Out of the gate, health and research apps face-plant

There are three billion smartphones bouncing inside pockets and bags around the world. Their owners are often within arm's reach 24-7. With such ubiquity, constant usage, and connectivity, researchers have publicly drooled over the potential for mobile devices to become gushing conduits of health information. They could wirelessly and effortlessly provide data on patients’ symptoms, the success or failure of new treatments, and the progression of diseases—streamlining clinical trials, research, and personalized care.

The potential is there. But reality is not, according to a study published this week in Nature Biotechnology.

Tracking the effectiveness of an asthma health app created using Apple’s ResearchKit, researchers reported problems with participant selection bias, extremely low participant retention, missing data, and data security.

“These issues require attention to realize the full potential of mobile platforms in research and patient care,” the authors, led by Joel Dudley and Eric Schadt of Icahn School of Medicine at Mount Sinai, concluded. Still, they remained optimistic. “Looking forward, the potential of ubiquitous smartphone technology to address the needs of clinical research to better understand health and disease appears to be more promising than ever,” they added.

Both Dudley and Schadt hold stock in LifeMap Solutions, Inc., the company that helped develop the health app.

One of the most striking findings of their look at the utility of the app for research was the low user retention. The app was designed to help monitor and control participants' asthma symptoms, as well as feed researchers data. The app harvested user-generated data using daily, weekly, and less-frequent “milestone” asthma symptom surveys. It also provided participants with medical information, plus gave reminders and notifications relevant to asthma management.

In the US, 40,683 people downloaded the app (mostly after a big, splashy Apple unveiling). Of those, 7,593 people enrolled in the study. But from there, only 131 participants took at least a week’s worth of surveys and a six-month milestone survey. That’s 1.7 percent. Such dismal retention echoes what other health apps have experienced.

But even if more of the participants provided more data, it’s unclear if it would be helpful to researchers. Compared with a study cohort of asthma patients that the Centers for Disease Control and Prevention rounded up for research, the app users “tended to be younger, wealthier, more educated, and were more often male.” So the findings wouldn’t have been generalizable. This is somewhat expected, since the study was selecting among the subgroup of people who have smartphones.

But the app users also had higher rates of asthma-related hospitalizations and emergency room visits compared with the CDC cohort, suggesting that the app users may have had more severe or less controlled asthma.

Otherwise, the researchers reported stumbles in collecting data. Due to technical issues, they only collected age and/or gender data from 1,398 participants. Only 545 frequent app users consented to giving up geolocation data, making it difficult for researchers to compare symptom prevalence with environmental factors, such as pollen counts and air quality.

Together, it’s difficult to draw any conclusions about how the app might have helped asthma sufferers or researchers. The authors did note that of the 173 users who filled out a “milestone” survey, the number of people who reported that their asthma was uncontrolled dropped from 42 to 24 percent.

In the future, the researchers say health app designers should get better at making apps engaging and perhaps more reliant on passive data collection, like movement. It might also help if device-based research offered financial or other incentives, which is typical for traditional clinical trials. The researchers also speculated that the apps could be helpful as a tool within traditional clinical trials.

For now, they say, these apps alone may be most suited for only a narrow type of research. A type that: presents minimal risk to participants, allowing the use of electronic consent; has a hypothesis that can be answered in three weeks or less; relies on data collection that is passive; has no assumption that results will be generalizable; and has a statistical analysis plan to deal with low retention and lots of missing data.

Nature Biotechnology, 2017. DOI: 10.1038/nbt.3826  (About DOIs).