Mobile apps for real-world evidence in health care

Madison Milne-Ives; Michelle Helena van Velthoven; Edward Meinert

doi:10.1093/jamia/ocaa036

. 2020 May 6;27(6):976–980. doi: 10.1093/jamia/ocaa036

Mobile apps for real-world evidence in health care

Madison Milne-Ives ^o1, Michelle Helena van Velthoven ^o1, Edward Meinert ^o1,^o2,^✉

PMCID: PMC7647309 PMID: 32374376

Abstract

The use of real-world evidence for health care research and evaluation is growing. Mobile health apps have often-overlooked potential to contribute valuable real-world data that are not captured by other sources and could provide data that are more cost-effective and generalizable than can randomized controlled trials. However, there are several challenges that must be overcome to realize the potential value of patient-used mobile health app real-world data, including data quality, motivation for long-term use, privacy and security, methods of analysis, and standardization and integration. Addressing these challenges will increase the value of data from mobile health apps to inform real-world evidence and improve patient empowerment, clinical management, disease research, and treatment development.

Keywords: real-world, evidence-based practice, mobile applications, telemedicine, US Food and Drug Administration

INTRODUCTION

The role of real-world evidence (RWE) in health care is growing. The 21st Century Cures Act permits the U.S. Food and Drug Administration (FDA) to allow the use of RWE instead of randomized controlled trials (RCTs).¹ The FDA defines real-world data (RWD) as “data … from electronic health records (EHRs), claims and billing data, … product and disease registries, patient-generated data … and … other sources that can inform on health status, such as mobile devices.”² RWE is the analysis of these data to assess the use, benefits, and risks of products, medications, and interventions.²

RCTs are the gold standard in health care research. They allow causal inferences and control for potential sources of bias, providing strong evidence of effectiveness.³ However, RCTs are expensive and time-consuming. More worryingly, because of strict eligibility conditions, their conclusions often fail to generalize to real-world populations.⁴ Patients whose multiple prescriptions, comorbidities, age, or disease severity make them “outliers” are often excluded from RCTs.³ Therefore, the study population does not necessarily reflect the intervention’s target population. Efficacy in a controlled sample does not necessarily imply real-world effectiveness. These concerns have increased focus on the potential of RWD to provide more efficient, cost-effective, and representative evidence.³^,⁵

RWE COLLECTED THROUGH MOBILE DEVICES

Electronic health records (EHRs) and claims data are currently the primary sources of RWE.⁶ However, mobile apps have the potential to collect a wide variety of data on patients’ daily experiences and decision motivations that are often missing in other sources of RWD.⁶^,⁷ There are many mobile health apps that track patient data or help consumers change health behaviors. These apps can also collect active and passive sensing and task-based data to measure daily activity and mental state, physiological status, and performance on physical or cognitive tasks.⁸

RWE from patient-used mobile apps can help evaluate the safety, effectiveness, and satisfactoriness of medical interventions in a more representative sample at a lower cost, and monitor symptoms, behaviors, and quality of life to improve patient care and reduce strain on medical services.⁶ These consumer-focused mobile health apps could provide valuable RWD to inform research and patient care. However, there are several challenges in the use of mobile health apps for RWE that must be overcome before this potential value can be realized.

CURRENT USE OF MOBILE RWE

The FDA is evaluating the use of RWD to provide evidence for drug approval and postapproval studies, support clinical trials, and develop decision-support tools for clinical practice, primarily using EHRs, patient registries, and claims data.⁹ Mobile health apps are also collecting a wide variety of RWD on patient health.¹⁰^,¹¹ However, data quality is not consistent. Validated questionnaires are available for measuring patient-reported outcomes such as quality of life, abilities, symptoms, health behaviors, and quality of care.¹²^,¹³ PROMIS (Patient-Reported Outcomes Measurement Information System) contains validated questionnaires that have already been integrated with EHRs.¹⁴ However, many apps adapt these questionnaires or develop their own without validation. Objective measures also need further data validation—step count and blood pressure data vary in accuracy and precision depending on the app or sensor being used.^15–17

There have been steps toward purposefully using patient-generated mobile health data in research and improving integration between platforms. Apple’s HealthKit collates health data from various apps, wearables, and EHR platforms. Since HealthKit’s release, Apple has also developed ResearchKit and CareKit—open-source frameworks for building apps to help researchers recruit patients and help patients manage their health conditions.¹⁸^,¹⁹ A study of the app Asthma Health demonstrated that data shared with a pulmonologist could be effectively used to identify and address deviations from patients’ norms.¹⁸ This type of access for health care professionals (HCPs) can improve patients’ quality of life and reduce clinic and emergency visits, reducing the strain on those services.¹⁸ Standards are being developed to improve interoperability, notably the Fast Health care Interoperability Resources Specification, which aims to integrate EHRs and other RWD across a variety of systems.²⁰^,²¹ Patient-powered research networks are another driver of improvement of mobile health apps for RWE.²² ArthritisPower is one example of these patient-guided registries that uses a mobile app to collect standardized patient-reported data more frequently than could be done at a clinic, but there are similar patient-powered research networks for many different chronic conditions.⁶

REGULATION OF MOBILE HEALTH APPS

Regulation of mobile apps depends on whether they are considered medical devices, and many are not.^23–25 The FDA defines mobile medical apps as “[accessories] to a regulated medical device [or apps that] transform a mobile platform into a regulated medical device.”²⁴ However, even mobile medical apps are not regulated if they are considered low risk.²⁵^,²⁶ There are also a number of federal laws that address data privacy and security (see Table 1).³⁴ The FDA and other regulatory agencies are working to update certification guidelines and processes for mobile medical apps.³⁵

Table 1.

Relevant laws and regulations relating to the safety, effectiveness, data protection, and privacy of mobile health apps and their data

Country	Law/regulation	Covers
United States	Health Insurance Portability and Accountability Act (HIPAA)²⁷	Data security and privacy of health information, health information breaches
United States	Federal Food, Drug, and Cosmetic Act (FD&C Act)²⁸	Safety and effectiveness of medical devices (includes some mobile apps)
United States	Federal Trade Commission Act (FTC Act)²⁹	Privacy, data security, validity of claims about safety and effectiveness
United States	Federal Trade Commission’s Health Breach Notification Rule³⁰	Personal health information breaches
Europe	General Data Protection Regulation (GDPR)³¹	Data protection, privacy, sharing of personal data outside the European Union/European Economic Area
Europe	Medical Device Regulation³²	Safety and effectiveness of medical devices (includes some mobile apps)
United Kingdom	Data Protection Act 2018³³	Data protection and privacy

Open in a new tab

Regulation is similar in the EU, with comparable conditions on what makes software a “device”—being an accessory, influencing treatment, or being used for diagnosis.³⁶ These apps must meet the standards of the Medical Device Regulation, which applies to Europe and the United Kingdom,³⁷ although this may change after Brexit.³⁸

BENEFITS OF MOBILE HEALTH APPS FOR RWE

Apps empower patients to take an active role in managing their health, and their RWD contribute to the development of diagnostics, preventions, and treatments. This RWE is valuable for patients in the short term and long term because it aids self-management, clinical care, and the improvement of treatments. For clinicians, mobile health app RWD can provide a more comprehensive picture of a patient’s condition and allow timely interventions. Economically, this can reduce clinic appointments and emergency visits by addressing issues early. RWD from mobile apps can provide payers (eg, insurance companies) more demographically specific information on effectiveness, safety, and value that can inform risk-benefit and payment calculations.³⁹ It can also help governmental agencies and medical companies with postapproval monitoring and regulation of new treatments.³⁹

For research, health app RWD are valuable because they provide more representative and realistic evidence without the time and monetary costs of an RCT. The ubiquity of mobile phones and the relative low cost of apps makes them a feasible, inexpensive way of collecting a lot of data. However, to be valuable, these data need to be high quality and standardized so that there can be interoperability between different sources of RWE in clinical care and research.

CHALLENGES FOR MOBILE RWE

The first challenge in using mobile health apps to collect RWD is an issue of purpose. Many health apps are not designed for research, which can affect data quality—a primary concern of patient-generated data. To be useful in research and health care, data need to be collected using valid and reliable measures.⁷ This is a challenge for mobile health apps because many variables rely on self-reports that are vulnerable to missing data and reporting and recall biases.⁶^,⁷ Sensing measures are subject to technological failure or variability, and task-based measures depend on the user correctly performing the procedure.⁸ Additionally, as with all mobile apps, maintaining long-term use is a challenge but is crucial to provide valid longitudinal data. Patients will also be unlikely to consent to these data being used for research without confidence that their data are private and secure.⁸ Most app users have concerns about privacy and data security and want to know how their data are being accessed and by whom.⁴⁰ These concerns are justifiable; a recent study found that health apps share data similarly to other apps, and some of the apps analyzed shared sensitive and potentially identifiable data that were vulnerable to access for commercial purposes.⁴¹ There is little regulation of the quality and effectiveness of mobile health apps that are not considered medical devices,²⁶ and another study found that only 20% of apps examined were considered medical devices and therefore subject to regulation.²³

Even if these problems are addressed, there is a lack of methods available to analyze the data concisely and comprehensibly so that it can be used by researchers, HCPs, and patients.⁸ Standardization, so that data from different sources can be integrated, is essential to make mobile health app RWD useful. This requires standardized reporting and interoperability between digital platforms, which are by no means universal features of apps.⁴² Additionally, older, less educated, and less e-health–literate people are less likely to use health apps in general, although they were more likely to use vital sign tracking apps.⁴³ To successfully use mobile health apps for RWE, the needs and contexts of all different groups of users must be considered. This will require integrating these new technologies into established frameworks for patients, HCPs, researchers, and other stakeholders.⁴⁴ These challenges are summarized in Table 2.

Table 2.

Summary of the challenges of using mobile health apps for real-world evidence

Challenge	Problem	Potential solutions
Intended purpose	Many mobile health apps are not designed to collect data in a way that is useful for health care and research	Involve medical and research experts in app development process
Data quality	Lack of validated and reliable measures through mobile health apps	Use previously validated questionnaires Test validity and reliability of newly developed questionnaires before use Further studies to validate sensor data More transparent reporting of how data were collected, so validity can be accurately evaluated
Privacy and security	Lack of patient trust and willingness to share data	Provide clear explanations of what information is being collected, who it is being shared with, and how it is being secured Provide an easy way to let patients consent to and control this data sharing over time Better technological security within apps for sensitive patient data
Methods of analysis	Data are overwhelming and difficult to use in research, in clinics, and by patients	More research into improving methods of analyzing and visualizing the types of real-world data collected by mobile apps
Standardization and integration of data	Real-world data cannot all be combined and assessed as a whole	Standardize methods of collecting and reporting data so that data can be shared Further develop infrastructures that allow integration of electronic health records, mobile health app data, and research databases
Health equity	Use of mobile health apps is weighted toward younger, more educated, and more e-health–literate people	Design app interfaces for older users with less technological experience Address privacy concerns, particularly for apps focused on reproductive health and other sensitive medical issues Target nontypical health app users to increase uptake and data representativeness Use purposeful sampling and appropriate statistical analyses to improve representativeness of real-world evidence in studies
Regulation of mobile health apps	Little regulation of mobile health apps that are not classified as “medical devices” with potential risk to public health	Design an international system of regulation of the quality and effectiveness of mobile health apps that are not regulated by the Food and Drug Administration, Medicines and Health care products Regulatory Agency, or other national bodies
Long-term use	Loss of motivation to keep using a mobile health app	Provide something of value that will motivate patients to continue use (eg, useful information about their health, specific behavior changes that could improve health, transfer of information to health care professionals for better treatment, a clear link between data and research) Easy-to-use apps that integrate into their daily lifestyle Personalizability of app features and availability of goal setting for health improvement
Integration into established routines	Lack of understanding of how new technology can best fit into existing workflows	Innovation in methods of assessing user needs to account for the amount and variety of patient-generated data Account for the context and established systems of all stakeholders when designing new or adapting current mobile health apps for real-world data collection

Open in a new tab

Other sources of RWD share many of these challenges.⁴⁵ For instance, EHRs, one of the main sources of RWE, are designed to record clinical interactions and aggregate data from other sources. However, much of the data collected are unstructured and incohesive.⁴⁶ These challenges—intended purpose, standardization, and integration—are similar to those described for mobile health apps. However, because the intended purposes and data collection and analysis structures of apps and EHRs are different, solutions to each will need to be specifically tailored.

IMPROVING MOBILE HEALTH APPS FOR RWE

Health care is beginning to transition toward a value-based system that pays providers according to patient outcomes. Value-based health care will depend on technology to evaluate costs and outcomes from an individual to a population level and must sufficiently address privacy and security concerns.⁴⁷ Mobile health apps are well suited to contribute to a value-based system because of their potential to cost-effectively collect a variety of RWE and integrate with existing systems. However, to achieve this potential, the challenges identified regarding data quality, privacy and security, analysis, standardization, and interoperability need to be improved.

The standards mobile health apps must meet to be valuable for RWE can be extrapolated from the challenges identified. To collect high-quality data, health apps should be purposefully designed for research data collection.⁴⁸ This includes the use of valid and reliable measures and the clear reporting of how data were collected so that quality can be accurately evaluated.⁷ Mobile health apps for RWD also need to be convenient and easy to use; have strong privacy, anonymity, and security features; and provide motivation to maintain engagement and data reporting.⁸^,⁴⁹ For patients, this could include supporting health behavior changes, improving communication with HCPs, and personalizing apps to suit their needs; for HCPs, this could include improved data analysis to deal with the abundance of data.⁵⁰ Apps must also address the specific needs of different demographic groups to improve the representativeness of the RWE.⁴³ Finally, to contribute to the body of RWE from other sources—such as EHRs, genomics, and claims data—health app data should be reported in a standardized and comprehensible way and integrated into existing databases.⁸

CONCLUSION

The potential of mobile health apps for RWE is great but currently unrealized. To be valuable sources of RWE, mobile health apps need to meet a variety of standards that differ depending on the user. Patients want control over their data and their health and observable benefits of participation. HCPs can benefit from reduced time and monetary burdens, as well as more data that can improve care. For research, the amount and variety of data and participants that can be studied at a low cost are valuable for understanding diseases and developing new interventions. To be beneficial in these ways, data need to be high quality and able to integrate with other systems, methods of collection need to be reported, mobile apps need to be easy to use and provide a benefit to the user, methods of analysis need to produce a clear and concise interpretation of a large amount of data, and privacy and security of sensitive information must be guaranteed. Striving to meet these standards, and adopting regulatory policies that ensure that they will be met even from mobile apps that are not classified as “medical devices,” will vastly increase the value of mobile health apps for RWE.

FUNDING

This work was supported by the European Institute of Innovation and Technology Health (Grant 18654, awarded to EM).

AUTHOR CONTIBUTIONS

EM conceived the article topic. MM-I drafted the initial article, which was subsequently revised by EM and MHvV. MM-I redrafted the article following feedback. MM-I responded to peer-review feedback under supervision of EM and redrafted the article.

CONFLICT OF INTEREST STATEMENT

None declared.

REFERENCES

1. Khosla S, White R, Medina J, et al. Real world evidence (RWE): a disruptive innovation or the quiet evolution of medical evidence generation? F1000Res 2018; 7: 111. [DOI] [PMC free article] [PubMed] [Google Scholar]
2.Office of Medical Products and Tobacco, Center for Devices and Radiological Health. Real-world evidence to support regulatory decision-making for devices: guidance for industry and Food and Drug Administration staff. FDA-2016-D-2153. http://www.fda.gov/regulatory-information/search-fda-guidance-documents/use-real-world-evidence-support-regulatory-decision-making-medical-devices Accessed November 26, 2019.
3. Corrigan-Curay J, Sacks L, Woodcock J.. Real-world evidence and real-world data for evaluating drug safety and effectiveness. JAMA 2018; 320 (9): 867–8. [DOI] [PubMed] [Google Scholar]
4. Bartlett VL, Dhruva SS, Shah ND, et al. Feasibility of using real-world data to replicate clinical trial evidence. JAMA Netw Open 2019; 2 (10): e1912869. [DOI] [PMC free article] [PubMed] [Google Scholar]
5. Sherman RE, Anderson SA, Dal Pan GJ, et al. Real-world evidence-what is it and what can it tell us? N Engl J Med 2016; 375 (23): 2293–7. [DOI] [PubMed] [Google Scholar]
6. Nowell WB. Information patients can provide will strengthen the real‐world evidence that matters to them. Clin Pharmacol Ther 2019; 106 (1): 49–51. [DOI] [PubMed] [Google Scholar]
7. Howie L, Hirsch B, Locklear T, et al. Assessing the value of patient-generated data to comparative effectiveness research. Health Aff (Millwood) 2014; 33 (7): 1220–8. [DOI] [PubMed] [Google Scholar]
8. Atreja A, Bates D, Clancy S, et al. Mobilizing mHealth Innovation for Real-World Evidence Generation Washington, DC: Duke-Margolis Center for Health Policy; 2017.
9.U.S. Food and Drug Administration. Framework for FDA’s Real World Evidence Program. Silver Spring, MD: U.S. Food and Drug Administration; 2018.
10. McKay FH, Wright A, Shill J, et al. Using health and well-being apps for behavior change: a systematic search and rating of apps. JMIR Mhealth Uhealth 2019; 7 (7): e11926. [DOI] [PMC free article] [PubMed] [Google Scholar]
11. Lee J-A, Choi M, Lee SA, et al. Effective behavioral intervention strategies using mobile health applications for chronic disease management: a systematic review. BMC Med Inform Decis Mak 2018; 18 (1): 12. [DOI] [PMC free article] [PubMed] [Google Scholar]
12. Kingsley C, Patel S.. Patient-reported outcome measures and patient-reported experience measures. BJA Educ 2017; 17 (4): 137–44. [Google Scholar]
13. Cella D, Hahn EA, Jensen SE, et al. Types of patient-reported outcomes In: Patient-Reported Outcomes in Performance Measurement. Research Triangle Park, NC: RTI Press; 2015. [PubMed] [Google Scholar]
14. Chung AE, Basch EM.. Potential and challenges of patient-generated health data for high-quality cancer care. J Oncol Pract 2015; 11 (3): 195–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
15. Balto JM, Kinnett-Hopkins DL, Motl RW.. Accuracy and precision of smartphone applications and commercially available motion sensors in multiple sclerosis. Mult Scler J Exp Transl Clin 2016; 2: 205521731663475. [DOI] [PMC free article] [PubMed] [Google Scholar]
16. Jamaladin H, van de Belt TH, Luijpers LC, et al. Mobile apps for blood pressure monitoring: systematic search in app stores and content analysis. JMIR Mhealth Uhealth 2018; 6 (11): e187. [DOI] [PMC free article] [PubMed] [Google Scholar]
17. North F, Chaudhry R.. Apple HealthKit and Health App: patient uptake and barriers in primary care. Telemed J E-Health 2016; 22 (7): 608–13. [DOI] [PubMed] [Google Scholar]
18. Genes N, Violante S, Cetrangol C, et al. From smartphone to EHR: a case report on integrating patient-generated health data. NPJ Digit Med 2018; 1 (1): 23. [DOI] [PMC free article] [PubMed] [Google Scholar]
19. Zens M, Woias P, Suedkamp NP, et al. Back on track’: a mobile app observational study using apple’s researchkit framework. JMIR Mhealth Uhealth 2017; 5 (2): e23. [DOI] [PMC free article] [PubMed] [Google Scholar]
20.HL7 FHIR Release 4. Overview-FHIR v4.0.1. https://www.hl7.org/fhir/overview.html Accessed January 22, 2020.
21. Bresnick J. 4 Basics to know about the role of FHIR in interoperability. Health IT Analytics March 22, 2016. https://healthitanalytics.com/news/4-basics-to-know-about-the-role-of-fhir-in-interoperability Accessed January 22, 2020.
22. Nowell WB, Benjamin Nowell W, Curtis JR, et al. Patient governance in a patient-powered research network for adult rheumatologic conditions. Med Care 2018; 56: S16–21. [DOI] [PMC free article] [PubMed] [Google Scholar]
23. van Drongelen A, de Bruijn A, Roszek B, et al. Apps Under the Medical Devices Legislation Bilthoven, the Netherlands: National Institute for Public Health and the Environment; 2018.
24.U.S. Food and Drug Administration. Policy for Device Software Functions and Mobile Medical Applications: Guidance for Industry and Food and Drug Administration Staff. Silver Spring, MD: U.S. Food and Drug Administration; 2019.
25.U.S. Food & Drug Administration. Device software functions including mobile medical applications. 2019. https://www.fda.gov/medical-devices/digital-health/device-software-functions-including-mobile-medical-applications Accessed January 28, 2020.
26. Kamerow D. Regulating medical apps: which ones and how much? BMJ 2013; 347: f6009. [DOI] [PubMed] [Google Scholar]
27.Office for Civil Rights, U.S. Department of Health and Human Services. Health Insurance Portability and Accountability Act. 1996. https://aspe.hhs.gov/report/health-insurance-portability-and-accountability-act-1996 Accessed January 29, 2020.
28.Office of the Commissioner, U.S. Food and Drug Administration.. Federal Food, Drug, and Cosmetic Act (FD&C Act). 2018. http://www.fda.gov/regulatory-information/laws-enforced-fda/federal-food-drug-and-cosmetic-act-fdc-act Accessed January 29, 2020.
29.Federal Trade Commission. Federal Trade Commission Act. 2013. https://www.ftc.gov/enforcement/statutes/federal-trade-commission-act Accessed January 29, 2020.
30.Federal Trade Commission. Health Breach Notification Rule. 2014. https://www.ftc.gov/enforcement/rules/rulemaking-regulatory-reform-proceedings/health-breach-notification-rule Accessed January 29, 2020.
31.European Parliament and Council of the European Union. General Data Protection Regulation, Regulation (EU) 2016/679. 2016. https://gdpr-info.eu.
32.Medical Device Regulation, Regulation (EU) 2017/745. 2017. https://eur-lex.europa.eu/eli/reg/2017/745/oj Accessed January 29, 2020.
33.Data Protection Act 2018. 2018. http://www.legislation.gov.uk/ukpga/2018/12/contents/enacted Accessed January 29, 2020.
34.Federal Trade Commission. Mobile Health Apps Interactive Tool. 2016. https://www.ftc.gov/tips-advice/business-center/guidance/mobile-health-apps-interactive-tool Accessed January 28, 2020.
35. Shuren J, Patel B, Gottlieb S.. FDA regulation of mobile medical apps. JAMA 2018; 320 (4): 337–8. [DOI] [PubMed] [Google Scholar]
36.Medical & Healthcare Products Regulatory Agency. Guidance: medical device stand-alone software including apps (including IVDMDs). 2018. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/717865/Software_flow_chart_Ed_1-05.pdf Accessed January 28, 2020.
37.Royal College of Physicians. Using apps in clinical practice: important things that you need to know about apps and CE marking. 2015. https://https://www.rcplondon.ac.uk/guidelines-policy/using-apps-clinical-practice-guidance Accessed January 28, 2020.
38.Medicines, Healthcare Products Regulatory Agency. Making a success of Brexit. 2016. https://www.gov.uk/government/news/medicines-and-healthcare-products-regulatory-agency-statement-on-the-outcome-of-the-eu-referendum Accessed January 28, 2020.
39. Hampson G, Towse A, Dreitlein WB, et al. Real-world evidence for coverage decisions: opportunities and challenges. J Comp Eff Res 2018; 7 (12): 1133–43. [DOI] [PubMed] [Google Scholar]
40. Zhou L, Bao J, Watzlaf V, et al. Barriers to and facilitators of the use of mobile health apps from a security perspective: mixed-methods study. JMIR Mhealth Uhealth 2019; 7 (4): e11223. [DOI] [PMC free article] [PubMed] [Google Scholar]
41. Grundy Q, Chiu K, Held F, et al. Data sharing practices of medicines related apps and the mobile ecosystem: traffic, content, and network analysis. BMJ 2019; 364: l920. [DOI] [PMC free article] [PubMed] [Google Scholar]
42. Daugherty SE, Wahba S, Fleurence R, PCORnet PPRN Consortium, et al. Patient-powered research networks: building capacity for conducting patient-centered clinical outcomes research. J Am Med Inform Assoc 2014; 21 (4): 583–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
43. Bol N, Helberger N, Weert J.. Differences in mobile health app use: a source of new digital inequalities? Inf Soc 2018; 34 (3): 183–93. [Google Scholar]
44. Valdez RS, Holden RJ, Novak LL, et al. Transforming consumer health informatics through a patient work framework: connecting patients to context. J Am Med Inform Assoc 2015; 22(1): 2–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
45. Cave A, Kurz X, Arlett P.. Real‐world data for regulatory decision making: challenges and possible solutions for Europe. Clin Pharmacol Ther 2019; 106 (1): 36–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
46. Guinn D, Wilhelm EE, Lieberman G, et al. Assessing function of electronic health records for real-world data generation. BMJ 2019; 24 (3): 95–8. [DOI] [PubMed] [Google Scholar]
47. Meinert E, Alturkistani A, Brindley D, et al. The technological imperative for value-based health care. Br J Hosp Med 2018; 79 (6): 328–32. [DOI] [PubMed] [Google Scholar]
48. Khozin S, Blumenthal GM, Pazdur R.. Real-world data for clinical evidence generation in oncology. J Natl Cancer Inst 2017; 109 (11): djx187. [DOI] [PubMed] [Google Scholar]
49. Mendiola MF, Kalnicki M, Lindenauer S.. Valuable features in mobile health apps for patients and consumers: content analysis of apps and user ratings. JMIR Mhealth Uhealth 2015; 3 (2): e40. [DOI] [PMC free article] [PubMed] [Google Scholar]
50. Cronin RM, Conway D, Condon D, et al. Patient and healthcare provider views on a patient-reported outcomes portal. J Am Med Inform Assoc 2018; 25 (11): 1470–80. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocaa036-B1] 1. Khosla S, White R, Medina J, et al. Real world evidence (RWE): a disruptive innovation or the quiet evolution of medical evidence generation? F1000Res 2018; 7: 111. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocaa036-B2] 2.Office of Medical Products and Tobacco, Center for Devices and Radiological Health. Real-world evidence to support regulatory decision-making for devices: guidance for industry and Food and Drug Administration staff. FDA-2016-D-2153. http://www.fda.gov/regulatory-information/search-fda-guidance-documents/use-real-world-evidence-support-regulatory-decision-making-medical-devices Accessed November 26, 2019.

[ocaa036-B3] 3. Corrigan-Curay J, Sacks L, Woodcock J.. Real-world evidence and real-world data for evaluating drug safety and effectiveness. JAMA 2018; 320 (9): 867–8. [DOI] [PubMed] [Google Scholar]

[ocaa036-B4] 4. Bartlett VL, Dhruva SS, Shah ND, et al. Feasibility of using real-world data to replicate clinical trial evidence. JAMA Netw Open 2019; 2 (10): e1912869. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocaa036-B5] 5. Sherman RE, Anderson SA, Dal Pan GJ, et al. Real-world evidence-what is it and what can it tell us? N Engl J Med 2016; 375 (23): 2293–7. [DOI] [PubMed] [Google Scholar]

[ocaa036-B6] 6. Nowell WB. Information patients can provide will strengthen the real‐world evidence that matters to them. Clin Pharmacol Ther 2019; 106 (1): 49–51. [DOI] [PubMed] [Google Scholar]

[ocaa036-B7] 7. Howie L, Hirsch B, Locklear T, et al. Assessing the value of patient-generated data to comparative effectiveness research. Health Aff (Millwood) 2014; 33 (7): 1220–8. [DOI] [PubMed] [Google Scholar]

[ocaa036-B8] 8. Atreja A, Bates D, Clancy S, et al. Mobilizing mHealth Innovation for Real-World Evidence Generation Washington, DC: Duke-Margolis Center for Health Policy; 2017.

[ocaa036-B9] 9.U.S. Food and Drug Administration. Framework for FDA’s Real World Evidence Program. Silver Spring, MD: U.S. Food and Drug Administration; 2018.

[ocaa036-B10] 10. McKay FH, Wright A, Shill J, et al. Using health and well-being apps for behavior change: a systematic search and rating of apps. JMIR Mhealth Uhealth 2019; 7 (7): e11926. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocaa036-B11] 11. Lee J-A, Choi M, Lee SA, et al. Effective behavioral intervention strategies using mobile health applications for chronic disease management: a systematic review. BMC Med Inform Decis Mak 2018; 18 (1): 12. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocaa036-B12] 12. Kingsley C, Patel S.. Patient-reported outcome measures and patient-reported experience measures. BJA Educ 2017; 17 (4): 137–44. [Google Scholar]

[ocaa036-B13] 13. Cella D, Hahn EA, Jensen SE, et al. Types of patient-reported outcomes In: Patient-Reported Outcomes in Performance Measurement. Research Triangle Park, NC: RTI Press; 2015. [PubMed] [Google Scholar]

[ocaa036-B14] 14. Chung AE, Basch EM.. Potential and challenges of patient-generated health data for high-quality cancer care. J Oncol Pract 2015; 11 (3): 195–7. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocaa036-B15] 15. Balto JM, Kinnett-Hopkins DL, Motl RW.. Accuracy and precision of smartphone applications and commercially available motion sensors in multiple sclerosis. Mult Scler J Exp Transl Clin 2016; 2: 205521731663475. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocaa036-B16] 16. Jamaladin H, van de Belt TH, Luijpers LC, et al. Mobile apps for blood pressure monitoring: systematic search in app stores and content analysis. JMIR Mhealth Uhealth 2018; 6 (11): e187. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocaa036-B17] 17. North F, Chaudhry R.. Apple HealthKit and Health App: patient uptake and barriers in primary care. Telemed J E-Health 2016; 22 (7): 608–13. [DOI] [PubMed] [Google Scholar]

[ocaa036-B18] 18. Genes N, Violante S, Cetrangol C, et al. From smartphone to EHR: a case report on integrating patient-generated health data. NPJ Digit Med 2018; 1 (1): 23. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocaa036-B19] 19. Zens M, Woias P, Suedkamp NP, et al. Back on track’: a mobile app observational study using apple’s researchkit framework. JMIR Mhealth Uhealth 2017; 5 (2): e23. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocaa036-B20] 20.HL7 FHIR Release 4. Overview-FHIR v4.0.1. https://www.hl7.org/fhir/overview.html Accessed January 22, 2020.

[ocaa036-B21] 21. Bresnick J. 4 Basics to know about the role of FHIR in interoperability. Health IT Analytics March 22, 2016. https://healthitanalytics.com/news/4-basics-to-know-about-the-role-of-fhir-in-interoperability Accessed January 22, 2020.

[ocaa036-B22] 22. Nowell WB, Benjamin Nowell W, Curtis JR, et al. Patient governance in a patient-powered research network for adult rheumatologic conditions. Med Care 2018; 56: S16–21. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocaa036-B23] 23. van Drongelen A, de Bruijn A, Roszek B, et al. Apps Under the Medical Devices Legislation Bilthoven, the Netherlands: National Institute for Public Health and the Environment; 2018.

[ocaa036-B24] 24.U.S. Food and Drug Administration. Policy for Device Software Functions and Mobile Medical Applications: Guidance for Industry and Food and Drug Administration Staff. Silver Spring, MD: U.S. Food and Drug Administration; 2019.

[ocaa036-B25] 25.U.S. Food & Drug Administration. Device software functions including mobile medical applications. 2019. https://www.fda.gov/medical-devices/digital-health/device-software-functions-including-mobile-medical-applications Accessed January 28, 2020.

[ocaa036-B26] 26. Kamerow D. Regulating medical apps: which ones and how much? BMJ 2013; 347: f6009. [DOI] [PubMed] [Google Scholar]

[ocaa036-B27] 27.Office for Civil Rights, U.S. Department of Health and Human Services. Health Insurance Portability and Accountability Act. 1996. https://aspe.hhs.gov/report/health-insurance-portability-and-accountability-act-1996 Accessed January 29, 2020.

[ocaa036-B28] 28.Office of the Commissioner, U.S. Food and Drug Administration.. Federal Food, Drug, and Cosmetic Act (FD&C Act). 2018. http://www.fda.gov/regulatory-information/laws-enforced-fda/federal-food-drug-and-cosmetic-act-fdc-act Accessed January 29, 2020.

[ocaa036-B29] 29.Federal Trade Commission. Federal Trade Commission Act. 2013. https://www.ftc.gov/enforcement/statutes/federal-trade-commission-act Accessed January 29, 2020.

[ocaa036-B30] 30.Federal Trade Commission. Health Breach Notification Rule. 2014. https://www.ftc.gov/enforcement/rules/rulemaking-regulatory-reform-proceedings/health-breach-notification-rule Accessed January 29, 2020.

[ocaa036-B31] 31.European Parliament and Council of the European Union. General Data Protection Regulation, Regulation (EU) 2016/679. 2016. https://gdpr-info.eu.

[ocaa036-B32] 32.Medical Device Regulation, Regulation (EU) 2017/745. 2017. https://eur-lex.europa.eu/eli/reg/2017/745/oj Accessed January 29, 2020.

[ocaa036-B33] 33.Data Protection Act 2018. 2018. http://www.legislation.gov.uk/ukpga/2018/12/contents/enacted Accessed January 29, 2020.

[ocaa036-B34] 34.Federal Trade Commission. Mobile Health Apps Interactive Tool. 2016. https://www.ftc.gov/tips-advice/business-center/guidance/mobile-health-apps-interactive-tool Accessed January 28, 2020.

[ocaa036-B35] 35. Shuren J, Patel B, Gottlieb S.. FDA regulation of mobile medical apps. JAMA 2018; 320 (4): 337–8. [DOI] [PubMed] [Google Scholar]

[ocaa036-B36] 36.Medical & Healthcare Products Regulatory Agency. Guidance: medical device stand-alone software including apps (including IVDMDs). 2018. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/717865/Software_flow_chart_Ed_1-05.pdf Accessed January 28, 2020.

[ocaa036-B37] 37.Royal College of Physicians. Using apps in clinical practice: important things that you need to know about apps and CE marking. 2015. https://https://www.rcplondon.ac.uk/guidelines-policy/using-apps-clinical-practice-guidance Accessed January 28, 2020.

[ocaa036-B38] 38.Medicines, Healthcare Products Regulatory Agency. Making a success of Brexit. 2016. https://www.gov.uk/government/news/medicines-and-healthcare-products-regulatory-agency-statement-on-the-outcome-of-the-eu-referendum Accessed January 28, 2020.

[ocaa036-B39] 39. Hampson G, Towse A, Dreitlein WB, et al. Real-world evidence for coverage decisions: opportunities and challenges. J Comp Eff Res 2018; 7 (12): 1133–43. [DOI] [PubMed] [Google Scholar]

[ocaa036-B40] 40. Zhou L, Bao J, Watzlaf V, et al. Barriers to and facilitators of the use of mobile health apps from a security perspective: mixed-methods study. JMIR Mhealth Uhealth 2019; 7 (4): e11223. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocaa036-B41] 41. Grundy Q, Chiu K, Held F, et al. Data sharing practices of medicines related apps and the mobile ecosystem: traffic, content, and network analysis. BMJ 2019; 364: l920. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocaa036-B42] 42. Daugherty SE, Wahba S, Fleurence R, PCORnet PPRN Consortium, et al. Patient-powered research networks: building capacity for conducting patient-centered clinical outcomes research. J Am Med Inform Assoc 2014; 21 (4): 583–6. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocaa036-B43] 43. Bol N, Helberger N, Weert J.. Differences in mobile health app use: a source of new digital inequalities? Inf Soc 2018; 34 (3): 183–93. [Google Scholar]

[ocaa036-B44] 44. Valdez RS, Holden RJ, Novak LL, et al. Transforming consumer health informatics through a patient work framework: connecting patients to context. J Am Med Inform Assoc 2015; 22(1): 2–10. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocaa036-B45] 45. Cave A, Kurz X, Arlett P.. Real‐world data for regulatory decision making: challenges and possible solutions for Europe. Clin Pharmacol Ther 2019; 106 (1): 36–9. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocaa036-B46] 46. Guinn D, Wilhelm EE, Lieberman G, et al. Assessing function of electronic health records for real-world data generation. BMJ 2019; 24 (3): 95–8. [DOI] [PubMed] [Google Scholar]

[ocaa036-B47] 47. Meinert E, Alturkistani A, Brindley D, et al. The technological imperative for value-based health care. Br J Hosp Med 2018; 79 (6): 328–32. [DOI] [PubMed] [Google Scholar]

[ocaa036-B48] 48. Khozin S, Blumenthal GM, Pazdur R.. Real-world data for clinical evidence generation in oncology. J Natl Cancer Inst 2017; 109 (11): djx187. [DOI] [PubMed] [Google Scholar]

[ocaa036-B49] 49. Mendiola MF, Kalnicki M, Lindenauer S.. Valuable features in mobile health apps for patients and consumers: content analysis of apps and user ratings. JMIR Mhealth Uhealth 2015; 3 (2): e40. [DOI] [PMC free article] [PubMed] [Google Scholar]

[ocaa036-B50] 50. Cronin RM, Conway D, Condon D, et al. Patient and healthcare provider views on a patient-reported outcomes portal. J Am Med Inform Assoc 2018; 25 (11): 1470–80. [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

Mobile apps for real-world evidence in health care

Madison Milne-Ives

Michelle Helena van Velthoven

Edward Meinert

Abstract

INTRODUCTION

RWE COLLECTED THROUGH MOBILE DEVICES

CURRENT USE OF MOBILE RWE

REGULATION OF MOBILE HEALTH APPS

Table 1.

BENEFITS OF MOBILE HEALTH APPS FOR RWE

CHALLENGES FOR MOBILE RWE

Table 2.

IMPROVING MOBILE HEALTH APPS FOR RWE

CONCLUSION

FUNDING

AUTHOR CONTIBUTIONS

CONFLICT OF INTEREST STATEMENT

REFERENCES

ACTIONS

PERMALINK

RESOURCES

Cite

Add to Collections

PERMALINK

Mobile apps for real-world evidence in health care

Madison Milne-Ives

Michelle Helena van Velthoven

Edward Meinert

Abstract

INTRODUCTION

RWE COLLECTED THROUGH MOBILE DEVICES

CURRENT USE OF MOBILE RWE

REGULATION OF MOBILE HEALTH APPS

Table 1.

BENEFITS OF MOBILE HEALTH APPS FOR RWE

CHALLENGES FOR MOBILE RWE

Table 2.

IMPROVING MOBILE HEALTH APPS FOR RWE

CONCLUSION

FUNDING

AUTHOR CONTIBUTIONS

CONFLICT OF INTEREST STATEMENT

REFERENCES

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases