Skip to main content
HHS Author Manuscripts logoLink to HHS Author Manuscripts
. Author manuscript; available in PMC: 2021 Aug 25.
Published in final edited form as: Healthc (Amst). 2016 Oct 3;5(3):81–85. doi: 10.1016/j.hjdsi.2016.08.004

Patient portals and health apps: Pitfalls, promises, and what one might learn from the other

Jessica L Baldwin a,b,*, Hardeep Singh a,b, Dean F Sittig c, Traber Davis Giardina a,b
PMCID: PMC8386919  NIHMSID: NIHMS1732677  PMID: 27720139

Abstract

Widespread use of health information technology (IT) could potentially increase patients’ access to their health information and facilitate future goals of advancing patient-centered care. Despite having increased access to their health data, patients do not always understand this information or its implications, and digital health data can be difficult to navigate when displayed in a small-format, complex interface. In this paper, we discuss two forms of patient-facing health IT tools—patient portals and applications (apps)—and highlight how, despite several limitations of each, combining high-yield features of mobile health (mHealth) apps with portals could increase patient engagement and self-management and be more effective than either of them alone. Patient portal adoption is variable, and due to design and interface limitations and health literacy issues, many people find the portal difficult to use. Conversely, apps have experienced rapid adoption and traditionally have more consumer-friendly features with easy log-in access, real-time tracking, and simplified data display. These features make the applications more intuitive and easy-to-use than patient portals. While apps have their own limitations and might serve different purposes, patient portals could adopt some high-yield features and functions of apps that lead to engagement success with patients. We thus suggest that to improve user experience with future portals, developers could look towards mHealth apps in design, function, and user interface. Adding new features to portals may improve their use and empower patients to track their overall health and disease states. Nevertheless, both these health IT tools should be subjected to rigorous evaluation to ensure they meet their potential in improving patient outcomes.

1. Introduction

There is growing interest in electronic access to health information and the use of digital data for both disease and health-related tracking. Widespread use of health information technology (IT) could potentially increase patients’ access to their health information and facilitate future goals of advancing patient-centered care.1 For example, health IT can be used to facilitate information exchange with clinicians and instruct patients when to act upon clinical issues, such as out of range physiologic parameters, follow-up of test results, and complications of medication use.2 Tools such as personal health records, patient portals, and various mobile health (mHealth) applications (apps) have been developed to help patients engage in their own care. Already, a significant number of patients use health IT; therefore, it is essential that patient-facing health IT be tailored to their needs. In this paper, we discuss two forms of patient-facing health IT tools—patient portals and apps—to highlight how, despite several limitations of each, combining high-yield features of mHealth apps with portals could increase patient engagement and self-management and be more effective than either of them alone. This could potentially improve both patient experience and outcomes related to patient-facing health IT.

Patient-facing health IT should be simply designed to encourage and sustain use and engage patients at various levels of health literacy.3 Patients increasingly express interest in being involved in medical decision-making and desire access to their health information.4 Despite having increased access to their health data, patients do not always understand this information or its implications, and digital health data can be difficult to navigate when displayed in a small-format, complex interface. For example, test results are not always displayed in a way that is easy for the patient to understand (e.g., with normal ranges clearly shown, along with implications of abnormal results). There is also little evidence that patient portal design addresses patients’ needs outside of meeting the “meaningful use” patient engagement criteria.5 It is imperative to keep patients’ needs in mind because patient-facing health IT users in the long run will not be just the early adopter health and technology “enthusiasts,” but regular people in need of better disease control and management.6

2. Emergence of patient-facing health IT

According to Pew Research, 7 in 10 U.S. adults say they track at least one health indicator.7 Although the number of patients interested in accessing their test results and tracking their health parameters has increased, patient portal use nationally is variable.5,8 For example, Athenahealth reports a 25% adoption rate across 1100 fee-for-service provider groups.9 Kaiser Permanente, an institution that has used portals for over a decade, reports that as of the third quarter of 2015, about 70% (5.2 million patients) of eligible adult members registered to use its My Health Manager patient portal.10 Group Health Cooperative (Seattle, WA) reports 73% of enrollees in Group Health Practices are registered and ID-verified to use its patient portal website as well as its mobile app, which offers the same constellation of services as the patient portal.11 However, registration rates and ID verifications do not account for the people who register but do not actively use the portal. Based on anecdotal data from a project we are conducting to determine patient preferences when viewing test results through portals, several patients have reported login issues and difficulty navigating portals.12

Conversely, the use of “easy to access” mHealth apps has increased dramatically over the past few years. Estimates suggest that by 2018, half of all smartphone and tablet users will have downloaded a mHealth app.13 The rise in health apps and health tracking software can be partially attributed to peoples’ growing interest in wearable devices and new applications that enable and engage patients to do more for their health care. Further, many apps rely heavily on social networking and the community experience allowing users to continuously track their activities and compare themselves with friends, family, and the larger community. Compared to most portals, apps appear to be more consumer-centric in design, and therefore, easier to use.

3. Pitfalls and promises of patient portals and health applications

As a secure online website providing patients access to their health information, the portal aims to improve quality of care by engaging patients as active participants in their care. While portal functions vary, most allow patients to view laboratory test results, immunizations, medications, and allergies, as well as to send secure messages to their physician.14 However, the portal can be difficult to navigate, and patients may struggle to understand their medical information. For instance, in our previous work we found that test result display and graphing were often confusing to patients, and they reported that portals were not user-friendly.15 A recent systematic review of patient and provider attitudes toward patient portal use found that the most negatively-perceived feature was user-friendliness, making the portal difficult to navigate.16 Our work exploring patient’s experiences using the portal to view test results echoes this finding, as many patients reported having difficulty locating their test results in the portal.17 When patients interact with their test results, they need to know the purpose of the test, the interpretation of the result, and next steps.18 Addressing these issues may help improve patient-centered care.

On the other hand and for a different engagement purpose, several companies have designed and created various tracking applications to encourage people to actively participate in their health. Applications, such as Mango Health (San Francisco, CA), Fitbit (San Francisco, CA), and Apple (Cupertino, CA) iPhone 6’s built in Health app, have consumer-friendly features with easy login access, real-time tracking, and simplified data display.19 From a patient standpoint, these features likely make the applications more intuitive and easy-to-use than patient portals. Furthermore, mHealth apps live on mobile devices, which make them easily accessible with little effort to login after setting up the account. This ubiquitous access is one of the reasons mobile technology is rapidly replacing desktop technologies.

Although apps might serve a different purpose, patient portals could adopt certain app features that lead to better engagement success with patients. Mobile apps have the capability to record several types of data, such as activity level, nutrition, and sleep, as well as data related to a consumer’s condition or disease, such as diabetes or asthma. For instance, Apple’s ResearchKit, although not designed as a health tracking application, offers several features that could be useful for health monitoring. It collects data and simultaneously encourages users to track their health by prompting daily health assessments. mHealth apps offer symptom management activities, which are not a standard feature universally available in patient portals.16 For example, LifeMap Solutions (San Jose, CA) has an application for Chronic Obstructive Pulmonary Disease management that provides medication reminders and tracks users’ symptoms to identify abrupt declines in their condition. Sentrian (Aliso Viejo, CA), a patient intelligence company, uses biosensors (i.e., blood glucose biosensor) to detect deteriorating health of patients to prevent avoidable hospitalizations.20 There are also apps that allow users to view their test results, such as Healthvana (Los Angeles, CA) and Labcorp (Research Triangle Park, NC). Healthvana, for instance, provides patients with interpretation of sexually transmitted infection results and follow-up instructions.21

While it is not clear if apps influence patient behavior, condition-specific apps may help patients improve outcomes.22,23 Nevertheless, mHealth apps’ features and functionality do not extend widely to provide users access to their institutionally-generated health data. At this point we also do not really know the value of the data generated by mHealth apps, and researchers are still determining how to best use the data from new apps like ResearchKit.24,25 These apps are also not heavily regulated and could contain poor quality or incorrect information, and some apps have been found to produce incorrect or inconsistent data.26,27 Despite increasing use of mHealth apps, up to 80% of apps are abandoned after only two weeks, suggesting more research is needed to understand what features engender longevity.28 Additionally, a recent study regarding health app use among vulnerable populations found that participants lacked confidence with the technology and expressed frustration with design and navigation. The authors called for participatory design, testing, and training with diverse patient populations to improve use.29

While mobile apps may offer more personalized interactions, it has been suggested that these apps need to be connected to personal health records to be effective and improve patient outcomes.30 However, there are legal concerns related to data protection and some uncertainty as to when and if mHealth apps fall under HIPAA or a developer’s own privacy policy, if available.3134 Thus, additional research must examine and determine the usefulness, relevance, credibility, and accuracy of health apps, as well as how they are used by patients and health care professionals.35

4. Socio-technical comparison of patient portals and health applications

In this section, we compare and contrast apps and portals using a sociotechnical lens. In our previous work, we have used an eight-dimension socio-technical model to evaluate other health IT innovations,36 interventions,37 applications,38 and devices39 implemented within a health care system.1 In Table 1, we compare apps and portals along the eight dimensions.

Table 1.

Comparison of patient portals and health apps on eight socio-technical dimensions.

Dimension Patient Portals Health Applications
Hardware and Software
  • Accessible via computers, smart phones, and tablets

  • Data is entered in system by labs and physicians

  • Accessible via computers, smart phones, and tablets

  • Data is entered by consumers and imported via tracking devices

Clinical Content
  • Patients can access their personal health information (i.e., test results, immunizations)

  • Direct Messaging with physician and health care team

  • Contain medical terminology and acronyms that are unfamiliar to most patients40

  • Consumers can enter health information

  • Data taken/entered in real-time

  • Data is taken in from the tracking devices (i.e., the accelerometer in the iPhone)

  • Generally no access to test results from physician

  • No communication access to physician or health care team

Human-Computer Interface
  • Accessible from the web and smartphones

  • Information is not always displayed in an understandable way, specifically test results

  • Outdated user interface design

  • Simple to access, use, and navigate

  • Information is often displayed in a way consumers without comprehensive medical knowledge can understand

  • Up-to-date, simple user interface design

People
  • Aim to connect patients to information from health care system

  • Consumers can often connect and compete with other patients

Workflow and Communication
  • Patients sign up through their doctor’s office

  • Patient password reset issues (e.g., in recent interviews, patients have complained about being “locked” out of their portal)

  • Consumers download the app and create their own account

  • Consumers’ activities are passively tracked, reducing data entry

  • Consumers have easy access to their data anytime, anywhere

Internal organizational policies, procedures, culture, and environment
  • Subject to an organization’s internal policies and procedures, which often create barriers to use (e.g., difficult sign-up procedures; reluctance of clinicians to participate in un-compensated work)

  • Currently subject to very little internal or external oversight (e.g., App developers can “sell” patient data)

  • Culture is “move fast, fix problems later”

  • Some have been found to sacrifice quality or safety in the pursuit of functionality22

External Rules, Regulations, Pressures
  • Must be HIPAA compliant

  • Must meet the legal and confidentiality needs of adolescents41

  • HIPAA compliance under review

  • Accessible to anyone with a smartphone

  • Constantly evolving smartphone operating system requirements

System Measurement and Monitoring
  • Few organizations monitor or measure how patient portal information is being used

  • Lack of real-time notifications and alerts to patients

  • Consumers use apps to monitor their own health

  • Depending on the app, different alerts are sent to the consumer’s phone

5. A vision for better portals

Patient portals are intended to engage patients by giving them access to medical information; however, if patients are unable to understand the information or the system is not usable, patients will not take advantage of them. Despite several aforementioned drawbacks, apps have used evolving innovative designs to engage consumers and offer unique features and functions that could be translated to patient portal design. For instance, Apple’s ResearchKit’s Diabetes app pings the user daily to update disease and symptom-related information. Check-in questions or user-friendly alerts in portals could similarly be explored for engaging more patients their health care. Alerts could ask if the patient understands an abnormal result, direct them to helpful resources, and encourage test result follow-up. Finally, test results in the portal need to be easily understood by laypeople or displayed using simplified medical terms. For example, a portal might display elevated cholesterol as “↑LDL cholesterol,” or even just display the number without a flag, whereas a health app may label it as “bad cholesterol.”

In addition to literacy issues, patients want to know how a test result will affect them and whether or not they need to seek further care after receiving a test result. A notification accompanying the result with this information would be helpful for patients. For example, “Your result is slightly outside of the normal range. No further tests are needed; however, monitor your diet and cut back on high cholesterol foods listed on the American Heart Association website (link provided).”

While in traditional systems, physicians often include information explaining test results, current methods of automated direct release of test results to patients make this personalized note difficult. A notification system that focuses on how to relay this contextual information may help ease patients’ concerns and anxiety.

To increase use, user-experience, adoption, and functionality, patient portals need to incorporate innovative design and user-friendly features, such as simplified data displays, easy log-in access, and alerts, pings, or notifications that explain results in layman’s terms and tell the patient if additional care is needed. Many of these features coupled with ubiquitous access made possible by portable digital devices such as smart phones, wearable technology, and tablets, contribute to the increased use of mHealth apps. Moreover, mHealth apps may provide a sense of control to patients because of fewer barriers to login and options related to health or disease tracking, which patient portals do not possess.42 Future portal development could look towards mHealth apps in design, function, and user interface.19 Adding new features to portals may improve their use and encourage patients to track their overall health and disease states.

To encourage improved features and adoption, vendors will need to work on usability and design. There is now a growing demand for improving patient-centered care and communication.43 Additionally, the push for reimbursement that relates to value-based care creates an opportunity to develop high-quality patient portals. For example, getting a better understanding of patient outcomes beyond traditional measures collected by the health care setting (length of stay or 30 day read-mission rate) requires new methods of data collection from patients, such as their performance on activities of daily living.44 New functional requirements will require existing patient portals to improve their capability and usability if they expect to capture new measures, such as observations of daily living (ODL) data, or items to measure patient-centered communication in a meaningful way.6

Adoption of portals may increase if certain barriers related to internal organizational policies, procedures, culture, and environment as well as external rules, regulations, and pressures are overcome. For example, Table 1 lists patient portal barriers, such as an organization’s internal policies related to timely test result notification to patients and difficult or complicated sign up procedures that result from meeting HIPAA requirements for user authentication. These issues may be resolved with changes to external regulations and meaningful use requirements that encourage easy log-in and registration procedures. Further, continued iterative usability testing with both users and nonusers will help vendors and health care systems identify problems.

There is considerable variation among the major vendors on addressing some of the portal issues discussed herein. However, at least some vendors appear to be actively working to identify usability and safety issues that could make portals more patient-centric.45 With the recent relaxation of certain patient access parameters in meaningful use regulations, patients and patient-advocates have strengthened their call for better access to more meaningful health care data. Currently, most vendors do not allow third party development on top of patient portals; however, with the advent of the new Substitutable Medical Apps & Reusable Technology (SMART) on Fast Healthcare Interoperability Resources (FIHR)46 standards, we may see more third party app development in both the EHR and the patient portal. Once these new (albeit evolving) standards make their way into routine use, user interface design of patient portals should improve as a result.

It appears that the excitement over mHealth apps has likely begun to influence patient portal developers. In June 2014, Apple announced the HealthKit cloud application programming interface (API) and its partnership with Epic (Verona, WI), an electronic health record vendor who also makes MyChart (a popular patient portal), and the Mayo Clinic (Rochester, MN). Apple’s HealthKit cloud service collects and logs data that has been recorded by multiple sensors, apps, or monitors, like the accelerometer in the iPhone, and allows this information to be stored in a database or health profile.47 Information from multiple sources will be available in one source, and Apple and its partners are working on allowing health profile data to interact with the Epic electronic health record.48 If such systems are tested with patients to help further identify patients’ needs when viewing and interpreting health data, the engagement potential would likely increase.

6. Conclusion

Apps appear to have certain features that lead to better engagement success with patients, and this could potentially inform portal development. To improve user experience with future portals, developers could look towards apps in design, function, and user interface. Combining certain high-yield features of mHealth apps with the wealth of provider-generated data available in portals may improve portal use, increase patient engagement, and empower patients to track their health and disease(s). Nevertheless, continued research is necessary to understand how best to combine these features and how data can be used meaningfully by patients to improve outcomes. For further progress, informatics and human factors researchers will need to work in coordination with mHealth vendors, health care delivery organizations, and their data to determine how patients are using these health IT tools and how to make them most useful for patient care. This type of evidence is essential for creating value for patients, clinicians, and health care organizations, as well as for initiating changes to improve the patient portal. Both these health IT tools should be subjected to rigorous evaluation to ensure they meet their potential in improving patient outcomes.

Acknowledgements

Dr. Singh is supported by the VA Health Services Research and Development Service (CRE 12-033; Presidential Early Career Award for Scientists and Engineers USA 14-274), the VA National Center for Patient Safety and the Agency for Health Care Research and Quality (R01HS022087 and R21HS023602). This work is supported in part by the Houston VA HSR&D Center for Innovations in Quality, Effectiveness and Safety (CIN 13-413). Mention of a product or app in this article should not be construed as an endorsement for it by the authors.

Footnotes

Conflict of interest and disclosure statement

This statement accompanies the article Patient portals and health apps: Pitfalls, promises, and what one might learn from the other authored by Jessica L. Baldwin and co-authored by Hardeep Singh, Dean F. Sittig, Traber Davis Giardina and submitted to Healthcare as an Article Type. Authors collectively affirm that this manuscript represents original work that has not been published and is not being considered for publication elsewhere.We also affirm that all authors listed contributed significantly to the project and manuscript. Furthermore we confirm that none of our authors have disclosures and we declare noconflict of interest.

Consultant arrangements: None

Stock/other equity ownership: None

Patent licensing arrangements: None

Grants/research support: None

Employment: None

Speakers’ bureau: None

Expert witness: None.

References

  • 1.Sittig DF, Singh H. A new sociotechnical model for studying health information technology in complex adaptive healthcare systems. Qual Saf Health Care. 2010;19:i68–i74. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Singh H, Naik A, Rao R, Petersen L. Reducing diagnostic errors through effective communication: harnessing the power of information technology. J Gen Intern Med. 2008;23:489–494. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Lazard AJ, Watkins I, Mackert MS, Xie B, Stephens KK, Shalev H. Design simplicity influences patient portal use: the role of aesthetic evaluations for technology acceptance. J Am Med Inform Assoc. 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Baldwin DM, Quintela J, Duclos C, Staton EW, Pace WD. Patient preferences for notification of normal laboratory test results: a report from the ASIPS collaborative. BMC Fam Pract. 2005;6:11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Otte-Trojel T, de Bont A, Rundall TG, van de Klundert J. What do we know about developing patient portals? A systematic literature review J Am Med Inform Assoc. 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Cohen DJ, Keller SR, Hayes GR, Dorr DA, Ash JS, Sittig DF. Developing a model for understanding patient collection of observations of daily living: a qualitative meta-synthesis of the Project HealthDesign program. Pers Ubiquitous Comput. 2015:91–102. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Pew Research Center. Health Fact Sheet. Available from: 〈http://wwwpewinternetorg/fact-sheets/health-fact-sheet/〉 [serial online] 2015; Accessed September 8, 2016.
  • 8.Keselman A, Smith AC. A classification of errors in lay comprehension of medical documents. J Biomed Inform. 2012;45:1151–1163. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Clain D athenaResearch Study: The Current State of Patient Portal Adoption, 2015.
  • 10.Garrido T, Raymond B, Wheatley B. Lessons From More Than A Decade In Patient Portals. 2016.
  • 11.Information on the Group Health Patient Portal. 2016.
  • 12.Singh H Improving Direct Notification of Abnormal Test Results via Patient Portals. Available from: 〈http://grantome.com/grant/NIH/R21-HS023602-02〉; [serial online] 2015.
  • 13.Jahns RG. 500m People will be Using Healthcare Mobile Applications in 2015. Available from 〈http://research2guidance.com/2010/11/10/500m-people-will-be-using-healthcare-mobile-applications-in-2015-2/〉; [serial online] 2010; Accessed September 9, 2016.
  • 14.HealthIT.gov. What is a patient portal? Available from: <https://wwwhealthitgov/providers-professionals/faqs/what-patient-portal> [serial online] 2015; Accessed September 9, 2016.
  • 15.Giardina TD, Modi V, Parrish D, Singh H. The patient portal and abnormal test results: an exploratory study of patient experiences. Patient Exp J. 2016. [PMC free article] [PubMed] [Google Scholar]
  • 16.Kruse CS, Argueta DA, Lopez L, Nair A. Patient and provider attitudes toward the use of patient portals for the management of chronic disease: a systematic review. J Med Internet Res. 2015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Giardina TD, Baldwin JL, Sittig DF, Singh H. Patient Perceptions of Test Result Notification Via the Patient Portal. 2016.
  • 18.Giardina TD, Modi V, Parrish D, Singh H. The patient portal and abnormal test results: an exploratory study of patient experiences. Patient Exp J. 2015Article 20. [PMC free article] [PubMed] [Google Scholar]
  • 19.Griffiths S Mobile App UX Principles: Improving User Experience And Optimising Converation. 2015.
  • 20.Baum S 5 Digital Health Tools That Could Improve COPD Management. Available from: 〈http://medcitynews.com/2015/12/copd-management-apps/〉; [serial online] 2015; Accessed September 8, 2016.
  • 21.Versel N With STDs on the rise, Healthvana seeks to put patients at ease. 2015.
  • 22.Singh K, Bates D, Drouin K, Newmark LP, Rozenblum R, Lee J. Developing a framework for evaluating the patient engagement. Qual, Saf Mob Health Appl. 2016. [PubMed] [Google Scholar]
  • 23.Shaw RJ, Steinberg DM, Bonnet J, et al. Mobile health devices: will patients actually use them? J Am Med Inform Assoc. 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Weiler A mHealth and big data will bring meaning and value to patient-reported outcomes. mHealth. 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Hayden EC. Mobile-Phone Health Apps Deliver Data Bounty. 2016. [DOI] [PubMed]
  • 26.Wicks P, Chiauzzi E. ‘Trust but veryfy’—five approaches to ensure safe medical apps. BMC Med. 2015;13:205. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Schoenfeld A, Sehgal N, Auerbach A. The challenges of mobile health regulation. JAMA Intern Med. 2016. [DOI] [PubMed] [Google Scholar]
  • 28.Infantino J Patient Engagement, Interoperabiltiy Needed For Mobile Health To Succeed. Available from: 〈http://epatientexperience.com/fv-community-news/post/patient-engagement-interoperability-needed-for-mhealth-to-succeed/〉; [serial online] 2014; Accessed September 1, 2016.
  • 29.Sarkar U, Gourley GI, Lyles CR, et al. Usability of commercially available mobile applications for diverse patients. Gen Intern Med. 2016:1–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Powell AC, Landman AB, Bates DW. In search of a few good apps. JAMA. 2014;311:1851–1852. [DOI] [PubMed] [Google Scholar]
  • 31.U.S.Department of Health and Human Services Food and Drug Administration Center for Devices and Radiological Health. General Wellness: Policy for Low Risk Devices. Available from: 〈http://www.fda.gov/downloads/medicaldevices/deviceregulationandguidance/guidancedocuments/ucm429674〉; [serial online] 2016; Accessed August 10, 2016.
  • 32.Terry NP. Mobile health: assessing the barriers. Chest. 2015;147:1429–1434. [DOI] [PubMed] [Google Scholar]
  • 33.Sunyaev A, Dehling T, Taylor PL, Mandl KD. Availability and quality of mobile health app privacy policies. J Am Med Inform Assoc. 2015;22:e28–e33. [DOI] [PubMed] [Google Scholar]
  • 34.Dehling T, Gao F, Schneider S, Sunyaev A. Exploring the far side of mobile health: information security and privacy of mobile health apps on iOS and android. JMIR Mhealth Uhealth. 2015;3:e8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Jutel A, Lupton D. Digitizing diagnosis: a review of mobile applications in the diagnostic process. Diagnosis. 2015;2:89–96. [DOI] [PubMed] [Google Scholar]
  • 36.Hazlehurst BL, Kurtz SE, Masica A, et al. CER Hub: An informatics platform for conducting comparative effectiveness research using multi-institutional, heterogeneous, electronic clinical data. Int J Med Inform. 2015;84:763–773. [DOI] [PubMed] [Google Scholar]
  • 37.Laxmisan A, Sittig DF, Pietz K, Espadas D, Krishnan B, Singh H. Effectiveness of an electronic health record-based intervention to improve follow-up of abnormal pathology results: a retrospective record analysis. Med Care. 2012;50:898–904. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Vartian CV, Singh H, Russo E, Sittig DF. Development and field testing of a self-assessment guide for computer-based provider order entry. J Health Manag. 2014;59:338–352. [PubMed] [Google Scholar]
  • 39.Sittig DF, Kahol K, Singh H. Sociotechnical evaluation of the safety and effectiveness of point-of-care mobile computing devices: a case study conducted in India. Stud Health Technol Inform. 2013;192:515–519. [PubMed] [Google Scholar]
  • 40.Keselman A, Slaughter L, Smith CA, et al. Towards consumer-friendly PHRs: patients’ experience with reviewing their health records. AMIA Annu Symp Proc. 2007:399–403. [PMC free article] [PubMed] [Google Scholar]
  • 41.Sittig DF, Singh H, Longhurst CA. Rights and responsibilities of electronic health records (EHR) users caring for children. Arch Argent Pediatr. 2013;111:468–471. [DOI] [PubMed] [Google Scholar]
  • 42.Shaw RJ, Steinberg DM, Bonnet J, et al. Mobile health devices: will patients actually use them? J Am Med Inf Assoc. 2016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 43.Singh H, Arora NK, Street RL Jr.. A vision for using online portals for surveillance of patient-centered communication in cancer care. Patient Exp J. 2015;2:125–131. [PMC free article] [PubMed] [Google Scholar]
  • 44.Bitton A, Onega T, Tosteson AN, Haas JS. Toward a better understanding of patient-reported outcomes in clinical practice. Am J Manag Care. 2014;20:281–283. [PMC free article] [PubMed] [Google Scholar]
  • 45.A Method for Identifying and Evaluating Potential Patient Safety Risks in a Patient-Facing Healthcare Portal. HFES 2016 (Human Factors and Ergonomics in Health Care: Shaping the Future); 16 Apr 14; 2016.
  • 46.Bosl W, Mandel J, Jonikas M, Ramoni RB, Kohane IS, Mandl KD. Scalable decision support at the point of care: a substitutable electronic health record app for monitoring medication adherence. Interact J Med Res. 2013;2:e13. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Carr DF. Apple Partners with Epic, Mayo Clinic for HealthKit. Available from: 〈http://www.informationweek.com/healthcare/mobile-and-wireless/apple-partners-with-epic-mayo-clinic-for-healthkit/d/d-id/1269371?page_number=2〉; 2014; Accessed September 8, 2016.
  • 48.Campbell KR. An apple a day: changing medicine through technology and engagement. Future Cardiol. 2015;11:259–260. [DOI] [PubMed] [Google Scholar]

RESOURCES