Abstract
Healthcare information technologies are now a routine component of patient–clinician interactions. Originally designed for operational functions including billing and regulatory compliance, these systems have had unintended consequences including increased exam room documentation, divided attention during the visit, and use of scribes to alleviate documentation burdens. In an age in which technology is ubiquitous in everyday life, we must re-envision healthcare technology to support both clinical operations and, above all, the patient–clinician relationship. We present 6 habits for designing user-centered health technologies: (1) put patient care first, (2) assemble a team with the right skills, (3) relentlessly ask WHY, (4) keep it simple, (5) be Darwinian, and (6) don’t lose the forest for the trees. These habits should open dialogues between developers, implementers, end users, and stakeholders, as well as outline a path for better, more usable technology that puts patients and their clinicians back at the center of care.
Keywords: health information technology, user centered design, human centered design, digital health, doctor-patient relationship
Over the last decade, healthcare delivery has undergone a technological revolution. In a 2009 survey of almost 3000 hospitals across the United States, fewer than 1 in 10 had an electronic health record system (EHR).1 That same year, the federal government implemented the Health Information Technology Economic and Clinical Health Act. The legislation called for the adoption and meaningful use of health information technology (HIT) to support coordination of care, tracking of clinical conditions, and clinical quality measures.2 By 2015, 96%3 of nonfederal acute care hospitals and 86% of office-based physicians reported adopting some type of EHR in their practices.4 Despite the Act’s goals for HIT systems to advance patient care and the flow of data,2 many barriers remain. In an age where technology-supported information sharing and interactions have become ubiquitous in other industries and in everyday life, HIT in medicine lags behind.
Safe adoption of any new HIT requires careful consideration of technology design within medicine’s complex social, political, and communication systems.5 Despite intentions to improve safety, new technologies can introduce safety risks.6,7 A 2012 National Academy of Medicine report lists usability, workflow, and interface complexity among the patient safety threats related to the introduction of HIT.8 Too often, the introduction of new systems into practice impedes good care by disrupting patient–clinician communication, failing to capture information richness, and increasing provider workload.9–11 A 2018 survey found that nearly half of the 521 primary care physicians surveyed saw digital storage as the primary value of their EHR.12 Only 8% of physicians reported that their EHR’s primary purpose was for clinical care and only 2% reported their EHR as supporting patient engagement. The majority (69%) of physicians reported that their EHR hindered relationships with their patients. To address these concerns, we need to envision new possibilities for the patient–clinician-EHR relationship in the digital age.10
To accelerate a move from widespread (and widely frustrating)13 HIT to more transformative HIT that improves patient safety and patient–clinician relationships, this paper draws from best practices and lessons learned across multiple industries and disciplines to offer 6 habits of successful HIT design and implementation (Table 1) for developers, implementers, and stakeholders (eg, vendors, local EHR implementation teams).
Table 1.
Habit | Description | How-to |
---|---|---|
1. Put patient care first | HIT must enhance patient care by making it safe, effective, efficient, and patient centered | Designs to facilitate clinician–patient communication. Consider potential solutions for shared clinician–patient HIT as well as team-based solutions that allow the clinician to focus on the patient, and not other tasks, eg, documentation. |
2. Assemble a team with the right skills | Innovation is an interdisciplinary team effort; it requires diversity in expertise and perspective. | Bring together a team diverse in expertise. Teams often include project managers, visual and interaction designers, human factors/industrial engineer experts, software engineers, writers/information architects, content experts, patients, clinicians, and key organizational stakeholders. |
3. Relentlessly ask WHY | Defining needs and generating innovative solutions calls for an iterative and inquisitive approach. Ask why is it done this way and could it be better? | Conduct needs analysis through observation, cognitive work analysis, task analysis, and interviews to understand current care processes, use patterns, workarounds, and cultural norms. Bring together users and key stakeholders in brainstorming sessions to generate design ideas. Avoid stifling early innovation with constraints, instead ask how innovation can advance policy, cultural norms, and technological limitations. Advance the best ideas into prototype design. |
4. Keep it simple | Leverage the science of human factors to support good design. | Consider user abilities and limits for visual perception, working memory, and motor skill. Consider the additive effect of each new system component on user cognitive load. Turn to other high stakes industries such as aviation and nuclear power for insight into designing for user performance. Look to industry standards when possible to support interoperability (eg, SMART on FHIR, ISO, NIST IR 7741, Usability, gov, AMIA). However, good design must drive the evolution of standards so design for the best solution first. |
5. Be Darwinian | Continue iterative improvements throughout the HIT life cycle. Test and retain only the best features at every stage. | Evaluate both in design and implementation. Formative evaluation should guide design iterations. Test for usability once design is stable. Continue summative evaluation once the system is implemented in the clinical environment. Consider target and novel outcomes of user behavior, patient care, and clinician wellness (eg, work after work, click count). Continue to evaluate across the postadoption life cycle as needs and technology continue to change. |
6. Don’t lose the forest for the trees | Putting patient care first requires safe design. Assess for safety and unintended consequences across the sociotechnical system with each design modification. | Introduction of new HIT in an existing sociotechnological system changes the dynamics of that system. Assess for unintended consequences and safety threats. Be aware that changes may occur not only on the user level, but also in interactions with others across the system (eg, changes in information exchange, shared work processes) |
AMIA: American Medical Informatics Association; FHIR: Fast Healthcare Interoperability Resource; HIT: health information technology; IR: Internal Report; ISO: International Organization for Standardization; NIST: National Institute for Standards and Technology.
HABIT 1: PUT PATIENT CARE FIRST
HIT should first and foremost support care that is patient-centered, safe, effective, and efficient. HIT must be designed to ensure the best medical decision-making while simultaneously supporting patient–clinician interaction. The introduction of HIT, especially in the exam room, splits providers’ attention between a computer screen and the patient.14 Empathic clinician communication is a key driver for better patient adherence, clinical outcomes, and reduced risk of malpractice lawsuits.15–17 A systematic review of the literature on HIT in the patient–clinician relationship demonstrates EHR use can negatively impact communication behaviors through interrupted speech patterns, multitasking, and lack of screen sharing.11 HIT can be a positive factor when technology is situated in a way that facilitates conversation and information-sharing, avoiding designs that turn the clinician away from the patient. Team-based care, in which a clerically or clinically trained person provides in-room data entry and retrieval support, allows the clinician to provide undivided attention to the patient.18,19 Performing front-line clinician and patient needs assessments during early design phases can help to drive new and innovative ways to support clinical interactions10. The clinician–patient relationship is the center of medicine; we should move away from technology driving patient care to patient care driving technology.20 Doing so will likely lead to new, patient-centered hardware and software configurations.
HABIT 2: ASSEMBLE A TEAM WITH THE RIGHT SKILLS
Developing effective, usable HIT requires a rich understanding of users and use context (eg, workflows, team interactions). In the complex, interrelated healthcare environment developing HIT for one user or user group ultimately impacts other users. To keep all team members’ needs at the forefront, it is critical to engage end users as active participants across the design, development, and testing phases of a new technology through user-centered design (UCD).21–24 Yet, many technology developers are not employing well-developed UCD processes.25,26 Challenges for developers employing UCD range from lack of understanding, resources, and access to the right users and contexts.26 Organizations and vendors must partner in the development of new HIT to assemble the teams necessary for transformative design. Innovation is a team effort; no single member can be an expert in all necessary areas. Team members should represent both developers and adopting organizations, bringing a diverse array of knowledge and skill sets to help design appropriately for users’ needs. Key roles often include project managers, visual and interaction designers, human factors or industrial engineers, software engineers, writers or information architects, content experts, end users (eg, patient, clinicians), and important organizational stakeholders.
At the heart of this team are users and stakeholders. To achieve usable and useful HIT, designers and human factors professionals should work together to identify and translate user’s needs and preferences into meaningful workflows. Design translates into development through the work of software engineers and information architects. With the disparate nature of such an interdisciplinary team, facilitation of communication and oversight of team priorities become a critical role for project managers.
HABIT 3: RELENTLESSLY ASK WHY
Transforming HIT requires questioning status quo and generating new, innovative ideas for HIT design. UCD and design thinking methods begin by understanding of the needs of users, their goals and motivations, and the current state of the sociotechnical system of care delivery. Given its complexities and dynamic nature, multiple methods for information gathering including observation, cognitive work analysis, interviews, and policy review may be required to fully understand the needs.27,28 Design thinking approaches engage users and key stakeholders directly in design processes through brainstorming.29 Encouraging ideas that challenge status quo helps to move toward stakeholders move toward innovative design.
While design must support users’ needs, asking why early can help avoid designing for inefficient and outdated processes and shift focus to new solutions.30 Ideas can then be refined with low-fidelity prototypes. Formative evaluation of prototypes with users, allows for an iterative design process identifying which features are successful and which need to be redesigned or eliminated early in development.28,31 Building from user-generated ideas can establish early user buy-in for later design solutions. Innovation can easily become stifled early through discussion of constraints (eg, assumptions that solutions cannot be achieved with available technology). Instead, asking why constraints exist can drive innovation forward to explore requirements necessary to support meaningful improvements in care. Asking why can help better address regulations around documentation and reporting requirements at the organizational, local, and national levels. However, even the most streamlined workflow and graphical user interface may not address regulatory issues that may be impeding clinician–patient interaction (eg, structured data entry for multiple organ systems on physical exam to meet a specific level of service for billing purposes).32
HABIT 4: KEEP IT SIMPLE
Good design is more than pleasing aesthetics. High-risk industries including military, aviation, nuclear, and aerospace have developed evidence and best practices for optimizing humans’ use of technology in complex environments.33,34 Designers should leverage the science of human factors in considering the user’s abilities and limits for visual perception, working memory, and motor skill. Seemingly simple features such as size and placement of buttons can affect both speed and error rates. For example, Fitts’ law predicts that the time required to move rapidly from one target to another is a function of the ratio between the distance to the target and the size of the target.35,36 This principle has been used in cockpit design for over 60 years but is rarely considered when determining the placement of buttons to click in the EHR. Utilizing established interaction design principles can guide design of key interactions including information presentation, navigation, feedback, and error prevention and recovery.37,38 Moreover, the use of smart data entry may aid error prevention by supporting cognitive processing during documentation.39 Inconsistency in HIT designs and lack of adoption of shared standards creates patient safety challenges.6,40 Technical standards are essential for error prevention, learnability, efficiency, and interoperability.41 Industry standards exist for technology platforms (eg, SMART on FHIR [Fast Healthcare Interoperability Resource])42 and usability testing (eg, International Organization for Standardization (ISO), Usability.gov). Yet, while the U.S. Department of Health and Human Services’ Office of the National Coordinator for Health Information Technology certification requirements call for UCD and usability testing, reports indicate less than half of certified EHRs used industry standards for UCD.25 While standards need to be adopted to support interoperability, designers should always consider if there is a better solution. Creating better solutions should then inform the evolution of better standards.
HABIT 5: BE DARWINIAN
Evaluate design across the life cycle only keeping what works. Developing HIT within an agile software development model allows for flexible design and iterative development, testing, and refinement across the product life cycle.43,44 Formative evaluation conducted during design should utilize working prototypes within realistic and representative clinical scenarios (use cases). An iterative approach provides design feedback for early identification of usability issues, prompt redesign, and further testing. Iterative refinement based on user feedback helps to optimize a new design. Once a stable design is achieved it is important to assess for usability issues with multiple users prior to implementation. The most important issues are often identified with only 3-5 users; groups of 15 participants find more than 90% of errors.45,46
Prepare for implementation by understanding potential barriers and solutions. Employing implementation science principles, integrate the system into user workflow and monitor effects through summative evaluation (eg, A/B testing, Plan-Do-Study-Act method). Consider both software and hardware issues. Even well-designed IT may present challenges in implementation when hindered by system issues (eg, placement of computer).40,47 Both healthcare organizations and vendors need to encourage user feedback. Remember to treasure every complaint and communicate how comments will be addressed. While not all feedback may be suitable for design modification, engaged users can offer learning points for design and implementation. HIT with poor usability is by definition error-prone.48 It is important to find and address these errors before they reach the patient.49
The healthcare landscape is dynamic; technical solutions supporting it must adapt as the field advances. Technology cannot be viewed as a 1-time investment. Implementing a design solution is not an end point but a stage in the life cycle. Continue to monitor the utility and performance of the system and revise as users’ needs change.
HABIT 6: DON’T LOSE THE Forest FOR THE TREES
Even if safety threats are identified and addressed prior to implementation, unintended consequences and safety risks can still arise through interactions with the broader work system.50 Incremental benefits when considered in the aggregate can be hazardous if they overload the humans in the system. High-risk fields (eg, aviation) design for safety through detailed focus on system impact on operator performance.33 No single HIT element exists in isolation—each feature adds to or eases workload. Focus on designing for safety by supporting a manageable cockpit for clinicians that accounts for an intervention’s effect on overall clinician cognitive workload, time pressures, administrative and clerical work, and need for workarounds.34
DISCUSSION
Physicians now spend nearly 45% of their workday and more than 11 h/wk outside of their workdays interacting with EHRs.51 Although much of this time is spent satisfying documentation requirements,32 technology should be capable of easing some of this burden. Our list of habits builds from prior work in HIT implementation as well as from the fields of design, usability, and human factors. Work specific to EHR implementation highlights the importance of usability,52 evaluation,47 the environment,53 and clinician training.
In a recent New England Journal of Medicine survey, 47% of the 519 executives, clinical leaders, and clinicians who responded identified HIT as 1 of the top 3 sectors needing disruptive innovation.54 Across industries from aviation to consumer mobile technologies, UCD techniques represent the standard for development; yet, they have seen only isolated adoption in HIT. Many EHR developers and adopting institutions, have focused on installing functional and reliable systems in time to qualify for federal incentive dollars10 and the ability of these systems to support financial and regulatory compliance needs.32 In this model, needs of the client have prioritized institutional level goals over the patient–clinician relationship often at the expense of both of these key end users.
Transforming HIT requires partnerships across the healthcare industry, both at organizational and user levels. Organizations developing and adopting HIT must understand that failure to dedicate resources to UCD can cost time and safety through issues discovered after adoption. To advance HIT usability, organizations must recognize the importance of the habits presented here and integrate throughout their culture.55 A business case needs to be made not for the adoption of HIT, but rather for the strategic investment in transformative HIT. Meaningful change in the HIT marketplace must be driven not by developers or organizational leaders alone, but rather by teams of developers, users, and stakeholders. To capture information needs, potential safety hazards, work processes, and the patient–clinician relationship, development teams must have access to representative users in the clinical workspace. Consumers need to be actively involved in the design process to ensure products are designed both for functionality and usability. Ultimately, these habits should drive development, testing, and redesign until the visions of HIT facilitating better communication, safer patient care, and more effective patient–clinician interactions are realized.
AUTHOR CONTRIBUTIONS
ERM, RMF and JMR, and conceived of the work. All authors substantially contributed to the design and implementation of the trial. JMR and EMR drafted the initial manuscript. All authors substantially contributed to the content of the manuscript, edited, and approved the final version submitted for publication. ERM takes responsibility for all aspects of the work.
CONFLICT OF INTEREST STATEMENT
RMW reports that he is a member of the Lucian Leape Institute of the IHI/National Patient Safety Foundation (no compensation except travel expenses); has a contract to University of California, San Francisco (UCSF), from the Agency for Healthcare Research and Quality to edit a patient-safety website; receives royalties from Lippincott Williams & Wilkins and McGraw-Hill for writing/editing several books; receives stock options for serving on the board of Accuity Medical Management Systems; receives a yearly stipend for serving on the board of The Doctors Company; serves on the scientific advisory boards for Amino.com, PatientSafe Solutions, and EarlySense (for which he receives stock options); consults with Commure (for which he receives a stipend and stock options) and Forward (stock options); has a small royalty stake in CareWeb, a hospital communication tool developed at UCSF; and holds the Benioff endowed chair in hospital medicine from Marc and Lynne Benioff and the Holly Smith Distinguished Professorship in Science and Medicine at UCSF. DIR reports support from Yale University Center for Biomedical Innovation and Technology; is an employee of the Department of Veterans Affairs, his views herein are his personal views and do not reflect those of Department of Veterans Affairs. MWF reports that since 2016, he has received financial support for research from the Agency for Healthcare Research and Quality, American Board of Medical Specialties Research and Education Foundation, American Medical Association, Center for Medicare & Medicaid Innovation, Centers for Medicare & Medicaid Services, Cedars-Sinai Medical Center, Commonwealth Fund, Milbank Memorial Fund, National Institute on Aging, National Institute on Drug Abuse, National Institute of Diabetes and Digestive and Kidney Diseases, National Institute on Minority Health and Health Disparities, Patient-Centered Outcomes Research Institute, and Washington State Institute for Public Policy. Since 2016, MWF has received payments from Consumer Reports for consulting services, from Wolters Kluwer for co-authorship of an UpToDate article about hospital quality measurement, and from Harvard Medical School for tutoring medical students in health policy. Since 2016, MWF has received support to attend meetings from the American Medical Association, Gordon and Betty Moore Foundation, and United States Department of Veterans Affairs. MWF also has a clinical practice in primary care at Brigham and Women’s Hospital and thus receives payment for clinical services, via the Brigham and Women’s Physician Organization, from dozens of commercial health plans and government payers, including but not limited to Medicare, Medicaid, Blue Cross and Blue Shield of Massachusetts, Tufts Health Plan, and Harvard Pilgrim Health Plan, which are the most prevalent payers in Massachusetts.
REFERENCES
- 1. Jha AK, DesRoches CM, Campbell EG, et al. Use of electronic health records in U.S. hospitals. N Engl J Med 2009; 36016: 1628–38. [DOI] [PubMed] [Google Scholar]
- 2. Blumenthal D. Launching HITECH. N Engl J Med 2010; 3625: 382–5. [DOI] [PubMed] [Google Scholar]
- 3. Henry J, Pylypchuk Y, Searcy T, et al. Adoption of electronic health record systems among US non-federal acute care hospitals: 2008-2015. ONC Data Brief 2016; 35: 1–9. [Google Scholar]
- 4. The Office of the National Coordinator for Health Information. Office-based Health Care Professionals Participating in the CMS EHR Incentive Programs. Health IT Quick-Stat #44; 2017. https://dashboard.healthit.gov/quickstats/pages/FIG-Health-Care-Professionals-EHR-Incentive-Programs.php. Accessed June 4, 2019. [Google Scholar]
- 5. Sittig DF, Singh H.. A new sociotechnical model for studying health information technology in complex adaptive healthcare systems. Qual Saf Health Care 2010; 19 (suppl 3): i68–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Howe JL, Adams KT, Zachary Hettinger A, et al. Electronic health record usability issues and potential contribution to patient harm. JAMA 2018; 31912: 1276–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Ratwani RM, Savage E, Will A, et al. Identifying electronic health record usability and safety challenges in pediatric settings. Health Aff 2018; 3711: 1752–9. [DOI] [PubMed] [Google Scholar]
- 8. Committee on Patient Safety and Health Information Technology, Board on Health Care Services, Institute of Medicine. Health IT and Patient Safety: Building Safer Systems for Better Care. Washington, DC: National Academies Press; 2012. [PubMed] [Google Scholar]
- 9. Rosenbaum L. Transitional chaos or enduring harm? The EHR and the disruption of medicine. N Engl J Med 2015; 37317: 1585–8. [DOI] [PubMed] [Google Scholar]
- 10. Gellert G, Webster S, Gillean J, et al. Should US doctors embrace electronic health records? BMJ 2017; 356: j242. [DOI] [PubMed] [Google Scholar]
- 11. Alkureishi MA, Lee WW, Lyons M, et al. Impact of electronic medical record use on the patient–doctor relationship and communication: a systematic review. J Gen Intern Med 2016; 315: 548–60. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.The Harris Poll, Stanford Medicine. How Doctors Feel About Electronic Health Records; 2018. http://med.stanford.edu/content/dam/sm/ehr/documents/EHR-Poll-Presentation.pdf. Accessed June 4, 2019.
- 13. Gawande A. Why doctors hate their computers. The New Yorker. 2018. Nov 5. https://www.newyorker.com/magazine/2018/11/12/why-doctors-hate-their-computers. Accessed June 4, 2019. [Google Scholar]
- 14. Toll E. A piece of my mind. The cost of technology. JAMA 2012; 30723: 2497–8. [DOI] [PubMed] [Google Scholar]
- 15. Zolnierek KBH, Dimatteo MR.. Physician communication and patient adherence to treatment: a meta-analysis. Med Care 2009; 478: 826–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Levinson W, Roter DL, Mullooly JP, et al. Physician-patient communication. The relationship with malpractice claims among primary care physicians and surgeons. JAMA 1997; 2777: 553–9. [DOI] [PubMed] [Google Scholar]
- 17. Derksen F, Bensing J, Lagro-Janssen A.. Effectiveness of empathy in general practice: a systematic review. Br J Gen Pract 2013; 63606: e76–84. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Hopkins K, Sinsky CA.. Team-based care: saving time and improving efficiency. Fam Pract Manag 2014; 216: 23–9. [PubMed] [Google Scholar]
- 19. Sinsky CA, Willard-Grace R, Schutzbank AM, et al. In search of joy in practice: a report of 23 high-functioning primary care practices. Ann Fam Med 2013; 113: 272–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Glass RM. The patient-physician relationship. JAMA focuses on the center of medicine. JAMA 1996; 2752: 147–8. [PubMed] [Google Scholar]
- 21. Nielsen J. Usability Engineering. New York, NY: Elsevier; 1994. [Google Scholar]
- 22. Norman DA, Draper SW. User centered system design: New perspectives on human-computer interaction. Boca Raton, FL: CRC Press; 1986. [Google Scholar]
- 23.ISO 9241-210:2010. Ergonomics of human system interaction-Part 210: Human-centred design for interactive systems. Geneva, Switzerland: International Organization for Standardization; 2009. [Google Scholar]
- 24. Shneiderman B, Plaisant C, Cohen M, et al. Designing the User Interface: Strategies for Effective Human-Computer Interaction. 6th ed.Upper Saddle River, NJ: Pearson; 2016. [Google Scholar]
- 25. Ratwani RM, Benda NC, Zachary Hettinger A, et al. Electronic health record vendor adherence to usability certification requirements and testing standards. JAMA 2015; 31410: 1070–1. [DOI] [PubMed] [Google Scholar]
- 26. Ratwani RM, Fairbanks RJ, Hettinger AZ, et al. Electronic health record usability: analysis of the user-centered design processes of eleven electronic health record vendors. J Am Med Inform Assoc 2015; 226: 1179–82. [DOI] [PubMed] [Google Scholar]
- 27. Effken JA. Different lenses, improved outcomes: a new approach to the analysis and design of healthcare information systems. Int J Med Inform 2002; 651: 59–74. [DOI] [PubMed] [Google Scholar]
- 28. Maguire M. Methods to support human-centred design. Int J Hum Comput Stud 2001; 554: 587–634. [Google Scholar]
- 29. Kelley T, Kelley D.. Creative Confidence: Unleashing the Creative Potential within Us All. New York, NY: Crown Publishing Group; 2013. [Google Scholar]
- 30. Ashton M. Getting rid of stupid stuff. N Engl J Med 2018; 37919: 1789–91. [DOI] [PubMed] [Google Scholar]
- 31. Samaras GM, Horst RL.. A systems engineering perspective on the human-centered design of health information systems. J Biomed Inform 2005; 381: 61–74. [DOI] [PubMed] [Google Scholar]
- 32. Downing NL, Bates DW, Longhurst CA.. Physician burnout in the electronic health record era: are we ignoring the real cause? Ann Intern Med 2018; 1691: 50–1.. [DOI] [PubMed] [Google Scholar]
- 33. Helmreich RL. On error management: lessons from aviation. BMJ 2000; 3207237: 781–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Sinsky CA, Privitera MR.. Creating a ‘manageable cockpit’ for clinicians: a shared responsibility. JAMA Intern Med 2018; 1786: 741–2. [DOI] [PubMed] [Google Scholar]
- 35. Fitts PM. The information capacity of the human motor system in controlling the amplitude of movement. J Exp Psychol 1954; 476: 381–91. [PubMed] [Google Scholar]
- 36. Schmidt RA, Zelaznik H, Hawkins B, et al. Motor-output variability: a theory for the accuracy of rapid motor acts. Psychol Rev 1979; 475: 415–51. [PubMed] [Google Scholar]
- 37. Zahabi M, Kaber DB, Swangnetr M.. Usability and safety in electronic medical records interface design: a review of recent literature and guideline formulation. Hum Factors 2015; 575: 805–34. [DOI] [PubMed] [Google Scholar]
- 38. Miller K, Capan M, Weldon D, et al. The design of decisions: Matching clinical decision support recommendations to Nielsen’s design heuristics. Int J Med Inform 2018; 117: 19–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Roman LC, Ancker JS, Johnson SB, et al. Navigation in the electronic health record: A review of the safety and usability literature. J Biomed Inform 2017; 67: 69–79. [DOI] [PubMed] [Google Scholar]
- 40. Ratwani RM, Savage E, Will A, et al. A usability and safety analysis of electronic health records: a multi-center study. J Am Med Inform Assoc 2018; 259: 1197–201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41. Lowry SZ, Quinn MT, Ramaiah M, et al. Technical evaluation, testing, and validation of the usability of electronic health records. J Res Natl Inst Stand Technol 2012; 65–87. [Google Scholar]
- 42. Mandel JC, Kreda DA, Mandl KD, et al. SMART on FHIR: a standards-based, interoperable apps platform for electronic health records. J Am Med Inform Assoc 2016; 235: 899–908. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43. Schwaber K, Beedle M.. Agile Software Development with Scrum. Upper Saddle River, NJ: Prentice Hall; 2002. [Google Scholar]
- 44. Beck K, Beedle M, Van Bennekum A, et al. Manifesto for agile software development. 2001. https://moodle2016-17.ua.es/moodle/pluginfile.php/80324/mod_resource/content/2/agile-manifesto.pdf. Accessed June 4, 2019.
- 45. Nielsen J, Landauer TK.. A mathematical model of the finding of usability problems In: Proceedings of the INTERACT ’93 and CHI '93 Conference on Human Factors in Computing Systems. New York, NY: ACM; 1993: 206–13. [Google Scholar]
- 46. Faulkner L. Beyond the five-user assumption: benefits of increased sample sizes in usability testing. Behav Res Methods Instrum Comput 2003; 353: 379–83. [DOI] [PubMed] [Google Scholar]
- 47. Ratwani R, Fairbanks T, Savage E, et al. Mind the Gap. A systematic review to identify usability and safety challenges and practices during electronic health record implementation. Appl Clin Inform 2016; 7: 1069–87. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48. Sittig DF, Ash JS, Singh H.. The SAFER guides: empowering organizations to improve the safety and effectiveness of electronic health records. Am J Manag Care 2014; 20: 418–23. [PubMed] [Google Scholar]
- 49. Ratwani RM, Hodgkins M, Bates DW.. Improving electronic health record usability and safety requires transparency. JAMA 2018. Nov 29 [E-pub ahead of print]. doi: 10.1001/jama.2018.14079 [DOI] [PubMed] [Google Scholar]
- 50. Ash JS, Berg M, Coiera E.. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc 2003; 112: 104–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51. Arndt BG, Beasley JW, Watkinson MD, et al. Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations. Ann Fam Med 2017; 155: 419–26. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52. Middleton B, Bloomrosen M, Dente MA, et al. Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA. J Am Med Inform Assoc 2013; 20 (e1): e2–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53. Duke P, Frankel RM, Reis S.. How to integrate the electronic health record and patient-centered communication into the medical visit: a skills-based approach. Teach Learn Med 2013; 254: 358–65. [DOI] [PubMed] [Google Scholar]
- 54. Dafny L, Mohta NS.. New marketplace survey: The sources of health care innovation. NEJM Catalyst; 2017. https://catalyst.nejm.org/disruptive-innovation-in-healthcare-survey/. Accessed June 4, 2019. [Google Scholar]
- 55. Healthcare Information and Management Systems Society. Promoting Usability in Health Organizations: initial steps and progress toward a healthcare usability maturity model. 2011. https://www.himss.org/sites/himssorg/files/HIMSSorg/Content/files/HIMSS_Promoting_Usability_in_Health_Org.pdf. Accessed June 4, 2019.