Enhancing patient safety and quality of care by improving the usability of electronic health record systems: recommendations from AMIA

Abstract

In response to mounting evidence that use of electronic medical record systems may cause unintended consequences, and even patient harm, the AMIA Board of Directors convened a Task Force on Usability to examine evidence from the literature and make recommendations. This task force was composed of representatives from both academic settings and vendors of electronic health record (EHR) systems. After a careful review of the literature and of vendor experiences with EHR design and implementation, the task force developed 10 recommendations in four areas: (1) human factors health information technology (IT) research, (2) health IT policy, (3) industry recommendations, and (4) recommendations for the clinician end-user of EHR software. These AMIA recommendations are intended to stimulate informed debate, provide a plan to increase understanding of the impact of usability on the effective use of health IT, and lead to safer and higher quality care with the adoption of useful and usable EHR systems.

Introduction

US healthcare delivery is in the midst of a profound transformation which results, at least in part, from Federal public policy efforts to encourage the adoption and use of health information technology (health IT). For example, HITECH regulations¹ within the American Recovery and Reinvestment Act² incentivize health IT use,^{3 4} and are changing the practice of medicine and clinical care delivery in both beneficial^{3 5 6} and untoward ways.⁷ Increased adoption of electronic health record (EHR) systems has been accompanied by heightened recognition of issues related to ‘goodness of fit’ in the user-friendliness of EHR systems.⁸ Some EHR users lament that health IT seems designed more for clinical transactions than for clinical care, and may not be easy to use in some care settings.^{9 10} In addition, many EHR systems require extensive training and lack standard user interfaces so that clinicians who work in multiple care settings using disparate technologies may struggle with the differences in interface design, with adverse impact on patient safety.¹¹ User interface design can influence provider productivity: well-designed interfaces speed work, while poorly designed interfaces steal minutes from busy schedules. The Institute of Medicine (IOM) report, Health IT and Patient Safety: Building Safer Systems for Better Care identified means by which health IT can lead to safer care, as well as introduce new safety risks. A critical component of safe and effective use of health IT is usability—‘the effectiveness, efficiency, and satisfaction with which the intended users can achieve their tasks in the intended context of product use.’¹² The IOM recommended that ‘[t]he Secretary of HHS [Health and Human Services] should specify the quality and risk management process requirements that health IT vendors must adopt, with a particular focus on human factors, safety culture, and usability’ (recommendation 6, p 9⁹).

Purpose of this AMIA statement

Given the anticipated adoption of health IT, and the potential for increased health IT-related harm or potential error, the AMIA Board of Directors convened a task force of members drawn from academia, clinical practice, and industry to produce a set of AMIA recommendations on enhancing patient safety and the quality of care with improved usability of EHR systems. These AMIA recommendations are intended to stimulate informed debate, form the basis of a plan to increase understanding of the impact of usability on the effective use of health IT, and lead to safer and higher quality care with the adoption of useful and usable EHR systems.

To address this issue, the task force convened for over a year. Subcommittees reviewed the literature on usability in health IT, current related activities underway at various US Federal agencies, lessons learned regarding usability and human factors in other industries, and current federally funded research activities. The key principles and recommendations described below are based on these reviews.

Relationship of usability to optimal healthcare practice

To frame this discussion, the AMIA Task Force on Usability considered the following issues related to health IT: (1) safe and effective use of EHR, (2) EHR usability, and (3) EHR usability-associated medical errors. Recent reports describe the safe and effective use of EHR as a property resulting from the careful integration of multiple factors in a broad socio-technical framework,¹³ including coordination and consideration across requirements assessment, application design, usability and human factors engineering, implementation, training, monitoring, and feedback to application developers.^{1 14–16} Following best practices for EHR implementation is essential to safe and effective use.^{2 17} Analyses of facilitators and barriers to physicians’ use of EHR systems suggest that usability is a major theme among system attributes, along with functionality, speed, support for hardware and software, required learning time, typing proficiency, understanding of the EHR system, motivation and personal initiative, and user-developed strategies and workarounds.^{3 4 18} Further analysis suggests that understanding user behavioral models is important to achieving effective use.^{3 5 6 19 20} Overcoming each of these issues is essential for improving the usability of EHR systems in clinical practice.

User error may result in untoward outcomes and unintended negative consequences. These may also occur as a result of poor usability,^{7 21–24} and may also be an emergent property only demonstrated after system implementation or widespread use.^{8 9} User errors may occur without adverse events,^{9 10 24–26} and some may not even be apparent to the user, analyzed by hospital or clinic review boards, or reported to the vendor.¹¹ While not causing undue harm, these errors are still problematic as they represent a mismatch between the user's model of the task and expected outcome,^{12 25} and the application's functionality and the resulting action or event.^{9 13 27} They may also represent a potential health IT-related error yet to happen.^{13 28} In some cases, clinicians use more than one commercial EHR system, with differences between the model of the task and the software functionality to execute the task, as well as differences in the terminologies used in the systems.^{29 30} Anecdotal reports suggest that these application differences result in an increased training burden for EHR users. Excessive alert fatigue can undermine the efficacy of clinical decision support in computer-based provider order entry (CPOE)^{31 32} and in other IT functions,^{11 33} and can result in very high override rates.^34–36 Some suggest that the expected gains sought with the adoption of EHR are not yet realized.^37–39

Actual adverse events or medical errors resulting from application design and usability, however, have also been described.^{9 10 14 15 40–42} Walker et al define an EHR-related system flaw as ‘Any characteristic of an EHR or of its interactions with other healthcare systems that has the potential to worsen care quality or patient outcomes. Other healthcare systems include individuals, care teams, facilities, policies, care processes, and healthcare organizations. Flaws may be introduced during the specification, design, configuration, or continuous-improvement phases of the EHR lifecycle’ (p 273).¹⁰ Sittig and Singh define EHR-related errors as occurring ‘anytime health IT is unavailable for use, malfunctions during use, is used incorrectly by someone, or when health IT interacts with another system component incorrectly, resulting in data being lost or incorrectly entered, displayed, or transmitted’ (p 1281).⁴³ The design of software applications requires both technical expertise and the ability to completely understand the user's goal, the user's workflow, and the socio-technical context of the intent.^{5 6 13 15 16 18 44} The design of clinical information systems is evolving from transaction-oriented systems toward a design focusing on patient-centered care, and on the needs of patients and healthcare teams.^{14 15 19 20 45} To achieve current US policy objectives,¹ transform our healthcare delivery system,⁹ and create a learning healthcare system,^46–49 clinicians need to use usable, efficient health IT that enhances patient safety and the quality of care. Some experts suggest that improving the usability of EHR may be critical to the continued successful diffusion of the technology.⁵⁰ Taken together, these factors suggest a need for renewed attention and focus on improving the usability of health IT to enhance patient safety and the quality of care.

AMIA's guiding principles

Framing usability assessment in health IT

This work focuses on the usability of EHR, and leaves aside medical devices, mobile devices, personal health records, and other related health information technologies. In particular, this work focuses on usability as it pertains to EHR systems and EHR module components.⁵¹ We touch upon usability principles, use cases, and interface guidelines⁵² as essential building blocks for effective user-centered design (UCD) and system implementation. For this work, we reference the Healthcare Information Management and Systems Society (HIMSS) usability definition, which includes nine attributes: simplicity, naturalness, consistency, forgiveness and feedback, effective use of language, efficient interactions, effective information presentation, preservation of context, and minimization of cognitive load.⁵³ A more comprehensive and evidence-based perspective on usability is provided by Zhang and Walji,⁵⁴ where usability refers to how useful, usable, and satisfying a system is for the intended users to accomplish goals by performing certain sequences of tasks.

The research team at the National Center for Cognitive Informatics and Decision Making in Healthcare, based upon an evidence review, proposed 14 usability principles which may guide the design and implementation of EHR.⁵⁴ These principles are listed in table 1.

Table 1.

Fourteen usability principles for the design of electronic medical records

1. Consistency—Design consistency and standards utilization	8. Message—Useful error messages
2. Visibility—System state visibility	9. Error—Use error prevention
3. Match—System and world match	10. Closure—Clear closure
4. Minimalism—Minimalist design	11. Reversibility—Reversible actions
5. Memory—Memory load minimization	12. Language—User language utilization
6. Feedback—Informative feedback	13. Control—User control
7. Flexibility—Flexible and customizable system	14. Documentation—Help and documentation

Adapted from Zhang and Walji.⁵⁴

Attributes of usability may be assessed with a wide array of qualitative and quantitative methods. Usability assessment methodologies have a rich scientific literature and these techniques are commonly used in software intensive industries⁵⁵ including healthcare.^56–63 These methods may be applied in simulation experiments, in laboratory-based evaluations, and in real-world use settings. They may combine subjective assessments made by an end-user (or group),⁶⁴ as well as quantitative assessments, such as time to task completion.⁵⁹ Often a ‘use case’ is defined to assess the ability of a user to use the software application to execute a particular task in a timely manner without error.⁶⁵ No one method has been seen as dominant over others, and typically a combination of methods may be used throughout the software development lifecycle.^{9 14–16 64} One area that has received less attention, however, is monitoring of use in practice or real-world settings to continually assess the user's usage patterns and monitor for unintended consequences, adverse effects, or medical error.⁶⁶

Use cases are particularly important as standardized instruments that may be used in usability evaluation and certification. The Office of the National Coordinator for Health Information Technology (ONC) published the Test Procedure for §170.314(g)(3) Safety-enhanced design, which was added to the 2014 edition of Health Information Technology: Standards, Implementation Specifications, and Certification Criteria for Electronic Health Record Technology (Meaningful Use 2014). Test Procedure §170.314(g)(3) was developed to emphasize the importance of UCD, and as a first step to improving EHR usability. Test Procedure §170.314(g)(3) includes eight use cases wherein an EHR vendor would need to document their UCD approach (and if a UCD approach did not previously exist one must be adopted) using the National Institute of Standards and Technology (NIST) Customized Common Industry Format Template for EHR Usability Testing (NISTIR7742).⁶⁷ The eight use cases are CPOE, drug and allergy interaction checks, medication list, medication allergy list, clinical decision support, electronic medication administration record (inpatient setting only), electronic prescribing (medication order entry), and clinical information reconciliation (patient problems, medications, and allergies). The ONC selects these use cases because of the associated risk for patient harm with poor usability, and the greatest immediate opportunity for error prevention and user experience improvement.

Finally, another important component for EHR design and effective use is the application of standard user interface guidelines (‘style guides’) that provide guidance on color, controls, screen layout, and application flow to developers and users who are customizing an application.⁵² Little research to date has focused on the similarities and differences in the application style guides for various EHR products. While standardized use cases define functional characteristics and prototypical workflow, they do not define more broadly the ‘look and feel’ of an application. For example, screen layout, controls such as buttons, dialog boxes, entry modules, and other interface artifacts should be designed to have consistent visual and functional attributes across all component modules of a complex system. A notable initiative involving the UK National Health Service (NHS) and Microsoft Corporation has focused on a ‘Common User Interface’ for health applications used in the NHS.⁶⁸ Since 1994, this effort has examined and developed recommendations for clinical documentation and standard user interface terminologies, consistent navigation, user interaction for medication ordering and management, patient identification, and other miscellaneous controls.

EHR industry perspectives

While some EHR vendors have adopted user-centered design when developing health information technologies, the practice is not universal⁶⁹ and may be difficult to apply to legacy systems. Current practice suggests that EHR vendors use multiple techniques with variable efficacy. In an Agency for Healthcare Research and Quality (AHRQ) workshop on usability in health IT in July 2010, all eight participating vendors agreed that usability was important and many suggested it was a competitive differentiator, although some considered that usability was in the eye of the beholder and that the discipline of usability evaluation was an imperfect science, with results that were not useful.⁶⁹ Nevertheless, the workshop participants agreed upon recommendations in three key areas: standards in design and development, usability testing and evaluation, and post-deployment monitoring and patient safety. They suggested that usability as a component of certification had to be viewed carefully and only when truly valid measures were available.⁶⁹ Clearly, the vendors surveyed by the AHRQ workshop said they were motivated to build and implement high quality software systems that would lead to improved patient safety and quality of care, yet current software practices appear to be highly variable. Some believe it is difficult or impossible to reliably compare one product with another on the basis of usability given the challenges in assessment of products as implemented.

Challenges with usability assessment

Challenges to assessing EHR usability include the complexity of the EHR interaction with the full socio-technical context in which it is used,^{24 26 57} the professional roles of the intended users,^{11 70} the peculiarities of the clinical collaboration patterns,^{25 71} and the difficulty in measuring the influence that systems have on downstream processes.^{27 72} Usability in complex socio-technical systems such as healthcare are particularly challenging since a software product is designed to meet the needs of multiple different user types who have varying requirements, work across geographic, temporal, organizational, and cultural boundaries, and who may in fact be participating in the product design.²⁸ The ability to perform meaningful, reproducible, objective usability metrics for EHR systems is limited by the socio-technical nature of the systems. They are typically highly configurable and different providers will implement them in different ways, with markedly different processes. Many effects of health IT can be considered to be ‘emergent’ or only discovered after monitoring a system in use.^{25 31 73 74} Typical metrics identified for usability are completion rate (or error rate), time on task, and subjective user satisfaction.^{33 44} These metrics require multiple measures or a composite score^{75 76} to be useful to developers and users.

Lessons learned from aviation

The field of human factors engineering and usability assessment has a rich set of scientific methods, a strong evidence base, and is widely applied effectively in other industries.^77–79 There are many process standards (HCI/GUI design, evaluation, and verification), software user interface standards and guidelines (human computer interface/graphical user interface appearance and behavior),⁵² standards for quality management in software design, and a standard EHR usability evaluation protocol which has been proposed by NIST.⁸⁰ The potential value of employing these methods in the design and effective use of EHR technologies is well described in the literature.^{9 10 17 20 60 61 74 77 78 81–87}

The experience in aviation—with similarities in the expertise required for decision making, significant customer safety risks, high utilization of technology, strong professional culture issues, and significant regulatory oversight—is particularly relevant.^{88 89} The aircraft industry developed industry groups where topics are discussed and conventions sanctioned by a large number of airframe manufacturers and aviation software and hardware vendors. These cooperative efforts have occurred despite the intensely competitive nature of the aviation sector as competitors see value in working together in this arena. These changes have occurred over the century of human powered flight, facing the same resistance change has evoked in other areas of activity. Nevertheless, each new generation of aircraft has resulted in safer and more efficient flight. No single measure alone accounts for this success, but the mixture of measurement, reporting, and regulatory incentives has resulted in an admirable level of safety and continuous improvement from which other industries—including healthcare IT—can learn. In the USA, the Federal Aviation Administration regulates aircraft airworthiness, issues aircraft safety alerts, operates the Aviation Safety Reporting System, and in many other ways regulates aviation. Some have suggested that a structure analogous to the Aviation Safety Reporting System should exist in healthcare.^90–92

AMIA recommendations

Given the potential impact of EHR technology to improve healthcare delivery and increase inadvertent patient harm, AMIA believes it is now critical to coordinate and accelerate the numerous efforts underway focusing on the issue of EHR usability. Vendors and users of health IT both seek to improve the quality of care delivered with EHR, but current evidence suggests that some health IT may facilitate certain types of adverse events and medical errors, and that these problems may be related to usability issues. The recommendations below do not address all aspects of the safe and effective use of EHR, but they help focus attention on critical usability issues that adversely affect patient safety and the quality of care.

1. Usability and human factors research agenda in health IT

Despite the wide variety of methods used in usability assessment and human factors engineering, additional experience is required in assessing their use in EHR applications. AMIA proposes accelerating the research agenda in three critical areas to support broad adoption of improved usability practices among EHR developers and users.

a. Prioritize standardized use cases

Standardized use cases should be established and maintained for selected EHR functionalities. Patient-safety sensitive EHR functions include patient selection (correct patient), clinical documentation (correct record, appropriate documentation quality), medication ordering and management (right drug, dose, route, etc), allergy documentation (correct allergen and reaction), results review (correct data and appropriate view), and advance directive documentation (correct status), among others suggested by the ONC. Standardizing and prioritizing a set of use cases for patient-safety sensitive EHR functions would provide the foundation for a process by which different EHR may be measured and certified. This work should be sponsored by the ONC.

b. Develop a core set of measures for adverse events related to health IT use

Use cases will facilitate the development and validation of standardized performance measures for assessing the incidence of adverse events and medical errors. These measures should be developed with the participation of experts and representatives drawn from the measure development community, clinical informatics, end-users, and the vendor community.

Such a collaborative effort would facilitate the development of a practical approach to the measurement of EHR use related to adverse events or medical error. This effort could also define methods to prioritize the importance of IT-related adverse events or medical errors with the goal of defining and distinguishing ‘EHR never events’ from events of lesser importance. This will be critical for establishing guidelines for IT-related adverse event reporting (see recommendation 4b) and developing a representative experience base across diverse products and various care settings. Ultimately, this work may inform development of a certification criterion or set of criteria and potential rule making regarding the usability of EHR. This work should be sponsored by the AHRQ.

c. Research and promote best practices for safe implementation of EHR

It is also critical that best practices are assessed and defined for the safe implementation and ongoing effective use of EHR. This includes identifying implementation best practices that address the broader socio-technical context of safe and effective use of EHR beyond just the usability component. They must address effective initial and ongoing training requirements, assessment of application configuration(s), technology (hardware/software) infrastructure and support, systems integration, workflow process(es), organizational culture and policies, and externalities which may confound the safe and effective use of EHR. This work should be sponsored by the AHRQ.

2. Policy recommendations

AMIA offers several recommendations for Federal policy initiatives that could help provide a sound evidence base to enhance the quality of care and patient safety using EHR.

a. Standardization and interoperability across EHR systems should take account of usability concerns

Because many patient-safety sensitive EHR functionalities may be related to competing or non-standard use of interface terminologies, it is critical that the ONC's Standards and Interoperability Framework includes usability considerations for patient-sensitive functions related to controlled medical terminologies, and application functions, expressed ‘at the glass’ for the end-user. This work should be sponsored by the ONC.

b. Establish an adverse event reporting system for health IT and voluntary health IT event reporting

A voluntary reporting process could leverage the AHRQ patient safety organizations (PSO), and would investigate and report on adverse events and medical errors related to usability. PSO could assume responsibility and accountability for establishing an IT-related voluntary error measurement and public reporting system. They should follow the NIST Common Industry Format.⁶⁷ Reports should be captured locally and reviewed by end-users (facilitated by application functionalities designed for this purpose), and summary reports should be sent to the application vendor and to PSO. PSO governance bodies can convene relevant stakeholders to determine best practices for end-user and vendor product anonymity, appropriate levels of data aggregation, report details and frequency, and what summary data are made public. Use of the AHRQ Health IT Hazard Manager is a potential application for this purpose.⁶⁶ This work should be sponsored by the AHRQ.

c. Develop and disseminate an educational campaign on the safe and effective use of EHR

An educational campaign is needed, with the goal of bringing increased attention to issues of patient-safety sensitive functions of EHR or EHR modules, and to usability in general. This industry is still relatively early in its evolution, with far less than 50% of users using EHR technologies, and many new companies entering the field, with different types of novel technologies. Thus, there is still an opportunity to provide meaningful guidance to the industry in this formative stage to facilitate achievement of the core policy goals of EHR adoption, interoperability, and meaningful use, and improved design of EHR for the benefit of vendors, end-users, and patients. This work should be sponsored by the ONC, utilizing the infrastructure of the Health IT Regional Extension Centers.

3. Industry recommendations

Vendors play a critical role in addressing challenges with EHR usability. AMIA makes two recommendations for the EHR industry.

a. Develop a common user interface style guide for select EHR functionalities

AMIA recommends that an industry coalition address the issue of developing a common style guide for patient-safety sensitive functions of EHR. This does not imply a common style guide for all aspects of the human–computer interaction in EHR, nor does it suggest there are not still ways in which vendors may innovate and compete with their product design and implementations. It does imply, however, that there should be a minimum set of design patterns shared among vendors that improve the usability for patient-safety sensitive functions within and across EHR, for the benefit of the patient and end-users. Much like the common features of information sources within automobiles (speedometer, odometer, fuel reserve, drive gear, etc), this set of common elements will lessen the training burden on end-users, and lead toward standardization of user interaction with the system in critical, patient-safety sensitive functions. Industry bodies such as the HIMSS EHR Association could spearhead this work, in collaboration with professional societies such as AMIA, and leading investigators in the field of human factors in the use of technology.

b. Perform formal usability assessments on patient-safety sensitive EHR functionalities

AMIA recommends that the EHR industry utilize usability assessments of a select set of patient-safety sensitive EHR functionalities that assess agreed upon attributes as defined in recommendations 1.a and 3.a above—for example the eight use cases from the ONC Meaningful Use Final Rule (2014 Edition), with more to come as patient-safety sensitive EHR functionalities are better understood through measurement and voluntary reporting (recommendations 1b and 2b above). If successful, the industry will improve usability of these patient-safety sensitive EHR functionalities over time. This work should be sponsored by the ONC and conducted in collaboration with NIST.

4. Clinical end-user recommendations

While the academic and research informatics community and the vendor community have critical roles to pursue as outlined in the recommendations above, AMIA believes that the end-user of EHR technologies also has a critically important role to play in enhancing the usability of EHR applications.

a. Adopt best practices for EHR system implementation and ongoing management

End-users of EHR systems are ultimately accountable for their safe and effective use, like any tool in clinical care. With the growing installed base of EHR, experience is being gathered on best practices for EHR implementation. Clinicians using EHR should exploit the vendor's experience base with implementations in similar settings, of course, but should also ‘take ownership’ for leading the configuration of the system for their environment to meet their clinical and business goals. With increasing experience, end-users can adopt best practices based upon the evidence to guide implementation of the EHR system and ensure safe and effective use.

b. Monitor how IT systems are used and report IT-related adverse events

The end-user should monitor how health IT systems are used. Measurement of patient-safety sensitive adverse events, and medical errors, related to IT use should occur regularly and be reviewed internally much like a clinicopathologic case review. Such reviews could be shared like the AHRQ M&M (morbidity and mortality) conferences on the web.⁹³ This monitoring can also be reported in a standard fashion to the vendor to seek the vendor's guidance on remediation, and to a PSO to help gather the evidence base on IT-related adverse events or medical error.

Next steps

These AMIA recommendations on patient safety and the usability of EHR aim to stimulate ongoing and informed discussions and bring about increased understanding of the impact of usability on the safe and effective use of EHR systems. The adoption of useful and usable EHR will lead to safer and higher quality care, and a better return on investment for institutions that adopt them. The recommendations are necessarily wide ranging, and the Task Force on Usability recognizes and appreciates the attention that these issues are already receiving from end-users, vendors, and several federal agencies such as the ONC. Nevertheless, the essential recommendation and next step is that these diverse activities already underway become more coordinated to collectively address the AMIA recommendations described in this article.