Resident Indentified Violations of Usability Heuristic Principles in Local Electronic Health Records

Abstract

Introduction

Difficulties with the electronic health record (EHR) are known to be associated with high physician burnout. Usability studies can evaluate and identify usability issues with the EHR at the end user level. This study was conducted to determine physician perspectives and usability issues of local EHR systems.

Methods

Survey and focus group methodology were employed. Participants were resident physicians who were members of a resident council in the Midwest. Survey data collected included demographics and perceptions. Focus group data included participants identification of usability principle violations and potential impact to end user.

Results

There were 15 survey respondents (across 11 residency programs) who reported use of three different EHR systems: Cerner^®, Meditech, and Computerized Patient Record System (CPRS). Satisfaction was greatest with Cerner^® as well as most reported level of experience. Focus group respondents reported a variety of usability violations which lead to provider confusion, increased time, alert fatigue, and potential patient safety issues.

Conclusion

Violations of usability principles can result in disruption of physician workflow processes and lead to increased documentation time as well as fatigue. These issues have been associated with increased provider burnout. Continuous usability assessments should be conducted at the end user level to promote the development of more effective and efficient EHR interface designs.

INTRODUCTION

Physician burnout has become a prominent topic in healthcare and has been defined as long-term job stress resulting in exhaustion, depersonalization, job detachment, and lack of accomplishment.1 Burnout is dramatically higher in physicians than other fields of work.2,3 As of 2019, 44% of physicians reported suffering from burnout, 11% feel colloquially depressed, and 4% were clinically depressed from some job-related factor.4 Professional consequences of burnout on the job equate to reduced quality of care, errors in healthcare related tasks, reduced patient satisfaction, reduced productivity, and career change.5,6 Personal consequences of burnout outside of work are in the form of depression, substance abuse, relationship problems, and, sometimes, suicide.5,6

In a recent survey study with input from over 15,000 physicians, the top contributors to burnout have been linked to administrative/clerical tasks, hours of work, and the electronic health record (EHR).5 Physician self-reported burnout was correlated positively with the number of hours worked. The wide spread adoption of electronic health records (EHRs) by healthcare providers due to the Health Information Technology for Economic and Clinical Health (HITECH) Act in 2009 was to promote meaningful use of EHRs to improve the quality and efficiency of care delivered through patient care documentation.7 However, a decade later, this improvement has manifested in decreased physician wellness.

The issues of the time demand of EHR documentation have been recorded through time-motion studies.8,9 Results from both time-motion studies showed that physicians spend more than half their day on clerical EHR documentation. For every hour of patient care, physicians may spend up to two hours documenting EHR information. The time cost of EHR documentation carries over outside work hours, during personal time for self-care and disengagement from the demanding healthcare setting. Real time documentation of patient health into EHRs affects workflow, is disruptive to the treatment planning process, and reduces face-to-face communication with the patient.10 The rigidity of the EHR, via the user interface, often results in physicians employing workarounds, which further interrupts efficient workflow.10,11

EHR usability is a major topic of research by academic researchers since the interface’s implementation into healthcare. Usability is defined as the extent to which a system, product, or service can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specific context of use.12 Heuristics are defined as methods that help to identify many kinds of user experience problems. A literature review of usability conducted by Zahabi et al.13 highlighted EHR usability issues in patient care documentation since 2000. In their review, they classified the issues in EHR usability literature based on Molich and Nielsen’s usability principles for usability evaluation of interactive systems.14 Based on their findings, they offered guidelines for EHR interface improvement in ten categories: naturalness, consistency, preventing errors, minimizing cognitive overload, efficient interaction, forgiveness and feedback, effective use of language, effective information presentation, customization/flexibility, and safety issues.13 In fact, guidelines from both academia13,15 and medical associations16 have been offered regarding EHR usability to provide solutions to the requirement of increasing documentation. However, issues in the human-system interaction of the EHR still persist.

A draft report set forth by 21st Century Cures Care Act on strategies to reduce physician’s clerical burden highlighted usability issues that are still prevalent in EHRs that exacerbate the task of patient health documentation.17 In this strategy report, they outlined the need of change in the EHR interface to support clinical workflow, EHR graphical user interface and physical environment, standardization of the interface, and the need to include the end user into design decisions.

The clerical burden put on physicians is a large contributor to physician burnout in the healthcare industry. Physicians as the end users are constrained by EHR documentation regulations, the incongruent workflow of the documentation task, and the interface implementation decisions of these systems. However, physicians may not be consulted on the design of EHR systems to fit their workflow. Therefore, there is a need to understand physicians’ technology uses, needs, and reactions better. The goal of this study is to understand the physicians’ interactions with local EHR systems and the usability issues that impede physicians’ task of patient health documentation.

METHODS

Survey and focus group methodologies were used to understand the experience and perceptions of residents on the use of EHR. This study was reviewed and deemed as not human subjects research by two Institutional Review Boards. Participants were resident physicians and part of a resident council at a medical school in the Midwest. Participants were recruited via email and participation was voluntary. The survey consisted of questions about residents’ demographics, satisfaction, experience, and concern about using an EHR. The focus group script was guided by a heuristic evaluation, including heuristic questions based on Nielsen’s ten usability heuristics (Table 1) for interface design.12 The focus group script included questions about their EHR experience, workflow, and usability.

Table 1.

Definitions of heuristic principles.

Principle	Definition
Recognition Rather than Recall	Minimize the user’s memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.
Match between System and the Real World	The system should speak the user’s language with words, phrases, and concepts familiar to the user, rather than system-oriented terms. It should follow real-world conventions, making information appear in a natural and logical order.
Visibility of System Status	The system always should keep users informed about what is going on, through appropriate feedback within a reasonable time.
Consistency and Standards	Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.
Aesthetic and Minimalist Design	Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.
Flexibility and Efficiency of Use	Accelerators, unseen by the novice user, often may speed up the interaction for the expert user such that the system can cater to both inexperienced users. Users should be permitted to tailor frequent actions.
Error Prevention	Even better than good error messages is a careful design which prevents a problem, from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.
User Control and Freedom	Users often choose system functions by mistake and will need a clearly marked ‘emergency exit’ to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.
Help Users Recognize, Diagnose, and Recover from Errors	Error messages should be expressed in plain language (no codes), should precisely indicate the problem, and should constructively suggest a solution.
Help and Documentation	Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user’s task, list concentrate steps to be carried out and not be too large.

The survey request was emailed prior to the focus group. The focus group took place during a regularly scheduled Resident Council meeting. It was led by a trained facilitator and two trained researchers taking notes. The lead facilitator provided definitions and examples for each of the usability principles. Participants were asked to provide an example of a problem (violation) they experienced related to the usability principle, as well as a possible solution. Due to time constraints, only 7 of the 10 usability design principles were addressed in the focus group.

Data Analysis

Survey responses were summarized using means (standard deviations) and frequencies (percentages) where appropriate. Research team members identified problems with underlying usability principles. Disparities in categorization were addressed by mutual consensus.

RESULTS

The survey was comprised of 15 resident physicians (male = 9, female = 6; age: M = 30.2 years, SD = 1.78 years). Table 2 shows the residency programs represented. There were three EHR systems used in this group: Cerner^® (https://www.cerner.com), Meditech (https://ehr.meditech.com/), and Computerized Patient Record System (www.va.com).

Table 2.

Survey demographics (n = 15).

Residency Program	Frequency (%)
Anesthesiology	2 (13%)
Family Medicine	1 (7%)
Internal Medicine	1 (7%)
Medicine/Pediatrics	1 (7%)
Obstetrics/Gynecology	2 (13%)
Orthopedic Surgery	1 (7%)
Pediatrics	1 (7%)
Psychiatry	2 (13%)
Radiology (Diagnostic)	2 (13%)
Sports Medicine*	1 (7%)
Surgery	1 (7%)

^*The Sports Medicine program is a fellowship rather than a residency.

Table 3 shows the representation of EHRs used, resident physician’s satisfaction with EHRs used, and resident physician’s perceived experience level. Most (11/15, 73%) reported using more than one EHR. Most (11/15, 73%) reported using Meditech. Cerner^® was reported most often as being the first-used EHR (7/15, 47%). The highest satisfaction was reported (4.25) with Cerner^®. Only two respondents were most satisfied with the first EHR system learned.

Table 3.

Resident reported EHR use, user experience, and satisfaction.

	Meditech	Cerner®	CPRSa	Other
EHR Usedb	11	9	7	6
EHR First Used (%)	5 (33%)	7 (47%)	1 (7%)	2 (13%)
User Levelc M (SD)	3.75 (0.40)	4.13 (0.35)	3.29 (0.49)	3.33 (0.52)
EHR Satisfactiond M (SD)	3 (0.87)	4.25 (0.46)	2.57 (1.40)	4 (0.71)

^aCPRS = Computerized Patient Record System

^bRespondents can choose more than one EHR being used.

^cUser level scale: 1–2 = Inexperienced, 3 = Intermediate, 4–5 = Experienced

^dSatisfaction scale: 1 = Very Dissatisfied, 2 = Dissatisfied, 3 = Neutral, 4 = Satisfied, 5 = Very Satisfied

Table 4 shows the training that respondents received with an EHR (respondents were able to choose more than one training choice). Most reported having been trained via orientation.

Table 4.

Resident reported training modalities.^*

Training Modality	Meditech	Cerner®	CPRSa	Other
Orientation	11	8	5	0
IT Support	4	5	1	2
Individual Instruction	3	4	2	1
Superuser 1:1	0	1	1	0
Webinar	0	0	1	0
Workshop	0	0	0	1
None	0	2	0	1

^*Respondents were able to choose more than one training choice.

^aCPRS = Computerized Patient Record System

Table 5 shows respondents’ use of the EHR per day and use outside of patient care. Most respondents reported spending more time per day (5.9 hours per day) with Cerner^®. Respondents also reported spending more time outside of patient care (1.8 hours per day) using Cerner^®.

Table 5.

Resident reported time use by system.^*

	Meditech	Cerner®	CPRSa
Hours per day	4.18 (4.47)	5.90 (4.01)	3.44 (5.15)
Hours per day outside of patient care	0.95 (0.91)	1.80 (1.40)	0.71 (1.25)

^*Mean (Standard Deviation)

^aCPRS = Computerized Patient Record System

Figure 1 shows the respondent’s attitudes toward the EHR system. Most agreed that the EHR takes up too much of their time. Less than half agreed they were concerned about their inability to manage the EHR. Most agreed that they are preoccupied with things other than the EHR and not concerned about the EHR at this time.

Figure 1.

Attitudes toward the EHR.

Focus Group

The focus group (n = 11) was comprised of 11 resident physicians (Male = 7, Female = 4). One information technology support technician was there solely for observation. Table 6 (appendix) summarizes the focus group findings regarding usability principle violations with the EHR systems and potential negative outcomes.

DISCUSSION

The main objective of this study was to understand physician’s viewpoint of local EHR systems and the usability issues that impede physicians’ task of patient healthcare. Findings revealed that resident physicians reported Cerner^® used the most time with daily tasks, yet they were most satisfied with Cerner^® as compared to the other local EHRs. Residents reported they were concerned with other things than EHR at this time, however, they indicated that the EHR takes up too much time. Resident physician’s comments highlighted the inability of the EHR to integrate into their workflow, which can compromise patient safety and certainly cause frustration.

Resident physicians were able to identify problems (usability violations) in 7 out of the 10 principle categories discussed. The main issue of increased interaction time with the EHR comes from physicians having to recall information and increased search times among cluttered, irrelevant, and redundant information. This negative outcome directly effects the daily hours a physician may need to document patient information while at work and outside of working hours. Many violations resulting in leaving out pertinent information and keeping physicians out-of-the-loop should be addressed as potential patient safety issues. Other issues, such as alarm fatigue and confusion, should be addressed, so potential outcomes of patient information are not overlooked.

Although Zahabi et al.13 used Molich and Nielsen’s usability principles for an extensive literature review of EMR/EHR use,14 it was decided more appropriate to use Neilsen’s updated usability heuristics12 in a focus group setting to understand specific problems faced by local resident physicians better. While there were similar findings between the literature review by Zahabi et al.13 and EHR guidelines, such as inefficient interaction, redundant information, clutter, information overload, and “alert fatigue”, the focus group allowed insight into more specific user problems based on their specific healthcare residency. The ability to identify specific usability problems based on certain roles and healthcare areas, may provide insight into design recommendations to optimize the amount of time required to document patient health information. This optimization for better EHR integration into specific workflows and less time on documentation duties, may reduce the EHR’s clerical and documentation burden as a contributor to burnout.

Limitations

There are aspects of this study that may limit generalizability of findings. First, a representative sample size of each residency may expand usability issues specific with resident EHR tasks. Second, usability principle violations cannot be generalized to all EHR interface types due to each EHR vendor having different EHR formats and customized options through information technology departments to support physician EHR related tasks. Third, burnout in physicians was not specifically measured, however, time on EHR related tasks has been linked to physician burnout.5

Future Directions

Future research should conduct similar studies with a broader and more representative sample from different hospitals and physicians, with a wider range of technology use experiences. It is also necessary to conduct similar studies across different hospitals with different EHR systems and formats. Given the nature of focus groups, it is necessary to conduct other usability methods and observation of EHR tasks with resident physicians.

CONCLUSIONS

Physician burnout has been linked to EHR use, specifically that excessive time needed for documentation takes away time from patient care and physician personal time. The design of the EHR may increase documentation time needed due to design flaws and cause usability issues that disrupt patient care and documentation. Overall, EHR problems can be delineated into specific usability violations which can be extrapolated into negative outcomes regarding patient safety, increased time, confusion, alert fatigue, and dissatisfaction. Including physicians/residents in EHR related design can improve the incongruent EHR/job demands and potentially reduce burnout. In general, the findings are expected to be useful for general EHR design and information technology support groups within hospital systems to provide changes to the EHR to make specific physician related EHR tasks more efficient and less time consuming.

Appendix

Table 6.

Resident identified EHR usability issues and potential negative outcomes.

Usability Design Principle: Recognition Rather than Recall
Problem Identified from Focus Group Quotes “Moving between notes and charting, often the en-counter will close and force you to reopen the chart repeatedly.”	Negative Outcome Increase note taking time/charting	Themes* T	Residency Surgery
“Meditech makes it easy to lose a searched patient.”	Causes extra time re-searching patient	T	Radiology
“Having to remember old notes or go back to old notes.”	Increase note taking time	T	Orthopaedic Surgery
Usability Design Principle: Not a Match Between System and Real World
Problem Identified from Focus Group Quotes	Negative Outcome	Themes*	Residency
“No free text for orders.”	May leave out pertinent information to the order	PS	Internal Medicine
“Not enough character text for imaging reasons for exam.”	May leave out pertinent information to the order	PS	Radiology
“No free text for orders.”	May leave out pertinent information to the order	PS	None given
Usability Design Principle: Visibility of System Status
Problem Identified from Focus Group Quotes	Negative Outcome	Themes*	Residency
“Pharmacy can change antibiotics from IV to PO without notification.”	Physician out-of-the-loop	PS	Surgery
“Orders are not transferred when patients are in ER, we cannot put in orders, or see who is in charge of their care.”	Physician out-of-the-loop	PS	Internal Medicine
“Unable to see med rec when someone else is using it. Only one person in a chart at a time able to order at a time.”	Physician out-of-the-loop, increased waiting time/ forgetting to chart	PS, T	Anesthesia
“Not notified when pharmacy changes orders.”	Physician out-of-the-loop	PS	Orthopaedic Surgery
“Pharmacy changes orders without notification to providers. Applies to lab cancelling orders as well.”	Physician out-of-the-loop	PS	None given
Usability Design Principle: Consistency and Standards
Problem Identified from Focus Group Quotes	Negative Outcome	Themes*	Residency
“Variations in names of tests (i.e., us vs. doppler vs. ultrasound.”	Inconsistent names and requires unneeded learning of extra name and increase time ordering tests	C	Internal Medicine
“Radiological exams may have overlapping investigative results.”	More than one exam needed to be ordered	C	Radiology
“Misunderstanding subspecialty specific exams like "Hi-res chest CT" is not actually "High definition" or helps us see thing better.”	Unclear wording and inconsistent language may cause confusion	C	Radiology
“Attendings patients show up on resident’s lists.”	May cause confusion of who need to take care of patients	C	Obstetrics/Gynecology
Usability Design Principle: Aesthetic and Minimalist Design
Problem Identified from Focus Group Quotes	Negative Outcome	Themes*	Residency
“Ordering magnesium w/warnings.”	Extra time need to address the pop-up warning, alarm fatigue	T, AF	Internal Medicine
“Ine_cient display of relevant physician notes, either SOAP/progress notes, H&P, and D/C summary. Need a way to make the aforementioned notes to the front. Billing and system audit options in Meditech should be in a separate menu or hidden.”	Causes extra time to search through irrelevant information and clutter	T	Radiology
“Excessive pop up warnings.”	Extra time needed to address the pop-up warning, alarm fatigue	T, AF	Orthopaedic Surgery
Usability Design Principle: Flexibility and Efficiency of Use
Problem Identified from Focus Group Quotes	Negative Outcome	Themes*	Residency
“Non-personalized templates/order sets in Meditech.”	May cause inefficient interaction with the EHR, causing extra time in interaction.	T	Ortho Surgery
“No dot phrases in Cerner except for most recent ECG results.”	Redundant information, increase in interaction time	T	Psychiatry
“No dot phrases.”	Redundant information, increase in interaction time	T	Obstetrics/Gynecology
“No dot phrases.”	Redundant information, increase in interaction time	T	Surgery
“When order sets changed, drops off favorites list.”	Inefficient interaction of use	T	Obstetrics/Gynecology
Usability Design Principle: Error Prevention
Problem Identified from Focus Group Quotes	Negative Outcome	Themes*	Residency
“Multiple drug interactions pop up. Make the major/potentially fatal drug interactions red.”	Excessive drug interaction pop ups may cause physicians to overlook potentially fatal interactions	PS, T, AF	Sports Medicine*
“Meditech: Adverse reaction listed when using medications include an overwhelming amount of pop up warnings. Need to prioritize adverse reaction in order of severity and rank them from top to bottom in warning page.”	Excessive drug interaction pop ups may cause physicians to overlook potentially fatal interactions	PS, T, AF	None given

^*Theme Codes: PS = Patient Safety issue; T = increased time; C = confusion; AF = alert fatigue.

Dot phrases: Dot (.) phrases are shortcuts provided by the EHR for long phrases and patient data, to reduce clutter and redundant information.¹⁸

^*The Sports Medicine program is a fellowship rather than a residency.