Abstract
Objectives
Examine electronic health record (EHR) use and factors contributing to documentation burden in acute and critical care nurses.
Materials and Methods
A mixed-methods design was used guided by Unified Theory of Acceptance and Use of Technology. Key EHR components included, Flowsheets, Medication Administration Records (MAR), Care Plan, Notes, and Navigators. We first identified 5 units with the highest documentation burden in 1 university hospital through EHR log file analyses. Four nurses per unit were recruited and engaged in interviews and surveys designed to examine their perceptions of ease of use and usefulness of the 5 EHR components. A combination of inductive/deductive coding was used for qualitative data analysis.
Results
Nurses acknowledged the importance of documentation for patient care, yet perceived the required documentation as burdensome with levels varying across the 5 components. Factors contributing to burden included non-EHR issues (patient-to-nurse staffing ratios; patient acuity; suboptimal time management) and EHR usability issues related to design/features. Flowsheets, Care Plan, and Navigators were found to be below acceptable usability and contributed to more burden compared to MAR and Notes. The most troublesome EHR usability issues were data redundancy, poor workflow navigation, and cumbersome data entry based on unit type.
Discussion
Overall, we used quantitative and qualitative data to highlight challenges with current nursing documentation features in the EHR that contribute to documentation burden. Differences in perceived usability across the EHR documentation components were driven by multiple factors, such as non-alignment with workflows and amount of duplication of prior data entries. Nurses offered several recommendations for improving the EHR, including minimizing redundant or excessive data entry requirements, providing visual cues (eg, clear error messages, highlighting areas where missing or incorrect information are), and integrating decision support.
Conclusion
Our study generated evidence for nurse EHR use and specific documentation usability issues contributing to burden. Findings can inform the development of solutions for enhancing multi-component EHR usability that accommodates the unique workflow of nurses. Documentation strategies designed to improve nurse working conditions should include non-EHR factors as they also contribute to documentation burden.
Keywords: electronic health record, documentations, documentation burden, nurses, usability, evaluation, user interface
Introduction
The use of electronic health record (EHR) systems has improved quality of care and patient safety, and reduced healthcare costs.1–3 Despite these advancements, EHR systems are associated with negative user outcomes including a significant increase in documentation burden4–7 and clinician burnout,8,9 with potential for adverse patient outcomes.10,11 According to a national survey of over 400 000 nurses, more than one-third (34%) of respondents planned to leave direct patient care by the end of the year because of burnout.12 Since nurse burnout is, in part, due to documentation burden,13 health systems are seeking solutions to optimize their EHR components as one strategy to address nurse turnover.
Nurses are one of the largest user groups in health information technology and have high levels of documentation requirements. Nurses spend approximately 35% of their shift time documenting patient care,14 limiting the time spent in direct patient care.14,15 A factor that could potentially reduce interaction time with EHRs is providing usable systems, which simply means that all of the components of EHRs are functioning as intended.8,16–19 However, current EHR testing efforts do not adequately consider user interface features and workflow conditions that fully support the intended user.20–22 As a result, the components of current EHRs that significantly contribute to documentation burden among nurses are not fully understood thereby hindering advancements to facilitate effective and efficient nursing care and desired outcomes.23–26 Given the complexity of nurses’ documentation burden due to multiple contributing factors, this study focused on identifying specific aspects of an EHR system that support clinical workflow of acute and critical care nurse clinicians. We also sought to identify features that could reduce documentation burden while advancing quality care.
Although nurses, who provide the majority of frontline care for patients, are disproportionately affected by the current state of EHR systems, there is little known about their lived experiences using them. Existing studies on clinician perspectives of EHR usability have predominantly focused on physicians,27,28 with the few studies on nurses being limited to quantitative analyses based on surveys,29 log files,30 or time-motion methods.31,32 The quantitative approach (including single data collection) of these prior studies does not fully capture the lived EHR experiences of nurses in day-to-day practice while caring for patients. Therefore, we added a qualitative component to the current study to fill this gap and augment our knowledge of the impact of EHRs on nurses.
Clinicians practicing in inpatient settings have one of the highest rates of burnout (>50%),33 spending a significant amount of their shift time documenting patient care within EHRs.30 In this study, we sought the input of nurse clinicians in acute and critical care units to deepen our understanding of their real-world experiences with EHRs including the strengths, barriers, and frustrations. The objective of this study was to examine nurses’ current use of an EHR system and identify factors contributing to documentation burden in acute and critical care settings at a large academic medical center.
Methods
A mixed-methods sequential explanatory design, which starts from quantitative analysis to qualitative analysis,34 was used to examine EHR use and perceptions of ease of use/usefulness of the key components of an EHR system in acute and critical care nurses, using analyses of data entry log files, one-on-one interviews, and surveys. Given the paucity of research on nurses’ current EHR use across multiple components in acute and critical care units and their perceived documentation burden, we used a qualitative research approach (ie, interviews) to explore salient factors in addition to the quantitative methods (ie, log file analyses and surveys).
Conceptual framework
This study was guided by the Unified Theory of Acceptance and Use of Technology (UTAUT),35 which posits that 4 factors including performance expectancy, effort expectancy, social influence, and facilitating conditions, are direct determinants of technology usage intention and behavior. It guided the design of our semi-structured interviews and the synthesis of data sources. The 4 factors operationalized in our study by the UTAUT model were based on its rooted construct and definition for each factor,35 which are presented in Table 1.
Table 1.
Key factors operationalized by the unified theory of acceptance and use of technology (UTAUT).35
| UTAUT construct | Operationalization in the study |
|---|---|
| Performance expectancy | User perception of the usefulness of electronic health record (EHR) components for documentation |
| Effort expectancy | User perception of the ease of use of EHR components for documentation |
| Social influence | User perception of EHR value by important others (eg, important others in the patient care team use the EHR components to augment team communication and inform care decisions) |
| Facilitating conditions | User perception of resource and support availability to use EHR components for documentation |
Study setting
The study setting was an academic medical center located in northeastern Florida, where Epic (Verona, WI) was implemented as the organization’s EHR system. The EHR components examined in the study were: (1) Flowsheets (a component detailing vital signs, intake/output, physiological assessments, treatment parameters such as daily care, wound, and ostomy care), (2) Medication Administration Records (MAR; a component detailing the drugs administered to patients), (3) Care Plan (a component creating, reviewing, and updating nursing care plans for patients), (4) Notes (a component primarily documenting patient care and communication with other care providers), and (5) Admission-Discharge-Transfer (ADT) Navigators (a component tracking inpatient visits from their arrival [admission], to their movements inside the hospital [transfer], to their departure [discharge]). The 5 EHR components were chosen because they are most frequently used by nurses in acute and critical care units.7,30,36,37
Sampling and recruitment
For interviews and surveys, we recruited nurses from units with a high likelihood of documentation burden as identified via analyses of data entry log files in Epic (details in “Data Collection”). Using a convenience sampling method, we sought to enroll 20 participants to ensure the likelihood of data saturation.38 The inclusion criteria included registered nurses (RNs) who (1) held an active RN license in the United States, and (2) had been employed and actively working as a staff nurse for more than 1 year in an acute or critical care unit at the targeted academic medical center. Utilizing our strong academic-practice partnership, we facilitated recruitment by attending executive committee meetings and unit leadership group meetings to introduce our study and distribute physical and electronic copies of recruitment flyers to staff nurses. When potential participants contacted a research coordinator by email, a screening phone call took place where they were asked basic questions about their work history to determine eligibility to participate.
Data collection
Quantitative data
The Nursing Efficiency Assessment Tool (NEAT)39 was used for analyses of data entry log files. The NEAT provides data entry log files of clinician use of various EHR components (eg, Flowsheets, MAR, Notes, Care Plan, ADT Navigators), and the raw dashboard data within the NEAT facilitates prioritizing continuous improvement opportunities and creating targeted action plans for specific clinicians. The NEAT collates time spent in Epic based on typical system behaviors for documentation (eg, mouse clicks; 30-second inactivity timeout) during the 12-hour nursing shifts. The inclusion criterion for log file analyses was inpatient units at the study site. We included only adult care units, given the differences in clinicians’ EHR use across adult and pediatric care units.40
The measurement of the average time spent documenting has often been used through EHR usage logs.7 Using the NEAT datasets, we extracted data entry log files spanning September-October 2022 including user identifier, clinical unit, time (minutes) spent in data entry, and activities (EHR components) to identify clinical units with the highest levels of nurse documentation burden. As time is one of the important indicators for documentation burden, the following measures of time spent in documentation were calculated: (1) Number of Records was calculated by identifying the number of records (entries) in the EHR during the study period. Number of Records was calculated individually for each of the 5 EHR components by clinical unit. (2) Minutes Per Record was calculated for each EHR component separately for each clinical unit by summing the number of minutes within an EHR component and dividing by the number of records within that EHR component. That value represents the average amount of time spent on making one entry in an EHR component by nurses within a clinical unit. (3) Total minutes was calculated by summing the amount of time spent making an entry into an EHR component within each of the 5 EHR components by nurses within a clinical unit over the study period. That value represents the total amount of time spent by nurses within a clinical unit making entries within an EHR component over the study period. (4) Level of documentation burden was calculated by summing the Total Minutes across the 5 EHR components and dividing by the total number of records present. That value represents the average amount of time making an EHR component entry, across the 5 components examined, by nurses within a clinical unit.
Qualitative data
We conducted one-on-one interviews from June to September 2023 with nurse participants who worked in the 5 units experiencing the highest documentation burden. Each interview was approximately 60-75 minutes in length. Once participants completed the informed consent process, all interviews were audio-recorded. The 5 key components of the Epic EHR (ie, Flowsheets, MAR, Care Plan, Notes, and ADT Navigators) were discussed during the interviews. Guided by the UTAUT,35 our interviews were semi-structured to cover each of the 5 EHR components included in quantitative data collection. The interview guide, which was designed to gain rich, detailed data through its open-ended nature and flexibility to ask probing questions (eg, contextual autocomplete feature which was not live in the Epic EHR at the time of our study and introduced with explanations),41,42 can be found in Supplementary File 1. All interviews were conducted in person with a clinical informatician trained in qualitative research (O.T.N. trained by H.C.) by encouraging participants to talk about their experiences and perceptions of their current use of the EHR components. Data collection continued until we reached saturation by the point at which no new themes emerged as discussed at our study team’s weekly meetings.
Following completion of interviews, participants were asked to complete a survey (through Qualtrics)43 on nurses’ demographics and background information (eg, gender, age, unit in acute/critical care, education, years of use of the EHR), and rate perceived usability and documentation burden for each EHR component. Perceived usability was measured using the System Usability Scale (SUS),44 consisting of 10 items rated on a 5-point Likert scale from strongly disagree (1) to strongly agree (5). The SUS is scored between 0 and 100, and the SUS score above 68 is considered acceptable usability.45,46 Perceived documentation burden was measured using the self-reported questionnaires rated on a 5-point Likert scale from not at all likely (1) to extremely likely (5) (ie, please rate your perceived documentation burden when using [each EHR component]). A higher scale value indicates a higher documentation burden of the EHR component.
Data analysis
Quantitative data
Descriptive statistics were used to calculate the frequencies, percentages, means, and standard deviations, including data entry log files of Epic use and surveys assessing demographics information, SUS scores, and perceived documentation burden questionnaires. The mean scores for perceived usability and documentation burden were reviewed across the 5 EHR components.
Qualitative data
All interviews were transcribed verbatim. Transcripts from audio recordings of interviews were independently reviewed and analyzed to develop codes per EHR component by 2 research team members (O.T.N. and J.P.), using a combination of inductive and deductive coding techniques.47 A set of codes—(a) usability issues contributing to documentation burden and (b) general barriers contributing to documentation burden and recommendations were generated. A codebook was developed and free text excerpted from the transcripts was entered into the codebook followed by each of the codes, seeking ≥85% agreement for deductive coding. Inductive coding was performed to identify and finalize the main themes with similar patterns analytically related and repeated across interview data in the codebook. Discrepancies were resolved by a third study team member (H.C.). Our research team also conducted peer debriefing and triangulated findings across multiple data sources (eg, interviews; surveys) to enhance the confirmability of interpretations and create a more comprehensive picture of the specific issues with nursing documentation that contribute to documentation burden.48
Ethical approval
The Institutional Review Board (IRB) of the University of Florida reviewed and approved all research activities including secondary data analyses (ie, data entry log files) and human subjects research (ie, interviews) (IRB#202202550). Informed consent was obtained from all participants. All participants were compensated with a $150 gift card for their time.
Results
Documentation burden by unit
Using the analyses of data entry log files for EHR use, we identified the top 5 acute and critical care units in which nurses spent the most time on documentation in the 5 EHR components (ie, Flowsheets, MAR, Care Plan, Notes, and ADT Navigators), which are presented in Table 2. Of 31 inpatient units (19 acute and 12 critical care), the 5 units identified as those with the highest levels of nurse documentation burden were 2 general medical-surgical units, 1 intermediate medical care unit (IMCU), 1 medical intensive care unit (MICU), and 1 surgical ICU. Distribution of time spent documenting in the 5 EHR components varied across the 5 units. Flowsheets and MAR were the 2 components nurses spent the most time documenting across all units; the third component differed by unit.
Table 2.
Descriptive statistics, including frequencies and the level of documentation burden for minutes in each of the 5 selected activities during the study period by unit.
| Unit (number of nurses) | Activity: electronic health record (EHR) component | Number of recordsa | Minutes per record (SD) | Total minutes | Level of documentation burden for the 5 EHR components (mean minutes) |
|---|---|---|---|---|---|
| Unit A (80) | Flowsheets | 405 | 167.2 (395.7) | 67 702.9 | 78.5 |
| MAR b | 415 | 47.1 (94.9) | 19 557.9 | ||
| Notes | 130 | 8 (15.3) | 1040.5 | ||
| Care Plan | 56 | 1.9 (2.7) | 106.7 | ||
| ADT Navigators c | 121 | 0.9 (1.2) | 110.0 | ||
| Unit B (58) | Flowsheets | 319 | 157.1 (404.1) | 50 108.2 | 80.5 |
| MAR | 274 | 54.3 (121.1) | 14 881.0 | ||
| Notes | 81 | 17.8 (40.6) | 1441.0 | ||
| Care Plan | 33 | 5.6 (10.3) | 184.6 | ||
| ADT Navigators | 124 | 1.9 (2.5) | 235.9 | ||
| Unit C (50) | Flowsheets | 260 | 173.1 (411.1) | 45 018.8 | 81.1 |
| MAR | 253 | 46.5 (95.2) | 11 764.6 | ||
| Notes | 87 | 17.6 (38.5) | 1532.3 | ||
| Care Plan | 34 | 3.2 (8) | 109.1 | ||
| ADT Navigators | 90 | 3.1 (8.8) | 282.7 | ||
| Unit D (41) | Flowsheets | 206 | 170.7 (411) | 35 159.9 | 88.4 |
| MAR | 176 | 85.6 (180) | 15 061.9 | ||
| Notes | 73 | 26.5 (40.5) | 1932.3 | ||
| Care Plan | 140 | 23.3 (46.4) | 3269.4 | ||
| ADT Navigators | 35 | 7.4 (9.4) | 258.1 | ||
| Unit E (13) | Flowsheets | 77 | 162.1 (414) | 12 480.8 | 83.1 |
| MAR | 73 | 81 (189.3) | 5913.3 | ||
| Notes | 24 | 25.3 (33.4) | 607.3 | ||
| Care Plan | 13 | 14 (10.3) | 182.2 | ||
| ADT Navigators | 49 | 8.8 (33.2) | 430.5 |
Number of records: the number of data entries a particular activity shows up within a particular unit during the study period.
MAR: medication administration records.
ADT navigators: admission-discharge-transfer navigators.
Sample
A total of 20 RNs who were working in the 5 identified units (4 RNs from each unit) participated in one-on-one interviews and surveys. The sample characteristics are presented in Table 3. The mean age for study participants was 34.1 years (SD = 10). The majority (75%; n = 15) in our sample had a bachelor’s degree in nursing and cared for more than 2 patients at a time (80%; n = 16). Of the participants, 40% (n = 8) had 1 year of nursing experience, and all participants (100%; n = 20) felt somewhat or very comfortable with digital technology.
Table 3.
Sample characteristics.
| Characteristic | n (%) |
|---|---|
| Age (in years), mean (SD) | 34.1 (10) |
| Gender | |
| Female | 17 (85) |
| Male | 3 (15) |
| Race | |
| White | 12 (60) |
| Black or African American | 2 (10) |
| Asian | 5 (25) |
| Not listed | 1 (5) |
| Ethnicity | |
| Hispanic/Latino | 2 (10) |
| Educational attainment in nursing | |
| Associate’s degree | 3 (15) |
| Bachelor’s degree | 15 (75) |
| Master’s degree | 2 (10) |
| Patient volume | |
| 1-2 | 4 (20) |
| 3-4 | 8 (40) |
| 5-6 | 8 (40) |
| Type of patient cared for | |
| Cancer | 17 (85) |
| HIV | 13 (65) |
| Dementia | 17 (85) |
| Cardiovascular diseases | 18 (90) |
| Respiratory diseases | 19 (95) |
| Digestive diseases | 18 (90) |
| Years of experience as a nurse | |
| 1 year | 8 (40) |
| 2-5 years | 6 (30) |
| More than 5 years | 6 (30) |
| Perceived digital literacy | |
| Very comfortable | 15 (75) |
| Somewhat comfortable | 5 (25) |
Findings
Perceived usability and documentation burden (from quantitative data analysis)
Nurses’ perceived usability and documentation burden in the 5 EHR components are presented in Table 4. Regarding nurses’ perceptions of documentation burden from using each of the 5 EHR components, Flowsheets and Care Plan were found to be the most cumbersome as more than 60% of nurse participants reported as “somewhat” to “extremely” burdensome for documentation. The SUS scores per EHR component are presented in Figure 1. The overall SUS scores indicated that only 2 EHR components, MAR and Notes had acceptable usability rated at above 68.45,46
Table 4.
Mean (standard deviation [SD]) for overall system usability scale (SUS) scores and frequency (%) for individual domain statements for perceived usability and documentation burden in 5 electronic health records (EHRs) components.
| Perceived usability a | Flowsheets | Medication administration records (MAR) | Care plan | Notes | Admission-discharge-transfer (ADT) navigators |
|---|---|---|---|---|---|
| Overall system usability scale (SUS) score | 62.13 (16.13) | 80.88 (13.31) | 46.88 (19.45) | 75.63 (17.68) | 52.38 (19.53) |
| Used the component frequently | 11 (55) | 16 (80) | 5 (25) | 13 (65) | 5 (25) |
| The feature was unnecessarily complex | 9 (45) | 1 (5) | 13 (65) | 4 (20) | 12 (60) |
| The feature was easy to use | 15 (75) | 17 (85) | 11 (55) | 15 (75) | 11 (55) |
| Needed a technical person to use the component | 2 (10) | 0 (0) | 3 (15) | 0 (0) | 3 (15) |
| Functions in the component were well-integrated | 13 (65) | 17 (85) | 5 (25) | 14 (70) | 11 (55) |
| Too much inconsistency in the component | 6 (30) | 0 (0) | 10 (50) | 0 (0) | 6 (30) |
| Learned to use the component quickly | 13 (65) | 17 (85) | 7 (35) | 14 (70) | 7 (35) |
| The component was cumbersome to use | 8 (40) | 1 (5) | 10 (50) | 1 (5) | 9 (45) |
| Felt confident using the component | 16 (80) | 19 (95) | 8 (40) | 17 (85) | 7 (35) |
| Had to learn a lot before using the component | 5 (25) | 3 (15) | 8 (40) | 1 (5) | 8 (40) |
|
| |||||
| Perceived documentation burden | Flowsheets | MAR | Care plan | Notes | ADT navigators |
|
| |||||
| Not burdensome at all | 1 (5) | 13 (65) | 1 (5) | 9 (45) | 2 (10) |
| A little burdensome | 7 (35) | 5 (25) | 6 (30) | 5 (25) | 9 (45) |
| Somewhat burdensome | 6 (30) | 2 (10) | 9 (45) | 6 (30) | 6 (30) |
| Very burdensome | 5 (25) | 0 (0) | 3 (15) | 0 (0) | 1 (5) |
| Extremely burdensome | 1 (5) | 0 (0) | 1 (5) | 0 (0) | 2 (10) |
Frequencies and percentages represent nurses who strongly agreed or agreed with each subdomain statement.
Figure 1.
System usability scale (SUS) score per electronic health records (EHR) component.
Factors contributing to documentation burden across EHR components (from qualitative data analysis)
The major themes of factors contributing to the nurse documentation burden identified from each of the 5 EHR components included: (1) in Flowsheets: repetitive data entry required; lack of “undo” functionality; unable to switch screens; poor customizability not tailored to the unit; insufficient training; insufficient time to document (especially related to the number of patients), (2) in MAR: lack of flexibility; difficulty searching for medications; missing barcode scanning for certain items; comfort with use, (3) in Care Plan: lack of a search bar for nursing terminologies; unclear information architecture; difficulty deleting duplicates of care plans created by colleagues; no value to patient care in checking care plans; value to only nursing care as not used by non-nurses in the care team; insufficient training; insufficient time to document, (4) in Notes: fear of notes seen by patients; insufficient time to document, and (5) in ADT Navigators: absence of non-applicable options; insufficient training; insufficient time to document. Additional details containing sample quotations for each theme are presented in Supplementary File 2. Major themes and illustrative quotations EHR component. We summarized the identified factors contributing to documentation burden across the 5 EHR components as—usability-related issues, and general barriers and recommendations.
Usability issues contributing to documentation burden
Several usability issues were reported across the 5 EHR components. Most of the usability problems were related to a lack of functionality, flexibility, and customizability. The nurses perceived that the usability issues resulted in poor workflow navigations and difficulty finding information within the EHR. For example, in Flowsheets documentation, nurses expressed that the lack of “undo” functionality produced more documentation work to fix the errors since one documentation error affected several rows and nurses had to remove the incorrect documentation from all affected rows (one by one). Several nurses also pointed out that EHR data entry elements in Flowsheets (eg, repetitive or not tailored to the specific unit), MAR (eg, missing barcode scanning for nutritional supplements), and ADT Navigators (eg, missing non-applicable options) were not aligned with the clinical workflow. Specifically, acute care nurses verbalized a need for the data entry elements in Flowsheets to be well-customized and tailored to their acute care units as they perceived the component was more likely designed for use in critical care units, saying “I wish they would be customized more to the unit. Some of those tabs are unnecessary depending on the unit.” Also, in Care Plan, nurses reported poor usability due to a lack of a search bar for nursing terminologies (eg, “It needs… parenthesis or single colons to help search for any word containing that”), unclear information architecture (eg, “It’s difficult to find care plan information”), and difficulty deleting duplicates of care plans created by colleagues (remaining several “same” care plans; eg, “I get that you can’t just delete it because it was on their chart and it’s like on the record”). No specific usability problem was reported in Notes.
General barriers contributing to documentation burden and recommendations
Our study showed that acute and critical care nurses faced well-known challenges such as high patient-to-nurse staffing ratios, patient acuity, and suboptimal time management in their everyday care with dynamic and complex patient cases. Nurses expressed that their documentation burden was often caused by limited time for completing documentation tasks within the number of the assigned patients and the number of the admissions and discharges across the shift. In addition, nurses pointed to the need for appropriate training for documenting in Flowsheets, Care Plan, and ADT Navigators, saying “We need in-depth training on it, not like ‘this is your care plan, this is what you do with a shift, and that’s that’.” Particularly for the Care Plan documentation, nurses emphasized that creating, checking, and updating care plans every shift were important to patient care; however, they felt that it was valued only to nursing care as non-nurse clinicians (eg, physicians) in the patient care team did not view the care plans, saying “No one looks at them in the team. I’m just wasting time doing them honestly because I’d rather be taking of patients.” Some nurses did not see the value of nursing care plans and found it discouraging to check existing care plans or just copied and pasted them (duplicates of care plans created but difficulty deleting them) during their shifts as required to meet documentation policies, saying “It makes me so mad when I’m trying to find out, is this patient is a walkie-talkie, and I have seven nurses that have charted on the patient. All it is, is copy and paste of the care plan.”
Discussion
We conducted an in-depth analysis and synthesis of data from log files, one-on-one interviews, and surveys and assessed factors contributing to the nurse documentation burden. Our study showed that nurses acknowledged the importance of documentation at the point of care and its meaningfulness to patient care, yet perceived the required documentation as burdensome. Overall, nurses reported varied levels of documentation burden stemming from each EHR component due to its poor usability with inadequate consideration of the unique aspects of acute and critical care with dynamic and complex patient cases. Possible factors contributing to documentation burden included general barriers such as high patient-to-nurse staffing ratios, patient acuity, and suboptimal time management, and usability issues related to the design and features (ie, functionality, flexibility, customizability) of the EHR components. Three EHR components, Flowsheets, Care Plan, and ADT Navigators, were found to be below the acceptable usability score range and contributed more to documentation burden as compared to MAR and Notes.
Our study showed distinct distribution of time spent documenting in the 5 EHR components among the top 5 units. This might reflect different nursing workflow and documentation patterns across care units.49 During the interviews, nurses reported that data entry elements in Flowsheets, MAR, and ADT Navigators were not well-aligned with their workflow. For example, nurses in critical care units expressed that they frequently managed admissions, transfers, and discharges but did not have sufficient time to document in ADT Navigators and its documentation was unfavorable due to several Yes-No questions not aligned with their practice. EHR systems considered the unique nursing workflow with tailoring according to the specific unit environment could help reduce nursing documentation burden. The finding also might be related to differences in the distribution of nursing tasks within a shift across units, patient acuity and patient-to-nurse staffing ratios as our nurses reported these as barriers contributing to their documentation burden.50,51
Poor usability often hinders clinician users from effectively and efficiently interacting with EHRs and is linked to clinician documentation burden.13,46 Nurses felt the usability-related issues resulted in creating duplicate documentation and unnecessary delays in their clinical workflows, compounding their documentation burden. Several strategies that nurses used to manage their documentation workloads in day-to-day practice included delegating some patient care tasks to technicians to free up time, charting-by-exception, copying-and-pasting, and saving less urgent documentation for the end or after their shift. Though most considered documentation to be important for capturing what was done for a patient, they indicated that documentation was, at times, excessive and often redundant (eg, the need to chart the same information in different places several times per day). Excess documentation demands interfered with direct patient contact time.
Overall, to decrease the nursing documentation burden and avoid adverse patient outcomes, it is crucial to develop effective strategies to improve the usability of each EHR component. This could be done by (1) minimizing redundant and/or excessive data entry requirements, using standard terminologies matched with real-world clinical practice, allowing nurses to have more flexibility (eg, adding “undo” function), more freedom (eg, make easy to switch screen to move from one screen to another), and more control (eg, adding an option for the one-time click of “delete” works for all relevant documentation “at once”), particularly in Flowsheets and Care Plan, and (2) offering visual cues to show what documentation has not been completed (eg, clear error messages; notifications of duplicates before creating a new one) and displaying patient information as prioritized and clinically relevant to the nurse on the screen (eg, clinical decision support tools). Several nurses also recommended that customizing the design of EHR components to individual nurses (not as a whole) would help reduce their documentation burden. Another proposed idea was to add a decision support search mechanism to each of the EHR components that would generate and present ranked evidence-based suggestions for nurses to quickly consider and select one that would automatically be added to the patient’s record (eg, contextual autocomplete feature). Our findings indicate that EHR’s usability across all components is likely to be improved with significant considerations of EHR interoperability and user-centered design that have the potential to reduce clinician burnout related to documentation burden.
The documentation burden among nurses was related to their perceived value of documenting in the EHR components to communicate patient care within the care team, consistent with prior studies among other clinicians (eg, physicians, nurse practitioner, physician assistant).52–54 In this study, interestingly, nurses had mixed reactions to the value of care plans, ranging from feeling that the Care Plan documentation facilitated the tracking of a patient’s goals throughout a hospital stay to feeling that it offered no value to patient care among non-nurse clinicians. For example, nurses believed that physicians did not view nursing care plan documents, which further demotivated some in completing them. Although the completion of care plan documentation was a regulatory requirement for patient care, a few nurses reported they did not often use the Care Plan component or could not trust care plans that their colleagues created because these care plans were characterized as having a lot of copied-pasted, inaccurate, or duplicated information. A key usability issue in Care Plan was the difficulty in searching for nursing terminologies such as nursing diagnoses and nursing interventions/outcomes classification and a lack of confidence with using the Care Plan component at the time of our study as many nurses reported having no training on how to create care plans beyond nursing school. Congruent with existing evidence,26,55–57 the EHR Care Plan component may not meet the main purpose of facilitating care plan documentation and promoting continuity of care and sound decision-making at the point of care although it is still mandated.58 Care plans are a vital part of nursing care and, when accurately documented and easily accessible, can support continuity in the delivery of care by the patient’s team and achievement of desired patient outcomes.59,60 Although care planning is a basic competency acquired by nurses in undergraduate training programs,59 the findings of our study suggest that offering continuous training at the organizational level on how to create care plans in day-to-day clinical practice (beyond nursing training programs) using technology-based tools (eg, software, videos) is needed to provide guidance in clinical decision-making for nurses. In this study, nurses hoped to get adequate training in Flowsheets, Care Plan, and ADT Navigators as commented “We need in-depth training on it, not like ‘This is your care plan, this is what you do with a shift, and that’s that’.” Given that these 3 EHR components were found to be below the acceptable usability score range, training may be crucial for systems with poor usability. Ideally, addressing usability issues should be preempted to boost benefits by offering adequate training to improve patient care outcomes and its meaningful use.61–65
Lastly, healthcare systems are complex with multiple conditions contributing to patient outcomes, which makes it challenging to detect a direct association between documentation burden and negative patient outcomes. Nonetheless, we explored nurses’ lived experiences with EHR documentation and found that their perceived documentation burden related to poor usability and lack of tailoring increased their perceived likelihood of negatively affecting patient outcomes.4–7 Further research to investigate the relationship between nurses’ documentation burden and adverse patient outcomes would benefit from excluding relevant confounding factors identified in this study (eg, usability issues, patient-to-nurse staffing ratios, suboptimal time management, and related stress). We specifically focused on nurse clinicians practicing in acute and critical care settings requiring astute clinical judgment to improve patient outcomes. This focus allowed us to uncover nuanced insights into how the intricacies of these settings impact nurses’ experiences with EHR documentation in recognition that documentation practices naturally differ across settings (eg, inpatient versus outpatient).
Limitations
Limitations to this study include the potentially limited generalizability of the results beyond inpatient units at a single medical center using a convenience sampling method. Results may differ in other groups, including nurses caring for different numbers of patients and hospitals using different EHR vendors. However, our focus on acute and critical care inpatient units that demand timely and accurate documentation for effective patient care provided practical insights for improving EHR documentation usability. Other limitations of our study were related to using EHR log files without direct observation and including the average documentation time as the sole indicator of documentation burden. There may have been measurement errors due to the inability of EHR logs to distinguish between idle and active time and the uncertainty related to after-hours work. Using other measures (eg, how timely the notes were finalized) may generate a different set of units. However, including Epic usage log file analyses to identify units with high likelihood of documentation burden and then recruit nurses from the specific units (rather than recruiting nurses from random units) for interviews and surveys, helped minimize possible variations related to inpatient care units and nurses in a hospital. Adding direct observation can yield more rich contextual data by comparing measures derived from logs with those from direct observation in future studies. Another limitation of our study is the possibility of response bias as with any research using survey responses, however, this was minimized through the synthesis of qualitative data sources from one-on-one interviews.
Conclusions
This research study examined nurses’ use of an EHR system in 1 university hospital inpatient setting of 31 units with a qualitative focus on 5 units with the heaviest documentation burden. From interviews and surveys with the 20 nurses from the 5 units, we found poor usability to be a major contributing factor to documentation burden. Our study underscores the crucial need for designing interventions that improve the EHR usability and that can easily be adjusted to accommodate the variable care conditions experienced by nurses in different units. As we learned from the nurses, the priorities for usability interventions are to reduce redundancy, improve workflow navigation, and streamline data entry. Further, these usability interventions must be nurse-endorsed and validated with representative populations of nurses to ensure the improved usability of EHR systems has the intended impact. Substantial opportunities thus remain to develop viable solutions for improving the usability of multi-component EHR systems with regards to transparent reporting on usability as required by the 21st Century Cures Act,66 and enhancing its usability and clinical workflow. Finally, as we noted, there are non-EHR factors that impact nurse documentation burden and these factors should also be considered in strategies intended to improve nurse working conditions to be harmonious with clinical workflow and coordinate efficiency in the work of all EHR stakeholders.
Supplementary Material
Acknowledgments
We wish to acknowledge our fabulous undergraduate students under the UF Active Learning Program Internship and Research Program, Jakyra McCloud and Lianny Propest who assisted with our interview preparation.
Contributor Information
Hwayoung Cho, College of Nursing, Department of Family, Community and Health System Science, University of Florida, Gainesville, FL 32610, United States.
Oliver T Nguyen, College of Engineering, Department of Industrial and Systems Engineering, University of Wisconsin at Madison, WI 53706, United States.
Michael Weaver, College of Nursing, Department of Family, Community and Health System Science, University of Florida, Gainesville, FL 32610, United States.
Jennifer Pruitt, College of Nursing, Department of Family, Community and Health System Science, University of Florida, Gainesville, FL 32610, United States; UF Health Shands Hospital, Gainesville, FL 32608, United States.
Cassie Marcelle, UF Health Shands Hospital, Gainesville, FL 32608, United States.
Ramzi G Salloum, College of Medicine, Department of Health Outcomes and Biomedical Informatics, University of Florida, Gainesville, FL 32610, United States.
Gail Keenan, College of Nursing, Department of Family, Community and Health System Science, University of Florida, Gainesville, FL 32610, United States.
Author contributions
Hwayoung Cho, Michael Weaver, Cassie Marcelle, and Gail Keenan were involved in the study concept and design. Hwayoung Cho, Oliver T. Nguyen, Michael Weaver, and Cassie Marcelle performed data acquisition and statistical analysis of the data entry log files, and Hwayoung Cho, Oliver T. Nguyen, and Jennifer Pruitt performed data acquisition and analysis of interview and survey data. Hwayoung Cho, Oliver T. Nguyen, Ramzi G. Salloum, and Gail Keenan performed data interpretation of both quantitative and qualitative data. All authors contributed equally to later manuscript drafts and agreed to the final submitted version of the manuscript. In this manuscript writing, generative AI has not been used.
Supplementary material
Supplementary material is available at Journal of the American Medical Informatics Association online.
Funding
This study was supported by the National Institute of Nursing Research, National Institutes of Health under award number R01 NR018416-01 (MPI: G.K./K.D.L./Y.Y.) and the University of Florida College of Nursing Internal Project Award (PI: H.C.). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Nursing Research, National Institutes of Health, or the University of Florida College of Nursing.
Conflicts of interest
The authors declare that they have no conflicts of interest in the research.
Data availability
The data sets generated and analyzed during this study are available from the corresponding author upon reasonable request. All de-identified data can be made available upon request of the authors.
References
- 1. Campanella P, Lovato E, Marone C, et al. The impact of electronic health records on healthcare quality: a systematic review and meta-analysis. Eur J Public Health. 2016;26(1):60-64. [DOI] [PubMed] [Google Scholar]
- 2. Wang SJ, Middleton B, Prosser LA, et al. A cost-benefit analysis of electronic medical records in primary care. Am J Med. 2003;114(5):397-403. [DOI] [PubMed] [Google Scholar]
- 3. Howley MJ, Chou EY, Hansen N, Dalrymple PW.. The long-term financial impact of electronic health record implementation. J Am Med Inform Assoc. 2015;22(2):443-452. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Dunn Lopez K, Chin CL, Leitão Azevedo RF, et al. Electronic health record usability and workload changes over time for provider and nursing staff following transition to new EHR. Appl Ergon. 2021;93:103359. [DOI] [PubMed] [Google Scholar]
- 5. Khairat S, Coleman C, Ottmar P, Jayachander DI, Bice T, Carson SS.. Association of electronic health record use with physician fatigue and efficiency. JAMA Netw Open. 2020;3(6):e207385. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Kadish SS, Mayer EL, Jackman DM, et al. Implementation to optimization: a tailored, data-driven approach to improve provider efficiency and confidence in use of the electronic medical record. J Oncol Pract. 2018;14(7):e421-e428. [DOI] [PubMed] [Google Scholar]
- 7. Moy AJ, Schwartz JM, Chen R, et al. Measurement of clinical documentation burden among physicians and nurses using electronic health records: a scoping review. J Am Med Inform Assoc. 2021;28(5):998-1008. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Melnick ER, Dyrbye LN, Sinsky CA, et al. The association between perceived electronic health record usability and professional burnout among US physicians. Mayo Clin Proc. 2020;95(3):476-487. [DOI] [PubMed] [Google Scholar]
- 9. Tajirian T, Stergiopoulos V, Strudwick G, et al. The influence of electronic health record use on physician burnout: cross-sectional survey. J Med Internet Res. 2020;22(7):e19274. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Kossman SP, Scheidenhelm SL.. Nurses’ perceptions of the impact of electronic health records on work and patient outcomes. Comput Inform Nurs. 2008;26(2):69-77. [DOI] [PubMed] [Google Scholar]
- 11. Hamine S, Gerth-Guyette E, Faulx D, Green BB, Ginsburg AS.. Impact of mHealth chronic disease management on treatment adherence and patient outcomes: a systematic review. J Med Internet Res. 2015;17(2):e52. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Incredible Health. Nursing in the Time of COVID-19: Stressors and Demand for Workers. Accessed March 29, 2024. https://www.incrediblehealth.com/blog/nursing-in-the-time-of-covid-19-report/
- 13. Gesner E, Dykes PC, Zhang L, Gazarian P.. Documentation burden in nursing and its role in clinician burnout syndrome. Appl Clin Inform. 2022;13(5):983-990. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Hendrich A, Chow MP, Skierczynski BA, Lu Z.. A 36-hospital time and motion study: how do medical-surgical nurses spend their time? Perm J. 2008;12(3):25-34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Lavander P, Meriläinen M, Turkki L.. Working time use and division of labour among nurses and health-care workers in hospitals—a systematic review. J Nurs Manag. 2016;24(8):1027-1040. [DOI] [PubMed] [Google Scholar]
- 16. Howe JL, Adams KT, Hettinger AZ, Ratwani RM.. Electronic health record usability issues and potential contribution to patient harm. JAMA. 2018;319(12):1276-1278. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Hettinger AZ, Melnick ER, Ratwani RM.. Advancing electronic health record vendor usability maturity: progress and next steps. J Am Med Inform Assoc. 2021;28(5):1029-1031. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Gomes KM, Ratwani RM.. Evaluating improvements and shortcomings in clinician satisfaction with electronic health record usability. JAMA Netw Open. 2019;2(12):e1916651. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Cho H, Keenan G, Madandola OO, et al. Assessing the usability of a clinical decision support system: heuristic evaluation. JMIR Hum Factors. 2022;9(2):e31758. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Yusof MM, Stergioulas L, Zugic J.. Health information systems adoption: findings from a systematic review. Stud Health Technol Inform. 2007;129(Pt 1):262-266. [PubMed] [Google Scholar]
- 21. Beauchemin M, Murray MT, Sung L, Hershman DL, Weng C, Schnall R.. Clinical decision support for therapeutic decision-making in cancer: a systematic review. Int J Med Inform. 2019;130:103940. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Topaz M, Ronquillo C, Peltonen L-M, et al. Nurse informaticians report low satisfaction and multi-level concerns with electronic health records: results from an International Survey. AMIA Annu Symp Proc. 2017. 2016;2016:2016-2025. [PMC free article] [PubMed] [Google Scholar]
- 23. Howard J, Clark EC, Friedman A, et al. Electronic health record impact on work burden in small, unaffiliated, community-based primary care practices. J Gen Intern Med. 2013;28(1):107-113. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Office of the National Coordinator for Health Information Technology. Strategy on Reducing Regulatory and Administrative Burden Relating to the Use of Health IT and EHRs. Accessed March 29, 2024. https://www.healthit.gov/topic/usability-and-provider-burden/strategy-reducing-burden-relating-use-health-it-and-ehrs
- 25. Apkon M, Singhaviranon P.. Impact of an electronic information system on physician workflow and data collection in the intensive care unit. Intensive Care Med. 2001;27(1):122-130. [DOI] [PubMed] [Google Scholar]
- 26. Agency for Healthcare Research and Quality. Challenges of Measuring Care Coordination Using Electronic Data and Recommendations to Address Those Challenges. Accessed March 29, 2024. https://www.ahrq.gov/research/findings/final-reports/prospectscare/prospects1a.html [Google Scholar]
- 27. Meigs SL, Solomon M.. Electronic health record use a bitter pill for many physicians. Perspect Health Inf Manag. 2016;13(Winter):1d. [PMC free article] [PubMed] [Google Scholar]
- 28. Sieck CJ, Pearl N, Bright TJ, Yen P-Y.. A qualitative study of physician perspectives on adaptation to electronic health records. BMC Med Inform Decis Mak. 2020;20(1):25-25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Moody LE, Slocumb E, Berg B, Jackson D.. Electronic health records documentation in nursing: nurses’ perceptions, attitudes, and preferences. Comput Inform Nurs. 2004;22(6):337-344. [DOI] [PubMed] [Google Scholar]
- 30. Collins S, Couture B, Kang MJ, et al. Quantifying and visualizing nursing flowsheet documentation burden in acute and critical care. AMIA Annu Symp Proc. 2018;2018:348-357. [PMC free article] [PubMed] [Google Scholar]
- 31. Bakhoum N, Gerhart C, Schremp E, et al. A time and motion analysis of nursing workload and electronic health record use in the emergency department. J Emerg Nurs. 2021;47(5):733-741. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Moy AJ, Schwartz JM, Elias J, et al. Time-motion examination of electronic health record utilization and clinician workflows indicate frequent task switching and documentation burden. AMIA Annu Symp Proc. 2020;2020:886-895. [PMC free article] [PubMed] [Google Scholar]
- 33. Peckham C. Medscape physician lifestyle report 2015. Medscape. 2015:2. Accessed July 29, 2016. http://www medscape com/features/slideshow/lifestyle/2015/public/overview
- 34. Creswell JW, Plano Clark VL. Choosing a mixed methods design. In: Designing and Conducting Mixed Methods Research. Vol 2. Thousand Oaks, CA: SAGE Publications; 2011:53-106.
- 35. Venkatesh V, Morris MG, Davis GB, Davis FD.. User acceptance of information technology: toward a unified view. MIS Quarterly. 2003;27(3):425-478. [Google Scholar]
- 36. Kossman SP, Bonney LA, Kim MJ.. Electronic health record tools’ support of nurses’ clinical judgment and team communication. Comput Inform Nurs. 2013;31(11):539-544. Quiz 545-536. [DOI] [PubMed] [Google Scholar]
- 37. Collins SA, Bakken S, Vawdrey DK, Coiera E, Currie L.. Clinician preferences for verbal communication compared to EHR documentation in the ICU. Appl Clin Inform. 2011;2(2):190-201. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Green J, Thorogood N.. Qualitative Methods for Health Research. Sage; 2018. [Google Scholar]
- 39. EpicShare. Accessed May 15, 2024. https://www.epicshare.org/tips-and-tricks/q-what-can-we-do-to-use-our-nursing-staff-most-efficientlyn_a-start-by-learning-where-they-spend-the-most-time-in-the-ehr-_#_ftn1
- 40. Rotenstein LS, Holmgren AJ, Downing NL, Longhurst CA, Bates DW.. Differences in clinician electronic health record use across adult and pediatric primary care specialties. JAMA Netw Open. 2021;4(7):e2116375. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41. Gill P, Stewart K, Treasure E, Chadwick B.. Methods of data collection in qualitative research: interviews and focus groups. Br Dent J. 2008;204(6):291-295. [DOI] [PubMed] [Google Scholar]
- 42. Kallio H, Pietilä AM, Johnson M, Kangasniemi M.. Systematic methodological review: developing a framework for a qualitative semi-structured interview guide. J Adv Nurs. 2016;72(12):2954-2965. [DOI] [PubMed] [Google Scholar]
- 43. Qualtrics, Provo, Utah, USA [computer program]. Accessed March 1, 2023. https://www.qualtrics.com/
- 44. Brooke J. SUS-A quick and dirty usability scale. In: Usability Evaluation in Industry. Vol 189(194). Boca Raton, FL: CRC Press; 1996:4-7.
- 45. Sauro J. A Practical Guide to the System Usability Scale: Background, Benchmarks & Best Practices. Measuring Usability LLC; 2011.
- 46. Melnick ER, West CP, Nath B, et al. The association between perceived electronic health record usability and professional burnout among US nurses. J Am Med Inform Assoc. 2021;28(8):1632-1641. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47. Swain J. A Hybrid Approach to Thematic Analysis in Qualitative Research: Using a Practical Example. SAGE Publications Ltd; 2018. [Google Scholar]
- 48. Kobayashi A. International Encyclopedia of Human Geography. Elsevier; 2019. [Google Scholar]
- 49. Kang MJ, Rossetti SC, Knaplund C, et al. Nursing documentation variation across different medical facilities within an integrated healthcare system. Comput Inform Nurs. 2021;39(12):845-850. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50. Kramer S, Raymond MJ, Hunter P, et al. Understanding the workflow of nurses in acute and subacute medical wards: a time and motion study. J Clin Nurs. 2023;32(21-22):7773-7782. [DOI] [PubMed] [Google Scholar]
- 51. Ivziku D, Ferramosca FMP, Filomeno L, Gualandi R, De Maria M, Tartaglini D.. Defining nursing workload predictors: a pilot study. J Nurs Manag. 2022;30(2):473-481. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52. Moy AJ, Hobensack M, Marshall K, et al. Understanding the perceived role of electronic health records and workflow fragmentation on clinician documentation burden in emergency departments. J Am Med Inform Assoc. 2023;30(5):797-808. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53. Hysong SJ, Spitzmuller C, Espadas D, Sittig DF, Singh H.. Electronic alerts and clinician turnover: the influence of user acceptance. Am J Manag Care. 2014;20(11 Spec No. 17):Sp520-530. [PubMed] [Google Scholar]
- 54. Livaudais M, Deng D, Frederick T, Grey-Theriot F, Kroth PJ.. Perceived value of the electronic health record and its association with physician burnout. Appl Clin Inform. 2022;13(4):778-784. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55. Urquhart C, Currell R, Grant MJ, Hardiker NR.. Nursing record systems: effects on nursing practice and health care outcomes. Cochrane Database Syst Rev. 2003;5(5):Cd002099. [DOI] [PubMed] [Google Scholar]
- 56. McDonald K, Schutz E, Chapman T, Davies S, Pineda N, Lonhart J. Prospects for Care Coordination Measurement Using Electronic Data Sources. Agency for Healthcare Research and Quality, 2012; 2015.
- 57. Polnaszek B, Gilmore-Bykovskyi A, Hovanes M, et al. Overcoming the challenges of unstructured data in multisite, electronic medical record-based abstraction. Med Care. 2016;54(10):e65-e72. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58. Joint Commission. 2021 Hospital Accreditation Standards. Joint Commission Resources; 2020. [Google Scholar]
- 59. American Association of Colleges of Nursing (AACN). The Essentials of Baccalaureate Education for Professional Nursing Practice. American Association of Colleges of Nursing; 2008. [DOI] [PubMed] [Google Scholar]
- 60.Nursing: Scope and Standards of Practice. Accessed March 29, 2024. Nursesbooks.org
- 61. Alami J, Hammonds C, Hensien E, et al. Usability challenges with electronic health records (EHRs) during prerounding on pediatric inpatients. JAMIA Open. 2022;5(1):ooac018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62. Mohan V, Woodcock D, McGrath K, et al. Using simulations to improve electronic health record use, clinician training and patient safety: recommendations from a consensus conference. AMIA Annu Symp Proc. 2016;2016:904-913. [PMC free article] [PubMed] [Google Scholar]
- 63. Faulkner L. Beyond the five-user assumption: benefits of increased sample sizes in usability testing. Behav Res Methods Instrum Comput. 2003;35(3):379-383. [DOI] [PubMed] [Google Scholar]
- 64. Russell P, Hewage U, Thompson C.. Method for improving the quality of discharge summaries written by a general medical team. Intern Med J. 2014;44(3):298-301. [DOI] [PubMed] [Google Scholar]
- 65. Keenan GM, Yakel E, Yao Y, et al. Maintaining a consistent big picture: meaningful use of a Web-based POC EHR system. Int J Nurs Knowl. 2012;23(3):119-133. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66. Public Law 114–255 S. To Accelerate the Discovery, Development, and Delivery of 21st Century Cures, and for Other Purposes. Enacted: December 13, 2016.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The data sets generated and analyzed during this study are available from the corresponding author upon reasonable request. All de-identified data can be made available upon request of the authors.