Abstract
Background
Use of an electronic health record may create unanticipated consequences for emergency care delivery. We sought to describe emergency department nursing task distribution and use of the electronic health record.
Methods
This was a prospective observational study of nurses in the emergency department using a time-and-motion methodology. Three trained research assistants conducted 1:1 observations between March and September 2019. Nurse tasks were classified into six established categories: electronic health record, direct/indirect patient care, communication, personal time, and “other”. Nurses’ perceived workload was assessed using the NASA Task Load Index (NASA-TLX).
Results
Twenty-three observations were conducted over 46 hours. Overall, nurses spent 27% of their time on electronic health record tasks, 25% on direct patient care, 17% on personal time, 15% on indirect patient care, and 6% on communication. During morning (7AM-12PM) and afternoon shifts (12PM-3PM), use of the health record was the most commonly performed task, whereas indirect patient care was the task most performed during evening shifts (3PM-12PM). Using the NASA-TLX, nurses reported an increase in mental demand and effort during afternoon shifts compared with morning shifts.
Conclusion
We observed that emergency department nurses spent more time using the electronic health record as compared to other tasks. Increased usability of the electronic health record, particularly during high occupancy periods, may be a target for improvement.
Keywords: Nursing, workload, burnout, electronic health record, time-motion study, emergency department, emergency medicine, operations
INTRODUCTION
Since 2003, there has been a push to implement electronic health records (EHR) throughout healthcare as part of a national task force deployed by the Institute of Medicine and Health.1 This was further expanded upon by the signing of the Health Information Technology for Economic and Clinical Health Act (HITECH) in 2009, which sought to promote implementation of EHRs in hospital systems.2 The adoption of commercialized EHRs across healthcare has helped facilitate complete patient records, improve shift-to-shift reporting, limit data entry errors, and enable more efficient delivery of healthcare.3 Nevertheless, this expansion has not been flawless, with drawbacks of inefficient implementation including poor usability, low participant satisfaction, and a demise in time dedicated towards patient care.4,5
The emergency department (ED), with its high patient acuity and dynamic workflow consisting of frequent interruptions, can be limited by an inflexible EHR that decreases productivity and hinders multi-tasking.4 Healthcare providers are at a significantly higher risk of professional burnout when dissatisfied with the time spent on clerical tasks and decreasing time spent interfacing with patients.5,6 Increased workload and resulting burnout amongst healthcare providers is detrimental to the quality of care delivery and clinical decision-making. For example, in intensive care unit (ICU) settings, increased workload in the form of higher patient-to-nursing ratios is associated with increased morbidity and mortality.7,8 Beyond the walls of the ICU, increased workload in the ED was found to influence physician prescribing patterns, with physicians more likely to prescribe opioids when patient volumes are higher.9
There is evidence that the use and alternatively, the usability of an EHR impacts meaningful patient-oriented outcomes. One study of neonatal intensive care nurses found that one of the most frequent unintended consequences of EHR use was a heavier workload along with changes to their workflow and modified communication patterns.10 Nurses spend less time reporting and providing patient-family teaching in lieu of their increasing documentation requirements.11 Cumbersome navigation of scattered information throughout the EHR slows cognitive processing of patient information,12 This can adversely impact a nurse’s ability to make rapid, real-time medical decisions and perform hand-offs effectively and efficiently.13,14
Across different EHR vendors, time to complete tasks and error rates can vary widely, leading to errors in medication and diagnostic orders.15,16 Additionally, key performance indicators such as waiting room time, treatment time, and total time for patients discharged from the ED, were increased immediately following the implementation of a commercialized EHR, when staff and healthcare providers are first learning how to use a new system. 17,18
Prior research amongst medical-surgical and outpatient nurses found that a substantial proportion of a nurse’s tasks may be devoted to the EHR.19,20 Given the importance of context and the highly disruptive nature of the ED, EHR use amongst emergency nurses has not, to our knowledge, been described previously and is the subject of this analysis. A previously unpublished study from 2007 from our institution examined nursing work patterns in the ED setting and found that nurses spent 10% of their time on computer tasks and 32% on direct patient care.21
In this study, our objective was to (1) describe overall patterns of nursing workload in the ED, (2) measure the task load dedicated towards use of the EHR, and (3) understand variation in ED nurse workload across different times of the day. Additionally, we sought to characterize any changes that may have occurred in time dedicated to the EHR since this was last studied at our institution 13 years ago.
METHODS
Design and Setting
We conducted a prospective observational study of ED nurse use of the EHR using a continuous time-and-motion methodology.22 Adapted from management science, a time-and-motion approach uses observation to measure the amount of time spent on tasks. The study was conducted at Vanderbilt University Medical Center (VUMC) in Nashville, TN. VUMC is a quaternary care academic medical center and its ED has 80,000 annual patient visits. The ED has used Epic (Epic Systems Corporation, Verona, WI), a commonly available commercial EHR, since November 2017. Observations were performed in the highest acuity section of the ED to be consistent with our initial 2007 study. This section of the ED consists of 20 treatment rooms and four trauma resuscitation bays with nursing to patient ratios of 1:3.
Observation procedures
Three trained research assistants conducted 1:1 observations using a convenience sample of emergency nurses who consented to be observed between March and September 2019. The research assistants were comprised of two medical students and a clinical research coordinator. Prior to conducting formal observations and to enhance reliability, research assistants were trained through simultaneous observations of a single nurse. Research assistants were given a guide defining workload categories and were observed by a senior researcher during a pilot observation training period to enhance the accuracy and reliability of the time-motion data collection. Observed nurses were typically assigned to 2–4 rooms per shift. Observations were conducted across different times in the day to capture potential variation in ED workload and flow, with a morning (8AM-12PM), afternoon (12PM-3PM), and evening (3PM-6PM) shift. Splitting observations is a standard methodology for time-motion studies conducted in the ED, where continuous observations can be disruptive and expensive to perform.23,24 Observers were instructed to limit conversations with nurses and to position themselves in a way to avoid interruptions in nurse workflow.
Participants
Staff were included if they were an emergency nurse and were assigned to the high acuity pod during the period of observation. Nurses were excluded if they were assigned to a teaching role as workflow patterns may deviate from their normal practice. Eligible nurses were selected based on availability through review of the electronic whiteboard or by recommendation from the charge nurse. Nurses were consented verbally prior to data collection. Nurses were aware their workload was being observed but were blinded to the purpose of the study. Charge nurses were not aware of the study purpose or aims.
Data collection and measures
Time-and-motion data were collected using REDCap (Research Electronic Data Capture). REDCap is a secure, widely used web-based software platform designed to support data capture for research studies.25 Research assistants used the tool to log, in real-time, what activity a nurse was performing. Timestamps and durations of tasks was automatically generated by the system. The data collection form can be seen in the Supplement.
Nursing activities were classified into one of six categories: EHR, direct patient care, indirect patient care (actions performed to benefit patient but do not involve direct patient contact), communication (discussions with other health-care providers), personal time (non-health care related tasks), and other (any other task not included above). These definitions were derived from literature review of prior time-motion studies and reviewed by a nursing manager for relevancy to our setting prior to their implementation.26,27 The distinction between direct and indirect patient care is well-described by the Nursing Interventions Classification System, a comprehensive, research-based standardized classification of nursing roles.28
Nursing staff’s perceived workload was assessed using the National Aeronautics and Space Administration Task Load Index (NASA-TLX), a validated tool for measuring and conducting a subjective mental workload assessment.29 The NASA-TLX is a multi-dimensional and widely accepted tool for measuring subjective occupant workload.30 The NASA-TLX is also a quick, easy, and flexible survey that limits interruptions to nursing workflow and has been effectively used in prior nursing studies to measure overall subjective workload of a shift and multiple tasks.31, 32
Towards the end of an observation session, nurses were given an electronic tablet with the NASA-TLX survey and asked to rate performance on a scale of 1–20 across six dimensions: mental demand, physical demand, temporal demand, effort, performance and frustration. To help with understanding our findings, we asked the observed nurses to comment on the perceived workload and whether there were any barriers to accomplishing their work during the shift. These comments were reviewed for themes and are presented in the Discussion.
Analysis
Statistical analysis was conducted using Stata/IC version 15.0 (College Station, TX). A Wilcoxon Rank Sum test was used to analyze time spent on EHR tasks when compared with other tasks. We used one-way ANOVA analysis to look at differences in NASA-TLX scores by time of day. Any missing NASA-TLX data were excluded from statistical analysis.
Ethical Considerations
This study was approved by the VUMC Institutional Review Board (IRB #181533). This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines for reporting observational studies.33,34 Observations were conducted on a convenience sample of nurses working in a research-centric ED in which participation in research is the norm rather than the exception. Prior to observation, study investigators collaborated with nursing leadership to develop a written script that was reviewed and approved by the institutional IRB and electronically sent to all potential staff explaining the purpose and voluntary nature of the study. They were also informed that they could opt out at any time. Staff were then contacted one day in advance and verbally consented. Although we had a 100% participation rate, recruitment focused on ensuring that participation was entirely voluntary. Further, nurses were blinded from specific study objectives. There were no incentives offered to participate in the study and no penalties to declining participation. However, we note that nurses often feel under-represented in research and are eager to participate to educate patients of the research products about the nature and challenges of emergency nursing. We did not record names of nurses observed or demographic information to protect participant privacy.
RESULTS
All recruited nurses enrolled in the study. Twenty-three observations were conducted totaling 46 hours between March and September 2019. Mean duration of observation per nurse was 120 minutes with a range of 56 to 169 minutes. In total, there were 9 morning observations, 9 afternoon observations, and 5 evening observations. A NASA-TLX survey was not completed in 3 out of the 27 observations due to participants declining or time constraints.
Tasks and time
Overall, nurses spent a median of 27% (Interquartile range [IQR] 23–33) of their time on EHR tasks, 25% (IQR 16–32) on direct patient care, 17% (IQR 6–24) on personal time, 15% (IQR 12–25) on indirect patient care, 6% (IQR 4–8) on communication, and 0% (IQR 0–2) on “other” tasks including cleaning patient rooms or documenting on paper. The median time spent on EHR tasks was greater than on indirect patient care (11.8 minutes, p =0.003; 95% CI 4.1, 20.5), communication (24.9 minutes, p < 0.001, 95% CI 18.4, 32.5), and personal time (15.1 minutes, p =0.005, 95% CI 4.8, 24.3) (Table 1). There was no significant difference from direct patient care. As a proportion of tasks performed, the use of the EHR was the most frequent task performed overall (31%), followed by indirect patient care (23%), direct patient care (21%), communication (13%), and personal time (11%). During morning shifts, the highest median fraction of time was spent on EHR tasks and direct patient care. During afternoon shifts, nurses spent most of their time using the EHR, whereas indirect patient care was the task most performed during evening shifts.
Table 1.
Median Time | IQR | |
---|---|---|
EHR | 27% | 23–33 |
Direct Patient Care | 25% | 16–32 |
Indirect Patient Care | 15% | 12–25 |
Communication | 6% | 4–8 |
Personal Time | 17% | 6–24 |
NASA-TLX Results
Overall, mean NASA-TLX scores (N=20) indicated that throughout a shift, nurses had low levels of demand and frustration and felt they could complete tasks effectively (Figure 1). However, when stratified by time of day, key differences were observed (Figure 2). Nurses had increased mental demand and reported effort (how hard they had to work to accomplish tasks) between morning, afternoon, and evening shifts (p = 0.02, F=5.04, p = 0.04, F=4.12). There were no statistical differences noted between physical demand, temporal demand, performance, and frustration across the three time points.
DISCUSSION
This study used direct observations of nurses working in the ED of a large quaternary care academic center to evaluate patterns of workload distribution. Our results identified that emergency nurses spent more time using the EHR than on direct or indirect patient care tasks (27%, 25%, 15%, respectively). Additionally, use of the EHR was the most frequent task performed by nurses during their shifts.
Our results demonstrated a substantial change in the amount of time spent on EHR tasks when compared with prior analyses of nurse workflow. In a 2010 study by Cornell et al, charting tasks took up 9.9% of medical-surgical nurse time, with the most time spent assessing patients.35 Our results are comparable to literature studying physician workflow. Research on emergency physician workflow in Denmark yielded similar numbers to ours, with physicians spending 25% of their time on direct patient care and 31% spent on documentation.36 Similar time-motion analysis of inpatient hospitalists done by Tipping et al. showed that physicians spent more time using the EHR than on direct patient contact (25% and 17%, respectively).37 Increasing medico-legal liabilities and billing requirements have necessitated that physicians spend more time documenting in the EHR. Nurses, however, do not have the same requirements and in prior research, have reported more positive attitudes towards the adoption of EHRs in clinical practice when compared to physicians.38 Despite this, our study reveals that nurses spend comparable amounts of time utilizing the EHR as physicians.
Using the NASA-TLX, nurses in our study reported increased mental, physical and temporal demands, increased effort and frustration, and decreased performance during afternoon shifts. During evening shifts, there was a sharp decrease in personal time tasks and time spent utilizing the EHR, with an increase in the duration of time performing indirect patient care. It appears that during higher occupancy times, emergency nurses spend more time on patient care, leaving them less time to document or take breaks. Qualitative comments, particularly around usability of the EHR, reflected on this cognitive toll associated with documenting during periods of increased patient load. Usability features that were a source of frustration included the multiple clicks required to collect information on patients, making nursing sign-out more tedious, as well as increased documentation requirements for patients waiting to be admitted (e.g., boarded patients). One nurse commented on how the portability of computers in the ED facilitated flow more so on the inpatient side, allowing for concurrent patient care to occur when documenting. The EHR, as one nurse commented, “helps more than it hurts, but can be challenging as someone getting to know the system.”
Prior research has shown that EHR-related stress was associated with nursing’ perceived physical demand and frustration.39 Early phases of EHR transitions are associated with increased cognitive workload in nurses, but no long-term follow-up has been done. According to Black Book market research, self-reported satisfaction with EHRs amongst nurses is low, with 90% of nurses believing that their EHR has damaged their ability to communicate with patients.40
Although not designed as an exact-replication study, our data collection methods were similar to the 2007 Vanderbilt study examining emergency nursing use of an EHR and provide insight into how nursing use of the EHR may have changed over time at a single institution. In the study, 96.7 hours of data were evenly split between 8 morning and 8 evening shifts from emergency nurses working in the highest acuity pod. The observations at that time found that emergency nurses spent approximately 10% of their time using the EHR and 32% on direct patient care. The EHR used at the time was a homegrown system compared with the commercially available product implemented in 2017. While we focused on a similar population of emergency nurses working with high acuity patients, potential confounders remain. For example, changes in team composition at the time of patient arrival both in the current study and in 2007, architectural design of the ED, nursing experience with the EHR, patient time spent in the ED (e.g., newly arrived vs. boarding patient), and historical changes in work processes are such examples that may introduce potential confounders which limit both a comparison between the two studies and further impact the results of the current study. Notably, our observations show an approximately three-fold increase in the proportion of time dedicated to the EHR since 2007. However, these findings may be limited in generalizability due to the potential confounders described above along with the specific operating nature of this quaternary care academic ED.
Further research is needed to understand why there has been such a substantial increase in time spent in the EHR. One issue frequently described in the literature and relevant to the ED setting is the increase of “alert” or “pop-up” fatigue resulting from frequent interruptions built into clinical systems.41 In a setting like the ED, where staff members are already burdened by frequent interruptions, additional interruptions created by an EHR may hinder interactions among ED staff and patients. To combat this, one study observed what unique alerts accounted for the majority of interruptions in nursing workflow and made changes accordingly, with significant reductions in weekly screen time.42 Additional usability features to consider for the ED include the forgiveness and feedback of the system, whereby exploration within the EHR does not lead to errors and informs users what actions are about to be undertaken, as well as the effectiveness of language and simplicity of the design interface.43 Nurses and physicians have historically felt excluded from participating in health system development initiatives and often cite the feeling that EHRs are designed to prioritize documentation and billing over patient care coordination and decision-making.44,45 Therefore, any future endeavors to address usability issues must have buy-in from all end-users of health informatics systems.
Limitations
Our study has several limitations. First, the study was limited to nurses working in the highest acuity pod in the ED. In these pods, nursing to patient ratios are lower (1:3 patients), which may alter how workflow is distributed when compared to other areas of the ED. Second, observer inter-rater reliability was not formally measured nor was a priori sample size calculated as this study was designed to be descriptive in nature. Further, training was conducted prior to observations in order to enhance reliability of the observations. Third, we used a survey instrument developed in REDCap for capturing time-motion analysis which has not been validated. However, there were no reported issues with using this instrument during observations. Other limitations include the small sample of emergency nurses observed as well as constraints on duration of the study and time of day when observations were conducted (there were no observations conducted after 6pm). Nurses in our study were not blinded to observers and may have subsequently modified their behavior or the content of their qualitative comments. Our study lacked randomization which may have introduced selection bias into our results. Finally, while possible that the same nurses were observed, subsequently reducing the generalizability, this was not part of our exclusion criteria and occurred infrequently.
IMPLICATIONS FOR EMERGENCY CLINICAL CARE
Our study demonstrated that more emergency nursing time was spent in the EHR than on direct or indirect patient care tasks. EHR documentation burden, usability, nurse satisfaction, are important areas for process improvement and innovation. Our study can serve as a single-site model assessment of the need for performance improvement to reduce EHR related job demands and frustrations for the emergency nursing workforce.
CONCLUSIONS
In this single center study, our findings demonstrated that time using the EHR was the most frequent task performed by emergency nurses. Further, our study provides some insight into the impact that health information technology has on cognitive demands, frustration, and nursing satisfaction in the ED. Identifying the etiology of this increased workload may identify ways to reduce time spent on EHR tasks and subsequently increase the amount of time available for patient care.
Supplementary Material
Table 2.
Difference (Minutes) | 95% Confidence Interval | Z-value | P Value | |
---|---|---|---|---|
EHR vs. Indirect Patient Care | 11.8 | 4.1, 20.5 | −2.4 | 0.003 |
EHR vs. Direct Patient Care | 4.2 | −7.5, 13.3 | −0.7 | 0.49 |
EHR vs. Communication | 24.9 | 18.4, 32.5 | −4.2 | < 0.001 |
EHR vs. Personal Time | 15.1 | 4.8, 24.3 | −2.9 | 0.005 |
CONTRIBUTION TO EMERGENCY NURSING PRACTICE.
The current state of the literature indicates that the implementation of commercialized electronic health records (EHR) has led to both advances and drawbacks to patient care. However, there is little in the literature examining the impact EHR usability has had on nursing workload and satisfaction in the emergency care setting.
The main finding of this research is that nurses spent more time utilizing the EHR as compared to other tasks, including direct and indirect patient care.
Key implications for emergency nursing practice from this research are identifying and addressing the specific usability issues within EHR systems that may hinder nursing workflow.
Funding:
The project described was also supported by the National Center for Research Resources, Grant UL1 RR024975-01, and is now at the National Center for Advancing Translational Sciences, Grant 2 UL1 TR000445-06. The content is solely the responsibility of the authors and does not necessarily represent the official views of the funding sources.
Footnotes
Conflict of Interest: None
CRediT author statement: NB, CG, ADJ, SA, DF, MJW: Conceptualization, Methodology; NB, CG, ES: Data curation; NB, CG, ADJ, MJW: Writing- Original draft preparation; SA, DF, MJW: Supervision; ADJ: Validation; All authors: Writing-Reviewing and Editing and approval of the final manuscript. All authors met ICMJE recommendations for authorship criteria.
Meeting Information: Mid-Atlantic Society for Academic Emergency Medicine Conference, Washington, DC, March 4th 2020.
SUPPLEMENT.
REDCap data collection instrument for workflow and perceptions of workload.
REFERENCES
- 1.Burton LC, Anderson GF, Kues IW. Using electronic health records to help coordinate care. Milbank Q. 2004;82(3):457–481, table of contents. doi: 10.1111/j.0887-378X.2004.00318.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Lite S, Gordon WJ, Stern AD. Association of the Meaningful Use Electronic Health Record Incentive Program With Health Information Technology Venture Capital Funding. JAMA Netw Open. 2020;3(3):e201402. doi: 10.1001/jamanetworkopen.2020.1402 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Menke JA, Broner CW, Campbell DY, McKissick MY, Edwards-Beckett JA. Computerized clinical documentation system in the pediatric intensive care unit. BMC Med Inform Decis Mak. 2001;1:3. doi: 10.1186/1472-6947-1-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Park SY, Lee SY, Chen Y. The effects of EMR deployment on doctors’ work practices: a qualitative study in the emergency department of a teaching hospital. Int J Med Inform. 2012;81(3):204–217. doi: 10.1016/j.ijmedinf.2011.12.001. [DOI] [PubMed] [Google Scholar]
- 5.Shanafelt TD, Dyrbye LN, Sinsky C, et al. Relationship Between Clerical Burden and Characteristics of the Electronic Environment With Physician Burnout and Professional Satisfaction. Mayo Clin Proc. 2016;91(7):836–848. doi: 10.1016/j.mayocp.2016.05.007. [DOI] [PubMed] [Google Scholar]
- 6.Wedding MG. Understanding the Association between EHRs and Nursing Burnout. 2019. [Google Scholar]
- 7.Lee A, Cheung YSL, Joynt GM, Leung CCH, Wong WT, Gomersall CD. Are high nurse workload/staffing ratios associated with decreased survival in critically ill patients? A cohort study. Ann Intensive Care. 2017;7(1):46. doi: 10.1186/s13613-017-0269-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Neuraz A, Guerin C, Payet C, et al. Patient Mortality Is Associated With Staff Resources and Workload in the ICU: A Multicenter Observational Study. Crit Care Med. 2015;43(8):1587–1594. doi: 10.1097/CCM.0000000000001015. [DOI] [PubMed] [Google Scholar]
- 9.Ward MJ, Kc D, Jenkins CA, Liu D, Padaki A, Pines JM. Emergency department provider and facility variation in opioid prescriptions for discharged patients. Am J Emerg Med. 2019;37(5):851–858. doi: 10.1016/j.ajem.2018.07.054. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Dudding Katherine M. BSN, RNC-NIC; Gephart Sheila M. PhD, RN; Carrington Jane M. PhD, RN Neonatal Nurses Experience Unintended Consequences and Risks to Patient Safety With Electronic Health Records, CIN: Computers, Informatics, Nursing: April 2018. - Volume 36 - Issue 4 - p 167–176 doi: 10.1097/CIN.0000000000000406 [DOI] [PubMed] [Google Scholar]
- 11.Schenk E, Schleyer R, Jones CR, Fincham S, Daratha KB, Monsen KA. Impact of Adoption of a Comprehensive Electronic Health Record on Nursing Work and Caring Efficacy. Comput Inform Nurs. 2018. Jul;36(7):331–339. doi: 10.1097/CIN.0000000000000441 [DOI] [PubMed] [Google Scholar]
- 12.Wisner Kirsten, Lyndon Audrey, Chesla Catherine A., The electronic health record’s impact on nurses’ cognitive work: An integrative review, International Journal of Nursing Studies, Volume 94, 2019, Pages 74–84, ISSN 0020–7489, 10.1016/j.ijnurstu.2019.03.003. [DOI] [PubMed] [Google Scholar]
- 13.Chao CA. The impact of electronic health records on collaborative work routines: A narrative network analysis. Int J Med Inform. 2016. Oct;94:100–11. doi: 10.1016/j.ijmedinf.2016.06.019. [DOI] [PubMed] [Google Scholar]
- 14.Staggers N, Clark L, Blaz JW, Kapsandoy S. Why patient summaries in electronic health records do not provide the cognitive support necessary for nurses’ handoffs on medical and surgical units: insights from interviews and observations. Health Informatics J. 2011. Sep;17(3):209–23. doi: 10.1177/1460458211405809. [DOI] [PubMed] [Google Scholar]
- 15.Cho I, Kim E, Choi WH, Staggers N. Comparing usability testing outcomes and functions of six electronic nursing record systems. Int J Med Inform. 2016;88:78–85. doi: 10.1016/j.ijmedinf.2016.01.007. [DOI] [PubMed] [Google Scholar]
- 16.Ratwani RM, Savage E, Will A, et al. A usability and safety analysis of electronic health records: a multi-center study. J Am Med Inform Assoc. 2018;25(9):1197–1201. , doi: 10.1093/jamia/ocy088. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Mohan MK, Bishop RO, Mallows JL. Effect of an electronic medical record information system on emergency department performance. Med J Aust. 2013;198(4):201–204. doi: 10.5694/mja12.10499. [DOI] [PubMed] [Google Scholar]
- 18.Ward MJ, Froehle CM, Hart KW, Collins SP, Lindsell CJ. Transient and sustained changes in operational performance, patient evaluation, and medication administration during electronic health record implementation in the emergency department. Ann Emerg Med. 2014;63(3):320–328. doi: 10.1016/j.annemergmed.2013.08.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Yen PY, Kellye M, Lopetegui M, et al. Nurses’ Time Allocation and Multitasking of Nursing Activities: A Time Motion Study. AMIA Annu Symp Proc. 2018;2018:1137–1146. Published 2018 Dec 5. [PMC free article] [PubMed] [Google Scholar]
- 20.Sun C, Cato K. How much time do nurses spend using electronic devices at work? Nurs Manage. 2020. Mar;51(3):22–29. doi: 10.1097/01.NUMA.0000651184.19361.4e [DOI] [PubMed] [Google Scholar]
- 21.Dale C, Dezube R, Levin S, France D, Weinger M. Impact of Emergency Department Occupancy and Patient Boarding on Registered Nurse Work Patterns and Subjective Ratings of Workload and Quality: 143. Ann Emerg Med. 2007;50(3). [Google Scholar]
- 22.Lopetegui M, Yen PY, Lai A, Jeffries J, Embi P, Payne P. Time motion studies in healthcare: what are we talking about?. J Biomed Inform. 2014;49:292–299. doi: 10.1016/j.jbi.2014.02.017 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.France Daniel J., Levin Scott, Hemphill Robin, Chen Kong, Rickard Dorsey, Makowski Renee, Jones Ian, Aronsky Dominik, Emergency physicians’ behaviors and workload in the presence of an electronic whiteboard, International Journal of Medical Informatics, Volume 74, Issue 10, 2005, Pages 827–837, ISSN 1386–5056, 10.1016/j.ijmedinf.2005.03.015. [DOI] [PubMed] [Google Scholar]
- 24.Coiera EW, Jayasuriya RA, Hardy J, Bannan A, Thorpe ME. Communication loads on clinical staff in the emergency department. Med J Aust. 2002. May 6;176(9):415–8. [DOI] [PubMed] [Google Scholar]
- 25.Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–381. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Westbrook Johanna I., Ampt Amanda, Design, application and testing of the Work Observation Method by Activity Timing (WOMBAT) to measure clinicians’ patterns of work and communication, International Journal of Medical Informatics, Volume 78, Supplement 1, 2009, Pages S25–S33, ISSN 1386–5056, 10.1016/j.ijmedinf.2008.09.003. [DOI] [PubMed] [Google Scholar]
- 27.Han B, Li Q, Chen X, Zhao G. Workflow for Intensive Care Unit Nurses: A Time and Motion Study. SAGE Open. July 2020. doi: 10.1177/2158244020947433 [DOI] [Google Scholar]
- 28.Butcher HK, Bulechek GM, Dochterman JMM, Wagner CM. Nursing Interventions classification (NIC)-E-Book. Elsevier Health Sciences; 2018. [Google Scholar]
- 29.Xiao YM, Wang ZM, Wang MZ, Lan YJ. [The appraisal of reliability and validity of subjective workload assessment technique and NASA-task load index]. Zhonghua Lao Dong Wei Sheng Zhi Ye Bing Za Zhi. 2005. Jun;23(3):178–81. Chinese. [PubMed] [Google Scholar]
- 30.Hart SG. Nasa-Task Load Index (NASA-TLX); 20 Years Later. Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 2006;50(9):904–908. doi: 10.1177/154193120605000909 [DOI] [Google Scholar]
- 31.Hoonakker P, Carayon P, Gurses A, et al. MEASURING WORKLOAD OF ICU NURSES WITH A QUESTIONNAIRE SURVEY: THE NASA TASK LOAD INDEX (TLX). IIE Trans Healthc Syst Eng. 2011;1(2):131–143. doi: 10.1080/19488300.2011.609524 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Young G, Zavelina L, Hooper V. Assessment of workload using NASA Task Load Index in perianesthesia nursing. J Perianesth Nurs. 2008. Apr;23(2):102–10. doi: 10.1016/j.jopan.2008.01.008. [DOI] [PubMed] [Google Scholar]
- 33.von Elm E, Altman DG, Egger M, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 2007;370(9596):1453–1457. [DOI] [PubMed] [Google Scholar]
- 34.von Elm E, Altman DG, Egger M, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: guidelines for reporting observational studies. International journal of surgery. 2014;12(12):1495–1499. [DOI] [PubMed] [Google Scholar]
- 35.Cornell P, Herrin-Griffith D, Keim C, et al. Transforming nursing workflow, part 1: the chaotic nature of nurse activities. J Nurs Adm. 2010;40(9):366–373, doi: 10.1097/NNA.0b013e3181ee4261. [DOI] [PubMed] [Google Scholar]
- 36.Füchtbauer LM, Nørgaard B, Mogensen CB. Emergency department physicians spend only 25% of their working time on direct patient care. Dan Med J. 2013. Jan;60(1):A4558. [PubMed] [Google Scholar]
- 37.Tipping MD, Forth VE, O’Leary KJ, et al. Where did the day go?—A time-motion study of hospitalists. J Hosp Med. 2010;5(6):323–328, doi: 10.1002/jhm.790. [DOI] [PubMed] [Google Scholar]
- 38.Darr A, Harrison MI, Shakked L, Shalom N. Physicians’ and nurses’ reactions to electronic medical records. Managerial and occupational implications. J Health Organ Manag. 2003;17(5):349–359, doi: 10.1108/14777260310505129 [DOI] [PubMed] [Google Scholar]
- 39.Colligan L, Potts HW, Finn CT, Sinkin RA. Cognitive workload changes for nurses transitioning from a legacy system with paper documentation to a commercial electronic health record. Int J Med Inform. 2015;84(7):469–476, doi: 10.1016/j.ijmedinf.2015.03.003.. [DOI] [PubMed] [Google Scholar]
- 40.Miliard M Nurses not happy with hospital EHRs. Healthcare IT News. 2014. [Google Scholar]
- 41.Baysari MT, Zheng WY, Van Dort B, Reid-Anderson H, Gronski M, Kenny E. A Late Attempt to Involve End Users in the Design of Medication-Related Alerts: Survey Study. J Med Internet Res. 2020;22(3):e14855, doi: 10.2196/14855. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Chaparro JD, Hussain C, Lee JA, Hehmeyer J, Nguyen M, Hoffman J. Reducing Interruptive Alert Burden Using Quality Improvement Methodology. Appl Clin Inform. 2020;11(1):46–58, doi: 10.1055/s-0039-3402757. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Belden JL, Grayson R, Barnes J. Defining and testing EMR usability: Principles and proposed methods of EMR usability evaluation and rating. Healthcare Information and Management Systems Society (HIMSS);2009. [Google Scholar]
- 44.Martikainen S, Kaipio J, Laaveri T. End-user participation in health information systems (HIS) development: Physicians’ and nurses’ experiences. Int J Med Inform. 2020;137, doi: 10.1016/j.ijmedinf.2020.104117. [DOI] [PubMed] [Google Scholar]
- 45.O’Malley AS, Grossman JM, Cohen GR, Kemper NM, Pham HH. Are electronic medical records helpful for care coordination? Experiences of physician practices. J Gen Intern Med. 2010;25(3):177–185, doi: 10.1007/s11606-009-1195-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.