Skip to main content
NCBI home page
AMIA Summits on Translational Science Proceedings logoLink to AMIA Summits on Translational Science Proceedings
. 2021 May 17;2021:634–643.

Impact of Data Entry Interface Design on Cognitive Workload, Documentation Correctness, and Documentation Efficiency

Bryan A Wilbanks 1, Jacqueline A Moss 1
PMCID: PMC8378654  PMID: 34457179

Abstract

Clinical documentation serves as the legal record of patient care and used to guide clinical decision making. Inadequately designed data entry user-interfaces may result in unintended consequences that negatively impact patient safety and outcomes because inaccurate information is used to guide clinical decision making. This study utilized an electronic simulated documentation interface (i.e., artificial electronic health record) combined with eye-tracking hardware to analyze documentation correctness, documentation efficiency, and cognitive workload of anesthesia providers (N = 20) generating documentation using different computer-assisted data entry types (drop-down box, radio button, check-box, and free text with autocomplete suggestions). Our study methodology incorporating eye-tracking with electronic health record user interfaces to assess documentation correctness, efficiency, and cognitive workload can be translated to other health care provider types.

Introduction

Inadequate data entry user-interface design impairs documentation correctness, efficiency, and cognitive workload associated with using electronic health records (EHR)1,2. Poor user-interface design may also result in unintended consequences that impair patient safety and outcomes3,4 because incorrect information is used to guide future clinical decision making7. For example, user-interfaces that are percieved as complicated by the user may result in documenting patient care information into the wrong area of the EHR which result in clinicians overlooking important information because it is not located where they expect it9. The lack of understanding in how to optimize data entry interface design has been identified as a contributing factor to unintended consequences that result in impaired patient outcomes3,8,9. We were unable to locate literature that identified user-interface design issues that were EHR vendor specific. One study that evaluated 11 EHR vendors in aggregate found wide variability in how vendors assessed interface design issues prior to implementation, and that many user-interface customizations during implementation were based primarily on purchaser requests without assessing impact on usability or patient safety10. There was relatively little evidence in the literature to guide the selection of specific data entry methods according to the type of data documented.

In order to fill this knowledge gap, this study was designed to evaluate how pairing specific computer-assisted data entry types (e.g., drop-down box, radio button, check-box, and free text with autocomplete suggestions) to specific anesthesia documentation data elements influenced documentation correctness, documentation efficiency, and cognitive workload of anesthesia providers. Understanding the impact of pairing specific types of data with specific data entry methods can improve patient outcomes by enhancing documentation correctness through improved documentation efficiency and reduced cognitive workload. Our study is EHR vendor agnostic because we are assessing the user-interface at a granular level (data entry for a single datum) and the results could be translated to various EHR vendors because the information to be documented are similar. This approach also has the advantage of focusing on each specific data entry method without the potentially unknown confounding sociotechnical issues associated with large system implementation evaluation.

Background

Computer-assisted Data Entry. Clinical documentation is the process of generating a record of patient care, to serve as the legal record , assist in reimbursement for services provided, inform clinical decision support, and to create a repository of information for secondary data analysis (e.g., clinical research or quality improvement initiatives4,11. Computer-assisted data entry consists of automated electronic tools to facilitate the generation of clinical documentation and if used inappropriately can produce errors in documentation13. For example, using documentation fields with default values to document antibiotic administration can result in inaccurate information being documented if the clinician does not edit the default values when they deviate from actual patient care. For anesthesia documentation, computer-assisted data entry methods that do not incorporate default values have been found to have higher documentation correctness and documentation efficiency when compared to paper based documentation13.

Cognitive Workload. Medical errors have been consistently linked to the increased cognitive workload of clinicians3,5,14-16. Increased cognitive workload may impair documentation quality and result in inaccurate information being used for clinical decision making17. Cognitive workload consists of all the psychological processes that occur to complete a task18. Characteristics of the task to be completed, the individual completing the task, and the environment in which the task occurs can alter cognitive workload18.

The link between pupil diameter changes and cognitive workload has been well established19. Pupil size is determined by autonomic nervous system activity that is altered by cognitive workload, emotion, and fatigue20,21. Increased cognitive workload will result in pupil dilation with a return to resting pupil size when cognitive workload decreases20,21. Eye tracking is beneficial in measuring cognitive workload because it overcomes the major limitation of subjective psychometric instruments by allowing a continuous and reliable assessment of objective workload22.

Methods

Design and Setting. This study utilized an electronic simulated documentation interface (ESDI) combined with eye-tracking hardware to analyze documentation correctness, documentation efficiency, and cognitive workload of anesthesia providers generating documentation. The ESDI was a Windows-based software program, specifically designed for this study, that presented a series of documentation tasks to the participants by first displaying a video-based clinical scenario (see Figure 1 for example), followed by the participant documenting the patient care events that were observed (see Figure 2 for example). Study participants used a standard keyboard and mouse to interact with the program graphical user interface to document simulated patient care events viewed in the video. A description of the development of the ESDI, creation of simulated patient care events, and incorporation of the eye-tracking hardware are discussed below. We chose to use an ESDI because it would allow the use of eye-tracking and the ability to assess documentation at a more granular level (i.e., documentation data element with data entry method) than is possible with an EHR that presents multiple documentation options simultaneously. The ESDI in a controlled setting also eliminated multiple confounding variables related to the clinical environment and helped to narrow the relation of the data collected to the specific data and documentation type presented.

Figure 1.

Figure 1.

Open in a new tab

This example image represents a video-based clinical scenario that was presented to the study participant (this video depicts intubation of the trachea). The patient care activities presented in this video were documented in the following data entry screen.

Figure 2.

Figure 2.

Open in a new tab

This example image shows the data entry screen that was used to document patient care activities that were viewed in a video-based clinical scenario. This user-interface evaluates the pairing of check-boxes (a type of computer-assisted data entry) with intubation of the trachea (a specific anesthesia documentation data element).

Study Participants. Convenience and snowball sampling were used to recruit nurse anesthetists (N =20). Inclusion criteria included nurse anesthetists who had more than one year of experience using any EHR. Exclusion criteria included participants who required the use of corrective eyeglasses, have a disease of the eyes, or on a medication that alters pupil reactivity. Individuals with eye disease were excluded because of the potential impact on eye-tracking and measuring pupil size. There was an additional study participant that was excluded from data analysis because of an eye tracking hardware malfunction that resulted in failure to collect pupil diameter sizes.

The Electronic Simulated Documentation Interface. Our study used an ESDI instead of an EHR because we were assessing how pairing specific data elements to data entry approaches performed in an EHR-agnostic environment. The use of an EHR would have introduced bias from other data elements and documentation options displayed on a single screen (i.e., our ESDI presented only one documentation pairing without other visual elements on the computer screen). The ESDI was developed in collaboration with an experienced software programmer and designed to automatically capture data for documentation correctness, documentation efficiency, and cognitive workload (via pupillary changes recorded by an eye-tracker) for every participant. A description of the operational definitions for these variables are described below. The video-based clinical scenarios were developed and recorded by the primary researcher of this study (BAW) and were based on routine anesthesia tasks. Video-based clinical scenarios were used to provide a consistent clinical scenario with standard content for documentation of clinical events. Additionally, narrative text that described the clinical scenario was not used instead of video because this might create bias from differences in our study participants literacy instead of documentation of observed clinical events.

The documentation elements of the ESDI were composed of a subset of the minimum required documentation data elements that are present in the intraoperative anesthesia documentation23. This sub-set of documentation data elements was selected because it consisted of the information that is most likely to be incorrectly documented in the EHR24. The nine minimum required data elements that the study participants were required to document included: (a) antibiotic administration, (b) inhaled gas flow rates, (c) neuromuscular function testing, (d) fluid intake/output, (e) intubation of the trachea, (f) extubation of the trachea, (g) insertion of a laryngeal mask airway, (h) removal of a laryngeal mask airway, and (i) medication administration. Since there were four different data-entry methods and nine different required documentation data elements there were a total of 36 unique pairings.

Before beginning the study each participant completed a tutorial built into the ESDI. The tutorial presented a series of five standardized video-based clinical scenarios with corresponding data entry screens to demonstrate how each specific computer assisted data entry type (drop-down box, radio button, check-box, and free text with autocomplete suggestions). The purpose of this tutorial was to familiarize the study participant with the computer assisted data entry methods to reduce any history bias caused by lack of familiarity with the ESDI functionality.

Study Variables. The study variables were documentation correctness, documentation efficiency, and cognitive workload. Documentation correctness is often defined in terms of correctness and completeness13,25. Documentation correctness is high if the documentation contains the minimum required amount of information, and that information is a true description of actual patient care13,26. Documentation efficiency is defined in terms of the total amount of time required to do a specific task and the total number of physical interactions4. Lastly, cognitive workload is defined conceptually as the amount of mental tasks that must be completed in a given time frame to complete a predefined objective27. All study variables were automatically calculated via the ESDI software to help improve the reliability of measurements.

Documentation correctness. Documentation correctness was operationalized as the percent-agreement score between the study participant's final documentation and the expected documentation elements associated with the standardized patient case scenario used to develop the ESDI.

Documentation efficiency.The ESDI presented multiple data entry screens that each contained a unique pairing of computer-assisted data entry method to a specific mandatory data element. Documentation efficiency was calculated using two approaches: (a) the amount of time between the first appearance of the data entry screen to the time the documentation was entered, and (b) the total number of mouse clicks and keystrokes for each data entry screen.

Cognitive workload.Cognitive workload was measured using eye-tracking equipment that measures real-time changes in pupil size during the study participant's use of the ESDI. The subtractive method of pupillary baseline correction was used in this study because it was one of the most common approaches used in the literature, and is the least effected by bias from inaccurate baseline pupil size measurement caused by high eye blinking rates28. Baseline correction is necessary to reduce the bias caused by normal variations in pupil size between individuals28. Subtractive baseline correction is done by subtracting the maximum pupil diameter size from the resting pupil diameter size2. The GP3 HD Eyetracker 150 Hz (Vancouver, British Columbia) was used for this study and has been shown to be reliable at measuring pupillary changes at intervals of 8 milliseconds with an accuracy in pupil diameter size measurement of +/- one pixel9.

The eye-tracking equipment was mounted below the computer monitor. The ambient room lighting and desktop computer screen luminescence were kept constant to prevent artifact due to pupillary accommodation to changes in lighting. Since normal pupil diameter size in well-lit rooms ranges from 2 mm to 5 mm, any values outside of this range were identified as artifacts and treated as missing data29.

Ethical Considerations. Prior to study participant recruitment this study received approval from the University of Alabama at Birmingham institutional review board (IRB-00001656).

Data Analysis. SPSS version 25 (Armonk, NY) was used to analyze the data with descriptive statistics, ANOVAs, effect sizes, and Pearson r. ANOVAs were used to detect differences in documentation correctness, documentation efficiency, and cognitive workload for each unique pairing of computer-assisted data entry method to mandatory documentation data element. Pearson r was used to detect associations between documentation correctness, documentation efficiency, and cognitive workload. The equivalent non-parametric tests were used for non-normally distributed data.

Results and Analysis

Twenty study participants completed the study protocol during a three-week time period. The study sample was 60% (n = 12) female with an average age of 43 (SD = 6.75) years. There were no differences in the study variables based on gender. The results and analysis of the study data are presented below.

Documentation Correctness.

Computer-assisted Data Entry Methods.There was a large effect size difference (η2 = 0.2) in documentation correctness between the different computer-assisted data entry methods (F(35, 716) = 4.91, p < .001). A post-hoc analysis identified check-boxes as having the lowest documentation correctness and radio buttons as the highest. Documentation correctness for the computer-assisted data entry methods rated highest to lowest were radio buttons (M = 92%, SD = 15%), free text (M = 89.5%, SD = 25%), drop-boxes (M = 85%, SD = 24%), and check-boxes (M = 83%, SD = 23%).

There was a medium effect size association between documentation correctness and total number of mouse clicks (r = -0.20, p < .001). A higher number of mouse clicks occurred with the use of radio buttons and free text with all of the anesthesia documentation data elements and resulted in the highest documentation correctness. The anesthesia documentation data elements that demonstrated the highest documentation correctness were associated with extubation of the trachea, medication administration, and neuromuscular function testing.

Anesthesia Documentation Data Elements. See Table 1 for a summary of documentation correctness for each specific anesthesia documentation data element. There was a medium effect size difference (η2 = .07) in documentation correctness between each type of documentation data element (F(8, 711) = 6.39, p < .001). A post-hoc analysis identified the anesthesia documentation data elements with the highest documentation correctness were extubation of the trachea (M = 95.7%, SD = 11.1%), medication administration (M = 94.7%, SD = 11%), and neuromuscular function testing (M = 93.8%, SD = 24.4%). Additionally, fluid intake/output (M = 78.6%, SD = 25.9%) and antibiotic administration (M = 81.6%, SD = 28.6%) had the lowest documentation correctness.

Table 1. Documentation correctness for each type of anesthesia documentation data element.

Documentation Data Elements Mean Standard Deviation Number
Extubation of the trachea 95.7% 11.1% 80
Medication administration 94.7% 11.0% 80
Neuromuscular Function Testing 93.8% 24.4% 80
Insertion of laryngeal mask airway 87.8% 21.1% 80
Inhaled gas flow rates 85.6% 26.7% 80
Intubation of the trachea 85.1% 16.0% 80
Removal of laryngeal mask airway 84.7% 18.0% 80
Antibiotic administration 81.6% 28.6% 80
Fluid intake and output 78.6% 25.9% 80
Average for All Data Elements 87.5% 21.9% 720
Open in a new tab

Note. This table displays the documentation correctness (percentage correct) for each type of anesthesia data. The data elements are listed in order of highest to lowest

Documentation Efficiency

Computer-assisted Data Entry Methods.There was a large effect size difference (η2 = 0.44) in the total time generating documentation between the different computer-assisted data entry methods (F(3, 716) = 185.96, p < .001). An increase in the total time spent generating documentation reflects decreased documentation efficiency (i.e., higher time durations are worse). A post-hoc analysis identified that radio buttons and check-boxes had no statistically significant differences between each other, but the other computer-assisted data entry methods differed. The total time spent documenting from the most to least efficient is check-boxes (M = 10.66 seconds, SD = 4.94), radio buttons (M = 11.57 seconds, SD = 6.57), drop-boxes (M = 16.11 seconds, SD = 7.76), and free text (M = 30.65 seconds, SD = 14.26).

The Kruskal-Wallis test was used to detect large effect size differences for keystrokes (chi-square = 692.40, df = 3, p < .001) and mouse clicks (chi-square = 470.39, df = 3, p < .001) between the different computer-assisted data entry methods. Free text had the highest number of keystrokes (M = 56.09, SD = 34.74) and drop-boxes had the highest number of mouse clicks (M = 4.39, SD = 1.60). There were no statistically significant differences in mouse clicks between radio buttons and check-boxes.

Several statistically significant associations were identified. There was a large effect size association between study participant age and total time generating documentation (r = 0.47, p < .001). A large effect size association existed between the total number of keystrokes and the total time generating documentation (r = 0.68, p < .001). There was a large negative effect size association between total number of keystrokes and mouse clicks (ρ = -0.49, p < .001).

Anesthesia Documentation Data Elements.A large effect size difference (η2 = 0.15) existed for the total time generating documentation between each specific anesthesia documentation data element (F(8, 711) = 16.16, p < .001). See Table 2 for a summary of total time spent generating documentation for each documentation data elements. A post-hoc analysis identified the data elements with the highest documentation efficiency were neuromuscular function testing (M = 7.2 seconds, SD = 6.57) and inhaled gas flow rates (M = 12.99 seconds, SD = 9.26). The documentation data elements with the worst documentation efficiency were fluid intake/output (M = 24.28 seconds, M = 9.71) and extubation of the trachea (M = 21.31 seconds, SD = 15.45).

Table 2. Documentation efficiency for each type of anesthesia documentation data element.

Documentation Data Elements Mean (in seconds) Standard Deviation Number
Fluid intake and output 24.28 9.71 80
Extubation of the trachea 21.31 15.45 80
Intubation of the trachea 19.64 13.50 80
Removal of laryngeal mask airway 19.16 15.92 80
Antibiotic administration 18.80 8.14 80
Insertion of laryngeal mask airway 17.05 11.89 80
Medication administration 14.80 4.92 80
Inhaled gas flow rates 12.99 9.26 80
Neuromuscular function testing 7.20 6.57 80
Average for All Data Elements 17.25 12.11 720
Open in a new tab

Note. This table displays the documentation efficiency (total time used to generate documentation) for each type of anesthesia data. The data elements are listed in order of highest to lowest total time spent on data entry. Lower documentation times reflect higher documentation efficiency

Cognitive Workload

Computer-assisted Data Entry Methods.Cognitive workload was calculated as the maximum pupil diameter size minus resting pupil diameter size (subtractive baseline correction). There was a small to medium effect size difference (η2 = .04) in cognitive workload between the different computer-assisted data entry methods (F(3, 698) = 9.96, p < .01). Free text (M = 0.547 mm, SD = .301) had the highest cognitive workload compared to check-boxes (M = 0.425 mm, SD = 0.298), radio buttons (M = 0.411, SD = 0.275), and drop-boxes (M = 0.401 mm, SD = 0.265). Cognitive workload was similar for check-boxes, radio buttons, and drop-boxes. There were 18 pupil diameter measurements treated as missing data because they were less than 2 mm or greater than 5 mm. This missing data was 2.5% of total pupil diameter measurements (18 of 720 measurements).

Anesthesia Documentation Data Elements.No statistically significant differences existed for cognitive workload between any of the anesthesia documentation data elements. Small effect size associations existed between cognitive workload and time spent documenting (r = 0.16, p < .001), total number of keystrokes (ρ = 0.15, p < .001), and total number of mouse clicks (ρ = -0.17, p < .001).

Discussion

Pairing computer-assisted data entry methods to anesthesia documentation data elements requires a consideration of the collective relationships between documentation correctness, efficiency, and cognitive workload. Additionally, the inherent properties of the computer-assisted data entry methods need to be considered to optimally pair data entry methods to the type of information to be documented. A discussion of these topics is presented below.

Documentation Correctness & Efficiency. This study found a large negative effect size association between the overall documentation correctness and efficiency, which is supported by the literature13,24,30. There is often a reciprocal relationship between documentation correctness and efficiency in EHRs where attempts to improve one often impairs the other13,30. This may be partially explained by documentation correctness decreasing as the study participant was forced to spend more time with data entry and more physical interactions with the data entry user-interface. Anesthesia providers may often ignore generation of documentation in favor of direct patient care; consequently, data entry user-interfaces that require a lot of time to complete may be more likely to be ignored or abbreviated by the anesthesia provider23.

While there is a negative association between documentation correctness and efficiency when evaluating an overall data-entry interface, there is a positive association when looking at some specific data entry fields31,32. Improving the efficiency for specific data entry fields related to nursing patient admission histories has been shown to improve documentation correctness31. Furthermore, documentation of quality measures for oncology patients has also been shown to be more accurate when the documentation process is more efficient33. Our study found a similar positive relationship for documenting neuromuscular function testing and fluid intake/output where increasing the efficiency of data entry resulted in improved documentation correctness. This may be because neuromuscular function testing and fluid intake/output both had the least amount of information for data entry compared to the other documentation data elements. Since documentation data entry for fluid intake/output was the most inefficient anesthesia documentation data element in our study, documentation correctness may be improved by determining a more efficient means of documentation.

Older study participants were less efficient using free text for data entry because they documented more contextual information. For example, one of the older study participants documented, "Pt. gas flow changed to increase sats. Now at 2L O2 and 1L Air." while a younger participant documented, "2L of O2 and 1L Air." While both responses were technically the same in documentation correctness the older participant provided a richer description to justify the patient care provided. Younger study participants were more efficient in using all of the computer-assisted data entry options but documented only the minimum amount of information in free text. We could not locate literature that described the impact of age or total years of clinical anesthesia experience on documentation generation practices.

Cognitive Workload. This study found negative associations between cognitive workload and documentation efficiency. An increase in the amount of time, number of keystrokes, and number of mouse clicks required to complete documentation increased cognitive workload. Data entry user-interfaces need to be designed to avoid excessive keystrokes or mouse clicks that increase cognitive workload because it may result in medical errors34. The cognitive workload related to using check-boxes, radio buttons, and drop-boxes with the anesthesia documentation data elements were similar. Individual anesthesia documentation data elements have been shown to have similar cognitive workload2. Free text had a medium effect size difference in cognitive workload from the other computer-assisted data entry methods and was also the most inefficient.

There were also no statistically significant differences in cognitive workload related to the anesthesia documentation data elements. Consequently, measuring cognitive workload is not useful at a granular level (i.e., single computer-assisted data entry method), but may be beneficial when evaluating an entire data entry user-interface that incorporates different types of computer-assisted data entry methods. Future research needs to explore the cognitive workload associated with multiple pairings of computer-assisted data entry methods to document anesthesia documentation data elements. Additionally, future research could incorporate high-fidelity clinical simulations that mimic real-world events when evaluating user-interfaces because the people and environment is known to alter how information technology is used.

Properties of the Computer-assisted Data Entry Methods. The inherent properties of radio buttons, check-boxes, drop-down boxes, and free text need to be considered before pairing them with specific documentation data elements. Cognitive workload is similar for each computer-assisted data entry method based on our findings (except for free text), therefore it does not need to be considered when choosing the other computer-assisted data entry methods. Radio buttons should be used with less than five data selection options and over use of radio buttons increases cognitive workload secondary to information overload35,36. Check-boxes are ideally used with binary data options and may result in information overload if too many options are available2,35. Our study found that the use of radio buttons or checkboxes with more than five data selection options resulted in impaired documentation efficiency. Drop-down boxes are suited for use when there are more than five possible data selection options35,36. Drop-down boxes that require scrolling will increase cognitive workload and decrease documentation correctness35,36.

Free text is an appropriate selection when there is a virtually unlimited number of possible values that may be documented36. This study identified free text as the second highest in documentation correctness, the most inefficient, and the highest cognitive workload compared to other computer-assisted data entry methods. In the literature, free text has been consistently linked with improved documentation correctness but less completeness of data (i.e., clinicians fail to document important information)30,37,38. It is recommended to limit the use of free text if there is an option to use other computer-assisted data entry methods because free text is more likely to be incomplete30. Free text also limits data reusability (e.g., retrospective studies or quality improvement initiatives) because it requires human interpretation30.

Limitations

This study had several limitations. Documentation generation in real-world settings combine multiple computer-assisted data entry methods simultaneously for each anesthesia documentation data element. This study evaluated unique pairings of a single computer-assisted data entry method with a single type of anesthesia documentation data element. This approach was chosen because of sample size limitations and the generalizability may have been impaired. Our eye-tracking hardware was not capable of measuring pupil diameter measurements consistently if the clinician wore corrective eyeglasses (which was an exclusion criteria in this study), so further studies on this topic should use eye-tracking hardware capable of compensating for corrective eyeglasses so the findings are more generalizable.

Conclusion

Inadequately designed data entry user-interfaces may result in impaired patient safety and outcomes because incorrect information is used to guide future clinical decision making. There is often tension between documentation correctness and efficiency in EHRs where attempts to improve one often impairs the other. We found this to be true in our study, documentation correctness was negatively associated with efficiency. However, we found that documentation data elements that contained a minimal amount of information (e.g., neuromuscular function testing or fluid intake/output) showed improved efficiency and correctness with the use of check boxes and radio buttons. Overall, free text was the least efficient, followed by drop boxes, with check boxes and radio buttons being the most efficient. Significant differences were noted in correctness between types of data entry methods with check boxes having the lowest documentation correctness and radio buttons the highest. Increasing the number of manual keyboard operations during documentation was shown to decrease efficiency and increase cognitive workload. However, cognitive workload associated with each individual computer-assisted data entry methods were similar when evaluating documentation at a granular level (i.e., a single type of computer-assisted data entry). This study showed how pairing specific entry methods with types of specific data can effect completeness, correctness, and cognitive workload. Inadequately designed data entry user-interfaces may result in impaired patient safety and outcomes. These study findings show how user interface design can be enhanced to increase the quality of clinical documentation.

Figures & Table

References

  • 1.Marian AA, Bayman EO, Gillett A, Hadder B, Todd MM. The influence of the type and design of the anesthesia record on ASA physical status scores in surgical patients: paper records vs. electronic anesthesia records. BMC Med Inform Decis Mak. 2016;16:29. doi: 10.1186/s12911-016-0267-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Wanderer JP, Rao AV, Rothwell SH, Ehrenfeld JM. Comparing two anesthesia information management system user interfaces: a usability evaluation. Can J Anaesth. 2012;59(11):1023–31. doi: 10.1007/s12630-012-9771-z. [DOI] [PubMed] [Google Scholar]
  • 3.Sittig DF, Wright A, Ash J, Singh H. New Unintended Adverse Consequences of Electronic Health Records. Yearb Med Inform. 2016;(1):7–12. doi: 10.15265/IY-2016-023. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Mamykina L, Vawdrey DK, Stetson PD, Zheng K, Hripcsak G. Clinical documentation: composition or synthesis? J Am Med Inform Assoc. 2012;19(6):1025–31. doi: 10.1136/amiajnl-2012-000901. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Campbell EM, Sittig DF, Ash JS, Guappone KP, Dykstra RH. Types of unintended consequences related to computerized provider order entry ? J Am Med Inform Assoc. 2006;13(5):547–56. doi: 10.1197/jamia.M2042. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Sittig DF, Singh H. Defining health information technology-related errors: new developments since to err is human. Arch Intern Med. 2011;171(14):1281–4. doi: 10.1001/archinternmed.2011.327. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Wilbanks BA, Geisz-Everson M, Clayton BA, Boust RR. Transfer of care in perioperative settings: A descriptive qualitative study. AANA J. 2018;86(5):401–7. [PubMed] [Google Scholar]
  • 8.Ellsworth MA, Dziadzko M, O’Horo JC, Farrell AM, Zhang J, Herasevich V. An appraisal of published usability evaluations of electronic health records via systematic review. J Am Med Inform Assoc. 2016;24(1):218–226. doi: 10.1093/jamia/ocw046. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 9.Zheng K, Abraham J, Novak LL, Reynolds TL, Gettinger A. A Survey of the Literature on Unintended Consequences Associated with Health Information Technology: 2014-2015. Yearb Med Inform. 2016(1):13–29. doi: 10.15265/IY-2016-036. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Ratwani RM, Fairbanks RJ, Hettinger AZ, Benda NC. Electronic health record usability: analysis of the user-centered design processes of eleven electronic health record vendors. Journal of the American Medical Informatics Association : JAMIA. 2015;22(6):1179–1182. doi: 10.1093/jamia/ocv050. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11.Rosenbloom ST, Denny JC, Xu H, Lorenzi N, Stead WW, Johnson KB. Data from clinical notes: a perspective on the tension between structure and flexible documentation. J Am Med Inform Assoc. 2011;18(2):181–6. doi: 10.1136/jamia.2010.007237. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Shoolin J, Ozeran L, Hamann C, Bria W. 2nd. Association of Medical Directors of Information Systems consensus on inpatient electronic health record documentation. Applied clinical informatics. 2013;4(2):293–303. doi: 10.4338/ACI-2013-02-R-0012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Wilbanks BA, Berner ES, Alexander GL, Azuero A, Patrician PA, Moss JA. The effect of data-entry template design and anesthesia provider workload on documentation accuracy, documentation efficiency, and user-satisfaction. Int J Med Inform. 2018;118:29–35. doi: 10.1016/j.ijmedinf.2018.07.006. [DOI] [PubMed] [Google Scholar]
  • 14.Weber-Jahnke JH, Mason-Blakley F, editors. International Symposium on Foundations of Health Informatics Engineering and Systems. Springer; 2011. On the safety of electronic medical records. [Google Scholar]
  • 15.Ancker JS, Edwards A, Nosal S, Hauser D, Mauer E, Kaushal R. Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Med Inform Decis Mak. 2017;17(1):36. doi: 10.1186/s12911-017-0430-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Peute LW, De Keizer NF, Van Der Zwan EP, Jaspers MW. Reducing clinicians’ cognitive workload by system redesign; a pre-post think aloud usability study. Stud Health Technol Inform. 2011;169:925–9. [PubMed] [Google Scholar]
  • 17.Gartner D, Zhang Y, Padman R. Cognitive workload reduction in hospital information systems : Decision support for order set optimization. Health Care Manag Sci. 2017. [DOI] [PubMed]
  • 18.Young MS, Brookhuis KA, Wickens CD, Hancock PA. State of science: mental workload in ergonomics. Ergon. 2015;58(1):1–17. doi: 10.1080/00140139.2014.956151. [DOI] [PubMed] [Google Scholar]
  • 19.Coyne J, Sibley C, editors. Proc Hum Factors Ergon Soc Annu Meet. Los Angeles, CA: SAGE Publications Sage CA; 2016. Investigating the Use of Two Low Cost Eye Tracking Systems for Detecting Pupillary Response to Changes in Mental Workload. [Google Scholar]
  • 20.Kok EM, Jarodzka H. Before your very eyes: the value and limitations of eye tracking in medical education. Med Educ. 2017;51(1):114–22. doi: 10.1111/medu.13066. [DOI] [PubMed] [Google Scholar]
  • 21.Mosaly PR, Mazur L, Marks LB, editors. Usability evaluation of electronic health record system (EHRs) using subjective and objective measures. Proceedings of the 2016 ACM Conference on Human Information Interaction and Retrieval. 2016. pp. 313–316.
  • 22.Matthews G, Reinerman-Jones LE, Barber DJ, Abich Jt. The psychometrics of mental workload: multiple measures are sensitive but divergent. Hum Factors. 2015;57(1):125–43. doi: 10.1177/0018720814539505. [DOI] [PubMed] [Google Scholar]
  • 23.American Association of Nurse Anesthetists . Documenting the Standard of Care: The Anesthesia Record. Park Ridge, IL: American Association of Nurse Anesthetists; 2010. p. 14. [Google Scholar]
  • 24.Wilbanks BA, Moss JA, Berner ES. An observational study of the accuracy and completeness of an anesthesia information management system: recommendations for documentation system changes. Comput Inform Nurs. 2013;31(8):359–67. doi: 10.1097/NXN.0b013e31829a8f4b. [DOI] [PubMed] [Google Scholar]
  • 25.Wilbanks BA, Geisz-Everson M, Boust RR. The Role of Documentation Quality in Anesthesia-Related Closed Claims: A Descriptive Qualitative Study. Comput Inform Nurs. 2016;34(9):406–12. doi: 10.1097/CIN.0000000000000270. [DOI] [PubMed] [Google Scholar]
  • 26.Wilbanks BA. An integrative literature review on accuracy in anesthesia information management systems. Comput Inform Nurs. 2014;32(3):56–63. doi: 10.1097/NXN.0b013e3182a041f7. [DOI] [PubMed] [Google Scholar]
  • 27.Wilbanks BA. An integrative literature review of contextual factors in perioperative information mangagement systems. Comput Inform Nurs. 2013;31(12):622–8. doi: 10.1097/CIN.0000000000000007. [DOI] [PubMed] [Google Scholar]
  • 28.Mathot S, Fabius J, Van Heusden E, Van der Stigchel S. Safe and sensible preprocessing and baseline correction of pupil-size data. Behav Res Methods. 2018;50(1):94–106. doi: 10.3758/s13428-017-1007-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.Watson AB, Yellott JI. A unified formula for light-adapted pupil size. J Vis. 2012;12(10):12. doi: 10.1167/12.10.12. [DOI] [PubMed] [Google Scholar]
  • 30.Wilbanks BA, Moss J. Evidence-Based Guidelines for Interface Design for Data Entry in Electronic Health Records. Comput Inform Nurs. 2018;36(1):35–44. doi: 10.1097/CIN.0000000000000387. [DOI] [PubMed] [Google Scholar]
  • 31.Karp EL, Freeman R, Simpson KN, Simpson AN. Changes in Efficiency and Quality of Nursing Electronic Health Record Documentation After Implementation of an Admission Patient History Essential Data Set. Comput Inform Nurs. 2019;37(5):260–5. doi: 10.1097/CIN.0000000000000516. [DOI] [PubMed] [Google Scholar]
  • 32.Weng CY. Data Accuracy in Electronic Medical Record Documentation. JAMA Ophthalmol. 2017;135(3):232–3. doi: 10.1001/jamaophthalmol.2016.5562. [DOI] [PubMed] [Google Scholar]
  • 33.Esper P, Walker S. Improving documentation of quality measures in the electronic health record. J Am Assoc Nurse Pract. 2015;27(6):308–12. doi: 10.1002/2327-6924.12169. [DOI] [PubMed] [Google Scholar]
  • 34.Zahabi M, Kaber DB, Swangnetr M. Usability and Safety in Electronic Medical Records Interface Design: A Review of Recent Literature and Guideline Formulation. Human factors. 2015;57(5):805–34. doi: 10.1177/0018720815576827. [DOI] [PubMed] [Google Scholar]
  • 35.Sewell JP, Thede LQ. Informatics and nursing: Opportunities and challenges. Wolters Kluwer Health/Lippincott Williams & Wilkins; 2013. [Google Scholar]
  • 36.Marian AA, Dexter F, Tucker P, Todd MM. Comparison of alphabetical versus categorical display format for medication order entry in a simulated touch screen anesthesia information management system: an experiment in clinician-computer interaction in anesthesia. BMC Med Inform Decis Mak. 2012;12:46. doi: 10.1186/1472-6947-12-46. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Tsou AY, Lehmann CU, Michel J, Solomon R, Possanza L, Gandhi T. Safe Practices for Copy and Paste in the EHR. Systematic Review, Recommendations, and Novel Model for Health IT Collaboration. Appl Clin Inform. 2017;8(1):12–34. doi: 10.4338/ACI-2016-09-R-0150. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Markel A. Copy and paste of electronic health records: a modern medical illness. Am J Med. 2010;123(5):e9. doi: 10.1016/j.amjmed.2009.10.012. [DOI] [PubMed] [Google Scholar]

Articles from AMIA Summits on Translational Science Proceedings are provided here courtesy of American Medical Informatics Association

RESOURCES