Effect of restricting electronic health records on clinician efficiency: substudy of a randomized clinical trial

Abstract

A prior randomized controlled trial (RCT) showed no significant difference in wrong-patient errors between clinicians assigned to a restricted electronic health record (EHR) configuration (limiting to 1 record open at a time) versus an unrestricted EHR configuration (allowing up to 4 records open concurrently). However, it is unknown whether an unrestricted EHR configuration is more efficient. This substudy of the RCT compared clinician efficiency between EHR configurations using objective measures. All clinicians who logged onto the EHR during the substudy period were included. The primary outcome measure of efficiency was total active minutes per day. Counts were extracted from audit log data, and mixed-effects negative binomial regression was performed to determine differences between randomized groups. Incidence rate ratios (IRRs) were calculated with 95% confidence intervals (CIs). Among a total of 2556 clinicians, there was no significant difference between unrestricted and restricted groups in total active minutes per day (115.1 vs 113.3 min, respectively; IRR, 0.99; 95% CI, 0.93–1.06), overall or by clinician type and practice area.

INTRODUCTION

Optimizing electronic health record (EHR) configurations requires balancing the need for patient safety without compromising clinician efficiency. The Joint Commission and Office of the National Coordinator for Health Information Technology recommended limiting clinicians to open 1 patient record at a time in the EHR to prevent wrong-patient errors.^1,2 However, this restricted configuration may not be consistent with the Department of Health and Human Services goal to mitigate EHR-related burdens on clinicians, including reducing effort and time.³ A national survey showed considerable variation in EHR configuration across healthcare systems, with respondents reporting perceptions that limiting to 1 record was safer, but allowing multiple records open was more efficient.⁴

To test safety, Adelman et al conducted a randomized controlled trial (RCT) and found no significant difference in wrong-patient orders in a restricted EHR configuration (limiting to 1 record open at a time) compared with an unrestricted configuration (allowing up to 4 records open concurrently).⁵ Clinicians in the unrestricted configuration reported significantly greater satisfaction, efficiency, and usability compared to those in the restricted configuration.⁶ However, it is unknown whether an unrestricted EHR configuration was more efficient. Therefore, we conducted a substudy of the parent RCT using EHR audit log data to test the hypothesis that a restricted EHR configuration decreased clinician efficiency when compared with an unrestricted configuration.

MATERIALS AND METHODS

Study design

This was a prespecified substudy of a larger RCT investigating the risk of wrong-patient orders in 2 different EHR configurations (NCT02876588). In the parent study, clinicians were randomized in a 1:1 ratio to an EHR configuration that limited to 1 record open at a time (restricted) or to an EHR configuration that allowed up to 4 records open concurrently (unrestricted). Clinicians were randomized using a computerized random number generator and were assigned one random number. Clinicians with odd numbers were assigned to the restricted arm, and clinicians with even numbers were assigned to the unrestricted arm. Clinicians entering the study after the start date were assigned a number after being assigned a user login. The parent study was conducted at a large academic medical center in New York, NY, and took place over 17 months from October 2015 to April 2017. In the parent study, all clinicians who placed orders in the EHR during the 17-month study period were included. Clinicians were excluded from the study if their workflow involved always opening 2 records simultaneously (eg, mother–infant services) or bypassing the standard order entry process (eg, radiologists). The trial was conducted in the Epic EHR system (EpicCare, Epic Systems Corporation). Trial methods and primary results have been previously reported (Supplementary Material S1).⁵ The study protocol was approved by the institutional review boards of the Albert Einstein College of Medicine and Columbia University Irving Medical Center. Waiver of informed consent was granted for clinicians.

This substudy took place over a 2-month period from October 2015 to November 2015 during the parent study period. All clinicians who logged onto the EHR from October 2015 to November 2015 were included in this analysis. For the purposes of this study, we define efficiency as the total time spent on the EHR by the clinician.⁷

Primary outcome

In this substudy, the primary outcome measure of efficiency was total active minutes per day, defined as the sum of time intervals beginning when a clinician either logs into the EHR or uses the mouse or keyboard to interact with the EHR, and ending 30 s after no further interactions were logged.

Secondary outcomes

Secondary outcome measures of efficiency were total mouse clicks, key presses, and screen changes per day. Total mouse clicks included all clicks recorded while interacting with the EHR. Total key presses included all key presses recorded while using the EHR. Screen changes recorded the number of times a clinician changing to a new screen, such as switching between patient records, switching from notes to laboratory values, or switching from notes to imaging tabs.

Data collection

Counts for both primary and secondary outcomes were extracted from audit log data aggregated in 1-h time intervals and then summarized as mean and standard deviation (SD) per day. Audit log data reflected clinician interactions with an EHR workstation, excluding mobile devices. Characteristics for each clinician, including age, sex, clinician type, and primary department, were extracted retrospectively from the healthcare system data warehouse after the study period.

Statistical analysis

Clinician characteristics were summarized using descriptive statistics (mean, SD, counts, percentages). Mixed-effects negative binomial regression was performed to determine differences between randomized groups for all outcomes. Incidence rate ratios (IRRs) were calculated and reported with 95% confidence intervals (CIs). As described in the reported RCT, there was crossover between randomized groups.⁵ Therefore, as-randomized and as-treated analyses were conducted. All statistical analyses were conducted using Stata version 17.0 (StataCorp). Data were analyzed in December 2021.

RESULTS

Study population

A total of 2556 clinicians were included in this substudy for analysis. Clinician characteristics are shown in Table 1. Clinician characteristics were similar between unrestricted and restricted groups, including age (mean, 43.7 vs 43.6 years), sex (female, 42.9% vs 43.4%), clinician type, and primary practice setting. Due to administrative error, 83 clinicians were misassigned at the start of the trial and 100 crossed over during the study (n = 183) (Figure 1).

Table 1.

Characteristics of clinicians

	Randomized group, N (%)
	Unrestricteda	Restrictedb
Characteristic	(n = 1273)	(n = 1283)
Age, mean (SD), years	43.7 (12.4)	43.6 (12.7)
Sex
Male	663 (57.1)	660 (56.6)
Female	498 (42.9)	506 (43.4)
Clinician type
Attending	637 (50.2)	647 (50.5)
House staff	353 (27.8)	346 (27.0)
Advanced practice provider	278 (21.9)	287 (22.4)
Primary practice area
Emergency department	118 (9.3)	129 (10.1)
Inpatient	421 (33.1)	445 (34.7)
Outpatient	631 (49.6)	617 (48.1)
Unclassified	103 (8.1)	92 (7.2)

^aUnrestricted: in the unrestricted group, the EHR configuration allowed up to 4 patient records open simultaneously.

^bRestricted: in the restricted group, the EHR configuration limited to 1 patient record open at a time.

Figure 1.

Substudy flow diagram. Randomization and allocation of clinicians in the efficiency substudy.

Primary outcome

For the primary outcome, there was no significant difference overall between unrestricted and restricted groups in total active minutes per day (mean, 115.1 vs. 113.3 min, respectively; IRR, 0.99; 95% CI, 0.93–1.06). Similarly, there were no between-group differences in subgroups defined by clinician type or practice area (Table 2). However, there was variation within groups by clinician type and practice area. Advanced practice providers had the most total active minutes per day in both unrestricted and restricted groups (134.5 and 139.9 min, respectively). Clinicians in the Emergency Department setting had the most total active minutes per day in both unrestricted and restricted groups (191.9 and 189.0 min, respectively). Results of the as-treated analysis for the primary outcome were similar (Supplementary Material S2, Table).

Table 2.

Primary and secondary outcome measures of efficiency

	Randomized group, mean (SD)
	Unrestricteda	Restrictedb	IRR (95% CI)
Primary outcome: total active minutes per day
Overall	115.1 (97.7)	113.3 (99.6)	0.99 (0.93–1.06)
Clinician type
Attending	117.8 (105.5)	114.0 (102.3)	0.97 (0.88–1.06)
House staff	95.4 (85.7)	90.1 (81.8)	0.99 (0.88–1.12)
Advanced practice provider	134.5 (98.9)	139.9 (98.3)	1.05 (0.91–1.19)
Primary practice area
Emergency department	191.9 (107.8)	189.0 (104.6)	0.97 (0.89–1.06)
Inpatient	110.2 (91.0)	108.8 (91.3)	0.99 (0.89–1.10)
Outpatient	111.9 (101.0)	108.3 (97.5)	0.96 (0.79–1.18)
Unclassified	76.6 (77.7)	76.8 (76.1)	1.03 (0.82–1.29)
Secondary outcomes per day
Mouse clicks	891.3 (891.0)	884.0 (891.4)	1.01 (0.93–1.08)
Key presses	4281.6 (4406.6)	4075.9 (4636.6)	0.95 (0.87–1.04)
Screen changes	255.1 (248.9)	250.4 (257.7)	1.00 (0.93–1.08)

^aUnrestricted: in the unrestricted group, the EHR configuration allowed up to 4 patient records open simultaneously.

^bRestricted: in the restricted group, the EHR configuration limited to 1 patient record open at a time.

Secondary outcomes

Overall, there were no significant differences between the unrestricted and restricted groups for secondary outcomes of mouse clicks, key presses, and screen changes. For mouse clicks, there were mean 891.3 versus 884, respectively (IRR, 1.01; 95% CI, 0.93–1.08). For key presses, there were mean 4281.6 versus 4075.9, respectively (IRR, 0.95; 95% CI, 0.87–1.04). For screen changes, there were mean 255.1 versus 250.4, respectively (IRR, 1.00; 95% CI, 0.93–1.08). Results of as-treated analyses for the secondary outcomes were similar (Supplementary Material S2, Table).

DISCUSSION

In this substudy of an RCT, we found no evidence that an unrestricted configuration led to fewer total daily minutes on the EHR, mouse clicks, key presses, or screen changes compared to a restricted configuration. Additionally, we saw no difference by clinician type or primary practice area. Taken together, the results of the RCT showed that restricting clinicians to 1 EHR record at a time did not reduce wrong-patient orders⁵; furthermore, despite perceptions that an unrestricted EHR configuration is more efficient,^4,6 restricting to one patient record at a time did not reduce clinician efficiency as measured here.

Despite no difference in measures of efficiency as reported by this study, the same study participants reported greater efficiency in the unrestricted configuration.⁶ Clinicians in the restricted configuration reported frustrations at their inability to multitask, which prompted some to use potentially hazardous workarounds to complete tasks.⁶ Therefore, although the unrestricted configuration was perceived as more efficient by clinicians, perhaps owing to the ability to open multiple records as needed, these results suggest no savings in time or effort on average based on EHR audit log metrics. Although we were not able to analyze measures of efficiency by the number of open patient records in the unrestricted arm, in the parent RCT 66.5% of order sessions occurred with 2 or more patient records open in the emergency department compared with 46.7% in inpatient and 16.6% in outpatient settings.⁵ In this substudy, there was no significant difference in total active minutes per day between randomized groups despite the substantially greater total active time among clinicians in the emergency department, and the majority of their orders placed with more than 2 records open.

Given clinician burnout and frustration related to the EHR, efforts to create clinician-centered EHR designs that mitigate burnout and reduce effort and time on the EHR are a high priority.^3,8–12 The findings of this study in addition with the parent RCT show that a restricted EHR configuration does not reduce wrong-patient errors, nor does it decrease clinician efficiency.⁵ Furthermore, clinicians in the unrestricted group reported significantly greater efficiency, usability, and satisfaction with the EHR configuration.⁶ In conjunction, these findings support an unrestricted EHR configuration being as safe as a restricted configuration, while being more clinician-centered.

Prior efficiency studies have focused on whether additional EHR training for clinicians can improve efficiency in the same EHR system.^9,13 The use of audit log data has been previously used in observational studies to describe clinician workflows in different practice areas, different times of day, and comparing clinicians by level of training.^14–16 This study is one of the few to use audit log data as measures of efficiency to directly compare the effect of different EHR configurations on clinician efficiency. This study focuses on improvements that can be made in the EHR system, rather than the clinician. It also reinforces the importance of audit log data as measures of efficiency when comparing clinician workflows.

This study has multiple strengths. First, clinicians were randomized to the different EHR configurations. Second, a large number of clinicians in different roles and settings were included and analyzed in this study. This study has multiple limitations. First, it was a substudy of a larger RCT and not the primary outcome of the parent study. Second, this study was conducted in a single-center setting, potentially limiting generalizability. Third, using total active minutes as a surrogate for efficiency captures only one dimension of a complex construct. Measuring clinician efficiency involves many factors, including quantifying specific tasks in patient care and tasks completed per unit of time. There may be unmeasured factors that explain clinicians’ perception of greater efficiency in the unrestricted arm in the parent RCT. However, total active minutes have been proposed as key metric to measure clinician efficiency.⁷

CONCLUSION

This substudy of an RCT comparing clinicians randomized to a restricted EHR configuration (limited to 1 record open) or an unrestricted (allowing up to 4 records open) showed that despite perceptions that an unrestricted EHR was more efficient, audit log data showed no significant difference in measures of efficiency. Further efforts must be made to configure the EHR to balance patient safety, while mitigating EHR-related burdens on clinicians.

FUNDING

This project was supported by grant numbers T32HS026121 and R21HS023704 from the Agency for Healthcare Research and Quality. The content is solely the responsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality. The Agency of Healthcare Research and Quality had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

AUTHOR CONTRIBUTIONS

JSA designed the overall study in consultation with JRA, CBS, HS, and WNS. JZK-H and CBS provided analysis of the data. JZK-H, JRA, ADC, and CBS provided interpretation of study results. JZK-H and JRA wrote the manuscript with input from CBS, HS, WNS, and JSA. All authors read, reviewed, and contributed critical revisions to the manuscript. JSA, JRA, and WNS contributed supervision and oversight.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.

CONFLICT OF INTEREST STATEMENT

None declared.

TRIAL REGISTRATION

Clinicaltrials.gov Identifier: NCT02876588

DATA AVAILABILITY

See Supplementary Material S3.