Abstract
Background
Wrong patient selection errors may be tracked by retract–reorder (RAR) events. The aim of this quality improvement study was to assess the impact of reducing the number of concurrently open electronic health records from 4 to 2 on RAR errors generated by a tele-critical care service.
Methods
The study encompassed 32 months before and 21 months after restriction. Chi-Square test of proportions and T statistical process control chart for rare events were used.
Results
There were 156 318 orders with 57 RAR errors (36.5/100 000 orders) before restriction, and 122 587 orders with 34 errors (27.7/100 000 orders) after. Rates were not statistically different (P = .20), but analysis was underpowered. When plotted on a T control chart, random variation was detected between RAR errors.
Conclusion
We found no significant difference in RAR errors in the tele-critical care setting after open record limitation. Other strategies should be studied to reduce wrong patient selection errors.
Keywords: information technology, medical order entry systems, telemedicine, patient safety, medical errors
INTRODUCTION
Problem description
Electronic health records (EHR) and computerized provider order entry (CPOE) should improve clinical efficiency and patient safety.1,2 CPOE has reduced medication-related errors, but increased risks such as potentially harmful wrong patient selection.3–5 Aiming to prevent wrong patient selection, our institution added patient photos to the EHR in December 2017. Months later, the number of concurrently open records allowed was reduced from 4 to 2. We are unaware of any reports of how this limitation affects wrong patient selection in a tele-critical care (TCC) setting. Therefore, this study evaluates wrong patient selection errors by TCC clinicians before and after the limitation. It is presented using the SQUIRE 2.0 guidelines for reporting quality improvement studies.6
Available knowledge
Retract–reorder (RAR) is a phenomenon whereby an order written for 1 patient is canceled, then reordered on a different patient within 10 minutes.7,8 CPOE permits quantification of patient selection errors by tracking RAR events. Reducing these errors may require optimizing EHR user interface, and restricting the number of concurrently opened records.9–11 The optimal number of open records that balances efficiency with safety is unknown, despite prior research.12 One retrospective emergency department (ED) study found 83 RAR errors/100 000 orders with no reduction by limiting open records.13 Similar studies in our ED and cardiology units found no significant reduction.14,15 Elsewhere, a prospective randomized study restricted clinicians to 1 open record, or up to 4 records in the unrestricted arm. Results were inconclusive, with 90 RAR errors/100 000 order sessions overall.16 RAR error rates ranged from 8/100 000 (outpatient settings), to 185/100 000 order sessions (inpatient settings), particularly obstetrics and intensive care units (ICU).
Rationale
The rate of RAR errors in TCC is not yet known. This is germane given growing adoption of TCC, especially for nighttime monitoring of ICU patients.17
Specific aims
This study aims to assess the impact of restricting the number of concurrently open records on RAR errors in a TCC service.
METHODS
Context
Cleveland Clinic’s TCC service, eHospital, was deployed from May 2014 to December 2016 to 15 adult mixed and specialty ICUs in our satellite community and tertiary hospitals. This 7 p.m. to 7 a.m. service is staffed by intensivists, acute care nurse practitioners (NPs), and critical care nurses, all of whom (except the nurses) maintain bedside practices. On average, TCC NPs work 22 ± 3 nights/year, (physicians 11 ± 1 nights/year), in blocks of 3 or 4 consecutive nights.
The service delivery is similar to published TCC descriptions.18 Each team (1 intensivist, 1 NP, 2 nurses) monitors 100–150 patients, collaborating with bedside nurses and any in-house providers in monitored ICUs. Some ICUs had NPs, or residents, or intensivists, while a few had no in-house providers at night. The beds monitored and number and mix of in-house providers varied in some ICUs during the study. From 2017 to 2019, a second TCC team was added, limiting teams to 150 patients. Depending on clinical needs, workloads, site-specific workflows, and interaction with onsite providers, TCC intensivists, NPs or in-house providers wrote orders in the EHR.
eHospital uses a locally developed platform, with a proprietary risk stratification algorithm that interfaces with our EHR, EpicCare (EPIC Systems Corporation, Verona, WI).19 It features context-synchronization which, with 1 click, simultaneously presents multiple patient data streams across 5 monitors at each workstation (Figure 1). The first monitor is a filterable dashboard. It has color-coded tiles for patients and displays notifications, such as when bedside clinicians use the in-room button to initiate audio–visual (AV) calls to the TCC team. The second monitor displays the EHR. The third displays a real-time feed of bedside monitors. The fourth displays radiologic images, and the last is for live 2-way AV interaction. With context synchronization, the patient of interest’s record is the active tab on the EHR monitor.
Figure 1.
Depiction of a typical eHospital workstation.
Interventions
In September 2018, Cleveland Clinic restricted the allowable number of concurrently open records from 4 to 2. The study period (January 1, 2016 to May 15, 2020), spans time during TCC deployment and beyond the restriction.
Measures
Relevant measures, TCC orders and RAR errors, were abstracted from our EHR. RAR errors were defined as orders written for 1 patient which are subsequently canceled then reordered on a different patient within 10 minutes by the same clinician. A purpose-built query was used to identify specified medication and procedure orders (for imaging, laboratory tests, or patient care instructions). Multiple simultaneously entered wrong patient orders were considered 1 instance of error. These were part of an order group, multiorder set or treatment care path, which are placed during a single order entry session. Medication orders were considered separately from procedures, even if placed simultaneously. A combined rate without overlap was calculated. The test run of the query revealed few eHospital RAR errors relative to published rates for other inpatient clinical contexts.
Analysis
Statistical analysis was done in SAS software (version 9.4; Cary, NC). P values < .05 were considered significant. Categorical variables were summarized with count (%) and continuous variables with median [IQR]. Daily RAR rate was computed by dividing the number of RAR errors by the length (days) of the study periods. RAR errors/100 000 orders were also calculated before and after the EHR restrictions and compared using Chi-square test of proportions. Assuming a baseline RAR error rate of 90 per 100 000 orders, 342 728 orders will be needed in each period for 90% power to detect a 25% decrease in error rate. Adequate power depended on sufficient number of TCC orders and RAR errors.
Days between RAR events were analyzed using T statistical process control (SPC) chart for rare events. SPC charts analyze patterns of variation of current event data in relation to the center line (CL) (50th percentile for T Charts), and the upper and lower control limits (UCL, LCL), typically +/−3 standard deviations (SD) from the mean.20 The CL, UCL, and LCL are calculated from historical data. Hence, a sufficiently long preintervention period (at least 20–30 data points) is crucial to ensure a stable process for estimating the CL, UCL, and LCL. The UCL and LCL define the range within which data falls randomly, if a process is stable and without significant changes, implying no statistical difference between event data. Thus, SPC charts differentiate random from special cause variation induced by specific interventions. Special cause variation is present if any data fall outside the UCL or LCL. It is also detected using other decision rules about data patterns that are statistically unlikely to occur randomly.20
Ethical considerations
The study was approved by Cleveland Clinic’s Institutional Review Board, with waiver of informed consent as a retrospective, cohort study of patients monitored by eHospital. Provider data were analyzed in aggregate by provider type.
Results
Study data are summarized in Table 1. There were 156 318 and 122 587 TCC orders before and after the intervention, respectively. There were only 91 RAR errors equating to 36.5 and 27.7 errors/100 000 orders, respectively, which were not statistically different (P = .20). Post hoc analysis confirmed the analysis was underpowered to detect a difference in proportions that small. The 91 RAR errors translate to 0.058 RAR errors/day, both before and after the intervention as shown in Figure 2. The median number of days between RAR errors was 17.68. The number of orders placed by TCC providers varied, but annual census of monitored patients was fairly stable.
Table 1.
Data summary of study setting and RAR error variables
| Variable | Value |
|---|---|
| Number of Study Days before Intervention, N (%) | 981 (69.2%) |
| Number of Study Days after Intervention, N (%) | 575 (30.8%) |
| Number of RAR errors pre-intervention N (%) | 57 (62.6%) |
| Number of RAR errors post-intervention N (%) | 34 (37.4%) |
| Minutes between order and retract, median [IQR] | 0 [0; 2] |
| Number of Days between RAR, median [IQR] | 17.68 [12; 27] |
| Order type, N (%) | |
| Medication | 82 (90.1%) |
| Procedure | 9 (9.89%) |
| Number of RAR errors by provider type, N (%) | 28 (30.8%) |
| Physicians | 63 (69.2%) |
| Nurse practitioners | |
| Annual census of monitored patients (N) | 20 826 |
| 2016 | 20 288 |
| 2017 | 20 379 |
| 2018 | 17 709 |
| 2019 | 5895 |
| 2020 (Year till 05/15/2020) | |
| Annual count of TCC orders | 32 428 |
| 2016 | 58 848 |
| 2017 | 65 042 |
| 2018 (pre restriction) | 18 834 |
| 2018 (post restriction) | 76 482 |
| 2019 | 27 271 |
| 2020 (YT 05/15/2020) |
Figure 2.
Graphical depiction of RAR event patterns over time. The median number of days between RAR errors was 17.68.
Figure 3 displays the T SPC chart of RAR event patterns, depicting days between events, in relation to the temporal change in number of concurrently open records, CL (value, 9.5 days), and UCL line (value, 187 days). The days between RAR errors vary randomly around the CL after the restriction. No data point even approaches the UCL. There are no discernible trends in the pattern of days between errors that match other decision rules for assignable variation. An effective intervention would have significantly increased days between events. Accordingly, no significant change in RAR errors occurred after the EHR restrictions.
Figure 3.
T control chart of days between RAR errors. There are no discernible trends in the pattern of the number of days between RAR errors.
DISCUSSION
Summary
We assessed the impact of restricting the number of concurrently open patient records from 4 to 2 in a TCC setting. The rate of RAR errors was not significantly reduced. This concurs with other settings (ED, inpatient cardiology) at our institution and is congruent with similar studies elsewhere. To the best of our knowledge, this is the first study addressing this in TCC.
Interpretation
To facilitate comparison with prior studies, we analyzed the rate of RAR errors/100 000 orders. However, that was underpowered to detect a difference owing to the infrequency of RAR errors in our TCC setting. We also analyzed days between errors using T charts: These are well-suited for rare events, having sensitive decision rules that do not necessarily require voluminous data or long periods, to detect data patterns consistent with trends towards improvement or deterioration. With rare events, such trends may need extended periods to accumulate sufficient data for statistical tests of proportions.
The infrequency of eHospital RAR errors merits consideration given that it seemingly differs from other inpatient settings. Incorporating patient photos in the EHR 8 months before the intervention, if effective, would have reduced RAR errors and enhanced the impact of subsequent EHR restrictions. Yet the photos were small and non-resizable, and we observed no difference in TCC RAR errors throughout the study. Further, our TCC clinicians also work in bedside settings, so our low error rates cannot be attributed to using exclusive personnel. Further, when monitored patient volume increased, we added another team to maintain consistent team–patient ratios. However, there are no reports yet from other TCC programs for comparison. If other programs find similarly low rates, this implies there may be features inherent to the TCC environment (perhaps setup, platform features, or workflows) that reduce the risk of RAR errors. If so, we speculate that context-synchronization may mitigate the risk by ensuring the streams of information displayed correspond to the patient with the active tab on the EHR monitor.
The foregoing suggests that solving wrong patient selection may lie in the realm of human factors. Sopan et al focused on human–computer interactions, user interface, and workflow design when devising mitigation strategies, including verification steps after patient selection and before order finalization.9 Interposing steps adds friction to workflows, is time-consuming, and may increase dissatisfaction with CPOE. They did recommend passive features, including patient pictures, prominent name displays, and colocating order finalization with patient names/photos to facilitate confirmation without intrusive steps. Optimal solutions should consider clinician efficiency and patient safety. These should not be mutually exclusive.
Limitations
Our study has limitations inherent to single center retrospective studies beyond its limited statistical power. In particular, our proprietary eHospital platform is not commercially available, and other TCC platforms may not feature context synchronization. In-house providers in monitored ICUs may differ elsewhere, so, although our staffing model corresponds with published reports, our volume of TCC orders and relative infrequency of RAR errors may not be generalizable. Further, we could not evaluate associations between risk of RAR errors and relevant factors, such as rates by provider type. Our TCC team collectively discusses patient issues, with necessary orders placed by either the physician or NP. Consequently, physician opportunities for error are fewer and not easily quantified. Therefore, we may have overlooked confounding variables that may have influenced our results.
CONCLUSION
The adoption of EHRs with CPOE has unintentionally increased wrong patient selection errors with risk for harm. Limiting the number of concurrently open records has been tried in order to reduce these errors, but studies using the RAR metric have not found benefit. Likewise, we found that restricting the number of open records from 4 to 2 in our TCC setting did not reduce RAR errors. Similar studies in other TCC programs are needed to confirm our findings. RAR errors were comparatively infrequent in our TCC service, which has user interface and workflow features unlike standard clinical workstations. This suggests there may be opportunities to optimize the EHR user interface to reduce wrong patient selection errors.
FUNDING
No funding was received for this article.
AUTHOR CONTRIBUTIONS
All authors have contributed to all parts of the research, including conceptualization, data analysis, and article preparation and review.
ACKNOWLEDGMENTS
The authors wish to recognize with gracious appreciation, the dedicated physicians, nurse practitioners, and nurses who deliver expert care nightly in the TCC operations center.
DATA AVAILABILITY
The data underlying this article cannot be shared publicly due to privacy of the healthcare providers whose clinical activity data was retrospectively analyzed. The data will be shared upon reasonable request to the corresponding author.
CONFLICT OF INTEREST STATEMENT
None declared.
REFERENCES
- 1. Agrawal A, Aronson JK, Britten N, Ferner RE, de Smet PA, et al. ; Members of Emerge EMERG. Medication errors: problems and recommendations from a consensus meeting. Br J Clin Pharmacol 2009; 67 (6): 592–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Velo GP, Minuz P.. Medication errors: prescribing faults and prescription errors. Br J Clin Pharmacol 2009; 67 (6): 624–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Pontefract SK, Coleman JJ, Vallance HK, et al. The impact of computerised physician order entry and clinical decision support on pharmacist-physician communication in the hospital setting: a qualitative study. PLoS One 2018; 13 (11): e0207450. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Koppel R, Metlay JP, Cohen A, et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA 2005; 293 (10): 1197–203. [DOI] [PubMed] [Google Scholar]
- 5. Ash JS, Berg M, Coiera E.. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc 2003; 11 (2): 104–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Ogrinc G, Davies L, Goodman D, Batalden P, Davidoff F, Stevens D.. SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence): revised publication guidelines from a detailed consensus process. Am J Med Qual 2015; 30 (6): 543–9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Koppel R, Leonard CE, Localio AR, Cohen A, Auten R, Strom BL.. Identifying and quantifying medication errors: evaluation of rapidly discontinued medication orders submitted to a computerized physician order entry system. J Am Med Inform Assoc 2008; 15 (4): 461–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Adelman JS, Kalkut GE, Schechter CB, et al. Understanding and preventing wrong-patient electronic orders: a randomized controlled trial. J Am Med Inform Assoc 2013; 20 (2): 305–10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Sopan A, Plaisant C, Powsner S, Shneiderman B.. Reducing wrong patient selection errors: exploring the design space of user interface techniques. AMIA Annu Symp Proc 2014; 2014: 1056–65. [PMC free article] [PubMed] [Google Scholar]
- 10. Wilcox AB, Chen Y-H, Hripcsak G.. Minimizing electronic health record patient-note mismatches. J Am Med Inform Assoc 2011; 18 (4): 511–4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Green RA, Hripcsak G, Salmasian H, et al. Intercepting wrong-patient orders in a computerized provider order entry system. Ann Emerg Med 2015; 65 (6): 679–86.e1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Adelman JS, Berger MA, Rai A, et al. A national survey assessing the number of records allowed open in electronic health records at hospitals and ambulatory sites. J Am Med Inform Assoc 2017; 24 (5): 992–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Kannampallil TG, Manning JD, Chestek DW, et al. Effect of number of open charts on intercepted wrong-patient medication orders in an emergency department. J Am Med Inform Assoc 2018; 25 (6): 739–43. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Canfield C, Udeh C, Blonsky H, Hamilton AC, Fertel BS.. Limiting the number of open charts does not impact wrong patient order entry in the emergency department. J Am Coll Emerg Phys Open 2020; 1 (5): 1071–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Blonksy H, Shirley R, Milinovich A, et al. Limiting the number of open charts in the electronic medical record does not decrease order entry errors in cardiovascular inpatient and intensive care units. Circulation 2019; 140 (suppl 1): A14696. [Google Scholar]
- 16. Adelman JS, Applebaum JR, Schechter CB, et al. Effect of restriction of the number of concurrently open records in an electronic health record on wrong-patient order errors: a randomized clinical trial. JAMA 2019; 321 (18): 1780–7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Kahn JM, Le TQ, Barnato AE, et al. ICU telemedicine and critical care mortality: a national effectiveness study. Med Care 2016; 54 (3): 319–25. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Fortis S, Goede MR. . Structure and design of the tele-ICU. In: Koenig M, ed. Telemedicine in the ICU. Cham, Switzerland: Springer; 2019: 107–28. [Google Scholar]
- 19. Udeh C, Udeh B, Rahman N, Canfield C, Campbell J, Hata JS.. Telemedicine/virtual ICU: where are we and where are we going? Methodist Debakey Cardiovasc J 2018; 14 (2): 126–33. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Tague NR. The Quality Toolbox. 2nd ed. Milwaukee, WI: ASQ Quality Press; 2005: 584. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The data underlying this article cannot be shared publicly due to privacy of the healthcare providers whose clinical activity data was retrospectively analyzed. The data will be shared upon reasonable request to the corresponding author.