Evaluation of Electronic Medical Record Downtime in a Busy Emergency Department

Abstract

Objectives: This study aimed to investigate electronic medical record (EMR) implementation in a busy urban academic emergency department (ED) and to determine the frequency, duration, and predictors of EMR downtime episodes.

Materials and Methods: This study retrospectively analyzed data collected real time by the EMR and by the operations group at the study ED from May 2016 to December 2017. The study center has used the First Net Millennium EMR (Cerner Corporation, Kansas City, Missouri, USA). The ED operations data have been downloaded weekly from the EMR and transferred to the analytics software Stata (version 15MP, StataCorp, College Station, Texas, USA).

Results: During the study period, 12 episodes of EMRD occurred, with a total of 58 hours and a mean of 4.8 ± 2.7 hours. The occurrence of EMRD event has not been associated with on-duty physician coverage levels (p = 0.831), month (p = 0.850), or clinical shift (morning, evening, or night shift) (p = 0.423). However, EMRD occurrence has been statistically significantly associated with weekdays (p = 0.020).

Discussion: In a real-world implementation of EMR in a busy ED, EMRD episodes averaging approximately 5 hours occurred at unpredictable intervals, with a frequency that remained unchanged over the first 20 months of the EMR deployment.

Conclusion: The study could define downtime characteristics at the study center. The EMRD episodes have been associated with inaccuracies in hourly census reporting, with a rebound phenomenon of over-reporting in the first hour or two after restoration of EMR operations.

Introduction

Widespread rapid increase of electronic medical records (EMRs) has many advantages for the Emergency Department (ED). However, when EMR downtime (EMRD) occurs, potential adverse operational and patient-outcome impacts also occur.^1–6 EMRD preparation is a well-known priority for ED operations planners.^1,5 Contingency planning for EMRD is facilitated by understanding the frequency and duration of EMRD episodes. The current study had a primary goal of characterizing EMRD at a busy academic center, to determine the frequency of EMRD episodes and their duration. When the study center transitioned to the EMR, relatively few data are available on predicted EMRD rate or duration, other than information provided by the company marketing the EMR.

The particular company marketing the EMR at the study center provides information on its website (https://www.cerner.com/gb/en/solutions/hosting-monitoring), stating that for hosted solutions (i.e., the data servers are housed at the EMR company), the downtime is predicted at 0.02% of total hours. This information, not published in the peer-reviewed evidence base, suggests a best-case expected EMRD rate of 2 in 10,000 hours. The authors were unable to find a peer-reviewed report of actual expected downtime in a busy system that did not use a hosted solution. Therefore, the current study aimed to provide a real-world estimate of EMRD in a setting wherein the servers were (by national law) unable to be sited at the EMR company.

We aimed to assess the first 20 months of EMR experience in a busy urban academic ED to determine the frequency and duration of EMRD episodes. The primary objective was to assess whether EMRD occurs more likely at certain times of day, days of week, or census levels. A secondary objective was characterization of EMR reporting the number of new patients registered at the ED during a given hour from the peri-EMRD time frame.

Materials And Methods

Study setting and population

This study was an analysis of data collected in real time by the EMR and by the operations group at the study ED. There was no patient contact, and the study was noninterventional. Institutional ethics approval was obtained from the study medical center. The study was conducted at the major teaching and largest tertiary care center (Hamad General Hospital) in the State of Qatar. Serving the national population of approximately 3 million, the center's ED serves as the only receiving hospital for major illness or injury, and the study center ED sees >400,000 cases annually.

The study center uses the First Net Millennium EMR (Cerner Corporation, Kansas City, Missouri, USA). The system was implemented on May 6, 2017, defined in this study as “EMR Day 1.” At the study center, we recorded EMRD episodes in real time, with the start time being the time of initial call to ED operations group and/or the information technology team that the EMR is not functional (EMR “slowdown” episodes do not count toward EMRD episodes in this study). We reported EMRD conclusion in the database as the time the EMR issue is resolved.

Data included in the study covered the 713,955 ED cases presenting during the 20-month period from May 2016 to December 2017. This assessment of operations data has no exclusion criteria.

Data collection

The EMR served as the main source for data. At the study center, the EMR database encompasses all of the operations data used in this study, as outlined in previous analyses.^7,8 In brief, the ED operations data for this study was downloaded from the EMR and transferred to the analytics software Stata (version 15MP, Stata Corp, College Station, Texas USA). This study used Stata for all analysis and graphics.

Operational definitions

Multiple terms used at the study center may be used differently than they are used elsewhere. These terms, as well as some terms specific to the study methodology, are clarified in this section.

At the study ED, hourly patient census refers to the number of new patients registering at the ED during a given hour. The hourly census is thus reflective of the number of new patient presentations (not the total number of patients in the ED at a given hour). In the study ED, the hour during which patients arrive is classified into one of the day's 24 circadian hours. An event (e.g., patient arrival) during the time frame from midnight to 00:59 is said to occur during circadian hour 1. Events during 01:00–01:59 occur during hour 2, and so on until hour 24 reflection of events occurring at 23:00–23:59.

Thus, this study used the circadian hour terminology, and the EMRD hours refer to the circadian hours that were involved with EMRD. If EMRD occurred during any part of a circadian hour, that hour was classified as an EMRD hour. For example, an EMRD episode starting a half-hour after midnight and lasting 45 minutes would be classified as consisting of 2 EMRD hours (the day's circadian hours 1 and 2). Interpretation of the study's reported “hours of involvement with EMRD” should be guided by the fact that the study center calculations of EMRD involvement hours cannot be simply translated into minutes of downtime. The peri-EMRD time frame is defined at the study center as the 6 hours leading up to EMRD, the hours of EMRD, and the 6 hours following EMRD. With six preEMRD and six post-EMRD hours, the minimum duration of peri-EMRD time (i.e., with an EMRD episode lasting 1 hour) is thus 13. Figure 1 shows a schematic of the peri-EMRD time frame as used at the study site.

Figure 1.

Study definition of time frame surrounding and including electronic medical record downtime (EMRD).

For purposes of assessing whether EMRD was more likely to occur at particular times of the day or days of week, the EMRD start hour was defined as the initial circadian hour of involvement with an EMRD episode. Similarly, the EMRD start weekday was defined as the day of the week on which the EMRD episode commenced. The inter-EMRD interval was defined as the number of days between ordinal episodes of EMRD. For example, the initial episode of EMRD occurred on EMR day 38, and the second EMRD episode occurred on EMR day 84; the corresponding inter-EMRD interval was 46 days.

Constitution of control group hourly census for EMRD hourly census analysis

The study's secondary endpoint was assessment for signs of inaccuracy in EMR-reported hourly census in the peri-EMRD time frame (Figure 2). This analysis required a “control group” of hourly census estimates that matched, in terms of census-predicting variables, each EMRD episode's 6 preEMRD, n EMRD, and 6 post-EMRD hours. The pool from which the control hourly census data were drawn was the 20-month study period (May 2016 to December 2017). We recorded the EMRD episode hours with respect to circadian hour, day of week, and Ramadan status. Subsequently, we selected nonEMRD hours in the study period to match the EMRD hours. The number of occurrences of a particular circadian hours and days of the week in the control pool accounted for a slight variation in the size of any given peri-EMRD hour's control group. The onduty physician coverage levels between the peri-EMRD hours and nonEMRD hours also remained the same.

Figure 2.

Method for generating electronic medical record downtime (EMRD) and control hourly census.

Data Analysis

Descriptive analysis for categorical variables used proportions, with 95% exact confidence intervals (CIs). We reported one-sided 97.5% CIs when a proportion's point estimate was 0%. For continuous or ordinal variables, the initial step in descriptive analysis was normality testing, executed with the Shapiro–Francia test (which can be used when n>10). For normally distributed data, we reported the central tendency and dispersion as mean (with 95% CI) ± standard deviation. For nonnormal data, we reported central tendencies as medians; for these data, 95% CI for the median was calculated using Stata's centile procedure (a binomial method that makes no-normality assumption).

With regard to hourly census tendencies during peri-EMRD vs. control time frames, the study focused on percentage change rather than absolute change. This standardized the estimates of influence of peri-EMRD status on hourly census, so those estimates could be applied regardless of actual circadian hour census tendencies. Statistical testing used p < 0.05 to define significance. Significance was also adjudicated by an overlap of a point estimate's CI with a null value. Univariate analysis for categorical variables used the chi-squared test. We used the Fisher exact test when any cell value was < 5.

We used graphing techniques to complement the exploratory hourly census analysis. Each point in the peri-EMRD period had its median hourly census plotted along with the median census from the control nonEMRD hours. Stata's default bandwidth (0.8) generated locally weighted scatter plot smoothing (lowess) trend lines for the preEMRD, EMRD, and post-EMRD time frames to assess (for hypothesis generation) whether any trends toward peri-EMRD census changes in the hourly census data appear. Post-hoc tests will be incorporated to confirm the difference between the groups.

Results

Frequency, duration, and timing of EMRD episodes

During the study period, 12 EMRD episodes occurred, with a total of 58 hours, accounting for 0.40% (95% CI, 0.30%–0.52%) of the study period's total (14,496) hours of ED operations. Stated differently, EMRD affects 4 hours (95% CI, 0.1–22) for every 1000 hours of ED operations. We found no differences between the peri-EMRD hours and nonEMRD hours with respect to the onduty physician coverage levels (p = 0.831).

We illustrated the occurrence days, circadian hours, and durations of the 12 EMRD episodes (Figure 3). One day (EMR day 157) had two separate episodes of downtime, occurring 16 hours apart. Two other EMRD episodes crossed the midnight hour and thus involved two calendar days.

Figure 3.

Occurrence and duration of electronic medical record downtime (EMRD) episodes over 20 months.

The 12 EMRD episode durations had normal distribution (Table 1). We found that the mean EMRD duration was 4.8 ± 2.7 hours, and the 95% CI for the mean was 3.1–6.5 hours. Nonparametric trend testing identified no association (p = 0.453) between EMRD episode duration and earlier vs. later occurrence within the set of 12 episodes.

Table 1.

Occurrences of electronic medical record downtime (EMRD): Timing and duration.

EMRD episode	Study day (Day 1 = 6 May 2017)	Calendar timing	EMRD initial circadian hour	Number of circadian hours affected by EMRD
1	38	Sun, Jun ‘16	11	3
2	84	Thu, Jul ‘16	17	7
3	157	Sun, Oct ‘16	1	4
4	157	Sun, Oct ‘16	20	6
5	175	Thu, Oct ‘16	19	2
6	293	Wed, Feb ‘17	10	11
7	426	Wed, Jul ‘17	10	5
8	427	Thu, Jul ‘17	10	6
9	444	Sun, Jul ‘17	23	3
10	459	Mon, Aug ‘17	18	6
11	468	Wed, Aug ‘17	9	4
12	510	Wed, Sep ‘17	7	1

We calculated the inter-incident interval between EMRD episodes as the number of days that elapsed between EMRD incidents. We noted 11 inter-incident intervals with 12 EMRD episodes. The inter-incident intervals ranged from 0 to 133 and were not normally distributed. The median (interquartile range [IQR]) was 17 (9–73) days, but the estimate for the median had a broad 95% CI of 11.5–73.9 days. Nonparametric trend testing identified no change (p = 0.420) in the inter-incident intervals with earlier vs. later occurrences during the study period.

EMRD occurrence association with time of day, weekday, or calendar month

Over the 20-month period, the months from May to December occurred twice; the months from January to April occurred once. When analyzed for a trend over the 20 months of the study, EMRD episodes showed no significant association between the months, and the likelihood of episodes changed over the 20-month time frame (p = 0.850). In the assessment of the 12 × 2 table of calendar months vs. EMRD occurrence, the p value was not significant (p = .175).

We observed a statistically significant association between weekdays and EMRD occurrence (p = 0.020). Post-hoc testing, performed for hypothesis generation, combined the three weekdays (Sunday, Wednesday, and Thursday) with multiple EMRD episodes and found that these three weekdays were significantly (p = 0.001) more likely to encounter EMRD than other weekdays. We found no association (p = 0.423) between EMRD and the clinical shift (morning, evening, or night).

Hourly census data reported from the EMR in the peri-EMRD time frame

We found no difference in the mean percent change in hourly census during comparison of pre and post-EMRD time frames; the mean difference in percent changes was 1.1 (95% CI, − 7.2 to 9.5; p = 0.787). However, we observed a significant difference in percent change during comparison of the EMRD time frame the with combined group of pre- and post-EMRD time frames (p < 0.0001). Compared with the combined group of hours before and after EMRD, the EMRD hour percent change from control hourly census was − 19.1% (95% CI, − 11.0 to − 27.2%) (Table 2).

Table 2.

Twelve episodes of electronic medical record downtime (EMRD): Median percent change in hourly census at each time point in the peri-EMRD time frame compared with the control hourly census.

Peri-EMRD time point (# episodes contributing data)	Mean ± standard deviation of % change in hourly census from control to peri-EMRD	95% CI for mean % change in hourly census
EMRD − 6 (12)	2.2 ± 22.0	− 11.8 to 16.2
EMRD − 5 (12)	5.9 ± 30.0	− 13.2 to 24.9
EMRD − 4 (12)	8.3 ± 37.;8	− 15.7 to 32.3
EMRD − 3 (12)	1.3 ± 18.4	− 10.3 to 13.0
EMRD − 2 (12)	3.1 ± 21.8	− 10.7 to 17.0
EMRD − 1 (12)	− 15.8 ± 31.4	− 35.8 to 4.2
EMRD hour 1 (12)	− 33.9 ± 29.2	− 52.4 to − 15.3*
EMRD hour 2 (11)	− 27.0 ± 27.5	− 45.5 to − 8.6*
EMRD hour 3 (10)	− 16.6 ± 29.0	− 37.3 to 4.2
EMRD hour 4 (8)	− 13.8 ± 30.1	− 38.9 to 11.4
EMRD hour 5 (6)	− 17.4 ± 27.6	− 46.3 to 11.5
EMRD hour 6 (5)	− 6.7 ± 12.4	− 22.2 to 8.7
EMRD hour 7 (2)	Only 2 data points: − 3.8 and 25.5	−
EMRD hour 8 (1)	Only data point: 30.8	−
EMRD hour 9 (1)	Only data point: 13.7	−
EMRD hour 10 (1)	Only data point: 1.9	−
EMRD hour 11 (1)	Only data point: 26.3	−
EMRD hour 1 (12)	15.4 ± 34.0	− 6.2 to 37.0
EMRD hour 2 (12)	6.2 ± 14.3	− 2.9 to 15.3
EMRD hour 3 (12)	− 5.8 ± 14.7	− 15.1 to 3.5
EMRD hour 4 (12)	0.2 ± 18.9	− 11.8 to 12.1
EMRD hour 5 (12)	− 0.1 ± 22.2	− 14.2 to 14.0
EMRD hour 6 (12)	− 4.0 ± 23.5	− 18.9 to 11.0

*95% CI for mean change does not include null value of 0%.

For the preEMRD time frame, none of the time points’ 95% CIs fell entirely outside the null value of 0%. However, the lowess line does suggest a downslope – a drop in the preEMRD hourly census compared with the control census – in the last 1–2 hours before EMRD. Data were insufficient to explore whether the possible downturn in the lowess line was statistically significant (Figure 4).

Figure 4.

Electronic medical record downtime (EMRD): EMR-reported census in the peri-EMRD time frame – Per-episode (each red x) hourly census data with locally weighted scatterplot smoothing trend lines.

A nonparametric trend line of hourly census in the EMRD period that included only the episodes with at least 6 hours of downtime identified a significant (p = .042) upward trend in percentage difference between the control and EMRD hourly census.

For the post-EMRD time frame, none of the time points’ 95% CIs fell entirely outside the null value of 0%. The lowess line does suggest that the initial 1–2 hours post-EMRD may be associated with an uptick in census (compared with the control census) that normalizes within a few hours post-EMRD. However, similar to the case with the suggestion of preEMRD findings just before EMRD, the possible trend involved too few data points to allow meaningful analysis in this study.

Discussion

The study could define downtime characteristics at the study center; however, it could not find any association or predictors for downtime. However, we observed statistical significance between weekdays. No circadian or calendar month associations were found to predict EMRD, and the hourly census data did not indicate any association between high-census ED status and EMRD.

The incorporation of EMRs into daily practice in EDs worldwide has had a major impact on clinical care and efficiency. With any EMR, however, comes the rare but inevitable occurrence of EMRD. Studies have shown that even when EMRD is uncommon, end-user perception, and actual patient-safety events suggest an insufficient EMRD preparation.^1,9 Therefore, EMRD preparation is considered an area warranting ongoing emphasis and improvements in operations planning.^2–4,10

Unexpected downtimes related to EMRs appear to be fairly common among institutions. When planning the implementation of a new EMR, leaders must plan for the awareness of the needs during unplanned downtime. End users are in direct contact with the information system and are responsible for documenting patient clinical information. End users perceive that they did not have adequate training or information about how to go through the downtime event. Training or education for healthcare staff may help resolving these events. Ensuring health information technology-related events are resolved and incorporate effective solutions should be a continued focus area for healthcare systems.

At the study center, the EMR is an enterprise solution, and thus the EMRs’ state of utilization is impacted by many nonED users. Failure to account for the load on the EMR from these other users (e.g., clinics, inpatient services) is a likely contributor to this report's inability to identify predictors of EMRD. Follow-up study will attempt to identify these contributing factors to EMRD to express and possibly improve risk assessment for EMRD.

The hourly census data found a profound depression in the reported hourly census during the first few hours of EMRD in the absence of any preEMRD census changes or differences in patient or ED-coverage characteristics. A dip in reported (vs. control) census appeared in the 1–2 hours before EMRD, with a post-EMRD period of 1–2 hours required for normalization of EMR-reported hourly census. Therefore, at the study center, actual ED volume drops precipitously just before EMRD, stays depressed during the initial EMRD hours, and rebounds even to above historical levels in the first few post-EMRD hours before stabilizing. However, the trends in this report's Table 2 and Figure 4 are quite suggestive. Although they require a confirmatory assessment in a follow-up study, a hypothesis can be made based on their explanation.

The most likely explanations for the data in Table 2 and Figure 4 are a combination of EMR and human error events. EMR “slowdown” probably occurred in the hour or two before a full EMRD episode. During the EMRD hours, patients are not registered as having arrived, and the patients are simply logged in as arriving in the initial post-EMRD hours in the post-EMRD initial hours, rather than back-time arrivals with questionable accuracy. The likelihood of this course of events is increased by the finding that ED volume appears to be increased (by the lowess results) during the initial hour or two of EMR recovery post-EMRD. This report's findings of EMRD and post-EMRD hourly census changes showed that it is likely more a function of human error (in failing to properly back-time ED arrival data) rather than EMR failure.

This study has several limitations. First, this analysis did not time EMRD to the minute. However, the measurement differences are not profound, and the study finding that EMRD affected 4 of every 1000 operating hours stands in stark contrast to the prestudy expectation that EMRD should affect approximately 0.2 per 1000 hours.

The study center should plan to incorporate an estimate that the mean EMRD duration is approximately 5 hours. The estimate for EMRD shares an important shortcoming with many other estimates in this report: imprecision and broad CIs. The EMRD duration estimate from this report ranged from 3.1 to 6.5 hours, and further assessment will be necessary to improve precision of this important parameter.

Study power limitations related to the low number of EMRD episodes accounted for the low study power to definitively identify circadian or calendar-related predictors of EMRD.

At the study center, the EMR is an enterprise solution, and thus the EMRs’ state of utilization is impacted by many nonED users. Failure to account for the load on the EMR from these other users (e.g. clinics, inpatient services) is a likely contributor to this report's inability to identify EMRD predictors. A follow-up study will attempt to identify these contributing factors to EMRD to visualize a more complete picture and possibly improve risk assessment for EMRD.

Conclusions

In a real-world EMR implementation in a busy ED, EMRD episodes averaging approximately 5 hours occur at unpredictable intervals, with a frequency that remained unchanged over the first 20 months of the EMR deployment. The EMRD episodes were possibly associated with inaccuracies in hourly census reporting, with suggestion of decreased reporting of hourly census just before and during EMRD, with a rebound phenomenon of over-reporting in the first hour or two after restoration of EMR operations. Future research should be conducted to identify the contributing factors causing downtime and improve the risk assessment for EMRD.

Funding

None.