Abstract
Objective
Electronic health record (EHR) implementation can improve care, but may also adversely impact emergency department (ED) efficiency. We sought to examine how a custom, ED provider electronic documentation system (eDoc) – which replaced paper documentation –affected operational performance.
Methods
We analyzed retrospective operational data for 1-year periods before and after eDoc implementation in a single ED. We computed daily operational statistics, reflecting 60,870 pre- and 59,337 post-implementation patient encounters. The pre-specified primary outcome was daily mean length of stay (LOS); secondary outcomes were daily mean length of stay for admitted (LOSa) and discharged patients (LOSd) and daily mean arrival time to disposition for admitted patients (TTD). We used a pre-specified multiple regression model to identify differences in outcomes while controlling for pre-specified confounding variables.
Results
The unadjusted change in LOS was +8.4 minutes; unadjusted changes in secondary outcomes were: 1) LOSa +11.4 minutes, 2) LOSd +1.8 minutes and 3) TTD +1.8 minutes. Using a pre-specified regression analysis to control for variations in operational characteristics, there were significant increases in LOS (+6.3 minutes [95%CI 3.5,9.1]) and LOSd (+5.1 minutes [95%CI 1.9,8.3]). There was no statistically significant change in LOSa or TTD.
Conclusions
In our single center study, the isolated implementation of eDoc was associated with increases in overall and discharge length of stay. Our findings suggest that a custom-designed electronic provider documentation may negatively affect ED throughput. Strategies to mitigate these effects – such as reducing documentation requirements or adding clinical staff, scribes, or voice recognition – would be a valuable area of future research.
Introduction
Background
In 2009, the federal government enacted a national incentive program to encourage the adoption of electronic health record systems (EHRs).1 Under this federal “meaningful use” program, adoption of EHRs has increased rapidly.2–4 Consistent with overall trends in health care, EHR adoption in emergency departments (EDs) has swiftly expanded from 46% in 2006 to 84% in 2011.5 However, the benefits of the federal incentive program have also been questioned because health information technology has contributed to inefficiencies, introduced unintended consequences, and only started to achieve promised benefits.6, 7 The American Medical Association has expressed particular concern with lack of EHR usability and called for delaying meaningful use implementation.8
Importance
Given the time-sensitive nature of emergency care, inefficient EHRs have the potential to impact ED throughput and quality of care. Previous evidence on the impact of EHRs is mixed – multiple studies have suggested that EHR implementation can improve ED efficiency9–13 and quality of care.14, 15 However, others have found that EHR implementation may have a neutral or negative impact, potentially increasing provider documentation time and patient length of stay.16–22 This mixed literature reflects heterogeneous EHR implementations at varying clinical sites and does not isolate the impact of individual EHR features from one another (e.g. patient tracking, computerized provider order entry (CPOE), and provider documentation) compared to paper-based documentation. Provider documentation is perhaps the most time-consuming component of EHR use.16, 17, 23
Goals of This Investigation
At our institution, we designed, built, and implemented a custom provider electronic documentation system (eDoc) to replace paper documentation in the setting of existing, well-established electronic patient tracking and CPOE systems, providing a unique natural experiment that could isolate the impact of replacing paper-based documentation with an electronic provider documentation system. The objective of our study was to examine the effect of implementing eDoc on ED efficiency as measured by daily mean ED length-of-stay eight weeks and one year before and after implementation. We initially hypothesized that implementation would result in transient increases in length-of-stay as providers learned the new system, but that over a one-year period it would have a neutral effect on or reduce length-of-stay.
Methods
Study Design & Setting
This study was a retrospective analysis of operational data obtained from Brigham & Women’s Hospital ED, a 43-bed, urban, academic ED in Boston, MA with an annual volume of approximately 60,000 patients. We had robust, custom-developed electronic patient tracking and CPOE systems in place. All order entry during the study period was performed using the exisiting CPOE system. Patient tracking (including admission and discharge times) was performed via the existing electronic tracking system throughout the entire study period.
Before the implementation of eDoc, provider documentation was completed on paper by residents and physician assistants (PAs) – the paper documentation template is shown in Appendix A. Completed paper documentation was scanned into our hospital EHR after the ED encounter by health information management staff. Attending physicians documented using traditional phone dictation which was transcribed by professional transcriptionists and electronically transferred into our EHR.
Our institution developed eDoc to work with our existing electronic ED patient tracking and CPOE systems. The eDoc system was custom-built based on the input of health IT experts and emergency medicine clinicians. An interdisciplinary team of attending and resident physicians, health information management, and information systems professionals led by a dually trained emergency physician/clinical informatician (AL) iteratively designed the electronic documentation system to meet workflow, quality of care, legal, and billing compliance requirements. The team designed all system components including data elements, workflows, and output producing detailed design specifications. An internal team of software developers built the system according to these specifications. Appendix B includes sample screenshots of the eDoc system.
Following implementation of eDoc on March 18, 2013, resident physicians and PAs entered documentation electronically. Attending physicians had the option of typing their notes in eDoc or using a real-time voice recognition tool that transcribed speech into text in eDoc (Speech Anywhere 360 Direct, Nuance Communications, Inc., Burlington, MA). Handwritten provider documentation and phone dictation were eliminated.
We compared daily operational data for a one-year pre-implementation period from March 18, 2012 to March 17, 2013 and the one-year post-implementation period from March 18, 2013 to March 17, 2014. We selected this study period 1) to ensure that our analysis would adequately capture effects outside of any immediate adjustment period to the new system; and 2) to allow comparison between similar timeframes given that their can be significant seasonal variability in ED utilization. All recorded patient encounters during the designated study period were included. To assess for any short-term effects of implementation on the outcome variables, we created a subset of these data, for 8-week periods before (1/21/2013 – 3/17/2013) and after implementation (3/18/2013 – 5/12/2013).
No other major information technology implementation projects, work flow changes or changes to staff coverage occurred during the study period. Further, none of the metric definitions or collection methods were influenced by eDoc implementation; these metrics were recorded by the electronic patient tracking system that remained in place during the entire study period. The study protocol was approved by the Partners Healthcare Institutional Review Board (IRB).
Methods and Measures
We obtained operational ED data for all patient visits including patient medical record number (MRN), gender, date of birth (DOB), arrival date and time, bed request date and time for admitted patients, discharge date and time for discharged patients, emergency severity index (ESI; on a scale of 1–5, with 1 reflecting patients with the highest level of acuity), mode of arrival (ambulance or other) and disposition (inpatient admission, ED observation, home or other). All protected health information was removed from the data set and each patient encounter was assigned a unique, randomly-generated identification number.
Length of stay for all patients and boarding time for admitted patients was derived from individual encounter data. Individual length of stay was defined as the recorded ED exit time minus the recorded arrival time. This included all patient encounters: admitted patients, discharged patients and cases in which patients left AMA (‘against medical advice’) or eloped from the ED. In addition, boarding time for all admitted patients was calculated based on the ED exit time minus bed request time. Time to disposition for admitted patients was calculated as the bed request time minus arrival time.
Daily descriptive statistics were then calculated for the full study period, yielding 730 total data points for final regression analysis. Patient data was included in daily totals based on the arrival date to the ED. Daily-level variables included month and day of visit, total daily visits, total visits for the previous day, mean patient age, count and proportion of female patients, count and proportion of admissions, count and proportion of ED observation admissions, count and proportion of discharges, proportion of patients with ESI of 2 or 1 (highest acuity), weighted mean ESI, count and proportion of patients of each ESI category (1 through 5), count and proportion arriving by ambulance, count and proportion of arrival by mode other than ambulance, mean time to disposition for admitted patients (TTD), mean daily length of stay (LOS), median daily LOS, mean LOS for admitted patients (LOSa), mean LOS for discharged patients (LOSd), total boarding time, and mean boarding time for admitted patients. Each outcome variable was calculated as an average per day providing 112 values for the 8-week period analysis (56 pre- and 56 post-implementation) and 730 values for the one-year periods (365 pre- and 365 post-implementation).
The proportion of high acuity patients (emergency severity index [ESI] 2 or 1) was calculated by the sum of patients with ESI of 1 or 2 divided by the total number of daily visits. Weighted mean ESI was calculated by multiplying the number of patients per ESI category by ESI number and dividing by the total patient visits. Total patient visits for the previous day was calculated based on the previous day's total visits and used as a proxy measurement of patient backlog in the ED that might affect department efficiency.
Outcomes
The pre-specified primary outcome variable was daily mean length-of-stay (LOS). The secondary outcomes included daily mean length-of-stay for admitted (LOSa) and discharged patients (LOSd), and daily mean time to disposition for admitted patients (TTD).
Primary Analysis
We first compared pre- and post-implementation data across a variety of operational characteristics using descriptive statistics including: total visits, mean daily visits, mean age, proportion of female patients, count and proportion of patients per ESI category, mean ESI, count and proportion of high acuity patients (ESI 1 or 2), mode of arrival, disposition, mean daily boarding time and mean boarding time per admitted patient. In addition, we calculated unadjusted, mean daily values for each of the pre-specified outcomes: LOS, LOSa, LOSd and TTD. Descriptive statistics were computed for both 8-week and one-year pre- and post-implementation periods. We generated a histogram of unadjusted mean daily LOS pre- and post- implementation. We also plotted outcomes by weekly average and overlayed a smooth trendline (using the LOESS method) for both 8-week and one-year pre- and post-implementation periods.
As our primary analysis, we used multiple regression modeling to examine primary and secondary outcomes. We created a pre-specified ordinary least squares (OLS) model regressing daily mean LOS with the following variables: pre- or post-implementation, month, day of the week, daily visits, daily visits from the preceding day, mean patient age, proportion of female patients, proportion of admissions, ED observation admissions and discharges, proportion of patients with ESI ≤ 2, proportion of patients arriving by ambulance and total daily boarding time. These variables were selected from all available departmental data based on their potential impact on departmental efficiency due to i) seasonal variation, ii) daily variation in patient demographics, volume and acuity, and iii) variation in hospital admissions and census (with boarding time serving as a proxy measure of hospital census). The pre-specified variables incorporated in the model were selected by consensus of the authors prior to analysis of the data set. We then applied this model for each of the outcome variables for one-year and 8-week pre- and post-implementation periods and report adjusted outcome variables.
Secondary Models
Recognizing limitations of our study design to determine causal relationships, we performed four adjunct analyses to better understand the impact of eDoc implementation on LOS. These secondary models were applied only to the full study period.
1) We performed coarsened exact matching (CEM) for similar days across the pre- and post-implementation periods as a sensitivity analysis.25 Coarsened exact matching is another method to control for confounding influences of pretreatment control variables in an observational data set (an alternative to multiple regression modeling).25 This method of analysis matches data points from the control (pre-implementation) and intervention (post-implementation) groups and prunes data that do not have an adequate match according to the specified control variables. The result is a subset of matched data points that can be used to estimate a causal effect.
We utilized CEM to identify similar days based on pre-specified variables of month, day, registered visits, visits from the previous day, daily admission rate, proportion of patients with ESI ≤ 2, and total boarding time. Matching was performed based on month of the year, day of the week and quartile of the remaining variables (visits, visits the previous day, admit percentage, high acuity patients and boarding time). Unmatched data was excluded. Linear regression modeling was then used to compare these matched samples and estimate the “sample average treatment effect on the treated” (SATT). We used a matching algorithm from the CEM package designed for R statistical software to perform this adjunct analysis.26
2) In addition, we employed a robust regression in order to control for the effect of outliers given observed variation in ED census. We first evaluated Cooks D distance and standardized residuals in order to explore the effects of outliers. We then used Stata’s “rreg robust” regression function where outliers had less weight in order to ensure that the results were not skewed by random variation in outlying values.27
3) In order to control for seasonal variation in ED census and to wash out the initial effects of adusting to the new system, we compared a period 6 months prior to implementation (9/12 through 3/13) to the same timeframe during the year following implementation (9/13 through 3/14). This strategy removes the immediate effects of implementation for changes in length of stay. In addition, it aligns similar timeframes to account for significant seasonal variability in ED volume.
4) Finally, we used augmented inverse probability weighting using the “teffects aipw” function in Stata, a doubly robust method in which one model is used to predict treatment and another model is used to predict outcome.28 AIPW is a modern statistical method that can be used to estimate treatment effects.29 It is considered “doubly robust” because the estimated effect is consistent even if the propensity score modeling component or the outcome regression is misspecified but the other model is properly specified.
Statistical analysis and modeling was performed in R (version 3.2.2) and Stata (version 14, StataCorp, College Station, TX) .30 Figures were generated using R and Stata.
Results
Characteristics of Study Subjects
There were a total of 120,207 patient encounters during the designated study period (60,870 pre-implementation and 59,337 post-implementation). Patient characteristics during the pre- and post-implementation periods were similar (Table 1) as well as during the 8-week pre- and post-implementation periods (Table 2). Notably, there were differences in mean daily boarding time and mean boarding time per admitted patient between pre- and post implementation periods for the 8-week and full one-year study periods. Appendix C shows a histogram of daily mean LOS comparing pre- and post- implementation periods.
Table 1.
Patient characteristics pre- and post-implementation: full study period
| Pre-implementation (3/18/2012 – 3/17/2013) | Post-implementation (3/18/2013 – 3/17/2014) | |||
|---|---|---|---|---|
|
| ||||
| N | % | N | % | |
| Total Visits | 60,870 | 59,337 | ||
| Mean Daily Visits (SD) | 166.8 (18.6) | 162.6 (18.3) | ||
| Mean Age (SD) | 48.4 (1.8) | 48.7 (1.7) | ||
| Sex | ||||
| Female | 36,578 | 60.1% | 35,548 | 59.9% |
| ESI | ||||
| 1 | 833 | 1.4% | 929 | 1.6% |
| 2 | 19,349 | 31.8% | 17,357 | 29.3% |
| 3 | 30,344 | 49.9% | 30,607 | 51.6% |
| 4 | 8,903 | 14.6% | 9,073 | 15.3% |
| 5 | 1,428 | 2.3% | 1,329 | 2.2% |
| High Acuity (≥2) | 20,182 | 33.2% | 18,286 | 30.8% |
| Mean ESI | 2.85 | 2.87 | ||
| Mode of Arrival | ||||
| Ambulance | 17,044 | 28.0% | 16,915 | 28.5% |
| Disposition | ||||
| Admit | 16,757 | 27.5% | 16,790 | 28.3% |
| OBS | 6,410 | 10.5% | 6,007 | 10.1% |
| Home | 35,583 | 58.5% | 34,543 | 58.2% |
| Mean Daily Boarding Time in Minutes (SD) | 9,708 (5,352) | 10,188 (6,036) | ||
| Mean Boarding Time per Admitted Patient (Minutes) | 211.2 | 221.4 | ||
SD = standard deviation
Table 2.
Patient characteristics pre- and post-implementation: short-term, 8-week assessment
| Pre-implementation (1/21/2013 – 3/17/2013) | Post-implementation (3/18/2013 – 5/12/2013) | |||
|---|---|---|---|---|
|
| ||||
| N | % | N | % | |
| Total Visits | 8,708 | 8,896 | ||
| Mean Daily Visits (SD) | 155.5 (21.2) | 158.9 (17.0) | ||
| Mean Age (SD) | 48.5 (1.9) | 48.2 (1.8) | ||
| Sex | ||||
| Female | 5,272 | 60.5% | 5,333 | 59.9% |
| ESI | ||||
| 1 | 172 | 2.0% | 164 | 1.8% |
| 2 | 2,672 | 30.7% | 2,571 | 28.9% |
| 3 | 4,285 | 49.2% | 4,547 | 51.1% |
| 4 | 1,386 | 15.9% | 1,374 | 15.4% |
| 5 | 190 | 2.2% | 231 | 2.6% |
| High Acuity (≥2) | 2,844 | 32.7% | 2,735 | 30.7% |
| Mean ESI | 2.83 | 2.87 | ||
| Mode of Arrival | ||||
| Ambulance | 2,580 | 29.6% | 2,596 | 29.2% |
| Disposition | ||||
| Admit | 2,441 | 28.0% | 2,367 | 26.6% |
| OBS | 924 | 10.6% | 918 | 10.3% |
| Home | 5,105 | 58.6% | 5,350 | 60.1% |
| Mean Daily Boarding Time in Minutes (SD) | 7,554 (3,126) | 5,988 (2,172) | ||
| Mean Boarding Time per Admitted Patient (Minutes) | 173.4 | 141.6 | ||
SD = standard deviation
Main Results
Figure 1 shows unadjusted outcomes for one-year pre- and post-implementation periods. Figure 2 shows unadjusted outcomes by weekly average for eight week pre- and post-implementation periods. Unadjusted values and net change for all outcome variables are shown in Tables 3-1 and 3-2 for one-year and 8-week pre- and post-implementation periods respectively.
Figure 1.
Unadjusted outcomes by weekly average for one-year pre- and post-implementation periods. Outcomes include: mean length-of-stay (LOS), mean length-of-stay for admitted (LOSa) and discharged patients (LOSd), and mean time to disposition for admitted patients (TTD). Trendlines were generated by LOESS method.
Figure 2.
Unadjusted outcomes by weekly average for eight week pre- and post-implementation periods. Outcomes include: mean length-of-stay (LOS), mean length-of-stay for admitted (LOSa) and discharged patients (LOSd), and mean time to disposition for admitted patients (TTD). Trendlines were generated by LOESS method.
Table 3-1.
Adjusted and unadjusted changes in outcome variables pre- and post eDoc implementation, full study period
| Pre, minutes (SD) | Post, minutes (SD) | Unadjusted Δ, minutes (95% CI) | Adjusted Δ, minutes (95% CI) | |
|---|---|---|---|---|
|
Mean daily LOS
|
||||
| All Patients
|
257.4 (43.2) | 265.8 (49.2) | + 8.4 (1.8 to 15.0) | +6.3 (3.5 to 9.1) |
| Admitted Patients
|
388.2 (105.0) | 399.6 (132.0) | + 11.4 (−6.0 to 28.8) | + 1.3 (−3.4 to 6.0) |
| Discharged Patients
|
209.4 (28.2) | 211.2 (24.6) | + 1.8 (−1.8 to 5.4) | + 5.1 (1.9 to 8.3) |
|
Time to Disposition (Admitted Pts)
|
180.0 (24.6) | 181.8 (0.38) | + 1.8 (−1.8 to 5.4) | + 2.8 (−0.6 to 6.3) |
LOS = length of stay; SD = standard deviation; CI = confidence interval
Table 3-2.
Adjusted and unadjusted changes in outcome variables, pre- and post eDoc implementation, short-term, 8-week assessment
| Pre, minutes (SD) | Post, minutes (SD) | Unadjusted Δ, minutes (95% CI) | Adjusted Δ, minutes (95% CI) | |
|---|---|---|---|---|
|
Mean daily LOS
|
||||
| All Patients
|
249.6 (35.4) | 232.2 (19.8) | − 17.4 (−28.2 to −6.6) | − 3.6 (−15.6 to 8.4) |
| Admitted Patients
|
355.2 (61.8) | 313.2 (42.0) | − 42.0 (−61.8 to −22.2) | − 11.4 (−30.6 to 7.8) |
| Discharged Patients
|
207.6 (29.4) | 201.6 (18.6) | − 6.0 (−15.0 to 3.0) | − 3.0 (−16.2 to 10.2) |
| Time to Disposition (Admitted Pts) | 183.6 (33.0) | 174.6 (19.8) | − 9.0 (−19.2 to 1.2) | −6.0 (−24.0 to 12.0) |
LOS = length of stay; SD = standard deviation; CI = confidence interval
The results of multiple regression analysis are also shown in Tables 3-1 and 3-2 with adjusted values compared to unadjusted analysis for these periods. With application of the pre-specified regression model, there were no significant changes in any outcome variables for the 8-week post-implementation period. For the one-year study period overall, there was a significant increase of +6.3 minutes (95% CI = 3.5 to 9.1 minutes) for overall LOS and +5.1 minutes (95% CI = 1.9 to 8.3 minutes) for LOSd only. There was no statistically significant difference in length of stay (LOSa) or time to disposition (TTD) for admitted patients.
Secondary Models
To ensure our primary findings were not model dependent we performed the following secondary analyses over the full study period: 1) coarsened exact matching (CEM), 2) robust regression (to control for outliers), 3) regression comparing 6 month pre-implementation period with the same period the following year, and 4) augmented inverse probability weighting (AIPW). These secondary analyses were concordant with the primary multiple regression results, finding statistically significant increases in overall LOS and and LOSd. The robust regression model (3) also found a statistically significant increase in TTD. The full results of the secondary analyses are presented in Appendix D.
Limitations
There are several potential limitations of this study. First, this study was performed in a single center using a custom developed electronic provider documentation system. While the design of eDoc is similar to other electronic documentation tools available, our results may not be generalizable to other EDs.
Additionally, due to the large sample size (120,207 total patient encounters) there were statistically significant differences in the pre- and post-implementation groups – notably in boarding time in the short-term analysis. However, we control for these differences in our primary regression model and in the secondary models used to validate our results. The association between increased length of stay and the implementation of eDoc was consistently demonstrated.
Importantly, this study also did not capture time that staff may have spent outside their shift hours to complete provider documentation using eDoc or documentation shortcuts. Anecdotally, we know that many staff reported staying longer hours to complete their documentation after the transition to eDoc. Further, staff may have used shortcuts, such as copy and paste, free text documentation instead of structured field completion, and less documentation. However, it was not possible to quantify these effects based on our available dataset. We hypothesize that any such after-hours charting or short cuts would have had the effect of buffering increases in length of stay.
Finally, this study represents a quantitative analysis of the impact of implementation on major quality and operational measures. As a result, our investigation does not capture physician satisfaction, system usability, and the technical challenges around implementation that often play a significant role in the success or failure of new EHR functionality,31 and which may be better characterized using qualitative methods. The study also does not characterize all potential benefits of electronic documentation such as potentially more thorough chart completion and improved billing.
Discussion
In this single center retrospective analysis, we took advantage of a unique opportunity to examine the isolated effect on ED operational performance of transitioning from paper-based ED documentation to a custom-developed electronic provider documentation. In unadjusted analysis, we observed significant variation in daily mean length of stay and an overall secular trend towards increased length of stay (Appendix C, Figure 1). After adjusting for operational metrics that might impact department efficiency, we found statistically significant increases in overall length-of-stay and length-of stay for discharged patients equivalent to 6 minutes and 5 minutes respectively per patient. While our study design cannot establish causation, we incorporated multiple additional supporting analyses which were concordant with the original results, bolstering our finding that the change to eDoc was a driver of increasing length of stay and that our findings were not model dependent.
Our custom-designed system was associated with a small, but consistent and statistically significant increase in length-of-stay. Although there was a trend towards decreasing length of stay in the immediate (8-week) implementation period in unadjusted analysis, this trend was not statistically significant after adusting for seasonal variation and other confounders. In long-term analysis, we found a statistically significant increase in LOS and LOSd that was consistent across multiple secondary modeling strategies.
Although this increase was relatively small in magnitude, a change of a few minutes could have a potentially important effect in a high-throughput ED. In our ED, an additional 6 minutes per patient encounter would add over 16 hours per day for an ED serving 165 patients per day. These increases in length of stay can potentially lead to decreased patient satisfaction and delays in care of time-sensitive conditions.32
It is also important to note that value is not measured in time saved alone; our study does not capture changes in the quality and completeness of documentation that could potentially add value in the domains of clinical care, medicolegal issues and billing. Although changes to physician workflow, such as migrating to an electronic charting system, have the potential to create a large ripple effect in a high-volume department, it is critical to balance this against the unmeasured benefits of more complete and high quality electronic documentation systems.
Our findings are consistent with recent research suggesting that electronic provider documentation in the ED may be more time consuming than traditional paper charting.16, 17 Previous studies of the impact of EHRs on ED operations have been mixed. Two single center studies found transient increases in patient length of stay that was not sustained over time18, 20 and a study from Australia found sustained increases over time.19 However, other studies have demonstrated the opposite: that the transition to electronic health records was associated with reduced length of stay.21, 22 Notably, the studies demonstrating improved length of stay examined multi-feature EHRs, which also included simultaneous implementation of other features. Computerized provider order entry implementation, for example, has been demonstrated to be associated with decreased length-of-stay13, which could offset increases associated with electronic provider documentation. Our study involved an isolated change to electronic provider documentation. Furthermore, previous studies did not incorporate data on the use of scribes or dictation software to mitigate the impact of EHR implementation. At our study site, attending physicians used speech recognition in a similar manner to the voice dictation system that was available prior to electronic provider documentation implementation, minimizing any changes in the provider documentation process.
In the era of meaningful use requirements, EHR vendors have faced increased scrutiny for poor usability and inattention to physician workflows.6, 7 We custom built eDoc for our providers and workflows, using robust, agile software development practices in which physicians were involved in design, testing, and implementation.24 Even with our focus on workflow and usability, we still experienced an impact on long-term operational performance. Importantly, our design was also greatly constrained by billing compliance regulations required for US fee for service billing. For example, electronic provider documentation systems must allow for all possible review of systems elements, and enable physicians to select at least 10 required for level 5 billing. The evaluation and management (E/M) billing rules – which require a seemingly arbitrary number of documentation elements – may be a confounding factor impacting the usability and efficiency of EHRs.33
As we move toward mandatory and universal use of EHRs, including the requirement to document patient encounters electronically,34 future research should be aimed at identifying interventions to mitigate the impact of electronic provider documentation on ED efficiency. Ideally, we would work with policy makers and insurers to reduce documentation requirements, especially with the transition to accountable care organizations and value-based care with providers assuming more of the financial risk. Further, the potential impact of adjusting staffing, adding scribes, and using voice recognition software to mitigate any effect of electronic provider documentation on ED operations should be formally evaluated in future studies.
In conclusion, electronic provider documentation is an important function enabled by the implementation of EHRs and is required under the current federal EHR incentive program. Our study isolates the effect of transitioning from paper-based to electronic provider documentation and quantifies a small, but consistent and statistically significant increase in length-of-stay at our single, high-volume, tertiary ED. This suggests that implementation of electronic provider documentation alone may have an adverse affect on ED operational performance, even if it has unmeasured benefits in regards to documentation quality and completeness. Employing custom development in the design and implementation process did not completely mitigate the efficiency losses. EHR design alone may not be sufficient to eliminate operations impacts – changes to billing compliance requirements, scribes, and voice recognition software may be valuable areas for future research to mitigate these effects.
Supplementary Material
Appendix A. Provider paper documentation template in use prior to eDoc implementation.
Appendix B. Sample eDoc screenshots for history of present illness, past medical history, social history, and review of systems sections.
Appendix C. Histogram displaying distribution of daily mean length of stay (LOS) for the pre- and post-implementation periods.
Appendix D. Additional adjunct analyses. Adjusted ordinary least squares (OLS) is our primary model. Secondary models included: 1) coarsened exact matching (CEM); 2) a doubly robust augmented inverse probability weighting (AIPW) model; 3) a robust weighted regression model to reduce the effect of outliers; and 4) a matched OLS model comparing a six-month period before eDoc implementation (Sept 2012 – Mar 2013) and the same six-month period after eDoc implementation (Sept 2013 – Mar 2013).
Acknowledgments
Financial Support: Dr. Ward is supported in part by NIH K23 HL127130. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. All authors had full access to all the data in the study and had final responsibility for the decision to submit for publication.
We would like to thank Hillary Cornish for her assistance with data collection for this study and Jonathan M. Teich, MD, PhD for his thoughtful review of an earlier draft of this manuscript. We would also like to thank the Brigham and Women’s Hospital Emergency Department Information Systems team for their hard work designing, developing, and implementing this electronic documentation module.
Footnotes
Related Conference Presentations: SAEM New England Regional Conference; April 1, 2015; Newton, MA, USA. ACEP Scientific Assembly 2015; October 25–27, 2015; Boston, MA, USA.
Conflicts of Interest: None
Author Contributions: ABL, JF, and SST conceived and designed the study. All authors provided feedback on the study design. ABL, JF, and SST supervised the conduct of the study and data collection and management. SST and MW provided statistical advice on study design and analyses; JF and SST conducted the analyses. JF drafted the manuscript, and all authors contributed substantially to its revision. ABL takes responsibility for the paper as a whole.
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Blumenthal D. Stimulating the adoption of health information technology. N Engl J Med. 2009;360:1477–1479. doi: 10.1056/NEJMp0901592. [DOI] [PubMed] [Google Scholar]
- 2.Desroches CM, Charles D, Furukawa MF, et al. Adoption Of Electronic Health Records Grows Rapidly, But Fewer Than Half Of US Hospitals Had At Least A Basic System In 2012. Health Aff (Millwood) 2013 doi: 10.1377/hlthaff.2013.0308. [DOI] [PubMed] [Google Scholar]
- 3.Wright A, Henkin S, Feblowitz J, McCoy AB, Bates DW, Sittig DF. Early results of the meaningful use program for electronic health records. N Engl J Med. 2013;368:779–780. doi: 10.1056/NEJMc1213481. [DOI] [PubMed] [Google Scholar]
- 4.Adler-Milstein J, DesRoches CM, Furukawa MF, et al. More than half of US hospitals have at least a basic EHR, but stage 2 criteria remain challenging for most. Health Aff (Millwood) 2014;33:1664–1671. doi: 10.1377/hlthaff.2014.0453. [DOI] [PubMed] [Google Scholar]
- 5.Jamoom E, Hing E. NCHS data brief, no 187. Hyattsville, MD: National Center for Health Statistics; 2015. Progress with electronic health record adoption among emergency and outpatient departments: United States, 2006–2011. [PubMed] [Google Scholar]
- 6.Wears RL. Health information technology and victory. Ann Emerg Med. 2015;65:143–145. doi: 10.1016/j.annemergmed.2014.08.024. [DOI] [PubMed] [Google Scholar]
- 7.Kellermann AL, Jones SS. What it will take to achieve the as-yet-unfulfilled promises of health information technology. Health Aff (Millwood) 2013;32:63–68. doi: 10.1377/hlthaff.2012.0693. [DOI] [PubMed] [Google Scholar]
- 8.American Medical Association. AMA and Other Medical Societies Call for Pause in the Meaningful Use Program. American Medical Association; Sep 17, 2015. [Google Scholar]
- 9.Baumlin KM, Shapiro JS, Weiner C, Gottlieb B, Chawla N, Richardson LD. Clinical information system and process redesign improves emergency department efficiency. Jt Comm J Qual Patient Saf. 2010;36:179–185. doi: 10.1016/s1553-7250(10)36030-2. [DOI] [PubMed] [Google Scholar]
- 10.Zikos D, Diomidous M, Mpletsa V. The effect of an electronic documentation system on the trauma patient's length of stay in an emergency department. J Emerg Nurs. 2014;40:469–475. doi: 10.1016/j.jen.2013.10.008. [DOI] [PubMed] [Google Scholar]
- 11.Boger E. Electronic tracking board reduces ED patient length of stay at Indiana Hospital. J Emerg Nurs. 2003;29:39–43. doi: 10.1067/men.2003.13. [DOI] [PubMed] [Google Scholar]
- 12.Daniel GW, Ewen E, Willey VJ, Reese CL, Iv, Shirazi F, Malone DC. Efficiency and economic benefits of a payer-based electronic health record in an emergency department. Acad Emerg Med. 2010;17:824–833. doi: 10.1111/j.1553-2712.2010.00816.x. [DOI] [PubMed] [Google Scholar]
- 13.Spalding SC, Mayer PH, Ginde AA, Lowenstein SR, Yaron M. Impact of computerized physician order entry on ED patient length of stay. Am J Emerg Med. 2010 doi: 10.1016/j.ajem.2009.10.007. [DOI] [PubMed] [Google Scholar]
- 14.Heo JH, Kim YD, Nam HS, et al. A computerized in-hospital alert system for thrombolysis in acute stroke. Stroke. 2010;41:1978–1983. doi: 10.1161/STROKEAHA.110.583591. [DOI] [PubMed] [Google Scholar]
- 15.Britton DJ, Bloch RB, Strout TD, Baumann MR. Impact of a Computerized Order Set on Adherence to Centers for Disease Control Guidelines for the Treatment of Victims of Sexual Assault. J Emerg Med. 2012 doi: 10.1016/j.jemermed.2012.06.016. [DOI] [PubMed] [Google Scholar]
- 16.Park SY, Lee SY, Chen Y. The effects of EMR deployment on doctors' work practices: a qualitative study in the emergency department of a teaching hospital. Int J Med Inform. 2012;81:204–217. doi: 10.1016/j.ijmedinf.2011.12.001. [DOI] [PubMed] [Google Scholar]
- 17.Perry JJ, Sutherland J, Symington C, Dorland K, Mansour M, Stiell IG. Assessment of the impact on time to complete medical record using an electronic medical record versus a paper record on emergency department patients: a study. Emerg Med J. 2013 doi: 10.1136/emermed-2013-202479. [DOI] [PubMed] [Google Scholar]
- 18.Spellman Kennebeck S, Timm N, Farrell MK, Spooner SA. Impact of electronic health record implementation on patient flow metrics in a pediatric emergency department. J Am Med Inform Assoc. 2012;19:443–447. doi: 10.1136/amiajnl-2011-000462. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Mohan MK, Bishop RO, Mallows JL. Effect of an electronic medical record information system on emergency department performance. Med J Aust. 2013;198:201–204. doi: 10.5694/mja12.10499. [DOI] [PubMed] [Google Scholar]
- 20.Ward MJ, Froehle CM, Hart KW, Collins SP, Lindsell CJ. Transient and sustained changes in operational performance, patient evaluation, and medication administration during electronic health record implementation in the emergency department. Ann Emerg Med. 2014;63:320–328. doi: 10.1016/j.annemergmed.2013.08.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Ward MJ, Landman AB, Case K, Berthelot J, Pilgrim RL, Pines JM. The effect of electronic health record implementation on community emergency department operational measures of performance. Ann Emerg Med. 2014;63:723–730. doi: 10.1016/j.annemergmed.2013.12.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Mathison D, Chamberlain J. Evaluating the impact of the electronic health record on patient flow in a pediatric emergency department. Appl Clin Inform. 2011;2:39–49. doi: 10.4338/ACI-2010-08-RA-0046. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Hollingsworth JC, Chisholm CD, Giles BK, Cordell WH, Nelson DR. How do physicians and nurses spend their time in the emergency department? Ann Emerg Med. 1998;31:87–91. doi: 10.1016/s0196-0644(98)70287-2. [DOI] [PubMed] [Google Scholar]
- 24.Schumacher R, Lowry S. US Department of Commerce, National Institute of Standards and Technology. Nov, 2010. NIST Guide to the Processes Approach for Improving the Usability of Electronic Health Records. NISTIR 7741. [Google Scholar]
- 25.Iacus SM, King G, Porro G. Causal Inference Without Balance Checking: Coarsened Exact Matching. Political Analysis. 2011 [Google Scholar]
- 26.King G. CEM Package for R. Feb 27, 2015. CEM:Coarsened Exact Matching Software. [Google Scholar]
- 27.Stata Data Analysis Examples Robust Regression. UCLA. Statistical Consulting Group; [accessed January 31, 2017]. from http://www.ats.ucla.edu/stat/stata/dae/rreg.htm. [Google Scholar]
- 28.Glynn, Adam N, Kevin M. Quinn. "An introduction to the augmented inverse propensity weighted estimator.". Political Analysis. 2010;18(1):36–56. [Google Scholar]
- 29.Bang Heejung, James M. Robins. "Doubly robust estimation in missing data and causal inference models.". Biometrics. 2005;61(4):962–973. doi: 10.1111/j.1541-0420.2005.00377.x. [DOI] [PubMed] [Google Scholar]
- 30.R Core Team. R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2013. [Google Scholar]
- 31.Sittig DF, Singh H. A new sociotechnical model for studying health information technology in complex adaptive healthcare systems. Qual Saf Health Care. 2010;19(Suppl 3):i68–74. doi: 10.1136/qshc.2010.042085. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Bernstein SL, Aronsky D, Duseja R, et al. The effect of emergency department crowding on clinically oriented outcomes. Acad Emerg Med. 2009;16:1–10. doi: 10.1111/j.1553-2712.2008.00295.x. [DOI] [PubMed] [Google Scholar]
- 33.Horwitz L. A Shortcut to Wasted Time. The New York Times. 2012 Nov 22;:A35. [Google Scholar]
- 34.Federal Register. Department of Health and Human Services, Centers for Medicare & Medicaid Services; 2012. Medicare and Medicaid Programs; Electronic Health Record Incentive Program—Stage 2. 42 CFR Parts 412, 413, and 495. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Appendix A. Provider paper documentation template in use prior to eDoc implementation.
Appendix B. Sample eDoc screenshots for history of present illness, past medical history, social history, and review of systems sections.
Appendix C. Histogram displaying distribution of daily mean length of stay (LOS) for the pre- and post-implementation periods.
Appendix D. Additional adjunct analyses. Adjusted ordinary least squares (OLS) is our primary model. Secondary models included: 1) coarsened exact matching (CEM); 2) a doubly robust augmented inverse probability weighting (AIPW) model; 3) a robust weighted regression model to reduce the effect of outliers; and 4) a matched OLS model comparing a six-month period before eDoc implementation (Sept 2012 – Mar 2013) and the same six-month period after eDoc implementation (Sept 2013 – Mar 2013).