Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2024 Jan 4.
Published in final edited form as: J Surg Educ. 2018 Nov;75(6):e97–e106. doi: 10.1016/j.jsurg.2018.10.010

Documenting or Operating: Where Is Time Spent in General Surgery Residency?

Morgan L Cox *, Alfredo E Farjat †, TJ Risoli †, Sarah Peskoe †, Benjamin A Goldstein †, David A Turner ‡, John Migaly *
PMCID: PMC10765321  NIHMSID: NIHMS1856296  PMID: 30522828

Abstract

OBJECTIVE:

The utilization of electronic health records (EHR) has become essential in the daily activities of physicians for documentation and as an information source. However, the amount of time spent by residents utilizing the EHR has not been thoroughly evaluated, particularly within surgical specialties. This study aims to analyze EHR usage by general surgery residents and to assess the association between this use and case volume at a single academic institution.

DESIGN:

For general surgery residents in clinical years (CY) 1–5, de-identified login and logout time data between September 2016 and June 2017 were retrospectively extracted from the Epic EHR (Verona, WI). A binary time series was created for each resident to indicate and track over time whether he or she was utilizing the EHR system. Comparisons between categorical variables were performed with Fisher’s exact test. Continuous variables were compared using Wilcoxon rank sum test. Longitudinal linear mixed-effects models were used to assess the EHR usage among the surgery residents. The association between EHR time and the number of operative cases logged was evaluated with Pearson’s correlation coefficient.

SETTING:

This study was performed by the Department of Surgery in conjunction with the Office of Graduate Medical Education at Duke University Health System.

PARTICIPANTS:

All active general surgery residents during the 2016–2017 academic year.

RESULTS:

Thirty-six general surgery residents (28 males, 8 females) spent a median of 2.4 hours per day and 23.7 hours per week using the EHR. CY2 had the highest median usage per week (28.9 hours), while CY3 had the lowest (16.7 hours) but no significant difference based on EHR usage was found among the analyzed CYs (p = 0.164). Residents spent significantly more time logged into the EHR during the week compared to weekends and during the day compared to nights (all p < 0.001). For the residency program as a whole, a median of 151.5 total work hours per day was dedicated to documentation. On average, interns on dedicated night rotations spent 7% of their login time outside regularly scheduled duty hours while interns on dedicated day rotations spent 27%. There was no overall correlation between monthly case logs and EHR usage (r = 0.06, p = 0.30); however, CY2 had a significant negative correlation (r = −0.2, p = 0.038).

CONCLUSIONS:

In the era of a maximum 80-hour work week, general surgery residents spend a substantial portion, at least 30%, of their time utilizing the EHR. One third of EHR usage by interns occurred outside the scheduled 12-hour shift, demonstrating the difficulties of completing paperwork as part of the scheduled work day. Additionally, the lack of correlation to case logs is likely due to an underestimation of the documentation burden associated with operating, which includes preparatory effort and operative notes. Ultimately, these quantitative EHR usage results will be correlated to burnout prior to implementing programs to improve efficiency and decrease the burden of charting.

Keywords: Electronic health record, General surgery, Residency, Time, Systems-Based Practice, Professionalism, Patient Care

INTRODUCTION

Electronic health records (EHR) have been a requirement for all public and private healthcare providers to continue receiving Medicaid and Medicare reimbursement since the American Recovery and Reinvestment Act in 2014.1 The EHR provides many positive features including error prevention, improved information sharing, and more efficient secondary work such as audits, research, and billing.2–6 However, the documentation burden created by the EHR has drastically changed the provider’s workflow with 92% of trainees reporting documentation obligations as excessive, while 90% of trainees believe it compromises time spent with patients.7 Existing literature evaluating the various effects of EHR implementation is mostly based on surveys or observational data, and no literature exists looking specifically at the impact within surgical or procedural specialties.6,8–14

Early studies evaluating the impact of EHR show an increased amount of time spent on the computer for clinical review and documentation at the expense of time spent performing direct patient care.15–19 Additionally, the impact of EHR on trainees and education is rarely discussed.2,7,20–22 Most institutions lack professional development to help residents efficiently function in the era of EHR and lack initiatives to streamline EHR documentation and workflow.23,24

Therefore, this study aims to describe and analyze the amount of time spent using the EHR system by general surgery residents at a single, academic institution by utilizing a large, objective dataset directly from the EHR itself. The results of this study will better define the EHR burden on surgical residents allowing programs to develop interventions to better balance EHR documenting and patient engagement.

METHODS

Data Acquisition

The Duke University Institutional Review Board approved this study. All clinically active general surgery residents between September 2016 and July 2017 were included and stratified by clinical year (CY) 1–5. July and August of the academic year were excluded due to incomplete EHR data, as no after noon timestamps were reported. Surgical residents in dedicated research years were excluded from analysis.

Utilizing the Epic EHR (Epic Systems Corporation, Verona, WI), we retrospectively extracted all login and logout timestamped data for the previously identified general surgery residents. A binary time series was created for each individual resident to indicate and track over time whether he or she was logged into the EHR system at any given time point during the study period.

Initial processing of data included identifying and defining concurrent logins. For the purposes of this investigation, a concurrent login was defined as 2 login times that occurred prior to a logout time. This concurrent login likely indicates the use of 2 different computers at the same time. In this case, the entire timeframe, from first login time to next consecutive logout time, was considered active EHR use. Each logout timestamp was classified as either a system or user logout depending on whether the resident purposefully logged out of Epic (user) or a timeframe of inactivity was met (system). A system logout could have occurred in 2 separate ways: (1) 20 minutes or 40 minutes of inactivity pending computer location or (2) resident performed a quick logout of the entire desktop without logging out of EHR first which correlates with no amount of inactivity. Since EPIC cannot determine the reason for system logout, we choose not to amend the data based on the presence of a system logout. Of note, mobile EHR logins via an application on handheld devices were not included for analysis.

Within our residency program, there are 4 to 6 residents who train at the Veteran’s Administration (VA) hospital every month. These residents continue to log into the EPIC system, although the majority of their EHR use is within the VA system, which was not captured in this analysis. These VA rotations could not be completely removed from analysis since the residency program rotation schedule changes with the calendar month rather than on a consistent day of the week. Removal of the VA rotations from analysis would have led to Epic EHR usage being reported during “incomplete weeks” (e.g., Tuesday through Saturday) as a resident rotated on or off a VA rotation. This change would have led to an underestimation of EHR time for the entire analysis. Therefore, we performed a sensitivity analysis removing the VA rotations to determine how much the use of this outside system actually underestimated the total EHR use calculated in this study.

We extracted additional study data, including resident demographics, monthly case logs, and clinical rotation schedules, and managed them in a Research Electronic Data Capture (REDCap)25 electronic data capture tools hosted at Duke University. Monthly operative case logs were extracted and compiled from the Accreditation Council for Graduate Medical Education (ACGME) website by the general surgery residency coordinator. Daytime hours were defined as 6:00 am to 5:59 pm, while nighttime hours spanned 6:00 pm to 5:59 am. Only clinical rotations identified as purely day or night rotations were included in the analysis to determine EHR usage outside of the designated shift time. Finally, the binary time series and REDCap data were then linked via deidentified, unique identifiers for statistical analysis.

Outcomes

The primary outcome of interest was the amount of time logged into the EHR system per resident and stratified by day of the week and time of day.

Statistical Analysis

Baseline resident characteristics were compiled and analyzed. Descriptive statistics were calculated for the overall cohort and for each subgroup separately. Continuous variables were summarized with their mean, standard deviation, median, and interquartile range (IQR). Categorical variables were reported as frequencies and percentages from total. Comparisons between categorical variables were performed with Fisher’s exact test. Continuous variables were compared using Wilcoxon rank sum test. Boxplots were employed to visualize the sample distribution of login times and a nonparametric approach was used to highlight the overall trend. Longitudinal linear mixed-effects models were used to assess the EHR usage among the surgery residents. A random subject effect was included in the models to account for the repeated measurements across time among residents. REML method was used to fit the linear mixed models. The Satterthwaite approximation to the degrees of freedom was used to evaluate the t tests and the F tests.

The proportion of EHR usage during and after hours was calculated based on clinical rotation and stratified by interns (CY1) and residents (CY 2–5). Six intern rotations were identified as solely day shift rotations and 2 were solely night shift rotations. Resident rotations included 2 and 17 rotations that were solely night and day shifts, respectively. Proportions were calculated by taking the amount of EHR usage time during the designated 12-hour shift divided by the total time of EHR usage by interns and residents on those specific rotations.

The association between EHR usage time and the number of operative cases logged was evaluated with the Pearson’s correlation coefficient, and the associated confidence interval was based on the asymptotic Fisher’s Z transformation. An overall correlation coefficient was produced for the entire cohort. Additional correlation coefficients between EHR usage time and monthly case logs based on CY were estimated. Scatterplots of the number of operative cases logged as function of the EHR usage stratified by CY were produced. Loess curves were produced by using locally weighted polynomial regression to fit smoothed curves to the scatterplots in order to highlight the association between the variables.

For all the analyses, two-tailed tests were used and the threshold for assessing statistical significance was set at level α = 0.05. Statistical analyses were performed using R software version 3.3.0.26

RESULTS

Resident Demographics

Thirty-six general surgery residents with a mean age of 30.1 (SD ±2.9) years were included for analysis. There were 28 males (78%) and 8 females (22%). This included 11 CY1 (5 preliminary, 6 categorical), 8 CY2, 4 CY3, 6 CY4, and 7 CY5 (Table 1).

TABLE 1.

Baseline Characteristics of General Surgery Residents

Total (n =36)
Age
Clinical year		 30.1 (±2.9)
1 11 (30.6%)
2 8 (22.2%)
3 4 (11.1%)
4 6 (16.7%)
5 7 (19.4%)
Type of intern Preliminary 5 (45.5%)
Categorical 6 (54.5%)
Sex Male 28 (77.8%)
Female 8 (22.2%)
Open in a new tab

Continuous data are reported as a mean (SD), while categorical data are reported as count (percent).

Overall Usage Time

General surgery residents spent a median of 2.4 hours per day (IQR 0.0, 6.13) and 23.7 hours per week (IQR 6.8, 37.3) logged into the EHR. We observed a marked trend toward males spending more time utilizing the EHR per week compared to females (25.2 vs 15.6 hours, p = 0.051), but this investigation was not powered to statistically differentiate between male and female residents. CY2 had the highest median usage per week (28.9 hours), while CY3 had the lowest (16.7 hours), but no significant difference based on EHR usage was found among the entire cohort of analyzed CYs (p = 0.164). The residents included in analysis logged an average of 61.1 duty hours per week during the study period. This translates to an average of 38.2% of the actual work week being spent logged into the EHR. Weekly EHR usage by CY is presented in Table 2.

TABLE 2.

Weekly EHR Use in Hours by Clinical Year

Weekly % of 80-hour Week* % of logged Week†
Clinical year
1		 26.3 (8.4, 42.7) 32.9% 41.4%
2 28.9 (4.7, 41.7) 36.1% 49.3%
3 16.7 (2.9, 30.0) 20.9% 38.9%
4 23.0 (9.6, 33.4) 28.8% 28.8%
5 21.7 (8.1, 32.0) 27.1% 37.5%
Overall 23.7 (6.8, 37.3) 29.6% 38.2%
Open in a new tab

Continuous data are reported as a median (IQR)

*

Calculated using the maximum 80-hour work week.

†

Calculated using the mean hours logged by the residents during the study duration.

Residents spent significantly more hours logged into the EHR during the week (median 19.5 hours) compared to weekends (median 2.4 hours, p < 0.001). Accounting for all clinically active residents, a median of 151.5 (IQR 108.7, 191.7) total work hours per day were dedicated to documentation (Fig. 1A and B).

FIGURE 1.

FIGURE 1.

Open in a new tab

Amount of time spent logged into the EHR per resident: (A) stratified by day of the week and (B) stratified by weekday vs weekend.

Median monthly EHR login time per resident ranged from 89.6 hours in June (IQR 53.2, 147.0) to 134.1 hours in March (IQR 29.8, 157.0). There was no significant difference in EHR usage by month across the study period (p = 0.710).

Usage by Time of Day

The majority of residents’ EHR usage took place during daytime hours with spikes at 5:00 am and 5:00 pm prior to shift changes (Fig. 2). Of all time spent logged into the EHR, the median proportion of time spent using the EHR during daytime hours on day shifts, compared to nighttime hours while on day shifts, was 60% (IQR 51, 72%). However, daytime usage proportions differed by CY (p < 0.001; Fig. 3).

FIGURE 2.

FIGURE 2.

Open in a new tab

Boxplot demonstrating the amount of time logged in to the EHR by all residents stratified by hour of the day.

FIGURE 3.

FIGURE 3.

Open in a new tab

Boxplots of the proportion of daytime EHR usage to total EHR usage among surgery residents stratified by clinical year.

On the 7 monthly rotations with dedicated day shifts for CY1 residents (n = 28), interns spent an average of 27% (SD ±10%) of their EHR usage time outside of regularly scheduled duty hours. Interns (n = 11) on the 2 rotations with dedicated night shifts spent, on average, 7% (SD ±3%) of their EHR login time outside the scheduled nighttime duty hours.

The remaining residents on dedicated day shift rotations (n = 124), not including interns, spent an average of 21% (SD ±9%) of their EHR utilization time outside regularly scheduled duty hours. As for rotations with dedicated night shifts, residents (n = 14) spent an average of spent 7% (SD ±6%) of their login time outside of regularly scheduled duty hours.

Sensitivity Analysis

After removal of the VA rotations, residents spent a median of 3.6 hours per day (IQR 0.1, 6.8) and 27.3 hours per week (IQR 15.6, 38.9) logged into the EHR system. Inclusion of the VA rotations in the overall analysis underestimates the daily EHR use by approximately 1.2 hours and the weekly EHR use by approximately 3.6 hours. The minimum, maximum, and standard deviations were unchanged.

Case Logs

For all residents, there was no overall correlation between the number of operative cases logged monthly and EHR usage (r = 0.06, 95% confidence interval [CI] = −0.05,0.17, p = 0.30; Fig. 4). Residents in clinical years 1, 3, 4, and 5 all had positive Pearson correlation coefficients between case logs and login time indicating a higher number of operative cases correlated with more time utilizing the EHR. This positive correlation was only statistically significant for CY1 (r = 0.27, 95% CI = 0.02, 0.49, p = 0.034) and CY4 (0.50, CI = 0.29, 0.67, p < 0.001). CY2 residents had a statistically significant negative correlation (r = −0.2, CI = −0.43, −0.01, p = 0.038) suggesting increased EHR use correlated with a decrease in operative case logs. Figure 5 displays the scatterplots and loess curves of the operative number of cases logged as function of EHR usage stratified by CY.

FIGURE 4.

FIGURE 4.

Open in a new tab

Scatterplot of operative case logs as a function of monthly EHR login time for all residents. The dashed line represents the linear regression curve between the variables.

FIGURE 5.

FIGURE 5.

Open in a new tab

Scatterplot of operative case logs as a function of monthly EHR login time stratified by clinical year. The dashed line represents the linear regression curve between the variables.

DISCUSSION

Our study utilized a large dataset derived directly from the Epic EHR in order to quantify the total amount of EHR usage time by surgical trainees during an academic year. To our knowledge, this investigation is the largest of its kind, and the only study to ever answer the question of how much time is spent on documentation within general surgery training. We found that general surgery residents spend approximately 2.4 hours per day and 23.7 hours per week working in the EHR. This time accounts 38% of the average weekly duty hours actually logged by the resident cohort during the study period.

The current results are concordant with similar, although smaller, studies aiming to assess time spent documenting within internal medicine. Chen et al.27 analyzed interns’ EHR usage via mouse clicks, keystrokes, and mouse miles over a select 4 months. They found a mean usage of 112 hours per month with increased usage earlier in the academic year. Using a national survey of over 16,000 medicine residents, Oxentenko et al.12 revealed that residents believed they spent over 4 hours per day documenting on the computer.

Other studies have used direct observation or intermittent pages instructing the resident to log the specific task being performed at that time in order to define EHR usage. In 2012, Fletcher et al.16 noted that 40% of an internal medicine resident’s day was spent performing computer work which was concordant with Oxentenko’s 2012 study18 reporting 49% of activities as computer related. While our results are consistent with the existing literature, our methodology utilized automated login and logout timestamp data directly from the EHR and allowed for every minute of the day to be accounted for over a 10-month span. This approach also enhances generalizability to surgical programs at other academic centers given that it allowed us to include all levels of trainees within the surgical program.

Furthermore, the current study adds original knowledge to this topic by identifying the amount of EHR usage taking place after scheduled duty hours and how overall EHR usage time impacts the monthly operative case volume of residents. We found interns and residents on rotations with solely daytime shifts spent substantially more time, over 20% of total time, using the EHR after work hours compared to 7% of time for those on dedicated night shifts. This difference is likely explained by the decrease in operative cases and less rounding time during night shifts, which allows for more time to complete documentation requirements. These data also suggest the burden of paperwork on residents, and on interns in particular, is too great to be routinely completed within allotted work hours. This time spent in the EHR during off hours is an important target for improvement initiatives given the potential contribution of this residual work on resident well-being.

Surprisingly, there was no overall correlation between EHR usage time and monthly case logs reported by residents. There was a negative correlation seen among CY2 residents, which was expected at our institution given the structure within the residency program. The schedules of CY2 residents consist of mostly consulting rotations and work in the intensive care units. The rotations lend themselves to a higher paperwork burden that would have a greater impact on the ability to operate compared to other clinical years. However, the overall lack of correlation across the program is likely due to an underestimation of the documentation burden associated with operating. For instance, a resident uses the EHR to review patient information and imaging studies prior to an operation. Additionally, postoperative documentation requires numerous EHR encounters by the resident in order to complete a brief operative note, operative note, and a postoperative progress note. While it is encouraging that case logs are not negatively correlated with EHR usage, the impact of the EHR on case logs may be mitigated by the amount of EHR usage taking place after scheduled work hours. It is likely residents are not willing to forego an operative experience to complete required documentation, which then leads to time in the EHR at the end of the workday or off hours.

While interesting, the results of this investigation are not without limitations. There is always the potential of unmeasured confounders when performing a large retrospective review of data. While we aimed to analyze the largest cohort over the longest timeframe possible, granularity had to be sacrificed. EHR usage was identified by login and logout timestamps, but the type of work being performed within the EHR is undetermined and outside the scope of this study. Furthermore, while all EMR time was included, we were unable to identify the type of Epic interface being used for each login timestamp such as in-patient, out-patient, or remote. There is likely an association between out-patient clinic time and EHR usage, while after hours use is more likely to be via remote login. Unfortunately, the current analysis was not granular enough to answer these hypotheses, which represent opportunities for future investigation. The analysis included to determine the amount of time spent using the EHR outside of regularly scheduled work hours had to be limited to the resident rotations with exclusively day or night shifts. This portion of the study is still generalizable to rotations with rotating schedules by extrapolating to day shifts and night shifts on a weekly basis. Additionally, the exclusion of VA and handheld device EHR usage time likely leads to an underestimation of overall EHR usage. By utilizing a sensitivity analysis, we were able to quantify the amount of time underestimated by inclusion of the VA rotations in the overall analysis as 1.2 hours per day and 3.6 hours per week.

Now that we have produced the first quantitative measure of the EHR documentation burden on general surgery residents, we can use these data to better characterize the impact of EHR use on resident education, well-being, and burnout, as well as the clinical and educational relevance of EHR documentation. This documentation burden has the potential to have a higher negative impact on the education of surgical or procedural residents given the extra element of training that includes the need for development of technical skills. As time during residency is a limited resource, every unnecessary minute spent documenting reduces the time that could be allotted for more important activities such as simulation practice, didactic sessions, or individual time with faculty in the operating room, on the wards, or in clinic. This study is only a starting point to define and quantify the landscape of documentation during general surgery residency, and the results of future studies should lead to the implementation of professional development programs, medical scribes, or changes to the intrinsic structure of the EHR system to improve efficiency and decrease the burden of charting by surgical residents.

CONCLUSION

In the era of a maximum 80-hour work week, general surgery residents spend almost 24 hours per week utilizing the EHR. This accounts for 38% of the actual duty hours worked by the residents in this investigation. A quarter of EHR usage by residents occurred outside of scheduled work hours, demonstrating the difficulties of completing paperwork as part of the regular work day. There was no overall correlation between EHR usage and monthly operative case logs.

ACKNOWLEDGMENT

We would like to thank Chandra Almond and Mary Beth Davis as our data managers along with Dr. Shanna Sprinkle as co-founder of the Duke Surgical Education Research Group (SERG) which provided the forum in which this project idea was born.

Funding:

This study was supported by the Duke AHEAD Learning Environment Grant (internal to our institution). MLC is supported by a National Institutes of Health T32 Training Grant with grant number T32HL069749.

Disclosures:

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

This study was presented as an oral paper presentation at the Association of Program Directors in Surgery Annual Meeting taking place at the 2018 Surgical Education Week in Austin, TX, May 1–5, 2018.

REFERENCES

  • 1.Obey DHR. In: 1 – 111th Congress (2009–2010): American Recovery and Reinvestment Act of 2009; 2009. https://www.congress.gov/bill/111th-congress/house-bill/1. Accessed March 11, 2018.
  • 2.Ellaway RH. Medium, message, panopticon: the electronic health record in residency education. J Grad Med Educ. 2016;8:104–105. 10.4300/JGME-D-15-00569.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Greenhalgh T, Potts HWW, Wong G, Bark P, Swinglehurst D. Tensions and paradoxes in electronic patient record research: a systematic literature review using the meta-narrative method. Milbank Q. 2009;87:729–788. 10.1111/j.1468-0009.2009.00578.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Siegler JE, Patel NN, Dine CJ. Prioritizing paperwork over patient care: why can’t we do both? J Grad Med Educ. 2015;7:16–18. 10.4300/JGME-D-14-00494.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Levin JC, Hron J. Automated reporting of trainee metrics using electronic clinical systems. J Grad Med Educ. 2017;9:361–365. 10.4300/JGME-D-16-00469.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Martin SK, Tulla K, Meltzer DO, Arora VM, Farnan JM. Attending physician remote access of the electronic health record and implications for resident supervision: a mixed methods study. J Grad Med Educ. 2017;9:706–713. 10.4300/JGME-D-16-00847.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Christino MA, Matson AP, Fischer SA, Reinert SE, DiGiovanni CW, Fadale PD. Paperwork versus patient care: a nationwide survey of residents’ perceptions of clinical documentation requirements and patient care. J Grad Med Educ. 2013;5:600–604. 10.4300/JGME-D-12-00377.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 8.Clynch N, Kellett J. Medical documentation: part of the solution, or part of the problem? A narrative review of the literature on the time spent on and value of medical documentation. Int J Med Inform. 2015;84:221–228. 10.1016/j.ijmedinf.2014.12.001. [DOI] [PubMed] [Google Scholar]
  • 9.Davis J, Riesenberg LA, Mardis M, et al. Evaluating outcomes of electronic tools supporting physician shift-to-shift handoffs: a systematic review. J Grad Med Educ. 2015;7:174–180. 10.4300/JGME-D-14-00205.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Kuhn T, Basch P, Barr M, Yackel T. Medical informatics committee of the American College of Physicians. Clinical documentation in the 21st century: executive summary of a policy position paper from the American College of Physicians. Ann Intern Med. 2015;162:301–303. 10.7326/M14-2128. [DOI] [PubMed] [Google Scholar]
  • 11.Oddone E, Guarisco S, Simel D. Comparison of housestaff’s estimates of their workday activities with results of a random work-sampling study. Acad Med. 1993;68:859–861. http://www.ncbi.nlm.nih.gov/pubmed/8216660. Accessed August 22, 2017. [PubMed] [Google Scholar]
  • 12.Oxentenko AS, West CP, Popkave C, Weinberger SE, Kolars JC. Time spent on clinical documentation: a survey of internal medicine residents and program directors. Arch Intern Med. 2010;170:377–380. 10.1001/archinternmed.2009.534. [DOI] [PubMed] [Google Scholar]
  • 13.Nuovo J, Hutchinson D, Balsbaugh T, Keenan C. Establishing electronic health record competency testing for first-year residents. J Grad Med Educ. 2013;5:658–661. 10.4300/JGME-D-13-00013.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Butler RJ, Johnson WG. Rating the digital help: electronic medical records, software providers, and physicians. Int J Heal Econ Manag. 2016;16:269–283. 10.1007/s10754-016-9190-8. [DOI] [PubMed] [Google Scholar]
  • 15.Carayon P, Wetterneck TB, Alyousef B, et al. Impact of electronic health record technology on the work and workflow of physicians in the intensive care unit. Int J Med Inform. 2015;84:578–594. 10.1016/j.ijmedinf.2015.04.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Fletcher KE, Visotcky AM, Slagle JM, Tarima S, Weinger MB, Schapira MM. The composition of intern work while on call. J Gen Intern Med. 2012;27:1432–1437. 10.1007/s11606-012-2120-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Guarisco S, Oddone E, Simel D. Time analysis of a general medicine service: results from a random work sampling study. J Gen Intern Med. 1994;9:272–277. http://www.ncbi.nlm.nih.gov/pubmed/8046530. Accessed August 22, 2017. [DOI] [PubMed] [Google Scholar]
  • 18.Oxentenko AS, Manohar CU, McCoy CP, et al. Internal medicine residents’ computer use in the inpatient setting. J Grad Med Educ. 2012;4:529–532. 10.4300/JGME-D-12-00026.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Sinsky C, Colligan L, Li L, et al. Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties. Ann Intern Med. 2016;165:753–760. 10.7326/M16-0961. [DOI] [PubMed] [Google Scholar]
  • 20.Robertson SL, Robinson MD, Reid A. Electronic health record effects on work-life balance and burnout within the I 3 population collaborative. doi: 10.4300/JGME-D-16-00123.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Atwater AR, Rudd M, Brown A, et al. Developing teaching strategies in the EHR era: a survey of GME experts. J Grad Med Educ. 2016;8:581–586. 10.4300/JGME-D-15-00788.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Schenarts PJ, Schenarts KD. Educational impact of the electronic medical record. J Surg Educ. 2012;69:105–112. 10.1016/j.jsurg.2011.10.008. [DOI] [PubMed] [Google Scholar]
  • 23.Gagliardi JP, Turner DA The electronic health record and education: rethinking optimization. J Grad Med Educ. 2016;8:325–327. doi: 10.4300/JGME-D-15-00275.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Stroup K, Sanders B, Bernstein B, Scherzer L, Pachter L. A new EHR training curriculum and assessment for pediatric residents. Appl Clin Inform. 2017;8:0994–1002. 10.4338/ACI-2017-06-RA-0091. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (RED-Cap)a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42:377–381. 10.1016/j.jbi.2008.08.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.R Development Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: The R Foundation for Statistical Computing; 2011. https://www.gbif.org/tool/81287/r-a-language-and-environment-for-statistical-computing. Accessed March 30, 2018. [Google Scholar]
  • 27.Chen L, Guo U, Illipparambil LC, et al. Racing against the clock: internal medicine residents’ time spent on electronic health records. J Grad Med Educ. 2016;8:39–44. 10.4300/JGME-D-15-00240.1. [DOI] [PMC free article] [PubMed] [Google Scholar]

RESOURCES