Effect of the Adoption of a Comprehensive Electronic Health Record on Graduate Medical Education: Perceptions of Faculty and Trainees

Abstract

Objectives:

Health systems are adopting electronic health records (EHRs). There are few studies on the effects of EHR implementation on graduate medical education. The authors sought to longitudinally assess perceptions of the impact of EHRs on graduate medical education during implementation and 2 years after implementation.

Methods:

A survey was distributed to faculty and trainees during the first year (2013) of adoption of the EHR system. A follow-up survey was distributed 2 years later (2015). The [chi]² test was used to compare the quantitative responses, and factor analysis was conducted to identify correlations between items. Free text responses were analyzed qualitatively.

Results:

The initial survey (in 2013) included 290 faculty and 106 trainees; the follow-up survey (in 2015) included 353 faculty and 226 trainees. In 2013, respondents had a positive impression of EHRs. During the implementation phase, participants believed that face-to-face teaching was negatively affected (P = 0.001). Faculty believed EHRs had a negative effect on trainees’ ability to take a history/conduct physical examinations (P = 0.002) and to formulate a differential diagnosis/plan independently (P = 0.003). In 2015, faculty opinions of the impact of the EHR remained unchanged; trainee responses were more positive than in 2013 in some areas. Qualitative analysis showed that the most frequent strategies to enhance the educational process were the development of EHR skills and improved chart access and note assistance.

Introduction

The role of the electronic health record (EHR) in medical education and healthcare delivery continues to grow in response to increasing demands for greater efficiency in health care.1 With growth of the EHR came early signs of its impact on trainee and faculty satisfaction because of advancements in technology and implementation of electronic order systems 2; however, little evidence has been documented regarding its impact on education. Strategies and methodology for effective teaching with the EHR have yet to be identified, and no formal training for leveraging its educational potential exists.1,3–6 To our knowledge, there are no curricula focused on incorporating the EHR into education. This deficiency may be the result in part of the perceived decrease in enthusiasm and distraction for teaching that EHR implementation has created.5

Given the need to better understand the effects of the EHR on graduate medical education (GME), our study examined the perceptions of faculty and trainees (residents and fellows) during the introduction of a comprehensive EHR into a large academic health center (as defined by the Association of Academic Health Centers).7 Faculty and trainees were surveyed to determine their perceptions of the effect of this comprehensive EHR on GME during implementation and 2 years after implementation.

Methods

This study consists of two cross-sectional surveys of faculty and trainees of a single large academic health center (89 programs and 927 trainees in 2016–2017)8 to assess perceptions of the effect of the EHR on GME over time. Participants were surveyed during the implementation phase of the adoption of a comprehensive EHR (in 2013) and again 2 years later (in 2015). Our institutional review board determined the study to be exempt.

Setting

This study was conducted at a large academic health center. Before implementing the comprehensive EHR system, the institution used a variety of medical record methods that varied across specialties, patient care settings, and locations. Outpatient medical record platforms were variable, most frequently consisting of physician dictation into office-based paper and electronic charts. Many participants also had prior exposure to the comprehensive EHR used by the US Department of Veterans Affairs medical system.

Participants and Survey

All clinical faculty and GME trainees at the institution were asked via e-mail to participate in the online survey using the institution’s Office of Clinical Research as an honest broker. The deidentified data were collected and managed using REDCap.⁹ No compensation was offered to the participants.

The survey was anonymous and consisted of questions on demographics, timing or “wave” of implementation, prior experience with EHR use, and perception of the effect of the EHR on GME, with separate faculty (supplemental A) and trainee versions (supplemental B). The survey instrument consisted of 10 questions. A 5-point scale, ranging from very negative (1) to very positive (5) was used to assess participant perceptions. In addition, an open-ended question was included to solicit strategies for the successful integration of EHRs into education. The nonvalidated survey was developed by the authors (A.R.A., J.S.W., R.B., W.R.L., D.T., J.A.R.), who are program directors and education experts. We searched PubMed using the terms “resident education” and “Electronic Health Record” and reviewed each article (and references) for concepts important to EHRs and designed the survey questions based on these concepts.^1,4,10–14

Initial Survey

The survey was initially distributed in 2013, during the first year of the institution-wide adoption of a comprehensive EHR. The 2013 survey included only outpatient providers due to the timing of EHR training and rollout. The survey was distributed to all 2306 School of Medicine faculty and 980 GME trainees. Participants (85) were excluded if they self-reported as inpatient-only providers.

Follow-Up Survey

A follow-up survey was distributed 2 years later, in 2015, to 1681 School of Medicine clinical faculty and 984 GME trainees. All of the questions from the initial 2013 survey were included. Because all of the clinical settings were using the new comprehensive system, both inpatient and outpatient providers were included.

Data Analysis

Responses were compared between faculty and trainees and between 2013 and 2015. Individual questions were evaluated by provider type (faculty vs trainee) and over time using the [chi]² test. A significant P value indicates that differential trends were detected in the ordinal scores of the survey items.

Factor analysis was used to explore which survey items were highly correlated with one another and, as such, tap into similar conceptual dimensions. This method is frequently used to create subscales such as with the 36-Item Short Form Survey.15 The aims were to determine the number of factors, or potential subscales, for the 2013 survey; inspect how those factors agree or differ by type of respondent (faculty vs trainee) for the questions common to both; and inspect how the factors differ by respondent type between 2013 and 2015. Technically, the varimax rotation was used to obtain a simplified factor structure. Questions were considered sufficiently belonging to a factor with a loading of 40 (raw factor loading score × 100). The factor analysis was conducted using SAS version 9.3 (SAS Institute, Cary, NC).

Responses to the open-ended questions were analyzed quantitatively by one author (A.R.A.) and similar responses were grouped by theme, identifying educational strategies. All reasonable responses were included to facilitate further innovation by the reader.

Results

The demographics of the participants are described in Table 1. The initial survey in 2013 included 290 outpatient faculty providers and 106 trainees (overall 14.6% response rate). The follow-up survey in 2015 included 329 faculty and 211 trainees (20.3% response rate).

Table 1.

	Faculty		Trainees
Characteristics, n (%)	2013 N = 290	2015 N = 353	2013 N = 106	2015 N = 226
Sex
Female	113 (40.07%	129 (41.48)	53 (50.48)	97 (49.24)
Male	164 (58.16)	179 (57.56)	50 (47.62)	99 (50.25)
Prefer not to say	5 (1.77)	3 (0.96)	2 (1.90)	1 (0.51)
P		Faculty and trainees in 2013, P = 0.08
		Faculty and trainees in 2015, P = 0.37
Age, y
25–30	1 (0.35)	2 (0.64)	58 (55.24)	95 (47.98)
31–36	38 (13.29)	28 (9.00)	42 (40.00)	89 (44.95
37–42	66 (23.08)	64 (20.58)	4 (3.81)	7 (3.54)
43–48	40 (13.99)	57 (18.33)	1 (0.95)	5 (2.53)
49–53	54 (18.88)	46 (14.79)	0	1 (0.51)
54–59	47 (16.43)	61 (19.61)	0	1 (0.51)
60–65	28 (9.79)	37 (11.90)	0	0
66–71	11 (3.85)	14 (4.50)	0	0
>71	1 (0.35)	2 (0.64)	0	0
P		Faculty and trainees in 2013, P = 0.001
		Faculty and trainees in 2015, P = 0.001
EHR experience
Previous
Significant	86 (29.6)	133 (37.8)	55 (52.4)	250 (62.0)
Some	82 (28.3)	104 (29.6)	35 (33.3)	61 (27.0)
Used for data gathering	51 (17.6)	53 (15.1)	8 (.73)	9 (4.0)
Minimal	71 (24.5)	62 (17.6)	7 (3.7)	16 (7.1)
P		Faculty and trainees in 2013, P = 0.001
		Faculty and trainees in 2015, P = 0.001

Demographics of study populations: 2013 faculty: 2306 recipients, 290 respondents, 12.6% response rate. 2015 faculty: 1681 recipients, 353 respondents, 21% response rate. 2013 trainees: 980 recipients, 106 respondents, 10.8% response rate. 2015 trainees: 954 recipients, 226 respondents, 23% response rate. In 2013, 85 respondents were excluded because they did not work in the outpatient setting. Attending vs trainee status was not recorded; therefore, 2013 response rate is artificially low in both groups because 85 participants were not included in either tall: EHR, electronic health record.

^aNot all the participants completed all portions of the survey.

There was a trend toward more men in the faculty group in 2013, but this was not statistically significant (P = 0.08). There was no difference in sex distribution between faculty and trainees in 2015 (P = 0.37). Age differences between faculty and trainees were statistically significant in both 2013 and 2015 (P < 0.001), with faculty older than trainees. Trainees had more EHR experience than faculty in both 2013 and 2015 (P < 0.001).

Perceptions of Faculty and Trainees Over Time

Initial Survey

Overall, the 2013 survey results indicate that the faculty perceived the effects to be negative or neutral (Table 2), with the most negative effects on “face-to-face learning time” between faculty and trainees and trainee “development of effective interactions with patients.” Similarly, overall, the trainees perceived the effects to be negative or neutral (Table 2), with the notable exception of the trainee “ability to reduce errors/improve patient safety,” in which the effect of the EHR was perceived to have a positive effect. It is interesting that 55% of faculty and 76% of trainees had a positive or very positive overall impression of the use of an EHR in 2013.

Table 2.

	2013					2015					p
	Percentage					Percentage					Peer group change over time	Faculty vs trainee change over time
	Very negative	Negative	No Effect	Positive	Very Positive	Very negative	Negative	No Effect	Positive	Very Positive
Ability to take history and conduct physical examination												<0.001
Faculty	6.2	28.2	41.7	12.8	1.1	7.7	34.8	45.2	12.0	0.3
Trainees	7.7	21.2	41.4	23.1	6.7	0	13.6	48.6	28.5	9.4	0.004
Formulate differential diagnosis and plan independently												<0.001
Faculty	5.5	27.8	57.5	8.8	0.4	7.3	29.7	52.6	9.8	0.6
Trainees	2.9	13.5	68.3	14.4	1.0	0	5.6	64.3	21.6	8.5	<0.001
Teaching interaction with faculty
Faculty	13.0	38.7	28.7	8.9	0.7	10.4	35.2	40.4	12.8	1.2
Trainees	ND	ND	ND	ND	ND	1.9	20.6	57.9	15.9	3.7
Face-to-face learning time
Faculty	15.9	48.3	29.9	5.2	0.7	15.2	43.9	33.8	6.4	0.6	0.007
Trainees	8.37	35.9	49.5	5.8	0	2.8	33.8	50.2	10.3	2.8
Ability to learn												<0.001
Faculty	5.6	31.2	50.2	11.9	1.1	5.2	29.5	52.9	11.7	0.6
Trainees	6.8	22.3	60.2	9.7	1.0	2.4	15.6	56.1	22.2	3.5	<0.001
Development of effective interactions with patients												0.017
Faculty	12.1	46.7	34.2	6.3	0.7	13.5	43.4	36.7	6.1	0.3
Trainees	10.6	50.0	25.0	14.4	0	7.1	35.9	38.7	14.6	3.8	0.016
Ability to reduce errors/improve patient safety
Faculty	ND	ND	ND	ND	ND	6.7	17.4	37.6	35.8	2.5
Trainees	2.9	9.6	25.0	47.1	15.4	2.3	8.9	25.7	51.4	11.7
Compliance with duty hours
Faculty	ND	ND	ND	ND	ND	4.6	23.5	60.1	11.2	0.6
Trainees	6.7	23.1	60.6	7.7	1.9	7.0	25.4	48.8	15.5	3.3
Satisfaction with career choice
Faculty	8.5	27.0	55.6	7.4	1.5	6.8	28.2	56.4	7.7	0.9
Trainees	5.8	21.2	60.6	9.6	2.9	2.4	16.9	62.0	15.5	3.3

P values are shown only for those items in which a statistically significant difference was noted between 2013 and 2015. For the following items, the difference in change over time for faculty was statistically significant: face-to-face learning time (P=0.007). For the following items, the difference in change over time for trainees was statistically significant: Ability to take history and conduct physical examination (P=0.004), ability to formulate differential diagnosis and plan independently (P<0.001), ability to learn (P<0.001), and ability to develop effective interactions with patients (P=0.016). For the following items, the difference in change over time for faculty versus trainees was statistically significant: ability to take history and conduct physical examination (P<0.001), ability to formulate differential diagnosis and plan independently (P<0.001), ability to learn (P<0.001), and ability to develop effective interaction with patients (P=0.017). ND, not done because of technical issue during survey administration.

^aNot all of the participants completed all portions of the survey

Follow-Up Survey

Virtually no change was noted in the faculty responses between 2013 and 2015, whether only outpatient faculty (data not shown) or the entire clinical faculty (Table 2) was included, with the perceived effects being negative or neutral on GME. The trainee responses did change during the 2-year period, how- ever, becoming more positive in five categories (Table 2). Fifty- three percent of faculty and 85% of trainees had a positive or very positive overall impression of the use of an EHR in 2015.

Factor Analysis for Potential Correlations

The factor analysis identified three potential factors or groups of questions in which survey items were highly correlated with one another (Table 3), diagrammatically represented in the Figure. For the faculty responses, the questions fell into three main groups: questions related to providing care (factor 2 in Table 3), the teaching interaction (factor 1), and career satisfaction (factor 3). For the faculty, these factors were similar in both 2013 and 2015, and in both years the “trainee ability to learn” item did not correlate clearly in either the “providing care” or “teaching interaction” factor, so it may be a single item that stands alone. For the trainee responses the factor analysis story is less clear because for both years the questions fell into two main groups: providing care (factor 3) and interpersonal inter- actions (factor 1), with several questions with high loads in two factors. The overall factor analysis story is that faculty-item responses can be logically summarized clearly in subscales, whereas trainee responses are more disparate and less easily able to be summarized.

Table 3.

Faculty	2013=290			2015 N = 353
	Factor 1	Factor 2	Factor 3	Factor 1	Factor 2	Factor 3
Trainees’ ability to take history and conduct physical examination	31	79	30	18	86	21
Trainees’ formulation of differential diagnosis and plan independently	24	85	20	25	83	22
Teaching interaction between you and the trainee	82	27	22	75	34	35
Face-to-face teaching time between you and the trainee	80	16	41	86	14	33
Trainees’ ability to learn	61	55	22	53	54	32
Trainees’ development of effective interaction with patients	74	34	19	58	53	16
Trainees’ satisfaction with career choice	38	23	72	29	17	86
Your satisfaction with career choice	20	25	86	32	36	72
Trainees	2013=106			2015 N = 226
	Factor 1	Factor 2	Factor 3	Factor 1	Factor 2	Factor 3
Ability to take history and conduct physical examination	23	35	78	20	22	85
Formulation of differential diagnosis and plan independently	24	16	86	30	8	84
Face-to-face teaching time with faculty	90	14	18	80	13	18
Ability to learn	65	54	21	65	45	27
Development of effective interaction with patients	58	23	43	78	24	18
Ability to reduce errors/improve patient safety	24	81	14	19	94	18
Compliance with duty hours	46	43	24	78	1	24
Satisfaction with career choice	19	79	25	62	21	46

Bold typeface indicates significant interaction.

Factor analysis identified 3 potential factors, or groups of questions, in which survey items were highly correlated with one another. Strength of interaction (higher number indicates greater interaction). For faculty: factor 1 = teaching interact, factor 2 = providing care, and factor 3 = career satisfaction. For trainees: factor 1 = interpersonal interactions, and factor 3 = providing care; factor 2 did not clearly differentiate for trainees. Bold typeface indicates statistical significance.

^aNot all of the participants completed all portions of the survey.

Strategies That Enhance the Educational Process in the Medical Setting Using an EHR

Free-text responses detailing strategies that faculty and trainees identified to enhance the educational process in the medical setting using an EHR are presented in Table 4. Higher-frequency strategies with shared agreement between faculty and trainees include “develop EHR skills, templates, preload notes,” “improved chart access” (eg, history, provider notes, chart previews, data trends, laboratory values, x- rays, photographs), and “note assistance” (eg, trainee completes note before staffing, scribes, trainee types in room, enhanced dictation, more trainees/midlevel providers, faculty type in room, extender/nursing note assistance).

Table 4.

	Faculty frequency	Trainee frequency
Develop EHR skills, “smartphrases”, templates, preload notes	31	11
Improve chart access (history, provider notes, chart preview, data trends, laboratory values, x-rays, phots)	28	36
Document at home/later	17	--
Discuss case in real time	16	--
Write and review notes with trainee	13	--
Links and online resources	11	4
Note assistance (trainee completes notes before staffing, scribes, trainee types in room, enhanced dictation, more trainee/midlevel providers, faculty type in room, extender/nursing note assistance	11	5
Teach trainee away from EHR
Remove copy and paste, copy forward	6	1
Decrease time to complete documentation, decrease note quality	5	2
Patient chart entry and review	4	--
Autoprompts, lists	4	5
Decrease patient volume	3	--
Search functions, filters	3	2
Enhance institution’s IT services, enhance Epic	3	1
Improve efficiency and communication	3	7
Protocols for productivity, safety, and quality	2	2
Patient education tools	2	2
Rearrange room	2	--
Faculty role model medical visit to trainee	1	--
Avoid using the EHR	1	--
Mobile computer workstations	1	1
Research opportunities	1	--
Shadow-note writing	1	--
Spend less face-to-face teaching time with trainees	1	--
Spend more time with patient	1	--
Weekly care management meeting	1	--
Set aside time for teaching	--	1
Lear from colleagues	--	1

EHR, electronic health record;

IT, information technology.

Discussion

During implementation of the comprehensive EHR, both faculty and trainees had negative or neutral perceptions of the effects of the EHR on GME. Faculty participants were more negative in their overall assessment, and both groups had concerns about face-to-face teaching time and trainee development of effective interactions with patients. The overall impression on the use of the EHR was positive for >50% of faculty and trainees at both points in time, however. This apparent discrepancy reflects the complex relation that doctors have with the EHR—from the hope based on its potential to the practical challenges of using it day to day.

It is interesting that 2 years after EHR implementation, faculty perception of face-to-face teaching time and trainee development did not change. Despite being more familiar with the EHR system and presumably having fewer work stoppages and other technical issues, faculty continued to have negative perceptions about the EHR impact on trainees. Trainees were more positive about these categories but still maintained their negative assessment. It is possible that trainees have adapted to the EHR system more readily and are more adept at using an EHR for patient care. Possible explanations for easier adaptation for the trainees are their younger age and lack of prior exposure to nonelectronic records. Decker et al showed that from 2002 to 2011, physicians younger than 45 years were more than twice as likely to adopt an EHR compared with providers older than 55 years.16 Although the EHR may not have differed during the period studied, it is possible that the preparation that later trainees received was improved compared with that of the faculty and initial trainees.

The faculty believed that education could be improved if they documented at home or documented later. This allows more time for teaching, but it is not likely to improve provider or patient satisfaction. Both faculty and trainees recommended, with more support from faculty, to teach trainees away from the EHR. Although this could allow for active, traditional medical teaching, it does not address the reality of current medical practice. The recommendation to remove copy and paste and copy forward functions has been introduced elsewhere in the medical literature.4,13 Proponents for the removal of these functions suggest that trainees should not use these functions before they have fully developed their medical skills and that these functions increase the risk of error.4 Others have suggested the need to train residents and fellows to appropriately use this technology so that it can be used safely.13

Both groups agreed that improved chart access could enhance the educational process, allowing for quick access to patient records for patient care and educational activities. Discussing cases in real time and writing and reviewing the notes with the trainee are similar concepts that faculty believed could improve teaching and learning. These approaches allow for more interaction and have the potential to be in line with faculty and trainee preferences. Finally, links and online resources have the potential to improve not only efficiency but also access to teaching tools and important medical content.

There are several limitations to this study. The survey took place at one institution with one EHR, potentially limiting the applicability of the results. The situation studied, that is, converting from electronic, dictated, and paper-based records to a single comprehensive system, likely will be a common occurrence as health systems respond to administrative drivers. The initial survey (2013) was not validated but was based on our detailed study of relevant GME and EHR concepts; to keep responses consistent we decided to use the same survey in follow-up (2015). Our response rate was lower (14%) in 2013 compared with the 20% in 2015, which is consistent with participants who were busy learning new technology in 2013; the low response rate overall was consistent with other physician survey studies.17,18 The survey was distributed via e-mail and no compensation was offered; participants may have self-selected for those who held strong opinions regarding the EHR in the setting of GME. The transient nature of GME complicated the study because it was guaranteed that at least a portion of the trainee participants in 2013 were no longer available for participation in 2015. Finally, it was not feasible to link participants’ responses between 2013 and 2015, which may have offered stronger evidence for trends in perceptions over time, and the two time points included both new and experienced trainees.

Further studies are required to determine which strategies will work best in the quest to enhance the educational process in the setting of the EHR. The potential next steps include using comments to develop interventions to enhance teaching and learning. Given the consistent negative perceptions of faculty, it may be of value to focus first on this group.

Conclusions

The overall impression of the EHR was positive in both faculty and trainees, but perceptions of specific aspects of the EHR on GME ranged from negative to neutral. Although the faculty appeared to have reservations about its impact on education, the trainees’ perceptions became more positive over time, offering hope for the future.

Figure 1.

Key Points.

• A survey of trainees and faculty provided meaningful results regarding the perception of the effect of the EHR on graduate medical education.

• The overall perception of the EHR was positive in both faculty and trainees, but perceptions of specific aspects of the EHR on graduate medical education ranged from negative to neutral.

• Although the faculty appeared to have reservations about its impact on education, the trainees’ perceptions became more positive over time, offering hope for the future.