Two-year longitudinal assessment of physicians’ perceptions after replacement of a longstanding homegrown electronic health record: does a J-curve of satisfaction really exist?

Abstract

This report describes a 2-year prospective, longitudinal survey of attending physicians in 3 clinical areas (family medicine, general pediatrics, internal medicine) who experienced a transition from a homegrown electronic health record (EHR) to a vendor EHR. Participants were already highly familiar with using EHRs. Data were collected 1 month before and 3, 6, 13, and 25 months post implementation. Our primary goal was to determine if perceptions followed a J-curve pattern in which they initially dropped but eventually surpassed baseline measures. A J-curve was not found for any measures, including workflow, safety, communication, and satisfaction. Only the reminders and alerts measure dropped and then returned to baseline (U-curve); a few remained flatlined. Most dropped and remained below baseline (L-curve). The only measure that remained above baseline was documenting in the exam room with the patient. This study adds to the literature about current controversies surrounding EHR adoption and physician satisfaction.

INTRODUCTION

The United States has experienced substantial growth in the number of hospitals and ambulatory care providers adopting electronic health records (EHRs), in part due to the Health Information Technology for Economic and Clinical Health (HITECH) Act.^1–3 The benefits of EHRs have been well described.^4–7 Yet EHRs remain controversial with respect to cost,⁸ productivity and efficiency,^9–17 patient-provider communication,^10,13,18 and physician job satisfaction.^19–23

Recent EHR adoptions provide opportunities to study “second-generation” implementations, in which users switch from legacy electronic systems to newer vendor systems,^24–27 and where users have existing familiarity with EHRs. This familiarity is important when judging the impact of an EHR, since the consternation that could arise from “first-generation” implementation, ie, switching from a paper-based to an electronic workflow, should be minimized. Additionally, such users should already be computer literate,^28,29 a topic of concern for first-generation implementations.^30,31

While there have been multiple studies of EHR and other health IT implementations,¹⁹ only some have followed the same providers longitudinally,^{10,24,30,32–40} and even fewer have measured these changes for 2 or more years.^30,37,38,40 Understanding how perceptions about an EHR change over time is important, because single time point measurements cannot reveal trends.

Here we describe a 2-year prospective, longitudinal survey of ambulatory care providers in 3 clinical areas (family medicine, general pediatrics, and internal medicine) where an EHR implementation occurred within a longstanding, mature health information technology environment. The survey assessed perceptions about a second-generation EHR among physicians with experience using an existing homegrown EHR. We tested the hypothesis that perceptions would follow a J-curve pattern, in which measures would initially decrease but then rise to surpass their baseline (pre-implementation) levels. Such a pattern, well described in other settings,⁴¹ should signify a successful EHR adoption, even if satisfaction temporarily dropped.

METHODS

Institutional setting

In 1998 the University of Michigan Health System (UMHS) implemented a homegrown EHR, CareWeb, which was used by all clinicians for creating and viewing documentation, as well as viewing test results, vital signs, and other data. CareWeb was integrated with multiple vendor systems, including an outpatient e-prescribing system.

In August 2012, UMHS transitioned all ambulatory providers to a primary vendor system, Epic (Epic Systems, Verona, WI, USA), locally renamed MiChart. Prior to implementation, all physicians received training, and clinical groups had invested substantial resources developing customized content, including order sets, documentation templates, and other anticipated time-saving components.

Survey development

We developed a brief survey with input from physicians in 3 clinical departments: family medicine, pediatrics, and internal medicine. Experts in clinical and health informatics and survey design also provided input. Questions covered themes including data entry, communication, safety, reminders and alerts, workflow and efficiency, job satisfaction, and overall perceptions about the transition from the old to the new EHR (see Appendix). Most questions were based on 5-point attitude scales (“disagree strongly” to “agree strongly”) and 5-point behavioral frequency scales (“never” to “all of the time”). Participants could also leave free-text comments. We used the online survey platform Qualtrics (Provo, UT, USA). This study was reviewed and exempted by our medical school’s institutional review board.

Survey deployment

Attending physicians in the 3 clinical areas were invited by e-mail to take the survey at baseline 1 month before the implementation, and then 3, 6, 13, and 25 months after the implementation, hereafter referred to as −1, +3, +6, +13, and +25 months. Residents (ie, physicians in training) were not included and no incentives were provided. Only those in the cohort invited to participate in the pre-implementation survey were invited to take subsequent surveys, so as to include only those who had experience with both the old and new EHRs.

Data analysis

Only completed surveys were included in our analysis, using the RR1 definition from the American Association for Public Opinion Research.⁴² For our analysis we used the 2 positive responses for each question (“slightly agree” and “strongly agree”) for the numerator when determining the percentages reported. We calculated 95% binomial confidence intervals with the Pearson-Klopper method, using the “binom” package (v1.1-1) within R v2.15.3. Line graphs depicting responses over time were also made using R. Free-text comments were selected to illustrate various sentiments from the respondents. For internal medicine, results from primary care physicians (eg, general internal medicine) were reported separately from specialty physicians (eg, nephrology) in addition to reporting the aggregate results. The Appendix details the approach for determining curve shapes.

RESULTS

Response rates varied from a high of 76% for general pediatricians at +13 months to a low of 23% for internal medicine specialty physicians at –1 month (Table 1). Longitudinal responses for each group are shown in Table 2, including overall trend lines and curve shape descriptions. The Appendix contains additional analyses demonstrating no evidence of selective attrition among those with positive or negative views at baseline; no difference between the minority of participants who responded at a single time point compared to the majority who responded at multiple time points; and additional longitudinal paired analyses for participants who responded at multiple time points.

Table 1.

Survey response rates, reported as number of responses/total surveys sent^a (% response rate)

Clinical Specialty	−1 month	+3 months	+6 months	+13 months	+25 months
Family Medicine	36/75 (48)	44/75 (59)	40/75 (53)	27/71 (38)	36/67 (54)
General Pediatrics	26/44 (59)	28/43 (65)	31/42 (74)	29/38 (76)	25/35 (71)
Internal Medicine	112/413 (27)	164/413 (40)	122/413 (30)	147/413 (36)	118/413 (29)
Primary Care	37/85 (44)	35/85 (41)	24/85 (28)	34/85 (40)	35/85 (41)
Specialty	75/328 (23)	129/328 (39)	98/328 (30)	113/328 (35)	83/328 (25)
Overall	174/532 (33)	236/531 (44)	193/530 (36)	203/522 (39)	179/515 (35)

^aThe number of potential respondents in the denominator varied each month for family medicine and pediatrics as faculty left the institution permanently or were out temporarily (eg, maternity leave). Internal medicine was not able to provide data on faculty attrition, and thus the denominator for that group remained unchanged throughout the study period.

Table 2.

Survey responses at 5 time points along the EHR implementation

Theme: Data Entry (Location and Timing)
1. Physicians entering data into the medical record (including dictation) on clinical workdays in the patient rooms with the patient, often or all of the time
Clinical Specialty	−1 month	+ 3 months	+ 6 months	+ 13 months	+ 25 months
Family Medicine	7 (19) [8-36]	17 (39) [24-55]	16 (40) [25-57]	16 (59) [39-78]	21 (58) [41-74]
General Pediatrics	4 (15) [4-35]	12 (43) [24-63]	15 (48) [30-67]	12 (41) [24-61]	14 (56) [35-76]
Internal Medicine	14 (13) [7-20]	53 (32) [25-40]	46 (38) [29-47]	49 (33) [26-42]	35 (30) [22-39]
Primary Care	6 (16) [6-32]	11 (31) [17-49]	9 (38) [19-59]	13 (38) [22-56]	13 (37) [21-55]
Specialty	8 (11) [5-20]	42 (33) [25-41]	37 (38) [28-48]	36 (32) [23-41]	22 (27) [17-37]
Overall	25 (14) [10-20]	82 (35) [29-41]	77 (40) [33-47]	77 (38) [32-45]	70 (39) [32-46]	“inverted L curve”

2. Physicians entering data into the medical record (including dictation) on evenings/nights of clinical workdays, often or all of the time
Clinical Specialty	−1 month	+ 3 months	+ 6 months	+ 13 months	+ 25 months
Family Medicine	22 (61) [43-77]	32 (73) [57-85]	28 (70) [53-83]	20 (74) [54-89]	24 (67) [49-81]
General Pediatrics	20 (77) [56-91]	22 (79) [59-92]	25 (81) [63-93]	22 (76) [56-90]	20 (80) [59-93]
Internal Medicine	72 (64) [55-73]	110 (67) [59-74]	74 (61) [51-69]	110 (75) [67-82]	86 (73) [64-81]
Primary Care	22 (59) [42-75]	25 (71) [54-85]	15 (63) [41-81]	25 (74) [56-87]	24 (69) [51-83]
Specialty	50 (67) [55-77]	85 (66) [57-74]	59 (60) [50-70]	85 (75) [66-83]	62 (75) [64-84]
Overall	114 (66) [58-72]	164 (69) [63-75]	127 (66) [59-72]	152 (75) [68-80]	130 (73) [66-79]	“flat line”

3. Physicians entering data into the medical record (including dictation) on days off (weekdays or weekends), often or all of the time
Clinical Specialty	−1 month	+ 3 months	+ 6 months	+ 13 months	+ 25 months
Family Medicine	15 (42) [26-59]	19 (43) [28-59]	19 (48) [32-64]	12 (44) [25-65]	15 (42) [26-59]
General Pediatrics	15 (58) [37-77]	13 (46) [28-66]	16 (52) [33-70]	18 (62) [42-79]	14 (56) [35-76]
Internal Medicine	58 (52) [42-61]	95 (58) [50-66]	58 (48) [38-57]	89 (61) [52-68]	73 (62) [52-71]
Primary Care	22 (59) [42-75]	24 (69) [51-83]	15 (63) [41-81]	25 (74) [56-87]	25 (71) [54-85]
Specialty	36 (48) [36-60]	71 (55) [46-64]	43 (44) [34-54]	64 (57) [47-66]	48 (58) [46-69]
Overall	88 (51) [43-58]	127 (54) [47-60]	93 (48) [41-55]	119 (59) [52-65]	102 (57) [50-64]	“flat line”

Theme: Communication
4. Physicians who agree that the EHRa enables the creation of high-quality documentation
Clinical Specialty	−1 month	+ 3 months	+ 6 months	+ 13 months	+ 25 months
Family Medicine	35 (97) [85-100]	9 (20) [10-35]	8 (20) [9-36]	10 (37) [19-58]	7 (19) [8-36]
General Pediatrics	24 (92) [75-99]	10 (36) [19-56]	16 (52) [33-70]	15 (52) [33-71]	13 (52) [31-72]
Internal Medicine	99 (88) [81-94]	25 (15) [10-22]	18 (15) [9-22]	28 (19) [13-26]	23 (19) [13-28]
Primary Care	33 (89) [75-97]	8 (23) [10-40]	6 (25) [10-47]	10 (29) [15-47]	7 (20) [8-37]
Specialty	66 (88) [78-94]	17 (13) [8-20]	12 (12) [6-20]	18 (16) [10-24]	16 (19) [11-29]
Overall	158 (91) [86-94]	44 (19) [14-24]	42 (22) [17-28]	53 (26) [21-3]	43 (24) [18-31]	“L curve”

5. Physicians who agree that the EHRa does not interfere with the ability to have face-to-face contact with patients
Clinical Specialty	−1 month	+ 3 months	+ 6 months	+ 13 months	+ 25 months
Family Medicine	33 (92) [78-98]	5 (11) [4-25]	5 (13) [4-27]	4 (15) [4-34]	6 (17) [6-33]
General Pediatrics	21 (81) [61-93]	3 (11) [2-28]	2 (6) [1-21]	5 (17) [6-36]	1 (4) [0-20]
Internal Medicine	96 (86) [78-92]	14 (9) [5-14]	14 (11) [6-19]	23 (16) [10-23]	12 (10) [5-17]
Primary Care	32 (86) [71-95]	4 (11) [3-27]	6 (25) [10-47]	5 (15) [5-31]	2 (6) [1-19]
Specialty	64 (85) [75-92]	10 (8) [4-14]	8 (8) [4-15]	18 (16) [10-24]	10 (12) [6-21]
Overall	150 (86) [80-91]	22 (9) [6-14]	21 (11) [7-16]	32 (16) [11-21]	19 (11) [7-16]	“L curve”
6. Physicians who agree that the EHRa supports the ability to interact with patients in a meaningful way
Clinical Specialty	−1 month	+ 3 months	+ 6 months	+ 13 months	+ 25 months
Family Medicine	34 (94) [81-99]	4 (9) [3-22]	6 (15) [6-30]	6 (22) [9-42]	9 (25) [12-42]
General Pediatrics	22 (85) [65-96]	2 (7) [1-24]	5 (16) [5-34]	5 (17) [6-36]	2 (8) [1-26]
Internal Medicine	93 (83) [75-89]	9 (5) [3-10]	6 (5) [2-10]	11 (7) [4-13]	13 (11) [6-18]
Primary Care	30 (81) [65-92]	5 (14) [5-30]	3 (13) [3-32]	5 (15) [5-31]	3 (9) [2-23]
Specialty	63 (84) [74-91]	4 (3) [1-8]	3 (3) [1-9]	6 (5) [2-11]	10 (12) [6-21]
Overall	149 (86) [80-90]	15 (6) [4-10]	17 (9) [6-14]	22 (11) [7-16]	24 (13) [9-19]	“L curve”

Theme: Safety
7. Physicians who agree that the EHRa has improved patient safety by helping to avoid problems or mistakes
Clinical Specialty	−1 month	+ 3 months	+ 6 months	+ 13 months	+ 25 months
Family Medicine	20 (56) [38-72]	6 (14) [5-27]	7 (18) [7-33]	7 (26) [11-46]	15 (42) [26-59]
General Pediatrics	10 (38) [20-59]	5 (18) [6-37]	12 (39) [22-58]	14 (48) [29-67]	15 (60) [39-79]
Internal Medicine	66 (59) [49-68]	21 (13) [8-19]	14 (11) [6-19]	34 (23) [17-31]	38 (32) [24-41]
Primary Care	18 (49) [32-66]	9 (26) [12-43]	6 (25) [10-47]	9 (26) [13-44]	19 (54) [37-71]
Specialty	48 (64) [52-75]	12 (9) [5-16]	8 (8) [4-15]	25 (22) [15-31]	19 (23) [14-33]
Overall	96 (55) [48-62]	32 (14) [8-19]	33 (17) [12-23]	55 (27) [21-34]	68 (38) [31-45]	“L curve”

Theme: Reminders and Alerts
8. Physicians who agree that the EHRa provides reminders about important actions or items (eg, labs, studies, procedures, vaccines, documentation) for which the patient is due that might have otherwise been forgotten or overlooked
Clinical Specialty	−1 month	+ 3 months	+ 6 months	+ 13 months	+ 25 months
Family Medicine	30 (83) [67-94]	14 (32) [19-48]	12 (30) [17-47]	14 (52) [32-71]	26 (72) [55-86]
General Pediatrics	7 (27) [12-48]	11 (39) [22-59]	9 (29) [14-48]	14 (48) [29-67]	15 (60) [39-79]
Internal Medicine	44 (39) [30-49]	37 (23) [16-30]	18 (15) [9-22]	45 (31) [23-39]	49 (42) [33-51]
Primary Care	19 (51) [34-68]	18 (51) [34-69]	12 (50) [29-71]	22 (65) [46-80]	27 (77) [60-90]
Specialty	25 (33) [23-45]	19 (15) [9-22]	6 (6) [2-13]	23 (20) [13-29]	22 (27) [17-37]
Overall	81 (47) [39-54]	62 (26) [21-32]	39 (20) [15-26]	73 (36) [30-43]	90 (50) [43-58]	“U curve”
Theme: Workflow and Efficiency
9. Physicians who agree that the EHRa enables the completion of documentation in a timely manner
Clinical Specialty	−1 month	+ 3 months	+ 6 months	+ 13 months	+ 25 months
Family Medicine	23 (64) [46-79]	15 (34) [20-50]	10 (25) [13-41]	13 (48) [29-68]	16 (44) [28-62]
General Pediatrics	13 (50) [30-70]	7 (25) [11-45]	12 (39) [22-58]	10 (34) [18-54]	13 (52) [31-72]
Internal Medicine	65 (58) [48-67]	16 (10) [6-15]	13 (11) [6-18]	18 (12) [7-19]	17 (14) [9-22]
Primary Care	19 (51) [34-68]	3 (9) [2-23]	3 (13) [3-32]	4 (12) [3-27]	4 (11) [3-27]
Specialty	46 (61) [49-72]	13 (10) [5-17]	10 (10) [5-18]	14 (12) [7-20]	13 (16) [9-25]
Overall	101 (58) [51-65]	38 (16) [12-21]	35 (18) [13-24]	41 (20) [15-26]	46 (26) [20-33]	“L curve”

10. Physicians who agree that the EHRa has eliminated work that they used to have to do
Clinical Specialty	−1 month	+ 3 months	+ 6 months	+ 13 months	+ 25 months
Family Medicine	N/A	9 (20) [10-35]	8 (20) [9-36]	10 (37) [19-58]	7 (19) [8-36]
General Pediatrics	N/A	5 (18) [6-37]	5 (16) [5-34]	4 (14) [4-32]	3 (12) [3-31]
Internal Medicine	N/A	10 (6) [3-11]	3 (2) [1-7]	9 (6) [3-11]	11 (9) [5-16]
Primary Care	N/A	2 (6) [1-19]	0 (0) [0-14]	1 (3) [0-15]	4 (11) [3-27]
Specialty	N/A	8 (6) [3-12]	3 (3) [1-9]	8 (7) [3-13]	7 (8) [3-17]
Overall	N/A	24 (10) [7-15]	16 (8) [5-13]	23 (11) [8-16]	21 (12) [8-17]	“flat line”
11. Physicians who agree that the EHRa has created new work that they now must do
Clinical Specialty	−1 month	+ 3 months	+ 6 months	+ 13 months	+ 25 months
Family Medicine	N/A	42 (95) [85-99]	40 (100) [91-100]	27 (100) [87-100]	36 (100) [90-100]
General Pediatrics	N/A	27 (96) [82-100]	29 (94) [79-99]	27 (93) [77-99]	24 (96) [80-100]
Internal Medicine	N/A	160 (98) [94-99]	118 (97) [92-99]	141 (96) [91-98]	115 (97) [93-99]
Primary Care	N/A	34 (97) [85-100]	23 (96) [79-100]	33 (97) [85-100]	35 (100) [90-100]
Specialty	N/A	126 (98) [93-100]	95 (97) [91-99]	108 (96) [90-99]	80 (96) [90-99]
Overall	N/A	229 (97) [94-99]	187 (97) [93-99]	195 (96) [92-98]	175 (98) [94-99]	“flat line”

Theme: Job Satisfaction
12. Physicians who report being satisfied with their overall job
Clinical Specialty	−1 month	+ 3 months	+ 6 months	+ 13 months	+ 25 months
Family Medicine	33 (92) [78-98]	18 (41) [26-57]	22 (55) [38-71]	13 (48) [29-68]	22 (61) [43-77]
General Pediatrics	22 (85) [65-96]	16 (57) [37-76]	17 (55) [36-73]	19 (66) [46-82]	20 (80) [59-93]
Internal Medicine	77 (69) [59-77]	66 (40) [33-48]	51 (42) [33-51]	63 (43) [35-51]	60 (51) [41-60]
Primary Care	20 (54) [37-71]	14 (40) [24-58]	10 (42) [22-63]	11 (32) [17-51]	19 (54) [37-71]
Specialty	57 (76) [65-85]	52 (40) [32-49]	41 (42) [32-52]	52 (46) [37-56]	41 (49) [38-61]
Overall	132 (76) [69-82]	100 (42) [36-49]	90 (47) [40-54]	95 (47) [40-54]	102 (57) [50-64]	“L curve”

13. Physicians who report that the EHRa has had a positive impact on their job satisfaction
Clinical Specialty	−1 month	+ 3 months	+ 6 months	+ 13 months	+ 25 months
Family Medicine	23 (64) [46-79]	5 (11) [4-25]	2 (5) [1-17]	5 (19) [6-38]	4 (11) [3-26]
General Pediatrics	13 (50) [30-70]	3 (11) [2-28]	5 (16) [5-34]	4 (14) [4-32]	2 (8) [1-26]
Internal Medicine	72 (64) [55-73]	6 (4) [1-8]	2 (2) [0-6]	6 (4) [2-9]	9 (8) [4-14]
Primary Care	21 (57) [39-73]	3 (9) [2-23]	1 (4) [0-21]	2 (6) [1-20]	3 (9) [2-23]
Specialty	51 (68) [56-78]	3 (2) [0-7]	1 (1) [0-6]	4 (4) [1-9]	6 (7) [3-15]
Overall	108 (62) [55-69]	14 (6) [4-10]	9 (5) [2-9]	15 (7) [5-12]	15 (8) [5-13]	“L curve”

Theme: Looking Forward
14. Physicians who agree that MiChart will allow them to provide better care for their patients than CareWeb
Clinical Specialty	−1 month	+ 3 months	+ 6 months	+ 13 months	+ 25 months
Family Medicine	14 (39) [23-57]	6 (14) [5-27]	6 (15) [6-30]	8 (30) [14-50]	9 (25) [12-42]
General Pediatrics	9 (35) [17-56]	6 (21) [8-41]	10 (32) [17-51]	6 (21) [8-40]	7 (28) [12-49]
Internal Medicine	19 (17) [11-25]	15 (9) [5-15]	8 (7) [3-13]	17 (12) [7-18]	22 (19) [12-27]
Primary Care	11 (30) [16-47]	7 (20) [8-37]	4 (17) [5-37]	4 (12) [3-27]	6 (17) [7-34]
Specialty	8 (11) [5-20]	8 (6) [3-12]	4 (4) [1-10]	13 (12) [6-19]	16 (19) [11-29]
Overall	42 (24) [18-31]	27 (11) [8-16]	24 (12) [9-18]	31 (15) [11-21]	38 (21) [16-28]	“flat line”

All responses are reported as No. (%) [95% CI]. The line graphs to the right are visualizations of the positive responses for each question; the y-axis shows the percentage of positive responses from 0 to 100, whereas the x-axis represents the time points. Below each line graph is the curve shape, as described in the Appendix. Note that the time points on the x-axis are shown at equal intervals although the actual study time intervals varied. Larger versions of each line graph can be found in the Appendix.

N/A: Not asked in this phase. ^aCareWeb in the pre-implementation phase and MiChart in the post-implementation phases. Filled square: family medicine. filled circle: general pediatrics; filled rhombus: internal medicine; filled triangle: internal medicine, primary care; filled inverted triangle: internal medicine, specialty.

For the data entry theme, the frequency of entering data while in the examination room with the patient increased (Q1), but respondents noted little change in the frequency of entering data during clinical workday evenings or on days off (Q2–3). Whereas most physicians agreed that the prior EHR supported various aspects of communication, there was a large drop in these measures across all groups that persisted 2 years post implementation (Q4–6).

Regarding safety (Q7), positive perceptions dropped substantially for about the first 6 months, then began to rise. While the overall graph displays an L-curve (remaining below baseline), a U-curve (return to baseline) was observed for family medicine, pediatrics, and primary care internal medicine. A drop and subsequent rise back to baseline was observed for the reminders and alerts measure (Q8).

For workflow and efficiency (Q9–11), 58% of respondents believed that the prior EHR allowed them to complete their documentation in a timely manner (Q9), which ultimately dropped to 26% with the new system. Across all specialties, few reported that the new EHR eliminated work they used to have to do (Q10), and nearly everyone reported that the new EHR created new work (Q11), a sentiment that remained stable over the study period.

Overall job satisfaction (Q12) dropped after the implementation. While it did rise slightly over time, it never reached baseline levels by +25 months. When participants were asked about the EHR’s contribution to their job satisfaction (Q13), positive views dropped from a high of 62% with the prior EHR to 8% with the new EHR. At no point did a majority of respondents believe that the new EHR would allow them to provide better care than the original EHR (Q14). A subset of the 559 comments left by the physicians is shown in Figure 1.

Figure 1.

Illustrative quotes representing sentiments expressed at the different time points during the longitudinal survey. These were drawn from 559 comments left by the respondents, totaling 56 303 words. Quotes from the same specialty are not necessarily from the same physicians each time. Note that CareWeb was the legacy homegrown EHR used from 1998 to 2012, and MiChart is the local name for the replacement vendor EHR.

DISCUSSION

Overall, we did not find evidence for J-curve patterns up to 2 years post EHR implementation. The only significant increase over baseline 2 years post implementation was for documenting while in exam rooms with patients. Most measures fell and remained below baseline, and only 1 (reminders and alerts) returned to baseline. These findings are worth discussing in the context of similar work. Some EHR implementation studies have shown improvements in productivity,^32,33 quality of care,³² safety,³⁷ and overall perceptions,³⁴ but others have had mixed results.¹⁹ Differences of opinion among providers exist: a recent survey found that 38% of family medicine physicians, 36% of primary care internal medicine physicians, and 29% of specialists felt that their EHR improved care, but higher percentages of physicians in each group thought it worsened care.⁴³ An older study from 2004 found lower satisfaction among internal medicine physicians compared to pediatricians who used an EHR within the same institution.⁴⁴ This raises questions about how well EHRs meet the diverse needs of various specialties.⁴⁵

A 2014 survey of providers revealed that many measures of EHR perception improved over time.³⁸ However, some measures did not improve, including patient engagement and care coordination. Even after 2 years, 27% felt that the EHR was not meeting their practice’s needs. Another EHR implementation study in a pediatric emergency department found multiple metrics returning to baseline as early as 3 months.³⁹

The variable time frames reported in prior studies make comparisons difficult. Experts have concluded that a “ramp-up” period³³ or a “shakedown phase”⁴⁶ is needed to achieve routine EHR use, lasting from 6 to 12 months. Our results suggest that even if routine use occurs by 12 months, perceptions can continue to change.

Three months remains a common duration before post-implementation studies began,^9,27,28 but that may be too early for perceptions to stabilize. Longitudinal assessments, therefore, are important for capturing changing patterns. Studies lasting less than a year may show decreases in some measures^28,47 that may increase after the study period.³⁵ One study reported the opposite; that is, productivity “gains” found at 3 months post implementation did not persist.⁴⁸ Several studies have observed that it may require 2 years before significant benefits from and satisfaction with an EHR are achieved.^6,49 One report suggested that 5 years may be needed for benefits to be observed,⁵⁰ whereas another study found EHR concerns persisting beyond 5 years.¹⁸ One thing is certain: “The optimal time period for assessment of time efficiencies post-implementation of EHRs remains a challenge and will require further research.”⁴⁸

Physicians in our study had increasing data capture and documentation requirements⁵¹ (eg, Meaningful Use)⁵² that coincided with the EHR implementation. Our prior EHR had fewer capabilities than its replacement, so it may be that physicians’ perceptions about increased work was due to the fact that they were accomplishing more.⁵³ One study found that physicians using an EHR with many functions, such as the one we implemented, had higher stress and time pressures, perhaps “trying to balance an increase in tasks with no increases in time allotted.”⁵⁴

Simpler EHRs may be perceived more positively by clinicians eager to spend more time with their patients and less with their computers.^55–58 There may also be differences between organizational and clinical needs for an EHR.⁵⁹ Strategies have been proposed to improve satisfaction with EHRs.^60,61 Major changes can be emotional for physicians,⁶² and our institution has ongoing efforts to improve physician efficiency and satisfaction. Additional surveys should clarify if satisfaction and perceptions continue to improve.

While our study was conducted at a single center, we surveyed 3 distinct clinical groups. We did not capture demographics, but age and gender may not be significant predictors of EHR satisfaction⁴⁴ or, as in earlier times, computer anxiety.⁶³ We also acknowledge that some of our response rates were low, but it is worth noting that for multiple measures the low response rate groups tracked closely with the high response rate groups, and that low response rates for physicians does not necessarily introduce bias.^64–69 Physicians with the lowest satisfaction may have chosen to leave,^12,70 and therefore may not have completed the surveys, resulting in a positive bias. Alternatively, unhappy users may have been motivated to respond. Despite these opposing possibilities, we did not observe differences in survey attrition rates among those responding positively vs negatively at baseline. Even if we are unable to generalize to the entire physician population, a substantial minority had persistent concerns even 2 years post implementation, and this should be considered in the context of an “emerging EHR monoculture.”⁷¹

CONCLUSION

In this prospective, longitudinal survey of physicians lasting 2 years post EHR implementation, we did not find evidence for a J-curve pattern with respect to positive perceptions eventually exceeding baseline measures. Some measures followed a U-curve (returned to baseline), or flatlined, while most followed an L-curve (fell and remained below baseline). Future research is warranted to determine if positive perceptions eventually surpass baseline, and what interventions can help physicians use EHRs more effectively.

CONFLICT OF INTEREST

The authors have no conflicts of interest to report.

CONTRIBUTORS

David Hanauer: Conception and design of the study, analysis, and interpretation of the data, drafting the manuscript

Greta Branford: Conception and design of the study, survey lead for internal medicine

Grant Greenberg: Conception and design of the study, survey lead for family medicine, drafting the manuscript

Sharon Kileny: Conception and design of the study, survey lead for general pediatrics

Mick Couper: Conception and design of the study, analysis and interpretation of the data, drafting the manuscript

Kai Zheng: Conception and design of the study

Sung Choi: Analysis and interpretation of the data, drafting the manuscript

All authors approved the final version to be published and agree to be accountable for all aspects of the work.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.