Abstract
Background
Little has been written about physician stress that may be associated with electronic medical records (EMR).
Objective
We assessed relationships between the number of EMR functions, primary care work conditions, and physician satisfaction, stress and burnout.
Design and participants
379 primary care physicians and 92 managers at 92 clinics from New York City and the upper Midwest participating in the 2001–5 Minimizing Error, Maximizing Outcome (MEMO) Study. A latent class analysis identified clusters of physicians within clinics with low, medium and high EMR functions.
Main measures
We assessed physician-reported stress, burnout, satisfaction, and intent to leave the practice, and predictors including time pressure during visits. We used a two-level regression model to estimate the mean response for each physician cluster to each outcome, adjusting for physician age, sex, specialty, work hours and years using the EMR. Effect sizes (ES) of these relationships were considered small (0.14), moderate (0.39), and large (0.61).
Key results
Compared to the low EMR cluster, physicians in the moderate EMR cluster reported more stress (ES 0.35, p=0.03) and lower satisfaction (ES −0.45, p=0.006). Physicians in the high EMR cluster indicated lower satisfaction than low EMR cluster physicians (ES −0.39, p=0.01). Time pressure was associated with significantly more burnout, dissatisfaction and intent to leave only within the high EMR cluster.
Conclusions
Stress may rise for physicians with a moderate number of EMR functions. Time pressure was associated with poor physician outcomes mainly in the high EMR cluster. Work redesign may address these stressors.
Keywords: electronic medical records; stress, psychological; primary care; outpatient clinics; physicians
Background
The electronic medical record (EMR) is promoted as an integral part of office-based care, and EMR implementation has markedly expanded in the last several years. In 2008, EMRs had been adopted by only a minority of practices,1 with 13% reporting a basic system and 4% a fully functional EMR. A basic system included patient demographics, problem list, medication list, clinical notes, prescription printing, and laboratory and radiology results. A fully functional system additionally contained medical history and follow-up notes, ordering functions for laboratories and imaging, ability to send prescriptions electronically, and inclusion of electronic images and clinical decision support tools. A 2012 study showed a dramatic increase in EMR implementation, with 72% of office-based physicians reporting use of an EMR. Forty per cent of these reported a system meeting basic criteria including patient history and demographics, problem list, clinical notes, medication and allergy lists, computerized prescription ordering, and the ability to view laboratory and imaging results electronically.2
Managing information overload is challenging,3 and EMRs may increase physician cognitive demands.4 This 'information chaos’ is frequent, and has implications for both physician performance and patient safety.5 A 2010 editorial considered the US investment in health information technology, the expectations around ‘meaningful use’ of EMR, and the ability of EMR to address quality initiatives given that current EMR do not support care coordination well.6 As incentives to implement and expand EMR increase,7 so do concerns about the ability of the EMR to meet workflow requirements.
We used data from the 2001 to 2005 Minimizing Error, Maximizing Outcome (MEMO) Study to assess two areas.8 First, we investigated associations between the number of EMR features available in a primary care office and physician stress, burnout, job satisfaction, and intent to leave the practice. Second, we examined organizational characteristics such as physician autonomy (work control) and productivity expectations/time pressure that may have modified the relationship between EMR features and physician work stress. Three competing hypotheses were deemed possible: EMR functions would be associated with increased work stress; EMR functions would be associated with decreased work stress; or the presence of an EMR would have no association with work stress.
Methods
Design and participants
The MEMO Study was designed to assess relationships between the structure and culture of the primary care workplace, physician stress and burnout, and the quality of care experienced by their patients.8 Investigators recruited outpatient primary care practices from five regions: inner city clinics in New York City and Chicago, Illinois; academic and managed care clinics in Milwaukee and Madison, Wisconsin; and small/rural clinics in central Wisconsin. These areas and clinics were selected for their diverse patient base and a wide range of payers. Physicians were recruited by local site directors during informational sessions at clinics and hospital grand rounds. Eligible physicians included family practitioners or general internists who spent at least four half-days per week delivering primary care. Clinics were enrolled if at least 50% of their physicians chose to participate. A total of 449 physicians from 119 practices consented to participate (59.6% of those approached) and 94.0% of these (n=422) completed the baseline survey for a final participation rate of 56.0%. Non-participants did not differ in specialty or sex from participants.
Main measures
Physicians completed a survey querying time pressure during patient visits (time allotted vs time needed for quality care for routine and follow-up visits), control over 14 workplace issues,9 job satisfaction,10 11 job stress,12 burnout,13 and intent to leave the practice. Stress and burnout were measured on five-point scales from 1 to 5. This survey also evaluated organizational culture using five scales: practice emphasis on quality; emphasis on information/communication; trust in the organization; practice cohesiveness; and physician-leader values alignment.14 15 Clinic managers completed a questionnaire querying office characteristics including the presence of an EMR. Managers reporting an EMR completed a checklist of 15 features commonly available on EMR systems, and noted number of years of EMR use at the site. Table 1 provides more information on these physician and EMR measures. Institutional review boards from all participating organizations approved the protocol and participants provided written consent.
Table 1.
Minimizing Error, Maximizing Outcome Study physician and EMR measures
| Physician survey | Questions |
|---|---|
| 1. Time pressure (ratio of time needed to time allotted) | Estimate average time allotted and average time needed to provide high quality care for: (1) complete physical exam; (2) routine follow-up visit |
| 2. Work control (4-point scale, none to great) | How much control do you have over: (1) selecting referral providers; (2) admitting decisions; (3) length of stay decisions; (4) selecting medications; (5) clinic schedules; (6) ordering tests; (7) paperwork volume; (8) work hours; (9) work interruptions; (10) panel size; (11) pre-authorizations; (12) workplace issues; (13) work pace; (14) time for difficult patients. |
| 3. Organizational culture | |
| (a) Workplace emphasis on quality scale (α 0.86) (4-point response scale, not at all to great extent) | (1) Physicians with inappropriate care practices are ‘talked to’; (2) Emphasize patient satisfaction; (3) Quality of physician work is closely monitored; (4) Identifiable practice style we try to adhere to; (5) Good methods to assure practice changes to include new technologies/research; (6) Commitment to measuring clinical outcomes; (7) Quality of care is goal one; (8) Common standard of care; (9) Leadership is concerned with quality of care; (10) Adequate training is provided for quality of care issues; (11) Making changes to reduce substandard care is difficult. |
| (b) Workplace emphasis on information and communication scale (α 0.68) (4-point response scale, not at all to great extent) | (1) Rely heavily on electronic information systems to provide cost-effective care; (2) Rely heavily on computer-based information when seeing patients; (3) Candid and open communications between physicians and nurses. |
| (c) Trust in the organization scale (α 0.79) (4-point response scale, not at all to great extent) | (1) Strong sense of group belonging; (2) Great deal of organizational loyalty; (3) Strong sense of responsibility to help physicians with personal problems; (4) High degree of organizational trust. |
| (d) Workplace cohesiveness scale (α 0.66) (4-point response scale, not at all to great extent) | (1) Widespread agreement about most moral/ethical issues; (2) Great deal of clinical information sharing; (3) Open discussions of clinical failures; (4) General agreement on treatment methods. |
| (e) Values alignment between physicians and leadership scale (α 0.86) (4-point response scale, not at all to great extent) | (1) Compensation formula well aligned with organizational goals; (2) Broad involvement of physicians in financial decisions; (3) Administrators provide information to improve care cost effectiveness; (4) Compensation plan rewards hard workers; (5) Compensation formula well understood; (6) Administrative decision-making process based on consensus building; (7) Business office/administration important parts of the practice; (8) Rapid change in practice when studies show can improve quality/reduce costs. |
| 4. Physician reactions | |
| Job satisfaction (5-point scale, strongly disagree to strongly agree) | (1) Work is rewarding; (2) Pleased with work; (3) Satisfied with practice; (4) Work is frustrating; (5) Practice has not met expectations. |
| Job stress (5-point scale, strongly disagree to strongly agree) | (1) Great deal of stress; (2) Few stressful things at work; (3) Job is extremely stressful; (4) Almost never feel job stress. |
| Burnout (select one response) | (1) Enjoy work; no symptoms; (2) Occasional stress; no burnout; (3) Burning out 1+ symptoms; (4) Constant symptoms; (5) Complete burn-out, need help. |
| Likelihood of leaving the practice (5-point scale, none to definitely) | Likely to leave the practice within 2 years. |
| Manager survey | Questions |
| Which EMR functions do your computers have? (check yes or no) | (a) Lab results. (b) Patient notes. (c) Radiographic reports. (d) Prevention reminders. (e) Drug interaction warnings. (f) Patient allergy warnings. (g) Access to treatment alternatives/guidelines. (h) Prescription writing. (i) Data exchange with other physicians. (j) E-mail communication with patients. (k) Consult notes from other services. (l) Medication list. (m) Problem list. (n) Test ordering. (o) Imaging ordering. |
EMR, electronic medical record.
Analysis
As a first step in the analyses, we used binary-based latent class analysis (LCA) to sort clinics into clusters based on 15 binary items reported by clinic managers regarding the presence or absence of 15 specific functions available in that clinic's EMR.16 LCA can detect homogenous subgroups in a heterogeneous group by evaluating and minimizing associations among responses across a set of indicators. The number of clusters were chosen based on the following selection criteria: (1) interpretability; (2) theoretical justification; (3) parsimony; (4) lowest adjusted Bayesian information criteria and Bozdogan's consistent Akaike information criteria17 scores; (5) average posterior probability in each cluster greater than 0.75 and no more than 10% overlap/cross-membership between non-contiguous clusters; and (6) at least 2.5% of total count in each cluster. Next, we assessed if the four physician outcomes (stress, burnout, satisfaction, and intent to leave) varied across the clusters identified by LCA. A two-level model (physicians nested within clinics) was used to calculate the mean adjusted response for each physician cluster to each outcome measure. The analyses were adjusted for physician age, sex, specialty, weekly work hours, and years using the EMR. Age, work hours and EMR duration were modeled as continuous variables. No collinearity was observed between these adjustment variables and EMR function group. We calculated Cohen's d-family effect sizes (ES) using mean differences normalized by the pooled SD, and applied the article on medical studies by Reed and Slaichert18 as a reference point to define small ES as 0.14, moderate as 0.39, and large as 0.61. Finally, we investigated if selected workplace factors (time pressure, work control, and the five scales reflecting organizational culture) differentially predicted physician outcomes based on EMR function cluster. For this analysis we calculated partial regression coefficients, again using a two-level modeling structure (physicians nested within clinics) controlling for physician age, sex, specialty, work hours and years using the EMR. All models were constructed using Mplus V.7.11.19
Results
A total of 379 physicians from 92 clinics were included in the sample. Mean age was 43 years, 56% were men, and 52% were general internists (48% family physicians). Based on the three parsimony indices, the best fit for the LCA was a three-class solution (figure 1), which was defined as clinics with a high number of EMR functions (50% of our clinic sample), clinics with a moderate number of EMR functions (24%), and clinics with a low number of EMR functions (26%). Managers from nearly 70% of the clinics in the moderate function cluster reported having an EMR system for less than 1 year. Conversely, most of the clinics with a high number of EMR functions had their systems quite a bit longer (an average of 4.3 years); however, nearly a quarter of these managers also reported having an EMR for less than a year. There was no clear association between years since EMR implementation and work stress.
Figure 1.
EMR functions from the clinic managers’ survey displayed with the probability of the functions in each cluster.
Table 2 illustrates the difference in physician scores for the scales measuring job stress, burnout, job satisfaction, and intent to leave. Compared to the low function group, significant associations included: higher stress scores among the moderate function group (moderate 3.44 vs low 2.97, p=0.03, with a small to moderate ES of 0.35); less job satisfaction among both the high and moderate function groups (3.61 moderate cluster vs 4.11 low cluster, p=0.006, with a moderate to large ES of −0.45; and 3.68 high cluster vs 4.11 low cluster, p=0.01, with a moderate ES of −0.39). There was a trend towards higher burnout scores in the moderate function group (2.32 moderate cluster vs 2.03 low cluster, p=0.08, with a small to moderate ES of 0.32). There was no difference in intent to leave between function groups. Figure 2 contrasts the adjusted means across clusters for physician responses to the stress, burnout, satisfaction, and intent to leave scales.
Table 2.
Variance in physician's adjusted mean scores for stress, burnout, satisfaction, and intent to leave scales by EMR function clusters
| High function cluster: 46 clinics, 207 physicians | Mod function cluster: 22 clinics, 98 physicians | Low function cluster: 24 clinics, 74 physicians | Contrast high versus mod cluster | Contrast high versus low cluster | Contrast mod versus low cluster | |
|---|---|---|---|---|---|---|
| Variable | Mean (SE) | Mean (SE) | Mean (SE) | |||
| 95% CI | 95% CI | 95% CI | ||||
| Stress score | 3.26 (0.07) (3.13 to 3.39) |
3.44 (0.10) (3.24 to 3.64) |
2.97 (0.19) (2.59 to 3.35) |
p=0.15 ES=−0.18 |
p=0.14 ES=0.23 |
p=0.03 ES=0.35 |
| Burnout score | 2.18 (0.06) (2.05 to 2.30) |
2.32 (0.10) (2.12 to 2.51) |
2.03 (0.09) (1.84 to 2.21) |
p=0.24 ES=−0.15 |
p=0.25 ES=0.17 |
p=0.08 ES=0.32 |
| Satisfaction | 3.68 (0.05) (3.58 to 3.79) |
3.61 (0.08) (3.44 to 3.78) |
4.11 (0.16) (3.79 to 4.44) |
p=0.47 ES=0.08 |
p=0.01 ES=−0.39 |
p=0.006 ES=−0.45 |
| Intent to leave | 2.00 (0.09) (1.82 to 2.19) |
2.05 (0.15) (1.76 to 2.34) |
2.09 (0.26) (1.57 to 2.61) |
p=0.80 ES=−0.03 |
p=0.75 ES=−0.05 |
p=0.87 ES=−0.02 |
Means adjusted for physician age, sex, specialty, hours worked weekly, and number of years with EMR system.
EMR, Electronic medical record; ES, effect size.
Figure 2.
Stress scored on five-point scale, ≥3 defined as ‘stressed’; Satisfaction scored on five-point scale, ≥3 defined as ‘satisfied’; Burnout scored on five-point scale, ≥3 defined as ‘burned out’; Intent to leave scored on five-point scale, ≥3 defined as ‘likely to leave’.
Table 3 illustrates the results from multivariate analyses assessing the associations between various workplace characteristics and physician outcomes by EMR cluster. As an example, for the variable ‘time pressure during physical examinations’, the table shows how the relationship between perceived time pressure and stress varies by EMR cluster. The numbers in each cell are partial regression coefficients based on a two-level clinic-physician model, adjusted for physician age, sex, specialty, hours worked weekly, and number of years with the EMR system. When looking at the outcomes of physician stress, satisfaction, burnout and intent to leave as a function of time pressure during physical examinations, we see a significant relationship between all four of these outcomes and time pressure, but only for physicians in the cluster with a high number of EMR functions. A similar pattern emerges for time pressure during follow-up visits, with significant relationships in three of the four outcomes, and again, only within the high function group. The directionality of this finding was that higher time pressure was associated with adverse outcomes.
Table 3.
Associations between workplace characteristics and physician outcomes by EMR cluster
| Predictors | EMR level | Stress | Burnout | Satisfaction | Intent to leave |
|---|---|---|---|---|---|
| Time pressure for visit | |||||
| High | 0.194 (−0.033, 0.420) |
0.285* (0.111, 0.458) |
−0.200* (−0.363, −0.036) |
0.295* (0.171, 0.420) |
|
| Moderate | 0.054 (−0.180, 0.287) |
−0.520 (−0.282, 0.179) |
−0.127 (−0.347, 0.094) |
0.026 (−0.298, 0.350) |
|
| Low | −0.046 (−0.645, 0.553) |
0.210 (−0.474, 0.893) |
−0.127 (−0.616, 0.363) |
−0.148 (−0.460, 0.165) |
|
| Time pressure for examination | |||||
| High | 0.167* (0.040, 0.295) |
0.214* (0.078, 0.349) |
−0.185* (−0.335, −0.036) |
0.208* (0.033, 0.384) |
|
| Moderate | 0.059 (−0.240, 0.359) |
−0.137 (−0.379, 0.104) |
−0.107 (−0.331, 0.117) |
−0.081 (−0.279, 0.117) |
|
| Low | 0.105 (−0.187, 0.397) |
0.414 (−0.027, 0.856) |
−0.053 (−0.592, 0.487) |
0.001 (−0.190, 0.192) |
|
| Work control | |||||
| High | −0.292* (−0.420, −0.165) |
−0.284* (−0.430, −0.138) |
0.434* (0.308, 0.560) |
−0.333* (−0.464, −0.202) |
|
| Moderate | −0.120 (−0.358, 0.119) |
−0.073 (−0.213, 0.066) |
0.349* (0.161, 0.536) |
0.056 (−0.233, 0.346) |
|
| Low | −0.344* (−0.627, −0.061) |
−0.571* (−0.990, −0.153) |
0.348* (0.027, 0.669) |
0.464* (0.224, 0.703) |
|
| Quality | |||||
| High | −0.124 (−0.277, 0.029) |
−0.318* (−0.426, −0.210) |
0.350* (0.227, 0.473) |
−0.342* (−0.444, −0.240) |
|
| Moderate | −0.193* (−0.352, −0.034) |
−0.063 (−0.283, 0.156) |
0.346* (0.190, 0.503) |
−0.149 (−0.443, 0.144) |
|
| Low | −0.367* (−0.726, −0.009) |
−0.341 (−0.740, 0.057) |
0.143 (−0.126, 0.412) |
0.046 (−0.493, 0.586) |
|
| Communication | |||||
| High | −0.102 (−0.262, 0.059) |
−0.178* (−0.337, −0.019) |
0.277* (0.145, 0.409) |
−0.093 (−0.206, 0.020) |
|
| Moderate | −0.180 (−0.376, 0.015) |
−0.189* (−0.314, −0.064) |
0.251* (0.103, 0.399) |
−0.264* (−0.517, −0.011) |
|
| Low | −0.081 (−0.275, 0.112) |
0.069 (−0.125, 0.263) |
−0.004 (−0.231, 0.222) |
0.016 (−0.302, 0.334) |
|
| Trust | |||||
| High | −0.144* (−0.280, −0.007) |
−0.302* (−0.431, −0.174) |
0.433* (0.300, 0.565) |
−0.323* (−0.465, −0.181) |
|
| Moderate | −0.196* (−0.343, −0.049) |
−0.119 (−0.289, 0.052) |
0.352* (0.212, 0.492) |
−0.353* (−0.532, −0.174) |
|
| Low | −0.267 (−0.740, 0.206) |
−0.385* (−0.675, −0.096) |
0.347* (0.085, 0.609) |
0.224 (−0.162, 0.611) |
|
| Cohesive | |||||
| High | −0.097 (−0.246, 0.051) |
−0.163* (−0.279, −0.047) |
0.324* (0.208, 0.441) |
−0.166* (−0.291, −0.041) |
|
| Moderate | −0.050 (−0.234, 0.133) |
0.061 (−0.162, 0.283) |
0.128 (−0.123, 0.379) |
−0.120 (−0.379, 0.140) |
|
| Low | −0.271 (−0.744, 0.202) |
−0.211 (−0.654, 0.233) |
0.134 (−0.216, 0.484) |
0.032 (−0.558, 0.623) |
|
| Leadership | |||||
| High | −0.196* (−0.343, −0.050) |
−0.255* (−0.397, −0.112) |
0.315* (0.191, 0.439) |
−0.428* (−0.560, −0.297) |
|
| Moderate | −0.066 (−0.219, 0.087) |
−0.106 (−0.251, 0.040) |
0.238* (0.103, 0.372) |
−0.032 (−0.277, 0.213) |
|
| Low | −0.133 (−0.522, 0.257) |
−0.462* (−0.732, −0.192) |
0.430* (0.110, 0.751) |
0.375* (0.018, 0.732) |
|
*p<0.05.
EMR, electronic medical record.
Analyses of the associations between physician outcomes and the other workplace factors (work control and five measures of organizational culture) reveal that time pressure and practice cohesiveness are the only job characteristics in which the findings are restricted to a single cluster (high EMR functionality). We noted a tendency, across all three clusters, for physicians to be more satisfied, less burned out, and more likely to stay in their practice if they reported more work control, greater practice emphasis on quality, higher practice emphasis on information/communication, more trust in the organization, greater workplace cohesiveness, and more values alignment with leadership.
Discussion
Two important issues arise from our findings. The first is that primary care physicians who are using EMR with a moderate number of functions report more stress and less job satisfaction than physicians with low numbers of EMR functions. We suggest that these physicians were grappling with the process of practice transformation. A hybrid system composed of paper and electronic records necessitates that providers function in two different worlds, each requiring different processes and cognitive demands. Physicians may have had to use computers to view results, but then act ‘on paper’. This could lead to problems with coordinating care, and concerns about tracking orders and summaries.
The second important issue from our findings is that time pressure was more related to adverse physician outcomes in the high EMR function cluster, and in particular, that time pressure during physical examinations was correlated with all negative physician outcomes only in this group. This suggests that physicians working with highly functioning EMR may be particularly challenged when the time allotted is not equal to the time perceived as necessary to provide quality care. One implication of this finding is that EMR systems may not match workplace processes and flow,20 thus testing the resilience of clinicians as they attempt to function quickly while the patient is present.21 This is consistent with O'Malley et al,3 who noted that managing EMR information overload is challenging. A fully functional EMR may make clinicians aware of the multiplicity of tasks required during each patient visit (health maintenance, quality measures, chronic disease management, social determinants of health, and other documentation requirements). This assessment of work required, potentially a more accurate assessment than could be made with the paper record, leads to the policy question of whether short visits (less time allotted than needed) allow the time necessary for providing contemporary primary care.
A recent study reported that the adoption of EMR to meet meaningful use regulations nearly doubled in primary care clinics from 2009 to 2011.22 Thus, addressing the challenges for physicians is critical. To mitigate the potential negative effects of practice transformation, it is critical to develop transition strategies that prepare offices for, and try to avoid, increased stress.23 The taxonomy of barriers to EMR acceptance of Boonstra and Broekhius24 provides a framework to help leaders address particular challenges. As EMR implementation progresses, stress could be diminished by regular optimization meetings, coupled with a responsive information technology group.
We found that while job stress decreases modestly for physicians with sophisticated systems compared to physicians with a moderate number of functions, time pressure during examinations and office visits was more highly associated with adverse physician outcomes in the high EMR function group. We hypothesize that while these physicians may be increasingly accustomed to their EMR, they are trying to balance an increase in tasks with no increases in time allotted. Future research could identify EMR features that are associated with improved physician outcomes, as well as physician and office work processes that decrease time pressure. Howard et al25 noted that ‘EHR use reduced some clinician work (ie, prescribing, some lab-related tasks, and communication within the office), while increasing other work (ie, charting, chronic disease and preventive care tasks, and some lab-related tasks).’ Further studies such as theirs may help reveal ways to remediate physician stress and burnout. Potential facilitating variables such as physician engagement and practice financing also warrant additional study. Longer office visits to accommodate information overload in sites with fully functional EMR, or designating ‘EMR catch-up’ time (desktop, or patient management slots) during patient care sessions, deserve consideration.26 Finally, the training, practice, reinforcement and optimization of practice for all members of the care team is a large, time-consuming, and iterative process. Further work is needed to identify the best ways to address these implementation issues.
The results of our study should be considered in light of several limitations. The EMR functions we measured did not include all possible functions, the physician outcome data were self-reported, EMR in different clinics may have come from different companies, and the research sites were restricted to primary care clinics. In addition, this is a cross-sectional study and causality cannot be determined. Strengths of the MEMO Study include the diverse provider base, the large number of clinics studied, and the timing of the study (coinciding with the implementation of EMR nationally). Our work highlights two important findings. First, the stress and burnout data suggest that a practice in transition may be a practice under pressure. Second, physicians working with highly functioning EMR experience a variety of adverse personal outcomes in association with time pressure during office visits and physical examinations. Because EMR may lead to decreased productivity,22 27 EMR systems should be designed to take into account physician workload and cognitive capabilities. Understanding which EMR features contribute to stress and burnout can help predict unintended consequences for physicians28 and offer opportunities to optimize EMR features that complement and enhance physician work life.
Footnotes
Contributors: SB is the lead author. LBM, RB and ML have been primarily involved in the statistics and analysis of this work. EM, EW MS and EH are a part of the MEMO team, and have had input into the writing, revisions and review of this manuscript. Each of these authors has participated sufficiently in the work to take public responsibility for the appropriate portions of the content.
Funding: Grant 5 R01 HS011955 from the Agency for Healthcare Research and Quality supported the design and conduct of the MEMO Study.
Competing interests: SB, RB, and ML had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.
Ethics approval: Institutional review boards from all participating organizations approved the protocol and participants provided written consent.
Provenance and peer review: Not commissioned; externally peer reviewed.
References
- 1.DesRoches CM, Campbell EG, Rao SR, et al. Electronic health records in ambulatory care—a national survey of physicians. N Engl J Med 2008;359:50–60 [DOI] [PubMed] [Google Scholar]
- 2.Hsiao CJ, Hing E. Use and Characteristics of Electronic Health Record Systems Among Office-Based Physician Practices: United States, 2001–2012. NCHS Data Brief, No 111 Hyattsville, MD: US Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics, 2012 [Google Scholar]
- 3.O'Malley AS, Grossman JM, Cohen GR, et al. Are electronic medical records helpful for care coordination? Experiences of physician practices. J Gen Intern Med 2010;25:177–85 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Holden RJ. Cognitive performance-altering effects of electronic medical records: an application of the human factors paradigm for patient safety. Cogn Technol Work 2011;13:11–29 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Beasley JW, Wetterneck TB, Temte J, et al. Information chaos in primary care: implications for physician performance and patient safety. J Am Board Fam Med 2011;24:745–51 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Bates DW. Getting in step: electronic health records and their role in care coordination. J Gen Intern Med 2010;25:174–6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Hsiao CJ, Hing E, Socey TC, et al. Electronic Health Record Systems and Intent to Apply for Meaningful Use Incentives Among Office-Based Physician Practices: United States, 2001–2011. NCHS Data Brief, no. 79 Hyattsville, MD: US Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics, 2011 [Google Scholar]
- 8.Linzer M, Manwell LB, Williams ES, et al. Working conditions in primary care: physician reactions and care quality. Ann Intern Med 2009;151:28–36 [DOI] [PubMed] [Google Scholar]
- 9.Linzer M, Gerrity M, Douglas JA, et al. Physician stress: results from the Physician Worklife Study. Stress Health 2002;18:37–42 [Google Scholar]
- 10.Konrad TR, Williams ES, Linzer M, et al. Measuring physician job satisfaction in a changing workplace and a challenging environment. SGIM Career Satisfaction Study Group. Society of General Internal Medicine. Med Care 1999;37:1174–82 [DOI] [PubMed] [Google Scholar]
- 11.Williams ES, Konrad TR, Linzer M, et al. Refining the measurement of physician job satisfaction: results from the Physician Worklife Survey. SGIM Career Satisfaction Study Group. Society of General Internal Medicine. Med Care 1999;37:1140–54 [DOI] [PubMed] [Google Scholar]
- 12.Motowidlo S, Packard JS, Manning MR. Occupational stress: its causes and consequences for job performance. J Appl Psychol 1986;71:618–29 [PubMed] [Google Scholar]
- 13.Rohland BM, Kruse GR, Rohrer JE. Validation of a single-item measure of burnout against the Maslach Burnout Inventory among physicians. Stress Health 2004;20:75–9 [Google Scholar]
- 14.Linzer M, Baier Manwell L, Mundt M, et al. Organizational climate, stress, and error in primary care: the MEMO Study. In: Clancy C.ed Advances in Patient Safety: From Research to Implementation. AHRQ Publication no. 050021 (1) Rockville, MD: Agency for Healthcare Research and Quality, 2005:67–77 [PubMed] [Google Scholar]
- 15.Kralewski JE, Wingert TD, Barbouche MH. Assessing the culture of medical group practices. Med Care 1996;34:377–88 [DOI] [PubMed] [Google Scholar]
- 16.Vermunt JK. Latent class and finite mixture models for multilevel data sets. Stat Methods Med Res 2008;17:33–51 [DOI] [PubMed] [Google Scholar]
- 17.Fraley C, Raftery AE. How many clusters? Which clustering method? Answers via model-based cluster analysis. Computer J 1998;4:578–88 [Google Scholar]
- 18.Reed JF, III, Slaichert W. Statistical proof in inconclusive ‘negative’ trials. Arch Intern Med 1981;141:1307–10 [PubMed] [Google Scholar]
- 19.Muthen LK, Muthen BO. Mplus user's guide, 6th edn Los Angeles, CA: Muthen and Muthen, 1998–2011 [Google Scholar]
- 20.Armijo D, McDonnell C, Werner K. Electronic Health Record Usability: Interface Design Considerations. AHRQ Publication no. 09(10)-0091–2-EF. Prepared by James Bell Associates and the Altarum Institute Rockville, MD: Agency for Healthcare Research and Quality, 2009 [Google Scholar]
- 21.Karsh BT, Weinger MB, Abbott PA, et al. Health information technology: fallacies and sober realities. J Am Med Inform Assoc 2010;17:617–23 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Patel V, Jamoom E, Hsiao C-J, et al. Variation in electronic health record adoption and readiness for meaningful use: 2008–2011. J Gen Intern Med 2013;28:957–64 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Karsh BT. Beyond usability: designing effective technology implementation systems to promote patient safety. Qual Saf Health Care 2004;13:388–94 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Boonstra A, Broekhius M. Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC Health Serv Res 2010;10:231. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Howard J, Clark EC, Friedman A, et al. Electronic health record impact on work burden in small, unaffiliated, community-based primary care practices. J Gen Intern Med 2012;28:107–13 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Reid RJ, Fishman PA, Yu O, et al. Patient-centered medical home demonstration: a prospective, quasi-experimental, before and after evaluation. Am J Manag Care 2009;15:e71–87 [PubMed] [Google Scholar]
- 27.Samaan ZM, Klein MD, Mansour MD, et al. The impact of the electronic health record on an academic pediatric primary care center. J Ambul Care Manage 2009;32:180–7 [DOI] [PubMed] [Google Scholar]
- 28.Harrison MI, Koppel R, Bar-Lev S. Unintended consequences of information technologies in health care—an interactive sociotechnical analysis. J Am Med Inform Assoc 2007;14:542–9 [DOI] [PMC free article] [PubMed] [Google Scholar]