Abstract
Objective
Burnout is common among emergency medicine (EM) physicians, and it is prevalent even among EM trainees. Recently proposed Accreditation Council for Graduate Medical Education requirements encourage faculty to alert residency leadership when trainees display signs of burnout. It remains uncertain how trainees experiencing burnout can be reliably identified. We examined if EM faculty advisers at one institution can accurately predict burnout in their EM resident advisees.
Methods
In this cross‐sectional, exploratory study at a single institution, we measured EM trainee burnout using the Maslach Burnout Inventory through a confidential, electronic survey. We subsequently asked EM faculty to predict if their designated advisees were experiencing burnout through a separate confidential, electronic survey. Burnout results were dichotomized from each survey and compared using a 2 × 2 contingency table and Fisher's exact test.
Results
Thirty‐six of 54 (66.7%) eligible EM trainees completed the burnout assessment. Eleven of 19 (57.9%) eligible faculty advisers completed trainee burnout predictions, resulting in 30 of 54 (55.6%) trainees who completed the burnout assessment and had a faculty burnout prediction. Trainees reported an overall burnout rate of 70.0% (95% confidence interval [CI] = 53.6% to 86.4%). Cumulative faculty predictions of trainee burnout resulted in an overall burnout rate of 16.7% (95% CI = –5.3% to 38.7%). The sensitivity and specificity of faculty predictions of trainee burnout were 19.1% (95% CI = 5.5% to 41.9%) and 88.9% (95% CI = 51.8% to 99.7%), respectively. Faculty prediction of trainee burnout had a positive predictive value of 80.0% (95% CI = 28.4% to 99.5%) and a negative predictive value of 32.0% (95% CI = 15.0% to 53.5). The difference between trainees’ reported rate of burnout and faculty predictions of trainee burnout was significant (p < 0.001).
Conclusion
Emergency medicine faculty prediction of trainee burnout was poor. Education on recognizing burnout and other methods of identifying trainee burnout may be necessary.
Burnout is a syndrome of emotional exhaustion, depersonalization, and a sense of low personal accomplishment.1 Burnout is prevalent among physicians (45%–55%), with emergency medicine (EM) physicians reporting the highest levels (65%–75%) among all specialties.2, 3, 4 Burnout is associated with decreased physician effectiveness at work and poor health.5, 6 Burnout is also associated with lower career satisfaction and greater intention of physicians to leave their work, including early retirement.7
Recent work demonstrated that burnout is prevalent (55%–65%) among EM trainees as early as the second year of training.3, 4, 8 Efforts to ameliorate the impact of burnout on trainees have been suggested, although their effectiveness remains unclear.9 One reason for the uncertain success of these wellness interventions may be due to their implementation among trainees en masse, without discrimination between those with high versus low levels of burnout. Recently proposed common program requirements issued by the Accreditation Council for Graduate Medical Education (ACGME) also encourage faculty to alert residency leadership when there are concerns about trainees displaying signs of burnout.10 However, the question remains if trainees with burnout can be reliably identified. We aimed to answer this question in this exploratory study by examining if EM faculty advisers at one institution can accurately predict burnout in their EM resident advisees.
Methods
Study Design
A standardized and validated burnout survey was conducted in a cross‐sectional sample of EM trainees. Results were compared to those of a separate survey asking a cross‐sectional sample of active EM faculty advisers to predict if their resident advisees were experiencing burnout.
Study Setting and Population
All EM postgraduate year (PGY) 1–4 trainees from a single urban, academic, Level I trauma, ACGME‐accredited residency program (88,000 patients annually) were eligible for the burnout survey conducted in November 2015. At this institution, residency leadership matches all trainees with a designated EM faculty adviser in the first year of residency based on academic and personal interests. Faculty advisers are instructed to meet with their advisees at least semiannually over the 4 years of training. Items for discussion suggested by residency leadership include, but are not limited to, difficulties in training, personal life issues, mechanisms to continue extracurricular learning, long‐term career goals, and progress on a scholarly project. All EM faculty members with designated advisees were eligible for the burnout prediction survey conducted in December 2015. There were no exclusion criteria.
Study Protocol
All PGY 1–4 EM trainees received an e‐mail invitation to participate in the study by completion of a confidential electronic survey assessing their levels of burnout. All eligible EM faculty advisers received a separate e‐mail invitation to participate in the study, also by completion of a confidential electronic survey asking them to predict if their designated advisees were experiencing burnout. All participants consented to the voluntary study by completing the respective open survey on a secure and Web‐based application (Research Electronic Data Capture). Up to three weekly reminder e‐mail invitations were sent to nonresponders. The Northwestern University Human Subjects Review Board approved the study.
Measurements
We assessed burnout among trainees using the Maslach Burnout Inventory (MBI), a widely used 22‐item standard instrument.1 Consistent with prior work, burnout was dichotomized and defined by high scores in the depersonalization (>12) or emotional exhaustion (>26) subscales of the inventory.11
We assessed faculty predictions of trainee burnout by asking faculty advisers to identify their assigned advisees and if they believed the advisee was experiencing burnout. Faculty respondents were provided a definition of burnout and a description of its characteristic features (Data Supplement S1, available as supporting information in the online version of this paper). Responses were dichotomized to “yes” or “no.” We collected limited demographic information from trainees and faculty advisers.
Data Analysis
Descriptive statistics for key variables were calculated. We calculated the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and likelihood ratios (LRs) of faculty predictions of trainee burnout using standard formulas. We used chi‐square analysis to compare the number of eligible versus responding trainees by sex. We used Fisher's exact test to compare the number of eligible versus responding faculty by sex, burnout by postgraduate year, and the number of trainees reporting burnout versus the number of trainees predicted by faculty to have burnout. Analyses were performed using SPSS v23.0.
Results
A total of 36 of 54 (66.7%) eligible EM trainees completed the burnout assessment. Eleven of 19 (57.9%) eligible EM faculty advisers completed trainee burnout predictions on their assigned advisees, resulting in 30 of 54 (55.6%) trainees who both completed the burnout assessment and had a faculty burnout prediction.
Mean trainee age was 28.1 years (95% confidence interval [CI] = 27.4 to 28.7 years) and the majority was male (68.8%, 95% CI = 53.7% to 83.9%). The percentage of responding trainees who were male was not significantly different than the percentage of eligible male trainees in the program (66.7%, p = 0.837). Faculty respondents were mostly male (81.8%, 95% CI = 59.0% to 100.0%), with the percentage of responding faculty who were male also not significantly different than the percentage of eligible male faculty in the program (68.4%, p = 0.447). Participating faculty reported a mean of 6.9 years (95% CI = 4.5 to 9.3 years) in practice since completion of residency training and a mean of 89.3 clinical hours (95% CI = 70.4 to 108.2 clinical hours) at the teaching site per month.
Twenty‐one of 30 trainees met criteria for burnout, for an overall burnout rate of 70.0% (95% CI = 53.6% to 86.4%), with no significant difference between trainees by postgraduate year (p = 0.790). Cumulative faculty predictions of individual trainee burnout resulted in an overall burnout rate of 16.7% (95% CI = –5.3% to 38.7%). The sensitivity and specificity of faculty predictions of trainee burnout were 19.1% (95% CI = 5.5% to 41.9%) and 88.9% (95% CI = 51.8% to 99.7%), respectively (Table1). Faculty prediction of trainee burnout had a PPV of 80.0% (95% CI = 28.4% to 99.5%) and a NPV of 32.0% (95% CI = 15.0% to 53.5%). Faculty prediction of trainee burnout had a positive LR of 1.7 (95% CI = 0.2 to 13.3) and a negative LR of 0.9 (95% CI = 0.7 to 1.2). The difference between the number of trainees reporting burnout and the number of trainees predicted by faculty to experience burnout was significant (p < 0.001).
Table 1.
Comparison of Trainee Burnout With Faculty Prediction of Trainee Burnout
| Faculty Prediction | Trainee Burnout | ||
|---|---|---|---|
| + | – | Total | |
| + | 4 | 1 | 5 |
| – | 17 | 8 | 25 |
| Total | 21 | 9 | 30 |
Discussion
This exploratory study is the first to investigate the ability of faculty advisers to accurately predict burnout among resident advisees. Results demonstrated that EM faculty significantly underestimated the number of trainees reporting burnout (16.7% vs. 70.0%). This discrepancy is notable in light of faculty respondents being asked to specifically predict burnout in their assigned advisees. Faculty advisers and advisees in this program are encouraged to meet regularly to discuss the trainee's progression in residency. These meetings also serve as an open forum to identify challenges faced by the trainee, both professionally and personally. Given the significant limitations in accurate faculty predictions of trainee burnout, our results suggest that training programs may not be able to rely on faculty reporting as a primary method of detecting trainee distress. Residency programs may need to improve faculty education on recognizing burnout or create other venues to identify trainee burnout. One such opportunity may include regular screenings of trainee distress using validated instruments like the MBI, with results of these screenings confidentially disclosed to respondents so that those who choose to self‐refer for help may do so. Although the sensitivity of faculty predictions of trainee burnout in our study was low, the PPV was relatively high. This suggests that trainees who faculty advisers believe may be experiencing burnout deserve the attention of program leadership in improving their training experience.
The impact of burnout on trainees’ future careers and job retention remains unclear.7 It is troubling, however, that EM trainees report similarly high levels of burnout as practicing EM attendings.2, 3, 4 Coupled with recent evidence demonstrating an association between physician burnout and lower quality of care,3, 8, 12 accurate identification and effective methods of addressing physician burnout are paramount.
Limitations
There are several limitations to this exploratory study. First, our results from a small sample of EM trainees and faculty at a single site with a specific mentorship structure may not be generalizable to programs, faculty and trainees in other environments. Second, the impact of our survey response rates on outcomes is unclear. The majority of our faculty and trainee respondents were male and we could not determine the influence of gender in our analyses. We were also unable to compare burnout between responding and nonresponding trainees as well as differences in burnout predictions between responding and nonresponding faculty advisers. Third, because each trainee is paired with a single faculty adviser, we could not determine inter‐rater reliability in faculty predictions of trainee burnout. Finally, it remains uncertain if designated faculty advisers were the best faculty to assess their advisees’ burnout. Although faculty advisers and advisees are paired by the residency program, they may not meet regularly as they are instructed or trainees may seek the mentorship of other “unofficial” faculty advisers.
Conclusions
In this pilot study EM faculty prediction of trainee burnout was poor. Improved faculty education on recognizing burnout or other methods of identifying trainee burnout may be needed. Future work involving multiple sites and larger samples is necessary to confirm these results.
Supporting information
Data Supplement S1. Faculty survey predicting resident burnout.
AEM Education and Training 2017;1:75–78
The authors have no relevant financial information or potential conflicts to disclose.
References
- 1. Maslach C, Jackson SE, Leiter MP. Maslach Burnout Inventory Manual, 3rd ed Palo Alto: Consulting Psychologists Press, 1996. [Google Scholar]
- 2. Shanafelt TD, Boone S, Tan L, et al. Burnout and satisfaction with work‐life balance among US physicians relative to the general US population. Arch Intern Med 2012;172:1377–85. [DOI] [PubMed] [Google Scholar]
- 3. Lu DW, Dresden S, McCloskey C, Branzetti J, Gisondi MA. Impact of burnout on self‐reported patient care among emergency physicians. West J Emerg Med 2015;16:996–1001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Kimo Takayesu J, Ramoska EA, Clark TR, et al. Factors associated with burnout during emergency medicine residency. Acad Emerg Med 2014;21:1031–5. [DOI] [PubMed] [Google Scholar]
- 5. Shanafelt TD, Balch CM, Dyrbye L, et al. Special report: suicidal ideation among American surgeons. Arch Surg 2011;146:54–62. [DOI] [PubMed] [Google Scholar]
- 6. Dewa CS, Loong D, Bonato S, Thanh NX, Jacobs P. How does burnout affect physician productivity? A systematic literature review. BMC Health Serv Res 2014;14:325. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Dewa CS, Jacobs P, Thanh NX, Loong D. An estimate of the cost of burnout on early retirement and reduction in clinical hours of practicing physicians in Canada. BMC Health Serv Res 2014;14:254. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Lu DW, Dresden SM, Courtney DM, Salzman DH. An investigation of the relationship between emergency medicine trainee burnout and their clinical performance in a high‐fidelity simulation environment. Acad Emerg Med E&T 2017;1:55–59. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Eckleberry‐Hunt J, Van Dyke A, Lick D, Tucciarone J. Changing the conversation from burnout to wellness: physician well‐being in residency training programs. J Grad Med Educ 2009; 1:225–30. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Accreditation Council for Graduate Medical Education Common Program Requirements – Section VI . Summary and Impact of Major Requirement Revisions. Available at: http://www.acgme.org/Portals/0/PFAssets/ReviewandComment/CPR_SectionVI_ImpactStatement.pdf. Accessed Nov 17, 2016.
- 11. Dyrbye LN, West CP, Shanafelt TD. Defining burnout as a dichotomous variable. J Gen Intern Med 2009;24:440.author reply 1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Nelson KM, Helfrich C, Sun H, et al. Implementation of the patient‐centered medical home in the Veterans Health Administration: associations with patient satisfaction, quality of care, staff burnout, and hospital and emergency department use. JAMA Intern Med 2014; 174:1350–8. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Supplement S1. Faculty survey predicting resident burnout.