Comparing the Maslach Burnout Inventory to Other Well-Being Instruments in Emergency Medicine Residents

ABSTRACT

Background 

The Maslach Burnout Inventory (MBI) is considered the “gold standard” for measuring burnout, encompassing 3 scales: emotional exhaustion, depersonalization, and personal accomplishment. Other well-being instruments have shown utility in various settings, and correlations between MBI and these instruments could provide evidence of relationships among key variables to guide well-being efforts.

Objective 

We explored correlations between the MBI and other well-being instruments.

Methods 

We fielded a multicenter survey of 9 emergency medicine (EM) residencies, administering the MBI and 4 published well-being instruments: a quality-of-life assessment, a work-life balance rating, an appraisal of career satisfaction, and the Primary Care Evaluation of Mental Disorders Patient Health Questionnaire 2 question screen. Consistent with the Maslach definition, burnout was defined by high emotional exhaustion (> 26) and high depersonalization (> 12).

Results 

Of 334 residents, 261 (78%) responded. Residents who reported lower quality of life had higher emotional exhaustion (ρ = –0.437, P < .0001), higher depersonalization (ρ = –0.18, P < .005), and lower personal accomplishment (ρ = 0.347, P < .001). Residents who reported a negative work-life balance had emotional exhaustion (P < .001) and depersonalization (P < .009). Positive career satisfaction was associated with lower emotional exhaustion (P < .0001), lower depersonalization (P < .005), and higher personal accomplishment (P < .05). A positive depression screen was associated with higher emotional exhaustion, higher depersonalization, and lower personal achievement (all P < .0001).

Conclusions 

Our multicenter study of EM residents demonstrated that assessments using the MBI correlate with other well-being instruments.

What was known and gap

The Maslach Burnout Inventory (MBI) is the accepted tool for measuring burnout; its proprietary nature and length present barriers to wide use in resident physicians.

What is new

A multicenter survey of 9 emergency medicine programs assessed correlations of the MBI and 4 other published well-being instruments.

Limitations

Single specialty study, single application of the assessment without an intervention.

Bottom line

Assessments using the MBI correlate with simpler instruments that could be used to assess resident well-being.

Introduction

Occupational burnout develops due to long-term, unresolvable job stress, and is defined as a triad of emotional exhaustion, depersonalization, and low personal achievement.¹ The prevalence of burnout continues to rise, and it affects medical students, residents, and more than half of all practicing physicians.^2–4 Burnout has negative implications for the physical and mental health of physicians,^5–8 career satisfaction,^9,10 and patient care,^11–15 and may have deleterious effects on patient care.¹⁶

The Maslach Burnout Inventory (MBI) is a 22-item instrument that is considered the “gold standard” for assessing burnout.¹ While the MBI provides valuable information, critiques include its proprietary nature, the lack of an established level of burnout that correlates with negative outcomes, and the observation that the 3 categories measured do not translate into target areas for interventions.^17,18 Identifying additional psychometric variables that correlate with the MBI may assist in identifying and addressing underlying attributes that manifest as burnout. Use of other instruments also may reduce the potential for survey fatigue from repeated use of the MBI. Correlating the MBI with other measures also can provide validity evidence for these tools.¹⁹

We sought to compare the MBI to 4 published well-being instruments^20–23 to determine to what degree scores correlate with the MBI. We hypothesized that these tools can be used to perform alternative assessments of resident well-being. We also measured the levels of burnout and positive depression screens in a cohort of emergency medicine (EM) residents.

Methods

Study Design

This was a multicenter survey study conducted at 9 Accreditation Council for Graduate Medical Education–accredited EM residencies who had members of the Emergency Medicine Education Research Alliance as core faculty.

Study Protocol

Eligible participants were postgraduate year 1 (PGY-1) to PGY-4 EM residents. Residents were recruited and participated during programs' weekly education conference in February 2017. Participation in the survey study was voluntary, and no incentive was offered for participation. Participants provided informed consent. Follow-up for nonresponders was program-specific, either in person or by e-mail.

The study was reviewed by each institution's Institutional Review Board and received approval prior to study initiation.

Data Collection Tools and Analysis

Burnout was assessed using the 22-question MBI, which has validity evidence for use in health- and service-related occupations.^1,24 Consistent with the Maslach definition, burnout was defined by both high emotional exhaustion (> 26) and high depersonalization (> 12). Additional categorizations are available as online supplemental material.^1,25,26

Visual analog scales (VASs) have been used since the 1970s to measure quality of life, with the assessment consisting of a single-item linear analog scale assessment: “How would you rate your overall quality of life during the past week?” with the score treated as a continuous variable.²⁰ One study compared the validity, reliability, and responsiveness of a single-item quality-of-life VAS to multi-item scales, including the Medical Outcomes Study 20-Item Short Form (MOS SF-20) and the Rotterdam Symptom Checklist (RSCL), showing that the VAS demonstrated high correlations with the MOS SF-20 health perceptions scale and moderate to high correlation with all other subscales of the MOS SF-20 and RSCL.²⁷

Depression was screened using the Primary Care Evaluation of Mental Disorders Patient Health Questionnaire 2 question screen (PRIME-MD PHQ-2): “During the past month, have you often been bothered by feeling down, depressed, or hopeless?” and “During the past month, have you often been bothered by little interest or pleasure in doing things?”²¹ A yes response to either question is considered a positive screen for depression.²¹ When compared with clinical interviews, a positive response on the 2-item instrument had a sensitivity of 96% and a specificity of 57% for detecting depression and performed similarly to longer case-finding instruments.²¹

We administered 2 questions, previously published by Shanafelt et al,^22,23 to assess career satisfaction and work-life balance. Career satisfaction was assessed by the question: “If given the opportunity to revisit your career choice, would you choose to become a physician again?” Responses of likely and very likely were categorized as positive for career satisfaction.²² Work-life balance was assessed with the question, “Does my work schedule leave me enough time for my personal/family life?” Responses of strongly agree and agree were categorized as positive for work-life balance.²³

We also obtained basic demographic information (age, sex, and PGY level). Descriptive statistics are presented as total number (n) and percentages for categorical variables. Continuous variables are displayed as means with SD for normally distributed variables or as median values with interquartile ranges (IQRs) for nonnormally distributed variables. Data were analyzed using Spearman correlation (for nonnormally distributed continuous variables), χ², or Student's t test as appropriate for continuous or categorical variables. Analysis was performed using a statistical package program (R version 3.3.2, The R Foundation, Vienna, Austria).

Results

Of 334 EM residents, 261 (78%) responded. The median respondent age was 29 years (IQR, 28–31 years). Of the 261 respondents, 76 (29%) were PGY-1, while 88 (34%), 85 (33%), and 10 (4%) were PGY-2, PGY-3, and PGY-4, respectively; 2 (1%) did not respond to this question. There were 185 (71%) male and 76 (29%) female respondents.

The median score on the quality-of-life VAS score was 75 (IQR, 60–83.75). Residents' impressions of their quality of life were significantly correlated to MBI burnout indices, with lower scores for quality of life associated with higher scores for emotional exhaustion (ρ = –0.437; 95% confidence interval [CI] –0.523 to –0.332; P < .0001), higher depersonalization (ρ = –0.18; 95% CI 0.234–0.451; P < .005), and personal accomplishment (ρ = 0.347, P < .001).

Among respondents, 136 (52%) reported a positive work-life balance. Those who reported a negative work-life balance scored higher on emotional exhaustion and depersonalization (Table). Work-life balance ratings were not significantly associated with personal accomplishment.

Table.

Comparison of Mean Maslach Burnout Inventory (MBI) Component Scores for Positive and Negative Screens of Other Measures

A total of 220 residents (84%) reported positive career satisfaction, which was associated with lower scores on emotional exhaustion and depersonalization, and higher scores on personal accomplishment (Table).

The prevalence of a positive depression screen in our sample was 40%. Screening positive on the PRIME-MD PHQ-2 was associated with higher scores on emotional exhaustion, higher depersonalization scores, and lower scores on personal achievement (Table). A total of 47 residents (18%) of the study population scored positive for burnout (emotional exhaustion > 26 and depersonalization > 12).

Discussion

In this study of EM residents, scores on the MBI consistent with burnout significantly correlated with lower scores for quality of life, negative assessments of resident career satisfaction, and negative scores for work-life balance. Our results support the use of these tools as additional or alternative well-being assessments for medical residents. Well-being is dynamic, and educators may want to evaluate well-being multiple times during training. By demonstrating correlation with the MBI, educators could utilize these shorter and nonproprietary instruments as an alternative to repetitive administrations of the MBI.

There is a precedent for comparing well-being instruments. The Satisfaction with Life Scale, which assesses global perceptions of satisfaction with life, has been found to have a negative correlation with the Beck Depression Inventory and the Symptom Checklist-90, a widely used measure of psychologic distress.²⁸

There was a positive depression screen rate of 40% and a burnout prevalence of 18% in our study population of EM residents. The prevalence of positive depression screens in our survey sample is significantly higher than previously reported rates. Katz et al,²⁹ who utilized the Center for Epidemiologic Studies Depression Scale (a 20-item questionnaire with a lower sensitivity of 85% and a higher specificity of 72% compared with the PRIME-MD PHQ-2), noted a 12% prevalence of depression in EM residents. A systematic review and meta-analysis by Mata and colleagues³⁰ demonstrated a pooled prevalence of depression and depressive symptoms of 29% among resident physicians. It is uncertain whether this prevalence variation reflects different measurement methods, as the PRIME-MD PHQ-2 has a lower specificity than several other utilized instruments, or if EM residents are currently more likely to screen positive for depression compared with residents in other specialties and historical values. Conversely, the burnout prevalence of 18% in our study population is notably less than the approximately 50% previously reported.¹³ The variability in reported prevalence of burnout may reflect its dynamic nature. While Maslach defined burnout as including both emotional exhaustion and depersonalization, other studies did not require both components to define burnout, potentially leading to a prevalence overestimation.^1,26

There are several limitations to this study. Our single specialty study may not generalize to other specialties. As well-being and burnout are dynamic processes, results may have differed if the assessments were done at a different time of the year or on multiple occasions. We obtained an acceptable response rate of 78%, and it is possible that nonresponders represented a skewed population not captured in the results. Individual responses involving the Likert scale and multiple-choice questions are subjective and may fail to fully measure the construct of interest.

Future research in this area could include longitudinal studies using the 2 additional well-being instruments in residents over time, and assessing the impact of a well-being intervention using 1 or more of these instruments as outcome measures.

Conclusion

A national multicenter survey study of EM residents showed that lower scores on a quality-of-life scale, a negative work-life balance rating, lower appraisal of career satisfaction, and a higher rate of screening positive on the PRIME-MD PHQ-2 all were associated with increasing levels of burnout as measured by the MBI and defined by high emotional exhaustion and high depersonalization. The instruments tested in this study may be useful in assessing well-being in residents.