Mindfulness, burnout, and effects on performance evaluations in internal medicine residents

Abstract

Purpose

Burnout has been documented at high levels in medical residents with negative effects on performance. Some dispositional qualities, like mindfulness, may protect against burnout. The purpose of the present study was to assess burnout prevalence among internal medicine residents at a single institution, examine the relationship between mindfulness and burnout, and provide preliminary findings on the relation between burnout and performance evaluations in internal medicine residents.

Methods

Residents (n = 38) completed validated measures of burnout at three time points separated by 2 months and a validated measure of dispositional mindfulness at baseline. Program director end-of-year performance evaluations were also obtained on 22 milestones used to evaluate internal medicine resident performance; notably, these milestones have not yet been validated for research purposes; therefore, the investigation here is exploratory.

Results

Overall, 71.1% (n = 27) of the residents met criteria for burnout during the study. Lower scores on the “acting with awareness” facet of dispositional mindfulness significantly predicted meeting burnout criteria χ²(5) = 11.88, p = 0.04. Lastly, meeting burnout criteria significantly predicted performance on three of the performance milestones, with positive effects on milestones from the “system-based practices” and “professionalism” domains and negative effects on a milestone from the “patient care” domain.

Conclusion

Burnout rates were high in this sample of internal medicine residents and rates were consistent with other reports of burnout during medical residency. Dispositional mindfulness was supported as a protective factor against burnout. Importantly, results from the exploratory investigation of the relationship between burnout and resident evaluations suggested that burnout may improve performance on some domains of resident evaluations while compromising performance on other domains. Implications and directions for future research are discussed.

Introduction

Burnout is defined as a state of work-related stress which causes health care professionals to experience emotional exhaustion, depersonalization, and decreased sense of personal accomplishment.^1,2 High levels of burnout have been documented in physicians and medical residents^3–5 with costly effects on resident quality of life, patient safety, and the health care system as a whole.^6–14 Burnout in internal medicine physicians is often reported at higher levels than in other medical specialties.^3,5 However, the reason for this difference is unclear and may be due to specific work duties or may represent factors unrelated to resident speciality.^3,5 It is important to advance our understanding of the negative correlates of burnout during medical training to inform appropriate interventions for decreasing burnout and its negative sequelae. By focusing on internal medicine residents for whom burnout is reported at high levels, the information gained may inform future research in medical residents across specialties.

Mindfulness has been defined as a state of nonjudgmental awareness in the present moment.¹⁵ Mindfulness-based interventions may be especially beneficial for decreasing burnout in health care professionals^16,17 by enhancing attentional capacities under stress and resiliency to stress.^18,19 Dispositional mindfulness, or the tendency to be mindful in one’s daily life, has been shown to increase following mindfulness training,²⁰ but also represents an individual’s tendency to be mindful unrelated to formal training.^21,22 Dispositional mindfulness is measured using validated self-report questionnaires and has been positively associated with adaptive health outcomes.^21,22 However, few studies have investigated the relationship between dispositional mindfulness and burnout in medical residents.²³ An investigation of this relationship could inform residency program directors on intervention development and help to identify resiliency factors for the prevention of resident burnout.

Across disciplines, occupational stress has been shown to negatively affect cognitive functioning and work-related performance.^24–26 However, research findings on the relationship between burnout and performance in medical residents have been inconsistent. Burnout tends to correlate negatively with perceived medical errors^8,9,12,14 and patient satisfaction.⁶ Conversely, when work performance is measured using chart audits²⁷ or observed medical errors,²⁸ no relation is found between burnout and performance. Others have found positive associations between burnout and poorer examination scores in medical residents.¹³

Notably, burnout is consistently related to anxiety, increased risk for cardiovascular disease, poor adrenal functioning, and suicide.^29,30 Therefore, it seems likely that burnout negatively affects performance, and the lack of consistent association found between burnout and objective measures of performance in medical residents may be due to issues of operationalizing performance. There may be particular aspects of resident performance that are negatively influenced by the presence of high burnout, specifically patient care or performance on more cognitively demanding tasks. The Internal Medicine resident milestones developed by the Accreditation Council for Graduate Medical Education (ACGME) may give important nuanced understanding of how burnout affects different educational competencies, yet to our knowledge, no research has investigated this relationship. The Internal Medicine resident milestones developed by the ACGME represent a shift toward competency-based education in residency training.^31,32 These milestones are used to guide evaluation of residents and were developed by a taskforce with expertise in the areas of internal medicine, program evaluation, and graduate medical education.³¹ They are widely used and provide national standardization for graduate training in internal medicine. However, no research has explored their relationship to burnout. Therefore, the present study sought to address this gap in the literature to better understand how burnout may affect internal medicine residents on relevant aspects of their performance.

Here findings are reported on burnout prevalence at a single institution throughout half a residency year, the relationship between dispositional mindfulness (as measured by a self-report scale) and burnout, and on an analysis of the relationship between burnout and Internal Medicine resident milestones developed by the ACGME. It was hypothesized that higher levels of dispositional mindfulness in residents would decrease likelihood of meeting criteria for burnout. Given the conflicting findings relating resident burnout to performance, and the exploratory nature of the final aim, there were no hypotheses made regarding the relationship between burnout and Internal Medicine resident milestones.

Methods

Participants/procedures

All internal medicine residents at Virginia Commonwealth University (VCU) were eligible to participate. Residents who agreed to participate provided written informed consent. Ethical approval was granted from the VCU Institutional Review Board (IRB), 7/10/2014-6/30/2015, IRB number HM20001840. Burnout was measured three times separated by 2.5 months beginning with the start of a residency year (July, October, and December). Data on dispositional mindfulness and demographic variables were obtained at baseline. Internal Medicine resident milestones were obtained at study end in December 2014.

Measures

Burnout

The Maslach Burnout Inventory (MBI) measures burnout in three domains: emotional exhaustion, depersonalization, and lack of personal accomplishment.^1,2 The MBI is a 22-item measure asking respondents to rate the frequency of several feelings and experiences on a 7-point Likert scale ranging from “never” to “daily”. This instrument is the gold standard for measuring burnout in medical professionals and is well validated in physician and resident populations.³³ Scores of 27 or higher on the emotional exhaustion subscale or scores of 10 or higher on the depersonalization subscale are considered criteria for meeting burnout. ^1,7,12,34 Consistent with previous literature, low scores on the personal accomplishment subscale are not considered burnout, because this may measure a distinct dimension.¹ The subscales of emotional exhaustion and depersonalization demonstrated adequate to good internal consistency (α = 0.85–0.91 and 0.77–0.84, respectively).

Mindfulness

The Five Facet Mindfulness Questionnaire Short Form (FFMQ-sf) is a 24-item measure, derived from the 39-item FFMQ, which assesses dispositional mindfulness on a five-level Likert scale.³⁵ The FFMQ-sf captures the tendency to be mindful in one’s life using five distinct subscales, “observe”, “describe”, “acting with awareness”, “nonjudging of inner experience”, and “non-reactivity to inner experience”. Higher scores indicate higher levels of mindfulness. The FFMQ-sf is well validated.³⁵ The five facets demonstrated acceptable to excellent internal consistency (α = 0.73–0.91).

Stress and depression

The Depression Anxiety Stress Scales short form (DASS 21)³⁶ was used to measure stress and depression at baseline as a potential covariate. Only the depression and stress subscales (14 items) were administered. Participants were asked to rate the degree to which statements applied to them over the last week on a 4-point Likert scale ranging from “did not apply to me at all” to “applied to me very much, or most of the time”. The DASS 21 is well validated and well normed in non-clinical populations.³⁷ The subscales demonstrated adequate to good internal consistency, alphas ranged from 0.78 to 0.81.

Internal Medicine resident milestones

For each resident, an aggregate of evaluations from attending physicians and faculty was discussed during a three-person committee meeting made up of department chiefs providing performance scores for each resident on 22 milestones developed by the ACGME, each of which falls under one of the six competencies: “patient care”, “medical knowledge”, “practice-based learning and improvement”, “interpersonal and communication skills”,” professionalism”, and “systems-based practice”. All milestones were rated on a 5-point scale ranging from “critical deficiencies” to “aspirational” based on nation-wide standards developed by the ACGME.³¹ It was expected that first, second, and third year residents would vary in their milestone ratings as their performance should improve as they advance in years in training. Therefore, performance on each milestone was compared with the residency class to which each participant belonged, to determine whether it fell below the class average or at or above the class average, making performance on each milestone a dichotomous variable.

Statistical plan

A hierarchical logistic regression was conducted to test the hypothesis that dispositional mindfulness would predict burnout criteria. Chi-square tests were conducted to evaluate whether burnout was associated with performance on the 22 Internal Medicine resident milestones. Data were analyzed using SPSS version 22 (IBM Corporation, Armonk, NY, USA). All data met assumptions for statistical analyses. Importantly, for the hierarchical logistic regression, assumptions of normality were tested via skewness and kurtosis statistics < 1.5, bivariate and multivariate outliers using Mahalanobis distance, and homoscedasticity of error variance by examining the residual plot. All assumptions were met. Finally, the Hosmer and Lemeshow test was used to determine goodness-of-fit.

Results

Of the 129 residents in the internal medicine residency program, 38 participated in the study (30% response rate). One participant did not provide a valid identifier, therefore performance evaluations were collected for 37 residents. Participants were mostly men (n = 30, 79%) and the average age was less than 30 years (M = 28.59, SD = 2.69). Half of the sample was White (n = 19), 26.3% (n = 10) was Asian, 2.6% (n = 1) Hispanic, 5.3% (n = 2) identified as other, and 15.8% (n = 6) chose not to disclose. Most of the participants were interns (n = 20, 52.6%) or third year residents (n = 16, 42.1%), with second year residents underrepresented (n = 2, 5.3%).

Prevalence of burnout

Of the 38 residents surveyed, 27 (71.1%) met criteria for burnout at least once during the course of the study. At any of the three assessments, 12–19 (31.6%–50.0%) residents met criteria for burnout. Average burnout ranged from moderate to severe (Table 1). These prevalence rates are consistent with other studies investigating burnout in internal medicine residents (Ishak et al).⁵

Table 1.

Prevalence of burnout at three time points

Burnout	Time point 1 (n = 32)		Time point 2 (n = 31)		Time point 3 (n = 24)
	M/n	SD/%	M/n	SD/%	M/n	SD/%
MBI EE	21.91	7.95	21.32	8.20	17.33	8.34
MBI DP	10.78	6.31	9.84	5.76	9.33	5.92
Total meeting criteria	19	50%	18	47.4%	12	31.6%

Notes: Scores on the MBI EE ≥27 are considered high burnout. Scores on the MBI DP ≥10 are considered high burnout.

Abbreviations: MBI, Maslach Burnout Inventory-Human Services; EE, emotional exhaustion subscale; DP, depersonalization subscale; M, mean; SD, standard deviation.

Dispositional mindfulness as a predictor of burnout criteria

A hierarchical logistic regression was performed to test whether dispositional mindfulness predicted burnout criteria. Gender, baseline stress, and baseline depression were entered at Step 1. The five subscales of the FFMQ were entered at Step 2 (Table 2). Results from the Homer and Lemeshow goodness-of-fit test compare the observed values with a perfect model and the test was non-significant, suggesting goodness of fit; χ²(8) = 6.59, p = 0.58. The second model significantly distinguished between residents who met criteria and those who did not meet criteria for burnout, χ²(5) = 11.88, p = 0.04. The model effect size was strong: Nagelkerke R² = 0.64. The model accurately predicted residents not meeting criteria for burnout 76.9% of the time, and was 89.5% accurate at predicting residents who did meet criteria for burnout, for an overall accuracy of 84.4%. The “acting with awareness” subscale of the FFMQ was the only significant predictor of burnout criteria, with lower dispositional mindfulness increasing the likelihood of meeting criteria for burnout.

Table 2.

Hierarchical logistic regression of dispositional mindfulness predicting burnout

Step	Predictor	B	SE B	p-value	Exp(B)	95% CI
Step 1	Gender	−2.02	1.85	0.28	0.13	0.003–5.004
	Stress	0.191	0.29	0.50	1.21	0.69–2.12
	Depression	−0.76	0.44	0.08	0.47	0.20–1.10
Step 2	FFMQ Nonreact	−0.07	0.20	0.74	0.93	0.63–1.39
	FFMQ Observe	−0.29	0.21	0.17	0.75	0.50–1.13
	FFMQ ActAware	−0.59	0.29	0.04*	0.55	0.32–0.97
	FFMQ Describe	0.05	0.23	0.84	1.05	0.67–1.63
	FFMQ Nonjudge	−0.11	0.17	0.51	0.89	0.64–1.25

Notes:

^*p < 0.05. Nonreact: nonreactivity to inner experience. ActAware: acting with awareness. Nonjudge: nonjudging of inner experience.

Abbreviations: B, unstandardized beta weight; SE, standard error; Exp(B), odds ratio; CI, confidence interval; FFMQ, Five Facet Mindfulness Questionnaire.

Association between burnout and performance evaluations

If residents met criteria for burnout at any one of the three time points, they were considered to have at least one episode of burnout during the study. One of the “patient care” performance milestones, “Manages patients with progressive responsibility and independence (PC3)” was significantly related to burnout, χ² (1) = 4.65, p = 0.031, such that meeting criteria for burnout was related to below average performance. Two other milestones, one measuring attendance at conferences and completion of paperwork, “Accepts responsibility and follows through on tasks (PROF2)”, χ² (1) = 4.40, p = 0.036, and another measuring patient referrals “Transitions patients effectively within and across health delivery systems (SBP4)”, χ² (1) = 4.77, p = 0.029 were unexpectedly positively related to burnout; that is, meeting criteria for burnout was related to above average performance. Tables 3–5 present data from the χ² classification tables for each milestone.

Table 3.

Chi-square classification table for milestone: “Manages patients with progressive responsibility and independence (PC3)”

Milestone	Performance score	Burnout criteria		Total
		Not met	Met
PC3 Criteria	Below average	2	12	14
	At or above average	11	12	23
	Total	13	24	37

Abbreviation: PC, patient care.

Table 4.

Chi-square classification table for milestone: “Transitions patients effectively within and across health delivery systems (SBP4)”

Milestone	Performance score	Burnout criteria		Total
		Not met	Met
SBP4 Criteria	Below average	7	4	11
	At or above average	6	18	24
	Total	13	22	35

Abbreviation: SBP, system-based practice.

Table 5.

Chi-square classification table for milestone: “Accepts responsibility and follows through on tasks (PROF2)”

Milestone	Performance score	Burnout criteria		Total
		Not met	Met
PROF2 Criteria	Below average	10	10	20
	At or above average	3	14	17
	Total	13	24	37

Abbreviation: PROF, professionalism.

Discussion

This was the first study to show that dispositional mindfulness predicts burnout criteria. We also provided exploratory data on the effect of burnout on the Internal Medicine resident milestones developed by the ACGME. Burnout rates were high in this sample, providing evidence to support the need for efforts to improve burnout in internal medicine residents. However, this must be qualified given the low response rate. The hypothesis that dispositional mindfulness would predict burnout criteria was supported; specifically, “acting with awareness”, the ability to attend to the moment, or the opposite of automatic pilot, may be especially helpful in reducing and/or protecting against burnout in internal medicine residents. Interventions developed to improve mindfulness may improve residents’ resiliency and decrease burnout.

Exploratory analyses revealed that meeting criteria for burnout increased the likelihood of performing below average on the milestone “Manages patients with progressive responsibility and independence”. Burnout was associated with a decrease in residents’ ability to act independently as physicians – perhaps relying more on their attending or senior resident to make decisions.

Surprisingly, the milestones “Transitions patients effectively within and across health delivery systems” from the “system-based practice” competency and “Accepts responsibility and follows through on tasks” from the “professionalism” competency were related to burnout criteria such that meeting criteria for burnout increased the likelihood of performing at or above average. Discussion with program directors revealed that these milestones assessed attendance at weekly conferences, completion of paperwork, and generally, higher levels of professional behavior. Striving to perform at a higher level of professional care may lend itself to higher burnout in residents at a cost to their self-care. However, more research is necessary to cross-validate this relationship in larger samples.

If these findings are replicated, it may suggest a need to discover individual- and system-level approaches that will decrease burnout without sacrificing professionalism and attendance-based performance. It should be recognized that burnout may be promoting certain competencies of residency at a cost to both resident health and other competencies of work-related performance, including patient care.

Study limitations include the small sample size, which limits the generalizability and interpretation of findings. Further, it limited the power to detect statistical significance, especially in tests of the relationship between performance and burnout. Notably, adjusting for family-wise error rate using a Bonferroni correction made these analyses non-significant. Therefore, it is possible that the three significant associations between Internal Medicine resident milestones and burnout were simply due to multiple comparisons. However, these tests are exploratory in nature and should be qualified as such. These results should be used to inform future studies investigating the relationship between Internal Medicine resident milestones and burnout. The sample size also limits the interpretation of the logistic regression investigating the relationship between dispositional mindfulness and burnout, as the sample size is smaller than recommended for this type of analysis. However, all assumptions were met and the model indicated a good fit; therefore, the relationship between burnout and dispositional mindfulness presented here warrants qualified consideration and future investigations to build upon these findings. This study is also limited by the static diagnosis of burnout caused by grouping residents as burnt out if they met criteria at any time point. However, residents who met criteria for burnout throughout the first half of a residency year may be different than residents who did not; therefore, these differences are especially compelling in how they relate to performance and what factors may be protective against burnout, including mindfulness.

The response rate in this study prevents elimination of selection bias as a potential confounding variable. Although it was developed by experts and has good face and content validity³¹ the measure of performance has not been validated against external criteria, given that this is the first study to investigate the relation between Internal Medicine resident milestones from program directors and burnout. Second year residents were underrepresented in this study due to difficulties in recruitment scheduling, thereby limiting generalizability. Finally, this study was limited by assessing only one internal medicine program, as well as by assessing only one specialty. However, these preliminary data may inform future larger scale studies. Subsequent research should evaluate several medical residency specialties, throughout their entire residency, on measures of burnout and investigate effects on resident performance evaluations.

Conclusion

Burnout may be associated with better performance on some competencies and poorer performance on other competencies; therefore, strategic intervention should be investigated to preserve work-related performance while improving resident health. Interventions that improve mindfulness may be especially helpful for decreasing burnout, as our findings suggest here, while also improving performance, as demonstrated elsewhere.^38–41 More research is needed to understand the effects of burnout on performance in medical residents and how program directors across specialties may help to ameliorate these effects.