Reliability and validity support for an abbreviated Copenhagen burnout inventory using exploratory and confirmatory factor analysis

Abstract

Objective

The Copenhagen Burnout Inventory (CBI) is an open‐access, valid, and reliable instrument measuring burnout that includes 19 items distributed across the following 3 domains (factors): personal burnout, work burnout, and patient burnout. The primary objective of this study was to determine the validity and reliability of an abbreviated CBI to assess burnout in emergency medicine residents.

Methods

This cross‐sectional study used data from the CBI that followed the 2021 American Board of Emergency Medicine In‐training Examination. Exploratory factor analysis (EFA) was followed by confirmatory factor analysis (CFA).

Results

Of the 8491 eligible residents, 7225 (85.1%) completed the survey; the EFA cohort included 3613 residents and the CFA cohort included 3612 residents. EFA showed 2 eigenvalues ≥1, an internal factor and an external factor. There were 6 CBI items that contributed to the 2 factors. The first factor was related to personal burnout and work‐related burnout and the second factor was related to working with patients. There were 4 CBI items that contributed to the internal factor and 2 CBI items that contributed to the external factor. Using the abbreviated CBI, the incidence of a resident having 1 or both types of burnout was 34.1%.

Conclusions

This study provides validity evidence and reliability support for the use of a 6‐item, 2‐factor abbreviated CBI. A shorter, reliable, valid, and publicly accessible burnout inventory provides numerous advantages for burnout research in emergency medicine.

1. INTRODUCTION

1.1. Background

Burnout among physicians is associated with numerous negative ramifications, including medical error, ¹ , ² poor job satisfaction, ³ , ⁴ decreased professional fulfillment, ⁵ increased alcohol and drug use, ⁶ , ⁷ , ⁸ and increased depression and suicidal ideation. ⁹ The prevalence of burnout among emergency medicine residents varies widely, depending on the methods used and definitions of burnout. ¹⁰ Using criteria applied to an abbreviated Maslach Burnout Inventory, the prevalence is 28% during the first‐year of an emergency medicine residency and increases to more than 40% in the final year of training. ¹¹ Another recent study reported that the overall incidence was 30% (Lu DW, Zhan T, Bilimoria KY, et al unpublished data, 2021). ¹² Determining the prevalence of burnout is complicated by the varied definitions that are applied to this enigmatic syndrome, ¹² as well as the arduousness and expense in using some burnout inventories. Identifying burnout requires the application of a valid and reliable measurement instrument. There are a limited number of burnout inventories used to assess physicians, some of which are proprietary and require the payment of fees for use. For large studies, these fees can be cost prohibitive.

The Copenhagen Burnout Inventory (CBI) is an open‐access instrument that involves 19 items distributed across the following 3 domains (factors): personal burnout, work burnout, and patient burnout (Table 1). The CBI has been used to assess varied types of health care personnel in several countries, amassing substantial validity evidence. ¹³ , ¹⁴ , ¹⁵ , ¹⁶ The CBI has been applied extensively to physicians. ¹⁷ , ¹⁸ , ¹⁹ , ²⁰ , ²¹ , ²² , ²³ , ²⁴ , ²⁵ , ²⁶ , ²⁷ , ²⁸ , ²⁹ , ³⁰ , ³¹ , ³² , ³³ , ³⁴ , ³⁵ , ³⁶ Despite widespread international use in measuring burnout among physicians, the CBI has been used infrequently to measure burnout among emergency physicians in the United States. ³⁷ , ³⁸

TABLE 1.

Copenhagen Burnout Inventory (CBI)

Personal burnout
1	How often do you feel tired? a
2	How often are you physically exhausted? a
3	How often are you emotionally exhausted? a
4	How often do you think: “I can't take it anymore”? a
5	How often do you feel worn out? a
6	How often do you feel weak and susceptible to illness? a
Work burnout
7	Is your work emotionally exhausting? b
8	Do you feel burned out because of your work? b
9	Does your work frustrate you? b
10	Do you feel worn out at the end of the working day? a
11	Are you exhausted in the morning at the thought of another day at work? a
12	Do you feel that every working hour is tiring for you? a
13	Do you have enough energy for family and friends during leisure time?a,c
Patient burnout
14	Do you find it hard to work with patients? b
15	Do you find it frustrating to work with patients? b
16	Does it drain your energy to work with patients? b
17	Do you feel that you give more than you get back when you work with patients? b
18	Are you tired of working with patients? a
19	Do you sometimes wonder how long you will be able to continue working with patients? a

^a 5‐point rating scale: never/almost never, seldom, sometimes, often, always.

^b 5‐point rating scale: to a very low degree, to a low degree, somewhat, to a high degree, to a very high degree.

^c Reverse scored.

1.2. Importance

Burnout is a frequently reported problem within emergency medicine. ¹⁰ , ¹¹ Conducting additional research on emergency physician burnout would provide an opportunity to better characterize the root causes of burnout and to explore more system‐based interventions that could benefit the specialty. More frequent, longitudinal assessments would be easier to conduct using a shorter inventory. Demonstrating validity and reliability evidence related to emergency medicine would provide greater confidence in the application of the CBI for emergency physicians. Finally, an open source and abbreviated CBI would facilitate the ease with which the CBI could be used.

1.3. Goals of this investigation

The primary objective of this study was to determine the validity and reliability of an abbreviated CBI to assess burnout in emergency medicine residents using factor analysis.

The Bottom Line

This study found that the Abbreviated Copenhagen Burnout Inventory, a 6‐item, 2‐factor abbreviated instrument, is a reliable, valid, and publicly accessible burnout inventory.

2. METHODS

2.1. Study design and setting

This was a cross‐sectional study using data from the optional post‐examination survey on the American Board of Emergency Medicine (ABEM) In‐training Examination (ITE). The post‐ITE survey has been used for more than 20 years and gathers information about the examination experience. The 2021 ITE survey also included the 19‐item CBI. The ITE was administered from February 23 to March 5, 2021 to residents in Accreditation Council of Graduate Medical Education (ACGME)‐accredited emergency medicine residency programs.

2.2. Selection of participants

Every resident who completed the ABEM ITE was invited to voluntarily complete the post‐examination survey. All residents in United States categorical ACGME‐accredited emergency medicine residency programs were included in the study; physicians in combined training programs and international programs were excluded. Only the results from physicians who completed the CBI were included for analysis. This study was deemed exempt by the Emory Institutional Review Board (Emory University, Atlanta, GA).

2.3. Interventions

There were no interventions.

2.4. Measurements

This study used the results of the 19‐item CBI (Table 1). The CBI is divided into the following 3 sections: personal burnout (6 items); work‐related burnout (7 items); and client (patient)‐related burnout (6 items). The CBI uses a 5‐unit Likert scale that varies depending on the item. There are 2 different scales; one scale is based on frequency of occurrence and the other is based on the intensity of a feeling. The scales apply to different items in the inventory (Table 1). All responses were self‐reported. Residents were instructed to answer all questions based on the academic year at the time of the survey (July 2020 to March 2021). Survey responses were sent to a secure server at ABEM. All CBI measurements were deidentified and segregated from the ITE performance data, as well as the other survey responses.

2.5. Outcomes

The primary outcomes were identification of unique measurement factors, as well as identification of the specific CBI items that contributed to measuring the identified factors. The final outcome was to determine rating thresholds for the various factors at which burnout was likely. Rating thresholds used an anticipated frequency between 30% and 40% because of prior studies using items from the Maslach Burnout Inventory and nearly identical resident cohorts (Lu DW, Zhan T, Bilimoria KY, et al unpublished data, 2021). ¹¹ There were several intermediary results needed to determine the primary outcomes. These intermediary outcomes included interitem correlations, determinants of factorability, and several data points needed to derive a final inventory.

2.6. Analyses

The survey responses were randomized into data sets of nearly identical sizes. An ABEM staff psychometrician performed the analyses.

Exploratory factor analysis (EFA) was performed on the first data set, as the investigators made no a priori assumptions about the existence or number of factors but did assume that if multiple factors existed, those factors would be related. Before EFA, the data's amenability to factor analysis was examined using the Kaiser‐Meyer‐Olkin (KMO) Measure of Sampling Accuracy. KMO values >0.60 are considered amenable to factoring. In addition, Bartlett's Test of Sphericity was performed. Bartlett's Test of Sphericity detects redundancy between the variables. A significant Bartlett's value also indicates that observed data can be factored.

EFA was performed using direct oblimin rotation. Direct oblimin rotation was used because the various dimensions of burnout were assumed to be related based on prior research. ³⁹ Multiple methods were used to determine the appropriate number of factors to extract, including a scree plot of the eigenvalues, Horn's parallel analysis, and Velicer's minimum average partial (MAP) procedure. ⁴⁰ , ⁴¹ A scree plot of the eigenvalues is useful in visually determining the number of factors to retain. Typically, factors with eigenvalues of ≥1 are retained. Horn's parallel analysis also determines the number of factors to retain in EFA. Briefly, this method compares eigenvalues from the observed data to the 50th and 95th percentile eigenvalues from an empirical sampling distribution that is randomly generated from matrices with the same structure as the observed data. In short, the number of observed eigenvalues greater than the average or 95th percentile of simulated eigenvalues indicates how many factors should be extracted. Velicer's MAP also determines the number of components to be retained in EFA, focusing on the magnitude of variance within a correlation matrix.

Once the factors were identified for extraction, as well as the CBI items that contributed to any of the identified factors, the second response cohort underwent confirmatory factor analysis (CFA). The CFA used the Satorra–Bentler (SB) robust scaling method. Model fit used the following 4 distinct methods: (1) SB‐scaled chi‐square (χ² _SB); (2) standardized root mean square residual (SRMR); (3) SB‐scaled root mean square error of approximation (RMSEA); and (4) the SB‐scaled comparative fit index (CFI_SB). These indices are used to determine whether the derived model fits the data. The following criteria were used to assess model fit: ≤0.08 for SRMR, ≤0.08 for RMSEA, and ≥0.90 for CFI. ⁴² , ⁴³ After the CFA, the factors were evaluated to ensure that they accounted for a sufficient amount of the variance in the responses (≥10% of the variance within the abbreviated CBI). Reliability for the EFA and CFA initial cohorts used Cronbach's alpha. Reliability for the inventory resulting from CFA was calculated as coefficient omega.

The rating thresholds for any identified factors that defined burnout were reviewed; the goals were similar frequencies for various factors and a total measured frequency of burnout similar to prior levels. SAS 9.4 was the primary software platform for descriptive analysis (SAS Institute Inc., 2021). Mplus 8.6 was used to estimate all factor analysis models (Muthén & Muthén, 1998–2021).

3. RESULTS

3.1. Characteristics of study subjects

The 2021 ABEM ITE was administered to 8863 residents, of whom 8491 were residents in categorical US ACGME‐accredited emergency medicine residencies. There were 7225 emergency medicine residents who completed the survey, for a response rate of 85.1%; 1266 residents (14.9%) did not complete the survey. The EFA cohort included 3613 residents and the CFA cohort included 3612 residents.

3.2. Main results

For internal reliability, Cronbach's alpha for the original 19‐item CBI was 0.94 for both samples. The KMO Measure of Sampling Accuracy was 0.96, which was well above the threshold for factorability (0.60). Bartlett's Test of Sphericity yielded a χ² = 54,649 (171 df; P < 0.001), further confirming factorability. Interitem correlations and descriptive statistics were calculated for all items in the CBI (Table 2). Using this matrix of correlation coefficients, EFA was performed and eigenvalues were calculated. Of the 19 resulting eigenvalues, only 2 were above the 1.0 threshold for inclusion in an abbreviated CBI model (Table 3). Horn's parallel analysis (Table 3) also found that only 2 factors had eigenvalues that exceeded the parallel analysis results at both the 50th and 95th percentiles (10.33 and 2.08). A scree plot visually confirmed the identification of the 2 factors above the 1.0 level threshold (Figure 1). Velicer's MAP also indicated that 2 factors should be extracted as the smallest average squared partial correlation (0.016) occurred with the second factor.

TABLE 2.

Inter‐item correlations: descriptive statistics and interitem correlations of the 19‐item Copenhagen Burnout Inventory, for exploratory factor analysis sample (n = 3613) and confirmatory factor analysis sample (n = 3612)

	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	Mean	SD
1	—	0.72	0.66	0.48	0.69	0.41	0.57	0.59	0.48	0.64	0.58	0.50	−0.29	0.33	0.33	0.35	0.32	0.32	0.32	3.54	0.81
2	0.72	—	0.71	0.55	0.68	0.52	0.57	0.62	0.50	0.62	0.59	0.55	−0.35	0.37	0.35	0.38	0.31	0.36	0.35	3.07	0.89
3	0.66	0.71	—	0.65	0.73	0.49	0.66	0.69	0.57	0.60	0.62	0.58	−0.36	0.42	0.40	0.43	0.37	0.41	0.40	3.07	0.94
4	0.47	0.54	0.64	—	0.61	0.54	0.53	0.63	0.57	0.48	0.58	0.61	−0.39	0.45	0.43	0.45	0.35	0.47	0.48	1.94	0.97
5	0.69	0.67	0.72	0.60	—	0.53	0.62	0.69	0.58	0.66	0.64	0.59	−0.37	0.42	0.42	0.44	0.37	0.43	0.42	3.06	0.97
6	0.42	0.50	0.52	0.55	0.54	—	0.43	0.50	0.45	0.41	0.47	0.50	−0.33	0.35	0.33	0.37	0.28	0.37	0.38	1.97	0.93
7	0.57	0.57	0.67	0.53	0.62	0.44	—	0.73	0.66	0.67	0.61	0.58	−0.29	0.48	0.48	0.50	0.43	0.47	0.46	3.15	0.98
8	0.60	0.61	0.69	0.65	0.70	0.50	0.73	—	0.75	0.66	0.71	0.68	−0.41	0.54	0.51	0.54	0.44	0.55	0.54	2.77	1.07
9	0.50	0.50	0.60	0.58	0.60	0.45	0.66	0.76	—	0.58	0.63	0.65	−0.32	0.58	0.58	0.58	0.47	0.59	0.56	2.62	1.03
10	0.62	0.61	0.60	0.47	0.65	0.41	0.65	0.64	0.57	—	0.66	0.60	−0.30	0.43	0.43	0.46	0.39	0.44	0.41	3.44	0.92
11	0.58	0.58	0.60	0.58	0.65	0.48	0.61	0.70	0.64	0.66	—	0.73	−0.36	0.49	0.48	0.51	0.40	0.51	0.49	2.83	1.04
12	0.49	0.54	0.57	0.61	0.59	0.50	0.59	0.68	0.66	0.60	0.75	—	−0.39	0.54	0.52	0.55	0.41	0.57	0.53	2.24	1.00
13	−0.32	−0.34	−0.37	−0.37	−0.38	−0.33	−0.29	−0.39	−0.31	−0.28	−0.35	−0.38	—	−0.25	−0.22	−0.26	−0.20	−0.28	−0.28	3.49	0.91
14	0.33	0.36	0.42	0.46	0.42	0.37	0.48	0.53	0.58	0.43	0.51	0.57	−0.23	—	0.83	0.81	0.57	0.73	0.63	2.04	0.85
15	0.33	0.34	0.41	0.44	0.43	0.34	0.47	0.52	0.60	0.44	0.50	0.54	−0.22	0.83	—	0.82	0.61	0.73	0.63	2.14	0.88
16	0.36	0.38	0.46	0.46	0.47	0.38	0.51	0.55	0.59	0.46	0.53	0.58	−0.25	0.78	0.81	—	0.61	0.75	0.64	2.13	0.92
17	0.33	0.30	0.37	0.32	0.40	0.29	0.43	0.45	0.49	0.41	0.42	0.42	−0.20	0.57	0.61	0.62	—	0.56	0.51	2.64	1.10
18	0.33	0.34	0.41	0.46	0.44	0.36	0.45	0.55	0.59	0.42	0.52	0.58	−0.23	0.73	0.74	0.75	0.58	—	0.77	1.92	0.90
19	0.34	0.36	0.42	0.48	0.44	0.38	0.47	0.55	0.58	0.43	0.52	0.58	−0.25	0.63	0.64	0.65	0.53	0.76	—	2.00	1.03
Mean	3.56	3.09	3.06	1.97	3.07	2.00	3.16	2.79	2.65	3.44	2.82	2.25	3.50	2.04	2.13	2.13	2.65	1.95	2.01
SD	0.83	0.89	0.95	0.99	1.00	0.95	0.97	1.09	1.04	0.91	1.06	1.00	0.90	0.86	0.89	0.91	1.10	0.92	1.04

Note: Descriptive statistics for exploratory factor analysis (EFA) sample are presented at the bottom of the table; interitem correlations are on the lower half of the diagonal. Descriptive statistics for confirmatory factor analysis (CFA) samples are presented on the right side of the table; interitem correlations are on the upper half of the diagonal. EFA sample, n = 3612; CFA sample, n = 3612.

TABLE 3.

Unrotated eigenvalues and Horn's parallel analysis results

		Parallel analysis results
No. of factors	Eigenvalue	50th percentile	95th percentile
1	10.33	1.13	1.15
2	2.08	1.10	1.12
3	0.89	1.09	1.10
4	0.69	1.07	1.08
5	0.65	1.06	1.07
6	0.52	1.05	1.06
7	0.50	1.03	1.04
8	0.47	1.02	1.03
9	0.40	1.01	1.02
10	0.35	1.00	1.01
11	0.33	0.99	0.99
12	0.29	0.98	0.98
13	0.26	0.96	0.97
14	0.23	0.95	0.96
15	0.23	0.94	0.95
16	0.22	0.93	0.94
17	0.20	0.92	0.93
18	0.19	0.90	0.91
19	0.16	0.88	0.89

FIGURE 1.

Eigenvalue screen plot

Two factors were extracted and rotated to a final solution using direct oblimin rotation. Because the factors were assumed to be moderately related, the delta parameter was fixed equal to 0. Items with pattern coefficients ≥0.40 were considered salient and retained. There were 6 CBI items that had pattern coefficients that were ≥0.40 (Table 4).

TABLE 4.

Pattern (structure) coefficients for the 6‐item, 2‐factor exploratory factor analysis solution

Item	Factor 1 internal factor	Factor 2 external factor
1	0.88 (0.84)	−0.08 (0.36)
2	0.86 (0.84)	−0.05 (0.38)
8	0.61 (0.75)	0.27 (0.38)
10	0.68 (0.75)	0.14 (0.48)
14	0.01 (0.46)	0.90 (0.91)
15	−0.00 (0.46)	0.91 (0.91)

Note: Bolded coefficients reflect salient loading. n = 3613.

The resulting model included 2 factors composed of 6 items. The factors were named to best characterize the items that contributed to the factors. The first factor (the “internal factor”) was related to feelings of burnout that were personal and work‐related; the second factor (the “external factor”) was related to working with patients. Specifically, CBI items 1, 2, 8, and 10 loaded onto the internal factor and items 14 and 15 loaded onto the external factor.

The pattern coefficients (Table 4) reflect the partial correlation between an item and the factor, controlling for all other factors, whereas a structure coefficient reflects the item's zero‐order correlation with the factor. For example, the pattern coefficient for item 1 and the internal factor is 0.88 and the structure coefficient is 0.84. Therefore, the correlation between item 1 and the internal factor is 0.88, controlling for all other factors, whereas the zero‐order correlation between item 1 and the internal factor is 0.84. As hypothesized, the 2 resultant factors from the EFA were moderately correlated with one another (r = 0.50). Before rotation, the internal factor accounted for 60.9% of the common variance and the external factor accounted for 19.0% of the common variance, combining to account for 79.8% of the total variation that was obtained by all 19 items.

After examining the EFA results, a CFA model was estimated using the second cohort sample. The results of the 6‐item, 2‐factor CFA model revealed the 2‐factor model fit the data: χ² _SB (8 df) = 557.77 (P < 0.01), SRMR = 0.047, RMSEA = 0.138, and CFI_SB = 0.95. With the exception of RMSEA, the fit indices displayed good fit, indicating that the 2‐factor model reproduced the observed relationships well. For CFA, the unstandardized pattern coefficients are interpreted as unstandardized regression coefficients, whereas the standardized pattern coefficients are interpreted as standardized regression coefficients (Table 5). Accordingly, the standardized pattern coefficients can be squared to yield an R ² value, which indicates the proportion of an item's variance explained by the factor. Both factors accounted for at least 60% of the variance relative to the CBI items in their factor group. For example, the internal factor accounted for 65.1% of the variance in item 1, whereas the external factor accounted for 84.7% of the variance in item 14. Overall, the internal factor accounted for 63.9% of the total variance within its items, and the external factor accounted for 83.0% of the total variance within its items. The 2 factors from the CFA were positively correlated with each other (r = 0.56). The magnitude of the correlation indicates that although the factors are related, they maintain a degree of distinction. The reliability of the 2 factors using coefficient omega for the unstandardized parameter estimates was 0.88 for the internal factor and 0.91 for the external factor.

TABLE 5.

Unstandardized (standardized) parameter estimates for 6‐item, 2‐factor confirmatory factor analysis solution

Factor	Inventory items	Pattern coefficients	Error variance	R 2
Factor 1 (internal factor)	1	0.66 (0.81)	0.23 (0.35)	0.65
	2	0.72 (0.81)	0.27 (0.34)	0.66
	8	0.85 (0.79)	0.43 (0.38)	0.62
	10	0.74 (0.80)	0.31 (0.37)	0.63
Factor 2 (external factor)	14	0.78 (0.92)	0.11 (0.15)	0.85
	15	0.80 (0.90)	0.14 (0.19)	0.82

Note: All unstandardized parameter estimates were statistically significant (P < 0.01). n = 3612.

To determine quantitative thresholds for burnout, the 2‐factor model was compared to a large study of emergency medicine residents using the Maslach Burnout Inventory. ¹¹ Given the 2‐factor model, each factor had to be regarded as an independent indicator of burnout. Therefore, each factor required an individual threshold. For the internal factor, there were 4 CBI items using a rating range of 1 (never/almost never, to a very low degree) to 5 (always, to a very high degree), which proved a possible score range of 4–20. The mean rating was 12.8 (SD 3.2). At a threshold of 16 or higher, the incidence of internally caused burnout was 19.7% (Table 6). For the external factor, there were 2 CBI items using the same Likert scoring with a possible range of 2–10. The mean rating was 4.2 (SD 1.7). At a threshold of 6 or higher, the incidence of externally caused burnout was 24.4%. Given that some residents had both types of burnout (10.1% had both), the overall incidence of a resident having 1 or both types of burnout was 34.1%, which is very similar to prior reports using validated burnout inventories for emergency medicine residents. ¹¹ In addition, the incidence of burnout increased as training progressed. Emergency medicine first‐year residents had a 29.3% incidence that increased to 37.5% for emergency medicine third‐ and fourth‐year residents (Table 6)

TABLE 6.

Burnout incidence measured by abbreviated Copenhagen Burnout Inventory

Emergency medicine level	No burnout, n	Internal burnout only, n	External burnout only, n	Both burnout types, n	% Any burnout
1 (n = 1216)	860	134	134	88	29.3
2 (n = 1128)	730	104	178	116	35.3
3/4 (n = 1205)	792	112	205	159	37.5
Total (n = 3612)	2382	350	517	363	34.1

Given the identification of 2 unique factors (internal and external), the CBI should be used in such a way as to identify the type of burnout a resident might have (eg, internal, external, both). By identifying burnout type in this way, greater investigation into cause and treatment can occur.

For comparison, the personal, work, and patient burnout ratings based on the 19‐item CBI were calculated for comparison. The incidence of personal, work, and patient burnout was 19.5%, 7.3%, and 18.1%, respectively. The overall incidence of a resident having at least 1 of the 3 original types of burnout was 30.2%.

4. LIMITATIONS

As with most surveys, the results were self‐reported and surveys such as this can be prone to social desirability bias. However, such bias is unlikely to affect the interitem correlations that created the 6‐item model. Still, this bias could have contributed to a lower rate of burnout.

The survey was administered when many emergency medicine residents were likely under considerable stress caring for patients during the COVID‐19 pandemic. This situational stress could have increased burnout more than in other times pre‐pandemic. However, the purpose of this study was not to determine the prevalence of burnout but rather to determine the interitem correlations and potential item redundancy that could be used to create an abbreviated inventory. Nonetheless, the findings are consistent with other burnout findings conducted during the COVID‐19 pandemic (Lu DW, Zhan T, Bilimoria KY, et al unpublished data, 2021).

The survey was administered after the ITE. A resident's self‐perception of performance could affect their survey responses. Specifically, if a resident found the ITE to be difficult, they could have greater feelings of burnout. In prior ABEM ITE surveys, no negative bias was detected. Moreover, the Dunning‐Kruger effect in other fields suggests that test‐takers tend to overrate their relative test performances.

The model is not statistically perfect. For the CFA, the RMSEA did not demonstrate excellent fit, despite the other statistical analyses demonstrating good fit. The RMSEA was 0.14, when ideally it should have been <0.08. The RMSEA is a measure of absolute fit and is sensitive to misspecified factor loadings. The likely source of this result is that item 8, which loaded on the internal factor, also partially loaded onto the external factor, albeit somewhat weakly. The other items (1, 2, 10, 14, and 15) tended to have large pattern and structure coefficients for a single factor, which were accompanied by small pattern and structure coefficients for the other factor. Nonetheless, given the strength of the other tests for fit, the RMSEA did not, by itself, negate the 2‐factor model.

The abbreviated CBI might slightly overestimate the incidence of burnout. The 19‐item CBI found an overall incidence of burnout to be 30.2%, whereas the abbreviated CBI calculated an incidence of 34.1%. Of note, the abbreviated CBI more closely approximately prior estimates and estimates using the Maslach Burnout Inventory.

This model did not attempt to independently establish the incidence of burnout de novo for the survey respondents. Rather, the 2‐factor model was compared with prior studies that used a similar survey methodology and similar cohort (Lu DW, Zhan T, Bilimoria KY, et al unpublished data, 2021). ¹¹ The proposed scoring rubric requires prospective application to determine reliability and provide additional validity evidence.

The abbreviated CBI should not be assumed to generalize to emergency physicians who have been in practice for a substantial length of time. Likewise, the abbreviated CBI should not be assumed to generalize to other specialties. Although there is substantial validity evidence for using the CBI in other medical specialties, using the abbreviated CBI requires further validity investigation. The abbreviated model requires further prospective factor analysis using the abbreviated CBI. This analysis of the ABEM 2022 ITE post‐examination survey is planned.

5. DISCUSSION

This study is the first to use factor analysis to assess the psychometric properties of the CBI in emergency medicine residents. In addition, this is the first study designed to create an abbreviated CBI. The results of this study are important in that they demonstrate substantial reliability and validity evidence to support the ongoing use of an abbreviated form of the CBI, as well as use of the CBI for emergency physicians. This study successfully identified a 2‐factor, 6‐item inventory that can assess burnout risk in emergency medicine residents (Table 7). Two findings provide substantial validity evidence for the derived model. First, the overall frequency of 34% is similar to studies using items from the Maslach Burnout Inventory involving residents taking the ABEM ITE (Lu DW, Zhan T, Bilimoria KY, et al unpublished data, 2021). ¹¹ Second, the prevalence of burnout increased as residents progressed through training, consistent with prior studies (Lu DW, Zhan T, Bilimoria KY, et al unpublished data, 2021). ¹¹

TABLE 7.

Abbreviated Copenhagen Burnout Inventory (CBI) items

1.	How often do you feel tired? a
2.	How often are you physically exhausted? a
8.	Do you feel burned out because of your work? b
10.	Do you feel worn out at the end of the working day? a
14.	Do you find it hard to work with patients? b
15.	Do you find it frustrating to work with patients? b

Note: Internal factor is determined from items 1, 2, 8, and 10. External factor is determined from items 14 and 15.

^a 5‐point rating scale: never/almost never, seldom, sometimes, often, always.

^b 5‐point rating scale: to a very low degree, to a low degree, somewhat, to a high degree, to a very high degree.

Although other investigators have applied EFA and CFA to define the psychometric properties of the CBI, those efforts were not directed toward identifying the essential items within the inventory that could be used for an abbreviated format. Prior work tended to use factor analysis to provide validity evidence for the CBI construct of the following 3 assessment categories of burnout: personal, work, and patient. For example, Todorovic et al used a Serbian version of the CBI to determine whether it could confidently assess burnout among Serbian medical students. ⁴⁴ The study applied EFA to the CBI and confirmed the presence of the 3 aforementioned subcategories that demonstrated a high degree of correlation.

Javanshir et al also evaluated the psychometric properties of the CBI to gather validity support for the use of an Iranian version of the instrument in assessing a diverse group of workers, including health care staff. ⁴⁵ The results from EFA and CFA similarly provided support for construct validity for the 3‐factor CBI construct. Internal reliability and test‐retest reliability were also high.

A study of pharmacists used a slightly shorter inventory by eliminating 2 items, but that modification was not based on factor analysis. ⁴⁶ This shortening was a pre hoc decision based on item validity. CFA of the amended CBI still provided reliability and validity evidence for the CBI. Although CFA supported the use of this 17‐item inventory, no further reduction in inventory items was attempted.

Not all studies supported a 3‐factor model for the CBI. In a study of Iranian nurses, EFA identified 4 factors that were supported by subsequent CFA. ⁴⁷ Of note, our study found 2 factors (internal and external). A Brazilian‐Portuguese version of the CBI used to assess Brazilian health care workers was also found to have a 2‐factor pattern ⁴⁸ that was similar to our study. Specifically, that study found that 1 factor was a combination of items from personal burnout and work‐related burnout; the other factor included items from patient burnout. However, that study did not aim to shorten the inventory; rather, its primary purpose was to validate the Brazilian‐Portuguese version of the CBI.

Physician burnout is a major area of emphasis within emergency medicine. The Quadruple Aim approach is not only patient‐centric but addresses improved physician experience as well. The notion is that without improving the clinical experience of the physician, it will be more difficult to improve the patient care experience, improve the health of a population, and reduce per capita health care costs. One key to improving the physician experience is reducing burnout. A seminal step to reducing burnout is to measure it reliably and accurately. Although there are available burnout inventories, many have limitations. For example, the Maslach Burnout Inventory and the Mayo Well‐Being Index have substantial costs for use. The Stanford Professional Fulfillment Index is relatively new and has 16 items.

The advantages of the CBI are its widespread use geographically and among varied health care professionals. Given the complexity of burnout, identifying the type of burnout (internal vs external) likely has an advantage for creating solutions. Another practical advantage is that the CBI is free. This study provides validity and reliability evidence for the use of the abbreviated format that provides greater ease of use. Finally, the ease of use and the open‐access of the abbreviated CBI make the inventory a viable instrument for program directors to monitor burnout among residents.

Our study provides both construct validity evidence and reliability support for the use of a 6‐item, 2‐factor abbreviated CBI. A shorter, reliable, valid, and publicly accessible burnout inventory provides numerous advantages for burnout research in emergency medicine. An additional prospective study using CFA is underway to provide additional validity evidence for the abbreviated CBI.

CONFLICTS OF INTEREST

Melissa A. Barton, Mary M. Johnston, and Earl J. Reisdorff are employed by the American Board of Emergency Medicine. Lewis S. Nelson is a former member of the Board of Directors of the American Board of Emergency Medicine.

AUTHOR CONTRIBUTIONS

Melissa A. Barton: Manuscript review. Michelle D. Lall: Manuscript review, institutional review board review. Mary M. Johnston: Study design, data analysis, supervised the conduct of the trial and data collection, managed the data, including quality control, provided statistical advice on study design, and analyzed the data. Dave W. Lu: Manuscript review. Lewis S. Nelson: Manuscript review. Karl Y. Bilimoria: Manuscript review. Earl J. Reisdorff: Conceived of the study, manuscript draft, manuscript review, institutional review board review, and takes responsibility for the paper as a whole.

Barton MA, Lall MD, Johnston MM, et al. Reliability and validity support for an abbreviated Copenhagen burnout inventory using exploratory and confirmatory factor analysis. JACEP Open. 2022;3:e12797. 10.1002/emp2.12797