Does the Maslach Burnout Inventory correlate with cognitive performance in anesthesia practitioners? A pilot study

Abstract

Background:

Chronic stress is a common condition among health-care operators, anesthetists in particular. It is known to cause cognitive weakening and pathological outcomes, as the Burnout syndrome. Nevertheless, the impact of clinicians’ health on their performance has received limited attention thus far. Our pilot study, aims at evaluating the influence of burnout on the cognitive performance in a population of anesthesia practitioners.

Methods:

In 18 practitioners we assessed attention by means of reaction times (RTs), pre- and post-shift, with a five-subtest computerized neuropsychological battery. RTs were controlled for the situational anxiety with the State-Trait Anxiety Inventory X1. The burnout level was evaluated with the Maslach Burnout Inventory (MBI). The three MBI sub-scores (emotional exhaustion, depersonalization and professional achievement) were combined to obtain two groups according to the burnout score (high and low).

Results:

Anesthetists showed a significantly worse performance in the fifth test post-shift (P=0.041) than pre-shift. The high-score burnout group reacted slower than the low-score burnout group in three of the five cognitive subtests, without reaching a statistical significance. Nevertheless, our effect size, which is independent from the sample size, is very large (d=1.165).

Conclusion:

We found that in a population of health-care operators, burnout may affect the cognitive and potentially, the working performance. Qualitative and quantitative measurements should be integrated to ensure a better management of burnout and its consequences in workplaces.

INTRODUCTION

“While anesthesia, like all of medicine, is considered to be increasingly safe for patients, it is becoming more dangerous for its practitioners.”

Jackson, 1999

In the last two decades there has been an increased attention for the working conditions of health-care operators, as they experience a major discomfort and a higher risk to develop work-stress related diseases.[1] Anesthesia practitioners in particular suffer from chronic stress, due to the peculiar work context (low control, high demand, risk and responsibilities, production pressure, conflicts) and content (proximity with situations of pain and death).[2,3]

This kind of distress can often lead to pathological outcomes such as the Burnout syndrome.[4,5] Burnout is a prolonged response to chronic emotional and interpersonal stressors on the job, and is defined by the three dimensions of exhaustion, cynicism and inefficacy.[6]

These conditions of chronic stress have also remarkable consequences on the functioning of one's cognitive systems, in particular memory and attention,[7,8,9,10,11,12] in terms of mental overload. These so-called non-technical skills, like the capability of handling several simultaneous stimuli, are absolutely essential for clinicians to properly carry out their work and manage every-day emergency situations[7,8] and their integrity is an elementary pre-condition for the overall performance of the cognitive system.[7]

Hence, burned-out workers could also present a weakened cognitive performance, which does not define the professional skill, but, if detected, can help in preventing errors. That is why recently, Byrne et al. have underlined that anesthetists’ performance is a prominent issue for a safe clinical practice[13] and cognitive overload caused by distress has been recognized as a significant cause of error in industries such as aviation. Measuring mental workload is becoming a key method of improving safety, also in hospital contexts.[13]

However, the cognitive functioning of distressed workers has received limited attention thus far.[14] The impact of clinicians’ health on their performance is now-a-days largely unknown and is often treated as a taboo.

Hence, to evaluate the influence of burnout level on cognitive performance, in the present preliminary study we applied an original protocol to assess the effects of burnout on the attentional cognitive system in a specific population of anesthesia practitioners. We therefore, collected not only qualitative data, but also objective measures of performance by means of reaction times (RTs). Our aim is to evaluate if and how burnout could affect the subject's ability of information speed processing and attention. We expect that the presence of burnout will lead to a poorer performance, indicated by slower RTs.

METHODS

We followed the Strengthening the Reporting of Observational Studies in Epidemiology STROBE Statement Checklist for observational cohort study.

Subjects

Our sample consisted of 23 specialised operators, 15 anesthetists and 8 anesthesia nurses, the whole Anesthesia Service at our Neurological Institute in Milan, a tertiary hospital devoted to neurological and neurosurgical procedures. As participation was voluntary, five practitioners decided not to take part in the study, three anesthetists and two nurses.

The experimental group comprised 18 subjects, 11 men (9 anesthetists and 2 nurses) and 7 women (3 anesthetists and 4 nurses) and with a mean age of 42 (±10.9). All participants were staff operators at the Neurosurgical Department and O. U. of Neuroanesthesia and Intensive Care, with a mean experience in the field of 17 years (±10) (9 years ± 8 years at the Neurological Institute C. Besta) and a mean of 44 ± 6 working hours per week. All subjects signed an informed consent to the anonymous treatment of the data.

Ethical approval for this study was provided from the Institutional Review Board (Record n. 17), Milan, Italy (Chairperson Prof. Muller) on 6^th July 2011.

Measures

Psychological testing

Every subject completed two tests: The State-Trait Anxiety Inventory X1 version (STAI-X1, Spielberger, 1979) and the Maslach Burnout Inventory (MBI, Maslach and Jackson, 1981, 1986), in the version specifically designed for health-care workers. The STAI-X1 assesses the state (situational) anxiety with 20 items to which answer on a 4-point scale (not at all– a little– quite– a lot). This allowed us to detect high levels of state anxiety that could affect the rest of the assessment.[15] Note that there is not a cut-off value to interpret STAI-X1 scores. Since, in our case, the STAI-X1 had not a diagnostic value, we considered scores >1 standard deviation as a deviation from the normality (statistically meant) and scores >3 standard deviations as outliers.

The MBI is a 22-item questionnaire that evaluates the burnout level on three scales, emotional exhaustion (EE), depersonalization (DP) and professional achievement (PA), as theorised by Maslach (1982). For the MBI cut-off values are available, specifically validated for health professions, and allow to categorize scores in low, medium and high. Specifically, they are: For EE, low ≤14, medium 15-23, high ≥24; for DP, low ≤3, medium 4-8, high ≥9; for PA, low ≥37, medium 36-30, high ≤29 (this scale is inverted; the higher the PA, the lower the burnout).

Every subject also completed a brief questionnaire collecting demographic data (age, sex, marital status) and professional information (profession, seniority, weekly working hours).

Cognitive assessment

A computerised neuropsychological battery with five subtests was used to perform selective and sustained attention and RTs in ms were acquired. RT is defined as the interval time between application of a stimulus and detection of response. It is used to assess attention and is considered a measure of cognitive resources.[11]

Our battery was a progressive order and included a difficulty scale. Stimuli in the tests were automatically randomized at every start of the program, in order to avoid learning effects on the performance. In the first cognitive test the subject only had to push the bar button on a keyboard when a yellow dot compared randomly on the display. The second, the third and the fourth were tests of selective and sustained attention: The subject had to push the button only when the target stimulus compared on the display, with, respectively, a color, color and shape and orientation criterion. The last was a spatial selective attention test, in which the participant had to click on every H letter he/she could see among other non-target letters in the shortest time possible. The battery was built by digitizing already existing and standardized tests, in detail: The first, second and third test from Posner (1984), the fourth test from Cooper and Shepard (1973) and the fifth test by Mesulam (1985).

Methodology

All participants were assessed one time before and one time after the routinary working day shift, in block randomization order. The method of block randomization allows to divide in blocks the participants according to a factor, that is than controlled for. In our case, we block randomized the order in which participants should complete the pre- or post-shift assessment, in order to avoid bias due to the order or the sequency of the assessment.

To avoid differences due to fatigue, all participants were tested before and after an ordinary morning shift in the operatory room (8 working hours per shift for all participants). The uniformity of the context, being stress a context-related phenomenon,[3] allowed us to control for this variable.

During every session, the subject filled the STAI-X1 and performed the cognitive tests. The MBI was given on the first session and had to be filled only once.

The experiment took place in a protected-from-external- stimuli environment at the Neurological Institute C. Besta of Milan.

Outcomes and hypothesis

Our primary outcome was to investigate the phenomenon of burnout in our sample population. In particular, we wanted to verify whether high levels of chronic stress (resulting in high burnout scores) could affect the performance of simple reaction time tests. Therefore, our primary outcome measures were MBI scores and RTs.

We speculated that RTs could vary for different level-of-burnout group (the higher the burnout level, the slower RTs) and also between the pre- and post-shift session, because of the fatigue due to mental workload developed during the working shift.

Data handling and statistics

We scored the two psychological tests taking into account demographic variables, i.e., age, sex, and education, using the correction tables provided by each manual. The three sub-scores obtained from the MBI were recoded into low-medium-high. In addition, the MBI does not return a unique score and our subjects all showed heterogeneous profiles. Therefore, we applied a recombination of the three MBI scores that allowed us to divide our sample into two groups, according to the burnout level: The low-score group, i.e., all subjects with none/only one subscale high-score (N=15), and the high-score group, i.e., all participants with at least two subscale high-scores (N=3) [Figure 1]. We applied this recoding based on previous considerations regarding the combination of the three MBI subscales.[16]

Figure 1.

Sample flow chart

The mean RT was calculated for every subtest, subject, and experimental session.

Data were analyzed using the Statistical Package for Social Science version 17.0 (SPSS Inc., Chicago, IL). All testing was two-tailed with the significance level set at P≤0.05 and P≤0.01 for multiple comparisons, according to the Bonferroni correction. We used non-parametric statistics: The Mann-Whitney's test for independent data, for between-group analysis, the Wilcoxon's signed-rank test for coupled data, for within-group analysis, and the Kendall's Tau for correlation analysis.

RESULTS

Our sample reveals high levels of burnout [Table 1]. The Kendall's Tau test did not show any correlation between the MBI subscales and demographic and job variables (age, sex, marital status, profession, seniority).

Table 1.

Sample burnout description for low, medium and high Maslach Burnout Inventory sub-scores

The STAI-X1 was used to control our reaction time measures for the situational anxiety. In our sample, there were not outliers (>3 standard deviations), but we had three high scores at the anxiety test (>1 standard deviation), the same in the pre- and the post-shift session. Despite this, we could include these subjects in the following analysis, due to the fact that STAI-X1 scores did not show any significant correlation with RTs and were stable over the two measurements (P=0.006).

RTs of the fifth cognitive test were significantly slower in the post-shift session for the overall sample (P=0.035), but it was the group of anesthetists that showed this effect prominently (P=0.041) [Figure 2].

Figure 2.

Anaesthetists’ reaction times (ms), expressed as mean and standard deviation, in the five neuropsychological tests (1, 2, 3, 4, 5), pre (♦) and post (■)

RTs of the two level-of-burnout groups did not show any statistical significant difference, although the graph is quite eloquent [Figure 3]. In order to understand the scope of our results, we then calculated the effect size (ES) of the differences in pre-shift RTs between low-score and high-score burnout group. Note that ES is independent from the sample size (unlike the P value) and gives a degree of departure from the null hypothesis. We found a mean ES d=1.17 (respectively, for RTs of the five pre-shift tests: d=1.02; d=1.11; d=1.36; d=1.05; d=1.29), which, according to Choen,[17] is a very large ES (d>0.8).

Figure 3.

Reaction times (ms), expressed as mean and standard deviation, in the five neuropsychological tests performed pre-shift by the high-score group (■) and the low-score group (♦) according to the Maslach Bur

DISCUSSION

Work-related stress is a growing phenomenon in our society and there are specific working populations, like health-care operators, that are more affected.[18,19,20]

In the literature, there are conflicting data concerning the correlation between burnout and other variables, such as demographic and occupational factors and methods are quite heterogeneous.[21] Despite this, it is well known that among anesthetists, and health-care operators in general, there is a high prevalence of burnout, between 20% and 60%, considering the large variance between cultures. In our sample, we detected levels of burnout higher than normal population's and in line with previous results.[6,21] On the other hand, also our results do not confirm any correlation between burnout level and demographic features, affirming the variability of these factors between different contexts.

Most of the studies on stress and burnout among clinicians, however, had used mainly questionnaires or postal surveys as evaluating tools.[3,6,18,19,20] Even so, a questionnaire is not meant to provide measures of cognitive overload, but a subjective evaluation of one's stress condition. Moreover, they make adequate comparisons difficult, limit the development of causal inferences and are exposed to bias.[22] Instead, and easy-to-acquire measure such as RTs can provide a quantitative index of working performance.[13,23,24] That is why a current of studies have focused on investigating new methods to measure the mental load during clinical practice.[13,23,24] These studies demonstrate that increased RTs in a double task correspond to higher mental workload, because of an increased demand of attentive resources.

The originality of our protocol stands in the combination of a validated and widely accepted questionnaire (the MBI) and a quantitative measure (the RT) controlled for the state anxiety, to provide an assessment of burnout and its cognitive consequences on a working population of health-care operators.

Our first result indicates a weakening of the performance in the post-shift assessment that becomes evident in the last attentive test. Anesthetists in particular show this effect. It arises only at the end of the battery probably because of the difficulty order of the tests and means that the first four tests reach a “ceiling effect”; instead the fifth test overcomes the threshold of the residual cognitive resources and ends in a slowdown of RTs. A possible explanation is that anesthetists experience a major stress and cognitive load during the working shift, which ends in a mental fatigue and in a worse performance.

Furthermore, congruent to our hypothesis, we found differences, though not statistically significant, in RTs between a high-score and a low-score burnout profile group, in that the first group shows slower RTs throughout all tests. It is remarkable that these differences are detected in pre-shift session, therefore, excluding the possibility to ascribe them to ordinary mental fatigue and pointing out a baseline mental overload due to chronic stress. RTs, as an index of cognitive overload, allowed us to detect a weakening of the attentive and potentially, working performance.

Being our research novel and exploratory in nature, it presents the major limitation of the small sample size and therefore, has a limited external validity. However, this limit does not undermine the importance of our results, as previous authors’ experience demonstrate.[24,25,26] This is even more true if we consider the large ES of our results. Taken together with the small sample size and the lack of statistical significance, it indicates a high probability of making an error type II (>20%). Our sample is too small to detect a statistical significance, but the ES of our results clearly demonstrates a trend towards a difference between subjects with and without burnout.

Moreover, this kind of study, being the participation voluntary, are susceptible of a self-selection bias.[22] In our case, this means that people who believed to suffer from burnout may had decided not to take part in the study (for example for fear of being judged), therefore, weakening the power of our results.

Thus, we suggest that further investigations based on our data are needed on a larger scale, concentrating on the cognitive outcomes of burnout and how to detect them. Should our results be confirmed, they may open the way to further analysis and suggestions for safety of both operators and patients.

CONCLUSIONS

Based on our evidences and on the literature, a questionnaire alone could not be enough for an all-round assessment of work-related stress. Qualitative and quantitative tools should be integrated in a protocol like the one we propose, which is more accurate and reliable for an effective management of stress in working contexts, since it provides both subjective and objective measures of stress and performance.

Currently the topic of burnout and mental overload is intensely under researched and results from this line of investigation are particularly relevant for the issue of a safe practice and for operators’ health.

Our results are only explorative, but give a preliminary and significant answer to the question we pose in the title, and can be a useful guide for future investigations and to implement an agile protocol to monitor operators’ health.