A Smartphone App to Reduce Burnout in the Emergency Department: A Pilot Randomized Controlled Trial

Abstract

Background:

Burnout is a significant concern among health care professionals, particularly those working in the emergency department (ED). Given the negative personal and professional consequences that burnout can have on all health care professionals, multidisciplinary solutions are needed to address burnout. Our objective was to evaluate the feasibility and potential impact of resilience training delivered through a smartphone application on burnout among health care professionals working at a tertiary-care pediatric ED.

Methods:

We conducted a single-center pilot randomized controlled study enrolling multidisciplinary health care professionals working in our ED. Participants assigned to the intervention group received self-driven access to a smartphone application that provided a structured resilience curriculum for a period of 3 months. The participants completed psychometric assessments both prior to and following the invention period. Changes in psychometric measures of the intervention group were then compared with a waitlist-control group.

Results:

Following the intervention period, a total of 20 participants were included in the final analysis. The change in participant scores on psychometric measures prior to and following the intervention period was calculated. A statistically significant mean decrease in burnout measure (emotional exhaustion subscale of Maslach-Burnout Inventory mean score −5.88, p < .001) and increase in mindfulness measure (Mindful Attention Awareness Scale mean score 0.51, p < .001) was observed among the intervention group participants.

Conclusions/Application to Practice:

Our study suggests that a resilience training program delivered using a smartphone application can be an effective intervention in reducing burnout and increasing mindfulness skills. Our study also demonstrated the potential feasibility of a randomized controlled study of burnout within a multidisciplinary group of health care professionals.

Burnout was first described by Herbert Freudenberger in 1974 as a state of mental and physical exhaustion caused by one’s professional life. Following this initial description, Maslach et al. (1997) formalized the definition of burnout as a syndrome involving emotional exhaustion, feelings of depersonalization, and a reduced sense of professional accomplishment. Burnout is particularly prevalent among health care providers, including physicians, nurses, and social workers (Brown et al., 2019; Shanafelt et al., 2015; Woo et al., 2020). Health care providers working in high-intensity clinical settings, including the emergency department (ED), experience higher rates of burnout than their colleagues in other clinical settings (Gómez-Urquiza et al., 2017; Zhang et al., 2020). High levels of burnout have been associated with a number of personal and professional consequences, including an increased rate of medical errors, higher inpatient mortality, risk of suicidal ideation, substance use, and a number of chronic health conditions (West et al., 2018). The economic impact of burnout on the health care system can be significant, with an estimated $4.6 billion annual cost of physician burnout attributed to physician turnover and a reduction in work hours by the physician workforce in the United States (Han et al., 2019).

Strategies to reduce burnout have been identified in the literature, including resilience training, mindfulness training, stress management techniques training, and exercise programs (West et al., 2018). Training in resilience has been found to enhance interpersonal relations and self-esteem, as well as reduce stress and anxiety in physicians and nurses (Magtibay et al., 2017; Sood et al., 2011; Waite & Richardson, 2004). However, an individual lack of resilience does not appear to be the sole cause of burnout among health care workers. West et al. (2020) found that while resilience and burnout measures were inversely correlated, burnout levels were still high among physicians with high levels of resilience. Indeed, Cleary et al. (2018) demonstrated that while resilience interventions can benefit health care professionals, enhanced resilience does not necessarily lead to decreased levels of burnout. The heterogeneous nature of research on resilience interventions is highlighted by the findings of a Cochrane review completed by Kunzler et al. (2020), which identified the positive effects of resilience training programs in health care students but acknowledged the very-low certainty of evidence.

Organizational interventions have also shown promise in decreasing burnout among health care workers (De Simone et al., 2021; Pieper et al., 2019; Thomas Craig et al., 2021). As burnout syndrome is a multifaceted condition, interventions targeting burnout may benefit from both individual and system-oriented solutions. Bruschwein and Gettle (2020) demonstrated that multifaceted approaches to burnout interventions, incorporating both individual and organizational solutions, have the potential to reduce burnout among health care workers (Bruschwein & Gettle, 2020). In particular, digital mental health interventions have shown promise in improving the mental health of participants within the organizational context (Pijpker et al., 2019; Stratton et al., 2021). However, there has been little evidence on our review of the literature regarding the use and impact of multifaceted and digitally delivered resilience training on burnout among a multidisciplinary group of health care professionals.

The goal of this study is to assess whether the implementation of resilience training delivered through a mobile smartphone application will affect burnout in health care professionals working in a tertiary pediatric ED in Canada. In addition, this pilot study serves as an assessment of the feasibility of performing a randomized controlled trial on burnout interventions. We predict that resilience training delivered through a smartphone application will be both an effective and feasible intervention to combat burnout among a multidisciplinary population of health care professionals.

Materials and Methods

Study Design

The study was designed as a randomized, waitlist-controlled pilot study enrolling health care professionals working at a tertiary pediatric ED. The participants were recruited through print and electronic (email) advertisements. The study included health care professionals (nurses, physicians, respiratory therapists, and social workers) who worked in the tertiary pediatric ED (≥4 clinical shifts/month in the ED). In addition, staff must have had the ability to operate the mobile application on their device. Participants who had previous exposure to the intervention application (headversity™) were excluded. A stratified randomization procedure was employed following enrollment, resulting in participants being randomized into the intervention or control group based on stated gender and occupation. A stratified randomization procedure was conducted to maintain participant anonymity among a potential small sample size, and as such, demographic data was not linked to group allocation. Enrolment and randomization were completed entirely online utilizing an electronic survey system (Qualtrics™) system, and group allocation was concealed from the investigators. No changes to the methods occurred following trial commencement. This study was approved by the University of Calgary Conjoint Health Research Ethics Board (REB19-2019).

Following enrolment and the completion of the pre-intervention questionnaire, the intervention group was provided immediate access to the smartphone application, while the control group’s access to the smartphone application was delayed by 12 weeks. Participants were able to access the application at their own leisure and to maintain anonymity, participant usage data was not tracked or recorded. All participants completed the same psychometric questionnaire following the 12-week intervention period. The linked questionnaire responses were anonymized using the random generation and assignment of an electronic study identifier to each participant, with only the participant knowing their unique identifier.

The primary outcome was a change in individual participant measures of burnout, as measured by the scores obtained using the three domains of the Maslach-Burnout Inventory (MBI). In addition, secondary outcomes included the change in the level of resilience of the participants as measured by the Connor-Davidson Resilience Scale (CDRS), as well as the change in mindfulness skills of participants, utilizing the Mindful Attention Awareness Scale (MAAS).

Psychometric Measures

Participants completed a psychometric questionnaire both prior to and following the intervention period. This psychometric questionnaire consisted of the MBI, CDRS, and the MAAS. MBI was developed in 1981 by Christina Maslach and Susan Jackson (1981) and has since been adapted specifically for professionals in health care, with a Medical Professional version (MBI-HSS). The 22-item MBI-HSS measures the participants according to the three domains of burnout: emotional exhaustion, depersonalization, and a decreased sense of professional accomplishment. Higher scores in the emotional exhaustion and depersonalization domains and a lower score in the professional accomplishment domain are indicators of burnout. The MBI-HSS has been validated in a large variety of professionals, including nurses, physicians, resident physicians, and the general population (Shanafelt et al., 2015).

The Connor-Davidson Resilience Scale (CD-RISC) was developed in 2003 to measure the level of resilience of participants, based on prior research on psychological characteristics associated with resilience (including strong self-esteem, social problem-solving skills, and ability to adapt to change, among others) (Connor & Davidson, 2003). The original CDRS includes a 25-item inventory, although a modified 10-item inventory was developed using factor analysis and has been externally validated (Campbell-Sills & Stein, 2007). Versions of the CD-RISC have been used in several previous studies to assess resilience skills in physicians and other health care providers (Sood et al., 2014).

MAAS is a 15-item measure of mindfulness where participants rate the frequency to which they engage in general or situation-specific mindfulness activities (on a scale of 1-6). Developed by Brown and Ryan (2003), the MAAS has been validated among several groups, including a general adult population sample, college students, and cancer patients.

Smartphone Application

The intervention utilized in this study was the headversity™ (headversity.com) smartphone application. This application was created to deliver a self-directed resilience curriculum entirely on the user’s mobile device using validated training tools. The curriculum includes training modules on six domains of resilience: mindfulness, self-expertise, mental fitness, mental health, hardiness, and energy management.

The curriculum is delivered through a variety of media, including text, images, videos, and audio media, with approximately 8 to 10 hours of total instructional content available on the application. Participants can track their progress and advance through the modules and exercises at their own pace and based on their own self-identified areas of interest. In addition to the training modules that provided didactic education around wellness topics, users could access self-directed activities, including meditation exercises, breathing exercises, and exercises around journaling and goal setting. Participants in this study were able to freely use the application and there was no tracking of the application usage during the study period.

Statistical Analyses

For each measurement scale used, the change between the participant’s pre-intervention psychometric scores and post-intervention scores was calculated, and the mean change in pre- and post-intervention scores was compared between the intervention and control groups using a two-sample t-test. As the MBI consists of three subscales (emotional exhaustion, depersonalization, professional accomplishment), a two-sample t-test was utilized to compare the change between two groups in each subscale. Measured variables were reported with mean and standard deviation. The Hedges g calculation was used for effect sizes in a population of a small sample.

Results

Our study enrolled 38 participants in total, with 31 females and 7 males. Participants were enrolled over a 2-week period in February 2021, with the 12-week intervention period occurring between March and June 2021. The multidisciplinary group of participants included 18 Nurses (Registered Nurses & Licensed Practical Nurses), 17 Physicians, and 3 Respiratory Therapists, as represented in Figure 1 (trainees were not included among participants). Following the exclusion of 4 participants due to trial protocol, 34 participants were randomized. To be included in the final data analysis, participants had to complete both the pre-intervention and post-intervention questionnaires. Nine participants in the invention group and five participants in the control group did not complete post-intervention questionnaire, therefore these participants were excluded from the final analysis and considered as participants who were lost to follow-up. This resulted in the final analysis including 8 participants in the intervention group and 12 participants in the control group (Table 1).

Figure 1.

PRISMA flow diagram.

Note. PRISMA = preferred reporting items for systematic reviews and meta-analyses.

Table 1.

Participant Demographics at Study Enrolment

Total participants assessed for eligibility	38
Identified gender
Female	31 (81.6%)
Occupation
Nurse (LPN, RN, NP)	18 (47.4%)
Physician	17 (44.7%)
Respiratory therapist	3 (7.9%)
Workload (shifts/month)
16 or greater shifts	4 (10.5%)
13–15 shifts	15 (39.5%)
10–12 shifts	10 (26.3%)
7–9 shifts	2 (5.3%)
4–6 shifts	6 (15.8%)
0–3 shifts	1 (2.6%)

Note. LPN = licensed practical nurse; RN= registered nurse; NP = nurse practitioner.

Following the intervention period, psychometric measures were repeated, and the change in individual participant scores was recorded. The mean change in score on the emotional exhaustion subscale of the MBI (MBI-EE) was −5.9 (SD = 11.22) in the intervention group compared with a mean change in score of +5.2 (SD = 5.7) in the control group, representing a statistically significant difference between groups (p < .001). As the primary outcome of the study was the change in scores on the MBI subscales, an effect size was calculated for the statistically significant difference. The effect size of the change in emotional exhaustion subscale score was 1.3 (Table 2).

Table 2.

Change Score Analysis

Variable	Intervention group	Control group	p value (t-test)	Effect size (Hedges’s g)
Sample size (n)	8	12
MBI—EE (M [SD])	−5.88 (11.22)	5.17 (5.73)	.01*	1.33
MBI—Dep (M [SD])	−2 (5.61)	−0.33 (3.92)	.44	0.36
MBI—PA (M [SD])	2.25 (3.54)	0.58 (2.64)	.24	0.55
CDRS (M [SD])	1.5 (3.30)	0 (3.86)	.38	0.41
MAAS (M [SD])	0.51 (0.62)	−0.2 (0.45)	.01*	1.36

Note. MBI–EE = emotional exhaustion subscale of Maslach-Burnout Inventory, MBI–Dep = depersonalization subscale, MBI–PA = professional accomplishment subscale.

^*Statistically significant results.

The mean change in participant mindfulness scores following the intervention period was 0.5 (SD = 0.6) in the intervention group, compared with a −0.2 (SD = 0.5) change discovered in the control group (p = .0081). No statistically significant changes were noted in the depersonalization and professional accomplishment subscales of the MBI, nor in the level of resilience of participants (measured by the CD-RISC).

Given the number of participants who were lost to follow-up during the trial, an analysis of the participants who had completed the pre-intervention questionnaire, but not the post-intervention questionnaire, was conducted. This compared the pre-intervention psychometric scores of the participants who were lost to follow-up to those who remained in the study. The emotional exhaustion subscale of the MBI was the sole psychometric measure that differed in the participants who were lost to follow-up, with a mean score of 18.4 (SD = 10) compared with 25.7 (SD = 10) in the participants who remained in the study (p ≤ .05). Additional results are available as supplemental material accompanying the online article (Figure S1 and Table S1).

Discussion

The intervention used in our study appeared effective at reducing a specific domain of burnout, with a statistically significant decrease in the MBI-EE score being discovered among participants who utilized the smartphone application. Although we are not able to draw conclusions from this pilot study, the results do suggest the potential effectiveness of delivering resilience training using a smartphone application. Resiliency training programs for healthcare professionals that have been described in the literature can vary significantly in length, learning objectives, and target populations. Waite and Richardson (2004) described an effective 35-hour workplace resiliency program that incorporated personal resilience concepts and emphasized building resilient relationships. Similarly, the U.S. Army Medical Department developed an effective multi-week resiliency training program specifically targeting medical professionals working in the military (Adams et al., 2010). The smartphone application used in our study allowed for users to engage independently with the content on their own schedule and interact with the programming over a 12-week period. It is possible that this may have enabled a greater opportunity for participants to interact and retain the curriculum content presented. Although mobile smartphone applications have also been explored as a platform to deliver various types of mental health training to non-healthcare audiences, our study suggests that this platform may be an effective delivery method for healthcare professionals (Champion et al., 2018). Further research into alternative and novel ways of delivering resiliency training should be explored in a larger randomized controlled trial.

With respect to secondary outcomes, our study discovered that there was a statistically significant increase in participant mindfulness scores following use of the smartphone application, as measured by the MAAS. Although our study was not designed to suggest causation between mindfulness skills and burnout, our findings are consistent with a systematic review by Luken and Sammons (2016) that suggested that mindfulness-based interventions can reduce burnout among health care professionals.

Our study findings are also consistent with prior research findings, which observed that health care professionals working in ED settings have elevated measures of burnout. Although there is no standard MBI score that indicates burnout, Maslach et al (1997) published mean reference values for the MBI based on a large sample size of health care professionals. Similar to the findings of a study on burnout among emergency physicians published by Lim et al. (2020), the mean emotional exhaustion and depersonalization scores in both our pre-intervention intervention and control group were higher than the MBI mean reference value of 22.19 (SD = 9.53) and 7.12 (SD = 5.22), respectively.

Our study also highlights the potential benefit and feasibility of targeting interventions to address burnout to a multidisciplinary audience. As organizational factors can influence burnout, it is reasonable to surmise that interventions targeting a larger group of professionals working within the same organization may lead to improved outcomes. Indeed, Colgan et al. (2019) demonstrated the high acceptability and feasibility of a multidisciplinary mindfulness intervention in a primary care practice. Further research should incorporate larger samples of multidisciplinary health care professionals and evaluate if there is a differential impact of multidisciplinary interventions over individual interventions.

It is interesting to note the lack of statistically significant change in participant resilience scores after receiving resilience training, as measured by the CD-RISC. Although our population had higher baseline levels of resilience prior to the intervention, it appears that the intervention affected variables other than resilience that were not measured in our study. Future studies could consider using a broader range of psychometric scales to ascertain the effects of resilience training more fully.

Our single-center study had several limitations that should be noted. As this study was conducted as a pilot study, it is not sufficiently powered to draw scientifically valid conclusions. A randomized controlled trial with a larger sample size of participants is necessary to ascertain more conclusive results. Furthermore, our study encountered a high loss to follow-up rate, which has the potential to bias our results. It is important to consider strategies to ensure the retention of study participants if a larger trial is conducted as our loss to follow-up analysis did reveal that the participants who were retained had higher measures of burnout than those who were lost. Although we did not collect data on the reasons behind participants not completing the final questionnaire, it is possible that the length of questionnaire and lack of incentive for completing the questionnaire could have contributed to our higher lost-to-follow-up rate. Indeed, a Cochrane review completed by Edwards et al. (2009) found that responses to electronic questionnaires were improved by utilizing incentives for competition and using shorter questionnaires. Finally, it should be noted that our study was conducted during a local wave of the COVID-19 pandemic. Morgantini et al. (2020) demonstrated that the COVID-19 pandemic has increased burnout among health care professionals and thus, the timing of our study could have affected the psychometric measures of our participants as well as our loss to follow-up rate due to additional stressors and competing priorities. We attempted to mitigate the impact of organizational influences of burnout (such as workload) through the randomization procedure; however, differential organizational pressures experienced during the intervention period could have led to variability in the levels of participant engagement with the intervention.

Implications for Occupation Health Practice

As health care professionals often operate in an interdisciplinary work environment, it is not surprising that burnout affects health care professionals across disciplines. Our study demonstrates that an intervention that utilizes innovative methods of content delivery (such as self-directed smartphone application) may be an effective strategy to improve the wellness of professionals working in a multidisciplinary environment. We hope that our study encourages a greater use of experimental design in evaluating interventions to reduce burnout among a multidisciplinary population and encourages the increased evaluation of innovative intervention methods in research on health care professional burnout and resilience.

Applying Research to Occupational Health Practice.

Burnout is an issue that can affect the professional and personal lives of healthcare workers. This study highlights the high levels of burnout experienced by healthcare professionals practicing in emergency department settings and investigates the implementation of an innovative intervention to reduce burnout. Our randomized controlled trial revealed that the use of a self-directed smartphone resiliency training application resulted in a statistically significant decrease in a burnout measure amongst our sample of healthcare professionals. Innovative methods of delivering interventions to combat burnout amongst healthcare professionals should be explored by occupational health leaders. In addition, our findings suggest that workplace interventions that are directed to a multidisciplinary audience are feasible and potentially effective at reducing burnout amongst healthcare professionals.