Adaptation and Evaluation of Military Resilience Skills Training for Pediatric Residents

Abstract

Background:

An evidence-based trauma-informed resilience skills training program developed for deployed military personnel was adapted and pilot-tested with pediatric residents. We anticipated high satisfaction ratings and changes in knowledge, beliefs, and self-efficacy related to coping with stress and trauma.

Methods:

The intervention included 6 skill-based modules covering emotion regulation, communication with angry patients and parents, reflective narrative, inspirational goal setting, problem-solving, and developing a self-care toolbox. An optional survey was administered before and after the training.

Results:

After training, 76% rated resilience skills as important, 60% were satisfied, and 82% indicated the training changed how they will respond to patient-related grief and trauma. They became more likely to believe attendings are affected by patient deaths and to know what helps them cope when they disagree with the medical decision making of others, more skilled in recognizing signs of stress and trauma, and more knowledgeable about evidence-based interventions.

Pediatric residents routinely care for extremely ill and dying children. Often, they have not yet acquired the vital skills of managing high distress in their patients and the families of their patients, successfully coping with the death of a child or adequately caring for themselves after a traumatic work–related event. The high patient acuity encountered by the residents can have lasting consequences. Pediatric residents are at heightened risk for burnout and depression compared with the general population, with rates of burnout ranging from 39% to 75% and rates of depression between 11% and 20%.^1–5 Those who are depressed or burned out report poorer health and higher medical error rates than those who are not burned out or depressed. In one study, depressed pediatric residents made 6 times more medical errors than their nondepressed peers.³ Risk for residents can be compounded by exposure to adverse events, such as medical errors and/or unexpected negative patient outcomes, potentially triggering trauma symptoms.^6–8

The National Quality Forum recommends that health care institutions better identify and support second victims, defined as hospital workers who are traumatized after an unanticipated adverse patient event, medical error, or patient-related injury.⁹ The harm experienced by traumatized hospital workers has been compared with the harm that occurs to military personnel involved in unintentional “friendly fire.” Rates of trauma symptoms among resident physicians are higher than expected in the general population, ranging from 12% to 28%.^6–8,10

Interventions, such as peer support programs, have been adopted by some hospitals to respond to the immediate needs of potential second victims.^11,12 However, less attention has been paid to interventions designed to prevent the second victim phenomenon. Trauma-focused cognitive behavioral interventions are known to be effective for trauma symptoms but are not systematically integrated into training. To address the need for effective prevention strategies, an evidence-based intervention currently in use to effectively improve resiliency and enhance positive coping in active duty military families was adapted for University of California, Los Angeles (UCLA) Health System workers. The initial pilot of this training program was conducted with pediatric residents at UCLA Mattel Children’s Hospital.

Psychological resilience, known to be an important factor to enhance quality of care and sustainability of the health care workforce, is defined as the ability to respond to stress in a healthy way with minimal psychological cost.^4,13,14 Research with resilient physicians has revealed helpful themes to consider when developing interventions to enhance resilience.¹⁴ In addition to well-known strategies such as cultivating social support and engaging in self-care and relaxation techniques, physicians scoring high on resilience also noted the importance of self-reflection, continuously evaluating the need for personal or systemic change, maintaining clear boundaries, and using helpful cognitive coping strategies.¹⁴

The Families OverComing Under Stress (FOCUS) resiliency training program is a strength-based, trauma-informed, prevention program.^15,16 Components include psychoeducation and skills training in goal setting, problem-solving, communication, narrative self-reflection, emotional regulation, and trauma trigger management techniques. The FOCUS interventions have been provided to more than 750 000 participants on active duty military installations and have demonstrated effectiveness at reducing psychological distress symptoms and enhancing positive coping in adults and youth.^15,16

We anticipated that an adaptation of FOCUS would (1) be feasible and acceptable to residents, (2) improve residents’ self-efficacy in coping with stress and patient-related grief and trauma, and (3) shift residents’ beliefs about their attendings’ coping thoughts and experiences. To our knowledge, this is the first pilot of a trauma-informed, evidence-informed resilience skills training program for residents.

Methods

Institutional Review Board exemption was obtained for an optional and anonymous survey, used for program evaluation and refinement. Residents were invited to complete an online survey before the first module and after the final module. Surveys were matched using a code known only to the resident. Participants included all pediatric and medicine-pediatric residents. A total of 83 residents completed at least one of the survey(s), out of 97 residents (although not all were available to attend the trainings due to off-site placements, illness, vacations, and international electives). There are 27 residents who completed both the premeasure and postmeasure, 33 who completed the baseline measure but not the posttest, and 23 residents completed the posttest without having completed the baseline measure. Thus, we have 60 completed baseline measures and 50 completed posttest measures. Cases were excluded from individual analyses if data were missing, resulting in varying sample sizes depending on the analysis.

The intervention consisted of six 1-hour modules. Each module was delivered the same week at 4 of our resident training sites. The average number of sessions attended was 2.43 (n = 52), with 2 residents completing more than 4 sessions. Among those participants who completed both pretest and posttest (n = 27; 33% of participants), the average number of sessions attended was 2.59, with 1 resident completing more than 4 sessions.

Training curriculum

The core components of FOCUS were adapted to meet the needs of pediatric residents using relevant examples from the clinical setting. The adaptation team consisted of FOCUS model developers, mental health experts, pediatric attending physicians, and the chief pediatric residents. Adult learning concepts were applied to optimize engagement and learning. An iterative process was used, with ongoing feedback from the chief pediatric residents to ensure relevance and acceptability of each module. Trainers for each session included 1 mental health clinician and 1 pediatrician. Residents were not required to attend the sessions.

Descriptions of the skill-based modules are shown in Figure 1.

Figure 1.

Skills training modules.

Anticipating that residents may be skeptical of the need for resilience training, this potential barrier was addressed by adopting the following tenets and procedures:

1. Skill-based. Each session teaches a specific skill that is practiced during the session.

2. Biological/science frame. The evidence and biological mechanisms supporting each skill are shared.

3. Videos. Each session includes a 5- to 10-minute-long video of a colleague/attending discussing a personal work–related challenge (death of a patient, depression, ethical dilemma, moral distress, career doubt, etc). Not part of the original FOCUS program, the goals of including these videos to the curriculum were to decrease stigma by role-modeling the verbalizing of emotionally difficult topics, to increase applicability of the skills being taught by illustrating situations in which the skills are relevant and to remind residents that these are common clinician challenges by showing an array of clinicians.

4. Applicability. Residents are regularly reminded that the skills can be used personally and with peers, their own family, and their patients/their families.

5. Use with other populations with high exposure to stress. Residents are reminded that this type of training, along with adoption of the skills, has been shown to be effective with active duty military personnel and their families.

6. Other. Each session is co-facilitated by a mental health professional and a pediatrician, with care taken to limit the use of psychological jargon and not to engage in excessive debriefing/processing. The sessions are closed to medical residents and other interested parties so that residents can speak openly without strain that could be caused if medical students or visitors were present.

Self-report measures

Burnout

Before the training, to better describe our sample, we measured burnout using the Abbreviated Maslach Burnout Inventory (aMBI), a 9-item scale measuring emotional exhaustion, depersonalization, and personal accomplishment. Higher scores in emotional exhaustion (being emotionally overextended and exhausted by one’s work) and depersonalization (an unfeeling and impersonal response toward patients) and lower scores in personal accomplishment (feelings of competence and successful achievement) are indicators of burnout. A Likert-type scale included 7 response options scored from 0 (never) to 6 (every day). Consistent with prior studies, burnout scores of 0 to 6 were considered low, 7 to 12 were considered moderate, and 13 to 18 were considered high for each domain.^16–19 Career satisfaction is included in the aMBI, but not part of the burnout construct. Higher scores indicate more satisfaction with being a doctor.²⁰

Resilience

We measured resilience using the Brief Resilience Scale (BRS), a 6-item questionnaire measured on a 5-point scale. Resilience was measured before the training to examine resilience by demographic variables and as a baseline for future research. The BRS has good internal consistency with Cronbach α ranging from 0.84 to 0.91 (0.89 in our sample). It has positive correlations with social relations, coping, and health, and negative correlations with anxiety, depression, and physical symptoms. Higher scores indicate greater resilience.²¹

Beliefs about attendings

Before and after the training, residents’ beliefs about their attending physicians were assessed with 4 independent items (not intended to form a scale; reviewed by developers for face validity) that were written for this evaluation. Items included beliefs about the impact of patient deaths on their attending physicians and about how they would be judged by supervisors after an adverse event if the resident cried or asked for support. These are questions 1 to 4 in Table 4.

Table 4.

Change in beliefs and self-efficacy (post minus pre).

Question	N	Mean	SD	P valuea
1. My attendings would consider me overly sensitive or unprofessional if cried after the death of a patient	27	0.15	0.86	.38
2. My attendings would consider me overly sensitive or unprofessional if I asked them for emotional support after the death of a patient	27	0.04	0.76	1.00
3. My attendings would consider me overly sensitive or unprofessional if I asked them for emotional support after a difficult/adverse medical event	27	0.00	1.07	.94
4. My attendings seem unaffected by the death of a patient	25	0.48	1.00	.03
5. Although tragic, I feel comfortable with my ability to cope with the death of a patient	26	−0.27	1.43	.33
6. I know what I can do to help myself with my own grief after the loss of a patient	26	−0.35	0.85	.06
7. I know what helps me when I disagree with the medical decision making of an attending or family of one of my patients	27	−0.67	1.18	.01
8. I know what helps me when I feel traumatized at work	27	−0.37	1.01	.099
9. I know how to recognize signs of stress and/or trauma in others	27	−0.44	0.70	.005
10. I know evidence-based approaches to assisting others with signs of stress and/or trauma in others	27	−1.41	1.34	<.001

^aWilcoxon signed rank sum test.

Self-efficacy

Also developed for this evaluation were 6 independent self-efficacy questions (not intended to form a scale). These items relied on face validity to assess beliefs, before and after the training, about one’s ability to cope with the loss of a patient, moral distress, and trauma symptoms. These are questions 5 to 10 in Table 4.

Training satisfaction and perception of change

After the training, we measured training satisfaction and perception of change with three 5-point Likert scale questions written for this project. Items ask about perceived importance of resilience training and satisfaction with training and perception of how much the training will change their responses to patient-related grief.

Assessment of modules

After the training, individual modules were assessed using a 5-point Likert scale rating helpfulness of the module and 3 open-ended questions (one thing they learned as a result of the training, their favorite part of the training, suggestions to improve the training).

Statistical analyses

Means, medians, and standard deviations were calculated to characterize the residents’ levels of burnout and resilience and to examine residents’ perceptions of the training curriculum. Independent sample t tests were used to compare male and female residents on resilience and burnout. One-way analysis of variance was used to compare year of residency by resilience and burnout. Wilcoxon signed rank test was also used to determine, as per our expectation, whether more residents disagreed with the 4 belief items and agreed with the 6 self-efficacy items after the training. Open-ended feedback was compiled and reviewed to inform further adaptation of the curriculum.

Results

The demographics of the residents completing both surveys are shown in Table 1.

Table 1.

Frequencies by postgraduate year (PGY) and sex (N = 58).

Postgraduate Year	Male (n = 17)	Female (n = 41)	Total
	No. (%)	No. (%)	No. (%)
1	6 (35)	20 (49)	26 (45)
2	6 (35)	12 (29)	18 (31)
3	4 (24)	7 (17)	11 (19)
4	1 (6)	2 (5)	3 (5)

N = 2 individuals were missing both sex and PGY year at baseline.

Burnout and resilience

Higher scores on emotional exhaustion and depersonalization and lower scores on personal accomplishment reflect higher levels of burnout. In our sample of 60 residents who completed the pretest, about a third of the residents scored high on emotional exhaustion, with most scoring in the moderate range. Close to half of the residents scored in the high or moderate range on depersonalization, and almost three-fourths scored low on personal accomplishment (see Table 2).

Table 2.

Burnout Risk Level.

	Low	Moderate	High
	No. (%)	No. (%)	No. (%)
Emotional Exhaustion Subscale	12 (20)	27 (45)	21 (35)
Depersonalization Subscale	31 (52)	21 (35)	8 (13)
Personal Accomplishment Subscale	444 (73)	15 (25)	1 (2)

No differences were detected between men and women on the 3 primary subscales of burnout or on the domain career satisfaction. Likewise, no differences were detected by year of training; however, greater emotional exhaustion among interns was nearly significant (n = 58; P = .057).

Male residents scored higher than female residents on resilience. No differences in resilience were detected by year of training.

Resilience, burnout, and career satisfaction scores examined by resident sex are shown in Table 3.

Table 3.

Sex by resilience, burnout, career satisfaction.

	Male			Female
	N	Mean	SD	N	Mean	SD	P valuea
Brief Resilience Scale (5-point scale; 5 = highest level of resilience)	17	3.75	0.52	43	3.11	0.79	.003
Emotional Exhaustion Subscale (scores can range from 0 to 18). Higher scores = higher levels of burnout	17	9.82	3.38	41	10.34	4.50	.67
Depersonalization Subscale (scores can range from 0 to 18). Higher scores = higher levels of burnout	17	8.06	3.77	41	6.41	4.28	.17
Accomplishment Subscale (scores can range from 0 to 18). Lower scores = higher levels of burnout	17	14.35	2.87	41	13.56	2.91	.35
Career satisfaction (scores can range from 0 to 18). Higher scores = more satisfaction with being a doctor	17	12.76	4.28	41	12.32	4.25	.72

^aTwo independent sample t test.

Beliefs and self-efficacy

After the resilience training, participants reported several significant changes in beliefs (items 1-4) and self-efficacy (items 5-10). The significant changes in the desired direction (decreased negative beliefs for items 1-4 and increased self-efficacy for items 5-10) were detected on 1 belief item and 3 self-efficacy items, presented in Table 4. In addition, findings nearing significance in the desired direction (defined as P < .10) were detected on 2 additional self-efficacy items.

Resident evaluation of training

After the training, 49 residents rated the training. Three-fourths of the residents rated resilience skills as an “important” or “very important” topic to include in their training, 60% were “satisfied” or “very satisfied” with the training (with another 16% indicating a neutral satisfaction rating), and over 80% of them indicated that the training changed how they will respond to patient-related grief and trauma. Two residents rated the topic as “very unimportant” and 2 indicated that they were “very dissatisfied” with the training. Individual ratings of the modules are shown in Table 5. Overall, the highest rated modules were emotion regulation (mean score = 3.2), personal toolkit (mean score = 3.1), and communication (mean = 3.0). The lowest rated modules were on reflective narrative (means = 2.6 and 2.8) and inspirational goal setting/problem-solving (mean = 2.9).

Table 5.

Frequencies and percentages for modules (N = 49).

	Not helpful	A little helpful	Somewhat helpful	Very helpful	Awesome	NA
	No. (%)	No. (%)	No. (%)	No. (%)	No. (%)	No. (%)
Module 1. Inspirational goal setting; problem-solving	2 (4)	5 (10)	10 (20)	4 (8)	1 (2)	27 (55)
Module 2. Reflective narrative practice—(viewing video)	4 (8)	7 (14)	4 (8)	9 (18)	1 (2)	24 (49)
Module 3. Reflective narrative practice—(creating timeline)	6 (12)	4 (8)	8 (16)	7 (14)	0 (0)	24 (49)
Module 4. Emotion regulation	2 (4)	3 (6)	7 (14)	5 (10)	3 (6)	29 (59)
Module 5. Communication	2 (4)	5 (10)	7 (14)	7 (14)	1 (2)	27 (55)
Module 6. Personal toolkit	3 (6)	8 (16)	9 (18)	6 (12)	5 (10)	18 (37)

Abbreviation: NA, not applicable.

In an open-ended question, residents were asked to name one thing they learned as a result of the training. Of the 23 residents who responded, the most frequently cited answers were increasing their knowledge and skills related to stress and self-care (35%) and learning the benefits of reflection (30%). Participants were also asked to name their favorite part of the training. Of the 17 respondents, 41% mentioned the use of videos, 24% named the interactive training components, and 18% listed emotional regulation skills. Fifteen residents offered suggestions to improve the training, with no consistent themes identified.

Discussion

We successfully adapted an evidence-based military resilience skills training program for use with residents. This initial pilot has revealed that trauma-informed resilience training is feasible and acceptable. Furthermore, residents’ beliefs and self-efficacy shifted in the desired direction, with several significant changes despite our small sample size.

Examining baseline rates of 3 burnout subscales, a little over a third of our residents reported high levels of emotional exhaustion and almost half reported moderate levels of emotional exhaustion, considered to be the most salient component of burnout and found in prior research to be related to poor job performance and turnover.^9,22 A recent meta-analysis concluded that high levels of job support and workplace justice are associated with lower levels of emotional exhaustion, and that high demands, low job control, high work load, low reward, and job insecurity are associated higher levels of developing emotional exhaustion.²³

About half of the residents reported high or moderate levels of depersonalization, characterized by the withdrawal from relationships and a negative, cynical, or callous outlook. These measured levels of emotional exhaustion and depersonalization underscore the urgent need for buffering interventions, such as resilience skills training. Countering these risks, almost three-fourths reported high feelings of work-related personal accomplishment.

Consistent with existing literature, we found no differences in rates of burnout by sex or year of training.³ Although resilience ratings also did not vary by year of training, men reported higher levels of resilience than women, consistent with the literature. This finding may be attributable to both genetic and social factors.^24,25

We anticipated that our training would improve residents’ knowledge and self-efficacy in coping with patient-related grief and trauma, a prerequisite to adopting skills known to mitigate the negative impact of chronic stress and trauma. Despite our small sample size and the low dose of training received, we detected statistically significant changes in the desired direction on several belief and self-efficacy items. This suggests that residents can benefit from short-term, skills-focused training.

After the training, residents were more likely to believe that their attendings are affected by patient deaths, which was likely due to the video narratives integrated into the training sessions. Residents were also more likely to know what helps them cope when they disagree with the medical decision making of an attending or family of one of their patients. Residents reported that they were more skilled in recognizing signs of stress and trauma and more knowledgeable about evidence-based interventions for stress and trauma. Knowledge about self-care related to grief after the loss of a patient was nearly significant in the desired direction and all other changes in beliefs and self-efficacy were in the desired direction, but not significant.

Congruent with the reports of increased self-efficacy, over 80% of the residents indicated that the training changed how they will respond to patient-related grief and trauma. Given the challenge of convincing the residents of the value of the resilience skills, this finding was a welcome surprise. Most residents thought that the topic of resilience is important to include in their training and were satisfied with the training. Their feedback resulted in helpful alterations to the curriculum. For example, participants discussed stress associated with highly distressed families who pressure them to provide nonstandard care. Therefore, we plan to create a new module that provides residents practical skills in effective boundary management. Residents’ favorite part of the training was the videos of their attendings’ modeling narrative reflection through discussion of one of their most challenging moments and describing how they coped during and following the event. We believe this is an important component of the training as it was designed to affect pediatric resident culture by reducing stigma related to feeling distressed and illustrating that work-related coping challenges (such as grief, trauma, and moral distress) are universal experiences and acceptable to discuss.

Our conclusions are limited by the lack of a comparison group, a small sample size, low intervention dose, and only short-term measurement of outcomes. The low sample size is attributable to the large number of residents who were clinically engaged or off-site during their optional noon conference time. We educated the residents about the need to regularly use these skills to benefit from them, but we did not assign practice or homework. In addition, we did not conduct any follow-up evaluation to compare residents who adopted skills with those who did not to examine changes in resiliency or burnout. Finally, converting the curriculum to an online format in the future may better reach residents who are unable to attend their noon conference.

Conclusions

This evaluation provides preliminary evidence that resilience skills training for health system employees is feasible, is well tolerated, and can lead to desired improvements in knowledge and self-efficacy. Using the data collected, we revised our curriculum to be more broadly applicable to health system employees. The new curriculum is currently being pilot-tested with a new group of health care providers. If we replicate our findings, we hope to expand our use of this training and to more carefully evaluate the impact of the skills adopted by participants on resilience, burnout, depression, trauma symptoms, and other important outcomes.