Exploring Residents’ Well-Being and Burnout via Qualitative Ecological Momentary Assessment

Abstract

Purpose

Most of what is known about resident burnout and wellness comes from cross-sectional snapshot surveys. The purpose of this study was to elicit qualitative perspectives on wellness from a cohort of internal residents over time using ecological momentary assessment.

Method

Drawing on principles of ecological momentary assessment, 13 different open-ended survey prompts were delivered between October and March during the 2019–2020 academic year. Participants were 88 randomly selected internal medicine residents from 4 internal medicine training programs in the Northeast.

Results

The response rate was 95%. Three main themes regarding wellness were self, program/education environment, and medical/structural system. A fourth theme, the desire to provide quality patient care, cut across all other themes. The patient care theme repeatedly stressed residents’ desire to spend more time with patients. The self theme primarily reflected messages about personal emotions and the need for work–life balance and wellness. The program/education environment theme reflected the value of learning, teamwork and community, and program culture. The medical/structural system theme showed that residents’ experiences were shaped by the efficiency of their days, and largely a product of their schedules and administrative support. Closing advice to future trainees was optimistic and reassuring.

Conclusions

While findings support much of what has been learned via single-occasion survey snapshots, an ecological momentary assessment design allowed a deeper dive into contextual associations. The results affirm the primacy of patient care and also highlight the value of teamwork and culture. Peers and program leaders are heavily influential in setting the tone for the learning experience, whether for the day or with a more enduring message of respect and support. There is opportunity to maximize high- or higher-value learning experiences for residents and find solutions to reduce and reframe the perceived “low-value administrative work” that is part of care coordination.

An enlarging literature demonstrates that medical residents, like other health care workers, are burned out.^1–3 The stress and time requirements of medical training put lives out of balance, and the associated burnout is itself associated with reduced sleep, coping, satisfaction, empathy, and patient safety. The identification of these concerns has occupied much research in medical education. Recently, investigators have pivoted from learning whether and how residents are burned out to identifying ways to combat the trend.^4–6

Toward those goals, program directors have aimed to improve the work environment by optimizing scheduling, improving program culture by creating more opportunities for social gatherings, and supporting more options for self-care (e.g., gyms, naps, healthier food options).^7–11 Importantly, while self-care is stressed, there is an increasing recognition that such efforts address symptoms more than their structural causes. In addition, nearly all of what is known about resident wellness comes from cross-sectional assessments that provide point-in-time estimates of the boundaries between well and unwell. These assessments provide a quantitative, but superficial, overview of the causes and correlates of burnout. Missing are extended examinations of how wellness develops or is challenged by the training context.

Ecological momentary assessment (EMA) provides a framework for designing studies to evaluate a phenomenon overtime and in depth. The methods have been well used in psychology and psychiatry for over 2 decades but are rare in medical education research.¹² The key features of EMA are gathering multiple assessments that are in real-time, real-world, and repeated.^13,14 According to Shiffman, one of the founders of EMA methods, “EMA aims to minimize recall bias, maximize ecological validity, and allow study of microprocesses that influence behavior in real-world contexts.”^14(p.1) It is important to note that EMA is not a single research method, but rather a research approach that has many uses and incorporates varied designs and degrees of retrospection. Using a time-based design and a focus on contextual associations, our purpose was to engage a cohort of residents over several months and elicit their perspectives on wellness by asking them to reflect on personal, programmatic, and system level drivers of well-being.

Method

Overview

The iCOMPARE trial (individualized Comparative Effectiveness of Models Optimizing Patient Safety and Resident Education) among 63 internal medicine training programs found high levels of burnout and dissatisfaction among residents.¹⁵ We invited respondents at 4 programs who had participated in the original trial to respond to open-ended prompts on 18 occasions over academic year 2019–2020. Prompts were time-based at irregular intervals approximately two weeks apart to elicit reactions to the training program across time rather than to specific rotations. The study was reviewed and deemed exempt by the University of Pennsylvania Institutional Review Board (#821156).

Participants

All residents at 4 internal medicine programs in the Northeast were invited to participate (Johns Hopkins University Program; Johns Hopkins University/Bayview Medical Center Program; Main Line Health System/Lankenau Medical Center Program; University of Pennsylvania Health System Program). Among those who were interested, we enrolled random samples of 22–25 per program, based on an expected 20%–30% drop-out rate and a goal of 15–18 residents per program. The 4 programs comprised 2 large university programs, 1 small university program, and 1 small community-based university affiliated program. Of the residents expressing interest in participation, we randomly selected 22–24 per program to participate. Of the 88 residents, 33 (37%) were postgraduate year 1 (PGY1), 27 (31%) were PGY2, and 28 (32%) were PGY3. Overall, 52 (59%) were female, and 60 (68%) identified as white. Residents received a $100 Amazon gift card each time they responded to 3 prompts.

Prompts

Eighteen different prompts eliciting open-ended qualitative responses were designed by the study team to address the range of constructs in the Shanafelt model of burnout.⁶ The sequential prompts were designed to be completed in less than 5 minutes (see Supplemental Digital Appendix 1, at http://links.lww.com/ACADMED/B203, for the sequential list of prompts). Brief, “momentary” tasks are a key component of EMA. We distributed these via Research Electronic Data Capture (REDCap). Initial outreach for the prompts was predominately sent during the week, most commonly Mondays and typically between 11 a.m. and 1 p.m. Only one survey was initially prompted on a weekend.

Our intention was to deliver prompts at approximate 2-week intervals over the academic year in order to capture the ebb and flow of wellness over time. However, in March 2020 the COVID-19 pandemic affected the programs. Four of the subsequent prompts were specifically tied to the pandemic. Those responses are excluded from these analyses as was one prompt requesting photographs. Each respondent was assigned a number of 100, 200, 300, or 400 to correspond with their program, and then a sequential number within a program, starting with 01. Respondent demographics were in a separate restricted file and could be linked by the 3-digit ID.

Analysis

Two trained coders (L.E.W., A.Y.) coded the data under the supervision of a project manager (W.E.). Early reading and synthesis started after the first prompt. After all data were collected, the coders met in alternating teams of 2 to initiate a formal content analysis of the responses.^16,17 We developed codes within each prompt, contributing iteratively toward a master codebook that captured the frequently occurring themes across prompts but also retained those unique to some prompts. Inter-rater agreement was assessed repeatedly and targeted to be above .80. We resolved coding disagreements internal to the coding team, who regularly checked with the larger study team about code definitions (and alignment with the goals for the project). There were 1,480 distinct responses. Sixteen codes were collected into 4 themes. Coding was managed with NVivo qualitative software, version 12 (QSR International, Doncaster, Victoria, Australia). The coders and the larger project team met biweekly to discuss emerging themes, starting with organic labels and then iterating into 4 themes comprising almost all of the codes and consistent with a biopsychosocial model of care.

Results

A total of 186 residents expressed interest; 88 residents were enrolled. The mean response rate to the 13 prompts included in this analysis was 94.8%; the minimum was 85.2% and the maximum was 96.6%.

Overview

Common themes emerged across multiple prompts. The prevalence of themes did not vary significantly across the 4 programs. Our conceptualization is similar to a biopsychosocial model of care in which resident responses were assigned to 3 nested themes: self, program/education environment, and medical/structural system,¹⁸ and also with the Shanafelt and Noseworthy conceptualization of burnout.⁶ Overlapping all areas was the fourth central patient theme and acknowledgement that quality patient care was the central outcome that mattered most. Within these 4 high-level themes are prevalent subthemes. Closing thoughts for future trainees suggest an optimistic and reassuring world view.

Patient

Residents craved more time with patients. They repeatedly wrote of the desire to provide quality care or learn to do so. The best days were those with time for deep patient and family engagement. At the end of the day, moving patient care forward was a top goal. Like the two responses below, comments often included reminders that a focus on patients was why they chose the profession in the first place:

[To be a better doctor] I think I can spend more time with my patients. Many times we are bogged down with chart work and orders and spend minimal time at bedside.

—Resident 419, PGY3, Program 4

Increase the amount of time per outpatient visit to at least 25 or 30 minutes of face time with patients per patient visit (especially in particular for complex patients in internal medicine), to eliminate the dearth of time to talk and examine patients. Current model of 15–20 minutes is definitely insufficient amount of time to build rapport with patients. Increasing time with patients will make patient care more meaningful and will help the resident be able to make a greater impact on the health of their patients.

—Resident 311, PGY2, Program 3

Self

Residents’ responses in the area of self primarily comprised 2 themes: emotions and the need for work–life balance and wellness.

Emotions.

The prevalent subthemes were positive and optimistic. Residents wrote positively about the meaning they find in their work, compassion for their patients, and gratitude for the opportunity to be part of the patient care team. When asked to name one or two ways of themselves showing gratitude, one person responded:

Give one specific piece of positive feedback to one of my colleagues daily. Thank my patients more when I see them.

—Resident 210, PGY2, Program 2

Residents also wrote about insecurities and anxiety about making mistakes or not “knowing enough,” and about feeling stressed and underappreciated. Some days were overwhelming, as this comment indicates:

Drivers for a difficult day include feeling overwhelmed (trying to care for many medically complex, actively sick patients, who are angry with you for various reasons), feeling stupid (explaining things poorly during rounds and being criticized or, worse, giving the wrong treatment—e.g., giving diuretics when you should be giving fluid—and hurting someone), and feeling generally unwell and fatigued (after having had 6 of these days in a row for example).

—Resident 301, PGY1, Program 3

Work–life balance and wellness.

Residents often acknowledged their need for a life outside of the hospital, the importance of leaving work behind at the end of the day, and taking care of themselves. The most trying days or weeks were those with long hours that limited time for relationships and a social life, and that reflected the little control residents have over the pace of their days. They longed for “golden weekends” or scheduled vacation time, and flexibility on days to allow for personal appointments. One resident noted:

I think the quality of time outside the hospital is just as important as the quantity. For example, being off of work on a weekday when no one else is off work isn’t great for your wellness, and might actually be worse! Sitting around by yourself at home since everyone’s working isn’t that enjoyable. Maybe asking a question about how a residency program can increase the quality of life and wellness when residents are off work would be revealing.

—Resident 201, PGY1, Program 2

Strategies involving exercise, sleep, and reflection were frequently mentioned. Residents believed they knew what they could do for their own wellness, but often expressed regrets about the lack of time and flexibility to do them:

I think good work days come hand in hand with the ability to have time for yourself outside of work and feel balanced, so that you have the mental energy to commit to work when you’re there.

—Resident 108, PGY1, Program 1

Program/education environment

Residents offered much commentary about education, the multiple avenues for learning, the importance of teamwork and community, and how each of these was influenced by the culture of the program.

Education.

Residents noted the availability of many educational conferences, although sometimes busy clinical services limited their ability to take advantage of them. There was also shared sentiment that the best learning occurred at the bedside. Common “wants” in the education theme were more free time for learning and elective time, fewer required conferences, more feedback, and a better understanding of how they were being assessed:

[A] factor that contributes to an awesome work day is how much I learned that day, whether it be through patients, during conference, or on rounds. My satisfaction with my day hinges on whether or not I feel like I learned something that day.

—Resident 110, PGY2, Program 1

I would recommend program directors focus on concrete ways in which they can improve the experience of residency. This includes ensuring high-quality educational programming (e.g., teaching conferences, resident report) and that residents be provided the mental space and time to give our attention to this programming.”

—Resident 222, PGY3, Program 2

Teamwork and community.

Multiple prompts elicited comments praising the value of teamwork. Effective teams adapted to the demands of the day, provided assistance to one another, communicated well, were able to resolve conflicts, and were the key element of a good day.

Residents also expressed appreciation for their colleagues as a whole. They enjoyed (and longed for more) activities such as retreats to bring them together for social connections, support and camaraderie. Several comments underscore this subtheme:

The last great work day I had was wonderful mainly because of the team I was working on. The resident and attending both really enjoyed teaching and incorporated it into rounds without significantly impeding rounding efficiency. They had both spoken with me early on about the things I wanted to work on and tailored teaching/questions to help challenge me. They also both expressed an interest in getting to know who team members were beyond the hospital, which made it more enjoyable to work together and easier to be honest about our knowledge gaps.

—Resident 105, PGY1, Program 1

The most difficult workdays for me have occurred when I’m working with a team in which I do not feel like I’m perceived as an equal, valuable member with strengths that contribute to making the team better. I feel especially demoralized when I am spoken down to or made to feel stupid when I am trying the best I can and working as hard as I can.

—Resident 305, PGY1, Program 3

Culture of the program.

Residents recognized themselves as operating on the low end of a steep and traditional hierarchy. Each program seemed to have its own style of communicating and messaging priorities, as well as how it did (or did not) prioritize wellness. Some comments about culture were linked to perceptions of being overworked, underappreciated, and mistreated. Like the observations below, mostly, however, residents praised program leadership support of wellness outside of the direct patient care arena through tangible items like food/snacks, access to gyms, nice work spaces, and access to mental health services:

My program also has a wellness coach which has been particularly helpful to some residents. I think if residents were able to have a psychologist on campus who accepted their insurance, residents would utilize this very frequently.

—Resident 410, PGY2, Program 4

For me, a big part of wellness and wellbeing are being around senior residents, fellows and attending physicians who are eager to teach in a nice, not intimidating way and being surrounded by colleagues who work on giving constructive rather than destructive feedback. A question that would be interesting to ask is “How is your wellbeing affected by senior colleagues, and what can we do to promote the change in the culture of medical education, moving on from an ‘older’ culture of humiliation and intimidation to a more friendly constructive environment?”

—Resident 304, PGY1, Program 3

Medical/structural system

Residents wrote frequently about the structure of their schedules and administrative tasks. Together these 2 subthemes were shaped by the efficiency of their days.

Scheduling.

Long days were wearing. “Getting out on time” was part of a good day. Residents wished for more vacation time and protected weekends, and schedule changes expressed variably across programs in the form of different/shorter block schedules, more freedom of choice, and more attention to the sequence of rotations. Although there was a subset of residents who enjoyed the slower pace and greater independence of overnight call, it was generally perceived to be a strain, as these two comments illustrate:

Resident wellness and wellbeing has to originate from systemic factors that create an environment of wellness. Our program does a very careful job in creating such an environment through the schedule—more time-intensive and “difficult” rotations (i.e., nights, ICU [intensive care unit], floors) are thoughtfully interspersed with more “relaxed” rotations (i.e. elective, ambulatory).

—Resident 401, PGY1, Program 4

Realistically, the medical residency industry and PDs [program directors] need to recognize (and publicly acknowledge) that working 60–80 hours per week is not conducive to wellness in the long term no matter what you do. It is not possible to find time to exercise and get a good night of sleep when you are on ICU/wards schedules and high patient loads. I would appreciate it if someone would call out the Sisyphean nature of residency wellness.

—Resident 306, PGY1, Program 3

Administrative tasks.

Almost universally, residents described too much time spent on tasks of low educational value, for example, validating parking tickets, making phone calls to set up transportation home, or calling for prior authorizations. More ancillary support would be a welcome addition in all programs. While they appreciated their roles in overall care coordination, they regretted the amount of time they spent outside of the patient’s room, documenting in the electronic health record. Three residents observed:

[Program directors should] Invest in auxiliary staff to complete non-MD tasks so that residents can spend time at the bedside/practicing medicine. This will improve skill, performance, and sense of job fulfillment.

—Resident 219, PGY3, Program 2

A lot of my frustrating days come from logistical nightmares/discharges…. “Systems” issues are exceptionally frustrating—I did not spend 4 years of undergrad and 4 years of medical school to get [to] deal with “prior auths” or setting up supplies at home. I studied this hard so I can treat patients with my medical knowledge, not how well I can coordinate discharges. I think a lot of this can be prevented with more social work or “resident assistants.”

—Resident 102, PGY1, Program 1

[The best thing my program has done to address resident wellness is to] Decrease the amount of social work/case management work we do. Minimize non-physician tasks.

—Resident 417, PGY3, Program 4

Efficiency.

Residents longed for more efficiency. They wrote of their own need to learn how to balance and prioritize multiple tasks, create their own efficiencies in writing note s (i.e., using more dotphrases) and developing heuristics (headsets/scripting). Favored strategies included focus, practice, delegation, and optimal use of downtime:

[Difficult days are driven by] systems problems making it difficult to get things done. Days like this could be avoided by improving systems efficiency to get things done faster.

—Resident 121, PGY3, Program 1

An awesome work day is one that is efficient, collaborative, and accomplished. A humming team that can get things done and get home will leave everyone happy, including the patient. Speed bumps are met with quick decisions and effective plans and coordination. This could be accomplished with more efficient use of time, improved communication between team members, and inviting communication between members.

—Resident 416, PGY3, Program 4

Pearls of wisdom

The final prompt to residents was “Finally, taking into consideration all that you’ve learned in the past year, what ‘pearls of wisdom’ would you offer the rising residents?” Responses were nearly uniformly positive and none referenced the pandemic, even though this prompt was given in early May 2020 when COVID-19 care dominated resident activity. Multiple responses suggested “taking one day at a time” and “enjoying the ride.” The second most frequent messages were about taking care of oneself and maintaining relationships/interests outside of work. Third were the messages about “don’t be afraid to ask stupid questions” and “asking for help is okay.” Less prevalent were reminders to “stay curious” and “practice kindness.” Finally was the reminder that it is really all about the patient—listen to them and get to know them. “It’s a privilege to be a physician.”

Discussion

Over 6 months, 88 internal medicine residents from 4 programs responded to repeated open-ended survey prompts about wellness and burnout using an EMA design.^12–14 The purpose of this qualitative approach was to understand from residents’ perspective real-time, real-world perceptions that might deepen understanding of findings from our previous quantitative work documenting correlates of burnout¹⁵ while avoiding recall bias, a key strength of EMA as designed by Shiffman.¹⁴ Some prompts asked for personal reflection and goal setting. Others were outward looking, reflecting on programs and systems. The 4 primary themes—patient, self, program educational environment, and medical/structural system—fit well within a model similar to a biopsychosocial view.¹⁸ These themes were largely consistent across the 4 programs, suggesting a set of core reactions to the academic environment.

The primacy of patient care expressed by these residents validated core professional values, even as the challenges of work hours and the other stresses of residency were experienced. Alongside a growing literature warning that contemporary health care demands redirect residents away from traditional goals,^19,20 the comments from residents in this study largely reaffirm heroic goals. Residents described an overriding desire to get to know their patients’ life stories even as they recognized the need to “move care forward.”

Residents also noted the importance of self-care and work–life balance. At the same time, many of the drivers of wellness seemed to be external and came from the program. Key ingredients to support wellness were teamwork, cultures, and schedules. How residents’ value teamwork and culture mirrors the extensive literature documenting the importance of teamwork for patient safety and health care worker well-being.^21–26 The “strength” of the appreciation of teamwork also overshadows the constant but low-level complaints about schedules. That said, the positive sentiment was at times a bit fleeting—when teams gel, the workday goes smoothly. How it goes today is not necessarily related to what will happen and how one will feel tomorrow. Culture has a more enduring influence and is slower to shift. Similar to professionals in all types of occupations, residents want to feel valued and respected.

This study has limitations. The data reported here were collected pre-pandemic. Educational programs are likely to change permanently based on experiences with the pandemic. Second, participants came from only 4 institutions in the northeastern part of the country, and were all in a single specialty, internal medicine. However, we purposefully chose large and small programs, and were able to randomly select participants within most programs. Third, we opted for prompts that could be answered in just a few minutes, consistent with an EMA approach.¹⁴ Prompts requiring longer responses might have provided deeper comments, but would likely have reduced the response rate from 95%. We also did not look at individual participants’ responses over time, as is common with EMA, as our prompts varied with each occasion. Fourth, in order to enable anonymity, we chose not to ask what service or unit each resident was on, and how many days they had been on that service, information that may have contextualized the responses. Finally, social desirability may have played a role in responses although residents were assured their program director(s) would not see their responses.

This study also has strengths, including its large size, multi-month design, and span across multiple residency programs with nearly complete response rates. Medicine residents submitted over 2,000 capsules of data about wellness. Past studies have provided snapshots of the resident experience through cross-sectional surveys. This study brings an EMA approach^12–14 to medical education and captures experiences repeatedly for a large group of residents.

Conclusions

The gaps we observed between residents’ experiences and desires are consistent with perceptions of external constraints—ceilings in resources from hospitals that, for example, limit availability of ancillary support, or regulations from outside organizations that increase administrative burden for programs without an obvious benefit. The goals and wishes expressed by the internal medicine residents in this study are also consistent with perceptions of the goals and wishes of program directors. That alignment is reassuring, as is the high value residents place on providing quality patient care and both receiving and providing quality education. Next steps should be targeted at program-level interventions focused on building or improving culture, teaching and monitoring the effectiveness of interprofessional teamwork training, and optimizing scheduling and task assignments to support patient care, resident wellness, education.