Pediatric Residents' Experiences With High-Value Care at an Academic Children's Hospital

Abstract

Background

The Accreditation Council for Graduate Medical Education mandates residents incorporate cost considerations into patient care. However, resident experiences with high-value care (HVC) in the clinical setting have not been well described.

Objective

To explore pediatric residents' experiences with HVC and its facilitators and barriers.

Methods

We performed a qualitative study with a grounded theory epistemology of pediatric residents recruited by email at a large academic children's hospital. We conducted focus groups (n=3) and interviews (n=7) between February and September 2020 using a semi-structured guide. Data were analyzed using the constant comparative method. Codes were built using an iterative approach and organized into thematic categories. Sampling continued until saturation was reached.

Results

Twenty-two residents participated. Residents' value-based health care decisions occurred in a complex learning environment. Due to limited experience, residents feared missing diagnoses, which contributed to perceived overtesting. Resident autonomy, with valuable experiential learning, supported and hindered HVC. Informal teaching occurred through patient care discussions; however, cost information was lacking. Practice of HVC varied by clinical setting with greater challenges on high acuity and subspecialty services. For children with medical complexity, identifying family concerns and goals of care improved value. Family experience/demands influenced resident health care decisions, contributing to high- and low-value care. Effective collaboration among health care team members was crucial; residents often felt pressured following perceived low-value recommendations from consultants.

Conclusions

Resident HVC learning and practice is influenced by multiple factors in a complex clinical learning environment.

Introduction

Pediatric health care costs have been rapidly increasing,¹ with overdiagnosis and overtreatment contributing to both higher costs and potential harm to patients.^2,3 One strategy to reduce costs is to train physicians to deliver high-value care (HVC), which has been defined as “the right care to the right patient at the right time”⁴ or the ratio of “quality over cost.”⁵ The Accreditation Council for Graduate Medical Education mandates that residents incorporate consideration of cost awareness into patient care.⁶ In response, national initiatives have been launched, such as the Choosing Wisely campaign (an initiative to advance dialogue around avoiding unnecessary medical tests and treatments for patients),⁷ and formal HVC curricula such as online modules have been developed for residency programs.^5,8,9

However, learning is theorized to occur when it is situated in authentic activity of the profession (situated learning) such as when caring for patients in the clinical environment.¹⁰ Yet, few studies have explored how residents learn about HVC and try to practice it in the clinical setting.^11–14 Informal teaching about HVC in the context of patient care occurs infrequently,^12,15 and residents may not often practice certain aspects of HVC.^13,16 A national survey of internal medicine residents reported that 24% shared information about estimated costs of tests and treatments with patients, and 46% incorporated costs into clinical decisions.¹³ A study of internal medicine and general surgery residents found that 88% and 68%, respectively, self-reported ordering perceived unnecessary laboratory tests.¹⁶ Inadequate cost transparency, diagnostic uncertainty, attending physicians having final treatment decisions, and lack of a cost-conscious culture may impede resident ability to practice HVC.^15,16 However, published studies are limited by use of survey methodology, which does not explore study participants' perspectives or experiences in depth, and lack of data that include pediatric residents.

Ensuring that residents learn how to provide HVC during their training is important because of educational imprinting, in which learners model observed behaviors even if doing so conflicts with what they are taught.¹⁷ A study of internists and family physicians found an imprinting effect for excessive health care spending that lasted at least 16 years after completion of residency.¹⁷

Thus, deeper understanding of resident experiences with practicing and learning about HVC in the clinical setting is needed. Insight gained from this study will inform residency programs tasked with graduating residents who are competent at delivering HVC. The aims of our study were to: (1) explore pediatric residents' experiences with HVC at a large academic children's hospital, and (2) describe facilitators and barriers to resident HVC practice.

Methods

Qualitative Research Methods and Theoretical Constructs

Our study was informed by situated learning, a conceptual framework which theorizes that learning is in part “a product of the activity, context, and culture in which it is developed and used.”¹⁰ In a workplace-based training environment, residents care for patients while also interacting with families, attending physicians, interdisciplinary health care team members, and other learners in a large and busy academic children's hospital. Therefore, we conducted a qualitative study because it is well suited to answer questions about informal learning within such complex learning environments¹⁸ and utilized grounded theory methodology designed to develop, primarily through qualitative exploration, an integrated set of concepts that provide an explanation of a social phenomenon.^19,20 As resident perspectives of HVC are likely to be multiple and diverse, we used an interpretivist research paradigm.²¹

Setting and Sampling Strategy

Our pediatric residency program, consisting of 98 residents, is associated with a university-affiliated, 434-bed, quaternary care, free-standing children's hospital. Residents in our program have experienced a curriculum around HVC, consisting of a didactic session followed by 5 case competitions focused on cost-effective diagnostic evaluation. However, attendance at these was not tracked. All pediatric residents were eligible to participate in focus groups or interviews. Residents were sampled as 2 groups: (1) postgraduate year (PGY) 1 and (2) PGY-2 and PGY-3. We sampled PGY-2 and PGY-3 residents (senior residents) together as they had similar clinical roles that differed from PGY-1 residents. For ease of scheduling, we purposefully and systematically recruited residents who were already present for a mandatory educational conference to participate in focus groups/interviews. We continued sampling until analyses indicated we reached thematic saturation (the point at which analysis of new data is not producing any new insights).¹⁹ Following review of published survey and focus group questions evaluating resident HVC learning and practice to offer insight into potential question topics,^5,13 we constructed a semi-structured focus group/interview guide (Box) to explore variables potentially affecting resident HVC learning and practice in the clinical setting. Probes were utilized, such as “Can you give an example?” and “Tell me more about that.” The guide was first pilot tested by conducting an informal focus group with 6 pediatric residents to ensure clarity of questions, which did not result in any question modifications. Only data from subsequent focus groups/interviews were analyzed.

Objectives

To explore pediatric residents' experiences with high-value care (HVC) and its facilitators and barriers.

Findings

Resident experiences with learning and practicing HVC are influenced by multiple dynamic factors of a complex work environment and include fear of missing diagnoses, resident autonomy, clinical setting, patient complexity, family satisfaction/demands, and interdisciplinary health care delivery.

Limitations

This study was from a single program and may have been affected by volunteer bias.

Bottom Line

Implementing curricula that focus solely on HVC general concepts and costs are likely to be insufficient in teaching residents how to deliver HVC; additional strategies are needed to overcome barriers in the clinical learning environment.

Box Focus Group and Interview Guide

1. How do you feel high-value care is being practiced at this institution?

   1. How would you compare the practice of high-value care between different clinical services?^a

   2. How do you incorporate high-value care into your own practice?

2. How have your clinical experiences impacted your attitude toward high-value care?

3. How have your clinical experiences contributed to your learning about high-value care?

4. What factors (personal, institutional, cultural) support your ability to practice high-value care?

5. What factors (personal, institutional, cultural) impede your ability to practice high-value care?

6. How do families influence your ability to practice high-value care?

7. How do other members of the health care team (attending physician/consultants/nurses/ancillary staff members) influence your ability to practice high-value care?^a

8. What else about high-value care do you feel is important to discuss?

Data Collection

Study investigators were E.N. (PGY-3 pediatric resident), A.B. (PGY-2 pediatric resident), S.J. (professional research assistant), and L.B.S. and M.T. (pediatric hospitalists). Two investigators (E.N., A.B.) conducted focus groups and individual interviews several months into the 2018–2019 and 2019–2020 academic years so residents would have completed multiple clinical experiences; E.N. and A.B. have previously participated as investigators in qualitative studies in which they conducted focus groups/interviews. Three focus groups, each consisting of 5 residents lasting approximately 40 minutes, were done in a hospital conference room from February to March 2020. A light meal was provided as a token of appreciation. To triangulate methods of data collection and accommodate resident schedules, 5 additional interviews involving 7 residents (2 sessions each with 2 residents and 3 individual interviews), each lasting about 15 minutes, were done in June and September 2020. The focus group/interview guide was iteratively modified as data were analyzed to explore emerging findings. Data were audio recorded and transcribed by an online audio transcription service.

Data Analysis

In accordance with qualitative methodology, we analyzed data using the constant comparative method.^19,20 Four investigators (A.B., E.N., S.J., L.B.S.) performed data analysis concurrently with data collection. Following initial immersion in the data, at least 3 investigators individually coded transcripts after each focus group/interview was transcribed. All codes were inductively developed. Study investigators who were physicians likely had some understanding of, and anticipated, potential factors affecting resident ability to practice HVC, such as workload, patient/family demands for testing, and working with consultants. As a group, the lists of codes were compared, and discrepancies were resolved by consensus. Through an iterative approach, initial codes were modified, and others were added to best reflect data content. Following analysis of the final transcript, investigators in pairs (E.N./L.B.S., S.J./L.B.S.) returned to the data, re-reviewing each transcript to verify the coding scheme. Thematic saturation was achieved when no additional changes to the coding scheme were made. The group of 4 investigators together organized the codes into categories that described themes. HyperRESEARCH software (Researchware Inc, Boston, MA) was used to organize the data and facilitate writing of thematic summaries. We established dependability of methods through purposeful sampling, involving multiple analyzers, peer debriefing following each focus group/interview, and member checking, in which we discussed themes with a subset of study participants to ensure accuracy of our interpretations.¹⁸

Managing Reflexivity

Reflexivity was managed by having a diverse research team that included investigators with and without subject familiarity or shared experiences.²² Investigators self-reflected and discussed the possible influences of researcher position on participant recruitment and ability to elicit comments. Awareness of personal backgrounds was stimulated as interpretations of data were discussed between researchers with and without topic familiarity. During the study one researcher (E.N.) graduated from residency and became an attending physician, which facilitated data analysis through a different lens.²² We also managed reflexivity through repeated review of the data and measures to establish trustworthiness (triangulation, multiple researchers involved in data analysis, and member-checking).^22,23 Participants provided verbal consent, and the institutional review board approved the study protocol (#19-1291).

Results

Of 71 invited residents, 22 (31%) participated in focus groups and interviews (10 PGY-1, 12 PGY-2/PGY-3). Data analysis yielded 7 themes surrounding the facilitators and barriers of resident HVC practice and learning: clinical experience; resident autonomy supporting and hindering HVC; informal HVC teaching; clinical setting; patient complexity, family experience, and demands; and interdisciplinary collaboration. Quotations are provided in the text and in the Table. Comments from PGY-1s and senior residents were similar; however, when senior residents reflected on their time as PGY-1s, they described improved clinical and communication skills that enhanced their ability to deliver HVC for some patients.

Table.

Themes With Illustrative Quotations

Theme (Definition)	Quotation
Clinical Experience (Amount of experience caring for patients)	“For me, getting more experience in different clinical scenarios to understand what to order would be helpful.” (PGY-1)
	“My style is very different. I'm going to be minimalistic in my workup at first. Over the past 3 years I've gotten more comfortable just holding off.” (Senior resident)
Resident Autonomy Supporting and Hindering High-Value Care (Perceived ownership of patients and support of health care decision-making)	“It's a teaching hospital. Tell me what you [intern] want to do. We can try it. I would not do that, but this is how you're shaping your skills as a doctor. Then in the future, ‘I don't think I'll do that next time because it didn't help.'” (Senior resident)
	“When I was the primary person for patients, that was when I felt the most invested in making sure families were updated and understood. That was probably higher value care.” (Senior resident)
Informal High-Value Care Teaching (Teaching and role modeling in clinical setting)	“Attendings somewhat keep me in check. You're ordering this test, but why are you ordering it. There is a culture of asking why and I think that's good.” (PGY-1)
	“What is the cost of the hospital stay? No one ever tells us that. What is the cost of each of these labs or treatments? How much does it cost to put a kid on low flow versus high flow nasal cannula?” (Senior resident)
Clinical Setting (Service or setting for health care encounters with patients)	“On neurology if the patient had any medical problems other than neurologic, I sometimes felt it wasn't the best care for the patient. Anything beyond why they were there was overlooked. Like nutrition.” (PGY-1)
	“Hospital medicine is pretty good at knowing when to order something and being conscious about it.” (Senior resident)
Patient Complexity (Patient with multiple complex health care issues/needs)	“If you ask [parents of child with medical complexity], what are your goals, what are your concerns. A lot of them don't want to be in the hospital. They'll say, I just want to make sure it's not X, Y, or Z. Otherwise, we can handle this at home.” (PGY-1)
	“18-year-old girl was super complex. We asked the mom, ‘If the swallow study says she can't swallow, are you still going to feed her?' The mom said, ‘Yeah because that's important.' We were, ‘Is it worth the cost and the radiation?' [The mom] ended up requesting they cancel the study. That's an example of saving the family money and the kid radiation.” (Senior resident)
Family Experience and Demands (Family satisfaction and expectations during the health care encounter)	“Sometimes families are your allies. If they don't want to have the child experience a poke or procedure that you don't think will change management, then you are on the same page. Family is satisfied and you've reduced unnecessary testing.” (PGY-1)
	“The first thing you hear is both parents yelling at you, ‘I want the shunt tapped, a urinalysis, and a GIP [gastrointestinal pathogen] panel.' Those situations make providing HVC extremely challenging.” (Senior resident)
Interdisciplinary Collaboration (Communication/working together with other members of the health care team)	“As third years we have more practice with how we communicate with nurses and parents and being able to better and more clearly explain ourselves after years of practice.” (Senior resident)
	“It's such a tough balance with consultants. You want their opinion. You need their opinion. But what if I disagree with your recommendation? There's some element of being collegial.” (PGY-1)

Clinical Experience

Residents described inconsistently practicing HVC, in part because of their level of clinical experience. Lack of clinical experience led to perceived overtesting as residents feared missing diagnoses that would result in patient harm. A PGY-1 commented, “Your ability to practice HVC is limited by the amount of experience you have. If you're inexperienced, you're relying on things you've read or stories you've heard. That seems a little dangerous to not do certain things.” Yet, few residents worried about liability. Senior residents felt more comfortable with monitoring patients' physical examinations, limiting testing, and using a stepwise diagnostic evaluation for some patients compared to when they were PGY-1s.

Resident Autonomy Supporting and Hindering HVC

Resident autonomy affected their attitudes toward HVC, and both supported and hindered their HVC practice. In some instances, feelings of autonomy and ownership of patients facilitated resident investment in practicing HVC. However, when residents' decisions were not supported by attending physicians, future HVC efforts were threatened. One senior resident commented, “On nights you fight the battle of ‘I'm going to do HVC.' You have a long conversation with the family. This is why I don't want to do this. Often the day team just does it in the morning. What was the point of me trying? You get a little burnt out.” Training at a teaching institution, residents felt their autonomy should be supported, so that they might learn from assessing the outcomes of chosen diagnostic studies or interventions. Thus, senior residents described supporting PGY-1 decisions, even those they recognized as lower value care.

Informal HVC Teaching

High-value care teaching occurred through informal patient-centered discussions and role modeling among attending physicians, senior residents, and interns. Residents valued when their clinical reasoning for ordering a test was challenged or when they were asked how a test would impact patient management. Senior residents provided teaching for interns. As one PGY-1 stated, “After we're done with everything [making a decision] I always ask my senior, can you please explain to me the thoughts behind it and what you would do differently. That's the most helpful thing.” Some attending physicians effectively discussed HVC while others did not. In one aspect of role modeling, residents felt that younger attending physicians, compared to more experienced ones, tended to do more testing. Teaching about financial costs was lacking. Residents also described a need for additional feedback on their care decisions, wondering if the interventions provided were necessary or if alternative care would have improved certain outcomes such as hospital length of stay.

Clinical Setting

Practicing at a quaternary care institution made rare diagnoses (eg, Huntington's disease) appear common and increased consideration of low-value testing for patients. Residents described varied HVC delivery among different clinical settings. Although HVC was often a focus on hospital medicine services, it was perceived to be less often practiced in high-acuity settings and subspecialty services. A senior resident in the ICU commented, “These kids are really sick and there's a reason they're in the ICU. I still think if the patient has bronchiolitis then they have bronchiolitis. We do chest x-rays every day that don't need to be done.” In the emergency department, residents felt overtesting occurred due to fear of missing a diagnosis or discomfort in discharging children without an accurate diagnosis. On inpatient subspecialty services (eg, oncology), lack of in-depth knowledge about a specialty limited resident ability to assess value of care decisions. In addition, the focus on a single organ system without addressing other important patient issues was felt to result in lower value care.

Patient Complexity

Providing HVC for generally healthy children with common diagnoses (eg, bronchiolitis) was felt to be easy. However, residents were less certain about what constituted HVC for patients with medical complexity. Published clinical care guidelines were not felt to be applicable to the management of these patients. One senior resident said, “You can't generalize everything the AAP [American Academy of Pediatrics] says about pneumonia to our patients. They have cerebral palsy and a million other things. So sometimes they deserve a little bit more.” Additionally, residents spent significant time on administrative tasks (eg, home care orders) for these patients, which limited time spent with families and resulted in perceived lower value care. Patients with medical complexity often had multiple active health care issues that utilized inpatient health care resources. Yet, residents recognized that determining and addressing families' specific concerns and goals of care promoted HVC. Once families' concerns were addressed, they were satisfied taking their children home and working on other issues as outpatients.

Family Experience and Demands

Residents noted how family attitudes contributed to health care decisions. Communication with families often, but not always, resulted in HVC. Emphasizing how a test result would not change patient management helped avoid unnecessary testing. Yet, in some instances, residents felt performing a medically unnecessary test added value if it improved the family experience. One PGY-1 stated, “Many families are so much more comfortable when they actually get that positive respiratory pathogen panel. It doesn't make us feel any better, but for them, they have an answer.” In several instances, residents resorted to ordering unnecessary tests for patients, despite communication efforts, to avoid further confrontation, effort, time spent, or dissolution of the therapeutic relationship. Residents, at times, struggled with discussing costs with families, wondering if the family felt the focus of the medical team shifted from the child's health to cheaper treatment options.

Interdisciplinary Collaboration

Effective collaboration among all interdisciplinary team members (nurses, respiratory therapists, primary care physicians, consultants) was crucial for HVC. Residents' acknowledging other team members' concerns and explaining their rationale for not pursuing certain interventions supported HVC. However, residents struggled delivering HVC when other team members provided mixed messaging regarding the plan of care to families. The primary care physician (PCP) could play an important role; when the PCP prepared families to expect certain interventions or studies in the hospital that residents felt were unnecessary, practicing HVC was difficult. Residents described how consultants variably affected HVC. In some instances, residents felt consultants would know the right diagnostic test to order, which supported HVC. However, consultants often recommended perceived low-value studies, and residents felt pressured to follow these recommendations. One senior resident stated, “You're deferring to their judgement, so we default to doing everything a consulting team wants, which may not be the best care.”

Discussion

This study found that pediatric residents experience delivery of high- and low-value care and learn about value-based health care decisions in a complex workplace-based learning environment. Through qualitative exploration, several findings from this study about resident HVC learning and practice in the clinical setting build on existing literature.

A key finding from this study was the relationship between resident autonomy and HVC. Previous studies reported resident autonomy is associated with HVC learning,¹¹ lower hospital resource use,²⁴ and a more positive attitude toward HVC.²⁵ Providing further elaboration, residents in this study described how autonomy can initially lead to either high- or low-value care, yet potentially facilitate HVC in the long-term from experiential learning. Residents described infrequent HVC teaching and role modeling not only in ICU and emergency department settings, similar to findings from an earlier study,¹⁵ but also on subspecialty services. Another finding from this study was the important role of communication skills with families and interdisciplinary team members (eg, nurses). Yet, the role of communication skills in facilitating or inhibiting delivery of HVC has not been well described in published studies.^{11,13,14,16,26} A previous study found that residents, when discussing unnecessary testing with standardized patients, primarily used rational rhetorical appeals.²⁷ Findings from this study demonstrated that residents used, often successfully, this form of rhetoric. However, lack of time for discussions, or rational rhetoric that did not work, hindered HVC; in addition, shared decision-making, as recommended by the Choosing Wisely campaign,⁷ may have limited efficacy with families who strongly desire test results. Despite its importance, few HVC educational interventions focus on communication skills training,²⁶ and a study of a curriculum focused on communication skills training did not result in improved resident performance.²⁸ Finally, this study adds to the literature by highlighting resident challenges working with consultants and the pressure to follow perceived low-value testing recommendations.

Situated learning theory, which posits that learning is a product of the context and culture in which knowledge is developed and used,¹⁰ helps explain our study's findings. Resident learning was promoted when the culture of the workplace-based training environment supported HVC. However, our study also identified a culture in some settings that hindered resident HVC learning and practice. Thus, an understanding of the influence of the workplace on resident learning is needed. Spatial metaphors are a model used to view the workplace, as they incorporate social, cultural, temporal, and power relationships, and account for the dynamic qualities of the environment.^29,30 One type of spatial metaphor is a practice development crucible, which conceptualizes clinical education as a learning space formed by the confluence of multiple fluid and interdependent forces.²⁹ As illustrated in the Figure, viewing the resident workplace with a crucible metaphor helps to provide an understanding of the types of forces that influence resident ability to learn and practice HVC. Each factor is in a dynamic state that can fluctuate over time and from situation to situation. Thus, the degree to which each factor contributes to the clinical environment is not constant. These dynamic and interdependent individual (eg, resident experience) and workplace (eg, family experience/demands) forces ultimately influence resident HVC learning and practice. The HVC practice crucible has implications for residency programs and for institutions tasked with teaching residents how to deliver HVC. Our study suggests that implementing HVC curricula that focus solely on knowledge about general concepts and costs is insufficient, and additional strategies are needed to try and overcome identified barriers. Institutions could create and offer faculty development programs for different clinical settings to encourage explicit HVC role modeling and discussions. Interdisciplinary HVC education for residents, attending physicians, nurses, respiratory therapists, primary care physicians, and other team members may be able to help create a culture that supports resident HVC practice. Evidence-based guidelines are needed to optimize interactions between residents and consulting services, and how consultant recommendations are utilized. A published model for an HVC consult that includes core components of cost, evidence, shared decision-making, and interdisciplinary communication may positively change resident clinical management.³¹ Finally, HVC curricula could also include communication skills training for residents to promote productive HVC discussions with families.

Figure.

Practice Development Crucible Illustrating Forces Influencing Resident High-Value Care Learning and Practice

Limitations

There are several study limitations. This study was performed at a single pediatric residency program, which may limit its transferability to other programs or institutions. We suspect that several study findings are transferable to other specialties (eg, internal medicine) as residents are likely to share similar perspectives (eg, benefits of autonomy) across disciplines. However, in pediatric programs residents are caring for children as patients while also interacting with children's families, which contrasts with health care delivery in other specialties and may result in different findings. Volunteer bias may exist with comments made by residents that could have differed from those of nonparticipants. Finally, resident experiences varied by attending physician, and thus comments could have differed with additional clinical interactions.

Next Steps

Future research could continue to explore how supporting resident autonomy influences resident short- and long-term HVC practice, the utility of interventions to improve HVC teaching across different clinical settings, and optimal HVC communication strategies with families and interdisciplinary team members. Research efforts could also evaluate resident HVC experiences at other types of institutions (eg, community hospitals).

Conclusions

Pediatric residents' experiences with learning and practicing HVC are influenced by multiple dynamic and interacting factors of a complex work environment.