ABSTRACT
Background: Teaching the practice of high-value care (HVC) is an increasingly important function of graduate medical education but best practices and long-term outcomes remain unknown.
Objective: Whether a multimodal curriculum designed to address specific drivers of low-value care would affect resident attitudes, skills, and performance of HVC as tested by the Internal Medicine In-Training Exam (ITE).
Methods: In 2012, we performed a baseline needs assessment among internal medicine residents at a community program regarding drivers of healthcare utilization. We then created a multimodal curriculum with online interactive worksheets, lectures, and faculty buy-in to target specific skills, knowledge, and culture deficiencies. Perceived drivers of care and performance on the Internal Medicine ITE were assessed yearly through 2016.
Results: Fourteen of 27 (52%) residents completed the initial needs assessment while the curriculum was eventually seen by at least 24 of 27 (89%). The ITE was taken by every resident every year. Long-term, 3-year follow-up demonstrated persistent improvement in many drivers of utilization (patient requests, reliance on subspecialists, defensive medicine, and academic curiosity) and improvement with sustained high performance on the high-value component of the ITE.
Conclusion: A multimodal curriculum targeting specific drivers of low-value care can change culture and lead to sustained improvement in the practice of HVC.
KEYWORDS: High-value care, cost effectiveness, curriculum assessment, curriculum development, graduate medical education, medical culture
1. Introduction
Moving towards higher-value medical care is a critical objective for the nation’s healthcare system and an important challenge for the medical education community [1,2]. While the medical education community has acknowledged the priority of high-value care (HVC), best practices to teach trainees and physicians remain unknown [3]. We hypothesized that framing a curriculum around specific drivers of low-value care would engage learners in a pragmatic approach to shaping attitudes, knowledge, and skills surrounding HVC.
2. Methods
The study protocol was approved by the Institutional Review Board of Providence Health and Services. In 2012, trainees in a 27-member community-based internal medicine residency program were surveyed to assess learner needs, baseline attitudes, and drivers of low-value care. A multimodal curriculum was then designed to teach HVC by addressing specific barriers to its practice. Separate online worksheets for PGY-1 and PGY-3 residents were created to promote interactive learning and were followed by faculty feedback. The PGY-1 worksheet incorporated presentations, articles, reflection, and role-play to teach knowledge and skills around the following drivers of low-value care: patient requests, cost unawareness, reliance on subspecialists, and academic curiosity. The PGY-3 worksheet provided evidence-based tools for residents as they approached graduation and specifically addressed the drivers of discomfort with uncertainty and defensive medicine. Didactic sessions at resident noon conferences were used to introduce the framework of the curriculum and reinforce key concepts. Examples of curricular materials can be found in the supplementary data.
In addition to resident activities, specific attention was paid to shaping the informal curriculum surrounding HVC. Core faculty were engaged as critical stakeholders in teaching HVC and reached consensus on fundamental aspects of medical practice–defining value as the balance between benefits and all costs, high-quality history and physical exam skills, proficient communication, use of evidence-based medicine, and physician numeracy. We established a quarterly Grand Rounds series devoted to highlighting value in health care, which reinforced the concepts and culture of HVC. Finally, residents were supported to implement their own quality improvement projects in this domain.
Outcome measures were annual surveys of perceived drivers of utilization at our institution as well as performance on the high-value section of the Internal Medicine In-Training Exam (ITE). These have been administered yearly through 2016. Temporal trends were analyzed using linear regression in PAST v3.14, with p-values <0.05 considered statistically significant.
3. Results
Fourteen of 27 (52%) residents in the program attended the initial noon conference and completed the survey at baseline (2012) about perceived drivers of healthcare utilization at our institution. Yearly follow-up surveys were completed by 41–70% of the residents at subsequent noon conferences. Since all categorical residents were required to meet with one of the authors to discuss their worksheet, at least 24 of 27 (89%) residents engaged in some part of the curriculum.
Over 3 years of follow-up, there was a significant decrease in the importance of several drivers (p < 0.05 for trend): patient requests, reliance on subspecialists, defensive medicine, and academic curiosity (Table 1). There was no significant change in cost unawareness, inadequate history/exam skills, time pressure, discomfort with uncertainty, availability of new technologies, financial incentives, and allegiance to individual patients as drivers of care. Of note, the four drivers that improved were all specifically addressed in the value curriculum while drivers which were not part of the curriculum (e.g., time pressure, financial incentives) did not improve.
Table 1.
Perceived drivers of healthcare utilization at our institution on a five-point Likert scale.
| Perceived drivers | 2012 | 2013 | 2014 | 2015 | p-value |
|---|---|---|---|---|---|
| Cost unawareness | 3.92 | 4.27 | 3.43 | 3.87 | 0.46 |
| Patient requests | 3.92 | 3.55 | 3.14 | 3.27 | 0.037 |
| Inadequate history and physical skills | 2.62 | 3.55 | 2.57 | 2.80 | 0.78 |
| Time pressure | 3.23 | 3.36 | 2.86 | 2.70 | 0.1 |
| Reliance on subspecialists | 3.92 | 3.73 | 3.21 | 3.17 | 0.004 |
| Discomfort with uncertainty | 4.23 | 3.82 | 3.57 | 3.80 | 0.18 |
| Availability of new technologies | 3.38 | 3.27 | 3.14 | 3.20 | 0.58 |
| Financial incentives | 1.92 | 2 | 2 | 1.93 | 0.98 |
| Defensive medicine | 4.38 | 4.64 | 3.57 | 3.60 | 0.003 |
| Allegiance to individual patient | 2.69 | 2.91 | 2.93 | 3.00 | 0.33 |
| Academic curiosity | 3.38 | 3.18 | 2.64 | 2.73 | 0.03 |
1 – Not at all important; 2 – Slightly important; 3 – Somewhat important; 4 – Very important; 5 – Extremely important
The Internal Medicine ITE is an important external assessment tool for our program and every resident takes the ITE every year. In 2012, our baseline year, the exam began incorporating HVC as a specifically tested content area. At baseline, our program performance was in the 26th percentile nationally which improved to the 84th percentile nationally 1 year after introduction of the value curriculum and has remained well above the national average ever since (Figure 1).
Figure 1.
Program-wide performance on the high-value care component of the internal medicine in-training exam (p = 0.03 for trend).
4. Discussion
HVC, both in practice and medical education, goes well beyond cost-containment to encompass broad principles of ‘good medicine’. As such, a HVC curriculum integrates well into many aspects of medical education but also poses challenges to an intentional and coordinated approach [4]. Framing our curriculum around specific drivers of low-value care gave residents tangible tools to improve their practice while also learning key clinical principles. Shifting resident attitudes and perceived drivers were most successful when a specific gap in knowledge or skills could be identified and addressed (e.g., how to respond to patient requests).
Our residents demonstrated rapid and sustained improvement on the high-value component of the ITE exam with initiation of our curriculum. Performance of appropriately conservative care on standardized tests has been shown to mirror the intensity of practice at the training institution [5]. The culture and practice patterns of training environments are powerful forces which shape physicians’ cost consciousness in their subsequent practices [6,7]. It appears the micro-environment of our program has been changed by the curriculum as the decrease in academic curiosity as a driver of utilization signalled a shift to a value-driven culture. In addition, specific attention to institutional culture and the development of role models were likely key to our program’s success.
Limitation of research: While it adds to the current understanding of education in HVC, our curriculum and study has limitations. First, we did not directly measure changes in practice or clinical outcomes. Rather, our outcome measures were resident attitudes and performance on a standardized test with patient scenarios. Second, we did not have a control group so it is not possible to rule out a ‘Hawthorne effect.’ However, the significant and sustained improvement on standardized test performance over 3 years indicates a real effect. Third, our curriculum’s multimodal approach simultaneously addressing skills, knowledge, and culture makes it difficult to tease apart the relative importance of each component. Perhaps it is the very multimodal nature of our approach which is necessary to successfully tackle a complex problem like HVC. Fourth, the small size of our program and faculty made concerted action easier to take than at a large institution. As prior studies teaching residents HVC have been small (none more than about 100 residents) [3], how to scale up a curriculum such as ours remains an open question. By delivering a great amount of our educational content online, our curriculum is set up to deploy the knowledge and skills components to a wide audience. The critical piece will be recruiting enough change champions across an organization to affect culture change.
5. Conclusion
Sustained improvement in the practice of HVC can be achieved by:
Identifying the drivers of low-value care in a practice environment
Designing a multimodal curriculum to address skills and knowledge deficiencies
Reshaping the institution’s culture of medicine
Supplementary Material
Funding Statement
The authors report no external funding source for this study
Disclosure statement
No potential conflict of interest was reported by the authors.
Supplementary material
Supplemental data for this article can be accessed here.
References
- [1]. Berwick DM, Hackbarth AD.. Eliminating waste in US health care. Jama. 2012;307(14):1513–1516. [DOI] [PubMed] [Google Scholar]
- [2]. Weinberger SE. Providing high-value, cost-conscious care: a critical seventh general competency for physicians. Ann Intern Med. 2011;155(6):386–388. [DOI] [PubMed] [Google Scholar]
- [3]. Stammen LA, Stalmeijer RE, Paternotte E, et al. Training physicians to provide high-value, cost-conscious care: a systematic review. JAMA. 2015;314(22):2384–2400. [DOI] [PubMed] [Google Scholar]
- [4]. Korenstein D. Charting the route to high-value care: the role of medical education. JAMA. 2015;314(22):2359–2361. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [5]. Sirovich BE, Lipner RS, Johnston M, et al. The association between residency training and internists’ ability to practice conservatively. JAMA Intern Med. 2014;174(10):1640–1648. [DOI] [PMC free article] [PubMed] [Google Scholar]
- [6]. Chen C, Petterson S, Phillips R, et al. Spending patterns in region of residency training and subsequent expenditures for care provided by practicing physicians for medicare beneficiaries. JAMA. 2014;312(22):2385–2393. [DOI] [PubMed] [Google Scholar]
- [7]. Ryskina KL, Halpern SD, Minyanou NS, et al. The role of training environment care intensity in US physician cost consciousness. Mayo Clin Proc. 2015;90(3):313–320. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.