5-Year Follow-Up of a Physician Performance Feedback Report Intervention to Reduce Overuse and Excess Cost: A National Cohort Study

Supplemental Digital Content is Available in the Text.

Abstract

BACKGROUND

Mohs micrographic surgery efficiently treats skin cancer through staged resection, but surgeons' varying resection rates may lead to higher medical costs.

OBJECTIVE

To evaluate the cost savings associated with a quality improvement.

MATERIALS AND METHODS

The authors conducted a retrospective cohort study using 100% Medicare fee-for-service claims data to identify the change of mean stages per case for head/neck (HN) and trunk/extremity (TE) lesions before and after the quality improvement intervention from 2016 to 2021. They evaluated surgeon-level change in mean stages per case between the intervention and control groups, as well as the cost savings to Medicare over the same time period.

RESULTS

A total of 2,014 surgeons performed Mohs procedures on HN lesions. Among outlier surgeons who were notified, 31 surgeons (94%) for HN and 24 surgeons (89%) for TE reduced their mean stages per case with a median reduction of 0.16 and 0.21 stages, respectively. Reductions were also observed among outlier surgeons who were not notified, reducing their mean stages per case by 0.1 and 0.15 stages, respectively. The associated total 5-year savings after the intervention was 92 million USD.

CONCLUSION

The implementation of this physician-led benchmarking model was associated with broad reductions of physician utilization and significant cost savings.

Mohs micrographic surgery (MMS) is an effective and efficient surgical procedure to treat skin cancer. Mohs micrographic surgery uses a series of staged excisions based on histologically determined margin status to ensure complete removal of cutaneous tumors while preserving cancer-free tissue. It is primarily used to treat basal and squamous cell carcinomas but has also proven effective for melanoma in high-risk anatomic sites due to higher cure rates than wide local excisions.^1,2 As nonmelanoma and melanoma incidence continues to increase in the United States, MMS use will continue to increase as well.³ The authors previously described wide variation in the number of staged resections among surgeons, which may be associated with increased reimbursement from additional stages, and if unnecessary, with potentially lower value care.⁴

To address overuse of stages during MMS, the American College of Mohs Surgery (ACMS) developed an overuse metric for the mean MMS stages per case, as part of the physician-led Improving Wisely quality collaborative in 2017.⁴ Surgeons were sent an individual performance report that depicted their mean number of stages per case relative to their national peers.⁵ The authors previously published that 83% of outlier surgeons reduced their mean number of stages per case in the first year. Since that initial study period, 2 additional rounds of notifications were conducted. In this study, the authors report the 5-year follow-up data surrounding the practice patterns and the associated cost savings resulting from the program.

Methods

Study Design, Setting, and Participants

Between January 2017 and December 2021, Johns Hopkins University in partnership with ACMS conducted 3 interventions to Mohs surgeons in the United States who billed MMS procedures to FFS Medicare. The quality improvement initiative was called Improving Wisely and was originally funded by the Robert Wood Johnson Foundation. Individual surgeon performance reports depicting a surgeon's mean stages per case for head and neck lesions, and a separate report for trunk and extremity lesions, were sent by mail to all ACMS members in February 2017, July 2018, and June 2019. Reports were suspended in 2020 and 2021 due to the COVID-19 pandemic and associated disruptions to medical and surgical practice. Surgeons who received all 3 reports were included as the study notification (intervention) group in this analysis, and surgeons who were not ACMS members and did not receive any reports were in the control group. Surgeons who performed 10 or fewer MMS procedures in either the preintervention or postintervention period were excluded as a requirement of the Medicare data use agreement to protect patient identity. The authors also identified surgeons' inlier/outlier status based on the preintervention performance period (January 1, 2016- January 31, 2017) and used the cutoff method (2 standard deviations greater than the mean number for all physicians billing MMS), as previously defined.⁴ The Johns Hopkins University Institutional Review Board approved this study with waived informed consent.

Data Sources

As described in the previous study, we used 100% Medicare fee-for-service claims from January 1, 2016, to December 31, 2021.^4,5 Current Procedural Terminology codes 17311 (first stage) and 17312 (second stage) were used to identify head and neck lesions and 17313 (first stage) and 17314 (second stage) to identify trunk and extremity lesions. The mean stages per case was calculated using [(17311 + 17312)/17311] for head and neck lesions and [(17313 + 17314)/17313] for trunk and extremity lesions. The overuse metrics used were approved by the ACMS Physician Engagement Council.

Surgeon characteristics evaluated included sex, years since graduation, practice location, solo or group practice, practice region from the Medicare Data on Provider Practice and Specialty, and the Physician Compare National Downloadable File.^6,7 The authors also identified the case volume in the preintervention period.

Outcomes

The primary outcome was surgeon-level change in mean stages per case in the 5-year study period between the intervention and control groups. The secondary outcome was the cost savings to Medicare over the same time period. The authors calculated the cost savings by multiplying the number of cases performed after the postintervention period, change in the mean stages compared with the preintervention period, and surgeons' mean reimbursed amounts in the postintervention period.⁵

Statistical Analysis

In this nonrandomized trial, the authors described and compared surgeon characteristics between the intervention and control groups using 2-sample t-tests for continuous variables and χ² tests for categorical variables. They used one-way ANOVA tests to compare surgeons' mean stages per case within each arm of the study of the intervention and control groups. The authors used independent 2-sample t-tests to compare surgeons' change in mean stages per case between the intervention and control groups in different study periods. Statistical analyses were performed using SAS Enterprise Guide version 7.1 (SAS Institute Inc., Cary, NC) with a statistical significance level of p < .05.

Results

A total of 2,014 surgeons were included in the cohort who performed Mohs procedures on head and neck lesions (Table 1). There were 85 physician outliers (33 in the notification group and 52 in the control group) and 1,929 physician inliers (987 in the notification group and 942 in the control group). Compared with the notification and control groups in both the outlier and inlier groups, the majority of surgeons were male (ranging from 66% to 78%), and the main practice location was metropolitan (ranging from 90% to 100%).

TABLE 1.

Characteristics of Mohs Surgeons for the Head and Neck Cohort

	Outliers, n (%) (N = 85)			Inliers, n (%) (N = 1,929)
	Notified (N = 33)	Non-notified (N = 52)	p	Notified (N = 987)	Non-notified (N = 942)	p
Male, no. (%)	22 (66.67)	36 (69.23)	.80	651 (65.96)	730 (77.49)	<.001
Years since graduationc (median, range)	21 (11–41)	27 (8–49)	.007	20 (3–53)	25 (4–56)	<.001
0–9	0	2 (3.85)		3 (0.30)	39 (4.14)
10–19	15 (45.45)	7 (13.46)		459 (46.50)	283 (30.04)
20–29	11 (33.33)	21 (40.38)		307 (31.10)	290 (30.79)
≥30	7 (21.21)	22 (42.31)		205 (20.77)	317 (33.65)
Unknown	0	0		13 (1.32)	13 (1.38)
Practice location			.42			<.001
Metropolitan	33 (100.0)	51 (98.08)		945 (95.74)	843 (89.49)
Micropolitan	0	0		24 (2.43)	77 (8.17)
Rural	0	1 (1.92)		8 (0.81)	11 (1.17)
Unknown	0	0		10 (1.01)	11 (1.17)
Practice region			.03			<.001
Midwest	2 (6.06)	3 (5.77)		180 (18.24)	188 (19.96)
Northeast	13 (39.39)	7 (13.46)		219 (22.19)	62 (6.58)
South	5 (15.15)	6 (11.54)		350 (35.46)	341 (36.20)
West	13 (39.39)	36 (69.23)		234 (23.73)	347 (36.84)
Other	0	0		4 (0.41)	4 (0.42)
Case volume in prenotification period (median, range)	424 (29, 1869)	124 (22, 1862)	<.001	406 (11, 2,929)	162 (11, 1,550)	<.001
11–200	8 (24.24)	35 (67.31)		181 (18.34)	551 (58.49)
201–400	7 (21.21)	12 (23.08)		300 (30.40)	219 (23.25)
≥401	18 (54.55)	5 (9.62)		506 (51.27)	172 (18.26)
Mean stages per case in 2016 prenotification period (median, range)	2.55 (2.35–3.24)	2.49 (2.36–3.95)	.77	1.62 (1.12–2.35)	1.60 (1.05–2.34)	.014

Outliers in the notification group had fewer years of practice (median 21 vs 27 years) and a higher procedural volume (median 424 vs 124 cases comparing with outliers in the control group). A similar pattern was found in the notification group and the control group among inliers. Outliers were more likely to practice in the west region (notification group n = 13 [39.4%], control group n = 36 [69.2%]), and inliers were more likely to practice in the south (notification group n = 350 [35.5%], control group n = 341 [36.2%]) and west region (notification group n = 234 [23.7%], control group n = 347 [36.8%]). The authors observed a similar demographic distribution to the trunk and extremity surgeon cohort (see Supplemental Digital Content 1, Table 1, http://links.lww.com/DSS/B406).

Following the first intervention in February 2017, the authors observed a decrease in mean stages per case for head and neck tumors in the outlier group, with a greater decrease in the notification group than the control group. A smaller decrease was observed after the second intervention mail out in the July 2018 for outliers, with lower mean stages per case in the notification group. The trends of mean stages per case after the third intervention in June 2019 have been stable for both outliers and inliers (Figure 1).

Figure 1.

Mean stages per case by physician group for head and neck lesions. Blue lines indicate notification mailings.

The mean stages per case decreased from 2.69 to 2.28 among outliers in the notification group and decreased from 2.66 to 2.32 among outliers in the control group (p < .001; Table 2). The inliers in the intervention and control groups also had a significant decrease in the mean stages per case between the postintervention and preintervention period (notification group: −0.06, 95% CI, −0.07 to −0.05; p < .001; control group: −0.07, 95% CI, −0.08 to −0.05; p < .001).

TABLE 2.

Change in Mean Stages per Head and Neck Lesion in Prenotification Period (January 01, 2016–January 31, 2017) and Postnotification Period (July 01, 2019–December 31, 2021)

Status in 2014	Notified in All 3 Interventions (Number of Physicians)	Prenotification Period, January 01, 2016–January 31, 2017		Postnotification Period, July 01, 2019–December 31, 2021		Cost Savings to Medicare	Difference in the Mean Stages per Lesion Between the Postnotification and Prenotification Period (p*)	Difference† in (Post–pre) Differences of Notified vs Not-Notified Groups (p)
		Number of Lesions	Mean Stages per Lesion	Number of Lesions	Mean Stages per Lesion
Outlier	Yes (N = 33)	17,627	2.688	36,150	2.278	10,344, 326.44	−0.440 (−0.561–0.320) p < .001	−0.132 (−0.280–0.016) p = .079
	No (N = 52)	10,939	2.660	19,943	2.316	4,531, 745.51	−0.308 (−0.398 to −0.219) p < .001
Inlier	Yes (N = 987)	471,869	1.648	1,025,213	1.586	44,811, 854.86	−0.064 (−0.073 to −0.054) p < .001	0.001 (−0.013–0.017) p = .820
	No (N = 942)	221,481	1.625	488,415	1.554	21,111, 631.06	−0.065 (−0.077 to −0.054) p < .001

Pre–Post change in mean stages per lesion for head and neck lesions by 2014 status and notification.

^*Paired t-tests were conducted using surgeon-level data and compared surgeons' prenotification and postnotification mean stages per case within each group.

^†Independent 2-sample t-tests were conducted using surgeon-level data and compared surgeons' change in mean stages per case between the notification and control groups.

A similar decrease in stages per case was found in the trunk and extremity surgeon cohort after the first intervention (see Supplemental Digital Content 1, Figure 1, http://links.lww.com/DSS/B406). A larger difference between the intervention and control groups was observed among outliers, and the trends of difference were smaller among inliers. The mean stages per case decreased from 2.42 to 2.02 among outliers in the notification group and decreased from 2.60 to 2.17 among outliers in the control group (p < .001). The inliers of both the intervention and control groups also had a significant decrease in the mean stages per case between the postintervention and preintervention periods, with a reduction of 0.04 stages per case (95% CI, −0.05 to −0.03; p < .001) for the notification group and 0.05 stages per case (95% CI, −0.07 to −0.03; p < .001) for the control group (see Supplemental Digital Content 1, Table 2, http://links.lww.com/DSS/B406).

Over the 5-year period, among outlier surgeons who were notified, 31 surgeons (94%) for head and neck and 24 surgeons (89%) for trunk and extremity lesions reduced their mean stages per case with a median reduction of 0.16 (IQR: 0.10–0.24) and 0.21 (IQR: 0.12–0.28) stages, respectively. Reductions were also observed among outlier surgeons who were not notified [45 (87%, for head and neck) and 31 (94%, for trunk and extremity)] reducing their mean stages per case by 0.1 (IQR: 0.06–0.17) and 0.15 (IQR: 0.08–0.31) stages, respectively (Figure 2 and see Supplemental Digital Content 1, Figure 2, http://links.lww.com/DSS/B406). Among inlier surgeons, 1,321 (68%) for head and neck and 764 (63%) for trunk and extremity lesions reduced their mean stages per case with a median reduction of 0.07 (IQR: 0.03–0.11) and 0.08 (IQR: 0.04–0.13) stages, respectively (Figure 3).

Figure 2.

Change in practice of individual outlier surgeons' overtime (head and neck). Each line represents a MMS surgeon. Green lines decreased between preintervention and postintervention. Red lines increased. MMS, Mohs micrographic surgery.

Figure 3.

Cumulative cost savings (in millions) after the implementation of a physician-led data feedback report intervention.

The authors identified a total cost savings of $80,799,558 for Mohs procedures on head and neck lesions from July 1, 2019, to December 31, 2021. This included a savings of $10,344,326 by the outliers in the notification group, $4,531,746 by the outliers in the control group, $44,811,855 by the inliers in the notification group, and $21,111,631 by the inliers in the control group (Table 2, Figure 3). The authors also identified a cost savings of $11,228,727 for Mohs procedures on trunk and extremity lesions during the same period of time, which included a savings of $2,256,526 by the outliers in the notification group, $1,431,777 by the outliers in the control group, $5,111,977 by the inliers in the notification group, and $2,428,448 by the inliers in the control group (see Supplemental Digital Content 1, Table 2, http://links.lww.com/DSS/B406, Figure 3).

Discussion

This analysis demonstrates the long-term sustainability and positive impact of a physician-specific data report intervention using a metric of over-use defined and endorsed by practicing specialists. In the initial publication, the authors reported the quarterly trends from 2016 to the first quarter of 2018 subsequent to the first intervention notification.⁵ In this analysis, they expanded the timeline to December 2021, which is a 4-year extension. The reduction for outliers was sustained over time, suggesting that repeated notifications and annual discussions about the quality improvement program at the national meeting of the ACMS and privately among surgeons resulted in a culture change. This culture change was likely marked by increased awareness that stages per case were being tracked, and there was some standard for appropriate practice patterns. Using the qualitative approach of surveys and key informant interviews, the authors previously described the implementation quality and perceived impact of the physician-specific data reports among Mohs surgeons. They were able to identify the barriers and facilitators to assist physicians to achieve the best clinical practice.⁸ The authors' findings showed the effectiveness of the key recommendations they previously addressed and the cost-effectiveness of the physician data report intervention.

Other studies have observed a short-term reduction in health care utilization following individual physician reporting initiatives in other areas of medical care, but have not performed additional long-term follow-up.^9–12 Stonko and colleagues⁹ identified the variability of physician use of endovenous thermal ablation performed per patient and implemented a one-time physician benchmarking report intervention that associated with a significant decrease in the number of EVTAs each patient received. A randomized clinical trial conducted by Sacarny and colleagues¹⁰ evaluating peer comparison letters on primary care providers led to an 18-month persistent decrease in quetiapine prescribing. The authors report the first long-term follow-up of a physician data report intervention. In this study, outlier surgeons had a 6 to 10 times greater decrease than inlier physicians.

Low-value care, as a pervasive and enduring problem in the US health care system, has been identified and addressed in different fields in medicine.^13–15 Hicks and colleagues identified a series of low-value practice patterns and highlighted the opportunity to re-evaluate the appropriateness of peripheral vascular interventions.^13,16–18 Kaczmarski and colleagues¹⁴ examined the surgeon re-excision rate among patients who underwent breast-conserving surgery, identifying a cohort of outliers who increased financial burden to the health care system and defining a quality metric associated with low-value care. Electronic health records–based interventions have been conducted in health care professionals and resulted in statistically significant reductions in inappropriate antibiotic prescribing.¹¹ This approach should also apply to other medical fields to assist patient decision support, improve clinical decision support, and strengthen clinician education and feedback. Population-based, supply side interventions with outcome monitoring, such as in accountable care organizations, have the potential to align incentives for cost-effective care and introduce novel quality initiatives.¹⁹

Finally, the estimated $92 million cost savings for Medicare observed in this study was significant. There were relatively modest costs to administer the quality improvement collaborative (totaling $375,000). Thus, the resulting return on investment was much higher than other known strategies designed to reduce over-use, such as preauthorization by insurance companies.

This study has several limitations. First, the authors did not include any clinical indicators or risk stratification in the analysis. Patients with complex tumors might appropriately require additional Mohs stages for clear margins. Meanwhile, the number of stages was not analyzed in the context of the complexity or reconstruction required. It would be a potential harm to patients and would diminish the cost savings of the intervention if physicians took fewer stages but thereby created larger wounds that required more extensive and expensive repairs. Second, the authors only included Medicare fee-for-service beneficiaries who primarily represent a population aged 65 years and older. This is a rich data set that could be used in different areas in dermatology to evaluate value-based care. However, there may be different patterns of surgeon behavior for other insurance providers and for younger patients. The effect of the intervention could be further evaluated in non-Medicare populations or Medicare Advantage patients. Third, there might be other environmental factors or referral bias that impacted the surgeons' performance patterns that were not captured and measured in this analysis. Increased case volume might be also a factor leading to a decrease in stage numbers.²⁰

Finally, the authors observed a concurrent improvement among the control groups in this study that is likely attributable to a spillover effect from the intervention group. The spillover effect also appeared to increase over time as they observed a growing culture change about the topic. Mohs surgeons exist in a relatively insular community, and for this reason, the control group (primarily members of the American Society of Mohs Surgery) was not truly an independent cohort. The authors' Improving Wisely intervention was publicized broadly, published in high impact dermatology journals, presented at dermatology educational conferences inside and outside the ACMS, and adopted by numerous academic departments and large dermatology groups, which often employ colleagues from both cohorts. In fact, the authors observed a perception among many control group surgeons that they could soon be measured on their number of stages per case. As a result, many likely became more aware of their performance and auto-corrected excessive practice patterns. Of note, a small subset of Mohs surgeons belongs to both societies. Therefore, the control group was not able to be fully blinded to the intervention throughout the long-term follow-up period, likely becoming aware through indirect exposure over the years. It is impossible to fully quantitate this “visibility” confounding effect. Future studies could explore and evaluate the factors associated with the spillover improvement noted the control cohort.

Conclusion

The implementation of the physician-led Improving Wisely intervention to show physicians where their performance stands relative to their peers resulted in a rapid, durable, and broad reduction in over-use of a specialty-identified appropriateness measure. This peer comparison model may be a reliable and sustainable approach with promise to reduce health care costs associated with low-value care in numerous specialties.

Supplementary Material

ds-50-558-s001.docx ^{(394.2KB, docx)}