Key Points
Question
Can the use of benchmarked peer-to-peer reports reduce overuse in Mohs micrographic surgery (MMS)?
Findings
In this nonrandomized controlled intervention study including 2329 US surgeons, sharing individual reports of MMS practice patterns with physicians resulted in an immediate and sustained reduction in mean stages per case among outlier physicians and an associated cost savings compared with physicians who did not receive these reports.
Meaning
A nonpunitive, confidential audit-and-feedback intervention may represent an opportunity to reduce overuse of MMS among physicians identified as outliers by their clinical specialty society.
This nonrandomized controlled intervention study compares Mohs micrographic surgery use by US surgeons who received confidential reports including stages-per-case performance data benchmarked to their peers nationally with physicians who did not receive these reports.
Abstract
Importance
Mohs micrographic surgery (MMS) is a skin cancer treatment that uses staged excisions based on margin status. Wide surgeon-level variation exists in the mean number of staged resections used to treat a tumor, resulting in a cost disparity and question of appropriateness.
Objective
To evaluate the effectiveness of a behavioral intervention aimed at reducing extreme overuse in MMS, as defined by the specialty society, by confidentially sharing stages-per-case performance data with individual surgeons benchmarked to their peers nationally.
Design, Setting, and Participants
This nonrandomized controlled intervention study included 2329 US surgeons who performed MMS procedures from January 1, 2016, to March 31, 2018. Physicians were identified using a 100% capture of Medicare Part B claims. The intervention group included physicians affiliated with the American College of Mohs Surgery, and the control group included physicians not affiliated with the American College of Mohs Surgery.
Interventions
Individualized performance reports were delivered to all outlier surgeons, defined by the specialty society as those with mean stages per case 2 SDs above the mean, and inlier surgeons in the intervention group.
Main Outcomes and Measures
The primary outcome was surgeon-level change in mean stages per case between the prenotification (January 2016 to January 2017) and postnotification (March 2017 to March 2018) periods. A multivariable linear regression model was used to evaluate the association of notification with this surgeon-level outcome. The surgeon-level metric of mean stages per case was not risk adjusted. The mean Medicare cost savings associated with changes in practice patterns were calculated.
Results
Of the 2329 included surgeons, 1643 (70.5%) were male and 2120 (91.0%) practiced in metropolitan areas. In the intervention group (n = 1045), 53 surgeons (5.1%) were outliers; in the control group (n = 1284), 87 surgeons (6.8%) were outliers. Among the outliers in the intervention group, 44 (83%) demonstrated a reduction in mean stages per case compared with 60 outliers in the control group (69%; difference, 14%; 95% CI of difference, −1 to 27; P = .07). There was a mean stages-per-case reduction of 12.6% among outliers in the intervention group compared with 9.0% among outliers in the control group, and outliers in the intervention group had an adjusted postintervention differential decrease of 0.14 stages per case (95% CI, −0.19 to −0.09; P = .002). The total administrative cost of the intervention program was $150 000, and the estimated reduction in Medicare spending was $11.1 million.
Conclusions and Relevance
Sharing personalized practice pattern data with physicians benchmarked to their peers can reduce overuse of MMS among outlier physicians.
Introduction
Skin cancer is the most common malignancy in the United States, accounting for more than 5.4 million cases annually at an estimated cost of more than $8.1 billion to the US health care system.1,2 Mohs micrographic surgery (MMS) is a surgical approach that treats skin cancer using a potential series of staged excisions determined by intraoperative histologic examination. Mohs micrographic surgery is an effective and often curative treatment for cutaneous basal and squamous cell carcinoma.3,4 In addition, the procedure is highly efficient, since the surgeon serves the simultaneous roles of oncologic surgeon, pathologist, and generally also reconstructive surgeon. Mohs micrographic surgery use has increased over the last decade owing to the procedure’s superior outcomes, its expanding application to melanoma, an emphasis of MMS in training programs, and increased rates of skin cancer.5,6,7,8,9,10,11,12
We have previously described wide variation among surgeons performing MMS in the mean number of stages used to resect a skin cancer.11 Potential overuse may stem from insufficient technical expertise or the current fee-for-service payment model, which compensates surgeons based on the number of stages performed per tumor, creating a perverse incentive for a surgeon to use an excessive number of staged resections to remove a lesion. Overuse of stages per case burdens patients with unnecessary surgical resections and taxes the health care system with avoidable costs.11 For these reasons, the American College of Mohs Surgery (ACMS), the largest specialty society for MMS, considers a surgeon’s annual mean stages per case as an appropriateness metric for Mohs surgery of the head and neck. To address this disparity in appropriateness and cost, we designed an audit-and-feedback intervention aimed at reducing extreme overuse in MMS by confidentially sharing stages-per-case performance data with individual surgeons benchmarked to their peers nationally for similar head and neck lesions. We examined the association of the intervention with a surgeon’s practice pattern and the overall cost to Medicare.
Methods
Study Design, Setting, and Participants
We conducted a nonrandomized controlled intervention study of surgeons in the United States who billed MMS procedures to Medicare. The intervention group consisted of all ACMS members, and they received an individualized notification that reported their annual mean stages per case benchmarked to national peer performance. The control group consisted of non-ACMS members who did not receive a notification. The prenotification performance period was January 1, 2016, through January 31, 2017. The intervention group received an individualized notification in February 2017. The postnotification period was March 1, 2017, through March 31, 2018. We excluded surgeons who performed 10 or fewer MMS procedures in either the prenotification or postnotification period, as required by our user agreement with Medicare designed to protect patient identity. This study received waived authorization of institutional review board approval by the Johns Hopkins Medical Institutions Institutional Review Board and informed consent was not needed because this study was based on administrative claims data only and did not involve active recruitment of study participants.
Expert Consensus on Best Practices
The Improving Wisely collaborative is a Robert Wood Johnson Foundation–funded initiative that uses national data to identify overuse patterns for various diagnostic, medical, and surgical procedures. In the Improving Wisely model, an overuse metric is developed by expert consensus, and Medicare data are queried to identify physician outliers and inliers.13 Confidential, benchmarked, personalized audit-and-feedback data reports are shared with physicians in an effort to change physician behavior and reduce overuse and cost waste.
As part of the Improving Wisely collaborative, ACMS created a Physician Engagement Council of highly respected national experts and clinical leaders, including the association’s executive leadership as elected by the ACMS membership, and representation from large, small, urban, and rural practice settings. Through this workgroup, the ACMS developed and endorsed a consensus threshold of what constitutes an outlier physician practice pattern of overuse. This consensus definition of an outlier was defined as a physician’s annual mean stages per case for MMS for skin cancers of the head, neck, genitalia, hands, and feet being greater than 2 SDs above the national mean.11 This metric was developed specifically for this study, taking into consideration our inability to control for surgeon case mix.
Intervention
In the month prior to the notification intervention, national leaders in the specialty introduced the Improving Wisely collaborative and objectives in a printed and electronic newsletter. In addition, surgeons in the intervention cohort also received an email notification detailing the upcoming confidential, personalized, audit-and-feedback report 2 weeks in advance of the mailing. All physicians in the intervention cohort were mailed their individualized performance report using the most recent addresses listed with their professional association. Accompanying the individual performance data, a cover letter signed by national peer leaders in the specialty described the benchmarking approach and the goals of the program.
Each report listed the individual surgeon’s mean number of stages per case, the national mean of stages per case, and the consensus metric definition and included a cover letter explaining the Improving Wisely program and the intervention (eAppendix in the Supplement). A graphical representation of an individual surgeon’s performance was depicted using an arrow pointing to the surgeon’s position on the national distribution. Surgeons were classified into 3 groups that were differentiated by color; green represented surgeons considered inliers (ie, performance fell within the acceptable range of clinical variation), yellow represented surgeons whose mean stages per case fell within 1.0 to 2.0 SDs above the national mean, and red represented surgeons considered outliers (ie, performance was more than 2.0 SDs above the national mean). Surgeons who fell in the lattermost category also had the word “outlier” appear on their report next to their mean stages per case along with their percentile rank.
Communications from the ACMS president to the membership at the time of notification described the Improving Wisely reports and invited surgeons to contact the leadership with any questions or concerns. Confidentiality, collegiality, respect, and a spirit of quality improvement were emphasized in all communications and announcements about the study. A follow-up letter was sent to all outlier members offering confidential mentoring and educational resources, including a slide deck on surgical and laboratory techniques and select readings to improve one’s proficiency.
Data Sources
Mean stages per case for head and neck procedures were calculated from the Physician Supplier Part B claims file accessed via the US Centers for Medicare & Medicaid Services Data Center.14 This file includes all fee-for-service claims submitted on a CMS-1500 claims form and includes the claims from physicians and free-standing facilities. We identified the first stage by Current Procedural Terminology (CPT) code 17311 and additional stages by CPT code 17312. The mean stages per case were calculated by dividing the sum of the first and additional stages by the number of first stages (ie, [total CPT codes 17311 + 17312]/CPT code 17311).
We identified surgeon characteristics, including sex, practice location, and region, from the Medicare Data on Provider Practice and Specialty.15 We obtained surgeon year of medical school graduation from the Physician Compare National Downloadable File.16 Surgeon information was linked to claims data using National Provider Identifier numbers.
Outcomes
The primary outcome was surgeon-level change in mean stages per case in the post-notification period compared with the prenotification period. Cost was defined as the Medicare-allowed amount in US dollars paid to surgeons directly for MMS (professional fee only). The cost savings for each outlier was calculated by multiplying their number of cases performed during the postnotification period by their change in mean stages per case and by their mean reimbursed amount for each additional stage (CPT code 17312) during the postnotification period. We calculated the mean cost savings per case for each group by dividing the total cost savings by the total case volume in the postnotification period.
Statistical Analyses
We compared physician characteristics between intervention and control groups using 2-sample t tests for continuous variables and χ2 tests for categorical variables. We used paired t tests to compare surgeons’ prenotification and postnotification mean stages per case within each arm of the study. We used independent 2-sample t tests to compare surgeons’ change in mean stages per case between the intervention and control groups stratified by outlier status. A power calculation was not performed because our study included the entire population of eligible MMS surgeons who billed to Medicare in the specified time period.
We used a multivariable linear regression model to evaluate factors associated with surgeon-level change in mean stages per case.16,17 We modeled surgeons’ postnotification minus prenotification change in mean stages per case as a continuous outcome, with the independent variables being surgeon sex, years since medical school graduation, practice region and location, case volume in the prenotification period, outlier status, notification status, and an interaction term for outlier × notification status. All P values were from 2-sided tests, and results were deemed statistically significant at a P value less than .05. All statistical analyses were performed using SAS Enterprise Guide version 7.1 (SAS Institute).
Results
The number of eligible MMS surgeons in the United States was 2329 (Table 1). There were 140 outliers (53 in the intervention group and 87 in the control group) and 2189 inliers (992 in the intervention group and 1197 in the control group). The outliers in the intervention group paralleled the outliers in the control group in regards to mean stages per case in the prenotification period, male predominance, and metropolitan location of practice (Table 1). Compared with outliers in the control group, intervention group outliers had fewer years in practice (median time, 16 vs 22 years), a higher procedural volume (median prenotification volume, 423 vs 102 cases), and were more likely to practice in the northeast region of the United States (38% [20 of 53] vs 15% [13 of 87]).
Table 1. Preintervention Characteristics of Intervention and Control Surgeons Performing Mohs Micrographic Surgery by Outlier Status.
| Characteristic | No. (%)a | |||||
|---|---|---|---|---|---|---|
| Outliers (n = 140) | Inliers (n = 2189) | |||||
| Intervention (n = 53) | Control (n = 87) | P Valueb | Intervention (n = 992) | Control (n = 1197) | P Valueb | |
| Male | 37 (70) | 62 (71) | .90 | 650 (65.5) | 894 (74.7) | <.001 |
| Time since graduation, yc | ||||||
| Median (range) | 16 (7-40) | 22 (5-48) | .01 | 18 (0-50) | 20 (1-55) | <.001 |
| 0-9 | 7 (13) | 13 (15) | 125 (12.6) | 241 (20.1) | ||
| 10-19 | 24 (45) | 24 (28) | 438 (44.2) | 336 (28.1) | ||
| 20-29 | 15 (28) | 21 (24) | 260 (26.2) | 307 (25.7) | ||
| ≥30 | 7 (13) | 29 (33) | 155 (15.6) | 298 (24.9) | ||
| Unknown | 0 | 0 | 14 (1.4) | 15 (1.3) | ||
| Practice locationd | ||||||
| Metropolitan | 53 (100) | 81 (93) | .30 | 949 (95.7) | 1037 (86.6) | <.001 |
| Micropolitan | 0 | 1 (1) | 25 (2.5) | 90 (7.5) | ||
| Rural | 0 | 1 (1) | 8 (0.8) | 13 (1.1) | ||
| Unknown | 0 | 4 (5) | 10 (1.0) | 57 (4.8) | ||
| Practice region | ||||||
| Midwest | 4 (8) | 5 (6) | .02 | 185 (18.7) | 227 (19.0) | <.001 |
| Northeast | 20 (38) | 13 (15) | 220 (22.2) | 102 (8.5) | ||
| South | 6 (11) | 15 (17) | 354 (35.7) | 433 (36.2) | ||
| West | 23 (43) | 54 (62) | 229 (23.1) | 429 (35.8) | ||
| Othere | 0 | 0 | 4 (0.4) | 6 (0.5) | ||
| Case volume in prenotification period | ||||||
| Median (range) | 423 (41-1864) | 102 (13-1858) | <.001 | 400 (11-2912) | 157 (11-1541) | <.001 |
| 11-200 | 12 (23) | 58 (67) | 191 (19.3) | 722 (60.3) | ||
| 201-400 | 13 (25) | 20 (23) | 308 (31.1) | 273 (22.8) | ||
| 401-2912 | 28 (53) | 9 (10) | 493 (49.7) | 202 (16.9) | ||
| Mean stages per case in prenotification period, median (range) | 2.4 (2.3-3.2) | 2.4 (2.3-3.9) | .24 | 1.6 (1.1-2.2) | 1.6 (1.1-2.2) | .10 |
Abbreviation: NA, not applicable.
Surgeons with mean stages per case of 2.3 or greater were defined as outliers, representing those with a metric greater than 2 SDs above the mean of the national distribution during the prenotification period.
Continuous variables were compared using 2-sample t tests. Categorical variables were compared using χ2 tests.
Years since graduation was calculated as 2016 minus the year of graduation from medical school.
Practice location was identified by surgeon’s core-based statistical area type available in Medicare Data on Provider Practice and Specialty. A metropolitan area is a US Census Bureau–defined urbanized area of at least 50 000 people. A micropolitan area is a Census Bureau–defined urban cluster of at least 10 000 and fewer than 50 000 people. A rural area is a non–core-based statistical area.
Included Puerto Rico.
Following the intervention, outliers in the intervention group demonstrated a decrease in mean stages per case in the first quarter of 2017, which was sustained through the first quarter of 2018 (Figure 1). Outliers in the control group demonstrated a smaller decrease in mean stages per case beginning in the second quarter of 2017, which continued for the remainder of the study period. Following the intervention, the overall mean stages per case decreased from 2.55 to 2.31 among outliers in the intervention group and decreased from 2.56 to 2.46 among outliers in the control group. Outlier surgeons in the intervention group exhibited a mean reduction of 0.26 stages per case, whereas outliers in the control group exhibited a mean reduction of 0.11 stages per case, corresponding to a differential reduction of 0.14 stages per case (95% CI, −0.23 to −0.05; P = .002) (Table 2). Of the 53 intervention group outliers notified, 44 (83%) demonstrated a reduction in mean stages per case in the postnotification period (median [range] reduction, 0.21 [0.05-1.22] stages per case). In contrast, 60 of 87 control group outliers (69%) demonstrated a decrease in mean stages per case in the postnotification period (median [range] reduction, 0.20 [0.002-0.94] stages per case), a difference of 14% in rate of response (95% CI of difference, −1 to 27; P = .07) (Figure 2). Among the 53 intervention group outliers, 9 surgeons (17%) demonstrated a median (range) increase of 0.06 (0.02-0.16) stages per case compared with 27 control group outliers (31%) with a median (range) increase of 0.08 (0.001-0.45) stages per case. Inlier surgeons in the intervention group exhibited a mean reduction of 0.03 stages per case, whereas inliers in the control group exhibited a mean reduction of 0.01 stages per case, corresponding to a differential reduction of 0.01 stages per case (95% CI, −0.03 to −0.003; P = .01) (Table 2).
Figure 1. Mean Stages per Case by Group.
Table 2. Change in Stages per Case by Outlier Status and Notification Group.
| Study Group | Total, No. | Prenotification Period | Postnotification Period | Mean Change in Stages per Case (95% CI)a | P Value | Difference in Changes Between Groups (95% CI)b | P Value | ||
|---|---|---|---|---|---|---|---|---|---|
| No. of Cases | Stages per Case | No. of Cases | Stages per Case | ||||||
| Outlier | |||||||||
| Intervention | 53 | 26 486 | 2.55 | 25 825 | 2.31 | −0.26 (−0.33 to −0.18) | <.001 | −0.14 (−0.23 to −0.05) | .002 |
| Control | 87 | 17 547 | 2.56 | 17 251 | 2.46 | −0.11 (−0.17 to −0.06) | <.001 | ||
| Inlier | |||||||||
| Intervention | 992 | 466 331 | 1.63 | 464 090 | 1.61 | −0.03 (−0.04 to −0.02) | <.001 | −0.01 (−0.03 to −0.003) | .01 |
| Control | 1197 | 268 567 | 1.62 | 286 774 | 1.61 | −0.01 (−0.02 to −0.01) | .001 | ||
Paired t tests were used to compare physicians’ postnotification and prenotification mean stages per case within each group. Tests were conducted using physician-level data.
Independent 2-sample t tests were used to compare physicians’ change in mean stages per case between the intervention and control groups. Tests were conducted using physician-level data.
Figure 2. Individual Surgeon Change in Mean Stages per Case Among Outlier Surgeons.
In the multivariable linear regression model that adjusted for baseline surgeon characteristics (Table 3), the intervention was associated with an additional 0.14 reduction in mean stages per case (95% CI, −0.19 to −0.09; P < .001) among outliers and an additional 0.02 reduction in mean stages per case (95% CI, −0.03 to −0.001; P = .03) among inliers. The intervention was associated with greater change among outliers compared with inliers (coefficient for interaction, −0.13; 95% CI, −0.18 to −0.07). Holding other factors equal, surgeons practicing for less than 10 years since graduation decreased a mean of 0.03 stages per case (95% CI, −0.05 to −0.01) more than those practicing for 30 years or more since graduation. Surgeon sex, practice region and location, and case volume were not significantly associated with a change in mean stages per case.
Table 3. Multivariable Linear Regression Model for Factors Associated With Surgeon-Level Change in Mean Stages per Case.
| Factor | Effect Estimate (95% CI)a | P Value |
|---|---|---|
| Sex | ||
| Male | 0 [Reference] | NA |
| Female | 0.01 (−0.004 to 0.02) | .20 |
| Practice region | ||
| West | 0 [Reference] | NA |
| Midwest | −0.001 (−0.02 to 0.02) | .96 |
| Northeast | −0.004 (−0.02 to 0.02) | .68 |
| South | −0.003 (−0.02 to 0.01) | .71 |
| Other | −0.01 (−0.10 to 0.09) | .90 |
| Practice location | ||
| Metropolitan | 0 [Reference] | NA |
| Nonmetropolitan | −0.002 (−0.02 to 0.02) | .84 |
| Time since medical school graduation, y | ||
| ≥30 | 0 [Reference] | NA |
| 0-9 | −0.03 (−0.05 to −0.01) | .001 |
| 10-19 | −0.01 (−0.03 to 0.003) | .11 |
| 20-29 | −0.01 (−0.02 to 0.01) | .49 |
| Case volume in prenotification period | ||
| 11-200 | 0 [Reference] | NA |
| 201-400 | 0 (−0.02 to 0.02) | .98 |
| 401-2912 | 0.01 (−0.01 to 0.02) | .59 |
| Notificationb | ||
| Nonnotified | 0 [Reference] | NA |
| Notified | −0.02 (−0.03 to −0.001) | .03 |
| Status in prenotification period | ||
| Inlier | 0 [Reference] | NA |
| Outlier | −0.10 (−0.13 to −0.07) | <.001 |
| Notification × statusc | −0.13 (−0.18 to −0.07) | <.001 |
Abbreviation: NA, not applicable.
The continuous outcome modeled was the surgeon-level change in mean stages per cases (ie, postnotification minus prenotification change). Therefore, the effect estimate represented the difference in change between the comparator vs reference groups.
The coefficient for notification estimated the adjusted effect of notification among the inliers.
The linear combination of the coefficient for notification and the coefficient for notification × status estimated the adjusted effect of notification among the outliers. The point estimate of this linear combination was −0.14 (95% CI, −0.19 to −0.09). The coefficient for notification × status estimated the difference in notification effect between outliers and inliers.
Excluding research salaries, approximately $150 000 ($144 per surgeon) was used to cover the costs of the intervention. Using the mean stages per case in the prenotification period as a reference, surgeons’ stage reduction in the postnotification period was associated with a total cost savings of $11 143 882 to Medicare. This included a savings of $2 458 672 by the outliers in the intervention group and $788 262 by the outliers in the control group as well as a savings of $5 114 177 by the intervention group inliers and $2 782 771 by the control group inliers. The mean cost savings per case was $95 for intervention group outliers vs $46 for control group outliers and $11 for intervention group inliers vs $10 for control group inliers.
Discussion
Engaging physicians to promote behavior change around best practices has been a significant barrier to reducing waste in health care.13,18,19,20,21,22 The emerging field of implementation science seeks to address this problem and has categorized quality improvement interventions by their approach. These categories include audit and feedback, use of local opinion leaders, reminders, and educational sessions.23,24,25,26,27 Of these approaches, interventions that use opinion leaders and those that use audit and feedback have been observed to have the greatest association with physician behavior change. In a 2012 meta-analysis of 70 studies using the audit-and-feedback approach,23 the Cochrane Effective Practice and Organization of Care group described a 1% to 4% change in desired physician behavioral outcomes. Similarly, a 2011 Effective Practice and Organization of Care review of 18 cluster randomized clinical trials involving 296 hospitals27 found that using opinion leaders in an intervention results in 9% to 14% change in desired behavior.
We combined 2 established approaches of using audit and feedback and opinion leaders, which resulted in an immediate and sustained reduction in physician behavior around low-value care in MMS. Among outliers in the intervention group, 83% demonstrated a reduction in mean stages per case compared with 69% of outliers in the control group. There was a mean reduction in stages per case of 12.6% among outliers in the intervention group compared with 9.0% among outliers in the control group. Adjusted for physician baseline characteristics, the intervention was associated with an additional reduction of 0.14 stages per case (95% CI, −0.19 to −0.09; P < .001) among outliers. We posit that the improvement we observed in this study may be attributed to 4 elements characteristic of the intervention: (1) endorsement of an overuse metric by a clinical specialty society, (2) awareness and education by the specialty society around overuse, (3) the use of an achievable target, and (4) a confidential, nonpunitive audit-and-feedback comparison process.
The potential latent crossover effect observed among control group outliers may be attributed to 2 factors. First, the educational efforts and increased awareness efforts by the specialty society at the time of the intervention may have influenced surgeons in the control group. We believe that having peers from the specialty association prepare members in the intervention group to receive their notification reports was critical to surgeons being receptive to them. Second, clinicians in the intervention group may have had an impact on control group participants through personal conversations or a perception of an increased culture of accountability fostered by the initiative. Further, we observed a small yet statistically significant decrease in stages per case among inliers in the intervention group, suggesting that while outliers were more likely to demonstrate a greater reduction in mean stages per case, inliers may have also modified their behavior. Notably, even a slight decrease in stages per case in the inlier group could result in a significant cost savings, given that outlier volume only comprises a small percentage of all MMS procedures.
Physician demand for benchmarking data has been described to be strong. A 2011 study of primary physicians28 revealed that 95% of physicians surveyed believe there is excessive variation in care, and 76% are interested to learn how their individual practice patterns compare with their peers. We posit these views are shared by other specialist groups and hypothesize that additional iterations of Improving Wisely report dissemination could elicit even greater reductions in overuse.29
This study and its approach through the Improving Wisely model may have implications for broader quality improvement and waste reduction efforts in health care.13 We have identified critical factors that are likely needed for subsequent replication of this design. First, as a prerequisite, the intervention requires a well-respected peer physician leadership group to endorse a metric and the broader initiative a priori.30,31 In the present study, the physician engagement council and ACMS endorsed the overuse measure and broader program. Second, a clear achievable benchmark that is supported by peer leaders in the field is required, with a consensus threshold of what constitutes outlier practice patterns. In this study, experts created a consensus threshold of 2 SDs above the national mean. Third, the addition of educational and training resources as intervention components may be valuable to help outliers seeking tools to improve. Finally, the manner in which reports are messaged and delivered to the physician is important.32 Our methods were consistent with audit-and-feedback delivery best practices.33 Specifically, we ensured the reports were easily accessible, short in length and easy to interpret (ie, reduced cognitive load), and provided a combination of text and visual feedback.33 The reports were nonpunitive and confidential, provided clear targets for the desired behavior, were endorsed by respected leaders and governing bodies, provided repeated messaging and communication to targeted surgeons, and emphasized a behavior that was in the control of individual surgeons. This concept of autonomy is critical, with data suggesting that behavior change effects are stronger if an individual believes they are acting autonomously rather than simply responding to external influences.34,35,36,37
Limitations
This study has several limitations. First, while claims data are powerful for benchmarking and not subject to reporting bias, they do not provide detailed clinical data, such as patient or tumor characteristics. While our proposed metric adopted an appropriateness range based on expert consensus regarding a surgeons’ typical case mix, it is possible that some outliers’ high mean stages per case might be attributed to a high percentage of large and complex cases and/or high-risk patients in their practice. Despite this, the intervention group still showed a greater reduction in mean stages per case. Additional data sources would expand the analysis beyond the Medicare population. Detailed registries, such as the ACMS Clinical Data Registry (MohsAIQ) or the American Academy of Dermatology DataDerm Registry, may facilitate risk-adjusted comparisons and may enable individual physicians to examine their mean number of stages across all payers and compare themselves to benchmarks more easily in the future. Additional research will be needed to determine whether any unintended negative consequences resulted from the reduction in mean stages per case, such as tumor recurrence. Second, our study intervened on a single metric, which addressed only 1 aspect of appropriateness in MMS. However, additional metrics (eg, appropriate use of MMS for trunk and extremities, complications, and recurrences) can and should be explored.38,39 Further, we emphasize that the metric used to identify outliers was developed for the purpose of this study, in consideration of the claims data limitations. Future studies should revisit these benchmarks and provide adjustments for those clinicians with extensive case mix. Third, to facilitate intervention delivery, a nonrandomized design was used. While we adjusted for some physician characteristics, differences between surgeons in the intervention and control groups, including history of fellowship training and clinical practice setting, could lead to residual confounding. Future quality improvement initiatives should aim to target a representative sample of surgeons performing MMS. Fourth, it is unknown which educational resources were used by intervention group outliers and which resources may have contributed to the improvement observed. The sustainability and durable impact of this intervention should be explored in the long term.
Conclusions
We observed an immediate and sustained reduction in MMS procedures using an intervention that integrated the use of experts, a specialty association partnership, and educational resources along with individualized, confidential data report cards that provided a benchmark for peer comparison. The relatively low cost of this intervention relative to its resultant cost savings suggests that application to other areas of medicine could yield larger savings to the health care system. In an era where reducing low-value care is increasingly paramount given the escalating costs of health care, this intervention has the promise to provide a high return on investment.
eAppendix. Sample report.
References
- 1.Guy GPG Jr, Machlin SR, Ekwueme DU, Yabroff KR. Prevalence and costs of skin cancer treatment in the US, 2002-2006 and 2007-2011. Am J Prev Med. 2015;48(2):183-187. doi: 10.1016/j.amepre.2014.08.036 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Rogers HW, Weinstock MA, Harris AR, et al. . Incidence estimate of nonmelanoma skin cancer in the United States, 2006. Arch Dermatol. 2010;146(3):283-287. doi: 10.1001/archdermatol.2010.19 [DOI] [PubMed] [Google Scholar]
- 3.Leibovitch I, Huilgol SC, Selva D, Hill D, Richards S, Paver R. Cutaneous squamous cell carcinoma treated with Mohs micrographic surgery in Australia I: experience over 10 years. J Am Acad Dermatol. 2005;53(2):253-260. doi: 10.1016/j.jaad.2005.02.059 [DOI] [PubMed] [Google Scholar]
- 4.Smeets NWJ, Krekels GAM, Ostertag JU, et al. . Surgical excision vs Mohs’ micrographic surgery for basal-cell carcinoma of the face: randomised controlled trial. Lancet. 2004;364(9447):1766-1772. doi: 10.1016/S0140-6736(04)17399-6 [DOI] [PubMed] [Google Scholar]
- 5.Reeder VJ, Gustafson CJ, Mireku K, Davis SA, Feldman SR, Pearce DJ. Trends in Mohs surgery from 1995 to 2010: an analysis of nationally representative data. Dermatol Surg. 2015;41(3):397-403. doi: 10.1097/DSS.0000000000000285 [DOI] [PubMed] [Google Scholar]
- 6.Viola KV, Rezzadeh KS, Gonsalves L, et al. . National utilization patterns of Mohs micrographic surgery for invasive melanoma and melanoma in situ. J Am Acad Dermatol. 2015;72(6):1060-1065. doi: 10.1016/j.jaad.2015.02.1122 [DOI] [PubMed] [Google Scholar]
- 7.Mosterd K, Krekels GAM, Nieman FH, et al. . Surgical excision versus Mohs’ micrographic surgery for primary and recurrent basal-cell carcinoma of the face: a prospective randomised controlled trial with 5-years’ follow-up. Lancet Oncol. 2008;9(12):1149-1156. doi: 10.1016/S1470-2045(08)70260-2 [DOI] [PubMed] [Google Scholar]
- 8.Pugliano-Mauro M, Goldman G. Mohs surgery is effective for high-risk cutaneous squamous cell carcinoma. Dermatol Surg. 2010;36(10):1544-1553. doi: 10.1111/j.1524-4725.2010.01576.x [DOI] [PubMed] [Google Scholar]
- 9.Viola KV, Jhaveri MB, Soulos PR, et al. . Mohs micrographic surgery and surgical excision for nonmelanoma skin cancer treatment in the Medicare population. Arch Dermatol. 2012;148(4):473-477. doi: 10.1001/archdermatol.2011.2456 [DOI] [PubMed] [Google Scholar]
- 10.Wang DM, Morgan FC, Besaw RJ, Schmults CD. An ecological study of skin biopsies and skin cancer treatment procedures in the United States Medicare population, 2000 to 2015. J Am Acad Dermatol. 2018;78(1):47-53. doi: 10.1016/j.jaad.2017.09.031 [DOI] [PubMed] [Google Scholar]
- 11.Krishnan A, Xu T, Hutfless S, et al. ; American College of Mohs Surgery Improving Wisely Study Group . Outlier practice patterns in Mohs micrographic surgery: defining the problem and a proposed solution. JAMA Dermatol. 2017;153(6):565-570. doi: 10.1001/jamadermatol.2017.1450 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Sebaratnam DF, Choy B, Lee M, Paver R, Fernández Peñas P. Direct cost-analysis of Mohs micrographic surgery and traditional excision for basal cell carcinoma at initial margin clearance. Dermatol Surg. 2016;42(5):633-638. doi: 10.1097/DSS.0000000000000756 [DOI] [PubMed] [Google Scholar]
- 13.Makary MA, Mehta A, Xu T. Improving wisely using physician metrics. Am J Med Qual. 2018;33(1):103-105. doi: 10.1177/1062860617704504 [DOI] [PubMed] [Google Scholar]
- 14.Research Data Assistance Center CMS Virtual Research Data Center (VRDC). https://www.resdac.org/cms-virtual-research-data-center-vrdc. Accessed April 4, 2019.
- 15.Research Data Assistance Center Medicare Data on Provider Practice and Specialty (MD-PPAS): 2008-2016. https://www.resdac.org/cms-data/files/md-ppas. Accessed April 4, 2019.
- 16.US Centers for Medicare & Medicaid Services Physician Compare datasets. https://data.medicare.gov/data/physician-compare. Accessed April 4, 2019.
- 17.Loehrer AP, Chang DC, Scott JW, et al. . Association of the Affordable Care Act Medicaid expansion with access to and quality of care for surgical conditions. JAMA Surg. 2018;153(3):e175568. doi: 10.1001/jamasurg.2017.5568 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Lyu H, Xu T, Brotman D, et al. . Overtreatment in the United States. PLoS One. 2017;12(9):e0181970. doi: 10.1371/journal.pone.0181970 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Porter ME. What is value in health care? N Engl J Med. 2010;363(26):2477-2481. doi: 10.1056/NEJMp1011024 [DOI] [PubMed] [Google Scholar]
- 20.Emanuel EJ, Ubel PA, Kessler JB, et al. . Using behavioral economics to design physician incentives that deliver high-value care. Ann Intern Med. 2016;164(2):114-119. doi: 10.7326/M15-1330 [DOI] [PubMed] [Google Scholar]
- 21.Rosenberg A, Agiro A, Gottlieb M, et al. . Early trends among seven recommendations from the Choosing Wisely campaign. JAMA Intern Med. 2015;175(12):1913-1920. doi: 10.1001/jamainternmed.2015.5441 [DOI] [PubMed] [Google Scholar]
- 22.Mostofian F, Ruban C, Simunovic N, Bhandari M. Changing physician behavior: what works? Am J Manag Care. 2015;21(1):75-84. [PubMed] [Google Scholar]
- 23.Ivers N, Jamtvedt G, Flottorp S, et al. . Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;(6):CD000259. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Forsetlund L, Bjørndal A, Rashidian A, et al. . Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst Rev. 2009;(2):CD003030. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Flodgren G, Eccles MP, Shepperd S, Scott A, Parmelli E, Beyer FR. An overview of reviews evaluating the effectiveness of financial incentives in changing healthcare professional behaviours and patient outcomes. Cochrane Database Syst Rev. 2011;(7):CD009255. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Giguère A, Légaré F, Grimshaw J, et al. . Printed educational materials: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;10:CD004398. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Flodgren G, Parmelli E, Doumit G, et al. . Effectiveness of the use of local opinion leaders to promote evidence-based practice and improving patient outcomes. Cochrane Database Syst Rev. 2011;(8):CD000125. doi: 10.1002/14651858.CD000125.pub4 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Sirovich BE, Woloshin S, Schwartz LM. Too little? too much? primary care physicians’ views on US health care: a brief report. Arch Intern Med. 2011;171(17):1582-1585. doi: 10.1001/archinternmed.2011.437 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Graham ID, Logan J, Harrison MB, et al. . Lost in knowledge translation: time for a map? J Contin Educ Health Prof. 2006;26(1):13-24. doi: 10.1002/chp.47 [DOI] [PubMed] [Google Scholar]
- 30.Althabe F, Buekens P, Bergel E, et al. ; Guidelines Trial Group . A behavioral intervention to improve obstetrical care. N Engl J Med. 2008;358(18):1929-1940. doi: 10.1056/NEJMsa071456 [DOI] [PubMed] [Google Scholar]
- 31.Gagliardi AR, Kothari A, Graham ID. Research agenda for integrated knowledge translation (IKT) in healthcare: what we know and do not yet know. J Epidemiol Community Health. 2017;71(2):105-106. doi: 10.1136/jech-2016-207743 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Colquhoun HL, Squires JE, Kolehmainen N, Fraser C, Grimshaw JM. Methods for designing interventions to change healthcare professionals’ behaviour: a systematic review. Implement Sci. 2017;12(1):30. doi: 10.1186/s13012-017-0560-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Brehaut JC, Colquhoun HL, Eva KW, et al. . Practice feedback interventions: 15 suggestions for optimizing effectiveness. Ann Intern Med. 2016;164(6):435-441. doi: 10.7326/M15-2248 [DOI] [PubMed] [Google Scholar]
- 34.Sikkens JJ, van Agtmael MA, Peters EJG, et al. . Behavioral approach to appropriate antimicrobial prescribing in hospitals: the Dutch Unique Method for Antimicrobial Stewardship (DUMAS) participatory intervention study. JAMA Intern Med. 2017;177(8):1130-1138. doi: 10.1001/jamainternmed.2017.0946 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Deci EL, Ryan RM. The empirical exploration of intrinsic motivational processes. Adv Exp Soc Psychol. 1980;13:39-80. doi: 10.1016/S0065-2601(08)60130-6 [DOI] [Google Scholar]
- 36.McDermott L, Yardley L, Little P, Ashworth M, Gulliford M; eCRT Research Team . Developing a computer delivered, theory based intervention for guideline implementation in general practice. BMC Fam Pract. 2010;11:90. doi: 10.1186/1471-2296-11-90 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Cabana MD, Rand CS, Powe NR, et al. . Why don’t physicians follow clinical practice guidelines? a framework for improvement. JAMA. 1999;282(15):1458-1465. doi: 10.1001/jama.282.15.1458 [DOI] [PubMed] [Google Scholar]
- 38.Council ML, Alam M, Gloster HM Jr, et al. . Identifying and defining complications of dermatologic surgery to be tracked in the American College of Mohs Surgery (ACMS) Registry. J Am Acad Dermatol. 2016;74(4):739-745. [DOI] [PubMed] [Google Scholar]
- 39.Leitenberger JJ, Rogers H, Chapman JC, et al. . Defining recurrence of nonmelanoma skin cancer after Mohs micrographic surgery: report of the American College of Mohs Surgery Registry and Outcomes Committee. J Am Acad Dermatol. 2016;75(5):1022-1031. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eAppendix. Sample report.