Reducing Imaging Utilization in Primary Care Through Implementation of a Peer Comparison Dashboard

Abstract

Background

High clinical variation has been linked to decreased quality of care, increased costs, and decreased patient satisfaction. We present the implementation and analysis of a peer comparison intervention to reduce clinical variation within a large primary care network.

Objective

Evaluate existing variation in radiology ordering within a primary care network and determine whether peer comparison feedback reduces variation or changes practice patterns.

Design

Radiology ordering data was analyzed to evaluate baseline variation in imaging rates. A utilization dashboard was shared monthly with providers for a year, and imaging rates pre- and post-intervention were retrospectively analyzed.

Participants

Providers within the primary care network spanning 1,358,644 outpatient encounters and 159 providers over a 3-year period.

Interventions

The inclusion of radiology utilization data as part of a provider’s monthly quality and productivity dashboards. This information allows providers to compare their practice patterns with those of their colleagues.

Main Measures

We measured provider imaging rates, stratified by modality, as well as order variation over time.

Key Results

We observed significant variation in imaging rates among providers in the network, with the top decile ordering an average of 4.2 times more than the lowest decile in the two years prior to intervention. Provider experience and training were not significantly associated with imaging utilization. In the first year after sharing utilization data with providers, we saw a 17.3% decrease in median imaging rate (p < 0.001) and a 21.4% reduction in provider variation between top and bottom deciles. Median ordering rate for more costly cross-sectional imaging, including CT, MRI, and nuclear medicine studies, decreased by 30.4% (p < 0.001), 20.2% (p = 0.008), and 41.8% (p = 0.002), respectively.

Conclusions

Peer comparison feedback can shape provider imaging behavior even in the absence of targets or financial incentives. Peer comparison is a low-touch, low-cost intervention for influencing provider ordering and may have applicability in other clinical areas.

Electronic supplementary material

The online version of this article (10.1007/s11606-020-06164-8) contains supplementary material, which is available to authorized users.

INTRODUCTION

As numerous payers transition from fee-for-service to value-based-care models, healthcare systems across the USA are striving to identify ways to improve quality while simultaneously improving efficiency and reducing costs. One opportunity is to uncover and reduce unwanted variation, defined as “medical practice variation across regions or provider groups…that is not explained on the basis of illness, patient risk factors or patient preferences.”¹ Such variation in practice patterns can lead to overutilization, adverse outcomes, and unnecessary costs.^{2, 3}

Studies have demonstrated significant unwanted variation in imaging utilization.^4–6 Between 2000 and 2007, Medicare experienced a rapid rise imaging costs.^{4, 7} Much of this increase was due to the widespread adoption of cross-sectional imaging modalities like CT and MRI.^{4, 7} However, findings from the US Government Accountability Office (GAO) show that imaging technologies are not evenly distributed, and significant regional variation exists within the USA.⁴ In 2005, the Medicare Payment Advisory Commission (MedPAC) reported “overuse, underuse, and misuse” of imaging services⁷ and decried “variation in the number of imaging services” as “the most significant reason to be concerned about potential overuse.”⁷ Evaluating regional differences in Medicare per-beneficiary imaging expenditures reveals that some regions spend nearly eight times more than others.⁴ Yet, outcomes do not reflect a clear benefit from higher rates of imaging utilization, suggesting that this unwanted variation may be a prime opportunity to reduce overuse and associated costs.⁷

As the medical community has recognized rapid increases in the deployment and costs of imaging technology, various attempts have been made to reduce utilization.⁸ Changes to Medicare payment policy have limited reimbursement for these services and bent the growth curve over the past 10 years.⁴ In addition, several professional societies have established guidelines intended to educate providers. The American College of Radiology established “appropriateness criteria” designed to help determine whether a particular imaging test is suitable for the clinical scenario at hand.^{8, 9} In addition, the Choosing Wisely campaign has outlined certain clinical scenarios in which imaging tests would be considered low-value and should be avoided.¹⁰ Furthermore, clinical decision support tools integrated directly into electronic health records can guide radiology decision-making at the point of care^11–13 and help implement guidelines.¹⁴ However, such systems are complex to design and expensive to implement. In addition, they may also create barriers to efficient care and can be disruptive to provider workflow.^{4, 15} Lastly, few have focused on benchmarking performance and presenting providers with a transparent view of how their utilization practices might compare with their peers.

In this study, we examine the variation in provider diagnostic imaging orders within a large academic primary care network and present the results of a low-tech, peer comparison provider feedback intervention designed to increase transparency and reduce unwanted variation in outpatient radiology orders, while improving standardization in clinical practice. The theoretical groundings of this approach arise from Social Comparison Theory,^{16, 17} specifically the notion that individuals evaluate and compare themselves with peers, thereby modifying their own behaviors. More recently, several theories and hypotheses around feedback-driven interventions in healthcare have been synthesized and formalized under the Clinical Performance Feedback Intervention Theory (CP-FIT), positing that the most effective interventions require limited work from providers, align with providers’ beliefs about how care should be delivered, and directly support clinical behaviors.¹⁸ Specific to radiology utilization, feedback-driven approaches have been used to improve CT usage among emergency department physicians.¹⁹ However, to the best of our knowledge, no similar approaches have been applied to reducing variation in outpatient radiology utilization among primary care providers.

METHODS

Duke Primary Care (DPC) is a network of 29 primary care practices in the Piedmont region of North Carolina. With 215 providers, DPC provides care to patients across more than 8 counties in the central part of the state. Every primary care provider in the DPC network receives a monthly dashboard that includes clinical quality and patient satisfaction metrics. Providers receive information about their own performance, in addition to practice and network averages, allowing for peer comparison. Because the metrics on this report are tied to compensation, the information has good penetration and is widely reviewed by our provider network. As such, we anticipated this dashboard could also be an avenue through which we could share utilization metrics.

Our hypothesis was that by providing increased transparency about practice patterns in a low-tech, low-touch manner, we might influence provider behavior. The radiology utilization metric reports a provider’s imaging rate, calculated as diagnostic radiology orders normalized to 1000 encounters. For example, if a provider ordered 300 studies and had 2000 patient encounters within a year, their rate would be 150 orders per 1000 encounters. The metric is calculated using data from the prior 12 months and reported monthly. Providers receive their own data as well as peer comparison data including the average imaging rate of their clinic site and the entire network. In addition, providers can review their individual historical trends from the previous year. To date, no numeric targets or financial incentives have been linked to imaging utilization. An example of a provider’s utilization dashboard can be seen in the appendix. This project was submitted for review and determined to be IRB-exempt as quality improvement.

In an effort to capture the variation in use of diagnostic imaging, we excluded screening studies such as mammography, bone densitometry, and low-dose chest CT from this metric. Radiology utilization data was shared with providers starting in July 2018 (the beginning of Duke’s 2019 fiscal year). We calculated a baseline comparison using data from two years prior to inclusion of the imaging metric on the dashboard. For presentation purposes, we refer to the 12-month period beginning July 2018 as “2019” (and similarly for the other two years studied). Because this analysis was designed to measure provider intent, we employed an intention-to-treat analysis. Providers without three full years of data were excluded from analysis. Analysis was restricted to family medicine and internal medicine attending providers. GME trainees and urgent care and pediatric providers were excluded.

Statistical analysis was done in Python version 3.7.4 utilizing Jupyter notebook version 6.0.1. Python packages used for analysis and visualization include Pandas version 0.25.1, Scipy 1.3.1, Numpy 1.17.2, Matplotlib 3.1.1, and Seaborn 0.9.0. First, radiology order data from July 2016 through June 2019 was normalized using encounter volume as described above. Imaging data were additionally stratified by modality (CT, MRI, nuclear medicine, PET, ultrasound, and X-ray). For year-over-year comparisons, Kruskal-Wallis H-testing was used to compare medians given non-normality in our data sets, as confirmed by Shapiro-Wilk testing. Regression analysis to evaluate effects of provider-level factors (years of experience, years at Duke, provider type) was done using Scipy and the Wald test was performed on the slope to determine significance.

RESULTS

Our data set spans 1,358,644 encounters with 159 providers. There were 179,211 radiology orders on 86,754 unique patients in the primary care network (summarized in Table 1). Providers ordered a median of 132.7 studies per 1000 encounters in the two years prior to intervention. However, there was significant variation among providers with the top decile ordering an average of 257.2 tests per 1000 patients, 4.2 times more than the lowest decile which ordered an average of 61.7 tests per 1000 patients. Between the two years prior to implementation of the dashboard, there were no significant changes in the median number of orders across all modalities. Prior to intervention, X-ray and ultrasound studies were the most common imaging modality ordered (51.5% and 24.2% of total, respectively) while cross-sectional imaging including CT and MRI composed a smaller fraction (15.0% and 7.8%, respectively). PET and other nuclear medicine studies only accounted for 1.4% of total orders.

Table 1.

Summary Statistics of Imaging Orders in a Large Primary Care Network

	2017	2018	2019
Number of providers	159
Total encounters	440,960	472,331	445,353
Unique patients*	36,367	38,680	35,135
Unique radiology orders	62,077	65,325	51,809
Patient demographics*
Percent female	65.1%	64.5%	64.0%
Average age	57.7	56.7	57.7
Non-Hispanic Black	25.1%	25.0%	24.6%
Non-Hispanic White	63.4%	62.9%	63.4%
Hispanic	2.2%	2.2%	2.3%
Asian	4.0%	4.1%	4.3%
Other/unknown	5.3%	5.8%	5.4%

*Patients for whom imaging studies were ordered

After incorporating radiology utilization data into the provider dashboard, we observed significant decreases in both imaging orders and overall variation (Fig. 1 and Table 2). Median orders decreased by 17.3% across all modalities (excluding PET, which had a median of zero pre- and post-intervention) compared with the two years prior to intervention (Table 2). Cross-sectional imaging modalities (CT, MRI, and nuclear medicine studies) had more pronounced median decreases of 30.4% (p < 0.001), 20.2% (p = 0.008), and 41.8% (p = 0.002), respectively. Dramatic decreases in ordering behavior were seen within the first month of implementation of the dashboard (Fig. 2) and were maintained throughout the study period. Additionally, the dashboard seemed to decrease imaging orders even among lower utilizers (Fig. 3). In addition, variance between top and bottom deciles decreased after the intervention. At baseline, providers in the top decile ordered 4.2-fold more tests than those in the bottom decile. After implementation of the dashboard intervention, the top decile ordered 3.3-fold more tests than the bottom decile. Similar results were obtained when performing a sensitivity analysis including all providers who entered or left the network during the study period (Supplemental Table 1).

Figure 1.

Annual imaging volume stratified by imaging modality. In the two years prior to intervention, there were no significant changes in utilization overall and by modality. After intervention, significant decreases were seen.

Table 2.

Intervention Analysis Highlighting Significant Decreases in Radiology Orders as well as Reduction in Variation Between Highest and Lowest Utilizers. All Order Numbers Are Normalized to 1000 Patient Encounters

	2017/2018	2019	Percent change (p value)
Median orders	132.7	109.7	− 17.3% (p < 0.001)
Top decile average	257.2	187.4	− 27.6%
Bottom decile average	61.7	56.9	− 7.8%
Top/bottom decile ratio	4.2	3.3	− 21.4%
Median orders by modality
X-ray	71.2	59.1	− 17.0% (p = 0.008)
Ultrasound	33.8	27.5	− 18.6% (p < 0.001)
CT	18.2	12.7	− 30.4% (p < 0.001)
MRI	9.8	7.8	− 20.2% (p = 0.008)
Nuclear medicine	0.9	0.5	− 41.9% (p = 0.002)
PET	0.0	0.0	---

Figure 2.

Statistical control chart by month highlights rapid effect of intervention. Order utilization across the network by month shows effect the month immediately after the dashboard information was shared with providers (vertical black line). Average utilization and two standard deviation range represented with the horizontal black and gray dotted lines, respectively.

Figure 3.

High provider-level variation exists within our network and decreases after intervention were seen throughout all levels of baseline utilization. Each column of dots in the top figure represents one provider with pre-intervention utilization in red and post-intervention in blue. Horizontal dotted lines represent pre- and post-intervention medians across providers pre- and post-intervention. The bottom figure is the corresponding percent change after intervention for each provider (dotted line represents average change).

We performed regression analysis to determine if provider characteristics predicted utilization patterns. For 150 providers with available data, years of experience, years at Duke, and provider type (MD/DO vs NP/PA) did not demonstrate any significant association with imaging rate (p = 0.41, p = 0.24, p = 0.79, respectively). When evaluating the race/ethnicity of patients for whom tests were ordered, we observed no differences pre- and post-intervention, suggesting that reductions in utilization remained equally distributed across groups.

DISCUSSION

Initiatives to reduce unwanted variation may provide an important opportunity to improve quality, while simultaneously reducing overuse and related costs. In this study, we demonstrate that a low-tech peer comparison and provider feedback intervention led to significant reductions in imaging utilization that were immediate and sustained over time.

While the intervention reduced utilization and variation, there may have been some unintended consequences. Though we hoped to see a reduction in variation after sharing imaging data with providers, we did not expect to see a reduction in utilization across the board. Regardless of whether they were high or low utilizers at baseline, exposure to their utilization rates led most providers to order fewer tests. The clinical appropriateness of utilization changes was not evaluated and remains a question for further study.

Our analysis has some notable limitations. Because our intervention included an entire practice network, we did not have a control group and thus could not determine what other external factors may have impacted imaging utilization during this same time period. In addition, because we lacked data on patients for whom imaging tests were not ordered, we were unable to analyze whether other variables (patient comorbidity, race, sex, socioeconomic status) may have predicted baseline imaging use or change in utilization after intervention. While full comorbidity data (and data on patients for whom imaging tests were not ordered) would have helped determine whether patient complexity had any association with ordering patterns, such data was unavailable for this analysis. Better understanding of patient-level factors, especially in the context of equity and clinical quality, is an opportunity for future study.

This study reveals some important lessons. Interventions to improve quality and produce value need not be complex or onerous for providers. While point-of-care tools can help guide decision-making, regular exposure to low-tech provider dashboards may also have the power to reduce unwanted variation. Such dashboards may reveal previously unrecognized patterns of utilization and cause providers to consider their own behavior in comparison with their colleagues. Our findings suggest that by increasing transparency about provider utilization patterns and providing the opportunity for peer comparison, health systems may be able to influence provider behavior even in the absence of discrete targets or financial incentives. While this study focused on imaging utilization, the findings have important implications for other utilization as well, especially in areas where unwanted variation or overutilization may exist, such as lab and referral utilization, as well as total costs of care.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they do not have a conflict of interest.