Audit-and-feedback - Optimizing a strategy to reduce low-value care

Despite a decade of attention to the issue of low-value care in the United States and internationally, there has been little change in provision of nonbeneficial health services [1]. The study by O’Connor and colleagues (2) in this issue of JAMA provides important high-quality evidence that a national audit and feedback program might help reduce widespread use of low-value testing.

The authors performed a cluster-randomized trial that included 3,819 Australian general practitioners (GPs) from 2,271 practices who had high request rates for 11 musculoskeletal imaging tests that often represent low-value care: lumbosacral spine X-ray and computed tomography (CT); cervical spine X-ray, CT and magnetic resonance imaging (MRI); shoulder X-ray and ultrasound; hip X-ray and ultrasound; knee X-ray; and ankle/hind foot ultrasound. Participant physicians were randomized to receive an audit and feedback intervention in the form of a mailed letter from the Chief Medical Officer of the Australian Government Department of Health and Aged Care, or no intervention. The letters used administrative data from the prior year to summarize individual physician’s rate of ordering overall and for each of the 11 individual tests, providing the number of requests per 1000 consultations, and the physician’s’ percentile ranking compared with peers. A 2×2 factorial design enabled novel evaluation of 2 variations of the letter, one related to frequency (a one-time letter compared to 2 letters one year apart) and a second related to visual design (with or without “enhancement” via yellow highlighting of tests for which rates were higher than 80% of peers).

The study reports a significant relative reduction of about 10% in ordering rates of the 11 imaging tests overall in intervention groups at 12 months compared with in the control group (adjusted mean, 27.7 [95% CI, 27.5–28.0] tests ordered per 1,000 consultations in the audit-feedback intervention groups versus 30.4 [95% CI, 29.8–30.9] in the no intervention control group; adjusted mean difference −2.66 [95% CI −3.24 to −2.07]; p<0.001). In prespecified secondary outcome analyses, there were additional marginal but statistically significant reductions from the second letter (with the difference seen at 18 months) and enhanced display interventions compared with a single letter without enhancement. Of note, the initial intervention occurred in late 2019; soon thereafter, rates of imaging decreased precipitously in all groups during the first few months of the COVID-19 pandemic (when stay-at-home orders were active in Australia), but then rebounded, with intervention groups maintaining consistently lower testing rates after return to prepandemic baseline imaging utilization rates.

This study is an important addition to the literature on reducing low-value care. While audit and feedback has been used to improve care by reducing errors of omission, the effects on errors of commission (and thus on reducing low-value care), have been inconsistent [3, 4]. It is therefore important to consider the mechanism for clinician behavior change via audit and feedback, which is empirically attributed to the behavioral economic theory of relative social ranking (also known as social desirability bias) [4,5]. specifically targeting outliers (i.e. high image-ordering physicians) leverages this phenomenon—most people, including physicians, do not want to behave too differently from their peers—and was the approach selected by O’Connor et al in including the 20% highest-ordering practitioners. The observed results were consistent with a prior randomized trial that reported a reduction in unnecessary antibiotic use among high-usage GPs in the UK randomized to receive “social norm” feedback [6] but differed from the results of a randomized trial in England and Scotland of musculoskeletal imaging audit and feedback that targeted GP practices and reported no effect [7], suggesting that providing personalized information to clinicians may be more effective than providing information to practices.

In addition to optimizing the target audience, information design and delivery has been hypothesized to influence efficacy of audit and feedback [5], which was confirmed by the current study. First, feedback was provided by a person of authority [4], in this case a person of very high national authority in a single-payer national health system, and comparator groups were physicians in the same geographic region. Importantly, randomization related to both the frequency (the intervention “dosing”) and the visual display of information affected rates of image ordering. Still, even 1 audit and feedback exposure without enhanced visual display resulted in a significant reduction in test ordering.

The simple nature of the intervention design in this study is notable, given prior evidence that multimodal approaches are generally more effective at reducing low-value care [8, 9]. For example, the randomized trial that reported that feedback reduced antibiotic prescribing among UK GPs added a patient educational brochure [7], and a randomized trial that reported improved intensive care unit pain management with audit and feedback randomized inclusion of a multicomponent clinician “toolbox” to facilitate implementation that increased efficacy of the feedback intervention [6, 10]. In contrast, the current study simple included links to general physician educational resources.

Perhaps most important, the targeting of multiple diagnostic services, without associated standards for determining appropriateness, is a unique aspect of the study by O’Connor et al. Letters to intervention groups provided only high-level guidance, stating that “most people who present with musculoskeletal pain in the absence of worrying features do not need any imaging as it does not help management” and suggesting physicians limit imaging to situations in which “imaging will change your patient’s treatment plan and improve their health outcomes.” In contrast, most prior studies of audit and feedback to reduce low-value care have concentrated on a single service in a specific clinical setting, generally focusing on optimizing management of a specific condition and often citing standards designated by the ABIM Foundation’s Choosing Wisely Campaign or national guidelines [11]; for example, there have been multiple studies of audit and feedback to reduce routine MRI use for evaluation of low back pain, a specific test used for a specific condition [12].

Perhaps providing feedback that includes multiple services across contexts is more likely to meaningfully influence care, given that the majority of medical decision-making occurs in situations that are outside specific guideline recommendations [13]. Adding to the potential effectiveness of this approach, the current study demonstrates an effect on utilization beyond the targeted imaging tests, specifically reducing rates of ankle x-ray, which were not included in the feedback report but could have been performed in lieu of ankle ultrasound (which was included in reports). Ankle x-ray was therefore a potential balancing measure (ie, an internal “control” measure) designed to detect unanticipated increases in use of nonreported tests resulting from the intervention. However, use of ankle x-ray also declined by a rate that was among the largest in the study, second only to hip x-ray.

An approach targeting multiple procedures without specific recommendations does, however, carry potential risks. First, physicians may discount the credibility of the feedback data, especially in the absence of case-mix adjustments, although absence of adjustment in the current study did not appear to impair efficacy. Second, necessary diagnostic tests may be erroneously omitted. Physicians are generally poor at estimating the likelihood of diagnoses and generally overvalue testing [14], so they may be unable to identify and eliminate only unneeded testing. For situations in which clinicians may not recognize services as low value, experts have called for interventions that fill knowledge gaps along with performance feedback [3], which was not done in the study by O’Connor et al and would be difficult when multiple tests are targeted. Inclusion of patient-reported clinical outcomes, such as functional mobility, would ensure that diagnostic imaging omission was safe, although collection of patient-reported outcomes in routine practice is not yet standard. Another approach would be to evaluate appropriateness of testing decisions among a small random sample of patients via chart review, which has been undertaken in other studies [7] (but not in the study by O’Connor et al) and which adds complexity and cost that can limit scalability.

The study by O’Connor et al in this issue of JAMA represents a rigorous and large-scale evaluation and suggests that use of an audit and feedback intervention targeting a specific clinician audience with a relatively nonspecific intervention can reduce low-value care, and contributes valuable insights to feedback design and implementation. Attention to optimal visual display seems warranted, and multiple intervention time points are probably best, although total duration needed for sustainability is uncertain. The observed 10% reduction in test ordering, while modest, could have significant benefit across a health system when delivered in areas where overuse of low-value care is frequent, although attention should be paid to patient outcomes.

In addition, generalizability beyond a universal health insurance system, such as the one in Australia, needs to be explored, particularly regarding financial incentives and the effects on clinician reimbursement, as well as the influence of the medical malpractice environment and the practice of “defensive medicine.” Furthermore, although physicians were the target audience in this Australian study, evaluation among nonphysician clinicians, who conduct a significant portion of primary care in the United States, is warranted. Despite these caveats, audit and feedback regarding multiple low-value services delivered to select clinicians has great potential to improve the value proposition and should be pursued.