Key Points
Question
Can an individualized audit and feedback intervention reduce musculoskeletal diagnostic imaging usage among high-requesting general practitioners in Australia?
Findings
In this cluster-randomized clinical trial that included 3819 Australian general practitioners from 2271 practices, an intervention that involved written individualized feedback and ordering rate comparisons with peers significantly decreased the overall rate of requests for any of 11 targeted imaging tests over 12 months compared with a control group that did not receive feedback (adjusted mean rate, 27.7 vs 30.4 requests per 1000 consultations; adjusted mean difference, −2.66 per 1000 consultations).
Meaning
Among Australian high-requesting general practitioners, an individualized audit and feedback intervention significantly decreased the rate of targeted musculoskeletal imaging tests ordered over 12 months.
Abstract
Importance
Audit and feedback can improve professional practice, but few trials have evaluated its effectiveness in reducing potential overuse of musculoskeletal diagnostic imaging in general practice.
Objective
To evaluate the effectiveness of audit and feedback for reducing musculoskeletal imaging by high-requesting Australian general practitioners (GPs).
Design, Setting, and Participants
This factorial cluster-randomized clinical trial included 2271 general practices with at least 1 GP who was in the top 20% of referrers for 11 imaging tests (of the lumbosacral or cervical spine, shoulder, hip, knee, and ankle/hind foot) and for at least 4 individual tests between January and December 2018. Only high-requesting GPs within participating practices were included. The trial was conducted between November 2019 and May 2021, with final follow-up on May 8, 2021.
Interventions
Eligible practices were randomized in a 1:1:1:1:1 ratio to 1 of 4 different individualized written audit and feedback interventions (n = 3055 GPs) that varied factorially by (1) frequency of feedback (once vs twice) and (2) visual display (standard vs enhanced display highlighting highly requested tests) or to a control condition of no intervention (n = 764 GPs). Participants were not masked.
Main Outcomes and Measures
The primary outcome was the overall rate of requests for the 11 targeted imaging tests per 1000 patient consultations over 12 months, assessed using routinely collected administrative data. Primary analyses included all randomized GPs who had at least 1 patient consultation during the study period and were performed by statisticians masked to group allocation.
Results
A total of 3819 high-requesting GPs from 2271 practices were randomized, and 3660 GPs (95.8%; n = 727 control, n = 2933 intervention) were included in the primary analysis. Audit and feedback led to a statistically significant reduction in the overall rate of imaging requests per 1000 consultations compared with control over 12 months (adjusted mean, 27.7 [95% CI, 27.5-28.0] vs 30.4 [95% CI, 29.8-30.9], respectively; adjusted mean difference, −2.66 [95% CI, −3.24 to −2.07]; P < .001).
Conclusions and Relevance
Among Australian general practitioners known to frequently request musculoskeletal diagnostic imaging, an individualized audit and feedback intervention, compared with no intervention, significantly decreased the rate of targeted musculoskeletal imaging tests ordered over 12 months.
Trial Registration
ANZCTR Identifier: ACTRN12619001503112
This randomized trial assesses the effect of individualized written audit and feedback interventions vs no intervention on rate of orders of musculoskeletal imaging tests among general practitioners who frequently request diagnostic imaging.
Introduction
Regional musculoskeletal conditions such as low back, neck, shoulder, hip, knee, and foot pain were common and significant contributors to global disability between 1990 and 2019.1,2 Evidence-based primary care guidelines for many of these conditions consistently recommend against early diagnostic imaging in the absence of features suggestive of serious and/or specific underlying cause or unexplained symptom progression.3 Unwarranted imaging does not benefit patients, may cause harm and/or lead to further unnecessary care, and wastes health resources that could be better used elsewhere.4 Yet overuse of diagnostic imaging by general practitioners (GPs) for people with musculoskeletal conditions has persisted.5,6,7,8,9
Audit and feedback is a widely used strategy for improving professional practice.10 The ability of audit and feedback used at scale to reduce musculoskeletal imaging by GPs is unclear, as available trials have included small numbers of practices and GPs.11,12,13,14,15 The primary objective of this trial was to evaluate the effectiveness of audit and feedback used at a national level for reducing requests for 11 commonly overused musculoskeletal diagnostic imaging tests by high-requesting Australian GPs compared with a control condition of no intervention. A secondary objective was to evaluate which forms of audit and feedback were most effective.
Methods
Trial Design
This was a 5-group factorial cluster-randomized clinical trial conducted within the context of Australian general clinical practice. A cluster design was used to minimize crossover of intervention effects within practices and a no-intervention control condition tested the effectiveness of any form of audit and feedback. A 2 × 2 factorial design was used to simultaneously assess the effect of 2 factors: (1) frequency of feedback delivery (once vs twice) and (2) enhanced visual display (no vs yes). The trial used routinely collected Australian Medicare Benefits Schedule (MBS) data to identify the study population, apply eligibility criteria, generate individualized feedback for the interventions, and source trial outcomes. Medicare is Australia’s universal health insurance scheme that guarantees all Australian residents access to a range of health care services by physicians and other health care professionals at low or no cost. The Bond University Human Research Ethics Committee (LA03323) approved the study and waived participant consent, as the research carried no more than low risk to participants, there was adequate protection of participants’ privacy, and obtaining consent would create a burden greater than the intervention itself.16 Participants were not aware they were in a trial. The trial protocol and statistical analysis plan are available in Supplement 1.
Participants
General practices were included if (1) at least 1 GP within the practice was in the top 20% of Australian GP referrers for 11 targeted musculoskeletal imaging tests overall, and for at least 4 individual tests, within the 12-month period from January 1 to December 31, 2018, according to Australian MBS administrative data and (2) the practice was located in an Australian state or territory. The MBS database is comprehensive and records all diagnostic imaging that has been requested by GPs, rendered by a radiologist, and claimed from Medicare. The 11 targeted imaging tests were lumbosacral spine x-ray and computed tomography (CT); cervical spine x-ray, CT, and magnetic resonance imaging (MRI); shoulder x-ray and ultrasound; hip x-ray and ultrasound; knee x-ray; and ankle/hind foot ultrasound. These imaging tests were selected for inclusion by the research team in consultation with the Australian Government Department of Health and Aged Care medical advisors and reflect tests that are generally considered overused in primary care. Only high-requesting GPs (ie, those in the top 20% of GP referrers for the targeted tests overall and for at least 4 individual tests within the specified 12-month period) practicing within one of the participating practices were included.
General practitioners were excluded if they (1) did not make any requests for the 11 targeted tests between January 1 and December 31, 2018; (2) had less than 1000 patient consultations over the specified 12-month period; (3) participated in user testing that informed intervention development; or (4) participated in an Australian government compliance activity between January 1, 2018, and October 25, 2019.
Randomization and Masking
General practices with at least 1 eligible high-requesting GP, clustered based on exact-matched primary practice addresses, were randomized on a single occasion to 1 of 4 interventions or a no-intervention control group in a 1:1:1:1:1 ratio by a statistician (B.M.) using the “cluster_ra” procedure in R statistical software, stratified by geographic region (metropolitan and regional/rural/remote).
Due to the nature of the audit and feedback intervention, participating GPs were not masked to group allocation; however, the risk of bias was considered to be minimal as GPs were not aware of the variations of audit and feedback being tested or outcome measures and analytical approach. The statistical analysis plan was developed by the senior trial statistician (L.B.), who was masked to group allocation (see Supplement 1).17 Analyses were independently conducted by 2 statisticians (B.M. and A.G.) using randomly shuffled group allocations. Real allocations were only revealed once analyses were completed and agreement between statisticians was reached.
Interventions
General practitioners allocated to the control group did not receive any audit and feedback on their musculoskeletal imaging requests.
General practitioners allocated to 1 of 4 intervention groups received individualized written audit and feedback on their imaging requesting rates in a letter from the chief medical officer of Australia (eAppendix 1 in Supplement 2). The audit and feedback provided to all intervention groups was consistent in content and length other than frequency and visual display, the potential effect modifiers evaluated in the trial. The letters were delivered by mail and graphically displayed recipients’ overall request rate for 11 targeted imaging tests, and tabular data for individual test request rates, per 1000 consultations, each compared with the median rate of their peers in the same geographic stratum. The letter described the benefits of tackling imaging overuse, provided links to educational resources, and encouraged recipients to reflect on the feedback and limit imaging requests to situations in which it had a reasonable likelihood of changing a patient’s management plan.
Factor 1: Frequency of Feedback
Two groups received feedback on 1 occasion (November 8, 2019), the current standard approach of the Australian government, based on GPs’ January to December 2018 audit data. Another 2 groups received feedback on 2 occasions (November 8, 2019, and November 9, 2020) to investigate the potential for increased effects; delivery of the second feedback letter provided updated requesting data for the period November 8, 2019, to March 7, 2020, a shorter period given the effect of COVID-19 on health care practice.
Factor 2: Inclusion of Enhanced Visual Display
Two groups received standard feedback display (request rates for all 11 tests without highlighting imaging types with rates >80% of peers), while 2 groups received enhanced display (use of yellow shading to direct recipients’ attention to imaging for which ordering rates were >80% of peers, suggesting behavior change may be required) (eAppendix 1 in Supplement 2).
The interventions were user tested and refined with GPs prior to evaluation.
Outcomes
The primary outcome was the overall rate of requests for any of the 11 targeted musculoskeletal diagnostic imaging tests for each GP over 12 months, with rates over 6 and 18 months considered secondary outcomes. The request rates for the 11 individual imaging tests plus ankle x-ray for each GP over 6, 12, and 18 months were secondary outcomes. Ankle x-ray was not targeted by the audit and feedback but was considered a possible substitute for ankle/hind foot ultrasound. All outcomes are expressed as requests per 1000 patient consultations.
The following baseline data were also collected: GPs’ age, sex, years of practice, geographical location of primary practice (metropolitan vs regional/rural/remote), and rates of musculoskeletal diagnostic imaging requests for primary and secondary outcomes at baseline (November 1, 2018, to October 31, 2019).
Power Analysis
A total of 2271 unique general practices across Australia were identified as eligible for inclusion in the study. Using all eligible practices and a 4-to-1 randomization ratio, the sample size provided greater than 90% power to detect a mean difference as small as 1.1 requests per 1000 consultations in the rate of imaging requests. This calculation assumed a standard deviation of 7 as observed in a previous data extract and a 2-sided type I error rate of 5%. It conservatively assumed that each cluster would be of size 1 (ie, a single GP per practice), thus negating the need to adjust for intracluster correlation. Given that several practices included 2 or more GPs with cluster sizes ranging from 1 to 12 (median, 1 [IQR, 1-2], equal between groups), this approach guaranteed at least 90% power regardless of the intracluster correlation.
This sample size also provided greater than 90% power to detect a difference of 1.1 requests per 1000 consultations in the mean rate of imaging requests between those receiving feedback on 1 vs 2 occasions and between those receiving feedback with vs without enhanced feedback display, assuming a 2-sided α = .025 for each of these secondary comparisons to control for multiplicity.
Statistical Analysis
Based on the trial protocol and statistical analysis plan, the main analysis was to include all randomized GPs and analyze them according to the group to which they were randomized, regardless of the number of patient consultations conducted. However, during data analysis, it was discovered that a limited number of GPs did not have any patient consultations during the follow-up period and could not be included in the main analysis. Therefore, the primary analysis set consisted of all GPs with at least 1 patient consultation over the relevant analysis period. Given that MBS data systematically capture all consultations, no missing outcome data were assumed.
The main analysis compared the mean rate of diagnostic imaging requests per 1000 patient consultations between the 4 audit and feedback intervention groups, considered as 1 group, with the control group. Data were aggregated at the GP level and analyzed using multilevel mixed-effects linear regression adjusted for the baseline imaging rate of each GP and regional/rural/remote location. All assumptions for multivariable regression analysis, including lack of multicollinearity, independence of observations, and multivariable normality, were met. To remove skewness and potential heteroskedasticity, a natural log transformation was applied to the 6-, 12-, and 18-month rates as well as to the baseline rate included as a covariate. Resulting estimates and confidence intervals were back-transformed to the original scale. Clustering of GPs by practice was accounted for by including a random intercept by practice. The effect of the intervention was estimated as the adjusted mean difference in the overall rate of musculoskeletal imaging requests between the intervention and the control groups with its 95% CI. The same mixed-effects linear regression model was used to estimate the effect of the number of audit and feedback occasions (1 vs 2) and the effect of the enhanced feedback display (no vs yes) by adding terms for each intervention as well as their interaction. The effect of each intervention was estimated using adjusted mean differences and 97.5% CIs to account for multiple comparisons. A similar approach was used to analyze the secondary outcomes. All analyses were repeated using an adjusted model, controlling for GP demographic characteristics (age, sex, and years of practice) and baseline volume of imaging requests (eAppendix 2 in Supplement 2).
Shifts in percentage ranks over time for the main comparison (intervention vs control) and according to the 4 intervention groups for the primary outcome were investigated using Sankey diagrams using 4 time points (baseline, month 6, month 12, and month 18), categorizing ranks into 3 categories (lowest: <80th percentile; medium: 80th to <90th percentile; high: ≥90th percentile).
A limited number of prespecified subgroup analyses were conducted to identify potential differences in intervention effects for factors documented in the analysis plan, such as geographical location (metropolitan vs regional/rural/remote), years practicing as a GP (0 to <5 years, 5 to <10 years, 10 to <15 years, or 15 or more years), sex, baseline overall percentile ranking (80th to <90th or 90th to 99th), and baseline volume of imaging tests with requesting percentile at least 80th (4-6 vs 7-11). Subgroup analyses were conducted by adding the subgroup variable of interest as well as its interaction with the randomized group to the main analysis model.
Additionally, prespecified sensitivity analyses were conducted to test whether the effect of the intervention remained stable under various conditions: (1) including only those who completed the trial per protocol, ie, excluding those whose mail returned to sender unopened or who requested not to receive a second audit and feedback letter and (2) including only those who had at least 1500 patient consultations within the 18-month trial duration. An additional post hoc sensitivity analysis was undertaken excluding practices randomized into 2 study groups due to an administrative error in practice locality classification, to assess the effect of practices misclassified in both strata (ie, metropolitan, nonmetropolitan [regional/rural/remote]) leading to GPs within the same practice having a chance of being randomized to different study groups. A post hoc estimate of the reduction in diagnostic imaging requests as a result of the audit and feedback intervention was calculated with its 95% CI based on the difference between the observed number of imaging requests made by GPs allocated to audit and feedback and expected number of imaging requests made by these GPs in absence of the intervention, based on regression model estimates. For the main analysis, all practices were included with GPs analyzed according to their allocated group.
All analyses were performed using Stata version 16 (StataCorp LLC) and R version 4.0.0 (R Foundation). Level of significance was set at P < .05 using a 2-sided test for the primary comparison and a Bonferroni-corrected P < .025 to compare 1 vs 2 feedback occasions and feedback with vs without enhanced display. We did not formally adjust for multiplicity across secondary outcomes; therefore, secondary outcome results should be interpreted as exploratory and as assisting the interpretation of the primary outcome results.
Results
On November 8, 2019, 3819 high-requesting GPs from 2271 unique practices were simultaneously randomized to control or active intervention groups. Due to an administrative error in general practice locality classification prior to randomization, 395 practices were stratified into both metropolitan and nonmetropolitan areas. This resulted in some GPs from the same practice having a chance of being randomized to different study groups. Of the 395 affected practices, 307 were randomized to 2 groups (111 practices to both control and intervention, with potential for crossover of 336 GPs in these practices; and 196 practices to 2 different intervention groups, with potential for crossover of 647 GPs); 88 practices were randomized into the same group.
Overall, 533 practices (n = 764 GPs) were randomized to control and 2133 practices (n = 3055 GPs) were randomized to active interventions (Figure 1). For the main analysis, all practices were included with GPs analyzed according to their allocated group. Requesting rates for 159 GPs (4.2%) who did not have any patient consultations during the trial period could not be calculated. The main analysis therefore included 3660 GPs from 2578 practices. The geographic location, sex, and baseline imaging request rates between GPs included in and excluded from the main analysis were comparable, yet excluded GPs were older and had more years in practice than included GPs (eTable 1 in Supplement 2). A per-protocol analysis included 3370 GPs from 2423 practices and excluded the 290 GPs who did not receive the allocated intervention (ranging from 37 [4.9%] to 112 [14.7%] per group) because their first audit and feedback letter was returned to sender unopened (n = 206) or there was no follow-up (n = 84), in addition to the 159 GPs with no patient consultations.
Figure 1. Participant Flow.
GP indicates general practitioner.
aThe trial captured all eligible high-requesting GPs and practices. The median practice size was 1 (IQR, 1-2) for all groups. The 2666 randomized clusters are more than the number of practices reported in the statistical analysis plan (n = 2271) and include 395 practices randomized more than once due to an address classification administrative error that occurred in July 2019, prior to randomization, and that was detected during the data cleaning. See text and eAppendix 3 in Supplement 2 for details.
bPractices were excluded if all GPs from the practice were excluded. A patient consultation was defined as professional attendance by a GP for which benefits are paid under the Australian Medicare Benefits Schedule.
cGeneral practitioners were excluded from the per-protocol analysis if they (1) had their audit and feedback reports returned undelivered; (2) were allocated to receive audit and feedback on a second occasion but requested not to receive a second report (and were not mailed the second report); (3) were allocated to receive audit and feedback on a second occasion but did not make any musculoskeletal imaging requests during the audit period of November 8, 2019, to March 7, 2020; or (4) were allocated to receive audit and feedback on a second occasion but had fewer than 333 patient consultations during the 4-month audit period of November 8, 2019, to March 7, 2020, so calculating their percentile for rate of imaging requests was not reliable and therefore it was not reported on the second occasion.
The characteristics of high-requesting GPs included in the trial were similar across all study groups at baseline (Table 1). The median overall rate of musculoskeletal imaging at baseline was 32.3 (IQR, 27.3-38.1) per 1000 patient consultations and was similar across groups. Request rates for individual imaging tests were also similar across groups at baseline (eTable 2 in Supplement 2).
Table 1. Baseline Participant Characteristics.
| Characteristics | Interventiona | Control (n = 764)a | ||||
|---|---|---|---|---|---|---|
| Standard display | Enhanced display | All interventions (n = 3055) | ||||
| Delivered once (n = 754) | Delivered twice (n = 760) | Delivered once (n = 780) | Delivered twice (n = 761) | |||
| GP age, y, No. (%) | n = 763 | |||||
| <40 | 66 (8.8) | 76 (10.0) | 62 (7.9) | 73 (9.6) | 277 (9.1) | 57 (7.5) |
| 40-49 | 175 (23.2) | 165 (21.7) | 146 (18.7) | 164 (21.6) | 650 (21.3) | 164 (21.5) |
| 50-59 | 220 (29.2) | 215 (28.3) | 252 (32.3) | 220 (28.9) | 907 (29.7) | 221 (29.0) |
| 60-69 | 217 (28.8) | 215 (28.3) | 247 (31.7) | 221 (29.0) | 900 (29.5) | 242 (31.7) |
| ≥70 | 76 (10.1) | 89 (11.7) | 73 (9.4) | 83 (10.9) | 321 (10.5) | 79 (10.4) |
| Sex, No. (%) | n = 759 | n = 779 | n = 760 | n = 3052 | ||
| Female | 299 (39.7) | 303 (39.9) | 281 (36.0) | 323 (42.4) | 1206 (39.5) | 283 (37.0) |
| Male | 455 (60.3) | 456 (60.0) | 498 (63.8) | 437 (57.4) | 1846 (60.4) | 481 (63.0) |
| Metropolitan geographical region, No. (%) | 572 (75.9) | 570 (75.0) | 595 (76.3) | 574 (75.4) | 2311 (75.6) | 586 (76.7) |
| State/territory, No. (%) | ||||||
| Queensland | 211 (28.0) | 225 (29.6) | 234 (30.0) | 256 (33.6) | 926 (30.3) | 220 (28.8) |
| New South Wales | 180 (23.9) | 205 (27.0) | 212 (27.2) | 192 (25.2) | 789 (25.8) | 223 (29.2) |
| Victoria | 222 (29.4) | 172 (22.6) | 185 (23.7) | 166 (21.8) | 745 (24.4) | 178 (23.3) |
| Western Australia | 82 (10.9) | 94 (12.4) | 73 (9.4) | 82 (10.8) | 331 (10.8) | 94 (12.3) |
| South Australia | 44 (5.8) | 39 (5.1) | 53 (6.8) | 36 (4.7) | 172 (5.6) | 34 (4.5) |
| Tasmania | 7 (0.9) | 15 (2.0) | 15 (1.9) | 19 (2.5) | 56 (1.8) | 12 (1.6) |
| Australian Capital Territory | 7 (0.9) | 9 (1.2) | 4 (0.5) | 7 (0.9) | 27 (0.9) | 2 (0.3) |
| Northern Territory | 1 (0.1) | 1 (0.1) | 4 (0.5) | 3 (0.4) | 9 (0.3) | 1 (0.1) |
| Total patient consultations, median (IQR)b,c | 4566 (2940-6607) | 4452 (2947-6422) | 4722 (2853-6679) | 4556 (2673-6742) | 4587 (2822-6605) | 4237 (2845-6532) |
| Diagnostic imaging tests, median (IQR)b | 148 (88-219) | 152 (91-219) | 151 (89-224) | 146 (86-226) | 149 (89-222) | 144 (87.5-213.5) |
Abbreviation: GP, general practitioner.
Denominators are as shown in the column headings unless otherwise stated.
Baseline time period: November 1, 2018, to October 31, 2019.
Refers to any professional attendance by a GP for which benefits are paid under the Australian Medicare Benefits Schedule.
Primary Outcome
The overall rate of musculoskeletal imaging requests by GPs following the audit and feedback intervention was statistically significantly lower than in the control group over 12 months despite substantial reductions in imaging requests across all groups during the COVID-19 pandemic (Figure 2 and Table 2; eFigure 1 in Supplement 2). The mean overall rate of imaging requests per 1000 consultations was 27.7 (95% CI, 27.5-28.0) with the intervention vs 30.4 (95% CI, 29.8-30.9) in the control group, for an adjusted mean difference of −2.66 requests (95% CI, −3.24 to −2.07; P < .001).
Figure 2. Overall Rates of Musculoskeletal Diagnostic Imaging Requests and Changes in Percentile Rank Over Time.
Corresponding model-based mean cumulative rates with standard deviations from baseline to 6, 12, and 18 months are reported in Table 2. See eFigure 1 in Supplement 2 for rates with 95% confidence intervals.
Table 2. Overall Request Rates for Trial Comparisons.
| Observed rates per 1000 patient consultations, median (IQR)a | Model-based estimatesa | |||||
|---|---|---|---|---|---|---|
| Mean per 1000 patient consultations (95% CI) | Adjusted mean difference (95% CI or 97.5% CI) | P value | ||||
| Overall | ||||||
| Intervention (n = 2933) | Control (n = 727) | Intervention (n = 2933) | Control (n = 727) | |||
| Baseline | 32.4 (27.3-38.0) | 31.8 (27.5-38.2) | ||||
| 6 mo | 24.0 (19.0-30.5) | 26.8 (21.7-33.1) | 24.8 (24.5-25.0) | 27.6 (27.1-28.1) | −2.81 (−3.36 to −2.26) | <.001 |
| 12 mob | 27.2 (21.8-33.4) | 29.4 (24.4-36.2) | 27.7 (27.5-28.0) | 30.4 (29.8-30.9) | −2.66 (−3.24 to −2.07) | <.001 |
| 18 mo | 26.2 (21.2-31.6) | 27.8 (23.1-33.8) | 26.2 (26.0-26.4) | 28.3 (27.8-28.8) | −2.11 (−2.63 to −1.58) | <.001 |
| Frequency of feedback | ||||||
| Twice (n = 1461) | Once (n = 1472) | Twice (n = 1461) | Once (n = 1472) | |||
| Baseline | 32.4 (27.3-38.1) | 32.5 (27.2-37.9) | ||||
| 6 mo | 24.1 (19.0-30.6) | 23.9 (18.9-30.3) | 24.9 (24.4-25.3) | 24.7 (24.3-25.1) | 0.14 (−0.04 to 0.70)c | .58 |
| 12 mob | 27.4 (21.7-33.3) | 27.2 (21.9-33.5) | 27.7 (27.3-28.4) | 27.7 (27.3-28.1) | 0.02 (−0.57 to 0.61)c | .93 |
| 18 mo | 26.1 (20.8-31.1) | 26.3 (21.4-31.9) | 25.8 (25.4-26.2) | 26.6 (26.2-27.0) | −0.79 (−1.30 to −0.26)c | .001 |
| Visual display | ||||||
| Enhanced (n = 1475) | Standard (n = 1458) | Enhanced (n = 1475) | Standard (n = 1458) | |||
| Baseline | 26.1 (21.0-31.0) | 32.5 (27.1-37.9) | ||||
| 6 mo | 23.7 (18.6-30.3) | 24.4 (19.3-30.7) | 24.4 (24.0-25.1) | 25.2 (24.8-25.5) | −0.79 (−1.35 to −0.24)c | .001 |
| 12 mob | 26.9 (21.3-33.0) | 27.6 (22.2-33.8) | 27.4 (26.9-27.8) | 28.1 (27.7-28.5) | −0.70 (−1.28 to −0.11)c | .008 |
| 18 mo | 26.1 (21.0-31.1) | 26.3 (21.2-32.0) | 26.0 (25.6-26.4) | 26.4 (25.9-26.7) | −0.36 (−0.90 to 0.17)c | .13 |
Rates are cumulative from the date of first feedback sent to general practitioners (GPs) (November 8, 2019) until the specified time point. Data aggregated at the GP level were analyzed using multilevel mixed-effects linear regression adjusted for the baseline imaging rate of each GP and regional/rural/remote location. See text for details.
Primary outcome.
97.5% CI.
Secondary Outcomes
Audit and feedback also statistically significantly reduced the overall request rate compared with control over 6 months (mean, 24.8 [95% CI, 24.5-25.0] vs 27.6 [95% CI, 27.1-28.1] requests per 1000 consultations, respectively; adjusted mean difference, −2.81 [95% CI, −3.36 to −2.26]; P < .001) and 18 months (mean, 26.2 [95% CI, 26.0-26.4] vs 28.3 [95% CI, 27.8-28.8] requests per 1000 consultations, respectively; adjusted mean difference, −2.11 [95% CI, −2.63 to −1.58]; P < .001). An estimated 47 318 (95% CI, 35 562 to 59 074) fewer diagnostic imaging tests were requested due to any audit and feedback intervention over the study period.
Audit and feedback on 2 occasions (at baseline and 12 months later) statistically significantly reduced the overall request rate per 1000 consultations at 18 months vs audit and feedback on 1 occasion (mean, 25.8 [95% CI, 25.4-26.2] vs 26.6 [95% CI, 26.2-27.0], respectively; adjusted mean difference, −0.79 [97.5% CI, −1.30 to −0.26]; P < .001).
Enhanced display significantly reduced the overall request rate per 1000 consultations vs standard display over 6 months (mean, 24.4 [95% CI, 24.0-25.1] vs 25.2 [95% CI, 24.8-25.5], respectively; adjusted mean difference, −0.79 [97.5% CI, −1.35 to −0.24]; P = .001) and 12 months (mean, 27.4 [95% CI, 26.9-27.8] vs 28.1 [95% CI, 27.7-28.5], respectively; adjusted mean difference, −0.70 [97.5% CI, −1.28 to −0.11]; P = .008) but not over 18 months (mean, 26.0 [95% CI, 25.6-26.4] vs 26.4 [95% CI, 25.9-26.7], respectively; adjusted mean difference, −0.36 [97.5% CI, −0.90 to 0.17]; P = .13). There was no statistically significant interaction between number of occasions and type of display (P = .51). Relative rate reductions in overall diagnostic imaging rates at each time point are presented in eFigure 2 in Supplement 2 (see also eTable 3 in Supplement 2).
Audit and feedback statistically significantly reduced the request rate per 1000 consultations of all individual imaging tests compared with control over the first 6 months, with the greatest reductions observed for hip x-ray (adjusted mean difference, −0.65 [95% CI, −0.86 to −0.45]; P < .001), ankle x-ray (adjusted mean difference, −0.62 [95% CI, −0.78 to −0.46]; P < .001) and lumbosacral spine CT (adjusted mean difference, −0.47 [95% CI, −0.58 to −0.37]; P < .001) (eTable 4 in Supplement 2). Request rates remained statistically significantly lower compared with control for all but 2 individual imaging tests (lumbosacral and cervical spine x-ray) over 12 and 18 months (Table 3; eTable 4 in Supplement 2). Audit and feedback on 2 occasions statistically significantly reduced request rates of all but 3 individual imaging tests (lumbosacral spine, knee, and ankle x-ray) vs once-only delivery over 18 months (eTable 5 in Supplement 2). Enhanced display statistically significantly reduced request rates of all but 4 individual tests (lumbosacral spine x-ray and CT, cervical spine x-ray and MRI) vs standard display over 6 months. However, there was no statistically significant difference in request rates between groups over 12 and 18 months except for hip x-ray and shoulder, hip, and ankle ultrasound (eTable 6 in Supplement 2). The intracluster correlation for the primary outcome was 0.10 (95% CI, 0.06-0.16) and ranged between 0 and 0.15 for other outcomes (eTable 7 in Supplement 2).
Table 3. Request Rates for Individual Imaging Tests for the Main Comparisona.
| Observed rates per 1000 patient consultations, median (IQR)b | Model-based estimatesb,c | |||||
|---|---|---|---|---|---|---|
| Mean per 1000 patient consultations (95% CI) | Adjusted mean difference (95% CI) | P value | ||||
| Intervention (n = 2933) | Control (n = 727) | Intervention (n = 2933) | Control (n = 727) | |||
| Lumbosacral spine | ||||||
| X-ray | ||||||
| Baseline | 1.6 (0.7-2.8) | 1.6 (0.7-1.7) | .13 | |||
| 12 mo | 1.2 (0.5-2.3) | 1.2 (0.5-2.3) | 1.41 (1.37-1.44) | 1.47 (1.40-1.54) | −0.06 (−0.14 to 0.02) | |
| Computed tomography | ||||||
| Baseline | 1.6 (0.7-2.8) | 3.6 (2.3-5.2) | <.001 | |||
| 12 mo | 2.9 (1.7-4.5) | 3.6 (2.1-5.0) | 3.01 (2.96-3.06) | 3.46 (3.35-3.56) | −0.45 (−0.56 to −0.33) | |
| Shoulder | ||||||
| X-ray | ||||||
| Baseline | 4.2 (2.8-5.7) | 4.2 (2.8-5.6) | <.001 | |||
| 12 mo | 3.5 (2.5-4.9) | 3.8 (2.5-5.1) | 3.38 (3.33-3.43) | 3.71 (3.60-3.82) | −0.33 (−0.44 to −0.22) | |
| Ultrasound | ||||||
| Baseline | 5.1 (3.9-6.5) | 5.2 (4.1-6.5) | <.001 | |||
| 12 mo | 4.4 (3.2-5.8) | 4.8 (3.5-6.2) | 4.27 (4.23-4.31) | 4.61 (4.53-4.69) | −0.35 (−0.43 to −0.26) | |
| Cervical spine | ||||||
| X-ray | ||||||
| Baseline | 0.8 (0.3-1.6) | 0.8 (0.3-1.5) | .16 | |||
| 12 mo | 0.6 (0.2-1.2) | 0.5 (0.2-1.4) | 0.74 (0.72-0.76) | 0.77 (0.74-0.81) | −0.03 (−0.07 to 0.01) | |
| Computed tomography | ||||||
| Baseline | 0.9 (0.4-1.7) | 0.9 (0.4-1.8) | <.001 | |||
| 12 mo | 0.7 (0.3-1.4) | 0.9 (0.3-1.7) | 0.89 (0.87-0.91) | 1.02 (0.96-1.07) | −0.12 (−0.17 to −0.07) | |
| Magnetic resonance imaging | ||||||
| Baseline | 0.4 (0.0-1.2) | 0.5 (0.0-1.3) | .05 | |||
| 12 mo | 0.4 (0.0-1.0) | 0.5 (0.0-1.2) | 0.62 (0.60-0.64) | 0.67 (0.63-0.71) | −0.05 (−0.09 to −0.0003) | |
| Knee | ||||||
| X-ray | ||||||
| Baseline | 5.2 (3.8-6.7) | 5.0 (3.7-6.6) | <.001 | |||
| 12 mo | 4.4 (3.0-5.7) | 4.6 (3.3-5.9) | 4.21 (4.15-4.26) | 4.44 (4.33-4.57) | −0.24 (−0.36 to −0.12) | |
| Hip | ||||||
| X-ray | ||||||
| Baseline | 3.9 (2.7-5.3) | 3.9 (2.7-5.4) | <.001 | |||
| 12 mo | 3.2 (2.2-4.5) | 3.7 (2.6-4.8) | 3.25 (3.20-3.29) | 3.60 (3.51-3.69) | −0.35 (−0.45 to −0.25) | |
| Ultrasound | ||||||
| Baseline | 3.2 (2.0-4.5) | 3.2 (2.0-4.5) | <.001 | |||
| 12 mo | 2.7 (1.6-3.9) | 2.9 (1.8-4.3) | 2.62 (2.58-2.65) | 2.90 (2.82-2.98) | −0.29 (−0.37 to −0.20) | |
| Ankle | ||||||
| X-rayd | ||||||
| Baseline | 4.0 (2.9-5.4) | 3.9 (3.0-5.4) | <.001 | |||
| 12 mo | 5.2 (3.8-7.1) | 5.5 (4.1-7.5) | 5.37 (5.31-5.43) | 5.98 (5.84-6.12) | −0.44 (−0.58 to −0.29) | |
| Ultrasound | ||||||
| Baseline | 1.6 (0.9-2.6) | 1.6 (0.8-2.6) | <.001 | |||
| 12 mo | 1.3 (0.6-2.1) | 1.4 (0.7-2.4) | 1.42 (1.40-1.44) | 1.54 (1.50-1.59) | −0.12 (−0.17 to −0.07) | |
See eTable 4 in Supplement 2 for request rates for individual imaging tests for the main comparison at all time points.
Imaging request rates for each general practitioner (GP) were calculated as the total number of diagnostic imaging requests from baseline to the end of the follow-up period (ie, 6, 12, and 18 months) divided by the total number of consultations over the corresponding period.
Data aggregated at the GP level were analyzed using multilevel mixed-effects linear regression adjusted for the baseline imaging rate of each GP and locality. See text for details.
Ankle x-ray was not targeted by the feedback intervention but was considered to be a potential substitute for ankle/hind foot ultrasound, so it was included as a secondary outcome to check for switching.
Within the first 6 months of the intervention, 46.9% of GPs in the intervention group (767/1634) and 33.0% of those in the control group (133/403) shifted from the highest percentile ranking (90th to 99th) to medium (80th to <90th) or lowest (<80th) (P < .001) (Figure 2), while 57.8% of GPs in the intervention group (623/1078) and 41.5% in the control group (120/289) shifted from medium to lowest percentile ranking. At 12 months, the proportion of highest percentile ranking GPs in the intervention group was statistically significantly lower compared with control (39.9% [1169/2933] vs 46.4% [337/727]; P = .001). The difference between groups increased following the second mailing, reaching 36.6% (1073/2933) for intervention and 49.7% (361/727) for control (P < .001). Percentile rankings for the 4 intervention groups at each time point are presented in eFigure 3 in Supplement 2.
Subgroup and Sensitivity Analyses
Subgroup analyses showed no statistically significant heterogeneity in intervention effects based on geographical location, years practicing as a GP, sex, and baseline overall percentile ranking and baseline volume of imaging tests with requesting percentile at least 80th (4-6 vs 7-11) (eFigure 4 in Supplement 2).
Prespecified sensitivity analyses showed that there was little change in effect estimates for the main and adjusted models after restricting analyses to per-protocol or to active GPs (ie, excluding those with <1500 consultations during follow-up). Post hoc sensitivity analyses excluding practices inadvertently randomized into 2 study groups due to an administrative practice locality classification error also led to consistent results (eTable 8 in Supplement 2).
Discussion
Among Australian GPs known to frequently request musculoskeletal diagnostic imaging, an individualized audit and feedback intervention, compared with no intervention, significantly decreased the rate of targeted musculoskeletal imaging tests ordered over 12 months in the context of substantial reductions in requests across all groups during the COVID-19 pandemic.
Audit and feedback also led to statistically significant reductions in overall imaging requests over 6 and 18 months, as well as for most individual imaging tests over 6, 12, and 18 months, compared with a control condition of no audit and feedback. A second round of feedback to GPs at 12 months led to a further modest statistically significant reduction in overall imaging requests over 18 months compared with once-only delivery. Enhanced display calling GPs’ attention to ordering rates greater than 80% of peers’ rates also resulted in further modest statistically significant reductions in overall imaging over 6 and 12 months but not 18 months.
Early in the COVID-19 pandemic, governments in Australia implemented a range of restrictions on health care delivery, greater access to telehealth services, and, in some jurisdictions, stay-at-home orders to limit virus transmission and protect hospital capacity. A substantial decline in imaging services was observed in the first few months of the pandemic and across all groups within the first 6 months of the trial but rebounded thereafter.
The favorable effects observed in the current study are at odds with results from 3 of 5 previous trials of audit and feedback for musculoskeletal diagnostic imaging.11,12,13 Robling et al13 found that providing audit and feedback accompanied by guidelines on 1 occasion to GPs in the UK did not lead to statistically significant improvements in guideline-concordant knee and lumbar spine MRI ordering compared with guideline dissemination alone. Kerry et al12 showed that providing audit and feedback to GPs in the UK on 1 occasion accompanied by guidelines on hip, knee, spine, chest, skull, and sinus imaging did not lead to a statistically significant reduction in the overall request rate compared with a no-intervention control (mean difference, 10; 95% CI, −1 to 21). Eccles et al11 showed that providing audit and feedback on 2 occasions over 12 months did not lead to statistically significant reductions in the volume or appropriateness of lumbar spine and knee x-ray requests by GPs in England and Scotland compared with guideline dissemination alone (absolute change in radiograph requests per 1000 patients, −0.07 [95% CI, −1.3 to 0.9] for lumbar spine and 0.04; [95% CI, −0.95 to 1.03] for knee x-ray requests; concordance odds ratios for lumbar spine, 0.75 [95% CI, 0.52-1.07] and knee, 0.82 [95% CI, 0.50-1.33]). However, the audit and feedback intervention evaluated in these trials was provided at the level of the GP practice, rather than individualized to the GP recipient as implemented in the current study, so this could explain differences in the observed effects. Furthermore, the interventions in the current study were user tested with GPs prior to evaluation and sent to GPs known to frequently request musculoskeletal diagnostic imaging by the most senior medical advisor in the Australian Government Department of Health and Aged Care, so this may have also contributed to the beneficial effects observed.
Two previous trials have reported benefit with more intensive audit and feedback interventions compared with control. Winkens et al15 showed that providing audit and feedback on requesting of cervical spine, lumbar spine, pelvis, knee, and ankle imaging to 79 family physicians in the Netherlands on 5 occasions over a 2-year period led to a statistically significant reduction in the proportion of inappropriate requests compared with control (effect estimate not reported). Schectman et al14 showed that providing audit and feedback as part of a multicomponent quality improvement strategy (including small-group education, guideline dissemination, patient educational materials, and individual follow-up visits) on 2 occasions led to a small, statistically significant improvement in guideline-concordant lumbar spine imaging compared with a no-intervention control (5.4% increase with intervention vs 2.7% reduction with control; P = .05).
The finding that providing audit and feedback on 2 occasions led to a statistically significant reduction in overall imaging requests compared with once-only delivery is in keeping with broader empirical and theoretical knowledge about audit and feedback10,18 and with theories suggesting audit and feedback is more likely to be effective when delivered on multiple occasions.19,20,21 To our knowledge, no prior randomized trial has compared different ways of displaying feedback to direct recipients’ attention to where action is needed.21 This study provides evidence in support of a simple design enhancement (ie, use of yellow shading to direct recipients’ attention to their high imaging requesting) that led to modest, statistically significant reductions in overall and individual imaging request rates compared with standard display, especially within the first 6 months of feedback.
It is unclear to what extent the observed effect of audit and feedback will be maintained over time and whether 1 or more additional interventions will be needed to sustain the effect. It is also not known whether audit and feedback interventions like that used in this study influence other related imaging behaviors or low-value care practices. Future research could explore these issues as well as investigate how audit and feedback can be further optimized to reduce overuse of musculoskeletal imaging.
Limitations
This study has several limitations. First, the study assessed volume and not appropriateness of imaging, so it is uncertain whether high request rates represent inappropriate use in all instances. Second, while audit and feedback letters were mailed to GPs’ nominated mailing address and it was noted when letters were returned to sender unopened, delivery of the letters was not tracked and no attempts were made to locate alternative mailing addresses for returned letters. It is also not known whether the letters that were not returned were opened and read. Therefore, estimates from the main analysis are likely to be conservative. Third, during data cleaning, an error was identified in the locality classification of some practices, resulting in some GPs located within the same practice being randomized into different groups. While it is possible that GPs within these practices discussed their feedback, sensitivity analysis suggests that exclusion of affected practices resulted in minimal overall changes to inference.
Conclusions
Among Australian general practitioners known to frequently request musculoskeletal diagnostic imaging, an individualized audit and feedback intervention, compared with no intervention, significantly decreased the rate of targeted musculoskeletal imaging tests ordered over 12 months.
Trial Protocol and Statistical Analysis Plan
eAppendix 1. Template for Intervention Description and Replication (TIDieR) Checklist
eAppendix 2. Data Analysis Notes
eAppendix 3. Clarification of Administrative Error
eFigure 1. Overall Rates of Musculoskeletal Diagnostic Imaging Requests Over Time With 95% CI
eFigure 2. Relative Reduction in Overall Rate of Musculoskeletal Diagnostic Imaging by Group Over 6, 12 And 18 Months
eFigure 3. Sankey Diagrams of Percentage Rank of Overall Musculoskeletal Diagnostic Imaging Requests Per 1,000 Consultations for 4 Active Intervention Groups: Standard Delivered Once (Top Left), Enhanced Delivered Once (Top Right), Standard Delivered Twice (Bottom Left), Enhanced Delivered Twice (Bottom Right)
eFigure 4. Forest Plot for Subgroup Analysis of Primary Outcome
eTable 1. Comparison Between Included and Excluded GPs
eTable 2. Baseline Rates of Individual Imaging Tests Across Groups
eTable 3. Estimated Rate Differences Between Each Intervention Group and Control at 3 Time Points
eTable 4. Request Rates for Individual Imaging Tests for the Main Comparison at All Timepoints
eTable 5. Request Rates for Individual Imaging Tests According to Frequency of Feedback (Twice vs. Once)
eTable 6. Request Rates for Individual Imaging Tests According to Display (Enhanced vs. Standard)
eTable 7. ICC (95% CI) for All Outcomes
eTable 8. Sensitivity Analyses of the Primary Outcome for the Main Comparison – Model-Based Estimates
Data Sharing Statement
References
- 1.GBD 2019 Diseases and Injuries Collaborators . Global burden of 369 diseases and injuries in 204 countries and territories, 1990-2019: a systematic analysis for the Global Burden of Disease Study 2019. Lancet. 2020;396(10258):1204-1222. doi: 10.1016/S0140-6736(20)30925-9 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Safiri S, Kolahi AA, Cross M, et al. Prevalence, deaths, and disability-adjusted life years due to musculoskeletal disorders for 195 countries and territories 1990-2017. Arthritis Rheumatol. 2021;73(4):702-714. doi: 10.1002/art.41571 [DOI] [PubMed] [Google Scholar]
- 3.Lin I, Wiles L, Waller R, et al. What does best practice care for musculoskeletal pain look like? eleven consistent recommendations from high-quality clinical practice guidelines: systematic review. Br J Sports Med. 2020;54(2):79-86. doi: 10.1136/bjsports-2018-099878 [DOI] [PubMed] [Google Scholar]
- 4.Maher CG, O’Keeffe M, Buchbinder R, Harris IA. Musculoskeletal healthcare: have we over-egged the pudding? Int J Rheum Dis. 2019;22(11):1957-1960. doi: 10.1111/1756-185X.13710 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Downie A, Hancock M, Jenkins H, et al. How common is imaging for low back pain in primary and emergency care? systematic review and meta-analysis of over 4 million imaging requests across 21 years. Br J Sports Med. 2020;54(11):642-651. doi: 10.1136/bjsports-2018-100087 [DOI] [PubMed] [Google Scholar]
- 6.Foster NE, Anema JR, Cherkin D, et al. ; Lancet Low Back Pain Series Working Group . Prevention and treatment of low back pain: evidence, challenges, and promising directions. Lancet. 2018;391(10137):2368-2383. doi: 10.1016/S0140-6736(18)30489-6 [DOI] [PubMed] [Google Scholar]
- 7.Jenkins HJ, Downie AS, Maher CG, Moloney NA, Magnussen JS, Hancock MJ. Imaging for low back pain: is clinical use consistent with guidelines? a systematic review and meta-analysis. Spine J. 2018;18(12):2266-2277. doi: 10.1016/j.spinee.2018.05.004 [DOI] [PubMed] [Google Scholar]
- 8.Morrisroe K, Nakayama A, Soon J, et al. EVOLVE: the Australian Rheumatology Association’s “top five” list of investigations and interventions doctors and patients should question. Intern Med J. 2018;48(2):135-143. doi: 10.1111/imj.13654 [DOI] [PubMed] [Google Scholar]
- 9.Müskens J, Kool RJ, van Dulmen SA, Westert GP. Overuse of diagnostic testing in healthcare: a systematic review. BMJ Qual Saf. 2022;31(1):54-63. doi: 10.1136/bmjqs-2020-012576 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;6(6):CD000259. doi: 10.1002/14651858.CD000259.pub3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Eccles M, Steen N, Grimshaw J, et al. Effect of audit and feedback, and reminder messages on primary-care radiology referrals: a randomised trial. Lancet. 2001;357(9266):1406-1409. doi: 10.1016/S0140-6736(00)04564-5 [DOI] [PubMed] [Google Scholar]
- 12.Kerry S, Oakeshott P, Dundas D, Williams J. Influence of postal distribution of the Royal College of Radiologists’ guidelines, together with feedback on radiological referral rates, on x-ray referrals from general practice: a randomized controlled trial. Fam Pract. 2000;17(1):46-52. doi: 10.1093/fampra/17.1.46 [DOI] [PubMed] [Google Scholar]
- 13.Robling MR, Houston HLA, Kinnersley P, et al. General practitioners’ use of magnetic resonance imaging: an open randomized trial comparing telephone and written requests and an open randomized controlled trial of different methods of local guideline dissemination. Clin Radiol. 2002;57(5):402-407. doi: 10.1053/crad.2001.0864 [DOI] [PubMed] [Google Scholar]
- 14.Schectman JM, Schroth WS, Verme D, Voss JD. Randomized controlled trial of education and feedback for implementation of guidelines for acute low back pain. J Gen Intern Med. 2003;18(10):773-780. doi: 10.1046/j.1525-1497.2003.10205.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Winkens RAG, Pop P, Bugter-Maessen AMA, et al. Randomised controlled trial of routine individual feedback to improve rationality and reduce numbers of test requests. Lancet. 1995;345(8948):498-502. doi: 10.1016/S0140-6736(95)90588-X [DOI] [PubMed] [Google Scholar]
- 16.National Health and Medical Research Council, Australian Research Council, Universities Australia . National Statement on Ethical Conduct in Human Research. National Health and Medical Research Council; 2007. Updated 2018.
- 17.Billot L, Maguire B, Schram D, et al. Evaluating an audit and feedback intervention for reducing musculoskeletal diagnostic imaging requests by Australian general practitioners: statistical analysis plan for a partial cluster randomised trial. OSF Preprints. Preprint posted August 6, 2021. doi: 10.31219/osf.io/prnd2 [DOI]
- 18.Ivers NM, Grimshaw JM, Jamtvedt G, et al. Growing literature, stagnant science? systematic review, meta-regression and cumulative analysis of audit and feedback interventions in health care. J Gen Intern Med. 2014;29(11):1534-1541. doi: 10.1007/s11606-014-2913-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Brown B, Gude WT, Blakeman T, et al. Clinical performance feedback intervention theory (CP-FIT): a new theory for designing, implementing, and evaluating feedback in health care based on a systematic review and meta-synthesis of qualitative research. Implement Sci. 2019;14(1):40. doi: 10.1186/s13012-019-0883-5 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Carver CS, Scheier MF. Control theory: a useful conceptual framework for personality-social, clinical, and health psychology. Psychol Bull. 1982;92(1):111-135. doi: 10.1037/0033-2909.92.1.111 [DOI] [PubMed] [Google Scholar]
- 21.Brehaut JC, Colquhoun HL, Eva KW, et al. Practice feedback interventions: 15 suggestions for optimizing effectiveness. Ann Intern Med. 2016;164(6):435-441. doi: 10.7326/M15-2248 [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Trial Protocol and Statistical Analysis Plan
eAppendix 1. Template for Intervention Description and Replication (TIDieR) Checklist
eAppendix 2. Data Analysis Notes
eAppendix 3. Clarification of Administrative Error
eFigure 1. Overall Rates of Musculoskeletal Diagnostic Imaging Requests Over Time With 95% CI
eFigure 2. Relative Reduction in Overall Rate of Musculoskeletal Diagnostic Imaging by Group Over 6, 12 And 18 Months
eFigure 3. Sankey Diagrams of Percentage Rank of Overall Musculoskeletal Diagnostic Imaging Requests Per 1,000 Consultations for 4 Active Intervention Groups: Standard Delivered Once (Top Left), Enhanced Delivered Once (Top Right), Standard Delivered Twice (Bottom Left), Enhanced Delivered Twice (Bottom Right)
eFigure 4. Forest Plot for Subgroup Analysis of Primary Outcome
eTable 1. Comparison Between Included and Excluded GPs
eTable 2. Baseline Rates of Individual Imaging Tests Across Groups
eTable 3. Estimated Rate Differences Between Each Intervention Group and Control at 3 Time Points
eTable 4. Request Rates for Individual Imaging Tests for the Main Comparison at All Timepoints
eTable 5. Request Rates for Individual Imaging Tests According to Frequency of Feedback (Twice vs. Once)
eTable 6. Request Rates for Individual Imaging Tests According to Display (Enhanced vs. Standard)
eTable 7. ICC (95% CI) for All Outcomes
eTable 8. Sensitivity Analyses of the Primary Outcome for the Main Comparison – Model-Based Estimates
Data Sharing Statement