Skip to main content
NCBI home page
The British Journal of General Practice logoLink to The British Journal of General Practice
. 2020 Nov 17;70(701):e858–e865. doi: 10.3399/bjgp20X713693

Strategies to reduce the use of low-value medical tests in primary care: a systematic review

Toshihiko Takada 1, Pauline Heus 2, Sander van Doorn 3, Christiana A Naaktgeboren 4, Jan-Willem Weenink 5, Simone A van Dulmen 6, Lotty Hooft 7
PMCID: PMC7679149  PMID: 33199293

Abstract

Background

It is recognised that medical tests are overused in primary care; however, it is unclear how best to reduce their use.

Aim

To identify which strategies are effective in reducing the use of low-value medical tests in primary care settings.

Design and setting

Systematic review.

Method

The databases MEDLINE, EMBASE, and Rx for Change were searched (January 1990 to November 2019) for randomised controlled trials (RCTs) that evaluated strategies to reduce the use of low-value medical tests in primary care settings. Two reviewers selected eligible RCTs, extracted data, and assessed their risk of bias.

Results

Of the 16 RCTs included in the review, 11 reported a statistically significant reduction in the use of low-value medical tests. The median of the differences between the relative reductions in the intervention and control arms was 17% (interquartile range 12% to 24%). Strategies using reminders or audit/feedback showed larger reduction than those without these components (22% versus 14%, and 22% versus 13%, respectively) and patient-targeted strategies showed larger reductions than those not targeted at patients (51% versus 17%). Very few studies investigated the sustainability of the effect, adverse events, cost-effectiveness, or patient-reported outcomes related to reducing the use of low-value tests.

Conclusion

This review indicates that it is possible to reduce the use of low-value medical tests in primary care, especially by using multiple components including reminders, audit/feedback, and patient-targeted interventions. To implement these strategies widely in primary care settings, more research is needed not only to investigate their effectiveness, but also to examine adverse events, cost-effectiveness, and patient-reported outcomes.

Keywords: investigative techniques, medical overuse, medical tests, primary care, systematic review, unnecessary procedures

INTRODUCTION

In primary care settings, the use of medical tests is increasing.1 However, a certain proportion of these tests are of low value, providing no benefit to patients or even causing harm.2,3

Although primary care physicians are aware that they overuse medical tests,4 there are some specific underlying mechanisms for this problem in primary care settings. First, as the pre-test probability of a serious disease is low and symptoms overlap between conditions, primary care physicians have to deal with greater diagnostic uncertainty than physicians in secondary and tertiary care settings.5,6 Second, primary care plays a major role in delivering screening and monitoring, for example, for various types of cancers and lifestyle diseases. When tests that were once considered effective have been found to be ineffective, for example, the use of routine mammography screening in women of average risk aged 40–49 years, primary care physicians are expected to discontinue them.7 However, it is not easy to keep up to date with the emerging evidence in the broad field of medicine in which primary care physicians are involved. Additionally, it has been reported that clinical guidelines have limited effect on physicians’ practice.8 Particularly, in relation to clinical guidance recommendations, de-implementation (reducing the use of low-value care) of existing practices is sometimes more difficult than implementing new practices.9

The awareness of low-value care has increased in recent years, and various initiatives have been introduced to address the issue.10 Although several systematic reviews about interventions to reduce low-value care have been undertaken,11–14 none have specifically focused on reducing the use of medical tests in primary care settings. Therefore, the primary aim of this study was to identify which strategies are effective in reducing the use of low-value medical tests in primary care settings. The study also investigated whether there was evidence about adverse events as a result of unperformed medical tests, other medical resource use, cost-effectiveness of de-implementation strategies, and patient-reported outcomes.

METHOD

The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement was followed.15

Data sources and searches

This review was part of a larger project on de-implementation for which studies evaluating strategies to reduce low-value care were identified regardless of type of care, setting, or study design. An information specialist conducted a literature search using MEDLINE, EMBASE, and Rx for Change databases in November 2019 (see Supplementary Box S1 for details of the search strategy). Reference lists of all included studies were also searched to identify reviews as an additional source.

How this fits in

It is uncertain which strategies have the greatest potential to reduce the use of low-value medical tests in primary care settings. The evidence from the randomised controlled trials in this review indicates that it is possible to reduce the use of low-value medical tests in primary care, particularly by combining multiple intervention components, including reminders and audit/feedback, and by targeting patients. However, to implement these strategies widely in primary care settings, more research is needed to investigate adverse events, cost-effectiveness, and patient-reported outcomes as consequences of reducing the use of low-value medical tests.
Open in a new tab

Study selection

Randomised controlled trials (RCTs) included in the review evaluated the effectiveness of a strategy for reducing low-value medical tests in primary care settings and were published in English, German, French, or Dutch after 1990. Studies on guideline adherence were only included when the aim of the study was explicitly stated as reducing low-value medical tests. Pairs of authors independently screened titles and abstracts, and subsequently full texts of potentially eligible publications. In the case of disagreement, the two authors discussed the issue, and consulted a third author when necessary.

Data extraction and critical appraisal

One of the authors extracted data, which was checked by a second author. A structured, pilot-tested electronic data extraction form was used that included study characteristics (study design, the type of medical tests being de-implemented [laboratory/imaging/physiological], the role of tests [diagnosis/screening/staging/monitoring], and targets and components of the de-implementation strategy) and outcomes (see Supplementary Box S2 for details). The component of de-implementation strategies was classified into nine categories using the taxonomy provided by the Cochrane Effective Practice and Organisation of Care Group.16,17 Each component of de-implementation strategies was further classified into four levels based on the target of a de-implementation strategy: provider, patient, organisation, and healthcare system (see Supplementary Box S3 for detailed examples for each category).18 The primary outcome was the effect of strategies to reduce the use of low-value medical tests or the total number of tests. The secondary outcomes were adverse events as a result of unperformed medical tests (for example, delay in diagnosis, referral, and treatment, or increased complications and mortality); other medical resource use (for example, other medical tests, or admission and visits to primary care/emergency room); cost-effectiveness of de-implementation strategies; and patient-reported outcomes (quality of life or patient satisfaction).

Two authors independently assessed the risk of bias using the Cochrane Collaboration’s risk of bias tool.19 In addition to the seven domains of this tool, three specific issues for cluster randomised trials were assessed.20–23

Analysis

The eligible studies reported the incidence of low-value medical tests in different ways, for example, only the incidence after intervention, the difference between baseline and post-intervention, or the incidence per arm/practice/physician/visits/patients. To compare the effect of de-implementation strategies across the studies, the relative reduction in the use of the low-value tests was calculated as the difference of the incidence between baseline and post-intervention divided by the incidence at baseline. The effectiveness of a strategy was defined as the difference between relative reductions in the intervention and control arms (net relative reduction). The studies in which de-implementation strategies were directly compared with each other were reported separately. When a study investigated the effect of a strategy on several low-value tests, the data of the low-value test with the median net relative reduction were selected as a representative of the study. In studies that compared several strategies, the strategy including the most interventions or addressing the most targets was selected. When there was only information about the total number of tests (without specifying if these were appropriate or inappropriate), the net relative reduction of total volume was selected. In addition to the analysis of the effect of strategies in the short term, the sustainability of effects was also assessed.

Factors potentially affecting the effect of strategies were explored: type of medical tests (laboratory/imaging/physiological tests), role of tests (diagnostic/screening/staging/monitoring), number of intervention components, number of targets, outcome measured (total number of tests or actual low-value tests), overall risk of bias in the included studies, and targets and components of the intervention. Studies with low overall risk of bias were defined as satisfying all of the following criteria: an adequate random sequence generation; a low risk of bias for all three domains related to cluster randomised designs, if applicable; and not rated as high risk of bias because of unconcealed allocation, detection bias, attrition bias, or reporting bias, with unclear risk of bias for a maximum of two domains.

RESULTS

Search results

A total of 4590 records were identified by the search. After the title and abstract screening, a full-text assessment was conducted for the remaining 936 articles, 16 of which were eligible for inclusion (Figure 1).24–41

Figure 1.

Figure 1.

Open in a new tab

Literature selection process. RCT = randomised controlled trial.

Characteristics of included studies

One-third of the studies were conducted in the UK (n = 6; 38%) (Table 1). More than half of the studies (n = 9; 56%) specified the indications of the tests to be de-implemented, of which low back pain was the most common (n = 4; 25%). The types of medical tests aimed for de-implementation were laboratory tests (n = 8; 50%), imaging tests (n = 11; 69%), and physiological tests (n = 3; 19%). Twelve studies (75%) specified the role of tests: diagnostics (n = 12; 75%), screening (n = 7; 44%), and monitoring (n = 6; 38%) with some overlapping, while no study focused on tests for staging (data not shown].

Table 1.

Summary of the characteristics of included studies (N = 16)

Characteristic Included studies, n (%)
Country
  UK 6 (38)
  The Netherlands 4 (25)
  US 3 (19)
  Australia 2 (13)
  Norway 1 (6)
Study design
  Cluster RCT 15 (94)
  RCT 1 (6)
Setting
  Single centre 2 (13)
  Multicentre 14 (88)
Indication for medical testsa
  Specified 9 (56)
  Low back pain 4 (25)
  Othersb 5 (31)
  Not specified 7 (44)
Type of medical testsa
  Laboratory tests 8 (50)
  Imaging 11 (69)
  Physiology 3 (19)
Open in a new tab
a

Some studies were applicable to >1 category.

b

Many other conditions were addressed, but most of them were evaluated in only one of the included studies (see Supplementary Table S1 for detailed information about each condition). RCT = randomised controlled trial.

De-implementation strategies

De-implementation strategies in the included studies were classified by their target and the number of interventions (single/combination of ≥2) (Table 2). All six studies with a single target and a single intervention were aimed at healthcare providers. Among them, educational materials and reminders were most frequently used (33% for both). Similarly, healthcare providers were targeted in all seven studies having a single target and using a combination of interventions. Educational materials and audit/feedback were the most frequently used strategies (86% for both). Among three studies addressing multiple targets with a combination of interventions, all targeted the healthcare provider. Two (67%) studies additionally targeted patients and one study additionally targeted the organisational context and healthcare system (see Supplementary Table S1 for detailed information about each study).

Table 2.

De-implementation strategies by the number of intervention components and targets

Intervention Single target, single intervention (n= 6), n (%) Single target, combination of interventions (n= 7), n (%) Multiple targets, combination of interventions (n= 3), n (%) All (n = 16), n (%)
Targeted at provider 6 (100) 7 (100) 3 (100) 16 (100)
Educational meetings 1 (17) 4 (57) 3 (100) 8 (50)
Distribution of educational material 2 (33) 6 (86) 3 (100) 11 (69)
Reminders 2 (33) 2 (29) 1 (33) 5 (31)
Audit/feedback 1 (17) 6 (86) 1 (33) 8 (50)
Targeted at patient 0 (0) 0 (0) 2 (67) 2 (13)
Targeted at organisational context 0 (0) 0 (0) 1 (33) 1 (6)
Organisational interventions 0 (0) 0 (0) 1 (33) 1 (6)
Structural interventions 0(0) 0(0) 0(0) 0 (0)
Targeted at healthcare system 0 (0) 0 (0) 1 (33) 1 (6)
Regulatory interventions 0(0) 0(0) 0(0) 0 (0)
Financial interventions 0 (0) 0 (0) 1 (33) 1 (6)
Open in a new tab

Risk of bias

In the domain of allocation concealment, seven studies (44%) were rated as low risk of bias, whereas nine studies (56%) did not give sufficient information. As blinding of participants was difficult because of the nature of the intervention, most studies (n = 11; 69%) were rated as high risk of bias in this item. Four studies (25%) satisfied the criteria of overall low risk of bias. See Supplementary Figures S1 and S2 for details of the results of the assessment of risk of bias.

Effectiveness of de-implementation

Eleven studies (69%) reported that their intervention showed a statistically significant reduction. Ten studies (63%) reported the necessary information to calculate relative reductions of the incidence of the low-value tests (see Supplementary Table S2 for details of the six studies without information to calculate the relative reduction).

Comparison of de-implementation with usual care

The median net relative reduction in the use of low-value tests was 17% (interquartile range [IQR] 12% to 24%). A comparison of net relative reductions based on study characteristics is shown in Figure 2 (see Supplementary Table S3 for details). Strategies with multiple targets and a combination of interventions tended to be more effective than those with a single target. Strategies using reminders and audit/feedback showed a larger reduction than those without these components: 22% (IQR 17% to 31%) versus 14% (IQR 12% to 20%), and 22% (IQR 13% to 37%) versus 13% (IQR 11% to 16%), respectively. Studies targeted at patients showed a larger reduction in the use of low-value tests than those not targeted at patients: 51% (IQR 30% to 72%) versus 17% (IQR 12% to 23%).

Figure 2.

Figure 2.

Open in a new tab

Comparison of relative reductions in the use of low-value medical tests based on characteristics of de-implementation strategies.

Direct comparison of de-implementation strategies

In three studies, a direct comparison of de-implementation strategies was reported.26,33,36 In one study, reminders were more effective than audit/feedback in de-implementation of imaging studies (41% versus 29% for lumbar radiograph, and 33% versus 15% for knee radiograph, respectively).26 However, the second study showed an opposite trend that reminders were less effective than audit/feedback in de-implementation of laboratory tests (15% versus 27%, respectively).33 In the third study, a computer-based decision support system based on guidelines reduced the number of laboratory tests by 20% compared with a system based on a reduced list of medical tests.36

Sustainability of effect

Three studies evaluated the sustainability of the effect of the strategy.34,40,41 One of them did not report results.41 The other two reported that the effect of the strategy was not sustainable despite an initially observed significant effect.34,40

Secondary outcomes

One study reported adverse events as a result of unperformed tests and found no increase in the number of hospitalisations, emergency room visits, or outpatient visits.34

One study assessed the cost-effectiveness of de-implementation strategies. In the comparison between an original multifaceted strategy (combining written feedback, group education, and distribution of guidelines) and a strategy using only feedback, the multifaceted strategy was more effective in cost reduction than only using feedback. However, the cost for the strategy surpassed the reduced cost.39

Two studies measured patient satisfaction. In one study, the intervention was designed to enhance primary care physicians’ patient-centredness and skills in handling patient requests for low-value diagnostic tests. Patients in the intervention group were more satisfied than in those in the control group.27 The other study stated in the method section that patient satisfaction was measured; however, no results were reported.32

DISCUSSION

Summary

Of 16 RCTs investigating the effect of strategies to reduce low-value medical tests in primary care, 11 studies (69%) reported a statistically significant reduction. The median net relative reduction in the use of low-value tests was 17%. Addressing multiple targets and using a combination of interventions tended to increase effectiveness. Strategies using reminders or audit/feedback showed larger improvements than those without these components (22% versus 14%, and 22% versus 13%, respectively), and patient-targeted strategies showed a larger reduction in the use of low-value tests than those not targeted at patients (51% versus 17%).

Strengths and limitations

Although there have been several reviews of quality improvement in primary care,42 to the best of the authors’ knowledge, this is the first review to evaluate the effect of strategies to reduce the use of low-value medical tests in primary care. This study has several limitations. First, for six studies it was not possible to calculate the relative reduction of the use of low-value tests, as they lacked the necessary information. This has also been encountered in other reviews.11,14 To promote the integration of evidence, recommendations about appropriate outcome measures for de-implementation are required. Second, in the analysis of factors related to the effect of strategies, there were only a very small number of studies in some categories, so the findings should be interpreted with caution. Finally, there was substantial heterogeneity among the included studies in terms of the type and role of medical tests, components, and targets of intervention. As a result, it was difficult to directly compare the effect of each of these factors in reducing the use of low-value medical tests.

Comparison with existing literature

The findings of this study corroborate the results of existing systematic reviews about strategies to promote the appropriate use of medical tests, which included mainly observational studies without a control group. Some of these reviews showed that interventions to reduce the use of laboratory tests are generally successful.11,14,43 However, they focused only on laboratory tests and the setting of two reviews was solely11 or mainly secondary/tertiary care.14 Another review found that multicomponent interventions were more effective than single-component interventions in increasing the appropriate use of diagnostic tests by physicians in various settings.44 However, as well as including studies that aimed to reduce the use of low-value tests, this review also included studies that aimed to promote underused tests.

In line with the findings of a review that evaluated the effect of de-implementation strategies with no restriction on types of low-value care (medical tests or treatment) and settings,13 the current study suggests that strategies that target patients as well as providers may be more effective. Although physicians may order low-value tests because of diagnostic uncertainty or misconceptions of the value of tests, patients may frequently request those tests. It has been reported that such patients are usually anxious and require reassurance.45 Although physicians sometimes rationalise the use of low-value tests to reassure patients, these tests hardly help to decrease patients’ anxiety.46 To improve patients’ understanding of low-value tests, the results of the current study suggest that it is of added value to include patient educational components as a part of de-implementation strategies.

Implications for research and practice

Although the effect of de-implementation strategies on the use of low-value care has been extensively evaluated, there is little evidence about the potential negative consequences of these strategies. One reason might be that negative effects are rare, so studies would need a large sample size and long follow-up for them to be evaluated. Because fear of juridical claims is one of the reasons for physicians to order tests,47 it is necessary to assure them that low-value tests can be omitted without adverse events, such as delays in diagnosis, referral, and treatment, or increased complications and mortality. Furthermore, sustainability and cost-effectiveness are crucial considerations for introducing de-implementation strategies on a larger scale. Nevertheless, only a few studies in this review evaluated these outcomes. Additionally, patient satisfaction is an important outcome in clinical practice, which can be impaired by declining medical tests requested by patients.48 Further research is therefore needed to assess the consequences of these interventions in the long term before the spread of de-implementation strategies for low-value tests is promoted.

Acknowledgments

Thanks to René Spijker for developing and performing the searches, and Eva Verkerk and Tijn Kool for their contribution to the selection of articles and data extraction.

Funding

This project received funding from The Netherlands Organisation for Health Research and Development (ZonMW — Citrienfonds 80-83920-98-101). The funder had no role in the design, conduct, analyses, or reporting of the study, or in the decision to submit the manuscript for publication. Toshihiko Takada was supported by the Uehara Memorial Foundation.

Ethical approval

Ethical approval was not required for this systematic review.

Provenance

Freely submitted; externally peer reviewed.

Competing interests

The authors have declared no competing interests.

Discuss this article

Contribute and read comments about this article: bjgp.org/letters

REFERENCES

  • 1.O’Sullivan JW, Stevens S, Hobbs FDR, et al. Temporal trends in use of tests in UK primary care, 2000–15: retrospective analysis of 250 million tests. BMJ. 2018;363:k4666. doi: 10.1136/bmj.k4666. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.de Vries EF, Struijs JN, Heijink R, et al. Are low-value care measures up to the task? A systematic review of the literature. BMC Health Serv Res. 2016;16:405. doi: 10.1186/s12913-016-1656-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.O’Sullivan JW, Albasri A, Nicholson BD, et al. Overtesting and undertesting in primary care: a systematic review and meta-analysis. BMJ Open. 2018;8(2):e018557. doi: 10.1136/bmjopen-2017-018557. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Sirovich BE, Woloshin S, Schwartz LM. Too little? Too much? Primary care physicians’ views on US health care: a brief report. Arch Intern Med. 2011;171(17):1582–1585. doi: 10.1001/archinternmed.2011.437. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Okkes IM, Oskam SK, Lamberts H. The probability of specific diagnoses for patients presenting with common symptoms to Dutch family physicians. J Fam Pract. 2002;51(1):31–36. [PubMed] [Google Scholar]
  • 6.McCowan C, Fahey T. Diagnosis and diagnostic testing in primary care. Br J Gen Pract. 2006;56(526):323–324. [PMC free article] [PubMed] [Google Scholar]
  • 7.Siu AL. U.S. Preventive Services Task Force. Screening for breast cancer: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med. 2016;164(4):279–296. doi: 10.7326/M15-2886. [DOI] [PubMed] [Google Scholar]
  • 8.Cabana MD, Rand CS, Powe NR, et al. Why don’t physicians follow clinical practice guidelines? A framework for improvement. JAMA. 1999;282(15):1458–1465. doi: 10.1001/jama.282.15.1458. [DOI] [PubMed] [Google Scholar]
  • 9.Roman BR, Asch DA. Faded promises: the challenge of deadopting low-value care. Ann Intern Med. 2014;161(2):149–150. doi: 10.7326/M14-0212. [DOI] [PubMed] [Google Scholar]
  • 10.Choosing Wisely Choosing Wisely: promoting conversations between patients and clinicians. 2020 https://www.choosingwisely.org (accessed 6 Nov 2020).
  • 11.Bindraban RS, Ten Berg MJ, Naaktgeboren CA, et al. Reducing test utilization in hospital settings: a narrative review. Ann Lab Med. 2018;38(5):402–412. doi: 10.3343/alm.2018.38.5.402. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Chambers JD, Salem MN, D’Cruz BN, et al. A review of empirical analyses of disinvestment initiatives. Value Health. 2017;20(7):909–918. doi: 10.1016/j.jval.2017.03.015. [DOI] [PubMed] [Google Scholar]
  • 13.Colla CH, Mainor AJ, Hargreaves C, et al. Interventions aimed at reducing use of low-value health services: a systematic review. Med Care Res Rev. 2017;74(5):507–550. doi: 10.1177/1077558716656970. [DOI] [PubMed] [Google Scholar]
  • 14.Kobewka DM, Ronksley PE, McKay JA, et al. Influence of educational, audit and feedback, system based, and incentive and penalty interventions to reduce laboratory test utilization: a systematic review. Clin Chem Lab Med. 2015;53(2):157–183. doi: 10.1515/cclm-2014-0778. [DOI] [PubMed] [Google Scholar]
  • 15.Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Ann Intern Med. 2009;151(4):W65–W94. doi: 10.7326/0003-4819-151-4-200908180-00136. [DOI] [PubMed] [Google Scholar]
  • 16.Effective Practice and Organisation of Care EPOC taxonomy. 2015 https://epoc.cochrane.org/epoc-taxonomy (accessed 6 Nov 2020).
  • 17.Effective Practice and Organisation of Care . The EPOC taxonomy of health systems interventions. EPOC Resources for review authors. Oslo: Norwegian Knowledge Centre for the Health Services; 2016. [Google Scholar]
  • 18.Grol R, Wensing M, Eccles M, Davis D, editors. Improving patient care: the implementation of change in health care. 2nd edn. Chichester: John Wiley & Sons; 2013. [Google Scholar]
  • 19.Higgins JPT, Altman DG, Gotzsche PC, et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928. doi: 10.1136/bmj.d5928. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Bolzern J, Mnyama N, Bosanquet K, Torgerson DJ. A review of cluster randomized trials found statistical evidence of selection bias. J Clin Epidemiol. 2018;99:106–112. doi: 10.1016/j.jclinepi.2018.03.010. [DOI] [PubMed] [Google Scholar]
  • 21.Brierley G, Brabyn S, Torgerson D, Watson J. Bias in recruitment to cluster randomized trials: a review of recent publications. J Eval Clin Pract. 2012;18(4):878–886. doi: 10.1111/j.1365-2753.2011.01700.x. [DOI] [PubMed] [Google Scholar]
  • 22.Higgins JPT, Thomas J, Chandler J, et al. Cochrane handbook for systematic reviews of interventions. 2nd edn. Chichester: John Wiley & Sons; 2019. [Google Scholar]
  • 23.Puffer S, Torgerson D, Watson J. Evidence for risk of bias in cluster randomised trials: review of recent trials published in three general medical journals. BMJ. 2003;327(7418):785–789. doi: 10.1136/bmj.327.7418.785. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Bearcroft PWP, Small JH, Flower CDR. Chest radiography guidelines for general practitioners: a practical approach. Clin Radiol. 1994;49(1):56–58. doi: 10.1016/s0009-9260(05)82916-2. [DOI] [PubMed] [Google Scholar]
  • 25.Dey P, Simpson CWR, Collins SI, et al. Implementation of RCGP guidelines for acute low back pain: a cluster randomised controlled trial. Br J Gen Pract. 2004;54(498):33–37. [PMC free article] [PubMed] [Google Scholar]
  • 26.Eccles M, Steen N, Grimshaw J, et al. Effect of audit and feedback, and reminder messages on primary-care radiology referrals: a randomised trial. Lancet. 2001;357(9266):1406–1409. doi: 10.1016/S0140-6736(00)04564-5. [DOI] [PubMed] [Google Scholar]
  • 27.Fenton JJ, Kravitz RL, Jerant A, et al. Promoting patient-centered counseling to reduce use of low-value diagnostic tests: a randomized clinical trial. JAMA Intern Med. 2016;176(2):191–197. doi: 10.1001/jamainternmed.2015.6840. [DOI] [PubMed] [Google Scholar]
  • 28.Flottorp S, Oxman AD, Havelsrud K, et al. Cluster randomised controlled trial of tailored interventions to improve the management of urinary tract infections in women and sore throat. BMJ. 2002;325(7360):367. doi: 10.1136/bmj.325.7360.367. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 29.French SD, McKenzie JE, O’Connor DA, et al. Evaluation of a theory-informed implementation intervention for the management of acute low back pain in general medical practice: the IMPLEMENT cluster randomised trial. PLoS One. 2013;8(6):e65471. doi: 10.1371/journal.pone.0065471. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 30.Kerry S, Oakeshott P, Dundas D, Williams J. Influence of postal distribution of the Royal College of Radiologists’ guidelines, together with feedback on radiological referral rates, on X-ray referrals from general practice: a randomized controlled trial. Fam Pract. 2000;17(1):46–52. doi: 10.1093/fampra/17.1.46. [DOI] [PubMed] [Google Scholar]
  • 31.Oakeshott P, Kerry SM, Williams JE. Randomized controlled trial of the effect of the Royal College of Radiologists’ guidelines on general practitioners’ referrals for radiographic examination. Br J Gen Pract. 1994;44(382):197–200. [PMC free article] [PubMed] [Google Scholar]
  • 32.Schectman JM, Schroth WS, Verme D, Voss JD. Randomized controlled trial of education and feedback for implementation of guidelines for acute low back pain. J Gen Intern Med. 2003;18(10):773–780. doi: 10.1046/j.1525-1497.2003.10205.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 33.Thomas RE, Croal BL, Ramsay C, et al. Effect of enhanced feedback and brief educational reminder messages on laboratory test requesting in primary care: a cluster randomised trial. Lancet. 2006;367(9527):1990–1996. doi: 10.1016/S0140-6736(06)68888-0. [DOI] [PubMed] [Google Scholar]
  • 34.Tierney WM, Miller ME, McDonald CJ. The effect on test ordering of informing physicians of the charges for outpatient diagnostic tests. N Engl J Med. 1990;322(21):1499–1504. doi: 10.1056/NEJM199005243222105. [DOI] [PubMed] [Google Scholar]
  • 35.Trietsch J, van Steenkiste B, Grol R, et al. Effect of audit and feedback with peer review on general practitioners’ prescribing and test ordering performance: a cluster-randomized controlled trial. BMC Fam Pract. 2017;18(1):53. doi: 10.1186/s12875-017-0605-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.van Wijk MA, van der Lei J, Mosseveld M, et al. Assessment of decision support for blood test ordering in primary care. a randomized trial. Ann Intern Med. 2001;134(4):274–281. doi: 10.7326/0003-4819-134-4-200102200-00010. [DOI] [PubMed] [Google Scholar]
  • 37.Verstappen WH, van der Weijden T, Dubois WI, et al. Improving test ordering in primary care: the added value of a small-group quality improvement strategy compared with classic feedback only. Ann Fam Med. 2004;2(6):569–575. doi: 10.1370/afm.244. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Verstappen WH, van der Weijden T, Sijbrandij J, et al. Effect of a practice-based strategy on test ordering performance of primary care physicians: a randomized trial. JAMA. 2003;289(18):2407–2412. doi: 10.1001/jama.289.18.2407. [DOI] [PubMed] [Google Scholar]
  • 39.Verstappen WH, van Merode F, Grimshaw J, et al. Comparing cost effects of two quality strategies to improve test ordering in primary care: a randomized trial. Int J Qual Health Care. 2004;16(5):391–398. doi: 10.1093/intqhc/mzh070. [DOI] [PubMed] [Google Scholar]
  • 40.Weller D, May F, Rowett D, et al. Promoting better use of the PSA test in general practice: randomized controlled trial of educational strategies based on outreach visits and mailout. Fam Pract. 2003;20(6):655–661. doi: 10.1093/fampra/cmg606. [DOI] [PubMed] [Google Scholar]
  • 41.Winkens RA, Pop P, Bugter-Maessen AM, et al. Randomised controlled trial of routine individual feedback to improve rationality and reduce numbers of test requests. Lancet. 1995;345(8948):498–502. doi: 10.1016/s0140-6736(95)90588-x. [DOI] [PubMed] [Google Scholar]
  • 42.Irwin R, Stokes T, Marshall T. Practice-level quality improvement interventions in primary care: a review of systematic reviews. Prim Health Care Res Dev. 2015;16(6):556–577. doi: 10.1017/S1463423615000274. [DOI] [PubMed] [Google Scholar]
  • 43.Cadogan SL, Browne JP, Bradley CP, Cahill MR. The effectiveness of interventions to improve laboratory requesting patterns among primary care physicians: a systematic review. Implement Sci. 2015;10:167. doi: 10.1186/s13012-015-0356-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Solomon DH, Hashimoto H, Daltroy L, Liang MH. Techniques to improve physicians’ use of diagnostic tests: a new conceptual framework. JAMA. 1998;280(23):2020–2027. doi: 10.1001/jama.280.23.2020. [DOI] [PubMed] [Google Scholar]
  • 45.van Bokhoven MA, Pleunis-van Empel MCH, Koch H, et al. Why do patients want to have their blood tested? A qualitative study of patient expectations in general practice. BMC Fam Pract. 2006;7:75. doi: 10.1186/1471-2296-7-75. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Rolfe A, Burton C. Reassurance after diagnostic testing with a low pretest probability of serious disease: systematic review and meta-analysis. JAMA Intern Med. 2013;173(6):407–416. doi: 10.1001/jamainternmed.2013.2762. [DOI] [PubMed] [Google Scholar]
  • 47.van der Weijden T, van Bokhoven MA, Dinant GJ, et al. Understanding laboratory testing in diagnostic uncertainty: a qualitative study in general practice. Br J Gen Pract. 2002;52(485):974–980. [PMC free article] [PubMed] [Google Scholar]
  • 48.Kravitz RL, Bell RA, Azari R, et al. Request fulfillment in office practice: antecedents and relationship to outcomes. Med Care. 2002;40(1):38–51. doi: 10.1097/00005650-200201000-00006. [DOI] [PubMed] [Google Scholar]

Articles from The British Journal of General Practice are provided here courtesy of Royal College of General Practitioners

RESOURCES