Table 3.
Characteristics of the included studies, outcome of the intervention and quality assessment result
| Author (year) | Country | Methods | Population | Clinical setting | Intervention | Low-value practice | Outcome of intervention(s) | Quality assessment |
|---|---|---|---|---|---|---|---|---|
| Arora et al. (2020) [126] | USA | Evaluation of a quality improvement project | 1535 children | Emergency department | Education, decision support, and performance feedback | Head CT | 4.6%-points reduction in use | * |
| Ashykian et al. (2019) [37] | USA | Retrospective review | 1000 reports | Orthopedic and family practice departments | Education | Repeat radiographs for routine follow-up of osteoarthrosis | 50% reduction in use | * |
| Bailey et al. (2013) [38] | USA | Longitudinal data analysis | 800 patients | Emergency department | Health information exchange | Repeated lumbar or thoracic imaging | 64% lower odds of repeat | * |
| Bailey, Wan et al. (2013) [39] | USA | Longitudinal data analysis | 1252 patients | Emergency department | Health information exchange | Diagnostic neuroimaging | 62% lower odds of repeat | * |
| Bairstow et al. (2010) [40] | Australia | Pre/post audit | 1061 patients | Emergency department | Education, request forms | Chest CTA, abdominal XR, imaging with a provisional diagnosis of renal colic and XR despite negative Ottawa Ankle Rule | 13–28% reduction in use | * |
| Baker et al. (2020) [41] | USA | Retrospective registry review | 445 patients | Emergency department | Education and guideline | C-spine CT | > 30% reduction in use | * |
| Ballard et al. (2019) [127] | USA | Nonrandomized clinical trial - secondary analysis | 3859 children | Emergency department | Electronic clinical decision support | Head CT | 2.6% unsignificant reduction in use | * |
| Bhatia et al. (2013) [44] | USA | Prospective, time series analysis | 1711 patients | Academic medical center | Education | Transthoracic echocardiography | 26% reduction in use | |
| Bhatia et al. (2014) [42] | USA | Randomized Control Trial | 88 internal medicine residents and 24 cardiology fellows | Academic medical center | Education | Transthoracic echocardiography | 62% reduction in use | * |
| Bhatia et al. (2017) [43] | Canada/USA | Multicenter, randomized controlled trial | 196 physicians | 8 hospitals | Education and feedback | Transthoracic echocardiogram | 1.3% significant reduction in use | * |
| Blackmore et al. (2011) [45] | USA | retrospective cohort study | 6141 patients | Medical Center | Decision support system | Lumbar MRI, Brain MRI in headache, and sinus CT | 23.2–26.8% significant lower use | |
| Bookman et al. (2017) [46] | USA | Longitudinal, before/after study | 235,858 patient visits | 5 emergency departments | Clinical decision support system | Head/c-spine CT, Chest CTA |
> 6% significant decrease in head and c-spine CT 2% non-significant reduction chest CTA |
* |
| Boutis et al. (2013) [47] | Canada | Interrupted time series with pair matched control design. | 2151 children | 6 emergency departments | Education, reminders, and computerized decision support system | Ankle XR | 22% reduction in use | * |
| Breakell et al. (2018) [48] | UK | Retrospective audit | 101 children | District General Hospital | Education and guideline | Chest XR for Bronchiolitis | 16% reduction in use | |
| Buntine et al. (2018) [49] | Australia | Before and after study | 2931 scans | 3 hospitals | Flowchart | Chest CTA and NM ventilation perfusion | 6 per 1000 scans reduction in use | * |
| Carnevale et al. (2015) [128] | USA | Before-and-after study | 29,395 encounters | Emergency department | Decision support and education | Several | 5–10% reduction in use | * |
| Carpenter et al. (2020) [50] | USA | Retrospective chart review | 1010 children | Hospital | Choosing Wisely | CT/MR or US in cryptorchidism | No significant difference | * |
| Chandra et al. (2019) [51] | Canada | Evaluation of a quality improvement project | 37 physicians | Community teaching hospital | Didactic seminar, Survey | Imaging for low back pain | 4%-points significant increase | * |
| Chang et al. (2018) [52] | USA | Before-and-after study | 202 family physicians and 8 general internists | Clinics | Feedback reports | CT, MRI, and PET | 14.5% reduction in use | * |
| Char et al. (2014) [53] | USA | Retrospective chart review | 510 patients | Emergency department | Increased D-dimer threshold value and clinical probability assessment | Chest CTA | 7%-points Increase in diagnostic yield | |
| Chen et al. (2020) [54] | USA | Before-and-after study | Physicians | Hospital | Alerts | Imaging for lower back pain | 9.6% reduction in total imaging rate and MRI (14.9%), No significant difference in use of CT/XR | * |
| Chien et al. (2017) [55] | USA | Block randomized controlled trial | 1205 clinicians | Multidisciplinary medical group | Financial incentives | CT or MRI for single headache or lower back pain, acute, uncomplicated rhinosinusitis, or DEXA for low risk for osteoporosis | No significant difference | * |
| Depinet et al. (2016) [56] | USA | Interrupted time series trial | 1886 children | Urban tertiary care hospital | Clinical decision support system and pathway | Abdominopelvic CT/US | 2%-points increased use of US and 5%-points reduction in use of CT | * |
| Doyle et al. (2019) [57] | USA | Randomized study | 3524 practitioners | 15 hospitals and 150 clinics | Best practice alerts | Several | 6% reduction in use | |
| Drescher et al. (2011) [58] | USA | Before and after study | 404 cases | Emergency department | Algorithm | Chest CTA | 4.4% increase in use | * |
| Dudzinski et al. (2016) [59] | USA | Before and after study | 65 cardiologists | Ambulatory cardiology practices in hospital | Education and feedback | Transthoracic echocardiography | 6%-points reduction in use | * |
| Dunne et al. (2015) [60] | UK | Before and after study | 5892 examinations | Hospital | Clinical decision support system | Chest CTA | 12.3% reduction in use | * |
| Durand et al. (2013) [61] | USA | Randomized controlled trail | 10 imaging tests | Tertiary teaching hospital | Cost display | Several | No significant difference | * |
| Ehrlichman et al. (2017) [129] | USA | Before-and-after study | 104,454 patients | Emergency department | Feedback | Several | 2.3% reduction | * |
| Fallon et al. (2016) [62] | USA | Prospective, longitudinal study | Children | Level I pediatric trauma center | Development of Trauma Protocol | Abdominal CT for abdominal trauma | 18%-points increase in diagnostic yield | * |
| Ferguson et al. (2017) [63] | USA | Interventional improvement project | 1 Emergency department | Hospital | Education and diagram | Abdominal XR constipation | 38%-points reduction | * |
| Flamm et al. (2013) [64] | Austria | Non-randomized controlled trial with a historical control group | 1363 patients | Hospital | PReOPerative evaluation’ (PROP) | Chest XR | 21.7%-points reduction | |
| French et al. (2013) [65] | Australia | Cluster Randomized trail | 112 general practitioners | Practices | Guideline/ facilitated interactive workshops | Lumbar CT or XR | No significant difference | |
| Gertz et al. (2016) [66] | USA | Before-and-after study | 941 patients | Hospital | Computerized order entry tool | Cardiac stress tests with imaging | No significant difference | |
| Goldberg et al. (2011) [67] | USA | Retrospective cohort chart review | 742 patients | Hospital | Guideline | Head CT | 16% reduction in use | |
| Graves et al. (2018) [68] | USA | Interrupted time series | 76,119 compensation claims | Regional | Policy implementation | Imaging for lower back pain |
5,6%-points significant decrease in use of MRI. 2.46% increase in the use of XR No change in use of CT |
|
| Hardin et al. (2017) [69] | USA | Pre−/post-test design | 339 patients | Hospital | Complex Care Map | CT scans | 62% reduction in use | |
| Hess et al. (2018) [70] | USA | Cluster Randomized Trial | 172 clinicians | Emergency department | Shared decision-making | Head CT (children) | No significant difference | |
| Hong et al. (2017) [71] | USA | Retrospective chart review | 1,547,870 patients | Several | Choosing wisely | Imaging for lower back pain | 4% reduction in use | |
| Hoo et al. (2011) [72] | USA | Retrospective chart review | 457 examinations | Hospital | Mandatory clinical decision rule and selective d-dimer use | Chest CTA | 13%-points Increase in diagnostic yield | * |
| Hui et al. (2014) [73] | USA | Prospective cohort study and retrospective review | 762 patients | Hospital | Education and guideline | Pelvic US | 58% reduction in use | |
| Hurley et al. (2017) [74] | USA | Before-and-after study | 10,554 patients | Hospital | MUSIC imitative Collaborative | Bone scan and CT for prostate cancer | 4.5–7%-points reduction in use | * |
| Ip et al. (2013) [78] | USA | Before-and-after study | 1.8 million patient-months | Hospital | Computerized order entry tool with clinical decision support systems and accountability tools | Multiple | 12% reduction in use | * |
| Ip et al. (2014) [75] | USA | Retrospective cohort study | 21,445 LBP-related primary care visits | Academic quaternary care hospital | Clinical decision support system | Imaging for low back pain | 30.8% reduction in use | * |
| Ip et al. (2015) [77] | USA | Before-and-after study | Emergency department patients | Hospital | Clinical decision support system | Head CT | 13.4% reduction in use | * |
| Ip et al. (2017) [76] | USA | Before-and-after study | 98,894 radiologyorders | Four institutions | Clinical decision support system | Several | 1%-point reduction in use | |
| Jennings et al. (2017) [79] | USA | Evaluation of a quality improvement project | 1346 Children | Community emergency department | Protocol, education, and individual feedback. | CT head of children, minor head injury | 12% reduction in use | |
| Judkins et al. (2013) [80] | Australia | Retrospective chart review | 659 children | Tertiary children’s hospital | NICE guidelines | Ultrasound urinary system, MUCG and dimercaptocsuccinic acid scintigraphy | 50% reduction in the use | |
| Kandiah et al. (2020) [81] | Canada | Evaluation of a quality improvement project | 4480 patients | Hospital | Education and information packages | MRI and CT of joints without red flags |
CT 43% reduction in use MRI 0.6% reduction of use |
* |
| Kanaan et al. (2013) [82] | USA | Retrospective chart review | 200 patients | Tertiary emergency department | Education | Chest CTA | No significant difference | * |
| Keveson et al. (2017) [83] | USA | Evaluation of a quality improvement project | All invasive mechanical ventilator patients | Tertiary hospital | Education and change in routines/referral system | Daily CXR of ventilated patients | 64% reduction in use | * |
| Kobes et al. (2020) [130] | Canada | Retrospective chart review | 28 medical imaging sites | Mobile radiography | Guideline | Chest XR | 3.2% reduction in use | * |
| Lacson et al. (2017) [84] | USA | Retrospective chart review | 63,222 orders | Hospital | Clinical decision support system | Several | No significant difference | |
| Levitt et al. (2015) [120] | USA | Before-and-after study | 415 patients | Hospital | Decision support and education | Stress echocardiography | 12%-points reduction in use | * |
| Lu et al. (2012) [85] | USA | Retrospective chart review | 267 patients | Hospital | Importing images from other institutions into PACS | Repeat imaging | 61%-points reduction in use | |
| Luther et al. (2019) [86] | USA | Retrospective chart review | 273 patients | Hospital | Standardized clinical assessment and management plans | Wrist XR | 60% reduction in use | * |
| Masood et al. (2020) [87] | Canada | Evaluation of a quality improvement project | Parents in the emergency department | Tertiary care center | Education, guideline and checklist, patient handouts, and feedback | Head CT in adults | 7–14% reduction in use | |
| McGrew et al. (2018) [88] | USA | Retrospective chart review | 1934 children | Pediatric Level 2 Trauma Center | Guideline | Head and abdomen/pelvis CT pediatric trauma | 11.5–18.8% reduction in use | |
| Mills et al. (2018) [121] | USA | Before-and-after study | 7987 patients | Emergency department | Decision support | Chest CTA | 2.5% increased diagnostic yield | * |
| Min et al. (2017) [89] | Canada | Retrospective chart review | 4562 patients | Emergency department | Checklist | Imaging in low back pain | 22% reduction in use | |
| Mittal et al. (2014) [91] | USA | Before and after design | 3641 patients | Tertiary care hospital | Clinical practice guidelines | Chest XR | 14.6–20%-points reduction in use | |
| Moriarity et al. (2015) [91] | USA | Retrospective chart review | 33,311 patients | Hospital | Clinical decision support | Inpatient MR, CT, and NM | No significant difference | |
| Mulders et al. (2020) [92] | The Netherlands | Before-and-after comparative prospective cohort study | 1261 patients | Emergency department | Amsterdam Wrist Rules | Wrist XR | 15%-points reduction in use | |
| Mäenpää et al. (2011) [93] | Finland | Retrospective, longitudinal study | 1 regional hospital | Hospital | Regional Health InformationExchange | Several | 16.4% reduction in use | * |
| Nigrovic et al. (2015) [94] | USA | Multifaceted quality improvement initiative | Children < 21 years | Urban tertiary care academic center | Guideline development, feedback, and education | Head CT | 6%-points reduction in use | * |
| O’Connor et al. (2014) [95] | USA | Prospective before and after study | 28,420 CT orders | Tertiaryacademic medical center | Requiring a clinical justification to override a repeat CT alert | Several | Prevented 1 in 13 scans | * |
| Ong et al. (2013) [122] | USA | Before-and-after study | 471 patients | Hospital | Algorithm and guideline | Chest CTA | 26% reduction in use | * |
| Ostby et al. (2020) [96] | USA | Evaluation of a quality improvement project | 235 patients | Emergency department | Specialist consolation before imaging | CT of gynecological cancer patients | 54%-points reduction in use | * |
| Palen et al. (2019) [97] | USA | Stepped-wedge study | 31,426 orders | several | Check boxes | Several | Modest increase in appropriateness | |
| Parikh et al. (2016) [98] | USA | Retrospective cohort study | 220,539 patients | Hospital | Guideline | Chest XR | 6.4% significant reduction in use | * |
| Poeran et al. (2019) [99] | USA | Retrospective interrupted time series | 27,549 orders | Emergency department | Clinical decision support | Low appropriateness imaging | 9%-points reduction in use | * |
| Prevedello et al. (2013) [100] | USA | retrospective cohort study | 2891 patients | Emergency department | Alerts | Chest CTA | 2.2/1000 reduction in use | * |
| Puffenbarger et al. (2019) [123] | USA | Retrospective chart review | 556 visits | Emergency department | Education, guideline, and hand-outs | Head CT | 21.6%-points reduction in use | * |
| Pugel et al. (2018) [101] | USA | Retrospective interrupted time series | 213,532 consultations | Ambulatory care | Education, feedback and guidelines | DEXA and head CT | 23.4% reduction in use | |
| Raja et al. (2012) [103] | USA | Retrospective cohort study | 6838 patients | Quaternary care institution | Clinical decision support system | Chest CTA | 20.1% reduction in use | * |
| Raja et al. (2015) [102] | USA | Prospective randomized controlled trial | 2167 patients | Urban level 1 adult trauma center | Feedback | Chest CTA | 2/1000 reduction in use | * |
| Reiter et al. (2018) [104] | Israel | Prospective cohort study | 544 children | Pediatric Emergency department | Education, guideline cards at computers | Chest XR for Bronchiolitis | 20%-points reduction in use | * |
| Rezaii et al. (2020) [105] | USA | Cohort study | 27 practices, 4601 cases | Academic and private practices | Online educational material and feedback | Chest CTA, advanced imaging of low back pain, follow-up of adnexal cysts | 3% reduction in use | * |
| Rosati et al. (2015) [106] | USA | Retrospective review | 233 children | Level I trauma center | Guideline | C-spine CT | 23% reduction in use | |
| Sclafani et al. (2010) [107] | USA | Retrospective chart review | 1092 patients | Hospital | Education | Head CT, brain MRI and carotid US of syncope patients | No significant difference | |
| Shah et al. (2016) [108] | USA | Chart review | 824 children | Emergency department | Diagnostic algorithm | Abdominal CT appendicitis | 51.2%-points reduction in use | |
| Shelton et al. (2015) [131] | USA | Before-and-after study | 2001 patients | Hospital | Feedback | Several | 38% reduction in use | * |
| Singer et al. (2014) [109] | USA | Before and after trial design | 34,961 children | Academic medical center | Opening a dedicated pediatric ED | Several | 3.2% reduction in use | |
| Sodickson et al. (2011) [110] | USA | Retrospective chart review | 1487 patients | Tertiary care, level I trauma center | Health InformationExchange | Several | 16–18% reduction in use | |
| Sy et al. (2016) [111] | Canada | Evaluation of a quality improvement project | 1492 patients | Intensive care unit | Education, posters, and change in order system | Chest XR | 26% reduction in use | * |
| Tajmir et al. (2017) [124] | USA | Randomized controlled trail | 613 patients | Hospital | Decision support | Ankle XR | Modest change in diagnostic yield | |
| Tyler et al. (2018) [112] | USA | Evaluation of a quality improvement project | 2211 patients | Children’s hospital | Education, feedback, sign pledge, visualizing algorithm, guideline | Chest XR | 12.1%-points reduction in use | |
| Vartanians et al. (2010) [113] | USA | Retrospective study | 118,975 orders | Several | Change in ordering system | Several | 3.5%-points reduction in use | * |
| Walen et al. (2016) [125] | USA | Prospective observational study | 250 patients | Hospital | Wells-score documentation | Chest CTA | 6.6% increase in diagnostic yield | |
| Walker et al. (2020) [114] | Canada | Retrospective chart review | 302 Consultations | Primary care | Electronic communication system | Abdominal, musculoskeletal, neuro, and thoracic imaging | 28% reduction in use | |
| Wang et al. (2018) [115] | USA | Retrospective chart review | 3 clinics | Primary care | On-site and online education and feedback | Lumbar MRI | 3.7 MRIs reduced per month | |
| Wu et al. (2020) [116] | USA | Evaluation of a quality improvement project | 6441 Chest XRs | Medical Intensive care unit | Survey, journal club, discussions, posters, alerts in electronic referral system, education, pocket cards, electronic decision tool | Chest XR | 36.1% reduction in use | |
| Xu et al. (2020) [117] | Canada | Retrospective review | 400 referrals | Tertiary care center | Mandatory check list | Knee MRI | 48% reduction in use | * |
| Zafar et al. (2019) [118] | USA | Randomized cohort study | 54 providers | Tertiary academic health system | Algorithm and alerts | Low back imaging | No significant difference | * |
| Zamora-Flores et al. (2015) [119] | USA | Retrospective chart review | 322 children | Rural community hospital | Guideline | Chest XR | No significant difference |
CT computed tomography, CTA CT angiography, DEXA dual-energy x-ray absorptiometry, MMAT Mixed Methods Appraisal Tool, MRI magnetic resonance imaging, MUCG micturating cystourethrogram, NM nuclear medicine, PET positron emission tomography, US ultrasound, XR X-ray
* Fulfill all MMAT criteria
Italic = Mixed results, Bold = Reduction in use/increase in rate of diagnostic yield, Normal = no significant difference/increase