Abstract
Background
Nudge interventions are those that seek to modify the social and physical environment to enhance capacity for subconscious behaviours that align with the intrinsic values of an individual, without actively restricting options. This study sought to describe the application and effects of nudge strategies on clinician implementation of health-related guidelines, policies and practices within studies included in relevant Cochrane systematic reviews.
Methods
As there is varied terminology used to describe nudge, this study examined studies within relevant systematic reviews. A two-stage screening process was undertaken where, firstly, all systematic reviews published in the Cochrane Library between 2016 and 2018 were screened to identify reviews that included quantitative studies to improve implementation of guidelines among healthcare providers. Secondly, individual studies within relevant systematic reviews were included if they were (i) randomised controlled trials (RCTs), (ii) included a nudge strategy in at least one intervention arm, and (iii) explicitly aimed to improve clinician implementation behaviour. We categorised nudge strategies into priming, salience and affect, default, incentives, commitment and ego, and norms and messenger based on the Mindspace framework.
Synthesis
The number and percentage of trials using each nudge strategy was calculated. Due to substantial heterogeneity, we did not undertake a meta-analysis. Instead, we calculated within-study point estimates and 95% confidence intervals, and used a vote-counting approach to explore effects.
Results
Seven reviews including 42 trials reporting on 57 outcomes were included. The most common nudge strategy was priming (69%), then norms and messenger (40%). Of the 57 outcomes, 86% had an effect on clinician behaviour in the hypothesised direction, and 53% of those were statistically significant. For continuous outcomes, the median effect size was 0.39 (0.22, 0.45), while for dichotomous outcomes the median Odds Ratio was 1.62 (1.13, 2.76).
Conclusions
This review of 42 RCTs included in Cochrane systematic reviews found that the impact of nudge strategies on clinician behaviour was at least comparable to other interventions targeting implementation of evidence-based guidelines. While uncertainty remains, the review provides justification for ongoing investigation of the evaluation and application of nudge interventions to support provider behaviour change.
Trial registration
This review was not prospectively registered.
Keywords: Implementation intervention, Nudge, Healthcare provider, Guidelines, Review
Contributions to the literature.
This review describes the application and potential impact of nudge strategies on clinician implementation behaviour for trials included in Cochrane systematic reviews, and seeks to add to the field by highlighting strategies that may be typically overlooked in design of implementation interventions.
This review suggests that nudge strategies, even when not part of a multicomponent intervention, could be useful in improving clinician behaviour.
There is a need to better describe and classify nudge strategy in order to fully understand its impact on implementation in the healthcare setting.
Introduction
Evidence-based guidelines are developed specifically to improve the effectiveness of healthcare providers professional practice, reduce the risk of unintended adverse effects to the community and are fundamental tools for the translation of research into practice [1]. Research studies, however, consistently document poor implementation of evidence-based clinical guidelines [2]. Consequently, a large volume of research examining ‘how’ to best support healthcare providers to implement evidence-based clinical guidelines now exists. The Cochrane Effective Practice and Organisation of Care (EPOC) group have published over 100 systematic reviews which describe interventions designed to improve professional practice and the delivery of effective health services by implementing evidence-based clinical guidelines and practices [3]. A systematic review of reviews in primary care alone found 91 reviews which evaluated strategies such as audit and feedback, educational meetings and other practitioner targeted interventions, to improve clinical guideline adherence [4]. The intervention approaches and interpretations synthesised within these reviews have largely assumed that practitioners behave in a rational manner. As such, these interventions have actively targeted rational constructs such as attitudes, intentions and self-efficacy in an attempt to change behaviour.
However, recent behavioural research suggests that in many instances, an individual’s behaviour and decision-making is not always perfectly rational [5]. Dual process models propose that individual decision-making and behaviour results from the interaction between two cognitive processes operating in parallel, one reflective, and the other impulsive or automatic [6]. In the provision of diabetes care, for example, studies have reported that ‘automatic’ decision-making processes operate alongside, and may mediate rational processes in influencing clinician provision of evidence-based care [7]. Additionally, a recent meta-analysis of nine studies assessing the association between ‘habit’ (which is an indicator of the automatic decision-making process) and clinician behaviour reported a moderate correlation (r = 0.33) [5, 8]. Such research suggests that interventions to change clinician behaviour need to move beyond strategies that focus purely on rational cognitive pathways, towards considering the context within which individual’s act, which are influenced by automatic pathways.
Nudge strategies have been suggested as one way to influence habitual or automatic behaviour, by targeting the subconscious routines and biases that are present in human decision-making and behaviour [9]. ‘Nudging’ was first coined in 2008 and is defined as a ‘function of any attempt at influencing people’s judgement, choice or behaviour in a predictable way, that is made possible because of cognitive boundaries, biases, routines and habits in individual and social decision-making’ (p158) [9]. These boundaries, biases and routines act as barriers for people to perform rationally consistent with their internal values. As such, nudge strategies work by targeting these boundaries, biases and habits by altering the underlying ‘choice architecture’, the social and physical environment in which the decision is made [9]. Specifically, ‘choice architecture’ involves the design of different ways in which choices can be presented to individuals. This can include influencing the range of choices (e.g. increasing number, types of choices), considering the manner in which the attributes are described (e.g. labelling, priming) and altering the way in which an object is presented (e.g. as a default, placement, presentation, sizing) [10]. This enhances the capacity for subconscious behaviours that aligns with the intrinsic values of an individual, without actively restricting options [11].
Nudge strategies are considered highly appealing from a policy and public health perspective as they are low cost and typically do not require ongoing resources to sustain. These interventions have been applied in public health policy to change behaviour and support healthier lifestyle choices. For example, many governments have applied nudges in the form of altered defaults, switching from opt in to opt out systems to increase organ donation rates. The United Kingdom (UK) Institute for Government and Behavioral Insights Team developed the Mindspace framework as a way to support the application of nudge strategies in public policy [12]. This framework describes the nine types of interventions (Mindspace: Messenger, Incentives, Norms, Defaults, Salience, Priming, Affect, Commitments, Ego) that are considered to have the most robust effects on the automatic system. Mindspace is underpinned by core principals of behavioural economics and aligns closely with other seminal lists describing nudge and choice architecture techniques [12, 13]. As such, this framework provides a structured way to support administrators, policy makers and researchers with selecting and applying nudge interventions to influence behaviour [12].
The potential impact of nudge strategies on clinician implementation behaviour has only recently started to be formally evaluated. A randomised controlled trial by Meeker found that a simple intervention encouraging practitioners to display a public poster stating their commitment to reduce inaccurate antibiotic prescription in waiting rooms, significantly reduced inappropriate prescribing rates relative to the control group (− 19.7% [95% confidence interval: − 5.8% to − 33.4]; p = 0.02) [14]. The UK Behavioural Insights team undertook a randomised 2 × 2 factorial trial examining the impact of providing social norm feedback to high antibiotic-prescribing GPs within their team. The study found that providing information concerning providers’ prescribing rate, compared with other local practices (norm) in the area from England’s chief medical officers (messenger), significantly reduced the rate of antibiotic items dispensed per 1000 population (p < 0.0001) [15].
Given the potential of ‘nudge’ strategies to impact on clinicians’ behaviours, efforts to describe the application and potential effect of such strategies on implementation are warranted. An overview of the type of settings and behaviours that have been targeted can highlight opportunities for future empirical research, and inform the design of low-cost implementation strategies.
This review aims to describe the application and effects of nudge strategies on healthcare provider and organisations’ implementation of evidence-based guidelines, policies and practices. The review will use data from randomised controlled trials included within Cochrane systematic reviews
Methods
This review has been reported in accordance to the PRISMA guidelines [16]. This review was not registered, and a protocol has not been previously published.
Information sources and search strategy
The nudge terminology was developed in 2008 and has gained popularity in the last decade. Implementation trials testing the impact of nudge strategies, however, may have been performed over many decades. Many interventions that would have been classified as nudge are not published under this term. Given this, conventional electronic searches of bibliographical databases are unlikely to be sufficiently sensitive or specific, and are likely to result in conflated numbers and potentially missed studies. To avoid this, we conducted a targeted and systematic search of studies included within eligible Cochrane systematic reviews. The Cochrane library was chosen as it is internationally recognised for publishing up to date, high-quality and current systematic reviews in healthcare settings, and has a review group dedicated to publication of studies to improve healthcare professional practices [17]. We undertook a two-stage screening process, where systematic reviews were screened for eligibility and following that, trials within eligible reviews identified in the first process assessed for inclusion. Two authors (SLY, FS) screened the titles, abstract and full text of all reviews published by the Cochrane library in the last 2 years (2016–2018) on December 2018. This timeframe was selected as Cochrane authors are encouraged to update their reviews every 2 years.
Study selection
Following this, full text of all included studies within eligible reviews were screened by at least two authors (SLY, FS, RS). Studies were included if they met all eligibility criteria described below. All disagreements were resolved via a consensus process between the two reviewers and involved a third reviewer (LW) where necessary.
Eligibility criteria
All studies that examined the impact of a nudge strategy targeting clinician and healthcare organisation’s implementation of health-related guidelines, policies and practices were included if they met the following criteria.
Types of studies
Only randomised controlled trials (RCTs) with a parallel control group that compared (i) an intervention that included a nudge strategy to improve the implementation of a healthcare-related guideline, policy and practice in healthcare settings/organisations compared with a non-nudge intervention or usual practice; and (ii) two or more different strategies, which included at least one arm with a nudge strategy, to improve the implementation of a healthcare-related guideline, policy and practice were included. Studies also had to specify the implementation of a health-related policy, guideline and practice as an explicit aim of the study, and as such were likely to be type 2 or type 3 hybrid trials [18].
Type of participants
Study participants were clinicians (medical doctors, allied health professionals) providing care in healthcare/clinical settings. Healthcare settings included acute care hospitals; long-term care facilities, such as nursing homes and skilled nursing facilities; physicians’ offices (i.e. primary care); urgent-care centres; outpatient clinics; home healthcare (i.e. care provided at home by professional healthcare providers) and emergency medical services [19].
Type of intervention
The intervention had to include at least one nudge strategy. The determination of whether a nudge strategy was present was undertaken by at least three reviewers (SY, FS, AA) for each study. Nudge strategies were defined as those that ‘applied principles from behavioural economics and psychology to alter behaviour in a predictable way without restricting options or significantly changing economic incentives’ (p6) [11]. These strategies were those that targeted the automatic decision-making processes rather than the rational decision-making processes [20]. Strategies were classified using the Mindspace framework (see Table 1) [20]. The Mindspace framework was chosen as it provides a practical checklist for summarising the application of nudge strategies in public health practice. Similar to a previous review [21], strategies were classed as (i) priming, (ii) norms and messenger, (iii) salience and affect, (iv) default, (v) commitment and ego and (vi) incentives [22]. For studies with multiple intervention arms (e.g. multi-arm RCTs, factorial trials or comparative effectiveness trials), only the arm/s that included an intervention with a nudge strategy were included. Multiple intervention arms with nudge strategies were combined as in most instances the intervention arms included the same type of nudge strategy. Where the intervention was multicomponent and included both nudge and non-nudge strategies, this was also included. Trials that included a nudge strategy in the control arm were excluded to allow the impact of the nudge strategies to be assessed relative to no nudge strategy.
Table 1.
Nudge categories and description applied in the review based on the Mindspace framework
| Categories of nudges using the Mindspace framework | Description |
|---|---|
| Priming | Subconscious cues which might be physical, verbal or sensational and are changed to nudge a particular choice |
| Salience/affect | Novel, personally relevant vivid examples and explanations that are used to increase attention to a particular choice |
| Default | A particular choice is ‘preset’, making it the easiest option |
| Incentive | Incentives to reinforce a positive choice, or penalties to discourage a negative choice. Such incentive however should not be enough to result in economic gains |
| Commitment/ ego | Making a commitment/ ego or public promise in order to elevate one’s desire to feel good about themselves |
| Norms and messenger | Using the practices of peers or others to establish a norm. People of status, professional organisations and peer leaders used to communicate with individuals |
Mindspace Messenger, Incentives, Norms, Default, Salience, Priming, Affect, Commitment, Ego
Type of outcomes
We included any subjective or objective measure of implementation outcomes. Similar to previous reviews carried out by the team [23, 24], implementation outcomes were those that described the fidelity or execution of a guideline, policy or practice at an organisational or practitioner level. Such outcomes could be assessed via self-reported surveys, observations or from other routine data sources including electronic medical records. Examples of such outcomes include appropriate prescribing or test ordering in line with guideline recommendations.
Studies were excluded if they were not published in English. There were no other exclusions beyond that specified by the original review the studies were extracted from.
Data extraction
Relevant information was extracted from the published Cochrane reviews, the primary trial and other associated papers referenced in the primary trial by at least two individuals (FS, RS, JJ, BM) using a standardised data extraction form. This included (1) study information—author name, study design, country, date of publication, type of healthcare provider/organisation, participant/service demographic/socioeconomic characteristics and number of experimental conditions; (2) characteristics of the overall implementation strategy, including the duration, number of contacts and approaches to implementation, and information to allow classification of the intervention strategy into nudge categories according to Table 1 (priming, norms and messenger, salience and affect, commitment and ego, incentives and default nudge); (3) trial primary or summary outcome measures and results, including the data collection method, validity of measures used, effect size (or raw data that allowed the calculation of an effect size) and measures of outcome variability; and (4) risk of bias assessment as published in the Cochrane reviews [25]. Where several implementation outcomes were reported, we extracted only the results and risk of bias assessment for those explicitly described as the primary outcome(s) of the trial, for all follow-up time periods. Where the primary outcome was not specified in the individual trial, we extracted the variable(s) described in the sample size calculation.
Data analysis
To describe the application of nudge strategies in practice, we calculated the number and percentage of trials using each nudge strategy, according to the Mindspace framework. We also described the application of nudge strategies according to setting and type of outcomes assessed. Where the primary outcomes were clearly identified in the aims or via sample size calculation, we calculated the within-study effect for this outcome. Where there were several primary outcomes, we focused on all implementation outcomes and calculated a pooled effect size for that study, if the outcomes were similar. Where they were different, we calculated the within-study effect for each outcome separately. We also calculated the within-study effects for outcomes reported at multiple time-points for each time-point.
Due to substantial clinical and methodological heterogeneity of included trials, it was not appropriate to conduct a meta-analysis. Instead, we summarised the effect estimates and used vote-counting methods, as outlined in the Cochrane Handbook, for where a meta-analyses was not possible [26]. We calculated within-study point estimates and 95% confidence intervals (CIs). For all studies, we extracted the raw values (mean, standard deviation, median interquartile range (IQR) and range for continuous outcomes; and percentages and frequencies for dichotomous outcomes). We used this data in the estimation of within-study effects. For continuous outcomes, we calculated the effect size as the difference in follow-up scores between intervention and control, except for one study that only reported the change in outcomes for control and intervention. The difference in change scores was used for this study [27]. For dichotomous outcomes, we calculated odds ratio (OR) as the measure of intervention effect. Odds ratios were chosen as it is a relative measure that is less sensitive to differences in baseline values than absolute measures such as risk differences. Additionally, ORs are also not as influenced by the underlying prevalence of the outcome as other relative measures [28]. We calculated within-study effects for all outcomes together with 95% CIs. The direction of a favourable intervention effect varied across studies, with some studies aiming for a reduction, and others aiming for an increase in a behaviour. Where studies aimed for a reduction, we reverse-scored these values. To provide an overview of overall impact, and by nudge strategy, we reported the number of studies and outcomes with an estimated effect in the beneficial direction as well as the percentage of effects favouring the intervention. These results were summarised in harvest plots, which visually demonstrate the directional effects of an intervention strategy and are recommended to help summarise review results when meta-analysis is not appropriate [29]. Finally, we calculated the median standardised mean effect size for continuous outcomes and median OR for dichotomous outcomes and IQR.
Statistical analyses were programmed using SAS v9.4 [30], Stata v13.0 and R [31, 32].
Clustered studies
All clustered trials were examined for unit of analysis errors to calculate within-study effects. For cluster randomised trials, the effective sample size was calculated and used for all estimates of effect sizes. This was undertaken to allow for inclusion in the harvest plots so that the potential impact of nudge strategies can be considered in light of the size of the studies. To calculate the effective sample size, the intracluster correlation co-efficient (ICC) derived from the trial (if available) or from another source (for example, the ICC used in the sample size calculation, or the mean ICC calculated from the other included studies) was used, and the design effect calculated using the formula provided in the Cochrane Handbook for Systematic Reviews of Interventions [33].
Studies with more than two treatment groups
Procedures described in the Cochrane Handbook for Systematic Reviews of Interventions [33] were followed for trials with more than two intervention or comparison arms that included a nudge strategy. This involved combining multiple intervention arms following the recommended formula set out by the Cochrane Handbook. Only intervention arms that included relevant nudge strategies as part of the intervention package were combined and compared to the control group. Multiple comparison arms were combined, rather than described separately, to help focus this review.
Results
Review characteristics
A total of 1730 systematic reviews published in the last 2 years (2016–May 2018) were obtained from the Cochrane library database. Forty-three full-text reviews were included in the full text screen and of those, 36 reviews were excluded for the following reasons: did not examine implementation outcomes (n = 33) [34–66], not undertaken in healthcare settings (n = 1) [67], not quantitative (n = 1) [68], or was a review of previous systematic reviews (n = 1) [69].
A total of seven reviews that examined a range of health-related practices including antibiotic prescribing [70], hand hygiene [71], management of obesity [72], management of musculoskeletal conditions [73], uptake of clinical guidelines across health behaviours [74, 75], and provision of mental healthcare [76] were included.
From these seven included reviews, 55 eligible RCTs that met all the inclusion criteria were included. Of these, 13 studies were excluded from the analysis: six did not report sufficient detail for calculation of a within-study effect [77–82], six included a nudge strategy in the control group [83–88], and one reported using an inconsistent outcome (time to event) [89]. Thus, a total of 42 studies, reporting across 57 outcomes, were included in the final analyses. Figure 1 contains a PRISMA flowchart of the study selection process.
Fig. 1.
PRISMA flowchart of study selection process
Study characteristics
Table 2 provides an overview of the included studies. Most studies were conducted in the USA (40%; n = 17), Canada (21%; n = 9) and the UK (14%; n = 6). Over half included two experimental arms, including the control (n = 26; 62%), 24% included three arms (n = 10) and 14% (n = 6) included four arms. Over half of the studies were cluster trials (52%, n = 22), one was a cross-over RCT [102] and the remainder were simple RCTs. Over half of the studies employed a control group that consisted of usual care or no intervention strategy (62%, n = 26). Over a quarter used a minimal intervention (26%, n = 11) which included strategies such as provision of guidelines, written recommendations, and introduction of facilities or equipment, and 7% (n = 3) used an intervention as part of the control, including education sessions to provide information; reminders and feedback targeting adequate products and facilities, provision of encouragement, monitoring and feedback and support. A further two studies [123, 125] did not provide sufficient detail of the control group.
Table 2.
Study characteristics of included trials in the review by nudge strategy
| Author name, year of publication, study design, country | Setting/healthcare professional | Number of experimental conditions | Guidelines targeted | Control group description | Multicomponent strategy (yes (Y)/no (N) ); Description of intervention | Description of nudge strategy | Implementation outcomes (primary), time-point/s | Data collection method | Effect size—SMD/OR (95% CI) reversed |
|---|---|---|---|---|---|---|---|---|---|
| Priming nudge | |||||||||
| Baer 2015 [90] , cluster RCT, USA | 12 primary care practices affiliated with an academic medical centre | 2 | Obesity management | Usual care | Y; Educational presentation; resources; additional information about the electronic health record & guidelines | Prompts to assess if body mass index is not assessed within the last year, reminders and other resources provided at point of care | Percentage of patients with a documented body mass index in the medical record within 12 months after initial visit | Collected as part of routine medical records (the electronic health records or scheduling systems) | OR 0.91 (0.30,2.75) |
| Barnett 1983 [91], RCT, USA | One physician group practice | 2 | None specific | Usual care | N; Physicians were sent reminders that they had deviated from standard care, and also an encounter form to record when next follow-up should occur. Another reminder was sent if follow-up specified by physician was not completed | Automated computer-generated reminder and encounter form to record when next follow-up should be | Adherence to quality assurance recommended programme; follow-up attempted or achieved at 6-12 months and 12-24 months | Self-encoding checklists unique to each specialty are completed by the physician or nurse at the time of the patient encounter and subsequently entered in to the system (usually on the same day) by record room personnel | OR 15.90 (6.32,39.99) |
| Burack 1996 [92], RCT, USA | Two sites of a health maintenance organisation serving an urban, minority population | 4 | Cancer screening | Usual care | N; Group 1: Patient reminders only; Group 2: Physician reminder only; Group 3: Patient and physician reminder | A brightly colour notice placed in medical record for women who had mammography due (Group 2 and 3) | Visit to the primary care doctor and completion of a mammogram in the study year; approximately 12 months | Electronic records | OR 1.33 (1.01,1.75) |
| Chambers 1989 [93], RCT, USA | One family practice centre with 28 healthcare providers | 2 | Cancer screening | Usual care | N; Date of the last mammogram ordered and entered into the database was displayed in the comments section of the encounter form for each visit. This information was printed as ‘last mammogram: date’, or, if no mammogram was on record in the encounter form database (i.e. none since 1984), the notation was listed as ‘last mammogram?’ | Date of the last mammogram ordered and entered into the database was displayed in the comments section of the encounter form for each visit | Up to date with the American Cancer Society guidelines for mammography (at end of intervention, 6 months follow-up) | Physician recorded ordering of mammograms on a patient encounter form which is entered into a patient registration database | OR 1.40 (1.08,1.82 |
| Chambers 1991 [94], cluster RCT, USA | Family Practice Center of the Department of Family Medicine at Thomas Jefferson University | 3 | Vaccination | Usual care | N; Reminders identifying patients as eligible for the vaccine were printed on the encounter form according to the assigned group of the patient's primary physician. These reminders were provided for appropriate patients at every visit during the 2-month study period until the physician responded by ordering the vaccine. When the billing record showed the procedure had been performed, the computer programme removed the reminder message from the encounter form | Reminders identifying patients as eligible for the vaccine were printed on the encounter form according to the assigned group of the patient's primary physician | Percentage who received influenza vaccine; post-intervention (2 months) | Patient chart review (computerised database), adherence to the Immunization Practices Advisory Committee recommendation | OR 1.80 (1.09,2.98) |
| Fisher 2013 [95], RCT, Singapore | Three wards within 1 hospital (Singapore) | 2 | Hand hygiene | Usual care | Y; Wireless monitoring system of hand hygiene with reminders and individual feedback | Wireless monitoring system of hand hygiene with reminders and feedback | Hand hygiene compliance on entry/exit of patient zone within 10-week period | Electronic hand hygiene monitoring system. Compliance was registered when hand hygiene occurred within preset times of entering (6 s) or exiting (1 min) a patient zone |
2.5 mth (entry) SMD 0.17 (−0.09,0.44) 2.5 mth (exit) SMD 0.39 (0.12,0.66) 1.5 mth (entry): SMD: 0.62 (0.35,0.89) 1.5 mth (exit) SMD: 0.45 (0.19,0.72) |
| Goodfellow 2016 [96], cluster RCT, England | 30 general practices in the East Midlands of England | 2 | Obesity management | Usual care | Y; Tailoring, training and educational resources for healthcare professionals (including a presentation, discussion and provision of the resources, e.g. patient booklets, body mass index charts, calories and portions leaflets, posters, information on referral pathways) | Posters for consulting rooms containing information on how to measure waist circumference were given as a visual reminder | Proportion of overweight or obese patients to whom the health professional had offered a weight loss intervention within the study period; 9-month follow-up | Data collection was blinded and used a standard electronic system that extracted data from the general practice electronic health records and, to minimise bias, all data were collected using full anonymisation using electronic data extraction queries suitable for the different types of general practice computer systems used in England | OR 0.88 (0.45,1.72) |
| King 2016 [97], RCT, USA | One surgical intensive care unit at a hospital in Miami | 4 | Hand hygiene | Usual care | N; Visitors to an intensive care unit were exposed to an olfactory prime – a clean citrus smell that was introduced to the environment through a commercially available aroma dispenser. Visitors to an intensive care unit were exposed to a photo of eyes prominently displayed above the gel dispenser. In half the sessions a photo utilising clearly, female eyes was used and in the other photo male eyes | Olfactory prime (clean citrus small via aroma dispenser); Visual prime (photo of male or female eyes displayed prominently above the gel dispenser) | Observed hand hygiene compliance, 12 sessions of 3-h observations over a 3-month period | Direct observation | OR 3.18 (1.82,5.55) |
| Lafata 2007 [98], cluster RCT, USA | 15 primary care clinics | 3 | Bone density screening | Usual care | N; Group 1: Initial and 1-month follow-up patient mailings were sent to women receiving the intervention. A third was sent to only those whose result indicated a need for follow-up. Group 2: As for Group 1 + physician prompts | Physician prompt in the electronic medical record and a biweekly letter to physician | Percentage who had bone mineral density testing/screening; 12 months | Health record data | OR 3.34 (2.29,4.88) |
| Le Breton, 2016 [99], cluster RCT, France | 144 GPs, who provided care for any reason to 20,778 patients eligible for colorectal cancer screening between June 2010 and November 2011 | 2 | Cancer screening | Usual care | N; Three reminders were mailed to GPs at 4-month intervals | Reminders contained lists of patients who had not performed a scheduled faecal occult blood test (FOBT) | Patient adherence to FOBT screening within the 17-month study period | Patient review database from the main French statutory health insurance programme and local screening programme | OR 1.08 (0.95,1.23) |
| Lobach 1997 [100], RCT, USA | 58 primary care providers (20 family physicians, 1 general internist, 2 physician’s assistants, 2 nurse practitioners, 33 residents), outpatient setting, diabetic patients | 2 | Diabetes management | Usual care | N; Computer-Assisted Management Protocol consistent with the diabetes guideline recommendations. Protocol was printed on the first page of the paper encounter form and provided the customised diabetes guideline recommendations based on practice standards and previously completed tests, and an area for handwritten updates by the clinician to capture data not previously stored in the medical record | The protocol generated a set of disease-specific care recommendations customised to an individual patient that advised the clinician regarding which studies/procedures should be done during the current visit and which studies/procedures were next due | Clinician compliance rate with regards to care guidelines for diabetes mellitus overall (number of recommendations completed/total number of recommendations due); 6-month study period | Medical chart audit & review of computer-generated lab test summaries | SMD 0.97 (0.75,1.19) |
| Martin-Madrazo 2012 [101], cluster RCT, Spain | 198 healthcare workers within 11 primary healthcare centres | 2 | Hand hygiene | Usual care | Y; Multimodal strategy based on World Health Organization (WHO) – posters; education sessions, availability of alcohol-based hand rub (4 × 50-min teaching sessions) | Hydroalcoholic solutions were placed in each consultation office | Hand hygiene compliance level; 6-month follow-up | Direct observations. | OR 2.93 (1.18,7.29) |
| Munoz-Price 2014 [102], cross-over RCT, USA | 40 anaesthesiology providers at one, 1500-bed teaching hospital in Florida | 2 | Hand hygiene | Minimal intervention: Wall-mounted hand sanitiser dispensers only | N; Intervention involved using a hand sanitiser dispenser on the anaesthesia machine in addition to the standard wall-mounted dispensers | Additional hand sanitiser dispenser on anaesthesia machine | Frequency of hand hygiene, defined as the number of hand hygiene events per hour of observation, within 30 days, the same subjects were evaluated again in the opposite allocation | Direct observation | SMD 0.44 (−0.19,1.07) |
| Rogers 1982 [103], RCT, USA | Physicians, 479 Northwestern University Clinic patients | 2 | Hypertension management | Usual care | N; A computer printout of a current computerised medical record system summary in addition to the traditional medical record | A computerised medical record system was developed to provide physicians with concise and current information on patient’s problems, to identify omissions in recording of observations and treatment recommendations, to show ordered procedures that were not carried out, to record deficiencies in medical reasoning, and to recommend corrective actions according to selected criteria | Hypertension renal function examination (done both years). Obesity: number of diets given of reviewed (done both years). Renal disease renal function examination (done both years); for 2-year period | Blind retrospective chart reviews by trained personnel using a standardised evaluation form. Measurement tool was developed by the research team |
Hypertension renal function examination OR 1.54 (1.03,2.31) Renal disease renal function examination OR 1.89 (0.85,4.20) Obesity number of diets given or reviewed OR 2.01 (0.96,4.23) |
| Rossi 1997 [27], cluster RCT, USA | 71 primary care providers within one general internal medicine clinic | 2 | Prescribing | Usual care | N; Reminder was attached to the medication refill forms that are given to providers at every patient visit | One-page guideline reminder placed in the patient chart by the clinic pharmacist. The reminder highlighted the prescription and offered alternative drugs and doses | Prescription change rate. The percentage changed from calcium channel blocker after 6 months | Patient chart review via computer system | OR 30.40 (4.08,226.35) |
| Schnoor 2010 [104], RCT, Germany | 8 Local Clinical Centres (11 hospitals & 34 sentinel practices) | 2 | Prescribing | Usual care | Y; Audit & feedback, educational meetings with dissemination of guideline, reminders | GPs and physicians received a poster, a short-printed version and an electronic version of the guideline | Adherence to the guideline was analysed for the following variables: initial site of treatment, empiric initial antibiotic treatment and duration of antibiotic treatment. After a training period of 1 month, process of care after guideline implementation (1 April 2007 to 29 February 2008) was compared with the treatment before (1 September 2006 to 28 February 2007) | Data of the recruited cases were entered by the personal tutor in-time, electronically using a standardised electronic report form (case report form) in a central database |
Antibiotic treatment in outpatients OR 1.27 (0.91,1.77) Duration of antibiotic treatment in inpatients OR 0.93 (0.65,1.32) Duration of antibiotic treatment in outpatients OR 2.11 (1.47,3.02) Antibiotic treatment in inpatients OR 1.70 (1.19,2.42) Initial site of treatment OR 1.75 (1.21,2.55) |
| Shojania 1998 [105], RCT, USA | 396 physicians in one tertiary-care teaching hospital | 2 | Prescribing | Usual care | N; A computerised guidelines screen appeared whenever a clinician in the intervention group initiated an order for intravenous vancomycin. Another guidelines screen is displayed after 72 h of therapy asking providers their indication for continuing vancomycin use | Showing computerised guidelines for vancomycin ordering at the time of initial vancomycin ordering and after 72 h of therapy | Number of vancomycin orders and duration of vancomycin therapy prescribed by providers; 9-month period | Vancomycin orders were obtained from computer log, monthly utilisation of vancomycin in the hospital was obtained from the pharmacy system. |
Total number of vancomycin orders SMD 0.22 (0.02,0.41) Vancomycin days per physician SMD 0.23 (0.02,0.44) |
| Thompson 2008 [106], cluster RCT, England | 19 acute mental health units in 4 local mental health trusts (667 nurses/doctors) | 2 | Prescribing | Minimal intervention: Received guidelines on antipsychotic polypharmacy | Y; An educational/cognitive behavioural therapy workbook; an educational visit to consultants; a reminder system on medication charts | A medication chart reminder system was developed. Ward pharmacists applied removable reminder stickers to medication charts when participants were prescribed more than 1 antipsychotic | Antipsychotic polypharmacy prescribing rates for each unit (cluster); 5 months | Information was collected from patients’ medication charts using a 1-day cross-sectional survey of antipsychotic prescribing pre- and post-intervention | OR 1.05 (0.66,1.68) |
| Yeung 2011 [107], cluster RCT, Hong Kong | Six residential long-term care facilities (188 nursing staff) | 2 | Hand hygiene | Intervention: Attended basic life support workshop (not hand hygiene) | Y; Education sessions, feedback, reminders | Pocket-sized containers of antiseptic hand rub were provided and kept close to clinicians. | Adherence to hand hygiene; 2-week intervention period followed by 7-month post-intervention | Direct observation | OR 1.17 (0.72,1.90) |
| Norms and messenger nudges | |||||||||
| Cranney 2008 [108], cluster RCT, Canada | 119 primary care practices (174 clinicians) | 2 | Osteoporosis management | Usual care | N; Letter to patient and physician at 2 weeks and 2 months post-fracture | A personalised letter notified the physician that their patient had a recent wrist fracture and highlighted that wrist fractures can be associated with osteoporosis, and that assessment for osteoporosis treatment is recommended for women with wrist fractures | Proportion of women who reported they were started on osteoporosis treatment (i.e. bisphosphonates, raloxifene, hormone therapy or teriparatide) within 6 months of fracture, 6 months post-fracture | Self-report via telephone survey | OR 3.29 (1.65,6.55) |
| Engers 2005 [109], cluster RCT, Denmark | 67 eligible GPs, 531 patients with nonspecific low back pain | 2 | Low back pain management | Usual care | Y; Two-hour workshop; distribution of a half-page patient education card; the guideline for occupational physicians; 2 scientific articles concerning GP management of nonspecific low back pain; and a collaboration tool to facilitate greater agreement with physical, exercise, and manual therapists on the management of nonspecific low back pain | In addition to the workshop, GPs received printed materials including patient education, a copy of the guidelines, scientific articles (educational material) and a collaboration tool | Number of referrals to a therapist (physical, exercise, or manual therapist) within 8-month study period | GPs completed self-registration forms post consultation; patient questionnaire completed immediately after the consultation | OR 5.17 (1.73,15.39) |
| Feldstein 2006 [110], RCT, USA |
Nonprofit, group-model health maintenance organisation in the Pacific Northwest with about 454,000 members, 35% of whom were aged 50 and older and more than 90% of whom had a prescription drug benefit. 15 primary care clinics and 159 primary care providers (range 1–3 patients per provider) |
3 | Osteoporosis management | Usual care |
Y; Group 1: electronic medical record message about participant risk of osteoporosis and distribution of educational materials Group 2: As for Group 1 + patient-directed component |
Primary care providers received patient-specific electronic medical record in-basket messages for their enrolled patients from the chairman of the osteoporosis quality improvement committee | The primary outcome was the proportion of the study population who received a pharmacological treatment or a bone mineral density measurement within 6 months after the intervention | Identified electronically from the outpatient pharmacy system of data from the referral site (on bone mineral density measurement) | OR 15.93 (2.13,118.93) |
| Majumdar 2008 [111], RCT, Canada | Two largest emergency departments and 2 largest fracture clinics in Capital Health (Edmonton, Alberta) (266 primary care physicians) | 2 | Osteoporosis management | Minimal intervention: Mailed osteoporosis guidelines | Y; Distribution of guidelines endorsed by local leaders; physician reminder; patient telephone counselling | Evidence-based treatment guidelines, representing an actionable summary of available osteoporosis guidelines and having endorsement from 5 local opinion leaders, were sent to these physicians | Starting treatment with a bisphosphonate within 6 months after the fracture, 6 months post-fracture | Patient self-report, confirmed through dispensing records at local pharmacies | OR 1.46 (0.70,3.08) |
| Mertz 2010 [112], cluster RCT, Canada | 30 adult hospital wards in 3 acute care sites | 2 | Hand hygiene | Minimal intervention: Alcohol-based gel dispensers were installed outside all patient rooms with at least 1 hand wash sink in each room | Y; Installation of gel dispensers as per control group + performance feedback, educational meeting & resources | Clinical managers were asked to develop a target adherence level. Meetings were held biweekly to provide unit-specific feedback. The adherence rates were shown on a large whiteboard both graphically and numerically. After 6 months, a comparison with the rates of other intervention units was provided | Rates of hand hygiene adherence, evaluated at unit level, assessed weekly for 1-year intervention period | Direct observations | OR 1.25 (0.90,1.74) |
| Shah 2014 [113], cluster RCT, Canada |
80 primary care practices 1592 patients at high risk for cardiovascular disease were selected |
2 | Prescribing | Minimal intervention: Control providers received the Canadian Diabetes Association newsletter, which included the revised guidelines for cardiovascular disease screening | N; Printed educational materials (1 toolkit including guidelines summary, laminated card with risk assessment algorithm, self-assessment tool & risk reduction strategies) | Letter from the Chair of the practice guidelines Dissemination and Implementation Committee, with guideline summary | Prescription for statin; 10 months | Trained registered nurse undertook patient chart review | OR 0.76 (0.42,1.37) |
| Salience/affect nudges | |||||||||
| Dey 2004 [114], cluster RCT, England | 24 General Practices with 2187 eligible patients | 2 | Test ordering | Usual care |
Y; Educational outreach visit; guidelines (educational material); poster of guidelines; referral forms with guidelines; access to fast-track physiotherapy and a back clinic) General Practitioners (GPs) were sent a letter offering them a visit from the guideline team, followed by a telephone call to the practice manager to arrange an appointment with the GP in their practice. At least 2 members of the guideline team attended each visit. Members of the guideline team facilitated a structured interactive discussion with the GP |
Face-to-face meeting included structured interactive discussion with the GP, which was based on the ‘elaboration likelihood model of persuasion’. This discussion was used to: raise awareness of the guidelines, adapt to the local context; emphasise the key messages in the guidelines; identify potential barriers to implementation; and suggest strategies for overcoming the barriers identified | The rate of referral for lumbar spine X-rays; 8 months | GPs were asked to log every patient presenting to them with acute low back pain: the practice was reimbursed £1 for each patient identified. A research assistant screened the records of these patients to confirm eligibility and to extract data on patient characteristics and clinical management during the 3-month period following first consultation | OR 0.89 (0.60,1.32) |
| Grant 2011 [115], RCT, USA | One hospital; with all 66 soap & gel dispensers randomly allocated to 1 of 3 signs | 3 | Hand hygiene | Minimal intervention: The control sign, which was developed by hospital managers, read, ‘Gel in, wash out’ | N; Three different signs over period of 2 weeks | Personal consequences’ sign read ‘Hand hygiene prevents you from catching diseases’. The patient-consequences sign read, ‘Hand hygiene prevents patients from catching diseases’ | Mean percentage of soap and gel used during 2-week periods before and after signs were introduced | Measured by blinded environmental services team | SMD 0.17 (−0.35,0.69) |
| Ince 2015 [116], RCT, England | 13 community mental health teams (82 individuals) | 2 | Delivery of psychological interventions | Minimal intervention: Summary of guidelines for psychological interventions for schizophrenia | N; Alternative text of National Institute for Health and Clinical Excellence Guidelines guidance for schizophrenia | Summary of guidelines re-written. Text was amended to personalise the message, use of behaviourally specific language. Checklist & decision tree produced & provided | Overall intention to follow the recommendations measured by a Theory of Planned Behaviour Total Scale Intention Score, number of participants providing psychological interventions (delivered, received training, supervision), at 1-month follow-up | Self-report questionnaire |
Intention to follow Theory of Planned Behaviour Total Scale Intention Score : SMD: 0.00 (−0.47,0.48) Received psychological training in last month OR 0.77 (0.16,3.76) Psychological interventions delivered OR 1.65 (0.58,4.66) Supervision for psychological interventions was used OR 1.69 (0.58,4.92) |
| Leslie 2012 [117], RCT, Canada | Unclear. There were n = 4264 patients randomised | 3 | Osteoporosis management | Usual care |
Y; Group 1: Notification letter to primary care physician (reminder) about the patient’s fracture accompanied by educational material Group 2: As for Group 1 + patient-directed intervention (educational material and reminder) |
A letter directed the physician to the provincial guidelines on bone mineral density testing and provided information on the management of osteoporosis. Additional information specific to the investigators research initiative was also provided. Enclosed with the letter were a requisition for a bone mineral density test and a flowchart showing the management of care | Combined end point of post-fracture bone mineral density testing or the start of medication for osteoporosis; 12 months post-fracture | Healthcare database information | OR 2.58 (2.17,3.07) |
| Priming (MPC: memorandum pocket card); default (TRF: test request form) | |||||||||
| Daucourt 2003 [118], cluster RCT, France | Six volunteer general hospitals and 1412 thyroid function tests ordered in 1306 patients | 4 (2 included in current review (TRF [test request form] & MPC [memorandum pocket card]) | Test ordering | Intervention: Physicians in all groups received guidelines and were invited to a local information meeting where guidelines were presented and discussed | N; Replacing previous order sheet with new TRF and providing small summary of recommendations on card |
TRF makes ordering of inappropriate tests impossible based on format of the form MPC designed to be summary of guidelines that can be kept in pocket |
Proportion of thyroid function test ordering in accordance with guidelines at 4 weeks after guideline implementation | Research Assistant completed data collection grid from patient medical files and test requests, or by speaking with the prescriber | OR 2.18 (1.16,4.10) |
| Priming, norms and messenger nudges | |||||||||
| Eccles 2001 [119], RCT, England | 247 General Practices enrolled. Data was abstracted from 1693 patients’ records of 162 GPs in 48 practices | 4 (2 × 2 factorial design) | Test ordering | Minimal intervention: Distribution of educational materials (guideline) |
N; Group 1: Distribution of educational materials; audit and feedback (number of practice referrals compared with peers) Group 2: Distribution of educational materials; reminders (messages on X-ray results) Group 3: Distribution of educational materials; audit and feedback; reminders All interventions had a 12-month duration |
Referral guidelines were posted to all GPs. Feedback contained the number of requests for lumbar spine and knee radiographs made by the whole practice compared with requests made by all GPs in the study was sent to GPs at start of intervention period and 6 months later. Educational messages were attached to the reports of every knee or lumbar spine radiograph requested during the 12-month intervention | The number of each radiograph (knee, lumbar, spine) requested per 1000 patients registered with every practice per year for 2 years; the second year was the intervention period | Records of radiology departments |
Mean lumbar spine radiographs SMD 0.34 (0.05,0.63) Mean knee radiographs SMD 0.39 (0.09,0.68) |
| Majumdar 2007 [120], RCT, Canada | 40 pharmacies (targeted sample size), patients with a self-reported diagnosis of heart failure or ischemic heart disease who was not taking a study medication | 2 | Prescribing | Minimal intervention: Physicians of the control subjects were faxed only their most recent medication profile | N; Five physicians were consistently identified as opinion leaders and worked with the investigators to develop the study’s evidence summaries | One-off fax of evidence summary to physician with the patient’s most recent medication profile | Improvement of prescribing for efficacious therapies in patients with a chronic cardiovascular disease within 6 months of the intervention | Patient-level medication profiles generated at each community pharmacy; outcomes measured by compliance with evidence-based prescribing recommendations | OR 1.46 (0.70,3.08) |
| McAlister, 2006 [121], RCT, Canada | Physicians at primary care practices, patients with established coronary artery disease | 3 | Prescribing | Minimal intervention: Physicians received a fax containing the coronary artery diagram for their patient |
N; Opinion leader statement group: The opinion leader statements were imprinted with the name of the participating patient, addressed directly to the patient’s physician, signed by the local opinion leaders for that city, and faxed automatically by a software programme that was developed for this trial. Unsigned statement group: The unsigned statements were identical to the opinion leader statements in content and form but did not contain the opinion leaders’ signatures. The unsigned statements were faxed to physicians in the same manner as the opinion leader statements |
Each physician received a fax containing objective evidence of the patient’s coronary artery disease (in the form of a coronary artery diagram) and either a signed or unsigned statement. These faxes were sent to physicians within a few days of the angiogram | Improved statin management, defined as initiation or increased dosage of a statin within the first 6 months after cardiac catheterisation | Medication outcomes were based on patient self-report (with cross-referencing to pharmacy records), and laboratory data and clinical outcomes were extracted from medical records | OR 1.31 (0.89,1.92) |
| Rodriguez 2015 [122], cluster RCT, Argentina | 705 healthcare workers in 11 intensive care units at acute care hospitals | 2 | Hand hygiene | Usual care | Y; Educational resources, reminders, feedback, executive support | Every month, coordinators of intervened sites received results of the indicator (compliance with hand hygiene) and they showed them in the storyboard comparing it to the best performance in study (if the site complied with <70%) or to an international performance of 95% (if the site complied with 71% or more. Reminders placed at the entrance of patient’s rooms and in common areas | Adherence to hand hygiene based on the WHO survey tool; monthly for 9 months | Direct observation (covert) | OR 1.58 (1.09,2.29) |
| Schouten 2007 [123], cluster RCT, Netherlands | Six medium-large hospitals in southeast of the Netherlands | 2 | Prescribing | Not reported | Y; Audit & feedback; educational meetings with dissemination of guidelines | Consensus ‘critical-care pathways’ were distributed to all doctors as a laminated, pocket card; desktop and personal digital assistant versions were also distributed. Feedback on indicator performance at the hospital level was presented and provided in writing to all doctors treating hospital lower respiratory tract infections. Feedback reports included benchmarks at the hospital level (best practice) and presented key issues for improvement | A sum score was calculated that determined the sum score for guideline adherence for empirical antibiotic therapy; 2 years | All data were collected by concurrent chart review; trained research assistants made twice-weekly reviews of the charts of all patients who were admitted to the internal and respiratory medicine wards | OR 2.16 (0.75,6.23) |
| Taveras 2015 [124], cluster RCT, USA | Primary care practice paediatric clinicians, children with obesity | 3 | Obesity management | Usual care | Y; Modified electronic health record to deploy a computerised, point-of-care clinical decision support alert to paediatric clinicians at the time of a well-child visit for a child with a body mass index at the 95th percentile or greater. Clinicians were trained to use brief motivational interviewing to negotiate a follow-up weight management plan with the patient and their family. A comprehensive set of educational materials were developed for paediatric clinicians to provide to their patients | An alert containing links to growth charts, evidence-based childhood obesity screening and management guidelines, and a prepopulated standardised note template specific for obesity | Body mass index percentile documentation, Healthcare performance/quality of care (nutrition/physical activity counselling documentation); baseline and 1-year follow-up | Child’s electronic health record from well-child visits, and Healthcare Effectiveness Data and Information Set (health performance) |
HEDIS Performance Measures BMI percentile documentation OR 1.49 (0.73,3.01) HEDIS Performance Measures nutrition PA counselling documentation OR 63.37 (3.81,1052.67) |
| Rahme 2005 [125], cluster RCT, Canada | GPs in eight small towns in Quebec, Montreal | 3 | Prescribing | Did not report |
Y; Group 1: Workshop which discussed evidence-based management of patients with osteoarthritis Group 2: Decision trees to support decision-making Group 3: Combination of workshop and decision tree |
Continuing medical education points and endorsement by medical bodies, delivered by peers (Groups 2 and 3) | Number of dispensed prescriptions for osteoarthritis from the Provincial Health Care fund database; 5 months pre-intervention and 5 months post-intervention (12 months) | Patient records | OR 1.52 (0.65,3.57) |
| Priming, salience, norms and messenger nudges | |||||||||
| Roux 2013 [126], RCT, Canada | Primary care physicians of an acute care hospital, 1446 patients aged 50 years or older with fragility fractures | 3 | Osteoporosis management | Usual care |
Y; Group 1: Verbal and written information on osteoporosis to patient (patient-directed component) and letter with specific management plan sent to their treating physician (GP reminder). Patient reminders at 6 and 12 months. Reminder to physician if patient untreated at 6 months Group 2: As for Group 1 + blood tests and bone mineral density test ordered for patient and results sent to the physician (patient-mediated intervention). Patient reminders at 4, 8 and 12 months and physician reminders at 4 and 8 months if patient remained untreated |
Verbal and written information on osteoporosis to patient and letter with specific management plan sent to their treating physician. Blood tests and bone mineral density test ordered for patient and results sent to the physician. Patients and physicians received reminder if patients remained untreated | Percentage change in treatment rates for osteoporosis; 1-year post-fracture | Delivery of osteoporosis medication was confirmed with the patient’s pharmacists | OR 3.05 (2.01,4.63) |
| Solomon 2001 [127], RCT; USA | 17 internal medicine services within one academic medical centre in USA | 2 | Prescribing | Usual care | Y; Educational meetings with policy dissemination; 1 x face-to-face or telephone academic detailing session with clinician who wrote the order for the 2 unnecessary antibiotics being studied | Academic detailing, patterns of antibiotic utilisation and resistance patterns in the institution | Number of days that unnecessary antibiotics (levofloxacin or ceftazidime) were administered in intervention & control services; 18-week period | Computerised pharmacy records (validated in a sub-sample of patients against the manually completed medication administration records in patient chart) | SMD 1.54 (0.44,2.64) |
| Norms and messenger, salience and incentive nudges | |||||||||
| Robling 2002 [128], cluster RCT, Wales | 39 general practices in South Glamorgan, Wales | 4 | Test ordering | Minimal intervention: single A4-sheet feedback on practice data | Y; Seminar workshop facilitated by academic GPs and researcher; videos; question and answer session | Continuing medical education point, feedback from experts, presentation of localised guidelines | Percentage concordant with local guidelines (MRI: medical resonance imaging requests); 11 months | Each MRI request was followed up, additional information assessed via follow-up interview with GPs | OR 0.59 (0.24,1.42) |
| Norms and messenger, priming and incentive nudges | |||||||||
| Solomon 2007 [129], cluster RCT, USA | 828 primary care physicians within primary care clinics | 4 (2 × 2 factorial design) (only relevant doctor arm described) | Prescribing | Usual care | Y; Educational resources that were used in a face to face educational session. Osteoporosis treatment algorithms, reminders flags and behavioural prescription packs were also provided | One hour of continuing medical education credit by Harvard Medical School were offered; reminder flags | Composite score consisting of either undergoing bone mineral density testing or initiation of medication for osteoporosis; 12 months | Patient Medicare and pharmacy claims data | OR 0.89 (0.74,1.06) |
| Norms and messenger, priming, salience and commitment nudges | |||||||||
| Stewardson 2016 [130], cluster RCT, Switzerland | Hospital ward | 3 | Hand hygiene | Intervention: Standard multimodal hand hygiene promotion activities, including monitoring and feedback, were done hospital wide throughout the study |
Y; Group 1: Audit and feedback; goal setting; executive support Group 2: As for Group 1+ patient participation (educational materials, alcohol-based handrub) |
Immediate verbal feedback and, where feasible, a card reporting individual hand hygiene compliance and individualised written advice for how to improve were provided. The card also illustrated the WHO Five Moments for Hand Hygiene and stated the institution-wide hand hygiene compliance goal (≥80%), with the signatures of the medical and nursing directors | Overall hand hygiene compliance of healthcare workers, at least once every 3 months during the baseline and intervention periods, and once every 4 months during the follow-up period | Direct observation during 20-min sessions | OR 1.10 (0.84,1.44) |
Note: RCT randomised controlled trial, USA United States of America, GP general practitioner, FOBT faecal occult blood test, WHO World Health Organization, MRI medical resonance imaging
Methodological quality of included studies
The risk of bias for each RCT as reported in the Cochrane reviews is presented in Table 3. Over half of the trials were judged as low risk for selection bias (random sequence generation (n = 33), allocation concealment (n = 28) and attrition bias (incomplete outcome data (n = 32)). A large number of trials were judged as unclear for selective reporting (n = 20) and protection against contamination (n = 14). In terms of blinding of outcome assessment, 21 trials were judged as having a low risk of bias (see Table 3). Overall, 29 studies (69%) met at least half of the criteria they were assessed against.
Table 3.
Risk of bias assessments from individual trials extracted from published Cochrane reviews
| Author name, year, study design, country | Random sequence generation | Allocation concealment | Protection against contamination | Baseline outcomes similar | Baseline characteristics similar | Blinding of outcome assessment | Incomplete outcome data | Selective reporting | Other bias |
|---|---|---|---|---|---|---|---|---|---|
| Baer 2016, cluster RCT, USA | Low risk | Unclear risk | Low risk | Not assessed | Not assessed | High risk | High risk | High risk | Unclear risk |
| Barnett 1983, RCT, USA | Unclear risk | Unclear risk | High risk | Unclear risk | Low risk | Unclear risk | Low risk | Unclear risk | Unclear risk |
| Burack 1996, RCT, USA | Unclear risk | Unclear risk | High risk | Unclear risk | Low risk | Low risk | Low risk | Unclear risk | Unclear risk |
| Chambers 1989, RCT, USA | Low risk | Unclear risk | High risk | Low risk | Low risk | Unclear risk | Unclear risk | Unclear risk | Unclear risk |
| Chambers 1991, cluster RCT, USA | Low risk | Unclear risk | Low risk | Unclear risk | Unclear risk | Unclear risk | Low risk | Unclear risk | Unclear risk |
| Cranney 2008, cluster RCT, Canada | Low risk | Low risk | Unclear risk | Low risk | Low risk | Unclear risk | Low risk | Unclear risk | Low risk |
| Daucourt 2003, cluster RCT, France | Low risk | Low risk | Not assessed | Unclear risk | Unclear risk | Unclear risk | Low risk | Low risk | Low risk |
| Dey, 2004, cluster RCT, England | Low risk | Low risk | Unclear risk | Unclear risk | Low risk | High risk | Low risk | Unclear risk | Low risk |
| Eccles, 2001, RCT, England | Low risk | Low risk | Low risk | High risk | Unclear risk | Low risk | Low risk | Unclear risk | Unclear risk |
| Engers, 2005, cluster RCT, Denmark | Low risk | High risk | Unclear risk | Unclear risk | Unclear risk | High risk | Low risk | Unclear risk | High risk |
| Feldstein 2006, RCT, USA | Low risk | Low risk | Unclear risk | Unclear risk | Low risk | Low risk | Low risk | Unclear risk | Unclear risk |
| Fisher 2013, RCT, Singapore | Low risk | Low risk | Unclear risk | Low risk | High risk | High risk | Low risk | Low risk | Low risk |
| Goodfellow 2016, cluster RCT, England | Low risk | Low risk | Low risk | Low risk | Unclear risk | Low risk | Unclear risk | Low risk | Not assessed |
| Grant 2011, RCT, USA | Unclear risk | Low risk | Low risk | Low risk | High risk | Low risk | Low risk | Low risk | Low risk |
| Ince 2015, RCT, England | Low risk | Low risk | Not assessed | Not assessed | Not assessed | Low risk | Unclear risk | Low risk | Low risk |
| King 2016, RCT, USA | Low risk | Low risk | Unclear risk | Low risk | Low risk | High risk | Low risk | Low risk | Unclear risk |
| Lafata 2007, cluster RCT, USA | High risk | Unclear risk | Unclear risk | Low risk | Unclear risk | High risk | High risk | Unclear risk | Low risk |
| Le Breton, 2016, cluster RCT, France | Low risk | Low risk | Low risk | Low risk | High risk | Low risk | Low risk | Low risk | Unclear risk |
| Leslie 2012; RCT; Canada | Low risk | Low risk | Unclear risk | Low risk | Low risk | Low risk | Low risk | Low risk | Low risk |
| Lobach, 1997, RCT, USA | Low risk | Low risk | Low risk | Low risk | Low risk | Low risk | Low risk | Unclear risk | Unclear risk |
| Majumdar 2007, RCT, Canada | Low risk | Low risk | High risk | Unclear risk | Unclear risk | Low risk | Low risk | Unclear risk | Low risk |
| Majumdar 2008, RCT, Canada | Low risk | Low risk | High risk | Unclear risk | Unclear risk | Low risk | Low risk | Unclear risk | Low risk |
| Martin Madrazo 2012, cluster RCT, Spain | Low risk | Low risk | Low risk | Low risk | High risk | Low risk | Low risk | Low risk | High risk |
| McAlister, 2006, RCT, Canada | Low risk | Low risk | Low risk | Low risk | Low risk | Low risk | Low risk | Low risk | Low risk |
| Mertz 2010, cluster RCT, Canada | Low risk | Low risk | High risk | Low risk | High risk | Unclear risk | Low risk | Low risk | High risk |
| Munoz-Price 2014, cross-over RCT, USA | Low risk | Low risk | High risk | Unclear risk | Low risk | High risk | Low risk | Low risk | Low risk |
| Rahme 2005, cluster RCT, Canada | Unclear risk | Unclear risk | Low risk | Unclear risk | Low risk | Low risk | Low risk | Unclear risk | Unclear risk |
| Robling 2002, cluster RCT, England | High risk | Unclear risk | Unclear risk | Unclear risk | Unclear risk | Low risk | Unclear risk | Unclear risk | Unclear risk |
| Rodriguez 2015, cluster RCT, Argentina | Low risk | Unclear risk | Unclear risk | Low risk | Unclear risk | Unclear risk | Low risk | Low risk | Low risk |
| Rogers 1982, RCT, USA | Unclear risk | Unclear risk | Unclear risk | Unclear risk | Unclear risk | Low risk | High risk | High risk | Not assessed |
| Rossi 1997, cluster RCT, USA | Low risk | Low risk | Low risk | Unclear risk | High risk | Low risk | Unclear risk | Unclear risk | Unclear risk |
| Roux, 2013, RCT, Canada | Low risk | Low risk | Unclear risk | Low risk | High risk | High risk | Low risk | Unclear risk | Unclear risk |
| Schnoor 2010, RCT, Germany | Low risk | Low risk | Low risk | High risk | High risk | Unclear risk | Unclear risk | Unclear risk | High risk |
| Schouten 2007, cluster RCT, Netherlands | Low risk | Low risk | Low risk | Low risk | Low risk | Unclear risk | Low risk | Unclear risk | Low risk |
| Shah 2014, cluster RCT, Canada | Low risk | Low risk | Not assessed | Low risk | High risk | Low risk | Low risk | Unclear risk | Unclear risk |
| Shojania 1998, RCT, USA | Unclear risk | High risk | High risk | Unclear risk | Low risk | High risk | Unclear risk | Low risk | Low risk |
| Solomon 2007, cluster RCT, USA | Low risk | Unclear risk | Low risk | Unclear risk | Low risk | Low risk | Low risk | Low risk | Unclear risk |
| Solomon 2001, RCT, USA | Low risk | Unclear risk | Low risk | Unclear risk | Low risk | Low risk | Low risk | Low risk | Unclear risk |
| Stewardson 2016, cluster RCT, Switzerland | Low risk | Low risk | Unclear risk | Low risk | Unclear risk | High risk | Low risk | Low risk | Low risk |
| Taveras 2015, cluster RCT, USA | Low risk | Low risk | Low risk | Low risk | Unclear risk | Low risk | Low risk | High risk | Not assessed |
| Thompson 2008, cluster RCT, England | Low risk | Low risk | Not assessed | Not assessed | Not assessed | Unclear risk | Low risk | Low risk | Unclear risk |
| Yeung 2011, cluster RCT, Hong Kong | Unclear risk | Low risk | Unclear risk | Low risk | High risk | High risk | Low risk | Low risk | High risk |
Risk of bias was assessed for the primary implementation outcome where specified
Note: Not assessed refers to where the risk of bias criteria were not assessed in the Cochrane review in which they were identified
Application of nudge strategies to improve implementation
All nudge strategies were used in at least one trial to influence healthcare provider adherence to guideline recommendations. Twenty-two studies across 30 outcomes incorporated nudge strategies as part of a multicomponent intervention, while the remainder examined the impact of nudge strategies in isolation (n = 20). Ten studies applied two nudge strategies and three applied three or more nudge strategies. The frequency to which each strategy was used and specific examples of their application are shown in Table 4.
Table 4.
Summary of application of nudge strategies in 42 randomised controlled trials included in this study
| Nudge strategy | Application in RCTs included in this review | Number (n, %) |
|---|---|---|
| Priming nudge | Stickers displayed at point of care, displaying problematic scans to primary care providers when high risk patients presents, treatment reminders/flags on online records, visual prime (picture/smells) to prompt targeted behaviour, reminder posters in display area, story board with priority problems and endorsement visually displayed, availability of resources to prime targeted behaviour (pocket hand rub, ball pen, posters), pocket-sized cards/laminated messages, mailing of questions regarding targeted behaviour to prime thinking of action, point-of-care prompts to assess/screen, reminders to measure/assess in electronic medical records at point of care | 29, 69% |
| Norms and messenger nudge | Sending guidelines by email/mail endorsed by a reputable organisation (e.g. chair, president or governing organisation), presenting information on performance relative to other providers and units in the area, letters signed by opinion leaders, resources endorsed by directors of the unit | 17, 40% |
| Salience/affect/affect nudge | Presenting vignettes with relevant patients’ cases to physicians, personal consequences signs | 8, 19% |
| Default nudge | Restricting options where not relevant to a particular patient/case (shading of boxes) | 1, 2.4% |
| Commitment/ego nudge | Providers to publicly declare their commitment/ ego to reducing inappropriate implementation behaviour/conducting implementation behaviour, displaying participation in improvement initiatives publicly | 1, 2.4% |
| Incentives nudge | Provision of continuing medical education points, certificates | 3, 7.0% |
Priming nudge
The most commonly used nudge strategy was priming, which was included in 69% (n = 29) of trials, across 41 outcomes. The impact of this nudge strategy was assessed on a range of implementation outcomes including adherence to antibiotic prescribing guidelines [104, 105, 123], prescribing rates of medication and test ordering of various conditions [27, 92, 93, 98, 99, 103, 106, 118, 119, 121, 129, 131], vaccinations [94], provision of care according to guidelines [90, 91, 96, 100, 103, 124, 126], and adherence to hand hygiene guidelines [95, 97, 101, 102, 107, 122, 130]. Priming nudges were also applied in various clinical settings including hospitals [95, 102, 104, 118, 122, 130], primary care practices [27, 91, 93, 94, 96, 99–101, 103, 119, 121, 124], mental health units [106] and community-based long-term care facilities [107].
Norms and messenger nudge
Norms and messenger nudge were the second most commonly used nudge strategy, included in 40% of studies (n = 17) across 19 outcomes. The impact of this strategy on a number of implementation outcomes including appropriate prescribing of medication [108, 111, 113, 125, 127, 132], and test ordering for various conditions [110, 119], antibiotic prescribing [123, 127], provision of preventive/lifestyle care according to guidelines [109, 124] and adherence to hand hygiene guidelines [112, 122, 130] was assessed. Interventions were undertaken in various clinical settings including hospitals [111, 112, 122, 127, 130], primary care practices [108–110, 113, 119, 121, 133] and pharmacies [131].
Salience and affect nudge
This was the third most frequently used nudge strategy, utilised in 19% of the studies reviewed (n = 8), across 11 outcomes. This was undertaken in interventions in hospitals [115, 127], primary care practices [114, 126] and community mental health teams [116] to improve test ordering (i.e. screening for bone mineral density) [114, 117], hand hygiene [115, 130], provision of care according to various guidelines (i.e. mental health) [116, 126, 128] and antibiotic prescribing [127].
Incentive, commitment/ego and default nudge
Less than 10% of studies incorporated either incentive (n = 3), commitment/ego (n = 1) or default (n = 1) nudge strategies as part of their interventions. The one study that used a commitment/ego nudge was conducted in hospitals to increase adherence to hand hygiene where clinicians publicly declared their commitment to either reduce inappropriate behaviour or increase recommended behaviour [130]. A default nudge was used in one study conducted in hospitals where test ordering options related to thyroid function not relevant to patients were shaded out based on ordering forms and incentive strategies [118]. The incentive strategy primarily included provision of certificates and professional development points [125, 128, 129] to primary care physicians to increase appropriate prescribing or test ordering.
Impact of nudge strategies on healthcare provider behaviour
Of the 57 outcomes assessed, 49 (86%) had an estimated effect on clinician behaviour in the hypothesised direction, of which 30 (53%) did not contain the null value (see Table 5). Figure 2 shows the distribution of studies by whether they had an estimated effect on the outcome. The median standardised mean difference across all continuous outcomes was 0.39 (IQ1 = 0.22, IQ3 = 0.45). For dichotomous outcomes, the median OR across all outcomes was 1.62 (IQ1 = 1.13, IQ3 = 2.76).
Table 5 .
A summary of the number and percentage of outcomes reporting an estimated effect in support of the intervention by number of nudge strategies and intervention type
| Strategy | Number of studies | Number of outcomes | Number (%) of estimated effects in direction of hypothesised effecta | Number (%) estimated effects in direction of hypothesised effect and significanta |
|---|---|---|---|---|
| All studies | 42 | 57 | 49 (86) | 30 (53) |
| Nudge as part of multicomponent intervention | 22 | 30 | 24 (80) | 15 (50) |
| Nudge only intervention | 20 | 27 | 25 (93) | 15 (56) |
| One nudge strategy included | 29 | 42 | 36 (86) | 23 (55) |
| More than one nudge strategy included | 13 | 15 | 13 (87) | 7 (47) |
| Type of nudge strategies: | ||||
| Priming | 29 | 41 | 37 (90) | 24 (59) |
| Norms and messenger | 17 | 19 | 16 (84) | 9 (47) |
| Salience/ affect/affect | 8 | 11 | 8 (73) | 3 (27) |
| Incentive | 3 | 3 | 1 (33) | 0 (0) |
| Default | 1 | 1 | 1 (100) | 1 (100) |
| Commitment/ego | 1 | 1 | 1 (100) | 0 (0) |
Note: aThere are several studies that report on more than one outcome and thus are represented more than once in this result
Fig. 2.
Harvest plot of estimated effect estimates for all include studies
When the nudge strategy was included as part of a multicomponent intervention, 24 out of 30 (80%) had a calculated effect estimate that was in the hypothesised direction, of which 15 (50%) did not contain the null. Comparatively, 20 studies across 27 outcomes tested a nudge only intervention where 25 of these outcomes (93%) were in the hypothesised direction and 15 (56%) did not contain the null (see Table 5, Fig. 1).
Twenty-nine studies across 42 outcomes included only one type of nudge strategy as part of their intervention packages. Of this, 36 (86%) were in the hypothesised direction, of which 23 (55%) did not contain the null. Comparatively, 13 studies across 15 outcomes employed intervention packages that included more than one nudge strategy. Of these, 13 of the 15 outcomes (87%) were in the hypothesised direction and 7 (47%) did not contain the null. See Table 5 and Fig. 3 for the number and percentage of outcomes by nudge strategies. Interventions that included a priming nudge showed the most promise, with 37 out of the 41 outcomes (90%) in the hypothesised direction.
Fig. 3.
Harvest plot of estimated effect estimates by nudge classification and whether studies were multi or single component studies
The most consistent evidence of effect was observed for behaviours such as handwashing (all outcomes in the hypothesised direction, with 70% containing the null) [95, 97, 101, 102, 107, 112, 115, 122, 130], and test ordering and prescribing for management of osteoporosis (all outcomes in the hypothesised direction, with 80% containing the null) [108, 110, 111, 117, 126]. The effects were least consistent for obesity management (two out of four in the hypothesised direction, with 25% containing the null) [90, 96, 124].
Discussion
This review of 42 RCTs included within eligible Cochrane systematic reviews found that a variety of nudge strategies have been employed in trials of clinical practice change interventions. For the majority of outcomes assessed (49/57), the effects were in the hypothesised beneficial direction. Additionally, a median effect size of 0.39 (IQ1 = 0.22, IQ3 = 0.45) for continuous and OR 1.62 ( 95% CI = 1.13, = 2.76) for dichotomous outcomes were calculated. These effects are comparable to other systematic reviews that have assessed the impact of a diverse range of implementation strategies on provider adherence to clinical guidelines [134, 135] and support the continued application of nudge interventions in practice. These comparable effects highlight the need to better consider and account for the potential impact of nudge strategies in the evaluation of complex implementation interventions, as these strategies are often overlooked.
Additionally, using vote-counting approaches, our study found that the effects of interventions were not related to the number of nudge strategies employed, or whether nudge strategies were part of a broader package of implementation support. While the evidence around this is mixed, these findings are consistent with a previous review of 25 reviews of implementation strategies, which found no compelling evidence that multicomponent strategies are more effective than single interventions [136]. As our findings relied on indirect comparisons, future studies testing the effects of nudge strategies using staggered or factorial trial designs are needed to better quantify the impact of individual nudge strategies by itself and in combination with other strategies.
Priming nudges were the most frequently evaluated, while few studies assessed the impact of incentive, commitment and ego nudges. While there were variable effects depending on outcomes, the most consistent effect was observed for handwashing, and test ordering and prescribing for osteoporosis management. It is possible that behaviours such as handwashing are more likely to be habitual, where practices are standardised and relatively simple to implement [137], and thus may respond to lower intensity stimuli. The outcomes measured for these behaviours are also likely to be more proximal and directly related to the intervention. These findings highlight the need to carefully consider the target behaviour and outcomes assessed when developing nudge interventions. Further, grounding the design and evaluation of nudge interventions in theory is needed to increase understanding of the context, type of behaviours and for whom nudge interventions may be effective for. Critically, future examination of the impact of nudge interventions according to different sociocultural characteristics (i.e. ethnicity, age, socioeconomic status) is needed to ensure such interventions do not exacerbate health inequalities.
Collectively, findings from this review highlight the positive effects of nudge interventions more broadly and support its application in clinical practice for some behaviours. It also identifies potential areas for development and testing of nudge strategies that have not been well studied such as for commitment nudges, where promising intervention effects have been reported [14]. For healthcare administrators, there is significant opportunity to embed priming, salience and default nudges within existing electronic systems or existing quality improvement tools such as reminders and audit and feedback. For example, scans or test results can be programmed to automatically pop up on electronic health records when a high-risk patient presents for care to facilitate follow-up. Additionally, electronic systems can be programmed to default to a more efficient treatment option where available (i.e. prescribing generic over brand name medications).
Where there is available infrastructure, embedding ‘nudge units’ similar to that described by Patel et al., within health services provides an innovative way of improving quality of care, while systematically developing and testing the impact of nudge interventions [138]. These units involve collaboration between health system administrators and leaders, front-line staff and researchers. Such collaborations enable the co-development of nudge solutions to address identified areas of suboptimal care. The unit is also then involved in subsequent implementation, evaluation and translation of the strategy, if shown to be effective [138]. The embedding of such units within clinical health services is similar to that previously described in public health practice [139] and provides significant opportunity to understand how nudge strategies can be best applied to improve clinical practice. Further, participation in innovative platforms such as the audit and feedback meta-lab [140], which involves collaboration between healthcare organisations and researchers, can provide an opportunity to further undertake head-to-head comparisons of nudge strategies applied within audit and feedback interventions.
Despite its promise, the application of nudge strategies into practice will firstly need to carefully consider issues around ethics and personal choice. As nudge interventions are inherently designed to influence the automatic systems, these attempts to change behaviour may be seen as challenging the role responsibility of clinicians and perceived as manipulative. Engaging clinicians early on with designing and implementing interventions is crucial to ensure the issues surrounding consent and freedom of choice are given due attention [12, 141].
Findings from this review should be considered in the context of a number of limitations. As there is no general consensus on the kinds of behavioural interventions that are classified as nudge, we used a pragmatic and systematic approach where we searched trials included in recently published Cochrane Reviews. This approach was selected as we sought to identify and characterise nudge implementation strategies across a range of medical disciplines and health conditions. As such, our approach undoubtedly failed to capture all eligible published trials, and findings described here are reflective of RCTs included in the published reviews. Future attempts to develop a more targeted electronic search strategy are likely to result in a more comprehensive review. The heterogeneity of outcomes, strategies and targeted guidelines precluded the use of meta-analysis to describe effects. We used non-meta-analytic methods of synthesis including providing a summary of overall effect estimate and vote-counting approaches. Vote-counting approaches are limited in its ability to examine intervention effectiveness as it is unable to account for differential weights in each study based on sample size. Further, seven studies had more than one primary outcome and were included more than once in the vote-counting procedure, which may have given more weight to such studies. While we attempted to extract risk of bias assessments for implementation outcomes, not all reviews specified this. As such, it is possible that the risk of bias assessment for these studies may be related to other non-implementation outcomes.
While such limitations exist, this review included 42 RCTs which provides a broad understanding of how nudge strategies have been applied to improve clinical practice and opportunities to further develop this important area of research. We used the Mindspace framework to provide a broad overview of the types of nudge intervention. To provide more insight into how nudge interventions can change behaviour, future reviews should consider using taxonomies such as the behaviour change techniques to classify these interventions by its psychological targets [142], or mapping to existing implementation taxonomies such as the Expert Recommendations for Implementing Change (ERIC) or EPOC taxonomy [143]. Our review also primarily focused on assessing the impact of the interventions on fidelity outcomes. Future reviews should consider assessing a broader range of implementation outcomes as specified by Proctor et al [144].
Conclusions
Main conclusion
This study is the first of its kind and offers new information for policy makers, practitioners and quality improvement agencies to support the application of nudge strategies to improve provision of clinical care. The review provides an overview on how such strategies have been applied and some evidence demonstrating the positive effects of nudge strategies. While more definitive research is needed, the results of this review suggest that nudges could be an effective tool to improve implementation of some clinical guidelines.
Future research
In addition to primary research exploring the effects of nudge interventions on clinical practice, there is considerable need to develop standard terminology that is applied consistently to describe these strategies as well as detailed guidance on describing such interventions. Currently, the diversity and inconsistency in the terminology represents a real barrier to synthesising, applying and advancing the field. Such work could be considered a priority in the field of implementation science given the considerable research investment and abundance of randomised trials assessing the effects of strategies target rational clinical decision-making.
Supplementary information
Additional file 1. Estimated effect estimates for continuous data used to generate results
Additional file 2. Estimate effect estimates for dichotomous data used to generate results
Acknowledgements
The authors wish to acknowledge Li Kheng Chai, Beatrice Murawski, Hannah Lamont, Jannah Jones and Melanie Lum for their support with data extraction and manuscript preparation.
Abbreviations
- RCT
Randomised controlled trial
- MINDSPACE
Messenger, Incentives, Norms, Defaults, Salience, Priming, Affect, Commitments, Ego
- OR
Odds ratios
- EPOC
Cochrane Effective Practice and Organisation of Care
- BCTT
Behaviour Change Technique Taxonomy
- ERIC
Expert Recommendations for Implementing Change
Authors’ contributions
SY conceived the idea for the review and drafted the first version of the manuscript. SY, FS, AH, AG, RW and LW contributed to the drafting of the manuscript. SY, FS, AH, RS, NN and AA undertook the data extraction and classification of nudge strategies. AH provided statistical advice and undertook the statistical analyses for the review. All authors reviewed and approved the final manuscript for submission. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.
Funding
This project was funded by internal research funds provided to SLY by the School of Medicine and Public Health, University of Newcastle. Hunter New England Population Health, the University of Newcastle, and Hunter Medical Research Institute provided infrastructure support. SLY is a postdoctoral research fellow funded by an Australian Research Council Discovery Early Career Researcher Award (DE170100382). LW is a Hunter New England Clinical Research Fellow and is supported by a National Health and Medical Research Council (NHMRC) Career Development Fellowship (1128348) and Heart Foundation Future Leader Fellowship (101175). RS is supported by a NHMRC TRIP Fellowship (APP1150661). RW is a postdoctoral research fellow funded by the National Heart Foundation (102156). The funders had no involvement in the conception and design of the study.
Availability of data and materials
The datasets supporting the conclusions of this article are included within the article (and its additional files).
Ethics approval and consent to participate
As this is a review of previously published articles, no ethics approval was required for this study.
Consent for publication
Not applicable
Competing interests
The authors declare that they have no competing interests.
Footnotes
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary information
Supplementary information accompanies this paper at 10.1186/s13012-020-01011-0.
References
- 1.Grimshaw J, Russell I. Effect of clinical guidelines on medical practice: a systematic review of rigorous evaluations. Lancet. 1993;342(8883):1317–1322. doi: 10.1016/0140-6736(93)92244-N. [DOI] [PubMed] [Google Scholar]
- 2.Mickan S, Burls A, Glasziou P. Patterns of 'leakage' in the utilisation of clinical guidelines: a systematic review. Postgrad Med J. 2011;87(1032):670–679. doi: 10.1136/pgmj.2010.116012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Cochrane Effective Practice and Organisation of Care: EPOC reviews. https://epoc.cochrane.org/epoc-reviews (2020). Accessed 20/04/2020 2020.
- 4.Lau R, Stevenson F, Ong BN, Dziedzic K, Treweek S, Eldridge S, et al. Achieving change in primary care--causes of the evidence to practice gap: systematic reviews of reviews. Implementation science : IS. 2016;11:40. doi: 10.1186/s13012-016-0396-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Potthoff S, Rasul O, Sniehotta F, Marques M, Beyer F, Thomson R, et al. The relationship between habit and healthcare professional behaviour in clinical practice: a systematic review and meta-analysis. Health Psychol Rev. 2019;13(1):73–90. doi: 10.1080/17437199.2018.1547119. [DOI] [PubMed] [Google Scholar]
- 6.Evans JS. Dual-processing accounts of reasoning, judgment, and social cognition. Annu Rev Psychol. 2008;59:255–278. doi: 10.1146/annurev.psych.59.103006.093629. [DOI] [PubMed] [Google Scholar]
- 7.Presseau J, Johnston M, Heponiemi T, Elovainio M, Francis J, Eccles M, et al. Reflective and automatic processes in health care professional behaviour: a dual process model tested across multiple behaviours. Ann Behav Med. 2014;48(3):347–358. doi: 10.1007/s12160-014-9609-8. [DOI] [PubMed] [Google Scholar]
- 8.Potthoff S, Presseau J, Sniehotta FF, Johnston M, Elovainio M, Avery L. Planning to be routine: habit as a mediator of the planning-behaviour relationship in healthcare professionals. Implementation science : IS. 2017;12(1):24. doi: 10.1186/s13012-017-0551-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Hansen P. The definition of nudge and libertarian paternalism: does the hand fit the glove? European Journal of Risk Regulation. 2016;1:155–174. doi: 10.1017/S1867299X00005468. [DOI] [Google Scholar]
- 10.Thaler RH, Sunstein CR, Balz JP. Choice architecture. The behavioral foundations of public policy. Princeton, NJ, US: Princeton University Press; 2013. p. 428-439.
- 11.Thaler R, Sunstein C. Nudge: improving decisions about health, wealth, and happiness. Yale University Press; 2008.
- 12.Hallsworth M. MINDSPACE: Influencing behaviour through public policy. 2010. [Google Scholar]
- 13.Sunstein CR. Nudging: a very short guide. J Consum Policy. 2014;37(4):583–588. doi: 10.1007/s10603-014-9273-1. [DOI] [Google Scholar]
- 14.Meeker D, Knight T, Friedberg M, Linder J, Goldstein N, Fox C, et al. Nudging guideline-concordant antibiotic prescribing: a randomized clinical trial. JAMA Intern Med. 2014;174(3):425–431. doi: 10.1001/jamainternmed.2013.14191. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Hallsworth M, Chadborn T, Sallis A, Sanders M, Berry D, Greaves F, et al. Provision of social norm feedback to high prescribers of antibiotics in general practice: a pragmatic national randomised controlled trial. Lancet. 2016;387(10029):1743–1752. doi: 10.1016/S0140-6736(16)00215-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Moher D, Liberati A, Tetzlaff J, Altman DG, Group P Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Bmj. 2009;339:b2535. doi: 10.1136/bmj.b2535. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Cochrane Effective Practice and Organisation of Care Group.: Cochrane Effective Practice and Organisation of Care. https://epoc.cochrane.org/about-us (2019). Accessed 12th July 2019.
- 18.Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217–226. doi: 10.1097/MLR.0b013e3182408812. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Centers for Disease Control and Prevention NCfIaRDN: interim guidance for infection control within healthcare settings when caring for confirmed cases, probable cases, and cases under investigation for infection with novel influenza A viruses associated with severe disease. https://www.cdc.gov/flu/avianflu/novel-flu-infection-control.htm (2014). Accessed 8th July 2019.
- 20.Vlaev I, King D, Dolan P, Darzi A. The theory and practice of “nudging”: changing health behaviors. Public Adm Rev. 2016;76(4):550–561. doi: 10.1111/puar.12564. [DOI] [Google Scholar]
- 21.Münscher R, Vetter M, Scheuerle T. A review and taxonomy of choice architecture techniques. J Behav Decis Mak. 2016;29(5):511–524. doi: 10.1002/bdm.1897. [DOI] [Google Scholar]
- 22.Mollenkamp M, Zeppernick M, Schreyogg J. The effectiveness of nudges in improving the self-management of patients with chronic diseases: a systematic literature review. Health Policy. 2019;123(12):1199–1209. doi: 10.1016/j.healthpol.2019.09.008. [DOI] [PubMed] [Google Scholar]
- 23.Wolfenden L, Goldman S, Stacey F, Grady A, Kingsland M, Williams C, et al. Strategies to improve the implementation of workplace-based policies or practices targeting tobacco, alcohol, diet, physical activity and obesity. Cochrane Database Syst Rev. 2018;11. 10.1002/14651858.CD012439.pub2. [DOI] [PMC free article] [PubMed]
- 24.Wolfenden L, Nathan N, Sutherland R, Yoong SL, Hodder R, Wyse R, et al. Strategies for enhancing the implementation of school-based policies or practices targeting risk factors for chronic disease. Cochrane Database Syst Rev. 2017;11. 10.1002/14651858.CD011677.pub2. [DOI] [PMC free article] [PubMed]
- 25.Higgins JPT TJ, Chandler J, Cumpston M, Li T, Page MJ, Welch VA,. Cochrane Handbook for Systematic Reviews of Interventions version 6.0. www.training.cochrane.org/handbook (2019). Accessed 20/04/2020 2020.
- 26.Arno A, Thomas S. The efficacy of nudge theory strategies in influencing adult dietary behaviour: a systematic review and meta-analysis. BMC Public Health. 2016;16(1):676. doi: 10.1186/s12889-016-3272-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Rossi R, Every N. A computerized intervention to decrease the use of calcium channel blockers in hypertension. J Gen Intern Med. 1997;12:672–678. doi: 10.1046/j.1525-1497.1997.07140.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Borenstein M, Hedges L, Higgins J, Rothstein H. Introduction to meta-analysis. Chichester: John Wiley & Sons. Ltd; 2009. [Google Scholar]
- 29.Ogilvie D, Fayter D, Petticrew M, Sowden A, Thomas S, Whitehead M, et al. The harvest plot: a method for synthesising evidence about the differential effects of interventions. BMC Med Res Methodol. 2008;8(1):8. doi: 10.1186/1471-2288-8-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.SAS Institute Inc. SAS software. 9.4 ed. Cary, NC, USA.: SAS Institute Inc; 2014.
- 31.StataCorp. Stata Statistical Software: Release 13. 13 ed. College Station, TX: StataCorp LP; 2013.
- 32.R Core Team. R: a language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2017.
- 33.Higgins J, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration; 2011.
- 34.Eamer G, Taheri A, Chen SS, Daviduck Q, Chambers T, Shi X, et al. Comprehensive geriatric assessment for older people admitted to a surgical service. The Cochrane database of systematic reviews. 2018;1:CD012485. doi: 10.1002/14651858.CD012485.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Vaona A, Banzi R, Kwag KH, Rigon G, Cereda D, Pecoraro V, et al. E-learning for health professionals. The Cochrane database of systematic reviews. 2018;1:CD011736. doi: 10.1002/14651858.CD011736.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Gonçalves-Bradley DC, Buckley BS, Fønhus MS, Glenton C, Henschke N, Lewin S, et al. Mobile-based technologies to support healthcare provider to healthcare provider communication and management of care. Cochrane Database Syst Rev. 2018. 10.1002/14651858.cd012927. [DOI] [PMC free article] [PubMed]
- 37.Jacobson Vann JC, Jacobson RM, Coyne-Beasley T, Asafu-Adjei JK, Szilagyi PG. Patient reminder and recall interventions to improve immunization rates. The Cochrane database of systematic reviews. 2018;1:CD003941. doi: 10.1002/14651858.CD003941.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Zaugg V, Korb-Savoldelli V, Durieux P, Sabatier B. Providing physicians with feedback on medication adherence for people with chronic diseases taking long-term medication. The Cochrane database of systematic reviews. 2018;1:CD012042. doi: 10.1002/14651858.CD012042.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Khalil H, Bell B, Chambers H, Sheikh A, Avery AJ. Professional, structural and organisational interventions in primary care for reducing medication errors. The Cochrane database of systematic reviews. 2017;10:CD003942. doi: 10.1002/14651858.CD003942.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Ciapponi A, Lewin S, Herrera CA, Opiyo N, Pantoja T, Paulsen E, et al. Delivery arrangements for health systems in low-income countries: an overview of systematic reviews. The Cochrane database of systematic reviews. 2017;9:CD011083. doi: 10.1002/14651858.CD011083.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Herrera CA, Lewin S, Paulsen E, Ciapponi A, Opiyo N, Pantoja T, et al. Governance arrangements for health systems in low-income countries: an overview of systematic reviews. The Cochrane database of systematic reviews. 2017;9:CD011085. doi: 10.1002/14651858.CD011085.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Pantoja T, Opiyo N, Lewin S, Paulsen E, Ciapponi A, Wiysonge CS, et al. Implementation strategies for health systems in low-income countries: an overview of systematic reviews. The Cochrane database of systematic reviews. 2017;9:CD011086. doi: 10.1002/14651858.CD011086.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Ellis G, Gardner M, Tsiachristas A, Langhorne P, Burke O, Harwood RH, et al. Comprehensive geriatric assessment for older adults admitted to hospital. The Cochrane database of systematic reviews. 2017;9:CD006211. doi: 10.1002/14651858.CD006211.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 44.Wiysonge CS, Paulsen E, Lewin S, Ciapponi A, Herrera CA, Opiyo N, et al. Financial arrangements for health systems in low-income countries: an overview of systematic reviews. The Cochrane database of systematic reviews. 2017;9:CD011084. doi: 10.1002/14651858.CD011084.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Goncalves-Bradley DC, Iliffe S, Doll HA, Broad J, Gladman J, Langhorne P, et al. Early discharge hospital at home. The Cochrane database of systematic reviews. 2017;6:CD000356. doi: 10.1002/14651858.CD000356.pub4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 46.Reeves S, Pelone F, Harrison R, Goldman J, Zwarenstein M. Interprofessional collaboration to improve professional practice and healthcare outcomes. The Cochrane database of systematic reviews. 2017;6:CD000072. doi: 10.1002/14651858.CD000072.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Archambault PM, van de Belt TH, Kuziemsky C, Plaisance A, Dupuis A, McGinn CA, et al. Collaborative writing applications in healthcare: effects on professional practice and healthcare outcomes. The Cochrane database of systematic reviews. 2017;5:CD011388. doi: 10.1002/14651858.CD011388.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Young C, Hall AM, Goncalves-Bradley DC, Quinn TJ, Hooft L, van Munster BC, et al. Home or foster home care versus institutional long-term care for functionally dependent older people. The Cochrane database of systematic reviews. 2017;4:CD009844. doi: 10.1002/14651858.CD009844.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Yuan B, He L, Meng Q, Jia L. Payment methods for outpatient care facilities. The Cochrane database of systematic reviews. 2017;3:CD011153. doi: 10.1002/14651858.CD011153.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Smith SM, Cousins G, Clyne B, Allwright S, O'Dowd T. Shared care across the interface between primary and specialty care in management of long term conditions. The Cochrane database of systematic reviews. 2017;2:CD004910. doi: 10.1002/14651858.CD004910.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 51.Chen CE, Chen CT, Hu J, Mehrotra A. Walk-in clinics versus physician offices and emergency rooms for urgent care and chronic disease management. The Cochrane database of systematic reviews. 2017;2:CD011774. doi: 10.1002/14651858.CD011774.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 52.Vaona A, Pappas Y, Grewal RS, Ajaz M, Majeed A, Car J. Training interventions for improving telephone consultation skills in clinicians. The Cochrane database of systematic reviews. 2017;1:CD010034. doi: 10.1002/14651858.CD010034.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 53.Flodgren G, Goncalves-Bradley DC, Pomey MP. External inspection of compliance with standards for improved healthcare outcomes. The Cochrane database of systematic reviews. 2016;12:CD008992. doi: 10.1002/14651858.CD008992.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 54.Weeks G, George J, Maclure K, Stewart D. Non-medical prescribing versus medical prescribing for acute and chronic disease management in primary and secondary care. The Cochrane database of systematic reviews. 2016;11:CD011227. doi: 10.1002/14651858.CD011227.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 55.Shepperd S, Iliffe S, Doll HA, Clarke MJ, Kalra L, Wilson AD, et al. Admission avoidance hospital at home. The Cochrane database of systematic reviews. 2016;9:CD007491. doi: 10.1002/14651858.CD007491.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 56.Wilson AD, Childs S, Goncalves-Bradley DC, Irving GJ. Interventions to increase or decrease the length of primary care physicians’ consultation. The Cochrane database of systematic reviews. 2016;8:CD003540. doi: 10.1002/14651858.CD003540.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 57.Gaitonde R, Oxman AD, Okebukola PO, Rada G. Interventions to reduce corruption in the health sector. The Cochrane database of systematic reviews. 2016;8:CD008856. doi: 10.1002/14651858.CD008856.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 58.Abdel-Aleem H, El-Gibaly OM, El-Gazzar AF, Al-Attar GS. Mobile clinics for women's and children's health. The Cochrane database of systematic reviews. 2016;8:CD009677. doi: 10.1002/14651858.CD009677.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 59.Wiysonge CS, Abdullahi LH, Ndze VN, Hussey GD. Public stewardship of private for-profit healthcare providers in low- and middle-income countries. The Cochrane database of systematic reviews. 2016;8:CD009855. doi: 10.1002/14651858.CD009855.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 60.Oyo-Ita A, Wiysonge CS, Oringanje C, Nwachukwu CE, Oduwole O, Meremikwu MM. Interventions for improving coverage of childhood immunisation in low- and middle-income countries. The Cochrane database of systematic reviews. 2016;7:CD008145. doi: 10.1002/14651858.CD008145.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 61.Gera T, Shah D, Garner P, Richardson M, Sachdev HS. Integrated management of childhood illness (IMCI) strategy for children under five. The Cochrane database of systematic reviews. 2016;6:CD010123. doi: 10.1002/14651858.CD010123.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 62.Campbell F, Biggs K, Aldiss SK, O'Neill PM, Clowes M, McDonagh J, et al. Transition of care for adolescents from paediatric services to adult health services. The Cochrane database of systematic reviews. 2016;4:CD009794. doi: 10.1002/14651858.CD009794.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 63.Balogh R, McMorris CA, Lunsky Y, Ouellette-Kuntz H, Bourne L, Colantonio A, et al. Organising healthcare services for persons with an intellectual disability. The Cochrane database of systematic reviews. 2016;4:CD007492. doi: 10.1002/14651858.CD007492.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 64.Smith SM, Wallace E, O'Dowd T, Fortin M. Interventions for improving outcomes in patients with multimorbidity in primary care and community settings. The Cochrane database of systematic reviews. 2016;3:CD006560. doi: 10.1002/14651858.CD006560.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 65.Opiyo N, Yamey G, Garner P. Subsidising artemisinin-based combination therapy in the private retail sector. The Cochrane database of systematic reviews. 2016;3:CD009926. doi: 10.1002/14651858.CD009926.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 66.Christensen M, Lundh A. Medication review in hospitalised patients to reduce morbidity and mortality. The Cochrane database of systematic reviews. 2016;2:CD008986. doi: 10.1002/14651858.CD008986.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 67.Alldred DP, Kennedy MC, Hughes C, Chen TF, Miller P. Interventions to optimise prescribing for older people in care homes. The Cochrane database of systematic reviews. 2016;2:CD009095. doi: 10.1002/14651858.CD009095.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 68.Munabi-Babigumira S, Glenton C, Lewin S, Fretheim A, Nabudere H. Factors that influence the provision of intrapartum and postnatal care by skilled birth attendants in low- and middle-income countries: a qualitative evidence synthesis. The Cochrane database of systematic reviews. 2017;11:CD011558. doi: 10.1002/14651858.CD011558.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 69.Tonkin-Crine SK, Tan PS, van Hecke O, Wang K, Roberts NW, McCullough A, et al. Clinician-targeted interventions to influence antibiotic prescribing behaviour for acute respiratory infections in primary care: an overview of systematic reviews. The Cochrane database of systematic reviews. 2017;9:CD012252. doi: 10.1002/14651858.CD012252.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 70.Davey P, Marwick CA, Scott CL, Charani E, McNeil K, Brown E, et al. Interventions to improve antibiotic prescribing practices for hospital inpatients. The Cochrane database of systematic reviews. 2017;2:CD003543. doi: 10.1002/14651858.CD003543.pub4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 71.Gould DJ, Moralejo D, Drey N, Chudleigh JH, Taljaard M. Interventions to improve hand hygiene compliance in patient care. The Cochrane database of systematic reviews. 2017;9:CD005186. doi: 10.1002/14651858.CD005186.pub4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 72.Flodgren G, Goncalves-Bradley DC, Summerbell CD. Interventions to change the behaviour of health professionals and the organisation of care to promote weight reduction in children and adults with overweight or obesity. The Cochrane database of systematic reviews. 2017;11:CD000984. doi: 10.1002/14651858.CD000984.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 73.Tzortziou Brown V, Underwood M, Mohamed N, Westwood O, Morrissey D. Professional interventions for general practitioners on the management of musculoskeletal conditions. The Cochrane database of systematic reviews. 2016;5:CD007495. doi: 10.1002/14651858.CD007495.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 74.Arditi C, Rege-Walther M, Durieux P, Burnand B. Computer-generated reminders delivered on paper to healthcare professionals: effects on professional practice and healthcare outcomes. The Cochrane database of systematic reviews. 2017;7:CD001175. doi: 10.1002/14651858.CD001175.pub4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 75.Flodgren G, Hall AM, Goulding L, Eccles MP, Grimshaw JM, Leng GC, et al. Tools developed and disseminated by guideline producers to promote the uptake of their guidelines. The Cochrane database of systematic reviews. 2016;8:CD010669. doi: 10.1002/14651858.CD010669.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 76.Bighelli I, Ostuzzi G, Girlanda F, Cipriani A, Becker T, Koesters M, et al. Implementation of treatment guidelines for specialist mental health care. The Cochrane database of systematic reviews. 2016;12:CD009780. doi: 10.1002/14651858.CD009780.pub3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 77.Bishop PB, Wing PC. Knowledge transfer in family physicians managing patients with acute low back pain: a prospective randomized control trial. The spine journal : official journal of the North American Spine Society. 2006;6(3):282–288. doi: 10.1016/j.spinee.2005.10.008. [DOI] [PubMed] [Google Scholar]
- 78.Kerry S, Oakeshott P, Dundas D, Williams J. Influence of postal distribution of The Royal College of Radiologists' guidelines, together with feedback on radiological referral rates, on X-ray referrals from general practice: a randomized controlled trial. Fam Pract. 2000;17(1):46–52. doi: 10.1093/fampra/17.1.46. [DOI] [PubMed] [Google Scholar]
- 79.Schectman J, Schroth W, Verme D, Voss J. Randomized controlled trial of education and feedback for implementation of guidelines for acute low back pain. J Gen Intern Med. 2003;18:773–780. doi: 10.1046/j.1525-1497.2003.10205.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 80.Stevenson K, Searle K, Curry G, Boyce J, Harbarth S, Stoddard G, et al. Infection control interventions in small rural hospitals with limited resources: results of a cluster-randomized feasibility trial. Antimicrob Resist Infect Control. 2014;3:10. doi: 10.1186/2047-2994-3-10. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 81.Tierney W, Hui S, McDonald C. Delayed feedback of physician performance versus immediate reminders to perform preventive care: effects on physician compliance. Med Care. 1986;24(8):659–666. doi: 10.1097/00005650-198608000-00001. [DOI] [PubMed] [Google Scholar]
- 82.Huis A, Schoonhoven L, Grol R, Donders R, Hulscher M, van Achterberg T. Impact of a team and leaders-directed strategy to improve nurses’ adherence to hand hygiene guidelines: a cluster randomised trial. Int J Nurs Stud. 2013;50(4):464–474. doi: 10.1016/j.ijnurstu.2012.08.004. [DOI] [PubMed] [Google Scholar]
- 83.Fuller C, Michie S, Savage J, McAteer J, Besser S, Charlett A, et al. The Feedback Intervention Trial (FIT)--improving hand-hygiene compliance in UK healthcare workers: a stepped wedge cluster randomised controlled trial. PLoS One. 2012;7(10):e41617. doi: 10.1371/journal.pone.0041617. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 84.Gulmezoglu AM, Langer A, Piaggio G, Lumbiganon P, Villar J, Grimshaw J. Cluster randomised trial of an active, multifaceted educational intervention based on the WHO Reproductive Health Library to improve obstetric practices. BJOG : an international journal of obstetrics and gynaecology. 2007;114(1):16–23. doi: 10.1111/j.1471-0528.2006.01091.x. [DOI] [PubMed] [Google Scholar]
- 85.Kritchevsky S, Braun B, Bush A, Bozikis M, Kusek L, Burke J, et al. The effect of a quality improvement collaborative to improve antimicrobial prophylaxis in surgical patients. Ann Intern Med. 2008;149:472–480. doi: 10.7326/0003-4819-149-7-200810070-00007. [DOI] [PubMed] [Google Scholar]
- 86.Paul M, Andreassen S, Tacconelli E, Nielsen AD, Almanasreh N, Frank U, et al. Improving empirical antibiotic treatment using TREAT, a computerized decision support system: cluster randomized trial. J Antimicrob Chemother. 2006;58(6):1238–1245. doi: 10.1093/jac/dkl372. [DOI] [PubMed] [Google Scholar]
- 87.Bekkering GE, Hendriks HJ, van Tulder MW, Knol DL, Hoeijenbos M, Oostendorp RA, et al. Effect on the process of care of an active strategy to implement clinical guidelines on physiotherapy for low back pain: a cluster randomised controlled trial. Quality & safety in health care. 2005;14(2):107–112. doi: 10.1136/qshc.2003.009357. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 88.Camins BC, King MD, Wells JB, Googe HL, Patel M, Kourbatova EV, et al. Impact of an antimicrobial utilization programme on antimicrobial use at a large teaching hospital: a randomized controlled trial. Infect Control Hosp Epidemiol. 2009;30(10):931–938. doi: 10.1086/605924. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 89.Senn L, Burnand B, Francioli P, Zanetti G. Improving appropriateness of antibiotic therapy: randomized trial of an intervention to foster reassessment of prescription after 3 days. J Antimicrob Chemother. 2004;53:1062–1067. doi: 10.1093/jac/dkh236. [DOI] [PubMed] [Google Scholar]
- 90.Baer HJ, Wee CC, DeVito K, Orav EJ, Frolkis JP, Williams DH, et al. Design of a cluster-randomized trial of electronic health record-based tools to address overweight and obesity in primary care. Clinical trials. 2015;12(4):374–383. doi: 10.1177/1740774515578132. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 91.Barnett G, Winickoff R, Morgan M, Zielstorff R. A computer-based monitoring system for follow-up of elevated blood pressure. Med Care. 1983;21(4):400–409. doi: 10.1097/00005650-198304000-00003. [DOI] [PubMed] [Google Scholar]
- 92.Burack R, Gimotty P, George J, Simon M, Dews P, Moncrease A. The effect of patient and physician reminders on use of screening mammography in a Health Maintenance Organization. Results of a randomized controlled trial. Cancer. 1996;78:1708–1721. doi: 10.1002/(SICI)1097-0142(19961015)78:8<1708::AID-CNCR11>3.0.CO;2-1. [DOI] [PubMed] [Google Scholar]
- 93.Chambers C, Balaban D, Carlson B, Ungemack J, Grasberger D. Microcomputer-generated reminders: improving the compliance of primary care physicians with mammography screening guidelines. J Fam Pract. 1989;29:3. [PubMed] [Google Scholar]
- 94.Chambers C, Balaban D, Carlson B, Grasberger D. The effect of microcomputer-generated reminders on influenza vaccination rates in a University-based family practice center. J Am Board Fam Pract. 1991;4:19–26. [PubMed] [Google Scholar]
- 95.Fisher DA, Seetoh T, Oh May-Lin H, Viswanathan S, Toh Y, Yin WC, et al. Automated measures of hand hygiene compliance among healthcare workers using ultrasound: validation and a randomized controlled trial. Infect Control Hosp Epidemiol. 2013;34(9):919–928. doi: 10.1086/671738. [DOI] [PubMed] [Google Scholar]
- 96.Goodfellow J, Agarwal S, Harrad F, Shepherd D, Morris T, Ring A, et al. Cluster randomised trial of a tailored intervention to improve the management of overweight and obesity in primary care in England. Implementation science : IS. 2016;11(1):77. doi: 10.1186/s13012-016-0441-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 97.King D, Vlaev I, Everett-Thomas R, Fitzpatrick M, Darzi A, Birnbach DJ. "Priming" hand hygiene compliance in clinical environments. Health psychology : official journal of the Division of Health Psychology, American Psychological Association. 2016;35(1):96–101. doi: 10.1037/hea0000239. [DOI] [PubMed] [Google Scholar]
- 98.Lafata JE, Kolk D, Peterson EL, McCarthy BD, Weiss TW, Chen YT, et al. Improving osteoporosis screening: results from a randomized cluster trial. J Gen Intern Med. 2007;22(3):346–351. doi: 10.1007/s11606-006-0060-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 99.Le Breton J, Ferrat E, Attali C, Bercier S, Le Corvoisier P, Brixi Z, et al. Effect of reminders mailed to general practitioners on colorectal cancer screening adherence: a cluster-randomized trial. European journal of cancer prevention : the official journal of the European Cancer Prevention Organisation. 2016;25(5):380–387. doi: 10.1097/CEJ.0000000000000200. [DOI] [PubMed] [Google Scholar]
- 100.Lobach D, Hammond W. Computerized decision support based on a clinical practice guideline improves compliance with care standards. Am J Med. 1997;102:89–98. doi: 10.1016/S0002-9343(96)00382-8. [DOI] [PubMed] [Google Scholar]
- 101.Martin-Madrazo C, Soto-Diaz S, Canada-Dorado A, Salinero-Fort MA, Medina-Fernandez M. Carrillo de Santa Pau E, et al. Cluster randomized trial to evaluate the effect of a multimodal hand hygiene improvement strategy in primary care. Infect Control Hosp Epidemiol. 2012;33(7):681–688. doi: 10.1086/666343. [DOI] [PubMed] [Google Scholar]
- 102.Munoz-Price L, Patel Z, Banks S, Arheart K, Eber S, Lubarsky D, et al. Randomized crossover study evaluating the effect of a hand sanitizer dispenser on the frequency of hand hygiene among anesthesiology staff in the operating room. Infect Control Hosp Epidemiol. 2014;35(6):717–720. doi: 10.1086/676425. [DOI] [PubMed] [Google Scholar]
- 103.Rogers J, Haring OM, Wortman P, Watson R, Goetz J. Medical information systems: assessing impact in the areas of hypertension, obesity and renal disease. Med Care. 1982;1(1):63–74. doi: 10.1097/00005650-198201000-00005. [DOI] [PubMed] [Google Scholar]
- 104.Schnoor M, Meyer T, Suttorp N, Raspe H, Welte T, Schafer T, et al. Development and evaluation of an implementation strategy for the German guideline on community-acquired pneumonia. Quality & safety in health care. 2010;19(6):498–502. doi: 10.1136/qshc.2008.029629. [DOI] [PubMed] [Google Scholar]
- 105.Shojania K, Yokoe D, Platt R, Fiskio J, Ma'luf N, Bates D. Reducing vancomycin use utilizing a computer guideline: results of a randomized controlled trial. J Am Med Inform Assoc. 1998;5:554–562. doi: 10.1136/jamia.1998.0050554. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 106.Thompson A, Sullivan SA, Barley M, Strange SO, Moore L, Rogers P, et al. The DEBIT trial: an intervention to reduce antipsychotic polypharmacy prescribing in adult psychiatry wards - a cluster randomized controlled trial. Psychol Med. 2008;38(5):705–715. doi: 10.1017/S003329170700147X. [DOI] [PubMed] [Google Scholar]
- 107.Yeung W, Tam W, Wong T. Clustered randomized controlled trial of a hand hygiene intervention involving pocket-sized containers of alcohol-based hand rub for the control of infections in long-term care facilities. Infect Control Hosp Epidemiol. 2011;32(1):67–76. doi: 10.1086/657636. [DOI] [PubMed] [Google Scholar]
- 108.Cranney A, Lam M, Ruhland L, Brison R, Godwin M, Harrison MM, et al. A multifaceted intervention to improve treatment of osteoporosis in postmenopausal women with wrist fractures: a cluster randomized trial. Osteoporosis international : a journal established as result of cooperation between the European Foundation for Osteoporosis and the National Osteoporosis Foundation of the USA. 2008;19(12):1733–1740. doi: 10.1007/s00198-008-0669-0. [DOI] [PubMed] [Google Scholar]
- 109.Engers A, Wensing M, van Tulder M, Timmermans A, Oostendorp R, Koes B, et al. Implementation of the Dutch Low Back Pain Guideline for General Practitioners. A Cluster Randomized Controlled Trial. Spine. 2005;30:595–600. doi: 10.1097/01.brs.0000155406.79479.3a. [DOI] [PubMed] [Google Scholar]
- 110.Feldstein A, Elmer PJ, Smith DH, Herson M, Orwoll E, Chen C, et al. Electronic medical record reminder improves osteoporosis management after a fracture: a randomized, controlled trial. J Am Geriatr Soc. 2006;54(3):450–457. doi: 10.1111/j.1532-5415.2005.00618.x. [DOI] [PubMed] [Google Scholar]
- 111.Majumdar SR, Johnson JA, McAlister FA, Bellerose D, Russell AS, Hanley DA, et al. Multifaceted intervention to improve diagnosis and treatment of osteoporosis in patients with recent wrist fracture: a randomized controlled trial. CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne. 2008;178(5):569–575. doi: 10.1503/cmaj.070981. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 112.Mertz D, Dafoe N, Walter S, Brazil K, Loeb M. Effect of a multifaceted intervention on adherence to hand hygiene among healthcare workers: a cluster-randomized trial. Infect Control Hosp Epidemiol. 2010;31(11):1170–1176. doi: 10.1086/656592. [DOI] [PubMed] [Google Scholar]
- 113.Shah B, Bhattacharyya O, Yu C, Mamdani M, Parsons J, Straus S, et al. Effect of an educational toolkit on quality of care: a pragmatic cluster randomized trial. PLoS Med. 2014;11:2. doi: 10.1371/journal.pmed.1001588. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 114.Dey P, Simpson C, Collins S, Hodgson G, Dowrick C, Simison A, et al. Implementation of RCGP guidelines for acute low back pain: a cluster randomised controlled trial. Br J Gen Pract. 2004;54:33–37. [PMC free article] [PubMed] [Google Scholar]
- 115.Grant AM, Hofmann DA. It's not all about me: motivating hand hygiene among health care professionals by focusing on patients. Psychol Sci. 2011;22(12):1494–1499. doi: 10.1177/0956797611419172. [DOI] [PubMed] [Google Scholar]
- 116.Ince P, Tai S, Haddock G. Using plain English and behaviourally specific language to increase the implementation of clinical guidelines for psychological treatments in schizophrenia. J Ment Health. 2015;24(3):129–133. doi: 10.3109/09638237.2014.958213. [DOI] [PubMed] [Google Scholar]
- 117.Leslie W, LaBine L, Klassen P, Dreilich D, Caetano P. Closing the gap in postfracture care at the population level: a randomized controlled trial. CMAJ : Canadian Medical Association journal = journal de l’Association medicale canadienne. 2012;184(3):290–296. doi: 10.1503/cmaj.111158. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 118.Daucourt V, Saillour-Glenisson F, Michel P, Jutand M-A, Abouelfath A. A multicenter cluster randomized controlled trial of strategies to improve thyroid function testing. Med Care. 2003;41(3):432–441. doi: 10.1097/01.MLR.0000053216.33277.A4. [DOI] [PubMed] [Google Scholar]
- 119.Eccles M, Steen N, Grimshaw J, Thomas L, McNamee P, Soutter J, et al. Effect of audit and feedback, and reminder messages on primary-care radiology referrals: a randomised trial. Lancet. 2001;357(9266):1406–1409. doi: 10.1016/s0140-6736(00)04564-5. [DOI] [PubMed] [Google Scholar]
- 120.Majumdar SR, Tsuyuki RT, McAlister FA. Impact of opinion leader-endorsed evidence summaries on the quality of prescribing for patients with cardiovascular disease: a randomized controlled trial. Am Heart J. 2007;153(1):22–e1-8. doi: 10.1016/j.ahj.2006.07.030. [DOI] [PubMed] [Google Scholar]
- 121.McAlister FA, Fradette M, Graham M, Majumdar SR, Ghali WA, Williams R, et al. A randomized trial to assess the impact of opinion leader endorsed evidence summaries on the use of secondary prevention strategies in patients with coronary artery disease: the ESP-CAD trial protocol [ NCT00175240] Implementation science : IS. 2006;1:11. doi: 10.1186/1748-5908-1-11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 122.Rodriguez V, Giuffre C, Villa S, Almada G, Prasopa-Plaizier N, Gogna M, et al. A multimodal intervention to improve hand hygiene in ICUs in Buenos Aires, Argentina: a stepped wedge trial. International journal for quality in health care : journal of the International Society for Quality in Health Care. 2015;27(5):405–411. doi: 10.1093/intqhc/mzv065. [DOI] [PubMed] [Google Scholar]
- 123.Schouten JA, Hulscher ME, Trap-Liefers J, Akkermans RP, Kullberg BJ, Grol RP, et al. Tailored interventions to improve antibiotic use for lower respiratory tract infections in hospitals: a cluster-randomized, controlled trial. Clinical infectious diseases: an official publication of the Infectious Diseases Society of America. 2007;44(7):931–941. doi: 10.1086/512193. [DOI] [PubMed] [Google Scholar]
- 124.Taveras EM, Marshall R, Kleinman KP, Gillman MW, Hacker K, Horan CM, et al. Comparative effectiveness of childhood obesity interventions in pediatric primary care: a cluster-randomized clinical trial. JAMA Pediatr. 2015;169(6):535–542. doi: 10.1001/jamapediatrics.2015.0182. [DOI] [PubMed] [Google Scholar]
- 125.Rahme E, Choquette D, Beaulieu M, Bessette L, Joseph L, Toubouti Y, et al. Impact of a general practitioner educational intervention on osteoarthritis treatment in an elderly population. Am J Med. 2005;118(11):1262–1270. doi: 10.1016/j.amjmed.2005.03.026. [DOI] [PubMed] [Google Scholar]
- 126.Roux S, Beaulieu M, Beaulieu MC, Cabana F, Boire G. Priming primary care physicians to treat osteoporosis after a fragility fracture: an integrated multidisciplinary approach. J Rheumatol. 2013;40(5):703–711. doi: 10.3899/jrheum.120908. [DOI] [PubMed] [Google Scholar]
- 127.Solomon D, Van Houten L, Glynn R, Baden L, Curtis K, Schrager H, et al. Academic detailing to improve use of broad-spectrum antibiotics at an academic medical center. Arch Intern Med. 2001;161:1897–1902. doi: 10.1001/archinte.161.15.1897. [DOI] [PubMed] [Google Scholar]
- 128.Robling M, Houston H, Kinnersley P, Hourihan M, Cohen D, Hale J, et al. General Practitioners' use of Magnetic Resonance Imaging: an open randomized trial comparing telephone and written requests and an open randomized controlled trial of different methods of local guideline dissemination. Clin Radiol. 2002;57:402–407. doi: 10.1053/crad.2001.0864. [DOI] [PubMed] [Google Scholar]
- 129.Solomon DH, Katz JN, Finkelstein JS, Polinski JM, Stedman M, Brookhart MA, et al. Osteoporosis improvement: a large-scale randomized controlled trial of patient and primary care physician education. Journal of bone and mineral research : the official journal of the American Society for Bone and Mineral Research. 2007;22(11):1808–1815. doi: 10.1359/jbmr.070717. [DOI] [PubMed] [Google Scholar]
- 130.Stewardson AJ, Sax H, Gayet-Ageron A, Touveneau S, Longtin Y, Zingg W, et al. Enhanced performance feedback and patient participation to improve hand hygiene compliance of health care workers in the setting of established multimodal promotion: a single-centre, cluster randomised controlled trial. Lancet Infect Dis. 2016;16(12):1345–1355. doi: 10.1016/s1473-3099(16)30256-0. [DOI] [PubMed] [Google Scholar]
- 131.Majumdar SR, McAlister FA, Tsuyuki RT. A cluster randomized trial to assess the impact of opinion leader endorsed evidence summaries on improving quality of prescribing for patients with chronic cardiovascular disease: rationale and design [ISRCTN26365328] BMC Cardiovasc Disord. 2005;5(1):17. doi: 10.1186/1471-2261-5-17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 132.McAlister FA, Fradette M, Majumdar SR, Williams R, Graham M, McMeekin J, et al. The Enhancing Secondary Prevention in Coronary Artery Disease trial. CMAJ: Canadian Medical Association journal = journal de l'Association medicale canadienne. 2009;181(12):897–904. doi: 10.1503/cmaj.090917. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 133.Taveras EM, Gortmaker SL, Hohman KH, Horan CM, Kleinman KP, Mitchell K, et al. Randomized controlled trial to improve primary care to prevent and manage childhood obesity: the High Five for Kids study. Arch Pediatr Adolesc Med. 2011;165(8):714–722. doi: 10.1001/archpediatrics.2011.44. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 134.Tuti T, Nzinga J, Njoroge M, Brown B, Peek N, English M, et al. A systematic review of electronic audit and feedback: intervention effectiveness and use of behaviour change theory. Implementation science : IS. 2017;12(1):61. doi: 10.1186/s13012-017-0590-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 135.Pedersen ER, Rubenstein L, Kandrack R, Danz M, Belsher B, Motala A, et al. Elusive search for effective provider interventions: a systematic review of provider interventions to increase adherence to evidence-based treatment for depression. Implementation science : IS. 2018;13(1):99. doi: 10.1186/s13012-018-0788-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 136.Squires J, Sullivan K, Eccles M, Worswick J, Grimshaw J. Are multifaceted interventions more effective than single-component interventions in changing health-care professionals’ behaviours? An overview of systematic reviews. Implement Sci. 2014;9:152. doi: 10.1186/s13012-014-0152-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 137.Jumaa PA. Hand hygiene: simple and complex. Int J Infect Dis. 2005;9(1):3–14. doi: 10.1016/j.ijid.2004.05.005. [DOI] [PubMed] [Google Scholar]
- 138.Patel MS, Volpp KG, Asch DA. Nudge units to improve the delivery of health care. N Engl J Med. 2018;378(3):214–216. doi: 10.1056/NEJMp1712984. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 139.Wolfenden L, Yoong SL, Williams CM, Grimshaw J, Durrheim DN, Gillham K, et al. Embedding researchers in health service organizations improves research translation and health service performance: the Australian Hunter New England Population Health example. J Clin Epidemiol. 2017;85:3–11. doi: 10.1016/j.jclinepi.2017.03.007. [DOI] [PubMed] [Google Scholar]
- 140.Grimshaw JM, Ivers N, Linklater S, Foy R, Francis JJ, Gude WT, et al. Reinvigorating stagnant science: implementation laboratories and a meta-laboratory to efficiently advance the science of audit and feedback. BMJ Quality &amp; Safety. 2019;28(5):416. doi: 10.1136/bmjqs-2018-008355. [DOI] [PMC free article] [PubMed]
- 141.Gorin M, Joffe S, Dickert N, Halpern S. Justifying clinical nudges. Hastings Cent Rep. 2017;47(2):32–38. doi: 10.1002/hast.688. [DOI] [PubMed] [Google Scholar]
- 142.Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The Behavior Change Technique Taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med. 2013;46(1):81–95. doi: 10.1007/s12160-013-9486-6. [DOI] [PubMed] [Google Scholar]
- 143.Powell BJ, Waltz TJ, Chinman MJ, Damschroder LJ, Smith JL, Matthieu MM, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implement Sci. 2015;10(1):21. doi: 10.1186/s13012-015-0209-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 144.Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Admin Pol Ment Health. 2011;38(2):65–76. doi: 10.1007/s10488-010-0319-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Additional file 1. Estimated effect estimates for continuous data used to generate results
Additional file 2. Estimate effect estimates for dichotomous data used to generate results
Data Availability Statement
The datasets supporting the conclusions of this article are included within the article (and its additional files).