De-implementing wisely: developing the evidence base to reduce low-value care

Abstract

Choosing Wisely (CW) campaigns globally have focused attention on the need to reduce low-value care, which can represent up to 30% of the costs of healthcare. Despite early enthusiasm for the CW initiative, few large-scale changes in rates of low-value care have been reported since the launch of these campaigns. Recent commentaries suggest that the focus of the campaign should be on implementation of evidence-based strategies to effectively reduce low-value care. This paper describes the Choosing Wisely De-Implementation Framework (CWDIF), a novel framework that builds on previous work in the field of implementation science and proposes a comprehensive approach to systematically reduce low-value care in both hospital and community settings and advance the science of de-implementation.

The CWDIF consists of five phases: Phase 0, identification of potential areas of low-value healthcare; Phase 1, identification of local priorities for implementation of CW recommendations; Phase 2, identification of barriers to implementing CW recommendations and potential interventions to overcome these; Phase 3, rigorous evaluations of CW implementation programmes; Phase 4, spread of effective CW implementation programmes. We provide a worked example of applying the CWDIF to develop and evaluate an implementation programme to reduce unnecessary preoperative testing in healthy patients undergoing low-risk surgeries and to further develop the evidence base to reduce low-value care.

Introduction

There is increasing recognition of the problem of overuse relating to ‘low-value care’ defined as a test or treatment for which there is no evidence of patient benefit or where there is evidence of more harm than benefit.^{1 2} The Canadian Institute f for Health Information report that as much as 30% of healthcare is considered low value, which can lead to poor patient outcomes due to adverse events of treatments or unwarranted secondary tests (with potential for overtreatment of incidental findings), and inefficient use of scarce healthcare resources threatening the sustainability of healthcare systems.³ Reports from the Institute of Medicine⁴ and international studies have repeatedly demonstrated similar levels of low-value care.^{1 5–7}

Recognition of the overuse of low-value care led to the establishment of Choosing Wisely (CW) by the American Board of Internal Medicine Foundation in 2012 and subsequently spread to over 20 countries.⁸ CW is an initiative that seeks to encourage a dialogue between clinicians and patients about avoiding unnecessary medical tests, treatments and procedures in healthcare in order to ensure high-quality care and avoid harm.⁹ Initial efforts have focused on developing CW recommendations and measuring baseline rates of overuse¹⁰ and some local efforts to reduce low-value care. In the USA, over 80 partners have developed 550 recommendations pertaining to unnecessary tests, treatments and procedures¹¹ and in Canada, over 70 medical specialty societies have developed more than 350 recommendations.¹² Similar efforts have been planned or launched in Germany, Austria, Australia, New Zealand, England, Wales, Italy, Netherlands, Norway, Portugal, France, Israel, Brazil and Japan with other countries planning or in early stages of launching national campaigns. Internationally, CW leaders from existing or planned national CW campaigns meet yearly to discuss implementation of the recommendations in their respective countries and collectively have formed a collaborative learning community.¹³

Despite such clear uptake of and enthusiasm in the CW initiative and some early successes in lowering low-value care through local interventions,^{10 14} studies have shown limited large-scale change in ordering rates of low-value care since the launch of these campaigns.^{15 16} The issue of implementing CW recommendations and evaluating the effects of these recommendations has received much less attention. Recommendations alone will not change practice.^{17 18} Recent commentaries have suggested that the focus of the campaign should be on identifying and applying evidence-based strategies to effectively reduce low-value care.^{13 19} There is substantial evidence and guidance on how to implement evidence-based strategies. However, few conceptual frameworks exist to guide de-implementation, and those that do exist focus on team culture or organisational change²⁰ or target change in a specific clinical setting,²¹ making it difficult to generalise the frameworks across a myriad of healthcare settings and contexts.

What is de-implementation?

Broad definitions of implementation and de-implementation exist in healthcare. The National Institute of Health defined implementation as “the use of strategies to introduce or change evidence-based health interventions within specific settings”²² whereas de-implementation in the healthcare context has been defined as the “abandonment of medical practices or interventions that have been found to be ineffective and harmful”.²³ Developing theory and evidence to support de-implementation interventions is of significant importance for healthcare systems.

While evidence exists about how to implement evidence-based practices in general and some interventions are reported as generally effective (eg, audit and feedback,²⁴ academic detailing²⁵), there has been less attention focused on the problem of implementing recommendations to reduce low-value healthcare. Further research is needed to explore the generalisability of research findings about general implementation activities that aim that to reduce low-value healthcare.

Using behavioural approaches to inform de-implementation efforts

De-implementation interventions can be delivered at any level within the healthcare system: from the individual health professional, healthcare groups or teams, organisations providing healthcare, up to and including the larger healthcare system.²⁶ Successful interventions (whether implementation or de-implementation) require key actors (patients, healthcare providers, managers and policy-makers) to change their behaviours and/or decisions while working in the complex (ordered chaos) of healthcare environments. Interventions to translate evidence into practice can be effective with the application of behavioural approaches.^27–29 While behavioural theories, for the most part, do not distinguish between implementation and de-implementation, techniques grounded in psychology can specifically target de-implementation (eg, behavioural substitution).^{30 31} Evidence and theory from behavioural science have informed methods for identifying factors that explain and influence behaviour, selection of techniques to address the barriers, and guidance about reporting behaviour change interventions.^{29 31–34} These approaches have yet to be explicitly applied in a systematic and theory-based manner to inform interventions for reducing low-value care. The current paper addresses this gap by presenting the Choosing Wisely De-Implementation Framework (CWDIF; figure 1), informed by state-of-the-art approaches from implementation science to develop and evaluate interventions to reduce low-value care.

Figure 1.

Choosing Wisely De-Implementation Framework with preoperative testing example.

Choosing Wisely De-implementation Framework

French and colleagues²⁹ proposed a process to develop theory-informed interventions to change healthcare professional behaviour involving four key steps: Who needs to do what differently?; Using a theoretical framework, what barriers and enablers need to be addressed?; What intervention components could overcome the modifiable barrier and enhance the enablers?; How will we measure behaviour change?.

The CWDIF builds on the French model²⁹ and uses tools from behavioural science (eg, Theoretical Domains Framework and behaviour change matrix)^{31 33 35} to present a systematic framework to develop, evaluate and scale up de-implementation interventions. The CWDIF is not a health system–specific, or country-specific, framework and can be used by any initiatives to systematically and rigorously identify better de-implementation strategies to reduce low-value care. Currently, the CWDIF is being applied to de-implement two CWC recommendations (preoperative tests and imaging for low back pain) in three Canadian provinces with differing healthcare systems (work on-going; protocol manuscript in preparation) and included in a European Union funding call to reduce low-value care in four countries. As a concrete exemplar, we present our on-going work with the CWDIF to reduce low-value care in Canada: unnecessary preoperative testing for low-risk surgical procedures (see box 1).

Box 1. Application of the CWDIF: unnecessary preoperative testing.

Many preoperative tests are routinely ordered for apparently healthy patients undergoing low-risk surgery without any clinical indication, and the subsequent test results are rarely used. In addition, unnecessary testing may lead physicians to pursue and treat borderline and false-positive laboratory abnormalities.

Phase 0: identification of potential areas of low-value healthcare

The Canadian Anaesthesiologists’ Society has established its Top 5 CWC recommendations, which focus on low-value tests in ambulatory surgery. They recommend that investigations should not be ordered on a routine basis, but should be based on the patient’s health status, drug therapy and with consideration to the proposed surgical intervention.

Phase 1: identification of local priorities for implementation of CWC recommendations

Using administrative data from the Institute of Clinical Evaluative Sciences, a population-based study demonstrated overuse of low-value tests and a significant interhospital variation across 137 Ontario hospitals.² For example, 31% of patients received an ECG with 26-fold variation in Ontario hospitals.² Key health system leaders met to identify CW Ontario priorities for implementation; a key initial hospital priority was preoperative testing prior to ambulatory surgery.

Phase 2: identification of barriers to implementing CWC recommendations and potential interventions to overcome these

A Theoretical Domains Framework (TDF) study with Ontario anaesthesiologists and surgeons identified key beliefs associated with overuse of preoperative tests.⁵⁹ Findings included conflicting comments about who was responsible for the test ordering (TDF domain—Social/professional role and identity), inability to cancel tests ordered by fellow physicians (Beliefs about capabilities and Social influences) and the problem with tests being completed before the anaesthesiologists see the patient (Beliefs about capabilities and Environmental context and resources). There were also concerns that not testing might be associated with harms (overnight admissions, re-admissions). Findings from the TDF study led to the development of a pilot intervention, which focused on increasing accountability in the healthcare system for preoperative test ordering (publication in preparation).

Phase 3: evaluation of CWC implementation programmes

Our pilot study in one hospital of the proposed intervention led to a 48% reduction in low-value preoperative ECGs. We are currently conducting a parallel two-arm cluster randomised control trial with repeated cross-sectional measurements before and after intervention in 22 Ontario hospitals. Our intervention will focus on increasing accountability in the healthcare system for preoperative test ordering.

Phase 4: spread of effective CWC implementation programmes

Plans for spread of the successful intervention will include the development of a multi-jurisdictional learning platform for the sharing of methods and tools developed as well as training support for region to implement the intervention.

Phase 0: identification of potential areas of low-value healthcare

Recognition that overuse is a real issue is an essential step in the framework and engaging with decision-makers and healthcare providers about the importance of the overuse issue is critical. International programmes such as the CW campaign,^{8 9 36} Preventing Overdiagnosis initiative³⁷ as well as the BMJ Too Much Medicine^{38 39} have helped illustrate that overuse in medicine and low-value care is a problem facing many countries and that ways to target these problems need to be developed. We found that engaging with medical societies to develop a process for members to actively contribute to the identification of low-value practices in their own discipline was successful. For example, the Canadian Anaesthesiologists’ Society in partnership with CW Canada released a list of five recommendations related to preoperative testing (see box 1; Phase 0).⁴⁰ Specifically, routine preoperative testing should be avoided in specific surgical populations where these tests provide no benefit or potential harm such as complications from unnecessary delays or invasive follow-up testing for false-positives.^{40 41} The remaining Phases outline a framework for moving the recommendations into practice behaviour.

Phase 1: identification of local priorities for implementation of CW recommendations

We recommend that healthcare systems and organisations identify their own local priorities about which CW recommendations they should implement, as it is not feasible to address all identified recommendations simultaneously. Such decisions should be informed by empirical studies demonstrating overuse of low-value tests and/or significant variations in practice; and/or consensus processes involving key local stakeholders where evidence around the priority, stakeholder engagement and professional agreement that the priority is important are discussed. In the Ontario example, both empirical data (hospital administrative data) and a consensus approach (a meeting of key hospital leaders) identified local priorities (see box 1; Phase 1). We suggest using local administrative data if available, and stakeholder engagement to identify priorities for implementation based on the empirical data, evidence of lack of benefit, professional consensus, variation and/or suboptimal levels of clinical performance.

Phase 2: identification of barriers and enablers to implementing CW recommendations and potential interventions to overcome these

Reducing low-value healthcare will require numerous stakeholders to change their behaviours, the largest stakeholder group being healthcare professionals.²⁹ There is increasing recognition of the value of behavioural theories and concepts to identify barriers to implementation and potential interventions to overcome them.⁴² Adopting a behavioural approach to implementation broadens the theories, methods and tools available to promote implementation. In addition, behavioural theories can help investigate main effects, mediators (mechanisms) and moderators (effect modifiers) between behavioural influences and interventions in the environments (policy, system, organisation, team)⁴³ in which healthcare professionals work. The options for which theories to use can be overwhelming and certain theories may be better suited to different units of practice, such as individuals, groups and organisations. The Theoretical Domains Framework (TDF) may be one option for researchers who are unsure about which theory to select to use. The TDF is a comprehensive behavioural framework based on 128 constructs from 33 psychological theories to identify barriers and has been widely used to identify determinants of targeted behaviours.^{33 35}

The TDF³³ consists of 14 domains: Knowledge, Skills, Beliefs about consequences, Beliefs about capabilities, Optimism, Social, professional role and identity, Intention, Goals, Memory/attention/decision processes, Environmental context and resources, Social influences, Emotions, Behavioural regulation and Reinforcement (see table 1 for explanation of the TDF domains).^{35 44 45} The TDF was designed to be adapted to any behavioural context under investigation and increasingly has been used to investigate determinants of behaviours of both healthcare professionals and patients.⁴⁶ These determinants, or beliefs, within each theoretical domain can then be addressed by implementation techniques and strategies in an intervention.

Table 1.

TDF domains and their explanations (reprinted with permissions from Cheung et al ⁴⁴ and Patey et al ⁴⁵)

Domain	Description
Knowledge	Existing procedural knowledge, knowledge about guidelines, knowledge about evidence and how that influences what the participants do
Skills	Competence and ability about the procedural techniques required to perform the behaviour
Social/professional role and identity	Boundaries between professional groups (ie, is the behaviour something the participant is supposed to do or someone else’s role?)
Beliefs about capabilities	Perceptions about competence and confidence in doing the behaviour and how that influences their behaviour
Optimism	Whether the participant’s optimism or pessimism about the behaviour influences what they do
Beliefs about consequences	Perceptions about outcomes, advantages and disadvantages of performing the behaviour and how that influences whether they perform the behaviour
Reinforcement	Previous experiences that have influenced whether or not the behaviour is performed
Intention	A conscious decision to perform a behaviour or a resolve to act in a certain way
Goals	Priorities, importance, commitment to a certain course of actions or behaviours
Memory, attention and decision processes	Attention control, decision-making, memory (ie, is the target behaviour problematic because people simply forget?)
Environmental context and resources	How factors related to the setting in which the behaviour is performed (eg, people, organisational, cultural, political, physical and financial factors) influence the behaviour
Social influences	External influence from people or groups to perform or not perform the behaviour How the views of colleagues, other professions, patients and families, and doing what you are told, influence the behaviour
Emotion	How feelings or affect (positive or negative) may influence the behaviour
Behavioural regulation	Ways of doing things that relate to pursuing and achieving desired goals, standards or targets Strategies the participants have in place to help them perform the behaviour Strategies the participants would like to have in place to help them

Each intervention designed to address a CW recommendation will have certain assumptions and conditions that need to be considered and addressed for the intervention to be most effective. We recommend that the choice of improvement programme or intervention be based on a number of criteria: (1) diagnostic assessment of theory-informed barriers and enablers; (2) understanding the mechanism of action of intervention components designed to address the barriers; (3) empirical evidence about effects of intervention components; (4) available resources to intervention developers; (5) practical and logistical issues within the context of the healthcare setting. The first three criteria are based on the findings from barriers assessment of phase 2 and are grounded in behavioural sciences⁴⁶ and the last two criteria are based in the practicalities of working in a resource-constrained and complex environment.⁴⁷ When designing interventions, we recommend that readers consider what the best intervention components or behaviour change techniques³¹ used to address the barriers may be, the appropriate method of delivery of those intervention components as well as how the components will be operationalised in the interventions. By systematically addressing these issues in a theory-informed manner and identifying the most appropriate behaviour change techniques or intervention components to specifically target barriers identified, we increase the likelihood that the designed intervention will change the behaviour.

Phase 3: evaluation of CW implementation programmes

Given the relative lack of attention that has been paid to reducing low-value healthcare programmes, it is important to evaluate any new initiatives to generate knowledge about the effects of such programmes and how they work. In general, cluster randomised controlled trials (or well-designed quasi-experimental studies) are the gold standard for evaluating programmes.⁴⁸ However, trial results tell us whether an intervention was effective but not how and why the intervention was effective. In the absence of a theoretical underpinning, it may be difficult to interpret positive or negative effects of interventions or the failure of an intervention to bring about change.⁴⁷

One may want to consider using a range of approaches to enhance the informativeness and value of evaluations including fidelity substudies (to determine whether the content of interventions was delivered as designed), mechanistic substudies (theory-based process evaluations to determine whether our interventions activated the hypothesised mediating pathways, and if so, was this sufficient to lead to practice change),⁴⁹ qualitative process evaluations (to understand participants’ experiences of being in a trial)^{50 51} or exploratory statistical analyses (eg, subgroup analyses and hierarchical regressions to explore the effects of interventions across different contexts and gender and equity gradients). In addition, given the limited resources for health, it is also crucial to assess the value for money of such de-implementation programmes. Economic evaluation provides a useful framework to inform de-implementation decisions because it can synthesise data from various sources, provide explicit estimates of long-term costs and benefits of alternative de-implementation programmes and address the uncertainty around costs and benefits as well as the decision-maker’s dilemma. It is recommended that these substudies should be decided on a priori and include pre-planned protocols.

Phase 4: spread of effective CW implementation programmes

During Phases 1–3, particular attention should be paid to ensuring scaling up and sustainability of interventions to increase the likelihood of wider spread. Providing detailed implementation packages to healthcare system partners (including professional groups) responsible for this phase is imperative to ensure dissemination of the research findings and the potential for replication in other systems and regions.^{8 13} We recommend engagement with knowledge users and other stakeholders throughout all phases of the framework which will assist in the final phase. The goal of this phase is to spark insight and discussion on findings and future approaches for scale-up and spread of effective interventions and generate thinking and action by those participating in this phase.

Discussion

In this paper, we present a ‘how-to’ framework for any organisation to follow to de-implement low-value care in a systematic and rigorous manner. In addition, we provide a working example of the CWDIF to reduce low-value preoperative tests for low-risk ambulatory surgeries, to illustrate the utility of the framework and further contribute to the evidence base on de-implementation.

There are a wide range of de-implementation strategies available but no ‘magic bullet’ or ideal intervention to be used across all de-implementation initiatives. Evidence shows that all available strategies work some of the time but none work all the time, the observed effects are often modest and it is not always clear why this is the case.^52–54 Despite increasing policy interest in de-implementation, with international programmes such as the Choosing Wisely campaign,^{8 9} relatively little has been reported that elucidates and addresses systematic methods for designing de-implementation interventions.³⁰ It is likely that de-implementation will involve different strategies than those used for implementation, but there is little evidence to indicate what they may be.^{18 39} The CWDIF can be used to identify and evaluate the most effective approaches to de-implementation.

There are other frameworks that address de-implementation but either focus on a specific change strategy²⁰ or clinical setting,²¹ or are broad in scope.⁵⁵ For example, Ellen and colleagues⁵⁵ provide a broad framework for understanding and addressing overuse from a broad lens, recognising that de-implementation interventions may target system, policy, hospital, practice, provider and patient levels. They explicitly recognise the importance of ensuring that de-implementation strategies (at whatever level) lead to provider (and patient) behaviour change but provide little practical guidance on how to design de-implementation interventions likely to result in behaviour change. The CWDIF addresses this gap by building on advances in behavioural and implementation science to provide a stepwise theory-driven approach to designing (and evaluating) de-implementation strategies. Specifically, the CWDIF incorporates behavioural and implementation science methods for identifying factors that explain and influence behaviour,^{33 35 56} and selecting behaviour change strategies⁵⁷ to address identified barriers. While these approaches have been used widely in implementation science (eg, there are over 800 peer-reviewed publications using the TDF to change health professional behaviour,⁴⁶ they have (with few exceptions⁵⁸) been used less often to reduce low-value care.

The key challenge for de-implementation research, as for implementation research, is identifying ‘what interventions work for whom and under what circumstances’. Implementation science models highlight the importance of developing strategies based on an understanding of the likely barriers and enablers to care. This suggests that the effectiveness of implementation strategies is likely a function of the validity and comprehensiveness of the barrier and enabler assessment, the mapping of effective intervention components to address the identified barriers and enablers, the fidelity of intervention delivery and the absence of unrecognised contextual factors that might modify the effects of an intervention. The CWDIF provides a clear process for individuals to consider each of these factors and proposes tools and methods one may consider using to address them. To better understand ‘what interventions work for whom and under what circumstances’, essential elements of evaluations should include careful contextualised implementation strategy development, which builds on theory and existing knowledge and identifies potential barriers and intervention components that activate mediating mechanisms to overcome the identified barriers.

CW began as a grassroots movement to promote dialogue between doctors and their patients about low-value care and to ensure delivery of high-quality necessary medical care. The next challenge faced by CW is to develop robust approaches to support de-implementation of CW identified low-value care. While CW campaigns in many countries have actively engaged clinicians and patients in identifying and reducing low-value care, successfully de-implementing unnecessary services requires attention to both factors that perpetuate performance of overuse and the barriers to their reduction. Kerr et al recently summarised the initial experience of CW campaigns and noted that ‘Making greater inroads in reducing the use of low-value care will necessitate developing new ways to address perceived barriers’ to de-implementation’ involving the development of ‘theory-based multilevel interventions that simultaneously decrease the use of low-value care and preserve the use of appropriate care’ and ‘rigorous and pragmatic approaches to test, implement, and evaluate these interventions’.¹⁹ The CWDIF provides a systematic process grounded in behaviour sciences and methodological rigour to guide the de-implementation of low-value healthcare services.

Conclusions

This paper presents a stepwise theory-based framework for de-implementing low-value care identified through CW recommendations for reducing low-value care. It is essential that efforts to implement CW recommendations use current state-of-the-science approaches and methods from implementation science and that healthcare systems maximally learn from implementation initiatives to avoid unnecessary duplication of effort and waste. In recognition of this need, CW Canada and the international CW campaigns are committed to developing implementation research networks to establish a learning healthcare system to support de-implementation of low-value care identified in CW recommendations. This framework offers opportunities for essential proof of concepts and can evaluate the feasibility of multi-jurisdictional shared programmes of implementation research in areas of common interest.