DeImplementing Ineffective and Low-Value Clinical Practices: Research and Practice Opportunities in Community Oncology Settings

Abstract

Patients, practitioners, and policy makers are increasingly concerned about the delivery of ineffective or low-value clinical practices in cancer care settings. Research is needed on how to effectively deimplement these types of practices from cancer care. In this commentary, we spotlight the National Cancer Institute Community Oncology Research Program (NCORP), a national network of community oncology practices, and elaborate on how it is an ideal infrastructure for conducting rigorous, real-world research on deimplementation. We describe key multilevel issues that affect deimplementation and also serve as a guidepost for developing strategies to drive deimplementation. We describe optimal study designs for testing deimplementation strategies and elaborate on how and why the NCORP network is uniquely positioned to conduct rigorous and impactful deimplementation trials. The number and diversity of affiliated community oncology care sites, coupled with the overall objective of improving cancer care delivery, make the NCORP an opportune infrastructure for advancing deimplementation research while simultaneously improving the care of millions of cancer patients nationwide.

Spurred in part by educational campaigns (1), reports on waste in health-care delivery systems (2), and medical reversals (3), there is increasing recognition of the use of ineffective, low-value, and even harmful clinical practices. This issue is of particular relevance in the area of oncology, where many interventions may have serious harms that must be weighed against any potential benefits, and the costs of treatment, often resulting in financial toxicity for patients, are escalating. A recent systematic review identified overuse of 154 cancer-related services, including imaging, procedures, and therapeutics (4). High-profile trials have also highlighted overuse, including the Trial Assigning Individualized Options for Treatment (5), which identified patient populations for whom chemotherapy may not be needed, and the trial by Venook and colleagues (6) comparing chemotherapy with cetuximab or bevacizumab, which showed no additional benefits but higher costs. Although recent efforts are laudable, the time is right to shift from a predominant focus on describing ineffective and low-value practices toward a more balanced approach that includes deimplementing such practices.

In this commentary, we discuss opportunities for advancing research on deimplementation within a specific network of community oncology care delivery settings, where the majority of cancer cases are managed and in which research findings can be quickly and directly moved into practice. Building on our prior work (7-9), we review key concepts in deimplementation, with an emphasis on multilevel barriers that inhibit deimplementation efforts, and highlight the need to develop and test strategies to drive deimplementation. We describe a specific national network of community oncology care delivery settings that is uniquely positioned to conduct research on deimplementation and affect the practice of deimplementation. We discuss research approaches and study designs for testing deimplementation strategies and close by highlighting resources and opportunities for strengthening research–practice partnerships that are essential for reaching these goals. This commentary serves to highlight a unique opportunity for oncology researchers and practitioners to leverage a national network of cancer care practices to advance our scientific understanding of deimplementation while maximizing our impact on improving care delivery.

Key Concepts in De-Implementation

Deimplementation is a complex process involving an interplay between different types of actions, clinical practice characteristics, multilevel barriers, and multilevel strategies, as shown in Figure 1 (9). Briefly, deimplementation may involve different actions, including removing, replacing, reducing, or restricting the use of a clinical practice. Deimplementation may be more challenging than the original implementation of common practices because it often works against prevailing intuitions and belief systems (as well as remuneration policies) that reinforced dissemination of the existing clinical habits.

Figure 1.

Framework for deimplementation in cancer care delivery. The framework for deimplementation in cancer care delivery is a conceptualization of multilevel and multifaceted factors that affect the deimplementation process and lead to deimplementation outcomes. These factors include strength of evidence, magnitude of problem, type of action, barriers and facilitators, and strategies. This framework was previously published by Norton, Chambers, and Kramer in the Journal of Clinical Oncology (9). As a US government work, it cannot be covered by copyright according to section 105 of the Copyright Act and is therefore considered to be in the public domain.

Barriers to deimplementation often exist across multilevel influences of cancer care delivery, including patients, health-care practitioners, organizations (eg, hospitals, clinics), and societal factors. Some patients may be reluctant to forego a cancer screening test for fear of missing a diagnosis, even if they are at very low risk, when administering the test runs counter to evidence-based screening guidelines. For example, despite clinical guidelines, some men may want to continue routine prostate-specific antigen screening after the recommended upper age limit (10), and some women may prefer to have cervical cancer screening every year rather than the recommended every 3-5 years (11). Overscreening among elderly patients is particularly concerning given the potential for harm relative to the likely benefits; indeed, a recent study reported more than 45% of approximately 175 000 individuals received cancer screening despite being older than what is recommended by clinical guidelines (12). Some patients may also prefer more advanced or technologically complex yet unnecessary prevention and treatment modalities, such as the use of magnetic resonance imaging vs mammography for breast cancer screening, even when they are of average or low risk (13), or they may prefer aggressive yet unnecessary treatment, such as radiation therapy or radical prostatectomy for localized prostate cancer rather than active surveillance or watchful waiting (14).

In addition to patient-level drivers of overuse, there exist health-care-practitioner-, organizational-, and societal-level drivers, as well. For example, some oncologists may err on the side of overuse of cancer treatment rather than underuse for fear of accusations of medical malpractice. Some cancer care delivery organizations may lack strong leadership to champion efforts to reduce low-value care. The pervasive social and societal norms that more care is always better care further inhibit successful deimplementation. Examples include societal norms that more frequent cancer screening is better, even when it is not recommended based on age and risk level. These and other barriers have been well documented in the literature.

However, we have less evidence about which deimplementation strategies can overcome multilevel barriers and effectively remove, replace, reduce, or restrict the use of ineffective and low-value ingrained clinical cancer practices. A systematic review found relatively few types of deimplementation strategies to reduce low-value care and still fewer that had been tested and shown to be effective in rigorous trials (15). Most of these studies, however, were limited to addressing patient- and/or provider-level barriers toward reducing low-value care and largely focused on select innovations (eg, medications) and delivery settings (eg, hospitals). Clearly, additional research is needed to broaden and deepen our scientific understanding of deimplementation strategies.

De-Implementation in Community Oncology Care Settings

Trials that test deimplementation strategies may take place in a variety of cancer care delivery systems, such as Kaiser Permanente, Veterans Health Administration, and academic medical centers. However, some of these systems tend to be relatively highly resourced, located in urban areas, and include a different patient mix compared with community cancer care settings in which the majority of patients receive treatment. To generate relevant, applicable, and generalizable knowledge on deimplementation strategies, trials should also take place in community cancer care delivery settings to reflect the range and diversity of contexts and patient populations in which oncology care is delivered. The National Cancer Institute (NCI) Community Oncology Research Program (16) (NCORP; https://ncorp.cancer.gov), a specific network of community cancer care delivery settings, is poised to conduct deimplementation research in service of generating real-world evidence that can readily inform and support changes to cancer care delivery.

Designed to support cancer prevention trials and cancer care delivery research, the NCORP is a national network of oncology practices that seeks to bring clinical trials and cancer care delivery research to people in their own communities across the United States. Currently, the NCORP includes 32 community oncology sites, where each site includes a mix of hospitals, oncology practices, and/or integrated health-care systems. The NCORP also includes 14 minority and underserved community sites, defined as sites that have a patient population that is at least 30% racial and ethnic minorities or rural residents, as a way to improve health equity and address cancer disparities by supporting trials that more accurately represent national diversity. The network is composed of 1017 hospitals, cancer centers, and oncology clinics, with broad geographic coverage (44 states, District of Columbia, Puerto Rico, and Guam) and diverse cancer care practitioners and patient populations. Almost 10 000 practitioners participate in NCORP, including physician investigators and health-care professionals, and the network has enrolled more than 30 000 patients in NCI clinical trials to date. The NCORP research portfolio includes more than 100 studies (www.ncorp.cancer.gov; accessed December 14, 2020) led by NCORP and non-NCORP investigators. Within NCORP, cancer care delivery research has evolved over time to include studies on a range of topics (eg, smoking cessation, guideline adherence, distress treatment) with diverse patient populations (eg, adults with prostate cancer, pediatric cancer patients, adults with solid tumors) using various study designs (eg, randomized controlled trials [RCTs], observational), with the overall goal of conducting research that has the potential to change practice (17). Investigators interested in learning more about NCORP, including participating community oncology sites, ongoing and completed studies, educational resources, opportunities for collaboration, and processes for conducting research studies within the NCORP (eg, extramural investigator-initiated grant applications, internal proposal and review process through NCI’s National Clinical Trials Network and NCORP Research Bases) are encouraged to visit www.ncorp.cancer.gov. Because these processes may change over time, investigators are encouraged to submit queries to ncorp@mail.nih.gov.

The NCORP offers several distinct advantages for advancing deimplementation research and affecting deimplementation practice. With respect to research, trials that test strategies to change practitioner and/or organizational behavior often require dozens—if not hundreds—of organizations (vs patients) to be sufficiently powered to detect statistically significant differences between study arms. Given these methodological requirements, conducting such trials is often limited to integrated delivery systems, which may not reflect the range of practitioners and organizations in which diverse patient populations receive oncology care. With more than 1000 cancer care delivery organizations, NCORP is uniquely positioned to conduct these large trials and broaden the generalizability of research findings to diverse cancer care delivery settings, practitioners, and patients. Moreover, patients receiving care at NCORP-affiliated sites represent a broader range and geographic distribution of patients relative to most clinical research settings. Trials conducted within NCORP have the potential to improve health equity and reduce cancer disparities, given that the NCORP infrastructure affords traditionally underrepresented and underserved patients and care delivery settings an opportunity to participate in and benefit from research.

The NCORP infrastructure is uniquely positioned to support practitioner-centered research (18). Much like patient-centered research, practitioner-centered research involves practitioners throughout all phases of a study. Practitioners can help prioritize which low-value or ineffective clinical practices should be deimplemented, identify candidate deimplementation strategies that are feasible and acceptable (but for which evidence of their effectiveness is unknown), formulate research questions and select study designs that are rigorous yet realistic, help interpret study results, and provide suggestions for future deimplementation studies. Focusing on issues that are important to practitioners may also increase trial participation (17), because poor study accrual is one of the most common reasons why trials are not completed.

With respect to deimplementation practice, the NCORP can accelerate dissemination of deimplementation research results, given that such results are generated by the very people who will put them into practice. Many frontline practitioners in NCORP are affiliated with national professional societies (eg, American Society of Clinical Oncology), engaged in collaboratives with insurance companies, communicate often with corporate entities seeking to reduce costs and waste, and considered experts in high-quality care within their own practices, all of which provide outlets for sharing information about deimplementation trials. NCORP practitioners can also leverage their social networks to share information about deimplementation trials, serve as knowledge brokers between similar yet unconnected groups, and champion efforts to deimplement low-value and ineffective clinical practices in their own care delivery settings.

Rigorous Designs for De-Implementation

Developing, testing, and identifying effective deimplementation strategies are critically important for guiding practice efforts to deimplement ineffective or low-value cancer practices. The most opportune study design for testing such strategies is the cluster RCT, where randomization to an experimental (ie, deimplementation strategy) or control condition occurs at the level of the organization (ie, cluster; eg, hospital, clinic, practice) rather than at the level of the patient (19-21). Cluster RCTs are preferable for most studies that test strategies to change practitioner behavior and/or change care delivery, because they minimize threats to internal validity by reducing the probability that results will be biased because of contamination between study conditions.

However, given their focus on emphasizing internal validity and minimizing bias, RCTs (including individual and cluster RCTs) are often criticized for inadequate generalizability. As such, they may be perceived as generating evidence that is neither relevant nor reflective of the realities of everyday patients, practitioners, and clinical care contexts. This is a common criticism of clinical trials, including those in oncology, because they may inadvertently further exacerbate health inequities by limiting participation, representation, and application of trials to select patient populations. However, one type of RCT—the pragmatic RCT—is designed specifically to address these concerns.

First articulated by Schwartz and Lellouch (22), pragmatic RCTs are conceptualized on a continuum from more explanatory to more pragmatic. Explanatory trials are designed to answer the question “Does this intervention work in ideal settings?” whereas pragmatic trials are designed to answer the question “Does this intervention work in typical or routine settings?” Trials are differentiated along this continuum by 9 domains [eg, recruitment, enrollment, primary outcome, and analysis (23)]. Each of the 9 domains represent important decisions in the trial planning phase; all of the domains are assessed on a continuum from more explanatory to more pragmatic via a validated tool, the PRagmatic Explanatory Continuum Indicator Summary-2 (PRECIS-2; www.precis-2.org) (23). An adapted version of PRECIS-2—the PRECIS-2-Provider Strategies tool (24)—may be particularly helpful in planning trials to test deimplementation strategies that target provider- and organizational-level changes.

NCORP-affiliated community oncology sites reflect the diverse landscape of cancer care delivery. NCORP sites that would opt to participate in pragmatic trials would generate evidence that is reflective and representative of the typical settings in which most cancer patients receive care. Moreover, participating clinics would have the opportunity to be part of the trial planning phase and ensure that the purpose and design of the trial matches the overall intent of the trial—that is, generating rigorous evidence that is applicable to everyday practice. Several pragmatic trials have already been conducted in NCORP [eg, Optimizing Lung Screening (25) trial that tested strategies to implement smoking cessation among 26 imaging clinics], demonstrating this type of study design to be feasible and acceptable within NCORP.

Opportunities for potential pragmatic RCTs for testing deimplementation strategies to reduce or stop the use of ineffective and low-value practices in NCORP are plentiful. Many ineffective or low-value clinical practices would be appropriate targets of a deimplementation trial, including any of the 75 recommendations from a search of “oncology” that are included as part of the Choosing Wisely campaign (https://www.choosingwisely.org/clinician-lists/). Value-based guidelines may also be sources for identifying practices for deimplementation trials, such as the use of costly chemotherapy regimens that are no more effective than less expensive chemotherapy regimens, even when the latter may be less clinically and financially toxic for patients. Deimplementation strategies, such as variations (eg, frequency, format) in audit and feedback, changes (eg, default, options) to electronic health record order sets, and the use of clinical champions, could be tested in deimplementation trials, where 1 or several types of strategies are randomized and deployed at the clinic or practice level.

In circumstances where RCTs are neither feasible nor acceptable, alternative study designs can be used to test deimplementation strategies in this specific network of community oncology care settings. Although these alternative designs are prone to more biases and confounders than RCTs, they can provide sound evidence on the effectiveness of deimplementation strategies when well designed. Quasi-experimental designs, such as interrupted time series and regression discontinuity (26), can test the impact of a deimplementation strategy on reduction or cessation of a practice without randomization to study arm. For example, an interrupted time series design could be used to test the impact of patient-provider communication training on reducing guideline-discordant use of breast cancer screening by leveraging electronic health record data to track changes before and after the training, and ideally between other practices that did not receive the training. Oncology modules in electronic health records could also track the effect of a clinical champion on type of treatment (eg, higher- vs lower-cost chemotherapy) or modality of diagnostic procedure (eg, open breast biopsies vs core needle biopsies).

Qualitative data can provide an in-depth understanding of the context in which deimplementation would occur among NCORP-affiliated sites and nicely complement quantitative data from trials testing deimplementation strategies (27). Semistructured qualitative interviews with health-care professionals may reveal specific challenges to deimplementation of low-value cancer care practices. For example, health-care professionals may resist deimplementing the use of high-cost antiemetics for regimens with low emetogenic potential. Qualitative data would give insight into why physicians continue to use high-cost regimens (perhaps default in order sets, lack of treatment management tools) and inform the selection of deimplementation strategies (perhaps changes to order sets, setting up clinical pathways programs) that should be tested in future deimplementation trials.

In addition, mixed-methods studies, which include collection, assessment, and integration of both qualitative and quantitative data, allow for a better understanding of deimplementation processes and outcomes (28). For example, for trials that do not show statistically significant differences between the deimplementation strategies arm and the control arm, semistructured interviews with participants (eg, oncologists, nurses, quality improvement leaders) could be conducted post hoc to explore why such strategies were ineffective. This mixed-methods approach would help characterize subtle yet important contextual differences, such as organizational culture, leadership, and team-based care, that may vary between NCORP-affiliated community oncology care sites. Such data would also be important for selecting alternative strategies that should be tested in future deimplementation trials.

Conclusion

Research is needed to identify deimplementation strategies that can effectively decrease or stop the use of ineffective and low-value clinical cancer practices. The NCORP provides an infrastructure for supporting this type of research, ensuring that proposed studies are scientifically sound, attending to the realities of cancer care delivery, and focusing on timely topic areas that are important to health-care professionals in oncology. Some resources and opportunities to advance deimplementation research, change clinical practice, and strengthen research-practice partnerships are already available (Table 1). For example, researchers may seek out training programs that focus on developing and testing multilevel interventions, such as the NCI Multilevel Intervention Training Institute, to be better equipped to conduct rigorous trials on deimplementation strategies. Other training programs and educational resources, such as those offered on pragmatic trials in health-care delivery settings (eg, Health Care Systems Research Collaboratory, National Institutes of Health), can help build a cadre of researchers who can conduct trials that are relevant and responsive to the needs of NCORP clinical and administrative partners. Increasing researchers’ and practitioners’ awareness of funding opportunities to support deimplementation research would be essential, too. Equally important is building equitable and sustainable research–practice partnerships; dedicated time for facilitated interactions, knowledge sharing, and informal “matchmaking” among researchers and health-care practitioners at professional meetings would go a long way in service of pursuing this goal.

Table 1.

Opportunities for supporting deimplementation research and practice in community cancer care settings

Type	Description	Examplesa
Research funding opportunities	Funding opportunities to support research on deimplementation	Dissemination and Implementation Research in Health, PAR-19-274, National Institutes of Health
		Deimplementation of Ineffective or Low-Value Clinical Practices along the Cancer Care Continuum, Notice of Special Interest, NOT-CA-20-021, National Cancer Institute
Research training programs	Training programs to enhance researchers’ scientific skills to study deimplementation in cancer care delivery	Training Institute for Dissemination and Implementation Research in Cancer, National Cancer Institute
		Multilevel Intervention Training Institute, National Cancer Institute
		AcademyHealth Delivery System Science Fellowship
		Pragmatic Clinical Trials Workshop, Health Care Systems Research Collaboratory, National Institutes of Health
		Cancer Prevention Fellowship, National Cancer Institute
Opportunities for research-practice partnerships	Venues for developing and supporting collaborations among cancer care delivery researchers and practitioners	American Society of Clinical Oncology Quality Care Symposium
		Implementation Science Consortium in Cancer, National Cancer Institute
		Society for Medical Decision Making
		NCI Community Oncology Research Program

^aSelect but not exhaustive examples of opportunities to support research on deimplementation of ineffective and low-value clinical care practices. NCI = National Cancer Institute.

In closing, the NCORP is an example of an opportune infrastructure that holds promise for advancing research on deimplementation and changing the practice of deimplementation, in service of optimizing cancer care and improving patients’ health outcomes in community oncology settings.

Funding

Not applicable.

Notes

Role of the funder: Not applicable.

Disclosures: Drs Norton, McCaskill-Stevens, Chambers, Stella, and Kramer have no disclosures to report. Dr Brawley is a consultant for Genentech-Roche on accrual to clinical trials.

Author contributions: WEN, DAC, BSK, and WMS developed the idea for the manuscript and outlined a first draft. PS and OB provided clinical content, case examples, and provider perspectives on research in community oncology settings. WEN drafted the first version of the manuscript; all authors added content and contributed to edits. All authors reviewed and approved the final version of the manuscript.

Disclaimers: The observations and conclusions in this article are those of the authors and do not represent the official position of the National Cancer Institute, National Institutes of Health, or other US federal agencies.

Data Availability

There are no new data associated with this article.