Factors That Support Sustainability of Health Systems Change to Increase Colorectal Cancer Screening in Primary Care Clinics: A Longitudinal Qualitative Study

Abstract

Background.

From 2015 to 2020, the Centers for Disease Control and Prevention’s Colorectal Cancer Control Program (CRCCP) supported 30 awardees in partnering with primary care clinics to implement evidence-based interventions (EBIs) and supporting activities (SAs) to increase colorectal cancer (CRC) screening. This study identified factors that facilitated early implementation and sustainability within partner clinics.

Methods.

We conducted longitudinal qualitative case studies of four CRCCP awardees and four of their partner clinics. We used the Consolidated Framework for Implementation Research (CFIR) to frame understanding of factors related to implementation and sustainability. A total of 41 semi-structured interviews were conducted with key staff and stakeholders exploring implementation practices and facilitators to sustainability. Qualitative thematic analysis of interview transcripts identified emerging themes across awardees and clinics.

Results.

Qualitative themes related to six CFIR inner setting constructs—structural characteristics, readiness for implementation, networks and communication, culture, and implementation climate—were identified. Themes related to early implementation included conducting readiness assessments to tailor implementation, providing moderate funding to clinics, identifying clinic champions, and coordinating EBIs and SAs with existing clinic practices. Themes related to sustainability included the importance of ongoing electronic health record (EHR) support, clinic leadership support, team-based care, and EBI and SA integration with clinic policies, workflows, and proce- dures.

Implications.

Findings help to inform future scale-up of and decision-making within CRC screening programs and other chronic disease prevention programs implementing EBIs and SAs within primary care clinics and also highlight factors that maximize sustainability within these programs.

BACKGROUND

Federal funding for public health programs is typically allocated through time-limited funding cycles. Initial investments may discontinue following short-term improvements to population health outcomes, after which implementation of evidence-based practices can diminish, resulting in loss of preliminary program gains (Birken et al., 2020; Buchanan et al., 2005; Stirman et al., 2012). Funders and implementers alike have a vested interest in understanding factors that affect sustained implementation of evidence-based practices to ensure that initial investments result in long-term health benefits.

From 2015 to 2020, the Centers for Disease Control and Prevention (CDC) funded the Colorectal Cancer Control Program (CRCCP) based on a health systems change model integrating public health and primary care to improve population health and reduce colorectal cancer (CRC) disparities. Although CRC screening reduces incidence and mortality through early detection or prevention, screening rate disparities persist among individuals with low incomes and lower educational attainment (United States Preventive Services Task Force, 2017; White et al., 2017). Thirty funded awardees partnered with primary care clinics to implement evidence-based interventions (EBIs) recommended by the Community Preventive Services Task Force in The Community Guide, coupled with other supporting activities (SAs), to increase uptake of CRC screening (Centers for Disease Control and Prevention [CDC], 2016). Awardees supported partner clinics in utilizing EBIs and SAs that could be sustained following the initial funding cycle, facilitating long-term systems change within clinics.

CRCCP outcomes were positive. In the first 4 years of implementation, awardees partnered with 831 clinics serving over 1.3 million CRC screening-eligible patients (CDC, 2020). Most clinics (71%) were Federally Qualified Health Centers (FQHCs) traditionally serving low-income, high-need populations. Among clinics enrolled in program year (PY) 1, average CRC screening rates increased more than 12 percentage points after 4 years of implementation (CDC, 2020).

To continue these positive trends, we sought to understand how awardees implemented CRC EBIs and SAs and positioned primary care clinics—particularly FQHCs—to sustain efforts beyond the initial funding cycle. The purpose of this paper is to describe factors that facilitated early program implementation and supported sustainability among four CRCCP awardees and their partner clinics. We define sustainability as full integration of EBIs/SAs into health system and/or clinic operations, with high-quality implementation in place and the supporting infrastructure needed to maintain the EBI/SA (Maxwell et al., 2020). We captured lived experiences of CRCCP implementers through qualitative case studies in early and late phases of program implementation. This study’s findings can contribute to public health by informing future expansion and replication of CRC screening programs, and efforts to maximize sustainability in clinics.

METHOD

This study addressed the evaluation question: “What factors within primary care clinics facilitate early implementation and sustainability of CRC EBIs and SAs?” We conducted longitudinal case studies of select CRCCP awardees and their partner clinics, including qualitative interviews with key stakeholders to explore early implementation and sustainability. Figure 1 provides an overview of our methodological approach.

FIGURE 1.

Methodological Approach

Guiding Framework

The Consolidated Framework for Implementation Research (CFIR) structured our approach to understanding CRCCP factors that facilitated early implementation and sustainability of CRC EBIs and SAs in clinics. CFIR is a conceptual framework that defines five major domains influencing implementation effectiveness, including intervention characteristics; inner and outer settings; characteristics of individuals involved in implementation; and processes by which implementation is achieved (Damschroder et al., 2009). Numerous constructs within each domain further delineate program factors and are customizable for designing and conducting evaluations of health care delivery interventions.

We focused on the inner setting domain to explore factors within clinics that facilitated early implementation and sustainability (Damschroder et al., 2009). This domain was most relevant to CRCCP partner clinics’ activities. Table 1 defines specific inner setting constructs explored.

TABLE 1.

Inner Setting Construct Definitions

CFIR inner setting construct	Definition
Structural characteristics	The social architecture, age, maturity, and size of an organization.
Networks and communications	The nature and quality of webs of social networks and the nature and quality of formal and informal communications within an organization.
Culture	Norms, values, and basic assumptions of a given organization.
Implementation climate	The absorptive capacity for change, shared receptivity of involved individuals to an intervention, and the extent to which use of that intervention will be rewarded, supported, and expected within their organization.
Readiness for implementation	Tangible and immediate indicators of organizational commitment to its decision to implement an intervention.

Note. CFIR = Consolidated Framework for Implementation Research.

Site Selection

We purposefully selected four awardees—Colorado Department of Public Health and Environment, Great Plains Tribal Chairmen’s Health Board, University of South Carolina, and West Virginia University —for participation in the longitudinal case studies. These awardees were selected because they participated in the CRCCP Learning Laboratory, a project to develop and apply a standardized approach to evaluate implementation, effectiveness, cost, and cost-effectiveness of multicomponent interventions to inform future intervention expansion and replication (Tangka et al., 2019). These awardees exhibited strong partnerships with FQHCs to implement EBIs/SAs (Subramanian et al., 2018) and experienced an average screening rate increase of 13 percentage points in the first 3 or 4 years of implementation, increasing the likelihood that they could identify facilitators for implementation and sustainability. Although awardees implemented five to eight EBIs/SAs, implementation varied across their clinic partners depending on clinic capacity and patient needs.

Data Collection

We conducted two sequential rounds of qualitative interviews with key stakeholders in awardee programs, implementation partner organizations, and partner clinics (all FQHCs). We selected these respondent types to gain multiple perspectives on program implementation. In 2017, we conducted 26 semi-structured early implementation interviews with awardee management and implementation staff; FQHC staff; awardees’ evaluators; and implementation partner organizations representing the four awardees covering the period between the start of PY1 (July 2015) through the end of PY2 (June 2017). We used convenience sampling to interview at least two awardee staff and one to two individuals from remaining respondent types per program until saturation was reached. Interviews explored clinic selection and recruitment; EBI and SA selection and implementation; clinic champions; CRC screening data validity; and partnerships. In 2019, we conducted 15 semi-structured late implementation interviews with awardee staff and implementation partner organizations covering the period between the start of PY3 (July 2017) through early PY5 (August 2019). Neither FQHC staff nor evaluation staff were interviewed in late implementation as we aimed to focus on awardees’ and implementation partners’ efforts to support sustainability in partner FQHCs. Interviews explored facilitators and barriers to sustainability, EBI and SA integration, clinic champions, funding environment, the role of electronic health records (EHRs) in sustainability, and leadership support. A subset of 10 interviewees participated in both early and late implementation interviews. Table 2 summarizes the respondent types by awardee and interview timing.

TABLE 2.

Interviews by Respondent Type and Site

	Site 1		Site 2		Site 3		Site 4		Total interviews
Respondent Type	E*	L*	E	L	E	L	E	L	E	L
Awardee staff	2	2	2	2	3	2	2	2	9	8
FQHC staff	1	0	2	0	1	0	2	0	6	0
Implementation partner staff	2	2	2	2	1	2	1	1	6	7
Evaluation staff	1	0	1	0	2	0	1	0	5	0
Total interviews	6	4	7	4	7	4	6	3	26	15

Note. FQHC = Federally Qualified Health Centers.

^*E = early implementation; L = late implementation.

Interview guides were developed based on respondent roles and evaluation questions. Verbal consent for interviews was obtained from each respondent. Research Triangle International’s Institutional Review Board determined that this data collection did not constitute human subjects research. Interviews were audiorecorded and transcribed verbatim for analysis.

Analysis

We conducted analysis following each round of interviews. We first developed a coding dictionary based on evaluation questions. Analysts pilot-coded two interview transcripts using the qualitative software NVivo QSR versions 10.0 and 11.0. Analysts met to reach consensus on refinement and application of codes. Eight early implementation interviews (33%) and three late implementation interviews (20%) were double coded (i.e., independently coded by two analysts), and analysts achieved a kappa coefficient of 0.8 or higher for each (Miles et al., 2013). Analysts then reviewed code reports to identify emerging themes and created tables including site-specific themes, illustrative quotes, and respondent site and role identifiers. Analysts reviewed the tables to identify cross-cutting themes, defined as having relevance across two or more awardees, and supported by two or more respondent types. We used an iterative approach whereby themes from early implementation interviews informed late implementation data collection.

Analysts mapped cross-cutting themes from early and late implementation interviews to five CFIR inner setting constructs (Table 1). To best interpret these themes, analysts combined two pairs of constructs—Structural Characteristics with Readiness to Implement, and Networks and Communication with Culture—for analysis.

RESULTS

We identified eight cross-cutting themes during early and late implementation interviews to help describe factors that supported early CRCCP implementation and sustainability. These themes are presented in alignment with CFIR inner setting constructs of interest (Table 3). Themes are further illustrated through respondent quotes in Supplemental Material. We present themes as they relate to early implementation or sustainability; however, themes for each construct are linked and influence one another over time (see Table 3). We further explore these longitudinal relationships in the Discussion.

TABLE 3.

Overview of Themes by CFIR Inner Setting Construct

Note. CFIR = Consolidated Framework for Implementation Research; EHR = electronic health records; CRC = colorectal cancer.

EARLY IMPLEMENTATION

Structural Characteristics/Readiness for Implementation

Using Clinic Readiness Assessment Results to Tailor Implementation Activities.

Respondents from three awardees described working with FQHCs to formally assess their implementation readiness during the application process or initial site visits. Although the scope of assessments and definitions of “readiness” varied, awardees generally examined: (a) CRC burden and risk, (b) CRC screening practices, (c) experience implementing similar EBIs and SAs, (d) effectively engaging partners, (e) current EHR systems, including problems with EHRs related to EBI implementation or measuring CRC screening rates, and (f) CRC screening rates. FQHC staff, in collaboration with awardees, used assessment results to inform selection of EBIs and SAs for implementation and address EHR issues. For example, one awardee conducted in-depth chart reviews to identify inconsistencies in FQHC staff documentation of CRC screening tests, validate screening rates, and inform provider trainings.

Providing Moderate Funding to FQHCs.

Awardees emphasized the importance of providing funding to FQHCs to support uptake of EBIs and SAs but cautioned that funding could be moderate to encourage sustainability of CRCCP practices. Respondents noted the importance of planning for sustainability from initiation of CRCCP implementation by allocating funding to integrate EBIs into FQHC practices. Respondents reported that FQHCs were encouraged to apply funding in ways that directly supported sustainability, such as building infrastructure and long-term FQHC practices (e.g., EHR assessments, quality improvement [QI] efforts, and workflow adaptations) versus funding staff positions (e.g., patient navigators) requiring new funding sources once CRCCP funding ends.

Networks and Communication/Culture

Identifying CRC Screening Champion(s).

Respondents described aspects of FQHC communication and culture that bolstered early implementation, including the importance of identifying champions from varied roles (e.g., QI expert, physician) during FQHC recruitment or early implementation who could help prioritize CRC screening within FQHCs. Three awardees described the importance of having a clinic champion to support EBI/SA implementation and sustained FQHC engagement. Respondents described a mix of “organic” champions—passionate FQHC staff committed to the program—and “cultivated” champions—FQHC staff selected to complete champion training.

Implementation Climate

Coordinating EBIs and SAs With Existing FQHC Prac- tices.

A key theme across all awardees was the essential nature of coordinating EBIs and SAs with existing FQHC practices to expand reach of clinics’ prevention efforts and increase efficiency. Opportunities for coordination were identified during clinic assessments. Examples included incorporating EBIs/SAs into existing programs (e.g., combining mammography or flu vaccination with fecal immunochemical testing) and streamlining FQHC reporting requirements across chronic disease programs. By coordinating EBI and SA delivery with other disease prevention services, FQHCs more efficiently utilized resources; improved data reporting practices; minimized burden to FQHC staff and patients; and improved patient outcomes by increasing the likelihood of screening patients.

SUSTAINABILITY

Structural Characteristics/Readiness for Implementation

Providing Ongoing Support for Optical EHR Use.

Respondents emphasized the value of providing technical assistance (TA) to FQHCs throughout the funding cycle to build QI capacity and a culture that supported sustainability. Awardee support focused on optimizing EHRs to generate high-quality CRC screening data. For example, one awardee conducted in-depth chart reviews in partner FQHCs to identify inconsistencies in documentation of CRC screening tests, validate screening rates, and inform provider trainings. Two awardees described working with FQHCs to review and standardize workflow processes in relation to EHR data to improve data quality. Finally, two awardees offered FQHCs specialized health information technology (IT) TA to increase FQHC capacity to correctly calculate CRC screening rates and produce accurate data reports using EHRs. To support sustainability, awardees also provided TA to FQHCs throughout the funding cycle for automating EBIs and SAs in EHRs, such as provider reminders and provider assessment and feedback. One respondent stated: “If you have an electronic metric system that does automated patient and provider reminders, we found that one is the bedrock of sustainability.” Awardees aligned the type and amount of TA offered with each FQHC’s IT capacity.

Awardees also provided ongoing support to overcome the complexity of many EHR systems, a challenge among FQHCs that stalled their ability to optimize data management and reporting features. Respondents from two awardees stated that EHR system complexity can lead providers to incorrectly document ordering and completion of CRC screening tests. Respondents also reported some EHR systems were unable to differentiate between CRC screening test types, which led to inaccurate screening rate calculations.

In addition to technical support, respondents noted the value of having at least one FQHC staff member with expertise to fully utilize the EHR system and inform QI efforts. This ensured proper collection and management of high-quality CRC screening data and generated useful reports to inform patient follow-up.

Respondents noted the negative impact of changing EHR systems on sustainability because it “throws everything back; it’s like starting all over.” Frequent EHR changes in some FQHCs hindered sustainability when EBIs and SAs were already integrated into a previous system but not yet integrated into the new system. To support FQHCs undergoing EHR changes, awardees connected them with other clinics already using that EHR system to assist.

Networks and Communication/Culture

Emphasizing FQHC Leadership Support.

Clear, visible leadership support helped facilitate sustainability by enabling institutionalization of EBIs and SAs within FQHCs. Champions leveraged external policies/reporting requirements (e.g., Health Resources and Services Administration Uniform Data System CRC screening performance measure) to make the case for sustained implementation of EBIs/SAs to upper levels of clinic administration.

Adopting a Team-Based Approach.

Respondents stated that adopting a team-based approach to care facilitated sustainability by safeguarding FQHCs from losing momentum when staff turnover and fostering shared commitment to CRC screening. Respondents described team-based models in which FQHC staff understood their role as part of a health care team and how their role fit into the FQHC’s goals. Team-based approaches facilitated sustainability by ensuring staff at all levels were trained and engaged in the process of promoting and implementing EBIs/SAs. Ongoing staff education and training were critical for sustaining EBIs/SAs embedded in clinic workflows and standard operation procedures (SOPs).

Implementation Climate

Integrating EBIs/SAs Into Existing FQHC Workflows, Policies, and SOPs.

Respondents across all four awardees discussed integration as a way to expand patient reach across chronic disease programs; decrease staff and patient burden; and support sustainability by pooling implementation resources for increased efficiency. Respondents also noted the benefits of integrating EBIs and SAs into SOPs that encompass multiple chronic diseases to support sustainability. Examples include incorporating EBIs/SAs into existing programs and streamlining health department reporting requirements across chronic disease programs. Through integration, FQHCs were able to maintain prioritization of CRC screening in the face of competing demands. Respondents emphasized the importance of promoting an integrated approach from the start of implementation, particularly among clinics with competing priorities and/or limited capacity. Certain EBIs, like patient and provider reminders, could be automated and therefore more easily sustained over time.

DISCUSSION

This longitudinal qualitative study identified factors facilitating early implementation and sustainability of CRC EBIs and SAs beyond the initial funding cycle. Qualitative findings provide unique perspective on how awardees and their partner FQHCs implement and prepare to sustain EBIs and SAs aimed at increasing CRC screening.

Sustaining evidence-based practices require comprehensive program planning that considers population health needs, program components, program drivers, and the clinic context and resources associated with program delivery (Gruen et al., 2008). This planning can focus on integration and sustainability beginning early in the funding cycle (Johnson et al., 2004; Pluye et al., 2004). Integrating evidence-based processes into multiple aspects of clinic operations—including funding allocation, staff training, care delivery models, and EHR systems—can be complex but improve the prospect of sustainability. Given the limited nature of public health funding cycles, it is critical that health systems change efforts not only work to achieve population health outcomes but also dedicate time and resources to integrating effective strategies for increased likelihood of long-term sustainability. Integrating EBIs and SAs into existing FQHC processes proved essential to CRCCP sustainability. Previous research identified an increase in the proportion of CRCCP partner clinics with sustainable EBIs and SAs in place over time, from 24 to 34 percentage points over a 2-year period (Maxwell et al., 2020). Understanding the extent to which EBI and SA implementation continues in CRCCP partner clinics following termination of CRCCP funding and TA could address long-term sustainability.

Our findings highlight the essential role of awardees in using readiness assessment findings to “meet clinics where they are” and mobilize strength-based partnerships—a critical tenet of public health service—to identify opportunities to integrate EBIs and SAs into existing clinic protocols (CDC, 2021). Others have recognized the importance of collaborative partnerships and meeting clinics where they are to support CRCCP implementation (DeGroff et al., 2008; Glover-Kudin et al., 2013; Rohan et al., 2013; Schlueter et al., 2018).

Team-based care delivery models also supported sustainability. Team-based care has been shown to improve efficiency, effectiveness, value of care, and patient and provider experiences (Jesmin et al., 2012; Schottenfeld et al., 2016). Multidisciplinary health care teams trained to implement EBIs and SAs through integrated workflows and SOPs normalized these processes as standard practice and enabled clinics to better prepare for sustainability. Respondents in this study largely discussed the utility of integration within existing team-based care structures. Clinics without team-based structures in place prior to implementing a CRC health systems change effort may require additional support as they adjust to providing integrated services. Future evaluation could focus on optimal approaches to successfully integrate CRC EBIs and SAs within different clinic settings.

This study reinforces the value of clinic champions previously identified (Miech et al., 2018; Soo et al., 2009) to continuously prioritize CRC screening efforts. Whether champions served in leadership or other FQHC staff roles, their ongoing promotion of EBIs and SAs, and clear and consistent leadership support, was paramount to prioritize these efforts. CRC screening champions have been associated with increased CRC screening rates and higher overall clinic performance (DeGroff et al., 2018; Maxwell et al., 2020). Given the clear benefits of having a screening champion, awardees can prioritize working with clinics to develop concrete plans to establish, support, and maintain champions despite staff turnover or funding changes. Understanding how clinic champions can have such a positive effect on CRC screening outcomes can help identify and maintain them. CDC is currently conducting a process evaluation of clinic champions’ activities within CRCCP partner FQHCs. However, additional qualitative perspectives exploring champions’ characteristics and varying clinic roles, and best practices for supporting integration and sustainability, can be beneficial to understanding the nuance of this role.

Effective EHR use is foundational for institutionalizing EBIs and SAs and supports high-quality data management and reporting, thereby making continued implementation more feasible. The availability of high-quality and timely data allows clinics to better monitor CRC screening rates over time. These data can be used to assess provider performance and provide feedback reports, an evidence-based approach shown to increase provider referral for screening (CDC, 2016). Accurate screening rate data are essential to measuring screening over time and highlighting improvements. Clinics may have opportunities to support future funding efforts by demonstrating screening rate improvements. Study results support the need for awardees and implementation partners to provide health systems and clinics with tailored TA for their specific EHR system throughout the funding cycle to support EHR optimization for CRC screening, including avoiding common pitfalls (e.g., improper data entry, overly complex systems, changing EHRs) in ever-changing health care environments.

Providing funding to partner clinics can facilitate program start-up and implementation. Our results suggest adequate funding can facilitate strong EBI and SA implementation, particularly in FQHCs providing comprehensive primary care to medically underserved communities and vulnerable populations that may be financially strained (Kim et al., 2020). However, this study emphasized the need for funding amounts to be “just right”– enough to support successful expansion and replication of EBIs and SAs without creating financial dependency on CRCCP awardees. Identifying appropriate levels of funding by clinic type could be helpful for funders allocating financial resources. Funders might consider structuring funding requirements to support sustainability planning throughout implementation to ensure funded clinics are prepared to implement long-term systems change.

This study has at least four limitations. First, as a qualitative study conducted with select CRCCP awardees, findings are not generalizable to all CRCCP awardees or FQHCs. Second, findings reflect a small number of respondents from a subset of CRCCP awardees and partner FQHCs and, therefore, reflect the opinions of a relatively small number of study participants. Third, we did not track CRC screening outcomes to reflect the specific time periods represented in this study and, therefore, cannot draw conclusions on the overall success of participating awardees’ approaches. Finally, EHR systems used by partner clinics varied so challenges associated with specific EHR systems were not explored.

IMPLICATIONS FOR PRACTICE

This study identified factors that support early implementation and sustainability of CRC EBIs and SAs among awardees and their partner FQHCs. Findings help to inform future expansion and replication of EBIs and SAs, and decision-making within CRC screening programs and other chronic disease prevention programs implementing sustainable evidence-based strategies within primary care clinics. Future research can explore the impact of these implementation factors on clinic outcomes and sustained activities.