A Conceptual Framework and Metrics for Evaluating Multicomponent Interventions to Increase Colorectal Cancer Screening Within an Organized Screening Program

Abstract

BACKGROUND:

Multicomponent, evidence-based interventions are viewed increasingly as essential for increasing the use of colorectal cancer (CRC) screening to meet national targets. Multicomponent interventions involve complex care pathways and interactions across multiple levels, including the individual, health system, and community.

METHODS:

The authors developed a framework and identified metrics and data elements to evaluate the implementation processes, effectiveness, and cost effectiveness of multicomponent interventions used in the Centers for Disease Control and Prevention’s Colorectal Cancer Control Program.

RESULTS:

Process measures to evaluate the implementation of interventions to increase community and patient demand for CRC screening, increase patient access, and increase provider delivery of services are presented. In addition, performance measures are identified to assess implementation processes along the continuum of care for screening, diagnosis, and treatment. Series of intermediate and long-term outcome and cost measures also are presented to evaluate the impact of the interventions.

CONCLUSIONS:

Understanding the effectiveness of multicomponent, evidence-based interventions and identifying successful approaches that can be replicated in other settings are essential to increase screening and reduce CRC burden. The use of common framework, data elements, and evaluation methods will allow the performance of comparative assessments of the interventions implemented across CRCCP sites to identify best practices for increasing colorectal screening, particularly among underserved populations, to reduce disparities in CRC incidence and mortality

INTRODUCTION

In 2015, 62% of the United States population ages 50 to 75 was up to date with recommended colorectal cancer (CRC) screening based on responses from selected states¹. Screening uptake was lower, ranging from 25% to 48%, among populations with lower education attainment and income, those without insurance, and several racial/ethnic groups.^2-5 CRC mortality can be reduced through the use of CRC screening tests, and the US Preventive Services Task Force recommends several tests for average-risk adults, including stool-based tests (eg, fecal immunochemical test [FIT], high-sensitivity guaiac fecal occult blood test [FOBT]), and tests that directly observe the colon (eg, colonoscopy, sigmoidoscopy).⁶ For example, individuals aged 50 to 75 years can be screened annually using a high-sensitivity, stool-based test or every 10 years using colonoscopy.

Evidence-based interventions focused on increasing the use of CRC screening tests are necessary to meet the Healthy People 2020 target of 70.5% of adults aged 50 to 75 years receiving recommended CRC screening.⁷ The Community Preventive Service Task Force recently recommended multicomponent interventions to increase screening for CRC⁸. Multicomponent interventions combine 2 or more interventions from strategies that increase community demand (eg, client reminders or small media), increase community access (eg, reducing structural barriers to screening), or increase provider delivery of screening services (eg, provider reminders or provider assessment and feedback). For example, small-media printed materials, such as brochures and newsletters, can be used alongside letters mailed to patients to remind them to complete CRC screening. These small-media materials can inform and motivate individuals to be screened and can provide information tailored to specific target groups to enhance patient reminders. Multicomponent interventions often target multiple levels, including the individual, provider, health system, and community⁸. The literature is limited, and additional high-quality studies are needed to further assess the effectiveness of multicomponent interventions and to identify cost-effective approaches that can be scaled to reach larger populations.

The Centers for Disease Control and Prevention (CDC) funded the Colorectal Cancer Control Program (CRCCP) in 2015 with the objective of increasing CRC screening among adults aged 50 to 75 years with known low CRC screening rates (eg, individuals with low incomes, those who are uninsured). Thirty awardees comprise the CRCCP, including 23 state health departments, 6 universities, and 1 American Indian tribe. To implement the CRCCP, awardees partner with health systems and their clinics or health plans to implement up to 4 evidence-based interventions (EBIs) recommended in the Guide to Community Preventive Services (The Community Guide) (ie, provider and patient reminders, provider assessment and feedback, removing structural barriers) and other supporting strategies (ie, small media, patient navigation).⁹ Patient navigation is a process by which an individual (the navigator) guides the patient through the screening process by providing psychosocial support and by reducing structural barriers, such as lack of transportation to the clinic. There can be differences across navigation programs based on who is serving as the navigator (for example, a trained nurse or a community worker), what types of services are provided, and when in the screening process the navigation services are offered. Awardees also provide health information technology support to help integrate EBIs within electronic medical records (EMRs) and improve the accuracy of screening rate measurement.

In 2016, the CDC established the CRCCP Learning Laboratory (Learning Laboratory) to develop and apply an approach that evaluates the implementation, effectiveness, and cost effectiveness of multicomponent interventions to increase CRC screening used by awardees. Office of Management and Budget (OMB) approval was obtained for the clinical data elements that were collected using common definitions (OMB control no. 9020-1074) and for details on program implementation collected through the grantee annual survey (OMB control no. 0920-1074). The CDC selected CRCCP awardees for participation in the Learning Laboratory based on availability of high-quality data, willingness to collaborate with CDC and members of the Learning Laboratory, and leadership commitment to track program outcomes. The Learning Laboratory currently includes 14 awardees and their implementation partners (eg, health system clinics). There is substantial variation among awardees and their partners in the number, combination, approach, and intensity of interventions implemented (Table 1). This poses a significant challenge to evaluating the programs and highlights the need to assess fully the implementation, effectiveness, and cost effectiveness of the interventions.

TABLE 1.

Overview of Awardees Participating in the Learning Laboratory

Awardee	Partners	Interventions and Process Improvements
California Department of Health	1 FQHC with 3 health centers	Incentives to providers, provider reminder system, reduction of structural barriers
Colorado Department of Public Health and Environment	3 FQHCs	Patient and provider reminder systems, provider assessment and feedback, integrated screening with quality-improvement processes
Great Plains Tribal Chairmen’s Health Board	2 IHS hospitals	Patient reminders, provider assessment and feedback, reduction of structural barriers with community health workers
Louisiana State University	1 FQHC	Patient reminders, implementation of electronic medical record system
Minnesota Department of Health	Multiple FQHCs	Patient navigation for integrated screening (breast, cervical, colorectal cancer screening and tobacco cessation)
Nebraska Department of Health and Human Services	Multiple FQHCs	Patient navigation for integrated screening (includes multiple cancers and cardiovascular diseases)
New York Department of Health	3 Medicaid managed care plans	Patient reminders, patient incentives
Oregon Health Authority and Kaiser Permanente	Collaborative teaming with several programs	Reduction of structural barriers through mailing of FIT kits and tailored instructions
Rhode Island Department of Health	2 FQHCs	Multiple approaches to patient navigation for integrated screening
University of Chicago Medical Center	1 Academic medical system and 1 FQHC	Patient navigation (medical center), provider reminder system, provider assessment and feedback (FQHC)
University of South Carolina	3 FQHCs	Patient reminders, provider reminders, provider assessment and feedback, continuous quality-improvement processes
Washington, DC Department of Health	1 Academic medical system	Patient navigation, patient reminders, provider reminders, provider assessment and feedback; reduction of structural barriers
Washington State Department of Health	1 FQHC with 16 health centers	Mailed FIT, FluFIT, and MammoFIT events where FIT kits were distributed during flu shots and mammograms
West Virginia University	1 FQHC	Patient reminders, provider assessment and feedback

Abbreviations: CRCCP, Colorectal Cancer Control Program; FIT, fecal immunochemical test; FluFIT, flu shot events at which FIT is offered; FQHC, Federally Qualified Health Center; IHS, Indian Health Service; MammoFIT, mammography screenings at which FIT is offered.

In this article, we provide an overview of the methods we are using currently and plan to use in the future to evaluate the differing intervention combinations implemented by the 14 CRCCP awardees and their partners. Specifically, we: 1) describe a conceptual framework of intervention implementation to guide the development of common data elements and identify process, performance, outcome, and cost measures; 2) provide standardized definitions of activities conducted by awardees to develop, implement, and monitor the multicomponent interventions; 3) provide an overview of the evaluation components, including process evaluation, effectiveness assessment and cost-effectiveness modeling, and qualitative data analysis; and 4) discuss strengths and limitations, including data and methodologic challenges, in conducting the evaluations. The methodological guidance provided is applicable not only to CRCCP funded programs but also to other CRC screening programs, because it is based on a comprehensive evaluation that includes all resources and activities required to implement and sustain programs regardless of the sources of the funding or in-kind support.

FRAMEWORK FOR EVALUATING MULTICOMPONENT CRC INTERVENTIONS

The framework for evaluating multicomponent CRC interventions (Fig. 1) was developed to guide the selection of a comprehensive set of data elements so that comparisons can be made across awardee programs implementing varied interventions. We adapted and expanded on preexisting CRC models to create a comprehensive frame-work that can support the evaluation of multicomponent interventions.^10-14 Prior frameworks offer a detailed set of activities to describe and monitor the screening implementation processes. In this report, we update these processes to include metrics and measures that can systematically evaluate interventions, with a specific focus on multicompetent approaches. We designed the framework to describe the steps involved in implementing, monitoring, and evaluating multicomponent interventions. Each of the 7 categories included in the framework is described in detail below.

Figure 1.

This is the conceptual framework for implementing and evaluating multicomponent colorectal cancer (CRC) interventions. FIT indicates fecal immunochemical test; SOP, standard operating procedure.

Readiness Activities

This category identifies activities needed to describe the infrastructure available for implementing and sustaining interventions, including screening data quality assessment, clinic procedure alignment, health system linkages, and monitoring and evaluation processes. Data quality assessment is needed to assess the reliability and validity of data used to determine screening rates and other process and outcome measures. The standard operating procedures and patient flow processes need to be reviewed to determine how new procedures may interact with existing practices so that modifications can be made to avoid unanticipated consequences that could reduce intervention effectiveness. Health system linkages related to care coordination need to be reviewed to ensure that patients who need follow-up diagnostic services are referred to providers and facilities where the procedures can be performed without long delays. Monitoring and evaluation activities need to be in place to assess program progress continuously, so that lessons learned can be translated into actionable steps in a timely manner.

Multilevel Interventions

The framework includes EBIs recommended in The Community Guide to increase community and patient demand for CRC screening, patient access to CRC screening, and provider screening delivery. Details on the types of interventions implemented by awardees and their implementation partners are essential to allow for comparisons. Interventions assessed by the Community Preventive Service Task Force to increase demand for CRC screening include patient reminders and small media. Interventions to increase patient access include removal of structural barriers by offering patient navigation services, flexible clinic hours, multiple options to collect and return FIT kits, and transportation support for clinic visits. Interventions that increase provider delivery of screening services include provider reminders and provider assessment and feedback. These interventions can be new activities that are introduced in clinics or existing interventions that are strengthened and enhanced. The interventions can target any steps in the screening continuum, from being screened, completing recommended diagnostic follow-up testing, or initiating treatment.

Process Measures

Process measures are quantifiable metrics that provide information to describe the implementation process, including underlying or intermediate steps leading to intervention outcomes. Process measures are used to identify implementation failures and/or problems that reduce the likelihood of patients completing the screening cycle. For example, when FITs are used, process measures can track the number of FIT kits handed out to patients; the number returned, sent to, and processed by the laboratory; and the number of test results returned to the clinic and communicated to patients. Improvements can be implemented to improve the quality of screening at all steps in the continuum.

CRC Screening Phases

The phases of CRC screening include patient identification, screening or rescreening, diagnostic follow-up, and treatment. Interventions target 1 phase or multiple phases of this screening continuum. Performance and outcome measures are then used to monitor implementation and describe the impact of interventions on the targeted screening phase. Patient identification is included as a screening phase, because identifying patients who are due or overdue for screening is essential to targeting intervention activities and achieving maximum impact.

Performance Measures

Performance measures typically include established benchmarks or targets and are used to assess short-term or intermediate outcomes. The measures need to provide timely information on aspects of intervention implementation and effectiveness. When performance measure targets are not met, it suggests potential implementation or data quality problems that need addressing. We include measures to capture the number of previously unscreened individuals who are screened, the proportion completing diagnostic procedures within 60 days, and the proportion initiating treatment for cancer within 60 days. We selected 60 days because it has been used previously as a screening quality metric in the CDC CRCCP.^15,16

Outcome Measures

These measures capture both intermediate and long-term outcomes of the interventions. We included clinic-level CRC screening rates, whether polyps are removed, treatment completion, CRC incidence, and CRC mortality. These measures can be used to conduct pre-post evaluations as well as comparisons with similar cohorts of patients who do not receive the interventions. Intermediate outcome measures can be derived using data collected on the screening implementation processes, whereas the long-term outcome measures will require longitudinal follow-up data. It is important to develop the infrastructure to monitor the long-term performance of screening programs, because information related to interval cancers and trends in stage at diagnosis can provide valuable information to improve the program operations.

Cost Measures

Activity-based cost data are collected to support economic evaluations of the interventions and the overall program. These estimates can be used to calculate the incremental cost per individual screened, and these cost estimates can be incorporated into microsimulation models to derive costs per life-years saved and per quality-adjusted life-year.

The framework is intended to be used to describe the program activities (EBIs) and measures needed for monitoring implementation and evaluating outcomes to increase CRC screening using a standardized approach. We have not included the activities performed by decision makers and the importance of CRC screening policies to support the implementation of EBIs.

DEFINING CRC PROGRAM ACTIVITIES

To compare interventions across awardees, we developed a list of activities required to develop, implement, and monitor the multicomponent interventions. We used a consensus-building process that included Learning Laboratory awardees as well as program and clinic staff participating in the project to define the included activities. Table 2 presents the list of activities, which are separated into 5 categories: 1) intervention development phase; 2) intervention implementation phase; 3) administration and management; 4) evaluation research and reporting; and 5) data quality assessment. The intervention development and implementation phase activities are those directly related to the start-up and execution of the program. The start-up activities are generally conducted during a specified timeframe to assess needs, select the interventions, formalize the intervention processes, and ensure that support systems are in place. These activities will not have to be repeated in the intervention clinics unless modifications are required to the interventions or the implementation procedures. The intervention implementation phase activities are conducted on a continuous basis to implement the interventions. The last 3 categories related to administration, evaluation, and data quality are overarching activities that may be applicable to 1 or both phases; that is, these activities support both the intervention development and implementation phases of the program.

TABLE 2.

Activities for Implementing Interventions to Increase Colorectal Cancer Screening

Intervention Development Phase	Intervention Implementation Phase
Select evidence-based interventions (including conducting internal assessment of processes)	Recruit patients/initial contact with patients
Identify eligible patients (eg, through database analysis)	Follow-up with patients (by mail, telephone or in person)
Identify physicians or other providers and sign contracts/MOAs/MOUs	Monitor and track patients and provider processes (related to the screening test)
Develop forms or databases for tracking progress or outcomes (patient-related or provider-related)	Process incentives
Develop CRC invitation and contact materials	Process FIT or FOBT kits (only applicable to laboratory procedures for fecal test kits)
Translate materials	Refer for diagnostic colonoscopy (only applicable to fecal-based testing)
Obtain internal or external approvals	Refer for treatment and follow-up
Modify or install new IT/EMR systems
Train implementation staff
Develop referral process and partnerships for diagnostic colonoscopy and cancer treatment
Create process for ordering/mailing FOBT or FIT kits

Administration and Management	Evaluation, Research, and Reporting	Data Quality Assessment
Develop partnerships and establish formal contracting agreements or MOAs	Develop evaluation plans	Identify clinics or sites
Hire staff	Collect and report baseline and annual clinic data to CDC	Conduct initial quality assessment and feedback
Provide implementation support by regular communication (eg, meetings, telephone calls)	Collect and report CRCCP survey and other information to CDC (nonfinancial data)	Perform patient chart reviews
Report fiscal and budget-related information	Collect data for awardee-specific evaluation and special studies (eg, CEA)	Provide training, quality-improvement education, and technical support
Develop implementation plans	Analyze and report results	Monitor data quality

Abbreviations: CDC, Centers for Disease Control and Prevention; CEA, cost-effectiveness assessment; CRC, colorectal cancer; CRCCP, Colorectal Cancer Control Program; EMR, electronic medical record; FIT, fecal immunochemical test; FOBT, fecal occult blood test; IT, information technology; MOA, memorandum of agreement; MOU, memorandum of understanding.

COMPONENTS OF CRC PROGRAM EVALUATION

The framework provided an overall description of the activities and measures needed to implement, monitor, and evaluate the program. Here, we describe 3 components of the evaluation, which include process evaluation, effectiveness assessment and cost-effectiveness modeling, and qualitative data analysis.

Process Evaluation

Table 3 includes common data elements that were developed based on the framework for implementing and evaluating the multicomponent interventions presented in Fig. 1 and intended to support consistent process evaluation across participating health systems and clinics. The data include variables necessary to quantify pre-established metrics as well as characteristics of individuals, providers, health systems/clinics, awardees, and community partners. At the individual level, several factors (including demographics, education level, and insurance coverage) can impact screening uptake and compliance along the continuum of care. Lack of insurance or high copayments can serve as economic barriers to increasing CRC screening rates and should be considered as mitigating factors when assessing the impact of interventions. We define providers as those involved in the implementation process, from the front-end clinic receptionist to the clinical assistant, nurse coordinator, and clinician. Our definition of facilities includes clinics, health systems, hospitals, and academic medical centers. Programs refer to awardees who receive funding from the CDC and/or other organizations to facilitate the implementation processes by performing needs assessment and data quality review, providing information on best practices, offering technical support, and evaluating the interventions. Integrated delivery systems and health plans can contain features of both facilities and programs. Community groups include organizations like the American Cancer Society, state primary care associations, and statewide cancer coalitions that promote and support CRC screening. Information about these groups is important, because these groups often provide implementation support to facilities where interventions are implemented.

TABLE 3.

Common Data Elements for Each Group of Stakeholders

Stakeholders	Data Elements
Individuala	Age; sex; race; education level; Hispanic ethnicity; primary language; CRC risk (personal and family history; no. of first-degree relatives diagnosed with CRC and age at diagnosis); insurance coverage, including plan type (Medicaid, Medicare, private, uninsured, other) and copayment requirements; CRC screening status (screening date, type of test, test results, date of diagnostic colonoscopy [when required], treatment initiation [when required], and rescreening recommendations)
Providera	No. of staff by type (eg, receptionists, nurses), role of staff (eg, recruit patient for screening; track screening results),b medical specialty of clinicians, CRC trainings received, clinician screening performance
Health system/clinica	Screening test offered, diagnostic colonoscopy availability, EMR system, patient identification process (manual, electronic, mixed), adoption of guidelines at point of care, integrating CRC screening within quality-improvement processes, patient invitations for screening (eg, in-person discussion, mailings), mode of communicating test results to patient (eg, in-person, mail), funding/resources for CRC intervention implementing, activity-based cost
Awardee	No. of staff; qualifications of staff activities conducted to support clinics and health systems (activity list)b; funding/resources, both labor and nonlabor, for CRC intervention support; activity-based cost
Community partners	No. of partners, key activity of partners (activity list)b

Abbreviations: CRC, colorectal cancer; EMR, electronic medical record.

^aUS Office of Management and Budget (OMB) approval was obtained for clinical data elements used to derive the screening uptake measures (OMB control no. 9020-1074) and other details collected through the grantee annual survey (OMB control no. 0920-1074).

^bFor details on the common set of activities, see Table 1 and Supporting Table 1.

To assess screening rates changes because of the interventions, a common definition (numerator/denominator) for measuring the screening rate was consistently used to facilitate comparative assessments. The CRCCP has developed guidance for assisting programs to report consistent screening rates, including a standard definition for the numerator (the number of adults who had received appropriate CRC screening), the denominator (the number of adults eligible for CRC screening), and the timeframe for measurement, because these definitions may vary by organization or quality measure.¹⁷ The denominator is defined as the number of patients aged 51 to 75 years who had a medical visit during the measurement year (excludes individuals with CRC and those who underwent total colectomy). The numerator is the number of patients aged 51 to 75 years who had 1 or more appropriate screenings for CRC and had at least 1 medical visit during the measurement year. Appropriate screening tests and intervals include the following: high-sensitivity FOBT or FIT during the measurement year, sigmoidoscopy during the measurement year or in the 4 years before, and colonoscopy during the measurement year or in the 9 years before. The CRCCP screening rate definition was used to measure those up to date with CRC screening in the preintervention period as well as during the intervention implementation timeframe.

To assess adoption and implementation of the interventions, we use process mapping to document the details of the intervention at each level of implementation. Process mapping involves documenting each step and interaction in the intervention implementation process. If multiple interventions are implemented, then each must be mapped. Process mapping includes describing the individuals who deliver or receive the intervention (eg, patients, providers), the frequency of intervention delivery (eg, monthly provider assessment and feedback reports), the duration of the intervention (eg, the average time navigators spend with patients), and the format and context of the intervention. For example, for a patient navigation intervention, we identify who provides the navigation (number of staff, qualifications), how the navigator interacts with the patient (eg, telephone), how many times they interact with the patient, the content of the interaction, barrier-reduction efforts, the use of technology to provide automated reminders or education, and timing of the interactions (details such as time period covered; before, during, and/or after the procedure). This level of detail will allow us to compare the dose or intensity of the interventions across implementation sites. The same type of intervention (eg, patient navigation) can be implemented differently across sites with differing outcomes. It is also important to understand whether the interventions can be maintained and sustained beyond the study period. We will use information collected at the facility level to review the extent to which CRC interventions are incorporated and embedded into the health care organization’s data systems, staff activities, and patient flow processes to evaluate sustainability.

Finally, a detailed cost analysis is performed to understand the resources required for planning implementation of the interventions. Activity-based cost data collected from the programs and facilities will be used to evaluate the appropriate allocation of funds to identify best practices for staff and structure interventions to maximize efficiency.

Effectiveness Assessment and Cost-Effectiveness Modeling

To evaluate the effectiveness and impact of the interventions, we have used a combination of process, performance, and outcome measures. For example, for an intervention using mailed FIT kits (reduction of structural barriers by eliminating a visit to the clinic), we calculated the proportion of FIT kits returned and estimated the cost per kit returned to assess the sustainability of the intervention.¹⁸ To evaluate another intervention on patient navigation, we determined the proportion of individuals who successfully received patient navigation and used a historic cohort to quantify its effectiveness.¹⁹ Although, in several instances, we identified adequate control cohorts to perform comparative assessments, including the use of randomization, for some of the evaluations, the study design only allowed for pre-post assessments.^20,21 In these instances, we report not only screening rates but also process measures to ensure that we capture the underlying activities and procedures that were affected by the interventions so lessons learned can be shared. In addition, we capture the cost of all interventions to identify the resources required for different combinations of interventions and also to generate incremental cost-effectiveness ratios.

For the Learning Laboratory, we will use a validated CRC simulation model to perform assessments of long-term effectiveness and cost of successful CRC screening interventions.^22-24 This analysis will allow us to generate quality-adjusted life-years saved and costs per life-year saved, which will allow us to compare CRC interventions with other types of prevention interventions to highlight the implementation of CRC interventions based on the relative efficiency in use of resources. In addition, this analysis enables us to develop the business case for health systems and other organizations that CRC screening may be a beneficial investment. For example, we are collaborating with multiple partners to build the evidence base to demonstrate that patient navigation decreases colonoscopy no-show rates and leads to the optimal use of endoscopy suites, which then may result in additional revenue.

Qualitative Data Analysis

To evaluate complex multicomponent interventions fully, the Learning Laboratory supports the collection of qualitative information to supplement quantitative data and further understand contextual factors, processes, barriers, and facilitators of partnering with health systems and implementing the interventions. For example, key informant interviews allow us to capture rich descriptions about implementation site selection, decision processes related to the choice of intervention strategies, and the identification of barriers and facilitators to those processes. The approach of integrating quantitative and qualitative data is important for implementation science research because it can more fully capture the dynamic and complex interactions that shape intervention outcomes at multiple levels of analyses and over time.²⁵

STRENGTHS AND LIMITATIONS

The Learning Laboratory works with programs that are implementing interventions to increase CRC screening in the real-world setting. The approach described in this report allows for an evaluation of the multicomponent interventions used by awardees so that effectiveness and cost can be compared across different combinations of interventions. This offers valuable information on the scalability of the interventions by providing not only evidence on cost effectiveness but also details on the budget impact to guide informed decision making.

Working with awardees in a real-world setting presents multiple challenges in terms of data collection. First, clinics and health systems may have difficulty systematically identifying individuals who are eligible for screening and tracking the provision and result of services provided by specialists or other entities outside the health system. A systematic evaluation of the available data and specific process changes to better identify patients and capture relevant data can lead to substantial improvements in the quality of the data. Second, EMR systems may not facilitate easy access to the data required to identify and invite to screening patients who are due. Enhancements to the EMR system may be required to allow facility staff to use actively the data captured and stored. Third, partner organizations that change or upgrade their EMR systems can experience multiple challenges in providing the information required to evaluate ongoing interventions. EMR changes disrupt data collection and often lead to delays in retrieving patient data; and, in some cases, historic data may no longer be available. These challenges may lead to underestimating or overestimating the number of patients due or eligible for screening, which, in turn, can produce inaccurate screening rates.

In addition, there are multiple challenges related to study design, especially in identifying optimal comparison groups. Study clinics are often unique in terms of the populations they serve, which reflects the demographic mix of their catchment area. One option that is being considered by the Learning Laboratory partners is to implement the same set of interventions, using consistent approaches across multiple clinics to test whether the expected changes are observed across all settings. Differences in CRC screening uptake and outcome measures will then be explored using process and performance measures, and statistical methods will be used to control for differences in populations across clinic sites. Furthermore, even pre-post evaluations can provide valuable lessons on the implementation processes and the effectiveness of interventions, especially when process measures can be used to track the pathway in which the interventions lead to improvements in screening outcomes.

CONCLUSION

Understanding the effectiveness of multicomponent EBIs and identifying successful approaches that can be replicated in other settings are essential to increase screening and reduce CRC burden. The CRC Learning Laboratory is using multiple methods, including performance measurement, process and outcome assessment, longitudinal simulation of outcomes and cost, and qualitative case studies, to evaluate systematically the multicomponent interventions implemented by 14 CRCCP awardees. We already have used a subset of the proposed approach in our studies and will initiate a full set of standardized data collection in future studies, which we are planning. Using common frameworks, data elements, and evaluation methods will allow us to perform comparative assessments of the interventions implemented across CRCCP sites to identify best practices for increasing CRC screening, particularly among underserved populations, to reduce disparities in CRC incidence and mortality.