Strategies and Opportunities to STOP Colon Cancer in Priority Populations: Design of a Cluster-Randomized Pragmatic Trial

Abstract

Background

Colorectal cancer is the second-leading cause of cancer deaths in the United States. The Strategies and Opportunities to Stop Colorectal Cancer (STOP CRC) in Priority Populations study is a pragmatic trial and a collaboration between two research institutions and a network of more than 200 safety net clinics. The study will assess effectiveness of a systems-based intervention designed to improve rates of colorectal-cancer screening using fecal immunochemical testing (FIT) in federally qualified health centers in Oregon and Northern California.

Material and Methods

STOP CRC is a cluster-randomized comparative-effectiveness pragmatic trial enrolling 26 clinics. Clinics will be randomized to one of two arms. Clinics in the intervention arm (1) will use an automated, data-driven, electronic health record-embedded program to identify patients due for colorectal screening and mail FIT kits (with pictographic instructions) to them; (2) will conduct an improvement process (e.g. Plan-Do-Study-Act) to enhance the adoption, reach, and effectiveness of the program. Clinics in the control arm will provide opportunistic colorectal-cancer screening to patients at clinic visits. The primary outcomes are: proportion of age– and screening-eligible patients completing a FIT within 12 months; and cost, cost-effectiveness, and return on investment of the intervention.

Conclusions

This large-scale pragmatic trial will leverage electronic health record information and existing clinic staff to enroll a broad range of patients, including many with historically low colorectal-cancer screening rates. If successful, the program will provide a model for a cost-effective and scalable method to raise colorectal-cancer screening rates.

Introduction

Despite the potential of colorectal cancer (CRC) screening to reduce CRC mortality, CRC remains the second-leading cause of cancer deaths [1]. In 2014, an estimated 137,000 adults in the U.S. will be diagnosed with CRC, and 50,000 will die from the disease [2]. Identification and removal of pre-cancerous polyps can reduce the rate of invasive disease [3].

Despite the clear benefits of screening, data from the National Health Interview Survey (NHIS) show that, in 2010, 41% of adults aged 50–75—nearly 35 million people—were not up-to-date on CRC screening [4]. Nearly 30% of eligible adults have never had any type of CRC screening [5]. These rates are well below goals set by the American Cancer Society (75% by 2015) [1] and by Healthy People 2020 (70.5%) [6]. NHIS data from 2000–2010 consistently show lower rates of screening among adults who are typically served by federally qualified health centers (FQHCs); that is, those with minimal education, low income, or no health insurance. Rates were also disproportionately low among recent immigrants, those with no usual source of care or physician visit in the past year, and Hispanics [4]. Low utilization of screening leads to delayed detection of CRC, diagnosis at more advanced stages, and higher CRC-related morbidity and mortality [1]. Accelerating adoption of screening could reduce CRC mortality more than 50% by 2020 [7].

Colonoscopy allows for removal of polyps at the time of screening and is considered the gold standard for screening by many professional organizations [8]. However, fecal immunochemical testing (FIT) is a low-cost screening method that is easily scalable, easy to do, and preferred by multiple patient populations [9]. Inadomi recently demonstrated that in patients offered either FOBT or a choice between FOBT and colonoscopy were more than twice as likely to complete CRC screening. Gupta and colleagues conducted a study involving a safety net health system and compared colorectal screening rates among 5,970 patients who were offered one of three testing options: (1) free FIT; (2) free colonoscopy; or (3) usual care, which was opportunistic screening. Findings from his study showed that over 40% of those offered free FIT were screened; this compared to 25% and 12% of those offered free colonoscopy and usual care, respectively [10]. Both studies, however, report rates of fecal testing over a single year, though annual testing over 10 years is needed to confer the same adherence as a single colonoscopy.

Previous evaluations of clinic-based programs to improve rates of CRC screening have shown that direct mailing of fecal occult blood tests (gFOBT) or fecal immunochemical tests (FIT) consistently led to 6–30% increases in CRC screening, regardless of clinical setting [11–15;15;16]. Some studies have shown that use of health educators and screening information tailored to specific cultural and language needs can be effective in raising CRC screening rates [11;13;17–19]. While some of these studies showed promising results, none have resulted in widespread adoption of CRC screening practices because the screening system relied on stand-alone tracking or was not integrated into routine care. The presence of practice-level systems to support the translation of physician recommendation into care delivery is an important influence on CRC screening uptake [12;20]. None of the previous interventions embedded registry functions directly into the electronic health record (EHR), and into existing clinical staff workflows, diminishing the opportunity for sustaining these interventions over time.

This paper describes the design of the Strategies and Opportunities to STOP Colon Cancer in Priority Populations (STOP CRC), a pragmatic study that seeks to automate and embed, using real-time EHR data, systems to identify patients who need CRC screening. We will also track CRC-related outcomes using routine processes of care at FQHCs. STOP CRC consists of a pilot study and a larger multi-site pragmatic study that began in 2014 and is testing a scalable option for promoting CRC screening in populations least likely to be screened.

Materials and Methods

STOP CRC is a large, multi-site, cluster-randomized pragmatic study that will test the effectiveness of automated strategies to raise CRC screening rates in safety-net clinics. This demonstration project was funded by the National Institutes of Health (NIH), Health Care Systems Research Collaboratory program, whose aim is to provide a framework of implementation methods and best practices that will enable the participation of many and varied health-care systems in clinical research [21]. The study was approved by the Institutional Review Board of Kaiser Permanente Northwest (Protocol # 4364), with ceding agreements from Group Health Research Institute, and OCHIN. OCHIN is a non-profit health information technology (IT) organization that provides EHR systems and support to FQHCs and small practices in several states. The OCHIN health IT organization and the OCHIN Practice Based Research Network collaborate to help FQHC clinics improve population health, patient care, and care efficiency. At the onset of our study, the OCHIN PBRN was affiliated with over 50 FQHCs and safety net health centers with more than 200 individual clinics, all using a single OCHIN-supported EHR system, Epic© (version 2010; Verona, WI). Due to the minimal risk of the intervention, the requirement for informed consent was waived. The trial is registered at ClincalTrials.gov (NCT01742065).

STOP CRC is based on two prior studies conducted by our study team that tested direct-mail CRC screening programs in two different clinical settings. The first was a pilot study conducted in 2007–2009 with an FQHC in western Washington. This study tested the program among 500 low-income Latinos who receive their care in safety-net clinics, but the methods relied on manual medical-chart review to identify patients and track screening outcomes [11]. A second study consisted of a randomized controlled trial conducted in a Health Maintenance Organization (Group Health Cooperative) that used an EHR-linked system for patient identification and tracking, but the tracking tools were managed by a research team (not embedded into the clinic workflows) [12]. Both studies and the researchers who conducted them helped guide the design of STOP CRC.

STOP CRC has two phases: The first, Phase 1, was a pilot phase [22]. During the pilot phase we developed our EHR tools and tested two interventions in a two FQHC clinics belonging to a single health organization. Phase 2 is a larger two-arm cluster-randomized study involving 26 FQHC clinics and 8 health organizations. Phase 1 pilot findings showed an overall 37 percentage point increase in CRC screening in intervention, compared to Usual Care (UC), clinics (38% vs. 1% over a 6 month period, based on intention-to-treat analyses) [22]. Here, we describe the Phase 2 study design and protocol.

Recruitment

To aid with issues regarding intervention adaptation and cultural relevance, we convened an Advisory Board comprised of project investigators, clinic staff, and community members. Our Advisory Board for Phase 1 helped establish the criteria for clinic eligibility. Using a list of clinics provided by OCHIN, we selected those that had at least 2 clinic sites with a minimum of 450 patients aged 50–74. We assessed other criteria through in-person meetings with clinic leadership including: 1) willingness to use a single type of fecal test in their intervention and UC clinics (given the pragmatic nature of the study, we did not specify the type of fecal test a health center used (most will use a single-sample or two-sample FIT that does not require dietary restrictions); 2) having sufficient capacity to obtain colonoscopies for patients who screen positive on FIT/FOBT; 3) having an electronic-results interface with the lab processing FIT or FOBTs; 4) an available lab that has sufficient processing capacity; 4) willingness to randomize eligible clinics, with the caveat that at least one will be intervention and one usual care; 5) having a plan for FIT or FOBT testing among uninsured patients; 6) willingness to fulfill research requirements (clinic interviews, data validation, regular advisory board meetings, interpret project findings); and 7) willingness to prioritize STOP CRC by reviewing baseline CRC screening rates and setting improvement targets. For administrative purposes, clinics also needed to agree to cede IRB to Kaiser Permanente Northwest and to maintain an active Federal-wide assurance. Beginning with a list of 51 health centers provided by OCHIN, we excluded health centers that had a single clinic (n = 14), did not meet the size requirements (n = 15), or were outside the geographic catchment area (n = 10, which included Washington, Oregon and Northern California). The STOP CRC study staff successfully recruited 8 health centers (of 12 that were invited) consisting of 26 clinics (estimated number of active patients aged 50–74 = 30,000).

For all participating clinics, both intervention and control, OCHIN will create a clinic-based registry of patients who are eligible for colorectal cancer screening, i.e., patients aged 50–74 who are due for screening. An initial list will be created at the outset of the study and updated on an ongoing basis thereafter. The eligibility criteria include those without EHR evidence of being up-to-date with CRC screening recommendations (FOBT within 1 year, flexible sigmoidoscopy within 4 years, or colonoscopy within 9 years) or of having a limited set of health conditions (e.g. prior CRC, inflammatory bowel disease, renal failure). To exclude patients already in the process of undergoing CRC screening, patients will be excluded if they have an un-resulted order for a FOBT/FIT in the past 6 months or have a referral to colonoscopy in the past year. A patient with an FOBT/FIT order that remained un-resulted for more than 6 months or with an open colonoscopy referral for more than 1 year would be considered eligible. OCHIN staff will apply automated codes from the EHR to create an initial registry of patients overdue for CRC screening. In addition, the list of eligible patients will be run against Health Maintenance, an Epic-embedded tool that tracks outside screening events, including CRC screening. All identified participants will be included in the final data analysis.

Randomization and Blinding

Randomization will be stratified by clinic organization, and within each organization assignments will be blocked to assure a balance of treatment assignments within each organization and equal numbers of intervention and control clinics overall. Within each clinic organization, clinics will be randomly ordered and then assigned to a predefined sequence of randomization assignments. This approach will fulfill our commitment to the clinic directors that each organization will have both intervention and control clinics. We considered use of constrained randomization techniques to better assure such balance [23], however, unpublished simulation models suggested that, for our relatively limited number of clusters, this approach might underperform relative to simple randomization techniques. The randomization process was done one time at the outset of the study, and clinics were made aware of their intervention assignment; if clinics were blinded, the study protocol would be impossible. Similarly, individual participants are, necessarily, un-blinded to intervention assignment. However, neither intervention nor control participants will be aware that they are participating in a research study; the intervention will be delivered as a standard clinic outreach program and outcome data will be passively gathered via the EHR. The randomization sequence that defined 2 groups of clinics will be generated by the project statistician and analyst; a member of the project’s Advisory Board (not a clinic representative) will “roll the dice” (using electronic dice) to determine which group is assigned to the STOP CRC intervention. For transparency, the real-time dice roll and outcome of the clinic-assignment will be broadcast over Webex for viewing by clinic representatives and advisory committee to support participatory based research principles.

Treatment arms

Participating clinics are randomized, in a one-to-one ratio, to either a Usual Care control arm or an automated, data-driven, EHR-embedded program (Auto Intervention and Practice Improvement Cycle) for mailing guaiac fecal occult blood test (gFOBT)/FIT kits to patients due for CRC screening. The practice improvement cycle aims to enhance the program’s implementation and reach. These arms are described in more detail below.

Usual care (UC) arm

UC clinics will continue whatever they are already doing to promote CRC screening. This typically consists of clinic staff providing patients with information on the importance of CRC screening, and ordering CRC screening tests on an opportunistic basis during routine clinic encounters. In recent years, federal reporting requirements for preventive services utilization, including CRC screening, have elevated the prioritization of CRC screening. Thus, usual care can vary slightly by clinic organization. The usual care clinics will not have access to the automated tools.

Auto Intervention and Practice Improvement Cycle arm

The auto intervention will be overlaid on whatever UC process exists at each intervention clinic. Patients who are CRC-screening eligible will be sent an introductory letter explaining the program and providing a clinic number to call if they were recently screened or wish to opt out of the screening (e.g. they were recently screened, have a terminal illness, or prefer not to do fecal testing, etc.). Those not opting out will be mailed a FIT kit (with pictographic instructions [24] and return postage). Patients who fail to return a completed FIT kit within one month will be sent a language-appropriate reminder letter.

We will also engage intervention clinics in an improvement cycle, also called a Plan-Do-Study-Act (PDSA) cycle [25]. This is a standard quality-improvement process for introducing a new program into primary care. Because of the heterogeneity of our sites, before conducting a PDSA cycle, we will conduct an assessment of the health organizations’ quality improvement and PDSA experience. A project consultant will then facilitate a PDSA at each site. She will work with clinic staff to Plan, that is, identify the question(s) to be asked and determine any data that needs to be collected and by whom, Do; that is, carry out the change or activity and collect the data, Study the data you collected; and Act by identifying next steps. Using this process, we aim to optimize the sustainability of our program as a standard part of clinical care. An improvement process, for example, may identify the need for a workflow that can improve efficiency (e.g., calling patients with invalid addresses) or the need for training (e.g., best practices for recording historical colonoscopies). The improvement process can also identify additional intervention components to improve effectiveness or reach (e.g., clinic posters that show how to do the test). The PDSA cycle will be conducted at each intervention site within 4 – 6 months following program launch. In year 2, we will continue to support clinics in maintaining the system in the intervention sites, by offering additional training (based on the PDSA cycle and ongoing reports).

Study outcomes

Primary outcome

The primary outcome is the clinic-level proportion of patients eligible for CRC screening who complete a gFOBT or FIT within 12 months of being identified by OCHIN staff as due for screening. Only patients initially identified during the first year of screening will be included in the primary outcome analysis. For analysis purposes, participants who are eligible for CRC screening will be identified using identical process for intervention and control clinics.

Secondary outcomes

Our secondary outcomes are: 1) The proportion of year-1-eligible patients who complete a gFOBT or FIT within 3, 6, and 9 months; 2) Healthcare Effectiveness Data and Information Set (HEDIS) scores for the calendar year of the intervention roll-out; 3) proportion of patients with a documented colonoscopy during the initial 12-month intervention period; 4) proportion of patients who screen positive on gFOBT or FIT; 5) proportion of patients who screen positive on gFOBT or FIT who are referred to gastroenterology; and 6) proportion of patients who screen positive on gFOBT or FIT who obtain a follow-up colonoscopy. Similar outcomes will be assessed for the year following the initial rollout. In year 2, we will assess changes in FIT testing rates (and changes in our secondary outcomes, including overall CRC screening rates) in both intervention and control sites, adjusted for Year 1 rates.

Intervention cost

We will measure the incremental cost of implementing the intervention in Phase 2 and use these data to analyze cost-effectiveness and return on investment of the intervention compared to usual care. Our Phase 2 analysis will be informed by an earlier assessment of the cost of the pilot intervention. Although this pilot assessment included development costs related to intervention design or to clinic EMR enhancements, we will not include such costs in Phase 2. Our analytic perspective will be that of a clinic or system considering implementation of an extant program with a compatible EMR.

Intervention costs will be calculated using micro-costing techniques; research-related costs will be excluded. We will categorize Phase 2 implementation costs as labor (e.g., medical director, clinical champion, quality-improvement lead, EMR site specialist, data analyst, medical assistant, and receptionist) or non-labor (e.g., printing, mailing, cost of FIT kits, cost of processing FIT tests for uninsured participants). We will further distinguish project labor as either 1) engagement and preparation or 2) launch and maintenance. Cost data will be collected from project reports and databases as well as from retrospective labor estimates. Unit cost multipliers (for example, MA wage rate plus fringe) will be applied to resource quantities, most of which will be tracked in a project database. As appropriate, univariate and multivariate sensitivity analyses will be conducted to test the robustness of our baseline results.

Process measures

Finally, consistent with the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) framework [26], we will also assess the adoption, implementation, and reach of the intervention, as well as assess to what degree the program is maintained by intervention clinics, and initiated by control clinics, two years following randomization. We will assess these factors using a mixed-method quantitative and qualitative rapid assessment process [27]. Our quantitative measure of implementation will rely on delivery of our core components: FIT kits mailed and PDSA cycle conducted, and we will assess implementation of non-core components: Reach will be estimated as N who received the intervention (N eligible − N with an invalid address)/N anticipated. At the beginning of implementation and 1 year following implementation, we will gather qualitative data (from in-person interviews with health center leadership) and administer on-line surveys of providers to assess the implementation and adoption of the program; we will also assess clinic-level barriers to ongoing maintenance and patient-level factors that influence program effectiveness (using both EHR-data and surveys with health center leadership).

Power

In developing our calculations, we assumed equal numbers of subjects per clinic and equal numbers of clinics (n=13) per group. In practice, the clinic sizes will not be equal, but since almost all clinics have at least 450 active age-eligible patients, we conservatively use this figure for all sites. We based our calculations on the simple paradigm of comparing two binomial proportions with a type I error rate of 5%, and adjusted both for intraclass correlation (ICC) and the reduced degrees-of-freedom (n=24) for the critical values. Based on analyses by Dr. Green using the data from her Systems Of Support study [12;28], we expect the ICC to be about .03. Using this figure, we will have very good power (>91%) to detect absolute differences as small as 10 percentage points even if the FIT completion rate in the UC arm is as high as 15% (fecal testing rates for 2013 for usual care clinics was 10%). For an ICC of .05 we would still have >91% power for detecting effect sizes of at least 13 percentage points.

Statistical analysis

Primary outcome analyses

The primary analysis will compare the proportion of year 1 eligible adults in intervention versus usual care clinics who complete a FIT within 12 months of being identified as due for screening. Secondary analyses of this outcome will assess differences in screening outcomes in select patient subgroups (e.g., by age, gender, Hispanic ethnicity/race, and insurance status), as well as outcomes during year 2 of follow-up.

The data may be viewed as coming from a 2-level hierarchical model with patients clustered within clinics. Because primary interest is on the difference in gFOBT or FIT completion rates between intervention and UC clinics, we will initially aggregate each clinic’s data into 8 separate screening rates (one each for subgroup defined by age (50–64 vs. 65+), gender, and race (minority vs. non-minority). The resulting analytic dataset will thus consist of 208 observations (26 clinics X 8 observations per clinic). Treating the resulting observations as approximately normally distributed, we will then use mixed-model ANCOVA to estimate screening probabilities as a function of intervention, age, gender, race, and baseline clinic screening rate, with clinic specified as a random cluster variable. To ensure that the clinic-specific proportions are estimated accurately, we will weight each clinic’s subgroup-specific means to reflect the corresponding frequency of that cell for that clinic.

For the secondary analyses we will include, in turn, fixed-effect interactions of treatment with age, gender, and race to test the impact of the intervention in subgroups defined by these variables. Following Murray (1998), for these latter analyses we also will include interactions between the covariates and clinic which will be treated as random effects [29].

Although the primary analysis focuses on screening results during the first 12 months post-intervention implementation, secondary analyses will assess screening outcomes during year 2 as well as the proportion of individuals who meet screening criteria during both the first and second year post-intervention implementation.

Secondary outcome analyses

The analyses of screening outcomes at 3, 6, and 9 months, as well as the analysis of HEDIS scores and colonoscopy completion probabilities, will all follow the same analytic model as the primary outcome analysis. The only difference is that the HEDIS score for each clinic is calculated based on the entire age-eligible population, and not just on those who were flagged for screening. We will use HEDIS scores as reported to NCQA, which are calculated on a calendar year basis, and hence will not correspond exactly to the timing of the intervention. We will use the HEDIS score for the calendar year prior to intervention rollout as an adjustment covariate in the analysis. For the above reasons, we expect a smaller effect size for this outcome than for our primary outcome, but we include it because this outcome is particularly important for informing policies relevant to CRC screening.

Analyses of the remaining secondary outcomes above will be descriptive in nature. The first of these is the proportion of completed and mailed-back gFOBT or FIT kits that are positive. While we anticipate a sizeable number of mailed-back gFOBT or FIT kits for each of the intervention clinics (typically in excess of 150), the numbers will be much smaller for control clinics (for many of the smaller clinics, this number will be less than 20) and hence much less precisely measured. The situation will be compounded for the remaining secondary outcomes, which are further limited to gFOBT or FIT kits that are mailed-back and are positive. Thus, for all of these outcomes the focus will be more on the experience of the intervention clinics than on formal comparisons between intervention and control clinics, though we will look at this latter question to the extent possible. Ultimately, we wish to know the extent to which the clinics are able to keep up with the extra screening and the impact it may have on their operations, and hope that the planned analyses will help to shed light on this issue.

Economic evaluation

We will use our intervention cost assessment to inform an economic evaluation of the STOP program relative to UC. Specifically, we will conduct both a cost-effectiveness analysis and a return-on-investment (ROI) analysis. We will report two primary metrics: 1) incremental program costs per incremental gFOBT or FIT completed (cost-effectiveness); and 2) program costs less expected costs saved by increased CRC screening (ROI). We will use published and other sources to generate estimates of averted CRC cases and premature mortalities based on increased screening rates, estimate medical care and productivity costs averted per case, and estimate premature death prevented. In the ROI calculation, we will use the estimated difference in total cost between study arms as the expected cost savings to offset against program costs. Costs (and benefits) will be inflation-adjusted and discounted at a 2% base rate and adjusted in univariate and multiariate sensitivity analyses.

Analysis of process data

We will monitor and report on the extent to which the intervention is delivered as intended. This will consist of simple descriptive statistics calculated for each intervention site and overall. Consistent with the broader RE-AIM framework, we will capture the following data for our intervention clinics: (1) Adoption: N clinics that participate/N anticipated [characteristics of adopters and non-adopters]; (2) Reach: N participants who receive intervention components/N anticipated. We will record N invalid address, N declines, and N who report prior screening; (3) Implementation: N activities for each intervention component delivered by clinics/N anticipated (e.g. N reminder fecal tests mailed/N anticipated); (4) Maintenance: N clinics that implement STOP CRC in YR 02/N implemented in YR 01. For those clinics who do adopt/maintain the intervention in year 2, we will describe similar intervention outcomes for year 2 (e.g., percent with FIT/FOBT returned within 12 months) and the proportion of eligible patients current for FOBT and any CRC screening for both years of the study.

Conclusions

The STOP CRC study has great potential to test a scalable approach to reduce disparities in stage of detection of CRC through improving health systems’ ability to encourage uptake of colorectal cancer screening. This intervention has been designed on the foundation of other work demonstrating the effectiveness of promotion techniques, while addressing the failure of other systems to achieve the scale necessary to reach large populations. This study is explicitly addressing the issue of scale by testing the intervention in multiple settings and by using a commonly employed information system and promotion techniques that incorporate the necessary personnel time and activities into routine clinic processes. The findings of this study will demonstrate the effectiveness, reach, and maintenance of the program using a pragmatic design that enrolls a broad range of patients. Our program also will document the proportion of patients who complete a second FIT in Year 2 of the program (maintenance). The evaluation of cost-effectiveness and return-on-investment of the intervention will aid in design of future appropriate interventions where financial constraints may be a concern. If successful, the program may represent a cost-effective method of raising levels of participation in CRC screening and of down-staging the detection of CRC among patients least likely to be screened. It could be rapidly rolled out to improve care for 2 million patients at multiple clinics within the OCHIN network, and more broadly among health care systems that use EHRs.