A Cost-Effectiveness Analysis of a Colorectal Cancer Screening Program in Safety Net Clinics

Abstract

STOP CRC is a cluster-randomized pragmatic study of a colorectal cancer (CRC) screening program within eight federally-qualified health centers (FQHCs) in Oregon and California promoting fecal immunochemical testing (FIT) with appropriate colonoscopy follow-up. Results are presented of a cost-effectiveness analysis of STOP CRC. Organization staff completed activity-based costing spreadsheets, assigning labor hours by intervention activity and job-specific wage rates. Non-labor costs were from study data. Data were collected over February 2014-February 2016; analyses were performed in 2016–2017. Incremental cost-effectiveness ratios (ICERs) using completed FITs adjusted for number of screening-eligible patients (SEPs), as the effectiveness measure were calculated overall and by organization. Intervention delivery costs totaled $305K across eight organizations (range: $10.2K–$110K). Overall delivery cost per SEP was $14.43 (range: $10.37–$19.10). The largest cost category across organizations was implementation, specifically mailing preparation. The overall ICER was $483 per SEP-adjusted completed FIT (range: $96–$1,021 among organizations with positive effectiveness). Lagged data accounting for implementation delay produced comparable results. The costs of colonoscopies following abnormal FITs decreased the overall ICER to S409 because usual care clinics generated more such colonoscopies than intervention clinics. Using lagged data, follow-up colonoscopies increase the ICER by 4.3% to $460. Results indicate the complex implications for cost-effectiveness of implementing standard CRC screening within a pragmatic setting involving FQHCs with varied patient populations, clinical structures, and resources. Performance variation across organizations emphasizes the need for future evaluations that inform the introduction of efficient CRC screening to underserved populations.

INTRODUCTION

Over the past two decades, colorectal cancer (CRC) incidence has declined in the United States, yet CRC remains the third most common cancer and the second most common cause of death, with over 140,000 new cases and over 50,000 deaths expected in 2018 (Siegel, Miller, & Jemal, 2018). It has been known for at least this long that effective CRC screening can reduce incidence and mortality, as reflected in US Preventive Services Task Force recommendations for CRC screening among adults aged 50–75. However, in 2015 only 63% of adults aged 50 and older were up-to-date on CRC screening (American Cancer Society, 2017), a rate below the targets of the National Colorectal Cancer Roundtable (80%) (National Colorectal Cancer Roundtable, 2018) and Healthy People 2020 (70.5%) (Office of Disease Prevention and Health Promotion, 2017). Despite recent improvement (30.2% in 2012 to 39.9% in 2016), CRC screening rates among adults served by federally qualified health centers (FQHCs) remain well below those of non-FQHC populations (National Colorectal Cancer Roundtable, 2018). Fecal immunochemical testing (FIT) may be a low-cost and effective population-based screening option in the FQHC context when combined with colonoscopy follow-up for positive FITs.

We conducted the Strategies and Opportunities to STOP Colon Cancer in Priority Populations (STOP CRC) study to evaluate the effectiveness of a mailed FIT intervention delivered by clinic staff at FQHCs. This cluster-randomized pragmatic study provided 13 clinics with electronic health record (EHR) tools to identify and contact patients who were due for screening; trained clinic staff to use the tools; and compared results to 13 clinics practicing usual care. This paper presents the results of a cost-effectiveness analysis of the STOP CRC intervention.

METHODS

Study design, recruitment details, and results have been published previously (Coronado et al., 2018; Coronado et al., 2016; Coronado, Vollmer, Petrik, Aguirre, et al., 2014; Coronado, Vollmer, Petrik, Taplin, et al., 2014; Coury et al., 2017; Petrik et al., 2016). The study was approved by the Institutional Review Board of Kaiser Permanente Northwest (Protocol # 4364), with ceding agreements from Kaiser Permanente Washington Health Research Institute and OCHIN (formerly Oregon Community Health Information Network).

Setting and Participants

The study included seven FQHCs representing 24 clinics and two clinics affiliated with an academic medical center, serving similar low-income populations. Participating health centers were willing to randomize clinics and to use a single fecal test across all participating clinics, had an electronic interface with the lab that processed the FIT kits, and had sufficient follow-up colonoscopy capacity (G. D. Coronado et al., 2016).

Participating clinics were randomized to either usual care (n = 13) or an electronic health record (EHR)-embedded intervention (n= 13) described below. Eligible adults were aged 50–74, had a clinic visit during the 12 months prior to accrual, and were due for CRC screening based on having no EHR evidence of completing a fecal test during the past 11 months, a flexible sigmoidoscopy during the past 4 years, or a colonoscopy during the past 9 years; and no evidence of a fecal test order in the past 6 months or a sigmoidoscopy or colonoscopy referral in the past year. Adults were excluded with evidence of a limited set of health conditions (e.g. colorectal cancer, colon disease, end-stage renal failure). Patients were identified using real-time EHR tools updated daily.

Usual Care

Usual care clinics continued standard CRC screening processes, which varied by health center and typically involved providing information and ordering screening tests during routine clinical encounters. Usual care clinics were offered training and intervention materials at the end of the follow-up period. Activity in usual care (and intervention) clinics may have been influenced by changes in the external environment during the study period, which saw secular growth in CRC screening within FQHCs, both nationally and in study clinics. In particular, the Affordable Care Act’s Medicaid expansion removed a critical structural barrier to CRC screening by offering insurance to many previously uninsured Oregonians and Californians who were age-eligible for CRC screening. Also, in 2014 CRC screening became an incentivized metric for Oregon’s Coordinated Care Organizations that administer services for Medicaid enrollees.

Intervention

The STOP CRC intervention consisted of an automated data-driven, EHR-embedded program (Epic© EHR software [version 2010; Verona, WI] Reporting Workbench) for mailing FIT kits to patients due for CRC screening. Reporting Workbench users work with customized templates to produce “real-time” reports on lists of patients, including orders, appointments, or diagnoses. Tools were designed so that eligible patients could be sent a letter introducing the study (available in English, Spanish, Russian, and Mandarin Chinese) with a number to call if they had clinical concerns, had been previously screened, or simply declined participation. Clinic staff were trained on how to mail eligible patients FIT kits, including pictorial instructions and return postage (G. D. Coronado, Sanchez, et al., 2014; Petrik et al., 2016). The EHR tools generated lists of patients not reported as completing the kit, to whom a single reminder letter could be mailed.

Reports at participating clinics were updated nightly through EHR data on eligibility, mailing, and FIT completion status, with completion representing processing and reporting of a returned FIT. Four to six months after clinic staff training, a plan-do-study-act (PDSA) improvement cycle was facilitated during which, participating clinics identified strategies to enhance reach or effectiveness (Coury et al., 2017). The STOP CRC intervention had three basic elements (introductory letter, FIT kit, and reminder letter); however, organizations tailored implementation to their individual systems.

Trial outcomes

The primary study outcome was clinic-level proportions of eligible adults during the accrual interval (February 2014–February 2015) who completed FIT testing within 12 months, or through August 2015 (after which, study tools were made available to usual care clinics). A secondary outcome was the clinic-level proportion of participants receiving any CRC screening (FIT, sigmoidoscopy or colonoscopy) during the evaluation interval. Implementation was calculated as the clinic-level proportion of participants mailed an introductory letter and who subsequently ordered a FIT during the evaluation interval. This allowed mailed FITs to be distinguished from those distributed in-clinic.

Lagged analysis

While the planned analysis included all individuals accrued after EHR tools were provided to clinics on February 4, 2014 (the date of randomization), no clinic began printing letters until at least June 2014; some did not begin until spring 2015. This delay in implementation allowed clinics to address site-specific issues, such as conducting staff training in EHR tools, obtaining supplies, and dealing with staff turnover. To account for this delay, analyses were repeated using a “lagged” dataset that included only individuals accrued between June 4, 2014 and February 3, 2015. As with the primary dataset, outcomes were assessed through August 3, 2015, after which intervention materials were made available to usual care clinics.

Economic outcome

The primary analytic outcome is an incremental cost-effectiveness ratio (ICER), the additional cost per outcome for an intervention that improves outcomes over a reference strategy (here, usual care). The ICER was calculated as (cost_i – cost_uc)/)/(effect_i –effect_uc), where i = intervention and uc = usual care. For tractability as well as to account for differences in clinic size across organizations, the number of completed FITs adjusted for number of screening-eligible patients (SEPs) was used as the effect measure, rather than the proportion of such adults with completed FITs. We calculated the ICER overall as well as for each participating organization using both the primary and lagged trial outcomes.

Costs

Intervention delivery costs were defined as the value of resources used to develop, implement, and maintain the screening intervention over the trial period and were measured from the organizational perspective (Basu, 2016). Research-related costs were excluded. Intervention components were classified as labor (e.g., mailing activities) or non-labor (e.g., FIT kits).

To capture labor resources, the research team developed a series of spreadsheets for clinic staff to complete. The spreadsheets were organized in an activity-based costing format (Lee, Austin, & Pronovost, 2016), disaggregating the STOP CRC intervention into a series of activities classified in a few categories: data organization and management, staff training, implementation process, program management, test processing, and delivery support (Table 1). Program management was defined as billing adjustments, PDSA meetings, and provider engagement meetings. Intervention activities reported by the clinics were based on the project workplan and were reviewed by the research team for validity and completeness. The cost of colonoscopy with polypectomy or biopsy is adapted from Naber et al., 2018 and reported in 2018 US dollars ($1,897) (Naber et al., 2018; US Bureau of Labor Statistics, 2018). Costs are reported in 2018 US dollars and are not discounted because of the limited time horizon. Confidence intervals are calculated applying Fieller’s theorem (Willan & O’Brien, 1996).

Table 1.

STOP CRC Intervention Activities

Data organization and management

Updating claims data (e.g., historical colonoscopies)

Initial EHR training

Testing EHR tools

Training of additional staff (e.g., MA)

Execution of lab interface agreements

Lab orders tracking

Results pool tracking

Staff training

On-going training/meetings

On-boarding of new staff

Dissemination labo

Adapting/approving mailed materials

Mailing introductory letter

Mailing FIT kits

Mailing reminders

In-clinic FIT kit distribution

Dissemination non-labor

Introductory letters with envelope

FIT kits

Reminder letters

Program management

Billing adjustments

Conducting a PDSA

Provider engagement meetings

Test processing

Processing of returned FITs

Reimbursement for returns from insured

Delivery support

Responding to patient phone calls

EHR: electronic health record

MA: medical assistant

FIT: fecal immunochemical test

PDSA: plan-do-study-act

TRIAL RESULTS

Primary dataset

Table 2 lists the numbers of screened participants and their proportions of SEPs by organization. Overall, intervention clinics obtained completed FITs from 14.3% of their SEPs (3,003/21,134), compared to 10.7% of SEPs (2,146/20,059) in usual care clinics. Both arms exhibited considerable variability in the proportion of completed FITs; proportions among intervention clinics ranged from 4.3% (101/2,352) to 22.9% (403/1,761) and from 2.7% (23/840) to 21.3% (427/2,004) among usual care clinics. Also, within three organizations the proportion of returned FITs among SEPs in their intervention clinics was lower than in their usual care clinics, with differences ranging from −2.9% to −7.4%.

Table 2.

STOP CRC Outcomes

PRIMARY
	USUAL CARE				INTERVENTION
Health Center	N screened	Eligibles	Proportion		N screened	Eligibles	Proportion	12-month change
1	23	840	2.7%		123	606	20.3%	17.6%
2	498	4,260	11.7%		1,227	5,762	21.3%	9.6%
3	427	2,004	21.3%		403	1,761	22.9%	1.6%
4	221	3,246	6.8%		265	1,882	14.1%	7.3%
5	350	2,991	11.7%		101	2,352	4.3%	−7.4%
6	372	3,349	11.1%		647	5,262	12.3%	1.2%
7	214	2,401	8.9%		128	2,129	6.0%	−2.9%
8	145	968	15.0%		108	1,380	7.8%	−7.2%
Total	2,146	20,059	10.7%		3,003	21,134	14.2%	3.5%
Primary outcome: Proportion of persons completing a FIT within 12 months of eligibility for screening
LAGGED
USUAL CARE					INTERVENTION
Health Center	N screened	Eligibles	Proportion		N screened	Eligibles	Proportion	12-month change
1	23	674	3.4%		122	496	24.6%	21.2%
2	441	3,429	12.7%		1,065	4,359	23.3%	10.6%
3	344	1,571	21.9%		377	1,392	27.1%	5.2%
4	185	2,508	8.0%		209	1,284	15.7%	7.7%
5	345	1,827	18.9%		133	1,851	7.2%	−11.7%
6	297	2,146	13.7%		564	3,337	17.3%	3.6%
7	207	1,984	11.2%		173	1,874	9.2%	−2.0%
8	130	765	17.0%		135	1,170	11.6%	−5.4%
Total	1,972	14,904	12.7%		2,778	15,763	17.5%	4.8%

FIT: fecal immunochemical test

Lagged dataset

Overall, intervention clinics obtained completed FITs from 17.5% of their SEPs (2,778/15,763), compared to 12.7% of SEPs (1,972/14,904) in usual care clinics. Both arms exhibited considerable variability in the proportion of completed FITs; proportions among intervention clinics ranged from 7.2% (133/1,851) to 27.1% (377/1,392) and from 3.4% (23/674) to 21.9% (344/1,571) among usual care clinics. Also, within three organizations the proportion of completed FITs among SEPs in their intervention clinics was lower than in their usual care clinics, with differences ranging from −2.0% to −11.7%.

Economic Results

Table 3 presents delivery costs and baseline ICERs, both in total and by organization. Delivery costs totaled $305K, ranging from $10.2K to $110K across organizations. Overall delivery cost per SEP was $14.43 and varied from $10.37 (HC6) to $19.10 (HC2) across organizations. The overall ICER across all eight organizations was $483 per SEP-adjusted completed FIT; however, this overall value includes three organizations for which their intervention clinics generated fewer SEP-adjusted completed FITs than their usual care clinics. (One organization reported fewer absolute, but a higher proportion of, FITs in its intervention clinics.) For the five organizations reporting more SEP-adjusted completed FITs in their intervention clinics, ICERs ranged from $96 to $1,021 per SEP-adjusted completed FIT. Using the lagged results (Table 4), three organizations produced fewer SEP-adjusted completed FITs in intervention clinics than in usual care clinics. The overall ICER was $441 per SEP-adjusted completed FIT, although organization-level ICERs ranged from $97 to $534.

Table 3.

STOP CRC delivery costs and incremental cost-effectiveness ratios: primary dataset (95% CI)^*

SYSTEM	HC1	HC2	HC3	HC4	HC5	HC6	HC7	HC8	TOTAL
Screening-eligible patients (SEP)
Intervention	606	5,762	1,761	1,882	2,352	5,262	2,129	1,380	21,134
Usual care	840	4,260	2,004	3,246	2,991	3,349	2,401	968	20,059
Intervention delivery cost
Total	$10,171	$110,035	$28,363	$30,147	$26,434	$54,557	$28,430	$16,860	$304,997
Per SEP	$16.78	$19.10	$16.11	$16.02	$11.24	$10.37	$13.35	$12.22	$14.43 ($12.45–$16.41)
Clinics (N)	1	3	1	2	1	2	1	2	13
# of completed FITs
Intervention	123	1,227	403	265	101	647	128	108	3,002
Usual care	23	498	427	221	350	372	214	145	2,250
Incremental	100	729	−24	44	−249	275	−86	−37	752
Completed FITs per screening-eligible patient
Intervention	0.20	0.21	0.23	0.14	0.04	0.12	0.06	0.08	0.14
Usual care	0.03	0.12	0.21	0.07	0.12	0.11	0.09	0.15	0.11
Incremental	0.18	0.10	0.02	0.07	−0.07	0.01	−0.03	−0.07	0.03 (−0.03–0.09)
Abnormal FITs
Intervention	11	219	35	39	15	53	12	19	403
with CS	3	66	20	27	1	10	7	10	144
Usual care	1	110	18	21	122	41	15	25	353
with CS	0	52	7	7	64	16	5	9	160
Incr. abn. FITs	10	109	17	18	−107	12	−3	−6	50
Incr. abn. FITs w/ CS	3	14	13	20	−63	−6	2	1	−16
Incremental cost-effectiveness ratios
Delivery cost / completed FITs	$102	$151	−$1,182	$685	−$106	$198	−$331	−$456	$406
Delivery cost per SEP / completed FITs per SEP	$96	$199	$1,021	$220	−$152	$873	−$460	−$171	$483 ($458–$511)
Delivery cost (w/ CS) / completed FITs	$159	$187	−$2,209	$1,547	$374	$157	−$375	−$507	$365
Delivery cost (w/ CS) per SEP / completed FITs per SEP	$149	$184	$1,967	$538	$385	$413	−$539	−$116	$409 ($388–$433)

HC: health center; FIT: fecal immunochemical test; CS: (follow-up) colonoscopy; CI: confidence interval

^*Negative incremental ratios mean that the intervention was more expensive and less effective than usual care.

Table 4.

STOP CRC delivery costs and incremental cost-effectiveness ratios: lagged dataset (95% CI)^*

SYSTEM	HC1	HC2	HC3	HC4	HC5	HC6	HC7	HC8	TOTAL
Screening-eligible patients (SEP)
Intervention	496	4,359	1,392	1,284	1,851	3,337	1,874	1,170	15,763
Usual care	674	3,429	1,571	2,508	1,827	2,146	1,984	765	14,904
Intervention delivery cost
Total	$10,171	$110,035	$28,363	$30,147	$26,434	$54,557	$28,430	$16,860	$304,997
Per SEP	$20.51	$25.24	$20.38	$23.48	$14.28	$16.35	$15.17	$14.41	$19.35 ($16.58–$22.12)
Clinics (N)	1	3	1	2	1	2	1	2	13
# of completed FITs
Intervention	122	1,065	377	209	133	564	173	135	2,778
Usual care	23	441	344	185	345	297	207	130	1,972
Incremental	99	624	33	24	−212	267	−34	5	806
Completed FITs per screening-eligible patient
Intervention	0.25	0.24	0.27	0.16	0.07	0.17	0.09	0.12	0.18
Usual care	0.03	0.13	0.22	0.07	0.19	0.14	0.10	0.17	0.13
Incremental	0.21	0.12	0.05	0.09	−0.12	0.03	−0.01	−0.05	0.04 (−0.03–0.11)
Abnormal FITs
Intervention	10	178	32	27	29	51	18	29	374
with CS	4	63	16	20	10	9	6	8	136
Usual care	1	71	32	17	87	30	14	40	292
with CS	0	27	16	4	46	12	6	11	122
Incr. abn. FITs	9	107	0	10	-58	21	4	−11	82
Incr. abn. FITs w/ CS	4	36	0	16	-36	-3	0	−3	14
Incremental cost-effectiveness ratios
Delivery cost / completed FITs	$103	$176	$859	$1,256	-$125	$204	−$836	$3,372	$378
Delivery cost per SEP / completed FITs per SEP	$97	$218	$393	$264	-$122	$534	−$1,262	−$264	$441 ($426–$456)
Delivery cost (w/ CS) / completed FITs	$179	$286	$859	$2,521	$197	$183	−$836	$2,234	$411
Delivery cost (w/ CS) per SEP / completed FITs per SEP	$169	$326	$441	$562	$199	$355	−$1,290	−$2	$460 ($444–$476)

HC: health center; FIT: fecal immunochemical test; CS: (follow-up) colonoscopy, CI: confidence interval

^*Negative incremental ratios mean that the intervention was more expensive and less effective than usual care.

Per-clinic delivery costs, averaging $23.3K across organizations, ranged from $8.4K (HC8) to $36.7K (HC2). Per-clinic delivery costs for HC2 were somewhat higher than for other organizations because HC2 reported 300 hours of full-time staff training in preparation for intervention start-up, which were substantially higher than for any other organization. Figure 1 presents STOP CRC’s per-clinic activity categories by organization. Regardless of the magnitude of overall costs, the largest reported cost category for each organization was implementation, specifically mailing preparation, which included printing letters, affixing labels on tubes or cards and envelopes, and placing lab orders.

Figure 1.

STOP CRC Activity Categories Per Clinic, by Health Center (2018 US$)

The ICERs reported earlier do not include costs of follow-up colonoscopy for abnormal FITs; however, potential implementers of a screening program such as STOP CRC are presumably interested in its implications for limited colonoscopy resources. Table 3 also presents primary data on the number of SEP-adjusted completed FITs per organization that were judged abnormal and the number receiving follow-up colonoscopy. This is a conservative cost estimate for colonoscopy because many colonoscopies do not involve polypectomy or biopsy. Adding the cost of colonoscopies for abnormal FITs decreases the overall incremental cost per returned FIT to $409. This counterintuitive result arises because 45.3% of abnormal FITs in usual care clinics were followed up with colonoscopy versus 35.7% in intervention clinics. However, this phenomenon is not observed in the lagged data (Table 4); follow-up colonoscopies increase the cost per SEP-adjusted completed FIT by 4.3% to $460.

DISCUSSION

Although the total cost of delivering the STOP CRC intervention varied substantially ($10.1K-$110K) across organizations, the delivery cost per SEP varied much less ($10.37–$19.10). It should be noted that HC2 is a county-wide health system with a workforce and patient population, both significantly larger than other participating organizations. The largest cost categories were related to disseminating the FIT to screening-eligible patients and general data management, involving developing program tracking reports for use in extant electronic data systems, and staff training in use of the reports, which were new to the clinics. They have subsequently been applied to other population-based care efforts, and less training may be required if staff has experience with the tools. Apart from one organization that devoted substantial resources to staff training, implementation was the most resource-intensive activity for all organizations. The resource burden of test processing was minimal.

As we previously report, STOP CRC achieved a 21% return rate on mailed FITs (Coronado et al., 2018), consistent with many previous evaluations of mailed FIT outreach (Goldman et al., 2015; Green, Anderson, et al., 2017; Green, Fuller, et al., 2017; Gupta et al., 2013; Levy, Xu, Daly, & Ely, 2013; Singal et al., 2016). As a pragmatic study, STOP CRC relied on clinic staff to deliver the intervention, and levels of intervention delivery varied substantially by clinic. Only one-third of eligible patients were mailed a FIT, with clinic-level performance ranging from 3% to 68% (Coronado et al., 2018). The STOP CRC primary outcomes evaluation relied on difference between intervention and usual care in clinic-level proportions of completed FITs, which was much lower than the FIT return rate (3.4% vs. 21%). These results suggest that additional resources directed towards staff training in mailed FIT outreach could improve the cost-effectiveness of an intervention such as STOP CRC.

The study-wide ICER for STOP CRC of $483 per SEP-adjusted completed FIT ($441 in lagged data) is somewhat higher than those of other CRC screening outreach studies (Lewis, Brenner, McGriffith, & Pignone, 2008; Tangka et al., 2013; Liss et al., 2016; Sequist, Franz, & Ayanian, 2010; Meenan et al., 2015). However, each of these studies differ from ours in significant ways, ranging from multi-modality (Lewis) to clinical cost assessment (Tangka) to simulation-based budget impact analysis (Liss) to a multispecialty group practice (Sequist) to an integrated health care system in which all patients had insurance, with colonoscopy costs all or mostly covered and with easy access to endoscopy services (Meenan). This differs from community clinics, in which screening colonoscopies are less common because of less insurance coverage and limited access (Bass et al., 2011; Davis, Morris, Rademaker, Ferguson, & Arnold, 2017; Ferreira et al., 2005; Robinson et al., 2011).

To our knowledge, this is the first cost-effectiveness analysis of a pragmatic CRC screening study conducted across a variety of FQHCs. The overall ICERs mask considerable heterogeneity in performance across the eight participating organizations. In both primary and lagged data, in three organizations the intervention did not increase the number of SEP-adjusted completed FITs over usual care. In the other organizations, the ICERs varied widely. This was in part due to the pragmatic aspects of the STOP CRC trial in which organizations implemented the intervention in a manner appropriate for their system and resources. The study’s pragmatic nature likely resulted in higher costs for clinic staff to conduct data cleaning and training, especially in smaller clinics without extensive organizational infrastructure.

The pragmatic design may also have contributed to our observation, within the primary dataset, that the proportion of abnormal FITs receiving follow-up colonoscopy was higher in usual care clinics than in intervention clinics, which decreased the overall incremental cost per returned FIT. This could reflect small-sample randomness or could reflect the fact that usual care clinics primarily distributed FIT kits during clinical encounters; such kits typically received better follow-up than mailed kits. An abnormal result in usual care would prompt a provider referral following a visit in which the kit was distributed. A mailed FIT may have had an abnormal result, but a more recent colonoscopy could have been found in the patient’s record, co-morbidities (e.g., cancer) that made colonoscopy follow-up inappropriate, or the patient may simply have been lost to follow-up.

Despite the heterogeneous results across organizations, a few lessons from our experience may be useful to future adopters. First, since mailing activities represent the largest portion of cost across organizations, dissemination methods that can be integrated well into regular staff activities would enhance the efficiency of programs such as STOP CRC. This is especially salient, given the varied ability of clinics to reach screening-eligible patients. As noted earlier, screening uptake among contacted patients was comparable to many previous studies, so staff training in efficient dissemination methods could yield significant benefits in terms of more screened patients. Second, it is important to exploit extant programs whenever possible, e.g., use existing quality improvement staff, if available, to enhance consistent and efficient program delivery. Third, “scrubbing” data records (i.e., removing or amending incorrect, incomplete, or duplicate data) is expensive. Processes that generate correct and current data the first time will facilitate efficient identification of relevant screening events. Fourth, clinics will benefit from the continued rollout of EHR systems within FQHCs and related tools, such as Reporting Workbench. As these systems become more prevalent, and clinic staff become more experienced users, challenges, for example, of data capture (e.g., colonoscopy underreporting, verification of FIT mailing) should gradually lessen, reducing costs. This is not to ignore the importance of external factors such as staff turnover and conflicting management priorities, but simply to acknowledge the potential for improved information flows that will enhance the success of programs such as STOP CRC. Fifth, always be in a “learning” mode; clinics should use documented evidence and the experience of other systems to inform their ongoing program activities. To that end, regular meetings with other implementing clinics can help participants learn from each other’s successes and failures.

This analysis has several limitations, particularly its reliance on retrospective questionnaire responses from implementing staff. Although this is appropriate for micro-level data collection (Frick, 2009), such an approach has inherent recall issues. We strove to minimize these issues by focusing on key informants at each organization and attempting to facilitate consistent understanding of terms and concepts. However, the heterogeneity of experiences and ability to recall events, magnified by high staff turnover at some sites, complicated data collection. Whenever possible, we re-contacted our informants to clarify responses, but considerable ambiguity remains. Future economic research should explore new methods of extracting micro-level implementation data vital to understanding the economics of screening programs. Data collection methods that are not perceived by clinic staff as intrusions into ongoing patient care are especially valuable. A checklist for the conduct and reporting of micro-costing studies may be helpful in this regard (Ruger & Reiff, 2016). In addition, “time-driven” activity-based costing (TDABC), a modified form of the standard ABC methodology applied in this study, has been described as a micro-costing approach well suited to accommodate complex health care cost accounting (Kaplan & Anderson, 2004; Kaplan & Porter, 2011). Standard ABC is considered a resource-intensive approach to data collection, which can inhibit its use. TDABC is intended to maintain the validity of cost data while reducing the resources needed to acquire them. TDABC requires only two key parameters: the capacity cost rate (the cost of capacity-supplying resources divided by their practical, not theoretical, capacity), and the time required to perform service delivery activities. To date, TDABC has been used primarily to analyze hospital and clinic services (Keel, Savage, Rafiq, & Mazzocato, 2017), but its utility in facilitating evaluation of FQHC screening programs such as STOP CRC should also be explored.

In addition, program-level data did not distinguish between costs of screening and diagnostic colonoscopy for each organization, which complicates understanding the true intervention effects in terms of improving CRC screening rates. Improved organization-level data systems will mitigate this issue. Also, our implementation cost estimates are based on eight organizations. Finally, cross-organizational differences in patient population, management commitment to STOP CRC, resource availability, and other latent factors may well contribute to our reported cost differences.

CONCLUSION

Our results indicate the implications for cost-effectiveness of implementing a standard CRC screening intervention within a pragmatic trial setting involving multiple FQHCs with varied patient populations, clinical structures, and resource availability. The variation in performance across organizations serves to emphasize the need for future similar evaluations that can contribute to our knowledge of how to introduce such screening programs to underserved populations most effectively and efficiently.

Highlights:

Implementing colorectal cancer screening across community health centers is complex Intervention delivery costs and overall effectiveness vary across health centers Data cleaning and training will likely be more expensive in smaller health centers Economic effects of follow-on colonoscopies depend on success of fecal test referrals