A Randomized Controlled Trial to Increase Cancer Screening Among Attendees of Community Health Centers

Abstract

BACKGROUND We assessed the efficacy of the Cancer Screening Office Systems (Cancer SOS), an intervention designed to increase cancer screening in primary care settings serving disadvantaged populations.

METHODS Eight primary care clinics participating in a county-funded health insurance plan in Hillsborough County, Fla, agreed to take part in a cluster-randomized experimental trial. The Cancer SOS had 2 components: a cancer-screening checklist with chart stickers that indicated whether specific cancer-screening tests were due, ordered, or completed; and a division of office responsibilities to achieve high screening rates. Established patients were eligible if they were between the ages of 50 and 75 years and had no contraindication for screening. Data abstracted from charts of independent samples collected at baseline (n = 1,196) and at a 12-month follow-up (n = 1,237) was used to assess whether the patient was up-to-date on one or more of the following cancer-screening tests: mammogram, Papanicolaou (Pap) smear, or fecal occult blood testing (FOBT).

RESULTS In multivariate analysis that controlled for baseline screening rates, secular trends, and other patient and clinic characteristics, the intervention increased the odds of mammograms (odds ratio [OR] = 1.62, 95% confidence interval [CI], 1.07–9.78, P = .023) and fecal occult blood tests (OR = 2.5, 95% CI, 1.65–4.0, P <.0001) with a trend toward greater use of Pap smears (OR = 1.57, 95% CI, 0.92–2.64, P = .096).

CONCLUSIONS The Cancer SOS intervention significantly increased rates of cancer screening among primary care clinics serving disadvantaged populations. The Cancer SOS intervention is one option for providers or policy makers who wish to address cancer related health disparities.

INTRODUCTION

Patients belonging to a racial or ethnic minority are more likely to have poor cancer outcomes.^{1– 7} Eliminating racial or ethnic disparities in health was a major emphasis of former Surgeon General Satcher and is now a public health goal for Healthy People 2010 (http://www.healthypeople.gov). Patients of low socioeconomic status and those who are uninsured or insured by Medicaid are also at greater risk of poor cancer outcomes.^{4, 5, 8– 14} The reasons the aforementioned groups have less favorable cancer outcomes is not certain but have been largely attributed to lower use of screening tests.^{15– 24}

Interventions to increase cancer screening have been systematically reviewed, and several limitations relevant to cancer screening in primary care can be identified.^{25– 29} First, many successful interventions have relied on computer systems and technology that may not be widely available to clinics caring for the underserved.^{30– 46} Likewise, other successful interventions have relied on personnel and resources that would not typically be available to primary care practices, such as lay health advisors, telephone counselors, or nursing staff dedicated solely to cancer screening.^{47– 54} Finally, whereas noncomputerized office systems interventions have been successful when targeted to typical primary care settings,^{33, 55– 57} their effectiveness in clinics serving disadvantaged populations is less certain, with some interventions having success⁵⁸ and others not.⁵⁹

To address these limitations, we developed a low-cost office systems intervention, Cancer Screening Office Systems (Cancer SOS),^* for primary care clinics serving disadvantaged populations. The intervention is noncomputerized and relies on personnel and resources that are available to most primary care clinics. We tested the efficacy of the system among patients attending community health centers, a representative setting of care for the target population.

METHODS

To target an underserved population, clinics were recruited from among 16 clinics participating in a county-funded health insurance plan in Hillsborough County, Fla. The county health plan provides health care for uninsured persons who do not qualify for Medicaid or Medicare and who have a serious chronic health condition. Clinics were eligible for the randomized trial if (1) they provided primary medical care 5 days a week, (2) a majority of the physician and non-physician providers agreed to participate, and (3) the clinic was expected to continue operating in the same fashion for the following 24 months. Each clinic individually decided whether to participate in the intervention; no clinic was obligated to join the intervention.

Eight clinics did not meet eligibility. One clinic refused to participate, 1 clinic was open only 1 day per week, and 6 clinics were uncertain whether they would be operating in the same fashion during the 2-year period of the grant (possibility of closing down, merging with another clinic, reducing days of service, loss of key personnel, etc). We performed a cluster-randomized experimental trial in which 8 clinics meeting eligibility criteria were randomized to either intervention or control conditions. Screening outcomes were assessed at 12 months and at 24 months. Results from 12 months of follow-up are reported here.

The intervention targeted 3 cancer-screening tests: mammograms, Papanicolaou (Pap) smears, and fecal occult blood tests (FOBT). Key components of the intervention included a cancer-screening checklist completed by patients and indicating whether patients were due for screening, and a series of red, yellow, and green stickers that indicated whether recommended screening tests had been ordered and completed. Appendix 1, which is available online only at http://www.annfammed.org/cgi/content/full/2/4/294/DC1, provides additional detail about the clinics that participated and additional detail about the intervention itself.

Intervention procedures were explained to office staff and providers during a 45-minute training session and were summarized in training manuals given to all staff. To insure that the Cancer SOS intervention was being implemented appropriately, project staff conducted unannounced audits of a random sample of 25 charts (of eligible patients seen in the most recent week) at 1 month, 2 months, and 3 months. Audits determined the percentage of eligible patients who appropriately had Cancer SOS checklists in their chart and whether the color-coded stickers were appropriately used. Compliance with the system was also assessed during formal feedback sessions with clinic staff that occurred every 6 months. Overall compliance with the system averaged 74% at 6 months and 71% at 12-months. At these same times, project staff also assessed control clinics for possible contamination of intervention methods (which was not found).

To provide ongoing reinforcement of the intervention to clinic staff, we conducted formal feedback sessions at 6 months and at 12 months after the intervention had been implemented. A random sample of 50 charts was abstracted before the 6-month session to provide clinicians and staff with feedback on how their screening rates were progressing under the intervention. Office staff and project staff jointly discussed how the intervention was proceeding, what problems were occurring, and what might be done to improve implementation.

Data Collection

During data collection periods, research assistants assembled sampling frames of all patient visits using office billing and scheduling records. Patient’s records were eligible to be abstracted if both the following criteria were met: (1) the patient was 50 to 75 years of age, and (2) the patient was established in the clinic (defined as having had at least 1 visit 12 months or more before the sampled visit). Based on sample size requirements for the intervention (the intervention was structured to provide 80% power to detect increases in screening rates of 20% or more), independent random samples of 150 charts were selected for each clinic at baseline and again 12 months after the intervention had been fully implemented in the clinic. Random selection was achieved using a list of random numbers generated by SAS/STAT software (SAS Institute, Inc, Cary, NC).

To prevent medical record reviews from influencing patient or provider screening behavior, and to allow adequate time for recommended screening tests to be completed, we abstracted charts 3 months after sampled patients had visited the clinic. Neither patients nor staff members were aware of when chart abstractions would occur. In addition, randomization of clinics into intervention and control arms of the study did not take place until after baseline data were collected.

For each of the targeted cancer-screening tests, the date the procedure was completed was recorded to determine whether the patient was up-to-date on screening. We defined being up-to-date as having completed the targeted screening test within either the 12 months before the audited visit or within the 3 months after the audited visit. The use of a grace period has been applied in other studies^{37, 55, 56, 59– 61} and allows sufficient time for screening tests that were recommended at an audited visit to have been completed by the patient.

Chart abstracters used a standardized method and instrument to abstract chart information and were trained by the project manager. Relevant clinical data were abstracted from all sections of the chart, including progress notes, laboratory reports, radiology reports, consultation letters, and hospital records. Before beginning data collection, interrater reliability for chart abstractors was assessed for the 3 cancer-screening tests by re-reviewing a sample of 30 charts and calculating the kappa statistic. The following values of kappa were obtained when assessing whether a patient was up-to-date on cancer screening: Pap smear, κ= 1.00; mammogram, κ= 1.00; FOBT, and κ= 0.91.

Our statistical analyses focused on 3 primary outcomes. We assessed whether the patient was up-to-date on one or more of the following cancer screening tests: mammogram, Pap smear, or FOBT. Women who had a personal history of breast cancer were excluded from our analysis of mammograms. Women with a personal history of cervical cancer or those who had had a hysterectomy were excluded from our analysis of Pap smear screening. Patients with a personal history of colon cancer and those who had received a colonoscopy or double-contrast barium enema in the previous 10 years were excluded from the analysis of FOBT.

The final data set consisted of the combined abstracted records from the 2 independent samples collected at baseline and at 12 months postintervention. We adjusted simultaneously for potential confounders with the method of generalized linear models using PROC GENMOD in SAS (SAS Version 8, SAS Institute Inc, Cary, NC). The following variables were included in regression models: age, sex (if appropriate), race-ethnicity, marital status, smoking status, health insurance, comorbidity (using the Charlson Comorbidity Index^{62, 63}), number of chronic illnesses, number of prescribed medications, number of health care visits in the previous year, clinic attended, primary language spoken, family history of targeted cancers, and for women the use of estrogen replacement therapy and history of benign breast disease.

Indicator variables were also created for clinic type (control vs intervention), and for survey year (baseline, postintervention 12-month follow-up). We also included an interaction term for the 2 variables of clinic type and survey year. The interaction term estimates the effect of the intervention while controlling for any baseline screening differences and secular trends in screening rates, and adjusting for other covariates. Because of the clustered nature of the data, with patients attending the same clinic and some patients being sampled by chance in both baseline and follow-up samples, we obtained parameter estimates and 95% confidence intervals using the method of generalized estimating equations.^{64, 65}

To calculate attributable numbers, we first estimated the relative risks from odds ratios using the formula by Zhang and Yu.⁶⁶ We then calculated attributable fractions for each screening test and used these to estimate the number of cancer-screening tests obtained among subjects that could be attributed to the Cancer SOS intervention.⁶⁷ We also assessed in preliminary fashion whether the Cancer SOS intervention is cost-effective as well as efficacious (for details, see Appendix 2, which is available online only at http://www.annfammed.org/cgi/content/full/2/4/294/DC1). This assessment involved estimating the costs of carrying out the intervention per patient and per screening test, and determining marginal costs (ΔC) and marginal effectiveness (ΔE) of the intervention. The incremental cost-effectiveness ΔC/ΔE of the Cancer SOS intervention was then compared, where possible, with comparable interventions reported in the published literature. This study was reviewed and approved by the University of South Florida Institutional Review Board, which waived the requirement for informed consent of individual patients.

RESULTS

Table 1 ▶ summarizes the clinical characteristics of patients attending control and intervention clinics. Screening rates 12 months after the intervention had been implemented were as follows: for Pap smears, intervention 62.4%, control 48.2%; for mammograms, intervention 75.7%, control 71.1%; and for fecal occult blood testing, intervention 40.1%, control 11.9%. During the 12-month follow-up period, screening rates in intervention clinics increased relative to screening rates in control clinics by the following amounts: FOBT, 14.4%; mammograms, 9.1%; and Pap smears, 9.9%. We also assessed the total number of screening tests obtained among women who were eligible for the 3 cancer-screening tests. At the 12 month follow-up, women in control clinics had received on average 1.25 cancer-screening tests compared with an average of 1.71 tests for women in intervention clinics (Wilcoxon rank sum = 7.94, P <.0001).

Table 1.

Characteristics by Group at Baseline, (N = 1,196)

	Control		Intervention
Clinical Characteristics	No.	%	No.	%	PValue
Group	596	49.8	600	50.2
Sex					.88
Male	129	21.6	132	22.0
Female	467	78.4	468	78.0
Age, years					.38
50–56	212	35.6	232	38.7
57–63	196	32.9	199	33.1
64–75	188	31.5	169	28.3
Race-ethnicity					.0009
African American	144	24.2	204	34.0
White	307	51.5	272	45.3
Hispanic	145	24.3	124	20.7
Marital status					.003
Married	142	23.8	189	31.5
Unmarried	454	76.2	411	68.5
Primary language					.23
English	464	77.8	484	80.7
Non-English (Spanish)	132	22.2	116	19.3
Health insurance					.48
County program	354	59.4	346	57.7
Medicaid	83	13.9	101	16.8
Medicare	122	20.5	112	18.7
Other	37	06.2	41	06.8
Smoking status					.61
Smoker	163	27.4	172	28.7
Nonsmoker	433	72.6	428	71.3
Health maintenance visit in past year					.16
Yes	329	55.2	307	48.8
No	267	44.8	293	51.2
Charlson comorbidity score					<.0001
0	249	41.8	175	29.2
1	129	21.7	146	24.3
2	120	20.1	156	26.0
>3	98	16.4	123	20.5
Chronic illnesses					<.0001
0–4	214	35.9	127	21.2
5–7	233	39.1	235	39.2
> 8	149	25.0	238	39.6
Medications prescribed					<.0001
0–5	272	45.6	110	18.3
6–8	182	30.5	153	25.5
> 9	142	23.8	337	56.2
Health care visits in past year					<.0001
0–4	213	35.7	150	25.0
5–7	185	31.0	185	30.8
> 8	198	33.2	265	44.2
Papanicolaou smear in past year*					.33
Yes	148	57.6	151	61.9
No	109	42.4	93	38.1
Mammogram in past year*					.13
Yes	337	75.9	325	71.4
No	107	24.1	130	28.6
Fecal occult blood test in past year*					<.0001
Yes	109	22.1	180	35.9
No	384	77.9	321	64.1

*Among patients who were eligible.

Multivariate analysis was used to assess the effectiveness of the intervention by determining the odds ratio for the interaction term previously described (Table 2 ▶). (Appendix 3, available online only at http://www.annfammed.org/cgi/content/full/2/4/294/DC1 describes other clinical predictors of screening). The intervention more than doubled the odds of FOBT screening, increased the odds of mammograms more than 60%, and increased the odds of screening Pap smears more than 50% (a finding that did not reach statistical significance, however). The impact of the intervention was also assessed in absolute terms. Among the intervention patients that were assessed at follow-up, the intervention resulted in 27 additional Pap smears among the 282 women eligible for screening, 33 additional mammograms among the 481 women eligible for screening, and 94 additional FOBTs among the 496 men and women who were eligible for screening. The corresponding numbers needed to treat (NNT) are as follows; Pap smear NNT = 10.4, mammograms NNT = 14.6, and FOBT NNT = 5.3.

Table 2.

Results of Multivariate Analyses on Intervention Effects

Screening Test	Number	Odds Ratio	95% CI	PValue
Papanicolaou smears	1,057	1.57	0.92–2.64	.096
Mammograms	1,832	1.62	1.07–9.78	.023
Fecal occult blood test	1,989	2.56	1.65–4.01	<.0001

CI = confidence interval.

On a per-patient basis, we estimate that the real costs of Cancer SOS are $5.39. When the expected cost per patient is allocated across the 3 screening tests, we find that the costs for mammography, Pap smears, and FOBT are $2.55, $1.96, and $2.96, respectively. When these per test costs are divided by the incremental effectiveness of Cancer SOS in respect to each test, we find ΔC/ΔE ratios of $55, $14, and $11 for mammography, Pap smears, and FOBT, respectively, which compare favorably with other screening interventions reported in the literature (see Appendix 2 for further details).

DISCUSSION

The Cancer SOS intervention successfully increased cancer screening in clinics serving primarily disadvantaged populations. After 12 months of follow-up, the odds of screening with mammography and FOBT both increased significantly, with a trend toward increased provision of Pap smears as well. In total, 154 additional cancer-screening tests could be attributed to the intervention among the 628 patients sampled at follow-up. Similar reminder systems have been tested in the past and are among the more successful strategies to promote cancer screening in primary care practices.^{56– 58} The novelty of our intervention results from its low cost, no need for computers, and involvement of the patient in the screening process.

The magnitude of effects of the Cancer SOS intervention was generally similar to those reported in systematic reviews.^{26, 29} It is difficult to compare strictly the magnitude of effects of the Cancer SOS intervention with other studies because most interventions are uniquely structured, and baseline screening rates vary considerably among studies. Among the 3 targeted screening tests, the magnitude of effect was clearly the greatest for FOBT. Screening rates for FOBT lag considerably behind those of Pap smears and mammograms, so there is much more room for improvement. Nationwide rates of colorectal cancer screening are among the lowest for recommended cancer-screening tests.^{68, 69}

There are several important questions concerning the viability of office systems as a general approach to increasing cancer-screening rates among disadvantaged populations. First is the durability of effects with time. Whereas many interventions increase cancer-screening behaviors for the short term, few have assessed durability of long-term effects.^{60, 70} Although the Cancer SOS intervention was successful for a 12-month period, it will be important to determine whether effects can be sustained, a question that will be addressed in a future study assessing outcomes at 24 months’ follow-up.

A second important factor is whether intervention strategies are cost-effective. Even though a more detailed analysis is planned, preliminary analysis suggests the Cancer SOS has reasonable cost-effectiveness. The least favorable cost-effectiveness ratio was observed for screening mammography, attributable mostly to the lower ΔE term for that test compared with Pap smears and FOBT. But even this ratio compares favorably to published cost-effectiveness ratios for interventions designed to increase the number of women undergoing screening mammography. More particularly, we found that the incremental cost-effectiveness ratio of Cancer SOS appeared to be lower than that reported by 9 of the 12 comparable studies found in our search of the US literature.^{71– 83}

Finally, the ability of successful interventions to diffuse from experimental settings into usual practice settings is largely unproved. To promote generalizability of our office systems approach, we attempted to utilize low-cost strategies and relied on personnel within the clinic as much as possible to carry out intervention tasks. It is also important to acknowledge that the Cancer SOS intervention targeted only 3 preventive services. Whether the intervention could be structured to improve preventive care more comprehensively is uncertain but would appear to be unlikely. The structure of Cancer SOS is more apt to be of value for providers who wish to improve a select number of preventive services.

We can also learn from situations in which office systems approaches have been less effective. First, there seems to be less success when clinic leadership is unstable.^{59, 84} Dietrich and colleagues⁵⁹ believed that office systems approaches face more obstacles in larger practices than in smaller practices, although their studies were unable to assess this issue rigorously. Studies have also reported less success when office systems interventions are applied to more representative samples of clinics as opposed to those samples that are recruited and presumably more highly motivated.^{59, 84} While not rigorously assessed in our study, all these observations ring true with our work with the Cancer SOS project. As such, we believe that interventions such as Cancer SOS will be most successful in clinics with stable leadership and at least minimal motivation to improve cancer screening.

This study has other limitations that should be considered. For simplicity we targeted a uniform age-group and uniform screening intervals (1 year) for 3 selected tests, and it is not known how the system would work if structured differently. Although our analysis was able to control for clustering that occurred at the level of the clinic, we did not collect detailed information on provider visits and were therefore unable to control for any clustering that may have occurred at the level of the individual physician. Finally, the Cancer SOS contained several components (the checklists, feedback of screening rates to clinic staff, etc), and our study is unable to assess the relative contribution of individual components.

In conclusion, we found that the Cancer SOS intervention significantly increased rates of cancer screening among primary care clinics serving disadvantaged populations. This finding suggests that office systems approaches may be one strategy to increasing cancer screening among underserved populations who are at greater risk of late-stage disease and poor cancer outcomes. Further study on the sustainability, cost-effectiveness, and limitations of this approach are warranted.