Long-term Results From a Randomized Controlled Trial to Increase Cancer Screening Among Attendees of Community Health Centers

Abstract

PURPOSE We assessed whether increased cancer screening rates that were observed with Cancer Screening Office Systems (Cancer SOS) could be maintained at 24 months’ follow-up, a period in which clinics were expected to be largely self-sufficient in maintaining the intervention.

METHODS Eight primary care clinics serving disadvantaged populations in Hills-borough County, Fla, agreed to take part in a cluster-randomized experimental trial. Charts of independent samples of established patients aged 50 to 75 years were abstracted, with data collected at baseline (n = 1,196) and at 24 months’ follow-up (n = 1,296). Papanicolaou (Pap) smears, mammography, and fecal occult blood testing were assessed.

RESULTS At 24 months of follow-up, intervention patients had received a greater number of cancer screening tests (mean 1.17 tests vs 0.94 tests, t test = 4.42, P <.0001). In multivariate analysis that controlled for baseline screening rates, secular trends, and other patient and clinic characteristics, the intervention increased the odds of mammograms slightly (odds ratio [OR]) = 1.26; 95% confidence interval [CI], 1.02–1.55; P = .03) but had no effect on fecal occult blood tests (OR = 1.17; 95% CI, 0.92–1.48; P =0.19) or Pap smears (OR = 0.88; 95% CI, 0.0.68–1.15; P = .34).

CONCLUSIONS The Cancer SOS intervention had persistent, although modest, effects on screening at 24 months’ follow-up, an effect that had clearly diminished from results reported at 12 months’ follow-up. Further study is needed to develop successful intervention strategies that are either self-sustaining or that are able to produce long-term changes in screening behavior.

INTRODUCTION

Patients belonging to a racial or ethnic minority are more likely to have poor cancer outcomes.^1–7 Patients of low socioeconomic status and those who are uninsured or insured by Medicaid are also at greater risk of poor cancer outcomes.^4,6,8–14 The reasons why these groups have less favorable cancer outcomes is not certain but has been attributed to lower rates of screening.^15–24

In an attempt to address these health disparities, we developed the Cancer Screening Office System (Cancer SOS), a low-cost office systems intervention to promote screening in primary care clinics serving disadvantaged populations. The intervention is not computerized and relies on personnel and resources that are available to most primary care clinics. The intervention attempted to change systematically the behaviors of all office staff (including physicians) to accomplish cancer screening and is modeled after similarly successful interventions.^25,26 We tested the efficacy of the Cancer SOS among patients attending community health centers, a representative setting of care for the target population.

The effectiveness of the intervention at 12 months’ follow-up has been previously reported.²⁷ In multivariate analysis that controlled for baseline screening rates, secular trends, and other patient and clinic characteristics, the intervention increased the odds of screening with mammograms (adjusted odds ratio [OR] = 1.62, P = .023) and fecal occult blood tests (OR = 2.5, P <.0001), with a trend toward greater use of Papanicolaou (Pap) smears (OR = 1.57, P = .096). Left unanswered, however, was whether the increased rates of screening that were observed could be sustained without ongoing support from research staff.

To assess whether the effectiveness of the intervention could be sustained, we assessed screening outcomes after an additional 12 months of follow-up, a period in which clinics were largely self-sufficient in regard to the intervention. During the first intervention year, office staff were taught how to maintain the intervention and how to train new employees in the intervention procedures. Training manuals were provided to all intervention clinics, and a person from each clinic (usually the office manager) assumed responsibility for maintaining the intervention structure. During the second follow-up year, intervention clinics were given all necessary supplies and materials but were expected to maintain the intervention themselves, without ongoing support from research staff. The durability of the intervention was assessed by chart reviews conducted at 24 months’ follow-up, the results of which are reported here.

METHODS

To target an underserved population, clinics were recruited from among 16 clinics participating in a county-funded health insurance plan in Hillsborough County, Fla. Clinics were eligible for the randomized trial if (1) they provided primary medical care 5 days a week, (2) the majority of physician and other clinicians agreed to participate, and (3) the clinic was expected to continue operating in the same fashion for the next 24 months. Each clinic individually decided whether to participate in the intervention, and none was obligated to join the study . We performed a cluster-randomized experimental trial in which 8 clinics meeting eligibility criteria were randomized to either intervention or control conditions. Together the clinics provided more than 3,000 patient visits per week.

The intervention targeted 3 cancer screening tests: mammograms, Pap smears, and fecal occult blood test (FOBT). Key components of the intervention included a cancer screening checklist that was completed by patients indicating whether patients were due for screening and a series of red, yellow, and green stickers that indicated whether recommended screening tests had been ordered and completed. Additional details about the about the clinics that participated and additional details about the intervention itself have been reported previously.²⁷

Data Collection

During data collection periods, research assistants assembled sampling frames of all patient visits using office billing and scheduling records. Patient’s records were eligible to be abstracted if both of the following criteria were met: (1) the patient was 50 to 75 years of age and (2) the patient was established in the clinic (defined as having had at least 1 visit 12 months or more before the sampled visit). Based on sample size requirements for the intervention (the intervention was structured to provide 80% power to detect increases in screening rates of 20% or more), independent random samples of 150 charts were selected for each clinic at baseline and again at 12 and 24 months after the intervention had been fully implemented in the clinic. Thus the samples were cross-sectional and did not assess the same patients during the 24-month period, as would occur in a cohort study. We also assessed compliance with the system at 6 months and again at 24 months by determining the percentage of intervention charts that contained a Cancer SOS checklist.

To prevent medical record reviews from influencing patient- or clinician-screening behavior, and to allow adequate time for recommended screening tests to be completed, we abstracted charts 3 months after study patients had visited the clinic. Neither patients nor staff were aware of the period during which chart abstractions would occur. For each of the targeted cancer screening tests, the date the procedure was completed was recorded to determine whether the patient was up-to-date on screening. We defined being up-to-date as having completed the targeted screening test during either the 12 months before the audited visit or the 3 months after the audited visit. The use of a grace period has been applied in other studies^25,28–32 and allows sufficient time for screening tests that were recommended at an audited visit to have been completed by the patient.

Chart abstracters used a standardized method and instrument to abstract chart information and were trained by the project manager. Relevant clinical data, including progress notes, laboratory reports, radiology reports, consultation letters, and hospital records, were abstracted from all sections of the chart. Before beginning data collection, inter-rater reliability for chart abstractors was assessed for the 3 cancer screening tests by a second review of a sample of 30 charts and calculating the κ statistic. The following values of κ were obtained when assessing whether a patient was up-to-date on cancer screening: Pap smear κ = 1.00, mammogram κ = 1.00, FOBT κ = 0.91.

We assessed whether the patient was up-to-date on one or more of the following cancer screening tests: mammogram, Pap smear, or FOBT. Women who had a personal history of breast cancer were excluded from our analysis of mammograms. Women with a personal history of cervical cancer or those who had had a hysterectomy were excluded from our analysis of Pap smear screening. Patients with a personal history of colon cancer and those who had received a colonoscopy or double-contrast barium enema in the previous 10 years were excluded from the analysis of FOBT. We defined being up-to-date on FOBT as having completed 3 home-collected specimens, not testing a stool specimen obtained in the physician’s office. For all screening outcomes, we considered only the actual completion of the screening test, not a physician’s recommendation or patient refusal to be screened. Finally, all outcomes were assessed strictly by chart review, not patient’s self-report of screening on their checklist.

Approximately 850 of the charts reviewed at baseline and at 24 months did not meet eligibility criteria. For this analysis, the final data set consisted of the combined abstracted records of eligible patients from the 2 independent samples collected at baseline (n = 1,196) and 24 months after the intervention (n = 1,296). To adjust simultaneously for potential confounders, we performed generalized linear models analysis using PROC GENMOD in SAS (SAS Version 8, SAS Institute Inc., Cary, NC). The following variables were included in regression models: age, sex, race-ethnicity, marital status, smoking status, health insurance, comorbidity (using the Charlson Comorbidity Index), number of chronic illnesses, number of prescribed medications, number of health care visits in the previous year, clinic attended, primary language spoken, family history of targeted cancers, and for women, estrogen replacement therapy and history of benign breast disease. Increasing scores on the Charlson Comorbidity Index reflect a greater burden of comorbid illness.^33,34

Indicator variables were also created for clinic type (control vs intervention) and for survey year (baseline, or 24-month follow-up). We also included an interaction term for the 2 variables of clinic type and survey year. The interaction term estimates the effect of the intervention controlling for any baseline screening differences and secular trends in screening rates while adjusting for other covariates. Because of the clustered nature of the data, we obtained parameter estimates and 95% confidence intervals (CIs) using the method of generalized estimating equations.^35,36 Finally, we repeated our analysis comparing patients who had a Cancer SOS screening checklist in their chart (indicating the intervention had actually been implemented for that patient) with those who did not, to assess the potential effectiveness of the intervention strategy when it was actually implemented. This study was approved by the University of South Florida Institutional Review Board.

RESULTS

The clinical characteristics of patients attending control clinics compared with those attending intervention clinics have been previously summarized.²⁷ Patients attending intervention clinics were more likely to be African American and married and were generally more ill, as evidenced by a greater number of comorbid illnesses, greater number of prescribed medications, and greater number of health care visits. Compliance with the Cancer SOS intervention decreased during the course of the intervention, with 334 of 615 (54.3%) charts showing evidence of the Cancer SOS cancer screening checklist at a 24-months follow-up, compared with 74% at the 6-month follow-up. Cancer screening rates at baseline, 12 months, and 24 months are reported in Table 1 ▶. For all 3 targeted tests, screening differences observed between control and intervention clinics were less prominent at the 24-month follow-up compared with the 12-month follow-up. We also assessed the total number of targeted screening tests that were up-to-date. At 24 months, patients in control clinics were, on average, up-to-date on 0.94 cancer screening tests compared with an average of 1.17 tests for patients in intervention clinics (t test = 4.42, P <.0001). Among women who were eligible for all 3 screening tests, the percentage completing screening tests at 24 months was as follows: for the control group, the mean number of tests was 1.22, 25.6% had 0 tests, 34.9% had 1 test, 31.0% had 2 tests, and 8.5% had all 3 tests; for the intervention group, mean number of tests was 1.37, 24.0% had 0 tests, 29.8% had 1 test, 31.5 had 2 tests, and 14.7% had all 3 tests.

Table 1.

Cancer Screening at Baseline, 12 Months’, and 24 Months’ Follow-up

	Percentage of Patients Up-To-Date
Variable	Baseline	12 Months	24 Months
Papanicolaou smears
Control	57.6	48.2	45.3
Intervention	61.9	62.4	47.3
Mammograms
Control	75.9	71.1	64.5
Intervention	71.4	75.7	67.0
Fecal occult blood tests
Total sample
Control	22.1	11.9	12.6
Intervention	35.9	40.1	28.2
Men
Control	22.5	12.2	11.0
Intervention	28.3	41.4	27.3
Women
Control	22.1	11.9	13.0
Intervention	38.1	39.7	28.5

Multivariate analysis was used to assess the effectiveness of the intervention on individual screening tests by determining the odds ratio for the interaction term previously described (Table 2 ▶). The intervention increased the odds of mammography but had no significant effect on the other 2 targeted screening tests. When results were repeated comparing patients having a Cancer SOS screening checklist with those who did not, presence of a checklist was associated with increased odds of Pap smear screening (OR = 2.03; 95% CI, 1.14–3.61; P = .014) and FOBT screening (OR = 3.28; 95% CI, 2.05–5.23; P <.0001), with a trend of increased odds of mammography screening (OR = 1.42; 95% CI, 0.98–2.07; P = .06). Finally, in a subgroup analysis, we found no significant difference in the effects of the Cancer SOS intervention among patients having a health maintenance visit (data not shown).

Table 2.

Results of Multivariate Analyses on Intervention Effects

Variable	Odds Ratio	95% CI	PValue
Papanicolaou smears (n = 1,103)	0.88	0.68–1.15	.34
Mammograms (n = 1,844)	1.26	1.02–1.55	.03
FOBT (n = 2,009)	1.17	0.92–1.48	.19
Men	1.33	0.78–2.27	.30
Women	1.13	0.87–1.46	.37

CI = confidence interval; FOBT = fecal occult blood test.

DISCUSSION

The Cancer SOS intervention had persistent, although modest, effects on screening at 24 months’ follow-up. Patients who received care at intervention clinics were up-to-date on a greater number of targeted screening tests at 24 months’ follow-up and had 26% greater odds of mammography screening compared with patients in control clinics. Although still apparent, intervention effects on screening at 24 months’ follow-up had clearly diminished relative to effects observed at 12 months follow-up.

Some evidence suggests that the decline in effectiveness of the intervention with time was more the result of decreased compliance with the system than then a system that became ineffective with time. For example, compliance with the Cancer SOS system, as determined by the presence of a Cancer SOS screening checklist in the patient’s chart, declined from about 75% at 6 months’ follow-up to just more than 50% at the 24-month mark. Furthermore, patients having a checklist in the chart had increased odds of screening relative to patients not receiving the intervention.

Although we were unable to assess rigorously the reasons for diminished compliance with the Cancer SOS system, there are several explanations. First, all intervention clinics had some degree of staff turnover during the course of the study. At site visits for data collection, for example, it was not uncommon to find newly hired personnel who were not familiar with the Cancer SOS system. Although the Cancer SOS intervention had provisions for training new staff (training manuals, assigned responsibilities for training, etc), it appeared that training did not always occur in practice. In addition, the enthusiasm with which a new program is implemented would likely diminish as its novelty wanes. Although the Cancer SOS system was purposely designed for ease of implementation and maintenance, it would appear that even such relatively simple interventions as Cancer SOS may require ongoing support from an outside entity to insure full implementation.

Few studies have assessed the durability of office systems interventions. Dietrich and colleagues found that intervention effects on FOBT and self-breast examination persisted 24 months after an office systems intervention was completed but that effects declined for mammography and clinical breast examination.³¹ Margolis and colleagues reported sustained improvements in preventive care for children at 30 months of follow-up with an office systems intervention.³⁷ Stange and colleagues also found that improvements in preventive care among intervention clinics were maintained at 24 months’ follow-up.³⁸ Goodson and colleagues reported some success in the ablility of the clinic to maintain an office intervention even up to 6 years after implementation; effects on screening rates were not reported, however.³⁹ While not specifically assessing ongoing effectiveness, other studies have assessed primary outcomes of office systems interventions at a point greater than 12 months after the intervention was implemented. Results were less encouraging in these studies.^26,40

Although a great number of interventions have found significant effects on screening rates in the short term,^41,42 few have proven long-term durability. It is not known whether office systems interventions have better durability than other approaches to improve cancer screening. Improved long-term outcomes (beyond 12 months) have been reported in studies of tailored telephone counseling, for example.^43,44 Other studies reporting primary outcomes more than 12 months after intervention implementation were not as successful.^45–51

In conclusion, the Cancer SOS intervention had persistent, although modest, effects on screening at 24 months’ follow-up. Effectiveness of the intervention strategy had clearly diminished from results reported at 12 months’ follow-up, most likely because of diminished compliance with the system. Further study is needed to develop successful intervention strategies that are either self-sustaining or that are able to produce long-term changes in screening behavior. Cancer SOS materials and methods are freely available for use at the following Web site (http://www.hsc.usf.edu/FAMILY/research/index.htm).