Abstract
The objective of the study was to determine the extent to which the FLU-FOBT Program, a colorectal cancer screening (CRCS) intervention linking the provision of fecal occult blood tests (FOBT) to the time of annual influenza vaccination, resulted in practice changes in six primary care clinics 1 year after it was introduced in a randomized controlled trial (RCT). We assessed CRCS rate changes for influenza vaccine recipients, administered brief serial clinic staff surveys and interviewed clinic leaders 1 year after the RCT. CRCS rates for influenza vaccination recipients between the ages of 50 and 75 years were 42.5% before the RCT, 54.5% immediately after the RCT and 55.8% 1 year after the RCT (P < 0.001 for difference between baseline and 1 year after RCT). Many FLU-FOBT Program components were maintained in most clinics at 1-year follow-up. Only 63% of clinic staff survey respondents (26 of 41) continued offering FOBT with influenza vaccines, but 85% (35 of 41) continued to provide mailing kits with FOBT. Many patient education materials were maintained and staff satisfaction with the intervention remained high. Clinic leaders acknowledged barriers to maintenance but also observed several beneficial practice changes. Many components of the FLU-FOBT Program were maintained, with beneficial outcomes for participating practices.
Introduction
Colorectal cancer is the second leading cause of cancer deaths in the United States, and mortality from colorectal cancer can be reduced with screening [1]. The US Preventive Services Task Force recommends colorectal cancer screening (CRCS) for adults between the ages of 50 and 75 [2]. CRCS tests recommended for this age group include home fecal occult blood tests (FOBT) annually, flexible sigmoidoscopy every 5 years, or colonoscopy every 10 years [2]. CRCS rates, although improving, remain suboptimal, with recent statistics from the Behavioral Risk Factor Surveillance System surveys showing that in 2010 only 65.5% of individuals aged 50–75 years were up-to-date with any of these three tests [3]. Individuals who are less educated, uninsured, members of ethnic minorities or are foreign-born have even lower CRCS rates [3]. FOBT is often the preferred CRCS option in clinical settings that serve these medically vulnerable populations, because it is readily available, inexpensive and similarly effective to endoscopy when completed yearly with appropriate follow-up [4].
The FLU-FOBT Program, a multi-component primary care intervention pairs the offering of FOBT with yearly influenza vaccine activities [5–7]. We recently completed a randomized controlled trial (RCT) demonstrating the effectiveness of the FLU-FOBT Program in six primary care clinics serving medically vulnerable patient populations in San Francisco [5]. The CRCS rate for participants in the FLU-FOBT group increased by 13.0 percentage points (from 32.5% to 45.5%), compared with an increase of 4.3 percentage points (from 31.3 to 35.6%) for patients who received influenza vaccines without the FLU-FOBT Program (P = 0.018). This new study sought to assess the lasting effects of study participation on these primary care clinics, especially the extent to which components of the FLU-FOBT Program were adopted, implemented and maintained 1 year after completion of the RCT.
Methods
Description of the RCT and the FLU-FOBT intervention
Details and results of the RCT are published elsewhere [5]. The RCT was conducted at six San Francisco Department of Public Health primary care clinics, located in diverse neighborhoods throughout the city. The clinics were randomly allocated to ‘intervention weeks’ during which they performed the FLU-FOBT activities and ‘control weeks’ when they provided influenza vaccination as usual. The FLU-FOBT intervention involved several components. These components included: training for non-physician clinic staff (mostly medical assistants) to identify patients who were due for FOBT; training in how to offer FOBT to eligible patients; standing orders to provide FOBT to eligible patients whenever an annual influenza vaccine was being offered; use of a FLU-FOBT log to record influenza vaccines; FOBT patient education materials to explain why FOBT is important and how to complete it (including visual aids, low literacy multi-language FOBT instructions, and a video that could be shown in the clinic) and stamped mailing envelopes allowing patients to send completed FOBT kits directly to the clinical laboratory. Clinics were encouraged to use the FLU-FOBT logs as a tool to assure test completion and follow-up of abnormal results, but this was not a required part of the study protocol. The primary RCT outcome was the CRCS rate change for influenza vaccine recipients within the control and the intervention groups, from the beginning of the influenza vaccination season in September 2009, until March 2010.
CRCS rate measurements for influenza vaccination recipients
In the current study, we measured CRCS rates for influenza vaccination recipients at three time points to assess CRCS trends over time: 31 March 2009 (prior to the RCT); 31 March 2010 (at the conclusion of the RCT) and 31 March 2011 (1 year later). At each time point, the CRCS rate was calculated for the group of clinic patients aged 50–75 years, with at least one clinic visit in the prior 2 years and an influenza vaccination in the last year. Individuals were included in the analyses regardless of whether or not they were included in the RCT. These patient-level data were obtained retrospectively from de-identified electronic medical records and patient registration data maintained by the San Francisco Department of Public Health and prepared by the Clinical and Translational Sciences Institute at the University of California, San Francisco.
Surveys of clinic staff
Non-physician clinic staff involved in FLU-FOBT activities completed brief written surveys just before the RCT began in September 2009; at the conclusion of the RCT in January 2010; and again in September and January during the year following the RCT. The surveys included questions about staff knowledge, attitudes and practices related to influenza vaccination, CRCS and the FLU-FOBT Program. For this study, we were interested in the extent to which the clinic staff implemented various FLU-FOBT Program components during the RCT and in the extent to which they maintained them in the following year, so we compared survey responses in January 2010 with those in January 2011.
Interviews with clinic leaders
During April and May 2011, clinic leaders (the medical director and nurse manager) at each primary care site participated in a 1-hour interview with two study investigators (J.J. and J.W.). The goal of the interviews was to understand the impact of the FLU-FOBT intervention from the perspective of clinic leaders, 1 year after the conclusion of the RCT. The interview guide was structured around the RE-AIM framework, which evaluates the robustness of clinical interventions through an assessment of their reach, effectiveness, adoption, implementation and maintenance [8].
Data analyses
CRCS rates
The number and proportion of patients who were up-to-date with CRCS was calculated for the six clinics individually and combined during the three points in time. Chi-square tests were used to compare the CRCS rates between each set of two different time points.
Clinic staff surveys
Cross-sectional analyses of the survey data were completed for each of the time points. First, we calculated descriptive statistics, including means and standard deviations for all continuous variables and percentages for categorical data. In some cases, examination of the categorical items prompted collapsing of the categories prior to comparing the survey data between the two time points. We assessed changes in responses to each survey item over time using McNemar’s test for categorical variables to account for the correlations between the two time points.
Clinic leader interviews
The interviews were transcribed and transcripts reviewed for accuracy by three authors (J.W., J.J. and M.P.). Common themes were identified by consensus and organized within the context of the RE-AIM framework.
A statistical significance level of 0.05 was used for all statistical tests. Data were analyzed using SAS version 9.2 (Cary, North Carolina). The study was approved by the UCSF Committee on Human Research (Clinical Trial Registration Number NCT01211379).
Results
CRCS rate changes
CRCS rates for influenza vaccination recipients between the ages of 50 and 75 years at each clinic for three time points, 1 year apart, are presented in Table I. These time points correspond to baseline, the conclusion of the RCT, and 1 year later. Overall, the number of influenza vaccination recipients in this age group increased over time. The CRCS rate among all influenza vaccination recipients aged 50 to 75 years (whether or not they were included in the RCT) increased from 42.5% at baseline to 54.5% at the conclusion of the RCT, and was 55.8% 1 year after the conclusion of the RCT. Statistically significant CRCS rate increases were observed in four of the six sites both from baseline to the end of the RCT and from baseline until 1 year after the conclusion of the RCT. Between the end of the RCT and the following year, two of these four sites experienced further significant CRCS rate improvements, whereas one site had no significant additional improvement, and one other site had a significant decline in its CRCS rate. At two of the clinic sites, there was no significant increase either from baseline to the end of the RCT or from baseline until 1 year after the conclusion of the RCT.
Table I.
Number and proportion of influenza vaccination recipients in the prior 12 months who were also were also up-to-date with colorectal cancer screening
| March 2009 baseline (n = 3260) | March 2010 post-RCT (n = 3634) | March 2011 1 year after RCT (n = 4333) | P-value baseline versus post-RCT | P-value baseline versus 1 year after RCT | P-value post-RCT versus 1 year after RCT | |
|---|---|---|---|---|---|---|
| Clinic 1 | 281 (51.2%) | 259 (51.1%) | 320 (53.6%) | 1.00 | 0.44 | 0.43 |
| Clinic 2 | 233 (41.1%) | 242 (42.5%) | 232 (42.8%) | 0.63 | 0.58 | 0.95 |
| Clinic 3 | 317 (56.5%) | 529 (71.6%) | 769 (75.9%) | <0.01 | <0.01 | 0.05 |
| Clinic 4 | 159 (45.2%) | 367 (76.9%) | 308 (61.9%) | <0.01 | <0.01 | <0.01 |
| Clinic 5 | 199 (43.5%) | 343 (61.6%) | 524 (68.1%) | <0.01 | <0.01 | 0.02 |
| Clinic 6 | 196 (25.3%) | 242 (30.8%) | 287 (31.4%) | 0.02 | .01 | 0.79 |
| Total | 1385 (42.5%) | 1982 (54.5%) | 2440 (55.8%) | <0.01 | <0.01 | 0.11 |
Up-to-date with colorectal cancer screening defined as having a fecal occult blood test in the past 12 months, flexible sigmoidoscopy in the last 5 years or colonoscopy in the past 10 years. P-values based on chi-square tests.
Clinic staff surveys: adoption, implementation and maintenance 1 year after the RCT
A total of 58 staff members were recruited to participate, and 42 (72%) completed all four surveys administered between September 2009 and January 2011. Failure to complete all four surveys was mostly attributable to staff turnover that occurred during the time of the study. Staff members from all six clinics participated. In Table II, we present clinic staff responses to survey questions completed during the RCT (January 2010) and compare them with responses to identical survey questions provided 1 year later (January 2011). One respondent missed several questions in this section, resulting in 41 matched pairs for analysis. During the RCT and 1 year later, more than three quarters of respondents reported using the clinic paper chart or electronic record to determine which patients were due for FOBT. Before the RCT began, none of the clinic staff routinely offered FOBT with influenza vaccines, but during the RCT 95% of respondents reported offering FOBT with influenza vaccines during the RCT intervention weeks. A total of 63% were still offering FOBT with influenza vaccines routinely in the year after the RCT (P < 0.01). Most respondents reported using the patient education materials introduced by the study during the RCT and in the year after. Those who reported using the FOBT instructional video with patients increased from 9.5% during the study to 51.2% in the year after the RCT (P < 0.01). Clinic staff gave the study-provided mailing kits to patients at high rates before and after the RCT. Slightly fewer than half of respondents reported using the FLU-FOBT log during and after the intervention. Staff reported more involvement over time in FOBT test follow-up, but this was not statistically significant.
Table II.
Clinic Staff Survey responses about FLU-FOBT Program activities performed during the RCT (January 2010) and 1 year later (January 2011)
| During RCT N (%) (January 2010) | 1 year post-RCT N (%) (January 2011) | P-value | |
|---|---|---|---|
| Personally checked electronic record or paper chart to determine which patients are due for FOBT | 33 (80.5%) | 32 (78.1%) | 0.76 |
| Personally offered FOBT to eligible patients with influenza vaccinations | 39 (95.1%) | 26 (63.4%) | <0.01 |
| Personally explained to patients why FOBT is important | 31 (75.6%) | 36 (87.8%) | 0.10 |
| Personally explained to patients how to do FOBT | 36 (87.8%) | 36 (87.8%) | 1.00 |
| Used low literacy multilingual instructions | 21 (51.2%) | 24 (58.5%) | 0.47 |
| Used visual aids for FOBT instructions | 27 (65.9%) | 26 (63.4%) | 0.81 |
| Used multilingual FOBT video instructions | 4 (9.8%) | 21 (51.2%) | <0.01 |
| Provided mailing kits with FOBT | 32 (78.1%) | 35 (85.4%) | 0.18 |
| Kept a log of which patients were provided with FOBT | 19 (46.3%) | 18 (43.9%) | 0.74 |
| Followed-up on FOBT not completed | 9 (22.0%) | 13 (31.7%) | 0.10 |
| Followed-up on abnormal FOBT | 2 (4.9%) | 7 (17.1%) | 0.06 |
Total N = 41 who responded to both surveys. P-values based on McNemar’s chi-square tests.
Clinic staff surveys: attitudes toward the FLU-FOBT Program
Clinic staff attitudes toward FOBT and the FLU-FOBT Program are presented in Table III. During and after the RCT, nearly all clinic staff respondents stated that FOBT was at least as important as influenza vaccination, that clinic staff should be involved in offering FOBT, that the FLU-FOBT Program was a worthwhile activity, and that participation in the study had resulted in improvements in their clinic systems for offering FOBT. Interestingly, only 46.3% of respondents agreed that the clinic had sufficient time to implement the FLU-FOBT Program, and this increased marginally to 56.1% in the year after the RCT (P = 0.10). Perhaps related to this, only 66.7% were satisfied with the way the FLU-FOBT program worked during the RCT, whereas 83.3% were satisfied with the FLU-FOBT Program as adopted by their clinic in the year after the RCT (P = 0.05), perhaps reflecting the greater comfort when the rigors of the RCT were not in place.
Table III.
Clinic Staff Survey responses relating to attitudes and satisfaction with the FLU-FOBT Program during the RCT (January 2010) and 1 year later (January 2011)
| During RCT N (%) (January 2010) | 1 year post-RCT N (%) (January 2011) | P-value | |
|---|---|---|---|
| Believe that FOBT is as important as or more important than influenza vaccination | 39 (95.1%) | 39 (95.1%) | 1.00 |
| Agree or strongly agree that FLU-FOBT Program is a worthwhile activity | 37 (90.2%) | 36 (87.8%) | 0.66 |
| Believe that FLU-FOBT Program (somewhat or much) improved the way FOBT is offered in their clinic | 37 (88.1%) | 38 (90.5%) | 0.71 |
| Satisfied or very satisfied with system to offer FOBT with FLU | 28 (66.7%) | 35 (83.3%) | 0.05 |
| Satisfied or very satisfied with patient education materials provided by the FLU-FOBT Program | 34 (85.0%) | 35 (87.5%) | 0.66 |
| Agree or strongly agree that the clinic has sufficient staff time to implement FLU-FOBT Program | 19 (46.3%) | 23 (56.1%) | 0.10 |
| Agree or strongly agree that offering FOBT should be a nursing responsibility or a shared responsibility (not solely the doctor’s responsibility) | 38 (97.4%) | 36 (92.3%) | 0.16 |
Total N = 42. P-value based on McNemar’s chi-square tests.
In the final survey, completed 1 year after the RCT, we asked clinic staff members to comment on the sustainability of the FLU-FOBT Program over the long term. Of 40 respondents to this question, 16 (40%) felt it was sustainable with no changes, 14 (35%) with minor changes, 7 (17.5%) with major changes and just 3 (7.5%) felt it was unsustainable. We asked staff to comment on challenges associated with the FLU-FOBT program, what they learned and what they thought their patients got out of the FLU-FOBT program. Time was described as an important challenge—‘just finding time to enter flu shots into the record can be challenging, and adding FOBT was hard’. Staff described that they learned to make FLU-FOBT ‘part of our clinic visit routine’. One clinic staff member wrote, ‘When I give a flu shot in the future, I will remember to give FOBT’, and another stated, ‘It is in our memory to offer FOBT.’ As a patient benefit, one staff member wrote, ‘Patients begin to relate that when they get their flu shot, they will get the FOBT given to them.’
Clinic leadership interviews—a RE-AIM perspective
One year after the RCT, the evaluation team (J.W. and J.J.) returned to each clinic and interviewed the medical director and nurse manager of each clinic together, focusing on their experience with the FLU-FOBT Program during and in the year following the RCT. The transcripts were analyzed using the RE-AIM framework, with key findings presented below.
Reach
Most clinic leaders felt that the FLU-FOBT project was able to reach many patients who came in for visits during the influenza vaccination season, but they were concerned about not reaching those patients who did not have visits during the time frame: As stated by a clinic medical director, ‘The unfortunate thing is, for patients not coming in, we couldn’t reach them.’ At all sites, however, and especially in sites with the lowest baseline CRCS rates, there was appreciation of the FLU-FOBT Program as a mechanism to begin to prioritize CRCS and reach a substantial proportion of patients with a focused, seasonal campaign.
Effectiveness
All interviewees found the FLU-FOBT Program to be effective, especially in the locations where the number of FOBT kits completed increased significantly. Even in the location where the number of influenza vaccination recipients who became up to date with CRCS rate did not increase, the clinic leaders felt that the program’s effectiveness could be measured in better processes for providing FOBT and greater awareness of the need to address CRCS for their clinic, which they felt would eventually lead to improvements. One medical director stated, ‘It got our staff thinking more about health care maintenance.’ There was consensus that the FLU-FOBT Program does not have to reach every eligible patient to have a positive effect on patient CRCS outcomes or clinic practices. Many had ideas about how to make the FLU-FOBT intervention even more effective, such as a more organized space to offer the kits and more efficient processes for the clinic staff. Some clinics were beginning to act on these ideas.
Adoption
Most clinic leaders indicated that they had autonomy to decide on whether or not to adopt practice changes. At baseline, most of the clinics had policies requiring clinic staff to provide influenza vaccines to patients without an order from the primary care provider. In contrast, none had standing orders for clinic staff to provide FOBT to eligible patients. The clinic leaders cited barriers to adopting the FLU-FOBT Program, such as limited staff time, space, reluctance of some clinic staff to adopt new roles and the initial burden of participating in an RCT that required staff to shift up and back between intervention and control weeks during the RCT: One medical director said, ‘Staffing issues and space are biggest [issues].’ Although they described these barriers, they also acknowledged that the program had been effective in other clinical settings, and many expressed a motivation to adopt something new to address an important preventive health issue.
Implementation
Clinic leaders all said they did the best they could to implement the FLU-FOBT Program, but limited staff time and the competing demands of clinical care were cited as reasons for not providing the FLU-FOBT intervention with every patient. For example, in clinics serving high acuity patients, staff often put preventive health issues such as influenza vaccines and CRCS lower on the priority list, and sometimes would forego offering them. However, they also said that their clinic staff learned how to assess CRCS eligibility, provide FOBT independently, use the patient education materials introduced by the study, providing appropriate instructions for test completion and return of kits to the laboratory using the new mailing envelope that was provided by the study. They described the process of implementation as a ‘jump start’ for CRCS, empowering clinic staff members, especially medical assistants, to take the initiative to offer FOBT and to better support patients to complete it when offered. One nurse manager said, ‘it was a big change, because we used to wait for the MD to offer [FOBT]’.
Maintenance
All clinic leaders listed some components of the FLU-FOBT Program that they maintained in the year after the RCT. Four out of the six clinics continued to emphasize the offering of FOBT with influenza vaccination as a special clinic project, whereas others simply set an expectation that clinic staff would be more involved in providing FOBT on a routine, year-round basis. In all clinics, clinic staff continued to be involved in assessing eligibility for FOBT and providing it to patients when indicated, using the educational materials provided by the study, and providing mailing envelopes. The system for mailing completed FOBT kits into the lab remained intact after the study, and many clinic leaders reported working hard to continue to find ways to finance the FOBT mailing kits with postage paid return envelopes. Some clinics introduced their own adaptations during the maintenance year. For example, one clinic developed a health maintenance checklist for clinic staff to use and act on independently at every visit, which included CRCS as well as other preventive services, including mammography and other immunizations. A medical director stated, ‘Now there is an agenda set for each patient before his/her visit where they (MAs) check a lot of preventive activities and see whether or not they are up to date.’ In the year after the RCT, clinic leaders felt that maintenance was facilitated by staff familiarity with the FLU-FOBT Program, their level of perceived success with the intervention, and the freedom to change and adapt it to suit locally defined needs without being required to adhere to a specifically defined research protocol.
Discussion
Evidence-based interventions often fail the test of translation into clinical practice [9, 10].
Our study of the FLU-FOBT Program provides an example of how an evidence-based intervention, developed through an iterative process involving the participation of the end users of the intervention, can be rigorously tested and evaluated, with lasting benefits for participating clinics. The FLU-FOBT Program was first adapted for primary care office visits [6], shown effective in an RCT [5] and then, in this present study, shown to have benefits for participating clinics that lasted for at least 1 year after completion of the RCT.
During the RCT, the proportion of influenza vaccination recipients who completed FOBT increased substantially and remained increased above baseline in the following year. Clinic staff and clinic leaders reported behavior and systems changes that they felt would support higher CRCS rates for their clinics into the future, although also described challenges to maintaining the FLU-FOBT program long term.
Adoption, implementation and maintenance of specific components of the FLU-FOBT Program specific component in the year after the RCT varied from clinic to clinic. While screening rates increased in several of the clinics, they did not increase in all clinics. While we do not know the exact reasons that not all clinics were successful in increasing and maintaining screening rates, barriers most commonly cited by clinic leaders were inadequate time, not enough space and staffing issues, including having enough available staff and having staff committed to the program.
The FLU-FOBT Program was implemented as a multi-component package, and it is not possible to separate out the effect of individual components of the program. The RCT was designed to allow considerable adaptation of the intervention by clinic staff, and this adaptation continued in the year after the RCT was completed. An important question is to what extent the success of the intervention can be attributed to the coupling of FOBT with influenza vaccinations, the empowerment of the clinic staff members to identify eligible patients and offer FOBT without a physician order, and/or to the use of a more organized way of providing FOBT to eligible patients. It is possible that the FLU-FOBT RCT became a more general means to an end, providing a simple and focused way for clinic staff to get involved in screening and for clinical teams to begin to work together on a whole range of CRCS issues.
Our study has several limitations. First, CRCS screening rates among influenza vaccination recipients may be subject to many external influences, such as fluctuations in the availability of influenza vaccination and changes in patient populations served in these clinics over time, or other unrelated quality improvement activities. However, the fact that CRCS screening rates remained higher than baseline, even 1 year after the RCT was completed, suggests that our intervention had a durable impact. Second, our intervention only targets individuals who come in for influenza vaccines, and these individuals may be different from those who do not come in for influenza vaccines. Third, our data on practice changes relied on self-report, and could have been biased. However, staff survey respondents were not identified individually and participants had anonymity to respond honestly and critically to all the questions. The interviews with nurse managers and medical directors were conducted by an investigator less known to them, rather than by the study principal investigator, so that respondents would feel able to speak freely about their experiences and future plans. The results of the surveys and interviews are largely corroborated by the increased number of influenza vaccines being given and FOBT kits being completed by primary care patients in these clinics. Finally, we evaluated the adoption, implementation and maintenance of the FLU-FOBT Program just for 1 year after the RCT, which may not be enough to be certain of the lasting impact of the intervention. On the other hand, we wanted to capture feedback from staff while they still had vivid memories of participating in the RCT. We feel therefore that the post-intervention evaluation was timed as well as possible to get accurate and relevant information from clinic leader and clinic staff participants.
In summary, many components of the FLU-FOBT Program were maintained after the trial was completed, and the evidence suggests that primary care clinic participation in the FLU-FOBT Program RCT resulted in continued improvements in CRCS processes and outcomes at least a year beyond their introduction. Future research should address the extent to which fidelity to and adaptation of individual FLU-FOBT Program components contribute to these improved outcomes and to their long term sustainability.
Funding
R18 grant from the Centers for Disease Control and Prevention [Grant Number: 5R18DP001566-03] A portion of Dr. Potter’s time was supported by the National Center for Research Resources and the National Center for Advancing Translational Sciences, National Institutes of Health, through UCSF-CTSI Grant Number UL1 RR024131.
Conflict of interest statement
None declared.
Acknowledgements
The Authors would like to acknowledge Dr Stephen McPhee for his role in the initial study design, Dr Russ Glasgow for his contributions to developing the approach to evaluation and the San Francisco Department of Public Health primary care clinics for their enthusiastic participation in the FLU-FOBT Program.
References
- 1.Whitlock EP, Lin JS, Liles E, et al. Screening for colorectal cancer: a targeted, updated systematic review for the U.S. Preventive Services Task Force. Ann Intern Med. 2008;149:638–58. doi: 10.7326/0003-4819-149-9-200811040-00245. [DOI] [PubMed] [Google Scholar]
- 2.U.S. Preventive Services Task Force. Screening for colorectal cancer: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med. 2008;149:627–37. doi: 10.7326/0003-4819-149-9-200811040-00243. [DOI] [PubMed] [Google Scholar]
- 3.Vital signs: colorectal cancer screening, incidence, and mortality – United States, 2002-2010. Morbidity and Mortality Weekly Report (MMWR) 8 July 2011. Available at: http://www.cdc.gov/mmwr. Accessed 16 May 2012. [PubMed] [Google Scholar]
- 4.Zauber AG, Lansdorp-Vogelaar I, Knudsen AB, et al. Evaluating test strategies for colorectal cancer screening: a decision analysis for the U.S. Preventive Services Task Force. Ann Intern Med. 2008;149:659–69. doi: 10.7326/0003-4819-149-9-200811040-00244. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Potter MB, Walsh JME, Yu TM, et al. The effectiveness of the FLU-FOBT program in primary care a randomized trial. Am J Prev Med. 2011;41:9–16. doi: 10.1016/j.amepre.2011.03.011. [DOI] [PubMed] [Google Scholar]
- 6.Potter MB, Yu TM, Gildengorin G, et al. Adaptation of the FLU-FOBT Program for a primary care clinic serving a low-income Chinese American community: new evidence of effectiveness. J Health Care Poor Underserved. 2011;22:284–95. doi: 10.1353/hpu.2011.0030. [DOI] [PubMed] [Google Scholar]
- 7.Potter MB, Phengrasamy L, Hudes ES, et al. Offering annual fecal occult blood tests at annual flu shot clinics increases colorectal cancer screening rates. Ann Fam Med. 2009;7:17–23. doi: 10.1370/afm.934. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89:1322–7. doi: 10.2105/ajph.89.9.1322. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Green LW, Glasgow RE, Atkins D, et al. Making evidence from research more relevant, useful, and actionable in policy, program planning, and practice slips ‘twixt cup and lip’. Am J Prev Med. 2009;37(6 Suppl 1):S187–191. doi: 10.1016/j.amepre.2009.08.017. [DOI] [PubMed] [Google Scholar]
- 10.Pasick RJ, Hiatt RA, Paskett ED. Lessons learned from community-based cancer screening intervention research. Cancer. 2004;101(5 Suppl):1146–64. doi: 10.1002/cncr.20508. [DOI] [PubMed] [Google Scholar]