Use of Evidence-Based Practices and Resources Among Comprehensive Cancer Control Programs

Abstract

Context

While efforts to promote use of evidence-based practices (EBPs) for cancer control have increased, questions remain whether this will result in widespread adoption of EBPs (eg, Guide to Community Preventive Services interventions) by comprehensive cancer control (CCC) programs.

Objective

To examine use of EBPs among CCC programs to develop cancer control plans and select interventions.

Design

Conducted Web-based surveys of and telephone interviews with CCC program staff between March and July 2012.

Setting

CCC programs funded by the Centers for Disease Control and Prevention’s National Comprehensive Cancer Control Program (NCCCP).

Participants

Sixty-one CCC program directors.

Main Outcome Measures

1) Use of and knowledge/attitudes about EBPs and related resources and 2) EBP-related technical assistance needs.

Results

Seventy-five percent of eligible program directors reported use of EBPs to a moderate or great extent to address program objectives. Benefits of using EBPS included their effectiveness has been proven, they are an efficient use of resources, and they lend credibility to an intervention. Challenges to using EBPs included resource limitations, lack of culturally appropriate interventions, and limited skills adapting EBPs for local use. Most respondents had heard of and used Web sites for The Guide to Community Preventive Services (95% and 91%, respectively) and Cancer Control P.L.A.N.E.T. (98% and 75%, respectively). Training needs included how to adapt an EBP and its materials for cultural appropriateness (state 78%, tribe 86%, territory 80%) and how to maintain the fidelity of an EBP (state 75%, tribe 86%, territory 60%).

Conclusions

While awareness, knowledge, and use of EBPs and related resources are high, respondents identified numerous challenges and training needs. The findings from this study may be used to enhance technical assistance provided to NCCCP grantees related to selecting and implementing EBPs.

There have been increasing calls in the fields of public health generally^1–4 and in cancer control specifically^5–9 for the dissemination, adoption, and implementation of evidence-based practices (EBPs). Promising practices designed to address public health priorities are increasingly subjected to comprehensive, systematic review processes by independent third parties (eg, the Task Force on Community Preventive Services, US Preventive Services Task Force, the Cochrane Collaboration) that determine the strength of the evidence for effectiveness and make recommendations for widespread utilization via publications and Web sites affiliated with the review bodies.^10–14 While the development, review, and compilation of EBPs have steadily increased over time, there is concern that the adoption and implementation of those practices, including among cancer control planners and practitioners, have not kept pace.^{2–5,7,8,15–17} A survey conducted in 2008 by Hannon and colleagues¹⁵ among cancer control planners in the Cancer Prevention and Control Research Network found that 65% of respondents had ever heard of EBP resources and less than half of respondents (48%) had ever used the resources. Furthermore, recent reviews of comprehensive cancer control (CCC) plans for states, tribes, and US Associated Pacific Islands (USAPI) and territories have found that EBP resources are not consistently cited as the basis for addressing goals and objectives.^17,18

Given the gap between the development/identification of EBPs and their use, public health and cancer control organizations need to place greater emphasis on the dissemination and adoption of these practices among those who can use them to improve population health.^{2,3,5,8,9,16,19} The Centers for Disease Control and Prevention’s (CDC’s) National Comprehensive Cancer Control Program (NCCCP) has promoted the use of EBPs for CCC among states, tribes, and USAPI/territories. Grantees of the NCCCP are strongly encouraged to use data and research to define the cancer burdens in their jurisdictions, set priorities, and develop goals, objectives, and strategies to ensure that their CCC plans are evidence-based and defensible.^20–22

While efforts to promote evidence-based cancer control practices have increased, questions remain whether these efforts will result in widespread adoption and implementation of these practices in the context of CCC. Potential barriers to the utilization of cancer control EBPs include those related to the intervention characteristics (availability, cost, ease of use), characteristics of cancer planners and practitioners (awareness and knowledge of EBP resources, training and skills, attitudes toward EBPs and related resources), and organizational characteristics (resources, leadership, commitment to EBPs).^8,15,23 In addition, the broad scope of CCC, which includes interventions across the spectrum of the cancer control continuum and in a variety of environments (public health, primary care, oncology), increases the complexity of adopting and using EBPs.^3,5 Thus, NCCCP grantees may face a number of challenges to incorporating EBPs into cancer control efforts in their jurisdictions.

The purpose of this study was to examine CCC planners’ (directors of NCCCP-funded programs and their key partners) use of scientific and practice-based information to inform development of cancer control plans and to select evidence-based interventions. Related to this purpose, the key study questions for this project are the following:

1. How do NCCCP planners use scientific and practice-based evidence to develop and implement their CCC plans?

2. What evidence-based resources do they use?

3. What technical assistance needs do they have related to selection and implementation of evidence-based interventions?

We answered the first study question by examining the extent to which EBPs are used by CCC planners to address cancer control objectives and priorities, the perceived benefits and challenges of using EBPs, and the roles played by partners in selecting and implementing EBPs. The second study question was addressed by asking about awareness and use of, as well as attitudes toward, 4 evidence-based resources: The Guide to Community Preventive Services (The Community Guide),¹² Cancer Control P.L.A.N.E.T.,¹⁰ the National Cancer Institute’s (NCI’s) Research-tested Intervention Programs (RTIPs),¹¹ and NCI’s Using What Works: Adapting Evidence-Based Programs to Fit Your Needs.²⁴ We also examined which characteristics of EBPs are important for selection for implementation and assessed CCC planners’ perceptions about maintaining the fidelity of interventions when EBPs are adapted for local use. To answer the third study question, we asked about the types of technical assistance and training CCC planners would like to receive. In addition to data collected to answer study questions, we assessed respondent and organizational characteristics.

In this study, we define EBPs as public health practices (interventions, programs, strategies, policies, procedures, processes, and/or activities) that have been tested or evaluated and shown to be effective.^2,8,25 A review of NCCCP action plans allowed us to inventory the types of practices being used to address program objectives and the sources used to justify those practices; however, it could not provide deeper insight into how the CCC programs are using EBPs and what could be done to facilitate the dissemination of EBPs for cancer control. Thus, to better understand how the NCCCP-funded programs are using EBPs in their respective jurisdictions, we collected data from directors of the NCCCP-funded programs (or their designees) and from program partners/collaborators identified by the program directors as instrumental to the selection and implementation of cancer control EBPs. This report is limited to findings for program directors.

Methods

Sample

We conducted 2 cross-sectional surveys that targeted directors of all CCC programs funded by CDC’s NCCCP and a convenience sample of program partners they referred. Sixty-five CCC programs receive NCCCP funding from the CDC, including all 50 states, the District of Columbia, 7 tribes/tribal organizations, and 7 USAPI/territories. One of the Pacific Island Jurisdictions, the Federated States of Micronesia, supports CCC programs in 4 constituent states (Yap, Chuuk, Pohnpei, and Kosrae). Therefore, a total of 69 programs were eligible to participate in this study. Although program directors are often the most knowledgeable about the current use of EBPs, they were given the option to designate a staff person with the most knowledge about and direct involvement with the use of EBPs for their programs. Hereafter, references to program directors include their designees.

Measures and instruments

Key survey measures were based on concepts from theories related to the diffusion of innovations and to dissemination and implementation of EBPs.^{5,8,15,19,23,25,26} Diffusion of innovation posits that the adoption and use of innovations are dependent on the characteristics of the innovation, the adopters, and their organizational settings. In measuring perceived benefits and characteristics of EBPs, we incorporated theoretical concepts such as relative or perceived advantage of EBPs, compatibility with organizational norms, complexity, and reinvention (eg, adaptation). The survey instrument was also informed by the Hannon and colleagues¹⁵ study of evidence-based cancer control practices among members of the Cancer Prevention and Control Research Network.

The survey of program directors consisted of 2 distinct components with 2 separate data collection instruments: a self-administered Web questionnaire that included mostly close-ended, structured items; and a telephone interview that consisted of 4 qualitative, open-ended questions allowing interviewers greater flexibility to explore relevant topics or issues. The 4 interview questions addressed the following: (1) how program directors work with program partners to identify and implement EBPs; (2) benefits of using EBPs; (3) challenges to using EBPs; and (4) how CDC can support utilization of EBPs among CCC programs.

To obtain an estimate of both respondent burden and feedback on the format, appropriateness, and relevance of the survey questions, we pilot tested the questionnaire and telephone interview script with 3 program directors (one each from a state, tribe/tribal organization, and USAPI/territory). We revised the questionnaire on the basis of the results of the test and excluded the pilot Web survey data from final analyses. No changes were recommended to the interview script; therefore, all interview data are included in the final qualitative analyses.

Procedures

The survey was conducted during March–July 2012. The questionnaire and telephone interviews were administered by a CDC-contracted agency to protect the integrity of the survey and not bias results based on CDC presence during the interview. During the interviews, the interviewers took detailed notes and made audio recordings if permission was granted. Recordings were used only to supplement the notes and fill in gaps or check errors as needed. Survey procedures were reviewed and approved by the appropriate institutional review board. This study was approved by CDC’s Human Subjects Committee and the Office of Management and Budget (OMB Control # 0920-0921).

Analysis

We used univariate statistical methods to analyze the questionnaire data. For exploratory bivariate analyses, we used cross-tabulations to examine relationships between selected variables. Given the high survey response rate among program directors (92%), we obtained a near-census sample for which inferential statistical testing was not necessary. We used qualitative content analysis methods^27,28 using NVivo (version 9) to examine and identify key themes in the telephone interview data.

Results

Online survey

Respondent characteristics

Ninety-two percent of the program directors eligible to participate in the study completed the survey, with a 98% response rate for states, 70% for USAPI/territories, and 83% for tribes/tribal organizations (Table 1). The majority of program directors (70.5%) held a graduate or professional degree; most degrees were in public health or medical fields (74.4%). About a third each of program directors had been working with their CCC programs for 1 to 3 years and for more than 5 years.

TABLE 1.

Survey Response and Respondent Characteristics

	n (%)
Survey response (n = 66)a	61 (92.4)
States	49 (98)
USAPI/territories	7 (70)
Tribe/tribal organizations	5 (83.3)
Highest level of education completed
High school graduate/GED	1 (1.6)
Some college	1 (1.6)
College graduate	16 (26.2)
Graduate or professional degree	43 (70.5)
Graduate/professional degree in public health/medical field	32 (74.4)
Length of time working with CCC program
<1 y	8 (13.1)
1–3 y	19 (31.1)
4–5 y	12 (19.7)
>5 y	22 (36.1)

Abbreviations: CCC, comprehensive cancer control; NCCCP, National Comprehensive Cancer Control Program; USAPI, US Associated Pacific Islands.

^aThe total number of eligible NCCCP-funded programs for the survey was 66. Three of the 69 CCC programs were excluded from the survey because of their participation in the pilot testing for the survey instruments.

How do NCCCP planners use EBPs?

Seventy-five percent of program directors reported that their programs use EBPs to a moderate or great extent to address objectives in the action plans for their programs, and the same percentage reported that their jurisdictions’ CCC plans include EBPs to a moderate or great extent (data not shown). From a list of 14 theory-based characteristics related to the selection of EBPs, respondents were asked to select which characteristics are important when choosing EBPs (Table 2). The characteristics rated most frequently as very important by state program directors include that EBPs are cost-effective (81.6%), consistent with the mission of the organization (75.5%), and have scientific evidence that proves they work (73.5%). Directors of territorial programs also rated cost-effectiveness and consistency with their program’s mission (both 85.7%) as very important, and 71.4% each reported that ease of evaluation and adaptation are significant selection criteria. Among directors of tribal programs, consistency with their organization’s mission and adaptability were considered very important by all. Other characteristics that they frequently rated as important included the following: ease of implementation, available for free or low cost, cost-effective, and people in the community had requested the EBP.

TABLE 2.

Characteristics Rated as Very Important When Selecting EBPs^a

EBP Characteristics	State, n (%)	Territory, n (%)	Tribe, n (%)
Easy to implement	28 (57.1)	3 (42.9)	4 (80)
Available for free or low cost	31 (63.3)	3 (42.9)	4 (80)
Other organizations like ours are using it	25 (51)	1 (14.3)	3 (60)
Easy to evaluate	30 (61.2)	5 (71.4)	3 (60)
Cost-effective	40 (81.6)	6 (85.7)	4 (80)
Consistent with our organization’s mission	37 (75.5)	6 (85.7)	5 (100)
Technical assistance is available	15 (30.6)	2 (28.6)	3 (60)
Innovative	13 (26.5)	2 (28.6)	3 (60)
We had used it before	8 (16.3)	1 (14.3)	1 (20)
People in community requested it	17 (34.7)	2 (28.6)	4 (80)
Encouraged by our funders	33 (67.3)	3 (42.9)	2 (40)
Easily adaptable	33 (67.3)	5 (71.4)	5 (100)
Scientific evidence saying it works	36 (73.5)	4 (57.1)	2 (40)
Lack of alternatives	7 (14.3)	0 (0)	2 (40)

Abbreviations: CCC comprehensive cancer control; EBP, evidence-based practice.

^aThe question was “22. In general, how important is each of the following factors to your CCC program when choosing an EBP?” where response options included: ‘not at all important,’ ‘somewhat important,’ ‘moderately important,’ ‘very important,’ ‘Don’t know/not sure.’”

What evidence-based resources do they use?

The majority of program directors (95.1%) had heard of The Community Guide and Cancer Control P.L.A.N.E.T. (98.4%), but fewer had heard of RTIPs (65.5%) (Table 3). Among those who were aware of the resources, most had used them to find EBPs for use in their jurisdictions (75%–91.4%). Most users of The Community Guide found it somewhat useful (35.8%) and very useful (34%), whereas most of those who used Cancer Control P.L.A.N.E.T. found it somewhat useful (35.6%) and moderately useful (37.8%). Among users of RTIPs, most found it somewhat useful (58.1%). Compared with state program directors, fewer tribal program directors and USAPI/territorial program directors found these resources very useful (data not shown).

TABLE 3.

Awareness, Use, and Attitudes Toward EBP Resources

	The Community Guide, n (%)	Cancer Control P.L.A.N.E.T., n (%)	RTIPs, n (%)	Using What Works,a n (%)
Ever heard of the resource	58 (95.1)	60 (98.4)	40 (65.6)	b
Ever used the resource to find EBPs for your jurisdictionc	53 (91.4)	45 (75.0)	31 (77.5)	25 (41.0)
Number of times in last 12 mo visited the resource Web site to find EBPsd
0	3 (5.7)	0.0 (0)	2 (6.5)	b
1–5	19 (35.8)	23 (51.1)	15 (48.4)	b
6–10	22 (41.5)	12 (26.7)	7 (22.6)	b
11–15	5 (9.4)	5 (11.1)	5 (16.1)	b
≥16	4 (7.5)	4 (8.9)	2 (6.5)	b
Don’t know/not sure	0.0 (0)	1 (2.2)	0.0 (0)	b
How useful is the resource for finding EBPs?d
Not at all useful	2 (3.8)	1 (2.2)	0.0 (0)	1 (4)
Somewhat useful	19 (35.8)	16 (35.6)	18 (58.1)	9 (36)
Moderately useful	14 (26.4)	17 (37.8)	7 (22.6)	7 (28)
Very useful	18 (34)	11 (24.4)	6 (19.4)	5 (20)

Abbreviations: EBP, evidence-based practice; RTIPs, Research-tested Intervention Programs.

^aUsing What Works: Adapting Evidence-Based Programs to Fit Your Needs. http://cancercontrol.cancer.gov/usewhatworks/start.htm.

^bQuestion was not asked.

^cThis question was asked if respondent answered “Yes” to “ever heard of.”

^dThis question was asked if respondent answered “Yes” to “ever used.”

Only 41% of all directors had used “Using What Works: Adapting Evidence-Based Programs to Fit Your Needs,” an NCI-developed guide for CCC planners adapting EBPs (Table 3). Of the 25 directors who had used the manual, 64% rated its usefulness as either moderately or somewhat useful and 20% rated it as very useful.

What EBP-related technical assistance needs do they have?

Respondents were asked to identify which of 10 technical assistance/training needs would help the CCC program in their jurisdiction identify, adapt, and implement EBPs. The needs most frequently selected included how to adapt an EBP and its materials for cultural appropriateness and how to identify what aspects should and should not be changed in an EBP (Table 4). Each technical assistance need was selected by at least 40% of the program directors.

TABLE 4.

Training and Technical Assistance Needs for Using EBPs^a

Type of Training/Technical Assistance	State, n (%)	Territory, n (%)	Tribe, n (%)
How to involve other stakeholders/partners	27 (55.1)	5 (71.4)	3 (60)
How to assess and use current available resources	32 (65.3)	5 (71.4)	2 (40)
How to find and secure additional resources (eg, funding, technical assistance)	34 (69.4)	4 (57.1)	3 (60)
How to obtain EBP materials (eg, education/promotional materials)	24 (49)	2 (28.6)	2 (40)
How to adapt an EBP and its materials for cultural appropriateness	38 (77.6)	6 (85.7)	4 (80)
How to identify what aspects should and should not be changed in an EBP	37 (75.5)	6 (85.7)	3 (60)
How to pilot test an EBP with the intended audience	29 (59.2)	6 (85.7)	2 (40)
How to develop an implementation and evaluation plan	27 (55.1)	4 (57.1)	3 (60)
How to recruit participants for interventions	25 (51)	5 (71.4)	2 (40)
How to implement and evaluate an EBP	33 (67.3)	4 (57.1)	4 (80)

Abbreviations: CCC, comprehensive cancer control; EBP, evidence-based practice.

^aThe question was “33. Which types of training would help your CCC program to identify, adapt, and implement EBPs? (Check all that apply).”

Telephone interviews

How do you work with your partners to locate and implement EBPs?

We identified 3 key themes among the responses to this question: (1) using partners as consultants to identify and select EBPs; (2) collaborating with a subset of the jurisdiction’s cancer coalition to capitalize on its expertise; and (3) providing support for partners’ use of EBPs. Program directors explained that their programs work with a variety of partners in relation to utilization of EBPs, including cancer coalitions, other health department programs, community members and organizations, and advisory councils.

What are benefits of using EBPs?

The program directors cited several benefits to using EBPs, including the following: their effectiveness has been proven; they are an efficient use of resources; they lend credibility to an intervention; and they are measurable. Respondents also explained that EBPs provide them with important guidance for doing their work. One respondent stated, “With limited time and resources available, it’s of comfort to use strategies that have shown to be effective in other states, communities, or partners.” They also noted that using EBPs allows them to be fiscally responsible and to better leverage existing resources. Some indicated that using EBPs validates their activities in the eyes of stakeholders.

What are challenges to using EBPs?

Program directors also identified several challenges to using EBPs, including the following: limited resources; lack of appropriate EBPs for specific populations and communities; the need to obtain community buy-in; and limited skills adapting EBPs. Respondents indicated that they lack staff and staff time to investigate EBPs; they have limited interventions for Native American, USAPI, rural, and non–English-speaking communities; and they lack guidance for adapting EBPs without losing fidelity and within a timely manner. One respondent explained, “Many EBPs seem too labor intensive to actually implement and programs just don’t have the man power.” Another stated, “Resources aren’t always based on tribes and territories but rather on the states.” With regard to adapting EBPs, one respondent expressed, “It is also very time consuming to try to find an EBP and then figure out how to adapt it to the local population.”

How can CDC help support or promote the use of EBPs?

The themes that emerged in response to this question included the need for the following: more workshops and trainings on using EBPs; tools and resources that are specific to working with EBPs; a platform to promote greater sharing among grantees; and stronger program consultant support and technical assistance. Respondents indicated that webinars are a good format for training. Some expressed a need for more culturally competent tools and resources for adapting EBPs, more flexibility and acceptance for using “promising practices” than using more well-established EBPs, and better understanding on the part of national organizations regarding the issues and challenges that some communities face when trying to use EBPs. One respondent indicated, “A train-the-trainers training on EBPs might be useful since oftentimes the information on EBPs needs to be taken back to partners and coalition members who are not that knowledgeable about EBPs.”

Conclusions

This is the first study to comprehensively and specifically assess the use of EBPs among NCCCP-funded programs and the key partners who help them use EBPs. Overall, most program directors were aware of and had used The Community Guide, Cancer Control P.L.A.N.E.T., and to a lesser extent RTIPs. The perceived usefulness of these resources varied considerably across study participants. Directors of USAPI/territorial and tribal programs perceived them to be less useful than state program directors, perhaps because of the limited availability of culturally appropriate interventions.

The characteristics that respondents perceived as important for choosing EBPs included theoretically based factors that have been shown to be predictive of successful dissemination of interventions.⁵ For example, program directors indicated that EBPS should be consistent with their organization’s mission, easy to implement, and amenable to adaptation. However, they also identified many challenges to using EBPs that could affect the timely adoption of innovations, including resource limitations, and the difficulty of finding EBPs that are culturally appropriate for, or can be adapted to, local populations and communities. The Hannon and colleagues¹⁵ 2008 survey of community-based cancer control planners also examined the use of EBPs and EBP-related resources to better understand planners’ needs and determine what actions might enhance the use of EBPs. They reported lower levels of awareness and use for The Community Guide, Cancer Control P.L.A.N.E.T., and RTIPs than we found. However, our findings may reflect increased promotion of EBP resources since 2008. Our results were consistent with Hannon and colleagues regarding what characteristics are important when selecting EBPs and what types of EBP-related training CCC programs would like to receive.

Increasing NCCCP grantees’ knowledge about EBP resources, including the strengths and limitations of these tools, may increase their use of EBPs in addressing cancer control priorities. Both CCC program staff and their partners should understand the uses and benefits of EBPs but have a realistic attitude about what these resources can offer. Ongoing workshops and trainings for CCC program staff via webinars or other formats and delivery options (eg, NCI’s Research 2 Reality Web site, or a YouTube channel) could be used to share this information. Topics could include the following: (1) how to identify EBPs; (2) how to adapt EBPs for cultural appropriateness and local implementation while preserving effectiveness, with special emphasis on the tribal and USAPI/territorial CCC programs; and (3) how to implement and evaluate cancer control interventions and programs.

Although evaluation of the use of EBPs by participants in national programs is necessary to inform program planning and refine technical assistance, published studies on this topic are limited.^15,29 With participation by more than 90% of NCCCP program directors, this study provides a rich source of information on knowledge, attitudes, practices, and training needs related to the use of EBPs among grantees’ from diverse settings. This study also has limitations that may affect the interpretation of our findings. First, some program directors opted to assign staff to represent them in the survey. We did not assess whether these programs differed organizationally or in other ways from programs for which surveys were completed by program directors. The inclusion of designated staff might have introduced a selection bias if their responses are not representative of those of the program directors who participated in this study. Second, a social desirability bias might have been introduced if respondents did not answer questions according to their true beliefs.

This study specifically targeted program directors among all NCCCP grantees. The findings will help expand our understanding of how EBPs are identified and used to address local and national public health priorities and will guide ongoing efforts to promote the use of EBPs for cancer control. The survey results could also be used to inform future studies related to the use of scientific and practice-based evidence in CCC, particularly for CCC programs seeking a wider selection of culturally appropriate interventions.