Capacity Building for and Implementation of Policy, Systems, and Environmental Change: Results from a Survey of the National Comprehensive Cancer Control Program

Abstract

Introduction:

Policy, systems, and environmental (PSE) approaches are commonly used to improve population health. Cancer-related examples include providing data and education to stakeholders about policies that support healthy living, or health systems changes such as universal reminders about recommended cancer screening. The National Comprehensive Cancer Control Program (NCCCP) funds health departments to form cancer coalitions that develop and implement cancer plans. The NCCCP initiated a demonstration program in 13 of 65 funded grantees to determine whether skilled, dedicated staffing and using a strategic process to examine data, form a workgroup, and develop an agenda would enhance their capacity to implement PSE approaches, recruit new partners, and provide data and education to stakeholders. The objective of this study was to compare demonstration program grantees to other NCCCP grantees on their ability to develop and implement PSE strategies, and the short-term results that were achieved.

Methods:

Program Directors (PDs) from each NCCCP-funded jurisdiction completed web surveys at two time points during implementation to assess changes in their capacity for PSE approaches, identify implementation activities, and document short-term outcomes. Responses from demonstration program PDs and other PDs at both time points were compared in a descriptive analysis.

Results:

Demonstration program grantees experienced greater increases in skills and capacity for addressing PSE approaches, engaged in necessary implementation activities more often, and achieved greater improvements in stakeholder and decision-maker awareness and support for PSE strategies, compared to nonparticipating NCCCP grantees.

Conclusions:

These findings support continued implementation of PSE approaches for sustainable cancer prevention and control.

INTRODUCTION

Cancer is the second leading cause of death in the United States¹, and although incidence rates are decreasing for many cancer sites², the absolute number of new cases continues to increase due to the aging US population³. Some cancer risks can be mitigated by reducing tobacco and alcohol use, obesity, and exposure to select infectious and environmental agents^4–6. Early detection of breast, cervical, and colorectal cancer can improve outcomes, and is particularly needed among underserved populations^7,8. For the growing number of cancer survivors, preventive care and support can reduce recurrence and new primary cancers, mitigate long term and late effects of cancer treatment, and improve quality of life^9–14.

Policy, systems, and environmental (PSE) approaches are promising strategies to address the nation’s cancer burden because they have the broadest population impacts by changing the overall context for health-related decision making and facilitating long-lasting protective interventions.¹⁵ PSE approaches are a commonly used public health strategy to reduce disease burden.¹⁵ Some examples of PSE approaches in cancer include providing data to stakeholders and decision makers about the harms of indoor tanning among minors and its relationship with melanoma incidence to inform policy; educating about systems-level changes to increase early detection of cancer, such as expanding mammography clinic hours in a health system; or promoting changes to the built environment that support healthy living, such as the development of walking trails and accessible recreational areas to increase physical activity that can reduce cancer risk. Many PSE approaches occur at the local level through voluntary organizational policies or environmental changes in schools, worksites, health systems, and communities.

The National Comprehensive Cancer Control Program (NCCCP), funded by the Centers for Disease Control and Prevention (CDC), initiated a demonstration program to enhance grantee capacity to develop and implement these approaches. Comprehensive cancer control (CCC) is a collaborative and strategic process that brings together various stakeholders in cancer control to develop and implement cancer control plans for their jurisdiction based on cancer burden and local factors, and it is based on a participatory innovation diffusion model.¹⁶ Traditionally, NCCCP grantees worked with their partners to implement interventions across the cancer control continuum. Beginning in 2010, grantees were encouraged to prioritize interventions that address primary prevention, early detection and screening, and cancer survivorship.¹⁷ While NCCCP grantees have been encouraged to include PSE strategies in their activities since the program’s founding in 1998, a greater focus on using them became a cross-cutting programmatic priority in 2010.¹⁷ The intent of the demonstration program was to begin building a practice- and evidence-based framework for supporting PSE approaches among all NCCCP grantees.

The demonstration program was designed to enhance cancer control capacity by assisting grantees with skill building among program staff and partners, prioritizing a set of PSE approaches based on cancer surveillance data and local context to form a PSE agenda (plan), using appropriate messaging and framing to discuss the cancer burden and strategies, and by strengthening and expanding partnerships among all relevant stakeholders, including with nontraditional partners. Multiple theoretical frameworks, most notably the Component Model of Infrastructure and the Community Coalition Action Theory, were used to guide development of the program’s conceptual model which served as a framework to evaluate its short and long term outcomes.^18–22 Thirteen NCCCP grantees were selected for this demonstration program in 2010 through a competitive application process that focused on organizational capacity to implement PSE change initiatives, ability to build upon existing partnerships or expand partnerships, proposed activities and outcomes, and plans to measure performance. The 13 grantees included one tribal nation and 12 states diverse in geography and racial/ethnic composition. This demonstration program included the addition of a dedicated policy analyst at the grantee level, and grantees received specialized technical assistance (TA) from CDC and its partners on how to develop and implement a data-driven, evidence-based policy agenda that is feasible based on local political context and cultural preferences. Trainings also focused on lobbying restrictions; building and expanding partnerships; creating compelling messages and developing a media/communications strategy; effectively working with PSE decision makers to provide data and educate them about evidence-based PSE strategies; and conducting evaluation. Peer-to-peer communication between grantees was common, and provided another platform for grantees to share effective practices with one other and with their partners and stakeholders. Policy analysts were encouraged to have competencies in the following areas: framing public health issues, media utilization, message tailoring, policy analysis, and policy formulation. Policy analysts played key roles in reviewing cancer burden data and developing an environmental scan of existing PSE approaches; forming and maintaining the PSE workgroup; developing the policy agenda; creating effective messaging and developing a media plan; and providing data and information on evidence-based strategies when approached by stakeholders and decision-makers.

The purpose of this study is to compare demonstration project grantees with other grantees funded by the NCCCP to determine whether the addition of specialized TA and a dedicated policy analyst led to more changes in the capacity to develop and implement PSE strategies over time.

METHODS

Survey description

We reviewed the literature, examined existing tools and instruments, and created an evaluation consultant group to inform development of a survey instrument that included measures to capture constructs around program capacity and infrastructure, and implementation of PSE change strategies. We administered a web-based survey (Office of Management and Budget Control Number: 0920–1016) in the fourth and fifth year of the demonstration program’s five-year funding period (Time 1 and Time 2, respectively).

Program Directors (PDs) in all 65 NCCCP-funded states, tribes and territories (13 demonstration program grantees and the remaining 52 NCCCP grantees not participating in this demonstration program) received an email explaining the survey’s purpose and requesting their participation. We sent up to five reminder emails to non-respondents on a weekly basis until the survey closed. We surveyed PDs because they were expected to be the most knowledgeable about program capacity and cancer control infrastructure in their jurisdictions. However, because staff roles and responsibilities vary across programs, we allowed PDs to designate a staff person who had a more direct role in the PSE-related activities to be the respondent for the survey. We allowed only one respondent per grantee entity to complete the survey.

The survey addressed two overarching evaluation questions: (1) How did demonstration program grantees build capacity required to develop an environmental scan, policy agenda, evaluation plan, and media plan? (2) What methods were used by demonstration program grantees to implement the policy agenda and media plan?

Survey domains – NCCCP program staff and PSE workgroup/coalition competencies and skills

To specifically assess capacity and infrastructure development and operationalize this concept, the survey first asked respondents to rate NCCCP program staff in six competency areas. Staff competency was measured on a 4-point Likert scale – Not at all; Basic-Staff have only general knowledge; Moderate-Staff could do this; Expert-Staff could teach this – with the option to mark each as “Don’t Know.” Next, respondents indicated their level of agreement with statements about the skills of program staff and partners (i.e., PSE workgroup/cancer coalition members) (Table 1); and the representation of appropriate sectors among cancer coalitions and PSE workgroup members. Extent of agreement with skill statements was measured on a 5-point Likert scale – strongly disagree; somewhat disagree; neutral; somewhat agree; strongly agree – with the option to mark each as “Not applicable.”

Table 1.

Program Directors/Designees assessments of PSE change competencies and skills among NCCCP program staff and PSE Workgroup/Cancer Coalition

Capacity and Infrastructure for PSE Change	Time 1		Time 2
	Demo PDs %	Other PDs %	Demo PDs %	Other PDs %
Competencies of NCCCP Staff*	n=12	n=39	n=10	n=32
Building relationships with partners, stakeholders	92	95	100	94
Issue framing	83	69	100	77
PSE strategy formulation	67	56	100	69
PSE strategy analysis	67	62	82	63
Media utilization	50	67	73	31
Message framing	67	72	73	83
Skills of NCCCP Staff**	n=12	n=39	n=10	n=32
Gathering data to show value of PSE strategies	100	82	100	88
Developing specific PSE goals	100	82	100	82
Assembling evidence to support PSE strategies	92	90	100	77
Convening a PSE workgroup with sector representation	83	80	100	80
Working with allies to support PSE change goals	92	87	100	88
Leading action planning for PSE change	83	74	90	80
Developing PSE change proposals	100	72	90	77
Providing data to decision makers	83	85	90	85
Technical assistance for implementation	83	74	90	82
Technical assistance for sustainability	83	67	90	68
Holding local forums to promote PSE change	67	67	80	71
Skills of PSE Workgroup or NCCCP Coalition**	n=12	n=38	n=10	n=31
Educating decision makers	83	71	100	77
Holding stakeholder meetings	83	74	90	74
Reaching out to allies	67	79	90	90
Developing specific PSE goals	58	66	80	61
Providing input to an Action Plan	75	76	80	90
Assessment of political environment, context	67	53	70	71
Providing technical assistance	33	55	70	45
Developing PSE change proposals	42	61	60	65
Organizing media events	67	53	60	45
Using media strategies	58	47	60	48
Monitoring implementation quality	42	42	50	58

^*Percent reporting moderate or expert levels for each competency. Level of competency was measured on a 5-point Likert scale – Not at all; Basic-Staff have only general knowledge; Moderate-Staff could do this; Expert-Staff could teach this – with the option to mark each as “Don’t Know.”

^**Percent reporting strong-moderate agreement with each statement. Extent of agreement with skill statements was measured on a 5-point Likert scale – strongly disagree, somewhat disagree, neutral, somewhat agree, strongly agree – with the option to mark each as “Not applicable.”

Abbreviations: NCCCP, National Comprehensive Cancer Control Program; PD: program directors/designees; PSE: policy, systems, and environmental.

The total sample size may vary depending on the survey item because not all program directors/designees provided responses to every item. During each survey time point, 13 demo PDs and 52 other PDs were contacted to participate.

Respondents also indicated (Yes/No) which trainings or resources were most helpful in building capacity for key skill sets.

Survey domains – implementation of PSE change strategies

The next set of survey items addressed implementation of PSE strategies prioritized in the grantees’ PSE agendas. Of the PSE strategies led by their internal cancer coalition (a diverse group of local cancer control practitioners and researchers who establish the cancer plan for the state, tribe or territory) or PSE workgroup (either a subset of the cancer coalition or a group of health department staff and partners with interest and expertise in using PSE approaches), respondents identified two they considered to have the most importance and the highest potential for success. For each strategy, the survey asked a series of questions about: methods used for stakeholder outreach and decision-maker education; interactions between PSE workgroup/cancer coalition members, allies (individuals or groups with a similar interest in the strategy) and opponents (individuals or groups who opposed the strategy); and key events that influenced the implementation of each strategy. Extent of agreement with each statement on level of awareness was measured on a 5-point Likert scale – strongly disagree, somewhat disagree, neutral, somewhat agree, strongly agree – with the option to mark each as “Not applicable.”

Survey domains – short term outcomes (decision-maker awareness and stakeholder support)

Finally, the survey addressed short-term outcomes: the level of awareness and knowledge of decision-makers and stakeholders involved in the two identified PSE strategies; and decision-maker position and stakeholder support for these strategies. Awareness was categorized as aware/accurately informed, aware but uninformed, aware but inaccurately informed, unaware, unsure/don’t know. Response categories for support included visible and vocal program champion [stakeholders only], supportive, neutral, no position, opposed, and don’t know.

Data analysis

Data analysis was descriptive, employing univariate statistical methods (Stata©, version 13) with tables and graphs for display and summarization. We calculated response percentages for each group at each time point to facilitate comparisons between demonstration program respondents “Demo PDs” and respondents not participating in the demonstration program“other PDS”, and to examine changes over time. We collapsed the competency categories as moderate/expert versus basic/not at all. Levels of agreement regarding skill level and interaction with allies and opponents were grouped as strongly/somewhat agree versus strongly disagree/ somewhat disagree/neutral. The percentage aware was the combined percentages for the response categories aware but uninformed, aware but inaccurately informed, and aware/accurately informed, versus the other categories unaware, unsure/don’t know. The percentage in support was the combined percentages for the response categories visible and vocal program champion [stakeholders only] and supportive, versus the other categories opposed, neutral, no position, don’t know.

No inferential statistical tests were done, as the survey was planned as a census of the directors of all NCCCP-funded programs, and any final set of survey respondents would be a non-probabilistic sample of that population. Thus, the findings should be regarded as representative of survey respondents only, rather than all NCCCP-funded programs.

RESULTS

Survey response was less than 100% at both time points, with a higher response at Time 1 than at Time 2. The overall response at Time 1 (T1) was 78%: 92% among demo PDs, and 75% among other PDs. The overall response at Time 2 (T2) was 65%: 77% among demo PDs, and 62% among other PDs.

Capacity and Infrastructure for PSE Change: Competencies of NCCCP Staff.

By T2, a higher percentage of demo PDs compared to other PDs reported that staff had either moderate or expert competency in building relationships (100% and 94%, respectively), issue framing (100% and 77%, respectively), PSE strategy formulation (100% and 69%, respectively) and media utilization (73% and 31%, respectively) (Table 1). Ratings for most competencies increased between T1 at T2 for both groups of PDs, though to a greater degree for demo PDs, especially for PSE strategy formulation, issue framing, and media utilization.

Skills of NCCCP Staff.

At T2, the vast majority of demo PDs expressed confidence (somewhat or strongly agree) that staff had each of the listed skills needed to promote PSE strategies, with 100% endorsement for 5 of the 11 skills: gathering data to show value of PSE strategies; developing specific PSE goals; assembling evidence; convening a PSE workgroup with sector representation; and working with allies to support PSE change goals (Table 1). While a majority of other PDs also expressed confidence for these skills, their percentages were lower than demo PDs by 5 to 23 percentage points at T2. Between T1 and T2, the percentage of demo PDs that endorsed staff skills increased or remained at 100% for 10 of the 11 skills, while the change in endorsements for the other PDs was more variable.

Skills of PSE Workgroup or NCCCP Coalition.

By T2, 90% of demo PDs had either created a PSE workgroup or had plans to do so in the coming year, compared to only 58% of other PDs (data not shown). Compared to other PDs, a higher percentage of demo PDs said that their PSE workgroup had the right sector representation to meet its PSE objectives (88% compared to 58%; data not shown). For both PD groups, between T1 and T2 there was little change in the percentage with a PSE workgroup or plans to form one, and sector representation also changed minimally.

At T2, the majority of demo PDs expressed confidence that their PSE workgroups and cancer coalition members had the necessary abilities for 10 of the 11 skills (Table 1). In contrast, the other PDs’ confidence in partner skills tended to lag behind the demo PDs. Between T1 and T2, the percentage of demo PDs that endorsed staff skills increased for 10 of the 11 skills, while the change in endorsements for the other PDs was more variable. Notably, for the three skills where the demo PDs had lower percentages of endorsements at T1 than the other PDs (reaching out to allies; developing specific PSE goals; providing technical assistance), by T2 the demo PD endorsements either matched or exceeded the other PDs.

Trainings and Resources.

Figure 1 shows the resources and training sources, per PSE skill set, selected by the highest percentages of demo and other PDs at T2 as the most helpful in building PSE capacity. More demo PDs selected standard trainings by CDC and national partners and/or tailored trainings by national partners as the most helpful sources for developing key messages for target audiences, working with various media, and PSE strategy formulation – the next highest sources for these three skill sets are trainings developed locally , and networking with other programs (both tied at 40%, 40%, and 50%, respectively). The demo PDs overwhelmingly selected networking with other programs as the most beneficial source for gaining stakeholder support and forming strategic alliances – the next highest source for those two skill sets is trainings developed locally (40% for both). Similarly, 70% of demo PDs selected guides, websites, tools as the most helpful for building skills to provide data and information to raise decision-maker awareness – the next highest source is networking with other programs at 40%.

Figure 1.

Top selected resources and training sources most helpful for building PSE change capacity at Year 5 of the Demonstration Program, by skill areas and Program Director/Designee group.

Note: Resources and training sources included: Standard trainings by CDC/national partners; Tailored trainings by national partners; Trainings developed locally; Networking with other programs; Guides, websites, tools; Other resource; None.

Abbreviations: CDC, Centers for Disease Control and Prevention; NCCCP, National Comprehensive Cancer Control Program; PD: program directors; NP, national partners; PSE: policy, systems, and environmental.

Demo PDs- Program Directors in Demonstration Program

Other.PDs- Program Directors not in Demonstration Program

The sources most frequently selected by the other PDs largely paralleled those selected by the demo PDs, but at lower percentages since their selections were distributed more evenly across multiple sources. However, the other PDs most frequently indicated that they had no source for working with various media, though approximately a quarter also selected standard trainings by CDC and national partners (26%), networking with other programs (23%), and guides, websites, tools (23%). The other PDs also differed from the demo PDs in the sources they tended to select for the providing data and information to raise decision-maker awareness skill set: the most frequently selected source was networking with other programs, but not far behind are, tailored trainings by national partners (26%), guides, websites, tools (26%), and standard trainings by CDC and national partners (23%).

Methods used to implement PSE Change strategies Stakeholder Outreach Methods.

At T2, the demo PDs reported using multiple outreach methods to increase awareness of PSE strategies among stakeholders (Table 2): 84% held meetings, workshops and community forums; 72% engaged stakeholders in collecting evidence to show value of PSE strategies; 40% created white papers, websites, or newsletters; and 24% secured earned media, public service announcements. The other PDs reported a similar usage pattern for these methods, though at much lower frequencies (69%, 55%, 22%, and 14%, respectively). There was very little change between T1 and T2 for stakeholder outreach and the other implementation activities (data not shown).

Table 2.

Implementation of PSE Change Strategies at Year 5 of the Demonstration Program

Methods used to implement PSE Change strategies	T2/Year 5
	Strategies used by demonstration program grantees %	Strategies used by other NCCCP grantees %
Stakeholder Outreach Methods*	n=25	n=85
Hold meetings, workshops, and community forums	84	69
Create white papers, websites, or newsletters	40	22
Secure earned media, public service announcements	24	14
Engage stakeholders in collecting evidence to show value of PSE change strategies	72	55
Decision-Maker Education Methods*	n=16	n=45
Provide data and other information to demonstrate value	81	67
Provide evidence that shows the health impact of PSE change strategies	44	58
Develop proposal for a specific PSE change strategies	56	47
Create white papers, websites, or newsletters	0	7
Create media, public service announcements	0	9
When requested, attend in-person meetings or briefings to educate decision makers	44	20
Provide data and information at board meetings and legislative hearings, upon request	31	27
Demonstrate support for PSE change strategies from stakeholders and constituents	19	20
Interactions with Allies**	n=22	n=64
Identified Allies	96	80
Reached out to Allies	86	78
Had one or more meetings to discuss common interests	100	70
Regularly shared information and coordinated activities to achieve mutual goals	91	75
Interactions with Opponents**	n=22	n=64
Identified Opponents	59	30
Reached out to Opponents	23	22
Identified strategies to address Opponent’s concerns	27	28

^*Percent reporting “Yes” for each method of stakeholder outreach and decision-maker education.

^**Percent reporting either strongly agree or somewhat agree with each statement. Extent of agreement measured on a 5-point Likert scale – strongly disagree, somewhat disagree, neutral, somewhat agree, strongly agree – with the option to mark each as “Not applicable.”

PDs answered questions on their top two PSE change strategies.

Abbreviations: NCCCP, National Comprehensive Cancer Control Program; PD: Program Directors/Designees; PSE: policy, systems, and environmental.

Decision-Maker Education Methods.

At T2, the demo PDs reported using multiple methods to educate decision-makers about PSE strategies (Table 2): provide data and other information to demonstrate value (81%); develop specific proposals for PSE strategies (56%); attend in-person meetings or briefings with decision-makers when requested (44%); and provide evidence showing the health impact of PSE strategies (44%). The other PDs selected the first three methods less frequently than demo PDs (67%, 47%, and 20%, respectively), but stated more often that they provide evidence showing the health impact of PSE strategies (58% compared to 44%). There was very little change between T1 and T2 (data not shown).

Interactions with Allies.

At T2, most respondents reported that their PSE workgroups/cancer coalition members had identified allies for the selected PSE strategies (Table 2), though demo PDs reported this more frequently than other PDs (96% and 80%, respectively). Similarly, at T2 a higher percentage of demo than other PDs reported that they: reached out to allies (86% compared to 78%); had one or more meetings with allies to discuss common interests (100% compared to 70%); and regularly shared information and coordinated activities to achieve mutual goals (91% compared to 75%).

Interactions with Opponents.

In general, all respondents reported interactions with opponents less frequently compared to interactions with allies (Table 2). However, at T2 the demo PDs stated more often than other PDs that they identify opponents (59% compared to 30%). The two groups were similar in terms of reaching out to opponents to better understand their position or find common ground (23% and 22%, respectively) and to say they looked for ways to address opponents’ concerns about the PSE approaches (27% and 28%, respectively).

Outcomes

Figure 2 provides an overview of reported awareness and support for selected PSE strategies among stakeholders and decision-makers at both time points.

Figure 2.

Changes in stakeholder and decision-maker awareness of and support for PSE change issues.

% Aware is the combined percentages for the response categories aware but uninformed, aware but inaccurately informed, and aware/accurately informed, versus the other categories unaware, unsure/don’t know.

% Support is the combined percentages for the response categories visible and vocal program champion [stakeholders only] and supportive, versus the other categories opposed, neutral, no position, don’t know.

Stakeholder awareness.

Overall, respondents in both groups reported high levels of awareness among stakeholders at both time points, and similar magnitudes of change between the time points (8 percentage points). The demo PDs reported higher percentages of stakeholder awareness at both time points than the other PDs. From T1 to T2, demo PDs reported an increase for aware/uninformed (14 to 24%), no change for aware/inaccurately informed (steady at 8%), and a slight decrease for aware/accurately informed (70% to 68%). From T1 to T2, the other PDs reported an increase for aware/uninformed (9% to 17%), an increase for aware/inaccurately informed (6% to 21%), and a substantial decrease for aware/accurately informed (65% to 51%).

Stakeholder support.

At T1, the two PD groups were reporting similar percentages of support among stakeholders. But by T2, the demo PDs reported an 11-point increase, while the other PDs reported a 4-point decrease. This overall shift in support reflects a reduction in the percentage of demo PDs reporting either opposed, no position, or don’t know by T2. In contrast, the other PDs were reporting an increase in the number of opposed and don’t know by T2.

Decision-maker awareness.

Overall, respondents in both groups reported high levels of awareness among decision-makers at both time points (although lower than stakeholders), and similar magnitudes of change between the time points (2–3 percentage points). However, the two groups had different directions of change: the demo PDs reported a slight increase in awareness by T2, while the other PDs reported a slight decrease. From T1 to T2, demo PDs reported a decrease for unaware (6% to 0%), a slight decrease for aware/uninformed (22% to 19%), no change for aware/inaccurately informed (steady at 6%), and an increase for aware/accurately informed (44% to 50%). From T1 to T2, the other PDs reported an increase for aware/uninformed (9% to 17%), an increase for aware/inaccurately informed (6% to 21%), and a substantial decrease for aware/accurately informed (65% to 51%).Both groups reported substantial percentages of unsure/don’t know at both time points with small increases from T1 to T2: 22% to 25% for demo PDs; 18% to 20% for other PDs.

Decision-maker support.

At T1, demo PDs stated less often than other PDs that decision-makers were supportive (67% and 74%, respectively) and stated more often that they were opposed to PSE approaches (11% and 2%, respectively). This gap closed quite a bit by T2 (69% and 71%, respectively), when fewer demo PDs reported opposition (6%), while the other PDs held steady (2%).

DISCUSSION

The results from the PD Survey showed that PDs who participated in the demonstration program reported greater increases in staff competencies over time compared to other PDs for most skills. Furthermore, demonstration program PDs were more likely than other PDs to say that TA and trainings from CDC and national partners were the most helpful resources for building their capacity. Evaluations of other national initiatives to promote the use of PSE approaches in public health have also demonstrated the importance of TA/training for building capacity and contributing to successful implementation and outcomes.^23–28 TA/training is also recognized as an essential component for improving translation of research findings into health promotion practice.^29,30 However, quality TA/training is key to building capacity.^30,31 Elements of the RE-AIM framework, particularly adoption and maintenance,³² may be helpful as well to measure impact and make refinements as needed to implementation activities so that NCCCP grantees adopt practices to build their capacity for PSE strategies and maintain them over time for effective PSE change.

Demonstration program PDs reported having a PSE workgroup, reported sufficient sector representation within their workgroups, and expressed higher confidence in the workgroup members’ PSE competencies more often when compared to other PDs. Collaboration, partnerships, and communication among key stakeholders, including the formation of dedicated workgroups, are important facilitating factors for achieving the goals of PSE strategies.^{24,26–28,33–35} While a high proportion of both PD groups surveyed reached out to allies to discuss areas of mutual interest, share information, and collaborate, demonstration program PDs had greater awareness of opponents. However, low levels of outreach to opponents were reported by both groups of PDs, as was looking for ways to address their concerns.

The overall percentage of PDs who said their stakeholders were aware of PSE approaches, whether informed or uninformed, was much higher among demonstration program PDs compared to other PDs, a gap that grew over time. Similarly, the percentage of demonstration program PDs reporting awareness among decision makers increased slightly, and those reporting opposition fell—while the percentage of other PDs reporting awareness and support among decision makers decreased. These changes may reflect the greater outreach and education efforts on the part of PSE workgroup within demonstration program grantees.

This survey has several limitations. Because we did not achieve a 100% census among all PDs, the findings cannot be considered representative of all NCCCP grantees. At baseline, demonstration grantees may have had better skills for PSE approaches because they were selected through a competitive application process to receive funding. The lower survey response rate at T2 limited our ability to use analytic approaches, such as analysis of covariance, to control for baseline differences between groups. During both survey time periods, numerous public health initiatives were also occurring to build capacity for PSE approaches. Therefore, these efforts may have also contributed to our findings. Data collection approval for the first time point was delayed, resulting in a shortened period between the data collection time points. Therefore, we may not have detected some changes of interest that occurred earlier in the demonstration project. Some staff turnover occurred from T1 to T2, resulting in a handful of survey respondents who were not the original PD or designee at T2. This may have biased the results to a small extent if the new survey respondents were not familiar enough with their staff and partners’ skills and competencies, or lacked knowledge about their stakeholders or decision makers. Finally, as the survey relies on self-reported information from the PDs (or their designees), social desirability bias could affect some of our findings.

This survey also has several strengths. One strength is that survey measures were linked to a theory- and evidence-based conceptual model developed to guide the overall evaluation.²² Furthermore, our study adds a unique and novel aspect in that previous studies in related areas have not examined proactive engagement with stakeholders who may be opposed to PSE initiatives.^24,28,34 Additionally, the survey allowed us to examine change in measures over time, and to compare the demonstration grantees to the other NCCCP grantees.

CONCLUSION

The NCCCP has consistently recognized the use of PSE strategies as an essential component of comprehensive cancer prevention and control.¹⁷ Since 2010, PSE strategies have been an explicit priority for the NCCCP and is reflected in the programmatic requirements and resource investments. For example, with the funding cycle that started in 2012, the NCCCP required grantees to hire a half-time subject matter expert in PSE approaches. This trend is also reflected in CDC’s priorities for chronic disease prevention and control in general.³⁶ The demonstration program examined whether additional resources and support could promote and enhance the use of PSE approaches to a large extent and within a relatively short period of time. The results presented here suggest that the demonstration program framework was successful in enhancing grantee capacity for using PSE strategies. The program framework, and the results of its evaluation, can be used to promote continued implementation of PSE strategies for sustainable cancer prevention and control. It may also be adapted for other chronic disease public health programs who desire to increase their capacity for using such strategies.