Implementation strategies to promote community-engaged efforts to counter tobacco marketing at the point of sale

Abstract

The US tobacco industry spends $8.2 billion annually on marketing at the point of sale (POS), a practice known to increase tobacco use. Evidence-based policy interventions (EBPIs) are available to reduce exposure to POS marketing, and nationwide, states are funding community-based tobacco control partnerships to promote local enactment of these EBPIs. Little is known, however, about what implementation strategies best support community partnerships’ success enacting EBPI. Guided by Kingdon’s theory of policy change, Counter Tools provides tools, training, and other implementation strategies to support community partnerships’ performance of five core policy change processes: document local problem, formulate policy solutions, engage partners, raise awareness of problems and solutions, and persuade decision makers to enact new policy. We assessed Counter Tools’ impact at 1 year on (1) partnership coordinators’ self-efficacy, (2) partnerships’ performance of core policy change processes, (3) community progress toward EBPI enactment, and (4) salient contextual factors. Counter Tools provided implementation strategies to 30 partnerships. Data on self-efficacy were collected using a pre-post survey. Structured interviews assessed performance of core policy change processes. Data also were collected on progress toward EBPI enactment and contextual factors. Analysis included descriptive and bivariate statistics and content analysis. Following 1-year exposure to implementation strategies, coordinators’ self-efficacy increased significantly. Partnerships completed the greatest proportion of activities within the “engage partners” and “document local problem” core processes. Communities made only limited progress toward policy enactment. Findings can inform delivery of implementation strategies and tests of their effects on community-level efforts to enact EBPIs.

INTRODUCTION

Tobacco use is the leading preventable cause of death in the USA, killing more than 480,000 people each year [1]. Tobacco marketing increases the risk of tobacco use and impedes users’ attempts to quit [2–5]. In a recent review of 20 studies, Robertson et al. (2015) found consistent evidence of a positive association between exposure to point-of-sale (POS) tobacco marketing (e.g., product displays, advertisements, and price discounts) and smoking [6]. The US tobacco industry spends over $8.2 billion annually marketing tobacco in convenience stores, gas stations, and other POS settings [7], and stores that sell cigarettes post an average of 29.5 advertisements for tobacco products [8]. The density of tobacco retailers (number per 1000 population) further increases exposure to tobacco marketing and also increases disparities in rates of tobacco use [2], as density is disproportionately high in low income and predominantly African American communities [9]. Tobacco retailers’ proximity to schools increases exposure to youth, who are more susceptible to the effects advertising has on tobacco use than adults [2, 3, 10–15].

Evidence-based policy interventions (EBPIs) are available that are known to reduce communities’ exposure to POS tobacco marketing. They include laws, ordinances, and resolutions that regulate tobacco product pricing, promotion, and placement and licensing restrictions on retailer types, density, and proximity to youth serving institutions [16–18]. Implementation research is now needed to identify effective ways to support community-level efforts to promote local enactment of these EBPIs. We report findings of an implementation research study to test the effects that Counter Tools’ implementation strategies had on community-level efforts to promote enactment of POS tobacco EBPIs. Counter Tools is a non-profit organization that provides tools, EBPIs, training, and other implementation strategies to support local enactment of POS tobacco EBPIs [19].

For decades, the Centers for Disease Control and Prevention has provided funding to states who, in turn, awarded grants to health departments and other community organizations to fund community-based tobacco control partnerships [20]. The funded organizations use these grants to pay for the partnership’s work and to also provide salary support for the tobacco prevention specialist or health educator who coordinates the partnership’s work (hereafter referred to as a partnership coordinator). Following their success enacting laws to raise cigarette taxes and create smoke-free spaces [1], many states now are prioritizing community-level enactment of POS tobacco EBPIs.

Changing policy is challenging and involves engaging community partners throughout an uncertain and often long policy change process [19, 21, 22]. Partnership coordinators may lack the capacity to facilitate the partnerships’ efforts to promote local EBPI enactment [23–25]. To be successful, partnership coordinators need tools, EBPI guidance, training, and other implementation strategies to support them throughout the process. Despite the effectiveness of POS tobacco EBPIs, little is known about which implementation strategies best support community partnerships’ efforts to promote local enactment of EBPI. In this paper, we describe a theory-based approach to providing implementation strategies and to testing their effectiveness.

Theory-based conceptual framework

The study’s theory-based conceptual framework (Fig. 1) is based on Leeman’s capacity building framework, which explains how implementation strategies (tools, EBPI guidance, training, and technical assistance) promote community-level enactment of EBPIs through their effects on two intermediate outcomes (mechanisms of change): (1) individual-level self-efficacy to coordinate and (2) community partnership-level performance of core policy change processes [26, 27]. Coordinators are particularly central to the success of community partnerships because they operate outside a formal organizational hierarchy and therefore must engage cross-sectoral partners in an ongoing, collaborative decision-making process [28, 29]. The study’s conceptual framework incorporates Kingdon’s multiple stream theory [30] and the work of multiple researchers to translate Kingdon’s theory into five core policy change processes [31–34]. Community partnerships need to document local problems (Kingdon’s first stream) and formulate policy solutions by selecting EBPIs that best fit the local problem and existing policy (Kingdon’s second stream). To build political will (Kingdon’s third stream), they need to engage strategic partners, raise awareness of problems and solutions, and persuade decisionmakers to enact new policy. Although most evidence for the effectiveness of these processes comes from quasi-experimental and case study research, the evidence is “clear and consistent” [35] in support of the association between these processes and successful EBPI enactment [33, 36–40]. The framework further specifies the role of contextual factors in influencing relationships across the framework’s constructs [22].

Fig. 1.

Theory-based conceptual framework: policy implementation strategies’ effects

Implementation strategies

Counter Tools, a 501c3 nonprofit organization, was launched in 2012 to support community partnership efforts to promote local enactment of POS tobacco EBPI. As detailed below, Counter Tools provides implementation strategies (tools, EBPI guidance, training, and technical assistance) to support partnerships with each of the five core policy change processes.

Tools to document the local problem.

An interactive, electronic mapping tool allows partnerships to find and display the locations of tobacco retailers and additional geospatial data needed for local decision-making (e.g., retailer density, correlations between density and race/poverty, policy solutions’ population reach). A mobile data collection system is provided for partnerships’ use in administering observational surveys (e.g., the Standardized Tobacco Assessment for Retail Settings (STARS) [35]) to document tobacco advertising, prices, and product promotions in the retail environment.

Tools to engage partners.

Pre-packaged engagement toolkits provide detailed guidance and materials that coordinators can use to plan a retailer marketing scavenger hunt, a POS tobacco Photovoice project, and a walking tobacco audit.

Tools to raise awareness and persuade decision makers.

Galleries of store images, maps, and print campaigns provide crowd-sourced photos of the retailer marketing problem as well as exemplar maps and print campaigns created to raise public awareness and gain support for change. PowerPoint slide shows with speaker notes provide adaptable tools for presenting the best available evidence.

EBPI guidance.

Policy solution fact sheets provide current evidence for EBPIs combined with step-by-step guidance on their use, community success stories, and links to additional resources. For each EBPI, guidance includes details on elements to include in a policy proposal to increase its effectiveness. Policy solution domains and sample EBPIs within each domain are listed in Table 1 [8]. Counter Tools’ and EBPI guidance are available at the following website: countertobacco.org.

Table 1.

Point-of-sale tobacco control policy domains and sample EBPIs [8]

1. Licensing and tobacco retailer density
a. Establishing or increasing licensing fees
b. Prohibiting the sale of tobacco at certain establishment types (e.g., pharmacies)
2. POS Advertising
a. Limiting the times during which advertising is permitted (e.g., after school hours on weekdays)
b. Limiting placement of outdoor store advertisements
3. Product Placement
a. Banning self-service displays for tobacco products
b. Limiting times during which products are visible (e.g., after school hours on weekdays)
4. Health warnings
a. Requiring graphic warnings at the point of sale
5. Non-tax approaches to increasing prices
a. Banning price discounting/multi-pack options
b. Establishing cigarette minimum price laws
6. ‘Other’ POS Policies
a. Banning flavored tobacco products
b. Requiring minimum pack size for other tobacco products

Training.

An initial, 2-day, in-person training orients participants to the POS tobacco problem, available EBPIs, and the importance of collecting local data to elucidate the problem and compare EBPI options. Trainings also include hands on experience using Counter Tools’.

Technical assistance (TA).

In-person trainings are followed by monthly TA webinars tailored to the needs of each state and designed to reinforce topics covered in the training and assist partnerships in overcoming any barriers encountered in using the tools. Table 2 provides an overview of the types of topics covered in TA webinars.

Table 2.

Technical assistance webinars delivered to study participants

1. Kick-off: Introduction, Project Timeline, Software Tools Overview
2. Materials/Resource Round-Up
3. Case Study Round Up: What is Happening in POS Across the Country
4. Tobacco Retailer Licensing
5. Store Mapper 101: How to Navigate the Store Mapper
6. Finding your Story: Data Analysis using the Store Mapper
7. Public Opinion Polling
8. Store Audit Center Connections: Using Store Mapper to Analyze Store Assessment Data
9. Telling your Story: Picking the Audience and Frame
10. Refresher: Media Advocacy for Health

PURPOSE

The purpose of this study was to explore changes in community coordinators’ self-efficacy, partnerships’ performance of core policy change processes, and communities’ progress toward EBPI enactment after receiving Counter Tools’ implementation strategies for 1 year. The study’s research questions included:

1. How did partnership coordinators’ self-efficacy change following 1-year exposure to implementation strategies?

2. To what extent did partnerships perform each of the core policy change processes over the year, and how did performance vary across partnerships?

3. How much progress did communities make toward POS tobacco EBPI enactment following 1-year exposure to implementation strategies?

4. What contextual factors do coordinators identify as barriers to progress toward local EBPI enactment?

METHODS

Design

The study employed a pre-test/post-test design. Data collection included surveys and in-depth interviews. An online survey was administered to partnership coordinators at baseline (July 2015) and then again 12 months following initial receipt of implementation strategies (June 2016). In-depth interviews were conducted with partnership coordinators at six (January 2016) and 12 months (June 2016). Each coordinator was required to attend a 2-day, in-person training at the beginning of the project year, 10 monthly 1-h technical assistance webinars (see Table 2), and a half-day virtual summit at the close of the project year.

Sample

The study was conducted with 30 partnerships and their coordinators in a single US state. All 30 partnership coordinators attended the 2-day training and agreed to participate in this study. Partnership coordinators worked for organizations that were funded by the state health department to facilitate tobacco prevention and control activities within a county or cluster of counties. Two-thirds of coordinators worked for local health departments and one-third worked for other types of community-based organization. The POS tobacco partnerships that coordinators facilitated varied, with some working with a small team comprised of members of their organization and a few volunteers and others doing their POS tobacco work in partnership with a well-established tobacco prevention and control coalition. The study was reviewed by the University of North Carolina at Chapel Hill Institutional Review Board and determined to be exempt.

Measures

Study measures included two newly developed measures (coordinator self-efficacy and partnership performance of policy change processes) and one previously developed measure that was applied to assess partnerships’ progress toward and contextual barriers to enacting POS tobacco EBPIs.

New measures: coordinator self-efficacy and partnership performance of core policy change processes.

The two new measures were developed following a literature review [33, 36–39] and interviews with stakeholders working to counter POS tobacco marketing at the state and local levels, to identify the behaviors and activities viewed as essential to successful enactment of POS tobacco EBPIs (unpublished evaluation, 2014). Building on findings from this formative work, the research team created a list of coordinator behaviors and partnership activities required to perform each of the five core policy change processes. We translated the list of coordinator behaviors into a five-scale self-efficacy survey and the list of activities into a structured interview guide.

The measure of partnership coordinator self-efficacy includes 36 items that ask coordinators to rate their confidence to perform specific behaviors on a five-point Likert Scale (see Table 3 for list of survey items).

Table 3.

Partnership coordinators’ self-efficacy paired samples descriptive statistics

	n	Time 1 mean (SD)	Time 2 mean (SD)	p*
Document the Problem
1. Describe how retail environments affect tobacco use	26	4.08 (0.94)	4.89 (0.36)	0.000
2. Describe how retail environments affect smoking quit attempts	26	4.08 (0.94)	4.89 (0.33)	0.000
3. Describe how store assessments document industry targeting	26	3.77 (0.91)	4.69 (0.55)	0.000
4. Track POS changes in community	26	3.23 (0.82)	4.31 (0.62)	0.000
5. Describe effects of food retail environments	26	3.46 (1.07)	4.46 (0.86)	0.002
6. Coordinate store assessments in community	24	3.79 (1.22)	4.58 (0.65)	0.017
7. Use store audit center to create a store assessment campaign	24	3.00 (0.83)	4.50 (0.72)	0.000
8. Use store audit center to assign stores	24	3.17 (0.96)	4.63 (0.58)	0.000
9. Use store audit center to survey stores	24	3.17 (0.96)	4.67 (0.56)	0.000
10. Analyze the store assessment data	24	3.13 (0.95)	4.38 (0.65)	0.000
11. Use store audit center to export data	24	3.00 (0.91)	4.52 (0.73)	0.000
12. Use store audit center for opinion polling	24	3.04 (0.86)	4.17 (0.76)	0.000
13. Teach other local partners to audit stores	24	3.46 (1.10)	4.71 (0.55)	0.000
14. Use store mapper to identify retailers	25	3.48 (0.96)	4.72 (0.46)	0.000
15. Use store mapper to calculate retail density	25	3.24 (1.01)	4.60 (0.58)	0.000
16. Use store mapper to understand tobacco retailer density disparities	25	3.32 (1.07)	4.56 (0.58)	0.000
17. Use store mapper to understand the proximity of retailers to youth	25	3.36 (1.04)	4.64 (0.57)	0.000
Formulate solutions
18. Identify existing POS policies	26	3.58 (0.99)	4.31 (1.01)	0.015
19. Assess strength of local POS policies	26	3.46 (0.91)	4.27 (0.87)	0.005
20. Use store assessment data to plan what POS policies to propose	26	3.35 (0.94)	4.08 (0.74)	0.008
21. Use store mapper to test the impact of potential policy solutions	25	3.28 (0.98)	4.24 (0.78)	0.002
22. Use store mapper to focus youth access enforcement efforts	25	3.44 (1.04)	4.40 (0.58)	0.001
23. Work with team to develop POS tobacco goals and objectives	24	3.83 (0.96)	4.50 (0.83)	0.023
24. Work with team to develop an action plan	24	3.79 (0.93)	4.21 (0.78)	0.076
25. Specify measurable POS objectives	24	3.67 (1.09)	4.21 (0.66)	0.034
Engage partners
26. Present local data on youth appeal of POS tobacco marketing	25	4.04 (0.94)	4.80 (0.41)	0.001
27. Describe how store assessments raise community awareness	26	3.85 (0.88)	4.73 (0.67)	0.001
28. Recruit adult volunteers to assess stores	24	3.92 (1.10)	4.54 (0.78)	0.048
29. Recruit youth volunteers to assess stores	24	4.00 (1.06)	4.46 (0.78)	0.126
30. Engage community in POS tobacco efforts	24	3.75 (1.07)	4.21 (0.93)	0.094
31. Teach others to use the store mapper tool	24	3.42 (1.06)	4.38 (0.82)	0.003
Raise awareness/persuade
32. Use store mapper to find a “story” to share	25	3.28 (0.98)	4.32 (0.75)	0.001
33. Describe at least one goal of sharing store assessment data	24	3.75 (1.03)	4.63 (0.77)	0.005
34. Monitor implementation of action plan	24	3.83 (0.96)	4.38 (0.82)	0.029
35. Build support by educating decision makers	24	3.79 (1.02)	4.46 (0.59)	0.010
36. Earn media coverage to raise awareness	24	3.88 (1.03)	4.21 (0.72)	0.119
Team Lead Self Efficacy—Sum	25	124.28 (26.92)	160.84 (14.65)	0.000

1 = cannot do, 2 = can do with a lot of help, 3 = can do with a moderate amount of help, 4 = can do with a little help, and 5 = can do without any help

POS point of sale

*Paired sample t test

The measure of partnership performance of policy change processes, was modeled on the Stages of Implementation Completion (SIC) measure [40], which documents progress in completing discrete activities involved in each stage of the process of implementing an intervention program. When used to assess the effectiveness of implementation strategies, SIC scores for “proportion of activities” completed were higher for those who received the strategies as compared to a control group [41]. Furthermore, higher SIC scores for proportion of activities completed early in a project were found to predict full implementation of evidence-based intervention programs [42]. In our measure, participants were asked semi-structured questions about whether they had completed a list of activities that were categorized by the five core processes (see Table 4 for list of activities by core process). Interviews were conducted by phone and lasted 15 to 20 min.

Table 4.

Partnership performance of policy change core processes

Core processes	Activities	Completed at time 2;% (n)
Document local problem	1. Map location of tobacco retailers	73.3% (22)
	2. Train volunteers in store audits	93.3% (28)
	3. Store audits completed	96.7% (29)
	4. Analyze local data	63.3% (19)
Formulate evidence-informed solution	5. Assess local policy	80.0% (24)
	6. Assess local officials opinions of POS policy	33.3% (10)
	7. Identify POS policy priorities	23.3% (7)
	8. Draft policy proposal	3.3% (1)
Engage partners	9. First planning meeting	96.7% (29)
	10. Partnership engaged	93.3% (28)
Raise awareness	11. Create promotional materials	23.3% (7)
	12. Participate in/hold events	86.7% (26)
	13. Use social media	6.7% (2)
	14. Create/distribute press release	13.3% (4)
Persuade decision makers	15. Draft a strategic campaign plan	13.3% (4)
	16. Meet with local policy makers	6.7% (2)

Existing measures: progress toward EBPI enactment and contextual factors.

To assess progress toward EBPI enactment the survey included a measure developed by Luke et al. [43], that asked coordinators to categorize the status of a list of 25 POS tobacco EBPI in their communities using one of the following responses: no formal activities, planning/advocating, policy proposed, policy enacted, or policy implemented. To assess contextual factors, the survey included another measure developed by Luke et al. [43], that asks coordinators to identify which of a list of contextual factors impeded their efforts to enact local POS tobacco EBPIs (yes or no). Contextual factors included (1) political will, (2) tobacco industry interference, (3) low awareness of the POS tobacco problem, (4) inadequate funding, (5) lack of capacity or authority, (6) competing priorities, (7) state preemption, (8) lack of evidence, and (9) enforcement issues.

Data analysis

For survey measures, we calculated descriptive statistics at Time 1 and Time 2 (coordinator self-efficacy, community progress toward EBPI enactment, and contextual factors). For the self-efficacy measure, we used paired sample t tests to test for differences and, for the community progress toward POS tobacco EBPI enactment, we used a nonparametric McNemars test with a binomial distribution to test for differences between Time 1 and Time 2. All analyses were performed using IBM SPSS Statistics Version 23 (Armonk, NY). For the interview administered measure of partnership performance of policy change processes, interviews were audio-recorded and then independently coded by two team members (JL, MW) using a directed content analysis approach [44]. Each partnership’s performance of an activity was coded as either completed or not, and new activities were added to the list as they were identified. For the 6-month interviews, coders met to compare coding, revise the list of activities, and further develop the coding guide to specify criteria for determining when an activity qualified as “completed.” For the 12-month interviews, coders compared coding and achieved inter-rater agreement of 95%. We calculated descriptive statistics for activities completed at 12 months by combining data from the 6- and 12-month interviews.

Twenty-six coordinators completed both Time 1 and Time 2 surveys (86.7% response rate). Twenty-six completed the first in-depth interview (86.7% 6-months) and 30 completed the second interview (100% response rate).

Partnership coordinator self-efficacy.

Coordinators’ self-efficacy significantly increased from Time 1 to Time 2 for all items except “recruit volunteers to conduct store assessments,” and “earn media coverage to raise awareness of POS issues” (Table 3). The four items with the lowest self-efficacy at Time 2 (mean = 4.21 for all 4 items) were “work with my partnership to develop a POS action plan,” “specify measurable outcomes/objectives for POS efforts,” “engage community members in POS tobacco efforts,” and “earn media coverage to raise awareness of POS issues.” For many questions, no one reported “cannot do,” restricting response ranges.

Partnership performance of policy change core processes.

We identified 16 activities that partnerships perform, ranging from two to four activities within each of the five core policy change processes (Table 4). At Time 2, completion rates were highest for documenting problems (ranging from 63.3% having analyzed local data to 96.7% having completed their store audits) and engaging partners (with 96.7% having held their first planning meeting and 93.3% having engaged their partnership.) Partnerships conducted between 12 and 252 store audits (Mean number of audits was 89; SD 66). Partnership completion rates were lower for activities in the other three core processes, with four or fewer of 30 coordinators reporting that partnerships had completed the following activities: drafted a policy proposal, used social media, created a press release, drafted a strategic campaign plan, or met with local policy makers.

In developing criteria for what qualified as activity completion, interviews identified wide variations in the ways that coordinators engaged community partners in activities across core processes:

• Coordinators varied in the extent to which they engaged members of their local tobacco control coalitions in their efforts to promote POS tobacco EBPIs, ranging from no engagement, to minimal engagement (e.g., periodically reporting to the coalition), to full engagement such that coalition members were actively engaged in policy change processes.

• Coordinators also varied in the range of partners they engaged in POS tobacco policy change processes, with some only engaging members of their home organization and others engaging representatives from other organizations and volunteers from the community at large.

• Using store audits as a specific example, some coordinators engaged community volunteers and representatives from other organizations (youth-led organizations, community colleges) and others reported using only in-house staff because they could complete audits more accurately and quickly than if they relied on volunteers.

Progress toward POS tobacco EBPI enactment.

None of the coordinators reported that a new EBPI was proposed or enacted over the 1-year project, and no statistically significant changes in progress toward POS EBPIs were identified between Time 1 and Time 2 (Table 5). However trends indicate movement in the preferred direction with more coordinators reporting that they were planning or advocating EBPIs in the domains of licensing and reducing tobacco retailer density (6 partnerships, Time 1 to 9 partnerships, Time 2) and non-tax approaches to raising the price of tobacco products (1 partnership, Time 1 to 7 partnerships, Time 2).

Table 5.

Partnership progress toward POS tobacco EBPI enactment where 0 = no formal activities and 1 = planning/advocating (N = 24)

Policy domain	T1 formal activities % of teams (n)	T2 formal activities % of teams (n)	McNemar Test Exact Sig. (two-tailed) p
1. Licensing and Tobacco Retailer Density	25.0% (6)	37.5% (9)	0.375
2. POS Advertising	33.3% (8)	41.7% (10)	0.727
3. Product Placement	58.3% (14)	58.3% (14)	1.000
4. Health Warnings	16.7% (4)	29.2% (7)	0.508
5. Non-tax Approaches	4.2% (1)	29.2% (7)	0.070
6. “Other” POS policies	45.8% (11)	45.8% (11)	1.000

Contextual factors.

Figure 2 summarizes survey findings on contextual factors that coordinators reported as barriers to EBPI enactment at Time 1 and Time 2. With the exception of political will, the number of coordinators identifying each contextual factor as a barrier was lower at Time 2 than at Time 1, with drops of 10% or more in the proportion of coordinators reporting that they encountered the following barriers: low public awareness, industry interference, competing priorities, inadequate funding, or limited capacity to address the POS tobacco marketing problem.

Fig. 2.

Contextual factors coordinators identified as barriers to EBPI enactment

DISCUSSION

Enacting POS tobacco EBPIs is essential to reducing community members’ exposure to tobacco marketing and tobacco use rates. Community partnerships have the potential to play a central role in promoting the enactment of POS tobacco EBPIs, and Counter Tools is providing an integrated set of implementation strategies to support partnerships’ efforts. This study documented changes in partnership coordinators and in partnership activities after receiving Counter Tools’ strategies for 1 year. The study found that partnership coordinators’ confidence in their ability to facilitate partnership efforts (i.e., self-efficacy) increased significantly. The study also identified the activities that partnerships performed across each of five core policy change processes and documented variations in the completion of those activities across partnerships. No new EBPIs were proposed or enacted in the partnership communities, which is unsurprising given the study’s 1-year time frame. However, partnerships did start working toward (planning and advocating) the enactment of new EBPIs. Notably, the number of partnerships working on EBPIs in the domains of “non-tax approaches to pricing” and “reduce or restrict the number, location, or density of tobacco retailers” increased from one to seven and from six to nine respectively. These two EBPI domains are among those with the greatest potential to reduce the impact of POS marketing on tobacco use rates [45, 46]. Finally, with the exception of “policy will,” fewer coordinators reported encountering each of the contextual-level barriers to EBPI enactment after receiving 1 year of implementation strategies as compared to baseline. This may reflect a change in the coordinators’ knowledge or beliefs about the barrier (e.g., “lack of evidence”) or actual changes to the local context (e.g., “low awareness of the POS problem”) that may have resulted from the partnership’s work. The fact that “political will” was identified more frequently as a barrier at time two is a concern since it is a central driver of policy change [30]. Further research is needed to understand to what extent changes in identified barriers reflect a change in coordinators’ knowledge or beliefs versus a change in the local context, and whether changes to the local context are the result of partnerships’ policy change processes.

Study findings on self-efficacy and partnership performance of core policy change processes offer insight into the areas where partnership coordinators may require additional support. Time 2 scores were lowest for self-efficacy and activity-completion related to three of the five core processes: formulate solutions, raise awareness, and persuade decision makers. Policy change processes are expected to take up to 3 years to complete and therefore low 1-year completion rates of some processes is not surprising. However, the low self-efficacy rates suggest that coordinators may benefit from greater exposure to implementation strategies addressing these processes. Although the majority of partnerships completed the engage partners core process activities, interviews revealed that a number of coordinators reported limited community engagement in activities specified for other core processes (e.g., store audits). Engaging community partners is key to increasing awareness of and building political will to address the problem [30, 31]. Thus, coordinators may need additional training, tools, and other implementation strategies to support their efforts to engage community partners across core processes. These findings also reveal that simply documenting whether or not partnerships performed an activity may not capture differences in the level of community engagement in those processes. Future assessments of partnership performance of policy change processes would benefit from greater attention to engagement across activities.

The limited research that has been done on partnership efforts to change local policy has focused on long-term outcomes, specifically, the enactment of new EBPI [33]. An overreliance on long-term outcomes is problematic because policy enactment may not be possible within the 3- to 5-year time frame of most funded research studies, and typically results from multiple factors, thereby limiting efforts to attribute enactment to the effects of implementation strategies [47–51]. The new measures developed in this study assess intermediate outcomes (self-efficacy and partnership performance) and therefore overcome the challenges inherent in relying on policy enactment as the primary measure of implementation strategy effectiveness. Findings from these intermediate outcome measures provide milestones that might be used to identify partnerships at risk for failure early in the process. The study’s intermediate outcomes also advance understanding of implementation strategies’ mechanisms of change—understanding that is critical to advancing theory, optimizing strategy effectiveness and efficiency, and tailoring strategies to different contexts [51, 52]. A measure of partnership performance of core policy change processes also could be used to motivate, strengthen, and sustain partnerships by providing ongoing feedback on partnerships’ progress on the road to policy enactment [31].

The present study was limited by its single group pre-test/post-test design and its short duration. In the absence of a control group, caution should be taken in attributing changes between Time 1 and Time 2 to the effects of the implementation strategies. Study measures are in the initial stages of development and future research is needed to establish their validity and reliability. Both the self-efficacy and partnership performance measures include multiple items organized around the five core policy change processes. Future studies with larger sample sizes are needed to further explore and confirm the measures’ factor structure. Furthermore, future studies might complement self-report measures, such as those reported, with more objective measures of both policy change processes and progress toward policy enactment. For example, data on media coverage of POS tobacco might serve as a marker of partnerships’ performance of the core policy change process “raise awareness.” To document progress toward policy enactment, town and city council minutes might be searched for evidence that a policy was proposed and/or enacted.

The contextual factors included in this study do not capture all of the factors that may influence a community partnership’s success. Characteristics of community partnerships, for example, are known to influence their effectiveness, including levels of member participation, member diversity, and group cohesion among others [53]. Future research is needed to identify and control for salient characteristics of the partnership. Future research also is need to test the effectiveness of Counter Tools in a randomized trial over a 3- to 5-year time period and to test relationships among the constructs that are hypothesized in this study’s theory-based conceptual model (Fig. 1) such as the role that coordinator self-efficacy and partnership performance play in mediating implementation strategies’ effects on the local enactment of EBPIs.

CONCLUSIONS

Study findings can inform the development of tools, training, and other implementation strategies to promote local enactment of EBPI. The study’s theory-based conceptual framework and measures might contribute to future tests of implementation strategy effects on community partnerships’ performance of core policy change processes and success at enacting new EBPI.

This study’s theory-based conceptual framework and measures have relevance not only to POS tobacco EBPIs but also to implementation strategies for EBPIs to create environments that support healthier eating, increased physical activity, and other healthy behaviors.

Compliance with ethical standards

Funding

The project described was supported by the Centers for Disease Control and Prevention (CDC) and the National Cancer Institute/NIH, through Cooperative Agreement Number U48 DP005017-SIP to the Center for Health Promotion and Disease Prevention at the University of North Carolina at Chapel Hill. The content is solely the responsibility of the authors and does not necessarily represent the official views of the CDC or NIH.

Conflict of interest

Two of the authors (Myers and Grant) are employed by the organization whose implementation strategies are being studied. The lead author (Leeman) and two other authors (Wangen and Queen) declare that they have no conflict of interest.

Ethical approval

All procedures performed were in accordance with the ethical standards of the University of North Carolina at Chapel Hill Institutional Review Board and with the 1975 Helsinki declaration and its later amendments or comparable ethical standards. UNC’s IRB reviewed study protocols and determined the study to be “Not Human Subjects Research” because data were not being collected on participants’ personal information and participants were only asked for information related to the performance of their jobs.

Informed consent

As noted above, this study was not human subject research.