Formative Process Evaluation for Implementing a Social Marketing Intervention to Increase Walking Among African Americans in the Positive Action for Today’s Health Trial

Abstract

Objectives. Evaluating programs targeting physical activity may help to reduce disparate rates of obesity among African Americans. We report formative process evaluation methods and implementation dose, fidelity, and reach in the Positive Action for Today’s Health trial.

Methods: We applied evaluation methods based on an ecological framework in 2 community-based police-patrolled walking programs targeting access and safety in underserved African American communities. One program also targeted social connectedness and motivation to walk using a social marketing approach. Process data were systematically collected from baseline to 12 months.

Results: Adequate implementation dose was achieved, with fidelity achieved but less stable in both programs. Monthly walkers increased to 424 in the walking-plus-social marketing program, indicating expanding program reach, in contrast to no increase in the walking-only program. Increased reach was correlated with peer-led Pride Strides (r = .92; P < .001), a key social marketing component, and program social interaction was the primary reason for which walkers reported participating.

Conclusions: Formative process evaluation demonstrated that the walking programs were effectively implemented and that social marketing increased walking and perceived social connectedness in African American communities.

Intervention programs targeting neighborhood walking and physical activity (PA) have the potential to prevent obesity in underserved African American communities.^1–3 African Americans experience the highest rates of obesity and the lowest rates of PA,^1,4 and face increased barriers to PA, including a lack of access to safe places to be active and a lack of community support for PA.^5–7 Formative process evaluation can inform program implementation and may lead to more effective health interventions for PA and obesity. In addition, social marketing (SM) approaches have been linked to increased effectiveness of health promotion interventions.^8–11 The purpose of this study was to describe the application of systematic evaluation methods in the Positive Action for Today’s Health (PATH) trial, which adopted an innovative SM approach to promote walking in underserved (low-income, high-crime) African American communities,¹² and to report evaluation data from the PATH walking programs. An additional aim was to present a framework for the evaluation of SM approaches that target a variety of public health issues.

Consistent with an ecological approach,¹³ the PATH trial developed a community-based intervention to address access and safety barriers to neighborhood walking in underserved African American communities, with one program including an additional SM component to increase social connectedness and motivation for walking. Process evaluation methods were developed to assess program implementation and reach. Social marketing strategies^14,15 were developed with community input,¹² and focus groups conducted in the PATH communities suggested that the strategies would meet community needs and would be culturally relevant.^16,17 Additionally, the utility of incorporating evaluation methods based on ecological theory has increasingly been recognized,^18–21 as has the effectiveness of SM approaches.^8–10 Although ecological factors significantly influence implementation practices and program outcomes,²² fewer than half of the studies targeting minority PA have evaluated them,²³ and only a handful of studies implementing SM strategies have reported summative^24–29 or formative process evaluation outcomes.^8,23,30 Such limitations can decrease the accuracy with which program mechanisms, efficacy, and effectiveness are understood.

As community-based interventions become more complex in design (e.g., by focusing on multiple ecological levels and components) and integrate strategies from varied disciplines (e.g., public health, marketing), evaluation becomes increasingly necessary for determining which levels and components affect outcomes, and to what extent,³¹ before dissemination.³² Evaluation of program implementation relies on the measurement of program dose, fidelity, and reach, which, respectively, quantify the completeness of program implementation, whether a program is delivered as intended, and the percentage of the target group engaged in the program.

A growing literature has reported evaluation methods for community-based PA programs targeting African American adults. The WATCH project promoted increased PA through participation in church-based walking groups and used process evaluation to demonstrate that intervention activities were implemented with varying degrees of dose and fidelity.³³ The Health-e AME church–based, multisite intervention assessed implementation and indicated that fidelity varied by site and was positively associated with meeting PA recommendations.³⁴ In the H.U.B. City Steps and Healthy Body, Healthy Spirit community-based trials that targeted walking and PA, formative process evaluation methods were based in part on an ecological framework, with ongoing feedback provided to counselors to address lapses in fidelity.^35,36 Finally, an intervention targeting PA adopted a media-based, community-based prevention marketing approach to promote walking and PA among women.³⁷ By contrast, the PATH trial adopted a community-based grassroots marketing approach and focused on neighborhood access, safety, and social connectedness rather than individual factors. The media-based program used process evaluation to demonstrate that campaign exposure significantly increased walking, PA, and use of a walking trail.³⁷

In this study, we expand on previous research by reporting formative process evaluation for monitoring the implementation of a community-based intervention that integrated grassroots SM strategies to improve perceptions of access and safety, and to increase social connectedness and motivation for neighborhood walking in underserved African American communities. We present evaluation methods and dose, fidelity, and reach data from baseline to 12 months in both PATH walking programs.

METHODS

We matched 3 communities by census tract on demographics such as crime, percentage of African American residents, and poverty.¹² We considered communities that experienced high levels of chronic disease, poverty, and crime relative to state and national averages to be underserved. For example, 23% to 39% of households in the selected tracts experienced poverty compared with national rates of 15%.³⁸ Communities were located in the southeastern region of the United States, with 1 or 2 census tracts targeted in each community, 5 targeted for the study, and approximately 45 to 90 miles separating each community. We considered communities to be rural on the basis of urban–rural classification criteria outlined by the US Census Bureau.³⁹ African American adults (aged 18 years or older) residing in specified census areas were enrolled in the PATH sample if they were free of medical conditions limiting participation in moderate intensity exercise⁴⁰ and did not have uncontrolled high blood pressure (< 180 mm Hg systolic, < 110 mm Hg diastolic) or blood sugar (< 300 mg/dL nonfasting, ≤ 250 mg/dL fasting). Participants were compensated $30 during the baseline measurement.

Study Design and Recruitment

The PATH trial was designed to evaluate the efficacy of a 24-month environmental intervention for increasing walking and PA by targeting access, safety, and social connectedness and motivational barriers in underserved African American communities.¹² Communities were randomized to a police-patrolled walking-plus-SM program (full intervention; walking-plus-SM), a police-patrolled walking-only program (i.e., no SM component; walking-only), or a health education program (health comparison program). PATH participants completed measures during health screenings occurring every 6 months. We designed PATH protocols consistent with a community-based participatory approach to ensure positive community relations and engagement, complete and valid measures, and increased participation in the walking programs. PATH community liaisons were key in building and maintaining community connections, and the health screenings were viewed as important resources for participants. These strategies resulted in high retention rates, consistent with or better than those of other community-based trials that implemented health interventions in minority populations.^41–43

Residents were recruited using random-digit dialing (54%) and through volunteer mechanisms (46%). The recruited sample is not inclusive of all individuals who participated in the PATH walking programs, although sampling procedures aimed to represent individuals from the larger community. This report focuses only on baseline to 12 months of the 2 walking programs.

Intervention

Theory and framework.

The intervention was based on an ecological framework,¹³ and program components targeted (1) access for walking in the neighborhood, (2) positive practices and perceptions related to safety, and (3) residents’ social connectedness and motivation for walking. We operationalized program components by defining essential program elements (Figure 1).

FIGURE 1—

Program components and essential elements for the walking-plus-social marketing program: Positive Action for Today’s Health Trial, United States.

Note. SM = social marketing. The walking-only program included access and safety program components only.

Staff and intervention implementation.

Local community centers operated as neighborhood hubs for program activities. Walking trails that began and ended at each center were identified. Walking leaders were trained in injury prevention and safety protocols and scheduled and led ongoing walks (e.g., Monday–Friday at 7:00 am). Program leaders worked with county officials to control stray dogs and address ongoing structural and aesthetic barriers on the trails, and off-duty police officers were hired to attend walks. To address seasonal and daily weather fluctuations, walking leaders used protocols directing them to lead indoor walks or aerobic activities (e.g., group exercise) during rainy or exceptionally cold (≤ 32°F) or hot (≥ 90°F) weather. Communities generally experienced hot, humid summers and mild winters, consistent with a humid, subtropical climate region.

Social marketing approach.

For the walking-plus-SM program, we developed a community-based grassroots campaign in collaboration with a SM firm. Five messages framed the campaign content, conveying that walking was safe, benefited physical health, benefited mental and spiritual health, promoted social connectedness, and promoted confidence (self-efficacy) for walking. The PATH Steering Committee and community residents developed print materials, and a 12-month calendar was distributed to deliver campaign messages. The calendar featured photographs of residents walking on the trail and also provided goal-setting opportunities for walking. Door hangers were delivered to residents’ homes to invite them to walk, to reinforce campaign messages, and to highlight program incentives (e.g., hand-held fan, pedometer) that walkers could earn when walking goals were met.¹² Grassroots networking was facilitated through the development of “Pride Stride” walks, which empowered community walkers to organize and lead special-event walks that promoted the campaign messages and engaged new walkers, with the support of the walking leaders.

Process Evaluation Procedures

Research staff attended program walks weekly (walking-plus-SM) and every other week (walking-only) to rate essential elements using internal process evaluation surveys. Staffing and budgetary constraints prevented weekly assessments in both programs, and we therefore paid greater evaluative attention to the walking-plus-SM program, given hypotheses that inclusion of the SM campaign would result in the greatest increase in walking.¹² Because the walking-only program did not include an SM component, we did not target social connectedness and motivation for walking.

After walks, supportive, corrective feedback was given to ensure ongoing achievement of implementation dose and fidelity. For example, feedback was provided if an exercise warm-up, part of the injury prevention protocol, was not completed before beginning a walk. Intervention staff reviewed data collected through these internal evaluations immediately.

Walking leaders attended all walks and documented walkers who attended. We also calculated the correlation of walk attendance with Pride Strides as part of the grassroots SM campaign in the walking-plus-SM program to assess the relation of reach to implementation and the SM strategy. Finally, a small sample of walkers reported their perceptions of the essential elements to supplement findings from process evaluation data.

Process Evaluation Measures

Internal process evaluation surveys.

Surveys included 12 key items. Adequate dose or fidelity was achieved if (1) items with yes–no response options were endorsed in the desired directions during 75% or more of evaluations and (2) items with 3-point scale response options had mean values of 2.50 or greater. Items were linked to dose if they evaluated the degree to which essential elements were completed and linked to fidelity if they evaluated whether the program was being implemented to support improvements in walking trail access and safety, as intended.

Walk attendance logs.

Walking leaders recorded walker information at each walk. Although Pride Strides often occurred in tandem with regularly scheduled walks, walkers were not counted as participants for both walks but as participants for either the Pride Stride or the regular walk on the basis of their reported affiliation with one or the other. We quantified program reach by using monthly sums of the number of walkers at each walk.

Walker surveys.

Surveys included 9 key items with yes–no response options that elicited perceptions of essential program elements. Program staff administered surveys to walkers and ensured the anonymity of walker responses. We included surveys completed during months 9 through 13 because we considered program exposure and implementation to be stable by months 9 through 12, and we determined that responses collected during that time and immediately afterward would be highly relevant to evaluation outcomes and future directions for the program.

RESULTS

Baseline characteristics for the walking-plus-SM (n = 133) community indicated that PATH participants were predominantly female (67%), with an average age of 54.2 years (SD = 15.6) and a body mass index (defined as weight in kilograms divided by the square of height in meters) of 31.1 (SD = 7.9). In the walking-only (n = 164) community, participants were predominantly female (65%), with an average age of 48.1 years (SD = 15.6) and a body mass index of 30.3 (SD = 8.3), as described previously.¹² The PATH sample aimed to represent the larger community and was not inclusive of all individuals who participated in the walking programs.

Intervention Dose

Data related to access indicated that in the walking-plus-SM program, adequate dose was achieved across all quarters and for all essential elements (Table 1). In the walking-only program, dose was achieved at rates similar to the walking-plus-SM program, with use of the neighborhood walking trail improving after the first quarter. Data related to safety indicated that the walking-plus-SM program achieved adequate dose across all quarters. In the walking-only program, adequate dose was achieved during all quarters except the fourth, during which the mean evaluation score assessing whether the trail was free from stray dogs decreased from 2.57 to 1.50. Data related to program social connectedness and motivation to walk indicated that adequate dose was achieved across all quarters in the walking-plus-SM program, with SM campaign messages reaching 81% to 87% of targeted walkers.

TABLE 1—

Quarterly Process Data for Implementation Dose and Fidelity from Baseline to 12 Months of the Walking Programs: Positive Action for Today’s Health Trial, United States, February 2009–January 2010

	1st-Quarter Outcomes		2nd-Quarter Outcomes		3rd-Quarter Outcomes		4th-Quarter Outcomes
Program Component, Essential Element, and Measure and Item	W + SM	WO	W + SM	WO	W + SM	WO	W + SM	WO
Implementation dose
Access for walking in neighborhood (walks completed on the identified walking trail),a % (no.)	100 (4)	50 (2)b	100 (12)	78 (10)	100 (12)	100 (6)	100 (11)	100 (5)
Safety and positive perceptions of safety
Walks at which police present,a % (no.)	100 (4)	100 (2)	92 (12)	100 (9)	100 (12)	100 (6)	82 (11)	100 (5)
Scores for presence of stray dogs,ac mean (no.)	3.00 (4)	3.00 (2)	2.91 (11)	2.88 (8)	2.64 (11)	2.67 (6)	2.57 (7)	1.50 (2)b
Resident social connectedness and motivation to walk,d % (no.)
Targeted walkers received a calendara	…	…	…	…	…	…	87 (134)	…
Targeted walkers received a door hangera	…	…	…	…	…	…	81 (134)	…
Implementation fidelity
Access for walking in neighborhood,e mean (no.)
Scores for presence of overgrowth and sidewalk qualityac	2.00 (4)b	2.00 (2)b	2.82 (11)	2.63 (8)	2.73 (11)	2.00 (6)b	2.57 (7)	2.00 (2)b
Scores for presence of litter and vandalismac	2.00 (4)b	2.50 (2)	2.55 (11)	2.50 (8)	2.09 (11)b	2.33 (6)b	2.57 (7)	2.00 (2)b
Safety and positive perceptions of safetyf
Times WLs followed pedestrian safety rules,a % (no.)	100 (4)	50 (2)b	67 (12)b	86 (7)	75 (12)	67 (6)b	75 (8)	67 (3)b
Times WLs completed 2 of 3 injury prevention protocols,a % (no.)	75 (4)	0 (2)b	55 (11)b	63 (8)b	92 (12)	83 (6)	30 (10)b	100 (3)
Percent times walkers were not injured,a % (no.)	100 (4)	100 (2)	100 (12)	100 (9)	100 (12)	100 (6)	100 (11)	100 (5)
Mean scores for walker comfort,ac mean (no.)	3.00 (3)	3.00 (2)	2.78 (9)	3.00 (7)	3.00 (10)	3.00 (5)	2.75 (8)	3.00 (3)
Resident social connectedness of program and motivation to walkg
Targeted walkers reached by phone and received program information,a % (no.)	…	…	…	…	36 (134)	…	34 (134)	…
PSs per quarter completed/scheduled,h % (no.)	…	…	100 (1)	…	93 (42)	…	89 (56)	…
Total walks that were PSs,h % (no.)	…	…	2 (62)	…	40 (106)	…	42 (134)	…
Association of total walkers with PSs,h r (no.)	…	…	.44 (59)	…	.61 (102)	…	.39 (125)	…

Note. PS = Pride Stride; SM = social marketing; W + SM = walking-plus-social marketing; WL = walking leader; WO = walking-only. Ellipses indicate that data were not collected during this quarter.

^aItems were collected from internal process evaluation surveys.

^bOutcomes for which adequate dose or fidelity was not achieved.

^cItem scores ranged from 1 to 3, with 1 indicating the greatest presence of trail barriers, and 3 indicating the least.

^dSM campaign will promote messages for walking to increase motivation. Program component was relevant only to the W + SM program, resulting in no data for the WO program.

^eWalking trail free from structural and aesthetic barriers.

^fWalking leaders adhere to personal safety guidelines and walkers are comfortable with police.

^gSM campaign increases connectedness and SM campaign promotes PSs to increase walking. Program component was relevant only to the W + SM program, resulting in no data for the WO program.

^hItems were collected from walking attendance logs.

Intervention Fidelity

Data related to access indicated that in both walking programs, fidelity was initially inadequate and subsequently improved, with the exception of the presence of litter and vandalism (Table 1). During the first quarter, data indicated that the walking trails were not free from the presence of overgrowth, litter, and vandalism, and did not have adequate sidewalks, with scores of 2.00 failing to meet the 2.50 criterion indicating adequate fidelity. Scores fluctuated during subsequent quarters. In the walking-only program, access fidelity was achieved at rates slightly lower than in the walking-plus-SM program, with the presence of overgrowth, litter, and vandalism problematic in both the 3rd and the 4th quarters. Data related to safety indicated that fidelity varied by essential element, with pedestrian safety and injury prevention protocols followed inconsistently in both programs at rates ranging from 0% to 100%. However, no walkers were injured in either program, and no incidents needed to be reported to the institutional review board. Additionally, police were present at 100% of walks during most quarters, with police presence never lower than 81%. Data related to social connectedness and motivation to walk indicated difficulty reaching participants by phone in the walking-plus-SM program.

Intervention Reach

Intervention reach in the walking-plus-SM program grew over time, as intended. The total number of walkers increased steadily into the 4th quarter, with as many as 424 total walkers in 1 month (Figure 2). Monthly totals for Pride Strides and walkers were positively correlated (r = .92, P < .001). In the walking-only program, reach did not increase over time, with the total number of walkers relatively consistent over each month and averaging 41 walkers (SD = 22).

FIGURE 2—

Total walkers by month in the walking programs from baseline to 12 months: Positive Action for Today’s Health Trial, United States, February 2009–January 2010.

Note. SM = social marketing.

Walker Survey Results

Walkers in the walking-plus-SM program reported positive perceptions of access, safety, social connectedness, and motivation components 87% to 100% of the time (Table 2). Additionally, social interaction was the number 1 reason for participating in regular walks (48%), compared with reasons such as health (25%) and feeling safe on the trail (13%). Data from the walking-only program are not reported because too few walkers were participating in the program at the time of data collection.

TABLE 2—

Walkers’ Perceptions of Program Components in the Walking-Plus-Social Marketing Program from Baseline to 12 Months: Positive Action for Today’s Health Trial, United States, February 2009–January 2010

Program Component Perceptionsa	% (No.) or Mean (No.)
Increased access for walking in neighborhood
Familiar enough with the trail to walk it independently	100 (15)
Enjoys scenery on the trail	87 (15)
Increased safety and positive perceptions of safety
Feels safe on the trail	100 (15)
Feels supported by police	100 (15)
Wants police support	100 (14)
Increased resident social connectedness of program and motivation to walk
Social interaction is the main reason for participation	48 (13)
Likes door hangers	89 (9)
Likes calendars	100 (15)
Program enjoymentb	9.07 (15)

^aItems endorsed dichotomously as “yes” or “no”; percentages represent the proportion of “yes” endorsements.

^bItem scores ranged from 1 to 10, with 1 indicating boredom and 10 indicating fun.

DISCUSSION

We presented the use of formative process evaluation for ongoing implementation monitoring of the walking programs and innovative SM campaign in the PATH trial. Based on an ecological framework, evaluation outcomes demonstrated that adequate dose, fidelity, and reach were generally achieved over time; the SM campaign led to substantial increases in walking program participation; and that walkers had positive perceptions of the program in the walking-plus-SM community. Thus, results support the effectiveness of the process used to implement the intervention rather than simply the content of the intervention. Such a framework can be applied to programs aiming to address a range of public health issues and may be tailored to meet the needs of low-income minority communities. In addition, social connectedness seemed consistently important to walkers and may have increased community capacity for promoting walking and health.

During the first 12 months of the PATH intervention, adequate dose was achieved for program components in both walking programs during a majority of quarters; implementation fidelity was achieved in a somewhat less consistent pattern. For fidelity, both communities overall were strong in access and many of the safety-related elements, although a few areas of weakness were found related to trail quality (access) and walking warm-ups (safety). However, data from walker surveys indicated that walkers’ perceptions of essential elements were positive, supporting evaluation evidence indicating that the walking-plus-SM program achieved adequate dose and fidelity.

Trail quality issues were recurrent in these underserved settings and were addressed through ongoing problem solving by community leaders, who developed an infrastructure for collaborating with local officials. Although implementation fidelity did not vary by program for most elements, the walking-only program experienced slightly increased challenges in trail quality related to the presence of overgrowth, litter, and vandalism and the quality of sidewalks (e.g., presence of cracks, uneven pavement). Both programs achieved moderate fidelity for personal safety, with injury prevention and safety protocols resulting in no walking injuries or safety threats. However, both programs struggled to consistently integrate protocols related to walking warm-ups and stretches, despite ongoing corrective feedback provided during evaluations. We observed that both walkers and walking leaders occasionally perceived some protocols as unnecessary or burdensome. Such perceptions likely gained influence over time as walkers and walking leaders began to take greater ownership of the programs. These challenges highlight the importance of incorporating flexibility into community-based intervention protocols.

In the PATH trial, walking leaders were supported in developing alternative protocols when they found existing protocols to be a poor fit for the walking group or for individual walkers. Walking leaders in the walking-plus-SM program relied less on police support over time, with police presence at walks dropping to 82% in the final quarter. This percentage did not fall below the criterion for adequate dose, and it may be a positive indicator of walkers’ increased comfort and feelings of safety on the walking trail. Efforts to decrease dependence on police support may also have been driven by goals to adopt sustainable protocols and to accommodate walkers’ varied schedules and requests to increase the number of walks.

Intervention reach differed greatly by program, with the walking-plus-SM program experiencing increased reach over time and consistently engaging more walkers than the walking-only program. When inclement weather may have prevented walking, walking leaders led indoor walks and exercises to ensure continuity of the programs. In general, the communities’ input on how to cope with inclement weather by being active indoors ensured that reach of participation decreased only slightly because of bad weather or seasonal effects.

The walking-plus-SM program demonstrated a large relationship between the number of completed Pride Strides and program reach, indicating that effective implementation of the social connectedness and motivation component may have contributed to increased program participation. The grassroots approach of the SM campaign may also have supported Pride Stride walkers in remaining engaged with the program and neighborhood, and in walking more regularly, as was often observed. Targeting social connectedness may have positively affected community resources such as social capital and collective efficacy, which have been linked to increased community capacity and positive health behaviors and outcomes.^44–47 The community-based participatory approach supported participants in providing input on program components (e.g., in the PATH trial, police support, SM messages), which may have fostered program engagement, increased participant retention, and increased the likelihood of collecting valid, reliable outcomes (e.g., accelerometry-estimated PA over 7 days) over the 24-month intervention.

The barriers of access and safety that were targeted, including structural and aesthetic neighborhood conditions and the presence of crime or stray dogs, have been linked to walking and PA,⁴⁸ although not all studies have supported these relations.⁴⁹ Operationalizing environmental factors such as access and safety can be challenging, but these steps are a necessary precursor to evaluation and can aid in building theory-based programs. Additionally, a community-based approach that incorporates ongoing community feedback facilitates the tailoring of process methods as described, ensuring that they are applicable to a variety of public health initiatives and across culturally diverse populations.²¹

The evaluative findings of the PATH trial are consistent with recommendations to incorporate systematic evaluation into community-based programs³² and public health programs targeting African American communities.⁵⁰ Findings are also consistent with studies suggesting that process evaluation can be used to understand outcomes for PA programs targeting African Americans^34,35,36 and to assess the implementation of SM strategies to increase PA.^26,27,51 Additionally, we confirmed previous evidence that exposure to a marketing campaign can increase walking and PA.³⁷ This study is the first to our knowledge to provide evidence that using a grassroots SM approach to promote PA in African American communities is feasible and, based on these evaluation findings, effective. Additional value added from this study is the demonstration that evaluation frameworks and criteria (e.g., whether adequate dose and fidelity are achieved) can be tailored to communities to adapt to their changing needs while maintaining fidelity to the core components of an intervention. This tailoring ensures that programs remain relevant and engaging, as was achieved in the PATH trial.

This study is one of the first to report the use of formative process evaluation methods in an intervention that targeted underserved African American communities and incorporated an innovative social marketing approach. Similar evaluation methods may ensure quality implementation in future studies and contribute to the prevention of obesity in underserved communities.