The StrongWomen–Healthy Hearts program in Pennsylvania: RE-AIM analysis

Abstract

Dissemination of evidence-based programs is needed to reduce CVD risk among midlife and older women. The aim of this study is to examine the public health impact of StrongWomen–Healthy Hearts in Pennsylvania using the RE-AIM framework. Reach, adoption, implementation, and maintenance were assessed using qualitative and quantitative measures; effectiveness was assessed using a pretest-posttest within-participants design. Reach into the target population was 5 in 100,000. Compared to the target population, a greater percentage of participants were white, married, middle-class, and had a graduate degree. Effectiveness was demonstrated (weight loss −2.0 kg, p < 0.001). Adoption among trained leaders was high (83.3 %), as was fidelity in implementation (average score 9.3 of 10). No leaders maintained the program. To increase impact of the StrongWomen–Healthy Hearts Program, it will be important to lower the costs and modify the recruitment and training strategies to better reach low-income and minority women. Such strategies may also improve program maintenance.

INTRODUCTION

Cardiovascular disease (CVD) is the leading cause of death and disability for women in the USA, claiming approximately 400,000 female lives a year [1]. The annual direct and indirect costs of CVD and stroke are estimated to be nearly $300 billion [1].

Although CVD develops over decades and prevention is important early in life, lifestyle modifications reduce risk at any age, even in older adults [2]. One public health approach to address this problem is educational and behavioral programs tailored to midlife and older women that can be implemented widely by community organizations. Few such programs currently exist.

StrongWomen–Healthy Hearts is a 12-week, evidence-based community program addressing CVD risk factors that has demonstrated effectiveness in a controlled, randomized trial conducted with midlife and older overweight and obese women in Arkansas and Kansas [3]. The intervention group saw an average weight loss of more than 4 lb, a decrease in overall caloric intake, saturated fat and cholesterol intake, and an increase of over 1600 steps per day, all of which were significant compared to the control group. The program classes are conducted by local health educators herein referred to as “leaders.”

To make any type of impact on CVD risk among the target population, it was essential that the StrongWomen–Healthy Hearts Program move beyond the limited trial to reach larger numbers of women. We designed a blueprint for dissemination in Pennsylvania and utilized the RE-AIM framework [4] to conduct a rigorous, prospective evaluation using mixed methods (quantitative and qualitative). RE-AIM encompasses the components Reach (participation rate and representativeness), Effectiveness (impact on outcomes), Adoption (participation rate of intervention leaders), Implementation (consistency of delivery by intervention leaders and costs), and Maintenance (continuing to offer the intervention over time).

Despite the increasing use of the RE-AIM framework to report on dissemination efforts, few studies were designed to rigorously evaluate each of its components [5, 6]. This study, specifically grounded in RE-AIM framework, sought to gain extensive information on best practices for the StrongWomen–Healthy Hearts Program’s in Pennsylvania to inform national dissemination so that the program would have maximal possible impact in reducing CVD burden. The purpose of this paper is to report these findings and to provide key lessons learned related to the process of translating an effective intervention into widespread programming to reduce the risk of CVD among women.

METHODS

Design

Reach, adoption, implementation, and maintenance were assessed using both qualitative (key informant interviews and observation) and quantitative (survey) measures, while effectiveness was assessed using a pretest-posttest within-participants design. Study procedures were reviewed and approved by the institutional review boards at Pennsylvania State University and Tufts University.

Sample

This dissemination study leveraged a long-term collaboration between Tufts University and Pennsylvania State University’s Cooperative Extension Service (herein referred to as Penn State University or Penn State Extension). Penn State Extension educators comprise a statewide network of experts who provide information and services, including health programming, to meet public needs at a local level. The original focus of the collaboration was the StrongWomen Strength Training Program, which was designed to increase women’s access to regular strength training opportunities [7]. In addition to this history of successful collaboration, Pennsylvania was chosen because it is demographically similar to the overall US population and offered an opportunity to explore both rural and urban settings [8]. In Pennsylvania, the age-adjusted death rate from CVD is 271.4 per 100,000, slightly higher than the national average of 262.7 [10].

To begin the dissemination process, 33 Penn State Extension educators who had been involved with the StrongWomen Strength Training Program were invited to attend a full-day training at Penn State University, University Park, PA, which took place in March 2010. Eighteen leaders attended. The workshop consisted of a series of seminars to train leaders on both the program and the assessment methods. All attendees received a copy of the curriculum which includes a detailed class-by-class guide.

Trained leaders who decided to run the program recruited participants for their classes. The participant sample, per eligibility requirements, was female, aged 40 years or older, sedentary, without contraindications to exercise (determined by the Physical Activity Readiness Questionnaire [9] or physician approval), had a body mass index (BMI) of 24.0 or higher, and was not participating in another lifestyle change program.

Intervention

The StrongWomen–Healthy Hearts Program is designed to take place 2 days per week for 12 weeks. Thirty minutes of each class includes aerobic dancing to a DVD created for the project or walking outside if location and weather permit. The other 30 min includes leader-directed and hands-on discussion and activities to modify dietary intake patterns, as well as weight control strategies such as portion control and keeping food logs. The Program emphasizes an eating pattern that is consistent with current American Heart Association guidelines and the Dietary Guidelines for Americans. The theoretical basis for the intervention is Social Cognitive Theory [10]. Thorough formative research helped inform development of the curriculum [11]. The program (ClinicalTrials.gov #NCT01439477) is listed on the National Cancer Institute’s Research-Tested Intervention Programs (RTIPS) website (http://rtips.cancer.gov/rtips/programDetails.do?programId=1194691). More information about the program itself and an on-line training for qualified leaders are available at strongwomen.org.

Recruitment

We sought initial IRB approval for a broad range of recruitment methods that we knew were typical in StrongWomen programming and the materials to support these methods. These included newspaper articles, flyers, announcements on the county extension websites, mailings through current contact listings, and verbal announcements. During the recruitment period, every leader completed a weekly questionnaire to provide up-to-date information on their recruitment strategies and the approximate number of potential participants reached by each method.

Measures and analyses

Reach

Reach into the target population was calculated as the number of eligible women who began the program divided by the total number in the target population in Pennsylvania (women age 40 and over, per US Census data [12]) multiplied by 100. For race, education, and employment, data for women age 45 and over was used as the basis for the target population since this was the categorization provided by the Census. For income level, census data specific to age and gender were not available, so we used overall state income levels instead. We also determined percent response to recruitment, as the number in the target population who responded to recruitment divided by the estimated number exposed to recruitment multiplied by 100. We calculated percent eligible as the number eligible among those who contacted the program leader multiplied by 100, and percent participation among those eligible as the number who participated and completed the program among the number eligible multiplied by 100.

Representativeness

Representativeness was determined at the state level by comparing the state population (from the 2010 Census and 2010 American Community Survey [12]) with StrongWomen–Healthy Hearts participants, who had completed a basic demographic questionnaire. A chi-square test was used for analysis.

Qualitative methods

Thirty-eight women who responded to recruitment and who screened as eligible declined to participate in the study. All of these 38 women were subsequently sent a letter inviting them to participate in an interview. Structured telephone interviews were conducted by the Penn State Project Coordinator (JLT). Thirteen granted full interviews and 3 responded by email. The interview guide asked about reasons for nonparticipation and barriers to participation. Phone calls were digitally recorded and then transcribed by the Penn State Project Coordinator.

Data were analyzed using the NVivo program (QSR International, version 10.0) using standard techniques for qualitative analysis [13]. A senior research team member with expertise in qualitative methods (SCF) was primarily responsible for coding and data analysis. An initial coding framework was developed based on the questionnaire structure, and in an iterative process, any additional codes were created as they emerged from the data. Themes were developed from these data and discussed with the Penn State research collaborators to help confirm findings.

Effectiveness

Body weight was the primary measure of effectiveness. Trained program leaders measured participants’ weight in triplicate to the nearest 0.5 kg using a digital floor scale (Seca 880, self-calibrating with a 200-kg capacity). Secondary measures were fruit and vegetable consumption, assessed using the validated 5 A Day for Better Health 7-item screener [14]; and physical activity, assessed using the International Physical Activity Questionnaire (IPAQ) [15–17]. The fruit and vegetable intake scores were compiled according to the NCI criteria [18]. Metabolic equivalent (MET) scores were compiled according to the IPAQ’s guidelines for data processing and analysis [19] and represent MET-minutes per week, defined as MET level multiplied by minutes of activity per day multiplied by days per week. MET levels were determined according to the Ainsworth standards [20].

The pre- and post-program means and standard errors of the three major outcomes—changes in weight, fruit and vegetable intake, and MET minutes per week—were compiled and tabulated. One-sample t tests were used to examine if changes in the three outcomes were significantly different from zero at each site, separately. The program in a specific site was deemed effective if the associated p value of the change in an outcome within a site was less than 0.05. For the overall changes, we pooled all data and examined whether mean changes were different from zero. We used SAS PROC MIXED to control for the clustering effects introduced by the sites, using SAS software version 9.2 (SAS Institute Inc., Cary, NC, USA).

Adoption

The overall adoption rate was calculated as the number of extension educators who ran the program within 1 year of being trained divided by the total number of extension educators invited to the training, multiplied by 100. We also determined the adoption rate among those trained as the number trained who ran the program in 1 year divided by the total number trained, multiplied by 100.

We collected qualitative data to gather detailed information about barriers to adoption. All three trained extension educators who failed to adopt the program granted a telephone interview, which was conducted and analyzed using methods described under reach.

Implementation

Extension educators were asked to complete a brief on-line survey weekly once they started running a StrongWomen–Healthy Hearts program to capture adherence to both the physical activity and nutrition sections of the curriculum, participant attendance and level of engagement with the material, estimate costs associated with the program, and estimated time to prepare for and run the classes. Tufts and Penn State research personnel independently rated fidelity-specific questions from the survey on a 1 to 5 scale (with 5 representing the highest level of fidelity) based on leader responses. Interrater reliability was determined using Cohen’s kappa, and all items that scored lower than 0.8 on this statistic were discussed until consensus was reached. We compiled extension educators’ weekly reports and summed costs for the 12 weeks. We calculated the average cost per site.

Site visits were also conducted by the Penn State Project Coordinator, who observed one class at each site that adopted the program, during weeks 4–8 of the 12-week program. The class was rated using a five-point scale [7] on questions related to fidelity to the curriculum. Interviews were conducted with all 15 program leaders in conjunction with the site visits to explore any changes that were made to the curriculum and reasons for the changes. These qualitative data were analyzed as described above for reach.

We calculated an overall fidelity score by summing ratings from both the implementation surveys and the site visits. Fidelity was also categorized by four components: Physical activity (Did they exercise as planned, both days? At the right intensity? For the right amount of time?), Cooking (Did they do the cooking activity? Did they omit or change any recipes?), Discussion (Was enough time allotted? Did they use the handouts provided?), and Structure (Were the classes conducted in order? Were they conducted independently—not combined?).

Maintenance

Maintenance of the program among adopters was calculated as the number of extension educators who ran the program for a second time within 1 year divided by the number of initial adopters, multiplied by 100. All nonmaintainers were invited to participate in key informant interviews, and five granted interviews. We asked about reasons for nonmaintenance and about barriers to maintenance. Data were analyzed using the process described above for reach.

RESULTS

Reach

Reach varied considerably, ranging from 9 in 1000 in the smallest county to 4 in 100,000 in the largest (Table 1). The county with the highest overall reach (Sullivan) used the full variety recruitment methods. In Pennsylvania overall, the response to recruitment was approximately 1.5 %. Of those who responded by contacting the program leader, 62.0 % were eligible; and of those eligible, 72.5 % participated in the program through completion. The primary reason for ineligibility was regular participation in physical activity (not currently sedentary).

Table 1.

Percent reach by site and for Pennsylvania overall

Site	# who began program	# in target populationa	% Reach
Adams	6	27,397	0.022
Allegheny	15	344,022	0.004
Beaver	25	50,921	0.049
Clearfield	20	22,412	0.089
Greene	8	9,997	0.080
Lancaster	42	130,949	0.032
Lebanon	5	36,177	0.014
Mercer	7	32,972	0.021
Northumberland	7	26,900	0.026
Sullivan	19	2,037	0.933
York	28	113,482	0.025
Pennsylvania Total	182	3,387,629	0.005

^aBased on the 2010 US Census

Compared to women in the target population in Pennsylvania overall, a greater percentage of participants were white, married, middle-class, and had a graduate degree (chi-square tests, p < 0.05; Table 2). The employment status of participants did not differ from the overall Pennsylvania population.

Table 2.

Representativeness of StrongWomen–Healthy Hearts participants compared to Pennsylvania overall

Demographic characteristic	StrongWomen–Healthy Hearts participants	Pennsylvania
Whitea	97.8 %*	86.3 %
Marriedb	73.0 %*	55.0 %
Bachelor’s degree or highera	35.9 %*	21.1 %
Income 25-100 K/yearc	68.7 %*	56.0 %
Employeda	45.2 %	44.3 %

^aBecause of Census data categorization, StrongWomen–Healthy Hearts participants are compared with women ages 45 and older in Pennsylvania

^bDoes not include those who are separated

^cStrongWomen–Healthy Hearts participant income level is compared with overall state household income level (not specific to women age 40+)

*Significantly different than Pennsylvania overall (p < 0.05)

Four of the 16 eligible women interviewed who did not participate in the program cited employment-related scheduling as the primary reason for nonparticipation (Table 3). Other reasons included an inability to pay class fees at all or all at once (three women); medical issues that would preclude participation, despite the fact that they had passed the screening procedure (three women); personal issues (three women); and distance to the class, already fairly active, and unwillingness to commit to 12 weeks (one woman each).

Table 3.

Themes and representative quotes from key informant interviews

RE-AIM component	Key informants	Major themes	Representative quotes
Reach	Eligible nonparticipants	Barriers to participation: employment-related scheduling; inability to pay class fees at all or all at once; medical issues that would preclude participation; personal issues; distance to the class; already fairly active; and unwillingness to commit to 12 weeks	“I also work part time, and at the time that I had called I didn’t believe I was going to be working those days. And it ended up that I was. [The fee] is probably reasonable…It’s kinda hard for me just to pay that all at once.”
Adoption	Nonadopters	Barriers to adoption: inability to recruit a new cohort within the same general geographic area; competition with other programs; focus on weight loss in recruitment rather than cardiovascular health Team-teaching is a good model	“A lot of health organizations in the area were running health programs at the time, and some of them less expensive, I think there was one free program in the area at the time.” “It just works so, so much better because you have two heads, and when one person is maybe not enthusiastic that day, the other one is. And you just go back and forth and it just keeps the enthusiasm, the information going.”
Implementation	Adopting leaders	Common changes to the curriculum: use of a non-program DVD for some portion of the program; providing additional information; adding visuals that complemented the lesson Changes to recipes were few and minor	“We did another Leslie Sansone DVD… This is the one with she did in conjunction with the American Heart Association. Or, I mean, it had their name up there, so she talked a lot about heart health, so it was a very good. It went right along with this material, this curriculum. It was good.” “And I always take the soda bottles telling us how much sugar’s in them, and I measure that sugar out into a glass or a jar, to show them exactly when it says 33 g in sugar, that’s exactly how much it is…I think seeing these things makes them more aware of what they’re doing then. So that was something I added to that lesson.”
Maintenance	Nonmaintainers	Factors that could have facilitated maintenance included adequate training, enjoyment of the program during the initial run, and supportive supervisors The major barriers to maintenance were a change in position, cost to run the program, time necessary to run the program, and not enough interest within the community	“But when we tried to run it again, I don’t remember what time we were shooting for, but it didn’t fly in the fall. So, I haven’t tried it again since then. I just don’t have any more time in my schedule. I have to cut programs. Not add them.” “You’re pulling at people that can’t afford it in other words…So we’d have to run that with scholarships or grant funding and I didn’t even try that at this point because I just didn’t have any time to apply for that.”

Effectiveness

Ten of 15 programs were effective at promoting statistically significant weight loss (Table 4). Overall weight loss was 2.0 kg (95 % CI 1.6, 2.4; p < 0.001). Five of 15 programs achieved a statistically significant increase in fruit and vegetable intake (MyPyramid cup equivalents/day). In the pooled analysis, the program was associated with an increase of one serving of fruits and vegetables per day (95 % CI 0.6, 1.4; p < 0.001). For physical activity, data were somewhat inconclusive since many women chose a “Don’t Know” response option on the IPAQ, which per protocol excluded them from analysis [19]. We detected a significant increase in MET minutes per week at only 3 of the 15 sites. However, in the pooled analysis, the intervention was associated with a 1337.3 unit increase in mean MET minutes per week, nearly double that of baseline (95 % CI 529.7, 2144.9; p = 0.003).

Table 4.

Effectiveness of the StrongWomen–Healthy Hearts Program at changing weight, fruit and vegetable intake, and physical activity

Site		Weight					Fruit and vegetable intakes					Physical activities
	n*	Weight in kg, mean (SE)			p < 0.05	n*	F&V scorea, mean (SE)			p < 0.05	n*	MET scoreb, mean (SE)
		Pre	Post	Change			Pre	Post	Change			Pre	Post	Change	p < 0.05
Adams	6	84 (10.1)	83.9 (10.6)	−0.1 (0.5)	No	6	2.9 (0.9)	2.5 (0.5)	−0.4 (0.9)	No	3	1473 (973.1)	2011 (1046.3)	538 (1506.8)	No
Allegheny	11	97.6 (9.5)	95 (8.1)	−2.6 (1.8)	No	11	2.9 (0.5)	4.7 (1.3)	1.7 (1.2)	No	8	850.2 (423.1)	4593.4 (1136.9)	3743.2 (1119.4)	Yes
Beaver	23	91.9 (3.3)	89.9 (3.3)	−2 (0.4)	Yes	23	3 (0.3)	2.8 (0.4)	−0.1 (0.4)	No	18	1373.1 (451.7)	1710.9 (400.9)	337.8 (444.7)	No
Clearfield-Brisbin	7	82.5 (5.9)	80 (5.8)	−2.5 (0.9)	Yes	7	2.6 (0.5)	3.7 (0.7)	1.1 (0.5)	No	6	2841.8 (1226.4)	2833 (1414.6)	−8.8 (2293.7)	No
Clearfield–Curwensville	6	86.6 (7.4)	83.6 (6.8)	−2.9 (0.9)	Yes	6	1.5 (0.6)	3.4 (0.6)	1.9 (0.6)	Yes	5	1836.2 (979)	4026 (1790.6)	2189.8 (2198.9)	No
Clearfield–Dubois	6	88.9 (8)	87 (7.3)	−2 (1.0)	No	6	2.1 (0.6)	2.7 (0.5)	0.6 (0.4)	No	3	937 (724.1)	3058 (1804.1)	2121 (2215.1)	No
Greene	8	91.3 (5.5)	89.2 (5.6)	−2.1 (0.7)	Yes	8	2.4 (0.5)	3 (0.8)	0.6 (0.8)	No	2	1053 (333)	5121 (2775)	4068 (3108)	No
Lancaster 1	20	87.3 (3.7)	85.3 (3.6)	−2 (0.4)	Yes	20	2.6 (0.4)	4.5 (0.6)	1.9 (0.6)	Yes	14	1028.5 (319.4)	1513.6 (203.1)	485 (303)	No
Lancaster 2	14	88.8 (4.4)	86.6 (4.4)	−2.2 (0.7)	Yes	14	2.4 (0.4)	4.4 (0.7)	2 (0.5)	Yes	9	1553.3 (524.2)	3563.4 (775.7)	2010.1 (1051.4)	No
Lebanon	5	112.5 (10)	109.8 (9.1)	−2.8 (1.1)	No	5	4.6 (0.5)	4.9 (1.1)	0.2 (0.8)	No	5	1897.6 (964.8)	4764 (1775.8)	2866.4 (916.5)	Yes
Mercer	5	94.8 (5.6)	93.4 (5.8)	−1.4 (0.5)	Yes	5	1.4 (0.2)	3.1 (0.9)	1.8 (1)	No	3	2974 (1045.4)	2075.5 (1284.3)	−898.5 (448.6)	No
Northumberland	5	81.7 (5)	77.8 (4.4)	−3.9 (0.6)	Yes	5	2.7 (0.5)	4.4 (0.7)	1.7 (0.3)	Yes	3	1127.8 (505)	6299 (2717.5)	5171.2 (2250.4)	No
Sullivan	16	87.3 (3.8)	87 (3.8)	−0.3 (0.4)	No	16	2.7 (0.4)	2.8 (0.3)	0.1 (0.4)	No	8	1794 (1259.5)	1744.1 (557)	−49.9 (1140.1)	No
York 1	16	89.8 (4.1)	87 (4.2)	−2.8 (0.5)	Yes	16	2.2 (0.3)	3.8 (0.6)	1.6 (0.6)	Yes	9	928.7 (454.7)	1411.6 (540.5)	482.9 (741.6)	No
York 2	9	88.9 (6.6)	87.1 (6.7)	−1.8 (0.7)	Yes	9	2.9 (0.7)	3.7 (0.7)	0.7 (0.4)	No	6	98.6 (34.8)	989.5 (321.1)	890.9 (340.3)	Yes
Overall change (from mixed effects regression)
Total	15	89.9 (1.5)	87.9 (1.4)	−2.0 (0.2)	Yes	15	2.6 (0.1)	3.6 (0.2)	1.0 (0.2)	Yes	15	1364.7 (184.5)	2807.4 (379.1)	1337.3 (377.4)	Yes

^aMyPyramid cup equivalents, since the curriculum reflected national guidelines at the time

^bMET score (MET-minutes per week) is defined as MET level × minutes of activity/week × days/week

*The n’s in each site represent total number of effective sample (i.e., class size), while the n’s for the “Total” row represents the total number of sites, which is 15

Adoption

The overall adoption rate among leaders invited to the training was 45.4 % (15 leaders ran a program of 33 invited to training). The adoption rate among those trained was 83.3 % (15 of 18 trained). Leaders ran programs in counties that represented both reasonable geographic distribution throughout the state and diversity with respect to population density. Half of them were from counties with a slightly higher poverty rate than the state overall; however, 14 of the 15 leaders who ran a program lived in counties that were >90 % white, compared to the state average of 81.9 % [12].

All trained nonadopters had co-taught with another site leader. In the key informant interviews, they indicated that the main reason for nonadoption was an inability to recruit a new cohort within the same geographic area (Table 3). However, other than this issue, they felt that team-teaching was a good model. Competition with other programs, such as hospital-based programs, also contributed to an inability to recruit. Leaders felt that it would be helpful to be able to recruit women of any BMI and emphasize cardiovascular health more than weight loss.

Implementation

The average overall fidelity score, representing a sum of scores from self-report and direct observation, was 9.3 (out of 10), ranging from 8.7 to 9.8 (Table 5). Based on key informant interviews (Table 3), leaders most commonly made changes to the physical activity component by using a non-program DVD for some portion of the program. Changes to recipes were few and minor. The most common change to the discussion component involved providing additional information, typically handouts from the StrongWomen Strength Training Program and from other extension programming. Another common change was to add visuals that complemented the lesson, for example, models depicting atherosclerosis. Estimated overall attendance based on leader report in the weekly implementation questionnaires was high (89 %, excluding those who dropped out).

Table 5.

Implementation of the StrongWomen–Healthy Hearts Program in Pennsylvania—fidelity scoring

Component	Score (out of 10) and range
Overall fidelity	9.3 (8.7 to 9.8)
Physical activity	9.0 (8.0 to 9.9)
Cooking	9.4 (8.1 to 10.0)
Discussion	9.7 (9.3 to 10.0)
Structure	9.8 (9.3 to 10.0)

Cost averaged $344 per 12-week program, ranging from approximately $100 to $900; or $35.89 per participant per 12-week program, ranging from $6.27 to $85.45. Costs included copies of handouts, groceries for cooking exercises, space rental, and equipment. Most of the variation in cost was accounted for by whether space rental was necessary and whether pedometers were purchased. Most program leaders charged participants a fee of $30 to $100 for the entire 12-week session.

Maintenance

The overall maintenance rate was 6.7 % (1 of the 15 leaders maintained the program). A major barrier to maintenance was related to a funding crisis that occurred in the state, which resulted in job elimination for four leaders. The nonmaintaining leaders who granted interviews indicated that they had been adequately prepared to run the program during the training, that the program was well received when they initially ran it, and that their supervisors were supportive (Table 3). The main barriers were a change in position, cost to run the program, time necessary to run the program, and not enough interest within the community.

DISCUSSION

This study confirms that the StrongWomen–Healthy Hearts Program is appropriate for community dissemination. The RE-AIM framework allowed us to identify strengths as well as areas that might be improved to achieve better public health impact as the program is introduced nationally. It also provides a number of lessons for the translation of similar programs.

Overall reach into the population was low. One consideration is that leaders had limited resources to promote the program. Also, typically in other studies, reach is based on a more bounded denominator such as church membership [21] or worksite employment [22]. Based on definitions of RE-AIM [4] and our dissemination model and philosophy, we used a metric designed to capture the entire target population within each county and the state overall. However, this conservative measure should be interpreted cautiously for several reasons. For example, the numerator was bounded by class size limits, which leaders placed to help ensure a meaningful class experience. Furthermore, the reach denominator, which included all women age 40+ in the county, was not always reflective of appropriately limited recruitment efforts. For example, in Allegheny County, the recruitment catchment by design included only the major urban area within that county. Finally, not all women age 40+ in the county or state would have screened as eligible. Among those who contacted a program leader in Pennsylvania, two-thirds were eligible. Among those eligible, there was a high rate of participation to program completion, suggesting that the screening procedure was effective at identifying women for whom the program would be safe and appropriate. We intend to maintain this procedure with modifications as described below throughout future dissemination.

The programs attracted participants that were white and of a higher socioeconomic status than the overall state population. Leaders who ran a program represented counties that had a higher percentage of white residents than the state overall, and it is possible that the curriculum is more appealing to leaders whose constituents are mainly white. This suggests the possible need for culturally adapted versions of the curriculum to attract leaders who serve other racial/ethnic groups. Financial issues, including an inability to pay the class fee, emerged as a barrier to participation for eligible nonparticipants. Accessibility may have been an issue as well, since most leaders kept costs lower by holding classes in facilities where they would not need to pay rent, which were not always conveniently located.

Taken together, our findings suggest that to increase public health impact of the StrongWomen–Healthy Hearts Program, it will be important to further reduce the cost of the program, to examine the effect of location and accessibility, to modify recruitment and training strategies, and possibly to adapt the curriculum to better enable leaders to reach low-income and minority women. We intend to continue to monitor representativeness as the program is disseminated nationally, identify leaders who are more successful at achieving it, learn their strategies, and incorporate these best practices into future leader training workshops. We may also need to consider other modes of delivery of the program to help address scheduling, accessibility, and cost issues. Technology-based interventions are a potential solution to overcoming these barriers for low-income women that also allows for the possibility of customization and adaptation to the needs of different groups [23].

The program demonstrated effectiveness at promoting weight loss overall and at the majority of sites. The magnitude of weight loss achieved in this community translation overall was essentially the same as that achieved in the original, more tightly controlled study [3]. Although there was some variability in effectiveness across sites, the program was effective among leaders who had been in their positions for varying lengths of time, from 2 years to over 30 years. It was also effective in a variety of communities, including rural, small town, small city, and urban locations, and at sites that varied with respect to resources available, such as access to kitchen facilities and outdoor walking routes.

The changes in weight, while modest, likely have public significance since 1 kg of weight loss has been associated with reductions in systolic and diastolic blood pressure of approximately 1 mmHg [24] and a 16 % reduction in diabetes risk [25]. Furthermore, any weight loss counters strong secular trends toward weight gain through adulthood [26]. To further determine public health significance, it will be important to conduct cost-effectiveness analysis in future studies. If reasonably cost-effective, this may also enhance adoption by providing information to help key decision-makers choose the program. This analysis may also promote maintenance by providing compelling data to help class leaders convince potential outside funders to support the program.

Overall adoption rate among those invited to train was moderate; among those trained, it was high. This suggests that trained leaders had adequate confidence, resources, and support to run the program the first time. It is notable that all trained “nonadopters” actually did teach the program, partnering to provide programming at another leader’s site. Leaders indicated that co-teaching made implementation more manageable. Based on this, both the training and the curriculum now encourage the co-teaching model.

The program was almost universally implemented as intended, which likely explains the preservation of effectiveness during dissemination. The universally high fidelity scores suggest that the training adequately prepared leaders and that we had met the goal of providing a curriculum that served as a useful guide in running the program. These findings suggest that features such as the basic format of the curriculum should be preserved. However, based on implementation feedback, we did make several minor changes. We added suggestions for visuals to enhance the lessons, and suggested that leaders should feel free to select among the recipes based on the resources available to them and group preferences.

Maintenance was problematic for the StrongWomen–Healthy Hearts program in Pennsylvania. Fundamentally, maintenance is a measure of “fit” within an organizational structure and also of public health impact over time. Both adoption and implementation suggest that the organizational fit was probably reasonable. Larger factors unrelated to the program itself played a major role in the low maintenance rates, which is an ongoing challenge with public health programming that relies on community partnerships. However, the data provided some evidence for strategies to address barriers that are within our control. We have taken measures to reduce program costs by making some of the cooking optional as described above, and by making pedometers optional. We made inclusion criteria more flexible by modifying the screening form to include tiered mandatory, highly recommended, and recommended criteria. For example, highly recommended criteria include BMI, reflecting a focus on weight control in the curriculum but allowing for flexibility and leader discretion.

A recent systematic review of studies using the RE-AIM framework found that only 44 of 71 articles reported on all five dimensions and that qualitative methods were used very infrequently to provide additional evaluation and understanding [5]. A strength of this study is that the research team received funding just when we were poised to begin major dissemination efforts, and these resources were critical to our ability to evaluate all RE-AIM components using multiple methods. It should be noted, however, that even with substantial planning and resources, our study falls short of meeting all 34 items used to evaluate RE-AIM [5]. For example, we were unable to examine maintenance at the individual level.

Two other studies have described the use of the RE-AIM framework for prevention of CVD among women [27, 28]. These studies demonstrated that it is feasible to use the framework to evaluate dissemination of programming for low-income women, Latina women, and postmenopausal women with diabetes, and that doing so provided a richer understanding of program success. Our study extends the literature by uniquely examining the public health impact of a program that is primarily focused on a lifestyle program run through a community-based organization to reduce CVD risk among all mid-life and older women.

There were several limitations to this study. Height and weight were measured by program leaders, who had an interest in the success of their programs. However, weight was measured by digital scale, essentially an objective measure. Diet and physical activity were measured by self-report, which is subject to recall and social desirability biases. Other methods, such as 24-h recall or accelerometry, were not feasible. To mitigate potential concerns, we chose measures that have been previously validated and widely used. All measures were pre-post with no comparison group, and the possibility that outcomes changed as a result of external influences cannot be ruled out. However, we monitored for the presence of other major initiatives, and none were reported. Finally, because we lack data on maintenance at the individual level, sustained health impact of the program cannot be determined. It will be important to evaluate this in future studies.

CONCLUSION

This study provides evidence for the external validity of the StrongWomen–Healthy Hearts program in Pennsylvania. Based on these data, the program has demonstrated that it can be readily implemented with high fidelity in a variety of settings while remaining effective. These data also provided information on how to modify recruitment, training, and the curriculum itself to improve public health impact as it is disseminated nationally.