Abstract
Integrated research-practice partnerships (IRPPs) may improve adoption of evidence-based programs. The aim of this study is to compare adoption of an IRPP-developed physical activity (PA) program (Fit Extension, FitEx) to a typical efficacy-effectiveness-dissemination pipeline model program (Active Living Every Day, ALED). Guided by the Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) framework, a randomized controlled trial assigned health educators (HEs) to FitEx (n = 18) or ALED (n = 18). Fourteen HEs adopted FitEx, while two HEs adopted ALED (χ 2 = 21.8; p < 0.05). FitEx HEs took less time to deliver (p < 0.05), stated greater intentions for continued program delivery (p < 0.05), and reached more participants (n = 1097 total; 83 % female; 70 % Caucasian; M age = 44 ± 11.8) per HE than ALED (n = 27 total; 60 % female; 50 % Caucasian; M age = 41 ± 11.3). No significant difference existed in FitEx or ALED participants’ increased PA (M increase = 9.12 ±29.09 min/day; p > 0.05). IRPP-developed programs may improve PA program adoption, implementation, and maintenance and may also result in programs that have higher reach—without reducing effectiveness.
Keywords: Translation, Physical activity, RE-AIM, Research-practice
Regular physical activity plays an important role in the prevention, onset, and management of many adverse chronic health conditions [1]. Conversely, physical inactivity is one of the primary behavioral causes of death in the USA [2] and contributes to the 483.8 billion dollars spent each year for the management and treatment of cardiovascular diseases and diabetes [3]. Unfortunately, less than half of the adult population is active at a recommended level [4] of moderate intensity for at least 30 min/day, 5 days or more per week [5]. While interventions (programs, policy, and practice) exist that aim to increase physical activity, there is a lack of translation of effective evidence-based programs into practice [6].
Reasons for this lack of translation are related to both research and practice. First, using research reporting guidelines that are predominantly focused on internal validity (e.g., CONSORT Statement [7]) results in a body of literature that values the strong internal validity but attributes only a passing focus on issues of external validity [8]. In fact, a series of reviews of intervention studies published in leading journals that targeted physical activity (as well as nutrition and smoking) bears this out, showing that external validity (e.g., participant’s representativeness, organizational adoption, and program sustainability) was rarely reported [9–11]. Second, a linear (or top down) model of dissemination, which highlights the researcher as the expert and developer of products, and the delivery system as a simple receptor of evidence-based programs are a common depiction of translation of research to practice [12]. A linear model has a number of drawbacks for translation of research to practice: (1) it fails to address the organizational and community capacity to implement evidence-based programs, (2) it does not address that the degree of congruence between organizational and program values can either facilitate or debilitate success, (3) the model assumes a pro-innovation bias and devalues current organizational practices, and (4) the assumed model of how delivery organizations make adoption decisions is overly simplified [12]. Third, research and practice communities typically operate in silos. A key difference between the silos is the attention paid to the broader health policy and cultural context of delivery organizations. In research, contextual factors may be described, at best, but more often are eliminated (i.e., efficacy trials testing an intervention under optimal conditions). In practice, activities are often driven by contextual factors related to organizational mission, resource availability, and the needs of the community or patient population targeted for health improvement [13]. Hence, researchers develop programs with the intent that evidence-based practice will eventually occur while the practitioners often perceive a lack of fit between the evidence-based programs and their organizational structure, values, and culture [4].
To address these obstacles, a number of implementation science frameworks have been developed and often they include an integration of research and practice personnel within a participatory model [14, 15]. The integration of research and community health professionals has the ability to address and overcome each of these obstacles and is proposed to speed the translation of evidence-based principles within typical practice. The benefit of an integrated approach to program development is that evidence from both research and practice environments are considered to determine effectiveness for behavior change (i.e., research evidence) and the fit of programs within the delivery system (i.e., evidence from within the system regarding mission, values, and resources) [4]. Thus, programs developed using integrated processes are thought to be more likely to be adopted in practice settings [16].
Integrated research-practice partnerships have worked in a variety of settings toward multiple outcomes including parent-child dyads for improved health behaviors [16], depression care for public community, long-term care facilities for older adults [17], and behavioral health care [18]. In research-practice partnerships, stakeholders are involved in an iterative and interactive process for decision-making [19] whereas a typical efficacy-effectiveness-dissemination pipeline model [20, 21] is more one directional. The pipeline approach first establishes internal validity under best practices (efficacy) and then positive outcomes in real-world demonstration (effectiveness) and the ability and willingness of people to implement the program with appropriate fidelity (implementation) [22]. One barrier to the pipeline approach was the lack of research team support for delivery/implementation [23]. This stepwise approach, and lesser attention on external validity, contributes to the persistent lack of translation into sustained practice or a long lag between the demonstration of efficacy and implementation that benefits the public health [24]. Therefore, integrated research-practice partnerships may speed translation. However, there has been little research to determine if they result in practical interventions that are more likely to be adopted and implemented when compared to interventions developed through a pipeline model [20, 21].
The purpose of this project was to compare an integrated research-practice partnership-developed physical activity program (Fit Extension, FitEx) to an evidence-based program developed through the typical efficacy-effectiveness-dissemination pipeline model (Active Living Every Day, ALED) using the Reach Effectiveness Adoption Implementation Maintenance (RE-AIM) framework [24]. More specifically, our primary aim at the organizational level was to determine the differences in health educator adoption. Secondarily, we also explored implementation and plans for sustained program delivery over time. At the individual level, we sought to determine the reach and effectiveness of each program. It was hypothesized that the integrated research-practice partnership-developed program would be more likely to be adopted within the system, would take less time to deliver due to differences in delivery structure, and would be related to perceptions of maintained delivery. At the individual level, we hypothesized that the integrated research-practice partnership-developed program would have higher reach and equitable effectiveness when compared to the pipeline developed model. A tertiary aim of this study was to determine potential RE-AIM-based predictors of program adoption. Specifically, we hypothesized that health educators with more positive perceptions of the program (as they related to RE-AIM) would be more likely to adopt the program.
METHODS
Design
Effectiveness-implementation hybrid type 3 trial [23]
This cluster randomized controlled trial was designed, a priori, to evaluate the utility of integrated research-practice partnerships to improve adoption of physical activity programs. A secondary aim was to evaluate (1) implementation, (2) intent to continue program delivery, (3) program reach, and (4) changes in physical activity between the two programs. See Fig. 1 for the different intervention development models. This figure illustrates an adapted version of the integrated research-practice partnership model as proposed by Estabrooks and Glasgow [14]. The RE-AIM framework [24] was used as a planning and evaluation tool for the trial with adoption, implementation, and intention for maintained delivery measured at the level of the health educator and reach and effectiveness measured at the level of the physical activity program participants. Randomization was conducted via a computer-generated randomization table.
Fig 1.
Research to practice models
Integrated research-practice partnership
An integrated research-practice partnership was developed between researchers in the Department of Human Nutrition, Foods and Exercise and the Family and Consumer Sciences unit within the state’s Cooperative Extension system. Cooperative Extension is available in every state and territory in the USA and associated with the Land-Grant University of that state or territory [25]. Cooperative Extension Family and Consumer Sciences health educators (n = 56) are tasked with delivering health promotion programs within the community. The purpose of this integrated research-practice partnership was to balance scientific evidence on physical activity promotion with the needs and systems evidence of the state’s Cooperative Extension. The integrated research-practice partnership consisted of the Cooperative Extension leadership in the state, extension specialists, health educators, program assistants, a behavioral scientist, and research staff. The integrated research-practice partnership was involved in all phases of this research including the generation of the scientific question, the design of the locally relevant physical activity program, the selection of research methods, and the interpretation of the findings. Trained research assistants collected all study data, and a third party conducted the analyses to reduce bias in interpretation.
Physical activity interventions
Fit Extension
FitEx was developed by the integrated research-practice partnership as an 8-week walking and fruit and vegetable consumption program delivered to teams using information from evidence-based group dynamics approaches to increase physical activity in community settings [26] as well as information from other Cooperative Extension-delivered physical activity programs [27, 28]. The group dynamics- and evidenced-based program principles used included the following: developing a sense of group distinctiveness, group goal setting, proximity to other group members, ongoing group interactions, communication (especially around feedback), information sharing, and collective problem-solving [27]. In addition, the program included the well-established [29] individual-level strategy of self-monitoring behaviors of physical activity and fruit and vegetable consumption.
The 8-week duration of the program was based on the experiences of the health educators, published studies on community physical activity (PA) programs [27, 30, 31], and the typical program length of other offerings in the system. To enhance reach, the program was developed to allow health educators to recruit team captains who, in turn, recruited team members (i.e., family, friends, and co-workers). Team size was limited to 6 (though exceptions were made at the discretion of the health educators) with a recommended goal for the team to record physical activity that would equal the distance across the state (∼450 miles). If all team members walked 10 miles/week at a moderate intensity (e.g., 15-min mile pace), the team would reach the recommended goal and each team member would meet the recommended physical activity guidelines of 150 min/week [32]. To address participants that may not prefer or be able to walk, miles were operationalized as either miles walked or run or any 15-min block of any moderate or higher intensity PA (e.g., biking, swimming, resistance training). Participants also received weekly newsletters that consisted of evidence-based strategies, recipes, and a 5-a day social cognitive theory-derived fruit and vegetable promotion message [33].
Active Living Every Day
Active Living Every Day (ALED) was developed by behavioral scientists at the Cooper Institute in Dallas, Texas, and offered through partnership with Human Kinetics [34]. ALED was a 12-week physical activity behavior change program delivered to community groups. ALED followed the traditional model of efficacy [35] to effectiveness [36] to demonstration to dissemination to reach the community [36]. Originally 20 weeks, the duration of program was adapted after community demonstration feedback [37, 38]. In its original conception, ALED was designed specifically for sedentary adults, meeting in groups of 12 to 20 for 1-h weekly sessions with a trained facilitator. Facilitators were representatives from partnering providers (e.g., hospitals, public health entities). Consistent with social cognitive theory and the trans-theoretical model of behavior change, ALED helped sedentary participants set goals, overcome barriers, find activities they like, and experience successes. Participants chose their own activities and created their own plans to work toward recommended physical activity guidelines of 150 min/week. Prior to program implementation, an online or in-person 6-h training was mandatory for facilitators and included an instructor manual with detailed lesson plans for each session, along with ongoing Web-based support. For participants, ALED workbooks and step counters were available for purchase from Human Kinetics (approximately $35 for the participant’s package including tax and shipping). For the study presented in this article, health educators in Cooperative Extension were trained on the (still available) commercialized program for delivery in their communities.
The primary program principles of ALED were as follows: emphasis on lifestyle activity rather than structured exercise, behavioral skill development, self-monitoring based on readiness to change, and facilitated group problem-solving. First, to encourage participation among sedentary adults who may lack self-efficacy to reach physical activity recommendations, the level of physical activity needed to benefit health was emphasized as not strenuous or time-consuming. Further, ALED emphasized and promoted small changes to increase activity levels (e.g., park farther away, take the stairs rather than the elevator). Additionally, the participants were taught skills to help them identify and overcome barriers, manage time, and deal with stress. Moreover, self-monitoring of thoughts, steps, and daily minutes of light/moderate/vigorous physical activity was a core program activity to incrementally increase activity from a stage of contemplation to action based on a participant’s readiness. Program activities were tailored to readiness to change levels in order to reduce resistance, facilitate progress, and prevent relapse. Finally, facilitated group-based problem-solving provided social support to brainstorm strategies to integrate physical activity into everyday living.
ALED was chosen as the comparison program as it was an evidence-based physical activity program that had followed an efficacy to effectiveness to demonstration study design and consistently resulted in an increase of physical activity. In addition, ALED included an established training infrastructure to ensure that participating health educators would receive the state-of-the-art training on the program. Finally, ALED has been examined in the context of program adoption previously and has been adopted in a wide variety of settings (governmental, non-government, fitness, medical, worksite, and educational organizations).
Health educator sample and training
Family and Consumer Science health educators within the state’s Cooperative Extension system (N = 56) were eligible to participate in the randomized controlled trial. All health educators have a bachelor’s degree, and health educators must receive their master’s degree within 6 years of their date of hire. Prior to announcing the opportunity to participate in a study to test FitEx and ALED, all Family and Consumer Sciences health educators were asked to complete a brief survey assessing their interest in delivering a physical activity program in their county. Thirty-six (100 % female; 45.9 ± 10.8 years) of 56 health educators (64 %) within the state’s Cooperative Extension system expressed interest in delivering a physical activity program in their county. Interested health educators were randomly assigned to receive training on either FitEx (n = 18) or ALED (n = 18). First, health educators were invited to attend a 2-h telephone conference call that provided details about the specific program to which they had been randomized (e.g., duration, frequency, program materials). Following the introductory call, health educators also had access to online materials (FitEx and ALED) and training support (ALED only).
A full day, in-person training was provided concurrently for each program. Two professional trainers employed by the Human Kinetics delivered the standard onsite training for community leaders interested in delivering ALED. Two members from the integrated research-practice partnership, a long-standing health educator and a research scientist with a background in group dynamics approaches to promote physical activity, led the FitEx health educator training.
Measures
Primary aim: organizational-level adoption
Adoption was measured in two ways: first, for health educators who made the decision to participate in training and second, for those who made the decision to adopt the program. At both levels, the proportion and representativeness of health educators were assessed. Training was consistent across both programs with telephone training lasting 2 h and in-person training of 6 h.
Secondary aims
Organizational-level implementation
Process evaluation data were captured for both program start-up (i.e., preparation time) and program delivery. Program preparation items include the amount of time health educators spent on reviewing materials, developing community partnerships, working on logistics, traveling to prepare for the program, preparing materials, marketing the program, and adapting program materials. Program delivery items included entering information, communicating with participants, leading activities, evaluating the program, following up with participants, and any other program-related activities. One unique area of health educator implementation monitoring was tailored for each program: for FitEx, health educators reported on the time necessary to process weekly team progress and for ALED, health educators reported on the time necessary to prepare for each class. These two summary items (i.e., preparation and delivery) were calculated by program and per participant.
Organizational-level maintenance
Health educators were surveyed to determine if they planned to deliver the program the following year. We calculated the proportion indicated in the affirmative, and those that either indicated that they would not deliver, or did not respond to the question, were coded as not intending to deliver the program the following year.
Individual-level reach and representativeness
Reach was assessed by the number and participation rate among eligible county/city residents where the program was offered. Additionally, we calculated the average participant reach per health educator delivering each program. Representativeness of participants was determined by comparing the demographic characteristics of program participants as obtained from program surveys and those reported by census data for the given county or city. While the program was for physical activity promotion, neither program had a pre-program activity level exclusion criterion (i.e., all adults were eligible). Reach of the complete program was operationalized as those attending the in-person classes at the end of the 12-week program for ALED. FitEx reach was operationalized as the proportion of teams reporting weekly mileage by the end of the 8-week program.
Individual-level effectiveness
The Rapid Assessment of Physical Activity (RAPA) [39, 40], a reliable and validated self-report measure, was used to determine physical activity levels. RAPA was self-reported via the pre- and post-program surveys. The RAPA was used to categorize participants as inactive (0 min of physical activity), insufficiently active (greater than 1 min, but less than 150 min, of moderate intensity of physical activity), and active (150 min of moderate intensity physical activity per week).
Tertiary aim: predictors of program adoption
At baseline, health educators were also asked the degree to which (1) they and (2) their stakeholders considered reach, efficacy/effectiveness, and implementation resources; maintenance of individual behavior change; and organizational maintenance when choosing to adopt a program. The items were all on a 5-point Likert scale from “not at all important” (1) to “extremely important” (5). Three items (perceptions of reach, representativeness, and the program’s ability to attract participants) were combined to indicate perceptions of program reach. Perceptions of effectiveness were based on two items: perception that the program was effective in controlled, research settings, and perceptions that the program was effective in the community. Two items were used for adoption: perceptions of training sessions and perceptions of fit within the system. Four measures were used for implementation including administrative support, support from community partners, ease of implementation, and if there was a specialist (doctoral-level Cooperative Extension employee) available to assist with program delivery. Individual-level maintenance was measured via one item that the program helped people maintain health behavior when the program was over. These items were collected after the phone training (T1), after the in-person training (T2), and after program delivery (T3).
Data analysis
Descriptive statistics were used for proportions and averages where appropriate. Independent sample t tests were used to determine any significant differences by randomization (i.e., FitEx or ALED) and delivery status (i.e., non-adopter or adopter) with change scores being calculated and used. Within-sample t tests were conducted to determine differences between health educators’ perceptions of RE-AIM program attributes. To conduct effectiveness comparisons, we matched ALED with FitEx participants on a 3:1 ratio based on specific demographics. Matching was completed based on the following: baseline physical activity category, gender, age (±5 years), and program completion status. This approach was used as there were over 40 times more FitEx participants when compared to ALED.
Prediction modeling
A forward stepwise multiple logistic regression was conducted to determine if the perceptions of RE-AIM program attributes at baseline predicted program delivery status. For the prediction model, significance was set at p <0.05. Multivariate-adjusted odds ratios, 95 % confidence intervals, log-likelihood statistics, model fit statistics, and individual predictor statistics were calculated. Due to the small sample size and to create a parsimonious model, only predictors meeting the probability predictor inclusion criteria (0.05 for inclusion) were used in the final model. All analyses were conducted in IBM SPSS 22.0.
RESULTS
Primary aim: organizational-level adoption
Health educators who were randomized to and delivered FitEx were similar to those who were randomized to deliver ALED, with the exception that one male (the only male Family and Consumer Sciences health educator in the state at the time of the study) was randomized to the ALED condition. Those who delivered the programs were not significantly different on demographic or health-related characteristics as compared to those who did not deliver the program (Table 1).
Table 1.
Health educator characteristics
| Variable | FitEx | ALED | ||
|---|---|---|---|---|
| Adopter, N = 14 | Non-adopter, N = 4 | Adopter, N = 2 | Non-adopter, N = 16 | |
| Female, % | 100 | 100 | 100 | 94 |
| Race, % | ||||
| Caucasian | 50 | 75 | 50 | 61 |
| Black | 25 | 25 | 50 | 39 |
| Unknown | 25 | – | ||
| Weight (lb) | 197 | 155.8 | 168.5 | 176.25 |
| Mean (SD) | (±67.62) | (±41.7) | (±61.52) | (±31.62) |
| Wanting to lose weight, % | 100 | 50 | 50 | 69 |
| Non-smoker, % | 75 | 93 | 100 | 81 |
| General health, % | ||||
| Poor | – | – | – | – |
| Fair | – | 14 | – | – |
| Good | 25 | 29 | 50 | 56 |
| Very good | 75 | 36 | 50 | 38 |
| Excellent | – | 21 | – | 6 |
| Meeting PA recommendations | 50 | 43 | 50 | 38 |
| Confidence for meeting PA recommendations, % | ||||
| Not at all | – | – | – | – |
| Somewhat | 25 | – | – | 13 |
| Moderately | 25 | 21.4 | 50 | 25 |
| Very | – | 57.1 | 50 | 19 |
| Completely | 50 | 21.4 | – | 43 |
| Cups of fruit, mean (SD) | 2.00 (±0.82) | 2.28 (±1.32) | 2.50 (±0.71) | 2.38 (±0.96) |
| Cups of vegetables, mean (SD) | 1.5 (±0.58) | 2.57 (±1.15) | 2.00 (±1.41) | 2.63 (±0.96) |
| Confidence for meeting F/V recommendations | ||||
| Somewhat | 25 | 8 | – | 6 |
| Moderately | 50 | 15 | – | 6 |
| Very | 25 | 46 | 100 | 38 |
| Completely | – | 31 | – | 50 |
| Work satisfaction, % | ||||
| Moderately | 25 | 43 | – | 25 |
| Very | 75 | 57 | 100 | 75 |
| Completely | – | – | – | – |
A total of 16 (89 %) health educators attended the 6-h, in-person FitEx training, and 18 (100 %) attended the ALED 6-h, in-person training. Fourteen health educators (78 %) adopted FitEx, whereas 2 ALED health educators (11 %) delivered the program. See Fig. 2 for more details.
Fig 2.
Recruitment and retention of health educators. The primary outcome of adoption through the recruitment and retention process is displayed
Secondary aims
Organizational-level implementation
There was no significant (p > 0.05) difference in the average time FitEx health educators spent preparing for the program (66 ± 80 h) when compared to their ALED counterparts (26 ± 38 h). FitEx health educators spent an average of 67 ± 94 h total delivering the program to 1097 participants. ALED health educators spent 7 ± 18 h total delivering the program to 27 participants. Taken together, from preparation to delivery, FitEx took 134 ± 162 h to deliver, or an average of 7 min per participant, and ALED took 33 ± 49, or an average of 1.2 h per participant.
Organizational-level maintenance
FitEx health educators were significantly (p < 0.05) more likely to intend to deliver the program the following year at 50 % affirming that they intended to deliver FitEx, while only 25 % of the ALED health educators intended to deliver their program in the following year.
Individual-level reach
All community members over the age of 18 were eligible to participate in either program (N = 587,688). Based on census data, on average, 71 % of county residents were Caucasian and 20 % were African American with 51 % being females. Participants in this study (i.e., those who engaged in either intervention condition) were representative of state residents with regard to age, race, and gender.
Participants in FitEx (n = 1097; 2 % of the county adult populations) were 73 % Caucasian, 27 % African American, and 89 % non-smokers. The mean age of participants was 44 years (±11.83), with 83 % female with an average weight of 165 lb (±49.49).
Participants in ALED (n = 27) were 60 % Caucasian, 40 % African American, and 78 % non-smokers. The mean age of participants was 41.33 years (±11.34) with 37 % female with an average weight of 175 lb (±33.33). The proportion of participants that reported poor, fair, good, very good, and excellent health status were 19, 22, 37, 11, and 11 %, respectively.
Baseline characteristics indicate that ALED participants were significantly (p < 0.05) more likely to be males and worked fewer hours (p < 0.05) than FitEx participants (41.04 (±9.38) vs. 27.85 h (±23.83)). FitEx reached significantly more individuals per health educator than ALED (78 vs. 13, p < 0.05). For completed program reach, ALED health educators had an average of 88 % (±17) attendance at the final program session and the FitEx health educators’ retention was 69 % (±31) of teams in week 8.
Individual-level effectiveness
A total of 108 participants were included in the analyses (FitEx, n = 81; ALED, n = 27; mean age = 41.72 ± 10.9; 56 % female; 71 % Caucasian; 91 % non-Hispanic). At baseline, 37, 28, and 35 % of participants were inactive, insufficiently active, and active, respectively. Post-program, the proportion of participants in FitEx that were inactive, insufficiently activity, and active were 8, 45, and 47 %, respectively, while the proportion of ALED participants were 5, 47, and 47 %, respectively. There was an overall significant shift (p < 0.01) from baseline to post-program in the proportion of people who were inactive, insufficiently active, and active to 6, 42, and 52 %, respectively. However, there was no significant difference in the proportion of individuals meeting recommendations by condition (p = 0.96). There was also no significant (p > 0.05) difference in the percent of participants improving, maintaining, or decreasing their physical activity recommendation status by condition. The mean increase of physical activity across both programs was 9.12 ± 29.09 min/day, p > 0.05 (Table 2).
Table 2.
Baseline and post-program physical activity status by group
| FitEx (n = 81) | ALED (n = 27) | Overall FitEx (n = 1070) | |
|---|---|---|---|
| Inactive, % | |||
| Baseline | 37 | 37 | 29 |
| Post-program | 8 | 5 | 6 |
| Insufficiently active, % | |||
| Baseline | 28 | 28 | 36 |
| Post-program | 45 | 47 | 43 |
| Active, % | |||
| Baseline | 35 | 35 | 35 |
| Post-program | 47 | 47 | 51 |
| Average increase in total PA minutes per day, mean ± SD | 7.50 ± | 14.07 ± 47.14 | 7.72 ± 19.02 |
Tertiary aim: RE-AIM predictors of program adoption
Table 3 shows the contribution of all individual predictors to the final model. Wald statistics indicate that perception of the program: (1) effectiveness in the community setting (p = 0.016), (2) easier to deliver than other physical activity programs (p = 0.014), and (3) the likelihood that the program could be sustained without additional funds (p = 0.046) as well as a lack of expertise necessary to deliver the program (p = 0.034) all made a significant contribution to the prediction model. Program adopters were less likely to perceive that the program was effective in community settings than non-adopters (OR 0.052, 95 % CI 0.005–0.576). Those who perceived that the program was easy to deliver were more likely to adopt the program. Furthermore, health educators that perceived the program to be sustained easily in the state’s Cooperative Extension system without additional funding were more likely to adopt the program. Finally, those that reported more not having the expertise that was needed to run the program were less likely to adopt the program.
Table 3.
Multiple logistic regression model results of the contribution of RE-AIM attribute perceptions to determining program adoption decision
| Variables | B | S.E. | Wald | Sig. | OR | 95 % CI | |
|---|---|---|---|---|---|---|---|
| Lower | Upper | ||||||
| Perception that the program | |||||||
| Was effective in community settings | −2.95 | 1.22 | 5.806 | 0.016* | 0.052 | 0.005 | 0.576 |
| Was easier to deliver than other physical activity programs | 5.20 | 2.11 | 6.066 | 0.014* | 181.552 | 2.892 | 11,395.978 |
| Was easier to deliver than other extension food or nutrition programs | 1.85 | 1.30 | 2.010 | 0.156 | 6.369 | 0.493 | 82.340 |
| Fits the mission of Cooperative Extension | −4.09 | 2.17 | 3.530 | 0.060 | 0.017 | 0.000 | 1.193 |
| Will be sustained in Cooperative Extension for longer than 1 year without special funding | 1.40 | 0.70 | 3.965 | 0.046* | 4.077 | 1.022 | 16.256 |
| Will receive good public relations opportunities and visibility for me and extension | −2.11 | 1.31 | 2.574 | 0.109 | 0.120 | 0.009 | 1.599 |
| Perceptions that the HE did not have the expertise that is needed to deliver the program | −2.38 | 1.12 | 4.499 | 0.034* | 0.092 | 0.010 | 0.834 |
*p < 0.05; χ 2 (7) = 25.754; p < 0.001
Multiple logistic regression results found that the model with the predictor variables were an improvement over the constant only model (χ 2 (7) = 25.754, p < 0.001), suggesting that the predictors were reliably able to distinguish between program adopters and non-adopters. Additionally, the Cox and Snell R 2 and the Nagelkerke R 2 both indicated that the final model helped to explain 52.1 to 69.9 % of the overall variance in the decision to adopt the program or not with prediction success at 88.6 % (95 % correct prediction of non-adopters and 80 % correct prediction of adopters).
DISCUSSION
This trial examined the degree to which a physical activity program grounded in evidence-based principles and developed using an integrated research-practice partnership would be adopted within an existing delivery system when compared to one developed through the typical efficacy-effectiveness-dissemination pipeline. Our results suggest that integrated research-practice partnership-developed programs may be more adoptable within existing delivery systems when compared to implementing manualized and highly structured evidence-based programs. We also demonstrated that health educators consider a number of factors when making adoption decisions and, while effectiveness was one factor, it appears that implementation features (ease of delivery, resources) and perceptions of administrative support were significantly related to program adoption. Finally, it also appears that integrated research-practice partnership-developed interventions are related to higher adoption rates within the system and have a higher reach into the target population.
It is possible to conclude that the differences between the two programs were so great (e.g., 8 weeks of newsletter and mileage computation for FitEx compared to 12 weeks of in-person class instruction for ALED) that the outcome of this research study is obvious. However, it is interesting to note that the adoption rate we reported for ALED was consistent with rates that were reported in a large-scale dissemination study of ALED. Further, although ALED was developed with specific recreation centers in mind for delivery, the Virginia Cooperative Extension system delivered the program in two settings—which is identical to the rate of adoption for ALED in other systems (i.e., two classes per organization per year). This underscores the degree to which a program developed through an integrated research-practice partnership resulted in a program that was much more adoptable when compared to ALED across a variety of settings [35, 36].
While our study supported the use of integrated research-practice partnership processes to develop physical activity programs with broad reach and organizational adoption, it does not devalue the contributions of interventions developed through the traditional pipeline model. First, it is possible that ALED would be a better fit in organizations such as the YMCA or Parks and Recreation settings where weekly community programs align with the existing delivery structure and regular programming [41, 42]. Second, evidence-based interventions provide information on evidence-based principles that can be used to adapt existing, or create new, programs [38]. Third, for organizations that have similar resources and structure to deliver an evidence-based intervention, the cost of adaptation and development was likely much lower—though this is a proposition ripe for empirical testing
Specific to the variables that predict adoption, it was unsurprising that the perceived ease of delivery was an important predictor in the degree to which health educators adopted a program as this is a central tenant of the diffusion of innovations [43] and was an important consideration for when developing the FitEx program. Our data also echoed the idea that program champions and concurrent administrative support [44] may enhance program adoption and implementation. Finally, those health educators who indicated their lack of expertise as an adoption barrier may benefit from training and support to bolster their knowledge, skill set, and confidence. An integrated research-practice partnership can provide tools, ongoing training, technical assistance, and quality improvement/quality assurance [15, 45]. Finally, contrary to the idea that observability of effectiveness enhances program adoption [43], the adopters in this study were less likely to perceive that the program was effective in community settings than non-adopters. This was a surprising finding; however, this could be attributed to the unfamiliarity with the integrated research-practice partnership-developed program and the outcomes of its predecessor in another state [27]. Further research is needed to explore this relationship.
The findings from our study also support the hypothesis that an integrated research-practice partnership-developed behavioral intervention can achieve the same level of effectiveness when compared to a behavioral intervention developed through a more traditional pipeline scientific paradigm. It could be hypothesized that interventions developed through an integrated process—to balance scientific evidence with the practical realities of a delivery system—will be more pragmatic and result in high implementation fidelity [46]. We recommend further research in this area that also applies comparative effectiveness study designs to examine existing programs when compared to new integrated research-practice partnership-developed programs to promote physical activity.
Using the RE-AIM planning and evaluation framework within our study, we attempted to balance internal and external validity within our approach. In doing so, our trial includes some limitations that are important to note. Specifically, Curran et al. [23] note that hybrid designs often require compromise at one level of analysis. As program adoption was our primary aim and both programs already had effectiveness data [27, 35–38], self-reported physical activity was used as a measure of program effectiveness for all participants. The low recruitment rates for ALED led to the lack of more substantive effectiveness comparisons; doing a stratified sampling would have improved our opportunity to assess effectiveness. Therefore, we chose to rely on the data from health educators who implemented the program by their own decision. We also used a single state-wide system for the integrated research-practice partnership as such FitEx may not be generalizable to community organizations—though is likely generalizable to other cooperative extension systems. This generalizability stems from the fact that Cooperative Extension is funded at the federal level and follows a similar model (e.g., hierarchy, program areas) in all states and territories. Another notable limitation is the lack of the full constitutive definition of maintenance variables as proposed by the RE-AIM framework. First, we did not include individual PA behaviors as a primary outcome measure. At the organizational level, though, we collected intent of continued program delivery. Previous research has used the intention to continue program delivery as a system-level indicator of maintenance [47, 48]. We also direct readers to a qualitative paper on this topic, including details on perceptions of program maintenance [49]. Finally, our implementation indicators were based on self-report weekly process evaluations (i.e., time spent on intervention tasks) rather than an objective checklist. However, this proxy measure did provide a description of program compliance as it related to material development and delivery.
CONCLUSIONS
This study was designed with the goal of state-wide uptake of a physical activity intervention. The integrated research-practice partnership considered each RE-AIM dimension at the outset of the intervention design [50]. The integrated research-practice partnership-developed program that was based on established intervention principles, but specifically designed to fit within the system, was more likely have positive effects on the intervention reach, effectiveness, adoption, implementation, and maintenance.
Acknowledgments
We would like to acknowledge the health educators who provided us with these data as well as Joan Wages who led the training sessions.
Compliance of ethical standards
Conflict of interest
The authors declared that they have no competing interests.
Adherence to ethical principles
This study followed accepted principles of ethical and professional conduct. The study received expedited review by the Virginia Tech Institutional Review Board (no. 08-466).
Footnotes
Implications
Practice: The integrated research-practice partnership approach to program development and delivery may improve the rate at which decision-makers adopt and implement evidence-based programs in a delivery system.
Policy: Infrastructure that supports the development and use of integrated research-practice partnerships may more swiftly impact public health.
Research: Partnering with representatives from the intended delivery system may allow for appropriate tailoring and improve the perceptions of evidence-based programming.
References
- 1.U.S. Department of Health and Human Services. Office of Disease Prevention and Health Promotion. Healthy People 2020. Washington, DC. http://www.healthypeople.gov/. Accessed 1 Jul 2015.
- 2.Mokdad AH, Marks JS, Stroup JF, et al. Actual causes of death in the United States, 2000. JAMA. 2004 doi: 10.1001/jama.291.10.1238. [DOI] [PubMed] [Google Scholar]
- 3.U.S. Department of Health and Human Services. The power of prevention. Steps to a HealthierUS: a program and policy perspective. 2003: 1–9.
- 4.Estabrooks PA, Glasgow RE, Dzewaltowski DA. Physical activity promotion though primary care. JAMA. 2003;289:2913–2916. doi: 10.1001/jama.289.22.2913. [DOI] [PubMed] [Google Scholar]
- 5.Morbidity and Mortality Weekly Report. Prevalence of regular physical activity among adults—United States, 2001 and 2005. 2007;56(46). [PubMed]
- 6.Kahn EB, Ramsey LT, Brownson RC, Task Force on Community Preventive Services et al. The effectiveness of interventions to increase physical activity. Am J Prev Med. 2002;22(4S):73–107. doi: 10.1016/S0749-3797(02)00434-8. [DOI] [PubMed] [Google Scholar]
- 7.Boutron I, Moher D, Altman DG, Schulz K, Ravaud P, for the CONSORT group. Methods and processes of the CONSORT group: example of an extension for trials assessing nonpharmacologic treatments. Ann Intern Med. 2008;W60-W67. [DOI] [PubMed]
- 8.Glasgow RE, Klesges LM, Dzewaltowski DA, et al. The future of health behavior change research: what is needed to improve translation of research into health promotion practice? ABM. 2004;27:3–13. doi: 10.1207/s15324796abm2701_2. [DOI] [PubMed] [Google Scholar]
- 9.Estabrooks P, Dzewaltowski DA, Glasgow R, et al. School-based health promotion: issues related to translating research into practice. J Sch Health. 2002;73:21–28. doi: 10.1111/j.1746-1561.2003.tb06554.x. [DOI] [PubMed] [Google Scholar]
- 10.Bull SS, Gillette C, Glasgow R, et al. Worksite health promotion research: to what extent can we generalize the results and what is needed to translate research to practice? Health Educ and Beh. 2003;30:537–549. doi: 10.1177/1090198103254340. [DOI] [PubMed] [Google Scholar]
- 11.Dzewaltowski DA, Estabrooks PA, Glasgow RE. The future of physical activity behavior change research: what is needed to improve translation of research into health promotion practice? Exerc Sport Sci Rev. 2004;32:57–63. doi: 10.1097/00003677-200404000-00004. [DOI] [PubMed] [Google Scholar]
- 12.Miller R, Shinn M. Learning from communities: overcoming difficulties in dissemination of prevention and promotion efforts. Am J Commun Psychol. 2005;35:3–4. doi: 10.1007/s10464-005-3395-1. [DOI] [PubMed] [Google Scholar]
- 13.Damschroder LJ, Aron DC, Keith RE, et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Imp Sci. 2009 doi: 10.1186/1748-5908-4-50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Estabrooks PA, Glasgow RE. Translating effective clinic-based physical activity interventions into practice. Am J Prev Med. 2006;31:45–56. doi: 10.1016/j.amepre.2006.06.019. [DOI] [PubMed] [Google Scholar]
- 15.Wandersman A, Duffy J, Flaspohler P, et al. Bridging the gap between prevention research and practice: the interactive systems framework for dissemination and implementation. Am J Commun Psychol. 2008;41(3–4):171–181. doi: 10.1007/s10464-008-9174-z. [DOI] [PubMed] [Google Scholar]
- 16.National Cancer Institute of Canada 2006. Final report to the NCIC Board from the Joint Working Group on Translational Research and Knowledge Integration of the Advisory Committee on Research and the Joint Advisory Committee for Cancer Control.
- 17.Pinard CA, Hart MH, Hodgkins Y, et al. Smart choices for healthy families: a pilot study for the treatment of childhood obesity in low-income families. Health Edu & Behav. 2012;39(4):433–445. doi: 10.1177/1090198111425686. [DOI] [PubMed] [Google Scholar]
- 18.Hasche LK, Lenze S, Brown T, et al. Adapting collaborative depression care for public community long-term care: using research-practice partnerships. Adm Policy Ment Health. 2014;41:687–696. doi: 10.1007/s10488-013-0519-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Harrison MB, Graham ID. Roadmap for a participatory research–practice partnership to implement evidence. Worldviews Evid Based Nurs. 2012;9(4):210–220. doi: 10.1111/j.1741-6787.2012.00256.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Stetler CB, Legro MW, Wallance CM, et al. The role of formative evaluation in implementation research and the QUERI experience. J Gen Intern Med. 2006;21:S1–S8. doi: 10.1007/s11606-006-0267-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Stetler CB, Mittman BS, Francis J. Overview of the VA Quality Enhancement Research Initiative (QUERI) and QUERI theme articles: QUERI series. Implement Sci. 2008;3:8. doi: 10.1186/1748-5908-3-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Osher DM, Quinn MM, Poirier JM, et al. Deconstructing the pipeline: using efficacy, effectiveness, and cost-benefit data to reduce minority youth incarceration. New Dir Youth Dev. 2003;99:91–120. doi: 10.1002/yd.56. [DOI] [PubMed] [Google Scholar]
- 23.Curran GM, Bauer M, Mittman B, et al. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012 doi: 10.1097/MLR.0b013e3182408812. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89:1322–1327. doi: 10.2105/AJPH.89.9.1322. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Smith-Lever Act. http://extension100years.net/en/administration/about_us/chancellors_office/extension/about-smith-lever/. Accessed 1 Jul 2015.
- 26.Estabrooks PA, Harden SM, Burke SM. Group dynamics in physical activity promotion: what works? Soc Personal Psychol Compass. 2012;6(1):18–40. doi: 10.1111/j.1751-9004.2011.00409.x. [DOI] [Google Scholar]
- 27.Estabrooks PA, Bradshaw M, Dzewaltowski DA, et al. Determining the impact of Walk Kansas: applying a team-building approach to community physical activity promotion. Ann Behav Med. 2008 doi: 10.1007/s12160-008-9040-0. [DOI] [PubMed] [Google Scholar]
- 28.Hongu N, Block L, Sanchez SA, Day SH, Harris RB. Walk across Arizona, community-based walking program: promoting and sustaining physical activity in older adults. The Forum for Family and Consumer Issues. 2010
- 29.Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011 doi: 10.1186/1748-5908-6-42. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Seymour RB, Hughes SL, Campbell RT, et al. Comparison of two methods of conducting the Fit and Strong! program. Arthritis Rheum. 2009 doi: 10.1002/art.24517. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Pazoki R, Nabipour I, Seyednezami N, et al. Effects of a community-based healthy heart program on increasing healthy women’s physical activity: a randomized controlled trial guided by community-based participatory research (CBPR) BMC Public Health. 2007 doi: 10.1186/1471-2458-7-216. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.2008 physical activity guidelines for Americans. Health.gov. http://www.health.gov/paguidelines/guidelines/. Accessed 1 Jul 2015.
- 33.Thompson B, Denmark-Wahnefried W, Taylor G, et al. Baseline fruit and vegetable intake among adults in seven 5 a day study centers located in diverse geographic areas. J Am Diet Assoc. 1999;99:1241–1248. doi: 10.1016/S0002-8223(99)00306-5. [DOI] [PubMed] [Google Scholar]
- 34.Human Kinetics, Champaign. www.humankinetics.com. Accessed 1 Jul 2015.
- 35.Dunn AL, Marcus BH, Kampert JB, et al. Comparison of lifestyle and structured interventions to increase physical activity and cardiorespiratory fitness: a randomized trial. JAMA. 1999;281:327–34. doi: 10.1001/jama.281.4.327. [DOI] [PubMed] [Google Scholar]
- 36.Dunn AL, Andersen RE, Jakicic JM. Lifestyle physical activity interventions—history, short- and long-term effects, and recommendations. Am J Prev Med. 1998;15:398–412. doi: 10.1016/S0749-3797(98)00084-1. [DOI] [PubMed] [Google Scholar]
- 37.Wilcox S, Dowda M, Griffin SF, et al. Results of the first year of Active for Life®: translation of two evidence-based physical activity programs for older adults into community settings. Am J Public Health. 2006;96:1201–1206. doi: 10.2105/AJPH.2005.074690. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Wilcox S, Dowda M, Leviton LC. Active for Life: final results from the translation of two physical activity programs. Am J Prev Med. 2008;35:340–351. doi: 10.1016/j.amepre.2008.07.001. [DOI] [PubMed] [Google Scholar]
- 39.Topolski TD, LoGerfo J, Patrick DL, et al. The rapid assessment of physical activity (RAPA) among older adults. Prev Chronic Dis. 2006;3:1–8. [PMC free article] [PubMed] [Google Scholar]
- 40.Glasgow RE, Ory MG, Klesges LM, et al. Practical and relevant measures of health behavior for primary care settings. Ann Fam Med. 2005;3:73–81. doi: 10.1370/afm.261. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Bozack A, Millstein S, Garcel JM, et al. Implementation and outcomes of the New York state YMCA diabetes prevention program: a multisite community-based translation, 2010–2012. Prev Chronic Dis. 2014 doi: 10.5888/pcd11.140006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Hays LM, Finch EA, Saha C, et al. Effect of self-efficacy on weight loss: a psychosocial analysis of a community-based adaptation of the diabetes prevention program lifestyle intervention. Diabetes Spectr. 2014 doi: 10.2337/diaspect.27.4.270. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Rogers E. Attributes of innovations and their rate of adoption. New York: Free; 2003. Diffusion of innovations; pp. 219–266. [Google Scholar]
- 44.Belkora J, Edlow B, Aviv C, et al. Training community resource center and clinic personnel to prompt patients in listing questions for doctors: follow-up interviews about barriers and facilitators to the implementation of consultation planning. Implement Sci. 2008 doi: 10.1186/1748-5908-3-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Edmunds JM, Beidas RS, Kendall PC. Dissemination and implementation of evidence-based practices: training and consultation as implementation strategies. Clin Psycho. doi: 10.1111/cpsp.12031. [DOI] [PMC free article] [PubMed]
- 46.Flay BR. Efficacy and effectiveness trials (and other phases of research) in the development of health promotion programs. Prev Med. 1986;15:451–474. doi: 10.1016/0091-7435(86)90024-1. [DOI] [PubMed] [Google Scholar]
- 47.Tibbits MK, Bumbarger BK, Kyler SJ, et al. Sustaining evidence-based interventions under real-world conditions: results from a large-scale diffusion project. Prev Sci. 2010;11:252–262. doi: 10.1007/s11121-010-0170-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Estabrooks PA, Smith-Ray RL, Dzewaltowski DA, Wilcox S, et al. Sustainability of evidence-based community-based physical activity programs for older adults: lessons from Active for Life. Transl Behav Med. 2011;1(2):208–215. doi: 10.1007/s13142-011-0039-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Johnson SB, Harden SM, Estabooks PA. TBM; under review
- 50.Klesges LM, Estabrooks PA, Dzewaltowski DA, et al. Beginning with the application in mind: designing and planning health behavior change interventions to enhance dissemination. Ann Behav Med. 2005;29:66––75. doi: 10.1207/s15324796abm2902s_10. [DOI] [PubMed] [Google Scholar]