Abstract
This study is a longitudinal investigation of the PROSPER partnership model designed to evaluate the level of sustainability funding by community prevention teams, including which factors impact teams’ generation of sustainable funding. Community teams were responsible for choosing, implementing with quality, and sustaining evidence-based programs (EBPs) intended to reduce substance misuse and promote positive youth and family development. Fourteen US rural communities and small towns were studied. Data were collected from PROSPER community team members (N=164) and Prevention Coordinators (N=10), over a 5-year period. Global and specific aspects of team functioning were assessed over 6 waves. Outcome measures were the total funds (cash and in-kind) raised to implement prevention programs. All 14 community teams were sustained for the first five years. However, there was substantial variability in the amount of funds raised and these differences were predicted by earlier and concurrent team functioning and by team sustainability planning. Given the sufficient infrastructure and ongoing technical assistance provided by the PROSPER partnership model, local sustainability of EBPs is achievable.
Sustained implementation of evidence-based prevention programs (EBPs) depends on successful resource generation, especially for public health innovations that have not been fully institutionalized. Frequently in the U.S., novel public health programs are first developed and implemented with seed grants. However, the dilemma of time-limited, grant funding often leads to program implementation without a clear plan for sustainability (Adelman & Taylor, 2003; Schierer & Dearing, 2011). To promote greater sustainability of EBPs, it is necessary to identify factors contributing to successful resource generation (Trickett, Beehler, Deutsch et al., 2011). Here we investigate community prevention teams’ ability to gain sustainability funding and examine what factors over five years predict the generation of sustainable funding by 14 community teams. These teams were mobilized by county-based Cooperative Extension System (CES) educators as part of an innovative model for disseminating substance misuse EBPs, PROSPER (PROmoting School–community-university Partnerships to Enhance Resilience; Spoth, Greenberg, Bierman, & Redmond, 2004). The PROSPER study evaluates whether well-functioning teams committed to science-based prevention can reduce adolescent substance use by deploying and sustaining high-quality implementation of EBPs.
The PROSPER Partnership Model
PROSPER is a community-university partnership model (see Spoth & Greenberg, 2005), focusing on the implementation of classroom and family-based EBPs. The PROSPER structural model has three tiers: local PROSPER community teams, university-based prevention scientists and Extension administrators, and prevention coordinators (PC). The primary function of Prevention Coordinators is to provide proactive technical assistance to the teams and to serve as a liaison between the teams and scientists (Spoth et al., 2004). The community teams are led by local team leaders who are educators working for the Cooperative Extension System (CES) and include a representative from the local middle school member (who serves as a co-leader), along with representatives of various community agencies (e.g., county-based alcohol and drug prevention agencies, mental health, and human services), parents and youth. The teams were newly formed for PROPSER and did not previously exist. The teams meet on a monthly basis and operate within their county CES. In the community-level randomized trial, we have reported that the PROSPER intervention-community teams could indeed foster high-quality implementation in both the initial years of implementation as well as continuing in subsequent cohorts (Spoth et al. 2007; Spoth et al., 2011). In addition, we have reported that PROSPER’s two-year sequence of a 6th grade family program and 7th grade classroom-based program led to reductions in substance initiation and substance misuse through 4.5 years after baseline (Spoth et al., 2007; 2011) and 6.5 years after baseline (Spoth et al., 2013).
The PROSPER partnership model was designed with recognition that community partnerships proceed through a series of developmental phases (Hawkins et al., 2008; Stevenson & Mitchell, 2003; Wandersman et al., 2008). The PROSPER model has four developmental phases. The first organizational phase lasts between six and eight months and involves partnership formation activities, including recruiting key members, receiving training in the model, establishing program goals based on local needs and resources, and coalescing as a team (Feinberg et al., 2007a). During the second, operations phase, which lasts between two and three years, the major tasks involve implementing chosen programs, applying a monitoring system to ensure implementation quality, and initiating sustainability training and planning.
The focus during phase three, early sustainability (Years 3 and 4), is on sustaining the effective activities of the local community team. It often involves engaging other community entities to create a permanent structure for the team’s operations and sponsored activities (Feinberg, et al., 2007; Spoth & Greenberg, 2005). During the early sustainability phase, proactive TA places a greater focus on planning for sustainability and gaining local funding support. The final phase, centering on ongoing operations and sustainability, involves strengthening the internal and external functioning of community teams, maintaining the quality of program implementation, and further developing local funds for ongoing support.
To create a gradual process of local ownership, PROSPER initiated a multi-year process of transition in financial support. After PROSPER teams had implemented programs with two different cohorts of youth, the university-based grant provided limited matching funds to incentivize local fundraising (in Years 3 and 4); these matching funds were discontinued in Year 5. However, during Year 5, the university-based grant continued to support two aspects of team operations: 25% of the salary of the Extension educator (PROSPER team leader) and support for basic team functions, including meeting costs, marketing, and recruitment. During subsequent years (Years 6 to 8) the percentage of the Extension educator’s salary provided by grant funding was gradually decreased to zero. Given the extensive support provided by the PCs to the local PROSPER teams (e.g., bi-weekly phone calls, workshops, discussions regarding program implementation, advice regarding funding possibilities, assistance with grant applications), as well as our introduction of the need for sustainability planning early in the team’s development, the teams were in an advantageous position for generating resources.
PROSPER adopted the sustainability model of Johnson (Johnson et al. 2004; Spoth & Greenberg, 2011) which includes two key indicators: maintaining program implementation by gaining long-term financial support and maintaining implementation quality; and maintaining an effective team, as indicated by strong institutional and financial connections, commitment to quality implementation of EBIs, and an effective communications plan (Scheirer, 2005).
We previously examined factors that predict well-functioning teams in the organizational and operational phases. We found that that greater community level poverty was negatively related to early team functioning (as measured by Global Team Functioning – see methods) and that greater perceived community readiness and positive team member prevention attitudes positively predicted the quality of team functioning 6 and 18 months after PROSPER began (Feinberg et al., 2007a; Greenberg, Feinberg, Meyer, Spoth, & Redmond, 2007).
Recently we investigated how the characteristics of PROSPER teams during the organizational and operations phases (phases 1 and 2) predicted the quality of team sustainability planning at Year 4 (phase 3; Perkins et al., 2011). We found that PROSPER global team functioning during the organizational and implementation phases predicted quality of sustainability planning in Year 4. In addition, team member recruitment and integration and participation also predicted later sustainability planning. These findings suggest that well organized teams anticipated change and had procedures in place for integrating new members to introduce new expertise and facilitate planning. Finally, teams that reported support from local businesses and agencies and good coordination between the CES educator and the school representatives, had more well-developed sustainability plans. These findings support existing hypotheses that effective sustainability is related to early planning and support (Berry et al.,2005; Johnson et al., 2004; Scheirer, 2005; Shediac-Rizkallah & Bone, 1998).
While early global team functioning was negatively related to the community poverty and community readiness (Greenberg et al., 2007), neither poverty nor readiness influenced Year 4 sustainability planning. Thus, while certain factors may influence early team functioning, it is the quality of team functioning, itself, that predicted the team’s sustainability planning. This finding indicates the need for a developmental model to understand community team development. This paper extends our evaluation of PROSPER to the next logical step. We examine (1) whether PROSPER teams generated resources to sustain local programs, and (2) which factors predict team financial success. Figure 1 presents the conceptual model of predictors of financial sustainability. We hypothesized that global aspects of local team functioning that previously predicted sustainability planning would predict funding success. In addition, we hypothesized that specific aspects of team functioning (the ability to integrate new team members, member participation, community buy-in and coordination with other local entities) would influence fundraising success. Finally, we expected the quality of team sustainability planning, itself, would predict concurrent and later funding outcomes.
Method
Procedure
Data were collected from the 14 PROSPER intervention communities (7 in Iowa and 7 in Pennsylvania). Twenty-eight communities were initially selected on the basis of four eligibility criteria: (a) school district enrollment between less than 5,200 students in non-metropolitan areas; (b) at least 15% of families eligible for free lunch; (c) < than 50% of the population employed by or attending a university; and (d) no current involvement with university-affiliated, youth prevention research. After selection, they were blocked by size and state and randomized into 14 intervention and control conditions. Overall, this was a rural/small town sample with a median household income of $37,070, and 29% of students receiving free or reduced lunches. See Greenberg et al. (2007) for more detailed information regarding community selection.
Waves 2 through 6 of data are reported here (see Table 1 for a list of measures by year). Wave 1 pretest findings are reported elsewhere (e.g., Feinberg et al., 2007; Greenberg et al., 2007). Team data at organizational and operations phases (Waves 2 and 3) were collected 6 and 18 months after team formation, when team operations and program implementation were fully supported by university funds. During Wave 2, teams were initiating the implementation of 6th-grade family programs (e.g., Strengthening Families Program 10–14); at Wave 3, teams were planning the second round of family programs and the first round of 7th-grade school programming (i.e., having chosen Life Skills Training, Project Alert, or All Stars).
Table 1.
2002 | 2003 | 2004 | 2005 | 2006 | 2007 | ||||||
---|---|---|---|---|---|---|---|---|---|---|---|
Apr – Jun | July – Dec | Jan – June | July – Dec | Jan – June | July – Dec | Jan – June | July – Dec | Jan – June | July – Dec | Jan – June | |
W1 | W2 | W3 | W4 | W5 | W6 | ||||||
Data Collection | |||||||||||
TM & AD Interviews | X | X | X | X | X | X | |||||
PC Ratings | X | X | X | X | X | X | X | X | X | X | X |
Funding Reports | X | X | X | X | X | X | |||||
Program Implementation | |||||||||||
Family Program (6th grade) | C1 | C2 | C2 | A | A | A | A | A | A | ||
School Program (7th grade) | C1 | C1 | C2 | A | A | A | A | A | |||
Funding Source | |||||||||||
Team Functions | G | G | G | G | G | G | G | G | G | G | G |
Family Programa | G | G | G | T+ | T+ | T+ | T+ | T+ | T+ | ||
School Program | G | G | G | G | G | G | T | T |
Notes: TM=team member; AD=agency director; PC=prevention coordinator. W1–W6=Data collection waves 1–6; C1=Cohort 1; C2=Cohort 2; A=Additional cohorts, not studied. G= research grant funds; T=team-raised funds only; T+= team-raised funds with matching incentive from research grant.
During Waves 4–6, program implementation continued with new cohorts. By Wave 4 (early sustainability phase), teams were required to financially support the family program costs, but matching funds were provided if teams increased the percentage of families recruited over the previous year. By Wave 5, teams were responsible for financial support of both family and school programs, but the matching incentive for family programs remained. At Wave 6, teams fully supported all program implementation. Costs to run each seven-week family program were approximately $3,000/group with the average team conducting about 2–5 groups per year. The classroom program involved fewer financial resources ($500 per year), but required the schools to commit teachers’ time. By Wave 6 (year 5) only 25% of the team leaders salary and support for basic team functions ($1,500 to $4,500 depending on community size) still were supported.
Participants
Three types of respondents were assessed: intervention team members (TM) and team leaders (TL), agency directors of human service organizations (ADs), and PCs who provided TA. Prevention team members (N=164) included a local CES team leader, a school co-leader, human service representatives (e.g., mental health and substance abuse) parents and youth. On average 12 members per community participated (range: 8–16) mean age of 42 years. Although, youth were on the teams, we did not collect this data from them. The majority were White (99%), female (71%), had a college degree (89%), and lived near the school (85%). Agency Directors (N=78) either supervised a team member or served as a team member. All ADs were White, 59% were male and all had college degrees. Ten part-time PCs in the two states provided TA. PCs had an average of 18 years experience with CES or prevention TA.
Measures
Team functioning
At each wave respondents participated in a face-to-face interview and were compensated $20 (See Table 1). PCs completed annual assessments. All team functioning scales averaged the scale scores and were aggregated to the team level. Wave 2 and 3 data were combined (averaged), whereas data from Waves 4 through 6 were analyzed separately.
At each wave, assessment of team functioning consisted of two global scales – one rated by TMs and one by PCs – and four specific scales. The Global Team Functioning (GTF) scales were created by averaging the three subscales. For PCs these scales: were Team Focus On Work, Team Goals, and Team Culture (PC average α across waves, = 0.91). These three subscales plus one additional subscale, Team Leadership, were used for TM’s responses (average α across waves= 0.90). Work Focus (5 items; adapted from Moos, 1981) assessed the degree of team work-orientation. The Goals subscale (2 items; Greenberg et al., 2007) measured whether the team had developed goals and procedures. The Culture and Leadership subscales (8 items each, Kegler, Steckler, McLeroy, & Malek, 1998) assessed team members perceived cohesion and unity and whether team leadership encouraged collaboration and displayed effective leadership behaviors (see Greenberg et al., 2007 for item examples).
Four scales assessed specific aspects of team functioning. Two scales were completed by TMs (New Member Integration and Enablers), one by PCs (Participation), and one by TMs and ADs (Community Buy-in). New Member Integration (6 items- average α across waves=.87; Taylor-Powell, Rossing, & Geran, 1998) assessed whether new members were effectively integrated into the team. Enablers (6 items: average α across waves=.79) assessed how various factors contributed to team’s task accomplishment; factors included team leadership, active involvement of members, support from local agencies and businesses, and coordination between the school and CES. Community Buy-In (5 items; average α across waves=.73) measured whether school administrators and agency leaders championed PROSPER. Participation (5 items average α across waves=.76) assessed team members participation in meetings, trainings, board meetings, and other team activities (see Greenberg et al., 2007 for item examples).
Waves 4 and 5: Sustainability planning
Three scales assessed team characteristics, plans, and attitudes regarding long-term sustainability. Team Expertise (TMs and PCs) assessed the team’s expertise to sustain PROSPER (4 items; W4 α=.71, W5 α=.69); for example, “Our local Prosper team has the expertise needed to fulfill the goal of sustaining PROSPER.” Team Confidence (TMs and ADs) assessed the expectations of sustainability (2 items; W4 α=.68, W5 α=.68). Team Funding Plans (TM only) assessed teams plans to secure funds beyond the grant (4 items; W4 α=.85, W5 α= 84); for example, “Has the PROSPER team developed a realistic, concrete sustainability plan for continuing prevention programs when (university) funds cease?”
Waves 4–6: Funding
From Wave 4, TLs and PCs reported team in-kind and financial contributions twice/year. This included the amount received and funding sources: federal, state, local government, school system, local NGO, foundation/charity, business, local fundraising events, and other. In-kind contributions were non-cash donations and included hours volunteered by program facilitators, local arrangers, child care providers, administrative support; and donations such as meals and incentives for participation, and supplies. Hourly rates used were as follows: $20.00 for SFP facilitator time (based on typical rates across sites); $18.77 for local logistics, recruitment, and team administrative support time (based on estimated value of volunteer time: Independent Sector, 2006); $9.06 for child care during SFP sessions (U.S. Bureau of Labor Statistics, 2006); and $34.06 for school program facilitator time (Greene & Winters, 2007).
Three variables for each year were calculated: in-kind contributions, cash contributions, and total (in-kind plus cash) contributions. To adjust for the size of the community, funding amounts were divided by the communities’ 7th-grade enrollment (in 2005–06)—yielding variables that indicate funding per child. Average grade enrollment was 218 students (SD=107; Iowa Department of Education, 2006; Pennsylvania Department of Education, 2006).
Community poverty
The percentage of families living below the federal poverty threshold was obtained for each school district from the National Center for Education Statistics (2003); these data were based on the 2000 United States Census.
Results
Table 2 presents descriptive statistics on community-level sustainability funding as well as a breakdown of cash and in-kind support by Wave. There was considerable resource generation in the average community as well as substantial variation between communities. Team funding amounts always were higher for cash than for in-kind contributions, and both types of funding increased over time, particularly from Wave 4 to Wave 5. Table 3 presents a further breakdown of the funding by source. It indicates that funds were obtained from numerous sources with no single sources accounting for greater than one-third of the overall funding.
Table 2.
Total Funding | Funding per Student | |||
---|---|---|---|---|
Mean | SD | Mean | SD | |
Wave 4 (n=13) | ||||
In-Kind | $4,784.77 | $3,728.59 | $19.53 | $10.06 |
Cash | $10,861.66 | $5,724.09 | $46.45 | $24.88 |
Total | $15,646.43 | $8,524.33 | $65.99 | $29.57 |
Wave 5 (n=14) | ||||
In-Kind | $7,382.90 | $5,342.73 | $35.56 | $27.87 |
Cash | $13,946.12 | $8,361.60 | $61.38 | $45.65 |
Total | $21,329.02 | $10,619.17 | $96.94 | $60.49 |
Wave 6 (n=14) | ||||
In-Kind | $7,606.01 | $4,027.75 | $39.10 | $22.80 |
Cash | $15,017.11 | $13,025.71 | $63.02 | $50.65 |
Total | $22,623.12 | $13,790.21 | $102.12 | $55.22 |
Note: Funding per student reflects total funding divided by number of students in 7th grade.
Table 3.
Source of dollars | Wave 4 | Wave 5 | Wave 6 | |||
---|---|---|---|---|---|---|
Amount | % of total | Amount | % of total | Amount | % of total | |
Federal | $20,148 | 15% | $2,665 | 2% | $0 | 0% |
State | $18,885 | 14% | $20,498 | 12% | $44,815 | 25% |
Local | $500 | 0% | $5,250 | 3% | $0 | 0% |
School district | $6,405 | 5% | $52,180 | 31% | $32,841 | 18% |
Non-governmental agency | $12,716 | 10% | $3,734 | 2% | $21,011 | 12% |
Foundation | $23,780 | 18% | $18,406 | 11% | $33,332 | 18% |
Business | $4,240 | 3% | $10,875 | 6% | $4,425 | 2% |
Individuals | $700 | 1% | $1,520 | 1% | $859 | 0% |
Fundraising activities | $2,802 | 2% | $6,921 | 4% | $10,002 | 6% |
PROSPER | $36,001 | 28% | $30,276 | 18% | $11,954 | 7% |
Extension | $3,138 | 2% | $2,826 | 2% | $10,819 | 6% |
Other | $1,018 | 1% | $12,206 | 7% | $10,149 | 6% |
| ||||||
Total | $130,334 | 100% | $167,358 | 100% | $180,207 | 100% |
Because our interest is in community-level patterns and our sample size was small (N=14), statistical analyses were limited to bivariate correlations. Given low power, we interpreted correlations as meaningful if they were above .30 and portrayed a consistent pattern. While the specification of .30 as the cutoff is somewhat arbitrary, we believe it is reasonable considering that an association of over .30 represents about 10% of the variance (Feinberg, Greenberg, Osgood, Anderson, & Babinski, 2002). Correlations between poverty and funding per student indicated an inverse relation between poverty and cash funding for Wave 4 and 5; the correlations were as follows (in-kind, cash, total): Wave 4 funding (r = 0.19, −0.33, −0.21), Wave 5 funding (r = 0.16, −0.34, −0.18), and Wave 6 funding (r =0.16, 0.04, 0.11).
Team Functioning and Sustainability Funding
Team member reported GTF
Associations between Wave 2/3 TM-reported GTF and sustainability funding were especially strong for W4 cash (r=.62) and total funding (r=.65) and had weaker relationships to W5 and W6 funding (Table 4). Similarly, W4 GTF was substantially related with W5 funding; in-kind (r=.46), cash (r=.48) and total funding (r=.56), but less related to W6 funding. W5 GTF was strongly related to concurrent funding, but was inversely linked to W6 in-kind funding and weakly associated with W6 cash and total funding. The overall pattern of relations between PC-reported GTF and sustainability mirrored the pattern for TM reports.
Table 4.
In-kind | Cash | Total | ||||||||
---|---|---|---|---|---|---|---|---|---|---|
Wave 4 | Wave 5 | Wave 6 | Wave 4 | Wave 5 | Wave 6 | Wave 4 | Wave 5 | Wave 6 | ||
Global Team Functioning: | Wave 2/3 | 0.41 | 0.45 | −0.24 | 0.62* | 0.18 | 0.31 | 0.65* | 0.34 | 0.19 |
Wave 4 | 0.47 | 0.46 | −0.29 | 0.67* | 0.48+ | 0.35 | 0.71* | 0.56* | 0.21 | |
TM-Report | Wave 5 | 0.31 | −0.47+ | 0.58* | 0.39 | 0.56* | 0.17 | |||
Wave 6 | −0.35 | 0.24 | 0.08 | |||||||
| ||||||||||
Global Team Functioning: | Wave 2/3 | 0.22 | 0.37 | −0.21 | 0.53+ | 0.06 | 0.18 | 0.51+ | 0.22 | 0.08 |
Wave 4 | 0.34 | 0.34 | −0.01 | 0.79* | 0.50+ | 0.52+ | 0.76* | 0.52+ | 0.47+ | |
PC-Report | Wave 5 | 0.46 | −0.11 | 0.56* | 0.27 | 0.62* | 0.20 | |||
Wave 6 | 0.26 | 0.27 | 0.36 | |||||||
| ||||||||||
Participation | Wave 2/3 | 0.45 | 0.23 | −0.45 | 0.64* | 0.22 | 0.39 | 0.68* | 0.27 | 0.18 |
Wave 4 | 0.24 | 0.24 | −0.05 | 0.74* | 0.50+ | 0.61* | 0.68* | 0.47+ | 0.54+ | |
Wave 5 | 0.37 | −0.14 | 0.74* | 0.48+ | 0.71* | 0.38 | ||||
Wave 6 | 0.15 | 0.39 | 0.43 | |||||||
| ||||||||||
New Member Integration | Wave 2/3 | −0.04 | 0.41 | −0.20 | 0.38 | 0.52+ | 0.39 | 0.29 | 0.56* | 0.28 |
Wave 4 | 0.53+ | 0.31 | −0.46 | 0.54+ | 0.45 | 0.43 | 0.62* | 0.47+ | 0.21 | |
Wave 5 | 0.26 | −0.40 | 0.65* | 0.36 | 0.59* | 0.17 | ||||
Wave 6 | −0.24 | 0.29 | 0.16 | |||||||
| ||||||||||
Community Buy-In | Wave 2/3 | 0.35 | −0.39 | −0.03 | 0.07 | −0.10 | 0.18 | 0.18 | −0.25 | 0.15 |
Wave 4 | 0.04 | −0.19 | −0.24 | −0.04 | 0.14 | 0.20 | −0.02 | 0.01 | 0.08 | |
Wave 5 | 0.25 | −0.17 | 0.39 | 0.67* | 0.40 | 0.55* | ||||
Wave 6 | 0.00 | 0.74* | 0.68* | |||||||
| ||||||||||
Enablers | Wave 2/3 | 0.59* | 0.36 | −0.47+ | 0.71* | 0.23 | 0.32 | 0.77* | 0.33 | 0.10 |
Wave 4 | 0.52+ | 0.03 | −0.54+ | 0.49 | 0.27 | 0.46 | 0.57* | 0.21 | 0.20 | |
Wave 5 | 0.28 | −0.36 | 0.49+ | 0.51+ | 0.48+ | 0.32 | ||||
Wave 6 | −0.21 | 0.52+ | 0.39 |
Note:
p≤.05;
p≤.10
Specific team indicators
The overall pattern of results for specific team indicators is similar to that of GTF with the strongest associations for W4 funding and then decreasing in strength over time, with stronger relations to cash and total funding than in-kind funding. New Member Integration showed the strongest associations with cash and total funding, but these somewhat diminished by W6. Community Buy-in was not associated at the earlier waves, but W5 and W6 buy-in was strongly related to W6 cash and total dollars. W2/3 Enablers was predictive of W4 funding, but showed little prediction to W5 and W6. W4 Enablers showed little relation to W5 funding, but was positively related to W6 cash and negatively related to W6 in-kind funds. W5 and W6 Enablers were associated with concurrent funding and predictive of W6 funds.
Sustainability Planning and Sustainability Funding per Student
Relations between perceived Team Expertise and cash and total funding were strong across all waves, with multiple correlations at or above r=.70 (see Table 5). With few exceptions, associations between this Team Expertise and in-kind funding were not noteworthy. W4 Team Confidence showed no relation to later funding. In contrast, W5 Confidence was associated with concurrent funding and predictive of W6 cash (r=.69) and total funding (r=.75). W4 funding plans were related to concurrent funding and W5 cash (r=.44) and total funding (r=.37). W5 funding plans were related to concurrent cash and total funding, and were positively related to W6 cash (r=.36) and negatively related to W6 in-kind (r=-.41).
Table 5.
In-kind | Cash | Total | ||||||||
---|---|---|---|---|---|---|---|---|---|---|
Wave 4 | Wave 5 | Wave 6 | Wave 4 | Wave 5 | Wave 6 | Wave 4 | Wave 5 | Wave 6 | ||
Team Expertise | Wave 4 | 0.29 | 0.23 | −0.17 | 0.75* | 0.51+ | 0.70* | 0.71* | 0.48+ | 0.58* |
Wave 5 | 0.56* | −0.17 | 0.78* | 0.56* | 0.82* | 0.45 | ||||
Wave 6 | 0.26 | 0.58* | 0.64* | |||||||
| ||||||||||
Team Confidence | Wave 4 | 0.02 | 0.24 | 0.06 | 0.10 | 0.00 | 0.16 | 0.09 | 0.11 | 0.17 |
Wave 5 | 0.23 | 0.27 | 0.50+ | 0.69* | 0.46 | 0.75* | ||||
Wave 6 | 0.30 | 0.41 | 0.50+ | |||||||
| ||||||||||
Team Funding Plans | Wave 4 | 0.20 | 0.11 | −0.61* | 0.56* | 0.44 | 0.18 | 0.52+ | 0.37 | −0.08 |
Wave 5 | 0.15 | −0.41 | 0.51+ | 0.36 | 0.44 | 0.16 | ||||
Wave 6 | −0.28 | 0.45 | 0.30 |
Note:
p≤.05;
p≤.10
Discussion
A key finding is that all 14 community teams were largely successful in raising local funds and in-kind community services during the two years of the early sustainability phase. Average community funding increased from about $15,000 (W4) to $20,000 two years later (W6). On average, 60% of total support generated was financial (“cash”) and approximately 40% was in-kind assistance. This represented substantial resource generation for relatively small, mostly rural communities, and by Wave 6, averaged approximately $100 per student. Thus, the funding, while showing substantial variability between communities, was sufficient for all 14 communities to continue to provide the family and school-based intervention program.
These results may not be readily generalizable to most cases where local communities attempt to find ongoing local support for innovative EBPs; but, they do indicate what is possible when a prevention system is designed to support a gradual process of sustainability and provides proactive TA. The study planned gradual withdrawal of financial support with ongoing TA that was provided by a federal grant throughout the five years. Local funds were derived from a multiplicity of sources with no single dominant source. Community teams received allocations by school boards, local foundations, United Way, and state agencies; they also raised money through local activities and donations by individuals and business. In-kind contributions were largely derived from volunteers devoting time to organizing, providing child care, and leading family sessions, but also included food donations, and incentives for participating families
It is useful to note that the development and evolution of the PROSPER teams likely enhanced CES Educator networking and related community social capital. Although a few CES educators already had some contacts with the schools, none had the degree of close association that was created and required by PROSPER – in which EBPs were implemented in very close collaboration with the schools. In fact, we were surprised that CES educators generally had quite weak relations with local middle schools. We observed a remarkable growth in relations between the CES educators and local school and service agency personnel. Over the five years, CES educators developed expertise in prevention programming and often were consulted by local agencies on the use of EBPs. This development of new social capital recently was documented, showing the substantial increase in positive views of both the schools and CES that resulted from PROSPER (Chilenski, Ang, Greenberg, Feinberg, & Spoth, 2013)
Team Functioning Predictors of Financial Sustainability
Regarding our second question, early assessments of team functioning and sustainability planning did predict variability in gaining local financial support, as delineated in Figure 1. Further, variables as independently assessed by TMs, PCs, and ADs all contributed to these predictions. We found a strong association between early team functioning and funding raised by local teams in later waves. Thus, the quality of team functioning during early phases predicted how effective teams were at raising funds subsequently. Wave 2 and 3 functioning strongly predicted Wave 4 funds, while Wave 4 functioning predicted Wave 5 funds. However, Wave 5 functioning did not predict Wave 6 funding. A similar pattern emerged from independent team ratings made by the PCs who provided technical assistance. Thus, it appears that the quality of team functioning impacted sustainability success three and four years, but not five years, after team formation.
Although team functioning predicted both dollars and total support, there was a negative relationship between team functioning and in-kind support from Waves 4 to 6. This pattern may be due to a decreased need for in-kind contributions when teams that obtain greater financial support—teams that raised more dollars were less likely to use volunteers.
Specific indicators of team functioning also predicted later funding success. Participation (as rated by PCs) predicted funding in each subsequent wave. New Member Integration was strongly related to funding at Waves 4 and 5, but not Wave 6. Well-organized teams were likely to plan for change, have a clear plan for integrating new members and this facilitated team processes and sustainability success. Active member participation was strongly linked to later fundraising success and was one of the only variables that continued to predict fundraising in Wave 6. It is likely that members who are actively engaged will increase a team’s success and this reflects collaborative leadership. These findings about member capacity, participation and team functioning are in accord with Foster-Fishman and colleagues’ (2001) findings and Community Coalition Action Theory (Butterfoss & Kegler, 2002), as well as previous work on coalition sustainability (Scheirer, 2005; Feinberg, Bontempo, & Greenberg, 2008).
There was a strong association between the Team Enablers scale and later funding outcomes. Teams reporting strong support from local businesses and agencies, and good coordination between the CES educator and the school staff raised greater funds. This supports Foster-Fishman’s finding about the relational capacity of teams; teams supported by good leadership and strong local support are more likely to plan ahead, have better relations with community leaders, and secure greater funds. Finally, the indicator of Buy-In by community leaders showed no prediction in the early waves, but by Wave 5 has a significant effect on funds raised in Wave 6. This pattern may reflect a developmental process in which teams that have strong early functioning are able to create better external relations that eventually support funding efforts. It should be noted that Community Buy-In was rated by the Agency Directors, not team members, and thus provides a broader and possibly somewhat less biased perspective on community processes. Thus, the broad hypotheses in Figure 1 were supported, but also qualified by somewhat different factors predicting financial outcomes in different years.
Our findings continue to deepen our understanding of the lifecycle of PROSPER teams. In earlier reports, we found that the community poverty, community readiness, and team attitudes toward prevention predicted community team functioning at Waves 2 and 3 (Greenberg et al., 2007; Feinberg et al., 2007). However, these early factors did not predict Wave 4 and 5 sustainability planning. Thus, while certain factors at team formation may affect early coalition team processes, it appears that the quality of team functioning, itself, becomes the predominant influence in predicting sustained action and success. This finding indicates the need for phase-specific support systems of TA (Schierer & Dearing, 2011).
As noted, team functioning was less predictive of financial success in the last wave. Here, factors that became more influential in later years, such as community buy-in and team expertise in funding, may have played relatively more central roles. Our experience is that teams that were more successful at funding sought new team members in the later years that had more expertise in funding and/or stronger relations with funding sources. Yet, we do not have a clear explanation as to why other factors showed weaker predictive relationships with Wave 6 fundraising. It may be that the changing role of external forces, such as the economic recession and its effects on the availability of local funds may have had a level of influence that is unmeasured here. Future research will examine if this is specific to Wave 6 or whether other factors become paramount in later time periods.
Factors that Influence Sustainability: The Role of Planning and Technical Assistance
The current results provide evidence supporting early functioning of locally-driven teams as a useful predictor of the sustainability of their initiatives. Local teams such as those that are part of the PROSPER face a daunting challenge to their sustainability (Cornerstone Consulting Group, 2002). Some community teams or coalitions flourish, but many fail to sustain their work once outside funding is withdrawn (Gomez et al., 2005). Community partnerships like PROSPER are likely to have long-term effects on public health only if they are sustained.
A common recommendation is that partnerships should plan early for sustainability (Berry, et al., 2005; Mittelmark, Hunt, Heath, & Schmid, 1993). Learning from this wisdom literature, PROSPER’s TA focused teams on the process of sustainability from the outset, increasing its emphasis as teams moved into the early sustainability phase. This proactive TA took many forms, including helping local teams market their data with local supporters, develop specific presentations to target local stakeholder groups, explore funding and fundraiser opportunities, and developing “boilerplates” for grant applications. PCs led bi-monthly, day-long learning communities involving team leaders to support peer-to-peer learning and team processes; they also regularly attended team meetings, they created clear agendas and workplans to keep teams involved and relevant, and they led annual state-wide meetings attended by all teams.
Starting in Wave 4 we presented teams with a sustainability planning process which provided a framework to guide their efforts as well as bi-annual self-assessment of their strengths and weakness in maintaining the PROSPER model. These assessments were instrumental in targeting TA to local needs. While sustainability planning has been discussed as a necessary aspect of team processes, there is little prior research about the utility of such planning and there is need for further work to understand the generalizability of the findings presented here.
PROSPER was planned from its outset to provide strong TA to teams and we gradually withdrew financial support to optimize sustainability actions. Further, the federal grant and close association with university research teams provided support and incentives that likely facilitated this process in an uncommon manner. The fact that all 14 local teams were sustained, and that local teams continued to implement programs with fidelity (Spoth et al., 2011) indicate that with substantial planning and ongoing TA, sustainability is achievable. This approach is further validated by reports of sustainability of health promotion models, given substantial planning and support (Berry, et al., 2005; Scheirer, 2005; Shediac-Rizkallah & Bone, 1998).
Limitations and Conclusion
A limitation of the study was its small sample size. Had it been large enough to permit the use of multivariate analytic models, we could better understand which factors are operative in predicting differences in coalition funding success. Further, as PROSPER purposely included only rural and small-town communities, we do not know whether having CES as the lead agency would be as effective for urban community teams where CES reputation is less well known.
In conclusion, these findings clearly indicate the importance of team functioning, team member expertise, and sustainability planning in predicting fundraising success for PROSPER community coalitions in the first years of increasingly independent, sustained existence. To our knowledge this is one of the first studies that has provided a detailed examination of the actual resources (dollars and in-kind contributions) raised during the early sustainability phase of coalitions, and how earlier team functioning and planning might predict such outcomes
Acknowledgments
Work on this paper was supported by the National Institute on Drug Abuse (grant DA 013709) and co-funding from the National Institute on Alcohol Abuse and Alcoholism (grant AA14702).
Contributor Information
Mark T. Greenberg, The Pennsylvania State University
Mark E. Feinberg, The Pennsylvania State University
Lesley E. Johnson, The Pennsylvania State University
Daniel F. Perkins, The Pennsylvania State University
Janet A. Welsh, The Pennsylvania State University
Richard L. Spoth, Iowa State University
References
- Adelman HS, Taylor L. On sustainability of project innovations as systemic change. Journal of Educational and Psychological Consultation. 2003;14:1–26. [Google Scholar]
- Berry WL, Senter S, Cheadle A, Greewald HP, Peason D. Evaluating the legacy of community health initiatives. American Journal of Evaluation. 2005;26:150–165. [Google Scholar]
- Butterfoss F, Kegler M. Toward a comprehensive understanding of community coalitions. In: DiClemente R, Crosby R, Kegler M, editors. Emerging theories in health promotion practice and research. San Francisco: Jossey-Bass; 2002. pp. 157–193. [Google Scholar]
- Chilenski SM, Ang PM, Greenberg MT, Feinberg ME, Spoth R. The impact of a prevention delivery system on perceived social capital: The PROSPER Project. Prevention Science. 2013 doi: 10.1007/s11121-012-0347-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cornerstone Consulting Group. End Games: The Challenge of Sustainability. Baltimore, MD: The Annie Casey Foundation; 2002. [Google Scholar]
- Feinberg MT, Bontempo D, Greenberg MT. Predictors and levels of sustainability of community prevention coalitions. American Journal of Preventive Medicine. 2008;34:495–501. doi: 10.1016/j.amepre.2008.01.030. [DOI] [PubMed] [Google Scholar]
- Feinberg ME, Chilenski SM, Greenberg MT, Spoth RL, Redmond C. Community and team member factors that influence the operations phase of local prevention teams: The PROSPER project. Prevention Science. 2007a;8:214–226. doi: 10.1007/s11121-007-0069-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Feinberg ME, Greenberg MT, Osgood DW, Anderson A, Babinski L. The effects of training community leaders in prevention science: Communities That Care in Pennsylvania. Evaluation and Program Planning. 2002;25:245–259. [Google Scholar]
- Feinberg ME, Greenberg MT, Osgood DW, Sartorius J, Bontempo D. Effects of the Communities That Care Model in Pennsylvania on youth risk and problem behaviors. Prevention Science. 2007b;8:180–191. doi: 10.1007/s11121-007-0073-6. [DOI] [PubMed] [Google Scholar]
- Foster-Fishman PG, Berkowitz SL, Lounsbury DW, Jacobson S, Allen NA. Building collaborative capacity in community coalitions: A review and integrative framework. American Journal of Community Psychology. 2001;29:293–299. doi: 10.1023/A:1010378613583. [DOI] [PubMed] [Google Scholar]
- Gomez BJ, Greenberg MT, Feinberg ME. Sustainability of prevention coalitions: An evaluation of Communities That Care. Prevention Science. 2005;6:199–202. doi: 10.1007/s11121-005-0003-4. [DOI] [PubMed] [Google Scholar]
- Greenberg MT, Feinberg ME, Meyer S, Spoth RL, Redmond C. Community and team member factors that influence the early phase functioning of community prevention teams: The PROSPER project. Journal of Primary Prevention. 2007;28:485–504. doi: 10.1007/s10935-007-0116-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Greene JP, Winters MA. How much are public school teachers paid? (Civic Report No 50) New York, NY: Manhattan Institute for Policy Research; 2007. [Google Scholar]
- Hawkins JD, Catalano RF, Arthur MW, Egan E, Brown EC, Abbott RD, Murray DM. Testing communities that care: rationale, design and behavioral baseline equivalence of the community youth development study. Prevention Science. 2008;9:178–190. doi: 10.1007/s11121-008-0092-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Independent Sector. History of the dollar value of a volunteer hour: 1980–2009 [Table] 2006 Retrieved from http://www.independentsector.org/volunteer_time.
- Iowa Department of Education. 2005–2006 Iowa public school PreK-12 enrollments by district, grade, race, and gender. 2006 Retrieved from http://www.iowa.gov/educate/index.
- Johnson K, Hayes C, Center H, Daley C. Building capacity and sustainable prevention innovations: A sustainability planning model. Evaluation and Program Planning. 2004;27:135–149. [Google Scholar]
- Kegler MC, Steckler A, McLeroy K, Malek SH. Factors that contribute to effective community health promotion coalitions: A study of 10 Project ASSIST Coalitions in North Carolina. Health Education and Behavior. 1998;25:338–353. doi: 10.1177/109019819802500308. [DOI] [PubMed] [Google Scholar]
- Mittelmark MB, Hunt MK, Heath GW, Schmid TL. Realistic outcomes: lessons from community-based programs for the prevention of cardiovascular diseases. Journal of Public Health Policy. 1993;14:437–462. [PubMed] [Google Scholar]
- Moos R. Work environment scale. Palo Alto, CA: Consulting Psychologists Press; 1981. [Google Scholar]
- National Center for Educational Statistics. 2003 Retrieved October 15, 2003, from http://www.nces.ed.gov/surveys/sdds/c2000.asp.
- Pennsylvania Department of Education. Public school enrollments by school 2005–06. 2006 Retrieved from http://www.portal.state.pa.us/portal/server.pt/community/enrollment.
- Perkins DP, Feinberg ME, Greenberg MT, Johnson LE, Chilenski SM, Mincemoyer CC, Spoth RL. Team factors that predict to sustainability indicators for community-based prevention teams. Evaluation and Program Planning. 2011;34:283–291. doi: 10.1016/j.evalprogplan.2010.10.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Scheirer MA. Is sustainability possible? A review and commentary on empirical studies of program sustainability. American Journal of Evaluation. 2005;26:320–347. [Google Scholar]
- Schierer MA, Dearing JW. An agenda for research on the sustainability of public health programs. American Journal of Public Health. 2011;101:2059–2067. doi: 10.2105/AJPH.2011.300193. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shediac-Rizkallah MC, Bone LR. Planning for the sustainability of community-based health programs: Conceptual framework and future directions for research, practice, and policy. Health Education Research. 1998;13(1):87–108. doi: 10.1093/her/13.1.87. [DOI] [PubMed] [Google Scholar]
- Spoth R, Greenberg M. Impact challenges in community science-with-practice: Lessons from PROSPER on transformative practitioner-scientist partnerships and prevention infrastructure development. American Journal of Community Psychology. 2011;48(1–2):106–119. doi: 10.1007/s10464-010-9417-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Spoth R, Greenberg M, Bierman K, Redmond C. PROSPER Community-university partnership model for public education systems: Capacity-building for evidence-based, competence-building prevention. Prevention Science [Invited article for Special issue] 2004;5(1):31–39. doi: 10.1023/b:prev.0000013979.52796.8b. [DOI] [PubMed] [Google Scholar]
- Spoth R, Guyll M, Lillehoj CJ, Redmond C, Greenberg M. PROSPER study of evidence-based intervention implementation quality by community-university partnerships. Journal of Community Psychology. 2007;35:981–999. doi: 10.1002/jcop.20207. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Spoth R, Guyll M, Redmond C, Greenberg M, Feinberg M. Six-Year Sustainability of Evidence-Based Intervention Implementation Quality by Community-University Partnerships: The PROSPER Study. American Journal of Community Psychology. 2011;48:412–425. doi: 10.1007/s10464-011-9430-5. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Spoth R, Redmond C, Shin C, Greenberg M, Clair S, Feinberg M. Substance use outcomes at 18 months past baseline from the PROSPER community-university partnership trial. American Journal of Preventive Medicine. 2007;32:395–402. doi: 10.1016/j.amepre.2007.01.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Spoth R, Redmond C, Clair S, Shin C, Greenberg M, Feinberg M. Preventing substance misuse through community-university partnerships and evidence-based interventions: PROSPER outcomes 4½ years past baseline. American Journal of Preventive Medicine. 2011;40:440–447. doi: 10.1016/j.amepre.2010.12.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Spoth R, Redmond C, Shin C, Greenberg M, Feinberg M, Schainker L. Longitudinal effects of universal PROSPER community university partnership delivery system effects on substance misuse through 6 1/2 years past baseline from a cluster randomized controlled intervention trial. Preventive Medicine. 2013;56:190–196. doi: 10.1016/j.ypmed.2012.12.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Spoth RL, Greenberg MT. Toward a comprehensive strategy for effective practitioner-scientist partnerships and larger-scale community benefits. American Journal of Community Psychology. 2005;35(3/4):107–126. doi: 10.1007/s10464-005-3388-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Spoth RL, Greenberg M, Bierman K, Redmond C. PROSPER community-university partnership model for public education systems: Capacity-building for evidence-based, competence-building prevention. Prevention Science. 2004;5:31–39. doi: 10.1023/b:prev.0000013979.52796.8b. [DOI] [PubMed] [Google Scholar]
- Stevenson JF, Mitchell RE. Community-level collaboration for substance abuse prevention. Journal of Primary Prevention. 2003;23(3):373–406. [Google Scholar]
- Taylor-Powell E, Rossing B, Geran J. Evaluating Collaboratives: Reaching the Potential G3658–8. Madison, WI: University of Wisconsin Extension-Program Development and Evaluation; 1998. [Google Scholar]
- Trickett EJ, Beehler S, Deutsch C, et al. Advancing the science of community-level interventions. American Journal of Public Health. 2011;101:1410–1419. doi: 10.2105/AJPH.2010.300113. [DOI] [PMC free article] [PubMed] [Google Scholar]
- United States Bureau of Labor Statistics. National compensation survey: Occupational wages in the United States, June 2006 (Summary 07–03) Washington, D.C: United States Department of Labor; 2006. [Google Scholar]
- Wandersman A, Duffy J, Flasphohler P, Noonan R, Lubell K, Stillman L, Blachman M, Dunville R, Saul J. Bridging the gap between prevention research for dissemination and implementation. American Journal of Community Psychology. 2008;41:171–181. doi: 10.1007/s10464-008-9174-z. [DOI] [PubMed] [Google Scholar]