Pathways to Sustainability: 8-year follow-up from the PROSPER Project

Abstract

The large-scale dissemination of evidence-based practices (EBPs) is often hindered by problems with sustaining initiatives past a period of initial grant funding. Communities often have difficulty generating resources needed to sustain and grow their initiatives, resulting in limited public health impact. The PROSPER project, initiated in 2001, provided community coalitions with intensive technical assistance around marketing, communications, and revenue generating strategies. Past reports from PROSPER have indicated that these coalitions were successful with sustaining their programming, and that sustainability could be predicted by early aspects of team functioning and leadership. The current study examines financial sustainability eight years following the discontinuation of grant funding, with an emphasis on sources of revenue and the relationships between revenue generation, team functioning, and EBP participation. This study used four waves of data related to resource generation collected between 2004-2010 by PROSPER teams in Iowa and Pennsylvania. Teams reported annually on the amount and sources of funding procured, as well as annual reports of team functioning and leadership and annual reports of EBP participation by youth and parents. Data revealed that teams' overall revenue generation increased over time. There was significant variation in success with revenue generation at both the community level and across the two states. Teams accessed a variety of sources. Cash revenue generation was positively and predictively associated with EBP participation, but relationships with team functioning and leadership ratings varied significantly by state. State level differences in in-kind support were also apparent. The results indicated that there are different pathways to sustainability, and that no one method works for all teams. The presence of state level infrastructures available to support prevention appeared to account for significant differences in sustainability success between Pennsylvania and Iowa.

Although the field of prevention science has made great strides toward developing an ever growing portfolio of evidence based preventive interventions (EBPIs) over the past 30 years, research suggests that the penetration of these programs into community-based settings is quite low (Hallfors, Sporer, Pankrantz, & Godette, 2000; Merikangas et al., 2011; Ringwalt et al., 2009). Prevention scientists and practitioners in the fields of prevention and public health have identified a number of barriers related to the effective delivery of EBPIs in applied settings, such as low readiness and resistance to change on the part of stakeholders (Bumbarger & Perkins, 2008), poor implementation quality (Akerlund, 2000; Backer, 2000; Pentz, Hawkins, & McClure, 2002), a lack of technical assistance (Fixsen, Blasé, Naoom, & Wallace, 2005), and a lack of infrastructure support (Spoth et al., 2013). There also is some evidence to suggest that in the presence of these barriers, communities face additional challenges generating revenues to support their programs (August, Bloomquist, Lee, Realmuto, & Hektner, 2006; Gruen et al., 2008; Spoth et al., 2013).

In this paper we examine the financial sustainability of community prevention initiatives using the PROSPER model. PROSPER (PROmoting School–community-university Partnerships to Enhance Resilience). was designed to support the dissemination and long-term sustainability of EBPIs for substance misuse prevention and healthy youth development by utilizing the infrastructure of a land-grant university's Cooperative Extension systems (CES) to catalyze and support local community prevention partnerships (Spoth, Greenberg, Bierman, & Redmond, 2004). Accordingly, after the initial start-up phase in which community teams received financial support for programming efforts, PROSPER technical assistance (TA) providers from the CES supported local communities in a proactive approach to sustainability. We have previously reported on an early stage of such sustainability efforts, during the transition from seed funding to autonomous functioning, focusing on the community and partnership factors that predicted sustainability. In this paper, we provide a description of communities' generation of sustainability resources over time, starting when they first began to seek external funding through the period in which they assume full financial responsibility for the support of their programming. The goal of this study is to inform the prevention field about the trends, predictors, and correlates of financial sustainability, along with the sources of support.

Sustainability: Definition, Theory and Empirical Findings

Research supports the idea that sustainability of EBPIs is associated with long-term public health benefits for stakeholders, but there is considerable variability in how the concept has been defined. Sustainability typically refers to the continuation of programs, often through their routinization or institutionalization within systems (Johnson, Hays, Center, & Daily, 2004; Schierer, 2005), but it is also conceptualized in terms of sustainability of public health benefits (Shediac-Rizkallah & Bone, 1998), sustained focus and prioritization of the problem addressed by the EBPI (Schierer & Dearing, 2011), as well as the sustainability of local structures and processes providing support for EBPI implementation (Feinberg, Bontempo, & Greenberg, 2008; Gomez, Greenberg, & Feinberg, 2005). Regardless of how sustainability is defined, it is invariably dependent on the availability of financial resources to support interventions and the infrastructures associated with them. Some research suggests that different sustainability outcomes (programming, infrastructure, and public health benefits) may interact with revenue generation in iterative ways. For example, the sustainability of effective and well-functioning infrastructures (e.g., coalitions, agency collaboratives) may allow for more effective fundraising and community outreach that enhances sustainability of the EBPI which, in turn, contributes to the sustainability of meaningful program impacts, such as reduced rates of substance use and problem behavior in young people (Schierer & Dearing, 2011). Additionally, some studies suggest that diversification of funding sources can promote sustainability by avoiding over-reliance on one source (Edwards & Stern, 1998; Goodman & Steckler, 1987).

Research on the sustainability of community based coalitions and their EBPIs indicates that sustainability is possible, although challenges abound. Studies of the Communities that Care (CTC) model of EBPI dissemination revealed that approximately 90% of sites were still active three years following the termination of seed funding, and that most sites were successful at expanding their resources beyond initial funding levels (Feinberg, Bontempo, & Greenberg, 2008). CTC findings suggest that sustainability is predicted by internal team dynamics (Gomez, Greenberg, & Feinberg, 2005). Other studies have also investigated the relations among sustainability, team leadership, and the way teams function, and have found links between sustainability and both the internal dynamics of teams, including perceptions of leadership, and their ability to form collaborative links with other entities (Feinberg, Bontempo, & Greenberg, 2008; Florin, Mitchell, Stevenson, & Klein, 2000; Roussos & Fawcett, 2000; Schierer, 2005; Shediac-Rizkallah & Bone, 1998).

The PROSPER Sustainability model

Earlier research on the PROSPER project emphasized outcomes related to the intervention design. These studies revealed strong, consistent, and positive impacts on multiple youth and family outcomes, including reduced rates of substance use and delinquent behavior, as well as improvements in family bonding and parenting quality (Redmond et al., 2009; Spoth et al., 2007, 2011). PROSPER adopted and evaluated the sustainability model developed by Johnson (Johnson et al., 2004; Spoth & Greenberg, 2011). This developmental model predicts that well-functioning teams that receive ongoing, proactive technical assistance will engage in a number of specific activities, including community-based delivery of EBPIs, implementation quality monitoring, development of community partnerships that promote participation and buyin, and strategic planning for sustainability (see figure 1). We predicted that teams successful with these activities in the early years of PROSPER implementation would also be effective at generating financial resources in subsequent years, which would in turn contribute to greater capacity to offer EBPIs to youth and families. Prior studies of sustainability in PROSPER found empirical support for this model, with early functioning of community teams related to subsequent sustainability planning and resource generation (Greenberg, Feinberg, Gomez, & Osgood, 2015; Perkins et al., 2011; Spoth et al., 2011).

Figure 1. The PROSPER Sustainability Model shows that improving child and family outcomes occurs through (a) building and sustaining well-functioning community prevention teams and, (b) sustaining high quality implementation of Evidence-Based programs.

Previous reports on PROSPER's sustainability focused on the first five years of its operation. Here we extend the examination of sustainability to eight years from the projects' initiation to more closely examine the sources and developmental trends in their generation of financial resources over time. Given recent theorizing regarding the importance of the availability of financial resources and infrastructure support for the sustainability and scale-up of EBPIs (Spoth et al., 2013), we were also interested in exploring differences in financial sustainability between projects in Pennsylvania and Iowa that might be related to state level infrastructure differences. In addition, we aimed to understand other factors contributing to resource generation; guided by our own and others' past research, we examine the degree to which the earlier quality of team functioning and the level of program delivery predicted resource generation. Specifically, we hypothesized that: 1) teams would generate more funding over time, as their experience and expertise with sustainability increased; 2) state-level differences in financial sustainability between Iowa and Pennsylvania would emerge as a result of infrastructure differences between the two states; 3) increases in funding would be linked to greater diversity of funding sources; 4) higher ratings of team leadership and functioning would be related to greater revenue generation, both concurrently and predictively; and 5) early financial sustainability would expand the community's capacity to offer EBPIs and would thus predict greater family participation in intervention programming at later time points.

Method

Sample

The PROSPER study began in 2002 and included 28 communities located in small towns and rural areas in Iowa and Pennsylvania. There were 14 communities in each state, and we randomized 7 to receive PROSPER interventions, while the other seven served as a comparison group. Communities were selected using the following criteria: 1) total school district enrollment was between 1,301 and 5,200 students in non-metropolitan areas; 2) at least 15% of families within the district were eligible for free or reduced cost lunches; 3) fewer than 50% of the community population was employed by or attending a college or university; and 4) the community was not involved with other university-affiliated, youth-focused prevention research projects. After the communities were selected, they were blocked by size and location and randomized into intervention and control groups. Median household income was $37,070 in 2001; on average 29% of students qualified for free/reduced school lunches. At baseline, 85% of the students were Caucasian, 5% Hispanic/Latino, 3% African American; and 7% identified as “other” or did not indicate their ethnicity (Redmond et al., 2009). Our study included data from the 14 communities (seven in Iowa, seven in Pennsylvania) randomized to the intervention condition.

Over the eight year time frame included in this study, two communities in Iowa discontinued their participation in PROSPER. In both cases, this was due to Team Leader turnover; the Team Leader left that community and no suitable replacement could be found, so the team's activities terminated.

PROSPER Interventions

In the PROSPER intervention communities, local teams led by CES educators were charged with the selection, implementation, evaluation, and sustainability of universal EBPIs designed to reduce middle school students' substance misuse and other problem behaviors. Communities selected two programs from a menu: the first was a family-focused program for 6^th grade students, while the second was a school-based curriculum for 7^th graders. Ultimately, all 14 communities selected the Strengthening Families Program: 10-14 (SFP 10-14; Molgaard, Kumpfer, & Fleming, 1997) as the family-focused intervention. This seven session, extracurricular program included both parents and youth, and involved instruction and materials, three paid facilitators and child care workers, a meal each week for 20-30 people, and small incentives for youth and parent participation. Thus the costs are considerable, averaging $2,200-$2,700 for each group of 10-12 families. For the school-based program, teams chose either the Life Skills Training program (Botvin, 2000), the All Stars Program (Hansen, 1992) or Project Alert (Ellickson & Bell, 1990). Once teachers were trained in these programs, the ongoing costs were minimal compared to SFP: 10-14, ranging from $100-$400 per year. Because the school-based programs were offered as part of the regular 7^th grade curriculum and did not require recruitment of youth or families, it was typical for these programs to reach all the students in the school.

Programming support

For the first two years of the project (2002-03 and 2003-04), the PROSPER grant covered all costs associated with program implementation (funded by National Institute of Drug Abuse). After two cycles of implementation (year 3—2004-05), teams were charged with raising funds locally, with the expectation that this would gradually increase in subsequent years as grant funding was progressively reduced. Teams received small financial incentives from the PROSPER grant for increasing their family participation by 5% or more. By year 5 (2006-07), the grant provided no financial support for program implementation, with the exception of two communities that had experienced an unexpected change in team leadership. In these communities (one in Pennsylvania and one in Iowa), an additional year of grant support was provided to facilitate a smooth transition for the new team leader. For the current study, we utilized data from four time points. Year 3 (2004-05) was the “baseline” sustainability year, when teams first generated funds for programming. Years 5 (2006-07) and 6 (2007-08) are “intermediate” sustainability years, because teams fully supported their own programming, but the PROSPER grant still provided “infrastructure support” for team leader and PC salaries. Year 8 (2009-10) represents “full” sustainability because by this time, the CES in both states supported the PC and Team leader positions.

Infrastructure support for the PROSPER teams also shifted after the first five years. CES educators serving as PROSPER team leaders received 25% of their salary from the PROSPER grant for five years. A phase-out of this support was planned with the CES administration at both universities, with support reducing over time to 20% in year 6, and 10% in years 7 and 8. As grant support declined, the CES in both states increased their ownership of PROSPER team leader time, finally institutionalizing all PROSPER activities into the educator's regular work plan.

Technical assistance in PROSPER involved regularly scheduled, ongoing contact between Team Leaders and Prevention Coordinators (PCs). PCs had expertise in prevention science and the CES, coached Team Leaders through the different phases of the program, and helped them to both anticipate barriers and to address them when they arose. Typically, PC's consulted with team leaders by phone on a bi-weekly basis and attended team meetings on a monthly or bimonthly basis. Because revenue generation and marketing were beyond the expertise of most PCs, technical assistance involving sustainability was supplemented with periodic presentations, workshops, and consultation with experts outside the PROSPER community.

Measures

We collected data from multiple measures of intervention process, implementation, and sustainability. For the current study, we utilized the following measures:

SFP: 10-14 Participation

Twice per year each community reported on the number of families enrolled in SFP: 10-14, the number who attended at least one session, and the number who graduated (completed at least four of the seven sessions). We adopted the 4/7 criterion based on prior research with SFP:10-14 indicating that four sessions were sufficient to detect intervention effects (Spoth, Redmond, & Shin, 2000, 2001). We aggregated the biannual data to create a single yearly index of SFP: 10-14 family participation.

School-based Programs

Due to the nature of the school based programs, there tended to be much less variability in their implementation. We collected yearly data from each participating school regarding the percentage of 7^th grade students receiving the school based program. This was typically either all or none, with the overwhelming majority of schools implementing the program with all students across the study period. Because the school based program was implemented one year later than SFP:10-14, the first year of sustainability data available for the school based program was 2006-07.

SFP: 10-14 Program Implementation Quality

A trained program observer observed 25% of all SFP:10-14 sessions and completed a session-specific implementation rating. This form indicated whether all session activities were completed, whether they were implemented in the correct order, and whether implementers made any modifications to the curriculum. Additionally, at each observation, raters completed four process items assessing participant engagement. Because there were no significant state-level differences in implementation quality, we summed these scores to compute an implementation quality rating, and then aggregated this across the two states.

Global Team Functioning and Team Leadership

Team functioning was assessed using annual ratings provided by Team Members. We combined two subscales to create the Global Team Functioning scale: Team Culture (8 items, α = .87 – .92; Kegler et al., 1998) assessed perceptions of the team's cohesion and unity; Focus on Work (5 items, α = .68 – .78; adapted from Moos, 1981) measured the degree of work-orientation at team meetings; Team Goals (2 items, r = .41 – 83; Greenberg et al., 2007) rated the degree to which teams had established goals and procedures; and Team Leadership (8 items, α = .78 – .87; Kegler et al., 1998) rated the effectiveness of team leaders. The average alpha for the global team functioning construct across the waves in the study was .91. We examined the relationships among the global team functioning scale, the team leadership subscale, and financial sustainability.

Sustainability Funding

PC's and team leaders jointly completed a twice-yearly log which included the following: actual dollars received, in-kind support received (this included donations of food, incentives, facilitator time), and the sources of each resource. These logs were combined to create an annual portfolio of sustainability funding. We coded support received into the following categories:

Teams usually procured federal funds through grant writing. For example, several teams received grants from the U.S. Department of Agriculture's Children, Youth and Families at Risk (CYFAR) initiative, or the Drug Free Communities initiative sponsored by the Substance Abuse and Mental Health Services Administration (i.e., SAMHSA).

State funds also were often obtained through competitive grants. For example, in Pennsylvania several teams procured grants from the Pennsylvania Commission on Crime and Delinquency (PCCD), which provided funding for EBPI implementation throughout the state, while in Iowa, teams accessed The Iowa Department of Public Health and State Incentive Grant Funds (i.e., SIG).

School Districts contributed funds to PROSPER teams, for example, to purchase program materials or pay program facilitators.

Foundations made grants to some communities. Most of these foundations were local and region-specific; they often were linked to health care conversion funds. A number of communities received funds from foundations that had health and wellness promotion or youth and family resilience as their focus.

Non-Governmental local agency

Some teams received funding from non-governmental social service providers. Often these were county-level agencies that provided social services or mental health or drug and alcohol services. We included local chapters of the United Way in this category. A “Community Action Network” and “Interfaith Forum,” churches and the Kiwanis are other examples.

Other local funding included resources from local sources, including governmental entities such as legislators, police departments, or mayoral offices; local businesses, including stores, restaurants, and movie theaters; individual donors or sponsors; and local fundraisers such as car washes and yard sales.

Cooperative Extension

Local and regional CES offices sometimes contributed funds to support the local PROSPER activities.

In-kind Funding

In-kind support almost always involved local or county-level resources. In-kind included donations of people's time to serve as SFP:10-14 facilitators and child care providers. Typically this involved a partnering agency that allowed its employees to implement PROSPER programs as part of their job, or a school district which allowed students to fulfill community service requirements by volunteering as PROSPER child care workers. Additionally, in-kind also involved donations of food for the SFP:10-14 meals and donations of incentives (gift cards, games, movie tickets, etc.) for participants.

Results

Coalition Sustainability

After eight years, 12 of the original 14 PROSPER prevention coalitions were still operating. Two sites in Iowa discontinued after year 6 as a result of turnover in team leadership. In both Pennsylvania and Iowa, the CES sustained the teams by institutionalizing both the Team Leader and the Prevention Coordinator positions. CES incorporated these roles into the plan of work for CES personnel, reflecting a considerable CES investment in PROSPER. CES leadership in both states indicated that the data on PROSPER's public health impacts, as well as local program sustainability efforts, were instrumental in mobilizing this infrastructure support.

Program Sustainability: Funding Trends Over Time

Figure 2 summarizes data related to our first hypothesis, that teams would generate increased revenue over time. Additionally, Figure 1 shows the average total monetary resources generated per team in Iowa and Pennsylvania across the four time periods. Both states' teams began with relatively small amounts of external funding, although Iowa teams on average generated 73% of Pennsylvania average revenue ($4,596 vs. $6,311). After that initial year, both states' teams substantially increased their average revenues in years 5 (2006-07) and 6 (2007-08). Iowa revenues per team nearly doubled from year 3 to year 5, while Pennsylvania revenues increased from $6,311 to $15,172 per team. Revenues continued to climb for both states in year 6, although Iowa revenues increased by about $714 per team, while Pennsylvania revenues increased an average of $5,667 per team. At Year 8 (2009-2010), Pennsylvania's revenues declined by 19% compared with Year 6, whereas Iowa's increased by 20%; however their mean dollars per team was still 28% lower than Pennsylvania's. By this time, two teams had discontinued operations in Iowa, leaving 5 active teams; Pennsylvania continued to have 7 teams in year 8. Thus, Pennsylvania experienced large increases in sustainability funding across the first three time periods, followed by a decline at the last wave. The Year 8 levels, however, remained substantially above those at Year 3. On the other hand, Iowa's funding amount per site was somewhat lower but showed less fluctuation over time.

Figure 2.

(a) Average team funding generated in Iowa and Pennsylvania across four select years

(b) Average in kind team resources generated in Iowa and Pennsylvania across four select years

Note. n=6 for Iowa Year 6; n=5 for Iowa Year 8; n=7 in all other cases.

Funding per student

Because the communities varied widely in size, we next examined the amount of funds teams raised per student across multiple years. For this analysis, the denominator included the total of 6^th grade students per community per year. As Figure 3 shows, in the first sustainability year (year 3), Iowa teams generated approximately $34 per student in sustainability dollars, while Pennsylvania teams raised about $27 per student. Iowa per student revenues remained relatively stable over time, with Iowa teams raising $46, $51 and $54 per student for years 5, 6, and 8, respectively. By contrast, Pennsylvania's revenues rose to $72 per student in year 5, $111 per student in year 6, and fell back to $75 per student in year 8.

Figure 3.

(a) Average funding per student generated in Iowa and Pennsylvania across four select years

(b) Average in-kind team resources per student generated in Iowa and Pennsylvania across four select years

Note. n=6 for Iowa Year 6; n=5 for Iowa Year 8; n=7 in all other cases.

Cash Funding Sources Across & Within States

Our second hypothesis was that increased revenue generation would be related to greater diversity in funding sources. Surprisingly, the results ran somewhat counter to this hypothesis. At Year 3, there was considerable diversity of funding sources, with both states' teams' reporting a total of six different sources of funding (see Figures 4 and 5). Federal, state, foundation, and school district funding were the primary types of revenue.

Figure 4. Hard Dollar funding sources for Pennsylvania across four select years.

Figure 5. Hard dollar funding sources for Iowa across four select years.

In Year 5, Pennsylvania received significant sums from state sources, school districts, foundations, and local agencies (see Figure 4). Additionally, teams raised over $8,000 from local fundraisers. Iowa teams showed modest but steady increases in state-level funding over time, along with small declines in local funding. Foundations were a consistently significant source of funding in Iowa across all time periods (see Figure 5).

By Year 6 the pattern had changed, with Pennsylvania reporting substantially more money than at previous waves, but only five different sources of funding (see Figure 4). Forty-three percent of all funds in Pennsylvania came from state level funding, and an additional 38% came from local sources including school districts, suggesting a concentration within a few specific sources. Iowa again reported a variety of different sources (see Figure 5).

In Year 8, both states showed a reduction in diversity of funding sources compared with Year 3. Pennsylvania reported six sources and Iowa reported five (see Figures 4 & 5). Nearly half (45%) of Pennsylvania's money came from the state, and an additional 30% from local agencies and school districts. In Iowa, 39% of revenue raised came from state sources, and about 21% from local agencies and schools.

Team-level differences within states

Within both Pennsylvania and Iowa, the amount of funds generated varied considerably across teams. As shown in Table 1, average annual funding per team (across the four years) ranged from $3,485 to $34,357 in Pennsylvania, and from $2,421 to $19,612 in Iowa. In terms of each team's percentage contribution to their respective statewide total, individual teams' shares of the total annual funding obtained ranged from 3% to 33% for teams in Pennsylvania, and from 4% to 36% for teams in Iowa. This may be an important predictor for low-functioning teams in particular; the two teams in Iowa that discontinued prior to Year 8 had the lowest levels of funding in their state in prior years.

Table 1. Average annual funding and average annual percent of state total per team, organized from the highest team amount to the lowest team amount, for hard dollars and in-kind resources.

	Hard Dollars		In-Kind
	Pennsylvania	Iowa	Pennsylvania	Iowa
Team Average	$14,806	$7,872	$6,580	$7,958
State Total	$103,641	$55,102	$103,641	$55,709
Yearly team average (% of state total)	$34,357 (33%)	$19,612 (36%)	$13,991 (30%)	$21,354 (38%)
	$26,080 (25%)	$8,906 (16%)	$11,147 (24%)	$8,838 (16%)
	$15,340 (15%)	$8,224 (15%)	$5,267 (11%)	$7,103 (13%)
	$11,113 (11%)	$6,257 (11%)	$5,109 (9%)	$5,473 (10%)
	$8,061 (8%)	$5,847 (11%)	$4,202 (8%)	$5,404 (10%)
	$5,145 (5%)	$3,833 (7%)	$3,955 (9%)	$4,373 (8%)
	$3,485 (3%)	$2,421 (4%)	$2,392 (5%)	$3,163 (6%)

Note. Each team average was created with financial and in-kind resources only for the years in which they were operational.

Relations With Team Leadership and Global Team Functioning

Based on earlier findings on the relations between team functioning and sustainability planning (Greenberg et al., 2015; Perkins et al., 2010), we hypothesized that the quality of global team functioning and leadership would be positively related to success with sustainability funding through Year 8. Results were mixed and indicated very different patterns across the two states. In Iowa, team leadership and global team functioning were strongly and positively correlated with concurrent funds raised per youth, whereas in Pennsylvania this relation was largely negative (see Tables 2 and 3). To understand this difference between states, we examined scatterplots. We discovered that of the three top revenue-generating teams in Pennsylvania, only one had high ratings of team functioning and leadership; moreover, one highly-rated PA team leader had the lowest level of resource generation. Examining project notes revealed that the two high-revenue teams were able to form stable partnerships with large social service agencies in their communities, thus accessing many resources, despite receiving relatively low ratings for team functioning and leadership.

Table 2. Spearman correlations between ratio dollars and ratio in-kind resources (i.e., estimated amount of money or in-kind resources per middle school student in the district), with team member-rated team leadership across the studied years.

Team Leadership	2004-05, Year 3 Ratio $ (In-Kind)	2006-07, Year 5 Ratio $ (In-Kind)	2007-08, Year 6 Ratio $ (In-Kind)	2009-10, Year 8 Ratio $ (In-Kind)
Pennsylvania
2004-05 Year 3	.20 (-.71)	-.50 (-.28)	.04 (-.07)	-.57 (-.50)
2006-07 Year 5	-.49 (-.36)	-.47 (.04)	-.13 (.18)	-.41 (-.38)
2007-08 Year 6	.09 (-.82)	-.68 (.46)	-.04 (.79)	-.43 (-.39)
2009-10 Year 8	-.46 (-.45)	-.59 (.40)	-.49 (.68)	-.31 (-.25)
Iowa
2004-05 Year 3	.79* (.50)	.86** (-.21)	.54 (.14)	.60 (-.10)
2006-07 Year 5	.71 (.07)	.50 -(.29)	.09 (.49)	.90* (.10)
2007-08 Year 6	.79 (-.02)	.74 (-.22)	.29 (.29)	.97 (-.21)
2009-10 Year 8	.60 (.00)	.90* (.10)	.60 (-.40)	.80 (-.30)

Note. n = 6 for Pennsylvania Ratio $ AY 2004-05; n= 7 for Pennsylvania Ratio $ AY 2006-07 through AY 2009-10; n = 7 for Iowa Team Leadership AY 2004-05 through AY 2007-08; n= 5 for Iowa Team Leadership AY 2009-10.

^*p < .05.

^**p<.01.

^***p<.001.

Table 3. Spearman correlations between ratio dollars and ratio in-kind resources (i.e., estimated amount of money or in-kind resources per middle school student in the district), with team member-rated global team functioning.

Global Team Functioning	2004-05, Year 3 Ratio $ (In-Kind)	2006-07, Year 5 Ratio $ (In-Kind)	2007-08, Year 6 Ratio $ (In-Kind)	2009-10, Year 8 Ratio $ (In-Kind)
Pennsylvania
2004-05 (Year 3)	.43 (-.75)	-.35 (.29)	.29 (.43)	-.21 (-.14)
2006-07 (Year 5)	-.26 (-.32)	-.43 (.36)	-.04 (.54)	-.29 (-.32)
2007-08 (Year 6)	-.20 (-.57)	-.68 (.46)	-.14 (.71)	-.43 (-.39)
2009-10 (Year 8)	-.49 (-.71)	-.57 (-.11)	-.36 (.21)	-.50 (-.43)
Iowa
2004-05 (Year 3)	.75 (.18)	.93** (-.32)	.37 (.09)	.80 (-.30)
2006-07 (Year 5)	.86** (.14)	.75 (-.28)	.43 (.37)	.90* (.10)
2007-08 (Year 6)	.75 (-.04)	.79* (-.25)	.26 (.14)	.90* (-.40)
2009-10 (Year 8)	.70 (.10)	.80 (.30)	.70 (.00)	.90 (.10)

Note. n= 6 for Pennsylvania Ratio $ AY 2004-05; n = 7 for Pennsylvania Ratio $ AY 2006-07 through AY 2009-10.; n = 7 for Iowa Team Leadership AY 2004-05 through AY 2007-08; n= 5 for Iowa Team Leadership AY 2009-10.

^*p < .05.

^**p<.01.

^***p<.001.

Relations With SFP: 10-14 Family Participation

To assess the relationships between financial sustainability and actual EBPI impact, we examined the relations between dollars raised per youth and rates of family participation in the SFP: 10-14 program. We hypothesized that teams that raised more money would have higher levels of youth and family participation in SFP: 10-14, and this was largely supported (see Table 4). A disaggregation between states revealed little difference in the magnitude of these associations, so the results were combined in order to boost statistical power. SFP: 10-14 graduation had a strong, significant, and positive association with revenue generation. At concurrent waves, correlations ranged from r = .48 to r = .71. Dollars raised in year 5 were highly correlated with SFP participation in year 6 (r=.61) and year 8 (r=.67), while the correlation of dollars in year 6 to SFP participation in year 8 was .38. Similarly, SFP graduation numbers in years 5 and 6 predicted dollars raised in future years; SFP participation in year 5 was correlated .51 with dollars raised in year 6 and .65 with dollars raised in year 8, while participation in year 6 was correlated .67 with revenue generated in year 8 (see Table 4).

Table 4. Spearman correlations between ratio dollars and ratio in-kind resources (i.e., estimated amount of money or in-kind resources per middle school student in the district), with number of SFP 10-14 families graduated (i.e., 4 or more sessions).

SFP 10-14 families graduated	2006-07, Year 5 Ratio $ (In-Kind)	2007-08, Year 6 Ratio $ (In-Kind)	2009-10, Year 8 Ratio $ (In-Kind)
2006-07 (Year 5)	0.54* (-.06)	0.51 (.26)	0.65** (.21)
2007-08 (Year 6)	0.61* (-.04)	0.48 (.45)	0.67** (.61*)
2009-10 (Year 8)	0.67* (-.15)	0.38 (-.18)	0.71** (.15)

Note. n = 7 for Pennsylvania Ratio $ AY 2006-07 through AY 2009-10.; n = 7 for Iowa Ratio $ AY 2006-2007 and 2007-08; n = 5 for Iowa Ratio $ AY 2009-10

^*p < .05.

^**p<.01.

^***p<.001.

Sustainability of the School-based programs was much less variable than the SFP:10-14 program, and partial implementation of the school based program was very unusual; schools tended to offer these programs either to all students or else to not offer them at all. Across the study period, 12 of the 14 districts offered the school based program to all 7^th grade students in 2006-07; in 2007-08, 11 communities offered it to all students, two to some students, and one did not offer it, and in 2009-10 all of the 12 remaining communities indicated that they offered the program to 90% or more of their 7^th grade students.

In-Kind Resources

The pattern for in-kind resources was somewhat different than for cash dollars. At all time points measured, Iowa raised more in-kind resources than Pennsylvania, although the differences were sometimes small (see Figure 2b). Additionally, in-kind resources consistently represented a larger proportion of the total per team revenues in Iowa than in Pennsylvania, with percentages of the total ranging from 47%-55% for Iowa teams and 24%-36% for Pennsylvania teams. As with cash dollars, the amount of in-kind resources generated by teams increased steadily over time, the only exception being a slight decrease in Pennsylvania in 2007-08.

The relationships among in-kind resources and team leadership and functioning variables were not consistent in either Pennsylvania or Iowa (see Tables 2 & 3). Depending upon the year, correlations could be large or small, and positive or negative, for both states. Similarly, and unlike cash dollars, there was no consistent pattern of relationships between in-kind revenue and SFP:10-14 family participation, either concurrently or predictively (see Table 4).

Possibly due to these lack of consistent relationships among in-kind and other variables, in-kind was also less “prognostic” of failing teams than cash resources. Of the two teams in Iowa that discontinued, one had low levels of in-kind resource generation during the previous year, while the other had average levels.

The associations between in-kind and cash dollars raised within each state were also somewhat inconsistent. In Pennsylvania concurrent correlations ranged from .14 (2006-07) to .96 (2009-10), while in Iowa they ranged from -.14 (2006-07) to .26 (2007-08).

Discussion

This paper examined the ways in which PROSPER communities achieved the long-term sustainability of both their teams and their programs. Given the generally poor track record for sustainability of prevention efforts documented in the literature, PROSPER coalitions appeared to fare relatively well. After 8 years, 12 of the original 14 teams were independently sustained and continuing to provide prevention programming to youth and families. Furthermore, sustainability occurred at multiple levels: families and schools received programming, teams met regularly to fill their roles, and the state infrastructure of CES sustained the roles of the team leaders and PC's.

We hypothesized that as time went on, PROSPER teams would increasingly generate more money and diversify their funding sources, but these hypotheses were only partially supported. Both states did increase their revenue generation substantially between year 3 and year 6, but Iowa's fundraising continued to increase in year 8, whereas Pennsylvania's declined from year 6 to year 8. The reasons for this decline in Pennsylvania are unclear. It is possible that teams decreased their fundraising efforts because they had “banked” sufficient resources in year 6 to cover their programming costs in subsequent years; alternatively, the economic recession occurring during the study period may have impacted fundraising efforts (Kuklinski, Hawkins, Plotnick, Abbott, & Reid, 2013). We did not find support for our hypothesis regarding increasing diversity of funding sources over time. If anything, teams relied on fewer sources of funding as time went on, but specific sources emerged as increasingly important. In Pennsylvania, the proportion of team funds that came from state-level sources (i.e., PCCD grants) increased dramatically over time, and was largely responsible for the disparity in funding observed between Pennsylvania and Iowa. In Iowa, where state-level infrastructure supports were less available, teams were more reliant on school districts, foundations, and local sources to support their programming.

Instead of diversifying funding sources, the PROSPER teams that were most successful at generating resources streamlined their fundraising efforts into long-term partnerships with school districts, social service agencies, or other partners that provided a substantial percentage of the resources needed, and focused less on other sources as they matured. For example, in Pennsylvania, the PROSPER team in Community 2 received a large annual contribution from the school district to support EBPs, while Community 1 maintained an ongoing relationship with an umbrella service agency that routinely supported SFP: 10-14 implementation. This may have been an efficient strategy, given that many of the earlier sources of sustainability funding yielded relatively small amounts of money and may not have been worth the time and efforts of the team leader or other team members. Additionally, school districts and county-level services such as mental health, children and youth services, and drug and alcohol providers may represent relatively stable, naturally-occurring partners for community coalitions such as PROSPER teams. These organizations often have family outreach and primary prevention as priorities, but may be unable to provide these services themselves; if so, forming collaborations with PROSPER to provide evidence-based prevention was mutually beneficial. However, despite the efficiency and apparent success of relying on one primary community partner for sustainability support, it is possible that PROSPER teams (and perhaps other community prevention coalitions) following this path may be at greater risk of losing their funding than those who choose a strategy that involves diversification of funding (Rog et al., 2004).

At all time points, Pennsylvania teams generated greater cash funds than those in Iowa, both overall and per student, while Iowa raised more in-kind funds. In fact, in-kind resources appeared to play a bigger role in Iowa than Pennsylvania generally, with in-kind consistently representing about half of Iowa's revenue generation but only one-quarter to one-third of Pennsylvania's. There are several possible factors that contributed to these differences. First, unlike Iowa, Pennsylvania has a statewide system for diffusion and support of EBPI's available through the Pennsylvania Commission on Crime and Delinquency (PCCD). This agency provides grants and implementation support designed to promote the successful community-based dissemination of EBPI's, and SFP: 10-14 and several of the school-based programs used in PROSPER are on the list of programs eligible for funding. Consequently, Pennsylvania communities successfully accessed these resources. By year 6, nearly half of Pennsylvania's sustainability dollars came from PCCD. Conversely, Iowa has no comparable system to support prevention, which curtailed the opportunities available to PROSPER teams in that state. Thus Iowa teams may have relied more on local resources, which often came in the form of in-kind support. Our findings on state level differences in sustainability resource generation support the point made by Spoth et al. (2013b), that financial structures and policies have large implications for effective EBPI dissemination and sustainability. This structural difference may also have contributed to the different relationships to team functioning found in the Iowa-Pennsylvania comparisons. In Iowa, the role of team leadership in securing funding appears to have been far more important than in Pennsylvania, where strong infrastructure support may have made this less imperative. This suggests that sustainability planning should include attention to either accessing existing state level resources or facilitating their development.

The negative correlation between team functioning and cash generation in Pennsylvania was a curious and counterintuitive finding. Evaluation of specific cases indicated that two teams in Pennsylvania with the lowest team functioning and leadership ratings had generated high levels of resources, primarily through their partnerships with prominent social service agencies. Conversely, a third team with high functioning and leadership ratings raised very little money; in this case, a popular team leader was not able to forge stable partnerships with agencies or schools to support prevention efforts. Had our sample been larger, it is possible that the results would not have been so strongly impacted by these particular cases. However, it is also possible that our measures of team functioning and leadership did not adequately capture the dynamics operating between PROSPER teams and the unique opportunities and resources available in each community.

Although there were noticeable state-level differences, the greatest variability in sustainability at every time period was at the community level. Averaging across all waves of data, we found that individual teams raised anywhere from 3- 42% of the total revenues generated within their respective states. This was important because cash revenue generation correlated significantly and positively with SFP: 10-14 participation at all waves of data collection; teams that raised more money had higher rates of EBPI penetration. Potentially, perceived future financial stability allows teams to catalyze member efforts, resulting in reaching more families with programming. More success with programming seems to reinforce success in generating needed resources. Additionally, success generating sustained funding may also forecast the coalition's long-term viability; the two teams in Iowa with the lowest levels of fundraising were also the ones in that discontinued their prevention activities by Year 8. Although the departure of the Team Leader from the Extension system was the precipitating factor for the termination of PROSPER in both of these communities, their low levels of sustainability success prior to this time may have made them particularly vulnerable to this transition. These findings are consistent with the theoretical model adopted by PROSPER, which posits that effective and well-organized teams with the capacity to deliver high-quality EBIs with fidelity, form strategic community partnerships, and access financial resources are most likely to sustain both themselves and their programming (Johnson et al., 2004; Spoth & Greenberg, 2011).

Interestingly, the findings for in-kind resources were somewhat different from those for cash revenues. Iowa teams generated more in-kind support than Pennsylvania teams at all time points, and in-kind represented a larger proportion of total community revenues in Iowa than in Pennsylvania. However, unlike cash revenues, in-kind was not consistently related to team leadership, team functioning, or SFP:10-14 family participation. Because virtually all in-kind resources were obtained at the local level, it is likely that they were less tied to infrastructure supports than cash dollars, which might explain this lack of relation to other sustainability or program participation variables. Although the ability to generate local in-kind resources may be very important to the sustainability of community prevention initiatives, our results suggest that cash dollars are more strongly related to EBPI participation and team functioning variables, possibly because they represent important links with infrastructures supporting prevention.

These results have several implications for effective dissemination and sustainability of EBPIs. First, they suggest what other researchers have asserted, that infrastructure support and ongoing technical assistance are important for helping communities make the transition from seed funding to financial independence (Feinberg, Bontempo & Greenberg, 2008; Gomez, Greenberg, & Feinberg, 2005; Spoth et al, 2013b). In the PROSPER study, team success in sustaining programs and operations was the rule rather than the exception. Support from the CES infrastructure provided local teams and team leaders with continued access to expertise in marketing, communications, grant writing, program evaluation and other skills related to program dissemination and sustainability. These teams also received targeted assistance when confronted with a challenge that otherwise may have threatened the viability of their teams or programming, including changes in leadership or local funding priorities. Second, our findings suggest that sustainability may be achieved very differently in different communities. Sustainability is in large part a local process and understanding the key stakeholders and funders in each community, and nurturing local relationships and building trust happened in different ways with different funders in each community. This suggests that sustainability may be largely driven by local contextual factors, and that technical assistance focused on sustainability of community teams needs to include a careful consideration of local resources as well as to nurture skills that make teams competitive for state and national level funding. On the other hand, state-level differences illustrate that state-level funding opportunities (and conceivably national level as well) for prevention can go a long way to facilitate the ongoing success of local teams, and, conversely, that a lack of state-level support can inhibit the viability and reach of prevention initiatives.

Fourth, our results indicate that local partnerships with schools and social service agencies yielded large amounts of funding for several PROSPER teams, particularly in Pennsylvania. This finding suggests that these partnerships may be crucial for the long-term sustainability of prevention programming and coalitions, and that teams that invest time and energy in the cultivation of these relationships may reap significant benefits. Future work on sustainability should explore the ways in which these partnerships are established and maintained, the ways in which they meet the needs of all major stakeholders, and whether teams that enter into these partnerships are more vulnerable to loss of funding than those that actively diversify their funding sources.

Finally, our results indicate that cash revenue generation is significantly linked to EBPI utilization within a community. Communities generating more money had more youth and families participate in SFP: 10-14, and these were predictive as well as concurrent relations. Fundraising at an earlier wave predicted SFP: 10-14 participation at later waves, and vice versa, suggesting a synergistic effect between resource generation and EBPI participation that may also reflect local enthusiasm and support for EBPIs. While it makes sense that having more financial resources allows communities to offer programs to more families, it may also be true that greater family participation generates more local awareness and enthusiasm for a program, which in turn yields further funding opportunities.

A number of limitations are inherent in this work. First, this study focuses on revenue generation and coalition sustainability, and does not consider other important aspects of the sustainability construct. For example, although we demonstrate that financial sustainability was linked to greater program delivery, the research trial was only able to follow the first two cohorts of youth (still being followed as of this writing, and in their early 20s). Therefore, we cannot ascertain whether cohorts during the sustainability phase (3-5 years of programming after the research cohorts) also showed improved outcomes. We know of no published studies that have examined the repeated effects of prevention programming on later community cohorts.

Second, with only 14 communities in the intervention sample, our analyses were by necessity descriptive and qualitative, and generalizations beyond PROSPER should be made cautiously. Relatedly, communities in the PROSPER study were generally small, ethnically homogeneous towns in largely rural areas characterized by relatively low levels of community risks such as poverty and crime. It is unclear whether the experiences of PROSPER teams and communities would be consistent with those of larger, more urban, more diverse, or more economically-disadvantaged communities.

This study did raise some interesting questions regarding factors related to sustainability of EBPIs that warrant further research. First, prior research with both CTC and PROSPER communities found consistent relationships between team leadership and other team functioning variables and later success with sustainability (Feinberg, Bontempo & Greenberg, 2008; Gomez, Greenberg & Feinberg, 2005; Perkins et al., 2011). In this study, we found this to be true only in Iowa, with factors other than team leadership appearing to drive sustainability in Pennsylvania. The role of coalition leadership has been investigated in other sustainability studies and suggest that leadership characteristics are related to sustainability (Butterfoss, Goodman, & Wandersman, 1993; Koo, Dunteman, George, Green, & Vincent, 1994; Kumpfer, Turner, Hopkins, & Librett, 1993; Rowe, 1997), but these inconsistencies warrant further investigation and perhaps better measurement. It is possible that the impact of team leadership on sustainability is moderated by other factors such as partnership opportunities, but we were unable to explore this possibility due to the small sample size. Also, our measurement strategy failed to capture factors related to teams' abilities to form strategic partnerships, which will be important to rectify in future studies.

Broad dissemination and sustained use of well-implemented EBPI's represent one of the current frontiers for the field of prevention science. Overall, the results of the PROSPER study suggest that sustainability of EBPIs implemented with high quality can be achieved when community teams actively plan for it, when community and state-level resources are available to support it, and when teams receive ongoing, targeted technical assistance to support their resource generation efforts.