Understanding community- and system-capacity change over time: A close look at changing social capital in Evidence2Success communities

Abstract

Evidence in majority White and low-population areas suggest that community prevention systems can create social capital that is needed to support high-quality implementation and sustainability of evidence-based programs. This study expands prior work by asking the question: How does community social capital change during the implementation of a community prevention system in low-income, highly populated communities of color? Data were collected from Community Board members and Key Leaders in five communities. Linear mixed effect models analyzed data on reports of social capital over time, first as reported by Community Board members then by Key Leaders. Community Board members reported social capital improved significantly over time during the implementation of the Evidence2Success framework. Key Leader reports did not change significantly over time. These findings suggest that community prevention systems implemented in historically marginalized communities may help communities build social capital that is likely to support the dissemination and sustainability of evidence-based programs.

1 |. INTRODUCTION

It is recognized more and more that changing how people and organizations work together to select, plan for, implement, and evaluate youth and family programs may help sustain the implementation of evidence-based programs that can lead to improved youth outcomes communitywide (Louison & Fleming, 2016–2017). Indeed, both the Communities That Care (CTC) and PROmoting School-community-university Partnerships to Enhance Resilience (PROSPER) prevention systems have been shown to improve elements of a community’s social capital and infrastructure related to sustaining evidence-based programs (Brown et al., 2007, 2011; Chilenski et al., 2014; Crowley et al., 2012). Both prevention systems have also been shown to improve outcomes for youth communitywide (Oesterle et al., 2018; Spoth et al., 2017).

This research evidence, however, has been collected in primarily White, rural, semirural, or suburban areas, creating a gap in our understanding of how community prevention systems function in predominately low-income Black and Hispanic communities in highly populated areas. This study begins to explore this gap with reporting on the implementation of Evidence2Success^®—a framework that promotes healthy child development by helping communities and public systems work together to use data to understand how young people are doing, select evidence-based programs to address challenges and improve outcomes, and develop financing and action plans to support the sustainability of selected programs. Does social capital and related infrastructure improve in communities who adopt and implement the Evidence2Success framework? If so, is this improvement sustained?

2 |. SOCIAL CAPITAL AND COMMUNITY PREVENTION SYSTEMS

2.1 |. What is social capital?

Social capital describes the “social fabric” of a community, including the interconnections among individuals and organizations, their characteristics, and how they facilitate action (Coleman, 1988; Putnam, 1993). Some characteristics that have been emphasized include norms, social trust, and the degree of coordination and cooperation. Social capital can also be described as the resources that become available to people, groups, and organizations through their interconnections. Given this broad framework, social capital has at times been described more concretely as “collective intelligence” (Lappé & Du Bois, 1997) and at other times been divided into three categories: bonding, bridging, and linking (i.e., Kawamoto & Kim, 2019). Both frameworks, social capital as collective intelligence and social capital categorized as bonding, bridging, and linking, are used in this study.

Collective intelligence, which we refer to as “collective capability” in this study, emerges when individuals come together within a group. Collective capability can be understood as a group’s power or ability to accomplish something. Collective capability occurs when a sense of optimism and hope and relevant skills and knowledge are combined with meaningful participation (Lappé & Du Bois, 1997). In other words, individuals bring their individual skills, knowledge, and experience to a group and collectively, these combined capacities help explain group performance and their capability to achieve goals and desired outcomes (Woolley et al., 2010). Collective capability, thus, through the social exchange and sharing of individual capacities, can support a collaborative’s adaptability, innovation, and sustainability.

On the other hand, the bonding, bridging, and linking framework focuses more on types of interconnections and relationships (Szreter & Woolcock, 2004; Villalonga-Olives & Kawachi, 2015). Bonding social capital describes resources that become available when similar individuals are connected to each other. Bridging social capital describes resources that become available when people and organizations with similar levels of power and influence, but otherwise are different from each other, come together. Bridging social capital can be thought of as the horizontal connections within a community. Linking social capital builds on bridging social capital to describe resources that come together when individuals and organizations with different levels of power and influence come together; linking social capital can be thought of as vertical connections within a community (Szreter & Woolcock, 2004; Villalonga-Olives & Kawachi, 2015).

2.2 |. Connecting social capital with community prevention systems frameworks

Given this understanding of social capital, we can see that community prevention systems, at their core, are social capital interventions. Community prevention systems include bringing together people, groups, and organizations in different ways for a common purpose: to improve the health and development of youth and families. The different connections and relationships that occur when people, groups, and organizations come together in a community prevention system can be categorized as bonding, bridging, and linking social capital. In addition, community prevention systems typically involve pooling existing and building relevant knowledge and skills (Currie et al., 2005; Petersen, 2002). These collaboratives typically develop mission and vision statements that can build a sense of meaningful participation, along with beliefs that the group can be successful in reaching their goals. The bonding, bridging, linking, and collective capability elements of the social capital surface through these connections and processes.

We can further apply these concepts to the community prevention system context. For example, a neighborhood-based group of parents that come together to discuss challenges of and solutions to community problems is an example of bonding social capital, whereas a group of agencies and organizations with different missions, goals, and services coming together in the same community may be an example of bridging social capital. Linking social capital, then, would be enacted if the neighborhood-based group of parents and the neighborhood-based organizational group came together to form one coalition or partnership, and it would be further reinforced if this group joined with leaders of public systems or other elected officials.

CTC, PROSPER, and Drug-Free Communities are community prevention systems that fit this understanding of social capital interventions. All three community prevention frameworks include bringing together parents, youth, and representatives from up to 10 additional sectors to assess community needs. These community groups then make decisions about the substance use/abuse prevention and youth development programming, activities, or resources that are available in their communities. This act of bringing together people and organizations from different sectors, aligns with the definition of bridging social capital. These interventions could further fit the definition of linking social capital if some of the sectors include representatives that have different levels of power and influence. The collective capability framework also maps on to this work: being involved in this activity would be thought of as a meaningful opportunity for participation. A sense of optimism and hope is built through creating mission and vision statements and is reinforced through working together collaboratively.

3 |. CURRENT EVIDENCE ON THE EFFECTIVENESS OF COMMUNITY PREVENTION SYSTEMS

Prior theoretical work connecting comprehensive community initiatives with social capital outcomes (Petersen, 2002) has been supported by more recent research. Evidence shows that community prevention systems can effectively facilitate the community capacities related to selecting, planning, implementing, and evaluating youth and family programs and outcomes for youth. In other words, community prevention systems can improve the infrastructure that supports delivering high-quality youth programming (Currie et al., 2005). We posit that these capacities and this infrastructure are elements of social capital.

3.1 |. Improvements in social capital and the prevention infrastructure

Several studies have found that community prevention systems have improved social capital or other elements of the community prevention infrastructure needed to support high-quality implementation of evidence-based prevention programs. We find evidence in the implementation of PROSPER, CTC, Getting to Outcomes, and other less formalized community coalition models (Anderson-Carpenter et al., 2017; Brown et al., 2007; Chilenski et al., 2014; Chinman et al., 2008).

The potential to improve bridging social capital exists in PROSPER community teams because they engage representatives from different organizations. Evidence was found in a 2014 longitudinal and experimental study (Chilenski et al., 2014). Specifically, improvements in the perceived trust and belief in the school system to be effective and meet their goals were observed over time. Similarly, the belief that the community could enact positive change for families also improved over time. Perceptions of linking social capital also improved over time, as measured by how effectively the school district and the Cooperative Extension System serve and meet the needs of their youth and families. It was theorized that the repeated interactions among these different organizational representatives changed these elements of trust and confidence in their communities (Chilenski et al., 2014).

Different elements of the prevention infrastructure, which are part of a community’s social capital, were measured in experimental studies of CTC (Brown et al., 2007, 2011, 2014). One study demonstrated improvements in organizational collaboration around prevention and adopting a science-based approach to prevention due to participating in CTC (Brown et al., 2007). How organizations collaborate around prevention can be described as a measure of bridging social capital. Organizations with similar levels of power and influence that work in different areas affecting youth and families were reported to collaborate more together around supporting these youth and families. The second measure can be described as a measure of public life skills, or an element of the collective capability of a community. A longer-term study found that the significant improvements in adopting a science-based approach to prevention persisted 4.5 years after adopting CTC (Brown et al., 2011), and yet one more study further indicated that these improvements in collective capability (e.g., adopting a science-based approach) led to lower levels of youth problem behaviors (Brown et al., 2014). In summary, not only did the communities engaged in CTC improve in levels of social capital, but what they learned and built using data to drive decisions is what led to sustained effects of implementation on youth outcomes (Brown et al., 2014).

The Getting To Outcomes work can also be viewed within the framework of increasing community social capital. A longitudinal, quasi-experimental design found through staff interviews that infrastructure supporting evidence-based program implementation improved in Getting To Outcomes sites (Chinman et al., 2008). Specifically, improvements in the public life skills such as planning and organization, clarity regarding program goals, grant writing skills, and preparing reports and presentations were found. These public life skills are part of developing collective capability. Further, using Getting To Outcomes helped to improve evaluation skills, including placing more emphasis on outcomes and data collection, and improving understanding of data trends. Finally, respondents identified improved consistency in their organizations, which they described as improved communication, a greater willingness to change, and an improved ability to come to an agreement (Chinman et al., 2008).

Other observational studies also examined associations with social capital in different prevention or community change initiatives. For example, one multisite statewide study showed how the development of relationships (i.e., bonding and bridging social capital) and knowledge (i.e., public life skills and collective capability) were associated with organizational-level improvements in effectively responding to intimate partner violence (Javdani & Allen, 2011). Another study in Japan found that linking social capital as measured by individual connections with government agencies produced efficiencies for recovering after an earthquake (Kawamoto & Kim, 2019). One study on the development of state infrastructure in nine states during the Strategic Prevention Framework State Incentive Grant program found that systems integration occurred both horizontally and vertically while using that framework, indicating improvements in bridging and linking social capital over time (Orwin et al., 2014). Further, another study found that gains were sustained and increased 1-year after Strategic Prevention Framework grant funding ended (Edwards et al., 2015). More recently, Schranz and colleagues (2018) have theorized that addressing the opioid epidemic in rural areas has been challenged at least partially due to the lack of needed infrastructure in terms of connecting the right people to the right programs. This combination of knowledge, skills, and interconnections can be considered bridging, linking, and collective capability social capital.

Additionally, the Active Involved Community Partnership model is another way communities and public systems may efficiently organize their collaborative work to support improved youth outcomes. Work with this model suggests that active involved community partnerships can build organizational, system, and community capacity, which we posit are elements of social capital (Boothroyd et al., 2017).

4 |. EXISTING GAPS

Less is known about how a community prevention framework improves social capital for racially and ethnically diverse highly populated communities, especially those in which families earn less income relative to community averages. Additional elements may need to be integrated in community prevention systems to achieve these same goals. Historically persistent inequities may continue without attention to community conditions and disparities which may then negatively impact coalition functioning and building social capital, which may then, in turn, lessen the potential positive impact on youth and community health outcomes.

First, elements of social capital, like bridging and bonding, and collective capability have historically been less prevalent in communities of color (Farquhar et al., 2005; Hinton, 2015; Warren et al., 2001). But these findings are equivocal given the varied descriptions and measurements of community social capital—in terms of economic capital, educational capital, civic and political capital, intellectual capital, and social cohesion. What the findings suggest, however, is that community conditions, such as poverty, inequality, neighborhood cohesion, and intergroup relations, can contribute to community social capital.

Congruently, research on social determinants of health indicates that communities plagued by structural and systemic inequalities face greater disadvantages and, in some cases, elevated risk. This work suggests that social conditions may influence how community coalitions and organizations are structured, functioned, and maintained (Feinberg et al., 2007; M. T. Greenberg et al., 2007; Raphael et al., 2014), which may then contribute to a community’s social capital. For example, in their systematic review, Hunter et al. (2011) found that social capital can act as a mediating variable between social determinants of health and health outcomes. More specifically, levels of income inequality, access to resources, and neighborhood safety associate with levels of social capital, which then have been shown to affect smoking behaviors, stress levels, premature mortality, cancer, and more. In this regard, community prevention frameworks that embed principles of racial equity and cultural competence in evidence-based programs are extremely important for increasing elements of social capital for racially and ethnically diverse communities; this importance is likely even greater in the case of low-income communities of color. While community prevention frameworks such CTC (Brady et al., 2018; Parker et al., 2018) are now implemented in diverse communities; empirical evidence connecting that implementation to improvements in social capital is lacking.

5 |. THE CURRENT STUDY: EVIDENCE2SUCCESS FILLING THIS GAP

The current study is an initial step in addressing this gap: can a social capital intervention in the form of a community prevention framework successfully build the related capacities and infrastructure needed to successfully adopt, implement, and sustain evidence-based prevention programs in low-income communities of color? Essentially, can a community prevention framework successfully build community social capital? We use data collected through the process evaluation of the Evidence2Success framework to begin to answer this question.

The Evidence2Success framework (The Framework) builds on the existing knowledge base of community prevention systems (Fagan et al., 2009; Spoth et al., 2011). The Evidence2Success framework promotes healthy youth development by outlining a process by which communities and public systems work together to select, plan for, implement, and evaluate youth and family programs. The framework represents a social capital intervention because it brings together people of different races, ethnicities, and socioeconomic statuses (SESs), as well as people from different sectors who hold varying levels of power and influence. Every meeting, workshop, and conversation that occurs within the work of the Evidence2Success framework provides an opportunity for interaction among the different individuals, organizations, and groups. It is through these repeated interactions that social capital changes, positively or negatively, to improve youth outcomes.

The Framework unfolds in a five-phased process underpinned by an operationalized commitment to racial and ethnic equity and inclusion and that prioritizes data use for decision making. Specifically, the Framework helps localities: (1) form partnerships that engage elected officials, public agencies and the community in working together to improve child well-being; (2) regularly collect and analyze data to establish priorities based on youth risk and protective factors; (3) select, implement, monitor, and evaluate evidence-based programs based on identified priorities that focus on prevention and early intervention; (4) analyze the effectiveness of current programmatic investments, and shift a percentage of public funding toward evidence-based programs; and (5) use a strategic financing process to plan for scaling and sustaining selected evidence-based programs.

The Framework has been implemented in different communities across the United States. Each community uses knowledge about how their public systems, services, and community organizations are organized to choose how to define “community” and its selected smaller geographic area, also known as their focus neighborhood(s), for initial implementation of tested, effective programs. While regional location and community size has varied, the majority populations in all focus neighborhoods have been people of color with low income. The variation gives us the opportunity to explore how social capital and other related capacities and infrastructure change during implementation of the Evidence2Success framework. First, given the emphasis on cross-sector collaboration within communities, we hypothesize that two measures of bonding and bridging social capital, networking collaboration, and active collaboration, will improve over time. Second, given the emphasis of different organizations, often at different levels of power and influence coming together to serve youth across differing racial/ethnic and socioeconomic backgrounds, we expect three measures of bridging and linking social capital to improve: racial inclusiveness and participation, care of youth across race/ethnicity and SES, and perceived willingness to invest in Evidence2Success.

Third, given the emphasis on including neighborhood and community residents as active members of this partnership together with elected leaders or leaders of public systems as one way to improve equity and inclusion, we hypothesize that two measures of linking social capital, community input, and community voice, will increase throughout the implementation of the Evidence2Success framework. Finally, given the content delivered during project orientations, trainings, and other workshops, we hypothesize five measures of collective capability (adoption of a science-based decision-making process, community confidence in prevention, knowledge about community prevention, confidence in tested, effective programs, and tested, effective program infrastructure) will increase throughout the implementation of the Evidence2Success framework.

Overall, we expect that measures of social capital needed to sustain evidence-based programs will improve during the implementation of the Evidence2Success framework, a linking social capital intervention. We first test these hypotheses in a sample of Community Board members, those closest to the day-to-day implementation of the Evidence2Success framework. Then, we test these hypotheses in an independent sample of community residents, Key Leaders (KLs) of up to 15 different sectors commonly found in communities. The Evidence2Success framework can be described as a linking social capital intervention because it explicitly outlines how to create connections among people and organizations of different races and ethnicities, priorities, experiences, and power, all in the pursuit of improving outcomes for youth in low-income communities of color.

6 |. METHOD

Five communities across the United States were involved in this study, covering the northeast, southeast, and west. The five communities started installing the Framework at three different time points: the first started implementing in 2012 and reached the fifth phase in 2015; the next three communities started implementing in 2015 and reached phase five in 2018; the fifth community started implementing in 2018 and reached the fifth phase slightly later due to COVID-19 pandemic delays, in 2021.

Three communities defined their broader area of concern as a county, inclusive of any major metropolitan areas and smaller municipalities within the county boundaries. Two communities defined their broader area of inclusion as a city. Each community selected at least one neighborhood within its boundaries as their focused neighborhood, the area planned to first receive an implementation of new tested, effective programs. The population of the communities that adopted the Evidence2Success framework ranged from around 37,000 to over 1.1 million (mean [M] = 500,680), while the smaller focused neighborhood areas ranged from around 6000 to 37,000 (M = 24,900) (United States Census Bureau, 2019). The primary races and ethnicities represented in each community and selected neighborhood varied. At the community-level, the primary races/ethnicities represented include White (range 45%–87%, M = 50.9%), Black, African American, or African (range 2%–70%, M = 38.7%), and Hispanic, Latino, or Spanish Origin (range 1%–43%, M = 14.4%). Primary races/ethnicities represented at the focused-neighborhood level include White (range 11%–67%, M = 26.8%), Black, African American, or African (range 1%–84%, M = 53.3%), and Hispanic, Latino, or Spanish Origin (range 1%–61%, M = 32.3%). Median income for the community and focused neighborhood, respectively, ranged from $31,000 to $71,000 (M = 47,000) and $11,000 to $62,000 (M = $32,300). Poverty levels ranged from 9% to 32% (M = 22%) and 10%–41% (M = 37%) for community and focused neighborhood areas, respectively (United States Census Bureau, 2019).

6.1 |. Participants

The current study draws from two types of participants: active Community Board members and KLs. More information about these samples follows.

6.1.1 |. Community Board members

Community Board members consisted of those individuals that were recruited by local Evidence2Success leadership to serve as a member of the Evidence2Success Community Board. Members typically cross different races, ethnicities, and SES, and include a range of different stakeholders at both the community- and focused neighborhood-levels, such as school superintendents, principals, or teachers, directors of county, state, or community mental health, public health, and substance abuse agencies, representatives from the faith community, other community organizations such as Boys and Girls Clubs, neighborhood groups, and parents, to name a few. Overall, 216 Community Board members participated in one or more data collection time points, with 43.0 participating on average across the three waves in each community (Min = 14, Max = 83).

The average age for Community Board members was 45.9 years of age (standard deviation [SD] = 12.2, range [R] = 23–80). At each data collection point, on average 29.4% was male, 70.6% reported female, and 60% self-reported White as their primary race/ethnicity, with another 36% self-reporting Black, African American, or African, 7% reporting Hispanic, Latino, or Spanish origin, and 3% reporting either Asian, Native Hawaiian or Pacific Islander, or American Indian or Alaskan Native. At each data collection point, on average 34 Community Board members (28.6%, R = 26.6%–29.8%) lived in the focus neighborhood, and 87 (78%, R = 74.7%–81.7%) lived in the community. At each data collection point, on average 67.9% of individuals in each community (R = 67.3%–69.0%), held a Masters degree or higher; another 27.6% (R = 27.4%–27.7%) held a Bachelor’s degree, attended a trade school, or had some college experience.

6.1.2 |. KLs

KLs of 10–15 different sectors were identified to participate in data collection. One person was identified as holding special knowledge and experience in each of the common sectors represented in the communities: Schools, health agency or health care, business, civic, recreation, law enforcement, judicial system, media, religious or faith organization, child welfare, health and human services, philanthropy, child and family advocacy, data monitoring and evaluation, and community or city coalition. KLs in each community were identified by local Evidence2Success leadership and were not involved in the day-to-day implementation of the Evidence2Success framework. Overall, 74 KLs participated in one or more waves of data collection, with 14.8 participating on average in each community (Min = 12, Max = 19).

The average age for KLs was 49.5 (SD = 9.4, R = 27–67). At each data collection point, on average 61.5% reported a male gender, 64% self-reported White as their primary race/ethnicity, with 30% reporting Black, African American, or African, 3% reporting Hispanic, Latino, or Spanish origin, and 2% reporting Asian, Native Hawaiian or Pacific Islander, or American Indian or Alaskan Native. Around 5 KLs (11%, R = 6.5%–15.6%) lived in the focus neighborhood, and 39 (76.4%, R = 74.0%–78.8%) lived in the community. At each data collection point, on average 69.7% (R = 63.3%–76.1%) held a Masters degree or higher; another 13 KLs (26.2%, R = 19.6%–32.7%) held a Bachelor’s degree, attended a trade school, or had some college experience.

6.2 |. Procedures

Community Board members and KLs participated in up to three face-to-face, 45 min to 1-h, computer-assisted semistructured interviews, including both quantitative and qualitative items. Data were collected using REDCap electronic data capture tools hosted at Penn State Health Milton S. Hershey Medical Center and Penn State College of Medicine (Harris et al., 2009). Face-to-face computer-assisted interviews were used as they promote high completion rates. The primary intention of the collected data was to inform next steps for Framework implementation locally and to improve Framework design. Secondarily, some collected data was used as an evaluative tool. As a result, the semistructured computer-assisted interviews included many items related not reported in this study, such as personal involvement in their local Evidence2Success implementation, the functioning of the collaborative, facilitators of success, challenges, and the community context. In addition, interviews asked about the indicators of social capital included in this study.

The first interview occurred 10–12 weeks of the Community Board Orientation; both Community Board members and KLs were invited to participate. Individuals were compensated with $20. The second interview occurred approximately 1 year later, after each Community Board reviewed their youth survey data and made decisions about their risk/protective factor and outcome priorities and selected their tested effective programs to fill existing gaps. Only the most active Community Board members were invited to participate at this data collection point. Individuals were compensated with $25. The third interview was collected approximately 1 year later, after initial planning and implementation of at least one of the selected tested effective programs began. The most active Community Board members and KLs from the 10 to 15 sectors were invited to participate. Individuals were compensated with $30.

6.3 |. Measures

The measures have been categorized into the different conceptualizations of social capital used in this study: bonding, bridging, linking, and collective capabilities. That said, because the construction of the Community Board includes community residents, representatives from community organizations across several sectors, and public systems and/or elected leaders, and because the Community Boards often involve individuals with differing racial and ethnic identities and at different levels of education and SES, the measures of bonding, bridging, and linking social capital, at times, overlap. Unless otherwise mentioned, all response options for the following variables were rated on a 1 (strongly disagree) to 4 (strongly agree) scale, where strongly agree indicates higher levels (i.e., more positive scores) of the variable in question.

6.3.1 |. Bonding and bridging social capital

Two measures of bonding and bridging social capital were collected. Networking collaboration (three items, $\bar{\alpha }=0.74$), examines whether a network exists that helps organizations keep each other informed about issues related to youth health and development. Active collaboration (two items, $\bar{r}=0.49$), refers to perceptions of how much organizations participate in joint planning and decision-making, and how much organizations share money and personnel when addressing issues related to youth health and development (Brown et al., 2008).

6.3.2 |. Bridging and linking social capital

Two measures of bridging and linking social capital were collected. Racial inclusiveness and participation (one item) assesses the involvement of individuals from diverse ethnic and cultural backgrounds in the planning and implementation of youth health and development programs and activities (Brown et al., 2008). Care for youth across races/ethnicities and SES (two items, $\bar{r}=0.67$) gauges individuals’ perception of the degree to which community residents and leaders take responsibility for health and well-being of youth from different racial, ethnic, and economic backgrounds. Perceived willingness to invest and commit resources (four items, $\bar{\alpha }=0.66$, adapted from M. Greenberg & Feinberg (2011) and Meyer (2003), gauges an individual’s sense of how far the Evidence2Success effort has progressed in assuring that organizations commit resources.

6.3.3 |. Linking social capital

Two measures of linking social capital were collected. Community input (two items, $\bar{r}=0.61$), describes the level of community input in decision-making concerning prevention programs within the region. This includes forums or procedures that are in place to provide residents a platform to share, review, and have decision-making power regarding programming in the community. Community voice (two items, $\bar{r}=0.73$), answered only by residents of focus neighborhoods, describes the level of community voice that is considered when decisions are made about youth programming in the region. Both community’s needs and preferences are taken into consideration when calculating community voice. Both scales are adapted from prior work (Watson & Foster-Fishman, 2013).

6.3.4 |. Collective capability

Five measures of collective capability were assessed. Adoption of a science-based decision-making process (22 items, variety of response options, range from 0 to 2, up to 5; Brown et al., 2014), measures individuals’ reports on how the community uses data from empirical research and knowledge about risk and protective factors when making decisions about youth and family health and development programs. These items are based on content about awareness and use of concepts of prevention science, use of epidemiologic data, use of tested and effective prevention programming, and system monitoring. Community confidence in prevention (one item; Arthur et al., 2005) gauges the perceptions of whether the people from the community had confidence that youth problems could be reduced through prevention efforts. Knowledge about community prevention (one item; Arthur et al., 2005) assessed the degree to which people who live in the region are knowledgeable about local prevention efforts. Confidence in tested-effective programs (one item; Rubin & Parrish, 2010) reviewed the desire for using tested, effective programs by individuals who are involved in making decisions about programming and by individuals across the targeted community. Tested, effective program infrastructure (three items, $\bar{\alpha }=0.81$; Rubin & Parrish, 2010), gauges attitudes and knowledge regarding tested, effective programs.

6.4 |. Data analysis

To model changes in measures of interest over time, linear mixed-effects models were used (West et al., 2014). These models included a random effect for individuals to account for the fact that individuals may have been interviewed at more than one survey wave. A fixed effect for the Evidence2Success community was included in all models to control for the potential homogeneity of responses within sites. All models used a compound symmetry covariance matrix and survey wave was treated as a linear predictor. Then, parameter estimates associated with the linear effect of the survey wave were calculated, along with the 95% confidence intervals (95% CIs) for these effects, to determine the change in measures of interest over time. In the adjusted models, fixed effects for individual-level education and race were included in the model, in addition to the fixed effect for the Evidence2Success community. Analyses were performed using the MIXED procedure in SAS 9.4 and SAS Enterprise Guide 8.3 (SAS Institute). Statistical significance was determined based on α = 0.05 for all analyses.

7 |. RESULTS

7.1 |. Descriptive statistics

Overall sample means for all waves are reported in Table 1. Moderate levels are reported at the first data collection point by Community Board members and KLs, with KLs reporting slightly higher levels in all but three variables. The highest scores at the first data collection time point are for networking collaboration (a measure of bonding and bridging social capital) and confidence in prevention and tested effective programs (two measures of collective capability). Respondents tended to have responses corresponding to “agree” and “strongly agree” to those items. Examination of the observed means suggests that positive change occurred in reports over time as reported by Community Board members.

TABLE 1.

Descriptive statistics for sample demographics and measures of social capital under investigation in this study.

		Wave 1			Wave 2			Wave 3
Variables		N	Mean	SD	N	Mean	SD	N	Mean	SD
Bonding and bridging
Active collaboration	CB	161	2.39	0.732	114	2.513	0.774	99	2.798	0.597
	KL	52	2.548	0.62	–	–	–	48	2.76	0.707
Networking collaboration	CB	163	3.018	0.638	117	3.15	0.657	98	3.306	0.498
	KL	52	3.237	0.509	–	–	–	50	3.27	0.633
Bridging and linking
Racial inclusiveness and participation	CB	156	2.276	0.913	113	2.433	0.885	96	2.75	0.767
	KL	52	2.423	0.848	–	–	–	47	2.72	0.948
Care for youth across race/ethnicity, and SES	CB	161	2.709	0.747	114	2.776	0.793	98	2.974	0.648
	KL	52	2.865	0.713	–	–	–	49	3.01	0.800
Perceived willingness to invest in Evidence2Success	CB	–	–	–	68	3.12	0.639	65	3.23	0.653
	KL	–	–	–	–	–	–	–	–	–
Linking
Community input	CB	150	1.867	0.812	110	1.905	0.895	92	2.098	0.839
	KL	46	2.087	0.896	–	–	–	45	2.078	0.852
Community voice	CB	35	2.628	1.009	31	2.935	0.814	24	3.146	0.634
	KL	3	3.167	0.288	–	–	–	7	2.571	1.096
Collective capability
Adoption of a science-based decision-making process	CB	161	1.416	1.547	117	2.265	1.882	98	2.408	1.988
	KL	51	1.235	1.68	–	–	–	50	1.54	1.918
Community confidence in prevention	CB	139	3.10	0.764	101	3.31	0.628	87	3.36	0.528
	KL	39	3.15	0.709	–	–	–	43	3.16	0.652
Knowledge about community prevention	CB	139	2.20	0.827	101	2.47	0.687	87	2.47	0.729
	KL	39	2.36	0.584	–	–	–	43	2.33	0.747
Confidence in tested-effective programs	CB	159	3.67	0.557	115	3.59	0.605	97	3.71	0.455
	KL	41	3.659	0.617	–	–	–	36	3.75	0.500
Tested, effective program infrastructure	CB	103	2.849	0.764	70	2.936	0.745	68	3.027	0.721
	KL	41	2.675	0.732	–	–	–	33	2.667	0.777

Abbreviations: KL, Key Leader; SD, standard deviation.

Hypothesis 1 – Reports of bonding and bridging social capital will improve during implementation.

See Table 2. Among Community Board members, active collaboration and networking collaboration increased significantly over time. Active collaboration increased by an average of 0.167 (95% CI 0.088, 0.246) points and networking collaboration increased by an average of 0.128 (95% CI 0.058, 0.198) points for each increase in survey wave. Among KLs, active and networking collaboration were not found to significantly increase over time (Table 2).

TABLE 2.

Results of linear mixed models examining change over time in measures of social capital for both the Community Board and Key Leader samples.

Outcome variable	Community board member sample			Key Leader sample
	N	Parameter estimate (95% CI)	p Value	N	Parameter estimate (95% CI)	p Value
Bonding and bridging social capital
Active collaboration	363	0.167 (0.088, 0.246)	<0.001*	91	0.101 (−0.026, 0.228)	0.116
Networking collaboration	366	0.128 (0.058, 0.198)	<0.001*	92	0.014 (−0.086, 0.114)	0.776
Bridging and linking social capital
Racial inclusiveness and participation	357	0.248 (0.139, 0.356)	<0.001*	91	0.132 (−0.044, 0.307)	0.138
Care for youth across race, ethnicities, and socioeconomic status	364	0.130 (0.036, 0.223)	0.007*	92	0.123 (−0.010, 0.256)	0.07
Perceived willingness to invest in E2S	133	0.134 (−0.065, 0.334)	0.184	N/A
Linking social capital
Community input	343	0.114 (0.017, 0.210)	0.021*	83	0.008 (−0.170, 0.185)	0.931
Community voice	89	0.185 (0.018, 0.351)	0.031*	N/A
Collective capability
Adoption of a science-based decision-making process	366	0.507 (0.286, 0.727)	<0.001*	92	0.157 (−0.162, 0.476)	0.328
Community confidence in prevention	317	0.136 (0.047, 0.225)	0.003*	73	0.033 (−0.098, 0.164)	0.612
Knowledge about community prevention	317	0.162 (0.071, 0.253)	<0.001*	73	0.012 (−0.111, 0.135)	0.847
Confidence in tested-effective programs	362	0.020 (−0.051, 0.091)	0.578	76	0.065 (−0.066, 0.197)	0.321
Tested, effective program infrastructure	236	0.084 (−0.020, 0.189)	0.113	73	−0.048 (−0.177, 0.082)	0.455

Abbreviation: CI, confidence interval.

^*Statistically significant at α ≤ 0.05.

Hypothesis 2 – Reports of bridging and linking social capital will improve during implementation.

In the sample of Community Board members, racial inclusiveness and participation and care for youth across race, ethnicities, and SES significantly increased over time, while perceived willingness to invest in E2S did not significantly increase (Table 2). The strongest effect was observed for racial inclusiveness and participation, where each one-unit increase in wave was associated with an average increase of 0.248 (95% CI 0.139, 0.356) points on the measure. Care for youth across race, ethnicity, and SES increased by an average of 0.130 (0.036, 0.223) points with each survey wave. Though the effect size of perceived willingness to invest in The Framework indicates similarly modest increases of 0.134 (−0.065, 0.334) points per survey wave, it was nonsignificant. In the sample of KLs, racial inclusiveness and participation and care for youth across race, ethnicity, and SES were not found to significantly increase over time (Table 2).

Hypothesis 3 – Reports of linking social capital will improve during implementation.

For Community Board members, both community input and community voice significantly increased over time (Table 2). For community input, a one-unit increase in survey wave was associated with an increase of 0.114 (95% CI 0.017, 0.210) points in the measure. Despite the small sample size, a strong effect for community voice was observed, indicating that the measure increased by an average of 0.185 (95% CI 0.018, 0.351) points for each survey wave. Among KLs, community input was not found to meaningfully change over time (Table 2).

Hypothesis 4 – Reports of collective capability will improve during implementation.

Among Community Board members, adoption of a science-based decision-making process, community confidence in prevention, and knowledge about community prevention were all found to significantly increase over time (Table 2). Specifically, each increase in survey wave was associated with an average of 0.507 (95% CI 0.286, 0.727) point increase in the adoption of a science-based decision-making process, an average of 0.136 (95% CI 0.047, 0.225) point increase in community confidence in prevention and an average of 0.162 (95% CI 0.071, 0.253) point increase in knowledge about community prevention. In contrast, confidence in tested, effective programs and tested, effective program infrastructure were not found to significantly increase over time. For KLs, none of the variables indicative of collective capability were found to meaningfully change over time (Table 2).

7.2 |. Covariates

All conclusions based on model results persisted in both the simple and covariate-adjusted models, and all results reported are from the adjusted models. More specifically, in the Community Board sample, the fixed effect of Evidence2Success community was a significant covariate in the models examining racial inclusiveness and participation, active collaboration, networking collaboration, community input, and adoption of a science-based decision-making process. Individual-level education was significantly associated with tested, effective program infrastructure and community voice. Individual-level race was a significant covariate in the models examining knowledge about community prevention, community input, tested effective program infrastructure, and perceived willingness to invest in Evidence2Success.

Specific to the KL sample, the fixed effect of the Evidence2Success community was a significant covariate only in the model examining community input, and education was significantly associated with racial inclusiveness and participation. Race was a significant covariate in the models examining active collaboration and racial inclusiveness and participation. Region-level covariates such as levels of community poverty and racial/ethnic composition were explored but not included in the final models due to the small sample size and the colinear association between the fixed effect of region and the community demographic characteristics. However, differences in the outcomes of interest across several community-level characteristics were suggested by these exploratory analyses, as shown in Supporting Information: Figure 2.

8 |. DISCUSSION

We posit that the Evidence2Success framework is a social capital intervention given that it engages individuals across typical power levels and provides opportunities for the intentional engagement of stakeholders across different community groups. Consequently, this study examined changes in social capital during the implementation of the Evidence2Success framework. We investigated this question using a longitudinal observational design with two different types of reporters. We found that Community Board member reports of multiple social capital measures improved significantly, across all communities. However, we found that reports on the same social capital measures did not significantly improve over time in the KL sample. Finally, we found that the specific Evidence2Success community was a significant covariate in the model, predicting different levels of social capital across time. Four types of social capital were explored: Bonding, bridging, linking, and collective capability.

8.1 |. Changes reported by Community Board members

We found that Community Board member reports of multiple social capital measures improved significantly and consistently, across all communities. The social capital measures under study describe the infrastructure needed for collaborative efforts to implement, evaluate, and sustain tested effective youth and family programs (Petersen, 2002). The measures crossed four common categories: Bonding, bridging, linking, and collective capability; all four types changed significantly over time.

8.1.1 |. Understanding changes in bonding, bridging, and linking social capital

The analyses showed that active collaboration, networking, racial inclusiveness, care for youth across race/ethnicity/SES, community input, and community voice, all examples of bonding, bridging, and linking social capital, significantly improved over time during the implementation of the Evidence2Success Framework. This replicates what has been found in prior experimental research of CTC (Brown et al., 2007) and other observational research on collaboration (Javdani & Allen, 2011; Orwin et al., 2014). PROSPER also found improvements in bridging and linking social capital, though different from the measures that were utilized in that study (Chilenski et al., 2014).

There are multiple possible reasons why these elements of social capital improved during the implementation of the Framework. The Evidence2Success Framework encourages the Community Board to understand disparities through a lens of equity and inclusion. It also explicitly directs the Community Boards to practice equity and inclusion within its operations and structure; for example, the Framework advises communities to recruit and engage people of color on the Community Board. It also suggests that neighborhood residents most affected by public systems participate on the Evidence2Success Community Board, along with public system and organizational representatives. Understanding disparities from a lens of equity and inclusion while practicing equity and inclusion may help successfully address historical inequities. This process may disrupt plausible reinforcement of inequities in coalition formation and functioning that could otherwise unintentionally negatively impact outcomes and continually reinforce disparities. The positive changes in reports of community voice and input in decision-making are likely connected to this fundamental operating procedure of the Evidence2Success Framework. It is also possible that the significant improvements in bonding, bridging, and linking social capital connect to the diversity of racial and ethnic groups that become part of the Community Board.

Given the Board’s operational structure, decisions are made with individuals that cross typical community power hierarchies, as community members who serve on the Board make decisions along with leaders in public systems, community-wide organizations, and organizations focused on the specific neighborhood(s) of interest. In addition, individuals and organizations with differing areas of focus in supporting youth and families typically become involved, such as public health, education, housing, mental health, the faith community, recreation, or other areas.

Community Board activities typically include monthly meetings and making decisions about programs and resources led by a Community Coordinator. The communication and cooperative planning that occurs through these repeated interactions at regular meetings among Community Board members may have changed how they perceived the work.

Bringing together different types of people and organizations within a collaborative working and decision-making context is likely central to building social capital. Practically speaking, this means that norms and expectations around active collaboration, networking collaborations, involving diverse individuals in planning and decision-making, care for the health and development of all youth, and community voice and involvement in decision-making all improved while implementing the Evidence2Success Framework in these five communities. For example, one Community Board member noted that “the adoption of the Evidence2Success framework was essential in constructing a board organizational structure focused on getting work done and being transparent in its dealings within our community. Transparency, explicit outreach, and word of mouth helped get the word out regarding results and to community volunteers interested in supporting the effort. Structure and prescribed approach are foundational to meeting opportunities head on.”

8.1.2 |. Understanding changes in collective capability

The analyses showed that the adoption of a science-based decision-making process, community confidence in prevention, and knowledge about community prevention programs, all examples of collective capability social capital, significantly improved over time during the implementation of the Evidence2Success Framework. This replicates what has been found in prior experimental research of CTC, with the adoption of a science-based decision-making process (Brown et al., 2007, 2011). Similar results were found in evaluation research of the Getting To Outcomes framework, with improvements in data use to make decisions (Chinman et al., 2008). This also replicates former observational research in the field of domestic violence, where improvements in knowledge specific to the problem and solution were found (Javdani & Allen, 2011).

There are multiple possible reasons why the collective capability elements of social capital improved during the implementation of the Framework. The Community Board structure includes additional workshops focused on special topics intended to build new skills, acquire new knowledge, and change the way that decisions about youth and family programs are made. It is likely that workshop content associates with the observed changes over time. For instance, orientation and workshop materials include information about how negative health behaviors can be prevented and healthy behaviors and decisions can be promoted, including a summary of the expected long-term fiscal benefits for some evidence-based programs. This content connects directly to building community confidence in prevention and knowledge about community prevention activities. Through the repeated interactions of the intentionally diverse Community Board in this shared space, the collective capability about prevention programs is likely to improve as reflected in measures of community confidence in prevention and knowledge about community prevention.

Meanwhile, advances in adopting a science-based decision-making process are likely connected to the central work of Evidence2Success much like CTC (Fagan et al., 2009); that is, all decisions are made through systematic collection and careful examination of data. For example, local data partners collect data on youth risk, protection, and outcome behaviors in the Youth Experiences Survey. A Finance Lead collects information on how funders distribute resources across programs. In addition, the Community Board and Finance Lead collect information about existing programs that connect to the selected priorities. All this information, and more, is used to make decisions about community priorities and the most appropriate evidence-based programs given priorities and resources. Once programs are selected, data-informed decisions are made about program goals and evaluation, and these decisions are put into place with Implementation Teams. Thus, the Framework guides the Community Board through a science-based decision-making process to select, implement, and evaluate youth and family programs. The repeated data-informed interactions with the intentionally diverse Community Board then are likely to build the community’s collective capability around adopting a science-based decision-making process. For example, the focus on youth survey data led one coalition to establish a Youth Council to review and make decisions with the data. After reviewing the data, the youth prioritized activities, programming, and events that supplement the evidence-based programs being implemented in their community.

The Evidence2Success Framework guides communities through a five-phased approach. The activities of one phase build on the prior phase, and documentation of progress is an explicit part of the work. The documentation likely helps increase trust in the process, as it promotes transparency. It is possible that this kind of gradual, step-by-step approach allows community collaborative efforts such as Evidence2Success to experience success and growth one step at a time, as the collaborative board can see the progress they have made along the way; the approach may also help reinforce motivation for the work. For example, it is possible that establishing the board and its mission in phases allows the Community Boards to build social capital as members simultaneously get to know each other and become “keepers” of the framework’s goals. This collective accountability is important to establishing trust and maintaining focus.

In summary, the repeated interactions connected to special content and skills-building, within the context of an intentionally diverse Community Board are likely to promote a shared perspective that work related to youth health and development is getting better and achieving desired outcomes. Community Board members are likely creating shared norms and beliefs.

8.2 |. Lack of change reported by KLs

In this study, KL reports of the same social capital measures did not significantly change over time. This sample was completely independent from the Community Board sample. There were some KLs that were also Community Board members; however, KLs that were also Community Board members were assigned “Community Board” member status for the analyses. Consequently, the KL sample in this study was not regularly involved in the activities of the Evidence2Success Framework.

On the one hand, we would think that significant improvements reported by KLs would be an indication that these changes truly occurred and were institutionalized in the community, that the community’s interconnectedness and ways of supporting youth and families truly changed. However, when we understand these changes as changes in social capital, it makes sense that changes in these measures were not found among KLs. Relationships and connection need to be present to build social capital. Social capital needs connection, relationships, and repeated communication. The existing Evidence2Success Framework does not give explicit directives regarding how to involve or engage KLs. Though we expect that the KLs involved in these data collection periods were familiar with one another based on their histories within and across their service areas, the KL sample, was not necessarily knowledgeable about all the work that was going into the Evidence2Success framework. Though there may be some opportunity during other meetings to create a sense of shared trust or norms, without explicit conversations about the Evidence2Success framework, it does not seem likely that any developed social capital is generalized to the greater Evidence2Success effort.

As suggested earlier, it is likely that the changes in Community Board member reports reflect those regular conversations board members have with each other. These KLs are not in the room with the board during meetings or when they are making decisions, and they do not attend or participate in any of the workshops or key Framework decision-making meetings. Those conversations reinforce the idea that things are changing, improving, getting better, and that the board is achieving their desired outcomes. However, KLs, not regularly engaged in the work are not necessarily bought in to the same degree. For example, one community qualitatively described feeling some support from local elected officials, but that local elected officials have been slow to promote and rally the community around the Evidence2Success effort.

Typically, KLs play a large role in bringing the Evidence2Success Framework into their community; however, many KLs step aside and let others become involved in the day-to-day work. Unless there was a regular touch point when progress was shared with the KLs, there is no way for the KL to understand the degree to which the interconnections, trust, and expectations are changing among those individuals that are engaged in the day-to-day work of Evidence2Success.

It is also possible that KLs may not emphasize the work of Evidence2Success since it starts small with programming focused on one neighborhood or a cluster of neighborhoods and involves community members that historically have not been in positions of power. This involvement shifts the typical leadership paradigm. Oftentimes, public system leaders or city or countywide elected leaders feel pressure to make changes across the community all at once, so as not to “advantage” one community over another. These leaders may expect it will be more efficient and run smoothly to institute the change everywhere at one time, which is in contrast with Community Board procedures.

It is also possible that the priorities selected for programming do not end up matching with previously articulated KL priorities. Perhaps KLs are more wedded to the areas that they represent or the formerly communicated priorities.

Understanding the role of KLs that are not regularly a part of the activities and how to increase their knowledge and support is important. Partnerships with KLs are needed to get things done. We expect that making progress with KLs will be important to support the initiative.

Finally, it is also possible that these KLs not regularly engaged in the Framework have different metrics for change or for success. They may have different definitions or expectations of what change looks like or what they want to see happen over the 2–3 year period.

In summary, KL reports of social capital did not significantly change over time. There are several possible reasons for this finding; these results provide a learning opportunity to improve community collaborative prevention systems.

8.3 |. The impact of the Covid-19 pandemic

Four of the five communities in this study completed their five-phase Evidence2Success process before the Covid-19 pandemic began. However, the Covid-19 pandemic began 2 years after the fifth community began work related to The Framework. This meant that the fifth community completed its second set of interviews just before the shutdowns in March 2020, and their third set of interviews occurred a bit more than 1 year into the COVID-19 pandemic. This fifth community, as did all the Evidence2Success Community Boards, quickly adjusted their workflow to pause on regular operations and support immediate needs of families in their communities, such as food distribution, digital and Internet resources, and childcare support. They also quickly changed all meetings from in-person to an internet interface, which largely continues today. Fortunately, we heard that attendance and engagement of Community Board members were not hindered with the web-interface, and at times people said they felt the web-interface improved attendance and engagement. After a few months of adjustments, all Evidence2Success communities continued from where they left off with the work—selecting, planning for, or implementing their selected evidence-based programs. However, many KLs’ attention remained diverted to the ongoing and ever-changing public health situation, likely affecting participation in that third interview, and lessoning their connection to the traditional work of the Evdence2Success framework. Also, all the Evidence2Success communities delayed their regularly planned collection of the youth survey, as students were at home and security of data collection and validity collected in that context was questionable. These factors may have lessened the degree of improvement experienced in this fifth community overall, and especially for the data reported by KLs.

8.4 |. Exploration of community characteristics

Post-hoc analyses revealed the possible empirical importance of the community context and demographic make-up as important factors to building social capital. Specifically, our exploration suggested that changes in community voice and input over time was different given the community demographic percent Black and the percent of families living below the federal poverty line. In communities with greater than 20% identifying as Black, African American, or African, there may be a steeper rate of increase on our measure of community voice from Wave 1 to Wave 3 than in communities with 20% or less who identified as Black. On the other hand, communities with greater than 25% of families below the federal poverty level, may have experienced less improvement in community voice when compared to communities with 25% or less of families below the federal poverty line. These results are exploratory. They indicate that it may be important to explore how community conditions contribute to achieving social capital, especially within community prevention systems; it is possible that initial expectations for community voice may have been lower in a largely Black community, perceiving change in a much stronger way.

It is also possible that the divergence in findings between the Community Board and KL samples is related to differing timing related to immigration patterns in each of the communities. Some of the targeted communities were “settled in”—that is that these communities have a long-standing history of the racial/ethnic minority community that resided there. Some communities, however, were “settling out”—that is, historically rooted minorities were moving to “safer communities” and being inhabited by newer minorities (some from the same racial/ethnic community). This movement may impact levels of trust, engagement, and levels of youth risk/protection. Further, in a study of community variations in coalition functioning, Gayles and colleagues (in preparation) show that different coalition capacities were differentially associated with community characteristics. This work aligns with public health’s understanding that social determinants of health and sociopolitical and historical conditions can shape community capital and agency to achieve health outcomes (Hunter et al., 2011). These nascent exploratory analyses point to a need to further understand and consider how the community context can variably influence prevention systems and improve social capital to achieve desired outcomes.

8.5 |. Implications for prevention

These findings have implications for prevention. First and foremost, these findings suggest that involving community members alongside traditional community leaders and decision-makers may be powerful component of improving a community’s social capital. Repeated interactions and communication, among people with different backgrounds and experiences seem uniquely positioned to support this improvement. Given these results, the next step to build on this work could be to create an explicit coleader role between a leader from the focused neighborhood to work alongside a more traditional systems-level leader.

Second, these findings also suggest that it may be important to change how and where we invite different groups to meet. Frequently, school boards and city/county councils ask community members to attend meetings on their turf and follow established procedures to share their thoughts or feelings about the policies or procedures up for discussion. Instead, the Evidence2Success Community Board typically convenes in the focus neighborhood, in a library, community center, or school building, rather than a city hall or administrative office.

Third, these findings suggest that providing a logical and structured, but adaptable, framework, especially one that focuses on using data to make decisions and promotes transparency of decisions, may be especially beneficial for communities. Frameworks such as CTC, PROSPER, and Evidence2Success provide community members with a valuable tool for achieving and spearheading desired outcomes in their communities. Without the structure and repeated documentation and communication of both the Framework’s process and results, it may be difficult to establish a sustained community effort.

Fourth, these findings suggest it is important to keep data at the core of community change efforts. The data are the avenue for individuals to understand the experiences of others, rather than base decisions on assumptions or the experience of one. The data provide a springboard for discussion, which can promote understanding and decision-making. The data can be eye-opening for public system leads and community members. Including data on levels of risk and protection, as well as outcomes, may promote a feeling of efficacy that something can be done to disrupt pathways that lead youth in the wrong direction.

We can also learn from these analyses about potential areas for improvement. These results suggest that additional outreach may be needed with KLs not regularly connected to the Evidence2Success work. For instance, it may be useful to invite them to key decision-making meetings, or additional communications through traditional community press outlets or social media may be useful.

8.6 |. Limitations

This study has a few limitations. First, this study utilized an observational repeated measures design. This study did not include comparison communities. Because of this design, we can only describe change that occurred during the implementation of the Evidence2Success Framework. Future research that includes a larger number of communities and a comparison group is suggested.

Second, in terms of external validity, we can most reasonably generalize these findings to similar communities. Though the involved communities ranged in size, region, and other demographic characteristics, they also had similarities of being historically marginalized. This framework and the results may work differently in differing community contexts.

Third, due to the research design, the KL was smaller compared to the Community Board members. The slightly smaller sample was partially due to only inviting KLs to participate in two interviews rather than three. Additionally, out of a maximum of 15 identified KL sectors, some of those individuals were regularly involved in Evidence2Success implementation, making them Community Board members for analyses. Also, often the number of Community Board members approached 20. Consequently, the smaller independent KL sample led to lower levels of statistical power and less precise effect estimates.

Fourth, as we continually adapted our instruments to try to improve our measurement, we did not have all the same measures for the first cohort as we did the latter two cohorts. Also, we made additional changes due to changing evaluation priorities for the third cohort. We addressed this by selecting the most consistent measures to what the Framework was trying to achieve, but the fluctuation may have weakened our design. The community voice, willingness to invest, and tested, effective infrastructure constructs were most affected by these decisions.

9 |. CONCLUSION

This study examined changes in social capital during the implementation of the Evidence2Success framework. Using a longitudinal and observational design, we found that Community Board member reports of bonding, bridging, linking, and collective capability social capital measures improved significantly, across all participating communities. However, we found that reports on the same social capital measures did not significantly improve over time in the KL sample. These findings suggest that providing historically marginalized communities a logical and prescribed data-based framework that allows for community adaptation and promotes transparency may help communities build important characteristics of social capital that are likely to lead to sustained improvements in youth outcomes communitywide. Future research with a design that can attribute causality and better explore the importance and meaning of different combinations of community characteristics is a crucial next step in building the knowledge base of effective coalition and community change models for historically marginalized communities.

DATA AVAILABILITY STATEMENT

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions. If this manuscript is accepted, we will work with our institution, our funder, and the communities involved in data collection to make the data as available as we can with appropriate protections.