Abstract
Background
Coalitions are increasingly utilized to promote positive community health outcomes. Typically, coalitions produce more desirable outcomes when coalition functioning is strong and the coalition model is implemented with fidelity. Although theory indicates that coalitions proceed through predictable stages of development, minimal research explicitly examines functioning and fidelity at these different stages.
Method
Within a larger evaluation of Washington State Community Prevention and Wellness Initiative, this cross-sectional study employs qualitative comparative analysis to illuminate the coalition functioning conditions necessary and sufficient to produce high model fidelity at different stages of development in 43 substance misuse prevention coalitions in one state.
Results
In the formation stage, only the presence of high levels of coalition leadership was sufficient to produce high model fidelity. In the maintenance stage, three combinations of conditions were sufficient: (1) sustainability planning if, and only if, accompanied by the absence of coalition participation costs, (2) coordinator leadership, and (3) a combination of coalition leadership and team cohesion. In the institutionalization stage, two solutions were sufficient: (1) coalition leadership if, and only if, accompanied by the absence of sustainability planning, and (2) sustainability planning if, and only if, accompanied by the absence of coordinator leadership.
Conclusions
This study illustrates several tangible steps technical assistance providers may take to increase the likelihood of achieving model fidelity. In the formation stage, skillful and inclusive coalition leadership is important. In the maintenance stage, technical assistance should focus on reducing participant-perceived costs; increasing sustainability planning; enhancing coordinator-specific leadership; and developing team cohesion and coalition leadership. For coalitions in the institutionalization stage, coalition leadership and sustainability planning may be prime targets for technical assistance.
Plain Language Summary: Community coalition approaches to addressing social problems are common and have some evidence of producing positive community outcomes. Research shows that coalitions produce more desirable outcomes when coalition functioning is strong and the coalition model is implemented as it was designed. Although theory suggests that coalitions proceed through predictable stages of development, few studies look at which supports are most needed, at each stage, to improve coalition functioning and implementation. This study aims to help answer that question by determining which aspects of coalition functioning, in which combinations, are key to strong, well-functioning coalitions at different developmental stages in a sample of prevention coalitions in Washington State focused on youth substance misuse. The results suggest that having a strong coalition leadership team is key for all coalitions, regardless of development stage. For coalitions earlier in their formation, this was especially true. For coalitions further into their development, the combination of strong coordinator leadership and strong team cohesion was particularly important. Finally, for the most well-established coalitions, if sustainability planning was weak, strong coalition team leadership was key; but if coordinator leadership was weak, strong sustainability planning was critical to offset the lack of strong leadership. The results can be used to inform the types of technical assistance provided to support coalitions at different developmental stages.
Keywords: coalitions, developmental stages, technical assistance, qualitative comparative analysis
Introduction
Demand for coalition-driven programming is great, with many prevention funding agencies requiring use of community coalition models (Butterfoss et al., 2008). There is ample evidence that coalitions are successful as a mechanism for realizing community change (Anderson et al., 2012; Butterfoss et al., 2008), particularly when implemented with fidelity (Feinberg et al., 2008; Gomez et al., 2005). Coalition-led interventions benefit communities by increasing connections between individuals and relevant care providers, thereby improving health outcomes (Spoth et al., 2013). Some coalition models have documented benefit-cost ratios such that the economic benefits associated with positive community-level health outcomes outweigh coalition implementation costs (Kuklinski et al., 2015; Miller & Hendrie, 2008).
Despite increased use of coalition models, little is known about the developmental needs of coalitions or the conditions necessary and sufficient to produce positive outcomes. Most coalition theories contain some schema of development that suggests nascent coalitions may have different needs than mature coalitions, and while the literature is awash with technical assistance advice, a gap exists around the specific needs of coalitions at different stages of development. This study examines the different aspects of functioning which need to be present at different developmental stages for coalitions to attain high model fidelity. Identification of these factors will allow coalitions and assisting agencies to actively support coalitions across stages of development. More effective support, in turn, will allow for better community health outcomes and greater stewardship of public funds.
This study was conducted within the context of a process evaluation of Washington State Community Prevention and Wellness Initiative. CPWI is a collaboration between state, county, and local agencies and stakeholders with the overarching goal of preventing alcohol and substance misuse among youth (Washington State Health Care Authority, 2021). The coalition uses a strategic planning process to examine local data on risk and protective factors related to youth substance use and then select and implement relevant prevention programming in the community. Initial outcome evaluations have demonstrated CPWI’s effectiveness in reducing alcohol use by 10th grade students, as well as in reducing concomitant risk factors (Shrestha et al., 2019).
Coalition Functioning and Model Fidelity
Coalition models differ in requirements, but most posit that model fidelity is needed to produce the desired community outcomes. In this case, model fidelity is conceptualized as the degree to which an intervention is implemented as originally intended and in accordance with existing theory and research supporting the intervention (Berkel et al., 2011). Model fidelity for a coalition intervention entails: (1) selection of programs for implementation that are aligned with community-specific risk and protective factor data and (2) implementation of these programs, in turn, with fidelity (Brown et al., 2010). In the Communities that Care (CTC) conceptual model (Brown et al., 2012; Hawkins & Catalano, 1992), depicted in Figure 1, fidelity is assigned an intermediate position in the pathway to community outcomes. Though CTC is a separate coalition model from the CPWI examined in this study, the development of CPWI was informed by CTC, so many similarities exist between the two: in both models, coalitions incorporate stakeholders from diverse sectors; complete local needs assessments; identify local priority risk and protective factors; and deliver an array of youth- and family-oriented, evidence-supported interventions (Communities That Care, 2018; Division of Behavioral Health and Recovery [DBHR], 2017).
Given similarity in core aspects of operation and program delivery, it is reasonable to adopt the CTC model’s emphasis on fidelity as a key outcome in this investigation of CPWI functioning and model fidelity. According to the CTC model, coalition functioning comprises of five factors: (1) leadership, (2) interpersonal relationships, (3) task focus, (4) participation benefits and costs, and (5) sustainability planning. This study investigates the relationship between these factors and model fidelity at each specified stage of CPWI coalition development.
Community Coalition Action Theory: A Developmental Perspective
This study uses Butterfoss’ (2007) Community Coalition Action Theory (CCAT) as a guiding framework for understanding coalition development. In CCAT, a coalition is defined as “a formal alliance of organizations that come together to work for a common goal” (p. 30). Work is completed collaboratively and resources are often pooled. The CCAT forms the foundation for many investigations into coalition functioning and effectiveness, including studies of the CTC coalition model (Brown et al., 2012; Hawkins et al., 2014). CCAT postulates that coalitions proceed through three developmental stages: (1) formation, (2) maintenance, and (3) institutionalization. The formation stage is usually initiated by a convening agency with roots in the community and includes activities such as organizational outreach and member recruitment (Butterfoss, 2007). During the maintenance stage, coalitions strive to retain membership while working towards coalition goals via assessment, planning, and selection and implementation of interventions (Butterfoss & Kegler, 2002). Coalitions in the institutionalization stage may be engaged in activities from the previous stages and have typically also mobilized resources in a manner which resulted in realization of the community-level outcomes which they prioritized. This having occurred, coalitions are more likely to achieve longevity or to have their strategies integrated into established community organizations; thus, they are institutionalized (Butterfoss & Kegler, 2002). As little is known about the factors which influence success at each stage, particularly the maintenance and formation stages (Butterfoss & Kegler, 2002), such research would greatly benefit the field.
Factors Varying Across Coalition Stage of Development
Minimal investigation has been conducted into the relationship between coalition developmental stage and coalition functioning. Some qualitative research exists supporting the role of factors such as leadership, staffing, and communication in coalitions’ ability to implement programs, but research on coalition developmental stage has been restricted either to one stage (Kegler, et al., 1998) or was ancillary to the study’s primary aims (Kramer et al., 2005), limiting the ability to compare differences across developmental stages. Among the few exceptions, a quantitative study by Chinman et al. (1996) found greater coalition member participation at later stages of development among individuals perceiving more benefits and lower participation among those members perceiving more costs. Kegler et al. (2010) found that contextual differences (e.g., history of collaboration) affected aspects of coalition functioning (e.g., leadership) in the formation stage. In both studies, however, consideration of coalition development was limited to either one development stage or one aspect of coalition functioning.
The Current Study
We completed this exploratory study within the larger context of a process evaluation of Washington State CPWI and aimed to determine the necessary and sufficient coalition functioning conditions for high model fidelity to occur in each stage of coalition development. As described above, the CTC conceptual model emphasizes the central role of coalition functioning in predicting model fidelity and other research suggests that coalition functioning may play different roles within different stages of coalition development. Therefore, we explore these differences by conducting qualitative comparative analysis (QCA), a technique capable of capturing causal complexity in samples as small as 5–10 cases (Hill et al., 2019; Kane et al., 2014), with survey data from coordinators of coalitions in each of the three developmental stages (Butterfoss, 2007).
Method
Participants and Procedures
The Washington State University Office of Research Assurances found that this project was exempt from the need for IRB review (certification 16225-001) and thus no informed consent was required or obtained. We recruited participants in the summer of 2017 based on their roles as coalition coordinators in Washington State’s CPWI. We identified participants for this study in collaboration with Washington State’s Division of Behavioral Health and Recovery (DBHR)1. A representative with DBHR sent an initial email to 61 CPWI coordinators inviting them to complete an online survey. Potential participants were told the survey was a part of the ongoing statewide evaluation of CPWI and that the goals of the survey were to identify (a) practices that support implementation of CPWI activities and (b) areas in which DBHR could better assist CPWI coalitions.
Respondents were spread across four cohorts: Cohort 1 began implementing CPWI in 2011, Cohort 2 in 2012, Cohort 3 in 2013, and Cohort 4 in 2015. Forty-three coordinators completed the survey (response rate: 70.5%). The 43 coalitions represented all regions of Washington State and included rural and urban locations. On average, coordinators were 42.6 years old (range 25–69) and mostly female (76.7%) and White (86%), with 11.6% identifying as Hispanic/Latino. Most reported completion of a bachelor’s degree or higher (78.5%). On average, coordinators had been in their position for 3 years and 10 months (range 3 months to 19 years 5 months).
Measures
Survey Characteristics
We adapted the current survey from existing, validated measures of coalition functioning. We calculated Cronbach’s alpha coefficients for each scale and found they had acceptable reliability (α = 0.64–0.90). For each scale, a summary score was created by calculating the mean score across all items in that scale. See Table 1 for alpha coefficients and sample items 1.
Table 1.
Construct (No. of items) |
Sample item | Response options | α | Mean (SD) |
% of coalitions with condition present | N |
---|---|---|---|---|---|---|
Coordinator leadership (4) | How skilled is your CPWI Coalition Coordinator or lead
staff person in the following
areas? …Organizational skills |
Needs Work (1) Adequate (3) Strong (5) Very Strong (7) |
0.64 | 5.8 (0.8) |
61% | 41 |
Coalition team leadership (9) | The leadership of our CPWI coalition intentionally seeks out our views. | Strongly Disagree (1) Somewhat Disagree (3) Somewhat Agree (5) Strongly Agree (7) |
0.88 | 6.0 (0.8) |
46% | 41 |
Team cohesion (5) | There is a feeling of unity and cohesion in this coalition. | Strongly Disagree (1) Somewhat Disagree (3) Somewhat Agree (5) Strongly Agree (7) |
0.90 | 5.4 (1.4) |
51% | 43 |
Task focus (4) | This is a highly efficient, work-oriented team | Strongly Disagree (1) Somewhat Disagree (3) Somewhat Agree (5) Strongly Agree (7) |
0.83 | 5.2 (1.3) |
51% | 43 |
CPWI participation costs (3) | How much has CPWI interfered with: …your family life? |
Not at all (1) A little (3) Some (5) A great deal (7) |
0.76 | 3.5 (1.5) |
50% | 42 |
CPWI participation benefits (3) | How much benefit have you gained from your involvement
with CPWI in these areas: …learning new skills? |
Not at all (1) A little (3) Some (5) A great deal (7) |
0.67 | 6.0 (0.9) |
56% | 43 |
Sustainability planning (5) | In the past year, has your CPWI coalition researched alternative funding sources or written proposals for additional funds to support CPWI activities? | 1 = Investigating Availability 2 = Begun Writing Proposal 3 = Submitted Proposal |
0.67 | 1.7 (0.7) |
35% | 37 |
Note. For all items measured via 7-point Likert scale, respondents were presented with response options (2), (4), and (6), as well. These were not accompanied by text and so are removed from the table for ease of consumption.
Coalition Functioning
This battery of scales examined degree of internal coalition functioning (Brown et al. 2012). All coalition functioning scales except one (assessing sustainability planning) were adapted from the CTC Board Survey 2010 (Greenberg & Feinberg, 2010). Except for the “sustainability planning” scale, response options ranged from 1 to 7 (anchors varied; see Table 1 for details).
Coordinator Leadership. This scale requested that coordinators self-report on their skills as coalition leader. Areas of inquiry included: interpersonal and communication skills, organizational skills, enthusiasm and passion for the CPWI mission, and an understanding and knowledge of prevention and CPWI.
Coalition Team Leadership. This scale asked coordinators to assess the competence of their coalition’s broader leadership team. This includes themselves, as well as committee chairs, officers, or otherwise installed officials. Areas of inquiry included: inclusiveness and openness of leadership, skill in resolving conflict and navigating political situations, and reputation in the community.
Team Cohesion. This scale assessed the degree to which group cohesion existed among coalition members. Areas of inquiry included: inter-member trust and feelings of unity, belonging, closeness, and group spirit.
Task Focus. This scale queried coordinators about the efficiency and work-directedness of their coalition’s operations. Areas of inquiry included: degree of teamwork-orientation, attentiveness to work requirements, degree of team work ethic, and time wasted (reverse scored).
CPWI Participation Costs. This scale assessed the personal costs coordinators’ perceived as a result of their involvement with CPWI. Areas of inquiry included: CPWI interference with other work responsibilities, interference with family life, and with personal free time.
CPWI Participation Benefits. This scale assessed the personal benefits or rewards coordinators perceived as a result of their involvement with CPWI. Areas of inquiry included: development of new skills, development of valuable relationships, and feelings of personal fulfillment.
Sustainability Planning. This scale was adapted based on the Annual Survey of Evidence-based Programs (Rhoades et al., 2012) and it reflected the degree to which coalitions have engaged in financial development planning. They were asked about six specific funding sources and for each source responses indicated whether a coalition: had investigated availability of the funding source, had begun writing a proposal for the funding source, or had submitted a completed proposal for the funding source.
Outcome Measure: Model Fidelity
As Brown et al. (2010) note, high coalition model fidelity ensures that programs selected for implementation are aligned with community-specific needs, as determined by risk and protective factor data, and that these programs are themselves implemented with fidelity. Following this logic, an eight-item scale was developed to assess coalition model fidelity. Six of the items were derived from the CTC Board Survey (Brown et al., 2010, 2012) and assessed the degree to which: the selection of programs to be implemented was informed by communities’ specific needs assessments; CPWI activities were selected after consideration of community-specific risk factors; coalitions monitored the implementation fidelity of programs they support; technical assistance and training were provided to program implementers; coalitions evaluate the impact of implemented programs on their prioritized risk factors; and coalitions reviewed their action plan annually. Two additional CPWI-specific items were added based on feedback from DBHR—these items assessed: the degree to which coalitions reviewed their CPWI strategic plan each year and the degree to which their CPWI strategic plan included a clear logic model.
For each item, participants were told: “Below is a list of some goals for CPWI coalitions. For each one, choose how far the coalition has progressed to date.” Responses were given based on the same scale used by Brown et al. (2010; 2012) with a “1” indicating “No,” “3” indicating “No, but working on it,” “5” indicating “Yes, to a limited extent” and “7” indicating “Yes.” The even numbers (i.e., 2, 4, and 6) were available for respondents to choose if progress was somewhere in between one of the odd number anchored responses. A summary score reflecting the degree to which coalitions achieved global model fidelity was created by calculating the mean across all items. Respondents who demonstrated a score at or above the mean fidelity score for the developmental stage in which their coalition resided were considered to have achieved high model fidelity; respondents below the mean fidelity score for the developmental stage in which their coalition resided were considered to have achieved low model fidelity.
Developmental Stage Identification
We asked coordinators to identify the statement which most accurately represented their current stage of development: formation, maintenance, and institutionalization. We created these three statements to reflect key principles of each developmental stage explicated in the CCAT. The chosen statement provided the basis for categorization into these stages (see Table 2 for details).
Table 2.
Construct | Prompt | Response options |
---|---|---|
Formation stage | Coalitions progress through a series of developmental stages. Please read the below statements and select the statement that best describes the work of your coalition over the past 6 months. | We are working to establish partnerships and recruit coalition members that bring together key sectors of our community (e.g., educators, law enforcement, youth) to address youth substance use in our community. Our coalition leadership is working to develop processes and procedures (e.g., communication strategies, decision-making protocols, creating workgroups/committees) to guide our CPWI initiative. |
Maintenance stage | Same as above | We have conducted planning and assessment activities and have the capacity to implement prevention programs and practices that address our short-term (e.g., increase community awareness) and long-term goals (e.g., reduce and prevent youth substance use). |
Institutionalization stage | Same as above | Our CPWI coalition is well-known in our community and actively involved in community-wide efforts to prevent youth substance use. We work collaboratively with community partners to share resources and offer prevention programs. Local leadership in our community consults with our coalition on matters related to youth substance use. |
Analytic Approach
Qualitative Comparative Analysis
QCA, despite its name, leverages quantitative procedures which account for causal complexity (Ragin, 2008) by examining “insufficient but non-redundant part(s) of an unnecessary but sufficient condition” (Mackie, 1980, p. 62), or INUS conditions. QCA is especially useful for understanding how multiple, interacting factors (i.e., conditions) lead to the same outcome (i.e., equifinality) and is increasingly used to evaluate complex interventions. For example, Hill and colleagues used QCA to determine which combinations of implementation, program delivery, and participant factors were necessary for positive outcomes in a complex multi-component family-based intervention (Hill et al., 2019). Several variations of QCA exist (see Schneider & Wagemann, 2012). We used crisp-set QCA, in which conditions and outcome are dichotomized. This decision provided three specific benefits and is aligned with recommendations from other implementation researchers (e.g., Hill et al., 2019; Kane et al., 2014): (1) the number of cases both with, and without, the outcome of interest became approximately balanced, (2) by splitting the outcome into lower- and higher-achieving fidelity groups, interpretability of the results was improved, and (3) it allowed us to make conclusions about differences in “kind” as opposed to differences in “degree” of the included coalition functioning conditions and how their combinations lead to high model fidelity (Rohlfing, 2020).
There are five broad steps in the crisp-set QCA process: (1) calibration (i.e., dichotomization of continuous variables, in our case), (2) creation of truth tables, (3) necessity analysis, (4) sufficiency analysis, and (5) analysis of model fit. These steps are summarized below as they relate to the current study. For a more detailed overview of this process, see Thiem (2017). The software used for these analyses was f/sQCA (Ragin & Davey, 2016).
Calibration. Both outcome and conditions were dichotomized via mean-split within developmental stage. Specifically, those with responses at or above the mean for their developmental stage were scored “1” for that condition/outcome and those with responses below the mean were scored “0” for that condition/outcome (see Table 1 for frequencies).
Truth Tables. Truth tables for each stage were created by using the f/sQCA software (Ragin & Davey, 2016). The truth tables provide a visual demonstration of the configurations of the conditions observed in the data and the frequency each configuration appeared as well as how often the configuration preceded the outcome of high model fidelity.
Necessity Analysis. Though no clear guidelines around the requirements for a claim of necessity have been established in the QCA community, it is recommended that the criterion be higher than the 0.8 cutoff recommended for consistency (Schneider & Wagemann, 2010). To satisfy this suggestion, we utilized a necessity cutoff of 0.9 in this study. This means that to be considered a necessary condition it must be present in 90% of the coalitions that achieved model fidelity.
Sufficiency Analysis. Three solutions were generated during this analysis: complex, intermediate, and parsimonious. Intermediate and complex QCA solutions are subsets of the parsimonious solution that have not been reduced to “prime implicants,” or those conditions most critical to the actualization of an outcome (Cooper et al., 2014). While debate exists about which QCA solution set to report (Ragin & Sonnett, 2005; Thiem, 2017), this debate is restricted to the intermediate and parsimonious solution sets. As this study is de facto faced with only the complex and parsimonious solution sets (see Results section for details), the parsimonious sets are reported and will be promoted for use by practitioners and program implementers.
Model Fit. Model fit for sufficiency analyses are assessed with two primary indicators: coverage and consistency. Coverage is analogous to variance explained in traditional regression analyses and ranges from 0 to 1 with higher values indicating greater empirical relevance. Two versions of coverage are reported in Table 3. Raw coverage denotes the overall proportion of cases achieving model fidelity via a given solution. For instance, coalition team leadership demonstrates a raw coverage of 1.0 in the formation stage, indicating that 100% of cases achieving model fidelity noted the presence of coalition team leadership. See below for details. Unique coverage indicates the percent of cases for which only that solution was present. Consistency conveys the strength of a set relationship and ranges from 0 to 1 with higher values indicating a stronger relationship. Consistency values above 0.8 are suggested to be acceptable for claims of sufficiency (Ragin, 2009).
Table 3.
Formation stage (n = 8) | Raw coverage | Unique coverage | Consistency |
---|---|---|---|
Coalition team leadership | 1.0 | 1.0 | 1.0 |
Total | 1.0 | 1.0 | |
Maintenance stage (n = 12) | |||
∼CPWI costs*Sustainability planning | 0.43 | 0.29 | 1.0 |
Coordinator leadership | 0.57 | 0.00 | 1.0 |
Coalition team leadership *Team cohesion | 0.57 | 0.00 | 1.0 |
Total | 0.86 | 1.0 | |
Institutionalization stage (n = 14) | |||
Coalition team leadership *∼Sustainability planning | 0.50 | 0.50 | 1.0 |
∼Coordinator leadership team*Sustainability planning | 0.40 | 0.40 | 1.0 |
Total | 0.90 | 1.0 |
Note. In Boolean algebraic notation, “∼” indicates the absence of a condition and “*” indicates the conjunction of two present conditions.
Results
Calibration
Table 1 includes descriptive statistics for the continuous version of the seven coalition functioning conditions and the percentage of coalitions reporting those conditions present based on the calibrated version used in the QCA model. After calibration, between 50% and 60% of the coalitions were scored as having the following conditions present (i.e., higher than average): coordinator leadership, team cohesion, task focus, coalition participation costs, and coalition participation benefits. Just under half of coalitions were scored as having coalition team leadership present and about a third had sustainability planning present.
After calibration of the conditions, there were eight cases out of 11 with complete data in the formation stage; 12 cases out of 15 with complete data in the maintenance stage; and 14 out of 17 with complete data in the institutionalization stage. We examined cases with and without complete data on the following variables: coalition coordinator age, race, gender, education, and time in position using chi-square and t-tests as appropriate. No significant differences were found. QCA requires complete data and therefore only those cases with complete data (N = 8 in formation; N = 12 in maintenance; N = 14 in institutionalization) are represented in the below results.
Truth Tables
In general, the truth tables generated for coalitions in each developmental stage demonstrated significant variability: few coalitions overlapped exactly in the configurations of conditions they reported. This suggests that coalitions operate in frequently differing climates and so a method such as QCA which attends to causal complexity is warranted.
Necessity Analyses
Necessity analyses revealed that only a single condition, coalition team leadership, was necessary for the attainment of model fidelity, but only during the formation stage. No other conditions at any other stage met the standard of 0.9 selected for consideration as necessary (see Table 4).
Table 4.
Formation stage | Consistency | Coverage |
---|---|---|
Coalition team leadership | 1.0 | 1.0 |
Coordinator leadership | 0.67 | 0.80 |
Team cohesion | 0.50 | 0.75 |
Task focus | 0.67 | 1.0 |
CPWI benefits | 0.83 | 0.83 |
∼CPWI costs | 0.33 | 0.67 |
Sustainability planning | 0.17 | 1.0 |
Maintenance stage | ||
Coalition team leadership | 0.71 | 0.83 |
Coordinator leadership | 0.57 | 1.0 |
Team cohesion | 0.71 | 0.71 |
Task focus | 0.71 | 0.83 |
CPWI benefits | 0.57 | 0.57 |
∼CPWI costs | 0.57 | 0.67 |
Sustainability planning | 0.57 | 0.80 |
Institutionalization stage | ||
Coalition team leadership | 0.80 | 0.80 |
Coordinator leadership | 0.40 | 0.57 |
Team cohesion | 0.40 | 0.50 |
Task focus | 0.60 | 0.67 |
CPWI benefits | 0.40 | 0.57 |
∼CPWI costs | 0.30 | 0.43 |
Sustainability planning | 0.50 | 0.63 |
Sufficiency Analyses & Model Fit
For all three stages of coalition development (formation, maintenance, and institutionalization stages), the complex and intermediate solutions were identical. Though this occurrence is not unheard of (see Cragun et al., 2016), it limits the translatability of results for a practitioner audience, and therefore we report on the parsimonious solution in Table 3 and below.
Formation Stage Results
In the formation stage, six coalitions achieved high model fidelity, with two achieving low fidelity. Coalition team leadership was the only condition sufficient to achieve high model fidelity in the formation stage (i.e., in every coalition in the formation stage where coalition team leadership was present, high model fidelity was observed; see Table 3). Coverage and consistency scores for the solution were 1.0 indicating the solution accounted for 100% of the cases that achieved high model fidelity in the formation stage.
The necessity analysis conducted for the formation stage converges with this finding, demonstrating coalition team leadership occurred in 100% of cases that realized model fidelity.
Maintenance Stage Results
In the maintenance stage, seven coalitions achieved high model fidelity, with five achieving low fidelity. Three combinations of conditions were sufficient to achieve high model fidelity for coalitions in the maintenance stage (see Table 3). The first solution, noted in the table as ∼CPWI costs*Sustainability planning, indicates that coalitions whose coordinators reported an absence of perceived costs as a result of participating in a CPWI coalition, combined with active coalition sustainability planning, achieved model fidelity in 100% of cases in which this pattern occurred. This combination accounted for 43% of cases which achieved model fidelity. The second solution was characterized by the single condition of coalition team leadership. This solution also achieved model fidelity in 100% of cases in which this pattern occurred and accounted for 57% of all cases which achieved model fidelity. The third solution for the maintenance stage, noted in Table 3 as Coordinator leadership*Team cohesion, indicates that the presence of coordinator leadership, when combined with team cohesion, achieved model fidelity in 100% of cases in which this pattern occurred and accounted for 57% of all cases which achieved model fidelity. The total coverage for this set of solutions was 0.86 indicating that they accounted for 86% of the cases that achieved high model fidelity in the maintenance stage.
Institutionalization Stage Results
In the institutionalization stage, 10 coalitions achieved high model fidelity, with four achieving low fidelity. Two combinations of conditions were sufficient to achieve high model fidelity for coalitions in the institutionalization stage (see Table 3). The first solution, noted in Table 3 as Coalition team leadership*∼Sustainability planning, indicates that coalition team leadership produced model fidelity in 100% of cases in which it occurred, if and only if sustainability planning was absent. This solution accounted for 50% of all cases that achieved model fidelity in the institutionalization stage. The second solution, noted as ∼Coordinator leadership*Sustainability planning, indicates that sustainability planning produced model fidelity in 100% of cases in which it occurred, if and only if coordinator leadership was absent. This combination of conditions accounted for 40% of all cases that achieved model fidelity in the institutionalization stage. The total coverage for this set of solutions was 0.96 indicating that they accounted for 96% of the cases that achieved high model fidelity in the institutionalization stage.
Discussion
We expected that different coalition functioning conditions would produce high model fidelity at each of the developmental stages articulated in the CCAT (Butterfoss, 2007). This was supported, with no single solution being common across stages. Though previous research had not examined the influence of coalition functioning on model fidelity as a function of developmental stage, the current findings are consistent with previous research and theory that broadly suggest both coalition functioning and developmental stage to be important elements of attaining model fidelity (Brown et al., 2012; Butterfoss, 2007). Also, these findings largely converged with the postulates of the three stages of the CCAT.
Formation Stage
Coalitions in the formation stage are engaged in recruitment of representatives from multiple sectors and development of organizational policies and processes to help facilitate coalition functioning (Kegler et al., 2010). This stage may be influenced by numerous factors, including member experience with prior interorganizational relationships and the prevailing political climate (Butterfoss et al., 2006; Butterfoss, 2007). Coalition leadership, the only condition found to be sufficient to attain model fidelity in the formation stage, included measures of political skill and knowledge, respect from community partners, and a vision for the coalition and therefore aligns well with existing literature.
It is somewhat surprising that other coalition functioning conditions were neither necessary nor sufficient for model fidelity in this stage. For example, research suggests that participant-perceived costs may exert a negative influence on coalition functioning (Chinman et al., 1996). However, this may depend on the coalition’s stage of development as suggested by Chinman et al. (1996) who notes that formation-stage participants’ shorter involvement in coalition activities may limit their exposure to participation-related costs. Extending this logic, it may be that the cumulative burden of participation costs has not yet reached a threshold required to negatively impact model fidelity. This may account for both the non-effect of participant-perceived costs in the formation stage and the positive effect that absence of those costs demonstrated in the maintenance stage.
Maintenance Stage
The current study found that coalitions in the maintenance stage achieved model fidelity in three different ways. The first was through a combination of sustainability planning and the absence of personal costs related to participation in the coalition. This is consistent with the emphasis the CCAT places on resource mobilization. Butterfoss and Kegler (2002) note another outcome of resource mobilization is that coalition members may experience elevated levels of satisfaction as they are provided with more opportunities and less burden.
Both the second and third pathways to fidelity included the presence of coordinator leadership, with the third pathway also including team cohesion. The importance of team cohesion may hearken back to Brown et al.’s (2013) note that strong relationships facilitate effective communication and decision making, both of which may increase the likelihood of achieving fidelity. This is also supported by Kegler et al. (1998) who observed that positive, effective communication and team cohesion helped to facilitate coalition delivery of programs during the implementation stage.
Institutionalization Stage
Coalitions in the institutionalization stage typically continue to be engaged in many of the same activities as the maintenance stage, however, the outcomes of interventions begin to become apparent. If executed successfully, strategies employed by the coalition may naturally disseminate and become integrated into the community (Butterfoss & Kegler, 2002).
We discovered two pathways to model fidelity in the institutionalization stage: coalition leadership in the absence of sustainability planning and sustainability planning in the absence of coordinator leadership. Included in the measure of coalition leadership is the ability to mobilize resources to aid the coalition in pursuit of its goals. It is possible that leaders’ ability to obtain resources for the coalition could compensate for lower levels of formal sustainability planning, thereby allowing the coalition to still attain model fidelity. Conversely, in the absence of leadership, it is logical that a coalition in the institutionalization stage, having experienced the implementation process and being integrated into the community, could continue to operate with fidelity if sustainability planning helped maintain basic costs.
The Importance of Leadership
Leadership proved to be the most consistently important condition in producing model fidelity, providing an avenue to high model fidelity in each developmental stage. Coordinator leadership was particularly contributive in the maintenance stage. Having a strong coalition leadership team was present in solutions in all developmental stages but was a sine qua non for high model fidelity in the formation stage.
Results from the QCA analyses aligned with previous studies of coalition leadership. Zakocs and Guckenburg (2007) for instance, found that the highest functioning coalitions exhibited a higher degree of participatory decision-making in their leadership structure; the current study found that higher levels of coalition leadership, a measure which included assessment of inclusive leadership tendencies, were more likely to achieve model fidelity. Woulfe et al. (2010) and Stokols et al. (2008) found that inclusive, vision-aligned, and/or supportive leadership styles predicted greater functioning and effectiveness; the current study found that coalition leadership, reflecting these same characteristics, was frequently necessary and/or sufficient to realize model fidelity.
One possible explanation for the importance of leadership across developmental stages could be its impact on capacity to implement programs, a major component of CPWI model fidelity. Research shows myriad effects of leadership upon coalition capacity (Kegler & Swan, 2011). For example, Zakocs and Guckenburg (2007) found that collaborative leadership (captured in the coalition team leadership scale in this study) predicts organizational and community capacity—specifically, leadership was related to presence of new skills acquired by coalition members, social capital, and sense of community—all of which support a coalition’s capacity to successfully implement programs in a community.
Limitations
This study faced several limitations. As a result of minimal extant research into the developmental needs of coalitions, this study was by default exploratory. As future studies further illuminate the influence of development on coalition functioning and model fidelity, more incisive and informative hypotheses will be cast. This study also examined only the reports of coordinators. Though coordinators are well-placed to detail the functioning and fidelity of a coalition, this lack of triangulation is a potential source of bias. For the present study, it is possible coordinators felt the need to report more positive functioning than was reality because their funders were supporting the data collection for this study. Despite assurance that their data would only be reported in summary form, they still may have felt some pressure to report higher levels of functioning and model fidelity. Some evaluations have employed board surveys (see Brown et al., 2010), which query multiple members of a coalition’s leadership. This approach affords greater angularity in assessing a coalition’s functioning and fidelity. This study’s reliance on coordinators’ perceptions may also diminish both generalization and comparability across investigations. Finally, the 7-point response scale based on Brown et al. (2010, 2012) for the model fidelity items did not have specified anchor descriptions for the even numbers and therefore it is possible that respondents had different perceptions of what it meant to be in between the odd numbered anchored items in terms of their progress.
Implications
This study illustrates several tangible steps technical assistance providers and coalition support agencies may take to increase the likelihood of achieving model fidelity. For coalitions in the formation stage, skillful and inclusive coalition leadership may provide added value. Trainings for coalition members assuming early leadership roles could couch elements of effective leadership in scenarios likely to be faced by junior coalitions. Such trainings may detail styles such as transformational, authentic, or servant leadership (Bass, 1990; George, 2003; Greenleaf, 1977).
For coalitions in the maintenance stage, technical assistance may be best targeted towards the reduction of participant-perceived costs; increasing of sustainability planning; enhancement of coordinator-specific leadership; and developing team cohesion and coalition leadership. This technical assistance could include team-building exercises, explication of self-care expectations, and/or nurturing of coordinator and board members’ competencies regarding “soft skills” such as positive communication and organization (Barrett, 2008; Hunt & Baruch, 2003).
For coalitions in the institutionalization stage, coalition leadership and sustainability planning may be prime targets for technical assistance. In different combinations, these conditions produced model fidelity 90% of the time. Similar to the formation stage, coalitions in the institutionalization stage may benefit from trainings highlighting elements of effective leadership. These trainings should incorporate scenarios likely to occur in mature coalitions, for example, member turnover, burnout, and loss of seed funding (Kegler et al., 1998). Sustainability planning technical assistance could be entwined with aspects of these trainings or could be separately accomplished.
Future Directions
Coalition-focused research remains nascent, but burgeoning. This study underscored two areas of coalition research promising fertile results: developmental differences and leadership needs. While crisp-set QCA provides intuitive, interpretable results that may be of immediate value to practitioners, in our case it required dichotomization of originally continuous variables. Use of supplementary analytic techniques, like fuzzy-set QCA which allow for varying degrees of condition and outcome (see Schneider and Wagemann, 2012), may provide more nuanced understandings of the relationships between conditions and outcomes.
Most importantly, the role of leadership in coalitions, permeating the results of the current study, should be explored more explicitly. Though leadership has frequently been measured in the context of coalition functioning and outcomes (see Kegler & Swan, 2011; see also Zakocs and Guckenburg, 2007), studies to date have been descriptive. The rich and voluminous body of broad leadership literature has not informed the testing or development of coalition-specific leadership strategies. Coalition leadership strategies should be manipulated or otherwise compared across contexts to understand the comparative influences of different leadership styles on coalition functioning and fidelity. Such an effort would provide valuable insight into the conditions optimal for coalition functioning, while simultaneously offering an avenue for coalition-supporting agencies to leverage proven effective leadership strategies.
DBHR is now a part of the Washington State Health Care Authority (HCA), but at the time of funding, DBHR was a part of the Washington State Department of Social and Health Services (DSHS).
Footnotes
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding: The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Data for the present study come from the 2017 Community Prevention and Wellness Initiative Process Evaluation, which was funded by Washington State Department of Social and Health Services, Division of Behavioral Health and Recovery. At the time of funding, the Division of Behavioral Health and Recovery (DBHR) was part of the Department of Social and Health Services. DBHR is now a part of the Washington State Health Care Authority (HCA).
ORCID iD: Brittany Rhoades Cooper https://orcid.org/0000-0002-9247-0127
References
- Anderson L. M., Adeney K. L., Shinn C., Safranek S., Buckner-Brown J., Krause L. K., (2012). Community coalition-driven interventions to reduce health disparities among racial and ethnic minority populations. Cochrane Database of Systematic Reviews 2015, 6, Article Number: CD009905. 10.1002/14651858.cd009905.pub2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Barrett D. (2008). Leadership communication. McGraw-Hill. [Google Scholar]
- Bass B. M. (1990). From transactional to transformational leadership: Learning to share the vision. Organizational Dynamics, 18(3), 19–31. 10.1016/0090-2616(90)90061-S [DOI] [Google Scholar]
- Berkel C., Mauricio A. M., Schoenfelder E., Sandler I. N. (2011). Putting the pieces together: An integrated model of program implementation. Prevention Science, 12(1), 23–33. 10.1007/s11121-010-0186-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Brown L. D., Alter T. R., Brown L. G., Corbin M. A., Flaherty-Craig C., McPhail L. G., Nevel P., Shoop K., Sterner G., III, Terndrup T. E., Weaver M. E. (2013). Rural Embedded Assistants for Community Health (REACH) Network: First-person accounts in a community-university partnership. American Journal of Community Psychology, 51(1–2), 206–216. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Brown L. D., Feinberg M. E., Greenberg M. T. (2010). Determinants of community coalition ability to support evidence-based programs. Prevention Science, 11(3), 287–297. https://doi.org/10.1007%2Fs11121-010-0173-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Brown L. D., Feinberg M. E., Greenberg M. T. (2012). Measuring coalition functioning: Refining constructs through factor analysis. Health Education & Behavior, 39(4), 486–497. https://doi.org/10.1177%2F1090198111419655 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Butterfoss F. D. (2007). Coalitions and partnerships in community health. John Wiley & Sons, Inc. [Google Scholar]
- Butterfoss F.D., Kegler M.C. (2002). Toward a comprehensive understanding of community coalitions: Moving from practice to theory. In DiClemente R. J., Crosby R. A., Kegler M. C. (Eds.), Emerging in health promotion practice and research: Strategies for improving public health (157–193). John Wiley & Sons, Inc. [Google Scholar]
- Butterfoss F.D., Kegler M.C., Francisco V.T. (2008). Mobilizing organizations for health promotion: Theories of organizational change. In Glanz K., Rimer B. K., Viswanath K. (Eds.), Health behavior and health education, 335–362. Jossey-Bass. [Google Scholar]
- Butterfoss F. D., Lachance L. L., Orians C. E. (2006). Building allies coalitions: Why formation matters. Health Promotion Practice, 7(2_suppl), 23S–33S. 10.1177/1524839906287062 [DOI] [PubMed] [Google Scholar]
- Chinman M.J., Anderson C.M., Imm P.S., Wandersman A., Goodman R.M. (1996). The perceptions of costs and benefits of high active versus low active groups in community coalitions at different stages in coalition development. Journal of Community Psychology, 24(3), 263–274. [DOI] [Google Scholar]
- Communities That Care. (2018). Programs to reduce violence, alcohol, and tobacco. https://www.communitiesthatcare.net/
- Cragun D., Pal T., Vadaparampil S. T., Baldwin J., Hampel H., DeBate R. D. (2016). Qualitative comparative analysis: A hybrid method for identifying factors associated with program effectiveness. Journal of Mixed Methods Research, 10(3), 251–272. 10.1177/1558689815572023 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Cooper B., Glaesser J., Thomson S. (2014). Schneider and Wagemann's proposed enhanced standard analysis for Ragin's qualitative comparative analysis: Some unresolved problems and some suggestions for addressing them (COMPASSS Working Paper 2014–77). http://www.compasss.org/wpseries/CooperGlaesserThomson 2014.pdf
- Division of Behavioral Health and Recovery [DBHR]. (2017). CPWI community coalition guide. Washington State Department of Social and Health Services. https://www.theathenaforum.org/cpwi-community-coalition-guide-updated-sept-2017
- Feinberg M.E., Bontempo D.E., Greenberg M.T. (2008). Predictors and level of sustainability of community prevention coalitions. American Journal of Preventive Medicine, 34(6), 495–501. 10.1016/j.amepre.2008.01.030 [DOI] [PubMed] [Google Scholar]
- George B. (2003). Authentic leadership: Rediscovering the secrets to creating lasting value. John Wiley & Sons, Inc. [Google Scholar]
- Gomez B. J., Greenberg M. T., Feinberg M. E. (2005). Sustainability of community coalitions: An evaluation of communities that care. Prevention Science, 6(3), 199–202. 10.1007/s11121-005-0003-4 [DOI] [PubMed] [Google Scholar]
- Greenberg M., Feinberg M. (2010). CTC web-based survey 2010: Technical report. Prevention Research Center, The Pennsylvania State University. [Google Scholar]
- Greenleaf R. (1977). Servant leadership: A journey into the nature of legitimate power and greatness. Paulist Press. [Google Scholar]
- Hawkins J. D., Catalano R. F. (1992). Communities that care: Action for drug abuse prevention. Jossey-Bass, Inc. [Google Scholar]
- Hawkins J.D., Oesterle S., Brown E.C., Abbott R.D., Catalano R.F. (2014). Youth problem behaviors 8 years after implementing the communities that care prevention system: A community-randomized trial. JAMA Pediatrics , 168(2), 122–129. 10.1001/jamapediatrics.2013.4009 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Hill L. G., Cooper B. R., Parker L. A. (2019). Qualitative comparative analysis: A mixed-method tool for complex implementation questions. The Journal of Primary Prevention, 40(1), 69–87. 10.1007/s10935-019-00536-5 [DOI] [PubMed] [Google Scholar]
- Hunt J. W., Baruch Y. (2003). Developing top managers: The impact of interpersonal skills training. Journal of Management Development, 22(8), 729–752. 10.1108/02621710310487882 [DOI] [Google Scholar]
- Kane H., Lewis M.A., Williams P.A., Kahwati L.C. (2014). Using qualitative comparative analysis to understand and quantify translation and implementation. Translational Behavioral Medicine, 4(2), 201–208. 10.1007/s13142-014-0251-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kegler M.C., Rigler J., Honeycutt S. (2010). How does community context influence coalitions in the formation stage? A multiple case study based on the community coalition action theory. BMC Public Health, 10(1), 90. 10.1186/1471-2458-10-90 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kegler M. C., Steckler A., Malek S. H., McLeroy K. (1998). A multiple case study of implementation in 10 local project ASSIST coalitions in North Carolina. Health Education Research, 13(2), 225–238. https://psycnet.apa.org/doi/10.1093/her/13.2.225. 10.1093/her/13.2.225 [DOI] [PubMed] [Google Scholar]
- Kegler M. C., Swan D. W. (2011). Advancing coalition theory: The effect of coalition factors on community capacity mediated by member engagement. Health Education Research, 27(4), 572–584. 10.1093/her/cyr083 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kramer J. S., Philliber S., Brindis C. D., Kamin S. L., Chadwick A. E., Revels M. L., Chervin D. D., Driscoll A., Bartelli D., Wike R. S., Peterson S. A., Schmidt C. K., Teresa Valderr L. (2005). Coalition models: Lessons learned from the CDC’s community coalition partnership programs for the prevention of teen pregnancy. Journal of Adolescent Health, 37(3), S20–S30. 10.1016/j.jadohealth.2005.05.010 [DOI] [PubMed] [Google Scholar]
- Kuklinski M.R., Fagan A.A., Hawkins J.D., Briney J.S., Catalano R.F. (2015). Benefit–cost analysis of a randomized evaluation of Communities That care: Monetizing intervention effects on the initiation of delinquency and substance use through grade 12. Journal of Experimental Criminology, 11(2), 165–192. 10.1007/s11292-014-9226-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mackie J. L. (1980). The cement of the universe: A study of causation. Oxford University Press. [Google Scholar]
- Miller T., Hendrie D. (2008). Substance abuse prevention dollars and cents: A cost-benefit analysis (DHHS Pub, (07-4298)). Center for Substance Abuse Prevention (CSAP), SAMHSA.
- Ragin C. (2008). What is qualitative comparative analysis (QCA)? http://eprints.ncrm.ac.uk/250/1/What_is_QCA.pdf
- Ragin C. (2009). Qualitative comparative analysis using fuzzy sets (fsqca). In Configurational comparative methods: Qualitative comparative analysis (QCA) and related techniques (Vol. 51, pp. 87–122). Sage Publications, Inc., https://doi.org/10.4135/9781452226569. [Google Scholar]
- Ragin C.C., Davey S. (2016). Fuzzy-set/qualitative comparative analysis 3.0. Department of Sociology, University of California. [Google Scholar]
- Ragin C.C., Sonnett J. (2005) Between complexity and parsimony: Limited diversity, counterfactual cases, and comparative analysis. In: Kropp S., Minkenberg M. (eds) Vergleichen in der politikwissenschaft. VS verlag für sozialwissenschaften (pp. 180–197). VS Verlag für Sozialwissenschaften. https://doi.org/10.1007/978-3-322-80441-9_9. [Google Scholar]
- Rhoades B. L., Bumbarger B. K., Moore J. E. (2012). The role of a state-level prevention support system in promoting high-quality implementation and sustainability of evidence-based programs. American Journal of Community Psychology , 50(3), 386–401. 10.1007/s10464-012-9502-1 [DOI] [PubMed] [Google Scholar]
- Rohlfing I. (2020). The choice between crisp and fuzzy sets in qualitative comparative analysis and the ambiguous consequences for finding consistent set relations. Field Methods, 32(1), 75–88. 10.1177/1525822X19896258 [DOI] [Google Scholar]
- Schneider C.Q., Wagemann C. (2010). Standards of good practice in qualitative comparative analysis (QCA) and fuzzy-sets. Comparative Sociology, 9(3), 397–418. 10.1163/156913210X12493538729793 [DOI] [Google Scholar]
- Schneider C. Q., Wagemann C. (2012). Set-theoretic methods for the social sciences: A guide to qualitative comparative analysis. Cambridge University Press. [Google Scholar]
- Shrestha G., Cooper B.R., Hill L.G. (2019). Community prevention and wellness initiative: Impact over time analysis report. Washington State Department of Social and Health Services. [Google Scholar]
- Spoth R., Redmond C., Shin C., Greenberg M., Feinberg M., Schainker L. (2013). PROSPER community-university partnership delivery system effects on substance misuse through 6½ years past baseline from a cluster randomized controlled intervention trial. Preventive Medicine, 56(0), 190–196. http://doi.org/10.1016/j.ypmed.2012.12.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stokols D., Misra S., Moser R. P., Hall K. L., Taylor B. K. (2008). The ecology of team science: Understanding contextual influences on transdisciplinary collaboration. American Journal of Preventive Medicine, 35(2), S96–S115. 10.1016/j.amepre.2008.05.003 [DOI] [PubMed] [Google Scholar]
- Thiem A. (2017). Conducting configurational comparative research with qualitative comparative analysis: A hands-on tutorial for applied evaluation scholars and practitioners. American Journal of Evaluation , 38(3), 420–433. https://doi.org/10.1177%2F1098214016673902 [Google Scholar]
- Washington State Health Care Authority. (2021). CPWI community coalition guide. Washington State: Washington State Health Care Authority, Division of Behavioral Health and Recovery. https://theathenaforum.org/cpwi-community-coalition-guide
- Woulfe J., Oliver T.R., Siemering K.Q., Zahner S.J. (2010). Multisector partnerships in population health improvement. Preventing Chronic Disease, 7(6), A119. [PMC free article] [PubMed] [Google Scholar]
- Zakocs R.C., Guckenburg S. (2007). What coalition factors foster community capacity? Lessons learned from the Fighting Back Initiative. Health Education & Behavior, 34(2), 354–375. https://doi.org/10.1177%2F1090198106288492 [DOI] [PubMed] [Google Scholar]