Modeling the sustainability of community health networks: novel approaches for analyzing collaborative organization partnerships across time

ABSTRACT

Sustainability is important if community health organizations are to be effective in collaborating to achieve long term health goals. We present a multi-method set of longitudinal analyses examining structural markers applied to a group of organizations brought together to reduce cancer disparities among older African American adults. At the overall network level, sustainability was seen in the growth of outgoing connections and multiplexity. Results of hierarchical clustering analyses identified distinct patterns of collaborative activation over time at the relationship level. Growth modeling indicated the effects of continuing network membership and participation in collaborative events on several structural markers of sustainability. Results of these analyses provide longitudinal indicators for how collaborations among partner organizations impacted their likelihood of continuing in the community network program. The strategy presented here introduces novel methods to assist with planning and evaluation of future community based public health endeavors.

Establishing and mobilizing collaborative partnerships (i.e., networks) among community health organizations that are brought together to achieve mutual goals is essential to broadening the impact of public health initiatives [1–4]. In practice, however, sustaining such collaborative networks can be challenging, particularly given competing demands and constraints on the time and resources of the members of the various organizations in the network [5–8]. Given the potential risk of investment loss if collaborative network partnerships dissolve before goals can be achieved, researchers are calling for more in-depth studies of “network sustainability” [9, 10]. That is, researchers want to know what is necessary for networks of community organizations to continue working together over time.

One problem with conducting such studies has been that conceptual and operational definitions of network sustainability are somewhat inconsistent in the literature. Various conceptual definitions include the continuation of program benefits, institutionalization/routinization of the network and network programs, building of community capacity, and continuation of network programs following termination of external funding [6, 11–16]. Perhaps most useful in considering a theoretical and operationalizable conceptualization of sustainability are Israel et al.’s [6] criteria of what is necessary for sustainable networks—specifically, “(1) sustaining relationships and commitments among the partners involved, (2) sustaining the knowledge, capacity and values generated from the partnership, and (3) sustaining funding, staff, programs, policy changes and the partnership itself”. In the case we present here, our conceptualization of sustainability is exemplified by continuing partner relationships in an ongoing community health network. We are specifically interested in examining the association between aspects of collaborative activity and sustainability.

Some evidence demonstrates that the collaborative efforts of partners in a community network are related to a network’s success in goal attainment (e.g., [17–19]). However, no convincing research has related such collaborative efforts to measures of sustainability. A previous study by O'Loughlin and colleagues [20] indicated that a measure of perceived network collaboration was not related to perceived sustainability of a community based health promotion intervention; however, no objective measures of sustainability were assessed. A separate review of successful strategies for community health promotion [21] suggested that collaborations among stakeholders, particularly in planning stages of network activities, are essential for sustainment of network efforts. Our study will empirically test the proposition that increased collaboration among network partners is associated with sustainability.

Measuring sustainability within a community network necessarily implies the passage of time [22]. Analyzing the relationship between markers of collaborative activity and ensuing sustainability requires methods that demonstrate associations between whether and how network partners interact over time, and whether partners continue in the network. However, unless researchers examine interorganizational collaborations at multiple levels of analysis [2] the results are likely to be somewhat simplistic, uninformative, or at worst, misleading. A multilevel network consists of dyadic relationships nested among partners who are nested within the network. Therefore, analyses of the association between collaboration and sustainability should include (1) the macro network level (i.e., analyses of the extent to which the entire network is characterized by collaborations among partner organizations over time), (2) the relationship level (i.e., analyses of the extent to which partner organization dyads interact and collaborate with each other, and variations in that dyadic collaboration over time), and (3) the network partner level (i.e., analyses of whether partners who remained members of the network had similar, detectable patterns of collaborative engagement over time).

Social network analysis is a method for assessing collaborative overlapping relationships at multiple levels [7, 23–25] and has been useful for assessing and evaluating community mobilization and partnership efforts [7, 25–29]. However, the process of collecting such data longitudinally are formidable for reasons such as time, cost [2], staffing needs, and partner availability. Nonetheless, more longitudinal data and commensurate modeling techniques for detecting underlying patterns are needed to better understand how community collaborations can be sustained across years and beyond initial funding periods. Using innovative modeling techniques to detect underlying patterns of collaborative activity with longitudinal community network data was our goal.

We used a unique longitudinal dataset collected as part of the Detroit Community Network Program (CNP) to examine sustainability over time. The Detroit CNP was a multi-year effort funded by the National Cancer Institute (NCI; U01CA114583) (e.g., [30]) to build partnerships and collaboration among community-based organizations in Detroit to reduce cancer disparities among older African American adults residing in the metropolitan area. The goals were to develop and implement education, training and pilot research efforts to address the cancer burden disproportionately affecting the population. The Detroit metropolitan area faces its own unique challenges as one of the most impoverished urban areas in the USA [31]. Although community health partnerships have developed in Detroit and many other urban areas in the USA, they can be relatively short-lived because of uncertain revenue and competition for scarce resources (e.g., [10, 32]).

The Detroit CNP is useful for studying the sustainability process because most of the organizations continued participating throughout the grant period (5 years) and a number of them have transitioned to a next phase of collaboration (i.e., a second 5-year NCI grant (U54CA153606) to support the community network). Those partners that continued to collaborate into the next phase were those that contributed to the sustainability of the network. We expected that collaborative activities across the years served to strengthen the relationships among network partners; hence we sought to identify markers of collaboration within the network that were associated with those continuing organizations that contributed to network sustainability. We also expected that similarities in patterns of within-dyad collaborative breadth (i.e., the number of different types of collaborative activities occurring between two partner organizations) over time would indicate when, among whom, and why collaborative activities expanded and contracted as the relationships evolved. We also examined whether organizations that continued their involvement in the Detroit CNP had differing 5-year trajectories (i.e., growth or change over time) for markers of collaboration compared with those that dissolved their involvement with the Detroit CNP. In short, we expected more positive markers of collaboration among partners that continued in the network and contributed to its sustainability.

METHODS

Study participants

We recruited three types of community partner organizations to join the Detroit CNP: (1) those serving older adults (n = 9); (2) those promoting general public health (n = 11); and (3) those targeting cancer prevention, control, clinical treatment, and/or survivorship (n = 8). Three additional partners included a major religious institution, a pharmaceutical company, and a university. One individual (usually a director or other senior leader) represented each organization in Detroit CNP activities across the time period. Each organization received a $250 honorarium annually if the representative participated in that year’s retreat.

The Detroit CNP partner network began in 2005 and continued until 2009. Self-reports of the extent of collaborative activity (see below) were collected annually starting in 2005 (Baseline/Year 1) and continued until 2009 (Years 2–5). The Detroit CNP began with 27 partner organizations at Year 1 and increased to 31 partners by Year 5. Partner organizations that continued their Detroit CNP participation following the end of the first 5-year funding period (i.e., remained with the network for the second 5-year grant funded period) were identified as “Continuing Partner Organizations” (CPOs); those that did not continue their participation were identified as “Non-Continuing Partner Organizations” (Non-CPOs; see Table 1). Network analysis, hierarchical cluster analysis and growth modeling were conducted to analyze the sustainability of relationships between and among Detroit CNP organizations across the 5-year time period.

Table 1.

CPO and Non-CPO partner organizations’ mean network scores over time

	Number	Output degree centrality	Input degree centrality	Reciprocated input	Mean multiplexity
2005
Non-CPO	12	0.29	0.21	0.06	0.49
CPO	15	0.20	0.26	0.07	0.37
2006
Non-CPO	14	0.12	.21	0.16	0.21
CPO	15	0.37	0.29	0.50	0.72
2007
Non-CPO	15	0.13	0.20	0.21	0.26
CPO	15	0.35	0.29	0.50	0.56
2008
Non-CPO	15	0.12	0.16	0.14	0.24
CPO	16	0.33	0.28	0.45	0.69
2009
Non-CPO	15	0.00	0.20	0.00	0.00
CPO	16	0.44	0.25	0.56	0.70

Numbers represent group mean centralities for the whole network (and not degree centralizations for between group sub-networks)

Annual assessments

At Year 1 (Baseline), all representatives completed a standard network analysis questionnaire [33, 34] containing a matrix of all Detroit CNP organizations (Fig. 1). Each representative reported (1) whether his/her organization had any collaborations (i.e., outgoing connections) with each of the other partners, and (2) how many different types (work-related, cancer-related, and/or other) of connections to each other partner there were. We measured relationship multiplexity [35–37] as the number of unique types of collaborations each partner reported for a relationship with another partner. Multiplexity scores at baseline ranged from 0 (no collaboration) to 3 for each type of connection or link that was reported.

Fig 1.

Example of collaboration matrix completed by network partners

For Years 2–5, project staff used an enhanced network matrix and personally interviewed the official senior representative from each Detroit CNP partner organization about their CNP collaborations with each of the other partner organizations. Specifically, they identified how many of four types (general information exchange/networking, education/training, research, and/or program development) of collaborations occurred. Scores on the multiplexity level of each relationship ranged from zero to four annually for Years 2–5.

Analyses

Network analysis of structural parameters

Using the PAJEK 2.05 software [38, 39] we produced sociograms as visual representations of all relationships reported for each year. We also analyzed the following network level parameters [33, 34, 40] as network level markers of collaboration: network density indexed the reported connections between all member organizations as a proportion of all possible connections in the network; network output degree centralization indexed the total number of outgoing connections across all network members as a proportion of all possible outgoing connections; and network input degree centralization was the corresponding measure of all reported incoming connections as a proportion of all possible incoming connections for the whole network.

We aggregated scores across partners in the network to create two additional network-level parameters. For each partner, we calculated the Reciprocated input proportion, which was the proportion of incoming ties that were reciprocated (i.e., also independently reported as outgoing by the reporting partner). For the whole network, Network reciprocated input proportion was defined as the mean of all the partners’ reciprocated input proportion. We also created a partner multiplexity score as the mean of the relationship multiplexity scores for each partner, from which we then created a mean network multiplexity score as the mean of the partners’ multiplexity scores. In sum, the macro level markers of collaboration were network density, network output degree centralization, network input degree centralization, network reciprocated input proportion and mean network multiplexity.

Hierarchical cluster analyses of relationship level parameters

We indexed the extent of collaborative activity in each relationship reported between any two organizations in the Detroit CNP with the relationship multiplexity score (see above). During the annual interviews, organization representatives reported the collaborations occurring in each of the partnerships they reported for that year. This resulted in a longitudinal multiplexity pattern (array) for each partner dyad. For example, if Organization A reported relationships with Organization B, the multiplexity scores would be arrayed as five data points across the 5 years (e.g., “2, 0, 3, 3, 4”). Before submitting the data to hierarchical clustering algorithms, we standardized the multiplexity scores within each year. Each data point, then, within each longitudinal multiplexity pattern represented the extent to which a reported relationship was less multiplex (meaning little or no collaboration) or more multiplex (meaning more collaboration) in standard deviation units, relative to the mean of all multiplexity scores for that year. Thus, each longitudinal multiplexity pattern indicated annual fluctuations in the breadth of collaborative activity in each relationship across time.

We submitted the total number of longitudinal multiplex patterns reported across the five years (n = 930) to Cluster 3.0 [41] to conduct hierarchical clustering analysis. Hierarchical clustering analysis is a data mining technique for large datasets used to assign similar objects to hierarchical clusters (i.e., groups). This procedure enabled us to identify and group longitudinal multiplex relationships that had similar array patterns. In essence, it results in arrays being grouped according to the similarity of their patterns (i.e., the similarity of their relationship multiplexity over time). Within each group, consecutive hierarchical levels further identified patterned array that are more similar and grouped them accordingly. This clustering procedure continues at each sub-ordinate group level until no further clustering is plausible [42]. Thus, we can consider the first level of similarly grouped patterns a primary subordinate level, with consecutive levels of grouping contained within each primary group. Clustered data can then be viewed with “heat maps”: different data values are presented in differing colors to facilitate pattern recognition. (Similar methods are also used, for example, to inductively examine clustering patterns in gene activation in DNA micro-array data [43, 44]). We used Treeview [45, 46] to visualize the resulting clusters within the heat maps [47].

Growth modeling of partner level parameters

As a final step, we examined whether partner organizations maintained collaborative connections with other CNP partner organizations over time. We accomplished this by extracting partner level parameters [33, 34, 40] for each year. The extent to which a network partner reported connections to other partners was measured by the (1) partner output degree centrality, i.e., the number of reported outgoing connections reported by each partner as a proportion of all possible outgoing connections to all partners; (2) partner input degree centrality, i.e., the proportion of incoming connections to that partner reported by other partners out of all possible incoming connections that could be reported; (3) partner reciprocated input proportion; and (4) partner multiplexity (both which were described above in “Network analysis of structural parameters”).

We used Hierarchical Linear Modeling (HLM 6.09) [48, 49] to fit growth models to examine how the partners’ network parameters changed over time and to examine differences in the trajectories of the network parameters [50, 51]. Preliminary examination of the trajectories suggested that the data would best fit quadratic growth models. Given the lack of independence in scores for the network parameters, we did not use model coefficients to make statistical inferences about corresponding population parameters. Rather, we modeled the mean trajectory coefficients and residual variance around the trajectories (i.e., level-1 error), and descriptively examined the model coefficients and between-group differences in coefficients.

The within-organization (i.e., level-1) longitudinal models took the following general form: Y = β₀ + β₁ time + β₂ time² + r, where Y = value for the network parameter, and time = the coded variable identifying the corresponding year for the outcome. Time was centered at the 5th year; hence, the intercept, β₀, is interpreted as the mean for the outcome in Year 5. β₁ and β₂ are, respectively, the linear and quadratic coefficients that define the shape of growth trajectories, and r represents the error term (i.e., deviation from the predicated value for the network parameters at each time point). Separate models were specified for each network parameter.

We modeled CPO (i.e., CPO) status and Non-CPO status at the between-organization level (i.e., level 2) of the model. The between-organization level of the models took the following forms: β₀ = γ₀₀ + γ₀₁ CPO status; β₁ = γ₁₀ + γ₁₁ CPO status; β₂ = γ₂₀ + γ₂₁ CPO status, where the intercept in each equation represented the value for Non-CPO status and the coefficient for CPO status in each equation represented the additive effect of CPO status on the coefficient.

RESULTS

Network-level analyses

Sociograms representing the reciprocated connections between and among Detroit CNP organizations for each year are in Fig. 2. Reciprocated dyadic links are presented in the figure given that these were the connections reported independently by both organizations in the relationship, and hence likely to be more reliable; however, we present network statistics based on all reported links. Network density and input degree centralization showed relative stability over time, ranging from 0.22 to 0.25. Network reciprocated input proportion ranged from 0.29 to 0.35 over time. Output degree centralization varied in a curvilinear fashion: 0.59 (2005), 0.50 (2007), 0.39 (2008), 0.78 (2006), and 0.70 (2009). Mean network multiplexity declined from highs of 0.42 (2005) and 0.47 (2006) to a low of 0.36 (2009).

Fig 2.

CNP over time. Darker, thicker lines, increased multiplexity; black-filled circles, CPOs; and gray-filled circles, Non-CPOs

We examined the network level parameters separately for CPOs and Non-CPOs. Given that we wanted the parameters to reflect collaboration across the entire network, we extracted the partner level parameters from the whole network and calculated means separately for CPOs and Non-CPOs. Mean scores for markers of collaboration for CPOs and non-CPOs are in Table 1. On average, CPOs and Non-CPOs had similar scores for all markers in 2005, but those parameter scores diverged in each of the subsequent 4 years. We model those divergences in greater detail below (see “Partner level analyses”).

Relationship level analyses

We used hierarchical cluster analysis to analyze the 930 relationship multiplexity scores arrayed across the 5 years; each pattern reflected the reported multiplexity over time for that relationship. As noted above, multiplexity scores were standardized within year so that positive multiplexity scores represented relationships with relatively more types of collaborations and negative scores represented relationships with relatively few or no types of collaborations. Given that hierarchical clustering analysis is an inductive data mining procedure, there are no model statistics that indicate precisely which clusters, or which level of clustering, should be interpreted as meaningful. Rather, it is for the researcher to examine a representation of the results (i.e., heat maps) to interpret the resulting clusters. As there were identifiable patterns that were concordant with network partner activities at the first level (i.e., the primary subordinate grouping) of the hierarchical clusters, we chose to examine the first level in our interpretation of the resulting groups.

A five-cluster solution emerged as shown in the heat map (Fig. 3). The first cluster (A) was composed of relationships with above average multiplexity scores in 2006—the cluster also contained some relationship with above average multiplexity in 2005, but there were fewer of these. Varying relationship multiplexity patterns followed in 2007 to 2009. The second cluster (B) was comprised of relationships with above average multiplexity scores mostly occurring in 2007. The third (C) and fourth (D) clusters showed above average multiplex relationships most occurring in 2008 and 2009. The fifth cluster (E) was comprised of relationships that had consistently below average multiplexity across years, indicating minimal or no collaborative relationships existing between those partner organization dyads. This between-dyad variation in the patterns of relationship multiplexity suggest that the number of types of relationships that particular partners shared expanded and contracted depending on events that took place in particular years. We explore this proposition with ancillary analyses following the main partner-level analyses below.

Fig 3.

Cluster analysis heat map illustrating five cluster solutions for activation of relationship multiplexity over time. Gray areas indicate no data for that year. Array examples: rows 1, array heat map/type of reporting partner organization − type of nominated partner organization; 2, years; 3, standardized multiplexity score within year. (Age aging services, Oth other; Can cancer)

Partner level analyses

We used growth models to examine how the trajectories of partner-level network parameters differed between CPOs and Non-CPOs. The effect of CPO status (i.e., the effect of continued participation in the Detroit CNP) is shown in the differences in model coefficients (Table 2) and represented graphically in Fig. 4. CPOs had significantly more positive growth trajectories for markers of collaboration than did Non-CPOs. Notably, both CPOs and Non-CPOs began participating in the Detroit CNP with approximately the same levels of network activity in the first year (2005), but quickly diverged in subsequent years, as demonstrated by the markers of output centrality, reciprocated input proportion and multiplexity. CPOs reported increases in outgoing connections, reciprocated connections and multiplexity, whereas Non-CPOs showed decreases in all those markers of collaboration over the same time. This divergence suggests that there is, in fact, an association between markers of collaboration and sustainability: Collaborative activities increased over the course of the 5 years among the network partners that met the criteria for sustainability (i.e., those who continued to the second grant-funded instantiation of the Detroit CNP), and collaborative activities decreased for the partners who did not meet that criteria. Note that we do not have evidence to suggest that increased collaborative engagement caused CPOs to remain in the network, as we may not rule out structural or institutional factors that precluded sustainment among non-CPOs. Importantly, however, given that CPOs and non-CPOs are defined by attributes occurring after the collaborative data were collected, we can reasonably infer that sustainment did not cause collaborative engagement.

Table 2.

Growth model coefficients for network degree centralities, reciprocated input, and mean multiplexity with CPO moderator

Fixed effects	Output degree centrality		Input degree centrality		Reciprocated input		Mean multiplexity
	γ	CoefE	γ	CoefE	γ	CoefE	γ	CoefE
Intercept
Non-CPO	0.03	0.03	0.19	0.19	0.03	0.03	0.05	0.05
CPO effect	0.38	0.40	0.05	0.25	0.49	0.52	0.63	0.68
Linear coefficient
Non-CPO	−0.04	−0.04	0.01	0.01	−0.08	−0.08	−0.11	−0.11
CPO effect	0.05	0.01	−0.06	−0.04	0.05	−0.03	0.06	−0.05
Quadratic coefficient
Non- CPO	0.00	0.00	0.01	0.01	0.00	0.00	0.00	0.00
CPO effect	−0.01	−0.01	−0.01	−0.01	−0.02	−0.02	−0.02	−0.03

γ growth model coefficient, CoefE coefficient estimate

Fig 4.

CPO effect on growth trajectories for output degree centrality, input degree centrality, reciprocated input proportion, and multiplexity

Ancillary partner level analyses: collaborative events

The results of the relationship-level hierarchical cluster analysis suggest increased collaborative activity among selected partners in 2007 and 2008—2 years in which specific collaborative events (CEs) took place. Thus, we conducted ancillary analyses to examine whether there were additional effects of participation in those separate CEs, over and above the effects of CPO status, on the growth trajectories for markers of collaboration. Specifically, we examined the simultaneous effects of CPO status and the 2007 CE by including a dummy code to identify 2007 CE involvement (coded 1 for involvement) and an interaction term to model non-additive effects of CPO status and the 2007 CE on each level-2 equation in the latent growth models. We used the same procedure for analyzing the effect of the 2008 CE.

The model coefficients and group coefficient estimates for the 2007 CE and 2008 CE are in Tables 3 and 4; the resulting trajectories are shown in Figs. 5 and 6. Generally, the partners that remained involved for all 5 years (i.e., the CPOs) and also participated in either of the CEs tended to have greater markers of sustainability. They typically outperformed partners who were CPOs but were not involved in the CEs. Partners who were involved in the CEs but who were also Non-CPOs had trajectories that spiked at the time of the CE and declined rapidly thereafter. Altogether, this evidence is intuitively appealing as it demonstrates that CEs provide opportunities for increased collaboration among network partners. However, and perhaps most importantly, this evidence indicates that event-centered examinations of partner collaborations might lead to erroneous conclusions about who in the network contribute to its sustainability—some partners had greater levels of collaborative engagement at the time of the event, but little engagement before and after the event.

Table 3.

Effects of CPO and 2007 CE on growth coefficients

Fixed effect	Output degree centrality		Input degree centrality		Reciprocated input		Mean multiplexity
	γ	CoefE	γ	CoefE	γ	CoefE	γ	CoefE
Intercept
Non-CPO	0.02	0.02	0.19	0.19	0.02	0.02	0.04	0.04
CPO effect	0.31	0.33	0.06	0.26	0.44	0.46	0.51	0.55
2007CE effect	0.04	0.06	−0.01	0.19	0.02	0.05	0.07	0.11
CPO*2007CE	0.18	0.56	−0.03	0.22	0.15	0.63	0.35	0.97
Linear coefficients
Non-CPO	−0.01	−0.01	0.01	0.01	−0.06	−0.06	−0.01	−0.01
CPO effect	−0.01	−0.03	−0.01	0.00	0.00	−0.07	−0.14	−0.15
2007CE effect	−0.12	−0.13	0.02	0.03	−0.09	−0.15	−0.45	−0.47
CPO*2007CE	0.22	0.07	−0.17	−0.15	0.18	0.03	0.80	0.20
Quadratic coefficients
Non-CPO	0.01	0.01	0.00	0.00	0.00	0.00	0.02	0.02
CPO effect	−0.02	−0.02	0.00	0.00	−0.03	−0.03	−0.07	−0.05
2007CE effect	−0.02	−0.01	0.00	0.01	−0.01	−0.01	−0.11	−0.09
CPO*2007CE	0.04	0.00	−0.04	−0.04	0.03	−0.01	0.19	0.03

γ growth model coefficient, CoefE coefficient estimate

Table 4.

Effects of CPO and 2008 CE on growth coefficients

Fixed effect	Output degree centrality		Input degree centrality		Reciprocated input		Mean multiplexity
	γ	CoefE	γ	CoefE	γ	CoefE	γ	CoefE
Intercept
Non-CPO	0.02	0.02	0.20	0.20	0.03	0.03	0.04	0.04
CPO effect	0.38	0.40	0.02	0.22	0.48	0.51	0.61	0.65
2008CE effect	−0.01	0.02	−0.18	0.03	−0.03	0.00	−0.01	0.03
CPO*2008CE	0.02	0.41	0.30	0.35	0.05	0.53	0.16	0.80
Linear coefficients
Non-CPO	−0.01	−0.01	0.01	0.01	−0.05	−0.05	−0.06	−0.06
CPO effect	0.06	0.05	−0.07	−0.06	0.05	−0.01	0.11	0.05
2008CE effect	−0.65	−0.65	0.01	0.02	−1.45	−1.50	−1.23	−1.29
CPO*2008CE	0.43	−0.17	0.07	0.03	1.31	−0.15	0.72	−0.45
Quadratic coefficients
Non-CPO	0.01	0.01	0.00	0.00	0.00	0.00	0.01	0.01
CPO effect	−0.01	0.00	−0.02	−0.01	−0.02	−0.02	−0.01	0.00
2008CE effect	−0.19	−0.18	0.01	0.01	−0.50	−0.50	−0.40	−0.39
CPO*2008CE	0.14	−0.05	0.01	0.01	0.47	−0.05	0.27	−0.14

γ growth model coefficient, CoefE coefficient estimate

Fig 5.

Simultaneous effects of CPO and 2007 CE involvement on growth trajectories for output and input degree centralities, reciprocated input, and multiplexity

Fig 6.

Simultaneous effects of CPO and 2008 CE involvement on growth trajectories for output and input degree centralities, reciprocated input, and multiplexity

DISCUSSION

This study used longitudinal network data collected over 5 years to examine which markers of collaboration are associated with network sustainability [2]. The network included mostly nonprofit organizations recruited to join an effort to reduce the disproportionate cancer burden affecting older, underserved African American adults in the Detroit metropolitan area. The network program’s goal was to create partnerships that could foster community outreach, education, training and provide opportunities for pilot research projects. Minimal financial incentives were available to assist the organizations and, under the rules of the grant, no clinical care (i.e., cancer screening services) could be offered to the older adults who were the constituents of many of the organizations. Despite these challenges, public service, training, educational events, and pilot grant initiatives provided opportunities for network partners to collaborate with each other, thus influencing whether the partners remained supportive of, and committed to, the Detroit CNP as it sought an extension of grant funding.

Modeling the relations between collaborative engagements and sustainability presented an additional challenge. We propose that the novel mix of methods described here represents an extremely useful strategy in response to the challenge. A network analysis provides an examination of structural parameters—i.e., parameters that provide information about the nature of ties among partners who comprise the network—that can be seen as markers of collaboration. These parameters can exist at different levels of analysis. With all the collaborative relationships that can be reported by organization members across multiple years, hierarchical cluster analysis provides a way to mine the large amount of patterned relationship data at the level of the individual dyads. Groups of dyads that have similar patterns of fluctuation in relationship breadth (i.e., multiplexity) across the 5 years give insight into how and when organizational pairs decided to increase or decrease the extent to which they collaborated. The hierarchical clustering analysis provided information on fluctuations in this multiplexity marker that we may not have observed if the dyadic relationships were aggregated by partners. At the partner level, growth modeling then showed to what extent the changes in collaborations over time were associated with partner organizations that contributed to network sustainability.

So what did we learn using this triangulated approach to understand what predicted sustainability in the Detroit CNP? First, the results showed some fluctuations in markers of collaboration at the macro network level. Thus, what was most illuminating was the finding that those markers revealed structural differences between those organizations that continued their partnership efforts and those that did not. These differences are marked by the extent to which organizations report outgoing relationships to other network organizations (i.e., output degree centrality), the extent to which relationships with other organizations are reciprocated (i.e., reciprocated input proportion) and collaborative (i.e., multiplexity).

The nature of collaborations within a community network is further clarified at the relationship level. Voluntary community partnerships can be somewhat fragile, with relationships among representatives based more on meeting attendance and minimal information sharing as opposed to meaningful collaborative activity. For us, then, relationship multiplexity was an important network parameter because it quantified the breadth of collaboration—we were able to examine a dimension of collaboration within the dyadic relationships that indicated efforts beyond minimal collaborative engagement. In using hierarchical cluster analysis to further examine the multiplex ties, we were able to better capture relational pattern variation across the 5 years. Given the significance of overlapping dyadic relationships which comprise networks [52, 53], the cluster analysis and the differing patterns evident on the heat map (Fig. 3) readily revealed when and within which relationships these dyad-level multiplex ties were most active across time. The results highlighted the facilitative effect of CEs on network engagement. We refrained from making inferences about particular network partners based solely on the results of the cluster analyses (in that any partner may be represented in multiple relationship clusters); however, we utilized insight gained from these analyses to examine network partners’ involvement with CEs.

The utility of the cluster analysis could extend beyond what we report here. For example, while beyond the scope of our analysis, the types of relationships in each cluster could be examined more closely to see the extent to which they are intersectoral and could be leveraged for other intervention or dissemination efforts. Future analysis could also explore the nature of the sub- clusters within each cluster for even more in-depth relational analysis.

The associations between collaborative activity over time and sustainability were further corroborated by the growth models (Table 2; Fig. 4). Interestingly, continuing and non-continuing partners essentially began at similar structural points in 2005 (i.e., similar levels of outgoing, incoming, reciprocated and multiplex ties). However, they diverged quickly on three of the four parameters—CPOs consistently had better performing markers of collaboration for output, reciprocated and multiplex ties and maintained a slightly higher level of incoming relationships over time. We do not know why some network partners decreased collaborative engagement so early and, eventually, halted their measureable network involvement. However, the rapid divergence of the trajectories highlights the fact that network assessments gathered at one time point, particularly early in the network, may present an inaccurate picture of which partners in the network will contribute to sustainability. The ability to discern differences in markers of collaboration between those partners that met criteria for sustainability and those that did not provides important data that could be used to track the potential success of a network over time. In our case, we were able to identify the 12 organizations that transitioned into a subsequent NCI grant which expanded the Detroit CNP into the Southeast Michigan Partners Against Cancer with support continuing until 2015, and we were retrospectively able to examine how those organizations collaborated leading up to the transition. Applications of these methods could examine network collaborations in real time, using objective metrics to track the collaborations, thereby potentially identifying partners who not only may benefit the most from resource allocation, but who may benefit the network most by contributing to its sustainability.

A final analytical strategy was to assess whether there was direct structural evidence that certain CNP driven events could affect markers of collaboration and thus be important for sustainability. Certainly, the potential for sustainability was bolstered because Detroit CNP partners collaborated on several types of projects to address cancer disparities across the 5 years. These included large and small educational/training events—for example, the largest educational event was a public symposium in 2007 on the cancer burden experienced by older underserved African American adults (“Facing Cancer with Faith, Hope, and Knowledge”), held at one of Detroit’s churches. Because the CNP was predicated on a community-based participatory research framework, partners also participated in developing and implementing pilot research projects with program investigators and postdoctoral trainees. In early 2008, several partners collectively developed an application to the Detroit Komen Race for the Cure Foundation for funds to expand screening services for low-income women. The effort was successful and, for a year, enabled the partner organizations to provide breast cancer screening for low-income women who were disabled and/or homebound. Given the direct relevance of each of these CEs to the mission of the Detroit CNP, we examined their moderating effects on the trajectories of the markers of collaboration. The results (Figs. 4 and 5) demonstrate yet another level of understanding relevant to sustainability. Clearly, CPO status and participation in each CE moderated the trajectories for collaborative network activity. Neither of the CEs, by itself, resulted in relatively greater network activity over time, and though there was more collaborative activity over time among partners who were CPOs but who did not participate in the events, it was when CPO status and CE involvement were combined that there was more collaborative engagement over time. Importantly, note that in the case of the 2007 event, three of the four markers indicated that the increased collaborative engagement was maintained even after the event was completed (Fig. 5). Involvement with the event led participating partners to be consistently more engaged with the network.

Limitations

This study is limited in three ways. First, we did not seek to test hypotheses; rather, we explored the utility of a novel mixed method strategy to assess the multifaceted and complex nature of community collaboration and its associations with sustainability. Given the number of organizations and the interdependence of the network parameters, we were unable to assess statistical significance with the methods we used. Though the results are not generalizable, they do have heuristic value and contribute some empirical evidence for guiding the choice of methods that may be used to evaluate collaborations within, and sustainability of community health coalitions. A second limitation is that the network data were collected as self-reports from a single representative of each organization. These individuals were in leadership positions and hence likely knowledgeable about collaborative activity. However, they may have been unaware, in error or disinclined to report that some contact had or had not occurred. (This limitation is similar to the one also described by Luque and colleagues in their analysis of the Tampa Bay Community Cancer Center [26].) A final limitation is that the multiplexity measure used for data collection in the first year was based on a 0–3 scale with different scale values than the 0–4 scale used during the subsequent 4 years. Concerns related to the impact on the hierarchical clustering analyses may be obviated given that the measures were standardized for that analysis. Furthermore, inasmuch as caution is warranted when interpreting the between-group means at baseline, it is unlikely that this measurement difference at baseline appreciably impacted the results.

Conclusion and future directions

In their recent and timely review of the literature on the sustainability of health programs and interventions, Stirman and colleagues concluded that much of this research is underdeveloped and in need of methods to demonstrate whether and how programs continue beyond initial implementation [10]. These authors also critically observe that many studies measure sustainability only at a single time point, thereby missing the dynamic quality and variation inherent in the process. They go on to note that appropriate data and mixed methodologies are “necessary to refine hypotheses, explore results, understand the relationships between sustainability drivers and facilitate the development of interventions to promote the sustainability of effective programs…” (p. 8).

In light of their criticism and call to action, our current work makes a timely contribution to the literature on the sustainability of health programs. Investigations of the extent to which collaborative community health program partnerships are sustained requires multiple levels of analysis, measures of relationship level collaboration, assessment of the extent to which relationship patterns cluster across time and whether change over time is moderated by relevant factors associated with the mission and goals of the programs. The investment of public funds for community health programs is justified if these programs are sustainable and positive changes occur. These results demonstrate that planners and funders of these types of programs need to carefully assist partners in forging meaningful relationships by providing visible, significant events to enable them to work together to better serve their own constituents. The most promising partnerships will be evident and worthy of future investment.

Our work, which focused on the processes that are related to network sustainability, helped us to better understand how connectedness, multiplexity and reciprocity in relationships among our CNP partners impacted their likelihood of being sustainable. A potential next step is to conduct moderational analyses that examine impacts of the types of organizations, or resources available to organizations, on the associations between partners’ collaborative engagements and network sustainability. An additional next step is to develop applications for the longitudinal analyses of social networks that could incorporate the multifaceted analyses we undertook. With the advent of such a tool, another logical direction would be to prospectively apply these analytic techniques and evaluate their ability to identify sustainable relationships during the evolutionary development of new or existing networks. The goal would be to refine techniques that would reliably identify direction for the strategic allocation of resources in network development.