The Value of Social Network Analysis for Evaluating Academic-Community Partnerships and Collaborations for Social Determinants of Health Research

Abstract

Objective

Community-based participatory research processes build healthy communities, as well as promote trust and genuine collaborative partnerships between stakeholders. Fostering relationships is essential to promoting these partnerships, which are necessary for collaborative, coordinated, and integrated efforts toward improving health outcomes in the community. The objective of our research was to demonstrate social network analysis as an evaluative tool to assess movement toward positive health outcomes through promoting relationships.

Methods

Using the example of the Gulf States Health Policy Center Coalition based at Bayou Clinic in Bayou La Batre, Alabama, we demonstrate the ability of social network analysis (SNA) methods to measure and map the formation of relationships, as well as the level and frequency of these relationships. Data were collected via email using a survey of Gulf States Health Policy Center Coalition members (N=80, 87%) and analyzed using UCInet software for social network analysis in April 2016.

Results

In this application of SNA to the community coalition of the Gulf States Health Policy Center, we find that, on average, coalition members doubled their own network within the coalition in a time period of <2 years and were working together more often and more collaboratively than they were before the coalition formed.

Conclusions

The increased frequency and level of collaboration among the Coalition network was accompanied by a higher level of collaboration among the coalition members as posited by social network and capital theories. As such, the community engagement fostered through the Coalition has increased and thus, to date, the Gulf States Health Policy Center has been effective in promoting partnerships and collaboration.

Introduction

Poor health outcomes and health disparities stem from a variety of factors.1 Despite considerable advances in conventional social science and health research, there remain significant health disparities related to social determinants of health, ie, those conditions in social environments where we are born, live, and age that affect health. Therefore, a multifaceted, multidisciplinary, and multi-systemic approach that involves members of the affected communities is vital to the public health infrastructure.1 Community-based participatory research (CBPR) allows for partnerships between academic and community partners to address social determinants and to design culturally and contextually specific health action plans.2 Essential to CBPR is community engagement, ie, the process of people affiliated by geographic proximity, special interest, or similar situations working collaboratively to address issues affecting the well-being of a community.3 As such, community members are considered subject matter experts with the ability to influence change. Community engagement has been successful in improving environmental health,4 post-disaster mental health disparities,5 economic development,6 prison reform,7 and changing policy.8-10

A common method of fostering community engagement is the creation of community coalitions, ie, groups of individuals and/organizations who agree to work together toward a common goal.11,12 To facilitate the work of these coalitions, partnerships – not only relationships among members – are necessary.13 Individuals with a common goal must first form a relationship and then collaborate toward a common goal.14 Partnerships are difficult to achieve in the absence of long-term bidirectional relationships built upon mutual respect and an understanding of the work of all partners involved.15 These relationships and partnerships are often assumed and not systematically evaluated.16 It is not uncommon for multiple stakeholders to work toward the same mission through parallel, albeit uncoordinated, efforts. It is, therefore, important to understand the complex relationships among coalition members, identify potential roadblocks, discern important roles that different organizations are serving, and discern between artificial relationships and true partnerships. Social network analysis (SNA) is an appropriate and valuable tool to do this.17 However, to date there has been limited use of SNA for the evaluation of the coalitions’ relationships and partnerships and measurement approaches for the performance and effectiveness of these partnerships in public health practice.

Social network analysis is presented herein as an evaluation tool that enables the quantitative evaluation of partnership/coalition development, network functioning, and even advocacy.18-20 SNA, which originated in sociology in the early 20th century, is now used across multiple disciplines, including criminology, organizational psychology, political science, biology, computer science, and communication.21 Across these disciplines, SNA is used to measure the formation and existence of relationships and used to provide data for the depiction of these relations using sociograms. These metrics and sociograms provide a simple way to illustrate the growth of a network, or lack thereof.

An example of a community coalition working to improve health and reduce health disparities is a coaltion established within the Gulf States Health Policy Center (GS-HPC), located in Bayou La Batre, Alabama. The GS-HPC is a comprehensive, education, and research center with a mission of improving health outcomes in the Gulf States region. The GS-HPC coalition comprises more than 100 community partners across the region, including individuals from civic and community-based organizations, higher education, state and local government, health and human services, private entities, and those simply representing the community. Using a CBPR approach, the GS-HPC works with the coalition to develop a health action plan that is culturally and contextually specific to the community. Since November 2014, the GS-HPC has formed and supported three coalition chapters: Bayou La Batre, Alabama; Birmingham, Alabama; Hattiesburg, Mississippi. Within this article, we present the evaluation of the GS-HPC coalition to provide an example of applying SNA to evaluate the development, process, and effectiveness of community coalitions. In particular, we evaluated the growth in the number of relationships, as well as the frequency and level of collaborations, since the formation of the GS-HPC coalition, and examined the association between frequency and level of collaborations.

Methods

This study is a cross-sectional analysis of data from a survey of the GS-HPC coalition members. It was approved by the Institutional Review Board (IRB) at The University of Southern Mississippi.

Participants

Participants were community coalition members who attended at least two coalition chapter meetings between November 2014 and February 2016. Between these dates, each coalition chapter met 12 times. Members were originally identified and recruited into the coalition in fall 2014 using the Workgroup on Community Health and Development Took Kit11 for the first meeting of the chapters in November 2014. Additional members were recruited after the first meeting based on identified gaps in expertise among existing members.

Coalition members were surveyed in April 2016. Ninety-two GS-HPC coalition members were invited to participate through an email invitation. The survey was administered through Survey Monkey and the first page of the survey was used to obtain informed consent from all respondents before continuing to data collection. The survey was open for one month and members received email reminders weekly. Coalition members who had not responded by the end of the third week were contacted by phone. Respondents were offered a $25 Amazon gift card for their completion of the survey.

Measures

A survey instrument with five social network questions was adapted from the PARTNER Tool developed by the University of Colorado Center on Network Science.22 Our outcomes were assessed at the coalition level rather than at individual chapters (respondents provided responses for their own chapter). These outcomes included: 1) existence of relationships; and 2) level and frequency of collaborations. First, respondents were asked to identity the coalition members that they knew from a list of the members in their coalition chapter. On the next screen, they responded to questions about frequency and level of collaboration with the selected coalition members for two time periods, ie, pre-coalition (before November 2014) and after the establishment of the coalition (November 2014 to April 2016). For individuals in the chapter that respondents did not know at any time point, a 0 was coded for both frequency and level of collaboration.

We measured the existence of relationships by network density, ie, the percent of all possible relationships that were present in the group as a whole.23 Values ranged from 100%, perfect connected coalition in which all members knew all other members, to 0%, unconnected coalition in which members did not know any other members. An increase in network density—an increase in the percentage of present relationships among coalition members— indicated that relationships had formed among the coalition members.

We measured the frequency of collaborations by asking respondents how often they collaborated with a selected coalition member, ie “How frequently do you work with this individual on issues related to the coalition’s goals?” Responses were 0 for never, 1 for once a year or less, 2 for about once a year, 3 for about a month, 4 for every week, 5 for every day, and 6 for working for the same organization. We measured level of collaborations by asking, “What kinds of activities does your relationship with this individual entail?” Responses were: 1 (Only work for the same organization); 2 ( Cooperative activities, involving exchanging information, attending meetings together); 3 (Coordinated activities, including coordinated activities beyond networking that improve outcomes for the community); and 4 (Integrated activities, with formal relationship, which may include shared funding, clients, and/or community events that improve outcomes for the community). Increases in either frequency of collaboration or level of collaboration demonstrate greater tie strength where ties are defined as information-carrying relationships and strength is defined by intensity of those relationships.24

Statistical Analysis

Of the 92 GS-HPC coalition members who received the survey invitations, 80 started (87%) and 74 completed the survey (80%). Thirty-eight of the 45 (84.4%) invited chapter members from Bayou La Batre participated, 22 of the 25 (88.0%) from Birmingham participated, and 20 of the 22 (90.9%) from Hattiesburg participated. As we only needed one part of each relationship dyad to assess that a relationship existed, we were able to examine the network for 86 individuals from the 74 completed responses (12 additional coalition members who did not respond to the survey). We examined a total of 3,655 possible relationships, calculated using ((N*(N-1))/2) in which N is the 86 coalition members represented in this analysis. The number of possible relationships was the same for the pre- and after-coalition time points: relationships not present in one of the periods were coded as 0 for frequency and level of collaboration.

We used the UCInet social network analysis software25 to develop network metrics and social maps (sociograms). We calculated the frequency for each of the responses and the mean level for frequency and level of collaboration. We then calculated the change in these outcomes by comparing the mean levels between the two time periods. Finally, we calculated the mean frequency and level of all collaborations of each coalition member for the two time periods, and the point change between the two time periods for each member to obtain scores of growth in both level and frequency of collaboration. These scores were then correlated using Pearson’s product-moment correlation.

Results

Existence of Relationships

Prior to the formation of the GS-HPC coalition, individuals in the network knew an average of 4.79 other individuals in the network and the network density was 5.6%. By April 2016, individuals in the network knew an average of 11.3 other individuals in the network and the network density was 13.5%.

Frequency of Collaboration

The change in frequency of collaboration is presented in Table 1. Overall, in the pre-coalition period, 88.7% of collaborations were non-existent (never). The number of “never” collaborations decreased by 189 (5.8% decrease) and the number that occurred about once a year decreased by 43 (76.2% decrease). The collaborations occurring about once a month or every week increased by 152 (65.7% increase), and 32 (177.8% increase), respectively.

Table 1. Change in frequency and level of collaboration.

	Pre-Coalition			April 2016		Difference	% change
	n		%	n	%
Frequency of collaboration (code)
Never (0)	3,241	88.7		3,052	83.5	-189	5.8
Once a year or less (1)	63	1.7		15	.4	-48	76.2
About once a year (2)	26	.7		69	1.9	+43	165.4
About once a month (3)	235	6.4		387	10.6	+152	64.7
Every week (4)	18	.5		50	1.4	+32	177.8
Every day (5)	1	0		1	0	0	0
Work for the same organization (6)	71	1.9		81	2.2	+10	4.1
Level of collaboration (code)
None (0)	3,243	88.7		3,073	84.1	-170	5.2
Only work for the same organization (1)	120	3.3		81	2.2	-19	15.8
Cooperative activities (involves exchanging information, attending meetings together) (2)	150	4.1		344	9.4	+194	129.3
Coordinated activities (includes coordinated activities beyond networking that improve outcomes for the community)(3)	83	2.3		127	3.5	+44	53.0
Integrated activities (formal relationship, which may include shared funding, clients, and/or community events that improve outcomes for the community)(4)	59	1.6		60	1.6	+1	1.7

Figure 1 and Figure 2 provide visual evidence for collaboration frequency by comparing the sociograms for respondents with a frequency of collaboration >3 (ie, meeting at least once a month) for the two time periods. The images clearly illustrate an increase in frequency of collaborations that were scored > 0, >1 and >2. Prior to the coalition, respondents reported working with others in their network at a mean frequency of collaboration of 2.68 (Std=1.90) between “About once a year” and “About once a month.” After the coalition was established, respondents reported working with others in their network at a mean frequency of collaboration of 4.09 (Std=1.25) between “About once a month” and “Every week.”

Figure 1. Pre-coalition frequency of collaboration.

Figure 2. April 2016 frequency of collaboration.

Level of Collaboration

Overall, in the pre-coalition period, 88.7% of the coalition members were not collaborating on activities related to the mission of the GS-HPC (Table 1). The number of “none” relationships decreased by 170 (5.2% decrease) and number of relationships that occurred by individuals who only worked for the same organization decreased by 19 (15.8% decrease). The number of relationships with cooperative activities, coordinated activities, and integrated activities increased by 194 (129.3% increase), 44 (53.0% increase), and 1 (1.7% increase), respectively.

The sociograms illustrate an increase in level of collaborations > 0 and >1 (Figure 3 and Figure 4). In the pre-coalition period, respondents reported a mean level of collaboration of 1.95 (Std=1.19) between “We only work for the same organization” and “Cooperative activities.” After the coalition, respondents reported their level of collaborations at a mean of 2.26 (Std=1.05)—between “Cooperative activities” and “Coordinated activities.”

Figure 3. Pre-coalition level of collaboration.

Figure 4. April 2016 level of collaboration.

Correlation between Level and Frequency of Collaboration

We calculated mean frequency and collaboration for each network member during the two periods of interest to develop a measure of change for each network member for frequency of collaboration and for level of collaboration. The Pearson’s product-moment correlation coefficient between change in frequency of collaboration and change in level of collaboration is r(86)=.56, P<.001. This finding indicates that an increase in frequency of collaboration is a statistically significant predictor of an increased level of collaboration among coalition network partners.

Discussion

In this application of SNA to the community coalition of the GS-HPC, we found that, on average, GS-HPC coalition members doubled their own network within the coalition in a time period of <2 years; and, they were working together more often and more collaboratively than they were before the coalition formed. Moreover, the correlation coefficient demonstrates that increased frequency was accompanied by a higher level of collaboration among the coalition members as posited by social network and capital theories.17-20 As such, the community engagement fostered through the GS-HPC has increased.

The GS-HPC coalition chapters share the common goal of fostering sustainable collaborations and partnerships. Thus, improving the relationships among network actors as coalition members is an important step not only toward meeting the stated goals of the GS-HPC, but also toward ensuring that the relationships exist in the communities in which the chapters are located. In this sense, the networks and relationships formed through the coalition meetings are providing a foundation for future projects and initiatives to improve the health of the community. Moreover, as the CBPR process includes trainings—ie, on conducting research, grant writing, health problem solving—the GS-HPC coalition represents a network of skilled individuals interested in improving the health of the community, which promotes sustainability of efforts. It is imperative that community efforts to promote health build capacity as a form of sustainability focus on the promotion of relationships and social capital.26 Thus, in evaluating the relationships formed through coalition work, we are not only, in essence, evaluating the aims of the GS-HPC in regard to the coalition, but also evaluating the potential of these networks to exist beyond periods of grant funding. Moreover, the effectiveness of partnerships is also a key indicator of the ability of a coalition to achieve its ultimate goals.6, 22,27

Whether between public health and medicine, health departments and municipal offices, or academia and community residents, there is no shortage of literature to support the value of partnerships – vs relationships – in improving health outcomes.1,13,20 Thus, the increase in the levels of collaboration in the GS-HPC is important for the goals of the center and the coalition members. An increase in the frequency of collaborations without the increase in its level would not be effective to address community health issues or reduce health disparities. However, without a consistent evaluation of the relationships of coalitions members, such assessment cannot be made and the reasons for having ineffective coalitions may not be discovered or addressed.6,15 For this reason, we recommend the continued collection of data to assess coalition members’ relationships as well as the long-term effects of the formation of these relationships on future endeavors and ultimately health outcomes.

In addition to illustrating network growth, SNA data can be used to identify areas for improving coalition efforts. Based on our findings, the areas of opportunity for the GS-HPC are to: 1) foster collaboration among the members of the three coalition chapters to increase opportunities for networking collaboration across chapters; and 2) monitor progress within each coalition chapter to develop a deeper understanding of the mechanisms by which partnership development influences policy change and community health improvement. To achieve this, the GS-HPC is planning more coalition events that bring together all three chapters. We will assess coalition members at six-month intervals to evaluate the effectiveness of such interventions on improving the social capital created through the coalition.

Study Limitations

There are some limitations to consider. Data were collected at one point in time: thus, relationships for the pre-coalition period were based on recollection of those relationships. Thus, recall bias introduces method variance in which it is possible that the variance we are observing is from the measurement, rather than frequency and level of collaboration as the constructs of interest.28-29 In social network analysis, time dependence (herein marked by joining the coalition network) is important for assisting in recall.30 Adding behavioral specificity to the relationship being recalled, such as joining the coalition network, reduces forgetting.29 Thus, it is likely that there is more accuracy in the data on the formation of relationships than in the specifics of the relationships, such as frequency and level of collaboration. To improve future applications of social network analysis as an evaluative tool for academic-community partnerships, we recommend that social network data are collected as partnerships form, rather than in a reflective manner.

In addition to method variance, respondents could have been motivated by social desirability in which they may have biased the results by over estimating their current relationships.30 If respondents underestimated the frequency and level of collaboration in the pre-coalition period or overestimated these factors in the current period, then we may have overestimated the improvement in collaborations in these analyses. In addition to social desirability, there are other extraneous factors that would lead us to overestimate the effect of the GS-HPC on the coalition network. For example, coalitions members may have met by other means or they may have collaborated even without the GS-HPC coalition. Similarly, data were not collected on the nature of the meetings that occurred beyond the monthly coalition meetings. As much of the increase in both frequency and level of collaboration was driven by the monthly coalition meetings, further research is needed to assess how frequently and to what purpose coalition members were meeting outside of the formal meetings. We are conducting follow-up qualitative research to better understand the nature of the collaborations and partnerships resulting from the coalition efforts.

Conclusion

In conclusion, through the GS-HPC efforts, coalition members are moving from superficial relationships to deeper, more meaningful relationships, and to partnerships within their chapters. SNA appears to be a valuable tool in evaluating the growth of community coalitions and the continued community engagement to address health disparities. The next steps in supporting these findings are to conduct more of an interpretative analysis that assesses the characteristics associated with increased frequency and levels of collaboration. While descriptive statistics derived from SNA have been demonstrated herein to have evaluative value, this further interpretive analysis will yield implications for improving current and future academic-community partnerships and collaborations. Finally, further examination between the interpretive analysis and outcomes is warranted. Future research is recommended to examine the association between network changes observed through the methods presented herein and the ability of the network to achieve its stated objectives and ultimately improve health outcomes.