Rapid organizational network analysis to assess coordination of services for HIV testing clients: an exploratory study

Abstract

Recognizing that HIV testing provides a gateway opportunity to connect with at-risk populations, we explored an approach to collect, analyze and present data on the network of connections between HIV testing organizations and other health and social service agencies operating in Durham County, NC. We surveyed 26 health and social service organizations, including 6 providing HIV testing services, and presented the results including frequency tabulations, network visualizations and metrics, and GIS maps to the participating organizations. Mapping the landscape of organizational relationships was seen as a practical and expedient approach to facilitating cross-sector collaborative efforts to improve community health.

INTRODUCTION

Growing concern over health inequities has led to increased recognition of the need to go beyond individual behavior change efforts to address the social determinants of health such as access to health care, quality food, and housing. The obvious corollary of this is a need for enhanced cross-sector collaborative efforts across the various social, medical, and public health organizations offering these services (Mattessich & Rausch, 2014). Indeed, the ability of social service and public health organizations to collaborate and provide comprehensive and coordinated services has been recognized as an important component of community well-being especially in the context of HIV prevention and care (Fitz Harris, Toledo, Dunbar, Aquino, & Nesheim, 2014; Kwait, Valente, & Celentano, 2001; Robert Wood Johnson Foundation Commission to Build A Healthier America, 2014; Thomas, Carter, Torrone, & Levandowski, 2008; Thomas, Isler, Carter, & Torrone, 2007).

HIV testing provides a unique opportunity to provide individuals at high-risk for disease acquisition with needed preventative social services. At the time of a positive HIV test result, most testing organizations connect the newly diagnosed person with a variety of social and support services that he or she may need. In contrast, people who receive a negative HIV test result typically receive brief HIV post-test counselling and are encouraged to return for a repeat test in the future. However, many HIV negative individuals living in disadvantaged environments have health and social service needs that overlap with the needs of people living with HIV. Linking people at risk for HIV to many of those same health and social services may reduce the number of new infections (Hatcher, Smout, Turan, Christofides, & Stoeckl, 2015).

Unfortunately, the health and social services needed by those at highest risk for HIV are frequently fragmented and hard to access due to resource limitations, unpredictable national and state funding priorities, and limited communication and collaboration between organizations (Johnson, Oliff, & Williams, 2011; Thomas et al., 2008; Thomas et al., 2007; Torrone, Levandowski, Thomas, Isler, & Leone, 2010). A study on the interconnectedness of HIV prevention agencies in a rural North Carolina county showed that among the 11 governmental, community-based, and faith-based agencies that took part in the study, there was little exchange of client referrals, information, or funding (Thomas et al., 2008). Research on STI rates in ten counties in North Carolina found that syphilis rates were significantly lower in counties where the network of referral and information connections between organizations was highest (i.e., high network density) and that STI rates were higher in locations where a larger proportion of referrals were flowing to and from one or a few agencies in the network (i.e., high network centralization) rather than spread across the network of agencies (Thomas et al., 2007).

The purpose of this study was to explore whether analyzing and providing feedback to participating organizations about their service network would be well-received and perceived as a potentially useful tool to facilitate collaboration between HIV testing organizations and other health and social service agencies operating in Durham County, NC. With an eye toward sustainability, we also sought to determine the feasibility and ease of collecting data from service organizations using a publicly available, online organizational network survey and analysis tool. Similar to other urban areas and much of the Southeastern U.S., Durham contends with high rates of poverty (Durham County Department of Public Health & Partnership for a Healthy Durham, 2014), homelessness (Tippett, 2014), substance abuse (Quality Management Team, 2013), STIs and HIV (Communicable Disease Branch, 2014). Racial disparities characterize these trends as well. African American’s carry a disproportionate burden of HIV cases in Durham County. From 2012-2014, nearly 70% of all newly diagnosed HIV cases were African American, although they account for only 37% of the county’s population (Jolly et al., 2016). Durham, however, also has a strong foundation of clinical services (Durham County Department of Public Health & Partnership for a Healthy Durham, 2014), and testing for HIV and other STIs is offered in a wide variety of community settings. The potential for such testing to reach those at highest risk in Durham was confirmed by a community-based survey conducted in 2011-2012 with 508 sexually-active Black young adults (273 women, 235 men). HIV testing experience varied by gender, with 173 (74%) of men and 236 (86%) of women reporting ever being tested (MacQueen et al., 2015). More recently, a program to increase HIV testing rates is being implemented, increasing the potential for testing to be a “gateway,” connecting individuals at high-risk with preventative health and social services (Partnership for a Healthy Durham, 2014).

METHODS

The [institutional IRB] reviewed this study and determined that the project did not meet the regulatory definition of research involving human subjects as defined under the Department of Health and Human Services Code of Federal Regulations [45 CFR part 46.102(f)].

Selection of Organizations

For this exploratory study, we identified a subset of local organizations providing health and social services to Durham County residents, specifically HIV testing and counseling, HIV/AIDS treatment, substance abuse services, food assistance, and housing assistance as one of their central services. We chose this subset of services because they have been identified in previous research as key components of HIV prevention and care strategies (Surratt, O’Grady, Levi-Minzi, & Kurtz, 2015; Wolitski, Kidder, & Fenton, 2007).

We identified 26 service organizations providing these health and social services to Durham County residents. We began with a list of six organizations that provided HIV testing services and were known to us either through reputation or previous collaborations. We then searched both on the United Way 211 website (a free, confidential national service to help people find local resources) and on Google using the key term, “Durham”, and each of the following terms: “HIV/AIDS”, “health”, “substance abuse”, “housing”, and “economic support services”; we identified twenty additional organizations from these searches. Many of the organizations identified provided more than one service of interest. We identified one contact at the director level or an HIV-coordinator at each organization to solicit participation in the survey as an organizational representative.

Survey Implementation

To collect the network data, we identified a social network data collection tool developed by the University of Colorado Denver and sponsored by the Robert Wood Johnson Foundation called PARTNER (Program to Analyze, Record, and Track Networks to Enhance Relationships), which was specifically designed to measure and monitor collaboration among organizations or people (Center on Network Science, 2014). We selected PARTNER because it was inexpensive and included a modifiable, structured, online survey and linked analysis tool with three pre-programmed features: network visualizations, calculation of network metrics and frequency tabulations of survey questions.

The standard survey included in the PARTNER tool was designed to assess the structure and strength of organizational collaboratives (i.e., groups of organizations known to be working together in a joint effort or toward a common end) and includes 19 questions about organizations and their relationships with others in a collaborative, of which more than half were completely or partially modifiable. Although we were not assessing relationships between organizations that identified as a collaborative, the flexibility of the survey questions allowed for the assessment of relationships in any informal network of service organizations. We added questions about organizational relationships; these included a question about referral relationships and a question about integrated relationships. For the latter question, we provided a definition of integrated relationships as those that involved all of the following: (a) exchanging information, attending meetings together, and offering each other resources, (b) engaging in intentional efforts to enhance each other’s capacity for the mutual benefit of programs, and (c) using commonalities to create knowledge and programming that supports work in related content areas.

An introductory email was sent to the identified contacts at the 26 organizations on July 9, 2014 describing the study and letting the participants know to expect an email inviting them to participate on the next day. An email with a link to the online survey was sent to organizational contacts on July 10. A follow up email reminder was sent two weeks later. Due to low response rates, we contacted each organization twice by email and then several times by phone over the following three weeks to encourage participation.

Network Analysis & Visualization

The PARTNER tool is programmed to house and output data as an Excel file which contains both the raw survey response data and several pre-programmed analyses including frequency tables and calculations of some network measures. At the network level, density and centralization were calculated. Density is the number of reported ties (i.e., relationships) between nodes (i.e., organizations or people) divided by the number of potential connections (Wasserman & Faust, 1994). Network centralization describes the distribution of relationships across the network (Borgatti & Everett, 2006).

At the organizational or respondent level, degree centrality, which refers to the number of direct relationships each organization has to other organizations, was calculated (Freeman, 1978). In organizational network analysis, the relationship distance between a pair of organizations is very close when they have a direct relationship with each other, less close when they share a relationship with a third organization but not each other, and increasingly distant if the relationship is more like that of a “friend of a friend of a friend.” We also looked separately at in-degree and out-degree centrality measures, which are counts of the number of reported relationships to (i.e., in-degree) and from (i.e., out-degree) network members (Freeman, 1978). Finally, we complemented the PARTNER network measure outputs by creating sociograms displaying various relationships between the participating organizations using Pajek software (Batagelj & Mrvar, 2014).

Obtaining Stakeholder Feedback

As part of this exploratory study, we convened a meeting of participating organizations in October 2014 to present the results from the survey, solicit feedback from service organization representatives about the utility of an organizational network mapping tool, and generate a conversation about the network’s current ability to link those at risk for or infected with HIV to needed care and services.

Mapping

To further understand the network, a spatial approach was added using ArcGIS v10.2 (ESRI, 2013), a geographic information system (GIS), to map connections geographically. Each organization was georeferenced on a map of Durham County and directional lines were drawn between organizations that referred clients to one another. Additionally, maps were produced to visualize the level of spatial connectedness between service providers and population-level risk factors for HIV infection. Ideally this would include neighborhood or census track HIV prevalence or incidence data, but for confidentiality reasons the Durham County Department of Public Health does make that data available for this type of exploratory study. We therefore used census data as a proxy to identify areas with populations known to be at greatest risk for HIV in Durham county (e.g., African-Americans, young men, percent of households in poverty). Knowing that populations who are most at risk of HIV are often disproportionately in lower socioeconomic status groups and therefore often rely on public transportation to access services, we also overlaid mapped bus routes on the map showing the location of the service organizations. This was done simply for illustrative purposes and was used to start conversations within the stakeholder group about the placement and coverage of services.

RESULTS

Of the 26 organizations invited to participate, 24 opened and sent back the survey. Not all of the surveys sent back were complete but 20 of the organizations provided information on the services provided and needed and on their referral networks, which were the foci of this analysis. All organizations that were invited to take the survey also were invited to participate in the stakeholder feedback meeting, and six sent representatives.

Services Provided & Needed

Organizations reported providing a broad range of services with the most frequent being referrals to services (77%), assistance with completing application forms for benefits (50%), and HIV testing and counseling (46%). When asked which services were most needed by Durham area residents at highest risk for acquiring HIV, the most frequently reported service was outpatient substance abuse services (54%), which happened to be the service least frequently provided by the organizations in this network (Table 1).

Table 1.

Percent of participating organizations who reported providing services and services most needed by Durham area residents at high risk for HIV (N= 20)^a,b

Services	Provided %	Most Needed %
HIV-related services
HIV testing and counseling	46	42
HIV education (community based)	31	42
HIV case management for HIV+	19	19
Health care services
Health care	23	42
Family planning	23	12
Housing services
Housing (homeless/emergency shelter, long-term housing, etc.)	31	42
Housing support (rent assistance, training)	27	39
Other services
Referral to services	77	35
Assistance with completing application forms for benefits	50	19
Help with transportation	42	23
Assistance to those seeking employment (mentoring, resume development, etc.)	42	12
Social worker	31	31
Food assistance (food pantry, kitchen, Meals-on-Wheels, etc.)	27	19
Services post-incarceration	23	15
Job bank employment services	12	12
Outpatient substance abuse services	8	54

^a20 organizations responded to these questions

^bOrganizations were allowed to choose as many response options as applied

Network Relationships

As part of the survey, organizations were provided with a list of the other organizations invited to participate in the network survey and asked about the nature, frequency and reciprocity of referral relationships with each. Based on connections made between organizations referring clients to one another, the density of this network was 40.7%, indicating that fewer than half had any referral relationship with other organizations in the network.

The network centralization score based on these connections was also fairly low at 47.4%, indicating that the referral relationships that did exist were distributed among organizations (in a more highly centralized network, the majority of network relationships would emanate from a small number of network members). On average, organizations in this network referred clients to about eleven other organizations (mean degree centrality = 10.69) in the network. In-degree centrality ranged from 0 to 15 and out-degree centrality ranged from 0 to 17. We also looked at the correlation between in-degree and out-degree and found there was little correspondence between a given organization’s in-degree and out-degree, indicating that the organizations who made the greatest number of referrals to other organizations in the network were not necessarily the organizations who received the greatest number of referrals from other organizations in the network.

Network Visualizations

The sociogram in Figure 1 shows the participating organizations organized by service sector and the number and type of relationships existing between them based on both client referrals and engagement in integrated activities. Notably, although only 20 organizations provided responses about whom they referred to, we included all 26 organizations in this sociogram since organizations who did not respond were still named by the other participating organizations. As shown, there appear to be many referral relationships in this network but when participating organizations were asked with which organizations they shared integrated activities, the relationships were much sparser. Notably, there was one organization whose primary service was HIV testing and counseling that did not receive or send referrals to any of the other organizations. Also of note, the organizations who primarily provided housing services did not engage in any integrated activities with organizations providing primarily health care or other social services.

Figure 1.

Existing Relationships between Organizations in this Network based on Referrals and Integrated activities.

Mapping

To spark discussion of the potential utility of geospatially presenting the data, we created a number of maps to share at the stakeholder meeting; an example is shown in Figure 2. Data in this map were aggregated by census tract for the urban southern half of Durham County. The total number of organizations per each census tract, represented by graduated red circle symbols, were largely clustered in the city’s downtown. The census tracts where the percent of households in poverty for 2012 was highest (chosen to indicate areas where referral services may be needed most) were clustered in the eastern and southern areas near downtown. Thirteen organizations were found in census tracts adjacent to the highest levels of poverty and stakeholders noted that bus routes were lacking to services located in the southeastern census tract.

Figure 2.

Map of Durham County showing Location of participating Organizations, Percent of Households in Poverty by Census tract and Bus routes.

Stakeholder Feedback

The agenda of the stakeholder meeting was to present the survey results and obtain feedback on the study process, survey and next steps. Though the number of organizations represented was small, the participants were engaged and interested in the results presented. The overall consensus from meeting participants was that the survey results were a useful tool to understand how organizations within the network related to one another, as well as the services and resources different organizations offer. We categorized the types of reactions and suggestions from the discussion into reactions that pertained to the following four areas: (1) improving the overall study design and process, (2) modifying the content of the survey, (3) presentation of data in maps, tables, sociograms and graphs, and (4) potential next steps for utilizing these results, and refining and expanding the process should it go forward (Table 2).

Table 2.

Examples of Types of Reactions and Suggestions from Stakeholders

Study Design Process	Survey Content	Visualizations (i.e. tables, maps, graphs, sociograms)	Potential Next Steps
Expand social service categories to include the following six services: mental health services, human trafficking, education, LGBTQ, probation, domestic violence	Collect information on organizational capacity (i.e. current staffing levels, number of clients)	Network analyses were of interest but sociograms and statistics were difficult to interpret and needed more explanation	Present these results to additional groups such as the United Way, Partnership for Healthy Durham, Ryan White Association to inform the utility of furthering this work
Expand original service providers to include twelve additional local service organizations	Avoid using potentially judgmental terms such as “Trust” and “Value”; opting for descriptive terms that are less easily misinterpreted	Geographic maps showing spatial relationships between network members were useful	Conduct a needs assessment with clients to determine their experience with referrals
Increase participation by conducting advanced engagement with potential survey respondents	Provide mutually exclusive service categories (i.e. break out “HIV case management” services)	Adding travel time metrics to GIS map would help with understanding of distances to services within the network	Make use of existing data sources from organizations such as United Way, Carolina Homeless Information Network and Durham Office of AIDS Alliance to enrich knowledge of the service network and client needs and demographics
Obtain input from providers on bounding the service provision network	Include questions about network governance (formalization of the relationships among the members and their accountability mechanisms)	Organizations should be identified on all data presentations otherwise it lessens the utility of the findings	Make this an iterative monitoring tool (i.e., update at regular intervals)

In terms of the study design process, the two most consistent pieces of feedback received from participants were to include a greater number of service organizations and to involve participating organizations earlier on in the study design process. Participants provided numerous suggestions regarding survey content; the most notable and agreed upon were questions on organizational capacity and network governance (i.e., formalization of the relationships among the members and their accountability mechanisms). At the meeting, we presented findings using several different types of visual displays ranging from tables and charts to maps and sociograms. The maps generated the greatest interest among participants.

Participants were highly supportive of using this approach as an iterative monitoring tool for facilitating collaboration because it would allow organizations to assess how everyone is connected and what changes are occurring in the network over time. Participants expressed concern, however, regarding the sustainability of the approach, as they were unable to immediately identify anyone who had the capacity to administer this tool on an ongoing basis.

DISCUSSION

The results from this study shed light on the structure of the current network of organizations providing HIV care and related health and social services to residents of Durham County, NC and garnered positive feedback from participating organizations. Findings from the analysis and stakeholder meeting further suggested a variety of ways in which network data collected and disseminated in this manner could be constructive simultaneously to organizations and the clients and communities that they serve.

For one, the survey questions allowed for identification of services currently being provided by each organization in the network. At the feedback meeting several organizations commented that they had been unaware of the services provided by other participating organizations. If questions were added to the survey to elucidate not only services provided but also capacity for referrals, as suggested by study participants, this could inform efforts to modify or increase the volume of client referrals across the network. After studying the maps, many of the study participants similarly revealed they were unaware of the physical distances between services and the lack of public transportation available to reach some service sites; this information similarly could immediately translate into more informed referrals for clients.

Visualization of the network through sociograms and calculation of some key measures were less intuitively comprehensible to participating organizations, but once explained were of interest and could be informative to network members in several respects. For example, identification of the most central node in a network could be useful if there were a need to disseminate information rapidly across a network or to create a bridge between one network of services to another. Conversely, identification of isolates (i.e., organizations without any relationships to others) is also illuminating, as was the case in our study since the one identified isolate was an organization that provided services deemed by the participating organizations to be critical to those at high-risk for HIV. Calculation of network density in this network revealed many organizations were not well-connected to other organizations; such information could be useful to networks in determining where collaboration could be potentially enhanced. In addition, if the ultimate goal were to achieve an integrated platform of service provision across which client lists and funding streams are shared, repeated assessments of the existence of integrated relationships would be crucial.

The efficient three-month time frame over which we collected and analyzed data, the high rate of survey response, and the positive reception of the analyses by the most engaged organizations supports our secondary objective of demonstrating the feasibility of this approach. While we felt that using a publicly available online organizational network survey and analysis tool was a good decision for the efficiency of this data collection effort, we also noted a variety of aspects of the PARTNER data collection and analysis functions that we would want to modify or augment in using this approach in the future. Most notably, GIS-based maps are not part of the PARTNER platform yet were found to be highly useful to participants. In addition, the network visualizations included in PARTNER were not customizable and did not display the aspects of the network structure that were of greatest interest (i.e., density, centralization); this necessitated exporting the data to another network analysis tool, which required additional network analysis expertise on the part of our study team.

Our study was not without limitations. As a result of study timeline and funding constraints, data were collected at one point in time and decisions about study participants, survey questions, and response options were made without input from community partners. This undoubtedly led to the omission of network participants and survey questions that may have provided important insights into the makeup, structure, and quality of relationships in this network. Notably this problem is not unique to our study in as much as when using quantitative network surveys it is always difficult to be assured that you have asked all the best questions to elicit and understand the network relationships. Additionally, many of the organizations included in this study were selected using limited information that was available through United Way 211 and on organizational websites. Because so many different services are important to HIV prevention, the method of organization selection may have contributed in part to the lack of connectivity between organizations.

The cross-sectional design of the study and the fact that no ideal set of network parameters exists meant that we could not determine whether these parameters were changing in a desired manner (e.g., greater density or reciprocity). The fact that only 20 of the 26 agencies surveyed provided complete responses on the survey and only one-quarter of the responding organizations attended the feedback sessions additionally limits our findings. Many of these limitations are reflective of the exploratory and rapid assessment nature of this study and therefore can be addressed in future studies. Despite the noted limitations of our approach, we believe our study’s use of a rapid assessment format (i.e., online survey followed by quick analysis and feedback to the organizations) to be a novel application of organizational network analysis and to have significant potential use in situations that demand quick and efficient investigations of networks. As a rapid assessment tool, the approach also has potential for communities to conduct ongoing monitoring and evaluation of strengths, weaknesses and gaps among organizations seeking to serve a diverse range of client needs. Furthermore, were this type of analysis to be conducted at the level of the local or county public health departments, who likely contract to most service organizations, we envision significant potential for it to shape policy and funding decisions.

PUBLIC HEALTH IMPLICATIONS

Insights obtained from our analysis and reactions from participating organizations suggest that mapping the landscape of organizational relationships and providing this information to local organizations may be a practical and expedient approach to facilitating cross-sector collaborative efforts to improve the health of community residents. Though our exploratory project collected data pertaining specifically to a small network of HIV care and related social service provider organizations in Durham County, NC, our findings regarding the simplicity and utility of this data collection effort have implications and applicability for a much broader audience. For those interested in building healthier communities by addressing broader social determinants of health, greater integration of public health, medical and social service provision at local and regional levels will be needed (Braunstein & Lavizzo-Mourey, 2011; Williams & Marks, 2011). HIV testing is one example of a focused screening effort that has the potential to identify people in need of a wide range of services regardless of the outcome of the particular disease being screened. By linking health professionals who provide such screening to broad health and social services, there is an opportunity to overcome the fragmentation of services and provide a better HIV prevention strategy for people at risk for multiple negative health outcomes. Indeed, in terms of the replication of our approach as a HIV prevention strategy, we would recommend those best-positioned to do so would be the health department unit(s) responsible for administering CDC and/or Ryan White-funded testing programs, as well as other social/supportive and medical services for people living with or at risk for HIV. Notably, however, as program funding evolves with shifts in state and national health funding policies, the challenge would be to find a home in local health departments that is sufficiently stable to sustain this activity.

For those interested in supporting the successful adoption, implementation, and sustainability of evidence-based programs, understanding which partnerships can be created and maintained will be critical (Brown et al., 2012; Palinkas et al., 2014; Valente, Fujimoto, Palmer, & Tanjasiri, 2010; Valente, Palinkas, Czaja, Chu, & Brown, 2015). To date, however, the literature on the use of network analysis to strengthen community partnerships is largely theoretical and consists of secondary analyses (Bevc, Retrum, & Varda, 2015; Granner & Sharpe, 2004; Manning et al., 2014; Mattessich & Rausch, 2014; Provan, Veazie, Staten, & Teufel-Shone, 2005; Retrum, Chapman, & Varda, 2013; Varda & Retrum, 2012; Varda, Shoup, & Miller, 2011). Little primary empirical evidence of how to build and sustain such partnerships exists (Thomas et al., 2007; Thomas, Reynolds, Alterescu, Bevc, & Tsegaye, 2015). We have shown that collection and provision of real-time network data directly to local organizations is feasible and shows great promise as a tool to facilitate the formation and sustainability of cross-sector collaborative efforts over time and potentially improve health outcomes.