A Network Assessment of Community-Based Participatory Research: Linking Communities and Universities to Reduce Cancer Disparities

Abstract

Objectives. We sought to determine whether a community-based initiative designed to reduce cancer disparities among Pacific Islanders in Southern California increased communications between community-based organizations and university researchers.

Methods. We conducted network analysis among 11 community-based organizations (CBOs) and 5 universities by interviewing 91 and 56 members of these organizations, respectively, at 2 points in time. We estimated random effects probit regression and stochastic actor-oriented network dynamic models.

Results. We found that, during the 2-year study period, CBOs increased their connectedness with one another (b= 0.44; P < .05) and to the universities (b = 0.46; P < .05), but that university researchers did not increase their connectedness to each other or to CBOs.

Conclusions. Cancer awareness, cancer education, and access to cancer services are low among Pacific Island groups, and this study provides an initial attempt to reduce these disparities. Community-based initiatives can strengthen a CBO network, creating the potential for increased community-informed cancer research and improved community access to cancer research resources.

Community-based participatory research (CBPR) is increasingly viewed as a promising approach to cancer health disparities research that can bridge the enduring divide between scientific discovery and community impact.¹ Although CBPR methodologies vary, at their core they promote translational research by forging ties between community members and university researchers throughout the research development, implementation, and translation processes.^2–4 Additionally, such processes yield longer-term benefits to CBPR partners, including the creation of communities that can effectively advocate their needs and create institutional changes that can reduce community cancer health disparities.⁵ As the CBPR field has grown, so, too, has the number of empirical studies on various aspects of its conduct and impacts⁶; however, there remains a dearth of studies evaluating the effectiveness of CBPR programs.

We report the results of the Weaving an Islander Network for Cancer Awareness Research and Training (WINCART) initiative designed to reduce cancer disparities among Pacific Islanders in Southern California.⁷ The WINCART initiative was created as a forum for university researchers and Pacific Islander community leaders to meet and establish linkages and has involved both a scientific advisory board (SAB; comprised of cancer-related researchers from 5 universities) and a community advisory board (CAB; comprised of representatives from 11 community-based organizations [CBOs]) to guide WINCART's collaborative activities. A stated objective of the WINCART initiative is to create linkages between CBOs and academic institutions conducting cancer education, research, and training.⁸

The CBOs are all nonprofit entities that conduct health and social service programs for Pacific Islanders in Southern California. The linkages created by WINCART would enable CBOs to disseminate information about cancer prevention, early detection, research, and treatment developments to their constituents. At the same time, WINCART was designed to create linkages from cancer researchers in 5 academic institutions (3 of which have comprehensive cancer centers) to CBOs so that cancer research, education, and training would be more community-informed. Public health researchers and advocates have long recognized that health disparities research often is not informed by community partners, and the gap between the academy and community persists today.

Since its inception in May 2005 (with $2.5 million in funding over 5 years from the National Cancer Institute's Center to Reduce Cancer Health Disparities), WINCART has conducted many activities to bridge the gaps between community and academy. Some of these activities have included bimonthly trainings of community leaders (usually by SAB members), retreats in which community and university researchers have focused on collaboration-building exercises and discussions, and events where community and university researchers have shared research activities and findings. Behind the scenes, WINCART has also facilitated relationship-building between community and university network members with mutual research interests, resulting in 6 National Cancer Institute–funded research projects on cancer epidemiology, prevention (nutrition and physical activity), and survivorship issues.

Several activities were initiated by core WINCART staff to promote collaboration between university and community partners. With regard to education, 3 researchers in the network were asked to train CAB members on the topics of “cancer 101,” physical activity, and cancer survivorship. These training sessions helped increase the capacities of CAB members to provide cancer-related outreach education to their communities, which, for the first year, included a total of 223 events that educated 27 886 individuals. With regard to research, SAB and CAB members attended 2 all-day meetings on networking and collaboration, including the annual WINCART fall event that reviewed the network's yearly activities and progress and an SAB orientation that introduced university researchers to the network's mission and to potential community collaborators.

We used social network analysis to evaluate the effectiveness of these activities. The stated goal of the project was to increase connections among and between CBOs and university organizations; thus, social network analysis was a logical methodology to use. Social network analysis is a set of theories, methods, and models for understanding how connections and relationships among entities are formed, evolve, and influence behaviors and actions.^3,9–11 There have been several notable prior network studies of interorganizational relations in the public health and health care delivery fields.^3,9,12–14 Generally these studies measure the degree of communication, collaboration, client referral, and formal agreements among organizations. The connections are mapped with specialty software and individual and network-level indicators are calculated (UCINET for Windows; Analytic Technologies, Lexington, KY).^9,15

This study was designed to determine whether community-based outreach activities can increase cancer-related networks connecting CBOs and university researchers. Specifically, we expected networks to become denser overall and more heterogeneous such that communications between CBOs and universities show an increase over time. To test this hypothesis, we collected network data at 2 time points and employed several statistical modeling techniques.

METHODS

We measured the extent to which individuals, as representatives of organizations or programs, reported interactions with a list of the 19 organizations involved in WINCART activities. Eleven of the organizations were CBOs, 5 were universities, and 3 were national cancer-related organizations (i.e., American Cancer Society, Cancer Information Service, and Susan G. Komen for the Cure).

Survey

Fourteen network questions were asked via a roster format in which each respondent was presented with a list of the 19 organizations. The 14 network questions were grouped into 3 sections on 3 separate pages with a statement at the top of each page: “For each of the following organizations or institutions, please check the box if you … (PLEASE CHECK ALL THAT APPLY).” The network question stubs were then presented as column headings. For example, general communication was measured with the stub “have ever communicated with in the past year.” The first network page measured general relationships such as: (1) general communication, (2) formal agreements, (3) client referrals to, and (4) client referrals from. The second page measured noncancer activities by asking what noncancer activities they worked on together regarding (1) education, (2) outreach, (3) training, (4) advocacy, and (5) research. The third page measured cancer activities by asking what cancer activities they worked on together regarding (1) education, (2) outreach, (3) training, (4) advocacy, and (5) research. (A copy of the complete survey questionnaire is available as a supplement to the online version of this article at http://www.ajph.org.)

Electronic invitations and surveys were sent to 121 individuals in 16 organizations in June 2005 (time 1; at the inception of the WINCART network) and 113 individuals in 17 organizations in July 2007 (time 2; approximately at the halfway point of funding for the 5-year project). Ninety-one respondents completed the survey at time 1 (75.2%) and 56 at time 2 (49.5%). At time 1, the 3 national organizations were not solicited to participate but invitations were sent to representatives of Cancer Information Service working in the community at time 2. We did not include these responses in this study because these data were only available at time 2. We removed all linkages to these 3 national organizations from the data for this analysis because there are no links from them. At least one individual from every participating organization responded. We aggregated the data to the organizational level so that individual responses are unknown.

Because the number of respondents from each organization varied, we summed the number of links between organizations and then divided by the number of respondents from each organization (details available in a supplement to the online version of this article at http://www.ajph.org). The dependent variable in this study was the percentage of links from one organization to another and the guiding research question was whether connectivity increased over time and shifted from within-group to between-group (e.g., from CBO to university or university to CBO).

Statistical Analyses

Three different statistical approaches are presented in this study. A somewhat traditional regression with probit regression was used to estimate factors associated with the linkage rate to test organizational attributes such as organization type, program participation, and tenure. The probit regression cannot account for network dependencies and so a network evolution model was estimated to test the likelihood of a link between organizations to determine whether organization type (CBO vs university) influenced linkage creation. Finally, the quadratic assignment procedure was used to test the correspondence between network linkages and event attendance to determine whether event coparticipation was associated with linkage change.

Initial analysis consisted of estimation with probit regression on the dichotomized median connection percentage (0.27) as a function of the following variables: (1) time (time 2 vs time 1), (2) average years working in current organization, (3) organizational follow-up response rate, (4) average number of WINCART activities participated in, and (5) network question type (general, noncancer, or cancer). To test for the effects of link type, we included indicator variables for link type based on organizational attributes: (1) university to university was the reference category, (2) CBO to CBO, (3) CBO to university, and (4) university to CBO. An interaction term between time and link type was included to test for program effects. A positive and statistically significant time-by-type interaction effect indicates that linkages increased over time between or among organizational types. The regression model was also re-estimated by using a random effects probit model to control for clustering of responses and attributes within organizations.

These 2 probit regression models were imperfect estimators of network effects, however, because they could not completely account for interdependencies that arose from being connected to the same other organizations in the network either directly or indirectly. The network evolution model^16,17 tests whether links were based on organization type after control for the network structural effects of density, reciprocity, and transitivity. For example, the likelihood that 2 organizations were linked depends in part on the overall density of the network (in a dense network everyone is more likely to be linked). The network evolution model can control for these structural effects and test the tendency of network choices made and received to be based on organizational type.^18,19

In the network evolution model, the 14 networks were analyzed separately and then combined by meta-analysis.²⁰ Each network was dichotomized on median values of the proportion of links for each network across both waves (14 medians, which ranged from 0.20 to 0.33). (A discussion of choices available to dichotomize these valued networks is available in a supplement to the online version of this article at http://www.ajph.org.) We applied the network dynamic model to each network and combined the results to produce vectors of parameter estimate means and standard errors across networks.

The probit regression, random effects probit regression, and network evolution model tested whether network linkage was a function of organization type and time. This analysis could indicate whether network change occurred as predicted. We conducted an additional set of analyses that used attendance logs to determine whether attendance at specific activities was associated with increased network linkage. Data from the attendance logs for each event were used to construct a co-occurrence network, which represented the number of people at each organization who jointly attended WINCART activities.^9,21 This joint attendance network was correlated with the reported network linkage via quadratic assignment procedure (QAP) regression,^22,23 which controlled for network dependencies by comparing the correlation between 2 networks with a sample generated by permuting the matrix rows and columns (available in a supplement to the online version of this article at http://www.ajph.org). We calculated a separate QAP regression on the change in each network linkage between times 1 and 2 and the number of people who co-attended the 15 events conducted during the study period.

The QAP correlation indicated whether co-attendance at events was associated with network change. To determine whether co-attendance was associated with network change based on organization status (CBO vs university), a 16-by-16 matrix was created with “1's” in cells representing links between CBOs and themselves and CBOs and universities. This matrix was multiplied (element-by-element multiplication) with the co-attendance matrix to construct an interaction term that indicated whether co-attendance by CBOs with other CBOs, and CBOs with universities, increased network linkage. The multiple regression QAP on linkage rate with these 3 matrices tests whether co-attendance by organization type was associated with network change.

RESULTS

Table 1 provides summary statistics indicating that there were 5.69 and 3.50 respondents per organization at times 1 and 2, respectively. Most respondents were female, 87.9% and 81.4% at times 1 and 2, respectively. Respondents were experienced with working in their organizations, averaging approximately 7 to 8 years working with their current organization. Most, 58.6% to 67.9%, had participated in WINCART activities, averaging 1.71 to 2.38 activities in the past year. There were 1426 links reported in response to the 14 network questions at baseline, and, despite the smaller number of respondents, there was an increase to 1617 links at time 2. There were 146 and 159 links within the organizations at time 1 and 2, respectively, omitted from the analysis (as well as the 75 links reported from the one national organization).

TABLE 1.

Sample Characteristics of Respondents From 16 Organizations Involved in WINCART Activities: June 2005 and July 2007

Characteristics	Time 1 (June 2005)	Time 2 (July 2007)
No. of responses	91	56
No. of responses per organization, mean	5.69	3.50
% female	87.9	81.4
Years working in current organization, mean	6.93	7.75
% participated in WINCART activities	58.6	67.9
No. of WINCART activities, mean	1.71	2.38
No. of links between organizations	1038	1129
No. of links within organization	146	159
No. of links from national organizations	0	75
No. of links to national organizations	242	254

Note. WINCART = Weaving an Islander Network for Cancer Awareness Research and Training.

The median linkage rate (percentage of respondents in the organization who nominated another organization) was 22% at time 1 and increased to 33% at time 2. At both time points, nominations were greatest in response to the question, “Which organizations have you communicated with in the past year?” (medians 30% and 50%, respectively). At time 1 the rate of connections was lowest for receiving clients (17%), but at time 2 it was lowest for referring clients (25%). Figure 1 illustrates the cancer education network—the ties between organizations derived from their responses to the question “which other organizations have you worked on non-cancer education activities with”—at times 1 and 2 with organizations depicted as CBOs (circles) or universities (squares).

FIGURE 1.

Cancer education network at (a) June 2005 (time 1) and (b) July 2007 (time 2).

Note. The links indicate greater than average (based on the median) communication between organizations. There is an increase in linkage from CBOs to universities but not from universities to CBOs. Figures were created with Netdraw visualization software version 2.087 (Analytic Technologies, Lexington, KY).

Probit regression and probit random effects regression results are presented in Table 2. The noncorrected results are only presented for comparison and are not discussed further. There was a nonsignificant increase in linkage rate over time (b = 0.17; P > .05), indicating a modest increase in density once other factors were controlled. There was a negative association with follow-up rate within the organization (b = −0.16; P < .01), indicating that organizations with fewer follow-up respondents had a high rate of connectivity. This may indicate that in organizations with a low response rate, the individuals with greater commitment and engagement were more likely to complete the follow-up survey.

TABLE 2.

Regression Results for Likelihood of a Link Among 16 Organizations Involved in WINCART Activities: June 2005 and July 2007

	Probit Regression, b	Random Effects Probit Regression, b
No.	2167	2167
Time 1 (Ref)	1.00	0.00
Time 2	−0.21	0.17
Follow-up rate within organization	−0.08***	−0.16**
Years at current organization time 1	−0.40***	−0.63*
Years at current organization time 2	0.20***	0.30
Networks
General networks (Ref)	0.00	0.00
Noncancer networks	−0.02	−0.03
Cancer networks	0.04	0.02
No. of WINCART activities at time 1	0.29***	0.41
No. of WINCART activities at time 2	0.75***	1.42**
Links
Link university to university (Ref)	1.00	0.00
Link CBO to CBO	−3.80***	−5.51*
Link CBO to university	−3.96***	−5.70**
Link university to CBO	−0.77*	−0.29
Status by time interaction
Link university to university, time 2 (Ref)	1.00	0.00
Link CBO to CBO, time 2	0.94***	0.44*
Link CBO to university, time 2	0.95***	0.46*
Link university to CBO, time 2	0.24	0.00

Notes. CBO = community-based organization; WINCART = Weaving an Islander Network for Cancer Awareness Research and Training.

*P < .05; **P < .01; ***P < .001, by the 2-tailed test.

Tenure, as measured by the average number of years that respondents in each organization worked at their organization, was negatively associated with linkage rate at time 1 (b = −0.63; P < .05) and positively (though not statistically significantly) at time 2 (b = 0.30; P > .05). There was a nonsignificant association with network type indicating that the linkage rate did not vary across the 3 network types (general, noncancer, and cancer). Participation in WINCART activities was positively associated with linkage rate at time 1 (b = 0.41; P > .05) and time 2 (b = 1.42; P < .01). The main effects for 2 combinations of organization type were negatively associated with linkage rate: CBOs to CBOs (b = −5.51; P < .05) and CBOs to universities (b = −5.70; P < .01). Dynamically, however, CBOs increased their rate of linkage with themselves (b = 0.44; P < .05) and with universities (b = 0.46; P < .05).

For the network evolution analysis (Simulation Investigation of Empirical Network Analysis [SIENA]), all 14 networks attained convergence with t-ratios being less than 0.1 in absolute value.¹⁸ The results of the meta-analysis across all networks indicated that there was a significant ego effect for CBOs indicating that they increased their outgoing linkages more rapidly than did universities (T² = 69.00; df = 14; P < .001; mean effect size = 1.15; P < .001). The alter, or incoming, effects (T² = 20.02; df = 13; P = .10; mean effect size = −0.64; P < .001) and similarity effects (T² = 14.49; df = 13; P = .34; mean effect size = 0.55; P < .01) based on organizational type were not significant, indicating no difference in nominations received by organization type and no difference in the likelihood of linkage between organizations of the same type. The estimated between-network standard deviation for the ego effect (outgoing) parameter along with those for incoming and similarity parameters were negligible, indicating similar effects across networks.

By using the 3 effect sizes (1.15, −0.64, and 0.55) and the values zero for universities and 1 for CBOs, one can calculate the formula for ego and alter effects based on status.¹⁸ The ego–alter selection tendencies were: university to university, 0.20; university to CBO, −0.99; CBO to university, 0.80; and CBO to CBO, 0.71. These ego–alter selection tendencies indicate that CBOs preferred relations with universities (0.80) or other CBOs (0.71), whereas universities preferred connections to other universities (0.20) and not to CBOs (–0.99). The regression and network evolution models analyses yielded similar results, indicating an increase in linkage from CBOs to universities but not from universities to CBOs.

The QAP regression of change in network linkage on co-attendance yielded 2 significant associations. Co-attendance was associated with an increased likelihood of linkage on (1) formal agreements and (2) cancer research. The QAP regression of change in network linkage on co-attendance, the matrix of “1's” indicating CBO links to themselves and universities, and the interaction of these 2 matrices produced 3 significant associations: (1) CBO links were associated with increased general communication, (2) CBO links were associated with increased cancer training, and (3) co-attendance was associated with increased likelihood of cancer research linkage.

DISCUSSION

Public health researchers have advocated CBPR as a means to bring evidence-based public health policies and programs to communities and to enable researchers to conduct community-informed research. Despite these goals, no studies have evaluated whether linkages among agencies involved in the CBPR process have changed as a result of interventions. In our study, we measured network linkages across 14 topics to determine whether linkages among and between CBOs and universities have changed as a result of project activities.

The WINCART initiative was created to initiate activities that bring CBOs in contact with each other and with university researchers to promote community-relevant studies that can explore and address the root causes of enduring cancer health disparities and, ultimately, contribute to a reduction in these disparities. Baseline and follow-up data were collected from multiple members of 16 organizations to measure network linkages among and between CBOs and universities. Statistical analysis showed that linkages from CBOs to other CBOs and from CBOs to universities increased during the study. By contrast, university faculty did not increase their ties to other universities or to CBOs. This suggests that WINCART was successful at motivating network change among the community partners but not among university researchers. There were several possible explanations for this finding.

First, although they were open to all network members, many of WINCART's activities were primarily attended by community leaders from CBOs. In addition, meetings were often held to bring CBOs together and foster sharing among themselves. The academic researchers present at these meetings were usually those who had a history of community work and so were not new to CBPR. Second, in many of the meetings, university researchers presented research and information relevant to the communities, but there was little face-to-face interaction and team building for new projects relevant to the academic faculty. Thus, the philosophy of WINCART, as well as that in other CBPR health disparities efforts, is often devoted to creating access for community members but not to increasing outreach among academics. Third, changing academic faculty is notoriously difficult, because academic researchers spend considerable effort becoming experts in well-defined areas, efforts that often preclude having time for community partnering. Finally, university culture and the incentives for faculty advancement are often at odds with the time it takes to conduct CBPR.¹

What was responsible for the increase CBO nominations to universities? Increased linkage was driven in large part by organized WINCART activities that had specific training and dissemination functions, rather than large meetings with formal presentations. The QAP regressions reported in this study indicate that joint attendance at meetings may have been less responsible for network changes than expected. Joint attendance at WINCART meetings was associated with increased linkage in the formal agreements and cancer research networks and increased general communication and cancer training for CBOs. Meetings designed to facilitate interaction need to have specific skill building and outcome goals built into their agendas and to be followed up with the additional effort needed to achieve these goals.

Interestingly, tenure also played a role, but in a complicated way as revealed in the probit regression. Tenure at time 1 decreased linkage, whereas tenure at time 2 increased it (though the association was marginally significant; P = .07). This indicates that organizations with respondents who reported having been at their organization longer may have been less connected at time 1 but increased that connectedness as a consequence of WINCART. Additional analyses not reported revealed this association to be strongest among CBO respondents, indicating that individuals at CBOs with more experience reported fewer linkages at time 1 yet increased their linkage rate to other organizations at time 2 (P < .01). Those with little experience were perhaps younger and better connected to different organizations at time 1, whereas the older, more-experienced employees gained more from project activities.

We used 3 different statistical approaches in this study to yield different perspectives to the evaluation. The probit regression analysis of linkage rate included several organizational attributes such as organization type, program participation, and tenure. This is the most familiar type of statistical analysis and tested whether linkage increased between CBOs and universities. The network evolution model tested the likelihood of a link between organizations based on organization type (CBO vs university) and controlled for network structural effects (density, reciprocity, transitivity) ignored in the probit regression. The QAP regression measured the correspondence between network linkages and event attendance to determine whether event coparticipation was associated with linkage change.

These 3 approaches are complementary, providing different insights into the determination of program effects. Linkages increased between CBOs and universities independent of the overall network structure (the tendency for mutuality and transitivity) yet dependent on respondent tenure and the number of events attended. Co-attendance at events by itself, however, only increased linkages regarding cancer research. Linkage increase in other networks presumably occurred because of other WINCART activities not explicitly measured as part of the program evaluation.

This study has at least 2 limitations that temper interpretation of results. Although there was at least 1 respondent from each organization, the response rate at follow-up (49%) was less than desired. A second limitation concerns the nature of community–academic partnerships. In some cases, university researchers may not have been in contact with CBOs directly because they prefer working with other researchers who have stronger ties with the community. For example, an epidemiologist may increase his or her community involvement by strengthening a partnership with another researcher within his or her university who already has strong ties to the community rather than establishing those ties himself or herself. Such an event would not be registered in this evaluation because the community-based researcher already has ties to the community. Further, this evaluation omitted ties within organizations and so such increases would not be detected.

In spite of these limitations, this research has shown that linkages between community and university researchers can increase through a CBPR network, which is a necessary precursor to determining whether the subsequent CBPR activities can effectively address cancer health disparities in these communities. By increasing their interaction, researchers and community members will gain a stronger understanding and appreciation for each others' strengths and constraints. Academicians learn how to incorporate community members' concerns and perspectives, making their research more community-informed and informative. Community members learn to anticipate researchers' needs and desires so that their research better serves community interests.

The research we describe also demonstrated the utility of network analysis as a tool for evaluating the effectiveness of community-based research and interventions. Many interventions are designed specifically to increase linkages within and between groups such as community, civic, governmental, judicial, legislative, academic, and so on.^9,14 However, it is critical to understand who is specifically linked to whom. For example, knowing that a specific CBO is well-connected to one university and not another has implications for understanding program effects and making programmatic recommendations. In this case, discovering change among CBOs without concomitant change among university researchers provides prescription for planning future activities such as more committed outreach to university faculty (and identifying specifically which university and CBO organizations should form partnerships). This study also suggests that network analysis methodology can make explicit the many intangible processes vital to successful CBPR.