Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2012 Nov 1.
Published in final edited form as: Soc Ment Health. 2011 Nov 1;1(3):185–199. doi: 10.1177/2156869311431613

An Exploration of the Effect of Community Engagement in Research on Perceived Outcomes of Partnered Mental Health Services Projects*

Dmitry Khodyakov 1, Susan Stockdale 2,**, Felica Jones 3, Elizabeth Ohito 4,**, Andrea Jones 5, Elizabeth Lizaola 6, Joseph Mango 7,**
PMCID: PMC3349344  NIHMSID: NIHMS336077  PMID: 22582144

Abstract

Mental health research projects address sensitive issues for vulnerable populations and are implemented in complex environments. Community-Based Participatory Research approaches are recommended for health research on vulnerable populations, but little is known about how variation in participation affects outcomes of partnered research projects. We developed a conceptual model demonstrating the impact of community engagement in research on outcomes of partnered projects. We collected data on key constructs from community and academic leaders of 21 sampled partnered research projects in two cycles of an NIMH research center. We conducted empirical analyses to test the model. Our findings suggest that community engagement in research is positively associated with perceived professional development, as well as political and community impact.

Community-Based Participatory Research (CBPR) is increasingly used to improve health services quality and outcomes for vulnerable populations, including those with mental health disorders (Jones and Wells 2007; Wallerstein 2006). CBPR entails genuine collaboration between academic and community partners in all aspects of the research project (Baldwin, Johnson, and Benally 2009; Shattell et al. 2008; Wells et al. 2004). With the CBPR approach, community health interventions can be designed and implemented to align well with the existing evidence base and better fit the needs of community, are more likely to be sustained, and can better facilitate coordination of research and policy efforts intended to reduce health disparities (Israel et al. 2001; Minkler 2010). Although the development and structure of academic-community partnerships vary, there is a growing trend toward more inclusive and equal collaboration between academic and community partners (Jones and Wells 2007; Minkler 2005a).

Previous studies suggest that participation of community partners in research aspects of partnered projects helps facilitate development of research questions that reflect issues important to community, improves cultural appropriateness of measurement tools, increases community trust and ownership of research projects, and improves accuracy and cultural sensitivity in the interpretation of findings. Consequently, academic-community research collaboration may increase the relevance of intervention approaches and therefore improve their chances of success (Butterfoss, Goodman, and Wandersman 1993; Minkler 2005a). Nonetheless, some research suggests that partnered projects are time consuming (Corbie-Smith et al. 2003), require high levels of commitment from both academic and community partners (Minkler 2004), and pose certain threats to the validity and reliability of jointly collected data (Sloane et al. 2003).

While partnered research projects have the potential to address pressing community health issues, little empirical evidence exists on the impact of degree of community partners’ engagement in research on outcomes of partnered projects, especially if they deal with mental health issues. Although some recent conceptual models link partnership context, group dynamics, intervention/research variables, and outcomes measures (Wallerstein and Duran 2010; Wallerstein et al. 2008), empirical research often focuses primarily on the impact of partnership structure on process measures, such as partnership synergy (Lasker, Weiss, and Miller 2001; Weiss, Anderson, and Lasker 2002), rather than perceived and/or objectively measured personal, scientific, or community benefits.

To fill this gap, we conducted an exploratory study that develops a conceptual model of partnership evaluation and examines how community partners’ engagement in research affects proximal partnership outcomes (i.e., partnership synergy (Lasker et al. 2001; Weiss et al. 2002)), perceived personal-level outcomes, and perceived community/policy-level outcomes. Our study directly responds to a recent call for empirical analyses of the impact of community engagement in research on the process and outcomes of partnered research projects (Minkler 2010; Minkler et al. 2008) by focusing on community-academic partnerships in the area of mental health in underserved populations. People seeking mental health treatment are often socially stigmatized (Byrd et al. 2010; Johnson et al. 2010), and community participation in this type of behavioral health research is not encouraged due to historically high levels of distrust in the medical field and in research (Shavers-Hornaday et al. 1997).

We used a partnered approach (Jones and Wells 2007) to conduct a mixed-methods evaluation of 21 mental health-related partnered research projects affiliated with the NIMH UCLA/RAND Center for Research on Quality (CRQ) in Managed Care and its successor, the NIMH Partnered Research Center (PRC) for Quality Care. By examining variations in degree of community partners’ engagement in research and different measures of partnership outcomes, our study may help community and academic partners better understand how to build effective research partnerships that have a meaningful impact in community.

LITERATURE ON RESEARCH PARTNERSHIPS AND OUR CONCEPTUAL MODEL

Our conceptual model draws primarily from CBPR evaluation research and the community health coalition literature. Most partnership evaluation models explain partnership effectiveness as a function of intra-group dynamics and focus on how group processes affect partners’ perceptions of group effectiveness (Butterfoss et al. 1993; Israel et al. 2005; Weiss et al. 2002). These models have developed rigorous measures of perceived outcomes, such as partners’ satisfaction, leadership quality, and goal clarity, among others (Wallerstein, Polascek, and Maltrud 2002). For example, a widely cited approach to evaluating CBPR partnerships developed by Schultz and colleagues (Schulz, Israel, and Lantz 2003) examines the impact of leadership, communication, conflict resolution, and power and resource sharing on perceived partnership capacity and effectiveness. More recent models, such as one developed by Wallerstein et al. (2008), however, emphasize partnership outcomes by measuring changes in policies and partners’ capacity, as well as improvement in population health.

Grounded in the health coalition literature, Lasker et al.’s (2001) theoretical model of partnership effectiveness emphasizes the importance of collaboration in explaining the advantage of partnerships over individual efforts to improve health issues in community and includes group dynamic measures (a.k.a. partnership functioning). It hypothesizes that partnership functioning affects partnership “synergy,” a proximal partnership outcome that refers to how the resources, perspectives, and skills of partners strengthen the work of the group. Synergy, in turn, positively impacts community-level outcomes. To date, empirical evaluation of the synergy model has tested only the relationship between partnership functioning and synergy and not the impact of partnership synergy on community/policy-level outcomes (Weiss et al. 2002). This may be partly explained by the lack of valid and reliable partnership outcome measures, a methodological challenge recently identified in the literature (Granner and Sharpe 2004).

Because we are interested in examining how variations in research partnerships impact outcomes for participants and communities, we modified the partnership synergy model (Lasker et al. 2001; Weiss et al. 2002) by adding partnership characteristics and perceived personal- and community/policy-level outcomes. Our conceptual model (see Figure 1) suggests that partnership characteristics impact partnership functioning, as well as synergy and perceived personal and community/policy-level outcomes; partnership functioning influences partnership synergy; and synergy influences both perceived personal- and community/policy-level outcomes. By including partnership characteristics, our model acknowledges the importance of three key features of the CBPR approach: 1) equal engagement of community partners in all aspects of research; 2) inclusion of diverse community perspectives; and 3) long term relationships between partners.

Figure 1.

Figure 1

Open in a new tab

Conceptual Model for Evaluating Academic-Community Research Partnerships

Partnership characteristics, specifically inclusiveness, size, and previous history of partnering, may have both direct and indirect effects on perceived personal and community level outcomes. Previous studies demonstrate that research projects benefit from community engagement at all levels - from formulating the research questions, to developing research protocols, to analyzing the data (Cashman et al. 2008), affecting both partnership functioning and outcomes. However, research on the inclusiveness of partnerships is mixed: while larger, more diverse partnerships may enjoy more personal- and community/policy-level outcomes because of the increased “critical mass” behind the community effort (Butterfoss et al. 1993), partnership functioning and the process of developing synergy may be faster and less complicated in smaller partnerships where fewer partners are engaged in the decision-making (Ndirangu et al. 2008). Larger projects, however, may also experience higher partner turnover (Bluthenthal et al. 2006), but losing one partner may be more detrimental for smaller partnerships. Furthermore, building trust among partners takes time and is crucial to producing tangible outcomes (Green, Daniel, and Novick 2001). Research shows that prior relationships between partners help reduce the amount of time needed to establish open communication, which is important for successful partnership functioning (Plowfield, Wheeler, and Raymond 2005).

As suggested by previous research (Granner and Sharpe 2004; Lasker et al. 2001), partnership functioning may also have an impact on partnership outcomes. Diversity among partners requires strong formal and informal project leadership (Khodyakov et al. 2009). Leaders must guide the development of a shared understanding of project goals and create a common language among partners to ensure that everyone is on the same page. They have to ensure that partners’ personal agendas overlap yet do not conflict with those of the partnership (Quandt, Arcury, and Pell 2001). Leaders should be able to effectively mediate disagreements as they arise (Armistead, Pettigrew, and Aves 2007) and create a sense of inclusiveness and openness to help partners share resources, ideas, and creatively solve problems. While leadership and management are important, numerous studies illustrate the negative impact of social pressure on the decision-making process (Asch 2003; Ridgeway 1978). For the partnership to have legitimacy, partners must be comfortable with how decisions are made and support the decisions made by the group (Butterfoss et al. 1993).

Efficiency is also crucial to partnership functioning, as participation in a community-academic partnered project often includes additional responsibilities for both sides. Competing demands for partners’ time and attention is a known barrier in academic-community partnerships (Quandt et al. 2001). More often than not, community partners’ time spent on a partnered project is not compensated financially (Kone et al. 2000). Therefore, partnered projects must use limited financial and non-financial resources of all partners adequately and efficiently. For similar reasons, partnered projects may need effective governance mechanisms to manage partnership activities (Higgins and Metzler 2001). For example, partners may have difficulty scheduling in-person meetings and therefore have to meet by phone (Quandt et al. 2001). Establishing effective channels of regular communication to coordinate partnership activities is essential for smooth functioning of the project. Project managers have a particularly difficult role because they facilitate interactions, prepare materials and meeting agendas (Srinivasan and Collman 2005), keep the project on track, and bring new partners up to speed (Weiss et al. 2002).

Effective use of non-financial resources becomes crucial for partnered projects, because community partners often feel that money is not being allocated equitably between academic and community partners and become frustrated about funding agency restrictions for spending project funds (Wolff and Maurana 2001). Most research partnerships are either underfunded from the very beginning or lack stable funding to ensure long-term sustainability (Srinivasan and Collman 2005). Projects that have access to necessary data and information, connections to politicians and government officials, diversity among participating community organizations, and are endorsed by prominent local leaders may be able to achieve intended results more effectively and efficiently (Minkler 2005b).

Finally, challenges arising from involving diverse partners can impact partnership outcomes. For many community organizations, participation in research projects creates a burden on staff members who must attend meetings, perform extra duties, and take on leadership roles in addition to their regular responsibilities. Such extra-role demands create challenges in recruiting, retaining, and motivating essential partners. Partners in large community-based projects tend to join and then drop out of projects, often due to economic challenges their organizations face (Bluthenthal et al. 2006; Jones and Wells 2007).

As a proximal partnership outcome, our conceptual model includes partnership synergy, an outcome of partnership functioning that facilitates the accomplishment of more than what can be accomplished by individual partners on their own (Lasker et al. 2001; Weiss et al. 2002). While academic partners contribute their research expertise and knowledge of evidence-based approaches to mental health treatment, community partners bring their understanding of community needs and knowledge of the local context. By supporting each others’ efforts and capitalizing on complementary strengths and areas of expertise, such collaborations also result in more comprehensive and sustainable programs, as well as stronger relationships with the community at large (Weiss et al. 2002). Collaboration allows partners to see different aspects of the problem and work constructively together to develop solutions that go beyond their limited individual capacities and resources (Gray 1989).

Our model suggests that partnership characteristics and synergy are associated with perceived personal and community/policy-level outcomes. Academic and community partners may join and remain in collaborative relationships to achieve their own personal goals. Rational choice theory suggests that collaboration is possible only if expected outcomes outweigh the transaction costs associated with working together (Feiock 2007). In addition, the goals of academic and community partners may be different (Wallerstein 1999). Academics may be motivated to collaborate by their desire to apply theoretical knowledge to the empirical problems in community. Community partners, in turn, may be motivated by an expectation of heightening their public profile within the community or additional funding/resources.

Ultimately research partnerships can build community capacity (Wallerstein et al. 2002), or the ability of communities to identify, mobilize, and address pressing health problems through planning, developing, implementing, and maintaining effective community programs (Poole 1997). Partnered projects can also bring new programs to community and introduce social change that improves quality of life and healthcare (Wallerstein and Duran 2010; Wallerstein et al. 2008).

METHODS

We tested our conceptual model using data on community partnerships affiliated with an NIMH-funded center. Due to sample limitations and size (see below), however, our results can only be considered exploratory. In the discussion section, we outline directions for further research to improve our understanding of how community engagement in research affects the outcomes of partnered projects at the personal and community/policy levels.

Sampling and Data Collection

We sampled projects from two cycles of an NIMH-funded center, providing a unique opportunity to evaluate a range of possible approaches to community-academic partnerships. The NIMH UCLA/RAND Center for Research on Quality (CRQ) in Managed Care focused on improving the sustainability and dissemination of evidence-based treatments for major psychiatric disorders and service delivery interventions that support the use of evidence-based treatments. Its approach involved adapting such interventions to diverse local practices, community settings, and populations; however, it did not adopt a specific approach to building and maintaining research partnerships. A community-academic collaborative that evolved in the later years of the center set the stage for the partnered planning of a renewal that would be guided by Community Partnered Participatory Research principles (Jones and Wells 2007; Khodyakov et al. 2009). The new center, the NIMH Partnered Research Center (PRC) for Quality Care, focused primarily on improving access to quality mental health services in communities through studying the impact of interventions at policy, practice, and community levels. Projects developed at PRC emphasized partnering in research, program development, implementation, and evaluation to achieve science that is informed by, and can inform, community-based services delivery. More fundamentally, the center was created to support community and academic partners in formulating, designing, implementing, and publishing such research. Each component of the PRC, as well as most center-affiliated projects, is led by a community and an academic PI including the Executive Committee, which is the decision-making body for the Center.

Projects affiliated with both centers represent a diversity of partner organizations, including universities, schools (K-12), the Veterans Health Administration, Los Angeles County and California State Departments of Health and Mental Health, RAND, health insurance companies, community advocacy agencies, faith-based organizations, homelessness agencies, and other community-based agencies that provide services for individuals with behavioral health problems. Independent sources of funding for individual projects were obtained as a result of collaboration between community partners and academics through an inclusive process for gaining community input into methods design and selection for new proposals. Many of the partnerships developed under or pre-dated the CRQ; most PRC-affiliated projects were based on previous partnerships with successful project outcomes, or well-established relationships between community and academic principle investigators. At the CRQ, many projects served as an introduction to research for the community partners. These projects developed community capacity as partners familiarized themselves with the research process through informal journal clubs, workshops, and works in progress meetings, among other activities. Community partners, in turn, helped build academic capacity by addressing issues of cultural sensitivity, raising practical problems, and suggesting unexplored areas of intervention development.

At the PRC, most community partners are well versed in the research process and are recognized as Co-PIs on projects, and two-way capacity building activities are a hallmark of the second center. Funding for this center supports administrative infrastructure and facilitates the sustainability of partnerships that continue to evolve as new projects are developed. Consequently, while PRC projects generally included at least one stable partnership with a community partner, they added new community partners to broaden their existing networks. Both centers enabled the development of academic-community network that is able to quickly and efficiently identify new research directions and pursue new funding opportunities. The resulting network density, which was unmeasured in this project due to the nature of our sampling approach (see below), may have contributed to the perceived success of partnership outcomes.

We chose the project, rather than the partnership, as the unit of analysis because our outcome measures were project-specific. We included center-affiliated projects only if they met the following 3 selection criteria: (1) the PI was either a Center PI, co-PI, or key personnel; (2) the project focused on mental health and/or substance abuse issues; (3) the project began between July 1, 2003 and June 30, 2009. After reviewing multiple center reports, we identified 72 projects led by 22 PIs that potentially met these inclusion criteria. To minimize the burden on participants, we randomly sampled up to two projects per academic PI (n=39) for an in-depth analysis, including the qualitative interview with the academic PI and the lead community partner and a survey of key academic and community partners working on that project. The remaining 33 projects were included for information verification purposes during the interview with the PI.

Between March and June of 2010, we conducted semi-structured qualitative interviews with academic PIs for all 39 sampled projects and with lead community partners for 15. We used the qualitative component to refine partnership functioning measures from Lasker and colleagues’ synergy model (Lasker et al. 2001; Weiss et al. 2002) and develop the survey items to measure community engagement in research and project outcomes. These interviews revealed that some projects did not actually meet our inclusion criteria (4 projects did not have community partners and 8 were data analysis or literature review studies), and PIs of 4 sampled projects decided not to participate in the quantitative component of this study. Therefore, our sample for the survey included only 23 of the original 39 projects.

During the qualitative interviews, we asked PIs and community leads for a list of the academic and community “key personnel” who worked on the sampled projects to use for survey sampling. 88 individuals, including project PIs, were identified, many of whom were affiliated with more than one partnered project. To minimize respondent burden, we assigned individuals to complete the survey for only one project. If an individual worked on multiple projects, we assigned him/her to the project that had the fewest personnel listed. This strategy allowed us to assign at least one academic and one community partner to 21 out of 23 sampled projects. Because we sampled up to two projects per PI yet asked each person questions about only one project in the survey, we were not able to get the PI’s perspective for every project.

Surveys were conducted online by using SelectSurvey software between June and August of 2010. Of 88 invited, 62 respondents (25 community and 37 academic partners) participated in this survey, yielding a 70% response rate. For 15 projects, both an academic and a community partner completed the survey; for the remaining 6 projects, only academic partners’ input was obtained. The response rate among academics was higher than among community partners (86% vs. 56%). While most of the community survey participants were white females between the ages of 40 and 49 with a master’s degree, most academics were white males either between the ages of 30 and 39 or 40 and 49 with a professional or doctoral degree. Due to a relatively low community response rate, the community survey participants may not accurately reflect the characteristics of community project personnel, many of whom are non-white.

Measures

We developed several survey items based on the results of our interviews; we describe them in detail below. We also adapted several multi-item measures from Weiss et al.’s synergy model (Weiss et al. 2002) and constructed indices from these items. The questionnaire with the adapted items is available from the first author upon request. Exploratory principal components analysis with promax rotation was conducted on each set of items, and we constructed composite measures based on our results (see Table 1 for means, SD, and alphas). Our results cannot be considered an empirical validation of Lasker et al.’s synergy model because the wording and response scales of some items were altered from the original.

Table 1.

Descriptive Statistics (N=62)

Scale/Index Mean SD Alpha
Index of Perceived Community/Policy-Level Outcomes (IPCPLB) [Range: 1-5] 3.23 1.36
Capacity-building [Range: 1-5] 4.31 .70 .862
Political impact [Range: 1-5] 3.51 .86 .736
Professional development [Range: 1-5] 4.34 .66 .574
Partnership synergy [Range: 1-5] 4.15 .63 .927
Community Engagement in Research Index (CERI) [Range: 4-12] 8.73 2.33
Number of partnering organizations 6.73 4.18
Ongoing project (partnership duration) [Yes=1] .42 .50
Leadership [Range: 1-5] 4.37 .71 .918
Efficiency [Range: 1-5] 4.27 .71 .855
Inclusion in decision-making [Range: 1-5] 4.39 .47 .681
Exclusion from decision-making [Range: 1-5] 1.76 .91 .790
Managing partnership activities (governance mechanisms) [Range: 1-5] 4.28 .60 .875
Open in a new tab

Community/policy-level outcomes

During our qualitative interviews, we asked participants to name any community-level outcomes they thought their partnered project had achieved. We then compiled these data, identified the most frequently named outcomes, and included the following as individual items in the survey: (1) improved the access, delivery, and/or quality of mental health / substance abuse services in the community; (2) resulted in sustained partnerships among agencies providing care/services for individuals with behavioral health disorders; (3) resulted in policy changes at county or state level (DMH or Department of Public Health/Health Services or other county-level government agencies); (4) improved the overall mental health of individuals in the community; (5) received public recognition or acknowledgement from local policy makers and/or government official. We instructed survey respondents to indicate whether these community-level outcomes were achieved or will be achieved for their projects, depending on completion status of the project (most of the projects were still on-going). From these items, we created an individual-level Index of Perceived Community/Policy-Level Benefits (IPCPLB) by summing the items each person endorsed.

Personal-level outcomes

These items were adapted from the synergy instrument (Weiss et al. 2002). Participants used a five-point scale (1=not at all and 5=to a very great extent) to assess their experience of 11 personal benefits resulting from participating in a partnered project. We constructed three indices based on the results of our factor analysis. The capacity-building index was calculated as the average score across five items: (1) acquisition of useful knowledge about services, programs, or people in my constituency or community; (2) development of valuable relationships; (3) enhanced ability to meet the needs of my constituency or clients; (4) ability to have a greater impact than I could have on my own; (5) ability to make a contribution to my constituency or community. Political impact was the average score across four items: (1) heightened public profile; (2) enhanced ability to affect public policy; (3) increased utilizati on of my expertise or services; (4) acquisition of additional financial support. Professional development was the average score across two items: (1) enhanced ability to address an important issue and (2) development of new skills.

Partnership Synergy

Using a five-point scale (1=not well at all and 5=extremely well) participants rated 11 statements from the original synergy instrument (Weiss et al. 2002) about the people and organizations that participated in their partnered project. The synergy score was calculated as the average across all 11 items.

Partnership Functioning Measures

We adapted items from Weiss and Lasker to measure multiple dimensions of partnership functioning (Weiss et al. 2002) and construct indices for leadership, decision-making, financial and non-financial resources, governance mechanisms, efficiency, and partnership challenges. All of the items comprising the indices were measured on different five-point scales (1=low, 5=high); the measures were constructed by taking the average of items that loaded together onto each factor. Not all partnership functioning measures were identical to those used by Weiss and Lasker.

Partnership Characteristic Measures

We created measures of community engagement in research, partnership size, and duration. In this project, community participation/engagement in research was operationalized as participation of lead community organizations’ representatives rather than community at large. We developed 12 items to measure community partners’ engagement in research activities based on our qualitative interviews with the project academic PIs and community leads: (1) grant proposal writing; (2) background research; (3) choosing research methods; (4) developing sampling procedures; (5) recruiting study participants; (6) implementing the intervention; (7) designing interview and/or survey questions; (8) collecting primary data; (9) analyzing collected data; (10) interpreting study findings; (11) writing reports and journal articles; (12) giving presentations at meetings and conferences. All survey respondents were asked to rate the extent to which community partners were involved in each research activity on the following three-point scale: 1=Community partners did not/do not participate in this activity; 2=Community partners consulted on this activity; 3=Community partners were/are actively engaged in this activity. We created the Community Engagement in Research Index (CERI) by summing these scores across the twelve activities and dividing by three. CERI scores have a possible range from 4=low engagement to 12=high engagement.

As a proxy for partnership size, we calculated the number of academic and community partner organizations working on each project based on the information provided by the academic PI during the qualitative interview. Partnership duration was assessed by asking survey participants whether their project was ongoing or completed.

Analysis

We explored the relationship between partnership characteristics, functioning, synergy, and personal- and community/policy-level outcomes by running several series of regression models. Due to the clustered nature of our sample (62 participants reporting on their experiences working on 21 projects), we used multi-level modeling with SAS PROC MIXED to calculate random intercept models (Wang 1997). We ran OLS regression to facilitate interpretation of findings for models with non-significant random effects. Our relatively small sample size did not allow us to test models that included all of the conceptually important variables simultaneously; therefore, our study can only be considered exploratory.

We first modeled the relationship between partnership characteristics and functioning (using each of the functioning variables as outcomes) and then between partnership functioning and partnership synergy, including each partnership functioning variable one-at-a-time while controlling for partnership characteristics. We also modeled the direct relationship between partnership characteristics and synergy. Next we examined the impact of partnership characteristics, functioning, and synergy on personal- and community/policy-level outcomes. In preliminary modeling, we included a partnership duration variable as a proxy for partnership length. This variable was not significant in any of the models and was excluded in the final modeling. We present results only for the final stage of modeling personal- and community/policy-level outcomes (see Table 2). Results for the preliminary modeling stages can be obtained by contacting the first author.

Table 2.

Models Predicting Perceived Outcomes of Partnered Projects

Personal-Level Outcomes Community/Policy-Level Outcomes
Model 1: Professional Development Model 2: Political Impact Model 3: Capacity-Building Model 4: IPCPLB
B SE B SE B SE B SE
Intercept 3.432 (.572) 1.151 (.736) 1.501 (.532) -1.197 (1.078)
Synergy .035 (.123) .319 (.166)† .516 (.121)*** .611 (.248)*
CERI .073 (.035)* .080 (.046)† .056 (.033) .139 (.068)*
Partnership Size .016 (.03) .048 (.028)† .032 (.018)† .103 (.036)**
Open in a new tab

Notes: Unstandardized coefficients and standard errors shown in the table.

†

p ≤ .1,

*

p ≤ .05;

**

p ≤ .01;

***

p ≤ .001

For models predicting professional development and political impact, coefficients are from mixed-effects models. For models predicting capacity-building and IPCPLB, coefficients are from OLS models.

RESULTS

Descriptive Statistics

Table 1 shows the distribution of our independent and dependent variables. On average, there were 6.73 (sd=4.18) partner organizations working on sampled projects. The mean value of the CERI was 8.73, sd=2.33 (range 4-12), suggesting that, on average, community partners were engaged at the level of consultation (as opposed to “full engagement” in research). Although not statistically significant, academic partners tended to report higher engagement scores as compared with community partners (9.27 and 8.38, respectively). Survey participants tended to rate partnership functioning highly: leadership, inclusion into decision-making, efficiency, governance, financial resources, social and human capital all had an average score equal to or greater than four (on five-point scales). We found no statistically significant differences between men and women, or between academic and community partners.

Survey participants also reported high levels of partnership synergy, professional development, and capacity-building (scores above four on a five-point scale) and moderate to high levels of political impact and IPCPLB values (greater than three on five-point scales). Community partners had significantly higher values on the IPCPLB than academic partners (3.65 vs. 2.88). This difference can be explained by the fact that all community partners reported that their project has led/will lead to sustained partnership among agencies and improved/will improve overall mental health in community. The personal-level outcomes are positively and relatively strongly correlated with each other, with Pearson correlation coefficients ranging from .497 to .652. Moreover, personal-level outcomes are also positively correlated with IPCPLB, with correlation coefficients ranging from .357 (professional development) to .512 (capacity-building).

Partnership Characteristics, Functioning, and Synergy

In the first stage of modeling the association between partnership characteristics (CERI and partnership size) and partnership functioning, we found only one marginally significant effect. Participants working on projects with higher levels of community engagement in research (CERI) tended to have a slightly higher yet only marginally significantly different score on partnership involvement challenges (b=.064, sd=.036, p=.082). This suggests that partnership characteristics are not directly associated with functioning. Partnership characteristics also were not associated with synergy.

In the second stage, we estimated a series of models for the effect of partnership functioning on synergy, controlling for CERI and partnership size. Largely supporting previous findings (Weiss et al. 2002), we found a significant association between each partnership functioning variable and synergy, except for exclusion from decision-making and partnership challenges, which were marginally significant. These associations remained significant with CERI and partnership size included in the model, suggesting that partnership functioning does have a significant impact on synergy, even after controlling for partnership characteristics.

Predicting Perceived Personal- and Community/Policy-Level Outcomes

In the third stage of modeling, we estimated random-intercept models with the SAS PROC MIXED procedure (Wang 1997) to examine the impact of partnership characteristics, functioning, and synergy on personal and community/policy-level outcomes (Table 2). The between-project variance components for “null” models (i.e., no predictor variables) predicting professional development and political impact were large (45% and 33% of the total variance, respectively) and statistically significant (p=.02 and p=.05, respectively). ICC in the professional development model decreased from .45 in the “null” model to .394 for the “full” model. Likewise, ICC in the political impact model decreased from .329 in the model with no predictors to .104 for the “full” model. This suggests that predictors in the “full” model account for at least some of the variance between projects. Project-level variance components from the random-intercept models were statistically significant for the professional development and political impact outcomes, but not for capacity-building or the Index of Perceived Community/Policy-Level Benefits (IPCPLB).

In the models predicting the effect of partnership characteristics on personal-level outcomes, CERI had a significant positive association with professional development (b=.08, sd=.03, p=.03) and capacity-building (b=.08, sd=.04, p=.046) and a marginally significant positive association with political impact (b=.09, sd=.05, p=.068). In the models predicting the effect of synergy on personal-level outcomes while controlling for partnership functioning and characteristics, none of the functioning variables were significant. Therefore, we excluded them from our analysis. Table 2 demonstrates that, while professional development (Model 1) is significantly associated only with CERI, capacity-building (Model 3) is significantly associated with synergy and marginally associated with partnership size. Political impact (Model 2) is marginally associated with all three predictors - synergy, CERI, and partnership size.

We used logistic regression to estimate models for each of the community/policy-level outcome items separately. Bivariate associations between CERI and some community/policy-level outcomes were significant and positive: active community engagement in research is associated with sustained partnerships among agencies providing care/services for individuals with behavioral health disorders (r=.393, p=.006); with policy changes at county or state level (Department of Mental Health or Department of Public Health/Health Services or other county-level government agencies) (r=.401, p=.013); and with receiving public recognition or acknowledgement from local policy makers and/or government officials (r=.307, p=.036). However, we found no significant effects in the logistic regressions modeling the effects of partnership characteristic, functioning, and synergy on any of the community/policy-level benefits.

In addition to the logistic regression models examining each community/policy-level outcome separately, we examined the association of partnership characteristics, functioning, and synergy with the Index of Perceived Community/Policy-Level Benefits (IPCPLB). In the model with partnership characteristics only, the effects of CERI and partnership size on IPCPLB were significant (CERI: b=.181, sd=.071, p=.014; Partnership size: b=.118, sd=.039, p=.004). A series of OLS regression models to examining the association between IPCPLB and synergy, while controlling for partnership functioning and characteristics, revealed no significant effects for the partnership functioning variables; therefore, we excluded these variables from our analysis. Model 4 (see Table 2) demonstrates that synergy, CERI, and partnership size all are significantly and positively associated with IPCPLB.

DISCUSSION

In this paper, we tested a conceptual model illustrating how community-based partnerships can improve research, professional opportunities for both academic and community partners, and the health care system in general. Our conceptual model expands on previous work evaluating outcomes of community-academic partnerships by conceptually linking the extent of community engagement in research with perceived projects’ outcomes. It suggests that community engagement in research and partnership size impact both partnership functioning and outcomes, and that partnership functioning influences partnership synergy, which, in turn, affects perception of outcomes.

To explore the empirical validity of our model, we conducted a mixed-methods evaluation of 21 academic-community partnerships around mental health projects affiliated with an NIMH-funded center. Our empirical analyses found evidence to support some, but not all, components of our conceptual model. We found that more active engagement of community partners in research is indeed positively associated with a number of perceived community/policy-level outcomes, including more sustained partnerships among agencies providing care/services for individuals with mental health disorders; policy changes at county or state level; and public recognition or acknowledgement from local policy makers and/or government officials. Higher levels of engagement are also associated with a greater number of perceived community/policy-level benefits.

Moreover, our findings suggest that partnerships with higher levels of CERI are more likely to contribute to the perception of positive professional development of their members, whereas larger partnerships with higher levels of synergy may positively affect partners’ perception of personal capacity. Finally, larger projects that actively engage community partners in research and have higher levels of synergy may yield higher levels of perceived political impact and a greater number of perceived community/policy-level outcomes. These results imply that active engagement of community partners in research may increase partners’ ability to address important issues and develop new professional skills, and that partners working on projects with synergistic relationships are more likely to perceive personal-level benefits, such as increased personal capacity and political impact. Both of these conclusions emphasize the value of active community engagement in research and building partnership synergy between community and academic partners.

However, we note a lack of significant relationship between community engagement in research and personal capacity building. While this relationship may become significant in a larger and more diverse sample of projects, it may be that the partnership size and the degree of synergy among partners, rather than the level of community participation in research activities, play a more significant role in strengthening partners’ capacity. Members of larger and more synergistic partnerships can widen the scope of their personal networks and therefore experience increased personal capacity by building valuable relationships with representatives of organizations that they otherwise may not have been collaborating with, improving their ability to have a greater personal impact in community.

Furthermore, because partnership characteristics and synergy are significantly associated with IPCPLB, active engagement of a large number of partnering organizations in research and the development of synergistic relationships may help partnered projects increase their relevance, cultural sensitivity, and feasibility in a given community. While partnership size has only a marginally significant impact on the perception of personal-level outcomes, it is the strongest predictor of the IPCPLB. Larger projects may have more visibility in community and thus are more likely to have enough critical mass necessary for making a significant impact (Butterfoss et al. 1993). Consequently, engaging more organizations in partnering around mental health projects may reduce the stigma associated with such disorders and encourage research participation among underserved populations.

As predicted by our conceptual model, we found a positive relationship between partnership functioning and synergy, which supports the previous research indicating that projects with better leadership, more efficient use of financial and non-financial resources, more inclusive decision-making, and better management of partnership activities achieve greater levels of synergy (Weiss et al. 2002). At the same time, we found no significant direct effects of partnership characteristics on synergy as suggested by our conceptual model. However, this may have been a result of unmeasured similarities among the partnered projects studied here. Partnership characteristics may be more important for comparisons between academic-community partnered projects and traditional academic-led research projects, and future research should explore this possibility. Further, partnerships around mental health issues may require higher levels of commitment from all partners to overcome social stigma, thus achieving higher levels of synergy, as compared with general health research.

Similarly, in contrast to our conceptual model, CERI and partnership size did not generally have a significant impact on partnership functioning, either positive or negative. This suggests that partnership characteristics do not affect partners’ perception of their project’s leadership, decision-making processes, participant involvement, and/or efficiency of using financial and non-financial resources. This conclusion contradicts the hypothesis that the more partner organizations participate in the project, in terms of number of partners and their level of engagement, the more challenging it should be to manage it. Nonetheless, the marginally significant relationship between community engagement in research and partnership involvement challenges provides limited support for this hypothesis. Engaging community in research can constitute a diversion of time and resources, frustration, and insufficient influence over partnership activities, underscoring the difficulty of sustaining partnered research efforts. Consequently, the relationship between partnership characteristics and partnership functioning may become significant, and could be either positive or negative, in a sample that includes projects not affiliated with the NIMH-funded center. Center resources may facilitate management of daily project activities by providing administrative assistance, advising on research-related issues, and helping troubleshoot problems, among other things.

CONCLUSIONS

Evaluations of research projects on vulnerable populations can and should examine the impact of a full range of partnership characteristics on project outcomes. Such investigations should include measures of community engagement in research, synergy, and personal, professional, and community benefits as perceived by project participants and (possibly) other stakeholders. We believe our conceptual model and corresponding empirical findings have a high degree of scientific and community validity, given the partnered nature of this partnership evaluation study. Our research provides additional empirical support for the value of community engagement in research by demonstrating a positive association between engagement and the perception of both personal- and community/policy-level partnership outcomes. We also found support for the partnership synergy model (Lasker et al. 2001; Weiss et al. 2002), which we augmented by adding partnership characteristics measures and modifying outcome measures for the purposes of evaluating mental health projects. Specifically, our results provide confirmation that greater synergy is associated with better outcomes, as has been found in previous studies. Although a negative association has yet to be found in empirical studies of the relationship between synergy and outcomes, synergy may not be sufficient by itself to result in changes at the community or policy level, nor in projects that are better designed, conducted, and associated with a research center that has a history of positive impact, such as the one described here.

As mentioned above, the projects included in the sample primarily represent established community-academic partnerships, most of which obtained independent funding but were supported by the strong administrative infrastructure of two consecutive NIMH-funded centers. This is especially important to consider because the majority of the community partner agencies serve minority, low-SES communities and are under-resourced themselves. Although exploratory, our study results may generalize to projects undertaken by community and academic partners with some history of successful partnership with such agencies. Our findings may also be useful for community and academic partners to better understand how to build more effective research partnerships that not only help achieve individually-desired outcomes, but also have a meaningful impact in community.

Several limitations, however, must be considered in interpreting our results. First, our measures are based on perceptions and are subject to social desirability and other biases inherent with self-reported data (Fisher 1993). Participants may have over- or underestimated the level of community engagement in research, as well as project benefits and outcomes. Second, our sample was relatively small, and, therefore, our study can only be considered exploratory, limiting our ability to empirically validate our conceptual model and generalize from our findings. In particular, some of the relationships predicted by our conceptual model but not supported by our data (e.g., association between partnership characteristics and functioning and the impact of CERI on capacity-building) may be revealed in a larger and/or more diverse sample. Third, because our sample of projects is limited to partnerships developed within two consecutive NIMH-funded research centers, our findings may not be applicable to all academic-community research partnerships. These projects may have unmeasured similarities in terms of the context in which they operate (e.g., a relatively strong commitment to, and a belief in, the value of partnered research) and have a high degree of uniformity in the focus on marginalized populations, social justice concerns, and stigma. Finally, our outcome measures consisted of both achieved and anticipated community/policy-level benefits, which we combined because only a small portion of projects had been completed at the time of data collection. To determine the sensitivity of our results to the combined measures, our models included an indicator for partnership duration (completed vs. ongoing). This variable did not have any statistically significant impact in our models, supporting our decision to use combined outcome measures.

In conclusion, results of our exploratory study reinforce the importance of engaging community partners in research on mental health issues. By incorporating different dimensions of partnership characteristics and using previously validated partnership functioning and synergy measures, our model advances the CBPR approach by demonstrating the potential impact of community engagement on partnered projects’ outcomes. As is the case with any conceptual model and any exploratory study, additional research is needed to empirically validate our model. Future research should include projects not affiliated with an academic research center, incorporate additional stakeholder perspectives, develop objective measures of outcomes, and use a prospective and longitudinal design or structural equation modeling to test the causality and significance of relationship pathways between partnership characteristics, functioning, synergy, and outcomes in larger samples of projects. Moreover, researchers should consider additional measures of partnership context (Wallerstein and Duran 2010; Wallerstein et al. 2008) to account for the variation between project environments. Using different methods of data collection and combining different sources of data may also mitigate the impact of the social desirability bias typical for self-reported data.

Footnotes

*

We would like to thank all of our study participants for sharing their experiences working on partnered projects and anonymous journal reviewers, as well as the editor, for their helpful comments and suggestions. We would also like to acknowledge support and advice we received from Ken Wells, Christine Vaughan, and Lingqi Tang. The project was funded by the National Institute of Mental Health (Award Number P30MH082760 and P30MH068639). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Mental Health or the National Institutes of Health.

This study was previously presented at the 21st NIMH Conference on Mental Health Services Research, the 106th Annual Meeting of the American Sociological Association, and the 139th Annual Meeting of the American Public Health Association.

Contributor Information

Dmitry Khodyakov, The RAND Corporation.

Susan Stockdale, US Department of Veterans Affairs and University of California, Los Angeles.

Felica Jones, Healthy African American Families.

Elizabeth Ohito, University of California, Berkley and University of California, Los Angeles.

Andrea Jones, Healthy African American Families and Charles Drew University of Medicine and Science.

Elizabeth Lizaola, University of California, Los Angeles.

Joseph Mango, University of California, Los Angeles.

References

  1. Armistead C, Pettigrew P, Aves S. Exploring leadership in multi-sectoral partnerships. Leadership. 2007;3:211–230. [Google Scholar]
  2. Asch Solomon E. Effects of group pressure upon the modification and distortion of judgments. In: Porter LW, Angle HLA, Robert W, editors. Organizational influence processes. M.E. Sharpe Inc; 2003. pp. 295–303. [Google Scholar]
  3. Baldwin JA, Johnson JL, Benally CC. Building partnerships between indigenous communities and universities: lessons learned in HIV/AIDS and substance abuse prevention research. American Journal of Public Health. 2009;99:S77–S82. doi: 10.2105/AJPH.2008.134585. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bluthenthal RN, Jones L, Fackler-Lowrie N, Ellison M, Booker T, Jones F, McDaniel S, Moini M, Williams KR, Klap R. Witness for Wellness: preliminary findings from a community-academic participatory research mental health initiative. Ethnicity and Disease. 2006;16:S1-18–S1-34. [PubMed] [Google Scholar]
  5. Butterfoss FD, Goodman RM, Wandersman A. Community coalitions for prevention and health promotion. Health Education Research. 1993;8:315–330. doi: 10.1093/her/8.3.315. [DOI] [PubMed] [Google Scholar]
  6. Byrd B, Tandon D, Mendelson T, Mance GSA, Jones J. Utilizing Community-Based Participatory Research to Adapt a Mental Health Intervention for African American Emerging Adults. Progress in Community Health Partnerships: Research, Education, and Action. 2010;4:131–140. doi: 10.1353/cpr.0.0112. [DOI] [PubMed] [Google Scholar]
  7. Cashman SB, Adeky S, Allen AJ, III, Corburn J, Israel BA, Montano J, Rafelito A, Rhodes SD, Swanston S, Wallerstein N. The power and the promise: working with communities to analyze data, interpret findings, and get to outcomes. American Journal of Public Health. 2008;98:1407–1417. doi: 10.2105/AJPH.2007.113571. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Corbie-Smith Giselle, Ammerman Alice S, Katz Mira L, St George Diane Marie M, Blumenthal Connie, Washington Chanetta, Weathers Benita, Keyserling Thomas C, Switzer Boyd. Trust, Benefit, Satisfaction, and Burden. Journal of General Internal Medicine. 2003;18:531–541. doi: 10.1046/j.1525-1497.2003.21061.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Feiock RC. Rational choice and regional governance. Journal of Urban Affairs. 2007;29:47–63. [Google Scholar]
  10. Fisher Robert J. Social Desirability Bias and the Validity of Indirect Questioning. The Journal of Consumer Research. 1993;20:303–315. [Google Scholar]
  11. Granner ML, Sharpe PA. Evaluating community coalition characteristics and functioning: a summary of measurement tools. Health Education Research. 2004;19:514–532. doi: 10.1093/her/cyg056. [DOI] [PubMed] [Google Scholar]
  12. Gray Barbara. Collaborating: finding common ground for multiparty problems. San Francisco: Jossey-Bass; 1989. [Google Scholar]
  13. Green L, Daniel M, Novick L. Partnerships and coalitions for community-based research. Public Health Reports. 2001;116:20. doi: 10.1093/phr/116.S1.20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Higgins DL, Metzler M. Implementing community-based participatory research centers in diverse urban settings. Journal of Urban Health. 2001;78:488–494. doi: 10.1093/jurban/78.3.488. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Israel BA, Parker EA, Rowe Z, Salvatore A, Minkler M, López J, Butz A, Mosley A, Coates L, Lambert G. Community-based participatory research: lessons learned from the Centers for Children’s Environmental Health and Disease Prevention Research. Environmental health perspectives. 2005;113:1463–1471. doi: 10.1289/ehp.7675. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Israel BA, Schulz AJ, Parker EP, Becker AB. Community-based participatory research: policy recommendations for promoting a partnership approach in health research. Education for Health. 2001;14:182–197. doi: 10.1080/13576280110051055. [DOI] [PubMed] [Google Scholar]
  17. Johnson CV, Bartgis J, Worley JA, Hellman CM, Burkhart R. Urban Indian Voices: A Community-Based Participatory Research Health and Needs Assessment. American Indian and Alaska Native Mental Health Research: The Journal of the National Center. 2010;17:49–70. doi: 10.5820/aian.1701.2010.49. [DOI] [PubMed] [Google Scholar]
  18. Jones L, Wells K. Strategies for academic and clinician engagement in community-participatory partnered research. Jama. 2007;297:407–410. doi: 10.1001/jama.297.4.407. [DOI] [PubMed] [Google Scholar]
  19. Khodyakov D, Mendel P, Dixon E, Jones A, Masongsong Z, Wells K. Community Partners in Care: Leveraging Community Diversity to Improve Depression Care for Underserved Populations. International Journal of Diversity in Organisations, Communities and Nations. 2009;9:167–182. [PMC free article] [PubMed] [Google Scholar]
  20. Kone A, Sullivan M, Senturia KD, Chrisman NJ, Ciske SJ, Krieger JW. Improving collaboration between researchers and communities. Public Health Reports. 2000;115:243–248. doi: 10.1093/phr/115.2.243. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Lasker RD, Weiss ES, Miller R. Partnership synergy: a practical framework for studying and strengthening the collaborative advantage. The Milbank Quarterly. 2001;79:179–205. doi: 10.1111/1468-0009.00203. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Minkler M. Community-based research partnerships: challenges and opportunities. Journal of Urban Health. 2005a;82:ii3–ii12. doi: 10.1093/jurban/jti034. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Minkler M. Community organizing and community building for health. Piscataway: Rutgers, The State University of New Jersey; 2005b. [Google Scholar]
  24. Minkler M. Linking science and policy through community-based participatory research to eliminate health disparities. Am J Public Health. 2010;100:S81–S87. doi: 10.2105/AJPH.2009.165720. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Minkler M, Vásquez VB, Tajik M, Petersen D. Promoting environmental justice through community-based participatory research: the role of community and partnership capacity. Health Education & Behavior. 2008;35:119. doi: 10.1177/1090198106287692. [DOI] [PubMed] [Google Scholar]
  26. Minkler Meredith. Ethical Challenges for the ‘Outside’ Researcher in Community-Based Participatory Research. Health Education & Behavior. 2004;31:684–697. doi: 10.1177/1090198104269566. [DOI] [PubMed] [Google Scholar]
  27. Ndirangu M, Yadrick K, Bogle ML, Graham-Kresge S. Community—Academia Partnerships to Promote Nutrition in the Lower Mississippi Delta: Community Members’ Perceptions of Effectiveness, Barriers, and Factors Related to Success. Health Promotion Practice. 2008;9:237–245. doi: 10.1177/1524839907300804. [DOI] [PubMed] [Google Scholar]
  28. Plowfield LA, Wheeler EC, Raymond JE. Time, tact, talent, and trust: Essential ingredients of effective academic-community partnerships. Nursing Education Perspectives. 2005;26:217–220. [PubMed] [Google Scholar]
  29. Poole DL. Building community capacity to promote social and public health: Challenges for universities. Health and Social Work. 1997;22:163–170. doi: 10.1093/hsw/22.3.163. [DOI] [PubMed] [Google Scholar]
  30. Quandt SA, Arcury TA, Pell AI. Something for everyone? A community and academic partnership to address farmworker pesticide exposure in North Carolina. Environmental health perspectives. 2001;109:435. doi: 10.1289/ehp.01109s3435. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Ridgeway CL. Conformity, group-oriented motivation, and status attainment in small groups. Social Psychology. 1978;41:175–188. [Google Scholar]
  32. Schulz AJ, Israel BA, Lantz P. Instrument for evaluating dimensions of group dynamics within community-based participatory research partnerships. Evaluation and Program Planning. 2003;26:249–262. doi: 10.1016/j.evalprogplan.2018.04.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Shattell MM, Hamilton D, Starr SS, Jenkins CJ, Hinderliter NA. Mental health service needs of a Latino population: A community-based participatory research project. Issues in Mental Health Nursing. 2008;29:351–370. doi: 10.1080/01612840801904316. [DOI] [PubMed] [Google Scholar]
  34. Shavers-Hornaday VL, Lynch CF, Burmeister LF, Torner JC. Why are African Americans under-represented in medical research studies? Impediments to participation. Ethnicity & Health. 1997;2:31–45. doi: 10.1080/13557858.1997.9961813. [DOI] [PubMed] [Google Scholar]
  35. Sloane DC, Diamant AL, Lewis LVB, Yancey AK, Flynn G, Nascimento LM, Carthy WJ, Guinyard JJ, Cousineau MR. Improving the Nutritional Resource Environment for Healthy Living Through Community Based Participatory Research. Journal of General Internal Medicine. 2003;18:568–575. doi: 10.1046/j.1525-1497.2003.21022.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Srinivasan S, Collman GW. Evolving partnerships in community. Environmental health perspectives. 2005;113:1814–1816. doi: 10.1289/ehp.7911. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Wallerstein N. Power between evaluator and community: research relationships within New Mexico’s healthier communities. Social Science & Medicine. 1999;49:39–53. doi: 10.1016/s0277-9536(99)00073-8. [DOI] [PubMed] [Google Scholar]
  38. Wallerstein N, Duran B. Community-based participatory research contributions to intervention research: the intersection of science and practice to improve health equity. American Journal of Public Health. 2010;100:S40–S46. doi: 10.2105/AJPH.2009.184036. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Wallerstein N, Polascek M, Maltrud K. Participatory evaluation model for coalitions: the development of systems indicators. Health Promotion Practice. 2002;3:361–373. [Google Scholar]
  40. Wallerstein N. Commentary: challenges for the field in overcoming disparities through a CBPR approach. Ethnicity & Disease. 2006;16:146–148. [PubMed] [Google Scholar]
  41. Wallerstein N, Oetzel J, Duran B, Tafoya G, Belone L, Rae R. What predicts outcomes in CBPR. In: Minkler M, Wallerstein N, editors. Community-Based Participatory Research for Health: From Process to Outcomes. San Francisco: Jossey-Bass; 2008. pp. 371–388. [Google Scholar]
  42. Wang J. Using SAS PROC MIXED to demystify the hierarchical linear model. The Journal of Experimental Education. 1997;66:84–93. [Google Scholar]
  43. Weiss ES, Anderson RM, Lasker RD. Making the most of collaboration: exploring the relationship between partnership synergy and partnership functioning. Health Education & Behavior. 2002;29:683–698. doi: 10.1177/109019802237938. [DOI] [PubMed] [Google Scholar]
  44. Wells K, Miranda J, Bruce ML, Alegria M, Wallerstein N. Bridging community intervention and mental health services research. American Journal of Psychiatry. 2004;161:955–963. doi: 10.1176/appi.ajp.161.6.955. [DOI] [PubMed] [Google Scholar]
  45. Wolff M, Maurana CA. Building effective community-academic partnerships to improve health: a qualitative study of perspectives from communities. Academic Medicine. 2001;76:166–172. doi: 10.1097/00001888-200102000-00016. [DOI] [PubMed] [Google Scholar]

RESOURCES