Establishing the Psychometric Properties of Constructs in a Community-Based Participatory Research Conceptual Model

Abstract

Purpose

The purpose of this study is to establish the psychometric properties of 22 measures from a community-based participatory research (CBPR) conceptual model.

Design

On-line, cross-sectional survey of academic and community partners involved in a CPBR project

Setting

294 CPBR projects in the U.S. with federal funding in 2009

Subjects

312 (77.2% of 404 invited) academic and community partners and 138 principal investigators/project directors (69.0% of 200 invited)

Measures

22 measures of CBPR context, group dynamics, methods, and health-related outcomes

Analysis

Confirmatory factor analysis to establish factorial validity and Pearson correlations to establish convergent and divergent validity

Results

Confirmatory factor analysis demonstrated strong factorial validity for the 22 constructs. Pearson correlations (p < .001) supported the convergent and divergent validity of the measures. Internal consistency was strong with 18 of 22 measures achieving at least a .78 Cronbach’s alpha.

Conclusion

CBPR is a key approach for health promotion in underserved communities and/or communities of color and yet the basic psychometric properties of CBPR constructs have not been well established. This study provides evidence of the factorial, convergent, and discriminant validity, and internal consistency of 22 measures related to the CBPR conceptual model. Thus, these measures can be used with confidence by both CBPR practitioners and researchers to evaluate their own CBPR partnerships and advance the science of CBPR.

PURPOSE

Community-based participatory research (CBPR) is a growing science for engaging in health promotion that especially addresses health disparities through partnership of researchers and community members in all phases of the research.^1-3 CBPR and community-engaged research are widely accepted by public health practitioners and academic researchers as effective in working with underserved communities and communities of color because they help to establish trust with community members who may feel disenfranchised, enhance cultural appropriateness of methods/interventions, and build capacity of university partners and community members.^4-7

Despite the growing interest, the science of CBPR has lagged behind its practice. In recent years, the development of theoretical models has enhanced the understanding of the pathways by which CBPR processes results in particular outcomes. First, Wallerstein and her colleagues² developed a conceptual model of CBPR that includes four domains: context, group dynamics, intervention/research, and outcomes (system/capacity and health outcomes) (see Figure 1). The model was developed in consultation with a national advisory board of CBPR community and academic experts and identifies specific characteristics within each domain, drawing upon prior work identifying CPBR constructs.⁸ Context includes socio-economic, policy, historical collaboration, and community/university capacity which provide the background for the group dynamics. Group dynamics includes structural features of the partnership, individual characteristics, and the relational dynamics among members. The partnering processes shape the intervention or research design and ideally reflect mutual learning and local community norms and practices. Finally, the intervention/research produces outcomes for the community and individual members.

Figure 1.

CBPR conceptual model

Second, Khodyakov and colleagues⁹ developed a model that explained how partnership characteristics result in outcomes such as the synergy of the partnership, partnership size, and community engagement in the research. In a study of 62 community and academic leaders of 21 federally-funded research centers on mental health, the authors found that community engagement in research was positively associated with professional development of members and community/policy-level outcomes. Synergy was associated positively with capacity building and community/policy-level outcomes. The two models are similar in perspective although the Wallerstein et al. model is more comprehensive in its coverage by including aspects of context and intervention/research. However, the current study benefited from Khodyakov et al.’s partnership measures.

The gains in theorizing and modeling are important for the science of CBPR. However, while the core constructs have been identified, there is a lack of strong measurement of these constructs. A review of literature on measurement tools of community coalitions revealed that the majority lacked information about validity and reliability.¹⁰ To prepare for the study reported here, the research team¹¹ reviewed 273 articles using CBPR and identified 46 instruments with 224 measures of CBPR constructs within the CBPR conceptual model.² They found that only 54 measures had evidence of internal consistency and only 31 had any evidence of validity. The validity tended to focus only on expert opinions or exploratory factor analysis. Thus, the vast majority of measures lack the basic psychometric properties expected of rigorous social science. Further, most of the measures focus on relational dynamics with many of the domains and constructs in the conceptual model not being measured. With the widespread use of CPBR, it is imperative to advance the quality of the measurement of partnership processes and outcomes.

A recent study offers stronger evidence of measurement quality than most previous studies.^{4, 9} This study offered evidence of internal consistency and factorial validity (exploratory factor analysis) for 10 measures of processes and outcomes including the following: perceived community and policy-level outcomes, capacity building, partnership synergy, influence in decision making, leadership, and managing partnership activities. Despite this improvement, there are several limitations of their study: small sample size (only 62 respondents from 21 projects), similar types of projects (all focusing on mental health), and inclusion of a partial set of constructs consistent with the CBPR conceptual model.

In summary, in order to develop the science of CBPR, there needs to be rigorous measurement of constructs in the CBPR conceptual model. Without rigorous measurement, researchers are not able to test the pathways of the conceptual model and practitioners are not able to effectively evaluate their partnership. Thus, the purpose of the current study is to provide evidence of the psychometric properties (internal consistency, factorial validity, and convergent/discriminant validity) of 22 measures used in a national survey of partners from 294 federally-funded CBPR or community-engaged research projects. These 22 measures cover many, but not all, of the constructs within the four domains as the conceptual model simply has too many constructs to effectively measure in single study. Choices were made as to the most important constructs and what could be measured in this study and are elaborated on in the measures section.

METHODS

Design

The research design is a three-stage, cross-sectional survey of federally-funded CBPR projects. The current study is based on the 3^rd stage. A brief summary of the first two stages is described here in order to provide sufficient background. Further details on the first two stages can be found elsewhere.^7,12 All data were collected via DatStat Illume, a web-based survey platform.

First, 103,250 federally-funded extramural projects funded in 2009 were screened using a computer algorithm and staff review to identify projects that involved CBPR or community-engaged research and included a range of institutes, funding agencies, health topics, and funding mechanisms (citation added after review); 333 were identified and invited to participate in the study. Second, among the invited projects, only 294 actually involved CBPR or community-engaged research (based on self-identification or re-screening of non-participating abstracts); 200 (68.0%) of the principal investigators/project directors (PI/PD) participated in a key informant survey (KIS) in the latter part of 2011. The KIS asked the PI/PD to identify project characteristics and also up to four individuals (one academic partner and three community partners) to participate in the third stage, the community engagement survey (CES). In addition, the PI/PD was invited to complete the CES and to nominate community partners to complete the CES. This survey included the measures evaluated in the current study and took about 30 minutes to complete. Survey follow-up included five e-mail reminders and contact via phone as needed. The study protocol was approved by two university IRBs and the National Indian Health Service IRB.

Sample

The PI/PD nominated a total of 404 partners with 312 completing the survey in 2012. Of the 200 eligible PI/PD’s, 138 completed the CES. These 450 participants represented 82% of the total projects where a KIS was completed and 56% of the original 294 projects. The CES sample included the following demographic characteristics (with missing data accounting for totals not adding to 450): a) 118 academic and 194 community partners; b) 205 female and 73 male, and c) 272 White, non-Hispanic, 37 American Indian/Alaska Native, 37 African American, 32 Hispanic, 28 Asian/Pacific Islander, and 23 mixed race or other. Figure 2 displays the flowchart of the stages and includes the reasons people did not participate. Figure 3 displays the number of responding projects and individuals per project.

Figure 2.

Selection flow chart of the community engagement survey

PI/PD = Principle investigators or project directors, KIS= key informant survey; CES = community engagement survey; Bolded box includes the sample for this study

Figure 3.

Community engagement survey (CES) flow chart for projects and participants

PI/PD = Principle investigators or project directors, CES= community engagement survey; Bolded box includes the sample for this study

Measures

The selection of measures and writing items for new measures were developed in the following manner. First, the literature on measures of CPBR and community-engaged research constructs within the conceptual model was consulted with a focus on which measures had evidence of validity and reliability. Thus, we began with the list of 224 measures noted in the purpose section. Second, the research team consulted with a team of CBPR experts for advice on measurement and research design. This advisory group was used to help determine the most important constructs and domain within the conceptual model, especially given constraints of survey length. Third, the survey was part of a larger study that included seven in-depth case studies of CBPR projects. Three of those case studies were completed before the initiation of the survey and thus helped to inform item generation for constructs without prior measures. These steps allowed for the creation of items of face and content validity consistent with the CBPR conceptual model. Further, areas where the case studies could better address the constructs were identified and left off the survey (i.e., many context and research design constructs). Fourth, the surveys used in this study were piloted with CBPR projects not funded in 2009 and this information was used to revised item wording and improve the flow of the on-line survey, and ensure the participant burden in completing the survey was reasonable. The large number of measures and items is appropriate given the breadth of CBPR and community-engaged research.

The measures included a total of 101 items. The first two columns in the tables presented in the results section help to organize the domain and measures described here. For context, seven items measuring the capacity of the partnership to meet its aims were included.¹⁰ In addition, a single item that was created for this study measured the degree of trust at the beginning of the partnership, which distinguishes from current levels of trust.

There were three measures of structural/individual dynamics. There were two new measures created for this study: “bridging social capital” (3 items) assessed the ability of community and academic members to work effectively with each other, and “alignment with CBPR principles” consisted of two sub-scales (partner and community focus; 4 items each) and was constructed from Israel et al.’s³ principles of effective CBPR and consistent with a recent conceptualization of principles.¹³ Finally, “partner values” (4 items) captures the degree to which there is agreement with the mission and strategies of the project.¹⁴

There were 11 measures of relational dynamics. They include the following: “research tasks and communication” consisting of three subscales that capture level of involvement for community partners in various stages of the research process—background (5 items), data collection (4 items), and analysis & dissemination (4 items);⁹ “Dialogue and mutual learning” consisting of tree subscales participation, cooperation, and respect (3 items each).¹⁵ “Trust” (4 items);¹⁶ “influence and power dynamics” (3 items);¹⁷ “participatory decision making”;⁹ “Leadership” effectiveness (12 items),⁹ and “resource management” (3 items; labeled efficiency in the original).⁹

In the final domains, intervention and outcomes, there were seven measures. For intervention, the degree to which the partnership had “synergy” was employed (5 items).⁹ Outcomes include “system and capacity changes” with four scales. Partner capacity building (4 items) was from an existing scale and agency capacity building (4 items) was from the same existing scale, but re-written for an agency instead of an individual ⁹ and “changes in power relations” (5 items) and “sustainability of partnership/project” (3 items) were created for this study. There were also two scales measuring more distal outcomes: “community transformation” (7 items) from an existing scale⁹ and “community health improvement” (1 item) created for this study.

Some of the original measures cited were altered for the purpose of the current study (i.e., to fit the CBPR conceptual model) and because of concerns about the length of the survey for participants. Specifically, the following changes were made: (a) capacity items were from an original scale that was labeled non-financial resources with subscales of social and human capital;⁹ (b) task communication included one additional item and the items were also divided into three subscales to reflect different phases in the research process whereas the original items were written as a single scale (community-engagement in research index);^{4, 9} (c) dialogue and mutual learning measures included only 3 items from the original measures of 5 (participation), 7 (cooperation), and 5 (respect) items;¹⁵ (d) participatory decision making items were from the original scale of decision making with two subscales of inclusion and exclusion from decision making;⁹ (e) synergy included only 5 of the original 11 items;⁹ (f) partner capacity building included only 4 of the original 11 items that were part of three subscales related to personal outcomes in the original;⁹ and (g) community transformation included 2 items written for this study to add to the original 5 items of a scale originally labeled perceived community/policy-level benefits.⁹ Thus, the revised measures cannot be considered an empirical validation of the original scales.

Analyses

To address factorial validity, all measures were analyzed with confirmatory factor analysis (CFA) using SPSS AMOS 20.0 in 2013. The CFAs were completed using the following protocol. First, all missing values were replaced with series mean; while replacing data in this manner can be problematic, missing data was determined to be missing at random with a very small portion of missing values in the entire data set (< 1%) and hence is not a major concern. Second, the individual measures were assessed using three fit indices: comparative fit index (CFI), Tucker-Lewis index (TLI), and root mean squared residual (RMR). Good fit was determined if CFI and TLI were at or above .90 and RMR was less than .08. If all three fit indices were not acceptable, measures were modified by removing items. Items were removed if there was a suggested modification of correlating error terms or suggested path to another item. In these cases, the item that had the largest modification impact was removed. The modification of measures is controversial and cautioned against by some.¹⁸ However, as this is the first study to validate a number of CBPR measures and there is a large and robust sample of CBPR projects, it seems reasonable to explore the factor structure if good fit was not achieved. To mitigate the concern of modifying measures, the sample was split into two random halves. The measure was modified with the first random sample; the second random sample was used to confirm the modifications. This approach allows for a confirmation of the modifications. Third, once all of the individual measures were confirmed, an additional CFA for measures within particular domains of the CBPR model was completed. In the case of relational dynamics, there were several multiple factor models given the large number of measures in this domain. These multiple factor models allow for further testing of the factorial validity of the scales. Finally, since the partners are nested within partnerships, the factors loadings and structural covariances of these multiple factor models were tested for measurement invariance by comparing a random selection of one partner in each partnership to the other remaining partners. All of these models had evidence of invariance and thus only the results for the entire sample are presented.

After completing the factor analysis, internal consistency of the measures was calculated with Cronbach’s alpha. Further, means and standard deviations of the averaged items were calculated. Finally, a correlation matrix was constructed to address construct validity. All of these analyses were completed with SPSS 20.0.

RESULTS

Table 1 displays the results of the CFAs and includes information on the fit indices on unmodified measures, modified measures (where relevant), and multiple factor CFAs. All tables include 22 multi-item measures and 2 single-item measures that are displayed because of their use in construct validity. The results focus primarily on the analysis of the 22 multi-item measures. Of the 22 multi-item measures, 15 of the measures achieved good fit without any modification. An additional two measures (partner capacity building and agency capacity building) also had good fit without modification, but had to be modified in order to achieve a good multiple-factor model. Five measures needed to have one item removed to achieve good model fit (project capacity, task analysis & dissemination, participatory decision-making, partner capacity building, and agency capacity building). One measure needed two items removed (leadership) and one measure needed three items removed (community transformation).

Table 1.

Psychometric properties of measures

Domain	Name of Measure	Number of Items and Range	Cronbach’s alpha	Mean	SD	Factorial Validity (single construct- unmodified)	Factorial Validity (single construct- modified)	Factorial Validity (multiple constructs within a domain)
Context	Project capacity. Project has the resources it needs to meet its aims	6 modified; 7 original (1-5)	0.78	4.34	0.52	χ2(14) = 89.35, p < .001; CFI = .91, TLI = .87, SRMR = .03	χ2(9) = 20.88, p = .013, CFI = .96, TLI = .93, SRMR = .03	N/A
	Initial trust. Level of trust at the beginning of the partnership	1 (1-7)	N/A	5.21	1.26	N/A	N/A
Structural/ Individual Dynamics	Bridging social capital. Academic and community partners have the skills and cultural knowledge to interact effectively	3 (1-5)	0.69	4.16	0.70	Saturated model	Not modified	χ2(84) = 206.27, p < .001, CFI = .97, TLI = .96, SRMR = .03
	Alignment with CBPR principles: Partner focus. Develops individual partner capacity and equitable partnerships in all phases of the research	4 (1-5)	0.82	4.51	0.59	χ2(2) = 0.65, p = .724, CFI = 1.0, TLI = 1.0, SRMR = .003	Not modified
	Alignment with CBPR principles: Community focus. Builds on resources and strengths of the community for the well-being of the community	4 (1-5)	0.85	4.30	0.67	χ2(2) = 3.58, p = .167, CFI = .99, TLI = .99, SRMR =.01	Not modified
	Core values. Shared understanding of the project mission, priorities, and strategies	4 (1-5)	0.89	4.41	0.60	χ2(2) = 27.77, p < .001, CFI = .98, TLI = .93, SRMR = .01	Not modified
Relational Dynamics	Task roles and communication: Background research. Community partners' level of involvement in the background research	5 (1-3)	0.81	2.07	0.57	χ2(5) = 35.49, p < .001, CFI = .96, TLI = .91, SRMR =.02	Not modified	χ2(51) = 93.12, p < .001, CFI = .95, TLI = .93, SRMR = .03
	Task roles and communication: Data collection. Community partners’ level of involvement in the data collection	4 (1-3)	0.69	2.50	0.51	χ2(2) = 11.61, p = .003, CFI = .97, TLI = .90, SRMR = .02	Not modified
	Task roles and communication: Analysis & Dissemination. Community partners' level of involvement in data analysis and dissemination of findings	3 modified; 4 original (1-3)	0.82	2.07	0.71	χ2(2) = 60.89, p < .001, CFI = .92, TLI = .77, SRMR = .03	Saturated model
	Dialogue and mutual learning: Participation. Degree to which all partners participate in the process	3 (1-5)	0.78	4.44	0.58	Saturated model	Not modified	χ2(59) = 160.48, p < .001, CFI = .97, TLI = .96, SRMR = .03
	Dialogue and mutual learning: Cooperation. Degree to which partners cooperate to resolve disagreements	3 (1-5)	0.83	4.15	0.68	Saturated model	Not modified
	Dialogue and mutual learning: Respect. Degree to which partners convey respect to each other (low score equals higher respect)	3 (1-5)	0.83	1.74	0.89	Saturated model	Not modified
	Trust. Degree of current trust within the partnership	4 (1-5)	0.86	4.36	0.63	χ2(2) = 8.80, p = .012, CFI = .99, TLI = .98, SRMR = .01	Not modified
	Influence and power dynamics. Degree of voice and influence in the decision making	3 (1-5)	0.58	3.79	0.71	Saturated model	Saturated model	χ2(13) = 41.35, p < .001, CFI = .96, TLI = .93, SRMR = .04
	Participatory decision making. Degree to which decisions are made in a participatory manner	4 modified; 5 original (1-5)	0.83	4.03	0.62	χ2(5) = 116.01, p < .001, CFI = .89, TLI = .78, SRMR = .05	χ2(2) = 12.63, p = .002, CFI = .97, TLI = .92, SRMR = .02
	Leadership. Overall effectiveness of project's leadership	10 modified; 12 original (1-5)	0.94	4.28	0.61	χ2(54) = 455.78, p < .001, CFI = .91, TLI = .89, SRMR = .03	χ2(35) = 139.84, p < .001, CFI = .94, TLI = .92, SRMR = .03	χ2(64) = 171.09, p < .001, CFI = .95, TLI = .94, SRMR = .02
	Stewardship. Effective use of financial & in-kind resources	3 (1-5)	0.86	4.36	0.65	Saturated model	Not modified
Intervention/ Research	Partnership synergy. Partners ability to develop goals, recognize challenges, respond to needs, and work together	5 (1-5)	0.90	4.46	0.59	χ2(5) = 33.27, p < 001, CFI = .98, TLI = .96, SRMR = .01	Not modified	N/A
Outcome	Systems and capacity changes: Partner capacity building. Develops the skills to benefit the individual members	3 modified; 4 original (1-5)	0.80	3.42	0.99	χ2(2) = 6.45, p = .040, CFI = .99, TLI = .98, SRMR = .02	Saturated model	χ2(125) = 255.59, p < .001, CFI = .93, TLI = .92, SRMR = .07
	Systems and capacity changes: Agency capacity building. Develops the reputation and skills of agencies involved in the partnership	3 modified; 4 original (1-5)	0.87	3.54	0.96	χ2(2) = 18.83, p < .001, CFI = .98, TLI = .94, SRMR = .04	Saturated model
	Systems and capacity changes: Changes in power relations. Degree to which power and capacity has been developed in the community members	5 (1-5)	0.81	4.17	0.60	χ2(5) = 12.74, p = .026, CFI = .99, TLI = .98, SRMR = .01	Not modified
	Systems and capacity changes: Sustainability of partnership/project. Likelihood of the project and partnership continuing beyond the funding period	3 (1-5)	0.71	4.00	0.76	Saturated model	Not modified
	Health outcomes: Community transformation. Policy changes and community improvement.	4 modified; 7 original (1-5)	0.79	2.92	1.00	χ2(14) = 122.08, p < .001, CFI = .92, TLI = .87, SRMR = .09	χ2(2) = 7.37, p = .025, CFI = .98, TLI = .95, SRMR = .06
	Health outcomes: Community health improvement. Improvement of health for the community as a result of the project	1 (1-5)	N/A	3.63	1.02	N/A	N/A

The multiple-factor models provide further support of the factorial validity of the measures. In most cases, the multiple-factor models confirmed the original measures without any need for further modification. For the outcome domain, the multiple-factor model did not achieve a good fit initially, χ²(179) = 839.04, p < .001, CFI = .86, TLI = .84, SRMR = .08. A single item from the partner and agency capacity building were removed and good model fit was achieved..

Table 1 also displays descriptive information about each of the measures. The internal consistency of the measures was generally very good. Only one measure failed to reach the minimum threshold of .60 (influence and power dynamics at .58), while most of the measures were at or above .78 (18 of 22). The individual items and factors loadings are available in Table 2.

Table 2.

Items and factor loadings

Domain	Name of Scale	Items
Context	Project capacity	Skills and expertise (0.56) Data and information (removed during modification) Diverse membership (0.42) Legitimacy and credibility (0.72) Ability to bring people together for meetings and activities (0.69) Connections to political decision-makers, government agencies, other organizations/groups (0.49) Connections to relevant stakeholders (0.64)
	Initial trust	At the beginning of your partnership, what type of trust did you have? (single item scale—no factor loading)
Structural/ Individual Dynamics	Bridging social capital	Does the community research team have the knowledge, skills, and confidence to interact effectively with the academic researcher team? (0.61) Does the academic research team have members who are from a similar cultural background as the community research team? (0.56) Overall, does the academic research team have the knowledge, skills, and confidence to interact effectively with the community research team? (0.87)
	Alignment with CBPR principles: Partner focus	This project builds on resources and strengths in the community (0.69) This project emphasizes what is important to the community (environmental and social factors) that affect wellbeing (0.78) This project views community-engaged research as a long term process and a long term commitment (0.69) This project fits local/cultural beliefs, norms, and practices (0.76)
	Alignment with CBPR principles: Community focus	This project facilitates equitable partnerships in all phases of the research (0.77) This project helps all partners involved to grow and learn from one another (0.81) This project balances research and social action for the mutual benefit of all partners (0.77) This project disseminates knowledge and findings to all partners and involves all partners in the dissemination process (0.70)
	Core values	Members of our partnership have a clear and shared understanding of the problems we are trying to address (0.78) There is a general agreement with respect to the mission of the partnership (0.86) There is general agreement with respect to the priorities of the partnership (0.86) Members agree on the strategies the partnership should use in pursuing its priorities (0.80)
Relational Dynamics	Task roles and communication: Background research	Developing community-based theories of the problem or intervention (0.55) Grant proposal writing (0.66) Background research (0.63) Choosing research methods (0.77) Developing sampling procedures (0.61)
	Task roles and communication: Data collection	Recruiting study participants (0.41) Implementing the intervention (0.32) Designing interview and/or survey questions (0.72) Collecting primary data (0.57)
	Task roles and communication: Analysis & Dissemination	Analyzing collected data (removed during modification) Interpreting study findings (0.70) Writing reports and journal articles (0.79) Giving presentations at meetings and conferences (0.69)
	Dialogue and mutual learning: Participation	We showed positive attitudes towards one another (0.81) Everyone in our partnership participated in our meetings (0.62) We listened to each other (0.88)
	Dialogue and mutual learning: Cooperation	Arguments that occurred during our meetings were constructive (0.72) When disagreements occurred, we worked together to resolve them (0.87) Even though we didn’t have total agreement, we did reach a kind of consensus that we all accept (0.76)
	Dialogue and mutual learning: Respect	There were disrespectful remarks made during the conversation (0.64) There was hidden or open conflict and hostility among the members (0.85) The way the other members said some of their remarks was inappropriate (0.91)
	Trust	I trust the decisions others to make about issues that are important to our projects (0.76) I am comfortable asking other people to take responsibility for project tasks even when I am not present to oversee what they do (0.75) I can rely on the people that I work with on this project (0.84) People in this group/community have confidence in one another (0.80)
	Influence and power dynamics	All partners had equal voice in deciding which funding opportunities were sought for this project. (0.42) I have influence over decisions that this partnership makes (0.50) Overall, I am satisfied with the amount of influence that each partner has over decisions that this partnership makes (0.94)
	Participatory decision making	Feel comfortable with the way decisions are made in the project (0.84) Support the decisions made by the project team members (0.76) Feel that your opinion is taken into consideration by other project team members (0.77) Feel that you have been left out of the decision making process (−0.61) Feel pressured to go along with decisions of the project team even though you might not agree (removed during modification)
	Leadership	Taking responsibility for moving the project forward (0.82) Inspiring or motivating people involved in the project (removed during modification) Encouraging active participation of academic and community partners in the decision-making (0.83) Communicating the goals of the project (0.82) Working to develop a common language (0.84) Fostering respect between partners (0.84) Developing trust between partner Creating an environment where differences of opinion can be voiced (0.80) Resolving conflict among partners (0.83) Helping the partners be creative and look at things differently (0.80) Recruiting diverse people and organizations into the project (0.59) Providing orientation to new partners as they join the project (0.83)
	Stewardship	The team's financial resources (0.82) The team's in-kind resources (0.72) The team's time (0.83)
Intervention / Research	Partnership synergy	Develop goals that are widely understood and supported in this partnership (0.76) Develop strategies that are most likely to work for your community or stakeholders as a whole (0.76) Recognize challenges and come up with good solutions (0.79) Respond to the needs and problems of your stakeholders or community as a whole (0.79) Work together as a team (0.70)
Outcome	Systems and capacity changes: Partner capacity building	Enhanced my own reputation (0.80) Enhanced my ability to affect public policy (removed during modification) Increased utilization of my expertise or services (0.86) Increased my ability to acquire additional financial support (0.67)
	Systems and capacity changes: Agency capacity building	Enhanced the agencies' reputation (0.90) Enhanced the agencies' ability to affect public policy (0.78) Increased utilization of agencies' expertise or services (0.88) Increased agencies' ability to acquire additional financial support (removed during modification)
	Systems and capacity changes: Changes in power relations	Community team members have increased participation in the research process (0.78) Community team members can talk about the project in other settings such as a community or political meeting (0.76) Community team members can apply the findings of the research (0.68) Community team members can voice their opinions about research in front of researchers (0.63) Community team members have sought continuing formal or informal education (0.54)
	Systems and capacity changes: Sustainability of partnership/project	I am committed to sustaining the community-academic relationship with no or low funding (0.71) This project is likely to continue forward after this funding is over (0.70) Our partnership carefully evaluates funding opportunities to make sure they meet both community and academic partners' needs (0.69)
	Health outcomes: Community transformation	Improved the access, delivery, and quality of health services (broadly defined) in the community (removed during modification) Resulted in sustained partnerships among agencies (removed during modification) Resulted in policy changes (0.69) Improved the overall health status of individuals in the community (0.79) Received public recognition or acknowledgment from local policy makers and/or government officials (removed during modification) Resulted in acquisition of additional financial support (0.43) Improved the overall environment in the community (0.87)
	Health outcomes: Community health improvement	Overall, how much did or will your research project (insert name) improve the health of the community? (single factor scale so no factor loading)

Table 3 displays the correlation matrix. This matrix helps to support the construct validity of the measures by offering information specifically about convergent and discriminant validity. First, the measures for constructs within domains generally demonstrate positive and moderate correlations. One exception is that within the relational dynamics domain, lack of respect was negatively and moderately correlated with other measures in this domain. These expected patterns illustrate that the measures have good convergent validity. Second, measures for constructs across domains, but that should be related, are positively and moderately correlated. For example, the principles measures are positively associated with the degree to which community partners were involved in the task roles (background, data collection, and analysis & dissemination) in the partnership. This pattern is further evidence of convergent validity. Finally, there is also good convergent and discriminatory validity for the relational dynamics measures and outcomes. A positive relationship between partnering behaviour and outcomes is expected. However, some outcomes should be better predicted than others. Specifically, improved health is a distal outcome that would not be expected to be largely associated with specific partnering behavior. In contrast, most of the other outcomes are proximal or intermediate to the partnership and thus would be expected to have a stronger and positive relationship with relational dynamics compared to improved health. The matrix supports this pattern with small to insignificant correlations for relational dynamics and improved health and moderate and positive relationships for relational dynamics and the other outcomes.

Table 3.

Correlation matrix of measures

Domain	Construct	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23
Context	1. Project  Capacity
	2. Initial Trust	.16*
Structural/Individual   Dynamics	3. Bridging social  capital	.44*	.21*
	4. Principles:  Partner	.53*	.14	.53*
	5. Principles:  Community	.51*	.18*	.48*	.74*
	6. Core Values	.49*	.18*	.49*	.64*	.64*
Relational Dynamics	7. Task  Background	.25*	.10	.22*	.38*	.41*	.28*
	8. Task Data  Collection	.16*	.01	.12	.36*	.31*	.30*	.56*
	9. Task Analysis  &  Dissemination	.15	−.01	.10	.28*	.31*	.21*	.65*	.59*
	10. Participation	.35*	.20*	.36*	.43*	.50*	.54*	.23*	.17*	.19*
	11. Cooperation	.20*	.10	.22*	.32*	.38*	.41*	.26*	.24*	.23*	.53*
	12. (Dis)Respect	−.13	−.12	−.23*	−.23*	−.26*	−.37*	−.06	−.02	−.01	−.43*	−.24*
	13. Trust	.43*	.21*	.44*	.54*	.57*	.64*	.26*	.24*	.22*	.51*	.40*	−.34*
	14. Influence	.32*	.16*	.27*	.44*	.54*	.48*	.51*	.38*	.46*	.44*	.42*	−.19*	.59*
	15. Participatory  Decision  Making	.37*	.21*	.41*	.52*	.55*	.61*	.24*	.21*	.19*	.55*	.37*	−.40*	.63*	.56*
	16. Leadership	.50*	.18*	.52*	.59*	.68*	.67*	.28*	.20*	.18*	.67*	.46*	−.39*	.68*	.50*	.65*
	17. Stewardship	.40*	.13	.35*	.46*	.51*	.52*	.24*	.23*	.18*	.47*	.32*	−.26*	.58*	.36*	.50*	.59*
Research	18. Synergy	.49*	.17*	.51*	.59*	.61*	.65*	.30*	.27*	.22*	.59*	.40*	−.33*	.60*	.43*	.59*	.76*	.60*
Outcomes	19. Partner  Capacity  Building	.33*	.15	.27*	.36*	.37*	.33*	.26*	.29*	.24*	.25*	.28*	−.05	.38*	.41*	.33*	.39*	.41*	.39*
	20. Agency  Capacity  Building	.38*	.11	.34*	.49*	.48*	.39*	.29*	.30*	.28*	.31*	.31*	−.13	.40*	.42*	.32*	.46*	.39*	.49*	.68*
	21. Change in  power  relations	.33*	.13	.30*	.52*	.56*	.44*	.49*	.49*	.46*	.43*	.36*	−.13	.50*	.52*	.45*	.48*	.43*	.52*	.44*	.53*
	22. Sustainability	.38*	.18*	.28*	.47*	.52*	.40*	.38*	.32*	.33*	.35*	.29*	−.10	.46*	.47*	.41*	.43*	.40*	.48*	.45*	.49*	.58*
	23. Community  transformatio  n	.28*	.01	.29*	.32*	.30*	.25*	.23*	.24*	.22*	.21*	.20*	−.01	.28*	.32*	.21*	.30*	.31*	.35*	.39*	.50*	.33*	.39*
	24. Improved  community  health	.27*	.09	.32*	.34*	.36*	.29*	.14	.14	.07	.21*	.20*	−.06	.30*	.29*	.22*	.36*	.30*	.40*	.59*	.36*	.51*	.30*	.38*

^*p <.001

DISCUSSION

There are several strengths of this study that contribute to the science of CBPR. First, this study is the first large scale study of CBPR projects in the U.S. Prior studies of CBPR processes and outcomes have looked at a small number of, or single, projects.^{19, 20} Even when there have been a larger number of projects involved,⁹ the projects were limited to a similar health condition. The current study involves a rigorous three-stage random sampling of CBPR or community-engaged research projects across the United States. The response rates were generally high for every component of the sampling process; thus, the findings have some generalizability to the CBPR and community-engaged research community.

Second, this study provides evidence of the factorial, convergent, and discriminant validity and internal consistency of 22 measures related to the CBPR conceptual model.² This evidence makes a significant contribution to the CBPR literature given the limited information on measurement validity and reliability of CBPR measures noted by prior reviews.^{10, 11} The vast majority of prior CBPR studies have utilized measures that have not been adequately tested and thus conclusions can be called into question. The current measures have strong psychometric properties that researchers can use to measure relevant constructs in the CBPR conceptual model. Of particular note is that the measures include proximal (synergy), intermediate (partner and agency capacity building, changes in power relations, and sustainability), and distal outcomes (community transformation and improving community health) that can be used for testing relationships among context, processes, and outcomes. In addition, the research team is in the process of developing an additional proximal outcome called culture centeredness²¹ that combines some of the validated scales for a measure of the level of voice and involvement of community partners and a trust typology that characterizes the process of trust over time.²²

In addition to contributing to the science of CBPR, this study has strong practical implications. A key component of CBPR is the ability to self-reflect and self-assess partnering processes and outcomes.²³ Community and academic partners want to use measures that are simple, and yet valid, assessments and evaluations of their partnering processes and outcomes. These measures have strong measurement validity and yet are straightforward. There is a website that includes the original survey instruments and their instructions and scaling so that interested partners can access and implement them in their own partnerships (web site to be added after review of the manuscript). These measures were designed to be a comprehensive list of processes and outcomes that community and academic partners can select from as they fit to their project. A good approach would be to focus on the outcomes that the project wants to achieve and then explore other measures that are most strongly correlated with those outcomes (i.e., in Table 3). Ideally all of the items in a measure would be used and yet space constraints may limit how many can be selected. In that case, selecting at least three items with the highest factor loadings is recommended.

The current study has a few limitations. First, there were some modifications in a few measures in order to achieve good model fit. Future research may be needed to examine the psychometric properties of these measures further. Some items did not make the final fit into the scales, but may be still useful for partnerships to collect as single items. For example, the item on “improved health services” was removed possibly because of our community-based research sample, but might be useful for health services partnerships. In addition, the sampling frame only included funded CBPR projects identified as CBPR or community-engaged research. This approach is reasonable given the research aims, yet the result may be less applicability for research projects with limited community engagement. Similarly, the study did not make comparisons to non-CBPR projects so that concurrent validity could not be established. Further, given the constraints of research design and resources, there was not any level of predictive validity of the measures; the psychometric properties are based solely on self-report data and not actual behaviour. Finally, the study included limited measures of particularly “context” and “intervention/research design” as it was difficult to identify measures that could be used across a variety of CBPR projects. Specifically, several measures that should be developed in the future include policy and funding context, historical collaboration, community and university readiness, fit of research to local/cultural norms and practices, and cultural renewal. The larger project included seven in-depth case studies that emphasized contextual and research design issues and yet is beyond the scope of this article.

In summary, despite these limitations, the current study provides evidence of the psychometric properties of 22 measures related to the CBPR conceptual model. The research design included strong response rates for two stages of surveys to provide further confidence in these data. While some modifications of the original measures were necessary to achieve model fit, there was a rigorous process of modifying and confirming the modifications with two random samples of the data. These steps strongly support the use of the measures by academic and community partners to evaluate and advance their own CBPR practice as a promising strategy for engaging in health promotion to address health disparities in underserved and minority communities. This study helps to establish the measurement validity of scales that can help to advance the practice and science of CBPR.

SO WHAT?

What is already known about this topic?

CBPR and other forms of community-engaged research are well accepted approaches to engage in health promotion to address health disparities in underserved communities and/or communities of color.

What does this article add?

While CBPR and community-engaged research are well accepted, there is a lack of understanding about how and why these approaches have positive outcomes. In order to better understand how CBPR works, good measurement of CBPR processes and outcomes is needed. This article offers 22 scales that are good measures of these processes and outcomes.

What are the implications for health promotion practice or research?

These measures can be used by practitioners to help evaluate their partnership which enables them to engage in self-reflection and identify strengths and areas for improvement. In addition, these measures will enable researchers to better understand the “best practices” of CBPR and to identify the reasons why CBPR works (and perhaps when it does not).