Assessing the Capacity of Health Departments to Engage in Community-Based Participatory Public Health

Abstract

Objectives. We created indicators of local public health agency capacity to engage in community-based participatory public health.

Methods. We sent a survey of 27 items reflecting aspects of community-based participatory public health to 429 employees in 4 local health departments. Two thirds (n = 282) responded. We performed a factor analysis to identify components of community-based participatory practice.

Results. We identified 4 factors: (1) the agency’s and (2) the individual employee’s skills in working with community groups and minority populations, (3) the extent and frequency of agency networking, and (4) community participation in health department planning.

Conclusions. Our findings suggest that it is possible to measure the competencies needed by health department staff to engage in community-based participatory public health.

Public health has always been thought of as focusing on the health of populations, but criticism has risen in recent years about the dominance in this field of scientific paradigms and of research and practice methods that emphasize the individual as the unit of practice and analysis.1,2 Along with this criticism has come the call for more of a community-based participatory approach to public health practice and research from public health institutions and scholars—a call that recognizes the value of involving the intended beneficiaries throughout all phases of program planning, implementation, and evaluation.2–5 Federally funded programs such as the Racial and Ethnic Approaches to Community Health (REACH 2010) initiative of the Centers for Disease Control and Prevention and various environmental justice and community intervention projects of the National Institute of Environmental Health Sciences seek to eliminate the growing health disparity between persons of color and majority populations in the United States. These programs are designed to engage communities and health agencies in a research enterprise that emphasizes collaborative inquiry and works to change organization-, community-, and individual-level factors that contribute to health disparities.

A crucial component of the public health infrastructure is workforce capacity and competency, defined by the Centers for Disease Control and Prevention as the expertise of the approximately 500 000 professionals who work in federal, state, and local public health agencies to protect public health.6(p6) Thus, any successful community-based participatory public health intervention must have the involvement of local public health department staff.7 Such involvement implies that health department staff need competencies that enable them to (1) enhance the capacity of community members to serve in partnership endeavors, (2) appreciate the role of participation by underrepresented or underserved populations, and (3) develop skills for mobilizing community resources to address community-defined priorities.8

Yet, little is known about the organizations and staff competencies of public health departments in community-based participatory public health. With regard to core functions outlined by the Institute of Medicine,7 public health researchers have given attention to the role of public health practice,9 the articulation of 10 essential public health services,6 and methods to assess the performance of health departments.10–13 For a local health department interested in attempting more of a community-based participatory approach, however, little guidance is available in terms of how to identify and monitor the acquisition of necessary skills and competencies. This article presents our attempt to operationalize such competencies and measure the performance of 4 health departments and their staff in community-based participatory public health practice.

METHODS

The Community-Based Public Health Initiative

In 1992, the W. K. Kellogg Foundation launched its 4-year, $16 million Community-Based Public Health (CBPH) Initiative. The CBPH Initiative was designed to “strengthen linkages between public health education and public health practice by forming formal partnerships with people in communities.”14 In North Carolina, community-based organizations in 4 counties, their county health departments, and faculty from the School of Public Health, University of North Carolina at Chapel Hill, sought to achieve this goal by coming together in a consortium to define and address the public health issues important to the residents of these counties. The goals of the consortium are to (1) improve minority health in 4 African American communities, (2) make public health services and education programs more responsive to the needs of these communities, and (3) ensure a key role for community-based organization partners in shaping public health services and working with health professionals in their communities.15

To achieve the latter 2 goals, the North Carolina consortium implemented 3 strategies to promote change in the 4 participating local health departments. One strategy created coalitions in each county, consisting of representatives from the 3 partners. The organization of these coalitions emphasized the importance of shared decisionmaking among community groups and agencies in identifying health problems and strategies to solve those problems; this prevents the health department, as the local health agency, from having to make these decisions alone. A second strategy developed a health department position in 1 of the counties in which a tenure-track university faculty member had a half-time appointment at the health agency and a half-time appointment at the School of Public Health. A third strategy established a series of retreats for health department staff to explore and discuss the definition of community-based participatory public health and ways to promote changes needed within their agencies to engage in such an approach.

The North Carolina consortium employed a multiple case study participatory evaluation design, with each county coalition, the overall consortium, and the academic partners all serving as single cases. Evaluation of the North Carolina consortium was done by staff of the University of North Carolina Center for Health Promotion and Disease Prevention. As part of the evaluation activities, a survey questionnaire was developed and administered to the 4 county health departments to examine the effects of the CBPH Initiative on each organization and its staff. The first draft of the instrument was developed by evaluation staff with prior experience in assessing community-oriented primary care programs in the United States. The initial draft was then shared with members of each coalition to elicit and incorporate their suggestions for additions or revisions to the instrument. The instrument’s primary focus was on measuring competencies in community-based participatory public health practice. This study describes the development and validation of this instrument.

Study Sample

The sample for this study consisted of employees in the health departments of the 4 participating counties. The survey was mailed to all employees whose positions required provision of public health services to community members; this included personnel in units such as maternal and child health, adult health, health education, dentistry, and sanitation. Excluded from the sample were personnel with clerical, security, or home health responsibilities. A total of 429 employees met the selection criteria. Of these, 282 completed and returned the survey, for a 66% response rate.

Measures

The survey contained 50 items. Of these, 7 elicited demographic characteristics of the respondents, including position with the agency (2 questions), major area of work, education level, number of years with the agency, ethnicity, and percentage of the respondent’s salary (if any) that was paid from CBPH Initiative project funds. The remaining 43 items focused on the respondents’ perceptions of the health department’s performance and their own performance in various aspects of public health. Among these 43 items, 27 items were specific to community-based participatory public health practice (Table 1 ▶). These 27 items focused on 5 dimensions: (1) community-based skills of the health department as a whole, (2) community-based skills of the individual respondent, (3) the health department’s frequency and extent of networking with other community agencies and groups, (4) community participation in health department planning, and (5) community assessment by the health department.

TABLE 1—

Original Survey Items and Measured Community-Based Participatory Practice Competencies

In general, on a 3-point scale of high, medium, or low, how would you rate the skills of your agency in the following areas?
1.	Working with community groups
2.	Community assessment
3.	Community organizing
4.	Program planning
5.	Advocating needs in the community
6.	Communicating with minority populations
7.	Influencing public health policy
In general, on a 3-point scale of high, medium, or low, how would you rate your own skills in the following areas?
8.	Working with community groups
9.	Community assessment
10.	Community organizing
11.	Program planning
12.	Advocating needs in the community
13.	Communicating with minority populations
14.	Influencing public health policy
Community participation in health department planning
15.	How often does your agency consult community members before new programs are introduced in their community? (never, rarely, often, always, don’t know)
16.	How often do the programs you work with use feedback from the communities you are serving to make decisions on these programs? (never, rarely, often, always, don’t know)
17.	How often do the programs you work with address problems identified by the community, when public health statistics point to different problems? (never, rarely, often, always, don’t know)
18.	Does your agency have a regular procedure for residents to give feedback on services and programs? (yes, no)
19.	Were community members asked for their opinions or perceptions concerning the health status of their community in the latest assessment? (yes, no)
Agency’s frequency of networking with other community agencies and groups (response categories: “always,” “often,” “rarely,” “never,” and “don’t know”)
20.	How often does your agency jointly plan program activities with other agencies/organizations?
21.	How often does your agency communicate or network about its activities in certain communities with other local agencies or organizations serving the same communities?
22.	How often does your agency exchange resources (subcontracts, personnel, equipment, etc.) with other agencies or organizations?
Assessment
23.	How often has your agency’s assessment tried to identify the strengths as well as the weaknesses of the communities it serves? (never, rarely, often, always, don’t know)
24.	How often does your agency conduct a community assessment (or community diagnosis) in your catchment area? (annually, every 2 years, every 3 years, never, don’t know)
25.	In the latest assessment, were you as staff asked your opinions or perceptions concerning the health status of the communities you serve? (yes, no)
26.	How often have you used the findings from the health department in your work? (never, rarely, often, always, don’t know)
27.	How often has your agency presented the findings from assessment activities to all agency staff? (never, rarely, often, always, don’t know)

Community-based skills.

Seven items assessed the respondent’s perception of how well his or her fellow health department staff as a group performed in areas such as working with community groups and minority populations, program planning, and assessment. These 7 items were then repeated, with the respondent being asked to rate his or her own skills. These 14 total items had a 3-point response category, with “high,” “medium,” and “low” as possible responses.

Community participation in health department planning.

Five items measured the respondent’s perception of the extent to which community members were involved in planning and implementation of health department programs. Three of the items used a 5-point Likert scale, with “always,” “often,” “rarely,” “never,” and “don’t know” as possible responses. Two additional items included 2 response categories, “yes” and “no.”

Health department’s networking with other community agencies and groups.

Three items measured the respondent’s perception of his or her health department’s frequency and extent of networking with other community agencies and groups. These items used a 5-point Likert scale, with “always,” “often,” “rarely,” “never,” and “don’t know” as possible responses.

Assessment process.

Five items focused on the health department’s community assessment process. Three of these items used a 5-point Likert scale, with “always,” “often,” “rarely,” “never,” and “don’t know” as possible responses. One additional item included “yes” and “no” as response categories. The final item, which asked how often an assessment was performed, used a 5-point Likert scale, with “annually,” “every 2 years,” “every 3 years,” “never,” and “don’t know” as possible responses.

Description of Data Analysis

In the first stage of data analysis, our objective was to determine whether the factor structure revealed in the data would justify the construction of scales to measure the hypothesized dimensions. Using SAS (SAS Institute Inc, Cary, NC), we conducted a factor analysis with an orthogonal rotation on responses to the 27 items intended to measure the 5 dimensions of community-based participatory public health skills and competencies. A response of “don’t know” was interpreted to signify a neutral level of involvement and knowledge of the respondent that nevertheless needed to be acknowledged; this response was therefore placed at a neutral point on the 5-point scales. Where appropriate, items were reverse-scaled to ensure consistency in the direction of responses. We excluded from the principal components analysis 102 respondents with missing responses on 1 or more items. Hence, 180 respondents were included in the principal components analysis. Their responses to all items were standardized with z scores for this analysis.

In the second stage of analysis, our objective was to determine which survey items to retain for the scales intended to measure the factors—i.e., the 5 dimensions of community-based participatory public health. The Cronbach α was calculated for each of the factor scales identified in the factor analysis to assess the internal consistency of the scales. Finally, to examine the scale scores among the participating health department employees, we calculated descriptive statistics for the scales.

RESULTS

On the basis of our articulation of the 5 dimensions of community-based skills (of both the agency staff as a group and the individual respondent), community participation, networking, and assessment, we expected 5 factors in this analysis. However, exploratory factor analysis followed by orthogonal rotation yielded only 4 factors, according to the proportion criterion. The fifth proposed dimension, assessment process of the health department, did not emerge from the factor analysis as a discrete factor.

Twenty of the original 27 items loaded at .40 or above on these 4 factors and were thus retained for the next step in the analysis. Items 18, 19, 22, 24, 25, 26, and 27 were dropped from future analyses, because they did not contribute to explaining variation in the factors. The next step involved a second factor analysis in which the 4 factors were specified. Twenty of the 27 items loaded at .40 or above on at least 1 of the 4 factors (Table 2 ▶).

TABLE 2—

Results of Factor Analysis to Identify Components of Community-Based Participatory Practice

Item	Factor 1: Others’ Skills	Factor 2: Own Skills	Factor 3: Community Participation	Factor 4: Networking
Eigen value	4.4887	2.5864	1.2779	.7296
Cronbach α	.87	.83	.63	.84
Community-based skills of your agency staff as a group in:
community organizing	.74	.08	.08	–.04
working with community groups	.74	.11	–.01	.03
community assessment	.73	.02	.22	.01
advocating needs in the community	.71	0	.15	.07
program planning	.70	.09	.25	.12
communicating with minority populations	.64	.10	.09	.02
influencing public health policy	.60	–.07	.24	.03
Community-based skills of yourself in:
community organizing	.01	.73	.08	.0
working with community groups	.05	.70	–.03	.11
community assessment	.04	.63	.14	.01
advocating needs in the community	.04	.63	.18	.8
program planning	–.01	.66	–.04	.13
communicating with minority populations	.06	.43	.26	–.06
influencing public health policy	.15	.46	.26	.01
Community participation in health department planning
How often do the programs you work with address problems identified by the community, when public health statistics point to different problems?	.10	.11	.58	.03
How often has your agency’s assessment tried to identify the strengths as well as the weaknesses of the communities it serves?	.13	.16	.49	.08
How often do the programs you work with use feedback from the communities you are serving to make decisions on these programs?	.20	.15	.44	.17
How often does your agency consult community members before new programs are introduced in their community?	.24	.06	.43	.13
Agency’s frequency of networking with other community agencies and groups
How often does your agency jointly plan program activities with other agencies/organizations?	.04	.12	.14	.79
How often does your agency communicate or network about its activities in certain communities with other local agencies or organizations serving the same communities?	.07	.06	.16	.78

The first factor had 7 items loading above .40. These items focused on the respondent’s perception of the community-based and core function skills of the health department’s staff. We named this factor “other staff’s skills.”

The second factor had 7 items that loaded above .40. These items focused on the community-based and core function skills of the respondent. We named this factor “own skills.” As with the first factor, and based on the results of the factor analysis, we retained all 7 of the items that were included to capture the respondent’s own community-based skills.

The third factor had 4 items reflecting what we called “community participation.” Three of these items were intended to reflect actual participation, and 1 was intended to reflect assessment (“How often has your agency’s assessment tried to identify strengths as well as the weaknesses of the communities it serves?”).

The fourth factor, defined as “networking,” contained 2 items that focused on networking by the health department with other organizations and agencies in the community. The third item intended to reflect networking—“How often does your agency exchange resources with other agencies and organizations?”—did not load at .40 or above, so it was not retained.

The Cronbach α ranged from .63 to .87 for the 4 factors, indicating moderate to high internal consistency for each scale.

The descriptive statistics for the 4 factors (Table 3 ▶) indicated a moderate level of participatory public health practice in this sample. The variation and range in the scales suggest that they may be useful measures of change in community-based capacity by health agencies.

TABLE 3—

Descriptive Statistics for 4 Principal Factors Identified as Components of Community-Based Participatory Practice

Principal Factor	Mean (Standard Deviation)	Range
Others’ skills	2.3 (±0.48)	1–3
Own skills	2.0 (±0.46)	1–3
Community participation	2.3 (±0.59)	0.5–4.0
Networking	2.2 (±1.17)	0–4

DISCUSSION

A growing number of public health researchers and practitioners have suggested that community-based participatory approaches to public health may add a useful, if not fundamental, aspect to the practice of public health.2,4,14,15 Advocates of community-based participatory public health approaches suggest that having the community function as a partner in public health endeavors will ensure that social and cultural considerations of public health problems will be more fully explored and that any public health interventions will be more responsive to community needs.2,14,15 As public health agencies and departments take heed of these recommendations and strive to incorporate community-based participatory approaches in their performance of core functions, they must be able to characterize and evaluate their capacities to undertake such approaches. This use of “community-based” requires moving away from a definition of “community” as a setting or target in which to place a public health program and toward a definition that recognizes community members as a diverse set of collaborators/partners with different but essential sets of skills and resources to contribute to public health practice.2,16

We identified 4 discrete dimensions of community-based participatory public health practice by health departments. Two scales measured public health staff skills fundamental to the practice of community-based participatory public health. Individuals’ assessments of the skills of others in their agency and of their own skills, such as community organizing and working with community groups, suggest that attention to staff skills is important. Individuals who seek employment at local health agencies may have developed these skills in their professional training, although the literature suggests that such is not likely to be the case.17 Continuing education strategies to enhance the skills needed for community-based work seem warranted.

The other 2 scales measure organizational practices or processes important to community-based participatory public health practice. The factor identified as “community participation” highlights organizational recognition of a community as possessing assets and deserving decisionmaking power. The key role of a community in decisionmaking is reflected in questions about “addressing problems identified by the community, when public health statistics point to different problems” or “[using] feedback from the communities . . . to make decisions on programs.” An agency’s attention to the assets of communities, such as human resources available among groups of people, is essential to viewing the collective power of a community. Public health agencies are constrained by 2 components of the health systems to which they belong: the standardization of assessments and interventions inherent in health agencies and the perception of individuals as dependent clients with needs.18 Conversely, the alternative of community-based participatory practice would identify strengths as well as weaknesses and would address problems identified by the community when standardized statistics point elsewhere. Such an awareness of a community with collective contributions to make to public health decisions would move agencies away from the limited focus on deficiencies and need-based practices.

The other organizational practice or process, “networking,” involves joint communication and planning among agencies serving the same communities. This scale measures a health department’s recognition of the need to combine and coordinate resources from multiple sectors to address the complex social and economic issues that contribute to the health of communities. Networking implies that an agency understands collaboration to be fundamental to community-based participatory public health practice.

One limitation of this study was the failure to capture a factor associated with the core public health function of assessment. This may reflect the fact that the items selected to measure that function portrayed a rather narrow view of assessment. For example, 3 of the 5 items asked how respondents used data, in contrast to asking how data were generated or obtained. One item asked about the frequency of community assessments, based on the unproven assumption that health departments that engage in more frequent community assessments may be more community based. It is noteworthy that the 1 item that clearly reflected a community-based principle—attention to the assets or strengths of communities—was retained by the factor analysis, but as part of the “community participation” factor.

A second limitation stems from our study’s reduced sample, which resulted from both the 66% response rate and the need to exclude respondents for whom data were missing. Although the direction of potential bias was not apparent, it is conceivable that responses from the more committed individuals—as indicated by their completion and return of the surveys—enhanced the validity of the factors. On the other hand, the scales may have overstated the level of community-based capacity in these departments if the nonrespondents are assumed to have been less committed to participatory research and practice.

A third limitation involves the use of differing response categories. Although the use of z scores diminished the possible effects of using different numbers of response codes (e.g., 3 codes vs 5 codes), the use of differently worded response categories within the same scale may have affected the psychometric capabilities of the method.

Through our analysis, we have identified and measured skills and competencies that may influence local health agencies’ commitment to community-based participatory public health. One important issue for the further development of these measures is how to refine the questions and scales that tap into the core function of assessment.

Our analysis has 2 implications for the practice of public health. First, our findings suggest that it is possible to operationalize community-based performance to guide health agencies as they determine their capacities to become more “community based.” For example, departments can assess their employees’ skills and provide training, or they can examine how their policies enhance or impede community participation. The second implication is that efforts by policymakers and professionals to hold health agencies accountable can and should encompass indicators of “community basedness.” Unless health agencies know that elected officials, community members, public and private funders, and others will evaluate their performance with regard to their community-based capacities and interventions, it is less likely that they will develop and implement such programs to enhance the health of communities.