Community Health Assessment and Improved Public Health Decision-Making: A Propensity Score Matching Approach

Abstract

Objectives. We analyzed the likelihood of chronic disease prevention activities delivery, as a proxy measure of public health decision-making and actions, given that local health agencies (LHAs) implemented a community health assessment and improvement plan in their communities.

Methods. Using a propensity score matching approach, we linked data from the 2010 National Association of County and City Health Officials profile of LHAs and the 2010 County Health Rankings to create a statistically matched sample of implementation and comparison LHAs. Implementation LHAs were those that implemented a community health assessment and improvement plan. We estimated the odds of chronic disease prevention activities delivery and the average treatment effect on the treated.

Results. Implementation group LHAs were 2 times as likely (95% confidence interval = 1.60, 2.64) to deliver population-based chronic disease prevention programs than comparison group LHAs. Furthermore, chronic disease prevention activities were more likely to be delivered among implementation group LHAs (6.50–19.02 percentage points higher) than in comparison group LHAs.

Conclusions. Our results signal that routine implementation of a community health assessment and improvement plan in LHAs leads to improved public health decision-making and actions.

The US public health services delivery system faces many challenges, such as limited funds and complex population health needs. Chronic diseases are the leading cause of death and disability in the United States, and they impose a heavy economic burden on the health system.^1,2 Chronic disease prevention activities are sparsely delivered despite the fact that 7 of 10 deaths are caused by chronic disease every year and 75% of US health care costs are attributable to preventable diseases,^1,2 a situation that emphasizes the need for improved public health decision-making and actions.

BACKGROUND

The 1988 Institute of Medicine report The Future of Public Health recommended that

every public health agency regularly and systematically [collect, assemble, analyze, and make] available information on the health of the community, including statistics on health status, community health needs, and epidemiologic and other studies of health problems.^3(p7)

A community health assessment is the practical application of this recommendation. It provides a framework for public health agencies to identify and describe population health needs, factors contributing to health outcomes, and resource availability or disparity in their community.³ A community health assessment also informs the development of a community health improvement plan, which is “a long-term, systematic effort to address health problems on the basis of the results of assessment activities.”^4(p14)

The National Association of County and City Health Officials (NACCHO) has defined a local health agency (LHA) as “an administrative or service unit of local or state government, concerned with health, and carrying some responsibility for the health of a jurisdiction smaller than a state.”^5(p3) LHAs can serve as effective vehicles for delivering community-based chronic disease prevention activities because of their wide geographic distribution across the United States, their statutory authority to implement public health laws and programs, and their relationships with diverse community stakeholders.^6,7

Since the 1988 Institute of Medicine recommendation, community health assessment and improvement plan implementation has been a part of public health practice.^3,8 Recent policy efforts now strongly encourage their routine implementation in public health agencies and nonprofit hospitals.^9,10 Specifically, to obtain and maintain accreditation the Public Health Accreditation Board requires public agencies to implement a community health assessment and improvement plan every 5 years.⁹ In addition, the Patient Protection and Affordable Care Act¹⁰ mandates that nonprofit hospitals implement a community health assessment and improvement plan every 3 years as part of their community benefit requirements to maintain tax-exempt status (see Internal Revenue Service Revenue Ruling 69–645, 1969–2.C.B.117).¹¹

In theory, the community health assessment and improvement plan process provides a logical order for public health practice decision-making and actions.³ The foundational process of community health assessment and improvement plan implementation and its barriers are well documented because most previous empirical studies on community health assessment and improvement plans have focused on their implementation instead of examining their impacts.^12–25 Empirical studies have shown that when LHAs implement assessment and planning mechanisms, they generate community engagement and prompt collaboration with other health systems stakeholders, such as nonprofit hospitals and businesses.^26–28 Overall, implementation barriers include low capacity, lack of funds and infrastructure, constrained resources, and differences in Public Health Accreditation Board and Internal Revenue Service cycle schedules.^19–21,29

Consequently, an evidence gap exists in how the assessment and planning mechanisms translate into decisions and actions. The impacts of community health assessment and improvement plan implementation are understudied, and its effectiveness has not been well documented or evaluated.^29–31 The Assessment Initiatives project of the New York State Department of Health resulted in maintenance of ongoing programs such as healthy heart and breast cancer prevention activities.¹⁸ Aside from this study, no other studies have empirically shown that implementing a community health assessment and improvement plan translates into improved decision-making and actions in public health.¹⁸ Such evidence is needed to elucidate the value of investing resources and undertaking community health assessment and improvement plans for LHAs, especially now that policy mechanisms are in place to encourage their routine implementation.^9–11 Community health assessment and improvement plan implementation in LHAs might also provide insights into the imbalance between chronic disease burden and level of investment in prevention activities (Figure 1).

FIGURE 1—

Theoretical framework for community health assessment and improvement plan implementation in local health agencies.

Note. CHA = community health assessment; CHIP = community health improvement plan; LHA = local health agency. Dotted lines indicate alternate causal pathways.

To help fill this evidence gap, we investigated whether the implementation of community health assessment and improvement in LHAs leads to improved public health decision-making and actions. Specifically, we used the likelihood of chronic disease prevention activities delivery in LHAs as a measure of improved decision-making and actions and assessed the likelihood of such activities in LHAs that implemented a community health assessment and improvement plan versus the likelihood in those that did not.

METHODS

We were not able to estimate the causal pathway between community health assessment and improvement plan implementation and public health decision-making and activities delivery. Instead, as a first step, we used variations in community health assessment and improvement plan incidence to test for an association between community health assessment and improvement plan implementation in LHAs and chronic disease prevention activities delivery in their communities.

Data and Study Population

We used observational cross-sectional data from the 2010 NACCHO Profile Survey of LHAs and the 2010 County Health Rankings to analyze whether the implementation of community health assessment and improvement in LHAs leads to improved public health decision-making and actions.

The 2010 NACCHO survey covered 2107 LHAs, with an 82% agency response rate from all states except Hawaii and Rhode Island.⁵ For this study, we considered only those LHAs that responded to questions 61 and 62 of the survey: “Has a community health assessment been completed for your local health department’s jurisdiction?” and “Has your local health department participated in developing a health improvement plan for your community?” On the basis of these criteria, the study sample included 1529 eligible agencies.⁵

We used the 2010 County Health Rankings data to obtain community-level prevalence of smoking, obesity, diabetes, and poor and fair health status as proxy community health status measures.³² The County Health Rankings data aggregated responses from the Centers for Disease Control and Prevention’s Behavioral Risk Factor Surveillance System.³² Smoking prevalence was based on adult smokers who had smoked at least 100 cigarettes in their lifetime.³³ Obesity prevalence was the percentage of the adult population with a body mass index greater than or equal to 30.³⁴ Diabetes prevalence was the percentage of the adult population who responded “yes” to the following question: “Have you ever been told by a doctor that you have diabetes?” Diabetes types were not differentiated, and those with gestational diabetes were included in diabetes prevalence.³⁴ Poor or fair health prevalence was the percentage of adults who identified themselves as having poor or fair health in the Behavioral Risk Factor Surveillance System.³⁵

Measures and Specifications

Chronic disease prevention activities delivery.

As outcome measures for this analysis, we used the LHAs’ assurance of chronic disease prevention activities delivery in their communities from the 2010 NACCHO survey. Assurance of activities delivery meant LHAs either contracted out the activities to other organizations or conducted them directly to ensure they were available in their communities.^3,5 NACCHO defined contracted out as the LHA’s paying another organization to perform a public health activity on its behalf.⁵ On the basis of these specifications, we selected screenings for cardiovascular diseases, high blood pressure, and diabetes; chronic disease epidemiology and surveillance (hereinafter referred to as chronic disease epidemiology); and population-based primary intervention programs topically focused on chronic disease prevention, tobacco, nutrition, and physical activities.⁵

Community health assessment and improvement plan.

Our primary explanatory variable was LHA implementation of a community health assessment and improvement plan within a 3-year period (2007–2009).

Other explanatory variables.

On the basis of previous public health services and systems studies,^36,37 we selected the following public health agency characteristics as observed covariates: size of population served; LHA expenditure per capita (adjusted to 2012 US dollars using the Consumer Price Index); number of full-time-equivalent staff per jurisdiction; presence of local board of health; health agency authority, classified as decentralized to local government or centralized at the state level; top executive’s (highest ranking employee with administrative and managerial authority) full-time employment status; and health officer’s full-time employment status.⁵ To normalize distributions, we used logarithmic values for the size of the population served, LHA expenditure per capita, and number of full-time-equivalent staff per jurisdiction. In addition, we used proxy measures of community health status from 2010 County Health Rankings data.³²

Statistical Analysis

The analyses in this study were conducted at the community level, and the unit of analysis was the LHA. We first used robust logistic regression models to estimate the association between community health assessment and improvement plan implementation and the delivery of each chronic disease prevention activity by LHAs in their community. Specifically, the unadjusted and adjusted odds ratios compared the likelihood of activities delivery in LHAs that implemented a community health assessment and improvement plan with the likelihood of activities delivery in LHAs that did not. The adjusted odds ratios were controlled for observed covariates, including LHA characteristics and community health status levels.

Given the observational data used for this study, the basic logistic regression models might not adequately control for biases resulting from systematic differences between agencies that implemented a community health assessment and improvement plan and those that did not. For example, agencies that implemented a community health assessment and improvement plan tended to be larger and have more resources than their counterparts.³¹ Moreover, random assignment in implementation and comparison groups was not possible or ethical because of the community-level analyses. Consequently, to address these possible sources of bias, we used a propensity score matching (PSM) approach to create a statistically matched set of implementation and comparison groups.^38,39 Agencies that implemented a community health assessment and improvement plan were assigned to the implementation group, and statistically matched nonimplementing agencies were assigned to the comparison group. We then used the statistically matched sample to retest the hypothesis that an association exists between community health assessment and improvement plan implementation and the delivery of chronic disease prevention activities in LHAs.

A propensity score is the probability of being in the implementation group, given a set of observed covariates.⁴⁰ In this study, we defined the propensity score as the conditional probability of community health assessment and improvement plan implementation in an LHA within a 3-year period (2007–2009), given a properly balanced set of observed covariates.^39–41 A PSM approach reduces the possible biases from observed covariates.^39,41,42 If successful, it mimics randomization in observational studies.^39,41,42

We estimated the propensity score for each agency with a logistic regression model, controlled for all selected observed covariates. We stratified the LHAs into 4 individual blocks on the basis of their propensity score distributions and confirmed appropriate balancing properties of individual covariates within each block.^39,41,42 We then used the estimated propensity scores to match each LHA in the implementation group with 1 in the comparison group, in a 1-to-1 nearest neighbor matching, without replacement, and a caliper of 0.01 matching procedure.^39,41,42 The matching was made within the propensity score region of common support, which is when the implementation and comparison groups’ propensity scores overlap.^43,44

We reestimated the odds ratios with robust logistic regression models, adjusted for all selected observed covariates and weighted with the propensity scores.⁴⁵ We also calculated the average treatment effect on the treated, which is the average effect of implementation on activities delivery in LHAs assigned to the implementation group.⁴⁵ In this study, the average treatment effect on the treated is more appropriate than the population average treatment effect because CHA and CHIP implementation barriers in some local health agencies. To estimate the effect of implementing CHA and CHIP on chronic disease activities delivery in all local health agencies is not realistic.³⁹ We conducted all analyses with Stata version 12 (StataCorp LP, College Station, TX).

RESULTS

From 2007 to 2009, a total of 1529 LHAs met the eligibility criteria, of which 508 LHAs had implemented a community health assessment and improvement plan and 1021 had not. After using the PSM approach, 460 LHAs were assigned to the implementation group and were successfully matched with 933 LHAs in the comparison group. On average, LHAs that implemented a community health assessment and improvement plan served more people, spent approximately $10 more per capita on total annual expenditures, and had roughly 30 more full-time equivalents per jurisdiction than those that did not, both before and after the PSM step, and chronic disease prevention activities were delivered more often among implementing agencies (Table 1). The prevalence of chronic disease–related community health status was moderately uniform across all eligible agencies, with slightly higher rates in nonimplementing agencies (Table 1).

TABLE 1—

Local Health Agency Characteristics and Community Health Status, Before and After Propensity Score Matching, 2007–2009: 2010 NACCHO Profile Survey and 2010 County Health Rankings, United States

	Before Propensity Score Matching (n = 1 529)		After Propensity Score Matching (n = 1 393)
Variable	With CHA and CHIP (n = 508), % or Mean (SD)	Without CHA and CHIP (n = 1 021), % or Mean (SD)	With CHA and CHIP (n = 460), % or Mean (SD)	Without CHA and CHIP (n = 933), % or Mean (SD)
Chronic disease prevention activities delivery
Cardiovascular disease screening	42.52	33.10	44.78	34.42
High blood pressure screening	73.62	66.60	76.09	67.1
Diabetes screening	53.35	43.49	55.22	44.27
Chronic disease prevention programs	70.08	50.64	72.39	52.52
Chronic disease epidemiology and surveillance	53.35	39.47	55.65	40.51
Physical activity program	67.52	53.18	70.22	54.88
Tobacco control programs	82.68	74.53	84.28	75.86
Nutrition programs	84.06	73.26	85.87	74.76
LHA characteristics
Presence of local board of health	78.74	75.02	78.87	75.81
Decentralized local authority	77.95	67.58	78.91	68.17
Top executive, full-time status	94.88	90.89	95.42	92.1
Health officer, full-time status	14.76	14.50	23.21	22.5
Size of population served	141 569 (318 957)	125 661 (413 811)	143 059 (315 106)	127 386 (424 362)
Expenditure per capita, adjusted to 2012 US$	71.96 (97.02)	62.27 (60.24)	78.10 (100.36)	62.36 (60.46)
No. of FTEs per jurisdiction	91.92 (348.25)	61.93 (208.03)	94.91 (362.20)	64.34 (215.71)
Community health status, prevalence
Obesity	28.59 (3.40)	28.88 (3.98)	28.48 (3.41)	28.86 (3.94)
Diabetes	9.43 (1.82)	9.86 (2.17)	9.37 (1.77)	9.81 (2.12)
Smoking	21.95 (5.31)	21.98 (5.87)	21.84 (5.38)	21.93 (5.76)
Poor or fair health	16.17 (5.49)	16.85 (5.76)	15.96 (5.48)	16.78 (5.74)

Note. CHA = community health assessment; CHIP = community health improvement plan; FTE = full-time equivalent; LHA = local health agency; NACCHO = National Association of County and City Health Officials.

The unadjusted and adjusted odds ratios estimated before the PSM procedure are reported in Table 2. Implementing LHAs were more likely to deliver chronic disease prevention activities than nonimplementing LHAs. The adjusted odds ratios ranged from 1.45 (95% confidence interval = 1.14, 1.83) for cardiovascular disease screening to 2.06 (95% confidence interval = 1.62, 2.63) for population-based chronic disease prevention programs.

TABLE 2—

Odds Ratios Before Propensity Score Matching of Chronic Disease Prevention Activities Delivery in Local Health Agencies and Bootstrapped Confidence Intervals Associated With CHA and CHIP Implementation, 2007–2009: 2010 NACCHO Profile Survey and 2010 County Health Rankings, United States

Outcome Variables	OR (95% CI)	AOR (95% CI)
Cardiovascular disease screening	1.53 (1.23, 1.92)	1.45 (1.14, 1.83)
High blood pressure screening	1.45 (1.13, 1.85)	1.51 (1.16, 1.96)
Diabetes screening	1.51 (1.21, 1.88)	1.51 (1.20, 1.89)
Chronic disease prevention programs	2.35 (1.86, 2.97)	2.06 (1.62, 2.63)
Chronic disease epidemiology and surveillance	1.78 (1.43, 2.21)	1.53 (1.22, 1.92)
Physical activity prevention programs	1.90 (1.50, 2.39)	1.63 (1.28, 2.08)
Tobacco prevention programs	1.68 (1.26, 2.23)	1.47 (1.09, 2.00)
Nutrition prevention programs	1.98 (1.48, 2.64)	1.79 (1.31, 2.45)

Note. AOR = adjusted odds ratio; CHA = community health assessment; CHIP = community health improvement plan; CI = confidence interval; NACCHO = National Association of County and City Health Officials; OR = odds ratio.

After the PSM procedure, we re-estimated the odds ratios (Figure 2). Similar to results from the pre-PSM analyses, the implementation group was also more likely to deliver chronic disease activities than the control group. For example, LHAs in the treatment group were 2 times more likely to deliver population-based chronic disease prevention programs in their communities than LHAs in the statistically matched comparison group. In addition, on average, the likelihood of chronic disease prevention activities being delivered among agencies implementing a community health assessment and improvement plan was 6.50 to 19.02 percentage points higher than in nonimplementing agencies (Figure 3).

FIGURE 2—

Postpropensity score matching estimated odds ratio of chronic disease prevention activities delivery in local health agencies and bootstrapped confidence intervals: 2007–2009.

Note. Whiskers indicate 95% confidence intervals.

FIGURE 3—

Postpropensity score matching average treatment effect on the treated for chronic disease activities delivery in local health agencies: 2007–2009.

DISCUSSION

Using chronic disease prevention activities delivery as a proxy measure of public decision-making and actions, we found that LHA implementation of a community health assessment and improvement plan does lead to improved public health decision-making and actions. Building on previous community health assessment and improvement plan studies, we provide early supporting empirical evidence that these mechanisms are beneficial to public health practice.^12–17 In addition, our findings lend preliminary support for recent public health policies that incentivize increased routine implementation of community health assessment and improvement plans in health agencies.^9–11

Evidence has shown that low capacity and constrained resources are implementation barriers to community health assessment and improvement plans,^19–21,29 which suggests that better resourced LHAs tend to implement these mechanisms more than those with fewer resources.⁵ Better resourced agencies might also be more likely to provide a broader scope of services, independent of whether they implemented a community health assessment and improvement plan. As previously mentioned, we were not able to establish the causal pathway between implementation of a community health assessment and improvement plan and public health decision-making and activities. Rather, we used variations of community health assessment and improvement plan implementation to test whether they were beneficial for public health delivery, which we found might be true. Consequently, a reversed causal pathway might have also occurred among the LHAs included in this study. That is, LHAs might have delivered chronic disease prevention activities before they implemented a community health assessment and improvement plan in their community. If this scenario occurred, we posit that community health assessment and improvement plan implementation might still be beneficial for public health decision-making and actions. A community health assessment and improvement plan would have helped LHAs identify gaps in their services and activities delivery, thus improving decisions and actions to better meet the health needs of their communities and maintain adequate service provision.

The main limitation of this study is the possible selection bias resulting from the cross-sectional nature of our data. We used PSM to correct the bias by creating statistically matched pairs of implementation and comparison groups. However, PSM relies on the assumption that unobserved characteristics do not influence community health assessment and improvement plan implementation. It ignores other unobserved factors that might confound the positive association we found between community health assessment and improvement plan implementation and chronic disease prevention activities delivery.^39–42

Another limitation is the lack of more granular measures of decision-making and actions, such as services and activities reach, volume, intensity, equity, and adherence to evidence-based public health decision-making and practice. Moreover, findings on community health assessment and improvement plan implementers versus nonimplementers might not be generalizable because of high variations in LHA characteristics, such as geography, population demographics, and local policies that are beyond the scope of this study.

On the basis of our findings, areas of future research might include (1) longitudinal studies of community health assessment and improvement plan implementation and more refined measures of services delivery before and after implementation to elucidate potential lagged effects and causal pathway, (2) studies of the measures and effects of other health system stakeholders such as hospitals and businesses, and (3) comparative effectiveness studies of alternative approaches to implementing community health assessment and improvement plans (e.g., more effective ways to engage community partners and stakeholders, mobilization through evidence-based actions, faster and more efficient but affordable resources for the same or better results).

Faced with limited public health funds and resources, LHA leaders may want to use the findings from our study to increase support for community health assessment and improvement plan implementation through partnerships with other public health entities, such as nonprofit hospitals, businesses, and academic institutions.^26,46 They may also want to use our findings in addition to the results of their community health assessment and improvement plan to galvanize political and community support for public health services and activities delivery and to make their agencies more competitive for limited funding opportunities.