Incentives to Encourage Participation in the National Public Health Accreditation Model: A Systematic Investigation

Abstract

Objectives. We sought to identify the incentives most likely to encourage voluntary participation in the national public health accreditation model.

Methods. We reviewed existing incentives, held meetings with key informants, and conducted a survey of state and local public health agency representatives. The survey was sent to all state health departments and a sample of local health departments. Group-specific differences in survey responses were examined.

Results. Survey response rates were 51% among state health department representatives and 49% among local health department representatives. Both state health department and local health department respondents rated financial incentives for accredited agencies, financial incentives for agencies considering accreditation, and infrastructure and quality improvement as important incentives. State health department respondents also indicated that grant administration and grant application would encourage their participation in the national accreditation model, and local health department respondents also noted that technical assistance and training would encourage their participation.

Conclusions. Incentives to encourage participation of state and local agencies in the national voluntary accreditation model should include financial support as well as support for agency infrastructure and quality improvements. Several initiatives are already under way to support agency infrastructure and quality improvement, but financial support incentives have yet to be developed.

The Institute of Medicine's 2003 report The Future of the Public's Health in the 21st Century called for increased accountability by state and local public health agencies in performing the core public health functions (assessment, policy development, and assurance) and the 10 essential public health services (e.g., monitoring health status, diagnosing and investigating health problems). An accreditation program was identified as 1 possible method of achieving this goal,¹ and the recent establishment of the Public Health Accreditation Board (PHAB) represents an important step in the process. The PHAB was established to provide state and local health departments with a process through which they could assess their capacities against recognized standards and apply for accreditation by a national entity.

The PHAB model and the proposed standards are based heavily on the outcomes of the Exploring Accreditation Project (EAP). The EAP assessed the feasibility of the national voluntary public health accreditation model, which was designed to assess state and local health departments' capacity and performance against national standards.² The EAP was supported by funding from the Centers for Disease Control and Prevention (CDC) and the Robert Wood Johnson Foundation, and work on the project was led by the Association of State and Territorial Health Officials (ASTHO), the National Association of Local Boards of Health, the National Association of County and City Health Officials (NACCHO), and the American Public Health Association. These organizations are providing continuing support to the PHAB. Given this investment of time and resources, it is critical to ensure that the conditions that will allow the national accreditation model to be successful are in place.

As a means of informing the development of the public health accreditation model, Mays conducted a review of accreditation programs from a variety of service industries.³ That review showed that accreditation programs with strong and visible incentives (e.g., increased eligibility for funding) encourage organizations to participate and are more likely to function successfully. For example, in the health care industry, eligibility for participation in Medicare has served as a strong incentive for hospitals to take part in the Joint Commission on Accreditation of Healthcare Organizations process.

EAP surveys of state and local agency personnel have identified quality and performance improvements, consistency among health departments, and recognition by peers as the most important incentives associated with accreditation. Another incentive that has been identified is access to the resources and services necessary to complete the accreditation process.² The EAP Finance and Incentives Work Group has concluded that the most feasible incentives would be those that provide financial support in the process of applying or preparing for accreditation, improve the efficiency of programs, and provide new grant funding to accredited health departments.⁴

The EAP Steering Committee and the Finance and Incentives Workgroup has recommended that incentives for public health accreditation be uniformly positive. Types of incentives not recommended include coercive or restrictive incentives, such as loss of funding for health departments that do not participate in accreditation; incentives that could create real or perceived conflicts of interest, such as access to special training; and incentives that could exacerbate problems for health departments with few resources, such as small or rural health departments.

Although these principles and survey results informed the EAP process, there has been no effort to systematically explore the incentives most likely to encourage voluntary participation in the national public health accreditation model. The EAP data collection efforts were limited by incomplete identification of potential incentives, and data were collected primarily from convenience samples.

Also informing the research described here were evaluations of the North Carolina Local Health Department Accreditation Program; these evaluations identified several key benefits (e.g., eligibility for additional funding) as potential incentives for participating in that accreditation program.⁵ We sought to identify additional accreditation incentives and to determine those most likely to encourage voluntary participation by state and local health officials in the proposed national public health accreditation model.

METHODS

We used a participatory research model in our study, engaging an advisory group to ensure that our results would inform the continued development of the PHAB and to facilitate access to research participants. Advisory group members included representatives from CDC, the National Network of Public Health Institutes, ASTHO, NACCHO, the PHAB, the National Association of Local Boards of Health, and the Robert Wood Johnson Foundation. We used both qualitative and quantitative data collection methods (discussion groups and a survey), given that such mixed data collection techniques increase a study's validity through providing an improved understanding of the phenomenon under investigation.^6–8

Our study involved 3 phases implemented during 2007–2008. The first (literature review) and second (meetings with federal and philanthropic organizations) phases were designed to identify all existing and potential incentives for accreditation and to examine the feasibility of those incentives. From the information gathered in these phases, we created a list of incentive categories (Table 1). In the third (research) phase, we attempted to determine the most compelling and relevant incentive categories that would encourage state and local health officials to participate in the national accreditation model.

TABLE 1.

Potential Incentives to Encourage Participation in Public Health Accreditation

Category	Examples
National support	Policy statements that indicate federal agency support for accreditation.
	Recognition from agencies and foundations to accredited agencies.
Training and technical assistance for agencies considering accreditation	Training, technical assistance, and consultation to prepare an agency for accreditation.
	Access to CDC field assignees to prepare for accreditation.
Benefits of participating in accreditation	Provision of a team-building opportunity for staff.
	Improvements in working relationships between agency personnel and partners (e.g., between state and local personnel).
Financial incentives for agencies considering accreditation	Funds to prepare to apply for accreditation.
	Funds to address potential agency deficits before applying for accreditation.
Grant administration streamlining	Fewer reporting requirements.
	Increased flexibility to use unobligated funds.
Financial incentives for accredited agencies	Eligibility to apply for grants and contracts. Access to funding support for quality improvements.
Marketing–recognition of accredited agencies	Awards (e.g., provision of awards to accrediting agencies).Classification (e.g., accreditation with distinction ratings).
Grant application improvements for accredited agencies	Streamlining of application process.
	Accreditation status considered as part of scoring criteria.
Infrastructure and quality improvement opportunities	Identification of areas for health department improvement.
	Receipt of benchmarking data and consultation on quality improvement activities.

Note. CDC = Centers for Disease Control and Prevention.

Discussion Groups

We conducted discussion groups with NACCHO and ASTHO members to obtain their feedback on the list of incentive categories, identify any additional incentives, and determine the most relevant incentives to include in a national survey of state and local public health agency representatives. Notes from discussion groups were compiled and summarized to inform survey development.

Three discussion groups were conducted via telephone with 12 NACCHO participants representing 9 health agencies. Two of these agencies were small (serving populations of less than 50 000), 4 were medium in size (serving populations between 50 000 and 500 000), and 3 were large (serving populations of more than 500 000). In the case of ASTHO members, an invited, closed session with state health officials or their representatives was held at the association's 2007 annual meeting. The 35 participants were split into 3 discussion tables, each with a facilitator and note taker. Twenty-one states, diverse in terms of geography and type of governance structure (centralized or decentralized), were represented. The remaining 14 participants represented government agencies or other organizations.

Survey Development and Administration

Incorporating feedback from the discussion groups, we developed a survey to examine the likelihood that specific categories of incentives would encourage health officials' participation in public health accreditation. The reimbursement category was not included in the survey because discussion group participants were concerned that this category was too variable, would apply only to states providing direct services, or would be perceived as punitive. Because a separate PHAB survey was being conducted with the same respondents at the same time, the 2 surveys were combined to avoid respondent confusion. Although the topics were different, both surveys addressed accreditation.

Survey items asked respondents to rate how likely each incentive category was to encourage them to volunteer for accreditation (1 = not at all likely, 7 = very likely), to select the 2 categories most likely and the 1 category least likely to encourage them to volunteer for accreditation (hereafter “final category”), to describe why they made each of these choices, to rate their level of agreement regarding their agency volunteering for accreditation, and to rate their familiarity with the national public health accreditation model and the PHAB. Additional agency information, such as population size served by the agency, was provided by NACCHO and ASTHO and preloaded by the survey vendor.

Survey Sampling Strategy

All 57 ASTHO members or state health department officials were included in the survey sample. NACCHO provided a sampling frame consisting of all unique e-mail addresses for an updated set of respondents to the 2005 National Profile of Local Health Departments survey.⁹ Because individuals who elected not to take part in the NACCHO profile survey also were unlikely to respond to our survey, we included only profile respondents in our sample.

A total of 1930 local health departments were included in the sampling frame, which was grouped into 6 strata. Subgroups were defined according to local health department governance structure (units of local government [decentralized] or units of the state health agency [centralized]) and population of jurisdiction: small (less than 50 000), medium (between 50 000 and 500 000), or large (more than 500 000). To ensure sufficient numbers for our subgroup analyses, we overweighted the sample for large local health departments and units of the state health agency. Participants were randomly selected via the Stata version 10 (StataCorp LC, College Station, TX) random number function.

The survey was administered to 55 state health department representatives and 574 local health department representatives (representatives from South Carolina and Micronesia did not receive the survey instrument because of an error in survey administration). An initial e-mail and 4 reminders were sent to all nonrespondents, and a fifth reminder was sent to state health department nonrespondents.

Data Analysis

We used SAS version 9.1.2 (SAS Institute Inc, Cary, NC) to conduct the survey data analysis. Survey responses were assessed separately with univariate analyses (for both state health department and local health department data) as well as bivariate analyses (for local health department data).

In the case of the local health department data analysis, sample strata and weight statements were included in each survey procedure. Sampling weights were computed as the ratio of the total number in each of the 6 sampling strata divided by the number responding to the survey. This strategy compensated for both the unequal probabilities of selection and differences in response rates according to strata. The final weighted distribution matched the population distribution.

We assessed bivariate associations between the 6 strata and the incentive categories, likelihood of seeking accreditation, familiarity with the national accreditation model, and familiarity with the PHAB. We also examined associations between the incentive categories and likelihood of seeking accreditation, familiarity with the national accreditation model, and familiarity with the PHAB. We included significant bivariate associations in the linear regression analysis so that we could assess relationships between variables.

RESULTS

Of the 629 individuals who received the invitation to complete the survey, 309 responded (overall survey response rate: 49%). Twenty-eight of these respondents were state health department representatives (state health department subsample response rate: 51%).

In the case of the local health department sample, 281 individuals responded (local health department subsample response rate: 49%). Within the NACCHO local health department respondent groups, response rates were 51% for the decentralized strata (ranging from 43% for local health departments serving small populations to 63% for those serving large populations) and 43% for the centralized governance strata (ranging from 36% for local health departments serving small populations to 50% for those serving medium-sized populations).

Incentive Ratings and Explanations

State health department respondents.

Among state health department respondents, mean incentive category ratings (on the 7-point scale) ranged from 3.74 for benefits to 6.0 for financial incentives for agencies considering accreditation and grant administration (Table 2). State health department respondents' mean ratings in 5 categories—financial incentives for agencies considering accreditation, financial incentives for accredited agencies, grant administration, grant application, and infrastructure and quality improvement—were 5.0 or higher. The highest percentage (44%) of state health department respondents named financial incentives for accredited agencies as their first-choice incentive category; 24% named infrastructure and quality improvement as their first choice.

TABLE 2.

Incentive Category Ratings of State Health Department Respondents

Incentive Category	Meana±SE (95% CI)	Respondents Selecting as First Choice, %	Respondents Selecting as Second Choice, %	Respondents Selecting as Final Choice,b%
Financial incentives for agencies considering accreditation	6.00 ±0.167 (5.66, 6.34)	16.0	16.0	4.0
Grant administration	6.00 ±0.166 (5.66, 6.34)	0.0	20.0	4.0
Financial incentives for accredited agencies	5.93 ±0.208 (5.50, 6.35)	44.0	16.0	0.0
Grant application	5.59 ±0.220 (5.14, 6.04)	12.0	8.0	4.0
Infrastructure and quality improvement	5.04 ±0.207 (4.61, 5.46)	24.0	12.0	0.0
Training and technical assistance for agencies considering accreditation	4.74 ±0.211 (4.31, 5.17)	0.0	12.0	12.0
National support	4.63 ±0.241 (4.13, 5.13)	0.0	4.0	24.0
Benefits	3.74 ±0.211 (3.31, 4.17)	4.0	4.0	12.0
Marketing–recognition	3.93 ±0.231 (3.45, 4.40)	0.0	8.0	40.0

Note. CI = confidence interval.

^aRatings were made on a scale ranging from 1 (not at all likely to encourage) to 7 (extremely likely to encourage).

^bRespondents' choice of the category least likely to encourage them to volunteer for accreditation.

To determine other priority choice categories, we examined the combined percentages of first- and second-choice categories. Twenty percent or more of the respondents also rated the grant application and grant administration categories as a first- or second-choice incentive. Forty percent of state health department respondents selected marketing–recognition as their final category, and 24% selected national support as their final category.

In open-ended questions, state health department respondents indicated that their first- and second-choice incentives were tangible, offset or covered the cost of accreditation efforts, rewarded efforts to undertake accreditation, and contributed to agency quality improvement efforts. According to one respondent who selected financial incentives to prepare for accreditation, “[We are not] likely to receive state funds for the accreditation process[,] so assistance would be necessary.” In the words of another respondent:

The data obtained from the accreditation process would help us identify areas we need to strengthen within our infrastructure. Consultation along with benchmarking data will help to advance the progress of our local and state health departments.

Local health department respondents.

Among local health department respondents, mean incentive ratings ranged from 4.34 for benefits to 5.93 for financial incentives for agencies considering accreditation (Table 3). Local health department respondents' mean ratings were 5 or higher for the following categories: financial incentives for agencies considering accreditation, financial incentives for accredited agencies, grant administration, grant application, and technical assistance and training for agencies considering accreditation. As was the case with the state health department respondents, the highest percentage of local health department respondents (35.7%) selected financial incentives for agencies considering accreditation as their first-choice category; 14.8% selected this category as their second choice.

TABLE 3.

Incentive Category Ratings of Local Health Department Respondents

Incentive Category	Meana±SE (95% CI)	Respondents Selecting as First Choice, %	Respondents Selecting as Second Choice, %	Respondents Selecting as Final Choice,b%
Financial incentives for agencies considering accreditation	5.93 ±0.112 (5.71, 6.15)	35.7	14.8	2.9
Financial incentives for accredited agencies	5.65 ±0.130 (5.40, 5.91)	19.4	18.2	3.2
Grant administration	5.62 ±0.111 (5.40, 5.84)	4.4	5.4	9.8
Grant application	5.25 ±0.125 (5.00, 5.50)	2.2	7.5	5.2
Training and technical assistance for agencies considering accreditation	5.16 ±0.115 (4.93, 5.38)	9.1	28.4	5.6
Infrastructure and quality improvement	4.96 ±0.126 (4.71, 5.21)	20.1	13.3	3.1
Marketing–recognition	4.05 ±0.141 (3.78, 4.33)	4.4	5.5	40.4
Benefits	4.34 ±0.126 (4.09, 4.59)	3.9	2.8	7.2
National support	4.25 ±0.136 (3.98, 4.52)	0.9	4.2	22.6

Note. CI = confidence interval.

^aRatings were made on a scale ranging from 1 (not at all likely to encourage) to 7 (extremely likely to encourage).

^bRespondents' choice of the category least likely to encourage them to volunteer for accreditation.

To determine other priority choice categories, we again examined the combined percentages of first- and second-choice categories. Approximately 38% of local health department respondents identified financial incentives for accredited agencies as their first or second choice, and 33.4% identified infrastructure and quality improvement. Training and technical assistance for agencies considering accreditation was selected by 37.5% of respondents as their first or second choice, but it was primarily seen as a second choice (28.4% of the 37.5% identified it as a second choice). Approximately 40% of local health department respondents selected marketing–recognition as their final category, and 22.6% selected national support. No significant differences in mean incentive ratings among the NACCHO local health department strata were found.

In open-ended questions regarding first- and second-choice incentives, many local health department respondents indicated that if they were to consider volunteering their agencies for accreditation, financial support and other forms of support, such as training and technical assistance, would be necessary to offset the perceived costs. As one participant put it, “Our health department is very small and has a very small budget, so financial incentives to apply to be accredited would be necessary to get the ball rolling.”

Other local health department respondents explained that they were looking to improve the general quality of their health departments or to improve infrastructure and services. According to one respondent:

I am looking at accreditation as a means of self-improvement. Our Health Department does an excellent job in our community, but we would like to know if what we are doing makes a difference.

Explanations of final-category choices.

State health department and local health department respondents both selected marketing-recognition and national support as the incentives least likely to encourage their participation in the national accreditation model. Respondents indicated that these incentives were not as important to them or as likely as others to persuade their stakeholders, communities, and elected officials to support the agency volunteering for accreditation. In addition, respondents noted that these incentives were vague and not practical. According to a state health department respondent: “Our leadership in legislature care[s] less about national recognition and more about what our own state stakeholders believe to be true.”

More important, given the perceived effort needed to undertake accreditation, respondents did not view these incentives as compelling reasons to initiate the process. In the words of a local health department respondent, “The time and expense needed for accreditation must be more substantive than ‘marketing’ and ‘recognition.’ ” Table 4 summarizes the incentives most and least likely to encourage state health department and local health department respondents' participation in the national voluntary accreditation model.

TABLE 4.

Summary of Incentive Categories Most and Least Likely to Encourage Participation in the National Accreditation Model

	Most Likelya		Least Likelyb
Incentive Category	State Health Department Respondents	Local Health Department Respondents	State Health Department Respondents	Local Health Department Respondents
Financial incentives for agencies preparing for accreditation	X	X
Financial incentives for accredited agencies	X	X
Infrastructure and quality improvement	X	X
Grant administration	X
Grant application	X
Technical assistance		X
National support			X	X
Marketing–recognition			X	X

^aIncentive categories that state health department and local health department respondents selected as first and second choices (most likely to encourage participation in the national accreditation model).

^bIncentive categories that state health department and local health department respondents selected as final choices (least likely to encourage participation in the national accreditation model).

Other Accreditation Measures

The survey also examined state health department and local health department respondents' familiarity with the national voluntary public health accreditation model and the PHAB along with their likelihood of seeking accreditation. The majority of state health department respondents (55.5%) were familiar or very familiar with the national voluntary public health accreditation model; more than one third (38%) were familiar or very familiar with the PHAB; and 37% agreed or strongly agreed that their health department would seek accreditation under the national model.

Local health department respondents were not as familiar with the national model or the PHAB, with 29.8% indicating that they were somewhat familiar with the national model and the majority indicating that they were not at all or not very familiar with the PHAB. Nevertheless, 31% of local health department respondents strongly agreed or agreed that they would seek accreditation. No differences were found in these ratings for the 6 jurisdictions in the NACCHO local health department strata.

Associations Among Variables

Among the local health department respondents, there were significant linear relationships between all of the incentive categories and the likelihood of seeking accreditation under the national voluntary public health accreditation model. In other words, as respondents' ratings of each incentive as likely to encourage volunteering for accreditation increased, their likelihood of seeking accreditation increased.

DISCUSSION

In this study, we sought to identify the incentives most and least likely to encourage state and local health officials to volunteer their agencies for accreditation. Among both state and local officials, the incentive categories most likely to encourage participation in the national public health accreditation model were financial incentives to prepare for accreditation, financial incentives for accredited agencies, and support for infrastructure and quality improvement. Respondents from state health agencies also indicated that the grant administration and grant application incentive categories would encourage their participation in the national accreditation model. Respondents from local health agencies reported that provision of technical assistance for agencies to prepare for accreditation or to address areas of quality improvement would also encourage their participation in the national model.

EAP survey respondents identified quality and performance improvement, consistency among health departments, and recognition by peers as the most important incentives for accreditation. These results are consistent with the findings described here with the exception of recognition by peers. Although the recognition incentive category used in our study primarily addressed national-level recognition (in contrast to recognition by peers), respondents indicated that recognition in general was not a priority incentive and that it is inherent in accreditation. However, the other incentives included in this research, such as financial incentives to prepare for accreditation, were not included in the EAP survey. Our study provided a more complete array of potential incentives to state and local health agency respondents, which may have decreased the perceived importance of the recognition incentive.²

Among the local health department respondents to our survey, there was a statistically significant linear relationship between incentives and likelihood of seeking accreditation. This finding is consistent with data collected for the EAP, in which an association was observed between likelihood of seeking accreditation and benefit levels perceived by health agencies. Future studies may be needed to examine whether there is a causal relationship among these variables.

Limitations

There were several limitations of the third (research) phase of our study. Both the NACCHO and ASTHO discussion groups were convenience samples. The participation of these individuals was based on their interest and availability. The online survey format limited explanation of incentive category definitions and did not allow display of all examples of incentives within specific categories. Respondents' ratings of incentives may have been affected by a less than complete understanding of the incentive categories. The wording of the survey items, such as familiarity with the national accreditation model, relied on respondents' self-interpretations, which may have introduced response bias. The online survey was paired with another survey on PHAB messages, which affected the design and implementation of the survey, including quality control and pilot testing.

Areas for Further Research and Exploration

Several aspects of accreditation incentives could not be addressed in this research. First, we were unable to assess incentive thresholds, that is, whether a state or local public health agency can receive incentives in isolation of other state or local agencies' participation in accreditation. Second, we did not examine whether proposed public health standards and measures were consistent with existing federal agency grant requirements. Third, additional research may be needed to explore and verify incentives that state agencies can provide to local agencies. Finally, although the financial assistance category distinguished between offering incentives for agencies preparing for accreditation and accredited agencies, there was limited exploration on the timing of such incentives.

Given the linear associations between our incentive categories and the likelihood of applying for accreditation, there is a need for additional research on these variables as well as health officials' familiarity with the national model and the PHAB. State and local public health agencies in which there is a high level of familiarity with accreditation and quality improvement processes (e.g., those participating in the Multistate Learning Collaborative¹⁰ and the NACCHO Operational Definition projects¹¹) may be important laboratories for conducting additional research.

Implementing Incentives

Infrastructure and quality improvement and training and technical assistance accreditation incentives have begun to be provided by national organizations such as ASTHO, NACCHO, CDC, and the Robert Wood Johnson Foundation.^12–15 Only a few of these initiatives, however, were in place at the time of our survey. Additional initiatives may be needed to sufficiently encourage state and local health agencies to participate in the national accreditation model, and other key incentives have yet to be developed.

A caveat is necessary with respect to financial incentives for accredited agencies. Several discussion group participants and survey respondents warned that this incentive category could in fact be interpreted as a disincentive for accreditation as a result of its potential for being punitive. Respondents were clearly aware that, if they failed in their attempts to become accredited or lost their accreditation status, they could lose access to funds. The punitive nature of such incentives could be minimized by making them available only in the case of competitive grants for new programs, particularly those targeted to state agencies.

As incentives are further developed and implemented, several considerations may maximize their impact. First, state and local agencies may need a menu of incentives, or multiple incentives may be needed at any given time. Financial incentives to prepare for accreditation, quality improvement, and technical assistance may be a particularly strong set of incentives to be offered simultaneously. Second, national organizations, federal agencies, and the PHAB should intentionally coordinate development of incentives and consider the phased development of specific incentives. Finally, incentives should be pilot tested, evaluated in terms of their effectiveness, and appropriately communicated to state and local health officials.