Medicaid Pay-For-Performance (P4P) Programs and the Use of Preventive Health Care Services: A Preliminary Analysis of Immunization Status among Young Children

Abstract

Introduction:

Although Pay-for-Performance (P4P) programs are being increasingly used by state Medicaid programs to provide incentives for managed care plans to provide high quality care, no national study has examined the effects of these plans on commonly targeted outcomes such as the use of particular preventive care services.

Methods:

We use information on state Medicaid P4P programs from the Centers for Medicare and Medicaid Services combined with information from the National Immunization Survey 1999–2011 to study the effect of Medicaid P4P programs on children’s immunization status. We use difference-in-difference models that compare the effect of P4P programs on children’s immunization status for children estimated to be Medicaid-eligible before and after introduction of a P4P program relative to the “pre-post” change for other children in the same group of states. We also estimate difference-in-difference-in-difference models that compare these changes to those for children in states that do not implement P4P.

Results:

We find that Medicaid P4P increases the probability that a Hispanic or non-Hispanic white child is up-to-date on several vaccinations. For example, we find that the chance that these children are up-to-date on the measures, mumps, and rubella series increased by about 2.7–2.9 percentage points. However, we do not find statistically significant effects on all vaccines, and no statistically significant effects for non-Hispanic black children.

Conclusions:

This study provides some evidence that Medicaid P4P programs may be helpful in improving childhood vaccination rates. Further study of the effects on other targeted outcomes as well as the effects of different P4P program designs may further increase our understanding of the potential role of these programs.

Introduction

In the past decade or two, providing incentives for health care providers or payers to improve performance, commonly known as pay-for-performance (P4P) programs, has become a popular strategy to improve quality and control cost of health care services. P4P programs typically link payment rates or other incentives with overall performance of the plan or provider in order to encourage the provision of high quality and cost-effective care. Beginning in the late 1990s, various states have adopted P4P programs for their Medicaid managed care plans. Medicaid agencies in each state measure performance of participating managed care plans based on different criteria, such as improvement of child immunization rates, or improvements in the process or outcomes for treating particular conditions such as diabetes or heart disease. Many states target Healthcare Effectiveness Data and Information Set (HEDIS) or HEDIS-like measures consisting of rates of performance of particular preventive health care services.¹

As of July 1, 2010, there were 19 states that had adopted P4P programs for Medicaid managed care plans (Table 1).² Evidence on the effectiveness of P4P programs in the past literature has been mixed. One study that examined the effects of implementation of P4P in one health plan using another as a control group concluded that providing incentives for clinicians to obtain common, fixed performance goals may produce small improvements in performance, but may also mostly reward clinicians who have already obtained relatively high performance standards.³ Other studies that used randomized controlled trials of P4P programs for private plans found mixed results of P4P’s effect on meeting cancer screening, immunization, and other preventive care guidelines.^4,5,6 One report summarized existing and new P4P activities in state Medicaid programs,⁷ though no quantitative study has attempted to measure the effect of state Medicaid P4P on the use of preventive care among the low-income population at the national level.

Table 1:

Adoption of Medicaid Pay-for-Performance (P4P) Programs by State

	Adoption Year	Name of Program(s)
Wisconsin	1996	BadgerCare Plus
New Mexico	1997	NEW MEXICO SALUD! Minnesota Prepaid Medical Assistance Program-1115(a), MinnesotaCare Program For Families And Children, Minnesota
Minnesota	1999	Prepaid Medical Assistance Program-1932(a), Minnesota Senior Care/Minnesota Senior Care Plus
Rhode Island	1999	Global Consumer Choice Compact
New York	2000	Federal-State Health Reform Partnership (F-SHRP), Partnership Plan - Family Health Plus, Partnership Plan Medicaid Managed Care Program
Michigan	2001	Comprehensive Health Plan
Missouri	2001	MO HealthNet Managed Care/1915b
Ohio	2002	State Plan Amendment for Ohio’s full-risk managed care program
Maryland	2002	HealthChoice
Washington	2004	Healthy Options Sacramento Geographic Managed Care, San Diego Geographic
California	2005	Managed Care, Two-Plan Model Program
Illinois	2006	Voluntary Managed Care
Pennsylvania	2006	HealthChoices, Voluntary HMO Contracts
Nevada	2006	Mandatory Health Maintenance Program
Tennessee	2006	TennCare II
Colorado	2007	Managed Care Program
Oregon	2008	Oregon Health Plan Plus
Indiana	2008	Hoosier Healthwise (1915(b)), Hoosier Healthwise (1115)
Georgia	2009	Georgia Families

We studied the impact of P4P for Medicaid managed care programs on the immunization status of young children. Studying the effect of P4P programs on preventive health care measures for children has the advantage that children on Medicaid are most likely to be in managed care plans compared to other groups eligible for Medicaid. Studying the effect of Medicaid P4P plans on childhood immunization status has the advantage that although measures targeted by state plans vary, nearly every state targets childhood immunizations.⁸ Two previous studies have examined the effect of Medicaid P4P on childhood immunization rates, each examining effects in one particular Medicaid Health Maintenance Organization (HMO): one study found positive effects⁹ while the other did not.⁴ We analyzed a nationally-representative sample of Medicaid-eligible children which included information on their immunization status as well as socioeconomic characteristics. We took advantage of the staggered adoption of P4P programs by state, and examined whether any increase in the probability that Medicaid-eligible children were up-to-date on vaccines after compared to before their state adopted a P4P program was larger than for a control group comprised of children not eligible for Medicaid, as well as for Medicaid-eligible children in states that did not adopt P4P.

We also estimated the impact of Medicaid P4P programs using samples stratified by race/ethnicity, since some past studies that have found positive effects of P4P on performance have also suggested that the programs may have an unintended consequence of increasing race or ethnic differences in the receipt of preventive health care services. For example, P4P may induce individual physicians or medical groups to avoid treating patients with poorer health status, who may lower their quality score if the quality measure does not adjust for patients’ overall health status. Furthermore, public reporting elements of P4P programs might benefit patients with more education relative to patients with less education.¹⁰

Methods

Data

We used data from the National Immunization Survey (NIS), a survey that monitors childhood immunization coverage among children ages 19–35 months living in the U.S. The survey is a list-assisted random-digit-dialing land-line telephone survey (with a cell phone sample added in 2011) followed by a mail survey of children’s immunization providers.¹¹ These data provide information on children’s vaccination status, along with other child and family characteristics. We would like to measure the effect of Medicaid P4P programs on targeted beneficiaries, i.e., children on Medicaid. Unfortunately, Medicaid status is not reported in the NIS. As an alternative, we estimate the effect of P4P on immunization status of children eligible for Medicaid by using family income as reported in the NIS along with information on Medicaid eligibility rules by state and year.^12,13 Studying the effect of P4P on Medicaid-eligibles rather than those participating in Medicaid has the advantage of avoiding any endogeneity of Medicaid participation. We select NIS data beginning in 1999 since that is the first year the NIS has information on the ratio of family income to the federal poverty line. We use data through 2011 since that is the last year for which we have complete information on Medicaid P4P programs by state. Survey response rates for children sampled via land line and that had adequate provider data ranged from 64 to 72% depending on the year. We also control for unemployment rates by state and year using information from the Area Resource File. Our sample size for NIS 1999–2011 consists of 106,495 Medicaid-eligible and 135,138 non-Medicaid eligible children after dropping some records with missing data on variables used in the analysis.

Outcome Variables

For our childhood immunization outcomes, we use dichotomous indicators of whether a child is up-to-date on six particular vaccines: Diphtheria, Tetanus, and Pertussis (DTaP) (four doses); Polio (IPV) (three doses); Measles, Mumps, and Rubella (MMR) (one dose); Haemophilus Influenzae Type B (Hib) (three doses); Hepatitis B (Hep B) (three doses); and Varicella (Chickenpox) (one dose). We also consider whether children are up-to-date on the complete series (known as 4:3:1:3:3:1 (DTaP/IPV/MMR/Hib/Hep B/Varicella)), though also consider effects of P4P on individual components of the series since some children may be up-to-date on some parts of the series but not others.

Statistical Analyses

To estimate the effect of Medicaid P4P adopted by certain states on childhood immunization status, we use difference-in-difference (DD) and difference-in-difference-in-difference (DDD) models. In both specifications, our treatment group, those affected by a Medicaid P4P program, consists of children who are eligible for Medicaid and the control group consists of children who are not eligible for Medicaid. We first estimate a DD specification using a sample limited to the 106,495 Medicaid-eligible and ineligible children who reside in states that have ever implemented a P4P program during our study period. In this specification, we estimate the effect of the policy by comparing the pre- and post P4P change in immunization status occurring in the treatment group to the outcome change occurring in the control group. Specifically, we estimate equation (1) as follows:

$$Y_{ijt}=\alpha +{\beta }_{1}P4P_{jt}*{\mathit{\text{Medicaid}}}_{\mathit{\text{ijt}}}+{\beta }_2{\mathit{\text{Medicaid}}}_{\mathit{\text{ijt}}}+{\beta }_{3}P4P_{jt}+{\beta }_4{X}_{ijt}+{\beta }_5\mathit{\text{Survey}}{\mathit{\text{Year}}}_t+{\gamma }_j+{\epsilon }_{ijt},$$ (1)

In Equation (1), Y_ijt measures immunization status (whether a child is up-to-date on a particular vaccine) for child i in state j in year t; Medicaid_ijt is an indicator that the child is eligible for Medicaid; P4P_jt is an indicator that turns one when the state has a Medicaid P4P program in effect; and X_ijt represents child, family or state characteristics. These characteristics include child gender, age category (19–23 months (omitted group), 24–29 months, and 30–35 months), race/ethnicity (white non-Hispanic (omitted group), black non-Hispanic, other race non-Hispanic, and Hispanic), mother’s education category ( not a college graduate (omitted group), and college graduate), mother’s marital status, and state unemployment rate. We also include a set of state dummies (γ_j) to control for any omitted time-invariant state-level variables that might be correlated with both adoption of a P4P program and outcomes, and year dummies to control for national trends in outcomes. Additional models also control for state-specific linear time trends to control for state-specific shocks over our study period that may be correlated with the adoption of P4P.

Even controlling for state-specific linear time trends, the DD estimates assume that Medicaid-eligible and non-eligible children experience the same idiosyncratic shock. However, if there are any omitted variables that are correlated with P4P adoption and affect Medicaid-eligible children differently than ineligible children, these effects will be captured in the DD estimate as well. Therefore, we move to a triple difference approach by adding all other states that did not adopt a Medicaid P4P program during the study period and assume that any outcome difference between Medicaid-eligible and non-eligible children due to any group-specific shock in the P4P states is the same as that occurring in non-P4P states. This approach allows us to compare pre- and post-P4P differences in immunization status between Medicaid-eligible and non-eligible children in states that implemented Medicaid P4P, while netting out this same difference in states that did not implement P4P. Specifically, we estimate the following equation:

$$Y_{ijt}=\alpha +{\beta }_{1}P4P_{jt}*{\mathit{\text{Medicaid}}}_{\mathit{\text{ijt}}}+{\beta }_2{\mathit{\text{Medicaid}}}_{\mathit{\text{ijt}}}+{\beta }_{3}P4P_{jt}+{\beta }_4\mathit{\text{Survey}}{\mathit{\text{Year}}}_t+{\gamma }_j+{\beta }_5{\mathit{\text{Medicaid}}}_{\mathit{\text{ijt}}}*\mathit{\text{Survey}}{\mathit{\text{Year}}}_t+{\beta }_6{\mathit{\text{Medicai}}}_{\mathit{\text{ijt}}}*{\gamma }_j+{\beta }_7{X}_{ijt}+{\epsilon }_{ijt}$$ (2)

In addition to terms included in equation (1), equation (2) also includes interaction terms between the state fixed effects and the Medicaid eligibilty indicator and interactions between the Medicaid eligibility indicator and the year fixed effects. Although our outcomes are binary, we estimate linear probability models to make interpretation of estimated coefficients on interaction terms straightforward. We use Huber-White corrected standard errors adjusted for clustering at the state level.¹⁴ All models are also estimated using survey sampling weights.

The key independent variables in equations (1) and (2) are the interaction terms between Medicaid-eligibility and P4P. The estimated coefficient on this interaction term in the DD model represents the effect of P4P on the immunization status of Medicaid-eligible children by comparing outcomes before and after P4P relative to the “pre-post” change in immunization status in the same state for other children. In the DDD models, we further net out the analogous comparison in states that did not implement Medicaid P4P programs. Since the DD and DDD models implicitly impose different assumptions on the estimates, we report both sets of estimates to examine the robustness of our results. Since most states that implemented Medicaid P4P programs specifically targeted childhood immunizations, we expect to see that implementation of P4P programs are associated with improvements in the probability that Medicaid-eligible children are up-to-date on immunizations relative to other children.

The main threat to the validity of our analyses is the possibility that the implementation of P4P is correlated with unobserved factors that are also correlated with outcomes for Medicaid-eligible children specifically in P4P states. This might occur if, for example, P4P programs are implemented in response to trends in outcomes among Medicaid-eligible children that are not common with trends among ineligible children, or if states change Medicaid eligibility rules at the same time as P4P is adopted, thus changing the composition of Medicaid-eligible children. We test for this second possibility by estimating equations (1) and (2) with child, family, and state-year characteristics as dependent variables. We examine the first possibility by performing a falsification test as in Gruber (2000).¹⁵ Specifically, we use a sample of children in non-P4P states and in P4P states prior to P4P adoption and create a P4P indicator which is equal to one rather than zero only in the year prior to implementation of P4P. If we find no significant effect by estimating the DD specification as specified above, it suggests that without the introduction of P4P programs, P4P and non-P4P states would have experienced similar changes in the immunization status. It would also support the argument that the policy change caused the differential increase in the immunization status among Medicaid population in P4P states, not the other way around.

Results

Table 2 reports estimates of β₁ from estimation of equations (1) and (2). The inclusion of state-specific linear time trends does not change our results substantially under either DD (columns 2 and 3) or DDD (columns 4 and 5) estimation. Only one comparison – Varicella under DD - is somewhat different, suggesting that state-specific shocks do not significantly bias our estimates. We find no significant effect of P4P on the probability of a child being up-to-date on the entire immunization series (4:3:1:3:3:1) or on three of the six component vaccines (DTaP, IPV, and Hib). Estimates of the effect of P4P on the probability of being up-to-date on MMR are significantly positive under both DD and DDD specifications with or without state-specific time trends. Considering estimates including state-specific time trends, the implementation of Medicaid P4P programs increased the chance that Medicaid-eligible children were up-to-date on the MMR by about 1.4 (DD) – 1.9 (DDD) percentage points, or roughly 1.5% – 2% relatively to the mean immunization rates before implementation (column 1). Estimates on other outcomes suggest that P4P could increase the probability of being up-to-date on Hep B (under DDD) by about 1.5 percentage points (about 1.6%, respectively) and Varicella (under DD) by about 1.6 percentage points (or 2%).

Table 2:

The Effect of Medicaid Pay-for-Performance (P4P) on Childhood Immunization Status

		Difference-in-Difference (DD)		Difference-in-Difference- in-Difference (DDD)
	Mean for Medicaid-Eligible Children Before P4P Adoption	P4P* Medicaid	p4p* Medicaid	p4p* Medicaid	p4p* Medicaid
Column	(1)	(2)	(3)	(4)	(5)
Diphtheria, Tetanus, and Pertussis (DTaP)	0.8512	0.0184	0.0204	0.0258	0.0269
	(0.004)	(0.012)	(0.012)	(0.014)	(0.014)
Polio (IPV)	0.9276	0.0107	0.0119	0.0136	0.0146
	(0.003)	(0.008)	(0.008)	(0.009)	(0.009)
Measures, Mumps, and Rubella (MMR)	0.9525	0.0143*	0.0143*	0.0192**	0.0189**
	(0.002)	(0.005)	(0.005)	(0.006)	(0.006)
Haemophilus Influenza Type B (Hib)	0.9376	0.0018	0.0020	0.0006	0.0010
	(0.003)	(0.006)	(0.006)	(0.009)	(0.009)
Hepatitis B (Hep B)	0.9346	0.008	0.0096	0.0139*	0.0149*
	(0.003)	(0.005)	(0.005)	(0.006)	(0.006)
Varicella (Chickenpox)	0.8263	0.0095	0.0163*	−0.0013	0.0037
	(0.004)	(0.007)	(0.006)	(0.009)	(0.010)
4:3:1:3:3:1 series	0.7023	0.0133	0.0205	0.0039	0.0066
	(0.005)	(0.011)	(0.011)	(0.015)	(0.014)
State-specific time trends		No	Yes	No	Yes

Table 2 reports results from estimation of equation (1) and (2) in the text using linear probability models with and without controlling for state-specific linear time trends. Other control variables included but not listed consist of gender, age categories (19–23 months (omitted group), 24–29 months, 30–35 months), race/ethnicity (white non-Hispanic (omitted group), black non-Hispanic, other race non-Hispanic, Hispanic), mother’s education categories (not a college graduate (omitted group), college graduate), mother’s marital status and state unemployment rate. Standard errors are in parentheses. The sample size is 18,194 for column (1), 106,495 for column (2) and (3), and 241,633 for column (4) and (5).

The symbols * and **indicate statistical significance at the 5 and 1% levels, respectively.

Table 3 shows results from two sets of robustness checks. Panel A of Table 3 shows the results of estimating equations (1) and (2) with individual characteristics as the dependent variables. (State-specific linear time trends are included.) Each coefficient represents an estimate of β₁ from a separate regression for both the DD or DDD specifications. Only two characteristics show any changes with P4P adoption that are significant at the 5% level: percent of children between 30 and 35 months (relative to 19–23 months, DD only) and the percent of female children (DDD only). The estimated sizes of these compositional differences are small – about 1.8 percentages points (about 5%) for age 30–35 months and 2.4 percentage points (about 5%) for female. However, because of these findings, we ran specifications in equation (1) and (2) without controlling for individual and state characteristics and found results with similar magnitude and significance levels as shown in Table 2, indicating that sample selection bias is likely not driving those results. That, is, to the extent that inclusion or exclusion of observed characteristics does not appear to alter findings, the estimated effects of P4P in Table 2 are not due to changes in the composition of children who live in P4P states. Second, we tested for the possibility of correlation between P4P implementation and pre-existing trends in outcomes for Medicaid-eligible children by estimating equations (1) and (2) using data prior to P4P in the adopting states for both DD and DDD specifications and all data in the non-adopting states for DDD. The P4P indicator is equal to one in the year prior to implementation. These results suggest that the estimated effects of P4P are not driven by pre-existing trends in outcomes for Medicaid-eligible children.

Table 3:

Robustness Checks for Sample Selection and Pre-Existing Trends

Panel A
Column	(1)	(2)	(3)	(4)	(5)	(6)	(9)	(10)	(11)
	Age 24–29 months	Age 30–35 months	Hispanic	Non-Hispanic Black	Non-Hispanic Other Race	Female	Mother is a College Graduate	Mother Married	State Unemployment Rate
Mean for Medicaid-Eligible	0.3456	0.3468	0.3791	0.2159	0.0995	0.5045	0.1080	0.5615	0.0525
Children Before P4P	(0.005)	(0.005)	(0.006)	(0.005)	(0.004)	(0.006)	(0.003)	(0.006)	(0.000)
DD
P4P * Medicaid	−0.0029	0.0179*	−0.0000	−0.0263	0.0096	0.0152	−0.0315	−0.0006	−0.0003
	(0.007)	(0.008)	(0.030)	(0.016)	(0.013)	(0.010)	(0.041)	(0.038)	(0.000)
DDD
P4P * Medicaid	−0.0100	0.0165	0.0166	−0.0163	−0.0110	0.0244*	−0.0298	0.0070	0.0005
	(0.008)	(0.009)	(0.029)	(0.013)	(0.013)	(0.009)	(0.044)	(0.038)	(0.000)
Panel B
	DTaP	IPV	MMR	Hib	Hep B	Chickenpox	4:3:1:3:3:1 series
DD
P4P * Medicaid	−0.0182	−0.0044	0.0029	0.0038	−0.0144	0.0272	0.0278
	(0.017)	(0.008)	(0.007)	(0.008)	(0.007)	(0.015)	(0.017)
DDD
P4P * Medicaid	−0.0239	−0.0034	0.0023	−0.0006	−0.0119	0.0025	0.0016
	(0.015)	(0.009)	(0.009)	(0.008)	(0.006)	(0.014)	(0.015)

Table 3 reports results from estimation of equation (1) and (2) in the text using linear probability models controlling for state-specific linear time trends. Other control variables included but not listed consist of gender, age categories (19–23 months (omitted group), 24–29 months, 30–35 months), race/ethnicity (white non-Hispanic (omitted group), black non-Hispanic, other race non-Hispanic, Hispanic), mother’s education categories (not a college graduate (omitted group), college graduate), mother’s marital status and state unemployment rate. Standard errors are in parentheses. The sample size for Panel A is 106,495 for DD regressions and 241,633 for DDD regressions. The sample size for Panel B is 46,691 for DD regressions and 181,829 for DDD regressions. See Table 2 for abbreviations for specific immunizations. DD stands for difference-in-difference estimation. DDD stands for difference-in-difference-in-difference estimation.

The symbol * indicates statistical significance at the 5% level.

Finally, Table 4 estimates equation (2) stratified by race/ethnicity. DD results (not reported) are similar. None of the interactions between P4P and Medicaid eligibility are statistically significant for non-Hispanic black children. For Hispanic children and for non-Hispanic white children, results are statistically significant for two of the six component vaccines. Interactions between P4P and Medicaid indicate that P4P increases the chance that a Hispanic or non-Hispanic white children child is up-to-date on MMR by about 2.7 – 2.8 percentage points, a nearly 3 percent effect for both groups. In addition, Table 4 shows statistically significant effects for Hispanic children for DTap and for non-Hispanic white children for Hep B. We cannot rule out that the lack of statistically significant results for non-Hispanic black children could be a result of reduced power for this group, though point estimates for some vaccines do not appear similar to those for non-Hispanic white and Hispanic children.

Table 4:

The Effect of Medicaid Pay-for-Performance (P4P) on Childhood Immunization Status by Race/Ethnicity

	Non-Hispanic White		Non-Hispanic Black		Non-Hispanic Other		Hispanic
	Meana	Medicaid* P4P	Meana	Medicaid* P4P	Meana	Medicaid* P4P	Meana	Medicaid* P4P
DTaP	0.8550	0.0375	0.8190	0.0281	0.8419	0.0165	0.8642	0.0341**
	(0.006)	(0.022)	(0.011)	(0.029)	(0.020)	(0.046)	(0.007)	(0.011)
IPV	0.9244	0.0261	0.9083	0.0175	0.9220	0.0065	0.9409	0.0079
	(0.004)	(0.016)	(0.008)	(0.020)	(0.017)	(0.019)	(0.005)	(0.009)
MMR	0.9420	0.0278*	0.9523	−0.0038	0.9461	0.0025	0.9640	0.0274**
	(0.004)	(0.013)	(0.005)	(0.019)	(0.014)	(0.020)	(0.004)	(0.007)
Hib	0.9404	0.0102	0.9257	−0.0113	0.9204	−0.0185	0.9434	−0.0070
	(0.004)	(0.013)	(0.007)	(0.022)	(0.016)	(0.030)	(0.005)	(0.008)
Hep B	0.9330	0.0360**	0.9301	−0.0185	0.9338	−0.0238	0.9383	0.0089
	(0.004)	(0.010)	(0.007)	(0.020)	(0.014)	(0.020)	(0.005)	(0.008)
Chickenpox	0.7665	0.0064	0.8365	0.0138	0.8456	0.0404	0.8779	−0.0098
	(0.007)	(0.017)	(0.009)	(0.030)	(0.018)	(0.028)	(0.006)	(0.014)
4:3:1:3:3:1 series	0.6600	0.0173	0.6838	−0.0030	0.7305	0.0117	0.7482	0.0147
	(0.008)	(0.021)	(0.012)	(0.039)	(0.022)	(0.061)	(0.008)	(0.014)

Table 4 presents estimation of equation (2) in the text stratified by race/ethnicity. Linear probability models are estimated contolling forgender, age categories (19–23 months (omitted group), 24–29 months, 30–35 months), mother’s education categories (not a college graduate (omitted group), college graduate), mother’s marital status and state unemployment rate, in addition to state and year effects for children eligible and not eligible for Medicaid. Standard errors are in parentheses. The sample size is 144,265 for non-Hispanic white children, 29,799 for non-Hispanic black children, 20,023 for non-Hispanic children of other race, and 47,546 for Hispanic children. See Table 2 for abbreviations for specific immunizations.

The symbols * and **indicate statistical significance at the 5 and 1 levels, respectively.

^aMean indicates mean for Medicaid-eligible children before P4P adoption. The sample size for those means is 38,273, 14,478, 7,097 and 22,431 for non-Hispanic white, non-Hispanic black, non-Hispanic other race, and Hispanic, respectively.

Discussion

This is the first nationally-representative study examining the impact of Medicaid P4P programs on the use of preventive care services. P4P for Medicaid managed care programs are intended to align reimbursement with the performance of managed care plans in terms of the appropriate provision of preventive care services. We find positive effects of P4P on the chance that Hispanic and non-Hispanic white children eligible for Medicaid are up-to-date on some vaccines. However, we do not find statistically significant effects on all vaccines, and no statistically significant effects for non-Hispanic black children. Unfortunately, we were not able to study the effect of different incentive schemes on the effectiveness of P4P programs. Although the Centers for Medicare and Medicaid Services notes the type of incentive scheme each state uses when a P4P program is initiated, it is not clear that accurate information is available for all states if states change the nature of their programs after a P4P program is first implemented. Our study, therefore, is of the average effect of programs that vary widely in design, in likely size of the rewards or penalties, and in fact whether the programs were mandatory or voluntary. It is possible that our mixed findings on the effect of Medicaid P4P on immunization status of children is due to our inability to distinguish program designs that are effective from ones that are less effective.

Preventive care plays an important role in the health care delivery system, especially for the low-income population. For example, Hillman et al. (1999, p. 931) pointed out that “childhood immunization is a cost-effective means of preventing disease… and also reflects the adequacy of pediatric health care in general”.⁴ Despite the benefits of preventive care, the low-income population, Medicaid’s primary target group, tends to have a lower utilization rates for those services. The fact that P4P programs may help increase the use of childhood immunizations provides a possible means of improving the health status of this population.

Studying the effect of Medicaid P4P on performance may take on additional importance under the Affordable Care Act (ACA) which is expected to increase the federal portion of Medicaid spending as a share of gross domestic product from 1.8 percent in 2014 to 2.1 percent in 2024.¹⁶ In addition, several ACA provisions are aimed at increasing both the quality and the affordability of health care. For example, the law requires each state that contracts with a Managed Care Organization (MCO) to have a strategy for the assessment of performance of Medicaid managed care services.¹⁷ The Centers for Medicare and Medicaid Services has developed a voluntary set of core performance measures for Medicaid and the Children’s Health Insurance Program, many of which are part of the HEDIS data set.¹⁸ This study and further analysis of state Medicaid P4P programs are opportunities to contribute toward an understanding of the effectiveness of existing efforts in improving the performance of Medicaid managed care plans.

Our study is limited in a number of ways. First, P4P programs that we studied were intended for beneficiaries covered by Medicaid managed care plans. However, we were not able to identify whether the child was covered by a managed care plan or not due to the lack of this information in the NIS survey data. Second, there are aspects of P4P programs that we were not able to study that may have an impact on the effectiveness of the program, such as whether performance is measured by a certain level of attainment of a goal or by improvement in performance relative to the past, as well as the specific type of incentive offered by the P4P program (for example, monetary bonuses, auto-assignment of unassigned beneficiaries to highly rated plans, or penalties). Some analysts conclude that it is important for P4P programs to incorporate both risk adjustment and stratification of standards, and to reward both the absolute quality score as well as improvement over time.¹⁹ Finally, we only examined the impact of the policy on childhood immunizations. There are many other performance outcomes targeted by many states that we didn’t study in this paper, such as breast, cervical, and colorectal cancer screening rates, and measures of the effectiveness of the management of blood pressure and cholesterol levels. Study of the effect of Medicaid P4P programs on these outcomes could provide further insight into the effective design of managed care programs for the Medicaid population in the future.