Early impact of the implementation of Medicaid episode‐based payment reforms in Arkansas

Abstract

Objective

To evaluate episode‐based payments for upper respiratory tract infections (URI) and perinatal care in Arkansas's Medicaid population.

Study Setting

Upper respiratory infection and perinatal episodes among Medicaid‐covered individuals in Arkansas and comparison states from fiscal year (FY) 2011 to 2014.

Study Design

Cross‐sectional observational analysis using a difference‐in‐difference design to examine outcomes associated with URI and perinatal episodes of care (EOC) from 2011 to 2014. Key dependent variables include antibiotic use, emergency department visits, physician visits, hospitalizations, readmission, and preventive screenings.

Data Collection

Claims data from the Medicaid Analytic Extract for Arkansas, Mississippi, and Missouri from 2010 to 2014 with supplemental county‐level data from the Area Health Resource File (AHRF).

Principal Findings

The URI EOC reduced the probability of antibiotic use (marginal effect [ME] = −1.8, 90% CI: −2.2, −1.4), physician visits (ME = 0.6, 90% CI: −0.8, −0.4), improved the probability of strep tests for children diagnosed with pharyngitis (ME = 9.4, 90% CI: 8.5, 10.3), but also increased the probability of an emergency department (ED) visit (ME = 0.1, 90% CI: 0.1, 0.2), relative to the comparison group. For perinatal EOCs, we found a reduced probability of an ED visit during pregnancy (ME = 0.1, 90% CI: −0.2, −0.0), an increased probability of screening for HIV (ME = 6.2, 90% CI: 4.0, 8.5), chlamydia (ME = 9.5, 90% CI: 7.2, 11.8), and group B strep‐test (ME = 2.6, 90% CI: 0.5, 4.6), relative to the comparison group. Predelivery and postpartum hospitalizations also increased (ME = 1.2, 90% CI: 0.4, 2.0; ME = 0.4, 90% CI: 0.0, 0.8, respectively), relative to the comparison group.

Conclusion

Upper respiratory infection and perinatal EOCs for Arkansas Medicaid beneficiaries produced mixed results. Aligning shared savings with quality metrics and cost‐thresholds may help achieve quality targets and disincentivize over utilization within the EOC, but may also have unintended consequences.

What This Study Adds.

• Episode‐based bundled payments may be a promising alternative to a fee‐for‐service payment model. However, recent evaluations of episode‐based payment in the Medicare and commercial populations have produced mixed results. To our knowledge, no study has evaluated episode‐based bundled payment in the Medicaid population.

• This study evaluates the early impact of Arkansas's Episode of Care (EOC) model for Upper Respiratory Infection (URI) and perinatal EOCs among Arkansas's Medicaid population. The mandatory nature of the EOC implementation presents a unique opportunity to evaluate the effectiveness of EOCs, avoiding potential selection bias associated with volunteer provider participation examined in previous studies.

• We found that tying shared savings to quality metrics may be helpful in achieving quality targets within the context of episode‐based payments in URI and perinatal EOCs, and can also be successful in disincentivizing over utilization of costly services within those EOCs.

• However, the implementation of EOCs may produce unintended consequences, such as potentially shifting care to outside the episode (perinatal), changing diagnostic coding practices (URI), or incentivizing patients to seek care in the emergency department (URI).

1. INTRODUCTION

The Centers for Medicare and Medicaid Services (CMS) have focused on efforts to move away from fee‐for‐service (FFS)—based payment toward value‐based purchasing to improve quality of care while controlling costs. One promising alternative payment model includes episode‐based bundled payments, as demonstrated by Medicare in the postacute care setting. ¹ , ² , ³ Episode‐based bundled payments aggregate all services related to a specific condition or procedure into a single payment to encourage quality improvement and cost control.

In 2011, Arkansas implemented episodes of care (EOC) for specific acute and chronic conditions as part of its Arkansas Health Care Payment Improvement Initiative (AHCPII), to control costs, and improve access to and quality of care. ⁴ , ⁵ , ⁶ As a result of these initial payment reform efforts, Arkansas was awarded a State Innovation Models Round 1 (SIM1) cooperative agreement from CMS for $42 million in early 2013 to support continued reform efforts. Arkansas used the SIM1 funds to refine, expand, and develop additional EOCs, as well as disseminate information regarding new payment models.

Participation in the EOC model was mandatory for Medicaid providers. The first two Arkansas Medicaid EOCs were implemented in October 2012, which included upper respiratory tract infections (URIs) and perinatal care. Indeed, URIs are the most common acute outpatient diagnosis nationally. ⁶ URIs—often simply the common cold—are typically viral, and antibiotic use is inappropriate unless it is clinically indicated. However, antibiotics are commonly dispensed for Medicaid beneficiaries, both nationally and in Arkansas specifically. ⁷ , ⁸ , ⁹ The goal of Arkansas's URI EOC is to promote more clinically appropriate use of antibiotics while controlling episode costs.

Similarly, controlling costs and improving perinatal quality of care were the primary goal of Arkansas's perinatal EOC. Nationally, payers expend a significant amount on perinatal care, and state‐Medicaid programs are responsible for 23 to 69 percent of all covered births by state. ¹ Additionally, geographic variability in hospital costs for low‐risk child births suggests episode‐based payments can generate overall cost savings. ¹⁰ , ¹¹ Episode‐based payments for perinatal care may help to reduce spending through the reduction in unnecessary cesarean sections and prevention of complications leading to emergency department visits or hospitalizations. ¹² , ¹³

The current evidence on episode‐based bundled payments is mixed. A recent evaluation of the comprehensive joint replacement (CJR) model suggests episode‐based payment can contribute to reductions in institutional spending, though there was no impact on postdischarge utilization. ¹ Similar results were found by Navathe et al ¹⁴ in a study of CJR patients at a large hospital system. An evaluation of the bundled payment for care improvement (BPCI) model also found reductions in Medicare spending per 90 days postdischarge episode, though the authors are careful to note that participation in the model was voluntary, precluding causal inference. ¹⁵ Maddox and others evaluated the association of participating in BPCI on episode costs, utilization, and 90‐day mortality for five medical conditions. ¹⁶ Their study found no association of bundled payments on average episode costs, use, or mortality. Although a brief baseline and follow‐up period was a key limitation to this study, in addition to voluntary participation. Finally, Carroll et al examined Arkansas's implementation of the perinatal EOC on episode spending and utilization among the commercially insured population. ¹² Their study found reductions in inpatient spending associated with the delivery stay, and limited evidence of improvements in quality of care. ¹² These findings are likely not generalizable to Arkansas's Medicaid population due to differences in income and health characteristics between the commercially insured and Medicaid populations.

Arkansas's EOC model is designed to reduce unnecessary utilization and improve quality of care. Specifically, it holds the principal accountable providers (PAPs) responsible for the total cost and quality of care for the entire duration of the episode. To determine risk or gain share, the state calculates the PAP’s average cost per EOC over the prior 1‐year period and assesses whether the costs are “acceptable” or “commendable.” ⁷ PAP payment is lost or gained based on whether their average EOC costs are deemed above, at, or below these cost thresholds. Moreover, PAPs are required to meet certain quality metrics on preventive screening measures and antibiotic use in order to be eligible for shared savings. As such, the Arkansas EOCs should be associated with improvements among the tracked quality measures, and reductions in unnecessary utilization such ED visits (URI and perinatal), antibiotic use (URI), hospitalizations, and cesarean sections. ¹⁷

The purpose of this study is to evaluate the early impact of Arkansas's EOC payment reform model on URI and perinatal EOCs among Arkansas's Medicaid population. The mandatory nature of the implementation presents a unique opportunity to evaluate the effectiveness of EOCs, avoiding potential selection bias associated with volunteer provider participation examined in previous studies. ¹⁴ To our knowledge, no studies have evaluated the effect of episode‐based payment reforms on utilization and quality of care within Medicaid. This paper reports outcomes for the EOC payment models among Medicaid beneficiaries in Arkansas for the first two years of implementation.

2. METHODS

2.1. Study design

We conducted a cross‐sectional observational study using a difference‐in‐difference design with propensity score weighting on retrospective annual cross sections of URI and perinatal EOCs from fiscal years (FY) 2011 to 2014.

2.2. Data sources

We used claims data from the Medicaid Analytic Extract (MAX) from CY 2010 to 2014 (the last year of available MAX data for Arkansas) with supplemental county‐level data from the 2016‐2017 Area Health Resource File (AHRF) to obtain information from 2010 to 2014.

2.3. Identification of comparison states

The two comparison groups consist of Medicaid beneficiaries from Mississippi and Missouri. Our comparison group for URI EOCs consisted of Medicaid beneficiaries who were diagnosed with a URI in an office, outpatient clinic, or in emergency department in Mississippi or Missouri. Our comparison group for perinatal EOCs consisted of Medicaid covered deliveries with a live birth that occurred in an inpatient setting in Mississippi and Missouri. We selected Missouri and Mississippi as comparison states because of MAX data availability through the study period, calculated Euclidean distance scores, ¹⁸ complete encounter data, ¹⁹ and similar income thresholds for Medicaid eligibility. ²⁰

Mississippi and Missouri Medicaid beneficiaries were primarily enrolled in Medicaid Managed Care (MMC) plans, while Arkansas beneficiaries were in Medicaid fee‐for‐service during the study period. Therefore, expenditure data from the comparison states were not available. Consistent with other work, ¹⁹ preliminary analyses indicate that for each EOC, Missouri and Mississippi had similar inpatient, professional, and drug utilization for each year, 2011 to 2014, indicating that encounter data for MMC plans enrollees are reliable.

Arkansas's adoption of a private option Medicaid expansion for adults with incomes up to 138 percent of the FPL in 2014 may impact the sample characteristics of the perinatal EOCs. ²¹ Indeed, compared to the 2011‐2013 period, women remaining in traditional Medicaid may have less income on average (because the higher income women are now enrolled in the Private Option) and, consequently, may be in poorer health. ²² As such, our findings for 2014 may be conservative and biased toward the null hypothesis.

2.4. Episode definitions

Our episode definitions, inclusion and exclusion criteria, were based on those available from Arkansas's Health Care Payment Improvement Initiative ²³ website with clarifications from Arkansas Dept of Medicaid Services as needed.

2.4.1. URI episodes

The URI EOC requires beneficiaries to be diagnosed with a URI in an office, outpatient clinic, or emergency department (ED). We used Other Services (OT) claims and identified all claims where there was a primary diagnosis of 034.0x, 460.xx, 461.0x‐461.3x, 461.8x, 461.9x, 462.xx, 463.xx, 464.0x, 464.00, 464.10, 464.20, 465.0x, 465.8x, or 465.9x and a corresponding procedure code equal to 99 201‐99 205, 99 211‐99 215, 99 241‐99 245, 99 281‐99 285, or T1015, T1015 U1‐U3, or G0463 and the place of service code equaled 11, 20, 22, 23, 49, 50, 71, or 72. The URI episode includes all outpatient and prescription claims occurring during the 21‐day EOC duration. An additional URI diagnosis that falls within the 21‐day window does not trigger a new episode. Arkansas defined three types of URI episodes: Nonspecific URI, sinusitis, and pharyngitis, we combine these subtypes in our analysis.

2.4.2. Applying URI exclusion criteria

We attempted to create URI episodes consistent with the exclusion and inclusion criterion rules applied in Arkansas. We identified 3 271 939 total episodes in the study population between 2011 and 2014. Episodes in which the beneficiary had restricted benefits (N = 121 166), was not continuously enrolled in full Medicaid coverage (N = 574 195) for the 21‐day episode period or was younger than 1 year (N = 367 507) were excluded. We excluded episodes in which a comorbid diagnosis occurred (eg, croup, epiglottitis, URI with obstruction, pneumonia, influenza, and otitis media) and other conditions that may lead to complications (N = 762 878). ¹⁸ We excluded beneficiaries with tonsillectomy or adenoidectomy (n = 7469) and those with any inpatient stays or observation stays (N = 220 731) during the episode. Our final sample included 1 681 962 total URI episodes among 802 357 unique Medicaid beneficiaries.

2.4.3. Perinatal episodes

Perinatal episodes were triggered by a procedure code indicating a singleton, live birth delivery; the episode begins 280 days before the delivery date and extends 60 days postpartum. A claim indicating either vaginal or cesarean deliveries identified a potential episode. The procedure codes for vaginal delivery included 59 400, 59 409, 59 410, 59 610, 59 612, and 59 614 (CPT) or 72, 720, 721, 722, 723, 7221, 7229, 7231, 7239, 724, 725, 7251‐7254, 726, 727, 7271, 7279, 728, 729, 7322, 735, 7351, or 7359 (ICD‐9). Cesarean section codes include 59 510, 59 514, 59 515, 59 618, 59 620, and 59 622 (CPT) or 74, 740, 741, 742, 744, or 7499 (ICD‐9). We create a single record for each unique Medicaid beneficiary and delivery date. If there was only one record for a unique beneficiary, we defined the delivery date as the service end date or principal procedure date on the inpatient claim. The admission date was used if the principal procedure date was missing. If there were multiple records for a unique beneficiary with the same procedure service date, then we used the service begin date as the delivery date. If there were multiple records for a unique beneficiary and the date the service began was less than 6 months from the previous service end date, then we applied a selection algorithm that is detailed in Appendix S1‐1.1.

2.4.4. Applying perinatal exclusion criteria

We identified 272 879 Medicaid‐covered live birth deliveries in an inpatient setting from fiscal year 2011 through 2014. We excluded beneficiaries with different types of coverage during the episode, including Children's Health Insurance Program, supplemental private insurance, or dual Medicare‐Medicaid enrollment (N = 39 289). To more closely resemble the Arkansas episode criteria, EOCs with select pregnancy‐related conditions (eg, obstetric blood clot embolism, placenta previa, amniotic fluid embolism, and severe preeclampsia) that may cause birth complications and other comorbidities were excluded (n = 36 838). To ensure maximum exposure to adequate prenatal services, we excluded beneficiaries with limited enrollment (n = 41 707), overlapping episodes (n = 2913), no claim for any prenatal care (n = 18 325), without full benefits during the delivery month (n = 3183), and for at least 6 months prior to delivery (n = 83 560). We also excluded episodes among adolescents 15 years and younger (n = 2436) because pregnant children may have social/economic factors that are unique and unobserved, relative to the rest of the population. ²⁴ Our final sample size consisted of 148 872 episodes. Postpartum outcome measures were further restricted to episodes with full Medicaid eligibility up to 60 days postdelivery (n = 7227). We conducted a sensitivity analysis that included all deliveries identified prior to the application of the exclusion criteria (Appendix S2). Results from this sensitivity analysis are largely consistent with our findings discussed below.

2.5. Key outcomes and covariates

2.5.1. Outcomes for URI

URI EOC utilization outcomes include having any antibiotic use, any URI‐specific physician visits, and any URI‐specific ED visits during the episode. We also modeled the appropriate treatment for children with pharyngitis: an indicator for children 3‐18 years of age who were diagnosed with pharyngitis, ordered an antibiotic, and received a group A streptococcus (strep) test for the episode. ²⁵ We measured appropriate treatment for children with URI (National Quality Forum (NQF) #0069), an indicator for children between the ages of 1 and 18 years with a URI diagnosis who were not prescribed an antibiotic within the first 3 days of the URI diagnosis.

2.5.2. Outcomes for perinatal

We examined a count of the number of ED visits during pregnancy, the probability of having any hospitalizations during pregnancy, and the probability of having any hospitalizations within 30 and 60 days postdischarge from the delivery inpatient stay. We also modeled the probability of whether there were any preventive screenings for HIV, group B streptococcus, and chlamydia during the pregnancy, and for whether a cesarean section was performed.

2.5.3. Covariates for the URI EOC

We controlled for demographic and health characteristics including gender, age (continuous and age‐squared), race (categorical), disability status, clinical setting at diagnosis, and the Chronic Illness and Disability Payment System (CDPS) score.

2.5.4. Covariates for the perinatal EOC

We controlled for demographic characteristics (categorical age, race, disability status, poverty‐related eligibility during the month of delivery, and number of months of full Medicaid eligibility) during the episode. Health status characteristics included CDPS score, and having any diagnosis for diabetes, asthma, or hypertension during the previous year.

For both URI and the perinatal EOC analyses, we controlled for county‐level characteristics such as metropolitan status of county of residence, percent of population at the federal poverty level, hospital beds per 1000 residents, median age, and percent uninsured under 65.

2.6. Statistical analysis

Propensity score weighting was used to balance observable characteristics between Arkansas EOCs and comparison group EOCs. We estimated annual propensity score models using logistic regression controlling for individual and health status characteristics, described above, including baseline indicators for any inpatient admission or ED use. The propensity weight for a comparison individual was a function of his or her predicted propensity score, where weight = p/(1‐p), and p is the predicted propensity. Limited variation in county‐level characteristics made balancing on these variables difficult. Therefore, to optimize the balance and avoid extreme weights, county‐level covariates were excluded from the propensity score model. Annual covariate balance and density plots of the propensity scores are available in Appendix S1‐1.2. In all years, we found the comparison group passed the common support assumption for almost the entire range of the intervention group's propensity scores.

For both the URI and the perinatal EOCs, we modeled binary outcomes with logistic regression models. We modeled the count of the number of ED visits in perinatal EOCs with a negative binomial regression model. Standard errors were clustered at the individual level to account for correlation within beneficiaries with multiple episodes.

2.6.1. Difference‐in‐difference approach

We used a difference‐in‐difference approach to compare differences between the intervention and comparison groups during the baseline (FY 2011 and FY 2012) and postperiods (FY 2013 and FY 2014). We assessed the parallel trend assumption by plotting trend graphs for all outcomes and by modeling core utilization outcomes during the baseline period with a linear time trend interacted with a dichotomous variable, indicating that the EOC occurred in Arkansas. These tests indicated the URI EOC met the parallel trend assumption (available in Appendix S1‐1.3a).

In contrast, the perinatal EOCs in general did not meet the parallel trend assumption as there were statistical significance differences in the baseline trends for most outcomes. As such, we used an alternative difference‐in‐difference model that allowed us to generate effect estimates that net out the potential baseline differences between Arkansas and the comparison group. ²⁶ The alternative D‐in‐D model is shown in Equation 1.

$$Y_{\mathit{ijt}}={\alpha }_0+{\beta }_1{I}_{\mathit{ij}}+I_{\mathit{ij}}\times t+\sum {\beta }_2{Q}_n+\sum {\alpha }_2{Q}_t+\sum \gamma I_{\mathit{ijt}}\times Q_t+\lambda X_{\mathit{ijt}}+{\epsilon }_{\mathit{ijt}}$$ (1)

This model is an annual fixed‐effects model, where Y_ijt is the outcome for individual i (test or comparison group) in state j in year t; I_ij (=0,1) is an indicator equal to 1 if the individual is in the test group and 0 if the individual is in its comparison group; Q_n is a series of yearly dummies for the baseline period (years 1 to 2); and Q_t is a series of yearly dummies for the postyears (years 3 to 4). The term that interacts the Arkansas indicator and time (I_ij*t) measures differences in trends between Arkansas and the comparison group over the entire period. The interaction of the test group indicator and Q_t (I_ij × Q_t) measures the difference in the pre‐ and postchange between the test group and its comparison group. With this model specification, the postyear*Arkansas interactions measure any deviation from the trend line in the postperiod.

3. RESULTS

3.1. URI Episode

Descriptive results. There were 804 559 total weighted URI episodes from FY 2011 to FY 2014. Table 1 shows the unweighted and weighted characteristics of the URI episodes in Arkansas and the comparison group during the last year of the baseline period (2012) (all years available in Tables A‐1 to A‐5 in Appendix S1). Once propensity weights were applied, the differences between Arkansas EOCs and the comparison group were minimal across sociodemographic and health status characteristics. Differences across characteristics of county of residence were expectedly larger; they are, however, controlled for in the outcome models.

TABLE 1.

Weighted means and standardized differences prior to Arkansas URI EOC implementation, Arkansas, and comparison group, 2012 ^a

	Unweighted			Weighted
	Arkansas URI EOC group	Comparison group	Standardized difference b	Arkansas URI EOC group	Comparison group	Standardized difference b
N	103 815	323 102		103 815	103 126
Individual‐level sociodemographic characteristics
Age	8.9	11.3	22.4	8.9	9.0	0.4
Male	45.6	42.0	7.4	45.6	45.8	0.3
Black	13.7	32.9	46.6	13.7	13.8	0.5
Hispanic	35.6	4.2	85.6	35.6	35.1	1.1
White	41.4	57.6	32.9	41.4	41.8	0.8
Other	9.3	5.4	15.2	9.3	9.3	0.1
Health status characteristics
CDPS score	1.0	0.9	6.3	1.0	1.2	5.1
Medicaid eligibility: Disability	11.8	9.3	7.9	11.8	12.1	1.1
ED as triggering location	8.8	16.1	22.2	8.8	9.6	2.6
ED visit, 2011	5.7	7.5	7.6	5.7	5.8	0.6
Inpatient admission, 2012	0.4	0.6	3.7	0.4	0.4	0.1
County‐level characteristics
Metropolitan status of county of residence	56.2	42.5	27.7	56.2	45.6	21.3
Percent of population at FPL, 2012	20.1	22.2	33.6	20.1	21.3	19.7
Hospital beds per 1000, 2010	3.7	4.3	19.9	3.7	4.2	16.2
Median age, 2010	37.9	37.4	12.2	37.9	37.4	12.8
Percent uninsured among under 65 years old, 2012	19.6	18.7	32.3	19.6	18.6	36.6

Abbreviations: CDPS, Chronic Illness and Disability Payment System; EOC, Episode of Care; URI, upper respiratory infection.

^aThe number of episodes reported here is weighted and for 2012. The total number of unweighted URI episodes in our analysis was 1 681 962. See Appendix S1, Tables A‐2 to A‐5 for annual weighted and unweighted counts of URI episodes

^bAbsolute standardized differences (SDs) are expressed as percentages. <10% SD is ideal for inferring balance between groups. To balance the population characteristics for the claims‐based analyses, we estimated propensity scores for all individuals from the comparison group for each year of the analysis. After propensity score weighting, the standardized differences between the weighted comparison group means and intervention group means were all well under the standard 10% threshold for individual‐level variables; however, a few county‐level variables exceed the threshold. County‐level variables are shown here to provide context and were not considered in the propensity score models. Because there was little variation in county‐level characteristics, balancing on these variables is difficult. Therefore, to optimize the balance and avoid extreme weights, county‐level covariates were excluded from the propensity score model.

3.1.1. Impact analysis

Table 2 displays the regression‐adjusted averages for URI outcomes in both the treatment and comparison groups during the baseline and postperiods for each year during postperiod. We report annual regression‐adjusted D‐in‐D estimates individually for the first 2 years after the implementation of the EOC model along with an overall D‐in‐D estimate with the relative difference. For example, the percent of URI EOCs in Arkansas with a URI‐related physician visit declined from 6 percent in the baseline period to 5.1 percent in postperiod, resulting in a 0.6 percentage point greater decline in the likelihood of a URI‐related physician visit for Arkansas relative to the comparison group after the URI EOC implementation (P < .001). In contrast, the overall likelihood of a URI‐related ED visit increased by 0.1 percentage points for Arkansas URI episodes relative to the comparison group (P < .001), a relative difference of 16.5 percent. Antibiotic use for EOCs in Arkansas declined from a baseline rate of 63.8 percent to 58.4 percent during the postperiod, corresponding to a 1.8 percentage point greater decrease in the likelihood of antibiotic use relative to the comparison group (P < .001). Additionally, Arkansas had 3.9 percentage point greater increase in the receipt of appropriate treatment, relative to the comparison group (P < .001). Likewise, Arkansas had a 9.4 percentage point greater increase in the overall percentage of pharyngitis episodes that included a strep test, relative to the comparison group (P < .001).

TABLE 2.

Difference in the pre‐ and postannual change in outcomes for URI Episodes of Care in Arkansas, relative the comparison group, first 2 years of implementation (October 2012 through September 2014), Weighted N = 804 559

	Preperiod‐adjusted mean, AR	Preperiod‐adjusted mean, CG	Test‐period–adjusted mean, AR	Test‐period–adjusted mean, CG	Regression‐adjusted difference‐in‐differences (90% confidence interval)	Relative difference (%)	P‐value
Any URI‐related physician visit for all three types of URI episodes (%)
Year 1	6.0	6.5	5.1	6.2	−0.6 (−0.9, −0.4)	−10.5	<.001
Year 2	6.0	6.5	4.8	5.9	−0.6 (−0.8, −0.3)	−9.7	<.001
Overall	6.0	6.5	5.0	6.0	−0.6 (−0.8, −0.4)	−10.1	<.001
Any URI‐related ED visit for all three types of URI episodes (%)
Year 1	0.9	1.2	0.9	1.1	0.1 (0.1, 0.2)	16.1	.01
Year 2	0.9	1.2	0.9	1.1	0.1 (0.1, 0.2)	16.8	.004
Overall	0.9	1.2	0.9	1.1	0.1 (0.1, 0.2)	16.5	<.001
Antibiotic use for all episodes (%)
Year 1	63.8	65.2	59.1	62.5	−1.9 (−2.40, −1.31)	−2.9	<.000
Year 2	63.8	65.2	57.7	60.9	−1.7 (−2.22, −1.10)	−2.6	<.000
Overall	63.8	65.2	58.4	61.7	−1.8 (−2.15, −1.37)	−2.8	<.000
Appropriate URI treatment for children ages 1 to 18 years (%)
Year 1	47.2	43.7	53.2	45.5	4.1 (3.21, 5.05)	8.7	<.001
Year 2	47.2	43.7	54.3	47.2	3.6 (2.64, 4.56)	7.6	<.001
Overall	47.2	43.7	53.7	46.3	3.9 (3.22, 4.55)	8.2	<.001
Strep tests for children diagnosed with pharyngitis (%)
Year 1	53.7	53.1	62.0	53.9	7.7 (6.37, 8.94)	14.3	<.001
Year 2	53.7	53.1	68.3	56.6	11.2 (9.93, 12.53)	20.9	<.001
Overall	53.7	53.1	65.1	55.2	9.4 (8.48, 10.30)	17.5	<.001

Appropriate Treatment for Children Denominator: All children age 1 to 18 years at the time of the URI diagnosis, who had an ED or outpatient visit with only a diagnosis of nonspecific (URI). There were 288 168 episodes that met the denominator criteria. Numerator: Children who were not dispensed an antibiotic. (NQF 0069)

Strep Tests for Pharyngitis Denominator: Children 3 to 18 years old, with a negative medication history, who had an outpatient visit, an observation stay, or an ED visit with only a diagnosis of pharyngitis and a dispensed antibiotic for that episode of care. Numerator: A group A streptococcus (strep) test in the 7‐day period from 3 days prior to the episode start date through 3 days after the episode start date. There were 145 483 total weighted URI episodes that met the denominator criteria.

How to interpret the findings: A negative value corresponds to a greater decrease or a smaller increase in the likelihood of a quality of care event in the intervention group relative to the comparison group. A positive value corresponds to a greater increase or a smaller decrease in the likelihood of a quality of care event in the intervention group relative to the comparison group. The relative difference is the D‐in‐D estimate as a percentage of the intervention group's baseline period‐adjusted mean.

Methods: A logistic regression model was used to obtain estimates of the difference in likelihood of a quality of care event. The regression‐adjusted D‐in‐D estimates represent the average treatment effect on the treated, whereas the regression‐adjusted means represent the average treatment effect. As a result, the regression‐adjusted D‐in‐D and the D‐in‐D calculated from the adjusted means will differ.

Data source: RTI analysis of MAX/AMAX Medicaid Claims, 2011‐2014.

The weighted N for all regression analyses was 804 559.

Abbreviations: AR, Arkansas; CG, comparison group; CI, confidence interval; D‐in‐D, difference‐in‐differences; ED, emergency department; URI, upper respiratory infection.

3.2. Perinatal EOC

3.2.1. Descriptive results

Our sample had 58 381 weighted perinatal episodes from FY 2011 to FY 2014. Table 3 displays the characteristics of the perinatal EOC episodes in Arkansas and the comparison group during the last year of the baseline period (2012). Once propensity weights were applied, there were minimal differences in individual sociodemographic and health characteristics. Similar to the URI model, the standardized differences across some county‐level characteristics were expectedly large and are controlled for in the outcome models.

TABLE 3.

Weighted means and standardized differences prior to Arkansas perinatal EOC implementation, Arkansas, and comparison group, 2012 ^a

	Unweighted			Weighted
	Arkansas perinatal EOC group	Comparison group	Standardized difference	Arkansas perinatal EOC group	Comparison group	Standardized difference
N	7438	31 163		7438	7423
Age indicator: 16 to 19 (%)	18.1	15.3	7.5	18.1	18.6	1.2
Age indicator: 20 to 24 (%)	43.3	42.7	1.1	43.3	43.0	0.5
Age indicator: 25 to 34 (%)	35.0	37.4	5.0	35.0	34.8	0.4
Age indicator: 35 and older (Referent)	3.6	4.5	4.8	3.6	3.6	0.0
White (Referent)	62.6	58.7	8.0	62.6	62.4	0.4
Black (%)	27.0	35.4	18.2	27.0	27.1	0.4
Hispanic (%)	5.7	2.6	15.7	5.7	5.9	0.8
Other (%)	4.8	3.4	6.8	4.8	4.6	0.8
Disability (%)	5.8	2.2	18.8	5.8	5.9	0.4
Concurrent Chronic Illness and Disability Payment System Score	1.6	1.9	33.6	1.6	1.6	3.4
Poverty‐related eligibility (%)	76.5	70.6	13.4	76.5	76.2	0.7
Months of full‐Medicaid enrollment during prenatal period	9.1	9.2	6.2	9.1	9.1	0.3
Diabetes (%)	3.9	2.5	8.0	3.9	3.9	0.2
Asthma (%)	2.3	5.6	17.2	2.3	2.4	0.5
Hypertension (%)	1.7	2.8	8.0	1.7	1.7	0.6
Metropolitan status of county of residence (%)	53.9	56.2	4.5	53.9	55.2	2.4
Percent of population at federal poverty level, 2012	20.7	21.0	4.9	20.7	20.7	0.6
Hospital beds per 1000, 2010	3.8	4.6	23.6	3.8	4.5	19.6
Median age, 2010	37.9	37.2	20.2	37.9	37.3	17.6
Percent uninsured, ages <65, 2012	19.5	18.1	48.3	19.5	18.2	46.5

Abbreviation: EOC, Episode of Care

^aThe number of episodes reported here is weighted and for 2012. The total number of unweighted perinatal episodes in our analysis was 148 872.See Appendix S1, Tables A‐6 to A‐9 for annual weighted and unweighted counts of perinatal episodes.

3.2.2. Impact analysis

Table 4 presents the results of the D‐in‐D regression analyses for the perinatal EOC outcomes. There was a small but statistically significant 1.2 percentage point greater increase in the likelihood of an inpatient stay during the prenatal period among Arkansas EOCs, relative to the comparison group (P < .05). Arkansas also had a 0.6 and 0.7 percentage point increase in the likelihood of a 30‐ and 60‐day readmission postdelivery in year 2, respectively, relative to the comparison group (P < .10). Likewise, there was a 0.4 percentage point increase in 60‐day readmissions across the two‐year performance period among EOCs in Arkansas, relative to the comparison group (P < .10). In contrast, the number of ED visits during pregnancy declined by 0.10 visits over both performance years, relative to the comparison group (P < .05). Arkansas had a 6.2 percentage point increase in the probability of HIV screening, relative to the comparison group (P < .001). Likewise, screening for chlamydia increased in Arkansas while declining in the comparison group, resulting in an increase of 9.5 percentage points in the likelihood of chlamydia screening for Arkansas episodes (P < .001). Arkansas had a 2.6 percentage point smaller decline in screening for group B streptococcus, relative to the comparison group (P < .05). There was not a statistically significant impact of the perinatal EOC on cesarean sections.

TABLE 4.

Difference in the pre‐ and postannual change in outcomes for perinatal Episodes of Care in Arkansas, relative to the comparison group, first 2 years of implementation (October 2012 thru September 2014), Weighted N = 58 381

Outcome (%)	Preperiod‐adjusted mean, AR	Preperiod‐adjusted mean, CG	Test‐period–adjusted mean, AR	Test‐period–adjusted mean, CG	Regression‐adjusted difference‐in‐differences (90% confidence interval)	Relative difference (%)	P‐value
Any inpatient admissions during prenatal period, %
Year 1	6.8	5.6	7.0	4.9	0.7 (−0.2, 1.7)	10.8	.22
Year 2	6.8	5.6	8.4	4.8	1.7 (0.4, 3.0)	24.7	.04
Overall	6.8	5.6	7.7	4.9	1.2 (0.4, 2.0)	17.5	.02
30‐day readmission, % a
Year 1	1.7	1.2	1.5	1.1	0.0 (−0.4, 0.4)	−0.4	.98
Year 2	1.7	1.2	2.6	1.0	0.6 (0.0, 1.1)	34.0	.08
Overall	1.7	1.2	2.1	1.0	0.3 (−0.1, 0.6)	16.2	.19
60‐day readmission, % a
Year 1	2.1	1.6	2.0	1.4	0.1 (−0.4, 0.6)	4.0	.78
Year 2	2.1	1.6	3.2	1.3	0.7 (0.1, 1.3)	34.3	.05
Overall	2.1	1.6	2.6	1.3	0.4 (0.0, 0.8)	18.6	.09
Number of ED visits during the pregnancy a
Year 1	0.9	1.3	0.8	1.2	−0.1 (−0.1, 0.2) Ŧ	−7.8	.13
Year 2	0.9	1.3	0.8	1.3	−0.1 (−0.3, −0.0)	−16.3	.07
Overall	0.9	1.3	0.8	1.2	−0.1 (−0.2, −0.0)	−11.9	.02
Cesarean section delivery, %
Year 1	32.4	29.6	30.8	29.3	−1.3 (−3.8, 1.3)	−4.0	.40
Year 2	32.4	29.6	29.9	28.9	−1.8 (−5.7, 2.0)	−5.7	.44
Overall	32.4	29.6	30.4	29.1	−1.6 (−3.8, 0.7)	−4.8	.26
HIV screening, %
Year 1	90.7	83.3	92.8	84.5	2.4 (0.4, 4.4)	2.6	.05
Year 2	90.7	83.3	94.0	80.5	10.4 (6.2, 14.5)	11.4	<.001
Overall	90.7	83.3	93.4	82.6	6.2 (4.0, 8.5)	6.9	<.001
Chlamydia screening, %
Year 1	77.8	83.1	81.7	81.7	5.8 (3.4, 8.2)	7.4	<.001
Year 2	77.8	83.1	84.1	77.7	13.5 (9.4, 17.6)	17.3	<.001
Overall	77.8	83.1	82.9	79.8	9.5 (7.2, 11.8)	12.2	<.001
Group B streptococcus screening, %
Year 1	82.2	86.3	78.6	84.1	−0.7 (−2.6, 1.2)	−0.9	.52
Year 2	82.2	86.3	78.2	76.0	6.1 (2.5, 9.8)	7.4	.006
Overall	82.2	86.3	78.4	80.2	2.6 (0.5, 4.6)	3.1	.04

How to interpret the findings: A negative value corresponds to a greater decrease or a smaller increase in the probability of any utilization (or the average expected number of visits for count models) in the intervention group relative to the comparison group. A positive value corresponds to a greater increase or a smaller decrease in payments or in the rate in the intervention group relative to the comparison group. The relative difference is the D‐in‐D estimate as a percentage of the intervention group's baseline period‐adjusted mean.

Methods: A logistic regression model was used to obtain estimates of the differences in probability of any utilization. The estimates are multiplied by 100 to obtain percentage probabilities. Negative binomial models were used for the number of ED visits.

For binary outcomes estimated using nonlinear models, the regression‐adjusted D‐in‐D estimates represent the average treatment effect on the treated, whereas the regression‐adjusted means represent the average treatment effect. As a result, the regression‐adjusted D‐in‐D and the D‐in‐D calculated from the adjusted means will differ.

Data source: RTI analysis of MAX/AMAX claims FY 2011‐FY 2014.

Abbreviations: CG, comparison group; CI, confidence interval; D‐in‐D, difference‐in‐differences; ED, emergency department.

^aThese outcomes were estimated only on episodes where the beneficiary had full Medicaid benefits during the 60‐day period postdelivery. N = 54 175.

4. CONCLUSION

After two years of EOC implementation, both URI and perinatal EOCs for Arkansas Medicaid beneficiaries showed significant improvements in targeted quality metrics relative to a comparison group. However, there were both unanticipated increases in utilization and potentially some unintended consequences.

Episode‐based payments can improve some process measures of quality in the outpatient setting, but the impact on utilization is mixed. For example, we found an increase in the number of strep tests performed in Arkansas relative to the comparison group, and a decrease in antibiotic dispensing, both of which are quality metrics for Arkansas’ URI EOC and are tied to episode risk and gain sharing. As other work has shown, tying process quality measures to payment can be an effective incentive for providers to improve performance. ²⁷ These results are similar to Arkansas's self‐evaluation ⁹ and are corroborated by interviews with participating providers in Arkansas. ¹⁸ However, these findings should be interpreted with caution. Providers may have moved toward using more specific diagnosis codes so as not to trigger the URI EOC, especially for nonspecific URIs. ¹⁸ According to Arkansas's internal analyses, the number of nonspecific URIs declined by 25 percent from 2012 to 2014. ⁹ Indeed, the overall decline in the probability of having any antibiotic use was driven by a larger decline among nonspecific URIs EOCs, relative to other subtypes (Table B ‐ 1, Appendix S2). If providers in Arkansas avoided diagnosing patients with a URI to avoid triggering the episode, then selection bias may be introduced during the postperiod, increasing the likelihood of producing favorable results.

Similarly, the perinatal EOC increased group B strep tests and screening for HIV and chlamydia among pregnant Medicaid‐covered women in Arkansas relative to the comparison group. These results were expected because the PAP needed to meet an 80 percent screening threshold for all three of these conditions for shared savings. The reduction in ED visits during pregnancy suggests improvements in management of outpatient prenatal care. However, we did not find evidence that the perinatal EOC was associated with a statistically significant reduction in cesarean sections, relative to the comparison group.

Consistent with other work, ¹⁴ our results suggest that episode‐based payments, coupled with performance metrics, can improve processes of care in the outpatient setting and reduce some types of acute services. These findings compliment a recent analysis of perinatal episode‐based payments in the commercial population. ²⁸

A second implication of our findings is that episode‐based payments in the URI and perinatal populations may introduce unintended consequences by increasing acute services. Relative to the comparison group, the URI EOCs reduced URI‐related physician visits, but there was a corresponding increase in URI‐related ED visits suggesting a potential substitution effect. The decline in URI‐specific physician visits is expected; the Medicaid fee schedule in Arkansas is such that one additional physician visit may place providers over the acceptable threshold for average URI costs. ⁹ Moreover, providers may be following up with beneficiaries by telephone or through patient portals, or educating their patients on when to return for an office visit, which may also explain the decline in visits. ¹⁸ However, beneficiaries may have used the ED as a substitute for additional physician visits. Indeed, provider interviews indicated that if a patient wanted an antibiotic or sought further medical assistance, they could visit the ED, particularly after clinic hours. ¹⁸ These findings are consistent with other work describing the reasons for ED use among low‐income patients. ²⁹

The perinatal EOC model may have had the unintended consequence of increasing hospitalizations, both during pregnancy and after delivery. One reason may be that providers may shift care for non–pregnancy‐related conditions to other providers or after the inpatient delivery to avoid incurring the costs during the episode. ¹⁸ We found small increases in hospital 30‐ and 60‐day readmissions postdelivery, which may be the result of deferred treatment of conditions unrelated to pregnancy to control the costs of pregnancy‐related care. As a sensitivity analysis, we analyzed the change in the proportion of postdelivery hospitalizations that are non–pregnancy‐related to see whether providers may be deferring nondelivery treatment to a separate hospital stay. We found some evidence that non–pregnancy‐related hospitalizations postdelivery were increasing in Arkansas relative to the comparison group. ¹⁸ While this is not conclusive evidence of an unintended effect of the EOC model, and moreover not all readmissions are unnecessary, further investigation is warranted. Even so, this finding should be interpreted with caution. The composition of Arkansas beneficiaries with perinatal episodes changed slightly with the introduction of Medicaid expansion in 2014. ¹⁸ The relative increases in hospital utilization could reflect the poorer health status among beneficiaries in Arkansas in 2014.

This study has several important limitations. First, we do not observe the intensity of professional and facility utilization or the impact of the EOC on overall Medicaid costs because cost data were unavailable. Second, our findings may not be generalizable to all Medicaid programs because the comparison group consisted of non‐Medicaid expansion states with large Medicaid managed care enrollment, while Arkansas was an expansion state where all EOCs were among beneficiaries enrolled in FFS Medicaid. Third, as noted above, our findings on antibiotic use in URI EOCs may be a result of selection bias. Fourth, many episodes were excluded due to data limitations, methodological concerns, and state‐based criteria ¹⁸ which may limit generalizability. However, we conducted a sensitivity analysis on all Medicaid deliveries in the state by removing our exclusion criteria and were able to replicate our findings for most outcomes (Table B‐2 and B‐3, Appendix S2). Fifth, we set our test period as October 2012 to September 2014, the period immediately following the state‐defined “baseline” period. However, for perinatal EOCs, the state defined the first “performance period” in which PAPs were accountable for quality and cost outcomes to start March 2013 due to implementation concerns. Thus, our test period included the 6‐month implementation ramp‐up period. This may attenuate any effects we can detect; importantly, our overall results are driven by effects identified in the 2nd‐test year. Finally, unobservable characteristics such as health, education, and other social factors, particularly in 2014, may bias our results.

CMS is prioritizing health care delivery reforms that control spending and improve quality of care for the Medicaid population. ³⁰ , ³¹ , ³² Based on our results, there are several lessons learned for Medicaid programs implementing EOC‐based payment reforms. First, tying shared savings to quality metrics may be helpful in achieving quality targets within the context of episode‐based payments. ³³ Second, EOCs can be successful in disincentivizing over utilization of costly services. Third, the implementation of EOCs may produce unintended consequences, such as potentially shifting care to outside the episode (perinatal), changing diagnostic coding practices (URI), or incentivizing patients to seek care in the ED (URI). Further research is needed to identify the effects of EOCs on Medicaid payments in perinatal and URI EOCs, as well as the impacts of other EOCs on cost and quality in the Medicaid population.

Supporting information

Author Matrix

Appendix S1 and S2

Toth M, Moore P, Tant E, et al. Early impact of the implementation of Medicaid episode‐based payment reforms in Arkansas. Health Serv Res. 2020;55:556–567. 10.1111/1475-6773.13296