Mandatory Medicare Bundled Payment Program for Lower Extremity Joint Replacement and Discharge to Institutional Postacute Care

Key Points

Question

What was the change in discharge to institutional postacute care after lower extremity joint replacement episodes among Medicare beneficiaries following implementation of the Comprehensive Care for Joint Replacement (CJR) bundled payments in 2016?

Findings

In this interim analysis of the first year of a 5-year randomized trial of 75 metropolitan statistical areas (MSAs) that were assigned the bundled payment model and 121 control MSAs that were not, the mean percentage of patient discharges to institutional postacute care was 33.7% in the control group and was 2.9 percentage points lower in MSAs covered by the CJR model, a significant difference.

Meaning

These interim findings suggest that CJR may reduce institutional postacute care following lower extremity joint replacement episodes among Medicare beneficiaries, although further evaluation is needed as the program is fully implemented over time.

Abstract

Importance

Bundled payments are an increasingly common alternative payment model for Medicare, yet there is limited evidence regarding their effectiveness.

Objective

To report interim outcomes from the first year of implementation of a bundled payment model for lower extremity joint replacement (LEJR).

Design, Setting, and Participants

As part of a 5-year, mandatory-participation randomized trial by the Centers for Medicare & Medicaid Services, eligible metropolitan statistical areas (MSAs) were randomized to the Comprehensive Care for Joint Replacement (CJR) bundled payment model for LEJR episodes or to a control group. In the first performance year, hospitals received bonus payments if Medicare spending for LEJR episodes was below the target price and hospitals met quality standards. This interim analysis reports first-year data on LEJR episodes starting April 1, 2016, with data collection through December 31, 2016.

Exposure

Randomization of MSAs into the CJR bundled payment model group (75 assigned; 67 included) or to the control group without the CJR model (121 assigned; 121 included). Instrumental variable analysis was used to evaluate the relationship between inclusion of MSAs in the CJR model and outcomes.

Main Outcomes and Measures

The primary outcome was share of LEJR admissions discharged to institutional postacute care. Secondary outcomes included the number of days in institutional postacute care, discharges to other locations, Medicare spending during the episode (overall and for institutional postacute care), net Medicare spending during the episode, LEJR patient volume and patient case mix, and quality-of-care measures.

Results

Among the 196 MSAs and 1633 hospitals, 131 285 eligible LEJR procedures were performed during the study period (mean volume, 110 LEJR episodes per hospital) among 130 343 patients (mean age, 72.5 [SD, 0.91] years; 65% women; 90% white). The mean percentage of LEJR admissions discharged to institutional postacute care was 33.7% (SD, 11.2%) in the control group and was 2.9 percentage points lower (95% CI, −4.95 to −0.90 percentage points) in the CJR group. Mean Medicare spending for institutional postacute care per LEJR episode was $3871 (SD, $1394) in the control group and was $307 lower (95% CI, −$587 to −$27) in the CJR group. Mean overall Medicare spending per LEJR episode was $22 872 (SD, $3619) in the control group and was $453 lower (95% CI, −$909 to $3) in the CJR group, a statistically nonsignificant difference. None of the other secondary outcomes differed significantly between groups.

Conclusions and Relevance

In this interim analysis of the first year of the CJR bundled payment model for LEJR among Medicare beneficiaries, MSAs covered by CJR, compared with those that were not, had a significantly lower percentage of discharges to institutional postacute care but no significant difference in total Medicare spending per LEJR episode. Further evaluation is needed as the program is more fully implemented.

Trial Registration

ClinicalTrials.gov Identifier: NCT03407885; American Economic Association Registry Identifier: AEARCTR-0002521

This study compares patient discharges to postacute care facilities after lower extremity joint replacement and spending among hospitals mandated vs not mandated by the Centers for Medicare & Medicaid Services to participate in a bundled payment model.

Introduction

The shift toward alternative payment models in Medicare is an important trend in US health care. By 2016, 30% of traditional Medicare reimbursement had been shifted from fee-for-service (FFS) models to alternative payment models.¹ Bundled payments are one of the leading alternative payment models. Under bundled payments, health care organizations (such as hospitals, physician groups, and postacute care providers) receive a single “bundled” payment for all services related to a specific treatment (eg, hip replacement). By holding multiple parties jointly accountable for quality and costs, bundled payments may encourage coordination of care and reduce unnecessary utilization. However, because these parties are paid a fixed amount irrespective of the volume of services, they may also respond by reducing necessary care or by trying to treat only healthier patients with lower expected costs.²

Bundled payments have been widely touted for their potential to have a substantial, positive effect on health care delivery,^3,4,5,6,7 yet there is limited rigorous evidence on their effects. Most studies of bundled payments have been observational, focusing on the experience of a small number of hospitals that voluntarily participated. These studies have tended to find large savings,^8,9,10,11,12 but voluntary participation makes separating treatment from selection effects difficult,¹³ and the small number of participating hospitals raises concerns about generalizability.

To address these gaps in scientific knowledge, this study took advantage of the Centers for Medicare & Medicaid Services’ (CMS’s) random assignment of some metropolitan statistical areas (MSAs) to a Medicare bundled payment model for lower extremity joint replacement (LEJR)—ie, hip and knee replacement. In 2014, there were 486 249 LEJR procedures, accounting for $6.2 billion of Medicare inpatient spending (4.52% of all inpatient Medicare spending).^14,15 CMS designed and randomly assigned the 5-year, nationwide bundled payment program for LEJR, known as Comprehensive Care for Joint Replacement (CJR), which began on April 1, 2016. Unlike previous bundled payment programs, participation was mandatory for all covered hospitals. The purpose of this study was to analyze results from an interim analysis of the first performance year of this 5-year program.

Methods

The primary objective of this study was to evaluate interim outcomes during the first year of implementation of a bundled payment model for LEJR, focusing on discharge to institutional postacute care following joint replacement hospitalization episodes.

Institutional review board (IRB) exemption was obtained from the Massachusetts Institute of Technology’s Committee on the Use of Humans as Experimental Subjects (COUHES 1710117275) and IRB approval for Medicare data analysis was obtained from Dartmouth’s IRB (15475). CMS did not require informed consent for patients in CJR; both IRBs waived informed consent for the analysis of the Medicare claims data.

Study Design

In July 2015, CMS publicly announced in the Federal Register its exclusion criteria and randomization procedure for selecting the 196 eligible MSAs¹⁶; MSAs were excluded primarily because of low LEJR discharge volume. Within eligible MSAs randomized to CJR, hospitals were required to participate in CJR if they were paid under prospective payment and not already participating in Model 1 or Phase 2 (Models 2 or 4) of the Bundled Payments for Care Improvement Initiative (BPCI), a preexisting Medicare voluntary bundled payments model for LEJR.

Within eligible hospitals, the patient inclusion criteria included Medicare Part A and Part B coverage, no readmission during the episode for LEJR, and no death during the episode. Applying the hospital and patient exclusion criteria to all Medicare FFS LEJR episodes in eligible MSAs in 2016, we estimated that 75% of episodes would be covered by CJR if the MSA was selected for treatment; section 1 of eAppendix 1 and eTable 1 in Supplement 1 provide more detail on the eligibility criteria and this estimate.

The 196 eligible MSAs were divided into 8 strata based on quartile of historical LEJR payments and above- vs below-median MSA population. CMS set different treatment probabilities (ie, probability of selection for CJR) for MSAs by strata (ranging from 30% to 45%); MSAs with higher historical LEJR spending had higher treatment probabilities. CMS performed the randomization in SAS Enterprise Guide version 7.1 (SAS Institute Inc) using the PROC SURVEYSELECT statement with METHOD=SRS.

In July 2015, CMS publicly announced that based on this randomization, 75 MSAs were initially assigned to treatment and 121 to control (Figure). Following prior independent analyses of government-implemented randomization protocols,^17,18 we verified via simulation that we could reproduce the randomization procedure to within statistical sampling error (section 1 of eAppendix 1 and eTable 2 in Supplement 1).

Figure. MSA Eligibility and Randomization in a 5-Year Randomized Trial of a Mandatory Medicare Bundled Payment Program for Lower Extremity Joint Replacement (LEJR) Episodes.

eTable 1 in the Supplement provides more details on the eligibility criteria and randomization process. MSA indicates metropolitan statistical area; CJR, Comprehensive Care for Joint Replacement program.

^aOriginal eligibility criteria were (1) at least 400 LEJR episodes in the baseline period between July 1, 2013, and June 30, 2014; (2) at least 400 non–Bundled Payments for Care Improvement Initiative (BPCI) LEJRs in the baseline period; (3) at least 50% of LEJR episodes in the baseline period were non-BPCI; and (4) at least 50% of otherwise eligible LEJR episodes not in Maryland hospitals. For eligibility criterion 2, BPCI participation was defined as hospitals participating in BPCI model 1 and phase 2 of BPCI models 2 or 4 as of July 1, 2015. For eligibility criterion 3, BPCI participation was defined in 2 steps: first, less than 50% of potentially eligible LEJR episodes were in hospitals participating in phase 2 of BPCI models 2 or 4 as of July 1, 2015; second, less than 50% of LEJR referrals to skilled nursing facility or home health agency services were made up of skilled nursing facilities or home health agencies participating in BPCI model 3 as of July 1, 2015.

^bThe 196 eligible MSAs were divided into 8 strata based on the full interaction of (1) average wage-adjusted historical LEJR episode payment, grouped into quartiles, and (2) MSA population size, grouped into above and below median. Randomization occurred within strata. Treatment probabilities varied within the payment quartiles: 30% in the first quartile (lowest payment), 35% in the second, 40% in the third, and 45% in the fourth (highest payment).

^cAfter randomization took place, the Centers for Medicare & Medicaid Services received comments that the original eligibility criteria did not take into account providers that entered into phase 2 of BPCI by October 1, 2015, which was the final quarter a phase 1 BPCI participant could transition into phase 2. CMS therefore revised the definition of BPCI participation in original eligibility criteria 2 and 3. The revised eligibility criterion 2 defined BPCI participation hospitals participants as of October 1, 2015, instead of as of July 1, 2015, and also included episodes associated with a physician who was in a physician group practice in phase 2 of BPCI model 2 as of October 1, 2015. Similarly, the revised eligibility criterion 3 defined BPCI participation based on the list of BPCI-participating hospitals, skilled nursing facilities, and home health agencies as of October 1, 2015, instead of as of July 1, 2015. The revised eligibility criteria resulted in exclusion of 8 MSAs from the CJR group, resulting in a final 67 MSAs in the CJR group. CMS did not announce which MSAs would have been excluded from the control group based on the revised criteria.

In November 2015, CMS publicly updated the MSA exclusion criteria in response to comments that the original criteria did not take into account hospitals and physician group practices that entered into Phase 2 BPCI by October 1, 2015; as a result, 8 MSAs were excluded from the treatment group without a corresponding set of MSAs excluded from the control group.¹⁹

Intervention

The CJR is a Medicare bundled payment model for LEJR that holds acute care hospitals financially responsible for Medicare spending over the entire episode of care. An episode begins with a hospital stay with a discharge in 1 of 2 included diagnosis related groups (DRGs) (MS-DRGs 469 and 470) and ends 90 days after discharge. The CJR was introduced in April 2016 and designed to last for 5 years.

Under CJR, hospitals face financial incentives to reduce Medicare FFS spending and to maintain or increase quality. At the end of each year, hospitals that (1) have per-episode Medicare FFS spending below the target price (set by CMS based on historical hospital and regional episode spending and the reason for admission) and (2) have met a minimum quality standard (5 of 20 on a composite quality score) receive “shared savings” from CMS for the difference between the target price and spending up to a stop-gain amount (ie, maximum bonus payment to the hospital), with higher scores making them eligible for greater savings. Hospitals that have FFS spending of more than the target price are responsible for paying the difference up to a stop-loss amount (ie, maximum financial penalty to the hospital). The upside and downside risks increase over time. In the first year, the stop gain was +5% and there was no downside risk; by the fifth year, the stop gain and stop loss were each scheduled to be 20% of the target price.^19,20

The use of random assignment by a government agency such as CMS is rare but not unprecedented and can be valuable for scientific research. For example, the state of Oregon used a random lottery to expand Medicaid coverage, enabling academic research on the Oregon Health Insurance Experiment.^{17,18,21,22,23}

Data and Outcome Measures

We studied the first performance year of CJR, which includes episodes that begin on or after April 1, 2016, and end no later than December 31, 2016. Specifically, we analyzed episodes starting between April 1, 2016 ,and September 15, 2016; the end date was chosen so that all episodes would fall within the performance year (given a mean length of stay for an LEJR admission of 3.1 days for DRG 470 and 7.0 days for DRG 469).²⁴

We used Medicare FFS claims data for 100% of enrollees from 2012-2014 and 2016 in the 196 eligible MSAs. We limited the sample to episodes that would have been covered by CJR if the MSA were included in the treatment group. We omitted data from 2015 because treatment MSAs were announced midway through 2015 and behavior was potentially affected during that year. We also used Hospital Compare data²⁵ to construct an estimate of the targeted quality measure, and data on hospital-specific end-of-year reconciliation (ie, bonus) payments.²⁰ Section 2 of eAppendix 1 in Supplement 1 provides more detail on data and outcomes.

The primary outcome was the share of LEJR admissions discharged to institutional postacute care—these are skilled nursing facilities, long-term care hospitals, and inpatient rehabilitation facilities. Existing observational studies^11,12 suggested that this would be the primary margin of adjustment, and this was a margin where power calculations suggested that reasonably sized effects could be detected (prespecified analysis plan available in eAppendix 2 in Supplement 1).

Secondary outcomes included the number of days in institutional postacute care during the episode, discharges to other locations, Medicare FFS spending during the episode (both overall and for institutional postacute care), and net Medicare spending during the episode, which adds to Medicare FFS spending any reconciliation payments made to a treatment hospital under CJR.

Other secondary outcomes included LEJR patient volume, patient comorbidity severity, and several quality measures. Patient comorbidity severity was measured by the Elixhauser Comorbidity Index, which is the sum of 31 different comorbidity indicators.^26,27 The targeted quality measure was a modified composite quality score, derived from measures of total hip/knee arthroplasty complication rates and patient experience ratings (see section 2 of eAppendix 1 in Supplement 1 for more detail). The score ranges from 0 to 18, with higher numbers indicating better quality. Because the score in the first year was based almost entirely on data from prior to the introduction of CJR,²⁰ we noted in the preanalysis plan that we did not expect it to be affected (eAppendix 2 in Supplement 1). We therefore also examined nontargeted quality measures: the 90-day emergency department visit rate, a quality measure used in prior analyses of voluntary bundled payments for LEJR^11,12; the 90-day all-cause readmission rate, a standard quality measure previously used for LEJR^11,28; and the 90-day complication rate for total hip and total knee arthroplasty, a part of the targeted quality measure that we could observe for admissions during the study period.

Statistical Analyses

The analyses were conducted at the MSA level. Not all randomly assigned MSAs were included in CJR (Figure). As prespecified (eAppendix 2 in Supplement 1), we therefore used a standard instrumental variable approach²⁹ to evaluate the relationship between inclusion in CJR and the outcomes; in the first-stage regression, assignment to CJR was used as an instrument for inclusion in CJR, and in the second-stage regression, inclusion in CJR was related to outcomes. As prespecified, all regressions included strata fixed effects because treatment probabilities varied by strata, and controlled for 2 years of lags of the dependent variable (specifically, 2013 and 2014) to improve statistical power. Based on historical data, we estimated the analysis to have power to detect a 2-percentage-point reduction in the primary outcome (2-sided α = .05 at 80% power). All analyses were prespecified (prior to obtaining postintervention data) other than analysis of the Elixhauser Comorbidity Index, which was requested during the review process, and the specific breakdown of discharges to noninstitutional postacute care, which was done for ease of exposition. Because of the strata sampling design, comparison of raw means of the treatment and control MSAs was not appropriate; instead, values for the control MSAs and the estimated differences (using the instrumental variable approach) between bundled payment and control MSAs are reported. Raw comparisons of means between original bundled payment and control MSAs—separately by strata—are presented in section 2 of eAppendix 1 in Supplement 1. Because we did not adjust for multiple testing of secondary outcomes, secondary outcome analyses should be considered exploratory.

A number of sensitivity and additional analyses were performed. Results were analyzed without controlling for the lags of the dependent variable, and intention-to-treat analyses were conducted to compare outcomes for the 75 MSAs originally assigned to treatment with outcomes for the 121 control MSAs. Additional detailed analyses of the number and type of patients were also reported. Section 2 of eAppendix 1 in Supplement 1 provides more detail.

All statistical analyses were conducted using Stata version 15.1 (StataCorp), with 2-tailed tests with a statistical significance threshold of P<.05.

Results

Study Population

Of the 196 eligible MSAs, 75 were initially assigned to treatment and 121 to control; 8 of the 75 were subsequently excluded from CJR so that 67 MSAs were covered by CJR. The eFigure in Supplement 1 shows the geographic distribution. Assignment to CJR thus increased the chance of being included in CJR by 89.1 percentage points (95% CI, 81.8-96.4 percentage points; P < .001).

In 2016, 167 hospitals were excluded from treatment or control MSAs because of preexisting BPCI participation, leaving 1647 eligible hospitals; 1633 of these hospitals had an eligible episode. Table 1 describes the CJR-eligible study population at the MSA level. On average, among eligible MSAs, the patient study population was 90% white and 65% female, with a mean age of 72.5 (SD, 0.91) years and a mean Elixhauser Comorbidity Index of 2.4 (SD, 0.29). Eligible hospitals had a mean of 289.1 beds and performed a mean of 20.4 CJR-eligible LEJR procedures per month over the study period (April-September 2016); 25.3% were for-profit hospitals and 9.3% were teaching hospitals. Among eligible MSAs, the mean number of acute care hospitals was 14.8 and the mean number of institutional postacute care providers was 50.9.

Table 1. Characteristics of the Study Population (All Eligible MSAs)^a.

Characteristics	Mean (SD)	Median (IQR)
Characteristics of eligible LEJR patients
Age, y	72.5 (0.91)	72.5 (71.8-73.0)
White, %	90.0 (0.07)	92.2 (86.3-95.1)
Female, %	64.8 (0.03)	64.9 (62.7-67.0)
Disabled, %	17.2 (0.06)	16.0 (13.1-20.2)
Also receiving Medicaid, %	11.7 (0.06)	10.8 (7.9-14.3)
Elixhauser Comorbidity Indexb	2.4 (0.29)	2.4 (2.2-2.5)
Characteristics of eligible hospitals
CJR-eligible LEJR volume per hospital per month, No.c	20.4 (11.6)	17.5 (12.8-23.8)
For-profit hospitals, %d	25.3 (25.8)	25.0 (0-50.0)
Teaching hospitals, %d	9.3 (16.1)	0 (0-13.7)
Hospital beds, No.d	289.1 (140.9)	258.1 (205.0-332.8)
Characteristics of eligible MSAs
Acute care hospitals, No.e	14.8 (16.0)	9.0 (6.0-18.0)
Institutional postacute care facilities, No.e	50.9 (53.3)	32.0 (20.0-58.0)
Home health agencies, No.e	57.8 (117.1)	24.0 (13.0-59.0)

Abbreviations: CJR, Comprehensive Care for Joint Replacement program; IQR, interquartile range; LEJR, lower extremity joint replacement; MSA, metropolitan statistical area.

^aTable reports characteristics of CJR-eligible patients with Medicare who underwent LEJR (n=131 285 episodes), CJR-eligible hospitals (n=1633 hospitals), and CJR-eligible MSAs (n=196 MSAs). All data are reported at the MSA level. Means for characteristics of eligible LEJR patients and eligible hospitals are taken first within MSAs and then across MSAs. Unless otherwise noted, all data are based on 2016 Medicare claims data; episodes starting between April 1, 2016, and September 15, 2016, are included. At the patient level, the mean age was 72.6 (SD, 8.5) years (n=130 343) and the mean Elixhauser Comorbidity Index was 2.4 (SD, 1.7) (n=130 343).

^bThe Elixhauser Comorbidity Index is a comorbidity measure based on International Classification of Diseases, Ninth Revision, Clinical Modification or International Statistical Classification of Diseases and Related Health Problems, Tenth Revision diagnosis codes. Binary indicators were created for a list of 31 comorbidities; the index was created by summing these 31 indicators. An index of 0 means no comorbidities were present for the episode and a higher index means more comorbidities. The range of this measure is 0 to 31.

^cMean number of CJR-eligible LEJR episodes across hospitals in each MSA.

^dBased on 2014 American Hospital Association data.

^eBased on 2016 Medicare Provider of Services file.

Balance

Prior to randomization, characteristics were balanced across control and treatment MSAs. Table 2 shows balance at baseline (2014) for the outcome variables; an F test failed to reject equality of all of the outcomes (P = .54). eTable 3 in Supplement 1 shows balance on MSA demographics and eTables 4 and 5 in Supplement 1 show raw means for each stratum.

Table 2. Balance of Study Population by Group Prior to Implementation of CJR (2014)^a.

	Mean (SD) Value in Control MSAs (n = 121)	Mean Difference (95% CI) Between Original Bundled Payment and Control MSAsb	P Valuec
Discharge destination, %
Institutional postacute cared	41.0 (13.1)	−0.4 (−1.7 to 0.84)	.51
Home health agencye	31.3 (17.0)	1.4 (−0.08 to 2.9)	.07
Home (without home health agency)e	25.1 (21.6)	−0.7 (−2.2 to 0.67)	.31
Othere,f	2.5 (4.4)	−0.03 (−0.41 to 0.34)	.88
No. of days in institutional postacute careg	10.4 (2.9)	0.04 (−0.41 to 0.48)	.88
Medicare episode spending in institutional postacute care, $	4719 (1410)	24 (−136 to 185)	.77
Total Medicare episode spending, $	24 160 (3465)	−29 (−300 to 243)	.84
90-d Complication rate, %h	1.4 (0.69)	0.01 (−0.18 to 0.21)	.89
90-d Emergency department visits during episode, %	20.1 (3.1)	0.56 (−0.20 to 1.3)	.15
90-d All-cause readmission rate, %i	10.9 (2.2)	−0.08 (−0.69 to 0.54)	.81
LEJR admissions, No. per 1000 enrollees	8.3 (4.3)	0.13 (−0.09 to 0.34)	.26
CJR-eligible LEJR admissions, No. per 1000 enrolleesj	6.9 (3.2)	0.06 (−0.23 to 0.36)	.69
Elixhauser Comorbidity Indexe,k	2.4 (0.28)	0.01 (−0.04 to 0.09)	.75

Abbreviations: CJR, Comprehensive Care for Joint Replacement program; LEJR, lower extremity joint replacement; MSA, metropolitan statistical area.

^aTable reports MSA-level average characteristics of CJR-eligible LEJR episodes in 2014. An F test of the hypothesis that all of the outcomes are jointly zero yielded an F statistic of 1.05 and P=.54. N = 196 MSAs included overall. All measures are based on 2014 Medicare claims data; episodes starting between April 1, 2014, and September 15, 2014, are included.

^bEstimated difference between the 121 control MSAs and the original 75 treatment MSAs from an ordinary least squares regression of the outcome on an indicator variable for original treatment status and controls for strata and for 2 lags of the dependent variable (measured in 2012 and 2013).

^cP values for differences are based on heteroskedasticity-robust standard errors.

^dInstitutional postacute care includes skilled nursing facilities, long-term care hospitals, and inpatient rehabilitation facilities.

^eOutcome not prespecified in analysis plan.

^fIncludes Medicare-approved swing beds and inpatient care at other acute care hospitals, intermediate care facilities, and other less common destinations such as psychiatric hospitals, hospice care, and federal hospitals.

^gSum of days in skilled nursing facilities, long-term care hospitals, and inpatient rehabilitation facilities. This measure is averaged across all episodes, not just episodes with institutional postacute care use.

^hComplication rate is the share of CJR-eligible patients who have at least 1 of the 8 underlying complications that contribute to the total hip arthroplasty/total knee arthroplasty 90-day complication measure used in the targeted quality score; unlike the input into the targeted quality score, the complication measure here is not risk-standardized and is measured for admissions during the study period rather than as a 3-year moving average. The sample for this measure includes patients who were readmitted for LEJR and those who died within the episode, both of which would otherwise be excluded from the CJR sample based on patient eligibility criteria.

ⁱThe sample for this measure includes patients who were readmitted for LEJR, who would otherwise be excluded from the CJR sample based on patient eligibility criteria.

^jSome hospitals and some LEJR patients are excluded from CJR. All other analyses in the article are limited to CJR-eligible admissions unless otherwise specified.

^kSee footnote to Table 1 for detailed description.

Health Care Use and Spending

Table 3 shows results of the instrumental variable analysis of the relationship between inclusion of MSAs in the CJR model and health care use and Medicare spending. For the primary outcome, the mean percentage of LEJR admissions discharged to institutional postacute care was 33.7% (SD, 11.2%) in the control group and was 2.9 percentage points lower (95% CI, −4.95 to −0.90 percentage points; P = .005) in the CJR group, a significant difference.

Table 3. Health Care Use and Spending During First Year of CJR (2016)^a.

	Mean (SD) Value in Control MSAs (n = 121)	Mean Difference (95% CI) Between Bundled Payment and Control MSAs	P Valueb
Discharge destination, %
Institutional postacute carec	33.7 (11.2)	−2.9 (−4.95 to −0.90)	.005
Home health agencyd	32.2 (19.4)	1.3 (−1.73 to 4.34)	.40
Home (without home health agency)d	32.2 (23.3)	2.6 (−0.79 to 5.90)	.14
Otherd	1.9 (3.0)	−0.40 (−0.84 to 0.05)	.08
No. of days in institutional postacute care	8.2 (2.7)	−0.48 (−1.06 to 0.11)	.12
Medicare episode spending in institutional postacute care, $	3871 (1394)	−307 (−587 to −27)	.04
Total Medicare episode spending, $e	22 872 (3619)	−453 (−909 to 3)	.06
Medicare spending net of reconciliation payment, $e,f	22 872 (3619)	234 (−214 to 683)	.31

Abbreviations: CJR, Comprehensive Care for Joint Replacement program; MSA, metropolitan statistical area.

^aChange with bundled payment was estimated with the use of 2-stage least-squares instrumental variable regression, with random assignment to CJR used as an instrument for inclusion in CJR. All regressions include indicators for strata and control for 2 lags of the dependent variable. All outcomes were measured during the episode. All measures are based on 2016 Medicare claims data unless otherwise noted; episodes starting between April 1, 2016, and September 15, 2016, are included; lags of the dependent variable are based on the corresponding periods in 2013 and 2014. N = 196 MSAs included overall. See Table 2 for variable definitions.

^bP values for differences are based on heteroskedasticity-robust standard errors.

^cPrimary outcome.

^dThe preanalysis plan specified that discharges to home health agencies would be combined with discharges to institutional postacute care rather than being analyzed separately, and analysis of the remaining categories was not specified. The changes are nonsubstantive and designed for ease of exposition.

^eBecause control MSAs do not receive reconciliation payments, control MSA total Medicare episode spending and Medicare spending net of reconciliation payment are identical.

^fMedicare spending net of reconciliation payment equals total Medicare episode spending (previous row) plus any reconciliation payments made by Medicare. Data on reconciliation payment were downloaded from https://innovation.cms.gov/Files/x/cjr-py1reconpym.xlsx. Because reconciliation payment did not exist prior to 2016, analyses instead controlled for lags of total Medicare episode spending.

Table 3 also shows secondary outcomes. The mean percentage of LEJR patients discharged to home without home health care was 32.2% (SD, 23.3%) in the control group and was 2.6 percentage points higher (95% CI, −0.79 to 5.90 percentage points; P = .14) in the CJR group, a statistically nonsignificant difference. There was no statistically significant relationship between inclusion in CJR and discharges to other destinations.

Medicare spending on institutional postacute care was $3871 (SD, $1394) in the control group and was $307 lower (95% CI, −$587 to −$27; P = .04) in the CJR group, a statistically significant difference. There was no statistically significant relationship between inclusion in CJR and total Medicare FFS spending per episode, either gross or net of CJR reconciliation payments. Mean total Medicare spending per LEJR episode was $22 872 (SD, $3619) in the control group. Excluding reconciliation payments, total Medicare spending per LEJR episode was $453 lower (95% CI, −$909 to $3; P = .06) in the CJR group, and inclusive of reconciliation payments, total Medicare spending per LEJR episode was $234 higher (95% CI, −$214 to $683; P=.31) in the CJR group, a nonsignificant difference.

Health Care Quality and Volume

Table 4 shows no statistically significant or substantively meaningful relationship between inclusion in CJR and any of the targeted quality or nontargeted quality measures. For example, the 90-day emergency department visit rate was 20.1% (SD, 2.9%) for the control group, and was 0.25 percentage points higher (95% CI, −0.44 to 0.93 percentage points; P = .48) in the CJR group, a statistically nonsignificant difference.

Table 4. Quality and Patient Case Mix During First Year of CJR (2016)^a.

	Mean (SD) Value in Control MSAs (n = 121)	Mean Difference (95% CI) Between Bundled Payment and Control MSAs	P Valueb
Quality measures
Targeted measure: composite quality scorec	9.3 (2.8)	0.72 (−0.12 to 1.55)	.10
90-d Nontargeted quality measures, %
Complication rate	1.2 (0.71)	−0.002 (−0.21 to 0.21)	.99
Emergency department visit rate	20.1 (2.9)	0.25 (−0.44 to 0.93)	.48
All-cause readmission rate	10.1 (2.0)	0.01 (−0.55 to 0.58)	.97
Admissions and patient case mix
LEJR admissions, No. per 1000 enrollees	9.0 (4.8)	−0.14 (−0.42 to 0.14)	.34
CJR-eligible LEJR admissions, No. per 1000 enrollees	7.2 (3.5)	0.05 (−0.32 to 0.42)	.80
Elixhauser Comorbidity Indexd	2.3 (0.27)	−0.01 (−0.07 to 0.05)	.73

Abbreviations: CJR, Comprehensive Care for Joint Replacement program; LEJR, lower extremity joint replacement; MSA, metropolitan statistical area.

^aChange with bundled payment was estimated with the use of 2-stage least squares instrumental variable regression with random assignment to CJR used as an instrument for inclusion in CJR. All regressions include indicators for strata and control for 2 lags of the dependent variable. All outcomes are based on 2016 Medicare claims data unless otherwise noted; episodes starting between April 1, 2016, and September 15, 2016, are included. N = 196 MSAs included overall.

^bP values for differences are based on heteroskedasticity-robust standard errors.

^cThis measure ranges from 0 to 18, with higher scores indicating higher quality. The instrumental variable estimate does not include lags of the dependent variable because it involves 3-year moving averages. As discussed in the text, this measure was not expected to be affected because the score for the first year was based almost entirely on data prior to the introduction of CJR.

^dSee footnote to Table 1 for detailed description. Outcome not prespecified in analysis plan.

Table 4 also shows no statistically significant relationship between inclusion in CJR and patient volume or case mix. For example, the mean number of CJR-eligible admissions per 1000 enrollees was 7.2 (SD, 3.5) in the control group and was 0.05 higher (95% CI, −0.32 to 0.42; P = .80) in the CJR group; the mean Elixhauser Comorbidity Index was 2.3 (SD, 0.27) in the control group and was 0.01 lower (95% CI, −0.07 to 0.05; P = .73) in the CJR group. eTables 6 and 7 in Supplement 1 further show no statistically significant relationship between inclusion in CJR and patient admissions under the various exclusion criteria (such as readmission for LEJR or death) and other measures of case mix (such as age or number of Charlson comorbidities^27,30).

eTable 8 in Supplement 1 shows that not controlling for lags of the dependent variable reduced precision but did not substantively change the results. eTables 9 and 10 in Supplement 1 show intention-to-treat analysis of the relationship between original assignment to CJR and the outcomes, and eTable 11 and 12 in Supplement 1 show raw means for the outcome data by strata.

Discussion

In an instrumental variable analysis of the first year of the CJR bundled payment model, MSAs that were covered by the CJR model, compared with those that were not, had a significantly lower percentage of patient discharges to institutional postacute care and significantly lower spending on institutional postacute care but no significant change in overall Medicare expenditures, particularly after accounting for the reconciliation (ie, bonus) payments. These results are consistent with previous work suggesting that reductions in postacute care are the first-line response of health systems to the introduction of alternative payment mechanisms.³¹ There was no significant relationship between inclusion in CJR and targeted or nontargeted measures of quality of care, nor did hospitals appear to change their rates of admissions for covered patients or strategically admit patients with lower illness severity, as some observers have feared.²

The relationship between CJR and health care use and spending was smaller than those from prior observational studies of Medicare bundled payment programs. A difference-in-differences analysis of the associations of BPCI for LEJR found a 4% decline in Medicare spending¹²; a pre-post analysis of BPCI for LEJR in one health system found a 21% decline and received considerable attention in the media.^11,32 In addition, a matched-control study of Medicare’s Heart Bypass Center Demonstration Project estimated net savings of 14% per episode⁸ and was cited as reason to expect substantial savings from subsequent bundled payment programs.³

There are several possible reasons the findings from the current study of a randomized, mandatory bundled payment program contrast with prior studies of voluntary bundled payment models for LEJR. One reason is selection on expected costs. Estimates of the savings from voluntary programs may be biased upward if hospitals with lower expected spending are more likely to sign up; fully controlling for this can be challenging.¹³ A second reason is selection on treatment effects.³³ The effect of bundled payment on the subsample of hospitals that select into the voluntary program may not be representative of the average effect across all hospitals.

A third reason is the size of the incentives. In the first year of CJR, financial incentives were about one-fourth the size of those in the voluntary BPCI model and had no downside risk; by years 4 and 5, CJR incentives will be the same as those in BPCI. An examination of the effects of CJR in subsequent years will be important, as the phase-in of downside risk and larger incentives may reduce Medicare spending (holding hospital behavior fixed) and may cause larger changes in hospital behavior. However, a comprehensive analysis of the CJR program as initially designed will not be possible. In December 2017, CMS modified CJR to be voluntary for 33 of the 67 included MSAs starting in performance year 3. This means that starting in performance year 3, the treatment vs control analysis performed herein can only be implemented for the remaining 34 MSAs where enrollment is mandatory.³⁴

Limitations

This study has several limitations. First, this study evaluated outcomes only during the first year following implementation of the bundled payment program, when the maximum stop-gain and stop-loss incentives had not yet been implemented. Second, the analysis was powered to detect differences in discharges to institutional postacute care but not the corresponding difference in total episode Medicare spending. Third, the analysis did not explore heterogeneity in effects of bundled payments across patients or hospitals, which might shed light on potential mechanisms; in this spirit, early work has compared characteristics of hospitals that did and did not achieve shared savings in the first year.³⁵ Fourth, potentially important health outcomes such as pain or functional limitations were not analyzed. Fifth, analysis of the relationship between bundled payments and the targeted quality measure is not yet meaningful, given that in the first year it was almost entirely measured prior to the start of CJR. Sixth, the analysis pertained only to health care use in the bundle during the covered episode; as more data become available, it will be of interest to see what happens to health care use and spending over longer horizons and health care use outside of the bundle (such as prescription pain medication covered by Medicare Part D).

Conclusions

In this interim analysis of the first year of the CJR bundled payment model for LEJR among Medicare beneficiaries, MSAs covered by CJR, compared with those that were not, had a significantly lower percentage of discharges to institutional postacute care but no significant difference in total Medicare spending per LEJR episode. Further evaluation is needed as the program is more fully implemented.

Supplement 1.

eAppendix 1. Supplemental Methods and Results

eFigure. Map of MSAs

eTable 1. Summary of Strata

eTable 2. Simulation

eTable 3. Additional Balance

eTable 4. Balance by Strata

eTable 5. Additional Balance by Strata

eTable 6. Eligibility Analysis

eTable 7. Additional Patient Composition

eTable 8. Sensitivity

eTable 9. Healthcare Use and Spending (ITT)

eTable 10. Quality and Patient Composition (ITT)

eTable 11. Healthcare Use and Spending by Strata (ITT)

eTable 12. Quality and Patient Composition, by Strata (ITT)

eAppendix 2. Prespecified Analysis Plan

Supplement 2.

Data Sharing Statement