Association of Medicare's Bundled Payments for Care Improvement initiative with patient‐reported outcomes

Abstract

Objective

To determine whether the Bundled Payments for Care Improvement (BPCI) initiative affected patient‐reported measures of quality.

Data Sources

Surveys of Medicare fee‐for‐service beneficiaries discharged from acute care hospitals participating in BPCI Model 2 and comparison hospitals between October 2014 and June 2017. Variables from Medicare administrative data and the Provider of Services file were used for sampling and risk adjustment.

Study Design

We estimated risk‐adjusted differences in patient‐reported measures of care experience and changes in functional status, for beneficiaries treated by BPCI and comparison hospitals.

Data Collection

We selected a stratified random sample of BPCI and matched comparison beneficiaries. We fielded nine waves of surveys using a mail and phone protocol, yielding 29 193 BPCI and 29 913 comparison respondents.

Principal Findings

Most BPCI and comparison survey respondents reported a positive care experience and high satisfaction. BPCI respondents were slightly less likely than comparison respondents to report positive care experience or high satisfaction. Despite these differences in care experience, there was no difference between BPCI and comparison respondents in self‐reported functional status approximately 90 days after hospital discharge.

Conclusions

These findings reduce concerns that BPCI may have unintentionally harmed patient health but suggest room for improvement in patient care experience.

1. INTRODUCTION

The Centers for Medicare & Medicaid Services (CMS) implemented the Bundled Payments for Care Improvement (BPCI) initiative to incentivize providers to deliver care more efficiently with equal or better quality.1 Four different models exist within the BPCI initiative, each with different rules and incentives. Each BPCI participant can choose to enroll in one Model and any combination of 48 clinical episodes or groups of Medicare Severity Diagnosis Related Group (MS‐DRG). Under Model 2 of the initiative, which began in 2013, acute care hospitals (ACHs) or physician group practices (PGPs) initiate episodes of care triggered by a hospital admission for an applicable MS‐DRG and take responsibility for Medicare spending during the episode of care.2 Medicare makes fee‐for‐service (FFS) payments to providers as usual under Medicare parts A and B, and those expenditures are compared to a target price based on discounted, historical average payments for an episode of care. If expenditures exceed the target price, the BPCI participant may have to repay CMS, but if expenditures are less than the target price, the BPCI participant is eligible for additional payments.

In responding to new payment models, participants must determine their approaches to care redesign implementation.3 Case studies of BPCI participants indicate that, among other implementation strategies, many participants reduced use of skilled nursing facilities in response to BPCI.4, 5

Quantitative evidence further demonstrates that BPCI led to lower intensity postacute care (PAC) services for Model 2 ACH episodes, with more patients discharged from the hospital to home, with care provided by home health agencies rather than institutional care, and with shorter stays for patients discharged to institutional care.6 These changes resulted in lower average risk‐adjusted Medicare payments for most clinical groups.6, 7 By encouraging providers to keep costs under a predetermined target price, BPCI raises concerns about potential care stinting, reduced care quality, and adverse patient experiences.8, 9, 10 Results to‐date do not indicate that BPCI was associated with increases in all‐cause mortality, emergency department use, or unplanned readmissions, nor was BPCI associated with changes in volume among lower extremity joint replacement episodes.6, 7, 11 Evidence from other evaluations of bundled payment interventions is generally consistent with these findings.12 For example, Medicare's Acute Care Episode (ACE) Demonstration Program (2007‐2012) was associated with lower total 30‐day PAC payments but not reduced 30‐day episode‐based Medicare payments.13 ACE also did not impact 30‐day mortality for cardiac or orthopedic surgery.13 Another study of ACE and BPCI at a single hospital system found reduced average Medicare episode expenditures and decreased PAC spending when the bundled payment included responsibility for PAC and found no impact on readmissions or emergency department use.14

Concerns remain that claims‐based measures of patient health are insufficiently granular to identify potential harm to patients.15 Patient‐reported outcomes can help identify impacts of bundled payment initiatives caused by premature discharge or inadequate postacute care that cannot be measured using claims data. Relevant patient‐reported outcomes include care experiences, functional recovery, and appropriateness of discharge planning. Consequently, evaluations of CMS’ most recent bundled payment initiatives (eg, BPCI, Comprehensive Care for Joint Replacement (CJR) Model, Oncology Care Model (OCM)) include large‐scale patient surveys to measure quality from the patients’ perspectives. Little research exists to‐date, however, that focuses on patient‐reported outcomes following bundled payment initiatives.12 This study compares patient‐reported functional outcomes and care experiences for Medicare beneficiaries whose care was provided and overseen by Model 2 ACH participants with outcomes and experiences of a similar group of beneficiaries who received care from nonparticipating ACHs. Model 2 ACH episodes of care began with an inpatient stay and lasted for 30, 60, or 90 days after hospital discharge. We focused on these episodes as they comprised more care episodes than the other combinations of Model and provider types. We measured the association of BPCI with patient‐reported outcomes across clinical groups initiated by Model 2 ACH participants. We also separately analyzed the five highest‐volume clinical groups, including major joint replacement of the lower extremity (MJRLE); chronic obstructive pulmonary disease, bronchitis, and asthma (COPD); congestive heart failure (CHF); simple pneumonia and respiratory infections (pneumonia); and sepsis, to assess the presence and degree of variation in outcomes across specific conditions. Together, these five clinical groups account for roughly two‐thirds of all Model 2 BPCI episodes in the ACH setting.

2. METHODS

2.1. Primary data collection

2.1.1. Survey instrument

The BPCI patient survey instrument contained 36 multiple‐choice, closed‐ended questions that covered five domains: functional status, overall mental and physical health, care experiences and discharge planning, overall satisfaction with recovery, and personal characteristics. The survey was designed to take 25 minutes to complete. The survey instrument included items adapted from validated survey instruments, including the CARE Tool,16 National Health Interview Survey,17 and Short Form 36 Health Survey.18 Three questions were also inspired by the Care Transitions Measure^®.19 New questions underwent cognitive testing with a small convenience sample of Medicare beneficiaries with recent hospital and PAC experience.

2.1.2. Survey sample

The patient survey used a stratified random sampling method with matched BPCI and comparison groups. Within each clinical group, we defined sampling cells as unique combinations of presence or absence of a major complication or comorbidity, beneficiary age category (<65, 65‐74, 75‐84, 85+), hospital size (above or below the median number of beds), and hospital academic affiliations. We drew a proportional random sample of BPCI and matched comparison beneficiaries within each cell. For each of nine survey waves, corresponding to episodes initiated between October 2014 and June 2017, we used claims submitted by BPCI and comparison hospitals during two consecutive one‐month periods (one month of claims was insufficient volume for many of the clinical groups) to identify patients in applicable groups. Claims were obtained from the Chronic Conditions Warehouse under contract with CMS. Surveys were mailed approximately 90 days after hospital discharge, and the patients in the two consecutive monthly samples received their surveys one month apart. This rapid sampling process was intended to minimize recall bias, but it limited the sample to patients whose claims were filed quickly, within one month after discharge. Although claims submitted within one month may not represent the entire Medicare population, due to hospital delays in submitting claims, this issue should affect BPCI and comparison samples equally, limiting the risk of bias in resulting estimates.

We sampled patients in the 25 clinical groups (of the 48 total in BPCI) projected to have at least 310 BPCI responses and 310 comparison responses across all nine survey waves combined, given expected response rates. This sample size supported a minimum detectable effect of 10 percentage points for a binary measure, with a mean of 50 percent. The 25 groups in our final sample (Appendix Table S1) represented 94 percent of all BPCI Model 2 ACH volume through June 2017, the last month of data collection.

2.1.3. Comparison group selection

Within each clinical group, we selected the comparison group in two steps. First, for all hospitals with an episode in our sampling universe, we identified the hospital's combination of size (above or below the median number of beds), academic affiliation, ownership type (for‐profit, nonprofit, and government/other), census region, and urban/rural location using the CMS Provider of Services file. To ensure similarity across hospital characteristics, we excluded comparison hospitals with a unique combination of characteristics that did not match at least one BPCI hospital. Second, we used coarsened exact matching to match comparison patients to BPCI patients within each sampling cell. We used a standardized mean difference of 0.25 to assess balance between the matched intervention and comparison groups.20

2.1.4. Exclusions

We excluded from the sample any beneficiary who had responded to our BPCI survey in a previous wave. CMS’ CJR model overlaps with BPCI and began on April 1, 2016, in certain regions of the country. We therefore excluded from the comparison group all MJRLE patients whose surgery was performed in CJR hospitals after that date.

2.1.5. Survey administration

We mailed each sampled beneficiary a paper survey, followed by a reminder postcard, and a second paper survey. We followed up with nonrespondents to complete the survey via telephone. Approximately ten percent of responses were collected via telephone.

2.2. Measures

The survey instrument asked about seven measures of physical function, which were adapted from the CARE Tool16 National Health Interview Survey,17 and Short Form 36 Health Survey.18 Respondents were asked to report their functional status for two time points. First, they were asked to recall their status the week before the hospitalization. They were then asked to report their current status at the time of the survey (which they received roughly 90 days after hospital discharge). The seven functional status measures included: (a) bathing/dressing/toileting/eating; (b) planning regular tasks; (c) moving using a mobility device; (d) walking without resting; (e) going up or down stairs; (f) the frequency with which physical or emotional health interferes with regular social activities; and (g) the frequency with which pain interferes with normal activities. For each functional status measure, we created two binary variables to measure change from the week before hospital admission to after the episode (a total of 14 outcome measures related to functional status). The first variable indicated improvement or maintenance of the highest functional status. This variable had a value of one if a patient moved to a better functional status level after the episode (eg, from “complete help needed” the week before hospital admission to “no help needed” after the episode) or if the patient recalled having the highest functional status in the week prior to hospitalization and remained in that status when completing the survey at the end of the episode (eg, “no help needed” both before hospitalization and after the episode). The variable had a value of zero otherwise. The second variable indicated decline, or maintenance of the worst functional status. The decline indicator had a value of one if the patient moved to a worse functional status group after the episode or if the patient recalled having the lowest functional status in the week prior to hospitalization and remained in that status when completing the survey.

We included maintenance of the highest or lowest functional status in these measures, because they represent the best or worst outcome a hospital could achieve for a patient presenting with the highest or lowest functional status. Rather than creating four total measures (each with different denominators), we opted to combine maintenance and improvement/decline of functional status to streamline the analysis. Analyzing binary change variables, rather than focusing on functional status at a single point in time, is consistent with CMS quality measures regarding functional status in the PAC setting.21, 22, 23

The instrument also asked nine questions about care experience and one regarding overall satisfaction with recovery. The following nine care experience measures were included: never received conflicting advice, always received appropriate care, staff always used preferred language, discharged at right time, medical staff considered preferences for services after discharge, understood care of self before going home, medication instructions provided after discharge, follow‐up care explained before going home, and able to manage health since returning home. The overall satisfaction measure reflected whether respondents were extremely or quite a bit satisfied with their overall recovery since leaving the hospital. From these categorical measures, we constructed binary measures indicating affirmative outcomes. Appendix Table S3 provides greater detail about the response items for each measure.

2.3. Analysis

2.3.1. Weighting and risk adjustment

We applied sampling weights and nonresponse weights before analyzing the survey data. Nonresponse weights equaled the inverse probability of response within a sampling cell. Sampling weights equaled the inverse of the proportion of episodes within a population cell selected into the sample. Comparison sampling weights were normalized to sum to the size of the BPCI sample. The final nonresponse‐adjusted weight equaled the product of the sampling weight and the nonresponse weight.

All outcomes were risk‐adjusted using logistic regression, controlling for potential differences between the BPCI and comparison samples that could remain despite sample matching and postweighting. We obtained information from Medicare enrollment and claims data on beneficiary‐level characteristics including demographics (age, gender, Medicare‐Medicaid dual eligibility); Hierarchical Condition Category (HCC) index;24 number of days spent in an inpatient hospital or skilled nursing facility during the 90 days prior to BPCI/comparison hospital admission; whether the beneficiary's hospital admission was directly from an institutional care setting; and MS‐DRG. The HCC index reflects the portion of the CMS‐HCC community risk score that corresponds to the HCCs present in the six months preceding hospital admission. We controlled for four additional clinical indicators mapped to ICD9/ICD10 codes that are not reflected in MS‐DRGs: hip fracture (for patients with MJRLE); hip fracture (for patients with a hip and femur episode besides MJRLE); and large vessel ischemic stroke and stroke due to intracerebral hemorrhage (for patients with stroke). We controlled for hospital size, academic affiliation, and ownership type from CMS Provider of Service data. Using survey responses, we further controlled for self‐reported prehospital functional status and whether a proxy respondent completed the survey. Controlling for prehospital functional status in models of change in functional status is consistent with the specification of CMS PAC assessment quality measures.21, 22, 23 We controlled for English as the respondent's primary language for the one care experience question asking about medical staff using the patient's preferred language. Lastly, we controlled for wave fixed effects.

2.3.2. Regression model

The primary analysis pooled all 25 available clinical groups using a fully interacted model that incorporated data from seven of the nine survey waves. We excluded the first two survey waves from this analysis, because MJRLE was the only clinical episode sampled at the clinical episode level for the first two waves. The regression coefficient for every risk‐adjusting variable was allowed to vary by clinical group, as shown in the equation below.

$$Y_{\mathit{ijkt}}={\delta }_k{\text{BPCI}}_{\mathit{ij}}\ast S_{\mathit{ijk}}+{\beta }_k{X}_{\mathit{ij}}\ast S_{\mathit{ijk}}+{\text{MS‐DRG}}_i+T_t+{\epsilon }_{\mathit{ijkt}}$$

Y _ijkt is the outcome of interest for individual i, treated by hospital j, in clinical group k, during time t. X refers to the risk‐adjustment variables, MS‐DRG designates individual indicators for each MS‐DRG, T are time (Wave) fixed effects, BPCI is an indicator for a beneficiary who was treated by a BPCI participating hospital, and S is a set of indicator dummies for each of the K clinical groups (K = 25). Since MS‐DRGs are unique to the 25 clinical groups, they do not need to be interacted and function as group fixed effects. The relationship between Y and X (indicated by β _k) is unique for each clinical group (eg, HCC index may affect changes in functional status differently for patients with sepsis than patients with elective MJRLE). δ _k indicates the difference between BPCI and comparison respondents in group k. Since the model is fully interacted, δ _k is identical to the estimates reported individually for each clinical group. The average difference between BPCI and comparison respondents across all Model 2 ACH episodes is equal to the sum of all K values of δ _k, with each δ _k weighted according to the proportion of BPCI episodes in clinical group k relative to the total number of BPCI episodes across all K clinical groups. That is, each clinical group's estimate is weighted according to the size of the clinical group relative to the entire underlying Model 2 ACH population represented by the survey. In addition to the pooled estimates, we estimated differences in outcomes individually for the five largest individual clinical groups, using identical risk‐adjustment models, without the need for group‐level interactions. All analyses included clustered standard errors at the hospital level.

2.3.3. Sensitivity analyses

To assess the robustness of our results to our choice of weighting and risk adjustment, we re‐estimated our pooled results using regression without weights, weights without regression, and using neither weights nor regression. We also estimated functional status models limited to improvement and decline, excluding observations that remained in the highest and lowest categories, respectively. Lastly, we estimated functional status models limited to functional status at the time of the survey, which also excluded prehospital functional status from risk adjustment.

3. RESULTS

We collected 29 193 BPCI and 29 913 comparison survey responses. The response rate was 46.8 percent among BPCI episodes and 47.8 percent among comparison episodes (P < 0.05). Despite the small significant difference in response rates, the two groups were well balanced on most beneficiary‐level and hospital‐level characteristics (Table 1). For‐profit hospital status was the only variable with a standardized mean difference greater than 0.25 between the two groups (17 percent in the intervention group and 5 percent in the comparison group).

Table 1.

Average beneficiary and hospital characteristics of survey respondents

Variable	BPCI respondents	Comparison respondents	Mean standardized differencea
Female, %	59.07	59.81	0.0149
Age < 65, %	11.13	11.17	0.0012
Age 65‐74, %	39.11	38.96	0.0030
Age 75‐84, %	30.67	30.42	0.0053
Age > 85, %	19.09	19.44	0.0089
Dually eligible for Medicare and Medicaid, %	14.41	14.64	0.0065
Hierarchical Condition Category (HCC) Index, mean (SD)	0.57 (0.89)	0.52 (0.83)	0.0532
Prior inpatient days, mean (SD)	0.77 (4.68)	0.50 (4.05)	0.0607
Prior Skilled Nursing Facility (SNF) Days, mean (SD)	0.97 (6.05)	0.79 (5.87)	0.0308
Admission from community, %	92.70	94.26	0.0631
Major complication or comorbidity, %	43.82	44.21	0.0078
Response by proxy, %	10.37	10.66	0.0096
Response by proxy unanswered, %	12.81	13.32	0.0153
Functional score 1, mean (SD)	1.87 (1.33)	1.87 (1.34)	0.0060
Functional score 2, mean (SD)	0.87 (1.34)	0.88 (1.36)	0.0056
Missing functional score, %	9.18	9.21	0.0023
Pain score, mean (SD)	0.70 (0.79)	0.71 (0.78)	0.0105
Missing pain score, %	4.40	4.30	0.0051
Hip fracture (major joint replacement of the lower extremity only), %	8.19	8.81	0.0222
Hip fracture (hip and femur only), %	82.29	85.34	0.1093
Large vessel ischemic stroke (stroke only), %	18.42	19.79	0.0347
Intracerebral hemorrhagic stroke (stroke only), %	9.60	7.32	0.0820
For‐profit hospital, %	17.11	4.92	0.3898
Not‐for‐profit hospital, %	67.78	74.55	0.1496
Government/other hospital ownership, %	15.11	20.53	0.1416
Academic medical center, %	51.35	51.64	0.0058
Hospital size exceeds median, %	88.02	88.05	0.0012

All means in table were weighted by sampling design and nonresponse. Standard deviations for continuous measures are reported in parentheses. The HCC index in our sample reflects the portion of the CMS‐HCC community risk score that corresponds to the HCCs present in the six months preceding hospital admission. HCC index values in our sample range from zero to 10.6 units. Prior inpatient days range from zero to 80 days. Prior SNF days range from zero to 90 days. Functional score 1 is the sum of functional status measures 1‐5 indicating “some help” needed (see Section 1). Functional score 2 is the sum of functional status measures 1‐5 indicating “complete help” needed. The two functional scores range from zero to five. Missing functional score indicates one of the input functional status measures had a missing response. Pain score indicates the sum of functional status measures 6‐7, indicating limitations “most” or “all” of the time. The pain score ranges from zero to two. Missing pain score indicates one of the missing input functional status measures had a missing response.

Abbreviations: BPCI, Bundled Payments for Care Improvement; SD, standard deviation.

^aThe mean standardized difference is calculated as the absolute difference between BPCI and comparison averages, divided by the pooled standard deviation of the outcome among both BPCI and comparison respondents.

Source: Analysis of beneficiary survey data for episodes that began in May 2015 through June 2017 for BPCI and comparison hospitals.

Respondents were generally healthier than nonrespondents (Table 2), although dual eligibility and hip fracture (among MJRLE patients) were the only variables with a standardized mean difference greater than 0.25 between respondents and nonrespondents. Differences between respondents and nonrespondents were similar across the BPCI and comparison groups.

Table 2.

Average beneficiary and hospital characteristics of survey respondents vs nonrespondents

Beneficiary and hospital characteristics	BPCI respondents			Comparison respondents			DID (P‐value)
	Respondents	Nonrespondents	Mean standardized differencea	Respondents	Nonrespondents	Mean standardized differencea
Female, %	57.10	58.33	−0.0249	57.03	57.87	−0.0169	−0.39 (0.5152)
Age < 65, %	9.74	14.92	−0.1567	9.70	15.06	−0.1624	0.19 (0.6574)
Age 65‐74, %	35.32	26.69	0.1872	35.39	26.54	0.1919	−0.22 (0.7204)
Age 75‐84, %	32.84	28.37	0.0973	32.93	28.18	0.1032	−0.28 (0.5975)
Age > 85, %	22.09	30.02	−0.1801	21.98	30.22	−0.1872	0.31 (0.6049)
Dually eligible, %	15.95	30.36	−0.3395	15.45	29.45	−0.3339	−0.42 (0.5878)
HCC Index, mean (SD)	0.65 (0.95)	0.91 (1.24)	−0.2277	0.60 (0.89)	0.84 (1.16)	−0.2267	−0.02 (0.1751)
Prior inpatient days, mean (SD)	0.92 (5.05)	2.35 (8.69)	−0.1974	0.61 (4.45)	1.43 (7.07)	−0.1367	−0.62 (0.0000)b
Prior SNF days, mean (SD)	1.17 (6.60)	3.19 (11.21)	−0.2150	0.94 (6.41)	2.47 (10.45)	−0.1736	−0.50 (0.0000)b
Admission from community, %	90.68	83.80	0.2049	92.77	87.50	0.1759	1.62 (0.0000)b
Major complication or comorbidity, %	53.70	64.35	−0.2168	53.64	64.28	−0.2165	−0.01 (0.9950)
Hip fracture (MJRLE only), %	6.39	20.43	−0.4763	7.17	21.62	−0.4735	0.42 (0.8419)
Hip fracture (hip and femur only), %	81.61	85.69	−0.1109	83.99	85.79	−0.0501	−2.29 (0.2629)
Large vessel ischemic stroke (stroke only), %	18.32	21.53	−0.0799	19.62	22.30	−0.0657	−0.53 (0.7547)
Intracerebral hemorrhagic stroke (stroke only), %	9.65	11.77	−0.0681	7.27	11.06	−0.1301	1.67 (0.1695)
For‐profit hospital, %	19.87	22.60	−0.0667	3.99	4.62	−0.0308	−2.10 (0.0700)
Not‐for‐profit hospital, %	66.42	62.88	0.0739	78.56	77.40	0.0282	2.37 (0.0859)
Government/other hospital ownership, %	13.71	14.52	−0.0231	17.45	17.99	−0.0142	−0.26 (0.7770)
Academic Medical Center, %	51.14	48.33	0.0562	51.53	50.02	0.0302	1.30 (0.3944)
Hospital size exceeds median, %	86.70	88.91	−0.0677	87.20	89.06	−0.0575	−0.35 (0.6630)

Nonrespondent refers to a beneficiary selected into the survey sample who did not respond to the survey. Reported means and estimated differences reflect unweighted outcomes. Standard deviations for continuous measures are reported in parentheses. The HCC index in our sample reflects the portion of the CMS‐HCC community risk score that corresponds to the HCCs present in the six months preceding hospital admission. HCC index values range from 0 to 11.67 units in our sample. Prior inpatient days and prior SNF days both range from zero to 90 days. DID indicates a difference in the differences between respondents and nonrespondent in the BPCI and comparison groups. P‐values for the DID are reported in parentheses.

Abbreviations: BPCI, Bundled Payments for Care Improvement; HCC, hierarchical condition category; MJRLE, major joint replacement of the lower extremity; SD, standard deviation; SNF, skilled nursing facility.

^aThe mean standardized difference is calculated as the absolute difference between respondent and nonrespondent averages, divided by the pooled standard deviation of the outcome among respondents and nonrespondents.

^b P < 0.05.

Source: Analysis of beneficiary survey data for episodes that began in May 2015 through June 2017 for BPCI and comparison hospitals.

Table 3 shows the distribution of responses to functional status questions for the pre‐ and posthospital periods. Respondents with the highest prehospital status ranged from 18.8 percent for walking up or down stairs, to 64.8 percent for bathing, dressing, using the toilet, or eating. At the time of the survey, 23.7 percent of respondents remained in the lowest functional status for walking up or down stairs, while only 8.3 percent of respondents reported the lowest functional status in bathing, dressing, using the toilet, or eating.

Table 3.

Pre‐ and posthospitalization status and change from pre to post, for all respondents

Functional status items and response options	Prehospital functional status	Posthospital functional status	Change from pre to post (based on prehospital status—column 1)
			Improved	Declined	Maintained
Help needed with bathing, dressing, using the toilet, or eating
N	55 428	55 649	7040	6744	40 134
No help, %	64.8	70.8	0.0	7.6	57.7
Some help, %	27.6	20.9	12.1	2.8	12.4
Complete help, %	7.6	8.3	3.1	0.0	4.3
Help needed with planning regular tasks, such as shopping or remembering to take medication
N	56 071	55 772	6107	9124	39 469
No help, %	58.1	58.7	0.0	10.3	48.2
Some help, %	29.8	27.8	9.7	3.8	16.2
Complete help, %	12.1	13.5	3.5	0.0	8.3
Use of a mobility device such as a wheelchair, scooter, walker, or cane
N	56 147	55 632	3472	9821	41 446
Never use, %	49.3	46.3	0.0	11.9	37.7
Sometimes use, %	24.9	25.4	6.5	5.2	13.2
Always use, %	25.8	28.3	6.0	0.0	19.5
Ability to walk by yourself without resting
N	55 683	55 293	10 234	9210	34 728
Walk several blocks by myself without resting, %	25.1	33.9	0.0	6.0	19.3
Walk one block by myself without resting, %	21.6	20.3	8.5	4.2	9.2
Walk from one room to another by myself without resting, %	34.2	27.0	11.7	4.6	18.0
Unable to walk by myself without resting, %	19.1	18.9	6.8	0.0	11.9
Difficulty walking up or down 12 stairs
N	54 571	54 429	10 588	7450	34 849
No difficulty walking up or down 12 stairs, %	18.8	28.8	0.0	5.1	13.8
Some difficulty walking up or down 12 stairs, %	30.7	35.1	9.5	3.9	17.5
A lot difficulty walking up or down 12 stairs, %	24.5	12.4	15.4	2.5	6.7
I was not able to walk up or down 12 stairs, %	26.0	23.7	7.1	0.0	18.4
Physical health or emotional problems interfered with social activities
N	55 411	55 404	15 765	10 541	27 622
None of the time, %	31.1	42.6	0.0	7.4	23.9
A little of the time, %	14.8	17.0	6.5	3.8	4.6
Some of the time, %	24.2	20.0	12.5	3.3	8.5
Most of the time, %	17.1	11.3	10.4	1.6	5.0
All of the time, %	12.7	9.2	7.7	0.0	4.9
Pain interfered with normal activities
N	55 496	55 557	17 779	9184	27 229
None of the time, %	19.3	29.7	0.0	5.4	13.8
A little of the time, %	13.9	25.4	4.3	3.4	6.1
Some of the time, %	24.2	24.6	11.9	2.7	9.6
Most of the time, %	24.5	12.4	17.2	1.6	5.9
All of the time, %	18.1	7.9	13.0	0.0	5.2

All percentages in table were weighted by sampling design and nonresponse. Percentages of change from pre to post (improved, declined, and maintained) sum to 100% across all response options, not within a single row. Since change measures required both pre‐ and posthospital functional status, the Ns across the improved, declined, and maintenance columns do not sum to the same values reported in the prehospital or posthospital columns.

Abbreviation: BPCI, Bundled Payments for Care Improvement.

Source: Analysis of beneficiary survey data for episodes that began in May 2015 through June 2017 for BPCI and comparison hospitals.

Although both groups reported mostly positive care experience outcomes, a slightly smaller proportion of beneficiaries in BPCI reported having the most favorable care experiences, relative to the comparison group (Table 4). Point estimates were negative and statistically significant (P < 0.05) for six out of nine care experience measures. BPCI respondents were more likely than comparison respondents to report receiving conflicting medical advice (−2.26 percentage points: 72.06 BPCI vs 74.32 comparison); less likely to indicate that their level of care was always appropriate (−2.00 percentage points: 62.31 BPCI vs 64.30 comparison); less likely to say they were discharged at the right time (−1.53 percentage points: 88.81 BPCI vs 90.34 comparison); less likely to agree that medical staff took their preferences into account in deciding what health services they should have after leaving the hospital (−0.65 percentage points: 93.05 BPCI vs 93.69 comparison); less likely to agree that they had a good understanding of how to take care of themselves before going home (−0.91 percentage points: 94.71 BPCI vs 95.62 comparison); and less likely to agree that medical staff clearly explained how to take their medications before going home (−0.90 percentage points: 93.90 BPCI vs 94.80 comparison). BPCI respondents were also 1.90 percentage points less likely to indicate that they were “quite a bit” or “extremely” satisfied with their overall recovery since leaving the hospital than were comparison respondents (70.31 BPCI vs 72.21 comparison; P < 0.05). Despite these differences, respondents did not report a large or significant difference in whether they felt able to manage their health care needs after returning home.

Table 4.

Differences in survey‐based quality outcomes between BPCI and comparison respondents, aggregate Model 2 Acute Care Hospitals, May 2015‐June 2017

Survey measure	BPCI rate, %	Comparison rate, %	Difference, PP
Changes in functional status
Improvement or maintenance of highest status in bathing, dressing, using toilet, or eating	72.94	72.78	0.15
Decline or remained in lowest status in bathing, dressing, using toilet, or eating	14.60	14.87	−0.27
Improvement or maintenance of highest status in planning regular tasks	61.15	61.58	−0.43
Decline or remained in lowest status in planning regular tasks	22.54	22.39	0.15
Improvement or maintenance of highest status in use of a mobility device (less likely to use)	49.79	50.53	−0.74
Decline or remained in lowest status in use of a mobility device (more likely to use)	37.09	36.20	0.89
Improvement or maintenance of highest status in walking without rest	46.56	45.72	0.84
Decline or remained in lowest status in walking without rest	26.56	26.85	−0.29
Improvement or maintenance of highest status in using stairs	45.88	45.72	0.16
Decline or remained in lowest status in using stairs	30.06	29.81	0.25
Physical/emotional problems limit social activities less frequently, or maintained lowest frequency	60.74	60.99	−0.24
Physical/emotional problems limit social activities more frequently, or maintained highest frequency	21.18	20.78	0.39
Pain limits regular activities less frequently, or maintained lowest frequency	60.41	59.86	0.55
Pain limits regular activities more frequently, or maintained highest frequency	18.08	18.41	−0.33
Care experience
Never received conflicting medical advice	72.06	74.32	−2.26b
Services always appropriate for level of care patient needed	62.31	64.30	−2.00b
Medical staff always spoke in patient's preferred language	92.97	93.56	−0.59
Agree that patient was discharged at the right time	88.81	90.34	−1.53b
Agree that medical staff took patient's preferences into account in deciding postdischarge health care services	93.05	93.69	−0.65a
Agree that patient had good understanding of how to take care of self before going home	94.71	95.62	−0.91b
Agree that medical staff clearly explained how to take medications before going home	93.90	94.80	−0.90b
Agree that medical staff clearly explained what follow‐up appointments would be needed before patient went home	94.58	95.05	−0.46
Agree that patient had been able to manage health needs since returning home	96.38	96.16	0.22
Overall satisfaction
Extremely or quite a bit satisfied with overall satisfaction with recovery since leaving the hospital	70.31	72.21	−1.90b

Estimated differences between the BPCI and comparison group are risk‐adjusted using logistic regression and reported in percentage point (PP) terms. All regressions are weighted for sampling and nonresponse. Regressions control for sex; age; dual eligibility for Medicare and Medicaid; hierarchical condition category index; inpatient and skilled nursing facility days prior to admission; admission from community; major complication or comorbidity; hip fracture (for joint replacement or hip and femur episodes); type of stroke (for stroke episodes); self‐reported prehospital functional status; response by proxy; Medicare Severity Diagnosis Related Group (MS‐DRG); hospital ownership type, academic affiliation, and size; and survey wave fixed effects.

Abbreviation: BPCI, Bundled Payments for Care Improvement.

^a P < 0.05.

^b P < 0.001.

Source: Analysis of beneficiary survey data for episodes that began in May 2015 through June 2017 for BPCI and comparison hospitals.

Results in the care experience and satisfaction domains for the five largest clinical groups are generally consistent with aggregate results, in magnitude and direction, with some variation among the groups. BPCI respondents in all five clinical groups were less likely than comparison respondents to report never receiving conflicting medical advice and to agree that the level of care they received was always appropriate. Differences in whether respondents felt able to manage their health care needs after returning home were small and insignificant across all five groups. BPCI respondents in all five of the largest clinical groups were also less likely than comparison respondents to report the highest levels of overall satisfaction with recovery, although none of the differences were statistically significant. These results suggest that most differences, at the aggregate level, were driven by consistent impacts for clinical groups, rather than differences in one or two clinical groups.

On the other hand, differences in the proportion of respondents reporting that medical staff took their preferences into account were of various directions and magnitudes across clinical episodes—none statistically significant—even though there was a significant (albeit small) difference in this measure at the aggregate level. Likewise, differences in whether respondents reported having a good understanding of how to take care of themselves before going home, or that medical staff clearly explained how to take medications before going home, were inconsistent in magnitude and direction across groups. Aggregate differences in these measures were primarily driven by differences among patients with sepsis or pneumonia.

Although BPCI beneficiaries reported a negative, statistically significant association with care experience and overall satisfaction with recovery, across a range of measures, differences between BPCI and comparison respondents at the aggregate level were relatively small (ranging from −0.7 to −2.3 percentage points). In addition, despite these differences in care experiences, there were no systematic differences in functional status outcomes. Aggregate differences in the rates of improvement and decline were not statistically significant for any of seven functional status measures, and point estimates were small and in both directions, with no consistent pattern. Among the five clinical groups, we found only two statistically significant differences (Table 5). BPCI respondents with MJRLE were 1.54 percentage points less likely to report a decline in bathing, toileting, or eating than comparison respondents (BPCI 6.17 vs 7.72 comparison; P < 0.05). Conversely, BPCI respondents with COPD were 2.22 percentage points more likely than comparison respondents to report a decline in bathing, toileting, or eating (BPCI 14.88 vs 12.66 comparison; P < 0.05). Differences for this measure in the other three clinical groups were small and insignificant, and differences for the other functional status measures in the MJRLE and COPD groups were insignificant and did not follow any pattern of favoring the BPCI or comparison group. Therefore, the insignificant findings at the aggregate level do not appear to mask significant impacts in the largest clinical groups.

Table 5.

Differences in survey‐based quality outcomes between BPCI and comparison respondents, five largest clinical episodes in Model 2 Acute Care Hospitals, October 2014‐June 2017

Survey measure	MJRLE	CHF	Sepsis	Pneumonia	COPD
Changes in functional status
Improvement or maintenance of highest status in bathing, dressing, using toilet, or eating	1.34	−0.63	−0.56	−0.13	−0.74
Decline or remained in lowest status in bathing, dressing, using toilet, or eating	−1.54a	0.37	−0.45	−0.35	2.22a
Improvement or maintenance of highest status in planning regular tasks	−0.37	1.02	1.80	−2.55	−1.95
Decline or remained in lowest status in planning regular tasks	0.16	−2.57	−1.24	0.82	1.90
Improvement or maintenance of highest status in use of a mobility device (less likely to use)	−2.32	0.01	−0.38	0.44	1.01
Decline or remained in lowest status in use of a mobility device (more likely to use)	2.03	1.02	1.44	1.08	−0.18
Improvement or maintenance of highest status in walking without rest	1.84	−0.68	0.85	2.24	0.28
Decline or remained in lowest status in walking without rest	−0.49	0.80	−0.16	−0.27	−1.19
Improvement or maintenance of highest status in using stairs	−0.13	0.98	0.45	0.69	−0.43
Decline or remained in lowest status in using stairs	1.41	−1.93	−0.17	0.17	0.63
Physical/emotional problems limit social activities less frequently, or maintained lowest frequency	−0.25	−0.42	−0.39	−0.43	0.40
Physical/emotional problems limit social activities more frequently, or maintained highest frequency	0.16	0.16	1.43	0.11	−0.08
Pain limits regular activities less frequently, or maintained lowest frequency	0.59	1.28	0.08	−0.23	1.93
Pain limits regular activities more frequently, or maintained highest frequency	−0.17	0.01	−1.15	0.18	−2.78
Care experience
Never received conflicting medical advice	−2.93a	−1.14	−3.40a	−3.25a	−3.15a
Services always appropriate for level of care patient needed	−2.14	−1.08	−2.73	−5.12a	−2.43
Medical staff always spoke in patient's preferred language	0.54	0.06	−2.87a	−1.44	−0.74
Agree that patient was discharged at the right time	−2.13a	−2.10	−0.74	−1.60	0.15
Agree that medical staff took patient's preferences into account in deciding postdischarge health care services	−0.57	−0.25	0.69	−1.30	0.10
Agree that patient had good understanding of how to take care of self before going home	−0.79	−0.80	−2.08a	−2.13a	0.45
Agree that medical staff clearly explained how to take medications before going home	−0.73	0.06	−2.08a	−2.28a	1.22
Agree that medical staff clearly explained what follow‐up appointments would be needed before patient went home	−0.43	−0.23	−0.03	−2.55a	1.35
Agree that patient had been able to manage health needs since returning home	0.56	0.59	0.27	−0.15	0.95
Overall satisfaction
Extremely or quite a bit satisfied with overall satisfaction with recovery since leaving the hospital	−1.96	−2.26	−1.09	−0.20	−2.15

Estimated differences between the BPCI and comparison group are risk‐adjusted using logistic regression and reported in percentage point terms. All regressions are weighted for sampling and nonresponse. Regressions control for sex; age; dual eligibility for Medicare and Medicaid; hierarchical condition category index; inpatient and skilled nursing facility days prior to admission; admission from community; major complication or comorbidity; hip fracture (for joint replacement or hip and femur episodes); type of stroke (for stroke episodes); self‐reported prehospital functional status; response by proxy; Medicare Severity Diagnosis Related Group (MS‐DRG); hospital ownership type, academic affiliation, and size; and survey wave fixed effects.

Abbreviations: BPCI, Bundled Payments for Care Improvement; CHF, congestive heart failure; COPD, chronic obstructive pulmonary disease; MJRLE, major joint replacement of the lower extremity; Pneumonia, simple pneumonia and respiratory infections.

^a P < 0.05.

P < 0.001.

Source: Analysis of beneficiary survey data for episodes that began in May 2015 through June 2017 for BPCI and comparison hospitals, with the exception of MJRLE, which was collected for episodes starting in October 2014.

Results of sensitivity analyses are reported in the electronic appendix. Our key findings were robust to the exclusion of survey weights and regression adjustment. Functional status results were also robust to different definitions of binary change measures and exclusion of prehospital functional status measures. Lastly, our conclusions were robust to adjustment for multiple comparisons.

4. DISCUSSION

This study evaluated patient‐reported experiences with BPCI, including nearly 30 000 responses from beneficiaries treated by hospitals participating in the BPCI initiative and 30 000 responses from a matched comparison group. BPCI was associated with a slightly lower probability that patients reported the most favorable care experiences. Since the measures of care experience used in this study are binary, our results indicate meaningful differences in care experience associated with BPCI, which affect a small proportion of patients, rather than small, trivial differences that affected all patients. However, BPCI respondents reported nearly identical changes in functional status from before to after their episodes, suggesting that meaningful differences in care experience did not correspond with clinically significant differences in functional status. Any changes hospitals implemented to reduce costs did not appear to either improve or jeopardize functional recovery. These findings were consistent across the five largest clinical groups. In particular, MJRLE procedures have considerable potential to influence functional status in a 90‐day timeframe, on measures such as walking, using stairs, and pain interfering with regular activities, and yet no consistent pattern of positive or negative differences emerged between BPCI and comparison respondents on these mobility measures.

Relative to fee‐for‐service Medicare, which reimburses providers more when more care is provided, financial incentives under BPCI could encourage care stinting, potentially leading to worse quality of care8 and functional outcomes. Both BPCI and comparison beneficiaries reported positive care experiences. Across all clinical groups, over 95 percent of beneficiaries felt that they and their caregivers were able to manage their health needs after returning home, nearly 90 percent indicated that they were discharged at the right time, and over 70 percent were extremely or quite a bit satisfied with their overall recovery. Despite these positive findings, survey respondents treated at BPCI hospitals were slightly less likely to report positive care experiences and overall satisfaction than were respondents treated at comparison hospitals, with statistically significant differences ranging from 0.7 to 2.3 percentage points. These differences were generally similar in magnitude across the five largest clinical groups, suggesting that results were not driven by one or two high‐volume groups. These findings refute the concern that bundled payments may result in markedly worse quality of care or functional outcomes, from the patient perspective.

Patients treated by BPCI hospitals were more likely to be discharged to home rather than to institutional settings and were discharged from institutional settings more quickly.25 Survey results indicate that BPCI patients’ preferences about discharge destination and timing were not taken into account as often as were those of patients treated at comparison hospitals. This suggests that hospitals could focus on improving communication with beneficiaries and involving them in shared decision making, especially during discharge planning.26, 27, 28 These results may also suggest that savings attributable to BPCI were primarily achieved through mechanisms other than improved care coordination (ie, reduced PAC intensity played a more prominent role). Care models such as the CJR and OCM models have begun to incentivize positive care experience by linking payments to submission of patient‐reported outcomes or performance on patient‐reported outcomes. It remains to be seen whether adjusting payments under new care models based on tracking patient‐reported outcomes may help encourage participating providers to focus on maintaining or improving care experiences while streamlining care.

Although respondents were generally healthier than nonrespondents, this does not affect the internal validity of our estimates because the analytic sample was well balanced between the BPCI and comparison groups. The comparability of differences between respondents and nonrespondents among the BPCI and comparison groups further bolsters the internal validity of our findings since the potential for nonresponse bias is similar between the two groups. Moreover, our results are robust to removal of both weights and regression adjustment.

This study has limitations. First, since BPCI is a voluntary program, these results might not be generalizable to all Medicare fee‐for‐service beneficiaries treated by all hospitals if bundled payments were mandatory nationwide.29, 30 These findings also may not generalize to all care settings, as the current sample is limited to Model 2 episodes initiated by acute care hospitals; our analysis did not include episodes initiated by PGPs, which accounted for roughly 43 percent of Model 2 episodes. Second, roughly half of the sampled beneficiaries did not complete the survey. Response rates and beneficiary characteristics were similar for the BPCI and comparison groups, and weights and risk adjustment helped to account for nonresponse. However, respondents were healthier than nonrespondents on average, which may indicate underlying differences that cannot be accounted for by weights or regression. To the extent that nonrespondents differed from respondents on factors correlated with functional status or care experience, we cannot be certain that these findings are generalizable to the entire BPCI and comparison populations. Generalization of our results to the full BPCI and comparison populations may also be limited by the one‐month claims runout used to define the sample, if a nonrandom set of hospitals (or those with a nonrandom subset of patients) was consistently slower to submit claims. Third, since data collection began after the start of the BPCI initiative, we cannot know whether patient‐reported outcomes at BPCI hospitals were better than, similar to, or worse than comparison hospitals prior to the initiative. It is possible that the proportion of patients with the most favorable care experiences was always smaller at the hospitals that volunteered for BPCI than at similar comparison hospitals, and so we cannot rule out improvement among BPCI hospitals. While our estimates should therefore be interpreted with caution, it is likely that some of the reported differences we measured were attributable to BPCI, and that our null findings in the functional status domain are not failing to capture any differences that BPCI might have caused.

BPCI was a broad initiative to alter the care and cost paradigm across a wide array of medical, surgical, acute, and chronic episodes of varying lengths spanning multiple care settings from inpatient to PAC and home. Participating hospitals and physician groups were free to test myriad strategies to reduce costs while maintaining quality. This sweeping initiative demanded careful assessment of impacts on patient experiences and recovery, but we identified no suitable, previously validated survey instrument that would be relevant for such a broad clinical population across all care settings covered by the episode. The instrument we designed met this need and may be a useful starting point for evaluating future initiatives that fundamentally reimagine the way health care is delivered and reimbursed.

5. CONCLUSION

Model 2 ACH BPCI and comparison survey respondents reported similar changes in functional status from before to after their care episodes. This should reduce concerns about the potential for BPCI incentives to result in harm to patient health. Further research is necessary, however, to investigate potential adverse patient outcomes among sicker or more vulnerable populations. While care experience was positive overall, and functional recovery was not different, a slightly smaller proportion of beneficiaries in BPCI reported having the most favorable care experiences, relative to the comparison group. Differences in care experience highlight the value of measuring patient‐reported outcomes as a component of quality of care when evaluating new payment and delivery models of care.

Supporting information

 

 

Trombley MJ, McClellan SR, Kahvecioglu DC, et al. Association of Medicare's Bundled Payments for Care Improvement initiative with patient‐reported outcomes. Health Serv Res. 2019;54:793–804. 10.1111/1475-6773.13159