Assessment of the Effect of Adjustment for Patient Characteristics on Hospital Readmission Rates

This observational study analyzes Medicare claims and US Census data to evaluate differences in hospital readmission rates associated with measurable patient characteristics not used by Medicare to adjust for risk of readmissions in pay-for-performance programs.

Key Points

Question

To what extent are differences in hospital readmission rates explained by measurable patient characteristics not used by Medicare to adjust for risk of readmissions in pay-for-performance programs?

Findings

This observational study of Medicare claims and US Census data found that adjustment for additional clinical and social variables reduced overall hospital variation in readmission rates by 9.6%, narrowed differences in rates between hospitals serving higher vs lower proportions of high-risk Medicare patients by 54%, and reduced expected penalties by 41% among the 10% of hospitals with the largest penalty reductions.

Meaning

Hospitals are penalized to some extent for serving sicker and poorer patients rather than for poorer quality of care, which highlights the need for improved risk adjustment in pay-for-performance programs.

Abstract

Importance

In several pay-for-performance programs, Medicare ties payments to readmission rates but accounts only for a limited set of patient characteristics—and no measures of social risk—when assessing performance of health care providers (clinicians, practices, hospitals, or other organizations). Debate continues over whether accounting for social risk would mitigate inappropriate penalties or would establish lower standards of care for disadvantaged patients if they are served by lower-quality providers.

Objectives

To assess changes in hospital performance on readmission rates after adjusting for additional clinical and social patient characteristics by using methods that distinguish the association between patient characteristics and readmission from between-hospital differences in quality.

Design, Setting, and Participants

Using Medicare claims for admissions in 2013 through 2014 and linked US Census data, we assessed several clinical and social characteristics of patients that are not currently used for risk adjustment in the Hospital Readmission Reduction Program. We compared hospital readmission rates with and without adjustment for these additional characteristics, using only the average within-hospital associations between patient characteristics and readmission as the basis for adjustment, thereby appropriately excluding hospitals’ distinct contributions to readmission from the adjustment.

Main Outcomes and Measures

All-cause readmission within 30 days of discharge.

Results

The study sample consisted of 1 169 014 index admissions among 1 003 664 unique Medicare beneficiaries (41.5% men; mean [SD] age, 79.9 [8.3] years) in 2215 hospitals. Compared with adjustment for patient characteristics currently implemented by Medicare, adjustment for the additional characteristics reduced overall variation in hospital readmission rates by 9.6%, changed rates upward or downward by 0.37 to 0.72 percentage points for the 10% of hospitals most affected by the additional adjustments (±30.3% to ±58.9% of the hospital-level standard deviation), and would be expected to reduce penalties (in relative terms) by 52%, 46%, and 41% for hospitals with the largest 1%, 5%, and 10% of penalty reductions, respectively. The additional adjustments reduced the mean difference in readmission rates between hospitals in the top and bottom quintiles of high-risk patients by 0.53 percentage points (95% CI, 0.50-0.55; P < .001), or 54% of the difference estimated with CMS adjustments alone. Both clinical and social characteristics contributed to these reductions, and these reductions were considerably greater for conditions targeted by the Hospital Readmission Reduction Program. Adjustment for social characteristics resulted in greater changes in rates of readmission or death than in rates of readmission alone.

Conclusions and Relevance

Hospitals serving higher-risk patients may be penalized substantially because of the patients they serve rather than their quality of care. Adjusting solely for within-hospital associations may allow adjustment for additional patient characteristics to mitigate unintended consequences of pay for performance without holding hospitals to different standards because of the patients they serve.

Introduction

Readmission rates are used by the Centers for Medicare & Medicaid Services (CMS) to assess quality of care and determine payment adjustments to health care providers (clinicians, practices, hospitals, or other organizations) in several Medicare pay-for-performance programs, including the Hospital Readmission Reduction Program (HRRP), the Merit-based Incentive Payment System, and the Medicare Shared Savings Program.^1,2,3 Although the purported goal of pay for performance is to improve quality of care, there is growing concern that it may ultimately result in worse quality for vulnerable groups by penalizing providers for serving higher-risk patients, independent of the quality of care provided.^{4,5,6,7,8,9,10} Underlying this concern is the fact that CMS adjusts readmissions for few patient characteristics.¹¹ In the HRRP, for example, CMS adjusts for patients’ age, sex, and diagnoses recorded in the 12 months of claims preceding an admission but not for other clinical factors, such as disability or patients’ full history of chronic conditions, nor for social factors such as poverty.^{4,12,13,14,15,16,17}

Because of limited risk adjustment, and evidence that hospitals serving higher-risk patients have received larger penalties,^1,18,19,20 some have proposed accounting for additional patient characteristics—including measures of social risk—in pay-for-performance programs.^{5,6,9,10,11,21,22,23} Toward this end, the 21st Century Cures Act²⁴ has directed CMS to begin comparing readmission rates among hospitals serving similar proportions of Medicare patients dually enrolled in Medicaid when calculating HRRP penalties.²⁵ However, this stratification approach (to begin in 2019) raises concerns that hospitals serving different patient populations will be held to different standards. If dually enrolled beneficiaries are disproportionately treated at lower-quality hospitals, the new adjustment would conceal between-hospital differences in quality that contribute to worse outcomes for socially vulnerable patients.^11,26,27,28 Similarly, using regression instead of stratification to adjust for the dually enrolled proportion of a hospital’s patient population would inappropriately adjust for some differences in quality between hospitals in this scenario, rather than only for the association of dual enrollment with readmission—an association that would be expected in both high- and low-quality hospitals.

Risk-adjustment approaches like these are problematic because they conflate patient- and provider-level factors associated with readmission, either explicitly through stratification or implicitly by adjusting for provider-level differences that may be correlated with quality.²⁶ Accordingly, some have argued that adjusting for social risk factors would establish a lower standard of care for providers serving socially disadvantaged groups,²⁷ but this is not a necessary consequence of risk adjustment.^3,26,29 Approaches that distinguish patient-level vulnerability from hospital-level factors can better support the goal of risk adjustment, which is to differentiate provider quality from performance differences that are due to differences in patients characteristics.^3,26,29

In this study, we compared hospitals’ readmission rates before and after adjusting for several sets of patient characteristics in addition to those CMS currently uses to adjust for risk of readmissions.¹⁶ We focused on clinical and social characteristics observable in Medicare claims and linked US Census data, acknowledging in our interpretation of results that these measures incompletely describe patient risk.^{4,23,26,30,31} We based the adjustments on the average within-hospital associations between patient characteristics and readmissions, thereby excluding hospitals’ distinct contributions to readmissions from our adjustments.^3,26,29 Using this approach, we quantified the impact of adjustment for additional patient characteristics on overall hospital variation in readmission performance, on performance scores among hospitals most affected by the adjustments, and on penalties that would be expected in a program structured like the HRRP.

Methods

Study Population

We analyzed Medicare admissions from 2013 through 2014 for a random 20% sample of beneficiaries. In each year, we identified beneficiaries who had at least 1 hospital admission and were continuously enrolled in the traditional fee-for-service program during the year of admission (while alive in the case of decedents) and in the preceding year. Consistent with CMS methods for assessing readmissions, we limited our sample to patients 65 years and older (eAppendix 1 in the Supplement).¹⁶ The study was approved by the Harvard Medical School Committee on Human Studies, and informed consent was waived because the analyses used deidentified secondary data, which involved no contact with patients.

For each beneficiary in each year, we included the first admission in the year that met criteria for an index admission and also conducted a sensitivity analysis in which we randomly selected 1 admission per beneficiary annually. We defined index admissions similarly to the HRRP but included admissions for any condition (not just those targeted by the HRRP), because the larger resulting sample enabled us to more reliably estimate systematic between-hospital variation in readmission rates, which is consistent with proposals to improve HRRP accuracy by including more index conditions.³² Similar all-condition readmission rates are publicly reported by CMS and are a quality measure in the Merit-based Incentive Payment System.³³ Although we included index admissions for all conditions, we analyzed 1 admission per beneficiary, rather than all available index admissions, so that the distribution of admissions per beneficiary in our sample would more closely resemble the distribution for a condition-specific index admission assessed by the HRRP. Because we sampled 1 index admission per patient annually, the readmissions rates we report are expectedly smaller than the rates reported by CMS.

We excluded from consideration as index admissions those that ended with transfer to another hospital, admissions in which the patient died in the hospital, and admissions occurring within 30 days of a hospital discharge for the same patient (ie, readmissions were not counted as index admissions). For consistency with the HRRP, which has focused largely on index admissions for unplanned reasons, we also excluded index admissions for planned procedures. To more reliably estimate between-hospital variation in readmission rates, we limited our analyses to index admissions occurring in hospitals with at least 200 admissions during 2013 through 2014 (eFigure 1 in the Supplement shows the derivation of the study sample).

Outcomes

For admissions meeting our inclusion criteria, we assessed whether the patient was readmitted to any acute care hospital within 30 days of the discharge date of the index admission, excluding readmissions for planned procedures.¹⁶ We also assessed a combined outcome of readmission or mortality within 30 days of discharge to capture potential effects of hospital quality on mortality, address the problems of competing risks and censoring present in analyses ignoring deaths, and explore whether social risk factors might predict death more strongly than readmission because they may reflect differences in demand for inpatient care (eg, income or supplemental insurance coverage).

Patient Covariates

Patient Characteristics Included in CMS Risk Adjustment

Following CMS specifications for the construction of hospital-wide readmission rates, for each index admission we used Medicare enrollment and claims data to determine the base set of characteristics used by CMS for risk adjustment: patients’ age and sex, the principal diagnosis (aggregated into clinical categories),³⁴ and the presence of 31 conditions recorded during the index admission or in the preceding 12 months (eAppendix 2 in the Supplement).¹⁶

Additional Clinical Characteristics

From the Chronic Conditions Data Warehouse,³⁵ which draws from Medicare claims since 1999 to measure beneficiaries’ cumulative chronic disease burden, we assessed the presence of 27 conditions reported prior to the index admission year. Using beneficiaries’ claims from the year prior to admission, we also constructed indicators for the condition categories included in the CMS Hierarchical Condition Categories risk score.³⁶ We used claims from the year of admission to determine whether beneficiaries were long-term nursing home residents.³⁷ Finally, we used enrollment data to determine whether beneficiaries were originally eligible for Medicare because of a qualifying disability and whether they had end-stage renal disease in the year of admission.

Social Characteristics

We used enrollment data from Medicare’s Master Beneficiary Summary File³⁸ to identify beneficiaries with dual enrollment in Medicaid (a marker of low income and assets). We categorized dual enrollment by disability status to differentiate beneficiaries who became eligible for Medicaid as a result of disability and low income from those who became eligible as a result of age and low income. We also used enrollment data to assess beneficiaries’ receipt of benefits from state Medicare Savings Programs, which provide partial Medicaid benefits to low-income Medicare beneficiaries, and receipt of the Medicare Part D low-income subsidy among beneficiaries who did not automatically receive the low-income subsidy as a result of Medicaid coverage (eAppendix 2 in the Supplement).^31,39

We included data from the American Community Survey⁴⁰ to assess the following area-level socioeconomic variables¹³: educational attainment, poverty rates, household income, and the proportion of residents living alone. We assessed these characteristics among individuals 65 years and older at 2 levels of geography: US Census tracts, which contain an average of 4000 residents each and approximate beneficiaries’ immediate neighborhoods; and zip code tabulation areas, with a mean population size of approximately 9400 people (eAppendix 2 in the Supplement).³¹

Statistical Analyses

We first fitted linear regression models predicting readmission as a function of hospital fixed effects, indicators for the month and year of admission, and different sets of patient characteristics: (1) base CMS variables alone, (2) base variables plus additional clinical variables, (3) base variable plus additional social variables, and (4) all patient variables (Box). By including hospital fixed effects in each set of models, we estimated the average within-hospital associations (ie, pooled across hospitals) between patient characteristics and readmission (eAppendix 3 in the Supplement). We used these average within-hospital associations to calculate each hospital’s predicted readmission rate based on only the observed characteristics of its patients, appropriately excluding the hospital’s distinct contribution to readmissions. For each patient-admission observation, we calculated a score by subtracting the patient’s predicted probability of readmission from an indicator of whether the patient was readmitted. The mean score for a hospital equals its adjusted readmission rate. We calculated 4 sets of scores, 1 for each set of adjusters. Results were similar when using logistic instead of linear regression for this part of the analysis (eAppendix 3 in the Supplement).

Box. Patient Characteristics Used to Adjust for Risk of Hospital Readmission Rates^a.

CMS Risk Adjustment

• Age and sex

• 31 Disease indicators reported on beneficiaries’ claims in the prior year^b

• Primary diagnosis of index admission^c

Additional Clinical Characteristics

• HCC indicators^d

• CCW conditions^e

• Disability was original reason for Medicare enrollment

• End-stage renal disease

• Long-term residence in a nursing home^f

Additional Social Characteristics

• Insurance characteristics^g

  • Dual Medicare and Medicaid enrollment^h

  • Recipients of a Medicare Savings Programⁱ

  • Recipients of the Part D low-income subsidy^j

  • No subsidies or prescription drug coverage^k

• Poverty rate, household income,^l educational attainment,^m and proportion of residents living alone in the beneficiary’s zip code and US Census tractⁿ

• Interactions among clinical and social variables^o

To model the relationships among the different scores at the hospital level, we fitted hierarchical regression models with hospital random effects to patient-admission–level scores. We used these models to estimate variances in hospital-level readmission rates adjusted for the different sets of characteristics and the covariances between the differently adjusted rates (eAppendix 3 in the Supplement). Importantly, this approach yielded estimates net of sampling error, thus eliminating the contribution of random annual fluctuations in readmission or case mix to assessments of the between-hospital variation in readmission rates. Such random fluctuations might alter annual hospital performance and penalties but without meaningful long-run consequences for policy or hospital finances. An approach that fails to account for the sampling error would exaggerate the systematic, policy-relevant effects of adjustments for patient characteristics.

Using the estimated variances and covariances for the differently adjusted rates, we simulated changes in hospital readmission rates that would be expected from adjustment for each additional set of characteristics and calculated the extent of changes among hospitals most affected by these adjustments (recognizing that some hospitals without the highest readmission rates may be among the most affected). From the relationship between publicly reported readmission rates and penalties, we then quantified the penalty changes that would be associated with the estimated changes in readmission rates (eAppendix 3 in the Supplement).

In a complementary analysis using the same approach to risk adjustment, we compared readmission rates among quintiles of hospitals categorized in 1 of 2 ways: by the hospital’s proportion of Medicare patients who were dually enrolled in Medicaid or by the hospital mean of a patient risk score calculated from all of the additional clinical and social characteristics listed in the Box (refer to eAppendix 2 in the Supplement for the accompanying description of methods). These comparisons quantified the impact of the additional adjustments on performance differences between hospitals that differed specifically in the additional characteristics we assessed or the 1 additional characteristic (dual enrollment) for which CMS plans to adjust through its stratification approach.²⁵

Supplementary Analyses

We performed 3 sets of supplementary analyses. First, we assessed whether the within-hospital association between patients’ social characteristics and readmission varied systematically with the average social risk of a hospital’s patients (eAppendix 4 in the Supplement). Such a relationship would complicate the interpretation and reporting of between-hospital differences in quality, as it suggests the possibility that between-hospital differences could vary for different patient subgroups. This would not necessarily make risk adjustment inappropriate and might even provide additional insight into mechanisms that underlie differences in hospital performance for different subgroups. Second, we limited our analyses to index admissions for conditions targeted by the HRRP. Third, we included dual enrollment status in the set of base CMS adjustments, given the planned adjustment for dual enrollment.

Results

Study Population and Association Between Patient Characteristics and Readmission

The study sample consisted of 1 169 014 index admissions among 1 003 664 unique Medicare beneficiaries in 2215 hospitals. During the study period, the mean unadjusted readmission rate was 11.9% and ranged from 8.9% to 14.8% for hospitals in the lowest to the highest quintiles of readmission rates (Table). The unadjusted standard deviation in readmission rates across hospitals (estimated net of sampling error) was 1.8%, and after standard CMS adjustments it was 1.2%.

Table. Patient Characteristics, Associated Readmission Risk, and Comparison Between Hospitals With High vs Low Readmission Rates^a.

Characteristic	Readmission Risk Associated With Characteristic (95% CI)b	Standard Deviation in Patient Characteristic Across Hospitalsc	Lowest Quintile Readmission Rate (Mean, 8.9%)d	Highest Quintile Readmission Rate (Mean, 14.8%)d	P Valuee
Patient Characteristicsf
Male, %	0.73 (0.61 to 0.84)	3.4	41.9	40.7	<.001
Age, y	0.04 (0.03 to 0.05)	1.4	80.1	79.7	<.001
CCW chronic conditions, No.	0.66 (0.64 to 0.67)	0.76	9.0	9.7	<.001
HCC scoreg	2.08 (2.04 to 2.11)	0.24	2.0	2.3	<.001
Disabled, %	1.94 (1.77 to 2.11)	5.0	13.7	16.8	<.001
End-stage renal disease, %	10.70 (10.37-11.04)	1.8	2.7	3.6	<.001
Long-term nursing home resident, %	1.58 (1.38 to 1.78)	5.0	8.6	11.0	<.001
Insurance characteristicsh
Enrolled in Medicaid, %	3.02 (2.86 to 3.18)	13.0	17.2	24.3	<.001
Enrolled in a Medicare Savings Program, %	1.00 (0.69 to 1.32)	3.9	3.4	4.5
Receives Part D low-income subsidy, %	1.21 (0.81 to 1.62)	1.3	2.1	2.4
No subsidies or prescription drug coverage, %	1.04 (0.87 to 1.22)	4.5	15.0	12.6
Area-Level Characteristics Linked to Beneficiariesi
Poverty rate among residents ≥65 y in ZCTA, %	4.24 (3.10 to 5.37)	4.4	8.9	11.0	<.001
Poverty rate among residents ≥65 y in US Census tract, %	0.70 (0.07 to 1.33)	4.9	10.5	11.5	.005
Household income among residents ≥65 y in ZCTA <$50 000, %	2.51 (1.96 to 3.06)	8.4	61.6	64.5	<.001
Household income among residents ≥65 y in US Census tract <$50 000, %	2.39 (2.00 to 2.79)	8.6	61.6	64.4	<.001
Education among residents ≥65 y in ZCTAi
Less than a high school education, %	1.87 (1.02 to 2.72)	8.2	18.3	23.3	<.001
College or more, %	−1.21 (−1.94 to −0.48)	7.9	23.6	19.9	<.001
Education among residents ≥65 y in US Census tractj
Less than a high school education, %	2.03 (1.40 to 2.66)	8.3	18.3	23.4	<.001
College or more, %	−1.32 (−1.87 to −0.76)	8.0	23.5	19.7	<.001
Residents ≥65 y living alone in ZCTA, %	1.88 (1.08 to 2.68)	3.6	27.3	28.1	.001
Residents ≥65 y living alone in US Census tract, %	0.05 (−0.42 to 0.53)	4.1	27.4	27.6	.41

Abbreviations: CWW, Chronic Conditions Data Warehouse; HCC, Hierarchical Condition Categories; NA, not applicable; ZCTA, zip code tabulation area.

^aAmong patients in a random 20% sample of fee-for-service Medicare beneficiaries with 1 or more index admissions in 2013 or 2014 who met study criteria (see eAppendix 1 in the Supplement).

^bAverage within-hospital association with readmission is reported as percentage-point increase in readmission rate per unit increase in characteristic. Within-hospital associations estimated by individually adding each characteristic (or group of characteristics for race/ethnicity and area-level education) to a linear regression predicting readmission as a function of the characteristic and hospital fixed effects. Coefficient estimates are reported in percentage points of the readmission rate for a 1-unit change in the outcome (or for a 100% change for characteristics expressed as proportions).

^cThe hospital-level standard deviation of patient- and area-level characteristics was estimated from a random effects model. Using the proportion of Medicare patients dually enrolled in Medicaid as an example, the proportion of dual enrollees in hospitals differed by ±13.0 percentage points in hospitals 1 standard deviation above and below the mean among all hospitals (18.8%).

^dMean value of given characteristic among hospitals in the lowest and highest quintiles of unadjusted readmission rates. To address sampling error that would cause some hospitals to have more high-risk patients than others in the 20% sample or in a given year, a random effects model was used to estimate unadjusted hospital-level readmission rates, which were then categorized into quintiles reflecting systematic differences in readmission rates that would be expected to persist, on average, over different samples or years.

^eP value for the difference in means or proportions between hospitals in the highest vs lowest 20% of unadjusted readmission rates.

^fMeans or proportions of beneficiary-level characteristics assessed from Medicare claims and enrollment data. See the Box footnotes for additional information about the characteristics.

^gHCC scores were constructed using Medicare enrollment and claims data from the prior year, with higher scores indicating higher predicted spending in the subsequent year.

^hBeneficiaries were categorized into mutually exclusive categories of insurance coverage. Receiving prescription drug coverage through the Medicare Part D program, an employer, or another source was omitted as the reference category. See the Box footnotes and eAppendix 2 in the Supplement for details of variable definitions.

ⁱCharacteristics of beneficiaries’ ZCTAs or US Census tracts are based on residential address data reported in Medicare enrollment files. Estimates of the average within-hospital associations reflect the expected change in the readmission rate associated with a 100 percentage-point change in the proportion of residents with the area-level characteristics shown.

^jThe omitted category is the proportion of residents 65 years and older with a high school education.

Patient characteristics varied substantially across hospitals (Table). In analyses adjusting for the admitting hospital, all of the patient-level indicators of greater clinical or social risk, and most of the area-level measures indicating greater social risk, were significantly associated with a greater likelihood of readmission (Table). For most characteristics we assessed, patients admitted to hospitals in the highest quintile of unadjusted readmission rates exhibited significantly greater clinical and social risk than patients admitted to hospitals with the lowest readmission rates (Table).

Changes in Hospital Readmission Rates and Expected Penalties Following Additional Adjustments

Compared with adjustment for the variables currently used by CMS, adjustment for the additional clinical and social characteristics reduced the variance in readmission rates across hospitals to 1.35 percentage points from 1.49 percentage points—a relative reduction of 9.6% (eTable 2 in the Supplement). As illustrated in Figure 1, after additional adjustments, readmission rates declined on average among hospitals initially above the national median (based on CMS adjustments), with the greatest reduction among hospitals starting in the highest decile, and increased on average for hospitals below the median, with the greatest increase among hospitals starting in the lowest decile.

Figure 1. Expected Changes in Readmission Rates After Adjusting for Additional Clinical and Social Characteristics of Patients.

This graph shows the distribution of expected performance changes among hospitals after adjusting for the additional clinical and social risk factors found in the Box. Hospitals are grouped on the y axis by decile of readmission performance adjusted for age, sex, and recent comorbidities (ie, standard Centers for Medicare & Medicaid Services [CMS] adjustments). On the x axis, changes are reported in percentage points of the readmission rate. The plotted distribution reflects 5000 draws from the empirical covariances of hospitals’ unadjusted readmission rates with readmission rates predicted from the base CMS variables, as well as the additional clinical and social characteristics listed in the Box. After additional adjustments in our analysis, approximately 6.0% of hospitals originally above the median would move below the median (plotted in dark blue), while a similar proportion of hospitals below the median would be expected to move above the median (plotted in orange). See eFigure 2 in the Supplement, which plots the distribution of expected changes in the rate of 30-day readmission or mortality following additional adjustments.

Among the 10% of hospitals most affected by the additional clinical and social adjustments, readmission rates changed upward or downward by 0.37 to 0.72 percentage points, which was equivalent to ±30.3% to ±58.9% of the hospital-level standard deviation under standard CMS adjustments (eFigure 3 in the Supplement). The additional adjustments would be expected to reduce HRRP penalties (assessed as a percentage of hospitals’ total inpatient Medicare revenue) by 1.2 percentage points for the 1% most affected hospitals (52% of their mean penalty of 2.29%), by 0.81 percentage points for the 5% most affected hospitals (46% of their mean penalty of 1.77%), and by 0.63 percentage points for the 10% most affected hospitals (41% of their mean penalty of 1.55%) (eTable 3 in the Supplement).

The additional clinical and social characteristics both contributed to these changes in performance scores and expected penalties but overlapped considerably in the variation in hospital readmission rates they predicted (eTable 2 in the Supplement). Adjustment for the social characteristics had a substantially greater impact on the composite outcome of readmission or death than on readmission alone (eTable 2 in the Supplement).

Performance Differences Between Hospitals Serving Higher vs Lower Shares of High-Risk Patients

Under current CMS methods for adjusting readmissions, the mean adjusted readmission rate was 0.98 percentage points higher (95% CI, 0.72-1.23 percentage points; P < .001) among hospitals in the highest quintile of patient readmission risk (predicted from the additional clinical and social characteristics) than among hospitals in the lowest quintile (Figure 2A). Adjusting for additional clinical characteristics narrowed this difference to 0.73 percentage points, and adjusting for both additional clinical and social characteristics narrowed it to 0.45 percentage points—a 0.53 percentage-point reduction (95% CI, 0.50-0.55 percentage points; P < .001), equivalent to 54% of the difference under base adjustments. The additional adjustments narrowed differences between hospitals in the highest and lowest quintiles of dually enrolled patients by 83% relative to the difference based on standard CMS adjustments (Figure 2B). Effects of the additional adjustments on between-quintile differences in rates of readmission or death were even greater (Figure 2C and D).

Figure 2. Differences in Outcomes Between Hospitals Serving Higher-Risk Medicare Beneficiaries and Those Serving Lower-Risk Patients, Before and After Adjustment for Additional Patient Characteristics.

These graphs display average risk-adjusted outcomes by quintile of hospitals before and after adjusting for clinical and social covariates found in the Box, in addition to base variables used by the Centers for Medicare & Medicaid Services (CMS) to adjust for risk of readmissions. The graphs display mean adjusted rates among all patients admitted to hospitals in each quintile. A and C, Hospitals are grouped into quintiles by patients’ clinical and social risk according to the additional characteristics examined. B and D, Hospitals are grouped into quintiles by the proportion of Medicare patients dually enrolled in Medicaid. The clinical and social risk of Medicare patients was assessed by constructing patient-level risk scores predicted by the average within-hospital associations between each outcome and the added clinical and social variables from the Box, controlling for base CMS variables and hospital fixed effects. Random effects models were used to estimate hospital-level averages of these risk scores or rates of dual enrollment in Medicaid, and the resulting estimates were used to group hospitals into quintiles (see eAppendix 3 in the Supplement for details).

Supplementary Analyses

We found no evidence that within-hospital differences between patients with higher vs lower social risk were systematically greater in hospitals that served more socially high-risk patients (eTable 4 in the Supplement). The effects of adjusting for the additional social and clinical variables were greater when index admissions were limited to those for conditions targeted by the HRRP (eTable 5 and eFigure 5 in the Supplement). Specifically, these additional adjustments reduced the hospital-level variance of readmissions for HRRP-targeted conditions by 16.6% and changed readmission rates upward or downward by 0.66 to 1.14 percentage points for the most affected 10% of hospitals.

In our hospital-wide readmission measure, dual enrollment explained only part of the effects of the additional adjustments. For example, after adding dual enrollment status to the base model, adjustment for the remaining additional social and clinical characteristics reduced the variance in readmission rates between hospitals by 6.8% (eTable 6 in the Supplement), changed the most affected 10% of hospitals' readmission rates upward or downward by 0.20 to 0.42 percentage points, and narrowed differences between hospitals in the highest vs lowest quintiles of patients risk by 36% (eFigure 6 in the Supplement). Finally, our results were substantively unchanged when we sampled a random index admission per patient in each year, rather than the patient’s first index admission of the year.

Discussion

We found substantial variation across hospitals in the prevalence of patients’ clinical and social risk factors for readmission that CMS does not use to adjust readmission rates. Despite the limited number and breadth of additional patient characteristics we could examine using Medicare and US Census data, adjusting for these characteristics reduced the variation in readmission rates across hospitals and had an appreciable effect on readmission rates and expected penalties for hospitals disproportionately serving clinically and socially vulnerable patients.^4,41,42 These findings demonstrate that the HRRP penalizes hospitals to some extent for serving poorer and sicker patients, adding to evidence that pay-for-performance programs with limited risk adjustment could exacerbate disparities by unjustifiably transferring resources away from providers disproportionately serving higher-risk patients and by establishing incentives for providers to avoid these patients.^{3,4,18,19,20,43}

Our study builds on prior studies of, and current policy for, risk adjustment in pay-for-performance programs in 2 important ways. First, we quantified the impact of adjustments on performance scores and expected penalties for the hospitals most affected by the adjustments. Prior studies of the HRRP that focused on overall changes in hospital performance rankings or penalty status may have missed salient financial consequences for an important minority of hospitals serving the highest-risk patients.^43,44,45 In programs like the HRRP, in which penalties increase proportionally for hospitals with higher readmission rates, risk adjustment can substantially reduce the penalties incurred by hospitals with high shares of high-risk patients without major changes in rankings or the proportion of hospitals receiving a penalty.

Second, we used methods to address the commonly voiced concern that adjusting for social risk factors could lower standards for providers with more socially vulnerable patients.^11,27,28 Starting in 2019, CMS plans to address the association between patients’ social risk and readmission by comparing readmission rates among hospitals serving similar proportions of Medicare patients dually enrolled in Medicaid.^24,25 If dually enrolled patients are served disproportionately by low-quality hospitals, CMS’s approach would set lower standards for hospitals serving those patients, weakening incentives to achieve outcomes for socially vulnerable groups that are equal to those achieved for similar groups by other, higher-quality hospitals. Unlike CMS’s approach, our methodology could flexibly accommodate adjustment for a broader set of patient characteristics without raising concerns about introducing disparate standards of care.

Our results support adjusting for additional patient characteristics to mitigate unwarranted penalties for hospitals serving higher-risk patients and suggest that social risk adjustment may be even more important when payments are tied to health outcomes such as mortality, as opposed to utilization-based measures. However, improved risk adjustment would likely be an incomplete solution, because patient populations differ on many relevant dimensions that are difficult to measure.^4,23 For example, our analysis of Medicare and US Census data could not assess the extent to which adjustment for functional limitations, cognition, household income, or social supports might further affect hospital performance and penalties.^4,23 Thus, even with better risk adjustment, pay-for-performance programs may continue to impose additional costs on hospitals serving higher-risk patients that are unrelated to hospital quality, either in the form of penalties or the higher costs of avoiding penalties.⁴⁶

Limitations

In addition to the limited number of patient characteristics we could assess, a key limitation of our study is that we could not assess the causes of within-hospital associations between social risk factors and readmission. These might include unmeasured differences in clinical risk factors, a hospital’s inability to cover the higher costs of achieving high performance for higher-risk patients, or culturally incompetent or even discriminatory behavior by hospital staff. Because within-hospital associations did not systematically vary with the average social risk of a hospital’s patient population, we would not expect our adjustments to reward any discriminatory behavior contributing to within-hospital differences. Addressing within-hospital disparities would require additional incentives or resources supporting high performance specifically for high-risk patients and is not achieved by risk adjustment of quality measures, whose specific function is to improve assessment of between-hospital differences in quality. Particularly given the growing evidence of pay-for-performance programs’ ineffectiveness in driving quality improvement,^47,48 policy makers could consider alternative improvement strategies that invest in the care of vulnerable groups to reduce within-provider disparities, rather than risk exacerbating them because of inadequate risk adjustment.

Conclusions

In conclusion, our study contributes to evidence of the unintended consequences of limited risk adjustment in pay-for-performance programs for providers serving clinically and socially higher-risk patients. Our results support policies to adjust readmission rates for a more comprehensive set of patient characteristics, including social risk factors, to minimize the potential for pay-for-performance programs rates to exacerbate health care disparities, as well as alternative strategies to improve quality and address disparities.

Supplement.

eAppendix 1. Derivation of analytic cohort

eFigure 1. Derivation of analytic cohort

eAppendix 2. Description of covariates

eAppendix 3. Statistical analyses

eTable 1. Within-hospital association between a unit change in patient characteristics and outcomes

eTable 2. Hospital-level standard deviation and correlation of risk-adjusted readmissions/readmission or mortality, comparing base CMS adjustments vs further adjustment for patients’ clinical and social characteristics

eFigure 2. Expected changes in the risk-adjusted rate of readmission or mortality after adjusting for additional clinical and social characteristics of patients

eFigure 3. Distribution of expected changes in hospital readmission rates following adjustment for additional patient characteristics

eFigure 4. Distribution of expected changes in the composite rate of readmission or mortality following adjustment for additional patient characteristics

eTable 3. Simulated change in HRRP penalties after adjusting for additional clinical and social risk factors for readmission

eAppendix 4. Supplementary analyses

eTable 4. Differences in predicted within-hospital associations between social factors and outcomes between hospitals in the top and bottom quintiles based on Medicare patients’ social risk

eTable 5. Hospital-level standard deviation and correlation of risk-adjusted readmissions following index admissions for HRRP-targeted conditions

eFigure 5. Distribution of expected changes in readmissions following admissions for HRRP-targeted conditions

eTable 6. Hospital-level standard deviation and correlation of risk-adjusted readmissions, comparing adjustment for standard CMS variables plus dual enrollment versus adjustments for additional clinical and social characteristics

eFigure 6. Risk-adjusted readmission rates for hospitals grouped into quintiles by Medicare patients’ readmission risk or dual enrollment in Medicaid (base model includes dual enrollment status)

eTable 7. Hospital-level scores, comparing adjustment for within-hospital vs within- and between-hospital associations between patient characteristics and outcomes