Regression to the Mean in the Medicare Hospital Readmissions Reduction Program

Sushant Joshi; Teryl Nuckols; José Escarce; Peter Huckfeldt; Ioana Popescu; Neeraj Sood

doi:10.1001/jamainternmed.2019.1004

. 2019 Jun 26;179(9):1167–1173. doi: 10.1001/jamainternmed.2019.1004

Regression to the Mean in the Medicare Hospital Readmissions Reduction Program

Sushant Joshi ^1,², Teryl Nuckols ³, José Escarce ⁴, Peter Huckfeldt ⁵, Ioana Popescu ⁴, Neeraj Sood ^1,^2,^✉

¹Sol Price School of Public Policy, Department of Health Policy and Management, University of Southern California, Los Angeles

²Schaeffer Center for Health Policy and Economics, University of Southern California, Los Angeles

³Division of General Internal Medicine, Department of Medicine, Cedars-Sinai Medical Center, Los Angeles, California

⁴Department of General Internal Medicine, University of California, Los Angeles

⁵School of Public Health, Division of Health Policy and Management, University of Minnesota, Minneapolis

Accepted for Publication: March 29, 2019.

^✉

Corresponding Author: Neeraj Sood, PhD, Sol Price School of Public Policy, Department of Health Policy and Management, University of Southern California, 635 Downey Way, Verna and Peter Dauterive Hall, Ste 512, Los Angeles, CA 90089 (nsood@healthpolicy.usc.edu).

Published Online: June 26, 2019. doi:10.1001/jamainternmed.2019.1004

Author Contributions: Mr Joshi had full access to all the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Joshi, Nuckols, Escarce, Huckfeldt, Sood.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Joshi, Nuckols, Huckfeldt, Popescu, Sood.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Joshi, Escarce, Huckfeldt, Sood.

Obtained funding: Nuckols, Huckfeldt, Sood.

Administrative, technical, or material support: Sood.

Supervision: Nuckols, Escarce, Sood.

Conflict of Interest Disclosures: Mr Joshi reported receiving grants from the Agency for Healthcare Research and Quality (AHRQ) during the conduct of the study. Dr Nuckols reported receiving grant R01 HS024284 from the AHRQ during the conduct of the study and grants from the National Institute on Drug Abuse, National Institute of Diabetes and Digestive and Kidney Diseases, Patient-Centered Outcomes Research Institute, National Institutes of Health (NIH)/National Center for Advancing Translational Sciences, National Heart, Lung and Blood Institute, AHRQ, NIH/National Institute on Aging, and American Association of Neuromuscular and Electrodiagnostic Medicine outside the submitted work. Dr Escarce reported receiving grant R01 HS024284 from the AHRQ during the conduct of the study. Dr Huckfeldt reported receiving grants from the AHRQ during the conduct of the study and funding outside of the submitted work from the Assistant Secretary of Planning and Evaluation (ASPE) in the Department of Health and Human Services, Centers for Medicare & Medicaid Services, the Florida Medical Malpractice Joint Underwriting Association Patient Safety Grant Program, and the NIH; serving as a paid consultant to the RAND Corporation on NIH-funded research and as a peer reviewer; and serving as a paid consultant to the Urban Institute on a project funded by ASPE. Dr Popescu reported receiving grants from the AHRQ during the conduct of the study. Dr. Sood reported receiving grants from the AHRQ during the conduct of the study; serving as a scientific advisor for Virta Health, a start-up aiming to reverse diabetes through medication management and lifestyle change; receiving compensation for serving as an expert for the American Medical Association advising on the competitive effects of the CVS-Aetna merger; receiving compensation for serving as an expert for a consortium of law firms representing cities and jurisdictions suing pharmaceutical firms and distributors for the opioid crisis; receiving compensation for serving as an expert for China Development Research Foundation (Government of China) and the Pharmaceutical Research and Manufacturers of America on improving policies for controlling drugs costs and improving access to prescription drugs in China; and receiving grants from NIH, National Institute for Health Care Management Foundation, and Patient-Centered Outcomes Research Institute outside the submitted work. No other disclosures were reported.

Previous Presentation: This article was presented at the American Society of Health Economists conference; June 26, 2019; Washington, DC.

^✉

Corresponding author.

PMCID: PMC6596330 PMID: 31242277

Key Points

Question

Could regression to the mean explain some of the decline in readmissions at hospitals initially classified as below-mean performers under the Medicare Hospital Readmissions Reduction Program (HRRP)?

Findings

In this analysis of change in hospital readmission rates for specific conditions among Medicare beneficiaries, strong evidence suggests that most declines in excess readmission experienced after implementation of the HRRP at poorly performing hospitals was explained by a statistical phenomenon called regression to the mean. Regression to the mean signifies that entities farther away from the mean in one period are likely to be recorded closer to the mean in subsequent periods, simply by chance.

Meaning

Random chance or luck rather than improvement in quality of care appears to be the primary driver of improvements in readmissions experienced at hospitals initially classified as below-mean performers under the HRRP.

This analysis quantifies the contribution of regression to the mean to declining readmission rates at hospitals initially penalized under the Medicare Hospital Readmissions Reduction Program.

Abstract

Importance

Excess 30-day readmissions have declined substantially in hospitals initially penalized for high readmission rates under the Medicare Hospital Readmissions Reduction Program (HRRP). Although a possible explanation is that the policy incentivized penalized hospitals to improve care processes, another is regression to the mean (RTM), a statistical phenomenon that predicts entities farther from the mean in one period are likely to fall closer to the mean in subsequent (or preceding) periods owing to random chance.

Objective

To quantify the contribution of RTM to declining readmission rates at hospitals initially penalized under the HRRP.

Design, Setting, and Participants

This study analyzed data from Medicare Provider and Analysis Review files to assess changes in readmissions going forward and backward in time at hospitals with high and low readmission rates during the measurement window for the first year of the HRRP (fiscal year [FY] 2013) and for a measurement window that predated the FY 2013 measurement window for the HRRP among hospitals participating in the HRRP. Hospital characteristics are based on the 2012 survey by the American Hospital Association. The analysis included fee-for-service Medicare beneficiaries 65 years or older with an index hospitalization for 1 of the 3 target conditions of heart failure, acute myocardial infarction, or pneumonia or chronic obstructive pulmonary disease and who were discharged alive from February 1, 2006, through June 30, 2014, with follow-up completed by July 30, 2014. Data were analyzed from January 23, 2018, through March 29, 2019.

Exposures

Hospital Readmission Reduction Program penalties.

Main Outcome and Measures

The excess readmission ratio (ERR), calculated as the ratio of a hospital’s readmissions to the readmissions that would be expected based on an average hospital with similar patients. Hospitals with ERRs of greater than 1.0 were penalized.

Results

A total of 3258 hospitals were included in the study. For the 3 target conditions, hospitals with ERRs of greater than 1.0 during the FY 2013 measurement window exhibited decreases in ERRs in the subsequent 3 years, whereas hospitals with ERRs of no greater than 1.0 exhibited increases. For example, for patients with heart failure, mean ERRs declined from 1.086 to 1.038 (−0.048; 95% CI, −0.053 to −0.043; P < .001) at hospitals with ERRs of greater than 1.0 and increased from 0.917 to 0.957 (0.040; 95% CI, 0.036-0.044; P < .001) at hospitals with ERRs of no greater than 1.0. The same results, with ERR changes of similar magnitude, were found when the analyses were repeated using an alternate measurement window that predated the HRRP and followed up hospitals for 3 years (for patients with heart failure, mean ERRs declined from 1.089 to 1.044 [−0.045; 95% CI, −0.050 to −0.040; P < .001] at hospitals with below-mean performance and increased from 0.915 to 0.948 [0.033; 95% CI, 0.029 to 0.037; P < .001] at hospitals with above-mean performance). By comparing actual changes in ERRs with expected changes due to RTM, 74.3% to 86.5% of the improvement in ERRs for penalized hospitals was explained by RTM.

Conclusions and Relevance

Most of the decline in readmission rates in hospitals with high rates during the measurement window for the first year of the HRRP appeared to be due to RTM. These findings seem to call into question the notion of an HRRP policy effect on readmissions.

Introduction

The Medicare Hospital Readmissions Reduction Program (HRRP), enacted under the Affordable Care Act in March 2010,¹ imposes financial penalties on hospitals with excess 30-day readmission rates. Excess readmission rates measure whether risk-adjusted admissions for a particular condition at a hospital during a preceding reference measurement window exceed the national mean. Beginning in fiscal year (FY) 2013, Medicare penalized hospitals with excess readmissions for heart failure (HF), acute myocardial infarction (AMI), and pneumonia during a reference measurement window from July 1, 2008, through June 30, 2011.² In particular, Medicare measured excess readmissions using the excess readmissions ratio (ERR), calculated as the ratio of a hospital’s predicted 30-day readmissions to the number that would be expected based on an average hospital with similar patients. Only hospitals with ERRs of greater than 1.0 for targeted conditions are penalized.

Risk-adjusted readmissions for targeted conditions started to decline in 2011, shortly after the announcement of the HRRP, and the reductions were larger at penalized than nonpenalized hospitals.^3,4 Some interpreted this evidence as suggesting that the HRRP’s financial penalties incentivized hospitals to improve transitions of care and implement other strategies that resulted in declines in readmissions.^3,4 However, most changes in excess readmissions at penalized hospitals may be explained by regression to the mean (RTM), a statistical phenomenon that occurs when an outcome for an entity is measured repeatedly and the outcome is a random variable.^{5,6,7,8,9,10,11} With repeated measurements over time, outcomes for entities with outcomes farther away from the mean in one period are likely to be recorded closer to the mean in subsequent or preceding periods simply by chance, because more extreme values have a lower probability of occurring than do values near the mean. Therefore, RTM has the potential to produce the appearance of improvement among initially low performers and deterioration among initially high performers. Given that hospitals’ excess readmissions involve randomness, hospitals with excess readmissions before the HRRP—that is, penalized hospitals—would be more likely to exhibit subsequent reductions in excess readmissions due to chance alone. Chance or random events might play an important role in determining excess readmissions because readmissions are not only a function of care processes and treatment provided to patients but also of random biological variation in treatment efficacy or adverse effects and random events such as poor outcomes due to severe weather or falls due to hazardous living conditions.

The objective of this analysis was to describe and quantify the role of RTM in explaining improvements in performance, relative to national means, at hospitals initially classified as below-mean performers under the HRRP. First, we tested whether hospitals with below-mean baseline performance during the initial HRRP measurement window experienced subsequent improvements in performance after announcement and implementation of the HRRP and, inversely, whether hospitals with above-mean performance experienced subsequent declines in performance after the announcement and implementation of the HRRP.

Next, we tested 2 hypotheses to examine whether RTM rather than HRRP policy effects played a major role in explaining the above trends in excess readmissions. First, if RTM played a major role, then hospitals with worse than mean baseline performance during an earlier alternate reference measurement window that predates the implementation of the HRRP would exhibit subsequent declines in ERRs. These declines would be qualitatively similar to those observed at hospitals with low performance after the implementation of the HRRP. Second, if RTM played a major role, then hospitals with worse than mean baseline performance during the HRRP would exhibit declines in ERRs going backward in time just as they do going forward in time. Finally, for hospitals with below-mean baseline performance, we used established methods to calculate the percentage of the subsequent improvements in excess readmissions that were attributable to RTM.

Methods

Population and Setting

The study was performed at the hospital level. We included fee-for-service Medicare beneficiaries 65 years or older with an index hospitalization for 1 of the 3 target conditions (HF, AMI, and pneumonia) or chronic obstructive pulmonary disease (COPD) that resulted in discharge alive from February 1, 2006, through June 30, 2014. We excluded index hospitalizations when the patient was admitted with the same condition during the prior 30 days. Study hospitals had at least 25 eligible index hospitalizations for a target condition in the excess readmission measurement period. The study was reviewed by the institutional review board (IRB) of the University of Southern California, Los Angeles, which deemed that our project meets the requirements outlined in 45 CFR 46.101(b)(4) and qualifies for exemption from IRB review. The project was also reviewed by the IRBs of all coauthors and was deemed exempt or approved with expedited review. Informed consent was not required for this study of deidentified patient data. This study met the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guidelines.

Data Sources

We used data from Medicare Provider and Analysis Review files from February 1, 2006, through June 30, 2014, to identify index hospitalizations and 30-day readmissions to July 30, 2014. Information on patient characteristics came from Master Beneficiary Summary files. Hospital characteristics were based on the 2012 Medicare Provider of Services files and 2012 American Hospital Association Annual Survey.

Measures

Following the HRRP methods, we calculated ERRs for each hospital and study period by using linear regression to estimate the predicted number of readmissions (based on patient characteristics and hospital-level random effects) and the expected number (based only on patient characteristics) and then dividing the predicted by the expected number. In these regression models, patient characteristics included age group (65-69, 70-74, 75-79, 80-84, and 85-89 years and a single group ≥90 years), sex, and condition-specific comorbidities documented during the index hospitalization. We used the ERR to measure performance because it reflected how Medicare assigned HRRP penalties and accounted for trends in mean readmission rates, whether owing to the policy or other factors.

Based on the reference-period ERRs, we stratified eligible hospitals into 2 groups reflecting below-mean (ERR>1.0) or above-mean (ERR≤1.0) performance. These group variables were the main independent variables in the analyses. The main outcome variable was the change in the ERR from the reference period to the subsequent or preceding period.

Statistical Analysis

Data were analyzed from January 23, 2018, through March 29, 2019. First, we summarized the characteristics of hospitals used in the analyses. Next, we conducted 3 sets of analyses, described below, with 3 analyses per set (HF, AMI, and pneumonia). In each analysis, we performed linear regression to examine whether the ERRs for a reference period predicted changes in ERRs from the reference period to a subsequent or a preceding period.

Analysis 1: Baseline Performance During the Initial HRRP Measurement Window

We examined changes in ERRs from the FY 2013 measurement window (July 1, 2008, through June 30, 2011) to the subsequent 3-year period (July 1, 2011, through June 30, 2014) after the announcement and implementation of the HRRP. We expected that for targeted conditions, hospitals with below-mean performance (ERR>1.0) would experience a decrease in ERRs, and hospitals with above-mean performance (ERR≤1.0) would experience an increase in ERRs. These effects are consistent with RTM and HRRP policy effects (Figure 1). In 1 alternate specification, we repeated this analysis by examining trends in readmissions for hospitals with ERRs of greater than 1.07 (75th percentile of the ERR distribution). Hospitals in the right tail of the ERR distribution had limited incentives to respond to the HRRP because the policy capped penalties; therefore, the hospitals would have to achieve substantial declines in readmissions to reduce the size of the penalties, and substantial declines might have been infeasible. We considered index hospitalizations for COPD in another alternate specification because the HRRP did not create incentives to lower COPD readmissions until FY 2015.

Figure 1. — In analysis 1, changes in excess readmissions rates are inversely associated with baseline excess readmission ratios (ERRs). In hospitals with worse (or better) than mean performance during the Hospital Readmissions Reduction Program (HRRP) measurement period for fiscal year (FY) 2013, ERRs declined (or increased) during the 3-year postmeasurement period. In analysis 2, hospitals with low performance during an alternate measurement period exhibited subsequent declines in ERRs. In hospitals with worse (or better) than mean performance during an alternate measurement period, declines (or increases) in ERRs occurred during the 3-year postmeasurement period (ie, during the FY 2013 measurement period). In analysis 3, hospitals with low performance during the HRRP measurement period for FY 2013 exhibited declines in ERRs when going backward in time. In hospitals with worse (or better) than mean performance during the HRRP measurement period for FY 2013, declines (or increases) in ERRs occurred in reverse chronological order during the alternate measurement period. ACA indicates Affordable Care Act.

Analysis 2: Alternate Measurement Period

We hypothesized that with RTM, hospitals with below-mean performance during a different measurement window predating the HRRP would also exhibit subsequent improvements despite the absence of policy incentives, and performance would deteriorate at hospitals with above-mean performance. To test this hypothesis, we examined changes in ERRs from an alternate measurement period that immediately preceded the FY 2013 measurement window (February 1, 2006, through June 30, 2008; data from before 2006 were not available) to the FY 2013 measurement window.

Analysis 3: Reverse Chronological Trends

Regression to the mean suggests that hospitals with below-mean performance during the FY 2013 measurement window would exhibit improvement going backward in time, just as they do going forward in time. Hospitals that are above-mean performers would exhibit deterioration going backward in time. To test this hypothesis, we examined changes in ERRs in reverse-chronological order from the FY 2013 measurement window to the preceding alternate measurement period (February 1, 2006, through June 30, 2008).

Quantifying Regression to the Mean

For hospitals that performed below the mean during the FY 2013 measurement window, we quantified the percentage of the subsequent improvement in performance that was due to RTM.^9,12 The formula, which is explained in detail in eMethods in the Supplement, captures the 3 phenomena that influence the size of the RTM effect and are easily understood intuitively. First, the expected RTM effect is greater when the outcome in the baseline period is more extreme (ie, farther from the mean of the distribution), because extreme outcomes rarely occur and consequently are very unlikely to be repeated. Second, other things being equal, the expected RTM effect is greater when the variance of the outcome in the baseline period is greater, because a greater variance means that the outcome distribution is more spread out. Consequently, any movement from the tail of the distribution at baseline to the mean in the postmeasurement period represents a greater difference in the outcome. Third, the RTM effect is smaller when the correlation in the outcome between the baseline and postmeasurement periods is higher, because a high correlation means that outcomes tend to be more persistent over time. We used methods described by Linden¹² to estimate the expected RTM effect. Two-tailed P < .05 indicated significance.

Results

Descriptive Analyses

Our analyses of eligible hospitals included 3116 for HF, 2196 for AMI, and 3233 for pneumonia. Medicare beneficiaries constituted about half of patients in these hospitals (Table 1). For each condition, more than 60% of hospitals were nonprofit, and more than 70% were located in metropolitan areas.

Table 1. Hospital Characteristics by Target’s Summary Statistics and by Conditions^a.

	Heart Failure	AMI	Pneumonia
No. of hospitals included in analysis	3116	2196	3233
No. of missing hospital characteristics	128	41	219
Mean No. of hospital beds per hospital	227.54	281.68	226.11
Inpatients with Medicare, %	51.01	51.13	50.79
Inpatients with Medicaid, %	19.66	19.22	19.60
Hospital type, No. (%) of hospitals
Nonprofit	1975 (63.4)	1511 (68.8)	2040 (63.1)
For-profit	647 (20.8)	439 (20.0)	665 (20.6)
Government	494 (15.9)	246 (11.2)	528 (16.3)
Teaching status, No. (%) of hospitals
Major	253 (8.1)	239 (10.9)	262 (8.1)
Minor	915 (29.4)	769 (35.0)	948 (29.3)
Nonteaching	1948 (62.5)	1187 (54.1)	2023 (62.6)
Location, No. (%) of hospitals
Metropolitan area	2205 (70.8)	1774 (80.8)	2273 (70.3)
Micropolitan area	612 (19.6)	360 (16.4)	638 (19.7)
Rural area	299 (9.6)	62 (2.8)	322 (10.0)

Open in a new tab

Abbreviation: AMI, acute myocardial infarction.

^{^a}

This includes all hospitals that are used to analyze the changes in excess readmissions rates. Individual hospitals may be listed for more than 1 target condition. Hospital characteristics are based on the 2012 survey by the American Hospital Association.

Changes in ERRs Between Contiguous Nonoverlapping Measurement Periods

Analysis 1: Baseline Performance During the Initial HRRP Measurement Window

For all 3 target conditions, hospitals with below-mean performance (ERR>1.0) in the FY 2013 measurement window exhibited improvements (decreases in ERRs) in the subsequent 3 years, whereas hospitals with above-mean (ERR≤1.0) performance exhibited deterioration (Figure 2). For HF, the mean ERRs declined from 1.086 to 1.038 (−0.048; 95% CI, −0.053 to −0.043; P < .001) at hospitals with below-mean performance and increased from 0.917 to 0.957 (0.040; 95% CI, 0.036 to 0.044; P < .001) at hospitals with above-mean performance. Similarly, for AMI, the mean ERRs declined from 1.106 to 1.043 (−0.063; 95% CI, −0.070 to −0.057; P < .001) at lower-performing hospitals and increased from 0.899 to 0.963 (0.063; 95% CI, 0.057 to 0.070; P < .001) at higher-performing hospitals. For pneumonia, the mean ERRs declined from 1.112 to 1.041 (−0.071; 95% CI, −0.077 to −0.066; P < .001) and increased from 0.898 to 0.955 (0.057; 95% CI, 0.052 to 0.062; P < .001), respectively. Results were qualitatively similar for hospitals with ERRs greater than 1.07 and for patients with COPD (see eTables 1 and 2 and eFigure 1 in the Supplement).

Figure 2. — Data from analysis 1 (described in Figure 1) are stratified by condition. MI indicates myocardial infarction.

Analysis 2: Alternate Measurement Period

For all 3 target conditions, performance improved at hospitals with below-mean performance during an alternate measurement period predating the HRRP and deteriorated at hospitals with above-mean performance (Figure 3). For HF, mean ERRs declined from 1.089 to 1.044 (−0.045; 95% CI, −0.050 to −0.040; P < .001) at hospitals with below-mean performance and increased from 0.915 to 0.948 (0.033; 95% CI, 0.029 to 0.037; P < .001) at hospitals with above-mean performance. Results were similar for AMI and pneumonia.

Figure 3. — Postmeasurement period is in reference to the fiscal year 2013 measurement period. Data from analysis 2 (described in Figure 1) are stratified by condition. MI indicates myocardial infarction.

Analysis 3: Reverse Chronological Trends

Results reveal an inverse association between performance during the FY 2013 measurement window and reverse chronological changes in performance from that window to the preceding period. For HF, hospitals with ERRs of greater than 1.0 saw mean ERRs decline from 1.086 to 1.051 (−0.035; 95% CI, −0.040 to −0.030; P < .001), whereas hospitals with mean ERRs equal to or below 1.0 exhibited mean increases from 0.918 to 0.951 (0.033; 95% CI, 0.029 to 0.038; P < .001). Results were similar for AMI and pneumonia (see eFigure 2 in the Supplement).

Quantifying Regression to the Mean

Among hospitals with below-mean baseline performance during the FY 2013 measurement window, 74.3% to 86.5% of the improvement observed from that window to the subsequent 3-year period was explained by RTM (Table 2). Similarly, 83.6% to 91.8% of the decline in performance among hospitals with above-mean baseline performance was explained by RTM.

Table 2. Changes in Excess Readmissions Ratios Explained by Regression to the Mean.

FY 2013 Measurement Period ERR	Condition^a
	Heart Failure (n = 2965)		AMI (n = 2064)		Pneumonia (n = 3013)
	Change in ERR (95% CI)	Change Due to RTM, % (95% CI)	Change in ERR (95% CI)	Change Due to RTM, % (95% CI)	Change in ERR (95% CI)	Change Due to RTM, % (95% CI)
>1.0	−0.048 (−0.053 to −0.043)^b	85.8 (80.57 to 91.02)	−0.063 (−0.070 to −0.057)^b	86.45 (80.25 to 92.65)	−0.071 (−0.077 to −0.066)^b	74.34 (70.12 to 78.55)
≤1.0	0.040 (0.036 to 0.044)^b	91.84 (85.80 to 97.88)	0.063 (0.057 to 0.070)^b	87.45 (80.51 to 94.39)	0.057 (0.052 to 0.062)^b	83.62 (78.22 to 89.03)

Open in a new tab

Abbreviations: AMI, acute myocardial infarction; ERR, excess readmissions ratio; FY, fiscal year; RTM, regression to the mean.

^{^a}

Change in ERR is calculated as the 3-year postmeasurement period minus the FY 2013 measurement period. The percentage due to RTM is calculated by dividing the RTM effect by the actual change in ERR multiplied by 100. The RTM effects were calculated using rtmci command in Stata with a cutoff of 1.0.¹²

^{^b}

P < .001.

Discussion

This analysis found that hospitals with below-mean performance in the measurement window for the first year of HRRP penalties experienced a decline of 4.8 to 7.1 percentage points in excess readmissions during the subsequent 3 years. However, we found strong evidence suggesting that most of this improvement was due to RTM rather than HRRP policy effects. The evidence supporting this conclusion includes the fact that we observed similar changes in excess readmissions at below-mean and above-mean performing hospitals when we defined performance during an alternative measurement period predating the HRRP and when we examined changes in performance going backward rather than forward in time. These observations held for all 3 target conditions, including HF, AMI, and pneumonia. Furthermore, readmissions for patients with COPD and for hospitals in the right tail of the ERR distribution, where there was no incentive to improve performance, show a similar trend across time. Finally, quantitative analyses indicated that at least three-fourths of the improvement observed at hospitals with below-mean performance during the FY 2013 measurement window was explained by RTM.

Regression to the mean was first conceived by Sir Francis Dalton in 1886 to explain why tall parents have children shorter than themselves and short parents have children taller than themselves.¹³ Despite its long history, RTM remains an overlooked statistical phenomenon. Observers often erroneously ascribe improvements among low performers to an intervention of interest when the true cause is RTM. Consistent with our findings of RTM, Press et al¹⁴ detected decreases in readmissions from 2009 (the year before the HRRP was announced) to 2011 (the year before it was implemented) at hospitals with initially high rates and increases at hospitals with initially low rates. Our results expand on these previous findings by examining a longer time frame, including after the HRRP was implemented, using multiple strategies to document the effects of RTM and quantifying the size of that effect.

The fact that RTM had such a large effect on trends in hospital performance has 3 important implications. First, it suggests that the HRRP explains at most a small portion of the more favorable trends in readmissions that occurred at penalized compared with nonpenalized hospitals. Hospitals that were penalized in FY 2013 experienced a financial shock due to excess readmissions, but our results suggest that this did not result in greater declines in excess readmissions. However, this does not mean that the HRRP did not lead to a decline in readmissions. Other researchers^15,16 have evaluated the HRRP using analytical strategies that are not biased by RTM and found that the policy was associated with a decline in readmissions. Other investigators^17,18 have suggested that the decline in risk-adjusted readmissions across all hospitals might have been inflated owing to nationwide changes in coding practices.

The second implication of our findings is that hospital performance and the assignment of penalties under the HRRP is more strongly influenced by chance. The results suggest that hospitals that were penalized based on their ERRs in the baseline period converged to the mean in the postmeasurement period. This possibility suggests that bad luck rather than poor-quality care explains the poor performance in the baseline period. Moreover, if hospitals have limited abilities to control their performance, they may be less likely to sustain efforts to improve transition-related care and reduce readmissions during the long term.

The third implication of our findings pertains to the ability to detect associations between trends in readmissions and mortality. Dharmarajan et al¹⁹ found hospital-level improvements in readmissions were only weakly associated with declines in mortality. Our findings suggest that chance may have played such a large role in changes in readmissions that the ability to detect a true association with mortality was limited, an example of regression dilution bias.⁵

Limitations

The results of our analysis should be viewed in light of its limitations. This study analyzes observational data on trends in readmissions at high- and low-performing hospitals after the implementation of the HRRP. Ideally, we would have preferred having data from a control group of hospitals that were unaffected by the HRRP; however, that was not possible given that the policy was implemented nationally. The study would also have been stronger if we had richer data on care processes at hospitals. With such data we may have been able to directly analyze changes in care processes at high- and low-performing hospitals and more definitively assess the extent to which RTM or changes in care processes explain trends in readmissions at these hospitals. In addition, the analysis focuses on low- and high-performing hospitals as a group. We cannot conclude whether declines in readmissions at individual hospitals were due to RTM, favorable changes in transition-related care, or other factors.

Conclusions

Hospitals with poor baseline performance during the first year of HRRP penalties appeared to exhibit substantial improvement relative to other hospitals nationally during the subsequent years. This analysis found that three-fourths of the relative improvement was due to RTM rather than the policy. Modifications to the HRRP that better account for the role of chance, such as excluding small hospitals with greater variability in readmissions from the HRRP or calculating mean readmissions across multiple conditions when estimating excess readmissions, might enhance its fairness and possibly also its effectiveness.

Supplement.

eMethods. Calculation of RTM

eTable 1. Change in Excess Readmissions Rates (ERRs) Explained by Regression to the Mean (RTM) for Hospitals in Right Tail of ERR Distribution

eTable 2. Change in Excess Readmissions Rates (ERRs) Explained by Regression to the Mean (RTM) for COPD

eFigure 1. Changes in Excess Readmissions Rate (ERRs) for COPD Before Inclusion of COPD in the HRRP

eFigure 2. (Analysis 3) Changes in Excess Readmissions Rates (ERRs) From FY2013 Measurement Period to Premeasurement Period

eReferences.

Click here for additional data file.^{(180.7KB, pdf)}

References

1.Centers for Medicare & Medicaid Services Hospital Readmissions Reduction Program (HRRP). https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/AcuteInpatientPPS/Readmissions-Reduction-Program.html. Modified January 16, 2019. Accessed January 16, 2019.
2.Centers for Medicare & Medicaid Services Hospital Readmissions Reduction Program (HRRP). https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/AcuteInpatientPPS/Readmissions-Reduction-Program.html. Modified January 16, 2019. Accessed January 16, 2019.
3.Desai NR, Ross JS, Kwon JY, et al. Association between hospital penalty status under the Hospital Readmission Reduction Program and readmission rates for target and nontarget conditions. JAMA. 2016;316(24):2647-2656. doi: 10.1001/jama.2016.18533 [DOI] [PMC free article] [PubMed] [Google Scholar]
4.Wasfy JH, Zigler CM, Choirat C, Wang Y, Dominici F, Yeh RW. Readmission rates after passage of the hospital readmissions reduction program: a pre-post analysis. Ann Intern Med. 2017;166(5):324-331. doi: 10.7326/M16-0185 [DOI] [PMC free article] [PubMed] [Google Scholar]
5.Barnett AG, van der Pols JC, Dobson AJ. Regression to the mean: what it is and how to deal with it. Int J Epidemiol. 2005;34(1):215-220. doi: 10.1093/ije/dyh299 [DOI] [PubMed] [Google Scholar]
6.Atkinson G, Loenneke JP, Fahs CA, Abe T, Rossow LM. Individual differences in the exercise-mediated blood pressure response: regression to the mean in disguise? Clin Physiol Funct Imaging. 2015;35(6):490-491. doi: 10.1111/cpf.12211 [DOI] [PubMed] [Google Scholar]
7.Bland JM, Altman DG. Regression towards the mean. BMJ. 1994;308(6942):1499. doi: 10.1136/bmj.308.6942.1499 [DOI] [PMC free article] [PubMed] [Google Scholar]
8.Kelly C, Price TD. Correcting for regression to the mean in behavior and ecology. Am Nat. 2005;166(6):700-707. doi: 10.1086/497402 [DOI] [PubMed] [Google Scholar]
9.Linden A. Assessing regression to the mean effects in health care initiatives. BMC Med Res Methodol. 2013;13(1):119. doi: 10.1186/1471-2288-13-119 [DOI] [PMC free article] [PubMed] [Google Scholar]
10.Yudkin PL, Stratton IM. How to deal with regression to the mean in intervention studies. Lancet. 1996;347(8996):241-243. doi: 10.1016/S0140-6736(96)90410-9 [DOI] [PubMed] [Google Scholar]
11.Bland JM, Altman DG. Some examples of regression towards the mean. BMJ. 1994;309(6957):780. doi: 10.1136/bmj.309.6957.780 [DOI] [PMC free article] [PubMed] [Google Scholar]
12.Linden A. RTMCI: Stata Module to Estimate Regression to the Mean Effects With Confidence Intervals. Boston, MA: Statistical Software Components; 2013. [Google Scholar]
13.Stigler SM. Regression towards the mean, historically considered. Stat Methods Med Res. 1997;6(2):103-114. doi: 10.1177/096228029700600202 [DOI] [PubMed] [Google Scholar]
14.Press MJ, Scanlon DP, Ryan AM, et al. Limits of readmission rates in measuring hospital quality suggest the need for added metrics. Health Aff (Millwood). 2013;32(6):1083-1091. doi: 10.1377/hlthaff.2012.0518 [DOI] [PubMed] [Google Scholar]
15.Gupta A. Impacts of performance pay for hospitals: the readmissions reduction program. Becker Friedman Institute for Research in Economics Working Paper No. 2017-07. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3054172. Posted October 16, 2017. Accessed May 29, 2019.
16.Zuckerman RB, Sheingold SH, Orav EJ, Ruhter J, Epstein AM. Readmissions, observation, and the Hospital Readmissions Reduction Program. N Engl J Med. 2016;374(16):1543-1551. doi: 10.1056/NEJMsa1513024 [DOI] [PubMed] [Google Scholar]
17.Ibrahim AM, Dimick JB, Sinha SS, Hollingsworth JM, Nuliyalu U, Ryan AM. Association of coded severity with readmission reduction after the Hospital Readmissions Reduction Program. JAMA Intern Med. 2018;178(2):290-292. doi: 10.1001/jamainternmed.2017.6148 [DOI] [PMC free article] [PubMed] [Google Scholar]
18.Ody C, Msall L, Dafny LS, Grabowski DC, Cutler DM. Decreases in readmissions credited to Medicare’s program to reduce hospital readmissions have been overstated. Health Aff (Millwood). 2019;38(1):36-43. doi: 10.1377/hlthaff.2018.05178 [DOI] [PubMed] [Google Scholar]
19.Dharmarajan K, Wang Y, Lin Z, et al. Association of changing hospital readmission rates with mortality rates after hospital discharge. JAMA. 2017;318(3):270-278. doi: 10.1001/jama.2017.8444 [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplement.

eMethods. Calculation of RTM

eTable 1. Change in Excess Readmissions Rates (ERRs) Explained by Regression to the Mean (RTM) for Hospitals in Right Tail of ERR Distribution

eTable 2. Change in Excess Readmissions Rates (ERRs) Explained by Regression to the Mean (RTM) for COPD

eFigure 1. Changes in Excess Readmissions Rate (ERRs) for COPD Before Inclusion of COPD in the HRRP

eFigure 2. (Analysis 3) Changes in Excess Readmissions Rates (ERRs) From FY2013 Measurement Period to Premeasurement Period

eReferences.

Click here for additional data file.^{(180.7KB, pdf)}

[ioi190031r1] 1.Centers for Medicare & Medicaid Services Hospital Readmissions Reduction Program (HRRP). https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/AcuteInpatientPPS/Readmissions-Reduction-Program.html. Modified January 16, 2019. Accessed January 16, 2019.

[ioi190031r2] 2.Centers for Medicare & Medicaid Services Hospital Readmissions Reduction Program (HRRP). https://www.cms.gov/Medicare/Medicare-Fee-for-Service-Payment/AcuteInpatientPPS/Readmissions-Reduction-Program.html. Modified January 16, 2019. Accessed January 16, 2019.

[ioi190031r3] 3.Desai NR, Ross JS, Kwon JY, et al. Association between hospital penalty status under the Hospital Readmission Reduction Program and readmission rates for target and nontarget conditions. JAMA. 2016;316(24):2647-2656. doi: 10.1001/jama.2016.18533 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ioi190031r4] 4.Wasfy JH, Zigler CM, Choirat C, Wang Y, Dominici F, Yeh RW. Readmission rates after passage of the hospital readmissions reduction program: a pre-post analysis. Ann Intern Med. 2017;166(5):324-331. doi: 10.7326/M16-0185 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ioi190031r5] 5.Barnett AG, van der Pols JC, Dobson AJ. Regression to the mean: what it is and how to deal with it. Int J Epidemiol. 2005;34(1):215-220. doi: 10.1093/ije/dyh299 [DOI] [PubMed] [Google Scholar]

[ioi190031r6] 6.Atkinson G, Loenneke JP, Fahs CA, Abe T, Rossow LM. Individual differences in the exercise-mediated blood pressure response: regression to the mean in disguise? Clin Physiol Funct Imaging. 2015;35(6):490-491. doi: 10.1111/cpf.12211 [DOI] [PubMed] [Google Scholar]

[ioi190031r7] 7.Bland JM, Altman DG. Regression towards the mean. BMJ. 1994;308(6942):1499. doi: 10.1136/bmj.308.6942.1499 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ioi190031r8] 8.Kelly C, Price TD. Correcting for regression to the mean in behavior and ecology. Am Nat. 2005;166(6):700-707. doi: 10.1086/497402 [DOI] [PubMed] [Google Scholar]

[ioi190031r9] 9.Linden A. Assessing regression to the mean effects in health care initiatives. BMC Med Res Methodol. 2013;13(1):119. doi: 10.1186/1471-2288-13-119 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ioi190031r10] 10.Yudkin PL, Stratton IM. How to deal with regression to the mean in intervention studies. Lancet. 1996;347(8996):241-243. doi: 10.1016/S0140-6736(96)90410-9 [DOI] [PubMed] [Google Scholar]

[ioi190031r11] 11.Bland JM, Altman DG. Some examples of regression towards the mean. BMJ. 1994;309(6957):780. doi: 10.1136/bmj.309.6957.780 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ioi190031r12] 12.Linden A. RTMCI: Stata Module to Estimate Regression to the Mean Effects With Confidence Intervals. Boston, MA: Statistical Software Components; 2013. [Google Scholar]

[ioi190031r13] 13.Stigler SM. Regression towards the mean, historically considered. Stat Methods Med Res. 1997;6(2):103-114. doi: 10.1177/096228029700600202 [DOI] [PubMed] [Google Scholar]

[ioi190031r14] 14.Press MJ, Scanlon DP, Ryan AM, et al. Limits of readmission rates in measuring hospital quality suggest the need for added metrics. Health Aff (Millwood). 2013;32(6):1083-1091. doi: 10.1377/hlthaff.2012.0518 [DOI] [PubMed] [Google Scholar]

[ioi190031r15] 15.Gupta A. Impacts of performance pay for hospitals: the readmissions reduction program. Becker Friedman Institute for Research in Economics Working Paper No. 2017-07. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3054172. Posted October 16, 2017. Accessed May 29, 2019.

[ioi190031r16] 16.Zuckerman RB, Sheingold SH, Orav EJ, Ruhter J, Epstein AM. Readmissions, observation, and the Hospital Readmissions Reduction Program. N Engl J Med. 2016;374(16):1543-1551. doi: 10.1056/NEJMsa1513024 [DOI] [PubMed] [Google Scholar]

[ioi190031r17] 17.Ibrahim AM, Dimick JB, Sinha SS, Hollingsworth JM, Nuliyalu U, Ryan AM. Association of coded severity with readmission reduction after the Hospital Readmissions Reduction Program. JAMA Intern Med. 2018;178(2):290-292. doi: 10.1001/jamainternmed.2017.6148 [DOI] [PMC free article] [PubMed] [Google Scholar]

[ioi190031r18] 18.Ody C, Msall L, Dafny LS, Grabowski DC, Cutler DM. Decreases in readmissions credited to Medicare’s program to reduce hospital readmissions have been overstated. Health Aff (Millwood). 2019;38(1):36-43. doi: 10.1377/hlthaff.2018.05178 [DOI] [PubMed] [Google Scholar]

[ioi190031r19] 19.Dharmarajan K, Wang Y, Lin Z, et al. Association of changing hospital readmission rates with mortality rates after hospital discharge. JAMA. 2017;318(3):270-278. doi: 10.1001/jama.2017.8444 [DOI] [PMC free article] [PubMed] [Google Scholar]

PERMALINK

Regression to the Mean in the Medicare Hospital Readmissions Reduction Program

Sushant Joshi, MA

Teryl Nuckols, MD, MSHS

José Escarce, MD, PhD

Peter Huckfeldt, PhD

Ioana Popescu, MD, MS

Neeraj Sood, PhD

Key Points

Question

Findings

Meaning

Abstract

Importance

Objective

Design, Setting, and Participants

Exposures

Main Outcome and Measures

Results

Conclusions and Relevance

Introduction

Methods

Population and Setting

Data Sources

Measures

Statistical Analysis

Analysis 1: Baseline Performance During the Initial HRRP Measurement Window

Figure 1. Periods Relevant to Analyses 1 to 3.

Analysis 2: Alternate Measurement Period

Analysis 3: Reverse Chronological Trends

Quantifying Regression to the Mean

Results

Descriptive Analyses

Table 1. Hospital Characteristics by Target’s Summary Statistics and by Conditionsa.

Changes in ERRs Between Contiguous Nonoverlapping Measurement Periods

Analysis 1: Baseline Performance During the Initial HRRP Measurement Window

Figure 2. Changes in Excess Readmission Ratio (ERR) From Fiscal Year (FY) 2013 Measurement Period to the 3-Year Postmeasurement Period.

Analysis 2: Alternate Measurement Period

Figure 3. Changes in Excess Readmission Ratio (ERR) From an Alternate Measurement Period to 3-Year Postmeasurement Period.

Analysis 3: Reverse Chronological Trends

Quantifying Regression to the Mean

Table 2. Changes in Excess Readmissions Ratios Explained by Regression to the Mean.

Discussion

Limitations

Conclusions

References

Associated Data

Supplementary Materials

ACTIONS

PERMALINK

RESOURCES

Similar articles

Cited by other articles

Links to NCBI Databases

Table 1. Hospital Characteristics by Target’s Summary Statistics and by Conditions^a.