Alternative Payment Models and Patient-Reported Quality of Preparation for Discharge: A Retrospective Longitudinal Study

Abstract

Preparing patients for posthospital care may improve readmission risk. Alternative payment models (APMs) incent hospitals to reduce readmissions by tying payment to outcomes. The impact of APMs on preparation for discharge is not well understood. We assessed whether patient-reported preparation for posthospital care was associated with reduced readmissions, and whether APM participation was associated with improved preparation for posthospital care. We used mixed-effects regression on retrospective (2013–2017) observational data for 2685 U.S. hospitals. We measured patient-reported preparation for posthospital care using the 3-Item Care Transition Measure and readmission using 30-day all-cause risk-adjusted readmissions from Hospital Compare. Participation in accountable care organizations (ACOs), Medical Homes, and Medicare's Bundled Payments for Care Improvement program was obtained from Medicare, the American Hospital Association's Annual Survey, and Leavitt Partner's ACO database. We found that APMs are not associated with improved preparation for posthospital care, even though it was associated with reduced readmissions (Marginal Effect: −0.012 percentage points). This may be because hospitals are not investing in patient engagement. This study has limited insight into causality and reduced generalizability among smaller, rural, and non-teaching hospitals.

Key Points:

• Patient-reported preparation for posthospital care is associated with improved readmission rates

• Hospital participation in Alternative Payment Models is not associated with improved patient-reported preparation for posthospital care

• Policymakers ought to consider explicit incentives to improve the patient experience into Alternative Payment Models

Introduction

Fragmented care with insufficient coordination may lead to poor care quality, particularly during hospital care transitions when patients are vulnerable to adverse events. One potentially effective strategy to improving the quality of care transitions is to improve the quality of patient preparation for posthospital care. The 3-Item Care Transition Measure (3-Item CTM) is a validated measure of the patient-reported quality of preparation for posthospital care.^1–3 Patients who are better prepared for hospital care transitions (eg, invited to participate in discharge planning, educated on medication and other follow-up instructions) report a greater ability to obtain and communicate medical information to their providers, leading to better care coordination,^4–8 and potentially lowering their risk of readmission. ⁹ Yet these activities are not reimbursable under fee for service payment models, leaving many hospitals without an incentive to prepare patients for posthospital care transitions.

Alternative payment models (APMs) create a powerful incentive for hospitals to improve the quality of posthospital care transitions. These programs offer financial bonuses to hospitals for improving quality and reducing cost, which is often achieved by reducing readmissions. Although patient experience of care and care transitions are touted as targets for APM policy intervention, many APMs, including Medicare programs, do not include measures of patient experience in performance evaluation. As a result, while hospitals have employed numerous strategies to improve care including investment in health information technology, ¹⁰ workforce investment, ¹¹ and telehealth, ¹² little is known about whether these strategies are directly influencing the patient experience of care. In this paper, we use 4 years of data from a national panel of hospitals to assess whether (a) higher scores on the 3-Item CTM are associated with lower readmissions and (b) APM participation is associated with higher scores on the 3-Item CTM. Results will inform future health policies that aim to improve healthcare cost and quality through patient-centered care.

Methods

We conducted a retrospective observational study using longitudinal data on a panel of U.S. hospitals from 2013 to 2017 to understand relationships between APM participation, patient-reported quality of preparation for posthospital care, and readmission rates. A hospital-year level dataset was analyzed after merging data from Medicare public use files, the American Hospital Association's Annual Survey, Leavitt Partner's ACO database, and Hospital Compare.

Study Population

The study population included all non-federal, non-critical access, acute care hospitals in the continental United States that reported data to Hospital Compare during 2013–2017 and had at least 1 year of data for all study variables.

30-Day All-Cause Readmission Rates and the 3-Item Care Transition Measure

Hospital-level 30-day risk-standardized all-cause readmission rates were extracted for 4 years (ending June 30) from 2014 to 2017 from Hospital Compare, the following year's readmission rate was used for each study year to account for lagged effects. Readmission rates are calculated based on Medicare claims data and therefore represent readmissions to any hospital with a Medicare claim. Patient-reported quality of preparation for posthospital care was measured at the hospital level using the 3-item CTM, which was extracted for 4 years (ending December 31) from 2013 to 2016. The 3-Item CTM is a validated measure endorsed by the National Quality Forum.^5,6,13–16 It is defined as the percentage of patients (range 1–99%) who answer “strongly agree” to all three of the following questions on the H-CAHPS survey: (a) The hospital staff took my preferences and those of my family or caregiver into account in deciding what my healthcare needs would be when I left the hospital; (b) When I left the hospital, I had a good understanding of the things I was responsible for in managing my health; and (c) When I left the hospital, I clearly understood the purpose for taking each of my medications.

APM Participation

Under recent U.S. healthcare reform policies, a hospital can participate in APMs that provide financial incentives for improving care quality. APMs include: accountable care organizations (ACOs), Medical Homes, and the bundled payment for care improvement (BPCI) program. APMs differ in several ways, which we discuss below.

ACOs offer shared savings to participating hospitals if per-patient spending is lower than a targeted amount and quality measures are met. ACO participating hospitals have an indirect incentive to improve quality of patient preparation for posthospital care to lower healthcare spending through reduced readmissions, and a direct incentive to improve quality of patient preparation for posthospital care in order to meet quality measures related to patient experience (eg, patient communication, incorporating patients into decision-making processes). ACO start dates were obtained from Leavitt Partners on Medicare or commercial ACO participation (current as of 2016). The Leavitt Partners ACO database contains validated information on ACOs in the US and is regularly updated using data from multiple sources including press releases, government announcements, interviews, and public records. ¹⁷

One of the core principles of Medical Homes is to coordinate acute care services. ¹⁸ To meet this goal, many Medical Homes provide performance-based rewards to providers who reduce readmissions, along with per member per month fees to put towards care management strategies which include preparing patients for posthospital care. For example, hospitals that are affiliated with Medical Homes as part of their hospital-based ambulatory care practices may integrate ambulatory providers, such as pharmacists, into inpatient care teams in order to better prepare patients for posthospital care.^19,20 These types of strategies have been shown to improve patient outcomes.^21,22 For this study, hospital Medical Home participation was obtained from the 2013–2016 AHA survey. Start and end dates for Medical Home participation were unavailable from the survey, so we assumed that hospitals reporting participation in a medical home participated for the full calendar year. For 899 hospital-year observations that were missing data on Medical Home participation due to AHA survey non-response, data was imputed using multiple imputations by chained equations. To avoid introducing bias, the imputation model included all variables in the main models as well as the following auxiliary variables: system ID if the hospital was in a system and hospital service code. ²³ Based on best practices, we generated 10 imputed datasets which is roughly equal to the percentage of incomplete cases (9%). ²⁴ Sensitivity analysis examined the sensitivity of our findings to this imputation method by dropping the hospital-year observations with missing data.

Finally, the Medicare BPCI program bundles inpatient and 30, 60, or 90 days of post-acute care services into a single “bundled payment” per care episode. Hospitals participating in the program have an incentive to improve patient preparation for posthospital care in order to lower costs through reduced readmissions. However, unlike the ACO and medical home programs, the BPCI program does not incorporate measures of patient-centered care or patient preparation for posthospital care into its program evaluation. Therefore, the effect of BPCI participation on patient preparation for posthospital care may be smaller than that of ACOs or Medical Homes. Quarterly participation in phase two of the BPCI program was obtained from the 2013–2016 Medicare BPCI public use file.

ACOs, BPCI, and Medical Homes all aim to reduce healthcare costs by improving quality during care transitions but differ in some respects on how this goal is incentivized. For this reason, we created two different measures of hospital participation in APMs. First, we calculated the portion of the calendar year (ending December 31) in which a hospital participated in any APM (either a Medicare or Commercial ACO, BPCI, and/or a Medical Home). This method allows for a simple interpretation of study findings, which can be used to inform policy implications more broadly. Second, because differences in APM design may have heterogeneous effects, we used a three-way interaction term between participation in ACOs, Medical Home, and BPCI. This method allows us to see if there are differences in the effect size based on which model(s) a hospital participates in. We report the marginal effect of each APM type. For both measures, APM participation was allowed to vary over time, such that hospitals could be in an APM in 1 year and drop out the next or vice versa.

Control Variables

We controlled for hospital characteristics that may introduce endogeneity into our analysis. Hospital data on teaching status (major teaching status was defined as membership in the Council of Teaching Hospitals, and minor teaching status was defined as accreditation by the Accreditation Council for Graduate Medical Education), system membership, network membership, and ownership were extracted from the 2013–2015 American Hospital Association annual survey (2016 AHA data was not available at the time of the analysis), and data on number of beds, Medicaid Disproportionate Share Hospital (DSH) payments, case mix index (a measure of patient clinical complexity), and urban/rural location from the 2013–2016 Medicare Impact file. Because data on these control variables are unlikely to change much over time, for hospitals that had at least 1 year of data, missing hospital-year observations were filled using the most recent prior year or future year of data (n = 2815 hospital-year observations). We believe this method is the most appropriate method for imputing missing data for these particular hospital characteristics given that these characteristics are unlikely to change much year after year. We chose these variables after a review of the literature on hospital quality with suggests that hospitals with more privilege (eg, larger, teaching hospitals) may have more resources to invest in patient engagement initiatives, and more privileged patient populations (eg, low DSH, urban location) may be more likely to rank hospitals higher on the 3-Item CTM.^6,25–27

Statistical Analysis

For each of the models described below, we estimated mixed-effects linear regression models using fixed effects for hospital controls and year, and individual hospital random effects. Using fixed effects for hospital controls removes variation that is due to time invariant hospital characteristics or temporal trends that may be endogenous. Using random effects for individual hospitals allows us to further account for any unobserved heterogeneity that are not captured in our list of hospital controls. In addition, we used robust variance estimators to account for clustering by hospital referral region. This allows us to account for heteroskedasticity across regions. We used the Hausman test to test whether covariates were correlated with error terms, an assumption of the random effects model. The Hausman test suggested that random effects for individual hospitals were the appropriate method to use.

To answer our first research question, we tested whether improved patient preparation for posthospital care is associated with reductions in the next year's readmission rates by regressing readmission rates on the quality of patient preparation for posthospital care, controlling for APM participation. To answer our second research question, we tested whether APM-participating hospitals improved patient preparation for posthospital care by regressing the 3-Item CTM on APM participation. We did this twice, examining any APM participation and participation by APM type. For the model measuring participation by APM type, we calculated average marginal effects of participation in each type to estimate the impact of each type of APM on levels of the 3-Item CTM.

We ran two sensitivity analyses. The first tested the sensitivity of our results to the number of APMs a hospital participated in any given year, as participation in multiple APMs may strengthen the effect of APM participation on the 3-Item CTM. Our second sensitivity analysis tested the sensitivity of our main results to our imputation of missing variables by running the analyses only on the subset of hospitals without any missing data in all years of the study period. It was not possible to involve patients or the public in the design, conduct, reporting, or dissemination plans of this research project.

Results

Out of 3969 hospitals that reported data to Hospital Compare in any year during the study period, 1133 hospitals were missing control variable(s) for all years and were dropped from our sample (Supplemental material, Appendix 1). Our final analytic sample included 9960 hospital-year observations for 2836 hospitals. Across all years, the unadjusted median readmission rate was 15% (standard deviation = 0.92) and the unadjusted median score on the 3-Item CTM was 51% (standard deviation = 6.21; Figure 1). The unadjusted median score on the 3-Item CTM was 50.8% for APM-participating hospitals before joining and 51.7% after joining, while the unadjusted median score on the 3-Item CTM was 50.6% for hospitals that never joined an APM. In our sample, assuming 310 hospitals missing data on Medical Home participation did not participate in a Medical Home, 51% of hospitals ever participated in an APM (n = 1451): 39% of hospitals participated in ACOs (n = 1093), 29% in Medical Homes (n = 817,), 3% in BPCI (n = 92) (Table 1). Hospitals participating in APMs differed significantly (P < .001) from hospitals that never participated in APMs in all control characteristics except DSH percentage: hospitals in APMs were more likely to have a primary care department, be urban, teaching, in a system, in a network, and non-profit; hospitals in APMs also had a larger number of beds and higher market share (Table 1).

Figure 1.

Hospital 30-day all-cause readmission rate and hospital level of patient engagement by year (2013–2017). Markers represent median, bars represent interquartile range; based on 2588 hospitals reporting data on patient engagement in 2013, 2588 and 2526 hospitals reporting data on patient engagement and readmission rates in 2014, respectively, 2526 and 2618 hospitals reporting data on patient engagement and readmission rates in 2015, respectively, and 2618 and 2228 hospitals reporting data on patient engagement and readmission rates in 2016, respectively, and 2228 hospitals reporting data on readmission rates in 2017.

Table 1.

Hospital Characteristics by APM Participation.

Characteristics	All hospitals	Ever in APM1	Never in APM1
	(n = 2836)	(n = 1451)	(n = 1385)
APM participation***
Ever in APM	51%	100%	0%
Ever in ACO	39%	75%	N/A
Ever in medical home	29%	56%	N/A
Ever in BPCI	3%	6%	N/A
Never in APM	49%	0%	100%
Total1	100%	100%	100%
Type of hospital (%)
Has primary care department2***
No	36%	24%	51%
Yes	64%	76%	49%
Total	100%	100%	100%
Urban/rural Classification***
Large urban	41%	53%	29%
Other urban	34%	33%	34%
Rural	25%	14%	37%
Total	100%	100%	100%
Teaching hospital***
Non-teaching	51%	38%	64%
Major (COTH)	8%	14%	3%
Minor (med. school affiliation)	41%	48%	33%
Total	100%	100%	100%
System membership***
not in a system	28%	20%	36%
In a system	72%	80%	64%
Total	100%	100%	100%
Network membership***
Not in a network	57%	46%	69%
In a network	43%	54%	31%
Total	100%	100%	100%
Ownership***
Government	15%	11%	19%
Non-profit	65%	82%	48%
For-profit	20%	7%	33%
Total	100%	100%	100%
Other hospital characteristics (Mean)
Number of beds***	203	253	151
DSH percentage	0.29	0.28	0.29
Case mix index***	1.55	1.62	1.47
Hospital marketshare***	1.49	1.76	1.22

Notes: Difference between hospitals ever in APMs and never in APMs are statistically significant at ***P < 0.001, **P < 0.01, *P < 0.05.

¹ Assuming 310 hospitals missing data on Medical Home participation did not participate in the year that the data was missing.

² Either owned or affiliated through a system or joint venture, not included in models. Hospital characteristics based on more recent available year of data per hospital. Marketshare calculated as the percentage of beds in hospital system in hospital referral region.

BPCI=bundled payment for care improvement; DSH=Medicaid Disproportionate Share Hospital.

In panel analysis, after adjusting for hospital-specific trends, characteristics, and year fixed effects, we found that each percentage point increase in the 3-Item CTM was associated with a −0.012 percentage point change in the following year's readmission rates (P < .005; Table 2, Model 1). To put this finding in context, we approximate the marginal effect on costs of care for a 1 standard deviation (6.2%) nationwide improvement in the 3-Item CTM. Based on prior work, we assume that the average number of 30-day all-cause hospital readmissions per year is 4,228,000 ²⁸ and the average cost of a readmission to be $15 000 per incident. ²⁹ We estimate that a 6.2 percentage point improvement in the 3-Item CTM nationwide would result in 0.074 percentage point lower readmission rate (0.012 percentage point reduction in readmission rates×6.2), which is equivalent to 3125 fewer readmissions per year (0.074% of 4,228,000 readmissions a year), resulting in roughly $47 million in cost saving a year ($15 000 per readmission * 3125 fewer readmissions).

Table 2.

Adjusted Association Between Patient-Reported Preparation for Posthospital Care, Readmissions, and Hospital Participation in Alternative Payment Models.

Marginal effects	(1)	(2)
	3-item care transition measure and readmission rates	APM participation and 3-item care transition measure
Full-year APM participation	−0.071*	−0.116
	[−0.126,−0.015]	[−0.172,0.404]
Patient engagement	−0.012***
	[−0.016,−0.008]
Controls
No. beds	0.001***	−0.005***
	[0.001,0.001]	[−0.006,−0.003]
Urban (Ref: large urban)
Other urban	−0.202***	1.067**
	[−0.300,−0.104]	[−0.366,1.769]
Rural	−0.189**	0.912*
	[−0.300,−0.078]	[0.164,1.659]
DSH percent	0.717***	−8.780**
	[0.462,0.972]	[−10.561,−7.000]
Case mix index	−0.484***	4.795***
	[−0.607,−0.362]	[3.917,5.673]
Teaching (Ref: non-teaching)
Major teaching	0.403***	−0.852**
	[0.273,0.532]	[0.267,1.437]
Minor teaching	−0.058	0.219
	[−0.113,−0.003]	[−0.058,0.495]
System (Ref: non-system)
System member	−0.025	−0.221
	[−0.066,0.017]	[−0.500,0.058]
Network (Ref: non-network)
Network member	−0.017	0.001
	[−0.065,0.032]	[−0.337,0.340]
Ownership (Ref: government)
Non-profit	−0.001	−0.290
	[−0.087,−0.085]	[−0.963,0.382]
For-profit	0.239***	−2.087***
	[0.126,0.351]	[−2.933,−1.241]
Marketshare (System HHI in HRR)	−0.043***	−0.122*
	[−0.066,−0.021]	[−0.234,−0.009]
Constant	16.400***	46.824***
	[16.076,16.725]	[45.204,48.444]
N (hospital-years)	9960	9960

Note: 95% confidence intervals in brackets. All models include hospital random effects and year fixed effects (not shown), and HRR clustered standard errors. *P < 0.05, **P < 0.01, ***P < 0.001.

ACO=accountable care organizations; DSH=Medicaid Disproportionate Share Hospital.

Adjusted analysis of the association between APM participation and the 3-Item CTM suggests that APM participation did not have a statistically significant association with the 3-Item CTM: full-year APM participation was associated with a −0.116 percentage point change in the 3-Item CTM (P = 0.342 Table 2, Model 2). When we examined participation by specific APM types, we found that the average marginal effects of ACO participation and Medical Home Participation on the 3-Item CTM was positive, but not statistically significant [0.18 percentage points (P = .353) and 0.13 percentage points (P = .458), respectively; Figure 2]; the average marginal effect of BPCI participation was −1.02 percentage points; this effect was statistically significant (P = .003, Figure 2). Full regression results are presented in Supplemental material, Appendix 2.

Figure 2.

Marginal effects of APM type participation on the 3-item CTM. Bars indicate marginal effects, lines indicate 95% confidence intervals; marginal effects based on mixed-effects regression model regressing the 3-item CTM on types of APM participation; model includes hospital characteristics, year fixed effects, hospital random effects, and HRR clustered standard errors; P < 0.05, **P < 0.01, ***P < 0.001.

Sensitivity Analysis

Consistent with our main results, sensitivity analyses suggested that participating in an additional APM was associated with an increase in the 3-Item CTM by 0.13 percentage points, this effect was not statistically significant (P = .168; Supplemental material, Appendix 3). When we examined only observations with complete data, results were also consistent with our main results; we found that each percentage point increase in the 3-Item CTM was associated with a significant −0.13 percentage point change in readmission rates (P < .001) and that APM participation is associated with a statistically insignificant 0.30 percentage point change in the 3-Item CTM (P = .054; Supplemental material, Appendix 4).

Discussion

In this study, we asked whether patient-reported preparation for posthospital care is associated with improved readmission rates, and whether patient-reported preparation for posthospital care is improving under APMs. We found that higher 3-Item CTM scores during hospital care transitions was associated with lower readmission rates, but that APM participation was not associated with improved 3-Item CTM scores. Taken together, these findings suggest that APM-participating hospitals may be overlooking a critical quality improvement strategy that achieves the dual purposes of reducing readmission rates and improving the patient experience of care.

Our finding that the 3-Item CTM was associated with lower readmission rates is consistent with prior evidence suggesting investing in patient education and preparation for posthospital care transitions may help hospitals succeed in reducing costs and improving patient-centered care,^4–8 both priority areas under current U.S. healthcare delivery reform policies. For example, prior work found that every 10 point increase in the 3-Item CTM was associated with a 14% reduction in readmission risk among a subsample of patients undergoing coronary revascularization. ⁶ Based on our findings, we estimate that if hospitals improved their score on the 3-Item CTM by 1 standard deviation (6 percentage points) in 2016, four thousand readmissions could have been avoided nationwide, saving the healthcare system roughly $47 million.

However, our finding that APM participation was not associated with improvements in the 3-Item CTM suggests that APMs do not create strong enough incentives to promote patient-reported quality. This is consistent with prior work that found that patients in a commercial ACO were not more likely to report a better patient experience. ³⁰ Specifically, our analysis of APM type found no association between ACO and medical home participation and the 3-Item CTM, and a negative association between BPCI participation and the 3-Item CTM. Our finding that ACO and medical home participation is not associated with patient engagement is not surprising, given that the responsibility of improving patient care tends to be placed on primary care providers under these models. However, our finding that BPCI participation is negatively associated with patient engagement is alarming and suggests that patient experience of the quality of posthospital care preparation may be suffering at the expense of other strategies hospitals are using to reduce costs. Sensitivity analysis suggests that participation in multiple APM types may improve performance on the 3-Item CTM, though this effect was not statistically significant.

These findings suggest that policy emphasis on the importance of patient-centered care is not misplaced. And that, under current payment reform programs, APMs have not focused on incentivizing improvements in the patient experience of care. Specifically, in the Medicare Shared Savings Program (MSSP), Medicare's largest ACO program, only three of 31 quality metrics are related to patient experiences (provider communication, shared decision-making, and health promotion and education ²² ) and of them, none are specific to patient preparation for posthospital care transitions, a critical and vulnerable phase of patient care. Meanwhile, neither Medical Homes nor the BPCI program requires hospitals to meet any patient experience quality metrics, and the BPCI advanced program only incorporates one measure relating to patient engagement (advanced directives), which is unrelated to care transitions.

Our findings seem to contradict prior evidence suggesting that APM participation leads to care coordination improvements during hospital care transitions (eg information sharing, care managers, performance-based contracts, and provider integration^25–31). This may be because prior studies have focused primarily on the provider perspective, not the patient perspective. Future research should also examine how APM participation may be spurring improvements in other dimensions of patient experience such as shared decision-making, patient communication, and access to patient records.

Findings from this study have important implications. Given that policy and delivery reform programs are increasingly focusing on the patient experience, improving patient preparation for posthospital care transitions may be “low hanging fruit” for hospitals to meet the dual goals of improve readmission rates and patient-centered care. And, in light of recent policy changes that are pushing for more patient-centered approaches to care and accelerating hospital participation in payment and delivery reform programs, policy makers may wish to include more measures of patient experience in evaluating APM performance. ³¹ Specifically, policymakers may wish to add the 3-item CTM in its assessment of patient experience for Medicare ACO and bundled payment programs, and strengthen incentives to improve hospital care transitions under the Patient Centered Medical Home program.

Limitations

It is important to interpret our results with several limitations in mind. First, compared to hospitals dropped due to missing data, hospitals in the sample were more likely to be larger, teaching, and system-owned (see Supplemental material, Appendix 1 which compares hospitals in and out of sample). These types of hospitals may have more resources to implement patient engagement activities. Therefore, we would expect that our analysis would be more likely to find a positive relationship between APM participation and the 3-Item CTM. Second, recent evaluations of the 3-Item CTM raised concerns that improvements in the 3-Item CTM may not have the same effect on readmission rates across all patient populations, necessitating the need for risk adjustment in studies using the measure.^6,27 We addressed this limitation by controlling for case mix index and DSH percentage and hospital random effects; however, it is possible that our results were influenced by unobserved patient characteristics that were omitted from the analysis such as socio-economic status. Further investigation on the equity impact of APMs and patient experience among minoritized and marginalized populations is critical. Relatedly, since the 3-Item CTM measure used in this study was reported by CMS at the hospital-level, we are unable to assess the 3-Item CTM at the patient level. This limitation is important to keep in mind when interpreting the association between ACO and BPCI participation and patient engagement because not all patients admitted to APM-participating hospitals are covered under APM payment policies. In other words, our study evaluates the direct and spillover effects of APM participation on all patients admitted to APM-participating hospitals. Finally, the study evaluates associations between changes in hospital-level variables over time and does not imply causality.

Conclusion

In this study, we sought to understand whether hospitals participating in APMs improved quality on patient-reported preparation for posthospital care transitions. Our findings suggest that greater patient preparation for posthospital care is associated with lower readmission rates, but APM participation is not associated with improved patient preparation for posthospital care. This suggests that APM-participating hospitals may be overlooking an important and effective patient-centered strategy to improving readmissions and stronger policy incentives will be needed to promote patient-centered care under new payment models.