The Early Impact of the CMS State Innovation Models Initiative on 30-Day Hospital Readmissions among Adults with Diabetes

Abstract

Background

The Centers for Medicare & Medicaid Services (CMS) State Innovation Models (SIM) Initiative funds states to accelerate delivery system and payment reforms. All SIM states focus on improving diabetes care, but SIM’s effect on 30-day readmissions among adults with diabetes remains unclear.

Methods

A quasi-experimental research design estimated the impact of SIM on 30-day hospital readmissions among adults with diabetes in three Round 1 SIM states (N=671,996) and three comparison states (N=2,719,603) from 2010–2015. Difference-in-differences (DID) multivariable logistic regression models that incorporated four-group propensity score weighting were estimated. Heterogeneity of SIM effects by grantee state and for CMS populations were assessed.

Results

In adjusted DID analyses, SIM was associated with an increase in odds of 30-day hospital readmission among patients in SIM states in the post- versus pre-SIM period relative to the ratio in odds of readmission among patients in the comparison states post- versus pre-SIM (ratio of aOR=1.057, p=0.01). Restricting the analyses to CMS populations (Medicare and Medicaid beneficiaries), resulted in consistent findings (ratio of aOR= 1.057, p=0.034). SIM did not have different effects on 30-day readmissions by state.

Conclusions

We found no evidence that SIM reduced 30-day readmission rates among adults with diabetes during the first 2 years of Round 1 implementation, even among CMS beneficiaries. It may be difficult to reduce readmissions statewide without greater investment in health information exchange and more intensive use of payment models that promote interorganizational coordination.

Background

The Centers for Medicare & Medicaid Services (CMS) State Innovation Models (SIM) Initiative is a federal-state partnership that, since 2013, has awarded over $1 billion in funding and technical assistance to states through a competitive process.¹ In order to receive funds, state health departments proposed plans to implement delivery system and payment reforms to improve health system performance, improve the quality of patient care, and decrease health care costs for all residents of their state.² The first round of SIM funding was awarded in April 2013 to six states (Arkansas, Maine, Massachusetts, Minnesota, Oregon and Vermont). The implementation period began in October 2013 and ended in 2018 for most grantee states.

Delivery system reforms within states’ SIM plans included patient-centered medical home (PCMH) implementation, behavioral health and primary care integration, hospital to outpatient care transitions management, expansion of health information technology, and workforce development.³ Testing value-based payment methods, such as shared savings programs involving multiple payers, was also central to each state’s SIM plan and all funded states had plans to implement these new methods. To do so, they implemented or built on existing primary care–based models. For example, the Primary Care Payment Reform Initiative in Massachusetts (2014–2016) used two-sided risk arrangements for some costs, and Arkansas used a Medicaid PCMH model with optional one-sided financial risk and payment for 14 episodes of care. Value-based payment models also involved commercial payers aligning with state Medicaid programs, including the Arkansas PCMH and episodes of care models and the Vermont ACO shared savings program.¹

Adult patients with diabetes are often a primary target population for care management as part of delivery system reforms.⁴ Value-based payment reforms also benefit from targeting adults with diabetes because of the high costs of diabetes-related care. The total estimated cost of diagnosed diabetes in 2017 was $327 billion, including $237 billion in direct medical costs.^5,6 Accordingly, improving care management and care coordination among adults with diabetes was a focus of SIM state plans. For instance, Maine added diabetes to its health priorities in its existing State Health Improvement Plan as a result of SIM and expanded the existing National Diabetes Prevention Program using SIM funding. Massachusetts developed a bidirectional referral system to link primary care providers with community resources, which included diabetes education programs. In addition, SIM encouraged states to integrate population diabetes metrics into payment reform models. For example, in Arkansas, PCMHs used process measures for patients with diabetes to help interdisciplinary care teams monitor patient care and diabetes control measures were central to Vermont’s ACO models. The largest component of expenditures for diabetes is hospital inpatient care (29 percent of the total medical cost),⁶ because uncontrolled diabetes can result in high-cost complications,⁷ including retinopathy, acute myocardial infarction, stroke, congestive heart failure, and non-traumatic lower extremity amputation.⁸ As a result, hospitalization rates among adults with diabetes are specifically monitored as part of all states’ SIM implementation and evaluation plans.

The implementation of SIM plans by select states presents an opportunity to conduct a natural experiment of the early impact of the SIM initiative on reduced 30-day readmissions among adults with diabetes. In this study, we examine the extent to which hospitalized adults with diabetes in Round 1 SIM states were less likely to be readmitted within 30 days of discharge in the first two years of SIM implementation compared to hospitalized adults with diabetes in select comparison states. SIM states focused on improving chronic care management in the short-term, which may have accelerated the reduction in 30-day readmission rates among adults with diabetes relative to comparison states above and beyond national health reforms and incentives for reducing readmissions. Previous studies have found that 30-day readmissions among adults with diabetes can be averted by timely primary care follow-up.⁹ Readmission rates, however, can decline because overall hospitalization rates decline.¹⁰ A recent study found that Round 1 SIM was associated with increased diagnostic prevalence of diabetes in SIM states, likely due to increased screening, but not lower all-cause hospitalization rates among adults with diabetes.¹¹ Reducing 30-day readmission rates requires improving chronic care management through interorganizational alignment and coordination, which are primary objectives of the SIM Initiative.^1,12,13. Consequently, SIM may have contributed to reduced 30-day readmissions among hospitalized patients with diabetes across payers, while not measurably reducing overall hospitalization rates.¹¹

Methods

Study Design

Using a quasi-experimental research design, we estimate the impact of SIM on 30-day hospital readmissions using data from hospitalized adult patients with diagnosed diabetes from six states in 2010–2015, based on Healthcare Cost and Utilization Project (HCUP) State Inpatient Data (SID) that included a linking variable to connect index and readmission visits. States were included based on HCUP SID visit linkage file availability and Round 1 states include Arkansas, Massachusetts, and Vermont. Comparison group states not receiving SIM funding are Florida, Georgia, and New Mexico. SIM funding began on April 1, 2013, which included an initial six month test period, resulting in the test implementation phase beginning October 1, 2013.¹² To allow for SIM implementation start-up time and because of higher readmission rates at the end of each calendar year, we coded the beginning of the early implementation period as January 1, 2014 and the end of the early implementation period as December 31, 2015. Table 1 summarizes differences in major statewide delivery system and payment reforms being planned and implemented across the six states included during the study period.

Table 1.

Statewide Delivery System and Payment Reform Activities and Milestones of State Innovation Models Initiative (SIM) and Comparison States

	Delivery System Reforms	Payment Reforms
SIM States, Round 1
Arkansas	• Comprehensive Primary Care Initiative • Comprehensive Primary Care Initiative Plus • Multipayer patient-centered medical homes • Performance feedback reports via multipayer portal • Policies to encourage providers’ use of real-time alerts of patients’ hospital and emergency department (ED) use	• Achieved multipayer alignment through SIM • Majority of SIM funding spent on supporting new payment models • 14 Episodes of Care tested • Included behavioral health (BH) performance measures in non-behavioral health home (BHH) value-based payment model • Medicaid Managed Care
Massachusetts	• Primary Care Payment Reform Initiative (PCPRI), a Medicaid PCMH with BH integration, shared savings • Required colocation of behavioral health and primary care providers for SIM payment • Performance feedback reports from Medicaid to providers participating in PCPRI • SIM funds for technical assistance to connect providers to health information exchange • Policies to encourage primary care physicians to access real-time alerts of hospital and ED use	• Did not achieve multipayer alignment through SIM • Majority of SIM funding spent on supporting new payment models • Medicaid Accountable Care Organizations • Delivery System Reform Incentive Program • Included behavioral health performance measures in non-behavioral health value-based payment model • SIM payment models offered care management fee • Medicaid Managed Care
Vermont	• Multipayer Patient Centered Medical Homes • Medicaid Health Homes • Training for behavioral health in learning collaboratives for care management • System to offer real-time alerts of patients’ ED and inpatient use to nonhospital providers • Supported exchange of mental health and substance abuse data outside the health information exchange • Technical assistance to connect providers to health information exchange • Policies aimed at improving data quality in health information exchange	• Achieved multipayer alignment through SIM • Multipayer Accountable Care Organizations • Delivery System Reform Incentive Program • Included BH performance measures in non-BHH value-based payment model • Had BHH model and supported BHHs or other implementation (not payment) with SIM funds
Comparison Group States
Georgia	• Medicaid Patient Centered Medical Homes	• No multipayer alignment • Medicaid Managed Care
Florida	• Medicaid Patient Centered Medical Homes	• No multipayer alignment • Medicaid Managed Care
New Mexico	• Medicaid Patient Centered Medical Homes • Medicaid Health Homes	• No multipayer alignment • Medicaid Managed Care • Delivery System Reform Incentive Program

Sources: Beil et al. (2019), Kissam et al. (2019), Rittenhouse et al. (2019), and Rutledge et al. (2019)

https://nashp.org/state-delivery-system-payment-reform-map/

https://www.kff.org/interactive/delivery-system-and-payment-reform/

Analytic Sample

Hospitalizations of adults with diagnosed diabetes were identified based on Agency for Healthcare Research and Quality (AHRQ) specifications using the HCUP Clinical Classifications Software (CCS), which categorizes ICD-9-CM diagnosis codes into a clinical group to understand patterns of diagnoses and procedures.¹⁴ Patients with diabetes aged 18 and older were identified using the Clinical Classifications Software code of diabetes without complication (CCS=49) and diabetes with complications (CCS=50), which were extracted from principal and secondary diagnosis fields. A patient’s first hospitalization in the calendar year was marked as an index admission. Patients in the index admission who died, left the hospital against medical advice, or were transferred to another acute care facility were excluded (n=196,013). Index admissions that occurred in December of each year were excluded (n=142,098) because the HCUP SID patient identifiers did not allow for matching of patients across calendar years, making it impossible to document 30-day readmissions for these visits. Patients with home addresses outside the states under study were also excluded (n=161,962). The analytic sample includes 3,391,599 index admissions, with 671,996 index admissions in SIM states, including 227,206 during SIM implementation. A total of 696 hospitals are included across the six states with an annual average of 932.1 index visits by adults with diabetes per hospital (SD=1,101.1).

Dependent Variable

The dependent variable equaled one if the index visit was followed by readmission less than or equal to 30 days after discharge and equaled zero if the readmission was greater than 30 days after discharge or no readmission.

Statistical Analyses

The effect of the SIM Initiative on 30-day readmissions was estimated using a difference-in-differences multivariable logistic regression model that incorporates propensity score weighting.¹⁵ The patient-level regression models had the structure of equation (1) in which p indexes patients, h indexes hospitals, s indexes states, and t indexes years; outcome_p,h,s,t indicates whether the patient was readmitted within 30 days or less; SIM_s x post_t indicates SIM Initiative implementation; hospital_h is a vector of indicator variables for each hospital to control for time-invariant readmission rate differences among hospitals (hence, no SIM main effect is included); year_t is a vector of indicator variables for each year (except 2010, the reference year) to control for trends in readmission rates over time; and X_p,h,s,t is a vector of patient characteristics. β₁ is the parameter of interest, which is a ratio of two odds ratios in a logistic regression model: the odds ratio of a readmission for patients in SIM states post- versus pre-SIM divided by the odds ratio of a readmission for patients in comparison states post- versus pre-SIM.¹⁶

Patient characteristics were measured at the index admission. Previous research examining Black-White differences in 30-day hospital readmission rates among adults with diabetes has yielded mixed results; some studies find higher readmission rates among Blacks, while others find no difference. ^16–18 Only one study examined differences between Hispanics and non-Hispanics and found lower readmission rates for Hispanics.¹⁹ To advance evidence about racial and ethnic differences, we include race/ethnicity (White, Hispanic, Black, Other) as an independent variable in our regression analyses.

Other covariates, which have been found to be associated with 30-day readmissions,^{9,16–18,20–26} were included: age, sex, length of stay of the index visit, admission via the emergency department, comorbidity burden, and insurance type (Medicare, Medicaid, dually eligible Medicare-Medicaid, private, uninsured, other). Standard errors were estimated by clustering at the hospital level to allow for correlation within hospitals over time.²⁷

To account for pre-treatment readmission rate trends, we included SIM-year trends by interacting SIM status with year as a continuous variable.²⁸ If pre-SIM 30-day readmission rate trends were not parallel between SIM and comparison states and the non-parallel readmission trends would have persisted in the post-intervention period absent the SIM Initiative, then the difference-in-differences regression estimates will be biased if these trends are not included in the model.²⁹

$$logit(outcom{e}_{p,h,s,\mathrm{t}}=1)={\mathrm{\beta }}_0+{\mathrm{\beta }}_{1}SI{M}_{\mathrm{s}}\times {\mathrm{post}}_{\mathrm{t}}+{\mathrm{\beta }}_2{\mathrm{hospital}}_{\mathrm{h}}+{\mathrm{\beta }}_3{\mathrm{year}}_{\mathrm{t}}+{\mathrm{\beta }}_4{\mathrm{X}}_{\mathrm{p},\mathrm{h},\mathrm{s},\mathrm{t}}+{\mathrm{\beta }}_{5}SI{M}_{\mathrm{s}}\times \mathrm{year}\_{\mathrm{continuous}}_{\mathrm{t}}$$ (1)

Model 1 was estimated as a weighted regression using propensity score weights to balance patient-level covariates (X_p,h,s,t) in SIM and comparison states before and after SIM implementation to be similar to SIM states in the pre-SIM period.¹⁵ Stata 14.0 -mlogit- was used to estimate propensity scores for each of four groups: SIM states in the pre-SIM period, SIM states in the post-SIM period, comparison states in the pre-SIM period, and comparison states in the post-SIM period. The four-group propensity score weighting method balances the populations on the observed patient covariates that may differ over time due to changes in group composition.¹⁵ Weights were calculated using each patient’s probability of being in a SIM state in the pre-SIM period relative to the patient’s probability of being in the group the patient is in. These weights were used as probability weights to estimate equation (1) above.³⁰ This approach accounts for differences among the groups that may affect the post-SIM period.

Alternative Model Specifications

Because downward readmission trends in Round 1 states relative to comparison states may not persist in the post-intervention period due to diminishing returns of pre-existing investments in readmission rate improvement, the inclusion of SIM-year trends in the model could bias the results. Diminishing returns in Round 1 states may be expected because these states were early adopters of Medicaid expansion and implemented statewide delivery system and payment reforms prior to SIM.^31,32, so a secondary specification (Model 2) removes the pre-treatment trend adjustment.

While many SIM investments were aimed at all grantee states’ residents, some were focused only on the payers involved in the new payment models, which did not always include commercial payers. Thus it is possible that CMS program beneficiaries may have benefited more from states’ investments. To assess whether the effect of SIM on 30-day readmissions was similar for CMS program beneficiaries, we re-specified Model 1 restricting the sample to Medicare and Medicaid patients and removed the payer type adjustment from the regression model (Model 3).

Finally, it is possible that the effect of SIM on 30-day hospital readmissions might differ by SIM state. For example, Vermont was the only SIM state in our sample that used SIM funding during the early implementation period to create and expand Medicaid ACOs, which aim to change care patterns and reduce costs.³³ Vermont also achieved significant coordination across payers through SIM, specifically Medicaid and Medicare, the latter of which includes the majority of patients hospitalized with diabetes.³³ Vermont also covered greater numbers of residents in its PCMHs compared to Massachusetts and Arkansas, and was more successful in achieving multi-payer participation in its value-based payment reforms through SIM.¹⁰ To examine the possibility of heterogeneous effects of SIM on 30-day readmissions by state, we modified Model 2. Instead of estimating an overall SIM effect, separate dummy variables for each of the three Round 1 SIM states (Arkansas, Massachusetts, and Vermont) were included (Model 4). Then, we modified Model 4 by including pre-treatment trends to account for differences between individual state readmission rates prior to SIM implementation (Model 5).

Results

Adults with diabetes who were hospitalized in SIM states differed from adults with diabetes who were hospitalized in comparison states in a number of ways (Supplemental Digital Content, Table A1). All patient-level covariates (X_p,h,s,t) were included in the generation of propensity scores, which substantially reduced standardized differences in patient demographic and clinical characteristics between SIM states’ pre-SIM patients and the other three groups: 1) SIM states, post-SIM, 2) comparison states, pre-SIM, and 3) comparison states, post-SIM (Table 2). Across these variables, standardized differences after propensity score weighting were small, ranging from −0.009 to 0.018. For example, the unweighted standardized differences in the proportion of Hispanic patients between SIM states, pre-SIM was −0.089 when compared to SIM States, post-SIM; −0.253 when compared to comparison states, pre-SIM; and −0.275 when compared to comparison states, post-SIM. These standardized differences indicate that there are a lower proportion of Hispanics in SIM states, pre-intervention compared to the other three groups. Once the propensity score weights were incorporated, the standardized differences in the proportion of Hispanic patients between SIM states, pre-intervention and each of the other three groups reduced to −0.005, 0.010, and 0.011, respectively.

Table 2.

Covariate Balance with Four-Group Propensity Score Weights

Variable	Unweighted (U) or Weighted (W)	Standardized Difference, SIM States, Post-SIM	Standardized Difference, Control States, Pre-SIM	Standardized Difference, Control States, Post-SIM
Black	U	0.012	−0.301	−0.302
	W	0.005	−0.009	−0.005
Hispanic	U	−0.089	−0.253	−0.275
	W	−0.005	0.010	0.011
Other race/ethnicity	U	−0.012	−0.027	−0.042
	W	0.001	−0.003	−0.003
Age	U	−0.015	0.051	0.036
	W	−0.001	−0.007	−0.003
Female	U	0.022	−0.005	0.012
	W	0.001	−0.006	−0.006
Medicaid	U	−0.078	0.002	−0.003
	W	−0.004	−0.002	−0.005
Medicare-Medicaid	U	0.053	0.391	0.413
	W	0.004	0.008	0.010
Private insurance	U	0.007	0.085	0.083
	W	−0.001	−0.004	0.000
Uninsured	U	0.122	−0.216	−0.181
	W	0.004	−0.000	−0.002
Other payer	U	0.040	0.031	0.044
	W	0.003	−0.003	−0.003
Length of stay	U	−0.037	−0.100	−0.100
	W	−0.012	0.010	0.006
Emergency Department admission	U	−0.028	−0.045	−0.178
	W	−0.000	0.018	0.016
Comorbidity count	U	−0.133	−0.335	−0.400
	W	−0.009	0.004	0.012

Note: Standardized differences are in comparison to SIM states, pre-SIM implementation.

Source: Authors’ analysis of Healthcare Cost and Utilization Project’s State Inpatient Databases

The figure (left graph) graphically displays unadjusted 30-day readmission rate trends for SIM and comparison states. The figure depicts the sharper pre-SIM reduction in readmission rates among adults with diabetes in SIM states. As the dotted lines illustrate, the unadjusted 30-day readmission rate in the post-SIM period does not decline for SIM or comparison states.

The main results of the differences-in-differences logistic regression model with four-group propensity score weights (Model 1) is detailed in Table 2. The estimate for the parameter of SIM x post (a ratio of aOR) is 1.057 (95% CI=1.013, 1.103, p=0.011) indicating a small, but statistically differential increase in odds of 30-day hospital readmission among adult patients with diabetes in SIM states in the post- versus pre-SIM period relative to the ratio in odds of readmission among adult patients with diabetes in the comparison states post- versus pre-SIM. Readmission rate trend differences between SIM and comparison states indicate that 30-day readmission rates were declining faster during the pre-SIM period in SIM states compared with comparison states (ratio of aOR=0.970, p<0.001).

Table 3, Model 2, which does not account for pre-SIM readmission rate trend differences between SIM and comparison states, estimates the parameter of SIM x post (a ratio of aOR) to be 0.965 (95% confidence interval (CI)=0.937, 0.994, p=0.02), indicating a small but statistically differential reduction in odds of 30-day hospital readmission among adult patients with diabetes in SIM states in the post- versus pre-SIM period relative to the ratio in odds of readmission among adult patients with diabetes in the comparison states post- versus pre-SIM.

Table 3.

Difference-in-Differences Logistic Regression Results: SIM Impact on 30-Day Hospital Readmissions among Adults with Diabetes

	Model 1 (Difference-in-Difference, Pre-Treatment Trend Adjustment, PS weights) n=3,391,434		Model 2 (Difference-in-Difference, no Pre-Treatment Trend Adjustment, PS weights) N=3,391,434		Model 3 (Model 1 with Medicare and Medicaid Populations Only) n=2,427,713
Variable	aOR	95% CI	aOR	95% CI	aOR	95% CI
SIM × Post	1.057**	1.013, 1.103	0.965**	0.937, 0.994	1.057*	1.001, 1.106
SIM × year trend	0.970***	0.958, 0.983			0.970***	0.956, 0.983
Race/ethnicity
Black	0.941***	0.925, 0.957	0.941***	0.925, 0.957	0.956***	0.935, 0.976
Hispanic	0.885***	0.862, 0.907	0.884***	0.862, 0.907	0.901***	0.877, 0.926
Other	0.840***	0.814, 0.866	0.839***	0.814, 0.866	0.858***	0.826, 0.892
Age	1.003***	1.002, 1.004	1.003***	1.002, 1.004	1.001*	1.002, 1.004
Female	0.917***	0.906, 0.927	0.917***	0.906, 0.927	0.915***	0.904, 0.927
Length of stay	1.037***	1.035, 1.039	1.037***	1.035, 1.039	1.030***	1.030, 1.035
ED admission	1.215***	1.190, 1.241	1.215***	1.190, 1.241	1.227***	1.198, 1.256
Comorbidity count	1.140***	1.135, 1.145	1.140***	1.135, 1.145	1.136***	1.131, 1.141
Payer
Medicaid	1.051***	1.024, 1.078	1.051***	1.024, 1.078
Medicare-Medicaid	1.107***	1.086, 1.128	1.108***	1.087, 1.129
Private	0.739***	0.725, 0.753	0.739***	0.725, 0.753
Uninsured	0.682***	0.644, 0.722	0.682***	0.644, 0.722
Other	0.816***	0.790, 0.842	0.816***	0.790, 0.843
Goodness of Fit
AIC	1394714		1394745		1099189
BIC	1395066		1395071		1099430

Note: Models 1, 2, and 3 are estimated using four-group propensity score weighting; models include year and hospital indicators (not shown). Standard errors are estimated by clustering at the hospital level. aOR: adjusted odds ratio; for interaction terms, it is the ratio of two adjusted odds ratios. CI=confidence interval. Reference group for race/ethnicity is white and for payer is Medicare. 165 observations were dropped in regression models due to perfect prediction within hospitals.

Source: Authors’ analysis of Healthcare Cost and Utilization Project’s State Inpatient Databases

^*p<0.10

^**p<0.05

^***p<0.01

Restricting the analyses to Medicare and Medicaid beneficiaries demonstrates consistent findings (Table 3, Model 3), with an estimate for the parameter of SIM x post (a ratio of aOR) of 1.057 (95% confidence interval (CI)=1.001, 1.106, p=0.034).

The figure (right graph) illustrates the unadjusted 30-day readmission rate trends for adults with diabetes for each SIM state compared to adults with diabetes in the comparison states. Analyses of state-specific SIM effects which do not account for pre-SIM readmission rate trend differences between SIM and comparison states (Table 4, Model 4), estimates the parameter of AR x post (a ratio of aOR) to be 0.951 (95% CI=0.907, 0.998, p=0.04 and MA x post to be 0.970 (95% CI=0.938, 1.003, p=0.07), indicating small reductions in odds of 30-day hospital readmission among adult patients with diabetes in Arkansas and Massachusetts in the post- versus pre-SIM period relative to the ratio in odds of readmission among adult patients with diabetes in the comparison states post- versus pre-SIM. Readmission rate trends for Vermont, however, are no different relative to trends in comparison states. Consistent with our main model (Table 3, Model 1), once pre-SIM readmission trends are considered, there is no evidence of different effects on 30-day readmissions between SIM and comparison states (Table 4, Model 5). Compared to patients in comparison states, Massachusetts (ratio of aOR=1.054, p=0.012) patients had increases in 30-day readmissions in the post- versus pre-SIM period. Arkansas (ratio of aOR=1.072, p=0.099) patients also had a marginally significant increase in 30-day readmissions. Readmission trends post- versus pre-SIM (ratio of aOR=0.867, p=0.258) were no different for Vermont patients compared to patients from comparison group states. The coefficients for state and time interaction terms for AR and MA are also not statistically different from one another (p=0.686) (data not shown).

Table 4.

Difference-in-Differences Logistic Regression Results: Heterogeneous SIM Effect on 30-Day Hospital Readmissions among Adults with Diabetes by State

	No pre-treatment trends adjustment (Model 4)			Pre-treatment trends adjustment (Model 5)
Variable	aOR	P-value	95% Confidence Interval	aOR	P-value	95% Confidence Interval
AR × Post	0.951**	0.040	0.907, 0.998	1.072*	0.099	0.987, 1.164
MA × Post	0.970*	0.077	0.938, 1.003	1.054**	0.012	1.011, 1.097
VT × Post	1.021	0.771	0.887, 1.174	0.867	0.258	0.676, 1.111
AR × year trend				0.961***	0.001	0.939, 0.985
MA × year trend				0.973***	0.000	0.961, 0.986
VT × year trend				1.073	0.179	0.968, 1.189
Race/ethnicity
Black	0.941***	0.000	0.924, 0.957	0.941***	0.000	0.924, 0.957
Hispanic	0.885***	0.000	0.862, 0.909	0.886***	0.000	0.863, 0.909
Other	0.840***	0.000	0.814, 0.866	0.840***	0.000	0.815, 0.866
Age	1.003***	0.000	1.002, 1.004	1.003***	0.000	1.002, 1.004
Female	0.917***	0.000	0.906, 0.927	0.917***	0.000	0.906, 0.927
Length of stay	1.037***	0.000	1.035, 1.039	1.037***	0.000	1.035, 1.039
ED admission	1.216***	0.000	1.190, 1.241	1.215***	0.000	1.190, 1.241
Comorbidity count	1.140***	0.000	1.135, 1.145	1.140***	0.000	1.135, 1.145
Payer
Medicaid	1.051***	0.000	1.024, 1.078	1.050***	0.000	1.024, 1.078
Medicare-Medicaid	1.107***	0.000	1.087, 1.129	1.107***	0.000	1.086, 1.128
Private	0.739***	0.000	0.725, 0.753	0.739***	0.000	0.725, 0.753
Uninsured	0.682***	0.000	0.644,0.722	0.682***	0.000	0.644, 0.722
Other	0.816***	0.000	0.790, 0.843	0.815***	0.000	0.790, 0.842
Goodness of Fit
AIC	1394744			1394701
BIC	1395096			1395105

Note: Models are estimated by using four-group propensity score weighting; models include year and hospital indicators (not shown). Standard errors are estimated by clustering at the hospital level. aOR: adjusted odds ratio; for interaction terms, it is the ratio of two adjusted odds ratios. AR: Arkansas; MA: Massachusetts; VT: Vermont AIC: Akaike’s information criterion; BIC: Bayesian information criterion; Post: post-SIM. Reference group for race/ethnicity is white and for payer is Medicare. Source: Authors’ analysis of Healthcare Cost and Utilization Project’s State Inpatient Databases

^*p<0.10

^**p<0.05

^***p<0.01

Discussion

We found no evidence that the CMS SIM Initiative reduced 30-day readmission rates among adults with diabetes during the early implementation period. On the contrary, we find evidence that, relative to the comparison states, two of the three states (Massachusetts and Arkansas) had small increases in 30-day readmission rates post- versus pre-SIM, while patients from Vermont had comparable 30-day readmission rates post- versus pre-SIM. These small increases may have been related to SIM reducing access barriers, resulting in additional, medically necessary readmissions. However, in the context of the SIM Initiative, it is unclear whether the pre-treatment readmission rate decreases in the three states would have persisted without SIM or would have attenuated because of diminishing returns from previous policy and payment reforms. Nevertheless, this counterfactual is unknowable and because of the common incorporation of pre-treatment trends in natural policy experiments, we conclude that the SIM Initiative did not decrease 30-day readmission rates across the three states overall.

The broad-based reforms initiated and accelerated by SIM¹ may not have been sufficiently focused on managing care transitions³⁴ to reduce 30-day hospital readmissions among adults with diabetes. Moreover, it may be difficult to reduce readmissions at a population level without greater investments in health information exchange and more intensive use of payment models that promote interorganizational coordination. The CMS Hospital Readmissions Reduction Program (HRRP)³⁰, in contrast, reduced readmission rates overall and reduced disparities for Black Medicare patients, highlighting that large scale federal incentive programs focused on reducing readmissions can have short run impact.^23,35,36

An unanticipated finding is that Black and Hispanic adults with diabetes experience lower 30-day readmission rates than non-Hispanic White adults with diabetes. A recent study of 30-day readmissions in five states similarly found that Black and Hispanic uninsured and Medicaid adult patients had lower 30-day readmission rates compared to Medicare and commercially insured patients.³⁷ The authors highlight that these lower readmission rates may be because of access-related barriers. The lower 30-day readmission rates for Black and Hispanic adults with diabetes in our study may likewise be a function of worse access to care resulting in some medically necessary readmissions not occurring.

There are important limitations to consider when interpreting our study results. First, it was not possible to use an interrupted time series with a comparison group design due to insufficient time points available in the post-SIM period.³⁸ This method allows for the intercept and slope to be different for intervention and comparison groups in the pre-treatment period and should be pursued in an analysis of the longer-term effects of SIM. Second, we could only include three of six Round 1 SIM states in the analytic sample due to HCUP SID data with readmission file linkage availability. It is unclear whether the inclusion of the omitted Round 1 SIM states (Maine, Minnesota, and Oregon) would alter our conclusions. Third, HCUP SID readmissions data availability for non-SIM states that could serve as comparisons were limited and the three comparison states used are demographically different (Supplemental Digital Content, Table A1) than the SIM states. Comparison states implemented few statewide delivery system and payment reforms during the study period (Table 1), reducing concerns about co-occurring interventions in comparison states. To address compositional differences between states, regression analyses are propensity score weighted, which balances measured patient characteristics between the SIM and comparison states. Finally, the early post-SIM period of two years is likely too short to detect significant 30-day readmission reductions, especially given recent reports of slower than expected progress for Round 1 SIM states.^1,13 Future research should clarify whether SIM effects change over each state’s four- to five-year implementation period.

In conclusion, we find no evidence that the CMS SIM Initiative reduced 30-day hospital readmission rates among adults with diabetes during the first two years of Round 1 implementation. Round 1 SIM states made targeted infrastructure investments in performance data and electronic hospital-event notifications and also made delivery system changes to enhance care coordination (Table 1),¹ but these efforts may not have been sufficiently focused on improving care transitions to accelerate reductions in 30-day readmissions among adults with diabetes. Although not statistically different, the state-specific results signal that SIM will not necessarily have consistent long-term effects on utilization and patient outcomes across grantee states. Future empirical research of the longer-term impact of SIM should explicitly examine state-specific effects. Methods for examining implementation variation for use in natural policy experiments have not been extensively developed. Given the wide range of criteria that can be considered for examining variation in SIM implementation across a small number of states, methods that parsimoniously classify states based on each states’ implementation foci and experience are needed.³⁹ These classifications of SIM across grantee states can then be linked with patient outcomes and utilization data to empirically assess the impacts of SIM based on variations in state implementation foci and experiences.

Figure. 30-Day Mean Hospital Readmission Rates among Adults with Diabetes, State Innovation Models Initiative (SIM) States vs. Comparison States.

Note: SIM= State Innovation Models Initiative. The left graph illustrates the three SIM states’ unadjusted 30-day readmission trends grouped together, while the right graph illustrates the three SIM state trends separately. For the right figure, trend lines were estimated by regressing the adjusted, annual, mean readmission rates on the relevant year range using ordinary least squares regression (Model 3). Dataset does not include 2010 data from Vermont because the rehospitalization linkage file was unavailable for the state.

Source: Authors’ analysis of Healthcare Cost and Utilization Project’s State Inpatient Databases