Early Impact of the State Innovation Models Initiative on Diagnosed Diabetes Prevalence among Adults and Hospitalizations among Diagnosed Adults

Abstract

Background:

The State Innovation Models (SIM) Initiative invested $254 million in 6 states in Round 1 to accelerate delivery system and payment reforms.

Objectives:

To examine the association of early SIM implementation and diagnosed diabetes prevalence among adults and hospitalization rates among diagnosed adults.

Research Design:

Quasi-experimental design compares diagnosed diabetes prevalence and hospitalization rates before SIM (2010–2013) and during early implementation (2014) in 6 SIM states versus 6 comparison states. County-level, difference-in-differences regression models were estimated.

Subjects:

Annual average of 4.5 million adults aged 20+ diagnosed with diabetes with 1.4 million hospitalizations in 583 counties across 12 states

Measures:

Diagnosed diabetes prevalence among adults and hospitalization rates per 1000 diagnosed adults

Results:

Compared with the pre-SIM period, diagnosed diabetes prevalence increased in SIM counties by 0.65 percentage points (from 10.22% to 10.87%) versus only 0.10 percentage points (from 9.64% to 9.74%) in comparison counties, a difference-in-differences of 0.55 percentage points. The difference-in-differences regression estimates ranged from 0.49 to 0.53 percentage points (P<0.01). Regression results for ambulatory care-sensitive condition and all-cause hospitalization rates were inconsistent across models with difference-in-differences estimates ranging from −5.34 to −0.37 and from −13.16 to 0.92, respectively.

Conclusions:

SIM Round 1 was associated with higher diagnosed diabetes prevalence among adults after a year of implementation, likely because of SIM’s emphasis on detection and care management. SIM was not associated with lower hospitalization rates among adults diagnosed with diabetes, but SIM’s long-term impact on hospitalizations should be assessed.

Summary: CMS’s State Innovation Model Initiative was associated with higher diagnosed diabetes prevalence, likely from its emphasis on detection.

I. Background and Objectives

The State Innovation Models (SIM) Initiative was launched by Centers for Medicare and Medicaid Services (CMS) to accelerate health system transformation (1–4). The SIM Initiative is an ongoing, naturally occurring federal-state “policy intervention” that provides financial and technical support to states for developing and testing state-led multipayer health care payment and service delivery models. The goals SIM are to improve health system performance, improve the quality of patient care, and decrease health care costs for all residents of the state. SIM states have emphasized different strategies to achieve their goals, including value-based payment models, accountable care organizations, health homes for individuals needing behavioral health services, and regional collaborations of medical and long-term service and support providers (1).

In order to achieve SIM Initiative’s goals, most SIM states accelerated their activities related to delivery system and payment reform strategies in order to improve the detection and management of chronic conditions, and in particular, diabetes because of its large contribution to patient morbidity and health care costs (5, 6). All state SIM plans included outcome and utilization performance measures focused on adults with diabetes (1, 2, 4).

Improving diabetes detection and management for adults with diabetes is a focus of most states’ SIM plans, because of diabetes’ increasing prevalence during the past 3 decades (7). In 2017, an estimated 24.6 million adults (or 9.7%) have been diagnosed, incurring $236 billion in direct medical costs attributable to diabetes, of which, ~29% were for hospital inpatient costs (5). In 2014, there were 7.2 million hospitalizations of adults with a diabetes diagnosis, a rate of 327 hospitalizations per 1000 diagnosed adults (8).

The 3 types of SIM awards include Design Awards that fund states to design a State Health Care Innovation Plan, Pre-Test Awards that fund states to continue developing their State Health Care Innovation Plans, and Test Awards that funds states to test and implement delivery system reforms. Since 2013, these awards have invested > 1 billion dollars in states’ efforts to design, test and implement delivery system reforms. The SIM Initiative is now supporting the testing and implementation of SIM plans in 17 states: in April 2013, 6 states received 42 months of test-award funding totaling $254 million (“Round 1” states); in January 2015, 11 states received 48 months of test-award funding totaling $622 million (“Round 2” states) (9).

In this study, we analyze whether counties in states that received a SIM test award in Round 1 experienced an increase in diagnosed diabetes prevalence among adults and a decrease in hospitalization rates among diagnosed adults, relative to a select group of counties in comparison states that applied for SIM funding, but did not receive a SIM test award in Round 1, although eventually did in Round 2. These research questions are of interest, because SIM may have accelerated improvements in the detection and management of diabetes—either by directly targeting diabetes or targeting chronic conditions more broadly—resulting in a higher diagnostic prevalence and lower hospitalization rates among diagnosed adults.

Our hypotheses are:

Hypothesis 1: The SIM Initiative improved the detection of diabetes in the short-term, resulting in more adults being diagnosed with diabetes; and

Hypothesis 2: The SIM Initiative improved the management of diabetes in the short-term, resulting in fewer hospitalizations among adults diagnosed with diabetes.

II. Methods

II.A. Measures

II.A.1. Outcome Variables

The outcome variables include diagnosed diabetes prevalence among adults, ambulatory care-sensitive condition (ACSC) hospitalizations per 1000 diagnosed adults, and all-cause hospitalizations per 1000 diagnosed adults. These outcomes were measured at the county-year level for adults (aged 20+ y old) from 2010 to 2014, because that was the geographic level and frequency of the diagnostic prevalence data (discussed below). An adult was considered diagnosed with diabetes based on their response to the Behavioral Risk Factor Surveillance System question that asked “Has a doctor ever told you that you have diabetes?”; women who only had been diagnosed with gestational diabetes were not considered diagnosed (10).

Hospitalizations among adults diagnosed with diabetes were identified based on Agency for Healthcare Research and Quality (AHRQ) specifications using the Healthcare Cost and Utilization Project (HCUP) Clinical Classifications Software (CCS), which categorizes ICD-9-CM diagnosis codes into a clinical grouper to understand patterns of diagnoses and procedures (11). The included hospitalizations were diabetes without complication (CCS=49) and diabetes with complications (CCS=50), which were extracted from up to 50 principal and secondary diagnosis fields per patient. Hospitalizations for pregnancies and hospital transfers were excluded. Hospitalizations that were due to ACSCs were analyzed separately because these hospitalizations are considered preventable through improved primary care management of diabetes, which was a major focus of SIM. We identified these hospitalizations from ICD-9-CM codes that were extracted from the principal diagnosis field of each patient using the following 2015 AHRQ Prevention Quality Indicators: #1 Diabetes Short-Term Complications, #3 Diabetes Long-Term Complications, #5 Chronic Obstructive Pulmonary Disease or Asthma in Older Adults, #7 Hypertension, #8 Heart Failure, #10 Dehydration, #11 Bacterial Pneumonia, #12 Urinary Tract Infection, #13 Angina Without Procedure, #14 Uncontrolled Diabetes, #15 Asthma in Younger Adults, and #16 Lower-Extremity Amputation among Patients with Diabetes (12). Annual county-level hospitalization rates were calculated by dividing the number of hospitalizations by the number of adults (in 1,000s) diagnosed with diabetes.

II.A.2. Key Independent Variable

The key independent variable was defined at the county-year level indicating whether a county was in a state that received a SIM Initiative, 42-month test award in Round 1 and whether the year was in the post-intervention period (9). The 6 SIM states included Arkansas, Maine, Massachusetts, Minnesota, Oregon and Vermont (see Fig. A1 in the Appendix, Supplemental Digital Content 1, http://links.lww.com/MLR/B678). SIM funding began on April 1, 2013, which included an initial 6-month test period, resulting in the test implementation phase beginning October 1, 2013 (1, 4). Because the diagnosed diabetes prevalence data are annual, we modeled the intervention period as beginning January 1, 2014. The analytic sample for the 6 SIM states included 240 counties from 2010 to 2014 (or 1200 county-year observations) with a mean adult population of 15.6 million during those years.

The comparison states included 6 states that received test awards in Round 2, whose funding began in January 2015 for 48 months: Colorado, Iowa, Michigan, New York, Rhode Island and Washington. These comparison states were selected because they were similar to the Round 1 test states, based on having received a design award in Round 1—thus demonstrating stakeholder engagement and a collaborative infrastructure for implementing state SIM plans—and providing data to the HCUP State Inpatient Databases. The analytic sample for the 6 comparison states included 343 counties from 2010 to 2014 (or 1715 county-year observations) with a mean adult population of 34.0 million during those years.

II.B. Data

Encounter-level hospitalization data were from the State Inpatient Databases maintained by HCUP. These administrative data include a census of all hospitalizations in the 12 states comprising of 594 counties from 2010 to 2014; 11 counties were excluded for incomplete data resulting in 583 counties with 6.8 million hospitalizations of adults diagnosed with diabetes, of which, 22.5% were for ACSCs.

Data on the number and prevalence of adults diagnosed with diabetes per county-year were from the Centers for Disease Control and Prevention’s “County Data Indicators” (10, 13). The analytic sample of 583 counties included an average of 49.6 million adults from 2010 to 2014, including an average of 4.5 million diagnosed with diabetes.

The Area Health Resources File was used to obtain data on county-level factors related to diagnosed diabetes prevalence and hospitalization rates, including gender, age, race/ethnicity, uninsured rate, and poverty rate (household income < 100% of the federal poverty level), which were available for each county’s population.

II.C. Analytic Approach

Hypotheses 1 and 2 were tested by estimating county-level, difference-in-differences regression models that had the structure of equation (1) in which c indexes counties and t indexes years; outcome_c,t is the percentage of adults diagnosed with diabetes (Hypotheses 1) or the number of hospitalizations per 1000 diagnosed adults (Hypotheses 2); SIM_c,t indicates implementation of a Round 1 SIM test award; year_t is a vector of year indicator variables for each year (except 2010, the reference year) to control for trends in the outcome measures across all counties; county_c is a vector of county fixed effects to control for time-invariant outcome differences among the counties; X_c,t is a vector of time-varying control variables; ε_c,t is the error term; and β₁ is the parameter of interest. The X_c,t vector includes gender, age, race/ethnicity, uninsured rate, and poverty rate, which are known to affect diagnosed diabetes prevalence among adults and hospitalizations of diagnosed adults (5, 14). These variables are measured at the county level as means for each county’s population. The regression models were not weighted for a county’s population so that each county would be considered an equal unit of analysis. Standard errors were estimated by clustering at the county level to allow for correlation within counties across time (15).

$$outcom{e}_{c,t}={\beta }_0+{\beta }_{1}SI{M}_{c,t}+{\beta }_{2}yea{r}_t+{\beta }_{3}count{y}_c+{\beta }_4{X}_{c,t}+{\epsilon }_{c,t}$$ (1)

The first set of difference-in-differences models are from equation (1). This model controls for time-invariant outcome differences between SIM and comparison counties, secular outcome trends, and time-varying cofounding variables. However, it does not control for pre-intervention differences that affect the post-intervention period, manifesting in two distinct concerns.

First, there is concern if pre-intervention outcome trends are not parallel between SIM and comparison counties—and those non-parallel trends would have persisted in the post-intervention period absent the intervention—then the difference-in-differences regression estimates will be biased (16, 17). We examined the pre-SIM diagnostic prevalence and hospitalization rate trends between the SIM and comparison counties by modifying equation (1) in the following manner: changed SIM to be non-time varying, changed year to be continuous, and added the interaction of SIM and year. As compared with comparison counties, SIM counties experienced a slighter higher pre-intervention trend for diagnostic prevalence (1.3 percentage points higher per year, P=0.41). However, comparison counties experienced a decreasing pre-intervention trend that has significantly steeper than SIM counties’ trend for the ACSC hospitalization rate (1.9 hospitalizations per 1000 diagnosed adults steeper decrease per year, P<0.01) and for the all-cause hospitalization rate (5.8 hospitalizations per 1000 diagnosed adults steeper decrease per year, P<0.01). To address this issue, in a second set of models we added county-year trends to equation (1) by interacting each county with year as a continuous variable (16, 18).

A second concern is that differences between SIM and comparison counties for the confounding variables in the pre-intervention period affect outcomes in the post-intervention period. Hence, in the second set of models we used propensity score methods to balance the covariates in the pre-intervention period (19). This approach allows for statistically assessing whether comparison counties are equivalent to SIM counties for the confounding covariates after balancing. Our goal was to have the pre-intervention period’s (i.e., 2010 to 2013) absolute standardized differences of the variables’ means between the SIM and comparison counties to be less than 0.10 to 0.25 (20, 21). To operationalize this approach, we used Stata 14.0: -probit- was used to estimate propensity scores; the propensity scores with common support were used to calculate stabilized inverse probability of treatment weights (IPTW) in the form of average treatment effect in the treated (22), which is consistent with a difference-in-differences regression (17); and -pbalchk- was used to calculate the standardized difference between the SIM and comparison counties for each covariate. The IPTWs were used as probability weights to estimate equation (1) above (including county-year trends), hence, making it “doubly robust” (23). The doubly robust approach has been used to study the impact of Medicare hospice enrollment on costs and quality (24).

While adding county-year trends and weighting the difference-in-differences models for pre-intervention covariate differences is designed to reduce bias, their incorporation can introduce bias when they are used to minimize pre-intervention outcome trends and covariate differences that are transient due to random noise (25–27). For example, minimizing the pre-SIM relative outcome trends between SIM and comparison counties relaxes the parallel trends assumption only if the relative trends would have continued in the intervention period absent the SIM intervention. In our context, it is unclear whether the hospitalization-rate trend differences would have persisted without SIM, for example, because of continuously improving diabetes care management in the comparison counties, or would have receded because of diminishing marginal returns, or would have even reversed, that is, regressed to the mean because the trend differences were actually random noise. Hence, we examine difference-in-differences results excluding county-year trends and IPTWs in the first set of models, as recommended by Lindner and McConnell (27).

This study was approved by the University of California, Berkeley, Institutional Review Board.

III. Results

Table 1 compares the unadjusted outcome measures, demographic characteristics and health characteristics for adults and for hospitalized adults diagnosed with diabetes in the SIM versus the comparison counties from 2010 to 2014. The diagnosed diabetes prevalence was higher in SIM versus the comparison counties (10.3% vs. 9.7%), while the all-cause hospitalization rate was lower (237.8 vs. 258.1) and ACSC hospitalization rate was similar (59.8 vs. 59.9). At the population level, the SIM counties were demographically similar to the comparison counties, with the largest difference being that 16.0% of households had incomes < 100% of the federal poverty level in SIM counties versus 14.7% in the comparison counties. Among hospitalized adults diagnosed with diabetes, the share with Medicaid was higher in SIM counties (22.6%) versus the comparison counties (14.3%), and the share admitted through the emergency department was also higher in SIM counties (49.9%) versus the comparison counties (32.9%). The mean number of comorbidities was lower in SIM counties (3.5) versus the comparison counties (3.8).

Table 1:

County-Level Outcomes, Demographics, and Health Characteristics of the Adult Population and Hospitalized Adults Diagnosed with Diabetes for SIM versus Comparison Counties, 2010 to 2014

Variable	SIM Counties (mean)	SIM Counties (std dev)	Comparison Counties (mean)	Comparison Counties (std dev)	Difference	P-value
Outcome Variables (levels)
Diagnosed diabetes prevalence (%)	10.3	2.3	9.7	2.0	0.7	<0.001
All-cause hospitalizations per 1000 adults diagnosed with diabetes	237.8	76.1	258.1	76.1	−20.3	0.002
ACSC hospitalizations per 1000 adults diagnosed with diabetes	59.8	20.9	59.9	20.7	−0.2	0.924
Outcome Variables (annual change)
Diagnosed diabetes prevalence (pp)	0.20	0.30	0.10	0.30	0.10	<0.001
All-cause hospitalizations per 1000 adults diagnosed with diabetes	−7.5	18.8	−10.9	22.1	3.3	0.058
ACSC hospitalizations per 1000 adults diagnosed with diabetes	−2.5	5.0	−3.5	6.5	1.0	0.039
Adult Population Independent Variables (%)
Female*	50.3	1.4	49.8	2.1	0.6	<0.001
Age 20−39 y	32.1	5.3	31.8	6.3	0.2	0.622
Age 40−64 y	44.9	2.7	45.4	3.3	−0.6	0.022
Age 65+ y	23.1	4.6	22.7	4.9	0.3	0.395
Nonwhite*	15.5	13.4	15.1	13.6	0.4	0.725
Uninsured*	14.0	5.4	13.4	4.0	0.6	0.098
Household income less than 100% FPL*	16.0	5.8	14.7	4.6	1.3	0.003
Hospitalized Adults Diagnosed with Diabetes (% unless indicated)
Female	51.0	4.0	49.9	3.4	1.1	<0.001
Age 20−44 y	7.9	2.7	7.0	2.5	0.8	<0.001
Age 45–64 y	31.3	5.4	31.5	5.5	−0.1	0.767
Age 65+ y	60.8	7.4	61.5	7.3	−0.7	0.264
Nonwhite	14.4	15.6	11.1	14.6	3.4	0.020
Medicaid	22.6	10.3	14.3	9.0	8.3	<0.001
Admitted from emergency department	49.9	20.4	32.9	27.9	17.0	<0.001
Number of comorbidities (mean)	3.5	0.4	3.8	0.3	−0.3	<0.001

^*Due to data availability, these statistics are based on the total population in a county, not just the adult population.

ACSC indicates ambulatory care-sensitive conditions; FPL: federal poverty level; pp: percentage points; SIM: State Innovation Model; Std dev: standard deviation

Notes: Statistics are based on county-level data, including 240 SIM counties and 343 comparison counties, for years 2010–2014 combined. The reported differences may not equal the differences between SIM and comparison counties due to rounding.

Sources: Authors’ analysis Centers for Disease Control and Prevention’s county-level estimates, Healthcare Cost and Utilization Project’s State Inpatient Databases, and Area Health Resources File

During the study period, the unadjusted, mean diagnosed diabetes prevalence among adults increased in both SIM and comparison counties, and the rate of increase during the pre-SIM period was similar between the two sets of counties (see Figure A2 in the Appendix, Supplemental Digital Content 1, http://links.lww.com/MLR/B678). However, in 2014, the prevalence continued to increase in SIM counties while it slightly decreased in comparison counties. Figure 1 plots the unadjusted, mean difference in prevalence between SIM and comparison counties from 2010 to 2014. Before SIM implementation in 2014, the prevalence estimates were 0.39 to 0.76 percentage points higher in the SIM counties as opposed to the comparison counties. In 2014, this difference increased to 1.13 percentage points.

Figure 1: Unadjusted, Mean Differences in Diagnosed Diabetes Prevalence Among Adults for SIM versus Comparison Counties, 2010–2014.

Notes: SIM funding began on Apr 1, 2013, which included an initial 6-month test period, resulting in the test implementation phase beginning Oct 1, 2013. Because the diagnosed diabetes prevalence data is annual, we modeled the implementation period as beginning Jan 1, 2014. Prevalence differences were calculated each year by subtracting the mean of the county-level prevalence in the 240 SIM counties from the mean of the county-level prevalence in the 343 comparison counties. The error bars are 95% confidence intervals of the difference in the mean prevalence by year. Apr indicates April; Dec, December; Jan, January; Oct, October; SIM, State Innovation Model

Source: Authors’ analysis of Centers for Disease Control and Prevention’s county-level estimates

During the study period, the unadjusted, mean ACSC hospitalization rate per 1000 adults diagnosed with diabetes decreased in both SIM and comparison counties, but the rate decreased faster in comparison counties up to 2013; in 2014 the comparison rate slightly increased while the SIM rate slightly decreased (see Figure A3 in the Appendix, Supplemental Digital Content 1, http://links.lww.com/MLR/B678). Figure 2 plots the unadjusted, mean difference in rates between SIM and comparison counties from 2010 to 2014. Before SIM implementation in 2014, these rates ranged from being 4.0 lower to 2.2 higher—trending toward higher—in the SIM counties as opposed to the comparison counties. In 2014, this difference decreased to −0.2.

Figure 2: Unadjusted, Mean Differences in ACSC Hospitalization Rate per 1000 Adults Diagnosed with Diabetes for SIM versus Comparison Counties, 2010–2014.

Notes: SIM funding began on Apr 1, 2013, which included an initial 6-month test period, resulting in the test implementation phase beginning Oct 1, 2013. Because the diagnosed diabetes prevalence data is annual, we modeled the implementation period as beginning Jan 1, 2014. Hospital rate differences were calculated each year by subtracting the mean of the county-level hospitalization rate in the 240 SIM counties from the mean of the county-level hospitalization rate in the 343 comparison counties. The error bars are 95% confidence intervals of the difference in the mean hospitalization rate by year. ACSC indicates ambulatory care-sensitive condition; Apr, April; Dec, December; Jan, January; Oct, October; SIM, State Innovation Model

Sources: Authors’ analysis of Centers for Disease Control and Prevention’s county-level estimates and Healthcare Cost and Utilization Project’s State Inpatient Databases

During the study period, the unadjusted, mean all-cause hospitalization rate per 1000 adults diagnosed with diabetes also decreased in both SIM and comparison counties, but the rate decreased faster in comparison counties up to 2013; in 2014 the comparison rate slightly increased while the SIM rate slightly decreased (see Figure A4 in the Appendix, Supplemental Digital Content 1, http://links.lww.com/MLR/B678). Figure 3 plots the unadjusted, mean difference in rates between SIM and comparison counties from 2010 to 2014. Before SIM implementation in 2014, the hospitalization rates were 12.7 to 33.1 lower in the SIM counties as opposed to the comparison counties, with the trend showing a decrease in the difference. In 2014, the difference was 19.3.

Figure 3: Unadjusted, Mean Differences in All-Cause Hospitalization Rate per 1000 Adults Diagnosed with Diabetes for SIM versus Comparison Counties, 2010–2014.

Notes: SIM funding began on Apr 1, 2013, which included an initial 6-month test period, resulting in the test implementation phase beginning Oct 1, 2013. Because the diagnosed diabetes prevalence data is annual, we modeled the implementation period as beginning Jan 1, 2014. Hospital rate differences were calculated each year by subtracting the mean of the county-level hospitalization rate in the 240 SIM counties from the mean of the county-level hospitalization rate in the 343 comparison counties. The error bars are 95% confidence intervals of the difference in the mean hospitalization rate by year. Apr indicates April; Dec, December; Jan, January; Oct, October; SIM, State Innovation Model

Sources: Authors’ analysis of Centers for Disease Control and Prevention’s county-level estimates and Healthcare Cost and Utilization Project’s State Inpatient Databases

Table 2 shows the regression parameter estimates for the difference-in-differences models for diagnosed diabetes prevalence among adults and the hospitalization rates per 1000 diagnosed adults. The first column of results are based on the standard difference-in-differences model, and the second column of results are based on the difference-in-difference model that includes county-year trends and is estimated using IPTWs. (See Table A1 in the Appendix, Supplemental Digital Content 1, http://links.lww.com/MLR/B678 that shows the covariate standardized differences between the SIM and comparison counties before and after applying IPTWs, which reduced these differences to less than 20% for each variable.) Compared with the pre-SIM period, the diagnosed diabetes prevalence during early SIM implementation increased in SIM counties by 0.65 percentage points (from 10.22% to 10.87%), but only increased in comparison counties by 0.10 percentage points (from 9.64% to 9.74%), resulting in an unadjusted difference-in-differences of 0.55 percentage points. In the table, the adjusted difference-in-differences estimate was 0.53 percentage points (P<0.01), similar to the 0.49 percentage points (P<0.01) adjusted difference-in-differences estimate that accounted for pre-SIM differences by including county-year trends and incorporating IPTWs.

Table 2:

Difference-in-Differences Regression Results for Diagnosed Diabetes Prevalence and Hospitalization Rates

	Difference-in-Differences (N=2,915)		Difference-in-Differences (accounts for pre-SIM differences) (N=2,900)
	Coefficient Estimate	95% Confidence Interval	Coefficient Estimate	95% Confidence Interval
Variable	Diagnosed Diabetes Prevalence
SIM Initiative	0.53***	(0.39, 0.67)	0.49***	(0.33, 0.65)
Female	−0.03	(−0.22, 0.15)	0.07	(−0.13, 0.28)
Age 40–64 y	0.02	(−0.07, 0.10)	0.08	(−0.04, 0.20)
Age 65+ y	0.03	(−0.06, 0.12)	−0.05	(−0.20, 0.10)
Nonwhite	0.03	(−0.10, 0.15)	0.10	(−0.06, 0.26)
Uninsured	−0.05***	(−0.08, −0.02)	−0.02	(−0.05, 0.01)
Household income < 100% FPL	0.00	(−0.02, 0.03)	−0.01	(−0.03, 0.01)
Year 2011	0.26***	(0.19, 0.33)	0.32***	(0.18, 0.46)
Year 2012	0.33***	(0.20, 0.46)	0.54***	(0.27, 0.80)
Year 2013	0.42***	(0.23, 0.61)	0.73***	(0.33, 1.12)
Year 2014	0.11	(−0.18, 0.39)	0.62**	(0.06, 1.19)
Constant	10.11*	(−0.55, 20.77)	2.55	(−11.18, 16.27)
Variable	Ambulatory Care-Sensitive Condition Hospitalization Rate
SIM Initiative	−0.37	(−2.64, 1.89)	−5.34***	(−8.54, −2.14)
Female	0.24	(−6.73, 7.21)	−1.86	(−8.30, 4.58)
Age 40–64 y	1.87*	(−0.15, 3.89)	0.36	(−2.75, 3.48)
Age 65+ y	2.39**	(0.07, 4.70)	1.60	(−2.67, 5.87)
Nonwhite	−1.35	(−4.01, 1.31)	−1.63	(−5.84, 2.58)
Uninsured	0.34	(−0.22, 0.90)	−0.43	(−1.19, 0.34)
Household income < 100% FPL	0.03	(−0.41, 0.46)	0.21	(−0.20, 0.63)
Year 2011	−3.07***	(−4.68, −1.47)	−6.78***	(−10.35, −3.21)
Year 2012	−9.39***	(−11.97, −6.81)	−17.16***	(−24.24, −10.08)
Year 2013	−10.72***	(−14.58, −6.85)	−21.78***	(−32.55, −11.02)
Year 2014	−9.44***	(−15.01, −3.86)	−24.96***	(−40.26, −9.66)
Constant	−69.06	(−445.14, 307.01)	137.44	(−230.64, 505.51)
Variable	All-Cause Hospitalization Rate
SIM Initiative	0.92	(−6.67, 8.51)	−13.16***	(−21.34, −4.97)
Female	−3.57	(−23.69, 16.54)	−7.01	(−20.65, 6.62)
Age 40–64 y	5.37	(−1.50, 12.25)	0.19	(−8.54, 8.92)
Age 65+ y	5.30	(−1.41, 12.01)	7.57	(−3.43, 18.57)
Nonwhite	−7.01*	(−14.92, 0.89)	−10.05**	(−18.87, −1.24)
Uninsured	1.68*	(−0.09, 3.45)	−0.37	(−2.12, 1.38)
Household income < 100% FPL	0.18	(−1.14, 1.50)	0.14	(−1.02, 1.30)
Year 2011	−12.52***	(−16.77, −8.27)	−20.35***	(−28.77, −11.94)
Year 2012	−27.63***	(−35.40, −19.86)	−48.06***	(−64.93, −31.19)
Year 2013	−29.32***	(−42.45, −16.20)	−59.05***	(−83.81, −34.30)
Year 2014	−21.33**	(−38.82, −3.83)	−63.59***	(−98.84, −28.34)
Constant	163.64	(−956.63, 1283.91)	607.29	(−228.75, 1443.34)

^*P<0.10,

^**P<0.05,

^***P<0.01

FPL indicates federal poverty level; SIM, State Innovation Model

County fixed effects and county-year trend parameter estimates not shown. For the county-year trends, year was coded as a continuous variable equal to the actual year minus 2010 so the first value would be zero; this coding only affects the estimate of the constant. Hospitalization rate is the number of hospitalizations per 1000 adults diagnosed with diabetes. The reference group for age was 20–39 year olds and for year was 2010. The sample size for the difference-in-differences model that accounted for pre-SIM differences had a smaller sample size due to dropping counties without common support.

Sources: Authors’ analysis of Centers for Disease Control and Prevention’s county-level estimates, Healthcare Cost and Utilization Project’s State Inpatient Databases, and Area Health Resources File

Compared with the pre-SIM period, the hospitalization rate for ACSCs during early SIM implementation decreased in SIM counties by 5.6 (from 60.9 to 55.3), similar to the decrease in comparison counties of 5.5 (from 61.0 to 55.5), resulting in an unadjusted difference-in-differences of −0.1. In the table, the adjusted difference-in-differences estimate was −0.4 (P=0.75). In contrast, the adjusted difference-in-differences estimate that accounted for pre-SIM differences was −5.3 (P<0.01) because the hospitalization rate for ACSCs was decreasing more quickly in comparison counties relative to SIM counties prior to the SIM intervention.

Compared with the pre-SIM period, the all-cause hospitalization rate during early SIM implementation decreased in SIM counties by 15.2 (from 240.8 to 225.6), similar to the decrease in comparison counties of 16.4 (from 261.4 to 245.0) resulting in an unadjusted difference-in-differences of 1.2. In the table, the adjusted difference-in-differences estimate was 0.92 (P=0.81). In contrast, the adjusted difference-in-differences estimate that accounted for pre-SIM differences was −13.2 (P<0.01) because the all-cause hospitalization rate was decreasing more quickly in comparison counties relative to SIM counties prior to the SIM intervention.

IV. Discussion

The results indicate that the first year of the CMS SIM Initiative Round 1 test award was associated with higher diagnosed diabetes prevalence among adults, but results related to lower ACSC and all-cause hospitalization rates among adults diagnosed with diabetes are mixed and sensitive to model specification decisions. Compared with comparison counties, SIM counties on average experienced between a 0.49 to 0.53 percentage point higher regression-adjusted increase in diagnostic prevalence before versus after SIM implementation. Based on the adult population in SIM counties totaling 15.9 million in 2014, the two estimated increases in diagnostic prevalence would have led to between 77,910 (95% confidence interval: 52,470—103,350) and 84,270 (95% confidence interval: 62,010—106,530) newly diagnosed adults. The implementation of SIM may have contributed to improving detection of diabetes through increased screening, particularly in high-risk populations with limited access to health care. Earlier detection of diabetes via screening has been linked to lower health care costs (28). Furthermore, improved detection of diabetes is associated with better outcomes, because undiagnosed diabetes has been found to be associated with more hospitalizations, longer lengths of stay, and increased mortality (29).

However, the first year of SIM implementation in Round 1 was not found to be consistently associated with a lower hospitalization rate, either for ACSCs or all-causes, which may have been due to the initiative’s design and incentives as well as the capabilities of the health care providers. Reviews of studies on accountable care organizations (30) and patient-centered medical homes (31, 32), both which share delivery-system and payment-reform attributes of SIM, found evidence that these programs can have a positive impact on health care utilization, quality, and health outcomes, but the results were mixed and depended on a program’s design, financial incentives, and patient population. For example, delivery and payment reforms have modestly reduced health care spending, including hospital expenditures, in the short term among early provider participants, likely because they had more advanced systems to manage care (33). Better management of hemoglobin A1c levels may reduce hospitalizations, because higher hemoglobin A1c levels are a predictor of hospitalizations in adults with diagnosed diabetes, undiagnosed diabetes, and pre-diabetes (34). Future research will evaluate the long-term effects of the SIM Initiative (35).

When pre-intervention hospitalization rate trend differences were accounted for in regression models, then SIM was estimated to reduce both the ACSC and all-cause hospitalization rates among adults diagnosed with diabetes. In the pre-intervention period, these hospitalization rates were decreasing faster in the comparison counties, but the relative trends reversed in the post-intervention period. Because we do not know whether the pre-intervention trends would have continued in the post-intervention period (absent SIM), the interpretation of this finding is unclear. On the one hand, the non-parallel trends may have continued if existing healthcare delivery, payment reform and unobserved demographic factors continued to improve in the comparison counties relative to the SIM counties. On the other hand, the non-parallel trends may not have continued if there were diminishing effects of these factors over time.

This study has limitations that warrant discussion. First, a difference-in-differences regression model assumes there was not an intervention that occurred contemporaneously with SIM that affected diagnosed diabetes prevalence among adults or hospitalization rates among diagnosed adults. During 2014, the majority of the Affordable Care Act’s insurance expansion occurred, but there was significant county-level heterogeneity. However, our models controlled for each county’s uninsured rate. Eleven of the 12 states in our models expanded Medicaid in 2014, with the exception being Maine (36). Therefore, we do not think the Affordable Care Act’s insurance expansion biased the parameter estimates. However, during 2014, the Center for Medicare and Medicaid Innovation sponsored several initiatives similar to SIM, including primary care initiatives, accountable care organization models, and efforts to improve the care of Medicare-Medicaid dually eligible patients (37). These initiatives may have affected diagnostic prevalence and hospitalization rates for diagnosed adults differently between SIM and comparison counties, but it was not feasible to measure and incorporate these initiatives into our analyses. In addition, the SIM Initiative is implemented differently across states and not uniformly among counties within states. Hence, this would attenuate the measured effects of SIM, because in some counties, it may not have been implemented to target diabetes detection and management. Finally, lack of statistical power limited our ability to estimate heterogeneous effects by stratifying SIM states based on implementation foci, but that will be possible in a future study that includes all 12 SIM Round 1 and Round 2 states as treatment states with a longer post-intervention period (2014 to 2017) (35).

Preventing, diagnosing and managing diabetes is a high priority for most states, because of its increasing prevalence and growing economic burden (5, 7). The CMS SIM Initiative appears to have had early promise in accelerating improvements in diabetes detection, but has yet to clearly show improved management at the population-level with respect to hospitalization rates among diagnosed adults. When the SIM Initiative Round 1 and 2 states have completed their SIM implementation and that data become available, it will be important to assess whether the foci and resourcing of SIM among states were differentially associated with improved diabetes detection. Given the latitude CMS gave states when developing their SIM plans, identifying the specific reforms and investments used by states with relatively more success in improving outcomes could improve the nation’s overall investment in state-led innovation.