Quality of Care Improves for Patients with Diabetes in Medicare Shared Savings Accountable Care Organizations: Organizational Characteristics Associated with Performance

Abstract

Accountable care organizations (ACOs), a primary care-centric delivery and payment model, aim to promote integrated population health, which may improve care for those with chronic conditions such as diabetes. Research has shown that, overall, the ACO model is effective at reducing costs, but there is substantial variation in how effective different types of ACOs are at impacting costs and improving care delivery. This study examines how ACO organizational characteristics – such as composition, staffing, care management, and experiences with health reform – were associated with quality of care delivered to patients with diabetes. Secondary data were analyzed retrospectively to examine Medicare Shared Savings Program (MSSP) ACOs' performance on diabetes metrics in the first 2 years of ACO contracts. Ordinary least squares was used to analyze 162 MSSP ACOs with publicly available performance data and the National Survey of ACOs. ACOs improved performance significantly for patients with diabetes between contract years 1 and 2. In year 1, also having a private payer contract and an increased number of services within the ACO were positively associated with performance, while having a community health center or a hospital were negatively associated with performance. Better performance in year 1 was negatively associated with improved performance in year 2. This study found that ACOs substantively improved diabetes management within initial contract years. ACOs may need different types of support throughout their contracts to ensure continued improvements in performance.

Introduction

Diabetes is a complex chronic condition associated with poor health outcomes and complications,^1–5 in part because of the challenging nature of coordinating care across providers,^5–8 the importance of patient adherence to lifestyle changes,^9–11 and the frequent occurrence of comorbidities.^12,13 To be optimally controlled, persons with diabetes must manage their blood sugar, cholesterol, blood pressure, and avoid tobacco products⁴; however, only approximately 14% of adults with diabetes are optimally controlled across these metrics.¹⁴ Research has shown that health care organizations, and especially primary care providers,^15,16 can improve quality of care by providing those with chronic conditions proactive, coordinated, team-based care that enables patients to self-manage their conditions effectively.^5,17 Under fee-for-service payment models, health care providers may have limited financial incentives to engage in activities such as coordinating care, spending extra time counseling patients on lifestyle changes, or providing ongoing care management.¹⁸

Accountable care organizations (ACOs), a prominent care delivery and payment reform effort, aim to transition providers away from individual care silos promoted by fee-for-service payment toward integrated population health management.^19,20 Medicare Shared Savings Program (MSSP) ACOs are provider groups contractually responsible for quality and total cost of care for a defined group of Medicare beneficiaries. As part of their contract, MSSP ACOs are measured on 5 clinical outcomes that indicate the level of diabetes control as well as an all-or-nothing composite.²¹ Proponents of ACOs hope that both ACOs' modified reimbursement model and quality performance incentives will encourage providers to invest in care management across the continuum of care,^22,23 particularly for patients with chronic conditions such as diabetes, which can be especially challenging to manage given the need for patient lifestyle changes and patient adherence for optimal control.^9–11 ACOs likely experience many challenges to improving care management, including lack of financial and staff resources to devote to care management, difficulty in developing infrastructure that facilitates coordination, and implementing team-based care processes.^24,25

Early evidence on ACOs shows there is remarkable variation in ACO performance.^26,27 Further, research has shown substantial variation in the composition, organizational structure, leadership, and reported capabilities of ACOs; it is uncertain how these differences may affect quality and cost performance.^25,28–34 One study found that ACOs that had more patients assigned, and fewer specialists, and those ACOs participating in advanced models performed better on preventive care measures.³⁰ Another study examined differences in cost savings for MSSP ACOs and observed different effects by ACO start date; ACOs that began in 2012 lowered spending significantly compared to a control group, while ACOs that began in 2013 did not differ significantly compared to controls. Researchers also found that ACOs not integrated with a hospital had greater savings than those that were integrated with a hospital.³⁵

No research to date has examined the association between ACO characteristics and quality of care for patients with diabetes. ACOs may be well positioned to improve overall care for patients with diabetes because ACOs have greater financial flexibility to provide care coordination activities and promote self-management among those with chronic conditions. Previous research does indicate, however, that improvements in care may be varied among ACOs. This study examines ACO factors hypothesized to be associated with performance variation (ie, ACO composition, staffing levels, care management capabilities, experiences with health reform) and how each of these factors was associated with the quality of care delivered to patients with diabetes.

Methods

MSSP ACOs' data were used to conduct longitudinal and cross-sectional analyses on organizational and patient characteristics' association with 2 years of quality performance for patients with diabetes. Specifically, data were used from the National Survey of ACOs (NSACO) on organizational characteristics along with publicly available data from the Centers for Medicare & Medicaid Services (CMS) on ACO quality performance and patient characteristics. This research was reviewed and approved by the Institutional Review Board at Dartmouth College.

NSACO

NSACO provided data on ACOs including contracts with payers, organizational characteristics, capabilities, activities, and prior experience with reform.^25,28 Executive-level ACO leaders, such as directors or medical directors, who understand ACO structure and capabilities were targeted as respondents. The survey collected data from newly formed ACOs in 3 waves from 2012–2015 and had a 69% response rate across survey waves for all Medicare ACOs. These data uniquely report organizational characteristics and capabilities of ACOs. Measures of ACO composition, types of services provided by the ACO, and contracts were used for the analyses. ACO composition was measured by whether or not the ACO included a hospital or a community health center. A measure was created to assess the breadth of services offered by the ACO by counting the number of services offered: emergency care, specialized care, behavioral health, skilled nursing, pediatric services, home health, and pharmacy. Finally, a measure indicating whether or not the ACO had an additional commercial payer contract was included in the analyses.

MSSP data

CMS collected and published data on MSSP ACOs for cost and quality performance metrics as well as some data on ACO patient population and provider characteristics.^21,36,37 CMS published quality performance data on 33 measures, which included 6 measures of diabetes care, with an all-or-nothing composite in performance years 2013 and 2014. The study team used first year performance and characteristics data for ACOs that began in 2012, 2013, and 2014; second year performance data were available for ACOs that began in 2012 and 2013.

These analyses focused on the 6 diabetes measures; all were continuous measures whereby a higher score indicated a greater proportion of eligible ACO patients met the metric. ACO patients with either 1 inpatient or 2 outpatient visits with a diabetes-related diagnosis code within 1 year prior to measurement were eligible. Five measures focused on how well patients with diabetes were controlled on: (1) blood sugar (HbA1c <8.0%); (2) cholesterol (low-density lipoprotein cholesterol <100mg/dL); (3) blood pressure (<140/90 mmHg); (4) tobacco nonuse; and (5) daily aspirin (when indicated). Patients who were well-controlled on each component measure also met a diabetes composite metric.

Measures of provider and patient characteristics from CMS were included in the analyses: proportion of providers who were primary care physicians, 3 controls for Hierarchical Condition Category (HCC) scores to risk adjust for patient morbidity (aged non-dual, aged dual, and disabled), proportion of patients aged 85 and older, proportion of patients aged 64 and younger, proportion female, and proportion of minority (non-white) patients.

Analyses

The analyses examined the association of ACO organizational and patient population characteristics with first year performance and change in performance from year 1 to year 2 on diabetes metrics. The study team analyzed MSSP ACOs with at least 1 year of performance data and valid responses on NSACO measures (N = 162). Sensitivity analyses showed that ACOs with survey data had slightly higher year 1 performance, but there were no consistent differences in year 2 performance compared to ACOs without survey data. The team used linear models to regress performance for each diabetes metric on ACO organizational and patient population characteristics. These analyses allowed for the examination of the association of both organizational and patient population characteristics for each diabetes metric.

The study team examined the relationship between other ACO characteristics (eg, care management capabilities, health information technology capabilities, inclusion of other providers, experience with other risk-based contracts) and found that these characteristics did not contribute significantly to performance on diabetes metrics in bivariate analyses. To simplify presentation of results, these variables were not included in the models presented.

Descriptive statistics for the organizational and patient population characteristics are presented, followed by significance tests on mean scores for diabetes measures between performance year 1 and performance year 2, and finally regression results. The study team examined the association between ACO organizational and patient population characteristics on performance for diabetes metrics in performance year 1 and the change score between performance years 1 and 2. The latter models also adjust for the effects of performance year 1 for each relevant diabetes metric. Because of the design of this study and limited data, these analyses cannot determine causal relationships – rather only correlations or associations.

Results

ACO organizational and patient population characteristics

Half of the MSSP ACOs included a hospital, one quarter included a community health center, half had a private payer ACO contract, and, on average, half of the physicians within ACOs were primary care physicians. These and additional characteristics are displayed in Table 1.

Table 1.

Descriptive Statistics for Organizational and Patient Characteristics of Accountable Care Organizations (N = 162)

	%/Mean (SD)
Organizational characteristics
Mean number of services in the ACO (out of 7 total services)	3.30 (2.78)
% of ACOs with a hospital	51% (50.15)
% of ACOs with a community health center	25% (43.61)
% of ACOs with a private payer ACO contract	48% (50.09)
% of ACO physicians who are PCPs	49% (23.28)
Patient characteristics
% minority beneficiaries	15% (13.94)
% female	54% (4.53)
% aged 85 and older	12% (3.16)
% aged 64 and younger	17% (6.37)
Mean HCC for disabled	1.11 (0.12)
Mean HCC for aged non-dual	1.06 (0.10)
Mean HCC for aged dual	1.05 (0.10)

Unadjusted descriptive statistics are presented.

Number of services in the ACO includes emergency care, specialized care, behavioral health, skilled nursing, pediatric services, home health, and pharmacy.

ACO, accountable care organization; HCC, Hierarchical Condition Category; PCP, primary care physician.

MSSP participants' performance on diabetes measures

Most ACOs improved performance significantly for the 6 diabetes measures in the second contract year (Figure 1, N = 119). Almost three quarters of ACOs improved on the composite measure, 62% improved on blood sugar control, 74% improved on cholesterol control, 63% improved on blood pressure control, 71% improved on tobacco nonuse, and 69% improved on daily aspirin use for patients with diabetes. Differences between the 2 years ranged from a 3.47 percentage point increase for blood pressure well controlled to an 11.61 percentage point increase for tobacco nonuse (which may be related to improved documentation of tobacco nonuse). For those ACOs that improved from performance year 1 to year 2, the average improvement was 9.58 percentage points for the composite, 8.11 for blood sugar control, 7.89 for cholesterol control, 5.76 for blood pressure control, 21.38 for tobacco nonuse, and 12.12 for aspirin use.

FIG. 1.

Average accountable care organization performance on diabetes measures for performance years 1 and 2. Each bar represents one of the diabetes quality measures. Light gray indicates performance in year 1 and dark gray shows the increase in performance during year 2. t Tests were used to test significance between unadjusted year 1 and year 2 performance scores for each diabetes measure. Significance testing was restricted to those ACOs with both survey data and Centers for Medicare & Medicaid Services performance data for consistency across analyses and because it was the more conservative approach. *P < .05; ***P < .001. ACO, accountable care organization; LDL, low-density lipoprotein cholesterol.

Association of ACO characteristics with diabetes quality performance in year 1

Ordinary least squares multivariate regression was used to explore the association between organizational and patient population characteristics with quality performance in year 1 (Table 2). Each column presents the regression coefficient, P value level, and t score for each organizational and patient population characteristic by outcome. Point estimates of coefficients were largely similar across outcome measures, although significance level varies. The number of services offered within the ACO and having a private payer ACO contract were positively associated with performance, while inclusion of a hospital or a community health center were negatively associated with performance. Patient population characteristics typically were not significantly associated with performance.

Table 2.

Multivariate Regression Coefficients for Accountable Care Organization Organizational and Patient Characteristics on Diabetes Quality Metrics for Performance Year 1 (N = 162)

	Composite		Hemoglobin A1c control		LDL control		Blood pressure <140/90		Tobacco nonuse		Aspirin use
	Coef	t score	Coef	t score	Coef	t score	Coef	t score	Coef	t score	Coef	t score
Unadjusted Mean for Quality Measure	22.90%		69.20%		55.11%		68.96%		67.55%		76.81%
Organizational characteristics
No. of ACO services	1.18**	2.64	0.73	1.81	0.56	1.39	0.34	0.73	2.86**	2.81	1.16	1.54
Hospital in ACO	−1.67	−0.73	−4.49*	−2.19	−2.94	−1.42	0.74	0.32	−1.25	−0.24	−3.25	−0.84
Community health center in ACO	−2.28	−1.1	−3.94***	−2.12	−3.70	−1.98	−2.30	−1.08	−4.72	−1.00	−1.50	−0.43
Private payer contract	4.28*	2.28	3.45*	2.06	4.09*	2.42	5.19**	2.7	9.87*	2.32	3.41	1.08
Proportion PCP	−1.61	−0.36	−7.12	−1.76	−3.54	−0.87	−3.69	−0.8	2.79	0.27	−5.10	−0.67
Patient characteristics
Proportion minority	−0.18*	−2.38	−0.11	−1.62	−0.12	−1.78	−0.05	−0.62	−0.32	−1.85	−0.34**	−2.6
Proportion female	−0.38	−1.61	−0.10	−0.49	−0.32	−1.51	−0.45	−1.87	−0.66	−1.23	−0.05	−0.13
Proportion aged 85 and older	0.25	0.62	−0.07	−0.21	0.05	0.15	0.09	0.23	0.071	0.08	−0.67	−0.99
Proportion aged 64 and under	−0.18	−0.95	−0.22	−1.31	−0.22	−1.34	0.00	−0.01	−0.14	−0.32	0.50	1.59
HCC for disabled	12.11	1.03	5.61	0.53	7.99	0.75	13.85	1.15	25.24	0.95	1.27	0.06
HCC for aged non-dual	−6.36	−0.42	−28.34*	−2.09	−26.48	−1.94	−14.52	−0.93	−9.60	−0.28	17.64	0.69
HCC for aged dual	−12.40	−1.02	−1.73	−0.16	−4.74	−0.43	−13.83	−1.12	−1.63	−0.06	−12.63	−0.62
Constant	48.73 **	2.8	109.49 ***	7.05	102.08 ***	6.52	105.89 ***	5.95	79.73 *	2.02	76.58 ***	2.62

Ordinary least squares regression was used to predict scores on diabetes quality metrics in performance year 1.

Number of services in the ACO includes emergency care, specialized care, behavioral health, skilled nursing, pediatric services, home health, and pharmacy.

^*P < .05; ^**P < .01; ^***P < .001.

ACO, accountable care organization; HCC, Hierarchical Condition Category; LDL, low-density lipoprotein; PCP, primary care physician.

Association of ACO characteristics with change in diabetes quality performance years 1 to 2

The relationships between ACO characteristics and changes in quality scores are shown in Table 3. A higher score in performance year 1 was consistently and significantly associated with less improvement between performance years 1 and 2. Inclusion of a community health center was consistently associated with lower performance on diabetes metrics. It is worth noting the lower average performance on the diabetes composite by ACOs with a community health center across years – a decrease of 3.92 percentage points on the difference between years 1 and 2. In performance year 1, having a private payer contract in addition to the Medicare contract was associated with better performance on 5 of 6 diabetes measures. The association between having a private payer contract in addition to the Medicare contract was no longer significant when predicting the change in performance.

Table 3.

Multivariate Regression Coefficients for Accountable Care Organization Organizational and Patient Characteristics on Change Between Performance Years 1 and 2 for Diabetes Quality Metrics (N = 119)

	Composite		Hemoglobin A1c control		LDL control		Blood pressure <140/90		Tobacco nonuse		Aspirin use
	Coef	t score	Coef	t score	Coef	t score	Coef	t score	Coef	t score	Coef	t score
Unadjusted Mean for Quality Change Score	4.37		2.58		3.64		1.86		9.18		6.66
Organizational characteristics
No. of ACO services	0.15	0.37	0.07	−0.3	0.12	0.34	0.13	0.4	1.00	1.26	0.12	0.25
Hospital in ACO	−2.73	−1.34	−2.72*	−2.22	−0.61	−0.33	−1.16	−0.69	−5.65	−1.44	−2.05	−0.79
Community health center in ACO	−3.92*	−2.09	−0.44	−0.39	−3.46*	−2.03	−1.34	−0.87	−6.41	−1.78	−5.06*	−2.15
Private payer contract	1.36	0.79	−1.21	−1.19	−0.90	−0.58	0.71	0.5	4.99	1.52	1.50	0.7
Proportion PCP	−3.36	−0.8	0.14	0.05	0.35	0.09	1.85	0.53	−3.98	−0.49	−2.78	−0.52
Patient characteristics
Proportion minority	0.05	0.79	−0.01	−0.22	0.03	0.60	−0.03	−0.48	0.02	0.19	−0.05	−0.59
Proportion female	−0.18	−0.85	−0.10	−0.83	−0.23	−1.24	−0.42*	−2.45	0.08	0.21	−0.15	−0.56
Proportion age 85 and older	0.56	1.42	0.39	1.66	0.57	1.59	0.63	1.94	0.21	0.28	0.24	0.47
Proportion under age 65	−0.05	−0.26	−0.08	−0.7	−0.04	−0.24	0.04	0.26	0.17	0.49	0.27	1.17
HCC for disabled	−19.10	−1.85	−15.23*	−2.48	−24.17*	−2.59	−6.63	−0.78	−13.65	−0.68	−4.33	−0.33
HCC for aged non-dual	−13.76	−1.03	−19.80*	−2.48	−4.54	−0.37	−7.09	−0.65	−12.06	−0.47	−16.35	−0.96
HCC for aged dual	15.90	1.47	9.64	1.49	9.85	1.01	7.63	0.85	13.75	0.66	2.38	0.17
Performance year 1 quality	−0.49 ***	6.78	−0.59 ***	−10.53	−0.37 ***	−4.34	−0.60 ***	−9.34	−0.77 ***	−12.31	−0.52 ***	−9.2
Constant	40.72 *	2.59	75.60 ***	7.31	52.38**	3.33	64.19 ***	4.7	65.47 *	2.18	70.38 ***	3.53

Ordinary least squares regression was used to predict changes in scores on diabetes quality metrics between performance years 1 and 2.

Number of services in the ACO includes emergency care, specialized care, behavioral health, skilled nursing, pediatric services, home health, and pharmacy.

^*P < .05; ^**P < .01; ^***P < .001.

ACO, accountable care organization; HCC, Hierarchical Condition Category; LDL, low-density lipoprotein; PCP, primary care physician.

Discussion

Despite the complex nature of diabetes, most ACOs significantly increased the percentage of patients who were well controlled for each of the diabetes quality measures (eg, an increase in the all-or-nothing composite score). Previous research found that only 14% of adults with diabetes were controlled across 4 of the ACO measures (excluding the aspirin use measure)¹⁴ while the present study found that, on average, 27% of ACO patients with diabetes were controlled on each ACO measure during contract year 2.

Performance improvement may be impacted by the nature of MSSP ACO contracts because they transition from pay for reporting to pay for performance between contract years 1 and 2.³⁸ This transition may have spurred ACOs to invest more heavily in clinical transformation activities that improve care for patients with diabetes. Interestingly, better performance in year 1 was associated with less of an improvement in year 2 performance on diabetes measures. This study found that different ACO organizational characteristics were correlated with diabetes performance in year 1 than with improved performance between years 1 and 2.

Implications of findings

Although this study revealed consistent improvement on diabetes quality of care by ACOs, the findings on the organizational and patient characteristics associated with performance were difficult to interpret. Present study results suggested that ACOs offering more comprehensive services and those that have additional ACO contracts may be better poised for initial performance on diabetes measures. Both findings may be an indication of an ACO's degree of preparation for contracts that include a pay-for-performance component.

Indeed, the relationships between ACO organizational characteristics and changes in performance between contract years were not the same as those observed in the first year performance. For example, despite the finding that ACOs with a private payer contract performed better in year 1, ACOs with a private payer contract did not improve significantly more in performance between years 1 and 2 than those without a private payer contract. These findings may be contradictory to expectations. For example, one might expect that ACOs with greater experience or with more value-based purchasing contracts might have better performance over time than other ACOs, such that any differences between ACOs with and without additional contracts was maintained or even increased. Other scholars have suggested that ACOs may need to reach a “tipping point” of patients under value-based purchasing contracts to successfully transform care.³⁹ These findings suggest that ACO characteristics, such as having a private payer contract, may make ACOs better poised for initial performance on diabetes measures, but that these differences may not be persistent over time.

Present study findings suggest that ACOs with a community health center may not improve performance as much on diabetes measures between years 1 and 2. ACOs with a community health center were less likely to improve performance on the composite measure, which requires patients with diabetes to meet each individual performance measure. Safety net providers may experience greater challenges to comprehensively improving quality of care such as fewer financial or structural resources to invest in quality improvement programs.^24,25 Further, the patients they serve are typically more disadvantaged, which may affect the patients' ability adhere to medical recommendations. This finding has important policy implications – safety net providers may need additional support to succeed under new payment models that include pay for performance.

Further, a consistent negative association was observed between performance in year 1 and improvement between performance years such that those ACOs with better year 1 performance had less improvement between performance years. There are multiple interpretations of this finding with implications on how CMS benchmarks performance over time. One interpretation is that ACOs can overcome initial differences in performance within a relatively short time frame. Conversely, there may be a ceiling effect on performance such that the higher an ACO was already performing, the harder it is to continue to improve. In other words, those ACOs with lower initial performance may have an easier time improving performance because there is more room for improvement. This continued performance may not be sustainable over time; further, more advanced health care organizations may be disincentivized to continue participation. Additional research is needed to better understand the nuances of ACO performance over time and how performance interplays with different benchmarking approaches. For example, do ACOs that improve performance in early contract years continue to improve performance? Are ACOs with better initial performance able to improve performance in later years?

Limitations

This study has several limitations. First, the ACO organizational and patient characteristics available for analysis were quite limited. CMS data included few patient characteristics, most of which were demographic or clinical risk-adjustment data. Despite a high survey response rate among MSSP ACOs as well as high levels of completeness of individual survey questions, many variables from the NSACO were underpowered. The relatively small number of ACOs in the early years of the program, as well as missing data on individual survey questions, may have affected the ability to detect significance. Unmeasured patient and organizational characteristics may have affected performance on diabetes measures. For example, some ACOs may have prioritized clinical transformation efforts for patients with chronic conditions, but prioritizations were not captured adequately in survey data. Some of the performance changes observed may reflect better documentation from ACOs rather than better patient care; for example, the increase in the percentage of patients reported as nonsmokers between performance years may have been related to more consistent documentation of smoking status. Additionally, because the data were aggregated at the ACO level rather than the patient level, patient-level differences in diabetes care cannot be examined; only ACOs' aggregate performance and patient mix can be addressed. The findings on quality performance over time may be impacted by methodological considerations such as serial correlation or reversion to mean performance. Given the limited nature of the data, it is challenging to disentangle relationships between organizational predictors and quality. Finally, performance cannot be causally evaluated because ACOs and non-ACOs cannot be compared in these analyses.

Conclusion

A better understanding of which factors contribute to successful performance is important for policy makers and ACOs to ensure the ACO model improves patient care while controlling cost growth. Policy makers should consider what additional support and resources ACOs may need to be successful at improving quality of care for complex patients. Support needed by ACOs to successfully transform care delivery may vary based on their organizational characteristics as well as how far along they are in their ACO contracts. This study contributes valuable insights into the early efforts of ACOs to impact population health by improving the quality and care for complex patients.⁴⁰ Additional research is needed to gain an in-depth understanding of how some ACOs improve care for complex patients. This study highlights that many ACOs improved the quality of care for patients with diabetes and offers initial insights into how performance may be affected by organizational characteristics.

Author Disclosure Statement

The authors declare that there are no conflicts of interest. The authors received the following financial support: This work was supported by grants from the Commonwealth Fund and the Agency for Health Care Research and Quality (AHRQ) (1U19HS024075).