Medicare Shared Savings Programs: Higher Cost Accountable Care Organizations are More Likely to Achieve Savings

Abstract

In the United States, Medicare’s flagship Accountable Care Organization (ACO) program, the Medicare Shared Savings Program (MSSP), is under close scrutiny to improve health care quality and decrease costs. First year measures, released in November 2014, reveal a wide range of financial and quality performance across MSSP participants. In this observational study we used 2013 results for 220 participating ACOs to assess key characteristics associated with generating savings. ACOs with higher baseline expenditures were significantly more likely to generate savings than lower cost ACOs. Average quality scores for ACOs that successfully reported on quality were not different between organizations that did and did not generate savings. These findings suggest ACOs that had lower utilization prior to program enrollment are less likely to be rewarded in the current program. This has important policy implications for the MSSP’s ability to attract and retain efficient ACOs and incent efforts to reduce waste and improve quality.

Introduction

In the United States, the Medicare Shared Savings Program (MSSP) is a keystone program established through the Affordable Care Act to incent health systems, hospitals, and health care providers to coordinate care and reduce costs. The program creates opportunities for physicians and health facilities to partner and organize to provide more coordinated care as Accountable Care Organizations (ACOs). It also encourages existing integrated systems to invest in models of care delivery that promote population health management. These incentives are dependent on the ability of ACOs to decrease their costs to Medicare, while maintaining or improving clinical quality [1].

Understanding which ACOs are most likely to generate savings in this program has important implications for CMS value-based payment policy development, as well as for ACOs deciding to begin or continue participation in the program. This is particularly important as participation in the MSSP has grown significantly since the program began. As of January 2015 there were 404 MSSP ACOs, representing 7.3 million beneficiaries across 49 states, Washington DC and Puerto Rico [2]. However, it is notable that the program has not been perceived as viable by some participants, as evidenced by Dartmouth-Hitchcock Medical Center’s ACO dropping out of the program [3].

Unlike other performance initiatives, which have focused on individual provider’s locus of control, ACOs seek to also address fragmentation and needed care coordination by requiring shared responsibility between physician groups and hospitals [4]. ACOs share features of some HMOs (e.g. delegated capitation), and had as their predecessor both provider-sponsored organizations (PSOs) and more recently Physician Group Practice Demonstrations (PGPD) [4, 5].

Results from earlier predecessor and ACO pilots have been mixed. In the late 1990s, PSOs, physician-run managed care plans offering a network of providers and hospitals for Medicare recipients, were able to directly contact with Medicare cutting out a health plan middleman. However, very few PSOs participated in the program, opting for insurance-based models instead [5]. In the mid-2000s, physician groups participating in the PGPD could receive bonus payments if they achieved lower comparable cost growth and met quality targets. Evaluations suggest that significant savings was based upon changes in diagnoses codes and cost reductions to the sub-population of dually eligible (Medicare-Medicaid) beneficiaries leading to modest overall savings [5, 6]. Quality, however, improved on all measures used [6, 7, 8, 9].

In 2012, the Pioneer ACO model launched with 32 diverse provider organizations agreeing to two-sided risk, sharing in savings if spending fell below a specific benchmark and incurring losses for excess spending. Findings from year one of the Pioneer program suggest only modest reductions in expenditures, with greater savings for ACOs with higher baseline spending. Performance on quality measures was either flat or slightly increased in only certain areas, [10] perhaps indicative of the fact that hospitals participating in the program as compared to similar non-participating hospitals were higher performers in quality at baseline [11]. In a recent assessment of Medicare ACOs, including the Pioneer and Medicare Shared Savings Program, participants indicated decreased spending across the total Medicare population, with more than three times greater decrease in spending among the clinically vulnerable populations [12]. Analyses also show that spending reductions in the MSSP exceeded bonus payments, and some MSSP cohorts have seen overall spending reductions, suggesting that the program has had initial positive impact for Medicare [13, 14]. However, it was also shown that significant heterogeneity exists - savings were greater in independent primary care groups than in hospital-integrated groups [14].

Failing to achieve significant savings when combined with influences such as payer-mix [15], discordance between payment reform and legislation intended to prevent fraud and abuse [16] and distribution of patient utilization of inpatient and outpatient services [17], may impact which organizations may choose to join the program in future years and continue to participate over time [14]. The participation volume and diversity of participants in newer ACO models offers more data to establish an evidence-base of key characteristics of shared savings success [5, 18, 19].

The purpose of this paper is to report our analysis of ACO characteristics that correlate with generating shared savings in the first performance year of MSSP participation. We consider how these results may play a role in future drop-out or uptake of program participation. We also discuss policy implications of these findings and considerations for future CMS value-based incentive programs.

New contribution

Health policy experts have debated the key components of the various ACO models. In particular, how baseline spending and initial benchmarks are associated with future ability to decrease costs is unclear [1, 5, 20]. Understanding the ability of ACOs with different baseline performance to achieve financial savings and effectively report on quality is essential for assessing the ACO concept as a key strategy towards achieving optimal population health management. This analysis provides deeper insight to ACO design and resulting impacts.

Conceptual framework

The study’s theoretical framework is grounded in Donabedian’s model for assessing the quality of care [21]. This framework suggests that health outcomes are the direct result of how the health care setting is structured and the care processes that occur as part of that setting. Over time, others have extended this framework to include structural attributes beyond a single health care setting [22] and have adapted the model to explicitly consider costs [23]. Key variables in the study reflect the Structure-Process-Outcome framework as indicated in Table 1 below.

Table 1.

Variables used for study analysis categorized by Donabedian’s model

CMS Provided Variable	Description	Donabedian’s Model
Total Assigned Beneficiaries (TAB)	Number of Medicare fee-for-service beneficiaries attributed to an ACO; count provided by CMS	Structure
Total Benchmark Expenditures (TBE)	Target total expenditures are based on an ACO’s historical performance and national trends, risk adjusted, and calculated as the per capita benchmark expenditures multiplied by the total person years; dollar amount provided in CMS dataset	Process
Quality Reporting	Whether or not an ACO met the CMS quality reporting requirement	Process
Total Assigned Beneficiary Expenditures (TABE)	Actual aggregate amount CMS spent on Total Assigned Beneficiaries (TAB); dollar amount provided in CMS dataset	Outcome
Created Study Variables
Minimum Savings Rate (MSR)	Minimum rate of expenditures less than target expenditures necessary to earn shared savings payments; calculated based on CMS-defined scale (CMS 2014c)	Process
Per Capita Benchmark Expenditures (PCB)	Target expenditures per beneficiary; Total Benchmark Expenditures (TBE) / Total Assigned Beneficiaries (TAB)	Process
Per Capita Expenditures (PCE)	Average actual expenditures per beneficiary; calculated by dividing Total Assigned Beneficiary Expenditures (TABE) by Total Assigned Beneficiaries (TAB)	Outcome
Generated Shared Savings Indicator (GSSI)	Binary indicator variable (true/false) indicating whether savings were positive or negative	Outcome
Quality Score (QS)	Quality performance score for each ACO calculated based on CMS defined methodology (CMS 2014e)	Outcome
Per Capita Savings Achieved (PCSA)	Average savings achieved per beneficiary, negative savings represents losses; calculated by subtracting target expenditures from actual expenditures (Per Capita Benchmark Expenditures (PCB) - Per Capita Expenditures (PCE))	Outcome

Source: Derived or calculated based on Medicare Shared Savings Program Accountable Care Organizations Performance Year 1 Results [20].

Notes: CMS provided detailed ACO level data, in some cases actual specific count and calculated variables were included in the public data set. In other cases specific variables are calculated by CMS and provided to individual ACOs, such as Minimum Shared Savings Rate (MSR) and Quality Score (QS), but were not included in the publically available dataset. These were calculated for our analysis using the data elements provided by CMS and applying the published methodologies. The Quality Score methodology includes weighting for individual performance on each of the 33 measures. Each of these are assigned to a domain, which also carries a weight and are combined for a final quality score. For those organizations that did not successfully submit quality measures in 2013, only partial scores could be calculated, based upon the 11 quality measures supplied by CMS [1]. The Minimum Savings Rate (MSR) is used by CMS to determine the percent change in expenditures, beyond target benchmark, that each ACO must exceed in order to share in savings or pay losses. The MSR was calculated based on CMS’s published methodology [21]. The MSR is different depending on participant Track. Track 2 (with both upside and downside risk) participants are all assigned an MSR of 2.0%. For Track 1 (upside risk only) participants the MSR is a on a sliding scale based on the number of attributed beneficiaries.

Study data and methods

In November 2014, the Center for Medicare and Medicaid Services (CMS) released year one financial and quality performance results (PY1) for 220 participating ACOs that joined the MSSP in 2012 and 2013 [24]. This data is publicly available at: https://data.cms.gov/ACO/Medicare-Shared-Savings-Program-Accountable-Care-O/yuq5-65xt. This dataset contains specific information about each of the 220 MSSP organizations, including agreement start date, number of assigned beneficiaries, the aggregate target benchmark for performance year one, the actual aggregate amount CMS spent during performance year one, and the rate of 2013 savings or losses. The CMS dataset also contains each ACO’s 2013 performance results, if available, for the required 33 individual and 2 composite quality metrics. Table 1 describes key variables used in the analysis, including those provided in the publically available dataset and those that were calculated. Of note, CMS calculates shared savings or losses using person-years rather than assigned beneficiaries as the denominator. Neither the actual PY1 per capita benchmark amount nor the numbers of person-years for each organization are included in the November 2014 dataset.

According to the initial Shared Savings Program Final Rule, to generate savings an organization’s Generated Savings Rate (GSR) must be greater than its Minimum Savings Rate (MSR), which depends on the track chosen by the organization. For the initial 3-year contract, ACOs were able to choose a one-sided risk model where only generated savings, but not losses, are shared (Track 1), or a two-sided model where any excess costs generated in a performance year must also be paid back (Track 2). For Track 1 ACOs, MSR is determined based on the number of Medicare beneficiaries attributed to the ACO. This ranges from 2.0% (for ACOs with 60,000 or more beneficiaries) and increases up to 3.9% (for ACOs with at least 5,000 beneficiaries) as the number declines toward the minimum of 5,000 needed to participate. For Track 2 ACOs, the MSR is set at 2.0%, regardless of the number of beneficiaries [1, 25].

Performance Year 1 quality reporting for ACOs with 2012 start dates required the reporting of quality measures for both calendar years 2012 and 2013, while ACOs with 2013 start dates were required to report only for calendar year 2013. Therefore, an indicator variable is included in the CMS dataset noting whether each MSSP organization: (1) did not successfully report quality measures in 2013, (2) did not successfully report quality measures in 2012 but did in 2013, or (3) successfully reported quality measures in each required year.

Calculated variables

We used information provided in the CMS dataset to create several per capita variables in order to compare performance among ACOs. We applied the quality score methodology to the 33 individual performance measures reported in order to compare final quality scores. We also created an indicator for those that did or did not generate savings (Generated Shared Savings Indicator, GSSI) and calculated the minimum savings rate (MSR). See Table 1 for variable definitions and their categorization as structure, process, or outcome according to Donabedian’s model.

Analyses

All analyses were conducted with SAS Version 10.0 and Stata Version 13. First, we tested for associations between GSSI and each of the structure, process, and outcome variables found in Table 1 using Fisher’s exact and Chi square tests. Results were considered significant at p<0.05. Next, we conducted three multivariate regression analyses to determine if the probability of an organization generating savings was associated with per capita benchmark expenditures, minimum saving rates, or quality score results. Stepwise selection procedures were performed, including only covariates with a p-value <0.20. To assess the goodness-of-fit of the multiple regression models the predictive fit index, adjusted R², was assessed. If the covariates did not increase the adjusted R² for the model, the model was reduced by excluding those variables. The final model only included per capita benchmark expenditure as a predictor of generated shared savings.

Study results

As noted in the CMS dataset, participating ACOs had a wide range of performance, with individual per capita benchmark costs to Medicare ranging from $5,014 - $22,993. Final financial performance also had wide variation, with 23.6% of MSSP organizations achieving a financial distribution from CMS in amounts ranging from $1.45M to $28.34M.

Accountable Care Organization characteristics

Table 2 provides a comparison of characteristics between ACOs that did and did not generate savings, including number of beneficiaries, financial results, and quality scores. Of the 220 MSSP organizations represented, 58(26%) qualified for shared savings based on financial performance. Average MSR was similar for ACOs that did or did not generate savings, at approximately 2.9%. The total number of average attributed beneficiaries was similar between groups that did and did not generate savings (16,129 and 16,912, respectively). In addition, of the ACOs that had greater than 20, 000 attributed beneficiaries, 16 (32%) generated savings and 34(68%) did not.

Table 2.

Description of MSSP ACO Characteristics and 2013 Performance Year Results Categorized by Donabedian’s Model

	All MSSP ACOs	Generated savings
		Yes (%)	No (%)	p-value
Structure
Number of MSSP ACOs
All	220	58 (26)	162 (74)
By start date				0.180
4/1/2012	27	8 (30)	19 (70)
7/1/2012	87	28 (32)	59 (68)
1/1/2013	106	22 (21)	84 (79)
By track				0.480
Track 1	215	56 (26)	159 (74)
Track 2	5	2 (40)	3 (60)
Number of total assigned beneficiaries				0.151
Average	16,706	16,129	16,912
3,900–10,000	82	25 (30)	57 (70)
10,001–20,000	88	17 (19)	71 (81)
20,001–140,000	50	16 (32)	34 (68)
Process
Minimum Savings Rate (MSR)				0.540
Average %	2.89	2.93	2.88
MSR ≤2.9%	110	27 (25)	83 (75)
MSR >2.9%	110	31 (28)	79 (72)
Per Capita Benchmarks Expenditures (Target)				0.007
Average $	$11,786	$13,386	$11,214
$5,000-$9,000	40	5 (13)	35 (88)
$9,001-$11,000	57	11 (19)	4652 (81)
$11,001-$13,000	59	16 (27)	43 (73)
$13,001-$23,000	64	26 (41)	38 (59)
Outcome
Per Capita Expenditures (Actual)				0.149
Average	$11,712	$12,500	$11,429
$4,000- $9,000	41	8 (20)	33 (80)
$9,001-$11,000	54	11 (20)	43 (80)
$11,001-$13,000	62	16 (26)	46 (74)
$13,001-$24,000	63	23 (37)	40 (63)
Shared Savings Result				0.000
Lost $	1	0	1 (100)
Neutral	167	6 (4)	161 (96)
Earned $	52	52 (100)	0
Quality Reporting				0.045
Did not successfully report quality in 2013	6	4 (66)	2 (34)
Did not successfully report quality in 2012, but did in 2013	5	2 (40)	3 (60)
Successfully reported quality in each required year	209	52 (25)	157 (75)
Quality Score				0.750
Average %	72	72	72
26%−66%	56	12 (22)	44 (78)
67%−74%	56	17 (30)	39 (70)
75%−79%	50	13 (26)	37 (74)
80%−91%	58	16 (28)	42 (72)

Table 2 Source: Analysis based on Medicare Shared Savings Program Accountable Care Organizations Performance Year 1 Results (Centers for Medicare and Medicaid Services, 2014c).

Notes: Values represent numbers/percentages unless otherwise indicated in the row. Percentages shown are calculated across for each characteristic. For each row, the denominator is the number of MSSP organizations that met the given characteristic and the numerators are the numbers of organizations meeting the characteristic that did or did not generate savings. Categories for total assigned beneficiaries, minimum savings rate, per capita benchmark and per capita expenditures were created to demonstrate distributions in addition to averages. Target per capita benchmark expenditures data resulted in a p-value of 0.007 when tested categorically using an x² test, however when tested continuously the resulting p-value was <0.0001. The Quality Score categories represent quartiles. In order to generate savings, an ACO’s Generated Savings Rate was greater than its Minimum Savings Rate.

Per capita expenditures and generating shared savings

Per our final regression model, the probability of an ACO generating savings was significantly associated with having higher per capita benchmark expenditures (p < 0.0001). Forty-one percent of MSSP organizations with per capita benchmark expenditure amounts greater than $13,000 were eligible for shared savings, as opposed to only 13% of ACOs with per capita benchmark expenditure amounts less than or equal to $9,000. As shown in Figure 1, MSSP organizations with greater per capita benchmark expenditures demonstrate more variation in savings and losses than those with lower per capita benchmark expenditures. The figure shows that correlation between high expenditures and achieving savings holds across the continuum of benchmark expenditures, not only when viewed by the categories presented in Table 2.

Figure 1.

Per Capita Savings Achieved for Each MSSP Organization, by 2013 per Capita Benchmark Expenditures

Note. Source: Medicare Shared Savings Program Accountable Care Organizations Performance Year 1 Results [20] and variables calculated from this dataset.

Quality reporting and generating shared savings

Table 2 indicates that an organization’s ability to generate shared savings was negatively associated with its successful reporting of quality measures. Five (3%) ACOs that did not generate savings also did not successfully report on quality in at least one required year, while 6 (10%) ACOs that generated savings did not successfully report in at least one required year. However, for those organizations that successfully reported quality scores in both years, quality scores did not differ significantly between organizations that did and did not generate shared savings (75% and 73%, p = 0.161).

Discussion

The MSSP is the flagship program in CMS’s portfolio of initiatives designed to reward health systems for moving from high-utilizing, volume driven activity toward value driven activity that delivers high quality and efficient care. MSSP ACOs had a wide range of final financial performance. Quality results also varied. This variability has significant implications for the program’s future and supports previous calls to examine underlying factors associated with variability and subsequent recommendations for policy changes [1, 5, 20].

We examined financial results from the first full year of the MSSP (calendar year 2013) and found that organizations with per capita benchmark costs greater than $13,000 were almost twice as likely to achieve savings as organizations with benchmark costs of $9,000 or less (37% versus 20%). Moreover, organizations with low benchmark expenditures were unlikely to achieve any savings, despite much lower costs than other groups. Overall, higher per capita benchmark expenditures were significantly associated with a higher likelihood of generating savings

Thus, the MSSP appears to have likely met the intended goal of reducing total expenditures for the highest cost ACO populations. However, as feared by key stakeholders [1, 5, 20] the MSSP appears to have the unintended consequence of rewarding those with comparatively high expenditures and penalizing those with comparatively low expenditures. This same outcome was confirmed for the Pioneer ACOs. In addition, for those ACOs that left that program, unsustainability of the financial model for already lower cost ACOs was a commonly reported reason [10].

In terms of Donabedian’s model [21], structural variables (ACO track and the number of total assigned beneficiaries) were not significantly associated with the outcome of generating savings. One process variable was associated with generating savings- the per capita benchmark expenditure. The other evaluated process variable, the minimum savings rate, was not associated with generated savings.

The program was designed to encourage participation by a wide variety of organizations, while protecting CMS from unwarranted payouts. Potential random variation in year to year costs due to differences in ACO enrollment size was addressed by the model developers through the use of a MSR range [25, 26]. The MSSP model does not take into account differences in baseline expenditures. Instead, earning savings is based on the rates of change in total spending, rather than absolute dollar performance. For example, an ACO that had per capita benchmark expenditures of nearly $23,000 earned shared savings by decreasing expenditures by approximately $1,000 per capita, yet another ACO with benchmark expenditures of $7,500 per capita had little change in spending during PY1 and did not achieve earnings in this program.

The variability in actual per capita savings (or losses) generated increases greatly along with benchmark costs, suggesting a greater degree of fluctuation may occur at higher benchmark expenditures. It also appears that few ACOs are able to reduce their per-capita expenditure below the $7,300 mark. This may suggest a threshold below which Medicare expenditures cannot be easily reduced in the MSSP program year over year, let alone at a rate which would allow for realizing shared savings.

We also examined the relationship between quality results and shared savings for MSSP 2013 performance and found no significant correlation between quality performance and the ability to generate shared savings. At the time of this analysis we only had quality performance data for one point in time. Future analyses may explore if quality performance changes similarly or differently for organizations that are and are not able also make significant reductions in total expenditures.

We did find that organizations eligible for shared savings were three times as likely to unsuccessfully report on required quality metrics compared with organizations ineligible for shared savings. However, the number of organizations that failed to successfully report required quality metrics was small. Nonetheless, the inability to organize around quality reporting despite achieving multi-million dollar cost savings suggests the savings may, in some instances, be related to random variation rather than fundamental changes in care delivery that would theoretically be much more difficult to accomplish in a short time period. These early findings support the lessons learned from earlier pilots that ACOs that will succeed long-term will need a “capability package” that includes strong quality improvement programs, acknowledging that the capacity and skillsets necessary to generate cost savings differ from those to improve quality [9, 27].

Study limitations

There were limitations to our analysis. In our calculations, per capita benchmark expenditures serve only as proxies for the true PY1 per capita benchmark expenditure amounts calculated by CMS for each MSSP organization, since the true values are calculated using person years rather than assigned beneficiaries. Also, we were unable to account for the number of participating organizations, such as physician practices and hospitals, within an individual ACO, as these data were missing for 29 (13%) of MSSP organizations [28]. ACOs that are comprised of many disparate physician and hospital groups may have very different characteristics than ACOs that were more highly integrated prior to joining the program, which may account for some of the differences in outcomes. Similarly, ACOs that include beneficiaries over a wide-geographic area may have different characteristics than those contained to a narrow region. We were unable to conduct an analysis based on these characteristics since available information was limited to the states where ACOs provided services, and no information was provided as to the distribution of where beneficiaries reside. We were also unable to account for differences in results based on variation in the health complexity of the populations attributed to each MSSP organization. Finally, future analyses that include more years of performance data may lead to additional or different findings.

Policy and future implications

Although the MSSP is still a relatively young program, these findings confirm theoretical concerns raised prior to implementation and suggest opportunities for policy makers to treat organizations with high and low baseline expenditures differently and create a high-performing ACO track.

Reflecting on the Institute of Medicine report estimating 30% of health care spending as unnecessary, it is clear that plenty of waste remains to expunge, even for currently high-performing organizations [29]. However, our data suggest that CMS should consider a program design that incorporates information about an ACO’s baseline expenditures, and consider adjusting program rules, as allowed by statute, to encourage the continued participation of institutions at all stages of evolution toward a high-value health system.

For future enrollment periods, we propose that CMS also consider adding an additional MSSP track that would support high-performing ACOs’ participation in the MSSP without risk for shared savings or losses. This track could provide participating MSSP organizations with claims data for their attributed Medicare beneficiaries; while obligating them to submit quality data and commit to continuing care coordination efforts similar to requirements for other MSSP tracks. If eligibility in this track were linked to quality and cost performance thresholds - based on national performance standards, such as top quartile in quality and lowest quartile in per capita expenditure - this would encourage participating ACOs to maintain and improve performance even as the bar rises over time. This may be an appealing model for high-performing ACOs committed to improving quality and reducing costs that intend to leverage more sustainable Medicare funding streams such as the use of transitional care management, complex chronic care management, and medication therapy management codes.

More recent Medicare programs have payment options that move toward risk adjusted capitation: the Next Generation ACO program and the Comprehensive Primary Care Plus Initiative (CPC+). The Next Generation ACO program - which began in 2016 and has a total of 45 participants as of January 2017 - offers a capitation payment mechanism option beginning in performance year 2. The CPC+ program, described by CMS as an “advanced primary care medical home model,” is being tested in 14 regions as of January 2017. This program includes a risk-adjusted capitated Care Management Fee and also shifts some FFS payment to lump sums payments for Track 2 participants. Evaluating the effectiveness of these payment models will be important as data become available [30, 31, 32].

Conclusions

MSSP’s current program design appears to have potential for incentivizing higher cost ACOs to measurably decrease utilization and organize around quality reporting, but may not reward already lower cost or high quality MSSP organizations. Individual ACO first year financial results, along with findings presented here, may cause lower expenditure ACOs to discontinue program participation. Program rules that reward baseline high performers and account for deeper investments and longer transformation trajectories may be necessary for the program to retain and continue to attract high-performing ACOs.