Changes in Low-Value Services in Year 1 of the Medicare Pioneer Accountable Care Organization Program

Abstract

Importance

Wasteful practices are widespread in the US health care system. It is unclear if payment models intended to improve health care efficiency, such as the Medicare accountable care organization (ACO) programs, discourage the provision of low-value services.

Objective

To assess whether the first year of the Medicare Pioneer ACO program was associated with a reduction in use of low-value services.

Design, Setting and Participants

In a difference-in-differences analysis, we compared use of low-value services between Medicare fee-for-service beneficiaries attributed to provider groups that entered the Pioneer program (ACO group) and beneficiaries attributed to other providers (control group) before (2009–2011) vs. after (2012) Pioneer ACO contracts began. We adjusted comparisons for beneficiaries’ sociodemographic and clinical characteristics and for geography. We decomposed estimates according to service characteristics (clinical category, price, and sensitivity to patient preferences) and compared estimates between subgroups of ACOs with higher vs. lower baseline use of low-value services.

Main Outcomes and Measures

Use of, and spending on, 31 services in instances that provide minimal clinical benefit.

Results

During the pre-contract period, trends in use of low-value services were similar for the ACO and control groups. The first year of ACO contracts was associated with a differential reduction of 0.8 low-value services per 100 beneficiaries for the ACO group (95% CI: −1.2, −0.4; P<0.001), corresponding to a 1.9% reduction in service quantity (95% CI: −2.9%, −0.9%) and a 4.5% differential reduction in spending on low-value services (95% CI: −7.5%, −1.4%; P=0.004). Differential reductions were similar for services less vs. more sensitive to patient preferences and for higher- vs. lower-priced services. ACOs with higher than their markets average baseline levels of low-value service use experienced greater service reductions (−1.2 services per 100 beneficiaries, 95% CI: −1.7, −0.7; P<0.001) than ACOs with below average use (−0.2 services per 100 beneficiaries, 95% CI: −0.6, 0.2; P=0.41; P=0.003 for test of difference between subgroups).

Conclusions and Relevance

During its first year, the Pioneer ACO program was associated with modest reductions in low-value services, with greater reductions for organizations providing more low-value care. ACO-like risk contracts may be able to discourage use of low-value services even without specifying services to target.

Reducing unnecessary health care utilization, a source of substantial spending,¹ is a central goal of many government^2–4 and private initiatives.^5,6 Recent efforts, such as the American Board of Internal Medicine’s Choosing Wisely campaign, have drawn attention to specific services that provide minimal clinical benefit.⁷ However, little is known about what strategies can effectively discourage the use of these services. Distinguishing high-value from low-value use of the same service is often challenging because value often depends on clinical context. As a result, efforts to directly limit overuse of specific services through coverage restrictions or other payment incentives may produce unintended consequences or achieve minimal gains.^8–10

Other strategies intended to improve health care efficiency do not target specific services. Among the most prominent of these strategies are alternative payment models such as the model used in the Medicare Pioneer accountable care organization (ACO) program, which places spending for all services under a global budget with incentives to stay under the budget and improve performance on quality measures. This approach has been associated with lower overall spending and improved or stable performance on standard quality measures.^11–15

However, it is unknown whether payment reforms like these are associated with disproportionate reductions in use of low-value services. Although ACO-like payment models are intended to discourage the provision of services that contribute to spending but not to health, the combination of lower overall spending and improved performance on quality measures that has been observed may have resulted from reductions in high-value services affecting unmeasured dimensions of quality rather than from reductions in low-value services. More generally, because risk-based contracts do not incentivize reductions in overuse directly, it is unclear whether providers under these contracts are targeting low-value services in their broader efforts to control overall spending. If ACO-like payment models succeed in reducing use of low-value services, there should be observable reductions in the delivery of low-value services that can be measured directly. Moreover, if providers respond to ACO contracts by targeting low-value services specifically, their efforts should result in greater reductions in spending on low-value services than in overall spending.

We constructed 31 claims-based measures of low-value services—services that provide minimal clinical benefit on average. Using these measures and 2009–2012 Medicare fee-for-service (FFS) claims, we conducted a difference-in-differences analysis comparing use of low-value services between beneficiaries served by Pioneer ACOs and beneficiaries served by non-ACO providers before vs. after the start of Pioneer contracts in 2012.

Methods

Background on the Pioneer ACO Program

In 2012, 32 provider organizations volunteered to participate in the Medicare Pioneer ACO program, in which participating organizations receive a bonus payment, or are penalized, if overall spending for an attributed patient population falls sufficiently below, or above, a financial benchmark, respectively. Performance on 33 quality measures determines the proportion of savings or losses shared by the ACO, although ACOs were only required to report on these measures to be eligible for maximum savings in 2012. None of the quality measures in Medicare ACO contracts assesses overuse of medical services.

Study Population

Data and Inclusion Criteria

We examined services provided from 2009–2012 using Medicare claims for a random 20% sample of beneficiaries; in a given year, this sample includes sample members from the prior year plus a 20% sample of newly eligible beneficiaries. For each year, we included beneficiaries in the study sample if they were continuously enrolled in Parts A and B of traditional Medicare while alive during that year and the entire prior year. We used the prior year of claims to collect diagnoses and procedures used for case-mix adjustment or for assessing the appropriateness of service use. In each study year, beneficiaries were excluded if they did not receive primary care services necessary for attribution to provider organizations, or if they were attributed to any of the 114 organizations that entered the Medicare Shared Savings Program (MSSP) later in 2012. MSSP ACOs faced weaker incentives than Pioneer ACOs to reduce spending, and for only part of 2012. Thus, if MSSP ACOs took early steps to limit low-value services, inclusion of their beneficiaries in the control group could have biased our estimates.

ACO Group and Control Group

Each of the 32 organizations that entered the Pioneer ACO program was defined as the collection of National Provider Identifiers (NPIs) for physicians listed by the ACO as participating in the ACO contract (eAppendix). Our definition of ACOs as sets of NPIs reflects the organizations’ ability to include only a subset of affiliated physicians in their ACO contracts. Following the MSSP attribution rules and previously described methods,¹⁵ for each year in the study period, each beneficiary was assigned to the ACO (ACO group) or non-ACO practice (control group) that accounted for the greatest fraction of that beneficiary’s annual allowed charges for primary care services (eAppendix). Non-ACO practices were defined by taxpayer identification numbers (TINs), which identify the billing practice, provider organization, or individual physician.

Study Variables

Measures of Low-Value Services

We constructed 31 claims-based measures of services that are low-value, which we define as providing minimal or no average clinical benefit in specific clinical scenarios. The measures, 26 of which were drawn from a prior study, ⁹ were derived from evidence-based lists of low-value services. As described previously,⁹ we surveyed the following sources for candidate low-value services: the American Board of Internal Medicine Foundation’s Choosing Wisely initiative,¹⁶ the US Preventive Services Task Force “D” recommendations,¹⁷ the Canadian Agency for Drugs and Technologies in Health technology assessments,¹⁸ and peer-reviewed medical literature.¹⁹ Services selected for measure development met three criteria: the service was relevant to the Medicare population; evidence that the service confers minimal clinical benefit had been established before the start of the study period; and claims and enrollment data were sufficient to distinguish high-value use from low-value use with reasonable accuracy.

For each measure, we created an operational definition of low-value service occurrence based on characteristics of the patient and the service they received. Relevant patient characteristics included demographic characteristics and diagnoses present in concurrent or past claims. In addition to the type of service received, some measure definitions incorporated the timing of the service (e.g. time since an inpatient discharge) and the site of care. We defined low-value services conservatively, opting for more specific definitions that reduced the likelihood of classifying a high-value service as low-value.⁹ We detected service occurrences meeting these definitions on the basis of claims data elements, including Current Procedural Terminology (CPT) service codes, International Classification of Diseases, Ninth Revision (ICD-9) patient diagnosis codes, data from Medicare enrollment files, and condition indicators from the Chronic Condition Data Warehouse (CCW).²⁰ Details regarding service identification, including codes used for service detection, are presented in the eAppendix (eTable 1). To avoid duplicative counting of services, we did not count any service instances occurring within seven days of the same service.

The primary outcome of this study was use of low-value services, defined as the annual count of all measured services. We chose this measure as our primary outcome because overall service counts equally weight clinical decisions to provide different services, while spending on low-value services is influenced heavily by use of more expensive services. We examined price-standardized spending on low-value services as a secondary outcome to compare changes in low-value spending with changes in overall spending associated with the Pioneer program that were estimated previously using similar methods (see eAppendix for price standardization methods).¹⁵

To assess whether any changes in low-value service use associated with Pioneer ACO contracts were concentrated in a specific clinical area or evident in multiple areas, we categorized the 31 low-value services into the following clinical categories: cancer screening, diagnostic and preventive testing, preoperative testing, imaging, cardiovascular testing and procedures, and other invasive procedures. We also categorized services as being priced higher (standardized price $180–$13,331) or lower ($5–$117) than the median service price, because ACOs would be unlikely to reduce higher-priced services in the absence of new payment incentives, whereas ACOs might restrict provision of lower-priced wasteful services even under FFS incentives to improve quality without major reductions in revenue. Thus, reductions in use of higher-priced low-value services would provide stronger evidence of changes related specifically to ACO contract incentives.

Finally, to explore the possibility that patient preferences moderated providers’ responses to ACO contracts, we categorized services as less vs. more sensitive to patient preferences (Table 1). For example, we considered testing for hypercoagulability following deep venous thrombosis as less sensitive to patient preferences because most patients would be unaware that such testing could be done. Table 1 presents each measure’s source and supporting literature, operational definition, and assigned categories of price and preference sensitivity.

Table 1.

Summary of Low-Value Care Measures

Clinical Category	Measure	Source	Operational Definition	Price Category	Preference Sensitivity Category	Mean Annual Count (per 100 Beneficiaries)
Cancer Screening	Cancer screening for patients with CKD receiving dialysis	CW33	Screening for cancer of the breast, cervix, colon, or prostate for patients over age 75 with CKD receiving dialysis servicesa	Lower priced	More sensitive	0.1
	Cervical cancer screening for women age 65 and over	CW, USPSTF34	Screening Papanicolaou test for women over age 65 with no personal history of cancer or dysplasia noted in claim or in prior claims, and no diagnoses of other female genital cancers, abnormal Papanicolaou findings, or human papillomavirus positivity in prior claimsb	Lower priced	More sensitive	4.3
	Colorectal cancer screening for adults over age 85	USPSTF35	Colorectal cancer screening (colonoscopy, sigmoidoscopy, barium enema, or fecal occult blood testing) for patients age 86 or over with no history of colon cancer	Lower priced	More sensitive	0.6
	PSA testing for men age 75 and over	USPSTF36	PSA testing for patients age 75 and over with no history of prostate cancer	Lower priced	More sensitive	7.7
Diagnostic and Preventive Testing	Bone mineral density testing at frequent intervals	Literature37,38	Bone mineral density test within two years of a prior bone mineral density test for patients with an established osteoporosis diagnosis	Lower priced	Less sensitive	0.6
	Homocysteine testing in cardiovascular disease	Literature39	Homocysteine testing with no diagnoses of folate or B12 deficiencies in the claim and no folate or B12 testing in prior claims	Lower priced	Less sensitive	0.8
	Hypercoagulability testing for patients with DVT	CW40	Lab tests for hypercoagulable states within 30 d following diagnosis of lower extremity DVT or pulmonary embolism; no prior evidence of recurrent thrombosis, defined by diagnosis of DVT or pulmonary embolism more than 90 d prior to the testing claim	Lower priced	Less sensitive	0.05
	PTH measurement for patients with stage 1–3 CKD	NICE41,42	PTH measurement for patients with CKD and no dialysis services before PTH testing or within 30 d following testing, as well as no hypercalcemia diagnosis during the year	Lower priced	Less sensitive	3.8
	Total or free T3 level testing for patients with hypothyroidism	CW43	Total or free T3 measurement in a patient with a hypothyroidism diagnosis during the year	Lower priced	Less sensitive	2.6
	1,25-dihydroxyvitamin D testing in the absence of hypercalcemia or decreased kidney function	CW44	Calcitriol testing for patients without hypercalcemia, secondary hyperparathyroidism of renal origin, or conditions related to non-PTH mediated hypercalcemia noted in claim (sarcoidosis, TB, selected neoplasms), and without a history of CKD; no diagnosis of hypercalcemia in the past 30 d	Lower priced	Less sensitive	1.0
Preoperative Testing	Preoperative chest radiography	CW, CADTH45,46	Chest x-ray not associated with inpatient or emergency carec and occurring within 30 d prior to a low or intermediate risk non-cardiothoracic surgical procedured	Lower priced	Less sensitive	1.7
	Preoperative echocardiography	CW47	Echocardiogram not associated with inpatient or emergency care and occurring within 30 d prior to a low or intermediate risk non-cardiothoracic surgical procedured	Higher priced	Less sensitive	0.3
	Preoperative PFT	CW48	PFT not associated with inpatient or emergency care and occurring within 30 days prior to a low or intermediate risk surgical proceduree	Lower priced	Less sensitive	0.1
	Routine preoperative stress tests	CW49	Stress electrocardiogram, echocardiogram, nuclear medicine imaging, cardiac MRI or CT angiography, not associated with inpatient or emergency care and occurring within 30 d prior to a low or intermediate risk surgical procedured	Higher priced	Less sensitive	0.3
Imaging	CT of the sinuses for uncomplicated acute rhinosinusitis	CW50	Maxillofacial CT study with a diagnosis of sinusitis and no complications of sinusitis,f immune deficiencies, nasal polyps, or head/face trauma noted in claim and no sinusitis diagnosis between 30 and 365 d prior to imaging	Higher priced	More sensitive	0.3
	Head imaging in the evaluation of syncope	CW51	CT or MR imaging of the head with a diagnosis of syncope and no diagnoses in claim warranting imagingg	Higher priced	More sensitive	1.0
	Head imaging for uncomplicated headache	CW52	Brain CT or MR imaging with non-post-traumatic, non- thunderclap headache diagnosis, and no diagnoses in claim warranting imagingh	Higher priced	More sensitive	2.9
	EEG for headaches	CW53	EEG with headache diagnosis in claim, and no epilepsy or convulsions noted in current or prior claims	Higher priced	Less sensitive	0.05
	Back imaging for patients with non-specific lower back pain	CW, NICE54	Back imaging with a diagnosis of lower back pain occurring within 6 wk of initial back pain diagnosis and with no indication of radiculopathy or other diagnoses in claim warranting imagingi	Lower priced	More sensitive	4.4
	Screening for carotid artery disease in asymptomatic adults	CW, USPSTF55	Carotid imaging not associated with inpatient or emergency care for patients without a history of stroke or TIA, and without a diagnosis of stroke, TIA, or focal neurological symptoms in claim	Higher priced	Less sensitive	5.8
	Screening for carotid artery disease for syncope	CW51	Carotid imaging with syncope diagnosis for patients without a history of stroke or TIA, and without a diagnosis of stroke, TIA, or focal neurological symptoms in claim	Higher priced	Less sensitive	0.6
	Imaging for diagnosis of plantar fasciitis	CW56	Radiographic or MR imaging with diagnosis of plantar fasciitis occurring within two weeks of initial foot pain diagnosis	Lower priced	More sensitive	0.4
Cardiovascular Testing and Procedures	Stress testing for stable coronary disease	CW57,58	Stress testing not associated with inpatient or emergency carej for patients with an established diagnosis of acute myocardial infarction (≥6 mo before testing)	Higher priced	More sensitive	0.7
	Percutaneous coronary intervention with balloon angioplasty or stent placement for stable coronary disease	Literature58,59	Coronary stent placement or balloon angioplasty, not associated with an ER visit,j for patients with an established diagnosis of acute myocardial infarction (≥6 mo before testing)	Higher priced	More sensitive	0.1
	Renal artery angioplasty or stenting	Literature60,61	Renal/visceral angioplasty or stent placement with a diagnosis of renal atherosclerosis or renovascular hypertension noted in procedure claim	Higher priced	Less sensitive	0.1
	Carotid endarterectomy for asymptomatic patients	CW55,62	Carotid endarterectomy, not associated with an ER visit,j for female patients without a history of stroke or TIA and without stroke, TIA, or focal neurological symptoms noted in claim	Higher priced	Less sensitive	0.05
	Inferior vena cava filters for the prevention of pulmonary embolism	Literature63,64	Any IVC filter placement	Higher priced	Less sensitive	0.2
	Pulmonary Artery Catheterization in the ICU	Literature65	Pulmonary artery catheterization for monitoring purposes during an inpatient stay that involved an ICU and a non- surgical DRG; claim contains no diagnoses indicating pulmonary hypertension, cardiac tamponade, or preoperative assessment	Lower priced	Less sensitive	0.01
Procedures Other Invasive	Vertebroplasty or kyphoplasty for osteoporotic vertebral fractures	Literature66–68	Vertebroplasty/kyphoplasty for vertebral fracture, with no bone cancers, myeloma, or hemangioma noted in procedure claim.	Higher priced	Less sensitive	0.3
	Arthroscopic surgery for knee osteoarthritis	NICE69	Arthroscopic debridement/chondroplasty of the knee with diagnosis of osteoarthritis or chondromalacia in the procedure claim and no meniscal tears noted in procedure claim	Higher priced	More sensitive	0.2
	Spinal injection for low-back pain	Literature70,71	Outpatient epidural, facet, or trigger point injections for lower back pain, excluding etanercept; no radiculopathy diagnoses in the claim	Higher priced	More sensitive	4.0

Abbreviations: CKD, chronic kidney disease; CT, computed tomography; DRG, diagnosis related group; DVT, deep vein thrombosis; ED, emergency department; EEG, electroencephalography; ICU, intensive care unit; IVC, inferior vena cava; MR, magnetic resonance; PFT, pulmonary function testing; PSA, prostate-specific antigen; PTH, parathyroid hormone; TB, tuberculosis; TIA, transient ischemic attack; USPSTF, US Preventive Services Task Force D recommendations.

^aThe age cutoff is included because transplantation is uncommon in this patient population.

^bPrior claims refers throughout the table to claims for services before the day of the measured service and during or after the prior calendar year.

^cInpatient-associated is defined here as occurring during within 30 d after an inpatient stay; ED-associated, during or 1 d after an ED visit.

^dIncludes breast procedures, colectomy, cholecystectomy, transurethral resection of the prostate, hysterectomy, orthopedic surgical procedures other than hip and knee replacement, corneal transplant, cataract removal, retinal detachment, hernia repair, lithotripsy, arthroscopy, and cholecystectomy.

^eIncludes procedures listed immediately above as well as coronary artery bypass graft, aneurysm repair, thromboendarterectomy, percutaneous transluminal coronary angioplasty, and pacemaker insertion.

^fIncludes inflammation of eyelid or orbit, orbital cellulitis, and visual problems.

^gExclusion diagnoses include epilepsy, stroke/TIA, history of stroke, head trauma, convulsions, altered mental status, nervous system symptoms (e.g. hemiplegia), speech problems.

^hExclusion diagnoses include those listed immediately above as well as giant cell arteritis, cancer and history of cancer.

ⁱExclusion diagnoses include cancer, trauma, intravenous drug abuse, neurological impairment, endocarditis, septicemia, tuberculosis, osteomyelitis, fever, weight loss, loss of appetite, night sweats, and anemia.

^jInpatient-associated is defined here as occurring during an inpatient stay; ED-associated, during or within 14 d after an ED visit.

Covariates

For each beneficiary, the following demographic and clinical covariates were assessed from Medicare claims and enrollment files: age (<65, 65–69, 70–74, 75–79, 80–84, and >84), sex, race/ethnicity, disability as the original reason for Medicare entitlement, presence of end-stage renal disease, presence of 27 chronic conditions in the CCW by the start of each study year (including indicators for each condition and indicators for having ≥2, 3, 4, etc. conditions up to ≥9), and the patient’s hierarchical condition category (HCC) risk score. Because most low-value service measures do not apply to all beneficiaries (e.g., low-value PSA tests were defined as PSA tests for men age 75 and over), we also created indicators for whether beneficiaries qualified for potential receipt of each low-value service (see eAppendix for definitions of these qualifying indicators).

ACO Baseline Levels of Low-Value Services

Because organizations providing more low-value services may have more opportunities to limit wasteful care, we assessed baseline use of low-value services for each ACO and tested whether changes in low-value service use associated with ACO contacts differed between ACOs with higher vs. lower baseline use. We decomposed an ACO’s baseline level of low-value service use into two components. First, we assessed whether the ACO had a greater or lesser risk-adjusted count of low-value services per beneficiary than the control group in the ACO’s service area (eAppendix). Second, we assessed whether the risk-adjusted rate of low-value service use among the control group in each ACO’s service area was greater or less than that of the median among ACO service areas.

This decomposition allowed us to examine whether an organization’s prior performance relative to its service area or service area performance relative to a national average predicted changes under ACO contracts. This distinction bears on whether ACO contracts might be associated with convergence in provider practices within regions or across regions. Baseline levels of low-value care were assessed in 2008 to avoid bias from regression to the mean between the pre-contract period (2009–2011) and 2012; we found no evidence of regression to the mean in the pre-contract period (eAppendix).

Statistical Analysis

We conducted a difference-in-differences analysis to quantify changes in the annual per-beneficiary rate of low-value services in the ACO group that differed from concurrent changes in the control group from the pre-contract period (2009–2011) to the post-contract period (2012), while adjusting for geography and any coincident changes in the groups’ measured patient characteristics. Specifically, we fit the following linear regression model:²¹

$$\mathrm{E}({\mathrm{Y}}_{\mathrm{i},\mathrm{t},\mathrm{k},\mathrm{h}})={\mathrm{\beta }}_0+{\mathrm{\beta }}_1{\text{ACO\_indicators}}_{\mathrm{k}}+{\mathrm{\beta }}_2{\text{HRR\_indicators}}_{\mathrm{h}}\times {\text{Year}}_{\mathrm{t}}+{\mathrm{\beta }}_3{\text{ACO\_contract}}_{\text{kt}}+{\mathrm{\beta }}_4{\text{Covariates}}_{\text{it}}$$

with E(Y_i,t,k,h) denoting the expected value of outcome Y (i.e., count of low-value services) for beneficiary i during year t assigned to ACO or non-ACO TIN k living in HRR h. “ACO_indicators” is a vector of indicators for each organization in the ACO group, omitting the control group as the reference group; “HRR_indicators×Year” is a vector of indicators for each HRR in each year of the sample with a reference HRR-year combination omitted; “ACO_contract” is an indicator for being attributed to a Pioneer ACO in 2012; and “Covariates” include patient sociodemographic and clinical covariates described above. The ACO indicators adjust for organizations’ average levels of low-value services in the pre-contract period, and for changes in the distribution of ACO-assigned beneficiaries across ACOs between the pre-contract and post-contract periods. The HRR indicators mean that estimates are based on comparisons of each beneficiary in the ACO group with control group beneficiaries in the same geographic area, and the interaction of HRR and year indicators adjust for region-specific trends in use of low-value services in the control group.

Thus, the quantity of interest, β₃, is the mean differential change in low-value services for ACO-attributed beneficiaries relative to local changes in low-value service use in the control group. To compare ACOs with higher vs. lower baseline levels of low-value service use, we added to the model two interactions between the β₃ term and each of the two measures of ACOs’ baseline low-value service use.

A key assumption of this difference-in-differences analysis is that differences in adjusted rates of low-value service use between the ACO group and the control group in the pre-contract period would have remained constant in the post-contract period in the absence of the Pioneer program.²² We tested this assumption by comparing trends in low-value service use between the ACO group and control group over the 2009–2011 pre-contract period (eAppendix).

We conducted several sensitivity analyses to test for potential sources of bias. First, we adjusted for any differences in trends in low-value service use between the ACO and control groups in the pre-contract period (eAppendix). Second, we excluded indicators of service qualification as covariates, in case ACO contracts were associated with changes in the likelihood of patients satisfying qualifying conditions. Third, we tested for differential changes in sociodemographic and clinical characteristics from the pre- to post-contract periods between the ACO and control groups. If the composition of the ACO and control groups did not change differentially in these observed dimensions, it is less likely that there were differential changes in other, unobserved, dimensions. All analyses employed robust variance estimators clustered at the level of ACOs (for the ACO group) or HRRs (for the control group).^23,24

Results

The study sample included 693,218 person-years in the ACO group and 17,453,423 in the control group. In analyses adjusted for geographic area, beneficiary characteristics during the 2009–2011 pre-contract period were similar in the ACO and control groups, and differential changes in the ACO group were minimal (Table 2).

Table 2.

Beneficiary Characteristics Before and After Start of Pioneer ACO Contracts

Characteristic	2009–2011		2012		Differential Change for ACO Group	P-Value
	Control Group N=13,041,918	ACO Group N=511,426	Control Group N=4,411,505	ACO Group N=181,792
Age, mean	72.2 ± 0.0	71.9 ± 0.2	72.0 ± 0.0	71.8 ± 0.1	0.1	0.16
Female sex, %	57.4	58.1	57.1	57.8	−0.1	0.54
Race/ethnicity, %
White	83.2	82.2	82.6	81.7	0.1	0.53
Black	8.6	9.1	8.9	9.2	−0.1	0.13
Hispanic	4.8	5.8	5.0	5.8	−0.2	0.10
Other	3.3	2.9	3.5	3.3	0.2	0.03
Medicaid recipient, %	16.3	16.3	16.1	16.0	−0.1	0.76
Disabled,a %	22.0	22.2	22.9	22.8	−0.3	0.06
End-stage renal disease, %	1.2	1.2	1.3	1.2	0.0	0.46
Nursing home resident, %	3.2	2.6	3.1	2.5	−0.1	0.38
CCW conditionsb
Total no., mean	5.6 ± 0.0	5.6 ± 0.1	5.8 ± 0.0	5.7 ± 0.1	0.0	0.32
≥6 conditions	47.8	46.9	50.1	48.9	−0.3	0.28
≥9 conditions	19.3	18.6	21.4	20.5	−0.3	0.25
Low-value service measures qualified for,c total no., mean	14.9 ± 0.0	14.9 ± 0.0	15.0 ± 0.0	14.9 ± 0.0	−0.0	0.47
HCC risk score,d mean	1.3 ± 0.0	1.3 ± 0.0	1.3 ± 0.0	1.3 ± 0.0	0.0	0.99
ZCTA-level characteristics, mean
% below FPL	9.1	8.9	9.1	8.8	0.0	0.83
% with high school degree	75.6	76.3	75.7	76.5	0.1	0.17
% with college degree	19.8	20.6	19.9	20.8	0.1	0.16

Abbreviations: ACO, Accountable Care Organization; CCW, Chronic Conditions Warehouse; HCC, Hierarchical Condition Categories; ZCTA, ZIP Code Tabulation Area. FPL, Federal Poverty Level. Means and percentages were adjusted for geography to reflect comparisons within hospital referral regions. Means are presented ± standard errors.

^aRefers to beneficiaries for whom disability was the original reason for Medicare eligibility.

^bChronic conditions include 25 conditions from the CCW: acute myocardial infarction, Alzheimer’s disease, Alzheimer’s disease and related disorders or senile dementia, anemia, asthma, atrial fibrillation, benign prostatic hyperplasia, breast cancer, chronic kidney disease, chronic obstructive pulmonary disease, colorectal cancer, depression, diabetes, endometrial cancer, heart failure, hip/pelvic fracture, hyperlipidemia, hypertension, hypothyroidism, ischemic heart disease, lung cancer, osteoporosis, prostate cancer, rheumatoid arthritis/osteoarthritis, and stroke/transient ischemic attack.

^cRefers to the number of low-value service measures that could potentially apply to a beneficiary each year. For example, preoperative testing measures only apply to patients who underwent specific surgical procedures. Qualification criteria for all measures are presented in the eAppendix.

^dHCC risk scores are calculated based on Medicare enrollment and claims files from the prior calendar year. Higher scores predict higher subsequent spending.

During the pre-contract period, the adjusted annual rate of low-value service use in the ACO group was 1.8 services per 100 beneficiaries lower (P=0.02) than the control group (Table 3), but trends in the pre-contract period were similar (0.1 services per 100 beneficiaries per year greater for the ACO group; P=0.74). Total spending on low-value services in the pre-contract period was similar for the ACO and control groups ($256 per 100 beneficiaries higher in the control group; P=0.13) and trends were also similar ($20 per 100 beneficiaries per year greater for the control group; P=0.88). In year 1 of Pioneer contracts, there was a differential reduction in the use of low-value services for the ACO group (−0.8 services per 100 beneficiaries; 95% CI: −1.2, −0.4; P<0.001), or a reduction of 1.9% (95% CI: −2.9%, −0.9%) relative to the expected 2012 mean for the ACO group of 41.0 services per 100 beneficiaries. This differential reduction in use corresponded to a 4.5% differential reduction in spending on low-value services (95% CI: −7.5%, −1.4%; P=0.004).

Table 3.

Differential Changes in Use of Low-Value Services in ACO vs. Control Group

Annual Count or Spending per 100 Beneficiaries	Mean for ACO Groupa	Baseline Difference between ACO and Control Group	P-Value	Differential Change (per 100 benes)	95% CI	Differential Change as Percent of ACO Meanb	95% CI	P-Value
Total low-value services, no.	41.0	−1.8	0.02	−0.8	(−1.2, −0.4)	−1.9	(−2.9, −0.9)	<0.001
Total low-value service spending, $	10301	−256	0.13	−459	(−773, −146)	−4.5	(−7.5, −1.4)	0.004
Low-value services by clinical category, no.c
Cancer screening	11	−0.3	0.27	−0.3	(−0.4, −0.1)	−2.4	(−4.1, −0.7)	0.01
Testing	8.7	−0.7	0.01	−0.2	(−0.5, 0.2)	−1.7	(−5.8, 2.3)	0.39
Preoperative Services	2.1	−0.1	0.01	0.0	(−0.1, 0.1)	1.0	(−3.9, 5.8)	0.69
Imaging	14	−0.6	0.05	−0.3	(−0.5, 0)	−1.8	(−3.6, 0)	0.05
Cardiovascular Tests and Procedures	1.0	0.0	0.43	−0.1	(−0.1, 0)	−6.3	(−12.6, 0)	0.05
Other Invasive Procedures	4.4	−0.1	0.22	−0.1	(−0.2, 0.1)	−1.3	(−4.3, 1.7)	0.38
Low-value services by price, no.c
Higher priced	15	−0.7	0.03	−0.2	(−0.5, 0.1)	−1.4	(−3.3, 0.4)	0.13
Lower priced	25	−1.1	0.03	−0.5	(−0.9, −0.2)	−2.1	(−3.5, −0.7)	0.004
Low-value services by sensitivity to patient preferences, no.c
More sensitive	28	−1.4	0.01	−0.5	(−0.9, −0.1)	−1.7	(−3.2, −0.3)	0.02
Less sensitive	13	−0.3	0.17	−0.3	(−0.5, −0.1)	−2.2	(−3.7, −0.7)	0.004

ACO = Accountable Care Organization, CI = Confidence Interval

^aCalculated as the sum of the 2012 control group mean and the adjusted pre-contract difference between the ACO and control group, which approximates the expected 2012 ACO group mean if there we no differential change.

^bCalculated as the differential change divided by the mean for ACO group.

^cNote that the sum of differential changes within each set of service categories equals the total differential change.

All clinical categories of low-value services except for preoperative services contributed to the overall differential reduction in the ACO group (Table 3). The differential reductions were statistically significant for three clinical categories (cancer screening, imaging, and cardiovascular testing and procedures). The greatest absolute reductions in service use occurred for the most frequently delivered services—cancer screening and imaging (Tables 1 and 3). Cardiovascular testing and procedures underwent the greatest reduction in relative terms (differential reduction of −6.3% for the ACO group; P=0.05). In relative terms, differential reductions in low-value service use were similar in magnitude for higher-priced services (−1.4%, 95% CI: −3.3%, 0.4%) and lower-priced services (−2.1%, 95% CI: −3.5%, −0.7%), as well as for services that were more and less sensitive to patient preferences (−1.7%, 95% CI: −3.2%, −0.3%; and −2.2%, 95% CI: −3.7%, −0.7%, respectively).

As shown in Figure 1, ACOs with higher baseline levels of low-value service use than their service area exhibited a differential reduction of 1.2 services per 100 beneficiaries (95% CI: −1.7, −0.7; P<0.001), while ACOs with lower baseline rates experienced a statistically insignificant differential reduction of 0.2 services per 100 beneficiaries (95% CI: −0.6, 0.2; P=0.41; P=0.003 for test of difference in differential reductions between ACO subgroups). Differential reductions in low-value service use were similar for ACOs serving areas with higher or lower baseline levels of low-value service use in the control group (P=0.41)

Figure 1. Differential Changes in Use of Low-Value Services in ACO vs. Control Group, by Baseline Use.

Adjusted differential changes in the annual rate of low-value service use for beneficiaries attributed to Pioneer ACOs vs. the control group from the pre-contract period (2009–2011) to the post-contract period (2012) are presented for the following ACO subgroups: (1) ACOs that served areas with a 2008 mean adjusted count of low-value services per beneficiary in the control group that was greater than vs less than that of the median service area among ACOs, and (2) ACOs with an adjusted count of low-value services per beneficiary in 2008 that was greater vs. less than that of the control group within the ACO’s service area. The number of ACOs within each subgroup is indicated parenthetically. Estimates are displayed with 95% confidence intervals and P-values for the difference between subgroups

Estimates were not substantially affected by adjusting for small differences in trends in low-value service use during the pre-contract period or by omitting service qualification indicators from regression models (eAppendix).

Discussion

Although many studies have examined the effects of various provider payment reforms on health care spending and patient outcomes, the use of low-value services has not been a focus of this literature.²⁵ Use of such services has been shown to fall somewhat following the publication of clinical trials demonstrating their lack of effectiveness,^26,27 but whether payment reforms further discourage use of these services has not been assessed.

We found that the first year of the Medicare Pioneer ACO program was associated with a modest reduction in use of low-value services that could be measured directly with claims data. These results are consistent with the hypothesis that alternative payment models with global budgets can discourage overuse even while preserving broad provider discretion in determining which services are low-value. Notably, the first year of the Pioneer program was associated with a 4.5% differential reduction in spending on low-value services, substantially larger than the 1.2% reduction in overall spending previously estimated with the same methods.¹⁵ This finding suggests that Pioneer ACOs targeted low-value services in their efforts to reduce spending, despite a lack of financial incentives or quality reporting requirements specifically concerning overused services.

Utilization changes occurred broadly across multiple clinical categories. Relative reductions were similar for higher-priced and lower-priced services, suggesting that overall reductions in low-value service use were driven not just by restrictions on service use that could have occurred without causing significant losses in reimbursement under fee-for-service payment.²⁸ Differential reductions in low-value service use were also similar for services that were more or less sensitive to patient preferences. This finding is consistent with clinicians in ACOs recommending fewer low-value services and with research demonstrating that patient preferences may not be major obstacles to reducing low-value service use.^25,29,30

Reductions in low-value service use were concentrated among ACOs with higher baseline levels of use of these services relative to their service areas, whereas baseline performance of ACO service areas did not predict reductions in low-value service use. First, these findings suggest that ACO initiatives may produce greater reductions in overuse if they encourage participation of provider organizations with more wasteful practices at baseline than other providers in their area. Second, these findings highlight the importance of practice variation within regional markets rather than across markets in predicting organizations’ prospects for improving efficiency.³¹ In service areas where overuse is especially common, providers may face difficulties in reducing low-value service use markedly below local norms.

Several limitations of this study warrant discussion. First, organizations selecting into the volunteer Pioneer program may have been uniquely well positioned to identify and reduce wasteful practices. Consequently, similar results may not be achieved if the Pioneer program or similar programs are expanded to include a different set of provider organizations. Second, although our difference-in-differences study design controls for fixed differences between the ACO group and control group, and even though we detected no difference in temporal trends in low-value service use between these groups, it is nevertheless possible that an independent contemporaneous factor affecting ACOs produced a differential change in 2012. It is also possible that organizations entering the Pioneer program may have differentially reduced low-value service use even in the absence of the program. However, we found no evidence that these organizations were experiencing faster reductions in low-value service use prior to the ACO contracts. In addition, reductions in use of higher-priced low-value services would cause substantial losses in fee-for-service revenue in the absence of ACO contracts, and we found that reductions were unrelated to service price.

Third, given the limited number of organizations participating in the ACO program, we could not assess the many organizational characteristics that might modify reductions in use of low-value services. Fourth, we only examined the first year of the Pioneer ACO program, the only year for which claims data were available. Although prior studies have shown increasing effects of commercial ACO contracts over time,³² the same pattern may not hold in Medicare. Finally, our results do not constitute conclusive evidence of value improvement among Pioneer ACOs. It is possible that important high-value services also experienced reductions in 2012.

Conclusions

The Pioneer ACO program was associated with a modest reduction in low-value services, with greater reductions within organizations providing more low-value care. Despite the limitations of our study, our findings, taken together with studies demonstrating spending reductions greater than Medicare bonus payments¹⁵ and improved or stable performance on measures of patient experiences and quality,¹¹ are consistent with the conclusion that the overall value of health care provided by Pioneer ACOs improved after their participation in an alternative payment model. Finally, our study demonstrates the utility of novel measures of low-value service use for evaluating the effects of health care policy initiatives.