For Diabetes Shared Savings Programs, 1 Year of Data Is Not Enough

Abstract

Fee-for-service payment models are moving toward pay-for-performance designs, many of which rely on shared savings for financial sustainability. Shared savings programs divide the cost savings between health care purchaser and provider based on provider performance. Often, these programs measure provider performance as the delivery of agreed-upon clinical practice guidelines that usually are represented as evidence-based medicine (EBM). Multiyear studies show a negative relationship between total cost and EBM, indicating that long-term shared savings can be substantial. This study explores expectations for the rewards in the first year of a shared savings program. It also indicates the effectiveness of using 1 year of claims to assess cost savings from evidence-based care, especially in a patient population with high turnover. This study analyzed 1956 adults with diabetes insured through Medicaid. Results of linear regression showed that the relationship between total cost of care and each element of evidence-based medical care during a 1-year period was positive (higher cost) or insignificant. The results indicate that diabetes EBM programs cannot expect to see significant cost savings if the evaluation lasts only 1 year or less. The study concludes that improvements in EBM incentive programs could come from investigating the length of time needed to realize cost savings from each element of diabetes EBM. Investigating other factors that could affect the expected amount of cost savings also would benefit these programs, especially factors derived from sources external to insurance program information such as the medical record and care management data.

Introduction

The Affordable Care Act escalated the US health care system's efforts to lower costs and improve outcomes.¹ The population of patients with diabetes mellitus presents great potential for these improvements because it presents a fast-growing health and financial burden. Diabetes prevalence grew from 1% of the US population in 1958 to 9% in 2014. In addition, the US Centers for Disease Control and Prevention (CDC) estimated that in 2014, 37% of the nondiabetic population was prediabetic, indicating continued growth of this disease in the population. The most troubling aspect of trends in diabetes is the growth in prevalence and severity among people younger than 65 years old. According to the CDC, more than 20,000 US people younger than 20 years old had type 2 diabetes (acquired, not childhood diabetes) in 2009. At that time, the expected increase in this young diabetes population was 5000 new cases per year. Furthermore, recent research indicates that Medicaid enrollees are developing diabetes at a younger age and have increased diabetes severity.²

This growth not only challenges the quality of life but also the ability to pay for managing this disease. The total cost of care (inpatient, outpatient, professional, and prescription) was estimated to be $245B for persons with diabetes in the United States in 2012. The cost of care for those with diabetes in the United States represents about 8.5% of total national spending on health care.³ Diabetes is also a precursor to other chronic diseases, such as coronary artery disease, renal disease, and hypertension, which also are chronic diseases that add cost burden for this population.

To address these challenges, private and public payers are working with health care providers to implement payment reform programs. One common approach is shared savings programs. Shared savings rewards health care providers for delivering evidence-based medicine (EBM). EBM defines practice that is widely agreed upon as necessary and sufficient to improve health outcomes for people with specific conditions. These practices are based on clinical evidence, consensus, and research. Evidence presented in the medical body of knowledge shows that by delivering EBM, patients will be healthier and therefore incur lower cost than patients who do not receive EBM. The health care purchaser/payer gives a portion of the shared savings to the health care provider as an incentive to deliver EBM. For diabetes, EBM often derives from the American Diabetes Association recommendations.⁴

In theory, shared savings is self-sustaining, paying for itself through the cost savings. Figure 1 illustrates the model for shared savings programs. Self-sustaining qualities of shared savings programs have made it the business model of choice for many payment reform programs. Examples of current shared savings programs are as follows:

FIG. 1.

Model for shared savings program. This figure illustrates the structure for shared savings programs. These programs use claims data to determine cost savings resulting from evidence-based care delivered during that year. The programs rely on the belief that evidence-based care leads to better outcomes, which in turn leads to lower total cost of care within a 1-year period. The underlying assumption is that sustainable incentives funding results from those cost savings.

• • PROMETHEUS Payment Program⁵

• • Anthem Blue Cross Blue Shield Patient-Centered Primary Care or “PC2”⁶

• • Cigna Collaborative Accountable Care or “CAC”⁷

• • Medicare Shared Savings Program⁸

• • Colorado State Medicaid Regional Coordinated Care Organization (RCCO) Patient-Centered Medical Home (PCMH)⁹

These programs use 1 year of claims data to measure provider performance because participants prefer annual reimbursements to align with corporate budgeting or to meet the constraints of grant report deadlines. In addition, the use of claims data reduces the reporting burden on practices participating in the program. For these programs, if cost goals are not achieved in any 1-year measurement cycle, the program sponsor typically will not reward the provider for that cycle. The shared savings program yearly metric for provider performance and outcome improvement is based on the assumption that there is an association between lower annual total cost of care and delivery of EBM in that same year. The present study investigates the validity of that assumption by analyzing the association between cost and receipt of EBM in a 1-year period.

Research shows that cost savings can result from delivery of EBM for diabetes. EBM, derived from the American Diabetes Association guidelines, includes the 7 elements listed in the following Methods section: HbA1c test, hepatic function panel, lipid panel, urine test, prevention/education, eye examination, and office visits. Kralewski et al¹⁰ showed a significant relationship between reduced cost of care and delivery of 3 elements of EBM (lipid test, HbA1c, and nephropathy test) when accompanied by high patient engagement and specific practice quality characteristics in a population of Medicare patients with diabetes. Salmon found that PCMHs showed promising cost results after 2 years of improved diabetes care; however, the investigation only included costs directly related to diabetes.

In contrast with the results of the multiyear programs, the results of year-by-year programs have not met expectations. Abraham et al¹¹ retrospectively examined clinical and administrative data for insured employee members of a benefits plan for 1–4 years. Results showed that high-quality preventive and primary care combined with effective patient self-management did not lead to lower near-term cost, and intermediate clinical outcomes did not translate to lower cost of care. Doran and Kontopantelis¹² reviewed research on incentive programs and found that those programs that provided significant monetary compensation for care delivery showed success in driving quality care, but not in lowering cost. The programs also had negative side effects in that they eroded providers' intrinsic motivation, narrowed focus, promoted unethical behavior, and ultimately increased health care inequalities. The articles reviewed described programs that collected assessment data for only 1 or 2 years.

The difference in findings between the Kralewski study and the Doran and Abraham studies suggests that the time frame may be a critical element in the relationship between EBM and lower cost. The present investigation examines the existence of a relationship between EBM and cost in 1 year of data. If no significant relationship exists between EBM and lower 1-year costs, payment reform programs would not effectively be based on these short-term cost savings from delivering EBM.

Methods

Framework

Payment incentive programs focus on incentives for providing quality care and not on changes in patient behavior. The reason for this focus is primarily that fairness clauses in the Patient Protection and Affordable Care Act make it difficult for reimbursement programs to selectively provide incentives for self-care to patients with diabetes (or any subgroup). The framework for this study, illustrated in Figure 2, examines only the provider delivery of EBM as affected by patient characteristics and risk factors.

FIG. 2.

Study framework. This framework for evaluating the relationship between evidence-based care and total cost of care shows a simple structure for examining shared cost savings models that relies on provider performance. This framework supposes that total cost of care is the result of providers meeting their responsibilities resulting in evidence-based care, but is affected by patient attributes and health risk factors. Shared savings models are assumed to adjust total cost of care for patient attributes and risk factors, and therefore would reward providers for meeting their responsibilities.

Study design

This observational study investigated the relationship between total cost of care and EBM using linear regression. The study analyzed 1 year of medical and pharmaceutical claims data for the population with diabetes. To avoid bias described by Linden,¹³ this study examined only patients with diabetes identified through a combination of claims data and medical record validation who were in the population for the entire period covered by the study (12 months). To control for a potential effect of patient attributes, the analysis included patient characteristics described in the following Variables section. The study used de-identified data and the Colorado Institutional Review Board classified this human subjects study as exempt from review. The data met all 18 Health Insurance Portability and Accountability Act (HIPAA) criteria for de-identification.

Study sources, time period, and selection of study cohort

The primary source of data is claims data from Colorado Medicaid Regional Collaborative Care Organizations Region 7 (RCCO 7). RCCO 7 is 1 of 7 multicounty regions that Colorado Medicaid designates to manage PCMHs. This RCCO encompasses 4 counties comprising both rural and urban populations, including the city of Colorado Springs. The data supplier was Community Health Partnership (CHP), a coalition of independent provider groups that manages RCCO 7, which partnered with the University of Colorado Health Sciences Graduate School to conduct this study. CHP engaged Treo Solutions (Troy, New York; since acquired by 3M HIS) to collect and process all CHP patient data. CHP provided de-identified medical and pharmaceutical claims data along with HIPAA-compliant patient characteristic data. Treo Solutions had previously cleaned all data of erroneous values and filtered to identify patients with diabetes.

CHP supplied patient data from January 1, 2014, through December 31, 2014, to the research team. The study cohort consisted of CHP PCMH Medicaid patients identified with either type 1 or type 2 diabetes. Treo Solutions processed this data for CHP, identifying clients with diabetes as those with diagnostic, procedure, or prescription claims having codes listed in the 3M Episode Disease Category 424 for Diabetes (unpublished data, Community Health Partnership, August 24, 2015). Treo supplied the patient cohort's data to CHP: total cost of care, all reimbursement codes, demographics, and the health risk measure. CHP directly identified which patients received care management. Patient identity was masked by replacing patient identifiers with sequential numbers and including only the diagnostic and procedure codes, drug short descriptions, and nonidentifying demographics.

Initial data for the population included 3829 people diagnosed with diabetes. Study exclusions were children ages 0 to 17 years and adults older than 89 years (231 people), pregnant women (220 people), people who had not been members for the entire year of the study (872 people), people whose primary chronic disease was not diabetes (274 people), and people who were dual eligible for both Medicaid and Medicare (504 people). Three people had missing cost data and thus were excluded from the study. These exclusions resulted in a final study population of 1956 people.

Primary outcome measures

Shared savings programs assume there is a negative relationship between total cost of care and the delivery of evidence-based care; that is, evidence-based care is associated with a decreased total cost of care. This relationship would manifest itself if the coefficients for the variables indicating delivery of EBM in the fitted regression model were negative.

Study variables

Predictors

The study investigated the relationship between the total cost of care and each element of diabetes EBM. Total cost of care was the allowed reimbursement amount of all claims for each patient with diabetes. The elements of diabetes EBM are the annual practices that form the basis for the diabetes core price in the PROMETHEUS model¹⁴:

• • Office visit—Primary care provider (PCP) office visit 2 or more times

• • HbA1c—Administered 2 or more times

• • Lipid panel—Taken at least once

• • Eye examination—Opthalmology visit with dilation done at least once

• • Hepatic function—Assessed at least once

• • Urine protein—Tested at least once

• • Preventive education—Provided at least once

CHP collaborated to form a list of medical and pharmaceutical reimbursement codes that would indicate the delivery of the elements of evidence-based care. If these codes existed in the frequencies indicated in the preceding bulleted list, the variable corresponding to the element of care would be set to 1; otherwise, it would be set to 0.

Control variables

A number of patient attributes and risk factors have been found to affect the relationship between quality and total cost of care. The study used the variables defined below to control for their effect on the statistical estimation of this relationship.

• • Patient attributes

  • • Age—Annual cost declines with age, depending on age of initial diagnosis¹⁵
  • • Sex—Females incur higher health care cost than males¹⁶

    • ○ Sex = 1 if female
  • • Urban/rural—People in rural areas have poorer access to care and tend to be sicker than their urban counterparts¹⁷

    • ○ Rural/urban = 1 if identified as living in an urban zip code
• • Risk factors

  • • Obesity—People who are obese incur higher health care cost on average^18,19

    • ○ Obesity = 1 if medical or prescription reimbursement codes associated with obesity are present in any claim for the person
  • • Pain—People in pain tend to incur higher cost of care²⁰

    • ○ Pain = 1 if medical or prescription reimbursement codes associated with pain are present in any claim for the person
  • • Depression/anxiety—People who suffer depression have more disease and inpatient stays²¹

    • ○ Depression/anxiety = 1 if medical or prescription reimbursement codes for depression or anxiety are present in any claim for the person
  • • Care management—Patients not receiving care management incur higher cost than those not receiving this care²²

    • ○ Care management = 1 if attributed to a care manager
  • • 3M disease risk category—This metric developed by 3M measures a person's severity of illness; higher severity is associated with higher cost^23–25

    • ○ Disease risk = standard normal transformation of the 3M Clinical Risk Group metric that rates disease severity on a scale of 1 to 7²⁶

The reimbursement codes used for these determinations appear in Table 1.

Table 1.

Codes Used to Identify Values of Explanatory Variables

EBM element	Codes to identify EBM service
HbA1c	83036, 86037, 3046F, 3047F
Office visit	99201–99205, 99212–99215, 99024, 99241–99244, 99271, 99272–99275
Hepatic function panel	80076
Preventive	99381–99429
Eye examination	99203, 99204, 99205, 99213, 99214, 99215, 99242, 99243, 99244, 99245, 92002, 92004, 92012, 92014, 92018, 92019, 92225, 92226, 92230, 92235, 92240, 92250, 92260, 67028, 67030, 67031, 67036, 67039, 67040, 67042, 67043, 67101, 67105, 67107, 67108, 67110, 67112, 67113, 67121, 67141, 67145, 67208, 67210, 67218, 67220, 67221, 67227, 67228, S0620, (HCPCS)S0621, S0625, S3000, (ICD9-CPT)14.1x, 14.2x, 14.3x, 14.4x, 14.5x, 14.9x, 95.02, 95.03, 95.04, 95.05, 95.16
Metabolic panel	80048
Lipid panel	80061
Urine protein	84155, 80150

These medical codes were used to identify the elements of evidence-based medicine that appeared in the claims for the study population. These codes were used to identify the delivery of evidence-based medicine to each subject.

CPT, Current Procedural Terminology; EBM, evidence-based medicine; HCPCS, Healthcare Common Procedure Coding System; ICD9, International Classification of Diseases, Ninth Revision.

Data analysis

This study examined the relationship between total cost of care and delivery of EBM using linear regression. This analysis indicated how much cost of care is predicted to change for a given change in any one of the predictors or covariates, keeping the other predictors and covariates constant. The equation representing the analysis model was

Ln

where TCoC represented the total allowed reimbursement amount of all claims for a patient with diabetes and each of the EBM variables represented the delivery of the following elements of EBM as described in the equation:

• • EBM1—Office Visit

• • EBM2—HbA1c

• • EBM3—Lipid panel test

• • EBM4—Eye examination

• • EBM5—Hepatic function test

• • EBM6—Urine protein test

• • EBM7—Preventative

The distribution of total cost of care data in a general population tends to be skewed with many data at low numbers and fewer at higher numbers. To remove this skewness, the natural log of the total cost of care [ln(TCoC)] in the regression model was used.

To understand the relationship between the covariates and the explanatory variables, the study team calculated correlations between variables, finding low to moderate correlations only. To further characterize the covariates with respect to obtaining EBM, the team examined odds ratios reported in Table 2. It was found that the odds of receiving EBM elements is higher for people who are female, obese, with depression/anxiety or pain, or in 3M risk categories 3–6. Urine protein tests were reported for only 6 people. Because this number is so small, urine protein tests were excluded from the odds ratio analysis.

Table 2.

Description of Binary Variables

			HbA1c					Office visits					Hepatic function test
	Population		HbA1c			OR 95% confidence interval		Office visits			OR 95% confidence interval		Hepatic function test			OR 95% confidence interval
Total N = 1956	N	%	N	Y	Odds ratio (OR) 0/1	Lower	Upper	N	Y	Odds ratio (OR) 0/1	Lower	Upper	N	Y	Odds ratio (OR) 0/1	Lower	Upper
Client sex
M	737	38	385	352	1.096	0.912	1.316	468	269	1.616	1.340	1.949	704	33	1.221	0.796	1.874
F	1219	62	609	610				632	587				1153	66
Urban
N	364	19	192	172	1.100	0.875	1.381	215	149	1.153	0.915	1.452	340	24	0.700	0.436	1.126
Y	1592	81	802	790				885	707				1517	75
Care management
N	1159	59	574	585	0.881	0.735	1.055	665	494	1.120	0.934	1.343	1099	60	0.942	0.623	1.425
Y	797	41	420	377				435	362				758	39
Obesity
N	223	11	215	8	32.912	16.148	67.081	192	31	5.627	3.807	8.319	218	5	2.501	1.006	6.216
Y	1733	89	779	954				908	825				1639	94
Depression/anxiety
N	1093	56	589	504	1.322	1.105	1.581	719	374	2.432	2.024	2.922	1051	42	1.770	1.176	2.664
Y	863	44	405	458				381	482				806	57
Pain
N	1034	53	540	494	1.127	0.943	1.346	740	294	3.929	3.252	4.747	1000	34	2.231	1.459	3.411
Y	922	47	454	468				360	562				857	65
Risk category 1
N	1636	84	789	847	0.523	0.408	0.670	818	818	0.135	0.095	0.192	1547	89	0.561	0.288	1.090
Y	320	16	205	115				282	38				310	10
Risk category 2
N	1414	72	708	706	0.898	0.736	1.094	740	674	0.555	0.451	0.682	1336	78	0.690	0.422	1.130
Y	542	28	286	256				360	182				521	21
Risk category 3
N	1579	81	812	767	1.134	0.906	1.420	909	670	1.321	1.055	1.655	1495	84	0.737	0.421	1.293
Y	377	19	182	195				191	186				362	15
Risk category 4
N	1613	82	834	779	1.225	0.969	1.547	951	662	1.870	1.478	2.367	1535	78	1.283	0.781	2.109
Y	343	18	160	183				149	194				322	21
Risk category 5
N	1749	89	910	839	1.588	1.185	2.129	1022	727	2.325	1.728	3.128	1668	81	1.961	1.152	3.340
Y	207	11	84	123				78	129				189	18
Risk category 6
N	1789	91%	917	872	1.229	0.894	1.689	1060	729	4.617	3.196	6.668	1704	85	1.834	1.018	3.306
Y	167	9%	77	90				40	127				153	14

	Eye examination					Preventive					Metabolic panel					Lipid panel
	Eye examination			OR 95% confidence interval		Preventative			OR 95% confidence interval		Metabolic panel			OR 95% confidence interval		Lipid panel			OR 95% confidence interval
	N	Y	Odds ratio (OR) 0/1	Lower	Upper	N	Y	Odds ratio (OR) 0/1	Lower	Upper	N	Y	Odds ratio (OR) 0/1	Lower	Upper	N	Y	Odds ratio (OR) 0/1	Lower	Upper
Client sex
M	63	674	1.906	1.315	2.760	667	70	3.166	2.397	4.182	566	171	1.349	1.092	1.666	245	492	1.156	0.950	1.407
F	57	1162				915	304				866	353				367	852
Urban
N	28	336	1.359	0.876	2.108	294	70	0.991	0.742	1.324	258	106	0.867	0.673	1.115	121	243	1.117	0.876	1.423
Y	92	1500				1288	304				1174	418				491	1101
Care management
N	78	1081	1.297	0.881	1.909	942	217	1.065	0.847	1.338	851	308	1.027	0.838	1.259	357	802	0.946	0.779	1.149
Y	42	755				640	157				581	216				255	542
Obesity
N	81	142	24.777	16.302	37.657	208	15	3.623	2.118	6.197	208	15	5.766	3.380	9.837	223	0	4.455	4.082	4.862
Y	39	1694				1374	359				1224	509				389	1344
Depression/anxiety
N	94	999	3.029	1.943	4.722	910	183	1.413	1.128	1.772	850	243	1.689	1.381	2.066	361	732	1.202	0.991	1.459
Y	26	837				672	191				582	281				251	612
Pain
N	100	934	4.829	2.962	7.872	840	194	1.050	0.838	1.316	838	196	2.361	1.922	2.900	337	697	1.138	0.939	1.378
Y	20	902				742	180				594	328				275	647
Risk category 1
N	81	1555	0.375	0.251	0.561	1322	314	0.972	0.715	1.320	1146	490	0.278	0.192	0.403	513	1123	1.020	0.787	1.321
Y	39	281				260	60				286	34				99	221
Risk category 2
N	87	1327	1.011	0.669	1.529	1139	275	0.926	0.718	1.194	992	422	0.545	0.427	0.695	423	991	0.797	0.646	0.984
Y	33	509				443	99				440	102				189	353
Risk category 3
N	96	1483	0.952	0.600	1.511	1276	303	0.977	0.733	1.302	1169	410	1.236	0.966	1.582	505	1074	1.186	0.926	1.520
Y	24	353				306	71				263	114				107	270
Risk category 4
N	109	1504	2.187	1.163	4.113	1313	300	1.204	0.904	1.603	1200	413	1.390	1.080	1.789	520	1093	1.298	1.000	1.685
Y	11	332				269	74				232	111				92	251
Risk category 5
N	114	1635	2.336	1.015	5.378	1417	332	1.086	0.759	1.556	1310	439	2.079	1.544	2.799	544	1205	0.923	0.679	1.255
Y	6	201				165	42				122	85				68	139
Risk category 6
N	113	1676	1.541	0.706	3.363	1443	346	0.840	0.551	1.282	1343	446	2.639	1.912	3.643	555	1234	0.868	0.621	1.214
Y	7	160				139	28				89	78				57	110

This table illustrates the chance of receiving EBM for each of the controls. Odds ratio confidence intervals show uneven odds of obtaining EBM based on sex, obesity, depression/anxiety, pain, and risk.

EBM, evidence-based medicine.

Results

Descriptive analysis

The descriptive analysis of the dichotomous variables appears in Table 2. Information on age appears in Figure 3. The average age was 47 years with a standard deviation of 11 years and a median of 49 years. The majority of people fell into the age category of 34–65 years old. People who were 20–30 years old averaged a higher cost of care than people 28–65 years old.

FIG. 3.

Age histogram and average ln(TCoC) for each distribution of age in the study population is skewed left with an average age of 47 years and a predominance of middle-aged people. Unexpectedly, the average ln(TCoC) for people ages 20–29 (8.82) is significantly higher at the 0.01 level (1-tailed Student t test P = 0.03) than for ages 30–64 (8.63). Younger people are expected to be healthier and less costly. This anomaly indicates a possible area for future investigation. ln(TCoC) represents the total allowed reimbursement amount of all claims for a patient with diabetes.

Of the 1956 patients with diabetes in the study, Table 2 shows that the odds of not receiving EBM elements were higher for people who were male, not obese, without depression/anxiety, without pain, or in 3M risk categories 1 and 2. Each of the following characteristics was present in 40%–50% of the study population: male; depressed; taking drugs for pain; engaged in care management. Eighty percent of the study population lived in urban areas and 90% were obese. People who lacked codes identifying obesity also lacked any codes for lipid panel. Furthermore, the nonobese were much less likely to show codes for any EBM. Each of the 3M risk categories held 10%–20% of the population, and the likelihood of receiving EBM increased with risk level.

Regression analysis

Results from performing the linear regression using the model described appear in Table 3, columns 2–6.

Table 3.

Linear Regression Results

		Unstandardized coefficients			95% confidence interval for B
	N	B	Standard error	Significance	Lower bound	Upper bound
(Constant)		6.972	0.135	<0.001	6.708	7.236
Explanatory variables
HbA1c	962	0.144	0.045	0.001	0.057	0.232
Office visits	856	0.524	0.048	<0.001	0.431	0.618
Hepatic function test	99	0.273	0.094	0.004	0.089	0.457
Eye examination	1856	0.268	0.095	0.005	0.082	0.455
Metabolic panel	524	0.241	0.050	<0.001	0.143	0.339
Preventive	374	0.102	0.053	0.052	−0.001	0.205
Lipid panel	1344	−0.031	0.054	0.562	−0.138	0.075
Urine protein	6	0.177	0.363	0.625	−0.534	0.889
Covariates
Age	1956	−0.003	0.002	0.063	−0.007	0.000
Client sex	1956	−0.068	0.043	0.115	−0.152	0.017
Urban	1592	−0.071	0.052	0.169	−0.173	0.030
Obesity	1733	0.266	0.082	0.001	0.104	0.428
Depression/anxiety	863	0.289	0.045	<0.001	0.202	0.377
Pain	922	0.150	0.044	0.001	0.064	0.237
Care management	797	−0.084	0.042	0.045	−0.165	−0.002
Risk category (1, 2, 3, 4, 5, 6)	320, 542, 377, 343, 207, 167	0.297	0.015	<0.001	0.268	0.327

Variables with coefficient estimates that were significant at the 0.05 level are indicated in bold.

The EBM variables HbA1c, office visits, hepatic function tests, eye examinations, and metabolic panels were statistically associated with an increased total cost of care (P < 0.05). Covariates that impacted total cost of care were care management, depression/anxiety, obesity, pain, and risk. Care management was associated with a lower total cost of care (negative coefficient estimate). Obesity, depression/anxiety, pain, and risk were associated with a higher total cost of care (positive coefficient estimate). Furthermore, delivering all elements of evidence-based care predicts a higher, although trivial, cost ($5) than delivering no elements of evidence-based care in the first year. The regression model accounted for 45% of the variation in the transformed total cost of care (R² = 0.45). There were no obvious patterns in the residual plots, which means the linear regression model was an appropriate way to evaluate the relationship between EBM and total cost of care.

The 3M Clinical Risk Group classification showed the largest effect of any of the covariates. Because of the importance of this variable, one alternative risk metric was substituted for the 3M metric (Charlson comorbidity index) with no appreciable change to the results. (The Charlson comorbidity calculation requires that procedure codes are present for each study subject. Some of the subjects [41 people] in this study were identified with diabetes through pharmacy claims only. Those subjects were included in the sensitivity study by using the average Charlson comorbidity for the rest of the population as the value for these subjects.) This finding is consistent with previous research.²⁷ Sex and care management showed a significant association with cost reduction in a year. Depression/anxiety was associated with increased cost of care. Age, urban location, and obesity did not impact total cost of care.

Discussion

One year of claims data showed no significant association between EBM and lower total cost of care. This result is consistent with general clinical expectations that outcome improvements from EBM rarely can be detected within 1 year. Although some predictor variables were related to total cost of care (HbA1c, office visits, hepatic function test, eye examination, and metabolic panel), other specific elements of evidence-based care (preventative care/education, lipid panel, and urine protein) were not. The fact that the model accounted for 45% of the variation in the transformed total cost reduces uncertainty associated with individual-level and population-level variation.

These conclusions imply risk to payment reform programs that reward providers for delivering EBM by sharing savings calculated from just 1 year of claims data. The model predicts that 1-year costs will be $5 higher as a result of delivering EBM. Based on multiyear studies that show cost reduction from EBM, one can conclude that shared savings payment reform participants cannot expect to receive cost savings rewards from programs that assess performance based on 1 year of data. In addition, PCPs who have high patient turnover may not be able to show lower cost by delivering EBM even in periods longer than 1 year. Low yearly patient turnover would be an important criterion to assess before qualifying a practice for a shared savings program.

Because it is costly to change practice structure and workflow, rewarding with shared savings over 1 year may motivate unintended behavior needed to produce cost savings, as seen in the Abraham study. That study illustrates the difficulty of showing the relationship between economic benefit and best clinical practice in the 1-year period used by current payment reform programs. A more effective approach might be to share savings realized over the expected time horizon required for each element of EBM to show results. This approach would require knowledge of these time horizons and of required control variables, and yet, still may exhibit large variances related to unpredictable individual and population differences. Furthermore, current thinking indicates that provider incentive and evaluation programs based on achieving patient engagement and behavioral changes may be more effective than shared savings programs to reduce total cost of care.

In addition to increasing knowledge of time horizons required to see the results of EBM, another potential area for further investigation is the effect of age on cost of care for Medicaid patients with diabetes. The present study observed that total cost of care was high for people between the ages of 20 and 30 years. This observation depicted in Figure 3 is not supported by the literature²⁸; however, it was not unexpected. Anecdotal evidence from CHP indicates that younger people are being first diagnosed in association with expensive episodes and are more resistant to stop risky behaviors. This observation offers potential opportunity for further population health research despite the fact that the model showed age to have an insignificant effect on total cost of care in the present study.

Limitations

To reflect the 1-year shared savings and provider evaluation practices, this study's framework does not include a few key influencing factors. First, the framework does not fully account for patient engagement with their health; however, evidence has shown that patient engagement is a key factor in health improvement.²⁹ Medicaid currently does not provide specific patient incentives to improve health behavior, and the current shared savings programs similarly do not provide patient incentives because of regulatory restrictions. Care management and education are currently the best practices for the provider to influence patient behavior and the model included this element of care. Second, although patient satisfaction with care influences health outcomes,³⁰ the framework also does not account for potential effects of patient satisfaction with their care, nor does it address the effects of social determinants of health that can determine cost of care.³¹ These limitations reduce the applicability to health care programs that involve patient motivational factors; however, the payment reform programs addressed by this study do not generally take these factors into account.

Because payment reform programs based on 1-year evaluation do not account for year-to-year variation, the present study did not examine this variation. The study team also did not examine the variation and accuracy of the covariates representing comorbidities (depression/anxiety, pain, and obesity); however, the methods of detection reflected the current practice for the care of this population. In addition, the study team only examined data for people in the program for the entire year, excluding any variation introduced through patient turnover. Considering these limitations, it is reasonable to have confidence in the model because it was able to explain almost half of the in-year variation. Finally, the variation introduced by small populations represented in most payment reform plans adds another risk to shared savings programs based on 1 year of data. These unexamined variabilities worsen the case for relying on 1 year of data to show cost savings from EBM.

Although CHP requires all of the EBM elements for diabetes care, only 6 people had claims for urine protein tests and only 19% of the study population had claims for preventive education. These 2 elements of care are not reimbursed and are typically underreported, according to CHP. In addition, only 5% of the study population had codes for hepatic function tests and 27% for metabolic panel tests. Despite these issues, the model provides relevant results for shared savings programs because those programs use the same type of data as the present study when assessing provider performance.

Medicaid study populations provide advantages and disadvantages to evaluations such as the one in the present study. For many reasons, Medicaid PCMH populations provide a good testing ground for the relationship between EBM and lower total cost of care for patients with diabetes. First, federal standards for PCMH require certification of 5 required functions and attributes³²: comprehensive care, patient-centered, coordinated care, accessible services, and quality and safety. These quality standards indicate that Medicaid PCMH providers would deliver higher quality care than most other providers, better enabling them to derive lower cost from EBM within a year. Second, diabetes prevalence tends to be high in these high-risk Medicaid populations, making it easy to identify a study population large enough to produce significant results. Third, Medicaid populations tend to be at higher risk than the general population, making them more likely than other populations to show positive results from EBM delivery within a year. Finally, Medicaid programs typically evaluate PCMH providers using 1 year of claims data, and therefore results from studying this population would provide useful information for these other yearly programs.

Research involving Medicaid populations is challenging, however. Many Medicaid patients disqualify for benefits periodically depending on their employment and income status. Therefore, Medicaid populations experience high turnover, making the study of outcomes from multiyear care difficult. Furthermore, according to the Centers for Medicare & Medicaid Services, about 30% of Medicaid patients with diabetes manage their disease without visiting a medical care provider.³³ Their knowledge of and loyalty to their assigned PCP in the PCMH is thought to be inconsistent according to anecdotal accounts from CHP. The model used in this study excluded any members who were unattributed to a PCP for the entire period to manage these risks of using Medicaid data.

All of the shared savings programs cited, and many other types of payment reform programs, rely on claims data to assess provider performance, cost, and outcomes. Claims data are not an entirely precise or accurate reflection of the care delivered.³⁴ Clinical data, such as health records, also provide health information; however, health records are rarely consolidated across all of a patient's providers. A combination of claims and health record data can yield more accurate and precise health care information than either data source alone. However, for research to be relevant to current shared savings programs, claims data alone should be used.

Claims data present advantages and disadvantages relative to shared savings programs. Claims provide practice and condition information in a more standard form than current health records (for instance, International Classification of Diseases, Ninth Revision codes), enabling quantitative comparison. Claims data also associate cost information with procedure and diagnosis. Claims data are known to underestimate care delivered, especially for people who are discontinuously covered by health care insurance.³⁵ Claims data challenges, however, include data inaccuracy and incompleteness. Hicks³⁶ found that claims data alone were insufficient to determine the approximation of two thirds of 553 quality indicators. Claims data are notoriously incomplete because of imperfections in the ability of standard claims codes to reflect the breadth of care and the lack of reporting for nonreimbursable services.³⁷ Despite the challenges, claims data for continuously enrolled people are considered to be a reasonable representation of the care delivered to a patient, especially in the absence of other information.³⁸

Conclusion

This study's results indicate that diabetes EBM programs cannot expect to see significant cost improvements if the evaluation lasts 1 year or less. Participants in diabetes care shared savings incentive programs can expect to incur costs in the first year of the program. Improvements in EBM incentive programs could come from investigation of the length of time needed to adequately evaluate each element of diabetes EBM. Investigating other factors that could affect the expected amount of cost savings also would benefit these programs, especially factors derived from sources external to insurance program information such as the medical record. These opportunities are especially important with respect to younger generations, for whom the rate and cost of diabetes may be increasing.

Acknowledgments

Thanks to Helen Harris and the Community Health Partnership for providing data and reviewing results. Thanks also to Dick Carpenter, PhD, Professor of Leadership, Research, and Foundations, College of Education, University of Colorado, Colorado Springs and to Jill Van Den Bos, ASA, MAAA, MA, Principal and Consulting Actuary at Milliman for their review and suggestions for this article.

Author Disclosure Statement

Ms. VanArsdale and Drs. Curran-Everett, Haugen, Smith, and Atherly declared no conflicts of interest with respect to the research, authorship, and/or publication of this article. The authors received the following financial support: Supported by NIH/NCATS Colorado CTSI Grant Number UL1 TR001082. The contents are the authors' sole responsibility and do not necessarily represent official NIH views.