Table 2.

Comparison of Matching Metrics and Causal Effects Estimated from Coarsened Exact Matching (CEM) and Propensity Score Matching (PSM)^*

		L1 Metric**		Wald Test
Matching Method†,‡	n	Pre	Post	Pre	Post	LL***	Estimated Causal Effect (PCMPM)†
CEM – A	15,027	0.42	0.00	0.81	0.84	−125.2	−$26.46
PSM – A	13,120	0.42	0.23	0.81	0.64	−106.7	−$19.85
PSM – B	13,151	0.42	0.23	0.81	0.71	−106.9	−$19.68
CEM – B	13,024	0.42	0.00	0.81	0.80	−106.5	−$24.67
PSM – C	13,151	0.42	0.23	0.81	0.76	−106.9	−$19.93
PSM – D	13,151	0.42	0.23	0.81	3.02	−130.9	−$34.25

^*All values based on the average of 250 simple random samples taken with replacement, with the treatment group sampled (simple random with replacement) at a rate equal to the original comparison group (analyzed comparison group sampled up to 1.5 times the original size). Members were between the ages of 18 and 64.9 years, evidenced to have a chronic condition of coronary artery disease, congestive heart failure, chronic obstructive pulmonary disease, and/or diabetes (based on administrative claims data), and excluded if medical costs in baseline or program year exceeded the 99^th percentile, or if their change in costs over these 2 periods exceeded the 99^th percentile.

^†Letters denote specific CEM or PSM stratification criteria and are described below (note that all variables not denoted as “04” are derived from the baseline period of 2005).

A=DURATION_SINCE_CHRONIC_DISEASE_INDICATION_DUMMY, AGE40_DUMMY, GENDER, IP_DUMMY, ED99_DUMMY, PLAN_STATE_DUMMY, PLAN_TYPE

B=BASE_MM, AGE, GENDER, IP_STAYS, IP_STAYS04, ED_VISITS, OP_VISITS, PHY_VISITS, PLAN_STATE_DUMMY, PLAN_TYPE, NDC_COUNT, WTAVGSEV, CPT_RANK, ICD_COUNT, ICD_COUNT04, SQRT_CLAIMS04, BASE_PMPM, EXCEED_AVG_COST, CRG, ASTHMA_IND, CAD_IND, CHF_IND, DM_IND (note that propensity score not included beyond use in creating the match)

C=B & propensity score as an explanatory variable

D=B & propensity score as the weighting instrument

^‡Corresponding Percent Concordant statistic (c-statistic) values for the propensity score generating model within the PSM-based Matching Methods, by letter, are as follows (note that all values are based on the aforementioned random sampling plan):

A: c–statistic=0.76

B: c–statistic=0.79

C: c–statistic=0.79 (identical propensity score generating model as B)

D: c–statistic=0.79 (identical propensity score generating model as B)

^**Pre-L1 metric computed based on the stratification criteria listed as opposed to searching across the relevant parameter space to define the matching variables, using Scott's Binning Algorithm to determine the cut points (eg, age separated at cut points 25, 35, 45, and 55) and then choosing the median L1 value from these specifications.

^***Log likelihood values reported in thousands and based on use of generalized estimating equations (GEE) regression model (normal distribution, identify link).

^†Estimated Causal Effect is the difference in per chronic member per month (PCMPM) medical costs between the 2 study years and treatment and comparison group members (referred to as the difference-in-difference value). Negative values imply the comparison group trend in medical expenditures exceeded the treatment group trend and thus reflect gross savings. For CEM and PSM, the Estimated Causal Effect is the coefficient for GROUP_TYPE from the GEE regression model (normal distribution, identify link). Note that for each result listed above, the set of evaluated explanatory variables was comprised of the following: GROUP_TYPE, BASE_MM, AGE, GENDER, IP_STAYS, IP_STAYS04, ED_VISITS, OP_VISITS, PHY_VISITS, PLAN_STATE_DUMMY, PLAN_TYPE, NDC_COUNT, WTAVGSEV, CPT_RANK, ICD_COUNT, ICD_COUNT04, SQRT_CLAIMS04, BASE_PMPM, EXCEED_AVG_COST, CRG, MONTHS_AFTER_TRIGGER_DUMMY, ASTHMA_IND, CAD_IND, CHF_IND, DM_IND [COPD_IND was the reference variable].