Abstract
Background.
While researchers use patient expenditures in claims data to estimate insurance benefit features, little evidence exists to indicate whether the resulting measures are accurate.
Objective.
To develop and test an algorithm for deriving copayment and coinsurance values from behavioral health claims data.
Subjects.
Employer-sponsored insurance plans from 2011-2013 for a national managed behavioral health organization (MBHO).
Measures.
12 benefit features, distinguishing between carve-in and carve-out, in-network (INN) and out-of-network (OON), inpatient and outpatient, and copayment and coinsurance, were created. Measures drew from claims (claims-derived measures), and benefit feature data from a claims processing engine database (true measures).
Study Design.
We calculate sensitivity and specificity of the claims-derived measures’ ability to accurately determine if a benefit feature was required and for plan-years requiring the benefit feature, the accuracy of the claims-derived measures. Accuracy rates using the minimum, 25th, 50th, 75th, and maximum claims value for a plan-year were compared.
Principal Findings.
Sensitivity (82% or higher for all but three benefit features) and specificity (95% or higher for all but two benefit features) were relatively high. Accuracy rates were highest using the 75th or maximum claims value, depending on the benefit feature, and ranged from 69% to 99% for all benefit features except for OON inpatient coinsurance.
Conclusions.
For most plan-years, claims-derived measures correctly identify required specialty mental health copayments and coinsurance, although the claims-derived measures’ accuracy varies across benefit design features. This information should be considered when creating claims-derived benefit features to use for policy analysis.
Keywords: employee health benefit plans, claims analysis, cost-sharing insurance
1. INTRODUCTION
Direct observation of private insurance benefits (e.g., copayments and coinsurance) allows health service researchers insight into an important access mechanism. Study of insurance benefits over time can reveal trends in insurance generosity1, study of benefits linked to health utilization and expenditure information can facilitate examination of causal relationships between benefits and access2, and study of insurance benefits linked to health quality measures provides insight into how insurance affects enrollees’ health outcomes3. Despite the importance of health insurance benefit design to policy-relevant research, reliable benefit design data can be difficult to obtain. When insurance companies do make these data available, it is often in the form of narrative “Description of Benefits” documents with open-ended text fields which are labor-intensive to transform into an analyzable database.
While the practice of using more readily available claims data to estimate various forms of patient cost-sharing is fairly common,4-11 the literature is bereft of articles that validate the estimated cost-sharing levels with a gold-standard data source to determine whether the resulting measures truly reflect the plans’ benefit design. Of the reviewed literature, only one study, by Chernew et al, reported results from a validation process12. This study compared relative cost-sharing measures for prescription drugs derived from claims to information from “Description of Benefits” data for a sub-set of sample plans. This study emphasizes the importance of the validity of claims-based measures, but it leaves unanswered two important questions: Can the same levels of accuracy be achieved for 1. Exact cost-sharing values (rather than relative measures) and 2. Claims for other services (i.e. not just prescription drugs)?
Indeed, much of the algorithms available in the literature report relative cost sharing measures from claims (i.e. how costly benefits are for a particular plan relative to other plans), either as a generosity index based on average patient cost-sharing for an employer/plan/quarter or employer/plan/year1,12-17 or by taking the ratio of patient out-of-pocket expenditures to total out-of-pocket expenditures18. The vast majority of the literature deriving cost-sharing measures from claims data as a proxy for gold-standard benefit design data do so for prescription drugs4,7. Methods described in the literature use claims data to generate measures of cost-sharing for a drug or drug class12-20. Two of these studies also report using cost-sharing measures for outpatient and inpatient medical services but as these methods are not the focus of the study, the description they provide is not sufficiently detailed to replicate14,19.
The present study is designed to fill gaps in the literature by addressing the following question for specialty mental health services (e.g. individual psychotherapy, medication management, family psychotherapy, group psychotherapy, rehabilitation homes, inpatient services, etc.): How closely do exact benefit features derived from claims correspond to those from a “gold standard” (for brevity, hereafter referred to as the “true” value) source of benefit feature information? This study makes two specific contributions: First, it provides researchers with recommended algorithms for deriving estimated copayment and coinsurance values from inpatient and outpatient specialty mental health claims data. Second, it validates the resulting claims-derived measures by determining their sensitivity, specificity, and accuracy.
2. METHODS
2.1. Data
This study leverages a unique dataset for behavioral healthcare plans from the behavioral health division of Optum® (hereafter called “Optum”), which is a large national managed behavioral health organization (MBHO) and fully owned subsidiary of UnitedHealth Group. It links plan information from two sources of Optum data.
The first source was benefit design information drawn directly from Optum’s benefit design database used for processing claims for payment. This source provided the exact copayment and coinsurance values reported by the plan in the “Description of Benefits” documents, but in an analyzable format.
The second source of data, specialty mental health claims records, provided information on patient out-of-pocket expenditures, separated by expenditures for copayments, expenditures for coinsurance, and expenditures for deductibles. The claims data also included information on the patient and provider, setting (e.g. inpatient vs. outpatient), date(s) of service, number of units of service (e.g., number of visits), diagnosis and procedure codes, and a flag distinguishing between in- and out-of-network services. Information from the “Book of Business” (BOB) file, which is maintained by Optum as a record of contracts, indicated the plan’s “carve-in” or “carve-out” (i.e. separate administration of behavioral and medical coverage) status.
Data spanned 2011-2013, a period after the Mental Health Parity Addiction and Equity Act of 2008 was fully implemented among employer sponsored insurance plans. During this period, specialty behavioral health benefits should be at parity with medical/surgical benefits, thus improving the external validity of this work.
2.2. Starting sample
All analyses are conducted at the plan-year level. Plan-years in the starting sample appeared in the true measure data source in 2011 through 2013 (n=6187 carve-in plan-years and 9553 carve-out plan-years), had enrollees (n=6177 carve-in plan-years and 9425 carve-out plan-years) and were not retiree or supplemental plans (n=6160 carve-in plan-years and 9241 carve-out plan-years).
2.3. Service-specific study samples
From the starting sample, we created separate study samples of plan-years with claims for four types of services: in-network (INN) inpatient, INN outpatient, out-of-network (OON) inpatient and OON outpatient. The INN inpatient sample required plan-years to have INN claims in inpatient settings and to have no use of tiered benefits (i.e. higher cost sharing for some visits than for others) (carve-in: 2466 plan-years; carve-out: 1717). The OON inpatient sample required plan-years to have OON inpatient claims in an inpatient setting, offer OON coverage, and have no use of tiered benefits (carve-in: 763 plan-years; carve-out: 477 plan-years).
The INN outpatient sample required plan-years to have INN claims for four common outpatient procedures (individual psychotherapy, family psychotherapy, medication management, diagnosis and evaluation) in an outpatient setting, have at least one claim reflecting care for a single unit of service, and have no use of tiered benefits (carve-in: 4455 plan-years; carve-out: 4761 plan-years). The OON outpatient sample required OON outpatient claims in an outpatient setting, have at least one claim reflecting care for a single unit of service, and offer OON coverage (carve-in: 2684 plan-years; carve-out: 2875 plan-years) (Web Appendix Table 1a and 1b).
2.4. Benefit variables derived from claims
Six benefit feature variables (hereafter referred to “claims-derived measures”) were created separately for carve-in and carve-out plans (for a total of 12 measures). We derived INN inpatient and outpatient copayment values, respectively, from the claims within the INN inpatient and outpatient samples (which were constructed based on INN inpatient and INN outpatient service use). Copayments were rarely used for OON services, so OON copayments were not derived from claims. The claims-derived copayment values drew from a claims variable indicating the amount of total patient out-of-pocket expenditures that were due to copayments.
Four claims-derived coinsurance measures were created from the relevant samples of claims: INN inpatient, INN outpatient, OON inpatient and OON outpatient. The claims-derived coinsurance measures drew from a claims variable indicating how much patients paid for their care in satisfying their required coinsurance (in dollars). To convert that dollar amount to a coinsurance percent, we divided it by the dollars allowed by the plan for the given service.
For both copayment and coinsurance, within a single plan and year, not all claims for a given unit of the same service listed the same amount. For example, for INN IP copayment, the within plan-year standard deviation is $56.5. To account for this variation within plan-years, five derived measures were assigned for each benefit feature, alternatively using the minimum, 25th percentile, 50th percentile, 75th percentile, and maximum copayment value from the plan-year distribution. As seen in Web Appendix 2a, descriptive statistics of all claims-derived measures vary based on which of the five measures is used, particularly with respect to the percent of plan-years requiring the benefit feature (Column 2 of Web Appendix 2a).
2.5. Benefit variables from the “true” value data source
For each of the 12 claims-derived measures described above, an analogous measure was taken from the “true” value data source. Separately for carve-in and carve-out samples, we used INN inpatient and outpatient copayment, INN inpatient and outpatient coinsurance, and OON inpatient and outpatient coinsurance, for a total of 12 measures. .One value was assigned to each plan-year. Descriptive statistics of these measures are in Web Appendix 2b.
2.6. Main analysis
We first examined whether the claims-derived measure agreed with the true measure on whether the plan-year required the benefit feature or not. This gave us the sensitivity (i.e. how often required benefit features were detected by claims-derived measures) and specificity (i.e. how often the absence of non-required benefit features was detected by claims-derived measures). The claims-derived measures were designated as “No benefit feature required” or ”Benefit feature required” when all of the claims for a given plan-year indicated the absence of the benefit feature, or when at least one claim for a given plan-year indicated the presence of the benefit feature, respectively.
Among plan-years where the claims-derived and true measures agreed that the plan-year did require a benefit feature, we looked at the proportion of plan-years where the claims-derived measure and the true measure agree on copayment and within 1 percentage point of the coinsurance value (i.e. the “accuracy rate”). This comparison was made using the minimum, 25th percentile, 50th percentile, 75th percentile, and maximum claims-derived values for a given plan-year. Among plan-years where the claims-derived values did not equal the true value, we reported the average overestimate and average underestimate.
To supplement this descriptive analysis of our claims-derived measures’ validity, we followed a procedure recommended in the literature for comparing new measures to gold-standard measures21. We estimated coefficients from random effects models where the claims-derived measure was the outcome and the true measure was the predictor, and plan-years were clustered at the employer level. Coefficients closer to one indicated higher levels of agreement between the two sources.
2.7. Sensitivity analyses
First, we examined sensitivity and specificity using alternative criteria for designating when the claims-derived measures indicated “No benefit feature required” (at least one plan-year indicated the absence of the benefit feature and all claims indicate the presence of the benefit feature). Second, we recalculated accuracy rates to see if “inaccurate” claims-derived coinsurance values were within 5 percentage points of the true value.
Third, we examined accuracy rates among plan-years where the claims-derived measure indicated that the benefit feature was required, regardless of what the true measure indicated, to more closely reflect the process that would be undertaken in the absence of true measures. Fourth, we loosened the sample inclusion/exclusion criteria from which we derive OON benefit features, requiring only that the plan-year had at least one claims observation for OON services. Fifth, we examined sensitivity, specificity, and accuracy rates among plan-years using only claims that did not have a positive deductible paid amount (either because there was no deductible to fulfill or because the patient had already met their deductible), to avoid basing claims-derived measures on copayment and coinsurances biased by deductible payments.
3. RESULTS
3.1. Sensitivity and specificity of all benefit features (Table 1)
Table 1.
Sensitivity and specificity of claims-derived measures in determining when the benefit feature is required or is not required
| True measure = not required |
True measure = required |
Specificity TN/(TN+FP) |
Sensitivity TP/(FN+TP) |
|||
|---|---|---|---|---|---|---|
| Claims-derived measure† = | Claims-derived measure† = | |||||
| Not required (TN) |
Required (FP) |
Not required (FN) |
Required (TP) |
|||
| Copayment, INN | ||||||
| Inpatient | % | % | ||||
| Carve-in | 1916 (78%) | 96 (4%) | 11 (0.5%) | 443 (18%) | 95 | 97 |
| Carve-out | 1330 (77%) | 1 (0.1%) | 16 (1%) | 371 (22%) | 99.9 | 96 |
| Outpatient | ||||||
| Carve-in | 2400 (54%) | 539 (12%) | 30 (1%) | 1486 (33%) | 82 | 98 |
| Carve-out | 2860 (60%) | 14 (0.3%) | 6 (0.1%) | 1881 (40%) | 99 | 99 |
| Coinsurance, INN | ||||||
| Inpatient | ||||||
| Carve-in | 221 (9%) | 20 (1%) | 150 (6%) | 2074 (84%) | 92 | 93 |
| Carve-out | 733 (43%) | 13 (1%) | 133 (8%) | 838 (49%) | 98 | 86 |
| Outpatient | ||||||
| Carve-in | 1157 (26%) | 839 (19%) | 235 (5%) | 2218 (50%) | 58 | 90 |
| Carve-out | 1901 (40%) | 8 (0.1%) | 593 (12%) | 2259 (47%) | 99 | 79 |
| Coinsurance, OON | ||||||
| Inpatient | ||||||
| Carve-in | 5 (1%) | 0 (0%) | 135 (18%) | 623 (82%) | 100 | 82 |
| Carve-out | 12 (3%) | 0 (0%) | 150 (32%) | 315 (68%) | 100 | 68 |
| Outpatient | ||||||
| Carve-in | 13 (0.5%) | 0 (0%) | 579 (22%) | 2092 (78%) | 100 | 78 |
| Carve-out | 2 (0.1%) | 0 (0%) | 830 (29%) | 2043 (71%) | 100 | 71 |
TN: True negative; FP: False positive; FN: False negative; TP: True positive; INN: In-network; OON: Out-of-network
The claims-derived measure was designated as “No benefit feature required” when all claims observations indicated the absence of the benefit feature. The claims-derived measure was designated as “Benefit feature required” when at least one claims observation indicated the presence of the benefit feature.
Table 1 reports sensitivity and specificity of the claims-derived measures compared to the true measures. The claims-derived measures of copayment have high sensitivity (inpatient: carve-in: 97%, carve-out: 96%; outpatient: carve-in: 98%, carve-out: 99%) and specificity (inpatient: carve-in: 95%, carve-out: 99.9%; outpatient: carve-in: 82%, carve-out: 99%).
Among the plan-years requiring INN coinsurance (based on the true measures), the claims-derived measures generally have high sensitivity (inpatient: carve-in: 93%, carve-out: 86%; outpatient carve-in: 90%). Sensitivity is lower for claims-derived INN outpatient coinsurance measure for carve-out plan-years, as only 79% of the plan-years correctly identify when the benefit feature is required. Although few plan-years do not require INN coinsurance, among these plan-years, the claims-derived measures of INN coinsurance has high specificity (inpatient: carve-in: 92%, carve-out: 98%; outpatient carve-in: 58%, carve-out: 99%).
The claims-derived measures of OON coinsurance claims-derived measures have lower sensitivity (inpatient carve-ins: 82%, carve-outs: 78%; outpatient carve-ins: 78%, carve-outs: 71%). All of the claims-derived measures of OON coinsurance have 100% specificity, although very few plan-years do not require OON coinsurance.
3.2. Accuracy rates for INN inpatient and outpatient copayment (Table 2)
Table 2.
Comparison of true† versus claims-derived †† measures when both sources indicate that in-network copayment is required
| Carve-in plan-years | Carve-out plan-years | |||
|---|---|---|---|---|
| % Total |
Among plans in row: Mean difference (SD) |
% Total |
Among plans in row: Mean difference (SD) |
|
| In-Network Inpatient Copayments | N=443 | N=371 | ||
| Plan-year value derived from the minimum value observed in claims | ||||
| Derived < True | 57 | −$257 (128) | 38 | −$250 (124) |
| Derived = True | 40 | $0 (0) | 57 | $0 (0) |
| Derived > True | 3 | $513 (941) | 5 | $51 (224) |
| Plan-year value derived from the 25th percentile value observed in claims | ||||
| Derived < True | 37 | −$226 (128) | 27 | $−223 (145) |
| Derived = True | 57 | $0 (0) | 67 | $0 (0) |
| Derived > True | 5 | $481 (747) | 6 | $41 (199) |
| Plan-year value derived from the 50th percentile value observed in claims | ||||
| Derived < True | 16 | −$155 (96) | 11 | −$154 (96) |
| Derived = True | 77 | $0 (0) | 79 | $0 (0) |
| Derived > True | 7 | $473 (690) | 10 | $35 (166) |
| Plan-year value derived from the 75th percentile value observed in claims | ||||
| Derived < True | 3 | −$152 (74) | 1 | −$104 (95) |
| Derived = True | 90 | $0 (0) | 88 | $0 (0) |
| Derived > True | 7 | $512 (680) | 11 | $56 (186) |
| Plan-year value derived from the maximum value observed in claims | ||||
| Derived < True | 1 | −$140 (71) | 0.8 | −$40 (15) |
| Derived = True | 79 | $0 (0) | 87 | $0 (0) |
| Derived > True | 20 | $437 (491) | 12 | $109 (234) |
| In-Network Outpatient Professional Copayments | N=1,486 | N=1,881 | ||
| Plan-year value derived from the minimum value observed in claims | ||||
| Derived < true | 79 | −$25 (8) | 31 | −$19 (6) |
| Derived = true | 21 | $0 (0) | 69 | $0 (0) |
| Derived > true | 0.3 | $20 (7) | 0 | $0 (0) |
| Plan-year value derived from the 25th percentile value observed in claims | ||||
| Derived < true | 9 | −$22 (10) | 2 | −$19 (6) |
| Derived = true | 90 | $0 (0) | 98 | $0 (0) |
| Derived > true | 1 | $16 (7) | 0.05 | $10 (0) |
| Plan-year value derived from the 50th percentile value observed in claims | ||||
| Derived < true | 4 | −$22 (11) | 0.5 | −$15 (7) |
| Derived = true | 94 | $0 (0) | 99 | $0 (0) |
| Derived > true | 2 | $13 (6) | 0.2 | $7 (3) |
| Plan-year value derived from the 75th percentile value observed in claims | ||||
| Derived < true | 2 | −$22 (10) | 0.05 | −$10 (0) |
| Derived = true | 94 | $0 (0) | 99.6 | $0 (0) |
| Derived > true | 4 | $14 (6) | 0.3 | $9 (2) |
| Plan-year value derived from the maximum value observed in claims | ||||
| Derived < true | 0.2 | −$7 (7) | 0 | NA |
| Derived = true | 53 | $0 (0) | 98 | $0 (0) |
| Derived > true | 47 | $37 (24) | 2 | $7 (3) |
SD: Standard deviation; INN: In-network
True measures are drawn from an insurer’s claims processing engine database, and reflect the values present in the “Description of Benefits” documents issued by the insurer.
Claims-derived measures are based on expenditures from copayments in specialty mental health claims data.
Among plan-years where both sources agreed that INN copayment is required, using the 75th percentile value results in the highest accuracy rates for inpatient (carve-in: 90%; carve-out: 88%) and outpatient (carve-in: 94%, carve-out: 99%) copayment (Table 2). The regression coefficients reported in Web Appendix 3a and 3b are closest to one (i.e. strongest agreement between the claims-derived and true measures) when the claims-derived measure of INN inpatient and outpatient copayment use the 75th percentile value.
For carve-in inpatient copayments, when the claims-derived and true values are not equal, the mean underestimate is -$152 and the mean overestimate is $512. The mean differences are smaller for inpatient copayment in carve-out plan-years and for outpatient copayment in both carve-ins and carve-outs.
3.3. Accuracy rates for INN inpatient and outpatient coinsurance (Table 3)
Table 3.
Comparison of true† versus claims-derived†† measures when both sources indicated that in-network inpatient coinsurance was required
| Carve-in plan-years | Carve-out plan-years | |||
|---|---|---|---|---|
| % Total |
Among plans in row: Mean difference (SD) |
% Total |
Among plans in row: Mean difference (SD) |
|
| In-Network Inpatient Coinsurance | N=2074 | N=838 | ||
| Plan-year value derived from the minimum value observed in claims | ||||
| Derived < true | 82 | −15(7) | 66 | −14 (7) |
| Derived = true | 17 | 0(0) | 34 | 0 (0) |
| Derived > true | 1 | 18(22) | 0 | NA |
| Plan-year value derived from the 25th percentile value observed in claims | ||||
| Derived < true | 76 | −12(7) | 64 | −14 (7) |
| Derived = true | 23 | 0(0) | 36 | 0 (0) |
| Derived > true | 1 | 18(22) | 0 | NA |
| Plan-year value derived from the 50th percentile value observed in claims | ||||
| Derived < true | 57 | −9(6) | 55 | −10 (6) |
| Derived = true | 42 | 0(0) | 45 | 0 (0) |
| Derived > true | 1 | 14(17) | 0 | NA |
| Plan-year value derived from the 75th percentile value observed in claims | ||||
| Derived < true | 29 | −7(5) | 35 | −8 (5) |
| Derived = true | 69 | 0(0) | 65 | 0 (0) |
| Derived > true | 3 | 17(16) | 0.1 | 14 (0) |
| Plan-year value derived from the maximum value observed in claims | ||||
| Derived < true | 18 | −7(5) | 27 | −8 (5) |
| Derived = true | 64 | 0(0) | 73 | 0 (0) |
| Derived > true | 18 | 22(18) | 0.5 | 18 (9) |
| In-Network Outpatient Professional Coinsurance | N=2218 | N=2259 | ||
| Plan-year value derived from the minimum value observed in claims | ||||
| Derived < true | 94 | −18(5) | 87 | −18 (4) |
| Derived = true | 5 | 0(0) | 13 | 0 (0) |
| Derived > true | 0.4 | 19(7) | 0 | NA |
| Plan-year value derived from the 25th percentile value observed in claims | ||||
| Derived < true | 84 | −17(6) | 73 | −18 (4) |
| Derived = true | 16 | 0(0) | 27 | 0 (0) |
| Derived > true | 1 | 17(7) | 0 | NA |
| Plan-year value derived from the 50th percentile value observed in claims | ||||
| Derived < true | 56 | −17(6) | 43 | −17 (5) |
| Derived = true | 43 | 0(0) | 57 | 0 (0) |
| Derived > true | 1 | 20(10) | 0 | NA |
| Plan-year value derived from the 75th percentile value observed in claims | ||||
| Derived < true | 21 | −15(7) | 16 | −17 (6) |
| Derived = true | 76 | 0(0) | 84 | 0 (0) |
| Derived > true | 3 | 22(16) | 0 | NA |
| Plan-year value derived from the maximum value observed in claims | ||||
| Derived < true | 3 | −6(4) | 2 | −9 (6) |
| Derived = true | 58 | 0(0) | 98 | 0 (0) |
| Derived > true | 39 | 29(21) | 0.1 | 14 (22) |
SD: Standard deviation; INN: In-network
True measures are drawn from an insurer’s claims processing engine database, and reflect the values present in the “Description of Benefits” documents issued by the insurer.
Claims-derived measures are based on expenditures from coinsurance in specialty mental health claims data.
As shown in Table 3, among the carve-in plan-years where both the claims-derived and true measures agree that inpatient coinsurance was required, using the 75th percentile value maximizes the accuracy rates for INN inpatient (69%) and outpatient (76%) coinsurance. However, among the analogous carve-out plan-years, using the maximum value yields the highest accuracy rate for inpatient (73%) and outpatient (98%) coinsurance. The regression coefficients reported in Web Appendix 3a and 3b are consistent with these findings for both carve-in and carve-out plan-years. Table 3 also shows that underestimates of coinsurance values are more common, but they are smaller in magnitude, compared to overestimates.
3.4. Accuracy rates for OON inpatient and outpatient coinsurance (Table 4)
Table 4.
Comparison of true† versus claims-derived†† measures when both sources indicated that out-of-network inpatient coinsurance was required
| Carve-in plan-years | Carve-out plan-years | |||
|---|---|---|---|---|
| % Total |
Among plans on row: Mean Difference (SD) |
% Total |
Among plans on row: Mean Difference (SD) |
|
| Out-of-Network Inpatient Coinsurance | N=623 | N=315 | ||
| Plan-year value derived from the minimum value observed in claims | ||||
| Derived < true | 89 | −28 ( 12) | 90 | −25 (13) |
| Derived = true | 10 | 0 (0) | 10 | 0 (0) |
| Derived > true | 1 | 13 (13) | 0 | NA |
| Plan-year value derived from the 25th percentile value observed in claims | ||||
| Derived < true | 89 | −27 (12) | 90 | −24 (13) |
| Derived = true | 10 | 0 (0) | 10 | 0 (0) |
| Derived > true | 1 | 13 (13) | 0 | NA |
| Plan-year value derived from the 50th percentile value observed in claims | ||||
| Derived < true | 87 | −22 (10) | 88 | −20 (12) |
| Derived = true | 12 | 0 (0) | 12 | 0 (0) |
| Derived > true | 1 | 11 (12) | 0 | NA |
| Plan-year value derived from the 75th percentile value observed in claims | ||||
| Derived < true | 75 | −18 (11) | 79 | −16 (11) |
| Derived = true | 23 | 0 (0) | 21 | 0 (0) |
| Derived > true | 23 | 14 (10) | 0.3 | 18 (0) |
| Plan-year value derived from the maximum value observed in claims | ||||
| Derived < true | 67 | −18 (11) | 72 | −16 (11) |
| Derived = true | 30 | 0 (0) | 28 | 0 (0) |
| Derived > true | 3 | 15 (9) | 0.3 | 18 (0) |
| Out-of-Network Outpatient Professional Coinsurance | N=2092 | N=2043 | ||
| Plan-year value derived from the minimum value observed in claims | ||||
| Derived < true | 97 | −37 (8) | 85 | −36 (9) |
| Derived = true | 3 | 0 (0) | 16 | 0 (0) |
| Derived > true | 0.2 | 32 (33) | 0.05 | 5 (0) |
| Plan-year value derived from the 25th percentile value observed in claims | ||||
| Derived < true | 82 | −36 (10) | 70 | −35 (11) |
| Derived = true | 18 | 0 (0) | 30 | 0 (0) |
| Derived > true | 0.4 | 24 (25) | 0.05 | 5 (0) |
| Plan-year value derived from the 50th percentile value observed in claims | ||||
| Derived < true | 48 | −35 (12) | 38 | −34 (12) |
| Derived = true | 51 | 0 (0) | 62 | 0 (0) |
| Derived > true | 1 | 20 (17) | 0.05 | 5 (0) |
| Plan-year value derived from the 75th percentile value observed in claims | ||||
| Derived < true | 24 | −33 (13) | 16 | −32 (13) |
| Derived = true | 75 | 0 (0) | 84 | 0 (0) |
| Derived > true | 2 | 18 (14) | 0.2 | 4 (2) |
| Plan-year value derived from the maximum value observed in claims | ||||
| Derived < true | 5 | −16 (10) | 3 | −19 (10) |
| Derived = true | 87 | 0 (0) | 96 | 0 (0) |
| Derived > true | 9 | 21 (14) | 0.8 | 12 (16) |
SD: Standard deviation; OON: Out-of-network
True measures are drawn from an insurer’s claims processing engine database, and reflect the values present in the “Description of Benefits” documents issued by the insurer.
Claims-derived measures are based on expenditures from coinsurance in specialty mental health claims data.
Assigning the claims-derived measure the maximum claims value for a plan-year yielded the highest accuracy rate for OON inpatient and outpatient coinsurance values for both carve-ins and carve-outs. This is supported by the regression coefficients reported in Web Appendix 3a and 3b, where the claims-derived measure using the maximum values have coefficients closest to one in all of the samples. The accuracy rate is high for OON outpatient coinsurance (carve-in: 88%; carve-out: 96%) but low for OON inpatient coinsurance (carve-in: 30%; carve-out: 28%). The average magnitudes are substantial for both underestimates and overestimates.
3.5. Sensitivity analyses
Using a stricter definition to determine when the claims-derived measure indicate a required benefit feature yields substantially higher specificity for INN outpatient coinsurance in carve-in plan-years (main analysis: 58% vs. sensitivity analysis: 99%), but only marginally higher or no change in specificity for other benefit features (Web Appendix 4). The trade-off for somewhat higher specificity is substantially lower sensitivity, with decreases ranging from between 23 and 84 percentage points (Web Appendix 4).
Allowing claims-derived coinsurance values to be within 5 rather than 1 percentage point of the true values increases accuracy rates substantially for INN inpatient coinsurance (84% vs. 69% for carve-ins and 84% vs. 73% for carve-outs) and for OON inpatient coinsurance (carve-in: 39% vs 30% and carve-out: 46% vs 28%) (Web Appendix 5a-5d).
Using all of the plan-years for which the claims-derived measure indicates a required benefit feature resulted in the same recommendations regarding which claims observation to assign as the value for the claims-derived measure (e.g. the 75th percentile or maximum), with the same resulting accuracy rates for nearly all benefit features (Web Appendix 6a-6d).
As with the main analysis, assigning the maximum value to the claims-derived measure yields the highest accuracy rate among plan-years with at least one OON claims observation (inpatient: 887 carve-ins and 505 carve-outs; outpatient: 3404 carve-ins and 2963 carve-outs) (Data not in a table). For all benefit features, except for OON outpatient coinsurance, the accuracy rates from this sensitivity analysis are very close to the main analysis (Web Appendix 7a & 7b).
After excluding claims with positive patient deductible expenditure amounts, the sample sizes shrink by between 2% and 8%, for most benefit features. Among this no-deductible sample, for copayment, sensitivity and specificity are nearly identical to the main results and for coinsurance, specificity is nearly identical to the main results (Web Appendix 8). However, the sensitivity rates for coinsurance increases for some services (INN outpatient in carve-ins and carve-outs and OON coinsurance in carve-ins) and decreases for others (OON inpatient for carve-ins and carve-outs). As in the main analysis, the highest accuracy rates were achieved using the 75th percentile or maximum values, and for INN coinsurance and OON coinsurance, the accuracy rates were around 10 percentage points higher (than the main analysis) (Web Appendix 8a-8f).
4. DISCUSSION
This paper develops and validates algorithms for using claims data to determine specialty mental health copayment and coinsurance in a national sample of employer sponsored insurance carve-in and carve-out plans. Described in greater detail in the Methods section, we recommend three steps for creating claims-derived measures of copayment and coinsurance: Step 1: Restrict plan-years in the claims database to those with claims with the relevant service. Step 2: Identify which plan-years require a particular benefit feature. Step 3: Among plan-years that require the benefit feature, assign the claims-derived copayment or coinsurance measure a value using either the 75th percentile or maximum value from the claims, depending on the benefit feature of interest.
In the main analysis, the claims-derived copayment measures achieved relatively high specificity and sensitivity rates. The claims-derived INN coinsurance measures also achieved relatively high specificity and sensitivity, with the exception of INN outpatient coinsurance among carve-ins. The sensitivity rates for claims-derived measures of OON inpatient and outpatient coinsurance were low (compared to copayment and INN coinsurance), with the lowest being 68%.
Claims-derived copayment measures had close to 90% accuracy in our sample. However, the accuracy rates for claims-derived coinsurance measures were not as uniformly high. In particular, OON inpatient coinsurance had an accuracy rate of around 30% for both carve-ins and carve-outs. The rates were somewhat higher when we allowed a 5 rather than 1 percentage point difference to count towards the accuracy rate, as well as in the no-deductible sample. However, even then, the accuracy rates were under 50%. Comparatively smaller samples in the OON samples may result in lower precision estimates.
Generalizability may be limited for two reasons. First, our results may not generalize to all private insurance plans, although we did use claims from a large managed behavioral health organization that offered both carve-in and carve-out plans and contracted with employers in all 50 states during our study period. Second, because we used specialty mental health claims observations, we cannot test our algorithm for medical/surgical health benefit features. By using plan-years subject to national behavioral health parity legislation, however, specialty mental health benefits should be at parity with benefits for medical/surgical services 22.
An additional limitation to this work is that a threshold for “how accurate is accurate enough” for the claims-derived measure does not exist. Statistical inference provides some frequently used thresholds that can be applied, and as noted in the results text and Web Appendix tables, our descriptive findings are supported by the corresponding statistical analyses.
Finally, although this study provides information validating copayment and coinsurance measures derived from claims data, it is not possible to estimate additional components of plan benefit design from the available claims data (e.g. deductibles, utilization review, provider networks, coverage of services, etc.). In particular, both behavioral health and medical claims are needed to derive deductibles from claims data, and we only have behavioral health claims.
Within the health services and health economics literature, there is a body of research using claims as the source of cost-sharing data, even though claims do not directly report the benefit feature values in the same way an “Explanation of Benefits” document issued directly from the insurance company would. One common practice in the literature is to construct a relative measure of cost-sharing rather than absolute values.
However, reporting changes in absolute plan cost-sharing requirements can make studies of trends of cost-sharing over time more meaningful than reporting directional trends15. It may also facilitate dissemination of findings, as exact values may be more intuitive for a broader audience to understand than relative generosity. A particular strength of this study is that we compare measures derived using our algorithms to the true benefit design feature values. This provides critical information on how well the claims-derived measures are able to predict the true values.
CONCLUSION
We found evidence that the claims-derived measures described in this paper are able to correctly distinguish whether most plan-years do or do not require each benefit feature (i.e. sensitivity and specificity), for most benefit features. Additionally, this paper reports that the claims-derived measures have accuracy rates above 69% for all but one benefit feature. This information should be considered and may be referenced by researchers interested in creating similar claims-based benefit design features as they determine whether or not copayments and coinsurance derived by claims provide the desired level of accuracy.
Supplementary Material
Appendix 1a: Sample flow chart for carve-in sample for claims-derived measure creation
Appendix 1b: Sample flow chart for carve-out sample for claims-derived measure creation
Appendix 2a: Descriptive statistics of claims-derived values across sample plan-years
Appendix 2b: Descriptive statistics of true measure values across sample plan-years
Appendix 3a: Regression coefficients from models regressing claims-derived measures on the true values among all plan-years
Appendix 3b: Regression coefficients from models regressing claims-derived measures on the true values among plan-years where both sources indicate that the benefit feature is used
Appendix 4: Sensitivity and specificity of claims-derived measures in determining when the benefit feature is required or is not required using alternative measure of benefit feature required in the claims data
Appendix 5a-5d: Comparison of true versus claims-derived measures when both sources indicated that coinsurance was required (counting coinsurance values within 5 percentage points of the true measure as equal)
Appendix 6a-6d: Comparison of true versus claims-derived measures when claims indicated that benefit was required
Appendix 7a -7b: Comparison of true versus claims-derived measures when both sources indicated that OON coinsurance was required, among plan-years with at least one OON claim
Appendix 8: Sensitivity and specificity of claims-derived measures in determining when the benefit feature is required or is not required, among claims without a positive deductible expenditure
Appendix 8a- 8f: Comparison of true versus claims-derived measures when both sources indicated that INN inpatient copayment was required, among claims without a positive deductible expenditure
Funding acknowledgements:
This work was supported by a grant from the National Institutes of Mental Health (1R01MH117013-01).
Other acknowledgements:
This work was accepted for presentation at the American Society of Health Economists Annual Conference and at the Academy Health Annual Research Meeting, but were not presented due to Covid-19.
Footnotes
Disclosure of Conflict of Interest:
Dr. Azocar is an employee of Optum - United Health Group and as such she receives compensation in the form of salary and stock. None of the other co-authors have any conflicts of interest to report.
REFERENCES
- 1.Chernew M, Gibson TB, Fendrick AM. Trends in Patient Cost Sharing for Clinical Services Used as Quality Indicators. J Gen Intern Med. 2010;25(3):243–248. doi: 10.1007/s11606-009-1219-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Keeler EB, Manning WG, Wells KB. The demand for episodes of mental health services. J Health Econ. 1988;7(4):369–392. doi: 10.1016/0167-6296(88)90021-5 [DOI] [PubMed] [Google Scholar]
- 3.Gibson TB, Jing Y, Kim E, et al. Cost-sharing effects on adherence and persistence for second-generation antipsychotics in commercially insured patients. Manag Care Langhorne Pa. 2010;19(8):40–47. [PubMed] [Google Scholar]
- 4.Agiro A, Xie Y, Bowman K, DeVries A. Leveraging benefit design for better diabetes self-management and A1C control. Am J Manag Care. 2018;24(2):e30–e36. [PubMed] [Google Scholar]
- 5.Buxbaum JD, Chernew ME, Bonafede M, et al. Cost sharing and branded antidepressant initiation among patients treated with generics. Am J Manag Care. 2018;24(4):180–186. [PubMed] [Google Scholar]
- 6.Dor A, Lage MJ, Tarrants ML, Castelli-Haley J. Cost sharing, benefit design, and adherence: the case of multiple sclerosis. Adv Health Econ Health Serv Res. 2010;22:175–193. [DOI] [PubMed] [Google Scholar]
- 7.Galbraith AA, Fung V, Li L, et al. Impact of Copayment Changes on Children’s Albuterol Inhaler Use and Costs after the Clean Air Act Chlorofluorocarbon Ban. Health Serv Res. 2018;53(1):156–174. doi: 10.1111/1475-6773.12615 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Huskamp HA, Keating NL, Dalton JB, Chernew ME, Newhouse JP. Drug plan design incentives among Medicare prescription drug plans. Am J Manag Care. 2014;20(7):562. [PubMed] [Google Scholar]
- 9.Joyce NR, Fishman J, Green S, Labiner DM, Wild I, Grabowski DC. Cost sharing for antiepileptic drugs: medication utilization and health plan costs. Am J Manag Care. 2018;24(6):e183–e189. [PubMed] [Google Scholar]
- 10.McClellan C, Fingar KR, Ali MM, Olesiuk WJ, Mutter R, Gibson TB. Price elasticity of demand for buprenorphine/naloxone prescriptions. J Subst Abuse Treat. 2019;106:4–11. doi: 10.1016/j.jsat.2019.08.001 [DOI] [PubMed] [Google Scholar]
- 11.Scoggins JF, Weinberg DA. Healthcare Coinsurance Elasticity Coefficient Estimation Using Monthly Cross-sectional, Time-series Claims Data. Health Econ. 2017;26(6):795–801. doi: 10.1002/hec.3341 [DOI] [PubMed] [Google Scholar]
- 12.Chernew M, Gibson TB, Yu-Isenberg K, Sokol MC, Rosen AB, Fendrick AM. Effects of Increased Patient Cost Sharing on Socioeconomic Disparities in Health Care. J Gen Intern Med. 2008;23(8):1131–1136. doi: 10.1007/s11606-008-0614-0 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Gatwood J, Gibson TB, Chernew ME, Farr AM, Vogtmann E, Fendrick AM. Price Elasticity and Medication Use: Cost Sharing Across Multiple Clinical Conditions. J Manag Care Pharm. 2014;20(11):1102–1107. doi: 10.18553/jmcp.2014.20.11.1102 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Gibson TB, Mark T, Axelsen K, Baser O, Rublee DA, McGuigan KA. Impact of Statin Copayments on Adherence and Medical Care Utilization and Expenditures. Am J Manag Care. 2006;12(December 2006-Special Issue 12 SP). Accessed July 25, 2017. http://www.ajmc.com/journals/issue/2006/2006-12-vol12-n12sp/dec06-2415psp11-sp19/ [PubMed] [Google Scholar]
- 15.Goldman DP, Joyce GF, Escarce JJ, et al. Pharmacy Benefits and the Use of Drugs by the Chronically Ill. JAMA. 2004;291(19):2344–2350. doi: 10.1001/jama.291.19.2344 [DOI] [PubMed] [Google Scholar]
- 16.Karaca-Mandic P, Jena AB, Joyce GF, Goldman DP. Out-of-Pocket Medication Costs and Use of Medications and Health Care Services Among Children With Asthma. JAMA. 2012;307(12):1284–1291. doi: 10.1001/jama.2012.340 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Karaca-Mandic P, Joyce GF, Goldman DP, Laouri M. Cost sharing, family health care burden, and the use of specialty drugs for rheumatoid arthritis. Health Serv Res. 2010;45(5 Pt 1):1227–1250. doi: 10.1111/j.1475-6773.2010.01117.x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Romley J, Goldman D, Eber M, Dastani H, Kim E, Raparla S. Cost-sharing and initiation of disease-modifying therapy for multiple sclerosis. Am J Manag Care. 2012;18(8):460–464. Accessed July 26, 2017. http://europepmc.org/abstract/med/22928761 [PubMed] [Google Scholar]
- 19.Crown WH, Ernst R. Berndt Onur Baser, et al. Benefit Plan Design and Prescription Drug Utilization Among Asthmatics: Do Patient Copayments Matter? : Forum for Health Economics & Policy. Forum Health Econ Policy. 2004;7(1). doi: 10.2202/1558-9544.1053 [DOI] [PubMed] [Google Scholar]
- 20.Pace LE, Dusetzina SB, Fendrick AM, Keating NL, Dalton VK. The Impact of Out-of-pocket Costs on the Use of Intrauterine Contraception Among Women With Employer-sponsored Insurance. Med Care. 2013;51(11):959–963. doi: 10.1097/MLR.0b013e3182a97b5d [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.St. Laurent R. Evaluating Agreement with a Gold Standard in Method Comparison Studies. Biometrics. 1998;54, No. 2:537–545. https://www.jstor.org/stable/3109761 [PubMed] [Google Scholar]
- 22.Friedman SA, Thalmayer AG, Azocar F, et al. The Mental Health Parity and Addiction Equity Act Evaluation Study: Impact on Mental Health Financial Requirements among Commercial “Carve-In” Plans. Health Serv Res. 2018;53(1):366–388. doi: 10.1111/1475-6773.12614 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Appendix 1a: Sample flow chart for carve-in sample for claims-derived measure creation
Appendix 1b: Sample flow chart for carve-out sample for claims-derived measure creation
Appendix 2a: Descriptive statistics of claims-derived values across sample plan-years
Appendix 2b: Descriptive statistics of true measure values across sample plan-years
Appendix 3a: Regression coefficients from models regressing claims-derived measures on the true values among all plan-years
Appendix 3b: Regression coefficients from models regressing claims-derived measures on the true values among plan-years where both sources indicate that the benefit feature is used
Appendix 4: Sensitivity and specificity of claims-derived measures in determining when the benefit feature is required or is not required using alternative measure of benefit feature required in the claims data
Appendix 5a-5d: Comparison of true versus claims-derived measures when both sources indicated that coinsurance was required (counting coinsurance values within 5 percentage points of the true measure as equal)
Appendix 6a-6d: Comparison of true versus claims-derived measures when claims indicated that benefit was required
Appendix 7a -7b: Comparison of true versus claims-derived measures when both sources indicated that OON coinsurance was required, among plan-years with at least one OON claim
Appendix 8: Sensitivity and specificity of claims-derived measures in determining when the benefit feature is required or is not required, among claims without a positive deductible expenditure
Appendix 8a- 8f: Comparison of true versus claims-derived measures when both sources indicated that INN inpatient copayment was required, among claims without a positive deductible expenditure