Abstract
Background:
There is systemic undercoding of medical comorbidities within administrative claims in the Department of Veterans Affairs (VA). This leads to bias when applying claims-based risk adjustment indices to compare outcomes between VA and non-VA settings. Our objective was to compare the accuracy of a medication-based risk adjustment index (RxRisk-VM) to diagnostic claims-based indices for predicting mortality.
Methods:
We modified the RxRisk-V index (RxRisk-VM) by incorporating VA and Medicare pharmacy and durable medical equipment claims in Veterans dually-enrolled in VA and Medicare in 2012. Using the concordance (C) statistic, we compared its accuracy in predicting 1 and 3-year all-cause mortality to the following models: demographics only, demographics plus prescription count, or demographics plus a diagnostic claims-based risk index (e.g., Charlson, Elixhauser, or Gagne). We also compared models containing demographics, RxRisk-VM, and a claims-based index.
Results:
In our cohort of 271,184 dually-enrolled Veterans (mean age=70.5 years, 96.1% male, 81.7% non-Hispanic white), RxRisk-VM (C=0.773) exhibited greater accuracy in predicting 1-year mortality than demographics only (C=0.716) or prescription counts (C=0.744), but was less accurate than the Charlson (C=0.794), Elixhauser (C=0.80), or Gagne (C=0.810) indices (all P<0.001). Combining RxRisk-VM with claims-based indices enhanced its accuracy over each index alone (all models C≥0.81). Relative model performance was similar for 3-year mortality.
Conclusions:
The RxRisk-VM index exhibited a high level of, but slightly less, accuracy in predicting mortality in comparison to claims-based risk indices.
Implications:
Its application may enhance the accuracy of studies examining VA and non-VA care and enable risk adjustment when diagnostic claims are not available or biased.
Level of Evidence:
Level 3
Keywords: Veterans, Medicare, Risk Adjustment, Drug Utilization
BACKGROUND
The application of risk adjustment indices that use diagnosis codes derived from the International Classification of Diseases (ICD) is standard in health services research.1, 2 However, risk adjustment using ICD codes may introduce bias due to variation in coding practices (i.e., “under” or “over” coding).3 This makes the accurate comparison of health outcomes across healthcare systems challenging. For example, there is systemic undercoding of comorbidities within administrative data derived from the electronic health records (EHR) of the Department of Veterans Affairs (VA). This is due, in part, to the uncoupling of diagnostic codes from reimbursement and because some Veterans only receive VA care for a subset of their conditions.4 More specifically, in Veterans who receive care in both VA and non-VA healthcare settings via Medicare (i.e., dual users), VA administrative data only capture 33% of their comorbidities, whereas Medicare data capture 80%.5, 6 This discrepancy results in an ascertainment bias, potentially skewing studies that compare outcomes between VA and non-VA health systems, as VA beneficiaries appear less sick per ICD claims.7
Risk adjustment using pharmacy records is a potential solution to overcome these measurement biases. Additionally, pharmacy records may be used for risk adjustment when ICD codes are incomplete or unavailable, such as in Medicare Advantage beneficiaries.7 One such medication-based index, RxRisk-V, was originally developed to predict cost of care and was subsequently validated to predict mortality in a cohort of Veterans using VA administrative data.8, 9 While RxRisk-V has been validated for use within VA, its prognostic accuracy has not been assessed using Medicare data.
Nearly 80% of VA beneficiaries also have an alternative form of private or public health insurance, and Veterans’ receipt of non-VA care is anticipated to grow as the VA Mission Act of 2018 seeks to enhance VA’s community care programs. Therefore, it is increasingly necessary to develop a non-biased method of risk adjustment that accurately accounts for Veterans’ comorbidities treated both within and outside VA when comparing the quality of care across VA and non-VA systems.10-14 Thus, we sought to modify the medication-based risk adjustment index, RxRisk-V, for use with combined VA and Medicare data and compare its accuracy to commonly used ICD claims-based risk adjustment methods in predicting 1 and 3-year all-cause mortality.
METHODS
Study Population and Data Sources
We conducted this study in a previously reported national cohort of all Veterans who were dually-enrolled in VA and Medicare Part D, had an active opioid prescription from either VA or Medicare in calendar year 2012, and were alive as of December 31, 2012.15 We excluded patients in Medicare Advantage plans because their medical claims were incomplete.
We linked national patient-level data from VA and the Centers for Medicare and Medicaid Services (CMS) from calendar years (CY) 2012 - 2013 (January 1, 2012 through December 31, 2013). From VA, we used data from the Corporate Data Warehouse (CDW) to capture sociodemographic characteristics and ICD-9 codes documented in face-to-face clinical encounters, using previously described methods.16 We used VA Pharmacy Benefits Management (PBM) data to record dispensed outpatient prescriptions and durable medical equipment (DME). We used the vital status file to determine 1- year (January 1, 2013 – December 31, 2013) and 3-year (January 1, 2013 – December 31, 2015) mortality. From CMS, we used the denominator file to capture enrollment and sociodemographic data. We used the MedPAR, carrier and standard analytic claims files (e.g., outpatient, home health, and hospice) to capture ICD-9 codes documented in face-to-face clinical encounters, and the Part D and Part B DME files for dispensed outpatient prescriptions and DME, respectively.
Modifying the RxRisk-V Index
We updated and modified the RxRisk-V index (RxRisk-VM) in 3 key ways. First, we classified all VA medications and DME dispensed to Veterans that were not previously classified in prior iterations of RxRisk-V.8,9 Three study investigators (TR, JH, and JN) mapped each unclassified medication or DME to 1 of 45 original RxRisk-V categories (Appendix Table A). Medications and DME that did not map to any of the 45 previously established categories were not included in the index. We reconciled all disagreements through a group discussion and consensus process (See Appendix A for a full description).
Second, we mapped all medications dispensed through Medicare Part D, in the same manner that we categorized VA medications and DME. Because DME is contained within VA PBM datasets and incorporated in RxRisk-V, we also incorporated all comparable DME dispensed through Medicare.
Third, as this was the first instance in which Medicare data has been integrated with VA data in the RxRisk-V model, we assigned new weights to each category using previously reported methods. 9 As per these methods, we used logistic regression to assigned weights based upon the magnitude of association (odds ratio) with 1-year all-cause mortality, controlling for age, sex, and race/ethnicity (Appendix Table B). As using the full sample to estimate new weights may over estimate predictive performance due to overfitting, we subsequently applied 10-k fold cross-validation to generate a more robust estimate of predictive performance. We compared these results with the estimates from the main analyses done in the overall cohort.
Outcome Measures and Covariates
Our outcome measures were the 1-year (January 1, 2013 – December 31, 2013) and 3-year (January 1, 2013 – December 31, 2015) all-cause mortality rates for the study cohort.
We created sociodemographic variables for age, sex, and race/ethnicity preferentially using VA data from CY2012, supplemented with CMS data as necessary. We used Medicaid enrollment status as a proxy of socioeconomic status and determined the percentage of Veterans who originally enrolled in Medicare due to disability. We also characterized Veterans as dual users of VA and Medicare health services based upon whether they received inpatient care, outpatient care, or medications from both a VA and a Medicare provider in CY2012.
Risk Adjustment
Using data from CY2012, we calculated the RxRisk-VM score applying both the newly derived and original weights. We also determined prescription counts based upon the number of unique prescriptions captured within VA and Medicare data in CY2012 and calculated scores for the following diagnostic claims-based comorbidity indices: Charlson (Deyo adaptation), Elixhauser (van Walraven adaption), and Gagne. The Deyo adaptation of the Charlson Index assigns empirically-derived weights to 17 conditions, resulting in an overall risk score.17,18 The van Walraven modification of the Elixhauser index combines the 30 standard Elixhauser conditions into a single weighted score.19,20 Due to the redaction of ICD-9 codes for alcohol and substance use disorders in Medicare data, these conditions were excluded from Elixhauser. Lastly, the Gagne index is based on 20 conditions derived from the Charlson or Elixhauser indices.21 This index has demonstrated superior accuracy in predicting 1-year all-cause mortality in comparison to the Charlson and Elixhauser indices.21 Therefore, we used the Gagne index for analyses in which we compared RxRisk-VM to a single index.
Statistical Analyses
We calculated descriptive statistics for sociodemographic variables for the overall cohort and sub-group of dual users of VA and non-VA care. We determined prescription counts, the mean weighted scores for each risk adjustment index, and the 1 and 3-year all-cause mortality rates for the overall cohort and dual use sub-group. We used the kappa statistic to compare the performance of RxRisk-VM and Gagne indices in identifying patients with specific conditions included in both indices.
Using the concordance (C) statistic, we compared the area under the curve (AUC) of RxRisk-VM in predicting 1 and 3-year all-cause mortality to alternative risk adjustment indices. Using VA and Medicare administrative data, we first fit a base model that contained baseline sociodemographics (i.e. age, gender, race/ethnicity, Medicaid enrollment, and disability status). In a series of separate models, we incorporated each of the following to the base model: RxRisk-VM, RxRisk-V incorporating Medicare and VA data using original weights, an unweighted count of unique prescriptions , and the following ICD claims-based comorbidity indices: Charlson (Deyo modification), Elixhauser (van Walraven modification), and Gagne. We compared each model’s AUC using the Chi-squared goodness-of-fit test. To determine if the RxRisk-VM index enhanced the AUC of ICD claims-based indices alone, we also compared the AUCs for models containing baseline demographics, RxRisk-VM and either the Charlson, Elixhauser, or Gagne Indices. We then replicated these models and determined their prognostic accuracy in determining 3-year all-cause mortality in a similar fashion.
To determine if the data source affected the AUC, we conducted a sensitivity analysis in dual VA and Medicare users, comparing the AUC of each model using data from VA alone, Medicare alone, or from both sources. Additionally, to assess the calibration of the RxRisk-VM index in predicting 1-year mortality, we generated calibration curves for both RxRisk-VM and Gagne models. For each calibration curve, we compared the observed versus expected 1-year mortality using the Hosmer-Lemeshow goodness-of-fit test.
3. RESULTS
Baseline Characteristics of the Study Cohort
Of the 271,184 Veterans in the study cohort, the mean age was 70.5 (SD 11.8) years, 96.1% were male, 81.7% were non-Hispanic white, and 35.7% were Medicaid eligible. Of the overall cohort, 172,041 (63.4%) were dual users of VA and Medicare health services (Table 1).
Table 1:
Characteristics and Weighted Comorbidity Index Scores for the Overall Cohort and Dual Users of VA and Medicare Health Services
| Characteristics | Overall Cohort (N = 271,184) |
Dual Users (VA + Medicare)* (N = 172,041) |
|---|---|---|
| Age, mean(SD) | 70.5 (11.8) | 70.0 (11.7) |
| <65 (%) | 29.0 | 30.0 |
| 65-74 (%) | 31.7 | 32.3 |
| 75-84 (%) | 26.5 | 26.1 |
| 85+ (%) | 12.8 | 11.6 |
| Male sex (%) | 96.1 | 96.1 |
| Race/Ethnicity (%) | ||
| Non-Hispanic White | 81.7 | 81.3 |
| Black | 13.1 | 13.5 |
| Hispanic | 2.4 | 2.5 |
| Other/missing | 2.8 | 2.7 |
| Medicaid Eligibility (%) | 35.7 | 36.0 |
| Disability Status†(%) | 40.2 | 42.2 |
| Comorbidity Index Scores, mean(SD) | ||
| RxRisk-VM (fully modified version) | ||
| Combined VA and Medicare pharmacy records‡ | 10.3 (4.8) | 10.7 (4.9) |
| VA pharmacy records only | - | 6.1 (5.6) |
| Medicare pharmacy records only | - | 6.9 (5.1) |
| Number of Unique Prescriptions§ | ||
| Combined VA and Medicare pharmacy records | 12.9 (6.2) | 13.8 (6.3) |
| VA pharmacy records only | - | 7.5 (6.4) |
| Medicare pharmacy records only | - | 7.7 (6.1) |
| RxRisk-V (Original Weights) | ||
| Combined VA and Medicare pharmacy records | 9.4 (5.0) | 9.9 (5.2) |
| VA pharmacy records only | - | 5.9 (5.5) |
| Medicare pharmacy records only | - | 5.9 (4.8) |
| Charlson (Deyo) Index | ||
| Combined VA and Medicare Claims | 3.0 (2.6) | 3.2 (2.7) |
| VA claims only | - | 1.6 (1.8) |
| Medicare claims only | - | 2.7 (2.6) |
| Elixhauser (van Walraven) Index£ | ||
| Combined VA and Medicare Claims | 4.7 (7.3) | 5.0 (7.5) |
| VA claims only | - | 1.4 (4.3) |
| Medicare claims only | - | 4.3 6.9) |
| Gagne Index | ||
| Combined VA and Medicare Claims | 2.4 (3.0) | 2.6 (3.1) |
| VA claims only | - | 0.9 (1.8) |
| Medicare claims only | - | 2.1 (2.9) |
| All-Cause Mortality | ||
| 1 Year (%) | 9.4 | 9.2 |
| 3 Year (%) | 24.3 | 23.8 |
Dual use was defined as any combination of an inpatient visit, outpatient visit or medication from both VA and Medicare during the study year. Among dual users, 49% had both visits and medication from both VA and Medicare.
Disability as the original reason for Medicare Enrollment. Among patients <65 years, 90.9% enrolled due to disability. Among patients of ≥65, 19.5% enrolled due to disability.
Only combined scores are depicted for the overall cohort as not all Veterans received care from both VA and Medicare sources.
Represents unique prescription count in 2012 and not weighted score.
Elixhauser Index excludes alcohol and drug use due to the redaction of related-claims.
Risk Adjustment Scores
For the overall cohort, the mean weighted RxRisk-VM score was 10.3 (SD 4.8). The mean number of unique prescriptions was 12.9 (SD 6.2). The mean weighted scores for RxRisk-V applying the original weights was 9.4 (SD 5.0). Mean weighted scores for the Charlson, Elixhauser, and Gagne indices were, 3.0 (SD 2.6), 4.7 (SD 7.3), and 2.4 (SD 3.0), respectively. There was poor agreement between the RxRisk-VM index and Gagne index for all conditions contained within both measures (Kappa<0.50), with the exception of dementia (Kappa=0.54) (Appendix Table C).
Among dual users, the mean RxRisk-VM score using VA and Medicare data was 10.7 (SD 4.9). Using VA data alone, the mean RxRisk-VM score among dual users was 6.1 (5.6), accounting for 57.0% of the total score. Using Medicare data alone, the mean RxRisk-VM score was 6.9 (SD 5.1), accounting for 64.4% of the total score. Due to overlap in medications prescribed in both VA and Medicare, the total of these percentages is greater than 100. For ICD claims-based comorbidity measures, VA captured fewer medical comorbidities than were captured within Medicare administrative claims (Appendix Table D). For example, among dual users, Gagne comorbidities captured within VA only accounted for 34.6% of the total score (0.9, SD 1.8), whereas comorbidities captured within Medicare accounted for 80.1% of the total score (2.1, SD 2.9) (Table 1).
Prognostic Accuracy of the RxRisk-VM Index
The 1 and 3-year all-cause mortality rates were similar for the overall and dual use cohorts – approximately 9% and 24%, respectively (Table 1). In the overall cohort, the AUC of both the RxRisk-V index using the original weights (C=0.759) and the RxRisk-VM Index (C=0.773) was greater than models using demographics alone (C=0.716) or a prescription count (C=0.744), but lower than models using Charlson (C=0.794), Elixhauser (C=0.800) or Gagne (C=0.810, all P<0.001) (Table 2). AUCs for RxRisk-VM derived using 10-k fold cross validation were nearly identical to those derived in our primary analysis (Appendix Table E).
Table 2:
Area Under the Curve (C-Statistic) of the Modified RxRisk-V Index (RxRisk-VM) in Comparison to and in Combination with Commonly Used Diagnostic Claims-Based Indices
| 1-year mortality (95% CI)* | 3-year mortality (95% CI)* | |
|---|---|---|
| Individual Models | ||
| Baseline Demographics (Demo)† | 0.716 (0.713, 0.719) | 0.729 (0.723, 0.731) |
| Demo + Unique Prescription Count | 0.744 (0.740, 0.747) | 0.753 (0.751, 0.755) |
| Demo + RxRisk-V (original weights) | 0.759 (0.756, 0.762) | 0.767 (0.765, 0.769) |
| Demo + RxRisk-VM (fully modified version) | 0.773 (0.770, 0.776) | 0.779 (0.778, 0.781) |
| Demo + Charlson | 0.794 (0.791, 0.797) | 0.792 (0.790, 0.794) |
| Demo + Elixhauser‡ | 0.800 (0.798, 0.803) | 0.796 (0.793, 0.797) |
| Demo + Gagne | 0.810 (0.807, 0.813) | 0.805 (0.803, 0.807) |
| Combined Models | ||
| Demo + Charlson + RxRisk-VM | 0.805 (0.802, 0.808) | 0.803 (0.801, 0.805) |
| Demo + Elixhauser + RxRisk-VM | 0.812 (0.809, 0.815) | 0.809 (0.807, 0.811) |
| Demo + Gagne + RxRisk-VM | 0.817 (0.814, 0.819) | 0.813 (0.811, 0.815) |
Models for 1-year and 3-year mortality were compared separately. All comparisons were statistically significant when applying the Chi Square Test of the AUC (p<0.001)
Baseline demographic models (demo) includes categorical age (10 categories: <50, 5-year categories between 50-89, and 90+), gender, race/ethnicity, Medicaid enrollment and disability status (original reason for Medicare enrollment).
Elixhauser Index excludes alcohol and drug use due to the redaction of related-claims.
The accuracy in predicting 1-year all-cause mortality for the model containing demographics, RxRisk-VM, and the Gagne Index together was greater (C=0.817, P<0.001) than models containing the RxRisk-VM and other ICD claims-based comorbidity indices (Table 2).
There was a similar pattern of relative model performance for 3-year all-cause mortality; however, the performance of models that contained the RxRisk-VM (C=0.779) and Charlson (C=0.779) scores now approximated each other (Table 2).
Among dual users of VA and Medicare services, models that contained both VA and Medicare data exhibited greater prognostic accuracy than models that contained VA or Medicare data alone, across all the risk adjustment methods. Specifically, the AUC for RxRisk-VM was 0.761 when including both VA and Medicare data, but was 0.716 when using VA data alone and 0.720 when using Medicare data alone (Table 3).
Table 3:
Area Under the Curve (C-Statistic) of the Modified RxRisk-VM Index in Predicting 1-year Mortality among Dual Users when Incorporating VA and/or Medicare Data (N = 172,041)
| VA + CMS data* | VA data only* | CMS data only* | |
|---|---|---|---|
| Individual Models | |||
| Baseline Demographics (Demo)† | 0.694 (0.689, 0.698) | 0.694 (0.968, 0.698) | 0.694 (0.689, 0.698) |
| Demo + Unique Prescription Count | 0.725 (0.721, 0.729) | 0.701 (0.697, 0.705) | 0.710 (0.706, 0.715) |
| Demo + RxRisk-V (original weights) | 0.745 (0.741, 0.749) | 0.713 (0.709, 0.717) | 0.716 (0.712, 0.721) |
| Demo + RxRisk-VM (modified version) | 0.761 (0.757, 0.765) | 0.716 (0.712, 0.720) | 0.720 (0.716, 0.724) |
| Demo + Charlson | 0.785 (0.781, 0.789) | 0.731 (0.727, 0.735) | 0.776 (0.773, 0.780) |
| Demo + Elixhauser‡ | 0.794 (0.791, 0.798) | 0.729 (0.725, 0.733) | 0.782 (0.778, 0.786) |
| Demo + Gagne | 0.805 (0.801, 0.808) | 0.740 (0.736, 0.744) | 0.792 (0.789, 0.796) |
| Combined Models | |||
| Demo + Charlson + RxRisk-VM | 0.797 (0.794, 0.801) | 0.735 (0.731, 0.739) | 0.777 (0.773, 0.781) |
| Demo + Elixhauser + RxRisk-VM | 0.807 (0.803, 0.810) | 0.734 (0.730, 0.738) | 0.783 (0.779, 0.787) |
| Demo + Gagne + RxRisk-VM | 0.811 (0.808, 0.815) | 0.743 (0.739, 0.747) | 0.792 (0.789, 0.796) |
Models using VA + CMS Data, VA data only, and CMS data only were compared separately. All comparisons were statistically significant when applying the Chi Square Test of the AUC (p<0.001)
Baseline demographic models (demo) includes categorical age (10 categories: <50, 5-year categories between 50-89, and 90+), gender, race/ethnicity, Medicaid enrollment and disability status (original reason for Medicare enrollment).
Elixhauser Index excludes alcohol and drug use due to the redaction of related-claims.
The observed versus predicted proportion of 1-year all-cause mortality did not significantly differ for most RxRisk-VM scores, with the predicted mortality falling outside the 95% confidence interval of observed mortality for RxRisk-VM scores exceeding 23. The calibration curve for the RxRisk-VM model was similar to the Gagne Index calibration curve, which also exhibited divergence of the observed and predicted mortality rates at higher scores (Figure 1).
Figure 1: Calibration Curves of the Modified RxRisk-V and Gagne Indices.
For the RxRisk-VM model, the absolute difference between the observed and predicted probabilities are all < 1% except for two groups: 1) RxRisk-VM score=15, 1.5% difference, 2) RxRisk-VM Score >21, −3.4%. For the Gagne Model, the absolute difference between the observed and predicted probabilities are all smaller than 1.6% except −5.4% for Gagne Score ≥11
DISCUSSION
In a national cohort of Veterans dually-enrolled in VA and Medicare Part D, the RxRisk-VM index exhibited a high degree of, but slightly less, accuracy in predicting 1 and 3-year all-cause mortality relative to other commonly used ICD claims-based risk adjustment indices. When incorporated into models that also contained ICD-based indices, RxRisk-VM enhanced the prognostic accuracy over the use of each index alone. Additionally, among dual users, incorporating data from both VA and Medicare sources enhanced prognostic accuracy over the use of data from only one source.
Our findings suggest that RxRisk-VM is a viable alternative to commonly used ICD-based risk adjustment indices for use in VA and Medicare data, especially when diagnostic claims are not available or the use of claims-based indices may lead to an unacceptable degree of bias. Such bias has been shown to be especially problematic in studies that evaluate outcomes between VA and non-VA health systems or the impact of dual VA and non-VA health system use.5-7 While the current standard when conducting such studies is to risk adjust by incorporating ICD claims from more than one healthcare system (i.e., VA and Medicare), potential pitfalls exist in applying this approach. First, ICD claims are often incomplete for Veterans who also receive care through Medicare Advantage plans or private insurers, resulting in their exclusion from studies.7 Second, even when claims are derived from both VA and non-VA sources, such analyses may under-account for important conditions commonly treated within VA, such as serious mental illness and substance use disorders.4,7
In addition to addressing several analytic challenges with ICD-based risk adjustment within VA, risk adjustment using RxRisk-VM or pharmacy records, in general, may become increasingly important given the complex nature of the recently implemented 10th Revision of the ICD (ICD-10). In comparison to ICD-9, ICD-10 contains nearly 5 times as many diagnostic codes (ICD-9 14,000 vs ICD-10 68,000).1,2 This increased complexity may widen the current gap in coding between VA and non-VA healthcare settings, as non-VA settings become more exacting in their coding standards. Despite identifying disagreement in the identification of certain conditions contained both in RxRisk-VM and the Gagne Index, RxRisk-VM was marginally less accurate in comparison to Gagne and other ICD-based comorbidity measures. Our findings are similar to those of Lu et al., who compared RxRisk-V to the Charlson Index in a cohort of Australian Veterans.23 Several other approaches to risk adjustment using pharmacy records have been validated and found to be comparable to ICD claims based indices; however, they have not been widely adopted in clinical or health services research.24-28
Our findings also have important implications for VA patients, researchers, and policy makers. Veterans’ access to non-VA care is expanding through VA Community Care Programs, through which Veterans receive care from a non-VA provider that is paid for by VA.13 Our findings may enable VA to more accurately assess the quality of non-VA care, empowering Veterans to make more informed healthcare choices when choosing to see a provider within or outside VA. Additionally, our findings align with prior work outside VA, which demonstrates that integrating diagnosis data, pharmacy records and other clinical values (e.g. laboratory values and vital signs) enhances the accuracy of models predicting outcomes such as cost of care and hospitalization.29-33 Given the discrepancy in coding between VA and non-VA health systems, the integration of diagnoses derived from the VA EHR via methods such as natural language processing may further enhance the prognostic accuracy of models containing diagnosis codes and pharmacy data derived from administrative claims.
Despite its advantages, medication-based risk adjustment, in general, has potential limitations. Providers exhibit varying propensities to prescribe medications that are influenced by patient (e.g., socioeconomic status, severity of illness), provider (e.g., training, specialty), and health system (e.g. formulary restrictions, decision-support tools) factors. These varying propensities may introduce alternative forms of bias that should be explored in future research. Second, medications with multiple indications for use, such as gabapentin, may introduce measurement error and mischaracterize the prevalence of certain disease states if the medication is used other than designated in the index. Third, as RxRisk-VM and other medication-based indices rely upon pharmacy claims, poor patient adherence may also introduce bias or decrease the measure’s prognostic accuracy.33 These factors likely explain, in part, why RxRisk-VM exhibited slightly lower prognostic accuracy than the ICD claims-based indices included in our study.
Our study also contained some specific limitations. First, our cohort was a national sample of Veterans who had received an opioid prescription in CY2012 from either VA or Medicare. This has the potential to limit the generalizability of our findings. Second, the redaction of diagnosis codes for substance use disorders in the Medicare data may have affected the mean score for each ICD claims-based risk index. Both of these limitations may be investigated in future studies using data derived from the general population and from data without claims redaction. Third, prior studies have demonstrated that varying patterns and degree of dual use confer differential risk of mortality.34 RxRisk-VM and other risk adjustment indices may therefore perform differently in Veterans with varying patterns of dual use.
CONCLUSION
Accurately accounting for the effect of comorbidity on health outcomes is essential but challenging within VA, where there is systematic undercoding of medical comorbidities within administrative claims.4-7 RxRisk-VM exhibited a high degree of, but slightly less accuracy in predicting 1 and 3-year mortality relative to commonly used risk adjustment measures and may enhance the prognostic accuracy of current ICD-based measures when their use is appropriate. As momentum builds to enable Veterans to seek care outside VA, policy makers and healthcare consumers require unbiased information regarding the quality of care available from VA and non-VA sources. The RxRisk-VM index may enhance the accuracy of future risk-adjusted studies examining VA and non-VA care and serve as a viable method of risk adjustment when diagnostic claims are not available or the use of claims-based indices may lead to bias.
Supplementary Material
Appendix Table A: Example Medications Mapped to RxRisk-VM Categories
Appendix Table B: Patients within each RxRisk-VM Category with Corresponding Odds Ratios and Updated Weights Derived from the Model for 1-year All-Cause Mortality
Appendix Table C: Agreement between the Modified RxRisk-V and Gagne Indices in Identifying Conditions contained Within Both Measures among the Study Cohort
Appendix Table D: Percentage of Veterans with a Diagnosis Code for an Elixhauser Conditions depicted by VA and/or Medicare Data
Appendix Table E: Comparison of the AUCs of the Modified RxRisk-VM Index (RxRisk-VM) alone and in Combination with Commonly Used Diagnostic Claims-Based Indices in Primary Analysis and when applying 10K Fold Approach to Train the Models.
Acknowledgments
Funding: This work was supported by a Department of Veterans Affairs Health Services Research and Development Merit Review Award [VA IIR 14-306] (Dr. Gellad); and the National Center for Advancing Translational Sciences of the National Institutes of Health under Award Number (KL2TR001856) (Dr. Radomski). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health or the Department of Veterans Affairs. Support for VA/CMS data is provided by the Department of Veterans Affairs, Veterans Health Administration, Office of Research and Development, Health Services Research and Development, VA Information Resource Center (SDR 02-237 and 98-004).
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
CITATIONS
- 1.International Classification of Disease, Ninth Revision, Available at: https://www.cdc.gov/nchs/icd/icd9.htm. Last accessed: October 16th, 2017.
- 2.International Classification of Disease, Tenth Revision, Clinical Modification (ICD-10-CM). Available at: https://www.cdc.gov/nchs/icd/icd10cm.htm. Last accessed: October 16th, 2017.
- 3.Bastian LA, Brandt CA, Justice AC. Measuring Multimorbidity: A Risky Business. J Gen Intern Med. DOI: 10.007/s11606-017-4110-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Radomski TR, Fine MJ, Gellad WF. Outcomes After Admission at Veterans Affairs vs Non-Veterans Affairs Hospitals. JAMA. 2016; 316(3): 345–6. [DOI] [PubMed] [Google Scholar]
- 5.Rosen AK, Gardner J, Montez M, Loveland S, Hendricks A. Dual-system use: are there implications for risk adjustment and quality assessment? Am J Med Qual. 2005;20(4):182–94. [DOI] [PubMed] [Google Scholar]
- 6.Byrne MM, Kuebeler M, Pietz K, Petersen LA. Effect of using information from only one system for dually eligible health care users. Med Care. 2006;44(8):768–73. [DOI] [PubMed] [Google Scholar]
- 7.Radomski TR, Zhao X, Thorpe CT, Thorpe JM, Naples JG, Mor MK, Good CB, Fine MJ, Gellad WF. The Impact of Medication-Based Risk Adjustment on the Association Between Veteran Health Outcomes and Dual Health System Use. J Gen Intern Med. 2017; 32(9): 967–973 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Sloan KL, Sales AE, Liu CF, Fishman P, Nichol P, Suzuki NT, Sharp ND. Construction and Characteristics of the RxRisk-V: A VA-Adapted Pharmacy Based Case-mix Instrument. Med Care.2003. ;41(6):761–74. [DOI] [PubMed] [Google Scholar]
- 9.Johnson ML, El-Serag HB, Tran TT, Hartman C, Richardson P, Abraham NS, Adapting the Rx-Risk-V for Mortality Prediction in Outpatient Populations. Med Care.2006;44(8):793–97. [DOI] [PubMed] [Google Scholar]
- 10.Hynes DM, Koelling K, Stroupe K, et al. Veterans’ access to and use of Medicare and Veterans Affairs health care. Med Care. 2007;45(3):214–23. [DOI] [PubMed] [Google Scholar]
- 11.U.S. Department of Veterans Affairs. 2011 Survey of Veteran Enrollees’ Health and Reliance Upon VA. Available at: http://www.va.gov/healthpolicyplanning/soe2011/soe2011_report.pdf. Accessed on 5/24/2015.
- 12.Gasper J, Liu H, Kim S, May L. 2015 Survey of Veteran Enrollee’s Health and Use of Health Care. http://www.va.gov/HEALTHPOLICYPLANNING/SoE2015/2015_VHA_SoE_Full_Findings_Report.pdf Accessed: 3/31/2017.
- 13.Gellad WF. The Veterans Choice Act and Dual Health System Use. J. Gen Intern Med 2015. DOI: 10.1007/s11606-015-3492-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Kizer KW. Veterans and the Affordable Care Act. JAMA. 2012;307(8):789–90. [DOI] [PubMed] [Google Scholar]
- 15.Gellad WF, Thorpe JM, Zhao X, Thorpe CT, Sileanu FE, Cashy JP, Hale JA, Mor M, Radomski TR, Hausmann LRM, Donohue JM, Gordon AJ, Suda K, Stroupe K, Hanlon JT, Cunningham FE, Good CB, Fine MJ. Impact of Dual Use of VA and Medicare Part D Drug Benefits on Potentially Unsafe Opioid Use. American Journal of Public Health. 2017. In Press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Pham HH, Schrag D, O’Malley AS, Wu B, Bach PB. Care patterns in Medicare and their implications for pay for performance. N Engl J Med. 2007; 356 (11): 1130–9 [DOI] [PubMed] [Google Scholar]
- 17.Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: Development and validation.1987;40(5):373–383. [DOI] [PubMed] [Google Scholar]
- 18.Deyo RA, Cherkin DC, Ciol MA. Adapting a Clinical Comorbidity Index for Use with ICD-9-CM administrative databases. Journal of Clinical Epidemiology. 1992; 45(6): 613–19 [DOI] [PubMed] [Google Scholar]
- 19.Elixhauser A, Steiner C, Harris RD, Coffey RM. Comorbidity Measures for Use with Administrative Data. Medical Care.1998;36(1):8–27. [DOI] [PubMed] [Google Scholar]
- 20.Van Walraven C, Austin PC, Jennings A, Quan H, Forster AJ. A Modification of the Elixhauser Comorbidity Measures into a Point System for Hosptial Death Using Administrative Data. Med Care. 2009; 47:626–633. [DOI] [PubMed] [Google Scholar]
- 21.Gagne JJ, Glynn RJ, Avorn J, Levin R, Schneeweiss S. A Combined Comorbidity Score Predicted Mortality in Elderly Patients Better than Existing Scores. J Clin Epi. 2011;64: 749–759. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Carey K, Montez-Rath ME, Rosen AK, Christiansen CL, Loveland S, Ettner SL. Use of VA and Medicare services by dually eligible veterans with psychiatric problems. Health Serv Res. 2008;43(4): 1164–83. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Lu CY, Barratt J, Vitry A, Roughead E. Charlson and Rx-Risk Comorbidity Indices were Predictive of Mortality in the Australian Health Care Setting. J. Clin Epidemiol.2011;64:223–28. [DOI] [PubMed] [Google Scholar]
- 24.Cortaredona S, Pambrun E, Verdoux H, Verger P. Comparison of Pharmacy-based and Diagnosis-based Comorbidity Measures from Medical Administrative Data. Pharmacoepidemiology and Drug Safety. 2017; 26:402–411. [DOI] [PubMed] [Google Scholar]
- 25.Von Korff M, Wagner EH, Saunders K. A chronic disease score from automated pharmacy data. J Clin Epidemiol 1992; 45: 197–203. [DOI] [PubMed] [Google Scholar]
- 26.Lamers LM. Pharmacy costs groups: a risk-adjuster for capitation payments based on the use of prescribed drugs. Med Care 1999; 37: 824–830. [DOI] [PubMed] [Google Scholar]
- 27.Dong YH, Chang CH, Shau WY, et al. Development and validation of a pharmacy-based comorbidity measure in a population-based automated healthcare database. Pharmacotherapy 2013; 33: 126–136. [DOI] [PubMed] [Google Scholar]
- 28.The Johns Hopkins ACG System White Paper. Available at: https://www.hopkinsacg.org/document/acg-white-paper/. Last accessed: March 20, 2019
- 29.Kharrazi H, Chi W, Chang H, Richards TM, Gallagher JM, Knudson SM, Weiner JP. Comparing Population-based Risk-stratification Model Performance Using Demograhic, Diagnosis, and Medication Data Extracted From Outpatient Electronic Health Records Versus Administrative Claims. Med Care.2017;55(8):789–796. [DOI] [PubMed] [Google Scholar]
- 30.Kharrazi H, Chang H, Heins SE, Weiner JP, Gudzune KA. Assessing the Impact of Body Mass Index Information on the Performance of Risk Adjustment Models in Predicting Health Care Costs and Utilization. Med Care.2018;56(12):1042–1050. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Kharrazi H, Anzaldi LJ, Hernandez L, Davison A, Boyd CM, Leff B, Kimura J, Weiner JP. The Value of Unstructured Electronic Health Record Data in Geriatric Syndrome Case Indentification. JAGS.2018;66(8):1499–1507. [DOI] [PubMed] [Google Scholar]
- 32.Kan HJ, Kharrazi H, Leff B, Boyd C, Davison A, Chang H, Kimura J, Wu S, Anzaldi L, Richards T, Lasser EC, Winer JP. Defining and Assessing Geriatric Risk Factors and Associated Health Care Utilization Among Older Adults Using Claims and Electronic Health Records. Med Care.2018;56(3):233–239. [DOI] [PubMed] [Google Scholar]
- 33.Chang H, Richards TM, Shermock KM, Dalpoas SE, Kan HJ, Alexander GC, Weiner JP, Kharrazi H. Evaluating the Impact of Prescription Fill Rates on Risk Stratification in Model Performance. Med Care.2017:55(12):1052–1060. [DOI] [PubMed] [Google Scholar]
- 34.Radomski TR, Zhao X, Thorpe CT, Thorpe JM, Good CB, Mor MK, Fine MJ, Gellad WF. VA and Medicare Utilization Among Dually Enrolled Veterans with Type 2 Diabetes: A Latent Class Analysis. J Gen Intern Med.2016;31(5):524–31. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Appendix Table A: Example Medications Mapped to RxRisk-VM Categories
Appendix Table B: Patients within each RxRisk-VM Category with Corresponding Odds Ratios and Updated Weights Derived from the Model for 1-year All-Cause Mortality
Appendix Table C: Agreement between the Modified RxRisk-V and Gagne Indices in Identifying Conditions contained Within Both Measures among the Study Cohort
Appendix Table D: Percentage of Veterans with a Diagnosis Code for an Elixhauser Conditions depicted by VA and/or Medicare Data
Appendix Table E: Comparison of the AUCs of the Modified RxRisk-VM Index (RxRisk-VM) alone and in Combination with Commonly Used Diagnostic Claims-Based Indices in Primary Analysis and when applying 10K Fold Approach to Train the Models.