Abstract
This study characterizes the accuracy of CMS’ Overutilization Monitoring System for correctly identifying prescription opioid users at risk of opioid use disorder (OUD) or overdose between 2011 and 2014.
To address the growing problem of opioid overuse and abuse in the Medicare population, the Centers for Medicare & Medicaid Services (CMS) launched an Overutilization Monitoring System in 2013, requiring its Part D plan sponsors to identify beneficiaries who are at high risk of opioid-related adverse events based on opioid overutilization criteria and to implement interventions.1 Whether the criteria, which depend on prescription-dispensing data, are accurate as a clinical marker for classifying beneficiaries with opioid use disorder (OUD) or overdose is unknown.
Methods
We used the 5% Medicare sample from 2011 through 2014 to estimate the sensitivity, specificity, and positive predictive value of the CMS opioid overutilization criteria in correctly identifying prescription opioid users at risk of OUD or overdose in three 6-month cycles (ie, January 1–June 30, April 1–September 30, and July 1–December 31) in each calendar year. We studied the performance measures over time, hypothesizing that accuracy might change with increasing efforts to combat the opioid crisis.2 Concordant with the most recent (2017) reporting requirements for Part D plans, we used 3 overlapping 6-month cycles per year.1
In each 6-month cycle, eligible patients were required to have at least 1 prescription opioid filled; be continuously enrolled in Parts A, B, and D; and have no cancer nor be receiving hospice care.1 We identified eligible beneficiaries who met CMS criteria as opioid overutilizers (ie, receiving prescription opioids with a mean daily morphine equivalent dose ≥90 mg and from >3 prescribers and >3 pharmacists or receiving a prescription of opioids with a mean daily morphine equivalent dose of ≥90 mg by >4 prescribers)1 and those who had a diagnosis of OUD or overdose3 in the same 6 months plus the subsequent 12 months to account for delays in OUD diagnoses. Linear regression was used to determine trends over time (SAS version 9.4; SAS Institute Inc). Statistical significance was defined as a 2-sided P<.05. This study was approved by the University of Florida Institutional Review Board with a waiver of informed consent.
Results
We identified between 142 036 and 190 320 eligible beneficiaries prescribed opioids across the 6-month measurement cycles from 2011 through 2014. The proportion of beneficiaries who met CMS overutilization criteria during any 6-month cycle ranged from 0.37% to 0.58%. The proportion who had a diagnosis of OUD or overdose during the 18-month follow-up increased from 3.91% in the first cycle to 7.55% in the last. We observed low sensitivity of the criteria, ranging from 4.96% (95% CI, 4.42%-5.58%) at the beginning of the study period to 2.52% (95% CI, 2.26%-2.81%) at the end (P for trend <.001) and positive predictive values ranged from 35.20% (95% CI, 32.14%-38.38%) to 50.94% (95% CI, 47.00%-54.86%; P for trend <.001). Specificity was greater than 99% in all cycles (Table).
Table. Performance Measures of Opioid Overutilization Criteria for Classifying Medicare Beneficiaries With Opioid Use Disorder or Overdose, 2011-2014.
| 6-Month Cycles by Study Year | Eligible Sample, No.a | Patients Meeting CMS Criteria for Opioid Overutilizers, No. (%)b,c | Patients With OUD or Overdose Diagnosis at 18-mo Follow-up, No. (%)c,d | No. of Patients Followed Up | Performance Measures of CMS Criteria, % (95% CI)e,f | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Diagnosis of OUD or Overdose | No Diagnosis of OUD or Overdose | |||||||||||
| Yes | No | Yes | No | Met CMS Criteria | Did Not Meet CMS Criteria | Met CMS Criteria | Did Not Meet CMS Criteria | Sensitivity | Specificity | Positive Predictive Value | ||
| 2011 | ||||||||||||
| January 1–June 30 | 142 036 | 739 (0.52) | 141 297 | 5559 (3.91) | 136 477 | 276 | 5283 | 463 | 136 014 | 4.96 (4.42-5.58) | 99.66 (99.63-99.69) | 37.35 (33.87-40.96) |
| April 1–September 30 | 162 841 | 929 (0.57) | 161 912 | 7055(4.33) | 155 786 | 327 | 6728 | 602 | 155 184 | 4.64 (4.16-5.16) | 99.61 (99.58-99.64) | 35.20 (32.14-38.38) |
| July 1–December 31 | 163 809 | 944 (0.58) | 162 865 | 7460 (4.55) | 156 349 | 350 | 7110 | 594 | 155 755 | 4.69 (4.23-5.20) | 99.62 (99.59-99.65) | 37.08 (34.00-40.26) |
| 2012 | ||||||||||||
| January 1–June 30 | 166 522 | 641 (0.38) | 165 881 | 8180 (4.91) | 158 342 | 262 | 7918 | 379 | 157 963 | 3.20 (2.84-3.61) | 99.76 (99.74-99.78) | 40.87 (37.06-44.80) |
| April 1–September 30 | 166 920 | 646 (0.39) | 166 274 | 8477 (5.08) | 158 443 | 256 | 8221 | 390 | 158 053 | 3.02 (2.67-3.41) | 99.75 (99.73-99.78) | 39.63 (35.85-43.53) |
| July 1–December 31 | 167 288 | 612 (0.37) | 166 676 | 8718 (5.21) | 158 570 | 236 | 8482 | 376 | 158 194 | 2.71 (2.38-3.08) | 99.76 (99.74-99.79) | 38.56 (34.71-42.56) |
| 2013 | ||||||||||||
| January 1–June 30 | 190 320 | 787 (0.41) | 189 533 | 9911 (5.21) | 180 409 | 323 | 9588 | 464 | 179 945 | 3.26 (2.92-3.63) | 99.74 (99.72-99.77) | 41.04 (37.59-44.58) |
| April 1–September 30 | 189 752 | 795 (0.42) | 188 957 | 10 310 (5.43) | 179 442 | 313 | 9997 | 482 | 178 960 | 3.04 (2.72-3.39) | 99.73 (99.71-99.75) | 39.37 (35.97-42.87) |
| July 1–December 31 | 189 103 | 735 (0.39) | 188 368 | 10 663 (5.64) | 178 440 | 332 | 10 331 | 403 | 178 037 | 3.11 (2.80-3.47) | 99.77 (99.75-99.80) | 45.17 (41.54-48.85) |
| 2014 | ||||||||||||
| January 1–June 30 | 174 415 | 637 (0.37) | 173 778 | 11 113 (6.37) | 163 302 | 285 | 10 828 | 352 | 162 950 | 2.56 (2.28-2.88) | 99.78 (99.76-99.81) | 44.74 (40.85-48.70) |
| April 1–September 30 | 174 676 | 650 (0.37) | 174 026 | 11 527 (6.60) | 163 149 | 294 | 11 233 | 356 | 162 793 | 2.55 (2.27-2.86) | 99.78 (99.76-99.80) | 45.23 (41.37-49.15) |
| July 1–December 31 | 171 917 | 642 (0.37) | 171 275 | 12 972 (7.55) | 158 945 | 327 | 12 645 | 315 | 158 630 | 2.52 (2.26-2.81) | 99.80 (99.78-99.82) | 50.94 (47.00-54.86) |
Abbreviations: CMS, Centers for Medicare & Medicaid Services; OUD, opioid use disorder.
In each cycle, the eligible sample was required to have continuous enrollment for 18 months, with the first 6 months devoted to the assessment of CMS criteria and the entire 18 months to determine the diagnosis of OUD or overdose diagnosis.
The CMS defined opioid overutilizers as receiving prescription opioids with a mean daily morphine equivalent dose ≥90 mg and from >3 clinician prescribers and >3 pharmacists or receiving a prescription of opioids with a mean daily morphine equivalent dose of ≥90 mg by >4 prescribers.
The denominator is the total eligible sample.
Opioid use disorder or overdose was defined as having at least 1 inpatient or outpatient encounter claim with one of the following International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes: 304.0x, 304.7x, 305.5x, 965.0x, E850.0-E850.2 or ICD-10-CM codes: F11.xx and T40.0X1, T40.0X4, T40.1X1, T40.1X4, T40.2X1, T40.2X4, T40.3X1, T40.3X4, T40.4X1, T40.4X4, T40.601, T40.604, T40.691, T40.694 in any diagnostic position of inpatient or outpatient encounter claims.
Sensitivity is calculated by dividing the number of patients who were diagnosed as having OUD or overdose and met CMS criteria by the total number of patients who were diagnosed as having OUD or overdose. Specificity is calculated by dividing the number of patients who were not diagnosed as having OUD or overdose and did not meet CMS criteria by the total number of patients who were not diagnosed as having OUD or overdose. The positive predictive value is calculated by dividing the number of patients who were diagnosed as having OUD or overdose and met CMS criteria by the total number of those who met CMS criteria.
Test for trend was examined using linear regression. Sensitivity was P<.001; specificity, P=.001; and positive predictive value, P <.001.
Discussion
Although the CMS criteria may target patients at true risk of OUD or overdose, they missed the majority of patients with OUD or overdose and flagged more than half of opioid prescription users as high risk who were not diagnosed as having OUD or overdose. The small positive predictive value is attributable partly to the low OUD prevalence in the Medicare sample.
For identified high-risk beneficiaries, CMS requires its Part D plan sponsors to implement patient-specific utilization review, case management, and dose-dependent safety alerts or reimbursement rejections at the time of opioid dispensing.1,4 The Substance Use-Disorder Prevention That Promotes Opioid Recovery and Treatment (SUPPORT) for Patients and Communities Act, which was signed into law on October 24, 2018, allows CMS to refuse payment for opioids prescribed to beneficiaries who are identified as high risk unless their prescriptions are issued by a designated clinician.5 Based on this study, the CMS criteria’s low sensitivity suggests different approaches are needed to identify high-risk patients and prevent or treat OUD.
The mechanisms explaining the present findings likely include reliance on prescription opioid data and failure to capture illicit opioid use, which is surpassing prescription opioids in their contribution to opioid overdoses.6 Because the CMS criteria are confined to patients who received opioid prescriptions, these results do not extend to patients with OUD or overdose who solely used illicit opioid sources; thus, reported incidence rates and sensitivity results are likely underestimated.
Section Editor: Jody W. Zylke, MD, Deputy Editor.
References
- 1.Centers for Medicare & Medicaid Services Analysis of proposed opioid overutilization criteria modifications in Medicare Part D. https://www.cms.gov/Medicare/Prescription-Drug-Coverage/PrescriptionDrugCovContra/Downloads/Proposed-Opioid-Overutilization-Criteria-Modifications-v-02012017.pdf. Accessed August 29, 2018.
- 2.Franklin G, Sabel J, Jones CM, et al. . A comprehensive approach to address the prescription opioid epidemic in Washington State: milestones and lessons learned. Am J Public Health. 2015;105(3):463-469. doi: 10.2105/AJPH.2014.302367 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Centers for Disease Control and Prevention Prescription drug overdose data & statistics: guide to ICD-9-CM and ICD-10 codes related to poisoning and pain. 2013. https://www.cdc.gov/drugoverdose/pdf/pdo_guide_to_icd-9-cm_and_icd-10_codes-a.pdf. Accessed August 29, 2018.
- 4.Chavez-Valdez AL, Centers for Medicare & Medicaid Services Memorandum to all Part D sponsors, July 7, 2017. Additional guidance on CY 2017 formulary-level cumulative morphine equivalent dose (MED) opioid point-of-sale (POS) edit. https://mopa.memberclicks.net/assets/docs/071517%20opioid%20med%20safety%20edit%20memo.pdf. Accessed September 14, 2018.
- 5.SUPPORT for Patients and Communities Act, HR 6, 115th Cong, 2nd Sess (2018). https://www.congress.gov/bill/115th-congress/house-bill/6/text. Accessed January 9, 2019.
- 6.Rudd RA, Seth P, David F, Scholl L. Increases in drug and opioid-involved overdose deaths—United States, 2010-2015. MMWR Morb Mortal Wkly Rep. 2016;65(50-51):1445-1452. doi: 10.15585/mmwr.mm655051e1 [DOI] [PubMed] [Google Scholar]