Key Points
Question
What factors most strongly predict opioid overdose in a linked statewide administrative data set?
Findings
In this predictive modeling study of 4 statewide Maryland databases with data from 2.2 million individuals, fatal opioid overdose in the next 12 months could be predicted with an area under the curve as high as 0.89. The factors most strongly associated with the baseline year (by odds ratio) included male sex, use of addiction treatment, at least 1 nonfatal overdose, and release from prison.
Meaning
Public health efforts to prioritize lifesaving interventions should consider the relative risk of overdose across different population groups.
This cohort study combines data from 4 statewide databases from clinical and criminal justice sources in Maryland to develop a predictive model for identifying opioid overdose.
Abstract
Importance
Responding to the opioid crisis requires tools to identify individuals at risk of overdose. Given the expansion of illicit opioid deaths, it is essential to consider risk factors across multiple service systems.
Objective
To develop a predictive risk model to identify opioid overdose using linked clinical and criminal justice data.
Design, Setting, and Participants
A cross-sectional sample was created using 2015 data from 4 Maryland databases: all-payer hospital discharges, the prescription drug monitoring program (PDMP), public-sector specialty behavioral treatment, and criminal justice records for property or drug-associated offenses. Maryland adults aged 18 to 80 years with records in any of 4 databases were included, excluding individuals who died in 2015 or had a non-Maryland zip code. Logistic regression models were estimated separately for risk of fatal and nonfatal opioid overdose in 2016. Model performance was assessed using bootstrapping. Data analysis took place from February 2018 to November 2019.
Exposures
Controlled substance prescription fills and hospital, specialty behavioral health, or criminal justice encounters.
Main Outcomes and Measures
Fatal opioid overdose defined by the state medical examiner and 1 or more nonfatal overdoses treated in Maryland hospitals during 2016.
Results
There were 2 294 707 total individuals in the sample, of whom 42.3% were male (n = 970 019) and 53.0% were younger than 50 years (647 083 [28.2%] aged 18-34 years and 568 160 [24.8%] aged 35-49 years). In 2016, 1204 individuals (0.05%) in the sample experienced fatal opioid overdose and 8430 (0.37%) experienced nonfatal opioid overdose. In adjusted analysis, the factors mostly strongly associated with fatal overdose were male sex (odds ratio [OR], 2.40 [95% CI, 2.08-2.76]), diagnosis of opioid use disorder in a hospital (OR, 2.93 [95% CI, 2.17-3.80]), release from prison in 2015 (OR, 4.23 [95% CI, 2.10-7.11]), and receiving opioid addiction treatment with medication (OR, 2.81 [95% CI, 2.20-3.86]). Similar associations were found for nonfatal overdose. The area under the curve for fatal overdose was 0.82 for a model with hospital variables, 0.86 for a model with both PDMP and hospital variables, and 0.89 for a model that further added behavioral health and criminal justice variables. For nonfatal overdose, the area under the curve using all variables was 0.85.
Conclusions and Relevance
In this analysis, fatal and nonfatal opioid overdose could be accurately predicted with linked administrative databases. Hospital encounter data had higher predictive utility than PDMP data. Model performance was meaningfully improved by adding PDMP records. Predictive models using linked databases can be used to target large-scale public health programs.
Introduction
Drug overdose deaths in the US quadrupled between 1999 and 20171 before reversing slightly in 2018.2 Two-thirds of all overdose deaths are linked to opioids (prescription opioids, heroin, and illicit fentanyl).1 Approaches to decrease overdose include safer prescribing, expanding treatment for people with opioid use disorder (OUD), and harm reduction programs (eg, naloxone distribution).3 Each requires a strategy to identify high-need populations, but policy often operates without a comprehensive understanding of relative risks across the community.
Predictive risk models (PRMs) provide 1 method for identifying those at greatest overdose risk. These models have been developed using prescription drug monitoring programs (PDMPs)4 and individual payers (eg, private insurance5 or the Veterans Administration6). Models typically focus on prescription-associated predictors, such as dosage of prescribed opioids and concurrent benzodiazepine use.5,6 Some incorporate claims-derived indicators, such as diagnosed OUD and nonfatal overdose.7,8 Predictive risk models can accurately predict fatal and nonfatal overdose risks for these specific payers or programs, achieving area under the curve (AUC) statistics ranging from 0.75 to 0.90.4,5,6,7,8,9
Despite their utility, current models are limited because they often exclude groups such as people without insurance, patients using specialty behavioral health programs that may be carved out of health plans, and people with criminal justice involvement. Overdose risk has substantially shifted away from people exclusively using prescription opioids to people using illicit opioids.1 To effectively identify high-risk populations, PRMs may need to incorporate data on indicators associated with illicit substance use.
Our objective was to develop a PRM for fatal and nonfatal overdose, using a novel, person-level data set that linked Maryland records from 4 statewide data systems. We hypothesized that best-performing models would use risk factors derived from all of the study databases compared with models with fewer variables.
Methods
Data Sources and Study Cohort
The study cohort spanned multiple programs that might be the focus of a statewide risk reduction effort by the Department of Health. Specifically, it included Maryland residents with 1 or more records in 2015 in any of 4 statewide data sets: PDMP prescriptions; statewide hospital inpatient and emergency department visits; public-sector specialty behavioral health care admissions (ie, programs predominantly serving individuals with Medicaid or no insurance); or arrest, postconviction incarceration in state prison, or a parole/probation case associated with a property or drug offense. The Maryland PDMP collects data on controlled substances on schedules II through V that are dispensed in Maryland by pharmacies and other health care services, across all payers, including the Veteran’s Administration. Data were linked using a probabilistic matching algorithm and deidentified by the Maryland state-designated health information exchange that maintains a data-sharing agreement with the Maryland Department of Health. We restricted the sample to individuals between 18 and 80 years and excluded individuals with out-of-state zip codes. We also excluded decedents recorded by the medical examiner in 2015, but we did not have access to noninvestigated deaths. Additional information is available in the eFigure and eTable 1 in the Supplement. We followed the Strengthening of Reporting in Observational Epidemiology (STROBE) reporting guidelines for cross-sectional studies.10 The study was approved by institutional review boards at the Johns Hopkins Bloomberg School of Public Health and the Maryland Department of Health, which provided a waiver of informed consent for study participants because the study was a retrospective analysis that involved no more than minimal risk.
Opioid-Associated Adverse Outcomes
We focused on 2 primary outcomes occurring in 2016. First was fatal opioid overdoses derived from the Maryland medical examiner datafiles. Second was having 1 or more nonfatal opioid overdose identified within the emergency department or inpatient setting within any Maryland hospital. More information on the derivation of these outcomes is in the eMethods and eTable 2 in the Supplement.
Risk Variables
Variables were derived from each study data set based on their conceptual association with overdose risk, interpretability for clinicians, and established statistical significance in other studies.6,11,12,13 Individuals who did not appear in a database were coded to have 0 values for all variables derived from that database (eg, a patient with no PDMP records was assumed to have never legally received a prescription for a controlled substance in 2015).
Demographic variables included age and sex. The PDMP variables included short-acting opioid analgesics (0, 1-3, ≥4 prescriptions); any long-acting opioid analgesics; buprenorphine indicated for OUD treatment; any benzodiazepine prescription; number of prescribers (0, 1-2, ≥3 prescribers); and total opioid volume, measured as the total quantity of opioids filled by the patient during the year in morphine milligram equivalents (MMEs), which captures the potencies of different opioids as a standard value (none, 1-300 MMEs, or >300 MMEs). Hospital variables included the number of all-cause emergency department visits (0, 1-2, ≥3 visits); any all-cause inpatient admission; and visits associated with a primary diagnosis for OUD, nonopioid substance use disorder, nonfatal overdose (in 2015), and nonoverdose injury. Behavioral health service variables included mental health treatment, as well as substance use disorder treatment defined as either non-OUD substance use treatment, OUD treatment that included medications, or OUD treatment that did not include medications. Criminal justice variables included having any arrest, an arrest for misdemeanor drug charges or felony drug charges, a parole/probation case, and being released from prison in 2015 after a term specific to drug and property crimes. Further details on variable construction are in the eMethods and eTables 3 and 4 in the Supplement.
Statistical Analysis
We trained separate logistic regression models for fatal and nonfatal overdose that sequentially added variables from the 4 databases. Our basic model included only demographic variables. To the basic model, either PDMP-associated variables or hospital-associated variables were added. We next trained a model that included both hospital and PDMP variables. To that model, we added variables from behavioral health. Our final model added criminal justice data. The AUCs were derived from bootstrapping analyses of 300 iterations; bootstrapping is beneficial because it allowed us to construct 95% CIs and is also robust to specification error.6,11,12 For our final model, we examined the sensitivity, specificity, positive predictive value, and negative predictive value of risk scores at different thresholds of risk.14
Data analysis took place from February 2018 to November 2019 using SAS version 9.4 (SAS Institute). Significance was set at a threshold of P < .05.
Results
Descriptive Analysis
The final sample size was 2 294 707 individuals. This was approximately 51.5% of the Maryland population age 18 to 80 years in 2015.15 There were 1839 fatal opioid overdoses in 2016, of which 1204 (65.5%) could be linked to our sample. In addition, 8430 individuals in the sample had 1 or more nonfatal opioid overdose treated in an emergency or inpatient setting in Maryland.
In the sample, 42.3% of individuals were male (n = 970 019), and 28.2% were aged 18 to 35 years (n = 647 083), 24.8% were aged 36 to 50 years (n = 568 160), 29.3% were aged 50 to 64 years (n = 671 204), and 17.8% were aged 65 to 80 years (n = 408 260) (Table 1). Individuals with criminal justice records were more likely than the full sample to be men (81.1% [(n = 20 011], compared with 42.3% across the full sample) and in younger age groups (eg, 63.3% of individuals [n = 15 633] with criminal justice records were aged 18-34 years compared with 28.2% of those in the full sample; conversely, 0.53% of people with criminal justice records [n = 132] were aged 65-80 years, compared with 17.8% of the full sample), while those with hospital records were more likely to be older (19.5% were aged 65-80 years [n = 286 636] vs 17.8% of the full sample).
Table 1. Characteristics of the Study Cohorts Within Each Maryland Databasea.
| Data source | Individuals, % | ||||
|---|---|---|---|---|---|
| Hospital | PDMP | Behavioral health | Criminal justice | Any | |
| Sample size, No. (%) | 1 466 750 (63.9) | 1 529 895 (66.7) | 170 752 (7.4) | 24 686 (1.1) | 2 294 707 (100.0) |
| Demographics in 2015 | |||||
| Male | 606 754 (41.4) | 622 726 (40.7) | 75 189 (44.0) | 20 011 (81.1) | 970 019 (42.3) |
| Age, y | |||||
| 18-34 | 424 684 (29.0) | 375 505 (24.6) | 73 190 (42.9) | 15 633 (63.3) | 647 083 (28.2) |
| 35-49 | 345 245 (23.5) | 394 173 (25.8) | 51 557 (30.2) | 6306 (25.5) | 568 160 (24.8) |
| 50-64 | 410 185 (28.0) | 486 414 (31.8) | 43 113 (25.3) | 2615 (10.6) | 671 204 (29.3) |
| 65-80 | 286 636 (19.5) | 273 803 (17.9) | 2892 (1.7) | 132 (0.5) | 408 260 (17.8) |
| All-payer hospital data in 2015 | |||||
| No. of emergency department visit | |||||
| 0 | 598 247 (40.8) | 1 066 469 (69.7) | 76 538 (44.8) | 15 788 (64.0) | 1 426 204 (62.2) |
| 1-2 | 734 326 (50.1) | 367 148 (24.0) | 58 279 (34.1) | 5938 (24.1) | 734 326 (32.0) |
| ≥3 | 134 177 (9.2) | 96 278 (6.3) | 35 935 (21.1) | 2960 (12.0) | 134 177 (5.9) |
| Any inpatient admission | 289 925 (19.8) | 183 854 (12.0) | 35 758 (20.9) | 2195 (8.9) | 289 925 (12.6) |
| Any other substance use disorder visit | 82 491 (5.6) | 48 024 (3.1) | 28 932 (16.9) | 2941 (11.9) | 82 528 (3.6) |
| Any opioid use disorder visit | 26 470 (1.8) | 18 823 (1.2) | 16 301 (9.6) | 2032 (8.2) | 26 492 (1.2) |
| Any nonfatal overdose | 5365 (0.4) | 3793 (0.3) | 2393 (1.40) | 456 (1.9) | 5366 (0.2) |
| Any nonoverdose injury | 389 332 (26.5) | 229 995 (15.0) | 44 345 (26.0) | 4928 (20.0) | 389 423 (17.0) |
| Prescription drug monitoring program in 2015 | |||||
| Any long-acting opioid prescription | 46 618 (3.2) | 69 087 (4.5) | 9616 (5.6) | 476 (1.9) | 69 087 (3.0) |
| No. of short-acting opioids | |||||
| 0 | 878 900 (59.9) | 524 033 (34.3) | 100 477 (58.8) | 19 943 (80.8) | 1 288 845 (56.2) |
| 1-3 | 436 471 (29.8) | 786 634 (51.4) | 38 782 (22.7) | 3314 (13.4) | 786 634 (34.3) |
| ≥4 | 151 379 (10.3) | 219 228 (14.3) | 31 493 (18.4) | 1429 (5.8) | 219 228 (9.6) |
| Any buprenorphine prescription for opioid use disorder | 12 303 (0.8) | 25 484 (1.7) | 10 545 (6.2) | 1309 (5.3) | 25 484 (1.1) |
| Any benzodiazepine prescription | 203 565 (13.9) | 397 195 (26.0) | 40 784 (23.9) | 1648 (6.7) | 397 195 (17.3) |
| No. of unique opioid prescribers | |||||
| 0 | 875 350 (59.7) | 515 889 (33.7) | 100 031 (58.6) | 19 927 (80.7) | 1 280 701 (55.8) |
| 1-2 | 463 204 (31.6) | 853 918 (55.8) | 44 036 (25.8) | 3471 (14.1) | 853 918 (37.2) |
| ≥3 | 128 196 (8.7) | 160 088 (10.5) | 26 685 (15.6) | 1288 (5.2) | 160 088 (7.0) |
| No opioid prescriptions filled | 876 669 (59.8) | 518 484 (33.9) | 100 164 (58.7) | 19 943 (80.8) | 1 283 296 (55.9) |
| Quantity, MMEs annually | |||||
| 1-300 | 281 400 (19.2) | 536 931 (35.1) | 26 423 (15.5) | 2333 (9.5) | 536 931 (23.4) |
| >300 | 308 681 (21.1) | 474 480 (31.0) | 44 165 (25.9) | 2410 (9.8) | 474 480 (20.7) |
| Behavioral health services in 2015 | |||||
| Any mental health services | 88 685 (6.1) | 78 839 (5.0) | 126 708 (74.2) | 3419 (13.9) | 126 708 (5.5) |
| Any non–opioid use disorder substance use disorder treatment | 13 814 (0.9) | 9000 (0.6) | 18 806 (11.0) | 1338 (5.4) | 18 806 (0.8) |
| Any opioid use disorder treatment | |||||
| With medication | 17 186 (1.2) | 14 509 (1.0) | 25 780 (15.1) | 2004 (8.1) | 25 780 (1.1) |
| Without medication | 8453 (0.6) | 8089 (0.5) | 11 793 (6.91) | 1524 (6.2) | 11 793 (0.5) |
| Criminal justice data in 2015 | |||||
| Any arrest | 4439 (0.3) | 2628 (0.2) | 2320 (1.4) | 8894 (36.0) | 8894 (0.4) |
| Released from prison | 731 (0.1) | 367 (0.02) | 449 (0.3) | 1193 (4.8) | 1537 (0.1) |
| Any parole/probation | 5605 (0.4) | 3928 (0.3) | 4153 (2.4) | 13 147 (53.3) | 13 147 (0.6) |
| Any drug misdemeanor only | 2890 (0.2) | 1824 (0.1) | 1622 (1.0) | 5671 (23.0) | 5671 (0.3) |
| Any drug felony | 766 (0.1) | 407 (0.03) | 323 (0.2) | 1653 (6.7) | 1653 (0.1) |
| Prospective outcomes in 2016 | |||||
| ≥1 Nonfatal opioid overdose | 7079 (0.5) | 6270 (0.4) | 3391 (2.0) | 635 (2.6) | 8430 (0.4) |
| Fatal opioid overdose | 961 (0.1) | 883 (0.1) | 106 (0.4) | 564 (0.3) | 1204 (0.) |
Abbreviations: MME, morphine milligram equivalent; PDMP, the Prescription Drug Monitoring Program.
Authors’ analysis of 4 linked databases from Maryland in 2015: all-payer admissions to acute nonfederal hospitals from the Health Services Cost Review Commission; the PDMP; public sector specialty behavioral health treatment from Beacon; and arrest, state prison, and parole/probation records from the Department of Public Safety and Correctional Services.
About 63.9% of the sample (n = 1 466 750) had hospital records in 2015, 32.0% (n = 734 326) had 1 to 2 emergency department visits, and 5.9% (n = 134 177) had 3 or more visits. Inpatient visits were experienced by 12.6% of the sample (n = 289 925). Injury visits were experienced by 17.0% of the sample (n = 389 423), while visits with a diagnosis of OUD were experienced by 1.2% (n = 26 492), visits with a diagnosis of nonopioid substance use disorders by 3.6% (n = 82 528), and visits involving a nonfatal overdose by 0.2% (n = 5366). Hospital visits for opioid and other substance use disorders were highest in the subgroups using behavioral health services (opioid use disorder visit, 9.6% [n = 16 301]; any other substance use disorder visit, 16.9% [n = 28 932]) and involved with criminal justice (opioid use disorder visit, 8.2% [n = 2032]; any other substance use disorder visit, 11.9% [n = 2941]; compared with 1.2% and 3.6%, respectively, for the full sample).
Two-thirds of the sample (1 529 895 [66.7%]) had PDMP records in 2015. About 34.3% of the full sample (n = 786 634) had 1 to 3 opioid prescriptions, and 9.6% (n = 219 228) had 4 or more opioid prescriptions. About 3.0% (n = 69 087) had 1 or more prescriptions for long-acting opioids. About 1.1% of the sample (n = 25 484) had buprenorphine for OUD, and 17.3% (n = 397 195) had any benzodiazepine. Overall, 37.2% (n = 853 918) had 1 to 2 opioid prescribers, and 7.0% (n = 160 088) had 3 or more prescribers. Controlled substance prescription fills were highest in the populations using PDMP and behavioral health services. For example, 26.0% of the PDMP sample (n = 397 195) and 23.9% (n = 40 784) of the behavioral health had prescriptions for benzodiazepines vs 17.3% of the full sample.
Individuals with criminal justice encounters (n = 24 686) and specialty behavioral health treatment (n = 170 752) made up relatively small subgroups. Only 1.1% of the overall sample (n = 24 686) had criminal justice involvement, 0.4% (n = 8894) had an arrest in 2015 for a drug or property offense, and 0.6% (n = 13 147) were under parole or probation for these offenses. About 7.4% of the overall sample (n = 170 752) had any behavioral health service use. Mental health treatment in the behavioral health system was received by 5.5% of the overall sample (n = 126 708), specialty OUD treatment with medication was received by 1.1% (n = 25 780), specialty OUD treatment without medication was received by 0.5% (n = 11 793), and 0.8% (n = 18 806) received specialty substance use disorder with medication. Criminal justice indicators were consistently more than 3 times more prevalent for individuals receiving behavioral health services than in the full sample (eg, any arrest: 1.4% [n = 2320] vs 0.4% [n = 8894], respectively) and, conversely, receipt of specialty behavioral health services was higher among those with criminal justice involvement (eg, any mental health service: 13.9% [n = 3419] vs 5.5% [n = 126 708] in the full sample).
Crude Fatal and Nonfatal Overdose Risk
Overall, 1204 individuals (0.05%) in the sample had a fatal overdose in 2016, and 8430 (0.37%) had 1 or more nonfatal overdoses (Table 1). Fatal and nonfatal overdose rates were lowest in the PDMP sample (respectively 883 [0.06%] and 6271 [0.41%]). The fatal overdose rate was highest in the behavioral health sample (106 [0.43%]), and the nonfatal overdose rate was highest in the criminal justice sample (635 [2.6%]).
Odds of Overdose
Odds ratios for models incorporating risk factors from all databases are shown in Table 2. Among demographic variables, male sex was the factor most strongly associated with both fatal overdose (odds ratio [OR], 2.40 [95% CI, 2.08-2.76]) and nonfatal overdose (OR, 1.41 [95% CI, 1.34-1.47]). Individuals aged 35 to 49 years had lower odds of nonfatal overdose than those aged 18 to 34 years (OR, 0.74 [95% CI, 0.70-0.79]), but this was not true for fatal overdose (OR, 1.11 [95% CI, 0.99-1.28]). The only prescription variables that was associated with fatal overdose were receiving any long-acting opioids (OR, 1.42 [95% CI, 1.11-1.73]), buprenorphine prescriptions for OUD (OR, 2.13 [95% CI, 1.71-2.81]), and benzodiazepine prescriptions (OR, 1.64 [95% CI, 1.42-1.85]). Patterns were generally similar for nonfatal overdose, except that having 4 or more short-acting opioids (vs none) was also associated with nonfatal overdose (OR, 1.44 [95% CI, 1.11-1.98]).
Table 2. Odds Ratios Associated With Variables in Models of Future Opioid Overdose Death and Nonfatal Opioid Overdose Events.
| Variable | Odds ratio (95% CI) | |
|---|---|---|
| Opioid overdose death | Nonfatal overdose events | |
| Sex | ||
| Female | 1 [Reference] | 1 [Reference] |
| Male | 2.40 (2.08-2.76) | 1.41 (1.34-1.47) |
| Age group, y | ||
| 18-34 | 1 [Reference] | 1 [Reference] |
| 35-49 | 1.11 (0.99-1.28) | 0.74 (0.70-0.79) |
| 50-64 | 0.94 (0.830-1.11) | 0.89 (0.83-0.94) |
| 65-80 | 0.16 (0.11-0.24) | 0.99 (0.93-1.08) |
| All-payer hospital data | ||
| No. of emergency department visit | ||
| 0 | 1 [Reference] | 1 [Reference] |
| 1-2 | 1.55 (1.32-1.77) | 1.75 (1.66-1.86) |
| ≥3 | 1.35 (1.09-1.70) | 2.48 (2.28-2.66) |
| Any inpatient visit | 1.02 (0.88-1.19) | 1.39 (1.29-1.50) |
| Any other substance use disorder visit | 2.35 (1.86-2.87) | 1.65 (1.50-1.82) |
| Any OUD visit | 2.93 (2.17-3.80) | 2.51 (2.31-2.80) |
| Any nonfatal overdose | 3.04 (2.26-3.94) | 3.89 (3.47-4.45) |
| Any nonoverdose injury | 1.32 (1.14-1.51) | 1.24 (1.17-1.31) |
| Prescription drug monitoring program data | ||
| Any long-acting opioid prescription | 1.42 (1.11-1.73) | 2.23 (2.07-2.40) |
| No. of short-acting opioids | ||
| 0 | 1 [Reference] | 1 [Reference] |
| 1-3 | 1.19 (0.40-8.08) | 0.73 (0.57-0.99) |
| ≥4 | 2.33 (0.95-16.35) | 1.44 (1.11-1.98) |
| Any buprenorphine prescription for OUD | 2.13 (1.71-2.81) | 1.92 (1.72-2.15) |
| Any benzodiazepine prescription | 1.64 (1.42-1.85) | 1.28 (1.22-1.35) |
| No. of unique opioid prescribers | ||
| 0 | 1 [Reference] | 1 [Reference] |
| 1-2 | 0.50 (0.17-0.73) | 1.23 (0.61-2.27) |
| ≥3 | 0.58 (0.19-0.95) | 1.27 (0.60-2.42) |
| Total opioid volume annually | ||
| None | 1 [Reference] | 1 [Reference] |
| 1-300 MMEs | 1.46 (0.69-3.26) | 0.97 (0.60-2.53) |
| >300 MMEs | 1.60 (0.73-3.33) | 1.23 (0.72-3.02) |
| Behavioral health services data | ||
| Any mental health services | 1.43 (1.15-1.76) | 1.38 (1.28-1.47) |
| Any non-OUD substance use disorder treatment | 2.64 (2.06-3.59) | 1.98 (1.72-2.24) |
| Any OUD treatment | ||
| With medication | 2.81 (2.20-3.86) | 4.20 (3.67-4.65) |
| Without medication | 2.44 (1.77-3.56) | 4.21 (3.63-4.80) |
| Criminal justice data | ||
| Any arrest | 1.54 (0.65-2.77) | 1.48 (0.92-2.12) |
| Released from prison | 4.23 (2.10-7.11) | 2.81 (1.87-3.87) |
| Any parole/probation | 2.00 (1.53-2.71) | 2.03 (1.79-2.40) |
| Any drug misdemeanor only | 1.07 (0.50-2.30) | 1.61 (0.99-2.53) |
| Any drug felony | 0.37 (0.07-1.37) | 1.11 (0.60-2.09) |
Abbreviations: MME, morphine milligram equivalents; OUD, opioid use disorder.
All hospital indicators were significantly associated with both fatal and nonfatal overdoses. Compared with no emergency department visits, 1 or 2 visits (OR, 1.55 [95% CI, 1.32-1.77]) and 3 or more visits (OR, 1.35 [95% CI, 1.09-1.70]) were associated with increased fatal overdose risk. The strongest substance use variable associated with fatal overdose was a hospital visit for OUD (OR, 2.93 [95% CI, 2.17-3.80]) and nonfatal overdose in 2016 (OR, 3.04 [95% CI, 2.26-3.94]). Patterns were similar for nonfatal overdose. Having 3 or more visits to the emergency department was strongly associated with nonfatal overdose (OR, 3.89 [95% CI, 3.47-4.45]).
The factor most strongly associated with fatal overdose was release from prison in 2015 (OR, 4.23 [95% CI, 2.10-7.11]). Parole/probation was significantly associated with fatal overdose risk (OR, 2.00 [95% CI, 1.53-2.71]). Arrests for any drug offenses, compared with property offenses, were not significantly associated with increased fatal overdose risk, but having a drug felony was associated with protection relative to property offenses (OR, 0.37 [95% CI, 0.07-1.37]). Similar associations were found for nonfatal overdose risk, except that there was no protective association with drug felonies. Finally, all variables indicating behavioral health system use were associated with significant increases in fatal and nonfatal overdose risk. Among the variables with highest associations with nonfatal overdose were receipt of specialty OUD treatments with medication (OR, 4.20 [95% CI, 3.67-4.65]) and without medication (OR, 4.21 [95% CI, 3.63-4.80]). Further details are in eTables 5 through 8 in the Supplement.
Model Performance
Comparing model performance for fatal overdose in 2016, the model with the lowest AUC was demographics alone (0.69), followed by demographics and PDMP variables (0.79), and demographics and hospital variables (0.82) (Table 3). The model that included demographics, PDMP, and hospital variables had an AUC of 0.86. Adding behavioral health indicators improved fit (0.89); however, there was no substantial improvement when adding criminal justice variables to this model (0.89).
Table 3. C Statistics for Different Models Predicting Fatal and Nonfatal Opioid Overdosesa.
| Models | C statistic (95% CI) |
|---|---|
| Predicting fatal opioid overdose in 2016 | |
| 1. Demographics | 0.692 (0.679-0.710) |
| 2. Demographics + PDMP | 0.789 (0.780-0.804) |
| 3. Demographics + hospital | 0.823 (0.811-0.839) |
| 4. Demographics + hospital + PDMP | 0.864 (0.855-0.875) |
| 5. Demographics + PDMP + hospital + behavioral services | 0.889 (0.880-0.900) |
| 6. Demographics + PDMP + hospital + behavioral services + criminal justice | 0.894 (0.883-0.903) |
| Predicting nonfatal opioid overdoses in 2016 | |
| 1. Demographics | 0.576 (0.570-0.582) |
| 2. Demographics + PDMP | 0.732 (0.726-0.738) |
| 3. Demographics + hospital | 0.769 (0.762-0.775) |
| 4. Demographics + PDMP + hospital | 0.820 (0.815-0.825) |
| 5. Demographics + PDMP + hospital + behavioral services | 0.847 (0.841-0.850) |
| 6. Demographics + PDMP + hospital + behavioral services + criminal justice | 0.851 (0.846-0.855) |
Abbreviation: PDMP, the Prescription Drug Monitoring Program.
All risk information is from 2015. Results are based on bootstrapping (selecting repeated subsamples).
The AUC statistics followed a similar pattern for nonfatal overdose, but improvements in model fit were more substantial when comparing the model with demographics alone (0.58) with models that included either the PDMP (0.73), hospital data (0.77), or both (0.82). Incorporating behavioral health care utilization slightly increased the AUC (0.85), but there was no further substantial improvement by adding criminal justice variables (0.85).
Models for both fatal and nonfatal overdose that included all variables demonstrated high specificity and negative predictive value (both >99% in all cases), but relatively low sensitivity (from 5.6% to 26.6% in fatal overdoses and from 4.7% to 24.7% in nonfatal overdoses) and positive predictive value (from 1.4% to 3.0% in fatal doses and from 9.1% to 17.0% in nonfatal overdoses; Table 4). This indicates that individuals identified as low risk generally did not experience the outcomes, but also that the models would classify many individuals who did not experience the outcome as high risk. Even when applying the most restrictive threshold of 0.1%, the model had a positive predictive value of 3.0% for fatal overdose (ie, only 3.0% of those screened as high risk would experience fatal overdose) and 17.0% for nonfatal overdose. The sensitivity indicates what percentage of all true positive outcomes would be captured at each threshold. At a threshold of 0.1%, the fatal overdose model identifies 5.6% of all cases, and at the same threshold for nonfatal overdose, 4.7% of all cases are identified. Applying less restrictive thresholds results in slight decreases in specificity, but substantial increases in sensitivity: a 1% threshold correctly captures 26.6% of all fatal overdose cases and 24.7% of all nonfatal overdose cases.
Table 4. Sensitivity, Specificity, and Positive and Negative Predictive Values for Different Probability Score Thresholds When Forecasting Death or Nonfatal Opioid Overdose Eventsa.
| Probability | % (95% CI) | |
|---|---|---|
| Overdose death | Nonfatal overdose | |
| Top 0.1% | ||
| Sensitivity | 5.59 (4.22-7.06) | 4.65 (4.18-5.05) |
| Specificity | 99.90 (99.90-99.90) | 99.92 (99.91-99.92) |
| Predictive value | ||
| Positive | 2.96 (2.14-3.66) | 17.02 (15.23-18.65) |
| Negative | 99.95 (99.95-99.95) | 99.65 (99.64-99.66) |
| Top 0.3% | ||
| Sensitivity | 11.69 (9.80-14.65) | 11.08 (10.48-11.70) |
| Specificity | 99.71 (99.70-99.71) | 99.74 (99.74-99.74) |
| Predictive value | ||
| Positive | 2.03 (1.67-2.54) | 13.60 (12.86-14.39) |
| Negative | 99.95 (99.95-99.96) | 99.67 (99.67-99.68) |
| Top 0.5% | ||
| Sensitivity | 16.94 (14.88-19.42) | 15.89 (15.15-16.70) |
| Specificity | 99.51 (99.51-99.51) | 99.56 (99.55-99.56) |
| Predictive value | ||
| Positive | 1.77 (1.52-2.06) | 11.68 (11.07-12.36) |
| Negative | 99.96 (99.95-99.96) | 99.69 (99.68-99.70) |
| Top 1% | ||
| Sensitivity | 26.64 (24.15-29.80) | 24.70 (23.72-25.60) |
| Specificity | 99.01 (99.01-99.02) | 99.09 (99.08-99.09) |
| Predictive value | ||
| Positive | 1.39 (1.25-1.58) | 9.05 (8.63-9.43) |
| Negative | 99.96 (99.96-99.96) | 99.72 (99.71-99.73) |
Estimates are from model 6 (on Table 2), which included demographic, the Prescription Drug Monitoring Program, hospital, behavioral services, and criminal justice variables.
Sensitivity Analyses
We conducted several sensitivity analyses. The AUC statistics substantially decreased when removing (1) measures of opioid dose and (2) variables of high risk that were present for less than 2% of the sample (eg, OUD visits). To examine the implications for a specific payer, we reexamined our main model focused on individuals with opioids reimbursed by Medicaid and found this model could accurately predict both fatal overdose (AUC, 0.85) and nonfatal overdose (AUC, 0.85) in the next year. The main model could successfully predict fatal polydrug overdose (ie, with an opioid and another drug; AUC, 0.89).
Discussion
Using 4 linked administrative databases in Maryland, we created a predictive risk model of fatal and nonfatal opioid-associated overdose risk. In our most comprehensive model, we obtained an AUC of 0.89 for fatal overdose and an AUC of 0.85 for nonfatal overdose. Individuals in the top 1% of the risk distribution accounted for more than one-quarter of all fatal overdoses in the sample. This study builds on prior PRMs by including individuals not necessarily identified in individual payer and prescription records. The PDMP-derived variables improved model fit but were not all statistically significant.
Overall, all-payer hospital data had greater predictive utility than the PDMP. This is likely because hospital records capture a large population with both highly prevalent risk indicators of moderate risk (eg, visits for nonoverdose-associated injuries), as well as acute risk indicators, such as nonfatal overdose. Hospitals have become an increasingly important focus for overdose reduction efforts because of their frequent contact with survivors of overdose and other substance use–associated injuries, such as skin abscesses from injecting drugs.13,16
Specialty behavioral health treatment data modestly improved model fit, but criminal justice records did not. This is likely because individuals who are justice involved made up a small proportion of our overall sample, despite high overdose risk. Within the behavioral health treatment group, the greatest markers of risk were associated with substance use disorder treatment. Although medication treatment is known to reduce overdose risk among people with OUD,17 it was associated with higher risk of overdose in our sample, likely because it is also a proxy for more severe and longer-term disorders.
Despite the relatively high AUC scores, our models had poor positive predictive values. In our highest-performing model, the positive predictive values for fatal overdose when applying a threshold of 0.1% was only 3.0%, and for nonfatal overdose, it was 17.0%. Our model makes broad predictions at a population level, which could be useful when public health agencies are assessing, for example, how to target naloxone programs across communities. However, our models cannot accurately predict levels of risk among people with specific markers, such as people with diagnosed OUD. Our model has inferior sensitivity and specificity to some prior models developed for discrete populations (eg, patients receiving opioid medications),7,9 which may be because of either the greater unmeasured heterogeneity of our study sample or measurement error in key domains. Further, modeling may be improved by considering issues, such as temporal sequence of events (eg, timing of addiction treatment discontinuation or opioid pharmacotherapy relative to death) as have been considered in other large population studies.18,19 Risk prediction may also be improved by examining how known risk factors, such as male sex, interact with other markers of risk.
Even with low positive predictive values, the model has utility for risk stratification. Public health programs are currently in an era of massive scale-up. For example, under the 21st Century Cures Act of 2016, states were called on to spend time-limited grant funds through the State Targeted Response to the Opioid Crisis grants and State Opioid Response Grants for overdose reduction efforts.20 States must determine how to prioritize among groups that could achieve benefit from treatment and harm reduction programs. Going forward, our model can suggest specific populations that can benefit from additional resources, particularly in coordinated responses across systems, such as criminal justice systems and hospitals.
Our study underscores the importance of considering multiple risk factors, particularly the integration of PDMP and hospital data. Integration of PDMP with high-risk flags (eg, recent discharges for nonfatal overdose) has already been implemented in Maryland and is likely to become more widespread through data exchanges. Certain types of information, such as criminal justice involvement and behavioral health treatment receipt, are especially sensitive and may be legally restricted.21 However, adding behavioral health and criminal justice variables to the model did not substantially improve model discrimination, which could indicate that their inclusion is not crucial for predicting overdose risk.
Limitations
While this study has important innovations, including the use of a large, linked patient database, it is subject to several limitations that may either limit generalizability or introduce bias. First, in an observational study using secondary data, risk indicators are not causal. Because our sample represents a group with higher-than-typical risk of overdose, some of the observed associations may not generalize to the statewide population. Second, the database did not include settings that could inform risk prediction, including emergency medical services and outpatient care with private, community physicians. The criminal justice data exclude pretrial detention and individuals who exclusively have been charged with certain offense categories, such as violent offenses. The behavioral health data did not include treatment received in programs that exclusively accept private payment. The data also lack information on access to harm reduction programs, including naloxone distribution. Third, the data are missing information on individual sociodemographics (eg, race/ethnicity, education, and income). Fourth, the study data were collected in 2015 and 2016 in Maryland and may therefore not generalize to the current context or other states. Fifth, although illicit-opioid and prescription-opioid overdoses may have different prevention strategies,18,19 they are often difficult to differentiate with medical examiner records. Our model does not distinguish these 2 types of events.
Conclusions
In this analysis, we have demonstrated that a predictive model for opioid overdose can be created using linked administrative data. These models can optimize program planning and resource allocation decisions. For example, outreach efforts can focus on individuals with multiple markers of risk, such as justice involved people with recent hospital exposure. Pairing effective interventions with objective risk data can steer limited resources toward lifesaving opportunities.
eFigure. Sample Construction Diagram
eMethods. Data Linking Algorithm
eTable 1. Study Data Sources
eTable 2. Specific Study Variables
eTable 3. Drugs in Each Major Category
eTable 4. Overdose Table Detailed Codes
eTable 5. Odds Ratios Associated with Variables in Models Predicting Future Opioid Overdose Death and Non-Fatal Opioid Overdose Events Excluding All Predictors with <2% Prevalence
eTable 6. Odds Ratios Associated with Variables in Models Predicting Future Opioid Overdose Death and Non-Fatal Opioid Overdose Events Excluding Opioid Dosage Variable
eTable 7. Odds Ratios Associated with Variables in Models Predicting Future Opioid Overdose Death and Non-Fatal Opioid Overdose Events – Restricting Sample to Only Patients with >1 Opioid Analgesic Paid for by Medicaid
eTable 8. Odds Ratios Associated with Variables in Models Predicting Polydrug Fatal Overdose (Opioid + At Least One Other Drug)
References
- 1.Hedegaard H, Minino A, Warner M Drug overdose deaths in the United States, 1999–2017. Published 2018. Accessed August 14, 2019. https://www.cdc.gov/nchs/data/databriefs/db329-h.pdf.
- 2.National Center for Health Statistics Provisional Drug Overdose Death Counts. Published 2019. Accessed August 14, 2019. https://www.cdc.gov/nchs/nvss/vsrr/drug-overdose-data.htm
- 3.Saloner B, McGinty EE, Beletsky L, et al. A public health strategy for the opioid crisis. Public Health Rep. 2018;133(1_suppl):24S-34S. doi: 10.1177/0033354918793627 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Geissert P, Hallvik S, Van Otterloo J, et al. High-risk prescribing and opioid overdose: prospects for prescription drug monitoring program-based proactive alerts. Pain. 2018;159(1):150-156. doi: 10.1097/j.pain.0000000000001078 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Liang Y, Goros MW, Turner BJ. Drug overdose: differing risk models for women and men among opioid users with non-cancer pain. Pain Med. 2016;17(12):2268-2279. doi: 10.1093/pm/pnw071 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Bohnert ASB, Logan JE, Ganoczy D, Dowell D. A detailed exploration into the association of prescribed opioid dosage and overdose deaths among patients with chronic pain. Med Care. 2016;54(5):435-441. doi: 10.1097/MLR.0000000000000505 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Glanz JM, Narwaney KJ, Mueller SR, et al. Prediction model for two-year risk of opioid overdose among patients prescribed chronic opioid therapy. J Gen Intern Med. 2018;33(10):1646-1653. doi: 10.1007/s11606-017-4288-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Zedler BK, Saunders WB, Joyce AR, Vick CC, Murrelle EL. Validation of a screening risk index for serious prescription opioid-induced respiratory depression or overdose in a US commercial health plan claims database. Pain Med. 2018;19(1):68-78. doi: 10.1093/pm/pnx009 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Lo-Ciganic W-H, Huang JL, Zhang HH, et al. Evaluation of machine-learning algorithms for predicting opioid overdose risk among Medicare beneficiaries with opioid prescriptions. JAMA Netw Open. 2019;2(3):e190968. doi: 10.1001/jamanetworkopen.2019.0968 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP; STROBE Initiative . The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol. 2008;61(4):344-349. doi: 10.1016/j.jclinepi.2007.11.008 [DOI] [PubMed] [Google Scholar]
- 11.Caudarella A, Dong H, Milloy MJ, Kerr T, Wood E, Hayashi K. Non-fatal overdose as a risk factor for subsequent fatal overdose among people who inject drugs. Drug Alcohol Depend. 2016;162:51-55. doi: 10.1016/j.drugalcdep.2016.02.024 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Chang H-Y, Krawczyk N, Schneider KE, et al. A predictive risk model for nonfatal opioid overdose in a statewide population of buprenorphine patients. Drug Alcohol Depend. 2019;201:127-133. doi: 10.1016/j.drugalcdep.2019.04.016 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Krawczyk N, Eisenberg M, Schneider KE, et al. Predictors of overdose death among high-risk hospital patients with substance-related encounters: a data linkage cohort study. Ann Emerg Med. 2020;75(1):1-12. doi: 10.1016/j.annemergmed.2019.07.014 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Pencina MJ, D’Agostino RB Sr. Evaluating discrimination of risk prediction models: the C statistic. JAMA. 2015;314(10):1063-1064. doi: 10.1001/jama.2015.11082 [DOI] [PubMed] [Google Scholar]
- 15.Maryland Department of Planning U.S. census bureau population estimates by age, race and gender April 1, 2010 to July 1, 2018. Published 2019. Accessed August 14, 2019. https://planning.maryland.gov/MSDC/Pages/pop_estimate/CensPopEst.aspxx
- 16.Weimer M, Morford K, Donroe J. Treatment of opioid use disorder in the acute hospital setting: a critical review of the literature (2014–2019). Curr Addict Rep. 2019;(July). doi: 10.1007/s40429-019-00267-x [DOI] [Google Scholar]
- 17.Committee on Medication-Assisted Treatment for Opioid Use Disorder Medications for Opioid Use Disorder Save Lives. National Academies of Science, Engineering, and Medicine; 2019. [PubMed] [Google Scholar]
- 18.Walley AY, Bernson D, Larochelle MR, Green TC, Young L, Land T. The contribution of prescribed and illicit opioids to fatal overdoses in Massachusetts, 2013-2015. Public Health Rep. 2019;134(6):667-674. doi: 10.1177/0033354919878429 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Nechuta SJ, Tyndall BD, Mukhopadhyay S, McPheeters ML. Sociodemographic factors, prescription history and opioid overdose deaths: a statewide analysis using linked PDMP and mortality data. Drug Alcohol Depend. 2018;190:62-71. doi: 10.1016/j.drugalcdep.2018.05.004 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.National Association of State Alcohol and Drug Abuse Directors NASADAD releases updated STR/SOR funding timeline. Published 2019. Accessed January 6, 2020. https://nasadad.org/2019/04/nasadad-releases-updated-str-sor-funding-timeline/
- 21.McCarty D, Rieckmann T, Baker RL, McConnell KJ. The perceived impact of 42 CFR part 2 on coordination and integration of care: a qualitative analysis. Psychiatr Serv. 2017;68(3):245-249. doi: 10.1176/appi.ps.201600138 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
eFigure. Sample Construction Diagram
eMethods. Data Linking Algorithm
eTable 1. Study Data Sources
eTable 2. Specific Study Variables
eTable 3. Drugs in Each Major Category
eTable 4. Overdose Table Detailed Codes
eTable 5. Odds Ratios Associated with Variables in Models Predicting Future Opioid Overdose Death and Non-Fatal Opioid Overdose Events Excluding All Predictors with <2% Prevalence
eTable 6. Odds Ratios Associated with Variables in Models Predicting Future Opioid Overdose Death and Non-Fatal Opioid Overdose Events Excluding Opioid Dosage Variable
eTable 7. Odds Ratios Associated with Variables in Models Predicting Future Opioid Overdose Death and Non-Fatal Opioid Overdose Events – Restricting Sample to Only Patients with >1 Opioid Analgesic Paid for by Medicaid
eTable 8. Odds Ratios Associated with Variables in Models Predicting Polydrug Fatal Overdose (Opioid + At Least One Other Drug)