Abstract
Background
Survivors of opioid overdose have substantially increased mortality risk, although this risk is not evenly distributed across individuals. No study has focused on predicting an individual’s risk of death after a nonfatal opioid overdose.
Objective
To predict risk of death after a nonfatal opioid overdose.
Design and Participants
This retrospective cohort study included 9686 Pennsylvania Medicaid beneficiaries with an emergency department or inpatient claim for nonfatal opioid overdose in 2014–2016. The index date was the first overdose claim during this period.
Exposures, Main Outcome, and Measures
Predictor candidates were measured in the 180 days before the index overdose. Primary outcome was 180-day all-cause mortality. Using a gradient boosting machine model, we classified beneficiaries into six subgroups according to their risk of mortality (< 25th percentile of the risk score, 25th to < 50th, 50th to < 75th, 75th to < 90th, 90th to < 98th, ≥ 98th). We then measured receipt of medication for opioid use disorder (OUD), risk mitigation interventions (e.g., prescriptions for naloxone), and prescription opioids filled in the 180 days after the index overdose, by risk subgroup.
Key Results
Of eligible beneficiaries, 347 (3.6%) died within 180 days after the index overdose. The C-statistic of the mortality prediction model was 0.71. In the highest risk subgroup, the observed 180-day mortality rate was 20.3%, while in the lowest risk subgroup, it was 1.5%. Medication for OUD and risk mitigation interventions after overdose were more commonly seen in lower risk groups, while opioid prescriptions were more likely to be used in higher risk groups (both p trends < .001).
Conclusions
A risk prediction model performed well for classifying mortality risk after a nonfatal opioid overdose. This prediction score can identify high-risk subgroups to target interventions to improve outcomes among overdose survivors.
Supplementary Information
The online version contains supplementary material available at 10.1007/s11606-020-06405-w.
KEY WORDS: nonfatal opioid overdose, mortality, machine learning, prediction, Medicaid
INTRODUCTION
The opioid crisis has been a major contributor to the recent decline in US life expectancy.1 For every fatal opioid overdose, there are approximately 30 nonfatal opioid overdoses.2 There is heightened interest in understanding how nonfatal overdoses impact the immediate and long-term health outcomes of overdose survivors, and in developing targeted interventions to improve outcomes among patients with nonfatal overdose.3–6
The limited amount of existing data suggests that opioid overdose survivors face a substantially increased risk of death3,4; adults who experienced a nonfatal opioid overdose had an approximately 24 times greater risk of 1-year all-cause mortality than in the general population, with a more than 40 times increased risk in younger individuals aged between 18 and 34 years.3
While some recent studies examined risk factors associated with mortality after a nonfatal opioid overdose,5,6 they did not focus on predicting an individual’s risk. Individual risk factors alone may not be strong predictors of the mortality outcome.7 Furthermore, these prior studies used conventional statistical models that have limited ability to handle the complex interactions and nonlinear associations that might be expected between predictors and outcomes.
To address these research gaps, we used data from the Pennsylvania Medicaid program and applied machine learning to predict all-cause mortality after a nonfatal opioid overdose and stratify individuals into subgroups with similar mortality risk according to the prediction score. We applied machine learning approaches because of their success in predicting opioid overdose in prior work.8 Moreover, because of prior data on the minimal decrease in opioid prescription use2,9 and minimal treatment for opioid use disorder10,11 after a nonfatal opioid overdose, we conducted exploratory analyses examining post overdose opioid prescription use and receipt of substance use disorder treatment by subgroup based on predicted mortality risk.
METHODS
Study Design and Participants
We conducted a retrospective cohort study using Pennsylvania Medicaid claims for fee-for-service and managed care beneficiaries.12,13 We identified beneficiaries who had opioid overdoses from January 1, 2014 to December 31, 2016 (n = 18,985) for which they received care in emergency department or inpatient settings. Index date was defined as the date of a patient’s first nonfatal opioid overdose (index overdose) during this period (Fig. 1). We excluded beneficiaries who were younger than 18 years or older than 65 years, had a diagnosis of malignant cancer, received hospice care, were eligible for both Medicaid and Medicare (we cannot capture complete claims), were eligible for only Healthy Horizons or the Select Plan for women in Medicaid (we cannot capture complete medical records for these partial benefit enrollees), or died within 7 days after the index date (we consider deaths proximal to the emergency department or inpatient claim to be overdose-related). As a limitation of the study, we had no access to data on the cause of death and our exclusion of those who died within 7 days of the index date may cause an underestimation of the mortality risk among these overdose survivors.4
Figure 1.
Flow chart of study population.
Because we measured predictor candidates in the 180-day window before the index overdose14 and our primary outcome was mortality during the 180 days after the index overdose, we required beneficiaries to be continuously enrolled in Medicaid 180 days before the index date. In addition, enrollees were included if they were either continuously enrolled until death or the end of 180 days following the index overdose. The final eligible cohort comprised 9686 Medicaid beneficiaries. The University of Pittsburgh Institutional Review Board approved this study.
Definition of Index Opioid Overdose
We identified the occurrence of all opioid overdoses in emergency department or inpatient settings, using International Classification of Diseases, Ninth Revision (ICD-9) and Tenth Revision (ICD-10) codes, as previously described.8,15 Overdoses were defined using either an opioid overdose code for the primary diagnosis or a substance use disorder code for the primary diagnosis and opioid overdose as the non-primary diagnosis.8,15
Predictor Variables
We constructed 348 predictors for inclusion in the model, including demographic characteristics, health status, patterns of opioid and non-opioid prescription use, characteristics of the historical and index opioid overdose events, and regional factors. The selection of these factors was informed by literature and clinical experience.5,6,8 We used Federal Information Processing Standards (FIPS) codes or zip codes to link our Medicaid dataset to several publicly available databases (i.e., County Health Rankings data,16 Area Health Resources Files,17 Area Deprivation Index datasets,18 and Pennsylvania Open Data Portal datasets19) to obtain the regional factors reflecting healthcare resources, health status, and socioeconomic status. We collected the remaining information from the Medicaid claims data.
Patient demographic information included age, sex, race, and ethnicity as well as the reasons for Medicaid eligibility, urbanicity of residence, regions used by Pennsylvania to contract with managed care organizations, and the calendar year of the index nonfatal opioid overdose. Patient health status variables included diagnoses of opioid use disorder and other substance use disorders, other comorbid conditions (e.g., HIV/AIDS, chronic pain, psychiatric disorders), and use of medical services (e.g., number of emergency department visits and inpatient visits). We included information about the index opioid overdose (e.g., heroin overdose, ambulance use, and severity rating for emergency room). We obtained information about prescription opioids and relevant medication use, such as the total and mean daily morphine milligram equivalent, and the cumulative and continuous duration of opioid prescription use.
Study Outcome
The primary outcome was all-cause mortality within 180 days after the index opioid overdose. We examined 90-day all-cause mortality as the secondary outcome. Mortality information was based on date of death recorded in the Medicaid enrollment file.
Measures for Post Hoc Exploratory Analyses
During the post overdose follow-up, we measured prescriptions filled for opioids, the receipt of medication treatment for opioid use disorder (i.e., methadone, buprenorphine, and naltrexone), and the receipt of risk mitigation and certain behavioral health treatment–related interventions (i.e., prescriptions filled for naloxone, counseling for substance use disorders, and claims for urine drug tests). We used current Procedural Terminology codes (H0020, J1230) in claims to capture methadone dispensed for opioid use disorder in an opioid treatment program (as opposed to prescriptions for pain management). We used National Drug Codes in pharmacy claims to identify opioid use disorder–approved buprenorphine and naltrexone.2
Statistical Analysis
The analysis included two main steps: (1) development and validation of a machine learning algorithm for predicting all-cause death; (2) risk stratification of patients into subgroups at similar risk according to the risk scores (i.e., the individual’s estimated probability of death) generated by the validated machine learning algorithm.
We randomly divided the cohort into a training set (two-thirds of the sample, n = 6460) and validation set (one-third of the sample, n = 3226). We used the training set to develop the machine learning model and conduct hyperparameter tuning with cross-validation (optimized hyperparameters are seen in Supplement, Box S1). The validation set was used to evaluate the model’s prediction performance. We used the gradient boosting machine (Python Package: XGBoost)20,21 in our final analyses, because it showed the best performance based on the C-statistic among several machine learning approaches that we evaluated, including random forest, least absolute shrinkage and selection operator–type regression, and elastic net (Supplement Table S1).
In addition to the C-statistic, we computed a series of prediction performance statistics including sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), number needed to evaluate (NNE) to identify one death, and the estimated rate of positive alerts.8 We examined these statistics at the prediction threshold with balanced sensitivity and specificity as determined by the Youden index,22 and at the sensitivity level of 90%. In the validation set, we classified the beneficiaries into six risk subgroups according to the estimated probability of death: < 25th, 25th to < 50th, 50th to < 75th, 75th to < 90th, 90th to < 98th, and ≥ 98th. We calculated the observed death rate across the risk subgroups to evaluate the calibration performance. We chose the cutoff of the 98th percentile rather than the 99th to avoid reporting small cell sizes.23
In the post hoc analyses, we examined proportions of beneficiaries receiving opioid prescriptions before and after the index overdose. We also examined the proportion initiating medications (methadone, buprenorphine, or naltrexone) for opioid use disorder, and proportions receiving risk mitigation and behavior health treatment interventions for substance use disorders after the index overdose, by risk subgroup.
We used SAS 9.4 (SAS Institute Inc.) and Python 3.6 (Python Software Foundation) for the analyses.
RESULTS
Patient Characteristics
Among the 9686 eligible beneficiaries, 47% were female, 82% were white, and 33% were enrolled in Medicaid due to disability. The mean (standard deviation, SD) age was 36.4 (11.2) years. Of these beneficiaries, 347 (3.6%) and 207 (2.1%) died within 180 and 90 days after the index opioid overdose, respectively. Among those who died within 180 and 90 days, the median (25th, 75th) time to death was 71 (29, 131) and 37 (17, 59) days, respectively. The training set and validation set had similar patient characteristics and mortality risk (Table 1).
Table 1.
Sample Characteristics, Overall and by Risk Groups for 180-Day All-Cause Mortality Among Medicaid Beneficiaries Presenting with Nonfatal Overdose
| Participant characteristics | Training | Testing | ||||||
|---|---|---|---|---|---|---|---|---|
| Total | Total | Risk subgroups of 180-day mortality | ||||||
| (N = 6460, 66.7%) | (N = 3226, 33.3%) | < 25th (n = 806) | 25th to < 50th (n = 807) | 50th to < 75th (n = 807) | 75th to < 90th (n = 484) | 90th to < 98th (n = 258) | ≥ 98th (n = 64) | |
| 180-day all-cause mortality, n (%) | 231 (3.6) | 116 (3.6) | 12 (1.5) | 16 (2.0) | 24 (3.0) | 26 (5.4) | 25 (9.7) | 13 (20.3) |
| Age in years, mean (SD) | 36.4 (11.2) | 36.3 (11.2) | 30.4 (8.2) | 33.6 (9.1) | 36.9 (9.7) | 41.4 (11.6) | 48.5 (11.9) | 49.5 (11.6) |
| Female, n (%) | 3049 (47.2) | 1503 (46.6) | 500 (62.0) | 366 (45.4) | 304 (37.7) | 200 (41.3) | 104 (40.3) | 29 (45.3) |
| Race, n (%) | ||||||||
| White | 5278 (81.7) | 2575 (79.8) | 672 (83.4) | 649 (80.4) | 638 (79.1) | 383 (79.1) | 186 (72.1) | 47 (73.4) |
| Black | 827 (12.8) | 453 (14.0) | 93 (11.5) | 105 (13.0) | 116 (14.4) | 72 (14.9) | * (*) | * (*) |
| Others | 355 (5.49) | 198 (6.42) | 39 (4.8) | 50 (6.2) | 50 (6.2) | 29 (6.0) | * (*) | * (*) |
| Hispanic ethnicity, n (%) | 361 (5.6) | 187 (5.8) | 43 (5.3) | 48 (6.0) | 50 (6.2) | 27 (5.6) | * (*) | * (*) |
| Medicaid eligible group, n (%) | ||||||||
| Disabled | 2125 (32.9) | 1111 (34.4) | 222 (27.5) | 211 (26.2) | 263 (32.6) | 209 (43.2) | 162 (62.8) | 44 (68.8) |
| Non-disabled adults | 4335 (67.1) | 2115 (65.6) | 584 (72.5) | 596 (73.8) | 544 (67.4) | 275 (56.8) | 96 (37.2) | 20 (31.3) |
| Newly eligible for Medicaid, n (%) | 3645 (56.4) | 1713 (53.1) | 389 (42.3) | 474 (58.7) | 478 (59.2) | 254 (52.5) | 96 (37.2) | 22 (34.4) |
| Index year, n (%) | ||||||||
| 2014 | 1188 (18.4) | 639 (19.8) | 239 (29.7) | 133 (16.5) | 127 (15.7) | 75 (15.5) | 44 (17.1) | 21 (32.8) |
| 2015 | 1914 (29.6) | 903 (28.0) | 323 (40.1) | 201 (24.9) | 174 (21.6) | 122 (25.2) | 67 (26.0) | 16 (25.0) |
| 2016 | 3358 (52.0) | 1684 (52.2) | 244 (30.3) | 473 (58.6) | 506 (62.7) | 287 (59.3) | 147 (57.0) | 27 (42.2) |
| Metropolitan residence, n (%) | 5942 (92.0) | 2960 (91.8) | 748 (92.8) | 737 (91.2) | 736 (91.9) | 445 (91.9) | * (*) | * (*) |
| Region, n (%) | ||||||||
| Lehigh Capital | 941 (14.6) | 433 (13.4) | 106 (13.2) | 125 (15.5) | 93 (11.5) | 66 (13.6) | 32 (12.4) | 11 (17.2) |
| Northeast | 688 (10.7) | 335 (10.4) | 97 (12.0) | 92 (11.4) | 70 (8.7) | 47 (9.7) | * (*) | * (*) |
| Northwest | 383 (5.9) | 207 (6.4) | 57 (7.1) | 59 (7.3) | 55 (6.8) | 26 (5.4) | * (*) | * (*) |
| Southeast | 2084 (32.3) | 1121 (34.7) | 238 (29.5) | 251 (31.1) | 282 (34.9) | 209 (43.2) | 112 (43.4) | 29 (45.3) |
| Southwest | 2338 (36.2) | 1124 (34.8) | 307 (37.7) | 279 (34.6) | 304 (37.7) | 136 (28.1) | 82 (31.9) | 16 (25.0) |
*Data suppressed due to insufficient cell size
Prediction Performance of the Machine Learning Algorithm
The C-statistic values were 0.71 (95% CI: 0.67, 0.74) for predicting 180-day mortality and 0.71 (95% CI: 0.67, 0.76) for predicting 90-day mortality after a nonfatal opioid overdose (Figure S1). The prediction performance measures are shown in Table 2. Regarding 180-day mortality, using the balanced sensitivity and specificity determined by the Youden index, the model had a sensitivity of 64.7%, specificity of 65.1%, PPV of 6.5%, NPV of 98.0%, and NNE of 16 with 36 positive alerts generated per 100 beneficiaries. If the sensitivity were set at 90%, then the model had a specificity of 25.6%, PPV of 4.3%, NPV of 98.5%, and NNE of 23 with 75 positive alerts generated per 100 beneficiaries. Regarding 90-day mortality at the balanced sensitivity and specificity determined by the Youden index, the model had a sensitivity of 66.7%, specificity of 66.6%, PPV of 4.2%, NPV of 98.9%, NNE of 24, and 34 positive alerts generated per 100 beneficiaries.
Table 2.
Prediction Performance of Gradient Boosting Machine Models for Predicting 180-Day and 90-Day Mortality Among Medicaid Beneficiaries Presenting with Nonfatal Overdose
| 180-day mortality | 90-day mortality | |||
|---|---|---|---|---|
| Performance matrix | At the threshold determined by the Youden index | If sensitivity = 90% | At the threshold determined by the Youden index | If sensitivity = 90% |
| Probability threshold | 3.9% | 2.7% | 1.2% | 0.3% |
| Sensitivity | 64.7% | 90.0% | 66.7% | 90.0% |
| Specificity | 65.1% | 25.6% | 66.6% | 29.0% |
| PPV | 6.5% | 4.3% | 4.2% | 2.7% |
| NPV | 98.0% | 98.5% | 98.9% | 99.2% |
| NNE | 16 | 23 | 24 | 37 |
| Positive alert per 100 | 36 | 75 | 34 | 71 |
NNE number needed to be evaluated, NPV negative predictive value, PPV positive predictive value
Risk Stratification Using Prediction Score
Figure 2 shows the actual mortality rates of individuals in each of the six risk subgroups in the validation set. The mortality rate observed in the highest risk group (i.e., the top 2% of the prediction score) was 20.3% for 180-day mortality, whereas in the lowest risk group (i.e., the bottom 25% of the prediction score), the mortality rate was 1.5%. The patients at higher risk of death were more likely to be older and disabled, have the index overdose in 2016 rather than in 2014 or 2015, and reside in southeast Pennsylvania (Table 1 and Table S2). The calibration curve for the gradient boosting machine model is shown in Figure S4 in the Supplement.
Figure 2.
Calibration performance of gradient boosting machine for predicting 180-day and 90-day mortality, by risk subgroup. Asterisk symbols indicate data suppressed due to insufficient cell size.
Exploratory Analyses
Figure 3 shows the proportion of beneficiaries who filled opioid prescriptions before and after the index opioid overdose in each of the six risk subgroups. The proportion filling opioid prescriptions was higher in those with a higher predicted risk of death at 180 days, both before (the range across six risk subgroups: 24.8–73.4%) and after (24.4–67.2%) the index overdose. Overall, the likelihood of filling opioid prescriptions was lower after the index overdose than before the index overdose.
Figure 3.
Proportion of beneficiaries with opioid prescription fills before and after index opioid overdose, by risk subgroup. OD, overdose. Chi-square test for overall trend across risk groups. For 180-day mortality, both before and after the index OD had p < .001; for 90-day mortality, both before (p = .231) and after (p = .161) the index OD had p > 0.15.
Figure 4 displays the proportions who received medication treatment for opioid use disorder and risk mitigation interventions for substance use disorders after the index opioid overdose (Supplement Table S3). Beneficiaries were less likely to receive these treatments or interventions as their predicted risk of mortality increased. For example, the proportion receiving risk mitigation interventions in the lowest risk group of 180-day mortality (i.e., the bottom 25% of the prediction score) was 56.8%, whereas the proportion decreased to 31.3% in the highest risk group (i.e., the top 2% of the prediction score).
Figure 4.
Proportion of beneficiaries with receipt of medications for opioid use disorder, and proportions with receipt of risk mitigation and behavioral management–related interventions for substance use disorders, after a nonfatal opioid overdose in the validation sample, by risk subgroup. Asterisk symbols indicate data suppressed due to insufficient cell size. OUD, opioid use disorder; SUD, substance use disorders. Risk mitigation and behavioral management–related interventions included filled prescription of naloxone, counseling for SUD, and urine drug test. Chi-square test for overall trend across risk groups. All four p trends < .001.
DISCUSSION
In a large state Medicaid program, a risk prediction model showed good performance in predicting mortality within 180 days after a nonfatal opioid overdose and could effectively classify beneficiaries into subgroups based on estimated probability of death. We also found an inverse relationship between predicted mortality risk after overdose and receipt of medication treatment for opioid use disorder or risk mitigation interventions for substance use.
Receiving care for opioid overdose during emergency department or inpatient visits is an opportunity to identify and intervene in patients with opioid use disorder.24 Because the number of opioid-involved overdoses remains high in the USA, it is important to be able to identify individuals at increased risk of death after nonfatal overdoses and target interventions, resources, and attention to those individuals most likely to benefit. It is important to note that shortly after our study period ended, Pennsylvania launched several programs to improve health outcomes of patients with opioid use disorder, including more intensive warm hand-off programs to ensure a seamless transition for opioid overdose survivors from emergency medical care to specialty substance use disorder treatment,25 programs to enhance access to naloxone,26 and Centers of Excellence program to include quality of opioid use disorder treatment.27 To our knowledge, this is the first study that has focused on developing a tool for identifying those high- and low-risk individuals, using claims and public data available prior to the overdose.
Our model exhibited good predictive utility, achieving a C-statistic of 0.71. Given that no single performance threshold may be suitable for every purpose, we present prediction performance statistics (e.g., PPV, NPV) at different levels of sensitivity (i.e., at the level of balanced sensitivity and specificity determined by the Youden index and at the level of 90% sensitivity) to provide comprehensive understanding on the model performance. At 90% sensitivity, the model performed well to eliminate most of the individuals at minimal risk of death (NPV > 98% for 180-day mortality, 99.2% for 90-day mortality). The risk stratification based on individual prediction score showed good performance in identifying beneficiaries at greater risk of death after a nonfatal overdose. In the subgroup having a risk score in the top 2%, the observed mortality rate was as high as 20% within 180 days of the index overdose. Future studies are needed to externally validate this prediction model.
When trying to predict risk of death among survivors of opioid overdose, it may be difficult to further differentiate between individuals who are already at generalized increased risk of premature mortality3,4 recurrent overdose,28 and other adverse health outcomes.29 This difficulty is similar to the challenge that has been described when predicting readmissions.30–33 Because the current model was developed based only on claims data, we were not able to incorporate important clinical and social-behavioral information that could potentially improve prediction performance, such as vital signs and lab results during the overdose visit, severity of comorbid conditions, and access to illicit opioids.
Many of the important features in our prediction model are regional level factors (e.g., rate of persons in deep poverty and z score of quality of life, seen in Figure S2). These regional factors may be proxies for other characteristics or social circumstances of individuals and reflect variables one might obtain from the medical record. The prediction performance of the model might be improved with using more robust linked data including behavioral factors and medical records. Indeed, Pennsylvania is currently working towards such integration of data from multiple sources.
Findings from the post hoc analyses point to potential uses of risk prediction following nonfatal overdose. Overdose survivors at the highest risk of death based on the prediction score were most likely to have prescription opioid fills after overdose. While medication treatment has been shown to prevent premature death in individuals with opioid use disorder,10,11 less than one-third of patients in our cohort received medication treatment after an overdose, and only half had received risk mitigation interventions for substance use disorders. Patients in the subgroups at greater predicted risk of death based on characteristics measured before their overdose were also less likely to receive treatment or intervention after their overdoses. This finding could be due to several factors. Because individuals in the lowest risk group had more follow-up time, they had greater opportunity to receive these therapies relative to the higher risk group with a higher rate of death. It may also be that the factors most predictive of death following an overdose are also correlated with low perceived need for treatment or care-seeking behavior. Nevertheless, this post hoc analysis points to the potential value of risk prediction tools for identifying patients most at risk and connecting them to treatment and recovery supports.
Our study has several limitations. First, used Medicaid claims primarily, and thus any services received, or overdoses occurring, outside of the healthcare system are not captured. Thus, the application of our prediction model would be limited to overdose survivors identified in a medical setting. Second, because of the lack of access to data on the cause of death, we were not able to predict cause-specific mortality, such as death from overdose. Third, because we used Medicaid data from one state and only up through 2016, the findings may not be generalizable to other populations or more recent time periods. Finally, our study is only the first step in risk prediction—development of the machine learning model. There are many challenges to implementing such a prediction model that have not yet been addressed, including addressing gaps and lags in claims data, updating the model with more current data, developing the infrastructure that automatically generates risk scores, and testing its usability and effectiveness in real-world practice. In addition, our model focuses on predicting risk of death and not predicting who would most benefit from an intervention nor which individual risk factors are causally associated with mortality. Future studies will need to determine if use of this model for risk prediction can also identify those most likely to benefit from interventions and how to reduce risk.
CONCLUSIONS
We applied a machine learning model to identify Medicaid beneficiaries at increased risk of death after a nonfatal overdose. This prediction score can identify high-risk subgroups to target interventions to improve outcomes among overdose survivors.
Supplementary Information
(DOCX 557 kb)
Acknowledgments
We thank Dr. David Kelley from the Pennsylvania Department of Human Services for his feedback about these analyses.
Funding
This study was supported by the National Institute on Drug Abuse (grant number R01DA044985). Access to the data was supported in part by an intergovernmental agreement between the University of Pittsburgh and the Pennsylvania Department of Human Services.
Compliance with Ethical Standards
Conflict of Interest
The authors declare that they do not have a conflict of interest.
Footnotes
Key Points
Question: Can a model be developed to predict an individual’s risk of mortality after a nonfatal opioid overdose among Medicaid beneficiaries?
Findings: A machine learning model performed well for classifying mortality risk after a nonfatal opioid overdose. In the subgroup with a risk score in the top 2%, the observed mortality rate was as high as 20% within 180 days of the index overdose. In the group with the lowest risk score (bottom 25%), the observed risk was only 1.5%.
Meaning: This prediction score can identify high-risk subgroups to target interventions to improve outcomes among overdose survivors.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.DeWeerdt S. Tracing the US opioid crisis to its roots. Nature. 2019;573(7773):S10–S12. doi: 10.1038/d41586-019-02686-2. [DOI] [PubMed] [Google Scholar]
- 2.Frazier W, Cochran G, Lo-Ciganic WH, et al. Medication-assisted treatment and opioid use before and after overdose in Pennsylvania Medicaid. JAMA. 2017;318(8):750–752. doi: 10.1001/jama.2017.7818. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Olfson M, Crystal S, Wall M, Wang S, Liu SM, Blanco C. Causes of death after nonfatal opioid overdose. JAMA Psychiatry. 2018;75(8):820–827. doi: 10.1001/jamapsychiatry.2018.1471. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Weiner SG, Baker O, Bernson D, et al. One-Year Mortality of Patients After Emergency Department Treatment for Nonfatal Opioid Overdose. Ann Emerg Med. 2019;75:1–5. doi: 10.1016/j.annemergmed.2019.04.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Leece P, Chen C, Manson H, Orkin AM, Schwartz B. One-Year Mortality After Emergency Department Visit for Nonfatal Opioid Poisoning : A Population- Based Analysis. Ann Emerg Med. 2019:1-9. 10.1016/j.annemergmed.2019.07.021. [DOI] [PubMed]
- 6.Krawczyk N, Eisenberg M, Schneider KE, et al. Predictors of Overdose Death Among High-Risk Emergency Department Patients With Substance-Related Encounters: A Data Linkage Cohort Study. Ann Emerg Med. 2019:1-12. 10.1016/j.annemergmed.2019.07.014. [DOI] [PMC free article] [PubMed]
- 7.Iams JD, Newman RB, Thom EA, et al. Frequency of Uterine Contractions and the Risk of Spontaneous Preterm Delivery. N Engl J Med. 2002;346(4):250–255. doi: 10.1056/NEJMoa002868. [DOI] [PubMed] [Google Scholar]
- 8.Lo-Ciganic W-H, Huang JL, Zhang HH, et al. Evaluation of Machine-Learning Algorithms for Predicting Opioid Overdose Risk Among Medicare Beneficiaries With Opioid Prescriptions. JAMA Netw Open. 2019;2(3):e190968. doi: 10.1001/jamanetworkopen.2019.0968. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Larochelle MR, Liebschutz JM, Zhang F, Ross-Degnan D, Wharam JF. Opioid prescribing after nonfatal overdose and association with repeated overdose: A cohort study. Ann Intern Med. 2016;164(1):1–9. doi: 10.7326/M15-0038. [DOI] [PubMed] [Google Scholar]
- 10.Ma J, Bao Y-P, Wang R-J, et al. Effects of medication-assisted treatment on mortality among opioids users: a systematic review and meta-analysis. Mol Psychiatry. 2018. 10.1038/s41380-018-0094-5. [DOI] [PubMed]
- 11.Bernstein SL, D’Onofrio G. Screening, treatment initiation, and referral for substance use disorders. Addict Sci Clin Pract. 2017;12(1):18. doi: 10.1186/s13722-017-0083-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Medicaid & CHIP. The Henry J. Kaiser Family Foundation. https://www.kff.org/state-category/medicaid-chip/. Accessed January 3, 2020.
- 13.Lo-Ciganic W-H, Gellad WF, Gordon AJ, et al. Association between trajectories of buprenorphine treatment and emergency department and in-patient utilization. Addiction. 2016;111(5):892–902. doi: 10.1111/add.13270. [DOI] [PubMed] [Google Scholar]
- 14.Zedler B, Xie L, Wang L, et al. Development of a Risk Index for Serious Prescription Opioid-Induced Respiratory Depression or Overdose in Veterans’ Health Administration Patients. Pain Med (United States). 2015;16(8):1566–1579. doi: 10.1111/pme.12777. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Dunn KM, Saunders KW, Rutter CM, et al. Opioid Prescriptions for Chronic Pain and Overdose. Ann Intern Med. 2010;152(2):85. doi: 10.7326/0003-4819-152-2-201001190-00006. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Use the Data | County Health Rankings & Roadmaps. https://www.countyhealthrankings.org/explore-health-rankings/use-data. Accessed October 8, 2019.
- 17.Area Health Resources Files. https://data.hrsa.gov/topics/health-workforce/ahrf. Accessed October 8, 2019.
- 18.Kind AJH, Buckingham WR. Making Neighborhood-Disadvantage Metrics Accessible — The Neighborhood Atlas. N Engl J Med. 2018;378(26):2456–2458. doi: 10.1056/NEJMp1802313. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.data.pa.gov | PA Open Data Portal. https://data.pa.gov/. Accessed October 8, 2019.
- 20.Python Package XGBoost. https://xgboost.readthedocs.io/en/latest/python/python_intro.html. Accessed October 8, 2019.
- 21.Natekin A, Knoll A. Gradient boosting machines, a tutorial. Front Neurorobot. 2013;7:21. doi: 10.3389/fnbot.2013.00021. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Fluss R, Faraggi D, Reiser B. Estimation of the Youden Index and its associated cutoff point. Biom J. 2005;47(4):458–472. doi: 10.1002/bimj.200410135. [DOI] [PubMed] [Google Scholar]
- 23.CMS Cell Size Suppression Policy. https://www.resdac.org/articles/cms-cell-size-suppression-policy. Accessed January 3, 2020.
- 24.D’Onofrio G, O’Connor PG, Pantalon MV, et al. Emergency department-initiated buprenorphine/naloxone treatment for opioid dependence: A randomized clinical trial. JAMA. 2015;313(16):1636–1644. doi: 10.1001/jama.2015.3474. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Warm Hand-Off. Pennsylvania Department of Drug and Alcohol Programs. https://www.ddap.pa.gov/Pages/Warm-Hand-Off.aspx. Accessed January 3, 2020.
- 26.Spotlight on Pennsylvania Leading-Edge Practices and Next Steps in Ending the Opioid Epidemic.; 2018. https://www.pamedsoc.org/. Accessed January 3, 2020.
- 27.Centers of Excellence. Pennsylvania Department of Human Services. https://www.dhs.pa.gov/Services/Assistance/Pages/Centers-of-Excellence.aspx. Accessed July 10, 2020.
- 28.Peterson C, Liu Y, Xu L, Nataraj N, Zhang K, Mikosz CA. U.S. National 90-Day Readmissions After Opioid Overdose Discharge. Am J Prev Med. 2019;56(6):875–881. doi: 10.1016/j.amepre.2018.12.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Elzey MJ, Barden SM, Edwards ES. Patient Characteristics and Outcomes in Unintentional, Non-fatal Prescription Opioid Overdoses: A Systematic Review. Pain Physician. 2016;19(4):215–228. doi: 10.36076/ppj/2019.19.215. [DOI] [PubMed] [Google Scholar]
- 30.Frizzell JD, Liang L, Schulte PJ, et al. Prediction of 30-day all-cause readmissions in patients hospitalized for heart failure: Comparison of machine learning and other statistical approaches. JAMA Cardiol. 2017;2(2):204–209. doi: 10.1001/jamacardio.2016.3956. [DOI] [PubMed] [Google Scholar]
- 31.Wolff P, Grana M, Ríos SA, Yarza MB. Machine Learning Readmission Risk Modeling: A Pediatric Case Study. Biomed Res Int. 2019;2019. 10.1155/2019/8532892. [DOI] [PMC free article] [PubMed]
- 32.Rafi P, Pakbin A, Pentyala SK. Interpretable Deep Learning Framework for Predicting all-cause 30-day ICU Readmissions. College Station: Texas A&M University. 2017.
- 33.Morgan DJ, Bame B, Zimand P, et al. Assessment of Machine Learning vs Standard Prediction Rules for Predicting Hospital Readmissions. JAMA Netw Open. 2019;2(3):e190348. doi: 10.1001/jamanetworkopen.2019.0348. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
(DOCX 557 kb)