High Risk Prescribing and Opioid Overdose: Prospects for Prescription Drug Monitoring Program Based Proactive Alerts

Abstract

In order to develop a simple, valid model to identify patients at high risk for opioid overdose-related hospitalization and mortality Oregon PDMP, Vital Records, and Hospital Discharge data were linked to estimate two logistic models; A first model that included a broad range of risk factors from the literature and a second simplified model. ROC curves, sensitivity and specificity of the models were analyzed. Variables retained in the final model were age categories over 35, number of prescribers, number of pharmacies, and prescriptions for long acting opioids, benzodiazepines/sedatives, or carisoprodol. The ability of the model to discriminate between patients who did and did not overdose was reasonably good (AUC = .82, Nagelkerke R² = .11). The positive predictive value of the model was low. Computationally simple models can identify high risk patients based on prescription history alone, but improvement of the predictive value of models may require information from outside the PDMP. Patient or prescription features that predict opioid overdose may differ from those that predict diversion.

INTRODUCTION

The proliferation of prescription opioids has become a critical public health issue in the United States (US), Canada, and, increasingly, Europe, Australia and New Zealand [11]. In the US, prescriptions for opioid analgesics have quadrupled over the past two decades. Prescription opioid overdoses claimed over 14,000 lives in the US in 2014 [6]. Previous studies have found an association between opioid prescription and rates of overdose mortality [3,12,14,17,20,21,27,29,31,33]. The increase in opioid related mortality has prompted scrutiny of prescription drug misuse, diversion, and “doctor shopping.” PDMPs have been implemented in 49 States, most of which allow clinicians to track patient prescription histories [22].

High-risk opioid use involves patient behaviors, clinician prescribing, and the clinician-patient interaction. Patterns of opioid prescribing to opioid-naïve patients are associated with probability of subsequent long term use [10]. Over time, tolerance and hyperalgesia may lead to patient drug-seeking behavior. In States with PDMPs, clinicians are privy to a patient’s prescription history at the time of an encounter. Routine use of the PDMP may allow clinicians to avoid writing for overlapping prescriptions or dangerous interactions.

Prescribers interpret data in the PDMP with little guidance. Some PDMP’s use proactive alerts [22] to help clinicians identify patients who exceed thresholds for number prescribers or pharmacies, high dose, or prescription overlap. No definitive set of evidence-based criteria for triggering proactive alerts exist. Accuracy of alerts in predicting overdose, diversion or abuse is only partly understood.

There may also be some risk of creating “alert fatigue,” as observed with drug interaction alerts in electronic medical record systems [28]. The utility of a proactive alert is blunted if the clinician finds it burdensome or irrelevant. A model that generates alerts that are sensitive, but not specific, may be of little value.

Many PDMP systems are focused on identifying drug diversion or abuse for law-enforcement purposes. Models that identify patients at high risk of overdose may differ in both variables employed and parameter values. Alerts directed to clinicians should address health outcomes. We developed a model for identifying patients at highest risk for overdose using prescription patterns and patient characteristics in PDMP data. We simplified the model so it could potentially be used as a proactive alert in PDMP systems where complex calculations would limit feasibility and slow system response time, presenting a deterrent to PDMP utilization.

Specific aims of the present research were:

1. To evaluate the strength of prescription-related risk factors associated with opioid overdose.

2. To develop a predictive model for opioid overdose that could be calculated using PDMP records alone.

3. To reduce the model to the fewest and simplest possible data elements that could inform a proactive alert.

4. To assess model sensitivity and specificity.

5. To assess model generalizability.

The guidelines established in the “Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD)” Statement represent the most rigorous standard for reporting findings from predictive modeling [8]. We present our findings in a manner consistent with this standard.

METHODS

Data were drawn from the Oregon PDMP, Oregon Vital Records, and the Oregon Hospital Discharge Database. Consistent with State law, an analyst at the Public Health Division prepared a de-identified data set for use by our research team. Potential identifiers were stripped or categorized such that patients, prescribers, and pharmacies were not individually identifiable. The data captured all controlled substance prescription fills, hospitalizations, and deaths between October 1, 2011 and October 31, 2014.

The commercial vendor that hosts Oregon’s PDMP matches multiple prescriptions to an individual using a proprietary and largely deterministic algorithm based on name, date of birth, and address. Consequently, the same individual may have numerous records in the system due to use of nicknames, changes of legal name or residence, errors in spelling, or reversed characters/digits. Prior to de-identification, a Public Health analyst therefore used the Link King v7.1 software to match individuals within and between datasets. The software uses name, date of birth, and zip code to classify possible matching records into “linkage certainty levels [1,5].” A random sample of linked record pairs was manually inspected within each certainty level for validation. Where linkage certainty levels fell below 95% positive predictive value, possible record pairs were reviewed individually. This process resulted in an improved unique patient identifier in the PDMP data, which was then used to link patients between PDMP, Vital Records, and Hospital Discharge data sets.

Drugs were classified by pharmaceutical class using Food and Drug Administration National Drug Codes [30]. All opioid medications were included in this analysis regardless of therapeutic use, except Tramadol, which was not tracked in the PDMP until August of 2014, and combinations of buprenorphine with naloxone/naltrexone. Non-human records, such as inventory transfers between pharmacies and drugs prescribed to pets were also excluded, as were prescriptions from out-of-state prescribers.

“Morphine milligram equivalents” (MME) were calculated for each prescription using the Centers for Disease Control and Prevention Conversion Reference Tables [23]. Where conversion factors were unavailable, a clinical pharmacist used the drug name and other properties to identify equivalent drugs with associated conversion factors. Where data about duration of action (immediate or extended release) were missing, the clinical pharmacist assigned duration of action based on equivalent drugs [24].

Population

The study population included Oregon residents over 12 years of age who received a prescription for an opioid medication from an Oregon physician during calendar 2013. The validation analysis used comparable data from 2012 alone. We restricted analysis to individuals who had only Oregon addresses, to capture the most complete patient records.

Defining Adverse Outcomes

Our goal was to identify adverse outcomes, defined as opioid overdose mortality or hospitalization. Prescription opioid overdose deaths were identified by ICD-10 codes for underlying cause of death and contributing cause of death indicating poisoning by prescription opioids (T40.2, T40.3) in the Oregon vital records.

Hospitalizations in the Oregon Hospital Discharge Database were included if discharge diagnoses indicated a definite or probable prescription opioid overdose. Definite overdose related hospitalizations were identified using ICD-9 codes indicating poisoning by opioid substances (965.0, 965.00, 965.02, 965.09, E850.1, E850.2). Probable overdoses were identified by codes indicating adverse effects of opioids (E935.1, E935.2) coupled with codes on the same date suggestive of overdose or poisoning (276.4, 292.11, 292.12, 292.81, 292.89, 486.0, 496.0, 518.81, 518.82, 780.01, 780.02, 780.03, 780.09, 780.97, 786.03, 786.05, 786.09, 799.01, 799.02, 799.10, E950.0, E950.1, E950.2, E950.3, E950.4, E950.5, E980.0) [9]. These definitions were based on a synthesis of approaches presented in the literature [12,18,21,27,31].

Predictive Measures

Predictive measures (independent variables) in the models were derived from PDMP data, including patient characteristics and risk factors related to prescribing patterns. The PDMP data set contained patient age category and urban/rural designation but no gender or other demographic information. The information collected by the Oregon PDMP is defined in statute and was intentionally limited. At the time of the study, even gender was not recorded in the database. Other variables such as education, race, and income are not recorded.

Prescription-related independent variables were drawn from the emerging body of literature on prescribing practices with high-risk for opioid overdose. All variables were calculated over the full 12-month study period. See Table 1 for a list of independent variables and corresponding definitions. The number of prescribers, pharmacies, and prescriptions for opioids per patient were continuous counts. The “four by four” metric (≥ 4 prescribers and ≥ 4 pharmacies) [20] is also discussed here, because it is a commonly cited risk metric.

Table 1.

Logistic Regression Results

	Model 1		Model 2
Parameter	aOR	95%CI	aOR	95%CI
Rural	0.94	(0.84, 1.04)	-	-
25–34 years	1.12	(0.77, 1.63)	-	-
35–44 years	1.59**	(1.12, 2.27)	1.47**	(1.15, 1.88)
45–54 years	2.13***	(1.52, 2.98)	1.95***	(1.57, 2.44)
55–64 years	3.13***	(2.25, 4.35)	2.82***	(2.29, 3.48)
65–74 years	4.15***	(2.98, 5.78)	3.68***	(2.97, 4.57)
75 + years	5.79***	(4.16, 8.06)	4.99***	(4.02, 6.19)
Number of Prescribers	1.13***	(1.10, 1.17)	1.15***	(1.12, 1.18)
Number of Pharmacies	1.11***	(1.06, 1.16)	1.11***	(1.06, 1.16)
Number of Prescriptions	0.99**	(0.98, 0.99)	-	-
Opioid-Opioid Overlap	2.48***	(2.16, 2.86)	-	-
LA/SA Opioid Overlap	2.70***	(2.32, 3.13)	-	-
Any Prescription of ER/LA opioid	-	-	4.41***	(3.93, 4.94)
Opioid-Benzodiazepine/Sedative Overlap	2.13***	(1.89, 2.39)	-	-
Any Prescription of Benzodiazepine/Sedative	-	-	2.50***	(2.23, 2.79)
Opioid-Benzo-Carisoprodol Overlap	1.59*	(1.08, 2.32)	-	-
Any Prescription of Carisoprodol	-	-	1.63**	(1.25, 2.13)
MMEDD >= 90 mg	1.52***	(1.25, 1.83)	-	-

Note.

^*p<.05,

^**p<.01,

^***p<.001;

LA/SA = Long Acting/Short Acting; ER/LA = Extended Release/ Long Acting; MMEDD = Morphine Milligrams Equivalent Daily Dose; aOR = adjusted odds ratio; CI = confidence interval; For complete reporting of regression coefficients see online supplemental tables.

“Overlap” variables indicate that a patient is receiving multiple prescriptions for controlled substances that may be concurrent (opioid-opioid, opioid-benzodiazepine, opioid-benzodiazepine-carisoprodol) [4,16,19,25,26,31]. Such patients may be at greater risk due to pharmacokinetic or pharmacodynamic interactions, or high cumulative opioid dose [31]. An alternative approach using binary indicators for any prescription of the individual drugs during the study year was also considered.

Total MME was calculated for each prescription. Each patient’s prescriptions were aggregated, and averaged over each month to generate an average daily dose. The maximum average daily dose for each patient (the highest monthly dose) during the study period was retained. The binary “High Dose” variable was set equal to 1 where the patients’ maximum average daily dose was greater than the CDC-recommended cutoff of 90 MME.

Analysis

We first examined the association of patient demographic variables and prescribing patterns with the outcome of overdose events using unadjusted odds ratios (OR). Metrics were then evaluated for usefulness as predictors. Two logistic regression models are presented here. One evaluated all candidate risk factors, and a second reduced the number of variables to a minimal model. The objective of the logistic regression models is to predict a binary outcome (opioid overdose) as a linear combination of risk factors. The multivariable model separates out the unique variance explained by each variable after controlling for the influence of all the others. The dependent variable in a logistic model is a continuous estimate of the log odds of an overdose occurring, and may be thought of as representing how often we would expect a patient with the given characteristics and history to have experienced an overdose.

We analyzed goodness-of-fit using an adjusted Hosmer-Lemeshow test. The Hosmer-Lemeshow test divides the population into subgroups based on predicted value of the dependent variable. Within each group, fit is assessed by how well the predicted frequency accords with the observed frequency. The resulting statistic is distributed approximately chi-square and permits hypothesis testing for model fit. This test is sensitive to large sample sizes and to the number of subgroups used.

In ordinary least squares (OLS) regression models, R2 provides a measure of the percentage of variance in the dependent variable explained by the model. In logistic regression, the R2 cannot be calculated. Several pseudo-R2 measures have been proposed, including the Cox and Snell and Nagelkerke methods, each with strengths and weaknesses. Cox and Snell performs best when the probability of the dependent variable is near .5, and the upper bound approaches 1. In the case of rare events, the upper bound of Cox and Snell R2 can be quite low, making its interpretation challenging. Nagelkerke R2 is a rescaled version Cox and Snell calculated by dividing by the upper bound of the test. This normalization of the statistic gives it an interpretation similar to that of the R2 statistic in OLS regression in the case of rare outcomes [7].

Receiver operating characteristic (ROC) curve analysis was also performed: a plot of sensitivity versus 1 - specificity of the models, indicating their ability to discriminate between patients who did or did not have an overdose event. The objective of logistic regression is to predict a binary outcome; whether the patient will or will not experience an overdose. The dependent variable in a logistic model may be thought of as representing how often we would expect a patient with the given characteristics and history to have experienced an overdose. To generate the binary outcome we decide on a threshold that delineates positive from negative predictions. As a model becomes more sensitive, by classifying more cases as positive, it also increases the number false positives, reducing specificity.

The Receiver Operating Characteristic (ROC) curve visualizes this tradeoff between sensitivity and specificity. The perfect predictive model would identify each event and non-event correctly. For such a model the ROC plot would be a right angle in the upper left corner. A model that performs no better than chance would have an ROC curve that was a diagonal line from the lower left to the upper right corner of the plot. Most models have an ROC curve that falls somewhere in between. The area under the ROC curve (AUC) quantifies the success of discrimination, ranging from 0.5 (no better than chance) to 1.0 (perfect discrimination) [15].

In assessing the generalizability of a model tests of model fit and statistical significance may be inadequate. Overfitting of the model to the data may be an issue [2]. To address this concern a separate dataset was compiled from the PDMP for 2012. This data set was used to validate the final model. Observations in the validation data set were classified using the final model and then performance of the model on the two data sets was compared using the AUC.

All analyses were performed using SAS 9.4 for Windows.

RESULTS

Patients, Prescriptions, and Events

The analytic data set for 2013 consisted of data on 6,334,197 prescriptions (all controlled drugs) written to 879,402 unique patients. Of the patients represented in the data set 61% were residents of urban areas. This was consistent between patients who did and did not experience an adverse event. Patient age was grouped into 7 categories 12–24 (12%), 25–34 (15%), 35–44 (15%), 45–54 (17%), 55–64 (18%), 65–74 (13%), and 75+ (9%). Age distribution varied considerably with between patients who did and did not experience an adverse event. Demographic information that may be recorded by the Oregon PDMP is defined in statute and was deliberately limited. During the study period even patient gender was not recorded. This is discussed further in the limitations section.

The most commonly prescribed medications by class were opioids (69.71%) followed by benzodiazepines (16.5%), non-benzodiazepine sedatives (5.99%), and stimulants (3%). The most commonly prescribed opioids were hydrocodone-acetaminophen combinations (33.84%) and oxycodone combinations (20.1%).

Opioid-related mortality from the Oregon Vital Records file included 99 prescription opioid related deaths (other than methadone) and 53 methadone-related deaths. Allowing for a small number of cases in which both methadone and non-methadone opioids were implicated there were 139 prescription opioid-related deaths during the study period. Opioid-related hospitalizations included 893 definite overdose events and 766 probable overdose events. A total of 1409 unique patients experienced an overdose during the study period. Heroin-related events that did not involve a prescription opioid were excluded.

Aim 1: Unadjusted Associations between Patient / Prescription Characteristics and Overdose Events

Odds ratios (ORs) were calculated for demographic variables and all prescribing variables. Full results of this analysis are reported in Online Supplemental Table 1. The odds of an opioid-related adverse event increased with each advancing age category, with all age groups showing statistically significant elevation of risk relative to the reference group (12–24 year olds. Rural residency was not significantly associated with likelihood of an overdose event (OR = 1.03, 95% CI: .93, 1.15).

Counts of prescribers, pharmacies, and prescriptions all showed an increasing trend in odds ratio of experiencing an opioid-related adverse event. There was no clear empirical threshold. The OR for the “four by four” metric was large and statistically significant (OR = 18.59, 95% CI: 15.24, 22.68) but slightly less than the sum of the individual ORs for four prescribers (OR = 9.82, 95% CI: 8.56, 11.27) and four pharmacies (OR = 9.96, 95% CI: 8.45, 11.72). A model containing both variables and an interaction term indicated that the combination metric was largely a linear combination of the two constituent variables with a small negative interaction.

Variables related to overlapping prescriptions for opioids with other opioids, benzodiazepines/non-benzodiazepine sedatives, or carisoprodol and between long-acting and short-acting opioids, were all significantly associated with the likelihood of an overdose event. We also tested metrics indicating any prescription for a benzodiazepine, carisoprodol, or long-acting opioid prescription at any time during the study year as possible alternatives to more complex overlap variables.

Aim2: Logistic Regression Model 1

A logistic regression model was estimated simultaneously incorporating demographics, prescribers, pharmacies, and prescriptions, overlap variables and high dose. Number of prescribers, pharmacies and prescriptions were mean-centered to address concerns of collinearity. Step up and step down approaches were also applied. All three approaches suggested the same model (Table 1). Age was associated with increasing risk, but only age categories over 35 were statistically significant. Magnitudes of effects in the logistic model were smaller than their unadjusted counterparts.

Aim 3: Logistic Regression Model 2

A second, more parsimonious model was created based on the results of Model 1 (Table 1). Age categories were reduced to only 35–44 (aOR = 1.47, 95% CI: 1.14, 1.88), 45–54 (aOR = 1.95, 95% CI: 1.57, 2.44), 55–64 (aOR = 2.82, 95% CI: 2.29, 3.48), 65–74 (aOR = 3.68, 95% CI: 2.97, 4.57), >75 (aOR = 4.99, 95% CI: 4.02, 6.192) with those under age 35 used as the reference. The number of prescribers (aOR = 1.15 for each additional prescriber, 95% CI: 1.12, 1.18), number of pharmacies (aOR = 1.11 for each additional pharmacy, 95% CI: 1.06, 1.16) were retained in the model. Variables that required calculating a prescription overlap period were removed and replaced with similar but easier to calculate binary indicators for any prescription for long acting opioids (aOR = 4.41, 95% CI: (3.93, 4.94), benzodiazepines or non-benzodiazepine sedatives (aOR = 2.50, 95% CI: 2.23, 2.79), or carisoprodol (aOR = 1.63, 95% CI: 1.25, 2.13) in the study year. The number of prescriptions and high dose were dropped due to small effect sizes after adjustment for other variables.

Aim 4: Model fit, Sensitivity & Specificity

For model 2, the Cox and Snell R² (.0026) and Nagelkerke R² (.1071) suggest different degrees of explanatory power. In this situation, with rare outcomes, the Nagelkerke R² may be the better estimate. A Hosmer-Lemeshow (H-L) test for model fit was calculated for each model. In Model 1 the model fit statistic indicated that the model fit the data well, χ2 (8) = 14.58, p = .07. For Model 2 the model fit statistic indicated that this model does not fit the data as well, χ2 (7) = 38.89, p <.0001. While the simpler model may not fit the data as well as the full model, its explanatory power was not adversely impacted. Explanatory power and goodness of fit of a model are often though not always associated.

The ROC curve in Figure 1a and AUC (0.82) indicate that Model 2 performs substantially better than a random model, χ2 (1) = 3182.34, p <.0001. Table 2 indicates that there is an almost symmetrical tradeoff between sensitivity and specificity moving from a cut-off value of .001 and .002. Using a cut off of .002 the sensitivity and specificity of the model were 62% and 82% respectively. These may be evaluated relative to the total number of patients who did experience an adverse event (n = 1409) and the total number of patients who did not (n = 837280).

Fig 1.

Fig. 1a & 1b: Receiver Operating Characteristic (ROC) Curves

Receiver Operating Characteristic (ROC) curves illustrate the tradeoff between sensitivity and specificity for model 2 on training (2013) and validation (2012) data sets.

Table 2.

Prediction Classification Table for Model 2

	Correct		Incorrect
P	Event	Non-event	Event	Non-event	Percent Correct	Sensitivity	Specificity	FPR	FNR	PPV
0.000	1409	0	837000	0	0.2	100.0	0.0	99.8	.	.002
0.001	1220	485000	353000	189	57.9	86.6	57.9	99.7	0.0	.003
0.002	891	688000	150000	518	82.1	63.2	82.1	99.4	0.1	.006
0.003	786	744000	93056	623	88.8	55.8	88.9	99.2	0.1	.008
0.004	659	775000	62331	750	92.5	46.8	92.6	99.0	0.1	.010
0.005	564	792000	45621	845	94.5	40.0	94.6	98.8	0.1	.012
0.006	511	799000	38418	898	95.3	36.3	95.4	98.7	0.1	.013

Note. FPR = False Positive Rate; FNR = False Negative Rate; PPV = Positive Predictive Value.

Prescribing data provided information about risk of adverse outcomes and identified a subgroup among whom adverse events were far more prevalent; however, even among patients identified as high risk by our model, overdose remained a rare event. The positive predictive value of a model is the proportion of positive tests that prove to be correct. The positive predictive value of model two using a cut off of .002 was .006, meaning that among 1,000 patients identified as “high risk”, only 6 were actually hospitalized or died from an opioid-related overdose (Table 2).

Aim 5: Generalizability

Generalizability of our model was assessed by applying the model estimated using the 2013 training data set to validation data from 2012. The area under the ROC curve for the validation data set (AUC = 0.82) was virtually identical to that for the training dataset, suggesting that the model generalizes well to new data.

DISCUSSION (1,323 for this revision)

Our preliminary analysis (specific aim 1) indicated that all of the prescribing risk factors identified from the literature had significant bivariate associations with opioid overdose. When combined into multivariable models, in pursuit of specific aim 2, many of these relationships were attenuated. Given the low prior probability of major overdose events for any given individual, we found that even among patients identified as being at higher relative risk of opioid overdose, the absolute risk remained small. The predictive value of any proactive alert for overdose risk based solely on prescription data appeared to be low. The results of our research suggest caution in using a PDMP-only model for prescribing decisions support. Nonetheless, the factors that predict overdose-related death or hospitalization should recommend caution in prescribing because they are likely to be important predictors of misuse, abuse, diversion, or less severe overdose as well.

While there is some overlap between predictors of overdose and other outcomes of interest, there are some important differences. In our data, increasing age and use of long-acting opioids were important predictors of opioid-related overdose. These factors are not emphasized in models built for identifying misuse or diversion for law-enforcement purposes, but have important clinical implications. Older adults may have lower risks of substance abuse disorder than younger adults [13], but age-related changes in drug metabolism and sensitivity to drug effects may render them more likely to experience inadvertent overdoses. More parsimonious use and lower dosing of opioids in this age group may be important. In our models, the single most powerful predictor of overdose, after adjusting for all others, was a prescription for a long-acting opioid. This has obvious implications for clinical practice, particularly in the treatment of chronic pain, where these medications are often employed.

In addition to advancing age and long-acting opioids, key independent predictors of opioid overdose in our study included the number of different prescribers, number of different pharmacies,, a prescription for a benzodiazepine or other sedative, and any prescription for carisoprodol., While many models have used threshold metrics for number of prescribers [12] and number of pharmacies (most commonly ≥4) [12,14,26], we found that continuous predictors resulted in better model fit. Threshold models may be more useful in identifying diversion, as thresholds may clearly demarcate unusual behavior, while risk of overdose appears to increase continuously.

In developing the parsimonious model 2 (specific aim 3),we found that replacing prescription overlap variables with indicators of any prescription for long-acting opioids, carisoprodol, benzodiazepine, or other sedatives in the same year did not decrease the predictive power of the model. Several factors may explain this finding. Prescription overlap definitions may be imprecise due to ambiguity or unavailability of the intended duration of a prescription. Overlap variables and binary indicators are likely to be highly collinear among those at high risk. Patients may save pills for later use, resulting in delayed effect of the prescription. Further, co-prescriptions may simply be markers of individuals whose medication-taking behavior is more likely to lead to opioid overdose, even if the co-prescription is not involved.

Specific aim 4 is a response to a gap in the literature with respect to the sensitivity, specificity, and positive predictive value of models predicting overdose risk. Models may be highly sensitive at the expense of a large number of false positives, which may induce alert fatigue. Even among patients flagged as high risk using this model, the absolute incidence of major overdose outcomes was low. Nevertheless, these patients may be at increased risk for addiction, misuse, minor overdose, or engaged in diversion of medications. Clinicians must integrate PDMP-derived flags with other patient information, including clinical history, physical examination, and laboratory findings. A model constructed in the Veterans Affairs health care system indicated that combining prescription data of the type we examined with patient demographic, clinical, and health care utilization data from medical records resulted in substantially greater predictive accuracy [32].

Our fifth specific aim was to assess the generalizability of our model. In many applications generalizability is not a matter of model fit to training data, but of how well it classifies a test data set that was not used in the process of model building. Such tests of classification provide evidence that an observed relationship is stable over time, or across different populations. The fact that the model generalized well to another year of data suggests that these few variables are good candidate metrics for predicting patient risk.

Limitations

There are important limitations to our analysis. The PDMP did not collect most patient demographic features, and did not have a unique identifier, such as social security number, during the study years. This necessitated probabilistic matching of patients within the PDMP and between the PDMP and hospital and death registries. Information on days’ supply for prescriptions was also missing, which had two effects. First, overlap variables were imprecise. Secondly, calculation of average daily dose had to be performed over the full study period, making the model less sensitive to high dose prescriptions of short duration.

Our exclusive use of PDMP data to define predictor variables restricts the types of variables we could include in the model. Consequently, the model does not incorporate variables that have been identified in previous studies such as health status.

This analysis is restricted to overdose events that resulted in death or hospital admission. Overdoses that were fully reversed by first responders or in the community with naloxone are not included. Overdose events that were treated and released from the emergency department or in outpatient clinics also do not appear in our data set, nor do overdose events that were never reported or never interacted with medical professionals.

Restricting PDMP data to Oregon providers and Oregon pharmacies leaves the possibility of out-of-state doctor shopping. In cases in which a patient was engaged in doctor shopping over state lines, the model would underestimate patient risk.

Outcomes were measured in the same year as exposures. Consequently, patients who died early in the year are less likely to display high log odds, and patients who did not experience an adverse event may have higher log odds. A patient who died in January of the study period might not appear to display many high risk characteristics simply because there is only a single month of data for them, while a patient who survives throughout the entire year may switch doctors or pharmacies and come to appear much more high risk. This limitation should result in a bias toward a finding of no effect.

Conclusion

Together, our findings may inform strategies for policy-makers designing practical proactive alerts using PDMP data, but also suggest that incorporation of other data may be necessary to achieve high predictive values. Twenty-six states use PDMP data to identify high risk patients and provide proactive alerts to clinicians [23]. In programming proactive alerts, use of simple indicators decreases the complexity of coding, processing time, and the need for data that may not be present in a PDMP system. Our models suggest that a small and computationally simple set of variables may be useful predictors of patient overdose risk, although the predictive value of a positive test is low for rare events.

Our data suggest that a predictive model based on prescription history alone can discriminate between patients at high and low risk for overdose event. However, like many models of rare events, our model suffered from weak positive predictive value. The majority of individuals identified by our model as being at high risk did not suffer an adverse event. Improvement of the predictive value of such models may require information from sources outside most PDMPs, such as a history of hospitalization for opioid related complications, or other clinical data from electronic health records. Nonetheless, our study helps to identify a parsimonious and computationally simple set of PDMP variables for such alerts, and may complement models built primarily for law-enforcement purposes. Testing the optimal design and impact of proactive alerts remains an important research agenda.