Validation and Threshold Identification of a Prescription Drug Monitoring Program Clinical Opioid Risk Metric with the WHO Alcohol, Smoking, and Substance Involvement Screening Test

Abstract

Background:

Prescription drug monitoring programs (PDMPs) are critical for pharmacists to identify risky opioid medication use. We performed an independent evaluation of the PDMP-based Narcotic Score (NS) metric.

Methods:

This study was a one-time, cross-sectional health assessment within 19 pharmacies from a national chain among adults picking-up opioid medications. The NS metric is a 3-digit composite indicator. The WHO Alcohol, Smoking, and Substance Involvement Screening Test (ASSIST) was the gold-standard to which the NS metric was compared. Machine learning determined optimal risk thresholds; Receiver Operating Characteristic curves and Spearman (P) and Kappa (K) coefficients analyzed concurrent validity. Regression analyses evaluated participant characteristics associated with misclassification.

Results:

The NS metric showed fair concurrent validity (area under the curve≥0.70; K=0.35; P=0.37, p<0.001). The ASSIST and NS metric categorized 37% of participants as low-risk (i.e., not needing screening/intervention) and 32.3% as moderate/high-risk (i.e., needing screening/intervention). Further, 17.2% were categorized as low ASSIST risk but moderate/high NS metric risk, termed false positives. These reported disability (OR=3.12), poor general health (OR=0.66), and/or greater pain severity/interference (OR=1.12/1.09; all p<0.05; i.e., needing unmanaged-pain screening/intervention). A total of 13.4% were categorized as moderate/high ASSIST risk but low NS metric risk, termed false negatives. These reported greater overdose history (OR=1.24) and/or substance use (OR=1.81–12.66; all p<0.05).

Conclusions:

The NS metric could serve as a useful initial universal prescription opioid-risk screener given its: 1) low-burden (i.e., no direct assessment); 2) high accuracy (86.5%) of actionable data identifying low-risk patients and those needing opioid use/unmanaged pain screening/intervention; and 3) broad availability.

1.1. Introduction

Community pharmacies are underutilized service settings to address the opioid epidemic. Community pharmacies commonly include chain, grocery, and independent settings. Despite recent national declines in opioid prescribing (IQVIA Institute, 2018), 9.7 million Americans in 2019 reported misusing a prescribed opioid (SAMHSA., 2020), and more than 36% of those misusing opioid medications obtained them by filling a prescription (SAMHSA., 2020). Community pharmacies are nearly ubiquitous across the US given more than 93% of Americans live 5 miles or less (Chain Drug Stores, 2011) of the more than 60,000 locations (CDC, 2013). A major limitation for community pharmacies addressing the opioid epidemic has been their inability to access patient health information, such as is routinely available within electronic health records (Roberts et al., 2019).

The prescription drug monitoring program (PDMP) is the most commonly available and useful clinical tool for pharmacists to identify possible opioid misuse (Ali et al., 2017; Young et al., 2017). The PDMP captures patient-level prescription dispensing information to inform monitoring, dispensing decisions, and possible intervention. Appriss Health is the largest PDMP platform vendor in the US and facilitates PDMP data sharing within 52 PDMPs, captures 400 million monthly transactions, and serves more than 30,000 pharmacies (Appriss Health, 2021). PDMP programs, such as the Appriss platform, have demonstrated mixed results for improving opioid safety, with some studies demonstrating reductions in opioid prescribing (Ali et al., 2017; Bao et al., 2016; Dowell et al., 2016; Kreiner et al., 2017; Lin et al., 2017; Manasco et al., 2016; Young et al., 2017), but with unclear effects on illicit substance use outcomes (Ali et al., 2017), including rates of opioid-related overdose (Nam et al., 2017; Patrick et al., 2016; Paulozzi et al., 2011). PDMP output information available to pharmacists and prescribers often presents un-summarized lists of patient fill data, which is of limited clinical utility, thus, requiring users to rely on “best judgment” when providing patient care and referrals.

Appriss Health has developed and preliminarily tested an opioid risk measure, the Narcotic Score (NS) metric, to identify potential risk for unintentional fatal opioid overdose (Huizenga et al., 2016). However, the NS metric has not been tooled to have clinically actionable risk threshold scores, nor has the validity of the NS metric been evaluated in relation to gold-standard clinical metrics of risky opioid use. Possessing a nationally scalable clinical metric of risky opioid use could support community pharmacists’ decision-making to address opioid-related safety for patients. For instance, a risk metric anchored to specific action steps based in clinically validated risk thresholds could promote appropriate responses by pharmacists for needed care of patients. Such a tool could have an important impact for increasing the clinical utility of PDMPs across the US—thus possibly reducing the mixed effects noted above that have been heretofore shown in the literature for these data systems.

The objective of this paper is to present results of an independent evaluation of the NS metric that identified clinically useful risk thresholds as well as concurrent validity using a gold-standard metric (ClinicalTrials.gov identifier: NCT03936985). This paper follows reporting guidelines set forth in the STrengthening the Reporting of OBservational studies in Epidemiology checklist for cross-sectional studies. This study was funded by the National Institute on Drug Abuse and by the NIH Helping to End Addiction Long-Term^SM Initiative. Appriss Health only provided NS metric scores to study investigators (detailed below) and took no part in the design, conduct, or analyses reported herein.

2.1. Material and Methods

2.1.1. Design, Sites, and Participants

This study was a one-time, cross-sectional, self-administered, health survey among adult patients currently being dispensed opioid medications from 19 pharmacies sites in Ohio and Indiana within a large national US chain from November 2019 to October 2020. Sites were selected by corporate pharmacy partners based on a convenience sampling approach and their judgement that the locations would be feasible for study implementation (had at least 300 patients filling opioid medications within a 6-month period) as well as generally accessible by car to research staff supporting the project. Participant recruitment followed a convenience sampling method and was initiated at point-of-dispensing with pharmacy staff charged with offering study information to all patients or caregivers picking up opioid medications. Interested patients were provided a computer tablet wherein they could enter their contact information that, once submitted, generated an automatic email that included a brief study overview and link to the online consent document. Patients unable to submit contact information on the tablet within the pharmacy or surrogates were given a study flyer with information on how patients could initiate the survey from personal devices. After completing informed consent, patients were directed to complete a study eligibility self-screening assessment.

Individuals were included in the study if they were 18 years of age and older, English speaking, and not receiving current cancer treatment (self-reported). Participants were excluded if they were solely filling buprenorphine or buprenorphine combination products (i.e., patients receiving opioid use disorder treatment with no other opioid medication use); had previously completed the survey (verified by study staff examining identifying information following health assessment submission); or had self-reported current involvement with the criminal justice system. Those meeting all study inclusion/exclusion criteria were advanced to the study survey. Participants who completed the survey were provided with a $50 gift card. This project was approved by the University of Cincinnati and University of Utah Institutional Review Boards.

2.1.2. Measures

Primary measures.

The primary outcome variable for this survey was the NS metric. The NS metric is a continuous indicator on a 000–999 scale, with the last digit representing the number of active opioid prescriptions (those with more than 9 prescriptions coded as 9) and the first two numbers representing a deterministic composite risk score comprised of component indicators, including opioid dosages, numbers of prescribers/pharmacies associated with opioid prescriptions/fills, overlapping opioid prescriptions, and benzodiazepine prescriptions—well-known indicators associated with opioid-related adverse events (Cochran et al., 2017; Huizenga et al., 2016; Sullivan et al., 2010). Higher scores indicate increased risk for adverse opioid-related outcomes (e.g., misuse). The NS metric was provided for each participant to the study team by Appriss Health. This process involved the study team sharing identifying information from participants with Appriss Health (name, address, date of birth, time of survey completion, and pharmacy location where their opioid was filled). Appriss Health in turn identified the specific participants within the Ohio and Indiana PDMP program data, and patient NS scores were returned, wherein quality assurance checks by the research team ensured accuracy of the data.

All other study assessments were completed online as self-report at the pharmacy location or a convenient location for participants. The survey contained 39–52 questions (dependent on number of substances used by participants). The World Health Organization Alcohol, Smoking, and Substance Involvement Screening Test (ASSIST) was used as the gold-standard to which the NS metric was compared. The ASSIST has been found to have criterion, construct, concurrent, and discriminant validity (Humeniuk and Ali, 2006). A priori risk threshold identification and validation was conducted using the ASSIST prescription opioids subscale, developed/validated by McNeely, et al. (McNeely et al., 2016a) Responses were calculated into three discrete ASSIST risk categories—low (no intervention recommended), moderate (brief intervention recommended), and high (referral to treatment recommend) (Humeniuk and Ali, 2006).

Substance use characteristics.

We likewise utilized the additional ASSIST subscales that included street opioids, cannabis, sedatives, cocaine, tobacco, alcohol, methamphetamine (e.g., “crystal meth”), hallucinogens, prescription stimulants (e.g., Adderall and Ritalin), and inhalants (Humeniuk and Ali, 2006) to characterize the population. Lifetime overdose frequency of any drug was assessed using the overdose frequency item from the criterion-valid Overdose Experiences, Self and Witnessed—Drug instrument (Fernandez et al., 2019). This instrument provides a definition of overdose and asks, “How many times in your life has this kind of situation happened to you?” with response options of 0 through 5, or 6 or more.

Physical and mental health characteristics.

Pain was assessed using the Brief Pain Inventory, a well-validated, reliable instrument consisting of a 4-item pain severity subscale and a 7-item pain interference subscale (Keller et al., 2014), with continuous scores of 0–10 and higher scores indicating worse pain severity or pain interference. General health status was measured using a 1-item subscale from the construct-valid Short Form-12 (Luo et al., 2003). Depression was captured using the 2-item criterion-valid Patient Health Questionnaire (PHQ)-2, with a score of 3 considered as the optimal cut-point for depressive disorders (Kroenke et al., 2003).

Morphine milligram equivalents.

Total patient morphine milligram equivalent (MME) dosage over the past 180 days from the PDMP record was also included in the study dataset. Request for these data, linkage, and quality assurance followed the same processes as the NS metric detailed above.

Demographics.

Participant demographics evaluated herein included age (years), sex (male vs. female), ethnicity (Hispanic vs. non-Hispanic), race (White vs. other [“other” analyzed herein given limited sample size amongst subgroups], see Limitations), marital status (married, divorced, widowed, separated, never married, and member of an unmarried couple), employment status (full-time, part-time, temporary leave, looking for work, retired, disabled, homemaker, and student), and insurance status (insured vs. not insured).

2.1.3. Power

Power estimates were based on the allocation ratio of the national rate of prescription opioid use disorder among those prescribed opioid medications in the last year (2.1%; (Han et al., 2017). We calculated an array of sample sizes with α=0.05 and a 0.70 (“fair”) area under the curve (AUC) value (Youngstrom, 2014), with a conservative null hypothesis assumption of 0.5 for discrimination power (Obuchowski, 2005). Our validation sample possessed >80% power to detect an AUC ≥0.70.

2.1.4. A Priori Analyses

A receiver operating characteristics (ROC) analysis was conducted to assess the overall discriminating ability of the NS metric with the ASSIST. AUC ROC curve values were calculated, and the following scale was used for evaluation: <0.70=poor, ≥0.70=fair, ≥0.80=good ≥0.9=excellent (Youngstrom, 2014).

The risk threshold was selected using a machine learning method. Participants were randomly split into training and validation datasets, stratified by ASSIST risk levels. A grid search cross-validation approach was used to select the risk threshold with a training set, in which the optimal value that gave the lowest average misclassification rate was selected. Following value selection, we validated selected risk threshold value using the independent validation set.

The concurrent validity of the NS metric was evaluated by Cohen’s Kappa (K) coefficient (Cohen, 1960; Landis and Koch, 1977) and Spearman (P) correlation (Kendall, 1970) using the independent validation set, with the ASSIST as the standard. Strength of agreement for the K coefficient was labeled as: 0–0.2=slight, 0.21–0.4=fair, 0.41–0.6=moderate, 0.61–0.8=substantial, and 0.81–1=near perfect (Landis and Koch, 1977). Strength of agreement for the P correlation was labeled as: 0–0.29=low, 0.3–0.49=moderate, 0.5–1=high degree (Schober et al., 2018).

2.1.5. Secondary Analyses

In order to describe misclassified individuals who were positive for opioid misuse risk on the NS metric but had low-risk on the ASSIST (i.e., false positives) and conversely those who were low on the NS metric but were moderate or high-risk on the ASSIST (i.e., false negatives), we created a confusion matrix and conducted univariate logistic regression analyses. These univariate analyses assessed the association between false positive/negative designation and participant demographic and health characteristics in the full dataset. In addition, we also conducted a ROC analysis with MME and the ASSIST as a sensitivity analysis for comparison to the NS metric.

3.1. Results

3.1.1. Participant Demographics and Behavioral/Health Characteristics

A total of 1,464 patients from the 19 pharmacies completed the survey and had sufficient data in the PDMP system to generate the NS metric (see Appendix 1 for detailed enrollment chart). Table 1 shows participants were on average 49.6 years of age, with most being White (93%) and female (62.2%). The largest proportion of participants reported currently being employed (35.2%), with the largest portion of those not employed having a disability (22.5%). Most participants reported having health insurance (94.3%). On the ASSIST, 54.1% were identified as having recent history of low-risk opioid medication use, 43.6% had moderate-risk use, and 2.3% had high-risk use. A total of 10% of patients reported an illicit or prescription drug overdose in their lifetime, and roughly 20% of respondents screened positive for depression.

Table 1.

Participant Characteristics (N=1,464)

Variable	N (%a)
Ageb	49.6 (14.8)
Femalec	911 (62.2)
Not Hispanic/Latinx	1447 (98.8)
White	1361 (93.0)
Marital status
Married	797 (54.5)
Divorced	244 (16.7)
Widowed	72 (4.9)
Separated	53 (3.6)
Never married	213 (14.6)
Member unmarried couple	77 (5.3)
Employment
Full-time	515 (35.2)
Part-time	79 (5.4)
Temp leave	72 (4.92)
Looking for work	45 (3.1)
Retired	208 (14.2)
Disabled	329 (22.5)
Homemaker	64 (4.4)
Student	16 (1.1)
Insured	1380 (94.3)
Overdose Experiences, Self and Witnessed—Drug instrument
No lifetime overdose	1318 (90.0)
1 overdose	56 (3.8)
2 overdoses	42 (2.9)
3 overdoses	13 (0.9)
4 overdoses	4 (0.3)
5 overdoses	8 (0.6)
6 + overdoses	17 (1.2)
Short Form12 general health
Poor	148 (10.1)
Fair	447 (30.5)
Good	565 (38.6)
Very good	246 (16.8)
Excellent	54 (3.7)
Patient Health Questionnaire 2 (depression)	286 (19.9)
Brief Pain Inventory
Pain severityb	4.85(2.2)
Pain interferenceb	4.84(2.8)
Alcohol, Smoking, and Substance Involvement Screening Test
Prescription opioids
Low	772 (54.1)
Moderate	623 (43.6)
High	33 (2.3)
Street opioids
Low	1430 (98.7)
Moderate	13 (0.9)
High	6 (0.4)
Cannabis
Low	1274 (88.5)
Moderate	158 (10.9)
High	8 (0.6)
Sedatives
Low	1199 (82.9)
Moderate	238 (16.5)
High	10 (0.7)
Cocaine
Low	1417 (97.9)
Moderate	30 (2.1)
High	1 (0.1)
Tobacco
Low	894 (62.3)
Moderate	495 (34.5)
High	45 (3.1)
Alcohol
Low	1300 (90.5)
Moderate	115 (8.0)
High	21 (1.5)
Methamphetamine
Low	1426 (98.5)
Moderate	18 (1.2)
High	3 (0.2)
Hallucinogensd
Low	1428 (99.4)
Moderate	9 (0.6)
Prescription stimulantsd
Low	1387 (95.7)
Moderate	62 (4.3)
Inhalantsc
Low	1453 (99.9)
Moderate	2 (0.1)

^aCategories may not total 1,464 due to “prefer not to answer” responses.

^bMean (SD) used in place of N (%).

^c“none of these describe me” analyzed as male due to limited sample size of subgroup (n=1).

^dNo high-risk use was reported.

3.1.2. Overall Discriminating Validity of NS Metric

ROC analyses of the NS metric discriminating high- compared to moderate-risk for prescription opioid use on the ASSIST showed fair discrimination (AUC=0.70, standard error [SE]=0.05, 95% confidence interval [CI]=0.59, 0.80), Figure 1. ROC analyses of the NS metric discriminating moderate- compared to low-risk for prescription opioid use on the ASSIST also showed fair discrimination (AUC=0.74, SE=0.01, 95% CI=0.71, 0.76), Figure 1.

Figure 1.

ROC Curve for Narcotic Score Discriminating High-risk vs. Moderate-risk and Moderate-risk vs. Low-risk Opioid Use Patients Validation Sample

3.1.3. Identified Risk Threshold Scores and Concurrent Validity

The identified risk threshold for the NS metric for high- vs. moderate-risk was a score of 602 and for moderate- vs. low-risk was a score of 291, selected based on the training dataset. The agreement between the established risk thresholds with the ASSIST was evaluated with the independent validation set and assessed as fair (K=0.35) and moderate (P=0.37; p<0.001).

The confusion matrix (Table 2) shows the largest proportions of the NS metric risk threshold scores corresponded to the relevant ASSIST risk thresholds, with 37% mapping to low NS metric and ASSIST risk (NS metric ≤291), 30.2% mapping to moderate NS metric and ASSIST risk (NS metric>291 to ≤602), and 0.1% mapping to high NS metric and ASSIST risk (NS metric >602).

Table 2.

Confusion Matrix for Narcotic Score Metric vs. Prescription Opioid Alcohol, Smoking, and Substance Involvement Screening Test (ASSIST) Risk Levels (N=1,464)

	Actual Risk Levels Defined by ASSIST
Predicted Risk Levels Defined by Narcotic Score Metric	Low Risk	Moderate Risk	High Risk
Low Risk Level	527 (37%)	189 (13.3%)	2 (0.1%)
Moderate Risk Level	243 (17.1%)	430 (30.2%)	24 (1.7%)
High-risk Level	2 (0.1%)	4 (0.3%)	2 (0.1%)

^aTable shows 1423 responses due to exclusion of “prefer not to answer” responses on the ASSIST questionnaire

3.1.4. False Positive/Negative

A total of 17.2% of participants were classified as false positives and 13.4% as false negatives. Table 3 shows participants more likely to be designated as false positive: were retired (odds ratio [OR]=3.40, 95% CI= 2.20, 5.26), had disabilities (OR=3.12, 95% CI= 2.10, 4.63), had >1 employment source (OR=2.50, 95% CI=1.27, 4.93), or were on temporary work leave (OR=2.04, 95% CI=1.02, 4.06). Also, false positive participants: had patterns of lower likelihood for substance use (OR=0.16–0.58, p<0.05), were more likely to be widows (OR=2.65, 95% CI=1.56, 4.52), had poorer general health (OR=0.66, 95% CI=0.56, 0.76), and had increased levels of pain severity (OR=1.12, 95% CI=1.05, 1.19) and pain interference (OR=1.09, 95% CI=1.04, 1.15). Additional descriptive analyses (results not shown) of pain among false positive patients showed 28.4% (n=58) with mild, 43.1% with moderate (n=88), and 28.4% (n=58) with high pain severity and 33.8% (n=69) with mild, 25% with moderate (n=51), and 41.2% (n=84) with high pain interference.

Table 3.

Univariate logistic regression for false negative and false positive classification by Narcotic Scores^a (N=1,464)

	False Positive		False Negative
Variable	OR (95% CI)	p	OR (95% CI)	p
Age	1.03 (1.02, 1.04)	<0.01	0.98 (0.98, 0.99)	<0.01
Marital status
Married	1 (ref)		1 (ref)
Divorced	1.28 (0.88, 1.87)	0.19	1.01 (0.67, 1.50)	0.97
Widowed	2.65 (1.56, 4.52)	<0.01	0.25 (0.08, 0.80)	0.02
Separated	1.28 (0.63, 2.63)	0.49	1.00 (0.46, 2.18)	0.99
Never married	0.80 (0.51, 1.26)	0.34	1.23 (0.82, 1.83)	0.32
Member unmarried couple	0.83 (0.41, 1.65)	0.59	0.94 (0.48, 1.83)	0.85
White race	0.82 (0.48, 1.41)	0.48	0.60 (0.36, 0.99)	0.05
Non-Hispanic	1.14 (0.24, 5.31)	0.87	.b	.
Employment
Full-time	1 (ref)		1 (ref)
Part-time	1.15 (0.52, 2.53)	0.73	1.36 (0.75, 2.47)	0.31
Temp leave	2.04 (1.02, 4.06)	0.04	1.53 (0.84, 2.80)	0.17
Looking for work	1.57 (0.63, 3.90)	0.33	2.18 (1.10, 4.32)	0.03
Retired	3.40 (2.20, 5.26)	<0.01	0.67 (0.41, 1.09)	0.11
Disabled	3.12 (2.10, 4.63)	<0.01	0.58 (0.37, 0.89)	0.01
Keeping house	2.35 (1.17, 4.72)	0.02	1.79 (0.97, 3.30)	0.06
Student	0.68 (0.09, 5.26)	0.71	0.36 (0.05, 2.74)	0.32
Other	3.08 (1.55, 6.15)	<0.01	0.41 (0.15, 1.17)	0.10
Multiple reported	2.50 (1.27, 4.93)	0.01	0.96 (0.47, 1.95)	0.90
Insured	0.93 (0.49, 1.75)	0.81	0.55 (0.31, 0.99)	0.05
Overdose history	0.81 (0.65, 1.01)	0.06	1.24 (1.09, 1.41)	<0.01
Poor general health	0.66 (0.56, 0.76)	<0.01	1.09 (0.94, 1.26)	0.27
Depression	0.99 (0.91, 1.07)	0.77	1.04 (0.96, 1.13)	0.28
Pain severity	1.12 (1.05, 1.19)	<0.01	0.99 (0.93, 1.05)	0.71
Pain interference	1.09 (1.04, 1.15)	<0.01	0.99 (0.94, 1.04)	0.71
ASSISTc Illicit/prescription drug use
Prescription opioids	.	.	12.66 (8.41, 19.07)	<0.01
Street opioids	0.32 (0.05, 1.93)	0.21	3.15 (1.60, 6.20)	<0.01
Cannabis	0.47 (0.27, 0.79)	0.01	1.81 (1.26, 2.61)	<0.01
Sedatives	0.37 (0.23, 0.60)	<0.01	1.87 (1.36, 2.57)	<0.01
Cocaine	1.16 (0.49, 2.77)	0.74	3.32 (1.62, 6.80)	<0.01
Tobacco	1.17 (0.91, 1.50)	0.21	1.11 (0.86, 1.44)	0.42
Alcohol	0.58 (0.35, .95)	0.03	2.03 (1.46, 2.83)	<0.01
Methamphetamine	0.27 (0.04, 1.84)	0.18	5.92 (2.63, 13.31)	<0.01
Hallucinogens	.	.	11.67 (2.89, 47.0)	<0.01
Prescription stimulants	0.16 (0.04, 0.67)	0.01	3.17(1.83, 5.48)	<0.01
Inhalants	.	.	.	.

^aFalse negative =low Narcotic Scores and moderate/high ASSIST scores. False positive=moderate/high Narcotic Scores and low ASSIST scores.

^b“.” represents associations that could not be estimated due to limited sample size or indicators that perfectly predicted the outcome.

^cAlcohol, Smoking, and Substance Involvement Screening Test.

Examining false negatives, those looking for work (OR=2.18, 95% CI=1.10, 4.32), with history of drug overdose (OR=1.24, 95% CI=1.09, 1.41) were among those with the highest odds for misclassification. There was a consistent pattern of increased odds (OR=1.81–12.66, p<0.05) for illicit/prescription drug use among those with false negative reports.

3.1.5. Sensitivity Analyses

ROC analyses of MME discriminating high- compared to moderate-risk for prescription opioid use on the ASSIST showed poor discrimination (AUC=0.65, 95% CI=0.54,0.75). ROC analyses of MME discriminating moderate- compared to low-risk for prescription opioid use on the ASSIST showed fair discrimination (AUC=0.70, 95% CI=0.68,0.73).

4.1. Discussion

This study sought to identify risk thresholds and validate a national opioid use risk metric currently deployed by the largest US PDMP vendor, the NS metric, which to date, had largely undefined clinical utility. Results demonstrated the NS metric is a screening tool with fair discriminative accuracy and fair to moderate concurrent validity for detecting recent risky prescription opioid use as measured by the WHO ASSIST. Totaling those correctly identified with low or elevated opioid risk (69.4%) and false positives (17.2.%; i.e., high opioid utilization without risky use reported—likely needing additional pain screening), the NS metric provides a high level of accuracy (86.5%) of clinically actionable information, with superior performance compared typical measures of opioid risk, such as MME (MME compared to NS metric: high vs. moderate-risk 0.65 vs 0.70 and moderate vs. low-risk of 0.70 vs. 0.74, respectively). These findings suggest the NS metric could play a valuable role as a clinically useful “universal screen” due to its wide availability to community pharmacists and its low burden relative to other potential “quick screens,” given it relies upon passive assessment. It is important to note this study was powered based on the national prevalence of prescription opioid use disorder, 2.1% (Han et al., 2017). In our study sample, we identified a similar rate of high-risk prescription opioid use of 2.3% among our participants—which provides additional confidence for the external validity of the sample.

False positive misclassifications appear to be among those with disability, unmanaged pain, poor general health, and work status-related issues. In clinical workflow, false positive classifications would likely need to be clarified/resolved through subsequent patient screening after the pharmacist receives the risk score notification. For instance, future research should investigate if the NS metric is detecting higher levels of opioid use among those with significant unmanaged pain potentially related to work status or disability. Ruling out false positive risk possibly driven by unmanaged pain, with tools like the Brief Pain Inventory (Krebs et al., 2009), could aid pharmacists to effectively triage these patients to non-opioid or complementary pain management resources and referral (U.S. Department of Health and Human Services, 2019), including evidence-based online programs (Winhusen et al., 2021) that could reduce pharmacy staff and patient burden related to office-based care. Such a model could be implemented within a clinical decision support tool and follow a pattern, such as is depicted in Figure 2. The NS metric triggers opioid safety information distribution for low-risk patients. Through passive PDMP screening (i.e. not requiring staff effort), eliminating the need to screen low-risk patients is a clinically significant advantage of the NS metric. Given the tens-of-thousands of patients filling opioids in community pharmacies annually, universal active patient screening is likely impractical within an already very busy practice environment (Kaplan et al., 2021). The NS metric can substantially decrease unnecessary disruptions to workflow and potential patient inconvenience by excluding approximately 50% of low-risk patients from staff-to-patient screening—thus significantly increasing the feasibility of screening in community pharmacies. The NS metric likewise triggers confirmatory screening for moderate and high-risk patients, with those who present as low-risk on confirmatory screening (false positives) receiving auxiliary pain screening, and moderate/high-risk patients (true positives) receiving intervention, warm handoff referral, and naloxone dispensation. The approach of brief universal screening followed by confirmatory assessment is a long-established and recommended standard practice for patients with substance use in healthcare settings (NIAAA, 2007; Smith et al., 2010). Cut-points previously established for the Brief Pain Inventory (0–4=mild pain, 5–6= moderate pain, 7–10=high pain; (Palos et al., 2006) might be utilized to categorize patients by pain level, with low pain patients being continually monitored within the PDMP while moderate and high pain patients could receive non-opioid or complementary pain management resources and referral (U.S. Department of Health and Human Services, 2019). For such a tool, implementation science-based training, monitoring, and follow up with pharmacy professionals could be useful to ensure these tools are effectively received/employed (Ducharme et al., 2016).

Figure 2.

Prescription Drug Monitoring Clinical Decision Support Tool for Prescription Opioid Risk: Flow Chart

^a Narcotic score. ^BThis flow chart only applies to Rx opioid risk. Non-Rx opioid substance screening may be feasible under other circumstances. ^CWHO Alcohol, Smoking, and Substance Involvement Screening Test prescription opioid use risk assessment. ^d Brief Pain Inventory-Short Form. ^e Brief intervention for prescription misuse. ^f Brief intervention for treatment linkage. ^e Treatment. ^h Prescription Drug Monitoring Program.

Implementation of such a clinical decision support tool would come with benefits and challenges for community pharmacy settings. Benefits likely include empowering pharmacists with clinically actionable data produced through passive screening (i.e. not requiring staff time) to engage patients with risky opioid use, which resources are limited within the field of pharmacy practice (Martin et al., 2021). Further, given possibility of implementing such a tool within a PDMP platform that transcends systems and state boundaries, this approach has the potential for significant scalability. In contrast, tool implementation would need support from leadership within community pharmacy settings given the potential draw on staff time for training and actual utilization—and likely hands-on learning to prepare for patient interactions as well as pre-recorded training information on how to use the tool. Further, to increase implementation feasibility, the ability to utilize less costly staff, such as pharmacy technicians, to coordinate or possibly deliver screenings and to make contacts with patient referral agencies (e.g. in anticipation of or actual referral to) may help to offset more costly pharmacist staff time, who likely would be delivering actual interventions. To this important point of cost and feasibility, the ability of the NS metric to eliminate screening need for low-risk patients greatly facilitates feasibility. Further, implementation science approaches for integrating such tools would be critical to staging success and sustainability (Kirk et al., 2016).

Conversely, false negative designation appears to point to a possible limitation to the NS metric given misclassification among individuals with regular substance use involvement and drug overdose history. The inability of the NS metric to detect use of non-opioid substances is not surprising and may not be problematic in that community pharmacy may not be an optimal venue to screen for and intervene with non-opioid substance misuse. Alternatively, this shortcoming of the NS metric may suggest a need for pharmacists to specifically screen patients for other substance use as a supplement to the PDMP. The possible heightened overdose risk within this subpopulation is concerning. It is important for pharmacists to monitor other metrics for detecting unintentional fatal opioid overdose risk (e.g. overdose risk metrics developed Appriss Health). An additional possible avenue for detection of risk could be to explore with data mining or machine learning approaches additional variables available within the PDMP platform or within pharmacy records that could signal opioid-related risk not detected by the NS metric—and devise/test methods for implementation into screening workflow. Nevertheless, specifically examining the true low-risk categorized participants vs. the total low-risk categorized group (i.e. sensitivity) showed a rate of 73.4%. Previous studies of widely used measures of opioid medication misuse have generally similar rates for their identified risk cutoffs. For instance, for the Current Opioid Misuse Measure, true misuse risk/total identified as misusing was 77% (Butler et al., 2007). For the Tobacco, Alcohol, Prescription Medication, and Other Substance Use (TAPS) Tool, true problem use/total identified problem use identified was 71%, and true high-risk use/total identified high-risk use was 48% (McNeely et al., 2016b). For the Prescription Opioid Misuse Measure, true possible misuse risk/total identified possible misuse risk was 82% (Knisely et al., 2008). These facts, in conjunction with the fair level of discrimination (≥0.70) and ability to identify elevated risk and need for confirmatory screening, suggest the NS metric may have a role as a low-burden universal initial screening for opioid medication risk in community pharmacy.

4.1.1. Limitations

While this study possesses many strengths, results herein should be considered in relation to its limitations. Except for the NS metric that is based on opioid dispensation data, other assessments herein were self-reported. Future studies seeking to increase objectivity may choose to collect health record-based or biological samples. Nevertheless, statistical analyses presented herein demonstrated expected relationships between self-reported data and the NS metric, which indicate the validity of participant responses. In addition, given the frequency of concomitant medication use among those misusing opioid medications (Ferries et al., 2017), capturing and exploring use of other medications may add insight into the behavioral profiles of patients identified as at risk in this study. An additional limitation is, despite its robust size, the sample was somewhat homogenous in terms of racial/ethnic distribution (i.e., comprised mostly of White participants, typical of the broader demographics of prescription opioid use (Friedman et al., 2019; Muench et al., 2020) as well as had a larger proportion of females vs. males. Next steps in validation must work to examine racial/ethnic and sex performance differences. Furthermore, while an important feature of this study was it recruited patients from geographically diverse settings, analyses were not conducted by rural vs. urban settings outside of the Midwest, and findings may only be applicable to states/regions wherein data were collected. Next steps should examine rural/urban performance differences in other areas of the US given disparities often noted for the rural impact of the opioid epidemic (Monnat and Rigg, 2016; Thomas et al., 2019). Additionally, PDMP programs do not capture methadone for opioid use disorder treatment given this medication is dispensed through specialty treatment programs. Only methadone for pain management is captured, which associated MMEs are included in the NS metric risk calculation for patients. Future research should actively seek to capture and understand how methadone treatment impacts the risk/benefit profile of community pharmacy patients dispensed opioid medications for pain. Finally, we acknowledge detected rates of substance use among our population, such as street opioids, may be higher herein than the general population (SAMHSA., 2020), which may be explained partially by the regular opioid medication use among the sampled population (NIDA, 2018).

5.1. Conclusion

This study demonstrated the NS metric may serve as a useful broad-based universal screen for risky opioid medication use among community pharmacy patients. Further research should seek to implement the identified threshold scores for the NS metric within a clinical decision support tool with PDMP platforms. Such a platform could include additional screening and guidance on how to provide brief intervention, warm handoff, or naloxone dispensation/training. Such steps in large-scale screening and intervention for risky opioid medication use may stand to make important advancements for protecting patient health and addressing the opioid epidemic.