Validation Study of a Predictive Algorithm to Evaluate Opioid Use Disorder in a Primary Care Setting

Abstract

Background:

Opioid abuse in chronic pain patients is a major public health issue. Primary care providers are frequently the first to prescribe opioids to patients suffering from pain, yet do not always have the time or resources to adequately evaluate the risk of opioid use disorder (OUD).

Purpose:

This study seeks to determine the predictability of aberrant behavior to opioids using a comprehensive scoring algorithm (“profile”) incorporating phenotypic and, more uniquely, genotypic risk factors.

Methods and Results:

In a validation study with 452 participants diagnosed with OUD and 1237 controls, the algorithm successfully categorized patients at high and moderate risk of OUD with 91.8% sensitivity. Regardless of changes in the prevalence of OUD, sensitivity of the algorithm remained >90%.

Conclusion:

The algorithm correctly stratifies primary care patients into low-, moderate-, and high-risk categories to appropriately identify patients in need for additional guidance, monitoring, or treatment changes.

Introduction

Chronic pain is a major public health issue, with nearly 11.2% of Americans affected.¹ Many patients suffering from chronic pain are prescribed opioids, resulting in a dramatic increase in the numbers of opioids prescribed. In a 10-year span, from 1997 to 2007, there was a 402% increase in the average per person sales of opioids and greater than 200% increases in retail sales of morphine, methadone, fentanyl, oxycodone, hydrocodone, and hydromorphone.² Furthermore, the most commonly cited reason for initial opioid use in those with opioid use disorder (OUD) is chronic pain.¹

The burden placed on physicians to balance treating pain with avoiding aberrant use of opioids is great. In fact, the number one source of opioids for nonmedical opioid misusers is a doctor’s prescription.³ Primary care doctors are especially under scrutiny, as 44.5% of the over 259 million opioid prescriptions⁴ prescribed in the United States are prescribed by primary care groups, including family practice physicians, internists, and general practitioners.^5-7 Although rates of lifetime nonmedical prescription OUD, as defined by the Diagnostic and Statistical Manual of Mental Disorders (DSM)-5 , have skyrocketed from 1.4% in 2001 to 2002 to 2.1% in 2012 to 2013,⁸ the prevalence of OUD in primary care patients with noncancer-related chronic pain is shockingly high, at 26%.⁹

Not surprisingly, primary care physicians reported experiencing stress over prescribing opioids and felt they lacked sufficient training to evaluate and manage patients needing opioid therapy.¹⁰ The Centers for Disease Control (CDC) released the CDC Guideline for Prescribing Opioids for Chronic Pain in 2016⁹ in order to clarify when and to whom physicians should prescribe opioids for chronic pain. The 3 main foci of the guidelines are (1) determining “when to initiate or continue pain management with opioids”; (2) managing “opioid selection, dosage, duration, and discontinuation”; and (3) “assessing risk and addressing harms of opioid use.”⁹ The question, then, is how to determine those patients most at risk of OUD.

Screening tools, such as Screener and Opioid Assessment for Patients with Pain–Revised (SOAPP-R),¹¹ Opioid Risk Tool (ORT),¹² and Brief Risk Interview,^13-15 are available to physicians to gauge potential misuse of opioids by patients. These tools are fairly straightforward to administer but are all plagued with the issues of self-report. The profile presented herein is a unique, commercially available tool that combines known genetic risk factors with phenotypic risk factors in a proprietary algorithm to stratify patients into low, moderate, and high risk of OUD.¹⁶ This tool is based, in part, on information that cannot be manipulated, and several studies have demonstrated the high degree of accuracy with which the profile predicts OUD: receiver operator characteristic (ROC) area under the curve (AUC) measurements ranging from 0.757 to upward of 0.967.¹⁷ Moreover, a previous study demonstrated that, although there was a significant positive correlation among the ORT, SOAPP-R, and profile, the profile detected OUD risk determination with the highest specificity. Furthermore, previous work^18,19 evaluated how physicians used the results of the profile to guide treatment decisions and found that physicians found the most utilization in actions that achieved the second aim of the CDC guidelines: managing “opioid selection, dosage, duration, and discontinuation.”²⁰

The addition of genetic information into a screening tool to predict the risk of OUD builds upon previous work that demonstrates the genetic contribution to substance use disorder. There have been a number of studies examining the association of single-nucleotide polymorphisms (SNPs) in the serotonergic, endorphinergic, GABAergic, and dopaminergic pathways with OUD and other addictive behaviors.^21-25 Furthermore, additional evidence for a genetic contribution to substance use disorder has been supplied through several studies in twins,^26-28 with Tsuang et al concluding that 44% of the variance in opioid misuse is due to genetics.²⁸ This validates our use of neural reward-associated SNPs in this profile screening tool.

This study is the first to examine the clinical validity of the profile in an exclusively primary care setting. The purpose of this study was to establish validated predictive accuracy and to provide additional evidence to support the performance characteristics of the algorithm in a primary care setting.

Materials and Methods

Study Population

This multicenter, observational study (protocols 1JAN15-20CR, 1JAN15-14CR) was reviewed, approved, and overseen by Solutions institutional review board (IRB), an IRB licensed by the US Department of Health and Human Services, Office for Human Research Protections. The investigation was carried out in accordance with the latest version of the Declaration of Helsinki. All participants signed informed consent forms prior to data collection.

Patients were recruited at several primary care clinics across the United States. Of these clinics, 16 enrolled at least 1 patient diagnosed with OUD, in addition to patients who were not diagnosed with OUD. All OUD case patients were Caucasian; however, those without diagnosed OUD were not all Caucasian. In order to control for race as a confounding variable in the study, controls were selected from the group of patients without diagnosed OUD such that they were also Caucasians enrolled at the same clinics. One of the risk factors in the profile algorithm is having an age between 16 and 45. To remove age as a confounder in downstream analyses, age distribution was also matched between OUD cases and controls (Table 1). The ratio of cases to controls was maintained among each clinic. In total, the study population consisted of 452 patients diagnosed with OUD and 1237 controls (Table 1). All enrolled patients were seen by a physician at a clinical research site and received medically necessary testing, as ordered by their physician. Written consent was obtained from all participants. Patients with OUD were identified solely by the International Statistical Classification of Diseases and Related Health Problems, Ninth Revision (ICD) series code 304 and ICD-10 series code F11.20. Opioid use disorder was independently diagnosed by each patient’s medical professional at their respective clinics. In addition to diagnostic coding, inclusion criteria for patients with OUD involved confirmation of present experience of chronic noncancer pain, consumption of opioid medication as part of a pain management plan, and fluency in English. In this study, OUD is the diagnostic term for opioid substance use disorder, rather than the physiological state of opioid dependence alone. The control group included patients whose medical records showed they have not been diagnosed with OUD, have not had nondependent opioid misuse (ICD-9 series code 305.5 and ICD-10 series code F11.10), and/or poisoning by opioids (ICD-9 series code 965.0 and ICD-10 series code T40.2).

Table 1.

Patient Demographics.^a

Population	N, Total	n, Female (%)	n, Male (%)	Mean Age	% Age ≤ 45 years
OUD	452	167 (37%)	285 (63%)	41	66%
Control	1237	686 (55%)	551 (45%)	45	66%

Abbreviations: OUD, opioid use disorder

^aIn total, 1689 patients were enrolled in the study. All patients were Caucasian. There were no biases due to age, clinic site, or ethnicity.

Data Collection

Each patient’s medical history was reviewed to determine eligibility based on inclusion and exclusion criteria. After enrollment, demographic information was obtained from patient questionnaires and medical records.

Two buccal swab specimens were obtained from each patient and transported to Proove Medical Laboratories (Irvine, California). Genomic DNA was isolated from buccal swabs obtained from each patient using a proprietary DNA isolation technique and DNA isolation kit (Macherey Nagel GmbH & Co, KG, Duren, Germany), according to the manufacturer’s instructions. Genotyping was performed using predesigned TaqMan assays (Applied Biosystems, Foster City, California). Allele-specific fluorescence signals were distinguished by measuring end point 6-FAM or VIC fluorescence intensities at 508 and 560 nm, respectively, and genotypes were generated using Genotyper software V 1.3 (Applied Biosystems). The DNA elution buffer was used as a negative control, and K562 cell line DNA (Promega Corporation; Madison, Wisconsin) was included in each batch of samples tested as positive control.

Phenotypic information was also collected, including whether patients had a personal history of alcoholism, illegal drug use, and prescription drug misuse; family history of alcoholism, illegal drug use, and prescription drug misuse; mental health disorders and/or depression; and whether they were 16 to 45 years old. This information was collected in a paper questionnaire that asked patients to give yes or no answers to the phenotypic factors indicated above.

Opioid Risk (Profile) Algorithm

A profile score and its associated risk stratification was calculated for each patient. The profile algorithm is a patent-protected, validated measure of opioid risk.^16,17 In short, it combines phenotypic and genotypic information to calculate a risk score that correlates with high-, moderate- or low-risk stratifications of OUD, such that a score of 1 to 11 is associated with low risk, 12 to 23 with moderate risk, and ≥24 with high risk of OUD. Low risk denotes the participant is at low risk of OUD and the clinician may proceed with prescription opioid therapy; moderate risk suggests the clinician should proceed with caution and may consider more routine urine drug testing and possibly limit the duration of opioid therapy; and high risk suggests the physician should prescribe opioids with extreme caution. Plans of treatment in this setting may include implementing frequent urine drug testing, prescribing alternative adjuvant medications in addition to opioids (rather than simply increasing opioid dose), limiting the duration of opioid therapy, titrating the patient off opioid therapy, maintaining vigilant awareness of patient outcomes, and possibly considering medically assisted treatment for detoxification.

The genetic markers used in the algorithm include 11 different SNPs that have been implicated in opioid abuse, misuse, dependence, or addiction (Table 2). Risk alleles for each SNP are weighted more heavily in an additive genetic model, and an overall higher panel score summed across SNPs represents an increased risk of OUD. This approach, which focuses on validated genetic variants, as opposed to comprehensive next-generation sequencing, is the preferred approach of many in the field.⁷³ The phenotypic factors tested (Table 2) are established risk factors for drug dependence or misuse and include age (whether they were 16-45 years old),^74,75 personal history of alcohol misuse,^78,79 personal history of illegal drug misuse,^12,77 personal history of prescription drug misuse,⁶⁷ and personal history of other mental health diseases, including attention deficit disorder,⁸⁰ obsessive compulsive disorder,⁷⁰ bipolar disorder,⁶⁹ depression, and schizophrenia.⁸¹

Table 2.

Profile Panel Markers.

Protein Name	Gene	SNP Marker	Associated Neuropsychiatric Disorders
Catechol-O-methyltransferase	COMT	rs4680	Alcohol abuse29,30 Methamphetamine abuse30 Heroin dependence31,32 Cocaine dependence33,34 Anxiety35,36 Depression37,38
Dopamine β-hydroxylase	DBH	rs1611115	Cocaine addiction39,40 ADHD41 Methamphetamine abuse42 Schizophrenia43
Dopamine D1 receptor	DRD1	rs4532	Depression44 Heroin addiction45 Alcohol dependence45
Ankyrin repeat and kinase domain containing 1/dopamine receptor D2	ANKK1/DRD2	rs1800497	Alcohol and cocaine Dependence46 Heroin abuse32,45,47
Dopamine D4 receptor	DRD4	rs3758653	Anxiety48,49 Heroin abuse50-52
Dopamine rransporter SLC6A3	DAT	rs27072	Methamphetamine addiction53
γ-Aminobutyric acid receptor A, γ2 subunit	GABRG2	rs211014	Alcohol abuse54 Heroin abuse55 Methamphetamine abuse55
Opioid receptor, κ1	OPRK1	rs1051660	Mood disorders56 Alcohol dependence57
Methylenetetrahydrofolate reductase	MTHFR	rs1801133	Bipolar disorder, depression58,59
Opioid receptor, Mu 1	OPRM1	rs1799971	Heroin addiction60 Opioid use disorder61 Substance use disorder61,62 Alcoholism57,63,64
Serotonin receptor 2A	HTR2A	rs7997012	Drug abuse29 Depression65
Phenotypic Traits	Risk Factors
Age	16-45 years old73,66
Personal history	Mental health disorders29,67-69 Depression70-72 Alcoholism73,74 Illicit drug use75,76 Prescription drug abuse77

ADHD, Attention Deficit Hyperactivity Disorder.

Statistical Analyses

A Mann-Whitney U test was used to determine the statistical significance of the difference between the profile scores of the OUD group and the control group. A cross-tab analysis was performed to assess the diagnostic performance of the profile as a comprehensive algorithm for the evaluation of OUD risk. Statistical analyses (implemented in R v3.2.5) included measures of sensitivity: the percentage of patients with OUD correctly identified by profile scores, specificity: the percentage of control patients correctly identified by profile scores, odds ratio: the odds of OUD given a profile score, positive likelihood ratio (PLR): the likelihood of identifying patients with OUD using the profile, and negative likelihood ratio (NLR): the likelihood of identifying controls using the profile. For the risk stratification portion of the analysis, profile scores of patients with OUD and controls were divided into low-, moderate-, and high-risk categories.

Results

Distribution of Profile Scores

The overall distribution of profile scores between patients in the OUD group (n = 452) and the patients in the control group (n =1237) is shown in Figure 1. The mean profile score for the controls was 15.6, with a standard deviation of 5.99, whereas the mean profile score for patients with diagnosed OUD was 25.8, with a standard deviation of 8.25, demonstrating that the profile-predicted risk is increased in patients with diagnosed OUD (P = 1.44 × 10⁻⁹⁷).

Figure 1.

Distribution of profile scores by OUD diagnosis. Patients with OUD had significantly higher profile scores than controls, an average of 25.8 (median = 26) compared to 15.6 (median = 14; P = 1.44 × 10⁻⁹⁷). Profile scores of the entire cohort ranged from 3 to 42. OUD indicates opioid use disorder.

Profile Algorithm Performance

In order to assess performance of the profile for the diagnosis of OUD, sensitivities and specificities were calculated across all possible profile scores. The AUC of the ROC curve provides information about the accuracy of the test, where an AUC of 1 is equal to 100% accuracy and an AUC of 0.5 is equal to random chance. The AUC of the profile was 0.832 (95% confidence interval: 0.808-0.857), indicating that the profile accurately identified patients in this cohort greater than 83% of the time (Figure 2). The sensitivity of the profile score increased as OUD risk increased, with a sensitivity of 91.8% at the moderate risk profile score cutoff (12 and above). The specificity of the profile was 88.4% at the high risk profile score cutoff (24 and above). Moreover, when evaluating profile performance at different prevalence rates of OUD—9% (prevalence of OUD in the general population), 27% (prevalence of OUD in the cohort), and 50% (a balanced prevalence rate)—the profile algorithm performs equally well (Figure 3).

Figure 2.

The ROC curve for profile-predicted OUD diagnosis. A profile score of ≥12, which corresponds to moderate of OUD, has a sensitivity of 91.8% and specificity of 26.7%. This cutoff is used for ruling out patients unlikely to exhibit aberrant behavior with opioids. At a profile score of ≥24, which corresponds to high risk, the sensitivity decreases to 65.5% and the specificity increases to 88.4%. This cutoff is used for ruling in patients for conservative treatment protocols and more regimented monitoring while on opioids. The AUC of the ROC curve is 0.832 (95% confidence interval: 0.808-0.857), indicating the profile algorithm is able to correctly diagnose OUD >83% of the time. Sensitivities and specificities for all profile scores are shown in Supplemental Table 1. AUC indicates area under the curve; ROC, receiver operating characteristic; OUD, opioid use disorder.

Figure 3.

Performance of profile across different prevalence rates of OUD. The profile performs equally well across different prevalence rates: 8% (population), 27% (cohort), and 50% (balanced). The consequences of increasing the profile score threshold to correctly identify patients with OUD are decreased sensitivity and increased specificity. Conversely, decreasing the profile score threshold to correctly identify patients with OUD results in increased sensitivity and decreased specificity. The incorporation of 2 thresholds (ie, 3 categories of risk) allows for clinicians to use the profile for both the higher sensitivity of ruling out of low-risk patients for continued opioid management with standard precautions and the more specific ruling in of high-risk patients for vigilant monitoring and/or alternative therapy intervention strategies. OUD indicates opioid use disorder.

Odds of Diagnosed OUD

When compared to controls, patients with OUD identified by the profile algorithm to be at moderate risk had an average of 9.48 increased odds of OUD (ranging from 4.08-12.4), whereas those in the high-risk category had an average of 18.2 increased odds of OUD (ranging from 14.5-21.9; Figure 4).

Figure 4.

Odds ratios of OUD in each profile risk category. Patients at moderate risk (profile score 12 to 23) had on average 9.48 increased odds of diagnosed OUD. Patients at high risk (profile score ≥24) had on average 18.2 increased odds of diagnosed OUD. OUD indicates opioid use disorder.

Likelihood of OUD

The PLR demonstrates the likelihood that a person with OUD would receive a positive profile test result, while the opposite is true for the NLR, which indicates the likelihood that a person with the condition would receive a negative test result. These ratios are based on the sensitivity and specificity of the test and are unaffected by changes in prevalence. Figure 5 shows small (<2) increases in the likelihood of OUD for profile scores of 9 to 15, moderate (2-10) increases for profile scores of 16 to 26, and large (>10) increases for profile scores of 27 or higher. For all profile scores below 42, the NLR is below 1 (Figure 5).

Figure 5.

The PLR and NLR of OUD across different prevalence rates. As the profile score increases, so does the PLR of OUD diagnosis, that is, the likelihood of correct OUD diagnosis given a minimum profile score. The NLR is the likelihood of correct diagnosis of no OUD given a minimum profile score. For all profile scores below 42, the NLR is below 1. The PLR and NLR are comparable across different prevalence rates, 8% (population), 27% (cohort), and 50% (balanced). NLR indicates negative likelihood ratio; OUD, opioid use disorder; PLR, positive likelihood ratio.

Discussion

Primary care physicians are on the frontline of the prescription opioid epidemic. Although the CDC guidelines for prescribing opioids recommend implementing a screening method to identify those most at risk of aberrant use of opioids,⁹ tools available to physicians today are often inadequate to reliably curb the substantial escalation in prescription opioid misuse. The ROC curve of the profile demonstrates that the profile is a highly reliable model for OUD screening: The profile is able to correctly identify those at risk of OUD greater than 83% of the time. Furthermore, the odds of OUD increases with higher profile-identified risk. Those whose profile scores were in the moderate-risk category had an average 9.48 increased odds of OUD than controls and those with profile scores in the high-risk category have an average 18.2 increased odds of OUD than controls. These statistics illustrate the ability of the profile to accurately stratify patients into low, moderate, and high risk of OUD.

The ORT and SOAPP-R detect OUD with much lower sensitivities and specificities than the profile. The specificity of the SOAPP-R to determine aberrant drug-related behavior is 52.0% and the sensitivity is 80%,¹¹ while the ORT reports a c statistic, which is a single measure of specificity and sensitivity of 0.82 for men and 0.85 for women.¹² In contrast, the sensitivity of the profile to detect those at moderate and high risk of OUD is 91.8% and the specificity of the profile to detect those at high risk is 88.4%. This confirms previous work that demonstrated that the profile predicted the risk of OUD with greater sensitivity and specificity than the ORT or SOAPP-R.

Interestingly, those patients in the control group with the highest profile scores also reported previous illicit drug use, suggesting that their opioid use may have been misclassified by their physician or their physician was unaware of the drug use. This indicates that these patients may, in fact, have had OUD but were not diagnosed properly or the ICD code was not reported in their medical record.

Given the burden faced by primary care providers to provide effective pain management while also mitigating the dangers of opioid abuse, misuse, and addiction, it is imperative that they have access to tools that provide accurate information regarding a patient’s risk of OUD. The profile performs better than other currently available screening tools and allows clinicians the ability to decrease the astronomical prevalence of OUD in primary care by identifying those most at risk prior to prescribing opioids.

The limitations of this study include the wide age range of study participants and the reliance on ICD code for diagnosis of OUD, as well as any confounding due to geographic distribution of the participating clinics. There are potential barriers to the adoption of this technology, including financial and practice barriers for physicians, and social stigma barriers for patients. These barriers must be addressed, perhaps through physician training and patient education. Future studies will include additional objective measures of drug use, including urine drug screening.

Conclusion

The profile provides primary care physicians with a highly accurate precision medicine approach to identifying patients most at risk of developing OUD. This identification allows clinicians to safely treat vulnerable patients while reducing the burden of OUD on patients, their families, and the American health-care system. This study is the first to evaluate the validity of this algorithm in a primary care setting.

Author Biographies

Dr. Maneesh Sharma, MD is a Johns Hopkins University trained anesthesiologist and Interventional Pain Medicine specialist. He treats various orthopedic, neurologic and gastrointestinal injuries, diseases and degenerative conditions that cause chronic pain using surgical, medical, regenerative and integrative approaches.

Chee Lee is a skilled data scientist with a background in statistics and genomics. Her expertise lies in integrating population-level statistics, such as demographic, phenotypic, and electronic health records, with complex genomic data, including associations with genetic variants, gene expression, and epigenetics. Chee Lee received her PhD in Bioinformatics from the University of Michigan, and has almost a decade of experience conducting scientific research.

Svetlana Kantorovich is a classically-trained neuroscientist who has conducted research in pain, epilepsy, cognitive dysfunction, and Parkinson's disease. She earned her Bachelor’s degree from Washington University in St. Louis, her Ph.D. in Neuroscience from the University of Florida, and completed her postdoctoral fellow training in neurobiology at the University of California, Irvine. At her current role at Proove Biosciences, Svetlana leads efforts related to laboratory methods and platform development, clinical trial design and operations, product development, and clinical affairs.

Dr. Maria Tedtaotao graduated from the University of Hawaii Burns School of Medicine in 1989. She works in Lynn Haven, FL and specializes in Family Medicine. Dr. Tedtaotao is affiliated with Gulf Coast Regional Medical Center.

Dr. Gregory A. Smith is the former Director of Pain Management and Assistant Clinical Professor at Harbor UCLA and UCLA. He formed the Comprehensive Pain Relief Group Inc. in 2001 and the GS Medical Center Inc. in 2004. In 2005 Dr. Smith created the Nutritional, Emotional, Social, and Physical (NESP) Program to simultaneously treat prescription addiction and chronic pain. He has authored a variety of scientific studies and has written 5 medical books. Currently he is the President and Medical Director of Dr. Gregory A. Smith, M.D. Inc. and founder and CEO of Red Pill Medical, Inc.

Ashley Brenton received her Bachelor’s Degree in Public Health Studies from the Johns Hopkins University, her PhD from University of California Davis in Entomology with a Designated Emphasis in Vector-Borne Diseases, and completed a viral pathogenesis fellowship at the Scripps Research Institute. Dr. Brenton has dedicated her career to the application of genomics to decrease major public health disease burden. She is currently the Director of Research and Development for Proove Biosciences, Inc.