Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2022 Apr 1.
Published in final edited form as: Drug Alcohol Depend. 2021 Feb 13;221:108583. doi: 10.1016/j.drugalcdep.2021.108583

Identifying individuals with opioid use disorder: Validity of International Classification of Diseases diagnostic codes for opioid use, dependence and abuse

Pooja Lagisetty a,b,*, Claire Garpestad c, Angela Larkin b, Colin Macleod a, Derek Antoku a, Stephanie Slat a, Jennifer Thomas a, Victoria Powell d, Amy SB Bohnert b,e, Lewei A Lin b,f
PMCID: PMC8409339  NIHMSID: NIHMS1681711  PMID: 33662670

Abstract

Background:

Policy evaluations and health system interventions often utilize International Classification of Diseases (ICD) codes of opioid use, dependence, and abuse to identify individuals with opioid use disorder (OUD) and assess receipt of evidence-based treatments. However, ICD codes may not map directly onto the Diagnostic and Statistical Manual of Mental Disorder (DSM-5) OUD criteria. This study investigates the positive predictive value of ICD codes in identifying patients with OUD.

Methods:

We conducted a clinical chart review on a national sample of 520 Veterans assigned ICD-9 or ICD-10 codes for opioid use, dependence, or abuse from 2012 to 2017. We extracted evidence of DSM-5 OUD criteria and opioid misuse from clinical documentation in the month preceding and three months following initial ICD code listing, and categorized patients into: 1) high likelihood of OUD, 2) limited aberrant opioid use, 3) prescribed opioid use without evidence of aberrant use, and 4) insufficient information. Positive predictive value was calculated as the percentage of individuals with these ICD codes meeting high likelihood of OUD criteria upon chart review.

Results:

Only 57.7 % of patients were categorized as high likelihood of OUD; 16.5 % were categorized as limited aberrant opioid use, 18.9 % prescribed opioid use without evidence of aberrant use, and 6.9 % insufficient information.

Conclusions:

Patients assigned ICD codes for opioid use, dependence, or abuse often lack documentation of meeting OUD criteria. Many receive long-term opioid therapy for chronic pain without evidence of misuse. Robust methods of identifying individuals with OUD are crucial to improving access to clinically appropriate treatment.

Keywords: Opioid use disorder, Opioid dependence, ICD codes

1. Introduction

In 2018, the National Survey on Drug Use and Health estimated 2 million people aged 12 or older in the United States met criteria for opioid use disorder (OUD), including both heroin use disorder and pain reliever use disorder (Substance Abuse and Mental Health Services Administration, 2019). Among Veterans receiving healthcare from the Veterans Health Administration (VHA), specifically, there has been a sharp rise in the reported number diagnosed with OUD, from 25,031 in 2003, to 69,142 in 2017 (Wyse et al., 2018). Much of the data regarding the prevalence of OUD, related hospitalizations, and treatment access rely upon electronic medical record (EMR) data. Metrics are based on the International Classification of Diseases (ICD) diagnostic codes assigned by clinicians or healthcare administrators (Barocas et al., 2018; Madras et al., 2020; Nosyk et al., 2013; Peterson et al., 2018; Weiss et al., 2020). It remains unknown whether ICD codes are accurate in both identifying individuals with OUD and assessing treatment eligibility and access.

Discrepancies between coding descriptions and diagnostic criteria for OUD make ICD coding for OUD complicated (Roland et al., 2016; Watson et al., 2020). The Diagnostic and Statistical Manual of Mental Disorders (DSM) is viewed as the “gold standard” for diagnosing OUD via the Structured Clinical Interview (First et al., 2016; Substance Abuse and Mental Health Services Administration, 2016). In the DSM-5, diagnostic language shifted from opioid “abuse” and “dependence” to “opioid use disorder” across a spectrum of severity (Peer et al., 2013; Substance Abuse and Mental Health Services Administration, 2016). It has been suggested, however, that DSM-5 criteria do not always fully convey the same character, range, and severity of opioid-related problems that may be documented in detailed clinical notes (Boscarino et al., 2015; Von Korff et al., 2010). Furthermore, ICD codes continue using “use,” “abuse,” and “dependence” to record diagnoses in patient charts based on outdated DSM-IV terminology (Howell et al., 2020). Retrospective analyses show coding labels are applied inconsistently to both individuals with moderate or severe OUD and those on long-term opioid prescriptions taken as directed, potentially because of ambiguity between psychological and physiological dependence (Watson et al., 2020). Although overlap in care exists, patients with chronic pain without OUD require, and are eligible for, different treatments than those with DSM-defined OUD (Hser et al., 2017; Wilson-Poe and Morón, 2018; Young et al., 2019). Prior work suggests that clearer guidelines for OUD-related coding are necessary to accurately assess and treat those with chronic pain and opioid misuse (Hser et al., 2017; Young et al., 2019).

Very limited prior research has investigated ICD code validity in predicting OUD. Howell and colleagues recently investigated the accuracy of opioid dependence and abuse ICD codes as OUD diagnoses, finding in their sample of 90 veteran patient charts that 29 % of OUD diagnoses were “likely inaccurate” (Howell et al., 2020). In this study, we expand on such work to examine the positive predictive value of ICD codes in identifying individuals with OUD across a larger national sample of U.S. Veterans. Specifically, we conducted manual review of EMR notes, extracting a wide range of signs and symptoms of OUD, both those directly informed by or indirectly associated with DSM-5 criteria. Through this chart review, we categorize how many individuals with an ICD code for opioid use, abuse and dependence had 1) high likelihood of OUD, 2) limited aberrant opioid use but did not meet enough criteria for OUD, 3) prescribed opioid use with no evidence of aberrant use, or 4) insufficient information in their chart to make an OUD diagnosis.

2. Methods

2.1. Study population

We selected a national sample of VHA patients with ICD-9 or ICD-10 codes for OUD (specifically opioid use, dependence, or abuse) between fiscal years 2012–2017 (Appendix A in Supplementary material) (Lagisetty et al., 2019; Weiss et al., 2020). We then applied three inclusion criteria. First, patients had to have two visits coded for OUD of the same type (e.g. two visits for opioid dependence), and the codes must be from different days within the study window. Use of two codes ensured a diagnosis was not erroneously entered. Second, patients could have no other OUD diagnosis code in the two years preceding the initial code to identify incident diagnosis. We hypothesized providers would be more likely to list reasons for diagnosis on the date of the initial code than in instances where the diagnosis was long-standing. Third, to ensure a minimum level of available information, we limited the sample to patients who had at least one VHA medical visit per year in the two years preceding diagnosis. The Ann Arbor VA Office of Research and Development Institutional Review Board approved this study (IRB # 2018–1078).

2.2. Medical record abstraction protocol

EMR notes were extracted for one month prior to and three months following the initial diagnosis date. Notes were filtered by keywords broadly encompassing pain, medication, and substance use related terms (Appendix B in Supplementary material). A chart abstraction tool was developed by the lead investigators, an Internal Medicine physician boarded in Addiction Medicine (PL) and an Addiction Psychiatrist (LL) (Appendix C in Supplementary material). This tool captured opioid-related signs and symptoms, past and current substance use history, past and current use of substance use disorder or pain treatments, care plan following diagnosis, and barriers and facilitators to engagement in the care plan. We also documented significant changes in patient diagnosis or treatment over the abstraction period.

This abstraction tool was piloted with 15 charts and revised iteratively until consensus was reached between the lead investigators. The lead investigators conducted the initial abstractions to set chart interpretation standards and identify issues with the abstraction tool. After finalizing the tool, we created a protocol manual of terms and instructions to maintain screening consistency. Seven individuals were trained to be screeners by abstracting five charts that had already been abstracted by the lead investigators. A lead investigator then reviewed these abstractions to address questions and inconsistencies. Following training, screeners proceeded with abstractions independently and brought questions to weekly meetings between screeners and lead investigators. Discrepancies were discussed until consensus was reached. Additionally, 10 % of abstractions were double-coded by the lead investigators to ensure coding consistency.

2.3. Diagnosis categories

Patients were placed into one of four categories based on information in their EMR available to the diagnosing clinician: 1) high likelihood of OUD, 2) limited aberrant opioid use but did not meet enough criteria for OUD, 3) prescribed opioid use with no evidence of aberrant use, or 4) insufficient information in their chart to make an OUD diagnosis (Table 1). We created these categories as each often requires different clinical treatments. For example, those with a high likelihood of OUD may warrant treatment medications for OUD (MOUD), such as buprenorphine, methadone, or extended-release naltrexone (Madras et al., 2020; Veterans Health Administration, 2019). However, an individual with prescribed opioid use but no evidence of aberrant use may not. We included the category of limited aberrant opioid use because these patients may meet criteria for mild OUD, and, although not common practice, some experts advocate using buprenorphine for these patients for pain management (Chen et al., 2014; Manhapra et al., 2018; Webster et al., 2020).

Table 1.

Category Definitions of OUD Related Signs and Symptoms.

1. Likely Opioid Use Disorder 2. Prescribed Opioids with limited Aberrant Use (<2 criteria) 3. Prescribed Opioids Without Aberrant Use 4. Insufficient Information for OUD Diagnosis

  • Three or More Misuse Criteria from Column 2

    Or

  • Notes Indicate Historical Diagnosis in Remission

  • Provider Documents that Patient Meets DSM-5 Criteriaa

  • Patient Seeking Care for OUD

  • Requesting Specific Treatment for OUD (e.g. Buprenorphine)

  • IV or Nasal Use of Opioids

  • Infection (e.g. Abscess) Secondary to Opioid Usec

  • Notes Indicate Opioid Misuse

  • Patient Using Opioids to Get High or Reason Other Than Painc

  • Family Reports Opioid Use to Clinician

  • Documented Loss of Control/Cravingsc

  • Physician Reports Drug Seeking Behavior

  • Patient Obtained Prescription Opioids from Non-Medical Sourcec

  • Opioid Overdose

  • Treatment Contract Violation

  • Patient Requesting Early Refills

  • Patient Taking More Opioids Than Prescribedc

  • Provider Comments on Aberrant Urine Drug Screen (UDS)

  • Obtained Opioids from Multiple Providers Surreptitiously

  • Diversion

  • Hospitalization or ED Visit Related to Opioids/Pain (Withdrawal, Detox)c

  • Other AE 2° Opioids-Symptoms = Withdrawalc

  • Patient on Prescription Opioids for Pain

  • Medical Visit Related to Pain Control or Opioids

  • Constipation, Nausea, Vomiting Secondary to Opioid Use

  • OUD listed in chart but not described

  • No mention of opioid use in chart

  • Non-opioid polysubstance use (PSU)b
Open in a new tab
a

Clinician noted patient met DSM-5 criteria in patient chart.

b

Using two or more substances.

c

Criteria that map directly or closely to DSM-5 OUD criteria.

We created a robust list of criteria to assign individuals to each category (Table 1). This list included DSM-5 criteria that could be ascertained via EMR review, such as taking more opioids than prescribed, noted loss of control/craving, and clinician documentation of the patient meeting DSM-5 criteria. Our list also included factors that do not directly map to DSM-5 criteria yet are associated with other DSM-5 criteria and, therefore, with OUD diagnosis. These criteria included seeking care for OUD, opioid overdose, intravenous or intranasal opioid use, or treatment contract violations. We included criteria beyond those outlined in DSM-5 in order to better capture the signs and symptoms frequently explained in patient clinical notes (Boscarino et al., 2015; Von Korff et al., 2010). We also captured unanticipated opioid-related signs and symptoms via open text fields.

Patients were categorized based on this checklist in the month leading up to the initial diagnosis date in the following ways. First, we posited that individuals who met multiple criteria for aberrant opioid use had higher likelihood of having an OUD compared to individuals with only one or two criteria. Therefore, if an individual met three or more criteria in the limited aberrant opioid use category, they were categorized as having a high likelihood of OUD. For example, if a patient had loss of control or cravings, obtained opioids from non-prescribed sources, and requested early refills, they were categorized as having a high likelihood of OUD. Second, if patients met the criteria for more than one category, they were placed in the most severe category for which they qualified. Third, lead investigators reviewed all unanticipated opioid-related signs and symptoms captured to determine if a patient belonged in a different category based on the additional information. Lastly, given the association between polysubstance use and OUD treatment (Lin et al., 2020), a non-opioid polysubstance use variable was created from a combination of 1) the presence of two or more current non-opioid substances documented or 2) if “polysubstance use” was specifically noted by clinicians in the chart.

2.4. Demographics and comorbidities

Administrative data was used to collect patient demographics, including age, sex, race and ethnicity, and medical and mental health comorbidities. Comorbidities were included if the patient had any of the following in the 12 months prior to diagnosis: depression, serious mental illness (e.g., schizophrenia, bipolar disorder), post-traumatic stress disorder (PTSD), anxiety disorder, non-opioid substance use disorder (SUD), acute pain, chronic pain, and cancer (Appendix D in Supplementary material). Chart abstraction captured prescribed and non-prescribed substance use as well as previous MOUD treatments on or before date of diagnosis. MOUD treatments were distinguished first by whether they were used medically or illicitly and then by whether those used medically were for OUD or something else (e.g. buprenorphine or methadone for pain, or naltrexone for AUD).

2.5. Analysis

Descriptive analysis included frequencies and crosstabs of the final sample as well as binary logistic regression. Positive predictive values with 95 % Wilson binomial confidence intervals were calculated as the proportion of individuals who met criteria for one or more OUD likelihood categories (Wilson, 1927). Data management was performed using Microsoft Access 2016 and R version 3.6.0 and analysis was performed using R version 3.6.0.

3. Results

We identified 171,620 unique individuals in the national VHA data set with an ICD-9 or ICD-10 code for OUD in their medical records between fiscal years 2012–2017 (Fig. 1). Of these individuals, 37,127 met study inclusion criteria for an incident OUD diagnosis. Of those that met study criteria, we randomly sampled 520 charts for medical record review. Sampled individuals were 91.4 % male, 77.1 % White, 16.7 % Black and 6.2 % other/unknown race, and were 51.6 years old on average at time of diagnosis (Table 2).

Fig. 1.

Fig. 1.

Open in a new tab

Flow Diagram of Patient Sample Selection.

Table 2.

Sample Demographics (n = 520).

Variable Total
Age at Time of Diagnosis (mean) 51.56
Age at Time of Diagnosis (age categories)
<35 100 (19.23 %)
35–55 172 (33.08 %)
56–75 234 (45.00 %)
≥76 14 (2.69 %)
Sex
Female 45 (8.65 %)
Male 475 (91.35 %)
Race
White 401 (77.12 %)
Black 87 (16.73 %)
Other/ Unknown 32 (6.15 %)
Ethnicity
Hispanic 20 (3.85 %)
Not Hispanic 500 (96.15 %)
Diagnoses 12 Months Prior to Diagnosis Date
Depression 270 (51.92 %)
SMI 88 (16.92 %)
PTSD 186 (35.77 %)
Anxiety 164 (31.54 %)
SUD 284 (54.62 %)
Acute Pain 115 (22.12 %)
Chronic Pain 124 (23.85 %)
Cancer 69 (13.27 %)
Rx Substance Use History (On or Any Time Prior to Diagnosis Date)
Opioids 356 (68.46 %)
Benzodiazepines 123 (23.65 %)
Non-Rx Substance Use History (On or Any Time Prior to Diagnosis Date)
Opioids 225 (43.27 %)
Amphetamines 63 (12.12 %)
Benzodiazepines 48 (9.23 %)
Cocaine 169 (32.50 %)
Alcohol 295 (56.73 %)
Cannabinoids 131 (25.19 %)
Other 19 (3.65 %)
MOUD (On or Any Time Prior to Diagnosis Date)
Buprenorphine 31 (5.96 %)
Methadone 29 (5.58 %)
Naltrexone 3 (0.58 %)
Open in a new tab

The abstraction found that on or before the date of diagnosis, 356 (68.5 %) patients in the sample had received opioids that had been prescribed to them, and 123 (23.65 %) had received benzodiazepines that had been prescribed to them. As for non-prescribed substances, 225 (43.3 %) had used opioids, 63 (12.1 %) amphetamines, 48 (9.2 %) benzodiazepines, 169 (32.5 %) cocaine, 295 (56.7 %) alcohol, 131 (25.2 %) cannabis or cannabinoids, and 19 (3.7 %) other substances. Regarding previous treatments for OUD the abstraction found 31 (6.0 %) had previously taken buprenorphine for OUD, 29 (5.6 %) methadone, and 3 (0.6 %) naltrexone.

Many patients had another mental health or SUD diagnosis in the preceding year: 88 (16.9 %) with serious mental illness, 270 (51.9 %) with depression, 164 (31.5 %) with anxiety, and 284 (54.6 %) with non-opioid SUD. In addition, 115 (22.1 %) experienced acute pain, 124 (23.9 %) had chronic pain, and 69 (13.3 %) had cancer.

Of the 520 individuals sampled with an incident diagnosis for OUD based on ICD codes, only 57.7 % [95 % confidence interval (CI):53.4–61.9 %] were categorized as having a high likelihood of OUD (Table 3). The remaining 220 individuals fell into the following categories: 16.5 % [CI 13.6–20.0 %] demonstrated limited aberrant opioid use, 18.9 % [CI 15.7–22.4 %] had prescribed opioid use but no evidence of aberrant use, and 6.9 % [CI 5.0–9.4 %] had insufficient information to make an OUD diagnosis.

Table 3.

Positive Predictive Values of Meeting Criteria for Different Opioid-related Diagnosis Categories.

PPV (as % of total sample) [95 % CI]
Classes
OUD 57.69 % 53.40–61.87 %
Aberrant Use 16.54 % 13.59–19.97 %
Prescribed Opioids 18.85 % 15.72–22.43 %
Insufficient Information 6.92 % 5.04–9.44 %
Open in a new tab

We evaluated the most common opioid-related signs and symptoms in each category (Table 4). In the high likelihood of OUD category, 51.7 % of the sample met criteria for three or more signs of aberrant opioid use. Additionally, in 51.3 % of these cases, the Veteran was seeking care for an OUD. Among those categorized as having limited aberrant opioid use, the most common criteria met was having an emergency room or inpatient hospital visit due to pain exacerbation, withdrawal, or detox (27.9 %). Of the 6.9 % of individuals categorized as having insufficient information to make a diagnosis, 50.0 % had no mention of opioid use in their chart at all, and 27.8 % had OUD listed but clinicians had not described patient symptoms in the period proximal to the diagnosis.

Table 4.

Within Class n Values of OUD Related Signs and Symptoms (n = 520).

Related Signs and Symptoms Total (n = 520) Opioid Use Disorder (n = 300) Limited Aberrant Opioid use (n = 86) Prescribed Opioids w/o Aberrant Use (n = 98) Insufficient Info. for Diagnosis (n = 36)
Three or More Limited Aberrant Use Criteria 155 (29.81 %) 155 (51.67 %)
Patient Seeking Care for OUD 154 (29.62 %) 154 (51.33 %)
Requesting Specific Treatment for OUD (e.g. Buprenorphine) 104 (20.00 %) 104 (34.67 %)
IV or Nasal Use of Opioids 103 (19.81 %) 103 (34.33 %)
Notes Indicate Historical OUD Diagnosis in Remission 61 (11.73 %) 61 (20.33 %)
Provider Documents that Patient Meets DSM V Criteria 50 (9.62 %) 50 (16.67 %)
Hospitalization or ED Visit Related to Opioids/Pain (Withdrawal, Detox) 138 (26.54 %) 114 (38.00 %) 24 (27.91 %)
Notes Indicate Opioid Misuse 200 (38.46 %) 178 (59.33 %) 22 (25.58 %)
Patient Obtained Prescription Opioids From Non-Medical Source 99 (19.04 %) 87 (29.00 %) 12 (13.95 %)
Provider Comments on Aberrant Urine Drug Screen (UDS) 43 (8.27 %) 31 (10.33 %) 12 (13.95 %)
Patient Taking More Opioids Than Prescribed 63 (12.12 %) 54 (18.00 %) 9 (10.47 %)
Obtained Opioids From Multiple Providers Surreptitiously 24 (4.62 %) 16 (5.33 %) 8 (9.30 %)
Patient Requesting Early Refills 25 (4.81 %) 18 (6.00 %) 7 (8.14 %)
Treatment Contract Violation 23 (4.42 %) 17 (5.67 %) 6 (6.98 %)
Opioid Overdose 13 (2.50 %) 11 (3.67 %) 2 (2.33 %)
Patient Using Opioids to Get High or Reason Other Than Pain 92 (17.69 %) 91 (30.33 %) 1 (1.16 %)
Documented Loss of Control/Cravings 51 (9.81 %) 50 (16.67 %) 1 (1.16 %)
Physician Reports Drug Seeking Behavior 17 (3.27 %) 16 (5.33 %) 1 (1.16 %)
Diversion 3 (0.58 %) 2 (0.67 %) 1 (1.16 %)
Family Reports Opioid Addiction to Clinician 11 (2.12 %) 11 (3.67 %) 0 (0.00 %)
Patient On Prescription Opioids for Pain 274 (52.69 %) 116 (38.67 %) 61 (70.93 %) 97 (98.98 %)
Medical Visit Related to Pain Control or Opioids 117 (22.50 %) 65 (21.67 %) 21 (24.42 %) 31 (31.63 %)
Other AE 2° opioidsa 125 (24.04 %) 103 (34.33 %) 16 (18.6 %) 6 (6.12 %)
No Mention of Opioid Use in Chart 21 (4.04 %) 1 (0.33 %) 1 (1.16 %) 1 (1.02 %) 18 (50.00 %)
OUD Listed in Chart but Not Described 30 (5.77 %) 5 (1.67 %) 9 (10.47 %) 6 (6.12 %) 10 (27.78 %)
Other 87 (16.73 %) 46 (15.33 %) 17 (19.77 %) 12 (12.24 %) 12 (33.33 %)
Open in a new tab
a

Adverse events secondary to opioids had several sub-criteria which were associated with different categories based on their severity. Additionally, there was an open ‘other’ option which was manually reviewed for any necessary reassignment.

Across the sample, there were multiple cases of non-opioid polysubstance use. The high likelihood of OUD category had the highest proportion of non-opioid polysubstance use at 45.7 % (p = .088). The relative use of prescription opioids for pain decreased as category severity increased, with 97 patients (99.0 %) exhibiting this symptom in the prescribed opioids without aberrancy category, 61 (70.9 %) in the limited aberrant opioid use category, and 116 (38.7 %) in the high likelihood of OUD category (Table 4).

4. Discussion

This study examined the validity of ICD codes in identifying individuals with OUD in a sample of VHA patients. Of those who received a diagnosis of OUD by ICD code, less than 60 % had medical chart documentation indicating that the patient met criteria for OUD based on our manual chart review. This proportion increases slightly, to about 75 %, when including individuals with limited indicators of aberrant opioid use. Notably, the remaining quarter of individuals did not have OUD or aberrant opioid use and were either only on prescribed opioids for pain, or their chart contained extremely limited information about their opioid use.

These findings have many implications for current practice and future research. Much of the current literature on opioid use in patient populations, including prevalence, hospitalizations, and treatment access and utilization, uses national databases extracted from EMR data and insurance claims (Barocas et al., 2018; Clemans-Cope et al., 2019a, 2019b; Jones et al., 2019; Madras et al., 2020; Nosyk et al., 2013; Peterson et al., 2018; Weiss et al., 2020; Wu et al., 2016). At the same time, OUD is substantially underdiagnosed (i.e., low sensitivity) in EMR data when compared to clinical samples who have been diagnosed with validated instruments (Hallgren et al., 2020; Wu et al., 2017). This study highlights that OUD research that uses ICD coding to define patient groups or measure outcomes may suffer from substantial misclassification. The degree to which this misclassification biases the results of prior studies varies by the study objectives and use of ICD-based measures. Policies and treatment guidelines stemming from such evidence may be based on biased or inaccurate information and potentially over-estimating the number of individuals who have a high likelihood of OUD compared to other opioid-related diagnosis when relying upon ICD coding alone (National Academies of Sciences, Engineering, and Medicine, Health and Medicine Division, Board on Health Sciences Policy, 2018; Veterans Health Administration, 2019). In addition, a number of proposed predictive models seeking to identify individuals at risk of developing OUD rely on ICD codes, EMR data, and pharmacy claims (Ciesielski et al., 2016; Cochran et al., 2014; Dufour et al., 2014; Glanz et al., 2018; Rice et al., 2012). Lastly, several genome-wide association studies looking for genetic basis of OUD also rely in part upon identifying individuals from EMR data (Boscarino et al., 2010; Gelernter et al., 2014, 2006; Zhou et al., 2019). Without studies validating the reliability of ICD codes in predicting OUD, assessing the accuracy and applicability of such predictive models and genetic research is difficult.

To our knowledge, a limited number of validation studies have been conducted in this area, many of which focus primarily on opioid overdose. Green and colleagues evaluated opioid poisoning code validity for predicting opioid overdose and found that code-based algorithms had a sensitivity and specificity of 97.2 % and 84.6 %, respectively, for opioid-related overdoses (Green et al., 2019). Green’s study concluded that such codes can accurately be used to monitor trends in opioid overdose and, therefore, help understand risk factors and evaluate mechanisms to reduce overdose occurrence (Green et al., 2019, 2017). In contrast, we find the positive predictive value of ICD codes for opioid use, dependence, and abuse in predicting OUD and opioid misuse to be lower than that of poisoning codes in predicting opioid overdose. Unfortunately, this lower predictive value in OUD diagnosis codes challenges our ability to identify patients earlier in their disease course and possibly prevent or reduce adverse events. A recent study by Howell and colleagues found significant inaccuracy in OUD diagnosis among veterans - 29 % of charts surveyed had an OUD diagnosis that was deemed likely inaccurate (Howell et al., 2020). Our work reinforces these findings with a larger national sample. In addition, rather than relying on a bivariate analysis of accurate versus inaccurate OUD coding, we use a robust classification methodology to further categorize this inaccuracy into four diagnostic categories along the opioid use continuum. By placing patients into these four categories (OUD, aberrant use, prescribed opioid use, insufficient information), this study captures significant information about individuals across the spectrum of disease, including those for whom diagnosis, and thus treatment plan, may be different or more nuanced than an ICD code alone would imply. This highlights the need for more robust methods of categorizing individuals with OUD in order to identify unique treatment needs, facilitate referrals and increase engagement.

By extracting an extensive list of symptoms from patient charts, we were able to identify interesting patterns and valuable insights into this patient population. One of the most common signs or symptoms among the high likelihood of OUD group was “Veteran seeking care for OUD.” Because OUD can be associated with limited insight into degree of opioid misuse, this sample of individuals seeking care for OUD likely vastly undercounts actual patients with OUD, including those who may not be treatment-seeking (Maremmani et al., 2012). Additionally, a significant portion of individuals with ICD codes for OUD did not meet criteria for OUD but, instead, demonstrated limited signs of aberrant opioid use, such as requesting early refills or obtaining prescriptions from non-medical sources (Table 3). This category, which experts advocate classifying as complex persistent dependence (CPD) or potentially “low severity OUD,” creates more nuance in OUD diagnosis and treatment, as patients may qualify for MOUD based on changing guidelines and expert recommendations (Chen et al., 2014; Manhapra et al., 2018; Veterans Health Administration, 2019; Webster et al., 2020). This diagnosis varies based on clinical interpretation of behavior as aberrant by the provider (Young et al., 2019). Evaluating the variability in treatment courses of individuals falling into each of the four categories outlined in this paper is an important next step.

Research into how to best utilize the full breadth of administrative and EMR data beyond ICD codes to classify individuals with OUD is another important future direction. Recent research has utilized various algorithms that employ ICD coding in addition to other variables such as overdose events or opioid-related hospitalizations in order to more accurately identify individuals with OUD (Song, 2017; Wakeman et al., 2020). Future studies should consider investigating such methodologies and other EMR based variables such as comorbidities or medication use that could improve the positive predictive value of ICD coding in identifying individuals with true OUD using electronic medical record data alone.

Limitations of this study include its focus on a VHA sample. Veterans have disproportionately high prescription opioid use and OUD rates (Gordon et al., 2007; Teeters et al., 2017; Wyse et al., 2018) as well as opioid overdose compared to the general population (Bohnert et al., 2011a, 2011b; Lin et al., 2019; Wyse et al., 2018). Therefore, the VHA has made significant efforts to both decrease opioid prescribing while dramatically increasing access to treatment for patients with OUD (Gellad et al., 2017; Hadlandsmyth et al., 2018; Lin et al., 2019; Wyse et al., 2018). Taken together, these sample factors may overestimate the positive predictive value compared to a non-Veteran patient population. We did, however, draw from a national sample, so geographic or regional clinical patterns are unlikely to be a factor as in prior studies (Lund et al., 2019). Other limitations include our inability to quantify the negative predictive value of ICD coding in the setting of likely substantial under-diagnosis of OUD in medical records, as this would have required viewing all charts of a large number of patients without an OUD diagnosis code, and capture whether there was an OUD. We were also unable to quantify the sensitivity and specificity of ICD coding for OUD because chart review is imperfect in routinely describing all DSM-5 criteria and other signs or symptoms of OUD due to variability of clinician documentation and potentially incomplete knowledge or capture of symptoms. In order to mitigate the effects of imperfect chart review as much as possible, we evaluated for signs and symptoms beyond those outlined in DSM-5 to capture more specific details about disease range and severity that are typically documented in medical charts. These signs and symptoms, such as seeking care for OUD, opioid overdose, intravenous or intranasal opioid use, or treatment contract violations, were viewed as prominent factors in the way clinical diagnoses of OUD are made in real-world practice yet may not have been captured had we relied strictly on DSM-5 criteria for our abstraction. Finally, we maintained consistency across abstractions by creating a clear manual, a robust training process and by holding weekly meetings between screeners and lead investigators. We also augmented our manual chart review process with administrative data for comorbidities to obtain more information about each patient.

5. Conclusions

The present findings highlight the need for more robust methods of categorizing individuals with OUD rather than relying on ICD coding alone. Diagnosing OUD, particularly in individuals who use prescribed opioids, can be difficult and can often be subjective. Moreover, as this study found, individuals labeled as having opioid use, abuse or dependence can span a spectrum from those using prescribed opioids for pain to those who meet multiple criteria for DSM-5 diagnosis of OUD. This variation must be recognized to enable the creation of policies and interventions that improve patient identification and treatment across this spectrum.

Supplementary Material

Appendix A. Supplementary Data

Acknowledgements

The authors would like to thank Ryan Sanii, MPH and Ruiqi Tang for their help completing chart abstractions.

Role of Funding Source

Authors report funding from NIA [Training Grant # AG062043] (VP), NIDA [Grant #1K23DA047475-01A1] (PL), the Veteran’s Health Administration, Ann Arbor, MI Center for Clinical Management and Research [Grant # LIP 19-113] (PL), and a Career Development Award [CDA 18–008] from the US Department of Veterans Affairs Health Services Research & Development Service (LAL). None of the funding agencies played a role in any aspect of the study.

Footnotes

Declaration of Competing Interest

No conflict declared.

Appendix A. Supplementary data

Supplementary material related to this article can be found, in the online version, at doi:https://doi.org/10.1016/j.drugalcdep.2021.108583.

References

  1. Barocas JA, White LF, Wang J, Walley AY, LaRochelle MR, Bernson D, Land T, Morgan JR, Samet JH, Linas BP, 2018. Estimated prevalence of opioid use disorder in Massachusetts, 2011–2015: a capture–recapture analysis. Am. J. Public Health 108, 1675–1681. 10.2105/AJPH.2018.304673. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Bohnert ASB, Ilgen MA, Galea S, McCarthy JF, Blow FC, 2011a. Accidental Poisoning Mortality Among Patients in the Department of Veterans Affairs Health System. Med. Care 49, 393–396. [DOI] [PubMed] [Google Scholar]
  3. Bohnert ASB, Valenstein M, Bair MJ, Ganoczy D, McCarthy JF, Ilgen MA, Blow FC, 2011b. Association Between Opioid Prescribing Patterns and Opioid Overdose-Related Deaths. JAMA 305, 1315–1321. 10.1001/jama.2011.370. [DOI] [PubMed] [Google Scholar]
  4. Boscarino JA, Hoffman SN, Gerhard G, Han J, Rukstalis M, Erlich PM, Stewart WF, 2010. PS2–13: genetic and environmental risk factors for prescription opioid dependence in the healthcare setting. Clin. Med. Res 8, 51. 10.3121/cmr.8.1.51. [DOI] [Google Scholar]
  5. Boscarino JA, Hoffman SN, Han JJ, 2015. Opioid-use disorder among patients on long-term opioid therapy: impact of final DSM-5 diagnostic criteria on prevalence and correlates. Subst. Abuse Rehabil 6, 83–91. 10.2147/SAR.S85667. [DOI] [PMC free article] [PubMed] [Google Scholar]
  6. Chen KY, Chen L, Mao J, 2014. Buprenophine-Naloxone in Pain Management. Anesthesiology 120, 1262–1274. 10.1097/ALN.0000000000000170. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Ciesielski T, Iyengar R, Bothra A, Tomala D, Cislo G, Gage BF, 2016. A tool to assess risk of de novo opioid abuse or dependence. Am. J. Med 129, 699–705.e4. 10.1016/j.amjmed.2016.02.014. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Clemans-Cope L, Lynch V, Epstein M, Kenney GM, 2019a. Opioid and substance use disorder and receipt of treatment among parents living with children in the United States, 2015–2017. Ann. Fam. Med 17, 207–211. 10.1370/afm.2389. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Clemans-Cope L, Lynch V, Winiski E, Epstein M, 2019b. State Variation in Medicaid Prescriptions for Opioid use Disorder from 2011 to 2018. Urban Institute. [Google Scholar]
  10. Cochran BN, Flentje A, Heck NC, Van Den Bos J, Perlman D, Torres J, Valuck R, Carter J, 2014. Factors Predicting Development of Opioid Use Disorders among Individuals who Receive an Initial Opioid Prescription: Mathematical Modeling Using a Database of Commercially-insured Individuals. Drug Alcohol Depend. 138, 202–208. 10.1016/j.drugalcdep.2014.02.701. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Dufour R, Mardekian J, Pasquale M, Schaaf D, Andrews GA, Patel NC, 2014. Understanding predictors of opioid abuse: predictive model development and validation. Am. J. Pharm. Benefits 6, 208–216. [Google Scholar]
  12. First MB, Williams JBW, Karg RS, Spitzer RL, 2016. User’s Guide for the SCID-5-CV Structured Clinical Interview for DSM-5® Disorders: Clinical Version, User’s Guide for the SCID-5-CV Structured Clinical Interview for DSM-5® Disorders: Clinical Version. American Psychiatric Publishing, Inc., Arlington, VA, US. [Google Scholar]
  13. Gelernter J, Panhuysen C, Wilcox M, Hesselbrock V, Rounsaville B, Poling J, Weiss R, Sonne S, Zhao H, Farrer L, Kranzler HR, 2006. Genomewide linkage scan for opioid dependence and related traits. Am. J. Hum. Genet 78, 759–769. 10.1086/503631. [DOI] [PMC free article] [PubMed] [Google Scholar]
  14. Gelernter J, Kranzler HR, Sherva R, Koesterer R, Almasy L, Zhao H, Farrer LA, 2014. Genome-wide association study of opioid dependence: multiple associations mapped to calcium and potassium pathways. Biol. Psychiatry 76, 66–74. 10.1016/j.biopsych.2013.08.034. Dopamine Deficits as an Addiction Phenotype [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Gellad WF, Good CB, Shulkin DJ, 2017. Addressing the Opioid Epidemic in the United States: Lessons From the Department of Veterans Affairs. JAMA Intern. Med 177, 611–612. 10.1001/jamainternmed.2017.0147. [DOI] [PubMed] [Google Scholar]
  16. Glanz JM, Narwaney KJ, Mueller SR, Gardner EM, Calcaterra SL, Xu S, Breslin K, Binswanger IA, 2018. Prediction model for two-year risk of opioid overdose among patients prescribed chronic opioid therapy. J. Gen. Intern. Med 33, 1646–1653. 10.1007/s11606-017-4288-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Gordon AJ, Trafton JA, Saxon AJ, Gifford AL, Goodman F, Calabrese VS, McNicholas L, Liberto J, Buprenorphine Work Group of the Substance Use Disorders Quality Enhancement Research Initiative, 2007. Implementation of buprenorphine in the Veterans Health Administration: results of the first 3 years. Drug Alcohol Depend. 90, 292–296. 10.1016/j.drugalcdep.2007.03.010. [DOI] [PubMed] [Google Scholar]
  18. Green CA, Perrin NA, Janoff SL, Campbell CI, Chilcoat HD, Coplan PM, 2017. Assessing the accuracy of opioid overdose and poisoning codes in diagnostic information from electronic health records, claims data, and death records. Pharmacoepidemiol. Drug Saf 26, 509–517. 10.1002/pds.4157. [DOI] [PubMed] [Google Scholar]
  19. Green CA, Perrin NA, Hazlehurst B, Janoff SL, DeVeaugh-Geiss A, Carrell DS, Grijalva CG, Liang C, Enger CL, Coplan PM, 2019. Identifying and classifying opioid-related overdoses: a validation study. Pharmacoepidemiol. Drug Saf 28, 1127–1137. 10.1002/pds.4772. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Hadlandsmyth K, Mosher H, Vander Weg MW, Lund BC, 2018. Decline in prescription opioids attributable to decreases in long-term use: a retrospective study in the veterans health administration 2010–2016. J. Gen. Intern. Med 33, 818–824. 10.1007/s11606-017-4283-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Hallgren KA, Witwer E, West I, Baldwin L-M, Donovan D, Stuvek B, Keppel GA, Mollis B, Stephens KA, 2020. Prevalence of documented alcohol and opioid use disorder diagnoses and treatments in a regional primary care practice-based research network. J. Subst. Abuse Treat 110, 18–27. 10.1016/j.jsat.2019.11.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. Howell BA, Abel EA, Park D, Edmond SN, Leisch LJ, Becker WC, 2020. Validity of incident opioid use disorder (OUD) diagnoses in administrative data: a chart verification study. J. Gen. Intern. Med 10.1007/s11606-020-06339-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Hser Y-I, Mooney LJ, Saxon AJ, Miotto K, Bell DS, Huang D, 2017. Chronic pain among patients with opioid use disorder: results from electronic health records data. J. Subst. Abuse Treat 77, 26–30. 10.1016/j.jsat.2017.03.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Jones CM, Byrd DJ, Clarke TJ, Campbell TB, Ohuoha C, McCance-Katz EF, 2019. Characteristics and current clinical practices of opioid treatment programs in the United States. Drug Alcohol Depend. 205, 107616. 10.1016/j.drugalcdep.2019.107616. [DOI] [PubMed] [Google Scholar]
  25. Lagisetty PA, Lin LA, Ganoczy D, Haffajee RL, Iwashyna TJ, Bohnert ASB, 2019. Opioid prescribing after opioid-related inpatient hospitalizations by diagnosis: a cohort study. Med. Care 57, 815–821. 10.1097/MLR.0000000000001182. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Lin L (Allison), Peltzman T, McCarthy JF, Oliva EM, Trafton JA, Bohnert ASB, 2019. Changing trends in opioid overdose deaths and prescription opioid receipt among veterans. Am. J. Prev. Med 57, 106–110. 10.1016/j.amepre.2019.01.016. [DOI] [PubMed] [Google Scholar]
  27. Lin LA, Bohnert ASB, Blow FC, Gordon AJ, Ignacio RV, Kim HM, Ilgen MA, 2020. Polysubstance use and association with opioid use disorder treatment in the US Veterans Health Administration. Addict. Abingdon Engl 10.1111/add.15116. [DOI] [PubMed] [Google Scholar]
  28. Lund BC, Ohl ME, Hadlandsmyth K, Mosher HJ, 2019. Regional and rural–Urban variation in opioid prescribing in the veterans health administration. Mil. Med 184, 894–900. 10.1093/milmed/usz104. [DOI] [PubMed] [Google Scholar]
  29. Madras BK, Ahmad NJ, Wen J, Sharfstein J, Prevention AT, Treatment, Epidemic, and R.W.G. of the A.C. on C. the U.S.O, 2020. Improving access to evidence-based medical treatment for opioid use disorder: strategies to address key barriers within the treatment system. NAM Perspect. 10.31478/202004b. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Manhapra A, Arias AJ, Ballantyne JC, 2018. The conundrum of opioid tapering in long-term opioid therapy for chronic pain: a commentary. Subst. Abuse 39, 152–161. 10.1080/08897077.2017.1381663. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Maremmani AGI, Rovai L, Rugani F, Pacini M, Lamanna F, Bacciardi S, Perugi G, Deltito J, Dell’Osso L, Maremmani I, 2012. Correlations Between Awareness of Illness (Insight) and History of Addiction in Heroin-Addicted Patients. Front. Psychiatry 3. 10.3389/fpsyt.2012.00061. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. National Academies of Sciences, Engineering, and Medicine, Health and Medicine Division, Board on Health Sciences Policy, 2018. Medication-Assisted Treatment for Opioid Use Disorder: Proceedings of a Workshop—in Brief, the National Academies Collection: Reports Funded by National Institutes of Health. National Academies Press (US), Washington (DC). [PubMed] [Google Scholar]
  33. Nosyk B, Anglin MD, Brissette S, Kerr T, Marsh DC, Schackman BR, Wood E, Montaner JSG, 2013. A call for evidence-based medical treatment of opioid dependence in the United States and Canada. Health Aff. Proj. Hope 32, 1462–1469. 10.1377/hlthaff.2012.0846. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Peer K, Rennert L, Lynch KG, Farrer L, Gelernter J, Kranzler HR, 2013. Prevalence of DSM-IV and DSM-5 alcohol, cocaine, opioid, and Cannabis use disorders in a largely substance dependent sample. Drug Alcohol Depend. 127, 215–219. 10.1016/j.drugalcdep.2012.07.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Peterson C, Xu L, Mikosz CA, Florence C, Mack KA, 2018. US hospital discharges documenting patient opioid use disorder without opioid overdose or treatment services, 2011–2015. J. Subst. Abuse Treat 92, 35–39. 10.1016/j.jsat.2018.06.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Rice JB, White AG, Birnbaum HG, Schiller M, Brown DA, Roland CL, 2012. A model to identify patients at risk for prescription opioid abuse, dependence, and misuse. Pain Med. Malden Mass 13, 1162–1173. 10.1111/j.1526-4637.2012.01450.x. [DOI] [PubMed] [Google Scholar]
  37. Roland CL, Lake J, Oderda GM, 2016. Prevalence of prescription opioid Misuse/Abuse as determined by international classification of diseases codes: a systematic review. J. Pain Palliat. Care Pharmacother 30, 258–268. 10.1080/15360288.2016.1231739. [DOI] [PubMed] [Google Scholar]
  38. Song Z, 2017. Mortality quadrupled among opioid-driven hospitalizations, notably within lower-income and disabled white populations. Health Aff. (Millwood) 36, 2054–2061. 10.1377/hlthaff.2017.0689. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Substance Abuse and Mental Health Services Administration, 2016. Impact of the DSM-IV to DSM-5 Changes on the National Survey on Drug Use and Health, CBHSQ Methodology Report. Substance Abuse and Mental Health Services Administration (US), Rockville (MD). [PubMed] [Google Scholar]
  40. Substance Abuse and Mental Health Services Administration, 2019. Key Substance Use and Mental Health Indicators in the United States. Results from the 2018 National Survey on Drug Use and Health. [Google Scholar]
  41. Teeters JB, Lancaster CL, Brown DG, Back SE, 2017. Substance use disorders in military veterans: prevalence and treatment challenges. Subst. Abuse Rehabil 8, 69–77. 10.2147/SAR.S116720. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Veterans Health Administration, 2019. Opioid Safety - Clinical Tools [WWW Document]. URL https://www.va.gov/PAINMANAGEMENT/Opioid_Safety/Clinical_Tools.asp (accessed 10.1.20).
  43. Von Korff M, 2010. Commentary on Boscarino et al. : Understanding the Spectrum of Opioid Abuse, Misuse and Harms among Chronic Opioid Therapy Patients. Addict. Abingdon Engl 105, 1783–1784. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Wakeman SE, Larochelle MR, Ameli O, Chaisson CE, McPheeters JT, Crown WH, Azocar F, Sanghavi DM, 2020. Comparative effectiveness of different treatment pathways for opioid use disorder. JAMA Netw. Open 3. 10.1001/jamanetworkopen.2019.20622e1920622–e1920622. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Watson A, Simon DM, Peratikos MB, Stringer EA, 2020. Medical Utilization Surrounding Initial Opioid-Related Diagnoses by Coding Method [WWW Document]. AJMC. URL https://www.ajmc.com/journals/issue/2020/2020-vol26-n2/medical-utilization-surrounding-initial-opioidrelated-diagnoses-by-coding-method (accessed 5.24.20). [DOI] [PubMed] [Google Scholar]
  46. Webster L, Gudin J, Raffa RB, Kuchera J, Rauck R, Fudin J, Adler J, Mallick-Searle T, 2020. Understanding buprenorphine for use in chronic pain: expert opinion. Pain Med. 21, 714–723. 10.1093/pm/pnz356. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Weiss AJ, Heslin KC, Stocks C, Owens P, 2020. Hospital Inpatient Stays Related to Opioid Use Disorder and Endocarditis, 2016 [WWW Document]. URL https://www.hcup-us.ahrq.gov/reports/statbriefs/sb256-Opioids-Endocarditis-Inpatient-Stays-2016.jsp (accessed 5.31.20). [PubMed]
  48. Wilson EB, 1927. Probable inference, the law of succession, and statistical inference. J. Am. Stat. Assoc 22, 209–212. 10.1080/01621459.1927.10502953. [DOI] [Google Scholar]
  49. Wilson-Poe AR, Morón JA, 2018. The dynamic interaction between pain and opioid misuse. Br. J. Pharmacol 175, 2770–2777. 10.1111/bph.13873. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Wu L-T, Zhu H, Swartz MS, 2016. Treatment utilization among persons with opioid use disorder in the United States. Drug Alcohol Depend. 169, 117–127. 10.1016/j.drugalcdep.2016.10.015. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Wu L-T, McNeely J, Subramaniam GA, Brady KT, Sharma G, VanVeldhuisen P, Zhu H, Schwartz RP, 2017. DSM-5 substance use disorders among adult primary care patients: results from a multisite study. Drug Alcohol Depend. 179, 42–46. 10.1016/j.drugalcdep.2017.05.048. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Wyse JJ, Gordon Adam J., Dobscha Steven K., Morasco Benjamin J., Tiffany Elizabeth, Drexler Karen, Sandbrink F, Lovejoy TI, 2018. Medications for opioid use disorder in the Department of Veterans Affairs (VA) health care system: historical perspective, lessons learned, and next steps. Subst. Abuse 39, 139–144. 10.1080/08897077.2018.1452327. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Young SR, Azari S, Becker WC, Edelman EJ, Liebschutz JM, Roy P, Starrels JL, Merlin JS, 2019. Common and challenging behaviors among individuals on long-term opioid therapy. J. Addict. Med 10.1097/ADM.0000000000000587. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Zhou H, Rentsch C, Cheng Z, Kember R, Nunex Y, Tate J, Dao C, Polimanti R, Justice A, Kranzler H, Gelernter J, 2019. Functional coding variant in oprm1 gene associated with opioid use disorder: evidence from gwas on large cohorts. In: Eur. Neuropsychopharmacol, Abstracts of the XXVIIth World Congress of Psychiatric Genetics (WCPG). 26–31 2019, October, Los Angeles, California: 29, S34–S35. 10.1016/j.euroneuro.2019.07.070. [DOI] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Appendix A. Supplementary Data
NIHMS1681711-supplement-Appendix_A__Supplementary_Data.docx (73.9KB, docx)

RESOURCES