Development of a 5-Step Electronic Medical Record–Based Algorithm to Identify Patients with Opioid Use Disorder in Pregnancy

Abstract

Objectives:

This study aimed to develop and validate an algorithm for the identification of opioid use disorder (OUD) in pregnant patients using electronic medical record (EMR) data.

Materials and Methods:

A cohort of pregnant patients from a single institution was used to develop and validate the algorithm. Five algorithm components were used, and chart reviews were conducted to confirm OUD diagnoses based on established criteria. Positive predictive values (PPV) of each of the algorithm's components were assessed.

Results:

Of the 334 charts identified by the algorithm, 256 true cases were confirmed. The overall PPV of the algorithm was 76.6%, with 100% accuracy for outpatient medication lists, and high PPVs ranging from 81.3% to 93.4% across other algorithm components.

Discussion and Conclusion:

The study highlights the significance of a multifaceted approach in identifying OUD among pregnant patients, aiming to improve patient care and target interventions for patients at risk.

Background and Significance

In the United States from 2010 to 2017, maternal opioid-related diagnoses have risen from 3.5 to 8.2 per 1,000 delivery hospitalizations.¹ The challenges associated with timely identification of opioid use disorder (OUD) during the perinatal period may contribute to adverse maternal and neonatal outcomes.^2,3 Improving care for families affected by OUD and designing targeted interventions necessitates the ability to accurately identify patients affected by OUD, allowing for the measurement of perinatal outcomes. Opioid use is not captured on birth certificate data, and extrapolation from state-level databases lags by several years with no linked data on pregnant patient outcomes.⁴ Additionally, no code from the International Classification of Diseases, Tenth Revision (ICD-10) exists specifically for opioid use in pregnancy. While some researchers have developed methods to query the electronic medical record (EMR) using ICD-10 codes and natural language processing (NLP) to identify nonpregnant patients with OUD^5–7, no standardized algorithm exists for OUD detection specifically in pregnant patients and relying on a single ICD-10 code from the EMR results in substandard sensitivity for OUD.^8,9

To address this issue, we developed an algorithm that can be incorporated into the EMR to identify patients with OUD during pregnancy and at delivery. We hypothesized that, by using multiple EMR inputs, we can identify patients with OUD during pregnancy and at delivery with high accuracy.

Materials and Methods

Study Design, Data Source, Sample and Statistical Analysis

To develop an automated method to identify OUD in pregnancy using the EMR, we initially identified a total of 16,915 deliveries from Tampa General Hospital from January 1, 2019, to September 1, 2021. Deliveries were identified using the EMR through Epic's stork delivery summary. We considered 5 sources of routinely collected data in an effort to identify OUD: (1) ICD-10 coding of the pregnant patient's chart, (2) ICD-10 coding of the neonatal chart, (3) keyword search of the pregnant patient's history and physical (H&P) documentation at the time of delivery, (4) urine toxicology testing, and (5) outpatient medication list at the time of delivery (Appendix 1). These 5 elements were chosen because of their ease of extraction from EMR, their ability to differentiate OUD from other substance use, and success in previous research.^10,11

Algorithm Components

ICD-10 codes included all F11.X codes for opioid use and T40.0X, T40.1X, T40.2X, T40.3X, T40.4X, and T40.6X codes indicating opioid poisoning. Neonatal coding included P96.1 (neonatal withdrawal symptoms from maternal use of drugs of addiction) and P04.14 (newborn affected by maternal use of opiates), which has been shown to have high positive predictive value (PPV) for neonatal opioid withdrawal syndrome (NOWS) in previous studies.^12,13

We utilized keywords that originated from previous research on NLP for the identification of OUD by Tang and Blackley.^10,11 To enhance the precision of our search, we excluded cases where keywords matched provider-ordered medication (eg, removing patients with a positive fentanyl keyword and a corresponding order for fentanyl epidural).

Additionally, urine toxicology testing was queried for positive screening for opioids. Toxicology testing involved several substances and pharmaceuticals, including buprenorphine (Subutex), buprenorphine-naloxone (Suboxone), methadone, oxycodone, oxycontin, hydrocodone, hydromorphone, heroin, opiates, tramadol, oxymorphone, and fentanyl (Appendix 1).

Furthermore, medication lists at the time of delivery were queried for medications for OUD, including methadone, buprenorphine, and buprenorphine-naloxone. Figure 1 shows the final algorithm.

Figure 1.

Final Algorithm for Identification of Patients with Opioid Use Disorder

ICD, International Classification of Diseases; NOWS, neonatal opioid withdrawal syndrome.

After identification by 1 of the 5 algorithm components described above, the charts were manually reviewed for evidence of OUD. In cases where a pregnant patient had multiple births during the specified period, only the records of the first birth were reviewed. OUD was defined using criteria from the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition, Text Revision (DSM-V)¹⁴ and American College of Obstetricians and Gynecologists Committee Opinion.¹⁵ Patients were categorized as having OUD if their laboratory and provider notes documented any the following: (1) active opioid use, such as heroin or oxycodone; (2) if patient was on medication for OUD such as methadone or buprenorphine; (3) a history of OUD in recovery without active opioid use; or (4) opioid use for greater than 3 months due to conditions like chronic pain. Furthermore, patients were categorized as not having OUD if (1) there was no evidence of opioid use in any laboratory or physician notes; (2) documentation supported only nonopioid substance use; or (3) opioid use was for short-term relief of acute pain for surgery, including cesarean delivery, labor pain, or acute conditions such as kidney stone or other injury for less than 3 months. It should be noted that a positive urine toxicology testing did not automatically qualify as OUD, unless other documentation supported an OUD diagnosis. This precaution was taken because many false positives could occur at the time of delivery if patients received opioids epidurals/spinal anesthesia or other intravenous opioids for pain relief in labor.¹⁶ Study data were collected and managed using REDCap (Research Electronic Data Capture) software hosted at the research institution.^17,18 REDCap is a secure, Web-based software platform designed to support data capture for research studies, providing (1) an intuitive interface for validated data capture; (2) audit trails for tracking data manipulation and export procedures; (3) automated export procedures for seamless data downloads to common statistical packages; and (4) procedures for data integration and interoperability with external sources. All OUD cases were categorized as described above.

The primary metric used to evaluate algorithm performance was the PPV or the proportion of algorithm-identified cases that were deemed to be true OUD cases after medical record review. In addition to the PPV and 95% CI for the overall algorithm, we estimated the PPV for each algorithm component when only 1 of the 5 components was positive (eg, maternal ICD-10 code indicative of OUD use but no keywords, diagnosis of neonatal abstinence syndrome, medications, or urine drug screen). We initially planned to report PPV for each component when it was present in addition to other components (eg, maternal ICD code and medications); however, PPV was perfect (100%) when more than 1 component was positive, so we reported these together to simplify our presentation of results.

Results

Refining the Algorithm

Initially, we considered O99.32 coding for substance use complicating pregnancy, but subsequently excluded it because of its lack of specificity for OUD in pregnancy. In our preliminary examination and manual chart reviews of charts with O99.32 diagnostic codes, none of the 27 charts examined indicated OUD, but only reflected patients with nonopioid substance use.

Despite eliminating all positive matches in the allergy section of the patient history and physical documentation, a high false positive rate persisted (2 out of 17 charts with OUD). Subsequently, we limited the keyword subset to only medications for OUD and other nonprescription medications not given for pain in labor or postpartum pain control (Appendix 1).

We examined first urine toxicology testing at the time of delivery, but noted high false positive screening due to iatrogenic opioid use. We then elected to examine urine toxicology testing from the date of conception to delivery hospitalization to query outpatient prenatal toxicology testing.

Algorithm Findings

After running all 5 algorithm components, 334 charts were identified out of 16,915 deliveries (16,534 pregnant patients) between January 1, 2019, and September 1, 2021. The demographic characteristics are shown in Table 1. Among the 334 deliveries, 256 (76.6%) had evidence of OUD. Patients with OUD were mostly non-Hispanic White (77.3%; 198/256) and covered by Medicaid (80.1%; 205/256). The majority of these patients (93.0%; 238/256) received prenatal care, with a median of 8 prenatal care visits (interquartile range [IQR], 5–10; range, 1–20). The median maternal age at delivery was 32 years and median gestational age at delivery was 38 weeks.

Table 1.

Demographic Characteristics of All Patients, Patients Without Opioid Use Disorder (OUD), and Patients With OUD

Characteristics	All patients (n, %)	No OUD (n, %)	OUD (n, %)
Total	334 (100)	78 (23.4)	256 (76.6)
Race/ethnicity
Non-Hispanic Black	41 (12.3)	27 (34.6)	14 (5.5)
Non-Hispanic White	222 (66.5)	24 (30.7)	198 (77.3)
Hispanic	54 (16.2)	20 (25.6)	34 (13.3)
Multiracial	11 (3.3)	5 (6.4)	6 (2.3)
Other/unknown	5 (1.5)	1 (1.2)	4 (1.6)
Asian	1 (0.3)	1 (1.2)	0 (0.0)
Fetal or neonatal death	7 (2.1)	1 (1.2)	6 (2.3)
Median gestational age at delivery, weeks (IQR)	38 (37–39)	38 (36–39)	38 (37–39)
Median maternal age at delivery, years (IQR)	32 (28–36)	29.5 (27–35)	32 (28–36)
Insurance
Commercial	27 (8.1)	3 (3.8)	24 (9.4)
Medicaid	262 (78.4)	57 (73.1)	205 (80.1)
Medicare	25 (7.5)	7 (8.9)	18 (7.0)
Tricare*	7 (2.1)	3 (3.8)	4 (1.6)
Naphcare†	3 (0.9)	0 (0.0)	3 (1.2)
Emergency Medicaid	8 (2.4)	8 (10.2)	0 (0)
Unknown	2 (0.6)	0 (0.0)	2 (0.8)

^*Insurance coverage for the military and their families.

^†Insurance coverage for incarcerated patients.

Regarding outcomes, the occurrence of fetal or neonatal death was 2.3% (6/256), and the majority of infants born to mothers with OUD (73.8%; 189/256) were discharged home to a biological parent (Table 1). Additionally, 54.3% (139/256) of patients with OUD were on medication for OUD, 22.6% (58/256) were in recovery without evidence of taking medication for OUD, 28.9% (74/256) had active opioid use, and 11.7% (30/256) were on long-term opioids for chronic pain (Figure 2).

Figure 2.

Status of Opioid Use in Sample

OUD, opioid use disorder. Categories are not mutually exclusive. Patients may be counted in more than 1 category.

The overall PPV of the complete algorithm, with 256 true cases among the 334 cases identified by the algorithm, was 76.6% (95% CI, 72.1%–81.2%). Regarding the 5 algorithm components, overall, the PPV was 100% for outpatient medication list at the time of delivery, 93.4% (95% CI, 90.2%–96.6%) for keyword search of the pregnant patient's history and physical documentation at the time of delivery, 87.8% (95% CI, 83.5%–92.0%) for ICD-10 coding of the pregnant patient's chart, 83.4% (95% CI, 78.2%–88.7%) for ICD-10 coding of the neonatal chart, and 81.3% (95% CI, 62.1%–100.0%) for prenatal urine toxicology testing.

Discussion

This study aimed to develop an algorithm that can be deployed in the EMR to identify patients with OUD during pregnancy. We hypothesized that by using multiple EMR inputs, we can accurately identify patients with OUD in pregnancy. Our algorithm used ICD-10 coding of the pregnant patient's chart, ICD-10 coding of the neonatal chart, keyword search of the pregnant patient's history and physical documentation at the time of delivery, urine toxicology testing prior to delivery hospitalization, and outpatient medication list at the time of delivery to identify OUD in pregnancy. The overall PPV of the algorithm was 76.6%, higher than results reported of another algorithm in a recent study,¹⁹ and indicates that, among the OUD cases identified by the algorithm, about 77% were confirmed to have OUD through manual chart review.

Our examination of the 5 algorithm components individually and in combination illustrates the importance of using different data sources. The outpatient medication list at the time of delivery had a PPV of 100%, emphasizing the significance of medication records as a highly reliable source in identifying OUD cases. This finding aligns with Blackley et al,¹¹ where they similarly identified documented medication history to be most important in an algorithm to identify patients with OUD. The keyword search of pregnant patients' history and physical documentation at the time of delivery also yielded a high PPV of 93.4%, underscoring the efficacy of keyword search processing in identifying relevant information. While the PPVs for ICD-10 coding in both pregnant patients' and neonatal charts were fairly good at 87.8% and 83.6%, respectively, the PPV for prenatal urine toxicology testing was slightly lower at 83.4%. Consistent with prior studies, these findings highlight the importance of not relying on a single data source ^19,20 and instead underscores the use of multiple data elements for a more comprehensive assessment of OUD, as each source contributes uniquely to the overall accuracy. However, the strengths of individual algorithm components should be weighed against their collective efficacy.

Strengths and Limitations

This study had several strengths and limitations. Strengths of this study included the focus on pregnancy-specific encounters rather than on OUD in general, and the use of a large cohort of patients over multiple years. A key limitation was the restriction to a single site. Further validation across diverse health care settings is needed for generalizability of findings to varying medical environments. The use of a single EMR system and its data extraction procedures may limit the study's applicability to other EMR systems with differing structures and terminologies. Although keyword searches were used for the physician notes algorithm component, this study did not use advanced NLP techniques, which might have implications for the depth of information extraction and pattern recognition within the data. Despite these limitations, the algorithm developed in this study offers the potential for improving early and accurate identification of OUD cases during pregnancy. This can facilitate timely interventions, enabling personalized care plans, and, ultimately, improving maternal and neonatal health outcomes.

Conclusions

Our study establishes the significance of a multifaceted approach in identifying OUD among pregnant patients. The algorithm, utilizing 5 algorithm components, demonstrates improving accuracy in identifying patients with OUD, with each component contributing differently to the overall effectiveness. Future research should aim to refine the algorithm, considering the differences across the perinatal period, and continually evolving diagnostic and coding standards. Our next steps include testing the algorithm across multiple health care settings with diverse EHR systems. Findings of this study offer a foundation for improving OUD identification strategies, ultimately contributing to improved prenatal and postpartum care and targeted interventions for pregnant individuals at risk for OUD.

Appendix 1

1. Maternal ICD-10 Codes:

   • F11* Opioid related disorders
   • O99.32 Drug use complicating pregnancy, childbirth, and the puerperium

     • O99.320 Drug use complicating pregnancy, unspecified trimester
     • O99.321 Drug use complicating pregnancy, first trimester
     • O99.322 Drug use complicating pregnancy, second trimester
     • O99.323 Drug use complicating pregnancy, third trimester
     • O99.324 Drug use complicating childbirth
     • O99.325 Drug use complicating the puerperium
   • T40.0* Poisoning by, adverse effect of and underdosing of opium
   • T40.1* Poisoning by and adverse effect of heroin
   • T40.2* Poisoning by, adverse effect of and underdosing of other opioids
   • T40.3* Poisoning by, adverse effect of and underdosing of methadone
   • T40.4* Poisoning by, adverse effect of and underdosing of other synthetic narcotics
   • T40.6* Poisoning by, adverse effect of and underdosing of other and unspecified narcotics
2. Neonatal ICD-10 Codes:

   • P96.1 “Neonatal withdrawal symptoms from maternal use of drugs of addiction”
   • P04.14 “Newborn affected by maternal use of opiates”
3. Original and Final Keywords:

   • Original keywords: opioid, opiate, narcotic, oxycodone, OxyContin, oxycet, Roxicet, Percocet, hydrocodone, lorcet, lortab, norco, Vicodin, fentanyl, morphine, MS contin, hydromorphone, Dilaudid, methadone, heroin, opiate abuse, opioid abuse, buprenorphine, tramadol, Ultram, codeine, oxymorphone, Opana, Subutex, Suboxone, Demerol, meperidine, MSContin, heroine, narcotic, narcotics, narcotism, Avinza, fenanyl, fentantyl, fentayl, opiates, opioids, oxycone, oxycodone, roxycodone, sufentanil, opioid, OUD, MAT, Narcan, naltrexone, vivitrol
   • Final keywords: Oxycontin, methadone, heroin, “opiate abuse”, “opioid abuse”, heroine, buprenorphine, Subutex, suboxone, MSContin. “MS Contin”

4. Urine Toxicology Testing Search Terms

   (eap.PROC_NAME like ‘%Urine%’ and eap.PROC_NAME like ‘%drug%’ and eap.PROC_NAME like ‘%screen%’)

   Also filtered components at the same time for each of those labs.

   (comp.NAME like ‘%buprenorphine%’ or comp.NAME like ‘%methadone%’ or comp.NAME like ‘%morphine%’ or comp.NAME like ‘%codeine%’ or comp.NAME like ‘%oxycodone%’ or comp.NAME like ‘%oxymorphone%’ or comp.NAME like ‘%fentanyl%’ or comp.NAME like ‘%hydrocodone%’or comp.NAME like ‘%hydromorphone%’ or comp.NAME like ‘%opiate%’ or comp.NAME like ‘%opioid%’ or comp.NAME like ‘%Subutex%’ or comp.NAME like ‘%Suboxone%’ or comp.NAME like ‘%tramadol%’ or comp.NAME like ‘%heroin%’ or comp.NAME like ‘%dilaudid%’or comp.NAME like ‘%oxycontin%’)

5. Medication Search Terms

   Methadone, buprenorphine, Subutex, and Suboxone