Abstract
Objective
No case definition exists that allows public health authorities to accurately identify opioid overdoses using emergency medical services (EMS) data. We developed and evaluated a case definition for suspected nonfatal opioid overdoses in EMS data.
Methods
To identify suspected opioid overdose–related EMS runs, in 2019 the Rhode Island Department of Health (RIDOH) developed a case definition using the primary impression, secondary impression, selection of naloxone in the dropdown field for medication given, indication of medication response in a dropdown field, and keyword search of the report narrative. We developed the case definition with input from EMS personnel and validated it using an iterative process of random medical record review. We used naloxone administration in consideration with other factors to avoid misclassification of opioid overdoses.
Results
In 2018, naloxone was administered during 2513 EMS runs in Rhode Island, of which 1501 met our case definition of a nonfatal opioid overdose. Based on a review of 400 randomly selected EMS runs in which naloxone was administered, the RIDOH case definition accurately identified 90.0% of opioid overdoses and accurately excluded 83.3% of non–opioid overdose–related EMS runs. Use of the case definition enabled analyses that identified key patterns in overdose locations, people who experienced repeat overdoses, and the creation of hotspot maps to inform outbreak detection and response.
Practice Implications
EMS data can be an effective tool for monitoring overdoses in real time and informing public health practice. To accurately identify opioid overdose–related EMS runs, the use of a comprehensive case definition is essential.
Keywords: emergency medical services, EMS, surveillance, opioid, overdose
In 2016, Rhode Island had the eighth highest rate of unintentional drug overdose deaths nationally (30.8 deaths per 100 000 people). 1 To help address this epidemic, the Rhode Island Department of Health (RIDOH) launched several initiatives to prevent overdoses and minimize drug-related harms in Rhode Island. 2 -8 To help identify, monitor, and respond to drug overdose trends in real time and supplement other surveillance systems, in 2016 RIDOH began using the Rhode Island Emergency Medical Services (EMS) Information System (RI-EMSIS), which captures information on all care and assessments provided during EMS runs in Rhode Island.
RI-EMSIS surveillance data include information currently not captured by other surveillance platforms in Rhode Island, including details on the location type (eg, public, private, semi-private) and address information for all deployments of EMS staff members. EMS providers are required per Rhode Island legislation 9 to upload data to RI-EMSIS within 2 hours of run completion, allowing this information to be available for use in near real time to generate heat maps that can inform targeted outreach and prevention efforts during overdose outbreaks. 10 In addition, RIDOH can use the patient identifiers available in RI-EMSIS to identify and characterize people for whom multiple EMS deployments have been made, such as people who have experienced multiple overdoses requiring EMS care.
Because RI-EMSIS captures data on all EMS runs, one challenge to using these data to respond to the opioid overdose epidemic is correctly identifying EMS runs for opioid overdoses. One way to identify opioid overdose–related EMS runs is by identifying all EMS runs in which naloxone was used. However, although naloxone can be used to rapidly reverse an opioid overdose, naloxone can also be used to safely rule out the occurrence of an opioid overdose among people with altered mental status, apnea, or severe respiratory depression. In Rhode Island, per statewide EMS protocol, naloxone can be administered to patients with altered mental status if drug use is suspected or unknown. 11 As such, to appropriately exclude people who were administered naloxone but had not experienced an opioid overdose, RIDOH needed to develop a comprehensive case definition for opioid overdoses. Although previous work had identified opioid overdose–related EMS runs using naloxone administration, 12,13 no comprehensive case definition existed at the time of our study.
The Enhanced State Opioid Overdose Surveillance programs’ cooperative agreement from the Centers for Disease Control and Prevention funded the development of this case definition, which supported states in improving surveillance, including better using EMS data systems for rapid surveillance of drug overdoses. Here, we describe the development and evaluation of the case definition with the goal of helping other health departments interested in implementing nonfatal opioid overdose surveillance using EMS data.
Materials and Methods
Rhode Island’s Drug Overdose Surveillance Program and Center for EMS, both of which are part of RIDOH, partnered closely on this initiative to develop a comprehensive case definition for nonfatal opioid overdose–related EMS runs. First, we approached the Ambulance Service Coordinating Advisory Board to receive input on which RI-EMSIS fields and codes to use in the opioid overdose case definition. In Rhode Island, this advisory board is required to meet quarterly to provide insight on its local EMS branches and communities. 14 The board is composed of municipal leaders, practicing and licensed emergency medical technicians, active physicians, firefighters, emergency nurses, members of the general public, members appointed by the speaker of the house and senate president, professional ambulance service providers, and regional EMS representatives from each Rhode Island county. EMS representatives from this advisory board provided input on the RI-EMSIS fields that are used in the case definition for opioid overdose–related EMS runs. In addition, these representatives helped identify the primary and secondary impression codes that are incorporated into the case definition, considering codes commonly used in the field to identify opioid overdose–related EMS runs. Based on this input, we used the following RI-EMSIS fields to develop the case definition for nonfatal opioid overdose–related EMS runs: primary impression, secondary impression, the dropdown field for medications given, the dropdown field for medications response, the field for naloxone given before EMS arrival, the field for naloxone administrator, and the report narrative.
To develop the case definition, we used RI-EMSIS data on EMS runs from 2018 that mentioned naloxone. To align with Centers for Disease Control and Prevention reporting requirements, we excluded data on EMS runs if they were an interfacility transfer, if the patient was aged ≤10, if no treatment was required, if the patient was biologically dead upon arrival, if the EMS run was canceled, and/or if no patient contact was made. A senior epidemiologist (L.L.) and the chief of the Center for EMS (J.R.) used an iterative process of jointly reviewing 20-50 EMS runs that met the case definition and 20-50 EMS runs that mentioned naloxone but did not meet the case definition each week. Through these weekly meetings, the case definition was gradually established and modified.
We defined the sensitivity and specificity of the case definition among 400 randomly selected EMS runs in which naloxone was mentioned (200 of which met the case definition and 200 of which did not meet the case definition), using medical record review as the gold standard. During this process, 2 reviewers independently reviewed all medical records and categorized EMS runs accordingly. In cases in which the reviewers disagreed, a third reviewer (J.R.) determined the final categorization. To validate the case definition, we compared the number of opioid overdose–related EMS runs in each month of 2018 with the number of emergency department (ED) visits for suspected opioid overdose reported through RIDOH’s 48-Hour Reporting System. 15
Finally, we characterized opioid overdose–related EMS runs identified using our case definition, including the frequency of repeat overdose. To identify people with repeat overdoses, we created unique identifications using the following patient information: first 5 letters of last name, first letter of first name, date of birth, and sex. We excluded data on people with unknown or invalid names (eg, Jane Doe) or missing date of birth.
Based on input from the Ambulance Service Coordinating Advisory Board and the internal case definition development process, we defined an EMS run as nonfatal opioid overdose–related if it met 1 of 5 criteria: (1) the primary or secondary impression is overdose related AND naloxone is in the dropdown field for medication given, (2) the primary or secondary impression is overdose related AND terms for both naloxone AND unresponsive are in the narrative report, (3) naloxone is in the dropdown field for medication given AND medication response is improved, (4) the terms for both naloxone and unresponsive are in the narrative report AND medication response is undocumented (excludes values of no change or worse), or (5) naloxone was given before EMS arrival AND who administered it is not a null value (Table). We completed all analyses using SAS version 9.4 (SAS Institute Inc). This response was part of RIDOH’s response to the opioid overdose epidemic in Rhode Island and did not require institutional review board approval.
Table.
Rhode Island Emergency Medical Services (EMS) Information System case definition for an opioid overdose–related EMS run, Rhode Island, 2018
| Case definition components | Case definition scenario a | ||||
|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | |
| Primary impression is 1 of the following ICD-10 codes: opioid-related disorders (F11); opioid abuse with intoxication, uncomplicated (F11.120); poisoning by other opioids, accidental (unintentional) (T40.2X1); poisoning by heroin, undetermined (T40.1X4); OR secondary impression contains 1 of the following: opioid or heroin b | ✓ | ✓ | |||
| Medication given: naloxone listed | ✓ | ✓ | |||
| Medication response | Improved | No value entered | |||
| Narrative indicates 1 of the following naloxone terms: Narcan or naloxone AND 1 of the following terms: unresponsive, apneic, apenia, semi-cons, semi cons, uncons, agonal | ✓ | ✓ | |||
| Naloxone before EMS: Naloxone administered before EMS arrived: yes AND who administered naloxone before EMS is not a null value | ✓ | ||||
Abbreviation: ICD-10, International Classification of Diseases, Tenth Revision.
aIf any of the case scenarios are met, the EMS run meets the case definition.
bWorld Health Organization. 16
Results
Using RI-EMSIS data from 2018, we identified 2513 EMS runs in which naloxone was administered, of which 1501 met the case definition for a nonfatal opioid overdose. Among the opioid overdose–related EMS runs that met the case definition, the location of information on naloxone administration varied within the medical record, with 49% selecting naloxone in the dropdown field of medications given and mentioning it in the narrative report, 4% selecting naloxone administration in the dropdown field only, and 47% reporting naloxone administration in the narrative report only. We observed a similar distribution in the location of information on naloxone administration within the medical record among EMS runs that did not meet the case definition (Figure 1).
Figure 1.
Location of naloxone information in the Rhode Island Emergency Medical Services (EMS) Information System reports for EMS runs in which naloxone was used, stratified by inclusion in the opioid overdose–related EMS run case definition, Rhode Island, 2018. Data source: Rhode Island Emergency Medical Services Information System.
Of 400 EMS runs that mentioned naloxone, the sensitivity of the RIDOH case definition was 90.0%, indicating that it accurately identified 9 of 10 true opioid overdose–related EMS runs (based on medical record review). The specificity of the case definition was 83.3%, indicating that it accurately excluded about 8 of 10 EMS runs that were not related to opioid overdose. When validating our case definition against other surveillance data on opioid overdoses, we found that the number of nonfatal opioid overdose–related EMS runs based on our case definition was similar to the number of ED visits for suspected opioid overdose by month in 2018 (1560 ED admissions vs 1502 EMS runs in 2018; Figure 2). Differences between the 2 measures can likely be explained by patient transport refusal (16 overdose patients in 2018), differences in overdose case definitions, and people arriving at the ED through other transport pathways.
Figure 2.
Number of emergency department (ED) admissions for suspected opioid overdose and opioid overdose–related emergency medical services (EMS) runs in Rhode Island, by month, 2018. Data sources: Emergency Department 48-Hour Reporting System, Rhode Island Emergency Medical Services Information System.
Of all EMS runs in 2018 that met the case definition for an opioid overdose, most (67.5%) were for male patients. The highest percentage of patients were aged 25-34 (33.7%), followed by aged 35-44 (21.8%), 45-54 (15.3%), 55-64 (11.6%), 11-24 (11.3%), and ≥65 (5.2%). Overall, 34.1% of these EMS runs were for patients who had experienced the opioid overdose in a public setting, 61.4% occurred in private settings, and 4.4% occurred in semiprivate settings (eg, hotel, assisted-living facility, prison, hospital, nursing home, residential institution). When looking at repeat opioid overdose–related EMS runs, 13.2% of the runs in 2018 were for a person who had an opioid overdose–related EMS run in the previous 12 months (10.5% having 1 previous opioid overdose–related EMS run, 2.7% having ≥2 previous opioid overdose–related EMS runs).
Discussion
In this study, relying on naloxone administration alone would have overestimated the number of opioid overdose–related EMS runs in Rhode Island by 67%. Failing to develop an accurate case definition that excludes people who were administered naloxone but did not experience an opioid overdose could dramatically overestimate the number of overdoses occurring in the community. Moreover, an inaccurate case definition may prevent states from appropriately defining geographic locations with increased opioid overdose activity and reduce the efficiency and effectiveness of targeted prevention and harm reduction efforts. 10
When developing an opioid overdose case definition, ensuring the adoption of the definition is essential. In addition to involving EMS personnel in the formation of the case definition and having the Drug Overdose Surveillance Program and Center for EMS jointly lead the development process, RIDOH provided quarterly drug overdose reports to EMS staff members. These reports included the number of opioid overdose–related EMS runs their EMS department responded to and the number of opioid overdose–related EMS runs that occurred in their municipality, with the goal of increasing awareness of the case definition and improving data quality and completeness. Furthermore, metrics RIDOH provided to each EMS department on their performance, including the percentage of opioid overdose–related EMS runs that included (1) naloxone in the dropdown field for medications given and narrative report, (2) complete information on medication response when naloxone was given, (3) complete information on how medication was administered when naloxone was given, and (4) complete information on whether the client was linked with a recovery coach (the latter is now managed by EDs in Rhode Island rather than EMS providers). In addition, to obtain local buy-in and develop a sense of shared value in the data that were created, RIDOH shared the opioid case definition on various platforms, including reports on the Rhode Island Advisory Notification System and the RIDOH website, distribution to agency chiefs and EMS coordinators, presentation at the annual EMS exposition, and inclusion in personal training plans for EMS license renewal.
Since development of the case definition, RI-EMSIS data have allowed RIDOH to more effectively respond to the opioid overdose epidemic in Rhode Island. As an example, in April 2019 a spike in nonfatal overdoses was detected in Woonsocket, Rhode Island. 10 Using data from RI-EMSIS, RIDOH was able to rapidly generate heat maps documenting where recent overdoses were occurring in the community. This information was then used by the community to guide its response efforts, which included the distribution of fentanyl test strips, naloxone, and needle exchange services; deployment of peer recovery specialists; and placement of messaging in high-traffic areas.
RIDOH has also used opioid overdose data from RI-EMSIS during Community Overdose Engagement summits. At these annual summits, RIDOH invites more than 300 community leaders to come together to collaboratively address the opioid overdose epidemic. Before this event, detailed surveillance reports are generated for each municipality that include hotspot maps of opioid overdose–related EMS runs in their community using RI-EMSIS data. At these meetings, community leaders use these reports to develop local response plans. The hotspot maps have led RIDOH and community leaders to engage with local businesses to increase geographically targeted messaging and improve awareness of services such as support for substance use disorder, availability of medication-assisted treatment for opioid use disorder, and naloxone.
EMS and ED data in Rhode Island closely align; however, using EMS rather than ED data has 3 advantages. First, EMS data provide incident location information, which can be used to identify hotspots and direct harm reduction efforts. Second, in contrast to ED data, EMS data provide patient identifiers, which can be used to describe the demographic characteristics of patients experiencing an overdose and can monitor and detect repeat overdoses. Third, ED data are currently reported to the state through the 48-Hour Reporting system, in which hospitals are required to manually report all overdose admissions to RIDOH within 48 hours of admission. By using EMS data, RIDOH can receive these data in near–real time and not rely on manual reporting from hospitals, which could cause some outbreaks to be missed. However, RIDOH is in the process of validating an entirely automated system that would identify overdose patients directly from medical records. It is likely that once operational, both reporting systems should closely align and could capture data on all overdoses in Rhode Island for which medical care was sought.
Since we developed the case definition to identify nonfatal opioid overdose–related EMS runs, other groups have also developed opioid case definitions using multivariable logistic regression 17 and natural language processing. 18 When compared with the modeled definition, our sensitivity was substantially higher (90% vs 45%) in identifying opioid overdose–related EMS runs. 17 Although the natural language processing algorithm correctly identified 98.6% of opioid misuse cases, because of the differences in our definitions (opioid overdose vs opioid misuse), it is difficult to compare results. 18
One strength of this work is the generalizability of the case definition. Because RI-EMSIS is Rhode Island’s user interface with the US national EMS reporting system (NEMSIS), most variables used in this definition should be collected and reported by other states in a similar format, which should increase the ease of adoption for other public health jurisdictions. However, because states can create custom variables and are not required to report or collect all NEMSIS-required variables, the case definition may need to be modified before adoption in some jurisdictions. In addition, this case definition relies on certain terms being present in the case narrative that may be affected by regional terminology. To ensure accuracy, public health jurisdictions should evaluate whether additional words or phrases may need to be added or removed from this definition before adoption.
This work had at least 2 limitations. First, data are often entered inconsistently into RI-EMSIS, which may result in the misclassification of some EMS runs. However, the general alignment of counts and trends in opioid overdoses based on RI-EMSIS data and ED data is encouraging. Nonetheless, training of EMS providers to help improve data quality is ongoing. Second, because NEMSIS undergoes updates, the case definition will need to be modified and validated to ensure that data on opioid overdose–related EMS runs are accurately captured. Although the case definition was originally developed using NEMSIS version 2.2.1, ED and EMS data began to fall out of alignment when NEMSIS version 3.4 was fully adopted; this version update resulted in RIDOH revisiting the opioid overdose case definition and revising in 2019.
Practice Implications
EMS data can be an effective tool for monitoring overdoses in real time, providing geographic information on where overdoses are occurring, and allowing for the identification of populations at risk for repeated opioid overdoses. This work highlights the importance of developing a comprehensive case definition to accurately identify opioid overdoses and provides the methodology and a case definition that could be adopted or adapted in other jurisdictions. The ability to rapidly identify geographic regions in which opioid overdoses are occurring can enable public health organizations to better serve their communities and more effectively address the opioid overdose epidemic. As EMS protocols and coding systems change, RIDOH is committed to reviewing and modifying the case definition to better identify opioid overdose–related EMS runs and respond to the overdose epidemic in Rhode Island.
Footnotes
Declaration of Conflicting Interests: The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding: The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The Enhanced State Opioid Overdose Surveillance program's cooperative agreement from the Centers for Disease Control and Prevention funded the development of this case definition.
ORCID iD
Benjamin D. Hallowell, PhD https://orcid.org/0000-0002-6943-9615
References
- 1. Centers for Disease Control and Prevention . 2016 Drug overdose death rates. 2017. Accessed June 30, 2020. https://www.cdc.gov/drugoverdose/data/statedeaths/drug-overdose-death-2016.html
- 2. McDonald JV. Using the Rhode Island prescription drug monitoring program (PMP). R I Med J (2013). 2014;97(6):64-65. [PubMed] [Google Scholar]
- 3. Bowman S., Engelman A., Koziol J., Mahoney L., Maxwell C., McKenzie M. The Rhode Island community responds to opioid overdose deaths. R I Med J (2013). 2014;97(10):34-37. [PubMed] [Google Scholar]
- 4. McCormick M., Koziol J., Sanchez K. Development and use of a new opioid overdose surveillance system, 2016. R I Med J (2013). 2017;100(4):37-39. [PubMed] [Google Scholar]
- 5. Jiang Y., McDonald JV., Koziol J., McCormick M., Viner-Brown S., Alexander-Scott N. Can emergency department, hospital discharge, and death data be used to monitor burden of drug overdose in Rhode Island? J Public Health Manag Pract. 2017;23(5):499-506. 10.1097/PHH.0000000000000514 [DOI] [PubMed] [Google Scholar]
- 6. Marshall BD., Yedinak JL., Goyer J., Green TC., Koziol JA., Alexander-Scott N. Development of a statewide, publicly accessible drug overdose surveillance and information system. Am J Public Health. 2017;107(11):1760-1763. 10.2105/AJPH.2017.304007 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Waye KM., Yedinak JL., Koziol J., Marshall BDL. Action-focused, plain language communication for overdose prevention: a qualitative analysis of Rhode Island’s overdose surveillance and information dashboard. Int J Drug Policy. 2018;62:86-93. 10.1016/j.drugpo.2018.08.010 [DOI] [PubMed] [Google Scholar]
- 8. Samuels EA., McDonald JV., McCormick M., Koziol J., Friedman C., Alexander-Scott N. Emergency department and hospital care for opioid use disorder: implementation of statewide standards in Rhode Island, 2017-2018. Am J Public Health. 2019;109(2):263-266. 10.2105/AJPH.2018.304847 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Rhode Island Code of Regulations, § 216-RICR-20-10-2, Title 216 (2018).
- 10. Lasher L., Hallowell BD., Chambers LC. et al. Using timely overdose data to address a spike in nonfatal overdoses and inform a coordinated community-level response in Rhode Island, 2019. Public Health Rep. 2021;136(suppl 1):24S-30S. 10.1177/00333549211012407 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Lasher L., Rhodes J., Viner-Brown S. Identification and description of non-fatal opioid overdoses using Rhode Island EMS data, 2016-2018. R I Med J (2013). 2019;102(2):41-45. [PubMed] [Google Scholar]
- 12. Pesarsick J., Gwilliam M., Adeniran O., Rudisill T., Smith G., Hendricks B. Identifying high-risk areas for nonfatal opioid overdose: a spatial case-control study using EMS run data. Ann Epidemiol. 2019;36:20-25. 10.1016/j.annepidem.2019.07.001 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Knowlton A., Weir BW., Hazzard F. et al. EMS runs for suspected opioid overdose: implications for surveillance and prevention. Prehosp Emerg Care. 2013;17(3):317-329. 10.3109/10903127.2013.792888 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Rhode Island Code of Regulations, § 23-4.1-2, Title 23 (2016).
- 15.Rhode Island Code of Regulations, § 216-RICR-20-20-5, Title 216 (2014).
- 16. World Health Organization . International Statistical Classification of Diseases and Related Health Problems, Tenth Revision. 2nd ed. Accessed April 19, 2021. https://apps.who.int/iris/handle
- 17. Schilke RE., Card K., Jiang J., Sturms J., McCoy S., Colston L. Validating syndromic data for opioid overdose surveillance in Florida. Online J Public Health Inform. 2019;11(1):e268. 10.5210/ojphi.v11i1.9779 [DOI] [Google Scholar]
- 18. Prieto JT., Scott K., McEwen D. et al. The detection of opioid misuse and heroin use from paramedic response documentation: machine learning for improved surveillance. J Med Internet Res. 2020;22(1):e15645. 10.2196/15645 [DOI] [PMC free article] [PubMed] [Google Scholar]