Development and Validation of a Syndrome Definition for Suspected Nonfatal Unintentional/Undetermined Intent Stimulant-Involved Overdoses

Abstract

Objectives

To monitor stimulant-involved overdose (SOD) trends, the Centers for Disease Control and Prevention (CDC) developed and evaluated the validity of a syndromic surveillance definition for suspected nonfatal, unintentional/undetermined intent stimulant-involved overdose (UUSOD).

Methods

We analyzed all emergency department (ED) visits in CDC’s surveillance system that met the UUSOD syndrome definition (January 2018–December 2019). We classified visits as true positive, possible, or not UUSODs after reviewing diagnosis codes and chief complaints. We first assessed whether visits were acute SODs, subsequently classifying acute SODs by intent. The percentage of true-positive UUSODs did not include intentional or possibly intentional visits. We considered all visits with UUSOD diagnosis codes to be acute SODs and reviewed them for intent. We manually reviewed and double-coded a 10% random sample of visits without UUSOD diagnosis codes using decision rules based on signs and symptoms. The overall percentage of true-positive UUSODs was a weighted average of the percentage of true-positive UUSODs based on diagnosis codes and the percentage of true-positive UUSODs determined by manually reviewing visits without codes.

Results

During 2018-2019, 40 045 ED visits met the syndrome definition for UUSOD. Approximately half (n = 18 793; 46.9%) of 40 045 visits had UUSOD diagnosis codes, indicating acute SOD; of these, 98.6% (n = 18 534) were true-positive UUSODs. Of 2125 manually reviewed visits without UUSOD diagnosis codes, 32.6% (n = 693) were true-positive UUSODs, 54.2% (n = 1151) were possible UUSODs, and 13.2% (n = 281) were not UUSODs. Overall, 63.6% of visits were true-positive UUSODs, 29.3% were possible UUSODs, and 7.1% were not UUSODs.

Practice Implications

CDC’s UUSOD definition may assist in surveillance efforts with further refinement to capture data on SOD clusters and trends.

The United States is experiencing an evolving drug overdose crisis. Interrelated with increases in fatal opioid-involved overdoses (particularly those involving synthetic opioids other than methadone, eg, illicitly manufactured fentanyl), stimulant-involved overdose (SOD) mortality rates have recently increased. ¹ These SOD deaths included cocaine and other psychostimulants (eg, methamphetamine), with increases in SOD mortality rates occurring with and independent of co-involvement with opioids. ² In addition, rates of nonfatal SODs treated in US emergency departments (EDs) have also increased. From 20182019, nonfatal overdose rates involving cocaine and amphetamines increased by 11.0% and 18.3%, respectively, in EDs from 29 states. ³

Given these increases, the Centers for Disease Control and Prevention (CDC) expanded surveillance efforts to monitor nonfatal SODs to identify overdose clusters, track trends, and inform public health prevention and response. Through the Overdose Data to Action cooperative agreement, CDC works closely with jurisdictions to improve the reporting of nonfatal drug overdoses using ED syndromic surveillance data. ^4,5 Syndromic surveillance systems allow for rapid identification of suspected overdoses using a combination of electronic health record diagnosis codes and qualitative, free-text data. ^6,7 These data help guide local resource mobilization and prevention efforts. ⁷

CDC’s Drug Overdose Surveillance and Epidemiology (DOSE) system ⁸ uses data in the National Syndromic Surveillance Program (NSSP) ⁹ to monitor trends in suspected drug overdose ED visits. NSSP is a collaboration between CDC and state and local health departments supporting the collection and analysis of electronic health data from visits to EDs, inpatient hospitals, urgent care centers, and laboratories. ^9,10 Some syndromic surveillance data are available in NSSP within 1 or 2 days after patients present to EDs, providing opportunities for early local detection of potential outbreaks of nonfatal drug overdoses. ⁹ Epidemiologists develop queries in NSSP to capture ED visits for specified syndrome definitions. For DOSE, CDC developed a syndrome definition for suspected nonfatal, unintentional/undetermined intent stimulant-involved overdose (UUSOD) in partnership with state and local jurisdictions. ⁸ The syndrome definition included both illicit and prescription stimulants. Although the stimulant syndrome definition was developed to capture UUSOD (ie, SOD without evidence of self-harm), we did not explicitly exclude ED visits for intentional SODs.

A comprehensive validation of this definition is warranted, to determine the extent to which the syndrome definition accurately identifies UUSODs and to assess its utility to monitor trends and inform overdose response and prevention efforts. Our objective was to determine the percentage of suspected nonfatal UUSODs that were true-positive UUSODs.

Materials and Methods

ED data in this study originate from CDC’s DOSE system, ⁸ which uses ED data in NSSP. ⁹ More than 3000 EDs submit data to NSSP, representing approximately 70% of ED visits in the United States as of February 2020. ⁹ States participating in the DOSE system must meet the requirement that their ED syndromic data cover ≥75% of all ED visits in their state. NSSP includes a chief complaint field, with text data describing the reason for the patient’s ED visit, and a discharge diagnosis field, with standardized diagnosis codes categorizing a patient’s clinical diagnosis. A concatenated “chief complaint/discharge diagnosis” field is made up of data from the chief complaint and discharge diagnosis fields.

Development of Suspected UUSOD Syndrome Definition

In 2019, CDC developed a UUSOD syndrome definition and corresponding NSSP query ⁸ (Table 1). We identified diagnosis codes including International Classification of Diseases, Ninth Revision, Clinical Modification ¹² and International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) ¹¹ and Systematized Nomenclature of Medicine–Clinical Terms (SNOMED-CT) ¹³ indicating an acute stimulant poisoning. We included ICD-10-CM codes indicating “initial encounter” (eg, codes with a seventh character of “A”) and did not include diagnosis codes for stimulant use or abuse (eg, F14 codes for cocaine-related disorders) in the syndrome definition.

Table 1.

Components of the Centers for Disease Control and Prevention’s (CDC’s) syndrome definition for suspected unintentional/undetermined intent stimulant-involved overdose (UUSOD), version 3 ^a

Criteria for inclusion	Variable	Specific terms, including diagnosis codes, with and without periods, abbreviations, and some common misspellings
Automatic inclusion	Discharge diagnosis, ICD-9-CM 12 poisoning	969.70, 969.71, 969.72, 969.73, 969.79, 970.0, 970.81, 970.89, 970.9, E854.2, E854.3, E855.2. Also included terms with no period (eg, 97080)
	Discharge diagnosis, ICD-10-CM 11 poisoning	T40.5X1A, T40.5X4A, T43.601A, T43.604A, T43.611A, T43.614A, T43.621A, T43.624A, T43.631A, T43.634A, T43.641A, T43.644A, T43.691A, T43.694A. Also included terms with no period (eg, T405X1A)
	Discharge diagnosis, SNOMED-CT 13	296321004, 441527004, 296290006, 61803000
Inclusion in combination with a stimulant term	Chief complaint—overdose term	Poisoning (poison), overdose (overdose, overdoes, averdose, averdoes, over does, overose), nodding off, snort, ingestion (ingest, injest), intoxication (intoxic), unresponsive (unresponsiv), loss of consciousness (syncopy, syncope), shortness of breath (SOB), short of breath, altered mental status (AMS)
Inclusion in combination with an overdose term	Chief complaint—stimulant term	Speedball, cocaine, coke, crack, amphetamine [will catch methamphetamine], meth, crystal meth, lisdexamfetamine, dextroamphetamine, levoamphetamine, biphetamine, dexedrine, Adderal, Vyvanse, Concerta, Methylin, Ritalin, cathinone, bath salt, stimulant, MDMA, ecstasy, Molly
Exclusions (only applied to visits not included automatically using discharge diagnosis codes)	Chief complaint	Chief complaint text containing any of the following phrases: Crackle, crackled, cracked, cracking, cracks, crackles, crackling, crackinling, crackiling, cracker, crackers, crackels, crackly, mccracken, buttcrack, a crack, crack in, knee crack, shoulder crack, butt crack, ankle crack, callous crack, crack due to, pop crack, loud crack, it crack, heard crack, felt crack, rib and crack, fell crack, snap crack, growth plate, to crack of, crack sidewalk, sidewalk crack, tooth, teeth, hands cracke, heard something, nail, cracky, crack area, crack cast, back crack, crack of, crack his, lip and crack, my crack, glass crack, buttock and crack, crack on, crack to, foot crack, no loss of consciousness, denied/denies loss of consciousness, negative loss of consciousness, denies any loss of consciousness, denies drug, deny drug, denied drug, denying drug, denies any drug, with dra, withdra, detoxification, detos, detoz, dtox, coker, coke bottle, coke can, coke case, case of coke, rum and coke, jack and coke, coke tea, coffee coke, delirium tremens coke, drink a coke, jack coke, dirnking coke, coke tab, scotch coke, drinking coke, aspirating on coke, coke colored, drank coke, cardiac-stimulant, cardiac stimulant, meth germantown, meth gmt, meth odist, meth obstet, meth group
CDC query for suspected stimulant-involved overdose (version 3) b		(,^l;/ ]969.7[01239][;/]^,or,^[;/ ]9697[01239][;/]^,or,^[;/ ]970.0^,or,^[;/ ]9700[;/]^,or,^[;/ ]970.8[19][;/]^,or,^[;/ ]9708[19][;/]^,or,^[;/ ]970.9[;/]^,or,^[;/ ]9709[;/]^,or,^[;/ ]E854.[23][;/]^,or,^[;/ ]E854[23][;/]^,or,^[;/ ]E855.2[;/]^,or,^[;/ ]E8552[;/]^,or,^[;/ ]T40.5X1A^,or,^[;/ ]T405X1A^,or,^[;/ ]T40.5X4A^,or,^[;/ ]T405X4A^,or,^[;/ ]T43.6[012349][14]A^,or,^[;/ ]T436[01239][14]A^,or,^[;/ ]296321004;^,or,^[;/ ]441527004;^,or,^[;/ ]296290006;^,or,^[;/ ]61803000;^,),or,(,(,(,^poison^,or,^verdo[se][es]^,or,^over dose^,or,^overose^,or,^nodding^,or,^ nod ^,or,^snort^,or,^in[gj]est^,or,^intoxic^,or,^unresponsiv^,or,^loss of consciousness^,or,^syncop^,or,^shortness of breath^,or,^short of breath^,or,^altered mental status^,),and,(,^speedball^,or,^speed ball^,or,^coc[ai][ia]ne^,or,^cocc[ai][ia]ne^,or,^cocane^,or,^cocanne^,or,^coke^,or,^crack^,or,^ meth ^,or,^amphetamine^,or,^c[ry][yr]stal meth^,or,^lisdexamfetamine^,or,^dextroamphetamine^,or,^levoamphetamine^,or,^biphetamine^,or,^dexedrine^,or,^adderal^,or,^aderal^,or,^vyvanse^,or,^concerta^,or,^methylin^,or,^ritalin^,or,^cathinone^,or,^bath salt^,or,^bathsalt^,or,^stimulant^,or,^ MDMA ^,or,^e[sc][cs]ta[cs]y^,or,^e[xsc]ta[sc]y^,or,^ex[sc]ta[sc]y^,or,^ moly ^,or,^ moll[iy] ^,),),andnot,(,^crackle^,or,^crackled^,or,^cracked^,or,^cracking^,or,^cracks^,or,^crackles^,or,^crackling^,or,^crackinling^,or,^crackiling^,or,^cracker^,or,^crackers^,or,^crackels^,or,^crackly^,or,^mccracken^,or,^buttcrack^,or,^a crack^,or,^crack in^,or,^knee crack^,or,^shoulder crack^,or,^butt crack^,or,^ankle crack^,or,^callous crack^,or,^crack due to^,or,^pop crack^,or,^loud crack^,or,^it crack^,or,^heard crack^,or,^felt crack^,or,(,^rib^,and,^crack^,),or,^fell crack^,or,^snap crack^,or,^growth plate^,or,^to crack of^,or,^crack sidewalk^,or,^sidewalk crack^,or,^tooth^,or,^teeth^,or,^hands cracke^,or,^heard something^,or,^nail^,or,^cracky^,or,^crack area^,or,^crack cast^,or,^back crack^,or,^crack of^,or,^crack his^,or,(,^lip^,and,^crack^,),or,^my crack^,or,^glass crack^,or,(,^buttock^,and,^crack^,),or,^crack on ^,or,^crack to ^,or,^foot crack^,or,^no loss of consciousness^,or,^denie[sd] loss of consciousness^,or,^negative loss of consciousness^,or,^denies any loss of consciousness^,or,^denies drug^,or,^deny drug^,or,^denied drug^,or,^denying drug^,or,^denies any drug^,or,^ with dra ^,or,^withdra^,or,^detoxification^,or,^detos^,or,^detoz^,or,^dtox^,or,^coker^,or,^coke bottle^,or,^coke can^,or,^coke case^,or,^case of coke^,or,(,^rum^,and,^coke^,),or,^jack and coke^,or,^coke tea^,or,^coffee coke^,or,^delirium tremens coke^,or,^drink a coke^,or,^jack coke^,or,^dirnking coke^,or,^coke tab^,or,^scotch coke^,or,^drinking coke^,or,^aspirating on coke^,or,^coke colored^,or,^drank coke^,or,^cardiac-stimulant^,or,^cardiac stimulant^,or,^meth germantown^,or,^meth gmt^,or,^meth odist^,or,^meth obstet^,or,^meth group^,),)

Abbreviations: ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification; ICD-10-CM, International Classification of Diseases, Tenth Revision, Clinical Modification; MDMA, 3,4-methylenedioxymethamphetamine; NSSP, National Syndromic Surveillance Program; SNOMED-CT, Systematized Nomenclature of Medicine–Clinical Terms.

^aBased on feedback from state and local health department partners, the CDC suspected UUSOD syndrome definition was revised twice. The current version, version 3, improved on previous versions (1 and 2) by adding more drug names and abbreviations (eg, all variations of the term “molly”), a new ICD-10-CM ¹¹ code for ecstasy poisoning that was implemented in 2019, and additional exclusion terms (noted after the “andnot” term).

^bAs described by NSSP, ⁹ these queries are not case sensitive and employ Boolean logic and regular expression search functions. Parentheses to group terms, brackets to allow for transposition of characters (eg, [se]), and wildcards (eg, ^) can be applied to improve search capabilities. For additional information, see NSSP resources for free-text coding: https://wwwcdcgov/nssp/tech-tips/free-text-coding/part1html.

Next, we identified overdose terms (eg, poisoning, intoxication) and stimulant drug terms (eg, cocaine, methamphetamine, ecstasy) indicating that a stimulant was involved in the overdose. We used a combination of diagnosis codes and text to identify suspected UUSODs. If a diagnosis code indicating a UUSOD was present, we classified the ED visit as a suspected UUSOD. In the absence of these diagnosis codes, we classified the visit as a suspected UUSOD only if the chief complaint had both an overdose term and a stimulant drug term. We applied a list of exclusions to the chief complaint text (eg, denies stimulant ingestion) when a diagnosis code was absent. This study evaluated version 3 of the syndrome definition, which is the most recent version used in analysis and reporting for CDC’s DOSE system. ⁸ Version 3 of the syndrome definition improved upon previous versions by adding more drug slang terms, negation terms, and newly implemented ICD-10-CM codes.

We completed our analysis in 2 stages. An initial test stage included ED visits from a subset of DOSE states, with data analyzed in multiple steps to inform refinement of visit classification decision rules. The second stage included validation of the UUSOD syndrome definition using a larger sample of visits retrieved from all DOSE jurisdictions sharing data with NSSP as of February 2020.

Stage 1: Development and Testing of Decision Rules

Development of decision rules

Stage 1 of the validation process involved developing, testing, and refining decision rules on how to classify suspected UUSODs. Using these rules, we first classified each visit as an acute SOD, a possible acute SOD, or not an acute SOD. We developed decision rules after reviewing the literature for information on clinical signs and symptoms (hereinafter, “symptoms”) of acute SODs ^{14 -18} and consulting with CDC physicians experienced in treating drug overdoses. We created a first draft of the decision rules in collaboration with an internal medicine/preventive medicine physician. A second internal medicine/primary care physician reviewed these rules; subsequently, we revised the decision rules after group discussion with both physicians.

We based decision rules on the presence of overdose terms, symptoms, comorbidities, other diagnoses, and other substances (Table 2). In collaboration with CDC physicians experienced in internal medicine, preventive medicine, primary care, and pediatrics, we developed tables cataloging the symptoms of acute SOD for adults aged ≥18 (Supplemental Table S1 online only) and for children and adolescents aged <18 years (Supplemental Table S2 online only). We used these tables listing symptoms of SOD along with final decision rules while reviewing and coding visits. Both tables classified potential symptoms of acute SOD by physiological type (eg, psychiatric, cardiovascular) and specificity. In the presence of stimulant ingestion, a single specific symptom (eg, myocardial infarction) was typically enough to classify a visit as an acute SOD. In contrast, a less specific symptom (eg, anxiety) typically did not provide strong enough evidence to classify a visit as an acute SOD, because nonspecific symptoms could indicate other conditions. Because the effects of stimulant ingestion can last for several days, we allowed up to 48 hours between stimulant ingestion and the occurrence of overdose symptoms to consider a visit as an acute SOD. Symptoms occurring >48 hours after ingestion were not considered specific or reliable indicators of acute SOD (Table 2). Notably, no specific NSSP variable defines the exposure period (ie, the length of time between stimulant ingestion and symptom onset), and most visits lacked detailed information on the exposure period in the text for chief complaint.

Table 2.

Final Centers for Disease Control and Prevention (CDC) decision rules for classification of suspected stimulant-involved overdoses as acute stimulant-involved overdoses ^a

Stimulant intoxication, ingestion, use, abuse, inhalation (and other modes of administration that could be mentioned)	Acute stimulant-involved overdose
General decision rules
+ the word “overdose” or “poisoning”	Yes
+ “possible overdose” or “possible poisoning”	Yes
+ plan to overdose in future	No
+ diagnosis code for overdose/poisoning by unknown/unstated drug(s)b	Yes
+ diagnosis code for intentional stimulant-involved overdose/poisoning c	Yes
+ heat stroke	Possible
+ unresponsive, loss of consciousness, or “found down”	Yes f
+ unresponsive/unconscious + no additional specific symptoms of stimulant-involved overdose d + diagnosis code for (initial) encounter due to poisoning by opioids e + no mention of opioid ingestion in chief complaint	Possible g
+ benzodiazepine administered + no indication that benzodiazepine was administered to address a chronic mental health condition	Yes
+ benzodiazepine administered + a positive indication that benzodiazepine was administered to address a chronic mental health condition	Possible
+ aged <2 + consumed ≥1 pill ecstasy/MDMA/molly	Yes
Psychiatric symptoms d
+ ≥1 specific psychiatric symptom + no mention of schizophrenia or alcohol or drug withdrawal/decreased use/delirium tremens + no F28/F29 code h	Yes
+ ≥1 nonspecific psychiatric symptom + no mention of schizophrenia or alcohol or drug withdrawal/decreased use/delirium tremens + no F28/F29 code h	Possible
+ ≥1 psychiatric symptom + schizophrenia	Possible
+ ≥1 psychiatric symptom + alcohol or drug withdrawal/decreased use/delirium tremens	Possible
+ psychosis + F28 or F29 code h	Possible
Other neurological symptoms d
+ age <65 + ≥1 specific other neurological symptom + no mention of plausible alternative neurological diagnosis (eg, epilepsy/seizure disorder) + no mention of alcohol or drug withdrawal/decreased use/delirium tremens	Yes
+ age <65 + ≥1 nonspecific other neurological symptom + no mention of plausible alternative neurological diagnosis (eg, epilepsy/seizure disorder) + no mention of alcohol or drug withdrawal/decreased use/delirium tremens	Possible
+ age ≥65 + ≥1 other specific neurological symptom besides cerebral edema/cerebral herniation + no mention of plausible alternative neurological diagnosis (eg, epilepsy/seizure disorder) + no mention of alcohol or drug withdrawal/decreased use/delirium tremens	Yes
+ age ≥65 + cerebral edema/cerebral herniation + the ordering of events is clear (eg, stimulant ingestion immediately preceded cerebral edema/herniation)	Yes
+ age ≥65 + cerebral edema/cerebral herniation + the ordering of stimulant ingestion and cerebral edema/herniation is not clear	Possible
+ age ≥65 + ≥1 other nonspecific neurological symptom besides cerebral edema/cerebral herniation + no mention of plausible alternative neurological diagnosis + no mention of alcohol or drug withdrawal/decreased use/delirium tremens	Possible
+ ≥1 other neurological symptom (specific or nonspecific) + a plausible alternative neurological diagnosis	Possible
+ ≥1 other neurological symptom (specific or nonspecific) + alcohol or drug withdrawal/decreased use/delirium tremens	Possible
Cardiovascular symptoms d
+ age <65 + ≥1 cardiovascular symptom + no mention of alcohol or drug withdrawal/decreased use/delirium tremens (consider alternative plausible diagnoses)	Yes
+ age ≥65 + ≥1 cardiovascular symptom + the ordering of events is clear (eg, stimulant ingestion immediately preceded cardiac symptoms)	Yes
+ age ≥65 + ≥1 cardiovascular symptom + the ordering of stimulant ingestion and cardiac symptoms is not clear	Possible
+ any age + ≥1 cardiovascular symptom + alcohol or drug withdrawal/decreased use/delirium tremens	Possible
Respiratory symptoms d
+ ≥1 respiratory symptom + no mention of asthma or chronic obstructive pulmonary disease diagnosis	Possible
+ ≥1 respiratory symptom + asthma or chronic obstructive pulmonary disease diagnosis	Possible
Gastrointestinal, renal, or electrolyte symptoms d
+ rhabdomyolysis, elevated creatinine phosphokinase, or elevated myoglobin	Yes
Other situations
+ uncertain stimulant ingestion (eg, “possible cocaine ingestion;” paraphernalia found at scene without further confirmation of ingestion; or if unclear whether stimulant use was current or historic/past use)	Possible
+ no mention of the word “overdose” or “poisoning”—only stimulant intoxication/ingestion/use, etc. + no symptoms of stimulant-involved overdose	Possible
+ no mention of the word “overdose” or “poisoning”—only stimulant intoxication/ingestion/use, etc. + symptoms of stimulant use, but not overdose (eg, trouble sleeping) + no symptoms of stimulant-involved overdose	No
+ psychiatric evaluation/mental health evaluation	Possible or no, depending on evidence of acute stimulant-involved overdose
+ “meth” overdose/ingestion + diagnosis code for poisoning by methadone i + no specific symptoms of stimulant-involved overdose d	Possible
+ response to Narcan	Yes, possible, or no j
+ no symptoms of stimulant-involved overdose d + no symptoms of opioid overdose k	Possible
+ unclear timing of overdose symptoms with respect to stimulant ingestion	Possible
+ if an ICD-10-CM F14/F15 code (eg, cocaine abuse uncomplicated) is written out and is the only place that a stimulant is mentioned (ie, the only reason that the record was captured in NSSP is that the F14/F15 code was written out)	Possible or no, depending on other evidence in the chief complaint/discharge diagnosis field (ie, not eligible to be coded as a “Yes” acute stimulant-involved overdose)
+ other non-overdose chief complaint that indicates that there was another primary medical reason for the ED visit (eg, patient with pneumonia and fever of 102 °F)	No
+ age ≤1 year + neonatal abstinence syndrome	No
+ if a record is wrongly captured because of an abbreviation (eg, “meth” as in “Methodist University Hospital”) + no other mention of stimulants	No

Abbreviations: ED, emergency department; ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification; ICD-10-CM, International Classification of Diseases, Tenth Revision, Clinical Modification; MDMA, 3,4-methylenedioxymethamphetamine; NSSP, National Syndromic Surveillance Program; SNOMED-CT, Systematized Nomenclature of Medicine–Clinical Terms; UUSOD, unintentional/undetermined intent stimulant-involved overdose.

^aThis table classifies suspected stimulant-involved overdoses (which were identified using the CDC syndrome definition for suspected UUSOD version 3) as acute stimulant-involved overdoses. The data source for this study was CDC’s Drug Overdose Surveillance and Epidemiology system, ⁸ which leverages available ED data in NSSP. ⁹ NSSP was queried using the CDC syndrome definition for suspected UUSOD version 3 for ED visits occurring from January 2018 through December 2019. A total of 40 045 ED visits met the syndrome definition. These visits were from Washington, DC, and 32 states (Alabama, Arkansas, Colorado, Connecticut, Delaware, Florida, Georgia, Illinois, Kansas, Kentucky, Louisiana, Maryland, Maine, Missouri, Mississippi, Montana, North Carolina, New Jersey, New Mexico, Nevada, New York, Ohio, Oregon, Pennsylvania, Rhode Island, South Carolina, Tennessee, Utah, Virginia, Washington, Wisconsin, and West Virginia). Each scenario is presented independently, but combinations could provide more (or less) evidence of acute stimulant-involved overdose. All available information contained in the chief complaint/discharge diagnosis field was assessed to determine whether there was ample support to code differently.

^bDischarge diagnosis codes for poisoning by unknown/unstated drug(s) include ICD-9-CM ¹² codes 977.[8,9], E858.[8,9], E950.[4,5], and E980.[4,5]; ICD-10-CM ¹¹ codes T50.90[1,2,4]A, T50.91[1,2,4]A, and T50.99[1,2,4]A; and SNOMED-CT ¹³ codes 431307001, 242975003, 269688005, 7895008, 708079007, 59274003, and 460991000124106.

^cDischarge diagnoses for intentional stimulant-involved overdose included ICD-10-CM ¹¹ codes T40.5X2A, T43.602A, T43.612A, T43.622A, T43.632A, T43.642A, and T43.692A, as well as SNOMED-CT ¹³ codes 290544006, 291241005, 291263001, 291266009, 295568001, 296293008, 296319009, 296323001, 296327000, 296331006, and 461061000124108.

^dA list of clinical signs and symptoms of stimulant-involved overdose, categorized by physiology (eg, cardiac symptoms), type (specific vs nonspecific), and age category (children and adolescents aged <18 vs adults aged ≥18 years) is available (Supplemental Tables S1 and S2). In the presence of stimulant ingestion, a single specific symptom was typically enough to classify a visit as an acute stimulant-involved overdose. Less specific symptoms typically did not provide strong enough evidence to classify visits as an acute stimulant-involved overdose.

^eICD-9-CM ¹² codes for opioid poisoning included 965.0[0,1,2,9] and E850.[0,1,2]. ICD-10-CM ¹¹ codes for initial encounters due to opioid poisoning included T40.0X[1,2,4]A, T40.1X[1,2,4]A, T40.2X[1,2,4]A, T40.3X[1,2,4]A, T40.4X[1,2,4]A, T40.60[1,2,4], and T40.69[1,2,4]. SNOMED-CT ¹³ codes for opioid poisoning included 295174006, 295175007, 295176008, 295165009, 242253008, 297199006, 295213004, 290182008, 242828004, 242829007, 60199004, 216464004, 242831003, 290171000, 290172007, 295161000, 295163002, and 295164008.

^fThese visits are generally classified as acute stimulant-involved overdoses, with minor exceptions noted in subsequent decision rules.

^gAlthough a stimulant-involved overdose can cause unconsciousness/unresponsiveness, it is a somewhat rare symptom of stimulant-involved overdose but a common symptom of opioid overdose. Furthermore, if (1) there is a diagnosis code for opioid/narcotic overdose but no diagnosis code for stimulant-involved overdose, and (2) if only a stimulant—but not an opioid—is mentioned in the chief complaint, this may represent an event in which a person mistakenly thinks they took a stimulant but for which it was ultimately determined (via diagnosis codes) that they took an opioid. We coded these rare instances conservatively, classifying them as possibly acute stimulant-involved overdoses.

^hICD-10-CM ¹¹ code F28: “Other psychotic disorder not due to a substance or known physiological condition;” ICD-10-CM ¹¹ code F29: “Unspecified psychosis not due to a substance or known physiological condition.”

ⁱDischarge diagnosis codes for methadone overdose include ICD-9-CM ¹² codes 965.02 and E850.1, ICD-10-CM ¹¹ codes T40.3X[1,2,4]A, and SNOMED-CT ¹³ codes 60199004, 216464004, 242831003, 290171000, 290172007, 295161000, 295163002, and 295164008.

^jThese ED visits were eligible to be coded as acute, possibly acute, or not acute stimulant-involved overdoses, based on other information in the chief complaint/discharge diagnosis field. In other words, response to Narcan did not preclude these visits from being classified as possibly acute or acute and was not a sole deciding factor in classifying these visits as acute stimulant-involved overdoses.

^kSymptoms of opioid overdose could include pinpoint pupils, unconscious/unresponsive, respiratory depression, and/or cyanosis.

A team of CDC scientists coded visits conservatively, classifying records as possible acute SODs if insufficient evidence existed to determine whether records were or were not SODs. Overdoses frequently involved multiple substances; therefore, consistent with the intent of the syndrome definition, we coded for whether overdoses were stimulant-involved.

Although we aimed to capture UUSODs, we could inadvertently capture self-harm/intentional SODs by our syndrome definition. We developed decision rules to classify whether acute SODs were intentional or possibly intentional (Table 3). We excluded intentional or possibly intentional SODs from the final calculations of the percentage of true-positive UUSODs.

Table 3.

Final Centers for Disease Control and Prevention (CDC) decision rules for classification of emergency department (ED) visits as acute intentional or possibly intentional stimulant-involved overdoses^a

Rule number	Scenario	Intentional
Intentional stimulant-involved overdose diagnosis code
1	Discharge diagnosis: intentional stimulant-involved overdose b	Yes
2	Discharge diagnosis: intentional stimulant-involved overdose b + discharge diagnosis/chief complaint: other known drug or unknown drug UUDO c	Yes
UUDO stimulant-involved overdose diagnosis code + intentional overdose diagnosis code (stimulant, other known drug, or unknown drug)
3	Discharge diagnosis: stimulant UUDO d + discharge diagnosis: intentional stimulant-involved overdose a,b (however, if chief complaint states intentional, then code as intentional)	Possible
4	Discharge diagnosis: stimulant UUDO d + discharge diagnosis: intentional other known drug or unknown drug overdose d,e (however, if chief complaint states intentional, then code as intentional)	Possible
5	Discharge diagnosis/chief complaint: intentional stimulant-involved overdose a,b + denies suicidal ideation in chief complaint	Possible
6	Discharge diagnosis: stimulant UUDO d + discharge diagnosis/chief complaint: intentional stimulant-involved overdose b + denies suicidal ideation in chief complaint	No
7	Discharge diagnosis: stimulant UUDO d + discharge diagnosis/chief complaint: intentional other known drug or unknown drug overdose d,e + denies suicidal ideation in chief complaint	No
UUDO stimulant-involved overdose diagnosis code + assault stimulant-involved overdose diagnosis code
8	Discharge diagnosis: stimulant UUDO d + discharge diagnosis: assault stimulant-involved overdose f + chief complaint mention of assault	Possible
9	Discharge diagnosis: stimulant UUDO d + discharge diagnosis: assault stimulant-involved overdose f + no mention of assault in chief complaint	No
Chief complaint stimulant-involved overdose (no intent listed) + intentional overdose diagnosis code for nonstimulant drug
10	Chief complaint: stimulant-involved overdose (intent not listed or inferred) + discharge diagnosis/chief complaint: intentional unknown drug overdose g	Yes
11	Chief complaint: stimulant-involved overdose (intent not listed or inferred) + discharge diagnosis/chief complaint: intentional unknown drug overdose g + discharge diagnosis/chief complaint: other known drug or unknown drug UUDO c	Possible
12	Chief complaint: stimulant-involved overdose (intent not listed or inferred) + discharge diagnosis/chief complaint: intentional overdose by other known drug h	Yes or possible, depending on context
13	Chief complaint: stimulant-involved overdose (if stimulant seems to be listed as an afterthought with respect to intention; intent not listed or inferred) + discharge diagnosis/chief complaint: intentional overdose by other known drug h	Possible
Intentional stimulant-involved overdose chief complaint
14	Chief complaint: mention stimulant + trying to kill self	Yes
15	Chief complaint: intentional stimulant-involved overdose + nonoverdose-related discharge diagnosis/chief complaint text	Yes
16	Chief complaint: stimulant-involved overdose intentional + discharge diagnosis/chief complaint: other known drug or unknown drug UUDO c	Yes
17	Chief complaint: intentional stimulant-involved overdose + discharge diagnosis/chief complaint: stimulant UUDO d	Possible
Suicidal ideation chief complaint
18	Chief complaint: suicidal ideation (or SI or substance inhalation) + chief complaint: stimulant-involved overdose	Possible
19	Chief complaint: suicidal ideation (or SI or substance inhalation) + stimulant + (discharge diagnosis/chief complaint: unintentional stimulant-involved overdose or unintentional other known drug/unknown drug overdose)	Possible
Chief complaint of stimulant-involved overdose (no intent listed) + suicidal ideation/suicide attempt diagnosis code
20	Chief complaint: stimulant-involved overdose (intent not listed or inferred) + (discharge diagnosis: suicidal ideation i or discharge diagnosis: suicide attempt i ) + no ICD-10-CM T36-T50 code for drug overdose	No
21	Chief complaint: stimulant-involved overdose + chief complaint mention of suicidal ideation + (discharge diagnosis: suicidal ideation i or discharge diagnosis: suicide attempt i )	Yes or possible, depending on chief complaint

Abbreviations: ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification; ICD-10-CM, International Classification of Diseases, Tenth Revision, Clinical Modification; SNOMED-CT, Systematized Nomenclature of Medicine–Clinical Terms; UUDO, unintentional/undetermined intent overdose.

^aThis table classifies acute stimulant-involved overdose as intentional or possibly intentional. The data source for this study was CDC’s Drug Overdose Surveillance and Epidemiology system, ⁸ which leverages available ED data in the National Syndromic Surveillance Program. ⁹ The National Syndromic Surveillance Program was queried using the CDC syndrome definition for suspected unintentional/undetermined intent stimulant-involved overdose version 3 for ED visits occurring from January 2018 through December 2019. A total of 40 045 ED visits met the syndrome definition. These visits were from Washington, DC, and 32 states (Alabama, Arkansas, Colorado, Connecticut, Delaware, Florida, Georgia, Illinois, Kansas, Kentucky, Louisiana, Maryland, Maine, Missouri, Mississippi, Montana, North Carolina, New Jersey, New Mexico, Nevada, New York, Ohio, Oregon, Pennsylvania, Rhode Island, South Carolina, Tennessee, Utah, Virginia, Washington, Wisconsin, and West Virginia).

^bDischarge diagnoses for intentional stimulant-involved overdose included ICD-10-CM ¹¹ codes T40.5X2A, T43.602A, T43.612A, T43.622A, T43.632A, T43.642A, and T43.692A, as well as SNOMED-CT ¹³ codes 290544006, 291241005, 291263001, 291266009, 295568001, 296293008, 296319009, 296323001, 296327000, 296331006, and 461061000124108.

^cA Microsoft Excel flag searched for ICD-10-CM ¹¹ codes for UUDO by other known or unknown drugs. These included T36.[0-8]X[1,4]A, T36.9[1,4]XA, T37.[0-5,8]X[1,4]A, T37.9[1,4]XA, T38.[0-7]X[1,4]A, T38.8[0,1,9][1,4]A, T38.9[0,9][1,4]A, T39.0[1,9][1,4]A, T39.[1,2]X[1,4]A, T39.3[1,9][1,4]A, T39.[4,8]X[1,4]A, T39.9[1,4]XA, T40.[0,1,2,3,4,5]X[1,4]A, T40.6[0,9][1,4]A, T40.[7,8]X[1,4]A, T40.9[0,9][1,4]A, T41.[0,1]X[1,4]A, T41.2[0,9][1,4]A, T41.[3,5]X[1,4]A, T41.4[1,4]XA, T42.[0-6,8]X[1,4]A, T42.7[1,4]XA, T43.0[1,2][1,4]A, T43.1X[1,4]A, T43.2[0-2,9][1,4]A, T43.[3,4,8]X[1,4]A, T43.5[0,9][1,4]A, T43.6[0-4,9][1,4]A, T43.9[1,4]XA, T44.[0-8]X[1,4]A, T44.9[0,9][1,4]A, T45.[0-4]X[1,4]A, T45.51[1,4]A, T45.52[1,4]A, T45.6[0-2,9][1,4]A, T45.[7,8]X[1,4]A, T45.9[1,4]XA, T46.[0-8]X[1,4]A, T46.9[0,9][1,4]A, T47.[0-8]X[1,4]A, T47.9[1,4]XA, T48.[0,1,3-6]X[1,4]A, T48.2[0,9][1,4]A, T48.9[0,9][1,4]A, T49.[0-8]X[1,4]A, T49.9[1,4]XA, T50.[0-8]X[1,4]A, T50.9[0,9][1,4]A, T50.A[1,2,9][1,4]A, T50.B[1,9][1,4]A, and T50.Z[1,9][1,4]A.

^dDischarge diagnoses for UUDO stimulant-involved overdose include ICD-9-CM ¹² codes 969.70, 969.71, 969.72, 969.73, 969.79, 970.0, 970.81, 970.89, 970.9, E854.2, E854.3, and E855.2; ICD-10-CM ¹¹ codes T40.5X1A, T40.5X4A, T43.601A, T43.604A, T43.611A, T43.614A, T43.621A, T43.624A, T43.631A, T43.634A, T43.641A, T43.644A, T43.691A, and T43.694A; and SNOMED-CT ¹³ codes 296321004, 441527004, 296290006, and 61803000.

^eICD-9-CM ¹² codes for intentional poisoning from other known or unknown drugs include E950.0, E950.1, E950.2, E950.3, E950.4, and E950.5. ICD-10-CM ¹¹ codes for intentional overdoses from other known or unknown drugs include T36.0X2A, T36.1X2A, T36.2X2A, T36.3X2A, T36.4X2A, T36.5X2A, T36.6X2A, T36.7X2A, T36.8X2A, T36.92XA, T37.0X2A, T37.1X2A, T37.2X2A, T37.3X2A, T37.4X2A, T37.5X2A, T37.8X2A, T37.92XA, T38.0X2A, T38.1X2A, T38.2X2A, T38.3X2A, T38.4X2A, T38.5X2A, T38.6X2A, T38.7X2A, T38.802A, T38.812A, T38.892A, T38.902A, T38.992A, T39.012A, T39.092A, T39.1X2A, T39.2X2A, T39.312A, T39.392A, T39.4X2A, T39.8X2A, T39.92XA, T40.0X2A, T40.1X2A, T40.2X2A, T40.3X2A, T40.4X2A, T40.5X2A, T40.602A, T40.692A, T40.7X2A, T40.8X2A, T40.902A, T40.992A, T41.0X2A, T41.1X2A, T41.202A, T41.292A, T41.3X2A, T41.42XA, T41.5X2A, T42.0X2A, T42.1X2A, T42.2X2A, T42.3X2A, T42.4X2A, T42.5X2A, T42.6X2A, T42.72XA, T42.8X2A, T43.012A, T43.022A, T43.1X2A, T43.202A, T43.212A, T43.222A, T43.292A, T43.3X2A, T43.4X2A, T43.502A, T43.592A, T43.602A, T43.612A, T43.622A, T43.632A, T43.642A, T43.692A, T43.8X2A, T43.92XA, T44.0X2A, T44.1X2A, T44.2X2A, T44.3X2A, T44.4X2A, T44.5X2A, T44.6X2A, T44.7X2A, T44.8X2A, T44.902A, T44.992A, T45.0X2A, T45.1X2A, T45.2X2A, T45.3X2A, T45.4X2A, T45.512A, T45.522A, T45.602A, T45.612A, T45.622A, T45.692A, T45.7X2A, T45.8X2A, T45.92XA, T46.0X2A, T46.1X2A, T46.2X2A, T46.3X2A, T46.4X2A, T46.5X2A, T46.6X2A, T46.7X2A, T46.8X2A, T46.902A, T46.992A, T47.0X2A, T47.1X2A, T47.2X2A, T47.3X2A, T47.4X2A, T47.5X2A, T47.6X2A, T47.7X2A, T47.8X2A, T47.92XA, T48.0X2A, T48.1X2A, T48.202A, T48.292A, T48.3X2A, T48.4X2A, T48.5X2A, T48.6X2A, T48.902A, T48.992A, T49.0X2A, T49.1X2A, T49.2X2A, T49.3X2A, T49.4X2A, T49.5X2A, T49.6X2A, T49.7X2A, T49.8X2A, T49.92XA, T50.0X2A, T50.1X2A, T50.2X2A, T50.3X2A, T50.4X2A, T50.5X2A, T50.6X2A, T50.7X2A, T50.8X2A, T50.902A, T50.912A, T50.992A, T50.A12A, T50.A22A, T50.A92A, T50.B12A, T50.B92A, T50.Z12A, and T50.Z92A. SNOMED-CT ¹³ codes for intentional overdose or intentional poisoning by any drug were also included. A search of https://phinvads.cdc.gov/vads/SearchVocab.action was conducted for all codes involving the words “overdose” or “poisoning”; then any codes for intentional self-harm or poisoning caused by drugs were included.

^fCodes for assault by stimulant-involved overdose included ICD-10-CM ¹¹ codes T40.5X3A, T43.603A, T43.613A, T43.623A, T43.633A, T43.643A, and T43.693A.

^gCodes for intentional overdose by unknown/unspecified drugs included ICD-9-CM ¹² codes E950.[4,5]; ICD-10-CM ¹¹ codes T50.902A, T50.912A, and T50.992A; and SNOMED-CT ¹³ codes 431307001, 59274003, and 460991000124106.

^hCodes for intentional overdose by other known drugs included all codes in footnote “e,” except for those in footnote “g” (ie, except for codes for intentional overdose by unknown or unspecified drugs).

ⁱCodes for suicidal intention/attempt included ICD-10-CM ¹¹ codes T14.91 (suicide attempt), X83.8 (intentional self-harm by other specified means), and R45.851 (suicidal ideation); ICD-9-CM ¹² code V62.84 (suicidal ideation); and SNOMED-CT ¹³ codes 82313006 (suicide attempt) and 6471006 (suicidal thoughts).

Testing the decision rules

Although the final analysis included data shared with NSSP from 32 states and Washington, DC, we first tested the decision rules by querying data from a subset of DOSE states (n = 18 states; testing data retrieved on July 23, 2019). Our test-stage NSSP data retrieval included examination of 24 952 ED visits captured using the suspected UUSOD syndrome definition for ED visits occurring from January 1, 2016, through December 31, 2018.

We first analyzed all test-stage visits using SAS regular expression code (SAS version 9.4; SAS Institute, Inc) to identify visits that were captured because they included UUSOD diagnosis codes (Table 1). SAS code searched these visits for codes with and without periods (eg, T40.5X1A or T405X1A) and for text from diagnosis code definitions (eg, “poisoning by cocaine accidental [unintentional] initial encounter”). We classified these 12 817 visits as acute SODs.

From the remaining 12 135 test-stage visits without UUSOD diagnosis codes, we randomly sampled 250 visits for manual review (these 250 visits were separate from the manual review completed in Stage 2: UUSOD Syndrome Validation). Five coders independently classified these visits based on information in the chief complaint/discharge diagnosis field using the initial decision rules. To facilitate manual review, we created Microsoft Excel spreadsheets with programmed flags that searched the chief complaint/discharge diagnosis field for key terms. We resolved discordant ratings via group discussion and consultation with CDC physicians.

Subsequently, we revised the initial decision rules to address questions that arose during the first review stage and trained an additional 5 coders. Each new coder independently rated the 250 randomly selected visits and discussed their ratings with the larger group. We made additional minor revisions to the decision rules throughout this process. We applied all revisions to the decision rules to visits reviewed in Stage 2 of the validation process.

Stage 2: UUSOD Syndrome Validation

On February 24, 2020, we queried NSSP to identify ED visits with suspected UUSOD occurring from January 1, 2018, through December 31, 2019, in 32 states and Washington, DC, that shared data with CDC’s DOSE system. ⁸

The Stage 2 validation process followed a similar process as Stage 1 (Figure). We used SAS regular expression code to identify visits with UUSOD diagnosis codes; we classified these visits as acute SODs. Among this subset, we used SAS regular expression code to identify visits that had both a UUSOD diagnosis code and an indication of intentional self-harm or assault-related SOD (“intentional,” hereinafter) based on available diagnosis codes or text. Using detailed decision rules (Table 3), 2 coders independently reviewed this subset of visits for SOD intention and resolved discordant ratings.

Figure.

Validation of a syndrome definition for suspected nonfatal unintentional/undetermined intent stimulant-involved overdose (UUSOD). Abbreviation: ED, emergency department.

We then randomly sampled 10% of remaining visits without UUSOD diagnosis codes. We divided these visits among 5 teams, each with 2 coders. Coders worked independently to first classify whether visits were acute SODs and subsequently to classify whether acute SODs were intentional or possibly intentional. Coders met with their partners to review all classifications. Coding teams discussed and resolved all discordantly classified visits using the decision rules and consultation with the project lead as needed. We extrapolated results from this 10% random sample to the entire subsample of visits without UUSOD diagnosis codes. We calculated the final estimated percentage of true-positive UUSODs as a weighted average of 2 numbers: the percentage of true-positive UUSODs based on diagnosis codes and the percentage of true-positive UUSODs based on manual review of the chief complaint/discharge diagnosis field.

CDC determined the intent of this project to be public health surveillance for disease and injury control; thus, the activity did not involve human subjects, and institutional review board approval was not required.

Results

The stimulant syndrome NSSP query (version 3) identified 40 045 suspected UUSOD ED visits during 2018-2019 in 32 states and Washington, DC. Of these visits, 99.5% had ≥1 word in the chief complaint/discharge diagnosis field, and 87.8% contained ≥1 diagnosis code. Children and adolescents aged <18 years accounted for approximately 6% (n = 2376) of suspected UUSOD visits, and the median patient age was 35. Nearly two-thirds (65.9%) of suspected UUSOD visits occurred among males.

A total of 18 793 (46.9%) ED visits had diagnosis codes for UUSOD, and we considered these acute SODs (Table 4). Of the 18 793 visits with UUSOD diagnosis codes, 1.2% were possibly intentional and 0.2% were intentional acute SODs; the remaining 98.6% (n = 18 534) were true-positive UUSODs (either unintentional or undetermined intent; Table 4). We manually reviewed 2125 visits of the remaining 21 252 visits without UUSOD diagnosis codes. Of these 2125 visits, 788 (37.1%) were acute SODs, 1126 (53.0%) were possible acute SODs, and 211 (9.9%) were not acute SODs. Of 788 acute SODs, 3.2% (n = 25) were possibly intentional, and 8.9% (n = 70) were intentional, whereas most were true-positive UUSODs (n = 693; 87.9%; either unintentional or undetermined intent; Table 4). After the study team accounted for intent, approximately 32.6% (n = 693) of all 2125 manually reviewed visits were true-positive UUSODs, 54.2% (n = 1151) were possible UUSODs, and 13.2% (n = 281) were not UUSODs. Overall, the final estimated percentage of true-positive UUSODs was 63.6%. This was a weighted average based on (1) visits with UUSOD diagnosis codes multiplied by the percentage of true-positive UUSODs among this subset (0.469% × 98.6% true-positive) and (2) visits without UUSOD diagnosis codes multiplied by the percentage of true-positive UUSODs among this subset (0.531% × 32.6% true-positive). In addition, 29.3% of overall visits were possible UUSODs, and 7.1% were not UUSODs.

Table 4.

Counts and percentages of acute stimulant-involved overdoses, acute intentional stimulant-involved overdoses, and true-positive acute unintentional/undetermined intent stimulant-involved overdoses, ^a 2018-2019

Stage of review	ED visits with UUSOD diagnosis codes, b no. (%)	10% random sample (n = 2125) of the 21 252 ED visits without UUSOD diagnosis codes, c no. (%)	Overall percentage d
Acute stimulant-involved overdose e
Yes	18 793 (100.0)	788 (37.1)	—
Possible	0	1126 (53.0)
No	0	211 (9.9)
Total	18 793 (100.0)	2125 (100.0)
Acute intentional stimulant-involved overdose f
Yes	42 (0.2)	70 (8.9)
Possible	217 (1.2)	25 (3.2)
No	18 534 (98.6)	693 (87.9)
Total	18 793 (100.0)	788 (100.0)
True-positive acute UUSOD g
Yes	18 534 (98.6)	693 (32.6)	63.6
Possible	217 (1.2)	1151 (54.2)	29.3
No	42 (0.2)	281 (13.2)	7.1

Abbreviations: ED, emergency department; ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification; ICD-10-CM, International Classification of Diseases, Tenth Revision, Clinical Modification; SNOMED-CT, Systematized Nomenclature of Medicine–Clinical Terms; UUSOD, unintentional/undetermined intent stimulant-involved overdose.

^aThe data source for this study was the Centers for Disease Control and Prevention’s (CDC’s) Drug Overdose Surveillance and Epidemiology system, ⁸ which leverages available ED data in the National Syndromic Surveillance Program. ⁹ The National Syndromic Surveillance Program was queried using the CDC syndrome definition for UUSOD version 3 for ED visits occurring from January 2018 through December 2019. A total of 40 045 ED visits met the syndrome definition for UUSOD. These visits were from Washington, DC, and 32 states (Alabama, Arkansas, Colorado, Connecticut, Delaware, Florida, Georgia, Illinois, Kansas, Kentucky, Louisiana, Maryland, Maine, Missouri, Mississippi, Montana, North Carolina, New Jersey, New Mexico, Nevada, New York, Ohio, Oregon, Pennsylvania, Rhode Island, South Carolina, Tennessee, Utah, Virginia, Washington, Wisconsin, and West Virginia).

^bThese visits were identified using SAS version 9.4 (SAS Institute, Inc) queries to detect ED visits with discharge diagnoses for UUSOD. These included ICD-9-CM ¹² codes 969.70, 969.71, 969.72, 969.73, 969.79, 970.0, 970.81, 970.89, 970.9, E854.2, E854.3, and E855.2; ICD-10-CM ¹¹ codes T40.5X1A, T40.5X4A, T43.601A, T43.604A, T43.611A, T43.614A, T43.621A, T43.624A, T43.631A, T43.634A, T43.641A, T43.644A, T43.691A, and T43.694A; and SNOMED-CT ¹³ codes 296321004, 441527004, 296290006, and 61803000.

^cA 10% random sample of all 21 252 ED visits without a UUSOD diagnosis code (n = 2125) was manually reviewed by teams of coders using the decision rules in Tables 2 and 3 and Supplemental Tables S1 and S2. The results in this 10% random sample were extrapolated to all ED visits (n = 21 252) without a UUSOD diagnosis code.

^dThe overall percentage of true-positive UUSOD was calculated as a weighted average of the percentage of true-positive UUSOD based on diagnosis codes and the percentage of true-positive UUSOD based on manual review of ED visits without UUSOD diagnosis codes. As an example, the final estimated percentage of true-positive acute UUSOD was 63.6%. This was a weighted average based on ED visits with UUSOD diagnosis codes multiplied by the percentage of true-positive UUSOD among this subset (0.469 × 98.6%), as well as ED visits without UUSOD diagnosis codes multiplied by the percentage of true-positive UUSOD based on manual review of the chief complaint/discharge diagnosis field (0.531 × 32.6%). Percentages of true-positive, possible, and not UUSOD sum to 100% without rounding.

^eIn this stage, ED visits were reviewed for whether they were acute stimulant-involved overdoses using Table 2 and Supplemental Tables S1 and S2.

^fIn this stage, acute stimulant-involved overdoses were reviewed for intent using the decision rules in Table 3.

^gED visits classified as not UUSOD include the sum of ED visits identified as not acute stimulant-involved overdoses and ED visits identified as acute intentional stimulant-involved overdoses. ED visits classified as possible UUSOD include the sum of ED visits identified as possible acute stimulant-involved overdoses and ED visits identified as possible intentional stimulant-involved overdoses. ED visits classified as true-positive UUSODs are acute stimulant-involved overdoses that were not intentional or possibly intentional (the “not intentional” group includes those of unintentional or undetermined intent).

Many manually reviewed visits (n = 593; 27.9%) contained an F14 or F15 ICD-10-CM code written out as text in the chief complaint/discharge diagnosis field (eg, “drug intoxication/F15.10/other stimulant abuse uncomplicated”). Most of these 593 visits (n = 497; 83.8%) were possible acute SODs and were only captured by the syndrome definition because the ICD-10-CM code text was written out. We coded certain other visits as possible acute SODs when it was unclear whether the chief complaint field described recent stimulant use versus general stimulant use disorder. Some visits indicated that stimulants or drug paraphernalia were found on the scene, but it was not always clear whether patients had used stimulants. In other visits, a patient’s parent, friend, or partner speculated that a patient may have ingested stimulants. Another common reason we coded visits as possible acute SODs was limited information on symptoms. We found several reasons for coding visits as not SODs, such as patients indicating that they wanted to overdose in the future and visits being wrongly captured by abbreviations (eg, “Meth” for “Methodist,” not “methamphetamine”).

Discussion

With an increased focus on monitoring nonfatal overdoses, CDC developed a syndrome definition to capture suspected UUSODs. Using ED visit data, we evaluated the validity of a syndromic surveillance definition for suspected UUSOD. Following a rigorous, multistage review process, we found that nearly two-thirds (63.6%) of suspected UUSODs were true-positive UUSODs, 29.3% were possible UUSODs, and 7.1% were not UUSODs. By developing, validating, and applying this definition with rapid syndromic surveillance data, we can estimate trends and detect sharp changes in UUSODs better than we could previously. Estimation and detection are particularly important given recent increases in fatal and nonfatal SODs and especially given dramatic increases in stimulant overdoses involving opioids, ^{1 -3,19 -21} likely attributable to synthetic opioids such as illicitly manufactured fentanyl. ^2,19,22 Using fentanyl can increase the chance of overdose, particularly among people who do not routinely use opioids. ²²

The percentage of true-positive UUSODs in our study was lower than the percentage of true-positive unintentional/undetermined intent heroin-involved overdoses (UUHODs) in a recent validation study of CDC’s suspected UUHOD syndrome definition (63.6% vs 94.7%); furthermore, the percentage of possible UUSODs in our study was much higher than the percentage of possible UUHODs in the heroin validation study (29.3% vs 3.0%). ²³ Although almost 80% of suspected heroin-involved overdoses had a diagnosis code for unintentional or undetermined intent heroin overdose, ²³ fewer than half of suspected SODs had UUSOD diagnosis codes. Compared with heroin-involved overdoses, SODs may be more difficult to classify based on clinical presentation because of prevailing nonspecific symptoms. Educating health care providers on the clinical presentation of SODs, increasing toxicologic testing in EDs, and including additional specific symptoms in our UUSOD syndrome definition might improve the percentage of true-positive UUSODs.

Another challenge in our analysis was the large number of visits recorded as possible UUSODs caused by the way facilities map data from their systems into the Health Level 7 (HL7) standard for syndromic surveillance. HL7 is an international data standard used for data exchange and national syndromic surveillance reporting in the United States. ²⁴ Sometimes F14/F15 diagnosis codes related to stimulant use or abuse (not necessarily overdose) were written as text in the chief complaint/discharge diagnosis field, causing some visits to be incorrectly captured as suspected UUSOD by our syndrome definition. Despite these challenges, only 7.1% of visits in our study were classified as not UUSODs (most of these were not acute SODs). Furthermore, a portion of the 29.3% of possible UUSODs were likely true-positive UUSODs, but we lacked sufficient information to classify them with certainty.

This study had several limitations. First, the diagnosis codes in the chief complaint/discharge diagnosis field represent preliminary diagnoses. In addition, 12.2% of visits lacked any discharge diagnoses, which could underestimate the number of true-positive UUSODs. Because information in the chief complaint/discharge diagnosis field was sometimes incomplete, we lacked enough evidence to classify 29.3% of visits with certainty as either true-positive UUSODs or not UUSODs (ie, 29.3% of visits were classified as possible UUSODs). Jurisdictions can continue to work with participating EDs to increase the quality and completeness of discharge diagnosis codes in syndromic surveillance data. A second limitation to our study was that discharge diagnoses in the ED are rarely based on toxicologic testing. ²⁵ Third, the stimulant syndrome definition may not have included all possible terms. Future revisions to the definition should consider additional symptoms and new drug terms as the stimulant drug market evolves. Fourth, we lacked an external gold standard, such as medical chart review, against which to compare our syndromic surveillance data; thus, we could not calculate sensitivity, specificity, or positive predictive value. Finally, we analyzed data from a sample of states, which may not be representative of all states. Because of differences in data quality, completeness, and reporting across jurisdictions, comparisons between jurisdictions should not be made for any syndromic, text-based definition.

Our study also had several strengths. First, across the jurisdictions included in our analysis, data coverage was typically ≥75% during 2018-2019. Second, we used a combination of SAS regular expression code and manual review to classify visits. Third, we manually reviewed more than 2000 randomly selected visits without UUSOD diagnosis codes. This manual review process was standardized via the use of detailed decision rules created in consultation with experienced CDC physicians, as well as through training activities and regular conference calls with the full stimulant review team. Each manually reviewed record was double-coded and, if necessary, discussed with a larger group to reach consensus. Fourth, we validated the syndrome definition among people of all ages and created symptoms classification documents for children and adolescents aged <18 years and separately for adults aged ≥18.

All jurisdictions submitting syndromic surveillance data to DOSE currently use CDC’s UUSOD definition. ²⁶ Recent DOSE data demonstrate that, of 25 states with sufficient sample sizes, 4 states had a significant increase in annual rates of UUSOD from February 2019 through February 2020, 19 had no change, and 2 had a significant decline. ²⁶

Practice Implications

Jurisdictions may use UUSOD syndromic surveillance data to rapidly identify and respond to potential clusters of UUSODs as well as to inform prevention activities. Identifying demographic groups with disproportionately high overdose rates may help direct prevention resources. Syndromic surveillance data may also inform local or national public health policies. Although syndromic surveillance data do not measure absolute overdose counts or rates, they are particularly useful for quickly assessing trends in nonfatal overdoses. ²⁷ Furthermore, although counts or rates of stimulant overdoses may be higher in syndromic surveillance datasets than in ED discharge datasets, overall trends (eg, direction of percentage change in rates) are similar in both data sources. ²⁷ Other data sources, including emergency medical services transports, ED and inpatient hospital discharge data, death certificates, medical examiner/coroner reports, and toxicology reports help provide a more complete picture of SODs. ⁴

Validation of syndrome definitions is an important step to understanding the context of suspected drug overdoses. The methodology used in this study can be adapted to future validation studies for other syndromic surveillance definitions. The suspected UUSOD syndrome definition may be revised further to improve its accuracy for use in detecting and responding to acute SODs across the United States.

The findings and conclusions in this article are those of the authors and do not necessarily represent the official position of CDC.

ORCID iD

Cassandra M. Pickens, PhD, MPH https://orcid.org/0000-0003-3154-2374