Validation of Criteria to Guide Prehospital Naloxone Administration for Drug-Related Altered Mental Status

Abstract

Introduction

We aimed to validate previously derived clinical criteria to predict successful prehospital response to naloxone in patients with altered mental status treated by EMS. We hypothesized that prehospital naloxone criteria would have high sensitivity for effective antidote response, but would be underutilized, in patients with drug-related altered mental status (DRAMS).

Methods

This study was a secondary data analysis of a prospective cohort of acute DRAMS at an urban ED. Naloxone criteria (respiratory rate (RR) <12, miotic pupils, or drug paraphernalia) and mental status, graded by either AVPU (Alert, Verbal, Painful, Unresponsive) or Glasgow Coma Scales, were abstracted from prehospital care reports. Interventions were compared for effective antidote response (EAR), defined as immediate improvement in RR, AVPU, or GCS.

Results

EMS transported 249 DRAMS over 17 months (48 % males, mean age 41.5, ALS 33.7 %). Forty-three (17 %) patients met naloxone criteria, of whom 44.2 % received the antidote. Naloxone criteria significantly predicted EAR (OR 7.0, p < 0.05) with 83 % sensitivity (95 % CI, 55–95 %). Miotic pupils (OR 20.0, p < 0.01) outperformed RR (OR 2.3, p = NS) as the best single criterion with 91 % sensitivity (95 % CI, 62–98 %).

Conclusions

This study validates prehospital criteria to guide naloxone administration. In addition, prehospital naloxone was underutilized for DRAMS. Further studies should address potential barriers to prehospital naloxone administration.

Introduction

Prehospital death from acute drug overdose is a substantial public health threat [1]. Drug overdose mortality, with an incidence of 10 per 100,000 young adults per year, is the most common precipitating cause of young adult out-of-hospital cardiac arrest in the USA [2] and in Europe [3]. Moreover, the number of fatal drug overdoses appears to be increasing. In Ohio, drug overdose deaths increased by 372 % from 1999 to 2010 [4]. In addition to death, the burden of lost productivity and medical costs associated with opioid-related poisoning is $20.4 billion annually (in 2009 dollars) [5].

Naloxone, the antidote for opioid toxicity, is an opioid receptor antagonist which can be administered intranasally, intramuscularly, and intravenously with rapid onset and relatively short clinical duration. Since its introduction in 1967, naloxone has greatly improved the treatment of patients with opioid overdose. Naloxone is included in EMS “Altered Mental Status” (AMS) protocols because it is an accurate diagnostic tool for the confirmation or exclusion of opioid overdose and provides therapeutic benefit to patients with respiratory depression from opioid toxicity [6]. Recently, distribution programs that allow family members and friends to administer naloxone have been successful in saving lives, as evidenced by a systematic review [7].

However, iatrogenic concerns exist regarding adverse effects of naloxone administration, such as a fulminate withdrawal, vomiting, pulmonary edema, re-sedation in overdoses involving sustained release prescription opioids (e.g., methadone), as well as perceived delays in prehospital care [8]. Adverse events related to naloxone treatment are quite common, occurring in up to 45 % of administrations [9]. Additionally, arousal from opioid poisoning frequently results in patients refusing medical care and facing the complications that can occur owing to a shorter duration of action for naloxone than most opioid agonists (e.g., methadone) [8].

Previously, clinical criteria have been derived to predict successful response to naloxone in patients with altered mental status treated by EMS [10]. The criteria which had 92 % sensitivity to predict a successful response to naloxone were the following: (A) respirations of 12 or less; (B) miotic pupils; and (C) circumstantial evidence of opioid abuse (i.e., paraphernalia) at the scene. Selective administration of naloxone for patients with AMS, plus at least one of these clinical criteria, would theoretically decrease naloxone use by 75 %, without missing any true opioid overdoses [10].

Although timely prehospital naloxone could theoretically be life-saving, more information is needed regarding the role of EMS in treating opioid overdose [11]. This study aimed to validate the above prehospital naloxone criteria. We hypothesized that prehospital naloxone criteria would have high sensitivity for effective antidote response, but would be underutilized, in patients with drug-related altered mental status (DRAMS).

Methods

Study Design and Population

This study was a secondary data analysis from a prospective cohort of acute emergency department (ED) visits for DRAMS at one urban tertiary care hospital [12, 13]. Consecutive adult (≥18 years of age) drug-related ED visits were initially screened for inclusion by trained research assistants who were physically present in the ED during normal business hours; additionally, they also screened all ED consults to the regional Poison Control Center (PCC) 24 h/day and 7 days/week from May 2009 through September 2010. Reporting of suspected acute drug-related ED visits to the regional Poison Center is mandated by public health law. The tertiary care ED is a 911-receiving center located in a large metropolitan zone with an annual ED census in excess of 100,000 visits.

Study Protocol

All patients had suspected DRAMS at the time of first presentation, as defined by two criteria: (1) suspicion of drug-related presentation; and (2) any altered mental status. The prehospital protocol for DRAMS utilized in the study is outlined in Fig. 1. DRAMS patients who were transported to the ED by EMS and whose prehospital care report (PCR) was scanned into the electronic medical record were included in the data analysis. The study protocol was performed on a preexisting de-identified database that was approved by the Program for the Protection of Human Subjects at the study institution.

Fig. 1.

EMS Regional Altered Mental Status Protocol. Adapted from the Regional Emergency Medical Services Council of New York City Advanced Emergency Medical Technician Protocols

Study Setting

The study setting was an urban metropolitan area with a Fire Department (FD) that provides EMS for the city and is comprised of a tiered response system, which affords two emergency medical technicians (EMT) providing Basic Life Support (BLS) for trauma and non-critical medical call types. For critical medical calls (i.e., respiratory distress or altered mental status), an Advanced Life Support (ALS) ambulance is dispatched which is comprised of two paramedics who can perform advanced airway maneuvers such as endotracheal intubation, supraglottic device placement and administer intravenous medications. For the highest acuity call types (i.e., cardiac or respiratory arrest), a “triple-tiered” response is dispatched which additionally includes Certified First Responders (CFR) responding in a FD engine. CFRs are licensed to provide compressions and defibrillate with an automated external defibrillator (AED) but their most significant contribution is their closer proximity to the scene of the call than the nearest ambulance, owing to more ubiquitous fire houses than ambulance designations. The State Emergency Medical Advisory Committee (SEMAC) previously approved the administration of naloxone using a mucosal atomizer device for patients experiencing opioid overdoses for use by certified BLS EMS providers; thus, both BLS and ALS providers are able to administer naloxone in the study catchment area.

Data Collection and Measurements

All ED electronic medical record (EMR) charts with triage notes or final diagnoses related to drug poisoning were screened for inclusion. Trained research assistants were paid personnel who had experience with chart abstraction. Important variables such as overdose, poisoning, and altered mental status were all defined precisely. Standardized abstraction forms were utilized to guide data collection. Data abstractors were blinded to the hypotheses being tested. An EMS PCR was generated for each patient encounter. The PCRs of all enrolled patients were reviewed for the following data points: (A) naloxone criteria (Table 1), (B) pupil examination, and (C) mental status graded by AVPU (Alert, Verbal, Painful, Unresponsive) or Glasgow Coma Scale (GCS). On-scene drug paraphernalia was recorded if noted on the PCR. While the PCR does not specifically ask if there is any drug paraphernalia on scene, prehospital providers generally include it since it provides useful contributory history. The information was entered into a computerized spreadsheet. Chart review methodology adhered to accepted guidelines [14].

Table 1.

Naloxone criteria

A.Miotic pupils

B.Respiratory rate (RR) <12

C.Circumstantial evidence (on-scene drug paraphernalia, track marks)

Adapted from Hoffman et al. [10]

Effective Antidote Response (EAR)

Emergency Medical Services (EMS) intervention (naloxone) was compared for EAR (i.e., dichotomous positive or negative response), defined as any immediate improvement in the following: RR, AVPU, or GCS (i.e., improvement of at least 1 respiration per minute, AVPU indicator, or GCS point). EAR was never scored as partial or equivocal. EAR was scored as positive if there was any clinical improvement in objectively measured vital signs (i.e., RR) or mental status (i.e., AVPU or GCS).

Statistical Analysis

Test characteristics of the naloxone criteria (Table 1) were evaluated for sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). Odds ratios (OR) and 95 % confidence intervals (CI) were calculated for all point estimates (e.g., sensitivity, specificity). We calculated that a sample size of 250 patients was needed in order to obtain narrow CI widths (i.e., 10 %) around the point estimates, assuming 15 % EAR, 80 % power, and 5 % alpha. Computer analysis was performed using SPSS v. 21 software (IBM Inc., Chicago, IL).

Results

The primary data analysis screened 720 ED patients with DRAMS over 17 months, of whom 267 were transported by EMS and 249 had a PCR scanned into the EMR. The included patients were 48 % male, with a mean age of 41.5 years, and ALS unit transport in 33.7 % (84/249) of patients.

Utilization of Naloxone

Of the 43 (17 %) patients meeting the naloxone criteria, only 44.2 % of these patients received naloxone. Additionally, ETI was attempted in lieu of naloxone on four apneic patients who were not in cardiac arrest. Other commonly used toxicologic antidotes (e.g., sodium bicarbonate, calcium) were not administered for any DRAMS patients.

Performance of Naloxone Criteria

The naloxone criteria were assessed in the 24 patients who received naloxone, and these main results are illustrated in Table 2. When ≥1 criterion was present, there was sevenfold increased odds of EAR (OR 7.0, p < 0.5). Overall, the criteria performed with 83 % sensitivity (95 % CI. 55–95 %). When evaluating each criterion individually, miotic pupils (71 % EAR), respiratory rate (66.7 % EAR), and circumstantial evidence (100 % EAR) all had high response rates; however, miotic pupils was the best single criterion with 91 % sensitivity (95 % CI, 62–98 %) and 89 % NPV (95 % CI, 57–98 %) outperforming any of the naloxone criteria as a whole. The EMS naloxone criteria test characteristics are summarized in Table 3.

Table 2.

Performance of naloxone criteria

Naloxone criteria:	GCS (median)	AVPU (mode)	+ EAR (%)	No EAR	Total	EAR OR (95 % CI)
Rule (≥1 criteria)	14	A	10 (66.7 %)	5	15	7.0 (1.1–46.9)
No criteria	14	A	2 (22.2 %)	7	9	–
Miotic pupils	3	A	10 (71 %)	4	14	20.0 (1.9–216.2)
RR <12	14	P	6 (66.7 %)	3	9	2.3 (0.4–13.6)
Paraphernalia	12	V	2 (100 %)	0	2	∞ –

AVPU awake-verbal-pain-unresponsive, CI 95 % confidence intervals, EAR effective antidote response, GCS Glasgow coma score, OR odds ratio, RR respiratory rate

Table 3.

Diagnostic test characteristics of EMS naloxone criteria prediction rules

Naloxone criteria	Sensitivity	Specificity	PPV	NPV
Current study	83.3 %	58.3 %	66.7 %	77.7
Original derivation [10]	92 %	76 %	–	–

NPV negative predictive value, PPV positive predictive value

Discussion

This study builds upon this existing literature by providing external validation to the original prehospital clinical criteria to guide naloxone administration. In the present study, the naloxone criteria had similar test characteristics compared with the original derivation (see Table 3). Thus, the naloxone criteria may remain a useful guide for prehospital administration by EMS providers. However, less than half of those meeting criteria actually received naloxone. Therefore, further studies should address underutilization of naloxone administration for DRAMS in the prehospital setting.

The administration of naloxone to a patient with hypoventilation due to opioid overdose is a potentially life-saving prehospital intervention. An Italian study demonstrated that the mortality for overdoses was significantly lower in a city whose EMS systems protocolized administration of naloxone [15]. Subsequently, a protocol for EMS administration of naloxone for opioid overdoses was first published in 1996 [16].

Selective, rather than reflexive, administration of naloxone provides obvious cost savings as well as a decrease in the unintended iatrogenic effects of naloxone administration to tolerant individuals. Iatrogenic opioid withdrawal may be precipitated by naloxone overuse or use without clear clinical indications. While not life-threatening, symptoms of iatrogenic opioid withdrawal include nausea/vomiting, abdominal pain, anxiety, tachycardia, hypertension, and agitation [8].

Recently, the use of naloxone has gained national attention from policy makers [17]; for example, at least one State Emergency Medical Advisory Committee has approved certified first responders to carry and administer intranasal naloxone with hopes to establish “Public Access Narcan” programs similar to “Public Access Defibrillation” programs [18]. First responders generally do not have the same education or training as ALS providers, nor are they equipped to treat sequelae of acute opioid withdrawal or severe pulmonary edema. In addition to providing clinical indications for prehospital naloxone use, our study may also provide guidance to policy makers considering propagating indications for naloxone in public access programs.

Improved EMS protocols for DRAMS patients may reduce the incidence of drug poisoning mortality, which is currently the leading cause of injury-related fatality in the USA [19]. This study represents an initial step in this regard. An excellent example of prehospital policy efforts comes from Melbourne, Australia, where the high mortality rate for heroin overdoses [2] prompted implementation of a number of preventative initiatives to curb the incidence of unintentional drug overdose. While data on the absolute mortality benefit is not available, such programs include first-aid training by local drug user groups, minimizing drug law enforcement (enabling heroin users to call for an ambulance without fear of police involvement or possible arrest), and limiting availability of potentially lethal drugs [20].

There is no accepted or validated definition for antidotal response. In the absence of a criterion/gold standard, we used EAR for this study as a reasonable surrogate. The authors would argue that a one-point improvement in the AVPU response is of greater clinical significance than a similar improvement in respiratory rate or GCS. However, an immediate improvement of any score is in all likelihood attributable to the antidote and could potentially be life-saving. In addition, it is plausible that many of the overdoses were mixed ingestions; therefore, it is difficult to determine the expected response to naloxone in mixed co-ingestions of different toxins. Finally, prehospital provider training for objective criteria such as respiratory rate and GCS is quite uniform and the inter-rater reliability is exceedingly high [21, 22]. For these reasons, we believe the use of EAR in this study was appropriate.

Our data suggest that examination of the barriers (training, access, non-adherence, etc.) to EMS antidotal administration would strengthen future research. We found that only 44.2 % of patients who met criteria were administered naloxone in this study. Possible explanations for this discrepancy include lack of appropriate training, certification, licensure, or lack of direct medical oversight. Studies that assess the impact of additional EMS provider training and education are warranted.

Limitations

There are several limitations to this study which must be considered. For example, the generalizability of our study to rural areas is a consideration, as well as to EMS systems with differing prehospital protocols for AMS; however, the study was conducted at a hospital located in the most racially and ethnically diverse neighborhood in the continental US, according to census data [23]. Additionally, our data abstraction relied upon PCRs, which may have been inaccurate or incomplete for a subset of patients in this study. Furthermore, the data was dependent on the training and competency of the prehospital providers in our region and the specific case-mix of patients; thus, it is possible that higher (or lesser) skilled paramedics could have improved (or reduced) ability to elicit the relevant clinical information and in turn have altered prehospital antidote administration. Generally, the providers filling out PCRs did not receive formal education in research, nor were they allotted extra time to assure the accuracy of their recordings for this study. Treatment response was known at the time of documentation and may have influenced what was recorded. There was no possibility of independent verification to the accuracy of objective measures like GCS scores; however, the inter-rater reliability for GCS has previously been shown to be exceedingly high [21, 22]. Similarly, the accurate documentation of vital signs by EMS providers in this study may have been outweighed by the need for urgent administration of naloxone to patients with impending respiratory arrest; however, the training of paramedics is fairly uniform throughout the region and the clinical information sought did not require additional training to obtain reliably. It is possible that trained research assistant availability was a limitation for nights and weekends; however, we expect that this would apply to only a small amount of patients as there is a strong onus on contacting the PCC for all drug poisonings at our particular institution.

Conclusions

In this ED-based study of prehospital antidotal interventions for DRAMS, we validated previously derived criteria for prehospital naloxone administration. In addition, prehospital naloxone was underutilized for DRAMS. Further studies should address potential barriers to prehospital naloxone administration, as well as the most effective clinical interventions for prehospital providers when they encounter DRAMS.

Compliance with Ethical Standard

Conflicts of Interest

The authors report no commercial conflicts of interest.

Sources of Funding

This study was funded, in part, by the National Institutes of Health (DA026476, PI: Manini). This content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute on Drug Abuse or the National Institutes of Health.