The Effect of Overdose Education and Naloxone Distribution: An Umbrella Review of Systematic Reviews

Abstract

Background. Opioids contribute to more than 60 000 deaths annually in North America. While the expansion of overdose education and naloxone distribution (OEND) programs has been recommended in response to the opioid crisis, their effectiveness remains unclear.

Objectives. To conduct an umbrella review of systematic reviews to provide a broad-based conceptual scheme of the effect and feasibility of OEND and to identify areas for possible optimization.

Search Methods. We conducted the umbrella review of systematic reviews by searching PubMed, Embase, PsycINFO, Epistemonikos, the Cochrane Database of Systematic Reviews, and the reference lists of relevant articles. Briefly, an academic librarian used a 2-concept search, which included opioid subject headings and relevant keywords with a modified PubMed systematic review filter.

Selection Criteria. Eligible systematic reviews described comprehensive search strategies and inclusion and exclusion criteria, evaluated the quality or risk of bias of included studies, were published in English or French, and reported data relevant to either the safety or effectiveness of OEND programs, or optimal strategies for the management of opioid overdose with naloxone in out-of-hospital settings.

Data Collection and Analysis. Two reviewers independently extracted study characteristics and the quality of included reviews was assessed in duplicate with AMSTAR-2, a critical appraisal tool for systematic reviews. Review quality was rated critically low, low, moderate, or high based on 7 domains: protocol registration, literature search adequacy, exclusion criteria, risk of bias assessment, meta-analytical methods, result interpretation, and presence of publication bias. Summary tables were constructed, and confidence ratings were provided for each outcome by using a previously modified version of the Royal College of General Practitioners’ clinical guidelines.

Main Results. Six systematic reviews containing 87 unique studies were included. We found that OEND programs produce long-term knowledge improvement regarding opioid overdose, improve participants’ attitudes toward naloxone, provide sufficient training for participants to safely and effectively manage overdoses, and effectively reduce opioid-related mortality. High-concentration intranasal naloxone (> 2 mg/mL) was as effective as intramuscular naloxone at the same dose, whereas lower-concentration intranasal naloxone was less effective. Evidence was limited for other naloxone formulations, as well as the need for hospital transport after overdose reversal. The preponderance of evidence pertained persons who use heroin.

Author’s Conclusions. Evidence suggests that OEND programs are effective for reducing opioid-related mortality; however, additional high-quality research is required to optimize program delivery.

Public Health Implications. Community-based OEND programs should be implemented widely in high-risk populations.

PLAIN-LANGUAGE SUMMARY

Overdose education and naloxone distribution (OEND) programs are crucial for preventing opioid fatalities. These programs provide training to people likely to witness an overdose and deliver critical information about overdose prevention, recognition, and response. However, given the proliferation of ultrapotent synthetic opioids such as fentanyl in illicit drug supplies, uncertainties exist concerning optimal naloxone formulation and patient management. Furthermore, although several reviews on the impact of these programs have been published, evidence regarding the effectiveness of these programs and their impact on vital public health measures remains uncertain. Therefore, we synthesized 6 systematic reviews to provide a broad-based conceptual scheme of the effect and feasibility of OEND programs and to identify areas for possible optimization. We found unanimous evidence suggesting that OEND programs produce long-term knowledge improvements, improve participants’ attitudes toward naloxone, provide sufficient training for participants to manage overdoses safely and effectively, and effectively reduce opioid-related mortality. We also found that high-concentration intranasal naloxone was as effective as intramuscular naloxone at the same dose, whereas lower-concentration intranasal naloxone was less effective. Most evidence concerned persons who use heroin. This study suggests that OEND programs effectively reduce opioid-related mortality and should be implemented widely in high-risk populations to prevent harm.

In North America, the rate of drug overdose deaths involving opioids has risen 5-fold over the past 2 decades.¹ Presently, opioids are the leading cause of injury-related deaths, resulting in more than 60 000 fatalities in the United States annually.² While several strategies have been put forward to prevent opioid-related mortality, one of the main strategies is naloxone distribution. Naloxone is an opioid-receptor antagonist that reverses overdose-induced respiratory depression, preventing secondary cardiac arrest and death. Overdose education and naloxone distribution (OEND) programs provide naloxone to individuals at risk for witnessing an overdose and training in 3 key areas: prevention, recognition, and response. However, given the proliferation of ultrapotent synthetic opioids in the illicit drug supply, uncertainties exist concerning optimal naloxone dosing and patient management.^3‒5 While the World Health Organization has recommended the expansion of public access to naloxone, evidence supporting the effectiveness of OEND programs remains unclear.

Several systematic reviews on the impact of these programs and their effectiveness on various outcomes have been published. However, to our knowledge, there have been no attempts to synthesize and critically evaluate the evidence from these systematic reviews on key questions of public health to date. Therefore, we conducted an umbrella review of systematic reviews to provide a broad-based conceptual scheme of the effect and feasibility of OEND and to identify areas for possible optimization. We reviewed the effects and feasbility associated with OEND in adults, assessed the efficacy of different naloxone formulations and the need for after-overdose hospital transport, and considered the implications of our findings for the development of future programs.

METHODS

An umbrella review is a comprehensive review of reviews that evaluates all evidence associated with a particular topic.⁶ In the conduct of an umbrella review, the analytical unit of inclusion and data analysis is the systematic review. We followed a protocol submitted to PROSPERO,⁷ but, because of delays, the protocol was transferred to Open Science Framework⁸ (https://osf.io/rfzbm). This umbrella review was reported in accordance with the Preferred Reporting Items for Overviews of Systematic Reviews reporting benefits and harms checklist.⁹

We addressed the following public health questions:

• 1.Are OEND programs effective at improving participants’ knowledge of overdose symptoms, risk factors, and response strategies; attitudes toward naloxone (e.g., willingness to accept or support public distribution); or ability to correctly manage a suspected overdose?
• 2.Are OEND program participants likely to administer naloxone, how effective is bystander naloxone use, and are there any potential harms with bystander naloxone use or distribution?
• 3.What are the optimal formulations for naloxone, and should overdose victims rescued in out-of-hospital settings receive hospital transport for additional care and monitoring?
• 4.Are OEND programs effective or cost-effective at reducing the incidence of opioid-related mortality?

After data extraction and in response to themes reflected in systematic reviews identified in our search, we modified our protocol to address questions regarding public attitudes, after-overdose care, and cost-effectiveness.

Search Strategy

We used a broad search for interventions addressing the opioid crisis. An academic librarian searched PubMed, Embase, PsycINFO, Epistemonikos, and the Cochrane Database of Systematic Reviews from database inception to October 31, 2019. The search algorithm is reported in detail in Appendix A (available as a supplement to the online version of this article at http://www.ajph.org). Briefly, we used a 2-concept search, which included opioid subject headings and relevant keywords with a modified version of the PubMed systematic review filter.¹⁰ We also manually searched reference lists of relevant articles for additional reviews. Citation management was done with EndNote ×9 (Clarivate Analytics, Philadelphia, PA).

Study Selection

Two reviewers (A. R., J .G.) independently screened the titles and abstracts of reviews against prespecified criteria. Reviews marked as potentially eligible proceeded to full-text assessment, with disagreements resolved by consensus. Eligible reviews (1) described comprehensive search strategies and inclusion and exclusion criteria, (2) evaluated the quality or risk of bias of included studies, (3) were published in English or French, and (4) reported data relevant to either the safety or effectiveness of OEND programs or optimal strategies for the management of opioid overdose with naloxone in out-of-hospital settings. We present our inclusion and exclusion criteria in Appendix B (available as a supplement to the online version of this article at http://www.ajph.org). Conference abstracts were excluded as they contain insufficient information for quality assessment.

Data Extraction and Quality Assessment

Two reviewers (A. R., J. G.) independently extracted the first author, publication year, outcomes reported, number of primary studies, and characteristics (i.e., design, follow-up, results) by outcome and review from eligible publications. Disagreements between reviewers were resolved by consensus or, when necessary, by a third reviewer (S. B. W.). The quality of included reviews was similarly assessed in duplicate using AMSTAR-2, a critical appraisal tool for systematic reviews.¹¹ Reviews were rated critically low, low, moderate, or high quality based on 7 critical domains: protocol registration, literature search adequacy, exclusion criteria, risk of bias assessment, meta-analytical methods, result interpretation, and presence of publication bias.

Data Synthesis

We constructed summary tables for the included reviews, synthesizing evaluated outcomes and the primary studies contributing to these outcomes. Confidence ratings were provided for each outcome using a previously modified version of the Royal College of General Practitioners’ clinical guidelines.¹² Evidence was classified as strong (⋆ ⋆ ⋆: high- or moderate-quality reviews with consistent results from multiple randomized controlled trials), moderate (⋆ ⋆: high- or moderate-quality reviews with consistent evidence from non‒randomized controlled trials or less consistent evidence from randomized controlled trials), limited or contradictory (⋆: mixed or inconsistent evidence from low- or moderate-quality reviews), or inconclusive (?; inconclusive evidence with theoretical support). One reviewer (A. R.) performed the initial confidence assessments, and final ratings were determined by consensus among members of the project team. Given heterogeneity among the systematic reviews’ primary studies, synthesis was qualitative only.

RESULTS

Our literature search yielded 8119 citations (Figure 1). Of these, we retrieved 491 full-text articles on any interventions addressing the opioid crisis. Six systematic reviews evaluated the use of naloxone for overdose rescue and were included in the umbrella review. Excluded citations are cataloged in Appendix C (available as a supplement to the online version of this article at http://www.ajph.org). The year of publication of the included reviews ranged from 2014 to 2019, and their primary studies ranged from 1996 to 2018 (Table 1). After we excluded primary study overlap (Appendix D, available as a supplement to the online version of this article at http://www.ajph.org), the reviews included 87 unique studies with randomized controlled trial, time-series, cohort, case‒control, or cross-sectional designs. One review included most of the included primary studies (n = 65; Table 1); however, only 11 were included in their narrative synthesis. All primary studies were conducted in high- or upper-middle-income countries (Australia, Canada, Denmark, Finland, Germany, Iran, Russia, United Kingdom, and United States). However, all but 1 of the primary studies specifically evaluating OEND program implementation were conducted in the United States or the United Kingdom.

FIGURE 1—

PRISMA Flow Diagram of Study Selection for Systematic Reviews Evaluating the Use of Naloxone-Based Interventions for Overdose Rescue

TABLE 1—

Characteristics of Systematic Reviews Evaluating the Use of Naloxone-Based Interventions for Overdose Rescue

Authors, Country	Focus	Context	Outcomes Examined	Primary Studies	Primary Study Quality Assessment	Data Synthesis	Review Qualitya
Clark et al.,13 United States	OEND program effectiveness	Community-based programs	Naloxone safety and efficacy, knowledge improvement, bystander overdose response, overdose mortality	19 (2001–2013)	Checklist adapted from Jinks et al.14	Qualitative	Lowb
Giglio et al.,15 United States	OEND program effectiveness	Community-based programs	Bystander naloxone efficacy, knowledge improvement	8 (2006–2014)	Checklist adapted from Jinks et al.14	Quantitative	Critically lowc
EMCDDA,16 Portugal	OEND program effectiveness	Community-based programs	Naloxone safety and efficacy, knowledge improvement, bystander overdose response, participant attitudes, overdose mortality	21 (2001–2014)	Cochrane Handbook for Systematic Reviews of Interventions17	Qualitative	Moderate
McDonald and Strang,18 United Kingdom	OEND program effectiveness	Community-based programs	Naloxone safety and efficacy, causality,d feasibility of OEND implementatione	22 (2001–2015)	Checklist adapted from Jinks et al.14	Qualitative	Lowf
Chou et al.,19 United States	Optimal overdose management	Out-of-hospital emergency response	Optimal naloxone formulation, need for hospital transport after overdose rescue	13 (1996–2016)	Methods Guide for Effectiveness and Comparative Effectiveness Reviews20	Qualitative	High
Haegerich et al.,21 United States	Various harm-prevention strategies	Community-based programs, EDs, primary care	Knowledge improvement, public and participant attitudes, overdose mortality	65g (2005–2018)	Study design and sample size examination adapted from GRADE22	Qualitative	Critically lowc

Note. ED = emergency department; EMCDDA = European Monitoring Centre for Drugs and Drug Addiction; EMS = emergency medical services; GRADE = Grading of Recommendations, Assessment, Development, and Evaluations Framework; OEND = overdose education and naloxone distribution.

^aAssessed in duplicate using AMSTAR-2, a critical appraisal tool for systematic reviews on health care interventions.

^bDowngraded from moderate because authors did not mention study preregistration or justification for restricting the literature search to English-language studies.

^cDowngraded from low because authors did not mention study preregistration, give justification for restricting the literature search to English-language studies, or provide a list of excluded studies.

^dEvaluated through 9 criteria developed by Sir Bradford Hill to assess the causal effects of public health interventions in the absence of experimental research.²³

^eEvaluated through 5 criteria developed by the World Health Organization to assess feasibility and implementation of public health interventions.²⁴

^fDowngraded from moderate because authors did not mention study preregistration or provide a list of excluded studies.

^gFrom a total of 251 primary studies on various interventions addressing the opioid crisis, 65 reported on overdose education and naloxone distribution.

The 6 included reviews^{13,15,16,18,19,21} evaluated a wide range of outcomes associated with the implementation of OEND programs. Five reviews^{13,15,16,18,21} evaluated the effectiveness of OEND programs through diverse outcomes, while the sixth review¹⁹ evaluated different modes of naloxone administration and the need for hospital transport (i.e., secondary care) after overdose reversal. Most reviews did not specify their target population; participants of included studies were thus primarily self-identified persons who use heroin unless stated otherwise in the results that follow. Because of substantial heterogeneity across the primary studies, only 1 review conducted a meta-analysis.¹⁵ We report the level of evidence and summary statements for each outcome identified in our umbrella review in Table 2.

TABLE 2—

Summary Results of Systematic Reviews Evaluating the Use of Naloxone-Based Interventions for Overdose Rescue

Outcome	No. Unique Studiesa	Main Conclusions and Comments	Level of Evidenceb
Knowledge improvement	4 systematic reviews13,15,16,21 11 NRSI (n = 19–525) 2 RCT (n = 187–197)	Strong evidence that OEND training produces long-term knowledge improvements regarding overdose recognition, overdose risk factors, overdose response, and naloxone administration. Assessments were primarily test-based. All retrieved research provided positive results.	⋆ ⋆ ⋆
Attitudes toward naloxone	2 systematic reviews16,21 6 NRSI (n = 19–525) 2 RCT (n = 187–1598)	Moderate evidence to suggest that educational interventions improve attitudes toward naloxone use among users of opioids. In the general public, factual information and a sympathetic narrative is most effective at producing positive attitudes. Conclusions are primarily based on 2 trials.	⋆ ⋆
Management of overdose	2 systematic reviews13,16 14 NRSI (n = 19–1942) 0 RCT	OEND training may improve OEND participant’s ability to respond to overdose; however, it is unclear which educational components are most effective. Results varied considerably across included observational studies.	⋆
Likelihood of naloxone use	1 systematic review16 10 NRSI (n = 19–385) 1 RCT (n = 187)	Moderate likelihood that two thirds of OEND participants will administer take-home naloxone in the event of a suspected opioid overdose. Conclusions are based primarily on 7 observational studies with active participant follow-up.	⋆ ⋆
Safety and efficacy	3 systematic reviews13,15,18 21 NRSI (n = 24–2912) 0 RCT	Strong evidence bystander-administered naloxone is both highly efficacious and safe. Consistent results were found across all retrieved studies in this domain. Ethical limitations (e.g., consent) bar the conduct of RCTs.	⋆ ⋆
Optimal naloxone formulation	1 systematic review19 4 NRSI (n = 93–609) 3 RCT (n = 100–182)	At the same dose, high-concentration intranasal naloxone is as effective as intramuscular naloxone, whereas lower-concentration formulations (2 mg/5 mL; > 0.5 mL/nostril) are less efficacious but associated with less risk of agitation. Conclusions are based on 2 medium risk of bias RCTs. Evidence was insufficient to compare other modes of naloxone administration.	⋆ ⋆
Need for hospital transport	1 systematic review19 6 NRSI (n = 84–2241) 0 RCT	The need for secondary care (e.g., hospital transport) after successful overdose reversal is inconclusive. Primary studies were at high risk for bias and did not examine linkage to treatment programs for opioid abuse.	?
Overdose-related mortality	4 systematic reviews13,16,18,21 2 NRSI (n = NRc to 2912) 0 RCT	Moderate evidence that OEND implementation significantly reduces overdose-related deaths in communities with high participant uptake. Evidence is primarily based on a quasi-experimental interrupted time-series analysis with low risk of bias.	⋆ ⋆
Cost-effectiveness	1 systematic review18 2 NRSI (n = NR) 0 RCT	OEND implementation is accessible and cost-effective even under conservative circumstances (e.g., rising naloxone prices; decreased number of observed opioid overdoses). Conclusions are based on consistent results from 2 cost-effectiveness modeling studies simulated in Russia and the United States.	⋆

Note. ⋆ ⋆ ⋆ = strong evidence: high- or moderate-quality systematic reviews demonstrating consistent results from multiple randomized controlled trials; ⋆ ⋆ = moderate evidence: high- or moderate-quality systematic reviews demonstrating consistent evidence from nonintervention studies or less consistent evidence from randomized controlled trials; ⋆ = limited or contradictory evidence: mixed or inconsistent evidence from multiple low- or moderate-quality reviews; ? = limited or inconclusive evidence: inconclusive research evidence at present, but some theoretical support; NR = not relevant; NRSI = nonrandomized studies of interventions; OEND = overdose education and naloxone distribution; RCT = randomized controlled trial.

^aThe primary studies assessing each outcome and overlap between reviews are provided in Appendix D (available as a supplement to the online version of this article at http://www.ajph.org).

^bLevels of evidence are based on a previously modified version of the Royal College of General Practitioners’ clinical guidelines.¹²

^cApproximately 3500 vials of naloxone were distributed by Chicago Recovery Alliance’s outreach workers to an unknown number of individuals.

Quality Assessment

We rated 1 review as high quality,¹⁹ 1 as moderate,¹⁶ 2 as low,^13,18 and 2 as critically low^15,21 (Appendix E, available as a supplement to the online version of this article at http://www.ajph.org). Critical domains with the lowest adherence were protocol registration (33%) and literature search adequacy (33%). The 4 reviews that received lower quality ratings did not preregister their reviews, did not provide justification for primarily including English-language articles, or did not provide a list of excluded studies. The majority of reviews (83%) justified the selection of primary study designs, accounted for risk of bias when interpreting and discussing results, and reported potential sources of conflicts of interest.

• Key question 1: Are OEND programs effective at improving knowledge, attitudes, or overdose management?

Knowledge (confidence, ⋆ ⋆ ⋆). Four reviews^13,15,16,21 assessed changes in knowledge after overdose education on risk factors, response strategies, and signs of opioid overdose. All reviews concluded that overdose education demonstrated a strong association with participant's improvement in knowledge about overdose symptoms, prevention, and care. These conclusions were based on consistently favorable test results across the available primary research, which included 2 randomized controlled trials^25,26 conducted among individuals prescribed opioids and family members of people who use heroin (Appendix F, available as a supplement to the online version of this article at http://www.ajph.org). In the trial²⁶ (n = 197) specifically identified by Haegerich et al.,²¹ the mean percentage of correct responses from individuals using prescription opioids increased from 41.8% to 73.8% when comparing pretest and posttest scores about overdose knowledge and response. Evidence from 4 studies^27‒30 included in the review by Clark et al.¹³ further suggest that knowledge improvements are retained long term (at least 2–6 months). The results suggest that educational interventions can result in substantial knowledge improvements among people likely to witness overdose. However, it remains unclear which of the tested curriculums were most effective.

Attitudes (confidence, ⋆ ⋆). Two reviews^16,21 assessed the effect of education on naloxone attitudes (e.g., willingness to accept and use naloxone or support public distribution). The first review by the European Monitoring Centre for Drugs and Drug Addiction (EMCDDA)¹⁶ focused primarily on users of injection drugs and concluded that there is weaker but consistent evidence that education produces naloxone-positive attitudes (Appendix F). The EMCDDA reported similar findings from 1 randomized controlled trial³¹ (n = 187) that found significantly higher attitude scores among family members of persons who use heroin compared with controls (mean difference = 7.5; 95% confidence interval [CI] = 3.1, 11.8; Appendix F). The second review by Haegerich et al.²¹ assessed interventions targeting the public’s attitudes to naloxone distribution. They found that educational communications that delivered information with a sympathetic narrative were most effective at increasing public support for naloxone policies when compared with factual information alone (e.g., support for naloxone distribution; odds ratio [OR] = 2.0; 95% CI = 1.4, 2.9). This conclusion was based on a large randomized controlled trial (n = 1598), thus giving strength to their conclusion. Overall, the results from the included reviews suggest that overdose education can produce positive attitudes toward the acceptance, use, and public distribution of naloxone in users of injection drugs and the public.

Overdose management (confidence, ⋆). Two reviews^13,16 assessed the impact of overdose education on participants’ capacity to correctly manage a suspected overdose. Both reviews concluded that educational interventions appear effective at improving participants’ use of recommended response strategies (e.g., attempting to restore consciousness using a sternal rub, cardiopulmonary resuscitation, placing victims in the recovery position), compared with no previous training. These conclusions were based primarily on prospective self-reported participant data (Appendix F) without comparison groups, thus limiting confidence in these results. In addition, when considering the impact of educational interventions on the rate of emergency medical service (EMS) calls, both reviews reported mixed findings and separately concluded that most participants do not call EMS. As a result, although training interventions may potentially improve participants’ capacity to manage overdose, these interventions probably do not address barriers that avert further patient care.

Key question 2: Are OEND participants likely or able to safely and effectively use naloxone?

Likelihood (confidence, ⋆ ⋆). One review¹⁶ assessed the likelihood that program participants administer naloxone in the event of a witnessed overdose. The review by the EMCDDA¹⁶ concluded that approximately 67% of participants who witnessed an overdose administered rescue naloxone. Data were based on 7 nonrandomized studies with prospective pre‒post designs and active participant follow-up (n = 387 witnessed overdoses; Appendix F). Data from their 1 included randomized controlled trial were excluded given the small number of overdoses witnessed among study participants (n = 13). Reasons for nonadministration were not readily reported, but 1 primary study suggested that most participants did not carry naloxone with them regularly; thus, it was not always available for use.²⁸ Taken together, these results suggest that OEND programs provide sufficient training for participants to confidently administer naloxone in the event of an overdose emergency. However, potential barriers to naloxone administration remain unclear.

Safety and efficacy (confidence, ⋆ ⋆). Three reviews^13,15,18 assessed the effectiveness of bystander naloxone administration. All 3 reviews analyzed the outcomes of reported naloxone administrations and given the consistent and favorable results across all the retrieved studies, concluded that bystander naloxone administration was highly effective (Appendix F). The review by McDonald and Strang¹⁸ synthesized all the available data (n = 2336 naloxone administrations), yielding a conservative overdose survival rate of 96.3% (95% CI = 95.5, 97.1) with naloxone administration. Two reviews^13,18 assessed the incidence of adverse events and found a weak association with bystander naloxone administration. The only reported physiological risks associated with naloxone were precipitated cases of opioid withdrawal, vomiting, agitation, and rare instances of seizure. Nonphysiological adverse events including problems with police, treatment programs, first responders, and shelters because of naloxone possession were also reported. Overall, conclusions regarding effectiveness are interpreted with caution given that most included studies utilized uncontrolled prospective study designs and self-reported participant data (Appendix F). Bearing in mind this limitation, it is not ethically possible to conduct a randomized controlled trial on the effectiveness of emergency bystander naloxone administration.

• Key question 3: What are the optimal formulations for take-home naloxone, and should overdose victims be transported for hospital care?

Optimal formulation (Confidence, ⋆ ⋆). One review¹⁹ assessed the comparative benefits and harms of different routes and doses of naloxone administration. Chou et al.¹⁹ found that at the same dose (2 milligrams [mg]), high-concentration intranasal naloxone (2 mg per milliliter [mL]) was as effective as intramuscular naloxone (OR = 0.7; 95% CI = 0.3, 1.5) at reversing overdose-induced respiratory depression, whereas lower-concentration intranasal naloxone (2 mg/5 mL) was found less effective than intramuscular injection (OR = 0.4; 95% CI = 0.2, 0.8), but associated with lowered risk of agitation. These findings were based on 2 randomized controlled trials (n = 327) conducted in Australia by paramedics before the rise of ultrapotent synthetic opioids (Appendix F). As a consequence, the generalizability to the present North American epidemic is unclear. The authors could not compare other modes of naloxone administration because of methodological limitations.

Hospital transport (Confidence, ?). Chou et al.¹⁹ also assessed the need for hospital transport after the reversal of overdose with naloxone in out-of-hospital settings. Chou et al.¹⁹ synthesized the data from 6 retrospective studies (n = 4397 overdose events; Appendix F) and found low risks of death and other serious adverse events (0%–1.3%) among nontransported victims after overdose rescue by paramedics or mobile emergency physicians. Reported follow-up periods varied widely among the included studies (12 hours to 1 year). The authors concluded that the risks associated with overdose victims refusing transport are likely to be low. However, given the lack of comparison groups across the included studies, the evidence is insufficient to compare the risks associated with transport versus nontransport.

• Key question 4: Are OEND programs effective and cost-effective at reducing opioid-related mortality?

Opioid-related mortality (confidence, ⋆ ⋆). Four reviews^13,16,18,20 assessed the impact of OEND programs on population-level opioid-related mortality and concluded that their implementation effectively reduces mortality in community settings. The supporting evidence comes predominantly from a single interrupted time-series analysis³² (n = 2912), which was included in all 4 reviews (Appendix F). This quasi-experimental study assessed the annual rate of opioid-related deaths across 19 communities in Massachusetts that implemented OEND programs over a period of 7 years. Communities with high or low OEND program recruitment experienced substantially greater reductions in overdose mortality than communities without OEND programs (high vs none: adjusted risk ratio [ARR] = 0.5; 95% CI = 0.4, 0.8; low vs none: ARR = 0.7; 95% CI = 0.6, 0.9). The large magnitude effect with high implementation and observed dose‒response relationship provided substantial strength to the review’s conclusions. One of the reviews, by McDonald and Strang,¹⁸ assessed the validity of the conclusion using the Bradford Hill criteria for causation, which was developed to assess causality when only observational data exist.²³ The authors concluded that evidence from their review adhered to all 9 criteria (e.g., temporality, strength of association) and, thus, supported a causal relationship between OEND program implementation and decreased opioid mortality.

Cost-effectiveness (confidence, ⋆). McDonald and Strang¹⁸ similarly assessed the cost-effectiveness of OEND programs as part of a 5-criterion feasibility checklist developed by the World Health Organization for public health interventions.²⁴ The authors concluded that OEND programs are cost-effective even under markedly conservative circumstances (i.e., when the cost of naloxone rises and rates of opioid overdose decrease) on the basis of consistent findings between 2 cost-effectiveness modeling studies^33,34 (Appendix F). Their conclusion was interpreted with caution given the limited amount of primary research. Separately, McDonald and Strang¹⁸ found that OEND programs fully or partially met the remaining 4 feasibility criteria: (1) absence of negative consequences, (2) feasibility of program expansion and coverage, (3) unanticipated benefits, and (4) capacity to reach high-risk populations (e.g., medical detoxification patients, prison inmates, homeless injection drug users, and methadone patients).

DISCUSSION

Our umbrella review was designed to synthesize and appraise evidence from systematic reviews pertinent to 4 key public health questions regarding the implementation of naloxone-based interventions for overdose rescue. Our results suggest that there is credible evidence that OEND programs produce long-term knowledge improvements regarding opioid overdose, improve attitudes toward naloxone use and distribution, provide sufficient training for participants to safely reverse overdose, and effectively reduce opioid-related mortality at the population level. In terms of naloxone administration, higher-concentration intranasal formulations (e.g., > 2 mg/mL) appear to be as effective as comparable intramuscular doses, while lower-concentration intranasal formulations are less effective. Evidence was inconclusive for other modes of naloxone administration, as well as the need for hospital transport after overdose reversal. The majority of available evidence pertained to self-identified persons who use heroin, with limited data available for persons who use prescription opioids. Together, the previously studied associations with the greatest internal validity suggest that OEND programs are an effective strategy to address the opioid crisis, although some knowledge gaps remain.

Increasingly, the United States and Canada have expanded legal routes for public access to naloxone. This includes third-party prescription laws, which allow health care providers to issue naloxone prescriptions to individuals likely to be first responders in the event of an overdose and standing orders that permit pharmacists to prescribe and educate persons who use opioids about naloxone. These policies have been associated with substantial increases in naloxone distribution.³⁵ Coprescription laws, which mandate clinicians to coprescribe naloxone to patients who are at high overdose risk from a prescribed medication (e.g., doses greater than 50 mg of daily morphine equivalent), have similarly increased public access.³⁶ Based on our findings, the continued expansion of these initiatives alongside existing OEND programs is likely to contribute substantially to reducing the incidence of opioid mortality.

Despite the recent expansion in public access to naloxone, additional research is needed to optimize OEND programs, particularly in epicenters of the opioid crisis where the presence of synthetic opioids (e.g., fentanyl) has caused disproportionate harms. More data are needed concerning barriers to naloxone administration: only two thirds of OEND program participants reported using their naloxone kits when witnessing an overdose. The distribution of more portable intranasal naloxone kits may improve responder rates as bystanders may be more willing to carry and administer convenient needle-free systems.^37‒39 We found that higher-concentration intranasal naloxone (2 mg/mL) had similar efficacy to intramuscular naloxone at the same dose (2 mg). The US Food and Drug Administration has recently approved use of an ultraconcentrated nasal spray (Narcan; 4 mg/0.1 mL) based on bioavailability comparable with the highest recommended intramuscular injection dose (2 mg).⁴⁰ The transition to needle-free intranasal systems would have the added benefit of preventing needle-stick injuries and, thus, the transmission of bloodborne diseases including hepatitis C and HIV.^41‒43 Future research should also focus on determining the optimal timing and need for repeat naloxone administration if adequate respiration or consciousness are not restored following an initial dose. Naloxone titration techniques that result in more gentle reversal of overdose may prevent debilitating withdrawal symptoms, which may precipitate further opioid-seeking behavior and impede naloxone acceptance among people who inject drugs.^37,38 There is some evidence that lower initial doses and routes of administration with a slower onset could minimize these effects.^44,45

Furthermore, additional research is required to inform the development of educational interventions that more effectively improve bystander response to the witnessed overdose. We found that while OEND programs may increase appropriate first-aid responses to overdose (e.g., use of sternal rubs, rescue breathing, recovery position), the vast majority of trained bystanders did not call EMS.^13,16 While observational data suggest that individuals who are not transported to the hospital following a naloxone-treated overdose have a low likelihood of related adverse events, evidence remains insufficient to conclude that transport is not needed.¹⁹ Contact with secondary care for overdose victims may improve linkage to treatment of problematic substance abuse and testing for infectious diseases (e.g., HIV, hepatitis C).^46,47 Therefore, future studies should assess barriers to contacting EMS and evaluate the effect of remediating strategies such as Good Samaritan laws. Despite the limitations of observational data, we found that OEND programs remain an effective strategy for reducing opioid mortality.

Strengths and Limitations

Umbrella reviews facilitate the efficient evaluation of large bodies of evidence and the production of comprehensive overviews, enabling evidence-informed decision-making and policy practices. Our study’s strengths included utilizing systematic reviews as the analytical unit for inclusion and data analysis, providing evidence addressing a range of key questions regarding the effects associated with OEND, and feasibility considerations that are required for program implementation and optimization. Our overview specifically provides a direct qualitative analysis of a multitude of outcome domains that have not previously been covered, to our knowledge, in any individual or pair systematic reviews. As a result, our umbrella review represents an evidence map of the highest available evidence regarding naloxone-based interventions for fatal overdose prevention. This report can thus facilitate evidence-based decision-making practices in this field, which may have been previously hindered by fragmented literature.

Nevertheless, by their nature, umbrella reviews have several limitations. First, the quality of evidence in an umbrella review is dependent on that of the included systematic reviews, of which the majority here were rated low. This limitation was addressed by considering the quality of included reviews when producing confidence ratings for our principal findings based on the adapted Royal College of General Practitioners’ clinical guidelines.¹² Second, we did not evaluate the risk of bias or quality of the individual primary studies, as this is typically outside the scope of umbrella reviews. As part of our inclusion criteria, however, we required that eligible systematic reviews include an assessment of their primary studies, which we found all but 1 review explicitly considered when drawing their conclusions.

Third, overlap between the included primary studies could have placed undue emphasis on the conclusions of commonly cited articles. However, study overlap was considered in our synthesis and citation matrices by review and outcome were reported to enhance transparency. Fourth, overviews can obscure substantial clinical and methodological heterogeneity in the included reviews and primary studies (that is, differences in baseline characteristics of participants, including age and sex, and differences in study design such as variable inclusion of comparison groups).

Finally, the majority of data available were from pre‒post studies conducted in self-identified persons who use heroin; therefore, our findings may be considered limited if approached from the perspective of traditional evidence hierarchies. However, randomized controlled trials or cluster randomized trials to evaluate the impact of OEND on various outcomes described previously would likely present formidable practical and ethical challenges.⁴⁸ With our use of the umbrella review methodology, our findings indicate that there are numerous reviews that provide unanimous support regarding the effectiveness of OEND programs. While these results may be difficult to generalize to persons who use prescription opioids, injection drug use—particularly with heroin in communities with illicit drug supplies contaminated with superpotent opioids—is the largest contributor to opioid mortality.⁴⁹ As a consequence, OEND programs are likely to have the most significant impact within these high-risk populations.

Conclusions

Our umbrella review was designed to provide a comprehensive overview of the available evidence from systematic reviews pertinent to the development and implementation of OEND programs in response to the opioid crisis. We found credible evidence that OEND programs improve individuals’ knowledge and attitudes about opioid overdose, enable participants to safely and effectively use naloxone, and reduce opioid-related mortality in community settings. Additional research is needed to identify and minimize the barriers to naloxone use and to better understand the need for after-overdose care. Community-based OEND programs should be implemented widely in high-risk populations to reduce the burden of the opioid epidemic.