Skip to main content
PMC home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2018 Apr 27.
Published in final edited form as: Drug Alcohol Depend. 2017 Apr 1;173(Suppl 1):S39–S47. doi: 10.1016/j.drugalcdep.2016.12.003

Randomized controlled trial of a computerized opioid overdose education intervention

Kelly E Dunn a,*, Claudia Yepez-Laubach a, Paul A Nuzzo a, Michael Fingerhood b, Anne Kelly b, Suzan Berman b, George E Bigelow a
PMCID: PMC5918281  NIHMSID: NIHMS958422  PMID: 28363318

Abstract

Background

Opioid overdose (OD) has become a significant public health problem in need of effective interventions. The majority of existing educational interventions target provision of naloxone and are conducted in-person; these elements present logistical barriers that may limit wide-spread implementation. This study developed and evaluated an easily disseminated opioid OD educational intervention and compared computerized versus pamphlet delivery

Methods

Participants (N = 76) undergoing opioid detoxification were randomly assigned to receive OD education via a Pamphlet (N = 25), Computer (N = 24), or Computer + Mastery (N = 27) with identical content for all delivery modalities. Primary outcomes were changes from pre- to post-intervention in knowledge of opioid effects, opioid OD symptoms, and recommended opioid OD responses, as well as intervention acceptability. Also assessed at 1 and 3-month follow-ups were retention of knowledge and change in reported OD risk behaviors.

Results

Knowledge increased following all three intervention-delivery modalities with few between-group differences observed in knowledge gain or acceptability ratings. Largest gains were in the domain of opioid OD response (from 41.8% to 73.8% mean correct responses; p < 0.001). Knowledge was well sustained at the 1 and 3-month follow-ups among completers, where a significant reduction was seen in the critical behavioral risk factor of using opioids while alone.

Conclusion

Opioid overdose education delivered by computer or written pamphlet produced sustained increases in knowledge and reduction in a key behavioral risk factor.

Results

Results support further evaluation of this educational intervention that can be used alone or to complement naloxone-training programs.

Keywords: Opioid, Overdose, mHealth, Naloxone, Opioid use disorder

1. Introduction

Abuse of heroin and prescription opioids increased over the past decade (Compton et al., 2016) and in 2013 more than 2.4 million adults sought treatment for opioid use disorder (OUD) (Substance Abuse and Mental Health Services Administration (SAMHSA), 2014). There has been a corresponding increase in opioid-related consequences, including overdose (OD). In 2014, accidental poisonings, largely driven by opioid OD, were the leading cause of accidental death among US adults, and the CDC estimated that 78 persons died from opioid-related OD each day (Centers for Disease and Prevention (CDC), 2014). Deaths related to prescription opioids and heroin has increased 3 and 6 fold in the past 10 years, respectively (Rudd et al., 2016). Major medical associations (Harris, 2016), government agencies (Rudd et al., 2014; CDC 2016; Furlow, 2016), and the White House (Office of the Press Secretary, 2016) have all formally acknowledged the opioid OD epidemic and called for action.

Educational and training interventions can help opioid users and people in their environment prevent and address symptoms of opioid OD. The majority of existing OD interventions focus on skill-building and generally convene a small group of participants for in-person meetings. Intervention content may include discussion of the signs and symptoms of OD, review of OD vignettes, and presentation of behavioral demonstrations with opportunities to practice appropriate OD reversal techniques including administration of the opioid antagonist naloxone (Green et al., 2008; Strang et al., 2008; Jones et al., 2014; Lott and Rhodes, 2016). A limited set of measures is available for evaluation of such training interventions. The Brief Overdose Recognition and Response Assessment (BORRA; Green et al., 2008) requires patients to successfully differentiate OD from non-OD in different vignettes, and the Opioid Overdose Knowledge Scale (OOKS; Williams et al., 2013) presents 45 naloxone training-related items. However, due in part to the urgency of the overdose epidemic, not all OD education and training programs have undergone formal evaluation.

Despite their widespread utilization, existing OD education and training interventions have some potential limitations. First, content has been heavily focused on skill-building and use of naloxone for OD reversal, with relatively little effort allocated towards standardization or evaluation of the remaining educational information. While naloxone-training programs have been associated with impressive reductions in fatal ODs (Walley et al., 2013), there are logistical barriers in some settings to the use of naloxone. In most states, naloxone is not yet available over-the-counter and, therefore, requires a prescription from a qualified physician (Davis et al., 2013; Hewlett and Wermeling, 2013). The cost of naloxone, which has continued to rise, may also be prohibitive for some programs (Thompson, 2015; Gupta et al., 2016). In addition, the in-person nature of most training programs can be a barrier since this requires dedicated staff time from trained interventionists, which can be costly as well as variable in training quality and fidelity.

There is value in developing a standardized educational intervention to provide uniform and consistent information about opioid OD prevention. This type of program could be delivered as a stand-alone intervention or as a complement to naloxone-training sessions and could be available in settings where there are barriers to naloxone distribution or training. A meta-analysis of naloxone training programs reports their educational components produce significant post-intervention and sustained knowledge gains (Giglio et al., 2015), which supports this approach. Importantly, delivery of educational interventions via computer can also circumvent some limitations of and barriers to in-person trainings. A recent meta-analysis reported that computer-based delivery of behavioral health interventions was effective for producing knowledge gain of health behaviors when compared to minimal intervention comparison conditions such as pamphlets (Krebs et al., 2010). Further, both meta-analytic (Krebs et al., 2010) and empirical (Silverman et al., 1991) studies support superiority in knowledge gains of interventions that require participants to demonstrate topic mastery relative to direct presentation of material to be learned. These findings are consistent with the behavioral analytic approach of instructional design to increase knowledge retention (Engelmann and Carnine, 1982 Johnson and Layng, 1992), which is commonly employed in web-based learning environments (Mi, 2016; Taveira-Gomes et al., 2016).

Ultimately, a computerized educational intervention for opioid OD could fill an important gap in the resources that are currently available to combat opioid OD, while maintaining a high potential for dissemination. The current study developed and evaluated an educational intervention that focused on knowledge of opioid effects, opioid OD symptoms and risk factors, and recommended opioid OD response in the absence of naloxone. The study question was whether knowledge would be increased following exposure to the training and whether delivery modality (computer vs. pamphlet) would differentially influence knowledge gain or risk behavior outcomes. Participants were randomly assigned to receive OD education with identical content delivered via a pamphlet or one of two computer-based programs, one of which incorporated a mastery training approach. Based on behavioral health literature, it was anticipated that computer-based training would produce greater knowledge gains relative to the pamphlet delivery comparison.

2. Methods

2.1. Participants

Participants (N = 76) were recruited between May 2015 and September 2015 from a 5-day outpatient, hospital-based, opioid detoxification unit located in Baltimore, MD. This population has a confirmed history of opioid use disorder (OUD) and was chosen because patients leaving detoxification are considered high risk for opioid OD (Davoli et al., 2007; Clausen et al., 2009; Britton et al., 2010; Ravndal and Amundsen, 2010; Degenhardt et al., 2011). Participants who were 18 or older and being treated for OUD were eligible. Participants who had physical limitations that prevented them from using a computer, participated in the pilot test of the intervention (described below), or completed the Baltimore-based Staying Alive naloxone OD prevention program (Tobin et al., 2009) were excluded from the study. The Johns Hopkins IRB approved this study and all participants provided voluntary informed consent to participate.

2.2. Study methods

2.2.1. Curriculum development

The curriculum content was based upon previous OD educational interventions (Strang et al., 2008; Green et al., 2008). Content was developed by the study team, was designed to be direct, simple, and precise, and emphasized knowledge in three domains: opioid effects, opioid OD risks and symptoms, and effective opioid OD response (Table 1). Content was intended to represent general concepts that would be applicable for diverse patient populations, versus concepts specific to drug users (such as injection risk behaviors). To increase potential generalizability and potential to complement naloxone training programs, naloxone administration was not discussed.

Table 1.

Intervention Content.

Knowledge Domain Target Topics Covered
Opioid Basic understanding of opioids Basic pharmacology of opioids
Effects of opioids on body that are relevant to OD risk Long vs. short-acting opioids
Examples of opioids (generic and brand names)
Opioid tolerance and withdrawal
Opioid OD Overdose (OD) What is an opioid OD, potential timecourse of an OD, OD can be fatal or nonfatal
Behaviors reported in literature to increase risk of a fatal or nonfatal OD Concurrent alcohol and polydrug use, loss of tolerance, beginning chronic opioid treatment, combining long and short acting opioids, history of previous non-fatal OD, presence of pain
Symptoms (internal and external), guidance on distinguishing an OD from an agonist effect (e.g., high) Miscolored lips or skin, vomiting, being nonresponse or unconscious, limpness, slow pulse, choking sounds, slow irregular breathing, stereotypic posture
Opioid OD Response Dispelling myths reported in the literature Do not inject victim with milk or salt water, put victim in cold water or ice bath, or induce vomiting
Effective responses Contact 911, guidance on what to tell operator
Administer sternal rub
Administer rescue breathing
Move victim to recovery position
Continue to monitor and provide support to victim while waiting for medical professionals
Open in a new tab

2.2.2. Pilot testing

The computerized version of the intervention was pilot-tested to determine that information was delivered in a clear and logical format and that multi-media features worked properly. OUD patients (N = 6) recruited via flyers from a local treatment program provided voluntary informed consent to participate and were compensated $25 in gift cards. Pilot participants completed the computerized intervention, rated its difficulty and clarity, and described problems they experienced. Participants had no suggestions for changes, reporting that content was clear and understandable and that program navigation was simple; pilot-testing therefore ended and trial recruitment began.

2.2.3. Study intervention groups

2.2.3.1. Pamphlet

A pamphlet was given to a control group to be read during the observed study session. To ensure content and visual organization was identical across all three groups, pamphlets were print outs of the content displayed in the computerized interventions. Participants were instructed to review the pamphlet and inform staff when they were ready to proceed with the post-test. No restrictions were placed on time spent reviewing the pamphlet and time was not recorded. All pamphlets were temporarily removed before the participant began the post-test and returned to the participant at the end of the session.

2.2.3.2. Computer intervention

The Computer intervention was hosted through the online survey manager Qualtrics. It contained 3 slides to introduce the participant to the computerized system and 25 educational slides that combined text, picture, and/or videos (Table 1). No restrictions were placed on the manner in which the participant interacted with the program and time spent completing the intervention was measured by Qualtrics. No mastery questions were presented during this intervention.

2.2.3.3. Computer + Mastery

The Computer + Mastery intervention was identical to the Computer intervention, however this group was required to achieve ≥80% accuracy on questions that were embedded in the domains of opioid effects (52 questions), opioid OD (46 questions), and opioid OD response (16 questions) to advance the program. Embedded questions were different from primary knowledge questions. Failure to meet this threshold prompted the information to start over, which occurred a maximum of three times before the intervention was automatically advanced. Due to limitations in Qualtrics, participants did not receive corrective feedback for incorrect responses. No additional restrictions were placed on the manner in which the participant interacted with the program.

2.2.4. Study procedures

At the single study session, participants completed baseline measures and the knowledge Pre-test (described below). Participants were then randomly assigned to one of the three intervention groups using the following stratification variables: sex, history of experiencing an opioid OD (yes/no), and presence of chronic pain (yes/no). The intervention began immediately following group randomization. A study staff member was in the room with the participant to provide technical assistance (e.g., problems with the computer mouse), but did not answer questions related to intervention content. Upon completion, all materials were removed and the participant completed the knowledge assessment a second time (Post-test), as well as an intervention acceptability questionnaire. Participants received $75 in gift cards for this session.

Before leaving the session, participants were scheduled for one and three-month follow-up visits. To encourage retention, participants were issued appointment cards with their follow-up dates that included study contact information and the compensation value of the visit. Participants were asked for multiple forms of contact (e.g., several phone numbers, addresses for letters) and study staff began reminder calls several weeks before scheduled visits. Taxicab service was made available as needed, and follow-up visit compensation was set intentionally high ($75, $100, respectively) to encourage attendance. During follow-up visits, participants completed the same battery of measures and provided a urine sample that was tested for evidence of relapse to opioid use. A staff member who was blinded to participant group assignment conducted all follow-up visits. Overall, 42% and 44% percent of participants completed 1 and 3-month follow ups, respectively, with 57% completing at least one follow-up.

2.2.5. Study measures

Study measures characterized participant demographic and drug use history, assessed OD knowledge, experience, and engagement in risk behaviors, and intervention acceptance.

2.2.5.1. Screening measures

Participants completed the Brief Pain Inventory (Cleeland and Ryan, 1994), a self-report measure that defines chronic pain as pain that has persisted for three months, which was used as a stratification variable. Participants also completed the Wide Range Achievement Test (WRAT), a standardized, observer-administered measure, to determine their relative reading grade level standardized against age and sex norms. Results were similar across groups, so this was not used as a covariate of outcomes.

2.2.5.2. Opioid and opioid OD knowledge

At the time of this study, existing OD knowledge measures (e.g., the BORRA, OOKS) targeted the successful provision of naloxone following a training intervention and were not considered appropriate outcomes for this intervention. Therefore, a 51-item self-report measure that targeted the domains taught in the curriculum (Table 1) was developed. Responses were presented as “True”, “False”, and “I don’t know”, the latter to discourage random guessing and prevent answers from being correct accidentally. The answer to every question was present in the intervention content. Results were coded dichotomously as correct and incorrect with “I don’t know” coded as incorrect, and percent correct was analyzed for each knowledge domain.

2.2.5.3. OD experience and engagement in risk behaviors

Past 30-day engagement in behaviors were assessed at baseline and follow-ups to evaluate whether risk behaviors may change as a function of knowledge gain. Risk behaviors included number of days using opioids alone (with no one available to administer aid if needed) (Davidson et al., 2003; Dietze et al., 2006; Shah et al., 2008), combining opioids with alcohol (Seal et al., 2001; Coffin et al., 2003; Davidson et al., 2003; Dietze et al., 2006; Coffin et al., 2007; Laberke and Bartsch, 2010), using the long-acting opioid methadone (Bunn et al., 2010; Webster et al., 2011), and using opioids after a change in opioid tolerance (e.g., having recently completed an opioid detoxification or being released from jail/prison) (Seal et al., 2001; Merrall et al., 2010; Ravndal and Amundsen, 2010; Kinner et al., 2012). Participants were also asked whether they perceived these behaviors as increasing risk of opioid OD (yes/no).

2.2.5.4. Acceptance questionnaire

During the post-test, participants completed 10 acceptance items rated on a scale from one (strongly agree) to five (strongly disagree) that stated “The educational intervention …”: “ was helpful”; “taught me information I didn’t know before”; “was easy to understand”; “was fun”; “was too long”; “was interesting”; and “was confusing”. Additional items included “I would recommend this intervention to someone else”, “I believe that more people should receive this educational intervention”, and “I do not think the educational intervention was useful”. Participants were also asked whether they thought the intervention would help prevent them from overdosing in the future (yes/no), whether they thought the intervention would change the way they would help other people who are overdosing (yes/no), how important they believe it is to learn how to prevent, recognize, and respond to an OD (very, somewhat, or not important), and whether they would recommend this intervention to a friend or family member (yes/no).

2.3. Data analyses

An a priori power analysis based upon previous computerized intervention (Silverman et al., 1991) and opioid OD knowledge (Strang et al., 2008) studies and assuming an alpha of 0.05 indicated 20 participants per group would yield 90% power to detect large between-group main effects on knowledge. Groups were compared on demographic and drug use characteristics and significant differences on variables that may have impacted knowledge would have been included as covariates in the analyses, though none were identified. The primary outcome for this study was the change in knowledge as a function of group (Pamphlet, Computer, Computer + Mastery) and session (pre-test, post-test, 1 month, 3 month), within each of the topic domains (opioid, opioid OD, and opioid OD response knowledge).

Demographic, drug use, OD history variables, acceptance questions, and attendance at the follow-up visits were reported descriptively and compared across groups using one-factor models for continuous and Chi-square analyses for dichotomous variables. Time to complete the computerized intervention was compared between the computer groups using independent groups t-tests. Time in session was not recorded for the pamphlet group.

Primary outcome data were analyzed in two ways. First, data from the pre- and post-test results and the post-treatment acceptance measure were analyzed across all participants as an intent-to-treat (ITT) analysis. Mean percent correct ratings for each of the knowledge domains, were compared using two-factor (group, session) models. Next, a Completer analysis (defined as completing at least one follow-up visit) was used to evaluate outcomes among the 57% of participants classified as Completers; the likelihood of completing the one (42.2%; p = 0.73) or three (43.7%; p = 0.43) month follow-ups did not differ across groups. Two factor models were used to evaluate main effects of group and session (pre-test, post-test, 1 month, 3 month) on knowledge outcomes, and group and session (pre-test, 1 month, 3 month) on risk behaviors and perceptions. All models were run using Proc Mixed for continuous variables and GEE for dichotomous variables. Posthoc testing was conducted with Tukey’s tests, alpha levels were set at 0.05, and analyses were conducted using SAS 9.3 for Windows (SAS Institute, Cary, NC) and SPSS version 23.0.

3. Results

3.1. Participants

Seventy-six participants were randomized into a Pamphlet (N = 24), Computer (N = 24), or Computer + Mastery (N = 27) group and completed the intervention; group demographic and drug use characteristics are presented in Table 2. Overall, 36% of participants reported experiencing an OD themselves in their lifetime (mean (SD) 2.63 (2.0), range of 1–8) and 54% reported previously witnessing someone overdosing (mean (SD) 2.73 (2.7), range of 1–14). A minority of participants (16%) had previously received OD information or been prescribed naloxone, and only two participants had previously administered naloxone. No OD variables differed significantly between groups at baseline (Table 2).

Table 2.

Participant Demographics, Drug Use, and Overdose History.

Pamphlet (N = 25) Computer (N = 24) Computer + Mastery (N = 27) p-value
Demographics
 Age (years) 41.9 (14.1) 37.8 (10.6) 40.0 (13.1) 0.53
 Male (%) 52.0 70.8 55.6 0.36
 Caucasian (%) 60.0 83.3 81.5 0.44
 Married (%) 20.0 33.3 18.5 0.40
 Employed (%) 20.0 37.5 25.9 0.38
 Chronic Pain (%) 52.0 54.2 48.1 0.91
 WRAT Reading Grade Equivalent 12.0 (1.3) 12.4 (0.5) 12.1 (1.3) 0.50
Drug Use and OD History
 Past 30-day Use (%)
 Heroin 100 95.8 100 0.33
 Prescription Opioids 60.0 70.8 74.1 0.53
 History of IV Drug Use (%) 60.0 58.3 66.7 0.81
 History of Agonist Maintenance Treatment (%) 88.0 70.8 66.7 0.18
OD History
 OD ever (%) 33.3 41.7 33.3 0.78
 Experienced Naloxone Reversal (%) 8.3 0.0 0.0 0.12
 Witnessed OD (%) 45.8 50.0 66.7 0.23
Source of Previous OD Intervention (%)
 Detoxificationa 72.7 60.0 50.0 0.54
 Methadone Maintenancea 40.0 40.0 33.3 0.94
 Buprenorphine Maintenancea 33.3 20.0 33.3 0.85
 Needle Exchangea 20.0 20.0 41.7 0.47
 Jail or Prisona 11.1 60.0 16.7 0.09
 Friendsa 33.3 20.0 10.0 0.46
 Othera 50.0 33.3 20.0 0.73
 Previous Prescription for Naloxone (%) 12.3 8.3 11.1 0.89
 Trained in CPR (%) 21.7 43.5 43.5 0.21
Open in a new tab

Values represent Mean (Standard Deviation) unless otherwise indicated. WRAT = Wide Range Achievement Test, IV = intravenous; OD = overdose, CPR = Cardiopulmonary Resuscitation.

a

Values represent percent of participants who endorsed previous OD training.

3.2. Intervention delivery

Participants in the Computer 16.3 (±18.5) and Computer + Mastery 19.8 (±12.0) groups spent similar amounts of time completing the program (p = 0.45). Within the Computer + Mastery group, 36% of participants successfully answered questions related to opioids on the first presentation; 6% and 58% of participants were successful on 2nd and 3rd presentations, respectively. Seventy-two percent of participants successfully answered questions related to opioid OD on the first presentation; 9% and 19% of participants were successful on 2nd and 3rd presentations, respectively. Finally, 88% of participants successfully answered questions related to opioid OD response on the first presentation; 11% and 1% of participants required 2nd and 3rd presentations, respectively.

3.3. Knowledge test results

3.3.1. ITT analyses (Fig. 1)

Fig. 1.

Fig. 1

Open in a new tab

Opioid effects, Opioid OD symptoms, and Opioid OD Response mean knowledge outcomes between the pretest (Pre) and post-test (Post)assessments. Data represent all enrolled participants. Symbols represent the Pamphlet (circle; N = 25), Computer (triangle; N = 24), and Computer + Mastery (square; N = 27) groups. Y-axis represents mean percent correct. A significant main effect of time was observed for all three outcomes; a significant main effect of group was observed between the Pamphlet and Computer + Mastery group in the opioid knowledge domain.

The ITT analyses of mean percent correct revealed a significant main effect of session between the pre and post-test (F(1,73) = 9.73, p < 0.01) and group (F(2,73) = 5.21, p < 0.01) on opioid knowledge, driven by differences between the Pamphlet and Computer Groups (p = < 0.01). A significant main effect of session (F(1,73) = 7.07, p = 0.01) was evident on opioid OD knowledge, with no significant main effect of group. As shown in Fig. 1, there was a large significant main effect of session (F(1,73) = 129.53, p < 0.001) on opioid OD response knowledge, where mean percent correct increased from 41.8% to 73.8% between the pre and post-test, respectively; no significant main effect of group or group x session interactions were observed.

3.3.2. Completer analysis (Fig. 2)

Fig. 2.

Fig. 2

Open in a new tab

Opioid effects, Opioid OD symptoms, and Opioid OD Response mean knowledge outcomes across all assessments (pretest (Pre), post-test (Post), 1 month, 3 month). Data derived from participants who completed at least one follow-up visit. Symbols represent the Pamphlet (circle; N = 15), Computer (triangle; N = 15), and Computer + Mastery (square; N = 13) groups. Y-axis represents mean percent correct. A significant main effect of time was observed for all domains; the opioid OD domain was significantly higher than baseline at the 1-month follow-up, and the opioid OD response domain was significantly higher than baseline at the 1 and 3-month follow-ups, independent of group. A significant main effect of group was also observed, whereby the Pamphlet group performed more poorly on the opioid knowledge domain relative to the Computer group, and on the opioid OD knowledge domain relative to both the Computer and Computer + Mastery groups.

The Completer analysis of mean percent correct similarly revealed a significant main effect of session (F(3,98) = 2.72, p = 0.05) and group (F(2,40) = 3.95, p = 0.03) on opioid knowledge, driven by differences between the Pamphlet and Computer Groups (p = 0.04). Significant main effects of session (F(3,98) = 4.30, p < 0.01) and group (F(2,40) = 9.18, p < 0.01) were also evident on opioid OD knowledge, driven by differences between the Pamphlet and Computer (p < 0.01) and Computer + Mastery (p = 0.01) groups. A significant main effect of session (F(3,98) = 35.74, p < 0.001) was also evident on opioid OD response knowledge, driven by differences between the pre-test and the 1 (p < 0.001) and 3 (p < 0.001) month follow-ups. Neither main effect of group nor group x session interactions reached significance.

3.4. Follow-up risk behaviors

Though none of the Completer sample reported experiencing an OD during the follow-up period, 9.4% and 3.3% of participants reported witnessing an OD between the intervention and the one and three-month follow-ups, respectively. By the one (81%) and three (77%) month follow-up visits, the majority of participants in the completer sample provided a urine sample that tested positive for an opioid, indicating relapse to opioid use. Within the Completers, a significant main effect of session (F(2.45) = 9.69, p < 0.001) was observed on days using opioids by themselves, which decreased from a mean 17.6/30 days at baseline to 6.5/30 days at the one month follow-up (p < 0.001), with a trend toward return to baseline at 3-months (11.4/30 days). A trend was also observed for days with combined use of opioids and alcohol (F(2.49) = 2.94, p = 0.07), which decreased from a mean of 10.3/30 days at baseline to 3.4/30 days at the 1-month follow-up (p = 0.05), and remained low at 5.2/30 days at 3-months. There were no significant group differences in participant perceptions of behaviors that increased risk of OD (Table 3).

Table 3.

Past 30 day Risk Behaviors Among Study Completers.

Baseline 1 month 3 month
Pamphlet (N = 15) Computer (N = 15) Computer + Mastery (N = 13) Pamphlet (N = 15) Computer (N = 15) Computer + Mastery (N = 13) Pamphlet (N = 15) Computer (N = 15) Computer + Mastery (N = 13) Group Session
Risk Behaviors
 Days used opioids and alcohol together (Mean, SD) 10.3 (13.8) 10.4 (13.3) 5.6 (11.4) 0.3 (0.7) 6.5 (12.4) 2.9 (6.2) 6.9 (13.1) 5.4 (11.6) 3.4 (6.7) 0.52 0.06
 Days used opioids alone with no one else around (Mean, SD) 19.1 (11.1) 20.1 (11.0) 14.1 (13.6) 8.5 (9.6) 4.7 (9.4) 6.3 (11.3) 7.9 (12.8) 14.6 (13.9) 11.6 (11.2) 0.31 <0.01a
Perceived Risks
 Mixing opioids with other psychoactive substances (%) 84.6 86.7 100 81.8 90.9 100 80.0 83.3 70.0 0.57 0.30
 Using opioids alone without anyone else around (%) 69.2 73.3 92.3 63.6 90.9 80.0 70.0 90.9 80.0 0.25 0.97
 Being abstinent from opioids for a period of time (%) 61.5 73.3 50.0 63.6 72.7 70.0 70.0 100 60.0 0.26 0.26
 Being incarcerated for a period of time (%) 61.5 35.7 50.0 63.6 54.5 70.0 50.0 54.5 70.0 0.50 0.43
 Using opioids with a chronic medical condition (%) 84.6 92.3 69.2 90.1 81.8 77.8 77.8 75.0 66.7 0.37 0.50
 Being inducted onto methadone maintenance or taking methadone pills (%) 53.8 53.8 66.7 60.0 63.6 77.8 33.3 58.3 62.5 0.54 0.26
Open in a new tab

Analyses refer to subset of respondents classified as Completers. No group x session interactions were significant so values are not shown. SD = standard deviation.

a

Post-hoc testing revealed significant difference between baseline and 1 month follow-up.

3.5. Intervention acceptability

All three groups rated their intervention experience positively (Table 4), with only two of 14 questions indicating a small advantage for the Computer interventions over the Pamphlet. Participants in the Computer and Computer + Mastery groups rated their intervention as somewhat easier to understand (χ2(6) = 13.8, p = 0.03), and were less likely to indicate the intervention was confusing (χ2(6) = 15.58, p = 0.05). Further, 100% of those who received a computer intervention vs. 96% of those receiving the Pamphlet thought the intervention would help keep them from overdosing in the future (χ2(2) = 8.2, p = 0.02). Almost every participant, independent of group, said it was “Very Important” to learn to prevent, recognize, and respond to an OD, 97% believed the intervention would change the way they helped people who were overdosing, and 100% of participants said they would recommend their intervention condition to a family member or friend.

Table 4.

Acceptance of the Intervention.

Pamphlet (N = 25) Computer (N = 24) Computer + Mastery (N = 27) p-value
The educational intervention (Range 1–5; lower values represent greater agreement):a
 Was helpful 1.6 (0.9) 1.4 (0.9) 1.6 (0.9) 0.73
 Taught me information I did not know before 1.6 (0.7) 1.5 (0.9) 1.5 (0.9) 0.21
 Was easy to understand 1.9 (0.9) 1.3 (0.5) 1.5 (0.5) 0.03
 Was fun 2.6 (1.2) 2.3 (0.9) 2.2 (1.1) 0.42
 Took too long 2.7 (1.3) 3.3 (1.0) 3.2 (1.4) 0.14
 Was interesting 2.2 (1.2) 1.8 (1.0) 1.7 (0.7) 0.33
I would recommend this educational intervention to someone elsea 2.2 (1.1) 1.5 (1.0) 1.7 (0.9) 0.20
I believe that more people should receive this educational interventiona 1.8 (1.2) 1.4 (1.0) 1.6 (1.0) 0.43
I DO NOT think that the educational intervention was usefula 4.0 (1.3) 4.2 (1.0) 4.3 (0.9) 0.47
The educational intervention was confusinga 3.7 (1.3) 4.2 (1.0) 3.9 (1.1) 0.05
How important is it to learn to prevent, recognize, and respond to an overdose (Range 0–2; higher values represent greater agreement)b 2.0 (0) 2.0 (0) 2.0 (0.2) 0.39
I believe this intervention will help prevent me from overdosing in the future (%) 96.0 100.0 100.0 0.02
I believe this intervention will change the way I help people who are overdosing (%) 96.0 95.8 100 0.58
I would recommend this intervention to a family member or friend (%) 100 100 100
Open in a new tab

Values represent Mean (Standard Deviation) unless otherwise indicated.

a

Rated on 5-point scale: 1 = Strongly Agree, 2 = Agree, 3 = Neither Agree not Disagree, 4 = Disagree, 5 = Strongly Disagree.

b

Rated on 3-point scale: 0 = Not at all Important, 1 = Somewhat Important, 2 = Very Important.

4. Discussion

This study developed and evaluated in a randomized, controlled design, the relative impact that an OD educational intervention (delivered via a Pamphlet, Computer, or Computer + Mastery format) had on three domains of OD knowledge and risk behaviors. The three modalities produced relatively comparable knowledge gains and there were no significant group x time interactions to support differential superiority of a delivery modality. The largest gains were observed in the opioid OD response domain, which focused on sternal rubs, recovery breathing, and the rescue position. Knowledge increases were largely sustained in the subgroup (57%) of participants completing the 1 and 3-month follow-ups. Exposure to all three modalities was also associated with changes in an important OD risk behavior of using opioids while alone. Participants also rated all methods very favorably, with 100% stating they would recommend their intervention condition to a family member or friend. Results provide preliminary support for the effectiveness of an educational intervention to increase knowledge about opioids, opioid OD, and opioid OD response in patients being treated for OUD.

This study enrolled participants undergoing opioid detoxification because they are at high risk of experiencing a fatal OD. Participants entered the study with high baseline knowledge about opioids and opioid OD, but not opioid OD response (Fig. 1). The fact that participants were regularly abusing opioids, may have personally experienced and/or witnessed an OD, and may have been the target of previous OD interventions could have contributed to existing knowledge and resulted in a ceiling effect that prevented the intervention from producing larger improvements in opioid and opioid OD knowledge. Nevertheless, the marked increase in knowledge of OD response (excluding naloxone) is notable and has important potential public health implications. Other populations, such as individuals receiving opioid prescriptions for chronic pain, may have less baseline knowledge about opioids and opioid OD (Dunn et al., 2016a) and may benefit more from the basic modules.

These results complement those of a recent web-based OD educational program (Roe and Banta-Green, 2016) and support the continued evaluation of opioid OD educational interventions more broadly. Computer-delivered intervention may have advantages in terms of cost and reach as a delivery method, though research to determine conditions under which various modalities may be most effective is warranted. Failure to support the hypothesis that computer would be superior to pamphlet delivery appeared due to relatively good performance of the pamphlet group, and absence of any evidence for superiority for the computer + mastery versus computer alone interventions may have been due to delivery limitations of mastery without feedback to participants. In the present study, content of the pamphlet was matched to that from the computerized slides and participants were provided with structured time during the session to read the pamphlet contents, both of which may have resulted in good performance under this condition. Other more ecologically valid pamphlet-delivered interventions should be considered in future research as comparators. While knowledge gain is a worthy end-point of trainings, it will also be important in the future to determine whether educational interventions delivered by various modalities improve behavioral response to OD in simulated and real life OD situations, as has been shown for naloxone administration behaviors following in-person skill-building trainings (Seal et al., 2005; Maxwell et al., 2006; Green et al., 2008, 2013; Wagner et al., 2010).

Educational interventions should produce changes in personal risk behaviors in order to have public health impact, and this was examined as a secondary outcome of knowledge gains. Outcomes analyzed within Completers revealed a significant decrease in the number of days participants used opioids while alone, a major risk behavior for experiencing a fatal OD. There was also a trend towards a decreased number of days combining opioids and alcohol. These findings are promising and support additional program development and research to promote sustained changes in risk behaviors over a longer duration of time.

Strengths of the study included a randomized, controlled design with a blinded rater for follow-up data collection. Significant limitations include the lack of a no intervention control, which limits interpretation of intervention effectiveness, and loss to follow-up of 43% of the sample, which impedes the ability to draw definitive conclusions about long-term effects. Other limitations include a participant population with high baseline knowledge that may limit generality of findings, relatively small sample size that lacked power to detect small between-group differences and interactions, use of an information-matched pamphlet that may lack ecological validity, lack of information about how participants used the pamphlet with anticipation of an impending post-test potentially influencing the manner or intensity with which participants read the pamphlet, inability to provide corrective feedback in the Computer + Mastery condition, and possibility that expectancy bias based on exposure to the intervention may have influenced responding about OD risk behaviors. Finally, several advancements have occurred since the onset of the study relevant to the educational curriculum and its evaluation. The next stage of development should potentially include information related to naloxone, since its use has become more widely supported through changes in laws (Davis et al., 2013; Davis and Carr, 2015) and development of new naloxone products (Traynor, 2016); reference to fentanyl and co-benzodiazepine use, which have become recognized as major drivers of OD deaths (Jones and McAninch, 2015; Gladden, 2016); and use of newly developed and psychometrically-supported outcome measures for evaluation (Dunn et al., 2016b).

5. Conclusion

A brief educational intervention produced gains in several domains of OD knowledge, and particularly knowledge of effective response to an OD in absence of naloxone, among patients being treated for OUD in a detoxification unit. The educational intervention was also associated with decreases in the important OD risk behavior “using while alone”, and was rated highly by participants for clarity and importance. A computerized delivery method may be increasingly feasible and acceptable with advances in technology infrastructure, with promise for a broad dissemination platform that can circumvent logistical barriers associated with in-person trainings and potential under-utilization of information delivered via written materials. Overall, the opioid OD educational prevention intervention, whether delivered as a stand-alone service or as a complement to naloxone training, fills an important gap in the continuum of resources available for combating the opioid OD epidemic.

Acknowledgments

Role of funding

This study was supported by the grants from the National Institutes of Health:R01DA035246 (Dunn), R21DA035327 (Dunn), and T32DA007209 (Bigelow). The funding source had no role in the data collection or interpretation.

The authors thank the staff members and patients from the Chemical Dependency Unit for their assistance with recruitment and implementation of the intervention.

Footnotes

Conflict of interest

The authors have no relevant conflicts of interest to declare.

Contributors

The study was designed by authors Dunn and Bigelow. All authors contributed substantively to the data collection, interpretation, analysis, and manuscript development.

References

  1. Britton PC, Wines JD, Jr, Conner KR. Non-fatal overdose in the 12 months following treatment for substance use disorders. Drug Alcohol Depend. 2010;107:51–55. doi: 10.1016/j.drugalcdep.2009.09.005. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Bunn TL, Yu L, Spiller HA, Singleton M. Surveillance of methadone-related poisonings in Kentucky using multiple data sources. Pharmacoepidemiol Drug Saf. 2010;19:124–131. doi: 10.1002/pds.1901. [DOI] [PubMed] [Google Scholar]
  3. Centers for Disease and Prevention (CDC) [Last Accessed on 12 August 2016];Wide-Ranging OnLine Data for Epidemiologic Research (WONDER) [Online] 2014 Available at https://wonder.cdc.gov/
  4. Centers for Disease Control and Prevention. Guideline for prescribing opioids for chronic pain. J Pain Palliat Care Pharmacother. 2016;30:138–140. doi: 10.3109/15360288.2016.1173761. [DOI] [PubMed] [Google Scholar]
  5. Clausen T, Waal H, Thoresen M, Gossop M. Mortality among opiate users: opioid maintenance therapy, age and causes of death. Addiction. 2009;104:1356–1362. doi: 10.1111/j.1360-0443.2009.02570.x. [DOI] [PubMed] [Google Scholar]
  6. Cleeland CS, Ryan KM. Pain assessment: global use of the brief pain inventory. Ann Acad Med Singap. 1994;23:129–138. [PubMed] [Google Scholar]
  7. Coffin PO, Galea S, Ahern J, Leon AC, Vlahov D, Tardiff K. Opiates, cocaine and alcohol combinations in accidental drug overdose deaths in New York City, 1990–98. Addiction. 2003;98:739–747. doi: 10.1046/j.1360-0443.2003.00376.x. [DOI] [PubMed] [Google Scholar]
  8. Coffin PO, Tracy M, Bucciarelli A, Ompad D, Vlahov D, Galea S. Identifying injection drug users at risk of nonfatal overdose. Acad Emerg Med. 2007;14:616–623. doi: 10.1197/j.aem.2007.04.005. [DOI] [PubMed] [Google Scholar]
  9. Compton WM, Jones CM, Baldwin GT. Relationship between nonmedical prescription-opioid use and heroin use. N Engl J Med. 2016;374:154–163. doi: 10.1056/NEJMra1508490. [DOI] [PubMed] [Google Scholar]
  10. Davidson PJ, McLean RL, Kral AH, Gleghorn AA, Edlin BR, Moss AR. Fatal heroin-related overdose in San Francisco, 1997–2000: a case for targeted intervention. J Urban Health. 2003;80:261–273. doi: 10.1093/jurban/jtg029. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Davis CS, Carr D. Legal changes to increase access to naloxone for opioid overdose reversal in the United States. Drug Alcohol Depend. 2015;157:112–120. doi: 10.1016/j.drugalcdep.2015.10.013. [DOI] [PubMed] [Google Scholar]
  12. Davis C, Webb D, Burris S. Changing law from barrier to facilitator of opioid overdose prevention. J Law Med Ethics. 2013;41(Suppl 1):33–36. doi: 10.1111/jlme.12035. [DOI] [PubMed] [Google Scholar]
  13. Davoli M, Bargagli AM, Perucci CA, Schifano P, Belleudi V, Hickman M, Salamina G, Diecidue R, Vigna-Taglianti F, Faggiano F VEdeTTE Study Group. Risk of fatal overdose during and after specialist drug treatment: the VEdeTTE study, a national multi-site prospective cohort study. Addiction. 2007;102:1954–1959. doi: 10.1111/j.1360-0443.2007.02025.x. [DOI] [PubMed] [Google Scholar]
  14. Degenhardt L, Bucello C, Mathers B, Briegleb C, Ali H, Hickman M, McLaren J. Mortality among regular or dependent users of heroin and other opioids: a systematic review and meta-analysis of cohort studies. Addiction. 2011;106:32–51. doi: 10.1111/j.1360-0443.2010.03140.x. [DOI] [PubMed] [Google Scholar]
  15. Dietze P, Jolley D, Fry CL, Bammer G, Moore D. When is a little knowledge dangerous? Circumstances of recent heroin overdose and links to knowledge of overdose risk factors. Drug Alcohol Depend. 2006;84:223–230. doi: 10.1016/j.drugalcdep.2006.02.005. [DOI] [PubMed] [Google Scholar]
  16. Dunn KE, Barrett FS, Yepez-Laubach C, Meyer AC, Hruska BJ, Sigmon SC, Fingerhood M, Bigelow GE. Brief Opioid Overdose Knowledge (BOOK): a measure to assess overdose knowledge in indiviiduals who use ilicit or prescribed opioids. J Addict Med. 2016a doi: 10.1097/ADM.0000000000000235. (in press) [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Dunn KE, Barrett FS, Yepez-Laubach C, Meyer AC, Hruska BJ, Sigmon SC, Fingerhood M, Bigelow GE. Brief Opioid Overdose Knowledge (BOOK): a questionnaire to assess overdose knowledge in individuals who use illicit or prescribed opioids. J Addict Med. 2016b;10:314–323. doi: 10.1097/ADM.0000000000000235. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Engelmann S, Carnine D. Theory of Instruction: Principles and Applications. Irvington Pub; 1982. [Google Scholar]
  19. Furlow B. FDA confronts opioid addiction and overdose deaths. Lancet Oncol. 2016;17:e95. doi: 10.1016/S1470-2045(16)00090-5. [DOI] [PubMed] [Google Scholar]
  20. Giglio RE, Li G, DiMaggio CJ. Effectiveness of bystander naloxone administration and overdose education programs: a meta-analysis. Inj Epidemiol. 2015;2:1–9. doi: 10.1186/s40621-015-0041-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Gladden RM. Fentanyl law enforcement submissions and increases in synthetic opioid-involved overdose deaths—27 states, 2013–2014. MMWR Morb Mortal Wkly Rep. 2016:65. doi: 10.15585/mmwr.mm6533a2. [DOI] [PubMed] [Google Scholar]
  22. Green TC, Heimer R, Grau LE. Distinguishing signs of opioid overdose and indication for naloxone: an evaluation of six overdose training and naloxone distribution programs in the United States. Addiction. 2008;103:979–989. doi: 10.1111/j.1360-0443.2008.02182.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Green T, Bowman S, Ray M, Kobayashi L, McKenzie M, Rich J. Board 339-research abstract patient simulation for assessment and reinforcement of layperson management of opioid overdose with intranasal naloxone in a recently released prison inmate cohort (Submission# 66) Simul Healthc. 2013;8:538. doi: 10.1097/SIH.0000000000000182. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Gupta R, Shah ND, Ross JS. The rising price of naloxone- risks to efforts to stem overdose deaths. N Engl J Med. 2016;375:2213–2215. doi: 10.1056/NEJMp1609578. [DOI] [PubMed] [Google Scholar]
  25. Harris PA. The opioid epidemic: AMA’s response. Am Fam Phys. 2016;93:975. [PubMed] [Google Scholar]
  26. Hewlett L, Wermeling DP. Survey of naloxone legal status in opioid overdose prevention and treatment. J Opioid Manag. 2013;9:369–377. doi: 10.5055/jom.2013.0179. [DOI] [PubMed] [Google Scholar]
  27. Johnson KR, Layng TJ. Breaking the structuralist barrier: literacy and numeracy with fluency. Am Psychol. 1992;47:1475. doi: 10.1037//0003-066x.47.11.1475. [DOI] [PubMed] [Google Scholar]
  28. Jones CM, McAninch JK. Emergency department visits and overdose deaths from combined use of opioids and benzodiazepines. Am J Prev Med. 2015;49:493–501. doi: 10.1016/j.amepre.2015.03.040. [DOI] [PubMed] [Google Scholar]
  29. Jones JD, Roux P, Stancliff S, Matthews W, Comer SD. Brief overdose education can significantly increase accurate recognition of opioid overdose among heroin users. Int J Drug Policy. 2014;25:166–170. doi: 10.1016/j.drugpo.2013.05.006. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Kinner SA, Milloy MJ, Wood E, Qi J, Zhang R, Kerr T. Incidence and risk factors for non-fatal overdose among a cohort of recently incarcerated illicit drug users. Addict Behav. 2012;37:691–696. doi: 10.1016/j.addbeh.2012.01.019. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Krebs P, Prochaska JO, Rossi JS. A meta-analysis of computer-tailored interventions for health behavior change. Prev Med. 2010;51:214–221. doi: 10.1016/j.ypmed.2010.06.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Laberke PJ, Bartsch C. Trends in methadone-related deaths in Zurich. Int J Leg Med. 2010;124:381–385. doi: 10.1007/s00414-010-0442-8. [DOI] [PubMed] [Google Scholar]
  33. Lott DC, Rhodes J. Opioid overdose and naloxone education in a substance use disorder treatment program. Am J Addict. 2016;25:221–226. doi: 10.1111/ajad.12364. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Maxwell S, Bigg D, Stanczykiewicz K, Carlberg-Racich S. Prescribing naloxone to actively injecting heroin users: a program to reduce heroin overdose deaths. J Addict Dis. 2006;25:89–96. doi: 10.1300/J069v25n03_11. [DOI] [PubMed] [Google Scholar]
  35. Merrall EL, Kariminia A, Binswanger IA, Hobbs MS, Farrell M, Marsden J, Hutchinson SJ, Bird SM. Meta-analysis of drug-related deaths soon after release from prison. Addiction. 2010;105:1545–1554. doi: 10.1111/j.1360-0443.2010.02990.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Mi M. Application of instructional design principles in developing an online information literacy curriculum. Med Ref Serv Q. 2016;35:112–121. doi: 10.1080/02763869.2016.1117298. [DOI] [PubMed] [Google Scholar]
  37. Office of the Press Secretary. The White House. President Obama proposes $1.1 billion in new funding to address the prescription opioid abuse and heroin use epidemic. J Pain Palliat Care Pharmacother. 2016;30:134–137. doi: 10.3109/15360288.2016.1173760. [DOI] [PubMed] [Google Scholar]
  38. Ravndal E, Amundsen EJ. Mortality among drug users after discharge from inpatient treatment: an 8-year prospective study. Drug Alcohol Depend. 2010;108:65–69. doi: 10.1016/j.drugalcdep.2009.11.008. [DOI] [PubMed] [Google Scholar]
  39. Roe SS, Banta-Green CJ. An initial evaluation of web-based opioid overdose education. Subst Use Misuse. 2016;51:268–275. doi: 10.3109/10826084.2015.1092986. [DOI] [PubMed] [Google Scholar]
  40. Rudd RA, Paulozzi LJ, Bauer MJ, Burleson RW, Carlson RE, Dao D, Davis JW, Dudek J, Eichler BA, Fernandes JC, Fondario A, Gabella B, Hume B, Huntamer T, Kariisa M, Largo TW, Miles J, Newmyer A, Nitcheva D, Perez BE, Proescholdbell SK, Sabel JC, Skiba J, Slavova S, Stone K, Tharp JM, Wendling T, Wright D, Zehner AM. Increases in heroin overdose deaths—28 states, 2010–2012. MMWR Morb Mortal Wkly Rep. 2014;63:849–854. [PMC free article] [PubMed] [Google Scholar]
  41. Rudd RA, Aleshire N, Zibbell JE, Gladden RM. Increases in drug and opioid overdose deaths—United States, 2000–2014. MMWR Morb Mortal Wkly Rep. 2016;64:1378–1382. doi: 10.15585/mmwr.mm6450a3. [DOI] [PubMed] [Google Scholar]
  42. Substance Abuse and Mental Health Services Administration (SAMHSA) Results from the 2013 National Survey on Drug Use and Health. 2014. NSDUG Series H-48. HHS Publication No. (SMA) 14–4863. [PubMed] [Google Scholar]
  43. Seal KH, Kral AH, Gee L, Moore LD, Bluthenthal RN, Lorvick J, Edlin BR. Predictors and prevention of nonfatal overdose among street-recruited injection heroin users in the San Francisco Bay Area, 1998–1999. Am J Publ Health. 2001;91:1842–1846. doi: 10.2105/ajph.91.11.1842. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Seal KH, Thawley R, Gee L, Bamberger J, Kral AH, Ciccarone D, Downing M, Edlin BR. Naloxone distribution and cardiopulmonary resuscitation training for injection drug users to prevent heroin overdose death: a pilot intervention study. J Urban Health. 2005;82:303–311. doi: 10.1093/jurban/jti053. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Shah NG, Lathrop SL, Reichard RR, Landen MG. Unintentional drug overdose death trends in New Mexico, USA, 1990–2005: combinations of heroin, cocaine, prescription opioids and alcohol. Addiction. 2008;103:126–136. doi: 10.1111/j.1360-0443.2007.02054.x. [DOI] [PubMed] [Google Scholar]
  46. Silverman K, Lindsley OR, Porter KL. Overt responding in computer-based training. Curr Psychol. 1991;9:373–384. [Google Scholar]
  47. Strang J, Manning V, Mayet S, Best D, Titherington E, Santana L, Offor E, Semmler C. Overdose training and take-home naloxone for opiate users: prospective cohort study of impact on knowledge and attitudes and subsequent management of overdoses. Addiction. 2008;103:1648–1657. doi: 10.1111/j.1360-0443.2008.02314.x. [DOI] [PubMed] [Google Scholar]
  48. Taveira-Gomes T, Ferreira P, Taveira-Gomes I, Severo M, Ferreira MA. What are we looking for in computer-based learning interventions in medical education? A systematic review. J Med Internet Res. 2016;18:e204. doi: 10.2196/jmir.5461. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Thompson CA. Naloxone access increases, as does price. Am J Health Syst Pharm. 2015;72:1426–1427. doi: 10.2146/news150056. [DOI] [PubMed] [Google Scholar]
  50. Tobin KE, Sherman SG, Beilenson P, Welsh C, Latkin CA. Evaluation of the staying alive programme: training injection drug users to properly administer naloxone and save lives. Int J Drug Policy. 2009;20:131–136. doi: 10.1016/j.drugpo.2008.03.002. [DOI] [PubMed] [Google Scholar]
  51. Traynor K. FDA approves first intranasal naloxone product. Am J Health Syst Pharm. 2016;73:e2–3. doi: 10.2146/news160002. [DOI] [PubMed] [Google Scholar]
  52. Wagner KD, Valente TW, Casanova M, Partovi SM, Mendenhall BM, Hundley JH, Gonzalez M, Unger JB. Evaluation of an overdose prevention and response training programme for injection drug users in the Skid Row area of Los Angeles, CA. Int J Drug Policy. 2010;21:186–193. doi: 10.1016/j.drugpo.2009.01.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Walley AY, Xuan Z, Hackman HH, Quinn E, Doe-Simkins M, Sorensen-Alawad A, Ruiz S, Ozonoff A. Opioid overdose rates and implementation of overdose education and nasal naloxone distribution in Massachusetts: interrupted time series analysis. BMJ. 2013;346:f174. doi: 10.1136/bmj.f174. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Webster LR, Cochella S, Dasgupta N, Fakata KL, Fine PG, Fishman SM, Grey T, Johnson EM, Lee LK, Passik SD, Peppin J, Porucznik CA, Ray A, Schnoll SH, Stieg RL, Wakeland W. An analysis of the root causes for opioid-related overdose deaths in the United States. Pain Med. 2011;12(Suppl 2):S26–35. doi: 10.1111/j.1526-4637.2011.01134.x. [DOI] [PubMed] [Google Scholar]
  55. Williams AV, Strang J, Marsden J. Development of opioid overdose knowledge (OOKS) and attitudes (OOAS) scales for take-home naloxone training evaluation. Drug Alcohol Depend. 2013;132:383–386. doi: 10.1016/j.drugalcdep.2013.02.007. [DOI] [PubMed] [Google Scholar]

RESOURCES