Patient Simulation for Assessment of Layperson Management of Opioid Overdose with Intranasal Naloxone in a Recently-Released Prisoner Cohort

Abstract

Introduction

Investigators applied simulation to an experimental program that educated, trained and assessed at-risk, volunteering prisoners on opioid overdose (OD) prevention, recognition and layperson management with intranasal (IN) naloxone.

Methods

Consenting inmates were assessed for OD-related experience and knowledge then exposed on-site to standardized didactics and educational DVD (without simulation). Subjects were provided with IN naloxone kits at time of release and scheduled for post-release assessment. At follow-up, subjects were evaluated for their performance of layperson opioid OD resuscitative skills during video-recorded simulations. Two investigators independently scored each subject’s resuscitative actions with a 21-item checklist; post-hoc video reviews were separately completed to adjudicate subjects’ interactions for overall benefit or harm.

Results

One hundred and three prisoners completed the baseline assessment and study intervention then were prescribed IN naloxone kits. One-month follow-up and simulation data were available for 85 subjects (82.5% of trained recruits) who had been released and resided in the community. Subjects’ simulation checklist median score was 12.0 (IQR 11.0–15.0) out of 21 total indicated actions. Forty-four participants (51.8%) correctly administered naloxone; 16 additional subjects (18.8%) suboptimally administered naloxone. Non-indicated actions, primarily chest compressions, were observed in 49.4% of simulations. Simulated resuscitative actions by 80 subjects (94.1%) were determined post-hoc to be beneficial overall for patients overdosing on opioids.

Conclusions

As part of an opioid OD prevention research program for at-risk inmates, investigators applied simulation to 1-month follow-up assessments of knowledge retention and skills acquisition in post-release participants. Simulation supplemented traditional research tools for investigation of layperson OD management.

INTRODUCTION

Individuals who unintentionally overdose (OD) during use of opioids, especially users recently released from prison or jail, are at significant risk for death.^1–5 The high rate of relapse to opioid use, combined with the loss of tolerance to opioids during incarceration, contributes substantially to the high morbidity and mortality in this population.¹ In light of these risks, there exists a significant opportunity for user-/community-based administration of resuscitative naloxone as an effective antidote by intervening through the peer groups and families of those most at risk.^6–15

Investigators conducting a National Institute on Drug Abuse-funded research study created a behavior change theory-based documentary-style, educational DVD for the prison population at risk of overdose.¹⁶ A companion feasibility study generated a just-in-time protocol to educate, equip and re-assess at-risk, volunteering prison inmates in opioid OD prevention, OD recognition and intranasal (IN) naloxone administration to prepare them for post-release exposures to OD situations. Using the DVD and standardized didactic materials during a pre-release ‘teachable moment,’ investigators accessed members of an especially high-risk cohort to educate and train them in the core knowledge and skills necessary to prevent, recognize and mitigate OD situations. As part of this experimental program, researchers developed and applied an opioid OD patient simulation methodology to explore its potential role in safely studying subjects’ pertinent knowledge and skills. In particular, the specific purpose of the simulation-based substudy was to examine the layperson subjects’ post-release retention of opioid OD training by having them demonstrate their ability to perform essential opioid OD resuscitation actions. This manuscript presents the development, conduct, analyses and findings of the exploratory simulation-based element within the larger research program.

METHODS

Setting and Sample

The targeted population was inmates who were within four weeks of release from the Rhode Island Department of Corrections (RIDOC) in Cranston, RI. The study specifically recruited English-speaking inmates over 18 years of age who were 1.) involved with and/or likely to be in proximity post-release to those involved in opioid abuse and 2.) volunteered to participate in the opioid OD prevention and naloxone administration training program with post-release follow-up. The exclusion criterion was not fulfilling all of the inclusion criteria. Those who expressed an interest in study participation after recruitment announcements and posters were approached privately for written informed consent by study personnel. Participation or non-participation in the overall research study did not impact the inmates’ release date, privileges, probation or parole status in any way. The study was approved by the Miriam Hospital Institutional Review Board (IRB) with a prisoner representative member, the Medical Review Advisory Group (MRAG) of the Rhode Island Department of Corrections, and the Federal Office of Human Rights Protection (OHRP).

Inmates agreeing to participate first completed baseline interviews detailing their sociodemographics and drug exposure and use. At a separate meeting time, participants were exposed on-site in small groups or one-on-one (as the prison schedule permitted) to an experimental intervention comprising the OD prevention educational DVD (accessible online at prisonerhealth.org) and standardized didactics on OD recognition, layperson opioid OD management (e.g., basic airway management, rescue breathing) and IN naloxone administration. Immediately after this session, subjects were evaluated on OD-related knowledge, including opioid OD recognition and the role of naloxone. Subjects unable to correctly answer all content questionnaire items were invited to attend additional sessions until able to do so. Immediate post-intervention simulation practice for demonstration and assessment of subjects’ skills in OD recognition and management was not feasible due to the correctional setting. For similar reasons, educational props such as simulation naloxone rescue kits and CPR barrier masks were not permitted into the facility during the study intervention. After successfully completing the intervention and initial assessment (total time approximately 30–40 minutes), each subject was provided with an IN naloxone kit around the time of release and scheduled for a one-month follow-up assessment. Because the highest risk of post-incarceration OD death is concentrated in the first two weeks after release, a follow-up period of one month was selected to permit monitoring for possible applications of preventive behaviors as well as OD recognition and management skills.

Simulation Scenario and Assessment Tool Development

After a thorough review of the literature, substance abuse, correctional health, emergency medicine and simulation researchers along with former inmates and active drug users collaboratively developed an opioid OD simulation scenario with accompanying 21-item assessment checklist for a full-body AirwayMan manikin (Laerdal, Wappingers Falls NY). The scenario called for a patient simulator seated on the floor, with palpable pulses and without respirations; there was no scripting for OD progression or response to naloxone. An IN naloxone kit (identical to those distributed to subjects at time of release) and a CPR barrier mask were made available to the participant during the simulation. Ambient street noise recordings that contained sounds of an approaching police car were played in the background; a cellphone mockup, drug-related props and distractor decoys (for common OD treatment errors) were provided. The scenario was designed to elicit and record subjects’ accessing of emergency services and the performance of basic airway maneuvers (e.g., head tilt, mouth opening), rescue breathing with barrier protection, IN naloxone kit assembly and use, avoidance of non-indicated actions, and continued monitoring. (See document files, Supplementary Digital Content, which contain the study session equipment list and setup process, orientation script and simulation protocol.)

The complementary checklist was based on the American Heart Association’s Life Support guidelines^17,18 and derived with a modified Delphi approach¹⁹ (see Table 1). Specific checklist elements, e.g., ‘recovery position,” were purposefully included as harm reduction interventions in acknowledgement of the concern that people who use drugs or who were formerly incarcerated may not be willing to stay on-scene after calling 911. Furthermore, formal validation of the scenario and checklist relative to real-world metrics was precluded by challenges associated with accessing confidential public safety service records that contained details of illicit substance abuse.

Table 1.

In-Simulation Performance of Layperson Opioid Overdose Resuscitation Actions by Study Subjects.

Opioid Overdose Management Actions for Simulation-based Assessment	Percentage of subjects completing action (n=85)	Time to completion from scenario start (sec; median [IQR1–IQR3])
Patient Assessment
1. Verbal stimulation	36.5%	5.0 [2.0–11.5]
□ Yells overdosing person’s name (victim) or yells at them
2. Physical stimulation	45.9%	10.0 [7.5–18.5]
□ Performs sternal rub (vigorous enough to shake manikin)
3. Assessment for respirations	65.9%	10.0 [7.0–18.3]
□ Checks for and verbalizes detection of the following signs of opioid overdose: not waking up and not breathing
4. Call for 911	8.2% 65.9%	75.0 [27.0–112.0]
Ventilatory Support
5. Establishment of airway
□ Lays manikin on back	62.4%	29.0 [16.0–50.5]
□ Tilts manikin’s head backwards (any angle)	38.8%
□ Opens manikin’s mouth (any opening attempt)	22.4%
6. Rescue breathing
□ Maintains backward tilting of manikin’s head (any angle)	37.6%	36.0 [21.0–59.3]
□ Supports manikin’s head with hand (hand under neck)	43.5%
□ Pinches manikin’s nose	63.5%
□ Blows into manikin’s mouth twice (chest rise required x 2)	91.8%
□ Continues rescue breathing every 5 sec for > 30 sec	42.4%
7. (Re-)Evaluation	56.5%	71.5 [49.5–96.3]
□ Checks for and verbalizes detection of the following signs of opioid overdose: not waking up and not breathing
Naloxone Administration
8. Naloxone retrieval and preparation		99.0 [70.5–101.5]
□ Retrieves and verbalizes naloxone in pre-filled syringe	91.8%
□ Removes yellow caps from syringe and red cap from naloxone vial	90.6%
□ Attaches nasal spray aspirator onto syringe	78.8%
□ Attaches naloxone vial into syringe	80.0%
9. Naloxone administration
□ Injects 1mL (half of vial) of naloxone into each manikin nasal cavity	51.5%	Time to administration from scenario start: 94.5 [70.8–125.0]
□ Injects any dose of naloxone into nasal cavity	70.6%	Time to administration from naloxone retrieval: 94.5 [71.8–125.0]
Additional Supportive Actions
10. Continued care	7.1% 28.2% 63.5%	115.0 [77.0–136.5]
- . Non-indicated actions	49.4% 37 subjects (43.5%) performed non-indicated chest compressions	n/a
□ Does not perform potentially harmful actions (e.g., applying ice or water on face; hitting or slapping manikin; leaving the manikin) or actions that were not discussed during intervention

IQR = interquartile range; mL = milliliter; sec = seconds

Two pilot sessions were completed to troubleshoot the simulation study protocol, scenario and assessment checklist.

Simulation Session Conduct

Each subject was contacted by mail and/or phone to confirm his/her scheduled one-month follow-up appointment at a community site. On arrival, subjects were queried as to whether they had experienced, been in the vicinity of, and/or managed ODs (especially with the study naloxone). After written follow-up assessments, a scripted orientation to the simulation setting was conducted with the subject. Ground rules were reviewed, e.g., termination of simulation in case of subject discomfort. The subject was then instructed to enter the pre-configured simulation space and start the scenario; one investigator (MR) videorecorded and completed real-time checklist coding of subject performance and timeliness. The scenario ended when subjects reported completing their resuscitative interactions with the simulated patient. Each participant received a $50 study incentive at session conclusion after completing a brief structured debriefing with the opportunity to self-correct (unprompted, then with open-ended prompting) any aspect of their resuscitation performance.

Data Analysis

Results from the overall experimental research program (including paper-based baseline and follow-up assessment data on subjects’ OD recognition and knowledge) are being analyzed and published separately, e.g., Green et al. 2014.²⁰ For the simulation study component, investigators used descriptive summary statistics on subjects’ sociodemographic information, opioid OD experience at baseline and follow-up, and simulation performance data.

All study session videos were independently re-coded offline by a second study reviewer (SEB) for comparison with the initial real-time checklist coding. Training of investigators for video review entailed their rating of three study session recordings, followed by comparison and group discussion of their ratings (thereby allowing for cognitive interviewing of the checklist completion process by each rater). This continued as an iterative process until all raters had consistent coding across the 21 checklist items and their time stamps. Subjects’ simulation checklist scores were analyzed with Fisher exact and Mann-Whitney U tests relative to their overdose experience (baseline and follow-up) and naloxone administration experience (follow-up).

Due to the exploratory nature of the study’s application of simulation to layperson OD resuscitation research, two investigators (TCG, SEB) independently completed post-hoc reviews of all videos for a separate, gestalt adjudication of each subject’s overall performance as either beneficial or harmful; discrepant reviews were resolved by a third reviewer. Bennett’s prevalence-adjusted bias-adjusted kappa (PABAK) coefficient, due to detected prevalence bias,^21,22 and inter-observer percent agreement were calculated for the overview assessments.

All subject information, responses and data were handled confidentially and securely stored in a Microsoft Access database with a de-identifying key code system; videos were deleted after data extraction.

RESULTS

One hundred seventy one inmates expressed an interest in study participation, of whom 107 (62.5%) consented to participate and completed the baseline assessment; 104 completed the study intervention training while incarcerated. Thirty-eight (35.5%) and 75 (70.1%) subjects had personally experienced or witnessed an opioid OD, respectively. None had previously been trained to respond to ODs or obtained a naloxone rescue kit in the community.

Ninety-nine subjects received an IN naloxone kit (96%) around the time of their release. Four additional subjects had not received their kit in person or by mail after release and were provided with one kit each at follow-up visit, and one subject could not be accessed for follow-up delivery of study naloxone. Subject demographics were representative of the target population: average age 34.8±8.9 years; 80% male; race: white 79%, African American 6.5%, Native American 6.5%, Hispanic/Latino 18%.

One-month Post-release Follow-up Assessment

At one-month follow-up, 85 participants of the 103 eligible inmates who had been trained and assessed, released and residing in the community were re-assessed. (Five subjects were re-incarcerated prior to follow-up, 12 did not present for follow-up, and 1 had expired from an opioid OD). Complete study and simulation data were available for all 85 follow-up subjects. Seven follow-up subjects (8.2%) had experienced opioid ODs after release, and 3 of the ODs were reversed using the study naloxone. In addition, subjects reported witnessing 8 nonfatal opioid ODs with 4 administrations of study naloxone, 1 performance of rescue breathing, and 1 activation of 911 services. No injuries, hospitalizations or naloxone-related complications were reported.

Simulation Session Data

Complete data extracted from 85 video-recorded simulation sessions conducted in 2012–2013 were analyzed. Independent reviewers exhibited less than 10% disagreement for checklist coding across sessions, which evolved to full agreement after joint review of the videos. Subjects’ median scores on task checklists during simulation scenarios were 12.0 (IQR 11.0–15.0) out of 21 total indicated actions. Checklist scores did not exhibit significant differences on bivariate analyses by age, gender, baseline OD and naloxone experience, and follow-up OD and naloxone experience (data not shown).

911 was activated by 72.9% of subjects via instruction to bystander (n=7) or prop phone (n=56). At least one basic airway maneuver and delivery of two rescue breaths with chest rise were completed by 56 (65.9%) and 78 (91.8%) subjects, respectively. Full naloxone doses were correctly administered into both nares by 44 (51.8%) of subjects at a median of 94.5 (70.8–125.0) seconds from scenario start; IN naloxone kit assembly and medication administration were completed in 58.0 (40.0–66.0) seconds from kit retrieval. Sixty subjects (70.6%) administered at least some naloxone intranasally: 8 participants administered naloxone into one nostril only, and 9 participants administered a partial dose. Several subjects attempted to administer naloxone intravenously (n=4), intramuscularly (n=1), orally (n=1) or intranasally without the atomizer (n=5) to the manikin. Six participants stated they did not know how to put the naloxone kit together during the simulation. Non-indicated actions that were not discussed during the experimental intervention were observed during 42 simulations (49.4%), primarily the performance of chest compressions. See Table 1 for details.

Post-hoc review of videos determined that resuscitative actions by 80 of 85 subjects (94.1%) were of potential overall benefit to the simulated opioid OD patient. The PABAK coefficient was 0.81 with inter-observer agreement of 90.6% for the two initial reviewers. Discrepancies for 8 subjects primarily revolved around whether and when during the scenario chest compressions were administered – these were resolved through review by a third investigator.

DISCUSSION

With widespread use of opioid pain relievers and a growing heroin crisis nationally, ODs are increasingly a major cause of morbidity and mortality.²³ Opioid ODs are on the rise in both incidence and case fatality despite the presence of an effective, accessible and commonly-recognized antidote in the form of naloxone. Institution of programs for specific populations to promote layperson administration of naloxone may help curb OD deaths associated with this epidemic. These efforts may be facilitated by patient simulation, which provides a safe and validated^24–27 mechanism for learners to demonstrate and be evaluated on their acquisition and application of knowledge, skills and poise for high-stakes situations. Accordingly, the study team used simulation to complement existing research tools for investigation into the meaningful performance of layperson opioid OD resuscitative actions in a high-risk cohort.

Even with significant environmental constraints on the program intervention, more than half of the study participants were observed to correctly deliver resuscitative doses of IN naloxone with timeliness comparable to paramedic students.²⁸ Concurrent with its primary evaluative purpose within the research program, study simulations were noted to enable subjects to act out what the experimental intervention had presented and to practice in effect for high-acuity events that are highly probable in their immediate future.²⁹ Study conduct also highlighted the potential access that applied simulation could afford when trying to reach and help at-/high-risk populations. Conversely, subjects’ difficulties with IN naloxone delivery device assembly and the frequent observation of impromptu and unrehearsed chest compressions revealed the complexity surrounding intervention efforts for this cohort. The limited and brief nature of opportunities available to train these individuals is expected to forestall traditional, resource-delimited methods such as structured Basic Life Support courses. Additionally, trepidation associated with accessing public safety services, i.e., calling 9-1-1, in the context of illicit drug-related activities was reported frequently by subjects and represents a prominent impediment to optimal resuscitative care – expansion of Good Samaritan overdose immunity laws may ameliorate this problem.³⁰ Next steps for community IN naloxone research efforts will need to address these unique operational challenges while ensuring correct device assembly (or alternative delivery mechanisms^31,32), appropriate naloxone dose, proper administration technique and the avoidance of non-indicated actions.

Study conduct demonstrated that simulation can be applied to outreach efforts directed towards inmate target populations housed in intrinsically limiting environments. With corroboration of the high frequency of opioid ODs in and around the study cohort over an extremely short follow-up period, the potential value of medical simulation techniques to assess (and potentially empower) those at risk appears significant. Study findings are being used nationally in ongoing programs to promote community naloxone administration for opioid ODs – continued application of the study simulation protocol is being considered for further investigation and standardized assessment of OD recognition and response in prison and community-based cohorts.

Limitations

Unforeseen factors related to subject recruitment and attrition may have impacted the study sample selection. As simulation could not be made available to subjects inside the correctional setting, subjects were not assessed for their acquisition and retention of opioid OD training immediately following the intervention. Records from public safety and emergency medical services were not available for subjects’ self-reported experiences with overdoses and administrations of naloxone. Pre-simulation assessments at follow-up may have impacted subject performance as recorded by scenario checklists, although the significant variability in checklist scores does not indicate a uniform effect. Study design and target population characteristics precluded follow-up beyond the research program’s one-time scheduled session.

CONCLUSIONS

Investigators applied simulation to the post-release assessment of opioid OD prevention training retention and skills acquisition in at-risk prison inmates. Findings from this exploratory study suggest the potential utility of simulation as a complementary evaluative methodology for investigation of layperson OD management skills.

Supplementary Material

Supplemental Data File _doc_ pdf_ etc._

Documents: Study scenario preparation/setup document; scripted orientation; simulation session protocol.