Evaluating rescuer performance in response to opioid overdose in a community setting: Evidence for medically appropriate process measures

Highlights

• •Content experts’ firsthand experiences of treating out-of-hospital opioid overdoses.
• •Process measures for treating acute respiratory depression and cardiac arrest.
• •Development of a rescuer evaluation tool for overdose education instructors.

Abstract

Overdose education and naloxone distribution (OEND) programs are widely accepted to reduce opioid overdose deaths. However, there is currently no validated instrument to evaluate the skills of learners completing these programs. Such an instrument could provide feedback to OEND instructors and allow researchers to compare different educational curricula. The aim of this study was to identify medically appropriate process measures with which to populate a simulation-based evaluation tool. Researchers conducted interviews with 17 content experts, including healthcare providers and OEND instructors from south-central Appalachia, to collect detailed descriptions of the skills taught in OEND programs. Researchers used three cycles of open coding, thematic analysis, and consulted currently available medical guidelines to identify thematic occurrences in qualitative data. There was consensus among content experts that the appropriate nature and sequence of potentially lifesaving actions during an opioid overdose is dependent on clinical presentation. Isolated respiratory depression requires a distinct response compared to opioid-associated cardiac arrest. To accommodate these different clinical presentations, raters populated an evaluation instrument with the detailed descriptions of overdose response skills, such as naloxone administration, rescue breathing, and chest compressions. Detailed descriptions of skills are essential to the development of an accurate and reliable scoring instrument. Furthermore, evaluation instruments, such as the one developed from this study, require a comprehensive validity argument. In future work, the authors will integrate the evaluation instrument in high-fidelity simulations, which are safe and controlled environments to study trainees’ application of hands-on skills, and conduct formative assessments.

1. Introduction

From 2020 to 2021, over 100,000 overdose deaths occurred in the United States, more than 70 % attributed to opioids (Ahmad et al., 2022). This increase in opioid overdose mortality is linked to illicit drug supply contamination and the negative effects of COVID-19-related social isolation on mental health (Ahmad et al., 2022, Linas et al., 2021). The distribution of naloxone, an effective opioid-reversal medication, to people likely to witness an opioid overdose is a critical mitigation strategy (Carroll et al., 2018). Naloxone is available in the United States at health centers (Wheeler et al., 2015), syringe services programs (Lambdin et al., 2020), and pharmacies (Morton et al., 2017).

Duration and content of overdose education and naloxone distribution (OEND) programs vary considerably (Edwards et al., 2020, Kerensky and Walley, 2017, Lewis et al., 2017, McDonald et al., 2017). Moustaqim-Barrette et al. (2021) identified OEND training parameters and strategies as a significant gap in the literature. Pellegrino et al. (2021) highlighted the need for “a validated tool(s) that works across populations to report educational outcomes” for use in future study designs and meta-analyses.

Currently, there are four validated assessment instruments: the Opioid Overdose Knowledge Scale (Williams et al., 2013), Opioid Overdose Attitude Scale (Williams et al., 2013), Brief Overdose Recognition and Response Assessment (Jones et al., 2014), and the Brief Opioid Overdose Knowledge Questionnaire (Dunn et al., 2016). While helpful, questionnaires may only reflect formal knowledge without directly measuring hands-on skills (Edwards et al., 2020). Furthermore, these instruments have limitations affecting their applicability across populations (ibid.) A detailed instrument with defined process measures (i.e., quantifiable steps for completing a skill) for opioid overdose response would provide a standardized tool for OEND program evaluation and research.

Simulation provides a controlled environment in which to evaluate overdose response, minimizing risk while providing a realistic experience (Wayne and McGaghie, 2010, Cook and Hatala, 2016, Orkin et al., 2021). Prior to this research, nine studies have evaluated learners in low to high-fidelity simulations of opioid overdose and attempted to measure administration of naloxone, ventilations, and response times (Edwards et al., 2020). In one exploratory study, Kobayashi et al. (2017) developed a 21-item checklist modified from the American Heart Association’s (AHA) medical guidelines and evaluated response-skills among inmates likely to witness an opioid overdose. This highlights a methodological gap previous researchers encountered: lack of a validated instrument to measure overdose response skills.

The aim of this study was to apply a rigorous process to the development of detailed steps a rescuer (i.e., someone first on scene) should take in response to an opioid overdose in the community. The outcome is a nuanced evaluation instrument specific to opioid overdose and designed to be applied in standardized simulations, including victims with isolated respiratory depression and those in cardiac arrest.

2. Materials and methods

2.1. Study site and setting

Researchers completed this study at a medical simulation laboratory in south-central Appalachia. In 2019, the overdose mortality rate was 53.5 deaths per 100,000 people in this region (National Opinion Research Center at the University of Chicago, 2018). The appropriate institutional review board approved this study.

2.2. Recruitment and eligibility criteria

Recruitment took place (January 2020-June 2020) using flyers, email, and word of mouth. Due to COVID-19 restrictions, researchers conducted interviews virtually (March 2020–June 2020).

Emergency Department (ED) physicians, first responders, OEND instructors, and peer recovery specialists were eligible if active in their role, trained in naloxone administration, and had treated at least one overdose. This low threshold helped recruit peer recovery specialists who have personal experience with addiction or mental health disorders. Additional criteria for OEND instructors included experience with training at least five people. At most, five participants were recruited for each category.

2.3. Redcap screening

Potential participants self-administered a REDCap screening questionnaire (Harris et al., 2009, Harris et al., 2019).

2.4. Structured interviews

Researchers developed an interview protocol with 17 to 20 questions, each with scripted sub-questions, and an outline of relevant rescue interventions published in medical guidelines (World Health Organization (WHO), 2014, Lavonas et al., 2015, Stancliff et al., 2016, Substance Abuse and Mental Health Services Administration (SAMHSA), 2018). Additional probing occurred in all interviews (Britten, 1995). Audio recordings were transcribed verbatim (a total of 17 transcripts) (https://www.thelai.com/). Participants received a $50 gift card.

Researchers asked participants, given their training and personal experience, to describe their ideal response to an opioid overdose. Participants were also prompted to explain the timing of interventions and the differences in approach to an overdosing person with and without a pulse.

2.5. Thematic coding

Using a constructivism paradigm, researchers conducted a qualitative analysis identifying thematic occurrences (Creswell & Guetterman, 2019). Using this framework, it is common to reach saturation in 6 to 20 interviews (Galvin, 2015, Guest et al., 2020). Researchers examined saturation after 6 and 15 interviews. Trained coders (FE and AS) used NVivo12 (March 2020) to complete three cycles of independent open coding with thematic analysis and consensus coding (Saldaña, 2013, QSR International Pty Ltd, 2020, Jackson and Bazeley, 2019). Each cycle of coding generated primary and secondary subthemes. FE and AS coded independently and discussed their findings at completion of the first transcript, the half-way point, and after all the transcripts, refining themes according to consensus and additional splitting. Researchers excluded process measures an untrained bystander could not perform without the appropriate training and medical equipment.

2.6. Identification of process measures

Inclusion of process measures was strongly supported by the high percentage of thematic occurrence. The list was further organized according to medical guidelines published in the last decade by the AHA, Substance Abuse and Mental Health Services Administration (SAMHSA), and the World Health Organization (WHO) (World Health Organization (WHO), 2014, Lavonas et al., 2015, Stancliff et al., 2016, Substance Abuse and Mental Health Services Administration (SAMHSA), 2018). Researchers compared thematic occurrences with published medical guidelines. Thematic occurrences in < 3 transcripts (< 15 %) were only included if supported by published medical guidelines. Lastly, thematic occurrences above 15 % but difficult to measure in simulation were excluded. See Additional file 1 for a complete list of thematic occurrences, and Additional files 2 and 3 for a detailed list of the consensus coding outcomes.

ED physicians and first responders described their current role (with their current training), not the ideal response of an untrained bystander in the community. This explains why some items, including ‘alerting EMS’, ‘staying with the person until help arrives’, and ‘not run away if police lights and sirens suddenly appear’, were not discussed by healthcare providers since they assumed themselves as EMS in their narratives. On the other hand, some items, including ‘intravenous naloxone administration’, ‘intraosseous naloxone administration’, and ‘breathing management with a bag-valve-mask’, were not discussed by non-healthcare providers since they assumed themselves as the layperson in their narratives. Therefore, some thematic occurrences (Table 2) have a denominator consistent with the type of content experts who discussed them.

Table 2.

Percentage of transcripts and references of adequate and inadequate performance of rescue maneuvers as described by content experts.

Adequate Performance	# of transcripts	% of transcripts	Expert	Inadequate Performance	# of transcripts	% of transcripts	Expert
Alert Emergency Medical Services (EMS)*
1.Alert EMS at the start of resuscitation	4 / 7	57.14 %	OEND PRS	1.Do not alert EMS	3 / 7	42.86 %	OEND PRS
2.Communicate essential information to operator	4	57.14 %	OEND
3.Request bystander to alert EMS (if unable to)	1	14.29 %	OEND
Assess safety of the scene
1.Check surroundings and person for syringes and drug paraphernalia	12 / 17	70.59 %	All	1.Do not recognize threats to safety	6 / 17	35.29 %	ED FR OEND
2.Aware of other bystanders	9	52.94 %	ED FR OEND	2.Do not check person for syringes and drug paraphernalia	4	23.53 %	FR OEND
3.Don personal protective equipment	4	23.53 %	ED FR OEND
4.Check person for weapons	4	23.53 %	ED FR
5.Reassess safety	2	11.76 %	ED FR
Visual assessment of the person
1.Check the person's skin	16 / 17	94.12 %	All	1.Do not recognize signs of an overdose	4 / 17	23.53 %	ED FR OEND
a.Change in skin pallor	13	76.47 %	All
b.Injection lesions	5	29.41 %	ED FR OEND
c.Temperature	3	17.65 %	ED FR OEND
d.Diaphoretic	3	17.65 %	FR OEND
2.Check the person's pupillary response	7	41.18 %	ED FR PRS
3.Check the person's body position	4	23.53 %	ED FR OEND
4.Check the person's clothing	3	17.65 %	ED FR OEND
Assess the person’s responsiveness
1.Check if person responds to painful stimuli	13 / 17	76.47 %	ED FR OEND	1.Do not check person’s responsiveness	4 / 17	23.53 %	ED OEND
a.Sternal rub	12	70.59 %	ED FR OEND	2.Do not use painful stimuli	4	23.53 %	ED FR
b.Press pen into person’s nail bed	4	23.53 %	ED FR
2.Check if person responds to verbal stimuli	8	47.06 %	All
3.Shake person	7	41.18 %	All
Check airway for obstruction
1.Look in the person’s mouth	11 / 17	64.71 %	ED FR OEND	1.Do not check the airway for obstruction	7 / 17	41.18 %	ED FR OEND
2.Reposition the person’s head	6	35.29 %	ED FR OEND	2.Leave the person's head tilted down	4	23.53 %	ED OEND
a.Jaw thrust	7	41.18 %	ED FR OEND	3.Blind finger sweep	3	17.65 %	FR OEND
Airway management
1.Reposition the person’s head	12 / 17	70.59 %	ED FR OEND
a.Jaw thrust	6	35.29 %	ED FR
b.Head tilt chin lift	6	35.29 %	ED FR OEND
2.Place person on back	3	17.65 %	ED OEND
3.Manage within first 30 s	3	17.65 %	ED FR
Assess breathing
1.Examine the way the person is breathing	14 / 17	82.35 %	All	1.Do not examine how the person is breathing	6 / 17	35.29 %	ED FR OEND
2.Check for chest rise and fall	13	76.47 %	All	2.Do not check respiratory rate	6	35.29 %	ED FR OEND
3.Check respiratory rate	8	47.06 %	ED FR OEND	3.Do not check for respirations	5	29.41 %	ED FR OEND PRS
4.Check respirations within 10 to 15 s	6	35.29 %	ED FR
5.Listen for respirations	3	17.65 %	OEND PRS
6.Check respirations throughout resuscitation	1	5.88 %	OEND
Assess heart rate or pulse
1.Check carotid pulse	13 / 17	76.47 %	All	1.Do not check heart rate or pulse	5 / 17	29.41 %	ED OEND PRS
2.Check radial pulse	8	47.06 %	ED FR OEND	2.Check weak pulse locations	3	17.65 %	ED OEND
3.Check pulse quality	6	35.29 %	ED FR OEND
4.Check for 10 s	6	35.29 %	All
5.Check femoral pulse	3	17.65 %	ED FR
6.Check different locations for pulse	3	17.65 %	FR OEND
7.Check pulse throughout resuscitation	2	11.76 %	OEND
8.Check pulse rate	2	11.76 %	FR OEND
Administer Narcan®
1.Insert Narcan tip into nose	5 / 17	29.41 %	FR OEND	1.Push plunger before insertion	4 / 17	23.53 %	OEND
2.Tilt the device perpendicular towards person's face	4	23.53 %	ED FR OEND	2.Do not insert the Narcan tip deep enough	3	17.65 %	FR OEND PRS
3.Place person on their back before naloxone administration	3	17.65 %	OEND PRS	3.Do not utilize the full dose	3	17.65 %	OEND PRS
4.Check the nose for obstruction	3	17.65 %	ED FR OEND
5.Administer within 5 min	2	11.76 %	ED
6.Administer the full dose [4 mg] into one nostril	2	11.76 %	FR PRS
Provide rescue breaths or other respiratory aid
1.Position person's head	5 / 17	29.41 %	ED FR OEND	1.Give forceful rescue breaths	5 / 17	29.41 %	ED FR OEND
2.Give rescue breaths immediately and before naloxone administration	5	29.41 %	ED FR	2.Give too rapid rescue breaths	2	11.76 %	OEND
3.Give 1 breath every 5 to 6 s	5	29.41 %	ED FR OEND
4.Rescue breaths make the person's chest rise	3	17.65 %	OEND
5.Continue rescue breaths after naloxone administration	2	11.76 %	OEND
6.Pinch person's nose closed	1	5.88 %	OEND
Repeat naloxone administration
1.Wait for 1 to 5 min before administration of the 2nd dose of naloxone	8 / 17	47.06 %	All
2.Observe person after administration of the 1st dose of naloxone	4	23.53 %	ED FR
Utilize recovery position
1.Roll the person on their side, bend their knee forward, place their arm under their head	7 / 17	41.18 %	FR OEND PRS	1.Do not place the person in the correct recovery position	5 / 17	29.41 %	FR OEND
2.Roll the person on their side if they vomit	4	23.53 %	ED FR OEND	2.Leave the person supine	3	17.65 %	OEND
3.Maintain a patent airway	4	23.53 %	FR OEND PRS
4.Roll the person on their side after naloxone administration	3	17.65 %	FR OEND
Staying with the person until help arrives*
1.Stay with the person	3 / 7	42.86 %	OEND PRS
2.Communicate effectively with EMS	2	28.57 %	OEND PRS
3.Continue to monitor pulse until EMS arrives	1	14.29 %	OEND
Provide chest compressions
1.Start compressions in 1 min (if pulseless)	5 / 17	29.41 %	ED FR OEND	1.Do not compress [2 to 2.4 in.] deep	6 / 17	35.29 %	FR OEND PRS
2.Compress chest [2 to 2.4 in.] deep	4	23.53 %	ED FR	2.Compress too fast or too slow	5	29.41 %	ED FR OEND
3.100 to 120 compressions per minute	3	17.65 %	ED FR	3.Do not place hands on person’s sternum	3	17.65 %	ED OEND
4.Compress mid-line over the sternum	2	11.76 %	ED FR	4.Leave the person on a soft surface	2	11.76 %	FR OEND
5.Chest recoils after each compression	2	11.76 %	ED FR
6.30 compressions and 2 rescue breaths (100 to 120 compressions per minute)	2	11.76 %	ED FR
7.Place person on a hard surface	1	5.88 %	ED
8.Keep wrists, shoulders, and elbows in a straight line over the person	1	5.88 %	ED
9.Do not stop compressions greater than 10 s	1	5.88 %	FR
10.Place one hand over the other when providing compressions	1	5.88 %	ED
Defibrillation
1.Place one pad on right side of person’s chest and one pad on lower left side of person’s chest	5 / 17	29.41 %	ED FR	1.Do not place one pad on right side of person’s chest and one pad on lower left side of person’s chest	4 / 17	23.53 %	ED FR
2.Apply shock if there is a shockable rhythm	4	23.53 %	ED FR
3.Defibrillate in < 2 min (if needed)	3	17.65 %	ED
4.Resume chest compressions immediately after shock or no-shock	3	17.65 %	ED FR

ED, emergency department; FR, first responder; OEND, OEND instructor; PRS, peer recovery specialist.

*Only non-healthcare providers (including OEND instructors and peer recovery specialists) discussed ‘alerting EMS’, ‘staying with the person until help arrives’, and ‘not run away if police lights and sirens suddenly appear’ because healthcare providers considered themselves EMS (i.e., ‘the help’) in their descriptions of responding to an opioid overdose.

2.7. Results

As part of a larger study of the same sample of participants, 50 % were female, 100 % were white and non-Hispanic, and the median age was 38 years (Fig. 1 and Table 1). First responders were active in their role the longest (median 17 years), but ED physicians had treated the most overdoses (median 350). See Additional file 4 for further sample demographics.

Fig. 1.

Recruitment and eligibility criteria flowchart.

Table 1.

Demographics.

Demographics	Total (n)	Percentage or median (IQR)
Gender
Female	8	50 %
Male	8	50 %
Ethnicity
Not Hispanic or Latino or Spanish Origin	16	100 %
Race
White	16	100 %
Age		38 (11)
Years of experience		8.5 (10)
ED physician	4	11 (3)
First responder	5	17 (7.5)
OEND instructor	5	2.5 (1.25)
Peer recovery specialist	2	2.5 (1.5)

IQR; interquartile range.

One participant did not complete the demographic or post-interview questionnaire.

Researchers collected and transcribed over 20 h of audio recording. Interviews averaged 71 min (range: 56–88). See Additional file 2 for a detailed listing of thematic occurrences including exclusions.

2.8. Themes and subthemes (Table 2)

2.8.1. Alert EMS

Non-healthcare providers will alert EMS at the beginning of their resuscitation (57 % of non-healthcare provider transcripts) and describe the clinical presentation (57 %). Furthermore, 43 % of non-healthcare providers mentioned not alerting EMS could jeopardize the health of the overdose victim. An OEND instructor highlighted reasons they alert EMS, stating:

Participant (OEND instructor): …if the patient either doesn’t improve or the Narcan doesn’t work or if the person becomes agitated or violent after using it, at least they have help on the way.

2.8.2. Assess safety of the scene

Fifty-three percent of participants mentioned awareness of other bystanders on scene. If available, 24 % will don personal protective equipment. Seventy-one percent will check the surrounding scene for syringes or other drug paraphernalia, and 24 % will check for weapons. Twelve percent will reassess the safety of the scene. Inadequate performance includes not recognizing threats to scene safety (35 %) and not checking the person’s surroundings or pockets for drug paraphernalia (24 %). An OEND instructor provided an overview of the possible threats to them, stating:

Participant (OEND instructor): I would be looking for potential harm such as open needles…Then obviously people like to crowd around when there’s something happening. You just wanna make sure that you are able to control that by saying like, “Stand back. Give us room,”….

2.8.3. Visual assessment of the person

Participants said they examine the person’s skin pallor (76 %) and look for injection lesions (29 %). Forty-one percent of participants look for a pupillary response, check body position (24 %), and clothing (18 %). According to participants, not evaluating a person’s physical appearance could interfere with their treatment (24 %). An ED physician provided insight on where to look for injection lesions, stating:

Participant (ED physician): …the first obvious injection site is in the AC [antecubital fossa]. A lot of people, after they’ve burned out those vessels, they’ll go to the inner arm, just beside their bicep. Then, looking at their wrists…the dorsum of the hand and in between the finger webs…and then the feet and in between the toes.

2.8.4. Assess the person’s responsiveness

Participants will evaluate responsiveness by checking for a response to a verbal stimulus (47 %), physical stimulus (41 %), and painful stimulus (76 %), such as a sternal rub (71 %). According to participants, inadequate performance includes using only one of these methods (24 %) and using overly-gentle stimuli (24 %). A first responder described a sternal rub, stating:

Participant (First responder): They’re usually on the ground, so you’re just pressing…hard and you just rub [on the sternum].

2.9. Check airway for obstruction

Participants will look in a person’s mouth for an airway obstruction but avoid inserting fingers to remove an obstruction (65 %). Additionally, participants will reposition the person’s head (35 %) with an airway maneuver, such as a jaw thrust (41 %). Inadequate performance includes not checking the airway for obstruction (41 %), leaving the head tilted down (24 %), and performing a blind finger sweep (18 %). An ED physician described checking the airway, stating:

Participant (ED physician): I just look in their mouth…If there’s no trauma up there and there’s no visible swelling and there’s nothing in their mouth, then I assume it’s probably clear.

2.10. Airway management

Participants will manage the airway within the first 30 s of arriving on scene (18 %), place the person supine (18 %), and use an effective airway maneuver to reposition the head (71 %), such as performing a head-tilt-chin-lift (35 %) or jaw-thrust (35 %). An OEND instructor described the steps they take to manage the airway, stating:

Participant (OEND instructor): I would say…positioning is the most critical thing, just making sure that the airway and breathing isn’t obstructed by positioning…early on in it, it’s just gonna be a head-tilt-chin-lift.

2.11. Assess breathing

Participants will assess the person’s breathing within 10 to 15 s of arriving on scene (35 %), listen for respirations (18 %), identify chest rise and fall (76 %), check the quality of the person’s breathing (82 %), and evaluate the person’s respiratory rate (47 %). One OEND instructor mentioned checking respirations throughout resuscitation (6 %). According to participants, inadequate performance includes not evaluating breathing quality (35 %), respiratory rate (35 %), or simply not checking or listening for respirations (29 %). An ED physician described evaluating respirations, stating:

Participant (ED physician): I would just look and see if they have any spontaneous respiration … Then take note whether they’re doing the shallow breathing…which would be typical of opiate overdose….

2.12. Assess heart rate or pulse

Participants mentioned locations to check for a pulse including the carotid (76 %), radial (47 %), and femoral (18 %). A few participants will check multiple locations for a pulse (18 %) and spend up to 10 to 15 s (35 %). Additionally, participants will assess the quality (35 %) and pulse rate (12 %). Two OEND instructors mentioned checking the pulse throughout their resuscitation (12 %). Inadequate performance includes not checking the heart rate or pulse (29 %) and only checking weak pulse locations (18 %). A first responder described the challenge of accurately assessing a pulse, stating:

Participant (First responder): …With the overdoses, they’re usually a lot slower, so you do have to wait a lot longer when you’re assessing the pulse. Usually, I try the 15 s rule. I try to wait 15 [seconds], or if it’s really going, I’ll do it for 10 [seconds], but with overdoses, sometimes you have to hold it the full minute….

2.13. Naloxone administration

Participants were asked to describe process measures for naloxone administration with routes familiar to them (i.e., intramuscular, intranasal, intravenous, etc.). For clarity, the process measures for Narcan® Nasal Spray (ADAPT Pharma Inc.) are outlined, as described by participants. Using Narcan® Nasal Spray includes placing the person supine with their head tilted back (18 %), checking their nose for obstruction (18 %), tilting the device perpendicular to the face (24 %), inserting the Narcan® tip into the nose (29 %), and administering the full dose into one nostril (12 %) within 5 min of arriving (12 %). Inappropriate technique includes pressing the plunger outside of the nostril (24 %), not inserting the Narcan® tip deep enough (18 %) or using the full dose (18 %). Many participants referenced Narcan® Nasal Spray as easy-to-use. A first responder clearly described how to use the nasal spray, stating:

Participant (First responder): …making sure that you’ve actually got it [nasal spray] in their nose…make sure that it’s deep….

2.14. Provide rescue breaths or other respiratory aid

Participants will adjust the position of the person’s head (29 %), pinch the nose to create an adequate seal (6 %), provide rescue breaths every 5 to 6 s (30 %), make the chest rise (18 %), and provide rescue breaths prior to naloxone administration (29 %) and immediately after (12 %). According to participants, inadequate performance includes providing too frequent rescue breaths (greater than 1 breath every 5 to 6 s) (12 %) or insufflating the person’s stomach (29 %). An OEND instructor described rescue breathing, stating:

Participant (OEND instructor): …you’ve positioned the person’s head properly to open up the airway. Look to see if there’s any obstructions…seeing the chest rise and fall….

2.15. Repeat naloxone administration

Participants will observe the person after administering the first dose of naloxone (24 %) and wait for 1 to 5 min before administering a second dose (47 %).

2.16. Utilize recovery position

Participants utilize the recovery position (41 %) if the person is ventilating independently immediately after naloxone administration (18 %) or if the person vomits (24 %) while maintaining a patent airway (24 %). Inadequate performance includes positioning the person another way (29 %) or in a position vulnerable to aspiration (18 %). A first responder described the recovery position, stating:

Participant (First responder): …the knee bent forward holding up the bottom half of their body and having the arm across their chest hold up the top of their body….

2.17. Reassess until help arrives

Non-healthcare providers mentioned staying with the person that overdosed until help arrives (43 %) and communicating effectively with the first responders (29 %). An OEND instructor described the importance:

Participant (OEND instructor): You stay with that person…They might be mad, they might be cussin’ you, but you don’t care. You stay there.

2.18. Provide chest compressions

In the event of cardiac arrest, participants will place the person on a hard surface for effective compressions (6 % of all transcripts), start compressions within one minute (29 %), and compress 2 to 2.4 in. deep (some stated up to 2.5 in. deep) (24 %) midline over the sternum (12 %). Participants will let the chest fully recoil after each compression (12 %), provide 100–120 compressions per minute (18 %) or 30 compressions with 2 rescue breaths (12 %). One healthcare provider mentioned they will place one hand over the other (6 %), and keep their wrists, shoulders, and elbows in a straight line, and not stop compressions for more than 10 s (6 %). Overall, participants mentioned they would not leave the person on a soft surface (12 %), place their hands anywhere but the sternum (18 %), compress outside of the 2 to 2.4 in. range (35 %), and compress too fast or slow (29 %). An ED physician described chest compressions, stating:

Participant (ED physician): Chest compressions are mid-line over the sternum about…One hand over the other…wrists, shoulders, and elbows should all be in a line straight over the patient…compressing down on the chest to about two inches at a rate of 100 to 120 times a minute…make sure they have full recoil….

2.19. Defibrillation

If defibrillation is needed, participants will place defibrillator pads in the position specified by the Automatic Electronic Defibrillator (AED)’s instructions (29 %), defibrillate within 2 min (18 %), and resume compressions after a shock or no-shock decision (18 %). Prior to defibrillation, participants will assess for a shockable rhythm (24 %).

2.20. Development of the rescuer evaluation instrument

From the interviews, raters identified medically appropriate process measures suitable to populate an evaluation instrument for opioid overdose response (see Additional file 2). Researchers created two separable tools for isolated respiratory depression and opioid-associated cardiac arrest in the piloting phase. Currently, the instrument has 60 process measures: 45 measures for overdose with isolated respiratory depression and 15 measures for an opioid-associated cardiac arrest (see Additional file 3, column 3).

3. Discussion

Researchers identified process measures that are likely measurable in simulation and medically appropriate during an overdose. The content experts provided valuable perspectives and experiences treating opioid overdoses. The sample of ED physicians and first responders witnessed and treated the greatest number of overdoses, and shared knowledge from formal medical education. Moreover, at least four participants from the sample of OEND instructors and peer recovery specialists self-disclosed a personal history of substance use and provided diverse perspectives from their lived opioid use experience.

The process measures identified in this study do not differ from current medical guidelines. Rather the goal is to provide a behavioral marker system that can enable implementation of guidelines into hands-on training. For example, the guidelines suggest ‘assess breathing’ which is accurate and would include several sub-steps; ‘check chest rise and fall’, ‘check respiratory quality’, and ‘check respiratory rate’. A behavioral marker system can also provide time indicators which are related, in this case, to quality of response. The inclusion of these detailed behavioral markers and timings enables a trainer to accurately measure trainees’ skills in simulation. In this study, the researchers’ goal was to create a usable scale from several perspectives, all of which represent individuals engaged in overdose response, including physicians, first responders, OEND instructors, and peer recovery specialists.

While referring to published medical guidelines, raters identified some noteworthy differences. For example, SAMHSA’s 2018 guidelines recommend rescuers look for: “...fingernails or lips turning blue/purple,” but no published guidelines recommend identifying injection lesions or checking for constricted pupils. This is interesting because experts in this study often cited checking the person’s skin and pupils as an important step in their response. Furthermore, SAMHSA and WHO recommend lay rescuers check and clear the airway; however, AHA stopped recommending this technique to lay rescuers in 2000 (American Heart Association, 2000). The experts in this study often said it was important to check the airway for an obstruction. AHA does not recommend untrained rescuers evaluate a pulse because this group cannot reliably establish the presence of a pulse. This differed greatly from 76 % of the experts’ recommendation in this study, which is to check the person’s pulse for at most 10 s. Lastly, AHA does not recommend untrained rescuers provide rescue breaths because it is a difficult skill to retain. However, SAMHSA and WHO do recommend rescue breathing for all rescuers regardless of training status. Less than half of the experts in this study recommended rescue breaths (30 %), but the authors believe it is important to include in an evaluation instrument until additional evidence is available in this population due to the hypoxic etiology of the cardiac arrest that occurs during opioid overdose. In summation, this study identified several outstanding research questions regarding the application and timing of skills in an opioid overdose which would benefit from further study.

This study is subject to several limitations. Overall, the sample of participants is small, ethnically homogenous, drawn from southwest Virginia, and the results are limited by the experience and expertise of the participants (excluding ED physicians and first responders). Generalizability to presentation in other localities, larger metropolitan areas, and other countries may be limited. For example, some participants described finding persons that overdosed on public transportation in larger metropolitan areas. As with any qualitative analysis, information may have been lost during the three cycles of coding. Furthermore, bias may have gone with the development of the process measures; however, researchers took steps to limit bias through consensus and additional splitting.

Researchers documented and made critical decisions during each step of designing the instrument. For example, during the development of each section of the evaluation instrument, raters examined each thematic occurrence for its reference percentage, its applicability to likely bystanders without access to advanced medical equipment, similarity with existing medical guidelines, and its potential in a simulated environment. See Additional files 2 and 3 for a detailed description of this process, and a summarization of the congruence between the results presented and available medical guidelines.

4. Conclusions

Researchers collected qualitative information and used coding analysis to distill process measures suitable for an out-of-hospital overdose emergency. This systematic approach generated substantive evidence for the development of two separable tools that could be used to measure trainees’ treatment of overdose victims with respiratory depression and those with opioid-associated cardiac arrest in simulation. The identified process measures provide nuance to the steps a rescuer should take while treating an opioid overdose, according to content experts. These process measures are also likely measurable in simulation, an environment that is safe and controlled. Furthermore, these process measures could assist with future meta-analyses of the transfer of learned and hands-on skills. Future work will include investigating the internal and external structural consistency of the evaluation instrument.

Funding

A Carilion Clinic and Fralin Biomedical Research Institute Center for Transformative Research on Health Behaviors pilot feasibility grant supported this work. The funding source had no such involvement in study design; in the collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit the article for publication. The Virginia Tech Open Access Subvention Fund supported publication of this work.

CRediT authorship contribution statement

G. Franklin Edwards III: Methodology, Formal analysis, Investigation, Resources, Data curation, Writing – original draft, Writing – review & editing, Visualization, Project administration. Cassandra Mierisch: Conceptualization, Methodology, Validation, Investigation, Writing – original draft, Writing – review & editing, Visualization, Supervision, Funding acquisition. Allison Strauss: Formal analysis, Data curation, Writing – original draft, Writing – review & editing, Visualization. Brock Mutcheson: Conceptualization, Methodology, Validation, Writing – original draft, Writing – review & editing, Visualization, Supervision, Funding acquisition. Keel Coleman: Validation, Writing – original draft, Writing – review & editing, Supervision. Kimberly Horn: Conceptualization, Methodology, Validation, Writing – original draft, Writing – review & editing, Supervision, Funding acquisition. Sarah Henrickson Parker: Conceptualization, Methodology, Validation, Investigation, Resources, Writing – original draft, Writing – review & editing, Visualization, Supervision, Funding acquisition.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Appendix A. Supplementary data

The following are the Supplementary data to this article:

Data availability

I have shared additional files with the complete dataset.