Evaluation of an interprofessional naloxone didactic and skills session with medical residents and physician assistant learners

Abstract

Background:

The CDC has reported 399,230 opioid-related deaths from 1999-2017. In 2018, the US surgeon general issued a public health advisory, advising all Americans to carry naloxone. Studies show that enhanced naloxone access directly reduces death from opioid overdose. Despite this, health care professional learners report low knowledge and confidence surrounding naloxone. Therefore, it becomes critical that medical education programs incorporate didactic and experiential sessions improving knowledge, skills and attitudes regarding harm reduction through naloxone.

Objectives:

1. Describe the components and evaluation of a replicable and adaptable naloxone didactic and skills session model for medical providers; 2. Report the results of the evaluation from a pilot session with family medicine residents and physician assistant students; and 3. Share the session toolkit, including evaluation surveys and list of materials used.

Methods:

In July 2017, a literature search was completed for naloxone skill training examining best practices on instruction and evaluation. A training session for family medicine residents and physician assistant learners was designed and led by University of Cincinnati College of Medicine and College of Pharmacy faculty. The same faculty designed a pre and post session evaluation form through internal review on elements targeting naloxone knowledge, attitude, and self-efficacy.

Results:

The training session included one hour for a didactic and one hour for small group live skills demonstration in four methods of naloxone administration (syringe and ampule, nasal atomizer, branded nasal spray and auto injector). Forty-eight participants showed statistically significant (p<0.05) improvement in knowledge (67.5% to 95.9%), attitudes (71.2% to 91.2%), and self-efficacy (62.1% to 97.8%) from pre to post assessment. Forty-four of 48 participants agreed that the pace of the training was appropriate and that the information will be of use in their respective primary care practices. Supply costs for the session were USD 1,200, with the majority being reusable on subsequent trainings.

Conclusions:

Our study of a naloxone didactic and skills session for primary care trainees demonstrated significant improvements in knowledge, self-efficacy, and attitudes. It provides an adaptable and efficient model for delivery of knowledge and skills in naloxone administration training. The pilot data suggest that the training was efficacious.

INTRODUCTION

Despite legislative, medical, and educational efforts of the past decade, the opioid epidemic continues expanding its reach across America. According to data from the Centers for Disease Control and Prevention (CDC), there were 702,568 drug overdose deaths from 1999-2017 with 399,230 of those deaths related to opioids.1 From 2016-2017, the rate of drug overdose deaths increased by 9.6%.2 A large attributing factor to this increase is the rise of fentanyl co-ingestion and other highly potent synthetic opioids, especially in Cincinnati, Ohio which resides near the epicenter of the nation’s opioid crisis. The greater Cincinnati area has seen an increase in fentanyl related overdose deaths by 1,000% since 2013.3 Overall, Ohio ranks second in the country for highest rates of opioid related overdose deaths with studies reporting 3,613 deaths in 2016, which is more than double the nation’s average.4 For three consecutive months in 2016, Hamilton County, alone, experienced an average of over 16 ER visits and one death everyday due to overdose.5 The harm reduction effect of naloxone distribution has been demonstrated regionally, with the number of doses distributed increasing nearly eightfold from 2015 to 2018.6 This in part has contributed to a 21% reduction in the number of deaths due to opioids from 2017 to 2018.6

Given the persistence of opioid related morbidity and mortality and the demonstrated positive impact of harm reduction efforts in the community, never has there been a more crucial time to focus on distribution and training about the opioid reversal agent, naloxone. Naloxone has been recognized for more than two decades as a safe agent to reverse the effects of overdose through its mechanism of action as an opioid antagonist.7,8 The practice of co-prescribing naloxone to primary care patients with chronic pain is recommended by the CDC.9 In 2018, the US surgeon general issued the first public health advisory in a decade, advising all Americans to carry naloxone and learn how to use it.10 With no serious side effects known, naloxone has been distributed not only to first responders, including law enforcement, but also to community members as part of a larger harm reduction methodology. It has been shown to be effective in decreasing the mortality rate associated with opioid overdose by as much as 11%.11-13 When coupled with education on overdose recognition and response, the provision of naloxone to both those with opioid use disorder and their friends and family has been associated with substantial reductions in opioid overdose mortality at the community level.13 The regulations for prescribing naloxone vary from state to state. In 2015, a new law aimed at facilitating the distribution of naloxone in Ohio allowed for pharmacists to provide naloxone without a prescription.14 Thus, pharmacists now play an expanded role in direct patient/caregiver training in the administration of naloxone. Furthermore, state legislatures have also adopted myriad Good Samaritan Laws as a means to facilitate use of naloxone by shielding individuals from adverse legal consequences surrounding emergency aide, such as naloxone administration, for those suffering from an overdose.15 Therefore, as states direct policy measures to increase access to naloxone and remove potential barriers to its use, it becomes essential that all health care team members are available to address gaps in training and foster efficacy in the safe and proper administration of naloxone by patients and families.

Despite these enhanced legislative policies for increased access to naloxone, training programs have been slow to incorporate naloxone training into their respective curriculums. Physicians continue to report high levels of discomfort in prescribing naloxone, including the concern of enabling risky behavior and fear of liability for prescribing policies and protocols for both patients and family members.16,17 Many physicians describe having not used naloxone since their training program, citing a lack of self-efficacy in use as a barrier to furnishing it to patients.17 Additionally, clinicians have also reported doubts about properly identifying those patients at elevated risk for dependence.18,19 Medical training programs have found health care professional learners reporting limited knowledge and low confidence in educating patients on naloxone and overdose risks, as well as discomfort with the patchwork quilt of laws and policies surrounding naloxone administration across the country.16,20

While many factors might contribute to low rates of naloxone prescription practice, training programs are where the process of integration for knowledge, attitudes, and efficacy begins. It has been strongly recommended that graduate level medical education programs incorporate trained and experienced faculty, adequate didactic and experiential sessions, and clinical exposure to care for patients with opioid dependence in the effort to introduce engaging platforms to support desired behaviors and skills.21 Professional society guidance and primary care behavioral educators often recommend curricular domains in substance use disorders, including the risk assessment and provision of naloxone for treating overdose.22 However, the currently mandated accreditation governing body family medicine residency components from the Accreditation Council for Graduate Medical Education (ACGME) are unfortunately silent on any explicit required training on substance use as it pertains to patient care. Interestingly, the only time substance use disorder training is mentioned in the more than sixty pages of requirements is in terms of self-care and identifying use in peers and other providers.23 Opioids, naloxone, or other components in care and treatment for those with substance use are not mentioned explicitly in the required educational curricula. Thus, without expanding beyond the minimum required opioid objectives, recent primary care training program graduates experience inadequate preparation to meet the needs of the opioid epidemic. While some training programs that have integrated components of opioid use disorders and overdose treatment report increase in knowledge, efficacy, and attitudes for health care professional trainees, the current climate of practicing physicians’ low comfort and confidence in educating patients in and prescribing life-saving naloxone suggests that further programming is necessary.16,20,24-27

After critical observation of existing training within our health system, a gap was identified in meeting the needs of patients with a substance use disorder. The objectives of this paper are to: (1) Describe the components and evaluation of a replicable and adaptable naloxone didactic and skills session model for medical providers; (2) Report the results of the evaluation from a pilot session with family medicine residents and physician assistant students; and (3) Share the session toolkit, including evaluation surveys and list of materials used, to enable other medical training programs to implement a similar educational session.

METHODS

In response to the lack of patient and family member access to and leaderships’ desire to develop efficacious training for prescribing and administering of naloxone, The Christ Hospital/ University of Cincinnati Department of Family and Community Medicine Residency implemented a naloxone skills and education session for all of its first-year residents in Fall 2017. The session was developed and delivered with faculty and staff supported time from a Health Resources and Services Administration (HRSA) Primary Care Training Enhancement grant. An interprofessional team conducted a literature search for primary care naloxone training programs in July and August 2017. From a review of this literature, the core components of an intervention were developed to include both didactic and skills training. This same group through an internal iterative process developed an evaluation form that could be administered both pre and post training. The evaluation form was designed through consensus to include items targeting knowledge, attitudes, and self-efficacy in the administration of naloxone. The evaluation form was reviewed by practitioners with substance use disorder experience for face validity and iteratively modified based on their feedback.

The 2017 pilot program with 7 family medicine trainees was then expanded to include interprofessional learners by partnering with a physician assistant training program in Fall 2018. Both iterations of the session were delivered by the same faculty members and lasted two hours; one hour for the didactic, and one hour for live skills demonstration in four methods of naloxone administration (syringe and ampule, nasal atomizer, branded nasal spray, and auto injector). Prior to the beginning of each session, learners completed a pre-session evaluation form that included knowledge, attitude, and self-efficacy questions regarding comfort level in the prescription of naloxone, care of patients with opioid use disorder and local practices and policy in surrounding county municipalities (Online appendix).

The didactic portion was led by a family medicine faculty member with a dual appointment in psychiatry. Content included: (1) a description of the recent national and regional background statistics of the opioid epidemic, (2) the function of naloxone, (3) identifying an opioid overdose, (4) an in-depth analysis of the advantages, functionality, and practicality of four widely used formulations of naloxone administration (auto-injector, syringe and ampule, nasal atomizer, and branded nasal spray), (5) representative naloxone access laws and Good Samaritan Laws, and (6) recommended indications for providing naloxone. Also, pharmacy faculty members were able to provide detailed instruction on writing a prescription for naloxone in the different methods described, demonstrated, and practiced during the session as described in Figure 1.

Figure 1. Prescribing information.

The autoinjector and branded nasal spray have training units available from the respective pharmaceutical companies. These were requested and received in advance of the training. The syringe and ampule and nasal atomizer supplies were purchased from medical training supply companies and consisted of inert saline in otherwise medically accurate devices. Sufficient supplies were purchased for each learner to have their own training materials including CPR-style mannequin heads for each small group to provide an avenue to demonstrate naloxone administration. Additionally, oranges were purchased to provide a practice model for intramuscular administration of naloxone. Figure 2 details a complete list of the supplies acquired for the session.

Figure 2. Supplies for naloxone skills training session.

The skills portion of the session included the assembly and active demonstration of four modes of naloxone: syringe and ampule, auto-injector, branded nasal spray, and nasal atomizer. This experiential portion of the training was led by a College of Pharmacy faculty member and two fourth-year pharmacy students. Each primary care learner practiced with their own complete demo kit for each method of administration in a small group setting. Stepped instructions were provided on assembly and administration of each within each small group to facilitate individual experiential learning.

Immediately following the didactic and skills portions of the session, all participants completed a post-session evaluation that included the same knowledge, attitude and self-efficacy questions as the pre-session evaluation. The post-session survey included four additional indirect assessment questions regarding usefulness of the information, effectiveness of the instructors, clarity of the session objectives, and pace of the training. The seven knowledge questions were multiple choice, with the remaining five statements on attitudes, four statements on efficacy, and six statements evaluating the session content and instructors having a four-point Likert scale response key (1=Strongly Agree to 4=Strongly Disagree). Descriptive and bivariate analysis was performed using IBM SPSS Version 25. These sessions were approved by the University of Cincinnati IRB under a larger primary care training enhancement training grant.

RESULTS

The naloxone didactic and skills session described occurred two times: The first was in October 2017 with seven family medicine intern participants who were required to attend the session during a behavioral science month rotation. The second session occurred in October 2018 and again included seven family medicine interns and 34 physician assistant learners.

Thus, a total population of 48 learners participated between the two training sessions. For the skills portion of the training, four methods of administration were initially planned. However, the October 2017 session included only the nasal atomizer, auto-injector, and syringe and ampule due to the branded nasal spray demo training devices being out of stock. Therefore, existing third party video was used to demonstrate appropriate use of the branded nasal spray device. All four methods were available for use during the October 2018 session. Faculty effort included time for two family medicine faculty members, one pharmacy faculty member, a research assistant, and a social scientist which amounted to just over USD 26,000. Material costs, including the demonstration devices, for both sessions totaled roughly USD 1,200.

Table 1 shows results of pre- and post-session evaluation knowledge questions. Table 2 displays results from the attitude and efficacy questions. Scores on all three domains improved from pre- to post-session assessment. Participants showed statistically significant (p<0.05) improvement from pre-knowledge evaluation to post assessment, with an average score of 5.6 correct answers on the seven knowledge questions. Statistically significant (p<0.05) differences were also measured in attitudes from pre-session evaluation to post-assessment. The greatest amount of attitude change was seen in responses to the statements, “I’m comfortable prescribing naloxone to a patient with opioid addiction” (p<0.001); and, “I’m comfortable prescribing naloxone to a family member of a patient with opioid addiction” (p<0.001). All participants after the session agreed that they felt comfortable prescribing naloxone to the patient, with only one participant disagreeing with feeling comfortable prescribing naloxone directly to family members of a patient under their care. The average percentage of those responding “agree” or “strongly agree” that they felt comfortable administering the four demonstrated naloxone devices increased from 53% to 97% (p<0.05).

Table 1. Results from knowledge questions.

Questions	Participants answered correctly/n respondents (%)	Participants answered correctly/n respondents (%)	p-value
	Pre	Post
Which of the following is a sign of opioid intoxication?	41/45 (91%)	46/46 (100%)	0.125
What is the mechanism of action of naloxone?	37/44 (84%)	42/45 (93%)	0.372
True/False. Narcan can precipitate acute opioid withdrawal.	36/45 (80%)	41/42 (98%)	0.008
Which of the following symptoms is generally NOT seen in opioid withdrawal?	24/45 (53%)	42/46 (91%)	<0.001
In Ohio, Narcan is available by prescription via standing order, which means that…(correct response) a patient without a prescription can buy Narcan at a pharmacy.	14/45 (31%)	42/46 (91%)	<0.001
True/False. The naloxone nasal atomizer is more expensive and convenient than the EVZIO device.	16/45 (36%)	46/46 (100%)	<0.001
True/False. The state of Ohio has a Good Samaritan Law that will protect a person who seeks out help for someone who is experiencing a drug overdose.	44/45 (98%)	45/46 (98%)	1.00

Table 2. Results from the self-efficacy and attitudes questions.

	Pre		Post		p-values
	Participants responding Strongly Disagree & Disagree/n respondents (%)	Participants responding Agree & Strongly Agree/n respondents (%)	Participants responding Strongly Disagree & Disagree/n respondents (%)	Participants responding Agree & Strongly Agree/n respondents (%)
I can recognize signs and symptoms of opioid overdose.	5/45 (11%)	40/45 (89%)	0	46/46 (100%)	<0.001
I would feel comfortable having opioid dependent patients come to my practice.	13/45 (29%)	32/45 (71%)	0	46/46 (100%)	<0.001
I would feel comfortable prescribing naloxone to a patient with opioid addiction.	10/45 (22%)	35/45 (78%)	0	46/46 (100%)	<0.001
I would feel comfortable prescribing naloxone to a family member of a patient with opioid addiction.	12/44 (27%)	32/44 (73%)	1/45 (2%)	44/45 (98%)	<0.001
I am comfortable administering the intramuscular formulation of naloxone.	20/43 (47%)	23/43 (53%)	4/46 (9%)	42/46 (91%)	<0.001
I am comfortable administering the intranasal formulation of naloxone.	18/43 (42%)	25/43 (58%)	0	46/46 (100%)	<0.001
I am comfortable administering the auto injector formulation of naloxone.	23/43 (53%)	20/43 (47%)	0	46/46 (100%)	<0.001
It is reasonable for a local government policy, based on available public resources, to not provide naloxone for repeat overdosers.	29/43 (67%)	14/43 (33%)	33/46 (72%)	13/46 (28%)	0.096
Increased public access to naloxone will increase risky opioid use.	28/42 (67%)	14/42 (33%)	38/44 (86%)	6/44 (14%)	0.048

Responding to the four additional post-survey indirect assessment questions, 44 of 48 (91.7%) participants agreed that the didactic and skills session met the expectations of the learners, objectives were clear, the pace of the training was appropriate, and that the information will be of use in their clinical practices.

DISCUSSION

Our study is one of the first to assess naloxone training methods in medical residents consisting of both a didactic and skills station. However, there have been several previous studies that have analyzed naloxone training in other types of trainees including emergency medical technicians, pharmacy students, medical students, and others that only included didactic type interventions.32-36 Our pilot of a naloxone didactic and skills session for primary care learners demonstrated substantial improvements in the domains of knowledge, self-efficacy, and attitudes. Prior studies have reported several barriers to practicing physicians incorporating naloxone into care of patients with substance use disorders including knowledge gaps in use and proper administration with those who have never received naloxone training.28 Given that the ACGME currently does not specifically require any education or curricular activities regarding opioid use disorder treatment, our training model provides a unique opportunity to address a critical gap in knowledge and skills for primary care providers who will encounter this epidemic in many dimensions in practice. This program was developed with less than USD 28,000 of grant support for faculty time and supplies. The workshop component and evaluation form were created over 3 months. However, once created, such a program can be used for ongoing training without significant time or costs to training programs looking to enhance their learners’ preparation to meet the need of patients with opioid use disorder. During our training, participants were not provided with personal active naloxone kits in an effort to keep costs low. Naloxone can be obtained without any prescription authorization and some community health centers have federal funds to provide free naloxone kits to community members. Participants who were interested in carrying a personal kit were provided this information as a source of free naloxone kits.

The results from the pre-post survey assessment suggests significant gains are possible in both knowledge and self-efficacy when implementing such a training program and this was true across both sessions. Based on similar improvement in pre and post session attitude and knowledge scores, we found this session format to be successful with both a smaller group of seven participants and a larger group of 41 participants. Increasing knowledge and comfort with naloxone amongst primary care providers outside of specific substance use disorder treatment settings is particularly encouraging considering where the reach of opioids has been found and persists, both in legitimate and illicit formulations. For example, a primary care physician at a community health center might provide a prescription for naloxone to a patient challenged with heroin use disorder; or they might facilitate naloxone access to a son or daughter caregiver of an elderly patient who has been prescribed opioids for pain. The pilot results of our training model demonstrated improvement in comfort levels administering four different methods of naloxone. Exposure to training, as suggested by others, in addition to personal experiences with patients and/or family members can impact necessary changes in behaviors and attitudes toward providing access to naloxone for patients who would benefit most.28,29

Evaluation of the pilot also suggests that an investment in a didactic and skills session exploring policy, access laws, identification of overdose symptoms, and the advantages and function of differing methods of naloxone administration may mitigate some of the stigma and treatment myths that persist at the practicing provider level. For example, despite the lack of any supporting evidence, many providers who report never having naloxone training still posit that facilitating naloxone access will enable risky drug behaviors.18,30 Our training content described evidence countering these claims and, based on participant responses to the evaluation question directly targeting this belief, appears to mitigate such attitudes by primary care trainees.

The multidisciplinary approach with a College of Pharmacy faculty member also proved advantageous and enriched the perspectives of the training session based on informal feedback from participants and facilitators. Specifically, studies have identified primary care providers have knowledge gaps in how to prescribe naloxone and materials required for use as well as cost issues from commercial insurance. Furthermore, recent reports support the efficacy of naloxone access laws for pharmacists in reducing opioid-related deaths.31 Pharmacists will be doing more direct patient/caregiver training on the use of naloxone, thus including pharmacy learners as skill station trainers under the supervision of pharmacy faculty for the primary care learners anecdotally enhanced their comfort in providing medication use counseling. Given the expanded roles for health care providers in the provision of naloxone it makes efforts in pharmacy and primary care training programs timely and more critical for the care of those who experience opioid overdose.

Limitations

Though the study provides initial support for targeted training to improve knowledge, attitudes, and efficacy in naloxone administration, there are several limitations with respect to our design and its broader generalizability. First, although we offer a readily available, adaptable, and financially accessible training model, the authors acknowledge that materials and faculty and staff resources were grant-supported and may vary across programs. However, the supplies listed are economical, the evaluation survey in the pilot session can be used and/or adapted, and the literature describing naloxone training in various environments is considerable; all are qualities facilitating access. The cost limitation can impact sustainability of the intervention as supplies are consumed through training sessions. Second, content for the pilot session regarding policies and laws around naloxone access and administration were unique to our region in Cincinnati, Ohio. While local policies reflect a growing number of state and local policies, it is advised to consult local and state laws to update content and evaluation as needed. Third, interprofessional training was used in the pilot session, however, the training could be facilitated by any one professional health care faculty including pharmacy, primary care, or behavioral health. We included pharmacy learners as skill station trainers to foster their counseling skills surrounding naloxone, but did not formally evaluate the impact of their participation. Although our results suggest concurrent training of family medicine and physician assistants can be efficacious we are not able to specifically measure how having interprofessional faculty and pharmacy learners as trainers might have impacted these results. Furthermore, our evaluation approach did not include an assessment of actual learner skill in administering naloxone. The groups were designed to be small enough to allow individual instruction and observation, but there was not a specific competency assessment of learner skill in naloxone administration. Thus, it is possible that learners made gains in knowledge, attitudes, and self-efficacy while maintaining gaps in competency related to physical administration of naloxone. This could be addressed through the use of an OSCE type evaluation in addition to the survey instrument used in this intervention. Fourth, the training does not and is not intended to address the larger opioid issue in terms of overprescribing or diversion. It also is not intended to fulfill all recommendations for substance use disorder treatment training, such as medication assisted treatment. However, it is a critical piece in specifically targeting a harm reduction approach to acute and life-threatening conditions as a result of opioid misuse. Finally, there could be issues of subject bias given the pre and post surveys were administered at a single group session required as part of the training program. Additionally, although the training included 48 learners the number of complete survey responses was less than this indicating some participants did not answer all questions. Although the number of missing responses was very small, this coupled with our smaller sample size could have artificially elevated the findings of improvement on the post test, which can be mitigated through serial confidential administration to successive groups of learners over time to increase the number of completed responses analyzed. No follow-up survey has yet been designed or administered for those who participated in the training programs and thus we can’t comment on permanence of the effect on knowledge and attitudes towards furnishing naloxone found in our work. An additional goal at project conception was to measure the association between participating in such training and the actual provision of naloxone at the patient level. Assessment of this goal was complicated because all trainees in a given year were required to participate in the training program so any comparison would have been between different years of trainees. Therefore, we are confidently able to describe gains in knowledge and attitudes, but not patient level changes in practice which could be done if this training was bundled as part of a QI project in a particular clinic location using naloxone prescription as an outcome pre-post.

Future Work

A few of the prior naloxone studies utilized patients in recovery to share their personal stories prior to the training workshop.20 We believe this could be a great way to capture participants’ attention by augmenting the urgency feel of satisfactorily completing training. Moreover, incorporating patient voice into future trainings to foster development of a more positive provider attitude towards patients struggling with substance use disorders represents a logical augmentation of this intervention. Perhaps a readily accessible manner of infusing patient voice would be to bring this training to existing Patient and Family Advisory Council (PFAC) meetings. Another prior study, focused on training student pharmacists to administer naloxone, utilized the objective structured clinical exam, commonly known as the OSCE, to assess skill and confidence level after training.33 Using the OSCE or some other formal skill competency station as another post-training assessment could serve a dual purpose: (1) provide additional information towards gains in skill level and attitudes and (2) further solidify participant’s knowledge in naloxone training by providing a direct case to practice their new skill set. Finally, having developed the naloxone didactic and skills session with an eye towards broad adoption, future work should aim to replicate our positive findings across additional classes of health care team members including medical students, community providers, and additional graduate medical education learners.

CONCLUSIONS

Our study provides a piloted, adaptable, and efficient model for delivery of a naloxone didactic and skills session. This is critical given the positioning of the primary care provider at the front line of managing their patient’s potential substance use disorder and being in the best position to facilitate enhanced use of harm reduction approaches such as naloxone to both their patients, family members of patients, and community members at large. The pilot data suggest that the training was efficacious in enhancing knowledge, attitudes, and self-efficacy of learners. Material costs are minimal and the entire intervention was completed in two hours of classroom time. Implementing naloxone training is a critical and timely component of education that should be integrated into curriculums to fill a current gap in ACGME education requirements.