Long-term effects of opioid overdose prevention and response training on medical student knowledge and attitudes toward opioid overdose: A pilot study

Abstract

Background:

Medical settings provide ideal opportunities to identify patients with substance use disorders and provide harm reduction and treatment resources. Medical students often volunteer in the community and can spend substantial time with patients, serving as touchpoints. Accordingly, medical schools have begun training in harm reduction. Initial studies show such training acutely improves knowledge, but sustained effects remain unclear. This pilot study explored longer-term impacts of Opioid Overdose Prevention and Response Training (OOPRT) on medical student knowledge about opioids, overdose, and naloxone.

Methods:

Students completed a survey about knowledge of opioid use disorder, overdoses, and attitudes towards patients. This included Opioid Overdose Knowledge (OOKS) and Opioid Overdose Attitudes (OOAS) scales. A subset of students was invited to attend OOPRT and complete a post-training survey. All who completed the baseline survey were invited to complete a 6-month follow-up. We analyzed long-term training effects on OOKS and OOAS scores.

Results:

89 students completed baseline and 6-month follow-up surveys; of these, 22 received training. OOPRT yielded significant improvements in knowledge of signs of opioid overdose (F(2,38) = 18.04, P < .001), actions to take during overdose (F(2,38) = 8.32, P = .001), and naloxone use (F(2,38) = 35.46, P < .001), along with attitudes regarding overdose competencies (F(2,38) = 99.40, P < .001) and concerns (F(2,38) = 8.86, P < .001). When comparing over time, students who attended OOPRT retained significantly higher competency scores than those who did not attend F(1,87) = 40.82, P < .001). No other significant differences were observed.

Conclusions:

This study demonstrates immediate efficacy of OOPRT in improving opioid overdose knowledge and attitudes and sustained changes at 6 months, compared to standard undergraduate medical curricula alone. Future research with larger sample sizes is underway to validate these preliminary findings and examine the difference in attitudes and knowledge retention over time. Given that students report interest in receiving OOPRT and consider it worthwhile, systematic study is warranted.

1. Introduction

Clinical encounters are crucial opportunities for physicians to provide education and harm reduction tools to persons who use drugs. Most patients with substance use disorders (SUDs) have experienced health consequences of use (e.g., overdose, infection) that required visiting healthcare professionals (Moses et al., 2018; Saloner & Karthikeyan, 2015; Wu et al., 2016). The quality of these interactions can have long-lasting impacts (Avery et al., 2019); thus, physicians should be trained in harm reduction early in medical training.

Despite recent attempts to integrate opioid overdose prevention and response training (OOPRT) in undergraduate medical education (UME), few studies have measured its longer-term impacts beyond the immediate post-training period. Although it is clear that training can lead to demonstrable gains in skills and knowledge, it is not clear whether those gains persist. This is important because the purpose of UME is to provide students with knowledge they can use in their clinical encounters; thus, goals of these trainings should include knowledge retention and skill utilization. Initial studies with medical students demonstrated training can increase knowledge about and preparedness to respond to overdoses (Berland et al., 2017; Berland et al., 2019) whereas training effects on student attitudes towards patients are less consistent. Although OOPRT should immediately improve knowledge, it remains unknown whether its educational benefit exceeds that of the standard medical curriculum.

This study evaluated the impact of a pilot OOPRT program at a large medical institution in the United States. Understanding how training impacts students’ opioid overdose knowledge, willingness to use naloxone, and attitudes toward patients with opioid use disorder (OUD) will help guide future curriculum enhancements. Aims were to (1) examine changes in knowledge and attitudes toward opioid use and overdose immediately after OOPRT, (2) identify changes in knowledge and attitudes associated with training over time, and (3) compare longer-term (6-month) changes in knowledge and attitudes between students who received training and those who did not.

2. Methods

2.1. Study design

In Spring 2019, we launched a pilot study at Wayne State University School of Medicine to identify student need for and interest in receiving additional training about SUDs and harm reduction. IRB approval was waived; this study received exemption status. All first- through third-year (M1-M3) medical students (~900 students: ~300 per year) were invited to complete a survey about their knowledge, attitudes, and beliefs regarding SUDs and harm reduction. Students voluntarily completed the survey, and those who completed were informed they had the opportunity to receive OOPRT. To minimize risk of coercion, students were informed about free community training programs they could attend regardless of whether they completed the survey.

For this pilot study, size of the training class was limited to 45 students due to time and funding constraints. Random number generation was used to select 15 students from each class year from among those who completed the baseline survey; participation was voluntary. Selected students were invited by email to the training and those who indicated they were unavailable were replaced by another randomly-selected student from the same class. This process continued until 15 students per class confirmed their availability to attend or no more students were available to invite for that class.

One formal OOPRT session was offered, which was attended by the 38 students from across all three class years. The training was provided by a skilled clinical pharmacist specializing in SUDs and delivered in discussion-guided format. Nine overarching competency-based areas focused on: opioid overdose identification, opioid overdose risk factors, opioid overdose response, naloxone use, harm reduction, naloxone access laws, good Samaritan laws, myth busting, and stigma reduction. Curriculum training details can be found in Moses et al., 2021. Immediately after training, students were asked to complete a post-training survey and 6-months later, all students who completed the baseline survey (including the 45 students who completed training) were invited to complete a follow-up survey. Fig. 1 illustrates the study design, timeline, and student groups.

Fig. 1.

Overview of Study Design Methods. Completion of the baseline survey was voluntary for all students. Training attendees were selected randomly from students who had completed the baseline survey; attendance at the training was voluntary. Random number generation was used to select 45 students (15 from each class year) to attend. Students who indicated they were unavailable for training were replaced by randomly selected students from the same class. This process continued until 15 students per class confirmed their availability to attend or no more students were available to invite. Due to expected “no-shows” at training and non-completion of surveys at each time point, the number of students included in the data analysis differs from the number invited to take part, and we indicate these differences below.

2.2. Measures

Baseline survey questions assessed students’ prior experiences in healthcare, working with people with SUDs, experiences with naloxone, and opinions regarding these topics. Because personal identifying information was collected, questions about students’ own substance use were collapsed into general personal experience questions to maintain privacy (i.e., “Do you know someone personally (including yourself) who has experienced a substance use disorder (addiction) at any point in their lives?”).

Primary outcomes of knowledge and attitudes toward opioid overdose and patients with SUDs were measured at all 3 timepoints using 3 standardized assessments. The Opioid Overdose Knowledge Scale (OOKS) measures respondents’ comprehension in 4 domains of opioid overdose knowledge: overdose risk factors, signs of overdose, actions to take during overdose, and naloxone use (Williams et al., 2013). The Opioid Overdose Attitudes Scale (OOAS) measures 3 domains of respondent self-perceived comfort and ability to respond to an opioid overdose situation: competencies, concerns, and readiness to respond (Williams et al., 2013). The Medical Condition Regard Scale (MCRS) modified for SUDs assesses healthcare provider attitudes and stigma towards people with SUDs via respondent agreement with 11 statements on a 6-point Likert scale (Berland et al., 2019). For additional details regarding the scales and scoring, see Moses et al., 2020.

2.3. Data analysis

Paired t-tests were used to evaluate initial outcomes of OOPRT (Aim 1) , comparing pre- to post-training changes in OOKS, OOAS, and MCRS domains. To evaluate changes over time associated with training (Aim 2), repeated measures one-way analyses of variance (rmANOVAs) with Bonferroni adjustment for multiple comparisons were used to compare changes in OOKS and OOAS domains across the 3 timepoints (baseline, post-training, 6-month follow-up). In all rmANOVAs the assumption of sphericity (Mauchly’s test) was met. To explore whether training influenced knowledge and attitudes beyond the traditional curriculum (Aim 3), we used mixed-model ANOVA with time (baseline and 6-month follow-up) as the repeated factor and condition (training vs. no training) as the between-subject factor. The criterion of P < 0.05 was used to reject the null hypothesis.

3. Results

3.1. Characteristics of students who received training

Of the 45 students who expressed interest and were invited to OOPRT, 38 attended and 34 (44.1% M1, 29.4% M2, 26.5% M3) completed baseline and post-training surveys. Supplemental Table 1 presents participant characteristics. Two-thirds knew someone who had SUD or had SUD themselves, 32.4% had previously seen someone overdose, and 41.2% knew someone who had overdosed. 100% were interested in receiving training.

3.2. Initial response to training (Aim 1)

Among the 34 students who completed training and baseline and post-training surveys, paired t-tests showed that OOPRT yielded significant improvements in 3 of 4 OOKS knowledge domains (signs of an opioid overdose, actions to be taken during overdose, and naloxone use) and all 3 OOAS attitude domains (competencies, concerns, and readiness to intervene). Three of 11 MCRS responses showed significant changes, indicating more positive attitudes towards patients with SUDs after training. Post-training, fewer respondents agreed with the statements, “Patients with SUDs are particularly difficult for me to work with” and “there is little I can do to help patients with SUDs”; whereas more respondents agreed with the statement, “I can usually find something that helps patients with SUDs feel better.”

3.3. Changes in knowledge and attitudes over time (Aim 2)

Twenty students (40.0% M1, 40.0% M2, 20.0% M3) completed OOPRT as well as the baseline, post-training, and 6-month follow-up surveys and were available for longer-term analyses. rmANOVAs showed that OOPRT led to statistically significant improvements in at least one post-training measure (either immediate post-training and/or 6-month follow-up) for 3 of 4 OOKS domains and 2 of 3 OOAS measures. OOKS results showed statistically significant improvements in knowledge of signs of an opioid overdose (F(2,38) = 18.04, P < .001, partial η² [η_p²] = 0.487) and naloxone use (F(2,38) = 35.46, P < .001, η_p² = 0.651), both immediately post-training and at 6-month follow-up. Knowledge of actions in overdose significantly improved (F(2,38) = 8.32, P = .001, η_p² = 0.305), but did not remain significant after 6-months. OOAS scores significant improved for competencies to manage an opioid overdose (F(2,38) = 99.40, P < .001, η_p² = 0.840) and concerns regarding opioid overdose response (F(2,38) = 8.86, P < .001, η_p² = 0.318). OOAS scores increased immediately post-training for all domains, but decreased between post-training to 6-month follow-up. Supplemental Figs. 1 and 2 show changes over time. MCRS statements were excluded from further analyses because minimal change was seen immediately post-training.

3.4. Effect of training compared to the standard curriculum (Aim 3)

There were no group differences in baseline demographics for students who completed OOPRT vs. those who did not (Supplemental Table 1). A group x time interaction for naloxone use knowledge (OOKS), (F(1,87) = 5.00, P = .028, η_p² = 0.054), indicated that scores for both groups significantly increased post-training relative to baseline, and training group scores increased more compared to the no training group, although this difference was no longer statistically significant after Bonferroni adjustment. A group × time interaction on competencies to manage opioid overdose (OOAS), F(1,87) = 40.82, P < .001, η_p² = 0.319), indicated that scores for both groups significantly increased from baseline, and were greater for students who received training compared to those who did not receive training (Fig. 2).

Fig. 2.

Change in Opioid Overdose Knowledge Scale (OOKS) naloxone use domain (upper panel) and Opioid Overdose Attitudes Scale (OOAS) competencies (lower panel) over time for those who received OOPRT (n = 22) and those who did not (n = 67); *significant (P < .05) increase from baseline, #significant (P < .05) difference between training and no training groups at indicated time point.

Significant effects for time were found for OOKS domains of signs of opioid overdose (F(1,87) = 22.35, P < .001, η_p² = 0.204) and naloxone use (F(1,87) = 49.32, P < .001, η_p² = 0.362), OOAS domains of competencies to manage opioid overdose (F(1,87) = 100.06, P < .001, η_p² = 0.535), and concerns about managing overdose (F(1,87) = 9.51, P = .003, η_p² = 0.103). Scores of all these indices increased over time independent of training.

4. Discussion

This pilot study assessed long-term effects of an OOPRT designed to enhance UME curriculum surrounding SUDs and harm reduction. Although some studies have examined immediate effects of SUD-related education during medical training (Berland et al., 2017, 2019; Oldfield et al., 2020), none to date have tracked longer-term learning retention. Furthermore, no study has compared the impact of a targeted OOPRT relative to the standard UME curriculum. Given the growing number of schools incorporating these trainings in their curriculum, it is important to identify not only whether OOPRT provides immediate benefits, but also to examine whether training results in sustained knowledge gains compared to standard curriculum.

Our findings suggest OOPRT improved retention of opioid overdose knowledge and attitudes towards responding to an opioid overdose, with minimal effect on attitudes towards patients with SUDs. Students showed significant improvements in three-quarters of OOKS domains and two-thirds of OOAS domains immediately post-training. Our findings are consistent with those of Berland et al., 2017, who found that a similar training improved scores on OOKS and OOAS assessments but not the MCRS for SUDs (Berland et al., 2017). Considering the impact that negative attitudes have on patient care (Avery et al., 2019), future training should consider including specific components targeted to reduce bias and improve empathy for individuals with SUDs. Although individual training will not suffice to combat structural stigma, it may still be a vital component within the broader context of widespread anti-stigma efforts (Goldberg, 2017; Khenti et al., 2019).

Assessment of the training group after 6-months demonstrates that most learning from OOPRT persisted, although learning in some domains was not retained. Six-month follow-up results showed that knowledge and attitude scores, even among students who did not receive training, improved over time; however, those with training scored higher on competencies to manage opioid overdose than those without training. Due to heterogeneity of pre-clinical and clinical experiences across different years in this stage of medical education, it is unlikely these general improvements are directly related to information gained from a particular class. Instead, we believe most participants were interested in this topic and in receiving training, so they likely gained additional knowledge through independent learning, volunteering, progression of their medical training, and training offered in the community. Although no other studies of medical students evaluate longer-term retention from similar trainings, one study conducted OOPRT with family members of patients with SUDs and assessed longterm (3-month) retention of knowledge and attitude changes (Williams et al., 2014). The results of our pilot study align with that previous research, which found that some gains from OOPRT are maintained over time.

Strengths of this study include the longitudinal study design and validated assessment measures, enabling reliable detection of quantifiable change. This pilot study has limitations. First, the small sample size yielded low statistical power limiting our ability to detect between-group differences; thus, we also report effect sizes. Second, data were collected from surveys completed by participants based on recall, increasing potential for self-reporting bias due to social desirability or reduced accuracy in recall. Third, we were not able to control how student engagement in outside training and volunteering activities may have impacted any changes in knowledge and attitudes. Fourth, our initial survey had an engagement rate of 21.8%, meaning that most invited students declined to participate. While this engagement rate for an optional survey is within the expected range for this medical student population, our sample is potentially biased towards students who are more interested and engaged with this topic. Last, we did not correlate knowledge and attitude changes with likelihood of engaging in desired behavioral outcomes; this is an area for further study.

Our preliminary findings suggest OOPRT is more effective than standard curricula alone for improving knowledge/attitudes related to SUDs and overdoses. Subsequent studies with larger sample sizes over longer time-periods are needed to confirm our findings and to better characterize changes in knowledge and attitudes. At present, we are conducting a larger longitudinal study to validate these findings and explore relevant knowledge and attitude changes as students progress through all 4 years of medical school; however, additional work across multiple institutions will still be needed.