Pilot Evaluation of Generation Rx Ambassadors Virtual Training to Prepare Students for Medication Safety Outreach

Abstract

Objective. To describe the creation of a virtual training program (Generation Rx Ambassadors) and evaluate a pilot offering’s impact on knowledge and perceived abilities in delivering medication safety outreach through the Generation Rx program.

Methods. Generation Rx (GenRx) is a prevention education program used by student pharmacists to teach safe medication practices in the community. An asynchronous virtual course, called Generation Rx Ambassadors (Ambassadors), was developed to train facilitators on best practices for GenRx delivery. The training was piloted in a mixed student cohort and evaluated using a preprogram/postprogram survey assessing participants’ objective knowledge gains and self-perceived abilities to appropriately deliver GenRx education.

Results. Fifty-two health sciences undergraduate and graduate students as well as Doctor of Pharmacy (PharmD) students completed the pilot offering of Ambassadors. Regardless of degree status or discipline, participants demonstrated significant knowledge gains for all outcomes except defining medication misuse behaviors (for which there was initial strong mastery). Prior to Ambassadors training, many participants indicated a perceived ability to effectively deliver GenRx education; however, corresponding objective knowledge assessment did not support this belief. Training through the Ambassadors program appropriately aligned participants’ perceived abilities with actual content knowledge for most program learning outcomes.

Conclusion. These pilot findings suggest that Ambassadors is an effective training tool on best practices for GenRx delivery. More generally, this work reiterates a need to formally train student pharmacists ahead of community outreach activities, particularly in the prevention education arena. Future evaluation will focus on replicating this study with an expanded cohort size and assessing participants’ ability to deliver GenRx education in community-based settings.

INTRODUCTION

Over the past two decades, the misuse of prescription medications has become an emerging public health crisis, with prescription opioids, stimulants, and sedatives representing the most misused classes of medications.¹ As the most accessible health care professionals, pharmacists play an important role in recognizing and addressing medication misuse, including prevention efforts that educate people of all life stages about safe medication practices. In 2007 and in response to this then-emerging public health crisis, faculty at The Ohio State University (Ohio State) established Generation Rx (GenRx), an evidence-informed prevention program to teach audiences of all ages about safe medication-taking behaviors and the dangers of prescription drug misuse.² Rooted in best practices in pedagogy, drug prevention, and community engagement, the GenRx resources include five comprehensive tool kits for engaging elementary, teen, university, adult, and older adult populations in effective and developmentally appropriate learning experiences. For example, by using the Older Adult Toolkit, a student pharmacist can deliver an interactive educational experience to adults aged 55 and above, after which participants will have increased knowledge of the safe medication use/misuse, storage, and disposal practices relevant to their stage of life.

From 2010-2020, and in collaboration with the American Pharmacists Association Academy of Student Pharmacists, GenRx activities expanded beyond a single student organization at Ohio State’s College of Pharmacy to approximately 120 schools and colleges of pharmacy across the United States. In that work, over 60,000 student pharmacists nationwide directly delivered GenRx programming in community engagement efforts. The work offered hands-on opportunities for students to develop and practice new skills in communication, education, and meaningful community engagement; however, formal training opportunities in GenRx delivery were lacking, and untrained facilitators could (and sometimes did) turn to non–evidence-based tactics in their efforts. Through community reporting and facilitators’ firsthand accounts, GenRx faculty anecdotally confirmed a concerning rise in the use of scare tactics or inappropriate norming techniques, particularly after the release of the youth-oriented tool kits. The presence of scare tactics and stigmatizing language were also observed in student-developed social media marketing for local GenRx events. These techniques are unsupported in the prevention education field, and some may even promote drug misuse behavior.³

It is imperative that any educational outreach program, and particularly those targeting youth, adhere to best practices and evidenced-informed approaches in both content and delivery; as such, conversations ensued about appropriately training GenRx facilitators in the program. Initially, GenRx training for the student pharmacy community was minimal at best, limited largely to guidance provided in the facilitator guides of each tool kit. Between 2016-2019, GenRx faculty piloted a train-the-trainer approach among nonpharmacy facilitators; however, this costly, time-intensive approach could not keep pace with the tool kit updates, making it an unsuitable solution for broadly training student pharmacists. Thus, drawing on best practices in virtual and adult-focused education, GenRx faculty developed an asynchronous virtual training program called Generation Rx Ambassadors (Ambassadors). This manuscript describes the creation of Ambassadors, with emphasis on evaluation of a pilot offering’s impact on participants’ knowledge and perceived abilities toward appropriately delivering GenRx medication safety outreach.

METHODS

Crafted with a backward design model,⁴ Ambassadors is centered on program-level outcomes in which students learn to discuss the mission and key messages of GenRx within its framework as a safe medication use and misuse avoidance prevention strategy for appropriate audiences, explain medication safety principles and the consequences associated with medication misuse behaviors in an age-appropriate manner, and implement a GenRx educational program that demonstrates best practices in prevention and education sciences while maintaining program integrity. The course design process identified asynchronous delivery of five self-paced virtual modules through Ohio State’s public-facing learning management system as the most sustainable and effective delivery modality (Figure 1).

Figure 1.

Design of the Generation Rx Ambassadors Program to prepare students for medication safety outreach.

A schematic illustrating the program design of the Generation Rx Ambassadors virtual training, which consists of five self-paced modules developed by applying backward design principles (writing learning outcomes, designing assessment, and developing learning activities and content).

Abbreviations: IVP=interactive video presentation; KC=knowledge checks.

Because both pharmacy and nonpharmacy students have historically delivered GenRx education, this pilot study recruited a convenience sample of health sciences undergraduate and graduate as well as Doctor of Pharmacy (PharmD) students from three large public universities located in Ohio and West Virginia. GenRx faculty sent a recruitment email to listserves of relevant students and to program advisors who disseminated the study information. All student participants voluntarily enrolled in the study and were compensated for their time by receiving up to $30 in gift cards for completion of the training program and all three survey instruments. Exemption from review by Ohio State’s institutional review board was granted.

This pilot study was implemented in two phases due to logistical considerations from working across three institutions. In each phase, participants were granted three weeks during either June (phase I) or July (phase II) 2020 to complete the Ambassadors training program and participate in the research study. Participants securely logged into the learning management system as frequently as needed during the three-week period to complete all five modules. However, the training program satisfaction survey indicated that on average, most participants took approximately three hours to complete all five modules, including all assessments identified in Figure 1 (57% took one to three hours, and 29% took three to five hours, data not shown). Out of 69 total enrolled participants, 52 students completed the training program and all three survey instruments (75% completion rate). Independent t tests indicated no significant differences across phase I and phase II participants; thus, participant data was pooled into a single cohort.

To assess the effectiveness of Ambassadors training, participants completed a preprogram and postprogram survey upon completion of modules 1 and 5, respectively, as well as a training program satisfaction survey. The pre- and postprogram survey consisted of five demographic questions related to educational program, age category, and self-reported experience with GenRx; 20 subjective questions that assessed participants’ perceived abilities toward delivering GenRx education; and five objective multiple-choice questions that assessed participants’ understanding of the training program learning outcomes (Table 1). The training program satisfaction survey consisted of 16 questions that assessed participant satisfaction of their training experience (data not shown). The GenRx faculty created these survey instruments with consultation support provided by the Center for Health Outcomes and Policy Evaluation Studies at Ohio State. All measures were collected asynchronously. Data were exported from the learning management system and deidentified, and statistical analyses were performed using SPSS version 27 (IBM Corp). Descriptive statistics were used to describe student demographics, percentage agreement (reported on a Likert scale, strongly disagree to strongly agree) on the subjective questions, and percentage correct on the objective questions. A two-sided paired t test was used to analyze changes in knowledge and perceived abilities within pilot participants between the pre- and postprogram surveys. The level of statistical significance was set at α=0.05.

Table 1.

Pilot Participants’ Responses to a Survey Administered Before and After Generation Rx Ambassador Training (N=52)

RESULTS

Fifty-two participants completed the pilot offering of Generation Rx Ambassadors. The participants ranged in age from 18-39, with most (60%) having an age between 18 and 22. A slight majority (54%) of participants were training in a PharmD program, while the remainder (46%) were health sciences undergraduate or graduate students. At the onset of training, many participants either had no experience implementing GenRx education (52%) or were entirely unaware of the GenRx program (19%).

After Ambassadors training, increases were observed in most items that assessed perceived abilities (ie, subjective measures) related to delivery of GenRx programming (Table 1). Regarding knowledge gains (ie, objective measures), participants demonstrated a strong ability to identify medication misuse behaviors and, as such, did not demonstrate significant knowledge gains related to that question. However, significant knowledge gains from pre- to postprogram completion were observed in the remaining four items related to Ambassadors program-level learning outcomes (Table 1).

Of note, the perceived abilities identified in five of the subjective questions (specifically items 1, 3, 6, 8, and 16 from Table 1) correlated with knowledge assessed in the five objective questions (specifically items 21-25 from Table 1). As shown in Figure 2, the preprogram data suggest a disconnect between participants’ perceived abilities and actual knowledge in four of the five content areas related to Ambassadors learning outcomes. After Ambassadors training, the postprogram data indicated alignment of participants’ perceived understanding with their actual knowledge within content areas related to the GenRx mission, key messages, and program implementation guidelines, but not appropriate GenRx program audiences.

Figure 2.

Preprogram and postprogram results for alignment of knowledge and perceived abilities in pilot participants in the Generation Rx Ambassadors training program.

A graph that overlays preprogram and postprogram results of participants’ perceived abilities (circles) and actual knowledge (bars) in the five content areas related to the learning outcomes of the Generation Rx Ambassadors virtual training. The knowledge assessed in the five objective questions corresponds with the perceived abilities identified in five (out of 20) subjective questions. Percentage correct for each objective multiple-choice question as well as the percentage answering strongly agree or agree for each related subjective question on the pre- and postprogram surveys from all pilot participants (N=52), analyzed via paired t test; *p<0.05, **p<.001 (objective questions only).

DISCUSSION

The findings from this pilot study suggest that the Generation Rx Ambassadors virtual training program increases participants’ knowledge and perceived abilities to appropriately deliver GenRx education and, thus, helps to train effective facilitators of the GenRx program. Some participants did enter this study with a foundational understanding of some GenRx program content, including the ability to identify medicine misuse behaviors; this is not an entirely unexpected result given that participants had an educational foundation in the fields of pharmacy and health sciences and/or potentially some experience with GenRx. However, prior to training, students did report low abilities to relay this content knowledge to others, to explain how GenRx functions as an evidence-informed prevention program, and to properly design/deliver best practice GenRx educational programs (Table 1).

Even with some baseline knowledge, participants still demonstrated significant learning gains related to their understanding of the GenRx mission, key messages, appropriate audience selection, and program implementation guidelines (Figure 2). In addition, in all areas except for Appropriate Audiences, the percentage correct for these objective measures on the postprogram survey was near or at 100%, suggesting that the training program helped to strengthen or reinforce participants’ understanding of important concepts related to teaching safe medication practices. Regarding the ability to identify appropriate GenRx audiences, the gap in perceived versus actual knowledge persisted posttraining, with nearly all participants (n=51, 98%) perceiving that they could identify appropriate audiences even though only a minority of participants could do so (n=18, 35%). As part of the course quality improvement process, GenRx faculty have subsequently strengthened Ambassadors training content to address this gap.

More generally, these findings reinforce the need for students, including PharmD and other health professions students, to receive training before community outreach efforts. Previous research indicates that people often overestimate their competence,⁵ and professional pharmacy and health sciences students are no exception. Additionally, drug prevention work may be particularly in need of training, as some techniques (eg, scare tactics) that seem instinctively effective can be dangerously counterproductive. Our pilot findings indicate that completion of the Ambassadors training program helped participants align their perceived understanding with their actual knowledge within most content areas while also preparing them for sound drug prevention efforts.

Despite these positive outcomes, this pilot study had some limitations that must be considered. First, because participants were recruited from health-focused academic programs, the lack of a control group makes it unclear whether participants’ responses would vary from the general population. Second, because data were self-reported, participants may have inaccurately shared their thoughts and skewed responses in a more socially acceptable manner. Third, the demographics collected from pilot participants were limited. Results may vary with larger, more diverse populations. Lastly, this pilot study did not assess knowledge retention long-term, or assess whether participants could successfully apply the training toward effectively delivering GenRx education in the community. Future studies may help address these limitations by strengthening the survey instruments to include a broader set of demographic questions as well as by implementing a longitudinal assessment to evaluate knowledge retention and appropriate delivery of GenRx education. In addition, as medication misuse remains an interdisciplinary public health issue, future efforts involve creating Generation Rx Ambassadors training programs that fit the unique needs and contexts of diverse health care (eg, pharmacists, veterinarians, and nurses) and community (prevention specialist) partners.

CONCLUSION

Implementation of the Generation Rx Ambassadors program provided health sciences undergraduate students, graduate students, and PharmD students with an opportunity to increase their knowledge and self-reported abilities toward effective delivery of GenRx education. Pilot study findings indicate that a three-hour asynchronous virtual training may serve as a sustainable and efficient approach toward formally training future GenRx facilitators. With GenRx resources used by students at many colleges of pharmacy nationwide, Generation Rx Ambassadors may help prepare future health professionals for effective medication safety outreach.