An Overview of the CRoME Lab’s MedSMA℞T Families Intervention for Promoting Opioid Medication Safety among Adolescents and Parents

Abstract:

As the opioid epidemic unrelentingly places profound weight on pharmacists, youth, parents, and the nation at large, innovative interventions are critical to prevent potential misuse and untimely death. Adolescents are especially vulnerable to misuse behaviors due to knowledge, attitudinal, and social influences. To best resonate with adolescents, appropriate tools and technologies must be developed and deployed. Few educational tools are more ubiquitous to adolescents than video games. Video games, specifically serious games, have the potential to connect with adolescents and impart critical knowledge related to opioid safety. Engaging youth, parents, and pharmacists in the design, refinement, and evaluation of a serious game is novel and necessary for building a robust, efficacious game. This commentary describes the conceptualization, inception, development, and evaluation of a novel opioid medication safety intervention, MedSMA℞T Families. This two-part intervention is comprised of an adolescent-tailored serious game entitled MedSMA℞T: Adventures in PharmaCity, and a family-focused tool named the personalized Family Medication Safety Plan. This intervention is the first theory-based, family-focused opioid medication safety toolkit co-designed by an interdisciplinary team of researchers, game developers, behavioral health scientists, pharmacists, student pharmacists, addiction medicine physician, adolescent health physician, parents, and teens. Responses from key stakeholders (pharmacists, parents, and teens) have demonstrated positive attitudes and impressions towards both components of the MedSMA℞T Families intervention. Empirical evidence of efficacy from two separate studies denotes improvement in key concepts related to safe behavior in adolescents. Findings to date support the widespread use, implementation, and dissemination of this efficacious intervention to supporting safe opioid prescription practices in families with adolescents. In this paper, we outline and summarize our process of intervention development and use of longitudinal data to validate each step of our process. Future directions include mobile compatibility, refinement based on key stakeholder feedback, and dissemination in various clinical settings where opioid medications are prescribed and dispensed such as emergency departments, pharmacies, hospitals, and clinics.

Background

The opioid crisis is an ongoing threat to the safety and health of children and adults alike. Adolescent death rates from opioid overdose have only continued to grow in recent years from both illicit and prescribed opioids.^1-3 From 2010 to 2020, overdose mortality rates increased from 12.4 per 100,000 to 27.86 per 100,000.² Another worrisome trend exists in both adolescent knowledge and behaviors as many are unaware of safe storage and disposal practices.⁴ Educational and prevention-focused interventions exist in the forms of school speaking programs, flyers and infographics, educational games, and social media messaging. However, many of these interventions are targeted at adults, lack scientific based foundations, or lack demonstrable efficacy.^5,6 Stakeholder engagement and co-design is a rigorous way to integrate research with end user needs while maintaining effectiveness.^7,8 As a critical population for opioid misuse prevention, a co-designed intervention for adolescents has the potential to promote safe behavior before and into adulthood. A potential intervention for adolescents is a serious game. Serious games are games in which the characterizing purpose is not entertainment, but to educate, train or otherwise improve educational or behavioral outcomes in players.⁹ Other games have approached game-based learning in a similar way, such as the Hope Lab’s game, Re-Mission.¹⁰ Serious games have targeted conditions in youth such as substance use, cancer treatment education, and mental health.^10-14

Innovation

In response to these trends and the critical need to engage adolescents in their own opioid prescription safety, the Collaborative Research on Medication use and family health (CRoME) lab embarked on the creation of a novel, serious game-based intervention for adolescents. The MedSMA℞T Families intervention is a family-based, adolescent-tailored intervention designed to proactively ignite family conversations about safe prescription opioid use in homes and clinical settings.¹⁵ This intervention is comprised of two parts, the MedSMA℞T: Adventures in PharmaCity (MedSMA℞T) serious game, and the personalized Family Medication Safety Plan (FMSP).

Overview of the MedSMA℞T Families Program

In this commentary we describe the conceptualization, development, and evaluation of MedSMA℞T: Adventures in PharmaCity and the FMSP to date. This paper surmises several studies conducted by the creators of MedSMA℞T Families leading to its current iteration and directions. We will cover each component separately, starting with the development and evaluation of the MedSMA℞T: Adventures in PharmaCity game then cover the Family Medication Safety plan before summarizing current and future directions for this intervention.

The Game: MedSMA℞T: Adventures in PharmaCity

The serious game, MedSMA℞T: Adventures in PharmaCity, is a digital, web-based game for adolescents and their families. Its most current version must be played on a computer browser. MedSMA℞T is built on the game engine, Unity WebGL.¹⁶ Characters in the game are anthropomorphic and genderless to reduce likelihood of cultural mismatches for players and improve relatability and identification with characters. The game is formatted similar to the genre of responsive narratives. Responsive narratives encompass games and novels where the audience chooses their own adventure, and their choices have an impact on outcomes throughout the story.

Players take the role of Shan, a sheep, who has broken their arm and oversees their own medication for a week. The game consists of five levels designed and adapted to feel realistic to adolescents.

Level 1: A Quiet Sunday Afternoon, teaches the player about safe storage requiring players to lock away medication left on the counter before Shan’s friends use it (Figure 1). Shan’s mom accidentally left her medication on the kitchen counter and the youth are contemplating giving it a try. Players must figure out how to safely store the medication to prevent misuse.

Figure 1.

Level 1: A Quiet Sunday Afternoon

Level 2: Monday Morning Bus Ride, Shan learns they have forgotten a major assignment while in pain (Figure 2). After an injury it is reasonable to forget about deadlines and Shan forgot about their important assignment. In this level, players are introduced to the rising action of a missed deadline.

Figure 2.

Level 2: Monday Morning Bus Ride,

Level 3: A Persuasive Speech at School, players encounter peer pressure and must resist the temptation to take another person’s medication to ease their pain (Figure 3). Shan is responsible for an important project, but their arm hurts too much to focus. Players need to think critically about whether Shan’s friend, Tracy’s plan to help is all that helpful.

Figure 3.

Level 3: A Persuasive Speech at School

Level 4: Bus Ride Home, reinforces the risks of sharing medications. This game level introduces Narcan® through an encounter with a stranger on the bus who is in pain and asks for relief (Figure 4). Players must decide whether to share their opioid with an injured stranger, discovering a way to help someone safely in this situation.

Figure 4.

Level 4: Bus Ride Home

Level 5: Last Minute Chore teaches the player the proper method of disposal once finished using an opioid prescription (Figure 5). In this level, players must leave the home to dispose of the medication in a pharmacy drop box, however, they must first encounter and reject common improper disposal methods.

Figure 5.

Level 5: Last Minute Chore

Purpose and Goals

The ultimate goal of the MedSMA℞T game is to facilitate opioid prescription safety for adolescents aged 12 to 18 years old. This is completed through supporting adolescent knowledge, behavioral intention, awareness, and self-efficacy related to the management, storage, and disposal of prescription opioid medications.

Foundational Research and Community-based Co-design

Early work in the space of adolescent opioid prescription safety began with assessing adolescents’ preferences for opioid prescription safety education. Focus groups were held in high schools where 54 youth participated in one of eight focus group sessions. This study uncovered adolescents’ perceptions of opioid safety and misuse. Adolescents reported misconceptions of opioid harm and safety, an awareness of teen misuse, and some youth suggested they were unsure what opioids were and which medications were considered opioids.¹⁷

To further understand youth’s opioid safety awareness and educational preferences, online surveys were conducted in schools with 190 Midwestern adolescents. Results from this study elucidated adolescents’ misconceptions around safe disposal and storage, showing that youth are often unsure of the dangers of practices such as flushing opioid medications down the toilet or leaving opioid prescriptions in unlocked cabinets. Students most often cited finding opioid information through web searches (67%), talking to their parents (56%), talking to healthcare professionals (54%), and online videos (42%).¹⁸

Through characterizing adolescents’ opioid safety awareness and categorizing preferences for education a serious game began to take shape. The following step was the creation of a scientifically based conceptual framework to enrich the development of game features for maximal impact on adolescent opioid knowledge and behavior. This framework integrates health behavior and game development models, relying on social cognitive theory and social learning theories. This conceptual framework, named the Serious Game Behavior Change Framework for Improving Medication Safety in Community Settings, was designed to create a serious game focused on medication safety in community settings (Figure 6).¹⁹ The framework incorporates two settings, the virtual game experience, and the real world, wherein the gaming experience represents intervention, and the real world represents outcomes. Within the game experience exists three domains: the in-game environment, game mechanics, and game feedback.

Figure 6.

The Serious Game Behavior Change Framework for Improving Medication Safety in Community Settings

With the framework in place, an interdisciplinary team of pharmacists, physicians, social-behavioral health experts, student pharmacists, teens, game designers, game developers, and medication safety and health outcomes experts was assembled. Guided by the aforementioned studies, characterizing adolescent baseline knowledge of opioid safety, learning goals were developed to specifically target the domains elucidated through adolescent reports and clinical and experiential expertise of the development team. The development team came to concurrence on education related to safe usage, storage, and disposal of opioid prescription medications. This led to the preliminary learning outcomes which would be assessed further through nation-wide surveys and other studies evaluating youth, parent, and pharmacist perceptions of game iterations. Choosing preliminary learning outcomes was necessary to create an initial prototype.

Evaluating the Game Prototype

A game prototype was developed over the course of a year, utilizing interdisciplinary and iterative feedback throughout the process, and guided by a game playbook. A prototype of MedSMA℞T: Adventures in PharmaCity was developed. The game utilized anthropomorphic, agender animal characters to pique the interest of youth regardless of race, ethnicity, gender expression and age and allow for identification with the characters. Through animated stylized realism, authentic scenarios were created to simulate real life experiences youth may encounter. A testing protocol was developed to ensure that the MedSMA℞T game was ready for research use. Both human and automated tests were conducted to ensure that the game was functional, and that the data collected was accurate. Then summative research and evaluation efforts began.

To evaluate the prototype, 10 informal playtests were held with over 300 adolescents and young adults (AYAs). These playtests were held in ubiquitous and comfortable locations for AYAs including classrooms in middle schools, high schools, and university courses as well as in public libraries. A brief questionnaire was administered to capture perspectives on the game and its components. Most youth reported the levels to be realistic, appropriate for the adolescent demographic, and that the learning objectives were achievable and salient.²⁰ After initial refinements to the prototype, it was ready for rigorous evaluation in terms of efficacy and feedback among key stakeholders.

Assessing Adolescents’ Opioid Awareness

To develop a common scale to measure the impact of MedSMA℞T, we conducted a national survey of 592 youth.²¹ Results indicated that most participants reported having some prior education on what opioids are, what opioids do, and the results of misuse, but few adolescents reported having education on how to safely store and dispose of opioids. Results reiterated the need to provide education to adolescents to improve their opioid knowledge, opioid misuse knowledge, behavioral intention regarding safe opioid practices and knowledge of appropriate disposal of opioids. In this nationwide survey 80% of youth indicated they believed playing an educational video game would increase their opioid safety knowledge.²¹

Another study aimed to create the first standardized scale that would be used to measure adolescents’ knowledge, behavioral intention, self-efficacy, and awareness of opioids. Thus, the Adolescent Opioid Safety and Learning scale (AOSL) was developed through exploratory factor analysis of survey responses from 188 high school students in the Midwest.²² This scale was psychometrically validated in a subsequent study using confirmatory factor analysis with a nationally representative sample of 774 adolescents.²³ Results from these studies indicated and validated the scale for use in adolescents across the United States to measure key concepts related to opioid knowledge and safe behavior. This scale is continually used in all studies which evaluate the efficacy of MedSMA℞T and its potential to impart crucial information and cognitions to youth.

Evaluating the Efficacy of MedSMA℞T: Adventures in PharmaCity

A study conducted by the MedSMA℞T team aimed to examine the effects of the game. Results from 117 adolescents demonstrated improvement in key concepts across gender expression, race, ethnicity, and other socio-economic factors.²⁴ All concept scores measured in this study (via the AOSL) improved except for Narcan knowledge and safe disposal. This effect remained whether the player had completed the game or not (67/117, 57.3%).²⁴

Parent and adolescent pairs were recruited to evaluate MedSMA℞T in another study. Results from 60 adolescents and 68 parents who met attention check requirements demonstrated similar effects on AOSL concept scores.²⁵ Attention checks are survey items which are used to tell if the participant is engaged, typically asking the participant to select a specific response to ensure they are reading questions. Adolescents improved in all concept scores except misuse harm, Narcan knowledge, and self-efficacy (MUSE). Parents improved in all concept scores except opioid knowledge, Narcan knowledge, and self-efficacy (MUSE). Similar to the previous study, the effect remained whether the player had completed the game or not (youth: 11/60, 18.3%, parents: 26/67, 38.8%).²⁵

Iterative Refinement and Implementation

Throughout the lifespan of MedSMA℞T, we have collected and characterized feedback from key stakeholders including adolescents, parents, pharmacists, and student pharmacists. These studies have allowed for adaptive improvement to the game design and mechanics, ensuring that key stakeholders find the game to be engaging, realistic, and worthwhile.

Student Pharmacists’ Perspectives

Student pharmacists were some of the first to play the game and offer their feedback. A study conducted as part of a pharmacy course required students to play the MedSMA℞T game for up to 30 minutes and participate in group discussions about their experience playing the game.²⁶ Student pharmacists held positive outlooks of the game’s graphics, appearance, and storylines. Students suggested that there was some difficulty in identifying level objectives, but the complexity of the game was appreciated, nonetheless. A key suggestion from this group was the inclusion of a recap dialogue at the end of the game to consolidate and reiterate educational points featured in the game.

Pharmacists’ Perspectives

The game was evaluated by pharmacists who suggested that the game was not only engaging and age-appropriate, but that the accurate information and attractiveness of it could make it applicable to implementation in community pharmacy practice.²⁷ Pharmacists suggested that the MedSMA℞T intervention could be beneficial for educating patients at greater depth without an outsized impact on pharmacy workflows.

Parents’ Perspectives

Parents who played the game and participated in individual, semi-structured interviews valued the MedSMA℞T intervention.²⁸ Parents responded positively towards the game’s scenes, characters, graphics, and educational content. Suggestions for improvement included simplifying navigation, increasing the pace of the game and the number of prompts and directions. Stakeholder engagement from parents was crucial since they are most often the primary medication managers in families with adolescents.

Adolescents’ Perspectives

Adolescents held a positive regard towards MedSMA℞T overall. They were able to clearly articulate the purpose of the game, and many told us they learned new, safer actions with opioids.²⁹ Youth found the characters to be engaging and appropriate albeit somewhat geared towards younger adolescents. Favorability was also supported by a significant number of suggestions from youth in this study that we expand the game to include more possible circumstances. Youth suggested a few improvements such as clarifying directions and more contemporary game controls.

Leveraging Data Science

The CRoME Lab IT staff developed an in-game data platform that tracks the use of the MedSMA℞T game from all players. The in-game data platform is an example of an in-game telemetry system which provides analytic data to understand how players are interacting with the MedSMA℞T game. In-game telemetry systems are data collection systems employed within games which allow for structured capture, processing, and analysis of players’ in-game behaviors.³⁰ Data collected is being used to inform future iterations of the game playable on mobile devices. Data from the open game data telemetry system is often leveraged by the game production team to record game telemetry and calculate basic analytic measures including the number of play sessions, the average time per session, and the greatest progression event per user for use during iterative prototyping. Data scientists in the CRoME lab utilize in-game data to identify features of gameplay of interest, develop analysis codes and data visualizations of gameplay. Examples of these data analyses include methods such as:

• Feature engineering to numerically describe attributes of a player’s experience

• Split testing to compare different versions of the game or different populations

• Replay of gameplay to reconstruct game experiences from telemetry events, supporting qualitative analysis

• Embedded and stealth assessments to measure player thinking

• Player typologies to determine “clusters” of similar players and play experiences

This innovative open infrastructure for collecting and processing game telemetry data for research and evaluation purposes is critical for observing uptake and reach in various regions of the state of Wisconsin. Collecting large amounts of in-game player data from multiple perspectives (youth, parents, and healthcare providers) will enable us to study and understand how people learn using educational health games. Our in-game data platform will allow us to leverage data science techniques to create meaningful insights out of large amounts of in-game player data and integrate and triangulate this with data obtained on actual medication safety behaviors, knowledge and attitudes collected through observations, interviews, and surveys. The in-game data platform will evolve to an open-source and community-maintained data storage and processing pipeline for educational game data. This platform will allow the CRoME Lab research team to build new theory while thinking about modularity, scalability, and performance.

The Personalized Family Medication Safety Plan (FMSP)

Development of the Family Medication Safety Plan

Following early playtests and community feedback, the FMSP was developed to help families review content and consolidate medication safety information. Many parents requested a follow-up to the game that they could use with their family. Whereas the youth were familiar with using a game to learn, parents wanted something that felt familiar to them. The first iterations of the FMSP (FMSP 1.0) were created and used in the form of an Excel spreadsheet where families could choose to either print and write-in answers or continue to use it as an online document (Figure 7). The contemporary iteration (FMSP 2.0) is delivered either through Excel spreadsheet or printed worksheet (Figures 8 and 9). FMSP 2.0 was designed for the express purpose of being printable for dissemination in the community pharmacy setting.

Figure 7.

FMSP 1.0 (Excel spreadsheet)

Figure 8.

FMSP 2.0 Page 1(Printable format)

Figure 9.

FMSP 2.0 Page 2(Printable format)

The FMSP was codesigned with youth, parents, pharmacists, and researchers. It is an education tool designed to foster open discussion about opioids and other medicines used by adolescents and other family members to consolidate important medication information and facilitate supportive safe family medication management. The FMSP is designed to prompt and facilitate important conversation about opioids and other medicines among adolescents, family members, and healthcare providers. There are five sections in the FMSP (both 1.0 and 2.0). The first section covers medication specific information such as interactions, potential side effects, and reason for use. Another section allows the family to create a schedule for taking each medication. Other sections asked families to fill in information related to storage, disposal, and who to contact for questions about the medications.

Pharmacists’ Perspectives

Pharmacists were shown the FMSP 1.0 and offered their feedback.³¹ Overall, the pharmacists in our study supported the use of the FMSP in their consultation practices while asserting that it would have little impact on their pharmacy’s workflow. They reported that the information included on the FMSP was accurate and the covered topics were important to opioid medication safety. They further reported that the FMSP touches on topics that are often omitted in consultation such as safe storage and disposal. Within families of adolescents, pharmacists stated that the parents may find the most educational benefit in the FMSP while still being a key resource during a medication-related emergency for the whole family. They suggest that this plan could be deployed alongside other medications than just opioids for a range of patient types. Pharmacists suggested that future iterations incorporate multiple modifiable formats to allow for increased use flexibility. Feedback suggested that implementation would require training and clarification on child, parent, and pharmacist roles in completing the FMSP.

Parents’ Perspectives

Parents evaluated the FMSP 1.0 for its potential use in their families’ medication management practices.³² Parents saw potential for the FMSP to be used when bringing home new medications or for family members who take multiple medications simultaneously. The content and sections were favorable to parents and represented information they suggested to be important. Parents also suggested that the FMSP could be a critical reminder when medication information is forgotten or needed in an emergency.

Adolescents’ Perspectives

Adolescents stated that the FMSP 1.0 was acceptable and covered important information that they do not typically know, such as storage and disposal.³³ Youth, however, suggested that the FMSP would be most beneficial to their parents (especially mothers) who were the primary medication managers in the home. They reiterated how the FMSP could be used for medications other than and alongside opioid prescriptions. Youth in this study expressed how a kid-friendly format and a Spanish translation version could improve the tool for future uptake.

The MedSMA℞T Families Program’s Current and Future Directions

Currently, the MedSMA℞T Families intervention is actively being used and evaluated for effectiveness within several health systems in Wisconsin. MedSMA℞T Families has been widely favored accepted for use within the pediatric and adult emergency department of one large health system in Wisconsin by ED staff, parents and youth prescribed opioid medications. In addition, the MedSMA℞T Families intervention has been widely deployed for use in community and retail pharmacies throughout the state of Wisconsin, with fourth year student pharmacists as key implementers during their rotations for Advanced Pharmacy Practice Experiences. Future iterative modifications will be made to the game elements as informed by key stakeholders. A key forthcoming adaptation will include mobile gameplay. Since the MedSMA℞T game is currently only available on computers, the CRoME team has partnered with game developers that are experts in creating mobile education games to enable easy and widespread gameplay on phones and tablets a reality, greatly expanding MedSMA℞T Families’ potential reach.

Preparation for additional studies is underway to evaluate the sustainability of this intervention in community pharmacies and emergency departments. Future studies will also evaluate the long-term effectiveness of this intervention in a variety of settings. Given the robust data already available, further study should include a randomized controlled trial to test the real-world effectiveness of our intervention.

The acceptability, attractiveness and potential of this intervention is underscored by the triangulation of perspectives from pharmacists, parents, and teens alongside empirical evidence of attitudinal and educational change. Future endeavors will continue to evaluate the game while taking into consideration implementation and long-term sustainment in a variety of community and clinical settings.

In summary, the overarching framework and process for building the MedSMA℞T Families intervention has been found to be effective and rewarding on multiple levels. Based on this significant previous experience we have developed a repeatable process for conceptualizing, designing, producing, and evaluating our games. The CRoME Lab is now adapting this effective approach to developing game-based family interventions for other public health issues that need immediate attention such as the vaping epidemic, mental health, and sleep challenges with teens, and improving cancer awareness among families. This will enable the CRoME lab to create learning games that tackle contemporary and critical health and medication use challenges with the overarching goal of improving the health and wellbeing of adolescents and their families nationally and globally.