Development and Pilot Testing of the OTC Coach Software to Support Student Pharmacist Learning

Abstract

Description of the problem: Pharmacy students are expected to learn how condition and patient-specific factors influence medication decision-making. Our objective was to create an interactive learning tool that would support students as they learn how individual factors change over-the-counter (OTC) medication recommendations.

Description of the innovation: OTC Coach was created to allow student pharmacists to practice making recommendations about OTC medications. First year student pharmacists enrolled in a required self-care therapeutics course were given access to the optional OTC Coach, which included electronic decision algorithms for 10 topics. Student perceptions were collected via an online survey.

Critical analysis: Two-thirds of the first-year students enrolled in the self-care therapeutics course activated their OTC Coach account (n=53/79, 67%). Among the students who completed the survey and reported using the tool (n=60/75, 80%), there was agreement that it help them learn the course material (78%), increased their confidence in making appropriate therapeutic recommendations (78%), increased their confidence when answering examination questions (63%), and improved their examination performance (61%).

Next steps: Implementing an electronic tool positively supported student learning about OTC medication recommendations, according to student self-report. The tool is being expanded to include an option for students to generate a series of scenarios with randomized patient and condition factors to further allow students to efficiently practice making repeated patient-centered recommendations.

Description of the Problem

The Pharmacists’ Patient Care Process (PPCP) provides a framework to support student pharmacists as they learn the skills necessary to make over-the-counter (OTC) recommendations.¹ Embedded within the PPCP are the steps of therapeutic decision making: gather information (collect), appraise the data (assess), make recommendations including by analyzing benefits and risks of different treatment options (plan), and enact and monitor the decision (implement and follow-up).^1-4 However, this is a general process applicable across practice settings and it does not provide detailed guidance about what information is critical to collect and assess prior to making an OTC recommendation.⁵ Given there are more than 300,000 OTC medications marketed in the United States, including products in more than 80 therapeutic classes, it is critical that student pharmacists learn how to make patient-centered OTC medication recommendations.^6,7

Virtual simulations has been shown to increase knowledge and skills while being enjoyable to the student.^8-14 Existing virtual simulation programs for students pharmacists that focus on teaching decision-making are typically designed for students to engage with one simulated patient for the duration of the exercise.^{15, 16} However, this makes it difficult for students to learn how different combinations of patient and medication-factors influence treatment recommendations. As a result, it could be helpful for students to be able to practice making many decisions with only small changes in the patient and conditions inputs in a short period of time so that students could see how these changes impacted the appropriateness of OTC medication recommendations. We sought to support student pharmacists as they learned to make therapeutic decisions by creating a tool, OTC Coach, which allowed them to systematically consider both patient- and condition-related factors to make evidence-based OTC treatment decisions. The aim of this paper is to describe the development of the OTC Coach and student perceptions of the use of the tool.

Description of the Innovation

Development of the OTC Coach

The OTC Coach is an electronic tool that allowed students to practice making patient-centered recommendations about OTC medications for 10 common health conditions: allergies, atopic dermatitis, constipation, diarrhea, fever, fungal skin infections, headaches, heartburn, insect bites and stings, and ophthalmic disorders. We developed the tool between January and December 2021 using grant funding from the American Association of Colleges of Pharmacy Scholarship of Teaching and Learning Grant and in-kind support from our institution.

As described by Alinier, our tool has the criteria of a “level 2” simulation as it is an interactive computer simulation that allows students to self-learn cognitive skills, including via feedback, but occurs outside of a realistic setting.¹⁷ This level of simulation aligns well with the needs of first-year student pharmacists who may benefit from more complex activities that can be offered via paper-based cases (level 0) but have not developed the fundamental knowledge or skills to effectively engage in more complex simulations (e.g., level 5, high-fidelity simulations).¹⁷

Decision trees were created for each therapeutic topic as a written document. For each topic, patient-related characteristics (e.g., age, allergies, concurrent medical conditions) and condition-related characteristics (e.g., severity and duration of symptoms) were identified that would influence whether a patient was a candidate for self-treatment of their minor ailment, and if so, which medications would be effective and safe. Each decision tree was customized for the specific topic, resulting in different questions and response options in each algorithm. As an example, the duration of symptoms was measured in days and weeks for pain and allergic rhinitis, respectively. Students were asked to predict whether the patient was a candidate for self-treatment. The output was customized to specify whether the student correctly identified that the patient could self-treat or needed a referral to another health care professional. In addition, if the patient was a candidate for self-treatment, a list of medications or therapeutic classes was presented along with a statement about whether each one was appropriate based on the specific inputs. In the output, medications were summarized by therapeutic class unless there was a compelling reason to list individual medications (e.g., different minimum age thresholds for non-steroidal anti-inflammatory drugs). Through this process, information was integrated from sources such as the OTC medication package labeling approved by the Food and Drug Administration, a textbook, clinical guidelines, and clinical experience.^18-20 The software developer (KD) subsequently converted each algorithm into an electronic format using Shiny Applications (Figure 1).²¹ The faculty member, student research assistant, and software developer extensively tested each algorithm to ensure that it matched the original written document and that the output aligned with clinical practice recommendations.

Figure 1.

Example OTC Coach Inputs and Outputs for the Treatment of a Fever

Use of the OTC Coach

First-year students were introduced to the OTC Coach during a class session about headaches, and they subsequently had 24 hour a day, seven day a week access to this website on their computers, tablets, and phones, which allowed convenience of use. The tool was demonstrated and its utility in studying was discussed. Students could repeatedly modify the combination of factors in a short period of time and receive immediate feedback about the appropriateness of common approaches to self-treatment for minor ailments. For example, they could create a virtual adult patient with no health conditions who is experiencing a tension headache with pain of four out of 10 for one day and learn that either acetaminophen or non-steroidal anti-inflammatory drugs (NSAIDs) were acceptable treatment options. Subsequently, they could keep the same patient except add that the patient takes medications for hypertension, which would result in acetaminophen being the recommended medication. If they were to keep the same patient, but increase the pain to eight out of 10, the recommendation would be a medical referral. A student would be able to learn from all three of these patients in under five minutes. Several weeks later, the tool was demonstrated a second time using the topic of fevers. Subsequently, students were periodically reminded of the tool throughout the remainder of the semester. The use of the OTC Coach was optional.

Data collection and analysis

Near the end of the semester, students were asked to complete an online survey via Qualtrics (Qualtrics, LLC, Provo, UT) about their experiences either using the OTC Coach or to provide information about why they decided not to use the tool. Students were provided a 5 bonus points (equivalent to 0.1% of their overall course grade) for completing the survey and could opt out of having their data included in the research study and still receive the bonus points. Students were asked the number of times they accessed the tool. Students who had not used the tool were asked to describe why (short answer response). OTC Coach users were asked average time spent using the tool (sliding scale from 0-180 minutes) and level of agreement with items related to ease of use, availability, whether the tool included relevant patient-related factors, condition-related factors, and effects of use (6-point Likert scale with scale anchors 1=strongly disagree and 6=strongly agree). Students were then asked to provide feedback about aspects of the tool that worked well, as well as potential areas for improvement. Descriptive statistics were used to summarize the quantitative data with dichotomized results for several variables (disagree = 1, 2, and 3; agree = 4, 5, and 6) for ease of interpretation using Stata version Stata SE 17.0 (StataCorp). Finally, the senior author (SV) reviewed and categorized the student’s comments. The University of Michigan Health Sciences and Behavioral Sciences Institutional Review Board reviewed this study and deemed it exempt.

Findings

Two-thirds of the first-year students enrolled in the self-care therapeutics course activated their OTC Coach account (n=53/79, 67%). Most students (n=76/79, 96%) responded to the survey, of which 75 students (95%) consented to having their data being used for research purposes. In the survey, 15 out of 75 students (20%) reported not using the OTC Coach. The reasons described included feeling like they already had enough study tools or had established study habits (n=9, 60%), lack of time (n=3, 20%), technical issues (n=3, 20%), or forgetting about the tool (n=2, 13%).

A total of 60 out of 75 (80%) students reported using the tool in the survey. Most students (n=48/60, 80%) reporting using it between 1 and 4 times during the semester while 4 students (7%) reported using it 11 or more times. The majority of students (n=44/60, 73%) reported using the tool for 10-30 minutes in a single session while 12 students (20%) reported using it for 31 minutes or more and 4 students (7%) used it for 5 minutes or less. Students agreed that the tool included relevant patient-related factors (n=57/60, 95%) and condition-related factors (n= 59/60, 98%) that influence treatment recommendations. Figure 2 provides additional perceptions.

Figure 2.

Perceptions of the Impact of the OTC Coach on Learning Among Students who Reported Using the Tool (n=60)^a

Figure 2. Perceptions of the Impact of the OTC Coach on Learning Among Students who Reported Using the Tool (n=60)^a

In response to an open-ended question, 52 students provided feedback about the aspects of the tool that worked well. Students reported liking the ability to quickly change patient and clinical factors (n=24/52, 46%) and the simplicity of the tool (n=18/52, 35%). Several students (n=5/52, 10%) reported that the list of reasons why a patient should be referred to a health care provider that was embedded in the data input screen was helpful. A total of 52 students also provided suggestions for improving the tool, including enhancing the interface (n=13/52, 25%) by adding the ability to save cases, add images, include more factors that could influence the treatment recommendation, and add an explanation of why the patient should or should not self-treat their symptoms. Several students (n=10/52, 19%) recommended adding additional topics or conditions.

Critical Analysis

Computer simulation is common in health professions education as it allows adult learners to engage in active, deliberative practice with focused feedback that promotes individualized learning in a safe environment. The OTC Coach was successful in allowing first-year student pharmacists to explore how patient- and condition-related factors influenced OTC medication treatment decisions.

There are several other virtual tools designed to help pharmacy students develop critical thinking and patient care skills. For example, MyDispense allows students to practice tasks in a community pharmacy setting such as verifying prescriptions, filling a prescription order, and making an over-the-counter recommendation. However, individual activities can take up to 60 minutes depending on the complexity of the activity.¹⁴ DecisionSim / vpSim uses a branched-narrative model in which the user makes a series of decisions for one specific patient and along the way receives feedback by learning the consequences of their choices.^8,13 EHRGo is an education electronic health record that helps students learn skills such as medication ordering and verification while also hearing the patient’s perspective in the process.²² The OTC Coach is distinct from these other virtual simulation tools in that it provides an opportunity for students to practice making many decisions for one therapeutic topic for many patients in a short period of time and consequently identify how small changes in patient presentation can influence the treatment plan.

Next Steps

Some students reported that the tool was helpful; however, they also acknowledged that comprehensive, standardized study materials were already provided to students for every therapeutic topic. Additionally, the OTC Coach focused on helping students with the medication decision-making process but did not include all content that was included on exams (e.g., dosing, potential side effects, counseling points, and monitoring plans). Based on student feedback, the tool is being expanded to include an option for students to generate a series of multiple-choice questions that includes a broader range of content for each therapeutic topic. Each formative quiz will include approximately 10 one-sentence scenarios with randomized patient and condition specific factors and several potential treatment options. Students will make a treatment decision for each scenario and then receive immediate feedback about their decisions. This will further allow students to efficiently practice making repeated patient-centered recommendations