An Active-Learning Laboratory on Immunizations

Abstract

Objective. To implement and evaluate an active-learning laboratory activity to teach pharmacy students about influenza, pneumococcal, and shingles vaccines.

Design. The laboratory session was divided into 6 immunization stations: 3 stations on influenza including a pediatrics station, and 1 station each for pneumococcal, shingles, and anaphylaxis.

Assessment. Although 118 of 123 (95.9%) students had completed an immunization training certificate prior to attending the laboratory, the average score on a pre-assessment to measure immunization knowledge and confidence was 56%. The post-assessment score was 87.4%. Students’ confidence improved by 18.7% to 51.2% in each of the 5 areas assessed. Most respondents rated the activity overall as good or excellent on a post-activity evaluation.

Conclusion. An active-learning approach to teaching immunizations allowed students to gain knowledge in simulated real-world experiences and reinforced key concepts on influenza, pneumococcal, and shingles vaccines.

INTRODUCTION

Pharmacists have become more involved with the delivery of immunizations, and state and federal governments have recognized the potential for pharmacists to further increase immunization rates.¹ In general, both the profession and the public have embraced the idea of pharmacists as immunizers.^1,2 All states, the District of Columbia, and Puerto Rico have authorized pharmacists to administer vaccines.³

In 1996, the American Pharmacists Association (APhA) created a Pharmacy-Based Immunization Delivery program that was endorsed by the Centers for Disease Control and Prevention.⁴ The program is an interactive certificate program where pharmacists or pharmacy students can learn the necessary skills to become a source for vaccine information and administration. The program consists of the completion of 3 key components: 12-hour self-study modules with case studies and an assessment examination, 8-hour live seminar with final examination, and hands-on assessment of intramuscular and subcutaneous injection techniques. The APhA has license partner agreements with 100 unique schools/campuses, which allows them to access certificate training program content (Bronhed Shaw, Associate Director of Education at APhA, written communication, August 2012).

The Accreditation Council for Pharmacy Education (ACPE) Standards 2007, Version 2.0, for programs leading to the doctor of pharmacy (PharmD) degree do not specifically address immunization training and instead refer to immunization education as a component of larger health promotion and disease prevention programs, which also include other topics such as tobacco cessation counseling and health screenings.^2,5 Immunizations are an important component of 2004 Center for the Advancement of Pharmaceutical Education Outcome 3 which is to “promote health improvement, wellness, and disease prevention in cooperation with patients, communities, at-risk populations, and other members of an interprofessional team of health care providers.”^2,5

At Virginia Commonwealth University School of Pharmacy, students are required to obtain their immunization certificate prior to beginning fourth-year advanced pharmacy practice experiences (APPEs). This may be accomplished through employer-sponsored programs or the APhA training program offered through local pharmacy organizations and the school. Like many other pharmacy schools, immunization training is not provided in the core curriculum.¹ However, Virginia Commonwealth University School of Pharmacy does offer the APhA training program outside of the core curriculum to interested students, but students must pay the required fee out of pocket.

When developing a method to provide additional immunization education and training, the course coordinators decided that the laboratory setting would be an excellent environment to provide hands-on training in a low-risk active-learning environment.⁶ Active learning encourages students to process and understand information, which results in increased retention of information as opposed to learning solely by rote memorization.⁷ Active-learning techniques may also help to ensure pharmacy students are able to “find, process, analyze and apply new information with their patients and their colleagues.”⁸ The key to successful active learning is actively engage students in the classroom, and this is facilitated by using small groups, which encourage student discussion and participation.⁷ By allowing students to apply their knowledge to simulated patient cases, students are able to increase their confidence before assisting actual patients.^5,6 This style of teaching is consistent with the ACPE standards which encourage faculty members to use active-learning strategies wherever possible to bridge the gap between the classroom and providing direct patient care.⁶ The specific approach and assessment methods used for this study as well as outcomes are described.

DESIGN

The active-learning laboratory included 126 third-year pharmacy students enrolled in the pharmacy practice skills laboratory during the fall semester. The skills laboratory is a 1- credit course intended to provide third-year pharmacy students with opportunities to improve acquired skills and gain additional skills necessary to be a competent, caring pharmacist. It is the fifth course in a 6-semester practice-based laboratory course sequence. The students were divided into 2-hour laboratory sections offered 4 times per week with 1 common hour or lecture time once weekly. Each of the 3 laboratory sections on campus had about 36 students, and the laboratory section at the distance learning site had 18 students. The goal of the immunization laboratory was to increase pharmacy students’ competency and confidence in key areas related to immunizations. The specific objectives for this laboratory were for the pharmacy student to be able to: (1) define indications and contraindications for the following vaccines: influenza, pneumococcal, and shingles; (2) recommend appropriate vaccination(s) based on given patient information; (3) counsel a patient on the risks and benefits of a given immunization; (4) describe appropriate reconstitution, administration, and documentation of a given immunization; and (5) recommend appropriate treatment of a patient experiencing anaphylaxis.

The 2 course coordinators and the postgraduate year 1 (PGY-1) ambulatory care resident at the school designed and implemented the immunization station active-learning laboratory. Faculty members involved in facilitating the activity included the course coordinators, an additional laboratory faculty member, PGY1 residents, an APPE student, and graduate teaching assistants for the course.

The faculty members reviewed the literature regarding teaching and learning methods as they related to immunization education. Although not all pharmacists will administer vaccinations, they still must be able to educate and counsel patients on key information regarding immunizations. The laboratory activity was divided into 6 immunization stations: influenza (intramuscular and intradermal) vaccines; nasal influenza vaccine (FluMist); pneumococcal vaccines; shingles vaccine; pediatric vaccines (influenza and pneumococcal); and a managing anaphylactic reactions station. Influenza, pneumococcal, and shingles vaccines were chosen for this laboratory exercise as these are immunizations commonly administered by pharmacists in the community. Each station was carefully designed to incorporate specific scenarios that pharmacists may be faced with in practice.

A pre-assessment that included knowledge-based immunization multiple-choice questions, a confidence survey instrument, and questions regarding immunization training status was administered 1 week before the active-learning laboratory. Students had no advanced knowledge of this assessment and therefore answered questions solely based on their current knowledge.

A 50-minute pre-laboratory lecture providing an overall review of immunizations was delivered the Friday prior to the active-learning laboratory sessions during the regularly scheduled lecture time. After completing the pre-assessment but prior to participating in the active-learning laboratory session, students were asked to review the Vaccine Information Statement (VIS) forms for influenza, herpes zoster, and pneumococcal vaccines.

On the day of the active-learning laboratory session, approximately 10 minutes were spent discussing the logistics of the laboratory, explaining the objectives for the activity, and answering questions. Students were divided into groups of 5 or 6 and rotated from one station to the next approximately every 15 minutes. Each station was led by a faculty member, pharmacy resident, APPE student, or graduate teaching assistant who was provided with specific instructions for conducting the exercise prior to the laboratory session.

Each station consisted of various vaccine products and packaging, completed vaccine waivers, and corresponding worksheet questions. The faculty member, resident, APPE student, or teaching assistant simulated 2 different patients at each station who were presenting to a pharmacy with a completed waiver form to receive a particular vaccination. Students were asked to determine the eligibility of the 2 simulated patients and the appropriateness of the prescribed vaccine. At each station, students were shown a copy of the patient’s vaccine waiver on a computer monitor, or if they preferred, they could view a paper copy. Students were then allowed to ask the patient relevant questions. The patients were also allowed to ask the pharmacy students questions pertaining to the vaccine prescribed. Some questions on the vaccine waiver form were left blank or filled in inappropriately to prompt the students to clarify these points with the patient. At every station, indications and contraindications for the vaccines were reviewed. If the vaccine was indicated, the students had to properly document all required information during the administration process on their worksheet and then counsel the patient on the immunization. Additional key teaching points for each of the 6 stations were also covered on the students’ worksheet (Table 1).

Table 1.

Information Presented at Immunization Stations as Part of a Pharmacy Active-Learning Laboratory

EVALUATION AND ASSESSMENT

This study was approved as exempt research by the Institutional Review Board of Virginia Commonwealth University. The knowledge-based portion of the pre- and post-assessment consisted of 14 multiple-choice and matching questions related directly to the objectives of the activity. The confidence portion of the survey instrument consisted of items on which students rated their level of confidence in 5 areas of immunization practice. The students ranked their confidence level on a Likert scale on which 1 = completely confident, 2 = very confident, 3 = somewhat confident, 4 = not very confident, and 5 = not at all confident.

One week after completing the laboratory session, students were given a post-assessment with the same knowledge-based questions that were asked on the pre-assessment. Students were aware in advance of this assessment and completion of the assessment was required as part of their course grade. Four additional multiple-choice questions were added to the post-assessment to determine student participation in immunization activities thus far in their careers. This helped to determine in what types of activities related to immunizations the students had participated, how many patients they had immunized, and where and when they completed their immunization certificate training.

Performance on the knowledge-based questions was described using the percent of students answering each item correctly. The mean (standard deviation) and median (interquartile range) were also calculated for students’ overall score (or total percent correct) for knowledge-based questions. Students’ confidence before and after the laboratory session was described from both a continuous perspective (ie, mean and standard deviation) and a categorical perspective (ie, those responding “confident,” “very confident,” or “completely confident”). Because a student’s responses before and after the laboratory session could be linked by his/her name in Blackboard, statistical comparisons were conducted using a matched approach. Results were de-identified after matching occurred. McNemar’s test was used to compare whether a student answered an individual item correctly before and after the session. Only 9 of the 14 knowledge questions were included in the individual item analysis as the other 5 questions were matching and were unable to be easily separated based on computerized responses. A paired t test was used to compare the overall knowledge score (ie, % correct across all knowledge questions) before and after the activity. To allow for potentially non-normally distributed data, the overall knowledge score was also examined using a Wilcoxon rank sum test. Similar approaches were used for student confidence whereby McNemar’s test was used to examine confidence from a categorical perspective and a paired t test and Wilcoxon rank sum test were used for confidence from a continuous perspective. All statistical analyses were conducted using Stata/SE, release 12 (StataCorp LP, College Station, TX), with p ≤ 0.05 denoting significance.

One hundred twenty-five students completed the pre-assessment and 124 (98.4%) completed the post-assessment. The average score on the 14 knowledge-based questions on the pre-assessment was 56% (14.3%-100%) and on the post-assessment was 87.4% (50%-100%). Scores improved for 95% of students, with 4 students scoring the same and 2 students scoring lower on the post-assessment. The greatest improvement in scores between pre- and post-assessment was 78.6%.

The individual comparison of questions 1-9 is reported in Table 2. Questions 10-14 dealt with matching a vaccine to the appropriate age group, assuming the patients are otherwise healthy with no comorbid conditions. There was a positive shift in each of these 5 questions from pre- to post-assessment. A statistical analysis of these questions was not performed due to the formatting of Blackboard questions (matching). These 5 questions focused on newer influenza products, pneumococcal, and shingles vaccines.

Table 2.

Pharmacy Students’ Knowledge of Immunizations Assessed Before and After Completing an Active-Learning Laboratory, N=103

Students’ confidence improved in all of the 5 areas assessed. The 2 areas on the post-assessment in which students reported the greatest level of confidence were in their ability to appropriately screen and determine eligibility and appropriateness for a patient to receive the influenza vaccine (91.1%), and in their ability to document administration of a vaccine in general (95.1%). The most significant improvement in confidence was in their ability to appropriately screen and determine eligibility and appropriateness of a patient to receive the pneumococcal vaccine (+51.2%). Confidence data are reported in Table 3.

Table 3.

Students’ Confidence in Immunizations Before and After Completing an Active-Learning Laboratory, N=103

Approximately 96% of students had completed an immunization certificate program. The majority (51.2%) had completed the program through the school of pharmacy program. Employer-sponsored programs (31.7%) represented the second most popular method of receiving training. Sixty-nine percent of students completed this training within 6 months of the survey, while about one-quarter had completed training more than 12 months before the survey (Table 4).

Table 4.

Completion of Immunization Certificate Training Program

To assess how students were using their immunization training skills, the survey instrument asked questions about the types of activities in which the students had participated and the number of patients the students had personally immunized. Activities could include patient education, patient screening, and administration. Of those students who had completed certificate training, 22% had not participated in any activities related to immunizations, while approximately 60% had administered immunizations. However, 45% of all respondents reported they had not personally immunized any patients. Most students who had personally immunized patients, had immunized between 1 and 20 patients. Two students, both of whom had completed the certificate program more than 12 months prior to the survey, reported personally immunizing more than 100 patients.

Students were also asked to complete an evaluation of the laboratory session. This evaluation consisted of 8 questions: 5 questions using a Likert scale on which 1 = strongly disagree, 2 = disagree, 3 = undecided, 4 = agree, and 5 = strongly agree; 1 question regarding the overall laboratory rating using a Likert scale with 1 = very poor, 2 = poor, 3 = fair, 4 = good, and 5 = excellent; and 2 open-response questions for students to comment on what they liked and did not like about the laboratory activity. Students were asked to rate items such as organization, relevance, appropriateness, contribution to professional development, and satisfaction.

Sixty-three (51%) students completed the evaluation. The majority felt the laboratory activity was well-organized, presented at an appropriate level, and relevant to pharmacy practice. Respondents also felt it contributed to their professional development and 87.3% rated the overall activity as good or excellent. When students were asked what they liked about the activity, they said they appreciated that the most common immunizations were included and they appreciated the practicality of the activity, including the patient cases and product demonstrations. They thought it was a helpful refresher in a low-stress format with knowledgeable facilitators. Students suggested that the activity include more practice administering injections. When discussing what they did not like about the activity, some pharmacy students stated that they felt rushed and that there was some redundancy across the laboratory stations.

DISCUSSION

Implementation of an active-learning laboratory focused on immunizations significantly increased pharmacy students’ knowledge and confidence in this area. Although almost 96% of students had completed an immunization certificate program prior to the laboratory activity, the average score on the pre-assessment knowledge-based questions was 56%. Also, nearly 50% of students had never personally immunized a patient. The difference in student immunization certificate training versus the actual number of patients immunized represented a void in experience and a lack of preparation prior to entering their fourth-year APPEs. Students are introduced to pharmacists as immunizers in their first-year health promotion course. The fall semester of the third year was chosen as an appropriate time to reinforce this content because it coincided with the beginning of influenza season and was immediately prior to students beginning APPEs.

The laboratory activity was designed to assess students’ prior knowledge, expand current understanding, evaluate patient case scenarios, and reassess students’ knowledge after the activity. This design allowed for analysis of differences in pre-activity and post-activity knowledge. The pre-assessment was intentionally unannounced so that students were assessed on only their current knowledge. This also gave students an opportunity for self-assessment prior to participating in the activity. The level of content covered in the immunization review the week before the small group laboratory sessions was considered suitable because of the high number of students who had completed immunization certificate training. Students with a different level of immunization training may need a more comprehensive presentation prior to participating in the activity.

The activity was designed to increase the level of engagement between facilitator and students. Students remarked that having realistic patient cases made the activity more comprehensive and relevant to pharmacy practice. Based on student and facilitator feedback, availability of demonstration products was important to application of information at each station. Although each station followed a standard format of screening 2 patients and then documenting information and counseling the patient about the vaccination, the time allotted for each step of the process at each of the stations could be improved. Decreasing the redundancy across stations – especially the influenza stations – could provide an opportunity to learn about other types of vaccinations. Even though standardized scripts for each of the stations were provided, the limitations of using graduate teaching assistants without experience working as pharmacists in the United States became apparent when students asked the teaching assistants questions for which answers were not provided on the scripts. A goal for future implementations is to increase the number of pharmacy faculty members and residents serving as facilitators during the laboratory activity.

The knowledge-based questions on the pre- and post-assessments were reviewed by a pharmacist who delivers the APhA Pharmacy-Based Immunization Delivery program. The pharmacist stated that the assessment questions were similar to the learning objectives covered in the program and were written at a similar level of Bloom’s taxonomy. Depending on when the students completed the immunization training program, they may not have been familiar with newer products, such as Fluzone Intradermal and Fluzone High-Dose. This may have impacted student’s scores on these questions. The students’ lowest performance on the post-assessment was on the question related to the pediatric pneumococcal vaccine dosing schedule. This may indicate a need to increase the emphasis of this topic at the pediatric immunization station.

Comparing pre-assessment performance to post-assessment performance demonstrates that the use of active-learning in a small group setting was an effective design strategy. Despite that almost 96% of students had completed an immunization certificate program, there was a significant increase in levels of confidence following participation in the laboratory activity. This suggests that even though students are certified to provide immunizations, many do not have confidence in doing so.

Students’ evaluations of the laboratory activity were generally favorable. The reasons why the response rate for the laboratory evaluation was lower than that for the pre- and post-assessments probably were because completion was voluntary, it was conducted outside of class time, and students had a limited timeframe to participate. To increase response rates in the future, it may be prudent to include the student laboratory evaluation within the post-assessment administered during class time. One limitation to this activity is that it does not include actual practice in administering immunizations. Faculty members discussed this topic when developing the activity and the decision was made not to include practice performing subcutaneous and intramuscular injections. The main reason was to avoid promoting a false sense of security for those students who had not yet completed the immunization certificate training program

Test-retest bias is a concern with pre- and post-assessments as students know they will receive a post-assessment, and for this assessment, the same questions were used for both administrations. Also, the ability to document immunizations and knowledge of managing anaphylactic reactions were not assessed in the knowledge-based questions on either assessment. Because the assessments were administered on Blackboard, individual question data were not available for the 5 matching knowledge-based questions. Pre- and post-assessment data could not be compared for 23 students because of incomplete data.

Confidence levels were rated as completely, very, somewhat, not very, or not at all confident based on a 5-point Likert scale. However, definitions of the meaning of each confidence level were not explicitly stated and were therefore open to student interpretation. In the future, it may be helpful to define each of the 5 levels of confidence. Also, the survey instruments used were not validated; validation should be considered for future administrations. Student confidence was not compared to performance on the post-assessment knowledge-based questions. This is a potential point for data comparison in the future.

Plans for expanding the laboratory include combining the influenza stations to reduce redundancy and the addition of tetanus-diphtheria and hepatitis B stations. Also, in the future, the Vaccine Information System that tracks which vaccines patients have received will be discussed to familiarize students with this resource.

SUMMARY

An active-learning laboratory activity to teach pharmacy students about common vaccines was implemented and evaluated. Although 96% of students had completed an immunization certificate program prior to the active-learning laboratory, their overall assessment scores and confidence levels improved significantly after participating in the activity. This suggests that even though students are certified to provide immunizations, many students could benefit from additional training to help develop knowledge and confidence. An active-learning approach to teaching immunizations allowed students to gain that knowledge and confidence in simulated real-world experiences and reinforced key concepts on influenza, pneumococcal, and shingles vaccines.