Impact of Novel Active-Learning Approaches Through iBooks and Gamification in a Reformatted Pharmacy Course

Abstract

Objective. To compare pharmacy students’ assessment and evaluation ratings of a newly formatted nonprescription products course that used innovative technology and gaming to that for a traditional nonprescription products course.

Methods. Examination scores and course evaluations of students who completed the traditional course on nonprescription products were analyzed and compared with those of students who completed a revised course on nonprescription products that used teaching techniques in educational technology.

Results. Students in the traditional course significantly outperformed those in the comparator group in eight nonprescription categories on examinations, while students in the new course significantly outperformed those in the comparator group in four categories. Students in the new course outperformed those in the traditional course in two additional examination categories, however these were not significant. Almost all students in both courses agreed or strongly agreed with each course evaluation item, including those items where use of gamification and iBooks were most likely considered.

Conclusion. Improvements in student examination performance and course satisfaction may not be immediately seen in a course in which new teaching techniques using educational technology including gamification and iBooks are introduced.

INTRODUCTION

With advances in technology and science, there is now a plethora of medical knowledge and information that must be sorted, analyzed, and synthesized by health professions students. This rapid increase in the volume of data, coupled with rising enrollments across colleges and schools of pharmacy, poses new challenges to educators. As a result, there have been many calls for educational reform, including from organizations such as the Carnegie Foundation for the Advancement of Teaching and the American Association of Colleges of Pharmacy (AACP).^1,2

Discussions regarding curricular reform are consistently associated with themes centered on moving away from traditional pedagogical approaches that are primarily passive, content centered, and lecture based, toward new approaches that are innovative, student centered, and active. Additionally, the traditional style of teaching and learning (eg, lecturing) has been heavily criticized for being too passive, potentially ineffective, and often boring.^3-5Approaches to pedagogy involving active learning have been proposed as a means to modify and improve traditional and sometimes less effective approaches to teaching and learning. Active-learning pedagogic approaches have been associated with learning efficacy and student satisfaction in both pre- and postsecondary educational environments.^6-8 Moreover, approaches that are advantageous, effective, and efficient are those having strategies that build on “offloaded” content, use case vignettes, and use team-based learning.^8-12

Complementary to the rise in popularity of novel active-learning approaches, several new educational technologies have emerged. The 2013-2014 AACP Academic Affairs Committee was charged with examining the role of games in pharmacy education and investigating which areas of curricular games might offer the greatest potential impact.¹³ The Committee endorsed the use of serious games in pharmacy education, advocated for faculty and student innovation in designing and/or implementing serious games in order to prepare future health care leaders, and provided a resource outlining existing serious games potentially useful in pharmacy and patient education because of their perceived educational and other benefits. The committee also suggested that colleges and schools of pharmacy encourage faculty members and students to consider the use of serious games for learning and professional development.¹³

Gamification has been described as the process of adding game-like elements to something (eg, learning activities) so as to encourage participation.⁹ “Serious gaming” is the use of game principles for the purposes of learning, skill acquisition, and training.¹⁴ Other descriptions and potential advantages of gamification have been described in varied ways in both the medical and general education literature. In medical curricula, educational games, medical mobile applications, and virtual patient simulations are all considered gamified training platforms. While rigorous studies that demonstrate improvement in learning are limited in this area, McCoy and colleagues reported that gamified training platforms such as virtual patient simulations have demonstrated improved learning outcomes.⁹ The authors also determined that games demonstrated the potential to promote learning, increase engagement, allow for real-world application, enhance collaboration, and provide opportunities for risk-free clinical decision making and swift feedback.⁹

Another teaching method that uses educational technology to optimize active learning is the use of electronic books (eBooks) such as iBooks (Apple Inc., Cupertino, CA). Electronic books may be used to facilitate content delivery and provide formative feedback. Apple’s proprietary ebook software, iBooks, has gained popularity both in the private sector and within educational systems. The iBooks features are optimized only for Apple devices. In the educational sector, iBooks is a technology that allows students to explore, learn, and experiment with concepts through use of multi-touch learning materials such as photos, text, video, and interactive widgets. Through these features, which bring content to life in ways printed pages cannot, iBooks can be extremely engaging for students while enhancing their learning and optimizing assessment.

At the University of Kentucky College of Pharmacy (UKCOP), a revised curriculum which launched in fall 2016, aimed to incorporate advanced active-learning approaches with an emphasis on revised Accreditation Council for Pharmacy Education (ACPE) standards and other recommendations made by the Academy.¹⁵ In the first professional year of the traditional (former) curriculum, a two-credit hour course containing content on nonprescription products had incorporated some active-learning techniques such as team-based learning (TBL). However, there were still passive aspects of the course that faculty members believed could be significantly improved with technology. Some examples of passive activities include reading for preclass assignments and listening to mini-lectures during in-class activities. Little effort was made in the existing course to test in real time (immediately after completion) students’ recall or evaluate their comprehension of the information they received. While games were only used occasionally in the traditional course, students reported through course evaluations that they enjoyed and desired more use of games for learning and content review. Students remarked that games should be structured and recurrent when used. Further, students in the traditional course often reported on course evlatuations that time in class should be better spent.

To address some of the aforementioned student concerns, in the first professional year of the new curriculum, a three-credit hour course was created that contained nonprescription products content (Wellness & Health Promotion I) and employed new teaching methods, including use of gamification and iBooks. These teaching methods were designed to address some of the needs for consistent active learning and access to formative assessments and tools that aid in reading comprehension and examination preparation, and to provide opportunities to enhance student engagement and classroom efficiencies. The iBooks software and games can facilitate efficient content delivery, be used for application and review, and provide engagement and immediate feedback for students. Subsequent aims associated with these modifications were to enhance student learning by improving comprehension, frequency of content review, and mobile accessibility to learning aids. Additionally, these modifications were intended to foster increased student engagement by providing course content in an interesting yet structured format that maximized time spent on assignments at home and in class.

The primary objective of this study was to investigate aspects of student assessment and evaluation of a new nonprescription product pharmacy course that used gamification and iBooks to aid in content delivery, content application, and content review in preparation for examinations, and to compare these aspects to those of the traditional nonprescription product pharmacy course previously taught that did not use these teaching techniques.

METHODS

To evaluate a new pharmacy course that used gamification and iBooks, we compared the level of nonprescription content knowledge on assessments of students in both the former and new courses. More specifically, we compared class performance on nonprescription examination questions for the traditional course to class performance on the same nonprescription examination questions used in the new course. Final course grades for each course were based on students’ weighted scores on examinations, individual and group quizzes, and projects. Additionally, to evaluate and compare student perceptions and satisfaction in a course that used gamification and iBooks, the same student evaluation metrics for each class in the traditional and new courses were compared. The COP classes of 2019 and 2020, enrolled in the professional program during the fall 2015, spring 2016, or fall 2016 semesters, were included in the study. Students meeting the following recruitment criteria were included: a student completing at least one semester of the traditionally taught Nonprescription Products & Supplies I or II, or the modified Wellness & Health Promotion I course. This study was approved by the University of Kentucky Institutional Review Board.

In this retrospective study, students’ grades on existing routine course assessments and students’ demographic data were obtained. Specifically, data retrieved from ExamSoft (ExamSoft Worldwide, Inc, Dallas, TX), CoursEval (Campus Labs, Buffalo, NY), faculty course files, and College SharePoint (Microsoft Corp, Redmond, WA) electronic databases were entered into an Excel spreadsheet. All data retrieved, including student scores, were de-identified.

In the nonprescription course sequence in the former curriculum, content was taught using a fairly equal hybrid of traditional (non-active-learning) and nontraditional (active-learning) teaching techniques (Appendix 1). In these courses, pre-assigned textbook readings, in-class quizzes, 20-30 minute mini-lectures, and TBL were employed to deliver and instruct content. In the reformatted nonprescription course, new teaching methods were added to some former approaches (ie, preassigned reading assignments, in-class quizzes, and TBL). Gamification and iBooks were employed to facilitate course instruction, content learning and review, and student engagement (Appendix 2). As a component of the new doctor of pharmacy (PharmD) curriculum, an Apple iPad was provided to every student. The provision of these devices facilitates and readily enables the inclusion of various technologic classroom modalities and activities such as those added to the new nonprescription course.

On most days of the 16-week course, the class was subdivided into four large sections of approximately 36 students each. For 90 minutes, each section met separately and was further divided into six small teams of four to six students for gaming activities. Gaming activities included student groups competing with one another in team challenges such as Name That Drug, active scavenger-hunt puzzles, and Nonprescription Jeopardy. One bonus point to an unannounced, intermittent, 50-point participation score was occasionally awarded to the winning team for that class period. The students’ iPads were used to facilitate the execution of games. Most of the games used in the course were later uploaded to the course’s iBooks to provide an avenue for students to review them as desired.

The iBooks were created by the course director and wirelessly pushed (transferred) to students’ iPads once per quarter during the 16-week semester. Each iBook was organized by the nonprescription products and wellness and health promotion topics taught during that four-week period. Each iBook was further subdivided into weekly segments that highlighted the topics to be covered during that time period. The course’s preassigned weekly readings, reading objectives, and reading introductions also were published in the iBooks. All iBooks included interactive text, photos, PowerPoint slides, videos, and widgets that reinforced concepts and content found within the weekly readings. In order to guide weekly readings, the interactive text would introduce or summarize a given content area. Non-interactive text was used to describe patient cases that were covered in a large group setting during the other weekly, in-class discussion periods using the TBL format. The iBook videos created for the course provided 3-D active demonstrations of mechanisms of pathology. Some videos also provided an enhanced audiovisual version of the weekly patient case that was otherwise provided in a text-only format. Finally, widgets embedded in the electronic book provided the students with interactive teaching material and games (eg, hangman, bingo, memory games, jigsaw puzzles, word searches) that tested and reviewed student knowledge. No credit was awarded for completing widgets, as they were primarily for independent student review. However, they provided an opportunity for deeper student engagement and immediate formative feedback.

Investigators reviewed and analyzed the following data for each student who had completed either the traditionally formatted course or the revised course: course examination scores, final course scores with associated final course grades, and anonymous student evaluation ratings. The following demographics were also collected: gender, age, grade point average (GPA), and Pharmacy College Admission Test (PCAT) scores. In terms of course examinations, only the nonprescription multiple-choice questions used in both the traditional and new course examinations were compared (n=102). These examination questions were grouped by content categories (eg, pediculosis, allergies) and compared accordingly. The instruments used for collecting and collating student evaluations were identical for each course, and thus corresponding identical evaluation items were compared.

Descriptive statistical analyses were performed to evaluate the study objective. Continuous variables were summarized using means and standard deviations. Likert-scale results are also presented as means and standard deviations. One- or two-tailed t tests were used for comparisons. Categorical data were analyzed using the chi-square test. A p value of <.05 was considered significant. All statistical analyses were conducted using IBM SPSS Statistics, Version 23.0 (IBM Corp, Armonk, NY).

RESULTS

Demographics and examination performance were collected for all students in both the traditional (n=138) and new course (n= 139). The majority of students were female, from 22 to 24 years of age, Caucasian, had an average 3.6 overall GPA and 3.5 science GPA, and an average PCAT composite percentile score of 77.2% (Table 1). There were no significant demographic differences between the two groups with the exception of age. The mean age of students in the traditional course cohort was 23.6 (SD=2.4) years as compared to 22.1 (SD=2.4) years in the new course cohort (p<.001).

Table 1.

Baseline Characteristics and Final Course Grades

Examination performance was expressed as the mean percentage (and standard deviation) of the class that answered a category of questions correctly. Fifteen question categories were compared (Table 2). For the categories allergies/allergic conjunctivitis, fungal skin infections, vulvovaginal candidiasis, and musculoskeletal (MSK) injuries/disorders, and analgesics, students in the traditional course performed significantly worse than those in the new course (Table 2). For the questions concerning burns/wounds/bacterial skin infections, contraceptives and pregnancy tests, and primary dysmenorrhea, students in the traditional course performed at least as well or slightly worse than those in the new course; however, these findings were not significant (Table 2). For questions regarding pediculosis, warts, adult urinary incontinence, allergic contact dermatitis, scaly dermatoses, sunburn/sunscreen, insect bites and stings, and acne, more students in the traditional course answered correctly as compared to those in the new course, and these differences were significant (Table 2). Overall examination performance on all categories studied was better in the traditional cohort than in the new cohort (87.3% (SD=4.8) vs 85.3% (SD=6.1), p=.002) (Table 2).

Table 2.

Mean Examination Scores of Pharmacy Students Enrolled in a Traditional Course vs a New Nonprescription Course

Course evaluation data contained Likert scale rankings ranging from 1 (strongly disagree) to 4 (strongly agree). The response rates for course evaluations was 89.9% in the traditional course and 96.4% in the new course. Students who evaluated the traditional course (n=124) agreed or strongly agreed slightly less often with the statement “Students had an opportunity to ask questions” (3.6 [SD=0.7] vs 3.6 [SD=0.5], p=.66) as compared to students who evaluated the new course (n=134), but this difference was not significant (Table 3). Students who evaluated the traditional course (n=124) agreed or strongly agreed with the following statements slightly more often than students who evaluated the new course (n=134): “The course assignments were clearly specified.” 3.6 (SD=0.5) vs 3.4 (SD=0.7); “The assigned readings were well integrated into the course.” (3.6 [SD=0.5] vs 3.4 [SD=0.7]; and “The audiovisual aids (charts, movies, slides, etc) used were effective in helping me learn.” (3.6 [SD=0.6] vs 3.2 [SD=0.8]). Each of these differences was significant (Table 3).

Table 3.

Students’ Mean Course Evaluation Scores of Traditional vs New Nonprescription Courses

A smaller number of students who evaluated the traditional course (n=124) agreed or strongly agreed with the following statements as compared to students who evaluated the new course (n=134): “This course was helpful in developing new skills.” (3.6 [SD=0.5%] vs 3.5 [SD=0.8%]; and “I have become more competent in the course content/topics/skills because of this course.” (3.6 [SD=0.7%] vs 3.5 [SD=0.7%], respectively) (Table 3). These differences were not significant. The mean final course scores and associated final course letter grades were higher in the traditional course as compared to those in the new course (90.3% [SD=3.8%] vs 88.5% [SD=4.2%], p<.001) (Table 1). Lastly, the approximate number of hours spent per week on homework/study outside regular class time was reported as 3.2 (SD=1) vs 4.3 (SD=0.9) hours (p<.001) by students who evaluated the traditional vs new course, respectively (Table 3).

DISCUSSION

As a component of a curricular reform process, the existing content on nonprescription products was transferred to a new course with an innovative design. New approaches that were student-centered and based on active-learning principles were deployed. Some proven active-learning approaches such as TBL and group discussions were retained from the traditional course.¹⁰ However, the new course design included the use of innovative educational technologies including gamification and iBooks in a way to aid in content delivery, content application, and content review in preparation for examinations. Additionally, efficient course plans were made in an effort to avoid undue burden and frustration and to optimize the execution of in-class activities.

Contemporary students are likely to be especially comfortable with technology, often relying on it for cognitive stimulation and engagement. Technology may also serve as a potential solution in addressing concerns related to efficiency. Furthermore, contemporary college students may prefer learning to be independent as well as practical and hands on.¹⁶ These same students are more likely to desire that their instructors help them engage with and apply content rather than simply share what they could otherwise find on their own, either online or within a textbook.¹⁶ Meeting this generation at this intersection of learning is not too abstract or complex and can be ideally and creatively accomplished by the educator.

There were no significant differences between the two class cohorts at baseline with regards to gender, GPA, or PCAT scores with the exception of age (Table 1). The cohort enrolled in the new nonprescription course only outperformed students in the traditional cohort in a few topical categories (Table 2). Of these, the category with the greatest difference in examination performance was MSK injuries/disorders and internal/external analgesics. Approximately 10% more students in the new course than in the traditional course answered questions related to this topic area correctly. This may have been because internal (systemic) analgesics were covered earlier in the semester in the new course. Therefore, being somewhat familiar with some of the concepts may have been the reason why students in the new course scored higher on the MSK topic on the examination compared with students in the traditional course.

Interestingly, students in the traditional course significantly outperformed the comparator group in twice as many categories (Table 2). The difference in mean examination scores between the students in the traditional course and those in the comparator group ranged anywhere from 3.5% to 22.4%. Most of the students in the traditional course outperformed students in the comparator group on the scaly dermatoses examination questions. Approximately 10% more students in the traditional course answered these examination questions correctly. There were only seven scaly dermatoses examination questions in the new course as compared to 14 in the traditional course. Having twice the number of questions doubled the odds for students in the traditional course to answer correctly, and may explain the difference in performance in this area between the two cohorts. The most notable difference in examination performance was in the insect bites/stings category where 87.9% (SD=17.1) of the class enrolled in the traditional course answered these examination questions correctly vs 65.5% (SD=21.6) of the class enrolled in the new course. Perhaps students in the new course performed less adequately on these examination questions because there were several content categories to learn for that period, making mastery more challenging. In the traditional course, fewer categories were covered per four-week period as compared to the new course. Also, the insect bites/stings category was on the first examination in the first semester of the new curriculum vs the third examination in the second semester of the traditional curriculum. Students in the new course were adjusting to a completely new and innovative curriculum. Students in the revised course may have been under added stress and pressure because they had to adjust to a new and innovative curriculum. Moreover, students in the new course had no historical context within which to evaluate the new course, which in theory could possibly reduce stress and anxiety. Students answering the insect bites/stings examination questions in the traditional course were likely to have been much more adjusted to the curriculum and program and not as stressed as students answering the same questions in the new course. Students in the traditional course had the benefit of relying on senior students for historical and contextual information related to both course content and assessments.

Quarterly published iBooks in the new course helped to guide content delivery through an introduction to and provision of objectives for the weekly chapter readings. The iBooks also contained interactive learning tools called widgets, which quizzed students on chapter readings, reinforced learning, and served as a review for examination preparation. The iBooks created for the first two quarters of the semester were more substantial (ie, contained higher-engaging features) than those created for the last two. We expected that students in the new course would perform better than their cohorts in content categories covered during the quarters in which iBooks had been used extensively (eg, pediculosis, allergic contact dermatitis, sunburn/sunscreen, insect bites/stings, allergies/allergic conjunctivitis, and burns/wounds/bacterial skin infections). However, this was only the case with allergies and allergic conjunctivitis examination questions where 4.3% more students answered correctly compared to the percentage of students answering correctly in the traditional course (78.9% (SD=14.5) vs 74.6% (SD=13.4), respectively). The iBook content on allergies and allergic conjunctivitis contained interactive text, a video case, and widgets such as games and puzzles. While the widgets were common for every iBook developed, video cases were not and could have made this iBook more engaging. Payne and colleagues described the use of iBooks as a reusable learning object (RLO) that can benefit individuals in medical education and healthcare settings.¹⁷ These authors characterize iBooks as beneficial because of their reusable features. One feature is the reusable nature of iBooks Author, the e-book authoring application that enables any user to easily create and publish an iBook. Another beneficial feature is the portable nature of the learning object (in contrast to the Internet-based platform of an RLO) where users can view an iBook “on the go” without the need for an Internet connection.¹⁷ Accordingly, one would expect students in the new course with anytime iPad and iBook access for learning, studying, etc, to have outperformed those without this access. However, we did not find this to be the case. We suspect that students in the new course may not have had time to take full advantage of the educational benefits that come from using iBooks. This is despite some of these benefits being the ability to learn and study “on the go” at any time. This may be because students in the new course reported spending an average of one additional hour per week on “homework/study outside regular class time,” which could have involved preclass readings and personal study aids alone, not leaving extra time for optional learning aids provided through iBooks, no matter how advanced they were. Although students in the new course reported spending more time on the course outside of class yet performed worse than students in the traditional course, their examination performance was not likely because of the new methodologies implemented. This is because participation in the gaming elements was only required during class and the iBooks were optional for use outside of class.

To further enhance student engagement and learning, gamification was adopted and inserted into the new course on designated weeks to encourage live class participation and reinforce concepts learned earlier in the week. Well-known game formats (eg, Jeopardy) and novel game formats (eg, Name That Therapy) were used. Most of these games were converted into iBook widgets for students to later access for review prior to examination. Vulvovaginal candidiasis was the only category in the study in which gamification was also applied in the traditional course. Students in the new course scored significantly higher in this category than did students in the traditional course. Furthermore, slightly fewer students in the new course, where student teams were always used in gaming activities, reported they agreed or strongly agreed with the item,“The team teaching approach was effectively implemented in this course.” (mean=3.2 [SD=0.8] vs 3.6 [SD=0.5], respectively; p<.001) (Table 3).

These findings related to gamification were likely because this was the first time the new course was taught in this pharmacy program. Teaching this course for the first time presented unique challenges that possibly could have affected student learning as faculty members and students adjusted to the new course dynamics. Many teaching methods for this course were new or enhanced. Therefore, related classroom obstacles (eg, learning curves for game rules and instructions, technical difficulties, and team dynamics) that can affect student learning, performance, or satisfaction ought to be considered when interpreting the results. Of note, one of the new course’s main technology software programs that was used to complement gaming underwent a major update. This required all students in the course to purchase and install the new update for the software in order for the software to be fully functional. This update was not announced in advance by the software producer, therefore it led to frustrations among faculty members and students. This occurred at the start of the semester and may have left an unpleasant impression on students that could have negatively affected their future perceptions of gamification and of iBooks. Prospectively, information technology (IT) personnel will be consulted weeks prior to the start of the semester to contact software manufacturers and address any impending updates.

Although students in the new course did not perform better overall on several categories of examination questions than students in the traditional course, they did express their appreciation for the use of games in the new course. Some students reported “…the review games on Thursdays helped a lot,” “The interactive games were good tests of material and I enjoyed working with my group,” and “I feel like interactive games are useful, but only for review or on-your-own studying materials” in the free text area of the course evaluation. As a result of findings related to gamification, lesson plans for the new course will be modified to improve the execution of gamification and reduce some of its perceived obstacles the next time it is offered. For instance, faculty members will post the instructions and rules of play for each game that is to be played prior to its class period. If students are more familiar with how to play the game before class, then hopefully they will be more at ease to learn and absorb information covered during the game. Also, the college’s IT support specialist will be consulted in advance to test new gaming technologies and to provide additional faculty or student training or support as needed. This IT support specialist may also be present to provide immediate, hands-on support during some class periods.

Through this gaming intervention, we learned that gamification can still serve as a teaching method that is engaging, fun, and suitable to provide a low-stakes opportunity for students to learn, think, and receive immediate feedback. To continue to provide opportunity for students to learn, test their knowledge, and review for examinations, all games (not just some) played in the classroom will be converted into iBook widgets that can be readily and easily accessible on students’ iPads within the new course.

Almost all students agreed or strongly agreed with each of the evaluation items (Table 3). For the evaluation items where the use of iBooks was likely considered (eg, “The instructional materials (eg, readings A/V tools) used were effective in helping me learn.”), students in the new course agreed and strongly agreed slightly less often than those in the traditional course (Table 3). As with any initial use of computer or software programs, system errors or flaws are inevitable and their management can increase user dissatisfaction. Although small in number, the technical difficulties we encountered using newer versions of technology in a new course did present challenges. While students in the traditional course scored slightly higher on average on their final course grade and rated the course as good to excellent, students in the new course rated the value of the course as good to excellent as well.

This study was conducted at one pharmacy school in the United States. The composition of this pharmacy school’s student body is unique in terms of demographics and program parameters and may not be similar to that of other institutions. Therefore, study findings may not be generalizable to all colleges and schools of pharmacy. Also, because of the nature of technology, its use in the classroom may come with unforeseen challenges. For instance, technological advancements occur rapidly, which may make it difficult for those who are not IT experts to keep up. Depending on the technical devices used, the technology may be functioning properly while limitations may be user-dependent. Technical needs and problems such as software updates, glitches, or user errors often arise and create problems during use that frustrate faculty members and students. Furthermore, the impact of the new technologies used in the course could also be affected by the course instructors’ digital literacy and understanding of the software. On the other hand, all iBooks and games were developed by the same course director. Perhaps in future offerings of the course, any instructor who uses gaming or iBooks to aid in course instruction should be required to go through ample software or program training and demonstrate proficiency prior to use.

Another limitation considered is the quantity of examination questions included in this study. There was no prescribed number of examination questions per nonprescription content category. The quantity of questions per category on each examination was determined by efforts to balance all content areas covered. The number of questions included in a category ranged from 4 to 13. Also, the questions included for study were not necessarily all items on a particular examination. Each of the four semester examinations had 50 regular questions each, but only questions that were present in both the traditional and new courses were included for study. Although student performance was evaluated only on those examination questions that were alike, the excluded questions and their level of complexity could have affected student performance. For instance, if students felt an examination was difficult overall, then perhaps that affected their performance on certain categories like those that immediately followed an examination section that was perceived to be difficult. Additionally, the independent effects of gamification and iBook use on performance were not analyzed and these effects could have been due to one or the other or both. Finally, this was a new course in a brand new curriculum involving first-year pharmacy students. Adjustments among both faculty members and students were likely necessary to meet the demands of executing new coursework. These uncontrollable factors may have influenced student performance. Students were aware that adjustments were needed and often shared this concern in formative student-liaison committee meetings.

CONCLUSION

From this study, we determined that the use of teaching methods incorporating innovative educational technologies may not directly or immediately positively affect student examination performance and course satisfaction. More time may be needed to fully appreciate the use of such novel techniques, especially when ever-evolving technology is used. The implementation of educational technologies such as iBooks and gamification may present technical challenges that need to be anticipated and managed when these technologies are used in a health professions course, especially if the course is newly developed. In future offerings of the new nonprescription products course in this particular program, the time required to work through the technical aspects of the use of iPads and iBooks and applications for games will be considered and built into preclass and in-class time.

Beneficial team-teaching methods such as TBL from the traditional nonprescription products course will continue to be maintained in the new course of this program. Though advanced technology is not imperative to its use, this method seems to be just as effective as the new ones introduced in the new course. Health professions students also continue to perceive that teaching methods using audiovisual technologies (eg, iBooks and gamification) are effective. In theory, such technologies can indirectly increase student learning and engagement. However, the desirable and undesirable implications of onboarding such progressive techniques for a new course and in light of student performance should be considered. Because of its potential challenges, use of iBooks and gamification perhaps should be reserved for low-stakes, formative assessments first. It may simply take time for the positive effects of these new techniques to be fully realized. To that end, additional studies are needed at other health professions colleges and schools using these techniques to further validate or refute their efficacy and student acceptance.

Appendix 1. Traditional Course Design

Appendix 2. New Nonprescription Course Design