Multi-faceted Approach to Improve Learning in Pharmacokinetics

Abstract

Objectives

To integrate a series of educational strategies ranging from content delivery to assessment, including a change in philosophy regarding the use of in-class time, to enhance learning of pharmacokinetics.

Methods

Several approaches were taken to develop a significant learning experience in the basic and clinical pharmacokinetics courses including games, a piloted multimedia module to offset content delivery and free-up class time, reflective writing, and an immediate feedback assessment. Games, a multimedia module, reflective writing assignments, and other innovative learning tools were incorporated into pharmacokinetics courses, as well as an assessment tool to provide immediate feedback.

Results

Median examination scores did not improve following the incorporation of the teaching innovations; however, based on survey results, student satisfaction increased.

Conclusions

Already high median examination scores (>90% from historical controls) did not improve; however, the effectiveness of the innovations implemented, which included deep learning and critical thinking and communication skills, may be more accurately measured over the long term, eg, in performance in advanced pharmacy practice experiences.

INTRODUCTION

Parallels can be drawn between the optimal management of diseases and learning environments. For many diseases, optimal treatment requires targeting different physiologic pathways and pathologies in a concerted, synergistic effort. Similarly, learning may be improved by focusing on various aspects of a student's educational needs. Fink¹ states that the key to learning is the ability to create significant learning environments. These environments should include formulating and communicating learning goals with appropriate feedback and assessment procedures, generating teaching and learning activities to support the learning goals, and integrating the component parts. Most reported strategies aimed at enhancing student learning have centered on teaching and learning activities (ie, active learning) but may exclude other potential teachable moments inside and outside of class. Enriching the learning environment through incorporation of a variety of teaching and learning strategies and methods both in and out of the classroom, rather than relying on a single, isolated classroom method, should yield enhanced learning. Combining learning activities that actively engage students, such as games and immediate feedback assessment, with out-of-class activities, such as reflective writing and asynchronous content delivery, can create a significant learning environment that addresses a broad range of students' learning needs.

Games are one active-learning strategy that offers a creative, interactive, in-class alternative to the traditional lecture or classroom activity. This technique can be used to reinforce previously learned material, introduce new ideas or concepts, create opportunities to apply and practice learned material, and motivate/engage participants. Games that incorporate and integrate skills necessary to the developing practitioner, especially if they emulate real-life experiences and/or draw upon learners' prior experiences, can be beneficial in that they can deepen understanding and promote achievement of desired abilities. Desirable skill sets promoted through games or other active-learning approaches include communication, critical thinking, creative thinking, and practical thinking.

While games can enhance learning within the classroom setting, they do not necessarily address what students are doing and thinking outside of class. Students must reflect both on what and how they are learning^1,2 since deep learning takes place only in the context of the learner's belief that information is relevant and important. Deep learning focuses on the significance of what is being learned and relates new information to previous knowledge, all with the motivating factors behind learning being intrinsic; this type of learning works better for long-term retention whereas surface learning is more for short-term retention.³ One approach to encouraging this personal connection to learning is through reflective writing. Reflective writing helps to articulate connections between new and existing knowledge as well as to relate new knowledge to the learner's future goals or value system. Reflective writing can help learners demonstrate what they have learned; encourage independent thought and ownership; enable expression of feelings, beliefs, and values; and provide a venue to work through ill-structured problems. Each of these cognitive activities promotes deep rather than surface learning.³

Most courses provide some degree of foundational knowledge, which can consume a significant amount of class time. Class time is limited and resource intensive; delivering content (ie, obtain basic, foundational information) may not be the most effective or efficient use of that time. There is evidence that learning factual content is well suited for self-paced efforts where material can be viewed repeatedly. This evidence was illustrated in a study by Knight et al in which students who did not attend lecture but were provided with lecture notes performed as well as students who attended lectures.⁴ The finding suggests that class time (ie, student-instructor interaction time) would be better used for promoting higher levels of learning (eg, analysis, evaluation) than for content exchange. To achieve the goal of higher order learning, rather than simply delivering content, it is necessary to free up class time to engage learners in activities that require them to apply their knowledge, analyze data, evaluate practice-related scenarios, and/or solve problems. If students come to class with a grasp of the basic knowledge (facts and concepts), then class time can be used to apply knowledge, develop problem-solving and critical thinking skills, and enhance communication and interpersonal skills through discussion and collaboration.

Assigned reading has been the predominant vehicle used by faculty members to prepare students with sufficient knowledge to participate in class. While book chapters and articles can be important learning resources, reading can be passive for novice learners, particularly if it is not directed by specific learning goals, questions to answer, writing or other active-learning assignments, or self-assessments. These static resources may appeal only to those learners whose learning style favors reading or advanced learners who can create a context for learning the material they are reading. Increasingly, university students are wired; they are constantly stimulated by technology and information. Formatting and delivering basic information and concepts in a way that appeals to these technology-inclined students can improve the effectiveness and efficiency of self-directed learning and free class time for greater interaction with faculty members and emphasizing active learning. The development of interactive, multimedia modules based on sound instructional design principles, should provide an appealing means of engaging the current and future generations of students.

An additional approach to creating significant learning opportunities is to utilize examinations both to promote learning and to assess student performance. Classical behavioral theory (ie, law of effects) states that immediate feedback improves behavior/learning compared to delayed feedback. An examination format that provides immediate, corrective feedback enables students to leave the examination knowing how well they did and enhances learning by identifying areas in which they have failed to gain knowledge or incorrectly understood or applied concepts.

The innovations described here are about optimizing learning by using a variety of teaching, learning, and assessment approaches to help students acquire knowledge and internalize concepts related to pharmacokinetics, and to apply that understanding to the solution of patient-care problems. Because pharmacokinetics is predominantly viewed as a math-based foundational course in many pharmacy curricula, the challenge to instructors is to help pharmacy students relate foundational content to clinical practice and acquire sufficient knowledge and skills in pharmacokinetics to support clinical decisions. These innovations aim to take a course that historically has not been well-appreciated by the students and transform into a significant learning environment.

DESIGN

Pharmacokinetics instruction in the School of Pharmacy at The University of North Carolina at Chapel Hill consists of 2 courses in the second-professional year. The foundational course is offered in the fall semester, with the clinical correlate course following in the spring semester of the second-professional year. Each is a 3 credit course that meets 3 times a week for 50 minutes per class. The class enrollment in 2005-206 was 141 students. The pharmacokinetic classes during the second professional year vary in structure including “traditional” lectures (most using PowerPoint), partially completed notes, case studies, and Socratic-type “discussion.” Regardless of format, students typically are given learning objectives, reading assignments to complete prior to class, and practice problem sets.

Games

Three games were developed for the pharmacokinetics courses and are detailed elsewhere.⁵ Each game focused on several desired learning outcomes including collaboration and teamwork, communication skills, and thinking skills. All games were group-based, with student assignment to a group based on gender and current course performance (ie, grades on examinations early in the semester). Games were assessed by comparison of final examination performance to historical controls and by use of an attitudinal survey instrument. The PK Poker Review Game was designed to review basic pharmacokinetic principles and to help students develop skills for common pharmacokinetic calculations and decisions (eg, dose conversion, estimating dosing interval, estimating half-life). The Pharmacy Scene Investigation game, played during the applied pharmacokinetics course, was designed around a mysterious cause of death of a popular television character with the purpose of using pharmacokinetic information about tricyclic (TCA) and selective serotonin-reuptake inhibitor (SSRI) antidepressants to solve the crime. The final game, also played during the clinical pharmacokinetics course, was the Clue Game. The game was based on a the popular murder mystery game Clue and incorporated selected Top 300 drugs where students had to use information related to contraindications in special populations (eg, elderly, pregnancy), organ dysfunction (eg, renal, hepatic), major pharmacokinetic drug interactions, and the results/extent of the interaction to answer sets of clues that led to a murderer, weapon, and location.

Immediate Feedback Examinations

Three multiple-choice examinations were administered during the foundational pharmacokinetics course. For each examination, an immediate feedback answer-until-correct format with 5-answer choices was used (Epstein Education Enterprises, NJ). A description of this assessment tool (answer sheet format) can be found elsewhere.⁶ Examination questions were constructed based on content area (approximately 2 to 4 questions per class period) and level of learning according to Bloom's Taxonomy, and arranged with simpler “knowledge” questions in the beginning, “application” questions in the middle, and “analysis/evaluation” questions at the end.

Grading of examinations was based on the number of attempts required, with an answer-until-correct format; first attempt correct answers (ie, one block scratch) scored 5 points; questions answered correctly on the second attempt scored 3 points; and questions answered correctly with 3 attempts scored with 1 point. Answers obtained after more than 3 attempts were assigned no points. Cumulative scores were scaled to the respective number of points allocated for that examination in the syllabus (ie, 100 points for examination 1 and 2, 150 points for the final examination) Examination scores were compared to historical controls to assess equivalency between approaches. Surveys also were utilized to capture student attitudes towards this format.

Reflective Writing

Each week, students were required to complete a reflective writing assignment addressing 3 out of 5 posed questions related to the most important parts of the past week's classes, any unanswered questions that arose during class or in their readings, application of the material to their own life (and future role as a pharmacist), any personal experiences with the material, and relation of material learned in their prior or concurrent courses. Some writing assignments had additional components such as expressing expectations for and fears about the course, writing examination questions to review for scheduled examinations, and writing a letter to the next year's class on their perceptions of the course and how to be successful. This latter assignment was not read by the instructors. Students were informed not to expect individual feedback (e-mail responses) on their writing but that their questions, comments, and thoughts might be anonymously incorporated into classroom discussion. The reflective writing assignments accounted for 12.5% of their grade; grades for reflective writing were based on submitting the assignment as motivation for completing the exercise. Writing assignments were submitted as quiz function in the course management system Blackboard, version 6.3 (Blackboard, Inc, Washington, DC); this venue allowed instructors to obtain one screen that could be scrolled through to view all the responses.

Assessment of the impact of the reflective writing on learning was performed by using the first examination as a comparison to historic controls as well as course evaluation data. The reason for using the first examination is the fact that both reflective writing and the immediate feedback method of testing was used in the same semester; thus, the immediate feedback format offers learning advantages after the completion of examination and for subsequent examinations. Performance on the first examination, however, would be impacted more by the reflection of the students rather than the immediate feedback.

Multimedia Enriched Learning Environment (MELearn)

A pilot project was undertaken to develop an enriched, multimedia, interactive module to engage students in content outside of class. This initial module was developed, in collaboration with the University's TL Interactive team within Information Technology Services (ITS) Teaching and Learning division. This module was developed to offset approximately 1.5 hours (out of 4 contact hours) of the hepatic clearance section in the foundational pharmacokinetics course. This module covered basic definitions and concepts (eg, unbound intrinsic clearance), the “well-stirred” model, and approximations based on high- and low-extraction ratio drugs. The module and each subsection started with learning objectives and ended with 2 to 3 practice exercises that related to the objectives outlined in the beginning of each section. The sections were comprised of 3 to 8 scenes that were fully narrated and animated in Macromedia Flash (version 8). Full navigation control was given to the user to rewind, stop, and fast-forward through the animated segments. The module also was equipped with several tools including digital graph paper, closed captioning, calculator, equation helper, and glossary. A print option was available so the user could print the narration or the embedded resource material. At the end of the module was a quiz that assessed each of the overall module objectives. In the backend design, student navigation was tracked by both recording the order in which sections were viewed, time spent on each section, and practice and quiz results.

Assessment of the effectiveness of the MELearn to replace lectures during class time as a source of foundational material was conducted as randomized, parallel group study. The class was given verbal and written descriptions of the study and students signed an informed consent consistent with the University's policy for the protection of human subjects. Of the 141 students enrolled, 132 consented to participate and 116 completed the study. The students completed a pre-instructional quiz on the hepatic clearance material approximately 2 weeks prior to the hepatic clearance section. Students were randomly placed into 2 groups for the first day of hepatic clearance material. The “in-class” group (n = 63) received traditional classroom instruction and the “module” group (n = 53) used the module in lieu of coming to class. After the class session or completion of the module, students completed the post-quiz related to the material just presented. An attitudinal survey instrument about the class or module format was also completed. After the first day of class on the hepatic clearance material, all students were invited back to class and all students were given access to the module; the individuals in the module group also were provided with an attendance-optional lecture on the hepatic clearance material similar to what the other group had received.

ASSESSMENT

The median scores from examinations for the foundational and applied pharmacokinetics courses from the past 3 years are presented in Table 1, year 1, in which no innovations were introduced, served as the control for comparisons. During year 2, games were introduced. In year 3, games were used again and reflective writing, immediate feedback assessment, and the MELearn module were introduced. When examining the foundational pharmacokinetic course, several differences were found. Year 3 scores on the first examination, in which the immediate feedback examination format and reflective writing were used, were significantly higher than those from year 2 (p < 0.001) but not those from year 1. For examination 2, year 3 scores were significantly higher than year 1 (p < 0.001) but not year 2. There was no difference between any of the final examination scores for the 3 years. When examining the impact of games during the applied course, final examination scores from year 2 when the games were used were significantly higher than those from the previous year when games were not used (ie, year 1).

Table 1.

Comparison of Examinations Scores for Three Consecutive Years in the Foundational and Applied Pharmacokinetics Course

Abbreviations: exam = examination; final = final exmaination

^aYear 1: no educational intervention; year 2: games were played and were evaluated on the final examination in both courses; year 3: reflective writing was introduced and the immediate feedback format was used for assessment in the foundation course only

^bNo statistical comparisons were made to the control year

^c p < 0.001 compared to scores on respective year 1 examination (Mann Whitney test with Bonferroni adjustment for multiple comparisons)

^d p < 0.001 compared to year 2′s respective examination (Mann Whitney test with Bonferroni adjustment for multiple comparisons)

To compare the use of the interactive learning modules with traditional lecture format, a pre- and post-study design approach was used. There was no difference between the in-class and module groups in either pre- or post-quiz scores (Table 2), but there was a significant increase (48%) in scores from pre- to post-quiz in both groups. Students were also asked about various aspects of the lecture and the module (Table 3) immediately after completion of the module/class, prior to the general release of the module, and prior to the second class period (for “module” students who wanted live instruction). Nearly all of the students agreed or strongly agreed that the lecture and the module were easy to follow (Table 3). The in-class students tended to find the lecture to be more engaging and worthy of their time to complete than the module (p < 0.001). The surveys summarized that students tended to favor coming to class over completing the module; however, students did enjoy the module, and in the open-ended, written comments they expressed concerns that modules would replace student-instructor interaction. Approximately 95% of the students in the lecture group indicated the lecture format was their first choice for acquiring knowledge, while 58% of students in the module group selected the lecture format as their first choice (Table 4).

Table 2.

Comparison of Pre- and Post-Quiz Scores for Students Who Attended Class or Completed the Module

^aData presented as median and 25% and 75% quartile

^b p < 0.01 compared to pre-quiz

^cSignificance was set at p < 0.0125

Wilcoxon-Signed Ranks test with Bonferonni adjustment

Table 3.

Pharmacy Students' Responses Regarding Presentation Methods Used to Teach Pharmacokinetics

^aRating scale: 5 = strongly agree; 1 = strongly disagree

^b p < 0.001; p value set at 0.0083 for multiple comparisons

Table 4.

Pharmacy Students' Ranking of the Effectiveness of Presentation Methods Used to Teach Hepatic Clearance

DISCUSSION

In pharmacy education, like other professional programs, it is often difficult to attribute learning to a single class or educational intervention since each provides only a small snapshot of what the students are experiencing. The ultimate measure of learning is the impact on patient care or advancement of the profession, either during experiential education or after graduation. One metric used to assess the innovations described here was examination scores; however these have a limited ability to measure true success of the innovations. First, examination scores have been historically high for the sequence of courses for which the innovations were developed, so there were not significant improvements in test performance (grades) to be gained. Secondly, these innovations allowed for reallocation of time to focus more on application, thinking skills, and communications skills, which are typically more difficult to assess than knowledge and may require different assessment formats. This was the first year for the mapping of examination questions based on taxonomy level (eg, knowledge, application, evaluation) and content; one method of using current techniques to examine learning improvement is to focus on performance on the higher levels of learning within the examinations. In addition, since these innovations seek to improve deeper and sustained learning, improvements in long-term retention and ability to apply information to complex problems may not be reflected well in short-term assessments of student knowledge. If the innovations help form a significant learning environment where students can achieve high orders of learning and/or develop life-long learning skills, then the innovations could be viewed as successful. As learners experience more teaching and learning innovations and they accomplish new levels of understanding, more advanced assessments will be needed to assess student performance related to higher orders of learning.

Although examination and other assessment scores may answer some questions about learning, attitudes about the course and innovations are also important. The best intended learning experiences, if not understood, accepted, or appreciated by the students, may not yield the highest learning. As a whole, student attitudes were positive toward all the innovations. Most students enjoyed the games, especially as a break from the more traditional class session.⁵ Negative student comments related to the games included that they were not learning as much compared to lecture, though course performance data suggested otherwise, or that games appeared to be more chaotic compared to lecture.⁵ It is understandable that students would experience this cognitive dissonance and discomfort, since they are most accustomed to fairly structured educational settings. Not having written “class” notes may also affect students' views of learning, as the amount of notes may indicate the quantity of what was learned.

While games help lighten the classroom experience, examinations, especially immediate feedback examinations, increase the level of anxiety for students. This increased anxiety can lead to poor performance. The primary goal of the immediate feedback assessment was to assess whether this format increased, at least in the short-term, retention of material (ie, performance on a cumulative final examination). A secondary goal to support long-term use of this format was to demonstrate that the anxiety that may be associated with immediate feedback is not detrimental to student performance in general. Although a statistically significant difference (grade improvement) was noted for the first examination, this difference may not be significant in terms of “more” or “deeper” learning. By the end of the semester, students enjoyed the examination format and requested that all classes that use multiple choices examinations use this format. This is consistent with others who have used this assessment approach.⁷

Reflective writing was first introduced in the same year the immediate feedback assessment was initiated. A primary challenge was the time or procedures needed to review the writings. Blackboard was used to collect the weekly writing assignments with the advantage that the program collated all the writings so the instructor could scroll down and read all the writings at once. The time to read 140 writings was approximately 20 to 30 minutes. Although the reflective writing in conjunction with immediate feedback examinations did not appear to affect course performance, the reflective writing did give the instructor insight into several areas. First, the writing elucidated points of confusion that could then be corrected in class. Second, the writing provided personal glimpses into the students' lives and how they related what they were learning about in class to personal experiences (eg, family members who were hospitalized and receiving infusions). Finally, and unexpectedly, the writing provided the instructor with new insights on how students viewed the course and the material. Many students had concerns and anxieties about the material and the course despite positive information from students who already had been through the course. This information was invaluable in establishing an environment where students felt safe.

The MELearn was a proof-of-concept experiment testing whether an interactive, asynchronous, multimedia entity could effectively replace instructor-based content delivery. Although following their limited experience with this method, students still preferred a live instructor, even though there was no difference in knowledge gained when comparing an instructor to the module based on students' scores on material presented by an instructor vs. that presented in a module. The intention of developing this platform was not to replace the instructor or the instructor-student interaction, but to restructure instructor-student interactions so that the instructor's time could be used for facilitating higher levels of learning rather than delivering content. Further investigation is needed to assess the benefits of this approach compared to the traditional approaches. The development of these types of modules is time and financial-resource intensive. Although subsequent modules may require less time per module to develop, the manpower and financial investment remains substantial. Lower cost alternatives, such as simple, narrated PowerPoint presentations that incorporate audio, visuals, simple animation, and/or interactivity through quizzes, button-clicks should also be explored.

In general, students found the innovations an enjoyable change from the more traditional class session. Students did not necessarily feel they learned more through the innovations, but it is often difficult to judge self-learning especially when students are use to pages of notes and slides as justification of “learning.” It is also plausible that students are not able to discriminate between deep, surface, or strategic learning in the short term of a semester and therefore may be a poor judge of what or how much they actually learned. The innovations did not reduce student performance and allowed class-time to be used for other types of learning; the innovations, therefore, can be viewed as beneficial. The basic pharmacokinetics course will continue to develop multimedia learning modules to shift content delivery to outside the classroom and use class time for more discussion, problem-solving and developing the meta-cognitive processes. Games will be further developed and incorporated into a more problem-based learning environment and used to complement the modules as in-class activities. The immediate feedback assessment will be further developed to map course outcomes (ie, linking examination questions to learning taxonomy and content area) and to focus on the higher orders of learning in alignment of class time being refocused on students' acquiring these higher order skills. The reflective writing will be developed to capture moments when students are the most engaged in class.²

The ultimate goal is for instructors to become coaches, facilitating learning, supporting students' efforts, and setting them up for future success. While there are many methods for creating an atmosphere that encourages deep-learning, the techniques described were chosen based on methodologies the instructor gained from various faculty development activities. A secondary reason for the selection of one of these activities, the development of the multimedia modules, stems from a fundamental change in the vision of the School with regards to clinical education.⁸ This change in vision relates to the shift in philosophy that classes should no longer “cover material” and class time should not be used for content exchange but for developing higher-order learning processes. These innovations will assist in bringing class sizes down, facilitating more engagement, and further developing what Fink has described as creating a significant learning experience.