A Technology-Enhanced Patient Case Workshop

Abstract

Objectives

To assess the impact of technology-based changes on student learning, skill development, and satisfaction in a patient-case workshop.

Design

A new workshop format for a course was adopted over a 3-year period. Students received and completed patient cases and obtained immediate performance feedback in class instead of preparing the case prior to class and waiting for instructors to grade and return their cases. The cases were designed and accessed via an online course management system.

Assessment

Student satisfaction was measured using end-of-course surveys. The impact of the technology-based changes on student learning, problem-solving, and critical-thinking skills was measured and compared between the 2 different course formats by assessing changes in examination responses. Three advantages to the new format were reported: real-life format in terms of time constraint for responses, a team learning environment, and expedient grading and feedback. Students overwhelmingly agreed that the new format should be continued. Students' examination scores improved significantly under the new format.

Conclusion

The change in delivery of patient-case workshops to an online, real-time system was well accepted and resulted in enhanced learning, critical thinking, and problem-solving skills.

INTRODUCTION

Computers and technology are an integral part of pharmacists' daily practice. Pharmacists use computers to process prescriptions and drug orders to dispense medications, access online continuing education material, and provide drug information to their patients or other health care providers. Most currently enrolled pharmacy students belong to the Net Generation, a cohort of young people born around 1982 and educated in a computer-technology based environment.¹ Members of this cohort usually are technologically more advanced than their instructors and may even learn in fundamentally different ways than their predecessors.² These students are at ease with advanced audiovisual media and prefer to be actively involved in the learning process rather than read or write about it. They are motivated to learn more during the process of answering questions and expect immediate responses. Despite spending hours playing computer games and surfing the Internet and social networking sites, this generation also values face-to-face interactions with their peers.²

The concept and use of computer-assisted instruction was tested as early as the 1970s in teaching pharmacology to second-year medical students.³ Since 1970, the Internet and its availability to the general public has grown by leaps and bounds. Learning management systems such as WebCT, Blackboard, or Desire2Learn, and more than 200 other commercially available systems, are being utilized by universities, faculty members, and students to manage syllabi, course materials, recorded classroom lectures, quizzes, examinations, e-mail, grade books, and student evaluations. These course management systems have allowed for enhancement of both the teaching and learning processes.

In medical, nursing, and dental education, Web-based learning was demonstrated to be equivalent to traditional learning as measured by knowledge gains tested through multiple-choice written tests.⁴ An increase in learner satisfaction with technology-based learning methods compared to traditional methods has also been demonstrated in the medical literature.^4,5 This satisfaction has been attributed to the perceived ease of use and access, navigation, interactivity, and the user-friendly interface design.

Pharmacy education has been at the forefront of embracing technology in delivering its curriculum. In 1994, the academic affairs committee of the American Association of Colleges of Pharmacy (AACP) developed a list of steps for colleges of pharmacy to take to ensure the active participation of students and faculty members in the implementation and use of communications technology.⁶ The Accreditation Council for Pharmacy Education (ACPE) in its Standards 2007 has emphasized the importance of innovation and experimentation within pharmacy curricula.⁷

This manuscript describes the enhancements made to the delivery of simulated patient care, critical thinking, and problem-solving experiences during workshops in one of a series of 6 required courses taught in the first professional PharmD program. The changes include introduction of laptop computers, utilization of wireless technology, utilization of a course management system supported platform to work-up simulated patient cases, and introduction of personal digital assistants (PDAs) containing drug information software. These modifications were conducted over a 3-year period. The purpose of these modifications was to enhance student ability to think critically, problem solve, and provide pharmaceutical care. These skills are desirable traits for pharmacy practitioners as well as being listed as competencies in Standards 2007 and the American Association of Colleges of Pharmacy's CAPE Outcomes.^6,7 These outcomes are also included in the college's program-level ability-based outcomes (ABOs). A secondary purpose of the course modifications was to begin to prepare students for their advanced pharmacy practice experiences (APPE) where they need to critically think and problem solve under time constraints, thus training them to think on their feet. The introduction of computers with wireless technology and use of a course management system facilitated expedient, uniform, consistent, and less labor-intensive grading. The objectives of this research activity were to assess the impact of the course modifications on student attitude and acceptance of the new format; student learning, critical thinking, and problem-solving skills; and student attitude to introduction of PDA and drug information software into the course. The hypotheses were (1) the modifications in the course format would be well received; (2) student learning, critical thinking, and problem-solving skills would remain unchanged (no negative effect); (3) the introduction of PDAs would be well received; and (4) faculty time required for grading and providing feedback would be decreased.

One means for assessing the impact of these course changes was to gather student perspectives by administering an end-of-course survey. The impact on student learning, critical thinking, and problem-solving abilities was assessed by mapping course-related activities to program-level ABOs. These linked ABOs then served as markers of the desired learning outcomes for each of the 3 class cohorts measured. The modifications to the course will be described in detail.

DESIGN

Pathophysiology and Therapeutics is a 6-quarter series of required courses taught over the second and third didactic years of the 4-year entry-level PharmD program at The Ohio State University. These courses are designed to orient students to many aspects of pharmacy as they pertain to the pathophysiology and pharmacotherapy of different disease states. These courses attempt to integrate and reinforce components of pharmacology, pharmacokinetics, drug delivery, and pharmacotherapy. Each course, taught over 10 weeks, consists of didactic lectures and weekly small-group workshops (24-30 students/workshop). Four 2.8-hour workshops were conducted weekly. These workshops were patient case-based and interactive. The goals of the workshops were to develop clinical reasoning, interpersonal skills, and communication skills in students while reinforcing knowledge learned during lectures by applying it to simulated patient cases. These workshops were facilitated by recitation faculty members consisting of postgraduate pharmacy residents and fellows.

This research activity involved format modifications to the workshops for 1 of the series of Pathophysiology and Therapeutics courses offered during the last quarter of the third didactic year before the students proceed to APPEs. Before the course modifications, patient cases were distributed a week prior to the designated workshop, allowing students an entire week to prepare a workup on each pharmacotherapeutic problem identified in the case. The workup consisted of up to 20 pages of an electronic document referred to as a SOAP (Subjective, Objective, Assessment and Plan) note. A paper copy of the SOAP note was submitted to the recitation faculty member on the day of the scheduled workshop for grading. Using a second paper copy of the SOAP note, students presented the case and discussed relevant pharmacotherapeutic issues in class. The recitation faculty member facilitated this discussion, covered key learning concepts, hand-graded the SOAP notes for 20 key learning points, and provided written feedback on each SOAP note. This activity was time consuming for the recitation faculty member, with potential for variability in grading between different faculty members. The weekly SOAP notes constituted 15% of the final course grade. The SOAP notes were returned to the students the following week. An official SOAP note “key” was electronically posted for students to view.

Starting with the spring 2005 course offering, a series of changes were made to the workshop format for Pathophysiology and Therapeutics course. The 2005 class (n=107) was divided among 5 workshop sections (Table 1). The patient cases were no longer distributed a week ahead; instead they were handed out and prepared during the workshop. The students had the first 1.3 hours to workup the case (create a SOAP note for the case). They were encouraged to work in groups and allowed the use of class notes and other references. The SOAP note was neither collected nor graded. During the remainder of the session (1.5 hours), the recitation faculty member facilitated student presentation and discussion of the key learning concepts. The students were encouraged to self-evaluate their SOAP notes and review the posted key. Their mastery at critical thinking and problem-solving was evaluated by administering 2 case-based examinations, a mid-term and a final examination held during regular workshop sessions, and followed the same format as the workshops with 2 exceptions. The students had the entire duration of the session to individually work on their cases. The examinations submitted as handwritten SOAP notes were hand-graded by the recitation faculty member. Each case examination constituted 10% of the student's final grade. Although time spent every week in grading and providing individual feedback was decreased compared with the previous format, grading of the examinations was still time consuming for the recitation faculty member, and had the potential for inconsistencies and variability in grading.

Table 1.

Changes to Patient Case Workshops in a Pathophysiology and Therapeutics VI Course

In spring 2006, laptops with wireless capabilities and an online course management system were introduced. A laptop equipped with Windows XP and Internet Explorer 6.0 was provided for each student to use during the workshop. The College of Pharmacy employs 8 full-time information technology (IT) specialists to meet the IT needs of the college. Of these, 2 helped set up and take down the classroom 4 times per week. They were assisted by the recitation faculty members in this activity. Two other specialists assisted with setting up the course and entering the patient case information into the new course management system. The online course management system at OSU is provided by Desire2Learn and named “Carmen” (reference to the University's alma mater, Carmen, Ohio). In 2006 Carmen version 7.3 was used, and in 2007 version 7.4 was used. Carmen automatically creates a blank course shell, complete with a class roster and a set of course tools, for every course in the registrar's master schedule of courses.

The Carmen course tool selected for use in the delivery of case workshop was online quizzes with auto-grading. Starting in 2006, rather than submitting handwritten SOAP notes, students answered multiple-choice questions pertaining to the patient case (Figure 1). Students accessed their cases online during their workshop session using wireless laptops. The case was created as a quiz using Carmen's quiz tools. The workup was designed in the SOAP note format, but instead of providing free-text answers, the students answered questions designed using a specific question type called a multi-select question. The cases and questions integrated disease states and treatments options from previous pathophysiology and therapeutics courses in the series into the current content. While answering a multi-select question, students could select more than 1 answer from a group of possible answers. Using preprogrammed answers, these questions were automatically graded by Carmen immediately after submission of the workup. A grading scheme of “right minus wrong” was used in which positive points were awarded for choosing the correct options, negative points for choosing the incorrect options, and positive and negative points canceling each other out.

Figure 1.

Example of patient case information in Carmen

Access to cases was restricted by a specific workshop session, allowing access only by those registered for the session, and by the date and the time set for each session. The first half of the workshop time (1.25 hours) was devoted to completing and submitting the workup, followed by auto grading. Students were allowed to use their textbooks, class notes, PDAs, and any accessible online references. Time limits were set and enforced in minutes (a clock display in minutes); however, a grace period of 5 minutes was allowed. Students, via Carmen, were able to view their score, the answers, and the feedback to each question. The feedback consisted of concise explanations for the correct and incorrect answers and was displayed at the end of each question. The recitation faculty member evaluated the performance of each group and the pattern of answer choices made for each question by using the statistics function associated with the quizzes. The key learning concepts, the group performance, and the feedback was discussed during the remainder of the session (1.55 hours). The patient cases, the group performance statistics on each question, the feedback, and the PDA screens were displayed in the classroom in order to lead the students through the activity.

At the end of the week, after the completion of all the workshop sessions, access to the case via Carmen was restored, allowing students to work up each case numerous times to better understand the concepts. The case examination format was unchanged from 2005 except that it was conducted through Carmen. Carmen allowed enforcing additional restrictions during the examination, namely not allowing right clicking, copying, printing, or electronically sending or saving the text or answers. E-mail and instant messaging, especially through Carmen, were not allowed. After all the examination sessions were completed, the workup was graded and scored and feedback was released.

The 113 students registered for the course in 2006 were divided among 4 workshops facilitated by 4 recitation faculty members. Individual scores received on the cases did not contribute to their final grades. Each case examination constituted 10% of their final grades.

EVALUATION AND ASSESSMENT

Student attitude and acceptance of the new case workshop format and the introduction of PDA with drug information software was assessed by administering a survey at the end of the course in 2006. The 111 students registered for the course in 2007 were divided among 4 workshops facilitated by 2 recitation faculty members. Improvements were made in the 2007 course offering based on feedback on the 2006 survey. Individual scores received on weekly cases contributed 5% rather than 0% towards their final grade and each case examination constituted 7.5% rather than 10% towards their final grade. The workup time during each workshop session was enforced before grading and discussion was initiated. The cases were not graded until the entire group had submitted their workup. Students completing their workup before the end of session were allowed to leave the class room but had to return before start of discussion. Hence students wanting to use the entire workup session did not feel rushed and could utilize the entire session. With the purpose of enhancing discussion during the workshop, the students were not able to view the correct answers or the feedback. They were only able to view their scores and their responses. Feedback and the correct answers could be viewed by accessing the “key” which opened after all sessions were complete. The students could then attempt the same case a second time to improve on their earlier performance, and an average of the scores for the first and second attempt was calculated to determine their grade. If they did not attempt a second time, their original score was counted toward their final course grade. The end-of-course survey was repeated in 2007.

On the survey instrument, students were asked to report difficulties with the course format and recommend improvements. Likert-scale questions for the survey were adapted or taken from the OSU Web-based tool called FYI (Feedback on Your Instruction, Ohio State University Office of Faculty and TA Development, Columbus). Student responses to the questions in this tool were intended to provide the instructors with formative feedback on teaching effectiveness and the quality of specific aspects of their course.⁸

The end-of-course satisfaction survey was completed by 95% (107/113) and 88% (98/111) of the students in the years 2006 and 2007, respectively (Table 2). A majority of the students (85% in 2006 and 95% in 2007) agreed that the format change presented them with an opportunity to think on their feet and develop skills needed for APPEs (79% in 2006 and 91% in 2007). Compared to the year 2006, a greater number of the year 2007 students agreed that the new format stimulated critical thinking, and developed rational decision making abilities and problem-solving skills. Students from both years overwhelmingly agreed that grading and feedback through this format was expedient. The students reported spending an average of 1.1 (± 0.9 SD) hours preparing for the weekly workshop after the format change compared to 6.7 (± 6.8) hours prior to the change. The recitation faculty member reported spending 10-20 minutes providing handwritten feedback on and grading each student's SOAP note in the previous format. Under the new format, grading and feedback was completely automated.

Table 2.

Pharmacy Students' Responses to Survey Questions Regarding Their Satisfaction With Technology Enhancements to a Pathophysiology and Therapeutics VI Course

^aResponses of strongly agree and agree were combined, as were responses of strongly disagree and disagree.

^bNote that a neutral category was also included on the survey instrument but is not presented here.

A greater number of the year 2006 students compared to the year 2007 agreed that their knowledge of PDA use was enhanced by this format. Only 76% of the year 2007 class, compared to 94% of the year 2006 class, agreed that their knowledge of PDA use was enhanced during case workshops. Students reported 2 advantages to this format for the course: real-life time constraints team-learning environment. Students from both years overwhelmingly responded that this format and delivery should be continued in the future.

Student learning, critical thinking, and problem-solving skills were assessed across 3 years (2005, 2006, and 2007) by evaluating performance on their patient case workups during the workshop final examination. The patient case and the disease states tested during the final examination were unchanged over the 3 years except for the modifications made to reflect scientific and pharmaceutical advances in management of the disease states involved. The following ABOs were used as indicators of student learning, critical thinking, and problem-solving skills: (1) ability to retrieve, evaluate, and apply current disease state information (apply pathophysiology knowledge); (2) ability to retrieve, evaluate, and apply current drug information (apply drug information); (3) ability to define therapeutic goals for a patient (therapeutic goals); (4) ability to prospectively recognize and manage potential adverse drug events (ADEs) associated with drug regimen (manage ADEs); (5) ability to prospectively recognize and manage drug-drug and drug-food interactions (manage drug interactions); (6) ability to develop a monitoring plan for a patient (monitor); and (7) ability to provide adequate patient education regarding the disease states and their medication therapy management (patient education).

For each ABO, key items were identified from within the final examination case workup. For example, 9 key items were identified within the case to assess the “apply pathophysiology knowledge” ABO. Student responses in the final examination case workup were mapped to these items. The percentage of students in the class responding to or identifying these items accurately was calculated for each year. Average scores on these items for each ABO are presented in Table 3 and compared across the 3 years. The effect of format change on critical thinking and problem-solving skills was assessed through a comparison of the ABO mean percentages across the 3 years using analysis of variance (ANOVA). The data were analyzed using the statistical software SPSS, v. 16.0 (SPSS Inc. Chicago, Illinois). A p value ≤ 0.05 was considered significant. The research activity was granted exempt status by OSU Institutional Review Board.

Table 3.

Comparison of Mean Scores for Ability-based Outcomes Among Course Years, %

^aNumber of key items identified and assessed for each ABO within the case workup.

^bItem is significant at p ≤ 0.05.

Based on the percentage of students correctly responding to or correctly identifying the key items mapped to the ABOs, significant differences in mean percentage responses were evident between the 3 years for the following ABOs: managing ADEs, managing drug interactions, monitoring, and patient education (Table 3). Within each ABO category, the trend in the mean percentage scores indicates improvement in student performance over time on items associated with these ABOs.

DISCUSSION

Workshops simulating pharmaceutical care through patient cases are an important component of the Pathophysiology and Therapeutics courses in the pharmacy curriculum at OSU. The workshop objectives are to integrate and reinforce key concepts in pharmacology, pharmacokinetics, drug delivery, and pharmacotherapy as they pertain to disease state management. These workshops also enhance student critical thinking and problem-solving skills while simulating pharmaceutical care scenarios. This study was undertaken to assess the effect of changes in the workshop format on these skills and on student satisfaction. The format changes were aimed at training the students to think on their feet, thus emulating APPEs, and in providing expedient, consistent, less labor-intensive grading and feedback for faculty members.

These format changes were successful. The students strongly agreed that the new format offered them an opportunity to think on their feet and develop skills necessary for APPEs, while providing feedback and grades expeditiously. Improvements made in the 2007 course offering based on the 2006 survey results may have led to an increase in student acceptance of the format in the subsequent year. Additionally, the new format prepared students in developing a focused approach to pharmaceutical care, encouraging them to choose relevant therapy options and provide concise rationale for their choices. It did not allow a “kitchen sink” approach of listing every necessary or unnecessary treatment option with lengthy explanations. The use of the online course management system permitted consistent automatic grading, thus avoiding grading variability between different recitation faculty members. The automation of grading and feedback availability after case submission decreased faculty workload considerably. The new format also allowed for additional review and assessment of knowledge on disease states taught in previous Pathophysiology and Therapeutics courses in the series. The case questions were designed specifically to encourage students to access the most current treatment guidelines before picking their answer choices. This provided them with opportunities to explore various online databases, and medical, nursing, and pharmacy journals. Another positive outcome of this research was the linking of examination questions to program-level ABOs. This technique allowed for the tracking of progress of student learning during the changes to the course, and it enabled the researchers to demonstrate group-level performance on program-level outcomes. Comparison of student performance mapped to ABOs across 3 class cohorts indicated no loss of learning due to format changes. The improvement in scores for the ABOs during 2006 and 2007 compared to 2005 could be attributed to the format. In the new format, the students were more apt to choose correct answers from a list of options offered to them compared to the 2005 format which required them to recall and write the information in the SOAP note to receive credit.

The changed format also enhanced the students' knowledge of PDA use; however this was more evident in 2006 compared to the other years. The PDAs were made available to the year 2006 students during the course offering instead of right before the start of APPEs. The use of PDAs during simulated patient case workups increased student confidence and ability in using this tool for patient care activities. Based on this positive experience, in 2007 the PDAs were made available to the students earlier than this course offering. That year, students had their PDAs for a full quarter before they enrolled in this course. This may be one of the reasons why students in 2007 did not perceive that the workshop format added to their ability to use the PDAs.

The inability to include free-text explanations for their answer choices after picking from a multiple-choice format, the potential for misinterpretation due to the wording used to phrase questions and answers, the ready availability of answer options to pick from, rather than to come up with the options, and the “right-minus-wrong” grading scheme were listed by the students as limitations of this format. Large network outages that slowed and made the system completely unavailable occurred once each quarter in 2006 and 2007. Although infrequent, they are a potential limitation of using technology and an online course management system. This was remedied by maintaining a paper copy of the quiz workup which could be photocopied and distributed during the session.

Broader educational goals can be fulfilled by using and modifying this format further. By allowing the students to access and submit the cases from home, part of the workshop time, after discussion of the initial case with colleagues and faculty members, could be used for follow-up activities. These could involve more mini-cases to strengthen key concepts or clarify complex problems, journal club presentations, complex drug information exercises, patient counseling/education exercises, or role-playing exercises.

Parts of this new format have been integrated into other Pathophysiology and Therapeutics courses in the series. These include working in groups, receiving patient cases during workshop, and writing the workup during workshop with time constraints rather than over a week. The initial time and resource commitment needed to set up the case workshops using the online course management system, the limited availability of classrooms fitted with wireless technology, and the limited number of laptops required to support running of 8 workshops a week are some of the factors that may have prevented complete integration of this new format into other Pathophysiology and Therapeutics courses in the series.

Future plans for Pathophysiology and Therapeutics include continuing with the new format and modifying it to include more activities such as role-playing and patient counseling. These would involve using simulated electronic medical records and digital storytelling techniques to simulate scenarios such as participation in hospital in-patient multidisciplinary team rounds, assessment of medication history, or physical assessment in an outpatient clinic setting, or a patient education session in a community pharmacy setting.

SUMMARY

Technology-based teaching and learning methods constitute a substantial portion of innovation and experimentation within pharmacy curricula. Laptop computers, wireless technology, and a course management supported platform were utilized to modify the delivery of a required course to enhance student learning, critical thinking, and problem-solving skills during the simulated patient case workshop. The format was designed to train the students to accomplish these skills under time constraints and to think on their feet, thus preparing them for their APPEs. The format changes were well accepted by the students who strongly agreed to continue the new format. Analysis of the examination responses mapped to ABOs showed significant improvement in most cases under the new format. Automation in grading and feedback availability decreased the faculty workload.