Incorporating Active-Learning Techniques and Competency Assessment into a Critical Care Elective Course

Abstract

Objective. To design, implement, and measure the effectiveness of a critical care elective course for second-year students in a 3-year accelerated doctor of pharmacy (PharmD) program.

Design. A critical care elective course was developed that used active-learning techniques, including cooperative learning and group presentations, to deliver content on critical care topics. Group presentations had to include a disease state overview, practice guidelines, and clinical recommendations, and were evaluated by course faculty members and peers.

Assessment. Students’ mean scores on a 20-question critical-care competency assessment administered before and after the course improved by 11% (p < 0.05). Course evaluations and comments were positive.

Conclusion. A critical care elective course resulted in significantly improved competency in critical care and was well-received by students.

INTRODUCTION

The provision of care to critically ill patients has been a major priority for the placement and development of clinical pharmacy services, dating back to the first major documented efforts of clinical pharmacy.¹ The current practice model for critical care pharmacy incorporates the entire scope of therapeutic knowledge, with significant emphasis on “best practice” medication use. Several publications have detailed the beneficial role of pharmacists in critical care settings.^2-4

The American College of Clinical Pharmacy (ACCP) Critical Care Practice and Research Network (PRN) described the difficulty of standardizing the training that should be required for a specialty degree/certification in critical care pharmacy practice.⁵ Flexibility within pharmacy curriculum design was strongly encouraged to allow for incorporation of critical care topic discussion and learning. Furthermore, ACCP’s 2009 “Pharmacotherapy Toolkit” provides a list of critical care pharmacotherapy topics ranked by importance of inclusion in the pharmacy curriculum.⁶ Critical care topics are commonly incorporated into the classroom curricula of US colleges and schools of pharmacy.⁷ Eighty-three percent of surveyed colleges and schools have integrated critical care into the pharmacotherapeutics sequence, and 51% of institutions have offered elective courses.

Teaching disease state management and pharmacotherapy of critically ill patients can prove challenging in the large classroom lecture setting that is common for therapeutics courses. Other potential limiting factors to teaching critical care to pharmacy students include the absence of evidence-based guidelines for some conditions, the rapidly changing and evolving nature of the management of patient conditions, and the steep learning curve required for mastering patient assessment and monitoring techniques. A small class setting, such as in an elective course, could help mitigate some of the learning challenges common with critical-care--related education by providing students more direct access to faculty members and an opportunity for inclusion of different education-delivery styles.

At the Sullivan University College of Pharmacy, an elective course was developed with the mission of enhancing student understanding of patient-centered care of the critically ill using independent student-led content development and delivery of information in a small classroom setting. We used a technique called cooperative learning, a type of active learning that involves structured social education where students in small groups work together to learn the assigned material and are held accountable for both their individual and group’s performance on assignments.^8-10 In adapting this technique to the course design, we wanted student groups to maintain independent control over content delivery and to be held accountable for delivering the information effectively to their classmates.

Active-learning improves critical-thinking skills, encourages student participation in the learning process, and influences students and faculty members to provide beneficial and constructive feedback for improvement.^11,12 The most recent Accreditation Council for Pharmacy Education standards for the programs leading to the PharmD degree encourage the use of active-learning strategies to support development of critical thinking and problem-solving skills.¹³ This paper describes the methods used to achieve the dual goals set out for the elective course: first, to enhance students’ understanding of patient care of the critically ill patient population; and second, to evaluate the effectiveness of using cooperative learning as the principal method of education delivery.

DESIGN

A once-weekly 2-hour elective course in critical care was offered during the spring quarter of the second-year curriculum in an accelerated 3-year PharmD program. The course was offered immediately prior to initiation of advanced pharmacy practice experiences (APPEs) and concurrently with the final course in the pharmacotherapeutics sequence. This part in the sequence included lectures covering many specific topics related to critical care, including sepsis/shock, cardiac arrhythmias, infectious diseases, and sedation/analgesia, among others.

During the design phase, course-level learning objectives were established to guide the development of activities within the course (Table 1) using Bloom’s Taxonomy.¹⁴ Where appropriate, these objectives were mapped to program-level objectives for curricular purposes. Twenty-four students were randomly assigned to 1 of 6 four-person groups and asked to prepare presentations on their assigned topic. Groups were instructed to create presentation objectives using Bloom’s Taxonomy and create 5 quiz questions that were mapped to the group’s presentation objectives. In addition to a detailed description of disease state therapeutics, students were required to include an analysis of at least 3 student-selected primary literature references and 1 faculty-selected review article in their presentation. Groups were also asked to include visual/auditory aids and activities to encourage active student-to-student discussion of the topics.

Table 1.

Student Learning Objectives for a Critical Care Elective Course

Topics for the course were carefully selected based on several factors. First, the major areas of critical care practice were identified by course faculty members. Then, the institution’s pharmacotherapeutics curriculum was examined for topics that were not explicitly included in the lecture series, particularly those noted in the ACCP Pharmacotherapy Toolkit.⁶ Topics chosen were intended to augment subject matter within the pharmacotherapeutics sequence, and the discussion schedule was structured to align with the concurrent lecture sequence in the pharmacotherapeutics course (Table 2).

Table 2.

Topic Outline for a Critical Care Elective Course and Similarities With the 2009 AACP Toolkit Topic List

Faculty members and fellow classmates evaluated each group presentation for clarity, organization, and achievement of presentation objectives using an assessment rubric included in the course syllabus. Faculty and peer assessment forms were reviewed and constructive feedback compiled and given to the presenting group members. Additionally, faculty members evaluated student responses on the quizzes after each class session to assess clarity of the group’s content and student understanding. The group members were instructed to use the feedback provided to develop and present a 20-minute “revision session” in class the following week to improve the quality and clarity of the content of their presentation.

Institutional Review Board (IRB) approval was obtained for this study and students provided informed consent for participation in the competency assessment. Students’ grades on any components of the course were not affected by participation in this study. Additionally, student participation was not mandatory for class credit.

EVALUATION AND ASSESSMENT

Fifty percent of the final grade for the elective course was based on the group presentation. Thirty percent of the grade for the group presentation was derived from students’ individual performance and 15% from the group performance, with an additional 5% derived from the quality of the revision session given 1 week after their presentation. The remaining 50% of the course grade encompassed an average of 6 quiz grades from the presentation content (10%), each group’s 5 quiz questions for their presentation (5%), an average of the peer assessment scores (5%), participation in class discussion and peer assessment (15%), and a comprehensive final examination (15%). Overall, 30% of each student’s final grade was determined by group scores and 70% by individual scores, which included individual contribution to the group presentation, presentation quizzes, final examination, and participation. The mean final course grade was 96.8%, and all students received an “A” grade for the elective course.

During the first week of class, students were asked to complete a 20-question multiple-choice assessment of basic critical care competency, which was defined as the necessary skills or abilities of an entry-level practitioner.^15,16 In designing the competency assessment, the faculty members’ primary goal was to create an appropriate assessment to evaluate the minimum critical care knowledge requirement of graduates entering the workforce. The assessment was not intended to evaluate the student’s in-depth understanding of complex disease states or patient assessment related to critical care.

The development of the competency assessment occurred prior to and independent of the selection of topics to be discussed and content provided by the student-led discussions in the elective. It was designed to test specific questions and facts related to critical care pharmacy practice that would be encountered and should be part of the knowledge base of an entry-level pharmacy practitioner. However, we did take into consideration that students rather than pharmacists would be assessed with the tool.

Each student assessment was scored, but neither the scores nor the correct answers were disclosed to students. The same assessment was administered again at the end of the course. After all group presentations were delivered, the content included in each presentation was compared with the content of the questions on the competency assessment. Eight of 20 questions (40%) on the competency assessment related to material covered by the student presentations and in-class discussions.

The difference in mean scores of all students between the pretest and posttest was determined using a paired t test with an a priori significance level of 0.05. A power calculation was not performed due to the limited enrollment size of the class and novel nature of the competency assessment.

Twenty-four students participated in the elective course, and all students provided informed consent to participate in the study. Overall student performance on the pretest was modest (mean score 57.5% ± 11.8%), with a slightly lower mean score (55.2% ± 15.6%) for the 8 questions related to material specifically covered in the elective course. On the posttest, scores of 17 of the 24 students (71%) improved, with an average improvement of 16.5% (range 5% to 40%). Six students scored the same on the pretest and posttest, and 1 student’s score decreased by 5%. The average score for the class significantly improved 11% (68.5% ± 12.8%, p < 0.05). On the 8 questions containing subject matter covered in the course, the mean score increased significantly from pretest by 17.7% (72.9 ± 15.5%, p < 0.05).

Course evaluations were completed by all 24 students in the course (Table 3). Approximately 50% or more of the class strongly agreed with all of the statements evaluated, and over 70% strongly agreed on 5 of the 12 statements (42%).

Table 3.

Students’ Evaluation of a Critical Care Course (n = 24)

In written comments, students stated that the topics presented in the course helped tie real-world practices to lecture material and furthered their learning of how to make sound evidence-based decisions. Students also recommended that more faculty-driven presentations and discussions be included to provide real-world insight and experience on some topics and provide students with a better perspective on the clinical impact of some of the recommendations suggested by the groups.

DISCUSSION

Active-learning strategies in various forms have been successfully incorporated into multiple disciplines and education delivery platforms in the health education continuum, from undergraduate courses to continuing professional education and workplace learning.^17-19 In the pharmacy literature, several reports describe successful use of active-learning strategies covering a wide variety of subject matter including pediatric pharmacotherapy,²⁰ women’s health,²¹ nutrition,²² cardiology,²³ and integration of clinical and basic sciences,²⁴ among others. In designing this elective course in critical care, faculty members sought to incorporate active learning in the form of cooperative learning as the principal method of content delivery. Specifically, groups of students were allowed to decide independently which specific content to teach to their classmates and then group members worked together to prepare and deliver the material. This course also fostered skill development related to delivery and appraisal of advanced clinical practice recommendations, as well as literature evaluation and drug information.

Students in the elective course used many creative techniques to add additional information and perspective on the material they presented. For example, in the presentation on chronic obstructive pulmonary disease and mechanical ventilation, a student in the group brought a portable ventilator device to serve as a visual aid for the class in understanding the impact of care on daily life for some patients and explaining the mechanics of the various settings. During the presentation on intensive care unit delirium, the group provided several recorded demonstrations of delirium screening tools used in the critical care setting.

Despite the long history of critical-care services and pharmacist participation on multidisciplinary teams, only one other description of a pharmacy elective course in critical care was found.²⁵ This course met weekly over a semester with assigned pre-class readings and used active-learning techniques including discussion and debate of clinical scenarios. End of course evaluations showed students felt strongly that the topics covered in the course were more in-depth than those taught concurrently in the therapeutics course and that the elective course increased their interest in including clinical care activities in their future pharmacy practice. These positive evaluations, echoed by evaluations of our own elective course, show a clear benefit obtained from a focused learning experience in critical care.

We evaluated the effectiveness of cooperative learning as the primary method of content delivery for the course through the use of a specially designed critical care competency assessment. While the framework of competency assessment has been widely discussed, such an assessment specifically related to critical care has not been previously described in the literature. While only 8 of the 20 questions on the competency assessment were included in the course content, the students independently selected this content and improved their understanding. The competency assessment fulfilled one of its intended purposes in that it provided faculty members with a method to analyze the ability of students to independently develop presentations of critical care content and deliver the information with minimal faculty oversight. This finding advanced our understanding of incorporating active-learning strategies in critical care coursework. Our findings suggest that students learning of this particular subject matter may require faculty involvement focused on both the structure and content within the course to achieve competency.

The overall success rate on the examination, while improved, could not be adequately compared to a previously existing objective standard to create an absolute breakpoint for student competency. Given the breadth of possible material related to critical care pharmacy practice, the development of an objective standard for baseline critical care knowledge and skills of graduating pharmacy students (or preparation for APPEs) becomes a logical goal to ensure that the course remains effective at reaching the primary goal of educating students in critical care. We hope to use our research and results to develop a better framework for the skills and knowledge relating to competency in critical care subject matter. Additionally, student feedback will be used to adapt the course for future students enrolled in the course. There is a potential to expand the use of these active-learning techniques throughout the curriculum at our institution. We also hope to include the competency assessment as a component in gauging the impact of course design changes and overall critical care knowledge level among our students on a year-by-year basis.

We believe that the basic framework and design of this course could be applied by other institutions without difficulty. Based on our institution type (accelerated versus traditional) or available affiliations (academic medical center, major research university, etc), the options for topics, guest speakers, use of technology, and active-learning methodology could be expanded and diversified. An elective course could be designed with any number of topics and focus points to accentuate material best suited to the expertise and available resources of the faculty members and practice sites.

There were some limitations to our research. As a pilot study, it was principally hypothesis-generating in nature. The sample size was small; thus, the results cannot be extrapolated with confidence to other parts of the curriculum. Also, the competency test administered with the course was developed solely for research and evaluation purposes, independent of course grading and evaluation. While it was reviewed by school administrators prior to administration, the test was not an independently validated assessment instrument. An objective assessment of knowledge in this area would have been more useful for the study, but the lack of a widely accepted standard for basic critical care competency leaves this limitation more difficult to overcome. As mentioned previously, we believe that our initial goals were reached with sound research design. Continued research in this area will undoubtedly reveal effective evaluation methods for competency assessment and effective learning tools in the field of critical care pharmacotherapy. Our goal is to ensure students’ critical care proficiency upon graduation so that new graduates are effective professional pharmacists with sufficient critical care knowledge.

SUMMARY

The use of cooperative learning through group presentations and class discussions proved to be an effective method for teaching critical care topics to pharmacy students. Topics were identified using rigorous planning and evaluation and assimilation of available academic evidence rooted in pharmacy literature and guidelines, including the 2009 ACCP Pharmacotherapy Toolkit. Students were assessed for basic competency in critical care before and immediately following the course, with the study results identifying significant improvement, particularly when considering just the questions relating to material covered in the course. Students were satisfied overall with the course and provided positive and constructive feedback.