A Pharmacotherapy Capstone Course to Advance Pharmacy Students’ Clinical Documentation Skills

Abstract

Objective. To implement and assess the effectiveness of a capstone pharmacotherapy course designed to integrate in-class curriculum using patient cases and drug-information questions. The course was intended to improve third-year doctor of pharmacy (PharmD) students' clinical documentation skills in preparation for beginning advanced pharmacy practice experiences (APPEs).

Design. This 2-credit, semester-long course consisted of 6 patient cases and 12 drug-information questions posted electronically on an Internet-based medical chart, a public health presentation, a knowledge examination, and an objective standardized performance assessment. In class, students engaged in active-learning exercises and clinical problem-solving. Students worked outside of class in small groups to retrieve and discuss assigned articles and review medication information in preparation for in-class discussions.

Assessment. A rubric was used to assess the patient cases and questions that students completed and submitted individually. Data for 4 consecutive course offerings (n=622) were then analyzed. A significant improvement was found in the “misplaced” but not the “missing” documentation ratings for both assessment and plan notes in the final assessment compared with baseline. In course evaluations, the majority of students agreed that the course integrated material across the curriculum (97%) and improved their clinical writing skills (80.5%).

Conclusion. A capstone pharmacy course was successful in integrating and reviewing much of the material covered across the PharmD curriculum and in improving students’ clinical documentation skills.

INTRODUCTION

Colleges and schools of pharmacy must develop curricula to educate student pharmacists to use critical thinking skills while providing medication therapy management (MTM) and performing other aspects of pharmacy practice in order to meet accreditation requirements. The Commission to Implement Change in Pharmaceutical Education outcomes and the Accreditation Council for Pharmacy Education guidelines provide information on what is expected from pharmacy curricula.^1,2 The creation and assessment of activities that require critical thinking can be challenging, and critical thinking is difficult to define, often resulting in subjective assessment. The use of patient cases to provide opportunities for students to enhance their clinical thinking skills is common across health professions.^3,4 However, cases take a significant amount of time to create, and it can be challenging to write them in such a way that they target specific learning objectives. The answers students produce in solving cases also take a significant amount of time to evaluate because of the variability of responses and the challenge of defining a spectrum of correct and acceptable answers. The intent of using multifaceted cases (ie, patients with multiple conditions and medications) is to increase students’ comfort with the variability they will see in practice.^5-7

Capstone courses are used to integrate material across the curriculum, thus creating opportunities for students to incorporate knowledge learned from 1-dimensional information from diverse classroom experiences into multifaceted cases, which resemble realistically complicated patients. In pharmacy education, capstone experiences are generally situated in advanced pharmacy practice experiences (APPEs) or research experiences, rather than in a classroom course.^8-10 As a college of pharmacy founded in 1892, our pharmacy curriculum is a 4-year PharmD program that follows a traditional academic calendar. We are a dual campus program with approximately two thirds of the student body on the main campus, and one third located at the distance campus. The capstone course was devised to help integrate content and better prepare students for APPEs. As a result, this unique capstone pharmacotherapy course has been offered in the spring semester of the third year of the curriculum, immediately preceding the APPEs for the past 6 years. It provides an opportunity for student pharmacists to integrate prior coursework and hone their clinical documentation skills while solving complicated patient cases.¹¹ Students also learn and apply documentation principles in the third, fourth, and fifth semesters of our 5-semester Pharmaceutical Care Skills courses (skills laboratory). This course was envisioned to create a bridge to APPEs, including additional preparation with clinical documentation and analysis of complex patients to identify and resolve drug therapy problems. The learning objectives for the course included that the participants would be able to: (1) communicate drug therapy recommendations (verbal and written), (2) interpret medical literature and apply new knowledge to clinical situations, (3) retrieve and provide reliable drug information, (4) assess medication appropriateness and identify drug-related problems, (5) integrate clinical, humanistic, and economic data to develop a drug therapy plan, and (6) determine therapeutic endpoints and monitor medication regimens. Other skills that were reviewed included conducting a patient-specific medical and medication history, understanding different pharmacy practice models, recognizing the importance of pharmacy law and ethics in decision-making, evaluating outcomes of clinical interventions, and applying public health knowledge. In this paper, we describe and assess a capstone pharmacotherapy course designed to integrate in-class curriculum using patient cases and drug information questions.

DESIGN

This 2-credit course consisted of 6 patient cases, 12 drug-information questions, a public health presentation, and curricular assessment initiatives, including a knowledge examination and objective standardized performance assessment. To simulate the unpredictability of conditions that patients may have or what medication questions may arise in practice experiences or actual practice, cases and drug information questions were not limited to content delivered in the in-class curriculum. This design allowed students to research unfamiliar conditions and medications. The class met in person for a 2-hour class period every other week to debrief on each case and to delve into issues surrounding the case. Because a substantial portion of the curriculum prior to the course had been lecture-based, students embraced having the opportunity to actively engage with the content and clinical problem-solving.

To facilitate preparation for the in-class discussions, students were randomly assigned to groups of 4 and completed tasks assigned prior to the in-class discussion. Typical pre-discussion tasks included retrieving and interpreting assigned articles, looking up information about medications, and researching patient assistance programs for medications that were involved in the case. The faculty members or preceptors who created each case were present in class to facilitate various active-learning techniques used to engage the students, such as write/pair/share, jigsaw teamwork, and group quizzes.

Because this course was designed in an attempt to ensure students understood and applied principles of good clinical documentation and clear communication, all assignments required individually written submissions. Students were allowed to discuss and debate with their classmates the details of the patient cases, but they were required to complete and submit their own work. Allowing students to discuss and debate provides the opportunity to learn from each other and exposed them to different approaches to solving cases. The learning benefit of allowing discussion and debate outweighed the risk of students sharing answers. The cases in this class were fictionalized patients with multiple conditions, medications, and drug therapy problems to better simulate the complexity that students would likely encounter during APPEs. As a result, this course required many faculty members from across the college to contribute expertise and content for the cases.

Weekly assignments, which were posted electronically in a custom, internally built Internet-based medical chart, alternated between patient cases and drug-information questions.^12,13 Students submitted their weekly work using the same Internet-based medical chart, which allowed assignments to be electronically triaged to evaluators who provided feedback. Evaluators were randomly linked to student submissions for grading. Once a student and evaluator were linked, they remained paired for the duration of the semester. Assignment feedback was provided by resident or graduate-student pharmacists who were guided by a rubric that students received on the first day of class. Students had 6 days to complete the case or drug information questions, and evaluators had 6 days to grade assignments. Evaluations were released once all graders had completed their feedback, and students could log into the same system to view their evaluations.

Six patient cases involving several medical conditions and a variety of medications were designed for students to solve during the course. The decision to assign 6 cases was based on the length of the semester, the amount of time needed to grade and provide feedback to students, and the number of cases needed to provide students with enough experience in documenting. Each case was created by teams of 2 to 3 faculty members or affiliated preceptors, generally consisting of 2 members based on the main campus and 1 based on the distance campus. They were also responsible for facilitating the in-class discussions regarding their cases on their respective campuses. The case format varied to provide students practice with differing levels of patient information. Some cases were isolated encounters with a patient involving only the subjective and objective data from the day that the students were working with that case. Other cases required students to review several past notes from different providers to put together the whole patient story. While this simulation of information and Internet-based charts is not comparable to an electronic medical record in practice, it did require students to create timelines and to sequence events occurring in each case. Students were required to provide their assessment and plan documentation for each case. Faculty members were given guidance from the course director as to how complicated the patient case should be (eg, the number of conditions) and how many and what types of drug therapy problems should be targeted. For example, case 1 was intended to have 2 to 3 conditions and 2 to 3 drug therapy problems. Although participating faculty members were not guided on the condition severity or prognosis of the patients, most patient cases involved ambulatory adults with good prognoses. The course director served as the coordinator of the cases and ensured that a variety of conditions and drug therapy problems were covered.

EVALUATION AND ASSESSMENT

Evaluation of individual case notes allowed for feedback on how well students were meeting the course learning objectives. The cases required students to communicate written drug therapy recommendations, assess medication appropriateness and identify drug-related problems, develop a drug therapy plan, and determine therapeutic endpoints and monitoring parameters. Each student’s case note was evaluated for both structure and clinical content. The structure criteria were focused on whether elements were present and in the correct section (assessment and plan). The clinical content criterion, which was focused on pharmacotherapy management, was the mechanism used to provide students with feedback about objectives, such as setting correct goals and following up. The faculty case writers provided expectations for structure and clinical content and developed an evaluation key for each case. Once the notes were submitted, the course director reviewed 20-30 notes to sample how students approached the case and what unanticipated answers were given. The evaluation key was then revised to reflect trends in actual submissions, as students often identified issues that were unintended.

Case evaluators used the rubric to assess the structure and clinical content of each student’s submission, including rating the assessment and plan structure as “complete,” “misplaced,” or “missing” documentation. The required elements for the structure of the assessment included status of condition, identification of problem(s), goals for condition, and rationale for therapeutic decision(s). Required elements for the structure of the plan included specific recommendation(s) and what/when to follow-up. If any elements were missing, the evaluator assigned a “missing” documentation rating. A “misplaced” documentation rating was assigned if the assessment contained plan recommendations or if the plan contained rationale for the recommendations. Each condition was individually evaluated according to the clinical content criteria of the rubric. This aspect of the design provided consistency in feedback on each condition, rather than focusing on 1 condition that the student may have mismanaged and potentially neglecting to provide additional feedback on other aspects of the case. The more conditions the case involved, the more times the clinical content criterion was applied. For example, if the patient in the case had hypertension and diabetes, the clinical content criterion would propagate in the online system for both the assessment and plan for hypertension and for diabetes.

While the type and number of conditions involved in each case changed throughout the semester, the structure expectations remained constant, allowing comparative analysis. “Missing” and “misplaced” documentation ratings indicated that students were not meeting the course goals because the elements are fundamental for complete written clinical communication. The data from the 4 previous offerings are included in this analysis. Case 1 was selected as a measure of baseline performance as students began the course, and case 6 was selected as measure of performance at the completion of the course. To determine if student clinical documentation performance improved, structure ratings from case 1 and case 6 were compared using a mixed-effects multinomial logistic regression model that takes into account that outcomes within the same students are correlated (SAS 9.2, SAS Institute Inc., Cary, NC). This model requires specification of a reference outcome and predictor categories. The reference outcome was the rating “complete” for documentation and the reference predictor was case 1. The level of significance was set at p<0.05.

Student note data for the 4 previous course offerings were available (n=622). The frequencies of “complete,” “misplaced,” and “missing” documentation for cases 1 and 6 are presented in Table 1. For assessment and plan notes, the odds ratios (ORs) of having a “missing” vs “complete” documentation rating were 1.07 (p=0.65) and 0.86 (p=0.328), respectively, for case 6 as compared with case 1. Students were 7% more likely to have a “missing” documentation rating in their assessment and 14% less likely to have a “missing” documentation rating in their plan. Neither was significant, indicating that students, on average, were as likely to have “missing” vs “complete” documentation ratings for both plan and assessment toward the end of the course as they were at baseline. For the same notes, the ORs of “misplaced” vs “complete” documentation rating were 0.081 (p<0.0001) and 0.175 (p=0.024) for case 6 as compared with case 1. On average, students were 92% less likely to have “misplaced” documentation rating in their assessment and 82.5% less likely to have “misplaced” documentation in their plan in case 6 compared with case 1. The improvement in note structure was driven by an improvement in “misplaced” documentation ratings and was significant for both assessment and plan.

Table 1.

Ratings Assigned to Patient Assessments and Plans Completed by Pharmacy Students Enrolled in a Capstone Pharmacotherapy Course in 2008-2011 (n=622)

Of the students providing feedback through the end of the semester course evaluation, 97% strongly agreed or agreed that the course integrated material across the curriculum, 80.5% of students strongly agreed or agreed that the course improved their clinical writing skills, and 64% strongly agreed or agreed that the feedback provided was useful. The themes in student feedback on course evaluations have shifted over the years. In the first offerings of this course, students were focused on feedback and consistency in grading. There was confusion concerning why an answer was graded as incorrect if the complete information was provided in the notes but under the wrong section. This feedback resulted in a revision of the rubric to its current form, which provides more detailed criteria for structure and clinical decision-making. While the criticism of grading inconsistency is still articulated, the student feedback now includes comments about how this course should be offered across the curriculum to allow students to work on content integration. Students also agreed that the course successfully integrated material across the curriculum and improved their clinical writing skills.

DISCUSSION

This capstone course successfully integrated the classroom-based curriculum, provided students with practice thinking through complex patient cases, and held students individually accountable for their clinical documentation. An investigation of student performance on case 1 and case 6 demonstrated an improvement in performance primarily because of students being less likely to have misplaced documentation. The assessment method used for evaluating student clinical documentation provided a unique opportunity to identify and characterize the mistakes students made. Differentiating between missing and misplacing a required note element provides more depth to instructors’ and students’ understanding of performance. The rubric and Internet-based medical chart grading interface allowed evaluators to select the specific element(s) students were missing as a sub-rating check box (data not shown), in contrast to the common practice of awarding points to an assignment and the potentially difficult delineation of performance differences, for example, between 75 and 80 points.

When the course was originally designed, cases were intended to become progressively more complicated, with case 6 providing a scenario that represented an authentically complex patient with 5 to 6 conditions and 5 to 6 drug therapy problems. The goal was to challenge students and induce their creativity while they were practicing clinical skills and documentation; however, too much challenge can be counterproductive and detract from learning.^14-16 Informal discussions and comments on student course evaluations indicated that this structure did not provide the learning experience desired. By the end of the semester, students felt frustrated and defeated because they were not able to demonstrate mastery as the expectations continued to increase. This discovery resulted in rethinking how to sequence the complexity of the cases. Currently, the complexity increases with cases 1, 2, and 3, and is reset with case 4, increasing again for cases 5 and 6.

Challenges to this course design include resource requirements, logistical challenges, and ensuring adequate curriculum integration. Faculty members frequently participate in addition to their regular workload, which may include research and clinic responsibilities. Preceptors, who are not employees of the college, volunteer their time. The creation and maintenance of cases requires planning and resource allocation. While this is a faculty responsibility, it should be recognized as a time investment. Fortunately, not every case needs to be revised every year, as written cases can be surprisingly dynamic. The publication of a new guideline can radically change the answer to a case so that revisions may not be necessary. If faculty members are pleased with the effectiveness of the case, only modest revisions are made.

Evaluating clinical notes is also time intensive. In reviewing our grading logs, evaluators spent 10 minutes to 15 minutes per note to grade, resulting in an investment of approximately 40 hours for grading each case. Resources are required to hire the pharmacy residents who grade the notes on their own time. We have found that a reasonable grading workload is approximately 20 students per evaluator. Creation of an evaluation key providing clear guidance on how a case should be assessed is a challenge and can take a significant amount of time. While previewing student submissions takes time, it can improve grading efficiency by providing guidance to the case evaluators about students’ most frequent tangents.

With 160 submissions to grade for each assignment, grading consistency continues to be a challenge and a focus of continued efforts toward improvement. In an attempt to minimize grading inconsistency, the course director uses computer-screen capturing software to record discussions of cases and keys for evaluators to watch. The case evaluators use a Google doc to access the key and to track grading of questions and resolutions.

An earlier analysis of the first 2 years of data found a significant decrease in the number of “missing” documentation ratings assigned from case 1 to case 6.¹⁷ The expectation was that the change in student performance would be attributed to a change in “missing” documentation ratings. However, the current analysis revealed that improvement in student clinical documentation was a reduction of the likelihood that a student would have misplaced documentation. One possible reason for this unexpected finding is a change in the faculty members contributing to this course, resulting in a completely new case 6 for last year’s offering. This new case was significantly more complicated and the number of conditions and drug therapy problems that students needed to address increased from 5 to 9. Additionally, in the new version of the case, the patient was hospitalized, which proved to be quite challenging to the students.

There are practical limitations to the interpretation of this analysis. Missing required elements in students’ clinical documentation suggests failure to document key information about their patients, which could translate to less-than-optimal patient care. Clinical documentation instruction provided to students is prescribed and directed. It is structured to help students develop their clinical thought process as they work through a patient case. Students’ misplacing of elements may have learning implications, as documentation structure reflects the clinical thinking process, but it is unclear how misplacing elements might translate into less-than-optimal patient care. Students are held to a high standard, understanding that as they enter practice, they will be able to determine when there is flexibility in documentation styles. We anticipate that many preceptors may write their clinical documentation with their assessment and plan combined. These preceptors also may instruct and encourage their students to combine the assessment and plan. This limits the interpretation of student improvement in this study because it identified improvement that may not have practical consequences on APPE performance. An additional limitation of this data analysis was that only the first and last cases were used. As a result, there may have been trends in student performance over time that are not described in this report. A final limitation is the focus on the structure of students’ notes rather than the clinical decisions documented in their notes. Use of these data was a practical choice secondary to the reality that appropriate management of conditions is regularly changing, while good documentation principles remain relatively static. Future work has the potential to analyze students’ rationale for some insight into their decision-making. However, narratives are limited in that they are evidence of the output of clinical decision-making without actually revealing how the decisions were reached.

As we continue to reflect on the course and consider which of their skills students need to refine prior to APPEs, we are acutely aware that in APPEs and later in professional practice, students will not have 1 or more days to work-up a patient, identify drug therapy problems, and write their clinical documentation. One planned revision includes having students complete a case and a drug information question in a 1-hour time period administered as an online examination through Moodle (http://moodle.org) our course management system. A time-compressed activity may provide a mechanism to address a different facet of student case performance. Additional feedback from preceptors also needs to be gathered to determine what future revisions of the course may be necessary.

CONCLUSION

Students demonstrated improvement of clinical documentation skills through the cases used in this course. Clinical documentation is an essential skill for pharmacists that is further developed by students during APPEs. Implementation of this educational strategy was successful in providing curriculum integration and additional experience with complicated cases and clinical documentation in preparation for students’ subsequent APPEs.