An Objective Standardized Clinical Examination (OSCE) in an Advanced Nonprescription Medicines Course

Abstract

Objective

To add an objective standardized clinical examination (OSCE) to a nonprescription medication elective and assess the impact on students' knowledge, skills, and satisfaction.

Design

A nonprescription medicine elective was altered to incorporate more active learning and skill-assessment measures. Small group recitation sessions were added to review didactic material from a prior required nonprescription medicine course, and an objective standardized clinical examination was used to assess skills.

Assessment

Thirty-four students completed the 3-case OSCE with an average grade of 88%. The standardized patients expressed differences in their satisfaction with the student pharmacists' care by ranking the students' overall performance. Students' grades for the course and course evaluations were similar to the previous year.

Conclusion

The addition of the OSCE to the elective course provided students with an enhanced mechanism for evaluation of their self-care education and skill development.

INTRODUCTION

Fifty-nine percent of Americans have taken at least 1 nonprescription medicine within the last 6 months, and approximately $16.8 billion is spent on nonprescription medicines each year in the United States.^1,2 Nonprescription medicines and self-care are important aspects of pharmacy practice because pharmacists are generally the only health care practitioners who interact with patients regarding selection of these products. Yet due to the curricular constraints in many US colleges and schools of pharmacy, there is not a course focused solely on nonprescription medicine and self-care, rather this subject is delivered as individual topics in a variety of courses.³ With nonprescription medicine training dispersed throughout a curriculum, students may not have a chance to adequately practice the application of self-care prior to beginning their experiential coursework.

At the University of Arkansas for Medical Sciences College of Pharmacy, we offer a comprehensive nonprescription medicines curriculum including a core course and an elective course in nonprescription medicines. In addition, students wishing to gain practical experience in this area also may select an advanced pharmacy practice experience (APPE) in nonprescription medications within a community pharmacy setting.⁴

The UAMS College of Pharmacy has been using OSCEs to teach and evaluate students' communication skills since the late 1990s, including in our therapeutics course and physical assessment course. We believe the OSCE offers an opportunity to bridge the gap between students' factual nonprescription medicine knowledge and its practical application with patients. Typically, factual knowledge is tested using multiple-choice, short answer, and/or true-or-false questions to determine students' ability to recall what was taught. With an OSCE, the process of the student “pulling together” factual and clinical knowledge can be observed.

Although OSCEs have been used in a variety of pharmacy courses, their use in nonprescription medicine courses has not been well described. For one course, complete and partial standardized encounters between pharmacists and patients involving nonprescription medicines were videotaped, and then students had to view the vignettes and make a recommendation for treatment (complete encounter) or identify the missing information needed to make a recommendation (partial encounter). Although content covered in the encounters was similar to that in OSCEs, the video-based clinical examinations did not allow students to interact with patients.⁵

This report focuses on our experience in implementing nonprescription medicine OSCE cases into an elective nonprescription course to fill the gap between students' factual knowledge and practical application of self-care principles and assesses students' responses to this change.

DESIGN

The 2-credit-hour required second-year course in nonprescription medicines is taught by a team of 4 faculty members. The course covers the history of nonprescription medicine, and teaches topics related to self-care therapy, including 24 distinct areas of nonprescription medicine therapeutics. The course is delivered primarily as lectures and case-based discussions, with a few supplemental Web-based activities. Student assessment was performed primarily through examinations consisting of multiple-choice, true/false, and matching items, some of which are case based. Given the scope of nonprescription medicines and self-care, it is difficult to provide much depth in any one therapeutic area within the time limits of the course.

To address this limitation, an elective course was added in 2001, and is designed to allow students to delve deeper into the topics of nonprescription medicine. The 2-credit-hour elective course is taught to third-year students by the same faculty members who teach the core course. The elective course objectives include being able to triage a patient in a self-care setting, select appropriate non-drug treatments for applicable self-treatable diseases, and make product recommendations according to patient specific information (Table 1). Topics taught in the elective course include switching patients from a prescription to a nonprescription medication, nonprescription drug use in special populations, use of natural medicines in various disease states, and documentation of self-care cases.

Table 1.

Objectives for an Advanced Nonprescription Drugs Course

The course is taught using lecture format, a Jeopardy game show style of review, and case-based small group periods. In the small group periods, nonprescription medicine topics covered during the required second-year course are reviewed using a case-based format. The small group cases are distributed to the students in advance, but include only minimal information, (ie, basic demographic information and chief complaint). Prior to the small group meetings, the students are expected to prepare by reviewing the cases and therapeutic areas. Community pharmacy practice residents serve as small-group facilitators and role-play the patients in each case. During the small group sessions, students ask the role-players questions utilizing the QuEST SCHOLAR method to elicit further patient-specific information needed to make an appropriate recommendation. The QuEST SCHOLAR method (Appendix 1) was used because it is a structured format for obtaining information from a patient before providing appropriate patient counseling.⁶ After the question-and-answer role-play, students were split into groups and the residents assisted with subjective, objective, assessment, and plan (SOAP) note development for each case. Finally, 1 student from each group presented their assigned case to the class and the residents facilitated a discussion of the “correct” answer for each case.

The OSCE comprised 33% of the elective course grade assignment. The other 2 portions of the grade were a didactic examination given earlier in the semester and the average grade for the SOAP notes prepared during small group periods. Each of these is also worth 33% of the final grade. The OSCEs took place in UAMS's Center for Clinical Skills Education (CCSE), which is a state-of-the-art facility used for case development, consultations, assessment, and teaching of clinical skills. The CCSE consists of 14 examination rooms, a recording room, and an observation room where the encounters can be viewed in real time.⁷ Standardized patients were recruited from our community by the CCSE and paid a stipend for their time. Each standardized patient graded the student in real time using detailed objective rubrics developed for the individual cases. The encounters between the students and standardized patients were digitally recorded and made available to course faculty members for viewing in order to verify grades or settle disputes about how well a student performed.

EVALUATION AND ASSESSMENT

The OSCE cases for the final examination were developed by individual faculty members and residents who were involved in teaching the elective course. The cases were then subjected to peer-review by all 4 faculty members. Changes suggested through the review process then were incorporated into the cases and reviewed a final time for correctness and conciseness.

Three cases were developed for the examination. Topics covered by the cases were headache, constipation, and smoking cessation. The cases were written narrowly enough to allow the student only 1 correct conclusion. In contrast to the small group cases, students were given no information about the cases prior to arriving at each individual station. In actual pharmacy practice, during patient encounters the pharmacist has minimal background data with which to work because most of the information that forms the basis of their recommendation comes from the patient. Thus, to replicate practice, the information provided to the student prior to entering the station was purposely limited.

Each case consisted of directions for the student, directions for the standardized patient, and a grading rubric. The cover page for the cased stated the station name, the task to be completed, station requirements, time limits, type of standardized patient with whom the student would be interacting (physician or patient), the author's name, and the date the case was created.

The directions to the student were on a separate page with the specific task they were to accomplish listed in bold font. Other pertinent information the student needed before entering the station was also listed on this page. The students were allotted 15 minutes for each station and could allocate the time as they deemed necessary. If they allocated too much time to reviewing the products available to treat the condition prior to entering the patient room, they might not have sufficient time to narrow down their recommendation to the most appropriate option based on patient-specific information. However, for this first experience with the new timing, if a student spent longer than 5 minutes preparing for the interaction, one of the faculty proctors encouraged the student to enter the patient room so that no one ran out of time. This approach was used to replicate practice and to teach students the importance of time management in a community setting.

Prior to the examination the standardized patients were provided detailed training in the CCSE on their respective case and on how to interact with students. The training included a practice experience on the case where faculty members role-played as the students. To give the standardized patient experience in the type of students they would encounter, and ensure consistency in standardized patient encounter assessments, the faculty role-played how an excellent, average, and below average student might perform. After the practice sessions, the performance was reviewed and discussed by all standardized patients and faculty members. The standardized patients were encouraged not to provide more information than the students requested. For example, if the student asked a closed-ended question, one that could be answered yes or no, the standardized patient was trained to answer only yes or no. Students were required to phrase questions in a way that allowed them to obtain all pertinent information from the standardized patient. If they did not obtain all pertinent information, their grade would reflect this. However, in order to prevent a student from missing a critical piece of information, occasionally specific prompting questions from the standardized patient were allowed. For example, on the constipation case, if the student only recommended increasing daily fiber intake, then the standardized patient asked the student to recommend a product. If a prompt was allowed, it was printed in the standardized patient's instructions and on the grading rubric.

EVALUATION AND ASSESSMENT

The grading rubric was a checklist of specific items on which the student was marked pass or fail. Each rubric contained 8 to 15 items related to clinical knowledge about the case. In addition, each rubric contained 9 communication items on which the student was marked pass or fail. Standardized patients also were asked to provide their “overall impression of student performance” by choosing a designation of outstanding, clear pass, borderline, or clear failure. This question allowed the results to be normalized if necessary. In addition, the standardized patients were asked to indicate whether they would return to this particular student pharmacist based on the encounter. However, neither of these global impressions counted towards the student's grade on the case.

Thirty-four students completed the elective. The average grade on the OSCE final was 78%, with scores ranging from 91% to 61%. The grades on the OSCE were not curved because the overall average grade for the course was 88%. Only 3 students failed the OSCE with grades less than 70%; however, because their grades in the other 2 parts of the course were high, no students failed the course. The final grades for the course were not significantly different than the previous year's overall class average of 89%. All students made a final letter grade of either A or B in the course both years.

Student performance on each case was similar. The average scores were 79%, 80%, and 77% for the constipation, smoking cessation, and headache cases, respectively. However, there was a difference in answers to the questions pertaining to the overall impression grade from the standardized patients between the cases. Differences were noted on the question, “Would you return to this pharmacist?” On the constipation case, 94% of the standardized patients responded positively, and on the smoking cessation case, 81% responded positively. However, for the headache case only 56% said they would return to the pharmacist. Figure 1 shows that the differences in patient satisfaction were related to the overall performance of the student. The individual standardized patients rated the student's performance as clearly passed or outstanding in 74% of the constipation cases, 62.5% of the smoking cessation cases, and 35.3% of the headache cases. The performance of approximately 20% of students was rated by the standardized patient as a “borderline” level for the constipation and smoking cessation cases. However, on the headache case, 56% of students were rated at a borderline level. The smoking cessation case had the highest percentage of student performance (19%) being rated as “clearly fail.”

Figure 1.

Overall impression scores on individual cases.

Overall, student assessments of the course were good (Table 2). Items were rated on a 5-point Likert scale (5 = strongly agree and 1 = strongly disagree). The course was evaluated by students using the same standardized evaluation form used in all didactic courses in the college. The median course evaluation score was 4, the same as for the previous year when the small group periods were added. Scores for the 2 years were slightly lower than scores in 2006, the last year the elective had been offered in lecture-only format.

Table 2.

Advanced Nonprescription Course Student Evaluations^a

^aScale: 5 = strongly agree; 4 = agree; 3 = neither agree or disagree; 2 = disagree; 1 = strongly disagree.

Students thought the course was beneficial, with several stating that recitation was one of the most valuable parts of the course. One student commented: “Got much more out the small group recitation periods than I thought I would.” Several students also mentioned they appreciated the OSCE examination. One student stated: “As much as I dislike counseling patients (because of nerves), I really need the practice.” Another student commented: “Overall, the entire course was valuable because it reviews everything that was important from last semester and included quite valuable subjects such as pregnancy.”

DISCUSSION

Given the current economic and health care challenges in the United States, the decreased emphasis on nonprescription medicine in some curricula is particularly troublesome. People may put off going to the doctor and therefore try to self-treat more often in an attempt to save money and time. Pharmacists are extremely accessible and often the first health care professional that people consult. Given this, training student pharmacists about nonprescription medications is possibly more important now than in the past.

Nonprescription medication counseling differs from prescription counseling primarily in that the pharmacist is responsible for triaging the patient to determine an appropriate course of action. Sometimes this results in a product being recommended, but at times the situation requires referral to a physician or the emergency department because it is beyond the pharmacist's scope of practice. Using an OSCE for the final examination in the elective course demonstrated that for the constipation and smoking cessation cases, the students made a correct diagnostic decision prior to making an appropriate recommendation. However, based upon the overall impressions of the standardized patients in the headache case, students had difficulty recognizing when to refer a patient who was beyond their capability to treat.

In the OSCE case, the patient presented with a headache of more than 12 hours duration. Upon questioning by the student, the standardized patients gave the students symptoms that indicated a migraine and, if asked, informed the student pharmacist that they had never been diagnosed as having migraines. Furthermore, the standardized patient stated the pain was “the worst headache they had ever experienced.” The information provided by the standardized patient should have led the students to immediately refer the patient either to a physician or an emergency department for diagnosis and treatment, rather than offer a recommendation for treatment. However, only 27% of the students referred the patient for immediate medical attention.

While faculty observers were “spot checking” students' live performances, they noted that in the headache case most students failed to refer the patient to a doctor or hospital. However, when grades from the standardized patients were reviewed, more than 80% of the individual standardized patients gave the students credit for referring them to a physician. Upon review of the recorded encounters, faculty members were able to determine that some of the standardized patients had interpreted “referral” to mean any reference to follow-up with a doctor. Usually the students' referral was in the context of instructing the standardized patient to follow up with a physician if the therapy recommended failed to alleviate the patient's headache as opposed to the faculty's expectation of an immediate referral.

Additionally, because a large number of students scored borderline on this case, it is likely that the standardized patient accounted for the timing of the student referral with this overall impression rather than on an individual assessment item. The discrepancy between the standardized patient's definition of “referral” and that of the faculty members will be addressed in future patient training sessions.

For this elective course, the items on the grading rubric for the OSCE were not weighted. All items were of equal value whether the item was related to therapeutics or communications. This is different than the method used for calculating grades in the OSCE for the college's therapeutics course where items related to therapeutics are worth more points than communication items. The faculty for this course felt that for nonprescription medication counseling, the communication section of the grading rubric was as important as the therapeutic section and, therefore, should not be weighted differently.

The results from this experience are based on a relatively small number of students and may not be applicable to other pharmacy education settings. Also, the lack of state of the art facilities, such as the CCSE available at our college, might make this experience harder to replicate at another institution.

The data presented from the elective nonprescription medicines course described in this manuscript shows that students learned much of the material presented to them. It also reveals that more emphasis needs to be placed on when it is appropriate to refer patients seeking nonprescription medications. Faculty in all nonprescription courses at the college will try to ensure that students understand that not making a recommendation for product is sometimes the most appropriate recommendation. Students found value in both the small group recitation periods and the OSCE assessment, even though at times they felt uncomfortable during the process. Also, faculty members will use the lessons learned in training standardized patients to improve the experience for future students. Faculty members were delighted to see that although the elective may have required more effort on the part of the students, such effort did not negatively impact the students' assessment of the course.

SUMMARY

Small group recitation periods were incorporated in a nonprescription medicine elective to make learning more interactive. Also, an OSCE to assess student pharmacists' counseling skills on nonprescription medicines was added to the course. Colleges of pharmacy have to be able to quantify students' learning to assure that the competencies addressed by the course were actually learned and can be applied by the students. Small group recitation periods and OSCE assessment are 2 good methods that build upon each other to ensure students have learned the material that was taught.

Appendix 1. Brief overview of QuEST SCHOLAR method (from: Leibowitz K, Ginsburg. Chantilly, VA: Proceedings of the APhA Inaugural Self-Care Institute; 2002. Counseling self-treating patients quickly and effectively. May 17-18.)

• Quickly and accurately assess the patient

• Establish that the patient is an appropriate self-care candidate

• Suggest appropriate self-care strategies

• Talk with the patient

Use SCHOLAR to ask the patient about their current complaint:

• Symptoms

• Characteristics

• History

• Onset

• Location

• Aggravating factors

• Remitting factors