An Advanced Objective Structured Clinical Examination Using Patient Simulators to Evaluate Pharmacy Students’ Skills in Physical Assessment

Abstract

Objective. To implement an advanced objective structured clinical examination (OSCE) in the curriculum and to evaluate Japanese pharmacy students’ skills in physical assessment such as measuring pulse and blood pressure, and assessing heart, lung, and intestinal sounds.

Design. An advanced OSCE was implemented in a hospital pharmacy seminar as a compulsory subject. We programmed patient simulators with 21 different patient cases in which normal and abnormal physiological conditions were produced. The virtual patients were then used to evaluate the physical assessment skills of fifth-year pharmacy students.

Assessment. Significant differences were observed between the average of all the detailed evaluations and the mean results for the following skills: pulse measurement, blood pressure measurement, deflating the cuff at a rate of 2-3 mmHg/sec, listening to heart sounds, and listening to lung sounds.

Conclusion. Administering an advanced OSCE using virtual patients was an effective way of assessing pharmacy students’ skills in a realistic setting. Several areas in which pharmacy students require further training were identified.

INTRODUCTION

The principal duties of pharmacists in Japan are to inspect and fill prescriptions received from physicians, and to provide patients with instructions for taking their medicine. Historically, unlike doctors and nurses, Japanese pharmacists generally have not conducted physical examinations of patients to evaluate patient condition. There is a new awareness, however, that verifying vital signs and conducting physical assessments help pharmacists assess drug efficacy and check for side effects. In April 2010, the Health Policy Bureau of the Ministry of Health, Labour and Welfare issued a notification on “the promotion of team medical care through the cooperation and coordination of medical staff,” in which specific examples of possible duties of pharmacists were outlined, such as verifying the onset of side effects.¹ One such example was “to perform pharmaceutical management (grasp of the situation of the side effect to the patient receiving the medicine instructions).” These instructions are intended for all patients including confined patients and those staying at home. Furthermore, in October 2010, the Japanese Society of Hospital Pharmacists published a response to the bureau’s notice. The response described the practice of pharmacists performing physical assessment and evaluating physical condition of patients receiving pharmacotherapy while accompanying doctors on their hospital rounds and at conferences.² This practice has been implemented by many hospitals and community pharmacies in Japan, and is now considered a standard responsibility of pharmacists.³ Duties include checking vital signs and conducting physical assessments from a pharmacotherapeutic perspective to determine drug efficacy and detect side effects early.

In 2006, Japanese pharmacy education shifted from a 4-year to a 6-year pharmacy curriculum, and included the introduction of objective structured clinical examinations (OSCE) in 2009 as part of the common achievement test administered to pharmacy students late in the fourth year.⁴ In the 6-year pharmacy curriculum, the students must pass the common achievement test in the fourth year before they start their practical training in the fifth year. The test is used universally by pharmacy schools throughout Japan. As part of the pharmaceutical common achievement test, the OSCE has been taken by approximately 10,000 students annually from 2009 to 2011, with an achievement rate of 96%-99% on the initial test, and for those unable to pass the initial assessment, 99%-100% on reexamination.⁵ In addition to the OSCE, computer-based testing (CBT) is available as an objective evaluation of certain basic academic skills. The OSCE comprises 6 tasks in 5 fields including: (1) serving a patient/customer at a pharmacy, (2) preparing a medicine, (3) performing a preparation audit, (4) demonstrating aseptic techniques, and (5) providing information to patient/customer. These are the basic duties of a pharmacist; however, physical assessment-related items are not included in the OSCE.

We believe that skills in monitoring vital signs and physical assessment required for the early detection of side effects, as well as skills in checking the efficacy of pharmacotherapy, should be objectively evaluated to confirm they have been acquired through training (educational programs). At the School of Pharmaceutical Sciences at the Kyushu University of Health and Welfare, evaluations of such skills have been incorporated into bedside training in the fourth year and seminars in hospital pharmacy in the fifth year.^6-8 To objectively evaluate students’ acquisition of skills in physical assessment at our school, we implemented an advanced OSCE trial.⁹ It underwent subsequent revision and was reimplemented.

This paper describes what the advanced OSCE revealed about physical assessment skills acquired through our program, including a few problem areas. For example, we found that while pharmacy students had identified heart sounds heard with a stethoscope, they had not assessed the mechanism of generating abnormal sounds and its subsequent course in a comprehensive manner. Moreover, the advanced OSCE revealed that difficulties associated with physical assessment could be lowered through better understanding of patient simulator characteristics.

DESIGN

An advanced OSCE was conducted as the final examination for a required seminar in hospital pharmacy course. The practical examination involved conducting a physical assessment in a hospital ward and was divided into 4 major steps: introduction, physical assessment, closing, and communication. Physical assessment involved pulse measurement, blood pressure measurement, listening to heart sounds, listening to lung sounds, and listening to intestinal sounds. The virtual patient model Physiko (Kyoto Kagaku Co., Ltd., Kyoto, Japan) was used to conduct the examination. The students used an aneroid sphygmomanometer to measure blood pressure. The advanced OSCE task lasted 13 minutes. The tasks were conducted in stations consisting of 3 groups each. One instructor was assigned to each group to act as an evaluator. The day before the advanced OSCE, the instructor demonstrated each task of the advanced OSCE as part of the course. In addition, examinees were allowed to read the list of tasks 1 minute before beginning the advanced OSCE (Figure 1).

Figure 1.

The list of tasks in the advanced objective structured clinical examination.

By the time they took the advanced OSCE, students had spent approximately 20 hours undergoing bedside training and completing seminars taught by doctors and pharmacists, which included lectures on physical assessment and ways to use a stethoscope and sphygmomanometer. In the seminar, heart, lung, and intestinal sounds were measured as follows: heart sound-normal (pulse 48, 60, 84, 108 bpm), third heart sound gallop, fourth heart sound gallop, aortic valve stenosis, atrial fibrillation, third and fourth heart sound gallop,* mitral valve insufficiency,* ventricular fibrillation;* lung sounds-normal, abnormal bronchial sounds, loss of right lung sound, wheezing, fine crackles, bubbling rale,* attenuation of left lung; intestinal sounds-normal, hyperactive sounds, hypoactive sounds, subileus,* ileus.* Items with an asterisk were not covered in the advanced OSCE.

The content for the advanced OSCE was created so that each case included 3 or more abnormalities involving pulse, blood pressure, heart, lung, and/or intestinal sounds (Table 1). Examinees were not given any information regarding how many of the questions would involve assessing normal conditions and how many would involve abnormal conditions. The cases were assigned sequentially to make the number as equal as possible.

Table 1.

Original Cases Used in the Advanced Objective Structured Clinical Examination

Evaluations used in the advanced OSCE are listed in Table 2. To state the purpose of the visit and obtain consent, examinees were instructed to say “I’ve come to check your physical condition to ensure the medicine is being accurately used. Is this a good time?”

Table 2.

Results of Detailed Evaluations of Physical Assessment in a Hospital Ward

Because body temperature measurement, respiratory rate measurement, electrocardiogram monitoring, and arterial oxygen saturation measurement could not be taken using the Physiko virtual patient, examinees were evaluated based on how appropriately they verbally communicated their intentions to the patient (ie, asking “May I use this portable electrocardiograph to take an electrocardiogram?” and saying “I’ve finished” afterwards).

For the practical tests that could use the virtual patient (pulse measurement, blood pressure measurement, and listening to heart, lung, and intestinal sounds), examinees were instructed to first ask the patient’s permission but were evaluated on whether they correctly or incorrectly conducted each test. The detailed evaluation of pulse measurement included being able to discern the difference between the left and right of the radial artery or carotid artery and being able to check the index, middle, and ring fingers. The students also checked for a normal pulse (actual rate=60 bpm; considered correct if response was within the range of 54 bpm - 66 bpm), tachycardia (actual rate=108 bpm; considered correct if response was within the range of 97-119 bpm), bradycardia (actual rate=48 bpm; considered correct if response was within the range of 43 bpm-53 bpm), and arrhythmia in atrial fibrillation (actual rate=84 bpm; considered correct if response was within the range of 76 bpm-92 bpm). The detailed evaluation of the patient’s blood pressure included being able to listen to the brachial artery and being able to deflate the cuff at a rate of 2-3 mmHg/sec. Additionally, as a practical test, the examinees assessed normal blood pressure (124/60 mmHg; correct within the range of 114-134/50-70 mmHg), hypertension (172/94 mmHg; correct within the range of 162-182/84-104 mmHg), and hypotension (82/52 mmHg; correct within the range of 72-92/42-62 mmHg). The detailed evaluation of skills for listening to heart sounds included being able to listen to the aortic, pulmonary, tricuspid, and mitral valve areas at the correct locations and being able to use the stethoscope diaphragm and bell properly (ie, checking the aortic, pulmonary, tricuspid, and mitral valve areas with the diaphragm; and the tricuspid and mitral valve areas with the bell). This test also included an evaluation of normal and abnormal sounds (atrial fibrillation, third heart sound gallop, fourth heart sound gallop, and aortic valve stenosis). The evaluation of skills for listening to lung sounds included being able to listen to the trachea, upper lung field, and lower lung field with the stethoscope in the correct position symmetrically from top to bottom and left to right and being able to listen to the front and back of the chest. This test also included an evaluation of normal and abnormal sounds (loss of right lung sound, abnormal bronchial sounds, fine crackles, and wheezing). The assessment of skills for listening to intestinal sounds included evaluating normal, hyperactive, and hypoactive sounds. To complete each evaluation item, students were required to say “the pulse is…,” “the blood pressure is…,” “I heard…heart sounds,” “I heard…lung sounds,” and “I heard…intestinal sounds.”

For the closing remarks section, students were required to say “Thank you very much. Do you feel ok? If you have any concerns please let us know. Take care.” In terms of behavior during the examination, students were evaluated based on interactions, such as gently guiding the patient into a sitting position after asking “Would you mind sitting up, please?”

A score of less than 70 points out of 100 was considered failing and meant the students would have to undergo reexamination. Ten points were allocated to each of the 5 practical test items and 2 points were given to each of the 25 specific evaluation items.

EVALUATION AND ASSESSMENT

Forty-nine students passed and 6 students failed the first advanced OSCE. In the 6 cases of failing, students got 3 or more wrong out of 5 practical examination items. These mistakes made the difference in decisive results. Students who were to take the examination again received an intensive lecture and practical exercise after the first examination attempt. Then, they took the examination a second time and all students passed. In the second examination, 2 students were questioned about atrial fibrillation, and both were able to identify arrhythmia.

A significant difference was observed between the achievement rate (percentage of students receiving an acceptable score) on the advanced OSCE of 91.7% and achievement rates on the following specific areas: pulse measurement, 67.3%; blood pressure measurement, 80.0%; ability to deflate the cuff at a rate of 2-3 mmHg/sec, 50.9%; listening to heart sounds, 56.4%; and listening to lung sounds, 72.7% (Table 2), respectively (p<0.05).¹⁰ It was revealed that 4 practical tests items (pulse measurement, blood pressure measurement, listening to heart sounds, and listening to lung sounds) and 1 specific evaluation item (ability to deflate the cuff at a rate of 2-3 mmHg/sec) in particular were difficult. The achievement rate for the pulse measurement practical test was 92.9% for normal (60 bpm), compared to 44.4% for tachycardia (108 bpm), 66.7% for bradycardia (48 bpm), and 11.1% for arrhythmia (84 bpm). The achievement rate for the blood pressure measurement practical test was 84.0% for 124/60 mmHg, compared to 68.8% for 172/94 mmHg, and 85.7% for 82/52 mmHg. Also, the achievement rate for listening to heart sounds was 38.9% for normal, compared to 88.9% for atrial fibrillation, 22.2% for third heart sound gallop, 80.0% for fourth heart sound gallop, and 66.7% for aortic valve stenosis. In addition, the achievement rate for test listening to lung sounds was 46.7% for normal, compared to 100% for loss of right lung sound, 75% for abnormal bronchial sounds, 80% for fine crackles, and 77.8% for wheezing. Furthermore, the achievement rate for the practical test listening to intestinal sounds was 83.3% for normal, in contrast to 100% for hyperactive sounds, and 62.5% for hypoactive sounds. In terms of normal vs abnormal sounds, 38.9% of students were able to assess normal heart sounds correctly, 100% of students were able to assess atrial fibrillation correctly as abnormal. Similarly, 77.8% correctly identified the third heart sound gallop as abnormal, 90.0% identified fourth heart sound gallop as abnormal, and 88.9% identified aortic valve stenosis as abnormal. Further, 46.7% of students were able to identify normal lung sounds, 100% of students could assess loss of right lung sound as abnormal, 75% were able to identify abnormal bronchial sounds, 90% were able to identify fine crackles as abnormal, and 100% were able to identify wheezing as abnormal. Moreover, 83.3% of students identified normal intestinal sounds, 100% identified hyperactive sounds as abnormal, and 93.8% identified hypoactive sounds as abnormal.

DISCUSSION

There has been keen interest regarding the introduction of OSCEs into pharmacy education and many countries have introduced this form of assessment into the curriculum.^11-13 However, few colleges and schools of pharmacy conduct OSCEs in an optimal manner, and most do not adhere to best practices in OSCE construction and administration.¹⁴ Further, schools have introduced simulators into pharmacy education to conduct specialized education, such as interprofessional team education and specialized pediatrics subjects.^15,16 Skills in blood pressure measurement and emergency medical care are also being evaluated using virtual patients.^17-19 However, a few reports have evaluated pharmacy students’ ability to conduct physical assessments using patient simulators.²⁰

In our study, simulators were used to analyze pharmacy students’ ability to perform patient assessments, including interaction with patients. It is particularly important to evaluate the extent to which students are able to assess normal and abnormal conditions using correct methods to measure pulse and blood pressure, and to listen to heart, lung, and intestinal sounds, as feedback can be given on these results immediately. Based on our analysis of the detailed evaluations, we confirmed that there were some items in the advanced OSCE that pharmacy students could not accurately assess. Of the 5 practical tests examining correct vs incorrect answers, a significant difference was observed in the mean detailed evaluations of 4 items: pulse measurement, blood pressure measurement, listening to heart sounds, and listening to lung sounds. No significant difference was observed in students’ mean achievement rate for listening to intestinal sounds. On the other hand, in the detailed evaluations, a significant difference was observed in students’ achievement rate for deflation of the cuff at a rate of 2-3 mmHg/sec. Because significant differences were found in tasks implemented in previous advanced OSCEs we had conducted,⁹ students in this study were carefully instructed on how to perform in these areas. Nevertheless, the same results were achieved. To improve performance in the future, we believe students need to learn how to make minute manipulations to the screw part of the rubber bulb of the sphygmomanometer.

Physiko models were configured so that arrhythmia (84 bpm) was synchronized with the atrial fibrillation heart sound program. The achievement rate for students correctly identifying atrial fibrillation was 88.9%, therefore, we assumed they could detect the onset of the atrial fibrillation, which often results in causing arrhythmia. However, arrhythmia (84 bpm) had the lowest achievement rate in the examination. This suggests that students only distinguished sounds heard using the stethoscope and they did not think in a comprehensive way about the mechanism behind the production of abnormal sounds, nor did they understand that atrial fibrillation could progress to arrhythmia. In the future, students should be taught not only to assess the patient’s heart sounds and pulse, but also to question why an abnormality is present and make their assessment accordingly.

Blood pressure varies and can be affected by symptoms and side effects the patient is experiencing. Therefore, these variations have to be identified correctly by pharmacists. In 4 out of 5 students who misinterpreted hypertension values, the systolic blood pressure measurement was incorrect. Although students were not asked to conduct pulse palpation in this examination because of time constraints, pulse palpation should be included in blood pressure measurement. Accordingly, we concluded the absence of pulse palpation method was associated with the tendency of students to measure systolic blood pressure incorrectly. However, the achievement rate for students’ ability to deflate the cuff at a rate of 2-3 mmHg/sec was also low, suggesting this was a skill with a high degree of difficulty.

Atrial fibrillation has a subtle abnormal sound caused by early irregular contraction of the atrium. Aortic valve stenosis has a distinctive whooshing sound, making it relatively easy to identify. However, while the fourth heart sound gallop is considered highly difficult to confirm, students had a higher rate of achievement for identifying it than for identifying aortic valve stenosis. On the other hand, we found students had a high degree of difficulty in identifying both normal and third heart sound gallop. Komatsu and colleagues reported that clinical training using a patient simulator for fifth-year medical school students following an OSCE common achievement test was effective because 96% of students were able to accurately use a stethoscope (only the diaphragm surface was evaluated); whereas only 45% were able to accurately distinguish heart murmurs, and less than 10% were able to identify an apex beat/chest wall pulsation, split second heart sound, and third/fourth heart sound gallop.²¹ Thus, based on the high degree of difficulty these skills posed for medical students, we concluded that these skills required repeated training for pharmacy students.

In our study, approximately 80% of students identified abnormal sounds as abnormal; however, approximately 60% of students assessed normal sounds as abnormal. Although diagnosing a new disease is not a pharmacist’s responsibility, pharmacists are required to conduct a differential diagnosis and, therefore, must be able to identify the difference between normal and abnormal sounds. We instructed students about the significance of an irregular rhythm and underlying mechanism of abnormal sounds. However, the low accuracy rate of students’ assessment of arrhythmia in atrial fibrillation suggested that they simply distinguished sounds heard using the stethoscope, and did not think in a comprehensive way about the mechanism behind abnormal sounds or subsequent progress toward arrhythmia. This lack of higher-level thinking was the greatest problem the advanced OSCE revealed, so educational methods need to be improved to address this issue.

When abnormal bronchial sounds were reproduced on the Physiko model, the sound could only be confirmed by listening to the dorsal lower right lung area. Because the abnormal bronchial sounds could not be identified from the front of the chest, the skill involved had a high degree of difficulty but was showed to be easier than identifying “normal” sounds. In this respect, students were introduced to the peculiarity of using patient simulators during auscultation training. Moreover, a disadvantage of using the patient simulator is that some cases believed to have a high degree of difficulty are easily assessed. Also, the reason that loss of right lung sound was identified by 100% of students may have been because it could be easily checked by comparing left and right lung sounds. We included these issues in a summary at the end of the seminar in hospital pharmacy. Despite these issues, we believe the students acquired proper physical assessment skills of lung sounds.

The virtual patients were programmed so that hypoactive sounds were simulated by producing 1 gurgle every 70 seconds. When normal, the model was configured so that 1 gurgle sound could be heard every 15 seconds; when simulating hyperactive sounds, the gurgling sounds were continuous. Thus, sounds could be identified by placing the stethoscope on the abdomen for 30 seconds; however 5 of the 6 students who misinterpreted hypoactive sounds had given the ileus or subileus as their answer.As for the ileus and subileus sounds with the virtual patient, they could not be confirmed for 5 minutes and more. Therefore, the time that the stethoscope was used may have been a deciding factor in accurate assessment.

SUMMARY

In this study, we implemented a new advanced OSCE in a hospital seminar course using a patient simulator and examined the educational outcomes. Significant differences were observed between the average of all the detailed evaluations and the mean results for the following skills: pulse measurement, blood pressure measurement, deflating the cuff at a rate of 2-3 mmHg/sec, listening to heart sounds, and listening to lung sounds. Several areas in which pharmacy students required further training were identified. However, administering an advanced OSCE using virtual patients was an effective way of assessing pharmacy students’ skills in a realistic setting.