The Impact of a Standardized Patient Program on Student Learning of Communication Skills

Abstract

Objective

Determine the quantitative and qualitative value of a lecture-laboratory course with standardized patients on student communication skills.

Methods

A blinded retrospective analysis was conducted on the counseling tapes of 127 students who took a lecture-laboratory course with standardized patients. A Communication Skills Assessment Form (CSAF) was used to evaluate baseline, midpoint and final tapes. Descriptive statistics and repeated measures analysis of variance were used to compare tapes. Students and standardized patients completed written survey instruments evaluating the laboratory at the end of the semester.

Results

Students had significant and progressively higher scores on the assessment across baseline, midpoint and final time points (p ≤ 0.001). Students had significantly higher final assessment scores across all subsections than at baseline (p ≤ 0.01). Students and standardized patients were favorable towards the laboratories and made useful recommendations.

Conclusions

A lecture-laboratory course with standardized patients had a significant impact on student communication skills across time and was well received by students and standardized patients.

INTRODUCTION

A patient-centered care philosophy is an integral part of providing optimal pharmaceutical care.¹ Patient-centered care requires the mastery of many important communication skills between a pharmacist and patient. Improving patient education and monitoring, providing support, and enabling patient feedback has a significant impact on a variety of health outcomes.^2-4 The American Association of Colleges of Pharmacy's Center for the Advancement of Pharmaceutical Care (CAPE) recognizes the clear and significant role of communication on patient care by setting guidelines that pharmacy graduates demonstrate an ability to communicate and collaborate with other healthcare professionals.⁵ These guidelines do not stipulate how and what communication skills should be taught. The Accreditation Council for Pharmacy Education (ACPE) does provide recommendations about communication skills that should be emphasized, such as interviewing techniques, active listening and empathy, assertiveness, and problem-solving.⁶ It is up to the schools of pharmacy to choose appropriate topics and methods in the delivery of this knowledge and practice in skills.^5,6 As with the CAPE document, ACPE standards do not require that any of these topics be taught according to a particular didactic method.⁷

Pharmacy schools often teach the following communication topics: (1) how to initiate communication, (2) establishing a trusting relationship, (3) using effective nonverbal communication, (4) eliciting information from the patient, (5) initiating educational interventions, (6) promoting adherence to appropriate drug therapy, (7) encouraging patient involvement in communication and problem solving, (8) organizing the encounter, (9) identifying cross-cultural communication issues, (10) verifying patient understanding, and (11) ending the encounter.^6,8,9 Current methods in teaching communication also include didactic learning, role playing, and the use of standardized patients. Role playing with other students or faculty members and the use of standardized patients typically involve more active learning than do didactic lectures since students are more likely to be learning by doing rather than passively receiving course content. Active learning has been defined as a “style of teaching that requires the learner to formulate answers to questions based on acquired knowledge while continuing to search for new knowledge that may provide better, more complete answers.”⁷

The techniques of role playing with other students or staff members and the use of standardized patients present different types of active-learning experiences for students learning communication skills.¹⁰ Role playing is when someone is given a role to act out such as that of a patient or physician.¹⁰ Role playing that occurs in the classroom often involves students taking turns portraying a pharmacist or a patient in a brief interaction. Sometimes role playing may involve students portraying the pharmacist and course staff members portraying the patient. The disadvantages of this approach include the lack of useful and/or critical feedback from other students, inconsistent learning experiences resulting from wide variations in acting skills among participants, and difficulty pretending a peer or faculty member is a “patient.”

The definition of a standardized patient is someone who has been trained to portray a character in a scripted case scenario and who is able to consistently deliver a similar performance when interacting with different students.^11,12 Standardized patients are valuable for several reasons.^10,11 The consistency of the acting by standardized patients allows faculty members to evaluate students more objectively. Also, standardized patients are trained to portray a broad spectrum of personalities and deliver appropriate corresponding reactions in various situations.¹¹ This permits students to be exposed to different situations and patient personalities and develop the skills to react accordingly. One study found that 92% of pharmacy students agreed or strongly agreed that they preferred an interactive examination using standardized patients to an examination involving “written patient case scenarios.”¹³ Ninety-three percent of the students reported that they felt comfortable working with standardized patients and forgot the patients were hired actors.¹³ The pharmacy schools sampled in a recent study identified the use of standardized patients as the area where they most wanted to see improvement in the course.⁶ ACPE guidelines do indicate, in general, that instructors should employ active-learning strategies and “where appropriate, these techniques should involve actual or simulated patients, pharmacists, and other healthcare professionals.”⁶ Despite the many advantages of standardized patients, it is still unknown whether their use is more effective than didactic lectures or role playing or how students and standardized patients feel about their experiences during communication laboratories.

Communication skills are recognized as a component of a complex set of skills that define clinical competence.¹³ In order to evaluate student progress in communication skills, there is a need to develop standardized assessments of these skills. A literature review and focus group identified 7 criteria as valid measures of communication skills of pharmacy students: (1) Did the student introduce himself or herself? (2) Did the patient understand the purpose of the consultation? (3) Did the student speak clearly? (4) Did the student use words that the patient could understand? (5) Did the student check whether the patient understood what had been discussed? (6) Did the student give the patient the opportunity to talk? (7) Did the student treat the patient with courtesy and respect?¹⁴ There is little or no published research on the validity and reliability of standardized assessments of student communication skills.

Based on these gaps in the literature, the present study focused on the primary objective to determine the impact of a lecture-laboratory course with standardized patients on student communication skills during their second year of a 6-year doctor of pharmacy (PharmD) curriculum. Our hypothesis a priori was that these pharmacy students would score significantly and progressively greater communication skills across the baseline, midpoint, and final assessments of the semester.

DESIGN

A lecture-laboratory communication course was developed for students in their second year of a 6-year PharmD program in preparation for some of the communication issues they would encounter while on their introductory pharmacy practice experiences (IPPEs) at the end of their second year. Course material was taught using traditional methods but also included some active-learning strategies such as trigger tapes and discussions. All students entered the University's first professional degree PharmD program directly from high school. Table 1 presents the objectives for the entire lecture-laboratory course (some of these not specifically achieved during the laboratory component). The lecture and laboratory components of the course specifically focused on students learning the process aspects of communication around medication counseling. The decision was not to focus on the technical aspects of medication information since students had not completed any pharmacotherapy courses yet and could not be expected to achieve competencies related to medication information.

Table 1.

Objectives for Lecture-Laboratory Communication Course

Students were expected to demonstrate several competencies on written and laboratory examinations that reflected their knowledge, comprehension, analysis, and application of various communication processes that are an integral part of a consultation (Table 2). The written examinations included both multiple-choice and short-answer questions. During laboratory examinations, students were assessed by the instructor using a communication skills assessment form (CSAF). This form evaluated each of the competencies listed in Table 2. The CSAF had 7 sections: opening (3 items), assessment (6 items), verbal skills (6 items), nonverbal skills (2 items), issue management (5 items), closing (5 items), and additional skills (3 items). The CSAF evaluated in all sections whether the skill was performed or not. Students were also evaluated on how professionally adequate and consistent they performed the items in the following sections: verbal skills, nonverbal skills, issue management, and additional skills.

Table 2.

Competencies Assessed During Laboratory Examinations

The course was primarily taught by 1 full-time pharmacy practice faculty member with a teaching and research background in pharmacy communications. A full-time cooperative education pharmacy student served as a teaching assistant. The faculty member had full responsibility for one 100-minute lecture and six 120-minute laboratories each week. The teaching assistant had previously taken a pharmacy communications course and assisted in multiple activities including arranging and posting student and standardized patient consultation times weekly, assisting in the taping of students weekly, organizing all laboratory related materials, and assisting with the administrative details related to payment of standardized patients. Thirty standardized patients were recruited by placing an advertisement in a local Boston newspaper. The standardized patients attended 3 hours of training. They received $15 per hour during training and when working with the students. Standardized patients were expected to practice acting out the roles given to them in advance for that week. They were not expected to give the students feedback after the consultations.

One classroom with approximately 25 computers, 3 private counseling rooms with built-in ceiling cameras, and 2 private counseling areas (in the classroom) were utilized for the weekly laboratories. To ensure students had access to a computer to watch the DVDs of their consultations each week, student enrollment per laboratory was capped at 25; thus, 6 laboratory sections were held each week to accommodate the 127 students enrolled. Other logistical concerns involved the scheduling of students with the standardized patients. To ensure sufficient time for student taping and student review of their tapes, and approximately 20-30 minutes for large group discussions of weekly concepts, 5 students were assigned to 5 standardized patients every 10-15 minutes during the first 60-80 minutes of laboratory.

Prior to laboratory each week, students were informed via Blackboard (a Web-based software application that integrates online course materials and course communications) when to come to the laboratory to meet their patients, what medications they were counseling on, and to complete a pre-laboratory assignment (most weeks). The advance notice about medications to be covered was done to allow third-year students not knowledgeable in pharmacotherapy to learn key counseling concepts regarding the medications. This advance notice also helped to reduce student anxiety about what content they needed to know and provide to their patient during the counseling sessions. Prelaboratory assignments included a review of the communication topics and readings associated with the laboratory for each week. Topics included learning how to listen, communicating with concerned patients and showing empathy, communicating with confused patients, communicating with aggressive patients, and communicating with non-adherent patients and using motivational interviewing techniques. This interaction with standardized patients each week enabled the students to solve problems in real time, with mock patients presenting cases related to themes taught during the week. For example, when the lecture for the week was on how to deal with aggressive patients, the laboratory session that week involved angry patients wanting to obtain a narcotic medication from the pharmacy for which there were no refills remaining that had been prescribed by their physician.

Five students came to the laboratory every 12-15 minutes at a pre-assigned time and were given a clipboard with a patient profile and prescription information for a specific counseling station. After allowing each group a few minutes to review the profile and prescription information, students were given timers and asked to meet with the standardized patient in their station for no more than 7 minutes. After all students completed their consultations, they watched their tapes and critically self-assessed their patient encounter using the CSAF. When all students in the laboratory were done reviewing their tapes, the entire laboratory section met to discuss communication issues that came up during the consultations.

To assess the lecture-laboratory communication course, 2 main analyses were completed. First, a retrospective analysis of the baseline, midpoint, and final tapes for 127 students was conducted to identify whether the students improved during the spring 2006 lecture-laboratory communication course. The baseline and final tapes of students involved the same patient-pharmacist scenario. This scenario involved a patient picking up a new prescription for hydrochlorothiazide and a refill for lisinopril. The standardized patient was trained to portray a patient visiting the pharmacist to pick up a second medication for her blood pressure who is worried about having a stroke and being disabled, which happened to one of her relatives. Students also were given prescription profiles that indicated the standardized patients were not adherent to their lisinopril prescriptions. The standardized patients were trained to present different reasons for nonadherence (eg, do not see the need for the medication). Standardized patients were instructed not to offer their concerns or reasons for nonadherence and to express not knowing what to expect from the medications. Students would only learn of patient issues through patient questioning and assessment.

The midpoint scenario involved counseling a patient who was confused about being tapered off one anticonvulsant and started on another one. With all scenarios throughout the semester, standardized patients were trained to have some issues (requiring intervention), but not to openly disclose them unless the information was elicited by the student.

In conducting this retrospective analysis of communication skills, 3 health communication graduate students and 1 communication graduate were recruited to evaluate tapes using the CSAF. Several discussions occurred to review the tool, revise the tool, and practice using the tool so it was easily understood and congruence was achieved among all raters. Every tape was rated by 2 different raters. There was approximately 94% interrater agreement across 2 CSAF score totals of each tape. All sections of the tool had excellent interrater reliability, above 80%, with the exception of the issue management section which had an interrater reliability of 75%, which is considered “substantial.”¹⁵ Based on the scales used for each section and the number of items in each section, students could earn 63 points on the CSAF. The maximum that could be earned for each section was 3 points for the opening section, 9 points for the assessment section, 18 points for the verbal skills section, 6 points for the nonverbal skills section, 15 points for the issue management section, 5 points for the closing section, and 7 points for the additional skills section. All tapes were blinded (the students' names and dates of taping were removed from the cassette holders and replaced with numbers) in terms of whether they represented baseline, midpoint, or final tapes from the lecture-laboratory course. Blinding was used to reduce any experimenter bias that midpoint and final lecture-laboratory tapes would be better than baseline lecture-laboratory tapes.

The second analysis used in evaluating the course involved 2 surveys conducted at the end of the semester: 1 survey instrument was administered to students and 1 to standardized patients. Both surveys asked Likert-scale and open-ended questions about respondent attitudes about the standardized patient program and general comments about the operations of the laboratory sessions. The standardized patients were also asked questions about the quality of their training session. These instruments were developed for the sole purposes of evaluating this specific course and require further validation and replication.

Data were entered into SPSS (version 14.0, SPSS, Inc, Chicago, Ill). Summaries of several study measures were analyzed using descriptive statistics (means, standard deviations). Repeated measures analysis of variance (ANOVA) methodology was used to examine the differences in means of baseline, midpoint, and final scores for both the total CSAF (Table 1) and each of the 7 section totals (Table 2). All pairwise tests were applied with the Bonferroni adjustment, which were used to assure the effect-wise error rate for tests did not exceed 0.05.¹⁶ Mauchly's sphericity tests were utilized to evaluate the sphericity assumption of the data.¹⁶ If the sphericity assumption was met, p values from the standard ANOVA were reported. If the sphericity assumption was violated, p values from the Huynh-Feldt correction were reported instead.¹⁶ Statistical significance was set a priori at p ≤ 0.05. The project was granted exemption by the University's Institutional Review Board.

ASSESSMENT

Basic background characteristics were collected on the 127 students involved in the course (data not shown). There were 75 females (59%) and the mean age of the group was 20.3 years (SD = 3.11). The race/ethnic mix was 69% Caucasian, 19.7% Asian, and 11% African American or others such as Indian and Middle-Eastern. Of the 127 possible set of tapes for the standardized patient group analysis, 21 sets of tapes could not be rated due to audiovisual difficulties, were removed from analysis having been used in training raters, or in a few cases missing for unknown reasons. Thus, there were 109 remaining students with baseline and final tapes for analysis and 108 students had complete sets of baseline, midpoint, and final tapes. One of these 108 students was missing complete information for all sections for the baseline and midpoint CSAF totals and therefore was removed from the analysis of CSAF totals for baseline and midpoint tapes.

Table 3 presents the mean total scores for the baseline, midpoint, and final tapes for the standardized patient group. This table also indicates that the repeated measures ANOVA showed a significant difference between the different tapes (p ≤ 0.001). Pairwise comparisons further indicate that students received significantly higher scores from baseline to midpoint, midpoint to final, and between baseline and final tapes (p ≤ 0.001). Comparisons of the means show that greatest improvements were between the baseline and midpoint tapes but that significant progress could be seen incrementally from baseline, midpoint, and final tapes.

Table 3.

Comparison of Pharmacy Students' Communication Skills Scores in a Lecture and Laboratory Course Using Standardized Patients

Table 4 highlights that there were overall significant differences with each section across the different time periods (p ≤ 0.01). Pairwise comparisons indicate that on all sections with the exception of verbal skills the scores on the baseline tapes were significantly lower than the midpoint (p ≤ 0.01). The midpoint scores were significantly lower than the final scores on only 2 sections involving assessment skills and issue management skills (p ≤ 0.01). There were significant changes noted between the baseline and final tapes on all sections of the CSAF (p ≤ 0.05).

Table 4.

Comparison of Pharmacy Students' Scores on Various Sections of a Communication Skills Assesment Form for a Lecture and Laboratory Course Using Standardized Patients (N=109)

Of the 127 students enrolled in the course, 77 completed the survey yielding a 61% response rate. Students reported favorable evaluations of the standardized patient program (Table 5). In general, they liked the standardized patient aspects of the program and felt it was helpful in improving the skills covered in class. In addition to these favorable evaluations, students reported that they had enough time to review their consultations and found such self-assessments as very helpful in measuring their progress throughout the semester (data not shown). Students wrote in their open-ended comments that they needed more time to prepare for the consultation, and found the pre-laboratory assignments not to be particularly useful and the classroom learning repetitious.

Table 5.

Student Attitudes About Standardized Patient Program (n=77)

^aScores based on a Likert scale on which 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = strongly agree.

Of the 30 standardized patients involved in the course, 20 completed the survey, yielding a 67% response rate. In Table 6, the standardized patients rated the laboratory sessions very positively and felt they were an integral part of the student's success in learning skills. The standardized patients felt the scenarios were realistic and prepared them well for the consultations. The standardized patients also reported favorable evaluations of the training program and its usefulness in preparing them for their work with the students (data not shown). In open-ended comments, standardized patients indicated they wanted to receive more information on what to expect from students, to reduce their wait time to see the next group of 5 students, and to receive more feedback on their role-playing skills. However, other standardized patient comments were similar to that written by one standardized patient: “I believe the students that had the opportunity to participate in the standardized patient program are much better prepared for real life situations than other students and will be more empathic and understanding when dealing with real world patients.”

Table 6.

Standardized Patient Attitudes About Scenarios and Consultations (n=20)

Scores based on a Likert scale on which 1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = strongly agree

SUMMARY

The present research is the first known study showing that pharmacy students who participated in a combined lecture and laboratory course on communication involving standardized patients had significant and progressively higher total scores from their assessment at baseline, midpoint, and final. These findings clearly support the study's hypothesis that students' communication skills would improve across a lecture-lab communication course using standardized patients. While there have been a few other studies showing the positive value of using standardized patients to evaluate student communication skills, there have been no known studies evaluating the use of standardized patients in improving student learning of communication skills especially over the course of a semester.^13,19 The use of standardized patients may help students more actively and realistically simulate the integration of important communication skills into a comfortable communication style that works best for them. A relatively recent study found that the current generation of pharmacy students prefer and are more receptive to active-learning approaches.¹⁸

The present study also found that student improvement in communication skills was significant across the different time points on many of the subsections of the communication assessment tool (CSAF). There was significant improvement noted from baseline to final assessment (an 8-week time period) on all subsections of the CSAF. Such a finding highlights that the subsections of the CSAF captured large changes in skills. With the exception of verbal skills, there was significant improvement noted on all the remaining subsections from the baseline to midpoint evaluation (a 5-week time period). This significant improvement across these 6 other sections from baseline to midpoint is also reflected in the total CSAF scores since the largest improvement in overall communication skills occurred during the semester from baseline to midpoint. It is possible that verbal skills did not yield significant baseline to midpoint changes since most students exhibited competency in this area at baseline, leaving less room for significant improvement. However, there was apparently room for larger changes in verbal skills as noted by the significant difference in these skills between baseline and final assessments.

The improvement from the midpoint to final evaluation (a 3-week period) was only significant in terms of assessment and issue management skills. Students may have continued experiencing significant improvement on assessment and issue management skills since additional attention was given toward these skills during the period from the midpoint to final assessment. The lack of significant improvement in verbal skills is likely attributed to the same issue as noted between baseline and midpoint assessment: that many students had achieved relative competency at baseline leaving little room for incremental growth throughout the semester. The lack of significant improvement noted from midpoint to final evaluation on the other 4 subsections (opening, nonverbal skills, closing, and additional skills) might be explained by the fact that most of the lecture and laboratory content during the period from baseline to midpoint is largely focused on building core communication skills in opening the consultation, verbal and nonverbal skills, closing the consultation, and additional skills. The greatest growth in skills on these subsections would, therefore, occur during the baseline and midpoint period. While there was continuous reinforcement of such skills during the midpoint and final, there may not have been significant improvement on these subsections since students already mastered the key skills leaving less room for significant improvement (as similarly noted with verbal skills from baseline to midpoint and midpoint to final). An alternative explanation for the lack of significant findings across the 5 sections between the midpoint and final assessments is that the CSAF tool may not have been discriminating enough to capture smaller but significant changes in these skills.

Quantitative and qualitative survey findings support the value of the course in improving student communication skills. It was interesting to see that student-reported and standardized patient-reported values of the standardized patient program in improving their communication skills and preparing them for practice was supported by the blinded evaluation, which showed improvement in student communication skills. There were several useful suggestions to improve the laboratory part of the course from both student and standardized patient perspectives. Many of these suggestions, such as eliminating prelaboratory assessments and giving standardized patient feedback about their role-playing, have been incorporated into the course since 2006. In 2007, standardized patients were given the opportunity to use a brief tool to provide feedback to students about skills. While students felt such standardized patient feedback was useful, there were concerns that the feedback was not particularly critical and often did not match course expectations.

Hastings and West (2003) and Lonie et al (2005) are the only known pharmacy studies evaluating different pedagogical approaches to teaching communication skills.^19,20 Hastings and West (2003) compared the efficacy of self-directed patient counseling sessions with instructor-directed counseling sessions on various student outcomes.¹⁹ The authors found that both groups of students improved across the course but were not statistically different from one another.¹⁹ It is unclear to what extent contamination from study groups influenced study results. Lonie et al (2005) investigated the impact of empathy training on student empathy scores.²⁰ They found that such training significantly increased several dimensions of student empathy.²⁰ This latter study only focused on empathy skills and did not use standardized patients. The present study involved a more rigorous evaluation of pedagogical techniques than these other studies since it involved blinding, and more than 2 time points at which communication skills were evaluated.

Despite the noted improvement in communication skills across multiple time points, positive evaluations of the standardized patient program and the methodological strengths of the present study, there are limitations to the present study design and analyses. First, the absence of a comparison group of students who did not receive the standardized patient program makes it difficult to clearly ascertain if the standardized patient program specifically improved communication skills over a laboratory not using standardized patients. A blinded post-hoc analysis attempted this by comparing the results of this study with assessment scores of fifth-year students who took a lecture-based communication course without laboratory sessions or standardized patients. This analysis showed that the total scores of the second-year pharmacy students who completed the communication course with laboratory sessions using standardized patients were significantly higher than the total assessment scores of the fifth-year students who completed the lecture-only course (p ≤ 0.001). However, this post-hoc analysis should be viewed with caution given 2 caveats. First, there was a gap of greater than 2 years between the time students completed the lecture-only course and when they were evaluated for the present study. Such a gap may have caused forgetfulness of key communication skills especially since the fifth-year students in the present study had received little structured reinforcement of communication skills during the gap. Second, the instructor of the lecture-laboratory course may have emphasized different concepts in lecture that the lecture-only instructor did not and may have little to do with the use/non-use of standardized patients. Related to this latter caveat is the possibility that the instructor of the lecture-laboratory course specifically focused on skills measured by the CSAF (since it was used in the grading of tapes) and lecture-only students were unaware of such specific criteria and therefore were not striving towards achieving excellence in such areas. A future study that would help to address these caveats and tease out the specific value of standardized patients on communication skills would involve having all students come to lecture at the same time and then have half the students randomized to laboratories with standardized patients and the other half to laboratories without standardized patients.

A second important limitation of the present study involves the need for more validation and reliability studies exploring the psychometrics of the CSAF. Third, while training efforts try to establish standardization, it is possible the standardized patients were not as consistent as expected across students and may have influenced individual student assessments. However, the assignment of students to standardized patients was random; thus, any standardization effects on student performance should be random across students. A fourth limitation involves the change in scenario used for the midpoint scenario. Since this scenario was different than baseline and final scenarios, student performance and subsequent midpoint evaluations may have been affected. It is unclear whether this difference would have changed the midpoint data in either direction (lower or higher scores). Since this study was conducted in one setting, a fifth limitation is that the study findings may not be generalizable to other university settings or students.

The current investigation is the first known study to highlight the value of a communication course going beyond the traditional approach of lecture and some role-playing to actually utilizing a more realistic, interactive approach of involving standardized patients in the learning process and not just for evaluation. The success of this lecture-laboratory course using standardized patients has convinced the School of Pharmacy to keep the standardized patient program in the communication course and recently has begun integrating the use of standardized patients in other pharmacy practice courses. In addition, efforts have been made to systematically incorporate process-related skills learned in the communication course in other parts of the curriculum as a way for continued reinforcement of communication skills. Future research should identify more clearly the key components of what makes the standardized patient methodology useful in pharmacy communication training and explore possible models for integrating the standardized patient methodology consistently throughout the pharmacy curriculum. More research needs to explore how to actively integrate communication skills training throughout the curriculum, including during APPEs to bring about sustainable change in pharmacist communication skills.²¹ Despite the relatively high costs associated with staffing a standardized patient laboratory program, the current study indicates it has significant value on student learning of communication skills. Students better trained in communication skills will hopefully translate into stronger pharmacist communicators and lead to more positive patient-centered care and improved patient outcomes.