Identifying Criteria for the Assessment of Pharmacy Students' Communication Skills With Patients

Abstract

Objectives

To identify criteria by which patients can assess the communication skills of pharmacy students.

Method

Potential assessment criteria were generated from 2 main sources: a literature review and a focus group discussion. A modified two-round Delphi survey was subsequently conducted with 35 professionals who were actively involved in teaching and assessing communication skills of pharmacy students to determine the importance and reliability of each criterion.

Results

Consensus ratings identified 7 criteria that were important measures of pharmacy students' communication skills and could be reliably assessed by patients.

Conclusions

A modified two-round Delphi consultation survey successfully identified criteria that can be used by patients to assess the communication skills of pharmacy undergraduates. Future work will examine the feasibility of using patients as assessors of communication skills of pharmacy students, preregistration pharmacists, and qualified pharmacists.

INTRODUCTION

The quality of communication between patients and healthcare professionals is fundamental to providing effective healthcare. The ability to communicate well with patients, which includes understanding their concerns and beliefs, eliciting relevant information, and explaining options so they can make informed decisions about treatment, is an essential clinical skill.

In schools of pharmacy in the United Kingdom (UK), communication skills are assessed in a variety of ways, including direct observation, use of simulated patients, and video recordings of interviews.¹ Such methods of assessment are successful in measuring many aspects of students' interactions with patients. However, our experience as pharmacists and previous work looking at patient experiences of student interviews indicates that from a patient's perspective, patients are more likely to openly discuss their disease and treatment if they feel comfortable and develop a rapport with the interviewer. Furthermore, research shows that patients consider attributes addressing the personal acceptance of the patient as the most important in a consultation. These include the warmth of the greeting, being listened to, respect shown to the patient, and concern for the patient as a person.² Thus, utilizing patients to provide feedback on the students' ability to relate effectively with them during a consultation may be of additional benefit to the student.

Patient feedback is a useful method of assessing communications skills and the quality of doctor-patient relationships.^3,4 Also, there is a positive impact of regular patient feedback on improving the interpersonal skills of doctors.⁵ In addition, stakeholders (including members of the public, students, academics, and practitioners) in medicine, pharmacy, and nursing, support the role of patients in student assessment, particularly their assessment of attitude and professionalism.⁶

Several validated patient rating scales have been developed to assess the interpersonal skills of doctors, medical students, and dentists.^7-10 However, there is a distinct lack of validated patient rating scales to assess the communication skills of pharmacy students and a lack of consensus on the essential components of effective pharmacist-patient communication.¹¹

The aim of this study was to identify criteria by which patients can assess the communication skills of pharmacy students. The Delphi technique, which has been used widely in healthcare education and training, was used to gather opinions on suitable criteria from practitioners and faculty members. This technique enables a large group of isolated experts to be contacted, usually by mail with a self-administered questionnaire, and places few limitations on the geographical distribution of the sample.¹²

METHODS

A list of 17 potential assessment criteria was generated from 2 main sources: (1) a literature review of tools to assess communication, interpersonal, and counseling skills in health professional education, and (2) a focus group discussion with 7 pharmacy practice staff members based at the University of Manchester, who were actively involved in the teaching and assessment of communication skills. The focus group had 2 specific goals: first, to consider the relevance to the pharmacy setting of criteria identified from other health professional literature; and second, to identify any other criteria that experienced pharmacy educators would consider important aspects of the communication skills of pharmacy students when consulting patients.

A modified two-round Delphi survey was subsequently conducted with a purposive sample of 38 academic and teaching staff members involved in pharmacy education at 3 UK universities (Aston, Cardiff, and Manchester). Participants were selected based on their involvement in the teaching and/or assessment of pharmacy students' communication skills relating to the pharmacy practice setting. The Delphi survey was posted to all participants in October 2005 together with a cover letter that explained the aim of the study. Each participant was asked to rate the extent to which each of the items were an important measure of communication skills for pharmacy students (face validity) and could be reliably assessed by patients (reliability). Ratings were made on a 9-point equal-interval rating scale, ranging from 1 = definitely to 9 = definitely not, based on the RAND appropriateness method.¹³ Space for additional comments was provided next to each item.

No items were removed from the survey instrument after the first round, but 7 items were added based on comments received, resulting in a total of 24 assessment criteria on the Delphi survey instrument used in the second round. Participants were given 3 types of feedback from the first-round results: a frequency distribution of all scores (on a 1-9 scale), the respondent's first-round ratings, and a summary of the qualitative comments, as shown in Figure 1. The comments were chosen to illustrate both the positive and negative opinions of the first-round respondents.

Figure 1.

Example of feedback on items included in the second round of the Delphi survey. The italicized figures in brackets below the score indicate how many respondents scored that particular value. The red dot above the score indicates your own response in the first round.

We used the second round of the Delphi survey to examine consensus among respondents. The following definitions were specified before analysis: consensus was judged to be present if the interquartile range of the participants' ratings fell within any 3-point range.

Items with an overall median rating of 1, 2, or 3 for measures of validity and reliability without disagreement were included in the final assessment criteria; items rated with an overall median of 7-9 and 4-6 were rated as invalid and equivocal, and subsequently excluded. Disagreement was defined as 30% or more scores in the top (1-3) and bottom (6-9) tertile.¹⁴ Data analysis was conducted using STATA version 9.0 (Stata, College Station, Tex).

RESULTS

Thirty-five respondents (response rate = 92%) submitted completed second-round questionnaires. Ten respondents were involved in pharmacy education at Aston University, 10 at Cardiff University, and the remaining 15 were from the University of Manchester. Of the 35 respondents, 12 were male (34%) and 23 were female (66%). The mean length of time the respondent had been qualified as a pharmacist was 18 years (range 4 to 36 years), and the mean length of time the respondents had worked in pharmacy education was 9 years (range 1 to 35 years).

Based on the comments received in the first-round Delphi survey, changes were made to the criteria included in the second round. Comments were used to clarify the positive rating given to criteria; eg, “good way of assessing students ability to tailor questions” and to explain low ratings, eg, “this question isn't measuring individual skills.” Some criteria present in the first round attempted to assess whether the students had managed to take a medication history from the patient. The majority of respondents felt such criteria were not a valid measure of students' communication skills; their comments included: “this is just testing if they can answer a question” or “it would be more useful to measure their ability to get a drug history – not communication skills.” Other respondents suggested variations on the wording of criteria or the inclusion of new additional items.

Median ratings for each of the 24 criteria in the second round of the Delphi survey were calculated for both face validity and reliability. To reiterate, criteria with an overall median rating of 1, 2, or 3 without disagreement were considered to have achieved consensus agreement. Table 1 shows that consensus for validity was reached for 12 of the 24 criteria. Consensus for reliability was also achieved on a total of 12 criteria, but only 7 criteria were assessed as both valid and reliable (Table 2).

Table 1.

Second-round Delphi Ratings^* for the Validity and Reliability of Assessment Criteria Used to Assess Pharmacy Students' Communication Skills

IQR = interquartile range

^*ratings were based on a 9-point equal interval rating scale ranging from 1 = definitely to 9 = definitely not

^†consensus established

Table 2.

Seven Criteria Rated Face Valid and Reliable for Assessing the Communication Skills of Pharmacy Students

DISCUSSION

This study identified 7 criteria that a group of 35 respondents who were actively involved in pharmacy education believed were both valid measures of communication skills in pharmacy students and could be reliably assessed by patients. Our study used 2 processes to identify suitable criteria for inclusion in the assessment tool. First, items generated from a literature search were discussed in a focus group of professionals involved in assessing communication skills of pharmacy undergraduates. The purpose of this group was to review these criteria and identify which items were applicable to the assessment of communication skills of pharmacy students as opposed to medical students and doctors. We then examined each of the criteria in a two-round Delphi survey. This succeeded in providing a consensus decision on each of the criteria and identified those that could be used by patients to reliably assess communication skills. Other studies reporting the development of similar patient assessment scales have generally constructed the instrument based on inspection of the content of the communication skills course⁸ or by use of a focus group of professionals to discuss the items to be included.¹⁰ We feel that the use of a multicenter two-round Delphi approach in addition to the focus group in this study further strengthens the results. An additional advantage of the Delphi technique is anonymity of the respondents. Since the technique gathers information through a structured approach of systematic, indirect interaction, group interaction processes that can inhibit an individual's freedom of expression were excluded. For example, individuals could retract, alter, or add to their ideas without group pressure. The process of gathering opinions and providing feedback in successive rounds also avoided the quick decision making that sometimes occurs in a focus group environment.¹⁵

Compared with similar questionnaires used in medicine, this survey tool contained fewer criteria. Both the Doctors Interpersonal Skills Questionnaire (DISQ)⁷ and the Liverpool Communication Skills Assessment Scale ⁸ consist of 12 items, while the Medical Student Interviewing Performance Questionnaire (MSIPQ) contains 10 items.¹⁰ Although relatively fewer criteria achieved consensus agreement for both validity and reliability in this study, the criteria identified were actually similar to many of those used in the medical scales. Criteria that are common to both the DISQ and the instrument used in this study include questions relating to how students introduced themselves, explained the purpose of the consultation, allowed the patient the opportunity to talk, and treated the patient with courtesy and respect. The importance of these criteria is supported by research that shows patients consider these aspects the most important elements of a good consultation.¹³ In addition, patients also identified the criterion of “being listened to” as important. This was included in the DISQ, but did not achieve consensus agreement for inclusion in this tool. Respondents agreed that this was an important measure of communication skills; however, they did not agree that patients could reliably assess it. The respondents qualified this with the following comments: “How does the patient know if the student was paying attention? They could only say if the student appeared to listen,” and “This could only be reliably measured if the student demonstrates that they have heard.” Other criteria included in the DISQ, but not in this tool related mainly to the role of the doctor in the treatment of the patient's condition, which would not necessarily be expected in a consultation with a pharmacist.

The response rate of 92% in this study was high. Previous research has suggested that the most important factors governing the response rate to mailed questionnaires are the perceived importance of the study and the number of approaches made by the investigator.¹⁵ We believe that the high response rate seen in this study reflects both the perceived importance of this research in the assessment of communication skills and the involvement of leading faculty members at each university in the coordination of the distribution and collection of the questionnaires. In addition, the consensus achieved from pharmacy educators at 3 major universities has helped to ensure the criteria identified are not specific to the practice and teaching of any one university. This suggests that the tool could be transferred and used in other universities and in clinical practice.

As practicing pharmacists, we are responsible for ensuring we are providing patients with good quality care. One of the key important elements of quality of care is the quality of interpersonal care and people presenting to a health professional expect good individual care. This study has identified criteria that can be used to measure patients' perceptions of their interaction with pharmacy students. Such measures are becoming increasingly recognized as important indicators of quality of care. Indeed, healthcare systems in both the United Kingdom and the United States, are incorporating patients' experiences as a performance measure. ¹⁶

The aim of this study was to identify criteria for the assessment of communication skills in pharmacy students; however, the need for such a tool is likely to stretch beyond the undergraduate curriculum. Preregistration pharmacists must provide evidence that they are competent in communicating with patients as an essential requirement to practice as a pharmacist. Thus, the tool used in this study could be extremely useful in assessing preregistration pharmacists' skills. For practicing pharmacists, obtaining regular feedback on patient interactions could only serve to improve their skills.

Our ongoing work involves applying these criteria to the assessment of students' communication skills. Development of the final format of the questionnaire requires evaluation of a suitable response scale. Use of a 5-point descriptor scale of excellent, very good, good, fair and poor is more discriminating than a 6-point numerical scale with anchors at each end of the scale.² Preparation of the final format of the questionnaire—exploring the underlying content structure and assessing the feasibility of using patients as assessors—is the basis of our continuing work.

CONCLUSION

This study has identified 7 criteria which can be utilized by patients to assess the communication skills of pharmacy students. Pharmacists are responsible for ensuring we provide patients with good quality care; thus, healthcare systems in both the United Kingdom and the United States are incorporating patients' experiences as a performance measure.