Abstract
Introduction
Consumer Quality Index questionnaires are used to assess quality of care from patients’ experiences.
Objective
To provide insight into the agreement about quality of pharmaceutical care, measured both by a patient questionnaire and video observations.
Methods
Pharmaceutical encounters in four pharmacies were video-recorded. Patients completed a questionnaire based upon the Consumer Quality Index Pharmaceutical Care after the encounter containing questions about patients’ experiences regarding information provision, medication counseling, and pharmacy staff’s communication style. An observation protocol was used to code the recorded encounters. Agreement between video observation and patients’ experiences was calculated.
Results
In total, 109 encounters were included for analysis. For the domains “medication counseling” and “communication style”, agreement between patients’ experiences and observations was very high (>90%). Less agreement (45%) was found for “information provision”, which was rated more positive by patients compared to the observations, especially for the topic, encouragement of patients’ questioning behavior.
Conclusion
A questionnaire is useful to assess the quality of medication counseling and pharmacy staff’s communication style, but might be less suitable to evaluate information provision and pharmacy staff’s encouragement of patients’ questioning behavior. Although patients may believe that they have received all necessary information to use their new medicine, some information on specific instructions was not addressed during the encounter. When using questionnaires to get insight into information provision, observations of encounters are very informative to validate the patient questionnaires and make necessary adjustments.
Keywords: community pharmacy, CQI, patient-provider communication, pharmaceutical care, patient perspective, video observation
Introduction
The pharmacist is an essential health care provider in the process of obtaining and maintaining optimal drug therapy and can play an important role in achieving optimal medication outcomes for patients.1,2 High quality of pharmaceutical care is of utmost importance to ensure good medication use. However, previous studies have shown that there is room for improvement in providing care at the pharmacy counter.3,4 To further improve pharmaceutical care, it is important to assess patient experiences in the community pharmacy setting.
In the Netherlands, the Consumer Quality Index (CQI) Pharmaceutical Care is a widely used questionnaire to measure patient experiences with the care provided in community pharmacies.5,6 This questionnaire was developed by Netherlands Institute of Health Services Research and Argo according to the Dutch national standard for developing questionnaires to measure patient experiences in health care settings involving relevant stakeholders including health care professionals, patient organizations, and health care insurers.7 This development phase included testing in patients visiting a pharmacy, assessing psychometric qualities, and adapting the questionnaire. The CQI Pharmaceutical Care includes 69 questions in the following domains: expertise of staff, communication style, information provision, OTC-medication, dispensing of medicines, accessibility of the pharmacy, privacy, medication counseling, patient’s own role in the encounters, cooperation of pharmacy with other health care professionals, patient dossier, overall judgment of the pharmacy, and background characteristics. The latest version of the CQI Pharmaceutical Care was released in 2013.
Previous studies have shown that CQI questionnaires can be used in different health care domains to assess quality of care. These studies showed good internal questionnaire validity and reliability.8–11 However, use of a questionnaire may result in recall bias or socially desirable answers.12 In the Netherlands, CQI questionnaires are used by health care insurers in contract negotiations, can be used by the Health Care Inspectorate, and are used by patient organizations to inform patients on health care quality. Because of this, questionnaire validity is of utmost importance. Previous research assessing validity of CQI questionnaires focused on whether the questionnaires included issues that were relevant to measure quality of care from patients’ perspective and whether the items in the questionnaire were adequate to measure quality.13 So far, no studies testing CQI’s validity have used video observations to test validity in clinical practice.
Objective
We aimed to provide insight into the level of agreement between patients’ experiences – measured with a questionnaire based on the CQI Pharmaceutical Care – and video-recorded observations of patient–provider interactions at the community pharmacy counter. Quality of care was assessed in three domains: 1) information provision, 2) medication counseling, and 3) pharmacy staff’s communication style.
Methods
Design and setting
We conducted an observational study in four community pharmacies located in the northwest of the Netherlands belonging to the Utrecht Pharmacy Practice network for Education and Research, a network consisting of pharmacies that regularly participate in research and internships for pharmacy students of Utrecht University.3,4 These pharmacies were selected based on their previous interest in projects about pharmaceutical care. Informed consent was obtained from the participating pharmacy staff members. In the Netherlands, pharmacists as well as pharmacy technicians can counsel patients during pharmacy encounters. However, most of the time, this information provision at the pharmacy encounter is performed by pharmacy technicians. In this study, 29 pharmacy technicians from four different pharmacies – referred to as pharmacy staff members – participated. The study was conducted according to the guidelines of the Institutional Review Board of the division of Pharmacoepidemiology and Clinical Pharmacology, Utrecht University.
Study population
Adult patients (age ≥18 years) visiting the pharmacy to collect prescription medication for themselves were invited to participate. Patients who agreed to participate received additional oral and written information about the study. Patients were asked to sign informed consent to videotape the encounter and to complete a questionnaire after the encounter. In addition, patients received a form which they could send to the research team in case they wanted to withdraw their videotape from the study. Patients were excluded when they met one or more of the following criteria: 1) visiting the pharmacy to collect only over-the-counter medication or medical devices; 2) only collecting prescribed medication for others; and 3) not being able to fill out the questionnaire (because of intellectual disability or limited understanding of the Dutch language).
Data collection: video observations and patient questionnaires
Patients who agreed to participate were directed to the pharmacy counter where a video camera was placed. Pharmaceutical encounters between the patients and the pharmacy staff were recorded on video, a method we have described before.3,4 In all four participating pharmacies, observations were collected over a 3-day period. The camera was positioned at the patient’s back to ensure patient’s privacy in such a way that only the pharmacy staff was identifiable and the patient was not. Patients were asked to complete a questionnaire consisting of questions from the CQI Pharmaceutical Care about their experiences in the pharmacy as well as sociodemographic questions, immediately after the encounter in the pharmacy.14 In case of lack of time, patients were asked to return the questionnaire by mail to the researchers within 2 weeks. Pharmacy staff noted information about the patient’s medication (number, type, and first/second/refill) directly after the encounter.
Questionnaire items
For this study, we used questions reflecting perceived quality of care in three domains: 1) information provision, 2) medication counseling, and 3) pharmacy staff’s communication style (Table 1). The domain “information provision” consists of three items: 1) explanation of the mechanism of action of the medication; 2) explanation of how to use the medication; and 3) pharmacy staff ask if there are any patients’ questions. The first two items were only scored for patients who received a first prescription. The domain “medication counseling” relates to repeat prescriptions and consists of four items which refer to patients’ experiences with the pharmacy staff’s behavior to explore patients’ experiences with the medication. We studied whether the pharmacy staff: 1) asked patients about their experiences with medicines; 2) discussed with patients whether the medicines have the desired effects; 3) asked patients whether they are able to use the medicines as prescribed; and 4) asked patients whether they suffer from side effects. The domain “communication style” is rated for all pharmacy encounters and consists of three items of the behavior of the pharmacy staff: 1) the patient was taken seriously by the pharmacy staff; 2) the pharmacy staff had enough time for the patient; and 3) the pharmacy staff listened attentively.
Table 1.
Questionnaire and video observation items to assess the quality of pharmaceutical care
| Domain | Questionnaire item | Observation item |
|---|---|---|
| Information provision | Did the pharmacy staff member explain the mechanism of action of the new medicine(s)? | Did the pharmacy staff member explain the mechanism of action of the new medicine(s)? |
| Did the pharmacy staff member explain how to use your new medicine(s)? | Did the pharmacy staff member explain how to use the new medicine(s)? | |
| Did the pharmacy staff member ask if you had any (other) questions? | Did the pharmacy staff member ask if the patient had any questions? | |
| Medication counseling | Did the pharmacy staff member ask about your experiences with the medicines? | Did the pharmacy staff member address patients’ experiences with medicine(s)? |
| Did the pharmacy staff member discuss with you whether the medicines have the desired effects? | Did the pharmacy staff member discuss with patients whether the medicines have desired effect(s) of medicine(s)? | |
| Did the pharmacy staff member ask whether you are able to use the medicines as prescribed? | Did the pharmacy staff member ask whether patients were able to use the medicine(s) as prescribed? | |
| Did the pharmacy staff member ask you about experienced side effects of the medicines? | Did the pharmacy staff member ask patients about the side effect(s) of medicine(s) experienced by them? | |
| Pharmacy staff’s communication style | Were you taken seriously by the pharmacy staff member? | Did the pharmacy staff member approach the patient as his equal? |
| Did the pharmacy staff member respond respectfully to different or wrong patient perception? | ||
| Did the pharmacy staff member respond respectfully to different or undesired patient decision? | ||
| Did the pharmacy staff member have enough time for you? | Did the pharmacy staff speak calmly, not in a hurry? | |
| Did the pharmacy staff member not interrupt the patient? | ||
| Was there room for a patient reaction? | ||
| Did the pharmacy staff member listen attentively? | Did the pharmacy staff member ask questions when necessary? | |
| Did the pharmacy staff member react to patient response? |
Data analysis
The videotaped patient–provider interactions at the pharmacy counter were coded for quality of provided care in the three domains using an observation protocol, similar to the protocol used in our previous studies3,4 (Table 1 for an overview of questionnaire items and video observation coding). All items were scored as yes/no or on a four-point Likert scale by the observers and in the case of a four-point scale, later on for analysis, the responses were dichotomized as yes (present) versus no (absent).
The observation protocol was pretested with videotapes of (simulated) student–patient interactions at the Utrecht School of Pharmacy; no adjustments were made. Two observers watched the first five video recordings together to reach an agreement on how to interpret the observation protocol. Thereafter, five videotaped patient–provider interactions were watched and coded individually and again discussed to reach a consensus on scoring for the different domains. One observer scored the remaining video recordings. To examine differences between the perceived quality of care, assessed by analyzing the videotapes and the patients’ questionnaires, the percentage of overall agreement between items in the different quality domains (Table 1) was calculated by the number of ratings with agreement on both measures divided by the total number of ratings. A percentage of ≥80 was considered appropriate (based on the cut-offs used in other studies).15,16 All data were analyzed with Statistical Package for the Social Sciences version 20.0 (IBM Corporation, Armonk, NY, USA).
Results
Participants
A total of 160 patients from four different pharmacies met our inclusion criteria and signed informed consent. Of these, one patient collected medication for someone else and was therefore excluded and one patient withdrew informed consent. Of the 158 patients who signed informed consent, 49 patients (31.0% nonresponse) did not fill in the post-encounter questionnaire, resulting in a total of 109 included patients (Table 2). The majority of patients were women and aged ≥55 years. Most patients (n=65, 59.6%) filled a prescription for one medicine. On average, patients received 1.7 medicines during the encounter (total of 180 filled medicines). Three-quarters of these medicines (n=135) were repeat prescriptions. Nonresponders did not differ on characteristics such as sex and number of filled prescriptions (data not shown).
Table 2.
Study participant characteristics (N=109)
| Characteristic | % (n) |
|---|---|
| Female sex | 62.4 (68) |
| Age (years) | |
| 18–24 | 4.6 (5) |
| 25–34 | 4.6 (5) |
| 35–44 | 11.1 (12) |
| 45–54 | 13.0 (14) |
| 55–64 | 32.4 (35) |
| 65–74 | 24.1 (26) |
| ≥75 | 10.2 (11) |
| Native ethnicity* | 90.6 (96) |
| Educational level** | |
| None | 1.0 (1) |
| Low | 14.3 (15) |
| Medium | 56.9 (62) |
| High | 25.7 (27) |
Notes:
Both parents born in the Netherlands.
None = no formal education, low = lower vocational, medium = intermediate vocational, intermediate/higher secondary, high = higher vocational/university.
Information provision
Agreement between the questionnaire items and video observations varied between 45% and 76% for the different items in the domain “information provision” (Table 3). Patients reported to have received medication information more frequently than we observed by watching their videotaped pharmacy encounter. This was especially true for the topic, whether the pharmacy staff asked patients if they had questions (45% overall agreement, 56% (59/105) based on patient experiences versus 1% (1/105) based on video observations).
Table 3.
Information provision in the pharmacy: questionnaire items compared with observations
| O: Provides information on mechanism of action (N=24) | |||
| Q: Did the pharmacy staff member explain the mechanism of action of new medicines?* | Yes | No | |
| Yes | 6 | 8 | |
| No | 0 | 10 | |
| % overall agreement | 66.7 | ||
| O: Explains how to use the medicine (N=25) | |||
| Q: Did the pharmacy staff member explain how to use new medicines?* | Yes | No | |
| Yes | 15 | 2 | |
| No | 4 | 4 | |
| % overall agreement | 76.0 | ||
| O: Checks for questions (N=105) | |||
| Q: Did the pharmacy staff member ask whether you had any questions? | Yes | No | |
| Yes | 1 | 58 | |
| No | 0 | 46 | |
| % overall agreement | 44.8 | ||
Note:
The first two items were only scored for patients who filled new medicines.
Abbreviations: Q, question; O, video observation.
Medication counseling
Agreement between the questionnaire and the video observations for medication counseling was very high (>95%) (Table 4). Both the observations and patient reports showed that the pharmacy staff did not explore patient experiences with the medication at all.
Table 4.
Medication counseling in the pharmacy: questionnaire items compared with observations
| O: Conversation about experiences with medicine (N=66) | |||
| Q: Did the pharmacy staff member ask about your experiences with the medicines? | Yes | No | |
| Yes | 2 | 2 | |
| No | 1 | 61 | |
| % overall agreement | 95.5 | ||
| O: Conversation about desired effects (N=65) | |||
| Q: Did the pharmacy staff member discuss with you whether the medicines have the desired effects? | Yes | No | |
| Yes | 0 | 0 | |
| No | 1 | 64 | |
| % overall agreement | 98.5 | ||
| O: Conversation about being able to use (N=65) | |||
| Q: Did the pharmacy staff member ask you whether you are able to use the medicines as prescribed? | Yes | No | |
| Yes | 0 | 2 | |
| No | 0 | 63 | |
| % overall agreement | 96.9 | ||
| O: Conversation about adverse effects (N=65) | |||
| Q: Did the pharmacy staff member ask you about the side effects of the medicines experienced by you? | Yes | No | |
| Yes | 0 | 0 | |
| No | 1 | 64 | |
| % overall agreement | 98.5 | ||
Note: The medication counseling items were only scored for patients who filled repeat medicines.
Abbreviations: Q, question; O, video observation.
Communication style
Table 5 shows that agreement between video observations and questionnaire data was very high for the communication style of pharmacy staff (>93% for all items).
Table 5.
Pharmacy staff’s communication style: questionnaire items compared with video observations
| O: Approaches patient as his equal (N=109) | O: Responds respectfully to different or wrong patient perception (N=0) | O: Responds respectfully to different or undesired patient decision (N=0) | |||||
| Q: Were you taken seriously by the pharmacy staff member? | Yes | No | Yes | No | Yes | No | |
| Yes | 107 | 1 | n/a | n/a | n/a | n/a | |
| No | 1 | 0 | n/a | n/a | n/a | n/a | |
| % overall agreement | 98.2 | – | – | ||||
| O: Speaks calmly, not in a hurry (N=109) | O: Does not interrupt the patient (N=109) | O: Room for patient reaction (N=109) | |||||
| Q: Did the pharmacy staff member have enough time for you? | Yes No | Yes No | Yes | No | |||
| Yes | 103 | 3 | 106 | 0 | 104 | 2 | |
| No | 3 | 0 | 3 | 0 | 3 | 0 | |
| % overall agreement | 94.5 | 97.2 | 95.4 | ||||
| O: Asks questions, explores patients’ needs/concerns (N=14) | O: Reacts to patient response (N=107) | ||||||
| Q: Did the pharmacy staff memberlisten attentively? | Yes | No | Yes | No | |||
| Yes | 13 | 1 | 106 | 0 | |||
| No | 0 | 0 | 1 | 0 | |||
| % overall agreement | 92.9 | 99.1 | |||||
Abbreviations: n/a, not applicable; Q, question; O, video observation.
Discussion
In this study, we investigated agreement about the quality of pharmaceutical care measured between a patient questionnaire and video observations of encounters. Our results show that agreement between patients’ reported experiences and video observations was very high for the domains “medication counseling” and “communication style”. However, for the domain “information provision”, agreement was lower than 80%: patients reported higher frequencies of provided information than was observed in the videotaped pharmacy encounters.
In general, patients themselves tended to be more positive about the provided care compared to the video observations, especially for the domain “information provision”. Makoul et al17 also described discrepancies between actual communication of health care providers – physicians in their study – and patients’ perception about the received information during consultation. They found that patients were more positive about the counseling received, compared to observations. The same was true for a study in Dutch midwifery practices, wherein clients were also more positive about a topic being addressed.18 Patient perception may, therefore, not reflect the actual situation or may reflect general satisfaction with – in our case – the pharmacy, and thus may not always be the best proxy to measure quality of provided care. Patients may have problems recalling what was actually discussed during the encounter or socially desirable answering of questionnaire may play a role here.
Strengths and limitations
To the authors’ best knowledge, this is the first study to externally validate a patient questionnaire on the provision of pharmaceutical care using video observation. Although such direct observation is expensive and may be potentially intrusive, it provides a gold standard for validity assessment of more easily performed nonobservational methods such as patient questionnaires.19 We were able to assess both the content of the communication as well as pharmacy staff’s communication style. To ensure patient anonymity and reduce potential intrusiveness, the video camera was placed at the patient’s back, simulating open communication from the patient’s perspective. Due to this procedure, limited data were collected about patients’ nonverbal behavior, which might also be an important response to questions. A possible limitation might be that the presence of the video camera during the encounter may have influenced pharmacy staff’s and patients’ communication style by increasing social desirability in the behaviors from those being observed. Finally, interpretation bias by the observer may have occurred. However, we tried to minimize this by using a structured observation protocol for coding and by checking agreement in coding for the first videos between two observers. To our best knowledge, this is the first study using video observations for validation of a questionnaire. Our sample size was relatively small; therefore, replicating the study with a larger sample size, for example, with patients picking up new prescriptions as compared to refills, is necessary.
Practice implications
We found good agreement between patients’ experiences and video observations regarding the communication style and medication counseling in the pharmacy. However, there were some clear discrepancies between patients’ experiences and observations regarding the provision of information. Patients’ own perception regarding the completeness of the information they have received may be different from the information we think they should receive. Although patients may believe that they have received all necessary information to use their new medicine, some information on specific instructions was not addressed during the encounter. In addition, one may question whether patients are always able to classify the information they have received correctly. The patient questionnaire in our study was mostly administered directly after the encounter, whilst in general, the time between the actual consultation and administering of a CQI questionnaire is much longer (about 2 months), which may lead to further recall bias. We showed good agreement between patients’ (recent) experiences and the video observations. As questionnaires are time- and cost-effective and result in less burden for patients when compared to video observations, questionnaires are most feasible for use. However, to assess patients’ experiences of received care, it might be necessary to administer these questionnaires within a shorter period of time after a consultation than 2 months, as well as to keep possible overreporting of actual received care in mind.
Conclusion
A patient questionnaire, such as the CQI Pharmaceutical Care, is informative for evaluating pharmacy staff’s communication style and medication counseling from the patient’s perspective. However, to get insight into the actual provision of information when patients collect a prescription, and thus, for estimation of the actual quality of care (instead of patients’ perception of care), questionnaire validation (and adjustment) with the help of observations of pharmacy encounters is needed to ensure that answers to the questionnaire reflect what happens in reality.
Footnotes
Disclosure
The authors report no conflicts of interest in this work.
References
- 1.van Grootheest AC, de Jong-van den Berg LT. The role of hospital and community pharmacists in pharmacovigilance. Res Social Adm Pharm. 2005;1(1):126–133. doi: 10.1016/j.sapharm.2004.12.009. [DOI] [PubMed] [Google Scholar]
- 2.Patwardhan A, Duncan I, Murphy P, Pegus C. The value of pharmacists in health care. Popul Health Manag. 2012;15(3):157–162. doi: 10.1089/pop.2011.0030. [DOI] [PubMed] [Google Scholar]
- 3.van Dijk M, Blom L, Koopman L, et al. Patient-provider communication about medication use at the community pharmacy counter. Int J Pharm Pract. 2016;24(1):13–21. doi: 10.1111/ijpp.12198. [DOI] [PubMed] [Google Scholar]
- 4.Koster ES, van Meeteren MM, van Dijk M, et al. Patient-provider interaction during medication encounters: a study in outpatient pharmacies in the Netherlands. Patient Educ Couns. 2015;98(7):843–848. doi: 10.1016/j.pec.2015.03.007. [DOI] [PubMed] [Google Scholar]
- 5.van Dijk L. Optimaliseren en inkorten van de CQI Farmacie. Utrecht: Nivel; 2013. Optimizing and shortening of the CQI Pharmaceutical Care. [Google Scholar]
- 6.van Linschoten P, Moorer P, Vervloet M, van Dijk L. Ontwikkeling indicatoren Patiëntervaring Farmaceutische Zorg. Groningen/Utrecht: ARGO/Nivel; 2011. Development Indicators Patient Experience Pharmaceutical Care. [Google Scholar]
- 7.ZIN [Accessed on March 1, 2016]. Available from: https://www.zorginstituutnederland.nl/kwaliteit/toetsingskader+en+register/de+cq-index/handboek+eisen+en+werkwijzen+cqi-metingen.
- 8.Berendsen AJ, Groenier KH, de Jong GM, et al. Assessment of patient’s experiences across the interface between primary and secondary care: Consumer Quality Index Continuum of care. Patient Educ Couns. 2009;77(1):123–127. doi: 10.1016/j.pec.2009.01.011. [DOI] [PubMed] [Google Scholar]
- 9.Booij JC, Zegers M, Evers PM, Hendriks M, Delnoij DM, Rademakers JJ. Improving cancer patient care: development of a generic cancer consumer quality index questionnaire for cancer patients. BMC Cancer. 2013;13:203. doi: 10.1186/1471-2407-13-203. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.van Cranenburgh OD, Krol MW, Hendriks MC, et al. Consumer Quality Index Chronic Skin Diseases (CQI-CSD): a new instrument to measure quality of care from patients’ perspective. Br J Dermatol. 2015;173(4):1032–1040. doi: 10.1111/bjd.13957. [DOI] [PubMed] [Google Scholar]
- 11.van der Veer SN, Jager KJ, Visserman E, et al. Development and validation of the Consumer Quality index instrument to measure the experience and priority of chronic dialysis patients. Nephrol Dial Transplant. 2012;27(8):3284–3291. doi: 10.1093/ndt/gfs023. [DOI] [PubMed] [Google Scholar]
- 12.Kimberlin CL, Winterstein AG. Validity and reliability of measurement instruments used in research. Am J Health Syst Pharm. 2008;65(23):2276–2284. doi: 10.2146/ajhp070364. [DOI] [PubMed] [Google Scholar]
- 13.Krol M. Numbers Telling the Tale? On the Validity of Patient Experience Surveys and the Usability of Their Results [PhD thesis] Tilburg: University of Tilburg; 2015. [Google Scholar]
- 14.van Greuningen M, Vervloet M, van der Hoek L, van Dijk L. Het discriminerend vermogen van de CQ-index Farmaceutische Zorg. Utrecht: Nivel; 2009. [Google Scholar]
- 15.Mokkink LB, Terwee CB, Gibbons E, et al. Inter-rater agreement and reliability of the COSMIN (COnsensus-based Standards for the selection of health status Measurement Instruments) checklist. BMC Med Res Methodol. 2010;10:82. doi: 10.1186/1471-2288-10-82. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.de Vet HC, Beurskens AJ. Roaming through methodology. VII. Reproducibility of measurements. Ned Tijdschr Geneeskd. 1998;142(37):2040–2043. Dutch. [PubMed] [Google Scholar]
- 17.Makoul G, Arntson P, Schofield T. Health promotion in primary care: physician-patient communication and decision making about prescription medications. Soc Sci Med. 1995;41(9):1241–1254. doi: 10.1016/0277-9536(95)00061-b. [DOI] [PubMed] [Google Scholar]
- 18.Martin L, Gitsels-van der Wal JT, Pereboom MTR, Spelten ER, Hutton EK, van Dulmen S. Midwives’ perceptions of communication during videotaped counseling for prenatal anomaly tests: How do they relate to clients’ perceptions and independent observations? Patient Educ Couns. 2015;98:588–597. doi: 10.1016/j.pec.2015.02.002. [DOI] [PubMed] [Google Scholar]
- 19.Stange KC, Zyzanski SJ, Smith TF, et al. How valid are medical records and patient questionnaires for physician profiling and health services research? A comparison with direct observation of patients visits. Med Care. 1998;36(6):851–867. doi: 10.1097/00005650-199806000-00009. [DOI] [PubMed] [Google Scholar]