Abstract
The operational definition of interpersonal communication is “the ability of the provider to elicit and understand patient concerns, to explain healthcare issues and to engage in shared decision-making if desired.”
Objective:
To examine how well interpersonal communication is captured in validated instruments that evaluate primary healthcare from the patient's perspective.
Method:
645 adults with at least one healthcare contact in the previous 12 months responded to instruments that evaluate primary healthcare. Eight subscales measure interpersonal communication: the Primary Care Assessment Survey (PCAS, two subscales); the Components of Primary Care Index (CPCI, one subscale); the first version of the EUROPEP (EUROPEP-I); and the Interpersonal Processes of Care Survey, version II (IPC-II, four subscales). Scores were normalized for descriptive comparison. Exploratory and confirmatory (structural equation) factor analysis examined fit to operational definition, and item response theory analysis examined item performance.
Results:
Items not pertaining to interpersonal communication were removed from the EUROPEP-I. Most subscales are skewed positively. Normalized mean scores are similar across subscales except for IPC-II Patient-Centred Decision-Making and IPC-II Hurried Communication. All subscales load reasonably well on a single factor, presumed to be interpersonal communication. The best model has three underlying factors corresponding to eliciting (eigenvalue = 26.56), explaining (eigenvalue = 2.45) and decision-making (eigenvalue = 1.34). Both the PCAS Communication and the EUROPEP-I Clinical Behaviour subscales capture all three dimensions. Individual subscales within IPC-II measure each sub-dimension.
Conclusion:
The operational definition is well reflected in the available measures, although shared decision-making is poorly represented. These subscales can be used with confidence in the Canadian context to measure this crucial aspect of patient-centred care.
Abstract
La définition opérationnelle de la communication interpersonnelle est la « capacité du clinicien à obtenir et à comprendre les inquiétudes du patient, à expliquer les problèmes concernant les soins de santé et à procéder à des prises de décisions communes s'il y a lieu. »
Objectif :
Examiner à quel point les instruments validés, qui servent à évaluer les soins primaires du point de vue du patient, font état de la communication interpersonnelle.
Méthode :
645 adultes, ayant vécu au moins un contact avec les services de santé au cours des 12 mois antérieurs, ont répondu à des instruments utilisés pour l'évaluation des soins de santé primaires. Huit sous-échelles ont servi à mesurer la communication interpersonnelle : Primary Care Assessment Survey (PCAS, deux sous-échelles); Components of Primary Care Index (CPCI, une sous-échelle); la première version de l'EUROPEP (EUROPEP-I); et Interpersonal Processes of Care Survey, version II (IPC-II, quatre sous-échelles). Les indices ont été normalisés pour procéder à des comparaisons descriptives. Les analyses factorielles exploratoires et confirmatoires (équation structurelle) ont permis d'examiner leur adéquation à la définition opérationnelle, et l'analyse de la réponse par item a permis d'en examiner le rendement.
Résultats :
Les items qui n'ont pas de lien avec la communication interpersonnelle ont été retirés de l'EUROPEP-I. La plupart des sous-échelles présentent une asymétrie positive. Les indices moyens normalisés sont similaires parmi les sous-échelles, sauf pour les sous-échelles « décisions axées sur le patient » et « communication hâtive » de l'IPC-II. Toutes les sous-échelles présentent un point de saturation raisonnablement acceptable pour un facteur unique, qui est probablement la communication interpersonnelle. Le meilleur modèle comporte trois facteurs sous-jacents qui correspondent à l'« obtention » (valeur propre=26,56), à l'« explication » (valeur propre=2,45) et à la « prise de décision » (valeur propre=1,34). La sous-échelle « communication » du PCAS et la sous-échelle « attitude clinique » de l'EUROPEP-I captent toutes deux les trois dimensions. Les sous-échelles individuelles au sein de l'IPC-II mesurent chacune des sous-dimensions.
Conclusion :
La définition opérationnelle est bien représentée par les mesures disponibles, quoique l'aspect ayant trait aux décisions communes y soit faiblement représenté. Ces sous-échelles peuvent être utilisées en toute confiance dans le contexte canadien pour mesurer cet aspect essentiel des soins axés sur le patient.
Effective communication between doctor and patient is a core clinical skill. In interviews with patients, doctors elicit diagnostic information and provide therapeutic advice. Effective doctor–patient communication is associated with positive health outcomes (Mead and Bower 2002; Stewart 1995) such as symptom resolution (Headache Study Group 1986; Starfield et al. 1981), improved general health (Safran, Taira et al. 1998) and physiologic measures (Stewart 1995), as well as with greater consumer satisfaction in different organizational models of primary healthcare services (Safran et al. 1994, 2000, 2002). Studies in many countries have shown that serious communication problems are common in clinical practice (Simpson et al. 1991). Indeed, most complaints by the public about medical services are not about competencies, but about communication. The majority of malpractice allegations arise from communication errors (Simpson et al. 1991).
Improving interpersonal communication between providers and patients is an important health policy issue (Simpson et al. 1991). Its importance has recently been underscored with the adoption of person-centred medicine and the “medical home” as key concepts of primary care (Stange and Acheson 2006). However, there are concerns that interprofessional teamwork, a core feature of new primary care models, may change the nature of one-on-one communication experienced by the patient (Safran 2003; Rodriguez et al. 2007). It is thus important to be able to assess interpersonal communication reliably and validly as part of monitoring the impact of new models of primary care.
Although interpersonal communication is closely related to relational continuity, trust and patient-centred care, it is considered a distinct attribute. Communication skills can be observed. Effective communication can be experienced even in a first encounter with a provider who may not be seen again. It precedes and leads to relational continuity and fosters patient-centred practice (Brown et al. 2001; Thom and Campbell 1997).
Evaluating interpersonal communication
In 2004, we conducted a consensus consultation of 20 primary healthcare (PHC) experts across Canada to formulate operational definitions of the attributes of care that should be measured in primary healthcare models (Haggerty et al. 2007). Good interpersonal communication was identified unanimously by the experts as an essential attribute, even though it is not specific to primary healthcare. The operational definition on which they agreed was “the ability of the provider to elicit and understand patient concerns, to explain healthcare issues, and to engage in shared decision-making if desired.” The experts also agreed unanimously that this attribute is most validly evaluated from the patient's perspective.
Several instruments measure interpersonal communication from the patient's perspective. Some are entirely devoted to this attribute of care (Stewart 1995; Stewart et al. 1999), and others are subscales of generic instruments developed to evaluate consumers' experience with primary care (Brody et al. 1989; Flocke 1997; Safran, Kosinski et al. 1998; Stewart 1995). Each of these instruments has a slightly different approach to the definition of interpersonal communication. Roughly speaking, they all propose definitions based on the two key moments of the clinical encounter: taking the patient's history and concluding the interview. Eliciting and acknowledging patients' concerns and explaining the diagnosis and management plan are part of all definitions (Brody et al. 1989; Flocke 1997; Safran, Kosinski et al. 1998; Stewart 1995; Stewart et al. 1999). Some instruments also include consideration of patients' preferences and abilities in the decision-making process (Stewart 1995; Stewart et al. 1999).
As is the case with most instruments developed to measure primary care attributes, there is little comparative information to guide evaluators in selecting the appropriate tool for evaluating the experience of interpersonal communication. Our objective was to compare validated instruments that purport to measure interpersonal communication. We compared scores of subscales from different instruments and examined whether they measured the same construct of interpersonal communication. We expected to find some overlap between interpersonal communication and respectfulness. After discussion, we excluded trust as being a result rather than a component of interpersonal communication. Finally, we examined the psychometric performance of individual items. Our intent is not to recommend one instrument over another, but to provide insight into how well different subscales fit our operational definition of interpersonal communication.
Method
The method and analytic strategy are described in detail elsewhere in this special issue of the journal (Haggerty 2011; Santor et al. 2011). Briefly: six instruments that evaluate PHC from the patient's perspective were administered to 645 healthcare users balanced by English/French language, rural/urban location, low/high level of education and poor/average/excellent overall PHC experience. The analysis consisted of examining the distributional statistics and subscale correlations, followed by common factor and confirmatory factor analysis (structural equation modelling) to identify dimensions common to the entire set of items. Finally, we examined the performance of individual items and response scales against constructs emerging across instruments using item response theory analysis.
The confirmatory factor analysis with structural equation modelling for this analysis differed from that used for the other attributes because of the large number of items for the subjects available. We used the robust maximum likelihood (RML) method, which assumes that the variables are continuous and normally distributed, rather than the weighted least squares regression (Flora and Curran 2004). We used the Satorra-Bentler chi-square statistic, which adjusts the model chi-square for non-normality.
Measure description
Among the six validated instruments in our study, four had one or more subscales on interpersonal communication. The Primary Care Assessment Survey (PCAS) had two subscales: Interpersonal Communication, which focuses both on history taking and conclusion of the interview, and Interpersonal Treatment, which focuses more on the physician's attitude towards the patient (warmth, patience, etc.) (Safran, Kosinski et al. 1998). The Components of Primary Care Index (CPCI) had one subscale made up of six items that could be related to the dimensions of eliciting and explaining (Flocke 1997). Finally, the Interpersonal Processes of Care version II (IPC-II) had four subscales: Elicited Concerns, Explained Results, Patient-Centred Decision-Making and Hurried Communication. Permission to use the instruments was obtained from all instrument developers.
The first version of the EUROPEP (EUROPEP-I) does not include a specific subscale on communication (Grol et al. 2000; Wensing et al. 2000), but its 15-item Clinical Behaviour subscale includes questions conceptually related to the eliciting, explaining and decision-making dimensions. Although our initial intent was to respect scales as conceived and validated by the instrument developers, these items clearly did not fit the analyses of interpersonal communication so we excluded them from our analysis. With these EUROPEP-I questions included, these four instruments contained eight subscales on interpersonal communication.
We assigned items to factors or underlying subdimensions based on the exploratory factor analysis and our judgment of fit with the operational definition when items had ambiguous loadings. We used as the reference item for confirmatory factor loading the one with the highest principal components loading and apparent content fit with the latent variable. We compared the appropriateness of a number of models in which the correlations between factors were allowed to vary or were fixed as orthogonal.
We based factor analysis only on subjects with no missing values; those with at least one missing value on any item (listwise missing) were excluded. This reduced our effective sample size. Given that this conservative approach can introduce bias, we repeated all the analyses using maximum likelihood imputation of missing values (Rubin 1987) to examine the robustness of our conclusions.
Results
Comparative descriptive statistics
The summary of the eight subscales item content and distributions are summarized in Table 1 and available in detail online at (http://www.longwoods.com/content/22636). Very few items had ≥5% missing values. Exceptions were the items in IPC-II Explained Results (5% each item) and IPC-II Patient-Centred Decision-Making (7%–9%). Additionally, many respondents declared as “not applicable” the EUROPEP-I questions on emotional problems related to health status (16%) and following advice (5%), which count as missing values in analysis. The vast majority of respondents selected the two highest response options and very few the lowest options. For subsequent analyses four items in the EUROPEP-I Clinical Behaviour subscale were removed because we judged that they do not address interpersonal communication (items EU_CB6 to EU_CB11). Confirmatory factor analysis supported this decision.
TABLE 1.
Summary of Interpersonal Communication subscale content and distribution of item responses (detailed distribution available at http://www.longwoods.com/content/22636)
| Subscale and Item Description | Response Scale | Range Missing Values | Overall Modal Response | Range Item Discriminability | Comments on Distribution |
|---|---|---|---|---|---|
|
PCAS Communication (6 items) Rate aspect of talking with regular doctor: thoroughness of questions about symptoms; attention to what you have to say; explanations of health problems or treatments; instructions about further care; advice and help in making decisions about care Frequency of leaving doctor's office with unanswered questions |
Likert evaluative, 1=very poor to 6=excellent | 1%–2% | 5 (very good) | 1.8 (frequency) to 5.9 (explanation, advice) | Approximately 60% of responses in two most positive categories; ≈% in most negative |
|
PCAS Interpersonal Treatment (5 items) Rate the personal aspects of care from regular doctor: amount of time spent; patience with questions or worries; friendliness and warmth; caring and concern; respect for you |
Likert evaluative, 1=very poor to 6=excellent | 0%–1% | 6 (excellent) | 3.7 (time) to 9.9 (caring and concern) | Approximately 60% of responses in two most positive categories; ≈4% in most negative |
|
CPCI Interpersonal Communication (6 items) Best description of regular doctor: Positive statements: I can easily talk about personal things; doctor always explains Negative statements: doctor does not always listen; I don't bring up things that I'm worried about; don't always feel comfortable asking questions; feel doctor ignores my concerns |
Semantic differential opinion, 1=strongly disagree, 6=strongly agree | 2%–3% | Most positive | 1.5 (worries) to 3.8 (ignores) | Most respondents (36%–53%) select most positive opinion; 6%–8% most negative |
|
EUROPEP Clinical Behaviour (10/15 items on communication) Rate care by GP: enough time; interest in personal situation; ease telling about problems; involvement in decisions about medical care; listening; explaining tests and treatments; telling about symptoms and/or illness; help with emotional problems; help understanding importance of advice; knowing what was done or said in previous contacts |
Semantic differential rating, 1=poor, 5=excellent | 2%–3% (true missing) 1%–16% (n/a) |
5 (excellent) | 2.9 (time) to 4.5 (listening) | Approximately 40% in most positive category; 2%–4% most negative. Two items with high proportion “not applicable”: emotional problems (16%), advice (5%) |
|
IPC-II-Elicited Concerns (3 items) Frequency of doctor's/doctors': Really finding out concerns; letting you say what was important; taking health concerns very seriously |
Likert frequency, 1=never, 5=always | 3% | 5 (always) | 3.1 (take seriously) to 4.5 (find out concerns) | Almost 80% of responses in two most frequent categories; 1% never |
|
IPC-II-Explained Results (4 items) Frequency of doctor's/doctors' explaining: test results; results of physical exam; effect of not taking a prescribed medicine; possible side effects from a medicine |
Likert frequency, 1=never, 5=always | 5% | 5 (always) | 2.0 (side effects) to 6.3 (physical exam) | Approximately 70% of responses in two most frequent categories; 3%–7% never |
|
IPC-II-Patient-Centred Decision-Making (4 items) Frequency of doctor's/doctors': working out a treatment plan together; asking if help needed deciding between treatment choices; asking about anticipated problems following recommendations; asking if you could do the recommended treatment |
Likert frequency, 1=never, 5=always | 7%–9% | 4 (usually) | 2.6 (plan together) to 5.5 (about problems) | Less skewed than others: ≈18% never, ≈30% usually, ≈18% always. High proportion of missing values likely due to individual case applicability, not health system context |
|
IPC-II-Hurried Communication (5 items) Frequency of doctor's/doctors' negative behaviours: speaking too fast; using hard-to-understand words; ignoring what told; appearing distracted; seeming bothered with questions |
Likert frequency, 1=never, 5=always | 3%–4% | 1 (never) | 1.5 (hard to understand) to 3.6 (bothered) | Approximately 80% of responses in least frequent categories; 1% always |
TABLE 1.
Distribution of responses for each item in subscales measuring interpersonal communication in primary healthcare services (n=645); modal response is shown in bold
| Item Code | Item Statement | Missing % (n) | Per Cent (Number) by Response Option | Item Discrimination | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
PCAS Communication Thinking about talking with your regular doctor… |
1=Very poor | 2=Poor | 3=Fair | 4=Good | 5= Very good | 6= Excellent | ||||
| PS_c1 | How would you rate the thoroughness of your doctor's questions about your symptoms and how you are feeling? | 1 (5) | 1 (6) | 3 (21) | 12 (76) | 24 (157) | 35 (223) | 24 (157) | 4.84 (.30) | |
| PS_c2 | How would you rate the attention your doctor gives to what you have to say? | 1 (5) | 1 (7) | 4 (26) | 11 (74) | 22 (143) | 29 (188) | 31 (202) | 5.30 (.35) | |
| PS_c3 | How would you rate doctor's explanations of your health problems or treatments that you need? | 2 (10) | 2 (10) | 4 (23) | 10 (64) | 23 (147) | 32 (207) | 29 (184) | 5.93 (.40) | |
| PS_c4 | How would you rate doctor's instructions about symptoms to report and when to seek further care? | 1 (7) | 2 (11) | 4 (24) | 11 (74) | 24 (152) | 32 (205) | 27 (172) | 5.72 (.37) | |
| PS_c5 | How would you rate the doctor's advice and help in making decisions about your care? | 2 (13) | 2 (11) | 4 (23) | 11 (73) | 24 (154) | 31 (200) | 27 (171) | 5.90 (.39) | |
| PS_c6 | How often do you leave your doctor's office with unanswered questions? | 1 (4) | 1 (9) | 3 (19) | 8 (49) | 20 (132) | 41 (263) | 26 (169) | 1.76 (.14) | |
|
PCAS Interpersonal Treatment Thinking about the personal aspects of the care you receive from your regular doctor… |
1=Very poor | 2=Poor | 3=Fair | 4=Good | 5=Very good | 6=Excellent | ||||
| PS_it1 | How would you rate the amount of time your doctor spends with you? | 1 (4) | 1 (9) | 5 (32) | 16 (104) | 27 (177) | 28 (178) | 22 (141) | 3.66 (.23) | |
| PS_it2 | How would you rate doctor's patience with your questions or worries? | 1 (5) | 0 (3) | 3 (19) | 13 (85) | 23 (150) | 28 (178) | 32 (205) | 5.64 (.34) | |
| PS_it3 | How would you rate doctor's friendliness and warmth toward you? | 0 (3) | 1 (8) | 3 (21) | 9 (57) | 24 (155) | 27 (176) | 35 (225) | 7.26 (.49) | |
| PS_it4 | How would you rate doctor's caring and concern for you? | 1 (4) | 0 (3) | 4 (25) | 10 (67) | 25 (158) | 27 (174) | 33 (214) | 8.85 (.71) | |
| PS_it5 | How would you rate doctor's respect for you? | 1 (6) | 1 (8) | 2 (11) | 6 (41) | 22 (139) | 28 (183) | 40 (257) | 5.41 (.35) | |
| CPCI Interpersonal Communication | 1=Strongly disagree | 2 | 3 | 4 | 5 | 6=Strongly agree | ||||
| CP_ic1 | I can easily talk about personal things with this doctor. | 2 (13) | 7 (48) | 7(48) | 10 (62) | 16 (102) | 21(137) | 36 (235) | 2.12 (.16) | |
| CP_ic2 | Sometimes, this doctor does not listen to me. | 2 (15) | 53 (339) | 17 (107) | 7 (47) | 9 (55) | 7 (48) | 5 (34) | 3.20 (.22) | |
| CP_ic3 | This doctor always explains things to my satisfaction. | 2 (14) | 3 (22) | 5 (35) | 10 (66) | 16 (104) | 25 (163) | 37 (241) | 2.09 (.16) | |
| CP_ic4 | Sometimes, with this doctor, I don't bring up things that I'm worried about. | 2 (16) | 35 (225) | 18 (119) | 9 (57) | 15 (96) | 13 (86) | 7 (46) | 1.51 (.13) | |
| CP_ic5 | I don't always feel comfortable asking questions of this doctor. | 3 (17) | 41 (265) | 19 (125) | 8 (54) | 10 (63) | 11 (70) | 8 (51) | 1.81 (.16) | |
| CP_ic6 | Sometimes, I feel like this doctor ignores my concerns. | 3 (19) | 49 (315) | 18 (119) | 9 (56) | 9 (55) | 7 (45) | 6 (36) | 3.75 (.27) | |
| EUROPEP Clinical Behaviour | 1=Poor | 2 | 3 | 4 | 5= Excellent | Not applicable | ||||
| EU_cb1 | Making you feel you had time during consultations | 2 (15) | 3 (20) | 6 (39) | 15 (97) | 28 (182) | 44 (282) | 2 (10) | 2.91 (.21) | |
| EU_cb2 | Interest in your personal situation | 2 (15) | 3 (19) | 8 (49) | 13 (84) | 31 (203) | 41 (264) | 2 (11) | 3.63 (.27) | |
| EU_cb3 | Making it easy for you to tell him or her about your problems | 2 (15) | 3 (19) | 6 (38) | 16 (102) | 27 (171) | 46 (295) | 1 (5) | 3.80 (.28) | |
| EU_cb4 | Involving you in decisions about your medical care | 2 (15) | 2 (13) | 5 (30) | 12 (76) | 32 (204) | 45 (292) | 2 (15) | 2.99 (.20) | |
| EU_cb5 | Listening to you | 2 (16) | 2 (14) | 4 (27) | 13 (84) | 28 (181) | 49 (317) | 1 (6) | 4.50 (.36) | |
| *EU_cb6 | Keeping your records and data confidential | 3 (19) | 0 | 1 (5) | 5 (31) | 23 (147) | 66 (425) | 3 (18) | 1.93 (.19) | |
| *EU_cb7 | Quick relief of your symptoms | 3 (19) | 2 (10) | 3 (21) | 16 (106) | 35 (224) | 37 (236) | 5 (29) | 2.41 (.18) | |
| *EU_cb8 | Helping you to feel well so that you can perform your normal daily activities | 4 (23) | 2 (11) | 3 (19) | 12 (76) | 34 (220) | 42 (271) | 4 (25) | 2.96 (.24) | |
| *EU_cb9 | Thoroughness | 3 (17) | 2 (12) | 4 (23) | 15 (99) | 29 (185) | 47 (300) | 1 (9) | 4.19 (.32) | |
| *EU_cb10 | Physical examination of you | 3 (17) | 3 (21) | 2 (13) | 12 (78) | 29 (186) | 48 (312) | 3 (18) | 2.95 (.22) | |
| *EU_cb11 | Offering you services for preventing diseases | 3 (21) | 4 (26) | 5 (34) | 14 (88) | 29 (184) | 37 (238) | 8 (54) | 2.44 (.19) | |
| EU_cb12 | Explaining the purpose of tests and treatments | 3 (18) | 2 (16) | 5 (30) | 9 (58) | 30 (196) | 47 (305) | 3 (22) | 3.84 (.28) | |
| EU_cb13 | Telling you what you wanted to know about your symptoms and/or illness | 3 (19) | 3 (17) | 4 (26) | 10 (67) | 30 (195) | 47 (305) | 2 (16) | 4.60 (.37) | |
| EU_cb14 | Help in dealing with emotional problems related to your health status | 3 (18) | 6 (41) | 6 (39) | 16 (102) | 22 (143) | 31 (197) | 16 (105) | 3.39 (.27) | |
| EU_cb15 | Helping you understand the importance of following his or her advice | 3 (20) | 2 (14) | 4 (27) | 17 (111) | 32 (204) | 37 (238) | 5 (31) | 3.66 (.26) | |
| EU_cb16 | Knowing what s/he had done or told you during previous contacts | 3 (18) | 4 (27) | 6 (38) | 15 (95) | 32 (208) | 37 (238) | 3 (21) | 2.98 (.20) | |
|
IPC-II-Elicited Concerns How often did the doctor(s)… |
1= Never | 2= Rarely | 3= Sometimes | 4= Usually | 5= Always | |||||
| IP_cel1 | Really find out what your concerns were? | 3 (22) | 2 (15) | 10 (66) | 13 (82) | 36 (231) | 36 (229) | 4.46 (.31) | ||
| IP_cel2 | Let you say what you thought was important? | 3 (21) | 1 (9) | 6 (39) | 11 (69) | 37 (236) | 42 (271) | 3.24 (.23) | ||
| IP_cel3 | Take your health concerns very seriously? | 3 (22) | 1 (9) | 5 (31) | 10 (62) | 32 (207) | 49 (314) | 3.09 (.22) | ||
|
IPC-II-Explained Results How often did the doctor(s)… |
1=Never | 2=Rarely | 3= Sometimes | 4= Usually | 5=Always | |||||
| IP_cex1 | Explain your test results, such as blood tests, x-rays or cancer screening tests? | 5 (34) | 3 (17) | 8 (49) | 11 (72) | 26 (170) | 47 (303) | 4.76 (.34) | ||
| IP_cex2 | Clearly explain the results of your physical exam? | 5 (32) | 3 (17) | 8 (52) | 10 (64) | 28 (181) | 46 (299) | 6.29 (.47) | ||
| IP_cex3 | Tell you what could happen if you didn't take a medicine that they prescribed for you? | 5 (35) | 7 (46) | 10 (64) | 12 (80) | 29 (190) | 36 (230) | 2.41 (.18) | ||
| IP_cex4 | Tell you about side effects you might get from a medicine? | 5 (30) | 6 (37) | 12 (78) | 15 (99) | 27 (177) | 35 (224) | 2.00 (.16) | ||
|
IPC-II-Patient-Centred Decision-Making How often did… |
1=Never | 2=Rarely | 3= Sometimes | 4= Usually | 5=Always | |||||
| IP_dm1 | You and your doctor(s) work out a treatment plan together? | 7 (42) | 17 (107) | 14 (91) | 14 (91) | 31 (198) | 18 (116) | 2.58 (.17) | ||
| IP_dm2 | If there were treatment choices, how often did the doctor(s) ask you if you would like to help decide the treatment? | 9 (59) | 18 (113) | 16 (101) | 12 (77) | 27 (177) | 18 (118) | 3.30 (.21) | ||
| IP_dm3 | The doctor(s) ask if you would have any problems following what they recommended? | 7 (46) | 18 (118) | 13 (87) | 18 (114) | 25 (162) | 18 (118) | 5.53 (.32) | ||
| IP_dm4 | The doctor(s) ask if you felt you could do the recommended treatment? | 7 (47) | 19 (125) | 13 (86) | 13 (83) | 26 (167) | 21 (137) | 5.38 (.34) | ||
|
IPC-II-Hurried Communication How often did the doctor(s)… |
1=Never | 2=Rarely | 3=Sometimes | 4=Usually | 5=Always | |||||
| IP_hc1 | Speak too fast? | 3 (22) | 41 (263) | 36 (235) | 15 (99) | 3 (18) | 1 (8) | 2.28 (.18) | ||
| IP_hc2 | Use words that were hard to understand? | 3 (21) | 35 (228) | 41 (267) | 17 (112) | 2 (14) | 0 (3) | 1.51 (.14) | ||
| IP_hc3 | Ignore what you told them? | 4 (24) | 47 (300) | 32 (204) | 14 (88) | 3 (21) | 1 (8) | 3.29 (.24) | ||
| IP_hc4 | Appear to be distracted when they were with you? | 4 (24) | 44 (284) | 36 (230) | 12 (75) | 4 (23) | 1 (9) | 2.59 (.20) | ||
| IP_hc5 | Seem bothered if you asked several questions? | 4 (26) | 52 (333) | 26 (168) | 12 (79) | 5 (31) | 1 (8) | 3.56 (.26) | ||
Item removed from further analysis.
The parametric estimate of the discriminatory capacity within the original subscale is shown in the right-hand column of Table 1 (Item Discrimination), with ≥1.0 indicating that the item discriminates between different levels of the subscale score. All items demonstrate good to excellent discriminability.
Table 2 presents the descriptive statistics for the subscale scores. The normalized means and standard deviations for the subscales differ substantially from one subscale to another. All subscales except the IPC-II Patient-Centred Decision-Making are skewed towards positive values, with the median consistently higher than the mean. The internal consistency of all the scales is excellent.
TABLE 2.
Mean and distributional values for interpersonal communication subscales, values normalized to a 0-to-10 scale (n=645)
| Developer's Subscale Name | Number of Items | Mean | SD | Quartiles | Cronbach's Alpha | ||
|---|---|---|---|---|---|---|---|
| Q1 (25%) | Q2 (50%) | Q3 (75%) | |||||
| PCAS Communication | 6 | 7.33 | 2.11 | 6.00 | 7.67 | 9.00 | .95 |
| PCAS Interpersonal Treatment | 5 | 7.44 | 2.17 | 6.00 | 7.80 | 9.60 | .96 |
| CPCI Interpersonal Communication | 6 | 7.19 | 2.32 | 5.30 | 7.67 | 9.30 | .96 |
| EUROPEP Clinical Behaviour | 10 | 7.70 | 2.30 | 6.50 | 8.25 | 9.50 | .96 |
| IPC-II Elicited Concerns | 3 | 7.81 | 2.16 | 6.70 | 8.33 | 10.00 | .86 |
| IPC-II Explained Results | 4 | 7.40 | 2.49 | 5.60 | 8.13 | 9.40 | .88 |
| IPC-II Patient-Centred Decision-Making | 4 | 5.41 | 3.15 | 2.50 | 5.63 | 7.50 | .91 |
| IPC-II Hurried Communication | 5 | 8.01 | 1.77 | 7.00 | 8.42 | 9.50 | .85 |
Table 3 presents the Pearson correlations between the subscales for interpersonal communication. All the subscales correlate relatively well. Not surprisingly, the two most correlated (.82) were from the same instrument: PCAS Communication and PCAS Interpersonal Treatment. The IPC-II Patient-Centred Decision-Making showed the lowest correlation overall (.46 to .63), although the subscales correlated most highly within their own attribute family. Again not surprisingly, they also correlated highly with subscales in Relational Continuity that measure knowledge of the patient (.48 to .71) and with the IPC-II Interpersonal Style subscale, which was mapped to the attribute of respectfulness (.59–.79). The interpersonal communication subscales also correlate strongly with PCAS Trust (.49–.73), as shown in Table 3.
TABLE 3.
Partial correlations§ between interpersonal communication subscales. Only correlations significantly different from zero are provided.
| PCAS* Communication | PCAS Interpersonal Treatment | CPCI** Interpersonal Communication | EUROPEP Clinical Behaviour | IPC-II*** Elicited Concerns | IPC-II Explained Results | IPC-II Patient-Centred Decision-Making | IPC-II Hurried Communication | |
|---|---|---|---|---|---|---|---|---|
| PCAS Communication | 1.00 | 0.82 | 0.62 | 0.72 | 0.59 | 0.50 | 0.46 | 0.63 |
| PCAS Interpersonal Treatment | 0.82 | 1.00 | 0.63 | 0.76 | 0.67 | 0.48 | 0.49 | 0.64 |
| CPCI Interpersonal Communication | 0.62 | 0.63 | 1.00 | 0.69 | 0.65 | 0.54 | 0.44 | 0.62 |
| EUROPEP Clinical Behaviour | 0.72 | 0.76 | 0.69 | 1.00 | 0.75 | 0.65 | 0.61 | 0.72 |
| IPC Elicited Concerns | 0.59 | 0.67 | 0.65 | 0.75 | 1.00 | 0.60 | 0.57 | 0.65 |
| IPC Explained Results | 0.50 | 0.48 | 0.54 | 0.65 | 0.60 | 1.00 | 0.63 | 0.59 |
| IPC Patient-Centred Decision-Making | 0.46 | 0.49 | 0.44 | 0.61 | 0.57 | 0.63 | 1.00 | 0.47 |
| IPC Hurried Communication | 0.63 | 0.64 | 0.62 | 0.72 | 0.65 | 0.59 | 0.47 | 1.00 |
| PCAS Trust | 0.68 | 0.68 | 0.68 | 0.73 | 0.63 | 0.55 | 0.49 | 0.60 |
Controlling for study design variables: province, educational achievement, geographic location.
PCAS: Primary Care Assessment Survey
CPCI: Components of Primary Care Instrument
IPC: Interpersonal Processes of Care
Do all items measure a single construct?
Missing values reduced our effective sample size for factor analysis from 645 to 427. Those patients included in the factor analysis tended to be in poorer health, to have been affiliated longer with their physician and to have consulted more often than those excluded. Imputing the missing values using a maximum likelihood method (Rubin 1987) increased our sample size and improved the fit statistics in the confirmatory factor analysis models, but it did not change the magnitude of loadings or the direction of our conclusions.
Most items load reasonably well (>.30) on a single factor with common factor analysis. However, in confirmatory factor analysis a unidimensional model did not demonstrate a good fit with a root mean square error of approximation (RMSEA) of .122, well above the .05 considered to demonstrate good fit even though the normed fit index (NFI) of .94 was above the .90 standard for good fit. Model fit improved when the items were grouped in their parent subscales and then linked to a single underlying construct presumed to be interpersonal communication, as shown in Figure 1. This model fit significantly better than the unidimensional model, as shown by the difference in chi-square values (χ2 = 6,299 – 2,150 = 4,149, 8 df, p<.0001); the RMSEA of .06 and the NFI of .98 indicated good fit.
FIGURE 1.
Parameter estimations for a structural equation model showing item loadings of items on parent subscales (first order) indicating the correlations between the scales
Item loadings are lowest for the CPCI Interpersonal Communication subscale, and IPC-II Explain Results and IPC-II Patient-Centred Decision-Making have weaker loadings than other subscales on the construct presumed to be interpersonal communication.
How do underlying factors fit with operational definition?
Exploratory factor analysis suggested three underlying factors in the pool of all these items. Using our operational definition as a guide, we judged that the first factor (eigenvalue = 26.56) seemed to assess the provider's ability “to elicit and understand patient concerns” (eliciting), the second (eigenvalue = 2.45) “to explain healthcare issues” (explaining) and the third (eigenvalue = 1.34) “to engage in shared decision-making” (decision-making). Most items load on a first factor that relates conceptually to the eliciting dimension, including all items in PCAS Interpersonal Treatment, IPC-II Elicit and IPC-II Hurried Communication. Most CPCI Interpersonal Communication items, half of the EUROPEP-I items and the IPC-II Explain subscales load on the factor that relates conceptually to the explaining dimension, but the loadings are modest, aside from three of the four questions of the IPC-II Explain subscale. Finally, the decision-making dimension is not well represented in the scales we studied. Only six items loaded on the factor: the four questions of the IPC-II Patient-Centred Decision-Making subscale – with high loadings – and one question each on the PCAS Communication and EUROPEP-I Clinical Behaviour subscales.
A confirmatory analysis model, in which the items are grouped by eliciting, explaining and decision-making, are then associated to a single factor representing interpersonal communication (as shown in Figure 2). Compared to the unidimensional model, we see a moderate improvement in goodness of fit using the chi-square criterion (χ2 = 6,299 – 5,039 = 1,250, 3 df, p<.001).
FIGURE 2.
Parameter estimates for a structural equation model where items are associated with the (first-order) constructs of eliciting, explaining and decision-making, which are linked to a single underlying construct IC_RESP (second order) presumed to be interpersonal communication
Figure 2 shows that the dimensions of eliciting and explaining are highly correlated (.92), but decision-making has much lower correlations with these two dimensions: .70 and –.79, respectively. Some items do not have high loadings and have a high proportion of residual error (shown to the right of each item), either because they are not discriminatory or because they relate better to another construct that is not part of the latent variable. Overall, the items in the PCAS and EUROPEP-I subscales seem to fit best with the subdimensions with which they were associated.
Individual item performance
We conducted item response analysis to evaluate the performance of individual items as a function of both the parent subscales and the dimensions of eliciting, explaining and decision-making.
For the eliciting dimension, the items from the PCAS and EUROPEP-I subscales demonstrated the best performance overall. The items showed relatively good discriminability, with the item score increasing with the total eliciting score. The probability of selecting low-rating options – e.g., 1 (very poor) to 3 (fair) – occurs appropriately in the negative zone of eliciting, and high ratings – 5 (good) and 6 (excellent) – in the most positive. However, the negative options are rarely endorsed, and in the PCAS, the “4=good” option is mostly endorsed when eliciting is below average; in the EUROPEP-I, “5=excellent” is endorsed at the average level of eliciting. The PCAS response scale consists essentially of responses 3 to 6, and the EUROPEP-I of 2 to 5. But peak probabilities of responding are well differentiated from one another.
The CPCI items perform the next best, showing relatively good discriminability, especially in the negative zone of eliciting. But the full response options are not fully exploited nor equally informative. The responses are either 1 (strongly agree) or 6 (strongly disagree), and the peak probabilities for the other response options are not clearly differentiated, suggesting this instrument functions as a three-point scale. This pattern is even more extreme in the IPC-II subscales, where any response other than “5=always” is associated with below-average eliciting and should be interpreted negatively.
These same patterns of performance are seen for each instrument in the dimensions of explaining and decision-making, although the latent variable is composed of a smaller number of items and interpretation necessarily requires prudence.
Discussion and Conclusion
Our results suggest that the validated scales that map to interpersonal communication do indeed have a single underlying construct that includes three distinct dimensions as stated in our operational definition: eliciting problems and concerns, explaining illness management and involving the patient in decision-making. This finding supports the hypothesis we formed based on our operational definition of interpersonal communication and adds to a body of knowledge along the same lines (Epstein et al. 2005; Hall et al. 1996; Mead and Bower 2002; Saba et al. 2006; Stewart 1995).
As mentioned earlier, our aim was not to evaluate the metric proprieties of entire instruments. All these instruments have been developed according to somewhat different conceptual frameworks and measurement approaches. Our aim was to compare their capacity to measure the construct of interpersonal communication according to the operational definition developed by PHC experts and providers (Haggerty et al. 2007). Unlike some others (Stewart 1995; Stewart et al. 1999), we did not explore patients' definitions of this attribute. Our aim was to ascertain whether subscales from different instruments that appear to measure this dimension actually do so.
The PCAS Communication subscale demonstrates good metric properties and discriminability and includes items that measure the eliciting and explaining dimensions of interpersonal communication plus one item on decision-making. The EUROPEP-I Clinical Behaviour subscale had only acceptable metric properties and discriminability; admittedly, however, the EUROPEP-I Clinical Behaviour component is not intended to relate specifically to interpersonal communication. The new version of the EUROPEP includes several revisions to communication items that may improve performance, but it still applies to general clinical encounters of which communication is only a part (Wensing 2006). The CPCI Interpersonal Communication subscale relates to the eliciting and explaining dimensions and, despite certain measurement problems, has good capacity to detect and discriminate problems in this dimension. The IPC-II has subscales addressing all three dimensions of interpersonal communication, but they load less on the construct and their items have poorer discriminability than subscales from other instruments. Some items map more to the respectfulness construct (PCAS Interpersonal Treatment and IPC-II Hurried Communication). The IPC-II subscales of Explained Results and Patient-Centred Decision-Making have high levels of missing values, creating problems for evaluators. The questions may not be clear or the response scale may be inappropriate for the questions, as indicated in our cognitive testing (Haggerty, Santor et al. 2011).
Patient involvement with decisions related to care is particularly poorly developed in all these instruments. In addition, much theoretical and empirical work has been done on the measurement of two important concepts related to interpersonal communication that are not considered in the instruments we studied: patient-centredness measurement tools, in which the notion of agreement on the definition of the problem and on the course of action is core (and has been associated with outcomes) (Epstein et al. 2005; Stewart 1995; Little et al. 2001), and shared decision-making measurement instruments (Elwyn et al. 2001).
As suggested earlier in this paper, very few general instruments that propose to measure communication between providers and patients have strong conceptual foundations. Those that have been associated more often with quality-of-care indicators, and less frequently with health outcomes, are the EUROPEP-I and the PCAS. For example, patients in a comparative European study based on the EUROPEP-I reported good interpersonal communication with their physicians despite important differences between countries in ratings of the organizational dimensions of care (Grol et al. 2000). Similar observations were made in the United States in studies using the PCAS (Safran et al. 1994, 2000, 2002). Although these observations may seem to suggest that patients are satisfied with their communication with their usual providers independently of the primary care model, recent comparative studies suggest that the type of organizational model makes a difference in most care experience indicators – the professional single-provider model being the best performing, and more complex organizational models performing less well (Lamarche et al. 2003). Indeed, some studies suggest that organizational interventions such as team care can disrupt the relationship between patients and their primary care providers and have a negative impact on the quality of interpersonal communication (Rodriguez et al. 2007; Safran 2003). It should be noted, however, that none of these instruments permits the evaluation of the experience of interpersonal communication from a team care perspective, an area where more research is certainly needed.
As organizational interventions increasingly change the day-to-day experience of care, they may have a growing impact on the quality of provider–patient interpersonal communication. Conversely, positive experiences with interpersonal communication and respectfulness may buffer patients from negative experiences associated with organizational changes. Indeed, we found that the Interpersonal Communication subscales provided the greatest discrimination between excellent, average and poor experience of care when our respondents were asked to rate their overall experience (Haggerty 2011). Hence the importance of being able to monitor this attribute of primary care, which is deemed essential by users, providers and decision-makers.
Acknowledgements
This research was funded by the Canadian Institutes of Health Research. Dr. Beaulieu was holder of the Sadok Basrour Research Chair in Family Medicine Research at the Université de Montréal during the conduct of this study. The authors thank Beverley Lawson for conducting the survey in Nova Scotia and Christine Beaulieu in Quebec; Fatima Bouharaoui for performing the analysis; Jocelyne Gagné and Donna Riley for support in the preparation of this manuscript.
Contributor Information
Marie-Dominique Beaulieu, Chaire Dr Sadok Besrour en médecine familiale, Centre de recherche du Centre hospitalier de l'Université de Montréal, Montréal, QC.
Jeannie L. Haggerty, Department of Family Medicine, McGill University, Montreal, QC.
Christine Beaulieu, St. Mary's Research Centre, St. Mary's Hospital Center, Montreal, QC.
Fatima Bouharaoui, St. Mary's Research Centre, St. Mary's Hospital Center, Montreal, QC.
Jean-Frédéric Lévesque, Centre de recherche du Centre hospitalier de l'Université de Montréal, Montréal, QC.
Raynald Pineault, Centre de recherche du Centre hospitalier de l'Université de Montréal, Montréal, QC.
Frederick Burge, Department of Family Medicine, Dalhousie University, Halifax, NS.
Darcy A. Santor, School of Psychology, University of Ottawa, Ottawa, ON.
References
- Brody D.S., Miller S.M., Lerman C.E., Smith D.G., Caputo C. 1989. “Patient Perception of Involvement in Medical Care: Relationship to Illness Attitudes and Outcomes.” Journal of General Internal Medicine 4: 506–11 [DOI] [PubMed] [Google Scholar]
- Brown J.B., Stewart M., Ryan B.L. 2001. Assessing Communication between Patients and Physicians: The Measure of Patient-Centred Communication. Working Paper Series 95–2. London, ON: Centre for Studies in Family Medicine [Google Scholar]
- Elwyn G., Edwards A., Mowle S., Wensing M., Wilkinson C., Kinnersley P., Grol R. 2001. “Measuring the Involvement of Patients in Shared Decision-Making: A Systematic Review of Instruments.” Patient Education and Counseling 43: 5–22 [DOI] [PubMed] [Google Scholar]
- Epstein R.M., Franks P., Fiscella K., Shields C.G., Meldrum S.C., Kravitz R.L., Duberstein P.R. 2005. “Measuring Patient-Centered Communication in Patient–Physician Consultations: Theoretical and Practical Issues.” Social Science and Medicine 61: 1516–28 [DOI] [PubMed] [Google Scholar]
- Flocke S. 1997. “Measuring Attributes of Primary Care: Development of a New Instrument.” Journal of Family Practice 45(1): 64–74 [PubMed] [Google Scholar]
- Flora D.B., Curran P.J. 2004. “An Empirical Evaluation of Alternative Methods of Estimation for Confirmatory Factor Analysis with Ordinal Data.” Psychological Methods 9(4): 466–91 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Grol R., Wensing M.Task Force on Patient Evaluations of General Practice 2000. Patients Evaluate General/Family Practice: The EUROPEP-I Instrument. Nijmegen, Netherlands: Centre for Quality of Care Research, Raboud University; [PMC free article] [PubMed] [Google Scholar]
- Haggerty J.L. 2011. “Measurement of Primary Healthcare Attributes from the Patient Perspective.” Healthcare Policy 7 (Special Issue): 13–20 [PMC free article] [PubMed] [Google Scholar]
- Haggerty J., Burge F., Lévesque J.-F., Gass D., Pineault R., Beaulieu M.-D., Santor D. 2007. “Operational Definitions of Attributes of Primary Health Care: Consensus among Canadian Experts.” Annals of Family Medicine 5: 336–44 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Haggerty J.L., Beaulieu C., Lawson B., Santor D.A., Fournier M., Burge F. 2011. “What Patients Tell Us about Primary Healthcare Evaluation Instruments: Response Formats, Bad Questions and Missing Pieces.” Healthcare Policy 7 (Special Issue): 66–78 [PMC free article] [PubMed] [Google Scholar]
- Hall J.A., Roter D.L., Milburn M.A., Daltroy L.H. 1996. “Patients' Health as a Predictor of Physician and Patient Behavior in Medical Visits: A Synthesis of Four Studies.” Medical Care 34: 1205–18 [DOI] [PubMed] [Google Scholar]
- Headache Study Group of the University of Western Ontario 1986. “Predictors of Outcome in Headache Patients Presenting to Family Physicians – A One Year Prospective Study.” Headache 6: 285–94 [PubMed] [Google Scholar]
- Lamarche P.-A., Beaulieu M.-D., Pineault R., Contandriopoulos A.-P., Denis J.-L., Haggerty J. 2003. Choices for Change: The Path for Restructuring Primary Healthcare Services in Canada. Ottawa: Canadian Health Services Research Foundation [Google Scholar]
- Little P., Everitt H., Williamson I., et al. 2001. Observational Study of Effect of Patient Centredness and Positive Approach on Outcomes of General Practice Consultations. British Medical Journal 323(7318): 908–11 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Mead N., Bower P. 2002. “Patient-Centred Consultations and Outcomes in Primary Care: A Review of the Literature.” Patient Education and Counseling 48: 51–61 [DOI] [PubMed] [Google Scholar]
- Rodriguez H.P., Rogers W.H., Marshall R.E., Safran D.G. 2007. “Multidisciplinary Primary Care Teams: Effects on the Quality of Clinician–Patient Interactions and Organizational Features of Care.” Medical Care 45: 19–27 [DOI] [PubMed] [Google Scholar]
- Rubin D.B. 1987. Multiple Imputation for Nonresponse in Surveys. Hoboken, NJ: John Wiley [Google Scholar]
- Saba G.W., Wong S.T., Schillinger D., Fernandez A., Somkin C.P., Wilson C.C., Grumbach K. 2006. “Shared Decision Making and the Experience of Partnership in Primary Care.” Annals of Family Medicine 4: 54–62 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Safran D.G. 2003. “Defining the Future of Primary Care: What Can We Learn from Patients?” Annals of Internal Medicine 138: 248–55 [DOI] [PubMed] [Google Scholar]
- Safran D.G., Kosinski J., Tarlov A.R., Rogers W.H., Taira D.A., Lieberman N., Ware J.E. 1998. “The Primary Care Assessment Survey: Tests of Data Quality and Measurement Performance.” Medical Care 36(5): 728–39 [DOI] [PubMed] [Google Scholar]
- Safran D.G., Rogers W.H., Tarlov A.R., Inui T.S., Taira D.A., Montgomery J.E. 2000. “Organizational and Financial Characteristics of Health Plans: Are They Related to Primary Care Performance?” Archives of Internal Medicine 160: 69–76 [DOI] [PubMed] [Google Scholar]
- Safran D.G., Taira D.A., Rogers W.H., Kosinski M., Ware J.E., Tartov A.R. 1998. “Linking Primary Care Performance to Outcomes of Care.” Journal of Family Practice 47: 213–20 [PubMed] [Google Scholar]
- Safran D.G., Tarlov A.R., Rogers W.H. 1994. “Primary Care Performance in Fee-for-Service and Prepaid Healthcare Systems. Results from the Medical Outcomes Study.” Journal of the American Medical Association 271: 1579–86 [PubMed] [Google Scholar]
- Safran D.G., Wilson I.B., Rogers W.H., Montgomery J.E., Chang H. 2002. “Primary Care Quality in the Medicare Program: Comparing the Performance of Medicare Health Maintenance Organizations and Traditional Fee-for-Service Medicare.” Archives of Internal Medicine 162: 757–65 [DOI] [PubMed] [Google Scholar]
- Santor D.A., Haggerty J.L., Lévesque J.-F., Burge F., Beaulieu M.-D., Gass D., Pineault R. 2011. “An Overview of Confirmatory Factor Analysis and Item Response Analysis Applied to Instruments to Evaluate Primary Healthcare.” Healthcare Policy 7 (Special Issue): 79–92 [PMC free article] [PubMed] [Google Scholar]
- Simpson M., Buckman R., Stewart M., Maguire P., Lipkin M., Novack D., Till J. 1991. “Doctor–Patient Communication: The Toronto Consensus Statement.” British Medical Journal 303: 1385–87 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stange K.C., Acheson L.S. 2006. “Communication in the Era of ‘Personalized’ Medicine.” Annals of Family Medicine 4: 194–96 [Google Scholar]
- Starfield B., Wray C., Hess K., Gross R., Birk P.S., D'Lugoff B.C. 1981. “The Influence of Patient–Practitioner Agreement on Outcome of Care.” American Journal of Public Health 71: 127–32 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stewart A.L., Nápoles-Springer A., Pérez-Stable E.J. 1999. “Interpersonal Processes of Care in Diverse Populations.” Milbank Quarterly 77(3): 305–39, 274 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Stewart M. 1995. “Effective Physician–Patient Communication and Health Outcomes: A Review.” Canadian Medical Association Journal 152: 1423–33 [PMC free article] [PubMed] [Google Scholar]
- Thom D., Campbell B. 1997. “Patient–Physician Trust: An Exploratory Study.” Journal of Family Practice 44: 169–76 [PubMed] [Google Scholar]
- Wensing M.E. 2006. EUROPEP 2006: Revised Europep Instrument and User Manual. Nijmegen, Netherlands: Centre for Quality of Care Research, Raboud University; Retrieved July 8, 2011. <http://www.topaseurope.eu/?q=node/13>. [Google Scholar]
- Wensing M., Mainz J., Grol R. 2000. “A Standardised Instrument for Patient Evaluations of General Practice Care in Europe.” European Journal of General Practice 6: 82–87 [Google Scholar]