There is Little or No Association Between Independently Assessed Communication Strategies and Patient Ratings of Clinician Empathy

Abstract

Background

Quality of care is increasingly assessed and incentivized using measures of patient-reported outcomes and experience. Little is known about the association between measurement of clinician communication strategies by trained observers and patient-rated clinician empathy (a patient-reported experience measure). An effective independent measure could help identify and promote clinician behaviors associated with good patient experience of care.

Questions/purposes

(1) What is the association between independently assessed clinician communication effectiveness and patient-rated clinician empathy? (2) Which factors are associated with independently assessed communication effectiveness?

Methods

One hundred twenty adult (age > 17 years) new or returning patients seeking musculoskeletal specialty care between September 2019 and January 2020 consented to video recording of their visit followed by completion of questionnaires rating their perceptions of providers’ empathy levels in this prospective study. Patients who had operative treatment and those who had nonoperative treatment were included in our sample. We pooled new and returning patients because our prior studies of patient experience found no influence of visit type and because we were interested in the potential influences of familiarity with the clinician on empathy ratings. We did not record the number of patients or baseline data of patients who were approached, but most patients (> 80%) were willing to participate. For 7% (eight of 120 patients), there was a malfunction with the video equipment or files were misplaced, leaving 112 records available for analysis. Patients were seen by one provider among four attending physicians, four residents, or four physician assistants or nurse practitioners. The primary study question addressed the correlation between patient-rated clinician empathy using the Jefferson Scale of Patient Perceptions of Physician Empathy and clinician communication effectiveness, independently rated by two communication scholars using the Liverpool Communication Skills Assessment Scale. Based on a subset of 68 videos (61%), the interrater reliability was considered good for individual items on the Liverpool Communication Skills Assessment Scale (intraclass correlation coefficient [ICC] 0.78 [95% confidence interval (CI) 0.75 to 0.81]) and excellent for the sum of the items (that is, the total score) (ICC = 0.92 [95% CI 0.87 to 0.95]). To account for the potential association of personal factors with empathy ratings, patients completed measures of symptoms of depression (the Patient-Reported Outcome Measurement Information System depression computerized adaptive test), self-efficacy in response to pain (the two-item Pain Self-Efficacy Questionnaire), health anxiety (the five-item Short Health Anxiety Inventory), and basic demographics.

Results

Accounting for potentially confounding variables, including specific clinicians, marital status, and work status in the multivariable analysis, we found higher independent ratings of communication effectiveness had a slight association (odds ratio [OR] 1.1 [95% CI 1.0 to 1.3]; p = 0.02) with higher (dichotomized) ratings of patient-rated clinician empathy, while being single was associated with lower ratings (OR 0.40 [95% CI 0.16 to 0.99]; p = 0.05). Independent ratings of communication effectiveness were slightly higher for women (regression coefficient 1.1 [95% CI 0.05 to 2.2]); in addition, two of the four attending physicians were rated notably higher than the other 10 participants after controlling for confounding variables (differences up to 5.8 points on average [95% CI 2.6 to 8.9] on a 36-point scale).

Conclusion

The observation that ratings of communication effectiveness by trained communication scholars have little or no association with patient-rated clinician empathy suggests that either effective communication is insufficient for good patient experience or that the existing measures are inadequate or inappropriate. This line of investigation might be enhanced by efforts to identify clinician behaviors associated with better patient experience, develop reliable and effective measures of clinician behaviors and patient experience, and use those measures to develop training approaches that improve patient experience.

Level of Evidence

Level I, prognostic study.

Introduction

Background

The documented association of better patient-reported experience measures (ratings of clinician trust, empathy, and communication effectiveness) with greater treatment adherence [9, 18, 21, 30] supports the notion that efforts to improve patient experience might boost the quality and value of healthcare [6, 10]. Payment programs such as the Medicare Hospital Value-Based Purchasing Program have financial incentives for hospital systems to deliver better patient experience. Given evidence that patient experience is associated with the quality rather than the duration of a visit with a musculoskeletal specialist [16, 24, 31], one can surmise that efforts to improve patient experience can benefit from the ability to identify and foster the communication and relationship strategies that are most strongly associated with patient-rated clinician trust, empathy, and communication effectiveness.

Rationale

What is lacking is a measurement tool that independent observers can use to help identify opportunities for clinicians to improve a patient’s experience. To our knowledge, the best-available current tools are designed to evaluate student-doctor interactions with actor patients in training scenarios, such as the Liverpool Communication Skills Assessment Scale. As a next step in the development of interventions to improve patient experience, we need a better understanding of the correlation between the currently available patient experience measures and independent ratings of communication effectiveness.

We therefore asked: (1) What is the association between independently assessed clinician communication effectiveness and patient-rated clinician empathy? (2) Which factors are associated with independently assessed communication effectiveness?

Patients and Methods

Study Design and Setting

Participating clinicians from a single academic orthopaedic office in a large urban area were selected for their willingness to have their patient visits videotaped as part of this study. We prospectively enrolled 120 adult (aged > 17 years) patients with various upper and lower extremity diagnoses between September 2019 and January 2020. Patients who had operative treatment and those who had nonoperative treatment were included in our sample. We did not record the number of patients or baseline data of patients who were approached, but most patients (> 80%) who were approached were willing to participate. Patients were seen by one provider among four attending physicians, four residents, and four physician assistants or nurse practitioners.

Participants

All new and returning English-speaking and Spanish-speaking patients were asked to participate by a research assistant, and informed consent was obtained. We pooled new and returning patients because our prior studies of patient experience found no influence of visit type [16, 31]. Enrollment of new and returning patients also provided another opportunity to measure the potential influences of familiarity with the clinician on empathy ratings. We excluded all patients with cognitive and/or language deficiencies that would impede completion of questionnaires.

Patients’ Baseline Data

Of approximately 140 patients who were approached (those who declined were not tracked, but there were not many), 120 patients completed the baseline questionnaires. In 7% (eight of 120 patients), there was a technical malfunction with the video equipment or files were misplaced, leaving 112 video records available for analysis. Most patients were men (54% [65 of 120]), with a mean age of 46 ± 15 years (Table 1). Most patients were college-educated (68% [81 of 120 patients]), and the greatest proportion were married or had a partner (43% [51 of 120 patients]).

Table 1.

Patient demographics^a

Variable	Value
Age in years, mean ± SD	46 ± 15
Women, % (n)	45 (54)
Race or ethnicity, % (n)
White	58 (69)
Hispanic	25 (30)
Black	8.4 (10)
Other	8.4 (10)
Marital status, % (n)
Married or partner	43 (51)
Single	41 (49)
Divorced or widowed	16 (19)
Education level, % (n)
High school or lower	33 (39)
2-year college	18 (21)
4-year college	26 (31)
Postgraduate degree	24 (28)
Work status, % (n)
Employed	62 (74)
Unemployed	13 (15)
Disabled	10 (12)
Retired	7.6 (9)
Other	7.6 (9)
Annual income in USD, % (n)
< 24,000	38 (40)
24,000 to 45,999	13 (14)
46,000 to 74,999	12 (13)
75,000 to 121,000	12 (13)
> 121,000	25 (26)
Insurance status, % (n)
Commercial	45 (53)
Safety netb	40 (48)
Medicare	14 (17)
Workers compensation	0.84 (1)
Follow-up visit, % (n)	17 (20)
Upper extremity diagnosis, % (n)	82 (94)
Fracture, % (n)	14 (17)
PROMIS Depression, mean ± SD	49 ± 10
PSEQ-2, median (IQR)	9 (6-12)
SHAI-5, median (IQR)	10 (8-12)
JSPPPE score, median (IQR)	30 (25-35)
Provider, % (n)
1	34 (40)
2	5.9 (7)
3	21 (25)
4	5.1 (6)
5	19 (23)
Other	14 (17)

^aOne patient completed a videotaped consult but did had missing baseline demographics. Therefore, all variables were calculated using N = 119 except for annual income (N = 106), upper extremity (N = 115), and provider (N = 118)

^bSafety net includes Medicaid, Medical Access Program, and uninsured patients. PROMIS = Patient-Reported Outcomes Measurement Information System; PSEQ-2 = Pain Self-Efficacy Questionnaire, two-item; SHAI-5 = Short Health Anxiety Inventory, five-item; JSPPPE= Jefferson Scale of Patient Perceptions of Physician Empathy.

Interventions

The visits were recorded with a small digital video camera on a tripod, without the research assistant present. At the end of the visit, patients were asked to complete the Jefferson Scale of Patient Perceptions of Physician Empathy, the Patient-Reported Outcome Measurement Information System depression computerized adaptive test, the two-item Pain Self-Efficacy Questionnaire, the five-item Short Health Anxiety Inventory, and a questionnaire of basic demographics (including gender, age, and level of education). Most patients met with a single clinician. If a second clinician was involved, the video ended before the second clinician entered the conversation. The clinicians, regardless of level of training, did not receive any special instruction in communication techniques.

Measurements and Variables

After data were collected, a team of two independent communication experts (LEB and EC) rated clinician communication effectiveness according to the Liverpool Communication Skills Assessment Scale. The Liverpool Communication Skills Assessment Scale was developed for use in objective structured clinical examinations and is a comprehensive and validated instrument that measures communication effectiveness [11]. To our knowledge, there are no studies that have used this outside a teaching setting, but all tested elements are important factors in patient-clinician interactions, based on prior research [11]; use of this scale to evaluate clinical interactions has face validity; and use of an established questionnaire has advantages over creating a new questionnaire. The questionnaire consists of 12 items that are key elements in clinical interactions, such as an appropriate greeting, introduction, eye contact, empathic reflection, and sensitivity of questions. For each element, the clinician is rated on a 4-point Likert scale, ranging from “unacceptable” to “good.” Scores range from 0 to 36, with higher scores indicating higher communication effectiveness. The interrater reliability of the two raters was considered good for individual items on the Liverpool Communication Skills Assessment Scale (intraclass correlation coefficient [ICC] 0.78 [95% confidence interval (CI) 0.75 to 0.81]) and excellent for the sum of the items (that is, the total score) (ICC = 0.92 [95% CI 0.87 to 0.95]) [14].

In addition, the participants completed the Jefferson Scale of Patient Perceptions of Physician Empathy, Patient-Reported Outcome Measurement Information System depression computerized adaptive test, Pain Self-Efficacy Questionnaire, five-item Short Health Anxiety Inventory, and a survey of basic demographics. The Jefferson Scale of Patient Perceptions of Physician Empathy is a five-item scale of patient-perceived empathy that asks patients to rate their agreement on a 7-point Likert scale ranging from “strongly disagree” to “strongly agree.” The scale contains statements such as “[The doctor] understands my emotions, feelings and concerns” and “[The doctor] asks about what is happening in my daily life.” Scores range from 5 to 35, with higher scores indicating greater patient-perceived empathy and satisfaction. The Patient-Reported Outcome Measurement Information System depression computerized adaptive test is a validated and reliable patient-reported outcome tool that measures symptoms of depression on a continuous scale [7, 8, 25]. Scores are normed to the United States general population, with higher scores indicating greater symptoms of depression. The mean is 50, and 10 above or below the mean represents one standard deviation. The two-item Pain Self-Efficacy Questionnaire is the two-item version of the full Pain Self-Efficacy Questionnaire and measures effective pain coping in response to nociception [4, 15, 22]. Scores range from 0 to 12, with higher scores indicating greater pain self-efficacy. The five-item Short Health Anxiety Inventory is the five-item short form of the full Short Health Anxiety Survey [1, 29] and measures hypochondriasis on a continuum [3]. Higher scores indicate greater health anxiety.

Ethical Approval

Approval for this cross-sectional cohort study was obtained from our institutional review board.

Statistical Analysis

We performed descriptive statistical analyses; all continuous variables are reported as the mean ± standard deviation or median (interquartile range). Discrete variables are presented as percentages and numbers. To seek associations between independently assessed communication effectiveness and patient-rated clinician empathy, we performed bivariate analyses of factors associated with the Jefferson Scale of Patient Perceptions of Physician Empathy (Supplemental Table 1; http://links.lww.com/CORR/A977). The Mann-Whitney U tests and Kruskal-Wallis H tests were used for categorical variables, where appropriate. For continuous variables, we calculated Spearman rank-order correlations, accounting for age and questionnaire scores. Because we expected substantial ceiling effects in measures of empathy based on prior work [24], we also dichotomized the Jefferson Scale of Patient Perceptions of Physician Empathy (cutoff = 30, median split). A t-test and Mann-Whitney U tests were used for parametric and nonparametric variables respectively. For categorical variables, chi-square tests and Fisher exact tests were used, where appropriate. All variables close to reaching statistical significance in the bivariate analysis (p < 0.10) were moved to a multivariable analysis. In addition, we sought factors associated with Liverpool Communication Skills Assessment Scale scores in bivariate and multivariable analyses (Supplemental Table 2; http://links.lww.com/CORR/A978). We used a multivariable logistic regression for dichotomous, dependent variables and linear regression for normally distributed variables. All providers with five or fewer patients were pooled for analysis. All p values below 0.05 were considered statistically significant. Two-way random-effects ICC models were constructed, reporting ICCs for the individual items on the Liverpool Communication Skills Assessment Scale and the total score.

An a priori sample size calculation indicated that 118 participants would provide 80% statistical power to detect a 0.5-point difference in the Liverpool Communication Skills Assessment Scale score, based on a linear multivariable regression analysis with 10 independent variables and alpha set at 0.05.

Results

Association Between Clinician Communication Effectiveness and Patient-rated Clinician Empathy

Accounting for potential confounders, including specific clinicians, marital status, and work status in the multivariable analysis, higher independently assessed communication effectiveness was associated with slightly higher ratings of patient-perceived empathy (OR 1.1 [95% CI 1.0 to 1.3]; p = 0.02), while being single was associated with lower ratings of clinician empathy (OR 0.40 [95% CI 0.16 to 0.99]; p = 0.05) (Table 2).

Table 2.

Multivariable logistic regression analysis of factors associated with achieving a JSPPPE score of 30 or higher

Variable	Odds ratio (95% CI)	p value
LCSAS	1.1 (1.0 to 1.3)	0.02
Marital status
Married or partner	Reference value
Single	0.40 (0.16 to 0.99)	0.05
Divorced or widowed	0.73 (0.20 to 2.7)	0.64
Work status
Employed	Reference value
Unemployed	0.57 (0.16 to 2.0)	0.39
Disabled	0.24 (0.056 to 1.0)	0.06
Retired or other	0.59 (0.18 to 2.0)	0.40

JSPPPE = Jefferson Scale of Patient Perceptions of Physician Empathy. The JSPPPE is scored on a scale from 5 to 35, with higher scores representing higher patient-rated provider empathy. LCSAS = Liverpool Communication Skills Assessment Scale. The LCSAS is scored on a scale from 0 to 36, with higher scores representing higher communication effectiveness. The odds ratio in the table represents the increase in likelihood that a visit is rated with a JSPPPE of 30 (50^th percentile visit) or higher per point increase in the LCSAS.

Factors Associated With Independently Assessed Communication Effectiveness

Accounting for potentially confounding variables such as woman gender, higher level of education, upper extremity conditions, and individual clinicians in the multivariable analysis, higher independently rated communication effectiveness was associated with being a woman; in addition, two of the providers (clinicians 1 and 4) were rated more highly than the others were (Table 3).

Table 3.

Multivariable linear regression analysis of factors associated with greater communication effectiveness (measured with the LCSAS)

Variable	Regression coefficient (95% CI)	p value	Partial r2
Gender
Women	Reference value
Men	-1.1 (-2.2 to -0.047)	0.04	0.043
Education level
High school or lower	Reference value
2-year college	1.0 (-0.69 to 2.7)	0.24	0.014
4-year college	0.71 (-0.75 to 2.2)	0.34	0.0096
Postgraduate degree	-0.59 (-2.1 to 0.97)	0.46	0.0058
Extremity
Lower	Reference value
Upper	0.92 (-0.74 to 2.6)	0.27	0.013
Provider
1	Reference value
2	-5.8 (-8.9 to -2.6)	< 0.001	0.12
3	-4.1 (-5.6 to -2.5)	< 0.001	0.22
4	-0.60 (-3.2 to 2.0)	0.65	0.002
5	-3.7 (-5.2 to -2.2)	< 0.001	0.20
Other	-5.2 (-7.1 to -3.4)	< 0.001	0.25

LCSAS = Liverpool Communication Skills Assessment Scale. The LCSAS is scored on a scale from 0 to 36, with higher scores representing higher communication effectiveness. Partial r² = the proportion of variation explained by each variable and the error variance. Adjusted r² = 0.39, which is the percentage of variance explained by the full model, adjusted for the number of explanatory variables.

Discussion

An improved understanding of the association between independently assessed communication effectiveness and patient-rated clinician empathy could help inform quality improvement initiatives. We performed a cross-sectional study of patients with upper and lower extremity illnesses at a single academic musculoskeletal specialty office and found that patient-rated empathy does not correlate with independently assessed communication effectiveness. Furthermore, the most highly rated clinicians in terms of communication effectiveness did not have higher patient ratings of clinician empathy. These results indicate that there is more to learn about patient experience, or that we need better measures to effectively study these relationships.

Limitations

This study has several limitations. First, the communication experts could not be blinded to the clinician, which could have introduced some bias. However, the interobserver agreement for the Liverpool Communication Skills Assessment Scale score was excellent, and the ratings were kept confidential from the clinician to limit the influence of social desirability bias [5, 34]. Second, there were substantial ceiling effects in the measure for patient-rated empathy [24], which limited our ability to detect weaker associations. We expect that if the association between independently assessed communication strategies and patient-rated clinician empathy was strong, we would have detected this, regardless of the notable ceiling effects of the empathy scale. Development of new patient-reported experience measures with limited ceiling effects is needed to better gauge the effect of improvement interventions on patient satisfaction, ratings of empathy, and trust in the clinician [32]. Third, we did not account for clinician personal factors. In our current study, we included and tracked clinicians with different levels of experience (resident physicians, nonphysician specialty clinicians, and attending physicians) to increase variation in communication effectiveness, but future studies could further investigate clinician factors. Fourth, we did not document the number of patients who refused participation in our study, and we did not collect baseline data of these patients, but few patients declined. Although this sample may not represent all patients visiting an orthopaedic surgeon, based on the variability in the sample, we expect the limited association between patient-rated empathy and independently assessed communication effectiveness is generalizable outside this study population. Finally, this study aimed to evaluate factors associated with divergence between patient-rated and independently assessed communication effectiveness as a tertiary (unplanned) analysis, but there were too few patients who had an evident discrepancy to perform a statistical analysis, and there were no apparent discernable patterns in the data. Future studies could be powered to seek factors associated with divergence in these ratings.

Association Between Clinician Communication Effectiveness and Patient-rated Clinician Empathy

The finding of no association between independently assessed clinician communication effectiveness and patient-rated clinician empathy indicates that effective communication strategies alone cannot provide patients with a good care experience, or they may reflect the inadequacy or inappropriateness of current measures. We interpret the observation of notable differences in communication effectiveness between clinicians but no difference in patient-rated empathy by clinician similarly. Our findings are in contrast with a randomized trial that offered residents and fellows from various subspecialties three 60-minute training modules on communication strategies for conveying empathy and demonstrated a relationship between training and better patient-rated empathy [28]. An alternative explanation for the lack of association between independently assessed communication effectiveness and patient-rated clinician empathy is that current measurements either do not address the factors associated with patient experience or they do not adequately measure variation in these factors. We have noted that current patient experience measures seem to address a single underlying factor (“relationship”), and they all have notable ceiling effects (limited variation) [2, 12, 13, 26]. We may benefit from better ways for independent observers to discern the key elements of a patient-clinician interaction. For instance, current measures may not adequately assess nonverbal communication such as facial expressions [19, 33], body language [35], and the linguistic and paralinguistic vocal cues in speech [17]. A study that documented correspondence between computer-rated emotional content of clinician facial expressions and patient thoughts and feelings regarding symptoms suggests computer-based tools can help identify aspects of patient-clinician interaction to improve patient experience [33]. Another study documented that natural language processing of verbatim text comments from patients about their care experience resulted in a more normal distribution of patient experience scores and no ceiling effect [27]. We regard such tools as a promising line of investigation into tools for identifying clinician behaviors associated with better patient experience.

Factors Associated With Independently Assessed Communication Effectiveness

The finding that individual providers accounted for the largest amount of variation signals opportunities for clinicians to learn and practice effective communication strategies. It also suggests the Liverpool Communication Skills Assessment Scale can identify clinician-level variation in communication strategies. Better experience measures and crafted independent ratings of communication effectiveness might help discern the key factors clinicians can learn and practice in order to improve the patient experience. One pretest and posttest study of residents at an academic center in the Netherlands observed a slight improvement in patient-rated clinician empathy (measured using the Consultation and Relational Empathy measure) after a three-day communication training program [23], yet there was no improvement in independently assessed communication effectiveness (measured by a trained rater based on videotaped consultations using the Maastricht History-taking and Advice Scoring tool). This supports our finding that individual clinicians can improve regarding communication effectiveness and corroborates our prior finding that communication effectiveness and perceived empathy are not correlated as strongly as one might assume.

Conclusion

The observation that ratings of communication effectiveness by trained communication scholars have little or no association with patient-rated clinician empathy suggests that either effective communication is insufficient for good patient experience or that existing measures are inadequate or inappropriate. To improve the patient experience, we need measures that correspond with aspects of communication that can be rated, taught, practiced, and improved. This is a priority because concordant feedback inspires and reinforces a growth mindset, but it can be disheartening for clinicians to receive feedback that is discordant with their skills and behaviors. Given the evidence that a variety of aspects of patient experience including trust, perceived empathy, perceived communication effectiveness, and satisfaction represent a single underlying factor (conceptualized as a relationship) [20, 25, 32], researchers in future studies could collaborate with communication scholars to help develop ratings of video or audio recordings of visits that assess key aspects of relationship-building. An independent rating crafted to identify areas for improvement in skills that are known to be effective based on communication scholarship might prove more useful for training and be more strongly associated with patient experience. More work on creating measures with greater variation and fewer ceiling effects (for instance, using natural language processing of verbatim comments [28]) could lead to better experience measures that are more sensitive to improvements in clinician communication effectiveness.