Comparing a Self-Administered Measure of Empathy with Observed Behavior Among Medical Students

Abstract

PURPOSE

Studies show that measures of physician and medical students’ empathy decline with clinical training. Presently, there are limited data relating self-reported measures to observed behavior. This study explores a self-reported measure and observed empathy in medical students.

METHOD

Students in the Class of 2009, at a university-based medical school, were surveyed at the end of their 2nd and 3rd year. Students completed the Jefferson Scale of Physician Empathy-Student Version (JSPE-S), a self-administered scale, and were evaluated for demonstrated empathic behavior during Objective Structured Clinical Examinations (OSCEs).

RESULTS

97.6% and 98.1% of eligible students participated in their 2nd and 3rd year, respectively. The overall correlation between the JSPE-S and OSCE empathy scores was 0.22, p < 0.0001. Students had higher self-reported JSPE-S scores in their 2nd year compared to their 3rd year (118.63 vs. 116.08, p < 0.0001), but had lower observed empathy scores (3.96 vs. 4.15, p < 0.0001).

CONCLUSIONS

Empathy measured by a self-administered scale decreased, whereas observed empathy increased among medical students with more medical training.

Electronic supplementary material

The online version of this article (doi:10.1007/s11606-009-1193-4) contains supplementary material, which is available to authorized users.

Empathy in the physician-patient relationship is the physician’s ability to recognize a patient’s perspectives and experiences, and convey such an understanding back to the patient.^1,2 This understanding allows patients to feel respected and validated,³ promotes patient and physician satisfaction, and may improve patient outcomes.^1,4–7 Empathy is one of the Association of American Medical Colleges' (AAMC) goals for the development and education of altruistic and compassionate physicians.⁸Studies of medical students and residents suggest that empathy decreases with increased medical training.^9–11 These studies used self-administered measures of empathy with an uncertain correlation with actual empathic behavior. One study by Hojat et al. showed a modest positive correlation between their self-administered measure of empathy at the beginning of the 3rd year of medical school and program directors’ assessment of these students’ empathy during the end of internship 3 years later.¹² While it is known that physician self-assessment does not compare favorably to observed measures of competence,¹³ psychometrically sound scales are thought to do better if they are validated against observable behaviors.

This study explores the relationship between a self-administered measure of empathy, the Jefferson Scale of Physician Empathy-Student Version (JSPE-S), and observed empathy, as evaluated by standardized patients during end of year Objective Structured Clinical Examinations (OSCEs).

METHODS

Study Participants

All students in the Class of 2009 at Boston University School of Medicine (BUSM) were eligible to participate. The BUSM curriculum is a traditional 4-year medical school with 2 years of preclinical study, with limited patient contact in the form of weekly physician shadowing for 10 weeks and a weekly patient interviewing and examination course for 6 weeks, followed by 2 years of clinical clerkships and electives.

Study Design

Class of 2009 students in their 2nd and 3rd year of medical school were asked to participate in a voluntary online survey measuring “student attitudes toward medicine” during March-April 2007 and March-April 2008. The survey was administered during their end of year Objective Structured Clinical Examinations (OSCEs).

The Jefferson Scale of Physician Empathy-Student Version (JSPE-S), is a self-administered 20-item instrument measuring components of empathy among health professionals in patient-care situations.² Respondents indicate their level of agreement on a seven-point Likert scale. Scores range from 20 to 140, with higher values indicating a higher degree of empathy. Participants also specified their gender, age, anticipated financial debt and likelihood of choosing various specialties.

Career specialty intentions were categorized into two groups, “People-oriented” specialties and “Technology-oriented” specialties.¹¹ Categorizations were based on categories determined in prior studies.^2,11

Toward the end of the 2nd and 3rd year of medical school, students are required to take an Objective Structured Clinical Examinations (OSCEs) where they are assessed on clinical skills, including their doctor-patient interactions, by standardized patients. Second-year students complete three cases—two history taking and physical exam cases and one substance abuse case—while 3rd-year students complete six cases—specialty-oriented cases in Medicine, Pediatrics, Family Medicine, Psychiatry, Ob/Gyn, and Surgery (see online Appendix A for descriptions of student cases). Each student is rated on a five-point Likert scale for empathy for each case (see online Appendix B for descriptions of OSCE empathy question). All standardized patients were trained at the University of Massachusetts Medical School and were familiar with the examination material and empathy question, and many had several years experience in evaluating Boston University School of Medicine students.

Descriptive statistics and analyses of variance (ANOVA) were used to compare the JSPE-S scores among the different classes and categorized groups, while controlling for the effects of gender, age, anticipated financial indebtedness, and career preference. Post-hoc ANOVA pairwise comparisons were made using Tukey’s HSD test. Correlations were made between JSPE-S and observed empathy scores. All computations were done with SAS statistical software version 9.1. This study was approved by the Boston University Medical Center Institutional Review Board.

RESULTS

The Class of 2009 had 167 students eligible in the 2nd year and 162 students eligible in the 3rd year to participate in the study; 97.6% and 98.1% of eligible students participated in their 2nd and 3rd years, respectively (Table 1).

Table 1.

Demographics and Characteristics of the Students in the Medical School Class of 2009

	Second year	Third year
Number of students eligible to participate	167	162
Number of completed surveys	163	159
Percentage of surveyed students who were female	54.0% (N = 88)	56.6% (N = 90)
Percentage of surveyed students preferring “people-oriented” specialties	51.0% (N = 78)	58.3% (N = 88)
Percentage of eligible students surveyed	97.6%	98.1%

The primary multivariate analysis of variance considered and adjusted for five factors: class, gender, anticipated financial debt, career preference, and age. The overall correlation between JSPE-S and OSCE observed empathy scores was 0.22 (p < 0.0001).

Second-year students had higher JSPE-S scores compared to 3rd-year students (118.63 vs. 116.08, p < 0.0001), but the average observed empathy score for 2nd-year students was lower than the observed empathy score for 3rd-year students (3.96 vs 4.15, p < 0.0001) (Table 2).

Table 2.

Results by OSCE^a

	Second-year assessment (N = 163)	Third-year assessment (N = 159)	P-value
JSPE-S averageb	118.63	116.08	<0.0001
OSCE empathy average per caseb	3.96	4.15	<0.0001

^aAdjusted for gender, age, anticipated financial indebtedness, and career preference

^bPossible JSPE-S scores range from 20 to 140

^cPossible OSCE scores range from 1 to 5

DISCUSSION

Self-administered tools are the most common method used to assess various educational interventions aimed at improving student empathy.^14–20 Nevertheless, the validity of such self-assessments is unknown.¹³ Our study found a trend towards a decline in measured empathy with increased clinical training with a self-administered instrument, but an improvement in observed empathy among these more clinically experienced students. Our finding of empathy decline by the JSPE-S is similar to prior studies using this instrument to assess change from the pre-clinical and clinical years.^10,11 So why the discrepancy with simulated patient ratings?

We can suggest several explanations. The differences noted between the 2nd- and 3rd-year medical school classes on their self-assessment of empathy could be attributed to different training experiences and how these experiences shape student perception of illness, compassion, and empathy. The JSPE-S is designed to assess the empathy of health-care providers in patient-provider situations. In the first 2 years of medical school, students interact with patients mostly when shadowing practicing physicians and participating in clinical skills training courses. Third-year students continually interact with patients and may often share the experiences of patients and families coping with serious and sometimes fatal illness. Therefore, it is possible that with their limited clinical exposure 2nd-year students interpret the subjective anchors of JSPE-S questions differently from 3rd-year students who have a broader and more intense clinical experience. The student, in effect, has recalibrated his emotional understanding of illness through clinical experience and hence altered his score on paper without having impacted his nature.

An alternative explanation could be that student acculturation to critical illness and a true emotional recalibration within the student. Such a change could be protective in the professional development of a physician. Medical illness and patient suffering are real, intense, and frequently sad. It may be necessary for physicians to undergo a professional acculturation that is being captured by the self-assessment tool and that, to date, has been interpreted as a measured decline in empathy associated with early clinical training. Students in this case do become emotionally hardened and feel less empathy. The working hypothesis has been that clinical training lacks an element of humanity and that a “hidden curriculum” exposed by jaded, experienced practitioners undermines the idealism, humanism, and empathy young clinicians bring with them to clinical medicine.^11,14 This last argument has been the motivation for curricular innovations designed to enhance and support the maintenance of empathy. It would be difficult to discern the difference between these two hypotheses, but each would predict potentially lower scores on self-reported empathy as one traverses the early stages of clinical training.

So why did we observe more empathic behavior in more advanced students? The rating of empathy during an OSCE broadly includes verbal and non-verbal communication, as well as physical behavior, but cannot assess the internal emotion or motivation of the student. Third-year students had more opportunities to practice, observe, and get feedback on their empathic behaviors, independent of their internal emotional connection. From the perspective of clinical proficiency, one would anticipate that more experienced students would better demonstrate clinical behaviors, including empathic behaviors.

Thus, the apparent independence of the self-assessment measure and observer ratings suggests that the use of the JSPE as self-assessment tool may not sufficiently predict empathic behavior. Further confirmation of our findings is needed as this has implications for curriculum evaluation since a self-administered tool is easier and cheaper to use. These findings raise more challenging questions for educators: What are the important aspects of physician empathy to measure? Is assessment of empathic behavior adequate even if internal emotions are discrepant? Is the correct attitude acceptable if we cannot relate it to competent behavior?

There are several limitations of our study. The small number of OSCE cases, especially in the 2nd year, raises issues of score reliability and, though OSCE score differences were present, it is unclear how clinically significant a 0.2 point difference is at the physician-patient level. Since we had only a single empathy question for each case, we are unable to determine which observable behaviors were driving the rating. Our study is limited to one medical school, but we feel that our results are applicable to all schools with a similar traditional structure. Future studies should examine the subtleties of the physician-patient interaction by discriminating those elements that comprise empathic behavior, such as tone of voice, empathic language, and non-verbal communication.

The patients’ need for an empathic physician will always be essential. Efforts to improve the empathic behaviors of trainees are important. More work is required so that curricular enhancements designed achieve these goals can be properly evaluated.