Comment on: Empathy and burnout: a study on residents from a Singapore institution

Dear Sir,

We read the article ‘Empathy and burnout: a study on residents from a Singapore institution’ by Lee et al(1) with great interest and would like to commend the authors for their conciseness and reasonable clarity. However, we would like to highlight the following points about the article that we believe do not adequately justify the conclusions reached by the authors.

Firstly, we believe that the interpretation of data and the conclusions lack scientific rigour. The authors failed to standardise their diagnostic criteria of burnout to the United States (US) studies that they compared their results with. The authors adopted the parameters of emotional exhaustion (EE) score ≥ 27, or depersonalisation (DP) score ≥ 11, or personal achievement (PA) score ≤ 33 as a criteria of burnout, but the US studies they compared their results to adopted high EE or high DP scores (EE ≥ 27 or DP ≥ 11) only. A low PA score alone is never used to define burnout. This raises doubt about whether the authors’ finding of a burnout rate of 80.7% in Singaporean residents, and that this is significantly higher than rates in the US, is actually exaggerated by false positives from respondents with low PA scores alone, thus leading to a misrepresentation of the truth. Lee et al did not justify why a different criteria of burnout was adopted and hence, a fair and direct comparison between the two cohorts could not be made due to this double standard. If the authors are able to rectify this problem, they could then strengthen their results by comparing them to a national US study involving 1,701 US residents(2) rather than comparing to three small studies. Another flaw is their comparison of mean burnout scores in their own study with median scores in the US study. It is inappropriate to compare means with medians from two different populations. In a unimodal distribution, one may infer a positive or negative skew if the mean and median from the same population are significantly dissimilar.

We believe that no meaningful or robust conclusions can be made from the data provided by the authors. The only direct comparison that was permitted statistically with Lee et al’s provided dataset was not performed in the article. We, therefore, tested the mean Jefferson Scale of Physician Empathy (JSPE) scores from the study against the US cohort cited by the authors(3) for a statistically significant difference using a two-tailed t-test (Singapore 104.9 ± 13.2, n = 449 vs. US 120 ± 15.6, n = 704). This revealed p < 0.01. However, in the context of a self-assessment tool that has not been extensively tested in Singapore residents, the lack of a control group and various other issues with the study, its significance is open to debate.

We recommend that the authors standardise their criteria of burnout to the US studies that they tried to compare their results to. They could then compare the proportions of individuals defined as having burnout in both populations using a chi-square analysis, rather than incorrectly comparing the mean ± standard deviation of Maslach Burnout Inventory (MBI) scores with medians from separate populations.

Secondly, the literature search and presentation of data are suboptimal. The authors stated that “To our knowledge, empathy and burnout studies have not been performed in Singapore”. However, a simple PubMed search using the terms ‘burnout’ and ‘Singapore’ demonstrated otherwise.(4-6) One of the co-authors was also first author of a study on empathy in a cohort of medical students in Singapore.(7) This statement discredits other local investigators. If the authors meant that studies investigating the association between lack of empathy and burnout have not been performed in Singapore, and if that was their intention, this was not described in the aims of the study, nor was it rigorously attempted. See et al conducted a better study that reported a burnout rate of 31.1% in attending physicians and 71.8% in trainees in Singapore, although it was submitted as a letter to the editor and seemingly unfunded.(5) These results were similar to some rates reported in the US. However, the study was limited by the use of a different measurement tool for burnout (Copenhagen Burnout Inventory), a smaller sample size (64 trainees, 45 attendees) and the lack of a control group.

Lee et al’s study used linear regression to study the association of empathy scores and PA, EE and DP. However, the respective r² scores of 0.200, 0.035 and 0.103 for PA, EE and DP suggest that the scattering of data was sizeable and therefore, the inclusion of scatter plots is needed for readers to discern if a meaningful interpretation of the linear regression analyses has been made. An argument could be made that correlation analysis would be more suitable than linear regression in this case; however, the relationship between the test variables, if there is any, may also be non-linear, and hence the scatter plot is required. Further, the presentation of data in Table I is confusing, displaying response rates by cohort in one segment of the table but comparing response rates against the proportion of total responses in other segments of the table (e.g. year of residency vs. ethnicity). Survey results should have been standardised to show the proportion of total responses. It is also well recognised that burnout is more prevalent in certain specialties (e.g. emergency medicine, obstetrics and gynaecology) so this study would have been better served if data was presented by specialty (as did the better US studies)(2,8) instead of grouping 34 different programmes into ‘medical’ and ‘surgical’. Local specialties could have gained better insight if the authors had compared burnout rates and the lack of empathy to their US counterparts.

Thirdly, some of the statements made by the authors lack validation. The authors stated, “We have previously published findings on the validity and reliability of the JSPE in Singapore”. The JSPE for students (JSPE-S) and JSPE for physicians and health professionals (JSPE-HP) are different self-assessment tools intended for different categories of medical professionals. In a study by Sng et al, the authors reported that the JSPE-S model was not a perfect fit for local medical students and proposed that the social construct of empathy might be viewed differently in the East and the West.(7) They suggested this could affect the JSPE results in an East Asian population, but it is unclear whether Lee et al made any effort to address their previous concern prior to using the JSPE-HP in this study. This is also important because both cohorts might be very different, with international medical graduates making up 25%–33% of residents in the local workforce (unpublished demographic data provided by Dr Yeo Khung Keong at the Academic Medicine Education Institution lecture on mentoring).

The use of unsubstantiated and bold statements is also a cause for concern. For example, the authors commented that “Medical students in Singapore have been shown to be less empathetic than their counterparts from the US, although they were relatively more empathetic compared to counterparts from East Asia”. Unfortunately, no statistical evidence was provided to demonstrate that JSPE-S scores in Singapore medical students differ significantly from the US or East Asian medical students. This statement should not have been made without robust evidence.

Lastly, the data collection process is questionable. Although the authors stated that participation in their study was voluntary, two weekly email reminders were sent to non-responders. This meant that during the data collection period from 1 January 2015 to 31 May 2015, a non-responding or non-consenting trainee would have been sent 42 emails (or ten emails if the authors meant ‘fortnightly’). This could have pressured some trainees into completing the survey and thus, the meaningfulness of some responses is questionable. In addition, the MBI inventory manual recommends that participants are given the sense and assurance of anonymity during the survey for responses to be honest and meaningful.(9) However, participants in the study were asked for personal and potentially negatively discriminatory details that were irrelevant to the stated study objectives (e.g. name, alcohol consumption, smoking habits, marital status, etc). This may have contributed to non-response bias. The authors also did not state their exclusion criteria and it is unclear if any of the authors themselves participated in the survey as a respondent during their residency in 2015. If so, there would be further ethical considerations and implications.

In summary, we commend Lee et al for their efforts in trying to understand burnout and help address an important issue in the Singapore healthcare system. However, given the limitations of the study, we would advise all readers to interpret the article with caution until more data is made available by its authors.

Yours sincerely,