Abstract
Background
Study aimed to assess the burnout prevalence and level of empathic attitude in family medicine doctors (FMDs) and its associations with demographic factors, working conditions and physician health, using the Jefferson Scale of Empathy – Health Professional version (JSE-HP).
Methods
Slovenian FMDs (n=316, response rate 56%) completed an online socio-demographic questionnaire, with questions on working conditions, physician health, and the Slovenian versions of the Maslach Burnout Inventory (MBI) and the JSE-HP. Univariate and multivariate analyses were used, applying linear regression to calculate associations between demographic variables, factors of empathy and burnout dimensions, P<0.05 was set as a limit of statistical significance.
Results
Of the 316 participants, aged 40±10.2 years, 57 (18%) were men. The FMDs achieved mean scores on the JSE-HP (JSEtot of 112.8±10.2 and on the MBI 27.8±11.6 for EE, 10.8±5.5 for D and 33.5±6.0 for PA. High burnout was reported in one dimension by 24.8% of participants, in two by 17.2%, and by 6% in all three dimensions. Multivariate analysis revealed a higher EE and D and lower PA in specialists as opposed to trainees. Higher EE was also identified in older physicians having longer work experience, working in a rural setting, dealing with more than 40 patients/day and having a chronic illness. The latter was also associated with higher JSEtot. JSEtot was negatively associated with D, while PA was positively associated with JSEtot and Perspective Taking.
Conclusion
The incidence of burnout warns both physicians and decision-makers against too heavy workload, especially in older professionals.
Keywords: empathy, professional burnout, emotional exhaustion, depersonalisation, physicians, family medicine, workload, primary care
Izvleček
Izhodišča
V družinski medicini empatična naravnanost pri komunikaciji z bolnikom pripomore k učinkovitosti na več ravneh: poveča se zadovoljstvo bolnikov in izboljša njihovo sodelovanje ter zmanjša se izgorelost zdravnikov in izboljša njihovo počutje. Namen raziskave je bil opredeliti stopnjo izgorelosti in empatične naravnanosti, odkriti socialnodemografske značilnosti in značilnosti dela, povezane z obema entitetama, ter prvič uporabiti in preveriti prevedeno in prirejeno lestvico za samooceno empatične naravnanosti pri zdravnikih.
Metode
V presečni raziskavi so slovenski specializanti in specialisti družinske medicine (n = 316, 56-odstotni odziv) odgovarjali na spletni vprašalnik, ki je spraševal po socialno-demografskih značilnostih, delovnih pogojih in zdravstvenem stanju, dodani sta bili lestvici za samooceno izgorelosti MBI (angl. Maslach Burnout Inventory) in empatične naravnanosti JSE-HP (angl. Jefferson Scale of Empathy – Health Professional version), za kateri so bile v procesu preverjanja s konfirmatorno faktorsko analizo določene psihometrične lastnosti. Univariatna statistična analiza je zajemala t-test za neodvisne vzorce, enofaktorsko analizo variance in Pearsonov korelacijski koeficient. Multivariatna analiza povezanosti demografskih spremenljivk in faktorjev empatije z dimenzijami izgorelosti (čustvena izčrpanost (ČI), razosebljenje (R), osebna izpolnitev (OI)) je bila izračunana z linearno regresijo. Vrednost p < 0,05 je določala mejo statistične pomembnosti.
Rezultati
Vzorec je zajemal 316 sodelujočih, od tega 57 (18 %) moških in 259 (82 %) žensk, povprečna starost je bila 40 ±10,2 let. Zdravniki so dosegli povprečno skupno vrednost na JSE-HP (JSEtot) 112,8 ±10,2. Povprečne vrednosti na MBI so bile 27,8 ±11,6 za ČI, 10,8 ± 5,5 za R in 33,5 ±6,0 za OI. Največ zdravnikov je navajalo visoko izgorelost v eni dimenziji (24,8 %), 17,2 % v dveh in 6 % v vseh treh. Specialisti so v primerjavi s specializanti kazali višjo ČI (β = 0,28; p < 0,001) in R (β = 0,17; p = 0,026) ter nižjo OI (β = -0,20; p = 0,004). Višjo ČI so navajali starejši zdravniki (r = 0,213; p < 0,001), zdravniki z daljšo delovno dobo (r = 0,185; p = 0,001), zdravniki v ruralnih ambulantah (β = 0,12; p = 0,043), ki opravijo 40 obravnav na dan in več (p = 0,014), ter tisti s kronično boleznijo (p = 0,002). Slednji so dosegali višjo JSEtot (p = 0,010). Pokazala se je negativna povezava JSEtot z R (r = –0,224; p < 0,001) in pozitivna z OI (r = 0,372; p < 0,001). Ta se je v dodatni multviariatni analizi pokazala pozitivno povezana s kognitivno komponento empatije, z zmožnostjo upoštevanja zornega kota drugega (β = 0,35; p < 0,001).
Zaključek
Lestvica za samooceno empatije se je pokazala kot učinkovit instrument. Zlasti pri starejših in bolj obremenjenih zdravnikih je povezava med empatično naravnanostjo in izgorelostjo pomembno opozorilo tako za zdravnike kot tudi za odločevalce v zdravstveni politiki. Rezultati kažejo tudi na pomen stalne krepitve veščin v medosebnih odnosih v obliki suportivnih in interaktivnih izobraževanj.
Ključne besede: empatična naravnanost, upoštevanje zornega kota drugega, izgorelost, čustvena izčrpanost, razosebljenost, osebna izpolnitev, zdravniki, družinska medicina
1. Introduction
Empathy has been recognised as an important and powerful part of communication in general practice, strengthening the physician-patient relationship (1, 2). Burnout in family medicine doctors (FMDs) at the front line of health care, disrupts this relationship (3, 4, 5, 6). An empathic attitude is described as the capacity to understand what another person is experiencing from within their frame of reference (7). Physicians with empathic attitudes experience greater patient satisfaction and better patient compliance and adherence to treatment (8, 9). In addition, they tend to face fewer medical errors (10), have improved health outcomes (11), report fewer symptoms of burnout and have better well-being (12). The concept of empathy consists of cognitive and affective components (13), with a known moderate correlation between the concepts of sharing understanding and sharing emotion in patient-care (14).
Almost 65% of European FMDs exhibit signs of burnout (3) with various and non-specific symptoms (6). As the response to chronic emotional and interpersonal stressors at work, burnout leads to reduced job performance (15); the physicians’ behaviour can have a detrimental effect on the health of patients and lead to more malpractice suits (16) and patient dissatisfaction (17). In physicians, the heavy workload and the lack of financial and organisational resources are important risk factors for burnout (15, 18), with an intense empathic attitude leading to emotional exhaustion and causing burnout syndrome (19). Some studies determined depersonalisation to be the main reason for a decrease in empathic attitude (20).
In Slovenia, burnout has only been evaluated in family medicine trainees, who scored highly (71%) in at least one burnout dimension (21). This is the first study in Slovenian family medicine focusing on the relationship between empathy and burnout and aiming to assess the extent of burnout and the level of empathic attitude in FMDs, and also to explore their associations with sociodemographic factors, working conditions and health. The Jefferson Scale of Empathy (JSE) for physicians was used as a validated self-assessment tool for the first time in Slovenia (see Additional File).
2. Methods
2.1. Participants and Procedure
This was a cross-sectional survey of Slovenian FMDs; 565 out of 1139 FMDs in Slovenia (22) were invited twice by e-mail to complete an online survey. The invitation was sent through the e-mail distribution lists of the Slovenian syndicate of FMDs (396 specialists’ e-mail addresses) and family medicine trainees (169 trainees’ e-mail addresses).
The questionnaire, which had been validated previously, comprised socio-demographic and other questions concerning working conditions, health and well-being, the Slovenian version of the Maslach Burnout Inventory (MBI) (4), and the Jefferson Scale of Empathy – Health Professional version (JSE-HP) (23, 24). The data were collected from April to June 2016.
The response rate was 56%; of 316 respondents, 123 (38.9%) were family medicine trainees and 193 (61.1%) were specialists (who had finished a four-year period of specialised training), aged 40±10.2 years.
2.2. Instruments
2.2.1. Socio-Demographic Characteristics Questionnaire
The participants answered demographic questions assessing gender (male/female), age (years), time working in family medicine (years), marital status (single/married/in a relationship/widowed), and children (yes/no). Further questions regarded working conditions and working environment (urban/rural) and workload (the number of patients per day (<40/40-60/>60)), emergency care duty during the regular workday (yes/no), the number of nightshifts per month (0/1-3/≥4)). At the end, there were some questions that concerned the self-reported health of the physician (the number of sick leave days per year (0/1-5/≥6), having a chronic illness (yes/no)) and their self-assessment of their general health, mood and emotional state on a five-point Likert-type scale (1=poor, 5=excellent).
2.2.2. Self-Assessment of Empathic Attitude
The JSE-HP (JSE in further text) was developed by Hojat et al. to evaluate the empathic capacity of practitioners in health professions, including physicians (23, 24). It consists of 20 items, which use a 7-point Likert-type scale (1=strongly disagree, 7=strongly agree) to elicit responses, with a score range of 20-140 (24). Previous studies have suggested a three-factor structure, with the components being Perspective Taking, Standing in the Patient’s Shoes and Compassionate Care (24, 25). The first two subscales address the cognitive aspect of empathic behaviour/attitude (23). The validity and reliability of the JSE were evaluated (11, 23, 24); it has been translated into 53 languages and used in more than 80 countries worldwide (26). The only Slovenian study so far used the JSE in medical students (JSE-S) and confirmed the three-factor structure of the 18-item scale (13). For our study, the JSE was translated into Slovenian and the authorisation for its implementation was obtained. Cronbach’s α coefficients for JSE subscales were 0.865 for Perspective Taking, 0.722 for Standing in the Patient’s Shoes, 0.784 for Compassionate Care and 0.798 for the total of 20 items.
2.2.3. Self-Assessment of Burnout
The MBI is the gold standard for assessing burnout (4), using 22 items scored on a 7-point Likert-type scale (0=Never, 6=Every day). The MBI consists of 3 subscales: Emotional Exhaustion ((EE), 9 items, score range from 0 to 54); Depersonalisation ((D), 5 items, score range from 0 to 30); and Personal Accomplishment ((PA), 8 items, score range from 0 to 48). High scores on the EE and D subscales, combined with low scores on the PA subscale, indicate high levels of burnout. Cut-offs for high burnout were determined by the upper quartile for each dimension, and were for EE>37, for D>15 and <30 for PA. Slovenian version of MBI (27) was used and Cronbach’s α for EE subscale was 0.929, 0.765 for D and 0.801 for PA. The MBI has previously been tested and used several times on groups of healthcare staff, including FMDs (3, 6, 21).
2.2.4. Results of Confirmatory Factor Analysis for MBI and JSE
The confirmatory factor analysis was used to address psychometric properties and measurement invariance of MBI and JSE (28). We calculated the average variance extracted (AVE), maximum shared variance (MSV) and average shared squared variance (ASV) for the items loading on a construct. Conducted AVEs in Table 1 were all above 0.5, which represents a good conversion of MBI and JSE items (29). The resulting 3-factor structure for both MBI and JSE were confirmed as adequately fitting the data. For measurement invariance, the comparative fit index (CFI) and the root mean square error of approximation (RMSEA) were measured. Given that that CFI values were above 0.9 and RMSEA values were below 0.08, this was considered as acceptable (30, 31).
Table 1.
Psychometric properties | AVE | MSV | ASV | |
---|---|---|---|---|
MBI: Emotional Exhaustion | 0.572 | 0.028 | 0.014 | |
MBI: Depersonalisation | 0.516 | 0.031 | 0.016 | |
MBI: Personal Accomplishment | 0.552 | 0.030 | 0.015 | |
JSE: Perspective Taking | 0.584 | 0.027 | 0.014 | |
JSE: Standing in the Patient’s Shoes | 0.512 | 0.032 | 0.017 | |
JSE: Compassionate Care | 0.550 | 0.030 | 0.015 |
MBI scale | JSE scale | |||
---|---|---|---|---|
Measurement invariance | ||||
CFI | RMSEA | CFI | RMSEA | |
Gender: males (n=57) versus females (n=259) | ||||
Configural invariance | 0.932 | 0.026 | 0.944 | 0.020 |
Metric invariance | 0.929 | 0.026 | 0.930 | 0.026 |
Scalar invariance | 0.910 | 0.030 | 0.911 | 0.031 |
Age: 27-39 years (n=182) versus 40 years or above (n=134) | ||||
Configural invariance | 0.948 | 0.019 | 0.934 | 0.025 |
Metric invariance | 0.940 | 0.021 | 0.930 | 0.026 |
Scalar invariance | 0.920 | 0.027 | 0.918 | 0.027 |
AVE: average variance extracted, MSV: maximum shared variance,
ASV: average shared squared variance
CFI: comparative fit index, RMSEA: root mean square error of approximation
2.3. Data analysis
The sample was presented by the frequency and percentage distribution or by the values of the mean and standard deviations. Univariate statistical analysis included various statistical tests: the t-test for independent samples, the one-way analysis of variance, Pearson’s correlation coefficient and Spearman’s correlation coefficient. G*Power software (version 3.1.9) was used to calculate the achieved statistical power (32). Firstly, the confirmatory factor analysis was used to address psychometric properties and measurement invariance. In multivariate analysis, linear regression was used to calculate the associations between demographic variables, factors of empathy and burnout dimensions (EE, D, PA). The results of linear regression were presented by β coefficient, t value and p-value. All the analyses were performed using the SPSS version 22.0 for MS Windows (IBM Corp., Armonk, NY), with the significance set at p<0.05. Additionally, multiple comparisons bias was addressed by changing the level of statistical significance (p<0.001).
3. Results
Most of the responding FMDs were females (78.9% trainees and 83.9% specialists), working in this speciality for 11.2±10.4 years. The majority were married or in an intimate relationship (272 (86.1%)), most of them had children (218 (69%)). At the time of the survey, more FMDs worked in urban (199 (63%)) than in rural (117 (37%)) settings. Approximately half of the participants had to provide emergency care during regular worktime (152 (48.1%)) vs. 164 (51.9%)). A quarter did not work night shifts (75 (23.7%)), the same proportion reported night shifts at least 4 times per month, while one to three night shifts were reported by 166 (52.5%) respondents. Most of the participants examined at least from 40 to 60 patients per a working day (230 (72.8%)), 53 (18.7%) FMDs dealt with 60 or more patients per day and only a minority worked with less than 40 patients per day (33 (10.4%)).
Participants mainly reported no chronic illnesses (230 (72.8%)) and used either none (136 (43%)) or less than six sick leave days in a year (119 (37.7%)). Only 61 (19.3%) needed 6 or more sick leave days per year. Self-assessment of their general health was well (3.5±0.9), the same for mood and emotional state (3.2±0.9).
3.1. Empathic Attitude and Burnout Level in the FMDs
A mean total JSE score (JSEtot) was 112.8±10.2; mean scores of burnout dimensions were 27.8±11.6 for EE, 10.8±5.5 for D and 33.5±6.0 for PA (Table 2). The highest proportion of physicians reported high burnout in one dimension (24.8%), 17.2% reported it in two and 6% in all three dimensions. Altogether, almost half (48%) reported high burnout in at least one dimension, while a quarter (25.2%) scored high in EE and even more in D (25.8%) and PA (26.2%).
Table 2.
EE | D | PA | JSEtot | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Categorical variables | n | M | SD | t (p) | M | SD | t (p) | M | SD | t (p) | M | SD | t (p) |
Score: | 316 | 27.8 | 11.6 | 10.8 | 5.5 | 33.5 | 6.0 | 112.8 | 10.2 | ||||
Gender: | ES=0.3 | P=0.5 | 1.686 | ES=0.0 | P=0.1 | 0.259 | ES=0.3 | P=0.5 | 1.819 | ES=0.0 | P=0.1 | 0.279 | |
male | 57 | 25.4 | 12.4 | (0.093) | 11.0 | 5.3 | (0.796) | 34.8 | 6.2 | (0.070) | 112.4 | 9.9 | (0.780) |
female | 259 | 28.4 | 11.4 | 10.8 | 5.5 | 33.2 | 5.9 | 112.9 | 10.3 | ||||
Patients/day: | ES=0.5 | P=0.7 | 2.481 | ES=0.3 | P=0.4 | 1.641 | ES=0.1 | P=0.1 | 0.613 | ES=0.2 | P=0.2 | 0.782 | |
<40 | 33 | 23.1 | 11.0 | (0.014) | 9.4 | 5.0 | (0.102) | 34.1 | 5.3 | (0.540) | 114.2 | 8.4 | (0.435) |
≥40 | 283 | 28.4 | 11.6 | 11.0 | 5.5 | 33.4 | 6.1 | 112.7 | 10.5 | ||||
Night shifts/month: | ES=0.2 | P=0.3 | 1.341 | ES=0.2 | P=0.3 | 1.404 | ES=0.2 | P=0.3 | 1.274 | ES=0.0 | P=0.1 | 0.292 | |
<4 | 241 | 27.3 | 11.8 | (0.181) | 10.6 | 5.6 | (0.161) | 33.3 | 6.2 | (0.204) | 112.8 | 10.5 | (0.771) |
≥4 | 75 | 29.4 | 11.1 | 11.6 | 4.9 | 34.3 | 5.3 | 113.2 | 9.6 | ||||
Emergency care duty: | ES=0.2 | P=0.4 | 1.742 | ES=0.1 | P=0.1 | 0.785 | ES=0.2 | P=0.4 | 1.812 | ES=0.1 | P=0.1 | 1.098 | |
yes | 152 | 29.0 | 11.1 | (0.083) | 11.1 | 5.4 | (0.433) | 34.1 | 5.4 | (0.069) | 113.5 | 10.2 | (0.273) |
no | 164 | 26.7 | 12.0 | 10.6 | 5.6 | 32.9 | 6.4 | 112.2 | 10.3 | ||||
Chronic illness: | 3.146 | 0.100 | 0.141 | 2.582 | |||||||||
yes | 86 | 31.3 | 11.4 | (0.002) | 10.8 | 5.9 | (0.921) | 33.4 | 5.8 | (0.888) | 115.4 | 9.9 | (0.010) |
no | 230 | 26.6 | 11.5 | 10.9 | 5.3 | 33.5 | 6.1 | 112.0 | 10.2 | ||||
Continuous variables | r | p | r | p | r | p | r | p | |||||
Age (years): | 0.213 (P=0.9) | <0.001 | -0.036 (P=0.1) | 0.534 | -0.009 (P=0.1) | 0.872 | 0.080 (P=0.3) | 0.161 | |||||
Years in current speciality: | 0.185 (P=0.9) | 0.001 | -0.038 (P=0.1) | 0.511 | 0.005 (P=0.1) | 0.931 | 0.087 (P=0.3) | 0.130 | |||||
Sick leave days/year: * | 0.081 (P=0.3) | 0.159 | 0.013 P=0.1) | 0.820 | -0.054 (P=0.2) | 0.348 | 0.037 (P=0.1) | 0.525 |
M: mean value, SD: standard deviation, t: Student’s t-test, r: Pearson’s correlation coefficient, df: degrees of freedom=314, ES: effect size (Cohen’s d), P: achieved power
EE – Emotional Exhaustion, D – Depersonalisation, PA: Personal accomplishment, JSEtot– total JSE score
Spearman’s correlation coefficient was calculated for ordinal scale of sick leave
3.2. Correlation between Socio-Demographic Factors, Working Conditions and JSE and MBI Scores
There were no statistically significant differences in the EE, D and PA scores according to gender, while older physicians and those with more work experience turned out to be more emotionally exhausted; however, the correlation was weak (Table 2). In regard to working conditions (patients per day, night shifts per month, emergency care duty), physicians who examined more than 40 patients per day had a higher EE. More characteristics are presented in Table 2.
Multivariate analysis revealed a higher EE, higher D and lower PA in specialists, compared to trainees. Physicians working in a rural environment were more emotionally exhausted than those in an urban area. A lower PA was associated with female gender, while physicians with children reported a higher PA, yet no significant correlations between marital status and burnout dimensions were identified. See Table 3.
Table 3.
EE | D | PA | |||||||
---|---|---|---|---|---|---|---|---|---|
Categorical variables | B | t | p | B | t | p | B | t | p |
Gender (female/male) | 0.09 | 1.62 | 0.106 | -0.03 | -0.55 | 0.583 | -0.12 | -2.16 | 0.032 |
Working experience (specialist/trainee) | 0.28 | 3.94 | <0.001 | 0.17 | 2.24 | 0.026 | -0.20 | -2.92 | 0.004 |
Marital status (in a relationship/single) | -0.03 | -0.55 | 0.582 | -0.09 | -1.51 | 0.133 | -0.01 | -0.12 | 0.907 |
Children (yes/no) | 0.05 | 0.75 | 0.454 | -0.03 | -0.37 | 0.711 | 0.17 | 2.58 | 0.010 |
Working environment (rural/urban) | 0.12 | 2.03 | 0.043 | 0.04 | 0.63 | 0.532 | -0.06 | -1.19 | 0.236 |
Years in current speciality | 0.00 | -0.01 | 0.995 | -0.12 | -1.64 | 0.103 | 0.00 | 0.02 | 0.988 |
Perspective Taking | 0.05 | 0.82 | 0.413 | -0.10 | -1.53 | 0.127 | 0.35 | 5.80 | <0.001 |
Standing in the Patient’s Shoes | -0.04 | -0.62 | 0.536 | -0.11 | -1.70 | 0.091 | 0.08 | 1.28 | 0.203 |
Compassionate Care | -0.07 | -1.13 | 0.259 | -0.10 | -1.57 | 0.118 | 0.08 | 1.35 | 0.178 |
R2=0.129, df=9, p<0.001 | R2=0.078, df=9, p=0.006 | R2=0.224, df=9, p<0.001 |
R2: Coefficient of determination
E – Emotional Exhaustion, D – Depersonalisation, PA – Personal Accomplishment
3.3. Correlation between Physicians’ Health, Empathic Attitudes (JSEtot) and Burnout
Physicians with a chronic disease scored a higher JSEtot and they were more emotionally exhausted. Subjective evaluations of physicians’ general health, mood and emotional state were not significantly correlated with their empathic attitude, nor were the number of sick leave days. Details are in Table 2.
3.4. Correlation between Empathic Attitude and Burnout Dimensions
JSEtot was weakly negatively correlated with D (r=-0.224, p<0.001), yet the correlation between JSEtot and PA was positive and of moderate strength (r=0.372, p<0.001). However, EE was not correlated to JSEtot. Additional multivariate analysis positively associated Perspective Taking with PA (Table 3).
4. Discussion
This study assessed the level of empathic attitude and extent of burnout in Slovenian FMDs, and tested associations between JSEtot and individual burnout dimensions with socio-demographic factors, working conditions and physician health (Tables 2, 3). Psychometric properties of Slovenian MBI and JSE scales are also presented (Table 1), introducing the Slovenian version of JSE as validated and highly recommended instrument.
The participants reported a higher level of burnout when comparing mean values for each dimension with European FMDs (Table 2) (3, 33) and Slovenian family medicine trainees (21). A large cross-national burnout study (34) included Slovenian psychiatry trainees, but used a 16-item MBI-GS scale, which made the comparison very difficult. Another hurdle is different speciality, given that psychiatrists could exhibit even higher burnout and have different work process than FMDs. The positive association between emotional exhaustion and age and work experience (Table 2) was identified, concordant with a Slovenian study of family medicine trainees results (21). Maslach (15) discovered greater burnout in those at the beginning of their careers, but warns of survival bias. With regard to work experience, specialists reported higher burnout scores in all three dimensions, in comparison to trainees (Table 3). This is plausible, considering the situation in the Slovenian healthcare system, with specialists taking on an even higher workload, i.e., an excessive number of patients and extensive bureaucracy (35). The overall higher burnout in specialists (Table 3) could be attributed also to the sampling method.
Slovenian female FMDs reported lower personal accomplishment than males (Table 3), while in an European burnout study (3), a strong association between male gender and a high score in all three burnout dimensions was found. Parenting was associated with higher personal accomplishment scores, whereas living in an intimate partnership was not associated with any dimension of MBI (Table 3). Parenting probably shapes personality more than having a partner, in terms of hardiness, self-esteem, non-avoidant coping style, which are found to protect against burnout (15). An increased tendency to experience burnout in those who are single or not married was reported previously (15), while Park et al. reported of no correlation between MBI and marital status or parenting (36).
The correlation between workload and emotional exhaustion (Table 2), supported by the findings highlighting difficult working conditions in Slovenian primary health care, with 90% of physicians dealing with at least 40 patients and 20% with at least 60 patients per day (Table 2), is concordant with a previous Slovenian study of family medicine trainees and Croatian FMDs (21, 37). An additional burden was demonstrated in those working in rural family medicine clinics (Table 3), which is concordant with the study in which those working in rural settings scored higher on the MBI (28).
Unlike burnout, there are few studies that deal with physicians’ empathy, yet research into this topic is on the rise (38). It is difficult to compare scores for empathic attitude, as different instruments and study populations were in use (39, 40). The FMDs in this study reached a JSEtot (Table 2) comparable to that of medical students (40, 41), but the scores were lower than in doctors in other studies (11, 24). These findings could be attributed to cultural, educational and organisational differences of studied populations. Some other researchers also shortened the scale to improve internal consistency (13, 36). Literature often describes greater empathy in female physicians and medical students (1, 13, 19, 24, 36, 40, 41, 42), but no correlation between gender and empathic attitude was discovered in this study (Table 2).
FMDs with a chronic illness reported higher emotional exhaustion and showed a higher empathic attitude (Table 2). Physical illness affecting burnout has not yet been adequately researched and there is a high possibility of burnout in the presence of a persistent stressor, such as chronic disease (43). The impact of chronic illness on empathic capacity was examined in some qualitative studies, showing greater empathy in those who had experienced illness themselves (44).
More empathic participants felt greater personal accomplishment (Table 2), similarly to studies of Spanish (42) and French (45) FMDs and to family medicine trainees (20); physicians with greater empathic attitudes were reported to be less emotionally exhausted (36). Cognitive component of empathy (Perspective Taking) was associated with greater personal accomplishment (Table 2), similarly to Paro et al. (46), who reported personal accomplishment to be significantly associated with decreasing personal distress and which was found to be a reliable predictor for perspective taking. Some research shows that deficits in perspective taking alone might be a risk for burnout, whereas higher perspective taking and empathic concern might be protective (19, 47).
Given that multiple comparisons bias was additionally addressed by holding alpha error rate at 5% and changing the level of statistical significance (p<0.001), this emphasised the most important results of this study, i.e., a positive association between emotional exhaustion and age and working experience ((r=0.213, p<0.001) and (β=0.28, p<0.001), respectively), and between personal accomplishment and cognitive component of empathy (β=0.35, p<0.001). That is concordant with several previous studies and strengthens the reliability of association between empathy and burnout. Contrary to previous studies, the correlations between age and burnout in FMDs could be explained by the Slovenian health care system organisation.
In this study, organisational issues are shown to be a high risk factor for burnout (Table 2), high workload being an issue in Slovenia as well as in other developed countries (3, 15, 37). The situation in the country, with an aging population, the lack of physicians working in primary care, a decrease in interest in family medicine residencies, and shortage of time for the patient (48), suggests an urgent need for psychological help and support for FMDs, taking into consideration empathic attitude, which is associated with the feeling of personal accomplishment.
This study, being the first in Slovenia to assess burnout and empathic attitude in trainees and specialists in family medicine, and with a satisfactory response rate (56%), opens a new dimension of the physician-patient relationship and emphasises the empathic approach to family medicine. The results from this study could be the starting point for a discussion on the restructuring of curricula for both medical students and family medicine trainees, and support for the work of specialists, since FMDs work in relative isolation in their outpatient clinics and are deprived of peer support, as capacity building was shown to be empowering (49, 50). Additionally, the Slovenian version of JSE is presented and could be applied in further research.
4.1. Limitations of the Study
One of the main limitations of our research is sampling, being of convenience, with only a part of FMDs (members of syndicate and trainees) and not all FMDs were approached. Secondly, as the study was cross-sectional, the responses were analysed at a certain moment. Furthermore, self-assessment scales, such as JSE and MBI, are subjective and biased, as they are based on the respondent’s self-observation. Finally, there was also a slightly larger proportion of women included in the study (82%), compared to female FMDs in Slovenia (73.5%, (51)). Therefore, further research is needed on representative, random samples that would confirm our findings, especially prospective longitudinal studies to explore the association between empathic attitude and burnout.
5. Conclusions
The associations between emotional exhaustion, workload, age and work experience, depict important issues of Slovenian family medicine. Given that the association between burnout and empathy was confirmed, the importance of learning empathic communication and peer-support-based capacity building in FMDs for the prevention of burnout have been shown. There is a need for further intervention studies in medical students and FMDs, in order to evaluate these findings, elaborate peer support interventions and enhance the quality of patient care.
Acknowledgements
We want to thank all the family medicine doctors of our region who agreed to participate in our research.
Abbreviations
- JSE-HP
Jefferson Scale of Empathy-Health Professional version
- MBI
Maslach Burnout Inventory
- JSEtot
total score on JSE-HP
- FMD
family medicine doctor
- EE
Emotional Exhaustion
- D
Depersonalisation
- PA
Personal Accomplishment
Appendix
Lestvica za samo-ocenjevanje empatične naravnanosti po Jeffersonu (JSE – HP version – Slovenian)
Navodila: Uporabite kemični svinčnik in označite stopnjo strinjanja z vsako od navedenih trditev tako, da s križcem označite številko, ki najbolj ustreza stopnji vašega strinjanja s trditvijo.
Prosimo, da uporabite navedeno 7-stopenjsko lestvico (višja številka na lestvici pomeni večjo stopnjo strinjanja s trditvijo. Označite samo eno številko pri vsaki trditvi.
se nikakor ne strinjam | se povsem strinjam | ||||||
---|---|---|---|---|---|---|---|
1. Moje razumevanje počutja bolnikov in njihovih družin ne vpliva na medicinsko ali kirurško zdravljenje. | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
2. Moji bolniki se počutijo bolje, če razumem in upoštevam njihova čustva. | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
3. Težko mi je gledati na stvari iz zornega kota bolnikov. | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
4. V odnosu med zdravnikom in bolnikom je razbiranje nebesednih sporočil enako pomembno kot besedna plat sporazumevanja. | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
5. Imam dober smisel za humor, kar po moje prispeva k boljšemu kliničnemu izidu. | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
6. Ker so ljudje različni, je zame težko gledati na stvari iz zornega kota bolnikov. | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
7. Pri pogovoru z bolniki in jemanju anamneze se trudim, da ne polagam pozornosti na njihova čustva. | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
8. Upoštevanje bolnikovih osebnih izkušenj ne vpliva na izid zdravljenja. | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
9. Pri obravnavi bolnikov si skušam predstavljati, kako je »v njihovih čevljih«. | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
10. Moji bolniki cenijo moje razumevanje njihovih čustev, kar je samo po sebi terapevtsko. | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
11. Bolezni lahko pozdravimo zgolj z medicinsko ali kirurško obravnavo; čustvene vezi z mojimi bolniki pri tem niso pomembne. | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
12. Menim, da je spraševanje bolnikov o dogajanju v njihovem življenju nepomemben dejavnik pri razumevanju njihovih telesnih težav. | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
13. Da bi lažje razumel(a), kaj bolniki mislijo in čutijo, sem pozoren(a) na na njihovo nebesedno sporočanje (način govora in govorico telesa). | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
14. Verjamem, da čustva niso pomembna pri zdravljenju bolezni. | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
15. Empatija je terapevtska veščina, brez katere je moja uspešnost zdravljenja omejena. | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
16. Za moj odnos z bolniki je pomembno, da poznam njihovo čustveno stanje in dogajanje v njihovih družinah. | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
17. Zato da bi jih lahko bolje obravnaval(a), poskušam razmišljati kot moji bolniki. | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
18. Ne dovolim si, da bi name vplivale tesne osebne vezi med bolniki in njihovimi družinskimi člani. | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
19. Ne uživam v branju nestrokovne (nemedicinske) literature ali ob umetniških delih. | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
20. Prepričan(a) sem, da je empatija pomemben terapevtski dejavnik v procesu zdravljenja. | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
© Jefferson Medical College. All rights reserved. Slovenian translation by Lea Penšek, MD (lea.pensek@gmail.com) and Polona Selič, PhD Clinical, Psychology (polona.selic@siol.net), University of Ljubljana, Slovenia
Algoritem točkovanja za lestvico JSE
Vprašani mora odgovoriti na vsaj 16 (80 %) od 20 trditev, sicer je obrazec nepopoln in ga je treba izključiti iz analize podatkov.
V primeru, da ne ogovori na 4 ali manj trditev, se manjkajoče vrednosti nadomestijo s povprečno vrednostjo trditev, na katere je sodelujoči odgovoril.
Točkovanje lestvice: Trditve 1, 3, 6, 7, 8, 11, 12, 14, 18 in 19 se točkujejo obratno (tj.se popolnoma strinjam=1…se nikakor ne strinjam=7), medtem ko se ostale trditve točkujejo skladno z Likertovo lestvico (tj. se nikakor ne strinjam=1…se popolnoma strinjam=7).
Skupno število točk je seštevek točk za posamezne trditve. Višja dosežena celokupna vrednost na vprašalniku kaže na večjo empatično naravnanost.
POMEMBNO: Algoritem točkovanja je namenjen izključno za vrednotenje JSE obrazcev kupljenih za en sam projekt. Kopiranje ali deljenje algoritma je prepovedano.
© Jefferson Medical College All rights reserved
Footnotes
Conflicts of interest: The authors declare that no conflicts of interest exist.
Funding: The authors acknowledge the financial support from the Slovenian Research Agency, research core funding Research in the Field of Public Health No. P3–0339.
Ethical approval: Research was conducted according to ethical principles and was approved by the National Ethics Committee in 2016, reference number 0120-206-2016.
References
- 1.Derksen F, Bensing J, Lagro-Janssen A.. Effectiveness of empathy in general practice: a systematic review. Br J Gen Pract. 2013;63:e76–84. doi: 10.3399/bjgp13X660814. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Halpern J.. What is clinical empathy? J Gen Intern Med. 2003;18:670–4. doi: 10.1046/j.1525-1497.2003.21017.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Soler JK, Yaman H, Esteva M, Dobbs F, Asenova RS, Katic M. et al. Burnout in European family doctors: the EGPRN study. Fam Pract. 2008;25:245–65. doi: 10.1093/fampra/cmn038. [DOI] [PubMed] [Google Scholar]
- 4.Maslach C, Jackson SE, Leiter MP. The Maslach Burnout inventory manual. 3rd ed. Palo Alto: Consulting Psychologists Press; 1996. [Google Scholar]
- 5.Shanafelt TD, Boone S, Tan L, Dyrbye LN, Sotile W, Satele D. et al. Burnout and satisfaction with work-life balance among US physicians relative to the general US population. Arch Intern Med. 2012;172:1377–85. doi: 10.1001/archinternmed.2012.3199. [DOI] [PubMed] [Google Scholar]
- 6.Pejušković B, Lečić-Toševski D, Priebe S, Tošković O.. Burnout syndrome among physicians - the role of personality dimensions and coping strategies. Psychiatr Danub. 2011;23:389–95. [PubMed] [Google Scholar]
- 7.Bellet PS, Maloney MJ.. The importance of empathy as an interviewing skill in medicine. JAMA. 1991;266:1831–2. doi: 10.1001/jama.1991.03470130111039. [DOI] [PubMed] [Google Scholar]
- 8.Halpern J.. From idealized clinical empathy to empathic communication in medical care. Med Health Care Philos. 2014;17:301–11. doi: 10.1007/s11019-013-9510-4. [DOI] [PubMed] [Google Scholar]
- 9.Kim SS, Kaplowitz S, Johnston MV.. The effects of physician empathy on patient satisfaction and compliance. Eval Health Prof. 2004;27:237–51. doi: 10.1177/0163278704267037. [DOI] [PubMed] [Google Scholar]
- 10.West CP, Huschka MM, Novotny PJ, Sloan JA, Kolars JC, Habermann TM. et al. Association of perceived medical errors with resident distress and empathy: a prospective longitudinal study. JAMA. 2006;296:1071–8. doi: 10.1001/jama.296.9.1071. [DOI] [PubMed] [Google Scholar]
- 11.Hojat M, Louis DZ, Markham FW, Wender R, Rabinowitz C, Gonnella JS.. Physicians’ empathy and clinical outcomes for diabetic patients. Acad Med. 2011;86:359–64. doi: 10.1097/ACM.0b013e3182086fe1. [DOI] [PubMed] [Google Scholar]
- 12.Krasner MS, Epstein RM, Beckman H, Suchman AL, Chapman B, Mooney CJ. et al. Association of an educational program in mindful communication with burnout, empathy, and attitudes among primary care physicians. JAMA. 2009;302:1284–93. doi: 10.1001/jama.2009.1384. [DOI] [PubMed] [Google Scholar]
- 13.Petek Šter M, Selič P.. Assessing empathic attitudes in medical students: the re-validation of the Jefferson Scale of Empathy student version report. Zdr Varst. 2015;54:282–92. doi: 10.1515/sjph-2015-0037. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Hojat M, Mangione S, Nasca TJ, M Cohen MJ, Gonnella JS, Erdmann JB. et al. The Jefferson Scale of Physician Empathy: development and preliminary psychometric data. Educ Psychol Meas. 2001;61:349–65. [Google Scholar]
- 15.Maslach C, Schaufeli WB, Leiter MP.. Job burnout. Annu Rev Psychol. 2001;52:397–422. doi: 10.1146/annurev.psych.52.1.397. [DOI] [PubMed] [Google Scholar]
- 16.Balch CM, Oreskovich MR, Dyrbye LN, Colaiano JM, Satele DV, Sloan JA. et al. Personal consequences of malpractice lawsuits on American surgeons. J Am Coll Surg. 2011;213:657–67. doi: 10.1016/j.jamcollsurg.2011.08.005. [DOI] [PubMed] [Google Scholar]
- 17.Anagnostopoulos F, Liolios E, Persefonis G, Slater J, Kafetsios K, Niakas D.. Physician burnout and patient satisfaction with consultation in primary health care settings: evidence of relationships from a one-with-many design. J Clin Psychol Med Settings. 2012;19:401–10. doi: 10.1007/s10880-011-9278-8. [DOI] [PubMed] [Google Scholar]
- 18.Thirioux B, Birault F, Jaafari N.. Empathy is a protective factor of burnout in physicians: new neuro-phenomenological hypotheses regarding empathy and sympathy in care relationship. Front Psychol. 2016;7:1–11. doi: 10.3389/fpsyg.2016.00763. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Gleichgerrcht E, Decety J.. Empathy in clinical practice: how individual dispositions, gender, and experience moderate empathic concern, burnout, and emotional distress in physicians. PLoS One. 2013;8:e61526. doi: 10.1371/journal.pone.0061526. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Zenasni F, Boujut E, Woerner A, Sultan S.. Burnout and empathy in primary care: three hypotheses. Br J Gen Pract. 2012;62:346–7. doi: 10.3399/bjgp12X652193. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Seliš P, Stegne-Ignjatović T, Klemenc-Ketiš Z.. Burnout among Slovenian family medicine trainees: a cross-sectional study. Zdrav Vestn. 2012;81:218–24. [Google Scholar]
- 22.NIJZ podatkovni portal SI-STAT. 2015 2018 March 7th; https://podatki.nijz.si/Table.aspx?layout=tableViewLayout2&px_tableid=BPI_TB07.px&px_path=NIJZpodatkovniportal5Virivzdravstvu1Izvajalcizdravstvenedejavnosti&px_language=sl&px_db=NIJZpodatkovniportal&rxid=d16ff457-ac65-48f1-87ed-ac0dcabff7c0 Accessed. at. [Google Scholar]
- 23.Hojat M. Empathy in Patient Care. New York: NY: Springer New York; 2007. The Jefferson Scale of Physician Empathy; pp. 87–115. [DOI] [Google Scholar]
- 24.Hojat M, Gonnella JS, Nasca TJ, Mangione S, Vergare M, Magee M.. Physician empathy: definition, components, measurement, and relationship to gender and specialty. Am J Psychiatry. 2002;159:1563–9. doi: 10.1176/appi.ajp.159.9.1563. [DOI] [PubMed] [Google Scholar]
- 25.Tavakol S, Dennick R, Tavakol M.. Psychometric properties and confirmatory factor analysis of the Jefferson Scale of Physician Empathy. BMC Med Educ. 2011;11:54. doi: 10.1186/1472-6920-11-54. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Jefferson Scale of Empathy - Thomas Jefferson University. 2017 November 7th; http://www.jefferson.edu/university/skmc/research/research-medical-education/jefferson-scale-of-empathy.html Accessed. at. [Google Scholar]
- 27.Lamovec T. Slovene. Ljubljana: Filozofska fakulteta, Oddelek za psihologijo; 1994. [Psychodiagnostics of personality] [Google Scholar]
- 28.Preti A, Vellante M, Gabbrielli M, Lai V, Muratore T, Pintus E. et al. Confirmatory factor analysis and measurement invariance by gender, age and levels of psychological distress of the short TEMPS-A. J Affect Disord. 2013;151:995–1002. doi: 10.1016/j.jad.2013.08.025. [DOI] [PubMed] [Google Scholar]
- 29.Hair JF. A primer on partial least squares structural equations modeling (PLS-SEM) CA: Sage Publications; 2014. [Google Scholar]
- 30.Browne M, Cudeck R. Bollen K, Long J. Testing Structural Equation Models. Newbury Park: CA: Sage Publications; 1993. Alternative ways of assessing model fit; pp. 136–61. [Google Scholar]
- 31.Hu LT, Bentler PM.. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Model. 1999;6:1–55. doi: 10.1080/10705519909540118. [DOI] [Google Scholar]
- 32.Faul F, Erdfelder E, Buchner A, Lang A-G.. Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behav Res Methods. 2009;41:1149–60. doi: 10.3758/BRM.41.4.1149. [DOI] [PubMed] [Google Scholar]
- 33.Goehring C, Bouvier Gallacchi M, Künzi B, Bovier P.. Psychosocial and professional characteristics of burnout in Swiss primary care practitioners: a cross-sectional survey. Swiss Med Wkly. 2005;135:101–8. doi: 10.4414/smw.2005.10841. 2005/07/smw-10841. [DOI] [PubMed] [Google Scholar]
- 34.Jovanović N, Podlesek A, Volpe U, Barrett E, Ferrari S, Rojnic Kuzman M. et al. Burnout syndrome among psychiatric trainees in 22 countries: risk increased by long working hours, lack of supervision, and psychiatry not being first career choice. Eur Psychiatry. 2016;32:34–41. doi: 10.1016/j.eurpsy.2015.10.007. [DOI] [PubMed] [Google Scholar]
- 35.Poplas Susić T, Švab I, Kersnik J.. The project of model practices in family medicine in Slovenia. Zdrav Vestn. 2013;82:635–47. [Google Scholar]
- 36.Park C, Lee YJ, Hong M, Jung CH, Synn Y, Kwack YS. et al. A multicenter study investigating empathy and burnout characteristics in medical residents with various specialties. J Korean Med Sci. 2016;31:590–7. doi: 10.3346/jkms.2016.31.4.590. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Ožvačić Adžić Z, Katić M, Kern J, Soler JK, Cerovečki V, Polašek O.. Is burnout in family physicians in Croatia related to interpersonal quality of care? Arch Ind Hyg Toxicol. 2013;64:255–64. doi: 10.2478/10004-1254-64-2013-2307. [DOI] [PubMed] [Google Scholar]
- 38.Bouma HK.. Is empathy necessary for the practice of “good” medicine. Open Ethics J. 2008;2:1–12. doi: 10.2174/1874761200802010001. [DOI] [Google Scholar]
- 39.Hojat M, Axelrod D, Spandorfer J, Mangione S.. Enhancing and sustaining empathy in medical students. Med Teach. 2013;35:996–1001. doi: 10.3109/0142159X.2013.802300. [DOI] [PubMed] [Google Scholar]
- 40.Hojat M, Gonnella JS.. Eleven years of data on the Jefferson Scale of Empathy-medical student version (JSE-S): proxy norm data and tentative cutoff scores. Med Princ Pract. 2015;24:344–50. doi: 10.1159/000381954. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Hegazi I, Wilson I.. Maintaining empathy in medical school: it is possible. Med Teach. 2013;35:1002–8. doi: 10.3109/0142159X.2013.802296. [DOI] [PubMed] [Google Scholar]
- 42.Yuguero Torres O, Esquerda Aresté M, Marsal Mora JR, Soler-González J.. Association between sick leave prescribing practices and physician burnout and empathy. PLoS One. 2015;10:e0133379. doi: 10.1371/journal.pone.0133379. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Sharpe L, Curran L.. Understanding the process of adjustment to illness. Soc Sci Med. 2006;62:1153–66. doi: 10.1016/j.socscimed.2005.07.010. [DOI] [PubMed] [Google Scholar]
- 44.Woolf K, Cave J, McManus IC, Dacre JE.. ”It gives you an understanding you can’t get from any book.” The relationship between medical students’ and doctors’ personal illness experiences and their performance: a qualitative and quantitative study. BMC Med Educ. 2007;7:50. doi: 10.1186/1472-6920-7-50. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Zenasni F, Boujut E, Buffel C, Catupinault A, Tavani JL, Rigal L. et al. Development of a French-language version of the Jefferson Scale of Physician Empathy and association with practice characteristics and burnout in a sample of general practitioners. Int J Pers Cent Med. 2012;2:759–66. doi: 10.5750/ijpcm.v2i4.295. [DOI] [Google Scholar]
- 46.Paro HBMS, Silveira PSP, Perotta B, Gannam S, Enns SC, Giaxa RRB. et al. Empathy among medical students: is there a relation with quality of life and burnout? PLoS One. 2014;9:e94133. doi: 10.1371/journal.pone.0094133. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Lamothe M, Boujut E, Zenasni F, Sultan S.. To be or not to be empathic: the combined role of empathic concern and perspective taking in understanding burnout in general practice. BMC Fam Pract. 2014;15:15. doi: 10.1186/1471-2296-15-15. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 48.Petek Ster M, Svab I, Zivcec Kalan G.. Factors related to consultation time: experience in Slovenia. Scand J Prim Health Care. 2008;26:29–34. doi: 10.1080/02813430701760789. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 49.Kelm Z, Womer J, Walter JK, Feudtner C.. Interventions to cultivate physician empathy: a systematic review. BMC Med Educ. 2014;14:219. doi: 10.1186/1472-6920-14-219. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 50.Benson J, Magraith K.. Compassion fatigue and burnout: the role of Balint groups. Aust Fam Physician. 2005;34:497–8. [PubMed] [Google Scholar]
- 51.NIJZ podatkovni portal SI-STAT. 2015 2017 November 7th; https://podatki.nijz.si/pxweb/sl/NIJZpodatkovniportal/NIJZpodatkovniportal6Zdravstvenestoritveinsistemzdravstvenegavarstva6dIzvajalcizdravstvenedejavnosti/BPI_TB06.px/?rxid=6ccac404-85b5-4c6e-8c85-767b40c21d2e Accessed. at. [Google Scholar]