Abstract
Support groups are an important resource for people living with systemic sclerosis (SSc; scleroderma). Peer support group leaders play an important role in the success and sustainability of SSc support groups, but face challenges that include a lack of formal training. An SSc support group leader training program could improve leader self-efficacy to carry out important leadership tasks, including the management of group dynamics. However, no measures exist to assess self-efficacy among SSc support group leaders. The objective of this study was to develop and provide preliminary evidence on the reliability and validity of the Scleroderma Support Group Leader Self-efficacy Scale (SSGLSS). The SSGLSS was administered to two sets of SSc support group leaders from North America, Europe, and Australia. Study 1 participants (n = 102) completed the SSGLSS only. Study 2 participants (n = 55) completed the SSGLSS and the Oldenburg Burnout Inventory (OLBI). For both studies, we evaluated internal consistency reliability using Cronbach’s coefficient alpha. Convergent validity was assessed in Study 2 using Pearson correlations of the SSGLSS with the OLBI exhaustion and disengagement subscales. Cronbach’s alpha was 0.96 in Study 1 and 0.95 in Study 2. Consistent with our hypotheses, there was a small negative correlation between SSGLSS scores and the OLBI exhaustion subscale (r = -0.25, p<0.01) and a moderate negative correlation between SSGLSS scores and the disengagement subscale (r = -0.38, p<0.01). These results suggest that the SSGLSS is a reliable and valid measure of self-efficacy for carrying out support group leadership tasks.
Keywords: Patient-reported outcome measures, Psychometrics, Scleroderma, Systemic, Self-help groups, Social support
Introduction
Systemic sclerosis (SSc; scleroderma) is a rare, chronic, autoimmune connective tissue disease characterized by abnormal fibrotic processes and excessive collagen production (1, 2). Many SSc patients turn to support groups to help them cope with the burden of living with their disease (3, 4).
Support groups adhere to the self-help concept that people who face a common challenge can empower each other through mutual support (5). Support groups typically involve an education or information-sharing component and opportunities to exchange emotional and practical support; they may take place face-to-face or virtually, and may be structured or unstructured (6, 7). In common diseases, support group services are often organized and led by health care professionals. In rare diseases, including SSc, professionally led services are typically not available (8, 9). As a result, people with a rare disease often rely on support groups that are peer-initiated and led (10, 11).
There are currently almost 200 active SSc support groups across Canada and the USA (3, 4). However, many people with SSc do not have geographic access to a support group, and many support groups that are initiated are not sustained due to obstacles that could be addressed by providing training to patient support group leaders (10, 11). Challenges for SSc support group leaders include practical difficulties, such as limited resources; poor coordination with medical professionals and patient organizations; difficulties with group leadership tasks, including managing group dynamics; and personal challenges, such as managing one’s own health condition while supporting others (12-14).
A training program could provide information and skills to improve the ability of SSc peer support group leaders to lead effective groups, reduce the emotional and physical burden on group leaders, and encourage new leaders to set up support groups where none exist or via the internet. Thus, the Scleroderma Patient-centered Intervention Network, Scleroderma Canada, and the Scleroderma Foundation have partnered to develop a scleroderma support group leader education program. The program will provide SSc support group leaders with knowledge and skills needed to effectively organize and lead SSc support groups. The program is intended to improve support group leader self-efficacy. Self-efficacy is defined as the perceived ability to perform the actions needed to accomplish specific goals (15). There are no validated measures available to assess support group leader self-efficacy to carry out support group leadership tasks in scleroderma or other patient groups.
The objective of the present study was to assess aspects of the validity of the Scleroderma Support Group Leader Self-efficacy Scale (SSGLSS). We describe the steps undertaken to develop the SSGLSS; we evaluate internal consistency and reliability; and we assess convergent validity with a measure of burnout. Effect sizes for correlations have been described as small (|r|≤0.3), moderate (0.3< |r| <0.5), or large (|r| ≥0.5) (16). A meta-analysis of 57 studies found a correlation between self-efficacy and job burnout of -0.33, including -0.33 between self-efficacy and exhaustion and -0.31 between self-efficacy and disengagement (17). Thus, based on previous research in other settings, we hypothesized that in SSc the SSGLSS would also have small to moderate negative correlations with measures of these burnout domains.
Methods
We administered the SSGLSS to 2 sets of SSc support group leaders. In Study 1 (initial validation and reliability), which targeted support group leaders from North America and Europe, leaders completed the SSGLSS only. In Study 2 (convergent validity), which included support group leaders from North America, France, and Australia, leaders completed the SSGLSS and the Oldenburg Burnout Inventory (OLBI) (18, 19).
Participants
Eligible Study 1 participants were current SSc support group leaders. Eligible Study 2 participants were current or past SSc support group leaders, some of whom may have participated in Study 1.
Study 1
North American support group leaders were recruited to complete Study 1 between April and August 2015 through: (i) postings on the Scleroderma Canada, Scleroderma Foundation, and Canadian provincial SSc society websites; (ii) postings on Scleroderma Canada and Scleroderma Foundation social media venues (e.g., Facebook, Twitter); (iii) the distribution of flyers at the Scleroderma Foundation annual conference; (iv) announcements in SSc patient newsletters; (v) emails to support group leaders and members across Canada and the USA; and (vi) postings in SSc-related chat rooms.
European support group leaders were recruited to complete Study 1 between March and August 2016 through: (i) the distribution of flyers and direct contact with patients at the 4th Systemic Sclerosis World Congress in Lisbon, Portugal; (ii) referrals from the Federation of European Scleroderma Associations (FESCA) generated via emails and postings on the FESCA website and other social media venues (e.g., Facebook, Twitter); (iii) announcements in SSc patient newsletters; (iv) emails to support group leaders and members throughout Europe; and (v) postings on European SSc society websites. In addition to European support group leaders, leaders from Australia who attended the World Congress completed the survey and were also included.
Study 2
Leaders of SSc support groups were recruited to complete Study 2 between December 2016 and April 2017. Respondents were recruited through: (i) postings on the Scleroderma Canada, Scleroderma Foundation, and Canadian provincial SSc organization websites; (ii) postings on Scleroderma Canada and Scleroderma Foundation social media venues (e.g., Facebook, Twitter); (iii) announcements in SSc patient newsletters; and (iv) emails to support group leaders associated with patient organizations in Canada, the USA, France, and Australia.
Procedure
In both studies, surveys were accessible via the online survey tool Qualtrics. The survey was translated into French, German, Italian, Dutch, Portuguese, and Spanish, using an accepted forward–backward translation method (20). Study 1 was available in English, French, German, Italian, Dutch, Portuguese, and Spanish. Study 2 was available in English and French. After clicking on the survey link and selecting their preferred language, respondents were shown a brief consent form that described study objectives and provided instructions on how to complete the survey. Respondents were given the option to close their browser and not participate or to provide consent by clicking an arrow to continue with the survey. Surveys were set up using cookies to prevent respondents from completing them more than once to reduce the possibility of duplicate responses.
Studies 1 and 2 were approved by the Ethics Committee of the Jewish General Hospital in Montréal, Québec. Respondents were not required to provide written informed consent because the surveys were done anonymously and did not involve collection of any data that could be used to identify respondents.
Measures
SSGLSS
Initial items considered for inclusion in the SSGLSS were obtained from the Group Leader Self-Efficacy Instrument, a 37-item self-report questionnaire that assesses self-efficacy for performing group leader skills (12). The Group Leaders Self-Efficacy Instrument is intended for use with group psychotherapy leaders, so many of its items are not relevant or appropriate for support group leaders. Items from this instrument were reviewed for relevancy, and relevant items were considered for inclusion, along with items from a questionnaire intended for leaders of cancer and multiple sclerosis support groups (13) and items that we generated from the results of a published study on the experiences of leaders of cancer support groups (14).
Initial survey items were reviewed by research team members, who edited individual items, made recommendations to remove items that were not relevant for SSc or were repetitive, and generated new items to reflect SSc-specific content, based on their own experience or on qualitative interviews that we conducted with SSc support group leaders (n = 10). Items were reviewed iteratively by all research team members until consensus on the final item pool was reached. Team members who participated in this process included representatives from Scleroderma Canada and the Scleroderma Foundation; a patient advisory team that consisted of 6 SSc support group leaders from Canada and the USA; and researchers with expertise in SSc.
The final version of the SSGLSS, which was used in both studies, consisted of 32 core items that assessed the confidence of SSc support group leaders to carry out tasks necessary for leading a support group successfully (see Tab. I). The list of items was intended to comprehensively reflect tasks important to the effective leadership of an SSc support group so that both overall self-efficacy and item-specific efficacy could be evaluated. Item response options included Strongly Disagree, Disagree, Slightly Disagree, Slightly Agree, Agree, and Strongly Agree (scored 0-5). Possible total scores range from 0 to 160 with higher scores indicating greater self-efficacy.
Table I.
Scleroderma Support Group Leader Self-efficacy Scale (SSGLSS) items
| Survey items | Study 1 (n = 102) | Study 2 (n = 55) | ||
|---|---|---|---|---|
| “I am confident in my ability to…” | Mean (SD) | Corrected item-total correlation | Mean (SD) | Corrected item-total correlation |
| 1. Obtain financial or other resources needed to run the group. | 3.3 (1.4) | 0.38 | 3.1 (1.4) | 0.18 |
| 2. Promote the group to health professionals as an important resource for patients. | 3.9 (1.2) | 0.51 | 4.0 (0.9) | 0.56 |
| 3. Share responsibilities, including administrative and practical tasks, with a co-facilitator or other group members. | 3.8 (1.3) | 0.52 | 4.0 (1.1) | 0.36 |
| 4. Manage group members who are overly talkative or monopolize the discussion | 3.8 (0.9) | 0.60 | 3.8 (1.0) | 0.65 |
| 5. Manage group members who assume the role of the “know-it-all.” | 3.8 (1.0) | 0.62 | 3.7 (1.2) | 0.69 |
| 6. Support members of the group who are grieving. | 4.2 (1.0) | 0.60 | 3.9 (1.0) | 0.48 |
| 7. Help overly shy group members feel comfortable interacting with the group. | 4.2 (0.8) | 0.70 | 4.2 (0.7) | 0.54 |
| 8. Help group members cope with difficult events, such as the death of a member. | 3.8 (1.1) | 0.66 | 3.9 (1.0) | 0.61 |
| 9. Effectively recruit new members. | 3.5 (1.3) | 0.70 | 3.4 (1.1) | 0.53 |
| 10. Address the different needs of groups members at varying stages of the disease. | 3.8 (1.0) | 0.75 | 4.0 (0.8) | 0.64 |
| 11. Manage conflicts and disagreements between group members. | 3.6 (1.0) | 0.78 | 3.6 (1.0) | 0.68 |
| 12. Help the group establish appropriate group rules, such as maintaining confidentiality. | 4.2 (0.9) | 0.60 | 4.2 (0.8) | 0.64 |
| 13. Effectively publicize the group. | 3.6 (1.2) | 0.68 | 3.6 (1.1) | 0.56 |
| 14. Intervene effectively when group rules are not being followed. | 3.9 (0.9) | 0.61 | 3.9 (0.9) | 0.71 |
| 15. Obtain the support I need to cope with the emotional demands of leading the group. | 3.5 (1.2) | 0.65 | 3.6 (1.1) | 0.69 |
| 16. Respond constructively to feedback from group members. | 4.1 (0.9) | 0.83 | 4.2 (0.7) | 0.75 |
| 17. Help group members relate to other members of a different age. | 4.0 (0.8) | 0.77 | 4.2 (0.6) | 0.64 |
| 18. Provide the structure needed for successful meetings. | 4.1 (0.8) | 0.65 | 4.1 (0.7) | 0.66 |
| 19. Keep the group meetings interesting and relevant to both new and returning members. | 3.9 (0.9) | 0.59 | 4.0 (0.7) | 0.71 |
| 20. Manage group members who oversimplify or minimize the concerns of other members. | 3.9 (0.9) | 0.74 | 3.9 (0.9) | 0.80 |
| 21. Facilitate the group meetings so that all members have an opportunity to speak. | 4.2 (0.9) | 0.60 | 4.3 (0.7) | 0.62 |
| 22. Help the group stay focused on topics that are relevant to members. | 4.0 (0.9) | 0.66 | 4.0 (0.7) | 0.83 |
| 23. Obtain feedback from members about the group. | 4.0 (0.9) | 0.58 | 4.0 (0.8) | 0.75 |
| 24. Organize and plan activities for group members, such as having guest speakers. | 3.9 (1.1) | 0.52 | 3.8 (1.0) | 0.54 |
| 25. Help members feel comfortable in the group and relate to one another. | 4.3 (0.8) | 0.73 | 4.2 (0.8) | 0.57 |
| 26. Obtain feedback from members about my leadership. | 3.8 (1.0) | 0.72 | 3.8 (1.1) | 0.78 |
| 27. Help group members relate to other members of a different cultural background. | 4.0 (1.0) | 0.73 | 3.9 (0.9) | 0.58 |
| 28. Communicate reasonable boundaries about my availability outside of the group. | 4.0 (0.9) | 0.69 | 4.0 (0.7) | 0.63 |
| 29. Talk to a group member about her or his behavior if it is disruptive to the group. | 3.5 (1.1) | 0.72 | 3.7 (1.1) | 0.74 |
| 30. Ask a member to leave the group due to her or his disruptive behavior. | 3.0 (1.5) | 0.57 | 3.0 (1.5) | 0.71 |
| 31. Help group members relate to other members of a different gender. | 4.0 (0.8) | 0.76 | 4.0 (0.7) | 0.79 |
| 32. Recruit a co-facilitator or other group members to help me with leadership responsibilities. | 3.6 (1.2) | 0.62 | 3.9 (1.2) | 0.45 |
SD = standard deviation.
OLBI
The OLBI (18, 19) is a 16-item measure of burnout that assesses exhaustion and disengagement due to burnout. The OLBI was initially designed for work-related burnout, but has been adapted for numerous settings, and in multiple countries and languages (19). Our research team revised the wording of each OLBI item so that it reflected the support group environment rather than a work environment (e.g., “I find my work to be a positive challenge” revised to “I find my role as a support group leader to be a positive challenge”). The OLBI has a 2-factor structure (exhaustion and disengagement) with good measurement properties (18, 19). Items are scored on a 4-point scale; higher scores indicate higher levels of exhaustion and disengagement. Internal consistency reliability (Cronbach’s α) in the present sample was 0.84 for exhaustion and 0.80 for disengagement.
Data analysis
Corrected item-total correlations, which reflect the correlation of each item with the total scale score minus that item, were calculated for all SSGLSS items. Internal consistency reliability for the SSGLSS was calculated in Study 1 and Study 2 separately using Cronbach’s α. Convergent validity was assessed in Study 2 via Pearson correlations of the SSGLSS with the OLBI exhaustion and disengagement subscales. Analyses were conducted using SPSS version 23.
Results
Sample characteristics
There were 88 eligible respondents who initiated the North American part of Study 1. A total of 80 completed the full survey and were included in the present analyses. There were 23 eligible respondents who initiated the European part of Study 1, of whom 22 completed the survey and were included (total n = 102). For Study 2, 69 eligible respondents began the survey, and 55 completed all survey items and were included in the analyses (See Tab. II for participant characteristics).
Table II.
Participant characteristics
| Characteristics | Study 1 (n = 102) | Study 2 (n = 55) |
|---|---|---|
| Female sex, n (%) | 91 (89.2%) | 40 (72.7%) |
| Age in years, mean (SD) | 57.1 (12.3) | 60.2 (10.8) |
| Country, n (%) | ||
| Canada | 18 (17.6%) | 15 (27.3%) |
| USA | 62 (60.8%) | 28 (50.9%) |
| France | 1 (1.0%) | 7 (12.7%) |
| Australia | 6 (5.9%) | 5 (9.1%) |
| Other a | 15 (14.7%) | 0 (0.0%) |
| White race/ethnicity, n (%) | 95 (93.1%) | 50 (90.9%) |
| Married or living as married, n (%) | 70 (68.6%) | 41 (74.5%) |
| Education in years, mean (SD) | 15.0 (2.9) | 15.1 (3.2) |
| Occupational status, n (%) | ||
| Homemaker | 10 (9.8%) | 4 (7.3%) |
| Part- or full-time employment b | 26 (25.5%) | 10 (18.2%) |
| Disability | 33 (32.4%) | 16 (29.1%) |
| Retired | 29 (28.4%) | 24 (43.6%) |
| Not employed | 4 (3.9%) | 1 (1.8%) |
| SSc diagnosis, n (%) | ||
| Limited SSc | 51 (50.0%) | 27 (49.1%) |
| Diffuse SSc | 43 (42.2%) | 20 (36.4%) |
| Not provided | 8 (7.8%) | 2 (3.6%) |
| Leader not diagnosed with SSc | 0 (0.0%) | 6 (10.9%) |
| Years since SSc diagnosis, mean (SD) | 14.5 (7.8) | 14.9 (8.2) |
| Years as a SSc support group leader, mean (SD) | 7.3 (6.5) | 8.0 (6.5) |
| Received any training for leader role, n (%) | 32 (31.4%) | 15 (27.3%) |
| Current leader of SSc support group | 102 (100.0%) | 47 (85.5%) |
SD = standard deviation; SSc = systemic sclerosis.
Includes countries with <5 participants in Study 1 (Germany, Netherlands, Portugal, UK, Belgium, Romania, Switzerland, Finland, New Zealand).
Includes 1 full-time student.
Reliability and validity of the SSGLSS
As shown in Table I, mean item scores for the 32 SSGLSS items ranged from 3.0 to 4.3 (standard deviations [SDs] 0.8-1.4). For 29 of the 32 items, the corrected item-total correlation was ≥0.50 in both surveys (range 0.18-0.83). The mean total SSGLSS score for the 102 participants in Study 1 was 122.9 (SD 21.7; range 29-160); for the 55 participants in Study 2 it was 123.9 (SD 19.4; range 81-160). Cronbach’s alpha was 0.96 for Study 1 and 0.95 for Study 2. There were small to moderate negative correlations between the SSGLSS and OLBI total score (r = -0.34, p = 0.012), exhaustion subscale (r = -0.24, p = 0.080), and disengagement subscale (r = -0.38, p = 0.004).
Discussion
The results of the present study provide preliminary evidence that the SSGLSS is a reliable and valid measure of self-efficacy for carrying out tasks related to leading SSc support groups. Internal consistency reliability was high. Convergent validity with a measure of burnout, including exhaustion and disengagement subscales, was consistent with what has been reported in studies from other patient groups (17).
There are limitations to consider when interpreting the results of this study. First, respondents were recruited through national and provincial SSc organizations, patient conferences, SSc-related chat-rooms and newsletters, and emails to support group leaders. It is possible that SSc support group leaders who are more active within the SSc community participated in the study, and that these leaders could differ from other leaders. Consistent with this, scores on individual items tended to be high. Second, the sample sizes in Study 1 and Study 2 were small, which did not allow for complex analyses of measurement properties, such as factor analysis. We did not calculate sample size requirements a priori, but recruited as many support group leaders as possible. Given the small number of SSc support group leaders, it is unlikely that we would be able to recruit larger numbers. Third, Study 1 and Study 2 were anonymous online surveys, and it is possible that some participants in Study 1 also participated in Study 2. Fourth, given the single administration of the online survey, we could not evaluate test-retest reliability or sensitivity to change.
In conclusion, results of the present study provide preliminary evidence in support of using the SSGLSS to evaluate the effectiveness of a training program designed to improve the effectiveness and sustainability of SSc support groups by improving leader self-efficacy.
Acknowledgments
Scleroderma Support Group Project Advisory Team Members
Kerri Connolly, Director of Programs and Services of the Scleroderma Foundation, Danvers, Massachusetts - USA; Laura Dyas, Executive Director of the Scleroderma Foundation Michigan Chapter, Southfield, Michigan - USA; Stephen Elrod, Southern California Patient Group, Los Angeles, California - USA; Catherine Fortune, Ontario Patient Group, Ottawa, Ontario - Canada; Karen Gottesman, Director of Pharma & Biotech Engagement Scleroderma Foundation, Los Angeles, California - USA; Anna McCusker, Executive Director of Scleroderma Canada and the Scleroderma Society of Ontario, Hamilton, Ontario - Canada; Michelle Richard, President of Scleroderma Canada, Halifax, Nova Scotia - Canada; Robert Riggs, Chief Executive Officer of the Scleroderma Foundation, Danvers, Massachusetts - USA; Maureen Sauve, VP Advocacy and Public Relations of Scleroderma Canada and the Scleroderma Society of Ontario, Hamilton, Ontario - Canada; Nancy Stephens, Michigan Patient Group, Detroit, Michigan - USA.
Footnotes
Disclosures: Financial support: The research was supported by the Scleroderma Society of Ontario and the Canadian Initiative for Outcomes in Rheumatology Care.
Conflict of interest: None of the authors has financial interest related to this study to disclose.
References
- 1. Scleroderma Seibold J.. In: Harris ED, Budd RC, Firestein GS, et al., eds. Kelley’s textbook of rheumatology. 7th ed. Philadelphia, PA: Elsevier. 2005;1279-1308. [Google Scholar]
- 2. Mayes M. Systemic sclerosis: clinical features. In: Klippel JH, Stone JH, Crafford LJ, White PH, eds. Primer on the rheumatic diseases. 13th ed. New York, NY: Springer. 2008;343-350. [Google Scholar]
- 3. Scleroderma Society of Canada. Find a support group. http://www.scleroderma.ca/Support/Find-A-Support-Group.php. Accessed September 2, 2017.
- 4. Scleroderma Foundation. Support groups. http://www.scleroderma.org/site/PageServer?pagename=ts_support_groups#.WVKfdxPytgc. Accessed September 2, 2017.
- 5. Davison KP, Pennebaker JW, Dickerson SS. Who talks? The social psychology of illness support groups. Am Psychol. 2000;55(2):205-217. [PubMed] [Google Scholar]
- 6. Aymé S, Kole A, Groft S. Empowerment of patients: lessons from the rare diseases community. Lancet. 2008;371(9629):2048-2051. [DOI] [PubMed] [Google Scholar]
- 7. Barg FK, Gullatte MM. Cancer support groups: meeting the needs of African Americans with cancer. Semin Oncol Nurs. 2001;17(3):171-178. [DOI] [PubMed] [Google Scholar]
- 8. Kwakkenbos L, Jewett LR, Baron M, et al. The Scleroderma Patient-centered Intervention Network (SPIN) Cohort: protocol for a cohort multiple randomised controlled trial (cmRCT) design to support trials of psychosocial and rehabilitation interventions in a rare disease context. BMJ Open. 2013;3(8):e003563. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Delisle VC, Gumuchian ST, Rice DB, et al. Perceived benefits and factors that influence the ability to establish and maintain patient support groups in rare diseases: a scoping review. Patient. 2017;10(3):283-293. [DOI] [PubMed] [Google Scholar]
- 10. Delisle VC, Gumuchian ST, Peláez S, et al. Reasons for non-participation in scleroderma support groups. Clin Exp Rheumatol. 2016;34 Suppl 100(5):56-62. [PubMed] [Google Scholar]
- 11. Gumuchian ST, Delisle VC, Peláez S, et al. Reasons for not participating in scleroderma patient support groups: A cross-sectional study. Arthritis Care Res (Hoboken). 2017. Feb 19. doi: 10.1002/acr.23220. [Epub ahead of print]. [DOI] [PubMed] [Google Scholar]
- 12. Page BJ, Pietrzak DR, Lewis TF. Development of the group leader self-efficacy instrument. J Spec Group Work. 2001;26(2):168-184. [Google Scholar]
- 13. Zordan RD, Juraskova I, Butow PN, et al. Exploring the impact of training on the experience of Australian support group leaders: current practices and implications for research. Health Expect. 2010;13(4):427-440. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Butow P, Ussher J, Kirsten L, et al. Sustaining leaders of cancer support groups: the role, needs, and difficulties of leaders. Soc Work Health Care. 2005;42(2):39-55. [DOI] [PubMed] [Google Scholar]
- 15. Bandura A. Self-efficacy: the exercise of control. New York, NY: WH Freeman and Company; 1997. [Google Scholar]
- 16. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Hillsdale, NJ: Erlbaum; 1988. [Google Scholar]
- 17. Shoji K, Cieslak R, Smoktunowicz E, Rogala A, Benight CC, Luszczynska A. Associations between job burnout and self-efficacy: a meta-analysis. Anxiety Stress Coping. 2016;29(4):367-386. [DOI] [PubMed] [Google Scholar]
- 18. Demerouti E, Bakker AB, Vardakou I, Kantas A. The convergent validity of two burnout instruments. Eur J Psychol Assess. 2003;19(1):12-33. [Google Scholar]
- 19. Reis D, Xanthopoulou D, Tsaousis I. Measuring job and academic burnout with the Oldenburg Burnout Inventory (OLBI): factorial invariance across countries and samples. Burnout Research. 2015;2(1):8-18. [Google Scholar]
- 20. World Health Organization. Process of translation and adaptation of instruments. http://www.who.int/substance_abuse/research_tools/translation/en/. Accessed September 2, 2017.