Validation of the Slovenian Version of Short Sense of Coherence Questionnaire (SOC-13) in Multiple Sclerosis Patients

Abstract

Aim

To validate the Slovenian version (SOC-13-SVN) of Sense of Coherence 13-item instrument (SOC-13) in Slovenian multiple sclerosis (MS) patients.

Methods

A consecutive 134 Slovenian MS patients were enrolled in a cross-sectional study in 2013. The reliability of the SOC-13-SVN was assessed for internal consistency by Cronbach’s alpha coefficient (α), dimensionality by the confirmatory factor analysis (CFA), and criterion validity by Pearson correlation coefficient (r) between SOC-13-SVN global score and MSQOL-54 composite scores – Mental Health Composite score (MHC) and Physical Health Composite score (PHC).

Results

For the SOC-13-SVN instrument as a whole, internal consistency was high (α_total=0.88) while it was low for three subscales (α_{comprehensibility}=0.79; α_{manageability}=0.66; α_{meaningfulness}=0.69). The results of the CFA confirmed a three-factor structure with good fit (RMSEA=0.059, CFI=0.953, SRMR=0.065), however, the correlations between the factors were very high (r_{comprehensibility/manageability}=0.938; r_{comprehensibility/meaningfulness}=0.811; r_{manageability/meaningfulness}=0.930). The criterion validity analysis showed a moderate positive strength of relationship between SOC-13-SVN global score and both MSQOL-54 composite scores (MHC: r=0.597, p<0.001; PHC: r=0.437, p<0.001).

Conclusion

Analysis of some psychometric properties confirmed that this instrument is a reliable and valid tool for use in Slovenian MS patients. Despite the three-dimensional structure of the instrument, the use of the global summary score is encouraged due to the low reliability of the subscale scores and high correlations between them.

1. Introduction

Multiple sclerosis (MS) is a chronic neurological disease starting predominantly in the period of early/middle adulthood (1). It affects patients in a complex way, causing minor or greater disability (2). In MS, the effect of disability in daily living is reported to be greater in comparison to other chronic diseases (3, 4). Additionally, MS is considered as a leading cause of non-traumatic disability (e.g. sensory, motoric, coordination, balance or vision problems, cognitive disturbances, and attention/memory deficits) in young adults in Europe (5). These facts pose a challenge to clinicians in terms of how to empower MS patients for coping with their illness over the long-term. Sense of coherence (SOC), the core construct of the salutogenetic model (6, 7), developed by the Antonovsky, an Israeli American sociologist, could play an important role in dealing with the disease (7). According to Calandri et al. (8), SOC seems to mediate the adjustment to MS among recently diagnosed patients.

The SOC was originally defined by Antonovsky as “a global orientation that expresses the extent to which one has a pervasive, enduring though dynamic feeling of confidence that (i) the stimuli from one’s internal and external environments in the course of living are structured, predictable, and explicable; (ii) the resources are available to one to meet the demands posed by these stimuli; and (iii) these demands are challenges, worthy of investment and engagement” (6). In this context, he also proposed three dimensions/components of the SOC construct: comprehensibility (the ability to understand the situation), manageability (the perception of having resources to cope with the situation), and meaningfulness (the ability to find meaning in the situation) (6). To measure the SOC construct, the Orientation to Life questionnaire was developed (6). The original version consists of 29 (SOC-29), while the abbreviated version consists of 13 items (SOC-13) (6, 7). The comprehensive systematic review of Eriksson & Lindstrom (9) on more than 470 publications showed that, until 2003, the SOC questionnaires had been translated in at least 33 different languages, while a 2017 update revealed that they had been translated in another 16 languages, and used in more than 48 countries in total (7). Both instruments were validated many times in many different population groups, from general population to various groups of patients (e.g. patients with diabetes, cardiovascular diseases, cancer, rheumatoid arthritis, and schizophrenia). The studies evaluated the reliability, mostly in terms of internal consistency, as well as various aspects of validity, e.g. face, criterion, and construct validity (7, 9). The latter was mainly evaluated in terms of the instrument’s factor structure (7, 9, 10). Exploratory (EFA) and/or confirmatory factor analysis (CFA) procedures were both applied (7, 10, 11, 12, 13, 14), using SPSS software for performing EFA (10, 11), and AMOS (11, 13, 15) or Mplus software (14, 16) for performing CFA procedures, for example.

The Slovenian expert group from the Faculty of Medicine, University of Ljubljana, completed the translation/cultural adaptation of the SOC-29 (SOC-29-SVN) and the SOC-13 (SOC-13-SVN) instruments into the Slovenian language, and made them available for research purposes in 2013 (17). However, they have not been validated in any population group in Slovenia yet.

The newest epidemiological data places Slovenia among the countries with the highest MS prevalence worldwide (>100/100,000) (18). In addition, due to a long lifespan, a disability burden of the Slovenian MS patients is very high nowadays (19). Measuring the level of psychosocial dysfunction of MS patients for focused empowerment for a long-term successful coping with this chronic illness is, therefore, mandatory.

To our knowledge, the SOC instrument has not been assessed among Slovenian MS patients yet and we could not find information on the validation of SOC questionnaires in the population of MS patients in online biomedical bibliographic/full-text databases. As it is very important to know whether an instrument reliably and validly measures what it intends to measure in a specific population, the aim of the present study was to validate the SOC-13-SVN instrument with the objective of assessing some of its psychometric characteristics in the Slovenian MS patients.

2. Methods

This study was carried out in the frame of a larger research project on the impact of SOC on quality-of-life and a self-perceived health in patients with MS at the Department of Neurology of the University Clinical Centre Maribor (UCCM), Slovenia, in the period of March to December 2013 (20).

2.1. Observed Population

All members of the MS patient population, followed-up at the UCCM, which met the inclusion criteria similar to criteria in other MS quality-of-life studies (i.e. MS diagnosis established according to the McDonald’s criteria (21), age 18+ years, without MS exacerbation in the last month prior to the scheduled neurological examination, and without chronic co-morbidity), were considered eligible for participating in the aforementioned research project and, consequently, in this study (20).

2.2. Study Instrument

2.2.1. Description of the SOC 13 Instrument

The SOC-13 is an instrument with 13 items, each being scored on a seven-point scale (6) (Table 1). The values can be considered in the analysis with their original (original scoring) or reverse values (reverse scoring) (6) (Table 1). The measure given by the SOC-13 instrument is a summary score, obtained by summing the values of individual responses to all 13 items, ranging from 13–91 points, with higher scores indicating a stronger SOC.

Table 1.

Sense of Coherence 13-item instrument: items, their placement within three dimensions and scoring (6).

Item No.	Question*	Dimension	Scoring
Item_1	Do you have the feeling that you don’t really care about what goes on around you? (1=Very seldom or never to 7=Very often)	Me	R
Item_2	Has it happened in the past that you were surprised by the behaviour of people whom you thought you knew well? (1=Never happened to 7=Always happened)	C	R
Item_3	Has it happened that people whom you counted on disappointed you? (1=Never happened to 7=Always happened)	Ma	R
Item_4	Until now your life has had: (1=No clear goals or purpose at all to 7=Very clear goals and purpose)	Me	O
Item_5	Do you have the feeling that you’re being treated unfairly? (1=Very often to 7=Very seldom or never)	Ma	O
Item_6	Do you have the feeling that you are in an unfamiliar situation and don’t know what to do? (1=Very often to 7=Very seldom or never)	C	O
Item_7	Doing the things, you do every day is: (1=A source of deep pleasure and satisfaction to 7=A source of pain and boredom)	Me	R
Item_8	Do you have very mixed-up feelings and ideas? (1=Very often to 7=Very seldom or never)	C	O
Item_9	Does it happen that you have feelings inside you would rather not feel? (1=Very often to 7=Very seldom or never)	C	O
Item_10	Many people – even those with a strong character – sometimes feel like sad sacks (losers) in certain situations. How often have you felt this way in the past? (1=Never to 7=Very often)	Ma	R
Item_11	When something happened, have you generally found that: (1=You overestimated or underestimated its importance to 7=You saw things in the right proportion)	C	O
Item_12	How often do you have the feeling that there’s little meaning in the things you do in your daily life? (1=Very often to 7=Very seldom or never)	Me	O
Item_13	How often do you have feelings that you’re not sure you can keep under control? (1=Very often to 7=Very seldom or never)	Ma	O

^*Legend: *=the questions from the questionnaire are reprinted with the permission of the copyright holder; C=comprehensibility; Ma=manageability; Me=meaningfulness; O=original; R=reverse

2.2.2. Translation to Slovenian Language

The translation process was performed at the Chair of Public Health, Faculty of Medicine, University of Ljubljana, in the period of September 2012 to March 2013, after obtaining the written permission from the copyright holders of the original SOC-13. The translation was carried out by a specially established group for this task, consisting of well-qualified translators (two medical doctors, both specialists in public health, one medical nurse, and one medical sociologist, all with extensive experience in translating medical texts) and a medical student. Back-translation was carried out by a professional linguist with a university degree in English who had never seen the SOC-13 English version. The group followed all the rules of quality translation. Final solutions were accepted with a full agreement of all group members in a final SOC-13-SVN version (17).

2.3. Instrument Administration and other Data Acquisition

Participants completed the SOC-13-SVN in the presence of a neurology resident and MS nurse. Assistance in reading/writing/explanation was provided if required.

Along with the SOC-13-SVN, the socio-demographic data (gender: male, female; age; education: primary, secondary, college or higher; employment status: employed, unemployed, retired; marital status: single, married/cohabiting; area of living: rural, urban) were also collected. The clinical data, i.e. MS duration in years, a disease course (primary progressive, secondary progressive, relapsing-remitting), clinical worsening of MS in the past year prior to the neurological examination, excluding the period of 30 days prior to the examination (a relapse of relapsing-remitting type of MS or an increase of the EDSS score by 1 point in progressive type of MS; yes, no), the immunomodulatory therapy (yes, no), and the EDSS score, were extracted from the patients’ medical records.

Acceptability of the SOC-13-SVN was assessed by calculating a percentage of missing data.

2.4. Psychometric Validation

The expectation-maximization technique was used to replace the missing values (22) and the descriptive statistics were utilized to describe the study participants’ characteristics.

The instrument’s reliability was assessed using the internal consistency method. First, the Cronbach’s alpha coefficient (α) was calculated for each of the three subscales. Then, these values were combined into the reliability of the total score as described in Nunnally & Bernstein (23).

In order to assess the factor structure of the instrument, the CFA was conducted. The robust maximum likelihood estimator (MLM) was used. The criteria for the fit measures were a root mean squared error of approximation (RMSEA) <0.060, a comparative fit index (CFI) >0.950, and a standardized root mean squared residual (SRMR) <0.080 (24). Akaike information criterion (AIC) was used for model comparison.

Criterion validity was assessed by calculating the Pearson correlation coefficient between the SOC-13 summary score, the Multiple Sclerosis Quality of Life (MSQOL-54) instrument composite scores (physical (PHC) and the mental health composite (MHC) scores) (25).

The statistical analysis was performed with the SPSS software, version 21.0 (SPSS Inc., Chicago, IL, USA), except for the factor analysis in which the lavaan package (26) in the R environment (27) was used.

3. Results

3.1. Study Participants Characteristics

Out of 207 MS patients initially considered for inclusion, 57 did not meet the inclusion criteria: 55 (96.5%) had comorbidity and two (3.5%) a recent exacerbation of MS. In total, 134/150 eligible patients participated in the study (response rate: 134/150; 89.3%), while 16 refused.

Among the participants, there were 42 males (31.3%) and 92 (68.7%) females. Mean age was 43.2±11.1 years (range: 21–72 years). All other participants’ characteristics are presented in Table 2.

Table 2.

Characteristics of the multiple sclerosis (MS) patients group for validation of Slovenian version of the Sense of Coherence 13-item instrument (n=134).

Characteristic	Category	No. (%) / Median; Min–Max; Q1–Q3
Education	Primary	16 (11.9)
	Secondary	94 (70.1)
	College or higher	24 (17.9)
Employment status	Employed	63 (47.0)
	Unemployed	18 (13.4)
	Retired	53 (39.6)
Marital status	Single	44 (32.8)
	Married/cohabiting	90 (67.2)
Area of living	Rural	80 (59.7)
	Urban	54 (40.3)
Disease duration (years)		8; 0-33; 4–12.25
Disease course	Primary progressive	6 (4.5)
	Secondary progressive	23 (17.2)
	Relapsing-remitting	105 (78.4)
Clinical worsening of the disease*	No	83 (61.9)
	Yes	51 (38.1)
Immunomodulatory therapy	No	42 (31.3)
	Yes	92 (68.7)
EDSS		3; 0-8; 1.625–4.5

Legend: Q1 – the first quartile; Q3 – the third quartile; *– clinical worsening of the disease in the past year prior to the neurological examination, excluding the period of 30 days prior to the examination (a relapse of relapsing-remitting type of MS or an increase of the EDSS score by 1 point in progressive type of MS); EDSS – Expanded Disability Status Scale score

The mean SOC-13 summary score was 67.8 (13.3; min: 28; max: 91). The characteristics of the individual item values distribution are displayed in Table 3.

Table 3.

Characteristics of the distribution of values of items of the Sense of Coherence 13-item instrument in the validation study in multiple sclerosis patients (n=134).

Item	Mean	Standard Deviation	Median	Interquartile range
Item_1	5.6	1.6	6	4.44–7
Item_2	4.7	1.7	5	3–6
Item_3	4.8	1.7	5	4–6
Item_4	5.7	1.4	6	5–7
Item_5	5.3	1.7	6	4–7
Item_6	5.4	1.6	6	4–7
Item_7	5.4	1.3	6	4–6
Item_8	4.9	1.7	5	4–6.25
Item_9	5.0	1.8	6	4–6.25
Item_10	5.0	1.6	5	4–6
Item_11	5.0	1.5	5	4–6
Item_12	5.7	1.5	6	5–7
Item_13	5.0	1.7	5	4–6.25

3.2. Missing Values Analysis

The percentage of missing data was generally low. For 7 items (53.8%) there were no missing data. In the other 6 items the range of missing data was 0.7–3.0% (1 or 0.7% in 3 items, 2 or 1.5% in 1 item, 3 or 2.2% in 1 item, and 4 or 3.0% in 1 item). The highest percentage of missing data was recorded in Item_8 (detailed item description is given in Table 1).

3.3. Psychometric Validation

3.3.1. Reliability

For the instrument as a whole, internal consistency was high (α_total=0.88) while it was low for three subscales (α_{comprehensibility}=0.79; α_{manageability}=0.66; α_{meaningfulness}=0.69).

3.3.2. Factor Structure

In the CFA, three factor analysis models were defined and tested: the one-factor model, the three-factor model, and a modified three-factor model with correlated uniquenesses. In the third model, we allowed correlated residuals for the Item_2 (comprehensibility dimension) and the Item_3 (manageability dimension), as well as for the Item_4 (manageability dimension) and the Item_13 (meaningfulness dimension) (a detailed description of the items is given in Table 1). The statistical properties of these three models are presented in Table 4. The first two models did not fit well, although the fit of the three-factor model was slightly better compared to the one-factor model. The former model was also to be preferred according to AIC. However, the modified three-factor model exhibited a good fit and was to be clearly preferred according to AIC (Table 4).

Table 4.

Comparison of three factor analysis models in the Slovenian version of the Sense of Coherence 13-item instrument validation study in multiple sclerosis patients (n=134).

Item	χ2	df	p	AIC	RMSEA	CFI	SRMR
1-factor	190.63	65	<0.001	6055.38	0.120	0.786	0.084
3-factor	177.10	62	<0.001	6045.75	0.118	0.804	0.083
Modified 3-factor	87.68	60	0.011	5931.90	0.059	0.953	0.065

Legend: AIC=Akaike information criterion; RMSEA=root mean squared error of approximation, CFI=comparative fit index; SRMR=standardized root mean squared residual

The Table 5 presents raw (with standard errors) and standardized factor loadings for the modified three-factor model. All loadings were reasonably high, although some items appeared to be better measures of their respective constructs. The correlations between the factors were very high: 0.938 between comprehensibility and manageability dimensions, 0.811 between comprehensibility and meaningfulness dimensions, and 0.930 between manageability and meaningfulness dimensions.

Table 5.

Factor loadings for the final model in the Slovenian version of the Sense of Coherence 13-item instrument validation study in multiple sclerosis patients (n=134).

Item	Item	Loading (SE)	Standardized loading
Comprehensibility	Item_2	0.804 (0.148)	0.476
	Item_6	1.185 (0.126)	0.726
	Item_8	1.451 (0.107)	0.834
	Item_9	1.370 (0.125)	0.763
	Item_11	0.730 (0.140)	0.505
Manageability	Item_3	0.693 (0.140)	0.421
	Item_5	0.820 (0.149)	0.479
	Item_10	0.740 (0.163)	0.467
	Item_13	1.325 (0.112)	0.787
Meaningfulness	Item_1	0.506 (0.134)	0.319
	Item_4	0.890 (0.124)	0.658
	Item_7	0.806 (0.105)	0.623
	Item_12	1.266 (0.142)	0.855

3.3.3. Criterion Validity

The analysis showed a moderate positive strength of relationship between SOC-13 score and both MSQOL-54 composite scores (MHC score: r=0.597; PHC score: r=0.437). In both cases, the association was highly statistically significant (p<0.001).

4. Discussion

Based on the results of this study, we can conclude that the SOC-13-SVN successfully passed the evaluation for cultural equivalence as well as fulfilled the necessary psychometric criteria for being used in the Slovenian MS patients’ population.

The results of the reliability analysis are consistent with the results reported in other similar studies. In particular, the reliability of the total score obtained in our study is in the upper part of the values range of this measure obtained in other similar studies (range 0.70–0.93) (9, 10, 11, 12, 15, 16, 28, 29, 30, 31, 32, 33, 34). Taking the small number of items into account, it can be considered to be reasonably high and close to the value recommended when making decisions about individuals (23).

Analysis of the factor structure has confirmed a three-factor structure of the SOC-13-SVN with good fit. The multidimensionality shown in our study is consistent with the results of a systematic review of Eriksson & Lindstrom (9), who concluded that the SOC seems to be a multidimensional construct. According to that review, factor analysis in a few studies confirmed the unidimensional model, while in others this failed, and two-factor, three-factor, and five-factor models of structure were found (9). However, our three-factor model was modified. Correlated residuals were allowed for two pairs of items, since the items in both pairs have something in common, regardless of whether they belong to different dimensions. In the first pair (Item_2, Item_3), both items address participants’ expectations about people who could help them in distress, while in the second pair (Item_4, Item_13), both items are focused on the management of life situations. These results are in line with Antonovsky who stated that items, although theoretically pertaining to one dimension, share elements with items from other dimensions (6). Despite the three-dimensional structure, the use of the single total score is encouraged in our study. The first reason is the high correlations between dimensions and low reliability of the subscale scores were found in our study and the second that the one-factor model was advocated by Antonovsky himself, since the questionnaire was not intended to measure dimensions individually (6, 28).

The criterion validity results were also consistent with the results of other similar studies, which used quality-of-life instruments for assessing this aspect of validity (range 0.51–0.77) (9).

Finally, if we make a rough comparison of the SOC-13 summary score mean value obtained in our MS patients, this almost coincides with the results of the only similar study that we found, i.e. the study of Broersma et al. (67.5±13.3) (35).

This study has some limitations. First, a relatively small number of participants were included in the present study, however, the number was still sufficient to permit fair conclusions. Next, one could argue that no method of measurement of stability of the instrument over time, e.g. the test-retest method, was used in the present study. However, the reliability of any patient-reported outcome measure can be evaluated using measurement stability methods and/or measurement equivalence methods. The later were developed in the social science research for the situations in which it is not possible to perform repeated measurements because the measured phenomenon changes or could change over time (36). As we assumed, based on results of previous studies (37, 38, 39), that the phenomenon measured in our study could change over time so, due to specificities of the observed group, only the measures of equivalence were used (36). Finally, not all aspects of validity were analysed in this study, however, we decided to report only usually reported results as in similar studies (9, 10, 11, 12, 15, 16, 28, 29, 30, 31, 32, 33, 34).

The study also has some important strengths. The most important is that this study provided novel knowledge about the psychometric properties of SOC-13 instrument when evaluated in MS patients. Given the results of this study, MS patients could join a number of population groups and settings in which the SOC is or was assessed (7). Moreover, new opportunities are opening toward a more personalized medicine approach in terms of integrating health promotion approaches (i.e. by using SOC for increasing/strengthening interventions (40, 41)) for disease management in MS patients.

There are still many challenges in researching the use of SOC-13 in MS patients. It is necessary first to check the dynamics/stability of the SOC in time, especially in those subgroups with more rapidly evolving and/or a more severe form of MS, as well as in those with comorbidity. In the latter group, the SOC has to a certain extent already been studied in MS patients with depressive symptoms (42). Additionally, with a focus on studying the properties of the SOC-13-SVN, further evaluation is needed. Our work can be continued by working on a larger dataset and analysing additional aspects of validity.

5. Conclusion

The rigorously performed translation process provided a good quality translation of the SOC-13 to Slovenian language. Analysis of its psychometric properties proved that this instrument is a reliable tool for use in Slovenian MS patients.