Stroke Impact Scale 3.0: Reliability and Validity Evaluation of the Korean Version

Abstract

Objective

To establish the reliability and validity the Korean version of the Stroke Impact Scale (K-SIS) 3.0.

Methods

A total of 70 post-stroke patients were enrolled. All subjects were evaluated for general characteristics, Mini-Mental State Examination (MMSE), the National Institutes of Health Stroke Scale (NIHSS), Modified Barthel Index, Hospital Anxiety and Depression Scale (HADS). The SF-36 and K-SIS 3.0 assessed their health-related quality of life. Statistical analysis after evaluation, determined the reliability and validity of the K-SIS 3.0.

Results

A total of 70 patients (mean age, 54.97 years) participated in this study. Internal consistency of the SIS 3.0 (Cronbach's alpha) was obtained, and all domains had good co-efficiency, with threshold above 0.70. Test-retest reliability of SIS 3.0 required correlation (Spearman's rho) of the same domain scores obtained on the first and second assessments. Results were above 0.5, with the exception of social participation and mobility. Concurrent validity of K-SIS 3.0 was assessed using the SF-36, and other scales with the same or similar domains. Each domain of K-SIS 3.0 had a positive correlation with corresponding similar domain of SF-36 and other scales (HADS, MMSE, and NIHSS).

Conclusion

The newly developed K-SIS 3.0 showed high inter-intra reliability and test-retest reliabilities, together with high concurrent validity with the original and various other scales, for patients with stroke. K-SIS 3.0 can therefore be used for stroke patients, to assess their health-related quality of life and treatment efficacy.

INTRODUCTION

Stroke is a leading cause of mortality, morbidity and disability worldwide, affecting quality of life, especially among the elderly [1,2]. Patients with stroke frequently remain with the sequelae which may include physical, cognitive, and emotional problems, as well as issues with social reintegration [3] and life satisfaction [4]. Assessing impairment and disability alone does not provide sufficient information regarding the absolute impact of stroke [5]. Although functional outcome measures are important in stroke rehabilitation [5], health-related quality of life is increasingly used to assess the status of stroke survivors. The proportion of stroke survivors in Korea is expected to increase rapidly with population aging.

The Stroke Impact Scale (SIS) is widely used to identify patient need, and to inform and evaluate rehabilitation interventions. Therefore, much research has been done to determine the reliability and validity of SIS in other countries [6,7]. So far, no research has been conducted to investigate the reliability and validity of the Korean version (K-SIS 3.0) among the hospital-based stroke survivors. The aim of this study was to establish K-SIS 3.0 using translation and back-translation, to determine its validity and reliability in the evaluation of stroke patients. The SIS, developed by Duncan et al. [8] at the University of Kansas Medical Center, provides a multi-dimensional, self-reported measure of health-related quality of life of stroke survivors. The SIS assesses 8 domains of health, including strength, hand function, memory, communication, emotion, and social participation. Scores for each domain range from 0 to 100, with higher scores indicating a better quality of life. The SIS is highly sensitive in assessing the motor function in patients with stroke who have been classified as ‘minimal’ or ‘no disability’ [9]. This is the first study to establish the reliability and validity of the K-SIS 3.0, using the translation and back-translation procedure.

MATERIALS AND METHODS

Translation procedures

Translation emphasizes the comprehensibility and appropriateness of the language in the Korean cultural context. The process of translating the SIS 3.0 into Korean included the following steps: The SIS 3.0 was translated into Korean by two independent native Korean speakers with excellent knowledge of English, and organized by a translation committee which comprised two rehabilitation medical doctors and a psychologist. The first K-SIS 3.0 was agreed by consensus. This version was back-translated into English by a Korean native with good knowledge of English, who did not have access to the original version of the SIS. The first back-translation was compared with the original version, and modifications were made to eliminate all discrepancies between the original and the back-translated versions (Fig. 1).

Fig. 1. Translation and back translation procedure. SIS, Stroke Impact Scale.

Procedures

This study included 70 stroke patients hospitalized at the National Rehabilitation Center of Korea. Patients were evaluated using the Mini-Mental State Examination (MMSE), the National Institutes of Health Stroke Scale (NIHSS), the Modified Barthel Index (MBI), and the Hospital Anxiety and Depression Scale (HADS). Health-related quality of life was evaluated with the Short Form Health Survey (SF-36), and the K-SIS 3.0. The subjects were evaluated twice (test interval, 15 days) to establish the test-retest reliability of K-SIS.

Inclusion criteria were (1) stroke (subacute, chronic) diagnosed on the basis of clinical and radiologic findings (brain computed tomography and/or magnetic resonance imaging); (2) age greater than 20 years; (3) hospitalization at the National Rehabilitation Center of Korea; and (4) sufficient cognitive ability to understand the questionnaire. The following criteria were excluded from the study: (1) severe disability caused by a previous neurologic disorder; (2) comorbidity with severe systemic illness (e.g., cancer, autoimmune disease); and (3) severe pre-existing neurological deficits.

Assessment

The MMSE measures the cognitive functions, including memory, orientation to place and time, naming, reading, copying (visuospatial orientation), writing, and the ability to follow a 3-stage command. This instrument has 19 items, with scores ranging from 0 to 30 points [10].

NIHSS is a 15-item stroke scale that provides a quantitative measure of the components of a neurologic examination. The NIHSS evaluates the patient's level of consciousness, external ocular movements, visual fields, facial palsy, strength of extremities, coordination, sensory loss, language (aphasia), dysarthria, extinction, and inattention. The score ranges from 0 to 42, with higher scores indicating greater impairment [11]. The MBI has been widely used to evaluate the activities of daily living (ADL) in stroke patients. The MBI measures 10 items of ADL related to self-care and mobility: control of bowels and bladder, grooming, toileting, feeding, transfer, mobility, dressing, stairs, and bathing. The MBI score ranges from 0 to 100, with lower scores indicating greater dependency. The validity and reliability of this tool is well established [12]. The HADS consists of 7 items for the assessment of anxiety, and 7 for depression, with each item scored from 0 (no problem) to 3 (severe problem). Scores on individual items are summed to calculate a score for anxiety (HADS-A) and for depression (HADS-D). Scores over 11 points on a subscale are indicative of mood disturbance [13]. The SF-36 is a generic health-related quality of life measure that includes 8 domains for the assessment of physical functioning, role limitations due to physical problems, bodily pain, general health perception, vitality, social functioning, role limitations due to emotional problems, and emotional well-being. Scores for each domain range from 0 to 100, where higher values indicate better functioning. We used the Korean version of SF-36 [14]. The SIS 3.0 is a 59-item self-report assessment of stroke outcome, used to assess health-related quality of life. The SIS has 8 domains: strength, hand function, mobility, physical and instrumental activities of daily living (ADLs and IADLs), memory and thinking, communication, emotion, and social participation. Scores for each domain range from 0 to 100, where higher scores indicate a better health-related quality of life. The strength items are rated in terms of strength; memory, communication, ADLs/IADLs, mobility, and hand function items in terms of amount of difficulty; and emotion and social participation items in terms of frequency. Four of the subscales (strength, hand function, ADLs/IADLs, and mobility) are combined as a physical domain. The SIS 3.0 also includes a question to assess the patient's global perception of recovery. The respondent is asked to rate his or her percentage of recovery on a visual analog scale of 0 to 100, where lower scores indicate lower perceived recovery [15].

Statistical analysis

Data analysis was carried out using SPSS ver. 15.0 for Windows (SPSS Inc., Chicago, IL, USA). The data were analyzed in a descriptive as well as inferential manner. The categorical variables were presented as absolute values and percentages, and the numeric variables as means and standard deviations. Univariate analysis was first carried out between each predictor and the SIS domain to identify the variables included in the multivariate model. Interclass correlation coefficient (Spearman's rho) was calculated for the test-retest reliability analysis. Internal consistency was analyzed with Cronbach's alpha test. A value of p less than 0.05 was considered statistically significant.

RESULTS

A total of 70 patients, having mean age 54.97±14.96 years, participated in the study. Demographic and clinical characteristics of the study population are shown in Table 1.

Table 1. Baseline and demographic characteristics of patients.

Values are presented as mean±standard deviation (range) or number of patients (%).

The 9 domains of K-SIS 3.0 were evaluated twice; the second follow-up was after 14 days, to assess the test-retest reliability (Table 2). For all domains examined, the average is higher than the first assessment.

Table 2. Scores of Korean version of the SIS 3.0.

Values are presented as mean±standard deviation (range).

SIS, Stroke Impact Scale; ADL, activities of daily living; I-ADL, instrumental activities of daily living.

Reliability of the SIS 3.0

Internal consistency of the SIS 3.0 (Cronbach's alpha) was obtained, and all domains had good co-efficiency, above the threshold of 0.70 (stroke recovery could not be evaluated as it is a 1-item scale) (Table 3). Test-retest reliability of SIS 3.0 resulted from the correlation (Spearman's rho) of the same domain scores obtained on the 1st and 2nd assessments. Results were above 0.5, with the exception of the social participation and mobility (Table 4).

Table 3. Cronbach's alpha of Korean version of the SIS 3.0.

SIS, Stroke Impact Scale; ADL, activities of daily living; I-ADL, instrumental activities of daily living; NA, not applicable (due to 1 domain category).

Table 4. Test-retest coefficients of the Korean version of the SIS 3.0.

SIS, Stroke Impact Scale; ADL, activities of daily living; I-ADL, instrumental activities of daily living.

^*p<0.001 (derived from correlation analysis).

Validity of the SIS 3.0

Concurrent validity of K-SIS 3.0 was assessed using the SF-36 compared to the same or similar domains. The strength domain of K-SIS 3.0 had a positive correlation with physical function (Spearman's rank correlation coefficients, ρ=0.356), bodily pain (ρ=0.318), general health (ρ=0.281), and physical domain (ρ=0.401) of the SF-36. Hand function domain of K-SIS 3.0 correlated with physical function (ρ=0.283), general health (ρ=0.353), and mental health domain (ρ=0.416) of the SF-36. Mobility domain of K-SIS 3.0 had a positive relationship with physical roles (ρ=0.361), general health (ρ=0.480), emotional roles (ρ=0.528), and mental domain of SF-36 (ρ=0.612). The memory domain of K-SIS 3.0 correlated with mental health (ρ=0.237) and physical domains of the SF-36 (ρ=0.534). Social participation correlated with physical function (ρ=0.378), physical roles (ρ=0.335), and physical domain (ρ=0.389). The stroke recovery domain of K-SIS 3.0 had a positive correlation with general health (ρ=0.442), role physical (ρ=0.377), vitality (ρ=0.329), and physical domains of the SF-36 (ρ=0.488). A weak association was observed between the emotional domain of K-SIS 3.0 and emotional roles of SF-36 (Table 5). Concurrent validity was observed between K-SIS 3.0 and other scales with the same or similar domains. The strength (ρ=0.370), hand function (ρ=0.556), ADL/I-ADL (ρ=0.628), and mobility domains (ρ=0.613) of K-SIS 3.0 correlated with MBI. Hand function, mobility, ADL/IADL, memory, communication, emotion, and stroke recovery domains of K-SIS 3.0 also correlated with HADS-A and HADS-D. Mobility (ρ=0.352) and ADL/I-ADL domains (ρ=0.247) of K-SIS 3.0 were correlated with MMSE. All domains of K-SIS 3.0, except the emotion and memory domains, correlated with NIHSS (Table 6).

Table 5. Correlations between Korean version of SIS and SF-36.

SIS, Stroke Impact Scale; SF-36, Short Form 36 Health Survey; ADL, activities of daily living; I-ADL, instrumental activities of daily living.

^*p<0.05, ^**p<0.001.

Table 6. Correlation between Korean version of the SIS 3.0 and the other scales.

SIS, Stroke Impact Scale; MBI, Modified Barthel Index; MMSE, Mini-Mental State Examination; HADS-A, Hospital Anxiety and Depression Scale-A; HADS-D, Hospital Anxiety and Depression Scale-D; NIHSS, National Institutes of Health Stroke Scale; ADL, activities of daily living; I-ADL, instrumental activities of daily living.

^*p<0.05, ^**p<0.01.

DISCUSSION

The Cronbach's alpha is an important measurement tool, since it assesses the extent of correlation between questionnaires, and their individual correlation with the overall results of the study. We found that the Cronbach's alpha of each domain was high, indicating acceptable internal consistency. Barring emotion (0.719), all domains were greater than or equal to 0.8, which demonstrates a high level of internal consistency, when compared with the Brazilian version of SIS 3.0 (Cronbach's alpha=0.49) [7].

To establish the test-retest reliability of K-SIS 3.0, the correlation (Spearman's rho) of the same domain scores from the initial test and that performed after 2 weeks was obtained; all results were above 0.5, except for social participation and mobility domains. Insufficient reliability of the social participation domain was expected, owing to the characteristics of the hospitalized patients. The results were similar to the findings of studies performed with similar inclusion criteria of the Italian and Brazilian SIS 3.0 [6]. Further evaluation is required to confirm the test-retest reliability of the social participation domain in an outpatient setting. Reliability of the mobility domain of the SIS 3.0 was also insufficient, mainly due to our fall prevention policy, which prohibits self-gait training without the supervision of a physical therapist in patients who are at a high risk for falls. The patient risk was assessed using the Johns Hopkins Fall Risk Assessment Tool [16].

Concurrent validity was observed between K-SIS 3.0 and the SF-36 with the same or similar domains. The strength domain of K-SIS 3.0, and the physical function and physical domain of SF-36, were strongly correlated. The mobility domain of K-SIS 3.0 was correlated with physical and emotional domains of SF-36. The stroke recovery index of K-SIS 3.0 also correlated with physical function, physical domain, role physical, and vitality domains of SF-36, demonstrating that recovery was considered as improvement in the physical status. There was no statistical significance between emotional domain of K-SIS 3.0 and SF-36, which was possibly due to use of antidepressant. Additional measures of emotion are required for adequate conclusion.

Concurrent validity was also observed between K-SIS 3.0 and the other scales with the same or similar domains (MMSE, MBI, and NIHSS). Correlation between all domains of K-SIS 3.0 and MBI was confirmed, except for communication and emotion domains, which were not included in MBI categories. Among the MBI categories, three domains, which are considered as important factors that enhance independence, showed significant correlation with K-SIS 3.0: hand function (ρ=0.556), mobility (ρ=0.613), and ADL/I-ADL (ρ=0.628). The emotion and stroke recovery domains of K-SIS 3.0 also correlated with HADS, indicating that neuropsychiatric problems (such as depression and insomnia) are important factors of health-related quality of life.

There was no significant correlation between the memory domain of K-SIS 3.0 and the MMSE. Mean value of the MMSE score was 26.70±3.31, which could have a ceiling effect on the results. Therefore, additional evaluations are needed for the cognitive range variable.

We acknowledge several limitations of our study. First, we were not able to determine the validity and reliability of the SIS 3.0 among outpatients. Further studies should consider recruiting outpatient stroke survivors. Second, confounding factors influencing health-related quality of life, such as environmental factors, family support, and rehabilitation services were not included in this study. These should be considered in future studies.

In conclusion, the newly developed K-SIS 3.0 showed high inter-intra reliability, and test-retest reliabilities, together with high concurrent validity with the original and other scales for patients with stroke. K-SIS 3.0 can therefore be used among stroke patients to assess health-related quality of life and treatment efficacy. Further research is required to determine the reliability of the social participation and mobility domains among outpatients.