The Spinal Cord Injury Spasticity Evaluation Tool: A Persian adaptation and validation study

Abstract

Objective: To adapt the Spinal Cord Injury Spasticity Evaluation Tool (SCI-SET) into the Persian language (SCI-SETp) and to examine the reliability and validity of the SCI-SETp in patients with spinal cord injury (SCI).

Design: A cross-sectional and prospective cohort validation study.

Setting: University Neurological Physiotherapy Clinic.

Participants: Adult patients with SCI.

Main Outcome Measures: SCI-SET.

Results: There was no missing data. No floor or ceiling effect was observed. Cronbach's α coefficient was 0.862. Factor analysis suggested 1 factor structure (Eigenvalue = 8.49) explained 24.27% of the total variance. The ICC_agreement for test-retest reliability was 0.84. The standard error of measurement and the smallest detectable change was 0.30 and 0.82, respectively. The divergent relationships demonstrated the SCI-SETp uniqueness construct.

Conclusion: The results support the reliability and validity of the SCI-SETp for assessing the impact of spasticity on daily life of patients with SCI.

Introduction

Spinal cord injury (SCI) is a major health and costly condition that annually affects about 40 million people worldwide and 2.1 to 130.7 million in developing countries.^1,2 The most common causes of SCI are motor-vehicle crashes, falls, gunshot injuries, knife injuries, and sports/recreation activities.^1,2 The SCI often leads to serious disability and poor quality of life (QoL) which is one of the main burdens to society with significant emotional, economic, and social consequences for the patients and their families. The SCI can cause paralysis, sensory loss, and significant long-term medical complications including spasticity, pain, pressure sores, and fatigue.³

Spasticity is a common problem and one of the most debilitating health complications after SCI affecting 65–78% of individuals with SCI.^4,5 Spasticity has been classically defined as a motor disorder with velocity dependent increase in muscle tone resulting from hyperexcitability of the stretch reflex, as one component of the upper motor neuron syndrome⁶ but is recently redefined to include other positive features of involuntary muscle activation including clonus and spasms.⁷ Spasticity may have an important and profound influence on QoL of patients with SCI. The decision to treat spasticity depends on its accurate assessment of spasticity with reliable and valid instruments.

There are various clinical, biomechanical, and neurophysiological approaches to measure the severity of spasticity.^8–11 However, they are examiner based methods and do not consider the patient's experience of spasticity. Recently, a condition-specific subjective measure has been developed by Adams et al (2007) named the Spinal Cord Injury Spasticity Evaluation Tool (SCI-SET) which assesses the spasticity impact on various activities of daily life after SCI taking the patient's experience of spasticity into consideration.¹² The SCI-SET is a 7-day recall self-report questionnaire taking into account the positive and negative effects of spasticity on daily life in patients with SCI. The SCI-SET used in clinical investigations with SCI persons^13,14 has been demonstrated reliable and valid in patients with chronic SCI.¹²

The SCI-SET is a validated instrument developed in English language.¹⁵ In order to be used in non-English countries, the SCI-SET must be cross-culturally adapted to ensure that the translated version is equivalent semantically and conceptually to the original language. There are no reports available on the use of SCI-SET in other languages including Persian. Hence, the purpose of the present study was to translate and adapt the original SCI-SET to Persian language, and to evaluate its reliability and validity in a sample of Iranian Persian-speaking patients with SCI.

Methods

The SCI-SET was cross-culturally adapted to Persian language following steps recommended in guidelines for translation of self-report health questionnaires¹⁶ and used previously.¹⁷ Step І: two professional translators with Persian as their mother tongue and one of them with medical background, independently translated the SCI-SET to Persian language labeled as T1 and T2. Step ІІ: The expert committee with both translators discussed and produced a consensus Persian SCI-SET labeled as T12. Step ІІІ: another two translators who had no medical background and were unaware of the questionnaire concept independently translated T12 into English producing two back translations of the T12 questionnaire to identify any semantic or conceptual differences between the English and Persian versions. Step ІV: the expert committee compared the back translations with the original English SCI-SET considering semantic, idiomatic, experiential and contextual aspects to identify errors as well as reviewed all documents including the forward translations, and T12 version. An agreement was reached by the members of the Committee and the pre-final Persian SCI-SET was produced. Step V: the pre-final Persian SCI-SET was administered to a sample of 30 patients with SCI to test for face and content validity. Patients were instructed to give their comments and suggestions about each question with regard to the content, wording clarity, and relevance. The feedback from patients showed relevance and clarity of the SCI-SET questions, and the final Persian SCI-SET (SCI-SETp) produced and went under further psychometric evaluations. The SCI-SETp has been displayed in the Appendix.

Study participants

Patients with SCI were included in this cross-sectional and prospective cohort study with the following criteria: a) SCI duration of at least 6 months; b) age 18 years or older; and c) able to read and write Persian. In this study 100 patients were included as recommended an appropriate size for reliability and validity analyses.¹⁸ The approval was obtained from the Review Board, School of Rehabilitation, Tehran University of Medical Sciences (TUMS) and the Tehran University of Medical Sciences Ethics Committee. All patients gave their written informed consent before study initiation.

Procedure

Study participants were recruited through the Rehabilitation clinics in Tehran, Iran, and Tehran Spinal Cord Injury Association. The SCI-SETp was administered to SCI patients fulfilling the inclusion criteria. Each patient also underwent assessment with the Persian Functional Independence Measure (PFIM) for construct validity.¹⁷ We hypothesized a non-significant correlation between the SCI-SETp and the PFIM-Motor subscale or the PFIM-cognitive subscale for divergent construct validity. A sample of 50 patients out of 100 was re-administered the SCI-SETp, 7 days after the first administration, to evaluate test-retest reliability.

The SCI-SETis a self-administered instrument consisting of 35 questions to evaluate the impact of spasticity on daily life in people with SCI. Patients were asked to recall their past 7 days when rating spasticity on a scale ranging from –3 (extremely problematic) to +3 (extremely helpful). The SCI-SET total score was computed by summing all the responses from the applicable items then dividing the sum by the number of applicable items and ranges from –3 to +3.¹²

The Persian version of the FIM¹⁷ was used to assess the construct validity. The FIM is an 18-item scale to grade the level of cognitive and physical assistance necessary for function. Item scores range from 1 (total assistance) to 7 (total independence). The individual item scores were summed to compute a total score ranging from18 (lowest) to 126 (highest). Two summary scores [motor subscale (mobility/self-care) and cognitive subscale (communication/cognition)] were then derived with total possible scores of 91 and 35, respectively. Higher scores indicate greater functional independence. The motor subscale and the cognitive subscale total scores were used for divergent construct validity analyses, respectively.

Statistical analysis

All data analyses were performed using SPSS for Windows (Version 17: SPSS Inc., Chicago, IL, USA) to determine the subjects’ characteristics and to calculate descriptive statistics for all the variables; categorical and continuous variables as counts (n) and means [standard deviations (SD)], respectively. The acceptability of the Persian SCI-SET was assessed by computing percent missing responses to items of the questionnaire. The content validity and responsiveness of the SCI-SETp were evaluated by evaluating the distribution of the scores, and ceiling or floor effects for the total score. To determine the internal consistency reliability, the Cronbach's α coefficient was used with a coefficient of 0.7 considered as acceptable. The item-total correlation (ITC) was used to examine the internal structure of the SCI-SETp; correlation value of at least 0.3 was interpreted satisfactory. Factor structure was analyzed from extraction method of principal component analysis (PCA) with varimax rotation, and the scree plot inflection, Eigenvalue >1.0, and variance >10% were considered as a-priori requirements for factor extraction.¹⁹ Test-retest reliability was assessed by obtaining the ICC_agreement (two-way random effects mode, absolute measure, single rater). ICC value of 0.70 was regarded a minimum standard for reliability. The SEM (σ√1-ICC) and the SDC (1.96 × SEM × √2) were computed as the absolute reliability measures. Construct validity was examined by assessing the relationships between the SCI-SETp with the PFIM using Pearson's correlation coefficients. Our hypothesis was that the SCI-SETp should demonstrate poor correlations with the FIM-Motor subscale and the FIM-Cognitive subscale.

Results

Participants characteristics

A total of 100 eligible patients (male/female58/42; mean age 39.0 ± 11.0 years, range 20.0–69.0; duration since SCI 14.4 ± 11.5years, range 0.7–54.0) participated in this study. The most common cause of SCI was motor-vehicle crashes (n = 49). Forty-nine participants had complete SCI (American Spinal Injury Association (ASIA) impairment grade A). Demographic characteristics of the patients are presented in Table 1.

Table 1.

Characteristics of the patients (n = 100)

Characteristics	Frequency
Education level
Diploma and under	69
University degree	31
SCI† reason
Motor-vehicle crashes	49
Fall	16
Other	35
Level of injury
Cervical	28
Thoracic	38
Lumbar	34
ASIA‡
A	49
B	18
C	25
D	8
E	0

*SD, standard deviation; ^†SCI: spinal cord injury; ^†ASIA, American Spinal Injury Association

Translation and adaptation

There was no difficulty in the process of translation and cross-cultural adaptation of the SCI-SETp. Patients had no problem in filling in the questionnaire and favorably accepted and understood the SCI-SETp. No missing data were found, and patients responded to all items of the questionnaire. The items of the SCI-SETp were clear and relevant as interpreted by SCI patients.

Floor or ceiling effects

Table 2 summarizes the mean, SD, and scores distribution of the SCI-SETp and PFIM. The SCI-SETp had no floor and ceiling effects and total scores ranged from –2.90 to 0.62.

Table 2.

Descriptive statistics and score distributions for SCI-SETp* and PFIM^† (n = 100)

	Mean	SD‡	Minimum	Maximum
SCI-SETp Total score	–1.04	0.78	–2.90	0.62
PFIM-Motor subscale	50.86	20.76	13.00	89.00
PFIM-Cognitive subscale	34.70	1.80	22.00	38.00
PFIM Total score	85.56	21.02	40.00	124.00

*Persian SCI-SET; ^† Persian Functional Independence Measure; ^‡ SD, standard deviation

Internal consistency

The Cronbach's α coefficient for SCI-SETp was 0.862 which was greater than the predetermined cut-off of 0.7. If an item was deleted, the α values ranged from 0.853–0.870 and α did not fall or rise substantially. The ranges of corrected item-total correlations were from –0.009 (item 19) to 0.576 (item 2). Most corrected item-total correlations were greater than the predetermined cut-off of 0.3 with the exception of items 8 (your small hand movements (writing, use of computer, etc, ITC 0.195), 11(your enjoyment of social outings, ITC 0.253), 12 (your ability to stand/weight-bear, ITC 0.200), 13 (your walking ability, ITC 0.108), 19 (your power wheelchair use, ITC –0.009), and 24 (your sex life, ITC 0.246) which did not meet this minimal cut-off, however, α after item deletion for each of these items remained relatively unchanged (range 0.861–0.870) confirming that all item scores were related to the overall score.

Factor analysis

The correlation matrix and sampling adequacy for the SCI-SETp was appropriate for PCA from the values of Kaiser-Meyer-Oklin (KMO) (0.68) and Barlett's test of Sphericity (χ² = 1849.792, df = 595, P < 0.001). The PCA extracted 10 components with the Eigenvalues >1.0 that accounted for 69.98% of the total variance. However, nine factors with an Eigenvalue >1.0 each accounted for <10% of variance and consequently were not extracted. Visual examination of the scree plot suggested the retention of one factor consistent with the above findings (Figure 1). Therefore, when all three a-priori criteria were applied to extract factors, one-dominant factor solution (Eigenvalue = 8.49) was considered to be satisfactory for the SCI-SETp accounted for 24.27% of the total variance (Table 3).

Figure 1.

Scree plot from the principal component analysis indicates a one-factor solution for the SCI-SETp

Table 3.

One-dominant Factor with Eigenvalue 8.494 and 24.27 of total variance extracted for SCI-SETp. Factors with Eigenvalues less than 1 are not shown

Factor	Initial Eigenvalues			Extraction Sums of Squared Loadings			Rotation Sums of Squared Loadings
	Total	% of variance	Cumulative %	Total	% of variance	Cumulative %	Total	% of variance	Cumulative %
1	8.494	24.269	24.269	8.494	24.269	24.269	3.427	9.792	9.792
2	3.124	8.926	33.196	3.124	8.926	33.196	3.224	9.213	19.005
3	2.486	7.102	40.298	2.486	7.102	40.298	3.091	8.832	27.837
4	1.982	5.664	45.962	1.982	5.664	45.962	2.602	7.434	35.271
5	1.738	4.965	50.927	1.738	4.965	50.927	2.458	7.022	42.293
6	1.613	4.609	55.536	1.613	4.609	55.536	2.204	6.298	48.590
7	1.378	3.938	59.474	1.378	3.938	59.474	2.064	5.898	54.488
8	1.295	3.700	63.174	1.295	3.700	63.174	2.019	5.769	60.257
9	1.250	3.572	66.746	1.250	3.572	66.746	2.011	5.746	66.003
10	1.130	3.229	69.975	1.130	3.229	69.975	1.390	3.971	69.975

Test-retest reliability

Test-retest reliability for the SCI-SETp total scores was excellent (ICC_agreement = 0.84, 95% CI 0.74–0.91, P < 0.001).

The SEM and SDC

Absolute reliability measures of the SEM and the SDC for SCI-SETp were 0.30 (CI 95% =  ± 0.59) and 0.82, respectively (Table 4).

Table 4.

Results of test-retest reliability and absolute reliability measures for the Persian SCI-SET total score (n = 50)

	Mean ± SD
Scale	Test	Retest	d (SD)	ICCagreement(95% CI)	SEM	SDC
SCI-SETp	–1.02 ± 0.74	–1.05 ± 0.73	0.03 (0.41)	0.84 (0.74–0.91)	0.30	0.82

SD, standard deviation; d, mean difference of the test and retest scores; SEM, standard error of measurement; SDC, smallest detectable change.

Construct validity

Pearson correlation test performed to assess the level of construct validity of the SCI-SETp did not find statistically significant positive correlation between the SCI-SETp and the PFIM-Motor subscale (r = 0.14, P = 0.18) or the PFIM-Cognitive subscale (0.13, P = 0.20).

Discussion

This study presents the first report on the translation and cultural adaptation of the SCI-SET to another language. In the present study, the SCI-SET was translated and cross-culturally adapted to Persian language and assessed reliability and validity of the SCI-SETp when completed by patients with SCI. The results of this study show that the SCI-SETp is a reliable and valid instrument for evaluating the impact of spasticity on daily life in patients with SCI, with psychometric properties in agreement with the original English version.

Adaptation and acceptability

The process of forward and back translation for the development of the SCI-SETp to ensure semantic and conceptual equivalence of the SCI-SETp to the original version was proceeded without difficulties. An agreement among the translators and expert committee members was easily found in terms of wording and style. No major problems were identified in the pilot testing phase for evaluating the pre-final version reflected in responding easily to all questions of the SCI-SETp. Feedback from patients about the content adequacy and wording clarity of SCI-SETp confirmed the usefulness and acceptability of the SCI-SETp and supported its face and content validity.

In further evaluating the psychometric properties of the SCI-SETp, patients again responded to all questions (response rate 100%) and no missing answers were found indicating an excellent completeness of item response. This indicates the acceptability and feasibility of the SCI-SETp as determined from the completion of the all questionnaire items by the respondents. The acceptability of SCI-SETp is in agreement with the original English version of SCI-SET.¹²

Floor and ceiling effects

Distribution of scores for SCI-SETp consistent with those from the original English version (–0.65 ± 0.56, range –2.35–0.00)¹² was well ranged without any floor or ceiling effect. Neither a ceiling nor floor effect was identified as no patient had a maximal or minimal SCI-SETp score. No floor or ceiling effect identified in this study reflects the ability of SCI-SETp in the meaningful detection of changes (deterioration or improvement) in patient's condition. Excellent completeness of item response by patients, well score distribution, and absence of floor or ceiling effect support the content validity and responsiveness of the SCI-SETp. Floor and ceiling effects have not been reported for the original English version of the SCI-SET.¹²

Internal consistency

The SCI-SETp showed high acceptable internal consistency (α = 0.862) similar to that (α = 0.90) reported for the original English version¹² which confirms that all the questions of SCI-SETp are interrelated as a homogenous instrument. The Cronbach's α for SCI-SETp that was between the proposed criterion (0.70–0.90) for a good internal consistency reliability¹⁸ indicates no redundancy in SCI-SETp items.

Factor analysis

Dimensionality of the SCI-SETp was assessed by examining the item-total correlations and the PCA. The Cronbach's α found for the SCI-SETp was acceptable, suggesting that the SCI-SETp items have high internal consistency. But high α does not imply that the SCI-SETp is unidimensional. In order to provide evidence that the SCI-SETp is unidimensional, additional analyses of ITC and PCA were performed. The ITC for the SCI-SETp showed that most ITC values exceeded the acceptable cut-off of 0.3. The high α together with the good ITC values suggests that the SCI-SETp is a unidimentional instrument. The ITC has not been reported for the original English SCI-SET.¹²

In addition to the α coefficient of reliability and the ITC, we investigated the dimensionality of the SCI-SETp using factor analysis. The internal structure of the SCI-SETp with regard to the a priori criteria was demonstrated as a single factor instrument. The Eigenvalue for the first factor and the total variance explained are notably larger than those for the next factors (Table 3). This suggests that the items of SCI-SETp are unidimensional. The first factor accounted for 24.7%of the total variance was acceptable that was approximately 3–8 times higher than that of the other factors, all of which being less than 10% of total variance. However, the total percent variance accounted for by the first factor though acceptable may be interpreted as low. One reason could be the sample size. In the present study, 100 subjects were included according to the recommendations of Terwee et al.¹⁸ In order to diminish the errors and reach a stable solution a larger sample size of at least 300 participants has been suggested with the exploratory factor analysis.²⁰ A study with a larger sample size is required to investigate the SCI-SETp dimensionality and to identify the possible latent constructs. The factor analysis has not been evaluated for the original English SCI-SET.¹²

Test-retest reliability

Test-retest reliability for the SCI-SETp total scores was established (ICC_agreement = 0.84) which is in agreement with that (ICC = 0.91) from the original English version.¹² The acceptable test-retest reliability indicates that repeated measurements using SCI-SETp in stable patients with SCI would provide similar responses.

The SEM and SDC

The SEM and SDC as absolute reliability measures provide valuable information about the clinical reliability limits of the SCI-SETp and enables clinicians to make well informed decisions regarding whether a real change after an intervention has occurred or whether the observed change is due to the measurement error.

The SEM was used to quantify measurement error and to determine the smallest difference between measurements that is required to assume that a real change exists in test scores. A reliable and sensitive measure would present small measurement errors to detect real changes. The SEM value found in this study was small that indicates the SCI-SETp is a reliable measure.

The SEM value depicted that the SDC required to assume that a real difference exists between test-retest scores for the SCI-SETp was 0.82. The SDC enable clinicians to identify whether an individual patient has achieved a real change after treatment. To determine whether a real change in outcome has occurred between testing sessions using the SCI-SETp, changes in scores achieved by a patient with SCI must be more than or less than 0.82 to be considered real.

The SEM and SDC are not reported for the original English SCI-SET.¹² However, based on the data values provided by Adams et al (2007) we calculated the SEM and SDC for original English SCI-SET as 0.17 and 0.47, respectively. The SEM and SDC values found for the SCI-SETp were greater than those of the original English version.

Construct validity

Construct validity was examined in terms of the divergent validity between the SCI-SETp and the subscales of the PFIM. We considered the FIM for divergent validity because in previous study with the original English SCI-SET poor correlation between the two measures was displayed.¹² In testing for construct validity between the SCI-SETp and the PFIM-Motor subscale, as in the original English version, we found no significant correlation between the two measures. The lack of relationship between the SCI-SETp and the PFIM-Motor suggests that different constructs are being tapped, and the FIM-Motor cannot capture the effect of spasticity on daily life of patients with SCI. This finding suggests that the impact of spasticity on daily life of patients with SCI is not reflected in the FIM score as a measure of functional performance. This result is not surprising as the FIM is developed to assess the level of physical and cognitive help necessary for function and ADL tasks regardless of spasticity.²¹ The lack of correlation between the SCI-SETp and the PFIM-Motor subscale, consistent with the original English version, suggest that the SCI-SET has no overlap with the FIM, and a general measure of function such as FIM may be inadequate for the evaluation of spasticity impact on daily life in patients after SCI. The implication of this finding is that the SCI-SETp as a SCI-specific questionnaire measures a unique construct enhancing the SCI-SETp's usefulness.

The original English version of SCI-SET was also evaluated for associations with other measures of spasticity and was found to be significantly correlated with self-assessed spasticity impact (r = –0.61) and spasticity severity (r = –0.48), as well as the Quality of Life Index (QLI) SCI version (QLI-SCI) health and functioning subscale (r = 0.68) and the Penn spasm frequency scale (PSFS) (r = –0.66).¹² Instruments that assess the influence of spasticity on daily life of SCI patient are lacking.²² However, there are two condition-specific subjective tools available, the SCI-SET¹² and the Patient Reported Impact of Spasticity Measure (PRISM),²³ which are the most promising tools for the assessment of spasticity influence on quality of life (QoL) after SCI.¹⁵ Future studies must focus on the convergent construct validity evaluation of the SCI-SETp using appropriate measures such as PRISM developed to measure the impact of spasticity on QoL from the perspective of patients after SCI.²³

As expected, no significant correlation was observed between the SCI-SETp and the PFIM-Cognitive subscale further supporting the divergent construct validity. The divergent validity has not been evaluated for the original English version of SCI-SET.¹² The poor correlations between the SCI-SETp and the PFIM subscales confirmed the construct validity of the SCI-SETp.

There are limitations in this study that must be mentioned. First, the discriminative validity, convergent construct validity, and responsiveness in terms of effect size methodology were not evaluated. Second, further investigation with a larger sample size is required to clarify the internal structure of the SCI-SETp.

Conclusion

The SCI-SET was successfully translated and cross-cultural adapted to Persian language. The Persian version of SCI-SET demonstrated excellent acceptability and psychometric properties of reliability and validity, which is now available to assess the impact of spasticity on daily life of patients with SCI in Persian population.

Appendix: Persian Spinal Cord Injury Spasticity Evaluation Tool (SCI-SET)

(Colour online)

Disclaimer statements

Contributors All authors have contributed to the study concept and design. MK collected the data. NNA performed all statistical analyses. NNA and MK were involved in drafting the manuscript. All authors reviewed and revised the manuscript for important intellectual content and approved the final version to be published.

Funding None

Conflicts of interest None

Ethics approval The approval was obtained from the Tehran University of Medical Sciences (TUMS) Ethics Committee.