Reliability and minimal detectable change of the Trunk Assessment Scale for Spinal Cord Injury (TASS) and the trunk control test for individuals with spinal cord injury

Abstract

Study design

Cross-sectional study.

Objectives

To evaluate the reliability and calculate the measurement error of the Trunk Assessment Scale for Spinal Cord Injury (TASS) and trunk control test (TCT-SCI) in individuals with spinal cord injury (SCI).

Setting

Rehabilitation Hospital in Japan.

Methods

The evaluations of TASS and TCT-SCI for individuals with SCI were video-recorded. The inter-rater reliability (two physiotherapists) was confirmed using the videos. ICC (2,1), kappa coefficient (κ) were used to determine the reliability of the total score and each item. Each minimal detectable change (MDC) was calculated.

Results

The TASS and TCT-SCI total scores showed excellent inter-rater reliability (ICC = 0.99, and 1.00). The kappa coefficients of TASS were acceptable to excellent for 8 items (κ = 0.76–1.00), below acceptable for 1 item (κ = 0.62). The kappa coefficients of TCT-SCI were excellent for 12 items (κ = 0.83–1.00), below acceptable for 1 item (κ = 0.68). The inter-rater MDC of the TASS total score was 4.07 points, and the MDC of the TCT-SCI total score was 1.13 points. The intra-rater MDC of the TASS total score was 3.86 points.

Conclusion

Both TASS and TCT-SCI showed high reliability. Differences of less than four points in TASS and one point in TCT-SCI were interpreted as measurement errors between the two raters.

Introduction

The International Standards for Neurological Classification of Spinal Cord Injury (ISNCSCI), developed by the American Spinal Injury Association (ASIA), are widely used for functional assessment of spinal cord injury (SCI) [1, 2]. The ISNCSCI express the degree of disability using the ASIA Impairment Scale (AIS) and the neurological level of injury. The motor functions of the upper and lower limbs are represented by their respective motor scores, but motor function tests to assess trunk function are not included [3].

Interventions on trunk function are commonly used in the rehabilitation of individuals with SCI [4]. It has also been reported that trunk function in individuals with SCI is related to sitting balance, walking ability, and activities of daily living [5–7]. Several tools are available for assessing sitting balance and trunk function in individuals with SCI [8–13]. Among them, the Trunk Control Test for individuals with SCI (TCT-SCI), developed by Quinzaños et al., has been confirmed its reliability and validity in Mexico [6, 7]. This scale was thus reported as the “Gold Standard” in this field [14]. However, the TCT-SCI was composed of many postural maintenance tasks using the upper limbs or reaching tasks, we were concerned about the difficulty of use for individuals with tetraplegia. The measurement properties of TCT-SCI have not yet been examined in regions with aging population and increasing number of incomplete tetraplegia [15–20].

We developed a new scale, the trunk assessment scale for SCI (TASS) without the influence of upper limb dysfunction and to assess trunk function in all individuals with SCI, and reported its inter-rater reliability and internal consistency in live sessions [21]. However, most participants of the previous study were individuals with incomplete SCI (70% had AIS: D), and we did not confirm whether the TASS could be used for all individuals with SCI. We also pointed out the possibility that the fact that the number of assessment trials differed for each item may have affected the degree of agreement for each item. Therefore, it is necessary to confirm the reliability of the TASS after excluding the effects of changes in participant performance.

The first objective of present study was to confirm the inter- and intra-rater reliability of the TASS by video sessions and to calculate the measurement error in order to overcome the issues pointed out in the previous studies. The second objective was to confirm the reliability of the TCT-SCI and to calculate the measurement error in regions with a large number of individuals with incomplete tetraplegia.

Methods

Study design and setting

The present study was conducted at the authors’ institution. The results of the evaluation were obtained from December 2019 to March 2021. This article was reported based on the recommendations of the Guidelines for Reporting Reliability and Agreement Studies [22] and the COnsensus-based Standards for the selection of health Measurement INstruments [23, 24].

The study protocol was approved by the Ethics Committee of the Ibaraki Prefectural University of Health Sciences (approval no. 902). Written informed consent was obtained from all participants before enrollment in the study.

Participants

The participants were adult individuals (age ≥20 years) with SCI (both traumatic and non traumatic) with an AIS of A to D. Individuals were excluded from the selection process if they had difficulty in understanding the instructions for conducting the study, who were unable to participate in the rehabilitation program, and had difficulty performing these evaluations safely due to a history of cardiovascular, cardiac, or orthopedic surgery. They were also excluded if they were judged by the staff to be unsuitable for participation in the study.

Sample size

The sample size requirement by COSMIN is considered to be the rule of thumb [25]. The sample size required for this study was thus estimated using Fisher’s z–r transformation [26, 27]. Using the intra-class correlation coefficients (ICCs) reported in a previous study [21], a sample of 6 was required to establish inter-rater reliability (n = 2 raters). Also, the sample size required to establish intra-rater reliability was confirmed a posteriori from a preliminary assessment conducted on six participants. Based on the results of the preliminary assessment (ICC of 0.97, 95% confidence interval [CI], 0.85–1.00), it was confirmed that a sample of 5 was required to establish intra-rater reliability (n = 2 sessions). Therefore, the sample size of this study was set at a minimum of 6 and to cover all types of paralysis (tetraplegia/paraplegia, complete/incomplete) among individuals with SCI. To avoid bias in assessing the severity of paralysis, which was an issue identified in previous studies, participants were recruited so that there would be more cases adjudged to have AIS of A to C than AIS of D and met the aforementioned sample size requirements.

Outcome measure

The TASS is a 9-item scale for assessing trunk function in individuals with SCI (Table 1) [21]. The task content and scoring were determined via a questionnaire (using the Delphi method) administered to Japanese experts (physical therapists for SCI). Each item was assigned a score of 0 to 2, 0 to 4, and 0 to 6, for a total score of 44, with higher scores indicating better trunk function. The participants were allowed to wear shoes, but not lower limb orthoses. Participants instructed by their physicians to wear a cervical or trunk brace during rehabilitation were assessed while wearing the respective brace. The participants sat on a flat seat without a backrest (height, 40–45 cm). The hip and knee joints were flexed at 90°, the ankle joint was dorsiflexed at 0°, and the sole was fully grounded. The use of the upper extremities as support was prohibited. The task contents were as follows: item 1, sitting position maintenance; item 2, ischial bone elevation; item 3, trunk rotation; item 4, trunk frontal flexion; item 5, trunk lateral flexion; item 6, trunk backward tilt; and item 7, reaching task. The assessment took less than 5 min.

Table 1.

Trunk Assessment Scale for Spinal Cord Injury (TASS).

Initial position:
(1) Sitting on flat seat without backrest (height 40–45 cm)
(2) 90° hip/knee joint flexion, 0° ankle joint dorsiflexion, all soles grounded
(3) No lower limb orthosis
(4) Support with upper limbs is prohibited (upper limb position is optional)
1. Maintain initial position during 10 s ※measured with a stopwatch
maintain position for more than 10 s	2
maintain position for more than 5 s and less than 10 s	1
5 s or more cannot be maintained	0
2. Elevation of the ischial bone ※The therapist will insert his or her hand to check
able to elevate the ischial bone on one side and return to the initial position (bilateral)	4
able to elevate the ischial bone on one side and return to the initial position (only unilateral)	2
unable to elevate the ischial bone/ able to elevate the ischial bone, but cannot return to initial position	0
3. Rotate the trunk 30 deg (measured by the line connecting both acromion) ※measured with an angle meter
more than 30 deg of rotation	4
less than 30 deg of rotation	2
unable to rotation	0
4. Touch the ankle joint and return to the initial position
able to touch the ankle joint and return to the initial position	4
able to touch the ankle joint, but cannot return to initial position	2
unable to touch the ankle joint	0
5. Touch the bed with one elbow (stand still for 3 s) and return to the initial position
able to touch the bed with one elbow and return to the initial position (bilateral)	6
able to touch the bed with one elbow and return to the initial position (unilateral)	3
unable to touch the bed with elbow/ able to touch the bed, but cannot return to initial position	0
6. Touch the bed with PSIS and return to the initial position
able to touch the bed with PSIS and return to the initial position	6
able to lean trunk backward, but cannot touch the bed with PSIS	3
enable to lean trunk backward/ fall backward	0
7. SRT/STT ※measured with a ruler or a tape measure
SRT: measure the reaching distance in three directions (FR, RR, LR) using the fingertip as an index
STT: measure the reaching distance in three directions (FR, RR, LR) using the acromion as an index
[FR]
more than 30 cm	6
more than 20 cm and less than 30 cm	4
more than 10 cm and less than 20 cm	2
less than 10 cm	0
[RR]
more than 30 cm	6
more than 20 cm and less than 30 cm	4
more than 10 cm and less than 20 cm	2
less than 10 cm	0
[LR]
more than 30 cm	6
more than 20 cm and less than 30 cm	4
more than 10 cm and less than 20 cm	2
less than 10 cm	0
	/44

Item 7: Basically, SRT is used, but STT is used for patients who have difficulty in raising the upper limbs.

PSIS posterior superior illiac spine, SRT seated reach test, STT shoulder thrust test, FR front reaching, RR right reaching, LR left reaching.

The TCT-SCI is a 13-item trunk function scale [6]. The maximum total score is 24 points, with higher scores indicating better trunk function. The participants were initially assessed in the sitting position, with feet on the support surface, knee joints flexed at 90°, without trunk support, and hands resting on the thighs.

The contents are as follows: (1) The three items of static equilibrium involve maintaining the initial position (item 1) and maintaining a sitting position with one lower limb crossed over the other (items 2 and 3). (2) The four items of dynamic equilibrium involve touching the feet (item 4), lying down in the supine position from the initial position and returning to the initial position (item 5), and rolling onto the right and left sides (items 6 and 7). (3) The six items of dynamic equilibrium involve carrying out activities with the upper limb are reaching tasks performed from the initial position with the unilateral upper limb in 90° shoulder flexion, elbow in full extension, forearm in pronation, hand joint in neutral plantar dorsiflexion, and fingers in extension (items 8–12, 3 directions on each side).

Procedure

After recruitment, case information, such as the name of the diagnosis and the results of the ISNCISC assessment, was collected from the participants’ medical files. The two raters involved in the assessment were physical therapists with at least 4 years of experience in SCI rehabilitation (rater A: 8 years, rater B: 4 years). The two raters confirmed the TASS manual and TCT-SCI before the evaluation.

Inter-rater reliability

Originally, the TASS and TCT-SCI are scales that directly assess individuals with SCI. However, in the present study, we decided to conduct video sessions to exclude the effects of changes in participants’ performance. Evaluations of TASS and TCT-SCI performed by rater A were recorded using a video camera (Sony, HDR-CX470, Japan). The TASS items 1, 4, 5, and 7, RR and LR, were photographed from the anterior side, items 2 from the posterior side, item 3 from the superior side, item 6 from the lateral side, and item 7, FR, from the lateral side. In the TCT-SCI, static items 1–3, and dynamic items 1, 2 were recorded from the anterior side, and dynamic items 3, 4 were recorded from the caudal side. Dynamic items with upper limb activity were recorded from the medial side so that the contact with the target could be seen. Raters A and B looked at the video to score each participant’s performance. Rater A watched the videos at least 2 weeks after the sessions to be blinded to previous ratings. The two raters scored the videos independently.

Intra-rater reliability

Rater B watched the videos on TASS and TCT-SCI twice (sessions 1 and 2) with an interval of more than 2 weeks and evaluated all participants. The raters were blinded to their previous ratings.

Statistical analysis

Descriptive statistics were used to describe the clinical characteristics of the sample.

Scores from the video assessments of raters A and B were used to confirm inter-rater reliability, and scores from the video assessments of rater A (sessions 1 and 2) were used to confirm intra-rater reliability. ICCs with their respective 95% CIs were calculated for the total scores. Cohen’s kappa coefficient was calculated to confirm the agreement for each item. Coefficients greater than 0.70 for both items are recommended as the minimum standard for reliability [23] and values greater than 0.80 were considered excellent [28]. The standard error of measurement (SEM) was calculated with the previously calculated ICC using the formula:

$$\mathrm{SEM}=\mathrm{SD}\times \sqrt{\left(1-\mathrm{ICC}\right)}$$

where SD is the standard deviation of the scores obtained on the TASS and TCT-SCI from each session, and ICC is the corresponding reliability coefficient. The minimal detectable change (MDC) was calculated with each SEM using the formula:

$${\mathrm{MDC}}_{95}=\mathrm{SEM}\times 1.96\times \sqrt{2}$$

where 1.96 is the z-value chosen. MDC₉₅ represents the smallest score change at a 95% CI, which can be considered a true change beyond the measurement error [27]. SEM% and MDC% were defined as the percentages of SEM and MDC₉₅ to the mean value of each evaluation. An MDC% of <30 is considered acceptable, while an MDC% of <10 is considered excellent [29].

Finally, the Bland–Altman plots of difference against mean with limits of agreement (LoA) were used as a visual demonstration of the agreement between sessions and pairs of raters [30, 31]. If 0 was included in the 95% CI of the mean of the differences, it was adjudged that there was a fixed error. Regression analysis was performed for the Bland–Altman plot, and if the regression coefficient was significant, it was adjudged to have a proportional error. The LoA was calculated using the following formula [32]:

$$\mathrm{LoA}=\mathrm{mean}\mathrm{difference}\left(d\right)\pm 1.96\times \mathrm{SD}$$

where SD is the standard deviation of the difference. The statistical analyses were performed using R version 3.6.3 (R Development Core Team, Vienna, Austria) [33]. Analysis items with p < 0.05 were considered statistically significant.

Results

Characteristics of participants

Nine participants with SCI completed the reliability study. The participants’ demographic characteristics at the time of assessment are presented in Table 2. Five participants were more than 65 years old. Five participants were adjudged to have AIS of A or C and four participants were adjudged to have AIS of D.

Table 2.

Participants’ characteristics at the time of reliability assessments.

No.	Ages [years]	Sex	Traumatic/non traumatic	Diagnosis	NLI	AIS	Motor score		TASSb	TCT-SCIb	Days from onset to assessment
							UEMS	LEMS
1	61	M	Traumatic	Cervical cord injury	C4	A	27	0	0	0	6818
2	65	M	Traumatic	Cervical cord injury	C3	C	24	30	23	2	246
3	84	M	Non traumatic	Cervical spondylotic myelopathy	C4	D	39	43	41	18	395
4	67	M	Traumatic	Cervical cord injury	C5	D	47	50	40	18	2145
5	42	M	Traumatic	Thoracic cord injury	T6	A	50	0	8	17	3490
6	68	M	Traumatic	Thoracic cord injury	T12	A	50	0	0	14	18,282
7	64	M	Non traumatic	Thoracic cord injury	T12	C	50	27	27	23	71
8	69	M	Traumatic	Thoracic cord injury	C7a	D	50	38	23	18	105
9	56	F	Traumatic	Thoracic cord injury	T10	D	50	42	39	24	91
Mean ± SD	64.0 ± 11.2						43.0 ± 10.6	25.6 ± 20.3	22.3 ± 16.5	14.9 ± 8.50	3515.9 ± 5984.2
Median (IQR)	–						50.0 (39.0, 50.0)	30.0 (0.00, 42.0)	23.0 (8.00, 39.0)	28.0 (14.0, 18.0)	834.0 (140.0, 3154.0)

NLI neurological level of injury, AIS ASIA impairment scale, UEMS upper extremity motor score, LEMS lower extremity motor score.

^aParticipant #8 was diagnosed with thoracic spinal cord injury, but due to the cervical spinal canal stenosis, he had hypoalgesia in the C8 region, and his NLI was determined to be C7.

^bTASS and TCT-SCI scores are the first assessment scores by rater A.

Inter-rater reliability

The inter-rater reliability was excellent for both the total scale score (TASS: ICC = 0.99, TCT-SCI: ICC = 1.00) (Table 3). The kappa coefficients of TASS were excellent for 7 items (κ = 0.85–1.00), acceptable for 1 item (κ = 0.76), and below acceptable for 1 item (κ = 0.62). The kappa coefficients of TCT-SCI were excellent for 12 items (κ = 0.83–1.00), and below acceptable for 1 item (κ = 0.68) (Table 4). On the Bland–Altman plot for TASS, the 95% CI of the mean difference was −2.35 to 0.80, the 95% LoA was −4.79 to 3.24, and the slope of the regression line was −0.26 (p = 0.48). On the Bland–Altman plot for TCT-SCI, the 95% CI of the mean difference was −0.05 to 0.72, the 95% LoA was −0.65 to 1.31, and the slope of the regression line was −0.28 (p = 0.47) (Fig. 1 and Table 3).

Table 3.

Reliability and minimal detectable change of the TASS and TCT-SCI total score.

	TASS				TCT-SCI
	Inter-rater		Intra-rater		Inter-rater		Intra-rater
ICC (95% CI)	0.99	(0.97, 1.00)	0.99	(0.97, 1.00)	1.00	(0.99, 1.00)	1.00	(–)
Systematic error
95% CI for the mean (lower, upper)	(−2.35, 0.80)		(−1.92, 1.25)		(−0.05, 0.72)		–
95% LoA (lower, upper)	(−4.79, 3.24)		(−4.37, 3.71)		(−0.65, 1.31)		–
Regression	−0.26	(p = 0.48)	0.40	(p = 0.29)	−0.28	(p = 0.47)	–
Measurement error
SEM (SEM%)	1.47	(6.47)	1.39	(5.98)	0.41	(2.78)	–
MDC95 (MDC%)	4.07	(17.9)	3.86	(16.6)	1.13	(7.68)	–

Inter-rater reliability was determined using the ICC(2,1) and Intra-rater reliability was determined using the ICC(1,1).

CI confidence interval, LoA limits of agreement, SEM standard error of measurement, MDC minimal detectable change.

Table 4.

Kappa coefficients and agreement ratio of each scale (TASS, TCT-SCI).

	Inter-rater		Intra-rater
	Kappa	Agreement (%)	Kappa	Agreement (%)
TASS
item 1	1.00	100.0	1.00	100.0
item 2	1.00	100.0	1.00	100.0
item 3	0.89	88.9	0.89	88.9
item 4	1.00	100.0	1.00	100.0
item 5	1.00	100.0	0.89	88.9
item 6	0.62	77.8	0.75	77.8
item 7-FR	1.00	100.0	0.88	88.9
item 7-RR	0.76	77.8	0.88	88.9
item 7-LR	0.85	88.9	0.89	88.9
TCT-SCI
item 1	1.00	100.0	1.00	100.0
item 2	0.83	88.9	1.00	100.0
item 3	0.68	77.8	1.00	100.0
item 4	1.00	100.0	1.00	100.0
item 5	1.00	100.0	1.00	100.0
item 6	1.00	100.0	1.00	100.0
item 7	1.00	100.0	1.00	100.0
item 8	1.00	100.0	1.00	100.0
item 9	1.00	100.0	1.00	100.0
item 10	1.00	100.0	1.00	100.0
item 11	1.00	100.0	1.00	100.0
item 12	1.00	100.0	1.00	100.0
item 13	1.00	100.0	1.00	100.0

Fig. 1. Bland–Altman plots regarding the TASS and the TCT-SCI.

Inter-rater agreement for the TASS (a), the TCT-SCI (b), and intra-rater agreement for the TASS (c). These Bland–Altman plots were constructed to show the degree of inter and intra-rater agreement between the two scales (TASS and TCT-SCI). The intra-rater agreement of the TCT-SCI was perfect, so the Bland–Altman plot was not constructed. These results indicate that no systematic error was included in the TASS and the TCT-SCI scores.

The SEMs of the total scores obtained from the inter-rater reliability of the TASS and TCT-SCI were 1.47 and 0.41 points, and the MDC₉₅ was 4.07 and 1.13 points, respectively (Table 3).

Intra-rater reliability

The intra-rater reliability was excellent for both the total scale score (TASS: ICC = 0.99, TCT-SCI: ICC = 1.00) (Table 3). The kappa coefficients of TASS were excellent for 8 items (κ = 0.88–1.00) and acceptable for 1 item (κ = 0.75), and the kappa coefficients of TCT-SCI were excellent for 13 items (κ = 1.00) (Table 4). On the Bland–Altman plot for TASS, the 95% CI of the mean difference was −1.92 to 1.25, the 95% LoA was −4.37 to 3.71, and the slope of the regression line was 0.40 (p = 0.29) (Fig. 1 and Table 3).

The SEM of the total scores derived from the intra-rater reliability of TASS was 1.39, and the MDC₉₅ was 3.86 points.

Discussion

In the present study, the reliability of the TASS and TCT-SCI was confirmed in a video session of individuals with SCI (AIS: A to D). The results suggested that both scales had high reliability and could be used for older individuals with SCI; however, only one item of each scale had a low inter-rater agreement. The MDC₉₅ values for both scales were 4.07 and 1.13, respectively. To the best of our knowledge, this is the first study to report the scale characteristics of TCT-SCI in regions with a large number of individuals with poor upper limb function and the measurement errors of both scales (TASS and TCT-SCI).

The present study showed that the inter-rater and intra-rater reliabilities of both scales (TASS and TCT-SCI) were excellent and exceeded the recommended value for clinical use (ICC > 0.90) [27]. In addition, although the proportion of severity of paralysis among participants differed from that reported in the previous study, the inter-rater ICC of the TASS in the present study (0.99) was comparable to that of previous studies (0.98) [21]. The high level of agreement found in this study may be due to the fact that the task content was designed by Japanese experts of SCI rehabilitation (physical therapists for SCI) in an easy-to-understand manner. These results suggest that this test can be easily used by physical therapists in any region. None of the scales showed systematic errors, as shown by the Bland–Altman analysis, and we are able to refer to measurement errors. Since the MDC₉₅ of TASS was 4.07, and the intra-rater MDC₉₅ was 3.86, the change of five points (difference beyond two levels) in the total TASS points was considered to indicate a change beyond the measurement error. On the other hand, since the inter-rater MDC₉₅ of the TCT-SCI was 1.13, a change of two points (roughly two levels of difference) in the total TCT-SCI score was considered to indicate a change beyond the measurement error. Because an MDC% of less than 30 was considered acceptable, and an MDC% of less than 10 was considered excellent [29], both inter-rater MDCs were acceptable (MDC%: TASS = 17.9, TCT-SCI = 7.68) [29]. Since a roughly 2 level difference was the measurement error for both scales, the fact that the MDC of TCT-SCI was smaller than that of TASS may be explained by the difference in the scoring for each item.

The agreement for item 6 of the TASS was below the acceptable level (κ = 0.62), and this result was similar to that reported in a previous study (κ = 0.55) [21]. In a previous study, the authors pointed out that the reason for the lack of agreement on this item was the possibility that the number of assessments per item in the live session increased and the participants’ performance changed. To confirm the truth of this consideration, the video sessions in this study were conducted in such a way that the participants’ performance did not change; however, there was poor inter-rater agreement on the same items. Therefore, this lack of agreement is most likely due to the content of the task, not the change in participants’ performance. It is also possible that the difficulty in confirming the contact with the bed and the Posterior Superior Illiac Spine in the video affected the results, but it is necessary to discuss whether the content of this item should be revised in the future. In addition, the TCT-SCI also had only one item with the agreement below the acceptable level. However, since the intra-rater reliabilities of both scales are acceptable even with the current content, if the same therapist repeatedly assesses one individual with SCI, these reliabilities could be assured.

The present study was very meaningful because it evaluated a highly reliable trunk function assessment scale that can be used for all individuals with SCI. The main limitation of the present study was that only individuals with SCI who were admitted to a single institution were included and that our sample size was smaller than that required by COSMIN. Therefore, in the future, it would be desirable to evaluate individuals with SCI at another center in Japan and overseas. In addition, the validity and responsiveness of the TASS have not yet been confirmed, we thus would like to confirm its usefulness after estimating the required sample size.

Conclusions

The results of the present study indicate that the TASS and TCT-SCI are highly reliable measures of trunk function in regions with a large number of individuals with poor upper limb function. The observed MDC₉₅ indicates inter-rater and intra-rater measurement errors. However, since both scales had one item with the insufficient inter-rater agreement, it should be interpreted with caution when the raters change.

Author contributions

All authors were involved in conceptualizing the research. HS and SK collect the data. HS performed the data analysis with interpretation provided by all authors. HS wrote the initial draft, all authors were involved in editing, producing, and approving the final manuscript. MM got funding.

Data availability

The datasets generated and analyzed during the current study are available from the corresponding author on reasonable request.

Competing interests

The authors declare no competing interests.

Ethical approval

We certify that all applicable institutional and governmental regulations concerning the ethical use of human volunteers were followed during the course of this research.