Reliability and responsiveness of Jebsen-Taylor Test of Hand Function and Box and Block Test for children with cerebral palsy

Abstract

AIM

To assess the reliability and to evaluate the responsiveness of both the Jebsen-Taylor Test of Hand Function (JTTHF) and the Box and Block Test (BBT) in children with cerebral palsy (CP).

METHOD

In this retrospective study, the reliability analyses were conducted with paired t-tests considering a short (mean 14d) and a long (mean 120d) time in between two assessment periods. In addition, an intraclass correlation coefficient (ICC) was used to assess the level of congruency. The responsiveness to therapy was conducted with a paired t-test in the whole sample regarding the age, the manual ability level as classified with the Manual Ability Classification System (MACS), and the topography.

RESULTS

Our main results confirmed the tests’ reliability in a short time period for the JTTHF in both hands and for the BBT on the less affected hand. These results were consistent with the ICC. The responsiveness was confirmed, except on the less affected hand for the JTTHF, with similar results for age, MACS, and topography approach.

INTERPRETATION

This study supports the use of the JTTHF and the BBT to examine changes after short-term interventions for children with CP. These results should be interpreted with association to normative values or with a control group when used over long assessment periods

Cerebral palsy (CP) is defined as a group of disorders of movement and posture secondary to early/congenital brain injury.¹ CP is the most common cause of pediatric motor deficits with a prevalence of these disorders ranging from 2 to 2.5 out of 1000 children.^2,3 Depending on brain lesion location, either bilateral or unilateral CP may be observed. Gross motor function impairments and manual dexterity deficits are frequently observed in these children, having a major impact on their activity level and autonomy.^4–6 Improving manual dexterity and activity level of children with CP is often a main focus in rehabilitation. Over the last decade, intensive interventions based on motor skill learning principles have demonstrated efficacy, inducing motor changes and improving activities of daily life in children with CP.^6,7

The efficacy of such interventions on manual ability is notably observed through changes in test scores assessing manipulative skills. These skills in children with CP, both in research and clinical practice, are frequently assessed using the Box and Block Test (BBT)⁸ and the Jebsen-Taylor Test of Hand Function (JTTHF).⁹ The BBT was originally developed for adult patients in order to assess gross manual dexterity.⁸ This test has an excellent test–retest reliability, inter/intrarater reliability, high responsiveness, and criterion and construct validity in typically developing children and adults for various pathologies.^10–15 The JTTHF assesses a battery of seven items of unimanual hand functions simulating activities of daily living. In adults and typically developed children, this test has an excellent test–retest reliability^16–18 and an excellent inter/intrarater reliability.¹⁹ However, criterion validity is quite poor in adults and typically developing children.^19,20 Hence, these tests could represent two approaches to evaluate hand function in children with CP, with different levels of psychometric qualities.

The intrinsic psychometric qualities of a test are essential to highlight modifications produced by an intervention; hence, the test used should be valid, reliable, and sensitive to changes.²¹ In addition, in clinical practice it is also important to determine whether changes as a result of an intervention are clinically meaningful. Despite the wide use of the BBT and JTTHF in children with CP, the stability of the measure over time, and the responsiveness, defined as the power of the measure to reveal changes over time, have not been sufficiently studied in this population. The reliability of the JTTHF was tested by Taylor et al. in a small group of children with different disorders. This sample included children with CP.¹⁸ Even though Taylor et al. demonstrated the reliability, results cannot be generalized because of the size and heterogeneity (many different disorders) of their sample.

The objectives of this retrospective study were, first, to assess the reliability of the BBT and the JTTHF in children with CP; and second, to evaluate the responsiveness of both tests to bimanual intensive interventions. Lastly, we aimed to evaluate the clinical significance of this change. We hypothesized that the JTTHF and the BBT would be sensitive enough to detect dexterity changes occurring after intensive upper extremity interventions.

METHOD

Participants

Participants included 154 children with unilateral (n=108) or bilateral (n=46) CP aged from 5 to 17 years (mean age 9y 10mo; SD 3y 4mo). The characteristics of the children are displayed in Table I. Briefly, 67 children were considered to assess the stability of the measures and 87 were considered to assess the responsiveness of the measures. Data were collected in two different research projects (2011–2017), one at Columbia University in the USA (n=56), and one at the Université catholique de Louvain in Belgium (n=98). Inclusion/exclusion criteria were the same as previous studies^7,22 (more details in Appendix S1, online supporting information).

Table I:

Characteristics of the participants

	All participants	Reliability	Responsiveness
Age, y:mo
Mean (SD)	9:6 (3:4)	9:6 (2:11)	9:6 (3:6)
Sex
Females (%)	78 (51%)	41 (61%)	37 (43%)
Males (%)	76 (49%)	26 (39%)	50 (57%)
Form of cerebral palsy
Unilateral form (%)	108 (70%)	38 (57%)	70 (81%)
Bilateral form (%)	46 (30%)	29 (43%)	17 (19%)
More affected hemibody
Right side (%)	73 (47%)	33 (49%)	40 (46%)
Left side (%)	81 (53%)	34 (51%)	47 (54%)
Manual Ability Classification System
Level I	25 (16%)	8 (12%)	17 (19%)
Level II	90 (58%)	38 (57%)	52 (60%)
Level III	39 (26%)	21 (31%)	18 (21%)
Total	154	67	87

All parents and children gave their written informed consent. The ethical review boards of the respective universities approved these studies.

Study design

This study assessed the reliability and sensitivity to change of the measures (BBT and JTTHF). Both assessments required two testing times, with or without intensive therapy in between (see Fig. S1, online supporting information). The assessments were performed by the same rater at each assessment time to avoid intrarater bias.

The reliability, defined as the stability of the measures, was tested between two assessment times, named A1 and A2, without intensive intervention in between. From the 67 children enrolled in this part, 38 had a test–retest within a brief period (mean [SD] 14d, [1]; range 13–14d) and 29 had a test–retest within a longer period (mean [SD] 120d, [2]; range 30–210d).

The sensitivity to changes of the measures was assessed before and after two intensive interventions: hand and arm bimanual intensive therapy²² (n=41, USA) or hand and arm bimanual intensive therapy including lower extremity^7,23 (n=46, Belgium) (Fig. S1). Both interventions were conducted in a day camp setting with a total of 90 hours of treatment. Both therapies promote more accurate movements and the appearance of new strategies using principles of motor skill learning. T1 was performed within the week just before the intervention; T2 was performed within the week just after the intervention.

Assessments

The BBT was performed according to the description of Mathiowetz et al.⁸ The child sat on an adjusted chair in front of a two-compartment box placed on a table, one compartment containing a total of 150 blocks. The goal was to transport as many blocks as possible, one at a time, from one compartment to the other with the same hand. The score is the number of blocks transported within a minute.⁸

To perform the JTTHF, the child sat in front of a table where the test was performed. As first described by Jebsen et al.,²⁴ the test consists of seven items including writing a 24-letter sentence, turning over five cards, picking up small common objects (two pennies, two bottle caps, and two paperclips), simulated feeding using a teaspoon and five kidney beans, stacking four checkers, picking up and moving five large empty tin cans and then five large full cans. In this study, all items were tested except the writing.¹⁸ The time needed to complete each item with each hand was recorded with a maximum time of 3 minutes. The total score was obtained for each hand as the sum of each item time.

In both tests the less affected hand was first evaluated, followed by the more affected hand.

Statistical analysis

Reliability

A whole sample and a group approach were used to assess the stability of both measures. In all children without intervention (n=67) the whole sample approach consisted of evaluating a possible difference between A1 and A2 (trial effect) using a paired t-test or Wilcoxon test when the Shapiro–Wilk normality test failed. In addition, the degree of association between A1 and A2 was observed using the intraclass correlation coefficient (ICC). We used these methods to assess the intrinsic stability of each child’s score. Since reproducibility may differ in time, we also performed a group approach, where the same analyses were performed within two assessment periods (i.e. short period of time or long period of time, see details in ‘Study design’).

Thus, no significant differences between A1 and A2, with high values of ICC, confirm the reliability of the tools.

Responsiveness

For the study of responsiveness, a whole sample and a group approach were also used. For both approaches, comparisons between T1 and T2 were performed using paired t-test or Wilcoxon test when Shapiro–Wilk normality test failed. In the whole sample approach (n=87) we evaluated all children together. In the group approach we considered children regarding their age, their Manual Ability Classification System (MACS)²⁵ level, and their topography separately. The group approach allowed us to determine whether a specific (sub)group demonstrates a higher responsiveness. In addition, the effect size was used as an index of responsiveness in both the whole sample and group approaches.²⁶ The effect size gauges the amount of changes of a scale to amounts considered to be clinically meaningful. It is calculated by dividing the mean change of T1 and T2 by the standard deviation of the change scores. The effect size was interpreted regarding Cohen’s guidelines:²⁷ results under 0.2 are considered as non-significant changes; results between 0.2 and 0.49 are considered as small changes; results between 0.5 and 0.79 are considered as moderate changes; and large changes are considered when results are over 0.8. Thus, significant differences between T1 and T2, with significant changes in the effect size, confirm the responsiveness of the tools.

Clinical significance of changes

We evaluated the clinical significance of changes between T1 and T2 using the minimal clinically important difference (MCID),²⁸ to detect if the difference measured by a tool is sufficient to be considered as clinically important for each child in a treatment outcome.²⁹ It is calculated as half the standard deviation of the changes,³⁰ considered as clinically significant when the observed change is above the MCID value.

RESULTS

Reliability

Whole sample approach

A significant difference was observed between A1 and A2 for the BBT on the more affected hand (p<0.001) and less affected hand (p=0.002), as well as on the more affected hand for the JTTHF (p=0.002), showing an improvement over time in the measures. In addition, an excellent ICC (test–retest) was observed for both tests in the more affected hand and in the less affected hand (all ICC>0.849; Table IIa).

Table II:

Test–retest reliability indices of the Box and Block Test (BBT) and Jebsen-Taylor Test of Hand Function (JTTHF)

		Mean (SD) or median [P25–P75]			Paired t-test or Wilcoxon		Intraclass correlation coefficient
		A1	A2	Mean of change (SD)	t (or z)	p	[lower and higher bound]	p
(a) Whole sample approach (n=67)
BBT	MA	20.00 [14.00–31.00]	23.00 [15.00–34.00]		3.804	<0.001	0.944 [0.910–0.965]	<0.001
	LA	43.00 [31.00–51.00]	46.00 [35.00–55.00]		3.109	0.002	0.849 [0.765–0.904]	<0.001
JTTHF	MA	281.09 [99.92–623.89]	204.37 [83.32–498.81]		−3.132	0.002	0.888 [0.817–0.933]	<0.001
	LA	55.76 [41.16–102.81]	51.32 [41.22–88.15]		−1.507	0.133	0.884 [0.810–0.930]	<0.001
(b) Group approach
Within 14 days (n=38)
BBT	MA	24.71 (14.41)	26.06 (15.14)	1.553 (3.244)	−2.950	0.006	0.971 [0.945–0.985]	<0.001
	LA	35.84 (14.57)	37.18 (14.81)	1.342 (5.085)	−1.627	0.112	0.938 [0.884–0.967]	<0.001
JTTHF	MA	284.65 (266.36)	257.34 (257.92)	23.246 (95.302)	1.484	0.147	0.932 [0.874–0.965]	<0.001
	LA	73.24 [47.30–121.19]	58.56 [45.34–117.56]	2.281 (44.046)	−0.911	0.366	0.939 [0.886–0.968]	<0.001
Within 120 days (n=29)
BBT	MA	20.00 [14.00–27.50]	24.00 [16.50–32.00]	4.379 (9.458)	2.979	0.003	0.622 [0.340–0.802]	<0.001
	LA	50.00 [44.50–53.50]	54.00 [50.50–59.00]	5.207 (9.466)	3.137	0.002	0.295 [−0.069 to 0.592]	0.054
JTTHF	MA	507.48 [275.35–804.91]	460.32 [188.49–766.04]	114.567 (162.633)	−2.656	0.006	0.784 [0.529–0.910]	<0.001
	LA	44.10 [36.81–57.13]	43.45 [36.71–50.96]	21.907 (65.389)	−1.241	0.229	0.284 [−0.173 to 0.644]	0.108

MA, more affected hand; LA, less affected hand.

Group approach

When we compared A1 and A2 over a short period of time (2wks), a significant difference was detected only for the BBT on the more affected hand (p=0.006). Within 2 weeks, an excellent ICC was observed for both tests in both hands (all ICC>0.932; Table IIb). These results indicate a reliability for the BBT on the less affected hand and for the JTTHF in both hands while performed over a short period of time.

When we compared A1 and A2 over a longer period of time (mean 120d), a significant difference was detected for the BBT on the more affected hand (p=0.003) and less affected hand (p=0.002) hands as well as for the JTTHF on the more affected hand (p=0.006). During this longer time interval, we observed levels of ICC between weak and good (from 0.284 to 0.784; Table IIb). These results indicate no reliability for either tests in either hands when performed over a longer period of time.

Responsiveness

Whole sample approach

In this approach, a significant difference was observed between T1 and T2 in the BBT for both hands (p<0.001). In the JTTHF the comparison showed a difference for the more affected hand (p<0.001). In addition, the effect size indicated moderate changes on the more affected hand (effect size=0.754) and the less affected hand (effect size=0.511) for the BBT, and small changes on the more affected hand for the JTTHF (effect size=0.472) (see Table SI, online supporting information).

Group approach

Group analysis by age

To perform the group analysis regarding the age, children included in this part of the study (n=87) were separate into two groups: a younger group (n=56) with ages between 5 and 10 years and an older group (n=31) with ages between 11 and 17 years (Table SI).

The younger group showed significant changes for the BBT on the more affected hand (p=0.003) and the less affected hand (p=0.001) as well as for the JTTHF on the more affected hand (p<0.001). The effect size showed moderate changes in both hands for the BBT (more affected hand, effect size=0.567; less affected hand, effect size=0.516) as well as in the more affected hand for the JTTHF (effect size=0.502).

In the older group, we observed significant changes for the BBT on both hands (less affected hand, p=0.003; more affected hand, p<0.001) and for the JTTHF in the more affected hand (p=0.002). In addition, the effect size showed large changes for the BBT on the more affected hand (effect size=0.936) and moderate changes on the less affected hand (effect size=0.581) as well as small changes for the JTTHF on the more affected hand (effect size=0.432).

Group analysis by MACS level

To perform a group analysis regarding the level of MACS, we separated all children (n=87) in three groups: MACS level I (n=17), MACS level II (n=52), and MACS level III (n=18) (Table SI).

The MACS level I group showed significant changes for both tests on the more affected hand only (BBT, p<0.001; JTTHF, p=0.049). The effect size showed a large change for the BBT in the more affected hand (effect size=0.814) and a moderate change for the JTTHF on the more affected hand (effect size=0.518).

The MACS level II group showed significant changes for the BBT on both hands (p<0.001) and for the JTTHF on the more affected hand (p<0.001). The effect size showed moderate changes for the BBT (more affected hand, effect size=0.612; less affected hand, effect size=0.520) and a small change for the JTTHF on the more affected hand (effect size=0.485).

The MACS level III group showed significant changes for the BBT in both hands (more affected hand, p=0.014; less affected hand, p=0.003) and on the more affected hand (p=0.014) for the JTTHF. The effect size showed a large change for the BBT in the less affected hand (effect size=0.829) and a moderate change on the more affected hand (effect size=0.639) as well as small change in the JTTHF on the more affected hand (effect size=0.484).

Group analysis by topography

To perform the group analysis regarding the topography, children (n=87) were divided in two groups: unilateral impairments (n=70) and bilateral impairments (n=17) (Table SI).

The unilateral group showed significant changes for the BBT on the more affected hand (p<0.001) and the less affected hand (p<0.001) as well as for the JTTHF on the more affected hand (p<0.001). The effect size showed moderate changes in both hands for the BBT (more affected hand, effect size=0.678; less affected hand, effect size=0.514) as well as a small change in the more affected hand for the JTTHF (effect size=0.489).

In the bilateral group, we observed significant changes for the BBT on the more affected hand (p=0.010) and a clear trend on the less affected hand (p=0.052). The JTTHF showed significant changes in the more affected hand (p=0.005). The effect size showed large changes for the BBT on the more affected hand (effect size=0.936) and moderate changes in the less affected hand (effect size=0.496) as well as moderate changes on the more affected hand for the JTTHF (effect size=0.594).

Clinical significance of change

In the whole sample approach, the clinical significant difference for the BBT was 1.9 (blocks) on the more affected hand and 3.0 (blocks) on the less affected hand. For the JTTHF the MCID was 54.7 seconds on the more affected hand and 20.9 seconds on the less affected hand (Table SI). In the group approach we compared whether these values were larger than the mean of change to determine if the changes were clinically significant (Table SI).

DISCUSSION

The aim of the present study was to test the reproducibility and the responsiveness of the BBT and JTTHF in children with CP. Our results confirmed the reliability of the tests when the assessments were performed within 2 weeks for the JTTHF, regardless of the hand, and for the BBT on the less affected hand. These results were consistent with the ICC. While assessing the responsiveness, both tests showed differences during intensive interventions, except for the JTTHF on the less affected hand.

The data of this study, notably observed in the whole sample analysis, confirm an impairment in dexterity of the children with CP, regardless of the time of assessment and the presence or absence of intervention. This is obvious when comparing our data to the norms described for typically developing children by Mathiowetz et al. for the BBT,³¹ and Taylor et al.¹⁸ and Reedman et al.¹⁶ for the JTTHF.

Reliability

When observing the reliability, the whole sample approach showed significant differences between the two assessments time for both tests. In the group approach, when measured in a brief period (2wks) the stability was present in both tests except for the BBT on the more affected hand. However, if we observe our main results, the mean change calculated over this brief period without intervention (Table IIb) was below the MCID calculated in between intervention (Table SI) for both tests in both hands. This could indicate that, despite the trial effect highlighted by the reliability study, the change observed did not reach a clinically meaningful difference. Therefore, this measure could be used to show an intervention effect over a brief period provided the MCID is taken into account. Conversely, the stability of the data was present only in the less affected hand for the JTTHF when assessed within a longer period. Moreover, the mean of change for a longer period (without intervention [Table IIb]) exceeds the MCID in all measures (Table SI). These results over a longer period could be explained by the physiological development of children with CP in between the two assessments.

High coefficients of ICC between A1 and A2 suggest that both tests have good test–retest reliability when assessed within a brief period. These results are congruent with those of Jongbloed-Pereboom et al.¹⁵ that provided normative values of the BBT for typically developing children aged 3 to 10 years in a sample of 215 children.¹⁵ Their results for the ICC test–retest and inter–rater reliability showed very good reproducibility. In the same way, the test–retest reliability of JTTHF has been tested by Reedman et al.¹⁶ using ICC and the standard error of measurement, within a sample of 87 typically developing children aged between 5 and 10 years considering all items of the test except the writing item. Their findings indicated a good test–retest reliability (within 12d, [SD 13]) for the JTTHF.¹⁶

Responsiveness

When observing the responsiveness, the same results were observed in the whole sample and the group approach regarding the age, MACS level, and topography. As expected, these results show an improvement in the dexterity of both hands in the BBT. This may be explained by the characteristics of hand and arm bimanual intensive therapy and hand and arm bimanual intensive therapy including lower extremity, intensive therapies based on motor skill learning principles using the constant engagement of both hands through bimanual activities.^2,23 Thus, the BBT, which assess a single task commonly performed in daily life activities either with one hand or the other, is sensitive to changes in both hands produced by bilateral intensive interventions. Conversely, the JTTHF, which assesses several tasks simulating some activities of daily living usually performed with the dominant hand, showed improvements only on the more affected hand, probably because the more affected hand has a larger potential for improvement (not usually used for precise tasks).

In the group approach regarding the MACS level, the effect size showed changes in both hands for the BBT. However, the MACS level I group presented a large change on the more affected hand when compared with the other MACS levels, while the MACS level III group presented a large difference on the less affected hand for this test when compared with the other MACS levels. We can speculate that children in MACS level I usually present a less affected hand with important manual abilities in function of the capacity of each child, hence bimanual training mostly induces changes on the more affected hand. This is in agreement with studies showing that young children in MACS level I have a rapid development of bimanual performance. Moreover, these children reach the maximal predicted improvement of bimanual performance faster than children classified in MACS levels II and III.^32,33 Children in MACS level III may have a lower performance in these tests on both hands, therefore improvements in both hands can be targeted. These results are congruent with those of MCIDs. Hence, our results may be related with the bimanual aspect of the therapy promoting the use of both hands and so allowing both to improve their performance.

In the group approach regarding the topography, we observed similar results in children with unilateral and bilateral impairments confirming the responsiveness for the BBT in both hands and for the JTTHF on the more affected hand. Although our results on the responsiveness are similar between both groups (improvement after intervention), the mean values at each assessment differed between groups, showing the specificity of each topography. Such differences between children with unilateral and bilateral CP have been also observed when children with CP where classified with the Bimanual Fine Motor Function.³⁴ Conversely, two studies observing outcomes in bimanual performance in children with unilateral and bilateral impairments assessed with the Assisting Hand Assessment and the Both Hands Assessment^32,35 showed similar results in both groups.

Both tests included in this study have been widely used in clinical and research contexts. Despite their wide use in testing manual abilities in children with CP, their psychometric properties have been tested mainly in adults and typically developing children.^{8,9,11,12,15,16,24,31} Our results demonstrate the reliability and responsiveness of both tests in children with CP in a short period of intervention. We suggest that a control group or normative values are needed when the assessment is performed in a longer period to disentangle the effect of an intervention from the physiological evolution of the participant.

A limitation of this study was the difference in the sample of children presenting with a unilateral and a bilateral form of CP. Indeed, most children presented with a unilateral form which may influence the results obtained for the more and less affected hand. However, when performing the analyses by topography separately, the results are similar in both groups. Despite similar demographic features (sex, age, and affected side) between both groups, there is an over-representation of children in MACS level II in comparison with other levels. However, when performing the analyses by MACS separately, comparable results were obtained, thus we expect that increasing the sample should lead to similar outcomes. Finally, in this study, the MCID is based on a numerical approach and not on a patient/therapist approach which may be a limitation to this measure.

Conclusion

In conclusion, reliability has been demonstrated when assessments are performed within a brief period of time for the JTTHF and the BBT. These outcomes confirm our hypothesis where both tests are sufficiently responsive to highlight the effects of an intensive intervention over a short period of time. Therefore, we suggest keeping both tests in the assessment battery for measuring changes in studies focusing on dexterity and hand function improvement in children with CP. Conversely, we suggest a cautious interpretation of results for interventions over longer periods since intervention results might be contaminated by spontaneous developmental changes in dexterity of children with CP.

Supplementary Material

The following additional material may be found online:

Appendix S1:

Inclusion and exclusion criteria

Table SI:

Responsiveness of the Box and Block Test and Jebsen-Taylor Test of Hand Function using a whole sample and a group approach

Figure S1:

Design of the study with the two separate folds, reliability and responsiveness of the measures

What this paper adds:

• The Box and Block Test (BBT) is reliable in a brief period of assessment in children with cerebral palsy (CP).

• The Jebsen-Taylor Test of Hand Function (JTTHF) is reliable in a brief period of assessment in children with CP.

• The JTTHF and BBT are responsive to changes in a brief period of intensive therapy in children with CP.

• The reliability and responsiveness of the JTTHF and BBT are weak over long assessment periods.

ABBREVIATIONS

BBT

Box and Block Test

ICC

Intraclass correlation coefficient

JTTHF

Jebsen-Taylor Test of Hand Function

MACS

Manual Ability Classification System

MCID

Minimal clinically important difference