Preliminary Reliability and Validity of a Battery for Assessing Functional Skills in Children with Sturge-Weber Syndrome

Abstract

Purpose

The purpose of this study was to evaluate inter-rater reliability and validity of a proposed functional outcome battery for clinical trials in children with Sturge-Weber Syndrome (SWS).

Methods

10 children were evaluated twice on the same day using a series of functional outcome measures selected for sensitivity to the range of age and function of children with SWS: Modified Rankin Scale, Pediatric Evaluation of Disability Index, Modified House Functional Classification, and a modified version of the Erhardt Developmental Prehension Assessment. Inter-rater reliability was calculated, and criterion validity was explored through correlations with the Sturge-Weber Syndrome-Neurological Rating Score (SWS-NRS).

Results

Inter-rater reliability was high across all measures. Correlations were identified between the SWS-NRS and the study measures.

Conclusions

The proposed battery of functional outcome measures captures child’s functioning at the levels of impairment, activity and participation and is robust to evaluation by different raters and across sessions on the same day. This battery is expected to be sensitive to treatment-related changes in qualitative patterns of hand use, functional skills, and/or change in independence in daily living.

Introduction

Sturge-Weber Syndrome (SWS) is a rare neurocutaneous disorder that frequently results in functional deficits. As clinical trials begin in SWS, a battery of functional outcome measures will be needed that is applicable to the heterogeneous population of SWS patients. Our goal is to propose and validate a battery of measures that quantifies upper extremity motor skills and independence with daily activities and measures function across the domains of impairment, activity and participation.

Due to brain involvement in SWS, functional deficits frequently result and may be due to hemiparesis or other motor impairments, cognitive/behavioral dysfunction, epilepsy, and/or visual field cut [1,27,31,21]. Prior work has highlighted significant variability in clinical presentation of children with SWS [37,32]. Furthermore, individuals with SWS may show notable changes in function in association with seizures and stroke-like episodes [32].

The underlying somatic mosaic mutation for SWS has been recently reported [30], providing important insights into pathogenesis and potential targets for treatment strategies. As therapeutic strategies for Sturge-Weber Syndrome are being proposed [e.g. [23,22]] and targeted, the field is preparing for clinical trials in this population. Along with important measures of disease severity, such as frequency of seizures and stroke-like events, an important goal of intervention is maintenance or improvement of functional skills. Clinical studies will thus need to incorporate a battery of functional outcome measures, which are sensitive to the range of function observed in this population and demonstrate reliability over multiple administrations.

Given the rarity of SWS, it is anticipated that clinical trials will enroll individuals over a broad age range and functional level. Furthermore, given that therapy may be most effective if started at a very young age, prior to onset of seizures and/or acquired functional deficits, the ability to assess the functional status of infants will be important. Additionally, ideal assessments will make use of readily available items, allowing for cost-effective assessment of children in a multi-site project.

We performed an extensive literature search for common measures of upper extremity and activities daily living (ADL) function. Of the assessments identified, many possible measures were limited by the need to capture the function of very young children. Based on the identified importance of using outcome measures across the World Health Organization (WHO) International Classification of Functioning to characterize pediatric neurological disorders [11], we chose an assessment battery that would evaluate functioning at the impairment, activity and participation levels. The WHO defines impairment as a problem in the structure or function of the body, activity as the performance of an action or task by a person, and participation as engagement in a life situation [34,35]. The purpose of this study was to explore the reliability and validity of the selected battery of tests in children with SWS.

Methods

This study was approved by the Johns Hopkins Medicine Institutional Review Board. Parental written informed consent was obtained for each parent-child dyad.

Participants

A convenience sample of ten children with SWS ranging from 9 months to 11 years old was enrolled in this study in conjunction with clinical evaluations in the Hunter Nelson Sturge-Weber Syndrome Center. Diagnosis of SWS was confirmed by the SWS Center Director (A.C.) based on clinical and imaging findings.

Overview of Measures; see also Table 1

Table 1.

Summary of Functional Outcome Measures

Measure	Description/ Outcome Assessed	ICF level	Age norms	Time to complete	Advantages	Limitations
Modified Rankin Scale (mRS):	6 category observational global measure of patient participation and residual disability post stroke	Participation	Designed for adults; some evidence for use with children	~5 minutes	NIH Common Data Element for stroke; this is the only measure within the current battery which captures death as a possible outcome.	Originally designed for adults. Clarification/ adaptation of description of outcomes typically used in pediatric studies to enhance consistency and applicability to children.
Pediatric Evaluation of Disability Index (PEDI)	197 item parent- report/structured interview regarding child’s level of participation in the functional domains of self-care, mobility and social function.	Participation	6 months-7 years (but prior use in older children with disabilities)	~30 minutes	Tasks are broken down into a developmental progression, reducing the likelihood of scoring at the floor of the measure and making the test sensitive to small changes in function. The information can be obtained despite the child’s level of participation during an evaluation session.	Relies on parent- report. Limited range of norms (but expanded PEDI-CAT now available which covers birth-20 years). Does not capture whether functional tasks are completed in a typical or modified fashion.
Modified House Functional Classification (MHC):	32 items/ 9 category observational assessment of function of a hemiparetic upper extremity	Activity and Impairment	2 years old and older	~10 minutes	Tests a large range of arm and hand skills in a short period of time. Easy to administer in a child-friendly fashion. The more affected arm is assessed in addition to evaluation of some bimanual tasks such as cutting with scissors.	Published studies use the MHC for children no younger than 2 years old, but there are items across 8 categories which are applicable to age- appropriate arm function in children as young as 6 months.
Modified Erhardt Developmental Prehension Assessment (mEDPA)	6 subscales of an observational, qualitative measure for tracking detailed gradations in fine motor function	Activity and Impairment	Natal-6 years	~10 minutes	Applicable for infants of any age. Sensitive to abnormal patterns of hand use. Provides assessment of both hands.	Requires precise observation by trained examiner.

Modified Rankin Scale (mRS): The mRS is a National Institute of Health common data element for stroke. It is a 6 point standardized scale that assesses patient participation and residual disability post stroke which includes death as a possible outcome. Raters complete an online training (http://rankin-asa.trainingcampus.net/uas/modules/trees/windex.aspx). The mRS can be quickly and efficiently completed; however, it was designed for adults, and the criteria used for assessing independence are not developmentally appropriate for children. While good inter-observer reliability has been reported with use in children [3], some authors have combined the mRS with age-appropriate markers of function (i.e. need for school modifications) [4]. In an effort to improve inter-rater reliability for the current study, supplemental descriptions for scores 1, 2, and 3, mutually agreed upon by the evaluators (T.R. and S.S.) were used when rating the mRS. A score of 1, “No significant disability despite symptoms; able to carry out all usual duties and activities,” was further described as a child who “performs at an age-appropriate level in activities of daily living (ADLs), school, and play activities.” A score of 2 “Slight disability; unable to carry out all previous activities, but able to look after own affairs without assistance,” was further described as a child who “requires assistance with ADLs but does not need special education services nor additional supervision for play.” A score of 3, “Moderate disability; requiring some help, but able to walk without assistance,” was further described as a child who “requires special education services.”

Pediatric Evaluation of Disability Index (PEDI) [12]: The PEDI is a parent report questionnaire and structured interview that assesses a child’s level of participation in the functional domains of self-care, mobility and social function. In less than 30 minutes, information is acquired about multiple aspects of home and community functioning. Tasks in these domains are broken down into a developmental progression, reducing the likelihood of scoring at the floor of the measure and making the test sensitive to small changes in function. A Caregiver Assistance and Modification subsection provides additional information about caregiver burden in relation to these areas of functioning. The unique design and holistic nature of this assessment allows it to include children with a combination of physical and cognitive disabilities [33]. The tool can also discriminate between a child with and without a disability [8]. There are numerous papers confirming the reliability and validity of this tool as well as its ability to detect change over time [for review, see [19]]. The PEDI has been used previously as an outcome tool for children with hemiplegia [26,5].

The PEDI is normed for children 6 months through 7 years; however, it was also used with older participants in this study. Use in older children has been reported in other studies of children with disabilities [e.g. [14,36]] in which many of the participants are functioning below age-based expectations and therefore did not reach the test ceiling. In the current study, raw scores were used for determining inter-rater reliability and for evaluating scores in relationship to the floor and the ceiling of the measure. Scaled scores were used for summary purposes and correlations with other measures. Scores were examined for child’s independence in self-care, mobility, and social “functional” domains as well as “caregiver assistance” needs for each of those three domains.

Modified House Functional Classification (MHC): The House Classification was initially presented as a means for clinically categorizing function of a hemiparetic arm/hand; the scale consists of 9 categories ranging from “does not use” to “spontaneous use, complete” [17]; in the MHC, specific tasks for use in assessment were delineated [20]. This observational tool assesses the affected arm/hand through functional unimanual and bimanual skills such as opening a marker, cutting with scissors, and picking up small items from a table top surface. Tasks are scored as achieved or not achieved. The MHC was described to have good construct validity with the Manual Ability Classification Scale and ABILHAND-Kids [10]. A high level of intra-rater (0.96) and inter-rater reliability (ICC=0.92) have been shown [20], and the MHC has been used as an outcome measure of treatment to improve upper limb function [29,28]. There is a high ceiling for test items, and the assessment is sensitive to typical hemiplegic compensatory movement patterns and limitations. Two scores were assessed; the total number of items that a participant was able to complete (“items”) and the highest category for which the participant could complete all tasks (“category”).

Modified Erhardt Developmental Prehension Assessment (mEDPA) [6]: The EDPA is a qualitative measure for tracking hand skill development that assesses detailed gradations of fine motor function as assessed by 17 tasks. The EDPA captures the functional capacity of very young children and those that have very limited hand function. This measure provides a description of not only what the child achieves but how the child achieves it; in providing a measure of the quality of movement, it complements the MHC well. Additionally, the EDPA assesses function of both the dominant and non-dominant hand, which is important in the SWS population, as some children show motor impairments bilaterally. Intraclass correlations for inter-rater reliability were found to range from .42–.85, for the original unmodified version [7].

With permission from the author (Erhardt, 2010, personal communication) the authors modified use of the EDPA by using 6 subscales which examine single-step, voluntary hand and finger movements. Typical use of the EDPA includes qualitative evaluation of highly specific motor components for each developmental level of task completion; for the purposes of this study, each subtest was scored by noting the highest developmental level at which the participant achieved all motor components. An ordinal scale was created for each subtest by sequentially numbering the developmental stages provided for that subtest. For the purpose of examining sensitivity and validity, summed scores of the ordinal rankings were created for the dominant hand and non-dominant hand separately.

Sturge-Weber Syndrome-Neurological Rating Score (SWS-NRS): As previously described [18], the SWS-NRS provides a clinically derived rating of visual field cut, frequency of seizures, hemiparesis, and age-based evaluation of cognitive functioning (see Table 2). The total score is a sum of subscores and can range from 0 (no neurological impairment) to 15. SWS scores were assigned by the SWS Center Director (A.C.) during clinical visits; the SWS score closest in time to the date of the study visit was used; time between SWS score and study visits ranged from 0–85 days (median=0 days).

Table 2.

SWS Neurological Rating System [18]

Seizure score

0 _ None ever 1 _ One or more seizures, currently controlled 2 _ Breakthrough seizures 3 _ Monthly seizures 4 _ At least weekly seizures

Hemiparesis score

0 _ No weakness or posturing 1 _ Mild posturing intermittently 2 _ Fine motor impairments only 3 _ Significant fine and gross motor impairments 4 _ Severe fine and gross motor impairment, poor helper arm function, and walks with great difficulty or not at all

Visual field-cut score

0 _ No field-cut 1 _ Partial homonymous hemianopsia 2 _ Full homonymous hemianopsia visual field-cut

Cognitive function score

Infant/preschool 0 _ Normal 1 _ Mild speech delay but comprehends well 2 _ Mild delay in speech and comprehension 3 _ Moderately delayed speech 4 _ Severely delayed speech 5 _ Profoundly delayed speech with little or no comprehension

Child

0 _ Normal 1 _ School difficulties, regular classes 2 _ Resource help needed in school 3 _ Special education required 4 _ Trainable for activities of daily living 5 _ Full care

Adult

0 _ Normal 1 _ Lives and works independently 2 _ Works in community with parental support 3 _ Significant difficulty maintaining employment or satisfactory social relationships 4 _ Trainable (i.e., group home, supervised work setting) 5 _ Full care

Procedures

The testing battery (mRS, PEDI, MHC, mEDPA) was performed twice on same day by two different examiners [a pediatric occupational therapist (T.G.R.) and a pediatric physiatrist (S.S.)]. These two examiners evaluated all children and were blinded to each other’s results on the day of testing. The two evaluation sessions were separated by several hours. Each testing session was less than one hour in duration.

Prior to test administration, examiners agreed upon using a specific, height adjustable table and providing foot support to participants when feasible in order to support optimal postural stability. When variations to this set-up existed, most often due to size/height of the child or cooperation level (e.g., preferred sitting on parent’s lap for testing), this information was provided from the first examiner to the second examiner, in an effort to ensure replication of testing position in order to limit position as a factor on assessment performance. The same parent completed PEDI in both testing sessions.

Items from MHC and mEDPA were combined for administration in a way that flowed sequentially between tasks in a play based fashion. For example, tasks requiring grasping and manipulating a marker, drawing, and cutting were placed in sequence to encourage natural engagement of the child in play-like activities. The order of selected tasks was also carefully determined by the occupational therapist (T.G.R) to reflect pediatric clinical experience in order to optimize the child’s participation. For example, drawing is an enjoyable experience that children often have difficulty terminating, so this task was placed last in the battery. Tasks were presented in a standardized order with standardized verbal prompts and demonstrations (see Online Resource 1), and the same materials were used across sessions to elicit desired behaviors. Examiners established a set number of trials (3) permitted for each task to control for inconsistency; best performance on any of the three trials was scored.

Statistical Analyses

All statistical analyses were performed in SPSS 20 with statistical significance at p<0.05. Descriptive statistics were used to examine sample characteristics. Inter-rater reliability was evaluated through two-way mixed, consistency, single-measures intraclass correlations (ICC) as an equivalent measure for the quadratic-weighted weighted kappa statistic [9,15]. Strength of Kappa was judged per Landis and Koch: ≤0=poor, .01–.20=slight, .21–.40=fair, .41–.60=moderate, .61–.80=substantial, and .81–1=almost perfect [24]. Spearman’s correlations and overall agreement are also reported as measures of inter-rater reliability. The number of participants with an assigned score from either rater at the floor or ceiling for the selected measures was examined to assess sensitivity to the full range of physical abilities and motor skills in this population. Criterion validity was explored through correlations of mean scores (mean of Rater 1 + Rater 2) for mRS, PEDI, MHC, and mEDPA with SWS-NRS scores. As all of the ordinal scales contained five or more rankings, Pearson’s correlations were performed and were controlled for child’s age via partial correlation. Strength of correlations, based on correlation coefficient, was assigned using the following parameters: r=0–0.2 (very weak), r=0.2–0.4 (weak), r=0.4–0.7 (moderate), r=0.7–0.9 (strong), r=0.9–1.0 (very strong). Partial correlations, controlling for age, were used to examine associations among the mean scores from the selected measures.

Results

Demographics

Children ranged in age from 9–136 months; there were 6 girls and 4 boys. Four participants had bilateral brain findings of SWS; the other 6 had unilateral brain involvement (3 right hemisphere, 3 left hemisphere). Three participants had bilateral skin involvement, 6 had unilateral skin involvement, and 1 had no skin involvement. Three participants had bilateral eye involvement, 4 had unilateral eye involvement, and 3 had no eye involvement.

Total SWS-NRS scores ranged from 2–10; hemiparesis subscale scores ranged from 0–4 (mean 1.6), cognitive subscale scores ranged from 0–4 (mean 1.9), seizure subscale scores ranged from 1–4 (mean 1.9), and visual field cut scores ranged from 0–2 (mean 0.5). One child was post-ictal and somnolent on the day of the study visit and was unable to participate in active assessment; for this child only the parent-report measures (mRS and PEDI) were completed. For a different child PEDI Caregiver Assistance data were not obtained. Participant ages and performance on the testing battery are summarized in Table 3.

Table 3.

Summary Data for Age and Performance on Testing Battery

	Mean	Median	Standard deviation	Range
Age (months)	71.5	64.5	49.3	9–136
SWS Neuroscore	5.9	6.00	2.9	2–10
Modified Rankin Scale (MRS)	3.4	3.0	.7	3–5
Modified House Classification (MHC) variables
Total items completed	21.7	22.5	8.1	9–32
Highest category for which all items completed	4.7	5.0	2.0	2–8
PEDIa variables
self-care scaled score	53.1	58.1	24.4	8.6–81.4
mobility scaled score	64.8	72.8	29.1	10.65–100
social scaled score	48.9	56.6	21.9	10.65–82.2
self-care Caregiver assistance	45.0	52.1	28.6	0–79.5
mobility Caregiver assistance	62.0	70.9	35.8	0–100
social Caregiver assistance	42.1	47.8	30.6	0–78.6
mEDPAb items (developmental level, in months)
Dc cube grasp	8.0	9.0	1.4	5.5–9.0
D cube release	9.8	12.0	3.3	3.0–12.0
D pellet grasp	9.7	10.5	2.8	5.0–12.0
D pellet release	11.1	14.0	6.4	0.0–15.0
D dowel grasp	9.1	10.0	1.4	7.0–10.0
D pencil grasp	2.7	2.5	1.8	1.0–5.0
NDd cube grasp	2.8	9.0	3.0	0.0–9.0
ND cube release	8.9	10.0	4.4	0.0–12.0
ND pellet grasp	8.8	10.0	3.4	2.50–12.0
ND pellet release	9.8	14.0	6.5	0.0–15.0
ND dowel grasp	8.1	10.0	3.4	0.0–10.0
ND pencil grasp	2.0	1.5	1.3	1.0–4.0

Data reflects the scores collected by both raters (two scores per measure per child).

^aPEDI =Pediatric Evaluation of Disability Index

^bmEDPA= modified Erhardt Developmental Prehension Assessment

^cD=Dominant Hand

^dND=Non-dominant Hand

Inter-rater reliability

Inter-rater reliability data are provided in Table 4. Kappa for mRS, PEDI functional domain scores, and MHC were almost perfect. Kappa for PEDI Caregiver Assistance scores ranged from substantial to almost perfect. Kappa for 10 of 12 items of the mEDPA was also almost perfect, with values for dominant hand cube grasp and non-dominant hand pellet grasp falling into the substantial range. Correlation coefficients for the mRS, PEDI functional domain scores, and MHC were very strong. Correlation coefficients for the PEDI Caregiver Assistance scales and mEDPA tasks ranged from strong to very strong.

Table 4.

Inter-rater reliability measures for the selected functional battery

Measure	ICC*	Spearman correlation		Absolute agreement	Range of discrepancy in ratings	Percentage of discrepant ratings with 1 point discrepancy
		r	p
Modified Rankin Scale (MRS) n=10	.82	.81	.005	80%	1 category	100%
Modified House Classification (MHC) variables n=9
House total items	.94	.95	<.001	22%	1–7 items	29%
House category all items met	.98	.98	<.001	78%	1 category	100%
PEDIa variables
PEDI self-care score n=10	1.0	1.0	<.001	30%	0–4 points	57%
PEDI mobility score n=10	1.0	1.0	<.001	30%	0–2 points	86%
PEDI social score n=10	.99	.97	<.001	20%	0–6 points	25%
PEDI self-care Caregiver assistance n=9	.99	.97	<.001	11%	0–3 points	25%
PEDI mobility Caregiver assistance n=9	.99	.97	<.001	67%	0–5 points	33%
PEDI social Caregiver assistance n=9	.77	.88	.002	33%	0–10 points	17%
mEDPAb items n=9
Dc hand cube grasp	.77	.82	.007	56%	1–2 categories	50%
D hand cube release	.86	.88	.002	78%	1–5 categories	50%
D hand pellet grasp	.93	.89	.001	56%	1–2 categories	75%
D hand pellet release	.94	.99	<.001	89%	3 categories	0%
D hand dowel grasp	.89	.98	<.001	89%	2 categories	0%
D hand pencil grasp	.88	.90	.001	89%	2 categories	0%
NDd hand cube grasp	.95	.85	.004	78%	1–2 categories	50%
ND hand cube release	.97	.82	.008	78%	1–2 categories	50%
ND hand pellet grasp	.93	.85	.003	44%	1–2 categories	60%
ND hand pellet release	.84	.92	<.001	78%	3–4 categories	0%
ND hand dowel grasp	.98	.92	.001	78%	1 category	100%
ND hand pencil grasp	.85	.86	.003	67%	1 category	100%

^*two-way mixed, consistency, single-measures ICC

^aPEDI =Pediatric Evaluation of Disability Index

^bmEDPA= modified Erhardt Developmental Prehension Assessment

^cD=Dominant Hand

^dND=Non-dominant Hand

Floor and Ceiling Effects

For the mRS, no child scored at the ceiling; while the absolute floor of the measure is death, 2 children were assigned the lowest possible score for a living child. For the MHC, one child received a score at the ceiling, and no child received a score at the floor. For the PEDI (raw scores), one child scored at the ceiling for mobility skills; no child performed at the ceiling for self-care or social skill domains, and no child scored at the floor in any domain. For the caregiver assistance scores, two children received scores at the ceiling, and three children received scores at the floor. Five children scored at the floor of at least one subtest of the mEDPA, but no child scored at the floor for all subtests. Six children scored at the ceiling of at least one subtest of the mEDPA, but no child scored at the ceiling across all subtests.

Validity

Correlations with SWS-NRS scores are shown in Table 5. mRS score showed a trend toward correlation with Total SWS-NRS. PEDI mobility scaled score was correlated with Total SWS-NRS, and all PEDI functional scale subscales were correlated with SWS-NRS Cognitive subscore at p=.05. MHC Items was significantly correlated with Total SWS-NRS and Hemiparesis subscore. Dominant summed score for mEDPA was significantly correlated with Total SWS-NRS score, and mEDPA Non-dominant summed score was significantly correlated with Total SWS-NRS and Cognitive subscore, with a trend toward correlation with Hemiparesis score.

Table 5.

Correlations between SWS-NRS scores and performance on selected measures

	Total Neuroscore		Hemiparesis Neuroscore		Cognitive Neuroscore
	r	p	r	p	r	p
Modified Rankin Scale (MRS)	.61	.08	.41	.28	.33	.38
Modified House Classification (MHC) variables
Total items completed	−.88	.004	−.73	.04	−.57	.14
Highest category for which all items completed	−.53	.18	−.29	.48	−.33	.43
PEDIa variables
self-care scaled score	−.53	.14	−.18	.65	−.66	.05
mobility scaled score	−.79	.01	−.49	.18	−.66	.05
social scaled score	−.48	.19	−.16	.67	−.67	.047
mEDPAb variables
Dominant hand sum	−.71	.05	−.19	.65	−.32	44
Non-Dominant hand sum	−.84	.008	−.65	.08	−.72	.04

All correlations are controlled for age

^aPEDI =Pediatric Evaluation of Disability Index

^bmEDPA= modified Erhardt Developmental Prehension Assessment

Shaded boxes note trends toward correlation

Bolded values note significant correlations

Correlations among measures

mRS, PEDI functional domain and caregiver assistance, MHC, and mEDPA scores were strongly to very strongly correlated with each other, with the exception of PEDI social functional skills not being correlated with either mRS or PEDI self-care caregiver assistance.

Discussion

The purpose of this study was to explore reliability and validity of a proposed assessment battery consisting of the mRS, PEDI, MHC, and a modified version of the EDPA for describing the function of children with SWS. In this pilot sample, the battery demonstrated excellent inter-rater reliability when performed twice on one day, with separate evaluations performed by a pediatric occupational therapist (OT) with expertise in children with hemiparesis and a pediatric physiatrist. This suggests that the chosen battery, when performed in a standardized fashion, is expected to be robust not only in the face of different evaluators but also to an individual child’s variability within a given day.

The SWS-NRS is the current standard for describing the disease-related function of children with SWS, with SWS-NRS scores correlating with brain atrophy [18], brain perfusion and brain metabolite ratios [25], and quantitative EEG [16]. Correlations of the proposed functional evaluation battery with SWS-NRS demonstrate criterion validity for this battery but also highlight that function in a child with SWS is not mediated by motor function alone; in particular, cognitive function is an important mediator of independence with daily living skills, as demonstrated by correlations between the SWS-NRS Cognitive subscale score and PEDI scores. Likewise, the Total SWS-NRS, describing not only motor and cognitive function but also seizure frequency and visual field cut, is correlated with at least one score from all tests examined, reinforcing the previously described importance of overall disease stability in function in children with SWS [32]. The lack of correlation between SWS-NRS Hemiparesis and Cognitive subscale scores and portions of the proposed functional evaluation battery underscore the importance of outcome measures which specifically target the real-world functional goal of treatment (e.g. improved hand use or self-care skills) rather than solely relying on the SWS-NRS as a proxy of a child’s function.

The combination of parent-report measures (mRS and PEDI) which reflect a child’s real-world skills with measures requiring direct evaluation of the child’s skills (MHC and mEDPA) is felt to be a strength of this testing battery. There are benefits and challenges of assessment via parent report; one benefit is that information can be obtained despite the child’s level of cooperation/participation during an evaluation session. In this study one child was post-ictal and could not participate in direct evaluation due to lethargy, yet functional information was still obtained through the parent-report measures. A challenge is that the available parent must be able to report on their child’s functioning, though from the clinical experience of the OT in this study, this is not often an issue with this population of children. During this study no parent had difficulty reporting on function, nor did test responses contradict behavioral observations. Furthermore, parent-report allows assessment within the ICF domain of participation, which otherwise would be very difficult to achieve without lengthy observations of the child in his or her typical environments.

For the mRS, agreed-upon child-relevant definitions of functional categories, as were used here, are felt to be key to achieving reliable use within a pediatric population. While scores on the mRS were restricted within this small cohort, additional variability is expected if applied to a broader SWS population. Additionally, this is the only measure within the current battery which captures death as a possible outcome.

A computer-based version of the PEDI has now been released (http://pedicat.com/category/home/) and is normed for children from birth through 20 years of age. This version assess the functional domains of self-care, mobility, and social skills and has an added domain of “responsibility” which captures how much responsibility a child takes for activities that enable independent living and replaces the Caregiver Assistance scales. It is expected that use of this newer, computerized version will reduce variability that was observed in the Caregiver Assistance interview portion of the PEDI. Additional benefits are that this computerized version does not require time of a skilled examiner for completion, and this version allows assessment of skills over a larger range of typical function, thereby raising the ceiling of the measure. Because the PEDI-CAT integrates item response theory, administration time is also expected to be much shorter compared to the PEDI. Given these benefits, for future studies we recommend use of the PEDI-CAT in place of the PEDI.

The MHC and mEDPA were felt to complement each other well for direct child observation in order to not only test a large range of arm and hand skills (MHC) but also to capture whether immature movement patterns were used (mEDPA). As the EDPA requires very precise observation for optimal description of a child’s motor patterns, the raters found it difficult to assess all aspects of movement qualitatively within a limited time period, especially given that the participants were unfamiliar to the examiners. Videotaping of MHC and mEDPA assessments would be useful if greater detail in description of qualitative movements is desired for either test; while qualitative assessment of movement is not currently captured as part of MHC, change in movement pattern for completion of tasks could change with improvement in function. Videotaping would also allow determination of intra-rater reliability as well as inter-rater reliability of scoring within a single session.

Despite the measures being strongly correlated with each other, the entire proposed battery is judged to be appropriate for use in SWS clinical trials if the course of change in function that will follow with treatment of SWS is not known. By maintaining the entire battery, until additional data suggest otherwise, outcome measures will allow capture of changes in qualitative patterns of hand use, functional skills, and/or change in independence in daily living. Furthermore, use of the complete battery in this cohort demonstrates the ability to detect improvement from current function, given that only rare scores at the ceiling were achieved, and no child achieved scores at the ceiling for every test. Over time, the battery of tests may be able to be shortened, based on observations from additional use.

Should future trials be designed to specifically evaluate one functional outcome, then the battery could be more focused. If only one measure is desired to rate global function, then the PEDI is recommended for use. Advantages of the PEDI include its psychometric validation in children as well as the availability of age-based norms, which would allow examination of whether a child is “catching up” to age-based norms or if he or she, though making functional gains over time, remains more “statically” delayed in comparison to age-based norms. Furthermore, the PEDI breaks functional skills into detailed elements, which may allow better assessment of change in developmental skills overtime, as opposed to the mRS in which global changes in overall independence, likely a longer term result of treatment, are required to observe a change in score. While in the current study absolute agreement was better for the mRS, likely due to the restricted number of scoring options, inter-rater reliability was better for PEDI. If one measure for evaluating hand function is desired, then the authors recommend use of the MHC, based on ease of administration and more consistent inter-rater reliability in the current study. Although published studies use the MHC for children no younger than 2 years old [10], potentially limiting its use for studies of infants, there are items across categories 1–7 which are applicable to age-appropriate arm function in children as young as 6 months; in the current study, a 9 month old child completed 10 items on the MHC.

Involvement of a pediatric OT in the selection of the current battery and orchestration of the testing protocol is felt to have been a key component contributing to the strong results achieved with this battery and is recommended for clinical trials. Occupational therapist (OTs) possess a unique skill set that covers multiple domains of functioning [2]. OTs’ training allows them to assess how a person’s component skills set such as vision, posture, dexterity, cognition, strength and range of motion interact with the environment and task demands to impact the execution of self-care and functional activities. In this study the OT was essential in selecting appropriate assessment tools that provided a comprehensive battery. The OT also chose the order of measures, materials, and flow of evaluation to encourage an accessible and play-based testing atmosphere. Furthermore, in qualitative examination of the data, the authors found that the OT was more perceptive to delays in hand function, deviant grasp patterns, posture, and seating impact on performance and common hand skill/self-care deficits associated with hemiplegia/neuromotor deficits. Thus, investigators are encouraged to utilize experienced pediatric OTs to complete such assessments, based on their astute clinical observation skills.

The current data reflect only an initial evaluation of the reliability and validity of these measures in this population. Limitations of these data include the small sample size and the need to document, in future studies, the responsiveness of these measures to change, though the PEDI and MHC have been used in other populations to document response to interventions [29,28,36,13]. While a large age range within childhood was captured in the current sample, the youngest child in this cohort was 9 months of age, and therefore additional evaluation is needed to demonstrate reliability in younger infants, especially given that this age group may be of particular interest with regard to SWS clinical trials. Nevertheless, these data reflect an important first step in creating a standardized battery of functional outcome measures for children with SWS.