Consensus-Based Evaluation of Outcome Measures in Pediatric Stroke Care: A Toolkit

Abstract

Following a pediatric stroke, outcome measures selected for monitoring functional recovery and development vary widely. We sought to develop a toolkit of outcome measures that are currently available to clinicians, possess strong psychometric properties, and are feasible for use within clinical settings. A multidisciplinary group of clinicians and scientists from the International Pediatric Stroke Organization comprehensively reviewed the quality of measures in multiple domains described in pediatric stroke populations including global performance, motor and cognitive function, language, quality of life, and behavior and adaptive functioning. The quality of each measure was evaluated using guidelines focused on responsiveness and sensitivity, reliability, validity, feasibility, and predictive utility. A total of 48 outcome measures were included and were rated by experts based on the available evidence within the literature supporting the strengths of their psychometric properties and practical use. Only three measures were found to be validated for use in pediatric stroke: the Pediatric Stroke Outcome Measure, the Pediatric Stroke Recurrence and Recovery Questionnaire, and the Pediatric Stroke Quality of Life Measure. However, multiple additional measures were deemed to have good psychometric properties and acceptable utility for assessing pediatric stroke outcomes. Strengths and weaknesses of commonly used measures including feasibility are highlighted to guide evidence-based and practicable outcome measure selection. Improving the coherence of outcome assessment will facilitate comparison of studies and enhance research and clinical care in children with stroke. Further work is urgently needed to close the gap and validate measures across all clinically significant domains in the pediatric stroke population.

Introduction

Pediatric stroke is associated with significant morbidity that can affect a child’s developmental trajectory.¹ Typical growth and maturation processes in the developing brain are frequently disrupted, resulting in a spectrum of neurological impairments.¹ Consequent deficits may include hemiparesis, language impairment such as aphasia, cognitive difficulties, and social-emotional problems, some of which only emerge later in childhood when developmental and educational demands increase.^2–4 In addition, variability in age at stroke, stroke etiology, and premorbid risk factors make predictions of poststroke recovery trajectories and outcomes challenging.^5–7 Outcome measures are one effective way to monitor recovery and screen for potential emerging deficits.⁸ Clinicians can evaluate initial deficits and track impairments during rehabilitation through the use of global functional outcome measures, which are brief, easy to administer, and broadly applicable. Outcome measures that focus on a particular area, such as motor function, language, cognition, adaptive function, or mood and behavior allow for a more precise understanding of a child’s strengths and weaknesses. These domain-specific measures often take longer to administer and might require expertise, but provide valuable insight into specific neurological impairments to inform individualized treatment recommendations.

In practical terms, outcome measures help clinicians make decisions about how to direct resources and care. Given their importance, measurement tools should possess high-quality psychometric properties. International programs such as the COSMIN initiative (Consensus-based Standards for the Selection of Health Measurement Instruments) have been developed to encourage the use of psychometrically sound health instruments.^9–11 COSMIN advocates for systematic evaluation of health instruments, including clinical outcome measures, through a framework with checklists that incorporate validity, reliability, internal consistency, responsiveness, and interpretability. This approach of identifying high-quality measures with a uniform set of criteria may also aid in the development of preferred or gold-standard instruments to standardize the collection of data across clinical sites and hospitals.

A systematic review in 2012 found wide variation in the use of outcome measures in pediatric stroke research, with 34 studies utilizing 38 different outcome measures.⁸ Unfortunately, such variation limits the comparison of studies. Harmonizing outcome measures across clinical sites would allow for easier cross-site comparison of pediatric stroke outcomes and treatment results. To encourage the adoption of a common set of outcome measures, we utilized a multidisciplinary team of health care providers with pediatric stroke expertise to evaluate a wide range of commonly used clinical outcome measures in pediatric stroke care using the COSMIN framework. The purpose of the evaluation was to develop an expert-informed compendium of outcome measures, a “toolkit” for the evaluation of children with stroke. Our goal was to provide clinicians and researchers with valuable information related to psychometric properties and practical features of measures to inform their selection within clinical settings and research studies.

Methods

The expert group for the current study was created and coordinated through the International Pediatric Stroke Study (IPSS) and International Pediatric Stroke Organization (IPSO) network, which consists of a multidisciplinary group of clinicians, scientists, and research staff. The IPSO members who participated in the evaluation process were selected to ensure diverse expertise in clinical backgrounds and research areas related to pediatric stroke. The group of 13 contributors included neurologists, a neurointensivist, neuropsychologists, a nurse practitioner, a physical therapist, and a speech and language pathologist.

To develop the measurement toolkit, the expert group evaluated the measures through the following stages.

Stage 1: identification of domains and generation of the initial list of measures for inclusion in the expert review

Commonly used measures for evaluating stroke-related impairment were identified within the clinical research literature and through existing recommendations for the following outcome domains of interest: (1) global performance; (2) motor function; (3) behavioral assessments and adaptive function; (4) cognitive function; (5) language; (6) quality of life (QoL); and (7) mood. The expert team met and reviewed the list of domains and their associated outcome measures. An initial list of measures was discussed to finalize a list for review before stage 2.

Stage 2: quality criteria and content for review measures and REDCap survey ratings of quality criteria

A survey was utilized to rate the importance of quality criteria for subsequent evaluation of the individual measures within the domains of interest. These quality criteria were adapted from the COSMIN guidelines international consensus on measurement properties.^10,11 Additional information to guide the ratings such as practical features and scoring information was reviewed further and approved by expert group members. The working group completed an anonymous REDCap survey in which each member rated the following quality criteria adapted from the COSMIN guidelines on a scale of 1 to 10 with 1 considered least important and 10 considered most important: responsiveness and sensitivity, reliability, repeatability, validity, feasibility, and predictive utility. The operationalized definitions for each quality criterion were provided to each group member to ensure consistent interpretation (refer to Supplementary Document 1).^10,11 The REDCap survey ratings of the importance of each quality criteria provided by each group member were rank ordered, and then each criterion was averaged across the 12 raters with a derived mean level of importance to the expert working group.

Stage 3: literature search and drafting of data tables

The literature search was performed within PubMed using search terms “stroke” OR “pediatric stroke” AND (“outcomes” OR “measures” OR “psychometrics” OR “rehabilitation” OR “neurological rehabilitation” OR “therapy” OR “recovery”). An additional search was undertaken in PubMed and on Google Scholar for each domain (e.g., “cognition” or “language”) with the aforementioned search terms, and for each individual measurement scale (e.g., “behavior rating inventory of executive function” or “clinical evaluation of language fundamentals”) that was included within the designated domain. Tables for each outcome domain were initially developed by a single working group member based on literature review. Scales were identified from the literature search according to quality criteria that included psychometric properties (i.e., responsiveness and sensitivity, reliability, repeatability, validity, feasibility, and predictive utility), time of administration, age range, scoring range, and practical features of each measure. The identified scales were then assigned to the appropriate domains (global performance, motor function, language, etc.) within the evaluation framework.

Stage 4: expert review of data tables

Three expert working group members were assigned to each domain of interest to verify the preliminary information in the table for accuracy and to ensure completeness (refer to Supplementary Tables 1–6). These assignments were determined according to the specialized knowledge and expertise of the members.

Stage 5: rating of individual measures using the modified COSMIN checklist

A modified version of the COSMIN checklist was utilized to assess the quality of each outcome measure (Table 1). With measure names removed to reduce bias, blinded coauthors reviewed all measures within each domain and rated the measures assigning a score for each measure from 1 to 5.

TABLE 1.

Modified COSMIN Checklist for Scoring of Outcome Measures

Score	Meaning	Description
5	Highly Recommend	• Excellent reliability (test-retest, inter-rater, intrarater, and/or responsiveness) • Excellent validity (content, construct, and criterion validity) • Easy to interpret scores (can be used by individuals of differing expertise or no training required) • Evidence of cross-cultural validity • Excellent clinical utility across a majority of the following: short, does not require a lot of supplies, applicable for many intellectual/physical ability levels, can readminister within short period of time, wide age range of assessment allows for follow-up within patient tracking over time) • Data to support psychometric utility is strong (validated in a pediatric stroke population or relevant/related populations) • Sensitive to change over time (minimal clinically important difference) has been established, low likelihood of ceiling or floor effects)
4	Recommend	• Good reliability • Good validity • Easy to interpret scores • Minimal evidence of cross-cultural validity • Strong clinical utility (meets half of the criteria) • Data to support psychometric utility is strong • Sensitive to change over time
3	Recommend, nonpreferred	• Minimal evidence of reliability • Minimal evidence of validity • Some barriers to score interpretation • Minimal or reported evidence of cross-cultural validity • Meets some clinical utility criteria • Data to support psychometric utility is strong but has not been validated in a pediatric stroke or relevant/related population or is an adult measure that has been adapted for use in pediatric populations • Sensitive to change over time—no published data
2	Unable to recommend at this time	• Limited information with regard to psychometrics but has clinical utility (might be a measure that is often used clinically but has little research available)
1	Do not recommend	Poor psychometrics and poor clinical utility or measure is rarely used so there is little to no data on the measure

Abbreviation:

COSMIN = Consensus-based Standards for the Selection of Health Measurement Instruments

Modified COSMIN Guidelines were used for rating the individual measures. A scale of1to 5 was utilized; if majority of the criteria (>5 items) were met within a given score category, the measure would receive that score. The reliability and validity criteria both had to be satisfied to meet a given score category. Definitions of each scoring criteria were provided (refer to Supplemental Document 1).

Stage 6: rating and recommendation review of measures for the toolkit

The rated measures were reviewed by additional members of the expert group who did not participate in stage 5, and these members provided comments and feedback.

Results

Stage 1

Outcome measures in the seven domains of interest were identified by the expert working group: (1) Global Performance (eight outcome measures); (2) Motor Function (eleven outcome measures); (3) Behavioral Assessments and Adaptive Function (eight outcome measures); (4) Language (six outcome measures); (5) Quality of Life (four measures); and (6) Cognitive Function (ten measures) and Mood (two measures).

Stage 2

Quality criteria that were selected included responsiveness and sensitivity, reliability, repeatability, validity, feasibility, and predictive utility. Based on working group input, the following content areas were added to the previously described modified COSMIN quality criteria: instrument description, scoring range, and practical features of the measurement tool. The REDCap survey results demonstrated that validity, sensitivity, and responsivity received the highest mean scores indicating their greater importance to the raters. Reliability and feasibility were also reported to be important, with slightly lower mean ratings than validity and sensitivity. Repeatability and describing a measure’s practice effect impact were the least important qualities to the majority of expert raters. However, across all measures, there was significant variability in importance ratings of a given criteria. For example, repeatability received the majority of the lowest ratings; however, two experts rated it highly (nine out of ten rating of importance).

Stage 3

7608 articles were screened via title review followed by abstract review when relevant on PubMed and Google Scholar during the literature search. Information from 304 articles and six technical manuals were referenced and used to populate the domain tables.

Stage 4

Domain tables were verified by group members (see Supplemental Tables 1–6).

Stage 5

Tables of domain-specific measures are summarized below and are organized from highest to lowest rated.

Global performance

The Pediatric Stroke Outcome Measure (PSOM) received the highest rating (mean = 4.66, range 4–5) among the eight scales (Table 2). The PSOM was recognized as being the only validated global composite performance measure developed specifically to assess outcomes in pediatric stroke. The PSOM was rated to have good to excellent validity and reliability and was noted to be easy to administer at the bedside. The Pediatric Stroke Recurrence and Recovery Questionnaire (psRRQ) and the Hammersmith Infant Neurological Examination (HINE) had the next highest ratings, respectively. Importantly, the psRRQ is a derivative of the PSOM, which can be conducted remotely, an important consideration in easing the burden of attending clinic and assessing outcome for children and families who live remote to the stroke center. The HINE was rated highly due to sensitivity and accuracy in identifying mild delays in infants with cerebral palsy relative to typically developing infants and based on evidence supporting its predictive validity, as it is highly correlated with gross motor function at two years and full-scale IQ scores.¹⁸ Other measures within the domain received lower scores because they were not specific, were generally more rough estimates of function, were not validated in pediatric samples, or required multiple informants to accurately identify a child’s functional capacity across a number of different domains making their use somewhat impractical except in team-based care settings. The shortcomings of many of the measures within the global performance domain are the limited range and lack of specificity of items that determine a child’s overall functioning; thus, substantial changes in functional ability would be necessary to shift scores to a different performance category (from mild to moderate or moderate to severe).^{22–24,28,29,31} The limited range and lack of specificity are desirable in that only clinically meaningful differences in scores are likely to be detected; however, this could limit detection of subtle changes that could prove important at an individual level.

TABLE 2.

Global Performance Measures

Outcome Measure	Ratings (Means and Range)	Pros	Cons
PSOM3,12–16	Mean: 4.66 Range: 4–5	• Wide age range • Rapid administration • Validated in pediatric stroke • Phone version available	• Cognitive and behavioral items limited in scope • Not validated for administration by nonneurologists • Limited dynamic range • Outcome values often clumped in 0–3 range
psRRQ16	Mean: 4 Range: 4	• Quick to administer • Phone use avoids missing outcomes • Excellent for working with individuals in rural communities who are difficult to access • Reliable estimator of the PSOM	• Validity data limited • Some internal consistency data rates relatively low (ICC = 0.5) • Limited dynamic range
HINE17–21	Mean: 4 Range: 4	• Good for identifying motor impairment in infants- can have the caretaker complete the examination—good test-retest and interrater reliability • Predicts walking and independent sitting in children with cerebral palsy • Standardized on large cohorts of children	• Limited age range, which makes it difficult to track recovery over time (ends at 24 months)
PCPC/POPC22–24	Mean: 3.33 Range: 2–4	• Does not require training • Does not require the patient to be present if medical chart is available	• No research published in pediatric stroke has used this measure, suggesting it is less commonly used • Not validated in pediatric stroke • Limited dynamic range
King’s Outcome Scale for Childhood Head Injury (KOSCHI)25–27	Mean: 3 Range: 3	• Wide age range- could be used in younger children if needed • Convergent validity with QoL measures • Similar to the psRRQ and phone version of the PSOM	• Validated in TBI, not pediatric stroke • Items measure different outcomes, grades 1 and 2 reflect physiological function, grades • 2 and 3 reflect awareness and response, and grades 3–5 measure functional independence • Scale places strong emphasis on concentration, behavior, and inhibition—common problems in pediatric stroke. Validity may be impacted in children with comorbid ADHD • Not a great predictor of behavior or emotions • Similar to the psRRQ and phone version of the PSOM • Limited dynamic range
GOS-Peds28,29	Mean: 2.33 Range: 2–3	• Good sensitivity to severity ofTBI in children, associated with changes in TBI sequelae over time • Validated for use in infants, toddlers, children, and adolescents	• Not validated in peds stroke • Limited dynamic range
mRS30	Mean: 2.33 Range: 2–3	• Well-validated adult disability scale	• Not validated for children • Limited dynamic range • Variable inter-rater reliability across mRS reliability studies
FSS31	Mean: 2.33 Range: 2–3	• Quick to use, minimal subjectivity • Applicable to broad age range • Commonly used in hospital environment	• Literature indicates it is not designed to predict outcome and should not be used to assess or predict outcomes for pediatric patients

Abbreviations:

ADHD = Attention-deficit/hyperactivity disorder

FSS = Functional Status Scale

GOS-Peds = Glasgow Outcome Scale-Peds

HINE = Hammersmith Infant Neurological Examination

ICC = Intraclass correlation coefficient

mRS = Modified Rankin Scale

PCPC = Pediatric Cerebral Performance Category

POPC = Pediatric Overall Performance Category

PSOM = Pediatric Stroke Outcome Measure

psRRQ = Pediatric Stroke Recurrence and Recovery Questionnaire

TBI = Traumatic brain injury

When only one number is given for range, all reviewers/raters agreed on that rating. Many are free but require training, refer to Supplemental Table 1 for specified cost.

Motor function

The Gross Motor Functional Measure (GMFM) was the highest rated outcome measure for motor function (refer to Table 3 and Supplemental Table 2) based on excellent validity and reliability in children with cerebral palsy.^32–38 The GMFM is designed to measure gross motor function over the course of typical development and therefore is sensitive to motor impairments and change over time. The GMFM also rates capacity to complete a movement rather than quality of performance.³⁶ For children with hemiparesis, this is a meaningful outcome because less fluid, but functional movements may allow children to complete activities of daily living. One significant drawback is that the GMFM is an hour-long assessment, which is not usually practical during a routine clinic appointment. Some of the alternative measures have not been validated in children or in relevant clinical populations (e.g., pediatric stroke or cerebral palsy). These measures also may be impractical or lack adequate sensitivity, reliability, or validity. Future work should aim to validate motor function measures such as the GMFM in pediatric stroke samples. The Dysphagia Disorder Survey is the only validated measure reported for use in pediatric stroke that assesses swallowing and feeding function; however, the Dysphagia Disorder Survey requires substantial training to administer and interpret, and therefore received a lower score (mean rating = 3).⁷⁴

TABLE 3.

Motor Function Measures

Outcome Measure	Ratings (Means and Range)	Pros	Cons
Gross Motor Functional Measure (GMFM)32–39	Mean: 4.33 Range: 4–5	• Designed for children with cerebral palsy, also validated in pediatric TBI • Wide age range • Two versions—one that is sensitive to impairments in young children and those with more complex motor disability • Used as an outcome measure in intervention studies in cerebral palsy • Teaching resource available • Reference curves in cerebral palsy available (Hanna et al., 2008)	• Not validated in pediatric stroke • Some users of the GMFM selectively administer only some of the subscales, and the reliability and the validity of the subscale scores are generally not as strong as they are for total score
Beery-Buktenica40–42	Mean: 3.66 Range: 3–4	• Measure that can differentiate between motor coordination impairments, perception impairments, and integration impairments • Standardized, excellent validity and reliability	• Upper limb only
AHA43–48	Mean: 3.66 Range: 3–4	• Good at detecting small changes in functional ability • Tested in children with cerebral palsy	• Very long and cannot be completed in the clinic; more useful for research purposes or if there is a dedicated physical therapy or occupational therapy program that patients with stroke could incorporate into the program
HAI49–51	Mean: 3.66 Range: 3–4	• Requires substantial training to administer	• Only for use in infants (however, can use the AHA for older children)
BOT-252,53	Mean: 3.33 Range: 3–4	• Used as a descriptive research tool • Addresses fine and gross motor function • Short and long forms available	• Cannot administer in clinic • Not used until age 4 years • Fine motor subtest not as strong as the others, needs to be performed by trained individual • BOT-2 test significantly overestimates score compared with the longer form in a healthy population of children
CB&M54–58	Mean: 3.33 Range: 3–4	• Validated in peds TBI (subset of participants with stroke), excellent reliability • Reliability estimates similar to adult stroke sample estimates • Can be rated in person or rated using video recording	• Long administration time and requires space to complete assessment • Ratings do not include quality of movement (i.e., patients with ataxia may meet the time requirement resulting in a high school, but the quality of their movement was low) • Arm function seems more important to assess as this tends to be more debilitating and arm function is not assessed • Administration is reliable if assessed by physical therapists
Fugl-Meyer Assessment59–63	Mean: 3.33 Range: 3–4	• Can calculate score for upper and lower limb function • Brief screening option available	• Difficult to use in the presence of aphasia or spatial neglect • Pain scale is variable • Only validated in adolescents and adults
Six-Minute Walk Test64–66	Mean: 3 Range: 3	• Simple test requiring no exercise equipment or advanced training • Reflects functional capacity for daily physical activities • Predictive of morbidity and mortality • Tested in child samples with cerebral palsy	• Highly specific, just related to walking—only looks at lower limb motor function and does not look at sensation, etc. • Does not provide specific information on pulmonary or cardiovascular systems during walking • Inter-rater reliability not great
Action Research Arm Test67–70	Mean: 3 Range: 3	Short administration time • Observation-based scoring, need some expertise in rating performance	Only assesses upper limb • Requires a lot of equipment • Not tested in younger children (ages 13+)
QUEST71–73	Mean: 3 Range: 3	• Designed specifically for cerebral palsy • Excellent reliability and validity in cerebral palsy • Assesses quality of movement unlike most measures that assess whether or not movement can be completed • Useful for assessing spasticity • Free	• Need to have experienced therapist administer the measure—does not have structured guidelines • Long administration time (3045 minutes) • QUEST is an assessment of quality; a change in score may not equate to a change in function/skill level • Age range is small • No validity data in acquired brain injury • No demonstration of predictive utility
PBS58	Mean: 3 Range: 3	• Good test-retest and inter-rater reliability • Short administration time • Validated in cerebral palsy Predictive of gross motor function	• Not yet validated in pediatric stroke • Only addresses balance skills so would have to do other tests to address other motor functions • Floor and ceiling effects • Correlates with age
Swallowing (oromotor)
Dysphagia Disorder Survey74–76	Mean: 3 Range: 3	• Excellent for assessing swallowing and feeding behaviors • Well validated over broad age range • Relatively short • One of the few available measures for assessing physiologic limitations in swallowing and feeding sensory-motor function • Developed for persons with developmental disability so it is appropriate for use in individuals with varying intellectual capacities	• Highly specific, not really generalizable • Needs to be administered by trained speech and language pathologist, occupational therapist, or physical therapist No literature on utility in assessing improvement over time

Abbreviations:

AHA = Assisting Hand Assessment

BOT-2 = Bruininks-Oseretsky Test of Motor Proficiency, 2^{nd edltlon}

CB&M = Community Balance & Mobility Scale

GMFM = Gross Motor Functional Measure

HAI = Hand Assessment for Infants

PBS = Pediatric Berg Balance Scale

QUEST = Quality of Upper Extremity Test

TBI = Traumatic brain injury

When only one number is given for range, all reviewers/raters agreed on that rating.

Adaptive functioning and behavior

The Adaptive Behavior Assessment System (ABAS) is one of the most commonly used measures to evaluate adaptive functioning (Table 4) in children and adults and is normed on a large sample representative of the US population. The ABAS received a mean rating of 4.66 and is considered a gold standard in terms of assessment of adaptive functioning. The ABAS possesses strong psychometric properties (i.e., validity, reliability) and is frequently used in pediatric stroke research.⁷⁹ The Child and Adolescent Scale of Participation (CASP) also received high scores (mean = 4.33) and is freely available, unlike the ABAS, which requires a cost per use.^80–84 The Vineland Adaptive Behavior Scale received lower ratings due to variable test-retest reliability and its long administration time.¹⁰⁴

TABLE 4.

Behavior, Adaptive Functioning, Quality of Life, and Mood

Outcome Measure	Ratings (Means and Range)	Pros	Cons
Behavior and adaptive functioning
ABAS77–79	Mean: 4.66 Range: 4–5	• Excellent validity and reliability • Good at identifying deficits in clinical populations • Some validation in developmentally delayed children	• Need computer program to score it efficiently, easy to make errors when hand scoring • Expertise required • Only available in two languages (English and Spanish) • Not sensitive to TBI-related impair- ments/issues • Intended for the assessment of older children rather than younger children
CASP80–84	Mean: 4.33 Range: 4–5	• Affordable • Short administration time • Concurrent validity with other measures evaluated • Excellent reliability • Widely available • Validated in ABI	• Moderate face validity • Some reports of ceiling effects in ABI • Item ratings compare child with same- age peers, may be less responsive to change than measures that are not age- referenced • Does not differentiate between the extent of participation and extent to which child is able to participate
WeeFIM85–88	Mean: 4.33 Range: 4–5	• Strong psychometric properties in cerebral palsy and children with disabilities • Predicts longitudinal functional recovery in children with disability • Short administration time • Broad age range	• Developmental differences in children less than 3 years old may create a floor effect • Expensive • Requires licensing to score
ASQ89–92	Mean: 4 Range: 4	• Has good predictive validity for gross motor development • Ease of administration • Shorter administration time than the Bayley	• Only available for birth-6 years. Limited age range—unable to track children over time • Not as sensitive for infants with extremely low birth weight
BASC93–97	Mean: 4 Range: 4	• Broad age range • Teacher, parent, child versions • Strong psychometric properties (excellent validity and reliability)	• Longer administration time • Requires psychologist for interpretation
CBCL98–102	Mean: 4 Range: 4	• Broad age range • Teacher, parent, and self-report forms • One of the most commonly administered measures of behavioral functioning • Corresponds to DSM-V criteria • Extensively validated in children across a broad age span and cross culturally	• Not validated in pediatric stroke or cerebral palsy • Some of the items are not relevant to younger children
VABS-3rd edition103,104	Mean: 3.66 Range: 3–4	• Good validity • Excellent reliability	• Test-retest reliability is inconsistent, susceptible to responder bias • Longer administration time • Responder bias
Vanderbilt Assessment Scale (3rd edition)105–110	Mean: 3.66 Range: 3,5	• Parent and teacher form—useful for collecting the same information from multiple sources • Used in clinical setting • Screens for other disorders in addition to ADHD • Free • Utility for screening for comorbid disorders • Teacher scale correlates highly with diagnosis of ADHD	• Low inter-rater reliability • No evidence found for discriminant validity • Should be used as a screening tool only • Items are more relevant for school- aged children than younger children
QoL
PSQLM111,112	Mean: 4.66 Range: 4–5	• Only validated QoL measure in pediatric stroke • Developed specifically for pediatric stroke • Takes into account what is important to the family in calculation of QoL score	• Not widely used, newer measure
Neuro-Quality of Life Measure (Neuro QoL Pediatrics)113–115	Mean: 4 Range: 4	• Consists of both generic and targeted domains, which allows investigators to compare children's health status with other disease groups (e.g., TBI, cerebral palsy)	• Upper and lower extremity function were not validated in the Neuro QoL pediatrics version
PROMIS116	Mean: 4 Range: 4	• Validated in pediatric epilepsy and muscular dystrophy populations • Item banks developed for children and separate item banks for adults • Psychometrically sound	• Pediatric ABI and TBI not included in neuro QoL development, and it has not been validated in these populations
PedsQL/PedsQL Infant117,118	Mean: 4 Range: 3–5	• Not validated in pediatric stroke but commonly used	• Quality of life item measures assume certain level of functional ability
Mood
RCADS119–121	Mean: 4 Range: 4	• Screens for anxiety and depression • Broad age range • Free • Available in 25 languages	• Some mixed findings on convergent validity
Child Depression Inventory122–125	Mean: 3.66 Range: 3–4	• Broad age range • Easy to score	• Depression only • Cost

Abbreviations:

ABAS = Adaptive Behavior Assessment System (3rd edition)

ABI = Acquired brain injury

ASQ = Ages and Stages Questionnaires

BASC = Behavioral Assessment System for Children

CASP = Child and Adolescent Scale of Participation

CBCL = Child Behavior Checklist

DSM-V = Diagnostic and Statistical Manual of Mental Disorders, fifth edition

PedsQL = Pediatric Quality of Life Inventory

PROMIS = Patient-Reported Outcomes Measurement System

PSLQM = Pediatric Stroke Quality of Life Measure

QoL = Quality of Life

RCADS = Revised Children’s Anxiety and Depression Scale

TBI = Traumatic brain injury

VABS = Vineland Adaptive Behavior Scale

WeeFIM = Functional Independence Measure

When only one number is given for range, all reviewers/raters agreed on that rating.

The Child Behavior Checklist (CBCL), Vanderbilt Assessment Scale (VAS), and the Behavioral Assessment System for Children (BASC) evaluate children’s internalizing and externalizing behaviors and assist in diagnosing behavioral and emotional problems in children. All three measures indicate items that correspond with the DSM-V diagnostic criteria for disorders diagnosed in children (e.g., attention-deficit/hyperactivity disorder, generalized anxiety disorder, major depressive disorder, oppositional defiance disorder).^107–109 The CBCL and BASC possess similarly strong psychometric properties, and both received ratings with a mean of 4 and were rated higher than the VAS, which was found to have low-inter-rater reliability and no evidence of discriminant validity (mean rating = 3.66).^107,109 The CBCL possesses good cross-cultural validity; it is available in multiple languages and has been validated in countries outside of North America.^100–102

Quality of life (QoL)

Four measures of QoL were evaluated, and all were rated positively (mean rating≥4). The Pediatric Stroke Quality of Life Measure (PSQLM) received the highest rating of 4.66. In addition to the PSOM, the PSQLM is one of the few scales developed specifically for children with stroke. The PSQLM has excellent sensitivity and validity, and the items were informed by the experiences of children with stroke and their families.¹¹¹ Generic QoL measurement tools such as the PedsQL tend to lack the elements of QoL related to cognition, language, and memory issues, which are of critical importance in pediatric stroke populations.

Mood

The Revised Children’s Anxiety and Depression Scale (RCADS) had a higher rating (mean rating 4.33) than the Child Depression Inventory (CDI) (mean rating = ADD). The RCADs has been validated in a large number of children representative of the US population, spans a wide range of ages, and is less costly than the CDI.^119–121

Cognition

Cognitive performance is often characterized as consisting of five subdomains: memory and learning, language, attention, executive functions, and perceptual and motor functions. Motor function and language were addressed separately, so this section focused primarily on batteries assessing overall intellectual ability, memory and learning, and executive function (Table 5). Two batteries that assess core domains of cognition were evaluated. The Wechsler Intellectual Ability Tests (WPPSI-IV, WISC-V, WAIS-IV) received a mean rating of 4.66. The Wechsler tests are the gold-standard measures to assess intellectual ability across the life span.¹⁶⁰ The child version has been normed on over 2000 children, possesses excellent psychometric validity, and is continually updated and improved upon.¹⁵⁵ The Weschler tests require significant training, have high associated costs, and are known to have practice effects. The NIH toolbox also received high ratings, with a mean score of 4. The NIH toolbox has a shorter administration time, is less expensive, and has good psychometric validity. The toolbox is far less widely used than the Wechsler tests as it is a newer battery and has only recently been validated for use in children with TBI.

TABLE 5.

Cognition and Language

Outcome Measure	Ratings (Means and Range)	Pros	Cons
Cognition
CVLT-Child / CVLT-33126–137	Mean: 4.66 Range: 4–5	• Short administration time • Well normed and studied in pediatric stroke and TBI • Child and adult versiondgood for tracking over time	• Cost but still less expensive than most other cognitive tests • Requires training to administer and interpret
WIAT-III138–142	Mean: 4.66 Range: 4–5	• Well standardized • Frequently used in research and clinically in pediatric stroke	• Long administration time
BRIEF-2143–153	Mean: 4.33 Range: 4–5	• Quick • Available in 40 languages • Solid psychometric properties • Easy to administer • Child and adult versiondgood for tracking over time	• Cost, parent-teacher inter-rater agreement was only moderate but was indicated to be consistent with expectation for different environmental settings
Wechsler Intelligence Tests (WISC-V/WPPSI- IV/WAIS-IV)154–162	Mean: 4.33 Range: 3–5	• Normed in >2000 children • Excellent psychometric properties • Most commonly used cognitive measure in North America • Can track patients from childhood to adulthood using same measure (different version depending on age) • Validated for use remotely (i.e., online administration)	• Cost • Requires significant training • Lengthy • Practice effects—frequent assessments are not feasible • Although available in several different languages, validation processes are of varying quality—suggested cautious use of Spanish version
Conner’s Scale for ADHD Assessment, third edition163,164	Mean: 4.33 Range: 4–5	• Easy to • No information on sensitivity to change or cross-cultural validity	•No information on sensitivity to change or cross-cultural validity
NIH Toolbox Cognitive Battery165–170	Mean: 4 Range: 4	• Shorter testing than most cognitive screening batteries • Solid psychometric properties • Has been assessed across multiple conditions, including pediatric stroke • Child and adult version—good for tracking over time	• Cost (less than some) • Experience needed in psychologyand requires training to administer
DKEFS171–178	Mean: 3.66 Range: 3–4	• 9 subtests—can administer individually or as a group • Sensitive in the detection of frontal lobe function • Developed for populations with brain • Sensitive in distinguishing between different clinical groups (focal frontal lesions vs fetal alcohol syndrome vs healthy controls) • • Alternative forms—reduces practice	• Validity is lower in youngest age groups (greater variability in scores in younger groups during standardization) • Only valid for children ages 8+ • Adequate reliability for some subtests • Test instructions can be complex and repetitive effects
TEA-Ch178–181	Mean: 3.66 Range: 3–4	• Tests a number of different types of attention • Scores are age-sensitive • 3 versions available—risk of practice effects is lower • Child and adult version—good for tracking over time	• Clinicians tend to find the most recent version (second version) cumbersome • Children with ADHD have trouble undergoing assessment • Poor test-retest • Expensive • Scoring instructions are unclear
WMS-IV182–185	Mean: 3.33 Range: 3–4	• Good but longer administration time for same cost as similar tests • Decent reliability	• Not studied in peds stroke • Only for children 16+ • Different depending on age, which makes it concerning for assessing longitudinal effects • Practice effects • Low scores are common in healthy children and adolescents, interpret with some caution
ChAMP186–190	Mean: 3.33 Range: 3–4	• Screening index as well as longer battery available • Ecologically valid • Shorter than other memory tests • Does not rely heavily on other neurocognitive domains (e.g., visual-motor integration) • Developed and validated in children with motor impairments	• Ongoing expense for electronic scoring • Less sensitive to memory deficits than the CVLT • Screening index less reliable than subtest measures • Practice effects observed if tested again <45 days later • Tasks can be monotonous and boring for younger children for visual memory subtests—sustained attention is required Instructions subtest may be culturally relevant for some children leading to inflated scores • Low scores are common in healthy children and adolescents, interpret with some caution
Speech and language
FOCUS191–192	Mean: 4 Range: 4	• Good validity and reliability • Parents can administer it • Low cost	• Relatively long delay between assessment points • Age range is small • Lengthy • Costly
EVT, third edition, and PPVT, fifth edition197,198	Mean: 3.66 Range: 3–4	• Reliable and valid. • Modifications available for children with disabilities and young children • Some cross-cultural validity • Can compare expressive to receptive language on two measures that were co-normed • Accessible for children with low intellectual ability	• Long administration time • High cost • Limited in its complexity as it is a naming task and does not evaluate higher-order aspects oflanguage (i.e., inference or comprehension of sen- tences/instructions)
Children’s Acquired Aphasia Screening Test199	Mean: 1.33 Range: 1–2	• Developed to evaluate aphasia in children	• No information available within the literature

Abbreviations:

ADHD = Attention-deficit/hyperactivity disorder

BRIEF-2 = Behavior Rating Inventory of Executive Function, Second Edition

CELF = Clinical Evaluation of Language Fundamentals

ChAMP = Child and Adolescent Memory Profile

CVLT-3 = California Verbal Learning Test—Child, Third Edition

DKEFS = Delis Kaplan Executive Function System

EVT = Expressive Verbal Test

FOCUS = Focus on the Outcome of Communication Under Six

PPVT = Peabody Picture Vocabulary Test

TBI = Traumatic brain injury

TEA-Ch = Test of Everyday Attention-Child

WAIS-IV = Weschler Adult Intelligence Scale-IV

WIAT-III = Wechsler Intellectual Achievement Test, Third Edition

WISC-V = Weschler Intelligence Scale for Children-V

WMS-IV = Wechsler Memory Scale, Fourth Edition

WPPSI- IV = Welschler Preschool and Primary Scale of Intelligence-IV

When only one number is given for range, all reviewers/raters agreed on that rating.

Within the memory subdomain, the California Verbal Learning Test (CVLT-C) was most highly rated with a mean score of 4.66. The CVLT-C is a standardized test with a short administration time, excellent reliability and validity in a pediatric TBI sample, and has been used in a research context in pediatric stroke.¹³⁷ Within the subdomain of executive function, the Behavior Rating Inventory of Executive Function received a score of 4.33 due to short administration time, strong psychometric properties, availability in several languages, and ability for parents to complete in clinic.¹⁴⁴ Only one objective attention measure, the Test of Everyday Attention-Child, was included in our evaluation, which received a mean rating of 3.66. The Test of Everyday Attention-Child has good validity; however, it possesses weak test-retest reliability, and some clinicians tend to prefer previous versions of the measure. However, the BASC, CBCL, Conners, and the VAS (covered under the Adaptive Function and Behavior domain) all will identify concerns about attention.

Speech and language

Very few language-specific measures were identified as being commonly used in pediatric stroke. The highest rated measure, the Focus on the Outcome of Communication Under Six has strong psychometric properties and received a mean score of 4. However, the age range is limited and therefore does not have wide applicability across the pediatric age span with low utility as a longitudinal measure. The Clinical Evaluation of Language Fundamentals (CELF) covers a broader range of ages (mean rating = 3.66); however, it is quite long to administer, requires specific expertise to interpret, and requires some degree of motor function.

Discussion

This study identified and evaluated 48 commonly used outcome measures across seven domains of function through a review of the literature and expert ratings by a multidisciplinary group of clinicians who care for children with strokes. Although there are a range of different outcome measures utilized in clinical care and clinical trials in pediatric stroke, existing literature provides little guidance regarding outcome measure quality and utility. Our comprehensive assessment of commonly used instruments addresses an important gap in knowledge regarding outcome measures by providing systematic multi-rater scoring of instruments for their utility across multiple domains of function. The selected outcome measures were evaluated based on their psychometric properties pertaining to the relevant clinical groups (i.e., pediatric stroke, cerebral palsy, TBI). Instrument strengths and weaknesses were summarized from the current literature by expert users. These evaluations can guide outcome measure selection for clinical trials or observational studies. Over the long-term, the use of a shared set of high-quality outcome measures could facilitate comparison between research studies, improve understanding of the recovery phases following pediatric stroke, and advance pediatric stroke recovery based on this knowledge.

Consistent with a prior systematic review completed a decade ago, most outcome measures have been validated in related populations of children with cerebral palsy or TBI.⁸ Only three measures were specifically validated for use in pediatric stroke populations: the PSOM; the psRRQ, which is a derivative of the PSOM; and the PSQLM. The PSOM received the highest rating of the global performance measures, with strengths including construct validity, inter-rater reliability, and ease of use either prospectively or retrospectively.¹³ The PSOM has been used in multiple outcome studies and has been previously strongly recommended for prospective clinical trials in pediatric stroke.¹³ The psRRQ as a remote administration option expands its use.¹⁶ One inherent limitation of global performance measures is that the corollary of their strengths as general screening measures is their limitation in being able to identify more subtle or focal deficits with sufficient sensitivity. Some global performance measures may misclassify a patient with minor neurological impairment into a more severe category and in turn predict an unnecessarily poorer outcome.²⁴ For example, on the KOSCHI, a child must meet all criteria outlined within a given category; otherwise, the child would be classified in a lower category suggesting an increased level of disability than might otherwise be warranted. With the PSOM, a child with mild functional impairment in four domains can receive the same score as a child with severe or profound impairment in one domain, which can lead to different levels of functional impairment within the same score. A variation of the PSOM, the Severity Classification Scheme (PSOM-SCS) has recently been developed to capture overall functional impact across domains better¹² and has been used in pediatric stroke outcome studies as well.⁷

A number of the global outcomes and motor scales have been adapted from adult scales such as the Glasgow Outcome Scale (GOS) and the Assisting Hand Assessment (AHA), which provides the advantage of supporting comparison with adult populations as well as facilitating the evaluation of teens into adulthood.²⁸ However, definitions may be hard to interpret, given the developmental stage of a child (e.g., “age appropriately independent for daily living”). Also, young children with stroke may have deficits that become more apparent over time as language and motor skills become more complex.³

Many of the motor performance and cognition measures evaluated in the current study are designed to assess different subdomains of function. Broad batteries of motor function or cognitive measures that screen many subdomains of function such as the Gross Motor Function Measure or the Wechsler Intelligence Scales are valuable and received high ratings from the experts as they are well validated and standardized. However, these measures are often impractical and time-consuming to administer within a routine neurology clinic visit or a clinical trial visit. A global performance measure may be useful to identify areas of impairment, whereas well-validated subdomain specific measures such as the California Verbal Learning Test, which is used to assess verbal memory, or the Community Balance and Mobility scale may be better suited to provide greater specificity and detail of specific impairments after areas of low performance are identified on a broad screening battery (e.g., WISC-V).

Often deficits in social-emotional functioning and behavior have a greater impact on health and well-being of children after stroke than deficits in physical functioning and mobility.¹¹² Therefore, an index of well-being such as a quality-of-life measure or a behavioral measure to track patient outcomes over time is recommended. Through the current evaluation of commonly used measures, it was noted that several of the adaptive functioning and behavior measures were highly correlated with one another (e.g., the BASC and CBCL) and are described to evaluate similar constructs.⁹⁷ During selection of outcome measures in the domains of behavior, QoL, and adaptive function for clinical practice or a clinical trial, care must be taken to avoid administering multiple highly correlated measures to reduce redundancy as well as patient burden.

There are some notable limitations to this study. Although a comprehensive evaluation of many of the outcome measures is provided, not every measure available to evaluate outcomes in pediatric stroke has been captured. The list of measures included in the current study was informed through a thorough literature search and modified by experts; however, the measures chosen were those most frequently used within the North American context. Measures used that are in accordance with the European International Classification of Disease system were outside the scope of the current study. Nevertheless, many of the measures selected for use are validated cross-culturally and are available in numerous languages. Our assessment suggests the need for new practical outcome measures to assess specific outcome subdomains and the necessity of further validation of commonly used measures within a pediatric stroke population.

Although making definitive outcome measure recommendations for all situations and studies remains difficult, pediatric stroke centers should include global measures at each follow-up time point as well as more domain-specific measures as appropriate. The rankings established in this article provide a reference for selecting outcome measures depending on the clinical or research question, assessment capabilities, and age of the child. Typical timing of assessments varies (also see Felling et al., (2023) which provides a roadmap for the timing of pediatric stroke outcome assessment). However, at hospital discharge, global assessments such as the PSOM are appropriate. More detailed assessments may occur in inpatient rehabilitation, three months poststroke, 12 months poststroke, and as clinical needs dictate, especially at critical time points of transition. Some measures have practice effects such that they cannot be used more often than every 12 months.

This work is intended to provide a toolkit for clinicians and clinical researchers to tailor outcome measure choices for children with stroke in clinical care, observational research studies, or clinical trials. The compendium of assessments and evaluation of their quality and utility should support more consistency across centers, which should facilitate research and care pathways. In the future, the development of additional pediatric stroke-specific outcome measures or validation of existing measures in pediatric stroke populations would be helpful.