An Examination of the PROMIS^® Pediatric Instruments to Assess Mobility in Children with Cerebral Palsy

Abstract

Purpose

The Patient Reported Outcomes Measurement Information System (PROMIS^®) provides adult and pediatric self-report measures of health-related quality of life designed for use across medical conditions and the general population. The purpose of this study was to examine the feasibility and validity of the PROMIS^® pediatric short form and computer adaptive test (CAT) mobility measures in children with cerebral palsy (CP).

Methods

Eighty-two children with CP completed self-report (PROMIS^® Mobility Short Form, PROMIS^® Mobility CAT, Pediatric Quality of Life Inventory^™) and performance-based assessments of mobility (Timed Up-and-Go, Gross Motor Function Measure). Parents provided three proxy reports of child mobility (Pediatric Outcomes Data Collection Instrument, Functional Assessment Questionnaire, Shriners Hospitals for Children CP-CAT). Validity of PROMIS^® instruments was examined through correlations with other measures and “known groups” analyses determined by Gross Motor Function Classification System (GMFCS).

Results

On average, the PROMIS^® CAT required less than seven items and two minutes to administer. Both PROMIS^® measures showed moderate to high correlations with child- and parent-proxy report of child mobility; correlations with performance-based measure were small for the PROMIS^® Short Form and non-significant for the PROMIS^® CAT. All measures except for the PROMIS^® CAT were able to distinguish between GMFCS categories.

Conclusions

Results support the convergent and discriminant validity of the pediatric PROMIS^® Mobility Short Form in children with CP. The PROMIS^® Mobility CAT correlates well with child- and parent-report of mobility but not with performance-based measures and does not differentiate between known mobility groups.

Approximately 4.7 million children in the United States currently live with physical disability [1; 2]. Of these, 1.5 million have limitations in activities of daily living, including self-care and mobility and approximately 3 million experience limitations in play and school activities. Cerebral palsy (CP), a non-progressive neurological disorder characterized by a variety of physical impairments including problems with posture, movement, spasticity, and mobility, is the most common cause of physical disability in children [3] and rates of CP have been increasing due to improved neonatal care and associated increased survival rates of low birth weight infants [3].

The high prevalence of CP in the pediatric rehabilitation population and the large number of ongoing clinical trials [4] underscore the need for quality outcome measures in CP. Existing instruments have serious limitations, including limited responsiveness to intervention and ceiling effects [5; 6]. Reliance on proxy report is another limitation of existing outcomes in children and child report is equally important to parent-report in overall outcomes assessment. Cognitive testing studies support child-reported outcomes, finding that children as young as six years of age can accurately report on their own health status [7] and understand subtle differences among ordinal response scales choices [8]. Clarity and brevity of assessment is a priority in self-report of children with CP, for whom fatigability or physical limitations may contribute to difficulty in completing traditional surveys. These factors indicate a need for modern measurement tools that are written at an age-appropriate level, measure outcomes in a way that is relevant to children, and are brief while maintaining high levels of validity, reliability, and sensitivity.

The Patient Reported Outcomes Measurement Information System (PROMIS^®) aims to address this need for efficient and meaningful pediatric outcome measures by developing contemporary instruments of health related quality of life that are freely available to the public. PROMIS^®, an NIH Common Fund initiative, was started in 2004 to advance the science and application of patient-reported outcomes (PROs) across chronic diseases [9; 10]. PROMIS^® measures employ modern computer technology and item response theory (IRT) to develop brief measures that assess constructs across a wide-ranging spectrum of severity. PROMIS^® has generated computer-adaptive tests (CAT) based on calibrated “item banks,” or large sets of items organized along unidimensional hierarchies from very low to very high levels of symptoms/ability. For example, the PROMIS^® Pediatric Mobility item bank has 23 items that range from low (e.g., I could get out of bed by myself) to high functioning (e.g., I could run a mile). After the first CAT item, the computer selects subsequent items based on previous responses. In this way, CAT administration is dynamically tailored to the respondent, administering a small selection of items (5–12) to generate a precise estimate of what the individual’s score would have been had they taken all items. Each PROMIS^® domain also has a short form, a static form with the most informative and content-balanced item bank items (determined through IRT) that cover the full measurement range.

Recent work has focused on expanding PROMIS^® in pediatric populations [11–14] and PROMIS^® currently includes measures of 9 pediatric domains for use with children 8 years and older. PROMIS^® pediatric instruments were developed and calibrated in a community sample of 4,129 children diverse in terms of gender, age, race/ethnicity, and medical/psychiatric diagnosis [15]. However, only 35% of children in the normative sample endorsed one or more chronic illnesses and, of those, the most common medical conditions were asthma (18%), Attention Deficit/Hyperactivity Disorder (4.6%), and arthritis (2.9%)[15]. Therefore, the sample used to develop and calibrate the PROMIS^® Pediatric Mobility item bank did not have a significant representation of children with physical disability associated with a neurological condition, such as CP.

PROMIS^® was designed to be “generic” so that it could measure functioning across populations. As such, the PROMIS^® emphasizes issues and symptoms that are universal to everyone across different domains of functioning (e.g. physical, emotional, and social functioning). It does not take a disease-specific approach to measuring outcomes, which would place emphasis on measuring functioning within specific conditions. There is concern that generic health-related quality of life measures may not be appropriate for use by individuals with significant impairment in a specific domain [1] and the validity of PROMIS^® measures needs to be established in groups that were not previously studied [16]. Therefore, it is important to examine the validity of PROMIS^® Pediatric Mobility CAT and Short Form in children with CP. Thus, the aim of this study is to determine the feasibility and validity of the PROMIS^® Pediatric Mobility instruments in a sample of 82 children with CP, by examining:

1. The internal reliability and distribution characteristics (e.g., floor/ceiling effects) of all measures and the administration feasibility of the PROMIS^® Mobility CAT in CP (indicated by test length and administration time).

2. The construct (i.e., convergent) validity of PROMIS^® measures by correlation of PROMIS^® scores with performance-based and self- and parent-report legacy measures.

3. The construct validity of PROMIS^® measures through “known group” method, by testing whether PROMIS^® mobility measures are able to distinguish GMFCS levels.

Method

Procedure

This study is part of a larger prospective study on the responsiveness of PROMIS^® instruments to orthopedic surgery at 8 collaborating sites (6 Shriners Hospitals for Children (SHC), Children’s Hospital Boston, and Cincinnati Children’s Hospital Medical Center). The study began at the 6 SHC sites, which had a strong infrastructure for data collection due to an ongoing study of responsiveness of the Shriners Hospitals for Children CP-CAT (described in the Measures section). SHC sites were selected based on the number of children with CP and their geographic location. The two non-SHC sites were added to help with enrollment. In total, 189 children with cerebral palsy who were scheduled to undergo orthopedic surgery were screened for the study. Of those cases, 97 children met the eligibility requirements. The primary reasons that children with CP who were undergoing eligible surgeries were deemed ineligible were being under 8 years of age or an inability to self-report due to cognitive impairment. The study enrolled children ages 8 to 21 years with CP undergoing elective orthopedic surgery to improve lower or upper extremity functioning. Exclusion criteria included: orthopedic surgery to simply improve positioning, pain and\or spasticity, cognitive impairment that limited the ability to read/understand questions, primary language other than English, functional limitations due to meningitis, brain tumors, or acquired injuries and\or disease of the brain. Each site obtained local Institutional Review Board approval. Prior to enrollment, parent written informed consent and child assent were provided. All instruments were administered just prior to surgery and again at three time points aligned with routine visits through the first 24-months after surgery; only pre-surgical “baseline” data were used in this study.

Data Collector Training

To ensure consistency across sites, one clinician with expertise in pediatric motor assessment trained all data collectors (physical or occupational therapists) according to a standardized research protocol. Bi-weekly calls throughout the data collection period provided opportunities to discuss study procedures.

Sample

The sample consisted of 97 children and their parent/guardian who completed the baseline measures. Fourteen children were candidates for upper extremity surgery, completed a different battery of outcomes measures related to upper extremity-specific surgical goals, and were excluded from analyses. Of the 83 candidates for lower extremity surgery, 82 had completed one or both PROMIS^® measures and were included in analyses. Mean age at surgery was 12.67 years (SD = 2.96; median = 13.00, range 8–19 years). Most children were male (n = 47, 57.3%) and White (n = 64, 78.0%; Black n = 10, 12.2%; Asian n = 5, 6.1%; American Indian n = 1, 1.2%; Hawaiian, Pacific Islander n = 1, 1.2%; Other n = 1, 1.2%).

Measures

Child Self-Report

Children independently completed the PROMIS^® version 1.0 pediatric Mobility CAT and Mobility Short Form on a stand-alone computer. Mobility items use a common stem: “In the past 7 days I could …” followed by a specific activity. The five response options range from ‘no trouble’ to ‘not able to do’ (see Figure 1 for all response options). Like all PROMIS^® instruments, these are scored on a T metric with a normative Mean=50 and Standard Deviation=10. The PROMIS® CAT default setting were used, which administered a minimum of five items and only stopped administering items when a standard error of 0.40 (indicating a minimum level of precision in estimating the person’s “true score”) or when a maximum of 12 items were completed. The Mobility Short Form is made up of a static set 8 of the 23-items of the Mobility item bank, with a possible T-score range between 14 and 59 [17]. These measures were developed and validated in a racially diverse sample of 3048 children, aged 8–17 years [17]. The PROMIS Mobility measure has been shown to be feasible and valid in a clinical sample of children with cancer [18]. Children also completed the Pediatric Quality of Life Inventory^™ (PedsQL^™) Cerebral Palsy Module Version 3.0 [14], a measure with well-established reliability and validity [19] that was designed to assess health-related quality of life across seven domains in children with CP. The subscale examined in this study, movement/balance, is the mean of 5 items that assess ability to move arms, legs and body and balance while sitting and standing.

Fig 1.

Response frequencies to the first item of the PROMIS^® Pediatric Mobility CAT by GMFCS category

Note. GMFCS = Gross Motor Function Classification System; CAT = Computer Adaptive Test

Parent Report

Three parent-report measures, the Shriners Hospitals for Children (SHC) CP-CAT (hereafter called SHC CP-CAT) [20], the Gillette Functional Assessment Questionnaire (FAQ; walking scale)[21] and the Pediatric Outcomes Data Collection Instrument (PODCI)[22], served as proxy measures of child mobility (the PODCI and FAQ are publicly available). The SHC CP-CAT was developed by the SHC to evaluate the rehabilitation and orthopedic outcomes of children with CP and includes measures of activity, upper-extremity skills, lower-extremity skills, and global physical health [20]. The lower extremity scale has 85 items that assess basic mobility, transfers, and ambulation skills that were written by content experts with knowledge about typical orthopedic interventions and outcomes in CP as well as by parents of children with CP; the items were exposed to iterative cognitive testing to ensure comprehension of items [23]. Similar to PROMIS^®, the SHC CP-CAT is scored on a T metric and uses CAT technology to administer few items to arrive at a precise score, resulting in a brief and accessible measure. It uses a five-point ordinal response scale ranging from “without difficulty” to “cannot do” and has strong test-retest reliability and concurrent and “known groups” validity in CP [24; 25]. The SHC CP-CAT was programmed to administer 15 items and was administered to 72 of the 82 cases.

The FAQ walking scale is a single-item 10-level measure of child’s mobility ranging from non-ambulatory (1 = “not able to take any steps at all”) to independently ambulatory in the community (10 = “walks, runs, and climbs on level and uneven terrain without difficulty or assistance); it has demonstrated good reliability and validity in children with walking disability [21]. The PODCI provides subscale scores on upper extremity function, pain, happiness, satisfaction, expectations and on two subscales used in this study: 1) the transfer/mobility subscale consists of 11 items that assess basic mobility, such as toilet and bed transfers, standing independently, walking short distance, and climbing a flight of stairs, and 2) the physical functioning subscale consists of 12 items that assess ability to engage in more intense physical activity such as riding a bike, walking long distances, scaling multiple flights of stairs, and participating in group sports activities.

Examiner Administered

Trained examiners administered two performance-based physical activity tests, the Timed Up-and-Go (TUG)[26] and the Gross Motor Function Measure (GMFM)[27]. For the TUG, the examiner recorded the time it took for a child to rise from a chair, walk straight for 3 meters at a normal pace, and return to and sit in the chair. TUG scores were recorded as the average time (seconds) across three trials. The TUG has demonstrated excellent reliability (ICC = 0.99), discriminant and convergent validity, and responsiveness to change in children with and children without physical disabilities [28]. The GMFM, developed to assess five domains of gross motor ability in children with CP, has been shown to have good reliability, validity, and responsiveness to change in CP [29]. For the two subtests administered in this study, Walk/Run/Jump and Stand, the examiner rated the child’s completion of physical tasks based on a 4-point scale where higher scores indicate higher functioning. Examiners, who were trained to use the Gross Motor Function Classification System – Expanded and Revised instruction manual (GMFCS – E & R) [30], provided a GMFCS [31] score, which rates ambulatory ability on a 5-point scale (See Table 1). Examiners determined GMFCS scores through direct interactions with participants and parents. The GMFCS – E & R has demonstrated excellent content, convergent, and predictive validity and inter-rater and test-retest reliability [32–35]

Table 1.

Scale Definitions for the Gross Motor Function Classification System (GMFCS)

Level I	Walks without limitations
Level II	Walks with limitations
Level III	Walks using a handheld mobility device
Level IV	Self-mobility with limitations; may use power wheelchair
Level V	Transported in a manual wheelchair

Data Analysis

Distribution characteristics, including frequency of scores at the floor (theoretical minimum) and ceiling (theoretical maximum) of the measure were calculated for all measures. Cronbach’s alphas were calculated to give an estimate of internal consistency; levels of at least .70 were considered acceptable [36]. Construct validity was examined using zero-order correlations of PROMIS^® measures with legacy measures that are theoretically related (convergent validity). To assess construct (i.e. discriminant) validity, we tested how well the PROMIS^® instruments discriminated between “known groups” with the expectation that mean scores on a valid measure should differ between GMFCS categories. Analysis of variance (ANOVA) procedures, with post hoc Tukey honestly significant differences tests for multiple comparisons, were used to compare PROMIS^® scores by GMFCS level. Because TUG scores were not normally distributed, a Kruskal–Wallis (K–W) non-parametric test was used to test differences between GMFCS categories and the Mann–Whitney U-test was used to determine significance between pairs of categories. Thirty-three children (40.2%) had a GMFCS score of I, 32 (39.0%) had a score of II, 14 (17.1%) had a score of III, 2 (2.4%) had a score of IV, and 1 (1.2%) had a score of V; categories III, IV, and V, were collapsed for analyses into a single category due to low frequencies.

Results

All measures demonstrated acceptable levels of internal consistency (Table 2). Consistent with GMFCS scores suggesting overall high levels of mobility, across the measures, there was a higher rate of ceiling effects compared to floor effects. PROMIS^® had lower frequency of ceiling effects compared to all survey measures except for the SHC CP CAT and the PODCI Physical Activities/Sports.

Table 2.

Descriptive statistics of the PROMIS^® and legacy instruments including mean, median, minimum, and maximum scores. reliability and floor/ceiling effects in a sample of 82 children with cerebral palsy.

Measure	N	Mean	Scores Median	Min	Max	Cronbach’s α (# items)	Floor		Ceiling
							N	%	N	%
Child Self-Report Measures (possible range)
PROMIS® Mobility SF (14 – 59)	82	41.43	40.00	26	59	.75 (8)	0	0	6	7.2
PROMIS® Mobility CAT (20 – 80)	79	43.38	42.80	28	62	-	0	0	2	2.5
PedsQL Movement/Balance (0 – 100)	81	82.35	90.00	30.00	100.00	.70 (5)	0	0	17	21.0
Parent Report Measures
PODCI Basic Mobility (0 – 100)	83	83.20	88.00	20	100.00	.90 (11)	3	7.9	8	21.1
PODCI Physical Activity/Sports (0 – 100)	83	53.10	50.00	3	97	.94 (12)	0	0	0	0
FAQ – mobility item (0 – 10)	83	8.01	9.00	2	10	-	0	0	13	36.1
SHC CP-CAT Lower Extremity (30 – 70)	72	53.12	53.18	36.20	68.46	-	0	0	0	0
Performance-based Measures
TUG	77	10.52	8.50	4.99	67.0	-	-	-	-	-
GMFM Stand (0 – 39)	81	30.46	34.00	1	39.0	-	0	0	4	4.9
GMFM Walk (0 – 72)	81	52.01	64.00	0	72.0	-	2	2.5	2	2.5

Note. PROMIS^® = Patient Reported Outcome Measurement Information System; PedsQL = Pediatric Quality of Life Inventory^™; PODCI = Pediatric Outcomes Data Collection Instrument; FAQ = Functional Assessment Questionnaire; SHC CP-CAT = Shriners Hospitals for Children CP-CAT; TUG = Time Up and Go; GMFM = Gross Motor Function Measure

Feasibility of the PROMIS^® Mobility CAT in CP

The average number of PROMIS^® CAT items administered was 6.70, and did not differ across GMFCS categories (Table 3). Approximately 70% of children with GMFCS scores II-V discontinued the CAT after the minimum number of items (5) whereas 42.4% of those with a GMFCS score of I discontinued after five items. In contrast, nearly 20% of those in the GMFCS I category and less than 6% of those in the GMFCS III-V category completed the maximum number of CAT items (12). Average time to complete the measure ranged from 1.35 to 2.10 minutes, with those with highest level of mobility taking the longest.

Table 3.

Number of items and length of time for administration of the PROMIS^® Mobility CAT, for entire sample (n = 79) and by Gross Motor Function Classification System (GMFCS) category

		Number of Items Administered						Time (Minutes)
	# of items in item bank	Mean	Median	Min	Max	N (%) who stopped at min items	N (%) who reached max items	Mean	Median	Min	Max
Total Sample	23	6.70	6.70	5	12	47 (57.3)	12 (14.5)	1.66	1.00	<1.0	25.0
GMFCS Category
I (n = 31)	23	7.23	6.00	5	12	14 (42.4)	6 (19.4)	2.10	1.00	<1.0	25.0
II (n = 31)	23	6.52	5.00	5	12	21 (65.6)	5 (16.1)	1.39	1.00	<1.0	6.0
III-V (n = 17)	23	6.06	5.00	5	12	12 (70.6)	1 (5.9)	1.35	1.00	<1.0	5.0

Note. PROMIS^® = Patient Reported Outcome Measurement Information System; 79 of the 82 children in the study sample had data for the PROMIS^® Mobility CAT.

Construct Validity

Correlations between PROMIS^® and legacy measures are depicted in the first two rows of Table 4 with correlations amongst the legacy measures in lower rows. PROMIS^® mobility measures were highly correlated with each other and moderately correlated with the legacy child self-report measure, the PedsQL. Correlations between PROMIS^® measures and parent self-report measures of mobility ranged from 0.38 to 0.60, with relatively higher correlations observed for the short form compared to the CAT. Correlations between performance-based measures of mobility were small for the Short Form and non-significant for the CAT.

Table 4.

Pearson bivariate correlation coefficients for PROMIS^® and legacy measures of mobility

	Child Self-Report		Parent-Reported				Performance-Based (Examiner-Administered)
	2 PROMIS® CAT	3 PedsQL Move	4 PODCI mobility	5 PODCI Sports	6 Gillette	7 SHC CP- CAT	8 TUG	9 GMFM Stand	10 GMFM Walk
1. PROMIS® Short Form	.88**	.58**	.52**	.60**	.48**	.54**	−.30**	.39**	.39**
2. PROMIS® Mobility CAT	-	.60**	.39**	.49**	.41**	.38**	−.16	.21	.19
3. PedsQL Movement and Balance		-	.39**	.32**	.40**	.41**	−.35**	.25**	.29**
4. PODCI Mobility			-	.74**	.76**	.84**	−.59**	.77**	.74**
5. PODCI Sports				-	.64**	.79**	−.36**	.67**	.61**
6. FAQ					-	.68**	−.65**	.67**	.61**
7. SHC CP-CAT Lower Extremity						-	−.56**	.80**	.83**
8. TUG							-	.68**	−.63**
9. GMFM Stand								-	.95**
10. GMFM Walk									-

Note. PROMIS^® = Patient Reported Outcome Measurement Information System; PedsQL = Pediatric Quality of Life Inventory^™; PODCI = Pediatric Outcomes Data Collection Instrument; FAQ = Functional Assessment Questionnaire; SHC CP-CAT = Shriners Hospitals for Children CP-CAT; TUG = Time Up and Go; GMFM = Gross Motor Function Measure; ns for all other correlations ranged from 78 to 82.

Known Groups Analysis

Table 5 shows that the PROMIS^® Short Form and all legacy measures were able to distinguish between GMFCS categories. PROMIS^® CAT scores did not show significant mean differences between the groups. Mean scores on the PROMIS^® Short Form and CAT were comparable for the GMFCS 1 category; however, compared to the CAT, the short form showed lower mean scores in the GMFCS II and III-V categories. In fact, for the GMFCS III-V category, the average short form score is approximately ½ standard deviation lower than the CAT score.

Table 5.

“Known group” results comparing means for PROMIS^® and legacy measures of mobility across GMFCS categories

Measure	GMFCS Category Means (SD) 95% Confidence Interval of Mean			Significance Test Resultsa	Contrastsb
	I	II	III-V
PROMIS® Mobility CAT (n = 31, 31, 17)	45.45 (7.20) 43.14 – 47.77	42.56 (5.79) 40.44 – 44.69	41.09 (6.22) 37.89 – 44.29	F(2,78) = 2.90, p = .06	I vs. II, p = 0.19 I vs. III-V, p = 0.07 II vs. III-V, p = 0.73
PROMIS® Mobility Short Form (n = 33, 32, 17)	45.06 (7.85) 42.28 – 47.85	40.53 (6.77) 38.09 – 42.97	36.06 (5.80) 33.07 – 39.04	F(2,81) = 9.55, p < .001	I vs. II, p = 0.03 I vs. III-V, p < 0.001 II vs. III-V, p = 0.09
PedsQL Movement and Balance (n = 32, 31, 17)	90.31 (14.59) 85.05–95.57	80.48 (14.96) 75.00–85.97	70.88 (21.95) 59.59–82.17	F(2,79) = 8.01, p < .01	I vs. II, p = 0.54 I vs. III-V, p < 0.01 II vs. III-V, p = 0.14
PODCI Mobility (n = 33, 32, 17)	92.61 (7.01) 90.12–95.09	85.50 (10.95) 81.55–89.45	63.41 (23.54) 51.31–75.51	F(2,81) = 27.13, p < .001	I vs. II, p = 0.09 I vs. III-V, p <0.001 II vs. III-V, p <0.001
PODCI Sports (n = 33, 32, 17)	65.00 (17.05) 58.96–71.04	51.31 (17.86) 44.87–57.75	35.47 (17.83) 26.31–44.64	F(2,81) = 16.35, p < .001	I vs. II, p < 0.01 I vs. III-V, p <0.001 II vs. III-V, p = 0.01
FAQ (n = 33, 32, 17)	9.21 (1.05) 8.84–9.59	8.13 (1.29) 7.66–8.59	5.82 (2.70) 4.44–7.21	F(2,81) = 25.03, p < .001	I vs. II, p = 0.02 I vs. III-V, p <0.001 II vs. III-V, p <0.001
SHC CP-CAT (n = 30, 28, 14)	57.89 (3.88) 56.20 – 59.57	52.67 (5.02) 50.93 –54.41	43.80 (5.24) 41.34 – 46.27	F(2,71) = 44.49, p < .001	I vs. II, p <0.001 I vs. III-V, p <0.001 II vs. III-V, p <0.001
TUG (n = 31, 31, 15)	7.66 (1.52) 7.10–8.22	9.04 (3.16) 7.88–10.20	19.49 (15.01) 11.18–27.81	χ2(2, 77) =26.19, p < .001	I vs. II, p = 0.71 I vs. III-V, p <0.001 II vs. III-V, p <0.001
GMFM Stand (n = 32, 31, 17)	35.94 (3.37) 34.72–37.15	31.81 (5.94) 29.63–33.99	19.35 (10.73) 13.84–24.87	F(2,79) = 36.74, p < .001	I vs. II, p = 0.04 I vs. III-V, p <0.001 II vs. III-V, p <0.001
GMFM Walk (n = 32, 31, 17)	67.06 (7.77) 64.26–69.86	53.71 (17.27) 47.37–60.04	23.65 (20.00) 12.29–32.37	F(2,79) = 46.92, p < .001	I vs. II, p < 0.01 I vs. III-V, p <0.001 II vs. III-V, p <0.001

Note. GMFCS categories III, IV, and V are collapsed into a single category. PROMIS^®=Patient Reported Outcome Measurement Information System; PedsQL=Pediatric Quality of Life Inventory^™; PODCI=Pediatric Outcomes Data Collection Instrument; FAQ=Functional Assessment Questionnaire; SHC CP-CAT=Shriners Hospitals for Children CP-CAT; TUG=Time Up and Go; GMFM=Gross Motor Function Measure;

^aOne-Way ANOVAs used for all tests except for the TUG, which used a Kruskal-Wallis (K-W) non-parametric test to examine significance between GMFCS categories;

^bpair-wise P-values calculated by Tukey’s HSD Post Hoc Test for all tests except for TUG, which used the Mann-Whitney U-test to determine significance between pairs of GMFCS Categories

Supplementary Analyses

Given the findings that the PROMIS^® CAT did not correlate significantly with performance-based measures of mobility and did not distinguish between known mobility groups, we examined some characteristics of the CAT administration in this sample. To estimate the location of CAT items (i.e., difficulty of the item along the hierarchy from low to high mobility), we averaged the four beta weights for each item from the source/calibration manuscript [17]. Table 6 depicts the 23 items of the PROMIS^® CAT arranged according to location. The first item administered to everyone, “I could do sports and other exercise that kids my age could do,” has a high level of difficulty. Response patterns to the first item of the CAT by known mobility capacity (Figure 1), indicates that nearly equal proportions of children with no mobility limitations (59.4% of GMFCS I) and of children with mobility limitations requiring a mobility aid (58.8% of GMFCS III-V) responded that they had “no trouble” or “a little trouble”. Also notable, the items administered with the highest frequency are clustered near the upper half of the location distribution. In fact, the CAT did not administer seven of the lowest difficulty items. Furthermore, some of the lowest difficulty items that were administered (e.g. items with locations of 13, 14, 17) were administered at higher frequencies in the GMFCS I category than in the GMFCS III-V category.

Table 6.

Percent of cases within each GMFCS category that received PROMIS^® Mobility CAT items (arranged by “difficulty”)

GMFCS Categories
I (n=32)	II (n=32)	III-V (n=17)
%	%	%	Item Location	Items, arranged from highest to lowest mobility difficulty	Format*
48	23	24	1	I could run a mile	CAT Only
100	100	100	2	**I could do sports and other exercise that kids my age could do	Both CAT/SF
94	81	88	3	I have been physically able to do the activities I enjoy most	Both CAT/SF
32	19	18	4	I could ride a bike	CAT Only
71	55	47	5	I could keep up when I played with other kids	Both CAT/SF
48	23	24	6	I could walk more than one block	CAT Only
32	16	18	7	I could walk up stairs without holding on to anything	Both CAT/SF
55	26	29	8	I could stand on my tiptoes	Both CAT/SF
35	52	53	9	I could stand up by myself	Both CAT/SF
100	100	94	10	I could get up from the floor	Both CAT/SF
10	23	24	11	I could walk across the room	CAT Only
58	84	71	12	I could move my legs	Both CAT/SF
26	16	6	13	I could carry my books in a backpack	CAT Only
26	13	6	14	I could get down on my hands and knees without holding on to something	CAT Only
-	-	-	15	I could get in and out of a car	Not Administered
-	-	-	16	I could get into bed by myself	Not Administered
13	10	6	17	I could bend over to pick something up	CAT Only
-	-	-	18	I used a wheelchair to get around	Not Administered
-	-	-	19	I used a walker, cane, or crutches to get around	Not Administered
-	-	-	20	I could go up one step	Not Administered
-	-	-	21	I could get up from a regular toilet	Not Administered
-	-	-	22	I could turn my head all the way to the side	Not Administered
10	23	24	23	I could get out of bed by myself	CAT Only

Note. GMFCS = Gross Motor Function Classification System

^*Both = item was administered as part of the CAT and Short Form;

^**Denotes first item administered as part of the CAT; % = the percent of children within a given GMFCS category who were administered a given item.

Discussion

The PROMIS^® instruments were designed to provide efficient, flexible, reliable, and precise measurement of health-related quality of life across medical conditions. The goal of this study was to examine the feasibility and validity of the PROMIS^® Mobility Short Form and CAT in children with CP. Results support the construct validity of the PROMIS^® Short Form. It demonstrated moderate to large correlations (r’s ranging from 0.48 – 0.60) with the self- and parent-report legacy measures, and moderate correlations with performance-based measures of mobility (r’s ranging from 0.30 – 0.39). The PROMIS^® CAT demonstrated good convergent validity when compared with child self-report and parent report legacy measures; however, it did not correlate significantly with any of the performance-based measures. The strength of these associations should be interpreted within the context of previous findings that correlations between child self-report measures tend to be higher than between child-report measures and either parent-report [37–40] or performance-based measures of physical functioning [37; 41].

In regards to discriminant validity, known-groups analyses showed that the PROMIS^® Short Form was able to distinguish between GMFCS categories. Remarkably, the PROMIS^® CAT was the only study measure that was not able to distinguish between GMFCS categories. Mean mobility scores for GMFCS I, the highest mobility group, are similar for the Short Form and CAT, but only the short form showed expected changes in mean scores for GMFCS categories II and III-V such that the lowest mobility group has a mean score that is nearly 1 standard deviation lower than the category with the highest mobility. In contrast, the CAT did not show changes of similar magnitude across the GMFCS categories; the differences between the highest and lowest mobility group is less than ½ standard deviation (i.e. 4.36 points). This suggests the possibility that the Short Form and CAT have similar accuracy in children with good mobility, but that the CAT is less accurate than the Short Form in children with functional limitations.

The pediatric PROMIS^® measures were developed and normed in a sample that did not have a significant representation of children with physical disability [17]. Therefore, calibration of the PROMIS^® CAT, including starting point (first item), was calculated in a sample of children whose average mobility ability was markedly different from the pediatric CP population. In this study, seven of 23 mobility items were never administered by the CAT. Furthermore, the unused seven items were lower-difficulty items related to the use of mobility aids (“I used a walker, cane, or crutches to get around”) and basic transfer skills (“I could get in and out of a car”). These items are highly relevant to measuring mobility in a sample of children with CP, particularly those with lower mobility. In this sample these items were not administered to those children who use mobility aids (GMFCS III-V).

Response patterns to the first CAT item, indicate that children with high mobility (GMFCS I) report little to no trouble doing “sports and other exercise kids [my] age could do” at rates that are similar to children with low mobility (GMFCS III-V). This item may be displaying Differential Item Functioning (DIF) and future work should conduct formal tests of DIF for all items in this item bank in children with CP compared with children from the general population. It is important to understand if children with CP interpret the question differently from children in the general population. Cognitive interviewing, which requires the respondent to “think aloud” while answering survey questions, can provide insight into how specific items are interpreted and to identify problems with item content [42] and has been used to remedy problems with self-report measures in children with physical disabilities [43]. Cognitive interviewing was conducted as part of the SHC CP-CAT development process and revealed that parents were concerned that items relating to sports and advanced physical activity were not applicable to their children and that many items were difficult to answer because they were overly complex and/or ambiguous [23]. Cognitive interviewing of the PROMIS^® CAT was performed in diverse populations, but was not focused on children with mobility impairments. Additional cognitive interviewing, which was not conducted as part of this study, may provide some insight into how PROMIS^® items relate to children with CP. For example, the first item may not be relevant to this clinical population due to its emphasis on sports and exercise. “Sports and exercise” may be ambiguous and open to interpretation (e.g. wheelchair basketball is a sport, working with resistance bands is exercise). Similarly, “kids my age” could refer to any kids of a similar age, including other children with CP, who may constitute a large proportion of a child’s social network with which they are making comparisons.

The fact that the PROMIS^® Short Form performed better than the PROMIS^® CAT suggests that the PROMIS^® CAT stopping rules stopped the test too soon before a precise estimate of mobility could occur. By adjusting the stopping rules to allow for administration of more items, the PROMIS^® CAT may perform satisfactorily, assuming the items do not display significant DIF in children with CP. Future work is necessary to determine if adjusting the PROMIS^® CAT discontinue rules, such as increasing the minimum and maximum number of items and/or decreasing the critical standard error would change how the CAT administers the item bank in children with limited mobility. Additional work may also indicate the need for CP-specific CAT calibrations or forced item inclusion to guarantee inclusion of CP-relevant item content in CAT administrations. Until more is known about how to improve the performance of the PROMIS^® pediatric Mobility CAT in children with CP, clinicians and researchers working with this population may elect instead to use a Short Form version of the measure. This would alleviate any concerns they may have about using the CAT and the Short Form showed good convergent and discriminant validity.

This work does not suggest that all CAT instruments function poorly in CP, but item choice and stopping rules may be important. The other CAT-administered measure in this study, the SHC CP-CAT (parent-proxy assessment) showed excellent construct validity. Notably, the SHC CP-CAT required all respondents to complete 15 items compared to an average of 6 for the PROMIS^® CAT. Had the PROMIS^® CAT required as many items, it may have performed comparably. Other differences between these two measures are that the SHC CP-CAT is characterized by 1) parent proxy administration, which is usually more closely associated with performance assessments, 2) calibration in children with CP, which may optimize item selection better than a more diverse calibration population, and 3) item development that was focused entirely on mobility as it relates to children with CP.

In terms of administration feasibility, the PROMIS^® CAT required few items (<7) and was completed quickly (<2 minutes) in this sample, which is consistent with CAT administration characteristics found in adults with moderate physical limitations [44]. This may suggest good accessibility of this measure in children with CP where lengthy self-report measures might present additional burden due to the physical limitations of the respondent. However, the fact that the CAT-administration did not take long is reflective not just of feasibility, but also of a problem with the CAT not administering a sufficient number of items to arrive at an accurate score. Indeed, those with the greatest mobility limitations (GMFCS III-V) completed the PROMIS^® CAT relatively more quickly because the CAT administered fewer items in this group compared to those with higher mobility capacity, resulting in a poor estimate of mobility for the lowest mobility group.

Approximately one in five children had the maximum possible score on the PedsQL^™ and two of the parent-report measures, the PODCI and FAQ, demonstrated similarly high rates of ceiling effects. In contrast, rates of ceiling effects were relatively low for the SHC CP-CAT and PROMIS^® measures. Ceiling effects indicate failure to detect variability at the high end of the mobility spectrum and that a measure should not be used to assess treatment outcomes (because there is no room for improvement). As such, compared to the legacy measures considered here, the SHC CP-CAT and PROMIS^® measures may demonstrate better responsiveness to change.

This study has a number of strengths. We collected mobility data from children and parents, and through performance-based measures. This allowed us to examine the validity of the PROMIS^® instruments against measures providing different methods of assessing mobility. One study limitation is that the sample had a relatively small representation of children with significant mobility limitations (GMFCS III-V), as these children typically do not undergo orthopedic surgery for improved function but for cosmesis, comfort, and/or pain and were excluded by design. This study also excluded children with cognitive impairment that hindered their ability to self-report. Future work is needed to determine how best to write items for and obtain self-report in children with cognitive impairment.

In conclusion, results support the validity of the pediatric PROMIS^® Mobility Short Form in children with CP and show that the PROMIS^® instruments have lower rates of ceiling effects compared to legacy measures except for the SHC CP CAT. The PROMIS^® Mobility CAT can be quickly administered to children with CP and although it correlates well with child- and parent-report of mobility, it does not correlate with performance-based measures or discriminate between known mobility groups. Further work is needed to determine whether adjustment to default CAT-administration settings improve the performance of the PROMIS^® Mobility CAT in this clinical population.