Measuring Activity Limitation Outcomes in Youth with Spinal Cord Injury

Abstract

Study Design

Cross-sectional

Objectives

The Pediatric Spinal Cord Injury Activity Measure (PEDI-SCI AM), which includes calibrated item banks (child and parent versions) for General Mobility, Daily Routines, Wheeled Mobility and Ambulation, can be administered using computerized adaptive tests (CATs) or short forms (SFs). The study objectives are: 1.) examine the psychometric properties of the PEDISCI AM item banks and 10-item CATs); 2.) develop and evaluate the psychometric properties of PEDI-SCI AM SFs.

Setting

U.S. Shriners Hospitals for Children (California, Illinois and Pennsylvania).

Methods

Calibration data from a convenience sample of 381 children and adolescents with SCI and 322 parents or caregivers were used to examine PEDI-SCI AM item banks, 10-item CATs and SF scores. We calculated group reliability, internal consistency (Cronbach's alpha), and interclass coefficients (ICCs) to assess agreement between 10-item CATs, SFs and item banks. The percent of the sample with highest (ceiling) and lowest (floor) scores was also determined. An expert panel selected items for 14 SFs.

Results

PEDI-SCI item banks, 10-item CATs and SFs demonstrate acceptable group reliability (0.73-0.96) and internal consistency (0.77-0.98). ICC values show strong agreement with item banks for 10-item CATs (0.72-0.99) and SFs. Floor effects are minimal (<15%). Ceiling effects are minimal for children with tetraplegia, but high in children with paraplegia for General Mobility (13.41-26.05%) and Daily Activities (12.99-32.71%).

Conclusions

The PEDI-SCI AM exhibited strong psychometric properties for children with tetraplegia. Replenishment of the General Mobility and Daily Routines item banks is needed to reduce ceiling effects noted for youth with paraplegia.

Relevant, precise and efficient measures are needed to examine the efficacy of rehabilitation programs for youth with spinal cord injury (SCI) and to support evidence-based practice. Generic measures currently used in this population have significant conceptual and psychometric limitations (1). For example, the PedsQL (2) includes items inappropriate for children with SCI (e.g. ‘it's hard for me to walk more than one block’ and ‘it's hard for me to run’), and the WeeFIM may not detect clinically meaningful change (3). The Pediatric Spinal Cord Injury Activity Measure (PEDI-SCI AM) was developed specifically to assess activity outcomes in youth with SCI (2-5), providing an alternative to generic pediatric outcome measures. The PEDI-SCI AM (4-8) includes activities important to youth with SCI and items assess a wide range of abilities in the following domains: General Mobility, Daily Routines, Wheeled Mobility and Ambulation.

Previous work determined item calibrations and verified PEDI-SCI AM item bank unidimensionality (8). PEDI-SCI AM calibrated item banks can be administered as Computerized Adaptive Tests (CATs), which use a computer program to select items. A CAT begins with a midrange item and subsequent items are administered based on the individual's responses. Items that are too hard, too easy, or irrelevant are not administered (9). As each item is administered, the score estimate precision increases. The program terminates, based on pre-determined rules specifying a level of precision (standard error) or maximum number of items, to yield a final score estimate. Scores for a given domain can be compared on repeated measures even though different items are administered because scores are based on the same metric. While there are many benefits to using CAT administration, static short form (SFs), comprised of items carefully selected from calibrated item banks, are used when computer administration is not feasible.

The objectives of this study are to: 1.) examine psychometric properties of PEDI-SCI AM item banks and 10-item CATs); 2.) develop PEDI-SCI AM SFs and evaluate their psychometric properties.

METHODS

Participants-Calibration Sample

PEDI-SCI AM calibration data were obtained from a convenience sample of 381 youth with acquired SCI (traumatic, transverse myelitis, tumor, etc.) for the child-reported version (age 8 to 21 years) and 322 parents/caregivers of children with SCI for the parent-reported version (age 4 to 21 years). Inclusion criteria were discharge from initial SCI rehabilitation with return to pre-injury environment for at least 3 months. Since the measure was developed specifically for children with acquired SCI, children with congenital (e.g., spina bifida) and progressive conditions (e.g., spinal muscle atrophy) were excluded from the study. Other exclusion criteria were: English not primary language; brain injury that interfered with ability to read, comprehend and respond to items. Data were collected within the Shriners Hospitals for Children System (Philadelphia, Chicago, and Northern California hospitals). The study was approved by the Institutional Review Board of each facility. Parents and children provided consent and assent, respectively. We certify that all applicable institutional and government regulations concerning the ethical use of human volunteers were followed during the course of this research.

PEDI-SCI CAT Development and Simulation

The CAT algorithms were developed at Boston University for each domain and programmed to administer the first item with highest Fisher Information value at the average sample score level. Post-Hoc simulations were conducted on the calibration dataset to estimate the 10-item CAT scores and standard error (SE). The person score and standard error were estimated using the weighted likelihood estimation method (10). Each subsequent item was selected based on maximization item Fisher information value at the current score level, and the score and SE were recalculated after feeding the program with known response until 10 items had been administered. The final scores were transformed to the T score scale with a population mean and standard deviation of 50 and 10, respectively. Higher scores represented higher function.

PEDI-SCI Short Form Development

An expert panel, comprised of 12 professionals (1 medical doctor, 1 psychologist, 4 physical therapists, 6 occupational therapists), attended a one-day meeting to select SF item candidates from PEDI-SCI AM item banks by reviewing two item parameters: item difficulty [measured in logits] and item discrimination [measured as the slope of the item characteristic curve (ICC)]. Groups identified SF item candidates using spreadsheets with PEDI-SCI items hierarchically organized based on logit scores with an additional column for ICC slopes. Logit scores were used to select items with an appropriate range of difficulty and ICC slopes identified items that best discriminated among persons with different ability levels. Finally, SF item candidates were reviewed from a clinical perspective to ensure that important aspects of function were included in each SF. Groups presented initial recommendations and used an iterative process to identify 7-12 items for the 4 domains, with separate SFs for child/parent respondents (8 initial SFs). Based on differences in abilities of youth with paraplegia and tetraplegia and experience developing adult SCI-FI SFs (11), separate paraplegia and tetraplegia SFs were developed for the Daily Routines and Wheeled Mobility domains (4 additional SFs: 2 parent/2 child). Bowel and bladder management is critical for youth with SCI and items assessing these abilities are in the item bank, but were not selected for inclusion in the SF because the items did not have a high information function. Also, these items are not relevant for youth with SCI not using bowel and bladder programs and could not be scored. Separate SFs were developed for Manual and Powered Wheeled Mobility for youth with tetraplegia to yield a total of 14 separate PEDI-SCI SFs. Iterative psychometric analyses to examine properties of the 14 SFs were conducted by two investigators (MDS, MJM) to ensure optimal item selection.

Psychometric Analyses

Calibration sample data was used to calculate group-level reliability for the full item banks, 10-item CATs and SFs for each domain, (separate for child and parent) defined as:

$$\frac{{\sigma }_{\theta }^2-E\left(S{E}^2\right)}{{\sigma }_{\theta }^2}$$

where E (SE²) is the mean of estimated score standard errors in each group and σ²_θ is the variance of the estimated score for child and parent respondents. We also calculated Cronbach's alpha for the full item bank and SF items (but not for10-item CATs since different items were administered). Interclass correlation coefficients (ICCs) were calculated to determine agreement between the total item bank and 10-item CATs and SFs. We examined precision of score estimates by limiting the score range (0 through 80) and calculating the standard error. Finally, we calculated the percent of the sample with the highest (ceiling) and lowest (floor) score for the full item banks, 10-item CATs and SFs.

RESULTS

PEDI-SCI sample

The mean age (SD) of the participants for the child-reported version was 15.5 (SD = 3.5) years; most were boys (55%) and white (82%); 57.6% had paraplegia; 54.2% had complete injuries, as defined by American Spinal Injury Association Impairment Scale (AIS). Mean age (SD) for the parent-reported version was 13.6 (SD = 4.5) years; most were boys (55%); white (82%); 56% had paraplegia; 52% had complete injuries (AIS). Details are reported elsewhere (7).

PEDI-SCI Item Banks

The PEDI-SCI AM item banks include four domains: General Mobility (19 child items, 18 parent items); Daily Routines (192 child items, 185 parent items); Wheeled Mobility (62 child items, 64 parent items); and Ambulation (25 child items, 25 parent items) [see supplemental material (8) for complete list of items]. Core items (N=178) were administered to all participants; supplemental items (N=229) were administered based on responses to screener questions (e.g., age-specific items, bowel and bladder program, ambulation status) (8).

Descriptive Statistics

Tables 1-3 present the range, mean values and SDs for the full item banks, 10-item CATs and SFs. Scores were calculated using the T metric (mean=50; SD=10). Mean values were near 50 with some exceptions. For Ambulation, parent respondents were below the mean for CAT (33.0) and SF (33.4) administrations. Where separate SFs were developed for different levels of lesion, lower mean scores were noted for tetraplegic-specific SFs, as expected (Child and Parent Daily Routines Tetraplegia SFs = 44.4 and 43.6, respectively; Child and Parent Power Wheeled Mobility Tetraplegia SFs = 34.4 and 35.6, respectively; Child and Parent Manual Wheeled Mobility Tetraplegia SFs=42.9 and 43.9 respectively). Higher mean scores were noted for paraplegic-specific SFs (Child and Parent Daily Routines Paraplegia SFs = 54.7 and 55.5, respectively; Child Wheeled Mobility Paraplegia SF = 54.0).

Table 1.

Descriptive Statistics of the Pedi-SCI Parent-reported and Child-reported General Mobility and Ambulation Scales^*

Respondent	Version	N	Mean	SD	Range	Items	Group Level Reliability	Cronbach's Alpha
General Mobility
Child	Full item bank	374	49.06	10.37	18.48-63.04	20	0.92	0.97
	10-item CAT	374	49.12	10.25	19.04-62.76	10	0.90	NA
	Short Form	374	48.83	10.02	20.69-61.63	10	0.88	0.95
Parent	Full item bank	319	49.94	10.79	26.94-69.32	18	0.94	0.97
	10-item CAT	318	49.90	10.95	27.17-69.20	10	0.94	NA
	Short Form	319	49.80	10.46	28.85-67.99	9	0.91	0.95
Ambulation
Child	Full item bank	123	50.12	13.72	11.96-74.69	25	0.95	0.97
	10-item CAT	123	50.05	1371	12.04-73.88	10	0.94	NA
	Short Form	123	50.22	12.53	22.03-72.46	11	0.91	0.95
Parent	Full item bank	116	32.98	7.21	17.30-50.92	25	0.96	0.97
	10-item CAT	116	33.03	7.19	17.30-50.92	10	0.95	NA
	Short Form	116	33.42	6.58	21.93-50.77	10	0.92	0.94

^*Separate short forms for paraplegia and tetraplegia were not developed for these domains.

Table 2.

Descriptive Statistics of the Pedi-SCI Daily Routines Scales

Respondent	Version	N	Mean	SD	Range	Items	Group Level Reliability	Cronbach's Alpha
Child	Full item bank	372	52.45	11.32	−7.44-75.35	192	0.94	0.991
	10-item CAT	372	52.08	10.79	6.43-74.40	10	0.91	NA
	Paraplegia Short Form	214	54.70	4.13	29.08-73.53	11	0.79	0.882
	Tetraplegia Short Form	156	44.42	10.01	20.58-58.43	12	0.76	0.94
Parent	Full item bank	318	51.36	11.15	11.56-75.01	185	0.96	0.993
	10-item CAT	317	51.19	10.97	10.01-68.77	10	0.93	NA
	Paraplegia Short Form	177	55.52	5.99	26.83-62.43	12	0.89	0.912
	Tetraplegia Short Form	136	43.58	8.61	25.04-58.24	11	0.86	0.95

¹To calculate Cronbach's Alpha, we removed 54 items not completed by all participants due to use of specific device

²removed 1 item skipped by children not using a bowel program

³removed 52 items due to use of specific device.

Table 3.

Descriptive Statistics of the Pedi-SCI Wheeled Mobility Scales*

Respondent	Version	N	Mean	SD	Range	Items	Group Level	Cronbach's Alpha
Child	Full item bank	329	48.14	13.46	−35.53-70.81	62	0.94	0.971
	10-item CAT	329	48.01	13.8	−32.91-70.23	10	0.89	NA
	Paraplegia Short Form	188	54.00	8.47	35.17-66.95	7	0.81	0.77
	Tetraplegia MWC Short Form	97	42.94	12.23	11.26-67.20	8	0.73	0.88
	Tetraplegia PWC Short Form	80	34.42	13.80	−28.03-60.88	10	0.85	0.82
Parent	Full item bank	277	47.72	11.5	3.64-70.21	64	0.96	0.982
	10-item CAT	277	47.69	11.61	1.61-69.06	10	0.94	NA
	Paraplegia Short Form	153	52.78	7.73	24.69-65.97	8	0.89	0.89
	Tetraplegia MWC Short Form	80	43.86	10.63	22.34-66.10	10	0.90	0.95
	Tetraplegia PWC Short Form	61	35.59	11.65	3.36-62.69	9	0.83	0.80

¹To calculate Cronbach's Alpha, we removed 26 i4ems not completed by all participants due to use of specific device

²removed 25 items due to use of specific device

³ score is lower because one subject responded to all power wheelchair items as “unable.”

Group-level reliability (0.73 to 0.96) was moderate to high for all domains and versions (full item banks, 10-item CATs and SFs). Internal consistency values for the full item banks and SFs were also acceptable (Cronbach's alpha=0.77 to 0.99). For General Mobility and Ambulation item banks (child and parent versions) agreement was high for the CATs and SFs (ICC range = 0.97-0.99). For the Daily Routines and Wheeled Mobility item banks (child and parent versions) agreement was high for the CATs (ICC=0.97-0.99); however, for Daily Routines, agreement was lower for paraplegia-specific SFs for children (ICC=0.72) and parents (ICC=0.83) compared to tetraplegia-specific SFs for children (ICC=0.93) and parents (ICC=0.95). For Wheeled Mobility SFs, agreement was strong (ICC range=0.90-0.99).

Score Estimate Precision

Figures 1-4 display the standard error observed for each domain and version. The dashed line represents reliability at 0.90; points below the line indicate that the score reliability>0.9; above the line score reliability<0.9. The General Mobility (Figure 1) and Ambulation (Figure 2) plots demonstrate excellent precision for low mid-range score estimates that varies somewhat by version (full item banks>10-item CATs>SFs). For General Mobility, the precision for parent and child respondents is similar. For Ambulation, precision is better for parent respondents, but it should be noted that the sample size is smaller as fewer children were ambulatory. Results for Daily Routines (Figure 3) should be viewed with caution due to the problems noted with ceiling effects; however, a similar pattern is noted. Finally, for Wheeled Mobility (Figure 4), precision estimates vary by mode of administration with high standard errors for tetraplegia SFs and 10-item CATs for both child and parent respondents; however, these instruments were completed by subsets of the sample (e.g., only manual wheelchair users).

Figure 1.

a. General Mobility (child-reported) Standard Error (dashed line represents reliability = 0.90)

1b. General Mobility (parent-reported) Standard Error (dashed line represents reliability = 0.90)

Figure 4.

a. Wheeled Mobility (child-reported) Standard Error (dashed line represents reliability = 0.90)

4b. Wheeled Mobility (parent-reported) Standard Error (dashed line represents reliability = 0.90)

Figure 2.

a. Ambulation (child-reported) Standard Error (dashed line represents reliability = 0.90)

2b. Ambulation (parent-reported) Standard Error (dashed line represents reliability = 0.90)

Figure 3.

a. Daily Routines (child-reported) Standard Error (dashed line represents reliability = 0.90)

3b. Daily Routines (parent-reported) Standard Error (dashed line represents reliability = 0.90)

Ceiling and Floor Effects

Tables 4-6 present floor and ceiling effects for the PEDI-SCI banks, 10-item CATs and SFs. Floor effects are acceptable (<15%) across all domains, respondent types and administration versions for children with paraplegia and tetraplegia (range=0% to 10.29%). Ceiling effects for children with tetraplegia are acceptable (<15%) across all domains, respondent types and administration versions. For children with paraplegia, ceiling effects for the General Mobility domain are acceptable (<15%) for parent respondents, but high for child respondents across all versions (23.61-26.85%). A significant problem with ceiling effects is noted for Daily Routines in children with paraplegia; ceiling effects are unacceptable, except for the Parent Daily Routines full item bank (13.0%). For Wheeled Mobility ceiling effects are acceptable for parent and child respondents for all versions, except the Child Wheeled Mobility SF (15.43%).

Table 4.

Pedi-SCI General Mobility and Daily Routines Scales^* Sample Size (N), Range, % Floor, % Ceiling

Domain	Version	Tetraplegia				Paraplegia
Respondent		N	Range	% Floor	% Ceiling	N	Range	% Floor	% Ceiling
General Mobility
Child	Full item bank	157	18.3-63.1	3.18	9.55	217	39.2-63.1	0.00	23.61
	10-item CAT	157	19.1-62.8	3.18	10.19	217	38.4-62.8	0.00	25.00
	Short Form	157	20.7-61.6	5.73	9.55	217	39.0-61.6	0.00	26.85
Parent	Full item bank	136	25.1-69.4	10.29	3.68	179	30.8-69.4	0.00	13.41
	10-item CAT	136	25.4-69.2	10.29	3.68	178	25.4-69.2	0.56	13.41
	Short Form	136	28.8-68.0	11.76	3.68	179	32.4-68.0	0.00	13.41
Daily Routines
Child	Full item bank	156	-12.0-78.2	0.64	3.85	215	28.7-79.6	0.00	21.50
	10-item CAT	156	6.4-64.0	1.28	9.62	215	28.1-74.4	0.00	32.71
	Short Form*	156	20.6-58.4	5.77	14.10	215	29.1-73.5	0.47	50.00
Parent	Full item bank	136	11.8-75.8	0.00	0.74	177	16.6-75.2	0.56	12.99
	10-item CAT	136	10.0-69.2	3.68	2.21	177	19.4-69.2	0.56	20.34
	Short Form*	136	25.0-58.2	8.09	6.62	177	26.8-62.4	0.56	30.51

^*Separate SFs for children with paraplegia and tetraplegia

Shaded area denotes ceiling effect >15%

Table 5.

Pedi-SCI Ambulation Scales: Sample Size (N), % Floor, % Ceiling

Respondent	Version	Paraplegia and Tetraplegia
		N	% Floor	% Ceiling
Child	Full item bank	123	2.44	7.32
	10-item CAT	123	2.44	8.94
	Short Form	123	5.69	12.2
Parent	Full item bank	116	6.03	2.59
	10-item CAT	116	6.03	2.59
	Short Form	116	10.34	2.59

Table 6.

Pedi-SCI Wheeled Mobility Scales^*: Sample Size (N), % Floor, % Ceiling

Respondent	Version	Tetraplegia						Paraplegia
		Manual			Power
		N	% Floor	% Ceiling	N	% Floor	% Ceiling	N	% Floor	% Ceiling
Child	Full item bank	97	0.00	5.15	80	1.25	6.25	189	0.00	7.41
	10-item CAT	98	0.71	5.00	80	0.71	5.00	191	0.00	9.52
	Short Form*	98	4.12	8.25	80	1.25	6.25	191	0.00	15.43
Parent	Full item bank	80	2.27	4.55	61	0.00	0.00	153	1.19	4.76
	10-item CAT	80	2.27	4.55	61	2.27	4.55	153	0.00	9.52
	Short Form*	80	4.55	4.55	61	0.00	0.00	153	1.20	8.43

^*Separate SFs for children with paraplegia and tetraplegia

Shaded area denotes ceiling effect >15%.

DISCUSSION

The PEDI-SCI AM is the first activity limitation measure developed specifically for youth with SCI. Initial examination of the PEDI-SCI AM demonstrated strong reliability and internal consistency for all item banks, 10-item CATs and SFs. Simulated 10-item CAT and most SF scores provide precise estimates and show strong agreement with the full item bank, indicating that CAT programs select appropriate items and SF items that adequately represent the full item banks. The exception is the Daily Routines SF for children with paraplegia where the lower agreement with the item bank may limit item selection due to high ceiling effects.

Analysis of floor and ceiling effects indicated that the range of difficulty represented in the Ambulation and Wheelchair items was appropriate for youth with paraplegia and tetraplegia. Floors effects are minimal (<15%) across all domains and respondent types and the full item banks, 10-item CATs and SFs, indicating that all PEDI-SCI AM measures had sufficient content range for all children, regardless of their level of lesion.

The PEDI-SCI AM measures had the necessary content range at the upper end of the scale for youth with tetraplegia, as evidenced by small ceiling effects. Likewise, for youth with paraplegia, ceiling effects for the child and parent versions of the Mobility and Ambulation SFs were acceptable. However, for children with paraplegia, ceiling effects for the child and parent Daily Routines and the child General Mobility SFs were unacceptably high for all versions and may underestimate a child's full ability in these domains, especially if the child was functioning at a relatively high level.

The ability to improve the psychometric properties of a measure by adding and calibrating new items (referred to as item bank replenishment) is a significant advantage of IRT-based measures. The next step in refinement of the PEDI-SCI AM involves replenishment of the Daily Routines and General Mobility item banks. Replenishment can enhance item banks without altering the underlying scale and we have successfully replenished other item banks (12). We will develop new items to assess the upper range of abilities and calibrate these items in a sample of youth with SCI and these new items will be added to the existing item banks and SFs.

Given the void in robust outcome instruments for pediatric SCI, the SCI PEDI-AM is an important contribution that can elevate pediatric SCI practices. SCI PEDI-AM is the first measure with items specifically developed for youth with SCI that uses sophisticated measurement approach. It can be administered as a SF or CAT and, while the PEDI-SCI AM CATs performed better than the SFs, some SFs were comparable to the simulated 10-item CAT, providing evidence that the SFs can be an acceptable alternative to CAT administration. Young children are capable of self-reporting on their health conditions; however, discordance between child- and parent-report of HRQOL outcomes has been established in clinical samples and typically developing children (13, 14). It is important to assess outcomes from both parent and child perspectives (15) and the PEDI-SCI AM offers parent and child versions. These versions are based on different metrics and scores cannot be directly compared, but the ability to administer the measure to parent and child respondents provides clinicians and researchers with different options based on the specific situation and purpose of the assessment. Finally, PEDISCI AM can be used with its companion PEDI SCI participation measure (4) and since item banks are linked to the Spinal Cord Injury-Functional Index (SCI-FI), an IRT-based measure developed for the adult population, function can be tracked across the lifespan (8). The next step is to validate these measures in a new sample and compare the responsiveness of the PEDI-SCI AM CATs and SFs to generic measures.

There are several limitations to this work: the sample, drawn from three Shriners Hospitals for Children in the United States, may not be representative; and the sample is relatively small, but acceptable given the overall number of children with SCI (estimated number of new injuries is 1,500 per year) (16).

CONCLUSION

The PEDI-SCI AM uses contemporary measurement approaches to create new instruments for pediatric SCI research and clinical practice. For the first time, activity outcomes can be assessed with a measure developed specifically for youth with SCI. The ability to administer any or all of the four domains and select child and/or parent as respondent(s) provides a comprehensive measure that can be customized for multiple purposes and the ability to use PEDI-SCI AM CAT or SF versions provides administrative options for different settings.