Abstract
Background
The Patient Reported Outcomes Measurement Information System (PROMIS®) was developed to provide patient-reported outcome (PROs) measures that are designed as being universally relevant across health conditions, low burden and precise. A major problem for research and clinical practice in cerebral palsy (CP) is the void of outcomes instruments that are capable of evaluating the wide range of abilities and broad age spectrum inherent in this clinical population. Given the tremendous potential of PROMIS® , the research questions for this study were “How do PROMIS® pediatric computer adaptive tests (CAT) and short forms (SF) detect change in children with CP following elective musculoskeletal surgery?” and “How do PROMIS® instruments compare to the Pediatric Quality of Life Inventory™ Cerebral Palsy Module Version 3.0 (PedsQL CP), Pediatric Outcomes Data Collection Instrument (PODCI), the Timed Up and Go (TUG) and the Gross Motor Functional Measure (GMFM).
Methods
PROMIS® Pediatric CATs and SFs and the PedsQL, PODCI, TUG and GMFM were administered before and after surgery. Effect Size (ES) and Standardized Response Mean (SRM) were calculated. Floor and ceiling effects were evaluated and, exposure rates for the PROMIS® item banks were examined.
Results
ES and SRM for all PROMIS® Pediatric Measures were non-significant. PedsQL CP detected significant, positive change in mobility at 6 (ES =0.26; SRM = 0.31) and 12 (ES = 0.36; SRM=0.36) months; pain at 12 months (ES=0.29; SRM=0.34); and fatigue at 6 (ES=0.24; SRM=0.22) and 12 (ES=0.36; SRM=0.41) months. Significant negative changes were detected by the PODCI (ES= - 0.20; SRM= - 0.26), GMFM (ES= - 0.13; SRM = -0.24) and TUG (ES= - 0.29; SRM= - 0.25). Ceiling effects were high. Exposure to an appropriate range of the PROMIS® Mobility item bank was limited.
Conclusion
PROMIS® measures were less able to detect change than other measures. PROMIS® measures may be improved by tailoring start\stop rules or by adding items to include content appropriate for children with mobility impairments.
Level of Evidence
Diagnostic Study, Level III
INTRODUCTION
The need for outcome measures that assess health-related quality of life is widely acknowledged. As a response, the Patient Reported Outcomes Measurement Information System (PROMIS®) was developed to provide a new generation of patient-reported outcome measures (PROs).1 PROMIS® pediatric PROs were developed using item banking2,3 and Item Response Theory4 methodology and may be administered either as computerized adaptive tests (CATs) or as brief, fixed-length “short forms” (SFs). CAT administration uses a computer algorithm to select items based on responses to previously administered items. With this approach, a small number (e.g., 5-12) of carefully selected items generate a precise estimate of the individual's ‘ability level’. PROMIS® pediatric CATs are administered with a standard ‘stopping rule’ that involves the administration of at least 5 items with additional items added until a standard error of 0.4 is reached or a maximum of 12 items are answered.5 Most PROMIS® pediatric PRO may also be administered as SFs comprised of 8-10 carefully selected items.
Given the tremendous potential of PROMIS® instruments, the likelihood of change following orthopedic surgery6-8 and the need for quality measures to assess the effectiveness of surgery for children with CP,9 this study examined the ability of the PROMIS® instruments to detect change in youth with CP as compared to that of the Pediatric Quality of Life Inventory™ Cerebral Palsy Module Version 3.0 (PedsQL CP), Pediatric Outcomes Data Collection Instrument (PODCI), the Timed Up and Go (TUG) and the Gross Motor Functional Measure (GMFM).
MATERIALS AND METHODS
Youth aged 8 to 21 years with CP undergoing elective musculoskeletal surgery to improve function were enrolled. Surgery was performed at 11 hospitals in the United States. Since the purpose of the study was to evaluate the outcomes instruments as opposed to evaluating the effectiveness of surgery, there were no attempts to standardize the surgeon, surgical procedures or post-operative therapy. Exclusion criteria were: surgery for reasons other than to improve function (pain; positioning; cosmesis); cognitive impairment that limited ability to read; non-English speaking and; diagnoses other than CP. Each site obtained local Institutional Review Board approval.
Measures
Measures were administered within a week before surgery and at 6 and 12 months following surgery. Table 1 provides the domain, item content descriptions and common context for each measure.
Table 1.
Domains, Content and Common Context of PROMIS and Legacy Measures.
| Measure & Domain | N Item | Content | Common Context |
|---|---|---|---|
| PROMIS Measures (child reported) | |||
| Mobility CAT | 6.7* | Trouble doing activities enjoyed most, standing, standing on tiptoes, walking up stairs, moving legs, getting up from floor, doing sports and exercise, keeping up with other kids in play | “In the Past Seven Days....” |
| Mobility SF | 8 | ||
| Pain Interference CAT | 7.6* | Frequency of pain resulting in feeling angry, trouble sleeping, doing schoolwork; hard to pay attention, run, walk one block, have fun and stay standing | |
| Pain Interference SF | 8 | ||
| Fatigue CAT | 8.6* | Frequency of being tired/weak and being hard to keep up with school work, pay attention, play/go out with friends; trouble starting/finishing things, too tired to do sports and exercise, things like to do, and things outside | |
| Fatigue SF | 10 | ||
| PedsQL CP Module (child reported) | |||
| Movement & Balance | 5 | Moving extremities, balance in sitting and standing | “Within the Last Month...” |
| Hurt & Pain | 4 | Joint and muscle pain and stiffness, trouble sleeping because of muscle hurt | |
| Fatigue | 4 | Tired, weak, need to rest, lack of energy | |
| Daily activities | 9 | Dressing, grooming, showering, toileting | |
| PODCI (parent reported) | |||
| Basic Mobility | 11 | Putting on coat, climbing one flight of stairs, walking short distances, sitting, on/off toilet, in/out of bed, standing, bending, use of assistive devices | “In the Last Week...: |
| Sports & Physical Functioning | 21 | Running short distance, riding a bike, walking long distances, multiple flights of stairs, group activities and sports | |
| Pain & Comfort | 3 | Pain and pain interference with activities; i.e.,: “How much pain did you child have last week and did it interfere with activities at home and school?” | |
| GMFCS (Performance-based) | |||
| Stand (Item D) | 13 | Standing with/without holding on, weight shirt, both legs and single limb stance, up/down from seated position, up from half kneeling, down to floor, pick up objects from floor | NA |
| Walk/Run/Jump and Stand (Item E) | 24 | Walking forward/backward different distances with/without holding object, parallel lines, straight lines, stepping over obstacles, jumping, hopping, stairs with/without railing | |
| Timed Up and Go (TUG) (Performance-based) | |||
| Walking | 1 | Average time to stand and walk specified distance | NA |
Measures are noted as child-report or parent report or performance based. CAT=computer adaptive testing; SF=short form; N=number
average number of items administered by the CAT.
The Gross Motor Function Classification System (GMFCS)10 rates ambulatory ability as five levels ranging between walks without limitations (Level I) and is transported in a manual wheelchair (Level IV). GMFCS levels were assigned at baseline and follow up.
PROMIS® PROs
Children completed the PROMIS® version 1.0 Pediatric CATs and SFs on tablet computers. Mobility, Pain Interference and Fatigue PROMIS® domains were selected based on their relevance to post-surgical changes likely to be observed.6,7-9,11 PROMIS® Pediatric PROs use a five-option response scale that ranges from ‘no trouble’ to ‘not able to do’ (mobility) and from “never” to “almost always” (pain interference and fatigue). PROMIS® measures are scored on a T metric where the mean=50 and Standard Deviation=10. PROMIS® CATs were administered using the standard stopping rules.5 The PROMIS® Pediatric Mobility and Pain SFs have 8 items; the Fatigue SF has 10 items.
PedsQL CP
The Pediatric Quality of Life Inventory™ Cerebral Palsy Module Version 3.0 (PedsQL CP) is a disease (CP) specific measure.12 The PedsQL CP uses a five-option response scale ranging from “never a problem” to “almost always a problem.” The PedsQL CP Movement and Balance (MB), Pain\Hurt and Fatigue domains were assessed via child report.
PODCI
The Pediatric Outcomes Data Collection Instrument (PODCI) has 114 items and five subscales.13 The PODCI uses a five-point response scale, ranging from ‘definitely yes’ to ‘definitely not.’ The PODCI transfer/basic mobility, sports/physical function and pain/comfort subscales were assessed via parent report.
Performance Measures
For the Timed Up-and-Go (TUG),14 the time to rise from a chair, walk straight for 3 meters, and return to sitting was recorded. TUG scores were recorded as the average time (seconds) across three trials. The Gross Motor Function Measure (GMFM)15 D (standing) and E (walking, running and jumping) domains were assessed through observation, and items were scored using a four-point scale: 0= “does not initiate”, 1= “initiates”, 2= “partially initiates” and 3= “completes.”
Data Analysis
We calculated the mean, standard deviation and range of scores for all measures. We examined whether the scores were in a valid range, examined mean scores for the legacy measures and compared them to published studies using similar samples, when available. For each measure we calculated the Effect Size (ES) and Standardized Response Mean (SRM) using the following equations.
where SD = standard deviation We used the bootstrap method to estimate the confidence intervals (CI) of the ES and SRM. To facilitate comparisons across measures, the direction of change scores for each measure was adjusted, if necessary, so that across all measures a positive ES/SRM value is interpreted as an improvement in mobility, pain and fatigue and a negative ES/SRM value is interpreted as a worsening in mobility, pain and fatigue. We also calculated ceiling and floor effects. Finally, we examined CAT performance by determining exposure rates (% of sample completing items in the item bank) and identified items not administered for the PROMIS® Mobility, Pain Interference and Fatigue measures.
RESULTS
Of 242 children who were screened for study participation, 115 met study eligibility. Children were excluded for the following reasons: under 8 years of age (n=40, 31%), unable to self-report (n=44, 35%), musculoskeletal surgery with non-functional goal (n=24, 19%), did not speak English (n=8, 6%), declined (n=6, 5%) and other (n=5, 4%). Two subjects were withdrawn due to an intraoperative change in the surgical plan (n=1) and being misdiagnosed (n=1). Of the 113 children for whom baseline data were available, 98 provided 6-month data (8 lost to follow up; 7 missed 6-month visit) and 86 provided 12-month data (2 additional lost to follow up; 16 missed 12-month follow-up visit; 1 refused).
Analyses of demographic and clinical variables for youth who completed the follow up (Table 2) were compared to non-completers and no significant group differences were found. A total of 46% of youth had multi-level surgeries and the majority had multiple procedures. The most common procedures were tendon lengthening and tendon release.
Table 2.
Participant Demographic and Clinical Characteristics.
| Variable | Baseline N = 113 | 6 months N = 98 | 12 months N = 86 |
|---|---|---|---|
| Age at surgery Mean (SD) | 12.7 (3.0) | 12.7 (3.1) | 12.4 (3.0) |
| Gender N (%) | |||
| Female | 43 (38.1) | 38 (38.8) | 35 (40.7) |
| Male | 69 (61.1) | 59 (60.2) | 50 (58.1) |
| Missing | 1 (1.0) | 1 (1.0) | 1 (1.2) |
| Race N (%) | |||
| White | 89 (78.8) | 78 (79.6) | 66 (76.7) |
| Black | 13 (11.5) | 10 (10.2) | 10 (11.6) |
| Asian | 5 (4.4) | 5 (5.1) | 5 (5.8) |
| American Indian | 1 (0.9) | 1 (1.0) | 1 (1.2) |
| Other | 4 (3.5) | 3 (3.1) | 3 (3.5) |
| Missing | 1 (0.9) | 1 (1.0) | 1 (1.2) |
| Ethnicity N (%) | |||
| Non-Hispanic | 93 (82.3) | 81 (82.7) | 70 (81.4) |
| Hispanic | 19 (16.8) | 16 (16.3) | 15 (17.4) |
| Missing | 1 (0.9) | 1 (1.0) | 1 (1.2) |
| Surgery N (%) | |||
| Upper extremity | 13 (11.5) | 12 (12.2) | 10 (11.6) |
| Lower extremity | 100 (88.5) | 86 (87.8) | 76 (88.4) |
| GMFCS Classification N (%) | |||
| Level I | 45 (39.8) | 41 (41.8) | 36 (41.9) |
| Level II | 41 (36.3) | 36 (36.7) | 30 (34.9) |
| Level III | 17 (15.0) | 14 (14.3) | 13 (15.1) |
| Level IV, V | 7 (6.2) | 7 (7.1) | 6 (7.0) |
| Missing | 3 (2.7) | 0 | 1 (1.2) |
N=Number; SD=Standard Deviation; GMFCS=Gross Motor Function Classification System
Table 3 displays the mean, range and standard deviation for each measure. As anticipated, the means for PROMIS® scores on the CATs and SFs were below 50. The PedsQL CP mean scores were slightly higher than previously reported.16 As expected, the PODCI mean scores were lower than those reported for typically developing children and similar to scores reported for children with CP.17 Walking speed was comparable to CP walking speed 18 and 50% slower than walking speed of typically developing children.19
Table 3.
Descriptive Statistics of Pediatric Outcome Measures Scores.
| Measure/Domain n=baseline, 6 and 12 mo. | Baseline Mean (SD) Range | 6 months Mean (SD) Range | 12 months Mean (SD) Range |
|---|---|---|---|
| PROMIS | T Score, 50, SD 10 | ||
| Mobility CAT1 (n= 94, 81, 69) | 43.5 (6.5) | 42.2 (7.1) | 42.9 (6.5) |
| 27.7-61.7 | 24.3-61.7 | 30.4-61.7 | |
| Mobility SF1 (n=99, 81, 69) | 41.6 (7.7) | 40.6 (8.1) | 41.3 (8.3) |
| 26.0-59.0 | 24.0-59.0 | 25.0-59.0 | |
| Pain Interference CAT (n=109, 90, 79) | 47.5 (10.5) | 47.0 (10.8) | 48.8 (11.4) |
| 32.2-67.2 | 32.2-68.4 | 32.2-71.3 | |
| Pain Interference SF (n=113, 93, 79) | 47.6 (10.0) | 47.6 (10.4) | 48.9 (10.3) |
| 34.0-65.0 | 34.0-69.0 | 34.0-73.0 | |
| Fatigue CAT (n=108, 90, 78) | 45.5 (11.9) | 43.0 (12.0) | 43.0 (12.3) |
| 25.6-68.9 | 25.6-66.0 | 25.6-66.8 | |
| Fatigue SF (n=112, 93, 79) | 45.4 (10.9) | 45.1 (10.4) | 43.7 (10.8) |
| 30.0-69.0 | 30.0-69.0 | 30.0-64.0 | |
| PedsQL CP Module | Possible Scaled Score 0 (almost always problem) – 100 (never problem) | ||
| Movement & Balance* (n=95, 81, 66) | 81.8 (17.9) | 86.2 (17.2) | 87.5 (16.0) |
| 30-100.0 | 20.0-100.0 | 20.0-100.0 | |
| Hurt & Pain (n=108, 92, 74) | 76.1 (22.1) | 80.0 (20.9) | 80.7 (20.9) |
| 0.0-100.0 | 18.8-100.0 | 12.5-100.0 | |
| Fatigue (n=108, 92, 74) | 72.4 (22.0) | 76.6 (19.6) | 79.6 (19.1) |
| 12.5-100.0 | 31.3-100.0 | 31.3-100.0 | |
| PODCI | Possible Scaled Score 0 (poor outcome\worse health)-100 (best possible health) | ||
| Transfer/Basic Mobility* (n=100, 82, 71) | 84.3 (16.9) | 80.9 (21.1) | 82.5 (19.4) |
| 20-100.0 | 18.0-100.0 | 21.0-100.0 | |
| Sports/Physical Function* (n=100, 82, 71) | 54.3 (20.9) | 50.8 (23.4) | 53.6 (23.4) |
| 3.0-97.0 | 0.0-97.0 | 6.0-97.0 | |
| Pain & Comfort (n=113, 92, 80) | 78.4 (22.5) | 78.5 (24.2) | 78.4 (23.9) |
| 17.0-100.0 | 12.0-100.0 | 8.0-100.0 | |
| GMFM | |||
| Standing function* (n=97, 80, 69) | 31.0 (9.0) | 31.3 (8.3) | 31.4 (9.2) |
| 1.0-39.0 | 5.0-39.0 | 3.0-39.0 | |
| Walk/Run/Jump and Stand* (n=97, 80, 69) | 53.1 (21.3) | 51.9 (21.6) | 53.1 (20.9) |
| 0.0-72.0 | 3.0-78.0 | 5.0-72.0 | |
| TUG (seconds)** (n=93, 75, 66) | 10.2 (7.5) | 11.9 (4.0)** | 11.8 (9.3) |
| 5.0-67.0 | 4.0- 85.6 | 5.0-54.0 | |
n=number of subjects; CAT-Computer Adaptive Test; SF=Short Form
subjects having surgery on lower extremity only
One outlier (TUG score = 116.7) was removed from analyses
Ability to Detect Change
ES and SRM values are summarized for the mobility (Table 4), pain interference/pain (Table 5) and fatigue (Table 6) measures. The PROMIS® Mobility CAT and SF did not detect change at either the 6- or 12- months. The PedsQL CP MB showed improvement at 6- and 12 months. The PODCI Transfer\Basic Mobility scale and GMFM D showed a small decline for the 6 months and no change at the 12 months. The TUG showed a small significant decline in mobility for the 6- and 12-months. The PODCI Sports\Physical Function and GMFM E did not detect change at any of the two intervals.
Table 4.
Standard Response Mean (SRM) and effect size (ES) estimates for Mobility Measures for Lower extremity Surgical Cases Only.
| Instrument | Baseline to 6 Month | Baseline to 12 month | ||||
|---|---|---|---|---|---|---|
| N | ES (CI) | SRM (CI) | N | ES (CI) | SRM (CI) | |
| PROMIS CAT | 78 | −0.14 (−0.34, 0.06) | −0.16 (−0.38, 0.07) | 67 | 0.00 (−0.23, 0.23) | 0.00 (−0.23, 0.25) |
| PROMIS SF | 80 | −0.10 (−0.27, 0.07) | −0.13 (−0.36, 0.09) | 68 | 0.06 (−0.14, 0.30) | 0.07 (−0.17, 0.31) |
| PedsQL CP Movement & Balance Scale | 78 | 0.26 (0.07, 0.45) | 0.31 (0.08, 0.56) | 66 | 0.36 (0.14, 0.58) | 0.36 (0.13, 0.62) |
| PODCI Transfer\Mobility | 82 | −0.20 (−0.40, −0.05) | −0.26 (−0.43, −0.08) | 71 | 0.0 (−0.13, 0.12) | 0.0 (−0.23, 0.23) |
| PODCI Sports & Physical Function | 82 | −0.12 (−0.28, 0.03) | −0.17 (−0.38, 0.05) | 71 | 0.12 (−0.04, 0.29) | 0.17 (−0.06, 0.41) |
| GMFM -stand | 77 | −0.08 (−0.26, 0.07) | −0.12 (−0.34, 0.11) | 68 | 0.03 (−0.13, 0.19) | 0.05 (−0.19, 0.30) |
| GMFM-run, jump, stand | 77 | −0.13 (−0.28, −0.003) | −0.24 (−0.51, −0.01) | 68 | − 0.02 (−0.11, 0.07) | −0.04 (−0.28, 0.20) |
| TUG | 72 | −0.29 (−1.29, −0.06) | −0.25 (−0.45, −0.12) | 64 | −0.18 (−0.62, −0.03) | −0.29 (−0.50, −0.08) |
CI=confidence interval; N=number of subjects; Bold = significant ES/SRM. Positive ES/SRM values indicate improved outcome (better mobility); Negative ES/SRM values indicate worse outcome (decrease in mobility).
Table 5.
Standard Response Mean (SRM) and Effect Size (ES) Estimates for Pain Interference and Pain Measures.
| Instrument | Baseline to 6 Month | Baseline to 12 month | ||||
|---|---|---|---|---|---|---|
| N | ES (CI) | SRM (CI) | N | ES (CI) | SRM (CI) | |
| PROMIS* CAT | 89 | 0.08 (−0.17,0.31) | 0.07 (−0.15,0.27) | 78 | −0.03 (−0.32,0.27) | −0.03 (−0.25, 0.21) |
| PROMIS* SF | 93 | 0.01 (−0.22, 0.25) | 0.01 (−0.20, 0.22) | 79 | −0.03 (−0.30, 0.24) | −0.02 (−0.24, 0.20) |
| PedsQL (CP Module)** | 89 | 0.18 (−0.003,0.36) | 0.20 (−0.003,0.41) | 74 | 0.29 (0.10,0.50) | 0.34 (0.11,0.61) |
| PODCI Pain Comfort* | 92 | 0.04 (−0.19, 0.25) | 0.04 (−0.16,0.25) | 80 | 0.01 (−0.23,0.22) | 0.01 (−0.21,0.22) |
CI=confidence interval
Measures pain interference
Measures pain
N=number of subjects; Bold = significant ES/SRM. Positive ES/SRM values indicate improved outcome (less pain); Negative ES/SRM values indicate worse outcome (more pain)
Table 6.
Standard Response Mean (SRM) and Effect Size (ES) Estimates for Fatigue Measures.
| Instrument | Baseline to 6 Month | Baseline to 12 month | ||||
|---|---|---|---|---|---|---|
| N | ES (CI) | SRM (CI) | N | Effect size (CI) | SRM (CI) | |
| PROMIS CAT | 88 | 0.15 (−0.05, 0.38) | 0.15 (−0.05, 0.37) | 76 | 0.18 (−0.03, 0.43) | 0.19 (−0.03, 0.42) |
| PROMIS SF | 93 | −0.02 (−0.21, 0.17) | −0.02 (−0.23, 0.18) | 79 | 0.17 (−0.03, 0.39) | 0.18 (−0.03, 0.40) |
| PedsQL (CP Module) | 89 | 0.24 (0.02, 0.45) | 0.22 (0.02, 0.44) | 74 | 0.36 (0.18, 0.55) | 0.41 (0.20, 0.64) |
CI=confidence interval; N=number of subjects; Bold = significant ES/SRM; Positive ES/SRM values indicate improved outcome (less fatigue); Negative ES/SRM values indicate worse outcome (more fatigue).
The PROMIS® Pain Interference (Table 5) and Fatigue (Table 6) CAT and SF did not detect change at either the 6- or 12-month intervals. The PedsQL CP Pain\Hurt showed a decline for the 12-month interval. The PODCI Pain\Comfort did not detect change at either the 6- or 12-month intervals. The PedsQL CP Fatigue detected small improvement at 6 months and moderate improvement at 12 months.
Figure 1 displays the SRM estimates and CI for the PROMIS® CATs and SF and the PedsQL CP. As shown, except for pain at 6 months, the SRM estimates for the PedsQL CP were significant. The SRM estimates for the PROMIS® instruments were not significant at any testing interval.
Figure 1. Standard Response Mean (SRM) Estimates and Confidence Intervals (CI) for the PROMIS® CATs and SF and the PedsQL CP.
The PedsQL CP SRM was significant for each domain at each testing interval except SRM value at 6- months for pain that approached significance. The SRM of PROMIS instruments were not significant at any testing interval. N=Number; mo=months.
Content Range
Each measure showed minimal floor effects. Ceiling effects were as follows: PedsQL CP ceiling effects ranged from 17.6% (Fatigue at baseline) to 33.7% (Pain\Hurt at 6 months); PODCI ceiling effects ranged from 0.0% (Sports\Physical Function at baseline, 6 & 12 months) to 38.0% (Pain\Comfort at 6 months); PROMIS® CAT measures ranged from 2.1% (Mobility at baseline) to 24.0 % (Pain at 6 months); PROMIS® SF measures ranged from 7.6% (Mobility at 6 months) to 30.0% (Pain at 6 months); GMFM ranged from 2.1% (GMFM E at baseline) to 8.8% (GMFM D at 6 months).
Item Exposure
Table 7 presents a summary of the PROMIS® Pediatric Mobility item exposure rates at baseline demonstrating that items at the lower end (less difficult items) of the mobility scale were not administered. All 13 PROMIS® Pediatric Pain Interference items were administered. Of the 23 PROMIS® Pediatric Fatigue items, 6 items were not administered by the CAT nor included in the SF. Most items not administered were at the upper end of the scale (more fatigue).
Table 7.
Supporting Material.
| Sample N=94 | Average Item Difficulty | Slope | Items, arranged from high to low difficulty | Format* |
|---|---|---|---|---|
| 31.9% | −1.01 | 1.13 | I could run a mile | CAT Only |
| 100% | −1.31 | 3.11 | **I could do sports and other exercise that kids my age could do | Both CAT/SF |
| 87.2% | −1.54 | 2.36 | I have been physically able to do the activities I enjoy most | Both CAT/SF |
| 24.5% | −1.79 | 1.67 | I could ride a bike | CAT Only |
| 60.6% | −1.89 | 1.96 | I could keep up when I played with other kids | Both CAT/SF |
| 31.9% | −2.00 | 1.81 | I could walk more than one block | CAT Only |
| 23.4% | −2.11 | 1.97 | I could walk up stairs without holding on to anything | Both CAT/SF |
| 37.2% | −2.13 | 1.82 | I could stand on my tiptoes | Both CAT/SF |
| 43.6% | −2.17 | 3.00 | I could stand up by myself | Both CAT/SF |
| 100% | −2.19 | 2.62 | I could get up from the floor | Both CAT/SF |
| 16.0% | −2.25 | 2.80 | I could walk across the room | CAT Only |
| 70.2% | −2.26 | 3.27 | I could move my legs | Both CAT/SF |
| 19.1% | −2.34 | 1.76 | I could carry my books in a backpack | CAT Only |
| 18.1% | −2.35 | 1.92 | I could get down on my knees without holding on to something | CAT Only |
| - | −2.49 | 2.24 | I could get in and out of a car | Not Administered |
| - | −2.55 | 2.28 | I could get into bed by myself | Not Administered |
| 11.7% | −2.70 | 2.01 | I could bend over to pick something up | CAT Only |
| - | −2.80 | 1.82 | I used a wheelchair to get around | Not Administered |
| - | −3.07 | 1.67 | I used a walker, cane, or crutches to get around | Not Administered |
| - | −3.33 | 1.84 | I could go up one step | Not Administered |
| - | −3.50 | 1.41 | I could get up from a regular toilet | Not Administered |
| - | −3.50 | 1.16 | I could turn my head all the way to the side | Not Administered |
| 3.2% | −3.88 | 0.88 | I could get out of bed by myself | CAT Only |
| −2.12 | 2.13 | Average for items administered | ||
| −3.03 | 1.77 | Average for items not administered | ||
Exposure Rates of Items in PROMIS Mobility Item Bank. Shaded areas indicate items that were not made available by the CAT and that are not on the SF
DISCUSSION
The finding that the PROMIS® CATs did not detect change is an extension of our previous work that showed that the PROMIS® mobility scales did not discriminate across GMFCS levels 20 and, is a further indication that the standard PROMIS® administration and scoring are problematic for youth with CP. While the PROMIS® Mobility CAT is designed to administer items based on a person's ability, seven items at the lower end of the scale, including items that would be most appropriate for this sample of children with CP (e.g., with appropriate content like walking with crutches), were not administered and were not included in the SF. This likely occurred because the PROMIS® item banks were calibrated on children without CP.21-23 Item exposure was also impacted by the standard PROMIS® CAT stopping rules as the administration may have discontinued before a proper estimate of the individual's true functioning was reached.
In addition to the functionality of the CAT, there was also a problem with the interpretation of some PROMIS® Mobility items, as response patterns did not match the abilities of the sample. For example, at baseline 31.9% of children were asked the item “I could run a mile,” which is the most difficult item in the mobility item bank. In response to this item, 87% responded that they could run a mile “with no trouble” or “with a little trouble.” Cognitive testing, a formal methodology used to evaluate how respondents interpret and formulate responses to test questions24 of PROMIS items was not conducted with children with CP and may have provided insight on how children with CP experience and interpret such questions and if they differ from that of children without CP.
Change detected by the PedsQL CP BM (child-report), the PODCI mobility subscales (parent-report) and GMFM and TUG (performance measures) varied. The GMFM D and the PODCI sports/physical functioning did not detect change. The PedsQL CP MB detected improvement at each testing interval while the PODCI Transfer\Basic Mobility and GMFM E detected slight decline in function at 6 months and no change from baseline at 12 months. The different results (improvement versus decline in function) among the instruments can be due to several factors. First, while all measures assess mobility, they contain items that focus on different dimensions of mobility. For example, the PedsQL CP BM includes items that assess very basic mobility (e.g., “moving a leg”; “leg stiffness”) while the PODCI Transfer\Basic Mobility, GMFM E and TUG assess more complex motor skills (e.g., “walking one block,” “running,” “walking speed”). The most common surgical procedures were tendon releases and lengthenings, both of which decrease spasticity and stiffness,25 which potentially contributed to the positive effect of surgery detected by items included in the PedsQL CP. However, these same procedures may also have adverse effects on mobility activities such as walking, running, or jumping.7,26 Others have reported similar findings, noting variability in direction of change among measures as well as among subscales within the same measure in children with CP.7,18,26
Another explanation for the difference in direction of change may be due to child-reported (PedsQL CP) and parent-reported (PODCI) measures being administered.27 The direct comparison between two child-reported instruments (PROMIS® and PedsQL CP) as done in this study, helps to evaluate alike instruments without the cofounding factor of proxy report.
Pain in CP is associated with muscle\joint stiffness, spasticity, hip instability, and stretching exercises.28,29 The PedsQL CP Pain\Comfort includes four items directly related to known causes of pain in CP (I ache and hurt in my joints and/or muscles; I hurt a lot; I have trouble sleeping because of pain or aching in my joints and/or muscles; My muscles get stiff or sore). In contrast, PROMIS® Pediatric Pain Interference includes an activity component (It was hard for me to run when I had pain). The PROMIS® scale may have difficulty detecting the impact of pain on function if activities within the scale have little relevance to youth with CP30 or if items that evaluate pain directly associated with CP are lacking.31,32
The PROMIS® portfolio assumes universality for common outcomes across diseases and conditions.33 This assumption implies that generic measures (non-disease or non-condition specific) such as those in the PROMIS® portfolio should be applied successfully across any clinical population, including youth with CP; this assumption was not supported by this study. However, there may be ways to increase the applicability of the PROMIS® measures in children with CP. The first item in a CAT should discriminate among children with different abilities, which does not occur.20 Changes to the standard stopping rules may allow the CATs to administer items with content that is more relevant to children with CP. PROMIS® SFs could be tailored to include a selection of items thought to be more relevant to the CP population. Existing PROMIS® item banks could be replenished and recalibrated with items of particular relevance to CP and\or, methodology to link PROMIS® scores to scores from disease-specific item banks34 can be developed.35,36
There are limitations to this study. Maximal functional return can occur up to 24 months after surgery and, although they did not detect early change (<24 months), the PROMIS instruments may detect late change. Intentionally, we did not power the study for stratification by GMFCS levels due to the assumption of universality. This study was not designed to evaluate the effectiveness of surgery. Effectiveness studies of musculoskeletal surgery for CP are needed and should be designed using appropriate methods and with careful selection of outcomes measures based on surgical procedures and the intended outcomes.
CONCLUSION
The PROMIS® Pediatric CATs may not select and the SFs may not contain appropriate items for youth with CP. In their current form, the PROMIS® Pediatric Mobility, Pain and Fatigue instruments are not recommended for evaluating the outcomes of orthopedic surgery in children with CP.
Acknowledgments
Sources of Support: The study was funded through the Patient Reported Outcomes Measurement Information System (PROMIS) (National Institutes of Health Grant #U01AR057929) and The Shriners Hospitals for Children (Grant #79120).
References
- 1.Cella D, Riley W, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. Journal of Clinical Epidemiology. 2010;63(11):1179–1194. doi: 10.1016/j.jclinepi.2010.04.011. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Bode R, Lai J, et al. Issues in the development of an item bank. Archives of Physical Medicine and Rehabilitation. 2003;84:S52–S60. doi: 10.1053/apmr.2003.50247. [DOI] [PubMed] [Google Scholar]
- 3.Lai J-S, Cella D, Chang C-H, Bode RK, Heinemann AW. Item banking to improve, shorten and computerize self-reported fatigue: an illustration of steps to create a core item bank from the FACIT-Fatigue Scale. Qual Life Res. 2003;12:485–501. doi: 10.1023/a:1025014509626. [DOI] [PubMed] [Google Scholar]
- 4.Reeve BB. An Introduction to Modern Measurement Theory. 2001 NIH/NCI Report. [Google Scholar]
- 5.PROMIS Instrument Customization: CAT Parameters. [17 October 2014];Instrument: PROMIS Ped Bank v1.0-Mobility. 2014 from https://www.assessmentcenter.net.
- 6.Firth GP, Sangeux E,M, et al. Multilevel surgery for equinus gait in children with spastic diplegic cerebral palsy: medium-term follow-up with gait analysis. Journal of Bone and Joint Surgery. 2013;95(10):931–938. doi: 10.2106/JBJS.K.01542. [DOI] [PubMed] [Google Scholar]
- 7.Seniorou M, Thompson N, et al. Recovery of muscle strength following multi-level orthopaedic surgery in diplegic cerebral palsy. Gait & Posture. 2007;26(4):475–481. doi: 10.1016/j.gaitpost.2007.07.008. [DOI] [PubMed] [Google Scholar]
- 8.Rutz E, Gaston M, et al. Hip flexion deformity improves without psoas-lengthening after surgical correction of fixed knee flexion deformity in spastic diplegia. Hip International. 2012;22(4):379–386. doi: 10.5301/HIP.2012.9453. [DOI] [PubMed] [Google Scholar]
- 9.Novak I, McIntyre S, Morgan C, Campbell L, Dark L, Morton N, Stumbles E, Wilson SA, Goldsmith S. A systematic review of interventions for children with cerebral palsy: state of evidence. Dev Med Child Neurol. 2013;55(10):885–910. doi: 10.1111/dmcn.12246. [DOI] [PubMed] [Google Scholar]
- 10.Palisano R, Rosenbaum P, Bartlett D, Livingston M. GMFCS - E&R: Gross Motor Function Classification System Expanded and Revised. 2007 doi: 10.1111/j.1469-8749.2008.03089.x. [DOI] [PubMed] [Google Scholar]
- 11.Heimkes B, Martignoni K, et al. Soft tissue release of the spastic hip by psoas-rectus transfer and adductor tenotomy for long-term functional improvement and prevention of hip dislocation. Journal of Pediatric Orthopaedics. 2011;20(4):212–221. doi: 10.1097/BPB.0b013e328344c529. [DOI] [PubMed] [Google Scholar]
- 12.Varni J, Burwinkle T, et al. Health related quality of life of children and adolescents with CP: hearing the voices of children. Developmental Medicine & Child Neurology. 2005;47(9):592–597. [PubMed] [Google Scholar]
- 13.Daltroy L, Liang M, et al. The POSNA pediatric musculoskeletal functional health questionnaire: report on reliability, validity, and sensitivity to change. Pediatric Outcomes Instrument Developmental Group. Pediatric Orthopaedic Society of North America. Journal of Pediatric Orthopaedics. 1998;18(5):561–571. doi: 10.1097/00004694-199809000-00001. [DOI] [PubMed] [Google Scholar]
- 14.Podsiadlo D, Richardson S. The timed “Up & Go”: a test of basic functional mobility for frail elderly persons. Journal of the American Geriatrics Society. 1991;39(2):142–148. doi: 10.1111/j.1532-5415.1991.tb01616.x. [DOI] [PubMed] [Google Scholar]
- 15.Russell DJ, Rosenbaum PL, et al. Gross Motor Function Measure (GMFM-66 & GMFM-88) User's Manual. Mac Keith Press; London: 2002. [Google Scholar]
- 16.Varni J, Burwinkle T, et al. The PedsQL in pediatric cerebral palsy: reliability, validity, and sensitivity of the Generic Core Scales and Cerebral Palsy Module. Developmental Medicine & Child Neurology. 2006;48(6):442–449. doi: 10.1017/S001216220600096X. [DOI] [PubMed] [Google Scholar]
- 17.Barnes D, Linton J, et al. Pediatric outcomes data collection instrument scores in ambulatory children with cerebral palsy: an analysis by age groups and severity level. Journal of Pediatric Orthopaedics. 2008;28(1):97–102. doi: 10.1097/bpo.0b013e31815a60e6. [DOI] [PubMed] [Google Scholar]
- 18.Cheng H, Ju Y, et al. Managing spastic hypertonia in children with cerebral palsy via repetitive passive knee movements. Journal of Rehabilitation Medicine. 2012;44(3):235–240. doi: 10.2340/16501977-0937. [DOI] [PubMed] [Google Scholar]
- 19.Nicolini-Panisson R, Donadio M. Normative values for the Timed ‘Up and Go’ test in children and adolescents and validation for individuals with Down syndrome. Developmental Medicine & Child Neurology. 2014;56(5):490–497. doi: 10.1111/dmcn.12290. [DOI] [PubMed] [Google Scholar]
- 20.Kratz A, Slavin M, et al. An examination of the PROMIS pediatric instruments to assess mobility in children with cerebral palsy. Quality of Life Research. 2013;22(10):2865–2876. doi: 10.1007/s11136-013-0397-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Irwin D, Stucky B, et al. Sampling plan and patient characteristics of the PROMIS pediatrics large-scale survey. Quality of Life Research. 2010;19(4):585–594. doi: 10.1007/s11136-010-9618-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Dewitt E, Stucky B, et al. Construction of the eight-item patient-reported outcomes measurement information system pediatric physical function scales: built using item response theory. Journal of Clinical Epidemiology. 2011;64(7):794–804. doi: 10.1016/j.jclinepi.2010.10.012. (35) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Lai J, Stucky B, et al. Development and psychometric properties of the PROMIS pediatric fatigue item banks. Quality of Life Research. 2013;22:2417–2427. doi: 10.1007/s11136-013-0357-1. (36) [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Willis GB. Cognitive Testing. A Tool for Improving Questionnaire Design. Sage Publications; London: 2005. [Google Scholar]
- 25.Viachou M, Pierce R, Davis RM, Sussman M. Does tendon lengthening surgery affect muscle tone in children with cerebral palsy? Acta Orthop Belg. 2009 Dec;75(6):808–14. [PubMed] [Google Scholar]
- 26.Oeffinger D, Gorton G, et al. Variability explained by strength, body composition and gait impairment in activity and participation measures for children with cerebral palsy: a multicentre study. Clinical Rehabilitation. 2014;28(10):1053–1063. doi: 10.1177/0269215513511343. [DOI] [PubMed] [Google Scholar]
- 27.Baca C, Vickrey B, et al. Differences in child versus parent reports of the child's health-related quality of life in children with epilepsy and healthy siblings. Value Health. 2010;13(6):778–786. doi: 10.1111/j.1524-4733.2010.00732.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Ramstad K, Jahnsen R, et al. Mental health, health related quality of life and recurrent musculoskeletal pain in children with cerebral palsy 8-18 years old. Disability and Rehabilitation. 2012;34(19):1589–1595. doi: 10.3109/09638288.2012.656794. [DOI] [PubMed] [Google Scholar]
- 29.Penner M, Xie W, et al. Characteristics of pain in children with cerebral palsy. Pediatrics. 2013;132(2):e407–e413. doi: 10.1542/peds.2013-0224. [DOI] [PubMed] [Google Scholar]
- 30.Tyler E, Jensen M, et al. The reliability and validity of pain interference measures in persons with cerebral palsy. Archives of Physical Medicine and Rehabilitation. 2002;83(2):236–239. doi: 10.1053/apmr.2002.27466. [DOI] [PubMed] [Google Scholar]
- 31.Geister T, Quintanar-Solares M, et al. Qualitative development of the 'Questionnaire on Pain caused by Spasticity (QPS), a pediatric patient-reported outcome for spasticity-related pain in cerebral palsy. Quality of Life Research. 2014;23:887–896. doi: 10.1007/s11136-013-0526-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Harvey A, Robin J, et al. A systematic review of measures of activity limitation for children with cerebral palsy. Developmental Medicine & Child Neurology. 2008;50:190–198. doi: 10.1111/j.1469-8749.2008.02027.x. [DOI] [PubMed] [Google Scholar]
- 33.Cook K, Kallen M, et al. The Patient Reported Outcomes Measurement Information System (PROMIS) Perspective on: Universally-Relevant vs. Disease-Attributed Scales. PROMIS Statistical Center Working Group. 2014 [Google Scholar]
- 34.Haley S, Chafetz R, Mulcahey MJ. Validity and Reliability of Physical Functioning Computer-adaptive Tests for Children with Cerebral Palsy. Journal of Pediatric Orthopaedics. 2010;30(1):71–75. doi: 10.1097/BPO.0b013e3181c85453. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Haley SM, Ni P, Lai JS, Tian F, Coster WJ, Jette AM, Straub D, Cella D. Arch Phys Med Rehabil. 10 Suppl. Vol. 92. S37-43: Oct, 2011. Linking the activity measure for post acute care and the quality of life outcomes in neurological disorders. doi: 10.1016/j.apmr.2011.01.026. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Tian F, Ni P, Mulcahey MJ, Hambleton RK, Tulsky D, Haley SM, Jette AM. Tracking functional status across the spinal cord injury lifespan: linking pediatric and adult patient-reported outcome scores. Arch Phys Med Rehabil. 2014 Nov;95(11):2078–2085.e15. doi: 10.1016/j.apmr.2014.05.023. doi: 10.1016/j.apmr.2014.05.023. Epub 2014 Jun 13. [DOI] [PubMed] [Google Scholar]