Abstract
Background
Patient-reported outcome (PRO) measurements linked to Common Terminology Criteria for Adverse Events (CTCAE) grading may improve symptom adverse event (AE) reporting in pediatric oncology trials. We evaluated construct validity, responsiveness, and test-retest reliability of the Ped-PRO–CTCAE measurement system for children and adolescents undergoing cancer care.
Methods
A total of 482 children and adolescents (7–18 years, 41.5% not non-Hispanic white) newly diagnosed with cancer and their caregivers participated from nine pediatric oncology hospitals. Surveys were completed at 72 hours preceding treatment initiation (T1) and at follow-up (T2) approximately 7–17 days later for chemotherapy, and 4+ weeks for radiation. Psychometric analyses examined the relationship of Ped-PRO-CTCAE items (assessing 62 symptom AEs) with Patient-Reported Outcomes Measurement Information System, Memorial Symptom Assessment Scale, Lansky Play-Performance Scale, and medication use. A separate test-retest study included 46 children.
Results
Ped-PRO-CTCAE and Memorial Symptom Assessment Scale were strongly correlated across age groups at T2: 7–12 years (r = 0.62–0.80), 13–15 years (r = 0.44–0.94), and 16–18 years (r = 0.65–0.98); and over time. The Ped-PRO-CTCAE was strongly correlated with Patient-Reported Outcomes Measurement Information System Pediatric measures at T2; for example, pain interference (r = 0.70, 95% confidence interval [CI] = 0.64 to 0.76), fatigue severity (r = 0.63, 95% CI = 0.56 to 0.69), and depression severity (r = 0.76, 95% CI = 0.71 to 0.81). Ped-PRO-CTCAE items differentiated children by Lansky Play-Performance Scale and by medication use. Test-retest agreement ranged from 54.3% to 93.5%.
Conclusions
This longitudinal study provided evidence for the construct validity and reliability of the core Ped-PRO-CTCAE symptom AE items relative to several established measures. Additional responsiveness data with clinical anchors are recommended. Incorporation of Ped-PRO-CTCAE in trials may lead to a better understanding of the cancer treatment experience.
Pediatric oncology trials collect adverse event (AE) data to ensure patient safety and inform sponsors, regulators, patients, caregivers, and clinicians about treatment effects. Clinicians grade AEs using the National Cancer Institute’s (NCI’s) Common Terminology Criteria for Adverse Events (CTCAE) (1,2), although symptom AEs (eg, nausea, anxiety) are subjective in nature and difficult to capture, especially in children. Prior research has found that compared with the child’s self-report, clinicians tend to underreport symptoms and downgrade their severity (3–6). Understanding a child’s ability to tolerate a given treatment, therefore, requires a measurement tool that can validly capture the child’s voice.
As previously reported, we designed the Pediatric Patient-Reported Outcomes (PRO) version of the CTCAE to directly integrate the child’s and adolescent’s voice into AE grading in pediatric oncology trials (7,8). The measurement tool is an extension of the NCI’s PRO-CTCAE, which is designed for adults participating in oncology trials (9,10). The Ped-PRO-CTCAE measurement system includes a library of 130 items (ie, questions and response options) that can assess up to 62 symptom AEs. This includes a core set of 15 symptom AEs that are prevalent across a broad range of pediatric cancers and treatments (eg, pain, fatigue) and 47 symptom AEs with lower prevalence (eg, dyspnea, pruritus).
To develop the Ped-PRO-CTCAE, 187 pediatric oncology clinicians reviewed the entire CTCAE and identified symptom AEs to be included in the Ped-PRO-CTCAE measurement system (7). Subsequently, 132 children and adolescents undergoing cancer treatment participated in a qualitative study including concept elicitation and cognitive testing (8,11). Concept elicitation allowed children to discuss experiences with disease- and treatment-related symptoms, informing content of the Ped-PRO-CTCAE (12). Cognitive testing confirmed children could understand the questions and provide valid responses (11). Together, this qualitative work supported content validity of the Ped-PRO-CTCAE measurement system.
The goal of the current study is to evaluate construct validity, responsiveness, and test-retest reliability of the Ped-PRO-CTCAE measurement system using longitudinal data collected from children and adolescents (7–18 years) undergoing cancer treatment. Because there is no gold standard for symptom assessment in pediatric oncology, we compared the Ped-PRO-CTCAE against multiple established patient-reported and parent-reported instruments to evaluate the validity of the Ped-PRO-CTCAE in narrow age ranges.
Methods
Longitudinal Study
Participants
Children and adolescents participated from nine pediatric oncology hospitals: Children’s Healthcare of Atlanta, Children’s Hospital Los Angeles, UPMC Children’s Hospital of Pittsburgh, Children’s National Hospital (Washington, DC), Dana-Farber Cancer Institute/Boston Children’s Hospital, Duke University (Durham, NC), the Hospital for Sick Children (Toronto, Ontario, Canada), St. Jude Children’s Research Hospital (Memphis, TN), and the University of North Carolina at Chapel Hill. Sites provided access to a diverse group of children in respect to cancer type, treatment modality, race, ethnicity, and geographic location.
Eligibility Criteria
Children and adolescents between 7 and 18 years of age with a first diagnosis of cancer and receiving frontline therapy that included chemotherapy or radiation as part of standard care were eligible. Patients must have completed 1 month of frontline treatment, be at least 3–6 weeks from any surgery, and be able to read (or listen to) and understand English with no clinically significant cognitive or memory impairment based on the judgment of the site investigator. Finally, the child’s caregiver must have agreed to participate.
All sites obtained institutional review board approval. All caregivers provided written informed consent, and children and adolescents provided assent. Participants received a $10 gift card at each time point.
Study Design
We did not administer all 62 Ped-PRO-CTCAE symptom AE-related items to participants because of potential response burden. Instead, every participant responded to the 15 “core” AEs and was randomly assigned to one of four forms containing a subset of the additional 47 less prevalent AEs. To ensure validity across ages, we divided our sample into three age groups (7–12, 13–15, and 16–18 years) and planned to enroll 160 patients per group.
Enrolled participants completed questionnaires via tablet computers during clinic visits on two occasions. The interval between the first (T1) and second time point (T2) varied based on treatment considerations. The goal was to capture symptom burden and functional impact at points along the care continuum that represented one point when the child was experiencing relatively low symptom burden and higher functional performance (T1) and another point when the child was likely experiencing relatively high symptom burden and functional limitations (T2). Generally, T1 was collected within 72 hours of beginning therapy. For patients receiving chemotherapy, T2 was collected 7–17 days later, and for patients receiving radiation therapy, T2 was collected approximately 4 weeks later.
The caregiver completed surveys on the child’s functioning and medication use on the same day as the child. A study team member helped ensure that the child and caregiver completed responses independently. If the child and caregiver were not in the clinic for their T2 assessment, survey links were sent to the caregiver requesting that the caregiver and child separately complete their surveys.
Measures
Ped-PRO-CTCAE
Questions assess symptom AE attributes of symptom presence, frequency, severity, and/or interference with daily activities. Each question uses the recall period “the past 7 days” and provides 4 response categories that match the symptom attribute. Conditional questions are used if a child reports experiencing the symptom. If a child reports not having the symptom, conditional questions are scored 0. More information about the measure is provided via a website (13) and in publications (8). Access to the Ped-PRO-CTCAE questions and additional information is available by the NCI website (https://healthcaredelivery.cancer.gov/pro-ctcae/).
Memorial Symptom Assessment Scale (MSAS 7–12; MSAS 10–18)
The MSAS 7–12, completed by children aged 7–12 years, assesses eight symptoms with eight questions plus conditional questions for reported symptoms (3). The MSAS 10–18, completed by participants aged 13–18 years, assesses 31 symptoms with 31 questions plus additional conditional questions for reported symptoms (14). For this study, both versions use a recall period of “the past week” and query symptom frequency, severity, and/or bother or distress.
Patient-Reported Outcomes Measurement Information System (PROMIS)
The PROMIS Pediatric measures include computerized-adaptive testing measures of physical function-mobility, pain interference, fatigue, depressive symptoms, and anxiety (15–19). The computerized-adaptive testing was set so that a child would answer five to six questions per domain. The recall period is “the past 7 days.”
Lansky Play-Performance Scale (PPS)
This performance status tool was developed to be completed by caregivers on an 11-point scale (20). For this study, we presented ranges from 10 (“No play, does not get out of bed”) to 100 (“Fully active, normal”). Performance status was categorized as “moderate to severe restriction” (PPS 10–40), “mild to moderate restriction” (PPS 50–70), and “able to carry on normal activity” (PPS 80–100).
Medication Use
Caregivers completed 10 questions about medications their child had taken in “the past 7 days” for nausea, insomnia, constipation, diarrhea, mucositis, neuropathy, headache, depression, anxiety, and pain.
Test-Retest Study in Independent Sample
Test-retest reliability indicates the reproducibility of a measure and its ability to provide consistent scores over time in a stable population (21,22). Because our longitudinal study included children whose symptoms would vary over time, we selected an independent patient group and treatment time period where clinical stability was expected.
Participants
Participants were between ages 7 and 17 years, receiving acute lymphoblastic leukemia (ALL) treatment, in the maintenance phase of therapy (weeks 50–126+), and patients at Children’s National Hospital, Duke, St. Jude Children’s Research Hospital, and University of North Carolina at Chapel Hill. Otherwise, inclusion criteria were the same as the longitudinal study. Participating sites obtained institutional review board approval. All caregivers provided written informed consent, and children or adolescents provided assent.
Study Design
Our goal was to have 50 participants complete all 62 Pediatric PRO-CTCAE symptom AE questions at two time points (5–9 days apart) when treatment did not include a steroid pulse. After completion of in-clinic or online surveys, the child and caregiver received a $20 gift card each.
Statistical Analyses
Psychometric analyses (using SAS, version 9.4) were performed at the individual-item level for each symptom attribute, consistent with the recommended scoring approach of Ped-PRO-CTCAE to directly reflect the child’s responses (11). For all cross-sectional analyses, we present results for only T2 because this was when the child was expected to be more symptomatic. Although not presented, we also examined results at T1 to make sure findings were consistent. We provide results by age group (7–12, 13–15, 16–18 years) when possible.
Convergent and discriminant validity of the Ped-PRO–CTCAE was examined by its association with the MSAS (using polychoric correlations for ordinal variables) and with PROMIS Pediatric measures (using polyserial correlations for ordinal and interval variables). If statistically different from zero, magnitudes of correlation estimates are classified as small (0.10–0.29), moderate (0.30–0.49), strong (0.50–0.69), and very strong (>0.70), consistent with Cohen’s recommended effect sizes for correlations (23,24).
Known groups validity was examined for PPS and medication use. Means and 95% confidence intervals (CIs) are provided. The groups were considered statistically different (P < .05) when confidence intervals did not overlap. Analysis of variance and t tests (two-tailed tests) were used to test for differences in mean scores for the PPS and medication use with P less than .05.
Responsiveness was examined by looking at correlations of change scores between Ped-PRO-CTCAE and MSAS items. Correlations of change scores were expected to be smaller than in cross-sectional data when participants’ symptom levels did not change from T1 to T2; thus, there is a restriction of range of change scores (25).
Test-retest reliability was evaluated by both intraclass correlations (ICCs) and percent agreement between T1 and T2 (26). For ICCs, we used a two-way mixed effect model (27). ICCs are more commonly reported but may not be an accurate reflection of stability when there is a restriction of range in the data (28). The issue of restricted range may be pertinent for the test-retest data collected on the Ped-PRO-CTCAE, because children in their maintenance phase of ALL treatment may have few symptoms. Thus, we provide percent agreement alongside ICCs, acknowledging that percent agreement does not adjust for chance (29). Agreement levels were categorized as poor (<.20), fair (.20–.40), moderate (.40–.60), good (.60–.80), or very good (>.80) (30).
Results
Patient Characteristics
A total of 580 child-caregiver dyads were approached for study participation, 88 declined, and 10 withdrew before completing the T1 survey, for a response rate of 83.1%. Table 1 provides characteristics of the 482 participants by age group. The sample included a diverse balance of sex (46.3% female), race or ethnicity (41.5% not non-Hispanic white), cancer types, and treatment modalities. Although we did not achieve equal numbers of 160 children in each group, we did attain a larger overall sample size with oversampling in the 7- to 12-year age group. The average number of days between T1 and T2 for children receiving chemotherapy was 10 days (range = 7–17) and for children receiving radiation was 30 days (range = 26–40). A total 14% completed T2 via email links.
Table 1.
Characteristics of the participants by age group*
| Child characteristics | 7–12 y | 13–15 y | 16–18 y | Total |
|---|---|---|---|---|
| (n = 203) | (n = 144) | (n = 135) | (n = 482) | |
| Age, mean (SD), y | 9.5 (1.7) | 14.2 (0.8) | 16.9 (0.8) | 12.9 (3.4) |
| Sex, female, no. (%) | 97 (48.3) | 67 (47.2) | 56 (42.4) | 220 (46.3) |
| Race, no. (%) | ||||
| White | 141 (70.1) | 99 (70.2) | 88 (65.7) | 328 (68.9) |
| Black | 35 (17.4) | 17 (12.1) | 28 (20.9) | 80 (16.8) |
| Asian | 9 (4.5) | 8 (5.7) | 5 (3.7) | 22 (4.6) |
| Other | 16 (8.0) | 17 (12.1) | 13 (9.7) | 46 (9.7) |
| Ethnicity, Hispanic, no. (%) | 29 (14.6) | 24 (16.9) | 18 (13.5) | 71 (15.0) |
| Cancer type, no. (%) | ||||
| Leukemia or lymphoma | 107 (53.2) | 67 (46.9) | 89 (66.4) | 263 (55.0) |
| Solid tumor | 49 (24.4) | 52 (36.4) | 34 (25.4) | 135 (28.2) |
| Neuro-oncology | 41 (20.4) | 19 (13.3) | 11 (8.2) | 71 (14.9) |
| Bone marrow transplant | 4 (2.0) | 5 (3.5) | 0 (0.0) | 9 (1.9) |
| Treatment received, no. (%) | ||||
| Chemotherapy | 179 (89.1) | 127 (88.8) | 130 (97.0) | 436 (91.2) |
| Radiation | 18 (9.0) | 11 (7.7) | 4 (3.0) | 33 (6.9) |
| Bone marrow transplant | 4 (2.0) | 5 (3.5) | 0 (0.0) | 9 (1.9) |
| Caregiver-reported PPS, mean (SD) | ||||
| Assessment point T1 | 73.6 (21.2) | 69.2 (20.4) | 69.8 (23.4) | 71.2 (21.7) |
| Assessment point T2 | 68.8 (21.6) | 66.7 (20.2) | 67.0 (23.9) | 67.7 (21.8) |
Each variable had a small amount of missing data (<5%), and we assumed the data were missing at random. PPS = Lansky Play-Performance Scale.
Convergent and Discriminant Validity: Ped-PRO-CTCAE and MSAS
The left half of Table 2 provides correlations between Ped-PRO-CTCAE and MSAS questions by age group at T2. Correlations for symptoms in common between the two measures ranged for 7- to 12-year olds from 0.62 to 0.80, for 13- to 15-year olds from 0.44 to 0.94, and for 16- to 18-year olds from 0.65 to 0.98. Consistently, relatively lower correlations were observed for comparison of Ped-PRO-CTCAE interference questions and MSAS bother or distress items. We observed strong correlations between measures for symptom frequency and severity.
Table 2.
Cross-sectional and longitudinal correlations of Ped-PRO-CTCAE with MSAS 10-18 and MSAS 7–12 items by age group*
| Symptom and attribute† | Ped-PRO-CTCAE and MSAS at T2 |
Ped-PRO-CTCAE and MSAS change T1 to T2 |
||||
|---|---|---|---|---|---|---|
| 13–15 y |
16–18 y |
7–12 y‡ |
13–15 y |
16–18 y |
7–12 y‡ |
|
| Polychoric r (95% CI) | Polychoric r (95% CI) | Polychoric r (95% CI) | Pearson r (95% CI) | Pearson r (95% CI) | Pearson r (95% CI) | |
| Constipation – S | 0.74 (0.58 to 0.90) | 0.98 (0.95 to 1.00) | — | 0.45 (0.30 to 0.58) | 0.59 (0.45 to 0.70) | — |
| Constipation – I | 0.52 (0.20 to 0.84) | 0.81 (0.63 to 1.00) | — | 0.21 (0.04 to 0.37) | 0.32 (0.14 to 0.47) | — |
| Diarrhea – F | 0.92 (0.86 to 0.99) | 0.89 (0.79 to 0.98) | — | 0.71 (0.61 to 0.78) | 0.72 (0.61 to 0.80) | — |
| Diarrhea – I | 0.87 (0.72 to 1.00) | 0.73 (0.42 to 1.00) | — | 0.62 (0.50 to 0.72) | 0.53 (0.38 to 0.65) | — |
| Mucositis oral – S | 0.80 (0.68 to 0.92) | 0.79 (0.65 to 0.94) | — | 0.49 (0.35 to 0.61) | 0.73 (0.63 to 0.81) | — |
| Mucositis oral – I | 0.75 (0.57 to 0.92) | 0.90 (0.80 to 1.00) | — | 0.55 (0.42 to 0.66) | 0.79 (0.71 to 0.85) | — |
| Nausea – F | 0.85 (0.77 to 0.93) | 0.79 (0.68 to 0.90) | 0.71 (0.60 to 0.81) | 0.61 (0.48 to 0.71) | 0.63 (0.51 to 0.73) | 0.47 (0.35 to 0.58) |
| Nausea – S | 0.81 (0.72 to 0.90) | 0.82 (0.73 to 0.92) | — | 0.63 (0.51 to 0.72) | 0.70 (0.59 to 0.78) | — |
| Nausea – I | 0.73 (0.59 to 0.87) | 0.77 (0.64 to 0.91) | 0.62 (0.47 to 0.78) | 0.50 (0.36 to 0.62) | 0.55 (0.41 to 0.67) | 0.32 (0.19 to 0.45) |
| Vomit – F | 0.91 (0.85 to 0.98) | 0.94 (0.88 to 1.00) | — | 0.78 (0.70 to 0.84) | 0.73 (0.63 to 0.81) | — |
| Vomit – I | 0.81 (0.68 to 0.95) | 0.83 (0.68 to 0.97) | — | 0.63 (0.51 to 0.72) | 0.67 (0.55 to 0.76) | — |
| Fatigue – S | 0.64 (0.50 to 0.77) | 0.71 (0.59 to 0.83) | 0.71 (0.60 to 0.82) | 0.28 (0.11 to 0.43) | 0.37 (0.20 to 0.52) | 0.37 (0.23 to 0.49) |
| Fatigue – I | 0.47 (0.26 to 0.67) | 0.65 (0.50 to 0.80) | 0.62 (0.46 to 0.78) | 0.07 (–0.10 to 0.24) | 0.26 (0.08 to 0.43) | 0.36 (0.22 to 0.48) |
| Pain – F | 0.89 (0.83 to 0.95) | 0.95 (0.90 to 0.99) | 0.80 (0.72 to 0.89) | 0.67 (0.56 to 0.76) | 0.68 (0.57 to 0.77) | 0.47 (0.35 to 0.58) |
| Pain – S | 0.90 (0.84 to 0.95) | 0.93 (0.88 to 0.98) | 0.73 (0.63 to 0.83) | 0.66 (0.55 to 0.75) | 0.71 (0.60 to 0.79) | 0.37 (0.24 to 0.49) |
| Pain – I | 0.81 (0.72 to 0.91) | 0.83 (0.73 to 0.93) | 0.74 (0.63 to 0.86) | 0.63 (0.52 to 0.73) | 0.54 (0.40 to 0.66) | 0.35 (0.21 to 0.47) |
| Not hungry – F | 0.80 (0.70 to 0.90) | 0.88 (0.80 to 0.96) | 0.63 (0.49 to 0.76) | 0.39 (0.24 to 0.53) | 0.51 (0.36 to 0.64) | 0.23 (0.09 to 0.37) |
| Headache – F | 0.93 (0.86 to 0.99) | 0.92 (0.85 to 0.98) | — | 0.64 (0.52 to 0.73) | 0.60 (0.47 to 0.71) | — |
| Headache – S | 0.91 (0.85 to 0.97) | 0.89 (0.82 to 0.97) | — | 0.64 (0.52 to 0.73) | 0.63 (0.51 to 0.73) | — |
| Headache – I | 0.73 (0.56 to 0.91) | 0.78 (0.61 to 0.94) | — | 0.63 (0.51 to 0.72) | 0.56 (0.42 to 0.67) | — |
| Numbness – S | 0.91 (0.83 to 0.98) | 0.92 (0.85 to 0.98) | — | 0.63 (0.51 to 0.72) | 0.45 (0.29 to 0.59) | — |
| Numbness – I | 0.44 (0.03 to 0.86) | 0.74 (0.51 to 0.96) | — | 0.20 (0.02 to 0.36) | 0.41 (0.24 to 0.55) | — |
| Anxiety – F | 0.76 (0.64 to 0.89) | 0.85 (0.74 to 0.97) | — | 0.43 (0.28 to 0.56) | 0.31 (0.14 to 0.47) | — |
| Anxiety – S | 0.80 (0.69 to 0.92) | 0.83 (0.70 to 0.96) | — | 0.38 (0.22 to 0.52) | 0.42 (0.26 to 0.57) | — |
| Anxiety – I | 0.67 (0.47 to 0.88) | 0.82 (0.65 to 0.98) | — | 0.42 (0.26 to 0.55) | 0.31 (0.13 to 0.47) | — |
| Worry – F | 0.76 (0.63 to 0.89) | 0.84 (0.72 to 0.96) | 0.71 (0.57 to 0.84) | 0.31 (0.14 to 0.46) | 0.34 (0.17 to 0.50) | 0.27 (0.13 to 0.40) |
| Worry – S | 0.78 (0.66 to 0.91) | 0.77 (0.62 to 0.93) | 0.70 (0.57 to 0.84) | 0.27 (0.10 to 0.42) | 0.34 (0.16 to 0.49) | 0.29 (0.15 to 0.42) |
| Worry – I | 0.50 (0.23 to 0.77) | 0.83 (0.69 to 0.98) | 0.75 (0.61 to 0.90) | 0.30 (0.14 to 0.45) | 0.14 (–0.04 to 0.32) | 0.21 (0.06 to 0.35) |
| Insomnia – F | 0.88 (0.82 to 0.95) | 0.78 (0.66 to 0.90) | — | 0.53 (0.39 to 0.64) | 0.57 (0.43 to 0.68) | — |
| Insomnia – S | 0.89 (0.83 to 0.95) | 0.74 (0.60 to 0.88) | — | 0.54 (0.40 to 0.65) | 0.59 (0.45 to 0.70) | — |
| Insomnia – I | 0.62 (0.43 to 0.80) | 0.77 (0.63 to 0.91) | 0.79 (0.67 to 0.90) | 0.25 (0.07 to 0.40) | 0.38 (0.21 to 0.53) | 0.38 (0.25 to 0.50) |
| Cough – F | 0.94 (0.89 to 1.00) | 0.98 (0.95 to 1.00) | — | 0.63 (0.51 to 0.72) | 0.81 (0.73 to 0.86) | — |
| Cough – S | 0.93 (0.88 to 0.99) | 0.97 (0.93 to 1.00) | — | 0.67 (0.56 to 0.76) | 0.75 (0.66 to 0.82) | — |
| Cough – I | 0.93 (0.81 to 1.00) | 0.78 (0.43 to 1.00) | — | 0.41 (0.25 to 0.54) | 0.07 (0.11 to 0.26) | — |
| Sad – S | 0.83 (0.72 to 0.93) | 0.89 (0.80 to 0.97) | 0.73 (0.61 to 0.85) | 0.36 (0.20 to 0.50) | 0.29 (0.11 to 0.45) | 0.36 (0.23 to 0.48) |
| Sad – I | 0.71 (0.51 to 0.91) | 0.88 (0.77 to 0.99) | 0.69 (0.52 to 0.86) | 0.53 (0.39 to 0.64) | 0.27 (0.09 to 0.43) | 0.14 (−0.01 to 0.28) |
Magnitudes of correlation are classified as small (0.10–0.29), moderate (0.30–0.49), strong (0.50–0.69), and very strong (>0.70). AE = adverse event; CI = confidence interval; F = frequency; S = severity; I = interference; PRO-CTCAE = Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events; MSAS = Memorial Symptom Assessment Scale; T1 = time 1; T2 = time 2.
†For “I”, Ped-PRO-CTCAE asks about “…interference with daily activities” and MSAS asks about “bother or distress.”
‡Children between 7 and 12 years of age completed the MSAS 7–12, which only included data on a limited number of symptoms (thus empty cells for this group) compared with children between 13 and 18 years who completed the MSAS 10–18 questionnaire.
For evidence of discriminant validity, associations at T2 between Ped-PRO-CTCAE and MSAS for dissimilar symptoms (eg, Ped-PRO-CTCAE pain item with MSAS fatigue item) ranged from 0.26 to 0.73 for 7- to 12-year olds, 0.01 to 0.64 for 13- to 15-year olds, and 0.00 to 0.83 for 16- to 18-year olds (data not shown). Consistently, correlations for dissimilar symptoms were lower than correlations for the same symptoms assessed by Ped-PRO-CTCAE and MSAS, supporting discriminant validity. We observed the lowest correlations for dissimilar symptoms, such as mucositis and constipation; higher correlations were observed for similar symptoms, such as depression with anxiety and fatigue with insomnia. Similar patterns were observed for other symptom attributes.
Convergent and Discriminant Validity: Ped-PRO-CTCAE and PROMIS Pediatric
Table 3 provides correlations between Ped-PRO-CTCAE and PROMIS Pediatric measures by age group at T2. Stronger correlations were observed when symptoms matched (ie, pain interference r = 0.70, 95% CI = 0.64 to 0.76; fatigue severity r = 0.63, 95% CI = 0.56 to 0.69; depression severity r = 0.76, 95% CI = 0.71 to 0.81; and anxiety severity r = 0.68, 95% CI = 0.62 to 0.74). Dissimilar symptoms associations were in the low to moderate range.
Table 3.
Correlation of Ped-PRO-CTCAE symptom AE items with PROMIS pediatric measures at T2 by age group*
| 7–12 y | 13–15 y | 16–18 y | Total | |
|---|---|---|---|---|
| Ped-PRO-CTCAEsymptom AE item | (n = 180) |
(n = 127) |
(n = 114) |
(n = 421) |
| r (95% CI) | r (95% CI) | r (95% CI) | r (95% CI) | |
| PROMIS Pediatric Physical Functioning – Mobility† | ||||
| Neuropathy – S | –0.43 (–0.60 to –0.27) | –0.25 (–0.46 to –0.04) | –0.42 (–0.62 to –0.23) | –0.37 (–0.48 to –0.27) |
| Neuropathy – I | –0.54 (–0.75 to –0.34) | –0.29 (–0.66 to 0.07) | –0.45 (–0.74 to –0.17) | –0.46 (–0.61 to –0.30) |
| Pain – F | –0.54 (–0.66 to –0.42) | –0.48 (–0.63 to –0.32) | –0.36 (–0.55 to –0.17) | –0.47 (–0.56 to –0.39) |
| Pain – S | –0.55 (–0.66 to –0.43) | –0.48 (–0.64 to –0.33) | –0.36 (–0.55 to –0.17) | –0.48 (–0.56 to –0.39) |
| Pain – I | –0.61 (–0.73 to –0.49) | –0.49 (–0.66 to –0.31) | –0.34 (–0.55 to –0.13) | –0.50 (–0.60 to –0.41) |
| Fatigue – S | –0.54 (–0.66 to –0.43) | –0.25 (–0.43 to –0.07) | –0.55 (–0.69 to –0.40) | –0.46 (–0.55 to –0.38) |
| Fatigue – I | –0.51 (–0.64 to –0.38) | –0.43 (–0.60 to –0.26) | –0.54 (–0.69 to –0.39) | –0.50 (–0.58 to –0.41) |
| PROMIS Pediatric Pain Interference† | ||||
| Pain – F | 0.63 (0.52 to 0.73) | 0.69 (0.59 to 0.80) | 0.65 (0.53 to 0.77) | 0.64 (0.58 to 0.71) |
| Pain – S | 0.62 (0.52 to 0.72) | 0.68 (0.58 to 0.79) | 0.60 (0.47 to 0.73) | 0.63 (0.56 to 0.69) |
| Pain – I | 0.69 (0.60 to 0.78) | 0.74 (0.65 to 0.84) | 0.65 (0.52 to 0.78) | 0.70 (0.64 to 0.76) |
| Abdominal pain – S | 0.41 (0.27 to 0.54) | 0.48 (0.32 to 0.63) | 0.44 (0.27 to 0.62) | 0.43 (0.34 to 0.52) |
| Abdominal pain – I | 0.49 (0.35 to 0.62) | 0.56 (0.41 to 0.71) | 0.46 (0.27 to 0.65) | 0.50 (0.41 to 0.59) |
| Headache – F | 0.43 (0.29 to 0.56) | 0.36 (0.18 to 0.54) | 0.31 (0.10 to 0.51) | 0.37 (0.28 to 0.47) |
| Headache – S | 0.39 (0.25 to 0.54) | 0.29 (0.11 to 0.48) | 0.31 (0.10 to 0.51) | 0.34 (0.24 to 0.44) |
| Headache – I | 0.53 (0.39 to 0.66) | 0.42 (0.23 to 0.62) | 0.40 (0.19 to 0.61) | 0.46 (0.36 to 0.56) |
| Mucositis oral – F | 0.35 (0.20 to 0.50) | 0.23 (0.03 to 0.42) | 0.34 (0.14 to 0.54) | 0.30 (0.20 to 0.40) |
| Mucositis oral – S | 0.36 (0.21 to 0.51) | 0.27 (0.08 to 0.47) | 0.34 (0.14 to 0.54) | 0.32 (0.22 to 0.42) |
| Mucositis oral – I | 0.43 (0.26 to 0.61) | 0.38 (0.17 to 0.58) | 0.46 (0.25 to 0.67) | 0.41 (0.30 to 0.53) |
| PROMIS Pediatric Fatigue† | ||||
| Fatigue – S | 0.65 (0.55 to 0.75) | 0.52 (0.38 to 0.66) | 0.68 (0.57 to 0.79) | 0.63 (0.56 to 0.69) |
| Fatigue – I | 0.65 (0.55 to 0.74) | 0.66 (0.55 to 0.78) | 0.74 (0.64 to 0.84) | 0.68 (0.62 to 0.74) |
| Depression – S | 0.56 (0.45 to 0.68) | 0.49 (0.32 to 0.65) | 0.62 (0.47 to 0.76) | 0.55 (0.47 to 0.63) |
| Depression – I | 0.57 (0.44 to 0.70) | 0.45 (0.21 to 0.68) | 0.72 (0.55 to 0.89) | 0.56 (0.46 to 0.65) |
| Anxiety – F | 0.57 (0.45 to 0.68) | 0.49 (0.33 to 0.64) | 0.51 (0.34 to 0.68) | 0.53 (0.45 to 0.61) |
| Anxiety – S | 0.56 (0.45 to 0.68) | 0.46 (0.30 to 0.62) | 0.44 (0.26 to 0.63) | 0.51 (0.42 to 0.59) |
| Anxiety – I | 0.64 (0.51 to 0.76) | 0.31 (0.07 to 0.56) | 0.57 (0.37 to 0.77) | 0.54 (0.44 to 0.64) |
| PROMIS Pediatric Depressive Symptoms† | ||||
| Depression – S | 0.75 (0.67 to 0.83) | 0.80 (0.72 to 0.88) | 0.76 (0.66 to 0.86) | 0.76 (0.71 to 0.81) |
| Depression – I | 0.66 (0.55 to 0.77) | 0.62 (0.45 to 0.78) | 0.90 (0.82 to 0.99) | 0.67 (0.59 to 0.75) |
| Anxiety – F | 0.64 (0.54 to 0.74) | 0.57 (0.43 to 0.70) | 0.74 (0.63 to 0.85) | 0.64 (0.57 to 0.71) |
| Anxiety – S | 0.61 (0.51 to 0.72) | 0.57 (0.44 to 0.71) | 0.64 (0.51 to 0.78) | 0.60 (0.53 to 0.67) |
| Anxiety – I | 0.65 (0.53 to 0.77) | 0.40 (0.20 to 0.61) | 0.73 (0.58 to 0.88) | 0.59 (0.50 to 0.68) |
| Fatigue – S | 0.41 (0.27 to 0.54) | 0.36 (0.19 to 0.52) | 0.42 (0.25 to 0.58) | 0.40 (0.31 to 0.49) |
| Fatigue – I | 0.39 (0.26 to 0.53) | 0.40 (0.23 to 0.56) | 0.45 (0.29 to 0.61) | 0.42 (0.33 to 0.50) |
| PROMIS Pediatric Anxiety† | ||||
| Anxiety – F | 0.69 (0.60 to 0.78) | 0.68 (0.57 to 0.79) | 0.72 (0.61 to 0.83) | 0.69 (0.63 to 0.75) |
| Anxiety – S | 0.67 (0.58 to 0.76) | 0.73 (0.64 to 0.83) | 0.64 (0.51 to 0.77) | 0.68 (0.62 to 0.74) |
| Anxiety – I | 0.67 (0.56 to 0.78) | 0.54 (0.38 to 0.70) | 0.65 (0.49 to 0.81) | 0.62 (0.54 to 0.70) |
| Depression – S | 0.64 (0.54 to 0.74) | 0.55 (0.41 to 0.69) | 0.60 (0.47 to 0.74) | 0.61 (0.54 to 0.67) |
| Depression – I | 0.59 (0.47 to 0.71) | 0.52 (0.34 to 0.69) | 0.64 (0.47 to 0.80) | 0.58 (0.50 to 0.67) |
| Fatigue – S | 0.40 (0.26 to 0.53) | 0.23 (0.05 to 0.41) | 0.39 (0.22 to 0.56) | 0.35 (0.26 to 0.44) |
| Fatigue – I | 0.38 (0.25 to 0.52) | 0.30 (0.13 to 0.48) | 0.43 (0.26 to 0.59) | 0.37 (0.28 to 0.46) |
Magnitudes of correlation are classified as small (0.10–0.29), moderate (0.30–0.49), strong (0.50–0.69), and very strong (>0.70). Polyserial correlations were used to look at association. AE = adverse event; CI = confidence interval; F = frequency; S = severity; I = interference; Ped-PRO-CTCAE = Pediatric Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events; PROMIS = Patient-Reported Outcomes Measurement Information System; T2 = time 2.
†The PROMIS Pediatric Physical Functioning - Mobility scale is scored so that higher scores represent better functioning. The PROMIS Pediatric symptom scales are scored so that higher scores represent worse symptom experiences.
Known Groups Validity: Ped-PRO-CTCAE and Play-Performance Status
Table 4 and Figure 1 provide mean Ped-PRO-CTCAE symptom AE scores categorized by PPS at T2. Consistently, mean symptom scores worsen from “normal activity” to the poorest functioning group (“moderate to severe restriction”). Twenty-one of the 38 (55.3%) Ped-PRO-CTCAE items had a statistically significant difference between the normal activity group and either of the restricted activity groups. Thus, of the 15 core symptom AEs, 11 (73%) had at least one item with a statistical difference by performance level, except for AEs of constipation, headache, numbness, and depression.
Table 4.
Evaluation of known groups validity of Ped-PRO-CTCAE items across Lansky PPS levels at T2*
| Ped-PRO-CTCAE symptom AE item | Moderate to severe restriction (n = 62) |
Mild to moderate restriction (n = 179) |
Normal activity (n = 179) |
Analysis of variance test |
|---|---|---|---|---|
| Mean (95% CI) | Mean (95% CI) | Mean (95% CI) | P | |
| Abdominal pain – F | 0.92 (0.70 to 1.14) | 0.66 (0.55 to 0.77) | 0.45 (0.37 to 0.54) | <.01 |
| Abdominal pain – S | 0.84 (0.63 to 1.05) | 0.68 (0.57 to 0.79) | 0.42 (0.34 to 0.51) | <.01 |
| Abdominal pain – I | 0.66 (0.43 to 0.88) | 0.35 (0.25 to 0.45) | 0.18 (0.11 to 0.24) | <.01 |
| Constipation – F | 0.60 (0.39 to 0.81) | 0.43 (0.33 to 0.53) | 0.36 (0.26 to 0.45) | .06 |
| Constipation – S | 0.56 (0.35 to 0.77) | 0.38 (0.29 to 0.47) | 0.31 (0.22 to 0.40) | .03 |
| Constipation – I | 0.25 (0.09 to 0.40) | 0.11 (0.06 to 0.17) | 0.07 (0.04 to 0.11) | .01 |
| Diarrhea – F | 0.58 (0.35 to 0.82) | 0.35 (0.25 to 0.45) | 0.25 (0.17 to 0.32) | <.01 |
| Diarrhea – I | 0.20 (0.05 to 0.35) | 0.11 (0.06 to 0.16) | 0.06 (0.02 to 0.10) | .03 |
| Mucositis oral – F | 0.53 (0.32 to 0.74) | 0.49 (0.38 to 0.60) | 0.35 (0.26 to 0.45) | .10 |
| Mucositis oral – S | 0.52 (0.30 to 0.73) | 0.52 (0.40 to 0.64) | 0.33 (0.23 to 0.42) | .03 |
| Mucositis oral – I | 0.27 (0.10 to 0.44) | 0.25 (0.16 to 0.34) | 0.09 (0.05 to 0.14) | .01 |
| Nausea – F | 0.87 (0.64 to 1.10) | 0.82 (0.70 to 0.95) | 0.52 (0.42 to 0.62) | <.01 |
| Nausea – S | 0.98 (0.73 to 1.23) | 0.81 (0.69 to 0.94) | 0.49 (0.40 to 0.59) | <.01 |
| Nausea – I | 0.67 (0.45 to 0.89) | 0.50 (0.40 to 0.60) | 0.24 (0.17 to 0.32) | <.01 |
| Vomiting – F | 0.68 (0.47 to 0.89) | 0.48 (0.39 to 0.58) | 0.28 (0.21 to 0.36) | <.01 |
| Vomiting – I | 0.52 (0.29 to 0.75) | 0.23 (0.16 to 0.30) | 0.17 (0.10 to 0.24) | <.01 |
| Fatigue – S | 1.28 (1.01 to 1.55) | 1.08 (0.96 to 1.20) | 0.74 (0.64 to 0.85) | <.01 |
| Fatigue – I | 0.90 (0.64 to 1.16) | 0.70 (0.59 to 0.80) | 0.37 (0.28 to 0.46) | <.01 |
| Pain – F | 0.87 (0.63 to 1.10) | 0.75 (0.63 to 0.88) | 0.49 (0.39 to 0.58) | <.01 |
| Pain – S | 0.93 (0.67 to 1.20) | 0.83 (0.69 to 0.96) | 0.49 (0.39 to 0.58) | <.01 |
| Pain – I | 0.55 (0.32 to 0.78) | 0.53 (0.41 to 0.65) | 0.29 (0.20 to 0.37) | <.01 |
| Anorexia – F | 1.08 (0.80 to 1.37) | 0.69 (0.55 to 0.82) | 0.47 (0.37 to 0.57) | <.01 |
| Headache – F | 0.58 (0.37 to 0.79) | 0.49 (0.38 to 0.59) | 0.40 (0.31 to 0.50) | .19 |
| Headache – S | 0.55 (0.33 to 0.77) | 0.51 (0.40 to 0.62) | 0.39 (0.29 to 0.49) | .18 |
| Headache – I | 0.33 (0.14 to 0.53) | 0.24 (0.16 to 0.33) | 0.21 (0.13 to 0.28) | .34 |
| Neuropathy – S | 0.43 (0.24 to 0.63) | 0.35 (0.26 to 0.45) | 0.32 (0.24 to 0.39) | .43 |
| Neuropathy – I | 0.17 (0.03 to 0.30) | 0.09 (0.04 to 0.14) | 0.05 (0.02 to 0.09) | .07 |
| Anxiety – F | 0.51 (0.28 to 0.73) | 0.55 (0.44 to 0.66) | 0.39 (0.28 to 0.50) | .13 |
| Anxiety – S | 0.45 (0.26 to 0.64) | 0.52 (0.41 to 0.64) | 0.32 (0.23 to 0.41) | .03 |
| Anxiety – I | 0.30 (0.12 to 0.48) | 0.20 (0.12 to 0.28) | 0.14 (0.07 to 0.21) | .11 |
| Insomnia – F | 0.93 (0.70 to 1.17) | 0.79 (0.67 to 0.92) | 0.44 (0.33 to 0.54) | <.01 |
| Insomnia – S | 0.75 (0.53 to 0.97) | 0.74 (0.61 to 0.86) | 0.43 (0.32 to 0.53) | <.01 |
| Insomnia – I | 0.47 (0.30 to 0.63) | 0.34 (0.25 to 0.43) | 0.21 (0.13 to 0.28) | .01 |
| Cough – F | 0.53 (0.35 to 0.72) | 0.46 (0.35 to 0.56) | 0.29 (0.22 to 0.37) | .01 |
| Cough – S | 0.52 (0.32 to 0.72) | 0.39 (0.30 to 0.49) | 0.24 (0.17 to 0.31) | <.01 |
| Cough – I | 0.08 (0.01 to 0.16) | 0.07 (0.02 to 0.12) | 0.03 (0.01 to 0.06) | .35 |
| Depression – S | 0.58 (0.36 to 0.81) | 0.49 (0.38 to 0.61) | 0.32 (0.23 to 0.40) | .01 |
| Depression – I | 0.37 (0.18 to 0.55) | 0.15 (0.09 to 0.21) | 0.14 (0.08 to 0.20) | <.01 |
Groups were considered statistically different when CIs did not overlap between children with different performance levels. Caregivers reported the child’s PPS. PPS scores are categorized as 80–100 “able to carry on normal activity,” 50–70 “mild to moderate restriction,” and 10–40 “moderate to severe restriction.” AE = adverse event; CI = confidence interval; F = frequency; S = severity; I = interference; Ped-PRO-CTCAE = Pediatric Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events; PPS = Lansky Play-Performance Scale; T2 = time 2.
Figure 1.
Evaluation of known groups validity of Pediatric Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (Ped-PRO-CTCAE) item mean scores across Lansky Play-Performance Scale (PPS) levels at time 2. F = frequency of symptom; S = severity of symptom. Means and 95% confidence intervals are provided for each PPS level. n = 179 in the normal activity PPS level, n = 179 in the mild to moderate restriction PPS level, and n = 62 in the moderate to severe restriction PPS level.
Known Groups Validity: Ped-PRO-CTCAE and Medication Use
Table 5 and Figure 2 provide mean Ped-PRO-CTCAE symptom AE scores at T2 for children whose caregivers reported they took medication for an experienced symptom during the past 7 days. Consistently, children in the group taking medication reported worse symptom AE scores than the group not taking medication. Twenty-six (81.2%) of 32 Ped-PRO-CTCAE items were statistically significantly different between the two groups. AEs that did not have a statistically significant association included depression and diarrhea, which had the smallest sample sizes for taking medications (n = 37 and 11, respectively).
Table 5.
Evaluation of known groups validity of Ped-PRO-CTCAE item mean scores by medication use at T2*
| Medication taken for† | Ped-PRO-CTCAE symptom AE item | Did not take medication in past 7 days |
Took mediation in past 7 days |
ttest | ||
|---|---|---|---|---|---|---|
| No. | Mean (95% CI) | No. | Mean (95% CI) | P | ||
| Nausea | Nausea – F | 118 | 0.29 (0.19 to 0.39) | 299 | 0.86 (0.77 to 0.95) | <.01 |
| Nausea – S | 118 | 0.29 (0.18 to 0.40) | 298 | 0.86 (0.77 to 0.95) | <.01 | |
| Nausea – I | 118 | 0.13 (0.06 to 0.20) | 299 | 0.53 (0.45 to 0.60) | <.01 | |
| Vomiting – F | 119 | 0.13 (0.07 to 0.20) | 299 | 0.54 (0.46 to 0.61) | <.01 | |
| Vomiting – I | 119 | 0.08 (0.02 to 0.13) | 299 | 0.31 (0.24 to 0.38) | <.01 | |
| Insomnia | Insomnia – F | 328 | 0.58 (0.49 to 0.66) | 84 | 1.04 (0.84 to 1.23) | <.01 |
| Insomnia – S | 328 | 0.52 (0.44 to 0.60) | 85 | 0.95 (0.76 to 1.14) | <.01 | |
| Insomnia – I | 329 | 0.24 (0.18 to 0.30) | 84 | 0.55 (0.40 to 0.70) | <.01 | |
| Constipation | Constipation – F** | 243 | 0.20 (0.14 to 0.26) | 174 | 0.74 (0.62 to 0.85) | <.01 |
| Constipation – S | 244 | 0.15 (0.10 to 0.20) | 173 | 0.69 (0.57 to 0.81) | <.01 | |
| Constipation – I | 243 | 0.02 (0.01 to 0.04) | 174 | 0.24 (0.16 to 0.32) | <.01 | |
| Diarrhea | Diarrhea – F | 406 | 0.33 (0.27 to 0.39) | 11 | 1.00 (0.15 to 1.85) | .11 |
| Diarrhea – I | 407 | 0.09 (0.06 to 0.12) | 11 | 0.27 (0.16 to 0.71) | .37 | |
| Mucositis | Mucositis – F | 326 | 0.30 (0.24 to 0.36) | 92 | 0.97 (0.77 to 1.17) | <.01 |
| Mucositis – S | 326 | 0.29 (0.23 to 0.34) | 91 | 1.02 (0.80 to 1.24) | <.01 | |
| Mucositis – I | 327 | 0.06 (0.03 to 0.09) | 92 | 0.64 (0.45 to 0.83) | <.01 | |
| Neuropathy | Neuropathy – S | 339 | 0.29 (0.24 to 0.35) | 74 | 0.62 (0.44 to 0.80) | <.01 |
| Neuropathy – I | 338 | 0.06 (0.03 to 0.09) | 74 | 0.23 (0.10 to 0.36) | .01 | |
| Headache | Headache – F | 296 | 0.28 (0.22 to 0.33) | 117 | 0.96 (0.80 to 1.11) | <.01 |
| Headache – S | 298 | 0.26 (0.20 to 0.32) | 117 | 1.01 (0.84 to 1.17) | <.01 | |
| Headache – I | 298 | 0.12 (0.08 to 0.16) | 117 | 0.55 (0.40 to 0.70) | <.01 | |
| Depression | Depression – S | 377 | 0.44 (0.36 to 0.51) | 37 | 0.57 (0.29 to 0.85) | .36 |
| Depression – I | 376 | 0.16 (0.12 to 0.21) | 36 | 0.33 (0.12 to 0.55) | .12 | |
| Anxiety | Anxiety – F | 345 | 0.44 (0.36 to 0.51) | 69 | 0.74 (0.51 to 0.97) | .02 |
| Anxiety – S | 348 | 0.40 (0.33 to 0.47) | 69 | 0.64 (0.43 to 0.84) | .03 | |
| Anxiety – I | 348 | 0.15 (0.10 to 0.20) | 69 | 0.38 (0.18 to 0.57) | .03 | |
| Pain | Abdominal pain – F | 328 | 0.52 (0.45 to 0.59) | 90 | 0.89 (0.73 to 1.05) | <.01 |
| Abdominal pain – S | 328 | 0.50 (0.43 to 0.57) | 90 | 0.90 (0.74 to 1.06) | <.01 | |
| Abdominal pain – I | 329 | 0.26 (0.20 to 0.32) | 90 | 0.48 (0.33 to 0.63) | <.01 | |
| Pain – F | 325 | 0.50 (0.43 to 0.58) | 90 | 1.21 (1.03 to 1.39) | <.01 | |
| Pain – S | 325 | 0.51 (0.43 to 0.59) | 90 | 1.36 (1.17 to 1.54) | <.01 | |
| Pain – I | 325 | 0.30 (0.23 to 0.36) | 89 | 0.88 (0.69 to 1.07) | <.01 | |
Groups were considered statistically different when CIs did not overlap between children with different performance levels. AE = adverse event; CI = confidence interval; F = frequency; S = severity; I = interference; Ped-PRO-CTCAE = Pediatric Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events; T2 = time 2.
†Medication use was reported by the caregiver or parent.
Figure 2.
Evaluation of known groups validity of Pediatric Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (Ped-PRO-CTCAE) item mean scores by medication use at time 2. F = frequency of symptom; S = severity of symptom. Means and 95% confidence intervals are provided. “Yes” indicates the child took a medication within the past 7 days for the symptom being assessed (eg, antiemetic drug for nausea). “No” indicates the child did not take a medication within the past 7 days for the symptom being assessed. Medications were specific to each symptom. Sample sizes varied by medication (see Table 5).
Responsiveness: Ped-PRO-CTCAE and MSAS
The right half of Table 2 presents correlations of change from T1 to T2 in Ped-PRO-CTCAE with MSAS 10–18 and MSAS 7–12 by age group. For 13- to 15-year-olds and 16- to 18-year-olds, strong correlations were noted between both measures, but less so for associations among the 7- to 12-year-olds. Because children self-reported few symptom changes over time, correlations of change over time between Ped-PRO-CTCAE and MSAS are notably lower than correlations of Ped-PRO-CTCAE and MSAS at each time point (25).
Substudy Test-Retest Reliability
The independent cohort included 46 ALL children or adolescents (70% female, 67% non-Hispanic white, mean age = 10.2 years). The median and average number of days between T1 and T2 for the sample was 7 days (range = 5–9 days). Supplementary Table 1 (available online) includes ICCs and percent agreement for the core Ped-PRO-CTCAE items. Most children did not report symptoms and the range was limited. Agreement between T1 and T2 ranged from 54.3% to 93.5%.
Discussion
Despite marked advances in cancer treatment and supportive care, children and adolescents (hereafter referred to as children) receiving cancer therapy experience multiple, subjective toxicities and symptoms secondary to treatment (31–33). Before starting therapy, children and their parents want to know how a given therapy may help them, and they are often most concerned with how they will feel during therapy (34,35). The answers to these questions rely heavily on recorded AEs in previous clinical trials; however, symptom AEs are underreported by clinicians (3–6), and evidence shows that more than 90% of children receiving cancer treatment did not report burdensome subjective toxicities unless directly asked by clinicians (12). Symptom AEs describe the child’s experiences with cancer and treatment, and the child is the most qualified person to report them. Patient-reported measures, such as the Ped-PRO-CTCAE measurement system, used in a standard manner with children receiving cancer treatment may be the best way to capture the child’s voice. Our supportive findings from this study of the Ped-PRO-CTCAE indicate that children in cancer treatment and their caregivers are willing (eg, >80% enrollment rate, high retention from T1 to T2) and able to self-report their subjective treatment experiences. The Ped-PRO-CTCAE items demonstrated strong convergent, discriminant, and known groups validity and responsiveness over time. Findings were consistent across age groups (ranging from 7 to 18 years) and at each time point.
Evidence for test-retest reliability was provided for Ped-PRO-CTCAE items in a substudy sample of children receiving ALL therapy in maintenance. ICCs were lower than expected; however, ICCs assume data are continuous and Ped-PRO-CTCAE items are ordinal. In addition, many children were not symptomatic; thus, there was a restricted range (28). Perfect agreement over time was also presented (26), and we observed most of the Ped-PRO-CTCAE items were in the good to very good range (30).
In sum, previously validated and established pediatric self-report symptom measures, the MSAS and PROMIS Pediatric measures, support the construct validity of the Ped-PRO-CTCAE items. Although the MSAS and PROMIS measures assess symptoms and functioning, they were not designed to align with clinician-graded CTCAE, which remains the gold standard for all AE grading. Thus, it is recommended that the Ped-PRO-CTCAE measurement system be used for safety and tolerability assessment to capture symptom AE experiences and questionnaires like PROMIS and MSAS be used as primary or secondary outcomes of intervention efficacy in pediatric trials.
The Ped-PRO-CTCAE measurement system is currently intended to provide data directly from the child to guide clinicians to grade AEs within the CTCAE framework and is not meant to replace clinician grading. The child’s answer to each Ped-PRO-CTCAE question should be summarized at the individual item level. A separate study has provided an algorithm that converts a child’s responses from the Ped-PRO-CTCAE items to recommended CTCAE grades based on input from a sample of pediatric oncology clinicians (36). The data from Ped-PRO-CTCAE are likely limited to CTCAE grades corresponding to grade 3 or lower. More serious AEs (ie, grade 4 that typically corresponds to life-threatening consequences, and grade 5 that typically corresponds with a death) are not captured by Ped-PRO-CTCAE.
Only pediatric oncology patients receiving cure-directed, up-front cancer treatment were included in this initial validation study, and sicker children (ie, those with relapsed cancer, on palliative care, or experiencing severe toxicities) are not represented. Additionally, not all items in the Ped-PRO-CTCAE library had a similar level of evidence. Core symptom AE items were collected from all participants, but less prevalent items were tested with fewer children to reduce response burden and need to be evaluated in future studies. We also delayed enrollment of participants for several weeks after their cancer diagnosis in an effort to reduce patient burden. This may have contributed to high symptom frequency at T1, meaning our hypothesis that symptoms would be statistically significantly worse at T2 did not prove to be true for all patients or symptoms. For this reason, responsiveness of the items should be further explored in longitudinal studies designed with strong clinical anchors to best capture change over time. Additional studies that include non-English-speaking participants are also warranted.
In conclusion, this study provides strong evidence to support use of the Ped-PRO-CTCAE measurement system with children and adolescents (7–18 years) receiving cancer treatment. Coupled with mounting support for the inclusion of patient-reported data for evaluating safety and efficacy of cancer treatments in trials (37), the demonstrated validity and reliability of the Ped-PRO-CTCAE support its incorporation into future pediatric oncology trials. Further, these items have been designed for ease of administration and interpretation and lend themselves well to longitudinal use in pediatric oncology care.
Funding
This work was supported by the National Institutes of Health including the National Cancer Institute (R01CA175759), and the National Institute of Arthritis and Musculoskeletal and Skin Diseases (U19AR069522).
Notes
The funders had no role in the design of the study; the collection, analysis, and interpretation of the data; the writing of the manuscript; and the decision to submit the manuscript for publication. All authors have no conflicts of interest to disclose.
We especially thank the children, adolescents, and their families for participating in this study and giving us their time to help us create better measures to capture their experiences.
Supplementary Material
References
- 1.International Council for Harmonisation. Medical Dictionary for Regulatory Activities (MedRA) https://www.meddra.org. Accessed August 5, 2019.
- 2.National Cancer Institute. Common Terminology Criteria for Adverse Events (CTCAE). 2018. https://ctep.cancer.gov/protocoldevelopment/electronic_applications/ctc.htm#ctc_50. Accessed March 1, 2020.
- 3. Collins JJ, Devine TD, Dick GS, et al. The measurement of symptoms in young children with cancer: the validation of the Memorial Symptom Assessment Scale in children aged 7-12. J Pain Symptom Manage. 2002;23(1):10–16. [DOI] [PubMed] [Google Scholar]
- 4. Dupuis LL, Taddio A, Kerr EN, Kelly A, MacKeigan L. Development and validation of the pediatric nausea assessment tool for use in children receiving antineoplastic agents. Pharmacotherapy. 2006;26(9):1221–1231. [DOI] [PubMed] [Google Scholar]
- 5. Hockenberry MJ, Hinds PS, Barrera P, et al. Three instruments to assess fatigue in children with cancer: the child, parent and staff perspectives. J Pain Symptom Manage. 2003;25(4):319–328. [DOI] [PubMed] [Google Scholar]
- 6. Zhukovsky DS, Rozmus CL, Robert RS, et al. Symptom profiles in children with advanced cancer: patient, family caregiver, and oncologist ratings. Cancer. 2015;121(22):4080–4087. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Reeve BB, Withycombe JS, Baker JN, et al. The first step to integrating the child’s voice in adverse event reporting in oncology trials: a content validation study among pediatric oncology clinicians. Pediatr Blood Cancer. 2013;60(7):1231–1236. [DOI] [PubMed] [Google Scholar]
- 8. Reeve BB, McFatrich M, Pinheiro LC, et al. Eliciting the child’s voice in adverse event reporting in oncology trials: cognitive interview findings from the pediatric patient-reported outcomes version of the Common Terminology Criteria for Adverse Events initiative. Pediatr Blood Cancer. 2017;64(3):e26261. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Basch E, Reeve BB, Mitchell SA, Clauser SB, et al. Development of the National Cancer Institute’s patient-reported outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). J Nat Cancer Inst. 2014;106(9):dju244. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Dueck AC, Mendoza TR, Mitchell SA, et al. Validity and reliability of the US National Cancer Institute’s patient-reported outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). JAMA Oncol. 2015;1(8):1051–1059. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Reeve BB, McFatrich M, Pinheiro LC, et al. Cognitive interview-based validation of the patient-reported outcomes version of the Common Terminology Criteria for Adverse Events in adolescents with cancer. J Pain Symptom Manage. 2017;53(4):759–766. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Weaver MS, Reeve BB, Baker JN, Martens CE, et al. Concept-elicitation phase for the development of the pediatric patient-reported outcome version of the Common Terminology Criteria for Adverse Events. Cancer. 2016;122(1):141–148. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Duke University Department of Population Health Sciences. Ped-PRO-CTCAE Measurement System. https://populationhealth.duke.edu/ped-pro-ctcae-measurement-system. Accessed December 10, 2019.
- 14. Collins JJ, Byrnes ME, Dunkel IJ, et al. The measurement of symptoms in children with cancer. J Pain Symptom Manage. 2000;19(5):363–377. [DOI] [PubMed] [Google Scholar]
- 15. Irwin DE, Stucky B, Langer MM, et al. An item response analysis of the pediatric PROMIS anxiety and depressive symptoms scales. Qual Life Res. 2010;19(4):595–607. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. DeWitt EM, Stucky BD, Thissen D, et al. Construction of the eight-item Patient-Reported Outcomes Measurement Information System pediatric physical function scales: built using item response theory. J Clin Epidemiol. 2011;64(7):794–804. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Lai J-S, Stucky BD, Thissen D, et al. Development and psychometric properties of the PROMIS(®) pediatric fatigue item banks. Qual Life Res. 2013;22(9):2417–2427. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18. Quinn H, Thissen D, Liu Y, et al. Using item response theory to enrich and expand the PROMIS® pediatric self report banks. Health Qual Life Outcomes. 2014;12(1):160. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Hinds PS, Wang J, Cheng YI, et al. PROMIS pediatric measures validated in a longitudinal study design in pediatric oncology. Pediatr Blood Cancer. 2019;66(5):e27606. [DOI] [PubMed] [Google Scholar]
- 20. Lansky SB, List MA, Lansky LL, Ritter-Sterr C, Miller DR. The measurement of performance in childhood cancer patients. Cancer. 1987;60(7):1651–1656. [DOI] [PubMed] [Google Scholar]
- 21. Cronbach LJ. Test “reliability”: its meaning and determination. Psychometrika [Internet]. 1947;12(1):1–16. [DOI] [PubMed] [Google Scholar]
- 22. Terwee CB, Bot SDM, de Boer MR, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. 2007;60(1):34–42. [DOI] [PubMed] [Google Scholar]
- 23. Cohen J. A power primer. Psychol Bull. 1992;112(1):155–159. [DOI] [PubMed] [Google Scholar]
- 24. Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. Hillsdale, NJ: L. Erlbaum Associates; 1988:567. [Google Scholar]
- 25. Goodwin LD, Leech NL. Understanding correlation: factors that affect the size of r. J Exp Educ. 2006;74(3):249–266. [Google Scholar]
- 26. Berchtold A. Test–retest: agreement or reliability? Method Innov. 2016;9:205979911667287. [Google Scholar]
- 27. Koo TK, Li MY. A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med. 2016;15(2):155–163. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Hallgren KA. Computing inter-rater reliability for observational data: an overview and tutorial. Tutor Quant Methods Psychol. 2012;8(1):23–34. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. McHugh ML. Interrater reliability: the kappa statistic. Biochem Med. 2012;22(3):276–282. [PMC free article] [PubMed] [Google Scholar]
- 30. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–174. [PubMed] [Google Scholar]
- 31. Hinds PS, Billups CA, Cao X, et al. Health-related quality of life in adolescents at the time of diagnosis with osteosarcoma or acute myeloid leukemia. Eur J Oncol Nurs. 2009;13(3):156–163. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Jones E, Isom S, Kemper KJ, McLean TW. Acupressure for chemotherapy-associated nausea and vomiting in children. J Soc Integr Oncol. 2008;6(4):141–145. [PubMed] [Google Scholar]
- 33. Cheng KK-F. Association of plasma methotrexate, neutropenia, hepatic dysfunction, nausea/vomiting and oral mucositis in children with cancer. Eur J Cancer Care. 2008;17(3):306–311. [DOI] [PubMed] [Google Scholar]
- 34. Schechter T, Grant R. The complexity of consenting to clinical research in phase I pediatric cancer studies. Pediatr Drugs. 2015;17(1):77–81. [DOI] [PubMed] [Google Scholar]
- 35. Miller VA, Baker JN, Leek AC, et al. Adolescent perspectives on phase I cancer research. Pediatr Blood Cancer. 2013;60(5):873–878. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. McFatrich M, Brondon J, Lucas NR, et al. Mapping child and adolescent self‐reported symptom data to clinician‐reported adverse event grading to improve pediatric oncology care and research. Cancer. 2020;126(1):140–147. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Kluetz PG, Slagle A, Papadopoulos EJ, et al. Focusing on core patient-reported outcomes in cancer clinical trials: symptomatic adverse events, physical function, and disease-related symptoms. Clin Cancer Res. 2016;22(7):1553–1558. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.