Recommended Scoring Approach for the Pediatric Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events

Abstract

Background

Collecting symptom, function and adverse event (AE) data directly from children and adolescents undergoing cancer care is more comprehensive and accurate than relying solely on their caregivers or clinicians for their interpretations. We developed the Pediatric Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (Ped-PRO-CTCAE) measurement system with input from children, parents, and clinicians. Here we report how we determined the recommended Ped-PRO-CTCAE item scoring approach.

Methods

Data from 271 patients were analyzed using three scoring approaches: 1) at the AE attribute (frequency, severity, interference) using ordinal and dichotomous measures, 2) a weighted composite AE item score by AE attribute (0.5 - frequency; 1.0 - severity; 1.5 - interference), and 3) overall number of AEs endorsed. Associations of each AE attribute, AE item score and overall AE score with the PROMIS^® Pediatric measurwere examined. The ability of the overall Ped-Pro-CTCAE AE score to identify patients with PROMIS symptom T-scores worse than reference population scores was assessed. Clinician preference for score information display was elicited through interviews with five pediatric oncology clinical trialists.

Results

The diverse scoring approaches yielded similar outcomes, including positive correlations of the Ped-PRO-CTCAE attributes, AE item score, and the overall AEs score with the PROMIS Pediatric measures. Clinicians preferred the most granular display of scoring information (actual score reported by the child and corresponding descriptive term).

Conclusions

Although three scoring approaches yielded similar results, we recommend the AE attribute level of one score per Ped-Pro-CTCAE AE attribute for its simplicity of use in care and research.

Introduction

In the United States, an estimated 11,050 children under age 15 were diagnosed with cancer in 2020.¹ Although pediatric cancer represents a small proportion (<1%) of overall cancer cases, the treatment burden associated with childhood cancers is large, resulting in impactful symptoms and adverse events (AEs) that can persist during and after treatment is concluded.^2–10 To deliver patient- and family-centered care and achieve established standards of care in pediatric oncology^11–14, it is critical to routinely assess and monitor symptoms and AEs throughout the course of a child’s care. Prior studies concluded that compared to children’s reports, parents, and clinicians under-report the frequency and severity of symptomatic treatment-related AEs (e.g., nausea, anxiety, depressive symptoms, pain, and fatigue).^15–19 Collecting symptom, function, and symptomatic AE data directly from children is a more comprehensive and accurate representation of their illness experiences, including the impact of cancer therapies, than relying solely on parent or clinician report.²⁰

Patient self-report is considered the gold standard for treatment-related symptom and function information in pediatric oncology.^12,21,22 However, given the multidimensional nature of symptomatic AEs in pediatric oncology and other childhood illnesses,^20,23 summarizing patient-reported data from multi-item, multi-domain instruments in real time for clinical use is challenging. While an overall score aggregating across multiple symptoms or AEs may be easier to calculate, it can mask specific symptom scores that merit clinical management (e.g. dose reduction, addition of supportive care).²⁴ Few existing, psychometrically validated, pediatric patient-reported outcome (PRO) instruments are designed to accurately report symptomatic AEs for children undergoing cancer care.²⁰ With these complexities in mind, we developed the Pediatric Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (Ped-PRO-CTCAE) measurement system with input from children/adolescents, caregivers and clinicians.^25,26 Here we report the steps taken to determine the recommended scoring approach for the Ped-PRO-CTCAE items in consideration of ease of clinical use (scoring and interpretation) by clinicians and equal attention to each symptomatic AE and its respective attributes.

The Ped-PRO-CTCAE and its Purpose

The PRO-CTCAE measurement system (adult and pediatric versions)^27–34 was designed to complement the National Cancer Institute’s (NCI’s) Common Terminology Criteria for Adverse Events (CTCAE), a severity grading scale for AEs. An AE is defined as an unfavorable, unintended symptom or disease associated in time with medical cancer care (procedure or treatment) that may or may not be related to the care itself (https://ctep.cancer.gov/protocoldevelopment/electronic_applications/ctc.htm).

The Ped-PRO-CTCAE consists of 62 patient-reported AEs assessing burdensome symptomatic AEs experienced by children and adolescents undergoing cancer therapy;^25,26,34 Fifteen of the AEs are considered ‘core’ because of how commonly they are experienced by children and adolescents undergoing cancer treatment. Depending on the AE, the Ped-PRO-CTCAE captures symptom frequency, severity, and/or interference with daily activities. The Ped-PRO-CTCAE features four response options and simplified items and directions, making it developmentally appropriate for children as young as 7 years-old.²⁶

Given its intended use in oncology clinical trials and healthcare settings, the Ped-PRO-CTCAE differs in three key ways from other PRO measures in the field such as the Patient-Reported Outcome Measurement Information System^® (PROMIS^®) Pediatric measures^35,36, the Pediatric Quality of Life Inventory ™ (PedsQL ™),³⁷ or the Memorial Symptom Assessment Scale (MSAS)^38,39. First, the Ped-PRO-CTCAE is designed as an item library appropriate for assessing a broad range of symptomatic AEs that may be experienced during cancer treatments. The library allows flexible selection of AEs for inclusion in a clinical trial or other use. This feature distinguishes the Ped-PRO-CTCAE from other PRO measures that have a fixed set of symptoms or functional domains. Second, the Ped-PRO-CTCAE includes only one to three questions for each AE to ascertain symptom frequency, severity, and/or interference with daily activities to inform AE grading, thus minimizing respondent burden. Third, the Ped-PRO-CTCAE, congruent with CTCAE grading, assesses the worst experience of a symptomatic AE ‘in the past 7 days’ whereas most PRO tools assess the average symptom experience over the past week.⁴⁰

AE Scoring and its Relevance to Clinical Care and Research

An accurate AE scoring method in pediatric oncology determines the impact of cancer treatments when the child’s reports are validly solicited and accurately documented. The AE scoring method must capture individual AEs that need intervention and AE change over time to learn the longer-term effects of cancer or symptom-directed interventions. An accurate scoring approach could position pediatric oncology PROs as an alternative and additional endpoint for early phase clinical trials, ³⁶ including tolerability and dose finding and dose modifications for toxicity. Additionally, scores could be used to validate a clinician’s clinical impressions, aid in the development of supportive care guidelines, and inform consent processes with descriptions of tolerance for a given therapy. Finally, the Ped-PRO-CTCAE could help to identify and predict children and adolescents at higher risk of having symptomatic AEs. Our goal is to offer clinically useful items to accurately address the symptomatic impact of cancer treatments on children in real time.

The Ped-PRO-CTCAE Dataset Used to Address Scoring Approaches

For this nine-site (US and Canada) Ped-PRO-CTCAE scoring methodology study, 271 children (ages 7–18) completed items for the 15 core AEs at two purposely distinct, sequential data points (anticipated to be low and high likelihood of symptomatic AEs) to evaluate the ability of the items to sensitively capture change over time in AEs and their association with established pediatric symptom measures. Sites contributed a diverse group of demographics, cancer types, and treatment. Time between the two data points (T1 and T2) ranged between 7 days and 4+ weeks, depending on disease/treatment categories.⁴⁰ More detail on the study design is published elsewhere.⁴⁰

Methods

Scoring Strategies

The alternative approaches we evaluated for scoring AEs were 1) at the AE attribute level (frequency, severity, interference) using ordinal (actual rating from the ill child from 0 to 3 per attribute) and dichotomous (0 – did not experience the AE attribute, 1 – experienced the level of AE attributes) approaches, 2) a weighted composite AE item score by AE attribute (0.5 - frequency; 1.0 - severity; 1.5 - interference) and 3) an overall AEs score (total number of AEs endorsed). Associations of each AE attribute, AE item score and the overall AE score with the PROMIS Pediatric measures of anxiety, depressive symptoms, and fatigue were examined. The ability of the overall AE score to identify patients with a PROMIS symptom T-score above (worse than) the minimal-moderate severity cut-off point in the reference population was also examined.^41,42

Scoring at the Ped-PRO-CTCAE AE Attribute Level

Each AE attribute is measured on a 4-point Likert scale (0–3). Ordinal Likert scale measures with less than 5 response options are treated as categorical measures,^43,44 thus we treated the Ped-PRO-CTCAE items as ordinal categories. Because most of the AE item attributes had skewed distributions towards lower response options, we also created a dichotomous variable (0-did not experience any AE attribute;1-experienced any level of AE attributes) for analysis of each AE attribute. We included the binary indicator (0, 1) at the AE attribute level (e.g., fatigue severity – 0/1) along with scoring the AE attribute item at the original response option level (e.g., fatigue severity – 0/1/2/3) (see Supporting Information File 1).

Scoring at the AE Level

We generated a single AE item score that aggregated the three attributes of an AE. Rather than a simple average, we combined individual attribute scores into a composite score by assigning weights to each of the AE attributes. Because there is no established method for calculating such weights, we used a weighting scheme (0.5 for frequency; 1.0 for severity; 1.5 for interference) based on expert clinician input that interference was the most relevant characteristic to inform patient care (see the example in Insert 1).

Overall Scoring by Number of Endorsed Core AEs

Individual AE item scores and AE attribute scores are clinically informative and important. However, an overall score for the measured core AEs may provide summary information for an assessment of the symptomatic AEs experienced during a given treatment. A count of the number of endorsed core AEs a child experiences may serve as an overall AE score. A threshold point of the overall AE score could be a checkmark for symptom assessment in clinical practice. We identified a threshold point using receiver operating characteristic (ROC) models.⁴⁵

Association Analyses

Our analyses focused on examining the associations of AE scores with previously validated PROMIS Pediatric measures representing the most frequently reported troubling symptoms by children during cancer care: fatigue, depressive symptoms, and anxiety.^35,36,41 Associations of the ordinal scale (0–3) of the Ped-PRO-CTCAE AE attributes with the PROMIS T-scores were tested using polyserial correlations.⁴⁵ The associations of the dichotomous measures of AE attributes (0 – did not experience the AE attribute; 1 – experienced any level of AE attributes) with PROMIS T-scores were assessed using biserial correlations.⁴⁷ The associations of AE item scores and the overall AE score with PROMIS measures were assessed using Spearman’s correlation.⁴⁸ If correlations were not statistically significant (p>0.05), the corresponding scores were considered less defensible.

Finally, the ROC model ^47,49 was used to examine the diagnostic ability of the overall AE score to predict the probability for patients to have PROMIS Pediatric scores higher than the minimal-moderate severity cut-off point in the reference population (i.e., >52 for anxiety; >53.5 for depression; and >47.5 for fatigue).⁴³ The area under curve (AUC) evaluated the accuracy of the diagnostic test; as the accuracy improves, the AUC approaches 1.0. ROC models established a cut-off for the overall AE score that may be a threshold to prompt clinical attention. Scores of the PROMIS Pediatric Anxiety, Depressive Symptoms, and Fatigue measures were recoded as 1 if scores were greater than the minimal-moderate cutoff point (i.e., >52 for anxiety; >53.5 for depression; and >47.5 for fatigue), otherwise 0; and the dichotomous measures were used as the dependent variables in the ROC models. Age group was controlled in the ROC model.

Clinician Preference for Score Display

Clinician members of the study team were asked to identify clinical colleagues in pediatric oncology with extensive experience as clinical trialists. The named clinical trialists were contacted by our study team via email and asked to participate in a 60-minute interview regarding implementing the Ped-PRO-CTCAE items in clinical trials. Interviews were conducted from February through July of 2021 by two members of the study team. Seven clinical trialists were contacted and five responded to the invitation. Clinicians were shown four different score display options on the Zoom screen and asked to select a preference regarding presentation of scoring information (see Insert 2). All interviews were conducted via Zoom and audio recorded.

Results

Patient Study Sample

The study sample comprised 271 children aged 7–18 years (mean age=13.4, sd=3.4); slightly more males (n=138, 51%) participated. The predominately represented races and ethnicity were non-Hispanic White (n=145, 53.5%), non-Hispanic Black (n=47, 17.3%), Hispanic/Latino (n=38, 14.0%), and non-Hispanic Asian (n=14, 5.2%). The cancers represented were varied with Leukemia/Lymphoma and solid tumor diagnoses being most represented (n=149, 55%); notably, 12.9% (n=35) were CNS tumors. Most children (92.3%) were receiving chemotherapy and smaller subsets were receiving radiation therapy or bone marrow transplant (Table 1).

Table 1:

Sample Characteristics

Sample Statistics (N=271)
Variable	Statistics
Child age (years)
Mean (SD)	13.4 (3.4)
Range	7.1–18.9
	N (%)
Child Gender
Female	131 (48.7)
Male	138 (51.3)
Child Race/Ethnicity
Non-Hispanic White	145 (53.5)
Non-Hispanic Black or African American	47 (17.3)
Non-Hispanic Asian	14 (5.2)
Non-Hispanic More than One Race	11 (4.1)
Hispanic/Latino	38 (14.0)
Other	8 (3.0)
Unknown Race/Ethnicity	8 (3.0)
Disease Diagnosis
Leukemia/Lymphoma	149 (55.0)
Solid Tumor	80 (29.5)
Neuro-Oncology	35 (12.9)
BMT	7 (2.6)
Cancer Treatment Received
Chemotherapy	250 (92.3)
Radiation Therapy	14 (5.2)
Bone Marrow Transplant	7 (2.6)

Clinician Study Sample

All five nominated clinicians (four women, one man) were from academic medical institutions. The five participants were pediatric oncologists with experience of chairing disease committees for cooperative clinical trials. Three clinicians were focused on early phase clinical trials and two on late phase trials. All oncologists were at least seven years post-hematology/oncology fellowship.

Descriptive Ped-PRO-CTCAE Findings

Response rates were high across AEs. Items about constipation had the highest rate of missing values (4.4%) than items about other AEs. For most AEs, the percentage of children reporting a high score (3: “Almost all the time”) for frequency was <5%, though the percentage was slightly higher for insomnia (7%). The percentages of children reporting a high score on severity (3: “very bad”) and interference (3: “A whole lot”) were low for all AEs except for insomnia severity (>5%). A substantial proportion of children (96%) reported having at least one symptomatic AE (score≥1). The most prevalent AE reported was fatigue. Sixty-six percent of the children reported having at least “A little bad” fatigue.

Association of Ped-PRO-CTCAE AE Attributes with PROMIS Pediatric Symptom Measures

Polyserial correlations of the Ped-PRO-CTCAE symptomatic AE attributes with the PROMIS Pediatric symptom measures are shown in Table 2. Attribute level scores of anxiety, depressive symptoms, and fatigue AEs showed large positive correlations (i.e., r ≥ 0.50)⁵⁰ with the corresponding PROMIS Pediatric measures. Interference attribute scores for anxiety and depression AEs also had large positive correlations with PROMIS Pediatric Fatigue scores (i.e., r >0.50). Most of the other AE items had low to medium positive correlations (i.e., r = 0.30–0.49) with the three PROMIS Pediatric measures. The interference attribute of pain had a strong positive correlation (r=0.52) with the PROMIS Pediatric Fatigue measure.

Table 2:

Polyserial Correlations^& of Ped-PRO-CTCAE Symptomatic AEs with PROMIS Pediatric Measures

		PROMIS Pediatric T-score
Pediatric PRO-CTCAE	Attribute	Depression	Anxiety	Fatigue
		r	r	r
Abdominal Pain	Frequency1	0.34	0.31	0.45
	Severity2	0.34	0.35	0.44
	Interference3	0.32	0.31	0.41
Anorexia	Frequency1	0.29	0.31	0.31
Nausea	Frequency1	0.35	0.32	0.42
	Severity2	0.39	0.35	0.45
	Interference3	0.37	0.30	0.45
Vomiting	Frequency1	0.27	0.22	0.31
	Interference3	0.36	0.31	0.37
Constipation	Frerquency1	0.13	0.15	0.26
	Severity2	0.16	0.20	0.32
	Interference3	0.31	0.21	0.29
Diarrhea	Frequency1	0.17	0.16	0.19
	Interference3	0.31	0.19	0.30
Cough	Frequency1	0.14	0.11	0.11
	Severity2	0.16	0.13	0.17
	Interference3	0.32	0.28	0.34
Pain	Frequency1	0.33	0.36	0.49
	Severity2	0.30	0.33	0.47
	Interference3	0.34	0.31	0.52
Headache	Frequency1	0.34	0.31	0.39
	Severity2	0.38	0.34	0.41
	Interference3	0.42	0.36	0.41
Neuropathy	Severity2	0.35	0.38	0.36
	Interference3	0.46	0.44	0.48
Mucositis	Frequency1	0.20	0.26	0.24
	Severity2	0.24	0.27	0.27
	Interference3	0.26	0.31	0.28
Anxiety	Frequency1	0.69	0.77	0.48
	Severity2	0.64	0.73	0.44
	Interference3	0.68	0.64	0.56
Depression	Severity2	0.74	0.63	0.47
	Interference3	0.71	0.63	0.50
Fatigue	Severity2	0.46	0.43	0.58
	Interference3	0.44	0.41	0.60
Insomnia	Frequency1	0.39	0.39	0.39
	Severity2	0.39	0.41	0.42
	Interference3	0.46	0.38	0.49

¹: 0=Never, 1=Sometimes, 2=Most of the time, 3=Almost all the time.

²: 0=Did not have any, 1=A little bad, 2=Bad, 3=Very bad.

³: 0=Not at all, 1=Some, 2=A lot, 3=A whole lot.

Note

r>0.50 large positive correlation, r >0.3–0.4= moderate positive correlation

Biserial correlations represent the correlations of the dichotomous measure (0-did not experience the AE attribute;1-experienced the level of AE attribute) of each AE attribute with PROMIS Pediatric measures (Table 3). Patterns observed in polyserial correlations remained in the biserial correlations. The dichotomous measures of anxiety, depression, and fatigue AE attributes had large correlations (r ≥ 0.50) with corresponding PROMIS Pediatric measures. Frequency and severity attributes for pain also had large positive correlations (r=0.53 and r=0.53, respectively) with the PROMIS Pediatric Fatigue score. When comparing biserial and polyserial correlations, anxiety interference also had a large positive correlation (r=0.52) with the PROMIS Pediatric Fatigue score although correlations between AE depression interference and PROMIS Pediatric Fatigue scores slightly declined (from r=0.50 to r=0.48). Additionally, the correlation of interference of AE insomnia with the PROMIS Pediatric Fatigue score slightly increased (from r=0.49 to r=0.52).

Table 3:

Biserial Correlations^& of Ped-PRO-CTCAE Symptomatic AE Dichotomous Items with PROMIS Pediatric Measures

		PROMIS Pediatric T-score
Pediatric PRO-CTCAE	Attribute	Depression	Anxiety	Fatigue
		r	r	r
Abdominal Pain	Frequency	0.36	0.36	0.45
	Severity	0.36	0.39	0.47
	Interference	0.30	0.30	0.41
Anorexia	Frequency	0.35	0.38	0.36
Nausea	Frequency	0.34	0.34	0.45
	Severity	0.41	0.37	0.46
	Interference	0.35	0.28	0.46
Vomiting	Frequency	0.26	0.23	0.31
	Interference	0.33	0.30	0.37
Constipation	Frequency	0.19	0.19	0.29
	Severity	0.12	0.16	0.30
	Interference	0.33	0.20	0.28
Diarrhea	Frequency	0.17	0.14	0.20
	Interference	0.29	0.18	0.29
Cough	Frequency	0.18	0.14	0.11
	Severity	0.16	0.13	0.17
	Interference	0.30	0.25	0.33
Pain	Frequency	0.38	0.41	0.53
	Severity	0.38	0.42	0.53
	Interference	0.35	0.31	0.49
Headache	Frequency	0.40	0.36	0.42
	Severity	0.40	0.36	0.45
	Interference	0.43	0.38	0.40
Neuropathy	Severity	0.33	0.37	0.32
	Interference	0.45	0.42	0.49
Mucositis	Frequency	0.23	0.29	0.27
	Severity	0.23	0.29	0.27
	Interference	0.28	0.35	0.30
Anxiety	Frequency	0.67	0.80	0.49
	Severity	0.61	0.75	0.46
	Interference	0.63	0.61	0.52
Depression	Severity	0.72	0.64	0.45
	Interference	0.68	0.60	0.48
Fatigue	Severity	0.54	0.48	0.58
	Interference	0.48	0.42	0.62
Insomnia	Frequency	0.38	0.36	0.37
	Severity	0.39	0.39	0.40
	Interference	0.50	0.41	0.52

Associations of Ped-PRO-CTCAE AE Item Scores and the Overall AE Score with PROMIS Pediatric Measures

Spearman correlations of the Ped-PRO-CTCAE weighted composite AE item scores with PROMIS Pediatric scores are shown in Table 4. All anxiety, depression, and fatigue item scores had strong positive correlations (i.e., r ≥ 0.50) ⁵¹ with corresponding PROMIS Pediatric measures, ranging from 0.59 to 0.72. The overall AE score is also highly correlated (r >0.50) with each of the PROMIS Pediatric measures (Table 4).

Table 4:

Spearman Correlations of Weighted AE Scores with PROMIS Pediatric Measures (n=217)

Pediatric PRO-CTCAE	PROMIS Pediatric T-score
	Depression	Anxiety	Fatigue
	r	r	r
Abdominal Pain	0.30	0.31	0.40
Anorexia	0.27	0.30	0.29
Nausea	0.33	0.29	0.40
Vomiting	0.22	0.19	0.25
Constipation	0.15	0.19	0.24
Diarrhea	0.15	0.12	0.16
Cough	0.14	0.12	0.12
Pain	0.33	0.34	0.47
Headache	0.36	0.31	0.38
Neuropathy	0.29	0.33	0.28
Mucositis	0.19	0.23	0.22
Anxiety	0.61	0.72	0.44
Depression	0.65	0.57	0.42
Fatigue	0.48	0.43	0.59
Insomnia	0.38	0.35	0.40
Overall AE Score	0.59	0.59	0.60

ROC Curve Analyses

The empirical ROC curves by dependent variable (anxiety, depressive symptoms, and fatigue) had AUCs of 0.82, 0.79, and 0.81, respectively, and are substantially larger than the acceptable level (AUC=0.70) of predictive ability.⁵² Thus, the overall AE score would have approximately 80% chance of correctly distinguishing PROMIS symptom scores higher than the minimal-moderate severity cutoff point in the reference population. The optimal cutoff value of AEs determined by the ROC analysis for the three PROMIS measures is 6 for depression, 6 for fatigue, and 7 for anxiety. That is, the optimal cutoff value (optimal decision threshold) is about the overall mean value (i.e., 6.3) of the total AEs in the population. For this optimal cutoff value, the sensitivity value is high ranging from 0.76 to 0.88, and the specificity value is adequate ranging from 0.63 to 0.65. Thus, when child-reported presence of AEs reaches the population average (approximately 6) or higher, regardless of the AE severity, the probability of classifying the child as having a higher than the minimal-moderate cutoff point (i.e., >52 for anxiety; >53.5 for depression; and >47.5 for fatigue) in the specific PROMIS Pediatric measure is high (i.e., 80%) (Table 5).

Table 5:

Cut-off Point for the Ped-PRO-CTCAE Overall Scores to Predict PROMIS Pediatric Measure T-scores Using ROC Analysis (n=271)

PROMIS Pediatric T-score	Total Number of AEs Endorsed from the Ped-PRO-CTCAE	Sensitivity	Specificity
Depressive Symptoms >53.5	6	0.76	0.63
Anxiety >52	7	0.86	0.65
Fatigue >47.5	6	0.88	0.65

Ped-PRO-CTCAE total AEs: mean=6.3, SD=3.7, range=0–15

Clinician Preferences for Score Display

All five clinicians indicated a preference for the more granular approach to score information display, meaning inclusion of the AE attribute, the AE attribute numerical score as reported by the child, and the word text beside the numerical score that represented the score (i.e., Severity = 2, “Bad”; Interference = 2, “A lot”) (Supporting Information File 2, example 1A).

Discussion

Having a clinically relevant scoring approach for the validated Ped-PRO-CTCAE items that can be quickly and accurately applied by clinicians directly brings in the child’s perspectives and experiences to routine AE monitoring. The three scoring approaches assessed in this study included: 1) individual AE item attribute scores (frequency, severity, and interference), 2) weighted AE item scores, and 3) an overall AE score that reflected the number of core AE items endorsed by children. Our analyses sought to compare the outcomes of these distinct scoring approaches in consideration of capturing the ill child’s voice, ease of use by clinicians, accuracy of identifying high scoring AEs meriting intervention, and simultaneously identifying multiple reported AEs regardless of the level of scoring. In our multiple statistical scoring approaches, outcomes had similar strong positive correlations between scored Ped-PRO-CTCAE item scores, item attribute scores and overall core AE scores with the PROMIS Pediatric measures (all correlations ranging from 0.48 to 0.72). In general, the magnitude of the correlations by AE item attribute and PROMIS Pediatric measures for fatigue, depressive symptoms, and anxiety were similar. The weighted AE attribute composite scores had somewhat higher positive correlations with the PROMIS Pediatric measures. Of note, the overall scoring approach and its numerical outcomes would only be clinically applicable if the complete set of core Ped-PRO-CTCAE items were administered to participants as actual AE scores are item-based.

The ROC models using the overall AEs score had an 80% chance of correctly distinguishing PROMIS Pediatric Anxiety, Depressive Symptoms and Fatigue measure scores that exceeded the minimal-moderate cutoff points of the PROMIS Pediatric scores in the reference population (i.e., >52 for anxiety; >53.5 for depression; and >47.5 for fatigue). An overall AEs score of 6 from the 15 measured core AEs achieved a high sensitivity value ranging from 0.76 to 0.88 and an adequate specificity value from 0.63 to 0.65 for each of the PROMIS measures, indicating that regardless of AE attribute severity, this number of endorsed symptomatic AEs would likely identify a child scoring above the minimal-moderate severity cutoff point of the PROMIS Pediatric measures. The value of this scoring approach may help clinicians identify a child with multiple AEs of low severity. When symptomatic AEs are assessed independently rather than considered together, clinicians may mistakenly conclude that the child is not experiencing high AE burden. Our scoring approach would remedy this misconception. However, fewer AEs (e.g., <6) does not necessarily mean a lower AE burden. Severity of any AE (e.g., pain) could be a clinical concern.

Although our tested scoring approaches yielded consistent results, concerns for clinical relevance directed us to endorse the ordinal approach that summarizes symptomatic AEs at the individual AE attribute level (e.g., severity, frequency and/or interference) and reporting the specific answer the child provided in response to the Ped-PRO-CTCAE. This efficient approach provides clinicians with a straightforward, accurate, and easy to calculate score that reflects exactly what the child reported without any further filtering or mathematical manipulation. Further, it allows designers of clinical trials to select Ped-PRO-CTCAE items that they anticipate would be most relevant to document treatment impact. Scoring at the individual attribute level is consistent with the CTCAE grading and the adult PRO-CTCAE scoring. It also accurately reflects the child’s reports as the scoring makes no alterations/refinements to their self-report. This scoring method renders the Ped-PRO-CTCAE items easier to use in both clinical care and clinical trials. Finally, this scoring method is consistent with the preferred score display of the five interviewed pediatric oncologists and trialists.

Our study network completed complementary work to create a Ped-PRO-CTCAE to CTCAE mapping algorithm to assess if AE attributes in the Ped-PRO-CTCAE could be combined into a single symptom AE score.⁵³ That single score was then compared with clinician-reported CTCAE grade for that AE. This exploratory approach, involving 10 pediatric oncologists, requires additional evaluation. To accelerate the use of the Ped-PRO-CTCAE, we recommend future work on visual presentation of PRO data to optimize interpretability and inform clinical care and research. Examining the actual clinical usefulness of this recommended scoring system and incorporation of Ped-PRO-CTCAE into the electronic health record are important next steps.

Limitations

Though the five pediatric oncology clinical trialists had complete agreement about their preference for scoring display, more information likely would have resulted if more individuals (such as clinical research coordinators and advanced practice clinicians) who are involved in these clinical trials (i.e., conduct, documentation and reporting of adverse events) had been included in the scoring display preference step.

Conclusion

The findings generated from multiple scoring approaches support the validity of the Ped-PRO-CTCAE items, and the consistent similarity in outcomes of the scoring methods support the robustness of our findings. For simplicity in clinical care, research, and clinical trials, we recommended using the AE attribute method of one score per AE attribute experienced (frequency, severity, or interference). This scoring approach gives attention to each measured AE attribute equally and quickly provides clinicians and researchers with the child’s perspective of the impact of our cancer therapies.

Supplementary Material

supinfo1

Supporting Information File 1: Scoring Example

Supporting Information File 2: Clinician Interview Scoring Display Options

Abbreviations:

AE

adverse event

AUC

area under the curve

CTCAE

Common Terminology Criteria for Adverse Events

MSAS

Memorial Symptom Assessment Scale

PedsQL

Pediatric Quality of Life Inventory

PRO

patient-reported outcome

PROMIS

Patient-Reported Outcome Measurement Information System