Voices of Children and Adolescents on Phase I or Phase II Cancer Trials: A New Trial Endpoint?

Abstract

Background

Pediatric participants on Phase I or Phase II clinical trials for incurable cancer are at risk of experiencing toxicities (adverse events, AEs) related to trial participation. Multiple AEs are subjective; thus, the real impact of trial treatment cannot be known unless patient subjective reports are solicited.

Methods

We assessed the feasibility and acceptability of soliciting symptom, function and quality of life (QoL) reports from 8- to 18-year-old participants enrolled on Phase I/II clinical trials at four cancer centers during the first course of chemotherapy. We also assessed the reliability and validity of six self-report PROMIS pediatric measures and four open-ended interview questions at two time points (trial enrollment, T1 and 3 to 4 weeks later, T2).

Results

The enrollment rate of 75.9% (n=20) exceeded our feasibility criterion, and missingness of measures by person, measure and items at T1 and T2 were lower than our acceptability criteria. New QoL themes were limited to the impact of treatment on families and being away from home, family and friends for treatment. All but one measure at T1 met the reliability criterion and all did at T2. Validity support was limited though as theorized, mobility decreased and fatigue increased as AEs increased.

Conclusions

Soliciting and documenting symptom, function and QoL reports from 8- to 18-year-olds enrolled on a Phase I/II clinical trial is feasible and acceptable to participants, particularly when embedded in trials. Reliable and valid findings can result, making patient self-reported outcomes a possible new trial endpoint.

Background

In the United States, cancer is the number one disease-related cause of death in children aged 1 to 19 years;¹ approximately 25% of the 12,400 children newly diagnosed with cancer will die of their disease.² A priority of the National Cancer Institute, the National Institute of Nursing Research and professional groups is to improve quality of life of pediatric patients with incurable cancer.^3-5 Measuring these patients' symptom and quality of life (QoL) reports or patient- reported outcomes (PROs) will help clinicians better anticipate and manage symptoms and thereby reduce patient and family suffering. PROs are included in therapeutic trials where the objective is cure.^3,6-8 It is exceedingly rare for PROs to be incorporated into pediatric Phase I or Phase II trials though these trials contribute to new drug indications and labeling. The U.S. Food and Drug Administration has released guidance concerning the use of PROs in trials to support drug labeling, including language requiring PROs in pediatrics .^9-11 To date, no pediatric oncology drug indication has been secured on the basis of PROs.

Experimental drug trials collect organ and system treatment-related adverse events (AEs) but not the child's report of symptom or function AEs. Two recent reports of pediatric Phase I/II trials included a generic QoL measure, but symptom and function PROs specific to impact of the study treatment were not solicited.^12,13 Failure to include these PROs means that children and adolescents enrolled on Phase I/II clinical trials who are very likely to experience AEs ^14-18 are not systematically asked to report on their symptoms or functioning while enrolled on the trial. Minus these patient reports, the full impact of the treatment on the patient is likely underreported and thereby undermanaged.

Using PRO measures, children as young as 8 can reliably report on most aspects of their health status, and younger children have reliably reported on selected health domains using paper or computer-based methods. ^7,8,19-21 Because cancer clinical trials have longitudinal designs, pediatric self-report measures must be able to capture statistically significant and clinically meaningful change or minimally important differences (MID) in PROs. MID is the smallest difference in scores reported as beneficial (improvement) or harmful (deterioration) and requires a change in treatment.^22,23 The PROMIS (Patient-Reported Outcomes Measurement Information System) initiative sponsored by the NIH Roadmap for Medical Research developed and validated instrumentation to measure symptoms, function and other aspects of QoL by patient report in 8- to 18-year-olds with cancer.^24,25 In this study, these measures in combination with an interview allowed us to go beyond the traditionally measured benefit of Phase I/II participation by children and adolescents with incurable cancer (tumor response) to measuring their symptoms, function and QoL.

Our study purposes were to demonstrate feasibility and acceptability of completing PROMIS pediatric measures by children enrolled on a Phase I/II trial at two data points where we anticipated clinical changes. In this longitudinal validity study (T1, time of trial enrollment; T2, 3 to 4 weeks later) involving 8- to 18-year-olds with incurable or refractory cancer enrolled on Phase I/II trials, we sought to:

Aim 1: assess the feasibility (enrollments and study refusals) and acceptability (attrition and unanswered measures or items) of the PROMIS pediatric measures to participants, and

Aim 2: develop initial estimates of the reliability, validity, responsiveness and range of MID of the PROMIS pediatric measures in this population.

Our study framework was the Theory of Unpleasant Symptoms^26,27 that posits symptom characteristics (duration, quality, distress, and intensity) and their outcomes are influenced by physiologic, psychological, and situational factors. Given our anticipated limited sample size, we did not test the framework but assessed theoretically-derived, previously supported relationships^6-8 as validity assessments of the PROMIS measures in this patient group e.g., higher fatigue is associated with higher toxicity.

Methods

We used a pre-post design and qualitative and quantitative methods to solicit patient-reports. We recruited children and adolescents enrolled on a Phase I/II trial at four cancer settings: Children's National Health System, Seattle Children's Hospital, Children's Hospital of Philadelphia, and Boston Children's Hospital.

Inclusion criteria

Eligible children had a cancer diagnosis, were between 8 and 18 years and able to speak and read English, were enrolled on a Phase I/II trial because of advanced cancer for which no curative therapy existed, and were able to manipulate a computer mouse for responding to computer adaptive test (CAT) items.

Exclusion criteria

Patients were excluded if the parent or child refused participation.

Study Procedures

Patients were recruited during an inpatient or outpatient visit after enrollment in a Phase I/II trial. If the parents gave permission to approach their child about the study, the team member explained the study to the child. Parent and child refusals and agreements were documented. Participants were given a study identification number and completed the PROMIS measures on a private clinic/hospital room computer or lap top.

Study Measures

PROMIS Pediatric Measures at T1 and T2

These measures document child and adolescent reports of their symptom and function experiences with illness and treatment. We administered the 6-item short form measures for mobility, pain, fatigue, depressive symptoms, anxiety, and peer relationships as previously administered via computer in a cross-sectional NIH-supported study.²⁴

Quality of Life Interview Questions for T2

We asked participants open-ended interview questions previously used with 8- to 18-year-old cancer patients regarding their QoL while receiving chemotherapy²⁸ and two additional questions (3 and 4) that addressed acceptability (Table 2).

Table 2. Coded Responses to QoL Interview Questions.

Please share with me what makes a good day for you since you began your new study medication? (n=15)
Theme	Definition	Code Frequency/Number of Participants
Being with Familiar Others	Spending time with family, friends and pets	13/9
Having Fewer Life Interruptions Secondary to Treatment	Experiencing less medications and side-effects, easier administration of the new drug and fewer medical appointments	6/3
Feeling well	Participating in sports, play and other activities	15/9
Seeing my Tumor get Smaller	perceiving the study drug is effective against own disease	2/1
A Bad Day? (n=15)
Feeling ill	Adverse effects of treatment mean not being able to do usual activities	14/7
Having to accommodate treatment	Daily activities, medication schedules and hospital appointments are dictated by the new protocol	7/5
Imagining the worse	Being uneasy about pain and its cause	3/3
Please share with me how being on this new study and its medication has been for you? (any surprises?) (n=16)
Being on study is positive	The new treatment is more convenient and manageable than conventional chemotherapy (fewer side effects) and may affect disease	14/7
Not experiencing any surprises on the study	Being on the new protocol was without unexpected events.	13/9
Loathing time needed for treatment	Not liking to be at the hospital as care takes valuable time away from family and friends	11/6
What else is important for your doctors and nurses to know about what it is like for you to be on this study medication? (n=15)
Feeling positive about this experience	patientsreport generally feeling well, engaging in more typical activities,and that the study is not causing them or their family difficulties	7/6
Nothing to add	No new content offered	6/6
Waiting to see if the treatment makes a difference	Biding time to learn if the disease responds to the study drug	2/2
Not liking to wait for treatment	Having to spend time to receive care and follow study guidelines make it hard to be on the protocol	2/1

Demographic and clinical information

Demographic (T1) and clinical information (T1, T2) were collected on each patient participant. Clinical information was comprised of diagnosis and toxicity reports, the latter routinely measured at each visit by clinician report using the Common Toxicity Criteria for Adverse Event (CTCAE) reporting form.²⁹

Statistical Analysis

Preliminary data were analyzed using descriptive and graphical methods. All item response variables were examined using measures of central tendency (mean), spread (standard deviation), and response category frequencies. Composite measures were used for all the PROMIS measures in validity and responsiveness tests.

We documented the number of parents who agreed to their child being invited to the study and those who declined. We also documented reasons for agreement and refusal per standard methods.³⁰ We considered feasibility and acceptability achieved if the rate of enrollment equaled or exceeded 70%, if attrition was 15% or less, and if missingness was less than or similar to previous palliative care studies.^6,31,32 We also used descriptive statistics to assess missingness by PROMIS measure, participant, item and time point.

The interview responses from the open-ended questions were analyzed using a semantic (theme or meaning based) content analytic technique with coding and inter-rater reliability assessments using Krippendorff's strategies.^33,34 We considered rater agreement (involving 2 raters) to be achieved at the level of 90% or higher. Resulting themes were compared with the PROMIS measures to assess if new QoL domains emerged.

The reliability of the PROMIS measures at T1 and T2 was assessed using Cronbach's alpha. AE data (type, grade and attribution) for each participant were extracted from the CTCAE with all AEs reported as per the primary treatment study guidelines and inclusive of the T1 and T2 dates. If an AE occurred multiple times during the course, only the highest grade for the longest duration was included in the analysis. The majority of the clinical trials used the CTCAE v. 4 and all but one used the 5-point attribution scale. The difference in the mean scores of the PROMIS measures by the number (>8, ≤8) of AEs was used to examine the criterion validity of the PROMIS measures. As sample size is small (N=20), a bootstrap approach³⁵ was applied for estimating Cronbach's alpha with standard error and testing validity of each PROMIS measure. Bootstrapping was performed based on 1,000 resamples randomly drawn from the original sample without replacement.

Internal responsiveness (ability of a measure to change) was examined using 1) paired T-test to assess the difference in PROMIS mean scores between T1 and T2; 2) effect size I (ESI), relating the average change in outcome between T1 and T2 to the variability in baseline scores; and 3) effect size II (ESII), relating the average change in outcome between T1 and T2 to the variability in change scores. ^36,37 We considered the magnitude of effect size < 0.2, 0.2-0.49, 0.5-0.79, and > 0.8 as negligible, small, moderate, and large, respectively.³⁸ In addition, the smallest real difference (SRD) (the 95% confidence limit of the standard error of the difference scores) was also used to assess sensitivity to change.³⁹ For individuals, the change score of each PROMIS measure was compared to the corresponding SRD. When the change exceeded the SRD, it was assumed that a real change occurred that was not attributable to ‘noise’. We also applied the established, patient-based MID score of a 3-point difference (+/- 3 points) in PROMIS measures to represent important score differences between the two data points.⁴⁰

Results

Aim 1: 29 patients met eligibility criteria during the 20 months of screening at the four study sites; 5 parents declined and 2 patients declined with the remaining 22 agreeing to participate (75.9% enrollment rate). Of these, 20 (90.9%) participated at T1 and 17 (77.3%) at T2. Most were male, adolescents, and white (Table 1). Most had a solid tumor (n = 13). Ten protocols were represented; 7 participants were enrolled on the same Phase I trial open at all four sites in which the PRO measures were embedded. The time to complete study measures at T1 ranged from 11 to 20 minutes and at T2 from 24 to 30 minutes as the interview questions occurred at T2.

Table 1. Sample Demographic Characteristics.

Characteristics	Participants (%) N = 20
Child Sex
Male	12 (60.0)
Female	8 (40.0)
Child Age Full Sample (years)
8-12	7 (35.0)
13-17	13 (65.0)
Age (Mean, SD)	13.6 (3.3)
Child Race
White	14 (70.0)
Black/African American	3 (15.0)
Asian	2 (10.0)
Other	1 (5.0)
Child Ethnicity
Non-Hispanic	17 (85.0)
Hispanic	3 (15.0)
Child's Diagnosis
Solid Tumor	13 (65)
Brain Tumor	5 (25)
Lymphoma	2 (10)
Adult Respondent's Relationship to Child
Mother/stepmother	16 (80.0)
Father/stepfather	4 (20.0)
Adult Respondent's Education Level
Some high school	1 (5.0)
High school degree/GED	3 (15.0)
Some college/technical degree	8 (40.0)
College degree	3 (15.0)
Advanced degree	4 (20.0)
Unknown	1 (5.0)

PROMIS measure missingness at T1 was limited to 1 person not completing two measures and two persons not completing one, for an overall rate of person missingness of 15% and measure missingness rate of 3.3%. The PROMIS person missingness at T2 (2 persons not completing 1 measure each) was 11.8% and measure missingness rate (2 persons not completing the peer relationship measure) of 2%. Individual item missingness at T1 was 1 pain item (.14%) and at T2 to 7 items (1.1%) across 4 different measures (4 items not answered by the same participant).

Coder agreement for the interview questions exceeded the 90% level. The most frequent responses to interview question 1 (‘a good day’) from 15 (88.2%) participants indicated favorable reactions to the new study drug including fewer side-effects and life interruptions than when on the frontline treatment. In response to the ‘bad day’ question, 7 participants (41.2%) reported AEs and 5 (29.4%) reported having to accommodate the protocol in terms of scheduled medication times and other interruptions to usual activities. Three reported being uneasy about the possible meaning of the pain they were experiencing. In response to the question about ‘any surprises’, 16 (94%) participants indicated that the experimental treatment was without surprises. Six (34%) described not liking to be at the hospital as it took valuable time away from family and friends (Table 2).

Six participants wanted doctors and nurses to know that overall being on the trial was a positive experience, while 3 others noted the inconvenience of following the protocol guidelines and uneasiness about waiting to learn of the treatment outcomes. Twelve participants indicated the PROMIS items captured what was important and 3 reported that the items were repetitive and too numerous. Six suggested items be added about being away from home, family and friends and the impact of treatment on family members; 2 recommended more items on anger and other feelings, 1 on appetite and 2 about how to answer questions that others posed to them about their illness (data not shown).

Aim 2: Bootstrap estimates of Cronbach's alpha by PROMIS measure at T1 indicated that five of six measures had an alpha value greater than the accepted cutoff point of 0.70 (Peer Relationships being slightly lower - 0.69). The alpha values were acceptable for all measures at T2 (Table 3). Paired t-test results and responsiveness indices (ESI, ESII, and SRD) indicated that average changes in the measures from T1 to T2 were small, and none of the corresponding paired t-tests was statistically significant (Table 4). The percentage of patients reaching the SRD criterion was low for all the scales, ranging from 5.3% (Depression, Anxiety, Fatigue, and Mobility) to 15.8% (Peer Relationship).

Table 3. Bootstrap estimate of Cronbach's Alpha by scale and assessment.

Scale	Alpha (SE)
Assessment 1 (N=20)	Assessment 2 (N=17)
Anxiety	0.72 (0.171)	0.73 (0.109)
Depression	0.88 (0.030)	0.78 (0.150)
Fatigue	0.79 (0.085)	0.87 (0.067)
Mobility	0.70 (0.203)	0.77 (0.197)
Pain	0.88 (0.059)	0.92 (0.030)
Peer relationship	0.69 (0.101)	0.86 (0.086)

Note.

The estimates were based on 1,000 bootstrap resamples.

Table 4. Responsiveness indices for PROMIS Measures.

Measure	Meant1 (SD)	Meant2 (SD)	Dift2-t1 (SD)	rxx	ESI	ESII	SRD
Depression	46.23(9.93)	43.98 (7.72)	-2.35 (6.01)	0.80	-0.23	-0.39	12.31
Anxiety	43.82 (7.93)	40.84 (6.00)	-2.40 (6.58)	0.63	-0.30	-0.36	13.37
Fatigue	49.71 (9.08)	51.19 (9.41)	1.46 (5.76)	0.82	0.16	0.25	10.67
Mobility	43.15 (6.96)	42.07 (7.94)	-0.59 (4.85)	0.80	-0.08	-0.12	8.65
Pain	45.41 (9.33)	46.88 (8.75)	1.12 (7.89)	0.64	0.12	0.14	15.51
Peer Relationship	48.64 (6.25)	49.97 (9.11)	1.21 (5.57)	0.79	0.19	0.22	7.94

Notes.

X_t1: Mean at Time 1.

X_t2: Mean at Time 2.

Dif_t2-_t1: Mean change from Time 1 to Time 2.

r_xx: Intraclass correlation coefficient.

ESI: Effective size I.

ESII: Effective size II.

SRD: smallest real difference (95% confidence limit, two-sided).

The number of AEs per participant from the CTCAE forms ranged from 0 to 23 with 7 participants having more than 8 and 12 having fewer. As theorized, mobility was significantly worse as the number of AEs increased, and as the number of AEs increased, fatigue was worse at both time points, though this did not reach statistical significance (Table 5). Using the patient-based MID value of +/- 3 change between the time points, 9 patients had MID changes in anxiety (6 improved), 10 in depression (7 improved), 9 patients in fatigue (3 improved), 9 in mobility (4 improved), 9 in pain (4 improved) and 10 in peer relationships (7 improved).

Table 5. Mean T-scores of PROMIS Measures by AE Frequency.

PROMIS T-score	Total AEs
	>8 Events (N=7)	≤8 Events (N=12)
	Mean (SD)	Mean (SD)
Baseline
Anxiety	43.6 (6.2)	44.5 (9.3)
Diff= -0.9, P=1.0000
Depression	46.0 (10.4)	47.2 (10.1)
Diff= -1.2, P=1.0000
Fatigue	56.0 (5.3)	46.0 (9.3)
Diff= 10.0, P=0.0940
Mobility	37.3 (6.3)	46.3 (5.3)
Diff= -9.0, P=0.0260
Pain	47.2 (12.2)	44.3 (8.1)
Diff= 2.9, P=0.9860
Peer relationship	49.7 (5.4)	48.8 (6.7)
Diff= 0.9 P=1.0000
Follow-up	Mean (SD)	Mean (SD)
Anxiety	41.2 (6.8)	40.6 (5.8)
Diff= 0.6, P=1.0000
Depression	45.8 (8.5)	42.7 (7.3)
Diff= 3.1, P=0.9220
Fatigue	56.1 (4.9)	47.8 (10.5)
Diff= 8.3, P=0.3080
Mobility	36.0 (7.5)	46.3 (5.1)
Diff= -10.3, P=0.0190
Pain	47.4 (9.9)	46.5 (8.4)
Diff= 0.9, P=1.0000
Peer relationship	52.2 (7.7)	48.5 (10.1)
Diff= 3.7, P=0.9360

Note.

^{>8 Events}: Total number of adverse events >8.

^{≤8 Events}: Total number of adverse events≤8.

^Diff: Mean difference between groups was tested using t-test based on bootstrap1000 resamples.

Discussion

This study allowed us to measure in a standardized approach the symptom and function reports of children and adolescents enrolled on Phase I/II cancer trials during the first course. Additionally, we solicited through interviews what aspects of symptoms and QoL were most important to them at this point in their lives and cancer treatment. Our enrollment findings indicated it is feasible to solicit PROs directly from participating children and adolescents as our enrollment rate (75.9%) exceeded our defined acceptable rate (70%). Refusals to enroll were more likely to come from parents (17.2%) than the eligible patients (8.3%) and only occurred when the self-report measures were not embedded in the clinical trial. Four of the five parents refusing expressed reluctance to experience an additional consent process. Our attrition rate (15%), met our goal of 15% or less and also supports feasibility.

Acceptability to the participants was also assessed using missinginess of data by person, measure and items. The number of persons not completing a PROMIS measure in this study (15% at T1 and 11.8% at T2) is somewhat higher than the 8% rate from our previous study during or following cure-oriented cancer treatment.²⁴ However, measure missingness rates in this study at T1 (3.3%) and T2 (2%) were lower than in the cross-sectional study (9.5%) as was individual item missingness (.14% at T1 and 1.1% at T2).^22,41 Importantly, these findings support the feasibility and acceptability to children and adolescents with incurable cancer to complete quantitative and qualitative measures regarding their symptom, function and QoL experiences while enrolled on a Phase I/II clinical trial.

We also documented through interviews the ability of children and adolescents to speak to their overall QoL while enrolled on an early stage trial. The 16 respondents to the interview questions indicated that for the most part, being on the study was not as demanding as being on frontline therapy protocols. One acknowledged worrying if the drug was effective but emphasized having no regrets about participating. Half described AEs that occasionally prevented them from engaging in preferred activities. Half indicated the items on the PROMIS measures represented what was important to them; others suggested additions about sports, appetite, impact of treatment on family and being away from home. In summary, the PROMIS measures used in this study captured most of what was important to children and adolescents enrolled on a Phase I/II clinical trial for incurable cancer with the exception of treatment impact on family and time with family and friends.

This study represents the first time that the PROMIS pediatric measures have been completed by children and adolescents with incurable cancer on a Phase I/II clinical trial. Importantly, the internal consistency estimates indicate that the measures were reliable at both time points in this patient group. Estimates of validity are more limited in the study findings most likely related to the small sample size, although as theorized, mobility was statistically significantly better at both data points when fewer AEs were experienced. A similar trend was noted for fatigue with lower fatigue associated with fewer AEs.

Important study limitations exist, particularly the small sample size that influenced validity tests. The 20 months needed to identify 29 eligible participants across the four study settings conveys the need to include sufficient settings in a future extension of this work. Another limitation is the parent refusal rate in which four of five refusing parents attributed their decline to not wanting to experience a second consent process after completing the clinical trial consent process. There were no parent or child refusals when the PROs were embedded. We recommend that future pediatric oncology Phase I/II trials embed PROs and allow age-eligible participants to choose whether to complete the embedded measures. Embedded PROs can be examined in relation to clinical and disease variables, making these PROs more clinically interpretable. Embedding will also allow pediatric oncology clinicians to better understand the full impact of novel therapies and to minimize or prevent likely toxicities. Results of this study confirm the feasibility and acceptability of soliciting and documenting these endpoints in pediatric oncology Phase I/II clinical trials.

Conclusion

Only by measuring and understanding self-reported symptoms and function in children and adolescents with incurable cancer can we adequately address threats to their QoL and improve symptom control and supportive care. By embedding PROs such as PROMIS in experimental clinical trials we provide voice to enrolled children and adolescents to inform clinicians about their subjective treatment experiences.