Abstract
BACKGROUND:
Approximately 50% of children with cancer in the United States who are aged <15 years receive primary treatment on a therapeutic clinical trial. To the authors’ knowledge, it remains unknown whether trial enrollment has a clinical benefit compared with the best alternative standard therapy and/or off trial (ie, clinical trial effect). The authors conducted a retrospective matched cohort study to compare the morbidity and mortality of pediatric patients with cancer who are treated on a phase 3 clinical trial compared with those receiving standard therapy and/or off trial.
METHODS:
Subjects were aged birth to 19 years; were diagnosed between 2000 and 2010 with acute lymphocytic leukemia (ALL), acute myeloid leukemia (AML), rhabdomyosarcoma, or neuroblastoma; and had received initial treatment at the Children’s Hospital of Philadelphia. On-trial and off-trial subjects were matched based on age, race, ethnicity, a diagnosis of Down syndrome (for patients with ALL or AML), prognostic risk level, date of diagnosis, and tumor type.
RESULTS:
A total of 428 participants were matched in 214 pairs (152 pairs for ALL, 24 pairs for AML, 32 pairs for rhabdomyosarcoma, and 6 pairs for neuroblastoma). The 5-year survival rate did not differ between those treated on trial versus those treated with standard therapy and/or off trial (86.9% vs 82.2%; P = .093). On-trial patients had a 32% lower odds of having worse (higher) mortality-morbidity composite scores, although this did not reach statistical significance (odds ratio, 0.68; 95% confidence interval, 0.45–1.03 [P = .070]).
CONCLUSIONS:
There was no statistically significant difference in outcomes noted between those patients treated on trial and those treated with standard therapy and/or off trial. However, in partial support of the clinical trial effect, the results of the current study indicate a trend toward more favorable outcomes in children treated on trial compared with those treated with standard therapy and/or off trial. These findings can support decision making regarding enrollment in pediatric phase 3 clinical trials.
Keywords: clinical trial, outcomes assessment, pediatric oncology, retrospective studies, trial effect
INTRODUCTION
Cancer remains the leading cause of disease-related death for children. Approximately 17.1 per 100,000 children aged <20 years are diagnosed with cancer each year in the United States, with 1 in 8 children with cancer dying of their disease.1,2 National Cancer Institute–sponsored/Children’s Oncology Group (COG) clinical trials have contributed to increased cure rates for children with cancer, with approximately 100 COG research trials (85% of which are therapeutic trials) sponsored each year.1,2 Approximately one-third to 86% of children in the United States aged <15 years who are newly diagnosed with cancer are treated on a therapeutic clinical trial.3–5 In contrast, <20% of adolescents aged 15 to 20 years are treated on a clinical trial, a differential that may contribute to the higher mortality and morbidity noted among adolescents and young adults (AYAs) with cancer compared with younger children.3,6
Although clinical trials have been central to the identification of long-term, continual improvements in pediatric cancer treatments, there are questions regarding whether individual children participating in cancer clinical trials have improved outcomes compared with those who receive treatment outside of clinical trials regardless of trial arm.7 The majority of studies have reported no mortality benefit to trial participation, but results have been mixed with limitations noted, including the ability to adequately adjust for potential confounders.6,8–14 Proposed mechanisms of the trial effect are multifactorial and include the benefits attributed to the experimental therapy, a monitoring effect, selection bias for healthier patients to enter a trial, behavioral changes on the part of oncology providers or the patient and family, and an outcomes assessment bias.6,15
In the current study, matching analytic techniques were used to compare a cohort of pediatric patients receiving primary therapy on trial compared with standard therapy and/or off trial on the outcomes of overall survival and morbidity. We chose 4 tumor types, acute lymphocytic leukemia (ALL), acute myeloid leukemia (AML), rhabdomyosarcoma (RMS), and neuroblastoma (NB), that are relatively common and had adequate numbers of patients participating in clinical trials to support a matched comparison of outcomes.1,2 Standard therapy and/or off trial was by protocol, typically based on the treatment arm of a prior, specified trial that established the efficacy for standard care.
MATERIALS AND METHODS
Study Population and Study Site
The study protocol was approved by the institutional review boards of the Children’s Hospital of Philadelphia (CHOP) and the University of Pennsylvania. The CHOP clinical trial database was used to identify eligible subjects. The CHOP tumor registry was used to confirm whether subjects received initial treatment on a phase 3 clinical trial (on trial) or standard therapy and/or off trial. Participation in a clinical trial was confirmed by chart review. Inclusion criteria were: 1) a cancer diagnosis between birth and age 19 years; 2) diagnosis between 2000 and 2010; 3) tumor type of ALL, AML, RMS, or NB; and 4) diagnosis at a time when a phase 3 clinical trial for the tumor type and risk level was open at CHOP. Exclusion criteria included having a prior malignancy or receiving initial treatment at an outside hospital. Patients with the myeloproliferative neoplasms of juvenile myelomonocytic leukemia, myelodysplastic syndrome, polycythemia vera, and refractory anemia were excluded. Those with mixed lineage leukemia and acute biphenotypic leukemia diagnoses also were excluded.
All phase 3 clinical trials that were open at CHOP during the period between 2000 and 2010, including opening and closing dates, were identified using the COG website. Potential subjects were evaluated against this information, as well as tumor registry data, to determine whether an appropriate trial for their tumor type was open at CHOP at the time of diagnosis. The CHOP Cancer Survivorship Program has developed guidelines for the surveillance of late effects for each tumor type and treatment approach. The investigative team reviewed CHOP guidelines to determine the classification of late effects identified through chart abstraction into affected clinical systems. This classification also was reviewed by members of our research team who are clinical experts (C.B. and R.B.) for each tumor type included in the study.
File Review
Chart reviews were conducted from March 2015 to April 2018 to ascertain sociodemographic and clinical variables to be included in the match and the occurrence of late effects. Data were censored at the time of chart review. Chart review was conducted sequentially in those patients with ALL, AML, RMS, and NB to minimize variation in the dates of data censoring. Mortality was assessed using data from the CHOP tumor registry and confirmed by chart review. If vital status in the tumor registry was shown as “alive” at the time of chart review, the status of “alive” was noted as 30 days prior to chart review. Data extraction forms underwent double review with additional quality checks to ensure accuracy. Data from approximately 25% of charts for each tumor type were double entered into Research Electronic Data Capture (REDCap) and checked for quality assurance. Discrepancies between entries were resolved by the principal investigators (L.P.B. and M.M.S.). Transition to the use of electronic medical records (EMRs) occurred in 2008 at CHOP. However, clinical data from the tumor registry and medical chart were entered retrospectively into the EMR for patients undergoing treatment, resulting in a primarily EMR review for those diagnosed from 2003 onward.
Matching Process
Factors extracted from chart review for the match included race (black or African American, white, or other), ethnicity (Hispanic or non-Hispanic), age, tumor-specific prognostic risk, diagnosis of Down syndrome (present or absent for patients with AML or ALL), site of the tumor (RMS), and diagnostic year. In consultation with our clinical experts (C.B. and R.B.) and based on both clinical knowledge and the published literature,16 we chose matching variables that met one of the following criteria: 1) clinical risk factors associated with the morbidity or mortality outcomes for each tumor type; or 2) demographic factors that may be associated with enrollment in clinical trials. Risk categories for each tumor type were abstracted from chart review, as listed in Table 1. We performed optimal subset matching to minimize the distance between trial patients and those treated off trial with regard to the matching variables. Matching was done using R statistical software (version 3.2.1) using the “design match” package (version 0.2.0).17,18 To assess the covariate balance of the match, we used the Wilcoxon rank sum test for continuous variables and the Fisher exact test for binary variables.19 We assessed the mean standardized difference after matching in units of standard deviations (SDs), with a goal of achieving SDs <0.20.20,21 A propensity score to be treated on trial was determined for each tumor type using variables in the match.22 All matching was performed first without viewing outcomes.
TABLE 1.
Matching Table for the Cohort Study
| Variable | All On-Trial Cases N = 432 | Matched On-Trial Cases N = 214 | Matched Off-Trial Controls N = 214 | All Off-Trial Controls N = 228 | Standardized Difference Before Matcha | P Before Matchb | Standardized Difference After Matcha | P After Matchb |
|---|---|---|---|---|---|---|---|---|
| Demographics | ||||||||
| Mean age, y | 5.9 | 6.5 | 6.9 | 7.2 | −0.24 | .0562 | −0.08 | .9070 |
| Mean year | 2004.7 | 2006.4 | 2006.8 | 2006.9 | −0.75 | .0000 | −0.14 | .2255 |
| Years 2000–2005 (ALL, AML, RMS) | 236 (58.7%) | 65 (31.3%) | 62 (29.8%) | 64 (28.8%) | 0.63 | .0000 | 0.03 | .8314 |
| Years 2006–2010 (ALL, AML, RMS) | 166 (41.3%) | 143 (68.8%) | 146 (70.2%) | 158 (71.2%) | −0.63 | .0000 | −0.03 | .8314 |
| Years 2000–2008 (neuroblastoma) | 13 (43.3%) | 2 (33.3%) | 3 (50.0%) | 3 (50.0%) | −0.13 | 1.0000 | −0.32 | 1.0000 |
| Years 2009–2010 (neuroblastoma) | 17 (56.7%) | 4 (66.7%) | 3 (50.0%) | 3 (50.0%) | 0.13 | 1.0000 | 0.32 | 1.0000 |
| Black race | 42 (9.7%) | 38 (17.8%) | 43 (20.1%) | 49 (21.5%) | −0.33 | .0000 | −0.07 | .6218 |
| White race | 358 (82.9%) | 161 (75.2%) | 151 (70.6%) | 159 (69.7%) | 0.31 | .0001 | 0.11 | .3277 |
| Other race | 32 (7.4%) | 15 (7.0%) | 20 (9.4%) | 21 (9.2%) | −0.07 | .4523 | −0.08 | .4809 |
| Male sex | 201 (46.5%) | 94 (43.9%) | 92 (43.0%) | 97 (42.5%) | 0.08 | .3657 | 0.02 | .9223 |
| Hispanic ethnicity | 27 (6.3%) | 14 (6.5%) | 18 (8.4%) | 20 (8.8%) | −0.10 | .2654 | −0.07 | .5820 |
| ALL | ||||||||
| No. | 306 | 152 | 152 | 156 | — | — | — | — |
| Age ≤3 mo | 2 (0.7%) | 2 (1.3%) | 5 (3.3%) | 5 (3.2%) | −0.19 | .0465 | −0.14 | .4477 |
| Age >3 to <12 mo | 4 (1.3%) | 3 (2.0%) | 4 (2.6%) | 5 (3.2%) | −0.13 | .1729 | −0.04 | 1.0000 |
| Age <12 mo | 6 (2.0%) | 5 (3.3%) | 9 (5.9%) | 10 (6.4%) | −0.22 | .0274 | −0.13 | .4128 |
| Age 1–2 y | 72 (23.5%) | 30 (19.7%) | 27 (17.8%) | 27 (17.3%) | 0.15 | .1500 | 0.05 | .7691 |
| Age 3–9 y | 163 (53.3%) | 77 (50.7%) | 70 (46.1%) | 70 (44.9%) | 0.17 | .0949 | 0.09 | .4911 |
| Age 10–19 y | 65 (21.2%) | 40 (26.3%) | 46 (30.3%) | 49 (31.4%) | −0.23 | .0222 | −0.09 | .5245 |
| Mean risk score | 3.3 | 3.5 | 3.5 | 3.5 | −0.28 | .0038 | 0.00 | 1.0000 |
| Down syndrome | 3 (1.0%) | 3 (2.0%) | 3 (2.0%) | 5 (3.2%) | −0.16 | .1265 | 0.00 | 1.0000 |
| AML | ||||||||
| No. | 49 | 24 | 24 | 27 | — | — | — | — |
| Age <1 y | 8 (16.3%) | 4 (16.7%) | 3 (12.5%) | 3 (11.1%) | 0.15 | .7368 | 0.12 | 1.0000 |
| Age 1–2 y | 7 (14.3%) | 6 (25.0%) | 8 (33.3%) | 9 (33.3%) | −0.45 | .0769 | −0.20 | .7516 |
| Age 3–10 y | 18 (36.7%) | 5 (20.8%) | 4 (16.7%) | 4 (14.8%) | 0.51 | .0638 | 0.10 | 1.0000 |
| Age 11–19 y | 16 (32.7%) | 9 (37.5%) | 9 (37.5%) | 11 (40.7%) | −0.17 | .6172 | 0.00 | 1.0000 |
| Low risk | 3 (6.1%) | 3 (12.5%) | 3 (12.5%) | 4 (14.8%) | −0.28 | .2377 | 0.00 | 1.0000 |
| Intermediate risk | 18 (36.7%) | 13 (54.2%) | 13 (54.2%) | 14 (51.9%) | −0.30 | .2315 | 0.00 | 1.0000 |
| High risk | 0 (0.0%) | 0 (0.0%) | 0 (0.0%) | 0 (0.0%) | 0.00 | 1.0000 | 0.00 | 1.0000 |
| Missing risk | 28 (57.1%) | 8 (33.3%) | 8 (33.3%) | 9 (33.3%) | 0.49 | .0577 | 0.00 | 1.0000 |
| Down syndrome | 4 (8.2%) | 3 (12.5%) | 3 (12.5%) | 4 (14.8%) | −0.21 | .4439 | 0.00 | 1.0000 |
| RMS | ||||||||
| No. | 47 | 32 | 32 | 39 | — | — | — | — |
| Extremity site | 12 (25.5%) | 5 (15.6%) | 5 (15.6%) | 6 (15.4%) | 0.25 | .2954 | 0.00 | 1.0000 |
| Trunk site | 5 (10.6%) | 5 (15.6%) | 5 (15.6%) | 6 (15.4%) | −0.14 | .5359 | 0.00 | 1.0000 |
| Head and neck site | 15 (31.9%) | 12 (37.5%) | 12 (37.5%) | 17 (43.6%) | −0.24 | .3703 | 0.00 | 1.0000 |
| Low risk | 27 (57.5%) | 15 (46.9%) | 15 (46.9%) | 16 (41.0%) | 0.33 | .1935 | 0.00 | 1.0000 |
| Intermediate risk | 10 (13.0%) | 6 (15.8%) | 6 (15.8%) | 6 (15.4%) | −0.01 | 1.0000 | 0.00 | 1.0000 |
| High risk | 24 (51.1%) | 20 (62.5%) | 20 (62.5%) | 21 (53.9%) | −0.06 | 0.8311 | 0.00 | 1.0000 |
| Missing risk | 1 (2.1%) | 1 (3.1%) | 1 (3.1%) | 6 (15.4%) | −0.48 | 0.0431 | 0.00 | 1.0000 |
| Neuroblastoma | ||||||||
| No. | 30 | 6 | 6 | 6 | — | — | — | — |
| Intermediate risk | 13 (43.3%) | 2 (33.3%) | 2 (33.3%) | 2 (33.3%) | 0.20 | 1.0000 | 0.00 | 1.0000 |
| High risk | 17 (56.7%) | 4 (66.7%) | 4 (66.7%) | 4 (66.7%) | −0.20 | 1.0000 | 0.00 | 1.0000 |
Abbreviations: ALL, acute lymphocytic leukemia; AML, acute myeloid leukemia; RMS, rhabdomyosarcoma.
Standardized difference in units of before-match standard deviation and after-match standard deviation.
P values were calculated using group tests, with the Fisher exact test used for binary variables and the Wilcoxon rank sum test used for continuous variables. The quality of the matching for those variables that were desired to be closely matched was excellent, with the majority of standardized differences <0.1 and all <0.2.
Statistical Analysis
Survival
The primary endpoint was overall survival, defined as the time from diagnosis to death from any cause. The primary analysis was conducted combining all tumor types with secondary analyses conducted on the individual tumor types. Our a priori power calculation determined we would have 80% power to detect a hazard ratio (HR) of 2 with 236 matched pairs if the baseline mortality rate was 15%. Survival differences between patients treated on trial and off trial were tested using the paired Prentice-Wilcoxon test, a nonparametric test on ranks of survival times.23,24 A secondary analysis was conducted excluding the 6 NB matched pairs. Survival differences also were tested at 1 year, 3 years, and 5 years after diagnosis using a bootstrap to calculate the 95% confidence interval (95% CI) and its P value. HRs and 95% CIs based on the Cox model were determined for the primary analysis and for the larger groups by tumor type if the assumptions for proportional hazards were met (ALL, AML, and RMS). A stability analysis was conducted using the composite outcomes of mortality or disease recurrence.
Late effects
The primary outcome of the current analysis was a scale representing the general burden of morbidity and mortality including late effects among these groups. Prior to coding, investigators prepared a list of late effects associated with each tumor type. Late effects identified in medical chart review were categorized by clinical system for each subject. Across all identified late effects, there were 14 systems. Because the severity of late effects was not systematically available in the medical record, the total number of clinical systems in which a subject experienced a late effect was determined. Based on the distribution of this outcome (which ranged from 0–5), we categorized morbidity accounting for mortality as follows: 0 to 1 organ systems versus ≥2 organ systems versus death. We then tested whether the relative odds of being in a worse category (higher score) were greater among patients treated on trial versus those treated with standard therapy and/or off trial using the method suggested by McCullagh for paired comparisons of ordinal categorical data.25 A logistic model for paired comparisons with ordered categorical data, based on the delta statistic, was used. When the exponentiated delta statistic is >1, it suggests increased relative odds of a worse outcome (a higher mortality-morbidity composite score) for the patients treated with standard therapy and/or off trial compared with those treated on trial. If the delta statistic is <1, it suggests a lower odds of a worse outcome in the patients treated with standard therapy and/or off trial compared with those treated on trial. A stability analysis was conducted using the composite outcomes of disease recurrence or death.
RESULTS
A review of the CHOP tumor registry led to the identification of 904 patients with the following tumor types: ALL (479 patients), AML (125 patients), RMS (95 patients), and NB (205 patients). Of these, 689 patients were eligible for matching: 462 patients with ALL, 75 patients with AML, 86 patients with RMS, and 66 patients with NB. After the matching procedure, there were 428 successfully matched subjects resulting in 214 matched pairs: 152 matched ALL pairs, 24 matched AML pairs, 32 matched RMS pairs, and 6 matched NB pairs (Figs. 1–4). The median follow-up for the patients treated on trial versus those treated off trial was 9.4 years and 9.2 years, respectively. Of the 428 patients in the current analysis, there were 78 who developed disease recurrence, 48 of whom died and 30 of whom were alive at the time of censoring. There were 75 deaths reported in the cohort.
FIGURE 1.
Consolidated Standards Of Reporting Trials (CONSORT) diagram for acute lymphocytic leukemia (ALL). CHOP indicates Children’s Hospital of Philadelphia.
FIGURE 4.
Consolidated Standards Of Reporting Trials (CONSORT) diagram for neuroblastoma (NB). CHOP indicates Children’s Hospital of Philadelphia.
Survival Analysis
Kaplan-Meier curves demonstrated no difference in survival between those patients treated on trial versus those treated with standard therapy and/or off trial, with a paired Prentice-Wilcoxon P of .0975 (Fig. 5). The survival HR for treatment on trial versus off trial was 0.69 (95% CI, 0.42–1.133; P = .142). In the primary analysis, the 1-year mortality rate was lower for the patients treated on trial (3.7%) versus those treated off trial (7.9%) (P = .018) and for the ALL subgroup (3.3% for patients treated on trial vs 7.9% for patients treated off trial; P = .024) but did not persist at 3 years (10.7% for patients treated on trial vs 14.0% for patients treated off trial; P = .216) or at 5 years (13.1% for patients treated on trial vs 17.8% for patients treated off trial; P = .110) after diagnosis. These findings persisted on a sensitivity analysis that excluded patients with NB (Fig. 5) (see Supporting Figs. 1–3). It is interesting to note that the HR for the ALL group at 5 years was 0.59 (95% CI, 0.32–1.1; P = .097). In the stability analysis when using a composite outcome of disease recurrence or death, the results were similar to those of the primary analysis, with a HR of 0.74 (95% CI, 0.49–1.11; P = 0.146 (see Supporting Fig. 4).
FIGURE 5.
Survival analysis of patients with the 4 tumor types who were treated on trial versus standard therapy (ST) and/or off trial. The tumor types were acute lymphocytic leukemia, acute myeloid leukemia, rhabdomyosarcoma, or neuroblastoma. The Cox proportional hazard ratio was 0.69 (95% confidence interval, 0.42–1.13; P = .142).
Comparison of Late Effects Between Treatment Groups
Patients experienced late effects subsequent to treatment involving from 0 to 5 organ systems. Late effects for patients were most common in the following systems: the musculoskeletal, endocrine, and cognitive systems for patients with ALL; the endocrine and cardiac systems for patients with AML; the musculoskeletal system for patients with RMS; and the auditory, cognitive, and endocrine systems for patients with NB (Table 2). There was no significant difference in the distribution of the composite late effects or death outcomes noted between those treated on trial compared with those treated off trial, although a trend was observed toward improved outcomes in the on-trial group (odds ratio, 1.46; 95% CI, 0.97–2.20 [P = .070]) (Table 3). In the stability analysis when using a composite outcome of disease recurrence or death, the results were found to be similar (odds ratio, 1.45; 95% CI, 0.98–2.15 [P = .062]) (see Supporting Table 1).
TABLE 2.
Percentage With Late Effects in Each System by Clinical Trial Participation and Tumor Type
| ALL N = 304 | AML N = 48 | RMS N = 64 | NB N = 12 | |||||
|---|---|---|---|---|---|---|---|---|
|
|
|
|
|
|||||
| On Trial | Off Trial | On Trial | Off Trial | On Trial | Off Trial | On Trial | Off Trial | |
| CNS | 5.9 | 6.6 | 4.2 | 0.0 | 0.0 | 3.1 | 0.0 | 16.7 |
| Auditory | 4.6 | 0.7 | 8.3 | 12.5 | 3.2 | 3.1 | 50.0 | 33.3 |
| Eye/visual | 5.3 | 2.6 | 12.5 | 4.2 | 12.5 | 15.6 | 0.0 | 0.0 |
| Cognitive | 10.5 | 11.2 | 8.3 | 0.0 | 6.3 | 6.3 | 16.7 | 33.3 |
| Endocrine | 21.1 | 18.4 | 20.8 | 16.7 | 12.5 | 6.3 | 16.7 | 50.0 |
| Cardiac | 4.6 | 4.6 | 12.5 | 16.7 | 0.0 | 0.0 | 0.0 | 0.0 |
| Secondary neoplasms | 0.7 | 1.3 | 0.0 | 0.0 | 9.4 | 9.4 | 0.0 | 0.0 |
| Psychological | 5.9 | 8.6 | 4.2 | 12.5 | 3.1 | 9.4 | 0.0 | 16.7 |
| GI/GU | 3.9 | 1.3 | 4.2 | 16.7 | 0.0 | 3.1 | 0.0 | 16.7 |
| Gynecologic | 0.0 | 0.0 | 4.2 | 8.3 | 0.0 | 0.0 | 0.0 | 0.0 |
| Musculoskeletal | 20.4 | 19.7 | 4.2 | 8.3 | 21.9 | 25.0 | 7.1 | 16.7 |
| Dental | 0.0 | 0.7 | 0.0 | 0.0 | 3.1 | 6.3 | 0.0 | 16.7 |
| Kidney | 2.0 | 0.7 | 8.3 | 16.7 | 0.0 | 0.0 | 0.0 | 16.7 |
| Chronic GVHD | 0.7 | 2.0 | 8.3 | 8.3 | 0.0 | 0.0 | 0.0 | 0.0 |
Abbreviations: ALL, acute lymphocytic leukemia; AML, acute myeloid leukemia; CNS, central nervous system; GI, gastrointestinal; GU, genitourinary; GVHD, graft-versus-host disease; NB, neuroblastoma; RMS, rhabdomyosarcoma.
The median number of clinical systems that incurred a late effect across all tumor types was 1 (range, 0–5) for both subjects treated on trial and off trial with standard therapy.
TABLE 3.
Differences in a Composite Late Effects and Death Outcomes by Off-Trial and On-Trial Group Matched Pairs
| Standard Therapy Off Trial | |||||
|---|---|---|---|---|---|
|
|
|||||
| 0 to 1 Systems Affected | 2 to 5 Systems Affected | Deaths | Total | ||
| On Trial | 0–1 systems affected | 91 | 28 | 26 | 145 |
| 2–5 systems affected | 19 | 11 | 7 | 37 | |
| Deaths | 18 | 4 | 10 | 32 | |
| Total | 128 | 43 | 43 | 214 | |
Late effects and death were categorized as: 1) 0 to 1 organ systems; 2) greater than or equal to 2 organ systems; and 3) death. The relative odds that the on-therapy group had lower (better) morbidity-mortality scores compared with the standard therapy group was 1.46 (95% confidence interval, 0.97–2.20; P = .070).
DISCUSSION
The current study addressed the question of whether participation in a clinical trial, regardless of study arm assignment, has a benefit for patients in terms of long-term morbidity or mortality. We identified pediatric and AYA patients with cancer who were aged birth to 19 years at the time of diagnosis; who had a diagnosis of ALL, AML, RMS, or NB; who received primary treatment at CHOP; and who were treated at a time during which there was an open phase 3 clinical trial for their primary diagnosis. We reported that the 1-year overall survival rate was higher for those treated on trial compared with those treated off trial in the primary analysis and in the ALL subgroup, but this finding did not persist at 3 years or 5 years after diagnosis. We further evaluated the difference in a composite measure of late effects or death and found no statistically significant difference between those treated on trial compared with those treated off trial, but did note a trend in favor of the on-trial group. We found similar results for death or disease recurrence combined with late effects.
The results of the current study build on prior studies regarding the trial effect in several ways. To our knowledge, the current study is the first to combine tumor types to evaluate a trial effect. The use of matching methods to control for confounders, including age at diagnosis, ethnicity, and prognostic risk level, applied a rigorous methodological approach that supported this type of analysis. Combining tumor types increased the sample size and generalizability of the study findings for a range of pediatric and AYA patients. Although the current study did not demonstrate a positive trial effect at 3 years or 5 years, we observed beneficial trends for both overall survival and a composite of late effects or death in a direction favoring the patients treated on trial compared with those treated with standard therapy and/or off trial. Observed trends are hypothesis-generating only; however, these findings may be considered in light of biases in the design of the current study that favor a null effect. Subjects in both groups were treated at a tertiary care, National Cancer Institute–designated comprehensive center of excellence. This may have decreased one proposed mechanism of a trial effect, namely increased monitoring during the treatment period. Furthermore, the presence of a trial effect may vary by age; adolescents are less likely to receive care at a COG institution, at which patients have access to pediatric protocols. Studies have indicated improved overall survival in AYAs who are treated in pediatric versus adult trial protocols and centers.26 We were unable to measure an interaction between trial enrollment and age in the current study because age was used as a matching variable in the analysis.
The results of the current study build upon previous studies evaluating the trial effect that have demonstrated inconsistent findings. A 2008 Cochrane review of adult patients across a range of study designs and clinical conditions (oncology, cardiology, other internal medicine areas, obstetrics/gynecology, psychology, and pediatrics) failed to find a trial effect for morbidity or mortality outcomes.8 Two recent studies of adults with esophageal cancer14 or who underwent bone marrow transplantation27 failed to find a trial effect. A recent study of trials from the SWOG national clinical trials consortium database comparing outcomes among patients with cancer who are treated in and out of clinical trials found trial participation to be associated with improved 1-year survival in 9 of 10 poor-prognosis studies but to have no effect among 11 good-prognosis studies.28 A positive trial effect was reported in a study of patients with castrate-resistant prostate cancer.13 Earlier studies of pediatric patients with ALL suggested a positive trial effect but were limited by their failure to control for confounding factors.6 A more recent retrospective cohort study controlled for potential confounding factors and failed to find a significant association between trial enrollment and event-free survival.9 Two studies using a national Canadian database reported mixed results among patients with ALL and AML.11 A subsequent study using a similar database and analytic plan for patients with AML found no association between clinical trial enrollment and either event-free survival or overall survival.10 Finally, in a study from the United Kingdom, patients aged 15 to 24 years who were diagnosed with ALL and treated on trial demonstrated a 17.9% improved 2-year survival rate and an 8.9% better 1-year survival rate compared with those treated off trial.29
Three primary mechanisms for a positive trial effect have been proposed: 1) a treatment effect; 2) a monitoring effect; and 3) a behavioral effect.6 Consideration of these mechanisms offers insights regarding mixed findings in the literature and the current study. A treatment effect is attributed to the incremental benefit of the experimental treatment arm of the clinical trial. Studies of the trial effect typically include a range of clinical trials that vary with regard to the efficacy of new treatments being evaluated. In addition, similar to the current study, analyses typically do not include randomization assignment, thereby limiting the ability to detect the percentage of a trial effect attributed to treatment efficacy.
Increased monitoring is the second proposed mechanism for a positive trial effect. Clinical trials are more likely to be available in tertiary care hospitals, such as in the current study, in which resources such as multidisciplinary teams may increase the quality and monitoring of the care provided. In contrast, many patients, particularly AYAs, are treated in community centers, in which resources and monitoring may be less available.30–32 In addition, this factor could vary with prognosis and the treatment course of various cancer types. For example, patients with AML have a worse prognosis and are more likely to have extended hospital stays for treatment compared with patients with ALL,10,11 thereby mitigating the benefit of increased monitoring in a clinical trial and therefore any positive trial effect observed.
Finally, increased engagement with the medical care and research teams in a clinical trial may have a carryover effect in the years of survivorship, leading to healthier behaviors and improved outcomes.33 Some late effects, including cardiovascular and psychological outcomes, may be modifiable through engagement and behavior changes. The current study focused on long-term survivors because the study design ensured at least 5 years of follow-up for all subjects. Longer follow-up is needed to capture the benefits of clinical trial participation for lifestyle behaviors that may minimize late effects.
Patients and families have many reasons for participating in clinical trials, including maintaining hope, receiving direct medical benefits, and helping future patients.34–36 Barriers to trial participation that have been cited include the fear of side effects, prolonged hospitalizations, and discomfort with experimentation.36 We believe the results of the current study provide valuable information that can inform clinical trial decision making for pediatric and AYA patients with cancer and their families as they balance the perceived risks of participation with benefits when considering enrollment in clinical trials.37,38
The current study has some limitations. First, it consisted of only 214 matched pairs, which limited our ability to detect statistical significance despite the trends observed. Second, the current study was a single-institution study and this may have limited the variability in care received for those treated on trial versus those treated off trial. A single-institution experience in which standardized assessment, monitoring, and supportive care guidelines are used could mitigate differences on a population level between those treated on or off trial. Third, despite matching, there could be unobserved confounders. Fourth, the current study was performed during an era of transition from paper to electronic health records and some patients treated in the early 2000s were excluded due to the inability to retrieve a complete medical record. Finally, we did not have access to information regarding the timing, severity, or grade of late effects, which necessitated that we create a composite outcome that scaled outcomes from death versus more organ system involvement versus less organ system involvement. Although we matched based on year of diagnosis and censored all data at the time of chart review, cases may have varied to some degree with regard to length of follow-up. However, this approach also had some strengths. It allowed us to formally compare matched pairs on 3 ordered outcome states using the delta statistic,25 thereby reducing bias compared with an unmatched analysis.39
The results of the current study indicated that in a single-institution experience, there was no significant difference in the overall 5-year survival or the occurrence of late effects among pediatric and AYA patients with cancer who were enrolled in a clinical trial compared with those treated off trial. However, we observed trends in favor of better clinical outcomes for the on-trial group compared with the group treated off trial. To the best of our knowledge, the current study is the first to use matching methods to compare outcomes between those treated on trial and those treated off trial and across pediatric tumor types. Studies of the trial effect, including the current study, combine outcomes across many trials, and therefore do not make a statement regarding the benefit of any specific trial. Nevertheless, these findings can help patients and their families to weigh the risks and benefits in clinical trial decision making. Further studies are needed to elucidate and understand the mechanisms of individual-level and population-level outcomes associated with enrollment in cancer clinical trials.
Supplementary Material
FIGURE 2.
Consolidated Standards Of Reporting Trials (CONSORT) diagram for acute myeloid leukemia (AML). CHOP indicates Children’s Hospital of Philadelphia.
FIGURE 3.
Consolidated Standards Of Reporting Trials (CONSORT) diagram for rhabdomyosarcoma (RMS). CHOP indicates Children’s Hospital of Philadelphia.
FUNDING SUPPORT
Supported by the Centers of Disease Control and Prevention Cooporative agreement number 1U48DP005053.
Footnotes
CONFLICT OF INTEREST DISCLOSURES
Shawna R. Calhoun is an employee of and owns stock in Merck Sharp & Dohme Corporation, a subsidiary of Merck & Company Inc (Kenilworth, New Jersey). The other authors made no disclosures.
REFERENCES
- 1.CURESEARCH for Children’s Cancer. Childhood cancer statistics. Published 2019. Accessed February 5, 2020. https://curesearch.org/Childhood-Cancer-Statistics [Google Scholar]
- 2.Ward E, DeSantis C, Robbins A, Kohler B, Jemal A. Childhood and adolescent cancer statistics, 2014. CA Cancer J Clin. 2014;64:83–103. [DOI] [PubMed] [Google Scholar]
- 3.Tai E, Beaupin L, Bleyer A. Clinical trial enrollment among adolescents with cancer: supplement overview. Pediatrics. 2014;133(suppl 3):S85–S90. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Shaw PH, Ritchey AK. Different rates of clinical trial enrollment between adolescents and young adults aged 15 to 22 years old and children under 15 years old with cancer at a children’s hospital. J Pediatr Hematol Oncol. 2007;29:811–814. [DOI] [PubMed] [Google Scholar]
- 5.Aristizabal P, Singer J, Cooper R, et al. Participation in pediatric oncology research protocols: racial/ethnic, language and age-based disparities. Pediatr Blood Cancer. 2015;62:1337–1344. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Peppercorn JM, Weeks JC, Cook EF, Joffe S. Comparison of outcomes in cancer patients treated within and outside clinical trials: conceptual framework and structured review. Lancet. 2004;363:263–270. [DOI] [PubMed] [Google Scholar]
- 7.Adamson PC. Improving the outcome for children with cancer: development of targeted new agents. CA Cancer J Clin. 2015;65:212–220. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Vist GE, Bryant D, Somerville L, Birminghem T, Oxman AD. Outcomes of patients who participate in randomized controlled trials compared to similar patients receiving similar interventions who do not participate. Cochrane Database Syst Rev. 2008;(3):MR000009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Koschmann C, Thomson B, Hawkins DS. No evidence of a trial effect in newly diagnosed pediatric acute lymphoblastic leukemia. Arch Pediatr Adolesc Med. 2010;164:214–217. [DOI] [PubMed] [Google Scholar]
- 10.Truong TH, Pole JD, Barber R, et al. Enrollment on clinical trials does not improve survival for children with acute myeloid leukemia: a population-based study. Cancer. 2018;124:4098–4106. [DOI] [PubMed] [Google Scholar]
- 11.Strahlendorf C, Pole JD, Barber R, et al. Enrolling children with acute lymphoblastic leukaemia on a clinical trial improves event-free survival: a population-based study. Br J Cancer. 2018;118:744–749. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Pulte D, Gondos A, Redaniel MT, Brenner H. Survival of patients with chronic myelocytic leukemia: comparisons of estimates from clinical trial settings and population-based cancer registries. Oncologist. 2011;16:663–671. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Koo KC, Lee JS, Kim JW, et al. Impact of clinical trial participation on survival in patients with castration-resistant prostate cancer: a multi-center analysis. BMC Cancer. 2018;18:468. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Toxopeus E, van der Schaaf M, van Lanschot J, et al. Outcome of patients treated within and outside a randomized clinical trial on neoadjuvant chemoradiotherapy plus surgery for esophageal cancer: extrapolation of a randomized clinical trial (CROSS). Ann Surg Oncol. 2018;25:2441–2448. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Meadows AT, Kramer S, Hopson R, Lustbader E, Jarrett P, Evans AE. Survival in childhood acute lymphocytic leukemia: effect of protocol and place of treatment. Cancer Invest. 1983;1:49–55. [DOI] [PubMed] [Google Scholar]
- 16.Unger JM, Cook E, Tai E, Bleyer A. The role of clinical trial participation in cancer research: barriers, evidence, and strategies. Am Soc Clin Oncol Educ Book. 2016;35:185–198. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.R Foundation. R: A Language and Environment for Statistical Computing. Published 2018. Accessed April 20, 2018. http://www.R-project.org [Google Scholar]
- 18.Matched Samples that are Balanced and Representative by Design. Package “designmatch.” Version 0.20. R Package DocumentationA comprehensive index of R packages and documentation from CRAN, Bioconductor, GitHub and R-Forge. Published 2016. Accessed November 28, 2016. https://rdrr.io/cran/designmatch/man/designmatch-package.html [Google Scholar]
- 19.Bishop YMM, Feinberg SE, Holland PW. Analysis of square tables: symmetry and marginal homogeneity. In: Bishop YMM, Fienberg SE, Holland PW, eds. Discrete Multivariate Analysis: Theory and Practice. The MIT Press; 2007:281–309. [Google Scholar]
- 20.Matching Rosenbaum P., in Design of Observational Studies. Springer; 2010. [Google Scholar]
- 21.Silber JH, Rosenbaum PR, Trudeau ME, et al. Multivariate matching and bias reduction in the surgical outcomes study. Med Care. 2001;39:1048–1064. [DOI] [PubMed] [Google Scholar]
- 22.Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika. 1983;70:41–55. [Google Scholar]
- 23.Kalbfleish JD, Prentice RL. The Statistical Analysis of Failure Time Data. John Wiley & Sons Inc; 1980. [Google Scholar]
- 24.O’Brien PC, Fleming TR. A paired Prentice-Wilcoxon test for censored paired data. Biometrics. 1987;43:169–180. [Google Scholar]
- 25.McCullagh P. A logisitic model for paired comparisons with ordered categorial data. Biometrika. 1977;64:449–453. [Google Scholar]
- 26.Stock W, La M, Sanford B, et al. ; Children’s Cancer Group; Cancer and Leukemia Group B studies. What determines the outcomes for adolescents and young adults with acute lymphoblastic leukemia treated on cooperative group protocols? A comparison of Children’s Cancer Group and Cancer and Leukemia Group B studies. Blood. 2008;112:1646–1654. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27.Khera N, Majhail NS, Brazauskas R, et al. Comparison of characteristics and outcomes of trial participants and nonparticipants: example of Blood and Marrow Transplant Clinical Trials Network 0201 Trial. Biol Blood Marrow Transplant. 2015;21:1815–1822. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28.Unger JM, Barlow WE, Martin DP, et al. Comparison of survival outcomes among cancer patients treated in and out of clinical trials. J Natl Cancer Inst. 2014;106:dju002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Hough R, Sandhu S, Khan M, et al. Are survival and mortality rates associated with recruitment to clinical trials in teenage and young adult patients with acute lymphoblastic leukaemia? A retrospective observational analysis in England. BMJ Open. 2017;7:e017052. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Howell DL, Ward KC, Austin HD, Young JL, Woods WG. Access to pediatric cancer care by age, race, and diagnosis, and outcomes of cancer treatment in pediatric and adolescent patients in the state of Georgia. J Clin Oncol. 2007;25:4610–4615. [DOI] [PubMed] [Google Scholar]
- 31.Albritton KH, Wiggins CH, Nelson HE, Weeks JC. Site of oncologic specialty care for older adolescents in Utah. J Clin Oncol. 2007;25:4616–4621. [DOI] [PubMed] [Google Scholar]
- 32.Parsons HM, Harlan LC, Schmidt S, et al. ;AYA HOPE Collaborative Group. Who treats adolescents and young adults with cancer? A report from the AYA HOPE Study. J Adolesc Young Adult Oncol. 2015;4:141–150. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Miller VA, Reynolds WW, Nelson RM. Parent-child roles in decision making about medical research. Ethics & Behav. 2008;18:161–181. [Google Scholar]
- 34.Truong TH, Weeks JC, Cook EF, Joffe S. Altruism among participants in cancer clinical trials. Clin Trials. 2011;8:616–623. [DOI] [PubMed] [Google Scholar]
- 35.Bell JAH, Forcina V, Mitchell L, et al. Perceptions of and decision making about clinical trials in adolescent and young adults with cancer: a qualitative analysis. BMC Cancer. 2018;18:629. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Forcina V, Vakeesan B, Paulo C, et al. Perceptions and attitudes toward clinical trials in adolescent and young adults with cancer: a systematic review. Adolesc Health Med Ther. 2018;9:87–94. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Barakat LP, Schwartz LA, Reilly A, Deatrick JA, Balis F. A qualitative study of phase III cancer clinical trial enrollment decision-making: perspectives from adolescents, young adults, caregivers, and providers. J Adolesc Young Adult Oncol. 2014;3:3–11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Barakat LP, Stevens E, Li Y, et al. Evaluation of the Pediatric Research Participation Questionnaire for measuring attitudes toward cancer clinical trials among adolescents and young adults. J Adolesc Young Adult Oncol. 2019;8:423–433. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Rosenbaum PR. Exact, nonparametric inference when doses are measured with random errors. J Am Stat Assoc. 2005;100:511–518. [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.