Abstract
Background
Clinician reporting of symptomatic adverse events (AEs) in phase I trials uses the Common Terminology Criteria for Adverse Events (CTCAE). The utility of the patient-reported outcomes (PROs) version of the CTCAE (PRO-CTCAE) in this setting is unknown. This prospective, observational study compared patient- and clinician-reported symptomatic AEs in phase I patients.
Methods
Phase I study–eligible patients at Princess Margaret were surveyed with the PRO-CTCAE full-item library (78 symptomatic AEs) at baseline (BL), mid-cycle 1, and mid-cycle 2 (C2). Patient and trial characteristics, best response, and survival data were collected. Presence or absence of patient- (PRO-CTCAE) or clinician-reported symptomatic AEs were compared (kappa) at defined timepoints and overall (BL+ mid-cycle 1 + C2).
Results
Of 292 patients approached from May 2017 to January 2019, a total of 265 (90.8%) were consented, with 243 (91.7%) evaluable and 552 PRO-CTCAE surveys (completion rate = 98.7%) included in analyses. Evaluation of overall patient-reported symptomatic AEs identified 50 PRO-CTCAE and 11 CTCAE items with 10% or greater reporting frequency. Nineteen CTCAE items were reported as 1% or less despite matched PRO-CTCAE items reporting as 10% or greater. Underreported categories included sexual health, bodily emissions, and cognition. Clinician- relative to patient-reporting frequency (ratio) demonstrated 9 symptomatic AEs with a 50-fold or more lower clinician reporting rate. Overall patient–clinician agreement for individual symptomatic AEs ranged from poor (κ = 0.00-0.19) to moderate (κ = 0.40-0.59), with discordance driven by lack of clinician reporting. Dyspnea (κ = 0.54) and peripheral neuropathy (κ = 0.63) at BL and limb edema (κ = 0.55) at C2 demonstrated the highest patient–clinician agreement.
Conclusions
Poor to moderate patient–clinician agreement for symptomatic AEs suggests clinician underreporting in phase I trials. Analyses of severity and interference PRO categories are ongoing.
Adverse event (AE) reporting is crucial to early-phase clinical trial accuracy and evaluation of experimental anticancer agents. Failure to precisely capture AEs can affect the risk–benefit assessment of tested drug(s), potentially exposing patients to subtherapeutic dosing or dose escalations that may lead to excessive toxicity (1). Standard AE reporting uses the Common Terminology Criteria for Adverse Events (CTCAE) (2) categorizing events into 3 groups: laboratory results (eg, hemoglobin), technical measures (eg, vital signs), and subjective patient-reported symptoms (eg, fatigue). Several reports have demonstrated underreporting of symptomatic AEs in cancer patients (1,3-7). Specialized health-care teams often collect and report AE causality, which may result in underreporting by failure to document or report mild or subjective complaints, discounting perceived unrelated AEs to study drug, patient reluctance to report for fear of treatment modification or discontinuation, and patient or clinician censoring of socially or culturally sensitive topics (eg, sexual health) (1,8). Toxicity and tolerability of experimental anticancer drugs may be better characterized by integrating patient-reported outcomes (PROs) into phase I trials (9), which are increasingly used in registration and the drug approval process (10-12).
The Patient-Reported Outcomes version of the CTCAE (PRO-CTCAE) is a multidimension symptom assessment tool designed and validated in a heterogenous cancer population to evaluate subjective patient toxicity in clinical trials (13,14). The PRO-CTCAE encompasses 78 symptomatic AE items evaluated through 124 survey questions on a toxicity spectrum of presence or absence, frequency, severity, and interference with daily activities (15). PRO-CTCAE was designed as a companion survey tool to standard CTCAE reporting as it includes only symptomatic toxicity for which correlation between the 2 reporting formats is moderate at best (3,14). Clinician reporting of symptomatic AEs is known to be associated with clinical endpoints such as death or hospitalization, whereas patient reporting correlates with day-to-day health status (16). The 2 reporting systems are complementary, providing information on symptomatic AE profiles and tolerability of experimental agents.
PRO-CTCAE has been widely tested in cancer patients (5,6,16-20) but has not yet been evaluated in phase I cancer trials. In contrast to late-phase clinical trials, the type, rate, and severity of AEs in phase I studies are generally unknown. Since phase I studies evaluate first-in-human compounds, safety and toxicity are rigorously monitored to inform future drug development and dose levels (21). Beyond the evaluation of drug safety, which is an assessment of medical risk to the patient, the PRO-CTCAE and other patient-reported measures from phase I trials may inform drug tolerability, which is the degree that AEs can be endured by patients (22).
We previously demonstrated that the administration of the full library of the PRO-CTCAE was feasible in phase I trial patients (23). In this study, we conducted a prospective, observational trial to compare clinician- and patient-reported symptomatic AE agreement in phase I trial patients.
Methods
Patients and Eligibility
Adult patients evaluated for inclusion in phase I solid tumor trials at Princess Margaret Cancer Centre in Toronto, Ontario, from May 1, 2017, to January 1, 2019, were approached in an outpatient, centralized phase I oncology clinic for participation (Supplementary Figure 1, available online). Key inclusion criteria were eligibility for phase I trial enrolment, English fluency, and absence of clinically significant cognitive impairment as assessed by phase I primary investigators. Patient characteristics (eg, age, gender, Eastern Cooperative Oncology Group, educational attainment, primary language, tumor type), treatment information (eg, type and number of prior therapies, phase I study treatment), Royal Marsden Index (RMI)-validated (24) phase I prognostic score variables (Lactate dehydrogenase [LDH], albumin, number of metastatic sites), best response (RECIST v1.1) on phase I study, and survival data were collected and stored in a centralized database.
Study Design
This prospective, single-center, observational trial evaluated patient-reported symptomatic AEs using the full PRO-CTCAE survey (25) at 3 study timepoints: 1) before initiation of investigational therapy (baseline [BL]), 2) mid-cycle 1 (C1), and 3) mid-cycle 2 (C2) as suggested by the National Cancer Institute (15) (Supplementary Figure 2, available online). PRO-CTCAE surveys were administered electronically to patients on tablet computers during encounters. Clinical evaluation of patient-reported toxicities by specialized phase I nursing and physician assessment teams (known henceforth as clinician), with physician-designated attribution of drug toxicity at parallel timepoints (same day), were captured using CTCAE v4.0. Both toxicity datasets (PRO-CTCAE and CTCAE) were stored in a central database. All CTCAE items were included regardless of drug attribution, and phase I clinician teams were blinded to PRO responses throughout the study period. For patients discontinuing from their respective phase I study before reaching the C1 or C2 timepoint, PRO-CTCAE and CTCAE survey attempts were adjusted for attrition (Supplementary Table 1, available online).
The primary endpoint of this study was to define patient- (PRO-CTCAE) and clinician- (CTCAE) reported symptomatic AE agreement based on PRO-CTCAE responses (presence or absence) overall (BL+C1+C2), at BL, and after initiation of therapy (C1, C2). Exploratory endpoints included evaluation of patient characteristics, RMI, and patient- or clinician-assessed toxicities in relation to survival.
This protocol was approved by the Princess Margaret Cancer Centre Research Ethics Board (CAPCR #17–5022) and conformed to the Helsinki Declaration; all participants provided written informed consent before enrollment.
Statistical Analysis
Study sample size was calculated using an estimate of 70% of patients completing 2 or more treatment cycles with a survey completion rate of 80% (14,16) and 5% margin of error for every PRO-CTCAE item, requiring 377 reports. Based on these assumptions, we estimated enrollment of 182 patients. Assuming a 10% dropout rate, we estimated a target enrollment of 200 patients.
Patient characteristics in addition to grouped treatment, response, and nominal aggregate patient (PRO-CTCAE) and clinician (CTCAE) AE reporting were evaluated descriptively. For this analysis, only the presence or absence of a PRO-CTCAE symptom was evaluated. Follow-up was defined as time from phase I study enrollment to last documented follow-up. Overall survival (OS) was estimated using the Kaplan-Meier method and calculated from time of PRO-CTCAE study enrollment to death from any cause. Univariable and multivariable survival analyses can be found in the Supplementary Methods (available online).
Comparative statistics were assessed using a kappa model (26) with agreement scores for particularly good (κ = 0.80-1.00), good (κ = 0.60-0.79), moderate (κ = 0.40-0.59), fair (κ = 0.20-0.39), and poor (κ = 0.00-0.19). Overall (BL+C1+C2) patient to clinician reporting frequencies (ratio) were evaluated for individual toxicities. Overall PRO-CTCAE and CTCAE individual toxicity percentage difference was also compared (27). Statistical analyses used SAS software (v9.4; Cary, NC).
Results
Patient Enrollment and Survey Data
From May 2017 to January 2019, 292 patients were approached for study inclusion with 265 (90.8%) consenting and 243 (91.7%) evaluable in final analyses. Of 561 attempted surveys, 552 (98.4%) were evaluable, including 545 (98.7%) with complete data and 7 with partially evaluable data (1.3%). Accounting for patient attrition during phase I trial(s), 243 evaluable patients had a BL survey (100%, n = 243), with 192 (89.3%, n = 215) completing 2 surveys and 117 (62.1%, n = 186) completing 3 surveys (Supplementary Table 1, available online).
For patients consenting to phase I trials, 24 (9.9%) screen failed and did not receive investigational treatment; BL surveys were included in final analyses. Of 219 patients who received investigational treatment, 207 (94.5%) completed 1 or more cycles, 171 (78.1%) completed 2 or more cycles, and 120 (54.8%) completed 3 or more cycles (Supplementary Table 2, available online).
Cohort Characteristics
The median patient age was 60 years (range = 18-82 years), 51.0% were female, 79.4% were Eastern Cooperative Oncology Group performance status 1, 43.6% had a university-level education, and 89.3% reported English as their primary language (Table 1). Predominant tumor types were gastrointestinal (31.7%), head and neck (13.2%), and breast (10.7%). Phase I trials generally involved immunotherapy (65.8%) and were frequently combined (62.6%) with other investigational agents. Single-agent targeted therapies (16.5%), combination immune checkpoint inhibitors plus immune costimulatory molecules (27.2%), and immune checkpoint inhibitors plus other immunomodulators (12.8%), were common study regimens (Supplementary Table 3, available online). Most patients (64.2%) were not previously enrolled in a clinical trial (Supplementary Table 4, available online).
Table 1.
Patient, treatment, and response characteristics (n = 243)
| Characteristic | No. (%) |
|---|---|
| Median age at enrollment (range), y | 60 (18-82) |
| Sex | |
| Male | 119 (49.0) |
| Female | 124 (51.0) |
| ECOG | |
| 0 | 49 (20.2) |
| 1 | 193 (79.4) |
| 2 | 1 (0.4) |
| Education level | |
| Elementary school | 5 (2.1) |
| High school | 76 (31.3) |
| Postgraduate (nonuniversity) | 56 (23.0) |
| University | 106 (43.6) |
| English as primary language | |
| Yes | 217 (89.3) |
| No | 26 (10.7) |
| Tumor type | |
| Gastrointestinal | 77 (31.7) |
| Head and neck | 32 (13.2) |
| Breast | 26 (10.7) |
| Genitourinary | 24 (9.9) |
| Gynecological | 21 (8.6) |
| Melanoma | 19 (7.8) |
| Lung | 18 (7.4) |
| Sarcoma | 18 (7.4) |
| Other | 8 (3.3) |
| RMI score | |
| 0-1 | 173 (71.2) |
| 2-3 | 70 (28.8) |
| Prior treatments | |
| 0 | 13 (5.3) |
| 1 | 62 (25.5) |
| 2 | 87 (35.8) |
| ≥3 | 81 (33.3) |
| Monotherapy or combination | |
| Monotherapy | 91 (37.4) |
| Combination | 152 (62.6) |
| Treatment type by group | |
| Immunotherapy based | 160 (65.8) |
| Targeted based | 60 (24.7) |
| Immuno-targeted combination | 23 (9.5) |
| Best responsea | |
| Complete response | 3 (1.4) |
| Partial response | 18 (8.2) |
| Stable disease | 79 (36.1) |
| Progressive disease | 117 (53.4) |
aRECIST evaluable population (n = 219). Screen fail; n = 24 (9.9%); not evaluable, n = 2 (0.9%); overall response rate (complete response + partial response) = 9.6%. ECOG = Eastern Cooperative Oncology Group; RMI = Royal Marsden Index.
For patients receiving investigational treatment (n = 219), the overall response rate was 9.6% (complete response, n = 3; partial response n = 18) with stable disease as best response for 36.1% (n = 79), progressive disease for 53.4% (n = 117), and unevaluable in 0.9% (n = 2) (Table 1).
AE Reporting and Agreement Scores
The 3 most frequently reported PRO-CTCAE categories were fatigue, pain (general), and anxiety (Figure 1, top). The median number of reported items at BL for PRO-CTCAE and CTCAE items was 11 and 3, respectively. Overall (BL + C1 + C2) analysis of patient-reported symptomatic AEs (Figure 1, top) identified 50 PRO-CTCAE items, and 11 CTCAE items with a reporting frequency of 10% or greater. Conversely, 19 CTCAE items (Figure 1, bottom) were reported at a frequency of 1% or less despite matched PRO-CTCAE items being reported at a frequency of 10% or greater. Underreporting of symptomatic CTCAE categories by clinicians included sexual health (decreased libido, vaginal dryness, hot flashes), bodily emissions (urinary frequency/urgency, flatulence, hyperhidrosis), cognition (memory impairment, concentration), and visual disturbances (epiphora, floaters, blurred vision).
Figure 1.
Patient and clinician reporting of symptomatic adverse events (AEs). Top: Frequently (≥10%) reported patient (patient-reported outcomes version of Common Terminology Criteria for Adverse Events [PRO-CTCAE]) AEs relative to clinician (CTCAE) symptomatic AEs. Bottom: Infrequently (≤1%) reported clinician (CTCAE) events relative to frequently (≥10%) reported patient (PRO-CTCAE) symptomatic AEs.
Evaluation of the number of overall patient-reported symptomatic AEs relative to the number of clinician-reported symptomatic AEs (ratio) for individual symptomatic categories demonstrated 9 PRO-CTCAE items (decreased libido, palpitations, wheezing, voice alteration, hiccups, hyperhidrosis, vaginal dryness, nail ridging, and urinary incontinence) with a 50-fold or more lower clinician- relative to patient-reporting frequency (Figure 2). Percent reporting differences of PRO-CTCAE and CTCAE are shown in Supplementary Figure 3, available online.
Figure 2.
Patient to physician ratio of symptomatic adverse event (AE) categories. Ratio of overall patient symptomatic AE (patient-reported outcomes version of Common Terminology Criteria for Adverse Events [PRO-CTCAE]) reporting frequency relative to matched clinician symptomatic AE (CTCAE) reporting frequency by individual category.
No symptomatic AE at any timepoint had very good agreement (κ = 0.80-1.0) between patient and clinician reporting (Figure 3). Discordance resulted from the presence of individual symptomatic AEs in PRO-CTCAE and absence in CTCAE reporting. Peripheral neuropathy (κ = 0.63) at BL was the only symptomatic AE with a good patient and clinician agreement score (κ = 0.60-0.79). Few symptomatic AEs such as dyspnea (κ = 0.54; BL), vomiting (κ = 0.46; C1), and limb edema (κ = 0.55; C2) reached moderate (κ = 0.40-0.59) agreement between patients and clinicians at BL (n = 7), C1 (n = 1), or C2 (n = 3), respectively. No correlation between patient- and clinician-reported symptomatic AEs was seen for palpitations, hiccups, vaginal dryness, memory impairment, hyperhidrosis, flatulence, or urinary urgency either overall or at any individual timepoint.
Figure 3.
Kappa agreement scores for individual symptomatic adverse event (AE) categories. Agreement (kappa) scores between patient- (patient-reported outcomes version of Common Terminology Criteria for Adverse Events [PRO-CTCAE]) and clinician- (CTCAE) reported symptomatic AEs overall and by timepoint. Kappa agreement scores are classified based on poor (0.00-0.19), fair (0.20-0.39), moderate (0.40-0.59), good (0.60-0.79), and very good (0.80-1.00). PRO-CTCAE items with a frequency of 10% or greater were included in kappa analysis. Symptomatic AEs with agreement less than 0.1 (kappa) on 3 or more categories have been omitted. C1 = mid-cycle 1; C2 = mid-cycle 2.
Survival
The median follow-up for patients receiving phase I study treatment (n = 219) was 8.1 months (range = 0.2-26.9 months) with a median OS of 9.4 months (95% confidence interval [CI] = 7.3 to 11.3 months) and a 90-day mortality of 18.1% (95% CI = 13.3 to 23.1%). The median OS for the evaluable cohort (n = 243) was 8.4 months (95% CI = 6.5 to 10.8 months) (Supplementary Figure 4, A and B, available online). There was no difference in OS between patients who reported symptomatic AEs below or above the PRO-CTCAE reporting median (≤11 or >11) on BL surveys (8.5 vs 8.8 months, P = .69) (Supplementary Figure 4, C, available online). Median OS was worse (P = .03) for patients above the clinician-reported (CTCAE) symptomatic AE median (>3 vs ≤3) at BL (7.2 months, 95% CI = 5.8 to 10.0 months; vs 10.6 months, 95% CI = 8.1 to 16.6 months, respectively) (Supplementary Figure 4, D, available online). Inferior survival was seen for patients with an RMI score of 2-3 relative to 0-1 (median OS = 5.8 months, 95% CI = 4.1 to 8.6 months; vs 10.4 months, 95% CI = 8.5 to 13.7 months, P < .001) (Supplementary Figure 4, E, available online).
Univariable analysis of patient characteristics demonstrated worse survival for an RMI score of 2-3 (hazard ratio = 1.86, 95% CI = 1.31 to 2.64, P < .001). No other key potential prognostic variables were found to be associated with OS (Supplementary Table 5, available online). Univariable and multivariable of patient- and clinician-reported symptomatic AEs can be found in Supplementary Table 6 (available online).
Discussion
This single-center observational trial shows clinician underreporting of symptomatic AEs in phase I trials. In this study, high acceptance (>95%) and completion (approximately 90%) rates of the full PRO-CTCAE survey establish the feasibility of integrating such a questionnaire in the phase I setting. We identified the top 50 PRO-CTCAE items occurring at a frequency of 10% or greater and 19 clinician-reported CTCAE items occurring at a frequency of 1% or less despite higher patient reporting (≥10%), with generally low levels of agreement. Whereas the total number of clinician-reported AEs was associated with survival, this was not shown for total patient-reported AEs.
Self-reporting of toxicities is known to improve patient-clinician communication and satisfaction (9,28), increase clinic efficiency, foster early detection of AEs, and improve OS (29-32). The discordant patient to clinician symptomatic AE reporting identified in our study is consistent with data from nonphase I studies (3-5) and is attributable to both human and technical factors (3,8). One source of higher patient reporting may relate to survey prompts, providing an opportunity to meticulously review individual PRO-CTCAE items. Conversely, in a time-restricted oncology clinic, capturing the full complement of CTCAE symptoms by clinicians may be difficult and instead rely on patient reporting as symptoms are recalled, tailored, and (possibly) omitted. PRO tools provide a structure to methodically capture a wider spectrum of symptomatic AEs and facilitate discussions of socially and culturally sensitive topics (eg, genitourinary function) (1,28).
In our study, a web-based version of the PRO-CTCAE was administered via computer tablets (electronic PRO or ePRO) (25,33). Electronic PROs have distinct advantages with fewer missing or unusable data points (27), reduced completion time (34), and fewer data entry errors (35), likely contributing to our high completion rates. Patient demographics and symptom burden can also affect patient reporting with preference for interactive voice-response systems over web- or computer-based systems in specific populations such as those aged older than 65 years, male gender, rural geography, visible minorities, and lack of formal postsecondary education (8). These preferences may reflect social attributes such as language barriers, internet access, online literacy, or comfort with newer technologies. Comparatively, patients in our study had a slightly lower median age, equal proportions of males and females, English predominance, and higher rates of postsecondary education. In addition, patient acceptance and perceived utility of completing ePROs at sequential clinic visits are generally quite high as was recently demonstrated at our institution (36).
Severity and interference PRO-CTCAE response categories were not designed for comparison with clinician-reported symptomatic AE data such as CTCAE grading. However, presence or absence of symptomatic AE items are directly comparable between patients and clinicians. Consistent with other studies (4,5,10,17), we report poor to moderate concordance with observable symptoms such as vomiting, limb edema, rash, and dyspnea; with very poor to no correlation for less visible or sensitive symptomatic AEs related to cognition, sexual health, and genitourinary function (6,8).
As the evaluation of drug tolerability is an overarching goal of phase I trials, the lack of patient and clinician symptomatic AE agreement in this study highlights the need for a comprehensive toxicity assessment tool. The 50 PRO-CTCAE items identified here could serve as a customized PRO survey for phase I trials regardless of the experimental drug(s) being tested. This may have efficiency and accuracy advantages over eliciting a full-item library by reducing survey fatigue and reporter inaccuracy for those experiencing physical or cognitive exhaustion (8,37). Customized surveys are difficult to build for phase I trials since the AE profile of an experimental agent is unknown. Item reduction can be performed by statistical methods such as factor analysis (38) or receiver operating curves to determine optimal threshold cutoffs (39) for elimination of infrequent toxicities (<1% vs <5% vs <10%). As phase I trials generally have small cohorts (<100) and small numbers of patients are treated at the recommended phase II dose, rare or infrequent toxicities are difficult to characterize precisely. A shorter survey would be less burdensome and faster for patients to complete but would need to be balanced against the failure to capture less frequent or unanticipated symptomatic AEs. The PRO-CTCAE library was designed to be tailored by researchers who can select items appropriate for the experimental agent or research purposes. Our chosen benchmark for symptomatic AE frequency (≥10%) was based on the US Food and Drug Administration guidance (40), and such a PRO tool could inform the tolerability of tested dose levels to better describe the timing, severity, resolution, and impact of symptomatic AEs on patients. PRO data collected during phase I trials may also inform the selection of specific PRO measures and tools when designing phase II or III trials. Empirical PRO data on drug-related toxicities from phase I trials could be used to define PRO hypotheses and endpoints in late-phase trials.
Study limitations include a single-center design and inclusion criteria mandating English fluency. Although these phase I trials evaluated an array of drug(s) and are representative of trials conducted at most phase I centers, combination immune and nonimmune therapies were frequent. This study administered PRO-CTCAE surveys at only 3 timepoints given the expected attrition in this population. Although less than two-thirds of patients were assessed beyond cycle 2, longitudinal collection to trial completion or symptomatic AE resolution could have yielded further data. Finally, PRO tools do not assign causation of symptomatic AEs, and interpretation of PRO-CTCAE responses in this context are not yet formulated. The corollary is that clinician underreporting of symptomatic AEs does not imply that phase I investigational agents were more toxic than reported. Symptomatic AEs may be additionally driven by tumor type and other comorbid issues. As patients enrolled in phase I clinical trials have good performance status and organ function with few comorbidities, the findings of this study may not be generalizable. Determination of optimal PRO-CTCAE constructs and analytic approaches are an active area of research (41).
To our knowledge, this is the first prospective trial evaluating PRO-CTCAE in patients enrolled in a wide spectrum of phase I trials. PRO responses have the potential to inform tolerability of experimental therapies and influence oncology drug approvals (10,11,42). Furthermore, symptomatic AEs collected from PRO tools could affect drug dose-escalation decisions through feedback (eg, investigator, sponsor) or statistical design (eg, continual reassessment model) and may affect maximum tolerated dose and recommended phase II dose determinations. Efforts to integrate the phase I specific PRO into early-phase studies are underway, with a view to administering surveys longitudinally during the study course and developing an analytic framework to better understand the clinical actionability of severity and interference responses.
Funding
No funding was procured for this study.
Notes
Role of the funder: Not applicable.
Disclosures: ZWV has received honorarium from Pfizer and Genomic Health. He has also served on an advisory board for Genomic Health. PLB reports grants from Bristol-Myers-Squibb, Sanofi, Genentech/Roche Novartis, GlaxoSmithKline, Nektar Therapeutics, Merck, Lilly, Servier, PTC Therapeutics, Seattle Genetics, Mersana outside the submitted work; and Uncompensated advisory boards for Bristol-Myers-Squibb, Sanofi, Pfizer, Genentech/Roche. LLS reports compensated consultancy/advisory board participation for Merck, Pfizer, Celgene, AstraZeneca/Medimmune, Morphosys, Roche, GeneSeeq, Loxo, Oncorus, Symphogen, Seattle Genetics, Glaxo-Smith-Kline, Voronoi, Treadwell Therapeutics, Arvinas, Tessa and Navire; and reports institutional support for clinical trials conduct from Novartis, Bristol-Myers Squibb, Pfizer, Boehringer-Ingelheim, GlaxoSmithKline, Roche/Genentech, Karyopharm, AstraZeneca/Medimmune, Merck, Celgene, Astellas, Bayer, AbbVie, Amgen, Symphogen, Intensity Therapeutics, Mirati, Shattucks and Avid. Spouse hold stock in Agios and Treadwell Therapeutics. ARH reports institutional support for clinical trials conduct from Novartis, Bristol-Myers Squibb, Pfizer, Boehringer-Ingelheim, GlaxoSmithKline, Roche/Genentech, Karyopharm, AstraZeneca/Medimmune, Merck, Astellas and Bayer; and compensated consulting/advisory boards for AstraZeneca, Merck and GlaxoSmithKline. ARAR has received research funding from Roche, Genentech, Eli Lilly, Merck, Boehringer Ingelheim, Novartis, AbbVie, Deciphera, Karyopharm, Astra Zeneca, Medimmune, Blueprint, Bristol Myers Squibb, GSK, Entremed/Casi Pharmaceuticals, Adaptimmune and BetaCat. He also has also served the advisory board for Eli Lilly, Merck, Adaptimmune, Boehringer Ingelheim. AS has served as an advisory board consultant for Merck (compensated), Bristol-Myers Squibb (compensated), Novartis (compensated), Oncorus (compensated), Janssen (compensated). She has also received research funding from Novartis, Bristol-Myers Squibb, Symphogen AstraZeneca/Medimmune, Merck, Bayer, Surface Oncology, Northern Biologics, Janssen Oncology/Johnson & Johnson, Roche, Regeneron, Alkermes, Array Biopharma, GSK. DS, CG, LW, LM report no conflicts of interest.
Author contributions: Study Concepts and Design: DS, ARH. Data Acquisition—CG, ZWV, DS. Quality Control of Data and Algorithms: DS, CG, ZWV, LW. Data Analysis and Interpretation—ZWV, LW, ARH. Statistical Analysis: ZWV, LW. Manuscript Preparation: ZWV, ARH. Manuscript Editing: ZWV, ARH, DS, LW, AAR, AS, PLB, LLS, LM
Acknowledgements: The study group would like to acknowledge the patients and their families who contributed to and made this research possible.
Prior presentation: This study was presented at the European Society of Medical Oncology (ESMO) annual meeting in Barcelona, Spain in Sept 2019.
Data Availability
The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.
Supplementary Material
References
- 1. Di Maio M, Basch E, Bryce J, Perrone F.. Patient-reported outcomes in the evaluation of toxicity of anticancer treatments. Nat Rev Clin Oncol. 2016;13(5):319–325. [DOI] [PubMed] [Google Scholar]
- 2. Topalian SL, Hodi FS, Brahmer JR, et al. Safety, activity, and immune correlates of anti-PD-1 antibody in cancer. N Engl J Med. 2012;366(26):2443–2454. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Atkinson TM, Ryan SJ, Bennett AV, et al. The association between clinician-based Common Terminology Criteria for Adverse Events (CTCAE) and Patient-Reported Outcomes (PRO): a systematic review. Support Care Cancer. 2016;24(8):3669–3676. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Fromme EK, Eilers KM, Mori M, Hsieh Y-C, Beer TM.. How accurate is clinician reporting of chemotherapy adverse effects? A comparison with patient-reported symptoms from the quality-of-life questionnaire C30. J Clin Oncol. 2004;22(17):3485–3490. [DOI] [PubMed] [Google Scholar]
- 5. Cirillo M, Venturini M, Ciccarelli L, Coati F, Bortolami O, Verlato G.. Clinician versus nurse symptom reporting using the National Cancer Institute—Common Terminology Criteria for Adverse Events during chemotherapy: results of a comparison based on patient’s self-reported questionnaire. Ann Oncol. 2009;20(12):1929–1935. [DOI] [PubMed] [Google Scholar]
- 6. Gravis G, Marino P, Joly F, et al. Patients’ self-assessment versus investigators’ evaluation in a phase III trial in non-castrate metastatic prostate cancer (GETUG-AFU 15). Eur J Cancer. 2014;50(5):953–962. [DOI] [PubMed] [Google Scholar]
- 7. Di Maio M, Gallo C, Leighl NB, et al. Symptomatic toxicities experienced during anticancer treatment: agreement between patient and physician reporting in three randomized trials. J Clin Oncol. 2015;33(8):910–915. [DOI] [PubMed] [Google Scholar]
- 8. Chang EM, Gillespie EF, Shaverdian N.. Truthfulness in patient-reported outcomes: factors affecting patients’ responses and impact on data quality. Patient Relat Outcome Meas. 2019;10:171–186. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Howell D, Molloy S, Wilkinson K, et al. Patient-reported outcomes in routine cancer clinical practice: a scoping review of use, impact on health outcomes, and implementation factors. Ann Oncol. 2015;26(9):1846–1858. [DOI] [PubMed] [Google Scholar]
- 10. Kim J, Singh H, Ayalew K, et al. Use of PRO measures to inform tolerability in oncology trials: implications for clinical review, IND safety reporting, and clinical site inspections. Clin Cancer Res. 2018;24(8):1780–1784. [DOI] [PubMed] [Google Scholar]
- 11. Zettler M, Basch E, Nabhan C.. Surrogate end points and patient-reported outcomes for novel oncology drugs approved between 2011 and 2017. JAMA Oncol. 2019;5(9):1358. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Vodicka E, Kim K, Devine EB, Gnanasakthy A, Scoggins JF, Patrick DL.. Inclusion of patient-reported outcome measures in registered clinical trials: evidence from ClinicalTrials.gov (2007-2013). Contemp Clin Trials. 2015;43:1–9. [DOI] [PubMed] [Google Scholar]
- 13. Basch E, Reeve BB, Mitchell SA, et al. Development of the National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). JNCI J Natl Cancer Inst. 2014;106(9):dju244. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Dueck AC, Mendoza TR, Mitchell SA, et al. Validity and reliability of the US National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). JAMA Oncol. 2015;1(8):1051. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Jacobsen P; National Cancer Institute, Division of Cancer control and Population Sciences, Healthcare Delivery and Research Program. Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAETM). 2019. http://healthcaredelivery.cancer.gov/pro-ctcae/. Accessed December 19, 2019.
- 16. Basch E, Jia X, Heller G, et al. Adverse symptom event reporting by patients vs clinicians: relationships with clinical outcomes. JNCI J Natl Cancer Inst. 2009;101(23):1624–1632. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Basch E, Iasonos A, McDonough T, et al. Patient versus clinician symptom reporting using the National Cancer Institute Common Terminology Criteria for Adverse Events: results of a questionnaire-based study. Lancet Oncol. 2006;7(11):903–909. [DOI] [PubMed] [Google Scholar]
- 18. Christodoulou M, McCloskey P, Stones N, et al. Investigation of a patient reported outcome tool to assess radiotherapy-related toxicity prospectively in patients with lung cancer. Radiother Oncol. 2014;112(2):244–249. [DOI] [PubMed] [Google Scholar]
- 19. Efficace F, Rosti G, Aaronson N, et al. Patient- versus physician-reporting of symptoms and health status in chronic myeloid leukemia. Haematologica. 2014;99(4):788–793. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Bennett BK, Park SB, Lin CSY, Friedlander ML, Kiernan MC, Goldstein D.. Impact of oxaliplatin-induced neuropathy: a patient perspective. Support Care Cancer. 2012;20(11):2959–2967. [DOI] [PubMed] [Google Scholar]
- 21. Le Tourneau C, Lee JJ, Siu LL.. Dose escalation methods in phase I cancer clinical trials. JNCI J Natl Cancer Inst. 2009;101(10):708–720. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22. Sacks CA, Miller PW, Longo DL.. Talking about toxicity — “What We’ve Got Here Is a Failure to Communicate.” N Engl J Med. 2019;381(15):1406–1408. [DOI] [PubMed] [Google Scholar]
- 23. Shepshelovich D, McDonald K, Spreafico A, et al. Feasibility assessment of using the complete Patient‐Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO‐CTCAE) item library. Oncologist. 2019;24(4):e146–e148. doi:10.1634/theoncologist.2018-0332. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Arkenau HT, Barriuso J, Olmos D, et al. Prospective validation of a prognostic score to improve patient selection for oncology phase I trials. J Clin Oncol. 2009;27(16):2692–2696. [DOI] [PubMed] [Google Scholar]
- 25. Bennett AV, Dueck AC, Mitchell SA, et al. ; on behalf of the National Cancer Institute PRO-CTCAE Study Group. Mode equivalence and acceptability of tablet computer-, interactive voice response system-, and paper-based administration of the U.S. National Cancer Institute’s Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO). Health Qual Life Outcomes. 2016;14(1):24. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. McHugh ML. Interrater reliability: the kappa statistic. Biochem Med. 2012;22(3):276–282. [PMC free article] [PubMed] [Google Scholar]
- 27. Atkinson TM, Dueck AC, Satele DV, et al. Clinician vs patient reporting of baseline and postbaseline symptoms for adverse event assessment in cancer clinical trials. JAMA Oncol. 2020;6(3):437. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Rotenstein LS, Huckman RS, Wagle NW.. Making patients and doctors happier — the potential of patient-reported outcomes. N Engl J Med. 2017;377(14):1309–1312. [DOI] [PubMed] [Google Scholar]
- 29. Denis F, Lethrosne C, Pourel N, et al. Randomized trial comparing a web-mediated follow-up with routine surveillance in lung cancer patients. JNCI J Natl Cancer Inst. 2017;109(9):djx029. doi:10.1093/jnci/djx029. [DOI] [PubMed] [Google Scholar]
- 30. Gotay CC, Kawamoto CT, Bottomley A, Efficace F.. The prognostic significance of patient-reported outcomes in cancer clinical trials. J Clin Oncol. 2008;26(8):1355–1363. [DOI] [PubMed] [Google Scholar]
- 31. Quinten C, Coens C, Mauer M, et al. Baseline quality of life as a prognostic indicator of survival: a meta-analysis of individual patient data from EORTC clinical trials. Lancet Oncol. 2009;10(9):865–871. [DOI] [PubMed] [Google Scholar]
- 32. Basch E, Deal AM, Dueck AC, et al. Overall survival results of a trial assessing patient-reported outcomes for symptom monitoring during routine cancer treatment. JAMA. 2017;318(2):197. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Coons SJ, Eremenco S, Lundy JJ, O’Donohoe P, O’Gorman H, Malizia W.. Capturing patient-reported outcome (PRO) data electronically: the past, present, and promise of ePRO. Patient. 2015;8(4):301–309. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Cella D, Hahn E, Jensen S, et al. Patient-Reported Outcomes in Performance Measurement. Research Triangle Park, NC: RTI Press; 2015. [PubMed] [Google Scholar]
- 35. Eremenco S, Coons SJ, Paty J, Coyne K, Bennett AV, McEntegart D.. PRO data collection in clinical trials using mixed modes: report of the ISPOR PRO mixed modes good research practices task force. Value Heal. 2014;17(5):501–516. [DOI] [PubMed] [Google Scholar]
- 36. Albaba H, Barnes TA, Veitch Z, et al. Acceptability of routine evaluations using patient‐reported outcomes of Common Terminology Criteria for Adverse Events and other patient‐reported symptom outcome tools in cancer outpatients: Princess Margaret Cancer Centre experience. Oncologist. 2019;24(11):e1219–e1227. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Atkinson TM, Schwartz CE, Goldstein L, et al. Perceptions of response burden associated with completion of patient-reported outcome assessments in oncology. Value Heal. 2019;22(2):225–230. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Kelton ML, LeardMann CA, Smith B, et al. ; the Millennium Cohort Study Team. Exploratory factor analysis of self-reported symptoms in a large, population-based military cohort. BMC Med Res Methodol. 2010;10(1):94. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39. Habibzadeh F, Habibzadeh P, Yadollahie M.. On determining the most appropriate test cut-off value: the case of tests with continuous results. Biochem Med. 2016;26(3):297–307. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Center for Drug Evaluation and Research, Center for Biologics Evaluation and Research. Adverse Reactions Section of Labeling for Human Prescription Drug and Biological Products.; 2006. ;doi:10.1088/0022-3727/42/4/043001. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/adverse-reactions-section-labeling-human-prescription-drug-and-biological-products-content-. Accessed December 10, 2020. [Google Scholar]
- 41. Kluetz PG, Chingos DT, Basch EM, Mitchell SA.. Patient-reported outcomes in cancer clinical trials: measuring symptomatic adverse events with the National Cancer Institute’s Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). Am Soc Clin Oncol Educ B. 2016;(36):67–73. [DOI] [PubMed] [Google Scholar]
- 42. Friends of Cancer Research. Broadening the definition of tolerability in cancer clinical trials to better measure the patient experience. 2018. https://www.focr.org/sites/default/files/Comparative Tolerability Whitepaper_FINAL.pdf. Accessed December 10, 2020.
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.