Adverse events reporting in stage III NSCLC trials investigating surgery and radiotherapy

Abstract

Background

Current treatment options for stage III non-small cell lung cancer (NSCLC) consist of different combinations of chemotherapy, surgery, radiotherapy and immunotherapy. Treatment choices are highly individual decisions, in which adverse events (AEs) are relevant for decision-making. This study aims to analyse reporting of AEs in prospective stage III NSCLC trials, focussing on trials including radiotherapy and/or surgery.

Methods

PubMed was searched for prospective studies dealing with stage III NSCLC from January 1987 to April 2019. Meta-analyses were screened as a positive control. Pearson's Chi-squared test and smooth kernel distribution were used to estimate distributions. Data was resampled using bootstrapping.

Results

Out of 1193 initially identified studies, 119 met the inclusion criteria. Of these, 31 had a surgical procedure in any study arm. Grade 3 and 4 AEs were reported in 94.12% and 92.44% of the included studies, respectively. Reporting of grade 5 AEs was provided in 87.39% of cases. Grade 1 and 2 AEs were less commonly reported at 53.78% and 63.03%, respectively. One study did not mention any AEs. Of the 31 treatment arms including any form of surgery, AEs were not reported in 10. Overall, 231 different AE items were reported, only 18 of them were included in at least 20% of the analysed studies.

Conclusion

Overall, AE reporting in stage III NSCLC was inconsistent and inhomogeneous. Studies including surgical study arms often reported only treatment-related deaths in regards of surgical AEs. Underreporting of AEs prohibits the extraction of patient-relevant information for decision-making and represents a suboptimal use of invested resources.

Short abstract

Adverse event reporting for stage III NSCLC is inconsistent and inhomogeneous. Surgical studies tend to underreport lower grade adverse effects. Patient-relevant information for decision-making is lost and invested resources are used suboptimally. https://bit.ly/3gLNIYy

Background

Stage III non-small cell lung cancer (NSCLC) is a potentially curable disease and patients may qualify for several treatment options. This collective of patients shows substantial heterogeneity as patients with different T and N stages, ranging from T4N0 to T4N3, are grouped into stages III A–C [1]. Depending on extent of disease, curative or palliative treatments are applicable and should be decided in a multidisciplinary setting. Over time, guidelines kept adapting to new evidence, forming today's recommendations. Treatment typically consists of chemotherapy combined with surgery or radiotherapy [2–4] and recently immunotherapy has become part of the treatment [5]. A multitude of factors may influence the availability of treatment options for each individual patient [6] (e.g. is a technique reimbursed?, Is a technique available?) and various criteria may be used to decide [7] (e.g. patient preference, laboratory findings). Therefore, treatment selection is a multilayered process, integrating physician- and patient-specific factors, emotional components and institutional criteria (e.g. implementation of guidelines).

The main differences between treatment strategies based on radiotherapy or surgery in stage III NSCLC are not quantitative (i.e. similar survival rates) [8], but rather of a qualitative nature. Patient preferences, the multidisciplinary setting and the treating physicians’ expertise add to the complex process of decision-making. A knowledge gap between the physician and the patient needs to be addressed by nuanced communication. This has a significant impact on patients’ expectation and fears, such as adverse events (AEs). These factors show why shared decision-making is of central importance to enable the selection of treatments best aligned with patient preferences. While decision aids can support the patient decision [9, 10], the question of what the respective risk–benefit ratios are (rate of AEs versus cure rates) remains difficult to answer.

To the patient, significant logistical differences exist between surgery and radiotherapy, for example, surgery requiring hospitalisation or radiotherapy requiring daily workday treatments for 6–7 weeks. Next to these logistical issues, potential AEs are essential to the tumour board's recommendation as well as the patient's decision. Information that is provided to patients is typically based on trials' primary outcomes (survival or cure rates) or registry data. AEs however, seem to be less the focus of current and past research. Published reviews also rarely deal with AEs, with only 1 in 15 addressing this issue [11, 12]. With this study, we aim to evaluate the quality of AE reporting in prospective trials for stage III lung cancer, related to their frequency and grading.

Methods

Search strategy

We performed a PubMed search for this systematic review, to identify trials on stage III NSCLC focussing on radiotherapy and or surgery (for the detailed search strategy see supplementary appendix A). To complete the list of publications, the meta-analyses gathered on www.pneumocancero.com were screened and were used as positive control assessing the completeness of the literature search. The search spanned from January 1987 until April 2019.

Study selection and inclusion criteria

Abstracts obtained by the PubMed search were screened for prospective studies focussing on stage III NSCLC, investigating curative radiotherapy or surgical regimens. Upon meeting those terms, the respective full texts were read. Studies were included for the analysis if radiotherapy (definitive, sequential/concurrent or trimodal setting) or surgery was part of the treatment. This covers single-arm studies, as well as studies including radiotherapy alone. Larger studies with several publications using the same collection of patients or separated toxicity reports were integrated as one publication (data were added to the first publication). If available, online appendices and supplements were screened for toxicity reports. Stage III needed to be defined according to the TNM classification, fourth edition or later. If no edition was declared, only studies starting inclusion after January 1, 1998 (after the appearance of the fifth edition) were included. Randomisation was not a requirement. Reviews, studies including other stages of NSCLC or published in other languages than English, German or French, were excluded from this systematic review. The process of the study selection is represented in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow chart (figure 1) [13].

FIGURE 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow chart visualising the study selection process.

Categorisation and assessment

To identify differences in the reporting of AEs, the studies were grouped according to predefined characteristics such as study design, year of publication, sample size, geographical features (Europe, North America, Asia and global) or funding of the study according to the declared sponsors (divided into pharmaceutical, public or unknown sponsors). All funding coming from governmental sources, nongovernmental organisations or universities were considered public. If a corporation granted any money, the study was attributed to the pharmaceutical group. Individual researchers’ honoraria did not label a study pharmaceutical.

Grading systems used by the authors were analysed. Textual description of AEs (mild, moderate, severe, life-threatening and death) were transcribed to grade 1 to 5 respectively, according to the Common Terminology Criteria for AEs (CTCAE), currently published in its fifth edition by the US National Institutes of Health [14]. If certain grades were specifically mentioned as not occurring (e.g. no toxic deaths), the respective grading/item was considered as reported. Patient-reported outcome measures (PROMs), defined as any questionnaire, where patients assess the toxicities themselves, were analysed. Any form of structured assessment (e.g. paper forms, electronic or by phone call) was considered a PROM, in accordance to the overview by Chen et al. [15]. The usage and frequency of the respective questionnaires were assessed. Data collection and management was handled using the REDCap electronic data capture tool [16].

Statistics

Pair-wise random permutation tests were used to estimate the statistical significance of the differences between each subgroup's mean number of reported AEs and the mean number of reported AEs of the remaining studies: The null-hypothesis distribution of mean differences was based on 50 000 permutations of the (AE number)-subgroup assignments. A one-tailed p-value was calculated as the fraction of null-hypothesis differences that were equal to or larger than the observed difference of means. The permutation test results were compared to the results of a nonparametric Mann–Whitney location test. Wolfram Mathematica was used to perform the statistical analyses. A p-value <0.05 was considered as a cut-off for statistical significance and p-values <0.01 were considered strongly significant.

Results

Study characteristics

The PubMed search identified 1193 studies. The meta-analyses from www.pneumocancero.com identified 120 studies, of which only 5 were not already included. A total of 119 met the inclusion criteria for this analysis (see supplementary appendix D). Of these, 31 involved surgery and 96 were randomised. Of those that declared the study phase, 39 were phase II and 34 were phase III studies. The oldest included studies were published in 1992 by Burkes et al. [17] and Strauss et al. [18], whereas 50% of the studies were published after 2009. The number of treated patients ranged from 12 to 709 (median 70), with a total of 222 individual treatment arms. Of these, 18 treatment arms used radiotherapy alone, 153 used radiochemotherapy (sequential or concomitant) and 23 used a trimodal regimen. There were 28 study arms that consisted of other treatments (surgery alone, surgery combined with different chemotherapy regimens, neoadjuvant radiotherapy with surgery or chemotherapy alone).

There were 8 global studies, 13 international studies (but confined to one continent) and 98 were performed at a national level. A total of 40 studies came from European countries, 30 were from North America and 41 studies were from Asian countries. Overall, 47 (39.5%) studies did not declare any funding, a third (40, 33.6%) were sponsored by pharmaceutical companies, while 32 studies (26.9%) received public funds.

Adverse event reporting

Toxicity report

Out of the 119 analysed trials, only one study did not mention toxicities at all. All other studies reported toxicities in general, but not for all treatment arms. Looking, for example, at studies with surgical arms, only 21 out of 31 studies (67.7%) reported surgical toxicities.

We identified 231 unique AE items, divided into two groups: nonsurgical or surgical. Nonsurgical AEs consisted of 119 items described as acute toxicity, 23 as late toxicity and 19 as acute and 10 as late organ effects (table 1 and supplementary appendix C). Organ effects were attributed to an organ system but not further specified. The surgical AE group included 60 items (58 acute and 2 late AEs). Some AEs were reported regularly, whereas others were mentioned by only a few (supplementary ppendix C). In total only 18 AEs were reported by at least one-fifth of the included studies. Focussing on nonsurgical acute or late toxicity, there were 25 items particularly associated with radiotherapy (e.g. pneumonitis, fistula or pulmonary fibrosis). There were 32 items associated with chemotherapy, such as polyneuropathy, febrile neutropenia or renal failure. AEs not specifically attributable to either of them remained unclassified.

TABLE 1.

Characteristics of nonsurgical adverse event reporting^#

Toxicity items	Acute (n=119)	Late (n=23)	Total (n=142)
Radiotherapy	17 (14.3%)	8 (34.8%)	25 (17.6%)
Chemotherapy	27 (22.7%)	5 (21.7%)	32 (22.5%)
Unclassified	75 (63.0%)	10 (43.5%)	85 (59.9%)

^#: subgroups “organ acute” and “organ late” not included.

Reporting systems

CTCAE is the most frequently used reporting system for AEs, with 66 studies using one of its versions. Versions 1 to 4 were used 3, 21, 30 and 5 times, respectively. Seven studies did not state what version was used. In 20 studies, the toxicity was reported using the World Health Organization's (WHO) scale. Scales developed by the Radiation Therapy Oncology Group (RTOG) and Eastern Cooperative Oncology Group(ECOG) or Japan Clinical Oncology Group (JCOG) were used 25 and three times, respectively. In 17 cases, the authors did not specify which grading system was used.

Late adverse events

A total of 31 studies provided any results of late AEs (26.1%). Definitions of acute and/or late AEs were provided in 30 studies. In 26 studies, the cut-off for late effects was 90 days (e.g. RTOG) and and in three studies it was 30 days. The rest did not clarify on what terms an AE was considered as a late AE. The minimum reported general follow-up was 10 months and the longest expanded to 107 months. Overall, 52 studies did not provide insight on how long patients were reassessed.

Grading

Of the 118 trials reporting AEs, they were graded in 114 trials; 94.12% reported grade 3, 92.44% grade 4 and 87.39% reported treatment-related deaths (grade 5), including the reports mentioning no deaths (figure 2). Lower grades than 3 were not commonly published. They were reported mostly for nonsurgical AEs for grade 1 and 2 in 53.78% and 63.03% of cases, respectively. In some cases, authors explicitly stated that specific side effects did not occur (interpreted as grade 0). This was in accordance with older studies with CTCAE version 2, when this grade existed or in its adaptation for newer studies. There was a difference in reporting between surgical and nonsurgical AEs (table 2). A Chi-squared test showed highly significant differences in AE reporting (p≤0.001) and AE grading (p≤0.001) Lower grades were more frequently reported for nonsurgical AEs. Most studies reported only severe AEs (grade 3–5).

FIGURE 2.

Reporting frequencies. Reporting frequency for each grade, divided into surgical and nonsurgical adverse events (AEs). “y/n” represents studies reporting occurrence or absence of certain AEs without any grading.

TABLE 2.

Comparison of radio(chemo)therapy- versus surgery-associated adverse events (AEs)

	Radio(chemo)therapy	Surgery
Studies n#	112	31
Reporting AEs	111 (99.1%)	21 (67.7%)
Grading	107 (95.5%)	9 (29.0%)
Only grade 5 reported	2 (1.7%)	3 (9.7%)
AE items in total n	1426	138
Reported acute AE/study	12.73±8.88 (0–65)	4.45±5.98 (0–28)

Data are presented as n (%) or Mean±sd (range), unless otherwise stated ^#: studies may be counted in both groups, according to their respective arms (e.g. comparing radiotherapy to surgery).

Mortality in surgery

While only 14 studies reported surgery-associated deaths, 10 mentioned a time span or specified when death occurred. Most frequently, in 8 studies, 30-day mortality was reported. In addition, one added 90-day mortality and two others had 120-day mortality. There have been two more studies describing deaths “while in hospital” and “6 months and later”.

Patient-reported outcomes

PROMs were reported in 10 studies [19]. The oldest trial including a quality of life questionnaire was published in 2004. Most frequently used questionnaires were the European Organisation for Research and Treatment of Cancer (EORTC) QLQ-C30, FACT-L and EORTC QLQ-LC14 (an updated edition of the original LC13).

Group/category-based results

Based on the permutation tests, we found two subgroup mean differences that were statistically significantly different (p<0.01): Phase II/III versus remaining studies and privately sponsored studies versus remaining studies (table 3). Phase II/III studies as well as privately sponsored studies reported a statistically significant higher average number of AEs than their respective out-of-the-subgroup studies. The results of the Mann–Whitney tests led to the same conclusions.

TABLE 3.

Reporting of adverse events (AEs) divided by category

	n	Mean AE/study	p-value	only >grade 3	no grading	<grade 3
1992–2008	58	10.95±5.87 (0–33)	0.0374	21	3	34
2009–2019	61	13.79±10.67 (0–65)	#0.0374	22	2	37
Phase II/III	75	13.80±9.18 (0–65)	0.0084	33	3	39
Not specified trials	44	10.02±7.47# (0–17)	#0.0084	10	2	32
AE as secondary end-point	68	11.91±6.99 (0–38)	0.2462	24	3	41
Not a secondary end-point	51	13.06±10.70# (0–65)	#0.2462	19	2	30
n<70	59	10.98±5.65 (1–24)	0.0388	12	2	45
n≥70	60	13.80±10.85 (0–65)	0.0551	31	3	26
Europe	40	11.25±6.42 (0–33)	0.1582	14	2	24
North America	30	15.63±12.24 (0–65)	0.0136	13	2	15
Asia	41	11.00±6.23 (1–25)	0.1016	11	1	29
Global	8	13.25±12.40 (4–40)	0.3411	5	0	3
Private sponsors	40	16.20±11.98 (1–65)	0.0003	17	0	23
Public sponsors	32	10.69±5.95 (0–25)	0.0941	14	2	16
Funding unknown	47	10.34±5.65 (0–23)	0.0153	12	3	32

^#: mean values for both subgroups formed show clear dependence due to their underlying distribution.

The same applies to three other subgroups (2009–2019, North American studies, n<70), with less significant differences (p<0.05) in average number of AEs. For the other groups we cannot reject the null hypothesis of equal underlying population distributions.

Discussion

The vast number of 231 different AEs, with only 18 of them being mentioned in at least a fifth of the studies, showed a large variety in reporting. Different grading systems for AE reporting exist, with partially different AE items. They all have five grades in common, which dates back to the grading system originally introduced by the WHO in 1979 [20]. In 1988 with the first appearance of the CTC-criteria (later CTCAE), an intergroup consensus on reporting was established, adopting the five grades. The terms “mild”, “moderate”, “severe” and “life-threatening” were consistently used with grade 1 to 4, therefore the studies stay comparable regardless of the classification system used. With the second version of the CTCAE, there was the development of a grading system for chemotherapy and radiotherapy as well. Only with the third version late effects were implemented, mostly from different sources as the RTOG scale or LENT-SOMA (which was seldomly used in our collective) [21]. The assessment however, remains physician-dependent and is therefore subjective and at risk of underreporting and/or misreporting [22]. Due to the nature of certain AEs, some remain subjective. To account for this, the PRO-CTCAE was introduced, defining categories amenable for patient-based self-reporting [23]. In conclusion, the CTCAE today offers an abundant catalogue, and the WHO and different collaboration groups depend on it today. Some authors of newer papers claiming to have used the WHO scale, without reference, may have therefore used the CTCAE. There is a high chance of underestimation of the true usage occurred in our review.

Regarding the differentiation between acute and late AEs, the clinically still often used “90-day-rule” was abandoned in 2006 with the introduction of version 3 of the CTCAE and left to the investigator's decision [18]. However, our review showed that there is a serious lack of providing a definition as well as reporting of late effects in general. We assume that in clinical work and therefore in studies, these 90 days still act as a reference point. Especially surgical studies usually define 30-day and 90-day mortality as common end-points. Several studies reported doubled mortality rates regardless of the extent of surgical resection, comparing 90-day to 30-day mortality indicating a high impact of timeline definition [24, 25]. In general, longer follow-up should be preferred; this applies to any treatment modality.

In stage III NSCLC, patients may qualify for different treatment options. Depending on initial stage, resection with neoadjuvant or adjuvant chemotherapy or definitive radiochemotherapy (potentially followed by immunotherapy) are standards of care [4, 5, 26]. Nodal stage or patients’ morbidity often sets limitation to resectability, shifting the recommendation to definitive chemoradiotherapy [3]. Providing prognostic information as well as most important side effects of the different treatments is crucial for a shared and informed decision. Haematological AEs (all blood cell lines) were most frequently reported AEs, mainly for chemotherapy. Radiation pneumonitis (66 studies) and oesophagitis (77 studies) were the most commonly reported as radiotherapy-associated acute AEs. In long-term observation, oesophageal stenosis, lung fibrosis and pneumonitis were most commonly listed. Surgical studies reported bronchial obstruction and fistulas (13 studies), empyema (9 studies), pneumonia (10 studies) and respiratory failure as main AEs (supplementary appendix C). Our study showed a lack of reporting of grade 1 and 2 AEs. They were reported in 53.78% and 63.03%, respectively of the included studies, mainly for nonsurgical AEs. Although grade 1 and 2 are seen by physicians as “tolerable” AEs, these side effects may highly impact the patient's decision-making process, which was impressively shown by Tol-Geerdink et al. [27]. Therefore, exact knowledge of side effects of oncological treatment is crucial. In our collective, only 27 studies (22.69%) reported whether patients did not have any AEs (equal to CTCAE “grade 0”).

There was a substantial difference in AE reporting, considering surgery related and nonsurgical AEs. While surgery-associated AEs, with 60 different items and only 67.7% of the studies reporting any AEs, perform poorly in our investigation, as many as 22 of them did not grade them properly. Nonsurgical AEs were more commonly reported with a total of 231 items. Surgical side effects are well known and especially in lower grades of little interest. Follow-up in surgical studies mostly is too short to report long-term AEs adequately. For oncological surgery, resection status and survival rates seem to be more of clinical interest. However, more and more studies appeared focussing on quality of life after surgery, showing a decline during short- and long-term follow-up [28–30]. Different AEs, such as chronic pain, anxiety/depression or general mental function are increasingly moving into focus. These AEs are often measured by using PROMs, which were initially designed for clinical trials. Therefore, it is not a surprise that these instruments gained in popularity, as the difference of effectiveness in treatments became smaller and smaller [31]. This interest in the clinical trial setting led to more frequent transfer of PROMs into daily practice [32, 33]. Different analyses, in trials and daily practice, showed high discrepancies of physician and patient-reported AEs and their respective severity [34, 35]. This discrepancy and intensified focus on patient centred care had an impact on trial design [36]. The EORTC initiated a consortium to provide recommendations on standardisation and analysis of quality of life end-points and other PROMs in clinical trials [37]. Our analysis shows a slightly higher reported mean of AE per study in newer studies from 2009 to 2019 (p=0.0374). This could be in correlation with a higher usage of PROMs and therefore, a larger number of structured AE assessments.

Around a quarter of the examined studies were publicly funded and one-third were privately sponsored. Another third of the collective did not declare any funds. To maximise the profit of this invested capital, the maximum of output in clinical research should be defined as goal (this includes an improvement in the quality of AE reporting). Although there are guidelines in using the CTCAE, no clear definition of quality of toxicity reporting exists. Besides completeness of toxicity reports, the coherence to the CTCAE grading and transfer of patients’ information into grade 1 to 5 need to be assessed carefully [22, 38]. During clinical examinations AEs are regularly assessed but not reported in the final publications, resulting in limited availability of existing data. Ideally, this awareness should be improved among investigators, sponsors as well as journal editors.

Although we showed important shortcomings of AE reporting, a number of limitations need to be considered. As in all reviews, selection bias may interfere with our findings. Namely potential selection bias of trials due to publication bias and search criteria, including language restrictions, may have distorted our collective of included studies. Some of the data concerning AE reporting were not available, as they were not published or not presented in their entirety. However, while this review certainly misses data, we extracted enough information providing a well-based overview. To our knowledge, this is the first detailed analysis of AE reporting in lung cancer with an extensive inclusion of trials. The findings were not only restricted to one treatment modality but also present the range of clinically relevant lung cancer treatments. Our findings were assessed and discussed in a multidisciplinary team, including radiation oncologists, surgeons and oncologists.

Conclusion

Even though the CTCAE provides an abundant and thorough catalogue of AEs and their clinical grading, overall reporting of AEs in stage III NSCLC for both radiotherapy/oncology and surgery was inconsistent and inhomogeneous. As all included studies were interventional, all of them may be expected to report AEs. Underreporting of AEs prohibits the extraction of patient-relevant information for decision-making and represents suboptimal use of invested resources.

Supplementary material

Please note: supplementary material is not edited by the Editorial Office, and is uploaded as it has been supplied by the author.

Appendices 00010-2020.supp ^{(623.2KB, pdf)}