Intermediate endpoints as surrogates for outcomes in cancer immunotherapy: a systematic review and meta-analysis of phase 3 trials

Summary

Background

Cancer immunotherapy shows unique efficacy kinetics that differs from conventional treatment. These characteristics may lead to the prolongation of trial duration, hence reliable surrogate endpoints are urgently needed. We aimed to systematically evaluate the study-level performance of commonly reported intermediate clinical endpoints for surrogacy in cancer immunotherapy.

Methods

We searched the Embase, PubMed, and Cochrane databases, between database inception and October 18, 2022, for phase 3 randomised trials investigating the efficacy of immunotherapy in patients with advanced solid tumours. An updated search was done on July, 15, 2023. No language restrictions were used. Eligible trials had to set overall survival (OS) as the primary or co-primary endpoint and report at least one intermediate clinical endpoint including objective response rate (ORR), disease control rate (DCR), progression-free survival (PFS), and 1-year overall survival. Other key inclusion and exclusion criteria included: (1) adult patients (>18 years old) with advanced solid tumour; (2) no immunotherapy conducted in the control arms; (3) follow-up is long enough to achieve OS; (4) data should be public available. A two-stage meta-analytic approach was conducted to evaluate the magnitude of the association between these intermediate endpoints and OS. A surrogate was identified if the coefficient of determination (R²) was 0.7 or greater. Leave-one-out cross-validation and pre-defined subgroup analysis were conducted to examine the heterogeneity. Potential publication bias was evaluated using the Egger's and Begg's tests. This trial was registered with PROSPERO, number CRD42022381648.

Findings

52,342 patients with 15 types of tumours from 77 phase 3 studies were included. ORR (R² = 0.11; 95% CI, 0.00–0.24), DCR (R² = 0.01; 95% CI, 0.00–0.01), and PFS (R² = 0.40; 95% CI, 0.23–0.56) showed weak associations with OS. However, a strong correlation was observed between 1-year survival and clinical outcome (R² = 0.74; 95% CI, 0.64–0.83). These associations remained relatively consistent across pre-defined subgroups stratified based on tumour types, masking methods, line of treatments, drug targets, treatment strategies, and follow-up durations. No significant heterogeneities or publication bias were identified.

Interpretation

1-year milestone survival was the only identified surrogacy endpoint for outcomes in cancer immunotherapy. Ongoing investigations and development of new endpoints and incorporation of biomarkers are needed to identify potential surrogate markers that can be more robust than 1-year survival. This work may provide important references in assisting the design and interpretation of future clinical trials, and constitute complementary information in drafting clinical practice guidelines.

Funding

None.

Research in context.

Evidence before this study

Immunotherapy shows unique efficacy kinetics such as delayed clinical effect and long-term favorable outcome. Accordingly, conventional endpoints based on Response Evaluation Criteria in Solid Tumours (RECIST) criteria fail to represent the long-term benefit of immunotherapy. In the last two years, many of the US Food and Drug Administration (FDA) accelerated drug approvals based on objective response rate (ORR) or progression-free survival (PFS) study data have been withdrawn by FDA. Here we conducted a systematic search in Embase, PubMed and Cochrane databases for phase 3 randomised trials investigating the efficacy of immunotherapy in patients with advanced solid tumours from database inception to July 2023. The keywords included “cancer”, “immunotherapy”, “randomised trial” et al. No language restrictions were used. Totally, 37,244 relevant records were identified from the initial search.

Added value of this study

Our study revealed that, in cancer immunotherapy, there is a strong correlation between clinical outcome and 1-year milestone survival rate, but weak associations with other intermediate endpoints including ORR, disease control rate, and PFS. Moreover, these associations remained relatively consistent across pre-defined subgroups stratified based on tumour types, masking methods, line of treatments, drug targets, treatment strategies, and follow-up durations.

Implications of all the available evidence

Our findings have potential implications for the design and interpretation of clinical trials, which could subsequently accelerate the drug development process and assist in drafting the clinical practice guidelines. 1-year survival was the only identified surrogate endpoint to date for cancer immunotherapy, ongoing investigations and development of new endpoints and incorporation of biomarkers are needed to identify potential surrogacy that can be more robust than 1-year survival.

Introduction

The application of immune checkpoint inhibitors (ICIs) has revolutionised cancer treatment in the last decade.¹ Agents targeting cytotoxic T-lymphocyte-associated antigen 4 (CTLA-4), programmed cell death protein 1 (PD-1), and programmed cell death ligand 1 (PD-L1) can rehabilitate or activate self-immunity against tumour cells,² which result in delayed clinical effect and long-term favorable outcome.³ Currently, ICIs are widely used in clinical practice and become the standard treatments in multiple tumour types.^1,4 Moreover, the broad efficacy of ICIs has led to unprecedented levels of research of immunotherapy, such as the combination with other treatment,⁵ and the development of new immune-based targets.⁶

Overall survival (OS) is a universally recognised endpoint to determine the clinical benefit in oncology trial. However, the long natural histories of some tumours make it difficult to achieve enough follow-up. Accordingly, intermediate endpoints that could serve as surrogates for OS are needed to prioritise combinations, detect signals of early activity, and interpret exploratory results. Intermediate endpoints based on tumour measurement, such as objective response rate (ORR), disease control rate (DCR), progression-free survival (PFS), have been routinely applied to assess new therapies in clinical trials and served as the endpoints to predict OS for the consideration of accelerated approval of drugs. In immunotherapy, it is well-known that conventional approaches such as Response Evaluation Criteria in Solid Tumours (RECIST) cannot fully characterise the clinical benefit.⁷ Additionally, numerous trials revealed that early death could occur in the first several months of immunotherapy.⁸ The novel mechanisms and unique patterns of anti-tumour activities in immunotherapy have renewed great interest in exploring surrogate endpoints to assist in on/off decision-making. Intermediate endpoints, including ORR, DCR, PFS, modified PFS, and milestone survival, have been proposed as surrogate endpoints for immunotherapy in clinical trials.9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20 However, these studies were specific to one tumour type, one country/region, or with limited high-qualities trials, and the results were often ambiguous or conflicted due to the biological complexity of cancer. Here, though a comprehensive analysis with phase 3 randomised control trials (RCTs), our primary objective was to assess the study-level of ORR, DCR, PFS, and 1-year OS as surrogate endpoint for outcomes in cancer immunotherapy.

Methods

Search strategy and selection criteria

Our meta-analysis was performed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement.²¹ This study, and its associated protocol, is registered with PROSPERO, number CRD42022381648 (https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42022381648). A systematic search of PubMed, Embase, and Cochrane databases for trails investigating cancer immunotherapy and describing overall survival and at least one intermediate clinical endpoint from inception to October 18, 2022 was conducted. An updated search was done on July, 15, 2023. The keywords used were: cancer, immunotherapy, randomised trial, adebrelimab, atezolizumab, avelumab, camrelizumab, cemiplimab, dostarlimab, durvalumab, envafolimab, ipilimumab, nivolumab, pembrolizumab, relatlimab, sintilimab, sugemalimab, tremelimumab, toripalimab, and tislelizumab. The detailed search terms were shown in Table S1. All investigators carried out the initial search independently, carefully reviewed the title and abstract for relevance, and classified the potential articles as included, uncertain and excluded. For uncertain studies, the full-texts were reviewed for the confirmation of eligibility.

Both inclusion and exclusion criteria were pre-specified. To be eligible, studies had to meet the following criteria: (1) study design: phase 3 randomised trials irrespective of blindness, tumour type, and line of treatment, OS as the primary or co-primary endpoint. (2) population: adult patients (>18 years old) with advanced solid tumour. (3) intervention: random assignment of patients to immunotherapy (monotherapy or combination treatments) or control treatment irrespective of dosage and duration. The immunotherapy combination treatments included immunotherapy + chemotherapy, immunotherapy + targeted therapy, immunotherapy + radiotherapy, and different ICIs combination, while the control arms included all the conventional treatment but immunotherapy. (4) outcomes: OS and information regarding intermediate clinical endpoints, and follow-up is long enough to achieve its primary endpoint. Of note, we conducted a subgroup analysis based on median follow-up duration (<24 months vs. ≥ 24 months). Trials published online ahead of print were eligible, but meeting abstracts were excluded. When multiple publications of the same databases appeared or if there was a case mix between different publications, we removed the overlapping data and only the most recent and/or complete report was included. Studies were excluded if they were: (1) other studies on this topic, including review articles, conference abstract, editorials, pre-clinical papers, phase 1 or phase 2 trials, quality of life studies, and cost effectiveness analyses; (2) studies in the pediatric population, or patients with hematological disease; (3) data from unpublished studies; (4) subgroup or post hoc analyses of clinical trials.

Rik of bias of eligible trials were evaluated by the Cochrane risk of bias tools,²² which covered the following items: random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective reporting, and other bias.

When disagreements occurred in terms of study selection, data extraction, and risk of bias assessment, all investigators double checked the original data independently, and discuss the potential problems together. The discrepancies were resolved when all authors came to an agreement.

Data extraction and outcome measures

For surrogacy assessment, the control and experimental arms were predefined for each RCT. All authors independently extracted study-level information regarding study characteristics (study name, tumour type, masking method, line of treatment, and sample size), treatment strategy, median follow-up, clinical endpoints information regarding ORR, DCR, PFS, 1-year survival rate, and OS. Treatment effects on OS and PFS were presented as OS hazard ratio (HR_OS) and HR_PFS, respectively. Treatment effects on 1-year OS was expressed as Ratio_1y-OS as previously reported,¹² Ratio_1y-OS = 1-year OS rate in the control arm/1-year OS rate in the experimental arm. Treatment effects on objective response and disease control were presented as OR relative risk (RR_OR) and RR_DC. All analysis were conducted on intention-to-treat population.

Statistical analysis

Candidacy for surrogacy was assessed with a widely-accepted two-stage meta-analytical approach,^23,24 which required two conditions to evaluate the magnitude of the association or treatment effect estimates on the intermediate endpoint and OS. Condition 1 required a strong correlation between surrogacy and endpoint (ORR vs. Median OS, DCR vs. Median OS, Median PFS vs. Median OS, and 1-year survival rate vs. Median OS). Condition 2 required a strong correlation of treatment effects between the surrogacy and the clinical endpoint (RR_OR vs. HR_OS, RR_DC vs. HR_OS, HR_PFS vs. HR_OS, and Ratio_1y-OS vs. HR_OS). The strength of correlation was quantified with coefficient of determination (R²), weighting each trial by the sample size.²⁵ We used the TrialLevelMA function of R package Surrogate to calculate R² and its associated 95% CI (Surrogate: Evaluation of Surrogate Endpoints in Clinical Trials. https://CRAN.R-project.org/package=Surrogate). Additionally, as sensitivity analysis we also evaluated another weighting system based on the numbers of events reported or derived from each trial. According to the Systematic Review and Recommendation for Reporting of Surrogate Endpoint Evaluation using Meta-analysis (ReSEEM) guidelines,²⁶ R² ≥ 0.7 suggest strong correlations (and thus surrogacy), R² between 0.50 and 0.69 mean moderate correlations, and R² < 0.5 represent weak correlations. Leave-one-out cross-validation was conducted as a sensitivity analysis.²⁷ Funnel plots were generated to show any potential source of reporting bias. The 95% prediction interval for the regression line was developed by accessing the limits of the regression function over a sequence of possible values for the intermediate clinical endpoint, using the same trial level weights as for calculating R².

Pre-planned subgroups were assessed for the candidacy of each intermediate clinical endpoint, including tumour types, masking methods, line of treatment, treatment strategy, drug target, and the duration of median follow-up. No post hoc analyses were conducted in this study.

Potential publication bias was assessed by visual inspection of a funnel plot, and also evaluated using the Egger's and Begg's tests.^28,29 Two-sided p values < 0.05 were considered statistically significant. Data were acquired and analysed with MedCalc 18.2.1 and R 4.0.1 software.

Ethics

Ethics approval was not required because all data included in this study were publicly available de-sensitised data.

Role of the funding source

No funding was received.

Results

The initial search from Embase, PubMed and Cochrane databases identified 29,857 relevant records in October 18, 2022. We conducted a second search in July 2023 and found 7387 related manuscripts. After carefully screening and reviewing based on our inclusion and exclusion criteria, 77 phase 3 RCTs were eligible for the final analysis (Fig. 1). All data used for analysis were obtained from published manuscripts, and the baseline characteristics of eligible trials were illustrated in Table 1. Totally, 52,342 patients with cancer were enrolled, 29,294 were treated with ICIs, the rest 23,048 patients were in the controlled arms. The median age at the trial-level was 63 years old. 15 types of tumours were identified, including lung cancer (n = 29), gastric cancer/gastroesophageal junction cancer/esophageal cancer (GC/GEJC/EC, n = 10), renal cancer (n = 6), urothelial cancer (n = 5), hepatocellular cancer (n = 4), melanoma (n = 4), ovarian cancer (n = 3), breast cancer (n = 3), prostate cancer (n = 3), head and neck cancer (n = 3), glioblastoma (n = 2), cervical cancer (n = 2), endometrial cancer (n = 1), biliary tract cancer (n = 1), and mesothelioma (n = 1). A total of 89 comparisons were included because 10 studies had three arms; 2 trials had four arms. Immunotherapy were administrated as first-line of treatment in 50 trials, as second-line or later treatment in 26 RCTs. ICIs were applied as monotherapy in 37 trials, as immune-combination treatments in 48 studies. Treatment targeting CTLA-4 were found in 4 trials, PD-1 in 42 RCTs, PD-L1 in 28 studies, and the combination of PD-1/PD-L1 and CTLA-4 in 11 trials. The treatment effects of the eligible trials were shown in Table S2. Most trials (43/77, 55.84%) had over 24 months' median follow-up. Generally, the eligible RCTs had moderate or low risk of bias. The main issue affecting the method quality was lack of blinding giving 54 trials (70.13%) were open-labeled. Both Egger's and Begg's tests were conducted to evaluate the potential publication bias, and no significant bias was discovered (Figure S1).

Fig. 1.

Flowchart diagram of selected trials included in this study.

Table 1.

Baseline characteristics of eligible phase 3 trials.

Study	Underlying malignancy	Masking	Line of treatment	Treatment agents	No. of patients	Median age (range), years	Sex, m/f	Median follow-up, months
A367100930	Melanoma	Open label	1	Tremelimumab	328	57 (22–90)	190/138	>40.0
				Chemotherapy	327	56 (22–90)	182/145
ARCTIC31	Lung cancer	Open label	3+	Durvalumab	62	64 (35–79)	42/20	>18.0
				Chemotherapy	64	62 (41–81)	48/16
				Durvalumab + tremelimumab	174	63 (26–81)	115/59
				Chemotherapy	118	65 (42–83)	81/37
ATTRACTION-432,33	GC/GEJC	Double blind	1	Nivolumab + chemotherapy	362	64 (25–86)	253/109	26.6
				Placebo + chemotherapy	362	65 (27–89)	270/92
CA184-02434,35	Melanoma	Double blind	1	Ipilimumab + dacarbazine	252	58 (33–87)	152/98	36.6
				Placebo + dacarbazine	250	61 (31–76)	149/103
CA184-09536	Prostate Cancer	Double blind	1	Ipilimumab	400	70 (44–91)	400/0	<54.0
				Placebo	202	69 (42–92)	202/0
CA184-10437	Lung cancer	Double blind	1	Ipilimumab + chemotherapy	388	64 (28–84)	326/62	12.5
				Placebo + chemotherapy	361	64 (28–85)	309/52	11.8
CAPSTONE-138	Lung cancer	Double blind	1	Adebrelimab + chemotherapy	230	62 (55–66)	184/46	14.4
				Placebo + chemotherapy	232	62 (56–67)	188/44	12.8
CASPIAN39, 40, 41	Lung cancer	Open label	1	Durvalumab + tremelimumab + chemotherapy	268	63 (58–68)	202/66	25.1
				Durvalumab + chemotherapy	268	62 (58–68)	190/78
				Placebo + chemotherapy	269	63 (57–68)	184/85
CheckMate 9LA42,43	Lung cancer	Open label	1	Nivolumab + ipilimumab + chemotherapy	361	65 (59–70)	252/109	30.7
				Chemotherapy	358	65 (58–70)	252/106
CheckMate 01744, 45, 46, 47	Lung cancer	Open label	2	Nivolumab	135	62 (39–85)	111/24	69.5
				Docetaxel	137	64 (42–84)	97/40
CheckMate 02548, 49, 50	Renal cancer	Open label	2+	Nivolumab	410	62 (23–88)	315/95	72.0
				Everolimus	411	62 (18–86)	304/107
CheckMate 03751,52	Melanoma	Open label	2+	Nivolumab	272	59 (23–88)	176/96	24.0
				Chemotherapy	133	62 (29–85)	85/48
CheckMate 05744,53	Lung cancer	Open label	2	Nivolumab	292	61 (37–84)	151/141	69.4
				Docetaxel	290	64 (21–85)	168/122
CheckMate 06654, 55, 56, 57	Melanoma	Open label	1	Nivolumab	210	64 (18–86)	121/89	32
				Dacarbazine	208	66 (26–87)	125/83	10.9
CheckMate 07858, 59, 60	Lung cancer	Open label	2+	Nivolumab	338	60 (27–78)	263/75	>37.3
				Docetaxel	166	60 (38–78)	134/32
CheckMate 14161, 62, 63	Head and neck cancer	Open label	2	Nivolumab	240	59 (29–83)	197/43	>24.2
				Chemotherapy	121	61 (28–78)	103/18
CheckMate 21464, 65, 66, 67	Renal cancer	Open label	1	Nivolumab + ipilimumab	550	62 (26–85)	413/137	43.6
				Sunitinib	546	62 (21–85)	395/151	32.3
CheckMate 22768, 69, 70	Lung cancer	Open label	1	Nivolumab + ipilimumab	396	64 (26–84)	255/141	54.8
				Nivolumab	396	64 (27–85)	272/124
				Chemotherapy	397	64 (29–87)	260/137
CheckMate 45971	HCC	Open label	1	Nivolumab	371	65 (57–71)	314/57	15.2
				Sorafenib	372	65 (58–72)	317/55	13.4
CheckMate 49872	Glioblastoma	Open label	1	Nivolumab + radiotherapy	280	60 (18–83)	190/90	13.0
				Temozolomide + radiotherapy	280	56 (23–81)	175/105	14.2
CheckMate 54873	Glioblastoma	Double blind	1	Nivolumab + radiotherapy + temozolomide	358	60 (24–79)	205/153	>12.5
				Placebo + radiotherapy + temozolomide	358	60 (18–81)	197/161	>19.5
CheckMate 64874	EC	Open label	1	Nivolumab + chemotherapy	321	64 (40–90)	253/68	>13.0
				Nivolumab + ipilimumab	325	63 (28–81)	269/56
				Chemotherapy	324	64 (26–81)	275/49
CheckMate 64975,76	GC/GEJC/EC	Open label	1	Nivolumab + chemotherapy	473	63 (54–69)	331/142	>24.0
				Chemotherapy	482	62 (54–68)	349/133
CheckMate 64975,76	GC/GEJC/EC	Open label	1	Nivolumab + ipilimumab	234	62 (22–84)	NR	>35.7
				Chemotherapy	239	61 (23–90)	NR
CheckMate 74377,78	Mesothelioma	Open label	1	Nivolumab + ipilimumab	303	69 (65–75)	234/69	29.7
				Chemotherapy	302	69 (62–75)	233/69
CONTACT-0379	Renal cancer	Open label	2+	Atezolizumab + Cabozantinib	263	62 (20–85)	204/59	15.2
				Cabozantinib	259	63 (18–89)	197/62
COSMIC-31280	HCC	Open label	1	Cabozantinib + atezolizumab	432	64 (57–71)	360/72	13.3
				Sorafenib	217	67 (60–73)	186/31
DANUBE81	Urothelial cancer	Open label	1	Durvalumab	346	67 (60–73)	249/97	41.2
				Durvalumab + tremelimumab	342	68 (60–73)	256/86
				Chemotherapy	344	68 (60–73)	274/70
EMPOWER-Cervical 182	Cervical cancer	Open label	2	Cemiplimab	304	51 (22–81)	0/304	18.2
				Chemotherapy	304	50 (24–87)	0/304
EMPOWER-Lung 383	Lung cancer	Double blind	1	Cemiplimab + chemotherapy	312	63 (57–68)	268/44	16.3
				Chemotherapy	154	63 (57–68)	123/31	16.7
ESCORT-1st84	EC	Double blind	1	Camrelizumab + chemotherapy	298	62 (56–66)	260/38	10.8
				Placebo + chemotherapy	298	62 (56–67)	362/35
IMagyn05085	Ovarian cancer	Double blind	1	Atezolizumab + bevacizumab + chemotherapy	651	60 (29–84)	0/651	19.9
				Placebo + bevacizumab + chemotherapy	650	59 (18–83)	0/650	19.8
IMbassador25086	Prostate cancer	Open label	2	Atezolizumab + enzalutamide	379	70 (51–91)	379/0	15.2
				Enzalutamide	380	70 (40–92)	380/0	16.6
IMbrave15087,88	HCC	Open label	1	Atezolizumab + Bevacizumab	336	64 (56–71)	277/59	15.6
				Sorafenib	165	66 (59–71)	137/28
IMmotion15189,90	Renal cancer	Open label	1	Atezolizumab + Bevacizumab	454	62 (56–69)	317/137	>40.0
				Sunitinib	461	60 (54–66)	352/109
IMpassion13091, 92, 93	Breast cancer	Double blind	1	Atezolizumab + Nab-Paclitaxel	451	55 (20–82)	3/448	18.8
				Placebo + Nab-Paclitaxel	451	56 (26–86)	1/450
IMpower11094,95	Lung cancer	Open label	1	Atezolizumab	277	64 (30–81)	196/81	30.0
				Chemotherapy	277	65 (30–87)	193/84
IMpower13096	Lung cancer	Open label	1	Atezolizumab + chemotherapy	451	64 (18–86)	266/185	18.5
				chemotherapy	228	65 (38–85)	134/94	19.2
IMpower13197	Lung cancer	Open label	1	Atezolizumab + carboplatin + nab-paclitaxel	343	65 (23–83)	280/63	26.8
				Carboplatin + nab-paclitaxel	340	65 (38–86)	277/63	24.8
IMpower13298	Lung cancer	Open label	1	Atezolizumab + chemotherapy	292	64 (31–85)	192/100	28.4
				Chemotherapy	286	63 (33–83)	192/94
IMpower13399,100	Lung cancer	Double blind	1	Atezolizumab + chemotherapy	201	64 (28–90)	129/72	23.1
				Chemotherapy	202	64 (26–87)	132/70	22.6
IMpower150101, 102, 103, 104	Lung cancer	Open label	1	Atezolizumab + chemotherapy	402	63 (32–85)	241/161	38.8
				Atezolizumab + bevacizumab + chemotherapy	400	63 (31–89)	240/160	39.8
				Bevacizumab + chemotherapy	400	63 (31–90)	239/161	40.0
IMvigor211105,106	Urothelial cancer	Open label	2+	Atezolizumab	467	67 (33–88)	110/357	33.0
				Chemotherapy	464	67 (31–84)	103/361
IPSOS107	Lung cancer	Open label	1	Atezolizumab	302	75 (69–81)	108/43	41.0
				Chemotherapy	151	75 (68–80)	108/43
JAVELIN Bladder 100108	Urothelial cancer	Open label	1	Avelumab	350	68 (37–90)	266/84	>19.0
				Placebo	350	69 (32–89)	275/75
JAVELIN Gastric 100109	GC/GEJC	Open label	1	Avelumab	249	62	164/85	24.1
				Chemotherapy	250	61	167/83	24.0
JAVELIN Lung 200110,111	Lung cancer	Open label	2	Avelumab	396	64 (58–69)	269/127	18.9
				Docetaxel	396	63 (57–69)	273/123	17.8
JAVELIN Ovarian 200112	Ovarian cancer	Open label	2	Avelumab + chemotherapy	188	60 (53–67)	0/188	18.4
				Avelumab	188	61 (53–70)	0/188	18.2
				Chemotherapy	190	60 (53–69)	0/190	17.4
JAVELIN Renal 101113,114	Renal cancer	Open label	1	Avelumab + axitinib	270	62 (29–83)	203/67	19.3
				Sunitinib	290	61 (27–88)	224/66	19.2
KESTR EL115	Head and Neck cancer	Open label	1	Durvalumab	204	62 (26–89)	175/29	NR
				Durvalumab + tremelimumab	413	61 (25–87)	340/73
				Chemotherapy	206	61 (22–84)	174/32
KEYLYNK-010116	Prostate cancer	Open label	2+	Pembrolizumab + Olaparib	529	71 (40–89)	529/0	15.7
				Chemotherapy	264	69 (49–84)	264/0
KEYNOTE-010117, 118, 119	Lung cancer	Open label	3+	Pembrolizumab	690	63 (56–69)	425/265	67.4
				Docetaxel	343	62 (56–69)	209/134
KEYNOTE-033120	Lung cancer	Open label	2+	Pembrolizumab	213	61 (28–83)	157/56	22.3
				Chemotherapy	212	61 (34–81)	164/48
KEYNOTE-042121	Lung cancer	Open label	1	Pembrolizumab	637	63 (57–69)	450/187	12.8
				Chemotherapy	637	63 (57–69)	452/185
KEYNOTE-045122,123	Urothelial cancer	Open label	2	Pembrolizumab	270	67 (29–88)	200/70	27.7
				Chemotherapy	272	65 (26–84)	202/70
KEYNOTE-048124,125	Head and neck cancer	Open label	1	Pembrolizumab + chemotherapy	281	61 (55–68)	224/57	13.0
				Cetuximab + chemotherapy	278	61 (55–68)	242/93	10.7
KEYNOTE-062126	GC/GEJC	Partial-Blind	1	Pembrolizumab	256	61 (20–83)	180/76	29.4
				Pembrolizumab + chemotherapy	257	62 (22–83)	195/62
				Chemotherapy	250	63 (23–87)	179/71
KEYNOTE-063127	GC/GEJC	Open label	2	Pembrolizumab	47	61 (32–75)	32/15	24.0
				Paclitaxel	47	61 (37–91)	37/10
KEYNOTE-119128	Breast cancer	Open label	2+	Pembrolizumab	312	50 (43–59)	0/312	31.4
				Chemotherapy	310	53 (44–61)	2/308	31.5
KEYNOTE-189129, 130, 131	Lung cancer	Double blind	1	Pembrolizumab + chemotherapy	410	65 (34–84)	256/156	31.0
				Chemotherapy	206	64 (34–84)	109/97
KEYNOTE-240132	HCC	Double blind	2	Pembrolizumab	278	67 (18–91)	226/52	13.8
				Placebo	135	65 (23–89)	112/23	10.6
KEYNOTE-355133,134	Breast cancer	Double blind	1	Pembrolizumab + chemotherapy	220	52 (44–62)	0/220	44.1
				Chemotherapy	103	55 (43–63)	0/103
KEYNOTE-361135	Urothelial cancer	Open label	1	Pembrolizumab + chemotherapy	351	69 (62–75)	272/79	31.7
				Pembrolizumab	307	68 (61–74)	228/79
				Chemotherapy	352	69 (61–75)	262/90
KEYNOTE-407136,137	Lung cancer	Double blind	1	Pembrolizumab + chemotherapy	278	65 (29–87)	220/58	14.3
				Chemotherapy	281	65 (36–88)	235/46
KEYNOTE-426138,139	Renal cancer	Open label	1	Pembrolizumab + axitinib	432	62 (30–89)	308/124	30.6
				Sunitinib	429	61 (26–90)	320/109
KEYNOTE-590140	EC	Double blind	1	Pembrolizumab + chemotherapy	373	64 (28–94)	306/67	22.6
				Chemotherapy	376	62 (27–89)	319/57
KEYNOTE-604141	Lung cancer	Double blind	1	Pembrolizumab + chemotherapy	228	64 (24–81)	152/76	21.6
				Chemotherapy	225	65 (37–83)	142/83
KEYNOTE-775142	Endometrial cancer	Open label	2+	Pembrolizumab + lenvatinib	411	64 (30–82)	0/411	12.2
				Chemotherapy	416	65 (35–86)	0/416	10.7
KEYNOTE-826143	Cervical cancer	Double blind	1	Pembrolizumab + chemotherapy	308	51 (25–82)	0/308	22.0
				Chemotherapy	309	50 (22–79)	0/309
KEYNOTE-966144	Biliary tract cancer	Double blind	1	Pembrolizumab + chemotherapy	533	64 (57–71)	280/253	25.6
				Chemotherapy	536	63 (55–70)	272/264
MYSTIC145	Lung cancer	Open label	1	Durvalumab	163	64 (32–84)	113/50	30.2
				Durvalumab + tremelimumab	163	65 (34–87)	118/45
				Chemotherapy	162	65 (35–85)	106/56
NINJA146	Ovarian cancer	Open label	2+	Nivolumab	157	58 (29–84)	0/157	<48.0
				Chemotherapy	159	60 (34–80)	0/159
OAK147, 148, 149, 150, 151	Lung cancer	Open label	2+	Atezolizumab	425	63 (33–82)	261/164	47.7
				Docetaxel	425	64 (34–85)	259/166
ORIENT-15152	EC	Double blind	1	Sintilimab + chemotherapy	327	63 (57–67)	279/48	16.0
				Placebo + chemotherapy	332	63 (56–67)	288/44	16.9
PACIFIC153, 154, 155, 156	Lung cancer	Double blind	3+	Durvalumab	476	64 (31–84)	334/142	34.2
				Placebo	237	64 (23–90)	166/71
ORIENT-3157	Lung cancer	Open label	2	Sintilimab	145	61 (38–74)	136/9	23.6
				Chemotherapy	135	60 (34–75)	122/13
RATIONALE 303158	Lung cancer	Open label	2+	Tislelizumab	535	61 (28–88)	416/119	16.0
				Chemotherapy	270	61 (32–81)	206/64	10.7
RATIONALE 306159	EC	Double blind	1	Tislelizumab + chemotherapy	326	64 (59–68)	282/44	16.3
				Chemotherapy	323	65 (58–70)	281/42	9.8

CRC, colorectal cancer; EC, Oesophageal cancer; GC, gastric cancer; GEJC, gastroesophageal junction cancer; HCC, Hepatocellular cancer.

Objective responses were reported in 75 studies with 51,024 patients. There were weak associations between ORR and Median OS (Fig. 2A, R² = 0.06; 95% CI, 0.00–0.16) and between RR_OR and HR_OS (Fig. 2B, R² = 0.11; 95% CI, 0.00–0.24). Information regarding disease controls were presented in 64 trials with 43,109 individuals. Weak correlations were identified between DCR and Median OS (Fig. 2C, R² = 0.14; 95% CI, 0.00–0.30) and between RR_DC and HR_OS (Fig. 2D, R² = 0.01; 95% CI, 0.00–0.01). With data from 73 RCTs with 49,379 patients, we found weak associations between median PFS and median OS (Fig. 2E, R² = 0.18; 95% CI, 0.03–0.34) and between HR_PFS and HR_OS (Fig. 2F, R² = 0.40; 95% CI, 0.23–0.56). In contrast, strong correlations were observed between 1-year survival rate and Median OS (Fig. 2G, R² = 0.76; 95% CI, 0.67–0.85) and between Ratio_1y-OS and HR_OS (Fig. 2H, R² = 0.74; 95% CI, 0.64–0.83). To evaluate whether any single RCT was more influential in the trial-level correlation between Ratio_1y-OS and HR_OS, leave-one-out cross-validation by excluding 1 study at a time was conducted. The median R² from the cross-validation is 0.75 (range, 0.72–0.78). As sensitivity analysis we also evaluated the surrogate endpoints with another weighting systems, which was based on the numbers of events reported or derived from each trial (Table S3). As expected, the strongest correlations were observed between 1-year survival rate and Median OS (R² = 0.76) and between Ratio_1y-OS and HR_OS (R² = 0.73).

Fig. 2.

Associations of the treatment effects between intermediate clinical endpoints and overall survival in cancer immunotherapy. The association between (A) objective response rate and median overall survival; (B) objective response relative risk and overall survival hazard ratio; (C) disease control rate and median overall survival; (D) disease control relative risk and overall survival hazard ratio; (E) median progression-free survival and median overall survival; (F) progression-free survival hazard ratio and overall survival hazard ratio; (G) 1-year overall survival rate and median overall survival; and (H) 1-year overall survival ratio and overall survival hazard ratio. Each dot represents one eligible phase 3 trials, size of the dot is proportional to the sample size in the trial. Blue lines indicate the correlation (R²), gray shaded area indicates 95% prediction intervals.

To assess the robustness of our findings, we further conducted 6 pre-defined subgroup analyses including tumour types, masking method, line of treatment, drug target, treatment strategy, and follow-up duration (Table 2). Generally, the correlations of the treatment effects between intermediate clinical endpoints and OS remained relatively consistent across these subgroup analyses. Of note, PFS showed moderate association with OS in some specific conditions including GC/GEJC/EC tumour type, ICIs were applied as monotherapy or as second or later line of treatment, immunotherapy targeting PD-1, and median follow-up duration over 2 years. Additionally, for some unknown reason, 1-year survival showed weak association with outcomes when patients were treated with multiple ICIs.

Table 2.

Subgroup surrogacy analysis of intermediate clinical endpoints.

	Trials, n	Comparisons, n	Sample size, n	Correlation of treatment effects (R2, 95% CI)
				RROR vs. HROS	RRDC vs. HROS	HRPFS vs. HROS	Ratio1y-OS vs. HROS
Tumour types
Lung cancer	29	34	19,006	0.11 (0.00–0.36)	0.01 (0.00–0.14)	0.25 (0.03–0.53)	0.81 (0.64–0.90)
GC/GEJC/EC	9	12	6482	0.52 (0.06–0.84)	0.52 (0.05–0.85)	0.56 (0.09–0.86)	0.79 (0.41–0.94)
Urothelial cancer	5	7	4215	0.51 (0.01–0.91)	0.47 (0.02–0.90)	0.44 (0.01–0.68)	0.78 (0.15–0.97)
Masking
Open-label	54	65	37,336	0.14 (0.02–0.32)	0.00 (0.00–0.08)	0.45 (0.25–0.63)	0.71 (0.57–0.81)
Double-blind	22	23	14,750	0.05 (0.00–0.36)	0.14 (0.00–0.54)	0.53 (0.20–0.77)	0.81 (0.60–0.92)
Line of treatment
1	50	60	36,339	0.09 (0.02–0.25)	0.00 (0.00–0.09)	0.32 (0.12–0.52)	0.70 (0.54–0.81)
>1	26	28	15,254	0.17 (0.00–0.46)	0.00 (0.00–0.18)	0.57 (0.28–0.78)	0.85 (0.69–0.92)
Treatment strategy
Monotherapy	37	37	21,154	0.25 (0.00–0.52)	0.02 (0.00–0.23)	0.53 (0.24–0.74)	0.84 (0.70–0.91)
Combination treatment	48	52	32,850	0.02 (0.01–0.15)	0.01 (0.00–0.14)	0.39 (0.17–0.59)	0.67 (0.49–0.81)
Drug target
PD-1	42	44	26,480	0.13 (0.01–0.36)	0.25 (0.04–0.51)	0.57 (0.35–0.74)	0.74 (0.56–0.85)
PD-L1	28	30	18,262	0.11 (0.00–0.38)	0.05 (0.00–0.30)	0.43 (0.14–0.69)	0.79 (0.61–0.89)
PD-1/PD-L1+CTLA-4	11	11	6794	0.00 (0.00–0.41)	0.36 (0.00–0.81)	0.40 (0.00–0.85)	0.25 (0.00–0.75)
Median follow-up
>24 months	43	52	30,221	0.22 (0.03–0.46)	0.35 (0.10–0.60)	0.40 (0.15–0.63)	0.72 (0.52–0.84)
≤24 months	33	35	21,298	0.03 (0.00–0.24)	0.02 (0.00–0.23)	0.63 (0.41–0.81)	0.78 (0.60–0.89)

CI, confidential interval; CTLA-4, cytotoxic T lymphocyte antigen-4; DC, disease control; EC, oesophageal cancer; GC, gastric cancer; GEJC, gastroesophageal junction cancer; HR, hazard ratio; OR, objective response; OS, overall survival; PD-1, programmed death-1; PD-L1, programmed death ligand 1; PFS, progression-free survival; RR, relative risk.

Discussion

Based on 52,342 patients with 15 types of tumours enrolled in 77 phase 3 randomised trials, our study reveals that 1-year milestone survival is a strong surrogate for clinical outcomes in cancer immunotherapy. In contrast, other intermediate RECIST-based endpoints including ORR, DCR, and PFS, show weak associations with overall survival. These results may provide important references in accelerating the drug development process, assist in design and interpretation of clinical trials, and constitute complementary information in drafting the clinical practice guideline.

Endpoints based on RECIST criteria has been commonly applied as the surrogacy for OS to reduce the follow-up duration, the sample size, and the cost of clinical studies.¹⁶⁰ However, it is well-established that, compared with other conventional treatments, immunotherapy shows unique efficacy kinetics such as delayed clinical effect and long-term favorable outcome.³ These characteristics may lead to the prolongation of trial duration and make it difficult to accelerate the drug development process.¹⁶¹ Consist with previous studies,18, 19, 20 here our data demonstrated that the conventional RECIST-based endpoints, including ORR, DCR and PFS, failed to represent the long-term benefit of immunotherapy. To further evaluate the robustness of these results, we conducted the sensitivity analysis using pre-defined subgroups, and could not identified any strong correlation between the RECIST-based endpoints and overall survival. Even in some cases PFS showed moderate associations with OS (Table 2), these associations usually had a wide range of confidential intervals, highlighted the inconsistency of RECIST-based endpoints. It was reported that ICIs could increase immune cell infiltration by activate the cytotoxic T-cell response, which might result in the initial upregulation of tumour volume or occurrence of new lesions followed by the shrinkage (pseudo-progression), or a severe and rapid pattern of progression (hyper-progression).¹⁶² The weak associations between OS and RECIST-based endpoints in our analysis might partly be due to the pseudo-progression and/or hyper-progression. It should be noted that many phase 3 randomised trials still set PFS as the primary solo-endpoint currently.163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174 Moreover, though Accelerated Approval Program, FDA had granted the application of ICIs in different types of tumours based on ORR or PFS.¹⁷⁵ In the last two years, many of these accelerated approvals have been withdrawn and no longer FDA-approved (https://www.fda.gov/drugs/resources-information-approved-drugs/withdrawn-cancer-accelerated-approvals).

Given that patients with modest or even absent PFS benefit may still gain long-term favorable outcomes from immunotherapy,¹⁷⁶ several proposals aiming to optimise the existing RECIST criteria were emerging. A modified RECIST (mRECIST) was developed to improve response assessment that may predict outcomes.¹⁷⁷ While RECIST defined progression by an increase in tumour diameters, mRECIST progression only required the disappearance or reduction of intra-tumoural arterial enhancement in tumour lesions. Immune-Related Response Criteria (irRC) combined the new lesions into the total tumour burden and included additional patterns of tumour response that appear after initial tumour expansion.¹⁷⁸ In 2017, the RECIST Working Group recommend iRECIST guideline to assess the tumour response in immunotherapy studies.¹⁷⁹ More recently, modified PFS, which omitted the events of progression within 3 months after randomisation, showed strong correlation with OS at both trial-level and patient-level.²⁰ However, the robustness of these consensus guidelines needed further independent validation. In the present study, since only a small number of the eligible phase 3 trials reported both OS and these intermediate endpoints, it is not feasible to evaluate the powers of them as surrogacies for clinical outcomes in cancer immunotherapy.

We explored milestone survival analysis due to the following reasons. First, traditional metrics of response and progression by RECIST are insufficient in characterising the clinical benefit or prioritising immune-combination treatment. Additionally, patients treated with ICIs demonstrate unique pattern of response and prognosis. On the trial level, these patterns led to the delayed separation of survival curves beyond the median, non-proportional curves, and a subgroup of individuals can achieve long-term benefit. Furthermore, it is well-established that milestone analyses had several advantages including simplicity of analysis and predictability (time driven rather than event driven). The relative and absolute differences of the outcomes can be measured in the context of clinical studies. An example demonstrated that the long-term survival rate of approximately 15% between two treatments led to an additional 2-year follow-up.¹⁶¹ Moreover, milestone measured at a mature time point can avoid some conditions such as the survival curves are non-proportional or separate late. In some circumstances, the milestone survival rate may capture the sub-population who can derive favorable outcomes.¹⁷⁶ Accordingly, the role of milestone survival at a given timepoint has been descriptive in nature in most studies.

One of the difficulties with milestone survival analysis is in milestone selection. The delayed clinical effect often occurred within 6 months after the application of ICIs, and the crossover of Kaplan–Meier curves may appear later. If the milestone placed at a timepoint before the treatment took effect, the potential benefit of immunotherapy would not have been discovered. Based on the clinical data derived from ipilimumab development program in which the OS were relatively stable beyond two years, a 2-year milestone was chosen in the retrospective real-time analysis.¹⁶¹ In 2017, a milestone survival study suggested that 1-year OS rate was a surrogate endpoint for non-small cell lung cancer (NSCLC). However, this investigation was conducted on trials with different treatment strategies including chemotherapy, targeted therapy, and immunotherapy.¹² Our study is unique since we only included phase 3 immunotherapy trials, and we examined the correlation between 1-year survival and OS in several subgroups to validate our results. Since the biggest challenge with milestone survival analysis is the difficulty in maintaining the study integrity after the milestone timepoint, we specifically analysed the trials with follow-up over 2 years (Table 1). The correlation between 1-year milestone survival and OS remained robust in this subgroup.

Metastasis or recurrence is an important step in tumour progression, and has long been known as the overwhelming majority of cancer-related deaths.¹⁸⁰ According, metastasis-free survival (MFS) and recurrence-free survival (RFS) have been examined as surrogates for outcomes in numerous tumours, and show strong correlation with OS in nasopharyngeal carcinoma,¹⁸¹ prostate cancer,¹⁸² colorectal cancer,¹⁸³ melanoma.¹⁸⁴ Unfortunately, no or very few immunotherapy trials involved in these studies. In 2020, Goart et al. evaluated the performance of relapse-free survival as a surrogate in adjuvant therapy of melanoma treated with ipilimumab,¹⁸⁵ and found a strong association at the patient-level but only moderate association at the trial-level. It should be noted that only 264 patients from EORTC trial were investigated in this study. Accordingly, more solid evidences were needed to confirm the strength of this association. Since the first ICI was approved only over 10 years ago,¹⁸⁶ currently most available trials mainly focused on patients with late-stage tumours. Accordingly, it is very difficult to extract enough information regarding MFS and RFS in our analysis. However, ICIs were widely used in clinical practice nowadays and became the standard first-line or second-line treatment in multiple malignancies. We believe a systematic analysis on the correlation between MFS or RFS and OS can be conducted in the near future.

Our study has several strengths and clinical implication. We conducted a comprehensive meta-analysis and included the most up-to date published phase 3 RCTs of immunotherapy across multiple advanced solid tumours with over 50,000 patients. Hence our study had enhanced statistical power, which provide more reliable and precise estimates. Moreover, since agents targeting different checkpoints were examined in a heterogeneous population, we could investigate for variability in outcomes and improved the generalisability of our conclusion. Second, we tested all the commonly used surrogate endpoints in cancer trials including ORR, DCR, PFS, and 1-year survival, which meant our result were more extensive and valid than others. Third, the correlation between 1-year milestone survival and OS remained robust based on various classification criteria. With the application of 1-year milestone survival, further studies might need more patients, more resources, and longer follow-up to obtain these data. Even so, considering the relatively low success rate of phase 3 trials in cancer research,¹⁸⁷ and the increasing number of trials using ICIs in tumours at early stage, the application of this surrogate is still cost-effective in guiding the selection of agents for a more time-consuming and costly study.

This study has several limitations. First, it was well-known that an optimal surrogacy should fulfill the condition of a strong correlation with the outcome at both the trial and patient level.¹⁸⁸ The patient level association does not always consist with the results derived from the trial levels. For example, the pathological complete response is a surrogate endpoint in neoadjuvant studies of early-stage breast cancer at patient level,¹⁸⁹ but not at trial level.¹⁹⁰ Here, we were unable to access to the individual patient data and hence could not examine patient-level association. Second, comprehensive and reliable data on cross over between experimental arms and salvage immunotherapy for control arm patients are unclear in most trials, which could impact the robustness of surrogate endpoints. Currently, we cannot conduct sensitivity analysis according to the presence of cross over. However, considering patients are transferred from the control arms to the experimental arms in most cases, these crossovers are likely to weaken the power of the association between intermediate endpoints and OS. Third, the immune-related response evaluation criteria lack consensus and are not widely used in these eligible trials. Hence, we did not evaluate the power of immune-related ORR, DCR, and PFS as surrogate endpoints here. Fourth, the heterogeneity across studies may not be fully explained. Some confounders, such as the different properties of ICIs, study designs, patient population, and the distribution of tumour stage, can also be the source of the heterogeneity. Further analysis is needed to determine their influences on the robustness of surrogate endpoints. Fifth, the 1-year milestone survival analysis is a cross-sectional analysis and only considers a given timepoint as the primary endpoint. It cannot account for the totality of the OS times or the effects of censoring before this timepoint. Lastly, the included studies were conducted at different hospitals by various clinicians, who might be associated with some subjectivity and have potential bias in reporting these intermediate endpoints. Our study is subject to any errors or biases of the original researchers, and the results are generalisable only to the patient eligible for these trials.

In summary, our study revealed there is a strong correlation between clinical outcome and 1-year milestone survival rate in cancer immunotherapy.

Contributors

BZ conceived and designed the study. ZZ, QP, ML and BZ developed the protocol and performed the data analysis. ZZ, QP and BZ collected data. ZZ, QP, ML and BZ make the figures. BZ, ZZ and QP wrote the manuscript. BZ and ZZ accessed and verified the underlying data. BZ supervised this work. All authors read and approved the final manuscript. BZ has accessed and verified the data and responsible for the decision to submit the manuscript.

Data sharing statement

All original data discussed in the text of this paper is also included as figure and/or table either in the main text or in the appendix. No additional data are available.

Declaration of interests

We declare no competing interests.

Appendix A. Supplementary data

Figure S1.