Which criteria should we use to evaluate the efficacy of immune-checkpoint inhibitors?

Immune-checkpoint inhibitors (ICI) have innovated the treatment of many different types of advanced cancer. Two important distinctions between ICI and other modalities are durable response (DR) and pseudoprogression (1,2).

DR refers to long lasting tumor control, which is unavailable with conventional modalities. DR has been reported to occur in 10–20% of patients treated with ICI (3,4), and in some patients who achieve DR, relapse is not observed after treatment discontinuation (3).

Despite the remarkable effect of ICI in some patients, the majority of patients do not see a benefit from ICI. Selecting patients who will benefit from ICI is a major issue in the application of this treatment. Several biomarkers have been investigated to select patients before treatment. Programmed death ligand-1 (PD-L1) and tumor mutation burden are useful, but not perfect, markers (5). Other biomarkers (e.g., the lymphocyte/neutrophil ratio, lactate dehydrogenase, and carcinoembryonic antigen) have been explored, but show limited predictive value (6-8). Hence, determining criteria to assess the benefit of ICI during treatment is important.

A few patients treated with ICI respond with an initial increase in total tumor volume, a phenomenon termed “pseudoprogression” (9). The existing standard criteria for evaluating response in cancer clinical trials are the World Health Organization (WHO) criteria and the Response Evaluation Criteria in Solid Tumors (RECIST) (10,11); however, neither is adequately equipped to appropriately evaluate pseudoprogression. Because they cannot distinguish pseudoprogression from progressive disease (PD). Three new criteria have been proposed to solve this problem (Table 1). Wolchok et al. reported the immune-related response criteria (irRC), an improved version of the WHO criteria (2). IrRC requires evaluation of the two-dimensional tumor burden, which requires more effort than one-dimensional evaluation (12). Nishino et al. reported the immune-related response evaluation criteria in solid tumors (irRECIST), which combines the features of irRC and RECIST. IrRECIST requires only one-dimensional measurement and need to confirm to judge PD (13). IrRECIST has not always been applied in the same way, leading to concerns about the comparability of results across studies (14). Seymour et al. reported the immune response evaluation criteria in solid tumors (iRECIST), an improved version of RECIST 1.1 (14). In iRECIST, the measurements of the new lesion(s) are not incorporated into the tumor burden, which is the main difference from irRECIST. IRECIST is developed by consensus, and the relationship with prognosis has not been clearly evaluated (14).

Table 1. Features of criteria for immune-related responses.

Features	irRC	irRECIST	iRECIST	imRECIST
Source	Wolchok 2009	Nishino 2013	Seymour 2017	Hodi 2018
Model based on	WHO criteria	irRC & RECIST 1.1	RECIST 1.1	irRC & RECIST 1.1
Dimension	Two	One	Same as irRECIST	Same as irRECIST
Progressive disease definition	25% increase from the nadir	20% increase from the nadir	20% increase from the nadir; results in unconfirmed progressive disease; confirmation is necessary for confirmed progressive disease	Same as irRECIST
New lesion	The presence of new lesion(s) does not define progression; the measurements of the new lesion(s) are included in the sum of the measurements	Same as irRC	The presence of new lesion(s) does not define progression; the measurements of the new lesion(s) are not incorporated in tumor burden	Same as irRC
Confirmation	4 weeks	4 weeks	4 weeks; no longer than 8 weeks	4 weeks
Development cohort	Melanoma treated with ipilimumab	Advanced melanoma treated with ipilimumab	Consensus base	Advanced NSCLC and mUC treated with atezolizumab
Outcomes of development cohort	OS	irRC response	Not applicable	OS

irRC, immune-related response criteria; irRECIST, immune-related response evaluation criteria in solid tumors, iRECIST, immune response evaluation criteria in solid tumors; imRECIST, immune-modified response evaluation criteria in solid tumors; WHO, World Health Organization; NSCLC, non-small cell lung cancer; mUC, metastatic urothelial carcinoma; OS, overall survival.

The recent study published in the Journal of Clinical Oncology by Hodi et al. proposed the immune-modified response evaluation criteria in solid tumors (imRECIST) (15). They developed the criteria to evaluate the outcomes of patients treated with atezolizumab, which was reported for the first time at the American Society of Clinical Oncology Annual Meeting (16). They evaluated the relationship of imRECIST and overall survival (OS) in non-small cell lung cancer (NSCLC) and metastatic urothelial carcinoma, and the progression pattern in renal cell carcinoma and melanoma. ImRECIST is almost identical to irRECIST; indeed, the authors overlap. The novelties of imRECIST include a more detailed definition of progression-free survival and evaluation of the relationship of prognosis in several cancers.

In clinical trials, surrogate endpoints such as overall response rate or progression-free survival are evaluated for the purpose of predicting OS, which is the ultimate endpoint (17). Several studies have reported a relationship between the criteria and OS (Table 2) (14,18-20). They showed the difference between the overall response and OS by Kaplan-Meier curve in several cancers. Currently, iRECIST and imRECIST are the most promising criteria with respect to convenience. Because we have limited data in regard to tumor type and the evaluated settings in advanced cancers, we cannot draw conclusions as to which criteria are superior.

Table 2. External validation of criteria for immune-related response.

Source	Cohort	Number of participants	Treatment	Validated criteria
				RECIST 1.1	irRC	irRECIST	iRECIST	imRECIST
Hodi 2016	Advanced melanoma	327	Pembrolizumab	+	+
Khoja 2016	Advanced melanoma	37	Pembrolizumab	+	+
Kataoka 2017	Advanced NSCLC	143	Nivolumab	+	+
Tazait 2018	Advanced NSCLC	160	PD-1 or PD-L1 inhibitor	+		+	+

“+” means the criteria were validated in the article. RECIST, response evaluation criteria in solid tumors; irRC, immune-related response criteria; irRECIST, immune-related response evaluation criteria in solid tumors; iRECIST, immune response evaluation criteria in solid tumors; imRECIST, immune-modified response evaluation criteria in solid tumors; NSCLC, non-small cell lung cancer; PD-1, programmed death-1; PD-L1, programmed death-ligand 1.

One limitation for use of these criteria is that all the criteria were developed for use in clinical trials. In general patient care, we should prudent to consulting these criteria to stop administration of ICI.

Further evaluation to clarify the difference of necessary effort, predictive accuracy in other types of cancers, and other treatment sequences (e.g., neo-adjuvant) are warranted.