EHA evaluation of the ESMO—Magnitude of Clinical Benefit Scale version 1.1 (ESMO-MCBS v1.1) for haematological malignancies

Abstract

Objective

Value frameworks in oncology have not been validated for the assessment of treatments in haematological malignancies, but to avoid overlaps and duplications it appears reasonable to build up experience on existing value frameworks, such as the European Society for Medical Oncology—Magnitude of Clinical Benefit Scale (ESMO-MCBS).

Methods

Here we present the results of the first feasibility testing of the ESMO-MCBS v1.1 for haematological malignancies based on the grading of 80 contemporary studies for acute leukaemia, chronic leukaemia, lymphoma, myeloma and myelodysplastic syndromes. The aims were (1) to evaluate the scorability of data, (2) to evaluate the reasonableness of the generated grades for clinical benefit using the current version and (3) to identify shortcomings in the ESMO-MCBS v1.1 that require amendments to improve the efficacy and validity of the scale in grading new treatments in the management of haematological malignancies.

Results

In general, the ESMO-MCBS v1.1 was found to be widely applicable to studies in haematological malignancies, generating scores that were judged as reasonable by European Hematology Association (EHA) experts. A small number of studies could either not be graded or were not appropriately graded. The reasons, related to the differences between haematological and solid tumour malignancies, are identified and described.

Conclusions

Based on the findings of this study, ESMO and EHA are committed to develop a version of the ESMO-MCBS that is validated for haematological malignancies. This development process will incorporate all of the usual stringencies for accountability of reasonableness that have characterised the development of the ESMO-MCBS including field testing, statistical modelling, evaluation for reasonableness and openness to appeal and revision. Applying such a scale will support future public policy decision-making regarding the value of new treatments for haematological malignancies and will provide insights that could be helpful in the design of future clinical trials.

Key questions.

What is already known about this subject?

• The European Society for Medical Oncology—Magnitude of Clinical Benefit Scale (ESMO-MCBS) v1.1 is a validated value scale for solid tumour oncology, but it has not yet been evaluated for the use in haematological malignancies.

What does this study add?

• Here, we present the results of the first feasibility testing of the ESMO-MCBS v1.1 for haematological malignancies based on grading of 80 contemporary studies for leukaemia, lymphoma, myeloma and myelodysplastic syndromes.

• The ESMO-MCBS v1.1 was found to be widely applicable to studies in haematological malignancies, generating scores that were judged as reasonable by European Hematology Association (EHA) experts; however, a small number of studies could either not be graded or were not appropriately graded because of shortcomings related to the differences between haematological and solid tumour malignancies.

How might this impact on clinical practice?

• Based on the findings of this study, ESMO and EHA are committed to develop a version of the score that is robustly validated to grade studies in malignant haematology.

Introduction

In recent years, rapid developments in haematology research resulted in a considerable expansion of treatment options. The development of instruments to measure clinical benefit is essential in the current scenario where increasing numbers of treatments for haematological malignancies (HMs) are becoming available, often targeting a small and defined subpopulation of patients. For this, several value frameworks have been published by different organisations and institutions taking into account or emphasising different aspects contributing to such an evaluation.1 These frameworks vary in terms of their definition of value, target audience and methodology, and each of them has specific limitations, which should be taken into consideration when interpreting their outputs.2 Until now, value frameworks developed in oncology have not been validated in the setting of HMs.

The European Society for Medical Oncology (ESMO) has developed such a value framework called the ESMO—Magnitude of Clinical Benefit Scale (ESMO-MCBS).3 Initially published in 2015, the scale is a validated and reproducible tool in solid tumour oncology with a particular focus on the clinical benefit. The ESMO-MCBS was developed to generate clear, valid and unbiased grading of the magnitude of clinical benefit demonstrated in therapeutic studies that could be used for a number of purposes including public health policy and health technology assessment (HTA), clinical decision-making, medical publication and journalism. The ESMO-MCBS grading highlights those treatments which substantially improve the duration of survival and/or the quality of life (QOL) of patients with cancer and aims to distinguish them from trials demonstrating more limited and sometimes even marginal benefits. The ESMO-MCBS was revised (version 1.1) in 2017, based on feedback and queries from clinicians, patients, researchers and representatives of the pharmaceutical industry, and a dynamic process of internal peer review.4 Version 1.1 incorporates 10 revisions and most importantly allows also for scoring of single-arm studies. The ESMO-MCBS assigns categorical benefit scores to European Medicines Agency (EMA) approved drugs, based on results from ‘positive’ randomised clinical trials: (1) superiority trials that have demonstrated a statistically significant result for the primary endpoint of the study, or secondary in case of overall survival (OS) and (2) non-inferiority trials, reaching a conclusion of non-inferiority. Primary or secondary endpoints included in the scoring system are OS, progression-free survival (PFS), QOL, treatment toxicity or response rates. In developing the ESMO-MCBS scale, ESMO aspired to meet standards for ‘accountability for reasonableness’,5 6 incorporating extensive field testing, statistical modelling7 and peer review of the ‘reasonableness’ of the generated results into the development process. The ESMO-MCBS is currently incorporated in ESMO’s clinical practice guidelines and is being used as part of HTA processes.8 9

The European Hematology Association (EHA) and ESMO have developed a joint initiative to develop a version of the ESMO-MCBS that is validated for HMs. As a first step in this process, we have field tested the current version of the ESMO-MCBS (version 1.1) across a wide spectrum of HMs. The aims of this evaluation were (1) to evaluate the scorability of data derived from contemporary clinical trials in HMs, (2) to evaluate the reasonableness of the generated grades for clinical benefit using the current version and (3) to identify shortcomings in the ESMO-MCBS v1.1 that require amendments to improve the efficacy and validity of the scale in grading new treatments in the management of HMs.

Methods

Study selection

The corresponding disease-oriented EHA scientific working groups identified experts who selected representative treatments currently used in clinical practice with a focus on recently approved drugs and novel strategies, to be evaluated for each of the common haematological malignancies: acute lymphoblastic leukaemia (ALL), acute myeloid leukaemia (AML), chronic lymphocytic leukaemia (CLL), chronic myeloid leukaemia (CML), Hodgkin and non-Hodgkin lymphomas, multiple myeloma (MM) and myelodysplastic syndromes (MDS). The treatments selected underwent a literature search to identify corresponding clinical trials and data.

ESMO-MCBS grading

Identified studies were graded by members of the EHA scientific working groups according to the ESMO-MCBS v1.1 forms4 in accordance with the instructions provided by ESMO. Magnitude of clinical benefit scores range from A to C for treatment strategies with curative intent and 5-1 for treatments with non-curative intent, with scores of A–B and 5-4 relating to a substantial level of clinical benefit. Initial grading by the expert groups were reviewed by the ESMO-MCBS working group for applicability and correctness.

Evaluations

For each disease entity, we evaluated the scorability of the evaluated studies and the reasonableness of the derived scores. Based on these findings, we identified shortcomings in the current version of the ESMO-MCBS that either precluded scoring or which generated grading which was considered not to be a reasonable estimation of benefit when such studies were identified.

Results

The extensive research concluded in 80 studies, 5 of which had either more than two arms or different publications for the same trial presenting results after longer follow-up times (87 studies and/or comparisons in total). In detail, we have scored 7 studies for AML, 5 studies for ALL, 8 studies for CLL, 4 studies for CML, 23 studies for non-Hodgkin and Hodgkin lymphoma, 23 studies for MM and 10 studies for MDS. The ESMO-MCBS v1.1 tool was applied in all the 87 distinct studies and/or subgroups.

Acute myeloid leukaemia

Studies evaluated: Seven studies were evaluated,10–16 three in a curative setting and four in a non-curative setting (table 1).

Table 1.

Feasibility testing of the ESMO-MCBS v1.1 for acute myeloid leukaemia (n=7)

Medication	Trial Name	Setting	Primary Outcome	PFS/EFS/ DFS Control	PFS/EFS/DFS Gain	PFS/EFS/DFS HR	OS Control	OS Gain	OS HR	RR (DOR)	QOL	Toxicity	ESMO-MCBS score	ESMO-MCBS form	Reference
SOC± midostaurin	RATIFY	Upfront, FLT3-mutated	OS	15.5 months (DFS)	11.2 months		25.6 months	49.1 months	0.78 (0.63– 0.96)				A	1	10
																SOC±gemtuzumab ozogamicin	ALFA-0701	Upfront, 50–70 years	EFS	17.1% 2 years	23.7%	0.58 (0.43– 0.78)	41.9% 2 years	11.3%	0.69 (0.49–0.98) Immature			Increased	A	1	11
SOC±sorafenib (+maintenance)	SORAML	Upfront	EFS	22% 3 years	18%	0.64 (0.45– 0.91)	56% 3 years	7%	Immature			Slightly increased	A	1	12
																Azacitidine versus SOC	AZA-001	Upfront elderly, low blast count	OS				16 months	8.5 months	0.47 (0.28–0.79)			Benefit (+1 point)	5	2a	13
Decitabine versus SOC	DACO-016	Upfront, elderly, intermediate/poor risk	OS				5 months	2.7 months	0.82 (0.68–0.99)				2	2a	14
LDAC ± Volasertib		Upfront, unfit	ORR	2.3 months EFS	3.3 months	0.57 (0.35–0.92)	5.2 months	2.8 months	0.63 (0.40– 1.00)	31% vs 13%, gain 18%		Slightly increased	3	2a	15
																Enasidenib		IDH2 mutated, relapsed/refractory	ORR				3.3 months (historical)			40.3% (5.8 months)			2	3	16

Across all tables, in case there is reported information for multiple endpoints, the evaluated endpoint results are indicated with bold.

DFS, disease-free survival; DOR, duration of response; EFS, event-free survival;ESMO-MCBS v1.1, European Society for Medical Oncology—Magnitude of Clinical Benefit Scale, version 1.1; FLT3, fms-like tyrosine kinase 3; IDH2, isocitrate dehydrogenase 2; LDAC, low-dose cytarabine; ORR, overall response rate; OS, overall survival;PFS, progression-free survival; QOL, quality of life; RR, response rate; SOC, standard of care.

Scorability: All studies were published with endpoints and data applicable to the ESMO-MCBS v1.1.

Reasonableness: The separation of studies with curative/non-curative intent corresponds closely to the distinction between intensive versus non-intensive chemotherapy regimens which are the terms usually applied in the treatment of AML. Grading effectively distinguished between high benefit treatment strategies in a curative setting and stratified between higher and lower benefit treatments in a non-curative setting.

Shortcomings: None identified.

Acute lymphoblastic leukaemia

Studies evaluated: Five studies were evaluated,17–23 and these included studies relating to three agents recently approved by EMA for relapsed and refractory ALL (table 2).17–20 22

Table 2.

Feasibility testing of the ESMO-MCBS v1.1 for acute lymphoblastic leukaemia (n=5)

Medication	Trial name	Setting	Primary outcome	PFS/EFS control	PFS/EFS gain	PFS/EFS HR	OS control	OS gain	OS HR	RR (DOR)	QOL	Toxicity	ESMO-MCBS score	ESMO-MCBS form	Reference (s)
Blinatumomab versus SOC	TOWER	Relapsed/refractory	OS	12% EFS 6 months	19%	0.55 (0.43–0.71)	4 months	3.7 months	0.71 (0.55– 0.93)	44% vs 25% CRR, gain 19%	Improved (+1 point)		5	2a	17 18
																Inotuzumab ozogamicin versus SOC	INO-VATE	Relapsed/refractory	OS/CRR	1.8 months	3.2 months	0.45 (97.5% CI: 0.34 to 0.61)	6.7 months (10% gain in 2-year survival)	1 month (13% gain in 2-year survival)	0.77 (97.5% CI: 0.58 to 1.03) p= 0.04	81% vs 29% CRR, gain 52%	Improved	Veno-occlusive disease 11% in experimental arm	4*	2a	19 20
Hyper-CVAD + ponatinib		Philadelphia chromosome-positive, upfront. Phase II single arm	EFS	81% 2 years EFS			80% 2 years						Not scoreable		21
																CAR T-cell tisagenlecleucel		Relapsed/refractory, age <21 years, single arm	ORR at 3 months				76% 1 year			81% ORR		>30% grade 3/4 cytokine release syndrome	3	3	22
Ponatinib	PACE	Philadelphia positive resistant to or side effects with dasatinib or nilotinib, or T315I mutation after TKI	Major haematological response within the first 6 months		7% at 12 months			40% at 12 months		Major haematological response: 41% (3 months)			2	3	23

*Based on >10% increase in 2 years of OS improvement.

CAR T- cell, chimeric antigen receptor T-cell therapy; CRR, complete remission rate; DOR, duration of response; EFS, event-free survival;ESMO-MCBS v1.1, European Society for Medical Oncology—Magnitude of Clinical Benefit Scale, version 1.1; Hyper-CVAD, hyperfractionated cyclophosphamide, vincristine, doxorubicin and dexamethasone; ORR, overall response rate; OS, overall survival;PFS, progression-free survival; QOL, quality of life; RR, response rate; SOC, standard of care; TKI, tyrosine kinase inhibitor.

Scorability: Four of the five studies were published with endpoints and data applicable to the ESMO-MCBS v1.1. The only not scoreable study was the single-arm study of ponatinib as add-on to standard of care upfront treatment with curative intent.21

Reasonableness: Both the first-in class bispecific antibody blinatumomab (TOWER trial)17 18 and the antibody-drug conjugate inotuzumab ozogamicin (INO-VATE trial)19 20 reached high scores based on positive OS data and favourable QOL data for blinatumomab (ESMO-MCBS v1.1 scores 5 and 4, respectively). The chimeric antigen receptor (CAR) T-cell treatment in children/young adults with relapsed or refractory B-cell ALL was graded with maximal credit of 3 for a single-arm study in a non-curative setting.22 The ponatinib treatment (single-arm PACE trial)23 was assigned grade 2 based on the major molecular response (MMR) in the non-curative setting.

Reasonableness: Grading effectively distinguished between high benefit treatment strategies in a curative stetting and stratified between higher and lower benefit treatments in a non-curative setting.

Shortcomings: One shortcoming was identified:

1. The ESMO-MCBS v1.1 does not have a form to grade single-arm treatments with curative intent. This shortcoming precluded scoring in one study21 and may also have been relevant to the grading of CAR T-cell salvage therapy which could also be considered as curative.22

Chronic lymphocytic leukaemia

Studies evaluated: Eight studies were evaluated (table 3).24–35

Table 3.

Feasibility testing of the ESMO-MCBS v1.1 for chronic lymphocytic leukaemia (n=8)

Medication	Trial name	Setting	Primary outcome	PFS control	PFS gain	PFS HR	OS control	OS gain	OS HR	RR	QOL	Toxicity	ESMO-MCBS score	ESMO-MCBS form	Reference(s)
FC± R	CLL8	Upfront, chemofit	PFS	32.9 months	23.9 months	0.59 (0.50–0.69)	86 months (66.9% 5 years)	>10% gain at 5 years	0.68 (0.54– 0.89)		No difference	Increased	4	2a	24–26
FC-R versus R-bendamustine	CLL10	Upfront, focus elderly subgroup >65 years	Non- inferiority in PFS	55.2 months	−13.5 months	Non-inferiority not met neither overall, nor in the >65 years post hoc subgroup			Not significant			Less toxicity in experimental arm	Not significant, not eligible for scoring	2c	27
Ibrutinib versus chlorambucil	RESONATE-2	Upfront elderly	PFS	18.9 months	8 months	0.16 (0.09– 0.28)	85% at 24 months	13%	0.16 (0.05–0.56) Immature		Improved (abstract only)		3	2b	28 29
Obinutuzumab± chlorambucil	CLL11	Upfront elderly not eligible for fludarabine	PFS	11.1 months	15.6 months	0.18 (0.13– 0.24)	NR	NA	0.41 (0.23–0.74) Immature			Increased but not meeting criteria for downgrading	3	2b	30
Ibrutinib versus ofatumumab	RESONATE	Relapsed/refractory (cross-over allowed)	PFS	8.1 months	4+ months	0.11 (0.08– 0.15)	81% at 12 months	9% at 12 months	0.43 (0.24–0.79) Immature		Pending	>10% SAE increase (−1 point)	3	2b	31 32
					(>10% gain at 12 months with plateau)
R-Venetoclax versus R-bendamustine	MURANO	Relapsed/refractory	PFS	17 months	6+ months	0.17 (0.11– 0.25)	87% at 24 months	5.30%	0.48 (0.25–0.90) Immature				4	2b	33
					(>10% gain at 12 months with plateau)
Ibrutinib	RESONATE-17	Relapsed/refractory with del17p	ORR	63% at 24 months			75% at 24 months			64%		No new safety flags	3	3	34
Venetoclax	M13-982	Relapsed/refractory with del17p	ORR	72% at 12 months			87% at 12 months			79%		No new safety flags	3	3	35

del17p, 17 p deletion;ESMO-MCBS v1.1, European Society for Medical Oncology—Magnitude of Clinical Benefit Scale, version 1.1; FC, fludarabine, cyclophosphamide; NA, not applicable; NR, not reached; ORR, overall response rate; OS, overall survival;PFS, progression-free survival; QOL, quality of life; R, rituximab; RR, response rate; SAE, serious adverse event.

Scorability: CLL is generally a relatively indolent disease with a very long survival—often decades long—and many patients do not need intervention for many years and when treatment is initiated it commonly generates very long periods of remission. For these reasons, PFS is generally the most relevant and measurable primary endpoint. Since CLL is generally not considered to be a curable disease, all scoring was performed using scales for non-curative disease. One study27 could not be scored because the primary objective of non-inferiority with regard to PFS was not met. Moreover, the published results limited to a subcohort of patients older than 65 years, which are relevant for clinical practice (particularly in view of presented toxicity data) did not show non-inferiority and they were derived from a post hoc exploratory analysis.

Reasonableness: Overall scoring was considered reasonable with the highest grades being achieved by studies demonstrating either mature OS data24–26 or PFS gains with long-term plateauing of PFS,33 or compelling PFS gains.28 29 Grading of the phase III study of ibrutinib versus ofatumumab (RESONATE trial)31 32 was considered to be low; it was credited for PFS advantage including gain in the tail of the curve but was penalised for toxicity associated with the more prolonged drug exposure in continuous treatment (ESMO-MCBS v1.1 score 3). However, the 9% improvement in OS at 12 months was not credited as these results are deemed immature by the ESMO-MCBS criteria. The benefit of novel agents in populations with high unmet need, like relapsed and refractory patients with CLL carrying deletion in chromosome 17 p, was graded reasonably using form 3 for single-arm studies in a non-curative setting.34 35

Shortcomings: One shortcoming was identified:

1. The EHA scientific working group members felt that compelling immature survival benefit ought to be credited even when the median survival of the control arm has not been reached.

Chronic myeloid leukaemia

Studies evaluated: Four landmark trials addressing the use of tyrosine kinase inhibitors imatinib, nilotinib, dasatinib and bosutinib upfront for chronic phase CML were graded.36–43 Only one of these had mature OS data (table 4).38

Table 4.

Feasibility testing of the ESMO-MCBS v1.1 for chronic myeloid leukaemia (n=4)

Medication	Trial name	Setting	Primary outcome	EFS/PFS control	EFS/PFS gain	PFS/ EFS HR	OS control	OS gain	OS HR	Major CytRR/ MMR	Complete CytRR	MMR	MR4	MR4.5	QOL	Toxicity	ESMO-MCBS score	ESMO-MCBS form	Reference(s)
Imatinib versus interferon/cytarabine	IRIS	Newly diagnosed chronic phase (cross-over allowed)	Initial: PFS/EFS long term: OS												Improved	Less toxicity			36–38
			18 months PFS	73.5%	18.6%					87% vs 35%, gain 52%	76% vs 15%, gain 62%						C/2	1/2 c
			10 years EFS	56.6%	23%		78.8%	4.5%	0.74 (0.56– 0.99)								B/2	1/2 c
Nilotinib 600 or 800 mg versus imatinib	ENESTnd	Newly diagnosed chronic phase	Initial primary: MMR at 12 months, secondary: complete cytRR												More cardiovasc. events for nilotinib 800 mg			39 40
			12 months 600 mg								80% vs 65%, gain 15%	44% vs 22%, gain 22%				2	2c
			12 months 800 mg								78% vs 65%, gain 13%	43% vs 22%, gain 21%				2	2c
			5 years 600 mg	92.6%	2.4%	NS	91.7%	2.0%	NS			77% vs 60%, gain 17%	66% vs 42%, gain 24%	54% vs 31%, gain 22%		2	2c
			5 years 800 mg		4.3%	0.37 (0.15–0.88)		4.5%	0.44 (0.21–0.93)				63% vs 42%, gain 21%	52% vs 31%, gain 21%		2	2c
Dasatinib versus imatinib	DASISION	Newly diagnosed chronic phase	Complete cytRR														41 42
			12 months								77% vs 66%, gain 11%	46% vs 28%, gain 18%			1	2c
			5 years				90%	1%	NS			76% vs 64%, gain 12%	42% vs 33%, gain 9%
Bosutinib versus imatinib	BFORE	Newly diagnosed chronic phase	MMR at 12 months								77% vs 66%, gain 11%	47% vs 37%, gain 10%			1	2c	43

cardiovasc., cardiovascular; CytRR, cytogenetic response rate;EFS, event-free survival;ESMO-MCBS v1.1, European Society for Medical Oncology—Magnitude of Clinical Benefit Scale, version 1.1; MMR, major molecular response; MR, molecular response; NS, not significant; OS, overall survival; PFS, progression-free survival; QOL, quality of life.

Scorability: CML is generally considered an incurable disease, but in a small proportion of cases with deep molecular responses the disease may be eradicated. Thus, when mature survival data were available, CML was scored for both curative and non-curative intent.36–38 Contemporary studies in CML treatments are conventionally evaluated using molecular response evaluations.44 45 This differs from the concepts of ‘pathological complete response’ or ‘response rate’ which are terms used in the ESMO-MCBS v1.1. Scoring of these studies was only possible by interpreting deep molecular responses (MMR 4–5) as pathological complete responses (form 1) or major responses (form 2 c).39–43 In one study,36–38 PFS/event-free survival (EFS) gains could not be credited because the PFS of the control arm was very long and had not reached median PFS after 11 years of follow-up.

Reasonableness: In the IRIS study of imatinib versus former standard interferon plus cytarabine, initial scoring at 18 months was credited on the basis of complete cytogenic response for curative intent with a grade of C and improvement in molecular response rate with grade 2.36–38 At 10-year follow-up, the imatinib scores B for curative intent based on survival improvement. While the grades for curative intent were considered reasonable, the EHA working group considered the ESMO-MCBS grade of 2 for non-curable intent to be too low for the benefits observed.

The remaining studies of nilotinib, dasatinib and bosutinib show minor improvements in complete molecular response rates when compared with imatinib (grade 2) in a non-curative setting.39–43 None of these agents had mature data beyond 5 years and consequently they were not graded for curative intent.

Shortcomings: These relatively low scores for imatinib in the non-curative grading appear to indicate two shortcomings in the ESMO-MCBS v1.1:

1. When PFS (or EFS) is very long, there is no mechanism to credit strong interim gains when the median PFS of the control arm has not yet been reached.

2. The surrogacy of complete cytogenic response and level 4–5 MMR, defined as 4 to 5-log reduction in BCR–ABL1 transcript levels from a standardised baseline, are much stronger surrogates for survival than pathological complete response and response rate in solid tumours.44 45 Consequently, form 2 c needs to be amended to incorporate evaluation of deep molecular responses.

Indolent non-Hodgkin’s, relapsed/refractory setting of non-diffuse large B-cell lymphoma (non-DLBCL) and Hodgkin’s lymphoma

Studies evaluated: Twelve studies of recently approved drugs for indolent non-Hodgkin’s, relapsed/refractory setting of non-DLBCL and Hodgkin’s lymphoma were evaluated (table 5).46–62

Table 5.

Feasibility testing of the ESMO-MCBS v1.1 for indolent non-Hodgkin and relapsed/refractory setting of non-DLBCL and Hodgkin’s lymphoma (n=12)

Medication	Trial name	Setting	Primary outcome	PFS control	PFS gain	PFS HR	OS control	OS gain	OS HR	RR (DOR)	QOL	Toxicity	ESMO-MCBS score	ESMO-MCBS form	Reference(s)
Obinutuzumab-Chemo versus R-Chemo	GALLIUM	Follicular lymphoma, first line	PFS	73% (3 years)	7%	0.66 (0.51–0.85)							Not scoreable	2b	46
VR-CAP versus R-CHOP	LYM-3002	Mantle cell lymphoma first line, not eligible for transplant	PFS	14.4 months	10.3 months	0.63 (0.5–0.79)	55.7 months	35 months (7 year survival gain >5% with plateau)	0.66 (0.51– 0.85)			Increased in experimental arm	A/4	1/2a	47 48
R-Bendamustine versus R-CHOP/R-CVP	BRIGHT study	Indolent and mantle cell lymphoma, first line	Non-inferiority in CRR (margin: 0.88)							Non-inferiority met 1.26 (0.93–1.73) 31% versus 25%, 6% gain	Improved		Not scoreable	2c	49 50
R-Bendamustine versus R-CHOP	STiL Trial NHL 1-2003	Indolent and mantle cell lymphoma, first line	Non-inferiority in PFS (margin: 1.32)	31.2 months	38.3 months	0.58 (0.44–0.74) Non-inferiority met						Less adverse events in experimental arm	4	2c	51
Bendamustine± Obinutuzumab	GADOLIN	Rituximab-refractory indolent non-Hodgkin’s lymphoma	PFS	14.9 months	NA	0.55 (0.40–0.74)	NR	>10% at 5 years	0.67 (0.47– 0.96)		Delayed deterioration in QOL		5	2a	52–54
																Ibrutinib versus Temsirolimus		Relapsed/refractory mantle cell lymphoma	PFS	6.2 months	8.4 months	0.43 (0.32–0.58)					Improved (+1 point)		4	2b	55
Lenalidomide versus Investigator’s choice	MCL-002 SPRINT	Relapsed/refractory mantle cell lymphoma	PFS	5.2 months	3.5 months	0.61 (0.44–0.84)					Improved (+1 point)		4	2b	56
Ibrutinib	PCYC-1104-CA	Relapsed/refractory mantle cell lymphoma	ORR	13.9 months						68%			3	3	57
Ibrutinib		Relapsed/refractory marginal zone lymphoma	ORR	14.2 months						48%		Relevant toxicity but not meeting criteria for downgrading	3	3	58
Idealisib	DELTA (101-09)	Relapsed/refractory indolent lymphoma	ORR	11 months						57% (12.5 months)			3	3	59
Pembrolizumab	KEYNOTE- 087	Relapsed/refractory Hodgkin lymphoma	ORR							69%	Improved (+1 point)		4	3	60
Nivolumab	Check Mate 205	Relapsed/refractory Hodgkin lymphoma	ORR	14.7 months						69%	Improved (+1 point)		4	3	61 62

chemo, chemotherapy; CRR, complete response rate;DLBCL, non-diffuse large B-cell lymphoma; DOR, duration of response;ESMO-MCBS v1.1, European Society for Medical Oncology—Magnitude of Clinical Benefit Scale, version 1.1; NA, not applicable; NR, not reached; ORR, overall response rate; OS, overall survival;PFS, progression-free survival; QOL, quality of life; R, rituximab; R-CHOP, rituximab, cyclophosphamide, doxorubicin, vincristine and prednisone; R-CVP, rituximab, cyclophosphamide, vincristine and prednisone; RR, response rate; VR-CAP, bortezomib, rituximab, cyclophosphamide, doxorubicin and prednisone.

Scorability: In one of the studies,46 PFS/EFS gains could not be graded because the PFS of the control arm was very long, the median PFS was not reached and only interim gains were reported. The BRIGHT study could not be scored because form 2 c makes no provision for scoring of non-inferiority studies based on response rates.49 50 The remaining 10 studies were published with endpoints and data applicable to the ESMO-MCBS v1.1 and were all evaluable.

Reasonableness: The grading was applicable and was judged by the EHA working group to be reasonable in the evaluated trials, endorsing relatively high benefit grades, that is, ESMO-MCBS v1.1. scores of 4–5 for 7 of the 10 evaluable studies.

Shortcomings: Two shortcomings were observed:

1. The ESMO-MCBS v1.1 has no mechanism for scoring non-inferiority studies based on response rate.

2. When PFS (or EFS) is very long, there is no mechanism to credit strong interim gains when the median PFS of the control arm has not yet been reached.

Diffuse large B-cell lymphoma

Studies evaluated: Eleven studies were evaluated63–75; two in the first-line setting with curative intent,63–66 two intensified therapies for first-line and salvage setting, respectively, with both curative and non-curative intent,67 68 two single-arm studies of CAR T-cell salvage therapy70 71 and five in a non-curative setting for relapsed and refractory disease (table 6).69 72–75

Table 6.

Feasibility testing of the ESMO-MCBS v1.1 for DLBCL (n=11)

Medication	Trial name	Setting	Primary outcome	PFS/EFS/DFS control	PFS/EFS/DFS gain	PFS/EFS/DFS HR	OS control	OS gain	OS HR	RR (DOR)	QOL	Toxicity	ESMO-MCBS score	ESMO-MCBS form	Reference(s)
CHOP± R	MInT study	First-line DLBCL, stage II–IV or I with bulky disease, IPI 0–1	EFS	55.8% (6 years)	18.5%	p<0.0001	80% (6 years)	10.1%	p= 0.0004				A	1	63 64
																CHOP± R	LNH-98.5	First-line DLBCL, stage II–IV, age 60–80	PFS	20% at 10 years	16.5%	p<0.0001	27.6% (10 years)	15.9%	p< 0.0001				A	1	65 66
EFS	38% at 2 years	19%	p<0.001
R-CHOP ±lenalidomide maintenance	REMARC	First-line DLBCL, stage II–IV, age 60–80	PFS	58.9 months	4+ months	0.71 (0.54–0.93)			NS				A/3	1/2b	67
																R-GDP+ASCT  versus R-DHAP +ASCT	NCIC-CTG LY12	Relapsed/refractory aggressive lymphoma	Non- inferiority (ORR) (margin: −10%)			No difference			No difference	ORR difference: −1.2 (-9, 6.7) 44% vs 45% (non- inferiority met)	Improved		B/not scoreable	1/2 c	68
Pixantrone versus investigators’ choice		Relapsed/refractory aggressive lymphoma	CRR	2.6 months	2.7 months	0.60 (0.42– 0.86)				20% vs 6%, gain 14%			3	2b	69
					>10% gain at 12 months, no plateau
CAR T-cell Axicabtagene ciloleucel	ZUMA-1	Relapsed/refractory aggressive non-Hodgkin’s lymphoma	ORR							82%		Toxicity but not meeting criteria for downgrading	3	3	70
																CAR T-cell Tisagen-lecleucel	JULIET	Relapsed/refractory DLBCL	ORR							52% (not reached, >10 months)		Toxicity not meeting criteria for downgrading	3	3	71
Lenalidomide versus investigators’ choice	DLC-001	Relapsed/refractory DLBCL	ORR	2 months	1.4 months	0.64 (0.41– 0.99)				28% vs 12%, gain 16%		More PFS-improvement in ABC subtype	2	2b	72
Panobinostat with or without R		Relapsed/refractory DLBCL	ORR							28% (15 months)			3	3	73
																Brentuximab vedotin		Relapsed/refractory DLBCL	ORR	4 months						44%			2	3	74
Ibrutinib		Relapsed/refractory DLBCL, subgroup ABC subtype	ORR	2 months						37% (4.8 months)			1	3	75

ASCT, autologous stem cell transplantation; CART- cell, chimeric antigen receptor T-cell therapy; CHOP, cyclophosphamide, doxorubicin, vincristine and prednisone; CRR, complete response rate; DFS, disease-free survival; DLBCL, diffuse large B-cell lymphoma; DOR, duration of response; EFS, event-free survival;ESMO-MCBS v1.1, European Society for Medical Oncology—Magnitude of Clinical Benefit Scale, version 1.1; IPI, International Prognostic Index; NS, not significant; ORR, overall response rate; OS, overall survival;PFS, progression-free survival; QOL, quality of life;R-CHOP, rituximab, cyclophosphamide, doxorubicin, vincristine and prednisone; R-DHAP, rituximab, dexamethasone, cytarabine and cisplatin; R-GDP, rituximab, gemcitabine, dexamethasone and cisplatin; RR, response rate.

Scorability: All studies incorporated required data for evaluation using the ESMO-MCBS v1.1. Single-arm studies of CAR T-cell therapy for refractory or resistant disease70 71 could not be evaluated for curative intent. The NCIC-CTG LY12 trial could not be graded in the non-curative setting because non inferiority was evaluated on the basis of overall response rate.68

Reasonableness: The grading was applicable and was judged by the EHA working group to be reasonable in the evaluated trials, endorsing high benefit grades for first-line therapies with curative intent.63–67 Lower benefit scores for trials in the relapsed and refractory therapies were considered reasonable.

Shortcomings: One shortcoming was identified:

1. The ESMO-MCBS v1.1 does not have a form to grade single-arm treatments with curative intent and this shortcoming does not allow for the representation of the full potential benefit of CAR T-cell salvage therapy.70 71

Multiple myeloma

Studies evaluated: Table 7 describes results from eight studies in the first-line setting.76–84 Of these, three were conducted for autologous stem cell transplantation (ASCT) eligible76–78 patients and five are for ASCT ineligible patients.79–84 Table 8 describes the results of a further 15 studies with relapsed or refractory myeloma.85–104

Table 7.

Feasibility testing of the ESMO-MCBS v1.1 for first-line multiple myeloma (n=8)

Medication	Trial name	Setting	Primary outcome	PFS/DFS control	PFS/DFS gain	PFS/DFS HR	OS control	OS gain	OS HR	RR	QOL	Toxicity	ESMO-MCBS score	ESMO-MCBS form	Reference(s)
VTD versus TD or VBMCP/VBAD/B	GEM2005-less65 PETHEMA/GEM	ASCT eligible	CR post ASCT (PFS)									More neuropathy but not meeting criteria for downgrading		1/2b	76
			TD	28.2 months	28.0 months	p=0.01	65% at 4 years	9%	NS	CRR 46% vs 24%, gain 22%			C/not scoreable
			VBMCP/VBAD/B	35.3 months	20.9 months	p=0.01	70% at 4 years	4%	NS	CRR 46% vs 38%, gain 8%			NEB/not scoreable
VTD versus TD	GIMEMA 2005	ASCT eligible	CR post induction (PFS)	56% at 3 years	12%	0.63 (0.45– 0.88)	84% at 3 years	2%	NS	(near) CRR 31% vs 11%, gain 20%		More neuropathy but not meeting criteria for downgrading	C/not scoreable	1/2b	77
																VCD versus PAD	MM5	ASCT eligible	Non-inferiority of ≥VGPR rates (margin: −10%)							VGPR difference: 2.8% (−6.8% to 12.3%) non-inferiority met		SAEs higher in the control arm	Not scoreable	1/2 c	78
VMP versus MP	VISTA	ASCT ineligible	TTP	16.6 months	7.4 months	0.48 (p<0.001)	43.1 months	13 months	0.70 (0.57– 0.85)				4	2a	79 80
VMPT versus VMP	GIMEMA VMPT	ASCT ineligible	PFS	27 months 41% at 3 years	>13 months 15%	0.67 (0.50– 0.90)	87% at 3 years	2%	NS		–	Vascular and cardiac events increased in experimental arm (−1 point)	2	2b	81
Lenalidomide-d continuous versus x18 or MPT x12	FIRST	ASCT ineligible	PFS												82
			Len-d x18	20.7 months	4.8 months	0.70 (0.60–0.82)	56% at 4 years	3% gain at 4 years	NS				3	2b
			MPT	21.2 months	4.3 months	0.72 (0.61–0.85)	47 months 51% at 4 years	7 months 8% gain at 4 years	0.78 (0.64– 0.96)				4	2a
VMP ± daratumumab	ALCYONE	ASCT ineligible	PFS	18 months 50% at 18 months	9+ months 21% at 18 months	0.50 (0.38–0.65)						More infections but not meeting criteria for penalty	3	2b	83
Lenalidomide-d ± bortezomib	SWOG S07777	ASCT ineligible	PFS	30 months	13 months	0.71 (0.56–0.91)	64 months	11 months	0.71 (0.52– 0.96)			Slightly increased	4	2a	84

ASCT, autologous stem cell transplantation; CR, complete remission; CRR, complete remission rate; d, dexamethasone; DFS, disease-free survival;ESMO-MCBS v1.1, European Society for Medical Oncology—Magnitude of Clinical Benefit Scale, version 1.1; Len-d, lenalidomide-d; MP, melphalan and prednisone; MPT, melphalan, prednisone and thalidomide; NEB, no evaluable benefit; NS, not significant; OS, overall survival; PAD, bortezomib, doxorubicin, dexamethasone;PFS, progression-free survival; QOL, quality of life; RR, response rate; SAE, serious adverse event; TD, thalidomide and dexamethasone; TTP, time to progression; VBMCP/VBAD/B, vincristine, BCNU, melphalan, cyclophosphamide, prednisone/vincristine, BCNU, doxorubicin, dexamethasone/bortezomib; VCD, bortezomib, cyclophosphamide, dexamethasone; VGPR, very good partial response rate; VMP, bortezomib, melphalan and prednisone; VMPT, bortezomib, melphalan, prednisone and thalidomide; VTD, bortezomib, thalidomide and dexamethasone.

Table 8.

Feasibility testing of the ESMO-MCBS v1.1 for relapsed/refractory multiple myeloma (n=15)

Medication	Trial name	Setting	Primary outcome	PFS control	PFS gain	PFS HR	OS control	OS gain	OS HR	RR (DOR)	QOL	Toxicity	ESMO-MCBS score	ESMO-MCBS form	Reference(s)
Dexamethasone ± lenalidomide	CC-5013- MM-010	Relapsed/refractory	TTP	4.7 months	6.6 months	0.35 (0.27– 0.46)	20.6 months	NA	0.66 (0.45–0.96)				3	2b	85
Lenalidomide-d ± carfilzomib	ASPIRE	Relapsed/refractory	PFS	17.6 months	8.7 months	0.69 (0.57–0.83)	40.4 months	7.9 months	0.79 (0.67– 0.95)		Improved (+1 point)	Slightly increased	4	2a	86 87
Lenalidomide-d ± ixazomib	TOURMALINE-MM1	Relapsed/refractory	PFS (interim)	14.7 months	5.9 months	0.74 (0.59–0.94)			Immature		Not improved		3	2b	88
Lenalidomide-d ± daratumumab	POLLUX	Relapsed/refractory	PFS (interim)	18.4 months	16+months	0.37 (0.27–0.52)			Immature			Higher haematological toxicities	3	2b	89
																Lenalidomide-d ± elotuzumab	ELOQUENT-2	Relapsed/refractory	Coprimary PFS and ORR (interim)	14.9 months 57% at 12 months	4.5 months 11% at 12 months	0.70 (0.57–0.85)	39.6 months	8.7 months	0.78 (0.63– 0.96)		No difference	Slightly higher SAEs	3	2a	90 91
Dexamethasone ± bortezomib	APEX	Relapsed/refractory	TTP	3.5 months	2.7 months	0.55 (p=0.001)	23.7 months	6.1 months	0.77 (p=0.027)				3	2b	92 93
Carfilzomib-d versus bortezomib-d	ENDEAVOR	Relapsed/refractory	PFS	9.4 months	9.3 months	0.53 (0.44–0.65)	40 months	7.6 months	0.79 (0.65– 0.96)		Improved (abstract only)	Slightly higher SAEs	3	2a	94 95
Bortezomib-d ± daratumumab	CASTOR	Relapsed/refractory	PFS	7.1 months 26.9% at 12 months	9.6 months 33.8% at 12 months	0.31 (0.24–0.39)			Immature			Higher haematological toxicity	3	2b	96 97
Bortezomib-d ±panobinostat	PANORAMA1	Relapsed/refractory	PFS	8.1 months	3.9 months	0.63 (0.52– 0.76)	30.4 months	3.25 months	Immature			3% increase in PN grade ≥3 (−1 point)	2	2b	98
Dexamethasone ± pomalidomide	MM-003	Relapsed/refractory	PFS	1.9 months	2.1 months	0.48 (0.39–0.60)	8.1 months	4.6 months	0.74 (0.56– 0.97)				4	2a	99
Pomalidomide-d ± cyclophosphamide	MMC-16705	Relapsed/refractory ≥2 prior lines of treatment	ORR	4.4 months	5.1 months	NS				64.7% vs 38.9%, gain 25.8%			2	2c	100
Daratumumab	SIRIUS	Relapsed/refractory	ORR	3.7 months						29% (7.4 months)			2	3	101
Daratumumab	GEN501	Relapsed/refractory (16 mg/kg)	Safety	5.6 months						36% (NR)			2	3	102
Daratumumab +pomalidomide + d	MMY1001	Relapsed/refractory ≥2 prior lines of treatment	Safety	8.8 months			17.5 months			60% (>13 months)			3	3	103
Pomalidomide +bortezomib + d	MC1082	Relapsed/refractory	ORR	13.7 months						86%			3	3	104

d, dexamethasone; DOR, duration of response;ESMO-MCBS v1.1, European Society for Medical Oncology—Magnitude of Clinical Benefit Scale, version 1.1; NA, not applicable;NR, not reached; NS, not significant; ORR, overall response rate; OS, overall survival;PFS, progression-free survival; PN, polyneuropathy; QOL, quality of life; RR, response rate; SAEs, serious adverse events; TTP, time to progress.

Scorability: Most studies incorporated required data for evaluation using the ESMO-MCBS v1.1. The PETHEMA/GEM study comparing VTD (bortezomib, thalidomide and dexamethasone) to TD (thalidomide and dexamethasone) or VBMCP/VBAD/B (vincristine, BCNU, melphalan, cyclophosphamide, prednisone/vincristine, BCNU, doxorubicin, dexamethasone/bortezomib) as induction therapies did not report HRs for the PFS, resulting in precluded scoring with non-curative intent using form 2b.76 The GIMEMA 2005 study could not be scored for non-curative intent because the median PFS of the control arm had not yet been reached.77 The MM5 non- inferiority study78 could not be scored for non-curative intent because non-inferiority was based on response rate.

Reasonableness: First-line treatments for patients who are ASCT eligible are graded both for curative and non-curative intent. The relatively low grades of C for curative intent achieved in two of the ASCT eligible studies76 77 reflect the prevailing consensus that MM is rarely cured. In most studies evaluated, the scale was feasible and the results were consistent with clinical practice.

Shortcomings: Three previously described shortcomings influenced scoring for a small number of these studies.

1. The ESMO-MCBS v1.1 has no mechanism for scoring non-inferiority studies in a non-curative setting based on response rate.

2. When PFS (or EFS) is very long, the ESMO-MCBS v1.1 has no mechanism to credit strong interim gains when the median PFS of the control arm has not yet been reached.

3. The EHA working group members felt that the capitation of PFS at a maximal preliminary grade of 3, with provision for an upgrade based on tail of the curve only if there is a plateau in the study medication PFS with gain of >10% at 12 months, may have undervalued some MM treatments.96 97 The plateau requirement for this adjustment precludes credit for substantial prolonged gains in PFS in this disease entity.

Myelodysplastic syndrome

Studies evaluated: Ten studies were evaluated in this setting.105–114 Of these, two studies were evaluated based on OS or PFS and the remaining eight studies were evaluated based on response rate (table 9).

Table 9.

Feasibility testing of the ESMO-MCBS v1.1 for myelodysplastic syndrome (n=10)

Medication	Trial name	Setting	Primary outcome	PFS control	PFS gain	PFS HR	OS control	OS gain	OS HR	RR (DOR)	QOL	Toxicity	ESMO-MCBS score	ESMO-MCBS form	Reference
Azacitidine versus SOC	AZA-MDS-001	High-risk MDS	OS				15 months	9.5 months	0.58 (0.43–0.77)				4	2a	105
Decitabine versus SOC		MDS FAB (IPSS ≥0.5)	Coprimary ORR and PFS	7.8 months	4.3 months	0.58 (0.37–0.91)					Improved (+1 point)		4	2b	106
Lenalidomide (10 mg/5 mg) versus SOC	LEN-MDS-004	Transfusion-dependent patients with low-risk/intermediate-risk MDS del5q (IPSS ≤1)	RR (RBC-TI)												107
			10 mg							56% vs 6%, gain 50%			2	2c
			5 mg							43% vs 6%, gain 37%			2	2c
Lenalidomide versus SOC	LEN-MDS-005	MDS-WHO (IPSS ≤1)	RR (RBC-TI at ≥8 weeks)							26.9% vs 2.5%, gain 24.4%	Improved		2	2c	108
Antithymocyte globulin versus SOC	SAKK 33/99	MDS <10% bone marrow blasts	RR at 6 months							29% vs 9%, gain 20%			2	2c	109
rHuEPO versus SOC	ICSG	MDS <10% bone marrow blasts	RR (TI)							37% vs 11%, gain 26%			2	2c	110
rHuEPO versus ± GCSF		MDS-FAB (IPSS ≤0.5)	RR (TI)							73% vs 40%, gain 33%			2	2c	111
EPO versus SOC	E1996	MDS <10% bone marrow blasts	RR (IWG 2000 modified)							36% vs 10%, gain 26%			2	2c	112
rHuEPO +GCSF versus SOC	GFM	MDS <10% bone marrow blasts	RR (IWG 2006 stringently modified)							42% vs 0%, gain 42%			2	2c	113
Darbepoetin versus SOC		MDS-WHO IPSS ≤1	RBC transfusion incidence							59% vs 36%, gain 23%			2	2c	114

del5q, 5 q deletion; DOR, duration of response;ESMO-MCBS v1.1, European Society for Medical Oncology—Magnitude of Clinical Benefit Scale, version 1.1; FAB, French–American–British classification for MDS; GCS-F, granulocyte-stimulating factor; IPSS, International Prognostic Scoring System; IWG, International Working Group; MDS, myelodysplastic syndrome;ORR, overall response rate; OS, overall survival;PFS, progression-free survival; QOL, quality of life; RBC-TI, red blood cell transfusion independency; rHuEPO, recombinant human erythropoietin; RR, response rate; SOC, standard of care; TI, transfusion independency.

Scorability: All studies incorporated required data for evaluation using the ESMO-MCBS v1.1. Clinical benefit measure was, however, partly confounded by the heterogeneity of the available definitions of haematological response and their clinical meaningfulness.

Reasonableness: In the two studies evaluating hypomethylating agents in intermediate-risk/high-risk patients,105 106 the ESMO-MCBS v1.1 graded them with substantial benefit based on either PFS gain or OS gain with improved QOL. In lower risk patients, the remaining eight studies included randomised trials investigating erythropoietin-stimulating agents, lenalidomide in MDS with del(5q) or non-del(5q) and immunosuppressive therapy with antithymocyte globulin plus cyclosporine, compared with best supportive care.107–114 All studies were evaluated based on response rates, but they used a range of different and inconstant criteria, some using International Working Group, or modifications thereof, and other study-specific criteria such as transfusion requirements. All these studies resulted in a final ESMO-MCSB v1.1 score of 2. In one of these studies108 QOL was evaluated and demonstrated to have improved but this was not reflected in grading since there is no QOL bonus for studies in which response rate is the primary outcome.

Shortcomings: The EHA working group identified one shortcoming derived from these evaluations:

1. In studies evaluating response rate as a primary endpoint, there is no provision of QOL bonus if improved QOL is demonstrated as a secondary outcome.

Discussion

The EHA with currently more than 5000 members is the largest European-based haematology association. In addition to its educational mission, it has a public policy and advocacy role that engages stakeholders, including patient representatives, to improve patient care and to raise awareness for haematology as a distinct medical discipline with specific needs.115 Reflecting these goals, EHA has observed the development of the ESMO-MCBS and its broad utility in solid tumour oncology with great interest, and in the absence of a value tool validated for malignant haematology, we sought to investigate the applicability of the ESMO-MCBS v1.1 as a first step to the development of a version validated for HMs.

There are several major differences in the behaviour of HMs as compared with solid tumour cancers. These differences arise largely from the more variable natural history of HMs which can range from fulminant (acute leukaemia and high-grade lymphomas) to almost benign (low-grade MDS). Furthermore, many of these malignant haematological diseases, even when they are not cured, they are characterised by very long PFS and OS that are rarely seen among incurable solid tumour malignancies. Finally, the endpoints used in the studies of treatments for HMs are sometimes different to those used in solid tumours and in some instances, such as CML, they are even disease-specific. Consequently, at the outset of this project we did not know if ESMO-MCBS v1.1 could be applied to studies in HMs, and if the grading of studies would generate grades considered reasonable by experts in the relevant diseases.

This evaluation of the behaviour of the ESMO-MCBS v1.1 in the grading of 80 studies across the full spectrum of HMs has demonstrated that the ESMO-MCBS v1.1 is widely applicable for the overwhelming majority of analysed studies (90% scoreable studies) and that the generated scores were generally adjudicated by clinical experts to reasonably accord with their evaluation of the magnitude of clinical benefit. In 5 of the 80 studies (6%), the ESMO-MCBS could not be applied at all21 27 46 49 50 78 and in 3 more studies (4%), it could not be applied to one of the evaluable parameters.68 76 77 In the evaluation of imatinib in CML,36–38 it generated scores that were considered to under-represent the true value of the intervention in the opinion of experts in the evaluated diseases.

Based on the analysis of the scorability of studies and the reasonableness of the generated results, this field testing identified six shortcomings in the current version of the ESMO-MCBS that will require redress to improve the applicability and reasonableness of ESMO-MCBS scoring for malignant haematological conditions.

1. Regarding single-arm studies with curative intent, such as CAR T-cell salvage therapies, the ESMO-MCBS v1.1 does not have a form to grade single-arm treatments with curative intent.

2. Regarding relatively indolent conditions with a very long PFS (or EFS) or OS such as CLL, CML, indolent lymphoma and MM, there is no mechanism to credit strong interim gains when the median of the control arm has not yet been reached.

3. The capitation of PFS at a maximal preliminary grade of 3, with provision for an upgrade based on tail of the curve only when there is a plateau in the arm with the study medication, may undervalue treatments with substantial late PFS gain but with no plateauing of the curves.

4. Regarding the standard molecular surrogate endpoints used for CML, the surrogacy of complete cytogenic response and level 4–5 MMR must be acknowledged and incorporated.

5. The scale does not make provision for the grading of non-inferiority studies based on response rate criteria.

6. In studies evaluating response rate as a primary endpoint, there is no provision of QOL bonus if improved QOL is demonstrated as a secondary outcome.

Finally, it must be acknowledged that the results of the scale may not be reasonable for some of the least malignant of the HMs such as low-risk MDS. Most of the studies for MDS were evaluated based on response rates, but there was heterogeneity of the available definitions of haematological response and their clinical meaningfulness. This underlines the need for a stand-alone form regarding studies with such heterogeneity in their response rates.

ESMO and the EHA are committed to the development of a version of the ESMO-MCBS that is validated for HMs. Based on the findings of this study, a revised version of the ESMO-MCBS will be developed to address the identified shortcomings in the current version of the scale regarding the assessment of HMs. This development process will incorporate all the usual stringencies for accountability of reasonableness that have characterised the development of the ESMO-MCBS. This, thus far, included field testing, statistical modelling, evaluation for reasonableness and openness to appeal and revision. Applying such a scale will support future decision-making and will provide insights that could be helpful in the design of future clinical trials.