Comparison of 2D (RANO) and volumetric methods for assessment of recurrent glioblastoma treated with bevacizumab—a report from the BELOB trial

Abstract

Background.

The current method for assessing progressive disease (PD) in glioblastoma is according to the Response Assessment in Neuro-Oncology (RANO) criteria. Bevacizumab-treated patients may show pseudo-response on postcontrast T1-weighted (T1w) MRI, and a more infiltrative non-enhancing growth pattern on T2w/fluid attenuated inversion recovery (FLAIR) images. We investigated whether the RANO criteria remain the method of choice for assessing bevacizumab-treated recurrent glioblastoma when compared with various volumetric methods.

Methods.

Patients with assessable MRI data from the BELOB trial (n = 148) were included. Patients were treated with bevacizumab, lomustine, or both. At first and second radiological follow-up (6 and 12 wk), PD was determined using the 2D RANO criteria and various volumetric methods based on enhancing tumor only and enhancing plus non-enhancing tumor. Differences in overall survival (OS) between PD and non-PD patients were assessed with the log-rank test and a Cox model. Hazard ratios (HRs) and their 95% CIs were determined.

Results.

For all patients together, all methods (except subtraction of non-enhancing from enhancing volume at first follow-up) showed significant differences in OS between PD and non-PD patients (P < .001). The largest risk increase for death in case of PD at both first and second follow-up was found with the RANO criteria: HR = 2.81 (95% CI, 1.92–4.10) and HR = 2.80 (95% CI, 1.75–4.49), respectively. In the bevacizumab-treated patients, all methods assessed showed significant differences in OS between PD and non-PD patients. There were no significant differences between methods.

Conclusions.

In the first 12 weeks, volumetric methods did not provide significant improvement over the RANO criteria as a posttreatment prognostic marker.

Importance of the study

Currently, the 2D RANO criteria are the method of choice for assessing progressive disease in glioblastoma. However, those treated with bevacizumab may show pseudo-response on postcontrast T1w images and non-enhancing tumor growth on T2w/FLAIR images upon progression that may not optimally be captured with the 2D RANO criteria. We compared the 2D RANO criteria with various volumetric methods based on enhancement, subtraction, and T2w/FLAIR abnormalities in this distinct patient group. The risk increase for death was determined based on the presence of PD upon follow-up. The largest HRs at first and second follow-up were found with the 2D RANO criteria: HR = 2.81 (95% CI, 1.92–4.10) and HR = 2.80 (95% CI, 1.75–4.49), respectively. We found no clear differences in prediction of OS between the 2D RANO and volumetric methods. The routine use of volumetric methods in clinical trials in recurrent glioblastoma is therefore not warranted.

Glioblastoma is the most common malignant primary brain tumor, comprising 15% of all primary brain and central nervous system tumors and almost half of all primary malignant brain tumors. The 5-year survival rate is only about 5% despite treatment of newly diagnosed patients with surgery, radiotherapy, and concomitant and adjuvant temozolomide.^1,2

In 2009, the FDA approved bevacizumab (Avastin, Genentech) for second-line treatment in patients with recurrent glioblastoma based on the observed response rates from phase II trials with bevacizumab and irinotecan.^3,4 Bevacizumab is a humanized monoclonal antibody directed against vascular endothelial growth factor (VEGF), which inhibits angiogenesis and normalizes abnormally permeable tumor blood vessels.^5,6 This may also lead to a decrease (or complete disappearance) of enhancing tumor on postcontrast T1-weighted (T1w) images without actual changes in tumor size. Due to this phenomenon of pseudo-responses, the conventional Macdonald assessment criteria,⁶ which rely primarily on the assessment of enhancing lesions, were no longer sufficient and have been replaced by the Response Assessment in Neuro-Oncology (RANO) criteria.⁷ These have become the standard in clinical neuro-oncology and include the assessment of T2w/fluid attenuated inversion recovery (FLAIR) (non-enhancing) abnormalities in addition to enhancing lesions. Progressive disease (PD) is defined as a ≥25% increase in the sum of the products of perpendicular diameters of enhancing tumor, a significant increase in non-enhancing tumor, the appearance of new lesions, or clinical deterioration. It has been shown that adding non-enhancing abnormalities based on T2w/FLAIR imaging to the response assessment in recurrent glioblastoma treated with bevacizumab may lead to earlier detection of PD.⁸

Despite the inclusion of T2w/FLAIR assessment into the RANO criteria, the main focus of response evaluation remains on enhancing tumor. The observed initial decrease of enhancing tumor and increase in T2w/FLAIR abnormalities at progression^9,10 in patients treated with bevacizumab suggests that more advanced methods of assessment, such as volumetry, might improve prediction of overall survival (OS). This is particularly relevant to the assessment of T2w/FLAIR abnormalities, since it is defined as a qualitative, and not a quantitative, change in volume. Volumetric assessments are likely to increase precision of measurements of enhancement in irregularly shaped tumors, such as glioblastoma, and T2w/FLAIR abnormalities are likely to be more reliably assessed quantitatively with volumetry.

The occurrence of pseudo-response (ie, the decrease of tumor enhancement due to vascular normalization rather than a true antitumor effect in glioblastoma patients treated with VEGF inhibitors) has now been well documented, and questions arise whether further improvement of the RANO criteria is needed. Several studies have investigated the evaluation of tumor response using volumetric measures from both enhancing and non-enhancing recurrent glioblastoma treated with bevacizumab.^9,11,12 Boxerman et al¹³ also directly compared 2D with volumetric methods in this specific patient group but did not assess the full RANO criteria. Ultimately, to be a reliable surrogate endpoint in phase I and II studies, response should reflect OS, the gold standard in oncology trials. Our aim was to determine whether in bevacizumab-treated recurrent glioblastoma volumetric methods are superior to the 2D RANO criteria in determining PD in association with OS.

For this purpose we used data from the BELOB trial, which is the first randomized and properly controlled phase II trial in recurrent glioblastoma, comparing single-agent bevacizumab or lomustine with bevacizumab plus lomustine.¹⁴

Methods

Patients

A total of 148 eligible patients with first recurrence of glioblastoma were included in the BELOB trial, a randomized controlled phase II trial in which patients received bevacizumab (Avastin) (n = 50), lomustine (n = 46), or both (n = 52). Patients were recruited between December 2009 and October 2011 from 5 university hospitals and 9 community hospitals in the Netherlands. Patients were at least 18 years of age and had received no prior treatment with anti-VEGF agents or nitrosoureas. All patients provided written informed consent according to national regulations. A more detailed description of the study and its findings can be found in Taal et al.¹⁴

The primary endpoint of the BELOB trial was 9-month OS. Additional outcome measures were median progression-free survival (PFS), PFS at 6 and 12 months, median OS, OS at 6 and 12 months, and proportion of patients with objective response. In the current radiological analysis, OS was used as the endpoint.

Scanning Procedure

Patients underwent standardized MRI scanning at baseline and follow-up with approximately 6-week intervals (ie, a follow-up scan was made after every treatment cycle). The standardized MRI protocol can be found in the Supplementary files. Imaging was performed by 1.5T and 3.0T scanners. During the study, MRI quality assessment was performed. Unfortunately, some patients still had incomplete imaging datasets, as specified in the Results section.

Progressive Disease Assessment

The presence of PD was determined with the 2D RANO method (method 1) and 4 volumetric methods: total contrast-enhancing (CE) volume measured on T1w postcontrast images only (method 2), total CE volume measured on subtraction (postcontrast minus precontrast) images only (method 3), total CE volume (as measured on T1w postcontrast images) complemented by non-enhancing volume measured on FLAIR images (method 4), and total CE volume as measured on subtraction images complemented by non-enhancing volume (method 5).

RANO assessment to establish PD was performed centrally (pre- and postcontrast T1w and T2w/FLAIR images) by 2 independent reviewers (M.B., M.S.). In case of disagreement, PD was decided by an adjudicator (B.J.).

Volumes of enhancing areas and non-enhancing (FLAIR) abnormalities were measured on 3D T1w postcontrast and 3D FLAIR images, respectively, by a single rater (R.G.) using a semi-automated technique in Brainlab iPlan 4.0 Cranial software. This technique involves the manual placement of “inclusion” and “exclusion” points (Figure 1), after which an algorithm is used to render a volume of interest (VOI). All VOIs were visually checked in 3 directions and adjusted if needed.

Fig. 1.

Example of segmentation in Brainlab iPlan 4.0 Cranial using inclusion (green) and exclusion (blue) lines/points in the axial plane (A) and the resulting segmentation (B).

In each scan, in case of more lesions, all lesions were measured separately and summed for the current analysis to obtain a single volumetric measure of both the enhancing tumor and one of the FLAIR abnormalities. Blood vessels, dura, and necrotic areas were excluded. Enhancing areas were included in the FLAIR VOIs, because these areas are also hyperintense on FLAIR. The commonly T2w-hyperintense cortical ribbon was excluded. Lesions clearly of vascular origin and periventricular apical capping (ie, not continuous with FLAIR lesions and remaining unchanged in time) were excluded, as well as the septum pellucidem. FLAIR abnormalities when present in both left and right hemispheres were measured separately when possible.

Subtraction images were created with FSL-FLIRT (FMRIB Software Library) with custom scripts in AFNI (Analysis of Functional NeuroImages; National Institute of Mental Health) created by Ellingson et al.¹⁵ VOIs of resulting enhancing tumor areas were drawn manually in MRIcron (Chris Rorden, www.mrico.com, v6.6.2013) by a single rater (R.G.). Necrotic areas, blood vessels, and dura were excluded.

PD was defined according to the 2D RANO criteria (as described earlier) for method 1. For the volumetric methods (methods 2–5), PD was defined as ≥40% increase in enhancing/subtraction volume, which was the most commonly used threshold in previous literature,^16–18 ≥25% increase of FLAIR volume,¹⁹ or the appearance of new lesions, whether enhancing or non-enhancing. The 40% threshold for volumetric assessment is based on an extrapolation from the ≥25% increase in the sum of perpendicular diameters of a 2D lesion to a 3D sphere-shape and the assumption that all increase in size is equal in every direction. Setting the threshold this high will ensure that only patients showing clear PD are categorized as such.

Statistical Analysis

Patients were classified as to PD or non-PD according to each of the 5 methods at both first and second follow-up scans. For each of the follow-up moments separately, OS was determined from the date of the scan to death from any cause. Patients still alive at last contact were censored. Kaplan–Meier survival curves were drawn from each follow-up time point for all patients from all treatment groups, as well as for patients treated either with bevacizumab (with/without lomustine) or lomustine separately. A log-rank test was used to determine the difference in OS between the PD and non-PD patients as established by each of the methods. To determine whether there were significant differences between methods in predicting OS, hazard ratios (HRs) and their corresponding 95% confidence intervals (CIs) were determined by means of a Cox regression analysis per method, using each method as a single covariate. The HR was determined for all patients together and for the different treatment groups per method. The overlap of the corresponding 95% CIs was determined and conclusions were drawn based on the extent of overlap. Statistical analysis was performed using SPSS Statistics for Windows v21.0 (IBM).

A post-hoc power analysis (G*Power²⁰) was performed for all patients together and for the 2 different treatment groups per method at both first and second follow-up to evaluate the validity of the results.

Results

Patients

One hundred forty eight patients were included in the BELOB trial. At first follow-up, 10 patients were excluded from the analysis due to a lack of follow-up and missing data required for 2D RANO assessment. Some further patients had missing 3D T1w pre- and/or postcontrast or 3D FLAIR images, resulting in the following numbers of patients available for analysis at first follow-up: n = 138 for method 1, n = 121 for method 2, n = 86 for method 3, n = 109 for method 4, and n = 89 for method 5. At second follow-up, at which time patients with prior PD or death had dropped out, the following numbers of patients with adequate imaging were available for analysis: n = 83 for method 1, n = 78 for method 2, n = 71 for method 3, n = 71 for method 4, and n = 67 for method 5.

Assessment of All Patients from All Treatment Groups

Significant differences in OS (Figures 2 and 3) were found between all patients with PD and non-PD as determined with methods 1, 2, 4, and 5 at first follow-up, and with all 5 methods at second follow-up (Table 1). At first follow-up, the highest risk increase for death was found for PD as determined with method 1 (the RANO criteria), with an HR of 2.81 (95% CI, 1.92–4.10). In comparing the methods, the extent of overlap in CIs (>50%) indicates that there are no significant differences in HR. At second follow-up the highest risk increase was again found for PD determined with the RANO criteria (method 1), with an HR of 2.80 (95% CI, 1.75–4.49). Again, no significant differences were observed between methods as determined by the extent of overlap in CIs. For a graphic depiction of the direct comparison between the HRs and 95% CIs, see the Supplementary files (Figure S1).

Fig. 2.

Kaplan–Meier curves of all progressive (PD) versus non-progressive (non-PD) patients for each of the methods at first follow-up. (A) 2D RANO, (B) contrast-enhancing volume, (C) subtraction volume, (D) contrast-enhancing + FLAIR volume, and (E) subtraction + FLAIR volume.

Fig. 3.

Kaplan–Meier curves of all progressive (PD) versus non-progressive (non-PD) patients for each of the methods at second follow-up. (A) 2D RANO, (B) contrast-enhancing volume, (C) subtraction volume, (D) contrast-enhancing + FLAIR volume, and (E) subtraction + FLAIR volume.

Table 1.

Hazard ratios of each of the methods for all patients at first and second follow-up (FU1 and FU2)

Method	HR (95% CI) FU1	P-value	HR (95% CI) FU2	P-value
1. 2D RANO	2.81 (1.92–4.10)	<.001	2.80 (1.75–4.49)	<.001
2. Contrast-enhancing volume	1.99 (1.29–3.05)	.002	2.17 (1.27–3.71)	.004
3. Subtraction volume	1.80 (.99–3.26)	.054	2.16 (1.27–3.68)	.005
4. Contrast-enhancing + FLAIR volume	2.45 (1.60–3.74)	<.001	2.57 (1.53–4.31)	<.001
5. Subtraction + FLAIR volume	2.66 (1.61–4.38)	<.001	2.48 (1.47–4.18)	.001

Assessment per Treatment Group

The post-hoc power analysis showed that power in the lomustine-only patient group was well below 80%, precluding meaningful analysis. Within the bevacizumab-treated group, power was also insufficient for a meaningful analysis at first follow-up for method 3 and at second follow-up for methods 2, 3, and 5. Differences in power between methods can be attributed to a difference in the number of patients available for analysis per method as well as differences in the number of patients classified as PD and non-PD.

In the bevacizumab-treated group, significant differences in OS (Figure 4) were found between patients with PD and non-PD as determined with methods 1, 2, 4, and 5 at first follow-up, and with methods 1 and 4 at second follow-up (Table 2). The highest risk increase at first follow-up was found for PD as determined with method 2 (CE volume), with an HR of 7.21 (95% CI, 3.20–16.22). This was, however, not significantly different from other methods. At second follow-up, only the results from methods 1 (RANO criteria) and 4 (combined analysis of CE and FLAIR volume) were deemed reliable (ie, power >80%), with HRs of 2.44 (95% CI, 1.46–4.08) and 2.08 (95% CI, 1.16–3.73), respectively. These were not significantly different from each other. For a graphic depiction of the direct comparison between the HRs and 95% CIs, see the Supplementary files (Figure S1).

Fig. 4.

Kaplan–Meier curves of progressive (PD) versus non-progressive (non-PD) bevacizumab-treated patients only for methods with sufficient power (>80%) at first follow-up. (A) 2D RANO, (B) contrast-enhancing volume, (C) contrast-enhancing + FLAIR volume, and (D) subtraction + FLAIR volume. And at second follow-up (E) 2D RANO, and (F) contrast-enhancing + FLAIR volume.

Table 2.

Hazard ratios of methods with sufficient power (>80%) for bevacizumab-treated patients only at first and second follow-up (FU1 and FU2)

Method	HR (95% CI) FU1	P-value	HR (95% CI) FU2	P-value
1. 2D RANO	5.53 (3.12–9.80)	<.001	2.44 (1.46–4.08)	.001
2. Contrast-enhancing volume	7.21 (3.20–16.22)	<.001
4. Contrast-enhancing + FLAIR volume	5.94 (3.06–11.53)	<.001	2.35 (1.32–4.19)	.004
5. Subtraction + FLAIR volume	5.63 (2.66–11.93)	<.001

Concordance Between Methods

Concordance rates between method 1 (2D RANO) and the volumetric methods (2–5) at first follow-up in all patients together and in the bevacizumab-treated group varied between 81.3% and 87.4%. For a more detailed report and individual examples, see the Supplementary files (Table S1 and Figures S2 and S3). There are several underlying reasons for discrepancies between methods in individual cases. The main reasons were: (i) PD based on the increase of FLAIR abnormalities, which was not picked up using methods 2 and 3 (enhancing/subtraction volume); (ii) PD determined in 2D RANO based on mixed response, not picked up with volumetric methods because change in total (non-)enhancing volume was analyzed; (iii) lesions that did not reach the threshold for PD in volumetry (≥40% increase), while having reached the threshold for PD in 2D RANO (≥25% increase); and (iv) significant increase in FLAIR volume in volumetry (≥25% increase), not scored as significant in 2D RANO (no threshold).

Discussion

We investigated whether PD, as determined with various MRI-based methods, was associated with OS, the gold standard in oncology studies, in order to identify the optimal method for radiological treatment response assessment, in particular in the context of anti-angiogenic treatment of recurrent glioblastoma. Currently, the 2D RANO criteria are the established method of choice for determining PD in studies on glioblastoma. We found that in patients with recurrent glioblastoma treated with bevacizumab determining PD with volumetric methods, with or without subtraction, did not provide significant improvement as a posttreatment prognostic marker at 6 and 12 weeks follow-up. The volumetric methods assessed were based on enhancing lesions only (methods 2 and 3) or on the combined analyses of enhancing and FLAIR lesions (methods 4 and 5). The volumetric methods were also not significantly different from each other. In the past, several studies on the added value of volumetry in recurrent glioblastoma in determining OS have been performed,^11,21,22 most notably by Boxerman et al,¹³ in which a direct comparison is made between 2D and volumetric methods. We analyzed all patients together and the bevacizumab-treated patients separately. Both of these analyses failed to show significant differences between methods. The lomustine-only group comprised one-third of all patients, but because this group was too small to draw reliable conclusions from separate analysis, its influence on the analysis of all patients together is unclear. This is, however, of less importance, since our main goal was to look at the added value of volumetry (and subtraction) in bevacizumab-treated patients. A larger dataset would be required to draw definitive conclusions about lomustine-treated patients.

Comparisons between 2D (and other linear) and volumetric measures have been investigated extensively in a whole range of different tumors,^23–25 including glioma, with contradictory results. Dempsey et al compared 1D, 2D, and volumetric measurements of enhancing volume of high-grade gliomas and found only volumetry to be predictive for OS.²⁶ Shah et al, on the other hand, compared linear measures with volumetry and found these comparable when correlated with median PFS, but found linear methods superior when correlated with OS.²⁴ Galanis et al compared 1D, 2D, area, and volume measures of enhancing (and non-enhancing) newly diagnosed gliomas and found with a time-dependent Cox model that PD measured by all 4 methods was predictive of OS in enhancing tumors.²⁷ Boxerman et al compared 2D and volumetric measurements of enhancement and the added value of non-enhancing volume in recurrent glioblastoma treated with bevacizumab plus a chemotherapeutic agent and found that 2D and volumetric measurements were equally good at predicting OS based on PD at 8 and 16 weeks.¹³ Our results are in accordance with these findings. The main differences between their and our studies are that we compared the current gold standard, the 2D RANO criteria, with various volumetric methods including subtraction, and that our study had 3 different treatment arms. It must be noted that the 2D (RANO) assessment of our study was performed by raters with extensive clinical experience (ie, a neuro-oncologist and 2 neuro-radiologists). This may have had a positive influence on the performance of the 2D method in predicting OS. Semi-automated volumetric assessment was performed by a single rater. Previous studies have, however, shown inter- and intrarater variability to be lower in computer-assisted (volumetric) methods compared with (non-automated) diameter methods.^17,28

Aside from an early pseudo-response, bevacizumab-treated glioblastoma patients have also shown an increase of T2w/FLAIR abnormalities at progression.^9,29 This non-enhancing tumor progression is likely due to the cooption of blood vessels by tumor cells, along which the tumor infiltrates to more distant areas.³⁰ The 2D RANO criteria adopted a subjective approach for assessing T2w/FLAIR abnormalities. Volumetric methods assessing T2w/FLAIR are potentially better suited to quantify these abnormalities, especially in view of the often complex shape of glioblastoma. We assessed the added value of volumetric FLAIR measures to the enhancement (and subtraction) measures and found no differences with the 2D RANO criteria. We did not assess FLAIR volume separately, because previous research suggests that volumetric FLAIR measures alone are insufficient for predicting OS. Ellingson et al, for instance, found that FLAIR volume (initial, residual, and change) was not predictive of either PFS or OS.¹¹ Similarly, Schaub et al determined that PD on FLAIR measures alone was not predictive of OS.²² Huang et al found a correlation between posttreatment T2w/FLAIR volume and OS, but this correlation disappeared when corrected for enhancing volume.²⁰

In addition to measuring enhancing tumor volume on T1w postcontrast and non-enhancing tumor volume on FLAIR images, we measured enhancing tumor volume on subtraction images. Patients treated with bevacizumab may develop T1 hyperintense lesions in a previously enhancing area with a reported prevalence between 20% and 80%.^15,31–33 These lesions likely represent calcifications.³¹ The presence of T1 hyperintensities might hinder assessment of residual enhancement. Also, enhancement may become more vague, interfering with measurement. Ellingson et al found that subtraction images improved visualization, tumor volume quantification, and prediction of OS in patients with recurrent glioblastoma treated with bevacizumab in comparison to T1w postcontrast images.¹⁵ We found that prediction of OS by assessing PD on subtraction (method 3) and subtraction plus FLAIR volumes (method 5) was similar to that with methods not using subtraction techniques. It must be noted that the number of scans suitable for this analysis was limited due to the multicenter study designs with hospitals using scanners from different vendors, with varying data quality and protocol violations. This was especially problematic for subtraction methods, because matching 3D pre- and postcontrast T1w images were commonly not available. Unfortunately, this meant that not all subtraction-based methods could be reliably assessed in the bevacizumab-treated patient group. These technical shortcomings unfortunately resemble the real-life conditions under which clinical trials are being conducted. The recently published standardized MRI brain tumor imaging protocol should overcome some of these technical difficulties.³⁴

There are several important considerations when designing novel ways to assess outcome in phase I and phase II trials. First and foremost, they must correlate with the final endpoint used in phase III trials, they must be reliable, and they must be feasible in large multicenter trials. A new method must either bring a clear benefit in being more accurate or bring an advantage in being simpler but with a similar precision. Volumetric assessment is more precise in irregularly shaped tumors, but it is also labor-intensive, time-consuming, and more complex, also in the case of semi-automated techniques. It is also not readily available in many institutions. Our data suggest that volumetric assessment is not better compared with classical 2D assessment. Therefore, there seems to be no rationale at this point to start using volumetric assessment in trials on recurrent glioblastoma.

There are some limitations to consider in this retrospective study. As previously mentioned, rater experience may have positively influenced the 2D RANO assessment. The interrater variability for assessment according to the 2D RANO from this trial will be reported separately. It must also be noted that the volumetry measurements were performed by a single observer. Secondly, our data were analyzed at first and second follow-up and presence of PD was determined at these time points only. Later follow-up data were not included in this study, because there were not enough patients who had not yet shown PD left to obtain reliable results.

Our future efforts are aimed at exploring more advanced imaging techniques, such as diffusion imaging–derived apparent diffusion coefficient, as well as relative cerebral blood volume values from enhancing and T2w/FLAIR lesions in bevacizumab-treated patients. Additionally, in future exploratory analyses, different volumetric thresholds for determining PD and a combination of enhancing volume plus qualitative FLAIR assessment will be investigated to provide more insight into the added value of volumetry in this patient group and that of FLAIR in particular.

We conclude that the current, widely used, and easily applicable 2D RANO criteria remain valid for response evaluation in patients with recurrent glioblastoma treated with bevacizumab. Volumetric and subtraction evaluation methods failed to yield a superior correlation with OS in the first 12 weeks in this patient group. Our data therefore do not support their routine use in clinical trials.

Supplementary Material

Supplementary data are available at Neuro-Oncology online.

Funding

This work was supported by Roche Nederland and KWF Kankerbestrijding.

Conflict of interest statement. None declared.

No unpublished papers cited.