Validation and Simplification of the Radiation Therapy Oncology Group (RTOG) Recursive Partitioning Analysis (RPA) Classification for Glioblastoma (GBM)

Abstract

Purpose

Previous recursive partitioning analysis (RPA) of patients with malignant glioma (glioblastoma multiforme [GBM] and anaplastic astrocytoma [AA]) produced six prognostic groups (I-VI) classified by six factors¹. We sought here to determine whether the classification for GBM could be improved by using an updated RTOG GBM database excluding AA and by considering additional baseline variables.

Patients and Methods

The new analysis considered 42 baseline variables and 1672 GBM patients from the expanded RTOG glioma database. Patients receiving radiation only were excluded such that all patients received radiation+carmustine. “Radiation dose received” was replaced with “radiation dose assigned.” The new RPA models were compared to the original model by applying to a test dataset comprising 488 patients from six other RTOG trials. Fitness of the original and new models was evaluated using explained variation.

Results

The original RPA model explained more variations in survival in the test dataset than did the new models (20% vs. 15%) and was therefore chosen for further analysis. It was reduced by combining classes V and VI to produce three prognostic classes (III, IV, V+VI), as classes V and VI had indistinguishable survival in the test dataset. The simplified model did not further improve performance (explained variation 18% vs. 20%) but is easier to apply because it involves only four variables:age, performance status, extent of resection, and neurologic function. Applying this simplified model to the updated GBM database resulted in three distinct classes with median survival times of 17.1, 11.2, and 7.5 months for classes III, IV, and V+VI, respectively.

Conclusions

The final model, the simplified original RPA model combining classes V and VI, resulted in three distinct prognostic groups defined by age, performance status, extent of resection, and neurologic function. This classification will be used in future RTOG GBM trials.

INTRODUCTION

Despite intense research efforts over the past 4 decades, the prognosis for patients with malignant glioma, particularly glioblastoma multiforme (GBM), remains dismal.^2-10 The median survival time for GBM patients has remained poor at approximately 12 months for a very long time. An important recent improvement is the concurrent and adjuvant use of temozolomide with radiation.¹¹ An update of that study showed that the survival benefit of temozolomide persisted, but the 5-year overall survival rate of 10% is still rather dismal, and further clinical research efforts are clearly warranted.¹²

A recursive partitioning analysis (RPA) of prognostic factors in 3 RTOG trials including 1578 patients with GBM or anaplastic astrocytoma (AA) was performed in the early 1990s,^1,13 This analysis generated six prognostic classes (I-II for AA, III-VI for GBM) with median survival time ranging from 58.6 months to 4.6 months and 2-year overall survival rates ranging from 76% to 4%. Among the 26 pretreatment patient/tumor factors and 6 treatment factors entered into this regression analysis, six were significant: age (< 50 versus ≥ 50 years) produced the most significant split, followed by histology (AA versus GBM) for younger patients and performance status for older patients, and then mental status. Treatment-related factors proven significant enough to be included were extent of surgery and radiation dose delivered. The reproducibility of this RPA classification system was later verified using patients from the RTOG 90-06 trial.¹³

Since its development in the early 1990s, this classification has been used in the design, stratification, and outcomes comparison for multiple GBM trials. With the availability of more patients from additional RTOG trials and increasing use of chemotherapy, it was unclear whether the original RPA model remained optimal. This was particularly concerning for patients with GBM (classes III-VI), the outcome for whom is much worse than that for patients with AA (median survival time, 3-5 years). We therefore undertook a new RPA involving 1672 GBM patients from 5 RTOG trials (the training dataset), all of whom received both radiation and carmustine, and none received temozolomide, with the goal of optimizing and updating the prior RPA classification specifically for GBM patients. We report here our evaluation of new RPA models versus the original model for goodness of fit and the ability to explain the most variation in survival with an additional test dataset comprising patients from six different RTOG trials. Our findings lead us to propose that a simplified model of the original RPA classification involving only four prognostic factors is sufficient for identifying three prognostic subgroups of patients with GBM.

PATIENTS AND METHODS

Patient population

Training database

Patients entered in one of the five consecutive RTOG trials for biopsy-proven, supratentorial GBM were used as the training dataset for building the new RPA model.^{8, 14-18} The original RPA was based on 1288 patients with GBM and 290 patients with AA in RTOG trials 74-01, 79-18, and 83-02. For the new analyses, we deleted patients who had received radiation only (arm 1 and 2 of trial 74-01) and added patients from RTOG studies 90-06 and 94-11. This resulted in the expanded (training) database of 1672 GBM patients who received radiation plus carmustine or another nitrosourea (Table 1). Primary outcome reports of these trials have been published. Eligibility criteria were consistent in the five studies and included the following: histologically confirmed supratentorial malignant glioma, age 18-70 years, an interval of 4 weeks or less from surgery to registration, and normal hepatic, renal and bone marrow function. Ineligibility criteria included prior malignancies except skin carcinomas and prior chemotherapy or head and neck irradiation.

Table 1.

RTOG studies comprising the training dataset (model building) and testing dataset (model testing)

Trial	No. GBM Patients	Treatment Groups
Training Dataset
74-01	231*	60 Gy + carmustine
		60 Gy + 4-methyl-lomustine + dacarbazine
79-18	243	60 Gy + carmustine
		60 Gy + misonidazole + carmustine
83-02	559	Hfx RT (64.8, 72.0, 76.8, 81.6 Gy) + carmustine
		Accel hfx RT (48.0, 54.4 Gy) + carmustine
90-06	531	60 Gy + carmustine
		Hfx RT 72 Gy + carmustine
94-11	108	Hfx RT 64.0 Gy + carmustine
		Hfx RT 70.4 Gy + carmustine
Total	1672
Testing Dataset
76-11	122	50 Gy + 15Gy photon or neutron boost
79-03	18	Neutrons 18 Gy + misonidazole
80-07	159	45 Gy + neutrons (3.6, 4.2, or 4.8 Gy)
84-09	44	60 Gy + aziridinylbenzoquinone
95-13	84	60 Gy + topotecan
96-02	61	60 Gy + paclitaxel
Total	488

^*Excludes patients who had received radiation only (arms 1 and 2).

Testing database

The testing database consisted of 488 patients with GBM from three older RTOG trials (76-11, 79-03, 80-07) and three more recent RTOG trials (84-09, 95-13, 96-02) (Table 1).

Prognostic variables

The original model used 32 variables; for the new analysis, we considered 42 variables (Table 2). Education level, hemoglobin level, and baseline score on the Mini-Mental Status Examination were added. Because the RTOG now uses Zubrod scoring for all trials, the variable ‘Karnofsky Performance Score’ was transformed to the equivalent Zubrod score in the new analyses. ‘Radiation dose received’ had been used as a variable in the original analysis; however, because this variable is not a pretreatment characteristic per se, ‘radiation dose received’ was deleted and the variable ‘radiation dose assigned’ was used instead.

Table 2.

Variables included in the new recursive partitioning analyses for model building^*

Demographics Age Sex Race Educational levela,b

Status Zubrod score f Neurologic function c Baseline MMSE*

Prior Treatment Surgery (biopsy vs resection)

Previous Neurologic Symptoms Headache Visual disturbance Speech impairment Sensory symptoms Memory symptoms Personality change Seizures

Neurologic Findings at Registration Mental status Somnolence Papilledema Motor deficit Cranial nerve deficit Sensory deficit Cerebral deficit

Laboratory Parameters Hemoglobin levela

Chronic Diseased Cardiac disease Hypertension Respiratory disease Diabetes

Tumor Characteristics Tumor lateralization Maximum tumor diameter (< vs. ≥ 5 cm)

Primary Tumor Locatione Frontal, temporal, parietal, occipital, deep, brainstem, corpus callosum

Treatment Radiation dose assigned f Radiosensitizer Carmustine

Extent of Resection Total or partial Biopsy

MMSE, Mini-mental status examination; GBM, glioblastoma multiforme; AA, anaplastic astrocytoma.

^*Variables used in the original model but not in the new model: symptom durations, RT fraction size, interfraction RT interval

^aNew variable (not considered in the original model)

^bMeasured only in RTOG 94-11.

^cclass 1, able to work; class 2, able to be at home; classes 3 & 4, hospitalized

^dRenal and liver disease excluded because fewer than 50 patients had either condition.

^eCerebellum, spinal cord, and other were excluded because fewer than 50 patients had lesions in these locations.

^fVariable changes: Zubrod score in place of KPS, RT dose assigned in place of RT dose received.

Statistical methods

RPA was used to establish prognostic groups as described previously.¹ Recursive partitioning is a method of building decision trees to model predictors.¹⁹ It uses Kaplan-Meier estimates of survival²⁰ and modified Wilcoxon tests²¹ to establish branches in the decision tree.²² More specifically, it examines all possible cut-points for all variables entered into the model. These cut-points divide the dataset into two relatively homogeneous populations that are significantly different with respect to survival. The best cut-point or split is chosen if (1) it provides the greatest separation in survival, based on the product-limit estimate of the survival function; (2) the p value calculated using modified Wilcoxon statistics is significant after adjustment for multiple comparisons^{23, 24}; and (3) each group includes sufficient numbers of patients (≥ 25). When no further splits are possible, then the “leaves” of the RPA tree are considered terminal nodes (the entire dataset is considered as the primary node). Terminal nodes that are similar in their survival profiles based on modified Wilcoxon tests are merged into distinct RPA classes.

After the new RPA models were built using the expanded GBM (training) dataset, those models were tested on a separate test dataset to see if a given model’s RPA classes were statistically distinguishable with respect to survival. The test set consisted of 488 GBM patients from six RTOG trials (76-11, 79-03, 80-07, 84-09, 95-13, and 96-02) (Table 1). To adjust for multiple comparisons, a significance level of 0.05/(N-1), where N equals the number of classes in the model, was used.

The new RPA models built on the expanded GBM dataset were then compared to the original model and to a simplified original model that combined classes V and VI by using a test dataset. Because the purpose of regression modeling techniques such as RPA is to account for heterogeneity in survival by using covariates, the squared error loss function defined by Korn and Simon²⁵ was used as the statistical metric for comparison of fitness among the different RPA models. This loss function permits calculation of the percentage of explained variation for survival by each model.

RESULTS

New RPA models using the updated RTOG GBM database

Expanded GBM (training) database

The first question we asked was whether restricting the analysis to only patients with GBM (i.e., excluding patients with AA) and adding patients from newer studies to the original database would result in a better RPA model with more distinct separation of risk groups while being easier to apply. An expanded training database was constructed consisting of 1672 GBM patients from 5 consecutive RTOG trials who received radiation plus carmustine or another nitrosourea (Table 1). Forty-two pretreatment patient/tumor factors were entered in the analysis as detailed in Table 2.

Patient characteristics

The key baseline patient characteristics are listed in Table 3. The median follow-up time was 10.2 months for all 1672 patients and 70.4 months for living patients (n=57). Median age was 57 (range 18-83). At the time of enrollment, 94% patients had normal mental status or only minor confusion. Almost all patients had some neurologic deficits. As for the treatment received, 80% patients had surgical resection, and 19% had biopsy only. The intended radiation dose was greater than 54.4 Gy in most patients (87%).

Table 3.

Key pretreatment patient characteristics.

	n	%
Age
<50	489	29
50+	1180	71
Missing	3	<1
Karnofsky Performance Score
<70	265	16
70	269	16
80	415	25
90	512	31
100	125	7
Missing	86	5
Prior Surgery
Biopsy	320	19
Partial Resection	974	58
Total Resecion.	360	22
Other	13	1
Missing	5	<1
Neurologic Function
Minor	816	49
Mod	694	42
Hospital	153	9
Missing	9	1
Mental Status
Normal Function	1017	61
Minor Confusion	581	35
Gross Confusion	66	4
Rousable with difficulty	4	<1
Missing	4	<1
Radiation dose assigned
<= 54.4Gy	214	13
> 54.4Gy	1458	87

New RPA models

The new RPA of the expanded GBM (training) database, including all 42 variables, resulted in a new model (New Model_[42]]) in which seven variables were found to be statistically significant predictors (age, Zubrod score, extent of surgery, neurologic function, memory symptoms, motor deficit, and personality change). This model produced five different prognostic groups. Noting that both the original model and this new model share four common significant variables (age, Zubrod/KPS, extent of surgery, and neurologic function), we used RPA to construct a second new model from the expanded GBM database that allowed only these four variables as input, leaving out memory symptoms, motor deficit, and personality changes (New Model_[4]). This simplified new model, using four variables, resulted in four prognostic groups. The obvious advantage of such a model is ease of use by clinicians attempting to stratify patients in terms of risk.

Simplified version of the original RPA model

When the original model (Fig. 1A) was applied to the test dataset as described in the following section, no statistically significant difference between the original RPA classes V and VI was observed. This lack of difference probably reflects the change from using ‘radiation dose received’ (a posttreatment variable) to ‘radiation dose assigned’ (a pretreatment variable). This change would have resulted in reclassifying patients who received lower radiation doses from RPA class VI to class V, which presumably would artificially improve the outcome for class VI and worsen the outcome for class V—and would also diminish the ability to distinguish the two subgroups. Clinically, combining classes V and VI makes sense, as modern-day radiation doses for GBM typically exceed 54.4 Gy. Because of the disappearance of a distinction between the original RPA classes V and VI when this one variable was redefined, we constructed a simplified version of the original RPA model (Original RPA model_[V+VI], Fig. 1B) in which RPA classes V and VI were combined. This conveniently left the original model with the same four variables as in the new RPA model_[4]: age, Zubrod/Karnofsky score, extent of surgery, and neurologic function.

Fig 1.

Tree diagrams of the different RPA models. IA, original RPA model; IB, original RPA model combining classes V and VI

Testing fitness of the four RPA models using a different RTOG GBM database

Distinction among the RPA classes in the different models

A testing database of 488 GBM patients from 6 RTOG studies different from the training database was used to test each RPA model. The initial testing involved determining whether a given model’s RPA classes were statistically distinguishable when applied to the test dataset. Interestingly, only the simplified original model that combined V and VI (simplified original model_[V+VI]) led to statistically significant differences between each RPA class, after adjustment for multiple comparisons (Fig 2). The two new models led to the least distinction between classes. Results were similar when a smaller test dataset consisting of only the three most recent studies (84-09, 95-13, and 96-02) was used (data not shown).

Fig 2.

Kaplan-Meier survival curves and associated median survival times for the different RPA models when applied to the test dataset for the original RPA model combining classes V and VI.

Comparison of the new and original models

Next, to assess the fit of each model to the data, we calculated the amount of variation explained by each model, with higher values indicating better fit. Calculations were done first with all six trials (76-11, 79-03, 80-07, 84-09, 95-13, and 96-02) and then again separately for only the latter three, more recent studies. In both scenarios, the original model explained more variation in survival (20%) than did any of the other three models (Table 4), most notably in the more recent studies (28%). The two new RPA models have the lowest of the explained variation values regardless of whether the full test dataset of all six studies were used or only the most recent three studies. These results clearly indicate that the original model is a better fit to the test database than the new RPA models. Combining classes V and VI of the original model resulted in a sufficiently homogeneous subgroup in terms of survival, with an explained variation of 18%.

Table 4.

Explained variation at 2 years using the test database

Patient Groups	RPA Models
	Original Model*	Original Model[V+VI]*	New Model[42]	New Model[4]
All Studies in Test Set (RTOG 76-11, 79-03, 80-07, 84-09, 95-13, and 96-02)	20%	18%	15%	16%
Recent Studies Only (RTOG 84-09, 95-13, 96-01)	28%	19%	10%	14%

^*Modified to use the variable ‘radiation dose assigned’ as opposed to ‘radiation dose received’

Final model: the reduced original model

We chose the reduced original model that combined classes V and VI (original model_[V+VI]) for its relatively high explained variation and its ease of use. It includes only four variables: age (< vs. ≥ 50 y), Karnofsky/Zubrod score (< vs. ≥ 90 for patients < 50 y or < vs. ≥ 70 for patients ≥ 50 y), extent of resection (resection vs. biopsy), and neurologic function (able to work vs. not). Applying this simplified version of the original RPA model to the updated/expanded GBM (training) database produced three prognostic classes: III, IV, and V+VI, with median survival times of 17.1, 11.2, and 7.5 months, respectively. The corresponding overall survival rates for patients in classes III, IV, and V+VI were 70%, 46%, and 28% at 1 year, and 20%, 7%, and 1% at 3 years (Table 5).

Table 5.

Application of the simplified original RPA model to the expanded RTOG GBM (training) database

RPA class	Median Survival	Median Survival Time	Overall Survival Rates
			1 year	3 years	5 years
III	< 50 y and KPS ≥ 90	17.1 months	70%	20%	14%
IV	< 50 y and KPS < 90;	11.2 months	46%	7%	4%
	≥ 50 y, KPS ≥ 70, resection, and working;
V+VI	≥ 50 y, KPS ≥ 70, resection, and not working;	7.5 months	28%	1%	0%
	≥ 50 y, KPS ≥ 70, biopsy only;
	≥ 50 y, KPS < 70

RPA, recursive partitioning analysis; RTOG, Radiation Therapy and Oncology Group; GBM, glioblastoma multiforme; KPS, Karnofsky Performance Score.

DISCUSSION

In the present study, we set out to determine whether the original RPA model’s classification of risk factors for patients with GBM established by Curran et al.¹ could be improved by restricting the model to only patients with GBM, by updating the RTOG glioma database by adding patients from newer studies, and by considering additional baseline variables. We found that the original RPA model outperformed the new models by explaining more of the variation in survival. Our final choice of model was a simplified version of the original RPA model in which classes V and VI were combined, which resulted in three distinct prognostic groups defined by four prognostic factors: age, performance status, extent of resection, and neurologic function.

Since its initial development in the early 1990s, the RTOG RPA classification system has proven useful and has been validated in multiple clinical trials.^{11, 17, 26-32} It has served as a historical control to compare findings from phase I/II clinical trials before phase III studies are begun that may otherwise have been based on false expectations. It also identified relatively homogeneous patient subgroups that may benefit most from a particular experimental approach, therefore sparing other patients from unnecessary treatment. The reduced RPA classification presented in this study is much easier to apply because it involves only four prognostic factors rather than the six in the original model, and has three risk groups instead of four. The reduced model does not consider either radiation dose or mental status. As stated above, radiation dose was dropped from the model because of the change from ‘radiation dose received’ to ‘radiation dose assigned,’ because patients with GBM are commonly treated to doses higher than 54.4 Gy. Mental status was no longer a splitting node in the reduced model, probably because it interacts with or is confounded by other prognostic factors such as age, performance status, and neurologic function.

Our findings in this study indicate that pretreatment patient and tumor factors continue to play an important role in the disease course and treatment outcome for patients with GBM. Even though the data used for the initial RPA were collected from 1974 to 1985, and advances have been made since then in diagnostic techniques, radiation therapy techniques, and chemotherapy, the initial classification, with slight modifications, can still be applied to patients undergoing current treatments. This contention is confirmed by several validation studies performed in the late 1990s,^{13, 33, 34} and by our new analysis presented here, which involved data collected from patients treated from 1974 to 1995 who all received radiation and carmustine or another nitrosourea. The RTOG RPA classification for GBM was further confirmed by the results of the EORTC/NCIC study,¹¹ which used a modified version of the RTOG RPA classification with the same four prognostic factors as in the simplified model.³⁵ In these more recently diagnosed patients who were treated with modern radiation techniques and temozolomide, median survival times were 17 months, 15 months, and 10 months for classes III, IV and V, which were remarkably similar to the results of our study.. The 10-month survival time for patients in class V in the EORTC study as compared to 7.5 months for patients in classes V+VI in our study probably reflects the omission of class VI patients, the worst performing group, in the EORTC RPA. The robustness of the initial RPA system, derived from data collected over periods in which different treatment regimens were used, suggests that the pretreatment patient-related and tumor-related factors probably continue to have a greater impact on outcome than do treatment factors (e.g., radiation dose, use of a radiosensitizer, and chemotherapy agents). The observation that median survival times for the three RPA classes in the EORTC/NCIC study were not improved as compared with those in our study indicates that vigorous research for more effective treatments for GBM is still needed.

One of the 42 variables entered used in the model building was assigned radiation doses with 54.4 Gy as the cutoff point. Although radiation dose was not found to be one of the four prognostic factors included in the final RPA model and was not a focus of the current study, one may argue about the clinical relevance of 54.4 Gy as it is different from current standard practice of 60 Gy at 2 Gy daily fractions. The cutoff point of 54.4 Gy was established in the original RPA by WJ Curran et al ¹. In Curran’s analysis, multiple radiation dose cut-off points (≤ 54.4, 54.5-59.9, or 60-72 or >72 Gy) were entered into the RPA model, and doses greater than 54.4 were found to be associated with better outcome. These doses were not necessarily associated with accelerated hyperfractionation in all studies, nor with neutrons. In the broad range from 54.5 to 72 Gy, no specific higher dose cut-off point, such as 60 Gy, was identified, suggesting that the difference in the outcome between 54.4 and 60 Gy is perhaps insignificant.

Nomograms have been generated based on the EORTC/NCIC study to predict outcome for individual patients with newly diagnosed GBM.³⁶ In patients who received radiation and temozolomide, similar prognostic factors as those used in RPA classification were identified, including age, performance status, extent of resection, and mental status as assessed by the Mini Mental Status Examination. The major advantages of these nomograms include more individualized prediction of particular patient’s survival and probably improved accuracy compared to RPA. This study also allowed inclusion of biological prognostic factors such as MGMT methylation status that are not addressed in the RTOG RPA model. The limitation of the nomogram as a prognostic system is that it has not been validated in a separate independent dataset because of the relatively short history of using concurrent radiation and temozolomide for GBM and therefore the lack of a large enough database. With the recent completion of the RTOG trial 0525 with more than 1000 patients, we expect to be able to use data from that study to validate the nomograms, expand on the model by including factors such as MGMT methylation, and also evaluate the impact of temozolomide, with a historic comparison to carmustine. The validated and improved prognostic system may well be useful for predicting an individual patient’s outcome, assisting in management decision-making, and improving the design of future clinical trials.

In addition to MGMT methylation status, other biomarkers such as EGFR and PTEN have been under vigorous investigation as emerging prognostic or predictive markers in GBM. The recent discovery of the R132H (arginine to histidine) somatic mutation in the IDH1 (isocitric dehydrogenase-1) gene through whole-genome sequencing analysis of GBM samples is of particular interest.^37-41 Subsequent studies showed that the IDH1 R132H mutation is present in more than 80% of grade II and III gliomas as well as in secondary glioblastoma, and is implicated in the early pathogenesis of these diseases. This mutation has also been found in cytogenetically normal acute myeloid leukemia. Wild-type IDH catalyzes the NADP-dependent conversion of isocitrate to alpha-ketoglutarate (aKG), which is lost in the mutant IDH1. Interestingly, same mutation also acts in a dominant-negative manner to gain a new ability to catalyze the NADPH-dependent reduction of aKG to 2-hydroxyglutarate (2HG), which resulted in ~100-fold elevation of 2HG, considered an onco-metabolite, in human glioma samples.^37-41 Further understanding of this pathway will provide not only a target for therapeutic intervention but also potential serum markers for screening and for early diagnosis, before the tumors undergo malignant transformation. Another area of active research is gene profiling studies to identify gene expression patterns that can help classify tumors into prognostic groups by using DNA microarrays. In this regard, high grade astrocytoma has been divided into three discrete prognostic subclasses resembling stages in neurogenesis: proneural, proliferative, and mesenchymal.⁴² Of these three subclasses, the mesenchymal subtype has the worst prognosis, and recurrent tumors frequently shift toward the mesenchymal subclass.⁴² Taken together, these studies may further aid in more refined assessment of patients’ risks at the time of diagnosis and subsequently in improved individualized and targeted treatment. An update of the RPA classification or nomogram to include these new markers will undoubtedly be needed once their roles are established.

One important limitation of the present study is the need to use a combined database from multiple trials spanning more than 4 decades, during which diagnosis and treatment for GBM have evolved substantially. The therapy (both the chemotherapy and the radiation) for many patients in both the training and the testing databases was outdated in comparison to current practice. Although this is an unavoidable consequence secondary to the uncommoness of the disease and would not preclude a retrospective analysis of prognostic factors and outcome, it is critical to be aware of this limitation when interpreting the results and applying them to clinical practice or the design of clinical trials. Also, advances in the pathologic identification of GBM over the past 4 decades may also raise questions as to the validity of the diagnosis of GBM, especially in long-term survivors.

Abbreviations

RTOG

Radiation Therapy Oncology Group

KPS

Karnofsky performance score

RPA

recursive portioning analysis

GBM

glioblastoma

AA

anaplastic astrocytoma