Internal validation of the prognostic index for spine metastasis (PRISM) for stratifying survival in patients treated with spinal stereotactic radiosurgery

Abstract

Purpose

We sought to validate the Prognostic Index for Spinal Metastases (PRISM), a scoring system that stratifies patients into subgroups by overall survival.

Methods and materials: The PRISM was previously created from multivariate Cox regression with patients enrolled in prospective single institution trials of stereotactic spine radiosurgery (SSRS) for spinal metastasis. We assess model calibration and discrimination within a validation cohort of patients treated off-trial with SSRS for metastatic disease at the same institution.

Results

The training and validation cohorts consisted of 205 and 249 patients respectively. Similar survival trends were shown in the 4 PRISM. Survival was significantly different between PRISM subgroups (P<0.0001). C-index for the validation cohort was 0.68 after stratification into subgroups.

Conclusions

We internally validated the PRISM with patients treated off-protocol, demonstrating that it can distinguish subgroups by survival, which will be useful for individualizing treatment of spinal metastases and stratifying patients for clinical trials.

INTRODUCTION

Stereotactic spine radiosurgery (SSRS) has been shown to relieve pain and provide durable control for select patients with spinal metastases[1–5] without substantial toxicity.[6–9] It has demonstrated significantly better progression free survival and improved toxicity in separate case-matched analyses as compared to conventional radiation. [10,11] Ergo, along with conventional external beam radiation treatment (EBRT) and surgery, SSRS is one of the main treatment modalities for spine metastasis and is fast emerging as the modality of choice for de novo and postsurgical adjuvant therapy as well as for reirradiation of recurrent disease.[12] However, SSRS requires additional resources in planning time, personnel involvement, and technical and financial investment.[13,14] Such a treatment should be restricted to patients with a better overall prognosis, such that the improved local tumor control and associated symptom control matches a better overall survival. Patients with poor survival potential may be better served with less intense approaches such as supportive care and conventional radiation. Using incremental cost-effectiveness ratio with effectiveness measured in quality-adjusted life years, it has been suggested that palliative SSRS may be the most cost effective approach once median survival is ≥11 months.[15] However, it remains unclear which patients will reach an acceptable threshold.

Other prognostic models have been created to stratify patients with spinal metastases after surgical resection or conventional radiation.[16–20] However, previously described models for evaluating survival after SSRS are limited in nature.[21] The PRISM (Prognostic Index for Spine Metastasis) model, a survival stratification score, was created to identify outlying patients with poor and excellent survival. The training dataset used to create this score consisted of 2 mature prospective institutional trials investigating SRSS.[22]

As a first step toward validating these results, we retrospectively analyzed pretreatment variables and subsequent survival of a large cohort of patients who had received SSRS off-trial at the same institution.

METHODS AND MATERIALS

Patient Selection

Details of the development cohort (the training set) used to derive the PRISM model are reported elsewhere.[22] Briefly, the training set comprised 206 patients, treated from 2002-2011, who had participated in one of two phase I/II trials to evaluate single-fraction or multifraction SSRS for spinal metastasis at a single institution (NCT01256554 and NCT01254903). Indications for treatment on these trials were oligometastatic disease, residual tumor after surgery, ineligibility for surgery owing to medical comorbidity, refusal of surgery, failure of previous local therapy, or radiation-resistant disease. Protocol inclusion criteria included a Karnofsky performance score (KPS) >40, histopathologic confirmation of cancer at the same institution, and available magnetic resonance imaging scans identifying spinal or paraspinal metastasis obtained within 1 month before enrollment. Protocol exclusion criteria included spinal cord compression or spinal instability as determined by evaluation, use of SINS score, and use of an MESCC grading system for epidural disease.[23,24] All eligible patients were deemed appropriate for spinal radiosurgery by a multidisciplinary SSRS tumor board.

For the current study (the internal validation of the PRISM model), we evaluated another 249 patients who had received single- or multi-fraction SSRS for spinal metastases at the same institution from March 19, 2003, through June 2, 2014. These patients (the validation set) were off-trial. Patients who received SSRS more than once were evaluated solely on the initial instance of SSRS performed at this institution. If SSRS was directed to different sites within 1 month, those sites were considered to have been treated simultaneously. Institutional Review Board approval was granted for this retrospective study, and the requirement for informed consent was waived.

Treatment

During the time span of this study, treatment platforms changed and evolved. Patients who had been treated off-trial (the subjects of the current validation analysis) underwent intensity-modulated, image-guided SSRS delivered with a CT-on-rail target system with 2100EX (Varian Medical Systems, Palo Alto, CA), Trilogy with CBCT (Varian), sequential dual x-ray ExacTrac patient positioning system (Brainlab, Feldkirchen, Germany) and CT-on-rail with 2100EX(Varian), or simultaneous dual x-ray 6-D ExacTrac patient positioning system (Brainlab) and CBCT Truebeam STX (Varian) (since April 2013 – present). Patient setup has been continuously evolving with technology advancements to improve safety, efficacy, and confidence in patient setup accuracies. For fractionation and dosages used refer to Table 1.Technique and dosimetric data has previously been published. [1,8,25,26]

Table 1.

Baseline patient and treatment characteristics

Characteristics	Training Set, no. (%) (n=205)	Validation Set, no. (%) (n=249)	P-value
Sex
Male	95 (46)	145 (58)
Female	110 (54)	104 (42)	.01
Age at treatment, year,
median (range)	59 (20-87)	59 (16-87)	.36
Karnofsky performance status
70-100	199 (97)	236 (93)
<70	6 (3)	13 (6)	.22
Primary tumor
Renal	77 (37)	87 (35)	.63
Thyroid	22 (11)	21 (10)	.50
Sarcoma	25 (12)	26 (10)	.66
NSCLC	19 (10)	37 (15)	.10
Breast	18 (9)	16 (6)	.44
Other	44 (21)	62 (25)	.45
Prior surgery at SRS Site
No	137 (67)	199 (80)
Yes	68 (33)	50 (20)	.002
Prior radiation at SRS site
No	148 (72)	188 (76)
Yes	57 (28)	61 (24)	.49
No. of vertebrae treated with SRS
1	127 (62)	135 (54)
2	53 (26)	74 (30)
>2	25 (12)	40 (16)	.054
RT regimens*
18 Gy in 1 fraction	35 (17)	41 (16)
24 Gy in 1 fraction	24 (12)	92 (36)
24 Gy in 3 fractions	1 (0.5)	3 (1)
27 Gy in 3 fractions	117 (57)	92 (36)
30 Gy in 5 fractions	28 (14)	17 (7)
Other	0	10 (4)
Other organ systems involved,
no., median (range)	1 (0-5)	1 (0-5)	.79
Intracranial	18 (9)	19 (8)
Liver	35 (17)	44 (18)
Lung	69 (34)	98 (39)
Adrenal	13 (6)	25 (10)
Other	80 (39)	51 (20)

^*In the validation cohort, 6 patients were treated with separate fractionations during the initial SSRS to different sites of disease.

Abbreviations: SRS, stereotactic radiosurgery; NSCLC, non-small cell lung cancer

Evaluation and Follow-Up

In the training set, patient evaluation had been mandated by protocol to include clinic visits at 3, 6, 9, 12, 18, and 24 months after treatment and then every 6 months thereafter. The validation cohort was followed per institution practice every 3 months for 2 years and then every 3-6 months thereafter or as clinically indicated. Survival status of patients who did not return to the clinic was updated by an institutional registry, for which patients or their families are called or national death registries are queried every 12-18 months.

Statistical Analyses

The primary outcome was death from any cause. The PRISM model for predicting overall survival had been created from the training set by means of Cox regression with backward selection at P values of < 0.05 as previously reported.[22] Pretreatment variables entered into the Cox regression model not found to be significant included ethnic group, age, histology, component of vertebrae involvement (epidural, vertebral body, paraspinal, or posterior element), and identity of organ systems involved outside of bone (intracranial, liver, lung, adrenal, and other). Multivariate analysis revealed that 7 pretreatment variables were significantly associated with survival: gender, Karnofsky performance status score, previous surgery at the SSRS site, previous radiation at the SSRS site, if SSRS is to a solitary metastasis, number of organ systems involved with metastases, and the interval between initial diagnosis and detection of metastasis. The resultant PRISM model used these variables to distinguish 4 subgroups of patients at different risks of death.

For the current analysis, patients in the validation cohort were assigned a PRISM score and stratified into survival subgroups. Survival for those subgroups was then estimated with the Kaplan-Meier method, and compared with log-rank tests. The C-index of the stratified (i.e., four ordered subgroups defined in the original paper) and unstratified score was calculated to ascertain discrimination in the validation cohort. To assess prediction calibration, 1- and 2- year Kaplan-Meier survival estimates for the four subgroups in both the training and validation cohorts were compared with using fitted regression lines generated by Excel version 2010 (Microsoft, Redmond, WA). All tests were 2-sided when appropriate, and results were considered significant at P <0.05. Analyses were done with SAS version 9.4 (SAS Institute, Cary, NC) and S+ (TIBCO, Palo Alto, CA) software.

RESULTS

Patient Characteristics

Patient and treatment characteristics are presented in Table 1. Gender was significantly different between the cohorts (54% female in training vs. 42% in validation, p<.01). Otherwise, the two cohorts were well-balanced in terms of number of organ systems involved (median 1, range 0−5), age at treatment (median 59 years), most common type of primary tumor (renal cell carcinoma in 35% training vs. 37% validation), and most common extraosseous organ system involved with metastasis (lung, in 39% training vs. 34% validation).

Regarding treatment, patients in the training set had higher rates of prior surgery (33% vs. 20% for the validation set, p=.002), but a decreased number of vertebrae receiving SSRS (mean of 1.5 vs. 1.7 vertebrae, p=.054). The most common radiation treatment regimens used in both the training and the validation cohorts were 27 Gy in 3 fractions (57% training and 36% validation), followed by 18 Gy in 1 fraction (17% and 16%), and 24 Gy in 1 fraction (12% and 36%). Six patients in the validation cohort had been treated with separate fractionations to different sites of disease during the initial SSRS.

Like the prior training cohort, patients in the validation cohort were assigned a PRISM score (See Table 2) and stratified into survival subgroups (See Figure 1). Respectively, PRISM subgroups 2 and 3 were more heavily represented in the validation cohort (36.5 and 37.3%) than the training cohort (31.1 and 35%).

Table 2.

Prognostic Index for Spinal Metastases (PRISM) score components and groupings

Variables	Score
Female	+2
Performance status	+1 for every 10 pts over 60 on KPS scale
Previous surgery at SSRS site	+2
Previous radiation at SSRS site	-2
Other organ systems involved with metastasis
(other than bone)	-1 per system
SSRS for solitary metastasis	+3
Time between diagnosis and metastasis >5 years	+3
Survival Groups	Score Range
Group 4 (poor prognosis)	<1
Group 3	1-3
Group 2	4-7
Group 1 (excellent prognosis)	>7

Figure 1.

[Histograms shows distribution of survival subgroups determined with the Prognostic Index for Spine Metastasis (PRISM) model among the (A) training set (205 patients treated on-protocol) and the (B) validation set (249 patients treated off-protocol).]

Survival

The training set had longer median overall survival times relative to the validation set: 25.5 months training vs. 17.0 months validation. The median survival time for subgroup 1 was not reached in either cohort. All other PRISM subgroups exhibited higher survival in the training compared with the validation dataset. Median OS for PRISM subgroups 2-4 in the validation cohort was 24.1, 13.1, and 6.5 months, respectively, while for the training cohort median OS was 32.4, 22.1, and 9.1 months, respectively. The Kaplan-Meier overall survival estimates at 5 years were similar for the validation and training cohorts in their entirety (23% vs. 22%), with the most marked difference found in subgroup 1 (50% vs. 66%) (See Figure 2). Notably, the median follow-up time among living patients for the validation cohort was shorter (70 months; range 37-133 months) than that for the training set (47 months; range 2-116 months). For both the training and validation cohort, PRISM subgroups exhibited statistically different survivals (both P<0.001).

Figure 2.

[Kaplan-Meier survival estimates after stereotactic spine radiation (SSRS) completion stratified by the 4 subgroups distinguished by the Prognostic Index for Spine Metastasis (PRISM) model in both the training (A) and validation (B) cohorts. Despite heterogeneity of patient characteristics and treatment PRISM stratifies each subgroup remarkably well.]

Model discrimination and calibration

To assess discrimination within the validation cohort, C-indexes were calculated for both stratified (0.68) and unstratified (0.70) score. These results were similar to findings for the original training dataset (0.69 and 0.70). To assess calibration, the 12-month and 24-month survival rates for the training and validation cohorts were plotted against each other (See Figure 3). The intercept of the fitted regression line for 24-month survival was (24-month validation) = 108 (24-month training) 16% and that for 12-month survival was (12-month validation) = 125 (12-month training) 31%. There was a survival offset such that survival rates were higher for the training set, but calibration was fair overall.

Figure 3.

[Calibration assessment plotting the 12-month and 24-month survival rates for the training and validation cohorts for the Prognostic Index for Spine Metastasis (PRISM) model against each other.]

DISCUSSION

We sought to validate the findings of a previous study involving the use of the PRISM model to stratify patients undergoing SSRS for spinal metastases according to survival. By using our validation set of patients treated off-trial, we reproduced the survival subgroups created for patients who had been treated on prior clinical trials. The model’s discrimination was excellent, with nearly identical C-indexes of both cohorts and log-rank P<0.001. Calibration was fair and Kaplan-Meier survival curves were nearly identical in shape.

The need for personalization of cancer treatment is becoming increasingly recognized. Optimal personalization of treatment depends in large measure on a clear understanding of a patient’s prognosis, an understanding that is made easier by the creation of prognostic score systems. The PRISM model was created with the goal of identifying, from baseline characteristics, which groups of patients to be treated with SSRS would have poor versus excellent survival. Aggressive and costly forms of treatment such as SSRS may be more burdensome and less beneficial for patients with a poor prognosis. On the other hand, another subset of patients, those with limited metastatic disease and good functional status, often demonstrate excellent survival. With continued maturation and a growing ease of delivering SSRS, practice patterns will continue to shift towards employing it more frequently. The PRISM can identify the group least likely to benefit from SSRS and thus define when its use is not justified.[27–29] Though the context is different, defining characteristics of patients well-suited for SRS of brain metastases rather than whole-brain irradiation is something that similarly continues to be broadened and defined.[30–32] Other approaches considered aggressive in the past, such as curative-intent resection of liver metastases from colon cancer, are now standards of care for carefully chosen patients.[33–36]

The PRISM model identifies a subset of patients with spinal metastases with poor survival who likely will not benefit from more aggressive treatments such as SSRS. Such a model can be useful in guiding treatment decisions. In future clinical trials of SSRS, having the PRISM tool will be useful in terms of stratifying or selecting patients. To date only one other such model has been presented to assess survival after SSRS, a model in which recursive partitioning analysis was used to identify three factors¾time to metastasis, KPS, and age¾to predict survival.[21] In contrast, PRISM takes into account two variables that have not been emphasized in previous models of spinal metastasis survival: previous surgery and radiation at the SSRS treatment site.[16–18,37] The inclusion of these variables in the PRISM model likely reflects the heavily treated nature of these study populations which correlates with increased survival in stage IV patients.

It was surprising that histology was not found to be significantly correlated with OS. Particularly aggressive histologies likely manifest in the model through the inclusion of pretreatment variables such as time to spinal metastasis and number of involved organ systems. In an era of cancer with driver mutations and increased availability of targeted drugs, a proportion of patients with certain mutations may have an increased expected survival. This has been demonstrated in NSCLC patients with ALK translocation of EGFR with brain metastases. [38]

Although all the patients in both the training and validation sets had been treated at the same institution, they were nevertheless rather different, as the training cohort was treated on trials and the validation cohort was treated off-trial. The training cohort was also predominantly male and had significantly higher rates of prior surgery. In addition, patients on clinical trials generally have better performance status and lack the comorbidities used as exclusion criteria for enrollment.[39–41] All of these comorbidities could not possibly be included in a practical model and often are not reflected in the assignment of KPS. This was evident in our analysis by the longer median survival time within the training cohort (25.5 months vs. 17 months) and across all survival subgroups in which median overall survival was reached. The calibration plot in further illustrates the superior survival of the patients in the training set when matched by PRISM subgroup. Importantly, Group 1, the poor survival group, displayed the smallest discrepancy of overall survival (9.1 v 6.5 months) proportionally and absolutely. Another group that treated patients off-trial found an even shorter median survival time of 10.7 months in their entire study population.[21] In either case, predicted survival of Group 1 falls below the proposed threshold of 11 months.[15]

This study has its limitations, primarily related to the retrospective nature of both PRISM creation and validation. Specifically, retrospectively assigning a KPS when it was not clearly notated in the medical chart has limitations and can lead to errors or selection bias. Second, all the data was from a single institution, which may limit broader application. Third, the calibration results seem to be suboptimal, perhaps because of a bias in trial enrollment (evidenced by the longer survival in the training cohort). However, a notable strength of this study is the heterogeneous nature of the compared patient cohorts. Despite this, the survival curves for all PRISM subgroups in each cohort matched well and the separation between subgroups was nearly identical. Finally, although survival after treatment of spine metastasis can be predicted with several models,[16–18,21,37] PRISM is SSRS specific and now has a validation cohort. To further demonstrate the generalizability of this model, external validation is a necessary next step.

We conclude that the scores derived from the PRISM model, a previously designed system for estimating survival after SSRS for patients being treated on protocols, has now been validated and reproduced in another cohort of patients treated off protocol. Despite heterogeneity of treatment, gender, and performance status, reproduction of survival curves in the validation cohort was strong for each of the four subgroups comprising the PRISM model. This validated prognostic score can help in choosing appropriate treatment approaches for individual patients with spinal metastases and can also help to stratify patients for future clinical trials. Specifically, once externally validated, this tool may be used to select out patients with poor prognosis as unsuitable for SSRS.