Abstract
Background
We tested the prognostic value of cumulative intracranial tumor volume (CITV) in the context of ds-GPA model for renal cell carcinoma (RCC) patients with brain metastasis (BM) treated with stereotactic radiosurgery (SRS).
Methods
Patient and tumor characteristics were collected from RCC cohorts with newly BM who underwent SRS. Univariable and multivariable logistic regression model was used to test the prognostic value of CITV, Karnofsky Performance Score (KPS), and the number of BM. Net reclassification improvement (NRI) and integrated discrimination improvement (IDI) were used to assess whether CITV improved the prognostic utility of RCC ds-GPA.
Results
In univariable logistic regression models, CITV, KPS, and the number of BM independently associated with RCC patient survival. In a multivariable Cox proportional hazard model, the association between CITV and survival remained robust after controlling for KPS and the number of BM (P=.042). The incorporation of the cumulative intracranial tumor volume (CITV) into the RCC ds-GPA model (consisting of KPS and number of BM) improved prognostic accuracy with NRI>0 of 0.3156 (95% CI: 0.0883–0.5428, P=.0065) and integrated discrimination improvement (IDI) of 0.0151 (95% CI: 0.0036–0.0277, P=.0183). These findings were validated in an independent cohort of 107 SRS-treated RCC BM patients.
Conclusion
CITV is an important prognostic variable in SRS-treated RCC patients with BM. The prognostic value of the ds-GPA scale for RCC brain metastasis was enhanced by the incorporation of CITV.
INTRODUCTION
Renal cell carcinoma (RCC) is the most common form of kidney cancer. It is a cancer with a known predisposition for metastasis to the brain.[1] Brain metastasis (BM) derived from RCC tends to be relatively refractory to whole brain radiation therapy.[2] In contrast, satisfactory control of BM has been reported following stereotactic radiosurgery (SRS).[3] Despite advances in molecular and clinical paradigms over the past two decades[4], the overall prognosis of SRS-treated RCC patients remains poor.[5] Nearly all patients succumb to the disease within 2 years of BM diagnosis.[6] As such, optimal treatment of RCC patients requires thoughtful consideration of treatment morbidity in the context of survival expectation.
The disease-specific Graded Prognostic Assessment scale (ds-GPA) for RCC was developed to meet the unique clinical challenges BM of different pathologies presented.[7] The scale suggests that survival prognostication for RCC patients with BM is reliant on two clinical variables: Karnofsky’s Performance Score (KPS) and the number of BM.[7] Our previous work suggests that the Cumulative Intracranial Tumor Volume (CITV) of BM, defined as the sum volume of all intracranial metastases at the time of presentation, served as a strong prognostic factor for lung cancer patients afflicted with BM [8–10]. However, since the fundamental premise of dsGPA is that prognostic factors critical for each cancer type differ, it remains unclear whether CITV serve as an important prognostic factor in the context of RCC dsGPA. Here, we tested this hypothesis.
METHODS
Study cohorts
Data collection by retrospective review was approved by each respective institution’s Institutional Review Board (IRB). The initial study cohort consisted of 360 newly diagnosed RCC BM patients treated by CC (University of California, San Diego), TS (Tsukiji Neurological Clinic) and MH (Katsuta Hospital Mito GammaHouse). The validation cohort consisted of 107 newly diagnosed RCC BM patients treated by VC (Yale University Medical Center). All patients were identified as having primary renal cell carcinoma with metastatic disease to the brain. Patients who underwent craniotomy as treatment of RCC BM was excluded from this series. All patients in this study underwent SRS as the primary treatment for their BM. Patients who underwent multiple SRS were included as a single entry in the database, with clinical data collected at the time of initial SRS treatment. The patient selection criteria applied at each of the treatment sites for SRS are comparable. Treatment were tailored specifically to each patient after review in a multi-disciplinary brain tumor board, the membership of which included at least one neurosurgeon, neuro-radiologist, radiation oncologist, and neuro-oncologist. In general, whole-brain radiotherapy (WBRT) was reserved for patients with miliary metastatic disease, patients who could not reliably follow-up with multiple MRI scans, or patients who were in the context of palliative care. Characteristics for our RCC cohort can be found in Table 1a/1b.
Table 1a.
Comparison of survival between constituent cohorts of the “discovery” RCC cohorta
| TS | CC/MY | p-value | |
|---|---|---|---|
| Survival.in.Months - median [range] | 6.20 [0.00, 144.50] | 5.20 [0.00, 111.60] | 0.179 |
| Patients with < 1 year survival - n (%) | 132 (69.8) | 120 (70.2) | 1 |
p-value of between groups comparison
Table 1b.
Baseline characteristics for patients within the RCC “discovery” cohorta
| RCC Cohort | |
|---|---|
| Number of patients | 360 |
| Socio-demographics | |
| Age - median [range] | 63.00 [24–90] |
| Sex - (% male) | 62.3% |
| Patient properties | |
| KPS - median [range] | 80 [40–100] |
| Number of metastases - median [IQR] | 2.00 [1–4] |
| CITV (cc) - median [IQR] | 5.09 [2.02–10.00] |
| Systemic disease status - N (% with active extracranial disease) | 321 (88.5%) |
| Survival | |
| Survival in Months - median [range] | 6.00 [0.00–144.50] |
| Patients with minimum one year survival - N (%) | 107 (29.7%) |
cc = cubic centimeters; CITV = Cumulative Intracranial Tumor Volume; Gy = Grey; KPS = Karnofsky Performance Score; RCC = renal cell carcinoma; IQR = interquartile range
Radiosurgery Technique
In all four centers, initial imaging was performed using 1mm axial and coronal T1-weighted pre- and post-contrast MR sequences. Treatment plan was formulated by a team consisting of neurosurgeons, radiation oncologists, and medical physicists. Elekta's Gamma Plan software was used for dosimetric planning. SRS dosimetric parameters were generally consistent with the published literature.[9, 11] The 50% isodose line was prescribed for each patient. Dose to the optic nerve was limited to 10 Gy; while dose to the brainstem was limited to 18 Gy. The mean dose received by the whole brain during any single SRS session was limited to <3 Gy.
Data collection and statistical analysis
Patient data was collected by electronic medical record (EMR). Patients were seen in 3–6 month intervals for follow-up. Data collected included patient age, Karnofsky performance status (KPS), number of brain metastases, CITV, and survival time (time from the initial SRS treatment to time of death). Overall survival (OS) calculation was determined from the time of SRS. CITV was defined as the cumulative volume of all treated lesions.[8, 9, 11] We used Elekta's Gamma Plan software on a pre-SRS T1-weighted post-contrast image to calculate this sum tumor volume. Patients who underwent multiple SRS were included as a single entry in the database, with age, KPS, and CITV collected at this time point. Overall survival was calculated as the interval between the time of initial SRS treatment to time of death. Based on principle component analysis, survival pattern was comparable between the three sites.
Pearson correlation analysis between CITV, KPS, and the number of BM was performed using established methods.[12–14] Analysis was also performed to determine whether these variables correlated with overall survival. Univariable proportional hazard analysis was performed to determine the risk of death associated with CITV, KPS, and the number of BM. A multivariable Cox proportional hazard analysis was performed to determine whether CITV remain a prognostic variable after controlling for KPS and the number of BM. The prognostic value of incorporating CITV into the published RCC ds-GPA model [7] was calculated using Net Reclassification Index (NRI>0) and Integrated Discrimination Improvement (IDI). The CITV incorporated model is termed RCC ds-GPA-CITV. NRI>0 and IDI are two established and generally accepted measures for quantifying classification improvement.[15–17] The NRI measures the proportion of subjects who will develop an event who are assigned a higher probability and the portion of subjects who will not develop an event who are assigned a lower probability with or without the addition of the new variable (in this case CITV).[15] IDI is based on the concept of discrimination slope, defined as the mean difference between model-based probabilities for events and nonevents. The IDI is defined as the difference in discrimination slopes between models with and without the variable of interest (i.e. CITV). [10, 18]
Statistical analyses were performed using R version 3.2.3 and the predictABEL package for NRI, and IDI calculations. The R plotROC package was used to generate ROC curves. Level of significance for all tests was set at P<.05; all tests performed were two-sided.
RESULTS
Demographics and disease characteristics
The demographics of the CC/MY and TS cohorts have been previously published. [10] The overall survival patterns of these RCC patient cohorts were comparable in terms of median and 12-month survival (Table 1a). As such, these two cohorts were combined into a single cohort of 360 patients, with the goal of maximizing the statistical power of this “discovery” cohort. We set out with the hope to not only examine the “discovery” cohort but then later validate the findings derived in an independent patient population. The demographics of this “discovery” cohort of 360 patients are shown in Table 1b. The mean age of this combined cohort was 62.5 years (SD = 12.0), and 62.3% of patients were male. Mean KPS was 81.3 (SD = 14.4), and mean overall survival was 1.1 years (SD = 1.7 years). Approximately 30% of patients had greater than 12-month survival. The mean number of brain metastasis was 3.8 (SD = 5.2), with a mean CITV of 7.5 cc’s (SD = 8.5cc). The demographic and clinical characteristics of this cohort is generally comparable to those reported in previous studies of SRS-treated RCC BM patients [7, 19]
Correlation between overall survival, CITV, KPS and number of BM
We first examine the extent to which CITV, KPS, and the number of BM were correlated using Pearson correlation analysis. As shown in Figure 1, increased CITV was correlated with decreased KPS. This inverse correlation between CITV and KPS was statistically significant (Table 2a, R2= −0.15 P=.005). Figure 2 shows that increased CITV was associated with increasing number of BM. This association was also statistically significant (Table 2a, R2= 0.13; P=0.016,). Interestingly, we did not observe a significant association between the number of BM and KPS.
Figure 1.
Cumulative intracranial volume (CITV) vs. KPS
Table 2a.
Pearson correlation analysis between KPS, metastasis number, 12-month survival and CITVa
| CITV | KPS | Number of lesions | |
|---|---|---|---|
| CITV | 1 | ||
| KPS | −0.15 | 1 | |
| Number of lesions | 0.13 | NS | 1 |
cc = cubic centimeters; CITV = Cumulative Intracranial Tumor Volume; KPS = Karnofsky Performance Score. All values significant with p <.05, unless labelled not significant (NS).
Figure 2.
Cumulative intracranial volume (CITV) vs. number of lesions
Next, we tested whether CITV, KPS, and the number of metastases significantly correlated with 12-month survival. Logistic regression revealed significant association between all three variables and overall survival. As shown in Table 2b, an increase in CITV or the number of metastasis was associated with worsened overall survival (P = .024 and P=.004, respectively). Increased KPS was associated with improved overall survival (P<.001).
Table 2b.
Univariable generalized linear models quantifying correlation between CITV, KPS, number of metastases and 12-month survival
| Cumulative Intracranial Tumor Volume (CITV) | ||
|---|---|---|
| Estimate Coefficient (β) |
p-value | |
| CITV | 0.0064 | P=.024 |
| KPS | −0.0098 | P<.001 |
| Number of Metastases | 0.0130 | P=.004 |
Hazard of death associated with CITV, KPS, and number of BM
Next, we performed univariable hazard analysis to determine the risk of death associated with CITV, KPS, and the number of BM (Table 3). We previously examined the optimal CITV cut-off for survival prognostication in SRS-treated lung cancer patients and found 4 cc’s to be the optimal cut-off [8]. We wished to determine whether this CITV cut-off is applicable to RCC patients. Univariable logistic regression analysis revealed that the odds ratio (O.R.) for death within a year for SRS-treated RCC BM patients with CITV < 4 cc’s were approximately 0.31 relative to those with CITV ≥ 4 cc’s (P=0.007). In other words, SRS-treated RCC BM patients with CITV < 4 cc’s were a third less likely to die within a year relative to patients with CITV ≥ 4 cc’s, suggesting the utility of this cut-off in survival prognostication.
Table 3.
Univariable (left – Model 1) and multivariable (right – Model 3) logistic regression analyses demonstrating prognostic significance of CITV point groupings for one-year survival in comparison to ds-GPA (middle – Model 2)
| MODEL 1 – CITV alone | MODEL 2 – ds-GPA | MODEL 3 – ds-GPA-CITV | ||||
|---|---|---|---|---|---|---|
| Hazard Ratio | Statistical Significance | Hazard Ratio | Statistical Significance | Hazard Ratio | Statistical Significance | |
| CITV (per point grouping increase) | 0.314 | P=.007 | – | – | 0.256 | P=.042 |
| KPS (per point grouping increase) | – | – | 1.659 | P<.001 | 1.620 | P<.001 |
| Number of metastases (per point grouping increase) | – | – | 0.456 | P=.002 | 0.451 | P=.003 |
| Akaike Information Criteria | 382 | 380 | ||||
CITV = Cumulative Intracranial Tumor Volume; ds-GPA = disease-specific graded prognostic assessment; KPS = Karnofsky Performance Score.
For analysis pertaining to KPS and the number of BM, patients were grouped based on specification of RCC ds-GPA. For KPS, each incremental increase in category (<70, 70–80, 90–100) was associated with a 65% increase the likelihood of one-year survival (O.R.=1.65, P <0.001). For the number of metastases, each incremental increase in category (1, 2–3, and >3) was associated a 54% decrease in the likelihood of one-year survival (O.R.=0.46, P =0.002).
We next performed a multivariable Cox proportional hazard analysis to determine whether CITV remain a prognostic factor after controlling for KPS and the number of BM. In this multivariable model, CITV remain an important factor in survival prognostication (OR = 0.256, P=.042). This result suggests that incorporation of CITV improves the prognostic utility of RCC ds-GPA.
NRI and IDI analysis
The previously published RCC ds-GPA scale and modified ds-GPA with incorporation of CITV (termed RCC ds-GPA-CITV) are shown in Table 4a. We used the NRI and IDI to quantitate the extent that CITV incorporation into the RCC ds-GPA model improve survival prognostication. These tests of discrimination have been previously published [15] and are described in more detail in Methods. The incorporation of CITV into the ds-GPA resulted in an NRI of 0.3156 (95% CI 0.0883–0.5428, P=.0065) and IDI of 0.0151 (95% CI 0.0036–0.0277, P=.0183), as shown in Table 4b. These statistics indicate that CITV improved the prognostic values of the ds-GPA model. Kaplan-Meier survival analysis of CITV-incorporated RCC ds-GPA is shown with stepwise point increments in Figure 3.
Table 4a.
Scoring components for SIR and ds-GPA-CITV models
| ds-GPA model | ds-GPA-CITV model | |||||
|---|---|---|---|---|---|---|
|
| ||||||
| 0 | 1 | 2 | 0 | 1 | 2 | |
|
| ||||||
| KPS | < 70 | 70–80 | 90–100 | < 70 | 70–80 | 90–100 |
| Number of metastases | > 3 | 2–3 | 1 | > 3 | 2–3 | 1 |
| CITV (cc) | – | – | – | ≥ 4 | – | < 4 |
cc = cubic centimeters; CITV = Cumulative Intracranial Tumor Volume; ds-GPA = disease-specific graded prognostic assessment; KPS = Karnofsky Performance Score.
Table 4b.
Prognostic performance of ds-GPA-CITV model as compared to ds-GPA alone in the RCC cohorta
| Value | 95% CI | |
|---|---|---|
| Continuous NRI>0 | 0.3156 | 0.0883–0.5428 (P=.0065) |
| IDI score | 0.0151 | 0.0036–0.0277 (P=.0183) |
ΔAUC = area under curve; IDI = integrated discrimination improvement; NRI = net reclassification improvement
Figure 3.
Kaplan Meier Survival Analysis of CITV-incorporated RCC ds-GPA (note: our primary endpoint of interest, 12-months, is indicated by a vertical line)
Validation of the prognostic value of CITV in the context of RCC ds-GPA
We wished to validate our findings using an independent patient cohort of 107 patients. The demographics of this patient population is shown in Table 5. We next used the NRI and IDI to quantitate the extent that CITV incorporation into the RCC ds-GPA model improve survival prognostication in this validation cohort. The incorporation of CITV into the ds-GPA resulted in an NRI of 1.162 (95% CI 0.841–1.48, P<.001) and IDI of 0.146 (95% CI 0.071–0.222, P<.001), as shown in Table 6. These statistics validated our observation that CITV improved the prognostic values of the ds-GPA model.
Table 5.
Baseline characteristics for patients within the validation cohorta
| Validation Cohort | |
|---|---|
|
| |
| Number of patients | 107 |
| Patient properties | |
| KPS median - [range] | 80 [50, 100] |
| Number of metastases - mean (SD) | 3.53 (3.6) |
| CITV (cc) - mean (SD) | 5.04 (5.27) |
cc = cubic centimeters; CITV = Cumulative Intracranial Tumor Volume; Gy = Grey; KPS = Karnofsky Performance Score; RCC = renal cell carcinoma; SD = standard deviation
Table 6.
Prognostic performance of ds-GPA-CITV model as compared to ds-GPA alone in the validation cohorta
| Value | 95% CI | |
|---|---|---|
| Continuous NRI>0 | 1.161 | 0.8412 – 1.482 (P<.001) |
| IDI score | 0.1462 | 0.07100 – 0.2215 (P<.001) |
ΔAUC = area under curve; IDI = integrated discrimination improvement; NRI = net reclassification improvement
DISCUSSION
Survival prognostication for RCC patients suffering from BM significantly impacts decisions in clinical management. The ds-GPA for RCC (consisting of KPS and the number of BM) has been useful in this regard. However, there is emerging literature suggesting that CITV may improve the prognostic ds-GPA for SRS-treated lung cancer patients suffering from BM. Here, we tested whether the prognostic utility of CITV apply to RCC patients. The question tested is important since the fundamental concept of ds-GPA is that prognostic factors critical for one cancer type may not necessarily apply to another cancer type. Our results demonstrated the prognostic utility of CITV in RCC BM patients. We showed that in two independent cohorts, the prognostic value of RCC ds-GPA can be improved by incorporating CITV.
Importantly, tumor volume in patients with newly diagnosed BM (including those arising from RCC) are generally well-defined. Individual tumor volumes in SRS candidates are typically calculated in an automated manner during dosimetric planning. Thus, the determination of CITV requires only summation of all tumor volumes. While many radiosurgery software suites do not provide automated summation of all tumor volume, this function can be provided through minor modification of the software. As such, incorporation of CITV into RCC ds-GPA can be instituted in the clinical setting without straining the clinical work-flow.
The correlation between number of BM and CITV is in RCC BM patients was informative in the following context. If each BM is reasonably homogenous in terms of tumor volume, then one would anticipate a significant correlation between number of BM and CITV. In contrast, if there is significant variability in tumor volume between different BM, then the correlation between these variables would be expected to be poor, as we reported to be the case in lung cancer.[8] The correlation between the number of BM and CITV suggests that RCC BM tend to be more homogenous in tumor volume relative to lung BM.
By far, the most important prognostic factor for survival in RCC patients with SRS-treated BM was KPS (Table 3), suggesting the primacy of overall clinical assessment in survival prognostication over individual clinical variables. The number of BM and CITV independently contributed to survival prognostication and are comparable in this regard. These results suggest that number of BM and CITV convey distinct information sets pertinent to clinical survival. Whether these information sets relate to distinct biology of patient tumor and differential clinical behavior (e.g. tumors with high CITV BM are intrinsically more aggressive clinically) or SRS dosimetric consideration (e.g. high CITV BM are typically treated with lower dose SRS) remains active areas of investigation.
While our study results are robustly reproduced in independent cohorts, the study is, nevertheless, a retrospective analysis of patients who were selected to undergo SRS. As such, validation through a prospectively designed study will be needed. Further, our results may not be applicable to a set of patients who were selected for SRS using a different set of criteria. Nevertheless, our study recapitulated results derived from patient cohorts enrolled in Radiation Therapy Oncology Group (RTOG) studies as well as cohorts of patients treated in Europe.[11] This consistency suggests that the selection criteria for SRS treatment in this report are generally similar to those utilized in other studies. Finally, with the advent of targeted and immune-therapy will undoubtedly impact the survival of RCC patients afflicted with BM. Meaningful integration of pertinent molecular and immunologic biomarkers with clinical variables presented here will be required in future studies. The data described here provide a foundation for these future studies.
CONCLUSION
Our results suggest that CITV is a key prognostic factor for RCC patients afflicted with BM. Prognostic utility of RCC ds-GPA is improved by the incorporation of CITV.
Highlights.
-
➢
Survival analysis of 360 patients with RCC brain metastases from the US and Japan
-
➢
Goal: Determine whether ds-GPA for RCC is augmented with CITV
-
➢
The addition of CITV to the ds-GPA model for RCC enhances its prognostic utility.
Abbreviation list
- ds-GPA
disease specific Graded Prognostic Assessment
- CITV
cumulative intracranial volume
- RCC
renal cell carcinoma
- BM
brain metastasis
- KPS
Karnofsky Performance Score
- NRI
Net reclassification improvement
- IDI
integrated discrimination improvement
- WBRT
whole-brain radiotherapy
- RTOG
Radiation Therapy Oncology Group
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
References
- 1.Gupta K, Miller JD, Li JZ, Russell MW, Charbonneau C. Epidemiologic and socioeconomic burden of metastatic renal cell carcinoma (mRCC): a literature review. Cancer Treat Rev. 2008;34:193–205. doi: 10.1016/j.ctrv.2007.12.001. [DOI] [PubMed] [Google Scholar]
- 2.Kim YH, Kim JW, Chung HT, Paek SH, Kim DG, Jung HW. Brain metastasis from renal cell carcinoma. Prog Neurol Surg. 2012;25:163–175. doi: 10.1159/000331190. [DOI] [PubMed] [Google Scholar]
- 3.Kim WH, Kim DG, Han JH, Paek SH, Chung HT, Park CK, Kim CY, Kim YH, Kim JW, Jung HW. Early significant tumor volume reduction after radiosurgery in brain metastases from renal cell carcinoma results in long-term survival. Int J Radiat Oncol Biol Phys. 2012;82:1749–1755. doi: 10.1016/j.ijrobp.2011.03.044. [DOI] [PubMed] [Google Scholar]
- 4.Graves A, Hessamodini H, Wong G, Lim WH. Metastatic renal cell carcinoma: update on epidemiology, genetics, and therapeutic modalities. Immunotargets Ther. 2013;2:73–90. doi: 10.2147/ITT.S31426. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Mori Y, Kondziolka D, Flickinger JC, Logan T, Lunsford LD. Stereotactic radiosurgery for brain metastasis from renal cell carcinoma. Cancer. 1998;83:344–353. doi: 10.1002/(sici)1097-0142(19980715)83:2<344::aid-cncr19>3.0.co;2-t. [DOI] [PubMed] [Google Scholar]
- 6.Sheehan JP, Sun MH, Kondziolka D, Flickinger J, Lunsford LD. Radiosurgery in patients with renal cell carcinoma metastasis to the brain: long-term outcomes and prognostic factors influencing survival and local tumor control. J Neurosurg. 2003;98:342–349. doi: 10.3171/jns.2003.98.2.0342. [DOI] [PubMed] [Google Scholar]
- 7.Sperduto PW, Chao ST, Sneed PK, Luo X, Suh J, Roberge D, Bhatt A, Jensen AW, Brown PD, Shih H, Kirkpatrick J, Schwer A, Gaspar LE, Fiveash JB, Chiang V, Knisely J, Sperduto CM, Mehta M. Diagnosis-specific prognostic factors, indexes, and treatment outcomes for patients with newly diagnosed brain metastases: a multi-institutional analysis of 4,259 patients. International journal of radiation oncology, biology, physics. 2010;77:655–661. doi: 10.1016/j.ijrobp.2009.08.025. [DOI] [PubMed] [Google Scholar]
- 8.Marcus LP, Marshall D, Hirshman BR, McCutcheon BA, Gonda DD, Koiso T, Hattangadi-Gluth JA, Carter BS, Yamamoto M, Chen CC. Cumulative Intracranial Tumor Volume (CITV) Enhances the Prognostic Value of the Lung-Specific Graded Prognostic Assessment (GPA) Model. Neurosurgery. 2016;79:246–252. doi: 10.1227/NEU.0000000000001123. [DOI] [PubMed] [Google Scholar]
- 9.Gonda DD, Kim TE, Goetsch SJ, Kawabe T, Watanabe S, Alksne JF, Hattangadi JA, Nitta M, Ott K, Hodgens DW, Carter BS, Yamamoto M, Chen CC. Prognostic factors for stereotactic radiosurgery-treated patients with cerebral metastasis: implications on randomised control trial design and inter-institutional collaboration. Eur J Cancer. 2014;50:1148–1158. doi: 10.1016/j.ejca.2014.01.001. [DOI] [PubMed] [Google Scholar]
- 10.Hirshman BR, Wilson B, Ali MA, Proudfoot JA, Koiso T, Nagano O, Carter BS, Serizawa T, Yamamoto M, Chen CC. Superior prognostic value of cumulative intracranial tumor volume relative to largest intracranial tumor volume for stereotactic radiosurgery treated brain metastasis patients. Neurosurgery. 2017 doi: 10.1093/neuros/nyx225. In Press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Nieder C, Andratschke NH, Geinitz H, Grosu AL. Diagnosis-specific graded prognostic assessment score is valid in patients with brain metastases treated in routine clinical practice in two European countries. Medical Science Monitor. 2012;18:CR450–CR455. doi: 10.12659/msm.883213. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Pearson K. Notes on regression and inheritance in the case of two parents. Proceedings of the Royal Society of London. 1895:240–242. [Google Scholar]
- 13.Israel Cohen YH, Jingdong Chen, Jacob Benesty. Noise Reduction in Speech Processing 2009 [Google Scholar]
- 14.Lin LIK. A Concordance Correlation Coefficient to Evaluate Reproducibility. Biometrics. 1989;45:255. doi: 10.2307/2532051. [DOI] [PubMed] [Google Scholar]
- 15.Pencina MJ, D’Agostino RB, Sr, D’Agostino RB, Jr, Vasan RS. Evaluating the added predictive ability of a new marker: from area under the ROC curve to reclassification and beyond. Stat Med. 2008;27:157–172. doi: 10.1002/sim.2929. discussion 207-112. [DOI] [PubMed] [Google Scholar]
- 16.Pencina MJ, D’Agostino RB, Pencina KM, Janssens AC, Greenland P. Interpreting incremental value of markers added to risk prediction models. Am J Epidemiol. 2012;176:473–481. doi: 10.1093/aje/kws207. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Pickering JW, Endre ZH. New metrics for assessing diagnostic potential of candidate biomarkers. Clin J Am Soc Nephrol. 2012;7:1355–1364. doi: 10.2215/CJN.09590911. [DOI] [PubMed] [Google Scholar]
- 18.Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Pencina MJ, Kattan MW. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology. 2010;21:128–138. doi: 10.1097/EDE.0b013e3181c30fb2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Ferrel EA, Roehrig AT, Kaya EA, Carlson JD, Ling BC, Wagner A, MacKay AR, Call JA, Demakas JJ, Lamoreaux WT, Fairbanks RK, Cooke BS, Peressini B, Lee CM. Retrospective Study of Metastatic Melanoma and Renal Cell Carcinoma to the Brain with Multivariate Analysis of Prognostic Pre-Treatment Clinical Factors. Int J Mol Sci. 2016;17:400. doi: 10.3390/ijms17030400. [DOI] [PMC free article] [PubMed] [Google Scholar]