A nomogram to predict development of brain metastasis in non-small cell lung cancer patients: a retrospective analysis using routinely available medical records

Summary

Background

Brain metastases (BrM) are a frequent complication among patients with non-small cell lung cancer (NSCLC). While guidelines exist for baseline CNS screening in advanced NSCLC, surveillance strategies for early-stage disease remain limited. This study aimed to develop a time-dependent BrM risk prediction nomogram using readily available clinical information.

Methods

We analyzed a retrospective cohort of NSCLC patients at Penn State Health. Our objectives were to (1) systematically evaluate the performance of existing BrM risk prediction algorithms and (2) construct novel nomograms for BrM risk prediction in NSCLC. Using Cox-proportional hazard models with L1-regularization, we predicted BrM risk at 6-month, 1-year, and 2-year follow-up intervals.

Findings

The patient cohort included 1904 patients (median age 68 years, range 38–94 years, BrM incidence 22.8%). The cohort included 1059 males (55.6%) and 845 females (44.4%). Of the cohort, 92.8% of patients identified as White (n = 1766), 1.0% as Asian (n = 19), 4.0% as Black (n = 77), and 2.2% as another race (n = 42). The Zhang 2021 model demonstrated the highest performance in predicting BrM incidence in our cohort, achieving an AUROC of 0.91 (95% CI: 0.87, 0.95). Two novel models were developed: a baseline model incorporating clinical and imaging data at diagnosis (cTNM stage, age at diagnosis), and an extended model including additional clinical and treatment data (number of extracranial metastatic sites, prior radiotherapy, chemotherapy, surgery, and histology) (https://nmikolajewicz.shinyapps.io/nomogram_wilding2024/). While both models showed similar short-term performance, the extended model demonstrated superior predictive capacity (AUROC 0.91 at 3-years) for longer-term outcomes. Our nomograms rely exclusively on clinical features routinely documented in patient records, thereby requiring no additional investigations.

Interpretation

These clinically accessible nomograms for BrM prediction will facilitate prognostic modeling, risk stratification, refinement of CNS screening guidelines, and patient counseling.

Funding

None.

Research in context.

Evidence before this study

MEDLINE via Pubmed, Cochrane, Web of Science, and EMBASE were searched in October 2021 for studies reporting statistical models to predict brain metastasis (BrM) incidence or risk of BrM in non-small cell lung cancer (NSCLC) (e.g., logistic regression, cox proportional hazards regression, etc.). We used the following search terms: (“brain metastasis” AND “nomogram”) OR (“predict” AND “brain metastasis”) OR (“risk” AND “brain metastasis” AND “lung cancer”). Our search was limited to prediction models for BrM among adult patients with NSCLC which were published before October 10, 2021. Review articles, commentaries, correspondences, and case reports were excluded. Our search identified 1643 unique publications, 14 of which met our inclusion criteria, and we then included a selection of author-curated conference abstracts that we felt would complement the existing paper. While our search demonstrated several algorithms predicting the risk of BrM development for patients with NSCLC, the widespread uptake of these tools in a clinical setting has been precluded by limitations related to the use of data not routinely available in patient charts, data lacking individual granularity or time-to-event data, and a lack of external validation across these algorithms.

Added value of this study

BrM are a frequent complication among patients with NSCLC. Surveillance strategies for early-stage disease are limited. Our study advances BrM prediction in NSCLC patients through three key contributions: i) validation of existing models, ii) development of a novel time-dependent predictive model, and iii) identification of clinically relevant risk factors. The validation of the Zhang 2021 model (AUROC 0.91, 95% CI: 0.87–0.95) represents a critical step towards clinical implementation. Our extended model builds on existing models to incorporate time-to-event data, while maintaining comparable accuracy, offering reliable long-term risk assessment.

Implications of all the available evidence

Our study validates the Zhang 2021 model, critically building upon the existing literature and advancing efforts to implement models for predicting BrM risk in NSCLC patients. Through using readily available clinical data, our model offers a practical, accessible, and accurate approach to BrM risk assessment. This accessibility, combined with reliable predictive performance, positions our model as a valuable tool for clinical decision-making. The model’s sustained accuracy overtime particularly supports its utility in long-term patient monitoring and treatment planning.

Introduction

Lung cancer is the most common cause of cancer-related death, with non-small cell lung cancer (NSCLC) comprising approximately 80% of these cases.1, 2, 3 Up to 55% of patients with locally advanced NSCLC develop brain metastases (BrM),^2,4, 5, 6 conferring a poor prognosis.^2,7,8 The National Comprehensive Cancer Network (NCCN) does not recommend central nervous system screening in patients presenting with Stage I-II disease who are asymptomatic.8, 9, 10 Yet, other factors such as histology and age at diagnosis influence the development of BrM,^2,4,8 and patients with lower stage disease may develop BrM without obvious neurological symptoms, missing an opportunity for early management.

Risk prediction tools can enable low-cost patient screening and trigger subsequent imaging studies for higher risk individuals. Several algorithms predicting the risk of BrM development for patients with NSCLC have been published to date.^8,11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24 However, the widespread uptake of these tools in a clinical setting has been precluded by limitations related to the use of i) small primary datasets comprised of data not routinely available in patient charts, or ii) large population datasets that lack individual granularity or time-to-event data.²⁵ Moreover, a lack of external validation is prevalent across these algorithms.

The objectives of our study were to i) review the literature on BrM-predictive algorithms, and ii) use our independent dataset, the largest primary patient cohort to date, to develop and validate a prognostic BrM predictive algorithm.

Methods

Software

Endnote 21 (Thomson Reuters) was used for reference management, R (version 4.2.2) for analysis, Excel (Microsoft) for data storage, and CorelDRAW X8 (Corel) for figure preparation.

Literature review

Search strategy

Medline was searched using a search strategy that included the following concepts: i) BrM prediction, ii) adult patients (age $\ge$ 18), and iii) NSCLC. Title/abstract screening was conducted by three independent reviewers (N.H., B.S., B.K.). Studies were included if statistical models were developed to predict BrM incidence or risk of BrM (e.g., logistic regression, cox proportional hazards regression, etc.). Review articles, commentaries, correspondences, and case reports were excluded. Peer-reviewed studies and conference abstracts were included; however recognizing the preliminary nature of conference proceedings, we indicated which risk models were peer-reviewed. No restrictions were imposed on study design (e.g., retrospective vs. prospective) or treatments taken by study participants. Only articles published in English were included. Eligibility was confirmed by full-text screening of articles for peer-reviewed studies. Screening conflicts were resolved by consensus as required. The list of abstracts identified in Medline was further supplemented with an author-curated selection of papers.

Data extraction

For each article, the following data were extracted and summarized: patient demographic data, statistical models/methods, and associated predictive variables. Model data are summarized in Table 1.

Table 1.

Literature review querying the Medline/PubMed database for literature predicting BM in adult patients with NSCLC. Only peer-reviewed studies are included.

Author	Year	PMID	Model application	Model fit	Study design	Validation method	Sample size	Association measures	Model performance
Arrieta11	2009	19386089	Prognostication	ANOVA, Mann–Whitney test, Fisher’s exact test, multivariable logistic regression	Prospective	NA	293	RR	NA
Arrieta12	2021	33558063	Risk stratification	ANOVA, Mann–Whitney test, Fisher’s exact test, multivariable Cox regression	Retrospective	NA	559	HR	NA
Bajard13	2004	15301872	Risk stratification	Chi-square test, multivariable logistic regression	Retrospective	NA	305	RR	NA
Chen14	2021	33832315	Risk stratification	Chi-square test, I2 for heterogeneity of studies, Odds ratio, weighted mean difference	Meta-Analysis	NA	11415	OR	NA
Dimitropoulos15	2011	21931502	Prognostication, Risk stratification	Mann–Whitney test, Kruskal–Wallis test, Student’s t-test, ANOVA, logistic regression, log rank tests	Retrospective	NA	161	OR	NA
Goncalves16	2016	27062154	Prognostication, Risk stratification	Chi-square test, Cox proportional hazard ratios, log-rank tests of Kaplan–Meier survival curves, logistic regression	Retrospective Cohort	NA	34681	OR, HR	NA
He17	2021	33655937	Risk stratification	independent sample t test, Chi-square test, multivariable logistic regression	Retrospective	Internal cross-validation	350	OR	AUROC
Hubbs18	2010	20629035	Prognostication, Risk stratification	Kaplan–Meier, univariate Cox proportional hazards model, multivariable Cox proportional hazards regression model	Retrospective	NA	975	HR	NA
Li19	2018	32922870	Risk stratification, Prognostication	Chi-squared test, Cox regression, Kaplan–Meier, Log-rank test	Retrospective	NA	373	RR	NA
Milano20	2020	32256710	Risk stratification	Chi-squared test, T-test, multivariable logistic regression	Retrospective	NA	49495	OR	NA
Mujoomdar21	2007	17229875	Risk stratification	Chi-squared test, Pearson correlation coefficients, bivariate logistic regression, hierarchical logistic regression	Retrospective	NA	264	Pearson correlation coefficients and regression coefficient	NA
Waqar22	2018	29526531	Risk stratification, prognostication	Univariate and multivariable logistic regression, Kaplan–Meier product limit, log-rank test	Retrospective Cohort	NA	457481	OR	NA
Zhang23	2016	27090794	Prediction, Risk stratification	Cox proportional hazards regression	Retrospective	Bootstrap resampling validation	637	HR	Concordance index
Zhang8	2021	33569207	Risk stratification	Pearson chi-square test, Fisher’s exact test, univariate Cox regression analysis, multivariable regression analysis	Retrospective Cohort Study	Internal cross-validation using bootstrap method	26154	OR	AUROC

ANOVA, analysis of variance; NA, not applicable within this study; RR, Risk ratio; HR, Hazard ratio; OR, Odds ratio; AUROC, Area Under the Receiver Operating Characteristic curve.

Institutional (retrospective) cohort

Data source

Single-institutional data from the Penn State Health electronic medical record (Table 2) were used to perform retrospective chart review of patients treated for NSCLC from 2011 to 2020 (01/01/2011–12/31/2020). Patients diagnosed with NSCLC after 2020 were excluded to allow for sufficient follow up time and assessment of BrM development. Institutional Review Board approval was provided by Penn State University under study number STUDY00019850.

Table 2.

Demographic, staging, disease, and treatment data for patients diagnosed with NSCLC in our single-institutional cohort.

Demographic & Survival	Median age at diagnosis in years (range)	Gender		Race				Survival
		Female	Male	White	Asian	Black	Other	Median follow up in days (IQR)	Median time to BrM in Days (IQR)
Total cohort (%)	68 (38–94)	845 (44.4)	1059 (55.6)	1766 (92.8)	19 (1.0)	77 (4.0)	42 (2.2)	550 (1325)	26 (270)

Staging	T-stage at diagnosis					N-stage at diagnosis					M-stage at diagnosis
	1	2	3	4	U	0	1	2	3	U	0	1	U
Total cohort (%)	794 (41.7)	153 (8.0)	335 (17.6)	554 (29.1)	68 (3.6)	913 (48.0)	157 (8.2)	439 (23.1)	270 (14.2)	125 (6.6)	1160 (60.9)	668 (35.1)	76 (4.0)

Disease characteristics	Histology				Primary NSCLC Treatment			Metastatic burden				Brain metastases
	Adenocarcinoma	Carcinoid	Carcinoma	Other	Surgery	Chemo	Radiation	0	1	2	3+	Y	N
Total cohort (%)	1007 (52.9)	115 (6.0)	681 (35.8)	101 (5.3)	572 (30.0)	719 (37.8)	884 (46.5)	854 (44.9)	694 (36.4)	228 (12.0)	128 (6.9)	434 (22.8)	1470 (77.2)

U, unknown.

Data curation

Adult patients (at least 18 years old) treated for NSCLC from 2011 to 2020 were identified by the Penn State Cancer Registry. For each patient, the following anonymized data were extracted from the electronic medical record and curated into a single dataset (https://github.com/hwild57/NSCLC/blob/main/Wilding%20Table%20S1%20copy.csv):

• 1.Sociodemographic: Date of birth, sex, race, and smoking history.
• 2.Disease-related: Age at NSCLC diagnosis, primary NSCLC tumor site, T, N, M, and overall stage at diagnosis, histologic grading, and treatment details (including surgery, radiation, and chemotherapy status). Information on molecular markers (e.g., EGFR status) was included when available.
• 3.Metastatic disease: Location of metastases, date diagnosed with metastatic disease, number and location of brain metastases, and date diagnosed with brain metastases. The variable “number of metastatic sites” reflects the number of extracranial metastatic sites.

Of note, M stage at diagnosis and number of metastatic sites represent distinct variables, each accounting for different aspects of the metastatic process. Staging was performed at diagnosis, hence M stage encompasses a patient’s metastatic status at diagnosis, while number of metastatic sites accounts for asynchronous development of metastases. This distinction, in addition to the pre- vs. post-operative status of variables, is summarized in Table 3.

Table 3.

Definitions of variables in chart review for patients diagnosed with NSCLC in our single-institutional cohort.

Variable	Definition	Timing of Variable	Included in model	Remarks
Age at NSCLC diagnosis	The age at which patient was first ever diagnosed with NSCLC	At diagnosis	Baseline & extended
Primary NSCLC tumor size	Measurement in centimeters (cm) of largest diameter of primary NSCLC tumor	At diagnosis	Baseline & extended
Overall stage	Clinical stage of patient’s NSCLC
T stage	Clinical T stage	At diagnosis	Baseline & extended
N stage	Clinical N stage	At diagnosis	Baseline & extended
M stage	Clinical M stage	At diagnosis	Baseline & extended
Histologic grading	Histologic grade of primary NSCLC tumor	Follow-up data	N/A	Classified as “unknown” when grade was not determined or data was missing
Histologic subtype	Classification of the histologic subtype of primary NSCLC	Follow-up data	Extended only	Placed into subgroups of adenocarcinoma, squamous cell carcinoma, large cell carcinoma, or other
Surgery	Y/N status of whether patient received surgery for resection of primary NSCLC tumor	Follow-up data	Extended only
Radiation	Y/N status of whether patient received radiation for treatment of primary NSCLC tumor	Follow-up data	Extended only
Chemotherapy	Y/N status of whether patient received chemotherapy for treatment of primary NSCLC tumor	Follow-up data	Extended only
EGFR	Mutational status of EGFR	Follow-up data	N/A
Metastatic Disease	Y/N status of whether patient was ever diagnosed with NSCLC metastatic disease (at time of NSCLC diagnosis or anytime thereafter)	Follow-up data	N/A
Number of extracranial metastatic sites	Number of metastatic sites of disease other than brain	Follow-up data	Extended only
Number of brain metastases	Number of NSCLC brain metastases (diagnosed at time of NSCLC diagnosis or anytime thereafter)	Follow-up data	N/A

Feature engineering

Raw medical record data (i.e., institutional data) were cleaned and converted into standardized variables. Metastatic locations were categorized by organ and counted. Unresolved or undetermined features were collapsed and coded as “other”. Tumor size estimates were converted into centimeters, and averaged across all dimensions (e.g., 3 cm × 4 cm × 5 cm = 4 cm).

Assessment of predictors of BrM risk

Descriptive statistics

Descriptive statistics were generated for all variables using the dfSummary function (summarytools v1.0.1 R package). Categorical variables were summarized as counts and percentages, and continuous variables were summarized as means, standard deviations (SD), ranges, medians, and interquartile ranges (IQR).

Logistic regression

Univariate and multivariable logistic regression models were fitted using the glm function in R with a binomial distribution and logit link function. The outcome variable was the presence or absence of BrM. Odds ratios (OR_uni; univariate odds ratios, OR_multi; multivariable odds ratios) and corresponding p-values (p_uni; univariate p-value, p_multi; multivariable p-value) were reported.

Evaluation of published BrM risk models

Model fitting

We specified and trained classification models according to the clinical features used in each original study (referred to as published models). Only models with clinical features available for >80% of our cohort patients were evaluated. For each model, we partitioned our retrospective cohort data into 70% training and 30% testing sets using stratified sampling [implemented with createDataPartition function (caret v6.0–93 R package)].

We used the training set to train classification models with glmnet (caret v 6.0–93 R package) with 10-fold cross validation with 10 repeats. The outcome variable was binary BrM incidence (true or false).

Model comparison

The performance of trained published models was evaluated on the held-out test set using evalm (MLeval v0.3 R package). Published models were then rank ordered by their respective area under the receiver operating characteristic curve (AUROC) measures to identify the top performing models.

Model validation

To validate the performance of existing predictive algorithms, we compared performance metrics obtained using our retrospective cohort with those reported in the original studies. Where available, we also applied published nomograms to our retrospective cohort data to externally validate their performance.

Time-to-event models to predict time-dependent BrM incidence

Model fitting

We performed time-to-event analyses using Cox proportional hazards models to identify clinical features associated with time-dependent risk of BrM development. Two main models were developed:

• 1.Baseline model using clinical features available at diagnosis.
• 2.Extended model incorporating treatment information and follow-up data.

Models were fit using Cox regression, and clinical features were selected using L1-regularization (fit_lasso function, hdnom v6.0.2 R package). The optimal regularization parameter (λ) was selected using k-fold cross-validation; the λ value that minimized the cross-validation error was used to parameterize the model. Model performance was then evaluated using time-dependent AUC (tAUC) derived using the Uno estimator.²⁶ Internal validation was performed using 50-fold cross-validation. The advantage of using an L1 regularized model is that it produces sparse models (few predictors) with reduced risks of overfitting. Non-regularized models are at risk of overfitting data, which inadvertently can lead to overestimating the weight of individual predictors and performance of models.

Nomogram development

Using the baseline and extended models, nomograms were developed to predict 6-month, 1-year, and 3-year BrM-free probability. This was performed using the as_nomogram function (hdnom v6.0.2 R package).

Statistics and data visualization

Unless otherwise specified, the ggplot2 R package (v3.3.5) was used for data visualization. Heatmaps were generated using pheatmap (pheatmap R package, version 1.0.12). Statistical models were fit and evaluated as detailed above. The performance of predictive models was assessed using AUROCs. Logistic regression models were reported using ORs. Significance was determined as ∗p < 0.05, ∗∗p < 0.01 or ∗∗∗p < 0.001. No sample size calculations were performed.

Role of funding source

No funding was received for this study.

Ethics

Institutional Review Board approval was provided by Penn State University under study number STUDY00019850. Informed consent was waived due to the retrospective nature of the study and the use of deidentified patient data.

Results

Overview of existing predictive models of BrM in NSCLC patients

The initial search of Medline identified 1643 unique publications. After screening, 1629 articles were excluded based on predetermined criteria: they either did not develop or report predictive models, had inadequately specified models (e.g., incomplete reporting of predictors or outcomes), or were not peer reviewed. The remaining 14 studies met all inclusion criteria and were included in our literature review. After identifying the 14 peer-reviewed studies by systematic review as described, we included a selection of author-curated conference abstracts that we felt would complement the existing paper.

The eligible peer-reviewed studies (14 total) encompassed a cohort of 583,143 patients, of which 293 were from prospective studies, and 582,850 were from retrospective studies/registries (Table 1). A total of 31 unique predictors of BrM were evaluated in these studies. The most common features used to train predictive models were histology (13 models), age of diagnosis,¹³ overall stage⁹ and N-stage⁹ at diagnosis, and grade.⁶ Twelve features were included in a single model; the majority of these (8/12 features) were infrequently reported in the general literature (e.g., CA125, CA199) (Fig. 1A). Only 10 models (7/10 peer review), out of 14 peer-reviewed articles and 8 conference abstracts, were developed using features available in at least 80% of patients found within the electronic medical records at Penn State (Fig. 1A, marginal barplots). These features included age at diagnosis (n = 1904, 100% availability); chemotherapy (n = 1827, 96.0%); radiotherapy (n = 1831, 96.2%); surgery (n = 1838, 96.5%); T (n = 1836, 96.4%), N (n = 1779, 93.4%), M (n = 1828, 96.0%), and overall (n = 1836, 96.4%) stage at diagnosis; tumor size at diagnosis (n = 1753, 92.1%); histology (n = 1803, 94.7%); number of metastatic sites (n = 1904, 100%); race (n = 1863, 97.8%); sex (n = 1904, 100%); and tobacco use (n = 1809, 95.0%).

Fig. 1.

Comparison of published BrM risk prediction models. (A) Dotplot of features used to construct published BrM risk prediction models. Black dots: Features that are routinely available in electronic medical records. Red dots: not readily available features. Marginal barplots show number of models per feature (top) and number of features per model (right). (B) ROC analysis of published BrM models trained using the Penn State Health retrospective cohort. Model composition was specified per original publications and trained using our retrospective cohort. (C) BrM models rank-ordered by AUROC. Data are mean AUROC ± 95% CI.

Validating existing algorithms using our institutional cohort

We sought to evaluate the performance of published predictive models using independent single-institutional data from the Penn State Health electronic medical record.

Our retrospective cohort included 1904 NSCLC patients, with 434 (22.8%) developing BrMs. Of patients with BrMs, 45 (10.4%) had leptomeningeal disease (LMD), and patients were included irrespective of LMD status. The cohort had a median age of 68 years (IQR 15). The cohort included 1059 males (55.6%) and 845 females (44.4%). Of the cohort, 92.8% of patients identified as White (n = 1766), 1.0% as Asian (n = 19), 4.0% as Black (n = 77), and 2.2% as another race (n = 42). The median follow-up duration was 550 days (IQR 1325 days), measured to either date of death or date of last follow-up. Among only patients who did develop BrM, the median time to development of BrM was 26 days (IQR 270 days). Table 2 summarizes the demographic and disease characteristics of our cohort. NSCLC overall staging at diagnosis followed a bimodal distribution. Most patients presented with either stage 1 (n = 794, 41.7%) or stage 4 (n = 554, 29.1%), while stages 2 (n = 153, 8.0%) and 3 (n = 335, 17.6%) were less common. Adenocarcinomas were the most frequent histological subtype (n = 1007, 52.9%), followed by squamous cell carcinomas (n = 681, 35.8%) and carcinoids (n = 115, 6.0%). The median tumor size at diagnosis was 2.5 cm (IQR 2.7 cm). The number of extracranial organs with metastases was right-skewed. Most patients (n = 854, 44.9%) had no metastases, 36.4% (n = 694) had one site, 12.0% (n = 228) had two sites, and numbers decreased progressively with only one patient (0.1%) having seven metastatic sites. Treatment modalities included radiotherapy (n = 884, 46.5%), chemotherapy (n = 719, 37.8%), and surgery (n = 572, 30.0%). Certain features were frequently unavailable, including grade (77.4% unknown, n = 1474) and molecular status (e.g., EGFR status was unknown in 77.5%, n = 1475).

For the evaluation of published models, we focused on 10 models (7/10 peer review) that used clinical features available in >80% of patient medical records. We made an exception for tumor grade (available in 22.6% of records), which, despite its frequent unavailability, was a key component of high-performing models. We trained each model using glmnet on our independent institutional cohort, employing a 70:30 train-test split. The Zhang 2021 model (features: age at diagnosis; surgical, chemotherapy, and radiation status; T and N stage; histological grade; and number of extracranial organs with metastases) demonstrated the highest performance in predicting BrM incidence in our cohort, achieving an AUROC of 0.91 (95% CI: 0.87, 0.95) (Fig. 1B and C).⁸ Importantly, the Zhang 2021 model demonstrated superior performance in our test cohort, compared to the original lung squamous cell carcinoma (LUSC) cohort used to develop the model [AUROC (95% CI): 0.805 (0.784, 0.825)] (Table 4 model comparison [our cohort vs. original study cohort]). Other models, including Bajard 2004¹³ and Milano 2020,²⁰ validated with satisfactory performance (i.e., AUROC > 0.8) and represent alternative predictive tools that may be considered if clinical features required for higher performing models are not available (Table 5 performance of published BM prediction models in our single-institution cohort of adults with NSCLC).

Table 4.

Comparison of model performance between our test cohort and the Zhang 2021 study.

Characteristics	Zhang et al., 2021	Wilding et al., 2024	p-valuea
Sample size (n)	Training: 17,543 Test: 8611	1904 Training: 1333 Test: 571
Mean age at diagnosis (years)	67	68
Gender (% cohort)	Male: 62.2 Female: 37.8	Male: 55.6 Female: 44.3 Trans-sex: 0.1	<0.001
Race (% cohort)	White: 83.3 Black: 11.2 Other: 5	White: 92.8 Black: 4.1 Other: 3.1	<0.001
T stage (% cohort)	T1: 20 T2: 35.6 T3: 23.4 T4: 21	T1: 41.7 T2: 8.0 T3: 17.6 T4: 29.1	<0.001
N stage (% cohort)	N0: 49.7 N1: 11.1 N2: 30.9 N3: 8.3	N0: 48 N1: 8.2 N2: 23.1 N3: 14.2
Brain metastases (% cohort)	No: 95.8 Yes: 4.2	No: 77.2 Yes: 22.8	<0.001
Number of organs with metastases besides brain (% cohort)	0: 83.8 1: 12.4 2: 3.2 3: 0.6	0: 44.9 1: 36.4 2: 12.0 3+: 6.7	<0.001
Chemotherapy (% cohort)	No: 58 Yes: 42	No: 62.2 Yes: 37.8	<0.001
Surgery (% cohort)	No: 62.5 Yes: 37.5	No: 70 Yes: 30	<0.001
Radiation (% cohort)	No: 56.3 Yes: 43.7	No: 53.5 Yes: 46.5	0.024
AUROC (95% CI)	Training: 0.810 (0.796, 0.823) Test: 0.805 (0.784, 0.825)	0.91 (0.87, 0.95)

^ap-values were calculated using chi-square tests.

Table 5.

Performance of published BM prediction models in our single-institution cohort of adults with NSCLC.

	AUROC	95% CI
Wilding et al. extended model performance at 3-years	0.91	0.87–0.96
Bajard13	0.8	0.75–0.85
Goncalves16	0.77	0.72–0.82
Li19	0.78	0.73–0.83
Milano20	0.82	0.77–0.87
Mujoomdar21	0.74	0.68–0.8
Zhang23	0.74	0.68–0.8
Zhang8	0.91	0.87–0.95
Hsiao (abstract)	0.77	0.72–0.82
Moskovitz (abstract)	0.74	0.69–0.79
Richardet (abstract)	0.66	0.6–0.72

Taken together, our independent cohort analysis validates the performance of several existing predictive algorithms and nominates the Zhang 2021 model as a clinically relevant and feasible model for predicting BrM risk in NSCLC patients.

Clinical features associated with BrM incidence

To evaluate the association between routinely available clinical features and BrM incidence, we performed univariate (uni) and multivariable (multi) logistic regression analyses using clinical features available in >80% of patient medical records at Penn State (Fig. 2, Supplementary Table S1—summary of logistic regression models).

Fig. 2.

Predictors of BrM incidence in NSCLC patients. Forest plot of rank-ordered features predicting BrM incidence in NSCLC patients. Features are rank-ordered by Z-score obtained from univariate and multivariable logistic regression models. Red: p < 0.05; grey: not significant; dot: univariate model; triangle: multivariable model. Reference groups for categorical variables are indicated in legend.

Several tumor characteristics were significantly associated with increased BrM incidence, including high grade [Grade 3 (OR_uni = 6.7, p_uni < 0.0001), advanced N stage [N2 (OR_uni = 5.1, p_uni < 0.0001), N3 (OR_uni = 6.7, p_uni < 0.0001)] and metastatic burden (OR_multi = 3.4 per each additional metastatic site, p_multi < 0.0001). Certain histological subtypes, particularly carcinoid (OR_uni = 0.42, p_uni = 0.002) and squamous cell carcinoma (OR_uni = 0.73, p_uni = 0.007), were less likely to progress to BrMs compared to adenocarcinomas. Age at diagnosis was the only patient-related characteristic negatively associated with BrM incidence (OR_multi = 0.98 per year, p_multi = 0.002). There were no sex-specific differences (p_uni = 0.24). Among treatments, chemotherapy (OR_multi = 1.4, p_multi = 0.049) and radiation (OR_multi = 1.6, p_multi = 0.006) were also associated with increased BrM incidence, whereas surgery was associated with lower rates of BrM incidence (OR_multi = 0.29, p_multi < 0.0001).

Taken together, these analyses demonstrate that several tumor-, patient-, and treatment-related factors influence the risk of BrM incidence. Among these, extracranial metastatic burden, age at diagnosis, and radiation and chemotherapy were the strongest independent predictors of BrM.

A novel and feasible algorithm for predicting BrM risk in NSCLC

To improve on existing algorithms, we leveraged our institutional cohort to develop a clinically accessible and dynamic toolkit that incorporates additional information as it becomes available throughout the patients’ treatment course and follow up. To accomplish this, we developed two models: the baseline and extended models. The baseline model was trained using clinical features available only at diagnosis (i.e., age at diagnosis and clinical N- and M-staging). In contrast, the extended model incorporated treatment information (chemotherapy, radiation, and surgery) and follow-up data (histology subtype and number of extracranial metastases), in addition to the features included in the baseline model. The resulting models were used to generate baseline and extended nomograms to predict BrM risk at 6-month, 1-year and 3-year follow-ups (Fig. 3A and B).

Fig. 3.

Nomogram to predict time-dependent incidence of brain metastasis in NSCLC patients. (A and B) Baseline (A) and Extended (B) nomograms to predict BrM incidence at 6 months, 1 year and 3 years. (C) Time-dependent AUROC for baseline and extended models (bottom) with differential (delta) plotted above. (D) Kaplan–Meier analysis of BrM-free period in low and high-risk individuals, stratified by Nomogram score. (E) Subgroup analysis evaluating features across low and high-risk groups. Abbreviations: AUROC: area under receiver operating characteristic curve.

The baseline model demonstrated strong predictive performance for BrM incidence, with AUROCs of 0.87 (95% CI: 0.77–0.94), 0.84 (95% CI: 0.75–0.93), and 0.85 (95% CI: 0.76–0.92) at 6-month, 1-year, and 3-year follow-up, respectively. The extended model showed significant improvements in predictive accuracy, achieving AUROCs of 0.89 (95% CI: 0.87–0.96), 0.88 (95% CI: 0.84–0.95), and 0.91 (95% CI: 0.87–0.96) at 6-month, 1-year, and 3-year follow-up, respectively. A notable trend emerged when comparing the two models over time. The baseline model’s predictive performance gradually declined as the follow-up duration increased. In contrast, the extended model maintained consistent predictive accuracies across different follow-up periods, thereby demonstrating greater stability over time Fig. 3C). Both models were able to stratify patients into high- and low-risk BrM groups (Fig. 3D), although the differences in time-to-BrM were more pronounced when using the extended model. Consistent with our earlier findings, high-risk patients were typically younger, with advanced stage tumors that underwent radiotherapy or chemotherapy, but not surgery. Higher risk of BrM incidence was also observed among those with higher metastatic burden (Fig. 3E).

These findings suggest that while both models offer strong predictive capabilities, the inclusion of treatment, histology, and metastatic burden in the extended model not only enhanced overall accuracy, but also provided more reliable long-term predictions for BrM incidence in patients with NSCLC.

Discussion

Overview

Our study advances BrM prediction in NSCLC patients through three key contributions: i) validation of existing models, ii) development of a novel time-dependent predictive model (https://nmikolajewicz.shinyapps.io/nomogram_wilding2024/), and iii) identification of clinically relevant risk factors.

Model validation and innovation

The validation of the Zhang 2021 model (AUROC 0.91, 95% CI: 0.87–0.95) represents a critical step towards clinical implementation. Their model was constructed based on the SEER database, using 8 readily available clinical features, including age at diagnosis, N stage, T stage, tumor grade, prior surgery, prior chemotherapy, prior radiation and number of extracranial metastases. In addition, while the variables influencing the overall risk of BrM development identified in our study have also been reported in a study based on the National Cancer Database as well²⁷; our extended model builds upon existing models by incorporating time-to-event data and updating the risk profile following the initiation of therapy, while maintaining comparable accuracy. Notably, our model’s sustained predictive performance offers reliable long-term risk assessment (Fig. 3C).

Clinical risk factors and implications

Our finding that younger age correlates with increased BrM risk warrants particular attention, as it suggests the need for more aggressive screening in younger NSCLC patients. While this association may reflect competing mortality risks in older populations or screening biases, it has been reported by others,⁸ so further investigation into the association between age and BrM risk is warranted. Furthermore, the associations between treatment (i.e., chemotherapy, radiation, surgery) and BrM development presents important considerations for therapeutic planning. Chemotherapy and radiation correlated with increased BrM incidence; one likely explanation is that patients with advanced stage or more aggressive disease are likely to both have BrM and receive more therapy. An alternative, albeit more speculative, explanation might be regarding the emergence of treatment-resistant clonal populations following selective pressures imposed by chemotherapy and/or radiation, which has been observed in several preclinical models.²⁸ Furthermore, mechanistic studies have also shown an association between increased risk of central nervous system invasion in patients undergoing chemotherapy, secondary to the disruption of the blood-cerebrospinal fluid barrier.²⁸ Alternatively, surgical resection of primary tumors appeared to have protective effects; patients with favorable comorbidities, a variable not readily captured in our database, may be more suitable candidates for surgical resection.

An additional clinical implication of our results is the process of clinical implementation. Demonstrated by the heparin anticoagulation study in the 1990’s,²⁹ nomograms have been long-established clinical tools which aim to maximize treatment efficacy and minimize risks to patients by identifying appropriate therapeutic goals. To facilitate clinician acceptance and adoption, these benefits must be empirically demonstrated. Historically, this validation process has commenced at the single-institution level, enabling direct comparison of treatment success rates using the nomogram against control groups. Integrating the nomogram as a link within the electronic health record could enhance accessibility and promote its adoption in clinical practice. Though yet to be prospectively demonstrated, earlier detection of BrM can render a higher proportion of patients eligible for stereotactic radiosurgery, a cost-effective intervention for this disease,³⁰ particularly in high-functioning patients with longer expected survival. Integration of our nomogram can also promote better surveillance.

Study limitations and future directions

Limitations of our study include our predominantly white cohort (92.8%), timing during evolving standard-of-care practices (2011–2020), the retrospective design, and single institutional bias.

The lack of comprehensive biomarker data including EGFR, KRAS, ALK, MET, RET, and BRAF is a significant limitation, as these mutations influence treatment response, options for targeted therapies and immunotherapies, and prognosis in NSCLC, potentially affecting the model’s clinical applicability, especially in precision oncology. The absence of comprehensive molecular profiling data reflects challenges in real-word clinical practice but also indicates an area for future model refinement.³¹

The widespread adoption of immunotherapies and targeted therapies, such as nivolumab, pembrolizumab, and osimertinib, greatly impacted the landscape of systemic therapies for NSCLC, with an approximate clinical impact starting in 2015. Our patient cohort represents NSCLC treatment both pre- and post-immunotherapy era, which may impact the generalizability of our findings to the modern era, where immunotherapy is standard; however, this remains an inherent limitation in predictive modeling, as treatment paradigms continue to evolve over time.

The single-center retrospective cohort design of this study has an influence on patient selection, treatment patterns, and follow-up strategies, which differ across centers. Thus, caution should be exercised when applying this model to different centers without further validation in diverse, prospective cohorts. External validation efforts are ongoing, and such efforts, especially multicenter validation, will be crucial for establishing widespread clinical adoption.

Clinical impact

Our model offers a practical approach to BrM risk assessment using readily available clinical data. This accessibility, combined with reliable predictive performance, positions it as a valuable tool for clinical decision-making. The model’s sustained accuracy overtime particularly supports its utility in long-term patient monitoring and treatment planning.

Contributors

Conceptualization: H.W., M.A., A.M.; Methodology: H.W., N.M., A.M., N.H., D.B.; Data Access and Verification: H.W., C.M., M.T., N.M.; Data Curation: H.W., C.M., M.T., N.H., B.S., B.K.; Formal Analysis: N.M., H.W., A.M.; Visualization: N.M.; Writing—original draft: H.W.; Writing—review and editing: H.W., N.M., K.T., C.M., D.B., A.O., L.M.F., S.S. M.T., N.H., B.S., B.K., A.M., M.A.; Supervision: A.M., M.A.; Decision to Submit Manuscript: H.W., A.M., M.A.

All authors have read and approved the final version of this manuscript.

Data sharing statement

The retrospective cohort curated for this study is stored at the following link: https://github.com/hwild57/NSCLC/blob/main/Wilding%20Table%20S1%20copy.csv. It is available upon reasonable request for IRB-approved studies.

The BrM risk prediction nomograms developed in the current study are publicly available: https://nmikolajewicz.shinyapps.io/nomogram_wilding2024/.

Declaration of artificial intelligence use

During the preparation of this work the authors used ChatGPT in order to assist with minor grammatical editing and phrasing. After using this tool/service, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication.

Declaration of interests

MA reports having received grants or contracts from Pfizer, Incyte, having received consulting fees from Bayer, Xoft, Nuvation Bio, Apollomics, Viewray, Varian Medical systems, Anheart Therapeutics, Theraguix, Menarini Ricerche, Sumitomo Pharma Oncology, GT Medical Technologies, Autem therapeutics, Modifi biosciences, Bugworks, EquiliumBio, Allovir, VBI vaccines, Servier pharmaceuticals, Incyte, Recordati, has participated on a data safety monitoring board or advisory board for VBI vaccines, owns stocks or stock options from Mimivax, MedInnovate Advisors LLC, Trisalus Lifesciences, LiveAI and Modifi Biosciences. The other authors have no conflicts of interest to report.