Deep learning based histomorphological phenotyping and prognostic stratification for combined SCLC and LCNEC

Abstract

Prognostic stratification for combined small and large cell neuroendocrine lung carcinoma (cSCLC-LCNEC) remains challenging. We introduce GTBIS, an interpretable deep learning model for histomorphological phenotyping and prognostic stratification of cSCLC-LCNEC from pathology images. In multicenter cohorts (n = 670), GTBIS accurately differentiated SCLC from LCNEC. When applied to cSCLC-LCNEC patients treated with chemoradiotherapy, GTBIS stratified them into subgroups with different prognoses. Patients in the favorable-prognosis subgroup (SCLC-like phenotype) exhibited significantly better five-year overall survival (100% vs. 39.5%) and disease-free survival (87.5% vs. 36.0%) than those in the poor-prognosis subgroup (LCNEC-like phenotype). Multivariable analysis confirmed GTBIS classification as a strong, independent prognostic factor. Multimodal interpretability analyses linked the favorable-prognosis phenotype to proliferative pathways, whereas the poor-prognosis phenotype to epithelial-mesenchymal transition, hypoxia, and metabolic reprogramming. In conclusion, GTBIS shows promise as a complementary histology-based tool for prognostic stratification, enabling more personalized management of cSCLC-LCNEC.

Introduction

Lung neuroendocrine carcinomas (Lu-NECs) account for approximately 20% of all lung cancers and are characterized by neuroendocrine morphology, including organoid nesting, trabeculae, peripheral palisading, and rosette formation. These tumors exhibit immunohistochemical expression of neuroendocrine markers, such as synaptophysin, chromogranin A, and CD56, for distinguishing them from non-neuroendocrine carcinomas^1,2. Two major high-grade subtypes of Lu-NECs are small cell lung carcinoma (SCLC) and large cell neuroendocrine carcinoma (LCNEC), each with distinct cytological and molecular features^3–8. SCLC is typically characterized by small cells with scant cytoplasm, finely granular chromatin and inconspicuous nucleoli. In contrast, LCNEC consists of larger cells with abundant cytoplasm, prominent nucleoli, vesicular nuclei, and more heterogeneous morphology⁹. SCLC management is stage-dependent, ranging from concurrent chemoradiotherapy for limited-stage disease to immune-checkpoint inhibitor combinations for extensive-stage disease¹⁰. For LCNEC, however, no consensus exists on optimal first-line therapy, and clinical management often defaults to SCLC-derived regimens, despite heterogeneous and frequently suboptimal responses^11,12.

Combined small cell lung cancer and large cell neuroendocrine carcinoma (cSCLC-LCNEC) represents a distinct subset in which both SCLC and LCNEC components coexist within a single tumor. This inherent biphenotypic nature suggests that a “one-size-fits-all” treatment approach is inadequate, as the relative proportion of SCLC (which is typically more sensitive to chemotherapy) and LCNEC components likely influences clinical behavior and outcomes⁴. Therefore, reliable prognostic stratification for cSCLC-LCNEC remains a critical unmet clinical need. Recent advancements in computational pathology have shown considerable promise in analyzing complex histopathological patterns in biphenotypic tumors. For example, artificial intelligence (AI) has been used to reclassify combined hepatocellular-cholangiocarcinoma¹³ and to subtype pancreatobiliary cancers directly from hematoxylin and eosin (H&E)-stained whole-slide images (WSIs), thereby guiding chemotherapy selection¹⁴. However, applying these technologies to stratify cSCLC-LCNEC tumors remains largely unexplored.

To address this issue, we developed the Graph-Transformer Biphenotypic Integrative Stratification (GTBIS), an interpretable deep learning model designed to stratify prognosis directly from H&E-stained WSIs. We hypothesize that GTBIS can objectively stratify cSCLC-LCNEC into subgroups with distinct clinical outcomes by learning the fundamental histomorphologic spectra of pure SCLC and LCNEC. Beyond providing a prognostic stratification tool, we further employ multimodal interpretability approaches by integrating pathomics, proteomics, and phenomics to elucidate the biological rationale underlying the model’s predictions, thereby linking morphology to molecular phenotype. GTBIS provides an objective and reproducible risk stratification tool and represents a step toward data-driven, precision medicine for managing these complex neuroendocrine malignancies.

Results

Cohort characteristics and study design

The clinical characteristics of the 670 patients from the two independent centers are summarized in Table 1. The CHCAMS cohort comprised 532 Lu-NECs, of which 425 were pure SCLCs (79.89%), 85 were pure LCNECs (15.98%), and 22 were cSCLC-LCNECs (4.14%). The HMUCH cohort included 138 Lu-NECs, comprising 109 pure SCLCs (78.99%), 22 pure LCNECs (15.94%) and seven cSCLC-LCNECs (5.07%). All 29 cSCLC-LCNEC patients included in our study had limited-stage disease and underwent curative surgical resection followed by standard postoperative adjuvant chemoradiotherapy. Among them, 75.9% (22/29) were pN0 or pN1 stage.

Table 1.

Baseline characteristics of patients enrolled in this study

Characteristics	CHCAMS cohort	HMUCH cohort
No. of patients	532	138
No. of WSIs	532	138
Pathological diagnosis, n (%)
Pure SCLC	425 (79.89)	109 (78.99)
Pure LCNEC	85 (15.98)	22 (15.94)
cSCLC-LCNEC	22 (4.14)	7 (5.07)
Age, years
Mean ± SD	59.90 ± 9.43	59.98 ± 7.96
Median (range)	59.00 (19–82)	62.00 (38–82)
Sex, n (%)
Male	453 (85.15)	100 (72.46)
Female	79 (14.85)	38 (27.54)
Smoking history, n (%)
Yes	387 (72.74)	97 (70.29)
No	145 (27.26)	41 (29.71)
Tumor location, n (%)
Right	279 (52.44)	71 (51.45)
Left	249 (46.80)	60 (43.48)
Unknown	4 (0.75)	7 (5.07)
AJCC stage, n (%)
I	179 (33.65)	61 (44.20)
Ⅱ	158 (29.70)	43 (31.16)
Ⅲ	177 (33.27)	34 (24.64)
Ⅳ	13 (2.44)	0 (0.00)
Unknown	5 (0.94)	0 (0.00)

SCLC small cell lung cancer, LCNEC large cell neuroendocrine carcinoma, cSCLC-LCNEC combined small cell and large cell neuroendocrine carcinoma, SD standard deviation.

The study design was shown in Fig. 1A and comprised three phases. First, GTBIS was trained to discern the histomorphological spectra of SCLC and LCNEC using WSIs from 425 pure SCLCs and 85 pure LCNECs in the CHCAMS cohort. Second, its discriminative performance was validated on the external HMUCH cohort. Finally, the trained model was applied to stratify cSCLC-LCNEC cases to assess its prognostic utility.

Fig. 1. Schematic of the study.

A Study design workflow. The GTBIS model was trained using five-fold cross-validation on 425 pure SCLCs and 85 pure LCNECs from the CHCAMS cohort to learn histomorphological distinctions. The trained model was then externally validated on an independent HMUCH cohort and subsequently applied to stratify cSCLC-LCNEC patients. Finally, model decisions were interpreted through multimodal analyses. B GTBIS model architecture. The framework integrates a pretrained pathology foundation model for feature extraction, graph convolutional networks for structured representation, and transformer layers for global context aggregation and classification.

Performance of GTBIS in differentiating SCLC and LCNEC

The GTBIS model outputs a continuous probability score ranging from 0 to 1, representing the morphological similarity of a given WSI to a prototypical SCLC pattern (scores approaching 1) versus a LCNEC pattern (scores approaching 0). We first assessed its performance using a stratified five-fold cross-validation on the CHCAMS cohort. For each fold, we partitioned the dataset at the patient level, using approximately 80% of the cases for training and holding out the remaining 20% for validation. This process was repeated five times, ensuring that each case served in the validation set at least once. Performance metrics are reported as the average across all five folds. The model demonstrated excellent discriminative ability, achieving a mean AUROC of 0.972 (Fig. 2A), with all folds exceeding an AUROC of 0.96, and a mean AUPRC of 0.99 (Supplementary Fig. 1A, B). Across the five folds, GTBIS attained an average accuracy of 0.965, a sensitivity of 0.981, a specificity of 0.882, and an F1-score of 0.979 (Supplementary Fig. 1C, D).

Fig. 2. Performance of GTBIS in differentiating SCLC from LCNEC.

A Mean ROC curve with standard deviations from five-fold cross-validation and the overall confusion matrix in the CHCAMS cohort. B ROC curves with 95% confidence intervals and confusion matrices in the HMUCH cohort. Shaded areas represent the 95% confidence intervals. Numbers within confusion matrices indicate the count of correctly classified (diagonal) and misclassified (off-diagonal) WSIs. C Representative H&E-stained whole-slide images of SCLC and LCNEC cases with corresponding GraphCAM attention heatmaps. Regions with higher attention scores (warmer colors) contributed more to the model’s prediction. High-attention tissue patches are shown at 5× magnification (middle panels), and one representative patch is shown at 20× magnification to illustrate key cytological features (right panels).

The model’s generalizability was further validated on an external cohort. In the HMUCH cohort (109 pure SCLCs, 22 pure LCNECs), GTBIS achieved an AUROC of 0.837 (95% CI: 0.729–0.945), an AUPRC of 0.955 (95% CI: 0.910–0.986), and an F1-score of 0.951 (Fig. 2B; Supplementary Fig. 2). These results confirm the robust performance and generalizability of GTBIS in distinguishing SCLC from LCNEC across diverse institutional settings.

To gain pathological insight into the model’s decision-making process, attention maps were reviewed by experienced thoracic pathologists. Their assessment confirmed that GTBIS appropriately focused on tumor regions with histologically characteristic features of SCLC and LCNEC. Specifically, GTBIS focused on regions with dense, hyperchromatic small cells for SCLC and regions containing larger, more differentiated cells with distinct nucleoli and abundant cytoplasm in LCNEC (Fig. 2C).

Prognostic value of GTBIS in cSCLC-LCNEC patients treated with CRT

We next assessed the prognostic value of the GTBIS in cSCLC-LCNEC patients who uniformly received adjuvant chemoradiotherapy (CRT). Using the continuous probability score from GTBIS, which reflects the overall SCLC-like versus LCNEC-like morphological phenotype, all cSCLC-LCNEC patients were stratified into two prognostic subgroups (Fig. 3A). A pre-defined threshold of GTBIS scores > 0.5 was used to define a subgroup with a phenotype predominantly aligned with SCLC (CRT beneficial (CRB), n = 10), hypothesized to have a more favorable prognosis under CRT. Patients with GTBIS scores ≤0.5 were defined as having a phenotype more consistent with LCNEC (CRT unbeneficial (CRU), n = 19), hypothesized to have a less favorable prognosis.

Fig. 3. Prognostic stratification of cSCLC-LCNEC by GTBIS.

A Workflow illustrating the application of GTBIS for treatment-oriented risk stratification in cSCLC-LCNEC. B Comparison of recurrence rate during follow-up between patients stratified as chemoradiotherapy-beneficial (CRB) and chemoradiotherapy-unbeneficial (CRU). C Kaplan-Meier curves comparing overall survival (C) and disease-free survival (D) between the CRB and CRU subgroups. Log-rank p-values are shown.

Patients in the CRU subgroup exhibited a significantly higher recurrence rate during follow-up compared to those in the CRB subgroup (57.9% vs. 10.0%; Fisher’s exact test, p < 0.05) (Fig. 3B). Survival analysis revealed that patients in the CRB group had significantly better overall survival (OS) (log-rank p = 0.012) and disease-free survival (DFS) (p = 0.028). The median DFS and OS for the CRB group were not reached. In contrast, the CRU group had a median OS of 38 months and a median DFS of 36 months. The CRB subgroup showed superior 5-year OS (100%) and DFS (87.5%) rates compared to the CRU subgroup (OS: 39.5%; DFS: 36.0%) (Fig. 3C, D).

GTBIS as an independent prognostic factor

To determine whether the prognostic value of GTBIS is independent of established clinical variables, we performed univariable and multivariable Cox proportional hazards regression analyses. Key clinicopathological variables, including age at diagnosis (≤ 60 vs. > 60 years) and AJCC 8th edition pathological stage (I, II, III), were included in the model as potential confounders. In the univariate analysis, classification into the CRB subgroup was associated with a substantially reduced risk of recurrence (HR, 0.14; 95% CI, 0.02–1.08; p = 0.059). After adjusting for AJCC stage and age, the GTBIS-based classification remained a significant independent prognostic factor for DFS (HR, 0.03; 95% CI, 0.00–0.39; P = 0.007) (Fig. 4A). We further explored the consistency of the GTBIS effect across age subgroups, a factor that demonstrated significant prognostic value in our model. Among patients aged ≤ 60 years, those in the CRB subgroup had significantly better DFS (3- and 5-year DFS: 50.0%) compared to those in the CRU subgroup (log-rank test, P = 0.036) (Fig. 4B). A similar benefit was observed in patients aged > 60 years, with the CRB subgroup showing 100% DFS at both time points, compared to 53.47% and 42.78% in the CRU subgroup (log-rank test, P = 0.038) (Fig. 4C). Taken together, these findings indicate that GTBIS provides clinically significant and independent prognostic stratification.

Fig. 4. Independent prognostic value of GTBIS in cSCLC-LCNEC.

A Forest plot from univariable and multivariable Cox regression analyses evaluating the association of GTBIS classification and clinicopathological variables with disease-free survival. Hazard ratios (HRs) and 95% confidence intervals are displayed. B, C Kaplan-Meier curves for disease-free survival in patients aged ≤ 60 years (B) and > 60 years (C), stratified by GTBIS-based CRB/CRU classification. Log-rank p-values are shown.

Pathological and biological interpretability of GTBIS decisions

We further investigated the pathological and biological relevance of the GTBIS-based reclassification of cSCLC-LCNEC tumors through integrative multimodal analysis. Initially, we used attention maps and Grad-CAM to identify the specific regions within the WSIs that influenced GTBIS classification. Attention maps revealed that GTBIS primarily focused on viable tumor regions, while largely ignoring necrosis and normal parenchyma (Fig. 5A, B). We then quantified the attribution of histomorphological features by calculating similarity scores for the SCLC- and LCNEC-specific morphological features in the top 3% of attention-ranked areas. Quantification via similarity scoring confirmed that high-attention patches in CRB tumors had higher similarity scores for SCLC-specific morphological attributes such as the Azzopardi phenomenon, high nuclear/cytoplasmic ratio, and round or fusiform cells, whereas patches in CRU tumors displayed higher similarity scores for LCNEC-specific features such as larger polygonal cells, prominent nucleoli, nuclear pleomorphism, and vesicular nuclei with clumped chromatin (Fig. 5C, D).

Fig. 5. Pathological and biological interpretability of GTBIS predictions.

A, B Representative H&E images of a predicted CRB (A) and a predicted CRU (B) cSCLC-LCNEC tumor overlaid with pixel-level GraphCAM attention heatmaps (left). Color intensity indicates the magnitude of the patch’s contribution to the model’s decision (scale: low to high). Adjacent panels show tissue patches corresponding to different intervals of attention weights. C, D Scatter plots showing the distribution of semantic similarity scores between high-attention patches (top 3%) and expert-defined morphological attributes characteristic of SCLC (blue), cSCLC-LCNEC (green), and LCNEC (red) for the CRB (C) and CRU (D) subgroups. Each dot represents a patch. E Gene Set Enrichment Analysis (GSEA) of Hallmark pathways based on proteomic data. The bar plot shows normalized enrichment scores (NES) for pathways differentially enriched between CRB and CRU subgroups. Significantly enriched pathways (FDR-adjusted p < 0.05) are shown in dark blue (CRB) and dark red (CRU), while trend-level enrichments are shown in light colors. Pathways are categorized by biological function. F Heatmap displaying the scaled expression of the top protein signatures identified by LASSO regression as predictive of GTBIS-derived histological features across the four patient groups: pure SCLC, pure LCNEC, CRB, and CRU.

To elucidate the biological basis of the GTBIS stratification, we performed GSEA analysis on the proteomic data. The CRB tumors showed significant enrichment in cell cycle-related pathways, including G2M checkpoint, MYC targets, and E2F targets (Fig. 5E). In contrast, the CRU subgroup exhibited significant enrichment immune-related pathways, hypoxia, epithelial-mesenchymal transition (EMT), and metabolic reprogramming (Fig. 5E). Subsequently, LASSO regression identified 61 protein signatures that correlated with histologic features, effectively clustering tumors into groups aligning with pure SCLC and LCNEC (Fig. 5F). Notably, CRB tumors showed protein expression patterns more similar to pure SCLC tumors, whereas CRU tumors displayed protein expression patterns more similar to pure LCNEC tumors.

Discussion

Lu-NECs are a heterogeneous group of malignancies known for their aggressive behavior and poor prognosis. Among them, cSCLC-LCNEC presents a unique clinical dilemma due to its intrinsic biphenotypy. The co-existence of SCLC and LCNEC components within a single tumor creates histological heterogeneity that challenges consistent diagnosis, obscures accurate prognosis, and complicates treatment decisions, as current “one-size-fits-all” strategies derived from SCLC protocols yield suboptimal results for many patients. In this study, we developed and validated GTBIS, a graph-transformer-based deep learning model, to objectively deconvolute this complexity directly from routine H&E-stained WSIs. GTBIS translates global histomorphological patterns into a continuous biological phenotype score and provides a novel, robust, and interpretable framework for the prognostic stratification of cSCLC-LCNEC.

Unlike convolutional neural networks or conventional multiple-instance learning models that process patches independently^15–17, GTBIS exploits the WSIs as a graph, using mincut pooling to aggregate structurally coherent regions. This enables the model to capture long-range spatial relationships and contextual cues that are critical for interpreting complex biphenotypic histology^18–20. GTBIS was trained on pure SCLC and LCNEC cases and first demonstrated high accuracy in distinguishing these pure histologies across multiple cohorts. Critically, the primary clinical contribution of GTBIS lies in its ability to stratify cSCLC-LCNEC patients into two subgroups with different outcomes following standard CRT. Patients classified into the favorable-prognosis subgroup (CRB, SCLC-like) experienced exceptional long-term survival, with median DFS not reached. In contrast, the poor-prognosis subgroup (CRU, LCNEC-like) had a shorter median DFS. Multivariable Cox regression analysis confirmed that this stratification provides incremental prognostic value independent of established clinicopathological factors. The stratification enabled by GTBIS has direct implications for risk-adapted clinical management. For cSCLC-LCNEC patients predicted to benefit from CRT, our data reinforce the utility of current SCLC-based CRT and may help identify patients who can achieve excellent outcomes with standard care. Recent retrospective surgical series have consistently demonstrated that optimally treated, surgically resected limited-stage SCLC patients can achieve favorable survival outcomes^21,22. GTBIS effectively identifies this favorable subgroup within the heterogeneous cSCLC-LCNEC population. For the high-risk CRU subgroup, GTBIS identifies patients who are unlikely to benefit maximally from conventional therapy. This subgroup represents a critical population for clinical trials investigating alternative strategies, such as regimens incorporating immunotherapy (leveraging the inflamed phenotype), agents targeting EMT or YAP1 pathways, or therapies more aligned with LCNEC or non-small cell lung cancer biology. Thus, GTBIS transitions from a prognostic tool to a potential enricher for precision oncology trials.

The integration of AI and deep learning into computational pathology is still in its early stages, particularly for complex biphenotypic tumors such as cSCLC-LCNEC. To understand the potential pathological and biological relevance of the GTBIS-based reclassification of cSCLC-LCNEC tumors, we performed detailed interpretability analyses at both histopathological and molecular levels. Recent large-scale molecular studies have revealed distinct molecular and biological characteristics of SCLC and LCNEC^3,7,8. Our integrative proteomic analysis provides mechanistic insights for the clinically distinct CRB and CRU subgroups for cSCLC-LCNEC. CRB tumors are characterized by heightened cell cycle activity and oncogenic signaling, aligning with a proliferative, neuroendocrine-high phenotype often associated with the classic SCLC-A (ASCL1-high) subtype²³, which is known for its relative sensitivity to DNA-damaging agents such as platinum-based chemotherapy. In contrast, the CRU subgroup exhibits a co-activation of EMT, hypoxia, inflammatory response, and metabolic reprogramming pathways. This molecular profile is characteristic of the therapy-resistant, mesenchymal-like states observed in non-neuroendocrine or SCLC-P/Y subtypes, and aligns strongly with aggressive subsets of pure LCNEC, particularly those with YAP1 activation or an “inflamed” tumor microenvironment^5,24–27. These molecular distinctions are consistent with recent proteogenomic classifications of Lu-NECs showing some of cSCLC-LCNEC tumors share a molecular expression profile with either SCLC or LCNEC⁷. Therefore, the GTBIS-based CRB/CRU stratification effectively divides cSCLC-LCNEC into a proliferative/classic entity likely responsive to standard therapy, and a mesenchymal/resistant entity that may require alternative strategies. Furthermore, by integrating attention mechanisms with visual-language models (CONCH), we quantitatively linked the model’s decisions to diagnostically relevant, text-described morphological attributes, bridging the gap between deep learning “black boxes” and pathological reasoning.

Our study has several limitations that need to be considered. Firstly, the rarity of cSCLC-LCNEC limited the size of our cohort, highlighting the need for larger, multicenter collaborations to expand the dataset and further validate the model’s generalizability. Secondly, as a retrospective study based on surgically resected specimens, the performance and generalizability of GTBIS on small diagnostic biopsies require future validation to fully realize its translational potential across disease stages. Thirdly, the small number of cSCLC-LCNEC cases (n = 29), particularly in the CRB subgroup (n = 10), limits the precision of survival estimates. The 100% 5-year OS observed in the CRB subgroup should be interpreted with appropriate caution, as this finding is based on a small sample size and requires validation in larger, independent cohorts. However, the highly significant HR in multivariable analysis, which is more robust to small sample fluctuations, supports the validity of GTBIS as an independent prognostic factor. Finally, although proteomics provided pathway-level insights, integrating transcriptomics or targeted sequencing could more precisely map GTBIS subgroups onto canonical molecular subtypes.

In conclusion, we developed and validated GTBIS, an interpretable deep learning model that provides robust, independent, and biologically grounded prognostic stratification for cSCLC-LCNEC directly from routine H&E slides. By quantifying the dominant histomorphological phenotype, GTBIS holds the potential to complement clinical guidelines, optimize post-operative management, and ultimately improve outcomes for patients with these complex biphenotypic neuroendocrine carcinomas.

Methods

Ethics approval

This multicenter retrospective study was conducted in accordance with the Declaration of Helsinki and was approved by the Ethics Committee and Institutional Review Board of each participating center: The Cancer Hospital, Chinese Academy of Medical Science (approval No. 22/250-3452), and Harbin Medical University Cancer Hospital (approval No. YD2025-14). Informed consent was waived due to the retrospective nature of the study.

Multicenter patient cohorts

This retrospective study utilized surgically resected, pathologically confirmed specimens and corresponding H&E-stained WSIs from archival formalin-fixed paraffin-embedded (FFPE) tissues, collected from two independent medical centers: The Cancer Hospital, Chinese Academy of Medical Science and Harbin Medical University Cancer Hospital (Fig. 1A). A total of 670 specimens and 670 WSIs were included. All H&E staining was performed using a fully automated and intelligent staining and sealing system to ensure procedural consistency. The study was restricted to primary tumors that underwent complete surgical resection. To ensure diagnostic accuracy and cohort homogeneity, a rigorous, multi-step pathology review and quality control protocol was implemented. First, the original diagnosis at each participating center was established through internal consensus by at least two pathologists. Second, all cases underwent an independent, centralized consensus review by an expert panel of senior thoracic pathologists from the two centers, who independently re-evaluated all H&E-stained slides with specific attention to cSCLC-LCNEC and borderline cases, strictly adhering to the 2021 WHO Classification of Thoracic Tumors. Any diagnostic discrepancies were resolved through joint discussion, and only cases with a definitive, unanimous consensus diagnosis were retained. Third, all digitized WSIs underwent systematic image-quality assessment, and slides with substantial artifacts (e.g., tissue folds, air bubbles, sectioning defects, blurring, or color drift) that could interfere with computational analysis were excluded. All pathologists were blinded to clinical outcomes and model outputs during reassessment.

WSI data preprocessing

FFPE tissue sections were stained using standard H&E protocols. Slides were digitized at 40× or 20× magnification using whole slide scanners (KF-LPE-006, Jiangfeng Biotechnology; Aperio CS2, Leica Biosystems; EasyScan 6, Motic microscopy). WSIs were stored in KFB or NDPI file formats. The digitized WSIs were downsampled to 5× magnification and segmented into non-overlapping patches of 224 × 224 pixels. Only patches with tissue coverage exceeding 50% of the total area were retained to ensure sufficient biological content and minimize background noise.

Deep learning model architecture

We propose a Graph-Transformer-based Biphenotypic Integrative Stratification (GTBIS) model for histomorphological phenotyping and risk stratification of biphenotypic tumors. GTBIS integrates domain-specific pretraining from pathology foundation models with graph-based learning and transformer networks to classify histopathological images. The architecture of GTBIS comprises three key steps: feature extraction and graph construction, graph convolution and pooling, and transformer aggregation and classification (Fig. 1B).

Feature extraction and graph construction

A pretrained pathology foundation model UNI²⁸, was used to extract high-dimensional feature representations from each 224 × 224 pixel image patch. UNI was trained on over 100 million tissue samples and 100,000 WSIs. Each WSI was then conceptualized as an undirected graph, where each patch feature representation (extracted by UNI) is treated as a node in the graph. The edges between nodes are constructed based on the spatial proximity of adjacent patches. Isolated nodes were discarded, and self-loops were added to maintain feature propagation.

Graph convolution and pooling

The node feature matrix $H_m$ and the adjacency matrix$\tilde{A}$ were processed through $M$ layers of graph convolutional network (GCNs):

$$H_{m+1}=\mathit{ReLU}\left({\tilde{D}}^{-\frac{1}{2}}\tilde{A}{{\tilde{D}}^{-\frac{1}{2}}H}_m{W}_m\right),m=1,2,\dots ,M$$ 1

Where $\tilde{D}$ is the diagonal degree matrix of $\tilde{A}$, and $W_m$ are learnable weights. This operation facilitates message passing, allowing nodes to integrate information from their topological neighbors. To reduce computational complexity while preserving structural hierarchy, a MinCutPool layer²⁹ was applied after the GCN layers, performing learnable graph clustering to coarsen the graph.

Transformer aggregation and classification

The pooled graph features were then processed by a transformer network which aggregates local and global contextual information across the entire graph via an attention mechanism. The sequence output by the transformer is then fed into a multilayer perceptron (MLP) head, which produces a continuous output score ranging from 0 to 1, representing the model’s estimate of the overall morphological similarity of the input WSI to a prototypical SCLC pattern (higher values) versus a prototypical LCNEC pattern (lower values). Tumors with a probability score greater than 0.5 were classified as having a phenotype more consistent with SCLC, while those with a score of 0.5 or less were classified as having a phenotype more consistent with LCNEC.

Model training and evaluation

The GTBIS model was trained using a 5-fold cross-validation on the CHCAMS cohort. In each fold, approximately 80% of patients were used for training and the remaining 20% for validation. Training used the Adam optimizer with an initial learning rate of 5e-5, weight decay of 5e-4, and a batch size of 16 for 200 epochs. To optimize training, we implemented a linear warm-up strategy for the first 10 epochs, where the learning rate gradually increased from 5e-5 to 5e-4, followed by cosine annealing with restarts to adjust the learning rate during the later epochs for improving convergence. The binary cross-entropy loss function (BCEWithLogitsLoss) was used to calculate the loss. All experiments were implemented using PyTorch (v2.4.0) on Windows with NVIDIA GeForce RTX 4090 GPU for model training and evaluation.

Pixel-resolution GraphCAM saliency mapping

To interpret GTBIS decisions at the pixel level and identify histologically relevant regions, we first applied the graph-based class activation mapping (GraphCAM) approach to compute the gradients ∇$A^{\left(l\right)}$ and layer relevance $R^{\left(\mathit{nl}\right)}$ associated with the target class³⁰, where ${\mathit{A}}^{\left(l\right)}$ represents the attention map of the multi-head self-attention block at layer $l$, and $n_l$ corresponds to the layer associated with the softmax operation. These gradients and relevance scores were propagated through the network’s layers and integrated with the attention maps to generate the transformer relevancy map $C_t$ for each layer:

$$C_t=\prod _{l=1}^L{\stackrel{\circledR }{A}}^{\left(l\right)}$$ 2

$${\stackrel{\circledR }{A}}^{\left(l\right)}={\mathbb{E}}_h\left(\nabla A^{\left(l\right)}\odot R^{\left(n_l\right)}\right)+I$$ 3

Where $\odot$ denotes the Hadamard product, ${\mathbb{E}}_h$ is the mean across the ‘heads’ dimension, and I is the identity matrix to prevent self-inhibition for each node.

Next, we computed the pooled node features through the MinCutPool layer, which, combined with the original graph’s feature matrix$X$, resulted in the learned dense assignment matrix $S$, as described in Eq. 4. The final graph relevance map $C_g$ was then obtained by mapping $C_t$ to each node in the graph using the dense assignments ($C_t\stackrel{S}{\to }{C}_g)$. Finally, GraphCAM reconstructed the class activation map using the graph’s adjacency matrix and the coordinates of each patch, along with the graph relevance information.

$$X_{\mathit{pool}}=S^{T}X$$ 4

Additionally, we integrated these attention-derived scores with pixel-level weights obtained from a principal component analysis of deep features extracted by the UNI model. The aggregated values were normalized to generate intuitive attribution heatmaps for pathologist review.

Attribute-driven similarity scoring

To quantitatively link model attention with diagnostic histopathology, we performed an attribute-driven similarity analysis. First, SCLC- and LCNEC-specific morphological attributes were compiled from the 5^th WHO Classification of Thoracic Tumors¹. We then used the visual-language foundation model CONCH³¹ to compute the semantic alignment between image patches and these textual attributes. For each WSI, the patch-level visual features $F_P=\left\{f_1,f_2,\dots ,f_p\right\}$ were extracted using CONCH’s image encoder. The textual attributes of SCLC and LCNEC were separately encoded into text feature sets $F_s=\left\{f_1,f_2,\dots ,f_s\right\}$ and$F_L=\left\{f_1,f_2,\dots ,f_l\right\}$, respectively. The cosine similarity between each high-attention patch (top 3% by GraphCAM score) and each set of textual features was calculated as:

$$\mathit{Similarity}\mathit{score}=cos\left(f_i,f_j\right),f_i\in F_p,f_j\in \left\{F_s\cup F_L\right\}$$ 5

This yields two scores per patch: its similarity to SCLC attributes and to LCNEC attributes, allowing us to objectively quantify which morphological phenotype dominates the model’s focused regions.

Proteomic data analysis

Proteomic data from 30 SCLC, 30 LCNEC, and 16 cSCLC-LCNEC cases were obtained from our previous study⁷. To compare pathway-level biology between GTBIS-stratified groups, Gene Set Enrichment Analysis (GSEA) was performed on the Hallmark gene sets curated from Molecular Signatures Database (MSigDB) (https://www.gsea-msigdb.org/gsea/msigdb). Proteins were ranked by fold-change between chemoradiotherapy beneficial (CRB) and chemoradiotherapy unbeneficial (CRU) subgroups, and enrichment was assessed using the GSVA (v1.48.3) R package. To identify direct links between histology and molecular profiles, we extracted 64-dimensional graph-convolutional feature vectors from GTBIS for all 76 proteomic cases. LASSO regression was applied to select protein expressions that were most predictive of these deep histological features. Significant histologic-proteomic signatures were defined as having a Spearman correlation greater than or equal to 0.6 with Benjamini-Hochberg adjusted p-value less than or equal to 0.001.

Statistical analysis

All statistical analyses were conducted using R (v4.4.1) and Python (v3.8.0) with the corresponding packages. Continuous variables were compared using the unpaired t-test (normal distribution), or Wilcoxon rank-sum test (non-normal). Categorical variables were analyzed using Fisher’s exact test when at least one expected cell count in the contingency table was less than five. Model performance was evaluated using area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), accuracy, precision, sensitivity, specificity, and F1-score, with 95% confidence intervals. Kaplan-Meier survival curves were generated, and differences between groups were assessed using the log-rank test with the survminer R package (v0.4.9). Univariable and multivariable Cox regression analyses were performed with the survival R package, and hazard ratios (HRs) with 95% confidence intervals (CIs) were calculated. All statistical tests were two-tailed, with significance set at p < 0.05.

Author contributions

M.Z., S.L., and L.Y. conceived and designed the study. L.Y., S.L., and F.Y collected the samples and acquired the pathological slide images. R.S., Z.Y, Y.Z., and T. W. developed and validated the model. L.Y., R.S., and Z.Y. analyzed and interpreted the data, and wrote the original draft of the manuscript; M.Z. and L.Y. provided supervision, funding acquisition, and guidance throughout the research. All authors read and approved the final version of the manuscript.

Data availability

The H&E images and clinical information analyzed during the current study are not publicly available for patient privacy purposes. Data access can be obtained through a reasonable request to L.Y. (yanglin@cicams.ac.cn). Access to the data will be restricted to non-commercial research, which removes patient-sensitive information. The mass spectrometry proteomic data for 76 Lu-NEC patients in the CHCAMS cohort are available from OMIX (https://ngdc.cncb.ac.cn/omix/) under accession code OMIX006183.

Code availability

The source code of this work can be downloaded from https://github.com/ZhoulabCPH/GTBIS.

Competing interests

The authors declare no competing interests.

Supplementary information

The online version contains supplementary material available at https://doi.org/10.1038/s41746-026-02800-5.