Cellular Architecture and Neighborhood-informed Virtual Spatial Tumor Profiling from Histopathology

SUMMARY

The tumor microenvironment (TME) critically shapes disease progression and therapeutic resistance. However, a comprehensive understanding of its spatial architecture remains elusive, and clinical translation is challenging. Here, we present CANVAS (Cellular Architecture and Neighborhood-informed Virtual AI-driven Spatial Profiling), an artificial intelligence platform that infers tumor ecological habitats from H&E histopathology. Built on an atlas of over 18 million cells profiled by 41-plex spatial proteomics across 457 patients with non-small cell lung cancer, CANVAS establishes 10 reproducible cellular neighborhoods (CNs) capturing conserved spatial organization of the TME. Through multimodal alignment and foundation model-based morphological encoding, CANVAS predicts CN-anchored habitat structures from H&E slides and enables clinical evaluation in over 5,000 patients spanning nine cancer types. Across patient cohorts, CANVAS supports prognostic modeling, spatial ecotype stratification, and immunotherapy outcome prediction. These results establish CANVAS as a clinically scalable platform for spatial profiling, bridging single-cell analysis to population-level insight and enabling precision oncology.

Graphical Abstract

In brief

CANVAS is a computational platform that decodes spatial tumor ecosystems from routine H&E histopathology to enable population-level prognostic modeling and immunotherapy response.

INTRODUCTION

The tumor microenvironment (TME) is a major determinant of cancer progression and therapeutic response, particularly in non–small cell lung cancer (NSCLC), a clinically heterogeneous malignancy^1–3. The TME is a complex ecosystem comprised of multiple cell types and distinct cellular states operating through dynamic interactions across broad scales⁴. Within this landscape, various cell types spatially organize into ecological habitats that represent local niches defined by coordinated phenotypes and functional programs⁵. However, how multicellular architecture organizes into prognostically and therapeutically relevant spatial habitats remains poorly defined. Addressing this problem requires technology platforms that can spatially resolve cellular distributions and tissue-level organizations across patient cohorts^6,7.

Spatial proteomic technologies have enabled spatially resolved, high-dimensional mapping of proteins in intact tissue^8–10. Imaging-based platforms such as co-detection by indexing (CODEX) can resolve cell states, spatial hierarchies, and functional programs at single-cell resolution^9,11. Using CODEX, previous studies have uncovered spatial relationships among tumor, immune, and stromal cells that are inaccessible with traditional bulk profiling^11–13. In NSCLC, spatial features including immunosuppressive stromal niches and organized tumor–immune interfaces have been shown to correlate with prognosis^14,15 and immunotherapy response¹⁶.

Despite its analytical power, the clinical deployment of spatial proteomics is hindered by high cost, technical complexity, and limited scalability. Moreover, most studies on spatial proteomics are confined to a small number of samples, which impacts the robustness and generalizability of the findings. By contrast, hematoxylin and eosin (H&E) histopathology remains the gold standard for cancer diagnosis, offering universal accessibility and compatibility with clinical workflows^17,18. While H&E captures gross tissue architecture and cellular morphology¹⁹, it does not measure protein expression and cannot directly resolve detailed molecular circuits. This trade-off presents a major barrier to translating spatial biology into precision oncology⁷.

Bridging this gap requires a framework that integrates the molecular depth of spatial proteomics with the accessibility of histopathology. A central challenge lies in utilizing deep learning to identify biologically grounded and clinically informative spatial domains from whole-slide histopathology images. Although recent studies have shown the feasibility of inferring spatial protein expression from H&E images, the prediction accuracy is moderately low^20–23. More importantly, these models lack the granularity needed to recover functionally coherent spatial structures.

To address this problem, we first delineated the single-cell spatial landscape of the lung tumor microenvironment by profiling over 18 million cells via 41-plex CODEX imaging across 457 tumors. We then constructed a taxonomy composed of 10 cellular neighborhoods (CNs), which capture the spatial topologies of tumor, immune, and stromal architectures conserved across histologic subtypes. Building on this taxonomy, we developed CANVAS, an AI platform that predicts CODEX-defined CNs based on standard H&E histopathology. CANVAS leverages a pretrained pathology foundation model for robust feature extraction, followed by multiscale spatial analysis to uncover informative ecological features. Applied to H&E slides from over 5,000 patients across nine cancer types, CANVAS enables prognostic modeling, molecular subtype stratification, and immunotherapy outcome prediction. Together, these findings establish CANVAS as a clinically scalable platform that bridges spatial proteomics and histopathology, with a potential for enabling the identification and translation of novel spatial biomarkers for precision oncology.

RESULTS

Distinct immune and stromal remodeling in lung cancer histologic subtypes

To spatially resolve the cellular architecture of the lung TME, we performed high-dimensional CODEX imaging on tumor samples from 457 NSCLC patients, comprising two tissue microarray (TMA) cohorts and ten whole-slide sections with matched H&E staining of the same tissue section (Fig. 1a). We used a validated 41-plex antibody panel targeting cell lineage and functional markers, enabling the classification of four major cellular compartments including epithelial, lymphoid, myeloid, and stromal, with further resolution of intra-lineage phenotypic diversity (Fig. 1b; Table S1). The panel included markers representing key functional programs such as proliferation, stemness, hypoxia, antigen presentation, and immune checkpoint activity. This enabled fine-grained annotation of cellular states, which is essential for decoding the spatial and functional heterogeneity of the TME.

Figure 1. Immune and stromal cell composition across lung cancer histologies.

a, Study design and workflow. b, The 41-marker CODEX panel spanning epithelial, lymphoid, myeloid, stromal, and functional programs. c, UMAP visualization of single-cell CODEX profiles, with 15 major cell types assigned by lineage-guided annotation. Abbreviations: Tcyto, cytotoxic T cell; Th, helper T cell; NK, natural killer cell; Treg, regulatory T cell; M1/M2, macrophage subtypes; EC, endothelial cell; SMC, smooth muscle cell; DC, dendritic cell; CAF, cancer-associated fibroblast. d, Heatmap of scaled expression of lineage markers across the 15 annotated cell types. e, Patient-level distribution of 14 non-epithelial cell types across adenocarcinoma (ADC) and squamous cell carcinoma (SCC), shown as a waterfall plot. Representative H&E images illustrate histologic architecture. f, Schematic of hierarchical lineage annotation into epithelial, immune, and stromal compartments using canonical and marker-defined features. g–j, Relative patient-level proportions of lymphoid (g), myeloid (h), stromal (i), and total immune (j) compartments in ADC and SCC. Mann–Whitney U test; *P < 0.05, **P < 0.01, ***P < 0.001.

We profiled and annotated tumor samples from 337 patients in TMA cohort 1 using a canonical marker–guided, unsupervised lineage classification framework (see Methods). This process allowed us to resolve 14 distinct microenvironmental cell populations in addition to epithelial tumor cells (Fig. 1c–f, S1a). Overall, endothelial cells (ECs) constituted the most abundant non-epithelial population, accounting for 10.4% of all cells. This was followed by helper T cells (Th, 9.6%), cancer-associated fibroblasts (CAFs, 8.2%), and neutrophils (6.5%) (Fig. S1b).

Comparative analysis of the two major histologic subtypes of NSCLC revealed distinct differences in microenvironmental composition between LUAD and LUSC. Immune cell infiltration was moderately higher in LUSC (43.7%) compared to LUAD (37.8%; P < 0.01), primarily due to an increased proportion of myeloid cells (21.8% vs. 14.8%; P < 0.001), while lymphoid and stromal compartments were largely similar (Fig. S1c). At the individual cell level, a more detailed analysis revealed neutrophils and M2 macrophages (M2) were more abundant in LUSC; whereas LUAD exhibited relative enrichment of Th, plasma cells, smooth muscle cells, and dendritic cells (DC) (Fig. S1b, S2a).

More granular histologic subtyping further reflected distinct patterns of cellular composition within the TME. In LUAD, progression from lepidic to solid patterns was associated with an increase in lymphoid lineages while LUSC showed minimal variation in the relative abundance of major TME compartments across subtypes (Fig. 1g–j). Single-cell analysis further delineated subtype-specific trends: in LUSC keratinization was associated with a significant expansion of CAFs (9.3% vs. 7.1%, P < 0.05) accompanied by a decreased infiltration of B cells (2.1% vs. 3.7%, P < 0.05) (Fig. S2b). Conversely, LUAD demonstrated an increase in adaptive immune constituents with increasing histologic grade, including cytotoxic T cells (Tc; 8.0% vs. 4.5% in solid vs. lepidic, P < 0.01), B cells (3.4% vs. 2.3%, P < 0.01), and Tregs (2.4% vs. 1.5%, P < 0.01) (Fig. S2b).

To investigate the association between TME composition and clinicopathologic features, we quantified cell-type proportions in each sample and integrated them with patient-level metadata, including demographics, tumor characteristics, molecular alterations, histologic subtypes, and survival outcomes (Fig. 2a–b). In LUSC, cancer cells were more abundant in advanced-stage tumors and those exhibiting lymphovascular or pleural invasion, whereas Tc, Th, and NK cells were enriched in early-stage tumors without pathologic evidence of invasion, reflecting a cytotoxic immune-infiltrated TME. CAFs exhibited selective expansion in keratinizing SCCs, suggesting stromal-driven modulation of squamous epithelial differentiation²⁴ (Fig. 2a). In LUAD, Tc and Th cells were enriched in solid-predominant histology, early-stage disease, and smaller tumors, where their presence pointed toward an active adaptive immune landscape and correlated with a favorable prognosis (Fig. 2b). In poorly differentiated tumors, neutrophils and M2 were enriched in subgroups with higher invasive potential, consistent with pro-tumorigenic inflammation and immunosuppressive remodeling associated with aggressive disease phenotypes²⁵ (Fig. 2b). Although TME compositional features correlated with clinical parameters, their limited prognostic utility suggested that additional higher-order spatial relationships may contribute to tumor progression²⁶.

Figure 2. Spatial cell–cell interaction networks and topological rewiring in the NSCLC tumor microenvironment.

a–b, Associations between cell-type frequencies and clinicopathologic variables in SCC (a) and ADC (b). Bubble plots show significance (−log₁₀FDR) and direction of association. Mutation status includes EGFR, ALK, ROS1, or KRAS alterations. MCT, mast cell tryptase percentage. c, Schematic of spatial interaction and co-occurrence analyses. d–e, Permutation-derived spatial interaction matrices showing scaled pairwise interaction landscapes across major NSCLC subtypes (ADC and SCC; d) and histologic subtypes (e), with ADC grouped into low-grade and high-grade tumors and SCC stratified into keratinizing and non-keratinizing subtypes. f, Forest plot of associations between cell–cell interaction strength and overall survival using Cox proportional hazards models. g, GNN-derived cell–cell topological network constructed from spatial cell graphs across the NSCLC cohort. h, Representative tumor-centered triplet motifs identified by integrating GNN-derived topology and survival associations. i–j, Representative CODEX images of tumor-centered triplet motifs, including lymphoid-enriched configurations (i) and myeloid–stromal arrangements (j).

Spatial topology sculpts immunoregulatory architecture in lung cancer

We delineated the intercellular spatial interaction network of the lung TME with a focus on subtype-specific differences in cellular organization. To do this, we applied a permutation-based approach to quantify pairwise spatial interactions between cell types, complemented by co-occurrence analysis at the sample level (Fig. 2c; see Methods). In LUAD, tumor cells were spatially insulated from Tc and Th cells, consistent with limited cytotoxic engagement; whereas neutrophils were preferentially localized near M2 macrophages and ECs, forming interaction patterns associated with immunosuppressive and angiogenic remodeling (Fig. 2d, box a-b). In LUSC, Tc and Th cells were more frequently positioned adjacent to stromal and myeloid compartments, including CAFs, M2 and ECs (Fig. 2d, box c; S2c), suggesting a subtype-specific redistribution of immune effectors away from tumor cells.

We further evaluated how intercellular networks evolve with histologic progression. In LUAD, compared with other histologic patterns, high-grade patterns (micropapillary and solid) showed increased crosstalk between cancer cells and neutrophils, ECs, M1 macrophages, and B cells in spatial interaction analysis. This was accompanied by diminished tumor–Tc interactions despite increased Tc in high-grade tumors (Fig. 2e; box a). The spatial proximity of neutrophils, ECs, and tumor cells suggests a pro-metastatic vascular gateway that may facilitate vascular invasion and tumor dissemination²⁷. In high-grade LUSC (non-keratinizing), spatial interactions between M2 macrophages and both T lymphocytes and NK cells were enhanced, consistent with the formation of immunosuppressive niches that may constrain effective anti-tumor immunity (Fig. 2e; box b). Additionally, co-occurrence analysis showed that dendritic cells tended to co-aggregate with CAFs, M2 macrophages, and ECs (Fig. S2d, box a). In contrast, keratinizing LUSC exhibited a fibrotic exclusion-like architecture, with CAFs remaining spatially adjacent to Tc, Th, B cells, and NK cells (Fig. 2e, box c). ECs showed strong co-occurrence with M2 macrophages, CAFs, and B-lineage cells, likely forming perivascular exclusion zones with decreased T cell infiltration (Fig. S2d, box b). Despite architectural differences, LUAD and LUSC shared a conserved immunoregulatory tumor microenvironment. Within this shared tumor-immune landscape, subtype-adjusted survival analyses revealed that tumor cell interactions with neutrophils as well as lymphoid interactions with M2 and CAFs were associated with adverse prognosis, whereas Tc engagement with Th, B, and plasma cells correlated with favorable outcomes (Fig. 2f).

To investigate higher-order cellular interactions beyond pairwise proximity and decode spatial topologies, we applied graph neural networks (GNNs) to spatially resolved cell–cell graphs across the NSCLC cohort (see Methods). This analysis revealed a conserved immunoregulatory hub comprising tumor cells, Tc, Th, M2 macrophages, CAFs, ECs, and neutrophils (Fig. 2g), which consistently occupied central network positions and orchestrated TME organization. To further dissect the architecture of these regulatory cores, we enumerated all three-cell motifs by integrating GNN-derived node centrality with edge density and ranked them using robust rank aggregation (RRA, see Methods). We identified 24 topologically enriched triplets (FDR < 0.01), of which 12 were significantly associated with patient prognosis (Fig. 2h). High-risk motifs were dominated by tumor-associated myeloid and stromal triplets (e.g., M2–tumor–CAF, neutrophil–tumor–CAF), reflecting a fibroinflammatory axis of immune suppression, whereas protective motifs incorporated effector lymphocytes and ECs (e.g., Tc–tumor–Th, EC–tumor–B), supporting cytotoxic engagement and vascular accessibility (Fig. 2h–j).

Together, these results underscore the importance of single-cell spatial analysis to capture fundamental TME biology. Our findings suggest that NSCLC progression emerges along two interrelated axes: one driven by shifts in cellular composition, and the other sustained by intercellular interactions that establish an immunoregulatory circuitry. These spatial topological architectures were conserved across histologic subtypes, suggesting a unifying structural framework that integrates tumor progression with spatially coordinated immune modulation.

Cellular neighborhoods underpin prognostic ecological units in lung cancer

In NSCLC, spatially organized cell states form recurrent multicellular patterns that collectively define hierarchical niches composed of distinct immune and stromal lineages¹. Due to their topological recurrence, these multicellular configurations provide a spatially conserved scaffold for decoding TME heterogeneity across tumors²⁸. To systematically delineate these multicellular networks, we applied an unsupervised clustering algorithm that defined spatially resolved cellular neighborhoods (CNs) by aggregating lineage-annotated cells within a fixed radius (Fig. 3a, S3a; see Methods). This analysis revealed 10 CNs that capture archetypal tissue ecologies with distinct tumor-immune or stromal imprints: tumor core (CN01), macrophage-enriched niche (CN02), B-cell niche (CN03), stromal-fibrotic domain (CN04), plasma cell cluster (CN05), neutrophil-rich zone (CN06), tumor–immune interface (CN07), T cell compartment (CN08), pan-immune activation zone (CN09), and vasculature niche (CN10) (Fig. 3b, d–f).

Figure 3. Cellular neighborhoods define prognostic ecological units and spatial immune barriers in NSCLC.

a, Schematic of cellular neighborhood definition from local cell-type composition within a fixed radius. b, Heatmap showing scaled lineage enrichment across the 10 CNs, delineating spatial compartments defined by major cell lineages. c, Forest plot of hazard ratios from univariable Cox analysis of CN abundance for overall survival. d, Representative CN Voronoi maps from four samples showing CN distributions across ADC and SCC. e–f, Representative CODEX images illustrating CN spatial organization based on marker co-expression patterns. g, Circos plot summarizing the distribution and prognostic significance of 64 cell phenotypes across CN-defined CEUs. Five concentric tracks are shown from the outermost to the innermost: (1) phenotype identity; (2) CEU relative enrichment within each CN; (3) scaled fold-changes between ADC and SCC, with blue bars indicating SCC enrichment; (4) statistical significance of subtype differences, where red points denote FDR < 0.01; and (5) representative CEUs with prognostic associations within each CN. An adjacent bubble plot summarizes the number of prognostically significant CEUs (log₂-transformed) per CN. h, Kaplan–Meier curves for overall survival of eight selected CEUs. i, Volcano plot comparing differential proximity of FAP⁺ CAFs to other cell types between CN07 and CN01. j, Scaled spatial proximity density of FAP⁺ CAFs or HIF1A⁺/Ki67⁺ neutrophils with CD8⁺ T cell subsets across CN domains. k, Density plots of intercellular distances between FAP⁺ CAFs or HIF1A⁺/Ki67⁺ neutrophils and CD8⁺ T cell across CN01, CN07, and other CNs. l, Schematics of barrier score calculation based on shortest spatial paths linking cytotoxic T cells and tumor cells under CAF-defined spatial constraints. m, Barrier score distributions in LUAD and LUSC stratified by early- and advanced-stage disease. n, Kaplan–Meier curves of PFS comparing high and low barrier score groups in stage I NSCLC. *P < 0.05, **P < 0.01.

Our analysis revealed that CN06 (neutrophil-rich zone) was enriched in poorly differentiated, large LUAD tumors, whereas CN08 (T-cell compartment) was associated with less invasive, smaller tumors in both subtypes (Fig. S3b–c). We performed univariate Cox regression analysis to evaluate the prognostic associations of CN-level enrichment and found that tumor-associated, fibrotic, and neutrophilic structures (CN01, CN04, and CN06) were linked to unfavorable prognosis, whereas B cell, plasma cell, T cell, and tumor–immune interface neighborhoods (CN03, CN05, CN07, and CN08) were linked to favorable clinical outcomes (Fig. 3c). These associations were reproduced in an independent NSCLC cohort, supporting the cross-cohort consistency of CN-based distribution patterns (Fig. S3d–f). Collectively, these findings support that CNs are biologically defined, clinically informative spatial domains that are conserved across histologic subtypes in NSCLC.

To further resolve intra-neighborhood complexity, we defined cell state–resolved ecological units (CEUs) by expanding the cellular hierarchy to include 64 phenotypic cell states (Fig. S3g; Table S3). Quantification of these refined phenotypes within each CN revealed how distinct functional programs shape the cellular composition and organization of spatial niches. Correlation of CEU profiles with clinical trajectories uncovered robust subtype-associated patterns and prognostic associations (Fig. 3g). A dichotomous pattern was evident, where CEUs defined by pro-tumorigenic and fibro-inflammatory programs within CN01, CN04, and CN06 were correlated with diminished survival expectancy. Conversely, lymphoid–endothelial CEUs located in CN03, CN08, and CN10 were preferentially associated with improved survival (Fig. 3g–h).

At the cell-state level within CEUs, CN01 was enriched in PanCK⁺ tumor cells, CD44⁺ stem-like subsets, and Ki67⁺ proliferative populations, whereas E-cadherin⁺ tumor cells marked a more preserved epithelial state with limited epithelial–mesenchymal transition (EMT)^29–31. Within CN04, the accumulation of matrix-associated CAFs (mCAFs) and HIF1A⁺ fibroblasts reflected a desmoplastic niche with a hypoxia-driven fibrotic program (Fig. 3g). CN06 was dominated by activated neutrophil states, including Ki67⁺/HIF1A⁺ subsets, consistent with a hypoxic and potentially pro-tumorigenic neutrophil program associated with worse outcomes (Fig. 3g–h)³². In contrast, immunocompetent ecological units demonstrated features of coordinated immune surveillance. CN03 featured an enrichment of B cells expressing HLA-A/E and HLA-DR, supporting antigen presentation and humoral immunity within the tumor milieu³³. CN08 was enriched with cytotoxic CD8⁺ T lymphocytes expressing granzyme B and CD4⁺ T helper cells, consistent with sustained effector activity and adaptive immune activation³⁴ (Fig. 3g–h).

Analysis of CN07 revealed a spatially distinct CEU at the tumor–immune interface, characterized by the coexistence of pro- and anti-tumor features (Fig. 3e–f, 3g). Compared with tumor core CN01, this niche uniquely integrated CD8⁺ T cells and their GZMB⁺ effector subset with Ki67⁺ tumor cells, HIF1A⁺/Ki67⁺ neutrophils, and FAP⁺ CAFs, establishing a microenvironment marked by ambivalent immunomodulation (Fig. 3g, 3i, S4a–b). To investigate the structural logic of this composite microenvironment, we resegmented all tissue sections using CN boundaries and quantified a spatial metric termed proximity density between CD8⁺ cytotoxic T cells and key cellular subsets, including FAP⁺ CAFs and HIF1A⁺/Ki67⁺ neutrophils (see Methods). Proximity density reflects the degree of spatial co-localization normalized by the local abundance of the interacting cells within each CN. Among all CNs, CN07 exhibited the highest proximity density between CAFs and cytotoxic T cells (Fig. 3j), suggesting a concentrated spatial association within this niche and indicating a dense, spatially restricted interface. These immune–stromal pairs were positioned at significantly shorter intercellular distances in CN07 than in tumor core CN01, suggesting a compact stromal architecture associated with local cytotoxic T-cell exclusion at the invasive margin (Fig. 3k, S4c).

To quantify the extent of stromal insulation, we computed a barrier score derived from topological tumor graphs³⁵, which measures the degree to which stromal elements, particularly FAP⁺ CAFs, spatially interpose between CD8⁺ T cells and proliferating tumor cells (Fig. 3l). The triplet-derived barrier score showed a stage-associated pattern, with relatively higher values in early-stage tumors (stage I–II) than in advanced-stage tumors (stage III–IV) (Fig. 3m; Fig. S4d). This pattern suggests that CAF-mediated insulation is more apparent in early lung cancer, consistent with the presence of a more intact desmoplastic rim in which boundary-localized CAFs and aligned ECM programs can enforce T cell exclusion at the tumor–stroma interface³⁶. Clinically, elevated barrier scores were associated with worse progression-free survival (PFS) in NSCLC (Fig. S4e), particularly among stage I patients (P = 0.004; HR [95% CI]: 2.06 [1.25–3.41]; Fig. 3n).

We next expanded the analysis to 64 phenotypic cell states to identify more granular CNs. Using this expanded taxonomy, we identified 20 distinct CNs representing discrete cellular niches. Among these, CN04 delineated a hypoxic–proliferative domain characterized by the co-localization of LAG3⁺ CD4⁺ T helper cells co-expressing VISTA, HIF1A⁺ neutrophils, and Ki67⁺ tumor cells. CN17 defined an immunoregulatory niche enriched for Breg-like cells, Tregs, Th2-like cells, and PD-1⁺ CD4⁺ T cells (Fig. S4f–g). Both CN04 and CN17 showed adverse prognostic effects, with their abundance correlated with poor patient outcomes (Fig. S4i–j). Although the 20-CN model resolved finer-grained niches, the original 10-CN model showed greater compositional purity and lineage homogeneity, supporting a more stable representation of the lung TME (Fig. S4h). Overall, these results establish CNs as spatially resolved tumor architectures, wherein cell-state ecological units bridge functional diversity with multicellular structure to inform clinical outcomes.

CANVAS enables prediction of tumor habitats from whole-slide histopathology

By defining CNs as spatially recurrent ecological domains, we established a taxonomy that captures TME heterogeneity through conserved cell-state programs and architectural patterns. While this provides important insight into TME biology, CODEX-based spatial proteomics is not practical for routine clinical applications. Hematoxylin and eosin (H&E) histopathology, however, is widely used in clinical diagnosis and advanced computational modeling offers an opportunity to reveal the subtle relationships between molecular phenotypes and morphological patterns^37,38. Motivated by this, we developed CANVAS (Cellular Architecture and Neighborhood-informed Virtual AI-driven Spatial profiling). CANVAS predicts tumor ecological habitats from standard H&E histology and comprises two integral modules (Fig. 4a; see Methods). First, we adapted a vision–language foundation model pretrained on millions of paired histopathology images and semantic annotations to predict CODEX-defined CNs as spatial habitats directly from H&E slides. Second, we constructed clinical outcome prediction models based on habitat-centric spatial features to capture biologically relevant patterns and enable histopathology-based patient stratification. Together, these modules support scalable and clinically translatable spatial diagnostics for precision oncology.

Figure 4. CANVAS enables accurate prediction of tumor habitats from whole-slide histopathology with prognostic and immunogenomic correlates.

a, Schematic of CANVAS predicting CODEX-derived CN annotations from paired H&E images. b, Confusion matrices showing CANVAS classification performance in the training, validation, and testing sets. c, Representative CANVAS-derived habitat maps, tumor probability maps, and H&E slides from two TCGA samples. d, Representative H&E regions defining tumor bulk and leading-edge tissue compartments. e, Bubble plots showing survival associations of habitat abundance in tumor bulk and leading-edge regions in the TCGA-LUAD cohort. f, Consensus clustering of habitat compositions identifying four spatial ecotypes (C-I to C-IV) across lung cancer tissues. g, Gene-level copy number alterations associated with spatial ecotypes, summarized as log2 odds ratios. h, Heatmap of somatic mutation enrichment across spatial ecotypes, with mutation frequencies z-score normalized. i, Representative pathways differentially enriched across the four spatial ecotypes. j, Distribution of immunotherapy-related gene signature scores across ecotypes. Associations between ecotypes and continuous molecular features were assessed by ANOVA. *P < 0.05; **P < 0.01; n.s., not significant.

The first module of the CANVAS approach comprised four core components: (1) annotation of CODEX-defined CNs as the ground truth for H&E-based habitat prediction; (2) automated nuclear segmentation and localization on H&E-stained sections; (3) single-cell spatial co-registration of CODEX and H&E images on the same tissue to align CN identities with histology; and (4) foundation model-driven prediction of CODEX-defined CNs using H&E as input (Fig. 4a; see Methods). This pipeline enables the precise transfer of spatially informative CNs from high-plex CODEX to standard H&E images, supporting histology-native spatial diagnostics.

To support model development, we curated a multimodal dataset of CODEX/H&E image patches (224 × 224 pixels) from whole-slide images (WSIs) and tissue microarray, with patch labels assigned through cell-level CODEX–H&E co-registration (see Methods). After quality control, each patch was assigned to one of 10 CN labels based on CODEX analysis. The WSI cohort was split into training (n = 8 WSIs) and validation (n = 2 WSIs) sets, whereas TMA cohort 2 was used as an independent test set. As a result, CANVAS demonstrated robust patch-level generalizability across training, validation, and independent test sets, achieving high classification performance with accuracy exceeding 0.83, F1 scores above 0.79, and Kappa coefficients above 0.80 (Fig. 4b; Fig. S5a). In benchmarking analysis, CANVAS consistently outperformed both established pathology foundation models and traditional convolutional neural networks for habitat prediction (P < 0.01; Fig. S7a). Beyond these benchmarks, CANVAS maintained strong performance for varying H&E resolutions, with comparable accuracy between ×40 and ×20 magnifications (Fig. S7b). Finally, CANVAS remained robust across different patch extraction strides, with nonoverlapping tiling yielding performance comparable to tiling with up to 75% overlap and accuracy consistently exceeding 0.85 (Fig. S7c; see Methods).

These results established the technical robustness of CANVAS for habitat inference from routine H&E images. We next applied CANVAS to three NSCLC cohorts, including TCGA, PLCO, and NLST, covering both LUAD and LUSC histologies. CANVAS accurately mapped spatial habitat architectures and identified recurrent habitat-level patterns linked to clinical outcomes (Fig. 4c–e). In TCGA-LUAD, higher abundances of H01, H04, and H06 were associated with poor survival, whereas H03, H05, and H08 were associated with favorable prognosis, consistent with our CODEX-based findings (Fig. 4e). These associations were refined by regional partitioning into tumor bulk and leading-edge compartments (Fig. 4d–e; see Methods). Similar prognostic patterns were observed in the PLCO and NLST cohorts, where higher relative abundances of H01, H04, and H06 were generally associated with adverse outcomes, whereas H03 and H08 were associated with favorable prognosis (Fig. S6a, S6d–e).

To assess the broader applicability of CANVAS, we applied the model to same-section CODEX–H&E data from 17 specimens spanning 12 tumor types. In this benchmark, CANVAS showed concordance between H&E-predicted and CODEX-derived habitat profiles across diverse tumor contexts (Fig. S7d). Orthogonal validation using same-section ST–H&E samples further showed agreement between H&E-predicted and ST-derived habitat profiles (Fig. S7e). We then extended the analysis to 3,724 H&E slides from eight TCGA cancer types, revealing CANVAS-inferred habitats showed recurrent but cancer type–dependent prognostic associations beyond lung cancer (Fig. S7f).

CANVAS defines spatial ecotypes associated with immunogenomic features

To assess relationships between histology-inferred tumor habitats and immunogenomic features, we compared CANVAS-predicted spatial architectures with transcriptome-deconvoluted cellular composition and signatures in TCGA-LUAD cohort (see Methods). This analysis revealed coordinated enrichment patterns, where H06 correlated with neutrophil abundance, while H03 and H08 clustered with adaptive immune modules enriched for B and T cells (Fig. S5b–d).

Building on these associations, we performed unsupervised consensus clustering of tumor habitat proportions in TCGA-LUAD cohort, which revealed four tumor clusters or ecotypes with distinct spatial architecture profiles (Fig. 4f). C-I (myeloid-inflamed) was dominated by habitats H05 and H06, indicative of an inflammatory neutrophil-rich and innate-oriented TME. C-II (fibrotic) featured a fibrotic habitat H04 with minimal immune infiltration. C-III (tumor-enriched) was mainly composed of habitats H01 and H07, characterized by high tumor purity and low stromal and immune content. C-IV (immune-active) exhibited coordinated enrichment of lymphocyte-abundant habitats (H03, H08) (Fig. 4f). Although spatial ecotypes showed partial correspondence with established TME classifiers³⁹, they demonstrated substantial independence from bulk transcriptome-based classifications (Cramér’s V = 0.18; Fig. S5e). These spatial ecotypes stratified survival outcomes across various clinical endpoints (log-rank P < 0.05; Fig. S6b–c), and remained independently prognostic for OS and DSS in multivariable Cox models adjusted for clinicopathologic covariates (Fig. S6f–g).

To enhance the biological interpretability of CANVAS-defined spatial ecotypes, we integrated each cluster with multi-omic features encompassing somatic alterations, oncogenic signaling, and immunotherapy-relevant transcriptional programs. C-I was characterized by co-amplification of BCL2 and BRAF, consistent with enhanced survival and proliferative signaling^40,41. Upregulation of CYP51A1 further suggested altered cholesterol biosynthesis⁴². C-II exhibited deletions in MTOR and RAP1GAP, in line with dysregulated PI3K and Rap1 signaling, while modest SOX2 amplification was compatible with increased epithelial plasticity⁴³. C-III, defined by gains in CDKN1A, TRAF5, and TGFB2, reflected a tumor-rich phenotype with active TGF-β signaling⁴⁴. In contrast, C-IV showed amplification of HIF1A and immune checkpoint loci (CD274 and PDCD1LG2), denoting a hypoxia-adapted, immune-infiltrated ecotype with potential immunotherapy relevance⁴⁵ (Fig. 4g). Complementary analysis of SNV profiles reinforced these ecotypic distinctions. C-I harbored mutations in BRCA1 and FGFR2, consistent with alterations in DNA repair and RTK signaling^46,47. C-II featured alterations in PTEN, JAK1, TSC1, and EPAS1, suggestive of disrupted immune and hypoxia-related signaling⁴⁸. C-IV was enriched for mutations in EP300, MSH6, and MACF1, associated with immunogenicity, mismatch repair deficiency, and cytoskeletal remodeling⁴⁹. Conversely, C-III displayed a sparse mutational burden dominated by EGFR mutations, consistent with an EGFR-driven and less immune-infiltrated tumor state⁵⁰ (Fig. 4h).

Each CANVAS-defined ecotype exhibited distinct transcriptional programs reflective of its spatially encoded microenvironment. C-I activated Toll-like receptor signaling, consistent with neutrophil-driven innate inflammation and limited lymphoid infiltration⁵¹. C-II upregulated mesenchymal reprogramming pathways, including EMT, MAPK4/6-associated signaling, and RUNX2 activity, characteristic of a fibrotic and immunologically quiescent stromal niche⁵². C-III, associated with EGFR mutations and a paucity of lymphoid infiltration, was characterized by heightened expression of cell cycle regulators, including G2/M checkpoint and mitotic spindle genes, suggestive of a hyperproliferative tumor-rich ecotype⁵³. By contrast, C-IV demonstrated a robust immune activation program encompassing IFN-γ response, cytokine–receptor interactions, and lymphocyte receptor pathways, consistent with its densely vascularized and immune-infiltrated microenvironment⁵⁴ (Fig. 4i).

Finally, immunotherapy-associated genomic and transcriptomic signatures further delineated the functional divergence across ecotypes (Table S4). C-IV exhibited enrichment of IFN-γ response (IFN-γ score), expanded immune gene signature (EIGS), and tertiary lymphoid structures (TLS) scores^54,55, highlighting a strong adaptive immune response. C-II displayed activation of a stromal EMT–TGFβ axis⁵⁶, consistent with desmoplastic remodeling and immune exclusion — features linked to resistance to ICB. Although both C-I and C-II harbor elevated tumor mutational burdens, their immunosuppressive features likely neutralize neoantigen-driven immunogenicity, thereby impairing the anti-tumor immune response^57,58 (Fig. 4j). These results highlight the therapeutic relevance of the CANVAS-defined spatial ecotypes in the context of immunotherapy.

CANVAS-guided multiscale spatial analysis reveals determinants of immunotherapy benefit

There is an unmet clinical need for reliable predictive biomarkers of immunotherapy response. Traditional biomarkers such as PD-L1 expression lack sufficient predictive power, while genomic and transcriptomic analyses obscure spatial heterogeneity by collapsing into bulk signatures⁵⁹. CANVAS addresses these limitations through an interpretable AI workflow by extracting habit-atinformed multiscale spatial features that reflect ecological diversity, interdomain coupling, and topographic organization, thereby enabling biologically grounded prediction of therapeutic response (Fig. 5a; see Methods).

Figure 5. Spatial analysis of H&E-derived tumor habitats identifies a signature predictive of immunotherapy outcomes.

a, Schematic overview of CANVAS for habitat inference and outcome prediction in immunotherapy cohorts. b, Integrated habitat co-occurrence and prognostic network in the discovery ICB cohort. Node size denotes habitat abundance, edge width indicates co-enrichment strength, edge color reflects Pearson correlation, and inner node color encodes PFS association. c, Heatmaps comparing pairwise habitat co-occupancy profiles between responders and non-responders based on Jaccard index analysis. d, Bubble plots showing associations between habitat-level spatial dispersion features and immunotherapy PFS. e, RRA-based ranking of samples using spatial features, with representative H&E images showing immune- and stromal-enriched patterns. f, Density plot of spatial transition entropy across samples in the discovery cohort. g, GNN-derived habitat connectivity network showing inter-habitat organization. h, Schematic of feature selection and model construction for PFS prediction. i, Forest plot of β coefficients for the 13 spatial habitat features retained in the Cox prognostic model. j, Time-dependent AUCs for PFS prediction at 6, 12, and 24 months in the discovery cohort. k–l, Kaplan–Meier curves for PFS stratified by the spatial signature in all patients from the discovery cohort (k) and in PD-L1-negative tumors (TPS < 1%; l). m, Time-dependent AUCs for PFS prediction at 6, 12, and 24 months in the CMB cohort. n, Kaplan–Meier curves for PFS stratified by the spatial signature in PD-L1-positive tumors (TPS ≥ 1%) from the discovery cohort. o, Kaplan–Meier curves for PFS stratified by the spatial signature in the CMB cohort. *P < 0.05; **P < 0.01.

To translate this spatial modeling paradigm into actionable analytics, we applied CANVAS to a discovery cohort of 149 advanced NSCLC patients treated with immunotherapy (Fig. 5a). For each patient, a total of 262 spatially grounded features were extracted (Table S5), which were divided into six groups: (1) Composition, capturing the relative abundance of each habitat within tumor sections (n = 10); (2) Diversity⁶⁰, representing the ecological complexity of habitat mixtures (n = 6); (3) Spatial dispersion⁶¹, capturing habitat-level aggregation and distribution patterns via multiscale spatial statistics (n = 90); (4) Interaction, modeling inter-habitat coupling based on pairwise proximity patterns (n = 100); (5) Distance, quantifying absolute spatial segregation between habitat pairs (n = 55); and (6) Transition⁶², an entropy-based, non-distance metric reflecting spatial intermixing (n = 1) (Table S5; see Methods).

Five habitats (H01, H04, H06, H07, H08) emerged as the most abundant spatial domains in NSCLC. Co-localization analysis revealed recurrent pairing of H03 with H08, consistent with coordinated lymphoid organization, and of H02 with H10, suggestive of immune–vascular interface zones (Fig. 5b, S8a). In contrast, H01 was spatially excluded from H04 and H08, reflecting immune-privileged tumor cores insulated from stromal or lymphoid infiltration (Fig. 5b, S8a). In the discovery cohort, enrichment of H01 and H06 was associated with inferior PFS, whereas the presence of H02, H07, and H08 delineated immune-active ecosystems predictive of clinical benefit from ICB (Fig. S8b). To further resolve habitat-level spatial co-occupancy associated with immunotherapy response, we stratified tumors by ICB outcome and calculated the Jaccard index, which quantifies the degree of shared occupancy between each pair of habitats across tumors. Compared with non-responders, responders showed increased spatial contiguity between immune-supportive habitats (H08) and resistance-associated niches (H01, H04, H06) (Fig. 5c), suggesting a more permissive spatial interface between immune-active and tumor-associated regions.

We next evaluated the predictive value of habitat-level spatial dispersion. High Ripley’s K and L values in H06 and H10 indicated tightly clustered neutrophil-rich and vascular regions, which were linked to shorter PFS and suggest the formation of inflammatory niches that may impair therapeutic response⁶³ (Fig. 5d, S8c). Conversely, higher F-function values in H01 and increased kernel density in H03 and H08 correlated with prolonged PFS, associated with favorable spatial organization of tumor and immune-supportive habitats⁶⁴ (Fig. 5d). To identify spatial phenotypes associated with ICB response, we applied RRA to prioritize candidate features across patients, revealing response-linked spatial specificity among prioritized features (Fig. 5e). Among responders, tumors exhibited high abundance of H03 and H08, elevated Shannon diversity, and increased kernel density in H08, consistent with lymphoid-enriched immune organization. In contrast, non-responders were characterized by dominant H01 and H04 habitats together with increased Fisher’s alpha, consistent with a heterogeneous but fibrotic and immune-restricted spatial phenotype⁶⁵ (Fig. 5e). These two phenotypes also exhibited opposing distributions along the STE ranking, supporting a link between spatial transition complexity and differential ICB responsiveness (Fig. 5f). We further assessed the inter-habitat spatial connectivity by conducting GNN-based connectivity analysis, which identified H01, H04, H06, and H08 as central interaction hubs in NSCLC (Fig. 5g). Collectively, these multiscale spatial features delineate a complex TME landscape that exerts a substantial impact on immunotherapy benefit.

Habitat-informed spatial signature predicts immunotherapy outcomes

Given the strong association between habitat features and immunotherapy response, we set out to develop a habitat-informed spatial signature that could predict immunotherapy outcomes. In the discovery cohort, we applied CANVAS to pre-treatment H&E images and computed 262 habitat-based spatial features for each patient. We sequentially applied two independent machine learning–based feature selection strategies following redundancy reduction, and then trained a multivariable Cox proportional hazards model for PFS (Fig. 5h; see Methods).

The final spatial signature included 13 habitat-based features (Fig. 5i; S8d). Among these, protective features (β < 0) primarily reflect immune-activated TME phenotypes. Representative features included co-localization of H08 and H03, suggesting coordinated T- and B-cell organization and TLS-like structures, together with H01–H03 spatial adjacency and elevated habitat richness, consistent with structured immune ecosystems associated with favorable outcomes^55,66 (Fig. 5i). By contrast, risk features (β > 0) captured TME architectural mechanisms of resistance to ICBs. High frequency of H06 was associated with accumulation of neutrophil-rich habitats, consistent with N2-like TAN features⁶⁷. H04–H09 interactions indicated fibrotic and immunosuppressive spatial coupling^36,68, whereas frequent H05–H03 contact suggested altered humoral immune organization involving B cells and plasma cells⁶⁹. Close spatial association between H02 and H09, together with elevated pair correlation in H02, was consistent with macrophage-rich immunosuppressive niches and M2-like polarization^70,71 (Fig. 5i).

In the discovery cohort, the spatial signature showed strong performance for predicting PFS after immunotherapy, with time-dependent AUCs of 0.75, 0.73, and 0.71 for 6-, 12-, and 24-month PFS, respectively (Fig. 5j). Notably, the spatial signature outperformed tumor PD-L1 expression (24-month AUC, 0.54) and the enrichment of individual spatial habitats, distinguished ICB response groups, and significantly stratified immunotherapy-treated patients by PFS in the discovery cohort (HR = 2.42 [1.55, 3.55], P < 0.001; Fig. 5k, S8e–f). This stratification remained significant across patient subgroups defined by PD-L1 expression (TPS < 1% and TPS ≥ 1%; Fig. 5l and 5n). The spatial signature also identified subgroups with immunotherapy-associated outcomes within EGFR- and KRAS-mutant tumors, molecular subtypes traditionally considered refractory to ICB^72,73 (Fig. S8g–h). The spatial signature remained predictive across therapeutic modalities, including ICB monotherapy and combination ICB–chemotherapy regimens (Fig. S8i–j). Importantly, the CANVAS-derived spatial signature outperformed established immunotherapy biomarkers (TMB, PD-L1, and TLS), further improved prediction of PFS when integrated with them (P < 0.05; Fig. S8k), and remained an independent predictor of PFS in multivariable models (P = 0.002; Fig. S8l).

Finally, we applied CANVAS to an external validation cohort of 40 immunotherapy-treated patients whose pre-treatment H&E images and clinical outcomes were obtained from the CMB Project. The derived spatial signature retained strong predictive performance, with AUCs of 0.79, 0.79, and 0.75 for 6-, 12-, and 24-month PFS, respectively (Fig. 5m). It also significantly stratified patients by PFS (HR = 4.56 [1.44–14.39], P = 0.01; Fig. 5o), underscoring its potential as a clinically relevant, spatially informed biomarker of immunotherapy benefit.

DISCUSSION

In this study, we developed CANVAS, a computational approach that infers tumor ecological habitats from standard H&E histopathology. Through spatial proteomic profiling of over 18 million cells from 457 patients, we identified 10 cellular neighborhoods (CNs) that capture multicellular architectures in the lung TME. CANVAS infers these CN-anchored spatial structures directly from whole-slide H&E images, enabling histology-native tumor microenvironment profiling. We applied CANVAS to H&E slides from over 5,000 patients and demonstrated its capability for population-scale prognostic modeling, molecular subtype classification, and immunotherapy response prediction. These findings establish AI-driven integration of spatial proteomics and histopathology as a scalable approach for defining spatial ecotypes that inform precision cancer therapy.

We resolved lineage-specific ecological programs in NSCLC, with LUAD progression linked to coordinated B cell and cytotoxic T cell infiltration, whereas LUSC keratinization was associated with CAF-, M2 macrophage-, and neutrophil-rich remodeling accompanied by lymphoid depletion, together defining distinct modes of TME reprogramming. Conserved tumor-centric triplet motifs across subtypes further suggested that recurrent higher-order spatial interactions provide a regulatory scaffold for tumor ecological evolution.

To define the interscale logic of spatial architectures, we identified 10 CNs that integrate local cellular programs and topological features into spatially coherent microdomains. Unlike prior studies based on small TMA cores prone to sampling bias, we analyzed both TMAs and whole-slide tissue sections from large patient cohorts, enabling robust CN construction that captures intratumoral heterogeneity while preserving intertumoral diversity. We show that CN06, an immunosuppressive niche enriched in neutrophils, was associated with inferior survival and resistance to anti-PD-1 immunotherapy, supporting TAN-targeted strategies such as CXCR1/2 inhibition with SX-682, which is under clinical evaluation in NSCLC and colorectal cancer⁷⁴. We also identified CN07, a tumor–immune interface niche enriched in proliferative tumor cells, activated cytotoxic T cells, and CAFs, together with a CAF-mediated tumor–T cell barrier score associated with worse outcomes in NSCLC. This supports stromal-targeting approaches such as setanaxib⁷⁵, a NOX1/4 inhibitor being evaluated in combination with pembrolizumab for advanced head and neck squamous cell carcinoma.

At finer resolution, we further identified a prognostically relevant niche in which LAG3⁺VISTA⁺ CD4⁺ T helper cells co-localize with HIF1A⁺ neutrophils and Ki67⁺ tumor cells, consistent with a locally exhausted and hypoxia-adapted ecosystem that couples immune suppression with aggressive tumor growth^76,77. In NSCLC, LAG-3 restrains T-cell activation and promotes immune escape, and therapeutic antibodies targeting this pathway, exemplified by relatlimab, are already being evaluated in combination with PD-1 blockade⁷⁸. Together, these multiscale spatial niches delineate complementary axes of immunotherapy resistance and provide a rationale for combinatorial treatment strategies that jointly target stromal insulation, neutrophil-mediated immunosuppression, and checkpoint-driven T-cell exhaustion.

Although spatial proteomics offers valuable insights into the TME, its clinical translation faces significant challenges including high cost, complexity, and limited scalability. CANVAS captures the relationship between spatially resolved cellular neighborhoods and morphological features derived from H&E through a foundation model pretrained on millions of pathology images⁷⁹. This is distinct from previous studies that attempted to infer the expression of individual proteins from H&E images. One study applied a generative adversarial network to generate virtual multiplex images of 11 proteins mostly consisting of cell-lineage markers²². Another recent study used a segmentation-based network to infer the status (high vs. low expression) for 21 proteins²¹. However, predicting protein expression from H&E is strongly marker-dependent and sensitive to staining, acquisition, and assay variability, leading to uneven and often modest performance across markers. By contrast, our study aims to infer tumor ecological habitats – spatial structures with biological interpretation – from H&E at a much higher accuracy exceeding 0.83. This enables robust discovery of spatially informed biomarkers in large-scale patient cohorts.

Immune checkpoint blockade (ICB) is a standard treatment for advanced lung cancer, but only a subset of patients derives durable benefit, and robust predictive biomarkers remain lacking. We applied the CANVAS to two independent ICB-treated cohorts, and found that spatial features beyond simple proximity, including intercellular dispersion, diversity, and topological transitions, better distinguished responders from non-responders. Our results show that responsive tumors exhibited B and T cell co-localization and compartmentalized TLS-like immune structures, whereas resistant tumors showed stromal insulation and neutrophil enrichment consistent with previous studies^80–82. In addition, we discovered new spatial features, including habitat richness and spatial entropy, which were also associated with ICB response. By integrating these habitat-level patterns, we derived an interpretable spatial signature that outperformed existing biomarkers in predicting ICB outcome.

In conclusion, our study provides a comprehensive characterization of the single-cell spatial landscape of the lung tumor microenvironment, and the computational CANVAS approach provides a biologically informed and clinically scalable platform for histology-native spatial tumor profiling to enhance precision oncology.

Limitations of the study

Our study has several limitations that should be considered when interpreting the results and that point toward future directions. First, CANVAS was trained on paired spatial proteomics and histopathology data in lung cancer and, although validated across multiple cancer types, additional training on more diverse datasets will be necessary to truly generalize the approach in a broader pan-cancer setting. Second, the 10-CN taxonomy is derived from unsupervised clustering of multiplexed imaging data, and while it has been validated through multiple layers, alternative solutions by integrating with additional assays such as spatial transcriptomics could reveal more granular and complementary aspects of TME organization^83,84. Third, some of the biological findings such as the association between neutrophil-rich niche and unfavorable outcomes are broadly consistent with established TME patterns. While we view this concordance as supporting the validity of the habitat framework, the more specific mechanistic interpretations (e.g., niche-restricted hypoxia–exhaustion ecologies) remain correlative. Functional validation of these spatial ecological hypotheses in experimental models will be an important next step. Finally, the immunotherapy response prediction model was developed in a discovery cohort of 149 patients and external validation was performed in a modest cohort of 40 patients. These results should be considered hypothesis-generating, and prospective validation in larger cohorts is required before clinical implementation.

RESOURCE AVAILABILITY

Lead contact

Further information and requests for resources and reagents should be directed to the lead contact, Ruijiang Li (rli2@stanford.edu).

Materials availability

This study did not generate new, unique reagents.

Data and code availability

Publicly available histopathology, multi-omics and clinical datasets were obtained from multiple sources. TCGA histology slides and clinical annotations were downloaded from the National Cancer Institute Genomic Data Commons Portal (https://portal.gdc.cancer.gov) and cBioPortal (https://www.cbioportal.org). Whole-slide images from the National Lung Screening Trial (NLST) cohort were acquired via The Cancer Imaging Archive (TCIA; https://wiki.cancerimagingarchive.net). PLCO datasets were accessed through the Cancer Data Access System (CDAS; https://cdas.cancer.gov/learn/plco) following appropriate data request approvals. The CODEX data and matched H&E images generated in this study are deposited in Zenodo (https://zenodo.org/records/20263843). The CANVAS pipeline, software, and downstream analysis scripts are publicly available at GitHub (https://github.com/lilabstanford/CANVAS).

STAR METHODS

EXPERIMENTAL MODEL AND STUDY PARTICIPANT DETAILS

Human specimens and study cohorts

This study integrated seven study cohorts to develop, validate, and clinically evaluate the CANVAS framework. A total of 457 lung tumor specimens were profiled by CODEX using a 41-plex antibody panel across three cohorts: tissue microarray cohort 1 (TMA1, n = 337; C1), tissue microarray cohort 2 (TMA2, n = 110; C2), and a whole-slide imaging cohort (WSI, n = 10; C3) (Table S2). TMA1 samples were profiled by CODEX alone, whereas TMA2 and WSI tumors underwent paired CODEX profiling and matched H&E staining. Collectively, these cohorts were used to define cellular neighborhoods from CODEX data. The paired WSI cohort was used to train and validate the H&E-based habitat inference model, which was further evaluated in the independent paired TMA2 cohort. For downstream clinical evaluation using H&E alone, we analyzed three lung cancer cohorts, TCGA (n = 945), PLCO (n = 382), and NLST (n = 366), comprising 1,693 patients (C4) in total (Fig. 4c), to assess associations between predicted tumor habitats, patient prognosis, and molecular subtypes. We further extended the analysis to a pan-cancer TCGA cohort spanning eight cancer types (n = 3,724; C5) to evaluate cross-cancer generalizability. Finally, two independent immunotherapy-treated cohorts were used for response modeling: an internal Stanford ICB cohort (n = 149; C6) and a lung cancer cohort from the Cancer Moonshot Biobank (n = 40; C7). Stanford specimens were analyzed in accordance with applicable data-use requirements, whereas TMA2 was obtained from TissueArray. Publicly available or controlled-access cohorts, including TCGA, PLCO, NLST, and CMB, were accessed and analyzed in accordance with their respective data-use policies. In summary, CANVAS comprises three integral components: (1) discovery of biologically grounded CNs from CODEX data; (2) inference of CODEX-derived habitats from H&E slides; and (3) derivation of spatial habitat features for prognostic modeling and biological interpretation. A schematic overview of the study design, patient cohorts, and analytical workflow is provided in Fig. 1a.

METHOD DETAILS

Sample preparation and tissue sectioning

Formalin-fixed paraffin-embedded (FFPE) tumor specimens from patients with NSCLC were used for CODEX imaging. For the Stanford TMA1 cohort, tumor regions were identified by gross inspection and confirmed by H&E review. Representative tumor regions were sampled from archival donor blocks using tissue punches and arrayed into recipient TMA blocks. In addition, surgical FFPE tumor blocks from patients with NSCLC were sectioned as whole-slide specimens to construct the Stanford WSI cohort. For CODEX staining, 5-μm FFPE sections from TMAs and whole-slide specimens were mounted onto Fisherbrand Superfrost Plus microscope slides (Fisher Scientific) according to the PhenoCycler-Fusion User Guide v2.1.0 and processed for downstream staining.

CODEX staining, imaging, and cell segmentation

Spatial proteomic profiling was performed at the Cell Sciences Imaging Facility (CSIF) at Stanford University using the PhenoCycler-Fusion (PCF) 2.0 platform, formerly known as CODEX. The antibody panel included pre-conjugated CODEX antibodies obtained from Akoya Biosciences and custom-conjugated antibodies generated using the PhenoCycler Antibody Conjugation Kit (Akoya Biosciences) according to the manufacturer's protocol. Antibody staining specificity and conjugation performance were routinely validated by immunofluorescence and PhenoCycler-Fusion test runs. Staining conditions were optimized individually for each antibody. Antibodies used in this study are listed in Table S1. FFPE sections from TMAs and WSIs were stained using the PhenoCycler Staining Kit. Briefly, after overnight baking at 60°C, slides were deparaffinized using a standard histological procedure. Antigen retrieval was conducted in Tris-EDTA buffer (pH 9.0) in a pressure cooker at 120°C for 20 min under high pressure. After cooling to room temperature, slides were incubated in a photobleaching buffer for 45 min. After extensive washing in 1× PBS, slides were hydrated and incubated with the conjugated antibody cocktail overnight at 4°C in a humidity chamber, followed by post-staining fixation and washing steps as instructed in the manufacturer’s user manual. Tissue sections were stored in CODEX Storage Buffer at 4°C until imaging. Reporter and buffer reagents for PhenoCycler-Fusion imaging were prepared using PhenoCycler Assay Reagent and the 10X Buffer Kit for PhenoCycler-Fusion (Akoya Biosciences) according to the manufacturer’s instructions. High-plex immunofluorescence imaging was then performed on the PhenoCycler-Fusion 2.0 platform. Images were acquired in DAPI, AF488, Atto550, and AF647 channels and automatically processed into multiplexed image files for downstream analysis.

Multiplexed fluorescence images were acquired using the CODEX Instrument Manager at a resolution of 0.507 μm per pixel. Raw image stacks were processed through the CODEX Processor pipeline for cross-cycle image alignment, tile stitching, and autofluorescence subtraction. Processed CODEX images were reviewed in ImageJ (v2.3.0) and QuPath (v0.4.3), and high-quality images were selected for downstream analysis. Each TMA core was treated as a single region of interest (ROI), with H&E staining performed post-acquisition to enable histological validation. Processed image stacks subsequently underwent standardized preprocessing and quality control, including tissue detection, ROI curation, and single-cell segmentation on the DAPI channel⁸⁵. Nuclear expansion was performed to approximate wholecell boundaries. Marker expression was quantified as the mean fluorescence intensity within each segmented cell mask. Shape and intensity features were computed for all compartments, and probability maps were retained to support downstream quality control and filtering. In total, over 18 million cells were analyzed across all samples, including 1.42 million cells from the TMA cohorts and 16.64 million cells from the WSI cohort.

H&E staining and imaging

Following PhenoCycler imaging, flow cells were removed according to the manufacturer’s instructions. Slides were washed extensively with 1× PhenoCycler buffer to remove residual imaging reagents and then processed for post-CODEX H&E staining using a standard histopathology workflow. Briefly, sections were rinsed, stained with hematoxylin, counterstained with eosin, dehydrated, cleared, and coverslipped. H&E-stained sections were digitized using a Leica Aperio AT2 whole-slide scanner at 40× magnification.

Cell type and cell state annotation using CODEX data

Single-cell spatial proteomic data were generated from pixel-level quantification of marker intensities and processed using the Seurat⁸⁶ package for downstream analysis. To reduce signal variation across markers, centered log-ratio normalization was performed using NormalizeData, followed by feature scaling with ScaleData. Given the compact antibody panel, all markers were retained for downstream analysis. Dimensionality reduction was performed by principal component analysis using RunPCA, and sample-level batch effects were corrected with the R package Harmony using RunHarmony. UMAP embedding was generated from the Harmony-corrected space for visualization, and the same corrected space was used for graph-based clustering. Phenotypic clustering was performed by shared nearest-neighbor graph construction with FindNeighbors followed by Louvain community detection with FindClusters. Cluster-enriched markers identified using FindAllMarkers were used to assign major lineage identities. Cells with potential segmentation artifacts, ambiguous marker profiles, or clusters lacking clear lineage-defining marker expression were annotated as unknown and were handled separately from high-confidence annotated cell populations in subsequent analyses. CD45⁺CD3e⁻GZMB⁺ immune cells were annotated as NK-like cells based on the available marker panel and referred to as NK cells in downstream analyses. Detailed functional cell states were annotated by integrating highresolution clustering with targeted Gaussian mixture modeling⁸⁷ for selected marker-defined populations, using curated marker combinations derived from the literature. Given the finite marker space of the CODEX panel, a subset of detailed annotations was assigned based on marker-informed phenotypic patterns. For whole-slide CODEX datasets, a two-stage annotation strategy was used. A representative subset of cells from each sample (10%) was first annotated to generate a reference set, and anchor-based label transfer was then performed for the remaining cells using FindTransferAnchors and TransferData.

Pairwise cell-cell interaction and co-occurrence

To quantify pairwise cell–cell interactions, we applied a radius-based, permutation-derived spatial interaction framework in which all neighboring cells within 40 μm of each cell centroid were defined as the local microenvironment^88,89. Pairwise interactions between annotated cell phenotypes were assessed independently within each image by comparing the observed frequency of neighboring cell-type pairs with a permutation-based null distribution. To generate the null model, cell phenotypes were randomized 1,000 times while preserving the underlying spatial coordinates. Cell-type pairs enriched relative to the permuted background were interpreted as spatial attraction, whereas depleted pairs were interpreted as spatial avoidance. By evaluating interactions within fixed local neighborhoods, this analysis captures short-range spatial organization and immediate multicellular topology in the TME.

Cell-cell co-occurrence was quantified at the tissue-core level to assess whether pairs of cell phenotypes were jointly enriched across samples, independent of their physical proximity within individual tissues. This analysis reflects coordinated expansion or depletion of lineages at the tissue scale, revealing global immune or stromal equilibria not detectable from local interactions alone. Together, these two perspectives are synergistic: interaction maps the proximate rules of cellular positioning, while co-occurrence situates these rules within broader ecological states. Their integration provides a coherent spatial logic that reconciles microscale topologies with macroscale tissue organization.

Identification of spatial cellular neighborhoods

To characterize multicellular spatial niches within the tumor microenvironment, we applied spatial Latent Dirichlet Allocation (spatial-LDA)^89,90, a topic modeling framework adapted for spatially resolved cell phenotype data. For each index cell, a local neighborhood was defined as all cells within a 40 μm radius, a scale that consistently captured ~25 neighboring cells across cohorts and aligns with prior studies delineating spatial domains in tissue microenvironments^91,92. Each neighborhood was encoded as a local phenotype-abundance vector across annotated cell types, forming a bag-of-cells representation analogous to the bag-of-words model in natural language processing.

Spatial-LDA was applied to model local cell composition as mixtures of latent motifs enriched for specific cell types, with a Laplacian prior imposed to enforce spatial smoothness across adjacent neighborhoods. The resulting motif proportion profiles were subjected to K-means clustering to define CNs, which capture spatially coherent and biologically distinct multicellular ecosystems within the tissue microenvironment. To improve cross-cohort independence and cross-format transferability, CNs were derived by co-clustering each TMA cohort separately with the WSI cohort. To determine the optimal number of CNs, we evaluated candidate solutions (k = 5–20) using complementary validity and stability metrics, including the Silhouette coefficient, Davies–Bouldin index, and Adjusted Rand index calculated between adjacent K partitions.

Graph-based triplet motif analysis

Spatial cell-cell interaction graphs were constructed for each TMA core by connecting CODEX-derived single-cell centroids within a fixed local distance threshold. Each cell was assigned to one of predefined cell types based on single-cell annotations. Local graphs from individual cores were aggregated into a global cell-type interaction network, where nodes represented cell types and edge weights corresponded to pairwise interaction frequencies. To quantify the topological prominence of each cell type, we employed a graph neural network (GNN) approach, implemented using a graph convolutional network (GCN)⁹³. The model was initialized with node degrees and weighted edges, and consisted of two GCNConv layers with ReLU activation functions. It was optimized to minimize the mean squared error between predicted and observed node degrees. The resulting node embeddings (GCN scores) captured each cell type’s network centrality within the global interaction topology.

All possible three-cell-type combinations were exhaustively enumerated, yielding 455 candidate triplet motifs. For each triplet, three global scoring metrics were calculated: the total edge-weight sum among the corresponding cell types, the mean GCN score, and their product as a composite score. Rankings from these metrics were integrated using robust rank aggregation (RRA), and 24 high-confidence triplet motifs with FDR < 0.01 were retained for downstream analysis. At the core level, each retained motif was quantified based on the local joint spatial exposure of its constituent cell types, and motif abundance scores were used for subsequent association analyses.

Proximity density and barrier score quantification

Proximity density (PD) quantifies the normalized local interaction density between two specified cell types within a given CN domain, defined as the number of identified proximity interactions divided by the total abundance of those two cell types within the same CN⁸⁹. This metric reflects localized cell-type juxtaposition and captures the extent of cell-type co-engagement within spatially constrained tissue niches.

To quantify stromal-mediated immune exclusion, we adapted a barrier score from the topological tumor graph framework³⁵. CD8⁺ T cells, proliferating tumor cells, and FAP⁺ CAFs were represented in a spatial graph. For each cytotoxic T cell not directly adjacent to a tumor cell, shortest immune–tumor paths were identified, and each path was classified as barrier-positive if at least one CAF was present along the intervening path. The sample-level barrier score was defined as the fraction of barrier-positive shortest immune–tumor paths across all eligible cytotoxic T cells, with higher scores indicating stronger stromal insulation of tumor cells from cytotoxic T cell access.

H&E image preprocessing, tumor detection, and tissue and cell segmentation

All H&E whole-slide images were analyzed at a standardized resolution equivalent to 40× magnification, corresponding to 0.25 micrometers per pixel. Slides acquired at lower resolutions were digitally rescaled to this reference magnification to ensure uniformity across cohorts. Regions affected by technical artifacts, including pen markings, tissue folds, and image blur, were systematically excluded using automated color-based filtering algorithms⁹⁴. To account for staining variability across institutions, H&E images were normalized using a structure-preserving color normalization method (https://github.com/CielAl/torch-staintools). Tissue-containing areas were segmented using the CLAM pipeline⁹⁵, followed by extraction of non-overlapping patches of 224 by 224 pixels. Tumor area detection was performed using a vision transformer model initially pretrained on a pan-cancer histopathology dataset and subsequently fine-tuned for lung tumor detection^96,97. Tumor-region detection generated patch-level tumor probability scores, which were spatially aggregated to identify tumor-enriched regions for downstream analysis. Cellular detection and segmentation in all datasets were performed using CellViT⁹⁸, a transformer-based model optimized for precise nuclear segmentation in H&E-stained tissues. The resulting cell coordinate matrices provided the basis for multimodality co-registration and habitat inference.

To define tumor bulk and leading-edge regions on H&E-stained whole-slide images, patch-level tumor probability maps were spatially aggregated across fixed-radius local neighborhoods to capture regional tumor context. These neighborhood-level features were subjected to unsupervised spatial clustering, and the resulting clusters were annotated as tumor bulk, leading-edge, or other tissue compartments based on tumor probability enrichment and spatial adjacency. The annotated regions were then used for downstream habitat profiling and prognostic modeling.

Multi-modality image co-registration

To facilitate deep learning between CODEX and H&E images, we implemented a pipeline for high-precision single-cell co-registration. Specifically, the DAPI channel from CODEX was aligned with the hematoxylin channel of the corresponding H&E section using PALOM (PyPI: https://pypi.org/project/palom/; Zenodo DOI: 10.5281/zenodo.17039386). In this framework, the source coordinates were defined as the centroid positions of segmented single cells from CODEX images. Affine transformation parameters, including translation, rotation, scaling, and shear, were used to project CODEX-derived cell coordinates into H&E space. This spatial alignment established the correspondence between different image modalities, enabling direct transfer of CODEX-defined CN annotations onto H&E histopathology.

Formally, given a source coordinate ${\mathit{x}}_{src}={\left[x,y,1\right]}^T$, the corresponding destination coordinate in H&E space $x_{dst}$ was computed using an affine matrix $\mathit{A}\in {\mathbb{R}}^{2x3}$ as:

$${\mathit{x}}_{\mathit{\text{dst}}}=\mathit{A}·{\mathit{x}}_{\mathit{\text{src}}}$$

where:

$$\mathit{A}=\left[\begin{array}{lll}{a}_{11}& a_{12}& t_x\\ a_{21}& a_{22}& t_y\end{array}\right]$$

and $t_x,t_y$ represent translation, while $a_{\mathit{\text{ij}}}$ encode rotation, scaling, and shear components.

Annotations that mapped outside the spatial boundaries of the H&E images were excluded from further analysis. Registration quality was manually reviewed to confirm alignment of corresponding tissue structures across modalities. H&E-derived single-cell segmentations were assigned CN labels by reference to the nearest registered CODEX cell centroid, using a centroid-to-centroid distance threshold of 5 μm. Only cells successfully mapped to one of ten spatial habitats were retained for downstream modeling. This image co-registration pipeline established high-confidence ground-truth habitat labels to learn spatially encoded morphological patterns from H&E.

Habitat label transfer based on CN assignment

CNs were derived by co-clustering the WSI cohort with TMA1 for discovery and with TMA2 for independent validation. CN labels were transferred to co-registered H&E images as habitat annotations for downstream model training and evaluation. Patch-level training labels were derived from CODEX-based single-cell annotations that were spatially mapped onto the corresponding H&E tissue sections. Patches with evident technical artifacts, including tissue folds, blurred regions, staining artifacts, or image misregistration, were manually excluded during quality control. For each image patch used for model training, the number of cells assigned to each spatial habitat class was quantified. Patches containing fewer than five annotated cells were assigned to a background class to represent non-informative tissue regions. Among habitat-informative patches, only those with more than fifteen annotated cells and a dominant habitat class accounting for at least 60% of the local cellular composition were retained as high-confidence habitat-labeled patches. Each eligible patch was labeled according to its most abundant habitat class. This filtering strategy emphasized the selection of high-confidence and compositionally consistent regions to enhance the reliability of model supervision. A multimodal dataset of spatially co-registered CODEX/H&E image patches (224 × 224 pixels) was assembled for model development. Following quality control, WSI patches were assigned to one of ten CODEX-derived CN labels and split into training (>180K patches) and validation (>82K patches) sets.

CANVAS model architecture and training strategy

The CANVAS model was developed to classify H&E image patches (224 × 224 pixels) into one of ten habitat types, enabling spatial habitat inference across whole-slide images. The model’s backbone is based on MUSK⁷⁹, a foundation model pre-trained using unified masked modeling on over 50 million pathology images from 11,577 patients and one billion pathology-related text tokens. MUSK enables robust extraction of high-dimensional visual embeddings that capture complex morphological features across diverse tissue types and staining variations. Let $\mathit{h}\in {\mathbb{R}}^d$ denote the visual feature embedding extracted by the MUSK encoder. The classification head consists of three sequential transformations:

$$z_1=\text{ReLU}\left(W_{1}h+b_1\right),z_1\in {\mathbb{R}}^{256}$$

$$z_2=\text{Dropout}\left(\text{ReLU}\left(W_2{z}_1+b_2\right)\right),z_2\in {\mathbb{R}}^{128}$$

$$o=W_3{z}_2+b_3,o\in {\mathbb{R}}^{11}$$

where ${\mathit{W}}_{\mathit{i}}$ and ${\mathbf{b}}_{\mathit{i}}$ are learnable weights and biases of each layer, and $\mathit{o}$ represents the unnormalized logits corresponding to the ten habitat classes and a background class.

To address class imbalance, we combined weighted random sampling with focal loss⁹⁹. For sampling, class frequencies $f_{c_i}$ were computed across the training set, and each sample $i$ (label $c_i$) received a weight $\frac{1}{f_{c_i}}$. Samples were drawn proportionally to these weights to ensure balanced exposure across classes. In parallel, focal loss was used to reduce the contribution of easy examples and emphasize underrepresented or misclassified instances. Given the predicted probability $p_t$ for the true class, the focal loss is defined as:

$$\text{L}\left(p_t\right)=-{\alpha }_t{\left(1-p_t\right)}^{\gamma }\text{log}\left(p_t\right)$$

The classification objective was optimized using the focal loss function, where ${\alpha }_t$ served as a class-specific weighting factor to address class imbalance, and $\gamma$ modulated the contribution of easy-to-classify samples to focus learning on harder examples. To enhance model robustness against staining and morphological variability, real-time data augmentation was implemented during training. Augmentation procedures included random horizontal and vertical flipping, rotation, and perturbation of image brightness, contrast, saturation, and hue. All augmented images were subsequently converted to tensors and normalized using channel-wise means and standard deviations derived from the MUSK pretraining dataset. This preprocessing pipeline improved model generalization and mitigated overfitting, particularly for underrepresented habitat classes.

Model evaluation and performance benchmarking

The CANVAS model was trained and internally validated using the Stanford WSI cohort, which was randomly partitioned into 80% training and 20% validation sets at the sample level. TMA2 served as an independent external test set to evaluate model classification performance across histologic platforms. The model was trained using the Adam optimizer. The initial learning rate was set to 1×10⁻⁴, with exponential decay applied at a factor of 0.95. A batch size of 64 and dropout rate of 0.5 were used throughout training. The model was trained for 60 epochs using a two-stage fine-tuning approach. For the first 10 epochs, only the classification head was trained while the entire backbone remained frozen. Subsequently, for the remaining 50 epochs, the final two layers of the encoder were unfrozen and fine-tuned along with the classification head. Evaluation was conducted using patch-level classification accuracy and habitat-type–specific precision and recall, measured on a held-out validation set with no patient overlap. Model performance was further validated across multiple cohorts via whole-slide inference and spatial prediction of habitats.

To evaluate the predictive performance of CANVAS-based habitat classification models, we computed four standard metrics: overall accuracy, macro-averaged F1 score, and Cohen’s kappa coefficient.

Accuracy was calculated as the proportion of correctly predicted habitat labels among all predictions:

$$\text{Accuracy}=\frac{\text{Number}\text{of}\text{correct}\text{predictions}}{\text{Total}\text{number}\text{of}\text{predictions}}$$

F1 score was computed as the harmonic mean of precision and recall for each habitat class, and macro-averaged across all classes:

$$\text{F}1=2\times \frac{\text{Precision}\times \text{Recall}}{\text{Precision}+\text{Recall}}$$

$$\mathit{\text{Macro}}-F1=\frac{1}{K}\sum _{i=1}^{K}F{1}_i$$

Cohen’s kappa coefficient quantified inter-rater agreement between predicted and true labels beyond chance, defined as:

$$\kappa =\frac{p_o-p_e}{1-p_e}$$

where $p_o$ denotes the observed agreement and $p_e$ represents the expected agreement by random chance.

We benchmarked the performance of CANVAS against three baselines: two pathology foundation models (UNI¹⁰⁰ and Virchow¹⁰¹) and ResNet-50. All models were trained and evaluated under identical conditions with standardized preprocessing, patient- and site-stratified splits, and consistent label definitions. Predictive performance was quantified by Macro-F1 and Cohen’s kappa. Robustness of performance differences was assessed by 1,000 bootstrap resampling of the validation and testing cohorts.

Evaluation of resolution and stride overlap in habitat prediction

The impact of image resolution on model performance was assessed by systematically downsampling WSIs and TMA cores from the validation and test sets. Slides originally scanned at 40× magnification (0.25 μm per pixel) were downsampled to 20× (0.50 μm per pixel) and 10× (1.00 μm per pixel), after which all image tiles were formatted to 224 × 224 pixels to preserve the model input tensor dimensions. Habitat predictions were then generated using the trained CANVAS model, and performance was quantified using standard patch-level classification metrics.

To evaluate the effect of tiling strategies on CANVAS habitat predictions, we varied the stride to modulate overlap between adjacent tiles, thereby testing whether increased sampling density around tumor boundaries or immune-enriched regions influenced model outputs. The patch size was fixed at 224 × 224 pixels, and three stride conditions were tested: 224 pixels (non-overlapping tiles), 112 pixels (50% overlap), and 56 pixels (75% overlap). For each stride setting, patch datasets were generated separately for the test set, and habitat predictions were performed. Model performance was quantified using accuracy, and robustness was further assessed by nonparametric bootstrap resampling with 1,000 iterations across stride pairs (224 vs. 112 pixels; 112 vs. 56 pixels).

Prognostic modeling of habitats on H&E-stained images

To evaluate the prognostic significance of CANVAS-derived spatial habitats, we quantified the abundance of 10 habitats within the tumor bulk and leading-edge compartments of each sample across the TCGA, PLCO, and NLST cohorts. Samples with insufficient tumor-associated patch coverage, defined as fewer than 100 tumor-bulk patches, were excluded to ensure adequate spatial representation. Within each histologic subtype and cohort, habitat–outcome associations were evaluated separately using available clinical endpoints, including overall survival (OS), disease-specific survival (DSS), progression-free interval (PFI), and disease-free interval (DFI). For each habitat–compartment combination, univariate Cox proportional hazards models were fitted, and hazard ratios (HRs), 95% confidence intervals, and Wald P values were reported.

Spatial ecotypes identified using H&E-derived tumor habitats

To identify recurrent spatial phenotypes with prognostic relevance, we performed unsupervised consensus clustering of TCGA-LUAD samples based on CANVAS-derived habitat abundances in tumor bulk and leading-edge regions. Samples with limited spatial coverage of tissue compartments were excluded. A feature matrix comprising the distribution of 10 habitats across the two spatial compartments was z-score normalized and subjected to consensus clustering using the ConsensusClusterPlus R package¹⁰² with partitioning around medoids (PAM) and Canberra distance. The optimal number of clusters was determined based on cluster stability and concordance metrics. The resulting consensus-defined subtypes were subsequently linked to clinical variables and survival outcomes. Prognostic value was further assessed using multivariable Cox proportional hazards models adjusted for major clinical covariates, including age, sex, smoking status, and tumor stage. To further contextualize the identified spatial ecotypes, we compared them with transcriptomic TME subtypes assigned to the same samples³⁹.

Orthogonal validation of habitat prediction

We assembled orthogonal validation datasets comprising same-section CODEX–H&E specimens and paired Xenium–H&E specimens (Table S2). For Xenium data, preprocessing, clustering, and cell-level annotation were performed separately for each sample using a common Seurat workflow that included quality control, SCTransform normalization, dimensionality reduction, and sample-level clustering. CN labels were inferred at single-cell resolution using a cosine-similarity-based CN inference framework, in which cell-level neighborhood profiles from spatial omics data were mapped to the most similar CODEX-derived reference CN composition profiles.

Gene set enrichment and immunogenomic analysis

To characterize the tumor microenvironmental correlates of CANVAS-defined spatial habitats, we computed patient-level habitat abundances for each TCGA-NSCLC sample by aggregating patch-level predictions of H01–H10. Samples with fewer than 100 informative patches were excluded. These spatial features were integrated with three orthogonal TME inference frameworks: (1) ESTIMATE¹⁰³-derived stromal, immune, and tumor purity scores; (2) CIBERSORTx (LM22)^104,105 immune cell scores; and (3) xCell¹⁰⁶ enrichment profiles spanning 64 immune and stromal cell types. All data were harmonized using TCGA sample barcodes to enable multi-omic correlation analysis. To assess relationships with transcriptomic signatures, normalized TPM profiles from TCGA were filtered to retain protein-coding genes and analyzed using single-sample gene set variation analysis with the GSVA R package¹⁰⁷. GSVA was used to compute pathway-level enrichment scores from normalized transcriptomic data, leveraging curated gene sets from MSigDB, KEGG, Reactome, and other collections³⁹. Differential pathway activity across CANVAS-inferred spatial habitat groups was assessed using the limma¹⁰⁸ R package with empirical Bayes moderation and linear modeling.

Genomic and immunologic correlates, including tumor mutational burden, aneuploidy, microsatellite instability, and SNV-derived neoantigen load, were obtained from public databases. Transcript-based signatures, including interferon response, tertiary lymphoid structure, EMT-TGFβ signaling, and angiogenesis signatures, were compiled from the literature and calculated using ssGSEA as implemented in the GSVA R package. Associations between spatial subtypes and molecular features were assessed using covariate-adjusted analysis of variance, with age and sex included as clinical covariates, implemented using the Anova function from the car¹⁰⁹ R package.

Somatic mutation and copy number alteration association analysis

To investigate genomic correlates of CANVAS-defined spatial ecotypes, gene-level copy number alteration and somatic mutation data from the TCGA-LUAD cohort were analyzed. Gene-level copy number calls were obtained from TCGA/GDC processed data and encoded as deletion, neutral, or amplification events. Associations between gene-level copy number status and spatial ecotype membership were assessed using regression-based models. Effect sizes were summarized as odds ratios with corresponding 95% confidence intervals. To facilitate biological interpretation, candidate genes were further annotated according to established oncogenic and immunoregulatory pathways¹¹⁰.

Somatic mutation profiles were obtained from MC3 consensus calls and binarized at the gene level. For each retained gene, mutation frequencies were calculated within each spatial ecotype, and associations between mutation status and ecotype membership were assessed using Fisher’s exact tests. Recurrent ecotype-associated mutation patterns were visualized by heatmap after row-wise z-score normalization of mutation frequencies. Candidate genes were curated from established TCGA PanCancer Atlas driver gene resources¹¹¹.

Habitat-level spatial feature engineering and extraction

To biologically interpret CANVAS-defined spatial habitats, we computed a total of 262 quantitative features from H&E-stained NSCLC tissue sections. All features were derived from patch-level annotations and subsequently aggregated to the image level to enable cohort-wide comparative analyses. (1) Composition features (n = 10): patch-level CANVAS annotations were aggregated per image to quantify the frequency of each of the ten spatial habitats. For each sample, raw counts of habitat-labeled patches were computed and normalized by the total number of annotated patches within the image, yielding a ten-dimensional vector representing the relative abundance of each habitat. (2) Diversity features (n = 6): to capture ecological complexity within each image, we computed six diversity metrics using habitat label counts as species abundances, including richness, Shannon index, Simpson index, inverse Simpson index, Fisher’s alpha, and Pielou’s evenness. (3) Spatial dispersion features (n = 90): to characterize intra-habitat spatial organization, we converted habitat-specific coordinates into planar point patterns and derived nine classes of spatial summary features per habitat, including summaries from Ripley’s K and L functions, pair correlation, G, F, and J functions, together with the Clark–Evans index, quadrat-based dispersion statistics, and kernel density summaries. (4) Interaction features (n = 100): to evaluate inter-habitat spatial coupling, pairwise interaction scores were computed for each tissue image using a fixed-radius neighborhood framework. For each habitat pair, interaction scores were derived using a permutation-based z-score approach (1,000 iterations), comparing observed spatial adjacency frequencies with a randomized null distribution. This yielded 100 ordered habitat-pair interaction features corresponding to the full 10 × 10 interaction matrix. (5) Distance features (n = 55): pairwise nearest-neighbor Euclidean distances between habitat pairs were calculated for each tissue image and summarized as image-level spatial distance features. (6) Transition feature (n = 1): Spatial transition entropy (STE) was computed to quantify the degree of inter-habitat mixing within each image. For each tissue image, habitat transitions were defined based on patch-level k-nearest-neighbor relationships (k = 6), and a habitat transition matrix was constructed from all observed transitions between neighboring patches. STE was then calculated as the Shannon entropy of the resulting global transition probability distribution. Higher STE values indicate a more intermixed and heterogeneous spatial architecture, whereas lower STE values reflect a more compartmentalized tissue organization.

Feature selection and model development

To construct a robust model for predicting immunotherapy outcomes, we implemented a multi-stage computational pipeline encompassing feature curation, redundancy reduction, feature prioritization, and survival modeling. As an unsupervised preprocessing step, multicollinearity was first reduced by calculating the Spearman correlation matrix across all predictors. Features with absolute pairwise correlation > 0.95 were grouped by Louvain community detection using the igraph package¹¹², and one representative feature from each cluster was retained to generate a reduced, non-collinear feature set.

Robust candidate predictors were then prioritized using two complementary resampling-based approaches. First, we performed 100 repeated resampling iterations of LASSO-penalized Cox regression using the glmnet package¹¹³. In each iteration, patients were randomly divided into training and validation subsets, and the training subset was used for model fitting with internal 5-fold cross-validation to determine the optimal penalty parameter based on the concordance index. Features selected in more than 40% of iterations were retained. Second, we applied permutation-based random survival forest analysis using the randomForestSRC package. Across 1,000 bootstrap iterations, feature importance was quantified by permuting individual predictors and measuring the resulting change in prediction error for progression-free survival. Features showing consistently higher importance than permuted controls were prioritized as robust predictors.

Thirteen features jointly prioritized by the LASSO and random survival forest procedures were subsequently used to train a multivariable Cox proportional hazards model. Model performance was assessed over 100 iterations of random 75/25 train-validation splits. Within each iteration, model coefficients were estimated in the training subset and evaluated in the held-out validation subset using the C-index and time-dependent AUCs at 6, 12, and 24 months. For survival visualization, patients were stratified into risk groups based on the training-set median risk score, which was then applied to the corresponding validation subset. Prognostic separation was summarized using hazard ratios, log-rank P values, and 95% confidence intervals. Final model coefficients were visualized in a forest plot with annotation categories.

QUANTIFICATION AND STATISTICAL ANALYSIS

All statistical analyses were conducted using R (version 4.4.0) and Python (version 3.10) for data preprocessing, visualization, and hypothesis testing. Two-group comparisons were performed using two-sided Student’s t tests or Mann–Whitney U tests, as appropriate based on data distribution and variance structure. For comparisons involving more than two groups, one-way analysis of variance (ANOVA) or non-parametric Kruskal–Wallis tests were applied as appropriate. P values were adjusted for multiple comparisons using the Benjamini–Hochberg false discovery rate (FDR) procedure where applicable. Survival analysis was performed using Kaplan–Meier estimators with log-rank tests, and survival associations were further evaluated using univariate and multivariable Cox proportional hazards models implemented in the survival R package. For visualization of survival differences, patients were stratified using analysis-specific thresholds as described in the relevant sections. Continuous covariates included in regression models were standardized by z-score transformation before model fitting. Specific statistical methods are described in the corresponding Method sections. Unless otherwise specified, all tests were twosided, and P < 0.05 was considered statistically significant.

Supplementary Material

2

Table S1. Antibody panel used for CODEX spatial proteomics, related to STAR Methods.

3

Table S2. Clinical characteristics of patient cohorts used in this study, related to Figure 1 and STAR Methods.

4

Table S3. Cell phenotype definitions with marker annotations and major lineage categories, related to Figure 3.

5

Table S4. Differential distribution of immunogenomic signatures and immune checkpoint markers across spatial ecotypes in TCGA-LUAD, related to Figure 4.

6

Table S5. Spatial features and feature categories used for immunotherapy biomarker analysis in CANVAS, related to Figure 5 and STAR Methods.

Key resources table

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Antibodies
CD8-BX026-Atto 550	Akoya Biosciences	Cat# 4250012
Pan-Cytokeratin-BX019-Alexa Fluor™ 488	Akoya Biosciences	Cat# 4150020
CD3e-BX045-Alexa Fluor™ 647	Akoya Biosciences	Cat# 4550119
CD163-BX023-Atto 550	Cell Signaling Technology	Cat# 25121
CD20-BX007-Alexa Fluor™ 488	Akoya Biosciences	Cat# 4150018
CD4-BX003-Alexa Fluor™ 647	Akoya Biosciences	Cat# 4550112
FAP-BX052-Atto 550	Abcam	Cat# ab271976
CD138-BX016-Alexa Fluor™ 488	Abcam	Clone# EPR6454
CD11 c-BX024-Alexa Fluor™ 647	Akoya Biosciences	Cat# 4550114
CD66b-BX020-Atto 550	R&D Systems	Cat# MAB4246
aSMA-BX013-Alexa Fluor™ 488	Abcam	Cat# ab5694
CD68-BX015-Alexa Fluor™ 647	Akoya Biosciences	Cat# 4550113
Ki67-BX047-Atto 550	Akoya Biosciences	Cat# 4250019
CD31-BX001-Alexa Fluor™ 488	Akoya Biosciences	Cat# 4150017
Collagen IV-BX042-Alexa Fluor™ 647	Akoya Biosciences	Cat# 4550122
Granzyme B-BX041-Atto 550	Akoya Biosciences	Cat# 4250055
MMP9-BX028-Alexa Fluor™ 488	BioLegend	Cat# 819701
PD-1-BX046-Alexa Fluor™ 647	Akoya Biosciences	Cat# 4550038
CD44-BX005-Atto 550	Akoya Biosciences	Cat# 4450041
PD-L1-BX067-Alexa Fluor™ 647	Akoya Biosciences	Cat# 4550128
E-cadherin-BX014-Atto 550	Akoya Biosciences	Cat# 4250021
LAG3-BX055-Alexa Fluor™ 647	Akoya Biosciences	Cat# 4550058
Mac2/Galectin-3-BX035-Atto 550	Akoya Biosciences	Cat# 4450034
FOXP3-BX031-Alexa Fluor™ 647	Akoya Biosciences	Cat# 4550071
CD14-BX037-Atto 550	Akoya Biosciences	Cat# 4450047
EpCAM-BX091-Alexa Fluor™ 647	Akoya Biosciences	Cat# 4550088
CD21-BX032-Atto 550	Akoya Biosciences	Cat# 4450027
CD45-BX021-Alexa Fluor™ 647	Akoya Biosciences	Cat# 4550121
MPO-BX098-Atto 550	Akoya Biosciences	Cat# 4250083
TCF-1-BX061-Alexa Fluor™ 647	Akoya Biosciences	Cat# 4550068
GATA3-BX049-Atto 550	Akoya Biosciences	Cat# 4250085
ICOS-BX054-Alexa Fluor™ 647	Akoya Biosciences	Cat# 4550117
HLA-A-BX029-Atto 550	Akoya Biosciences	Cat# 4250100
Bcl-2-BX085-Alexa Fluor™ 647	Akoya Biosciences	Cat# 4550089
HLA-E-BX034-Atto 550	Akoya Biosciences	Cat# 4250065
CD45RO-BX017-Alexa Fluor™ 647	Akoya Biosciences	Cat# 4550127
VISTA-BX040-Atto 550	Akoya Biosciences	Cat# 4250063
HIF1A-BX062-Alexa Fluor™ 647	Akoya Biosciences	Cat# 4550069
CD39-BX099-Atto 550	Akoya Biosciences	Cat# 4250076
CD40-BX010-Alexa Fluor™ 647	Akoya Biosciences	Cat# 4550064
HLA-DR-BX033-Alexa Fluor™ 647	Akoya Biosciences	Cat# 4550118
Biological samples
Formalin-fixed paraffin-embedded (FFPE) human lung tumor samples	This Manuscript	N/A
Chemicals, peptides, and recombinant proteins
CODEX barcode BX-023 (used for CD163 custom conjugation)	Akoya Biosciences	Cat# 5250003
CODEX barcode BX-052 (used for FAP custom conjugation)	Akoya Biosciences	Cat# 5250012
CODEX barcode BX-016 (used for CD138 custom conjugation)	Akoya Biosciences	Cat# 5150001
CODEX barcode BX-020 (used for CD66b custom conjugation)	Akoya Biosciences	Cat# 5250002
CODEX barcode BX-013 (used for aSMA custom conjugation)	Akoya Biosciences	Cat# 5450017
CODEX barcode BX-028 (used for MMP9 custom conjugation)	Akoya Biosciences	Cat# 5150005
Deposited data
CODEX and H&E data	This paper	https://zenodo.org/records/20263843
Software and algorithms
R 4.4.0	R Core Team	https://www.r-project.org
Python 3.10.14	Python Software Foundation	https://www.python.org/downloads/release/python-31014/
CODEX Instrument Manager	Akoya Biosciences	https://help.codex.bio/codex/cim/
CODEX Processor	Akoya Biosciences	https://help.codex.bio/codex/processor/
ImageJ v2.3.0	ImageJ	https://imagej.net/ij/
QuPath v0.4.3	QuPath	https://qupath.github.io/
Seurat 5.1.0	Satija Lab	https://satijalab.org/seurat
Harmony 1.2.1	CRAN	https://cran.r-project.org/web/packages/harmony/index.html
mclust 6.1.1	CRAN	https://cran.r-project.org/package=mclust
ConsensusClusterPlus 1.68.0	Bioconductor	https://bioconductor.org/packages/release/bioc/html/ConsensusClusterPlus.html
GSVA 1.52.3	Bioconductor	https://www.bioconductor.org/packages/release/bioc/html/GSVA.html
survival 3.6–4	CRAN	https://cran.r-project.org/package=survival
survminer 0.5.0	CRAN	https://cran.r-project.org/package=survminer
randomForestSRC 3.3.2	CRAN	https://cran.r-project.org/package=randomForestSRC
CANVAS code and downstream pipeline	This paper	https://github.com/lilab-stanford/CANVAS
Other
PhenoCycler Antibody Conjugation Kit	Akoya Biosciences	Cat# 7000009
PhenoCycler Assay Reagent	Akoya Biosciences	Cat# 7000002
10X Buffer Kit for PhenoCycler-Fusion	Akoya Biosciences	Cat# 7000019
CODEX Storage Buffer	Akoya Biosciences	Cat# 232107

Highlights.

• Single-cell spatial proteomics atlas identifies cellular neighborhoods

• CANVAS predicts cellular neighborhoods from standard histology

• CANVAS enables virtual spatial profiling at the population level

• CANVAS identifies H&E-based spatial signature of immunotherapeutic outcome