Image-based inference of tumor cell trajectories enables large-scale cancer progression analysis

Abstract

Current approaches to estimating cell trajectories, tumor progression dynamics, and cell population diversity of tumor microenvironment often depend on single-cell RNA sequencing, which is costly and resource intensive. To address this limitation, we developed an artificial intelligence (AI) model that leverages cell morphology features and histological spatial organization to classify tumor cell differentiation status, infer cell dynamic trajectories, and quantify tumor progression from hematoxylin and eosin (H&E)–stained whole-slide images. In three independent lung adenocarcinoma cohorts, our AI-based model accurately predicted cell differential status and provided quantifiable measures of tumor progression that were prognostic of patient survival. Spatial transcriptomic integrative analyses revealed cell components and gene signatures enriched in different cell differentiation statuses. Bulk transcriptomic analyses revealed that fast-progressing tumors exhibit up-regulated cell cycle pathways, while slow-progressing tumors retain characteristics of normal lung epithelium. This cost-effective method enables large-scale analysis of tumor progression dynamics using routinely collected pathology slides and provides insights into intratumor heterogeneity.

An AI approach turns pathological slides into dynamic maps of tumor evolution, enabling large-scale cancer progression analysis.

INTRODUCTION

Lung cancer has been the deadliest cancer type for several decades. In 2024, an estimated 234,580 cases of lung and bronchus cancer are expected to be diagnosed in the United States, with approximately 340 deaths per day attributed to lung cancer. Despite advances in diagnosis and treatment, the prognosis for patients with lung cancer remains poor, with a 5-year survival rate of less than 25% (1). A key challenge in improving outcomes is tumor heterogeneity, composed of spatially and functionally distinct populations of cells that drive drug resistance, metastatic potential, disease progression, and, ultimately, treatment failure (2, 3). Understanding the dynamic progression of tumors is a cornerstone of advancing cancer research and improving patient outcomes.

Estimating how tumors evolve over time is critical for identifying therapeutic targets, predicting disease trajectory, and personalizing treatment strategies. Cutting-edge single-cell RNA sequencing (RNA-seq) offers powerful solutions for modeling the dynamic process of cell differentiation by ordering cells along a developmental or evolutionary trajectory based on their gene expression profiles (4, 5). This approach has been applied to uncover critical insights into tumor progression and evolution, offering a deeper understanding of how cellular differentiation influences tumor behavior (6–9). However, these methods come with notable limitations: They are costly, require specialized equipment and expertise, and are not widely accessible, especially in resource-limited settings. To address this gap, we leverage histopathological images to investigate the spatial and temporal dynamics through the transition of cell differentiation status, which is a critical aspect of histopathological diagnosis and plays a central role in monitoring tumor progression and guiding treatment strategies in clinical practice (10). While previous studies have explored predicting cell differentiation status from pathology images across various cancer types (11–14), none have addressed whether pathology images can characterize tumor spatial evolution trajectories or how these trajectories are associated with tumor progression and clinical outcomes.

In this work, we developed a computing framework to quantify tumor progression from the hematoxylin and eosin (H&E) stained whole-slide tumor images (WSIs), which have been widely used in lung cancer diagnosis, providing critical insights into tumor grades, cancer subtype, and the tumor microenvironment (15–17). Recent advancements in artificial intelligence (AI) have opened avenues for extracting cell quantitative information from histopathological images (18, 19). Those AI-based computing approaches can now identify patterns and features previously imperceptible to the human eyes, enabling the analysis of tumor architecture, cellular heterogeneity, and spatial relationships between tumor and stromal components, offering a deeper understanding of lung cancer biology (16, 20–22). Powered by AI, we trained a deep learning model to predict cell differentiation status. Then, the image-derived features encoding cell morphological phenotypes were extracted from the trained model to estimate cell pseudotime. Last, we derived an objective tumor progression score by integrating cell differentiation status, cell pathological spatial organization, and pseudotime. We validated the proposed method’s capability through patient stratification using three independent lung adenocarcinoma (ADC) cohorts: The Cancer Genome Atlas Lung Adenocarcinoma (TCGA-LUAD), the National Lung Screening Trial (NLST-ADC), and the University of Texas Special Program of Research Excellence (SPORE) in Lung Cancer. The stratification further guided the exploration of related biological pathways and genes associated with tumor progression by integrating matched bulk RNA-seq data with image-derived differentiation metrics in the LUAD and SPORE cohorts, providing plausible explanations for the observed associations between these metrics and clinical outcomes. In addition, we also integrate our methods with spatial transcriptomic data to explore the gene signatures and immune cell components among different cell differentiation statuses (Fig. 1).

Fig. 1. Overview of tumor cell dynamic framework.

RESULTS

Development of the cell differentiation status classification model

Cell differentiation status is a critical aspect of histopathological diagnosis and plays a central role in monitoring tumor progression and guiding treatment strategies in clinical practice (10). Tumor grades reflect the level of cellular differentiation to normal cells and are typically categorized into three levels: low-grade tumors: well-differentiated (G1), intermediate: moderately differentiated (G2), and high-grade tumors: poorly differentiated (G3) and undifferentiated (G4) (23). Low-grade tumors closely resemble native lung tissue, tend to grow and spread slowly, and maintain some degree of structured organization. In contrast, high-grade tumors are more aggressive, exhibiting substantial loss of structured organization and little to no similarity to normal lung tissue (24).

A critical aspect of cell status classification is developing a robust five-class prediction model, encompassing the four established tumor grade categories (G1 to G4) and an additional “other” class that includes normal, necrosis, and other pathological conditions. To achieve this specific goal, a deep learning-based model was constructed using the high-quality, annotated dataset from the NLST (25), which includes 381 WSIs of lung ADC (Fig. 2A). We fine-tuned the Phikon histopathology foundation model (26) using the image patches randomly extracted from the annotated regions of interest (ROIs) on WSIs (Fig. 2B). Details on the dataset and the model training process are provided in Materials and Methods.

Fig. 2. Illustration of using pathology image deep learning model to predict cell differential status.

(A) Flowchart of predicting cell differential status from digital pathology images. Image patches extracted from annotated ROIs of H&E images are used to train a deep learning model for predicting cell differential status. (B) Detailed network structure of the proposed deep learning model. Image patches of size 224 × 224 serve as the inputs of our fine-tuned model. The model is built on a pretrained Phikon model followed by a feed-forward layer with 128 nodes and a layer of 5 nodes representing the five classes. A softmax function is applied to compute the probabilities across the five classes, determining the predicted class. (C) Summary of the number of training and testing ROIs (left) and patches (right) used in the model. (D) Confusion matrix of the patch-level performance using the independent testing patches of low (G1), intermediate (G2), and high (G3 and G4) grades and other pathological conditions. (E) AUROC of slide-level performance of the NLST dataset. (F) Confusion matrix of the slide-level prediction performance of the NLST dataset.

Validation of cell differentiation status prediction accuracy

A total of 440 ROIs were annotated in the NLST dataset and subsequently divided into training (n = 395) and testing (n = 45) subgroups. The model was trained on 1976 patches randomly extracted from training ROIs, and the model’s performance was evaluated using 2250 patches from independent testing ROIs (Fig. 2C). Because of the limited number of G4 tumors in our model, we classify G1 as low grade, G2 as intermediate grade, and G3 and G4 as high-grade tumors following the guidance of the National Cancer Institute. We compared the performance of five pretrained foundation models (Table 1), and the Phikon model outperformed the Phikon-V2 (27) and other convolutional neural network–based models (28). The result shows the patch-level classification weighted accuracy of 0.61 for all classes (Fig. 2D and table S1) of the Phikon model. To evaluate our model’s performance on WSIs, we aggregated the grades of all extracted tumor patches and assigned each slide the rounded average grade. On a cohort of 148 WSIs, the model achieved a weighted accuracy of 0.70, with an area under the receiver operating characteristic curve (AUROC) of 0.89 for distinguishing low-grade from intermediate/high-grade tumors and an AUROC of 0.83 for distinguishing low/intermediate-grade from high-grade tumors (Fig. 2, E and F). Most discrepancies arose from the limited number of training samples of undifferentiated tumors (G4) and the misclassification of moderately (G2) and poorly (G3) differentiated tumors. However, the classification of G2 and G3 is inherently challenging due to the subtle boundaries between these two categories. In addition, the manual annotation of ROIs may also introduce variability stemming from subjective decisions made by pathologists.

Table 1. Performance comparison of different deep learning backbone models.

Model	Patch size	Feature size	Weighted accuracy	Cohen’s K
Phikon (26)	224	768	0.61	0.468
Phikon-V2 (27)	224	1024	0.36	0.00
Resnet-50 (28)	500	2048	0.35	0.050
Resnet-101 (28)	500	2048	0.36	0.088
Resnet-152 (28)	500	2048	0.41	0.088

Pseudotime analysis of WSIs using cell differentiation status model

The pretrained patch-level tumor grading model enables application at the WSI level for lung ADC analysis. An example of WSI processing is shown in Fig. 3A. The pretrained model rapidly scans the masked tissue region pixel by pixel, providing spatialized results that display the distribution of differential status across the WSI. During this scanning process, image-derived features were extracted and saved for downstream quantitative analyses. Analogous to gene expression values in single-cell RNA-seq, image-derived features capture cell morphological phenotypes that can be used to estimate pseudotime of image patches. We used the extracted image features for pseudotime analysis to spatially investigate tumor cell differentiation status and their trajectories (Fig. 3A; more details in Materials and Methods). Image-based pseudotime analysis sorts image patches according to cell morphological heterogeneity, enabling the exploration of tumor evolution spatially.

Fig. 3. Cell differential status and pseudotime on whole-slide H&E images.

(A) Flowchart of cell differential status prediction and pseudotime inference at the WSI level. To predict WSI-level cell differential, the trained deep learning model scans each image patch extracted across the tissue region. At the same time, patch features used for prediction are extracted for the downstream pseudotime analysis. Patches were clustered by the Leiden algorithm using the extracted features, and pseudo time was estimated according to the Leiden clusters using the diffusion pseudotime algorithm, more details in Materials and Methods. UMAP, uniform manifold approximation and projection. (B) Results of G1 tumor (top) and G4 tumor (bottom). Both G1 and G4 identified the tumor ROI region, and the pseudotime results of G4 identified the necrosis region.

We conducted a detailed analysis of the predicted cell differentiation status and annotated ROIs confirmed by our expert pathologist. Figure 3B illustrates two whole-slide H&E image examples: a well-differentiated (G1) tumor and an undifferentiated (G4) tumor. In both cases, the predicted status accurately matched the annotated ROIs. Pseudotime analysis further revealed a later time point in the tumor regions compared to normal regions, reflecting a more advanced stage of tumor cell differentiation and progression, consistent with the status predictions. In the G4 example, although both the “normal” and “necrosis” regions were classified as other, the pseudotime analysis still effectively distinguished the necrotic region, which exhibited the latest cell stage compared to normal and tumor regions. These findings demonstrate that our model successfully aligns predicted status with annotated ROIs. In addition, the image feature–based pseudotime analysis provides valuable insights into cell trajectories, further enhancing our understanding of tumor progression.

Prognostic image biomarker integrating spatiality and pseudotime of tumor

The fine-tuned cell differentiation status classification model enables objective quantification of tumor progression using an H&E image. To further analyze tumor dynamics, we propose a general fitness score to quantify the level of tumor progression (Fig. 4A). This fitness score is defined as the product of two components: the progression speed (S), which reflects the biological aggressiveness of tumor growth based on pseudotime and the spatial organization of image patches, and the Shannon diversity (I), which represents tumor heterogeneity based on cell morphological diversity. Additional details on the mathematical model and calculations can be found in Materials and Methods.

Fig. 4. Tumor progression fitness model on WSI images and survival analysis.

(A) Illustration of tumor progression quantification integrating spatial organization on WSI; more details in Materials and Methods. (B to J) Results of Kaplan-Meier log-rank test and univariate Cox-PH regression analysis of NLST, SPORE, and LUAD datasets. Patients were stratified into slow (blue) and fast (orange) groups. HR, hazard ratio; CI, confidence interval. (K to M) Correlation plot between the tumor progression speed and Shannon index of three datasets. Patients were stratified into slow (blue), “moderate” (green), and fast (orange) groups by the median value of both speed and Shannon index. (N to P) Results of Kaplan-Meier curve of three-group patient stratification on three datasets. ns, not significant.

We applied our model to three independent ADC datasets [NLST, University of Texas Southwestern Medical Center (UTSW)-SPORE, and TCGA-LUAD] to examine the association between the derived scores and patient overall survival. Table 2 summarizes the detailed clinical characteristics of patients included in each dataset. Patients were divided into two groups based on the median value of progression speed, diversity index, and fitness score: “slow” (≤median value) and “fast” (>median value). We performed the log-rank test between two patient stratification groups in the three datasets: NLST (fitness: P = 0.01; speed: P = 0.04; diversity: P = 0.05), LUAD (fitness: P = 0.02; speed: P = 0.02; diversity: P = 0.05), and SPORE (fitness: P = 0.03; speed: P = 0.04; diversity: P = 0.08) (Fig. 4, B to J). The slow progression group shows better survival than the fast progression group in all three datasets. Both fitness and speed scores showed substantial success in stratifying patients in all three datasets, while the tumor Shannon diversity is significant in both the NLST and LUAD datasets.

Table 2. Clinical characteristics of patients in NLST, SPORE, and LUAD datasets.

	NLST	SPORE	LUAD
Total patients	N = 148	N = 52	N = 167
Stage
I	103 (69.5%)	32 (61.5%)	92 (55.0%)
II	18 (12.1%)	8 (15.3%)	44 (26.3%)
III	20 (13.5%)	11 (21.1%)	21 (12.5%)
IV	7 (4.7%)	1 (1.9%)	10 (5.9%)
Gender
Male	77 (52%)	20 (38.4%)	75 (45.0%)
Female	71 (48%)	32 (61.5%)	92 (55.0%)
Smoke history
Yes	80 (54.1%)	47 (90.3%)	–
No	68 (45.9%)	5 (9.4%)	–
Age
≤59	32 (21.6%)	–	47 (28.1%)
>60	107 (78.4%)	–	120 (71.8%)
Race
Caucasian	–	44 (84.6%)	134 (80.2%)
African American	–	5 (9.6%)	22 (13.1%)
Asian	–	3 (5.8%)	7 (4.2%)
Not reported	–	–	4 (2.3%)

In the univariate overall survival analysis using the Cox proportional hazard (Cox-PH) regression model, the fitness score is a significant independent predictor of hazard risk in the NLST (P = 0.05) and SPORE (P = 0.01) datasets, and speed is also substantial for the SPORE (P = 0.02) datasets (Fig. 4, B to J). Integrating the image-derived scores with clinical risk factors (e.g., cancer stage, gender, and age) in the multivariate Cox-PH analysis also improved the risk prediction results of higher concordance index (C-index) in both the NLST (from 0.67 to 0.7) and SPORE (from 0.72 to 0.82) datasets. In the LUAD dataset, the C-index remained at 0.62 with no improvement after adding image features. Detailed outputs of the multivariate Cox-PH analysis are provided in tables S2 to S4.

We then investigated the correlation between progression speed and Shannon diversity across three datasets. A weak positive correlation was observed, suggesting that aggressive tumor progression is potentially associated with increased and more evenly distributed tumor clonotypes, as the Shannon index reflects tumor morphological heterogeneity calculated from image features (Fig. 4, K to M). Using only speed and diversity scores, patients were further stratified into the three progression groups of extremely slow (≤median of both speed and Shannon index), extremely fast (>median of both speed and Shannon index), and moderate (all other cases). This combined stratification approach further improved patient differentiation into slow, moderate, and fast progression groups across the three datasets (Fig. 4, N to P).

Robustness of patient stratification using tumor progression quantification

To further validate our framework, we repeated our patient stratification analysis using Louvain clustering results (fig. S1) (29). In the NLST cohort, all three stratification metrics derived from Louvain clusters robustly distinguished patient groups, and in the SPORE dataset, Shannon diversity similarly stratified clinical outcomes. Overall, greater tumor progression remained consistently associated with poorer survival in the Louvain analysis.

We then evaluated how the number of principal components (PCs) influences pseudotime and Shannon diversity metrics for patient stratification. Across PC counts of 10, 20, 30, 40, and 50, all three metrics exhibited positive correlations. Moreover, slower tumor progression quantification consistently predicted better survival (Fig. 4, table S5, and figs. S2 to S4). Notably, the fitness score proved more robust than either speed or Shannon diversity across all three datasets, showing minimal sensitivity to PC count (see Materials and Methods for details).

Molecular function of genes associated with tumor progression

We downloaded the match transcriptomic data of the SPORE and LUAD datasets and performed gene set enrichment analysis (GSEA) using genes ranked by Spearman correlation between gene expression and Shannon diversity, leveraging pathway annotations from REACTOME pathway database (30), cell type signature (C8) gene sets from the Molecular Signature Database (MSigDB) pathway database (31). In the fast progression group, up-regulated pathways included those related to cell cycle regulation, such as the synthesis phase, DNA replication (Fig. 5A, SPORE dataset), and G₂M checkpoint (fig. S5, LUAD dataset). These findings indicate increased cell division and proliferation in these tumors, corresponding to the classification of the fast-progress group. In contrast, the slow progression group showed higher expression levels of genes involved in surfactant metabolism, reflecting a more differentiated cell state characteristic of normal lung epithelial cells, which aligns with their classification as slow-progress tumors (Fig. 5A and fig. S5A). Supporting this notion, genes associated with slow progression were also significantly enriched in alveolar type 2 cell (AT2) signatures (Fig. 5B and fig. S5B) (28), a known cell type responsible for maintaining the gas exchange function of the lung, surfactant production, alveolar repair and regeneration, and innate immune regulation (32–34). The GSEA analysis also identified important genes, including members of the surfactant protein family (SFTPA, SFTPB, and SFTPC) and CD81 expressed on the surface of B cells (35), involved in antitumor activity. Analysis using both SPORE and TCGA datasets revealed that CXCL17 reported to exhibit anti-inflammatory activity in macrophages (36) and a prognostic marker in ADC (37). Although CXCL17 is commonly expressed in the lung, it has reported both protumor and antitumor activities (38, 39), its underlying mechanisms remain poorly understood.

Fig. 5. Transcriptomic analysis of the patient stratification based on the tumor progression quantification.

(A) GSEA analysis results of the REACTOME pathway database using genes ranked by the correlation between gene expression and Shannon index from the LUAD dataset. NES, normalized enrichment score. (B) Normalized gene expression profile of genes enriched in the AT2 cell gene signatures between the slow progression and fast progression patients of the LUAD dataset. The AT2 gene signatures are acquired from the MSigDB. (C) Wilcox rank sum test results of AT2 scores between the slow and fast groups. (D) Spearman correlation of AT2 score and Shannon index. R², coefficient of determination. (E) Genes negatively correlated with Shannon diversity index were significantly enriched in the AT2 cell type signatures.

To further investigate this relationship, we calculated an AT2 score as the average expression of leading-edge genes from the GSEA results. A negative correlation between the AT2 score and derived speed, diversity, and fitness score was observed in both datasets. Notably, slow progression patients exhibited higher expression of AT2 cell type gene signatures compared to fast progression patients (Fig. 5, C and D, and fig. S5, C and D). This suggests an enhanced normal cell function, and it may also indicate that the tumor cells retain features of AT2 cells and are well differentiated (low-grade tumor), which usually has a slower growth rate and less aggressive behavior than poorly differentiated (high-grade tumor).

Integrating cell differentiation model with spatial transcriptomic data

To assess our model on spatial transcriptomic data, we applied it to the Xenium 5K ADC dataset downloaded from 10X Genomics, predicting cell differentiation status across WSI (Fig. 6A). Patch-level gene expression was calculated by averaging spot-level profiles, and highly variable genes were identified between differentiation states. The top five markers for G2 tumor regions (GLB1, SEPTIN9, LTA4H, PRR13, and VEGFB) and normal regions (CX3CL1, APP, MAFF, ZFAND5, and CAVIN2) are highlighted in Fig. 6B. Notably, LTA4H is up-regulated in tumor versus normal tissue, reflecting its roles in inflammatory signaling and cell proliferation (40, 41). Both PRR13 and VEGFB correlate with poor survival in ADC patients (42, 43), and VEGFB further drives metastasis, making it a potential therapeutic target (43–45). In contrast, CX3CL1—a chemokine with context-dependent pro- and antitumor effects—may regulate natural killer cell infiltration (46), while MAFF acts as a tumor suppressor by inhibiting ADC cell proliferation and blocking cell cycle progression (47).

Fig. 6. Integrative analysis with spatial transcriptomic data of ADC.

(A) Results of the cell differentiation status prediction from the trained model. (B) Highly variable genes differentiate between the predicted G1 and the other normal region. (C) Cell type annotation of the Xenium 5K ADC dataset. (D) UMAP of the identified cell clusters. (E) Highly variable genes in each annotated cell cluster. (F to I) Comparison of cell components from image patches predicted as G2 tumor or other normal regions.

We next applied Leiden clustering to the spatial transcriptomic expressions and annotated cell type based on its highly variable genes, enabling us to assess immune-cell composition across cell differential statuses (Fig. 6, C to E). We then quantify the cell components of each slide and compare the compositions between predicted grades (Fig. 6, F and G, and fig. S6). As anticipated, tumor regions were dominated by epithelial tumor cells (Fig. 6F). These regions also recruit an increased infiltration of memory T cells, identified by CD44 and CXCR4 expression, relative to adjacent normal tissue (Fig. 6, E to G). Conversely, vascular endothelial cells and alveolar fibroblasts—marked by DHCR24, SDC4, TSPAN3, ATP1A1, and TPPP3—were significantly enriched in normal regions (Fig. 6, H and I), which tends to correlate with better overall survival (48).

DISCUSSION

In this study, we developed an AI model that leverages the potential of cost-effective H&E images to analyze cell differentiation status, trace cell trajectories, and assess tumor progression. Overcoming the limitation of expensive, bad data quality and limited accessibility of single-cell RNA-seq, our proposed approach aids in patient stratification and provides guidance for clinical cancer treatment strategies in lung ADC,

To class the cell differentiation statuses, we fine-tuned the Phikon (26) foundation model that performs better than Resnet models in this task since it was specifically developed for pathological image analysis and was pretrained on H&E images from TCGA, while Resnet (28) models were pretrained from the natural ImageNet (49) dataset. Our framework effectively captures the morphological features of cells, enabling accurate performance at both the patch and WSI levels. When applied to WSIs, our model successfully identified ROIs with validation from expert pathologists. Therefore, our AI model offers an efficient tool to assist pathologists in manually examining H&E slides and ROI annotations in clinical settings.

In addition, the image-derived features demonstrate the ability to infer pseudotime, effectively depicting cell differentiation trajectories across the WSI. Moreover, although all other pathological conditions were labeled as other, image-based pseudotime analysis can differentiate necrotic regions from normal and tumor cells as the trained model can capture the morphological difference among different cell types. This highlights the model’s capacity to extract nuanced biological insights beyond standard classifications.

By integrating spatial coordinates and clustering results of image patches, we developed three metrics to capture tumor progression status: (i) speed, which quantifies the spatial tumor expansion velocity within a given area; (ii) the index of tumor diversity, reflecting tumor morphological heterogeneity; and (iii) fitness, a combined measure of speed and diversity index. We validated these metrics in three independent ADC cohorts—NLST, SPORE, and LUAD—with promising results. First, we demonstrated the patient stratification capability of these metrics. Predicted slow progression patients, characterized by low speed, low diversity, and low fitness, exhibited significantly better survival rates than fast progression patients, as confirmed by log-rank tests. Furthermore, incorporating these image-derived features into clinical phenotypes improved survival risk prediction, highlighting their value in prognostic modeling.

To further elucidate the biological basis of tumor progression, we conducted GSEA using transcriptomic data from the SPORE and LUAD cohorts. Fast progression patients displayed up-regulated pathways associated with active cell cycle processes, such as increased proliferation and reduced surfactant metabolism, contributing to more aggressive tumor behavior. In contrast, slow progression patients exhibited high expression of AT2 cell gene signatures, indicating that their tumors retained more normal-like and well-differentiated characteristics. In addition, the integrative analysis with spatial transcriptomics identified both meaningful genes and cell types that spatially vary between tumor and normal regions. Overall, these findings were consistent with our predicted cell differentiation status and stratified patient groups, providing robust support for the biological relevance of the proposed metrics in understanding tumor progression and stratifying patients by their clinical outcomes.

There are limitations to our study. First, our study has a relatively small number of verified ROIs in our training dataset, which may affect the prediction efficacy. However, the impact on downstream analyses is minimal, as pseudotime calculations rely more on the extracted image features than predicted status. Expanding the high-quality dataset to include a larger variety of tumor-grade slides and other pathological conditions would further enhance the model’s performance and generalizability. Second, given that cell differential status reflects the degree of similarity between tumor and normal cells, our proposed tumor progression quantification incorporates image features from both tumor and nontumor regions. Incorporating cell type segmentation could further refine the model by isolating tumor-specific features and focusing on interactions among tumor, stromal, and immune cells.

In conclusion, we developed an innovative AI-assisted model for cell differentiation status classification, tracing cell trajectories, and assessing tumor progression. This study uses only image-derived features to estimate pseudotime and tumor progression from H&E slides. We demonstrated the clinical applicability of our model across three independent lung ADC cohorts, showcasing its robustness and utility. By enabling accurate and efficient processing of histopathological images, our model has the potential to advance cancer research and improve clinical cancer treatment practices.

MATERIALS AND METHODS

Dataset

NLST-ADC dataset

The lung ADC pathology image datasets used to support cell differentiation status classification were obtained from the NLST-ADC. The publicly available dataset from the NLST (https://biometry.nci.nih.gov/cdas/nlst/) comprises 318 H&E-stained pathology images (×40 magnification) along with the clinical data from 148 patients. Multiple pathology slides may correspond to a single patient. To enhance our analysis, a total of 440 ROIs encompassing tumor, normal, necrosis, and other pathological conditions were annotated and confirmed by pathologists using the I-viewer software (50).

TCGA-LUAD dataset

Clinical data are downloaded from the TCGA data portal. Gene-normalized RNA-seq data from TCGA LUAD, processed by RSEM (RNA-seq by expectation maximization) algorithm was downloaded from Firebrowse (http://firebrowse.org/) (doi:10.7908/C19P30S6). All the imaging data are available at National Cancer Institute Imaging Data Commons (https://datacommons.cancer.gov/repository/imaging-data-commons). More information can be found at the TCGA website (https://cancer.gov/ccg/research/genome-sequencing/tcga). All participating sites obtained local institutional review board approval for participation in this study. All samples in TCGA have been collected and used following strict human subject protection guidelines and informed protocols.

UTSW-SPORE dataset

The SPORE in the Lung Cancer dataset includes 111 patients with ADC (51). Tissue samples were obtained via surgical resection from patients who provided written informed consent. All tumor tissue slides were formalin-fixed and paraffin-embedded and scanned at either ×20 or ×40 magnifications. Microarray data for this dataset were retrieved from the Gene Expression Omnibus (GSE42127) (52). The Illumina probe IDs from this dataset were converted to gene symbols using the “illuminaHumanv3.db (1.26.0)” R packages.

Cell differentiation status classification from histopathology images

We integrated the Phikon model (26), followed by sequential feed-forward layers with 128 and 5 nodes (number of classes), to predict patch-level cell differentiation statuses (Fig. 2B). The Phikon model is a foundation model pretrained on 40 million TCGA histopathology patches. To tailor it for our specific task, we fine-tuned the model using tumor grade–specific patches extracted from annotated ROIs. The dataset was split into 395 ROIs for training and 45 ROIs for testing. In addition to the four traditional grade (G1 to G4) classifications, we included other cell statuses as a fifth class that covers normal and necrosis pathological conditions. Patches were randomly extracted from ROIs at a resolution of 224 × 224 pixels to match the input size requirements of the Phikon and Phikon-V2 models. For the final dataset, 5 patches were extracted from each training ROI, and 50 patches were extracted from each testing ROI (Fig. 2C). When testing the performance of Resnet models (28), patches were extracted at 500 × 500. The final model was trained for 100 epochs with a batch size of 20. Cross-entropy loss was applied as the objective function, which was minimized by the Adam optimizer with a step-wise learning rate of 0.001. To mitigate overfitting from the imbalanced ROI distribution across cell differential status, we applied cross-entropy weights derived from the inverse class frequencies in our training set: 0.0607, 0.0184, 0.0293, 0.8695, and 0.0221 for G1, G2, G3, G4, and other, respectively. We selected this weighted model after comparison with unweighted versions, which failed to predict any G4 tumors and showed lower overall accuracy (fig. S7). The “scikit-image” library (v0.22.0) in Python was used for patch extraction, while “torch” (v2.2.1) and “torchvision” (v0.17.1) were used for model training. The Phikon model was implemented using the “ViTModel” from the “transformers” package (v4.38.2) in Python.

Extracting WSI image features from histopathology images

After training the patch-level prediction model, we applied it to scan WSIs, processing all individual patches extracted at 400-pixel intervals. For each patch, 768 Phikon features were extracted from the fine-tuned model to predict the cell differentiation status of the specific patch region. These features were subsequently used to infer pseudotime at the WSI level (Fig. 3A). Details and scripts of Phikon feature extraction are available at https://huggingface.co/owkin/phikon.

Pseudotime inference from extracted WSI histopathology image features

Similar to pseudotime inference from gene expression of single-cell RNA-seq, we inferred histopathology pseudotime using image patch features extracted from our fine-tuned model. First, the feature dimensions were reduced using principal components analysis. We computed the nearest-neighbor distance matrix using four neighbors and the first 20 PCs, capturing ≥63% of the variance across all H&E images. Patch clustering was then performed at a resolution of 1 using the Leiden algorithm, a computationally efficient method that ensures all patches are well-connected for optimal downstream trajectory inference (29). To construct a topological graph based on patch partitions, we used partition-based graph abstraction (53). A randomly selected normal patch was designated as the trajectory root node. Subsequently, the diffusion pseudotime (54) method was applied to estimate pseudotime values, capturing the dynamic tumor cell progression in histological space. All analyses were conducted using the “Scanpy” Python package (v1.9.8) (55).

Integrating spatial data and pseudotime for tumor progression quantification

The quantification of tumor progression describes the tumor expansion velocity within an area, integrating pseudotime and spatial organization over the WSI. The tumor progression fitness score is calculated as

$$\text{Fitness}=S\ast I$$ (1)

$$S=\text{log}({\displaystyle \sum _{i,j}^n}\frac{d [i,j]}{\mid t_i-t_j\mid }\ast \frac{1}{n})$$ (2)

$$I=-\sum _i^n\left[p_i\ast \text{log}\left(p_i\right)\right]$$ (3)

where S represents the average pairwise tumor progression speed among Leiden clusters, t denotes the mean pseudotime of each cluster, and d is the Euclidean distance between the centroids of clusters i and j. Hereby, a smaller S value indicates slower tumor progression, characterized by a limited spatial tumor expansion area on H&E images within a unit of pseudotime. In addition, a larger S value represents more aggressive cell growth and expansion activity. I is the Shannon index calculated on the basis of the Leiden clusters, reflecting the tumor clonal diversity inferred from the cell morphology heterogeneity. A low Shannon index indicates a high dominance of certain pathological cell clonotypes, corresponding to a relatively purer tumor microenvironment, suggesting a slow progression with relatively less active cell status transition or cell growth activity. In contrast, a higher Shannon index indicates more unique clonotypes and active cell activities. Together, the fitness score integrates tumor progression speed (S) and clonal diversity (I) to provide a comprehensive measure of tumor dynamics in H&E images, capturing both diversity and the presence of homogeneous cell components.

Patient selection and survival analysis

We conducted survival analysis on the three lung ADC cohorts to investigate the association between our tumor progression quantifications and clinical outcomes. H&E images with less than 10,000 patches extracted from the TCGA and NLST datasets and less than 3500 patches extracted from the SPORE dataset were excluded from the cohort analysis. Patients were stratified into two groups based on the median values of the fitness score, speed (S), or Shannon diversity (I). Patients with values less than or equal to the median were classified into the slow tumor progression group. In contrast, those with values above the median were placed in the fast progression group. The Kaplan-Meier (KM) curve and log-rank test were used to evaluate the survival difference between slow and fast groups. In addition, the Cox-PH model was applied to perform univariate and multivariate survival regression analyses. The C-index is used to measure the prognostic ability of survival models. All analysis was conducted using the Python package “lifelines” (v0.28.0).

Gene set enrichment analysis

The hallmark gene set (H) and cell type signature gene sets (C8) from the REACTOME pathway database (30) and MSigDB (31) were used for GSEA analysis. Genes were ranked on the basis of their Spearman correlation between RNA expression and the Shannon index. Correlation is calculated using the “spicy.stats” Python packages. GSEA was performed using the “fgsea” R package (v1.28.0).

Spatial transcriptomic analysis

The “Squidpy” (v1.6.5) (56) and Scanpy (v.19.8) (55) Python packages were used to process the Xenium ADC 5K dataset. First, we computed a nearest-neighbor distance matrix using the four closest neighbors in the space defined by the top 20 PCs. Cells were then clustered via the Leiden algorithm at a resolution of 0.1, and highly variable genes were identified by Wilcoxon rank sum testing across clusters. Last, cell type annotation was performed by applying the GPTCelltype prompt to the marker genes discovered for each cluster (57). The exact prompt used is given below.

“Identify cell types of human lung tumor tissue using the following markers. Identify one cell type for each row. Only provide the cell type name. \nMarkerGeneList”

Statistical analysis

The gene signature score of AT2 cell types (AT2 score) is the average expression of leading-edge genes from the GSEA results. The Kruskal-Wallis test is used to compare gene signature scores between patient stratification groups. All statistical analyses are implemented using the “ggplot2 (3.5.0)” R package.

Supplementary Materials

This PDF file includes:

Figs. S1 to S7

Tables S1 to S5