A hybrid multi-instance learning-based identification of gastric adenocarcinoma differentiation on whole-slide images

Abstract

Objective

To investigate the potential of a hybrid multi-instance learning model (TGMIL) combining Transformer and graph attention networks for classifying gastric adenocarcinoma differentiation on whole-slide images (WSIs) without manual annotation.

Methods and materials

A hybrid multi-instance learning model is proposed based on the Transformer and the graph attention network, called TGMIL, to classify the differentiation of gastric adenocarcinoma. A total of 613 WSIs from patients with gastric adenocarcinoma were retrospectively collected from two different hospitals. According to the differentiation of gastric adenocarcinoma, the data were divided into four groups: normal group (n = 254), well differentiation group (n = 166), moderately differentiation group (n = 75), and poorly differentiation group (n = 118). The gold standard of differentiation classification was blindly established by two gastrointestinal pathologists. The WSIs were randomly split into a training dataset consisting of 494 images and a testing dataset consisting of 119 images. Within the training set, the WSI count of the normal, well, moderately, and poorly differential groups was 203, 131, 62, and 98 individuals, respectively. Within the test set, the corresponding WSI count was 51, 35, 13, and 20 individuals.

Results

The TGMIL model developed for the differential prediction task exhibited remarkable efficiency when considering sensitivity, specificity, and the area under the curve (AUC) values. We also conducted a comparative analysis to assess the efficiency of five other models, namely MIL, CLAM_SB, CLAM_MB, DSMIL, and TransMIL, in classifying the differentiation of gastric cancer. The TGMIL model achieved a sensitivity of 73.33% and a specificity of 91.11%, with an AUC value of 0.86.

Conclusions

The hybrid multi-instance learning model TGMIL could accurately classify the differentiation of gastric adenocarcinoma using WSI without the need for labor-intensive and time-consuming manual annotations, which will improve the efficiency and objectivity of diagnosis.

Introduction

Gastric cancer is a serious global health problem, with more than 1 million people diagnosed worldwide each year. Globally, gastric cancer has been the fifth-most commonly diagnosed cancer type despite a decline in incidence and mortality over the past five decades [1]. It is estimated that over 95% of gastric cancers are gastric adenocarcinoma (GAC), which is the most common histological type. GAC has a high level of tumor heterogeneity, and it is often first diagnosed at an advanced stage, which is a major reason for its poor outcome [2]. An accurate prognosis assessment or prediction is crucial in the detection and treatment of GAC, as it seriously affects the survival outcome of patients.

Tumor, nodes, and metastasis (TNM) staging is the most commonly used method to assess the prognosis of gastric cancer according to the American Joint Cancer Committee (AJCC) guidelines [3]. Post-operative staging from pathological images has always been considered the gold standard for staging gastric cancer by the International Alliance Against Cancer [4]. Based on some analysis of large cases, it seems that the grade of tumor resection, the depth of tumor invasion, and lymph node metastasis have the greatest effect on prognosis [5]. The correlation between prognosis and differentiation has also gradually attracted the attention of researchers. Differentiation grading in tumors is based on the resemblance of cancer cells to normal cells of the same tissue type. For example, cancer cells of well differentiated tumors are closely resemble normal cells, this tumors grow slowly, and are typically associated with better prognoses. On the other side, the cancer cells of poorly differentiated tumors have minimal resemblance to normal cells, grow aggressively, and are linked to poorer prognoses. This grading system aids in assessing tumor aggressiveness and predicting behavior. Findings by researchers [6] from Xijing Hospital revealed a significant association between the differentiation of gastric tumor and its clinicopathological characteristics. Specifically, in stage III gastric cancer patients, those with well differentiation exhibited better overall survival rates, whereas those with poorly differentiation had poorer overall survival rates. The study conducted by Zhijie Wang et al. [7] found that the histological differentiation level of colorectal neuroendocrine tumors plays a crucial role in determining the prognosis of patients with metastatic colorectal neuroendocrine tumors. Luo et al. [8] discovered a correlation between the grade of differentiation and prognosis in colorectal cancer patients who undergo neoadjuvant chemotherapy. Traditional pathological analysis of differentiation is time-consuming and suffers from a lack of consensus among different observers [9], leading to a decrease in the reliability of pathological diagnoses. Moreover, finding experienced pathologists is a constant challenge. Consequently, it is imperative to develop computer-assisted methods that are both automated and objective to enhance decision-making in the field of histopathology [10].

This is where digital pathology can play a crucial role. With the widespread adoption of whole-slide imaging (WSI; also, whole-slide image) [11], the potential for computer-aided pathological analysis has significantly increased. Numerous studies have successfully employed deep learning algorithms for automated WSI analysis for a wide range of pathological conditions, including breast cancer, prostate cancer, and colon cancer [12–14]. Previous studies in this field have primarily relied on patch-based methods, which necessitate manual annotation of WSI. To alleviate the burden of annotation, researchers have explored various deep learning techniques for label-free training. Chen et al. [15] constructed a deep learning architecture to classify adenocarcinoma and squamous cell carcinoma based on 9662 lung cancer WSIs. Encouragingly, the results yielded excellent performance without the need for annotations. Comes et al. [16] also built a deep learning model based on WSI to predict disease-free survival in cutaneous melanoma patients.

We employed a hybrid multi-instance learning (MIL) architecture based on the Transformer [17] and graph attention (GAT) [18] called TGMIL [19] to capture the contextual and spatial information of WSI to classify the differentiation of gastric adenocarcinoma without annotation. The field of pathology is currently facing a shortage of skilled professionals, which, coupled with their heavy workload, high risks, and immense responsibilities, hampers the overall progress of the field. The objectives of this study were to reduce the burden on pathologists, enhance the efficiency and reliability of pathological diagnosis, and address the supply–demand imbalance by developing an automated auxiliary diagnostic system. By integrating cutting-edge deep learning techniques with medical methods, we aim to ensure timely diagnosis and treatment for the majority of gastric cancer patients.

Results

Patient cohort

In this experiment, we utilized the WSI data obtained from 613 gastric adenocarcinoma patients at Guizhou Provincial People’s Hospital (GPH) and Southwest Hospital (SH). The dataset consisted of 166 cases of well-differentiated gastric cancer, 75 cases of moderately differentiated gastric cancer, 118 cases of poorly differentiated gastric cancer, and 254 cases of normal gastric tissue slices. These details are summarized in Table 1.

Table 1.

Data sources and distribution of 613 gastric adenocarcinoma patients.

		Normal	Well differentiation	Moderately differentiation	Poorly differentiation	Total
GPH	Training set	42	37	40	119	238
	Test set	11	9	10	29	59
SH	Training set	161	61	22	12	256
	Test set	40	11	3	6	60
	Total	254	118	75	166	613

Performance of the TGMIL architecture

In our study, we employed TGMIL and five cutting-edge MIL models to accurately classify the grade of differentiation in gastric carcinoma. These five models were MIL [20], CLAM_SB [20], CLAM_MB [20], DSMIL [21], and TransMIL [22]. To comprehensively assess the performance of these models, we utilized a range of evaluation metrics such as the AUC, ACC, Pre, F1, SEN, and SPE. By using these metrics, we evaluated the effectiveness and reliability of these models in accurately categorizing the level of differentiation in gastric carcinoma.

By utilizing the TGMIL model for classifying gastric carcinoma differentiation, we achieved an impressive accuracy rate of 73.33%, precision rate of 70.32%, sensitivity rate of 73.33%, specificity rate of 91.11%, and F1 score of 71.27%. In comparison, the accuracy rates of MIL, CLAM_SB, CLAM_MB, DSMIL, and TransMIL were as follows: 63.45%, 67.39%, 66.00%, 69.99%, and 69.65%, respectively; the precision rates were 60.78%, 66.68%, 66.40%, 68.98%, and 66.07% respectively. The sensitivity rates were 63.45%, 67.39%, 66.00%, 69.99%, and 69.65%. The specificity rates were 87.82%, 89.13%, 88.67%, 90.00%, and 89.88%. The F1 scores were 59.91%, 66.04%, 65.30%, 68.43%, and 69.65% (Table 2 and Fig. 1).

Table 2.

Classification results of TGMIL compared with five other models (mean ± std).

Model	ACC	Pre	F1	SEN	SPE
MIL	0.6345±0.0281	0.6078±0.0298	0.5991±0.0357	0.6345±0.0281	0.8782±0.0094
CLAM_SB	0.6739±0.0486	0.6668±0.0308	0.6604±0.0341	0.6739±0.0486	0.8913±0.0162
CLAM_MB	0.6600±0.0317	0.6640±0.0400	0.6530±0.0349	0.6600±0.0317	0.8867±0.0106
DSMIL	0.6999±0.0358	0.6898±0.0400	0.6843±0.0421	0.6999±0.0358	0.9000±0.0119
TransMIL	0.6965±0.0223	0.6607±0.0265	0.6688±0.0233	0.6965±0.0223	0.8988±0.0074
TGMIL	0.7333±0.0236	0.7032±0.0670	0.7127±0.0446	0.7333±0.0236	0.9111±0.0079

Fig. 1.

Confusion matrix diagram of the TGMIL model and the other five models. In the confusion matrix, 1 stands for normal; 2 stands for poorly differentiation; 3 stands for moderately differentiation; and 4 stands for well differentiation.

The TGMIL model demonstrated superior performance in accurately classifying gastric carcinoma differentiation and surpassed the other models in terms of the AUC values. The AUC values of the TGMIL, MIL, CLAM_SB, CLAM_MB, DSMIL, and TransMIL models were 86.08%, 77.16%, 83.34%, 83.20%, 83.55%, and 81.37, respectively (Fig. 2). The TGMIL model takes into full consideration the reading habits of pathologists and pays overall attention to the tumor area. We performed attention visualization on our dataset, with a specific focus on three subtypes: well differentiation, moderately differentiation, and poorly differentiation. For each subtype, we obtained slide annotations from two experienced pathologists to mark the cancer region. To visualize the attention patterns, we generated corresponding heat maps. Figure 3 displays the attentional heat map for the three subtypes of gastric carcinoma. The attentional heat map clearly illustrates the TGMIL model's strong focus on the tumor region.

Fig. 2.

AUC comparison diagrams of MIL, CLAM_SB, CLAM_MB, DSMIL, TransMIL, and TGMIL.

Fig. 3.

Heat maps of the attention patterns. Slide annotations from two experienced pathologists are denoted in the left. HD represents the well differentiated group; MD represents the moderately differentiated group; and LD represents the poorly differentiated group. In these heat maps, the high attention regions are represented by red, and the low attention regions are represented by blue.

Discussion

Determining the grade of histopathological differentiation in gastric adenocarcinoma is a challenging task due to the inherent variability within the tumor group and its components as well as the labor-intensive process of annotating WSIs. To address this burden, we developed the TGMIL model that automatically determines the differentiation grade of gastric adenocarcinoma. This model eliminates the need for manual annotation and enables objective and efficient assessment of differentiation using WSI. The remarkable achievement of our model is reflected in its accuracy of 73.33% and AUC value of 86.08%. The TGMIL model represents an advancement in the field, offering a more efficient and accurate approach to determining histopathological differentiation in gastric adenocarcinoma. Its potential impact on diagnosis and treatment decisions could improve patient outcomes.

Due to the diverse histological and molecular characteristics of gastric cancer, diagnosing it rapidly and accurately poses a significant challenge [23]. The traditional diagnostic approach heavily relies on pathologists’ subjective judgment, which is time-consuming and lacks objective consistency [24]. In contrast, deep learning offers several advantages such as automation, accuracy, adaptability, ability to process unstructured data, efficiency, and scalability [25]. As a result, it has promising applications in various fields. Consequently, numerous scholars have proposed utilizing deep learning systems for automatically identifying tumor differentiation. For instance, Su et al. [26] employed a deep learning model to classify the WSI of gastric cancer differentiation from 467 patients. Hyun-Jong Jang et al. [27] also did similar research. They both achieved favorable outcomes. However, the approach suggested by Feng Su et al. still required manual labeling of the training data. In addition, they both only categorized gastric cancer differentiation into well and poorly groups, which differs from the actual pathological differentiation grading. In contrast, our study divided gastric cancer into four groups (normal, well, moderately, and poorly differentiation) based on actual differentiation levels without the need for manual annotation. Our proposed model is an enhanced version that addresses the limitations of the previous model. Previously, our team developed the StoHisNet method [28] for classifying gastric cancer and various tumors. In our new model, TGMIL, we have made improvements in two key areas. First, we have eliminated the need for detailed annotations and labels for WSI, thereby reducing the annotation costs. Second, TGMIL incorporates the contextual relevance and spatial information of the instances, thus enhancing the accuracy of classification. Moreover, we have taken inspiration from the reading habits of pathologists to improve the model’s performance. In TGMIL, we have introduced the MMFP module, which seamlessly integrates information from multiple magnifications. This module effectively simulates the process of pathologists adjusting microscope magnifications to observe both global and local details in the tissue sections.

Based on the Transformer and the GAT, we proposed a hybrid multi-instance learning model called TGMIL to classify the differentiation grade of gastric adenocarcinoma, which can automatically classify gastric adenocarcinoma into differentiation grades without the need for manual delineation. Previous studies have predominantly focused on classifying differentiation grades into well-differentiated and poorly-differentiated groups. However, our approach accurately categorizes gastric cancer into three groups, well, moderately and poorly differentiation, following established pathological classification standards. We made efforts to simulate the reading habits of pathologists within our model. However, our work does have certain limitations. First, our pathology data, although sourced from two hospitals, lacked independent external validation. Second, the quality of pathological images varies due to the storage conditions and staining constraints. Consequently, our future work will concentrate on enhancing our model’s capability to handle diverse data, improving generalization and consistency, conducting extensive verification using multi-center data and exploring the correlation between tumor differentiation and patient outcomes.

Materials and methods

Patient cohort, tissue resources, and scanning of tissue slides

This retrospective study collected paraffin specimens from 613 patients with gastric adenocarcinoma from the Department of Pathology of Guizhou Provincial People’s Hospital and Southwest Hospital Affiliated to Third Military Medical University from 2016 to 2019. Gastric adenocarcinoma specimens were obtained from the anatomical area of the tumor, and tumor-free gastric specimens were collected from the non-gastric tumor regions. The diagnosis of gastric adenocarcinoma was based on the 2016 guidelines of AJCC. The gastric cancer classification was confirmed by two pathologists. The cells were evenly spread on the slide, providing clear visibility of the field of vision, and the slide contained a moderate number of cells with minimal overlap. Routine hematoxylin and eosin staining was performed according to best practice recommendations, followed by scanning of the tissue sections at 400× magnification using a slide scanner (Aperio SC2, Leica Biosystems, Nussloch, Germany).

MIL problem formulation

A key concept in MIL is the concept of bags and instances. Taking the binary classification problem as an example, a bag is labeled as positive if at least one sample in the bag belongs to the positive category, and the bag is labeled as negative only if all the samples in the bag do not belong to the positive category.

The MIL-based dichotomy can be defined as follows:

Let $B_i=\left\{x_{i,1},x_{i,2},...,x_{i,k}\right\},i\in m$ be a bag, where $x_{i,j}$ is an instance with label $y_{i,j}\in \left(0,1\right),j\in k$. The instances are relevant to each other. $m$ is the total number of bags; $k$ is the number of instances in the ith bag; and the number of $k$ may very depending on the bag. The label $Y_i$ of the bag $B_i$ is given by the following:

$$\left\{\begin{array}{c}{Y}_i=\left\{\begin{array}{c}0,iff\sum y_{i,j}=0\\ 1,\text{otherwise}\\ \end{array}\right)\\ {\widehat{Y}}_i=S\left(B_i\right)\\ \end{array}\right)$$ 1

where $S$ is a scoring function, and ${\widehat{Y}}_i$ denotes the predicted label of the ith bag.

TGMIL is built on top of the MIL framework, which treats each WSI (bag) as a collection of many small patches (instances) with correlation.

TGMIL for WSI classification

The overall architecture of TGMIL is presented in Fig. 4. MMFP is a feature pyramid module with multiple low magnifications. TGMIL learns by employing an attention mechanism to obtain the instance context and spatial information. Its attention streams are the Transformer and the GAT. Finally, the outputs of the two attention streams are aggregated by the MIL strategy for prediction.

Fig. 4.

Overall architecture of TGMIL.

MMFP module

The design of the module effectively mimics the diagnostic process of a pathologist who repeatedly changes the magnification of the microscope to observe the global information and local detail information. We leverage the characteristics of the WSI multi-resolution storage structure and integrate multiple poorly-magnification WSIs into the MMFP module through a feature extractor, and that serves as the input for the TGMIL model. The use of multiscale features in WSI analysis is beneficial [21, 29, 30]. The structure of the MMFP module is shown in Fig. 5. Considering the limitation of memory and counting resources, in this study, three magnifications of 20×, 10×, and 5× are selected. A WSI is cropped into many 512 × 512 sized image blocks, and a pre-trained ResNet-50 [31] model on ImageNet is used as the image block feature extractor. Each 512 × 512 size image block is compressed into a 1024-dimensional feature vector by the feature extractor.

Fig. 5.

MMFP module structure.

Transformer and the GAT streams for TGMIL

The features $\text{Z}\in {\text{R}}^{\text{K}\times 1024}$ extracted by WSI through the MMFP module serve as inputs to the TGMIL model, where K is the number of the WSI blocks. In TGMIL, the dimension of each feature embedding is first reduced from 1024 to a 512-dimensional feature vector $\text{H}={\text{F}}_1\text{Z}$ through a fully connected layer $F_1\in R^{512\times 1024}$. The feature embedding of each bag B can be denoted as $\text{H}=\{h_1,h_2,\cdots ,h_k\}$, $\text{H}\in R^{512\times 1024}$ and then, the feature H′ is output after the attention streams of TGMIL. TGMIL consists of two attention streams, Transformer and GAT, and H′ can be obtained from the following equation:

$${\text{H}}^{\prime }={\text{H}}_{\text{T}}^{\prime }+{\text{H}}_{\text{G}}^{\prime }$$ 2

where ${\text{H}}_{\text{T}}\prime$ represents the output of the Transformer stream, and ${\text{H}}_{\text{G}}\prime$ is the output of the GAT stream. The Transformer stream mainly consists of two layers of the Transformer encoder. The output feature ${\text{H}}_{\text{T}}\prime$ of the Transformer stream is represented as follows:

$$\left\{\begin{array}{c}{\text{H}}_0=\left[h_1,h_2,\dots ,h_K\right]+E_{\mathit{pos}},H_0,E_{\mathit{pos}}\in R^{K\times d},h_{\text{i}}\in {\text{R}}^{\text{d}}\\ {\text{Q}}_{\ell }=H_{\ell -1}{W}^Q,K_{\ell }=H_{\ell -1}{W}^{\text{K}},V_{\ell }=H_{\ell -1}{W}^V,\ell =1\dots L\\ head=SA\left({\text{Q}}_{\ell },K_{\ell },V_{\ell }\right)=softmax\frac{{\text{Q}}_{\ell }{\left(K_{\ell }\right)}^T}{\sqrt{d_q}}{V}_{\ell },\ell =1\dots L\\ MSA\left({\text{Q}}_{\ell },K_{\ell },V_{\ell }\right)=concat\left(hea{d}_1,hea{d}_2,\dots ,hea{d}_m\right)W^o,\ell =1\dots L\\ H_{\ell }=MSA\left(LN\left(H_{\ell -1}\right)\right)+H_{\ell -1},\ell =1\dots L\\ H_{\ell }=MLP\left(LN\left(H_{\ell }\right)\right),\ell =1\dots L\\ H_T^{\prime }=F_T{H}_L\\ \end{array}\right)$$ 3

where $W^Q\in R^{d\times d_q}$, $W^K\in R^{d\times d_K}$, $W^V\in R^{d\times d_V}$, $W^o\in R^{h{d}_V\times d}$, and $head\in R^{K\times d_V}$; $d=512$ is the feature embedding dimension for each bag B; SA stands for the self-attention layer of the Transformer; MSA stands for multi-head self-attention; L is the number of blocks of the MSA; m is the number of heads of each MSA module; MLP stands for multi-layer perceptron; LN stands for layer norm; and $H_L$ is the output of the Transformer. $H_L$ passes through a fully connected layer $F_T\in R^{N\times d}$, and $H_T^{\prime }\in R^{N\times K}$ represents the output features of the Transformer stream in TGMIL, where N denotes the number of categories.

Another stream of TGMIL is GAT, which can effectively mine the WSI contextual and spatial information. The GAT stream output $H_G^{\prime }$ is represented as follows:

$$\left\{\begin{array}{c}{e}_{\mathit{ij}}=Leaky\text{Re}LU\left(a\left(W{h}_i,W{h}_j\right)+WSI\_adj\right)\\ a_{\mathit{ij}}=softmax\left(e_{\mathit{ij}}\right)=\frac{exp\left(e_{\mathit{ij}}\right)}{\sum _{j\in N_i}exp\left(e_{\mathit{ij}}\right)}\\ H_G^{\prime }=\underset{i}{\overset{K}{||}}\underset{m}{\overset{M}{||}}\sigma \sum _{j\in N_i}{a}_{\mathit{ij}}^m{W}^m{h}_j\\ \end{array}\right)$$ 4

where $e_{\mathit{ij}}$ is the attention coefficient for each node. $W\in R^{N\times d}$ is a learnable shared linear transformation. $a:R^{N\times N}\to R$ is a shared attention mechanism. $WSI\_adj\in R^{K\times K}$ is the weighted adjacency matrix of the WSI. In graph convolutional neural networks, the contribution of the neighboring nodes around each node is considered to be the same. However, as Liang et al. [32] stated, the correlation between the nodes in the graph and the surrounding nodes is not always the same in practice. Therefore, in the GAT stream, we use the traditional aggregation methods of MIL (max-pooling and mean-pooling) [33] to design the WSI adjacency matrix by assigning different weights to different neighbors. Notably, this approach avoids node traversal and additional storage requirements. WSI_adj is specifically represented as follows:

$$\left\{\begin{array}{c}{H}_{max}=\underset{i}{\overset{K}{||}}max-pooling\left(h_i\right),H_{max}\in R^K\\ H_{\mathit{mean}}=\underset{i}{\overset{K}{||}}mean-pooling\left(h_i\right),H_{\mathit{mean}}\in R^K\\ WSI\_adj=H_{max}{H}_{\mathit{mean}},WSI\_adj\in R^{K\times \text{K}}\\ \end{array}\right)$$ 5

where $\Vert$ denotes a connection. $H_{\mathit{Max}}$ is the max pooling layer output, and $H_{\mathit{Mean}}$ is the mean pooling output. To facilitate a forward comparison of the attention coefficients for different nodes, ${\text{e}}_{\mathit{ij}}$ is subjected to normalization through the softmax function to obtain ${\text{a}}_{\text{ij}}$. We employ a multi-head attention mechanism to stabilize the process of attention learning. Ultimately, the features ${\text{H}}_{\text{G}}\prime$ are obtained through the GAT stream.

MIL aggregation strategy for TGMIL

The outputs from TGMIL’s two attention streams are aggregated using the MIL strategy to map them into the category space. Here, we draw inspiration from the aggregation approach employed in the CLAM_MB model [20] to achieve the category mapping in TGMIL. The specific representation of TGMIL's embedding aggregator is as follows:

$$\left\{\begin{array}{c}{\text{A}}_{\text{i},k}=\frac{exp\left(\text{H}_{_{\left(\text{i}\right)}^{}}^{\prime }\right)}{\sum _{k=1}^{K}exp\left(\text{H}_{_{\left(i\right)k}^{}}^{\prime }\right)}\\ {\text{h}}_{\text{bag},i}=\sum _{k=1}^K{\text{A}}_{\text{i},k}{H}_k\\ {\text{S}}_{\text{bag},\text{i}}={\text{W}}_{c,\text{i}}{\text{h}}_{\text{bag},\text{i}}\\ \end{array}\right)$$ 6

Within the embedding aggregator, N parallel independent classifiers ($W_{c,1}$, $W_{c,2}$, …, $W_{c,N}$) are built for $H^{\prime }$ to obtain the corresponding bag scores for each class. $A_{i,k}$ is the attention score of the kth patch of class i, and $h_{bag,i}\in R^{512}$ serves as a slide-level representation aggregated from the distribution of the attention scores for the ith class. The bag scores are subsequently determined by the classifier $W_{c,i}\in R^{1\times 512}$. Ultimately, the softmax function $S_{\mathit{bag}}$ is applied to compute the predicted probability distribution for each class.

To encourage the model to learn class-specific features, a binary clustering objective is introduced in TGMIL to implement an instance-level classifier. The specific details of the instance-level classifier are given in [20].

Implementation details

TGMIL is trained using the Ranger optimizer [34] with a learning rate of 2e−4 and a weight decay of 1e−5. The batch size is set to 1, and the training period is 100 epochs. We incorporate an early stopping mechanism that halts training if the accuracy does not improve over 20 consecutive validation stages. During training, the random sampling method is used to solve the category imbalance problem. The total loss $\mathcal{L}$ for the TGMIL model is the sum of the WSI-level classification loss ${\mathcal{L}}_{WSI-level}$ and the patch-level clustering loss ${\mathcal{L}}_{patch-level}$. The WSI-level classification loss is computed using the standard cross-entropy loss by the embedding aggregator, and the patch-level clustering loss is calculated using the smooth top-1 SVM loss by the instance-level classifier [35]:

$$\mathcal{L}\ominus =\lambda {\mathcal{L}}_{WSI-level}+\left(1-\lambda \right){\mathcal{L}}_{patch-level}$$ 7

where $\lambda$ is the WSI-level classification loss weight factor, and in this experiment, let $\lambda =0\text{.7}$. All experiments were performed on an NVIDIA A100 80 GB.

Statistics

The area under the curve (AUC) of the receiver operating characteristic (ROC) with a 95% confidence interval (95% CI), accuracy (ACC), precision (Pre), F1 score (F1), sensitivity (SEN), specificity (SPE), and confusion matrices were used to evaluate the diagnosis performance of the TGMIL architecture to classify the differentiation of gastric adenocarcinoma using WSI.

Strengths and limitations of this study

Strengths

1. The TGMIL model eliminates the need for manual annotation, reducing the burden on pathologists.

2. The model integrates contextual and spatial information from WSIs, improving classification accuracy.

3. The study uses a large dataset from two hospitals, enhancing the generalizability of the results.

Limitations

1. The study lacks external validation from independent datasets.

2. The quality of WSIs may vary due to differences in staining and storage conditions.

Abbreviations

WSI

Whole-slide image

GAC

Gastric adenocarcinoma

TNM

Tumor, nodes, and metastasis staging

AJCC

American Joint Cancer Committee

MIL

Multi-instance learning

GAT

Graph attention

AUC

Area under the ROC curve

ROC

Receiver operating characteristic

95% CI

95% Confidence interval

ACC

Accuracy

Pre

Precision

SEN

Sensitivity

SPE

Specificity

Author contributions

First author, Mudan Zhang: conceptualization, funding acquisition, writing - original draft; Co-first author, Xinhuan Sun: formal analysis, methodology; Wuchao Li: data curation, investigation; Yin Cao: data curation; Chen Liu: data curation; Guilan Tu: data curation; Co-corresponding Author, Jian Wang: data curation, writing - review and editing; Corresponding Author, Rongpin Wang: funding acquisition, supervision, writing - review and editing.

Funding

The study was supported by the Provincial Health Commission funding of Commission's Science and Technology (gzwkj2024-210); The Guiyang Science and Technology Project (ZKXM[2020]4); Guizhou Province's "Hundred Levels" plan for high-level innovative talents(GCC [2022] 041-1).

Data availability

The data supporting the findings of this study are available from the corresponding author upon reasonable request. Due to privacy and ethical restrictions, the data are not publicly available.

Declarations

Ethics approval and consent to participate

All procedures were performed in compliance with relevant laws and institutional guidelines. This study was approved by the Ethics Committee of Guizhou Provincial People's Hospital. Due to the study’s retrospective nature, informed consent forms were not required. However, all the images and clinical data were anonymized, and the results of the study did not alter any treatment plans for the subjects. The study was performed according to the principles of the Declaration of Helsinki.

Informed consent

This study was approved by the Ethics Committee of Guizhou Provincial People's Hospital [(2023)223]. Due to the study’s retrospective nature, informed consent forms were waived. However, all the images and clinical data were anonymized, and the results of the study did not alter any treatment plans for the subjects.

Patient consent

Patients or the public WERE NOT involved in the design, or conduct, or reporting, or dissemination plans of our research.

Competing interests

All authors declare that they have no potential competing of interest include employment, consultancies, stock ownership, honoraria, paid expert testimony, patent applications/registrations, and grants or other funding.