Pathological diagnosis and prognosis of Gastric cancer through a multi-instance learning method

Gastric cancer (GC) is one of the most common malignant tumors with poor prognostic results. Endoscopic examination is mainly utilized for early detection, while pathological confirmation and CT scanning are suggested for further treatment. Machine learning and deep learning methods have been widely applied to explore automatic GC diagnosis and prognosis analysis. Endoscopic images based deep learning models for early GC detecting have achieved a performance equal with experienced endoscopists [1] and radiomic methods on CT images have proposed clinically significant imaging biomarkers with diagnostic and prognostic values [2], [3], [4]. In this article of Ebiomedicine, Huang et al. [5] proposed a simple multi-instance learning (MIL) approach for GC diagnosis and prognosis analysis on whole slide imaging (WSI) pathological images. Experiments on three datasets achieved good performances for both tasks.

The researchers collected a total of 2508 pathological images from 1128 patients and cropped the images into small tiles (224 × 224 pixels). Image-level diagnosis by pathologists was treated as labels for tiles. RegNetY, an improved model from ResNet, extracted features from each tile, followed by aggregation models to fuse features from the most significant tiles. They adopted a recurrent neural network (RNN) to merge features from different tiles for the diagnostic model, achieving an accuracy of 0.976 and 0.920 in the internal and external validation datasets, respectively. Multi-layer perception (MLP) was trained and validated for prognosis with a C-index of 0.671 and 0.657. Besides, the predicted risk was a strong predictor for survival prediction.

The traditional diagnosis of pathological images, relying on pathologists’ visual observation, is time-consuming and labor-intensive. Multiple explorations for GC prediction on pathological images have emerged nowadays. In practice, the WSI image is cropped into small tiles and trained via a weakly supervised training method to deal with its large size. During training, the challenges are as follows: extracting features from tiles and merging these tile-level features to make image-level predictions. For GC detection [6,7], researchers applied ResNet for feature extraction at tile-level and semantic segmentation models to detect GC regions at image-level, which requires delineating the margins of the tumor region on WSIs by pathologists. Muti et al. [8] used ShuffleNet for tile-level prediction and the fraction of predicted positive tiles as the patient-level prediction score for GC classification. Besides, Wang et al. [9] developed a framework including segmentation, classification, and risk quantification to predict the prognosis of GC using lymph node pathological images. Compared with previous related works, Huang et al. adopted a relatively simple MIL strategy with RNN and MLP but demonstrated powerful performance in GC diagnosis and prognosis analysis.

Determining the best practice of MIL in pathological images is still an open question. There are semantic segmentation models such as DeepLab to detect tumor regions, the fraction of predicted positive tiles for molecular subtyping, merge top-k features using sequential models or other simple models, and advanced MIL approaches [10]. Choosing the MIL strategy for pathological images is task-determinant, requiring previous experience and explorative experiments. Huang et al. used RNN for diagnosis but MLP for prognosis analysis. As explained by the researchers, a sequential model is more applicable to eliminate the effect of outliers, while MLP could be more efficient in the prediction based on all tiles. Interestingly, when researchers tried RNN algorithms in prognosis, the model could hardly converge, resulting in poor performance with a nearly 0.5 C-index. These results are consistent with the intuitive idea that the general observation on most tiles could predict diagnosis, but prognosis might require considering extreme tiles. Moreover, the results showed the potential of simple models to achieve good performance, guiding us to reconsider the model design for MIL practices in WSI images.

The research of Huang et al. provided a new solution for diagnosis and prognosis analysis of GC based on pathological images, and also raised several open questions for future research. First, the number k of selected top-k significant tiles could largely influence model performance. However, it lacked a determined method for choosing this value. There is a trade-off between sufficient coverage and superfluous insignificant tiles. Generally, it relates to the tumor type, WSI magnification, tile size, fusion model, and specific tasks, thus no one-fit-all solution exists. Comparative experiments and discussions for different parameter settings might provide a deeper insight into the problem. Also, it could be treated as a tuning parameter to be selected by cross-validation. Second, RNN for diagnosis and MLP for prognosis analysis were applied in the research. Nevertheless, its power in other datasets and tasks is open for exploration. What about the general ability of the approach to other datasets and tasks? Can it outperform advanced MIL techniques? What are the similarities or differences between the tasks that facilitate or hinder the utilization? These explorations might guide rethinking the complexities and generality of the model in the MIL scenario, and may also provide more insights into tumor diagnosis and prognosis.

Contributors

All authors contributed to conceptualization, writing, reviewing, editing and have read and agreed to the published version of the manuscript.

Declaration of Competing Interest

The authors declare no conflict of interest.