Deep Learning Powers Cancer Diagnosis in Digital Pathology

Abstract

Technological innovation has accelerated the pathological diagnostic process for cancer, especially in digitizing histopathology slides and incorporating deep learning-based approaches to mine the subvisual morphometric phenotypes for improving pathology diagnosis. In this perspective paper, we provide an overview on major deep learning approaches for digital pathology and discuss challenges and opportunities of such approaches to aid cancer diagnosis in digital pathology. In particular, the emerging graph neural network may further improve the performance and interpretability of deep learning in digital pathology.

Diagnostic and surgical pathology relies on human interpretations of patterns and morphology. The process is slow, error-prone, and largely unaided by digital technology of the 21st century. Coupling the rapid improvement of deep learning models, vastly enhanced computing power, and reduced storage costs with the rising burden of cancer incidence, the shrinking number of pathologists, the wide-availability of whole slice digitization techniques, and the increasing pathologist case load while reimbursements are dropping, the potential of deploying deep learning in automating the interpretation of digitized pathology slides is tremendous. Tissue or biopsy slides provide an important basis for pathological diagnosis. Many clinical tasks requiring high reproducibility, low fault tolerance, and a high degree of dependence on experienced readers are suitable for applying deep learning for classification, segmentation, or object detection.[1–4]

Before digging in the digital pathology, let us briefly review the history of deep learning from the proposal of the multilayer perception (MP) neuron mathematical model in 1943 [5] to now, where it has shined in several fields. In 1958, Rosenblatt proposed the first generation of neural network single-layer perceptron [6], which can distinguish basic shapes such as triangles and squares. But in 1969, Minsky proved that a single-layer perceptron could not solve the XOR problem and did not have a good learning mechanism.[7] In 1986, the second-generation neural network proposed by Hinton et al.[8], with the back-propagation training method and sigmoid nonlinear mapping, provided an effective solution and triggered the second wave of neural networks. In 1989, LeCun and colleagues [9] invented a convolutional neural network (CNN) and applied it to digital handwriting recognition. However, the backpropagation method has a serious gradient vanishing problem, and the development of neural networks once fell into a trough. Then, in 2006, Hinton proposed a pre-training method by the auto-encoder to suppress the vanishing gradient problem and resurrected interest in CNN [10]. In 2011, the Rectified Linear Unit (ReLU) activation function proposed by Glorot et al. effectively suppressed the vanishing gradient problem [11]. In 2012, Hinton et al. participated in the ImageNet image recognition competition and won the championship through the CNN network AlexNet [12] that far exceeded the classification performance of the second-place support vector machine (SVM) method. Methods such as ReLU and Dropout used in the AlexNet model to suppress the vanishing gradient problem and reduce overfitting can still be found in the recently developed deep learning models. Since then, deep learning has ushered in explosive growth. There are improvements made in the convergence process of deep learning, such as batch normalization [13], transfer learning [14], and skip-connection in ResNet [15–17]. Meanwhile, there are diversification of tasks, such as RNN-based time series deep learning model [18–21], used for image generation generative adversarial networks (GANs) [22–32], object detection models [33–35], natural language processing (NLP) models [36–40], and reinforcement learning models [41–43]. Deep learning systems even surpasses humans in many applications, e.g., Alphago zero [44].

Digital pathology initially referred to digitized histopathology slides using whole slide imaging technology. A large-scale comparison study on tumor specimens from over 1,992 patients concluded that the equivalent diagnostic performance between the pathologists using digitized whole-slide images (WSI) and the pathologists viewing conventional tissue slides [45]. However, studies have shown promise of integrating deep learning-based approaches with human pathologist’s diagnoses in improving diagnostic accuracy while reducing human error rate in cancer diagnosis [46–48]. The fast evolving field of digital pathology now explores the use of deep learning-based approaches for the classification and diagnosis of digitized images [4, 49–52].

For example, Nagpal et al.[53] designed a two-stage deep learning model to help pathologists differentiate between benign and malignant prostate tumors and grade the cancer quantitatively. The first stage was a deep convolutional neural network-based regional Gleason pattern classification. It was trained using 912 slides with 112 million human pathologist-annotated images. At the second stage, the classifications of human pathologists on 1,159 slides were used for training the model. The reference standard included a dataset of 331 slides from 331 patients that were reviewed by independent pathologists and genitourinary pathologists. Performance of the deep learning model vs. 29 human pathologists were compared. The mean accuracy of the deep learning model was 0.7, significantly superior to the accuracy of 0.61 (p = 0.002) by the 29 pathologists. In terms of the Gleason Grade decision thresholds, the receiver operating characteristic (ROC) curves by the deep learning model achieved 0.95–0.96 at each threshold. Especially at Gleason Grade ≥ 4, the deep learning model showed greater sensitivity and specificity than 9 out of 10 pathologists [53]. On the other hand, Mahmood et al.[54] developed a multistage mitotic-cell-detection method based on Faster region convolutional neural network (Faster R-CNN)[55], post-processing and deep convolutional neural networks (deep CNNs) for breast cancer detection. Two breast cancer mitotic cell-detection image datasets, i.e., ICPR 2012 MITOSIS[56] and ICPR 2014[57], were used to train the Faster R-CNN model. In stage 1, an image was input into the trained Faster R-CNN detector. In stage 2, post-processing was performed to reduce the false positives based on statistical, texture, shape, and color features. In stage 3, independently trained Resnet-50[15] and Densenet-201[58] scores were fused by deep CNNs, and the final classification of the mitotic and non-mitotic cells was performed in stage 4 [54]. This method obtained high detection accuracy of mitotic cells in multiple datasets including ICPR 2012, ICPR 2014, and TUPAC16 datasets, and the precision was higher than other ten different methods published previously.

At this point, we can state that CNNs have greatly improved the diagnosis based on histopathological slides in digital pathology [59]. However, in most situations the image is Euclidean while the content is usually not. Traditional CNN needs to learn the content through multiple layers. Due to cropping, however, traditional CNN cannot obtain information outside the field of view, resulting in performance degradation. Moreover, it is precisely because of this process of extracting non-Euclidean information that the learning process of CNN is difficult to comprehend. There are many research papers regarding the interpretability of deep learning [60–63], and notably the emerging development of graph neural networks (GNNs) has brought us a promising path [64, 65].

The first appearance of graphs in the field of mathematics can be traced back to the Königsberg bridges problem [66]. A graph is a kind of data structure that models a set of objects (nodes) and their relationships (edges). It can be stated that GNN is to graphs what CNN is to image processing. In the development of GNN, traditional deep learning methods have also been analogously applied to GNN, such as Attention mechanism [67], Long short-term memory (LSTM) control gate [68], etc. GNN is widely used in social networks, recommendation systems, physical systems, chemical molecular properties prediction, knowledge maps, and many other fields [64]. As a data structure with powerful performance capabilities, graphs combining with deep learning would become the next hot spot in the digital pathology.

The feature extraction ability of GNN is also stronger than CNN. For instance, Yanning et al. [69] developed automatic methods for grading colorectal cancer (CRC). The authors stated that CRC grading usually uses small image patches, and it is impossible to obtain information on the micro-architecture of the entire tissue during the grading process. The authors proposed a cell-graph convolutional neural network (CGC-Net) model that combines the Adaptive GraphSAGE graph convolution structure to obtain multi-level features[70]. In comparison with traditional CNN, the proposed model achieves much better accuracy of 97%. Under the theme of interpretability, GNN clearly shows its understanding of the relationship between objects in the graph without the support of additional models. For example, Mookund Sureka et al.[71] pointed out that the medical community needs interpretable models to conduct due diligence and improve understanding of disease mechanisms and treatment, which is difficult for CNN to do. Thus, the authors adopted an approach to model a histology tissue as a graph of nuclei and developed a GNN framework based on the attention mechanism and node occlusion for disease diagnosis. The method highlights the relative contribution of each cell nucleus in the image and fits the mental model of the pathologists.

In summary, recent development and applications of deep learning have progressively transformed digital pathology, particularly with promising GNN models. Although GNN has certain shortcomings, such as over-smoothing and scalability, the relatively easy-to-analyze structure and stronger generalization ability of GNN reveal its potential in digital pathology.

Highlights.

• The article emphasizes the application of AI in digital pathology. In the past, present and future, AI will continue to change the pathology field to make it more efficient, accurate, and interpretable.

• The article enumerates the application of AI in digital pathology and demonstrates the improvement of AI in this field.

• The emergence of graph neural network makes it possible for AI to be interpretable while being efficient and accurate.