Artificial intelligence at the intersection of pathology and radiology in prostate cancer

Abstract

Pathologic grading plays a key role in prostate cancer risk stratification and treatment selection, traditionally assessed from systemic core needle biopsies sampled throughout the prostate gland. Multiparametric magnetic resonance imaging (mpMRI) has become a well-established clinical tool for detecting and localizing prostate cancer. However, both pathologic and radiologic assessment suffer from poor reproducibility among readers. Artificial intelligence (AI) methods show promise in aiding the detection and assessment of imaging-based tasks, dependent on the curation of high-quality training sets. This review provides an overview of recent advances in AI applied to mpMRI and digital pathology in prostate cancer which enable advanced characterization of disease through combined radiology-pathology assessment.

Prostate cancer is the sixth most common cancer worldwide and the second leading cause of cancer death in American men (1). Prostate cancer has a diverse range of risk based upon disease presentation, ranging from low-risk indolent to high-risk aggressive disease. Early detection and localization of prostate cancer at a treatable stage is important as excellent cancer-specific survival is expected for most locally confined diseases. The most common method to diagnose prostate cancer uses transrectal ultrasonography (TRUS) to sample twelve regions distributed throughout the prostate, most often recommended after an elevated prostate specific antigen (PSA) blood test or an abnormal digital rectal exam (2). Common treatments for patients with localized intermediate and high-risk disease include radical prostatectomy and radiation therapy, whereas active surveillance is commonly considered for patients harboring low-risk disease. Selecting among these treatment options is difficult due to the clinical and biologic heterogeneity of prostate cancer disease.

Clinical risk assessment is primarily determined by pathologic grade based on the Gleason grading system, as initially proposed in the 1960s and modified multiple times thereafter (3). While the Gleason grading system has limitations, it generally captures the full range of biologic aggressiveness and has historically correlated with the risk of recurrent cancer after definitive treatment (4). Gleason pattern 3 consists of well-formed, individual glands of various sizes. Cancers containing only Gleason pattern 3 (i.e., overall Gleason score assignment 3+3) are generally considered low risk. Gleason pattern 4 includes poorly formed, fused, and cribriform glands. Within the spectrum of intermediate-risk disease, the presence of cribriform pattern in radical prostatectomy specimens has shown association to poor surgical and clinical endpoints, including higher rates of extra-prostatic extension, positive surgical margins, biochemical recurrence, and cancer-specific mortality (5, 6). Gleason pattern 5 consists of high-risk histologic patterns including sheets of tumor, individual cells, and cords of cells. In general, tumors containing only Gleason 3+3 have a minimal risk of progression to metastatic cancer, while tumors with dominant Gleason patterns 4 and 5 cancers carry a higher likelihood of progression to metastatic disease and cancer-specific mortality. Prostate cancer is known to exhibit heterogeneous distribution of morphologies both within and across tumors of the same patient, with increasing number of observed morphologies in larger cancers of worsening grade (7). Due to this diversity, the final grade as reported by the Gleason system is based on the combination of the most dominant and second most dominant patterns observed. In some settings a tertiary pattern, otherwise defined as a minor component of high-grade cancer, is reported in combination with the overall Gleason score when a small population of high-grade glands are observed upon evaluation of radical prostatectomy specimens. Although it is central in risk assessment and patient management for localized prostate cancer, the Gleason scoring system is known to suffer from high variability across pathologists, and even within the same pathologist, due to the subjective nature of scoring assignment (8, 9).

Multiparametric magnetic resonance imaging (mpMRI) is well-established as an important aide for prostate cancer diagnosis. mpMRI provides high-contrast, high-resolution anatomical images of the prostate and pelvic regions using T2-weighted imaging, diffusion-weighted imaging (DWI), and dynamic contrast enhanced (DCE) imaging sequences for the detection of clinically significant prostate cancers (10). However, the combined assessment and interpretation of all mpMRI sequences requires a considerable level of expertise from radiologists. The current classification of abnormal regions on mpMRI imaging is defined by the Prostate Imaging Reporting and Detection System version 2 (PI-RADSv2) (11). This system, while widely accepted, is also prone to inter-reader variability, leading to variable sensitivity, specificity, and a wide range of cancer detection rates (12–14). Despite these shortcomings, mpMRI has shown practice-changing value for urologists to more accurately direct needle biopsies into suspicious regions of the prostate and superior performance in disease sampling compared with TRUS biopsy techniques (15–17). However, even with targeting based on mpMRI findings, biopsy sampling error and tumor heterogeneity leads to inaccurate pathologic characterization of final whole-organ assessment after surgery.

Unique from many other primary cancer models, the surgical practice of radical removal of the entire prostate gland in localized disease allows for 1:1 spatial correspondence between radiologic imaging and pathologic imaging. This has led to a large body of literature characterizing tissue characteristics with functional imaging characteristics of mpMRI, most prominently using DWI imaging characteristics such as the apparent diffusion coefficient (ADC). Restriction of water molecules leads to differential signal properties of DWI, specifically reflecting the increased cellular density and nuclear-to-cytoplasmic ratio disturbed in cancerous tissues. In prostate cancer, several studies have established a negative relationship between ADC values and tumor Gleason score (18–20). However, the outright use of quantitative DWI characteristics for lesion detection is limited by the lack of clear distinction in ADC values of normal and tumor regions (21). Furthermore, the relationship between ADC and tumor-level Gleason score overall remains moderate, likely due to local heterogeneity in the proportion (density) and spatial distribution (sparsity) of cancerous cells within the tumor (22). Truong et al. (23) have reported preliminary evidence that advanced morphologic features, such as cribriform, influences the visibility of prostate lesions more so than tumors of the same grade with differing architectural patterns.

Overall, the complex heterogeneity of localized prostate cancer has led to variability in clinical practice for both radiology and pathology. The use of artificial intelligence (AI) applications in both radiology and pathology has increased substantially in the last decade (24), including more recently with the adoption of deep learning techniques applied to medical imaging. Here, we provide a summary of recent AI applications to radiology, pathology, and combined radiology-pathology characterization of localized prostate cancer where the intrinsic connection between histology and functional imaging in prostate cancer motivates the use of combined techniques for enhanced characterization.

Current uses of artificial intelligence in localized prostate cancer

Radiologic applications

In general, AI shows great promise in decreasing reader interpretation times, increasing performance of non-expert radiologists, and enabling large-scale screening practices without additional burden to radiologists. Applications of AI to prostate mpMRI are specifically expected to increase sensitivity of prostate cancer detection and decrease inter-reader variability (25). There are several examples of AI-based systems for prostate mp-MRI in the literature. In general, mp-MRI based AI tasks either focus on automated detection, aiming to localize suspicious regions within an image set, or automated diagnostic classification, aiming to predict the aggressiveness of a region of interest.

Traditional machine-learning based detection systems typically require several pre-processing steps, including extraction of prostate gland regions and quantitative metrics from MRI sequences. To date, several studies have evaluated in-house AI algorithms developed for automated detection on prostate cancer on mpMRI (26–31). Despite differences in terms of features used for voxel and candidate region classification, MRI modalities, and methods used for classification and multimodal fusion, these studies demonstrate robust detection on the order of 75% to 80% or more, within the range of reported radiologist performance (11). In a recent multi-reader, multi-institutional study, Gaur et al. (32) have shown in their multi-institution study that AI-based detection, from algorithm presented in (33), improved specificity in conjunction with PI-RADSv2 categorization as well as slightly improved radiologist efficiency, and they found an index lesion sensitivity for PIRADSv2 ≥3 of 78%. Here the greatest benefit was seen in the TZ where it helped moderately experienced readers to achieve 83.8% sensitivity with automated detection versus 66.9% with mpMRI alone. Litjens et al. (28) were able to demonstrate the combining AI-based prediction and PI-RADS improved both detection of cancer, as well as discrimination of clinically revelant (i.e., aggressive) disease.

While fewer studies have reported on the use of deep learning algorithms for automated detection of prostate cancer on mpMRI, early literature supports improved detection rates compared with previous works (34–36). Notably, Song et al. (37) report high diagnostic performance of 87% sensitivity in a cohort of 195 patients. Here, combination of automated detection and with expert radiologist PI-RADSv2 classification improved the detection of clinically significant cancer than with the radiologist alone. Multi-task algorithms, such as the method reported by Alkadi et al. (38), combine prostate gland segmentation for spatial context with lesion detection for improved performance. Yang et al. (39) report novel work demonstrating separate development and training on T2-weighted and ADC images improved detection performance when compared with methods learning across image sets. As the literature matures, it is expected that further increase in performance will be observed.

While these examples utilize various techniques and imaging features, they have all demonstrated promising ability to detect prostate cancer on mpMRI. However, they are inherently limited by training against a pathology validation set in which only a single Gleason score is assigned to each lesion, so called “weak labels”, that underestimate known intra-lesion heterogeneity (Fig.). This has resulted in underperformance of algorithms for certain pathologic grades (33). For these reasons, current AI algorithms are not ideally suited for intra-lesion spatial targeting of aggressive regions on mpMRI at biopsy. Ideally, voxel-based classifiers such as those previously described should be used to train the algorithm against a system that defines each “patch” of tumor in terms of its regional Gleason score to allow for more accurate spatial learning.

Figure.

Example mpMRI (T2-weighted and ADC images) with corresponding whole mount hematoxylin-eosin (H&E) section demonstrating two pathologically-defined tumors. Overall assessment in this patient resulted in Gleason grade 3+4 score assignment, resulting in weak labels derived from total tumor extent. However, detailed pathologic assessment demonstrates majority of Gleason pattern 4 is located in the anterior portion of the tumor, with predominately Gleason pattern 3 throughout remainder of the tumor field. Opportunities for improved spatial learning include density mapping of dominant pathologic grading and exclusion of non-cancerous structures within tumor field.

AI-enabled pathology-to-radiology correlation

To improve spatial labeling at the mp-MRI level, detailed pathologic information of each lesion would theoretically be required as input. Spatial annotation of this level is impractical due to the time-consuming nature of spatially annotating at the full resolution of digital pathology images, which can range up to 10 GB on the order of 100 000×100 000 pixels per section for whole mount prostate specimens (40). For these reasons, historical applications of AI techniques have included approaches for semantic segmentation of histology slides, such as automated detection and segmentation of all nuclei and tissue components to allow for further classification and/or analysis of diseased regions (41).

Commonly used machine learning workflows in prostate cancer rely heavily on semantic segmentation of epithelial, stromal, and lumen components on tissue slides due to the varying representation of each within normal and malignant structures (42). Gertych et al. (43) developed machine learning techniques for differentiating stroma and epithelial components, followed by classifier of benign and malignant regions using quantitative features derived from epithelial components. Gorelick et al. (44) reported improved classification of malignant tissue when including segmentation of lumen. Others have focused on segmentation of entire glandular structures (45), which have been used to enable morphology quantification for further classification of malignant tissues (46, 47). More recently, Li et al. (48) were able to demonstrate that an expectation-maximization approach to deep learning can provide improved gland segmentation in prostate cancer.

These applications have additionally shown promise for serving as pre-processing steps to enable more complex histopathologic spatial analysis with radiologic imaging. Kwak et al. (31) have shown that MRI signal characteristics are significantly associated with tissue composition density derived from automated tissue-level compartment segmentation of stroma, epithelium, epithelial nuclei, and lumen. In a similar analysis, McGarry et al. (49) shows that radiopathomic maps of epithelium and lumen density can be used to distinguish cancerous regions of varying grade. The use of AI for correlation between pathologic features and radiologic signature has shown that both histologic grade and mpMRI diffusion characteristics correlate more strongly with glandular components and crowding than with nuclear count and crowding (50–52). Chatterjee et al. (50) demonstrated that differences between cancers with Gleason patterns 3, 4, and 5 were also greater for the gland component volumes than for the cellularity metrics. Meanwhile, Hectors et al. (51) were able to establish a stronger relationship using advanced diffusion models mimicking human tissue. Importantly, establishing the relationship between tissue components and imaging signatures across these studies were achieved utilizing multiple different algorithms applied to pathologic data, providing strong evidence for distinct biophysical foundations for radiologic signatures and motivating utility and development of new imaging techniques (53).

Advanced pathologic applications

In several cancers, automated detection and quantification of tissue components demonstrate utility in predicting patient outcome. For example, nuclear and stromal features have shown correlation to patient prognosis in breast cancer (54, 55). Tumor infiltrating lymphocyte detection via deep learning models allows for spatial analysis of immune profiling, demonstrating independent prediction of treatment outcomes (56). However, these techniques so far have not demonstrated value in prostate cancer, where the major focus has turned to automated Gleason grading on digital pathologic assessment.

Despite increasing popularity of machine learning applications to complex medical imaging problems, automated algorithms for Gleason grading demonstrate limited success largely because of the subjectivity of the system (57–59). This includes sub-optimal prediction in binary classification of low- versus high-risk disease, with some studies showing highest error rates in the group of intermediate-to-high risk prostate cancers (57). However, these machine learning methods are highly sensitive to multiple pre-processing and feature extraction steps. Traditional machine learning methods have relied on building classifiers from handcrafted features derived in relatively homogeneous settings, such as those derived in tissue microarrays. This leads to lack of generalizability to whole slide images of heterogeneous tissue components. More recent advances in deep learning applications have shown promise for improved performance of prostate cancer grading algorithms.

Work by Nir et al. (60) evaluating multiple machine and deep learning methods demonstrate best results of 92% accuracy in cancer detection and 79% accuracy in classification of low and high grade prostate cancers when trained on tissue micro-array data, though agreement with expert pathologists was moderately low and overall system accuracy suffered when validated in an external dataset of whole-mount images. Other preliminary works differentiating low-to-intermediate (Gleason Grade ≤ 3+4) from unfavorable intermediate-to-high risk (Gleason Grade ≥ 4+3) only achieved 75% classification accuracy (61). Improved classification was found when performance was evaluated in a set of well-curated tissue microarrays, where Arvanti et al. (62) reported 70% overall sensitivity in classifying tissues as benign, Grade 3, Grade 4, or Grade 5. Here, Grade 5 achieved the highest per-class recall of 88% in contrast to 58% in Grade 4. Despite this, the deep learning model was able to achieve high agreement with pathologist interpretations (kappa, 0.71–0.75). High level annotations derived within 40 patients were used to build a custom network for both detection and grading of prostate cancer, demonstrating 99% epithelial detection accuracy and 71%–79% accuracy for low- and high-grade disease (63). In the largest study to date, Nagpal et al. (64) have developed and validated their deep learning algorithm against a cohort of 29 board-certified pathologists in Gleason scoring whole-slide images from prostatectomy patients and was able to provide more accurate quantitation of Gleason patterns and better risk stratification for prediction of biochemical recurrence. Even still, performance remains 70% for Gleason grading.

Remaining challenges for AI applications in pathology include stain normalization and color matching (65). Heterogeneity in these properties are underexplored due to the small nature of most datasets currently available. As noted, both machine learning and deep learning suffer in the cohort of intermediate-risk prostate cancers. However, given that morphology and classification pattern tasks vary widely across differing tumor types, direct application of algorithms developed in other tissue types may not translate. Computationally, reading in the entire image is prohibitively expensive and limits most AI applications to patch-based techniques. As technology continues to increase computing power, new opportunities for image-based tasks at full-scale resolution will become available.

Future opportunities for integrated techniques, radiogenomics, and beyond

Currently, both radiology and pathology are limited in their ability to detect and classify intermediate prostate cancers, which represent a critical point in clinical treatment decision-making. Here, opportunities for new integrated techniques combining imaging modalities for outcome prediction could be readily developed as datasets mature. However, as discussed, there is a lack of high-quality annotations and most models are constructed using weakly labeled data ignoring tissue heterogeneity. Improving performance of pathologic classifiers can serve as the basis for high-resolution radiologic labels, whereby pixel-wise predication of prostate cancers can replace current weak labels. Furthermore, the promise of integrated techniques comes with the idea that improved pathologic prediction and grading can lead to improved radiologic prediction.

While outside the scope of this review, new development of AI frameworks demonstrate that proof-of-principle integration of diverse data is feasible. Methods for coupling weak and strong labeled data demonstrated the highest level discrimination of low risk (Gleason ≤ 7) from high risk (Gleason ≥ 8) in digital pathology assessment of prostate cancer specimens (62). Combined machine learning and deep learning techniques for longevity prediction show proof-of-principle for combining multi-dimensional data for outcome prediction (66). Ensemble-based methods of cascaded deep learning algorithms can be readily used to evaluate the utility of combined prediction of radiologic and pathologic prediction for patient outcome or prediction of disease risk, including molecular characterization. Recently, novel integration of pathologic features and genomic characteristics for prediction of cancer outcomes further demonstrates a role for AI methods in integrated precision medicine initiatives (67).

In the combination of multi-modality multi-dimensional data, pathologic imaging and evaluation represents an ideal intermediary step to enable further radiogenomic associations. Radiogenomics, i.e., the correlation of radiographic signatures with genomic features, has been evaluated for prostate cancer in several small studies (68). Following radiogenomic work in glioma by Smedley et al. (69), there is an opportunity to apply deep learning for discovery-based prediction within radiogenomic frameworks. Recently published work demonstrates widespread transcriptome heterogeneity within prostate cancer (70). This formative work by Berglund et al. (71) demonstrates differential gene expression across regions of the tumor microenvironment, particularly in peripheral vs. more central regions of the tumor, of a single patient. However, the degree to which this results in differential growth patterns remains largely unexplored. Preliminary work correlating spatially distinct mpMRI normal and suspicious regions with whole exome sequencing additionally demonstrate heterogeneity within prostate lesions.

Conclusion

In conclusion, there exists a clinical need in prostate cancer for improvement in standardized assessment and characterization on pathologic and radiologic interpretation. The intimate connection between histology and functional imaging characteristics creates a unique opportunity for AI applications to improve detection, classification, and overall prognostication. As demonstrated across AI literature, the need for mature datasets with high quality annotations are necessary to continue advancements in this field.

Main points.

• Recent studies show artificial intelligence (AI) has potential to aide in detection and classification of prostate cancer on radiology and pathology imaging.

• Current radiology AI applications suffer from training data that does not reflect the heterogeneity in tissue composition observed on pathology.

• Pathology-based AI applications enable high quality radiology-pathology correlation and have the potential to serve as input to train improved radiology-based AI algorithms.