Radiomics and machine learning applications in rectal cancer: Current update and future perspectives

Abstract

The high incidence of rectal cancer in both sexes makes it one of the most common tumors, with significant morbidity and mortality rates. To define the best treatment option and optimize patient outcome, several rectal cancer biological variables must be evaluated. Currently, medical imaging plays a crucial role in the characterization of this disease, and it often requires a multimodal approach. Magnetic resonance imaging is the first-choice imaging modality for local staging and restaging and can be used to detect high-risk prognostic factors. Computed tomography is widely adopted for the detection of distant metastases. However, conventional imaging has recognized limitations, and many rectal cancer characteristics remain assessable only after surgery and histopathology evaluation. There is a growing interest in artificial intelligence applications in medicine, and imaging is by no means an exception. The introduction of radiomics, which allows the extraction of quantitative features that reflect tumor heterogeneity, allows the mining of data in medical images and paved the way for the identification of potential new imaging biomarkers. To manage such a huge amount of data, the use of machine learning algorithms has been proposed. Indeed, without prior explicit programming, they can be employed to build prediction models to support clinical decision making. In this review, current applications and future perspectives of artificial intelligence in medical imaging of rectal cancer are presented, with an imaging modality-based approach and a keen eye on unsolved issues. The results are promising, but the road ahead for translation in clinical practice is rather long.

Core Tip: Rectal cancer is a common malignancy requiring a multidisciplinary approach to ensure the best clinical management. Diagnostic imaging has contributed to increased survival rates and provided crucial information on the course of rectal cancer patients. Artificial intelligence, and in particular radiomics and machine learning, are promising techniques that could further enhance the value of medical imaging, allowing the building of decision support tools based on quantitative data. We herein present and discuss the potential role of artificial intelligence in rectal cancer applied to different medical imaging modalities.

INTRODUCTION

In 2020, more than 40,000 cases of rectal cancer (RC) were expected in the United States alone, with a higher incidence in men than in women and a median age at diagnosis of 63 years[1]. However, over the past years there has been an improvement in RC management associated with a reduction of mortality and higher survival rates, mainly related to earlier diagnosis and more effective treatment[2]. While endoscopy represents the gold standard for RC diagnosis, there are several factors to be considered that influence prognosis and therapeutic strategy, including local tumor extent (T), lymph nodes status (N) and presence of distant metastases (M)[3]. Indeed, radical surgery with curative intent (i.e. total mesorectal excision, TME) is recommended as a first-line strategy in patients with locally confined disease after neoadjuvant chemoradiotherapy (nCRT) for locally advanced RC (LARC). Metastatic patients, on the other hand, usually undergo systemic therapies such as chemotherapy, targeted therapy, or immunotherapy[4,5]. Diagnostic imaging plays a crucial role for pretreatment disease staging, with a multimodal approach commonly being necessary[6]. Magnetic resonance imaging (MRI) is regarded as the most valuable imaging modality for primary loco-regional staging of RC and restaging after nCRT[7,8]. Computed tomography (CT) scans are routinely performed to detect distant metastases, with the most common metastatic sites being the liver and lungs[2]. Currently, hybrid imaging by positron emission tomography/CT (PET/CT) could provide useful prognostic data for RC, even if its role still remains to be defined[6,9]. Likewise, the potential of simultaneously acquired PET and MRI still has to be explored[10]. However, conventional image assessment has recognized limitations that are driving the research towards the identification and validation of novel strategies to further increase the value of diagnostic imaging[11-13]. In this setting, a post processing quantitative technique known as radiomics appears particularly promising, with encouraging evidence collected in recent years[14,15]. Radiomics has been frequently and successfully coupled with artificial intelligence (AI), and in particular machine learning (ML) approaches in the field of oncologic imaging[16-19]. This review aims to introduce readers to the concepts of radiomics and ML and to present the state-of-the-art of RC radiomics-ML applications, with an imaging modality-based approach, highlighting their strengths and drawbacks.

RADIOMICS AND ML: WHAT, WHY, AND HOW

Trying to quantify what is visually assessed in medical imaging is a rather difficult task, and radiologists have traditionally provided qualitative information and semi-quantitative data in their reports[20]. However, this leads to a large amount of unused data remaining hidden in medical images[21]. Furthermore, semantic descriptors of cancer imaging phenotype (e.g., “central necrosis”, “irregular margin”, and “diffusely heterogeneous”) are prone to poor intra- and interobserver reliability, experience dependent, and might not significantly reflect actual tumor biology[22]. Indeed, tumors are not considered to be homogeneous entities but instead composed of various cell clones with biologically relevant differences[23]. Radiomics allows the conversion of images into mineable data with the high-throughput extraction of quantitative parameters (i.e. radiomics features) that capture the heterogeneities and provide important information on cancer phenotype[24]. Radiomics is a multistep process beginning with image acquisition and followed by image segmentation, which is the two- or three-dimensional delineation of the region of interest (ROI), usually represented by the primary tumor. Image segmentation can be manual, performed by a human operator; semiautomatic, performed by AI and manually adjusted; or automatic, exclusively performed by AI[25] . Subsequently, hundreds of radiomics features can be extracted from the ROI using specifically designed formulae conveying different information, including shape, first-order (based on the distribution of pixel intensities), second and higher-order features (accounting for pixel intensities spatial distribution)[26]. Correlating radiomics features to the outcomes of interest is the endpoint of radiomics, and many believe it could open the gateway to precision medicine[27,28]. However, such a huge amount of data can be more easily handled by AI rather than traditional statistical methods[21]. Indeed, ML is a branch of AI focused on algorithms that can be trained for a task they were not specifically programmed to perform[29]. The algorithms are essentially used for classification problems, with the main oncologic imaging application being decision support in various settings that include detection, characterization, and monitoring[30-32]. To properly train an ML algorithm, “the curse of dimensionality,” which is a set of issues arising when working with a number of features much higher than the patient population must be avoided. Feature reduction can be achieved in several ways that may also be combined to achieve better results[33,34]. Indeed, an excessive number of features increases the chances of finding nongeneralizable correlations (i.e. overfitting). On the other hand, complex relationships might need more features to build a proper prediction model[35]. Finally, trained ML classifiers need to be tested to verify generalizability on external data not used in the training process and possibly provided by different institutions[36]. A kind of ML algorithm called deep learning (DL), based on neural networks (NN), does not necessarily require image segmentation and learns autonomously the best features for performing data classification[37]. A brief description of the most commonly applied ML algorithms in RC radiomics can be found in Table 1.

Table 1.

Overview of the most widely adopted machine learning algorithms in rectal cancer imaging

Algorithm name	Description
Random forest	An ensemble method that combines multiple decision trees (a class of predictive learning models used in supervised ML) to obtain more accurate results for classification and regression tasks
Support vector machine	A linear approach used mainly for classification problems with the aim to find the best hyper plane which most accurately separate input data into two classes
Logistic regression	A classifier used to obtain the best fitting model for the relationship between multiple predictor variables and a dichotomous outcome
LASSO	A regularized regression method that performs both variable selection and regularization in order to optimally fit the resulting generalized statistical model
Naive Bayes	A classifier relying on the Bayes Theorem to model the probability of an outcome based on the strong (naive) independence assumptions between the features data
Quadratic discriminant analysis	A subtype of Dimensionality Reduction Algorithms that turn high-dimensional data into to low-dimensional data retaining the most significant features of original data for the prediction of the class label
ANN	A subgroup of ML composed of neuronal-like multi-layered networks allowing to automatically extract features without prior labelling and perform complex operations
CNN	As subset of ANN containing multiple computational hidden layers that filter and compute high-dimensional data to enhance the learning of high-level tasks (deep learning)

ANN: Artificial neural network; CNN: Convolutional neural network; LASSO: Least absolute shrinkage and selection operator; ML: Machine learning.

RADIOMICS AND ML APPLICATIONS IN RC: MRI

Thanks to its superb contrast resolution, MRI plays a pivotal role in the diagnostic pathway of RC patients, particularly for primary local staging and restaging after treatment[38]. Indeed, in addition to T and N staging, MRI provides valuable information such as the circumferential resection margin, defined as the minimum distance between the tumor and the mesorectal fascia, as well as extramural venous invasion (EMVI), an independent negative prognostic factor for RC[39,40]. In the following paragraphs, radiomics and ML approaches proposed to further increase the value of MRI in the assessment of RC are described.

Staging

Currently, MRI represents the first-choice imaging modality for determining RC local extent. However, the assessment of T stage is a challenging task, and staging failures often occur in the differentiation between T2 in which the tumor involves the muscularis propria and T3, in which the tumor involves perirectal tissue beyond the muscularis propria[41]. Decision support tools based on MRI radiomics and ML might be able to aid radiologists in this endeavor[42-44]. Using multilayer perceptron, a DL model powered by T2-weighted (T2w) radiomics features from pretreatment MRI, Ma et al[42] were able to discriminate between patients with T1 or T2 and those with T3 or T4 RC with 76% sensitivity and 74% specificity. Similar results were found using diffusion-weighted imaging (DWI) to extract radiomics features in a recent investigation on 115 patients. A logistic regression (LR) algorithm reached a sensitivity of 79% and a specificity of 74% for the same classification problem[43]. Finally, an LR model built with T2w images, both with and without fa-suppression, radiomics features achieved a sensitivity of 88% and specificity of 61% for classifying T1-2 vs T3-4 in a group of 174 patients[44].

MRI is also considered the imaging gold standard for the assessment of lymph node involvement in RC, but it suffers from a relatively low specificity, with potential negative implications on patient outcome[45]. Indeed, the management of patients with different nodal status is a highly debated and complex topic[46]. Radiomics has been proposed as a feasible solution to enhance the accuracy of MRI for N staging in RC patients[47]. In a recent retrospective single-center study in 152 patients, T2w radiomics were coupled to a random forest (RF) algorithm to create an ML classifier that was able to discriminate N0 from N1-2 patients with a sensitivity of 79% and a specificity of 72%[42]. Once again, similar results (81% sensitivity and 68% specificity) were found with LR and a different ML model derived from DWI radiomics features[43]. In both studies, pretreatment MRI scans were used, and the primary tumor was segmented. With a different approach, Zhu et al[48] extracted collective radiomics features from all noticeable lymph nodes on T2w images acquired before and after nCRT in patients with LARC; the LR model was trained to predict pathological node status after nCRT with a group of 143 patients, and had a sensitivity of 95% and a specificity of 60% in the validation cohort of 72 patients. The sensitivity was slightly lower and the specificity slightly higher than those reached by radiologist in the same patient cohorts (100% and 43%). Notwithstanding the specificity insufficient for clinical needs, such models might be useful tools for radiologists in the assessment of N stage in RC.

Finally, the identification of distant metastases in RC patients usually relies on imaging modalities other than MRI. Nevertheless, it should be mentioned that radiomics of the primary tumor was able to provide valuable information for the prediction of synchronous (already present at the time of diagnosis) or metachronous (developed after treatment) liver metastases[49-51] as well as synchronous metastases to other sites[52]. With specific regard to metachronous liver metastases, radiomics of T2w and post-contrast T1-weighted dynamic contrast enhanced (DCE) images were combined to build two ML predictive models, a support vector machine (SVM) and LR, with cross-validation in 108 patients[50]. The LR algorithm had the best performance, but not significantly better than SVM, with 83% sensitivity and 76% specificity, confirming the potential of radiomics and ML for the identification of RC patients who will develop liver metastases after treatment.

Predicting response to nCRT in patients with LARC

While TME should follow nCRT in patients with LARC, the role of surgery in patients with a complete response to nCRT is currently debated, and a “watch and wait” strategy has been proposed[53]. Indeed, patients who achieve a pathological complete response (pCR) after nCRT have better long-term outcomes compared with non-pCR patients, and could therefore be managed differently[54]. Unfortunately, pCR cannot be accurately predicted before surgery by conventional evaluation of MR images[55]. Recently, several radiomics features extracted from T2w, DWI, and DCE sequences have been investigated as possible imaging biomarkers for pCR prediction, with promising results[56-58]. The main studies that aimed to build classification models using ML algorithms for preoperative prediction of pCR after nCRT are shown in Table 2. Overall, the performance of the different models is encouraging. While a trend can be observed, with lower values found in those studies that validated the model in an external dataset and thus with the better chances of high generalizability, it is difficult to draw a final conclusion from the available evidence[59,60]. Most studies focused on MRI scans acquired before nCRT had started, extracting radiomics features from highly available sequences (i.e. T2w). The ideal approach exploits an advantage of radiomics that allows developing predictive models using medical images as they are acquired in the clinical routine[61]. On the other hand, each of the retrospective studies presented its own model, with a certain degree of heterogeneity that does not facilitate translation into clinical practice. An overview of the main studies proposing MRI radiomics and ML algorithms for the prediction of nCRT outcomes other than pCR is reported in Table 3. In those studies, the ML models were generally designed to classify patients into two groups (i.e. good and poor responders to nCRT), with one study prospectively designed but lacking external validation[62].

Table 2.

Key characteristics of the main studies using radiomics and machine learning algorithms on magnetic resonance images to predict pathologic complete response after neoadjuvant chemoradiotherapy in patients with locally advanced rectal cancer

Ref.	Study design (n of sites)	Number of patients	Definition of pCR	MRI field strength (n of scanners)	MRI timing	MRI sequence	ML algorithm	Data powering algorithm	Validation	Performance (AUC)
Antunes et al[59], 2020	Retrospective (3)	104	TRG 0 according to AJCC	1.5 and 3 T (> 10)	Pre-nCRT	T2w	RF	Radiomics features	External validation	0.71
Ferrari et al[106], 2019	Retrospective (1)	55	TRG 4 according to Dowrak-Rodel	3 T (1)	Pre-, mid- and post-nCRT	T2w	RF	Radiomics features	Internal validation (train/test split)	0.86
Horvat et al[107], 2018	Retrospective (11)	114	ypT0N0	1,5 and 3 T (4)	Post-nCRT	T2w	RF	Radiomics features	Internal validation (cross-validation)	0.93
Nie et al[108], 2016	Retrospective (1)	48	ypT0N0	3 T (1)	Pre-nCRT	T2w, DWI, pre and post-contrast T1w	ANN	Radiomics features	Internal validation (cross-validation)	0.84
Petkovska et al[109], 2020	Retrospective (11)	1022	ypT0N0	1,5 and 3 T (4)	Pre-nCRT	T2w	SVM	Radiomics and semantic features	Internal validation (train/test split)	0.75
Shaish et al[110], 2020	Retrospective (2)	132	ypT0N0	1,5 and 3 T (multiple3)	Pre-nCRT	T2w	LR	Radiomics features	Internal validation (train/test split)	0.80
Shi et al[111], 2019	Retrospective (1)	51	TRG 0 according to Ryan	3 T (1)	Pre- and mid-Ncrt4	T2w, DWI, pre- and post-contrast T1w	CNN	Radiomics features	Internal validation (cross-validation)	0.83
van Griethuysen et al[60], 2019	Retrospective (2)	133	ypT0/TRG1 according to Mandard	1,5 T (3)	Pre-nCRT	T2w and DWI	LR	Radiomics features	External validation	0.77
Yi et al[112], 2019	Retrospective (1)	134	ypT0N0	1,5 and 3 T (2)	Pre-nCRT	T2w	SVM	Radiomics, clinical and semantic features	Internal validation (train/test split)	0.88

¹< 10% of scans from other institutions.

²All previously included in Horvat et al[107], 2018.

³Inclusion of patients with MRI performed elsewhere but treated at study sites.

⁴Both MRI scans were not available for all patients. In all studies, three-dimensional manual segmentation of the primary tumor was performed to extract radiomic features, except for Shaish et al[110], mesorectal compartment) and van Griethuysen et al[60] (semiautomatic segmentation). ANN: Artificial neural network; CNN: Convolutional neural network; AUC: Area under the receiver operating characteristic curve; DWI: Diffusion-weighted imaging; LR: Logistic regression; ML: Machine learning; MRI: Magnetic resonance imaging; nCRT: Neoadjuvant chemoradiotherapy; RF: Random forest; SVM: Support vector machine; T1w: T1-weighted; T2w: T2-weighted; TRG: Tumor regression grade.

Table 3.

Key characteristics of the main studies using radiomics and machine learning algorithms on magnetic resonance images to predict outcome other than pathologic complete response after neoadjuvant chemoradiotherapy in patients with locally advanced rectal cancer

Ref.	Study design (n of sites)	Number of patients	Prediction task	CT phase (n of CT scanner)	Segmentation method	ML algorithm	Data powering algorithm	Validation	Performance
Bibault et al[85], 2018	Retrospective (3)	99	pCR after nCRT	Unenhanced (3)	Manual – 3D	DNN	Radiomics and clinical features	Internal validation (cross-validation)	AUC: 0.72
Hamerla et al[86], 2019	Retrospective (1)	169	pCR after nCRT	Unenhanced (1)	Manual – 3D	RF	Radiomics features	Internal validation (cross-validation)	Accuracy: 0.87
Yuan et al[87], 2020	Retrospective (1)	91	pCR after nCRT	Unenhanced (1)	Manual – 3D	RF	Radiomics features	Internal validation (train/validation split)	Accuracy: 0.84
Wu et al[90], 2019	Retrospective (1)	102	MSI status	Venous phase - DECT (2)	Manual - 3 2D ROIs for lesion	LR	Radiomics features	Internal validation (train/validation /test split)	AUC: 0.87
Fan et al[91], 2019	Retrospective (1)	100	MSI status	Portal venous phase (2)	Semiautomatic – 3D	NB	Radiomics features	Internal validation (cross-validation)	AUC: 0.75
Wu et al[92], 2020	Retrospective (1)	173	KRAS mutation	Portal venous phase (3)	Manual + DL – single 2D ROI	LR	Radiomics features	Internal validation (train/test split)	C-index: 0.83
Wang et al[94], 2019	Retrospective (1)	411	Prediction of survival	Unenhanced (1)	Manual – 3D	10-F CV	Radiomics and clinical features	Internal validation (cross-validation)	C-index: 0.73

In all studies, three-dimensional manual segmentation of the primary tumor was performed to extract radiomic features, with the exceptions of Alvarez-Jimenez et al[113] (rectal wall), van Griethuysen et al[60] (semiautomatic segmentation) and Yang et al[115] (two-dimensional manual segmentation). ANN: Artificial neural network; AUC: Area under the receiver operating characteristic curve; CNN: Convolutional neural network; DWI: Diffusion-weighted imaging; EMLM: Ensemble machine learning model; GR: Good responders; LASSO: Least absolute shrinkage and selection operator; RF: Random forest; LR: Logistic regression; ML: Machine learning; MRI: Magnetic resonance imaging; QDA: Quadratic discriminant analysis; SVM: Support vector machine; T1w: T1-weighted; T2w: T2-weighted; TRG: Tumor regression grade.

10F-CV: 10-fold cross-validation; CT: Computed tomography; DECT: Dual-energy computed tomography; DNN: Deep neural network; LR: Logistic regression; ML: Machine learning; MSI: Microsatellite instability; NB: Naive Bayes; nCRT: Neoadjuvant chemoradiotherapy; pCR: Pathologic complete response; RF: Random forest.

Additionally, recent studies explored the feasibility of radiomics nomograms, based on the combination of a radiomics signature and either a pretreatment MRI T stage[63] or a post treatment tumor length[64], to predict pCR to nCRT. In particular, Liu et al[64] built and validated a radiomics signature in LARC patients using T2w in 152 and DWI images in 70 the T2w and DWI images were acquired both before and after nCRT. An SVM ML algorithm incorporating signatures and post treatment tumor length in a nomogram was able to reach a final diagnostic accuracy of 94% in the prediction of pCR. Finally, Wang et al[65] developed a radiomics signature to classify good responders and poor responders to nCRT with an LR ML algorithm and radiomics features from T2w, DWI, and DCE sequences. When combined in a nomogram with MRI T stage and circumferential resection margin as well as apparent diffusion coefficient values, they were able to predict a good response with a sensitivity of 71% and a specificity of 88%.

Genotyping

Radiogenomics aims to correlate imaging features of a disease with its genotypic characteristics and represents the next step in a radiology-pathology correlation[66]. Radiomics and radiogenomics are not equivalent, and both qualitative and quantitative imaging features can be used for radiogenomic analysis, with quantitative data having promising associations with genetic mutations in RC[67]. Among the negative genetic prognostic factors in RC, KRAS mutations are associated with poor response to epidermal growth factor receptor-targeted antibodies[68] and an increased risk of developing distant metastases[69]. In a recent multicenter study by Cui et al[70], three classifiers (decision tree, SVM and LR) powered by T2w-based radiomics features were trained to predict KRAS mutations in data from 213 patients and validated in both an internal cohort of 91 patients and external cohort of 86. The SVM obtained the greatest area under the receiver operating characteristic curve (AUC) in the training dataset (0.72), which was substantially confirmed in the internal (AUC = 0.68) as well as external (AUC = 0.71) validation cohorts. The finding supports the potential generalizability of such models. Interestingly, in the same study, no associations were found between KRAS status and baseline clinical and histopathological data. More optimistic results were recently published using a decision tree classifier (AUC = 0.88) by a different study group[71], but the sample size was substantially smaller (60 patients) and the model was not externally validated. Finally, T2w-based radiomics have been also paired with DL, with an artificial NN discriminating between patients with or without KRAS mutations and a classification error of 13%[72].

Assessing high-risk histopathological variables

Several histopathological characteristics, EMVI, differentiation degree, and perineural invasion (PNI) for example, are associated with poor clinical outcome and need to be considered in the risk stratification of patients with RC. It is fair to assume that a reliable pretreatment evaluation of these high-risk variables would ease the transition toward precision medicine[5]. ML classifiers applied to MRI radiomics features have been recently explored in this setting[73-75]. As previously highlighted, MRI can be used to identify EMVI; however, its sensitivity is not as high as desirable[76]. To overcome current MRI limitations, Yu et al[73] built a nomogram based on both a DCE MRI radiomics signature and clinical data, finding that it outperformed conventional quantitative perfusion parameters such as Ktrans in the prediction of EMVI, with a sensitivity of 88.9% and a specificity of 78.3% in the validation cohort.

Well-differentiated tumors are associated with better outcomes of RC patients[77]. In a large cohort of 345 patients retrospectively enrolled at a single institution, Meng et al[74] explored the performance of three ML classifiers, RF, SVM, and least absolute shrinkage, and selection operator (LASSO) to identify well-differentiated RC. Radiomics features were extracted from multiple MRI sequences, including T2w, DWI, and DCE. The LASSO algorithm had the best performance, with an AUC of 0.72 in the validation dataset. Finally, PNI, the tumor spreading along the nerve sheath, is a histopathological factor known to be associated with poor prognosis[78]. Using T2w radiomics and AI, Chen et al[79] developed a nomogram to predict the presence of PNI in RC patients (AUC = 0.85). A decision curve analysis confirmed the clinical utility of their nomogram, but the sample size of only seven PNI-positive patients in the test dataset requires validation of this preliminary findings in larger datasets.

RADIOMICS AND ML APPLICATIONS IN RC: CT

In the management of RC, CT is commonly used as the initial staging modality, allowing accurate nodal and metastases staging and target volume delineation before radiation therapy in patients with LARC[6]. Conversely, the role of CT in RC pretreatment local staging as well as restaging after nCRT is limited because of its intrinsically lower contrast resolution compared with MRI[80,81]. Nevertheless, much effort has been directed toward the use of CT data beyond clinical indications, with the aim of developing CT-based radiomics signatures reflecting tumor heterogeneity[82]. CT images contain robust volumetric data that are highly reproducible across patients and are an ideal source of data to feed AI systems[83,84]. In that perspective, ML models that can find correlations of RC CT radiomics features that can be used to predict outcomes such as complete response to nCRT in LARC patients, genetic profiles, overall survival, and segmentation (Table 4).

Table 4.

Key characteristics of the main studies using radiomics and machine learning algorithms on computed tomography for v prediction tasks

Ref.	Study design (n of sites)	Number of patients	Prediction task	CT phase (n of CT scanner)	Segmentation method	ML algorithm	Data powering algorithm	Validation	Performance
Bibault et al[85], 2018	Retrospective (3)	99	pCR after nCRT	Unenhanced (3)	Manual – 3D	DNN	Radiomics and clinical features	Internal validation (cross-validation)	AUC: 0.72
Hamerla et al[86], 2019	Retrospective (1)	169	pCR after nCRT	Unenhanced (1)	Manual – 3D	RF	Radiomics features	Internal validation (cross-validation)	Accuracy: 0.87
Yuan et al[87], 2020	Retrospective (1)	91	pCR after nCRT	Unenhanced (1)	Manual – 3D	RF	Radiomics features	Internal validation (train/validation split)	Accuracy: 0.84
Wu et al[90], 2019	Retrospective (1)	102	MSI status	Venous phase - DECT (2)	Manual - 3 2D ROIs for lesion	LR	Radiomics features	Internal validation (train/validation /test split)	AUC: 0.87
Fan et al[91], 2019	Retrospective (1)	100	MSI status	Portal venous phase (2)	Semiautomatic – 3D	NB	Radiomics features	Internal validation (cross-validation)	AUC: 0.75
Wu et al[92], 2020	Retrospective (1)	173	KRAS mutation	Portal venous phase (3)	Manual + DL – single 2D ROI	LR	Radiomics features	Internal validation (train/test split)	C-index: 0.83
Wang et al[94], 2019	Retrospective (1)	411	Prediction of survival	Unenhanced (1)	Manual – 3D	10-F CV	Radiomics and clinical features	Internal validation (cross-validation)	C-index: 0.73

10F-CV: 10-fold cross-validation; CT: Computed tomography; DECT: Dual-energy computed tomography; DNN: Deep neural network; LR: Logistic regression; ML: Machine learning; MSI: Microsatellite instability; NB: Naive Bayes; nCRT: Neoadjuvant chemoradiotherapy; pCR: Pathologic complete response; RF: Random forest.

Predicting response to nCRT in patients with LARC

Bibault et al[85] explored the reliability of deep NN (DNN) integrating clinical features (T stage) and robust radiomics CT-based features in assessing the pCR to nCRT in a multicenter cohort of patients with LARC. The DNN model predicted pCR with an accuracy of 80% compared with 69.5% achieved with an LR model using only the TNM stage and an SVM model with the same parameters as the DNN that had an accuracy of 71.58%. Similarly, Hamerla et al[86] reported an accuracy of 87% for prediction of pCR after nCRT using an ML algorithm and CT radiomics data, but they noted that the model was not generalizable because of bias introduced by an imbalanced distribution of the minority class (pCR: 13% and non-pCR = 87%) in the study population. In another study, Yuan et al[87] tested and compared different ML algorithms using robust CT-based radiomics features significantly correlated with pCR. The best performing model was an RF with an accuracy of 83.9% in the test population. Interestingly, these studies used radiomics features extracted from unenhanced CT scans used for radiotherapy planning. The process highlights the potential value of nonroutine CT data for pretreatment risk stratification.

Genotyping

Recent studies have shown encouraging results with regard to the high predictive ability of AI-radiomics CT-based models of the biologic behavior of RC, in terms of microsatellite instability (MSI) status and KRAS gene mutations, which are considered significant molecular markers of improved prognosis and adjuvant therapy[11,88,89]. Wu et al[90] developed a pretreatment predictive model of MSI status in RC using ML radiomics features extracted from venous phase images of iodine-based material decomposition with dual-energy CT (DECT). Performance of the model was tested on images acquired with a different DECT scanner, and achieving a diagnostic accuracy of 79%. The result suggests a possible link between iodine DECT images and augmented tumor vascularization. In a preliminary retrospective study, Fan et al[91] found that an ML model combining clinical and CT radiomics features had a better classification performance for MSI status (AUC = 0.75) in stage II RC patients than models using only clinical features (AUC = 0.60) or only radiomics features (AUC = 0.70).

Another research group[92] investigated the performance of a model that used handcrafted radiomics signatures combined with those in a DL algorithm. The combined model was able to the discriminate patients with mutant or the wild-type KRAS with a sensitivity of 80% and a specificity of 72% in the validation cohort, thus showing a good predictive performance.

Prognosis

An active field of AI oncology-related research is the discovery of new clinical and imaging tumor biomarkers that are correlated with prognosis, with the goal of developing accurate predictive models of treatment response based on personalized tumor profiles[93]. Wang et al[94] explored the use of CT-based ML models powered by clinical and radiomics features to assess the prognostic outcomes of LARC patients treated with nCRT. Radiomics features were extracted from nonenhanced CT images used for planning the treatment of 411 LARC patients. Images analyzed by unsupervised ML did not find a relationship between the clinical and radiomics features. A supervised ML model with embedded radiomics and clinical parameters had an improved overall survival prediction in the testing set and a c-index of 0.73 which was significantly better (P = 0.044) than the performance of the model using only clinical factors (c-index = 0.67).

RADIOMICS AND ML APPLICATIONS IN RC: MULTIMODAL AND HYBRID IMAGING

The advantage of multimodality and hybrid imaging in oncology is mainly related to the combined evaluation of anatomical and functional tumor characteristics. Radiomics and ML could further increase the potential value of the techniques[95]. However, the number of studies evaluating RC is still limited, and the role of multimodal radiomics and ML models has mainly been investigated for the prediction of response to nCRT in patients with LARC[96,97]. In a single-center study in 169 patients, Shen et al[96] developed an RF model based on baseline PET/CT images that accurately predicted pCR to nCRT in LARC patients, with a sensitivity of 81.8% and a specificity of 97.3%. Another study confirmed the feasibility of combining pretreatment MRI data from T2w sequences and PET radiomics features to build a prediction model able to identify responders or nonresponders. ML algorithms were used for semiautomatic segmentation of the primary tumor in both sets of images[97]. The final LR model had a sensitivity of 86% and specificity of 83%. Beyond nuclear medicine, Li et al[98] described a multimodal radiomics-based nomogram with features extracted from baseline MRI and CT images, which better performed better than individual imaging techniques in the prediction of response to nCRT. Although multimodal radiomics for RC is in its infancy, the encouraging preliminary reports support the idea that it could allow an even more comprehensive assessment of tumor characteristics compared with individual images.

CURRENT LIMITATIONS AND FUTURE PERSPECTIVES

The available evidence confirms that AI is a feasible tool to broaden the spectrum of information that medical imaging can provide for the management of RC patients. Nevertheless, there is a risk that negative results are not published because of publication bias[99]. Furthermore, what could theoretically be done is not ready for clinical practice at present. Indeed, there are many exploratory studies and very few confirmatory ones to support the use of one radiomics-ML model over another. A possible solution to the problems of verifying generalizability and comparing the performance of different models proposed for the same prediction task might be the use of open-source data[100]. Indeed, a publicly available large dataset from multiple institutions could serve as a common benchmark to verify whether the available models can reproduce previous results while we wait for well-designed prospective clinical trials that will overcome the limitations of retrospective studies. Currently, there is a great interest in public imaging datasets, but their quality might be heterogeneous[101]. It should also be considered that significant variations in radiomics and ML pipelines make it difficult to compare studies. Adherence to shared guidelines for AI study design is this highly advisable[102]. Another issue of concern that could prevent widespread adoption of radiomics-ML prediction models is manual segmentation. It is often necessary, but is a time-consuming procedure that requires automatization. However, AI could also solve that problem. Recent studies have described the use of DL for fully automated segmentation of RC on both CT and MR images, with encouraging accuracy and computational time results[103,104]. Several of the radiomics-ML models described in this review had promising accuracy, but it should be noted that the potential clinical utility of such models depends on multiple factors, such as their added value in comparison with current gold standards, the cost-effectiveness of their implementation, and their actual impact on clinical practice. Decision curve analysis might be helpful in the analysis[34]. Finally, a recent study found that the overall quality of radiomics studies in oncology is below the desired standards, suggesting that most of the problems identified in the field of RC radiomics are shared among the studies involving different types of cancer[105].

CONCLUSION

Medical images contain mineable data with great potential. AI appears to be a convenient tool to harness their value for RC management. AI in imaging can support physicians in the transition toward precision medicine for RC patients, but there is still a long road ahead and it is time to start moving to the next step. Robust prospective multicenter studies and clinical trials are needed to confirm the clinical implications of this new methodology.