Predicting Rectal Cancer Response to Total Neoadjuvant Treatment Using an Artificial Intelligence Model Based on Magnetic Resonance Imaging and Clinical Data

Abstract

Purpose

To develop a model for predicting response to total neoadjuvant treatment (TNT) for patients with locally advanced rectal cancer (LARC) based on baseline magnetic resonance imaging (MRI) and clinical data using artificial intelligence methods.

Methods

Baseline MRI and clinical data were curated from patients with LARC and analyzed using logistic regression (LR) and deep learning (DL) methods to predict TNT response retrospectively. We defined two groups of response to TNT as pathological complete response (pCR) versus non-pCR (Group 1), and high sensitivity [tumor regression grade (TRG) 0 and TRG 1] versus moderate sensitivity (TRG 2 or patients with TRG 3 and a reduction in tumor volume of at least 20% compared to baseline) versus low sensitivity (TRG 3 and a reduction in tumor volume <20% compared to baseline) (Group 2). We extracted and selected clinical and radiomic features on baseline T2WI. Then we built LR models and DL models. Receiver operating characteristic (ROC) curves analysis was performed to assess predictive performance of models.

Results

Eighty-nine patients were assigned to the training cohort, and 29 patients were assigned to the testing cohort. The area under receiver operating characteristics curve (AUC) of LR models, which were predictive of high sensitivity and pCR, were 0.853 and 0.866, respectively. Whereas the AUCs of DL models were 0.829 and 0.838, respectively. After 10 rounds of cross validation, the accuracy of the models in Group 1 is higher than in Group 2.

Conclusion

There was no significant difference between LR model and DL model. Artificial Intelligence-based radiomics biomarkers may have potential clinical implications for adaptive and personalized therapy.

Introduction

Neoadjuvant chemoradiotherapy followed by total mesorectal excision (TME) increased the chance of a significant downstaging and reduced the risk of recurrence.^1,2 Recently, the National Comprehensive Cancer Network (NCCN) guidelines recommend total neoadjuvant treatment (TNT), ³ which is an extensive and optimized therapeutic modality for locally advanced rectal cancer (LARC). However, the duration of TNT treatment is long. And individual sensitivities to neoadjuvant treatment may be different among patients. If the response to neoadjuvant treatment regimens can be predicted before treatment, we could provide more individualized and accurate treatment regimens for patients. Therefore, we urgently explore more effective prediction methods to predict TNT response.

Magnetic resonance imaging (MRI) plays a very important role in the clinical staging of rectal cancer. MRI provides significant information and the basis for following clinical treatment.^4,5 In addition, radiomics, which originated from computer-aided detection and diagnosis, is a combination of image quantitative analysis and artificial intelligence (AI) methods.^6,7 Increasingly, radiomic approaches have been applied in clinical diagnosis and assisted doctors in decision support. ⁸ Studies reported that the radiomics feature could be used as a biomarker, and highlighted the potential value in tumor diagnosis and prognosis prediction.^6,9 Moreover, some scholars suggested that the baseline radiomic features could predict the response of neoadjuvant treatment of rectal cancer.^10–15 Nevertheless, the accurate and well-recognized radiomic features or markers for predicting response have not yet been discovered.

For feature extraction, most studies extracted handcrafted radiomic (HCR) features by using non-deep learning (DL) methods. However, HCR features are lower-order image features and more limited compared to deep learning radiomic (DLR) features. ¹⁶ Studies showed that the DL method may have advantages in feature extraction, which can extract more robust, deep and high-level features and does not require prior knowledge.^16,17 It has been revolutionized in image classification and prediction. ¹⁸ However, the study combined radiomics with DL is very rare in rectal cancer.^14,19 In addition, previous studies have focused on primary tumor features. But positive lymph nodes ²⁰ and peritumoral regions^21–25 may also have value in predicting response. Most importantly, few studies have focused on building a model to predict TNT response.

Therefore, we are trying to conduct a study for solving the problems mentioned above. We develop two types of novel AI models combining baseline MRI radiomic features and clinical data for predicting TNT response in LARC. Moreover, we analyze not only the primary tumor, but also the positive lymph nodes and peritumoral regions.

Materials and Methods

This is a retrospective study approved by Ethics Committee on Biomedical Research, West China Hospital of Sichuan University (2021-586). Ethics committee has agreed that the informed consent is waived. Because this is a retrospective study, and we collected de-identified data of patients. Human participants’ names and other identifiers have been removed from all sections of the manuscript.

Patients

This study enrolled patients with LARC who underwent baseline MR examinations retrospectively. Inclusion criteria were as follows: pathologically confirmed adenocarcinoma; stage II–III rectal cancer with distal extension less than 12 cm from the anal verge, received TNT (neoadjuvant concurrent chemoradiotherapy and at least 4 cycles of neoadjuvant chemotherapy of capecitabine combined with oxaliplatin (CAPOX)), underwent TME, and had pathological reports. Patients were excluded if they received less than 4 cycles of neoadjuvant chemotherapy, or if the chemotherapy regimen was not CAPOX, or if they did not have baseline MR images, or if they received targeted therapy, or if they received pelvic radiotherapy for other reasons in the past. Details of the treatment regimen are described in the supplementary documents.

Data Processing

MRI scans were performed before TNT. All MRI images were taken from picture archiving and communication system (PACS) in our hospital. MRI parameters are list in the supplement. The reprocessing method for the images was standardization: (grey value-average value) / standard deviation. Regions of interest (ROIs) were manually segmented on T2-weighted images (T2WI) using ITK-SNAP software by a radiation physician (5-year experience) and reviewed and revised by another radiation physician (10-year experience).

ROIs consisted of primary tumor (ROI_T), positive lymph nodes (ROI_LN) and peritumoral regions (ROI_Peri). ROI_T included rectal tumors and mucus, but not the intestinal cavity. ROI_Peri was defined as the outward expansion of ROI_T and ROI_LN by 3.5 mm in all directions. The definition of positive lymph node and clinical data are described in the supplement.

Construction of Logistic Regression (LR) Model

Figure 1 shows the structure and workflow of LR model. The methods were as follows. (1) Feature extraction: the HCR features were extracted using PyRadiomics package ²⁶ in python software (version 3.6). (2) Feature selection: intraclass correlation coefficients (ICC) were calculated to estimate feature robustness with a cutoff value of 0.8. ²⁷ Then, the selected features were further selected with the random forest feature selection algorithm, so as to construct a radiomic and clinical data signature and calculate the importance of the features. (3) LR model construction: AI models were built using LR algorithms. The LR formula is as follows:

$$y=\sigma (f(x))=\sigma (w^{T}x)={\displaystyle \frac{1}{1+{\mathrm{e}}^{-w^{T}x}}}$$

x: a variable and w: the selection feature.

Figure 1.

The structure and workflow of logistical regression model.

The calculation of the importance of a feature x is as follows. (1) The out-of-bag (OBB) data of each random forest decision tree was used to calculate the out-of-bag data error (errOOB1). (2) In order to randomly change the sample data of feature x, the noise interference was randomly added to feature x of all OOB data, and then the out-of-bag data error (errOOB2) was calculated. (3) The greater the importance of a feature, the greater the impact on classification results. The formula for the importance of a feature x is as follows:

$$\mathrm{The}\mathrm{importance}\mathrm{of}\mathrm{a}\mathrm{feature}x={\displaystyle \frac{\sum (\mathrm{errOOB}2-\mathrm{errOOB}1)}{N}}$$

N: the number of random forest decision tree.

Construction of Deep Learning (DL) Model

We built the DL model using DL networks and ensemble learning algorithms (Figure 2). DL model is constructed as follows. (1) The two-dimensional convolutional neural network (2D-CNN) ResNet34 ²⁸ and the three-dimensional convolutional neural network (3D-CNN) C3D ²⁹ were used to extract the 2D and 3D DLR features in the training set. These features consisted of overall appearance (OA), heterogeneity in shape (HS) and around appearance (AA). In C3D, ROI was converted into volume of interest (VOI). And we adjusted VOI pixels to 256 × 256 × 32 pixels. Then the corresponding feature classifiers were built. (2) At the same time, we used ICC (excluding features with ICC less than 0.8) and random forest to select clinical features to construct the clinical feature signature (clinical feature classifier). (3) We built the DL model using XGBoost ³⁰ (extreme gradient boosting), which could integrate classifiers of 2D DLR features, 3D DLR features and clinical features.

Figure 2.

The structure and workflow of deep learning model. The deep learning model was built using deep learning networks and the ensemble learning algorithm “XGBoost.”

Additionally, Gain was used to rank the importance of each feature to the prediction results. Gain took values between 0 and 1, where “0” and “1” indicated the best probability of a good response. The formula of Gain is as follows:

$$\mathrm{Gain}={\displaystyle \frac{1}{2}\left[{\displaystyle \frac{G_{\mathrm{L}}^2}{H_{\mathrm{L}}+\lambda }+{\displaystyle \frac{G_{\mathrm{R}}^2}{H_{\mathrm{R}}+\lambda }-{\displaystyle \frac{{(G_{\mathrm{L}}+G_{\mathrm{R}})}^2}{H_{\mathrm{L}}+H_{\mathrm{R}}+\lambda }}}}\right]-\gamma }$$

$G_{\mathrm{L}}^2/(H_{\mathrm{L}}+\lambda ):$ left subtree fraction; $G_{\mathrm{R}}^2/(H_{\mathrm{R}}+\lambda ):$ right subtree fraction; $(G_{\mathrm{L}}+G_{\mathrm{R}}{)}^2/(H_{\mathrm{L}}+H_{\mathrm{R}}+\lambda ):$ results obtained without residual learning; $\lambda$ : the complexity cost of adding new leaf nodes

Evaluation of TNT Response

There were two groups of responses to TNT. Group 1 consisted of pCR (pathological complete response, ypT0N0) and non-pCR. And in group 2, TRG 0/1 was considered high sensitivity; TRG 2 or patients with TRG 3 and MRI tumor volume decreased at by least 20% compared to baseline^31,32 was considered moderate sensitivity; whereas TRG 3 and MRI tumor volume did not decrease by 20% compared to baseline for low sensitivity. The system used for TRG as recommended by the AJCC Cancer Staging Manual, 8th Edition and the CAP Guidelines is that as modified by Ryan et al. ³³ Details of the statistical analysis are set out in Supplementary documents.

Results

Patient Characteristics

Out of 118 LARC patients treated with TNT, they were enrolled retrospectively between July 2015 and February 2020. Eighty-nine patients were used as a training cohort, and 29 patients as a testing cohort (Table 1). Thirty-nine patients (33.05%) achieved pCR. There was no low sensitivity but only high sensitivity (65 cases, 55.08%) and moderate sensitivity (53 cases, 44.92%). Patient characteristics of two groups are summarized in Supplementary Tables S1 and S2, respectively.

Table 1.

Patient Characteristics in Training and Testing Cohort.

	Training cohort	Validation cohort
Variable	N = 89 (75.42%)	N = 29 (24.58%)	P
Age (years)	53 (28-74)	55 (33-78)	0.147
BMI (kg/m2)	23.50 (17.15-31.22)	22.72 (17.94-28.51)	0.522
Gender			0.935
Male	56 (62.92%)	18 (62.07%)
Female	33 (37.08%)	11 (37.93%)
Baseline T stage			0.976
T2	1 (1.12%)	1 (3.45%)
T3	47 (52.81%)	17 (58.62%)
T4a	17 (19.10%)	3 (10.34%)
T4b	16 (17.98%)	8 (27.59%)
Baseline N stage			0.046*
N0	11 (12.36%)	1 (3.45%)
N1	25 (28.09%)	5 (17.24%)
N2	53 (59.55%)	23 79.31(%)
Group 1			0.004*
pCR	23 (25.84%)	16 (55.17%)
Non-pCR	66 (74.16%)	13 (44.83%)
TRG			0.149
TRG 0	21 (23.60%)	16 (55.17%)
TRG 1	28 (31.46%)	0
TRG 2	29 (32.58%)	10(34.48%)
TRG 3	11 (12.36%)	3 (10.34%)
Group 2			0.991
The high sensitive group	49 (55.06%)	16 (55.17%)
The moderate sensitive group	40 (44.94%)	13 (44.83%)
The low sensitive group	0	0
ypT stage			0.003*
T0	25 (28.09%)	16 (55.17%)
T1	4 (4.49%)	0
T2	16 (17.98%)	8 (27.59%)
T3	40 (44.94%)	4 (13.79%)
T4	4 (4.49%)	1 (3.45%)
ypN stage			0.197
N0	64 (71.91%)	25 (86.21%)
N1	21 (23.60%)	1 (3.45%)
N2	4 (4.49%)	3 (10.34%)

NOTE: Fisher exact tests, as appropriate, were used to compare the differences in categorical variables (Gender, Baseline T stage, Baseline N stage, Group 1, TRG, Group 2, ypT stage, ypN stage), whereas a two-sample t test was used to compare the differences in numerical variables (age and BMI).

Abbreviations: BMI, ballistic missile interceptor; pCR, pathologic complete response.

*P < 0.05.

LR Model Construction

A total of 100 HCR features and 50 clinical features per patient were extracted from T2WI and clinical data. Then, we built radiomic and clinical signatures, leaving 15 features in group 1 and 10 features in Group 2 (Table 2) (the importance of features decreases in turn). Finally, two separate predictive LR models were built for composite features from HCR and clinical features.

Table 2.

Features of LR Models.

Feature	Type	Importance
Group 1-LR model
GLCM_MCC	Primary tumor	0.987
Shape_Maximum3DDiameter	Lymph mode	0.979
GLSZM_SmallAreaEmphasis	Lymph mode	0.950
CA199	Clinical data	0.919
Platelet count	Clinical data	0.797
GLSZM_SizeZoneNonUniformity	Peritumoral region	0.779
GLCM_JointEntropy	Primary tumor	0.745
Erythrocyte count	Clinical data	0.667
Firstorder_10Percentile	Primary tumor	0.659
GLSZM_GrayLevelNon UniformityNormalized	Primary tumor	0.509
GLDM_LowGrayLevelEmphasis	Lymph mode	0.354
Firstorder_Robust MeanAbsoluteDeviation	Peritumoral region	0.350
Lactic dehydrogenase	Clinical data	0.314
Weight	Clinical data	0.224
GLSZM_SmallAreaHigh GrayLevelEmphasis	Primary tumor	0.154
Group 2-LR model
Original_Shape_Flatness	Primary tumor	0.978
Original_GLCM_Imc2	Primary tumor	0.869
CA199	Clinical data	0.768
Original_GLCM_ClusterShade	Lymph mode	0.728
Shape_Elongation	Peritumoral region	0.697
Lactic dehydrogenase	Clinical data	0.534
Glcm_MCC	Primary tumor	0.467
Firstorder_10Percentile	Primary tumor	0.461
Globulin	Clinical data	0.205
GLSZM_SmallAreaEmphasis	Lymph mode	0.192

Abbreviation: LR, logistic regression.

DL Model Construction

ResNet34 and C3D extracted 2D and 3D DLR features of tumor, lymph node and peritumoral regions, respectively. Clinical features were extracted and selected using ICC and random forest. Then the corresponding response classifiers were built. The features of Groups 1 and 2 (the importance of features decreases in turn) are shown in Table 3. Finally, XGBoost integrated all DLR and clinical classifiers to construct the two DL models.

Table 3.

Features of DL Models. Features are the Confidence Output of Neural Network Classifiers. The Result of the Confidence Output is the Real Number on the set [0,1], Which Represents the Importance of the Feature. the Closer the Real Number is to 0 or 1, the Greater the Probability of a Good Response.

Feature	Type	Importance
Group 1-DL model
2D-tumor texture features	Primary tumor	0.158
2D-lymph node shape features	Lymph mode	0.834
3D-lymph node texture features	Lymph mode	0.810
3D-lymph node peritumoral features	Peritumoral region	0.210
2D-tumor shape features	Primary tumor	0.270
Clinical feature signature	Clinical data	0.319
3D-tumor texture features	Primary tumor	0.328
3D-tumor shape features	Primary tumor	0.375
2D-peritumoral features	Peritumoral region	0.395
2D-lymph node texture features	Lymph mode	0.440
2D-lymph node peritumoral features	Peritumoral region	0.555
3D-lymph node shape features	Lymph mode	0.491
3D-peritumoral features	Peritumoral region	0.508
Group 2-DL model
3D-tumor texture features	Primary tumor	0.84
2D-lymph node texture features	Lymph mode	0.814
3D-lymph node texture features	Lymph mode	0.208
3D-lymph node shape features	Lymph mode	0.745
2D-peritumoral features	Peritumoral region	0.741
2D-tumor texture features	Primary tumor	0.685
Clinical feature signature	Clinical data	0.339
2D-tumor shape features	Primary tumor	0.353
3D-lymph node peritumoral features	Peritumoral region	0.588
3D-peritumoral features	Peritumoral region	0.575
2D-lymph node shape features	Lymph mode	0.573
2D-lymph node peritumoral features	Peritumoral region	0.529
3D-tumor shape features	Primary tumor	0.477

Abbreviation: DL, deep learning.

Predictive Performance of Models

Figure 3 shows the receiver operating characteristic (ROC) curves of the four models. The LR models in group 1 and group 2 yielded mean AUCs of 0.866 and 0.853. Models built with DL achieved mean AUCs of 0.838 (group 1) and 0.829 (group 2), respectively (Table 4). Moreover, there was no significant difference between LR model and DL model in either group 1 (P = 0.18) or group 2 (P = 0.23). After 10 rounds of cross validation, the mean accuracy of LR model and DL model in group 1 was 80.92% and 79.17%, and the mean accuracy of LR model and DL model in group 2 was 78.56% and 77.15%. The accuracy of the two models in group 1 was better than that in group 2, indicating that the grouping mode in group 1 was better (Figure 4).

Figure 3.

The receiver operating characteristic (ROC) curves of models.

Table 4.

Predictive Performance of DL and LR Models to TNT Response.

Model	AUC	SEN	SPE	PPV	NPV	ACC	P
Group 1-DL	0.838	13/15 (86.7)	10/14 (71.4)	13/17 (76.5)	10/17 (58.8)	23/29 (79.31)	0.009*
Group 2-DL	0.829	12/14 (85.7)	12/15 (80.0)	12/15 (80.0)	12/14 (85.7)	24/29 (82.76)	0.008*
Group 1-LR	0.866	14/15 (93.3)	12/14 (85.7)	14/16 (87.5)	12/13 (92.3)	26/29 (89.66)	0.011*
Group 2-LR	0.853	10/14 (71.4)	14/15 (93.9)	10/11 (90.9)	14/18 (77.8)	24/29 (82.76)	0.013*

Abbreviations: TNT, total neoadjuvant treatment; DL, deep learning; LR, logistic regression; AUC, the area under the receiver operating characteristic curve; SEN, sensitivity; SPE, specificity; PPV, positive predictive value; NPV, negative predictive value; ACC, accuracy.

*P < 0.05.

Figure 4.

Model accuracy after 10-fold cross validation.

Discussion

In this study, we built four AI models to predict TNT response based on MRI and clinical data in LARC patients. In addition, we innovatively used DL combined with the XGBoost algorithm to build the DL model. Our study showed that the LR model and DL model had similar predictive performance. Importantly, the results showed that response information could be obtained not only from primary tumor but also from positive lymph nodes and peritumoral regions. To our knowledge, this is the first study to predict TNT response in LARC, and the first to consider radiomics analysis of positive lymph nodes and peritumoral regions in conjunction with tumor radiomics analysis in predicting response in LARC.

As a non-invasive image analysis method, several studies^{10–12,19,34–36} have confirmed the value of radiomics in predicting and evaluating response. Different radiomics methods have recently been used in the literature. In terms of non-DL approaches, random forest model ¹⁰ and LR model^11,12,15 were the two most common methods for extracting HCR features and predicting CRT response in rectal cancer. For example, Hamerla et al were able to predict CRT response with an accuracy of 87% using CT and random forest. ¹⁰ While Cui et al developed a LR model to select MRI radiomics features and built a radiomics signature, which AUC was 0.944 in the validation dataset. ¹² As for DL methods, some studies that predicted CRT response in rectal cancer have already been published.^13,14,37 The research of Nie et al focused on multi-categories of MRI, and built an artificial neural network (ANN) model. ¹³ The AUC of testing set was 0.84 and 0.89 for predicting pCR and good responser (GR), respectively. However, in our study, we combined clinical data with imaging features to build LR and DL models, which may be comprehensive in predicting TNT response. The results of our study were similar to previous studies. Unlike previous research methods, we used convolutional neural network for feature extraction and the XGBoost algorithm for model building. XGBoost algorithm could effectively integrate the prediction results of different classifiers. Its advantages included a variety of strategies to prevent overfitting, fast training speed, high accuracy and selecting more important features by adjusting the weight. At present, the algorithm is widely used in data and science competitions and industry, but it is rarely applied and explored in the medical field. ³⁸

Studies have reported that DLR features achieved better performance than HCR features in breast carcinoma and hepatocellular carcinoma diagnosis and response prediction^14,39–41 In addition, Bibault et al revealed that a deep neural network (DNN) and two non-DL models trained by CT and clinical data from three academic institutions extract DLR and HCR features. Compared to non-DL models, DNN had better performance, predicting pCR with 80% accuracy and 0.72 AUC. ¹⁴ This is probably because the DLR features are high-level shape and texture features, and capture more imaging heterogeneity than HCR features. To validate these results, we extracted HCR and DLR features, and presented a LR and DL model in this study. However, there was no significant difference between LR and DL model. In addition, the cross-validation results suggested that the prediction performance of LR model is more stable than DL model. We concluded that this may be due to the small sample size in our study. In theory, training a DL model required huge datasets to ensure higher accuracy and prediction performance.^42,43 We are now conducting a phase III randomized multicenter trial of TNT (NCT03177382). In the future, we could expand the sample size and obtain multicenter data to validate the results.

Peritumoral region features may be important for response prediction.^{21–25,44,45} In addition, extramural venous invasion (EMVI) plays a crucial role in tumor recurrence^46–48. However, micrometastasis in lymphovascular tumors is not directly discernible by ROI delineation and diagnostic imaging techniques. ²³ This is one of the biggest differences between our studies and previous ones. Currently, there is no literature to clearly report the specific range of ROI_Peri around the rectal tumor. In a recent study, peritumoral radiomics features predicted distant metastasis in locally advanced non-small cell lung cancer treated with stereotactic body radiation therapy. ⁴⁹ They defined the tumor rim as the tumor located 3 mm away from the tumor contour, and defined the tumor exterior as the tumor extending from 3 mm to 9 mm outside the tumor contour. Similarly, Pizzi et al tried to present a novel model combining baseline MRI-based tumor core and tumor border radiomic features for the early prediction of response in LARC patients. ²⁵ First, both a 2-mm dilatation (“dilate”) and a 2-mm erosion (“erode”) were obtained from the tumor core. Then, they subtracted erode from dilate to obtain the tumor boundary. The tumor border was 4 mm thick. Therefore, in our study, we tried to extend 1 mm, 3.5 mm, 5 mm and 7mm in all directions of ROI_T and ROI_LN, respectively. Finally, we concluded that 3.5 mm away from ROI_T and ROI_LN provided valuable predictive information.

Moreover, we analyzed not only primary tumor but also positive lymph nodes. However, few studies have explored the benefit of adding lymph node radiomics features in rectal cancer. In clinical practice, the tumor may not have the same regression with lymph nodes after CRT or TNT. In the study, 39 patients (33.05%) achieved pCR, yet 37 patients (31.36%) achieved TRG 0. The primary tumor of 2 patients achieved completed regression, but lymph nodes did not. Furthermore, positive lymph nodes were considered a high risk factor for LARC recurrence. ³ One study indicated that baseline MRI lymph node positivity were less likely to achieve a good response to CRT. ²⁰ Coroller et al demonstrated that baseline radiomic features obtained from the lymph node were significantly predictive of CRT pathological response in locally advanced non-small cell lung cancer. ⁵⁰ Therefore, we concluded that baseline lymph node information may be associated with TNT response in LARC. Our study shows that radiomics features of lymph nodes are helpful in predicting TNT response.

There were several limitations that needed to be addressed in the study. First, this is a retrospective study with a small sample size collected from a single center. Feature extraction from MRI is complex and reproducible. These results cannot easily be reproduced in another center. Therefore, we can further expand the sample size and obtain multicenter data to validate the results. Second, ROIs were manually contoured, which could have subjective bias. Third, we only analyzed the T2WI sequence, which was somewhat limited. In the future, we will include more sequences for analysis. Fourth, we built models to predict TNT response, which included chemotherapy and concurrent chemoradiotherapy. We did not separate the response of radiotherapy alone, chemoradiotherapy alone, or chemotherapy alone. In the future, we will further explore the response prediction of different neoadjuvant treatment modes. Fifth, we built DL models to extract features, and then integrated models using integrated learning methods to predict response. This is a two-stage method for building the DL model. In the future, we will improve this method and build an integrated end-to-end learning method based on DL to train the most optimal response prediction model.

Conclusion

We have built models for predicting TNT response for LARC based on MRI and clinical data using LR and DL models. Four models were constructed using not only primary tumor features and clinical features, but also peritumoral regions and positive lymph nodes. Models have moderate predictive performance. The prediction model constructed by the pCR versus non-pCR grouping method has higher accuracy. Although the combination of radiomics and AI is of great value in predicting TNT response to LARC, it is challenging when used clinically. Therefore, the use of AI and radiomics should be further explored to expand the new field of response prediction in rectal cancer.

Abbreviations

TNT

total neoadjuvant treatment

LARC

locally advanced rectal cancer

TRG

Tumor regression grade

T2WI

T2-weighted images

LR

logistic regression

DL

deep learning

AUC

the area under receiver operating characteristics curve

CR

complete response

AI

artificial intelligence

NCCN

The National Comprehensive Cancer Network

HCR

handcrafted radiomic

DLR

deep learning radiomic

CAPOX

capecitabine combined with oxaliplatin

TME

total mesorectal excision

PACS

picture archiving and communication system

ROIs

regions of interest

ICC

intraclass correlation coefficients

OBB

out of bag

2D-CNN

two-dimensional convolutional neural network

3D-CNN

three-dimensional convolutional neural network

OA

overall appearance

HS

heterogeneity in shapes

AA

around appearance

VOI

volume of interest

CT

computed tomography