A deep learning model for predicting xerostomia due to radiotherapy for head-and-neck squamous cell carcinoma in the RTOG 0522 clinical trial

Abstract

Purpose:

Xerostomia commonly occurs in patients who undergo head-and-neck radiotherapy and can seriously affect patients’ quality of life. In this study, we developed a xerostomia prediction model with radiation treatment data using a three-dimensional residual convolutional neural network (3D rCNN). The model can be used to guide radiotherapy to reduce toxicity.

Methods and Materials:

A total of 784 patients with head-and-neck squamous cell carcinoma enrolled in the RTOG 0522 clinical trial were included in this study. Late xerostomia is defined as xerostomia of ≥grade 2 occurring in the 12th month of radiotherapy. The computed tomography (CT) planning images, 3D dose distributions, and contours of the parotid and submandibular glands were included as 3D rCNN inputs. Comparative experiments were performed for the 3D rCNN model without one of the three inputs and for the logistic regression model. Accuracy, sensitivity, specificity, F-score, and area under the receiver operator characteristic curve (AUC) were evaluated.

Results:

The proposed model achieved promising prediction results. The performance metrics for 3D rCNN model with contour, CT images, and radiotherapy dose; 3D rCNN without contour; 3D rCNN without CT images; 3D rCNN without the dose; logistic regression with the dose and clinical parameters; and logistic regression without clinical parameters were as follows: accuracy: 0.76, 0.74, 0.73, 0.65, 0.64, and 0.56; sensitivity: 0.76, 0.72, 0.77, 0.59, 0.72, and 0.75; specificity: 0.76, 0.76, 0.71, 0.69, 0.59, and 0.43; F-score: 0.70, 0.68, 0.69, 0.56, 0.60, and 0.57; and AUC: 0.84, 0.82, 0.78, 0.70, 0.74, and 0.68, respectively.

Conclusions:

The proposed model uses 3D rCNN filters to extract low- and high-level spatial features and to achieve promising performance. This is a potentially effective model for predicting objective toxicity for supporting clinical decision-making.

1. Introduction

Head-and-neck cancer (HNC) has been effectively treated using radiotherapy. However, treatment-related toxicity is a significant issue due to the close proximity of the cancer to normal tissues and organs. Modern radiotherapy techniques, such as intensity-modulated radiotherapy (IMRT), are used to spare the normal tissues surrounding the cancerous tissue to a greater extent than that with conventional techniques, thus, reducing radiation-induced toxicities ^[1]. Nonetheless, xerostomia remains a common sequelae, negatively impacting patients’ quality of life by affecting chewing and swallowing functions; compromising taste, speech, and sleep patterns; and contributing to worsened dentition ^[2,3].

Accurately predicting toxicity will facilitate clinical decision-making for planning personalized treatment. The currently available normal tissue complication probability models ^[4] do not provide any spatial information that may be crucial for accurately predicting toxicity. Recently, several studies have explored the use of radiomics features ^[5–9] extracted from images [computed tomography/magnetic resonance imaging (CT/MRI)] and/or the dosiomic features ^[9–11] extracted from three-dimensional (3D) dose distributions to improve the prediction models of treatment complications. Generally, a majority of these methods manually extract the features, which are then modeled using traditional machine learning methodologies. However, only low-level information is captured in such handcrafted features. Progressively matured deep learning models ^[12–17] can automatically extract hierarchical features from the data. Several studies have attempted to use this automated feature of 3D convolutional neural network (CNN) ^[18,19] to extract radiomics features for predicting the response to cancer treatment and have achieved promising results. Radiotherapy dose distribution is also an important predictor considering outcomes of this therapy. Zhen et al.^[20] have introduced a two-dimensional (2D) CNN model to analyze dose distribution on 2D rectal surface and predict rectal toxicity. Ibragimov et al. ^[21] have investigated the use of 3D dose plans as CNN inputs for predicting hepatobiliary toxicity. However, the information obtained from both CT images and radiotherapy dose distribution has not been fully utilized.

We hypothesized that xerostomia can be related to both CT images (radiomics) and radiotherapy dose distribution (dosiomics); therefore, we proposed a hybrid predictive model comprising a 3D residual CNN (rCNN) with the aforementioned inputs. In this study, the proposed model was used for predicting xerostomia in patients with HN squamous cell carcinoma (HNSCC) enrolled to the RTOG 0522 clinical trial. The number of cases for training from RTOG 0522 is sufficient for a robust deep learning model. The 3D rCNN inputs included CT planning images, 3D dose distributions, and the contours of the parotid and submandibular glands.

2. Methods and Materials

2.1. Patients

The retrospective patient cohort in this study comprised patients with HNSCC enrolled in RTOG 0522, a clinical trial registered with the National Cancer Institute (NCT00265941) and approved by the central and institutional review boards of the 151 participating centers. All patients provided written informed consent prior to participation.

Eligible patients had an untreated, histologically confirmed stage III or IVa/b (T2N2-3M0 or T3-4, any N, M0) oropharyngeal, hypopharyngeal, or laryngeal squamous cell carcinoma based on the AJCC Staging System (6th Edition) with a Zubrod performance score of 0–1. They additionally met the predefined blood chemistry criteria. Patients who presented the participating centers from November 2005 to May 2009 were enrolled. The experimental regimen comprised radiotherapy with concurrent cisplatin without (arm A) or with cetuximab (arm B). For conformal radiotherapy (CRT), doses of 72 Gy were administered in 42 fractions over 6 weeks with twice-a-day irradiation for 12 treatment days. For IMRT, a different accelerated schedule comprising a dose of 70 Gy was administered in 35 fractions (2 Gy per fraction) over 6 weeks with twice-a-day irradiation once a week for 5 weeks. The equivalent dose of 2 Gy per fraction (EQD2) was calculated for each patient using the following formula: EQD2 = n × d × (α/β + d)/(α/β + 2), with an α/β ratio of 3 for OARs. EQD2 was applied to all the analyses in this study. Patient characteristics are indicated in Supplemental Table S1.

2.2. End-points

Follow-up reports were collected at 8–9 weeks post-treatment and at 6, 9, and 12 months after initiating the treatment for the first year, every 3 months for the second year, every 6 months for third to fifth years, and annually thereafter. Important adverse events commonly associated with HNC treatment were assessed according to the Common Terminology Criteria for Adverse Effects version 3.0. The primary end-point in this study was moderate-to-severe xerostomia (≥grade 2) occurring by the 12th month after initiating radiotherapy. All toxicities occurring at 6, 9, and 12 months (excluding those occurring before 6 months) were included, thus, censoring patients without follow-up data at 12 months. Patients with incorrect CT images or dose distribution data were also excluded. Finally, 784 patients were included to model xerostomia; of them, 279 patients with xerostomia toxicity ≥grade 2 were categorized as toxicity cases and 505 patients with xerostomia toxicity grade 0–1 were classified as non-toxicity cases.

2.3. Data Pre-processing

Xerostomia is associated with parotid and submandibular gland functions ^[6]. Because some of the glands were either not or inconsistently contoured, a deep learning-based auto-segmentation approach ^[22] was used to contour the parotid and submandibular glands in all patients included in the study. Training data for segmentation models were obtained from “Auto-Segmentation Challenge 2015” ^[23], which included a subset of 40 images from the RTOG 0522 clinical trial. Delineation guidelines were developed by performing an extensive literature research. All structures used were re-segmented by experts to provide uniform quality and consistency. Data augmentation was adopted using left-right flipping, random scaling of input images (0.5–1.5), and random cropping.

The CT planning images, dose distributions, and delineated structures were used to model xerostomia. Data were pre-processed using MATLAB R2017b (MathWorks, Inc., Natick, Massachusetts, United States). Both 3D CT images and the corresponding 3D dose distributions were axially resampled to a pixel size of 2.00 χ 2.00 mm² and a cross-sectional thickness of 5.0 mm.

2.4. 3D rCNN Model for Predicting Xerostomia

A 3D rCNN model was trained for xerostomia prediction in this study. Figure 1 shows 3D CT images, 3D dose distributions, and contours (parotid and submandibular glands) as inputs and xerostomia prediction as the output. Specifically, Table S2 shows the 3D rCNN framework with 23 layers. A residual network was adopted to improve the performance. The deeper bottleneck architecture (DBA) is the core deep residual network. 3D convolution and pooling kernel size were calculated as k × k × d, where k is the kernel spatial size and d is the kernel depth. Input dimensions were 112 × 112 × 16 for CT images, dose distributions, and contours, respectively. The first convolution layer (Conv3D_1) had a kernel size of 7 × 7 × 3. Four DBAs (Conv3D_i, i = 2, 3, 4, and 5) were computed following Conv3D_1. Figure S1 illustrates that each DBA comprised a block of five convolution layers: 1 × 1× 1 (Conv_a), 3 × 3× 3 (Conv_b), 3 × 3× 3 (Conv_c), 3 × 3× 3 (Conv_d), and 3 × 3 × 3 (Conv_e). The stride in the longitudinal direction was set to 1 in Conv3D_1 and Conv3D_2 to avoid early merging of the longitude features. Pooling, fully connected (FC), and softmax loss layers followed the last convolution layer (Conv3D_5) for predicting xerostomia. Approximately 20 M weighted parameters were identified in the proposed network.

Fig. 1.

The flowchart for predicting xerostomia using the 3-dimensional residual convolutional neural network model. The network inputs include computed tomography images, radiotherapy dose distributions, and contours and the output is xerostomia probability. The values under the layer names represent the number of output features.

2.5. Experiments

A train-validation-test experiment was conducted to evaluate the performance of the 3D rCNN model. Approximately 80% of the data were randomly selected as the training set, 10% were randomly used as the validation set to tune the parameters and to identify the optimal model, and the remaining 10% were used as the test set to assess the final performance of the proposed method. During the training phase, random rotation (−10° to 10°), random cropping, and random left–right flipping were adopted for data augmentation to expand the existing training dataset to avoid network overfitting. To address the imbalanced outcome (with toxicity event of approximately 35%), cases in the toxicity class were oversampled. Networks were trained from scratch using a batch size of 1, momentum of 0.9, weight decay of 0.00005, and initial learning rate of 0.001. The learning rate was divided by 10 after each 50 K iterations, and the training was terminated after 100 K iterations.

Comparative experiments were conducted without one of the three inputs (CT images, dose distributions, or contours). In models without any one of the three labels, the body outline was used as the input channel.

For comparison, a logistic regression (LR) model using dose metrics and clinical variables was also performed for predicting xerostomia. The mean dose and relative volume of the parotid and submandibular glands receiving 20 Gy of radiation (MD_par and V20_par) and (MD_subm and V20_subm), respectively, were used as dose metrics. Clinical variables included sex, age, race, treatment arm (RT + cisplatin or RT + cisplatin + cetuximab), treatment technique (CRT or IMRT), tumor site (larynx, hypopharynx, or oropharynx), T stage, N stage, and Zubrod performance score. Because the clinical variables were not included in the proposed 3D rCNN, an LR model that only used the dose metrics was also constructed. The training, validation, and test sets in the LR model were the same as the ones used for the 3D rCNN. Data analysis was performed using MedCalc (MedCalc Statistical Software, Mariakerke, Belgium). The forward selection approach was used to select the significant (p < 0.05) variables entered into the model sequentially.

2.6. Quantitative Evaluation

The model’s performance was quantified in terms of the mean accuracy [accuracy = (TP + TN)/(TP + FP + FN + TN)], sensitivity [sensitivity = TP/(TP + FN)], specificity [specificity = TN/(TN + FP)], F-score [F-score = 2TP/(2TP + FN + FP)], and area under the receiver operator characteristic curve (ROC) (AUC), where TP is true positive, TN is true negative, FP is false positive, and FN is false negative.

3. Results

3.1. Prediction Performance

Table 1 presents the prediction results obtained after applying different methods on the test set. Figure 2 shows the ROC curve of predicting xerostomia. The 3D rCNN model with all CT images, dose distributions, and contours as inputs demonstrated a relatively good performance with an AUC value of 0.84, whereas the model without any of the three aforementioned labels demonstrated lower performance with AUC values of 0.82 (without contour), 0.78 (without CT images), and 0.70 (without dose distributions). This comparative experiment indicated that the model that included all three labels as inputs produced an optimal xerostomia prediction, whereas the model without dose distributions showed poor outcomes, indicating that dose distribution is a crucial factor for predicting xerostomia. For the LR model with clinical variables, six parameters were entered into the model using the forward selection approach, i.e., MD_par, V20_par, race, tumor site, T stage, and N stage. For the LR model without clinical variables, two parameters, i.e., MD_par and V20_par, were selected. The dose metrics of the submandibular gland (MD_subm and V20_subm) were non-significant parameters for the model. One reason for this may be that there was no specific dose limit to the submandibular glands in this clinical trial; thus, the glands of all patients received a similar high-radiation dose, making the dose metrics non-significant. The AUC values were 0.74 and 0.68 for the LR models with and without clinical variables, respectively. Under the same conditions, that is, without adding clinical variables, the result obtained with the 3D rCNN model was much better than that obtained with the traditional LR model (AUC: 0.84 vs. 0.68), which further underlined the quality of the proposed method.

Table 1.

Results of xerostomia prediction

Method	Accuracy	Sensitivity	Specificity	F-score	AUC (95% CI)
3D rCNN	0.76	0.76	0.76	0.70	0.84 (0.74–0.91)
3D rCNN without contour	0.74	0.72	0.76	0.68	0.82 (0.72–0.90)
3D rCNN without CT	0.73	0.77	0.71	0.69	0.78 (0.67–0.88)
3D rCNN without dose	0.65	0.59	0.69	0.56	0.70 (0.58–0.80)
LR without clinical variables	0.56	0.75	0.43	0.57	0.68 (0.56–0.80)
LR with clinical variables	0.64	0.72	0.59	0.60	0.74 (0.64–0.84)

AUC: area under the curve; CI: confidence interval; CT: computed tomography; 3D rCNN: three-dimensional residual convolutional neural network; LR: logistic regression

Fig. 2.

Receiver operating curve analysis for predicting xerostomia using the test set. Area under the curve values were 0.84, 0.82, 0.78, 0.70, and 0.74 for the 3-dimensional residual convolutional neural network (3D rCNN), 3D rCNN without contour, 3D rCNN without CT, 3D rCNN without dose, and logistic regression models, respectively.

3.2. Visualization Features

The deep CNN model adopted CT images, dose distributions, and contours as three stream inputs, which were gradually convolved by multiple 3D kernels within the subsequent convolution layer and residual blocks. Figure 3 shows the visualization of the feature maps of two cases. There were 64, 64, 128, 256, and 512 hierarchical feature maps (from low-level to high-level) of Conv3D_1, Conv3D_2, Conv3D_3, Conv3D_4, and Conv3D_5, respectively. Four feature maps of each Conv3D_n were selected for visualization. As the depth of the network increased, the features became more and more complex and abstract. High-level features of the 3D_Pool layer were connected to the FC layers, with the final predicted xerostomia probability as the output. The feature maps of the 3D_Pool layer measured 1× 1 × 2 × 512 and were reshaped to 32 × 32 for better visualization.

Fig. 3.

Feature map visualization. Feature maps of the first convolution layer (Conv3D_1), Conv3D_2, Conv3D_3, Conv3D_4, Conv3D_5, and 3D_Pool are presented. The 3D_Pool layer was reshaped from 1× 1 × 2 × 512 to 32 × 32 for better visualization.

Figure 4 illustrates the feature maps extracted by the Conv3D_1 in the 3D CNN model with and without contours. Feature maps in the first row were used for the model with contour labels, whereas those in the second row were used for the model without contour labels. This comparison shows that CNN can still automatically extract the features of the most critical areas (such as the parotid gland and its surrounding tissues) even without the input contour.

Fig. 4.

Feature maps of the first convolution layer of 3-dimensional residual convolutional neural network model with and without contours. First row: with contour label; second row: without contour label.

4. Discussion

Xerostomia is one of the most common toxicities of radiotherapy, and it significantly and negatively affects patients’ quality of life. Therefore, a deep 3D rCNN model was developed to automatically extract hierarchical features from both CT images and treatment plans. To the best of our knowledge, this is the first study that used 3D CT planning images, radiotherapy dose distributions, and related contours to extract low- and high-level features for predicting xerostomia. Experimental results have demonstrated that the proposed model can accurately predict xerostomia.

A majority of previous prediction models have focused on dose–volume parameters or conventional handcrafted radiomics features, which may not be discriminative or optimal for the classification of toxicity. Unlike the handcrafted feature extractors, a deep learning-based prediction model can extract and fine-tune features for specific classification tasks by optimizing convolution filters during the training procedure. Another important advantage of a deep learning-based prediction model is its efficiency. Moreover, the features can be automatically extracted by the convolution operations. For deep network, Conv was used to preserve the spatial information. Compared with the 2D mode, 3D Conv has the ability to better combine longitudinal features owing to 3D convolution and 3D pooling operations. In 3D Conv, convolution and pooling operations are performed in three directions, whereas they are only conducted in two directions in transverse section images in 2D Conv.

The comparative experiment shows that a lack of any of the three labels (CT images, radiotherapy dose distributions, or contours) will lead to lower prediction performance. This shows that the 3D CNN model can extract useful information from three types of inputs that improve the model’s robustness and accuracy. The model without radiotherapy dose distribution as the input had the worst performance, which can be explained by the high radiotherapy dose as the main cause of xerostomia. Imaging biomarkers have improved radiation-induced xerostomia prediction ^{[6–7, 24–27]}. Our study also proved that including CT images in the 3D CNN model can improve its accuracy in predicting xerostomia. The performance of the model without contour was a little inferior than that of the model with contour, suggesting that contour labels may facilitate the extraction of more accurate and useful features related to xerostomia.

Deep learning model is difficult to explain; therefore, feature maps (Fig. 3) were added, using which the features of the models with and without contours (Fig. 4) were compared. The maps show that CNN can automatically extract some important features of the parotid gland and its surrounding tissues. The advantage of this approach includes its ability to consider the radiotherapy dose surrounding the structure, as the uncertainty caused by positioning error, parotid shrinkage, and migration during the treatment was implicitly included in the relative dose distribution.

To the best of our knowledge, this is the first study that predicts xerostomia using the 3D CNN model. The performance of this model was compared with that of the LR model for predicting xerostomia, which revealed a better performance of the proposed model (AUC: 0.84 vs. 0.68). The performance of this model was also compared with the reported performance of xerostomia prediction models in the literature (Table 2). Beetz et al. ^[4] obtained an AUC of 0.68 based on the dose parameters. Using a combination of the dose and clinical parameters, Hui et al. ^[28] modeled xerostomia prediction with an AUC of 0.69. Recently, with the development of radiomics, manually extracted image features have been introduced to improve the prediction accuracy. The experiments have shown that adding image features to dose and clinical parameters could help obtain higher AUC values ^{[6,7,24–27]}. Our results also indicated that the inclusion of CT image could improve the model performance, from 0.78 (0.67–0.88) to 0.84 (0.74–0.91). In general, the proposed model demonstrated a comparable or better AUC value of 0.84 (0.74–0.91). Moreover, the proposed method could extract both radiomics and dosiomics features automatically, which is a big advantage over the traditional LR model.

Table 2.

Comparison of xerostomia prediction with other studies

Method	Parameters	AUC (95% CI)
Logistic regression [4]	dose	0.68 (0.60–0.76)
Logistic regression [28]	dose and clinical parameters	0.69 (CI not available)
Logistic regression [6]	dose and clinical parameters	0.75 (0.69–0.81)
	dose, clinical parameters, and CT IBMs	0.77 (0.71–0.82)
Logistic regression [7]	dose and clinical parameters	0.69 (0.62–0.77)
	dose, clinical parameters, and CBCT IBMs	0.78 (0.64–0.91)
Logistic regression [24]	dose and clinical parameters	0.73 (0.65–0.81)
	dose and clinical parameters and PET IBMs	0.77 (0.69–0.84)
Logistic regression [25]	dose and clinical parameters	0.76 (0.67–0.86)
	dose, clinical parameters, and CT IBMs	0.82 (0.72–0.91)
Logistic regression [26]	dose and clinical parameters	0.65 (0.41–0.88)
	dose, clinical parameters, and MRI IBMs	0.83 (0.67–0.99)
Logistic regression [27]	dose and clinical parameters	0.77 (0.65–0.88)
	dose and clinical parameters and CT image texture	0.91 (0.75–0.98)
Our method (3D rCNN)	CT images, dose distributions, and contours of the glands	0.84 (0.74–0.91)

AUC: area under the curve; CI: confidence interval; CT: computed tomography; IBMs: image biomarkers; CBCT: cone-beam CT; PET: positron emission tomography; MRI: magnetic resonance imaging; 3D rCNN: three-dimensional residual convolutional neural network

The manual contouring and feature extraction are both labor-intensive and time-consuming. This study aimed to construct a fully automatic framework, including auto-segmentation and -prediction, to predict toxicity. The auto-segmentation method has been proven to have high performance ^[15] and has the potential to reduce inter- and intra-observer variations in the delineation. We also consulted an experienced radiation oncologist to examine the auto-segmentation of few cases and confirmed the appropriateness of the delineation of glands, particularly that of the parotid gland. The delineation of the submandibular gland was slightly smaller than the actual size of the gland in some areas, but still acceptable. To investigate the influence of contouring accuracy on the prediction performance, a comparative experiment was conducted with/without the contour. The result showed that even without the input contour, CNN could achieve slightly inferior, if not comparable, performance (AUC: 0.82 vs. 0.84). On the other hand, the model without CT (AUC: 0.78 vs. 0.84) or dose distribution (AUC: 0.70 vs. 0.84) performed obviously worse. This indicates that full CT scan and 3D dose distribution are of paramount importance in the model and that their inclusion could make the model less sensitive to delineation errors.

Although the proposed 3D rCNN model demonstrated a promising performance for predicting xerostomia, further studies are warranted owing to the following concerns. First, the 3D network requires a large number of training data. The performance of a 3D model with much larger datasets merits further investigation. Second, clinical variables (e.g., tumor site and clinical stage) can play an important role in the toxicity profile; they were not included in the proposed 3D CNN model. Therefore, we will attempt to re-design the network structure to include other relevant variables such as clinical parameters. Finally, oral cavity contours were not included in our model because previous studies have reported that the features of the parotid and submandibular glands are the most important parameters for predicting xerostomia at 6 or 12 months after radiotherapy ^[4,6,29,30]. Moreover, our data included 3D CT images and dose distributions of the oral cavity, which are already considered in deep learning.

5. Conclusions

A toxicity prediction model based on 3D rCNN was designed and tested in this study. Low- and high-level spatial features were extracted from CT planning images, radiotherapy dose distributions, and contours using 3D filters. The comparative experiment demonstrated a promising performance of the proposed model for predicting xerostomia. The model can be further improved by conducting studies to investigate a more accurate definition for regions that are closely associated with xerostomia.

Summary.

• A model that accurately predicts toxicities may be used to support clinical decisions for personalized treatment planning.

• An automated xerostomia prediction model was developed using 3D residual Convolutional Neural Network (CNN) and demonstrated promising performance.

• This novel model uses CT planning, 3D dose distributions, and contours as inputs and toxicity probability as output.