Automated Segmentation of the Parotid Gland Based on Atlas Registration and Machine Learning: A Longitudinal MRI Study in Head-and-Neck Radiation Therapy

Abstract

Purpose

To develop an automated magnetic resonance imaging (MRI) parotid segmentation method to monitor radiation-induced parotid gland changes in patients after head and neck radiation therapy (RT).

Methods and Materials

The proposed method combines the atlas registration method, which captures the global variation of anatomy, with a machine learning technology, which captures the local statistical features, to automatically segment the parotid glands from the MRIs. The segmentation method consists of 3 major steps. First, an atlas (pre-RT MRI and manually contoured parotid gland mask) is built for each patient. A hybrid deformable image registration is used to map the pre-RT MRI to the post-RT MRI, and the transformation is applied to the pre-RT parotid volume. Second, the kernel support vector machine (SVM) is trained with the subject-specific atlas pair consisting of multiple features (intensity, gradient, and others) from the aligned pre-RT MRI and the transformed parotid volume. Third, the well-trained kernel SVM is used to differentiate the parotid from surrounding tissues in the post-RT MRIs by statistically matching multiple texture features. A longitudinal study of 15 patients undergoing head and neck RT was conducted: baseline MRI was acquired prior to RT, and the post-RT MRIs were acquired at 3-, 6-, and 12-month follow-up examinations. The resulting segmentations were compared with the physicians' manual contours.

Results

Successful parotid segmentation was achieved for all 15 patients (42 post-RT MRIs). The average percentage of volume differences between the automated segmentations and those of the physicians' manual contours were 7.98% for the left parotid and 8.12% for the right parotid. The average volume overlap was 91.1% ± 1.6% for the left parotid and 90.5% ± 2.4% for the right parotid. The parotid gland volume reduction at follow-up was 25% at 3 months, 27% at 6 months, and 16% at 12 months.

Conclusions

We have validated our automated parotid segmentation algorithm in a longitudinal study. This segmentation method may be useful in future studies to address radiation-induced xerostomia in head and neck radiation therapy.

Introduction

Xerostomia (dry mouth) is a common debilitating adverse effect in patients who have received radiation therapy (RT) for head and neck malignancies (1-8). Severe xerostomia is associated with oral discomfort, increased rates of dental caries, oral infection, and difficulty in speaking and swallowing (6, 8). Several studies have shown that changes in parotid gland morphology (e.g. volume reduction) are associated with decreased saliva production and xerostomia (1-8). Nevertheless, longitudinal imaging studies with larger cohorts are needed to better understand this debilitating side effect, monitor its progression, and evaluate its response to interventions, such as partial parotid sparing (available with intensity modulated RT).

Longitudinal imaging studies of radiation-induced parotid toxicity require an accurate, reliable, and validated imaging method to segment the parotid glands. In the clinic, manual segmentation remains the gold standard for parotid delineation with magnetic resonance images (MRI). However, it is impractical to apply the manual segmentation method to longitudinal or large-scale studies due to time constraints. In addition, manual segmentations are also prone to rater drift and bias. A number of CT-based parotid gland segmentation methods have been investigated (9-16), yet few studies have been conducted using MRIs. Therefore, the goal of this study was to develop an automated, reliable, and robust segmentation method to monitor radiation-induced parotid gland changes by using multiple MRIs.

We proposed combining atlas registration, which captures global variations of anatomy, with machine learning, which captures local statistical features, using kernel support vector machine (SVM), to automatically segment the parotid glands in MR images. This method uses the baseline parotid contours as the atlas and automates the parotid segmentation for post-RT MRIs. Our technology was tested in a longitudinal study of 15 head and neck patients with 1-year follow-up examinations.

Methods and Materials

Our segmentation method consisted of 3 major steps: (1) atlas-based registration; (2) feature SVM training; and (3) parotid gland volume segmentation using trained feature SVM.

Step 1: Atlas-based registration

A hybrid deformable image registration combining a normalized mutual information (NMI) metric with a normalized sum-of-squared-differences (NSSD) metric was used to map the pre-RT MR to the post-RT MRI. The transformation was then applied to the parotid gland binary volume manually contoured from pre-RT MR images (Fig. 1).

Fig. 1.

Flow chart of atlas registration (Step 1).

The term “atlas” in the context of this work is defined as the pairing of structural MRI volumes with their corresponding segmented parotid gland binary volumes. We built a subject-specific atlas for each patient, which included the patient's baseline (pre-RT) MR images and the manually contoured parotid binary volume. The advantage of using the pre-RT MR as the basis for the SVM approach is that the pre-RT MRI often shows anatomy structure that is clearer than that of post-RT MRI, where radiation damage may occur. The MR parotid segmentations at various follow-up times for a specific patient were based on this atlas. Using this patient's post-RT MR scan obtained at follow-up, we aligned the pre-RT MRI to the post-RT MRI, using deformable registration (detailed in the following section) and applied the transformation to the parotid gland binary volume of the pre-RT MRI.

In order to handle the local intensity contrast induced by RT (17) and inhomogeneity changes associated with various MR scans (18, 19), we applied a hybrid image matching metric (10), which is a combination of NMI and NSSD metrics.

$$E(I,J)=\alpha \cdot \mathit{\text{NSSD}}(I,J)-\beta \cdot \mathit{\text{NMI}}(I,J)=\alpha \cdot \frac{1}{N}\sum _x\Vert \frac{I\left(x\right)-{\mu }_{I(x)}}{{\sigma }_{I(x)}}-\frac{J\left(x\right)-{\mu }_{J(x)}}{{\sigma }_{J(x)}}\Vert -\beta \cdot \frac{H(I(x))+H(J(x))}{H(I(x),J(x))}$$ (1)

where I and J denote the pre-RT and follow-up images; H(I) and H(J) denote the marginal entropies I and J; and H(I, J) denotes their joint entropy, which is calculated from the joint histogram of I and J. α and β are the relative weighting of the two terms α= β = 0.5. μ_I = G_s * I denotes the local intensity mean, and σ_I = G_s*(I − μ_I)² denotes the local intensity variation of image, I. The same denotations are for the target image J.G_s, which denotes a Gaussian filter with the kernel size s (the kernel size s was chosen to be 2-3 times the image voxel size). The hybrid matching metric provides a better image alignment than the NSSD or NMI, because they are sensitive only to edges or local image contrast, respectively (10).

After this atlas registration, the registered baseline (pre-RT) MRI had similar global anatomy information as the follow-up MRI. However, the size and intensity of the parotid glands in MRI often change over time after RT (17). In order to capture local changes (intensity contrast variation) of parotid glands and improve the segmentation accuracy, we used this pair with large similarity as the follow-up MRI to train the kernel-based SVM.

Step 2: Support vector machine training

Multiple operators (gradient, enhanced Sobel and Gabor wavelets) were used to extract the local features from registered baseline MRI, and multiple different features were extracted to aid in the kernel-based SVM classification process (Fig. 2). A total of 41 features, some sensitive to the boundary and some sensitive to the microstructures, were made up of the original intensity feature, gradient features, enhanced Sobel features, and Gabor features. Four gradient and Sobel operators were convolved with aligned pre-RT MR images to detect horizontal, vertical, and diagonal edges and strength of edges (20). We also obtained a total of 32 Gabor features on 4 levels and 4 rotations for the original intensity image. Gabor wavelet was used to capture image features in multiple scales and multiple orientations (21, 22). The mother function of the 2-dimensional Gabor wavelet is:

Fig. 2.

Flow chart of SVM training (Step 2). SVM = support vector machine.

$$g(x,y)=\frac{1}{2\pi {\sigma }_x{\sigma }_y}exp[-\frac{1}{2}(\frac{x^2}{{\sigma }_x^2}+\frac{y^2}{{\sigma }_y^2})+2\pi \sqrt{-1}Wx]$$ (2)

with the corresponding Fourier transformation as.

$$G(u,v)=exp[-\frac{1}{2}(\frac{{(u-W)}^2}{{\sigma }_u^2}+\frac{v^2}{{\sigma }_v^2}\left)\right]$$ (3)

where σ_u = 1 / (2πσ_x)and σ_v =1/(2πσ_y) and σ_x and σ_y characterize the spatial extent and frequency bandwidth of the Gabor wavelet, and W is a shifting parameter along the u axis in the frequency domain. The Gabor wavelet can be obtained by dilation and rotation of the mother function. Use of the Gabor wavelet offers at least 2 advantages for parotid gland segmentation. First, with the Gaussian factor in the Gabor wavelet, the noise in the MRI (19) can be smoothed or removed. Second, the multiscale and multi-orientation structure of the Gabor wavelet enables the extractions of edge direction as well as edge strength. The Gabor wavelet provides rich edge maps that correspond to a variety of directions, rather than a single “maximum edge intensity” map.

SVM is a popular, supervised machine learning model with associated statistical learning algorithms that analyze data and recognize patterns for classification and regression analysis (23). The idea behind SVMs is to map the original data points from the input space to a high-dimension (hy-perplane) feature space such that the classification problem becomes simpler in the hyperplane space. The training phase of SVMs looks for a linear, optimal separating hyperplane as the maximum margin classifier with respect to the training data (24). Because the training data are not linearly separable, kernel-based SVM methods are used to classify these features. Kernel-based SVM methods map data from an original input feature space to a kernel feature space of higher dimensionality and then solve a linear problem in that space. In this study, kernel-based SVM was used to identify the features of parotid gland tissue. Although these features may vary greatly among various follow-up MR scans, the kernel-based SVM nonlinearly classifies subjects by extracting different features. The method defines a hyperplane to classify the subjects by minimizing the following function.

$$\frac{1}{2}(w^{T}w+b^2)-C\sum _{i=1}^N{\xi }_i$$ (4)

subject to.

$$y_i(w^{T}k(x_i)+b)\ge 1-{\xi }_i,{\xi }_i\ge 0\text{and}i\in N$$ (5)

where C is a penalty parameter and ξ_i is a slack variable to measure the deviation of training samples. w is the vector of coefficients, and b is a constant offset. To find the optimal input parameter values, the grid search method is used. The index i labels the N training cases. y_i∈; ± 1 is the class label, and x_i is the independent variable. The kernel k is used to transform data from the input to the feature space. There are a number of kernels that can be used in SVM models. In our implementation, radial basis function (RBF) was used as follows

$$k(x_i,x_j)=exp(-\gamma {\Vert x_i-x_j\Vert }^2,\gamma =1/(2{\sigma }^2)$$ (6)

We used the registered pre-RT MRI with multiple features (gradient, Sobel, and Gabor features), as well as the transformed parotid binary volume, to train the RBF kernel-based SVM.

Step 3: Volume segmentation

In order to segment the parotid glands, we performed the same feature extraction process for the follow-up MRIs (Fig. 3). The follow-up MR images and their multiple features (gradient, Sobel, and Gabor features) were the input of trained kernel-based SVM, and the trained SVM adaptively labeled the parotid tissue based on its texture and location. The output of trained SVM is a binary image (volume) consisting of many “0” (nonparotid tissues) and “1” (parotid tissue) points. The 3D parotid segmentation was obtained from the classification-based binary volume using the following steps: (1) the pre-RT segmented parotid volume plus 10-mm margin was defined as the volume of interest (VOI); (2) the VOI was applied to the binary volume to set the volume outside the VOI to 0; (3) a 3D filter was applied to smooth the binary volume; (4) a 0.5 threshold was applied to the filtered volume; and (5) 3D morphology operation was used to eliminate the holes or disconnections in the parotid glands.

Fig. 3.

Flow chart of parotid volume segmentation (Step 3).

Reliability evaluation of the segmentation algorithm

The automatic parotid segmentation results were compared with the gold standard of physicians' manual contours. A common evaluation measure for a segmentation method is the Dice overlap ratio. The Dice overlap ratio is defined as follows:

$$\mathit{\text{Dice}}({\mathit{\text{Vol}}}_1,{\mathit{\text{Vol}}}_2)=\frac{2|{\mathit{\text{Vol}}}_1\cap {\mathit{\text{Vol}}}_2|}{\left|{\mathit{\text{Vol}}}_1\right|+\left|{\mathit{\text{Vol}}}_2\right|}$$ (7)

where Vol₁ represents the voxels of the parotid gland segmented by the automated algorithm, and Vol₂ represents the voxels of the corresponding manual segmentation of the experts.

Results

Subjects

We conducted a longitudinal study of 15 patients receiving radiation therapy for head and neck malignancies, such as laryngeal and oropharyngeal cancers. This MR imaging study was approved by the Ethics Committee of our university. The 15 patients (10 men and 5 women) ranged in age from 40 to 65 years (mean, 49.5 ± 6.6 years). The median radiation dose to the left parotid gland was 50.3 Gy (43.1-61.1 Gy) and 51.9 Gy (38.5-63.2 Gy) to the right parotid gland. All patients received a baseline MR scan prior to RT and an additional 3 MR scans post-RT at 3- and 6-month and 1-year follow-up examinations. Among the 15 patients, 2 patients missed the 6-month follow-up scan, and 1 patient missed the 1-year follow-up scan. A total of 57 MR scans were included in this study.

MRI scan

All MRI examinations were performed with a dedicated head and neck coil, with a Signa model HDxt 1.5-T machine (GE Healthcare). T1-and T2-weighted MR sequences were obtained after an intravenous injection of contrast material (gadopentetate dimeglumine). The parameters for the T2-weighted sequence were TR/TE/ETL = 3000 ∼ 4000/90 ∼ 120/8 ∼ 16 ms, 3.0-mm slice thickness, 256 × 256 matrix, maximum voxel resolution of 0.8 × 0.8 × 3.0 mm³.

Interobserver reliability study

The interobserver reliability study showed consistency in the physicians' manual baseline contours, which were used as the ground truth, as well as in the automatic parotid segmentations, using the 2 different sets of baseline contours.

Parotid gland segmentation: a case report

Figure 4 shows the case of a 41-year-old patient who received intensity modulated RT for laryngeal cancer. The mean dose to the right parotid was 55.26 Gy, and the mean dose to the left parotid was 53.05 Gy. Prior to RT, the baseline volume of the right parotid gland was 28.14 cm³ and 26.98 cm³ of the left parotid. Post-RT, the right parotid gland volumes obtained from the automatic segmentation compared to those from manual segmentation (Dice volume overlap) were 21.55 versus 22.82 cm³ (89.9%), respectively, at 3-month follow-up; 20.86 versus 22.26 cm³ (91.3%), respectively, at 6-month follow-up; and 23.35 versus 25.29 cm³ (89.6%), respectively, at 1-year follow-up. Similarly, the left parotid gland volumes obtained from the automatic compared to those of the manual segmentations were 19.90 versus 21.14 cm³ (91.8%), respectively, at 3 months; 19.71 versus 18.93 cm³ (90.4%), respectively, at 6 months; and 22.96 versus 24.77 cm³ (90.1%), respectively, at 1-year follow-up. Differences between automatically segmented parotid gland volumes and the physicians' manual contours ranged between −8.3% and 7.7% at the 3 follow-up time points.

Fig. 4.

An example of parotid gland volume reduction post-RT. (Top row) MR images of the pre-RT parotid glands (a) and at 3-month (b), 6-month (c), and 1-year (d) follow-up examinations. The automatic segmentation is shown in red, and the manual contour is shown in yellow. (Bottom row) 3D visualization and comparison between automatic (red) and manual (blue) parotid gland segmentation 3 months post-RT. A color version of the figure is available at www.redjournal.org.

Parotid gland segmentation: first follow-up study for all patients

Fifteen pre-RT MR image volumes corresponding to 15 patients were first manually contoured by an experienced radiation oncologist (NW) to obtain respective the left and right parotid gland binary volumes (masks). We built 15 atlases (pre-RT T2-weighted MRI volume plus corresponding parotid gland binary volume comprised an atlas for each patient) for 15 patients in our longitudinal study.

We compared the automatic parotid gland segmentations with the physicians' manual contours for the 42 follow-up MRIs. Figure 5 shows the Dice volume overlap, maximum, and mean surface distance comparisons of the first follow-up time point for the 15 patients. The averaged volumes obtained from the automatic segmentations compared to the manual contours were 20.56 ± 7.63 versus 22.23 ± 8.39 cm³, respectively, for the left parotid glands; and 21.31 ± 6.16 versus 23.01 ± 7.87 cm³, respectively, for the right ones. The averaged Dice volume overlap was 91.1 ± 1.6% (left) and 90.5 ± 2.4% (right); the averaged absolute volume difference was 7.98% (left) and 8.12% (right); the averaged maximum surface distance was 3.46 ± 1.22 (left) and 3.79 ± 1.47 mm (right); and the averaged surface distance was 0.29 ±0.11 mm (left) and 0.32 ± 0.17 mm (right).

Fig. 5.

Dice volume overlap (a), maximum (b) and mean surface distance (c) comparisons of left and right parotid glands for 15 patients at the first follow-up time between the automatic and manual segmentation.

Parotid gland segmentation: all follow-up studies for all patients

For the longitudinal study, the pre-RT parotid gland volume was used as the reference with which to observe the parotid gland volume reduction in all patients at various follow-up times. Figure 6 shows a comparison of the average percentage of volume reduction in the 15 patients, obtained by automatic and manual segmentations. For the manual and automatic segmentations, the percentage of parotid volume reduction was 24.9% ± 8.2% and 26.1% ± 8.4%, respectively, at 3 months post-RT (P=.42); 27.4% ± 7.9% and 27.9% ± 9.3%, respectively, at 6 months post-RT (P= .83); and 16.1% ± 15.6% and 17.3% ± 15.2%, respectively, at 1 year post-RT (P=.55). There were no significant differences (P = .37) between the parotid gland volume reductions at 3 and 6 months post-RT, whereas there were significant difference in volume reductions between the 3-month and 1-year follow-up examinations (P=.019) and the 6-month and 1-year follow-up examinations (P = .014).

Fig. 6.

Comparison of average percentages of volume reduction in 15 patients between the automatic and manual segmentation at different follow-up times.

Discussion

We proposed an automatic MR parotid gland segmentation algorithm with which to study RT-induced parotid volume changes in head and neck cancer RT. In this algorithm, an atlas registration combined the NMI with the NSSD and was used to register the pre-RT MRI to the post-RT MRI, and multiple features were extracted from the registered pre-RT MRI to train the kernel SVM. The trained kernel SVM was subsequently used to perform the segmentation for the post-RT MRI. In this automatic segmentation, the atlas registration was used to capture the radiation-induced global anatomical variation of the parotid glands, whereas the trained SVM with multiple features was used to capture the local statistical structural variation. Automatic segmentation results were compared with physicians' manual contours (gold standard). The average Dice volume overlaps between our segmentations and the manual contours of the bilateral parotid glands were more than 90%. Compared to the pre-RT parotid gland volume, the percentage of parotid volume reduction was 25% at 3 months post-RT, 27% at 6 months post-RT, and 16% at 1 year post-RT.

Parotid gland volume reduction was observed in all 15 patients after RT, which is consistent with previous studies (6-8, 17, 25, 26). Wu et al (8) demonstrated that the post-RT parotid glands had an average of 35% volume reduction compared with normal glands. Vasquez et al (25) observed that the average parotid volume reduction was 14% to 17% at the end of treatment. In a clinical study of 82 patients, Wang et al (6) indicated the average parotid gland volume loss was 20.1% after 3 weeks of RT, 26.93% upon completing RT, and 27.21% at 2 months post-RT.

Histologically, normal parotid glands consist entirely of serous cells with densely packed translucent secretory granules (3, 27). Radiation doses of more than 26 Gy can cause a significant loss of serous acini and reduce the volume of parotid glands (6) during the treatment and at the early follow-up examinations after RT. The percentage of parotid gland volume reduction produced big difference among those studies, which may be caused by (1) different mean doses to parotid glands correlated with parotid damage; and (2) different follow-up times. In the current study, no significant differences in gland volumes were observed between the 3- and 6-month follow-up examination, which is consistent with reports by Wang et al (6) and Nomayr et al (17). The parotid volume increase between 6-month and 1-year follow-up examinations may indicate the parotid recovery after radiation therapy (27).

Due to the complexity of head and neck MRI, most MR-based segmentation methods rely on a single or multiple atlases. Single - or multiple-atlas-based segmentation methods need to address 2 issues: (1) how to register single or multiple atlases to the images that need to be segmented; and (2) how to refine a segmented for the boundary or surface for the single-atlas-based method and how to combine or select the multiple aligned atlases multi-atlas-based method. Therefore, a good atlas-based segmentation system requires not only a robust registration algorithm but also an effective scheme to select the optimal atlas templates that are close to the segmenting image. In our single-atlas-based method, a hybrid deformable registration algorithm based on the NMI and NSSD was used to register the atlas (pre-RT MRI) to the post-RT MR image. This algorithm could capture the variations of anatomy and cope well with the local image contrast changes associated with radiation-induced tissue damage. A kernel SVM was used to combine multiple features extracted from an aligned subject-specific atlas. Our kernel SVM mapped the feature data from the aligned atlas space to the kernel feature space of higher dimensionality and then solved a linear problem separating the parotid and nonparotid tissues. Multiple features from subject-specific atlas pairs were used to train the kernel SVM, and the well-trained SVM based on RBF kernel could then robustly differentiate the parotid tissue from the surrounding tissues by statistically matching multiple texture features.

Future areas of study include speeding up the SVM training and segmentation by testing the sensitivity of multiple features and decreasing the number of feature numbers. Meanwhile, we are conducting a clinical longitudinal study with a larger cohort to further investigate the relationship between parotid volume changes and parotid gland function and validate if volume changes could predict xerostomia.

Conclusions

Studies have shown that volume changes of the parotid glands are correlated with the severity of radiation damage and may serve as a predictor for xerostomia. To better monitor radiation-induced volume change of the parotid gland and fully understand xerostomia in head and neck cancer radiation therapy, we developed a novel automatic MR parotid gland segmentation algorithm based on atlas registration and machine learning. We also demonstrated the feasibility and accuracy of our automatic segmentation algorithm in a clinical study. This segmentation method may be useful as we try to address xerostomia in patients after radiation therapy for head and neck malignancies.

Summary.

Parotid gland volume reduction has been associated with decreased saliva production and xerostomia (dry mouth) in head and neck cancer radiation therapy. In this study, an automated parotid segmentation method, based on atlas registration and machine learning, was developed to accurately quantify radiation-induced parotid gland change, using pre- and post-treatment magnetic resonance imaging. This automated segmentation tool will be valuable to carry out longitudinal or large-scale clinical studies to understand toxicity of the parotid gland and treat radiation-induced xerostomia.