Comparison of atlas-based auto-segmentation accuracy for radiotherapy in prostate cancer

Highlights

• •Auto-contouring accuracy and contouring time were evaluated using two procedures.
• •Dice coefficient was better for the multiple atlases procedure than for one atlas.
• •Contouring time of the multiple atlases procedure is clinically acceptable.

Abstract

Atlas-based auto-segmentation (ABS) procedure used in radiotherapy can be classified into two groups, one using one atlas per patient (sSM) and the other using multiple atlases (sMM). This study evaluated auto-contouring accuracy and contouring time in patients with prostate cancer using the two procedures. The Dice similarity coefficient of sMM was significantly better than that of sSM (prostate [median, 0.81 (range, 0.66–0.91) vs. 0.64 (0.27–0.71), p < 0.01], seminal vesicles [0.49 (0.31–0.80) vs. 0.18 (0.01–0.60), p < 0.05], and rectum [0.81 (0.37–0.91) vs. 0.57 (0.31–0.77), p < 0.01]). The median contouring times were 2.6 (sMM) and 1.3 min (sSM).

1. Introduction

Intensity-modulated radiation therapy (IMRT) is a technique that specifically targets tumors while sparing the normal tissue, thereby leading to improved treatment outcomes for patients with prostate cancer [1], [2]. Consequently, several clinics have widely used IMRT for such patients. However, considerable time is required to contour the relevant structures to establish an IMRT plan [3].

The atlas-based auto-segmentation (ABS) software is used to automatically contour target tumors and normal tissues on the computed tomography (CT) images of a new patient using predefined atlases and a non-rigid registration technique [4]. Several studies have used this software to reduce interobserver contouring variation and time [5], [6], [7], [8], [9], [10], [11]. Other studies have focused on improving the accuracy of non-rigid registration to enhance the accuracy of auto-contouring [12], [13], [14], [15]. Some reports in recent years have focused on improving ABS procedures based on neuroimaging research [16], [17], and these improved procedures have been applied to commercial treatment planning systems (TPSs) [8]. However, because structures created using the ABS software for treatment planning are far from satisfactory, they cannot be used for treatment planning without manual editing performed by a radiation oncologist or a planner [8], [9], [10], [18].

The ABS procedure can be classified into two groups, one using one atlas per patient and the other using multiple atlases per patient. Studies have compared the accuracy of ABS using these two procedures [19] and reported that the latter has a better auto-contouring accuracy [4]. Recently, the hybrid non-rigid registration algorithm, which combines image information by intensity with the structure provided by the contours, has been widely used [12] and reported to perform well compared with conventional algorithms [8].

Because the accuracy of ABS varies with a change in the procedure and algorithm used, verifying the accuracy for each procedure or algorithm is essential. Delpon et al. [10] reported the accuracy of ABS for prostate cancer using the hybrid non-rigid registration algorithm; however, the effect of ABS procedures was not clarified. To the best of our knowledge, the superiority of the procedure of multiple atlases over one atlas has not been observed in the hybrid non-rigid registration algorithm. Therefore, this study aimed to compare the accuracy of ABS for prostate cancer between the two procedures using the hybrid non-rigid registration algorithm.

2. Materials and methods

2.1. ABS procedure

RayStation version 6.2.0.7 (RaySearch Laboratories, Stockholm, Sweden), as a default procedure, implements the selected single atlas from multiple atlases (sSM) procedure (i.e., one atlas per patient). The sSM procedure can be divided into four steps: (1) searching for an atlas similar to the patient’s CT images by similarity measurement, (2) selecting the atlas most similar to the patient’s CT images from predefined atlases using the correlation coefficient, (3) performing non-rigid registration using ANAtomically CONstrained Deformation Algorithm (ANACONDA) as a transformation model, and (4) contouring the patient’s CT images [12]. The present study investigated another procedure (i.e., multiple atlases per patient) by changing from predefined atlases to structures. The process of the procedure can be divided into four steps: (1) making multiple structures per structure, (2) searching and selecting each similar structure using the correlation coefficient in the RayStation functionality, (3) performing non-rigid registration using ANACONDA, and (4) contouring with each structure. Hence, this procedure was named as selected multiple structures from the multiple structures (sMM) procedure.

2.2. Patients and manual contouring

We retrospectively reviewed the volumetric CT datasets of 30 patients with prostate cancer who received IMRT at our center between April 2015 and April 2017. The following seven anatomical structures of the prostate region were contoured on the patient’s CT images: the prostate, seminal vesicles, rectum, bladder, pubis, ischium, and femoral head. The Aquilion LB CT system (Canon Medical Systems, Tochigi, Japan) was utilized under the following conditions: tube voltage, 120 kV; tube current as determined by auto-exposure control (standard deviation = 10); slice width, 2.0 mm; matrix size, 512 × 512; and field of view, 550 cm. All structures of the 30 patients were manually contoured using the delineation consensus guideline for patients with prostate cancer as a reference [20]. The rectum was defined only in the 10-mm area of the craniocaudal direction of the planning target volume with the policy of the center of the present study [21]. The study was approved by the local ethics committee (no. 2018–1-415).

2.3. Creating predefined atlases and structures

The structure sets of 20 patients that were randomly selected from the 30 patients were used for predefined atlases [22] and structures using the ABS software of RayStation to create two different predefined atlases or structures. The two different ABS procedures (sSM and sMM) were then performed for the remaining 10 patients.

2.4. Evaluation of auto-contouring accuracy and contouring time

To evaluate auto-contouring accuracy, similarity between ABS and manual contouring was assessed using the Dice similarity coefficient (DSC) calculated as follows (Eq. (1)):

$$\mathrm{D}\mathrm{S}\mathrm{C}=\frac{2|A\cap B|}{|A|+|B|}$$ (1)

where A and B are the two structures evaluated. DSC may take any values between 0 (no overlap) and 1 (complete overlap). A is the manually contoured structure and B is the structure that was auto-contoured using the sSM or sMM procedure. Following previous studies [23], [24], DSC values > 0.7, 0.7–0.4, and <0.4 were defined as substantial agreement, moderate agreement, and large variation, respectively. The Hausdorff distance computation utilizes the maximum–minimum function as defined by Eq. (2):

$$h\left(a,b\right)={\mathrm{m}\mathrm{a}\mathrm{x}}_{a\in A}\left\{{\mathrm{m}\mathrm{i}\mathrm{n}}_{b\in B}\left\{d(a,b)\right\}\right\}$$ (2)

where a and b are the points of structures A and B and d(a,b) is the Euclidian distance between a and b.

The time required to perform ABS was measured in each procedure. Auto-contouring on RayStation was performed using a desktop computer with Windows 7 Professional with an Intel Xeon CPU at 3.2 GHz and 64 GB RAM.

2.5. Statistical analysis

The Wilcoxon signed-rank test was used to compare differences in DSC values and Hausdorff distance between the sSM and sMM procedures. Two-sided p values < 0.05 were considered significantly different. All statistical analyses were performed using EZR version 1.3.6 (Saitama Medical Centre, Jichi Medical University, Saitama, Japan), a graphical user interface for R (The R Foundation for Statistical Computing, Vienna, Austria) [25].

3. Results

3.1. Auto-contouring accuracy

In the sSM procedure, DSC values indicated substantial agreement (>0.7) for the pubis, ischium, and femoral head; moderate agreement (0.7–0.5) for the prostate, rectum, and bladder; and large variation (<0.5) for the seminal vesicles. By contrast, in the sMM procedure, DSC values indicated substantial agreement (>0.7) for the prostate, rectum, bladder, pubis, ischium, and femoral head and moderate agreement (0.7–0.5) for the seminal vesicles. Significant differences in DSC values between the two procedures were noted in the prostate (median, 0.64 (range, 0.27–0.71) vs. 0.81 (0.66–0.91), p < 0.01), seminal vesicles (0.18 (0.01–0.60) vs. 0.49 (0.31–0.80), p < 0.05), and rectum (0.57 (0.31–0.77) vs. 0.81 (0.37–0.91), p < 0.01) (Fig. 1a).

Fig. 1.

Boxplots of the (a) Dice similarity coefficient and (b) Hausdorff distance between the manual contours and the two procedures. The points represent the maximum, upper quartile, mean, median, lower quartile, and minimum values. sSM, selected single atlas from multiple atlases; sMM, selected multiple structures from the multiple structures.

The qualitative examples of the segmentation results are shown in Fig. 2. In particular, the contouring accuracy of sMM was improved compared with that of sSM with respect to the prostate, seminal vesicles, and rectum. Likewise, the result of the Hausdorff distances showed a significant difference among the prostate (p > 0.05), seminal vesicles (p > 0.05), and rectum (p > 0.05), but no significant difference was found in other contours (Fig. 1b).

Fig. 2.

Segmentation results for (a) manual contouring, (b) sMM procedure, and (c) sSM procedure; differences between (a–b) and (a–c) are shown in (d) and (e). Red line, the prostate; blue line, the seminal vesicles; orange line, the rectum; and yellow line, the bladder. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

The median time required to contour the seven structures in the sSM and sMM procedures was 1.3 (range, 1.2–1.4) and 2.6 (2.5–2.8) min, respectively.

4. Discussion

The auto-contouring accuracy of the sMM procedure using the hybrid non-rigid registration algorithm was found to be better than that of the sSM procedure. The median DSC values of the sSM procedure were nearly equivalent to those found in a previous study [10]. However, accuracy with regard to the prostate, seminal vesicles, and rectum is insufficient for clinical use and necessitates improvement. The sSM procedure has the disadvantage of selecting only one atlas that is similar to the patient. Because it is practically not possible to register and choose an atlas that roughly matches the patient with respect to all regions, the auto-contouring accuracy becomes low. In the sMM procedure, the auto-contouring accuracy was improved because the most similar structure could be selected from multiple structures.

In the bladder, by contrast, no significant difference was observed between the sSM and sMM procedures. Raudaschl et al. reported that the auto-contouring accuracy of large organs is better than that of small organs in ABS [11]. Each patient in the present study was instructed not to empty their bladder 3 h prior to treatment. The median bladder volume of all patients was 175 (range, 91–298) mL. Because the bladder volume was large, auto-contouring accuracy was good in both the sSM and sMM procedures but without significant differences. Moreover, in the pubis, ischium, and femoral head, no significant differences were noted. Because of the high contrast of the bony structure with the neighboring areas and low degree of shape variation, DSC values were good in both procedures but without significant differences [10]. DSC is sensitive to both over- and undercontouring and has the major advantage of having the ability to evaluate both false positives and false negatives between specific voxels. However, its disadvantage is that the DSC value of a large structure volume is higher than that of a small structure volume [4]. In particular, bigger volumes that only have a small disagreement will still have a larger DSC value. Smaller volumes with a slight disagreement, on the other hand, will have a larger decrease in the DSC value. Hence, the Hausdorff distance was used as a non-DSC indicator. However, no significant difference was noted in the Hausdorff distance (Fig. 1b).

Although the number of non-rigid registrations in the sMM procedure increased more than that in the sSM procedure, the calculation load was not extremely significant. In fact, the contouring time of the sMM procedure was approximately 1 min longer than that of the sSM procedure, but this increase is clinically acceptable.

Delays in radiotherapy initiation should be as short as reasonably achievable because delays are associated with an increased local recurrence rate [26]. However, the manual contouring of a large number of structures is time-consuming, burdening radiation oncologists and planners. Hence, auto-contouring technology is considered important in clinical practice. The mainstream of current research on auto-contouring is deep learning, showing an improved accuracy over ABS [27]. However, deep learning requires a large-scale dataset that is suitable for the target site. Moreover, commercial TPS does not generate datasets for all sites. By contrast, good results can be obtained with only a few dozen datasets with ABS, as shown in this study. In addition, the accuracy of deep learning can be improved by continuous learning on a cloud basis. However, cloud-based operation is not possible in certain countries because of legal regulations. Thus, it is believed that ABS may be an option in cases where deep learning datasets are not generated or when cloud management is difficult.

The present study has two major limitations. First, the number of predefined atlases was set to 20 in this study, as per Larrue et al.’s report [22]. However, the optimal number of atlases remains an open problem and depends on certain variables, including the target anatomy and image modality, among others. A detailed data analysis of other sites (e.g., head and neck) is necessary for future studies to acquire more useful results. Second, this study was conducted in a single facility that used a single commercial TPS. Because the similarity measurement and non-rigid registration algorithm adopted for each TPS are different, a multicenter study is needed to compare the robustness and effectiveness of the sMM procedure.

In conclusion, we evaluated auto-contouring accuracy and contouring time for prostate cancer using two different ABS procedures using the hybrid non-rigid registration algorithm. The sMM procedure demonstrated more enhanced auto-contouring accuracy compared with the sSM procedure. Furthermore, although the contouring time of the sMM procedure was slightly increased compared with that of the sSM procedure, this increase is clinically acceptable

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.