Reliable Delineation of Clinical Target Volumes for Cervical Cancer Radiotherapy on CT/MR Dual-Modality Images

Abstract

Accurate delineation of the clinical target volume (CTV) is a crucial prerequisite for safe and effective radiotherapy characterized. This study addresses the integration of magnetic resonance (MR) images to aid in target delineation on computed tomography (CT) images. However, obtaining MR images directly can be challenging. Therefore, we employ AI-based image generation techniques to “intelligentially generate” MR images from CT images to improve CTV delineation based on CT images. To generate high-quality MR images, we propose an attention-guided single-loop image generation model. The model can yield higher-quality images by introducing an attention mechanism in feature extraction and enhancing the loss function. Based on the generated MR images, we propose a CTV segmentation model fusing multi-scale features through image fusion and a hollow space pyramid module to enhance segmentation accuracy. The image generation model used in this study improves the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) from 14.87 and 0.58 to 16.72 and 0.67, respectively, and improves the feature distribution distance and learning-perception image similarity from 180.86 and 0.28 to 110.98 and 0.22, achieving higher quality image generation. The proposed segmentation method demonstrates high accuracy, compared with the FCN method, the intersection over union ratio and the Dice coefficient are improved from 0.8360 and 0.8998 to 0.9043 and 0.9473, respectively. Hausdorff distance and mean surface distance decreased from 5.5573 mm and 2.3269 mm to 4.7204 mm and 0.9397 mm, respectively, achieving clinically acceptable segmentation accuracy. Our method might reduce physicians’ manual workload and accelerate the diagnosis and treatment process while decreasing inter-observer variability in identifying anatomical structures.

Background

Introduction

Cervical cancer is acknowledged as one of the foremost gynecologic malignancies and poses a significant threat to women’s health. Radiotherapy is extensively employed in all stages of cervical cancer, as it can effectively decrease the local recurrence rate and enhance the survival rate of patients with cervical cancer [1]. Accurate and reliable delineation of the clinical target volume (CTV) from medical images is a crucial prerequisite and pivotal step for ensuring safe and effective radiotherapy treatment of patients with cervical cancer [2]. Incomplete coverage of the tumor and its infiltrating area in the treatment plan may result in residual disease or recurrence following treatment. Conversely, excessive target volume exceeding the actual tumor and its infiltrating area may cause unnecessary damage to the normal tissues of the patient.

Currently radiation oncologists (ROs) follow the standard practice of manually delineating the clinical target volume (CTV) and planning radiation therapy accordingly. However, manual CTV delineation is time-consuming and laborious and exhibits significant intra- and inter-observer variability. Typically, ROs outline the CTV layer-by-layer on the 2D axial plane of CT images. It is estimated that on average, it takes about 4 h for ROs to complete CTV outlining for a patient, and longer for complex cases. Moreover, during radiotherapy, outlining and developing the radiotherapy plan may need to be repeated multiple times as the patient may change tumor volume, weight loss, tissue edema, etc. The required time for treatment planning is too long for patients, and delays treatment, which to some extent can impact treatment outcomes and prognosis. In addition, standardizing the CTV contour criteria for cervical cancer is difficult due to the presence of potentially invasive tissues, such as established tumors and areas of subclinical malignancy, with unclear borders and irregular shapes, which heavily rely on the radiologist’s level of clinical experience, resulting in significant differences in contours among different practitioners.

Hence, it is of significant clinical value to investigate a rapid and effective computer-assisted method for automatic delineation of cervical cancer target areas, which would greatly alleviate the workload of manual delineation, expedite the treatment process, minimize inter-observer variability among radiologists with varying levels of experience, and ensure the accuracy and consistency of the delineation.

Literature Review and Related Works

With the advent of artificial intelligence, several deep-learning techniques have been employed to segment the CTV in various anatomical sites, including cervical cancer. For instance, Zhikai et al. presented a DpnU-Net, which replaced the entire U-Net encoder section with a dual-path architecture combining a residual module and a densely connected module to segment cervical CTVs and organs-at-risk (OARs), achieving a Dice similarity coefficient (DSC) of 0.86 and a Hausdorff distance (HD) of 5.34 mm for cervical cancer CTVs [3]. Xue et al. proposed a RefineNet-based automatic outlining model for cervical cancer radiotherapy, utilizing a multipath refinement network architecture to extract contextual information from input images directly, and obtaining DSC and HD values of 0.86 and 6.01 mm, respectively, for cervical cancer CTVs in the test set [4]. In the CTV segmentation study for cervical cancer, Shihong et al. suggested a two-stage segmentation method, utilizing a densely connected network (DenseNet) in continuous CT to extract slices of interest in the first stage, followed by a modified U-Net network in the second stage that achieved a DSC value of 0.85 [5]. Agisilaos et al. proposed a coarse-to-fine segmentation method by first using 2D U-Net to locate CTV and its nearby OARs, including bladder, rectum, femoral head, etc., in prostate CT image sequences [6]. Subsequently, CTV was further segmented using segmented OARs as well as the local CTV volume of interest based on 3D U-Net, and this method achieved an average DSC of 0.87 ± 0.05 on a test dataset of 50 patients. Jialin et al. proposed a novel CTV outlining an approach for cervical cancer based on the 3D U-Net architecture with a region-aware re-weighting strategy, which achieved a DSC of 0.793 on the test set [7]. Yankui et al. proposed a 3D Res-U-Net model for automatic segmentation of cervical cancer CTVs, incorporating techniques such as hopping connections, residual modules, and deep supervision based on the 3D U-Net network architecture, and obtained an average DSC value of 0.852 in the test case [8]. Achieving accurate automated segmentation of CTVs is still challenging due to the fact that CTVs contain potentially tumor-diffusing tissues or subclinical disease tissues, with unclear borders and irregular shapes.

While much research has been conducted in recent years on computer-aided clinical care for cervical cancer target area mapping, the limitations of a single modality medical image in conveying information persist. Currently, CT images are the primary basis for CTV mapping and RT planning in most hospitals in China. However, compared to MR images, CT images have limited soft tissue resolution, which makes it difficult to visualize the depth of tumor infiltration into adjacent tissues. Theoretically, the fusion of MR and CT images can provide complementary information, enhancing the information representation of the images and the discriminative power of the network, and providing more effective support for clinical treatment [9]. However, the clinical use of MR scans is often limited by their cost, thus presenting a significant challenge in the acquisition of MR data for image fusion [10].

Recent advancements in image generation technology have provided a potential solution to the challenges associated with medical image acquisition. These techniques can generate the required medical images without the need for physical scanning. Pix2Pix and CycleGAN are popular methods for image-to-image transformation, that are particularly suited for the medical image domain, where paired data is often lacking. Dong et al. utilized Pix2Pix model and added a gradient loss function to the generator architecture for MR To CT conversion [10]. Jelmer et al. also used Pix2Pix to convert low-dose CT images into corresponding high-dose images [11]. CycleGAN has been successfully applied to the synthesis of CT-PET [12], MR-CT [13], CT-MR [6], and T1W-T2W MR [14]. For instance, Jelmer et al. found that CycleGAN models trained with unpaired data outperformed models trained with paired data when generating brain MR-CT images [13], while Agisilaos et al. used the CycleGAN model to generate MR images from unpaired cardiac CT images for a segmentation task [6]. The experimental results showed that using the generated MR Images to assist segmentation improved their segmentation accuracy by 16%. For supervised learning, the main difficulties are the acquisition and labeling of paired datasets and the deformable alignment, which reintroduces geometric uncertainty, that would be involved. The acquisition of unpaired datasets used for unsupervised modeling is relatively easier, but since there is no definite mapping between inputs and outputs, making it more difficult to train.

In this study, we propose a method for automatic outlining of CTV for radiotherapy of cervical cancer using intelligently generated MR images. Our approach involves designing an attention-guided single-loop image generation model that streamlines the cycle structure of CycleGAN to improve training efficiency. We also introduce an attention mechanism and improve the loss function to guide the generator to produce more realistic and high-quality images. Furthermore, we fuse the two modal images, CT images and generated MR images, to compensate for the low resolution of the soft tissue display in CT images and to improve the segmentation accuracy. Compared with the target segmentation of a single modal image, the proposed method fuses the information of different modalities, and the image features of different modalities can complement each other, thereby helping the model to segment the target more accurately. In this paper, when generating MR Modal images, the traditional CycleGAN structure is improved, and the generated images are closer to the real MR Images by modifying the network structure and loss function.

Materials and Methods

Data Acquisition

The experimental dataset randomly selected CT and MR cases of cervical cancer patients who underwent preoperative examinations at Jiangsu Cancer Hospital between 2020 and 2022 and had never been treated as experimental data. To avoid the potential dataset bias, rare cases are excluded during selection. This dataset consisted of 200 cases of CT image data with annotated CTV contours, 114 cases of CT image data, and 199 cases of MR image data for image generation. All the experimental protocols mentioned in our manuscript have been reviewed and approved by the Ethics Committee of Jiangsu Cancer Hospital. The review elements include avoiding subjects’ exposure to unnecessary risks and researchers’ meeting research requirements. To protect patient privacy, all collected data were desensitized to remove personally identifiable information. The review process of the Ethics Committee follows ICH-GCP/ China GCP and relevant Chinese laws and regulations. The relevant clinical norms and medical ethics have been strictly followed to ensure that patients’ privacy was not violated, or disclosed, and patients’ rights and interests were not harmed.

The CT images were acquired using the Philips IQon Spectral CT scanner, GE Discovery CT750 HD, and LightSpeed VCT scanners, with a window position of 40 and a window width of 350. The CT image size was 512 × 512 pixels, and the slice spacing was 1.25 mm. The MR images were scanned and imaged using Philips Ingenia equipment, from which only T1W sequences were collected. The size of the obtained MR image was 784 × 784 pixels, with a slice thickness of 5 mm and a slice spacing of 6 mm. All acquired CT and MR images were stored in DICOM format. The CT images used for CTV segmentation were annotated by experienced radiologists who referred to the RTGO guidelines for CTV contours in cervical cancer [15]. All of these doctors had more than 20 years of working experience, and a total of 5 doctors participated in the labeling. The radiologists accurately annotated the CTV and surrounding organs, such as the bladder and rectum, based on anatomical and lesion morphology, as well as other clinicopathological knowledge. During the data collection process, each case was rigorously tracked by professional technicians according to standardized collection protocols, and all notes were reviewed and modified multiple times to ensure data quality and consistency, providing a reliable data source for the research.

Image Generation Model

To generate high-quality and realistic MR images, an attention-guided single-loop image generation model, named AGSC-GAN (attention-guided single-cycle generative adversarial networks) is proposed based on the CycleGAN framework, as illustrated in Fig. 1. AGSC-GAN employs a single-loop containing two generators and a discriminator, which greatly reduces the training time. To address the issues of unclear details and low authenticity of certain organs and tissues in the generated MR images, channel, and spatial attention mechanisms are incorporated into the image feature extraction module of the generator and discriminator. These mechanisms guide the model to focus on important areas of the image, reduce information loss in irrelevant image domains, improve detail clarity, and thus generate more realistic images. Furthermore, in the loss function, we use the multi-scale structure similarity index measure (MS-SSIM) to calculate the cyclic consistency loss. In addition, maximum mean difference (MMD) losses are introduced for image comparison in the two domains [16]. The modified discrepancy functions are more effective in monitoring the MR image generation process from CT images and facilitate the generation of high-quality images.

Fig. 1.

The structure diagram of AGSC-GAN

Specifically, the workflow of the generator can be explained as a three-step process of “encoding, converting, decoding.” The generator consists of three convolution layers, with instance normalization layers connected after each convolution layer, and the ReLU function is chosen as the activation function [17]. The partial structure of the generator encodes the input image to extract high-level semantic information. Subsequently, nine ResNet blocks are connected after the encoding structure, and the extracted feature vectors are combined [18]. The decoder, consisting of three deconvolution layers and an instance normalization layer, decodes the feature vector into the target domain image. The activation function used in the output layer is Tanh. The discriminator is composed of four convolution layers, with batch normalization layers (BN) connected after each convolution layer, and the LeakyReLU activation function is used [19, 20].

The attention guidance module used in this paper is intended to achieve style transfer and image reconstruction in cross-domain image conversion tasks, allowing the generator to produce more realistic and better-reconstructed images. The attention guidance module in this study incorporates the idea of combining channel attention with spatial attention in the CBAM module [21]. We employ a non-local auto-correlation matrix to perform matrix multiplication operations, thereby avoiding complex operations such as CBAM manual design pooling and multi-layer perceptron [22, 23].

In the AGSC-GAN model presented in this paper, the loss function comprises four components. The first component is the fundamental generative adversarial loss${\mathit{Loss}}_{\mathit{GAN}}\left(G_{X\to Y},,,D_Y\right)$. The second and third components are cyclic consistency loss and identity loss, respectively, which were proposed in CycleGAN. We also improve the calculation method of cyclic consistency loss by incorporating structural similarity. Lastly, we introduce a domain loss. The methods of calculating each loss and their respective roles in model training will be described in the subsequent sections.

$$\begin{array}{c}\hfill \&{\mathit{Loss}}_{\mathit{GAN}}\left(G_{X\to Y},,,G_{Y\to X},,,D_Y\right)=E_{x\sim P_{data\left(x\right)}}\\ \hfill \left[l,o,g,\left({1-D}_Y,\left(G_{X\to Y},\left(x\right)\right)\right)\right]++E_{y\sim P_{data\left(y\right)}}\left[l,o,g,\left(D_Y,\left(y\right)\right)\right]\end{array}$$ 1

In the equation above, $X$ and $Y$ denote the source domain (i.e., CT images) and the target domain (i.e., MR images), respectively, such that $x\in X$ and $y\in Y$, $P_{data\left(x\right)}$ and $P_{data\left(y\right)}$ respectively represent the data distributions of the source and target domains, and $E_{x\sim P_{data\left(x\right)}}$ as well as $E_{y\sim P_{data\left(y\right)}}$ represent the expectations of $x$ and $y$.

The generator $G_{X\to Y}$ generates MR images from CT images, while $G_{Y\to X}$ generates CT images from MR images. The discriminator $D_Y$ is used to discern the generated MR images. The subsequent sections will elaborate on the specific calculation methods and roles of each component in the loss function of the AGSC-GAN model proposed in this study.

We aim to achieve consistency in the learning process of the generative deduction model, which necessitates that, given the input $x$, the output $x\prime$ should closely resemble the original input $x$ after two successive generations by generating $G_{Y\to X}\left(G_{X\to Y},\left(x\right)\right)$. The cyclic consistency loss quantifies the difference between the input $x$ and the forward prediction $G_{Y\to X}\left(G_{X\to Y},\left(x\right)\right)$, which is expressed as ${\mathit{Loss}}_{\mathit{Cycle}}$. In other networks, L1-norm is commonly used for calculating ${\mathit{Loss}}_{L1}$, which evaluates the overall difference between the image by computing the distance between the restored and real images. However, in our paper, MR images are generated from CT images, and the two domain images differ greatly, with L1-norm exhibiting limited effectiveness. Therefore, we introduce multi-scale structural similarity ${\mathit{Loss}}_{MS\_SSIM}$, which utilizes the local structural information of the image to aid the network in better learning the image structure. Multi-scale SSIM computes structural similarity (SSIM) at multiple scales. In Eq. 2, $l$, $c$, and $j$ respectively denote brightness, contrast, and structure comparison.

$$MS\begin{array}{c}\text{\_}SSIM\left(x,,,y\right)=\begin{array}{c}[l_M(x,y){]}^{{\alpha }_M}·\prod _{j=1}^M[c_j(x,y){]}^{{\beta }_j}[s_j(x,y){]}^{{\gamma }_j}\end{array}\end{array}$$ 2

$${\mathit{Loss}}_{MS\text{\_SSIM}}(G_{X\to Y},G_{Y\to X})=1-MS\text{\_SSIM}\left(x,,,G_{Y\to X},\left(G_{X\to Y},\left(x\right)\right)\right)$$ 3

$${\mathit{Loss}}_{\mathit{Cycle}}={\mathit{Loss}}_{L1}+\lambda {\mathit{Loss}}_{MS\text{\_SSIM}}$$ 4

The generator $G_{X\to Y}$ is responsible for generating images in the $Y$ domain. To ensure that the output of $G_{X\to Y}$ is still in the $Y$ domain when given an input $y$, we introduce the identity loss ${\mathit{Loss}}_{\mathit{Identity}}$ to minimize the difference between $G_{X\to Y}$ and $y$.

$${\mathit{Loss}}_{\mathit{Identity}}\left(G_{X\to Y}\right)=E_{y\sim P_{data\left(y\right)}}\left[\Vert ,G_{X\to Y},\left(y\right),-y\Vert \right]$$ 5

Moreover, MMD is utilized to measure the difference between two related but different distributions. It has advantages in assessing the distance between the distribution of source and target domains. Specifically, the equation for MMD is as follows:

$$\begin{array}{c}\hfill {\mathit{Loss}}_{\mathit{MMD}}\left(X,,,Y\right)=\frac{1}{n\left(n,-,1\right)}\sum _{i,j=1}^{n}k(x_i,x_j)\\ \hfill -\frac{2}{\mathit{nm}}\sum _{i=1}^n\sum _{j=1}^{m}k(x_i,y_j)+\frac{1}{m\left(m,-,1\right)}\sum _{i,j=1}^{m}k(y_i,y_j)\end{array}$$ 6

where $n$ and $m$ denote the number of samples in the sample sets $X$ and $Y$, respectively. $k\left(x_i,,,x_j\right)$, $k\left(x_i,,,y_j\right)$, and $k\left(y_i,,,y_j\right)$ represent the distances between corresponding samples in the feature space of the kernel function $k$. The first term denotes the distance within the sample set $X$, the third term denotes the distance within the sample set $Y$, which represents the variance within the sample set, and the second term denotes the distance between the sample sets $X$ and $Y$, which represents the covariance between the two sample sets.

In summary, we give the full loss used to train the AGSC-GAN model, which contains the GAN loss, the cycle-consistency loss to ensure that the restored X-domain images are consistent with the true X-domain images, as well as the authentication loss and the domain adaptation loss to ensure that the generated Y-domain images are consistent with the X-domain image distribution. The formula is as follows:

$${\mathit{Loss}}_{\mathit{Full}}={\mathit{Loss}}_{\mathit{GAN}}+{{\lambda }_{1}Loss}_{\mathit{Cycle}}+{{\lambda }_{2}Loss}_{\mathit{Identity}}+{{\lambda }_{3}Loss}_{\mathit{MMD}}$$ 7

where the weighting factors ${\lambda }_1$, ${\lambda }_2$ and ${\lambda }_3$ are used to control the weight of different losses in the total loss.

CT-MR Image Fusion

To register the generated MR images to the CT images, we used the CT images as the reference and the MR images as the floating images. The registration algorithm for 2D images has reached a relatively mature stage. In this study, we employed an input-level fusion method to combine the MR and CT images. This approach is simple to use, preserves the inherent image characteristics, and allows the model to focus on the subsequent design of the network architecture. We compared three fusion methods: pixel value fusion, channel fusion based on the image spatial domain, and wavelet transform fusion based on the transform domain. The spatial domain-based method involves direct operations on image pixel values or channels. For wavelet transform fusion, we decomposed the image into low and high-frequency coefficients. The low-frequency coefficient represents the contour while the high-frequency coefficients include horizontal, vertical, and diagonal directions, which represent the details. Different fusion strategies were applied to various coefficients to complete the fusion at different levels and directions. Finally, the inverse transform was used to reconstruct the fused image.

CTV Segmentation

To achieve accurate and automatic delineation of cervical cancer target areas, this study proposes a segmentation model, named GSN (generating image-assisted segmentation network), based on a generated image. The structure of the network is illustrated in Fig. 2. Firstly, CT and MR images generated by a generative network are fused to compensate for the limited information provided by a single modality image. Then, based on the architecture of the fully convolutional network, the study employs the hollow space pyramid module to integrate features of multiple scales, thereby improving the accuracy of the segmentation boundary.

Fig. 2.

The structure diagram of GSN

Experiments and Analysis

Experiment Setup

This experiment was conducted on a server with an NVIDIA GeForce RTX 3090 graphics card, an Intel Xeon Gold 622R CPU, and the Windows 10 operating system. Python programming language and PyTorch deep learning framework were utilized to implement the image generation network and CTV segmentation network. Our experiment consists of two parts, the first part is to generate corresponding MR images from CT images using an image generation model, and the second part is to use the generated MR images to fuse with real CT images to help CTV delineation on CT images. For image generation, we used 114 CT cases and 199 MR cases for a total of 4650 CT images and 4676 MR Images from different patients with no overlap between them. For target delineation, we collected 200 doctor-annotated CT cases with a total of 8651 images, which were not related to the aforementioned CT and MR cases, and from different cases.

During the training of the generation network, the training and verification sets were split in an 8:2 ratio based on cases to avoid data-leakage [24]. The original resolution of the CT and MR images was 512 × 512 pixels and 784 × 784 pixels, respectively. All images were scaled to 256 × 256 pixels before being input into the network. The AGSC-GAN model was iteratively trained for 200 epochs using the Adam optimizer, with a learning rate of $2\times {10}^{-4}$ for the first 100 epochs, which was linearly reduced to 0 after the 100th epoch. A batch size of 4 was used to expedite the training process. The generator and discriminator were trained once each at the training step of each epoch, and the model was saved once at the end of each epoch. During the training, the image conversion effect of the generator was evaluated with the verification sets every 10 epochs.

For the CTV segmentation model, the dataset was divided into training and verification sets in an 8:2 ratio based on cases. The Adam optimizer was used to update the model parameters, with an initial learning rate of $1\times {10}^{-3}$, and the learning rate attenuation technology was applied to enhance training stability. The batch size was set to 8, and the model underwent 80,000 iterations. To improve the generalization ability and robustness of the model, the built-in data enhancement library of Pytorch was utilized to enhance the dataset, including random scaling, brightness adjustment, and noise addition. The early stop strategy was adopted to prevent overfitting, where the training was stopped when the loss value on the verification set stopped decreasing or began to rise.

Quality Assessment

The primary focus of this study is to develop an automated system for delineating the CTV using generated images as assistance. To evaluate the quality of the generated images, we employ two types of assessment indicators. The first type includes traditional image quality evaluation metrics such as PSNR (peak signal-to-noise ratio) and SSIM (structural similarity index). The second type involves computing the distribution distance of the generated images in the feature space, which includes FID (Fréchet inception distance) and LPIPS (learned perceptual image patch similarity) [25–27].

PSNR measures the difference in pixel values between the generated image and the ground truth image. SSIM, on the other hand, measures the similarity of the image structure based on brightness, contrast, and structure. $MSE(x,y)$ represents the mean squared error between the generated image x and the real image y. In Eq. 9, ${\mu }_x$ and ${\mu }_y$ represent the mean of the two images, and ${\sigma }_x$ and ${\sigma }_y$ represent the variance of the two images, respectively. ${\sigma }_{\mathit{xy}}$ represents the covariance of the two images, which is calculated in accordance with the standard definition. $C_1$, $C_2$, and $C_3$ are constants in this study.

$$PSNR\left(x,,,y\right)=20{\mathit{log}}_{10}\frac{{\mathit{MAX}}_x}{MSE(x,y)}$$ 8

$$\begin{array}{c}SSIM\left(x,,,y\right)=\frac{\left(2,{\mu }_x,{\mu }_y,+,C_1\right)\left(2,{\sigma }_{\mathit{xy}},+,C_2\right)}{\left({\mu }_x^2,+,{\mu }_y^2,+,C_1\right)\left({\sigma }_x^2,+,{\sigma }_y^2,+,C_2\right)}\end{array}$$ 9

FID evaluates the quality of the generated image by calculating the distribution distance between the generated image and the real image in the feature space. A smaller FID means a more similar distribution of the generated image and the real image. When calculating FID, the InceptionV3 network is usually used to generate feature vectors for the real data set and the generated image set, and the mean values ${\mathrm{\mu }}_{\mathrm{x}}$ and ${\mathrm{\mu }}_{\mathrm{y}}$ are obtained respectively. Then, FID is calculated by the following equation:

$$\begin{array}{c}FID(x,y)=||{\mu }_x-{\mu }_y|{|}^2+T_r({\mathrm{\Sigma }}_x+{\mathrm{\Sigma }}_y-2({\mathrm{\Sigma }}_x{\mathrm{\Sigma }}_y{)}^{\frac{1}{2}})\end{array}$$ 10

where ${\mathrm{\Sigma }}_{\mathrm{x}}$ and ${\mathrm{\Sigma }}_{\mathrm{y}}$ represent the covariance of the real image set and the generated image set, respectively, while ${\mathrm{T}}_{\mathrm{r}}$ denotes the trace of the matrix.

LPIPS, also known as perceptual loss, is another image quality evaluation index that measures the difference between two images. Its calculation equation is as follows:

$$LPIPS\left(x,,,y\right)=\frac{1}{N}\sum _{i=1}^N\sqrt{\sum _j^C{\left(f_j,\left(x_i\right),-,f_j,\left(y_i\right)\right)}^2}$$ 11

where $N$ represents the number of non-overlapping regions shared by the two images, $C$ represents the number of channels extracted by the neural network, and $f_j\left(x_i\right)$ represents the activation value of channel $j$ in the network. A lower value of LPIPS indicates a greater similarity between the two images, whereas a higher value indicates greater differences.

For evaluating the quality of CTV segmentation, several widely used segmentation evaluation metrics are employed to assess the accuracy and robustness of the model, including DSC (Dice similarity coefficient), IOU (intersection over union), HD (Hausdorff distance), and ASD (average surface distance) [28–30]. DSC and IOU are sensitive to the internal filling of the segmented region, and they are simple to implement with fast calculation speeds. HD and ASD are used to evaluate the edge smoothness of segmentation algorithms. DSC and IOU are used to measure the overlap between the predicted segmentation results and the true segmentation results. The closer the values of DSC and IOU are to 1, indicating greater accuracy in the segmentation results. If X and Y are used to denote the predicted segmentation result and the true segmentation result, respectively, the calculation process of DSC and IOU is as follows:

$$DSC\left(X,,,Y\right)=\frac{2\left|X,\cap ,Y\right|}{\left|X\right|+\left|Y\right|}=\frac{2TP}{2TP+FN+FP}$$ 12

$$IOU\left(X,,,Y\right)=\frac{\left|X,\cap ,Y\right|}{\left|X,\cup ,Y\right|}=\frac{\mathit{TP}}{TP+FN+FP}$$ 13

where TP, FP, and FN denote true positives, false positives, and false negatives in the confusion matrix of the segmentation, respectively. The key difference between DSC and IOU lies in the difference in the denominator. DSC penalizes more for different parts of the real segmentation region and the predicted segmentation region and is relatively more sensitive. The IOU has a smaller penalty for different parts of the real segmentation region and the predicted segmentation region and is more biased towards evaluating the similarity of the intersection part of two regions.

HD measures the maximum distance between two sets and has good interpretability but is more complex to compute. Suppose that there are two sets A = {$a_1$, $a_2$ …, $a_3$}, B = {$b_1$, $b_2$…, $b_3$}, then the HD between these two sets is defined as follows:

$$\begin{array}{c}\hfill HD\left(A,,,B\right)=max\\ \hfill \left(\begin{array}{c}\mathit{max}\\ a\in A\end{array},\left(\begin{array}{c}\mathit{min}\\ b\in B\end{array},\Vert a-b\Vert \right),,,\begin{array}{c}\mathit{max}\\ b\in B\end{array},\left(\begin{array}{c}\mathit{min}\\ a\in A\end{array},\Vert a-b\Vert \right)\right)\end{array}$$ 14

ASD measures the average distance between two segmented surfaces, which can accurately reflect the precision of segmentation results, especially in the case of complex surface geometries. A smaller value of HD and ASD implies better segmentation performance. Assuming that the two segmented surfaces are A and B, where $N_A$ denotes the total number of pixel points on the segmentation surface A, the ASD calculation process is as follows:

$$ASD\left(A,,,B\right)=\frac{{\sum }_{a\in A}\underset{bϵB}{\mathit{min}}d(a,b)}{|N_A|}$$ 15

In this study, we use multiple evaluation metrics to assess the error of segmentation algorithms from different perspectives, which can more comprehensively assess the performance and advantages and disadvantages of segmentation algorithms.

Experimental Results

To ensure the quality of the generated MR images, we conducted several experiments to verify the performance of the AGSC-GAN image generation model. We designed a comparative study between AGSC-GAN and CycleGAN networks to evaluate the advantages and disadvantages of AGSC-GAN in terms of image generation compared to CycleGAN. We trained and compared the performance of the two models on the same dataset.

Table 1 presents the results of our experiments using various evaluation metrics, including PSNR and SSIM, which focus on the quality of image pixels and structure, and FID and LPIPS reflect the quality of image visual perception. Our findings demonstrate that the AGSC-GAN model proposed in this study outperforms CycleGAN in terms of quantitative indicators and produces higher image quality when generating CT images from MR images. Specifically, AGSC-GAN achieved an increase in PSNR and SSIM by 12% and 16%, respectively, compared to CycleGAN. Additionally, FID and LPIPS also showed an improvement of 39% and 21%, respectively.

Table 1.

Performance comparison of AGSC-GAN and CycleGAN

Model name	Generation type	PSNR	SSIM	FID	LPIPS
CycleGAN	CT→MR	14.87	0.58	180.86	0.28
AGSC-GAN	CT→MR	16.72 $(12\mathbf{\%}\uparrow )$	0.67 $(16\mathbf{\%}\uparrow )$	110.98 $(39\mathbf{\%}\downarrow )$	0.22 $(21\mathbf{\%}\downarrow )$
CycleGAN	MR→CT	19.90	0.74	124.29	0.21
AGSC-GAN	MR→CT	20.30 $(2\mathbf{\%}\uparrow )$	0.75 $(1\mathbf{\%}\uparrow )$	107.15 $(14\%\downarrow )$	0.20 $(5\mathbf{\%}\downarrow )$

To further validate the effectiveness of our proposed model, we conducted an MR-to-CT conversion task, which holds significant implications for medical practitioners and patients alike, though not directly relevant to the subsequent auxiliary segmentation in this study. As presented in the third and fourth rows of Table 1, AGSC-GAN outperforms CycleGAN in the MR-to-CT generation task, as evidenced by improvements in all four evaluation indices, namely 2%, 1%, 14%, and 5%, thus providing further evidence of the superiority of the AGSC-GAN model. Additionally, we noted that the quality of CT images generated by both CycleGAN and AGSC-GAN was higher than that of the MR-generated ones, which we attribute to the greater complexity of organizational information in MR images. Furthermore, compared to MR-to-CT generation, the improvement in CT-to-MR generation was more noticeable for the AGSC-GAN model than for CycleGAN, likely because CycleGAN performed relatively well in the MR-to-CT task and had limited potential for enhancement.

To provide a more intuitive comparison of the image generation results obtained using the AGSC-GAN model developed in this study, we display the results of CT-to-MR image conversion in Fig. 3 by comparing the image generation effects by using CycleGAN and AGSC-GAN. Details are highlighted using yellow arrows. In Figs. 3 and 4, it is evident that the images generated by AGSC-GAN exhibit higher consistency with the real images and retain more details in complex regions. This is possibly attributed to the attention mechanism incorporated into the network, the loss of MS-SSIM structure consistency, and the loss of MMD domain migration. The generated MR images can be relied upon to assist in target segmentation on CT images, which might provide a valuable reference for future research in medical image processing tasks.

Fig. 3.

Comparison of the quality of the MR images generated by CycleGAN and AGSC-GAN

Fig. 4.

Comparison of the quality of the CT images generated by CycleGAN and AGSC-GAN

MR images are registered with CT images when generated using the proposed AGSC-GAN model. To evaluate the impact of various fusion methods on segmentation results, we compared the effects of pixel fusion, channel fusion, and wavelet transform fusion, as shown in Fig. 5. Pixel fusion, which is a simple weighted average of pixel values of two images, is advantageous due to its simple implementation and high efficiency. Channel fusion, which fuses different channels of two images, is capable of better fusion of image information. Wavelet transform fusion is capable of better handling high-frequency details of the image.

Fig. 5.

Results of different approaches for image fusion

To verify the effectiveness of various fusion methods on target segmentation, we utilized the fused images obtained from the three fusion methods mentioned earlier as inputs for the segmentation network and evaluated the output segmentation results using various indices. For the experiments, we selected two different split network models: the conventional U-Net and our proposed GSN. The segmentation results of the two models under different fusion methods were compared, and several evaluation indices, such as DSC, IOU, HD, and ASD, were used for the evaluation. The obtained results are presented in Table 2. We use the CTV segmentation result metrics of the unfused original CTs as a baseline, from which we calculated the impact of different fusion methods on the results.

Table 2.

Effect of different fusion methods on CTV segmentation

Network model	Fusion methods	DSC	IOU	HD	ASD
U-Net	Original CT without fusion	0.8363	0.8975	5.6004	3.0246
	Pixel fusion	0.8649 $(3\%\uparrow )$	0.9205 $(2\%\uparrow )$	5.3362 $(5\%\downarrow )$	2.0940 $(31\%\downarrow )$
	Channel fusion	0.8534 $(2\%\uparrow )$	0.9119 $(2\%\uparrow )$	5.4819 $(2\%\downarrow )$	2.8092 $(7\%\downarrow )$
	Wavelet transform fusion	0.8701 $(4\mathbf{\%}\uparrow )$	0.9239 $(3\mathbf{\%}\uparrow )$	5.2882 $(6\mathbf{\%}\downarrow )$	1.7562 $(42\mathbf{\%}\downarrow )$
GSN	Original CT without fusion	0.8737	0.9266	5.1895	1.3911
	Pixel fusion	0.8896 $(2\%\uparrow )$	0.9368 $(1\%\uparrow )$	4.9179 $(5\%\downarrow )$	2.2316 $(60\%\uparrow )$
	Channel fusion	0.9006 $(3\mathbf{\%}\uparrow )$	0.9437 $(2\mathbf{\%}\uparrow )$	4.7571 $(8\mathbf{\%}\downarrow )$	1.3067 $(6\mathbf{\%}\downarrow )$
	Wavelet transform fusion	0.8797 $(1\%\uparrow )$	0.9305 $(1\%\uparrow )$	5. 0692 $(2\%\downarrow )$	2.1168 $(52\%\uparrow )$

From Table 2, it is evident that pixel fusion, channel fusion, and wavelet transform fusion can enhance the segmentation performance of using both the U-Net network and our proposed GSN network. It is worth noting that the U-Net model exhibited the best performance with wavelet transform fusion, whereas the GSN model performed the best with channel fusion. This observation indicates that different segmentation networks display different adaptability towards individual fusion modes. Furthermore, our proposed multi-modal image fusion technique can enhance the precision and stability of cervical cancer target segmentation.

To improve the accuracy of cervical cancer target segmentation, this study enhanced the data and network individually while designing the CTV segmentation model. In terms of data, CT images were supplemented with MR images generated to augment the image information. In terms of network structure, the design was based on a full-convolution network, utilizing ResNet structure as the backbone network for image feature extraction. Following the acquisition of advanced semantics, the feature maps from multiple receptive fields were fused by incorporating ASPP. To evaluate the efficacy of the aforementioned steps in improving the model performance, we conducted independent experiments for each step, and the results are recorded in Table 3.

Table 3.

Comparison of GSN segmentation model improvements

Experiments	Ways of improvement	IOU	DSC	HD	ASD
Experiment 1	None	0.8737	0.9266	5.1895	1.3911
Experiment 2	Generated MR assist	0.9006 $(3\%\uparrow )$	0.9437 $(2\%\uparrow )$	4.7571 $(1\%\downarrow )$	1.3067 $(6\%\downarrow )$
Experiment 3	Network join ASPP	0.9021 $(3\%\uparrow )$	0.9447 $(2\%\uparrow )$	4.7493 $(8\%\downarrow )$	1.2452 $(10\%\downarrow )$
Experiment 4	MR assist + ASPP	0.9043 $(3.5\mathbf{\%}\uparrow )$	0.9473 $(2.2\mathbf{\%}\uparrow )$	4.7204 $(9\mathbf{\%}\downarrow )$	0.9397 $(32\mathbf{\%}\downarrow )$

Table 3 demonstrates that the multi-modal image fusion significantly enhances the network’s performance, and the multi-scale semantic module also contributes to better segmentation results. When the ASPP and multi-modal image fusion method are applied, the IOU and DSC increase by 3.5% and 2.2%, respectively, compared to their initial values. Moreover, HD and ASD improve by 9% and 32%, respectively.

Having confirmed the effectiveness of each component of GSN, we conducted experiments to compare it with other mainstream segmentation networks. The results are presented in Table 4, which reveals that the proposed GMDSegNet method outperforms other methods in terms of segmentation accuracy. We also use the segmentation result metrics of FCN as a baseline, from which we calculated the impact of different networks’ results.

Table 4.

Comparison of segmentation performance between GMDSegNet and existing networks

Network model	IOU	DSC	HD	ASD
FCN	0.8360	0.8998	5.5573	2.3269
U-Net	0.8363 $(4\%\uparrow )$	0.8975 $(0.2\%\downarrow )$	5.6004 $(0.7\%\uparrow )$	3.0246 $(30\%\uparrow )$
Attention U-Net	0.8705 $(4\%\uparrow )$	0.9245 $(3\%\uparrow )$	5.3227 $(4\%\downarrow )$	1.9494 $(16\%\downarrow )$
DeeplabV3 +	0.8755 $(5\%\uparrow )$	0.9276 $(3\%\uparrow )$	5.1241 $(7\%\downarrow )$	2.3652 $(2\%\uparrow )$
PSPNet	0.8969 $(7\%\uparrow )$	0.9414 $(4.6\%\uparrow )$	4.9282 $(11\%\downarrow )$	1.8620 $(20\%\downarrow )$
GMDSegNet	0.9043 $(8\mathbf{\%}\uparrow )$	0.9473 $(5\mathbf{\%}\uparrow )$	4.7204 $(15\mathbf{\%}\downarrow )$	0.9397 $(60\mathbf{\%}\downarrow )$

In Fig. 6, we present the segmentation diagrams of a part of the network model and our proposed model. Based on the segmentation results in the figure, our GSN model demonstrates high consistency with the ground truth marked by medical experts and a lower false segmentation rate compared to other networks.

Fig. 6.

Segmentation results of GMDSegNet and existing networks; the green line is ground truth, the blue line is network output

We invited several radiologists from Jiangsu Cancer Hospital to assess the quality of the generated images and the accuracy of CTV segmentation. The evaluation conducted by the doctors affirmed the significance of MR image generation research. Furthermore, the segmentation results achieved a clinically applicable level of accuracy, thereby serving as a valuable reference for doctors during the outlining process and contributing to enhanced efficiency.

Discussion

Radiotherapy plays a vital role in the treatment of cervical cancer. However, the manual delineation of target areas by radiologists is time-consuming, laborious, and can lead to delays in patient treatment, affecting the therapeutic outcome and prognosis. Therefore, developing fast and effective computer-aided tools to automatically delineate CTVs can greatly reduce the workload of doctors and accelerate the clinical diagnosis and treatment process. There are two major difficulties in realizing accurate automated delineation of CTV. Firstly, potential tumor spread tissues will lead to unclear edge interface and irregular shape in CTV. Second, the information provided by the image of a single modality is limited. In this study, the image fusion of two modalities is used, which can delineate the target area more accurately than the image of a single modality. To improve segmentation accuracy, a flow field-based method is used to decouple the main body and boundary features, while increasing the boundary loss weight to enhance the accuracy of segmentation results. The experimental results of CTV delineation show that using the proposed GSN method can yield better results exceeding other mainstream networks currently available, and achieve more accurate target segmentation.

On the other hand, this study does not use real multi-modal images, because the acquisition of multi-modal images is not easy in most cases. For instance, CT has a small amount of ionizing radiation and should not be irradiated multiple times in a short period of time. MR has stricter examination taboos. For example, if a patient has implanted ferromagnetic substances in their body or is in an early pregnancy state, MR examination cannot be performed. We try to use novel image generation technology to make up for the limitations of image acquisition, which can play a certain help. The research on multimodal image generation can not only assist in other image processing tasks such as segmentation and detection, but also provide image reference for patients due to limitations such as heart disease or wearing a pacemaker.

It is worth mentioning that there are still potential future directions for efforts in our study. For instance, our study was designed to assist radiation therapy for cervical cancer. However, radiation therapy for rectal cancer as well as whole brain and whole spinal cord is also an area of interest in clinical medicine. An attempt can be made to construct a more complete CTV delineation system that can be adapted to a wide range of conditions. More multi-modal image assisted segmentation and more refined image generation also can be explored in the future. In addition to the one-to-one conversion, one-to-many and many-to-one conversions can be attempted. For example, multiple sequences of MR are obtained by CT image conversion, or serve to complement each other when multiple modalities are converted to one modality.

Conclusions

High-precision CTV delineation is a prerequisite and key element for the successful implementation of precision radiotherapy. A fast and effective method for automatic delineation of the target area can reduce the workload of manual delineation and provide a reference for physicians with varying levels of experience.

In this paper, we propose an AGSC-GAN model to generate realistic MR images. This model uses a single cycle to transform CT images into MR images, which improves the efficiency of model training without compromising the quality of image generation. Furthermore, we introduce an attention mechanism into the feature extraction module to facilitate attention learning from two dimensions: space and channel. Radiologists from Jiangsu Cancer Hospital have evaluated the medical images generated by our method and confirmed the value of our study on multi-modal medical image generation.

To aid in the delineation of cervical cancer targets, we propose a GSN segmentation model based on image generation. We use a fully convolutional network structure as the basic framework and integrate multi-scale features using the ASPP void space pyramid module in the GSN network structure. The cervical cancer CTV delineation method proposed in this study might help physicians make radiotherapy plans efficiently and accurately, reduce the damage to patients’ normal tissues, and improve patients’ quality of life.

Author Contribution

Ying Sun, Kexin Gan, and Yuening Wang work together on data processing and manuscript writing. Yuxin Wang and Ying Chen collect data from hospital and instruct Ying Sun, Yuening Wang, and Kexin Gan on data processing. Hanzi Xu and Yun Ge provide medical instruction and consultancy. Jie Yuan runs the whole project.

Data Availability

The image data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.

Declarations

Ethics Approval

All the experimental protocols mentioned in our manuscript have been reviewed and approved by the Ethics Committee of Jiangsu Cancer Hospital. The review elements include avoiding subjects’ exposure to unnecessary risks and researchers’ meeting research requirements. To protect patient privacy, all collected data were desensitized to remove personally identifiable information. The review process of the Ethics Committee follows ICH-GCP/China GCP and relevant Chinese laws and regulations. The relevant clinical norms and medical ethics have been strictly followed to ensure that patients’ privacy was not violated, or disclosed, and patients’ rights and interests were not harmed.

Consent to Participate

All data are collected from anonymous patients.

Consent for Publication

Not applicable.

Competing Interests

The authors declare no competing interests.