A Neural Network for Segmenting Tumours in Ultrasound Rectal Images

Abstract

Ultrasound imaging is the most cost-effective approach for the early detection of rectal cancer, which is a high-risk cancer. Our goal was to design an effective method that can accurately identify and segment rectal tumours in ultrasound images, thereby facilitating rectal cancer diagnoses for physicians. This would allow physicians to devote more time to determining whether the tumour is benign or malignant and whether it has metastasized rather than merely confirming its presence. Data originated from the Sichuan Province Cancer Hospital. The test, training, and validation sets were composed of 53 patients with 173 images, 195 patients with 1247 images, and 20 patients with 87 images, respectively. We created a deep learning network architecture consisting of encoders and decoders. To enhance global information capture, we substituted traditional convolutional decoders with global attention decoders and incorporated effective channel information fusion for multiscale information integration. The Dice coefficient (DSC) of the proposed model was 75.49%, which was 4.03% greater than that of the benchmark model, and the Hausdorff distance 95(HD95) was 24.75, which was 8.43 lower than that of the benchmark model. The paired t-test statistically confirmed the significance of the difference between our model and the benchmark model, with a p-value less than 0.05. The proposed method effectively identifies and segments rectal tumours of diverse shapes. Furthermore, it distinguishes between normal rectal images and those containing tumours. Therefore, after consultation with physicians, we believe that our method can effectively assist physicians in diagnosing rectal tumours via ultrasound.

Introduction

Rectal cancer ranks as the third most prevalent cancer globally, accounting for approximately 10% of all cancer cases [1]. Most patients with rectal cancer do not manifest overt symptoms until the disease reaches a terminal stage, when treatment options narrow. Consequently, rectal cancer screening is vital for effective treatment.

With the advances in artificial intelligence, various computer-aided diagnostic methods have been developed. These methods aim to automatically detect tumours [2] or organs [3], thereby assisting physicians in reducing their workload and improving the efficiency and reliability of diagnosis.

Among the numerous techniques available, the U-Net [4] architecture, which is based on the convolutional neural network(CNN) [5] and includes an encoder-decoder structure resembling a ‘U’ shape, is particularly notable as a highly effective approach for image segmentation tasks. Researchers have actively refined the U-Net architecture by incorporating cutting-edge techniques such as grid-like skip connections [6] and dilated convolution [7]. Furthermore, inspired by its remarkable success in natural language processing, some researchers have integrated the attention mechanism [8], also known as the transformer architecture, into image processing. For example, ViT [9] extracts global features on the basis of patches consisting of 16 × 16 pixels, the Swin Transformer [10] reduces computational overhead by shifting windows to fuse information from multiple subwindows, SKNet [11] utilizes a global pool of feature maps to derive the corresponding weights for each pixel, and DAEformer [12] applies attention mechanism to feature channels.

Furthermore, researchers have developed algorithms designed to cater specifically to various tasks. For example, TransUnet [13] seamlessly integrates transformer blocks into the U-Net encoder, enhancing its capabilities for computed tomography (CT) image processing. SwinUnet [14], on the other hand, substituted convolutional blocks with the Swin Transformer, optimizing its performance for magnetic resonance imaging(MRI). Gu et al. [15] employed dilated convolution to segment the optic disc. In the context of rectal tumours, Li et al. [16] proposed an enhanced version of U-Net specifically for MRI rectal tumour segmentation. Similarly, Yu et al. [17] incorporated feature fusion techniques into U-Net to improve CT rectal tumour segmentation. Rao et al. [18] utilized channel concatenation to further refine the segmentation process.

Ultrasound imaging remains the most commonly utilized method for routine cancer screening due to its ease of use [19]. Chen et al. [20] employed a U-Net with adaptive attention to segment tumours in ultrasound breast images, whereas Hui et al. [21] introduced a boundary point supervision module specifically for intracranial aneurysm segmentation. For ultrasound rectal tumour images, Yi et al. [22] proposed an image-level classification method. Nevertheless, the majority of existing models are designed for application to medical images with minimal noise, such as CT and MRI, or concentrate on objects with consistent shapes or sizes, such as abdominal organs, and those with golden diagnostic standards, such as breast tumours. Unfortunately, few studies have been dedicated to segmenting rectal tumour ultrasound images.

Rectal tumours pose a greater challenge in localization compared to organs and tissues [23] and tumours with golden standards, such as breast tumours [20]. The variable size and complex shape of rectal tumours make segmentation challenging, and the difficulty of segmentation is further exacerbated by the limited field of view of ultrasound scans and with the inherent strong noise of ultrasound images. Existing methods focus primarily on enhancing encoding or bottleneck layers by substituting or refining convolutional blocks with attention mechanisms. However, these techniques necessitate the use of a larger volume of training data for the model to extract sufficient features, thereby increasing the burden on physicians. Furthermore, incorporation of attention mechanisms can compromise the model’s capacity to capture local features and potentially introduce spurious noise. Conversely, the limited receptive field of convolutional blocks excels at segmenting targets with simple shapes but encounters difficulties with more intricate shapes. Consequently, there is a pressing need for an efficient, automated ultrasound image segmentation method that can aid physicians in detecting tumours in ultrasound rectal images with greater accuracy.

This paper proposes a novel method for segmenting ultrasound images containing rectal tumours. The proposed method still employs a U-shaped architecture. Considering the strong noise characteristics of ultrasound images, we retained the convolutional layers in the encoder. Since lesion areas are sparse compared to normal areas, using convolution with a limited receptive field helps to avoid extracting features from larger regions, thereby reducing the impact of ultrasound image noise. Moreover, unlike the encoder including a transformer, our approach requires less training data for feature extraction. Although the attention mechanism’s global receptive field may capture irrelevant information or noise, it is effective in handling complex-shaped lesion areas. Therefore, we introduce a global attention mechanism into the decoders. The effectiveness of the attention mechanism is closely related to the size of the feature map. A higher-resolution feature map provides richer information to the attention mechanism, necessitating a cascaded approach with multiple transformer blocks. Furthermore, we utilized a channel fusion module to integrate the outputs of the various decoders. This reduced the noise captured by the global attention mechanism and enhanced the segmentation accuracy. Experiments indicated the proposed method has good performance, while also showing considerable differences from existing models in the task of ultrasound rectal tumour segmentation. Our research primarily examines the feasibility of leveraging deep learning techniques in the relatively unexplored domain of ultrasound rectal tumour image segmentation. We introduce a segmentation method that outperforms existing approaches for the task of segmenting ultrasound rectal tumours. In summary, deep learning-based image segmentation has considerable value in the specific context of ultrasound rectal tumour segmentation, offering assistance for physicians in lesion area confirmation, and is a promising avenue for future development.

Materials and Methods

Rectal Ultrasound Dataset

The Bioethics Committee of the University of Electronic Science and Technology of China granted approval for this study (reference No. 106142023121227787). The Sichuan Province Cancer Hospital kindly provided the research data, comprising records of 268 patients and a total of 1507 images. The imaging equipment utilized was the ESAOTE MYLAB™ Twice HD, equipped with TRT33 probes operating at a frequency range of 3 ~ 13 MHz. Notably, the dataset includes images captured at different points in time for the same patient. Below, this dataset will be referred to as ‘RTUI’. Notably, RTUI excludes any normal rectal ultrasound images.

Two physicians, one with over 6 years of clinical expertise and the other with more than 2 years of experience, independently annotated the actual rectal tumour region using the LabelMe software [24]. The ground truth labels were consolidated by synthesizing both annotations, ensuring utmost accuracy.

In the subsequent experiments, the data were randomly divided into test and training sets at a ratio of 2:8 based on the number of patients. Additionally, 10% of the training set was designated as the validation set. Consequently, the test set is comprised by 53 patients with 173 images, the training set is comprised by 195 patients with 1247 images, and the validation set is comprised by 20 patients with 87 images.

Additionally, we added an external validation set to improve model validation. The external validation set included 30 normal rectal ultrasound images and 90 images with rectal tumours for a total of 36 patients.

Data Augmentation

Similar to other researchers, we faced the problem of insufficient data in our experiments [25]. Therefore, we selected seven methods to augment the training dataset, including 180° rotation, horizontal and vertical flips (abbreviated as Flip-H and Flip-V, respectively), contrast enhancement, Gaussian blur, histogram equalization, and clipping 10% of the total images. The Albumentations [26] package was used for data augmentation.

Network Model

Figure 1 illustrates our network architecture. Once the input image is fed into the network, the encoders progressively downsample the image. By the end of the encoding process, the resolution of the input image is reduced to 1/32 of its original size.

Fig. 1.

Our network architecture

The encoded feature map was subsequently fed into the global attention decoders via the bottleneck. The decoders gradually rebuild the feature maps into the original resolution by upsampling. Finally, the predicted label was obtained by combining the outputs of all decoders using the efficient channel information fusion module. We wanted to use the global attention mechanism and multiscale information fusion in our model to solve the problems of strong noise in ultrasound images and changes in the rectal tumour shape.

The PyTorch2.0 framework was used to build the overall network model. The batch size was 24, and AdamW [27] was used as the optimizer, with an initial learning rate of 5.0 × 10⁻⁵ and a weight decay of 1.0 × 10⁻⁵. The overall model was trained for 240 epochs on a single NIVIDA GeForce RTX 3090. Prior to training, we standardized the input image by adjusting its pixel values to have a mean of 0 and a variance of 1. All of the images were subsequently resized to a uniform size of 224 × 224 with the pixel values normalized to the range between 0 and 1.

Global Attention Decoder

Figure 2a shows a single global attention decoder (GAD). The decoding process was carried out using both the global information entered and the information from the encoder at the same level.

Fig. 2.

(a) GAD structure. (b) Upsampling process. (c) The ECIF module

As shown in Fig. 2b, the input was first upsampled through the linear layer to twice the input resolution. This process can be defined by Eq. 1:

$$X_2\left(c_2,,,h_2,,,w_2\right)=up\left[X_1,\left(c_1,,,h_1,,,w_1\right)\right]=TROp\left\{M,L,P,\left[\left[X_1,\left(c_1,,,h_1,,,w_1\right)\right]\right]\right\}$$ 1

where X is the feature map, MLP is the multilayer perceptron, and c, h, and w are the channel, height, and width of the feature map, respectively, and TROp is the tensor reshaping operation. The tensor X₁ is mapped to match the number of X₂ (equal to c₂ × h₂ × w₂) using an MLP, and subsequently, the shape of mapped X₁ is aligned with X₂ through TROp. In Eq. 1 and Fig. 2b, h2 = 2h₁, w₂ = 2w₁, and c₂ = 4c₁.

After upsampling, we finished the weighted sum of the feature map in the resolution dimension, i.e. h axis and w axis. The calculation method is the attention mechanism, which outputs the importance score of each pixel or patch through matrix multiplication and the dot product. This mechanism helps to focus on the most valuable part of the current input for the output result. To reduce the computational cost, we use the Swin Transformer [10]. Then, the decoded result was concatenated with the output from the sibling encoder. This concatenated result is then fed into a convolutional layer using 1 × 1 kernels to reconstruct the high-resolution feature map.

Equation 2 can be used to define the entire decoding process:

$$D\left(X_1,,,X_2\right)=Conv\left\{S,w,i,n,\left[u,p,\left(X_1\right)\right]\right\}\oplus X_2$$ 2

where X₁ is the input image, X₂ is the feature map from the encoder corresponding to the decoder, Conv is a convolution layer with a 1 × 1 kernel, Swin is the Swin-transformer layer, and (x ⊕ y) represents the concatenation of x and y on the channel dimension.

Efficient Channel Information Fusion

Figure 2c shows the efficient channel information fusion process (ECIF). The outputs of different decoders were concatenated and then remapped to effectively merge multiscale features and improve the model’s segmentation accuracy.

First, we reshaped each decoder’s outputs from 3D (c × h × w) to 2D (c × L) tensors with the same length L and different channels. Each output is then concatenated into a tensor X_concat. This tensor is subjected to a channel information weighting operation, using the same calculation method as in the ‘Global Attention Decoder’ section, but with this operation applied to the channel dimension rather than to the resolution dimension. We used the calculation method suggested in [28], which can be defined as:

$$E_c\left(Q,,,K,,,V\right)=V\oplus softmax\left(K^T,\oplus ,Q,/,\tau \right)$$ 3

where Q, K, and V are obtained by using MLP to map the input, which is converted to 2D beforehand. τ is a constant number.

The channel self-attention operation combines information from multiple decoders and converts it into a single feature map. Finally, a MLP converts this map into a segmentation mask.

Equation 4 can be used to define the entire process:

$$out=MLP\left\{E,c,\left[D_1,\oplus ,D_2,\oplus ,D_3,\oplus ,D_4\right]\right\}$$ 4

where D_i represents the reshaped output of the i-th decoder, whose dimensions are c_i × L.

Loss function

We chose the cross-entropy loss function and the Dice loss function as our loss functions in this study. These loss functions were of a similar order of magnitude on our task; thus, the final loss is 0.5×Loss_{cross-entropy} + 0.5×Loss_Dice.

Metrics

Dice similarity Coefficient

The Dice similarity coefficient (DSC) is a statistical metric that compares the similarity of two sets commonly used in image segmentation. It can be expressed as:

$$Dice=\frac{2\left|X,\cap ,Y\right|}{\left|X\right|+\left|Y\right|}$$ 5

where |X ∩ Y| denotes the intersection between X and Y and |X| and |Y| are the numbers of elements in X and Y, respectively. To normalize the result into the range of 0 to 1, we multiply the numerator by two. The resulting range is from 0 (worst) to 1 (best).

DSC is commonly used to calculate the degree of overlap between the ground truth and predicted regions.

Hausdorff Distance 95

The Hausdorff distance 95 (HD95) is used in image segmentation tasks to measure the similarity between the boundaries of the predicted and ground truth segmentations. The ‘95’ indicates that the HD95 excludes 5% of the data corresponding to the largest outliers from the computation. HD95 has a value between 0 and infinity, with a value closer to 0 indicating greater similarity between the boundaries of the two regions.

The HD can be defined as:

$$HD=max\left\{{\mathit{max}}_{x\in X},{\mathit{min}}_{y\in Y},d,\left(x,,,y\right),,,{\mathit{max}}_{y\in Y},{\mathit{min}}_{x\in X},d,\left(x,,,y\right)\right\}$$ 6

where x and y are the pixels along the boundaries of images X and Y, respectively, and d(x, y) is the Euclidean distance between corresponding points on these boundaries.

Mean Intersection over Union

The mean intersection over union (MIoU) is a common metric for representing the intersection and union of the actual and predicted values. The MIoU was calculated via Eq. 7:

$$MIoU=\frac{1}{k+1}\sum _{i=0}^k\frac{\mathit{TP}}{\left(F,P,+,F,N,+,T,P\right)}$$ 7

where TP, FP, and FN are true positive, false-positive, and false-negative, respectively. k is equal to 1 in a binary classification problem.

Mean Pixel Accuracy

The mean pixel accuracy (MPA) was calculated by taking the average number of correctly classified pixels in each category divided by the total number of pixels in that category. The MPA was calculated via Eq. 8:

$$MPA=\left(\sum _{i=0}^N,P_i\right)/N,P=TP/\left(T,P,+,F,P\right)$$ 8

where N is the number of categories including the background, P_i is the precision of the i-th category, and TP and FP are the same as those in the ‘Mean Intersection over Union’ section.

Significance Analysis Method

In the subsequent experiments, we employed the paired t test method [29] to assess whether significant differences existed between paired models, with a p value set at 0.05 and the null hypothesis stating no significant differences between the tested objects. Additionally, we employed the Bonferroni correction [30] to address the issue of multiple testing and reduce the risk of type I error. This correction adjusts the p value to p/k, where k is the number of pairwise comparisons The Bonferroni correction was only applied in the ‘Ablation Experiment’ section.

Furthermore, we calculated Cohen’s d value [31] to quantitatively evaluate the magnitude of the differences between the objects under comparison. We established benchmarks for small (≥ 0.2), medium (≥ 0.5), and large (≥ 0.8) effect sizes. A Cohen’s d value exceeding 0.2 indicates a meaningful difference between the compared objects.

Results

Segmentation Results

Figure 3 shows the segmentation result, including the mask and the original image with the contour. The experimental results showed that our network performed well in segmenting ultrasound images of rectal tumours from various patients, with DSC, HD95, MIoU, and MPA values reaching 75.49%, 24.75, 71.46%, and 87.41%, respectively, when not loading the pretrained weight. Visualization showed that the prediction was also close to the ground truth.

Fig. 3.

Segmentation results from four different patients with rectal tumours

However, our model also has several shortcomings, such as the inability to identify tumour areas with drastic changes in edges and missegmentation of lesions in small areas, particularly those that occupy less than 5% of the total image area.

Fivefold Cross-Validation

Typically, cross-validation test the model’s sensitivity to the training data and judge whether the model is overfitting. We used Kaiming initialization [32] and the same hyperparameters in each fold. Following the method in the ‘Rectal Ultrasound Dataset’ section, the RTUI was partitioned into five sections. Each section alternately served as the test set, while the remaining four were divided into validation and training sets (1:9 ratio). This process was repeated five times for comprehensive evaluation. The model in each fold uses the same hyperparameters and Kaiming initialization [32].

Table 1 presents the five cross-fold validation results. The best and worst results for various metrics did not differ obviously. Therefore, our model was reliable.

Table 1.

Experimental results of five cross-fold validation on the test set

	DSC (%)	HD95	MIoU (%)	MPA (%)
Fold 1	74.56	26.64	70.63	86.76
Fold 2	75.48	27.44	71.66	87.34
Fold 3	74.17	25.6	70.69	88.38
Fold 4	74.41	25.49	70.95	86.85
Fold 5	75.49	24.75	71.46	87.41
Mean ± STD	74.82 ± 0.56	25.98 ± 0.94	71.08 ± 0.41	87.35 ± 0.58

Ablation Experiment

To test the effectiveness of the components proposed in the ‘Data Augmentation’ section, we gradually removed them from our model. When the base model was compared with our final model, the Bonferroni correction was used to adjust the p value. Since $=C_4^2=6$ , the adjusted p value was 0.0083. All of the models used the same hyperparameters and Kaiming initialization. The base model (Our U-Net) used the original UNet’s decoders.

Table 2 shows the results of the ablation experiments, where the difference in the results was determined via paired Student’s t tests. Cohen’s d values were derived from the base model and the comparison model. Compared with the base model, the combination of both modules resulted in improvements by 4.2%, 10.08, 4.24%, and 3.78% in the DSC, HD95, and MIoU metrics, respectively.

Table 2.

Results of the ablation experiment (mean ± STD). The symbol ‘*’ indicates significant differences between the base model and the comparison models (p-value ≤ 0.05), and the values in parentheses are the corresponding Cohen’s d values between the base model and the compared models

	DSC (%)	HD95	MIoU (%)	MPA (%)
Our U-Net (base)	71.29 ± 22.59	34.83 ± 29.24	67.22 ± 15.32	83.63 ± 13.98
Our U-Net + GAD	70.24 ± 24.76	30.58 ± 24.78*(0.189)	68.18 ± 14.86	83.43 ± 13.63
Our U-Net + ECIF	73.51 ± 21.49*(0.158)	27.22 ± 23.77*(0.364)	70.31 ± 15.29*(0.301)	84.78 ± 12.24
Our model	75.49 ± 21.74*(0.27)	24.75 ± 23.69*(0.395)	71.46 ± 14.37*(0.36)	87.41 ± 11.95*(0.4)

According to Table 2, when only one module was added, among the groups that showed significant differences, only the ‘Our U-Net + ECIF’ group achieved a small effect size on the HD95 and MIoU metrics. However, the combination of the two modules led to a significant difference, with a significance level of p < 0.0083 for all metrics. The Cohen’s d values for all metrics exceeded the threshold for a small effect size and were larger than those of the groups with only one module added, further suggesting that the combination strategy we adopted was highly effective.

Therefore, the experimental results support our hypothesis that global information extraction and multiscale information fusion can aid in accurately segmenting rectal tumours in ultrasound images.

Comparison with the Other Models

Table 3 compares our model with reference models; all of the models use the same hyperparameters and Kaiming initialization. Additionally, a paired t test was conducted to assess the difference between each reference model and our proposed model.

Table 3.

Comparison between our model and reference models (mean ± STD). The bolded results are the best results obtained on the test set of the RTUI. The symbol ‘*’ indicates that there is a significant difference between our model and the compared model under the paired t test on the corresponding metric (p-value ≤ 0.05), and the values in parentheses refer to Cohen’s d values between our model and the compared models

	DSC (%)	HD95	MIoU (%)	MPA (%)	Time/ms
SegNet [33]	72.46 ± 21.6*(0.218)	29.79 ± 23.21*(0.201)	68.07 ± 13.36*(0.326)	86.54 ± 12.22	7.525
U-Net [4]	71.46 ± 19.95*(0.334)	33.18 ± 24.74*(0.346)	66.73 ± 12.93*(0.479)	86.0 ± 11.21*(0.249)	4.538
U-Net +  + [6]	73.5 ± 19.78*(0.208)	33.25 ± 25.31*(0.336)	68.55 ± 13.68*(0.306)	85.94 ± 11.71*(0.229)	11.891
AAU-Net [20]	71.7 ± 24.79*(0.267)	31.78 ± 26.74*(0.233)	68.22 ± 15.46*(0.26)	84.86 ± 13.86*(0.256)	29.531
CeNet [15]	70.38 ± 22.21*(0.329)	33.79 ± 26.09*(0.346)	66.6 ± 13.26*(0.44)	83.95 ± 13.09*(0.388)	8.860
SwinUnet [14]	67.15 ± 20.53*(0.507)	34.55 ± 26.61*(0.38)	62.97 ± 13.42*(0.7)	83.07 ± 12.94*(0.441)	17.297
DAEformer [12]	69.75 ± 24.22*(0.359)	31.32 ± 24.33*(0.267)	67.08 ± 14.14*(0.398)	83.17 ± 13.34*(0.5)	27.633
TransUnet [13]	72.04 ± 23.12*(0.224)	28.54 ± 23.73	67.99 ± 14.64*(0.289)	86.14 ± 12.57	26.645
Deeplabv3 + [7]	70.57 ± 23.08*(0.288)	29.94 ± 23.22*(0.2)	67.91 ± 15.22*(0.282)	84.41 ± 13.05*(0.367)	8.402
Our model	75.49 ± 21.26	24.75 ± 23.59	71.46 ± 14.42	87.41 ± 11.69	26.715

As demonstrated in Table 3, our model outperformed all of the other models across all of the evaluated metrics. For the DSC, HD95, MIoU, and MPA metrics, our model achieved improvements of 1.99%, 3.79, 2.91%, and 1.27%, respectively, compared to the respective second-best models. However, when inferring from a single image, our model is 22.177 ms longer than the fastest model (U-Net), but this is a common issue with almost all models that use attention mechanism.

Figure 4 shows the ROC and PR curves, which indicate a classifier’s ability to identify samples at a given threshold. When category imbalance is disregarded, as shown by the AUC curve, there is little difference between the models. Nevertheless, in real-world scenarios where category imbalance is prevalent, our model demonstrates superior performance, as is evident from the PR curve.

Fig. 4.

ROC and PR curves in RTUI

Figure 5 presents a comparative visualization of the real and model-predicted results. Our model demonstrated superior accuracy, particularly in the 6th row of Fig. 5. When dealing with irregular and complex areas, all of the models except for our model exhibited clear segmentation errors, such as oversegmentation or undersegmentation, as can be observed in the 1st, 4th, and 5th rows of Fig. 5. Consequently, our model was able to accurately segment rectal tumours of diverse sizes and shapes, effectively detecting lesion areas from ultrasound rectal images.

Fig. 5.

Visualization of the segmentation results of the different models. From left to right are images of the input, ground truth mask, SegNet, U-Net, U-Net +  + , AAUnet, CeNet, SwinUnet, DAEformer, TransUnet, Deeplabv3 + , and ours

External Validation

We introduced an additional external validation set, as described in the ‘Rectal Ultrasound Dataset’ section, to better demonstrate the model's performance compared with existing models.

Table 4 displays the external validation results of the models, whereas Fig. 6 provides a visualization of the results. Introducing normal ultrasound rectal images resulted in improvements in some models. As presented in Table 4, our model is still the best in every metric. The last two rows of the visualization results in Fig. 6 show that our model could identify normal rectal images well.

Table 4.

Comparison between our model and reference models (mean ± STD). The bolded results were the best results obtained on the external validation. The symbol ‘*’ indicates that there is a significant difference between our model and the compared model under the paired t test on the corresponding metric (p-value ≤ 0.05), and the values in parentheses refer to Cohen’s d values between our model and the compared models

Models	DSC (%)	HD95	MIoU (%)	MPA (%)
SegNet	80.49 ± 19.86	28.87 ± 29.13	75.4 ± 18.44	89.8 ± 10.47
U-Net	77.6 ± 19.78	30.21 ± 25.77	72.47 ± 19.74*	89.58 ± 10.36
Unet +  +	78.43 ± 19.05	28.55 ± 25.62	73.51 ± 19.01	89.03 ± 10.64
AAU-Net	75.69 ± 23.92*(0.269)	36.04 ± 29.29	72.13 ± 21.05*(0.293)	86.64 ± 12.92*(0.341)
CeNet	74.25 ± 25.28*(0.301)	32.72 ± 28.34	72.12 ± 20.32*(0.321)	87.38 ± 12.25*(0.307)
Swin	75.44 ± 21.71*(0.339)	37.16 ± 33.63*(0.331)	71.51 ± 20.06*(0.357)	86.59 ± 11.93*(0.45)
DAE	76.93 ± 21.51*(0.233)	31.72 ± 28.99	73.07 ± 18.96*(0.261)	87.97 ± 10.18*(0.25)
TransUnet	78.25 ± 21.69	35.83 ± 32.25	73.81 ± 19.54	88.41 ± 11.1
Deeplabv3 +	74.36 ± 23.11*(0.453)	38.54 ± 37.21*(0.271)	70.55 ± 21.1*(0.54)	85.71 ± 12.82*(0.6)
Our model	80.56 ± 17.45	28.4 ± 26.65	76.01 ± 20.05	89.59 ± 9.61

Fig. 6.

Visualization of the segmentation results of the different models on the external dataset. From left to right are images of the input, ground truth mask, SegNet, U-Net, U-Net +  + , AAUnet, CeNet, SwinUnet, DAEformer, TransUnet, Deeplabv3 + , and ours

However, compared to the RTUI test set, the advantages of our model are no longer clear, particularly for the MPA metrics.

Effect of GAD Module

To further demonstrate the effectiveness of our GAD module, we also replaced the GAD with the module named ‘StackConv’, which achieved the equivalent global receptive field with stacked convolutional layers. It is obvious that GAD is better than that of the method of expanding the receptive field by deepening the network. According to Table 5, the difference between them is statistically significant, which is manifested in the p-value of several metric less than 0.05 and the Cohen’s d value exceeds the small effect size. Physicians believed that some encoders effectively concentrate on edges of rectal tumours, suppressing nonessential regions while exhibiting resilience towards areas of minimal variation.

Table 5.

Comparison between our model and the method of extending network depth (mean ± STD). The symbol ‘*’ indicates that there is a significant difference between our model and the compared model under the paired t test on the corresponding metric (p-value ≤ 0.05), and the values in parentheses refer to Cohen’s d values

Models	DSC (%)	HD95	MIoU (%)	MPA (%)
Test set of RTUI
Our model	75.84 ± 21.07	24.71 ± 22.76	71.56 ± 14.36	87.46 ± 11.62
StackConv	70.34 ± 24.66*(0.382)	30.11 ± 22.82*(0.266)	66.54 ± 14.02*(0.455)	85.49 ± 13.68*(0.239)
External validation
Our model	80.49 ± 19.86	28.87 ± 29.13	76.01 ± 20.05	89.8 ± 10.47
StackConv	75.61 ± 23.04*(0.441)	32.36 ± 27.62	70.59 ± 20.56*(0.558)	87.85 ± 11.53*(0.316)

To visualize the intermediate stages of our model, we produced a class activation map, effectively illustrating the model’s operational process through an intuitive heatmap. As the area becomes redder, its importance to the segmentation result increases. Figure 7 depicts the segmentation process of our models, encompassing both the GAD and StackConv. Notably, Fig. 7 reveals that during the decoding process, the GAD progressively encircles the lesion area from the periphery, whereas the Conv expands towards it from the centre. Therefore, for rectal tumours with complex shapes, Conv was prone to incomplete segmentation, as shown in the 3rd and 4th rows of Fig. 7. Furthermore, a single transformer block cannot also accurately locate the lesion area, but by cascading transformers, the lesion area is localized with increasing accuracy. This is because the GAD receptive field more accurately covers the lesion area as the resolution of its feature map increases.

Fig. 7.

Class activation maps of our model whose decoder is composed of GAD or StackConv

As depicted in Fig. 7, the ECIF module effectively enhances the detection performance. Since GAD enlarges the receptive field, it carries the potential risk of introducing noise. The ECIF module mitigates the noise effects by integrating outputs from decoders with diverse resolutions.

Discussion

Our proposed algorithm outperforms other algorithms because of the reasonable introduction of the attention mechanism. The results of the fivefold cross-validation in Table 1 verified that our model did not exhibit large fluctuations due to dataset division, indicating its reliability and that the model was not overfit. Moreover, the Cohen’s d values in Table 2 further demonstrate that our improvement strategy is not affected by the test sample size.

According to Table 3, for RTUI, our model demonstrated significant differences in terms of most metrics compared with the other models, with significance levels of p-value < 0.05. Figure 5 illustrates that our model can effectively handle rectal tumours of complex shapes. As outlined in the ‘Effect of the GAD Module’ section, by introducing the attention mechanism into the decoder, our model decoded the feature map from the outside to the inside, enabling better identification of the rectal tumour that usually extends peripherally in diffuse patterns within ultrasound images. Retaining a portion of the convolution module and incorporating a channel fusion module help reduce noise resulting from the enlarged receptive field while placing a greater emphasis on the lesion area. Furthermore, our model also showed the best performance on the external validation set. With regard to the effect size, there is a notable gap between our model and the other models that cannot be ignored, which means that the difference between our model and the reference model is not only statistically significant but also objectively realistic.

Compared to the convolutional model, our designed decoder can better handle the variable shapes and sizes of rectal tumours, and the segmentation results are comparable to those obtained by clinicians. In the task of ultrasound image segmentation, where the shapes and sizes of the segmentation targets vary greatly, introducing an attention mechanism with a built-in global receptive field can be more effective than expanding the receptive field through stacking convolutional layers, which is why our model was better than U-Net and DeeplabV3 + . However, the global receptive field of the attention mechanism is achieved through global information weighting, which lacks the gradual expansion of the receptive field of the convolutional models. This is precisely why the performance of SwinUnet and DAEformer, which rely solely on stacked attention modules, falls short of that of UNet. Our decision to retain the convolutional encoder aims to bolster the model’s robustness against variations in shape and size while simultaneously preserving its sensitivity to intricate changes in target details. The more detailed the information about the original image the encoder acquires, the better the decoder is able to reconstruct the global image and identify the lesion area. This is the reason why our model outperforms TransUnet, which directly incorporates the attention mechanism into the encoder and lacks the ability to extract fine details. Although our model may not perform well in segmenting targets with subtle edge changes and very small sizes, such rectal tumours are relatively rare, so that this shortcoming does not significantly limit the applicability of the model. Additionally, the scarcity of training data has constrained the full utilization of the capabilities of the attention mechanism.

Conclusion

In this paper, we propose a model for efficiently identifying and segmenting tumour regions in rectal ultrasound images, which could help physicians make more accurate rectal cancer diagnoses. We designed a deep learning network composed of encoders, a bottleneck, and GAD, and we employed the ECIF to emphasize the lesion area. These enhancements enabled the network to segment rectal tumours with complex shapes in ultrasound images. The experimental outcomes on both the ultrasound rectal tumour test set and the external validation set revealed that our approach outperformed the other methods. In the future, our objective is to deploy the model for real-time segmentation tasks and present comprehensive details, including tumour benignancy or malignancy assessments, along with tumour subtype classification, thereby efficiently assisting physicians in their diagnostic workflow.

Author Contribution

Yuanxi Zhang: methodology, writing—original draft preparation; Xiwen Deng: data curation; Tingting Li: visualization, investigation; Yuan Li: software, validation; Lifeng Yang: funding acquisition, supervision; Man Lu: conceptualization, funding acquisition; Xiaohui Wang: funding acquisition, formal analysis.

Funding

This work was financially supported by the Medico-Engineering Cooperation Funds from the University of Electronic Science and Technology of China (No. ZYGX2021YGLH206) and the Natural Science Foundation of Sichuan Province (No. 2022NSFSC0870).

Data Availability

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Declarations

Ethics Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. The study was approved by the Bioethics Committee of the University of Electronic Science and Technology of China of No. 106142023121227787.

Competing Interests

The authors declare no competing interests.