A lung nodule segmentation model based on the transformer with multiple thresholds and coordinate attention

Abstract

Accurate lung nodule segmentation is fundamental for the early detection of lung cancer. With the rapid development of deep learning, lung nodule segmentation models based on the encoder-decoder structure have become the mainstream research approach. However, during the encoding process, most models have limitations in extracting edge and semantic information and in capturing long-range dependencies. To address these problems, we propose a new lung nodule segmentation model, abbreviated as MCAT-Net. In this model, we construct a multi-threshold feature separation module to capture edge and texture features from different levels and specified intensities of the input image. Secondly, we introduce the coordinate attention mechanism, which allows the model to better recognize and utilize spatial information when handling long-range dependencies, enabling the deep network to maintain its sensitivity to nodule positions. Thirdly, we use the transformer to fully capture the long-range dependencies, further enhancing the global information integration of the network. The proposed method was verified on the LIDC-IDRI and LNDb datasets. The Dice similarity coefficient (DSC) values achieved were 88.29% and 78.51%, and the sensitivities were 86.33% and 75.05%, respectively. The experimental results demonstrated its high practical value for the early diagnosis of lung cancer.

Introduction

Lung cancer accounts for approximately 1.8 million deaths annually worldwide, as reported by the World Health Organization (WHO)¹. The ability to improve patient survival rates critically hinges on the early detection and conducting of effective pathological analyses and diagnoses. The challenge of accurately segmenting lung nodules mainly stems from the diversity of nodule shapes and sizes and the visual similarity between nodules and the surrounding tissues². Pulmonary nodules can be divided into six categories according to their morphologies and locations³. Figure 1 shows the morphological characteristics of various types of nodules.

Fig. 1.

Different types of nodules.

Currently, lung nodule segmentation techniques can be classified into two main categories: traditional methods and deep learning-based methods⁴. Traditional methods for lung nodule segmentation rely on manually designed features such as contrast, texture, shape, and boundary information. These features are typically implemented through region-growing or level-set-based methods during segmentation. However, these approaches exhibit significant performance limitations when handling complex or diverse images, such as low-contrast, small nodules or lesions with ambiguous morphology. This is because manual feature extraction cannot adequately adapt to the rich and varied details present in the images.

In contrast, deep learning-based methods automatically extract multi-scale features through convolutional neural networks (CNNs), enabling models to simultaneously focus on local edge information and global contextual semantics. This significantly improves sensitivity to low-contrast and small lesions. The essential distinction between this automated learning approach and traditional feature fusion lies in the fact that feature extraction is no longer restricted by predefined rules but is instead optimized end-to-end, achieving efficient mapping from pixels to lesion regions.

As a result, whether in classification or segmentation tasks, deep learning methods, with their powerful feature learning capabilities, adaptability, and global perception, have surpassed traditional methods of feature fusion and selection, becoming the mainstream technical approach in the field of lung cancer imaging analysis⁵.

The mainstream segmentation methods in deep learning currently rely on encoder-decoder architectures, where the encoder is responsible for extracting features from the input image, which serve as the foundation for the decoder to reconstruct the target segmentation. If the features extracted by the encoder are of poor quality (for instance, if they fail to effectively capture crucial information in the image and struggle to capture long-range dependencies), the segmentation results will be affected, regardless of how powerful the decoder is. Therefore, the design of the encoder plays a decisive role in the overall performance of the network in segmentation tasks⁶. Currently, commonly used encoders include the VGG series⁷, ResNet series⁸, and U-Net series⁹. Among them, U-Net, with its efficient data flow and strong scalability, has emerged as the dominant architecture for lung nodule segmentation.

Researchers tend to process input images to obtain additional information to aid feature extraction by the encoder. For example, they may use the Fourier Transform to convert time-domain (or spatial-domain) signals into frequency-domain signals,¹⁰ thereby capturing frequency and amplitude information of the image, or employ the Wavelet Transform to acquire both time-domain and frequency-domain information by decomposing the signal into frequency bands at different scales¹¹, or directly use the Discrete Wavelet Transform to extract wavelet features from the image¹². Our designed multithreshold feature separation module (MFS) separates and extracts local-to-global edge and texture features within the spatial domain using different thresholds, preserving the spatial structural information of the image. This extraction process has linear computational complexity, making it lighter and more computationally efficient than the Fourier Transform. Compared to the Wavelet Transform, the MFS module uses a simple multi-threshold strategy for feature extraction, avoiding the complex hierarchical decomposition and recursive calculations in wavelet transforms, while enabling effective feature extraction directly in the spatial domain with minimal computational cost and high efficiency.

The introduction of attention mechanisms to enhance important features and improve localization accuracy is also critical for enhancing encoder performance. Common mechanisms include channel attention, spatial attention, and their combinations or variations. Channel attention strengthens important features by assigning weights to different channels, but it does not consider the position of each pixel. Its main task is to perform weighted selection along the channel dimension of the feature map, optimizing feature representation while ignoring spatial location. Spatial attention and combinations like convolutional block attention module (CBAM)¹³ handle spatial information by compressing the channel dimension to 1 in the spatial attention module, which can result in some positional information loss. The convolution operations in these modules typically capture only partial local spatial relationships.

To enhance the capture of local spatial relationships while retaining the benefits of channel attention, we introduce the coordinate attention mechanism(CA)¹⁴. This mechanism performs global pooling along the horizontal and vertical directions to generate positional encodings similar to those in Transformers,¹⁵ and propagates them through the deep network along with the extracted features. This approach combines the advantages of channel and spatial attention, strengthens spatial relationships within local regions, and has a positive impact on handling long-range dependencies in subsequent layers.

In deep learning, the long-range dependency problem refers to the model’s difficulty in capturing relationships between distant features or information within the input data. In lung nodule segmentation, this problem is particularly significant because lung nodules are often distributed in lung regions with significant spatial variability, having semantic or spatial relationships with distant tissues or structures. Multi-threshold feature separation enhances the capture of both global and local information, while coordinate attention preserves positional information and increases spatial sensitivity. Together, these mechanisms optimize the modeling and capture of long-range dependencies. The self-attention mechanism in Transformers, unrestricted by receptive field limitations¹⁶, selects the most relevant parts through Query, Key, and Value vectors,¹⁵ making it a powerful tool for capturing dependencies across long distances. However, due to the Transformer’s heavy computational demands and its reliance on large datasets to achieve optimal performance, we apply it only at the skip connection of the bottleneck layer, where high-level features better represent the global structure.

In summary, the main contributions of this paper are as follows:

1. In the input stage, we design a MFS module to extract different levels of edge and texture features from the input medical images. Multiple thresholds are applied to capture image features in a local-to-global manner, including small texture differences and significant edge changes. By increasing the richness of the input image features, the ability of the model encoder to capture image features can be improved. The computational complexity of threshold separation is linear, so the module does not need large-scale calculations, and it is a lightweight module.

2. CA mechanism is introduced; this mechanism enhances the structural information recognition capabilities of the encoder, maintains the position sensitivity of the deep feature extraction process, and realizes cross-space feature integration by encoding features in the horizontal and vertical dimensions, which optimizes the ability of the model to capture long-distance dependencies in the input image and allows the model to more effectively recognize and integrate key structural information.

3. A transformer module is introduced at the fourth skip connection position of the U-shaped deep learning architecture, and its self-attention mechanism is used to capture the global dependencies and long-range dependencies of high-dimensional data and promote the circulation and integration of global information throughout the whole network. This module not only retains the spatial inductive deviation ability of the original bottleneck layer but also improves the ability of the overall network to process complex medical images.

Related work

In this section we will explore related work on lung nodule segmentation models and current work on attentional mechanisms used for lung nodule segmentation.

Research on lung nodule segmentation models

Lung nodule segmentation focuses mainly on standard CNN architecture. With the further development of CNN, Ronneberger et al.⁹ proposed a U-Net model based on an encoder-decoder structure, which uses an encoder to extract target features, integrates information and recovers the original resolution through a decoder, and retains some of the observed features through skip connections between the encoder and decoder. Due to its low data requirements and effective segmentation performance, U-Net has been widely adopted. Consequently, researchers are enthusiastic about using the encoder-decoder architecture as the foundational structure for segmentation.

Jiang et al.¹⁷ proposed the STCA-net, which adopts a dual-layer encoder-decoder architecture. In the data preprocessing stage, it uses “Cut-norm” and the Sobel operator to extract contrast and boundary information, generating mixed features that enhance the model’s sensitivity to low-contrast nodules. It also utilizes a 7 77 window in an axial multi-scale spatial attention module to capture directional features in the coronal, sagittal, and axial planes, allowing for spatial feature extraction and long-range dependency capture. However, the fixed-size window somewhat limits its adaptability to dynamic feature changes, making it less effective in handling nodules with significant variability or uncertainty. In contrast, the self-attention mechanism of Transformers can adaptively adjust weights based on variations in the input image features. While the Sobel operator enhances boundary information, it may also introduce misleading edge information outside the nodule boundaries, causing the model to capture irrelevant features during training. Therefore, increasing the information in the model input through simple threshold extraction, without introducing artificial biases, is a feasible strategy.

Xu et al.¹⁸ proposed DEF-NET, which employs two independent encoders: the Structure Encoder and the Particle Encoder, designed to extract global contour information and local detail information, respectively. Each encoder utilizes two types of dilated residual blocks, CEDR-Block and MEDR-Block, to model long-range dependencies, expanding the receptive field while minimizing the increase in computational burden. This allows the model to efficiently capture long-range dependencies with lower computational costs. However, the discrete sampling method of dilated convolutions may cause grid effects, limiting performance when handling long-range dependencies between multiple dispersed nodules in an image. Additionally, the model can only implicitly capture nodule positional information through the convolutional receptive field, which may lead to confusion of different feature positions when merging features from the two encoders. Similarly, Selvadass et al.¹⁹ proposed the SAtUNet model, which applies a series of atrous convolution units after each stage of the encoder and decoder, also encountering the aforementioned long-range dependency issues.

Keetha et al.²⁰ proposed U-Det, which utilizes a Bi-directional Feature Pyramid Network (Bi-FPN). The top-down feature flow path conveys global high-level semantic information, while the bottom-up path ensures the retention of detailed features. This bidirectional feature flow (top-down and bottom-up) achieves efficient multi-scale feature fusion, facilitating cross-layer feature transmission and sharing across different levels, thus addressing the issue of long-range dependencies to some extent. However, U-Det performs less effectively when capturing small nodules or regions with fuzzy boundaries. This limitation arises because Bi-FPN’s feature fusion method relies more heavily on medium- and large-scale features, which can weaken small-scale feature information during fusion, leading to reduced segmentation accuracy. The insufficient recognition of small nodules and edge details constrains U-Det’s performance in segmenting complex lung nodule types, such as ground-glass nodules and micro-nodules. Therefore, effectively separating and capturing edge and fine texture features at the model’s input stage, providing hierarchical edge and detail information, is a feasible strategy.

SETR et al.²¹ introduced a transformer-based image segmentation method for the first time, providing a new idea for image segmentation. Ma et al.²² proposed the SW-UNet network for segmenting lung nodules. They designed a transformer module using a sliding window and combined it with U-Net to overcome the lack of interactions between different regions in the convolution operation and improve the ability of the decoder and encoder to interact with each other. The number of stacked modules was reduced by increasing the embedding dimensionality of the multihead self-attention mechanism, thereby decreasing the number of model parameters. However, owing to the complexity of the model structure, the tuning process of this model may become more complex, and its time consumption may be greater.

Table 1 summarizes the advantages and disadvantages of the aforementioned methods, detailing their applicability and limitations in various contexts.

Table 1.

Summary of advantages and disadvantages of various models.

Model	Advantages	Disadvantages
U-Net9	-Low data requirements, suitable for small datasets	May not capture long-range dependencies well, limiting perception of wide-range features
	Effective segmentation performance, especially in medical image segmentation tasks
	Skip connections between encoder and decoder help retain features, achieving high-quality reconstruction
STCA-net17	Uses a dual-layer encoder-decoder architecture, further enhancing feature extraction capabilities	Fixed-size window limits adaptability to dynamic features, reducing effectiveness with nodules of significant variability
	In the preprocessing stage, “Cut-norm” and Sobel operators extract contrast and boundary information, increasing model sensitivity to low-contrast nodules	Sobel operator may introduce misleading boundary information, causing the model to capture irrelevant features
	Multi-scale spatial attention module captures directional features at different scales
DEF-NET18	Uses independent encoders to separately extract global contour and local detail information, ensuring feature completeness	Dilated convolutions may cause grid effects, resulting in discontinuous feature sampling that affects long-range dependency capture
	Efficiently captures long-range dependencies with dilated residual blocks (CEDR and MEDR), expanding the receptive field with minimal computational overhead	Limited performance in handling multiple dispersed nodules, with less effective nodule position capture
SAtUNet19	Uses atrous convolutions for multi-scale feature extraction, enhancing model performance at different scales	Long-range dependency issues persist, with limitations in handling distant dependencies
	Attempts to address long-range dependency issues, improving the capture of wide-range features	Limited capture of small nodules and fine details, impacting segmentation accuracy of small structures
U-Det20	Bi-directional Feature Pyramid Network (Bi-FPN) improves multi-scale feature fusion, enhancing cross-layer information flow	Poor performance in segmenting small nodules or fuzzy boundaries, as Bi-FPN may weaken small-scale features, reducing segmentation accuracy
	Efficient multi-scale feature fusion improves segmentation accuracy
SETR21	Transformer-based segmentation approach, introducing a new perspective on image segmentation	Complex model structure, leading to high training and inference costs
	Global attention and context information improve feature extraction quality
SW-UNet22	Combines advantages of Transformer and U-Net, enhancing feature representation capabilities	Complex model structure makes tuning challenging
	Reduces model parameters by increasing embedding dimensions, improving encoder-decoder interaction	Increased model complexity leads to longer training time

Attention mechanisms

In the task of lung nodule segmentation, utilizing an attention mechanism to focus on the key part of an image can significantly improve the accuracy and efficiency of model segmentation²³. The squeeze-and-excitation (SE) attention mechanism was first proposed by Hu et al.²⁴; it captures the dependencies between channels through global average pooling, reweighting the features of different channels to enhance or inhibit specific features. Yang et al.²⁵ integrated the SE attention module into the U-Net architecture to improve its segmentation effect by focusing on relevant features and reducing the uncertainty caused by intraclass and interclass variations. Although the SE attention mechanism improves the ability of the model to adapt to the importance of different channel features by focusing on the important features of lung nodules, it ignores the importance of spatial location information.

REMU-Net²⁶ adopts ResNeSt as the backbone network for feature extraction and integrates a Spatial Attention Module (SAM) after the channel attention mechanism. Combined with ResNeSt’s “Split-Attention Block,” REMU-Net utilizes multiple convolutional kernel branches within the same convolution layer, allowing the model to extract information from different perspectives and feature scales. This feature extraction capability improves the model’s performance, especially when handling small nodules, ground-glass nodules, and other complex cases.

Although the channel and spatial attention modules indeed enhance the focus of feature extraction, the spatial attention mechanism in REMU-Net mainly focuses on local features, with limited integration of features at longer distances. This limitation can lead to information decay, resulting in insufficient spatial associations when the model is tasked with handling multiple dispersed lung nodules or accurately locating complex boundary features. In other words, when nodules are distributed sparsely or when high-precision long-range information is needed, the model may struggle to establish effective feature associations.

Zhang et al.²⁷ used the Convolutional Block Attention Module (CBAM) in the encoder-decoder model, combining channel attention and spatial attention to reweight the channel and spatial dimensions. CBAM utilizes location information by reducing the channel dimensionality of the input tensor and then computing spatial attention via convolutions. However, convolutions can only capture local relationships, which play a minimal role in modeling the long-range dependencies necessary for segmentation tasks.

In contrast, the Coordinate Attention (CA) mechanism can integrate horizontal and vertical feature information over a larger range, effectively establishing cross-regional connections in space. This enables the model to capture long-range dependencies more accurately, preventing features from becoming blurred or erroneous due to a lack of spatial association for distant information.

Methods

In this section we give the general architecture of the proposed model and its details, as well as the optimization strategy of the model in which the loss function is used.

Model architecture

We give the overall framework of the MCAT-Net based on the encoder-decoder architecture in Fig. 2. In the encoder component, the input image is fed into the MFS module which performs the preliminary feature separation and then reaches the bottleneck layer through four pooling layers, three CA mechanisms and three two-dimensional convolution blocks, where each two-dimensional convolution block consists of two 3*3 convolution layers, layer normalization and a ReLU activation function. The input of the decoder component is the 6*6*512 feature map yielded by the bottleneck layer, which needs to be restored to its original size via 4 linear interpolation operations and 5 two-dimensional convolution blocks. Finally, the output result is obtained by a SoftMax layer.

Fig. 2.

Structural diagram of the MCAT-Net model (each cuboid represents an operation, and the subsequent number represents the size of the feature map produced after the operation is completed).

Multi-threshold features separation module

Inspired by traditional threshold segmentation and spatial pyramid pooling (SPP)²⁸, an MFS module is proposed in this paper. This module separates the digital features of the input image through different thresholds, determines the key image features, and then improves the performance of the whole network in the image segmentation task. It is designed to provide a lightweight but powerful tool for achieving improved segmentation accuracy and efficiency in critical application domains such as medical image analysis without adding a significant computational burden.

Computed tomography (CT) images are generally stored in computers as pixel values. Each pixel has a specific value that indicates the intensity of the corresponding point. Medical image segmentation tasks usually involve separating the regions of interest in an image, and these key regions generally have obvious intensity differences. By setting multiple reasonable thresholds, the original image features with different intensities can be obtained, and some invalid interference factors can be screened out. On this basis, further feature extraction steps can yield more abstract semantic information. The specific working principles of this module are shown in Fig. 3.

Fig. 3.

The structure of the MFS module. Four thresholds are used to binarize the input image, then the generated binary feature map is copied and spliced with the original image, and the element summation operation is performed. Finally, the result is output by convolution.

In this study, the input lung nodule image is a single-channel grayscale image, and the grayscale range of each pixel is within 0–255. For this grayscale image with a size of , the grayscale value of each pixel in the corresponding image matrix M is represented by an 8-bit binary number in the computer, which is denoted as ,,...,. We extract the elements of pixel value matrix one by one from high to low and binarize them according to the thresholds to obtain a digital feature map. The specific process is as follows. We take Threshold7 as the benchmark. A pixel in the image matrix M is assigned a value of if and 0 if . Thus, the first digital feature map Map1 is obtained. Then, this digital feature map is subtracted from the original image matrix to obtain a new image matrix , and Threshold6 is taken as the benchmark. If , the value is set to . If , the value is set to 0. Thus, a second digital feature map Map2 is obtained. According to this logic, eight feature maps are obtained. According to this logic, eight feature maps are obtained over eight iterations, by progressively eliminating already recognised features from the image, this approach reduces overlap between features, allowing each step to focus on new information that has not yet been extracted, thus improving the overall efficiency of information utilisation.

Notably, an excessive number of selected thresholds does not mean that the extracted features will be richer and more effective. In contrast, more noise will be introduced in the subsequent convolution, and the weights of some important features will be replaced, weakening the sensitivity of the model to important features. Therefore, according to the grey feature distribution of the input image, only four thresholds are selected to obtain four digital feature maps. We copy the four digital feature maps twice each, then copy the original image eight times and splice them in the channel dimension to obtain a 16-channel feature map FM. Eventually, we copy the original image 16 times and add it the results the feature map FM in an elementwise manner to obtain a 16-channel feature map . This is done because channel splicing can fuse information at different scales so that the model can effectively capture features at different scales. Element-by-element addition can highlight the salient features and integrate global information to improve the performance of the model in complex tasks.

We implement through two convolutions, normalization, and activation functions to expand its number of channels to 32, thus obtaining deeper features that can be used as inputs for the subsequent layers.

Coordinate attention mechanism module

CA¹⁴ encodes accurate location information concerning channel relationships and remote dependencies through two coordinate information embedding and CA weight generation steps, and the structure of the CA mechanism is shown in Fig. 4.

Fig. 4.

Schematic of the CA mechanism. The coordinate attention mechanism integrates the information of different spatial dimensions of features through one-dimensional mean pooling while realising the embedding of coordinate information, the attention generation is achieved through convolution operation and sigmoid function, and subsequently the attention weights are applied to the input feature maps through multiplication.

Since the location information of nodules is very important for capturing spatial structures in segmentation tasks, when coordinate information is embedded into channel attention, to ensure that the location information is not lost when compressing global spatial information to channel information and that the attention module accurately captures remote interaction information in space, CA decomposes the global pooling operation into one-dimensional feature coding. This is implemented as follows. For the input feature map x, each channel is encoded along the horizontal and vertical coordinates by using pooling kernels with two spatial ranges (H, 1) and (1, W), respectively; therefore, the output of channel c at a height of h can be expressed as follows.

1

The output of channel c at a width of w can be expressed as follows.

2

where is the output associated with channel c and the input x comes directly from a convolutional layer with a fixed kernel size.

The above two encoding methods aggregate features along two spatial directions to generate a pair of direction-aware feature maps. Moreover, the attention block can capture long-range dependencies along one spatial direction and retain precise location information along the other spatial direction, which helps the network locate the object of interest more accurately. Next, CA weight generation is performed. The aggregated feature maps generated by the above two equations are first concatenated and sent to the shared convolution F, which is calculated as follows:

3

where [,] represents the concatenation operation performed along the spatial dimension, is the ReLU nonlinear activation function, and is the intermediate feature map for encoding spatial information in the horizontal and vertical directions. Notation r is the reduction ratio that controls the number of channels. A larger reduction ratio reduces the number of parameters and the computational complexity but may lead to the loss of some important feature information; a smaller reduction ratio can retain more feature information but increase the computational overhead. In this study, r is set to 32 by considering hardware computing performance and the complexity of the model. We then split into two tensors along the spatial dimension: and . Two convolutions and are used to transform and , respectively, into tensors with the same number of channels as that contained in the input x.

4

where is the sigmoid function; then, the outputs and are separately expanded and used as attention weights. Eventually, the output y of the CA block can be written as follows:

5

Transformer module

The core strength of the transformer module is its ability to handle and integrate long-distance information dependencies. Due to the complexity of the calculation method used by the transform module¹⁵, adding it to each skip connection would consume too many computing resources. Moreover, the fourth skip connection of the encoder has the widest receptive field in its feature map and covers sufficient information, so it is input into the transformer module. This enables the establishment of long-range dependencies between nodule regions and non-nodule regions, thus improving the interactivity of the system. Therefore, this paper introduces the transformer module only at the fourth skip junction of the model, and its structure is shown in Fig. 5.

Fig. 5.

Structure of the transformer module. Input features are processed into sequences by image sequentialization layer, converted into higher dimensional features by patch embedding, and long range dependencies are captured by multiple transform layers, where transform layers are an important part of this module.

The transformer module includes image serialization, patch embedding, and multiple transformer layers. Specifically, the feature map output by the third CA block is rearranged into a series of flattened two-dimensional patches ; the size of each patch is , and the number of patches is , where N is the length of the input sequence. Since the input image has a low resolution and a smaller patch size can better capture local features and fine-grained features, we choose a patch size of .

We map the vectorized to the latent d-dimensional embedding space via a trainable linear projection, and the output of this linear projection is called a patch embedding. To encode and preserve the spatial location information of these patches, we learn specific location embeddings and embed them into the patches as follows:

6

where is the patch embedding projection and represents the location embedding.

At this point, serves as the input of the transformer layer. We refer to the number of layers in the vision transformer (ViT)-based model and set the number of layers to 12²⁹. In each transformer layer, we use a multihead self-attention (MSA) mechanism and a multilayer perceptron (MLP), and we apply layer normalization (LN) and residual connections before and after these modules, respectively. The MSA enables the transformer module to efficiently integrate features derived from different locations without being limited by local receptive fields. This means that it can capture both global information and local details, enabling the model to not only focus on the edge features of lung nodules but also consider the distribution of the nodules in the whole lung along with background information. The output of the l-layer transformer can be expressed as:

7

8

The MLP consists of two linear layers with Gaussian error linear unit (GELU) activation functions. We reshape the output into a feature map with a size of , which is then added with the features in front of the input module as the input of the first-layer decoder.

Loss function

Choosing an appropriate loss function can provide a more effective optimization path for the model³⁰, thereby improving its performance. The Dice loss function is a statistical tool used to measure the similarity between two samples. In the lung nodule segmentation task, a nodule area occupies a small proportion of the overall area of the input image, which leads to an extreme imbalance between the numbers of positive and negative samples. Thus, the imbalance between foreground pixels and background pixels is alleviated by the loss function. Therefore, this paper uses the Dice loss function to optimize the model, whose expression is as follows:

9

where is the true label of sample i and is the prediction probability of sample i, and N is the total number of samples.

Experimental results and analysis

This section gives the implementation of the model in detail and its comparison with other models in both visualization and metrics measurement.

Datasets

• The LIDC-IDRI dataset is obtained from the Lung Image Database Consortium and the Image Database Resource Initiative and is currently one of the largest publicly available lung CT image datasets. The dataset consists of 1018 CT images provided by seven academic centres, with lesions annotated by up to four physicians in each case. The data sample screening process used in this paper is based on the screening strategy of the LUNA16 dataset. Finally, 888 CT images containing 1186 lung nodule samples are obtained³¹.

• The Lung Nodule Database (LNDb) is a public database for lung nodule detection and segmentation tasks that contains CT images and their annotations; it was designed to support the early detection and diagnosis of lung cancer. This database, created by the Instituto Superior Tecnico (IST) in Portugal in collaboration with several medical institutions, provides 294 CT images for the research and development of automated lung nodule detection algorithms.³²

The data preprocessing method used in this study is shown in Fig. 6. First, a CT value histogram is drawn according to the CT samples of the given case (as shown in Fig. 7). According to the distribution range of the CT values in the histogram, the information between 1100 Hu and 450 Hu is extracted to remove irrelevant information and external environmental noise, such as air and water. Next, the mask data concerning the lung parenchyma region in the dataset are used to further eliminate the regions outside the lung parenchyma in the CT images to accurately preserve the lung parenchyma region. After preprocessing the CT images, we crop them to images according to the locations of the nodules. For the LIDC-IDRI dataset and LNDb dataset, we extract 9797 and 1361 images, respectively, and the ratio of the training set to the test set is 9:1.

Fig. 6.

Data preprocessing workflow. Extract the pulmonary parenchyma image from the original CT based on the lung mask file, and crop the image according to the location of the nodules.

Figure 7.

Distribution of the HU values in the selected CT images. Based on the distribution of HU values, information was extracted for HU values ranging from -1100 to 450.

Implementation details

All the experiments are implemented in Python via the PyTorch library on a server equipped with a 32-GB NVIDIA Tesla V100S graphics card. During training, the adaptive moment estimation (Adam) optimizer is used, the learning rate is set to 0.001, the batch size is set to 128, and the MultiStepLR learning rate scheduler is used to adjust the learning rate in multiple steps. The patch size P in the transformer module is set to , and the dimensionality of the MLP-based feedforward neural network is set to 1024.

Evaluation metrics

This paper uses four metrics to evaluate the performance of the model: the Dice similarity coefficient (DSC), intersection over union (IOU), sensitivity (SEN), and positive predictive value (PPV). The DSC is mainly used to evaluate the proportion of overlap between the segmentation results predicted by the model and the actual segmentation results, making it the most important of these evaluation indicators³³. The four measures are calculated as follows:

10

11

12

13

In the calculation formulas, TP is the number of true positives, FP is the number of false positives, and FN is the number of false negatives. The values of the above four evaluation metrics are between 0 and 1. The higher the model segmentation accuracy is, the more the values of these metrics tend toward 1.

Comparative experiment

We show the lung nodule segmentation results produced by MCAT-Net and comparison with those of CE-Net³⁴, nn-UNet³⁵,SegNet³⁶, U-Net⁹, U-Net ++³⁷ and TransUNet³⁸ to demonstrate the superior performance of our network in all respects below.

To prove the effectiveness of MCAT-Net and ensure the fairness of the comparison, we reproduce the six deep learning-based segmentation models described above according to the best parameters in their original papers. Comparative experiments are carried out on the data obtained via the data preprocessing method mentioned above. Additionally, we use a two-tailed Wilcoxon Mann-Whitney U test to compare the segmentation performance of MCAT-Net with other models in terms of the p-value (based on the Dice coefficient). The results shown are the best results obtained across the testing process, and the results are presented in Table 2, We randomly selected an image from LIDC-IDRI to test the parameter count and computational complexity during the model inference process, as shown in Table 3.

Table 2.

Results of comparisons with advanced models on the LIDC-IDRI and LNDb datasets.

Method	Dataset	DSC%	IOU%	SEN%	PPV%	P-values
CE-Net	LIDC-IDRI	81.64	69.24	77.48	87.39
nn-UNet	LIDC-IDRI	84.04	73.66	82.40	87.80
SegNet	LIDC-IDRI	83.82	72.21	80.56	87.63
U-Net	LIDC-IDRI	85.44	74.83	82.85	88.67
U-Net++	LIDC-IDRI	86.07	75.62	83.64	89.21
TransUNet	LIDC-IDRI	86.06	75.60	85.53	87.05
Ours	LIDC-IDRI	88.29	79.10	86.33	90.39	/
CE-Net	LNDb	61.65	44.57	51.35	79.26
nn-UNet	LNDb	64.48	52.44	73.11	60.38
SegNet	LNDb	55.98	39.26	64.08	50.28
U-Net	LNDb	69.07	53.06	59.24	82.71
U-Net++	LNDb	72.45	57.40	67.24	80.48
TransUNet	LNDb	74.39	59.77	68.75	80.68
Ours	LNDb	78.51	65.55	75.05	84.10	/

The best results is bolded.

Table 3.

The parameter count and computational complexity of different models.

Method	Para (M)	FLOPs (G)
CE-Net	38.95	8.21
SegNet	1.70	1.89
nn-UNet	11.96	1.22
UNet	7.76	1.90
UNet++	13.39	4.37
TransUNet	73.38	4.35
MCAT-Net	55.41	8.81

The changes in the DSC values of each model on the test set during the training process are shown in Fig. 8 . Overall, the model proposed in this paper is superior to the other models in terms of training stability and segmentation accuracy.

Fig. 8.

Each model is trained and tested on the LIDC-IDRI and LNDb dataset. The abscissa is the total number of epochs used for training, and the ordinate is the DSC value of the corresponding model on the test set.

To further prove the accuracy of the segmentation results, Fig. 9 gives the visualization of the segmentation results produced by the five models. Among them, nodules a-g are from the LIDC-IDRI dataset, and nodules h-k are from the LNDb dataset. we present the segmentation results of six models on different types of lung nodules. Nodule a represents a non-solid ground-glass nodule, where the contrast is uneven, and the nodule boundary contrast is low with little differentiation from the surrounding tissue. Both U-Net and MCAT-Net achieved a DSC of 0.93 for segmentation, while other models struggled to segment the low-contrast nodule edges, being more sensitive to the high-contrast areas. Nodule b is a partially solid nodule. SegNet, U-Net, nn-UNet,U-Net++, and MCAT-Net can visually differentiate the nodule type, while the other two models failed to segment the non-solid part, making it difficult to identify the nodule type. Nodule c is a small, isolated nodule, whose segmentation difficulty arises from its high similarity to lung blood vessels, making the model prone to misclassification. Despite its small volume and unclear boundaries, all models correctly identified the nodule, with U-Net++ and MCAT-Net achieving higher DSC values than the others. Nodules d, g, and i are calcified nodules, which are easier to segment, and the results from all models are similar. Nodule e and j images contain multiple nodules. In image e, which also contains a lung wall nodule, SegNet, U-Net, and MCAT-Net successfully segmented multiple nodules, while the other models failed to segment the lung wall nodule. In image j, SegNet, nn-UNet,U-Net, and U-Net++ missed some nodules, but TransUNet, SENet, and MCAT-Net identified all nodules, though with some misclassification in non-nodule regions. Nodule f is a needle-shaped nodule with a complex boundary. The segmentation result of SegNet deviated significantly from the real nodule boundary, but other models had a small morphological difference, with MCAT-Net achieving the highest DSC of 0.9152. Nodule k represents a lung wall nodule, where the nodule’s position close to the lung wall causes its contrast to be similar to the surrounding environment, making its boundary indistinct. Models need to compensate for boundary information, especially near the lung wall. MCAT-Net’s segmentation result is more accurate than other models, closely matching the true nodule. In conclusion, MCAT-Net demonstrates significant advantages in nodule localization and edge detail segmentation.

Fig. 9.

It shows the visualization of the segmentation results produced by each model. The top-right corner of each figure is the DSC value of the corresponding segmentation results. Our model achieves the best segmentation results.

Comparison with existing models

To ensure that the developed segmentation model is accurate and reliable, we compare it with methods used by other researchers on homology datasets. In this paper, the DSC is used as the main evaluation index for conducting the comparison with the existing models. As shown in Table 4. The table shows that DEF-Net¹⁸, SAtUNet¹⁹ and STCA-NET¹⁷ outperform the other models to some extent, but their performance is still lower than that of the model proposed in this paper.

Table 4.

Comparison with several existing models.

Method	Dataset	Input Size	DSC%	Strategy for Nodule Selection	Year
SW-UNet22	LUNA16	128x128	84.00	CT layer thickness 2.5 mm, CT with missing slices removed	2023
ResFCN39	LIDC-IDRI	64x64	84.76	All data in LIDC-IDRI	2022
REMU-Net26	LIDC-IDRI	64x64	81.10	All data in LIDC-IDRI	2022
SAtUNet19	LUNA16	64x64	87.00	CT layer thickness < 2.5 mm	2024
U-Det20	LIDC-IDRI	128x128	86.12	Nodule diameter > 3 mm, CT layer thickness < 2.5 mm	2020
DEF-Net18	LUNA16	96x96	87.39	CT layer thickness 2.5 mm, CT with missing slices removed	2023
STCA-Net17	LUNA16	64x64x64	86.89	CT layer thickness 2.5 mm, CT with missing slices removed	2024
Ours	LIDC-IDRI	96x96	88.29	Nodule diameter > 3 mm, CT layer thickness < 2.5 mm	2024
Ours	LNDb	9696	78.51	Nodule diameter > 3 mm, CT layer thickness < 2.5 mm	2024

The best results is bolded.

Ablation experiment

To maximize the effect of the model, we conducted ablation studies on each module using the LIDC-IDRI dataset. The dataset was divided into training and testing sets in a 9:1 ratio, with the performance on the testing set evaluated after each epoch. The experimental strategy we adopt is as follows. The basic U-shaped architecture is taken as the baseline network; one module at a time, the multiscale separation module, the CA mechanism, and the transformer module are added in turn, and the optimal parameter configuration of each module is explored. The baseline index of the ablation experiment regarding one module is the best result obtained in the ablation experiment concerning the former module.

Ablation experiments concerning the MFS module

The MFS module captures the edge and texture features of the input image at different levels and specific strengths through different thresholds and concatenates these extracted features with the original input image so that the whole model receives a rich amount of information. We experiment with different combinations of thresholds to find the optimal threshold, and the experimental results are shown in Table 5.

Table 5.

Performance metrics for different combinations of thresholds.

Order of Thresholds Combination	DSC%	IOU%	SEN%
Baseline	85.53	74.97	82.85
T6, T5, T4, T2	84.94	74.05	81.70
T6, T5, T4, T3	85.30	74.54	83.12
T7, T5, T4, T3	85.13	74.28	82.39
T7, T6, T4, T1	85.70	75.11	82.93
T7, T6, T5, T3	86.25	76.04	83.65
T7, T6, T5, T4	86.32	76.08	83.69

The best results is bolded.

In this study, four thresholds are selected for combination purposes. Since the grey value range of the grey image is 0-255 and its storage size in the computer is 8 bits, we use T0 to represent and T7 to represent We find that when the threshold combination is T7,T6,T5,T4, each index is greater than the baseline; the DSC is 1.09% greater than the baseline, and the IOU and SEN are 1.14% and 0.84% greater than the baselines, respectively. This may be attributed to the fact that the features separated by these thresholds can better satisfy the information richness requirements of the subsequent deep convolution for the input images. When the threshold combination contains a low threshold value, it causes a decline in each index, dropping even lower than the baseline level. When the feature separation threshold is too low, the extracted features lose many important details.

Moreover, different pixels in adjacent areas are incorrectly merged, resulting in confusion between the foreground pixels and background pixels such that the objects that should be separated are connected, which increases the misclassification rate and affects the accuracy of the model to a certain extent. Moreover, a low threshold causes the image contrast level to be too low so that the proportion of white pixels in the image is too high and the pixel contrast of the image is unbalanced. In multiple groups of experiments, the overall indicators of T6,T5,T4,T3 are greater than those of T7,T5,T4,T3. We determine that the texture details of the feature map extracted by threshold T6 are more sufficient than those extracted by T7, and this statement is also verified in Fig. 10.

Fig. 10.

Map1, Map2, Map3, and Map4 are the digital feature maps extracted by the T7, T6, T5, and T4 thresholds, respectively.

Ablation experiments concerning the CA mechanism

To explore the effectiveness of the CA mechanism, we attempt to embed the CA mechanism at other positions before the different skip connections of the model encoder and at positions before the bilinear interpolation operation of the decoder (we represent these two positions according to the encoder (i) and the decoder (i)).

Table 6 shows that adding the CA mechanism at encoder position (2,3,4) can achieve the best results. When the attention mechanism is added to encoder position (1,2,3) and encoder position (2,3), the improvements achieved in terms of various indicators are small compared with the baselines, which may be due to the small number of channels contained in the shallow module and the insufficient number of features that can be represented by each channel, resulting in insufficient attention being paid by the CA mechanism to the important features at the shallow position of the model. The effect is also not obvious at encoder position (3,4), which may be due to the lack of the CA attention mechanism in the second layer, resulting in the important features at the earlier stage not being fully enhanced; this then affects the feature representations at the subsequent levels.

Table 6.

Results of adding the CA mechanism at different points in the model. model.

Embedding Position	DSC%	IOU%	SEN%
Baseline	86.32	76.08	83.69
Encoder (123)	87.12	77.25	84.22
Encoder (23)	87.16	77.32	84.21
Encoder (234)	87.63	78.06	85.72
Encoder (34)	86.90	76.92	83.92
Encoder (1234)	87.23	77.42	85.53
Encoder (1234) Decoder (1234)	87.05	77.13	84.62

The best results is bolded.

Adding attention at decoder position (1,2,3,4) and all positions of the encoder and decoder results in little improvement, in which our reasoning is because as the number of deep model channels increases and the feature representation ability of each channel improves, adding an attention mechanism at all layers may lead to redundant feature enhancements. Similarities may be observed among the feature representations of different layers, and repeatedly enhancing similar features leads to information redundancy, which does not improve the performance of the model and may even interfere with useful features.

Ablation experiment concerning the transformer module

The transformer module is added at the fourth skip connection position to explore the impacts of the mapping dimensionality and number of heads of this module on the resulting segmentation performance. The experimental results are shown in Table 7. We test a classic transformer mapping dimensionality of 768 and a mapping dimensionality of 256 with the same number of input channels as that of the corresponding fourth skip connection.

Table 7.

Comparison among the impacts of different embedding dimensions and head counts on the performance of the transformer module.

Embedded dimensions/The numbers of heads	DSC%	IOU%	SEN%
Baseline	87.63	78.06	85.72
256 dimensions/4 heads	88.15	78.51	85.37
256 dimensions/8 heads	88.02	78.65	85.82
256 dimensions/16 heads	87.96	78.55	84.81
768 dimensions/16 heads	88.04	78.69	85.35
768 dimensions/8 heads	88.29	79.09	86.33
768 dimensions/4 heads	88.25	79.04	86.09

The best results is bolded.

As shown in the table, when the mapping dimensionality is 768 layers, all the indicators are greater than those achieved with 256 dimensions on the whole. When the transformer has a mapping dimensionality of 768 layers and contains 8 heads, all the indicators reach their optimal levels. Compared with those of the baseline network, the DSC value increases by 0.66%, and the IOU and SEN increase by 1% and 0.61%, respectively. Our analysis shows that higher dimensions can enable the model to represent more information and finer-grained features and learn more complex feature relationships. When too many heads are set, the performance of the model can also degrade. When the number of heads is 16, the information richness attained by each head is low, which decreases the overall feature representation ability of the model; the dimensionality of each head is also relatively low, resulting in the failure of a single head to learn important features, reducing its ability to capture global context information.Table 8 presents the parameter counts for each module in the optimal structure of MCAT-Net during the training process.

Table 8.

Parameter counts for each module in MCAT-Net.

Module	Total parameters (M)
Base model	8.54
Multi-threshold Feature Separation (MFS)	0.007
CoordAttention (CA)	0.012
Transformer Block	46.85
Total parameters	55.41

Conclusion

We proposed and developed a new pulmonary nodule segmentation model in this study. It mitigates the problems of the model not being able to adequately capture edge and semantic information during the encoding process, losing information during long-distance feature interaction, and not being able to effectively integrate global information.Through the experiments, it was verified that the proposed model could effectively segment some nodules with complex shapes and different sizes. The model attained strong generalization performance, which was better than that of most current pulmonary nodule segmentation models in our comparison.

However, this model also has its limitations. In the multi-threshold feature separation module (MFS module), different thresholds are used to separate and extract features, but these thresholds need to be manually selected and adjusted based on the gray-level characteristics of the input images. The choice of thresholds has a significant impact on model performance, and manual tuning can be time-consuming and imprecise. Additionally, fixed thresholds may perform poorly when handling images of varying types and quality. MCAT-Net incorporates multi-threshold feature separation, coordinate attention, and Transformer modules, which, although improving segmentation performance, also significantly increase model complexity and computational cost. In particular, the Transformer module, while capturing long-range dependencies, requires substantial computational resources, which can lead to longer inference times, making the model unsuitable for real-time applications or resource-constrained environments. Although the Transformer module and coordinate attention mechanism have advantages in capturing global dependencies and spatial information, these mechanisms may prioritize global dependencies over some local features, thus affecting segmentation accuracy. Additionally, model performance is sensitive to numerous hyperparameters, such as the number and selection of thresholds, the reduction ratio in the CA mechanism, and the embedding dimension and number of heads in the Transformer. Tuning these hyperparameters requires extensive experimentation; otherwise, suboptimal choices could lead to performance degradation, posing challenges to the model’s generalizability.

Author contributions

H and L were primarily responsible for writing the main text of the manuscript and for preparing all figures and tables. All authors participated in the conceptual framework and design of the study, and were involved in the analysis and interpretation of the data. Each author has reviewed and approved the final version of the manuscript.

Data availability

The LIDC-IDRI dataset used in this paper is available from website1(https://www.cancerimagingarchive.net/collection/lidc-idri) and the LNdb dataset is available from website2((https://zenodo.org/records/6613714). Similarly, the datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

Competing interests

The authors declare no competing interests.