A fused attention-based hybrid model for semi-supervised medical image segmentation

Abstract

Addressing the significant challenge of extensive data annotation in medical image segmentation, semi-supervised techniques have become an effective alternative for leveraging both labeled and unlabeled data. However, accurately capturing complex local structures and global contextual information remains difficult, often leading to inconsistent segmentation results. To mitigate these limitations, we propose a fused transformer-based hybrid architecture for semi-supervised medical image segmentation. The model integrates a parallel backbone comprising Deformation Convolution Blocks (DCB) and Fused Transformer Blocks (FTB), followed by a segmentation head for precise mask prediction. The DCB enhances spatial adaptability for irregular and artifact-rich regions, while the FTB – with its fused attention mechanism – captures long-range dependencies efficiently. A Ghost Layer Perceptron (GLP) embedded within the transformer further improves computational efficiency without compromising representation quality. In addition, the incorporation of a consistency loss and unsupervised contrastive learning facilitates robust feature discrimination on unlabeled data, improving generalization across modalities. Extensive experiments on four publicly available medical imaging datasets demonstrate that the proposed model achieves comparable or better accuracy than recent state-of-the-art methods, while requiring substantially fewer parameters and lower computational cost, underscoring its practicality for real-world clinical applications.

1. Introduction

Semi-supervised medical image segmentation (SSMIS) is an evolving research field focused on developing models that learn from a small set of labeled images alongside a larger pool of unlabeled images [1]. Unlike fully supervised segmentation, which relies on extensive annotated datasets, SSMIS explicitly leverages unlabeled data to improve model generalization and reduce the annotation burden. In medical imaging modalities such as dermoscopy, MRI, and ophthalmic imaging, acquiring expert-level pixel annotations is time-consuming and expensive, making SSMIS an attractive approach. Various strategies, including consistency regularization, pseudo-labeling using teacher–student learning, and contrastive representation learning, have recently been explored to effectively exploit unlabeled data [2–5]. These approaches allow SSMIS frameworks to achieve competitive segmentation performance while requiring significantly fewer labeled samples, making them particularly suitable for real-world clinical applications.

Accurate segmentation is crucial for diagnosis and treatment planning, and recent advancements in deep learning have significantly improved medical image segmentation performance [6]. Architectures such as U-Net [7], DeepLab [8], and RefineNet [9] have shown strong capabilities in segmenting diseased tissues by learning rich representations from annotated datasets. However, their success largely depends on extensively labeled data, which are scarce in many clinical domains. Moreover, CNN-based methods, though effective in local feature extraction, have a limited ability to capture long-range dependencies due to their localized receptive fields [10,11]. To address this, researchers have developed hybrid architectures that combine CNNs for extracting fine local details with vision transformers for modeling global contextual relationships [12,13]. These hybrid approaches are well-suited to SSMIS, where labeled data are limited, but global feature representation remains critical [11,14].

Contrastive learning has further enhanced SSMIS frameworks by improving representation learning from limited labeled data. By bringing similar samples closer and pushing dissimilar ones apart in the feature space, contrastive learning helps the model extract more discriminative features from both labeled and unlabeled datasets [4,5]. Despite these advancements, existing semi-supervised methods often struggle to handle the inherent challenges of medical images, including low-contrast lesions, subtle boundaries, and inter-class similarities. Approaches focusing solely on data-level perturbations [15] or network-level regularizations [16] fail to capture sufficient contextual information for accurate segmentation. Similarly, strategies such as dual consistency learning [17] and dense contrastive learning overlook broader long-range dependencies, limiting their effectiveness in modeling complex tissue structures [18]. Consequently, effectively combining local precision with global contextual awareness remains an open challenge in SSMIS [16,19,20].

To address these limitations, we propose a novel fused transformer-based hybrid model designed for accurate semi-supervised segmentation. The framework integrates a Fused Transformer Block (FTB) and a Deformable Convolution Block (DCB) into a dual-branch backbone. The DCB employs a hierarchical CNN structure to capture fine spatial hierarchies and segment small lesions and irregular medical artifacts effectively [21,22]. The FTB incorporates a fused attention (FA) mechanism that combines Cross-Channel Attention and Cross-Dimensional Attention to efficiently model long-range dependencies. By processing the image as non-overlapping patches, the FTB captures detailed local features and enhances cross-patch interactions through its FA design. To improve computational efficiency, we embed a Ghost Layer Perceptron (GLP) [23] into the FTB, which reduces parameters and FLOPs without sacrificing representational capability. Furthermore, the proposed framework jointly leverages labeled and unlabeled data through a combination of supervised segmentation loss, unsupervised contrastive loss, and consistency regularization, enabling robust learning even with limited labeled samples.

Extensive experiments and ablation studies demonstrate that the proposed model achieves state-of-the-art performance on multiple benchmark datasets. Our main contributions are summarized as follows:

• We propose a novel fused attention (FA)-based hybrid model that integrates a dual-branch backbone combining a Deformable Convolution Block (DCB) and a Fused Transformer Block (FTB) for semi-supervised medical image segmentation.

• The DCB is designed to capture fine spatial details and small lesions, improving segmentation accuracy in challenging regions.

• The FTB employs a fused attention mechanism combining Cross-Channel Attention and Cross-Dimensional Attention to model long-range dependencies across channels and spatial dimensions effectively.

• We incorporate a joint optimization strategy using supervised loss, consistency loss, and contrastive loss to exploit both labeled and unlabeled data, improving overall generalization while reducing the need for extensive annotations.

The rest of this paper is organized as follows: Section 2 reviews related work, including CNN-based approaches, transformer-based methods, and contrastive learning strategies. Section 3 presents the proposed methodology, detailing the integration of the DCB and FTB. Section 4 outlines the experimental setup, datasets, and evaluation metrics. Section 5 presents the quantitative and qualitative results, along with ablation studies and computational analysis. Finally, Section 6 concludes the paper and discusses future research directions.

2. Related works

SSMIS has evolved through the integration of advanced methodologies aimed at leveraging unlabeled data in the face of scarce labeled datasets. Traditional methods often fall short when handling complex medical imaging scenarios characterized by low contrast and high noise, relying too heavily on shallow, manually designed features. In contrast, modern approaches using deep learning enhance feature representation and modeling capabilities, with encoder–decoder frameworks and CNN-transformer hybrids achieving substantial improvements in efficiency [24–28]. In this section, we provide an overview of well-established methods in SSMIS, organized into three main categories: CNN-based approaches, Transformer-based approaches, and contrastive learning-based methods. Firstly, we analyze CNN-based techniques, which have been fundamental in developing early feature extraction strategies for medical imaging. These methods excel at processing spatial details but often struggle with capturing comprehensive contextual information over longer distances. Next, we investigate transformer-based approaches, which improve upon CNNs by utilizing self-attention mechanisms to more effectively integrate global contextual details into the segmentation process. Finally, we explore the role of contrastive learning in using unlabeled data to improve robustness and accuracy of models. This overview sets the context for our proposed model, a FA-based hybrid architecture that synthesizes these approaches to address specific challenges in SSMIS more effectively.

2.1. CNN-based approaches

Learning-based approaches have seen significant advancements in SSMIS. The Mean Teacher (MT) model [29] is a leading approach in contrastive and consistency learning, effectively adapting to data variations by gradually incorporating the weights of the student model [30]. Initially, the MT model uses supervised learning on labeled data, followed by the teacher model providing pseudo-labels for unlabeled data. Regularization strategies ensure consistency between the teacher and student models’ predictions, enhancing the student model through a combination of supervision and consistency loss data. Advancements in this field include methods to improve predictions on unlabeled data and reduce overfitting [11,31,32]. Notable contributions include Li et al.’s TCSM_v2 [32], Chen et al.’s Cross Pseudo Supervision [11], and Wang et al.’s uncertainty-aware framework within the MT model structure (UA-MT) [33]. Additionally, Wu et al.’s Mutual Consistency Network (MC-Net) [3], Luo et al.’s Dual-Task Consistency (DTC) [15], and Basak et al.’s Perturbed and Strict Mean Teacher (PS-MT) framework [34] have further advanced the field by introducing new methods for improving segmentation performance and reliability. However, these methods still face limitations, such as the reliance on high-quality pseudo-labels which can perpetuate errors if the initial annotations are inaccurate or if the model misinterprets complex medical images. Moreover, maintaining the balance between stability and plasticity in models that require iterative updating of weights can be challenging, potentially leading to inconsistent performance across different datasets or under different conditions.

Recent evidential and multi-branch CNN-based frameworks such as ETC-Net [35] further enhance pseudo-label reliability through evidential uncertainty modeling and tri-branch cross-supervision. Although effective, these approaches introduce substantial architectural and computational complexity, making them less scalable for high-resolution medical images. Similarly, deformation-resilient multigranularity learning models [36] improve robustness to spatial misalignment but rely on multimodal data and heavy alignment modules that do not naturally extend to semi-supervised medical segmentation scenarios.

2.2. Transformer-based approaches

Transformer-based approaches have also made significant strides in medical image segmentation. Originally developed for natural language processing (NLP) [37], transformer architectures have been adapted for computer vision (CV) applications by incorporating Patch Embedding to handle long-range dependencies between image patches [12]. Various lightweight transformer models have been proposed to address the substantial computational resources required by standard transformers [38,39]. The Vision Transformer (ViT) and subsequent models like Transformer in Transformer (TNT) [40], DETR [41], and SETR [42] have showcased the effectiveness of transformers in CV tasks, including object detection and semantic segmentation. In medical imaging, TransUNet [43] pioneered the integration of ViT within a segmentation framework, blending CNNs with transformers for effective global and local feature extraction. Further advancements include the development of SAMed [44], which leverages transformer technology for multi-organ segmentation, and models incorporating transformer-based modules in skip connections or as auxiliary branches to enrich global information [45]. Innovations such as Gated Axial-Attention [46] and Adaptive Gated Multi-head Attention [47] have improved long-range dependency modeling, while the introduction of the Swin Transformer [48] and its applications in models like Swin UNet [49] and SwinPA-Net [50] have demonstrated superior performance in medical image segmentation by effectively balancing computational efficiency and capturing multi-scale feature extraction capabilities. Despite these advancements, current transformer-based models [47,50–52] in SSMIS still face significant limitations. Many models do not adequately address the challenge of extracting subtle, small-scale features in medical images, which are crucial for accurate diagnosis and treatment planning. Additionally, while transformers handle long-range dependencies, their integration often does not optimally balance the computational load, which can be prohibitive in clinical settings. Moreover, existing models have limited ability to effectively leverage unlabeled data, missing opportunities to enhance learning through unsupervised techniques.

Recent prototypical and cross-image transformer-driven frameworks such as SCP-Net [53] and SCCL [54] enhance semantic consistency through self- and cross-image prototype interactions. However, their dependence on early-stage pseudo-label quality and batch composition can propagate semantic drift, and their prototype-centric mechanisms often overlook fine-grained boundary precision crucial in medical segmentation. Diffusion-based global representation models like I4D [55] demonstrate strong distribution modeling but are not tailored for pixel-wise prediction or semi-supervised segmentation, limiting their relevance to dense medical imaging tasks.

2.3. Contrastive learning-based approaches

Contrastive learning has become a potent approach that is extensively used in self-supervised learning to explore discriminative feature representations without the need for annotations. The main concept of contrastive learning is to acquire separate feature representations by increasing the separation between various classes in the feature space, enabling the differentiation of positive and negative pairings. To retain a large number of negative samples, existing approaches usually use a memory branch updated by momentum or keep a sizable mini-batch [1]. The success of contrastive learning in image-level tasks has motivated its application to dense prediction tasks such as image segmentation and depth estimation, where the challenge lies in making detailed predictions for every pixel or voxel in the image, thereby requiring highly discriminative and contextually relevant features for accurate analysis. This adaptability shows promise in enhancing the performance of semi-supervised learning frameworks in medical imaging, where annotated examples are limited. For instance, Wang et al. [33] modified image-level contrastive learning to the voxel level to improve segmentation performance. Pixel contrastive learning has been broadly adopted for semi-supervised segmentation beyond full supervision [4,20]. Methods like the uncertainty-aware Mean Teacher (UA-MT) [33], the Perturbed and Strict Mean Teacher (PS-MT) [34], and Zhong et al.’s approach integrating confidence sampling with consistency training [20] have further advanced the field by focusing on task-level consistency and reducing overfitting. These advancements demonstrate the transformative potential of deep learning and contrastive learning techniques in enhancing the accuracy and efficiency of SSMIS. Despite these advancements, several limitations persist. Many existing contrastive learning approaches in SSMIS fail to fully exploit the spatial and contextual relationships critical for accurate segmentation in medical images. Additionally, the computational intensity of managing large mini-batches or memory banks poses scalability challenges, particularly in resource-constrained settings. Moreover, while these methods improve model robustness, they often do not sufficiently address the integration of unsupervised data, potentially leading to underutilized discriminative feature learning from unlabeled datasets.

More recent advances incorporate multi-domain contrastive learning, such as MAC [56], which optimizes inter-domain mutual information to improve feature quality from unlabeled data. However, these models are primarily developed for 1D signal representation and lack spatial structural awareness needed for dense segmentation. Other hierarchical feature optimization methods such as TFA-HFO [57] enhance feature separability in time–frequency representations but are not designed for pixel-wise medical image prediction. Compared with these approaches, our model integrates consistency learning with lightweight contrastive regularization specifically tailored to preserve spatial structure and handle unlabeled medical data efficiently.

3. Proposed method

The proposed model for SSMIS integrates a dual-parallel branch architecture, incorporating the DCB and the FTB. This configuration is designed to improve segmentation accuracy by processing both labeled and unlabeled data simultaneously. The network utilizes a head to reduce a supervised error, ensuring accurate segmentation predictions that align with ground truth. Additionally, it utilizes a contrastive loss using unlabeled data, which encourages the model to acquire distinctive characteristics by distinguishing between similar and dissimilar features. The model also employs a consistency loss to enhance its stability and generalizability by utilizing both labeled and unlabeled data.

3.1. Network architecture

Fig. 1 provides a detailed overview of the proposed model architecture. It consists of a dual-parallel branch structure comprising a sequences of DCB and FTB. The DCB branch employs to enhance segmentation accuracy for irregular medical patterns by refining feature maps and capturing the complex spatial details that are characteristic of medical images. It uses a sequence of operations: Deformation Convolution (DC), Batch Normalization (BatchNorm), Gaussian Error Linear Unit (GELU), and Max-Pooling (MP). The FTB branch treats images as sequences of patches, employing FA to incorporate long-range dependencies across channels and dimensions. The FTB employs a series of LayerNorm, FA, and GLP layers concurrently. The FTB branch considers images as series of patches and utilizes FA to incorporate long-range dependencies across channels and dimensions, thereby enhancing the differentiation between features and background. By incorporating a GLP into the FTB branch, the computational complexity of the model is significantly reduced. Feature merging (FM) techniques are used to concatenate features from both the DCB and FTB branches, which are then fed back into the DCB and FTB, ensuring comprehensive feature integration and effective segmentation. The network employs a head to minimize supervised loss for accurate segmentation aligned with ground truth, and optimizes a contrastive loss using unlabeled data to distinguish between similar and dissimilar features. A consistency loss ensures alignment between the outputs of both branches, enhancing robustness and generalization. The total loss optimized by the network is a weighted sum of the supervised, contrastive, and consistency losses.

Fig. 1.

Overview of the proposed model for semi-supervised medical image segmentation. It features a dual-parallel branch architecture consisting of a Deformation Convolution Block (DCB) and a Fused Transformer Block (FTB). The DCB refines feature maps for precise segmentation of irregular medical patterns, while the FTB, using Fused Attention (FA), captures long-range dependencies. Here US represent the upsampling. The model employs a head to accurate segmentation predictions. It optimizes supervised and contrastive losses to improve segmentation accuracy and feature differentiation, with a consistency loss to ensure robustness and generalization. The FM and FE depict the feature merging and feature expanding, respectively.

3.2. Deformation convolution block

The Deformation Convolution Block (DCB) is employed to address the limitations of standard convolutional methods. Convolutional-based methods struggle to accurately capture features from small regions or irregularly tissue shapes, particularly along their boundaries because of their fixed receptive fields. While deformable convolutions [21] and dynamic convolutions [22] have made strides in allowing for flexible receptive fields, they often do not achieve an optimal balance between network efficiency and architectural compactness—a balance that is crucial for the high accuracy required in medical image segmentation tasks. In order to tackle these difficulties, the DCB is developed by dynamically adjusting the weights and offsets of convolutional kernels during the training process, thereby enhancing the model’s ability to adapt to complex and irregular features characteristic in medical images.

Fig. 2(a) illustrates the DCB as a sequence of operations applied in this branch. It consists of a DC, BatchNorm, GELU [58], and a MP layer. Following the DC layer, a BatchNorm step is applied to normalize the features, reducing internal covariate shift and improving training stability and speed. After normalization, the GELU activation function is employed to incorporate nonlinearity into the model, thereby enhancing its ability to comprehend and learn intricate functions. Finally, an MP layer decreases the spatial dimensions of the feature map that reduce the computational burden for subsequent layers and mitigates overfitting by providing a abstracted representation of the input features.

Fig. 2.

Overview of the proposed DCB and FTB: (a) DCB branch with Deformation Convolution (DC), Batch Normalization, Gaussian Error Linear Unit (GELU), and Max-Pooling (MP) layers; (b) FTB branch with Layer Normalization, Fused Attention (FA), and Ghost Layer Perceptron (GLP).

Let consider the input feature map $I(H\times W\times C)$, and the typical square convolutional kernel is defined as $K$. When this kernel is applied to $I$, the standard convolution at position ${\psi }_o=(x,y)$ in the output feature map $\mathit{\text{OM}}$ is defined as:

$$\mathit{\text{OM}}\left({\psi }_o\right)=\sum _{i=-1}^1\sum _{j=-1}^{1}K(i,j)\cdot I\left({\psi }_o+i,j\right),$$ (1)

where ${\psi }_o$ represents a coordinate in the 2D space of the feature map, and $i,j$ iterate over the spatial dimensions of kernel $K$. To adapt to varying shapes and textures in medical images, offsets $\Delta {\psi }_{ij}=(\Delta i,\Delta j)$ are introduced to the kernel positions, resulting in a deformable convolution defined by:

$$\mathit{\text{OM}}\left({\psi }_o\right)=\sum _{i=-1}^1\sum _{j=-1}^{1}K(i,j)\cdot I\left({\psi }_o+i+\Delta i,j+\Delta j\right),$$ (2)

where $\Delta {\psi }_{ij}$ represents the learned offsets that allow the convolutional kernel to adjust dynamically to the input features. This mechanism ensures that the kernel can better conform to spatial variations, essential for accurately delineating complex structures in medical images. For an enhanced convolution operation, dynamic weighting coefficients ${\beta }_{ij}$ are applied to modify the influence of each kernel weight based on the input data. It is a set of learnable parameters that allows the network to place more or less emphasis on certain features based on their relevance to the task at hand, further refining the segmentation process. It can be particularly beneficial for handling the complex and varied features found in medical images. The expression for DC incorporating dynamic weighting is given by:

$$O{M}^{\prime }\left({\psi }_o\right)=\sum _{i=-1}^1\sum _{j=-1}^1{\beta }_{ij}\cdot K(i,j)\cdot I\left({\psi }_o+i+\Delta i,j+\Delta j\right),$$ (3)

where $\mathit{\text{OM}}\prime$ indicates the dynamically enhanced output feature map of DC. The coefficients ${\beta }_{ij}$ are computed by an auxiliary network that assesses the relative importance of each convolutional kernel element, thereby optimizing the convolutional operation for specific feature enhancements in medical imaging.

Unlike standard deformable convolutions [21], which only learn spatial offsets $\Delta {\psi }_{ij}$ to adjust the sampling locations, our proposed DCB introduces an additional dynamic reweighting mechanism through the learnable coefficients ${\beta }_{ij}$. These coefficients are predicted by a lightweight auxiliary gating network that analyzes the context of the input feature map $I$. Specifically, we first apply global average pooling to extract a compact context vector $z\in {\mathbb{R}}^C$ from the input feature map. Then, $z$ is passed through a two-layer GLP with GELU activation and sigmoid normalization (4):

$$u=\sigma \left(W_2\cdot \delta \left(W_{1}z\right)\right),$$ (4)

where $\delta (\cdot )$ denotes the GELU activation and $\sigma (\cdot )$ is the sigmoid function. The resulting vector $u\in {\mathbb{R}}^{K^2}$ is reshaped into $\beta \in {\mathbb{R}}^{K\times K}$ and integrated into Eq. (3) to adaptively scale the kernel weights. This dual adaptation mechanism—jointly learning the spatial offsets $\Delta {\psi }_{ij}$ and the dynamic kernel weights ${\beta }_{ij}$—enables DCB to better capture small lesions, irregular tissue boundaries, and complex spatial hierarchies, which are often missed by conventional deformable convolution. In addition, the auxiliary network is extremely lightweight, introducing fewer than 0.1M additional parameters for $K=3$ and negligible inference overhead, thereby maintaining architectural compactness while improving segmentation performance.

Furthermore, the auxiliary gating network is tightly integrated with the main DCB branch and is trained jointly with the entire network in an end-to-end manner. By dynamically predicting the weighting coefficients ${\beta }_{ij}$ for each kernel element, the auxiliary network effectively modulates the contribution of each sampled feature in Eq. (3). This mechanism ensures that the DCB focuses more on highly relevant spatial patterns while suppressing less informative regions, thereby enhancing the segmentation of small lesions and complex anatomical boundaries without introducing significant computational overhead.

3.3. Fused transformer block

Fig. 2(b) illustrates the proposed FTB, which enhances feature differentiation between regions of interest and complex backgrounds and capitalizes fully on rich contextual information. The FTB consists of Layer Normalization followed by a Fused Attention (FA) mechanism for capturing complex dependencies and fusing long-range dependencies across channels and dimensions. Subsequently, another Layer Normalization is applied, followed by a residual connection, all integrated within a GLP to improve computational efficiency.

3.3.1. Macro-level workflow: Patch partitioning and shifted windows

To provide a clearer understanding of how feature maps are prepared before entering the fused-attention mechanism, we present a macro-level overview of the FTB. This overview illustrates the sequence of operations applied to the input feature map prior to the computation occurring within the four fused-attention branches.

Given an input feature map $X\in {\mathbb{R}}^{H\times W\times C}$, the FTB first performs a non-overlapping patch partition that divides the feature map into patches of size $P\times P$. Each patch is then passed through a linear embedding layer, which transforms the spatial region into a compact token representation suitable for transformer processing. To further enrich local contextual modeling while maintaining computational efficiency, we employ a shifted-window mechanism inspired by Swin Transformers [48] within each embedded patch. Specifically, windows of size $s\times s$ are extracted in both standard and spatially shifted configurations, where the shift is $\lfloor s/2\rfloor$ pixels along each axis. This staggered partitioning enables information exchange across neighboring windows without increasing the quadratic cost typically associated with global self-attention.

The resulting embedded and shifted-window tokens are then forwarded to the four parallel branches of the FTB – two CCA branches and two CDA branches – where long-range dependencies and detailed structural cues are jointly modeled. The updated Fig. 3 visualizes this macro-level workflow, including patch generation, linear embedding, shifted-window grouping, and the subsequent delivery of tokens into the FTB for fused attention computation.

Fig. 3.

Macro view of the FTB. The input feature map is first divided into non-overlapping patches and embedded into tokens, which are processed with regular and shifted windows to capture local and cross-window context. The FTB module then model long-range dependencies and refine features. The internal FA architecture is detailed in Fig. 4.

3.3.2. Fused attention

Fig. 4 depicts the FA within the FTB branch, specifically designed to enhance medical imaging analysis by addressing challenges such as low contrast, prevalent noise, and focus on small objects-areas where traditional self-attention mechanisms [12] often struggle. The figure illustrates the process of applying both cross-channel and cross-dimensional attention across four parallel branches to manage complex feature interactions effectively. Each branch handles a different aspect of the feature interactions: two focus on the relationships within and across the channel dimensions, and the other two on spatial interactions.

Fig. 4.

Overview of the Fused Attention (FA) mechanism used within the FTB. This mechanism integrates cross-channel and cross-dimensional attention processes across four parallel branches to handle different aspects of feature interactions. The input image, divided into patches, undergoes attention calculations to extract long-range dependencies. The final output feature is a weighted sum of branch outputs, optimized during training.

The input image $I$ is initially divided into $n\times n$ patches. Each patch is of size $h\times w$. This division allows the model to focus on specific regions of the image, reducing the complexity by limiting the scope of initial processing to smaller, manageable areas of the whole image. Within each patch, further refinement of the attention mechanism is achieved through the use of shifted windows, which divide each patch into smaller windows of size $s\times s$, where $s$ is typically set to 7. To elucidate the computational efficiencies of our proposed FA mechanism, we compare it with the Multihead Self-Attention (MSA) from Vision Transformers (ViT) [12] and Shifted Window-based MSA (SMSA) from Swin Transformers [48]. The computational complexities of these mechanisms are formulated as follows:

$$\begin{gathered} \text{MSA}(\text{ViT})=4N{c}^2+2(N{)}^{2}c, \\ \text{SMSA}(\text{Swin})=4N{c}^2+2s^{2}Nc, \\ \text{FA}(\text{Ours})=\frac{N{c}^2}{4}+s^{2}Nc, \end{gathered}$$

where $N$ indicates the total number of windows within each patch, $s\times s$ is the size of each window, and $c$ is the number of channels. This represents the computational load, highlighting the reduction in complexity achieved by our FA mechanism, particularly in processing channel dimensions. To understand the computational complexities more deeply, we consider $D$ as the dimensionality of the feature space and $N$, the number of tokens, equivalent to the number of windows. The complexities can be expressed as:

$$\begin{gathered} \text{MSA}\left(\text{ViT}\right)=O\left(D^2\cdot N+N^2\cdot D\right), \\ \text{FA}(\text{Ours})=O\left(\frac{D^2}{4}\cdot N+s^2\cdot D\cdot N\right). \end{gathered}$$

Our FA mechanism integrates into the FTB and operates through four distinct branches. Each branch is specialized: two focus on channel-wise and spatial relationships through a mechanism we refer to as Cross Channel Attention (CCA). The other two branches handle the Cross Dimensional Attention (CDA), exploring the interactions between the channel dimension $c$ and the spatial dimensions $h$ and $w$. The CCA and CDA mechanisms are calculated as follows by Eqs. (5) and (6):

$${\text{Attention}}_{\text{cca}}(q,k,v)=\text{SoftMax}\left(\frac{q{k}^T}{\sqrt{c/8}}+B_{\text{b}1}\right)v,$$ (5)

$${\text{Attention}}_{\text{cda}}(q,k,v)=\text{SoftMax}\left(\frac{q{k}^T}{\sqrt{h\times w/8}}+B_{\text{b}2}\right)v,$$ (6)

where $B_{\text{b}1}$, and $B_{\text{b}2}$ are the relative position bias matrices for the channel-wise and spatial branches, and $q,k$, and $v$ are the query, key, and value matrices adapted to their specific branches. This approach allows the FA to dynamically adapt to the most informative features, which is crucial for precise semi-supervised segmentation tasks.

Given an input feature map $X\in {\mathbb{R}}^{h\times w\times c}$, we compute the query $\left(q\right)$, key ($k$), and value ($v$) matrices separately for the CCA and CDA branches. For the CCA branch, $X$ is reshaped into $X_c\in {\mathbb{R}}^{c\times (h\cdot w)}$ and projected to:

$$q=X_c{W}_q^c,k=X_c{W}_k^c,v=X_c{W}_v^c,$$

where $W_q^c,W_k^c,W_v^c\in {\mathbb{R}}^{(h\cdot w)\times d_c}$ are learnable weight matrices. These matrices are used in Eq. (5).

For the CDA branch, $X$ is reshaped into $X_s\in {\mathbb{R}}^{(h\cdot w)\times C}$ and similarly projected to:

$$q=X_s{W}_q^s,k=X_s{W}_k^s,v=X_s{W}_v^s,$$

where $W_q^s,W_k^s,W_v^s\in {\mathbb{R}}^{C\times d_s}$. These matrices are then used in Eq. (6) to compute the CDA outputs. This explicit formulation clarifies how $q,k$, and $v$ are generated and used in the attention computations.

Algorithm 1:

FA in the FTB: QKV generation and fusion

Input: Feature map $X\in {ℝ}^{B\times H\times W\times C}$
Output: Fused output $Y$
Reshape for branches
$X_c\leftarrow \text{reshape}(X,[B,C,H\cdot W])$;	// CCA: channel-wise
view
$X_s\leftarrow \text{reshape}(X,[B,H\cdot W,C])$;	// CDA: spatial view
Linear projections (QKV)
$q\leftarrow X_c{W}_q^c,k\leftarrow X_c{W}_k^c,v\leftarrow X_c{W}_v^c$;	// CCA branch
$q^{\prime }\leftarrow X_s{W}_q^s,k^{\prime }\leftarrow X_s{W}_k^s,v^{\prime }\leftarrow X_s{W}_v^s$;	// CDA branch
Attention per branch (uses Eqs. (5), (6))
${\text{Attn}}_{\text{cca}}\leftarrow \text{SoftMax}\left(\frac{q{k}^{\top }}{\sqrt{c/8}}+B_{\text{b}1}\right)v$
${\text{Attn}}_{\text{cda}}\leftarrow \text{SoftMax}\left(\frac{q^{\prime }{k^{\prime }}^{\top }}{\sqrt{(H\cdot W)/8}}+B_{\text{b}2}\right)v^{\prime }$
Fuse heads and project
$Y\leftarrow \text{Linear}\left(\text{Concat}\left({\text{Attn}}_{\text{cca}}^{\left(1\right)},{\text{Attn}}_{\text{cca}}^{\left(2\right)},{\text{Attn}}_{\text{cda}}^{\left(1\right)},{\text{Attn}}_{\text{cda}}^{\left(2\right)}\right)\right)$
return $Y$

3.3.3. Ghost Layer Perceptron (GLP)

The GLP is introduced to enhance computational efficiency while maintaining high representational capacity within the Fused Transformer Block (FTB). The idea is inspired by GhostNet [23], which observes that many intermediate features in deep neural networks are highly redundant. Instead of generating all features through costly full-rank transformations, GLP creates intrinsic feature maps via standard linear projections and then generates complementary “ghost” feature maps using inexpensive operations such as depthwise linear filters.

Given an input feature tensor $X\in {\mathbb{R}}^{N\times d}$, the GLP splits the output generation into two parts:

$$Y=\left[X{W}_{\text{intrinsic}},g\left(X{W}_{\text{intrinsic}}\right)\right],$$

where $W_{\text{intrinsic}}\in {\mathbb{R}}^{d\times d/2}$ is a reduced linear projection and $g(\cdot )$ denotes a set of cheap linear operations that expand the intrinsic features into ghost features. The concatenation of the intrinsic and ghost features forms the final GLP output.

In our framework, the GLP is applied after concatenating the outputs of the CCA and CDA branches. This reduces the number of parameters and multiplications compared to a full linear projection, thereby lowering computational overhead. At the same time, the generated ghost features improve the diversity of the learned representations, which enhances segmentation accuracy while maintaining lightweight model complexity.

The detailed process within each FTB unfolds through two primary operations:

$$\begin{gathered} {\widehat{A}}^l=\text{FA}\left(\text{LayerNorm}\left(A^{l-1}\right)\right)+A^{l-1}, \\ {A}^l=\text{GLP}(\text{LayerNorm}({\widehat{A}}^l))+{\widehat{A}}^l, \end{gathered}$$

where ${\hat{A}}^l$ represents the intermediate features post-FA, and $A^l$ denotes the subsequent output features post GLP operation. The final output feature of the FTB is a composite of the outputs from each branch, computed as a weighted sum:

$$\text{Output}=f_1\cdot L{P}_1+f_2\cdot L{P}_2+f_3\cdot L{P}_3+f_4\cdot L{P}_4,$$ (7)

where the $f_1,f_2,f_3$, and $f_4$ refer to the four branch’s output that contains $h*w*c$. During the back-propagation phase, these outputs alongside learnable parameters $L{P}_1,L{P}_2,L{P}_3$, and $L{P}_4$, are adjusted during training, allowing the network to adaptively focus on the most relevant spatial and channel features for optimizing the network’s performance.

3.4. Loss functions

We further refine the feature learning process by employing a combination of supervised and unsupervised loss functions. The proposed model effectively leverages unlabeled data by utilizes a contrastive loss [59]. This loss encourages the model to differentiate between similar and dissimilar features within the images, fostering a deeper understanding of distinctive characteristics. Specifically, each image $I$, processed through the FTB, is divided into $n\times n$ patches. Each patch is independently passed through the network to generate local feature maps $l_i$, while the entire image generates a global feature map $g$. The $l_i$ and the $g$ are projected into a lower-dimensional space using a projection module $\varphi$, which is designed to facilitate effective feature comparison:

$$\varphi \left(g\right),\varphi \left(l_i\right)\text{for}\text{each}i.$$

Contrastive loss is then computed to align each local feature map with the global feature map, treating the corresponding parts as positive pairs and all non-corresponding combinations as negative pairs:

$$L_{\text{contrastive}}(i)=-\text{log}\left(\frac{\text{exp}\left(\varphi (g{)}_i^{\top }\varphi \left(l_i\right)/\tau \right)}{\sum _{k=1}^{n^2}\text{exp}\left(\varphi (g{)}_i^{\top }\varphi \left(l_k\right)/\tau \right)}\right),$$ (8)

where $\tau$ is a temperature parameter that controls the separation of positive and negative pairs in the embedding space. Additionally, a consistency loss is employed to enhance the alignment between the local and global predictions, ensuring that the segmentation is consistent across different scales of the image:

$$L_{\text{consistency}}=\sum _i\text{MSE}\left(\psi \left(l_i\right),\psi (g{)}_i\right),$$ (9)

where $\psi$ denotes a function that maps features to segmentation predictions. And, MSE represents the mean square error. For the labeled parts of the dataset, the model uses a supervised loss that combines Dice loss and cross-entropy loss, effectively capturing the essential segmentation accuracy while penalizing classification errors:

$$L_{\text{supervised}}=\frac{1}{2}\left(\text{Dice}\left(\psi \left(g\right),Y\right)+\text{CE}\left(\psi \left(g\right),Y\right)\right),$$ (10)

where $Y$ denotes the ground truth labels. The total loss function integrates these individual losses, weighted by parameters $\lambda$ and $\gamma$, which are tuned to balance the influence of each component:

$$L_{\text{total}}=L_{\text{supervised}}+\lambda L_{\text{contrastive}}+\gamma L_{\text{consistency}}\text{.}$$ (11)

The training process is implemented in two discrete stages, each with its specialized focus and objectives. During the initial stage, the network’s backbone is trained using a combination of the supervised loss and the dense cross-level contrastive loss, with hyperparameters $\lambda$ and $\gamma$ set to 1 and 0, respectively, prioritizing contrastive learning dynamics. In the second stage of training, the model leverages the consistency loss to deepen the exploitation of cross-level relationships inherent in the image data, thereby bolstering its proficiency in capturing and utilizing nuanced partial information. In this stage, the hyperparameters shift to $\lambda =0$ and $\gamma =1$, channeling the learning emphasis towards internal consistency.

4. Experiments

4.1. Datasets and preprocessing

Our investigation employs four distinguished public datasets, namely Kvasir-SEG [63], Cell Nuclei [64], BraTS [65], and ISIC [66], to evaluate the performance of our method. The Kvasir-SEG dataset [63] includes a collection of 1000 white-light colonoscopy images, each meticulously annotated at the pixel level to serve as a reliable benchmark for segmentation algorithms. The ISIC dataset [66] features 2594 dermatological images of skin lesions, offering a challenging array of data for testing segmentation accuracy due to the diversity of lesion shapes and sizes. Delving into cellular structures, the Cell Nuclei dataset [64] encompasses both the DSB and MoNuSeg collections; the former includes 670 images from a variety of imaging modalities, such as brightfield and fluorescence, with targets that are notably difficult to delineate, while the latter provides 30 H& E stained histopathologic images in its training set and 14 in the test set, hailing from multiple low contrast between the nuclei and surrounding tissue. Lastly, the BraTS dataset [65] contributes a robust set of 462 multimodal MRI scans from patients with glioblastoma (GBM) and lower-grade glioma (LGG), complete with expertly verified ground truth labels, posing a significant test for the segmentation of complex brain tumors in varying stages and grades. Each dataset presents unique challenges, from subtle contrasts and diverse modalities to complex structures, ensuring a comprehensive assessment of our segmentation approach across varied medical imaging scenarios. Within the semi-supervised settings, we allocated 90% of the datasets for training purposes and reserved the remaining 10% for testing. Within the training subset, 20% of the data were utilized as labeled samples, whilst the vast majority, 80%, were employed as unlabeled. To enhance the robustness of our proposed model and mitigate overfitting, we applied augmentation techniques to the training dataset such as rotating the images by 15 degrees to both the left and right, along with applying horizontal flips.

4.2. Experimental details

We developed the methods using Python and the PyTorch framework, with NVIDIA RTX 3080 GPUs facilitating both training and evaluation phases. For training our network, we employed the AdamW optimizer with a weight decay set at 0.1, a learning rate of 0.0001, and a batch size of eight. We conducted the training over 50 epochs, during which all images were resized to a resolution of 256 × 256. To ensure fair and unbiased comparisons across all methods – both proposed and baseline approaches – we standardized several key aspects of our study. These include the combination of semi-supervised and supervised training losses, the method of dataset partitioning, and the duration of training, which was consistent across all evaluated methods. We utilized pre-trained weights [67], as initial values for our model’s and also the baseline models’ weights. This initial use of pre-trained weights provided a robust starting point, enhancing the learning efficiency and effectiveness the models. The evaluation of all methods involved key metrics such as mean absolute error (MAE), mean intersection over union (mIoU), and the Dice coefficient for the foreground. To further validate our findings, results were averaged over three different sets of randomly selected labeled images. For each of these sets, we calculated and reported the corresponding means and standard deviations.

5. Results and discussions

5.1. Quantitative comparison

Table 1 provides a detailed quantitative comparison of semi-supervised segmentation models evaluated on four benchmark datasets – ISIC, Kvasir-SEG, Cell Nuclei, and BraTS – using three common metrics: Mean Absolute Error (MAE), Dice Score (DS), and mean Intersection over Union (mIoU). Across most datasets, the proposed model demonstrates consistently strong performance and often leads in key metrics. On the ISIC dataset, it attains the highest mIoU of 76.89 ± 1.58 and a competitive DS of 85.02±2.16, confirming precise lesion boundary delineation. For Kvasir-SEG, it achieves the lowest MAE (5.19 ± 0.81) and the highest DS (76.54 ± 3.60), closely followed by Su et al. [61] and Miao et al. [62], which report slightly higher mIoU values but at greater computational cost. On Cell Nuclei images, our model obtains both the lowest MAE (3.65 ± 1.10) and the highest mIoU (64.11 ± 4.18), demonstrating its robustness in capturing fine-grained cell boundaries.

Table 1.

Comparative performance analysis of semi-supervised medical image segmentation models on four datasets: ISIC, Kvasir-SEG, Cell Nuclei, and BraTS. The table highlights key metrics including Mean Absolute Error (MAE), Dice Score (DC), and mean Intersection over Union (mIoU).

Approaches	Datasets
	ISIC			Kvasir-SEG			Cell Nuclei			BraTS
	MAE	DC	mIoU	MAE	DC	mIoU	MAE	DC	mIoU	MAE	DC	mIoU
Proposed model	4.37 ± 0.53	85.02 ± 2.16	76.89 ± 1.58	5.19 ± 0.81	76.54 ± 3.60	66.01 ± 1.37	3.65 ± 1.10	79.58 ± 2.46	64.11 ± 4.18	6.78 ± 0.77	71.24 ± 2.89	59.12 ± 4.18
Hu et al. [60]	4.69 ± 0.71	84.99 ± 2.65	76.58 ± 1.49	5.85 ± 1.10	76.02 ± 3.91	65.91 ± 1.95	3.98 ± 1.81	79.62 ± 2.76	64.03 ± 4.25	7.46 ± 1.36	71.21 ± 3.37	59.19 ± 4.26
Su et al. [61]	4.62 ± 0.95	85.33 ± 3.10	76.78 ± 1.83	5.66 ± 1.01	76.36 ± 3.81	66.15 ± 1.80	3.75 ± 1.50	79.70 ± 2.66	64.19 ± 4.89	6.95 ± 0.89	72.05 ± 3.18	60.38 ± 4.06
Miao et al. [62]	4.27 ± 0.88	85.01 ± 2.44	76.83 ± 1.58	5.52 ± 0.89	76.47 ± 3.65	66.26 ± 1.53	3.71 ± 1.23	79.73 ± 2.69	64.16 ± 4.10	6.81 ± 0.81	71.58 ± 2.48	60.79 ± 4.54
Luo et al. [15]	4.85 ± 0.93	85.26 ± 3.35	76.61 ± 1.91	6.02 ± 1.08	76.24 ± 3.89	66.13 ± 2.12	3.79 ± 1.85	79.61 ± 2.93	63.98 ± 4.40	6.82 ± 1.13	70.59 ± 3.26	58.76 ± 4.61
Zhu et al. [16]	5.92 ± 1.24	83.43 ± 3.32	75.19 ± 1.82	5.95 ± 0.97	76.01 ± 2.69	65.52 ± 1.46	3.97 ± 1.58	78.63 ± 2.49	63.47 ± 4.88	6.87 ± 1.35	71.29 ± 3.56	58.88 ± 4.95
Chaitanya et al. [19]	6.56 ± 1.88	83.01 ± 4.21	74.38 ± 2.08	7.58 ± 1.33	75.03 ± 3.80	65.49 ± 1.94	5.37 ± 2.22	78.55 ± 2.18	61.42 ± 5.16	7.16 ± 2.19	70.23 ± 2.58	58.14 ± 3.95
Zhong et al. [20]	8.29 ± 1.95	77.98 ± 3.30	68.44 ± 2.47	8.36 ± 3.98	70.19 ± 4.42	59.10 ± 1.79	7.14 ± 2.06	72.98 ± 3.27	56.76 ± 4.95	9.16 ± 3.74	66.87 ± 2.92	54.48 ± 3.67
Aralikatti et al. [17]	12.84 ± 2.08	70.12 ± 2.51	60.89 ± 3.95	13.22 ± 3.27	64.54 ± 3.62	53.52 ± 1.90	10.87 ± 3.49	66.19 ± 4.10	52.94 ± 3.34	13.94 ± 2.48	61.32 ± 4.28	48.13 ± 5.76
Wu et al. [18]	8.45 ± 1.85	81.27 ± 4.29	73.13 ± 3.19	8.87 ± 2.58	68.35 ± 3.59	64.38 ± 3.76	6.58 ± 1.97	77.74 ± 4.28	57.28 ± 5.18	9.08 ± 3.49	70.56 ± 3.71	55.75 ± 4.53

For BraTS, it records a DS of 71.24 ± 2.89 and an mIoU of 59.12 ± 4.18, comparable to or exceeding those of the latest competitors. Compared with new baselines such as Hu et al. [60], Su et al. [61], and Miao et al. [62], as well as established methods including Luo et al. [15], Zhu et al. [16], Chaitanya et al. [19], Zhong et al. [20], Aralikatti et al. [17], and Wu et al. [18], the proposed approach achieves superior or near-state-of-the-art accuracy across all datasets while maintaining computational efficiency. These results verify the model’s strong generalization capability across different imaging modalities and tasks, underscoring its effectiveness for real-world semi-supervised medical image segmentation.

5.2. Qualitative comparison

Fig. 5 presents a comprehensive qualitative comparison of semi-supervised segmentation results on four medical image datasets: ISIC for skin lesions, Kvasir-SEG for gastrointestinal lesions, Cell Nuclei for cellular analysis, and BraTS for brain tumors. The proposed model consistently demonstrates superior segmentation performance, closely aligning with the ground truth across various medical imaging modalities. In the ISIC dataset, the proposed model accurately replicates lesion contours, showing high fidelity in segmentation precision. For the Kvasir-SEG dataset, it provides near-perfect lesion representations, effectively handling the complexity of gastrointestinal images. In the Cell Nuclei dataset, the model excels in accurately identifying cell boundaries, highlighting its robustness in cellular segmentation tasks. The BraTS dataset results further emphasize the model’s effectiveness, with precise tumor segmentation that closely matches the ground truth. Compared to six established methods—Luo et al. [15], Zhu et al. [16], Chaitanya et al. [19], Zhong et al. [20], Aralikatti et al. [17], and Wu et al. [18]—the proposed model consistently outperforms in terms of segmentation accuracy and detail. While methods like those of Aralikatti et al. show competitive segmentation, others, such as Wu et al. struggle with over-segmentation and noise. Overall, the proposed model stands out for its consistency, reliability, and precision in segmentation, which are essential for clinical applications, ensuring accurate and actionable medical imaging analysis.

Fig. 5.

Illustration of segmentation performance across four medical image datasets (ISIC, Kvasir-SEG, Cell Nuclei, and BraTS) comparing the proposed semi-supervised model against six established methods. The figure shows the original images, ground truth segmentations, and results from the proposed model, along with those from Luo et al. Zhu et al. Chaitanya et al. Zhong et al. Aralikatti et al. and Wu et al. The proposed model consistently illustrates the superior performance of the proposed model, which closely matches the ground truth and effectively handles complex medical images.

5.3. Computation cost

Table 2 compares the computational costs of the proposed model with several recent semi-supervised segmentation methods in terms of parameters, training time (TT), FLOPs, and inference time (IT). The proposed model achieves an optimal balance between representational power and computational efficiency, requiring only 18.91M parameters, 39.8G FLOPs, and the shortest training time of 5.6 h. It also achieves the fastest inference speed of 28.7 ms per image, demonstrating its suitability for real-time or near-real-time medical applications. Although Luo et al. [15] has a slightly smaller parameter count (18.88M), it incurs higher FLOPs (42.9G) and slower inference (28.9 ms/img). Other recent methods, such as Zhu et al. [16], Chaitanya et al. [19], Zhong et al. [20], and Aralikatti et al. [17], exhibit substantially larger model sizes (26M–71M parameters), longer training times (8.3–21.6 h), and slower inference speeds (32–46 ms/img). Similarly, Wu et al. [18] requires 34.34M parameters, 9.1 h training, and 34.2 ms/img inference time. These results collectively demonstrate that our model achieves the most favorable trade-off between accuracy, speed, and computational cost, confirming its efficiency and practicality for real-world semi-supervised medical image segmentation tasks.

Table 2.

Comparison of computational costs for recent semi-supervised segmentation models, including the proposed model. The table compares the number of parameters (in millions), training time (TT) (in hours), and FLOPs (in billions) and inference time (IT) (in milliseconds per image).

Approach	Computation cost
	#Parm (M)	TT (h)	FLOPs (G)	IT (ms)
Proposed model	18.91	5.64	39.8	28.7
Hu et al. [60]	26.10	14.52	48.7	31.4
Su et al. [61]	28.60	32.21	52.1	35.8
Miao et al. [62]	26.10	18.70	48.7	33.5
Luo et al. [15]	18.88	5.93	42.9	28.9
Zhu et al. [16]	26.26	8.31	55.1	32.6
Chaitanya et al. [19]	34.65	9.70	61.3	36.7
Zhong et al. [20]	56.87	16.48	69.5	41.2
Aralikatti et al. [17]	71.06	21.57	73.8	45.8
Wu et al. [18]	34.34	9.10	50.1	34.2

5.4. Ablation study

Table 3 presents a comprehensive ablation study conducted on the ISIC Dataset to assess the impact of various architectural components on the performance of our SSMIS model. The study evaluates different configurations involving the DCB, FT with FA, Multihead Self-Attention (MSA) [12], and Shifted Window-based Self-Attention (SMSA) [48] across the model’s dual parallel backbone, analyzing their effects on MAE, DS, mIoU, and computational cost. From the computational perspective, the ablation study outlined meticulously evaluates the effects of various architectural components. Case 1, featuring the integration of the DCB with FA and GLP, demonstrates the highest segmentation accuracy, registering an MAE of 4.37, a DS of 85.02, and an mIoU of 76.89, which is indicative of the model’s robustness and efficiency. In contrast, Case 2 substitutes DCB with UNet and observes an increased MAE of 6.75 and reduced DS and mIoU, suggesting a decrease in segmentation precision. Cases 3 through 11 explore various configurations by selectively incorporating features like FT, ViT, SwinT, and different self-attention mechanisms (MSA, SMSA). For instance, Case 6 introduces SMSA within the FTB branch, leading to a slight improvement in DS and mIoU compared to Case 5, which employs MSA. Notably, the full inclusion of advanced attention mechanisms, as seen in Cases 7 and 8 which incorporate ViT and SwinT respectively, leads to a noticeable reduction in DS and mIoU. This pattern is also seen in Cases 10 and 11, where the addition of a UNet to a simpler configuration does not compensate for the reduction in performance metrics, underlining that the complexity added by advanced modules does not necessarily translate to better segmentation accuracy.

Table 3.

Comprehensive ablation study of the proposed model on the ISIC dataset.

Case	DCB branch			FTB branch									MAE	DS	mIoU	Computation cost
	DCB	UNet	FT	FT					ViT	SwinT	DCB	UNet				#Param (M)	TT (h)
				FA	MSA	SMSA	GLP	MLP
1	✓	✕	✕	✓	✕	✕	✓	✕	✕	✕	✕	✕	4.37	85.02	76.89	18.91	5.6
2	✕	✓	✕	✓	✕	✕	✓	✕	✕	✕	✕	✕	6.75	80.31	71.34	37.16	8.9
3	✕	✕	✓	✓	✕	✕	✓	✕	✕	✕	✕	✕	5.46	82.97	72.25	23.74	6.5
4	✓	✕	✕	✓	✕	✕	✕	✓	✕	✕	✕	✕	5.79	83.12	73.04	22.05	6.3
5	✓	✕	✕	✕	✓	✕	✓	✕	✕	✕	✕	✕	6.07	81.00	71.92	24.31	6.6
6	✓	✕	✕	✕	✕	✓	✓	✕	✕	✕	✕	✕	5.33	83.16	72.57	23.46	6.1
7	✓	✕	✕	✕	✕	✕	✕	✕	✓	✕	✕	✕	7.68	78.08	69.34	45.89	11.2
8	✓	✕	✕	✕	✕	✕	✕	✕	✕	✓	✕	✕	7.01	79.26	70.63	39.58	10.0
9	✓	✕	✕	✕	✕	✕	✕	✕	✕	✕	✓	✕	5.64	83.03	72.75	17.47	5.4
10	✓	✕	✕	✕	✕	✕	✕	✕	✕	✕	✕	✓	8.16	77.88	68.42	38.56	9.0
11	✕	✓	✕	✕	✕	✕	✕	✕	✕	✕	✕	✓	8.56	76.28	67.94	49.97	12.8

From the computational perspective, the ablation study reveals significant variations across different cases. The optimal configuration (Case 1) not only achieves superior segmentation accuracy but also maintains a moderate computational demand, with 18.91 million parameters and a training time of 5.6 h. On the other hand, configurations that incorporate more complex transformer-based architectures such as ViT and SwinT (Cases 7 and 8) lead to a substantial increase in both the number of parameters and training time—45.89 million parameters with 11.2 h and 39.58 million parameters with 10.0 h, respectively. This significant resource usage does not yield proportional improvements in segmentation performance, illustrating a diminishing return on computational investment. Furthermore, Cases involving simple configurations like Case 9, which uses DCB alone, show minimal computational cost with 17.47 million parameters and 5.4 h of training time but at the expense of reduced performance metrics. This comprehensive analysis of computational costs across different configurations highlights the need to balance architectural complexity with computational efficiency, ensuring that any increase in resource demand is justifiable by notable enhancements in model performance. This careful consideration is critical for practical applications, particularly in scenarios where computational resources are a limiting factor. This ablation study demonstrates that while sophisticated modules can enhance segmentation, their computational cost must be justified by significant performance improvements, thus highlighting the practical application and efficiency of the proposed model in medical image segmentation.

6. Conclusions and future works

This work introduces a semi-supervised medical image segmentation (SSMIS) framework that integrates a dual-branch architecture coupling a DCB with a FTB. The DCB enhances sensitivity to intricate spatial structures and irregular boundaries, while the FTB employs a fused attention (FA) mechanism – implemented via four parallel branches spanning channel and spatial dimensions – and a GLP to capture long-range dependencies with reduced computational cost. The training objective combines consistency regularization with unsupervised contrastive learning to leverage unlabeled data effectively. Extensive experiments and ablations on four public datasets indicate that the proposed model attains comparable or better accuracy than recent state-of-the-art methods while maintaining lower FLOPs, shorter training time, and faster inference, highlighting its practicality for clinical workflows.

The proposed model demonstrates effectiveness in SSIM. However, it is constrained by its dependence on high-quality training data, which could limit its performance when dealing with lower-quality images. Furthermore, despite attempts to decrease computational requirements, there are still obstacles to overcome in order to achieve real-time processing, which is crucial for clinical application. Future technical advancements will prioritize enhancing the model’s ability to withstand changes in image quality and further refining computational efficiency for real-time analysis. Furthermore, expanding the variety of datasets to incorporate real-world clinical data will be essential for validating and improving the model’s flexibility and performance in a variety of medical imaging scenarios.

Data availability

https://github.com/NabaviLab/SSIM.git.