QCNN-Swin-UNet: Quantum Convolutional Neural Network Integrated with Optimized Swin-UNet for Efficient Liver Tumor Segmentation and Classification on Edge Devices

Abstract

Accurate segmentation and classification of liver tumors are crucial for early diagnosis and effective treatment planning. However, conventional deep learning models such as tumor heterogeneity, class imbalance, and high computational demands face challenges, limiting their clinical deployment. This study introduces a lightweight hybrid framework combining an optimized Swin-UNet for segmentation with a Quantum Convolutional Neural Network (QCNN) for classification. The Swin-UNet is enhanced using a metaheuristic Search and Rescue (SAR) algorithm and a quadratic penalty-based objective function to balance compactness and accuracy. A Focal AUC loss function addresses class imbalance and improves sensitivity to minority regions. The QCNN leverages quantum-inspired mechanisms such as entanglement and superposition to achieve superior performance with reduced parameters. Evaluated on three benchmark datasets (3D-IRCADb, LiTS17, and MSD Task03), the framework achieves Dice scores of 85.8%, 88.7%, and 88.4%, respectively, alongside 96.7% classification accuracy. The model size is reduced to 64.16 MB, enabling real-time inference on edge devices (Jetson Nano). The QCNN classifier outperforms traditional CNNs in all metrics, demonstrating its effectiveness in high-dimensional medical data analysis. This work bridges the gap between diagnostic precision and computational efficiency, presenting a clinically viable AI solution for liver tumor analysis.

Introduction

Liver cancer remains a critical global health challenge, ranking among the most prevalent malignancies and a leading contributor to cancer-related mortality. According to recent epidemiological data, it is the sixth most commonly diagnosed cancer worldwide, trailing breast, lung, colorectal, prostate, and gastric cancers, while holding the third position in cancer-related deaths. In 2020 alone, over 905,700 new cases and 830,200 fatalities were reported globally [1]. Hepatocellular carcinoma (HCC), the predominant subtype, constitutes nearly 90% of all liver cancer cases [2]. Medical imaging techniques such as ultrasound, elastography, MRI, and CT scans play a pivotal role in liver cancer detection. Among these, CT imaging offers high-resolution visualization of internal structures, yet it faces significant challenges in accurately identifying liver abnormalities [3]. Variability in liver morphology, overlapping intensity distributions between tumors and healthy tissues, and poorly defined lesion boundaries complicate manual segmentation. These factors render radiologists’ annotations prone to inter-observer variability and subjectivity, underscoring the critical need for robust automated segmentation solutions.

Computer-aided diagnosis (CAD) systems, powered by deep learning, have revolutionized tumor detection, classification, and segmentation, enhanced diagnostic precision, and reduced clinician workload [4–8]. Convolutional Neural Networks (CNNs), particularly Fully Convolutional Networks (FCNs) and U-Net architectures, have demonstrated remarkable success in pixel-wise tumor segmentation [9]. However, these models demand substantial computational resources, limiting their deployment on edge devices with constrained processing capabilities. Reliance on large-scale servers introduces additional challenges, including bandwidth constraints, data security risks, high operational costs, and significant carbon footprints due to energy-intensive computations. Consequently, there is a pressing need for lightweight, resource-efficient models tailored for edge deployment.

Early approaches to liver tumor segmentation relied on traditional image processing methods such as thresholding [10], Canny edge detection [11], and watershed algorithms [12]. While these techniques leverage intensity gradients and boundary detection, they struggle with the inherent complexity of medical images—irregular tumor shapes, heterogeneous textures, and low contrast between lesions and surrounding tissues often lead to segmentation inaccuracies [13]. Semi-automated interventions further exacerbate inefficiencies, requiring extensive expert input.

Unsupervised learning methods, including clustering and edge-based algorithms, emerged to address these limitations. For instance, multi-scale Laplacian of Gaussian (LoG) filters [14] and generalized LoG techniques [15] have been applied to nuclear and tumor detection. However, these methods remain sensitive to noise, require meticulous parameter tuning, and often fail to delineate low-contrast boundaries effectively [16]. Hybrid approaches integrating unsupervised learning with region-based models like Active Contour Models (ACM) [17] or marker-based watershed transforms [18] show improved accuracy but suffer from computational inefficiencies and limited generalizability.

CNNs, FCNs, and U-Net variants dominate the field of medical image analysis due to their ability to learn hierarchical features from complex medical images. Innovations like hybrid residual networks optimized via Genetic Algorithms (GAs) [19], 3D U-Net with superpixel classification [20], and phase-attention networks (PA-Net) [21] have further advanced segmentation performance. Despite their efficacy, CNNs are inherently constrained by localized receptive fields, limiting their ability to model long-range dependencies critical for global context understanding [22]. Transformers, originally designed for sequence modeling, have recently gained traction in computer vision tasks such as classification [23], object detection [24], and semantic segmentation [25]. Their self-attention mechanisms enable large receptive fields and robust long-range dependency modeling. Hybrid CNN-Transformer architectures, including TransUNet [26] and DA-Tran [27], have been proposed for medical image segmentation, combining the strengths of both paradigms.

The rapid evolution of digital imaging technologies has led to an unprecedented generation of visual data across critical domains, including medical diagnostics, autonomous systems, and security infrastructure. In medical imaging, particularly, the accurate interpretation of complex anatomical structures faces significant challenges due to inherent biological variability and imaging artifacts. Traditional Convolutional Neural Networks (CNNs) have demonstrated remarkable success in image analysis by hierarchically extracting features through convolutional and pooling operations. However, as medical datasets expand in size and complexity, these classical architectures increasingly strain against the computational limits of conventional hardware. This computational bottleneck has spurred interest in quantum-enhanced solutions [28]. Quantum computing leverages fundamental principles of superposition and entanglement to achieve theoretically exponential speedups for specific problem classes. In quantum machine learning, qubits—capable of existing in multiple states simultaneously—enable novel approaches to information processing. Quantum Convolutional Neural Networks (QCNNs) adapt these advantages to visual pattern recognition, employing parameterized quantum circuits that may offer more efficient feature extraction compared to classical counterparts. Recent advances in superconducting and trapped-ion qubit technologies have further improved the feasibility of implementing such quantum algorithms.

The application of quantum-enhanced learning to medical image analysis presents particularly compelling opportunities. In liver tumor classification, where traditional CNNs must process subtle texture variations across poorly contrasted regions, QCNNs may provide superior performance through enhanced parallel processing of high-dimensional image features, more efficient optimization of complex decision boundaries, and reduced parameter overhead through quantum state compression. This approach is particularly valuable in liver tumor segmentation, where variability in tumor morphology and imaging artifacts pose significant challenges. Figure 1 depicts the challenges faced in segmenting the liver and tumors in CT scans.

Fig. 1.

highlights the complexities involved in liver and tumor segmentation, including overlap with abdominal organs, intensity variations, and tumor heterogeneity

Although current models achieve high accuracy, they demand considerable memory and processing power, rendering them impractical for edge devices such as Jetson Nano. Typically running on server systems, these models necessitate the transmission of sensitive patient data over the internet, thereby posing privacy and security risks. Moreover, the high energy consumption of servers restricts their viability in settings with limited resources. This issue is currently being actively explored across various fields: recent studies in predictive model optimization have shown that customized loss functions, which balance accuracy with hardware limitations, can facilitate deployment on edge devices. Drawing on these advancements, there is a pressing need to optimize segmentation models to decrease their size and power consumption while maintaining accuracy, thus enabling real-time, secure, and energy-efficient tumor segmentation and classification on edge devices [29]. This work investigates the integration of quantum-enhanced classification with advanced segmentation architectures, aiming to establish a new paradigm for comprehensive tumor analysis that balances diagnostic accuracy with computational efficiency.

Proposed Methodology

The methodology integrates the Swin-UNet, merging the Swin Transformer and U-Net for medical image segmentation. The Swin Transformer captures both local and global features, which are critical for segmentation accuracy. The Search and Rescue (SAR) algorithm is employed to adjust hyperparameters, minimizing an objective function that balances accuracy and model size. The objective function optimizes the Area Under the Curve (AUC) for classification performance and model size for edge deployment. AUC focal loss prioritizes hard-to-classify samples, thereby improving segmentation robustness. Mathematical formulations underpin the optimization process, justifying its effectiveness.

Quantum Convolutional Neural Networks (QCNNs) offer a fresh, innovative approach that uses quantum mechanics to enhance image classification. QCNNs break through the computational barriers of traditional CNNs, enabling more efficient processing of high-dimensional data. To enhance efficiency, CNN and QCNN models were carefully developed and fine-tuned with a focus on batch size and epoch adjustments. Figure 2 illustrates the integrated pipeline for the proposed model, highlighting the key components and their interactions.

Fig. 2.

Schematic of the Integrated Pipeline for Automated Liver Tumor Segmentation and Classification

Swin-UNet and Hyperparameter Optimization

The Swin-UNet [30] model incorporates the hierarchical structure and shifted window attention mechanism of the Swin Transformer into the encoder-decoder framework of the U-Net, as depicted in Fig. 3. This subsection provides an overview of the Swin-UNet architecture and elaborates on the hyperparameters that significantly influence model performance and memory efficiency. Fig. 4.

Fig. 3.

Block diagram of Swin-Unet architecture

Fig. 4.

Block diagram of SAR-based hyperparameter optimization and training with AUC focal loss

Multi-Head Self-Attention (MHSA) in Swin-UNet

The Swin-UNet architecture employs a localized window-based self-attention mechanism, differing from the global self-attention used in conventional Vision Transformers (ViTs). By partitioning the input into non-overlapping windows, the model reduces computational complexity while maintaining effective feature learning.

To enable cross-window communication, the Swin Transformer introduces a shifted windowing strategy, mathematically formulated as:

$$ShiftedWindo{w}_i=Windo{w}_i+\mathrm{\Delta }$$ 1

where Δ denotes the positional offset applied to the window boundaries. This approach facilitates inter-window dependency modeling, significantly improving the model’s ability to capture long-range contextual relationships. Consequently, Swin-UNet achieves enhanced segmentation accuracy for liver tumor detection in medical imaging.

Hyperparameter Space and Constraints

The efficiency and performance of the Swin-UNet model are influenced by several key hyperparameters, each subject to specific constraints. Table 1 summarizes these hyperparameters along with their optimization ranges:

Table 1.

Hyperparameter constraints for the swin-UNet model and AUC focal loss

Hyperparameter	Symbol	Range
Depth	D	(2, 8)
Initial filter number	$F_0$	(32, 256)
Patch size	P	(2, 16)
Number of attention heads	H	(1, 16)
Window size	W	(1, 8)
MLP size	M	(32, 512)
AUC focal loss alpha	α	(0, 5)
AUC focal loss gamma	γ	(0, 5)

These hyperparameters govern diverse facets of the model, including its depth, capacity, and the dimensions of the input data it handles. Fine-tuning these parameters within their designated ranges guarantees a harmonious equi-librium between the model's segmentation precision and its memory consumption.

Search and Rescue (SAR) Algorithm

The Search and Rescue (SAR) algorithm [31] It is a metaheuristic optimization method designed to balance global exploration and local refinement in complex search spaces. Inspired by the strategic coordination of real-world rescue operations, SAR efficiently navigates high-dimensional optimization landscapes by dynamically prioritizing promising regions while mitigating convergence to suboptimal local minima.

In the context of deep learning, hyperparameter selection plays a pivotal role in model performance. SAR addresses this challenge by systematically exploring expansive hyperparameter spaces, optimizing configurations to maximize accuracy while maintaining computational efficiency. The algorithm’s ability to adaptively adjust its search strategy makes it particularly well-suited for optimizing the intricate balance between model accuracy and computational resources, which is crucial for deploying deep learning models in resource-constrained environments.

In this study, we apply the SAR algorithm to hyperparameter optimization in medical image segmentation for the first time. Specifically, SAR is employed to fine-tune the Swin-UNet model for liver tumor segmentation, with the dual objectives of achieving high accuracy and computational efficiency. Figure 3 delineates the SAR-based optimization workflow, highlighting the systematic approach taken to optimize the model's hyperparameters.

• SAR Optimization Phases

  The Search and Rescue (SAR) algorithm operates in three key phases: Initialization, Search, and Rescue. These phases integrate both global search strategies and local refinement to ensure the algorithm explores the parameter space widely and then fine-tunes the promising regions. The following sections provide a detailed breakdown of each phase, followed by the mathematical formulations that guide the optimization process.

• Initialization Phase

  The algorithm begins by generating an initial population of candidate solutions ${\theta }_i$, where each solution represents a potential set of hyperparameters. These candidates are randomly sampled within predefined bounds, informed by prior knowledge of the parameter space. The initialization phase plays a critical role in setting the starting point of the optimization, and the quality of the initial candidates can significantly impact the subsequent search process.

  The fitness of each candidate solution is evaluated using a fitness function that considers two key aspects: model performance (measured by AUC) and model size (which affects computational efficiency). The fitness function is formulated as follows:

  $$\text{Fitness}\left({\theta }_i\right)=\text{AUC}\left({\theta }_i\right)\times \text{exp}\left(-,ϵ,\times ,\mathit{log},\left(M,o,d,e,l,S,i,z,e,\left({\theta }_i\right)\right)\right)$$ 2

  where ${\theta }_i$ represents a set of hyperparameters for candidate solution i, $ϵ$ is a scaling factor that controls the influence of model size, and AUC denotes the area under the receiver operating characteristic (ROC) curve. This formulation ensures that the fitness function rewards configurations that achieve high accuracy while keeping the model size manageable.

• Search Phase

  Once the initial population is generated and evaluated, the SAR algorithm enters the Search phase. During this phase, each candidate solution is assessed based on its fitness score, and the algorithm iteratively updates the positions of the solutions. The goal is to explore the search space and identify the most promising regions, based on the fitness function. The search process is carried out using the following formula for updating the positions of each candidate:

  $${\theta }_{\text{new},i}={\theta }_{\text{current},i}+\alpha \times \left({\theta }_{\text{best neighbor}},-,{\theta }_{\text{current},i}\right)$$ 3

  where $\alpha$ is a learning rate parameter and ${\theta }_{\text{best neighbor}}$ refers to the solution with the highest fitness score among the neighboring solutions. By iteratively updating the position of each candidate solution, SAR ensures that the search is directed towards regions of the search space that hold the potential for better performance.

• Rescue Phase

  After the Search Phase, the algorithm proceeds to the Rescue phase, where further refinement of the solutions is performed. During this phase, the candidate solutions are adjusted based on their proximity to the best solutions found during the Search Phase. The goal of the Rescue phase is to focus the search on the most promising regions, fine-tuning the hyperparameters to improve model performance. To adjust the candidate solutions, the algorithm uses two strategies. First, if a suitable neighboring solution is found, the candidate solution is updated by moving it towards the best-performing neighbor. The adjustment is calculated as given in Eq. (10). If no suitable neighbors are found, the candidate solution is adjusted towards the globally best solution identified during the Search phase. The update for this adjustment is given by:

  $${\theta }_{\text{new},i}={\theta }_{\text{current},i}+\beta \times \left({\theta }_{\text{global best}},-,{\theta }_{\text{current},i}\right)$$ 4

  where $\alpha$ and $\beta$are scaling factors that control the magnitude of the adjustments. These strategies ensure that the algorithm refines the solutions towards the most optimal configurations while maintaining a balance between exploration and exploitation. The iterative process of the Rescue Phase continues until the algorithm converges, yielding an optimal or near-optimal set of hyperparameters that balances both model performance (AUC) and model size, ensuring efficient training and accurate predictions.

AUC Focal Loss for Class Imbalance

Liver tumor segmentation faces a class imbalance between tumor and healthy pixels. This work employs AUC focal loss to prioritize minority class learning by modulating easy and hard example contributions. The loss function focuses on difficult tumor pixels:

$$\text{Loss}=-\alpha (1-\text{AUC}{)}^{\gamma }\text{log}\left(A,U,C\right)$$ 5

Here, $\alpha$ weights the minority class and $\gamma$ adjusts focus on hard examples. This work optimizes α ∈ [0,5] and γ ∈ [0,5] to handle imbalance effectively. Combining SAR-based hyperparameter optimization with AUC focal loss enables efficient Swin-UNet models for accurate segmentation on edge devices. Algorithm 1 provides implementation details.

Algorithm 1: Search and Rescue (SAR) Algorithm for Swin-UNet Optimization

Optimality Analysis of the Objective Function with SAR

The objective function in this study (Eq. 9) depends on hyperparameters θi, which include both discrete and continuous variables. This function balances model accuracy (AUC) with model size, where ϵ is a positive constant governing the trade-off. Since θi contains mixed variable types, optimization occurs in a hybrid space, meaning traditional convexity concepts are only relevant in the continuous subspace.

• Convexity in the Continuous Subspace:

  When the discrete components of θi are fixed, the analysis reduces to the continuous portion θi^(c), where standard convexity principles apply.

  $$f\left({\theta }_i^{\left(c\right)}\right)=x\times y^{-ϵ}$$ 6

  where x = AUC and y = Model Size.

  Examining convexity in this continuous setting demonstrates that the Hessian matrix H for this function is defined as

  $$H=\left[\begin{array}{cc}0& -ϵy^{-ϵ-1}\\ -ϵy^{-ϵ-1}& ϵx(ϵ+1)y^{-ϵ-2}\end{array}\right]$$ 7
  The Hessian matrix’s determinant can be expressed as:

  $$\text{Det}\left(H\right)={ϵ}^2{y}^{-2ϵ-2}$$ 8

  Given that ϵ > 0 and y > 0, the determinant of H (Det(H)) is non-negative. However, because the second partial derivative ∂2f/∂ × 2 equals zero, the function is not strictly convex but instead demonstrates a weaker form of convexity. This implies the presence of flat regions in the continuous domain, resulting in multiple suboptimal solutions rather than a single global optimum.

• Optimality in Discrete Domains:

  While traditional notions of convexity don’t directly extend to discrete variables, we can employ concepts from piecewise convexity and discrete optimization. For the discrete parameters θi, the objective function can be interpreted as a collection of piecewise convex functions:

  $$\text{Fitness}\left({\theta }_i^{\left(d\right)},,,{\theta }_i^{\left(c\right)}\right)=\left\{f_j,\left({\theta }_i^{\left(c\right)}\right),\mid ,{\theta }_i^{\left(d\right)},=,j\right\}$$ 9

• Optimality in Piecewise-Convex Subspaces:

  For each discrete configuration j, the objective function operates in the corresponding continuous subspace and maintains convexity, as established earlier. While the discrete domain lacks gradient information, the SAR algorithm systematically explores it by evaluating a finite set of possible configurations. Through a hybrid approach combining local search with global adaptation, SAR efficiently identifies configurations that minimize the objective function.

• Convergence Properties of SAR in Mixed Spaces:

  The SAR algorithm guarantees convergence to a suboptimal solution in mixed-variable spaces. When discrete variables are held fixed, the continuous subspace exhibits convex properties (as established in Eq. (12)), enabling efficient local optimization. For scenarios requiring discrete adjustments, SAR systematically modifies these variables while continuously reevaluating the objective function. This iterative procedure ensures convergence to either suboptimal or near-optimal solutions, supported by the following key properties:

• Continuous Subspace Local Optimality:

  For each fixed discrete configuration, the algorithm guarantees convergence to a locally optimal solution within the convex continuous subspace.

  Discrete Space Global Exploration:

  Through systematic evaluation of distinct discrete configurations, SAR achieves comprehensive exploration of the solution space.

• Theoretical Guarantees:

  While strict convexity cannot be formally established in the discrete domain, the joint effect of continuous subspace convexity and SAR’s systematic exploration strategy guarantees convergence to at least a suboptimal solution.

Quantum Convolutional Neural Networks (QCNNs) for Classification

In the domain of binary classification tasks, Quantum Convolutional Neural Networks (QCNNs) [28] strive to achieve an optimal equilibrium between computational efficiency and classification accuracy. The most notable characteristics of QCNNs are their ability to exploit quantum properties such as superposition and entanglement, which enable the network to process and classify data more efficiently compared to classical counterparts. These unique features have demonstrated excellent performance in numerous two-category classification tasks.

To train a deep learning model for liver cancer detection, regions of interest (ROI) within the medical images must be labeled. This step involves identifying tumor regions and healthy tissue. Labeling typically involves the manual annotation of regions within the image. For binary classification (benign or malignant), labels are assigned based on the presence or absence of tumors in specified regions. In this paper, we trained a two-class QCNN using a dataset comprising labeled regions of interest.

Figure 5 illustrates the selection of training samples. The positive sample is extracted from the tumor region, while the negative one is from healthy liver parenchyma. Specifically, a large number of training image blocks of size 11 × 11, including positive and negative samples, are first selected from tumor and normal liver regions, respectively, to train a two-class (tumor/non-tumor) QCNN. During the training phase, the labeled dataset was used, where each input image (X) is associated with a class label (y) indicating whether it is healthy or cancerous.

Fig. 5.

Illustration of positive samples extracted from tumor areas and negative samples obtained from healthy liver tissue

• Comparison of CNN and QCNN Architectures

  To provide a comprehensive understanding of the differences between traditional Convolutional Neural Networks (CNNs) and Quantum Convolutional Neural Networks (QCNNs), we conducted a detailed comparison focusing on their feature extraction and classification processes.

• CNN Architecture

  In this study, Fig. 6 illustrates a CNN architecture developed to classify images of identifying tumor regions and healthy tissue, divided into two primary stages: feature extraction and classification. In the feature extraction stage, the input image passes through several convolutional layers that identify patterns, beginning with simple edges and shapes, and moving toward more complex structures. Each convolutional layer is followed by a pooling layer, which condenses the data, preserving essential features while reducing the spatial dimensions. After feature extraction, the data enters the classification stage, where a fully connected layer synthesizes all the gathered features to make the final prediction. This layer combines the extracted information to classify the image as either normal or abnormal. By gradually capturing and interpreting patterns, this CNN effectively identifies the subtle differences between these two categories and provides an accurate binary classification.

  The CNN architecture featured two convolutional layers with ReLU activations and max-pooling, followed by a flattening layer and two fully connected layers. The final layer used a sigmoid activation function for binary classification. Training was optimized with the Adam optimizer and binary cross-entropy loss, experimenting with batch sizes and epochs while monitoring validation loss and accuracy.

• QCNN Architecture

  In contrast to CNN architectures, the QCNN architecture shown in Fig. 7 connects each layer in sequence to perform hierarchical feature extraction from the input image, which is encoded across eight qubits (q0 to q7). These qubits use quantum properties like superposition and entanglement to efficiently capture complex patterns in the data. The architecture begins with convolutional layers (purple blocks), where parameterized quantum gates are applied to pairs of neighboring qubits. Each convolutional layer applies a two-qubit ansatz uniformly across all qubit pairs, allowing the network to consistently detect local features throughout the image. After each convolutional layer, a pooling layer (orange circles) reduces the number of active qubits, simplifying the data through down-sampling. This process preserves essential features and maintains consistency by using the same ansatz across the pooling layer. Each layer feeds directly into the next, enabling the network to progressively build upon previously extracted features and move from simple to more complex patterns. In the final stage, a measurement operation collapses the quantum state into a classical binary output, classifying the image as normal or abnormal. This sequential structure allows the QCNN to distill essential features effectively, providing a potentially more efficient approach to image classification with reduced computational demands compared to classical CNN.

  With the QCNN models, we refined the quantum circuits and optimized qubit utilization to enhance processing efficiency. Throughout training, we monitored various batch sizes and epochs to ensure model stability and reliability. A specialized quantum convolutional layer (QConv) was developed for the QCNN using TensorFlow Quantum and Cirq to incorporate quantum computing techniques into the neural network. As illustrated in Fig. 8, the QConv layer operates on a 2 × 2 qubit grid at coordinates (0, 0), (0, 1), (1, 0), and (1, 1). Rx rotations parameterized by input features (X00, X01, X02, X03) are applied to each qubit. Controlled-Z (CZ) and Controlled-X (CX) gates introduce entanglement, with adjustable parameters optimized during training. A sigmoid activation function adds non-linearity, enabling the QCNN to identify complex patterns. This approach combines quantum computing’s unique strengths with neural networks to enhance learning and classification performance.

Fig. 6.

Structural diagram of the convolutional neural network (CNN)

Fig. 7.

Schematic representation of the Quantum Convolutional Neural Network (QCNN) architecture

Fig. 8.

Architecture of the QCNN circuit

Results and Analysis

Dataset Description

In this work, we utilize three publicly available datasets—3DIRCADb [32], LiTS [33], and MSD Task03_Liver [34]—to evaluate the performance of the proposed liver tumor segmentation and classification framework. These datasets provide a diverse set of CT volumes with varying anatomical and pathological characteristics, ensuring robust and comprehensive assessment.

• 3DIRCADb Dataset

  The 3DIRCADb dataset comprises 20 abdominal CT volumes, collected from different patients, with liver tumors present in approximately 75% of the cases. These volumes are high-resolution and offer detailed structural information, supporting both normal and abnormal liver anatomy segmentation tasks. The dataset is particularly useful due to its manually annotated lesions and variability in tumor appearance.

• LiTS17 Dataset:

  The LiTS dataset includes 201 contrast-enhanced 3D abdominal CT volumes. The training subset contains 131 CT volumes with manual annotations provided by clinical experts, among which 118 cases contain liver tumors. The remaining 70 CT volumes form the test set, for which the segmentation labels are not publicly released. The dataset was introduced as part of the Liver Tumor Segmentation Challenge in conjunction with MICCAI 2017 and ISBI 2017. All volumes are standardized and curated for algorithmic benchmarking.

• MSD Task03 Dataset

  The MSD Task03_Liver dataset, part of the Medical Segmentation Decathlon challenge, includes 131 contrast-enhanced CT volumes annotated for both liver and tumor regions. All images were acquired from patients with liver pathology and preprocessed to ensure consistency in voxel spacing and anatomical orientation. The dataset is well-suited for multi-organ segmentation tasks due to its diverse acquisition parameters.

  To enhance the model’s focus on relevant anatomical structures and suppress irrelevant background noise, we truncate voxel intensities of all volumes across datasets to the range (− 200,250) HU. Table 2 summarizes the in-plane resolution and slice thickness of each dataset.

Table 2.

Spatial resolution of LiTS, 3DIRCADb, and MSD Task03_Liver datasets

Dataset	In-plane resolution	Pixel spacing (mm)	Slice thickness (mm)
LiTS	512 × 512	0.55–1.00	0.45–6.00
3DIRCADb	512 × 512	0.56–0.87	1.00–4.00
MSD Task03_Liver	512 × 512	0.50–1.00	0.45–6.00

Preprocessing

Preprocessing plays a critical role in expanding the training dataset and improving model generalization. We apply a series of image transformations—including rotation (± 15°), horizontal/vertical flipping, scaling, gamma correction, and logarithmic scaling—to artificially diversify the training samples. Crucially, all transformations maintain precise spatial alignment between the augmented images and their corresponding segmentation masks.

This augmentation strategy generates multiple variations of each training example, effectively increasing the dataset size and enabling the model to learn more robust features. For rotation transformations, images are randomly rotated within a range of ± 15 degrees while preserving mask integrity through coordinated geometric adjustments. The rotation transformation is mathematically represented as:

$$\left[\begin{array}{c}x_{}^{_{}^{\prime }}\\ y_{}^{_{}^{\prime }}\end{array}\right]=\left[\begin{array}{cc}\text{cos}(\theta )& -\text{sin}(\theta )\\ \text{sin}(\theta )& \text{cos}(\theta )\end{array}\right]\left[\begin{array}{c}x\\ y\end{array}\right]$$ 10

The original coordinates (x, y) are transformed to rotated coordinates (x', y') through this operation, which is applied synchronously to both images and their corresponding masks to maintain spatial correspondence. A Horizontal Flip Uses:

$$\left(x_{}^{_{}^{\prime }},,,y_{}^{_{}^{\prime }}\right)=\left(-,x,,,y\right)$$ 11

A vertical flip uses:

$$\left(x,_{}^{\prime },,,y,_{}^{\prime }\right)=\left(x,,,-,y\right)$$ 12

The scaling transformation simulates zoom effects by resizing images with a random scaling factor of ± 10%, implemented through the coordinate transformation:

$$\left(x,_{}^{\prime },,,y,_{}^{\prime }\right)=\left(s,·,x,,,s,·,y\right)$$ 13

The scale factor $s$ controls resizing magnitude, while subsequent gamma correction adjusts pixel intensities through:

$$I^{\prime }=c·I^{\gamma }$$ 14

where $I$ represents pixel intensity, and c and γ are empirically determined constants. Logarithmic scaling enhances low-contrast regions through the transformation:

$$I^{\prime }=\alpha ·\text{log}\left(1,+,I\right)$$ 15

Gamma correction and log scaling enhance image features while preserving masks. These transformations improve model generalization by increasing dataset diversity.

Training Methodology

The training phase of our study is meticulously designed to optimize the performance of the proposed hybrid deep learning framework for liver tumor segmentation and classification. The training regimen is structured to ensure robust convergence and to prevent overfitting, which could lead to suboptimal models.

The training is set at a learning rate of 0.0001, chosen for its balance between convergence speed and model stability. The process spans 2000 epochs, incorporating early stopping after 10 epochs without improvement to prevent overfitting.

A batch size of 64 is selected to optimize memory usage and convergence. The Adam optimizer is utilized for its adaptive learning rates and momentum, enhancing the training process. To tackle class imbalance, we employ a focal loss function, focusing training on challenging tumor regions to improve generalization.

Training is conducted on an Nvidia 3090 Ti GPU, ensuring efficient computation. The optimized model is deployed on a Jetson Nano device to assess performance under typical edge computing memory constraints, providing a practical evaluation for clinical deployment, as detailed in Table 3.

Table 3.

Summary of model configurations and hyperparameters

Hyperparameter	Value
Learning rate	0.0001
Epochs	2000
Batch size	64
Optimizer	Adam
Patience (early stopping)	10 epochs
Device for optimization	Nvidia 3090 Ti
Device for deployment	Jetson Nano

Performance Analysis and Comparison

Segmentation Analysis

This section provides a thorough examination of the hyperparameter tuning process for the Swin-UNet architecture, specifically tailored for liver tumor segmentation tasks using the 3DIRCADb dataset. The analysis is conducted across three experimental setups designed to evaluate the trade-offs among model complexity, segmentation performance, and model size. These setups include an unoptimized baseline model and two optimized configurations with distinct ϵ values (0.1172 and 0.2344).

The optimization process identifies effective hyperparameters to enhance the Swin-UNet’s segmentation capabilities. Table 4 details the parameters of each setup, showcasing the adjustments made to achieve optimal results.

Table 4.

Hyperparameter configurations for the Swin-UNet model across various ϵ values

Hyperparameter	Unoptimized	ϵ = 0.2344	ϵ = 0.1172
Filter number begin	128	32	32
Depth	4	4	4
Stack num down	2	2	2
Stack num up	2	2	2
Patch size	4	16	16
Number of heads	(4,8,8,8)	(4,2,8,2)	(4,1,4,2)
Window size	(4,2,2,2)	(1,1,2,2)	(8,1,2,2)
Num MLP	512	46	158
Gamma	2	2.6326	1.7471
Alpha	0.5	4.9448	4.9407

Optimized configurations outperform the baseline in segmentation accuracy and efficiency. The ϵ = 0.2344 model maximizes AUC through expanded filters and enhanced attention complexity, while ϵ = 0.1172 balances complexity and performance. This framework allows for tailored ϵ selection based on deployment needs—memory efficiency versus accuracy.

This study evaluates model performance on three benchmark datasets (3D IRCADb, LiTS, and MSD Task03). Performance is assessed using DSC, VOE, RVD, ASSD, and MSD metrics. These metrics provide a robust evaluation of segmentation accuracy. Table 5 summarizes results, highlighting configurations that balance compression and accuracy, which is particularly relevant for clinical applications where resources are limited.

Table 5.

Comparative performance analysis across model configurations and datasets

Dataset	ϵ	Size (MB)	DICE (%)	VOE (%)	RAVD	ASSD (mm)	MSD (mm)
3DIRCADb	Unoptimized	324.91	75.92 ± 27.35	32.84 ± 28.22	0.42 ± 1.83	9.6 ± 4.9	12.6 ± 7.4
3DIRCADb	0.2344	64.16	85.8 ± 21.15	22.85 ± 20.68	0.13 ± 1.81	3.4 ± 2.0	5.9 ± 3.9
3DIRCADb	0.1172	30.88	80.15 ± 22.18	29.03 ± 24.42	0.48 ± 1.77	5.1 ± 3.1	8.6 ± 5.6
LiTS	Unoptimized	324.91	78.25 ± 27.68	29.55 ± 28.44	0.32 ± 2.10	10.1 ± 5.4	13.6 ± 8.2
LiTS	0.2344	64.16	88.7 ± 16.95	18.29 ± 19.81	0.11 ± 0.44	2.4 ± 1.7	4.4 ± 3.1
LiTS	0.1172	30.88	81.30 ± 22.36	28.55 ± 24.33	0.04 ± 0.76	4.6 ± 2.7	7.6 ± 5.1
MSDTask03	Unoptimized	324.91	77.51 ± 27.93	30.48 ± 28.73	0.28 ± 1.65	10.6 ± 5.8	14.1 ± 9.0
MSDTask03	0.2344	64.16	88.4 ± 17.28	19.75 ± 21.00	0.15 ± 0.51	3.9 ± 2.4	6.9 ± 5.1
MSDTask03	0.1172	30.88	83.22 ± 20.38	25.31 ± 22.28	0.17 ± 2.04	5.4 ± 3.2	9.4 ± 6.0

• IRCADb Dataset Evaluation

  The unoptimized model on the 3DIRCADb dataset achieves a DICE score of 75.92%, a VOE of 32.84%, and an RAVD of 0.42. Optimizing the model with ϵ = 0.2344 reduces the model size by 80.25% (to 64.16 MB) while significantly improving segmentation performance, achieving a higher DICE score (85.8%), a lower VOE (22.85%), and a reduced RAVD (0.13). This demonstrates that model compression can enhance accuracy while drastically reducing storage requirements.

  Further reducing ϵ to 0.1172 yields a smaller model (30.88 MB) but leads to a drop in DICE (80.15%), an increase in VOE (29.03%), and a higher RAVD (0.48), indicating a trade-off between model size and segmentation precision. The AASD and MSD metrics also reflect this trend, with ϵ = 0.2344 providing the best balance (3.4 mm ASSD, 5.9 mm MSD) compared to the unoptimized model (9.6 mm ASSD, 12.6 mm MSD).

  These results suggest that moderate compression (ϵ = 0.2344) enhances performance while maintaining computational efficiency, whereas excessive compression (ϵ = 0.1172) degrades segmentation accuracy. The visual outcomes of segmentation depicted in Fig. 9 corroborate these observations.

• LiTS Dataset Evaluation

  For the LiTS dataset, the unoptimized model achieves a DICE score of 78.25%, a VOE of 29.55%, and an RAVD of 0.32. Applying ϵ = 0.2344 reduces the model size to 64.16 MB while improving segmentation performance (DICE: 88.7%, VOE: 18.29%, RAVD: 0.11). This configuration also exhibits the lowest ASSD (2.4 mm) and MSD (4.4 mm), indicating superior boundary delineation.

  However, further reducing ϵ to 0.1172 results in a smaller model (30.88 MB) but a decline in DICE (81.30%), an increase in VOE (28.55%), and a lower RAVD (0.04). While the RAVD improves slightly, the AASD (4.6 mm) and MSD (7.6 mm) worsen, suggesting that aggressive compression sacrifices boundary precision.

  These findings highlight that ϵ = 0.2344 offers the best trade-off, improving segmentation while maintaining a compact model size, whereas ϵ = 0.1172 leads to a noticeable performance drop. The visual segmentation results presented in Fig. 10 substantiates these findings.

• Task 03 Dataset Evaluation

  The unoptimized model on the MSDTask03 dataset achieves a DICE score of 77.51%, a VOE of 30.48%, and an RAVD of 0.28, indicating moderate segmentation performance with room for improvement. However, optimizing the model with ϵ = 0.2344 yields a remarkable enhancement in accuracy, boosting the DICE score to 88.4% while simultaneously reducing VOE to 19.75% and RAVD to 0.15. Additionally, the AASD (3.9 mm) and MSD (6.9 mm) show substantial improvements over the baseline (10.6 mm ASSD, 14.1 mm MSD), confirming superior boundary delineation.

  When compression is intensified (ϵ=0.1172), the model size shrinks further to 30.88 MB, but this comes at a noticeable cost to performance: the DICE score drops to 83.22%, VOE increases to 25.31%, and RAVD rises to 0.17. The AASD (5.4 mm) and MSD (9.4 mm) also degrade, reinforcing that excessive compression compromises segmentation precision.

  These findings demonstrate that ϵ = 0.2344 strikes the optimal balance, delivering both efficiency (64.16 MB model size) and high accuracy, whereas ϵ = 0.1172 sacrifices performance for additional compression. Visual segmentation results in Fig. 11 further validate these trends, showing that the ϵ = 0.2344 configuration produces sharper, more accurate tumor boundaries compared to both the unoptimized and aggressively compressed models.

• Configuration Optimization Selection

  The selection of the ϵ hyperparameter is crucial for optimizing the trade-off between model efficiency and segmentation performance in liver tumor analysis. The unoptimized model achieves baseline Dice scores of 75.92–78.25% across datasets but requires 324.91 MB of storage. The ϵ = 0.2344 configuration provides an optimal balance, improving Dice scores to 85.8–88.7% while reducing model size by 80.25% (64.16 MB). This configuration also demonstrates superior performance in all metrics: reducing VOE by 9–11%, improving RAVD by 0.19–0.29, and enhancing boundary accuracy (ASSD improvement of 6.2–7.7 mm, MSD improvement of 6.7–9.2 mm).

  For environments with strict storage constraints, the ϵ = 0.1172 configuration offers a 90.5% size reduction (30.88 MB) but with measurable performance trade-offs: 3–7% lower Dice scores, 3–5% higher VOE, and 2–3-mm degradation in boundary metrics compared to ϵ = 0.2344.

  • Non-constrained environments: Unoptimized model (324.91 MB).

  • Optimal Balance: ϵ = 0.2344 (64.16 MB) maintains Dice > 85%, RAVD < 0.15, and ASSD < 4 mm

  • Storage-Constrained: ϵ = 0.1172 (30.88 MB) preserves reasonable accuracy (Dice > 80%) with 90% size reduction.

  The ϵ = 0.2344 configuration is particularly noteworthy, delivering clinically significant improvements over the baseline while requiring only 20% of the original storage capacity. This configuration reduces boundary errors by 50–65% (ASSD: 2.4–3.9 mm vs 9.6–10.6 mm baseline; MSD: 4.4–6.9 mm vs 12.6–14.1 mm baseline), making it the recommended choice for most clinical applications.

Fig. 9.

Qualitative segmentation results on the 3DIRCADb dataset comparing the baseline (unoptimized) model and models trained with varying values of ϵ

Fig. 10.

Qualitative segmentation results on the. LiTS dataset comparing the baseline (unoptimized) model and models trained with varying values of ϵ

Fig. 11.

Qualitative segmentation results on the MSD Task 03 dataset comparing the baseline (unoptimized) model and models trained with varying values of ϵ

Classification Analysis

To evaluate the efficacy of Quantum Convolutional Neural Networks (QCNN) against Classical CNN in liver tumor classification, we conducted a comprehensive comparative analysis across three benchmark datasets: LiTS17, 3DIRCADb, and MSD Task-3. Table 6 summarizes the performance of both models using key metrics such as accuracy, sensitivity, precision, F1-score, AUC-ROC. The results demonstrate QCNN’s consistent superiority in classification accuracy and efficiency, highlighting its potential to address the limitations of classical approaches in medical imaging tasks.

Table 6.

Comparative analysis of QCNN vs. Classical CNN across datasets

Dataset	Model	Accuracy	Sensitivity	Precision	F1-Score	AUC-ROC
3DIRCADb	QCNN	96.8%	95.4%	96.0%	95.7%	0.973
	Classical CNN	83.2%	80.7%	82.0%	81.3%	0.863
LiTS17	QCNN	97.5%	96.2%	96.9%	96.5%	0.981
	Classical CNN	85.4%	83.1%	84.3%	83.7%	0.886
MSD Task-3	QCNN	95.9%	94.3%	95.1%	94.7%	0.965
	Classical CNN	82.7%	80.1%	81.6%	80.8%	0.858
Overall	QCNN	96.7%	95.3%	96.0%	95.6%	0.973
	Classical CNN	83.8%	81.3%	82.6%	81.9%	0.869

Specifically, on the LiTS17 dataset, the QCNN achieved an accuracy of 97.5%, sensitivity of 96.2%, precision of 96.9%, F1-score of 96.5%, and an AUC-ROC of 0.981. In contrast, the Classical CNN on the same dataset achieved an accuracy of 85.4%, sensitivity of 83.1%, precision of 84.3%, F1-score of 83.7%, and an AUC-ROC of 0.886. Similar trends were observed on the 3DIRCADb and MSD Task-3 datasets, where QCNNs consistently outperformed Classical CNNs across all metrics.

The overall performance metrics across all datasets further underscore the superiority of QCNNs, with an average accuracy of 96.7%, sensitivity of 95.3%, precision of 96.0%, F1-score of 95.6%, and an AUC-ROC of 0.973. In comparison, Classical CNNs achieved an average accuracy of 83.8%, sensitivity of 81.3%, precision of 82.6%, F1-score of 81.9%, and an AUC-ROC of 0.869, as depicted in Fig. 12. The quantum-inspired architecture of QCNNs, leveraging entanglement and superposition, enhances feature extraction from heterogeneous tumor regions, thereby reducing misclassifications. These advancements position QCNNs as an ideal tool for tasks requiring both precision (e.g., differentiating benign vs. malignant tumors) and scalability in large-scale diagnostic workflows.

Fig. 12.

Comparative performance evaluation: Quantum CNN vs. Classical CNN across the overall dataset

Statistical Analysis of Model Performance

• Statistical Significance Testing Using t-Test

  To rigorously assess the significance of performance differences between the unoptimized model and the optimized models (ϵ = 0.2344 and ϵ = 0.1172) across the datasets, we employed t-tests for the data presented in Table 5. The t-test is a robust statistical method widely utilized to determine whether the means of two groups are significantly different from each other. In this context, it enables us to evaluate whether the observed performance improvements in the optimized models are statistically significant compared to the unoptimized model.

  The t-test statistic is calculated using the following formula:

  $$t=\frac{{\overline{X}}_1-{\overline{X}}_2}{\sqrt{\frac{s_1^2}{n_1}+\frac{s_2^2}{n_2}}}$$ 25

  where:

  •${\overline{X}}_1$ and ${\overline{X}}_2$ are the sample means of the two groups being compared.

  •$s_1$ and $s_2$ are the sample standard deviations.

  •$n_1$ and $n_2$ are the sample sizes.

  The p-value is then determined from the t-distribution, and a p-value less than 0.05 indicates that the difference between the two groups is statistically significant. The results of these t-tests are summarized in Table 7, which provides a detailed statistical analysis of the performance metrics presented in Table 5.

• Statistical Significance Testing Utilizing McNemar’s Test

  To meticulously compare the proposed Quantum Convolutional Neural Network (QCNN) with the Classical Convolutional Neural Network (CNN) as detailed in Table 6, we utilized McNemar's Chi-Square test on the classification outcomes derived from the test set. McNemar’s test is specifically suited for assessing the statistical significance of performance disparities between two classifiers applied to identical datasets, particularly in binary classification.

  This test zeroes in on discordant pairs instances where one model correctly classifies a sample while the other does not. The test statistic is computed using the following formula:

  $${\chi }^2=\frac{(b-c{)}^2}{b+c}$$ 26

  where:

  • b: The count of samples misclassified by the Classical CNN but accurately classified by the QCNN.

  • c: The count of samples misclassified by the QCNN but accurately classified by the Classical CNN.

  A p-value below 0.05 signifies that the observed difference in classification performance is statistically significant and not attributable to random fluctuation. The outcomes of McNemar’s test are encapsulated in Table 8, offering a thorough statistical dissection of the performance metrics delineated in Table 6.

  The McNemar’s test results confirm that the performance improvements achieved by the QCNN over the Classical CNN are statistically significant for all datasets, with p-values well below the 0.05 significance threshold. This indicates that the QCNN model outperforms the Classical CNN in terms of classification accuracy and other performance metrics. The detailed statistical analysis provides a rigorous basis for the observed performance enhancements, further validating the effectiveness of the proposed QCNN model.

Table 7.

Results of t-test for statistical significance

Dataset	Metric	Comparison	p-value
3DIRCADb	DICE (%)	Unoptimized vs. ϵ = 0.2344	0.002
		Unoptimized vs. ϵ = 0.1172	0.015
	VOE (%)	Unoptimized vs. ϵ = 0.2344	0.031
		Unoptimized vs. ϵ = 0.1172	0.048
	RAVD	Unoptimized vs. ϵ = 0.2344	0.023
		Unoptimized vs. ϵ = 0.1172	0.036
	ASSD (mm)	Unoptimized vs. ϵ = 0.2344	0.001
		Unoptimized vs. ϵ = 0.1172	0.012
	MSD (mm)	Unoptimized vs. ϵ = 0.2344	0.003
		Unoptimized vs. ϵ = 0.1172	0.021
LiTSa	DICE (%)	Unoptimized vs. ϵ = 0.2344	0.001
		Unoptimized vs. ϵ = 0.1172	0.010
	VOE (%)	Unoptimized vs. ϵ = 0.2344	0.025
		Unoptimized vs. ϵ = 0.1172	0.038
	RAVD	Unoptimized vs. ϵ = 0.2344	0.018
		Unoptimized vs. ϵ = 0.1172	0.029
	ASSD (mm)	Unoptimized vs. ϵ = 0.2344	0.002
		Unoptimized vs. ϵ = 0.1172	0.014
	MSD (mm)	Unoptimized vs. ϵ = 0.2344	0.004
		Unoptimized vs. ϵ = 0.1172	0.020
MSDTask03	DICE (%)	Unoptimized vs. ϵ = 0.2344	0.003
		Unoptimized vs. ϵ = 0.1172	0.017
	VOE (%)	Unoptimized vs. ϵ = 0.2344	0.030
		Unoptimized vs. ϵ = 0.1172	0.045
	RAVD	Unoptimized vs. ϵ = 0.2344	0.022
		Unoptimized vs. ϵ = 0.1172	0.034
	ASSD (mm)	Unoptimized vs. ϵ = 0.2344	0.001
		Unoptimized vs. ϵ = 0.1172	0.013
	MSD (mm)	Unoptimized vs. ϵ = 0.2344	0.003
		Unoptimized vs. ϵ = 0.1172	0.022

The t-test results confirm that the performance improvements achieved by the optimized models (ϵ = 0.2344 and ϵ = 0.1172) over the unoptimized model are statistically significant for all datasets and metrics, with p-values well below the 0.05 significance threshold. This indicates that the optimizations lead to meaningful improvements in segmentation performance. The detailed statistical analysis provides a rigorous basis for the observed performance enhancements.

Table 8.

Results of McNemar’s test

Dataset	Model comparison	p-value
3DIRCADb	QCNN vs. Classical CNN	2.1e-6
LiTS17	QCNN vs. Classical CNN	1.3e-4
MSD Task-3	QCNN vs. Classical CNN	4.5e-5
Overall	QCNN vs. Classical CNN	1.1e-7

Comparative Analysis with SOTA Approaches

To evaluate the performance of the proposed model, we conducted comparative analyses using state-of-the-art liver tumor segmentation methods, as detailed in Table 9.

Table 9.

Comparative analysis of liver tumor segmentation techniques

Dataset	Approach	Dice (%)	VOE (%)	RVD (%)
3DIRCADb	DefED-Net [35]	66.25	34.28	0.81
	X-net [36]	69.11	36.09	− 0.68
	TD-Net [37]	68.20	40.80	− 8.40
	DRDA-Net [38]	75.40	22.60	0.288
	MS-FANet [39]	78.00	31.30	− 15.50
	Lgma-net [40]	83.20	24.30	0.76
	MAPFUNet [41]	83.90	23.70	3.00
	MS-UNet [42]	84.10	27.36	0.22
	Our Method	85.80	22.85	0.13
LiTS	TD-Net [37]	70.90	39.60	11.70
	MS-FANet [39]	74.20	36.70	− 10.70
	MAPFUNet [41]	85.80	22.03	11.00
	Lgma-net [40]	87.40	23.19	0.57
	DefED-Net [35]	87.52	23.85	0.52
	Our Method	88.70	18.29	0.11
MSD Task03	S. Muhammad et al.[43]	87.00	12.09	6.93
	Our Method	88.40	19.75	0.15

Discussion

The proposed QCNN-Swin-UNet framework represents a significant advancement in liver tumor segmentation and classification, addressing key challenges such as model size optimization, class imbalance, and computational efficiency. Our framework introduces several novel contributions that collectively enhance diagnostic precision and computational efficiency, setting it apart from existing methods.

Firstly, we optimized the Swin-UNet architecture to achieve a balance between model size and segmentation accuracy, enabling deployment on memory-constrained edge devices like the Jetson Nano. This optimization is crucial for real-time applications in resource-limited settings, a significant improvement over traditional models that often require substantial computational resources. Our model size reduction to 64.16 MB ensures efficient deployment without compromising segmentation performance.

Secondly, we introduced a quadratic penalty-based objective function that balances model size and Area Under the Curve (AUC). This function encourages the development of compact yet accurate models, ensuring that segmentation performance is not compromised while reducing computational requirements. This approach provides a practical solution for optimizing model performance in resource-constrained environments.

Thirdly, the integration of the Search and Rescue (SAR) algorithm to identify optimal model configurations is a key innovation. The SAR algorithm, a metaheuristic optimization method, systematically explores the hyperparameter space to find configurations that maximize segmentation accuracy while maintaining computational efficiency. The resulting SAR-Swin-UNet model achieves superior performance in terms of both segmentation accuracy and computational efficiency, setting it apart from other optimization techniques used in the field.

To address the issue of class imbalance, which is a common challenge in medical imaging datasets, we proposed a Focal AUC loss function. This loss function enhances the model's ability to segment minority class pixels, thereby improving overall segmentation accuracy and robustness. By focusing on hard-to-classify samples, the Focal AUC loss function ensures that the model performs well even in regions with low contrast and overlapping intensity distributions, which are typical characteristics of liver tumor images.

A groundbreaking aspect of our framework is the integration of a Quantum Convolutional Neural Network (QCNN) for classification. QCNNs leverage quantum computing principles such as superposition and entanglement to achieve enhanced parallel processing of high-dimensional image features and reduced parameter overhead through quantum state compression. This integration positions our framework to fully utilize the powerful capabilities of quantum technology as it becomes more accessible, thereby improving classification performance. The QCNN classifier outperforms traditional CNNs in all metrics, demonstrating its effectiveness in high-dimensional medical data analysis.

Our comprehensive evaluations on three benchmark datasets (3D-IRCADb, LiTS17, and MSD Task03) demonstrate the superior performance of our framework. Achieving Dice scores of 85.8%, 88.7%, and 88.4%, respectively, alongside a classification accuracy of 96.7%, our framework outperforms other state-of-the-art methods. Table 8 presents a comparative analysis of liver tumor segmentation techniques across three datasets: 3DIRCADb, LiTS, and MSD Task03, using Dice (%), VOE (%), and RVD (%) as performance metrics. The proposed method demonstrates superior performance, achieving a Dice score of 85.8%, a VOE of 22.85%, and an RVD of 0.13% on the 3DIRCADb dataset, outperforming other state-of-the-art methods such as DDefED-Net (66.25%, 34.28%, 0.81%), X-net (69.11%, 36.09%, − 0.68%), TD-Net (68.20%, 40.80%, − 8.40%), DRDA-Net (75.40%, 22.60%, 0.288%), MS-FANet (78.00%, 31.30%, − 15.50%), Lgma-net (83.20%, 24.30%, 0.76%), MAPFUNet (83.90%, 23.70%, 3.00%), and MS-UNet (84.10%, 27.36%, 0.22%). On the LiTS dataset, the proposed method again leads with a Dice score of 88.7%, a VOE of 18.29%, and an RVD of 0.11%, surpassing TTD-Net (70.90%, 39.60%, 11.70%), MS-FANet (74.20%, 36.70%, − 10.70%), MAPFUNet (85.80%, 22.03%, 11.00%), Lgma-net (87.40%, 23.19%, 0.57%), and DefED-Net (87.52%, 23.85%, 0.52%). Similarly, on the MSD Task03 dataset, the proposed method achieves a Dice score of 88.4%, a VOE of 19.75%, and an RVD of 0.15%, outperforming S.Muhammad et al. (87.00%, 12.09%, 6.93%). These results highlight the proposed method's enhanced accuracy, precision, and consistency in volume estimation for liver tumor segmentation.

Limitations

While the proposed QCNN-Swin-UNet framework has demonstrated promising results and robust performance, several limitations must be acknowledged. First, the current classification approach is limited to binary outcomes, distinguishing only between tumor and non-tumor tissues without differentiating among various tumor subtypes, such as benign or malignant tumors. This limitation is primarily due to the scarcity of publicly available datasets that feature multiclass annotations for liver tumors, thereby restricting the granularity of diagnostic insights that the system can provide.

Second, the segmentation and classification processes are based solely on 2D CT slices derived from the 3D-IRCADb, LiTS17, and MSD Task03 datasets. This approach may overlook valuable volumetric information and inter-slice continuity that 3D models could leverage. Incorporating full 3D context is essential for enhancing segmentation accuracy and expanding the model's applicability in real-world clinical scenarios.

Third, the evaluation of the framework's performance was conducted using publicly available datasets, which may not fully represent the diversity and complexity of real-world clinical data. Real-world datasets often encompass a broader spectrum of tumor characteristics, imaging artifacts, and patient demographics, factors that could influence the model's generalizability and robustness.

Fourth, while the model optimization and quantum-inspired mechanisms have led to significant improvements in computational efficiency, further enhancements are still necessary to facilitate seamless deployment on edge devices with constrained computational capabilities. Despite the model size being reduced to 64.16 MB, representing an 80.25% reduction, it may still present challenges for deployment on devices with more stringent memory limitations.

Conclusions

In response to the growing global burden of liver cancer, this study presents QCNN-Swin-UNet—a lightweight, hybrid deep learning framework that significantly advances the accuracy and efficiency of liver tumor segmentation and classification, particularly for deployment on edge devices. By integrating an optimized Swin-UNet architecture for segmentation with a Quantum Convolutional Neural Network (QCNN) for classification, the proposed model effectively addresses critical challenges in medical imaging, including tumor heterogeneity, class imbalance, and high computational demands. The use of a metaheuristic Search and Rescue (SAR) algorithm and a quadratic penalty-based objective function enabled an optimal trade-off between model compactness and segmentation performance, resulting in a reduced model size of 64.16 MB suitable for real-time inference on resource-constrained hardware such as the Jetson Nano. Additionally, the incorporation of a Focal AUC loss function enhanced the model’s sensitivity to underrepresented regions, while the QCNN leveraged quantum-inspired mechanisms like superposition and entanglement to achieve high classification accuracy with fewer parameters. Extensive validation across three benchmark datasets (3D-IRCADb, LiTS17, and MSD Task03) yielded Dice scores of 85.8%, 88.7%, and 88.4%, respectively, along with a classification accuracy of 96.7%, consistently outperforming conventional CNN-based approaches. These performance gains were further supported through statistical significance testing using paired t-tests and McNemar’s test, underscoring the robustness and reliability of the proposed framework. Moving forward, future work will explore other metaheuristic optimization algorithms, expand the model’s application to other medical imaging domains, and pursue clinical integration to fully realize its potential for improving early diagnosis and treatment planning for liver cancer patients.

Author Contribution

Conceptualization, W.M.I.; Methodology, W.M.I.; Software, W.M.I. and K.A.A.; Validation, W.M.I., K.A.A.; Formal analysis, W.M.I., K.A.A.; Investigation, W.M.I., Y.Q.Z.; Data curation, W.M.I. and K.A.A.; Writing—original draft preparation, W.M.I.; Writing—review and editing, W.M.I., Y.Q.Z., K.A.A., and H.M.E.; Visualization, W.M.I.; Supervision, Y.Q.Z.; Project administration, Y.Q.Z.

Data Availability

The data used in this study are publicly available datasets. The 3D-IRCADb dataset was obtained from: https://www.ircad.fr/research/data-sets/liver-segmentation-3d-ircadb-01/. The LiTS17 Dataset can be accessed at https://academictorrents.com/details/27772adef6f563a1ecc0ae19a528b956e6c803ce. The MSD Task03 Dataset is available at http://medicaldecathlon.com.

Declarations

Consent for Publication

Not applicable. This study uses data from previously published research, and no new individual data is disclosed.

Consent to Participate

Not applicable. This study does not involve new data collection or human participants.

Competing Interests

The author declares that there are no competing interests.