GSA-YOLO enables high-efficiency real-time X-ray security inspection via structured sparsity and adaptive knowledge distillation

Abstract

X-ray security inspection requires accurate real-time detection of prohibited items, but existing models often struggle to balance the challenges of severe occlusion, complex clutter, and strict speed requirements. To overcome these challenges, this paper proposes GSA-YOLO, a novel lightweight framework built upon the YOLOv8n architecture, specifically engineered to enhance detection robustness and inference efficiency. GSA-YOLO strategically integrates structured sparsity and adaptive knowledge transfer through three core components: Group Lasso (GL) applied to the network neck for robust feature extraction; Sparse Structure Selection (SSS) applied to the detection head for significant model slimming; and an Adaptive Knowledge Distillation (Ada-KD) mechanism for comprehensive accuracy recovery. This integrated approach synergistically enhances feature representation while pruning redundant channels, maximizing model efficiency without sacrificing performance. Rigorous evaluations on the HiXray and PIDray datasets confirm GSA-YOLO’s comprehensive capability, achieving a leading inference speed of 189.62 FPS, accompanied by a reduction in computational cost from 8.7G to 8.0G. Crucially, GSA-YOLO secures mAP50:95 results of 0.531 and 0.679 on HiXray and PIDray, demonstrating 2.4% and 1.8% improvements over the baseline, respectively. Compared to other models, GSA-YOLO exhibits enhanced accuracy while maintaining computational efficiency, making it a promising solution for practical X-ray security inspection.

Introduction

Security inspection at public transportation hubs and sensitive areas is an essential component of public safety¹. X-ray baggage screening systems are widely deployed for this purpose, but conventional approaches rely on manual interpretation, which is susceptible to human error, fatigue, and inconsistency, contributing to missed detections². The advancement of computer vision, particularly deep learning, has opened the possibility of more accurate and consistent automated detection of prohibited items³.

However, deploying deep learning models in X-ray security inspection presents challenges that differ from those in natural image recognition. Complex backgrounds and occlusion from the random arrangement and overlap of luggage contents hinder reliable feature extraction⁴. Prohibited item detection is also a multi-target problem with large variation in object scale, which increases the likelihood of missed detections for small or partially visible items⁵. In addition, high-throughput security checkpoints require real-time inference, imposing a practical constraint on model complexity that must be balanced against detection accuracy⁶.

To address these difficulties, methods from three research directions have been explored. On the model compression side, structured sparsity techniques such as Group Lasso^7,8 and Sparse Structure Selection⁹ have been applied to reduce model complexity, while knowledge distillation^10–12 has been used to recover accuracy lost during compression. On the detection architecture side, numerous methods have been proposed specifically for X-ray imagery, including frequency-aware dual-stream transformers¹³, split-attention anchor-based detectors¹⁴, collaborative knowledge distillation frameworks¹⁵, hard negative selection methods¹⁶, incremental learning for occluded instances¹⁷, feature fusion and edge-aware modules^18,19, and recent attention-enhanced YOLO variants^20–22. While these methods have advanced specific aspects of X-ray detection, they generally do not simultaneously address detection robustness under complex conditions and the inference efficiency required for real-time deployment.

A key difficulty lies in the inherent tension among these challenges: designs that improve robustness tend to increase model complexity, while compression techniques that improve inference speed often degrade accuracy, particularly for subtle or occluded targets. In X-ray security inspection, this tension is particularly evident: prohibited items exhibit overlapping layouts, low contrast, and high material diversity, making fine-grained feature representations important and sensitive to structural compression. Furthermore, while knowledge distillation can partially mitigate accuracy degradation, it introduces additional training stages and hyperparameter sensitivity. Most existing methods address only one or two of these aspects without jointly optimizing all three. To overcome these limitations, we propose GSA-YOLO, a novel lightweight framework based on the YOLOv8n architecture, that strategically integrates structured sparsity and knowledge transfer to simultaneously improve detection robustness and inference efficiency. Each component is designed to address a specific aspect of this tension: GL applies soft regularization in the neck to preserve feature quality under occlusion; SSS performs structured pruning in the detection head to reduce computational cost; and Ada-KD mitigates the accuracy degradation caused by structural compression. The main contributions of the proposed GSA-YOLO method are summarized as follows:

1. We propose an integrated sparsity framework that combines Group Lasso (GL) soft regularization in the network neck with Sparse Structure Selection (SSS) hard pruning in the detection head. This dual-stage approach systematically reduces model complexity while maintaining essential feature representations, leading to measurable improvements in FPS and computational efficiency.

2. We employ an Adaptive Knowledge Distillation (Ada-KD) mechanism to mitigate the accuracy loss introduced during the sparsity training stages. By adaptively transferring knowledge from a larger pretrained teacher model, Ada-KD facilitates accuracy recovery in the compressed student model.

3. Experiments on the HiXray and PIDray datasets demonstrate that GSA-YOLO achieves a favorable accuracy-efficiency trade-off, maintaining competitive detection performance while improving inference speed compared to recent YOLO variants.

The remainder of this paper is organized as follows. “Related work” reviews the relevant research, specifically summarizing the current state of network sparsity, knowledge distillation, and X-ray object detection architectures. “Preliminaries” introduces the YOLOv8n architecture and the overall GSA-YOLO pipeline as preliminaries. “Proposed method” then presents the technical details of the proposed GSA-YOLO framework. In “Experiment”, we conduct comprehensive experiments to demonstrate GSA-YOLO’s competitive performance on different aspects against mainstream models. “Conclusion” concludes this paper by summarizing our research, discussing limitations, and suggesting future research directions.

Related work

Advances in network pruning and structured sparsity

The continuous expansion of deep neural networks necessitates effective model compression techniques for efficient deployment on resource-limited security inspection systems. The foundational concept of network pruning can be traced back to Optimal Brain Damage (OBD)²³ and later popularized by Deep Compression²⁴. However, this unstructured pruning offered limited practical acceleration due to the resulting irregular sparse matrix structure. To realize actual inference speedup, research shifted toward structured sparsity methods, which remove contiguous groups of parameters on entire channels or filters, thereby preserving the dense matrix structure compatible with parallel hardware. Network Slimming²⁵ pioneered channel pruning. To achieve more explicit structure removal, the principle of Group Lasso (GL), which enforces group-wise sparsity using a mixed-norm penalty²⁶, was adapted for deep neural networks^7,8,27,28. GL and its non-convex variants are generally viewed as soft sparsity methods, excellent for subtle feature refinement, but are often incompatible with parallel hardware for practical inference acceleration, making them less suitable for hard real-time deployment. For high-level structural removal, hard sparsity techniques that directly remove network structures are required. Sparse Structure Selection (SSS)⁹ formulates pruning as an explicit optimization problem to select the most critical sparse structures, allowing for direct and significant reduction of GFLOPs and parameters^29,30. However, relying solely on hard sparsity is also proven insufficient: while hard sparsity minimizes model size, it frequently struggles to maintain detection stability and robustness against subtle or complex features, highlighting the necessity for a strategic, multi-stage application that balances compression and fidelity. This trade-off between compression and accuracy motivates the use of knowledge distillation as a recovery mechanism, which we review next.

Knowledge distillation and adaptive knowledge transfer

Despite the gains in efficiency from network pruning, deep compression and structural changes inevitably lead to a degradation in model accuracy and robustness. To mitigate this critical performance drop, Knowledge Distillation (KD)¹⁰ has become an indispensable technique. KD transfers generalized knowledge from a high-performing teacher model to a compressed student model¹¹. This technique is widely employed to recover lost precision for sparsified architectures³¹. While conventional KD uses a fixed loss structure, recent efforts have focused on adaptive knowledge transfer to account for the capacity gap between Teacher and Student. Adaptive Knowledge Distillation (Ada-KD) represents a notable development, introducing methods that dynamically adjust the distilled knowledge complexity based on the Student’s learning progression. Boutros et al.¹² proposed an AdaDistill approach that dynamically weighs knowledge transfer. Recent works have explored cross-modal knowledge distillation³² and symmetrical learning approaches³³ to address data sparsity and asymmetry challenges in knowledge transfer. Furthermore, iterative contextual and adaptive strategies have shown promise in image analysis tasks³⁴. Additionally, advanced attention mechanisms including frequency domain learning, dual meta attention, and meta PID networks have demonstrated effectiveness in handling complex image denoising and enhancement tasks^35–37. This adaptive paradigm is well-suited for highly sparse models, as it offers a more flexible mechanism for feature recovery compared to fixed-loss KD. However, the effectiveness of integrating such advanced Ada-KD techniques with multi-stage sparse architectures remains an underexplored challenge in object detection literature. Moreover, existing KD methods are largely validated on general detection benchmarks and do not account for the specific challenges of X-ray security inspection, where occlusion and low contrast impose additional demands on feature preservation, as reviewed in the following subsection.

Object detection architectures in X-ray security imaging

X-ray security inspection is a safety-critical application facing distinct and severe challenges, primarily severe object occlusion and high background clutter. The development of specialized datasets, such as SIXray³⁸, HiXray³⁹, and the challenging PIDray dataset⁴⁰ with its explicitly hidden test sets, has highlighted the need for robust detection under extreme interference. The architectural evolution in this field has been driven by the push for real-time performance. This began with slower two-stage detectors like Faster R-CNN⁴¹, but the push for speed quickly favored highly efficient one-stage detectors. You Only Look Once (YOLO)⁴² revolutionized this paradigm, and its variants, including YOLOv3⁴³, continuously refined efficiency. A major breakthrough in one-stage detection came with RetinaNet⁴⁴, which introduced Focal Loss to successfully address the extreme foreground-background class imbalance inherent in dense prediction, boosting one-stage detectors’ accuracy to parity with two-stage methods. Subsequent one-stage models like YOLOv5⁴⁵ and the state-of-the-art YOLOv8⁴⁶ have further refined efficiency through innovations like multi-scale feature pyramids and anchor-free, decoupled-head designs. Dual attention mechanisms have also been explored for context-aware feature learning in agricultural detection applications⁴⁷. Recent advances have explored alternative architectures for achieving efficiency-accuracy trade-offs, such as Mamba-based frameworks that achieve real-time performance without requiring post-hoc compression⁴⁸. Existing X-ray detection methods typically build on these architectures, prioritizing accuracy enhancements through architectural improvements like attention mechanisms^20,21 or refined feature fusion networks^49–52, often sacrificing the strict real-time efficiency required for mass deployment. We select the highly efficient YOLOv8 architecture as our core baseline (Table 1).

Table 1.

Summary of representative related works across three research directions.

Direction	Method	Core contribution	Limitation
Structured sparsity	Network slimming	Channel-level pruning via BN scaling factors	Uniform application without position awareness
Structured sparsity	Group Lasso	Soft group-wise sparsity regularization	Incompatible with hard real-time deployment alone
Structured sparsity	SSS	Explicit structure selection via learnable scaling	Accuracy drop requires post-pruning recovery
Knowledge distillation	Hinton KD	Soft-label knowledge transfer via temperature scaling	Fixed loss weight, no adaptation to student progress
Knowledge distillation	AdaDistill	Dynamic adaptive weighting of distillation	Not validated on structurally compressed models
Knowledge distillation	Cross-modal KD	Multi-modal knowledge transfer for data sparsity	Domain-specific, not generalized to pruned architectures
X-ray detection	Faster R-CNN	Two-stage accurate detection	Insufficient inference speed for real-time deployment
X-ray detection	YOLOv8	Anchor-free decoupled head, high efficiency	Not optimized for X-ray occlusion and clutter
X-ray detection	ESI-YOLO/GEMA-YOLO	X-ray specific architectural enhancements	No structural compression for deployment efficiency

The three research directions reviewed above share a common underlying tension that has not been jointly addressed. Structured sparsity methods (GL, SSS) can reduce model size and FLOPs, but hard pruning alone causes accuracy degradation that is particularly severe for subtle prohibited items in X-ray imagery. Knowledge distillation can recover this accuracy loss, yet existing adaptive KD strategies have not been systematically validated in combination with multi-stage structured sparsity pipelines. Meanwhile, X-ray detection architectures prioritize robustness against occlusion and clutter, but none of the existing lightweight detectors simultaneously address compression, accuracy recovery, and domain-specific robustness in a unified framework. The intersection of these three gaps defines the core scientific problem: how to achieve aggressive structural compression while maintaining detection robustness in X-ray security inspection scenarios.

The preceding review highlights a fundamental gap in achieving deployable, high-speed X-ray detection models. Current pruning methods fail to strike the necessary balance between extreme compression and maintaining feature robustness, leading to unacceptable accuracy loss on subtle prohibited items. Furthermore, while advanced KD techniques exist, their systematic integration and validation for recovering performance in models subjected to aggressive, structured sparsity remains unexplored. To address this efficiency-robustness gap, we propose a novel Sparse-Constraint YOLO Detection Framework, GSA-YOLO. This framework integrates a hybrid structured sparsity approach with an Ada-KD mechanism, targeting real-time, accurate X-ray security detection.

Preliminaries

GSA-YOLO is built upon the YOLOv8n architecture ⁴⁶, a compact anchor-free detector comprising three components: a Backbone (Conv, C2f, SPPF) for feature extraction, a Neck (FPN-based multi-scale fusion) for feature aggregation, and a decoupled detection Head with three scale-specific prediction branches (P3, P4, P5). YOLOv8n achieves a strong speed-accuracy baseline at 8.7G GFLOPs and 6.2MB model size, making it a suitable foundation for structured compression. The three-stage GSA-YOLO pipeline applies Group Lasso (GL) regularization to the Neck, Sparse Structure Selection (SSS) pruning to the Head, and Adaptive Knowledge Distillation (Ada-KD) fine-tuning to recover accuracy, as illustrated in Fig. 1.

Fig. 1.

GSA-YOLO framework overview based on YOLOv8n for X-ray security inspection. The architecture consists of three main components: Backbone (Conv, C2f, SPPF modules for basic visual feature extraction), Neck (Upsample, Concat, C2f, Conv modules implementing multi-scale feature fusion with FPN structure, targeted by Group Lasso), and Head (three Detect branches for multi-scale object detection, targeted by Sparse Structure Selection).

Proposed method

This section details the three proposed components of GSA-YOLO: Group Lasso (GL), Sparse Structure Selection (SSS), and Adaptive Knowledge Distillation (Ada-KD). The framework employs a multi-stage optimization process divided into two cascaded layers: the Sparsity-Compression Layer, where GL first establishes a differentiable soft-sparsity foundation and SSS subsequently executes the irreversible structural compression; and the Accuracy-Recovery Layer, where Ada-KD fine-tunes the compressed model to restore performance. This sequential approach is necessary because SSS is a non-differentiable operation, preventing simultaneous optimization of all components. The strategic application of different sparsity techniques to distinct architectural components is theoretically motivated by their functional roles, as shown in Table 2, where GL targets the neck’s feature fusion layers where multi-scale information integration creates channel redundancy, while SSS targets the detection head where hard pruning can eliminate redundant channels without compromising detection performance. Throughout this paper, “detection head” refers to the overall detection module (model.22) as a single architectural unit, while “detection heads” refers to the three scale-specific prediction branches (P3, P4, P5) contained within it. The complete mapping between YOLOv8n model layer indices and their architectural roles is provided in Table 2, where layers model.12–21 correspond to the Neck and model.22 corresponds to the detection head.

Table 2.

Strategic application of sparsity techniques across YOLOv8n architecture.

Layer	Module	Function	Sparsity strategy
0-9	Backbone	Feature extraction	No modification
10-11	Neck (Upsample + Concat)	Multi-scale fusion	–
12	Neck (C2f)	Feature refinement	GL applied
13-14	Neck (Upsample + Concat)	Multi-scale fusion	–
15	Neck (C2f)	Feature refinement	GL applied
16	Neck (Conv)	Downsampling	GL applied
17	Neck (Concat)	Feature fusion	–
18	Neck (C2f)	Feature refinement	GL applied
19	Neck (Conv)	Downsampling	GL applied
20	Neck (Concat)	Feature fusion	–
21	Neck (C2f)	Feature refinement	GL applied
22	Head (Detect)	Multi-scale detection	SSS applied

To further quantify the rationale behind this allocation, we analyze the channel redundancy of each layer group. As shown in Table 3, Head layers consistently exhibit higher redundancy scores than Neck layers, empirically supporting the application of hard SSS pruning to the Head and soft GL regularization to the Neck.

Table 3.

Channel redundancy analysis of Neck and Head layers in YOLOv8n, supporting the design rationale for GL-Neck and SSS-Head allocation.

Group	Layer	Channels	High corr ratio	Redundancy score
Head	head.box.cv2.0	64	0.36	0.42
Head	head.box.cv2.1	64	0.34	0.42
Head	head.box.cv2.2	64	0.31	0.40
Head	head.cls.cv3.0	8	0.33	0.41
Head	head.cls.cv3.1	8	0.32	0.41
Head	head.cls.cv3.2	8	0.32	0.41
Neck	neck.model.16	64	0.18	0.24
Neck	neck.model.17	192	0.18	0.23
Neck	neck.model.18	128	0.17	0.23
Neck	neck.model.19	128	0.18	0.24
Neck	neck.model.20	384	0.18	0.24
Neck	neck.model.21	256	0.17	0.23

Group Lasso

GL is strategically applied to the Neck part of the YOLOv8 model, specifically targeting the key convolutional layers model.12, model.15, model.16, model.18, model.19, and model.21, which correspond to the main feature fusion points in the Feature Pyramid Network. As a training-time feature refinement mechanism, GL does not directly remove channels but instead identifies and suppresses less important feature channels through soft regularization, preparing the network for subsequent structured pruning. GL is a regularization technique that enforces group-wise sparsity, which is crucial for achieving channel-level sparsity in neural networks. The objective is to shrink the weights of entire groups, effectively identifying and eliminating redundancy in the feature extraction process. Unlike unstructured pruning, which operates on individual weights, GL applies the penalty to a group of weights simultaneously. In our implementation, we treat the entire weight vector of each output channel as a single group. The loss function for a single convolutional layer is formulated as:

1

The total loss for the entire model is the global detection loss plus the sum of GL regularization terms across all neck layers:

2

where represents the global base detection loss computed once; is the entire weight vector of the g-th output channel of the l-th convolutional layer, which is treated as a group; is the number of output channels in the l-th layer; and β is the regularization hyperparameter that controls the strength of the penalty for Group Lasso. The -norm ensures that if a channel’s weights are shrunk, they are shrunk together, thereby encouraging channel-wise sparsity. L represents the total number of convolutional layers in the neck part of the YOLOv8n model.

The gradient component contributed by the Group Lasso penalty term with respect to is calculated as shown in Equation (3). This penalty gradient is added to the gradient of the base detection loss during optimization, pushing the weights of less important channels toward zero, thereby enabling channel-wise sparsity. The selection of the regularization hyperparameter β is critical for balancing model performance and the degree of induced sparsity; we set the default value of β to 1e−4.

3

where \varepsilon is a small positive constant (typically ) added to prevent division by zero when is a zero vector.

Sparse structure selection

SSS is the hard pruning module applied after GL, specifically targeting the detection head (model.22) which contains the convolutional layers for the three different scale detection outputs in YOLOv8n’s multi-scale detection architecture. These layers in the detection head are checked to exhibit greater channel redundancy, presenting a prime opportunity for aggressive hard pruning. The primary goal of SSS is to achieve model slimming and computational efficiency by eliminating less important channels, based on the learned channel scaling factors λ during the pretraining process, while preserving essential feature representations.

SSS employs -norm regularization on the channel scaling factors λ, which are designed to scale the outputs of specific structures, which is the channels in our module. The loss function for a single convolutional layer l is formulated as:

4

The total loss for the entire model across all layers is:

5

where represents the global base detection loss computed once; is the dedicated scaling factor for the c-th output channel in the l-th layer; is the total number of output channels in the l-th layer; is the -norm of the scaling factor, which promotes channel sparsity by forcing less important factors toward zero; γ is the regularization hyperparameter that controls the strength of the sparsity penalty for SSS; and L represents the total number of convolutional layers targeted for pruning in the head part of the YOLOv8 model, which is exactly 6.

After pretraining, we determine a pruning threshold τ based on the distribution of the scaling factors . Specifically, given a target pruning ratio, we collect all scaling factors from the detection head branches, sort them by magnitude, and set τ to the corresponding percentile value, ensuring that channels with are pruned while preserving a minimum number of channels for network functionality. The default value is set to τ = 0.001, which consistently achieves the best detection performance on both datasets as validated by sensitivity analysis in “Hyperparameter sensitivity analysis”. Channels whose scaling factors are below the threshold τ are subsequently pruned from the network as Eq. (6) illustrates. After pruning, the model undergoes fine-tuning to recover performance loss caused by the pruning process. The gradient of the loss with respect to is shown in Eq. (7).

6

7

where 0 is chosen as the subgradient at , which is a valid selection within the subgradient range and is standard in proximal gradient optimization.

The local details of the SSS pruning process, applied to the convolutional layers in model.22 Detection Head, are illustrated in Fig. 2. The choice of γ is important for balancing the trade-off between model accuracy and speed, whose default value is set to 1e−3. Rather than applying γ at full strength throughout training, we adopt a warmup-and-ramp schedule: sparsity regularization is introduced after an initial warmup of 10 epochs and then linearly increased to the target value over the subsequent 40 epochs, which stabilizes early training and avoids premature channel suppression.

Fig. 2.

The detailed illustration of SSS module. SSS performs channel pruning based on the regularized channel scaling factor λ. Gray channels whose scaling factors are inactive and can be pruned.

Adaptive knowledge distillation

Ada-KD is employed as the final stage to effectively recover the performance of the compressed YOLOv8n model after the application of GL and SSS. The core principle of Ada-KD is to transfer valuable knowledge from a pretrained and high-performing teacher model, the pretrained YOLOv8m model after the comparison in “Comparison experiment”, to the sparse student model via a progress-based adaptive weighting mechanism. The teacher model (YOLOv8m) remains unmodified and retains its full channel structure without any GL or SSS modifications. Since our distillation operates on the output-level classification probability distributions rather than intermediate feature maps, both teacher and student produce probability vectors of identical dimensionality C (the number of classes) after softmax, regardless of the internal channel differences caused by pruning in the student’s detection head. This mechanism dynamically adjusts the influence of the distillation loss based on the student’s learning progress, ensuring a balanced optimization between learning from the teacher’s soft labels and the ground-truth hard labels. The detailed illustration of Ada-KD mechanism is shown in Fig. 3.

Fig. 3.

The detailed illustration of Ada-KD mechanism.

The total loss function used during the Ada-KD phase is defined as:

8

where represents the overall loss during the distillation phase. is the base detection loss calculated using the ground-truth hard labels; is the knowledge distillation loss, which is computed using the soft labels provided by the teacher model; is a hyperparameter defining the maximum influence of the distillation loss; and is the adaptive weight function that dynamically controls the decay rate of the distillation loss, and it is time-dependent.

The adaptive weight function is calculated using the Sigmoid function in Eq. (9), which is designed to adaptively decrease the KD influence as the training progresses:

9

where t is the normalized training progress, defined as the current epoch divided by the total number of distillation epochs, thus ranging from 0 to 1; is a constant, set to 0.5, representing the normalized progress point where the decay factor α (t) drops to 0.5; and θ is the scaling factor that controls the steepness and transition rate of the Sigmoid curve.

The distillation loss is computed using Kullback–Leibler (KL) divergence¹⁰, which measures the difference between the teacher’s and student’s predicted probability distributions for both classification and bounding box regression logits. A temperature coefficient T=2.0 is applied to soften the probability distributions before computing the KL divergence, controlling the smoothness of the soft labels during knowledge transfer. The formulation is as follows:

10

where C is the number of classes; T=2.0 is the temperature coefficient; and , are the temperature-scaled softmax probabilities for class i from the teacher and student model, respectively. The factor compensates for the gradient magnitude reduction caused by temperature scaling.

By minimizing , the student model is encouraged to mimic the fine-grained decision boundaries encoded in the teacher’s soft labels, which is crucial for recovering performance and enhancing robustness in complex X-ray imaging scenarios. The scaling factors and θ primarily control the overall contribution and balance between hard label learning and knowledge transfer throughout the training phase; we set their default values to 4 and 15, respectively. Rather than applying differential processing across the three detection scales, we adopt a unified distillation strategy that applies the same KL divergence-based loss across all detection heads, prioritizing implementation simplicity and training stability while the adaptive weighting mechanism provides sufficient dynamic control over the distillation process.

Experiment

Building upon the GSA-YOLO framework proposed in the previous section, which integrates GL for neck-level feature regularization, SSS for detection head pruning, and Ada-KD for accuracy recovery, this section presents experiments to empirically evaluate each design decision and the overall framework performance.

Experimental setup

Datasets

To evaluate our model across challenging real-world X-ray security scenarios, we evaluate our model on two widely used X-ray datasets: HiXray³⁹ and PIDray⁴⁰. The HiXray contains 45,364 X-ray images, with 102,928 labeled instances collected from real-world airport scans. It features 8 categories of prohibited items, providing a rich diversity of complex and occluded scenarios. The PIDray contains 47,677 images across 12 categories, specifically emphasizing the detection of hidden and occluded objects, allowing us to rigorously test the robustness of the model. Both HiXray and PIDray datasets provide independent and dedicated test sets for final assessment, which have 9096 and 18,220 images, respectively. Specially, PIDray offers three distinct testing partitions, easy, hard, and hidden at a ratio about 5:2:3, to separately evaluate model’s performance under various occlusion and clutter scenarios. We divided the original training partitions of both datasets into new training and validation sets at an 8:2 ratio. The detailed dataset partition and class distribution are presented in Table 4.

Table 4.

The detailed dataset partition and class distribution of HiXray and PIDray.

Dataset (total/train/val/test)	Class	Count
HiXray (45,364/29,014/7254/9096)	Portable_Charger_1	12,421
	Portable_Charger_2	7788
	Water	3092
	Laptop	10,042
	Mobile_Phone	5383
	Tablet	4918
	Cosmetic	9949
	Nonmetallic_Lighter	883
PIDray (47,677/23,566/5891/18,220)	Baton	2399
	Pliers	6814
	Hammer	6299
	Powerbank	8116
	Scissors	7060
	Wrench	6437
	Gun	3757
	Bullet	2957
	Sprayer	4227
	Handcuffs	3388
	Knife	5549
	Lighter	6157

Evaluation metrics

Following standard object detection practices, we evaluate the proposed model using the following metrics to assess detection accuracy: Precision (P), Recall (R), mAP50, mAP50:95, and model efficiency: Frames Per Second (FPS) and GFLOPs. The definition of these metrics relies on True Positives (TP), False Positives (FP), and False Negatives (FN). Furthermore, the Intersection over Union (IoU), representing the ratio of intersection and concatenation between the predicted bounding box and the ground truth box, is used to determine if a detection counts as a TP.

Precision (P) assesses the model’s accuracy, reflecting the ratio of correctly predicted positive samples to all detected samples, and is calculated as shown in Eq. (11):

11

Recall (R) evaluates the model’s completeness, measuring the ratio of correctly predicted positive samples to the total number of actual positive samples, and is calculated as shown in Eq. (12):

12

The Average Precision (AP) for a single class is defined as the area under the Precision-Recall curve, quantifying the detection performance across all Recall levels, as shown in Eq. (13):

13

The Mean Average Precision (mAP) is the average of the AP values across all N object categories, providing a holistic measure of detection accuracy across all classes, as shown in Eq. (14):

14

where the in the equation denotes the AP value with category index value i, and N denotes the number of categories of the samples in the training dataset, in this paper, N is 8 and 12 for HiXray and PIDray respectively. Specifically, mAP50 denotes the mAP when the IoU threshold is fixed at 0.50, serving as a robust baseline for overall performance. mAP50:95 is a more stringent metric, representing the average mAP across IoU thresholds ranging from 0.50 to 0.95 with 0.05 increments. This multi-threshold averaging allows mAP50:95 to comprehensively reflect the model’s ability to accurately localize objects under varying degrees of overlap and strict bounding box requirements.

For model efficiency, we use GFLOPs to measure the theoretical computational complexity of the model, as shown in Eq. (15). GFLOPs quantifies the total number of floating-point operations required to process a single image, providing an insight into the inherent efficiency of the model architecture. Specifically, GFLOPs is calculated as the total number of floating-point operations (FLOPs) divided by , which standardizes the value to a more manageable scale. Moreover, Frames Per Second (FPS) measures the actual inference speed on the target hardware. It is the reciprocal of the average inference time per image, as shown in Eq. (16), and measures the model’s real-time processing capability in practical security inspection scenarios:

15

16

Implementation details

The GSA-YOLO framework is built upon the lightweight YOLOv8n architecture, while YOLOv8m was selected as the teacher model for the Ada-KD mechanism as discussed in “Comparison experiment”. Our proposed GSA-YOLO and the YOLOv8n baseline were implemented via the Ultralytics Python package (version 8.3.199)⁴⁶. All experiments, including the training and evaluation of comparative models reported in Tables 6 and 7, were executed on a consistent computational platform anchored by an NVIDIA RTX 3090 GPU, utilizing Python 3.10.18 and PyTorch 2.7.1 integrated with CUDA 12.6. For model training, the input image size was fixed at 640 × 640, a batch size of 64 was employed, and the YOLOv8n baseline was trained for 100 epochs on both datasets. Our multi-stage training pipeline consists of three sequential phases: (1) Group Lasso training using SGD optimizer (lr = 1e−4, momentum = 0.9, weight decay = 5e−4) with cosine annealing scheduler for 60 epochs, with the Backbone weights remaining unfrozen and jointly optimized throughout this stage; (2) SSS training initialized from the GL-trained model weights, using the same SGD configuration with gamma schedule (10 epochs warmup, 40 epochs ramp to target gamma = 1e−4) for 85 epochs, where pruning is performed at the end of sparse training and no separate fine-tuning stage is added after pruning since the Ada-KD stage serves this recovery function; (3) Ada-KD training initialized from the SSS-pruned model, using SGD optimizer with cosine annealing scheduler for 60 epochs, totaling 205 epochs across all stages. Notably, during the SSS training stage, the GL-trained Neck weights are not frozen but continue to be jointly optimized, ensuring that the feature distribution of the Neck remains adaptive and does not adversely affect the evaluation of channel importance in the Head. To ensure the reliability and fair comparison of efficiency metrics, all models were tested using the same 640 × 640 input resolution at a batch size of 1, running in standard FP32 precision. A warm-up phase of 50 forward passes was executed prior to formal timing to stabilize GPU performance. The formal FPS measurement is then conducted over the full test set (9096 images for HiXray and 18,220 images for PIDray) with continuous sequential inference, and the reported FPS is computed as the average frame rate over all test images. The reported FPS specifically reflects the latency of the model’s forward pass, the inference time, without including post-processing operations like NMS or box decoding. The detailed experiment configuration and settings are shown in Table 5.

Table 6.

Evaluation results of the different methods on the HiXray dataset. (The best performance in each metric is marked in bold, and the second-best is underlined.).

Model	Model size	GFLOPs	FPS	Precision	Recall	mAP50	mAP50:95
Faster R-CNN41	319.2MB	168.7G	15.02	0.841	0.763	0.788	0.467
RetinaNet44	128.7MB	151.5G	34.05	0.824	0.723	0.756	0.451
YOLOv5n45	3.9MB	4.5G	161.31	0.859	0.773	0.799	0.486
YOLOv8n46	6.2MB	8.7G	170.58	0.882	0.769	0.806	0.507
YOLOv8n-Pruned (L1)	4.4MB	6.1G	231.92	0.869	0.760	0.795	0.496
YOLOv8n-Pruned+QAT (INT8)	2.3MB	6.1G	312.50	0.856	0.748	0.782	0.487
YOLOv8m46	50.3MB	79.1G	96.23	0.912	0.806	0.839	0.550
Fine-YOLO53	46.2MB	56.9G	152.42	0.877	0.771	0.811	0.514
SRW-YOLO54	9.4MB	12.0G	158.97	0.883	0.782	0.814	0.516
ESI-YOLO55	6.7MB	8.9G	164.48	0.886	0.779	0.815	0.517
DSF-YOLO56	35.4MB	43.4G	54.60	0.889	0.786	0.817	0.515
GEMA-YOLO22	11.8MB	16.1G	82.57	0.895	0.788	0.822	0.523
GSA-YOLO(ours)	5.7MB	8.0G	189.62	0.902	0.795	0.827	0.531

Table 7.

Evaluation results of the different methods on the PIDray dataset. (The best performance in each metric is marked in bold, and the second-best is underlined.).

Model	Model size	easy		hard		hidden		Average
		mAP50	mAP50:95	mAP50	mAP50:95	mAP50	mAP50:95	mAP50	mAP50:95
Faster R-CNN41	319.2MB	0.805	0.679	0.820	0.632	0.582	0.403	0.736	0.571
Retina-Net44	128.7MB	0.799	0.672	0.813	0.621	0.575	0.401	0.729	0.565
YOLOv5n45	3.9MB	0.817	0.691	0.847	0.651	0.593	0.406	0.752	0.583
YOLOv8n46	6.2MB	0.844	0.750	0.862	0.718	0.644	0.514	0.783	0.661
YOLOv8n-Pruned (L1)	4.4MB	0.832	0.741	0.851	0.705	0.635	0.504	0.769	0.648
YOLOv8n-Pruned+QAT (INT8)	2.3MB	0.822	0.729	0.838	0.696	0.622	0.492	0.758	0.638
YOLOv8m46	50.3MB	0.876	0.781	0.888	0.745	0.674	0.533	0.813	0.675
Fine-YOLO53	46.2MB	0.848	0.752	0.866	0.721	0.646	0.512	0.787	0.662
SRW-YOLO54	9.4MB	0.847	0.754	0.868	0.722	0.642	0.509	0.786	0.662
ESI-YOLO55	6.7MB	0.850	0.759	0.871	0.724	0.651	0.518	0.791	0.667
DSF-YOLO56	35.4MB	0.852	0.756	0.876	0.724	0.648	0.516	0.792	0.665
GEMA-YOLO22	11.8MB	0.861	0.763	0.875	0.729	0.654	0.517	0.797	0.670
GSA-YOLO(ours)	5.7MB	0.868	0.771	0.879	0.734	0.665	0.524	0.804	0.679

Table 5.

Detailed experimental configuration and settings.

Configuration	Setting
Operating system	Ubuntu 20.04
CPU	Intel(R) Core(TM) Ultra 7 255H 2.00 GHz
GPU	NVIDIA RTX 3090
CUDA	12.6
Framework	PyTorch 2.7.1
Language	Python 3.10.18
Input resolution (training and test)	640 × 640
Training batch size	64
Inference batch size	1
Measurement precision	FP32 (32-bit floating-point)
FPS measurement scope	Forward pass latency only
GPU pre-heating	50 warm-up passes

Comparison experiment

The effectiveness of the proposed GSA-YOLO was rigorously validated by comparing it with mainstream methods for detecting prohibited items and several well-known object detection models^22,53–56on the HiXray and PIDray datasets. Our comparison strategy focuses on X-ray security detection methods and established baselines to ensure domain-relevant evaluation. All reported results represent averages from multiple runs with different random seeds to ensure statistical reliability. The evaluation metrics included GFLOPs, FPS, Precision, Recall, mAP50, and mAP50:95, covering both computational efficiency and detection accuracy. As presented in Tables 6 and 7, GSA-YOLO consistently achieves a new competitive balance between efficiency and accuracy. The proposed method reduces computational cost from 8.7G to 8.0G GFLOPs while maintaining competitive performance, achieving an effective efficiency-accuracy trade-off.

On the HiXray dataset, GSA-YOLO achieves competitive performance across the evaluated metrics. The inference speed reaches 189.62 FPS, an 11% improvement over YOLOv8n, attributed to the SSS module which reduces GFLOPs from 8.7G to 8.0G through structured pruning of redundant channels in the detection head. In terms of detection accuracy, GSA-YOLO achieves mAP50 of 0.827 and mAP50:95 of 0.531, representing improvements of 2.1% and 2.4% over the YOLOv8n baseline. The Ada-KD mechanism contributes to accuracy recovery by transferring soft-label knowledge from the teacher model, mitigating the degradation introduced during the sparsity stages.

On the PIDray dataset, which includes easy, hard, and hidden test partitions, GSA-YOLO achieves the best performance across all eight evaluated metrics. On the hidden test set, the model obtains mAP50 of 0.665 and mAP50:95 of 0.524. The GL module, by applying soft regularization to the neck’s feature fusion layers, helps preserve multi-scale feature representations under occlusion and cluttered backgrounds. Overall, GSA-YOLO achieves mAP50 of 0.804 and mAP50:95 of 0.679, exceeding the baseline by 2.1% and 1.8% respectively. Compared to the two compression baselines, GSA-YOLO outperforms YOLOv8n-Pruned by +0.037 mAP50:95 on HiXray and +0.032 on PIDray, and exceeds YOLOv8n-Pruned+QAT by +0.050 and +0.045 respectively, confirming that Group Lasso-guided sparsity with Ada-KD recovery achieves a superior accuracy-efficiency trade-off over aggressive compression without knowledge distillation.

To select an appropriate teacher model for Ada-KD, we compare the YOLOv8 series from YOLOv8n to YOLOv8l in terms of the trade-off between distillation gain and training overhead.

As illustrated in Fig. 4, the student model’s mAP50:95 plateaus beyond YOLOv8m, while training time continues to increase, making YOLOv8m a reasonable choice that balances accuracy recovery and training cost. We thus selected YOLOv8m as our teacher model in our Ada-KD mechanism.

Fig. 4.

The trade-off analysis of teacher model complexity (GFLOPs) vs. student model performance (mAP50:95) for Ada-KD mechanism.

Hyperparameter sensitivity analysis

To optimize the GSA-YOLO framework performance and guide practical deployment, we conducted hyperparameter sensitivity analysis across the three core modules: GL, SSS and Ada-KD. The objective is to identify optimal parameter values that balance detection accuracy and efficiency. The analysis follows rigorous experimental protocols: the control variable method is adopted so that only one hyperparameter is varied at a time while all others remain fixed at their baseline values; a fixed random seed is used throughout all sensitivity experiments to ensure reproducibility; grid search is employed over predefined value ranges for each hyperparameter; and all configurations are evaluated on both HiXray and PIDray datasets under identical training procedures. The tested ranges and optimal values for all hyperparameters are summarized in Table 8. The detailed trends and trade-offs are visualized in Fig. 5.

Table 8.

Summary of hyperparameter sensitivity analysis.

Module	Hyperparameter	Search range	Optimal value
GL	β
SSS	γ
Ada-KD			4
Ada-KD	θ		20

Fig. 5.

Hyperparameter sensitivity analysis for the GSA-YOLO framework. The shaded regions in each subplot represent the optimal parameter ranges where the models achieve balance between performance and efficiency.

The GL parameter controls soft sparsity regularization intensity for feature refinement in the neck part. Performance curves on both HiXray and PIDray show a convex shape, with peak performance at . This value provides regularization strength that maintains model performance against complex clutter while preserving feature pathways. As shown in Fig. 5a, the optimal performance range corresponds to .

For the SSS module, we examine the pruning intensity γ as shown in Fig. 5b, which affects the speed-accuracy relationship. The plot uses dual Y-axes to track mAP50:95 (left) and FPS (right). As γ increases, FPS shows continuous improvement due to model slimming. mAP50:95 peaks at , followed by decline, indicating that pruning beyond reduces model capacity. The value is selected for balancing FPS improvement and accuracy maintenance.

In addition to γ, the pruning threshold τ directly determines which channels are removed after SSS sparse training. Table 9 presents the sensitivity analysis of τ on both datasets. The results show that τ = 0.001 consistently achieves the best performance, while smaller values provide insufficient pruning and larger values cause over-aggressive channel removal with measurable accuracy degradation.

Table 9.

Sensitivity analysis of pruning threshold τ on HiXray and PIDray datasets.

τ	HiXray				PIDray
	Precision	Recall	mAP50	mAP50:95	Precision	Recall	mAP50	mAP50:95
0.0005	0.897	0.791	0.822	0.526	0.832	0.736	0.799	0.674
0.0010	0.902	0.795	0.827	0.531	0.837	0.741	0.804	0.679
0.0015	0.899	0.792	0.824	0.528	0.834	0.738	0.801	0.676
0.0020	0.894	0.787	0.820	0.523	0.830	0.733	0.797	0.670
0.0030	0.886	0.780	0.813	0.515	0.821	0.726	0.789	0.662

The Ada-KD module involves two interdependent parameters: Maximum Distillation Intensity and Decay Steepness θ. Performance for both datasets peaks at , which falls within the range shown in Fig. 5c. Low values limit knowledge transfer, while high values cause over-distillation, affecting the student model’s learning of hard targets. For θ, which controls the Ada-KD weight decay rate during training, results show that θ =20 yields the highest mAP50:95. This value balances knowledge transfer in early phases with student model optimization during convergence. The performance range corresponds to [15, 25] as illustrated in Fig. 5d.

We further conduct sensitivity analysis on the inflection point of the adaptive weight function. As shown in Table 10, consistently achieves the best performance on HiXray, validating that an equal split between teacher-guided and student-independent phases provides the optimal knowledge transfer schedule for X-ray security inspection scenarios.

Table 10.

Sensitivity analysis of inflection point on HiXray dataset.

	Precision	Recall	mAP50	mAP50:95
0.3	0.896	0.788	0.820	0.525
0.4	0.900	0.792	0.824	0.529
0.5	0.902	0.795	0.827	0.531
0.6	0.899	0.793	0.825	0.528
0.7	0.895	0.788	0.820	0.523

To validate the decay-type weight design, we compare it against alternative weighting patterns on the HiXray dataset. As shown in Table 11, the decay pattern consistently outperforms increasing, bell-shaped, and fixed alternatives, confirming that strong initial teacher guidance followed by gradual student independence is the most effective strategy for accuracy recovery in sparse X-ray detection models.

Table 11.

Comparison of Ada-KD weight design patterns on HiXray dataset.

Pattern	Description	Precision	Recall	mAP50	mAP50:95
Decay (ours)	High→Low (0.8→0.2)	0.902	0.795	0.827	0.531
Increasing	Low→High (0.2→0.8)	0.892	0.783	0.816	0.521
Bell-shaped	Low→High→Low (0.2→0.8→0.2)	0.896	0.787	0.820	0.524
Fixed	Constant (0.5)	0.888	0.779	0.813	0.517

Ablation experiment

Ablation experiments were conducted on both datasets to evaluate the individual and combined impact of the modules: GL, SSS and Ada-KD on YOLOv8n baseline performance. Precision, Recall, mAP50 and mAP50:95 were selected as evaluation metrics. The composition and description of the module groups utilized in the experiments are as follows:

• ∘ Baseline: The original YOLOv8n model, serving as the benchmark without any proposed modifications.

• ∘ GL: Implementation of the GL module solely on the Neck of the YOLOv8n model, designed for initial feature refinement and soft sparsity.

• ∘ SSS: Implementation of the SSS module solely on the detection head of the YOLOv8n model, applying hard pruning for efficiency.

• ∘ GL+SSS: The combination of GL on the Neck and SSS on the Head, creating a structurally compact and sparse architecture.

• ∘ GL+Ada-KD: Applying the Ada-KD mechanism on the GL-integrated model, targeting recovery of neck-refined features.

• ∘ SSS+Ada-KD: Applying the Ada-KD mechanism on the SSS-integrated model, targeting recovery of head-pruned channels.

• ∘ GL+SSS+Ada-KD (GSA-YOLO): The complete model integrating all three components for efficiency, accuracy and robustness.

Table 12 presents the ablation results on the HiXray dataset. Applied individually, GL and SSS each show only marginal differences from the Baseline: SSS slightly improves Recall to 0.776, while GL achieves an mAP50 of 0.809, but both also introduce a minor accuracy trade-off during the compression phase. The combination of GL+SSS shows complementary effects, surpassing the Baseline across all metrics with mAP50 of 0.811 and mAP50:95 of 0.509.

Table 12.

Ablation experiment on HiXray dataset.

GL	SSS	Ada-KD	Precision	Recall	mAP50	mAP50:95
×	×	×	0.882	0.769	0.806	0.507
\checkmark	×	×	0.871	0.768	0.809	0.501
×	\checkmark	×	0.873	0.776	0.808	0.504
\checkmark	\checkmark	×	0.885	0.781	0.811	0.509
\checkmark	×	\checkmark	0.886	0.786	0.813	0.513
×	\checkmark	\checkmark	0.888	0.789	0.815	0.514
\checkmark	\checkmark	\checkmark	0.902	0.795	0.827	0.531

Ada-KD consistently mitigates the accuracy degradation introduced during compression. GL+Ada-KD improves upon the GL-only model by 1.4% in Recall and 1.2% in mAP50:95, while SSS+Ada-KD achieves a 1.5% and 1.3% increase in Precision and Recall over SSS-only, confirming Ada-KD’s role in recovering performance in compressed models. The complete GL+SSS+Ada-KD (GSA-YOLO) architecture achieves the best overall performance, with gains of 2.0% in Precision, 2.6% in Recall, 2.1% in mAP50, and 2.4% in mAP50:95 over the Baseline. These improvements represent relative gains of 4.7% on HiXray and 2.7% on PIDray in terms of mAP50:95.

Table 13 presents the ablation results on the PIDray dataset. The trend is consistent with HiXray: individual GL or SSS modules result in slight performance degradation compared to the Baseline, while Ada-KD consistently recovers accuracy. Both GL+Ada-KD and SSS+Ada-KD surpass the Baseline across all metrics, confirming Ada-KD’s role in maintaining detection performance under compression, particularly on PIDray’s hard and hidden subsets. The complete GL+SSS+Ada-KD (GSA-YOLO) architecture achieves the best overall performance, with gains of 1.2% in Precision, 1.7% in Recall, 2.1% in mAP50, and 1.8% in mAP50:95 over the Baseline.

Table 13.

Ablation experiment on PIDray dataset.

GL	SSS	Ada-KD	Precision	Recall	mAP50	mAP50:95
×	×	×	0.825	0.724	0.783	0.661
\checkmark	×	×	0.820	0.728	0.781	0.657
×	\checkmark	×	0.818	0.721	0.777	0.652
\checkmark	\checkmark	×	0.830	0.731	0.788	0.663
\checkmark	×	\checkmark	0.828	0.734	0.789	0.665
×	\checkmark	\checkmark	0.827	0.729	0.785	0.664
\checkmark	\checkmark	\checkmark	0.837	0.741	0.804	0.679

To validate the effectiveness of our sigmoid-based adaptive decay schedule, we conducted comparative experiments with alternative decay strategies including cosine and linear schedules on the HiXray dataset. Results show that our complete GSA-YOLO model with sigmoid decay achieves Precision of 0.902, Recall of 0.795, mAP50 of 0.827, and mAP50:95 of 0.531, outperforming cosine decay at 0.526 and linear decay at 0.523, showing an improvement over cosine and linear decay schedules in this setting. Table 14 presents the detailed comparative results.

Table 14.

Comparative analysis of Ada-KD decay schedules on HiXray dataset.

Method	Precision	Recall	mAP50	mAP50:95
Decay schedule comparison
Sigmoid decay (ours)	0.902	0.795	0.827	0.531
Cosine decay	0.897	0.789	0.822	0.526
Linear decay	0.893	0.785	0.818	0.523

The detection performance across both datasets indicates that the integration of GL, SSS, and Ada-KD addresses X-ray detection challenges including severe occlusion, complex backgrounds, and real-time inference requirements.

Deployment and training cost analysis

To evaluate GSA-YOLO’s practical deployment readiness, we compare two inference configurations: the original PyTorch FP32 model and a TensorRT-optimized FP16 version. As shown in Table 15, the TensorRT FP16 version reduces model size by approximately 49% and improves inference throughput by approximately 70%, with a modest accuracy drop of 0.013 in both mAP50 and mAP50:95. These results confirm that GSA-YOLO is compatible with standard inference optimization pipelines and suitable for lightweight deployment without significant accuracy degradation.

Table 15.

Deployment configuration comparison for GSA-YOLO on HiXray.

Model	Backend	Model size	FPS	mAP50	mAP50:95
GSA-YOLO	PyTorch FP32	5.7MB	189.62	0.827	0.531
GSA-YOLO	TensorRT FP16	2.9MB	322.35	0.814	0.518

We further report the training cost of GSA-YOLO relative to representative baselines on HiXray. As shown in Table 16, GSA-YOLO requires approximately 2.1× more training time than YOLOv8n due to its three-stage pipeline. However, this is a one-time cost: at deployment, GSA-YOLO achieves the highest inference FPS (189.62) among all compared methods—3.5× faster than DSF-YOLO and 2.3× faster than GEMA-YOLO—while also achieving the best mAP50:95 (0.531) with the smallest model size (5.7 MB). The training overhead is thus justified by the superior accuracy-efficiency trade-off at inference time.

Table 16.

Training cost and inference performance comparison on HiXray.

Model	Model size	Training time	Infer FPS	mAP50	mAP50:95
YOLOv8n	6.2MB	5.23h	170.58	0.806	0.507
Fine-YOLO	46.2MB	5.35h	152.42	0.811	0.514
DSF-YOLO	35.4MB	5.78h	54.60	0.817	0.515
GEMA-YOLO	11.8MB	5.47h	82.57	0.822	0.523
GSA-YOLO (ours)	5.7MB	10.15h	189.62	0.827	0.531

Confusion Matrix Analysis

To examine the classification performance and detection completeness of GSA-YOLO, we visualize confusion matrices for the YOLOv8n baseline and our model on HiXray and PIDray hidden test sets, as shown in Fig. 6. The analysis focuses on two performance indicators: True Positive (TP) counts on the diagonal, reflecting classification accuracy; and False Negative (FN) counts in the background row for non-background classes, representing missed detections.

Fig. 6.

Confusion matrix plots on the HiXray and PIDray hidden test sets. Subfigures (a) and (b) show the results on the HiXray test set for the YOLOv8n baseline and our GSA-YOLO model, respectively. Subfigures (c) and (b) show the corresponding results on the challenging PIDray hidden test set.

On the HiXray dataset, as shown in Fig. 6a,b, GSA-YOLO shows generally higher TP counts and reduced missed detections across most categories. For example, correctly identified Mobile_Phone instances increased from 10,359 (baseline) to 10,405 (ours). GSA-YOLO reduces missed detections: for the Cosmetic category, missed objects decreased from 790 (baseline) to 555; missed detections for Portable_Charger_1 and Portable_Charger_2 decreased from 153 to 113 and from 170 to 117 respectively. This pattern is observed on the PIDray hidden test set containing complex and occluded instances. As shown in Fig. 6c,d, GSA-YOLO increases correctly detected Wrench and Gun from 401 to 457 and 224 to 269 respectively. These results indicate that GSA-YOLO maintains or improves detection completeness relative to the baseline across both datasets.

Feature activation analysis

To examine the feature learning behavior of GSA-YOLO, we conduct Class Activation Mapping (CAM) visualization across three representative scenarios: multi-target occlusion (Water + Mobile_Phone), concealed targets (liquid bottle in backpack), and clear single targets (isolated liquid bottle). As shown in Fig. 7, GSA-YOLO produces more focused activation patterns with better alignment to target boundaries compared to ESI-YOLO and YOLOv8n, which exhibit broader or more diffuse activations. These observations are consistent with the expected effects of GL’s soft regularization on feature representations and SSS’s structured pruning on channel redundancy.

Fig. 7.

Class Activation Mapping visualization comparison for X-ray prohibited item detection. The heat maps demonstrate the spatial attention patterns of different models: GSA-YOLO (Ours), ESI-YOLO, and YOLOv8n baseline, with Ground Truth (GT) annotations for reference.

Visualization analysis

To demonstrate the detection effectiveness in real X-ray security inspection scenarios, we conducted visual comparison among three models: YOLOv8n baseline, ESI-YOLO, and GSA-YOLO. We selected representative X-ray images covering different complexity levels including clear scenarios, complex occlusion scenarios, and overlapping small target scenarios. The comparative detection results are presented in Fig. 8.

Fig. 8.

X-ray security inspection detection visualization comparison. The figure shows detection results across different complexity scenarios: Image (original X-ray images), GT (Ground Truth annotations), Ours (GSA-YOLO), ESI-YOLO, and YOLOv8n baseline. Different colored boxes represent prohibited items: red (Tablet), blue (Mobile Phone), yellow (Portable Charger), dark blue (Cosmetic). The comparison covers clear scenarios, complex occlusion scenarios, and overlapping small target scenarios.

The results are presented across three complexity levels. In clear scenarios (first row), GSA-YOLO produces bounding boxes that are well-aligned with Ground Truth, while YOLOv8n shows duplicate detections and positional offsets. In complex occlusion scenarios (second row), GSA-YOLO maintains stable detection for items obscured by cables, whereas YOLOv8n exhibits more missed detections. In overlapping small target scenarios (third row), GSA-YOLO localizes overlapping portable chargers and mobile phones more accurately, while YOLOv8n shows larger positional deviations for small targets. ESI-YOLO performs comparably to GSA-YOLO in most scenarios but shows slight disadvantages in small target recall.

Conclusion

In this paper, we introduced GSA-YOLO, a lightweight framework designed for efficient and robust prohibited item detection in X-ray security imagery. To address the need for real-time models resilient to severe occlusion and complex clutter, GSA-YOLO integrates three core components: Group Lasso (GL) for feature refinement, Sparse Structure Selection (SSS) for structured pruning, and Adaptive Knowledge Distillation (Ada-KD) for accuracy recovery. Experiments on the HiXray and PIDray datasets confirm the framework’s overall effectiveness. GSA-YOLO successfully reduced the computational cost from the YOLOv8n baseline’s 8.7G to 8.0G, while simultaneously achieving an mAP50:95 of 0.531 on HiXray, a 2.4% improvement over the baseline, and 0.679 on PIDray, a 1.8% lift, demonstrating its potential as a promising candidate for efficient X-ray security inspection under controlled benchmark conditions.

Despite these results, the GSA-YOLO framework has certain limitations. The current implementation relies on a sequential, multi-stage optimization process, GL → SSS → Ada-KD. Due to the non-differentiable nature of hard pruning, this pipeline increases sensitivity to hyperparameter tuning and requires careful calibration across stages. Furthermore, X-ray prohibited item detection relies on fine-grained visual cues—such as overlapping objects and low-contrast materials—that are sensitive to aggressive channel pruning; this domain-specific constraint motivates the adoption of low-to-moderate compression, as verified by the experimental comparisons against stronger compression baselines. Regarding practical deployment feasibility, the pruned GSA-YOLO model achieves a model size of 5.7 MB and 8.0 GFLOPs, both lower than the YOLOv8n baseline (6.2 MB, 8.7 GFLOPs), indicating a compact memory footprint compatible with edge inference platforms; however, comprehensive evaluation of latency fluctuations under variable input conditions and hardware compatibility with proprietary security inspection pipelines remains an important direction for future work. Future work will focus on two main directions: first, investigating GSA-YOLO’s deployment on resource-limited edge devices using mixed-precision quantization; and second, extending the GSA components to other advanced object detection architectures beyond the YOLO family. Additionally, evaluating the multi-stage training pipeline under alternative optimizers such as AdaBoB ⁵⁷, which combines adaptive gradient confidence and dynamic learning rate boundaries, represents a promising direction for further improving training stability and convergence. Beyond optimizer choice, recent metaheuristic methods such as the Secant Optimization Algorithm (SOA)⁵⁸ and the Schrödinger Optimizer (SRA)⁵⁹ offer complementary potential: their adaptive exploration-exploitation balance is conceptually aligned with the dynamic weighting in Ada-KD, making them promising candidates for automated hyperparameter search in the GSA-YOLO pipeline.

Author contributions

J.K. conceived and designed the study, performed data collection and statistical analysis, interpreted the results, and wrote the main manuscript text. J.K. also prepared all tables and figures. The author reviewed and approved the final manuscript.

Funding

The authors did not receive support from any organization for the submitted work.

Data availability

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Declarations

Competing interests

The authors declare no competing interests.