Metaheuristic-driven LSTM framework for advanced cyberthreat detection and response using a hybrid firefly–whale–grey wolf optimization pipeline

Abstract

Conventional intrusion detection systems (IDSs) based on static, signature-based rules are becoming insufficient in response to more advanced, dynamic cyber threats like advanced persistent threats (APTs), zero-day exploits and polymorphic malware The paper presents a new three-step hybrid optimization model based on the combination of Long Short-Term Memory (LSTM) networks, which is a gated form of recurrent neural networks (RNNs) specifically developed to learn sequential patterns, with three complementary bio-inspired metaheuristic algorithms, each contributing to the specific role of optimization. Firefly algorithm (FA) is used to select the features subset and decrease the input dimensionality, whale optimization algorithm (WOA) is used to optimize LSTM parameters such as hidden units, learning rate, dropout rate, and batch size, and grey wolf optimizer (GWO) to perform weight optimization at the ensemble level to improve the robustness of the classification. The framework has been tested on the NSL-KDD and CICIDS2017 benchmarks on a stringent protocol of 30 independent runs. The Wilcoxon signed-rank test (p < 0.001) and paired t-test (p < 0.001) prove statistical significance. The hybrid model is 98.62% (standard deviation 0.21) on NSL-KDD and 99.1% (standard deviation 0.18) on CICIDS2017 with a false positive rate (FPR) of 1.2 and an area under the ROC curve (AUC-ROC) of 0.992. The suggested framework has a high potential of being used as the basis of detecting near-real-time cyberthreats, though it will require additional validation on real-life network settings.

Introduction

The contemporary digital environment is marked by a continuous growing interconnection of critical infrastructure, cloud services, and devices of the Internet of Things (IoT), which increase the attack surface available to malicious actors. The degree of cyber threats has become significantly more advanced, prevalent, and extensive: the cost of cybercrime worldwide is expected to reach USD 10.5 trillion per year by 2025¹. Routine examples of threats that do not fall under the traditional security controls are advanced persistent threats (APTs), zero-day vulnerabilities, distributed denial-of-service (DDoS) campaigns, and polymorphic malware. The classic intrusion detection systems (IDSs) which are mainly rule based or signature driven in nature are inherently reactive: they can only detect attack patterns which are listed in the catalog and cannot handle new or obfuscated attack vectors. This is a critical weakness in the contemporary cybersecurity infrastructure and it requires more adaptive proactive solutions.

The artificial intelligence (AI) and deep learning (DL) have become revolutionary technologies in proactive threat detection. The deep neural architecture is capable of learning hierarchical features representations that are directly based on the network traffic data, which allows detecting new patterns that have never been previously seen. Gated versions of a recurrent neural network (RNN) also known as Long Short-Term Memory (LSTM) networks which solve the issue of the vanishing gradient are highly suitable in modeling the sequential character of network traffic flows. Autoencoders are a complement to the LSTM architectures where the compact structural representation is learned to detect anomalies. These benefits notwithstanding, deep learning models are highly vulnerable to architectural decisions and hyperparameter settings, and their performance has to be carefully fine-tuned to be able to generalize to a variety of threat settings.

Bio-inspired metaheuristic optimization algorithms provide a conceptual method to the hyperparameter tuning and feature selection difficulties of deep learning-based IDS. Firefly algorithm (FA), whale optimization algorithm (WOA) and grey wolf optimizer (GWO) emulate the collective behaviours of nature to search non-convex, multidimensional search spaces. Nevertheless, the current literature indicates that there is a research gap: the existing studies are largely based on the application of a single metaheuristic optimizer (e.g., GWO-tuned LSTM², WOA-tuned LSTM³, or FA-guided feature selection⁴ without taking the complementary benefits of multiple optimizers into account and using them in an integrated pipeline. In particular: (1) single-optimizer models tend to converge prematurely and lack search diversity; (2) the sequential combination of FA (discrete combinatorial searching feature selection), WOA (continuous spiral-based search hyperparameter tuning), and GWO (hierarchical rank-based ensemble optimization) has not been systematically studied in the literature as a three-stage pipeline; and (3) statistically validated repetitions of experiments have not been done on most studies of IDS metaheuristics.

The paper presents a new three-phase hybrid metahybrid-optimized LSTM model of enhanced cyberthreat detection. The framework allocates each optimizer a different, non-overlapping task according to its natural strengths: FA is feature subset selection in a discrete search space; WOA is LSTM hyperparameter optimization in a continuous search space using its spiral bubble-net dynamics; and GWO is ensemble-level weight optimization using its 3-level 3-leaders hierarchical leadership model. It is this architectural division of concerns and the addition of a multi-objective fitness function to combine accuracy, F1-score, and FPR which make up the fundamental innovation of the proposed approach.

Main contributions

The principal contributions of this work are:

• A new three-phase sequential metaheuristic pipeline (FA → WOA → GWO) of LSTM-based IDS, in which each optimizer solves a different optimization sub-problem serving a particular task.

• A multi-objective fitness function, which is formally defined and optimizes the detection accuracy, the macro-averaged F1-score, and the false positive rate together.

• A stringent experimental procedure of 30 free trials of Wilcoxon signed-rank test and paired t-test statistical validation - more rigorous than any other similar previous studies.

• A Study where each component of the optimizers is studied individually and the three-stage design is empirically justified.

• Clearly stated limitations with a future validation plan on current data such as CICIoT2023.

• Explicit acknowledgment of limitations with a roadmap for future validation on contemporary datasets including CICIoT2023.

The rest of this paper is structured in the following way. Section  2 is a literature review. Section  3, implies the objectives of the research. Section  4 tells us about the datasets. The methodology is provided in Sect.  5. The mathematical framework is described in Sect.  6. In Sect.  7, there is performance evaluation and benchmarking. The results are presented in Sect.  8. Section  9 provides the limitations. Discussion is provided in Sect.  10. Future directions are the last section of Sect.  11.

Literature review

Deep learning architectures for intrusion detection

Intrusion detection has been utilizing deep neural architectures to learn more intricate time and structure variations of network traffic. Singh and Jang-Jaccard⁵ suggested the idea of a multi-scale CNN-LSTM autoencoder that learns spatio-temporal dependencies in an unsupervised approach, which has proven to be more effective on NSL-KDD and UNSW-NB15 databases. Al-Kahtani et al.⁶ improved the performance of the IDS in wireless sensor networks by combining gated recurrent units (GRUs) with LSTM networks and showed their effectiveness in a variety of attack types. Hussein et al.⁷ demonstrated that LSTM networks with SMOTE-based rebalancing demonstrated a considerable enhancement in intrusion detection performance on both CICIDs2017 and NSL-KDD, especially on unrepresented attack classes. The LSTM autoencoders and stacked bi-directional models of LSTM designed by Kumar et al.⁸ were used to identify IoT intrusions and performed better on UNSW-NB15 and Bot-IoT datasets than classical machine learning baselines. Gueriani et al.⁹ made a CNN-LSTM hybrid architecture tested on the current CICIoT2023 dataset with 98.42% accuracy, proving the practicability of the deep hybrid structures in the modern IoT threat context.

Metaheuristic-optimized IDS frameworks

Bio-inspired metaheuristic algorithms and deep learning of IDS have been actively studied. Elsaid et al.¹⁰ introduced the optimized grey wolf-based deep learning hybrid (LSTM-GRU) IDS that has a higher detection accuracy. In a CNN-LSTM IDS, Aljanabi and Kumaran¹¹ used the grey wolf optimizer to select features, and they achieved better precision and recall. Abdel-Husseini et al.¹² used whale optimization algorithm to optimize LSTM classifiers hyperparameters, and the highest accuracy was 99.6% on NSL-KDD. Khan et al.¹³ came up with an explainable LSTM-based IDS, which is optimized using the firefly algorithm in an IoT environment, where LIME and SHAP are used to provide model transparency and a model with an accuracy of 98.42. In network traffic, Kumar and Sinha¹⁴ used a firefly algorithm-adjusted LSTM to identify anomalies, showing it to be superior in NSL-KDD and UNSW-NB15. Vijayanand et al.¹⁵ presented a wrapper feature selection method of IDS, which uses a whale optimization algorithm. Mojtahedi et al.¹⁶ showed that whale optimization algorithm that is used in conjunction with a genetic algorithm and KNN outperforms in feature selection to IDS on KDD Cup 1999. Kunang et al.¹⁷ introduced whale and remora-inspired algorithms to a CNN-based IDS with a better detection rate and reduced false positives. Sharma and Tripathi¹⁸ suggested an enhanced ensemble model with swarm intelligence that employs a metaheuristic, demonstrating an improvement in using a combination of traditional and deep learning classifiers.

Recent advances in metaheuristic-tuned ML/DL for cybersecurity (2023–2025)

A number of literature published in recent times also enlighten the development of the proposed framework. According to Mabrouk et al.¹⁹, hybrid CNN-XGBoost IDS fine-tuned using a modified sine cosine algorithm have proven the effectiveness of metaheuristic tuning on boosted tree classifiers on cloud security datasets. A two-level IoT/metaverse intrusion detection framework that was suggested by Dahou et al.²⁰ and applies metaheuristic optimization to independently tune feature selection and classification phases - a design rationale that is similarly close to the separation of stages offered herein. A survey of metaheuristic optimization was conducted by Slimani et al.²¹ on the topic of IoT/IIoT intrusion detection, with feature selection and hyperparameter optimization being the most significant optimization objectives. Ghosh et al.²² used a CatBoost classifier that was optimized using an adapted reptile search algorithm of IDS and performed competitively on CICIDS2017. Using boosting-based smart city IDS which was optimized by metaheuristics Ibrahim et al.²³ suggested that ensemble techniques could be improved by optimizer-coordinated weight coordination, which is of direct interest to the GWO ensemble optimization step in the suggested framework. Mabrook et al.²⁴ used NLP-based insider threat detection as an implementation of the metaheuristic optimization to NLP-based optimization of network-flow data, which is not limited to network-flow data.

Other supporting works are: Growth optimizer and whale optimization algorithm were integrated in deep learning IDS by Rajasekhar and Ghosh²⁵ when using cloud/iot datasets; CNN-based IDS were applied by Rani and Das²⁶ using hybrid filter techniques; Arjunan and Suresh²⁷ developed a variational autoencoder-cnn hybrid IDS; and Jain et al.²⁸ created a whale optimization algorithm-CNN solution to malware classification. Taken together, these publications both affirm the topicality of the offered framework and point out the particular gap that remains unexploited: no previous study has implemented FA, WOA, and GWO in a pipeline separated by stages with complete statistical validation of all of the independent runs.

Recent advancements in ensemble-based intrusion detection for cyber-physical systems further highlight the importance of coordinated optimization strategies. Ji et al.²⁹ proposed cascading ensemble techniques for IDS, while Ji et al.³⁰ provided a comprehensive review of intrusion detection in cyber-physical systems. Hybrid ensemble optimization approaches such as CNN-GWO-voting models³¹ and feature selection frameworks³² demonstrate improved detection performance. Furthermore, meta-learning-based ensemble strategies³³ and quantum-inspired secure frameworks³⁴ indicate the growing importance of adaptive and hybrid optimization techniques in modern cybersecurity environments.

Research objectives

The main aim of the proposed study is to develop, deploy and strictly test a new 3-phase hybrid metaheuristic-optimized LSTM model of high-level cyberthreat detection. The particular aims are: (1) to design a stage-separated pipeline with FA, WOA, and GWO solving a different optimization sub-problem according to their natural computational capabilities; (2) to evaluate the detection performance of the framework on NSL-KDD and CICIoT2017 in terms of accuracy, macro-averaged F1-score, Matthews Correlation Coefficient (MCC), per-class precision and recall, FPR, and AUC-ROC; (3) to test statistical significance testing across 30 independent runs so that performance improvement.

The proposed framework can be distinguished by three key aspects of the previous works (GWO-LSTM¹⁰, WOA-LSTM¹², FA-LSTM¹³: (1) role specialization - FA is used to trial combinatorial feature selection and WOA is used to trial continuous hyperparameter optimization and GWO is used to trial ensemble weight coordination - has not been studied in the previous literature as an integrated three-stage pipeline; (2) the multi-objective fitness function that is used to optimize accuracy, F1-score, and FPR simultaneously offers.

Research datasets

The proposed framework is tested on two benchmark datasets NSL-KDD and CICIDS2017. The primary benchmark is NSL-KDD which is necessary to allow making a direct comparison with the enormous amount of previous literature on IDS. The more recent categories of attacks are evaluated at CICIDS2017. It is clearly admitted that NSL-KDD is based on network captures of 19,981,999, thus, it does not represent current encrypted traffic, zero-day activities, or IoT-specific attack patterns. Although more recent, CICIDS2017 is pre-2020 in terms of cyberthreats. Future confirmation on CICIoT2023⁹- including modern patterns of IoT attacks such as MQTT-based attacks and advanced reconnaissance - is quoted as a high-priority extension of this work.

NSL-KDD dataset

NSL-KDD is an improved version of KDD Cup 1999, which is used to overcome the problems of duplicate records and class imbalance³⁵. It has 41 items (38 numerical, 3 categorical) that are network connection characteristics, denoted by the term normal and in the four attack types: Denial of Service (DoS), Probe, Remote-to-Local (R2L), and User-to-Root (U2R). The training (KADDTrain +) has 125,973 instances and the test set (KADDTest +) has 22,544 instances. The dataset has issues of class imbalance especially with U2R (< 0.1% of training instances) which is resolved by using SMOTE-based oversampling as part of the preprocessing pipeline (Fig. 1, Fig. 2 and Fig. 3).

Fig. 1.

NSL-KDD Dataset sample (top) and CICIDS2017 Dataset sample (bottom).

Fig. 2.

CICIDS2017 Dataset sample showing feature columns and labelled traffic flows.

Fig. 3.

Distribution of classes in the NSL-KDD dataset by attack type and instance count.

CICIDS2017 dataset

The dataset of CICIDS2017³⁶ includes records of network flows during a period of 5 days of monitored traffic which consists of 79 features of each flow obtained with the help of CICFlowMeter, and which have labels Benign traffic and 14 attack categories. The flow-based design of the dataset and the presence of real timestamps allow constructing sequences to train LSTM more principled compared to NSL-KDD. It also has a substantially different feature space, as well as an attack distribution, which makes it useful in assessing cross-dataset generalization.

Methodology

Data preprocessing — NSL-KDD

The preprocessing pipeline for NSL-KDD comprises: (1) Data sanitization: removal of duplicate records, imputation of missing values, and label consistency verification. (2) Categorical encoding: the three categorical features (protocol_type, service, flag) are one-hot encoded. (3) Label mapping: normal traffic → 0; attack traffic → {DoS, Probe, R2L, U2R} for multi-class classification. (4) Normalization: Min-Max scaling to [0,1] applied to all numerical features using training-set statistics only.

Data preprocessing — CICIDS2017

CICIDS2017 preprocessing includes: (1) flow-based feature aggregation grouping packets with matching source-destination IP-port tuples; (2) timestamp processing converting raw timestamps to inter-arrival delta features; (3) attack label generalization mapping fine-grained labels (e.g., DoS-Slowloris) to coarse categories (DoS, Brute Force, DDoS); and (4) SMOTE-based oversampling to address class imbalance for underrepresented categories such as Heartbleed and Infiltration.

Sequence construction for LSTM input

The LSTM networks demand sequential input in the format (samples, timesteps, features). NSL-KDD is not sequential and is tabular in nature, which requires synthesized sequences. KDDTrain+ records are indexed by their original row ID which is a rough approximation of time-ordering since the data were compiled by recording network captures in time. It uses a sliding window of size T = 10 and a stride S = 1, which generates overlapping sequences, S i = {x i, x i + 1, x i + 2, and so on}, in which x j is the preprocessed feature vector of the i th record. The label of sequence S i is the label of the last record x i + T -1. This will give N records as N -T + 1 sequences. It is clearly stated that this method presupposes that there is local temporal locality among the records at nearby location - a flawed assumption when using tabular data. In order to evaluate the value of temporal modeling by LSTM, non-sequential baselines (Random Forest, SVM) that are trained on the same feature subset that FA selected are also presented in comparative assessment. In the case of CICIDS2017, the pseudo time per-flow timestamps allow constructing chronological sequences, and the method is more principled in the case of this dataset.

Class imbalance handling

SMOTE generates synthetic minority-class samples by interpolating between existing minority instances in feature space. Class weights inversely proportional to class frequency are additionally applied in the LSTM loss function to further compensate for residual imbalance, particularly for U2R instances (Fig. 4 and Table 1).

Fig. 4.

Flowchart depicting the complete hybrid FA–WOA–GWO–LSTM pipeline, from data ingestion through metaheuristic optimization to final model evaluation.

Table 1.

Mathematical notation used throughout Sect.  6.

Symbol	Definition
X ∈ ℝⁿˣᵀˣᶠ	Input tensor: n = samples, T = time steps (window size = 10), f = features after FA selection
Y ∈ {0,…,C-1}ⁿ	Ground-truth label vector; C = number of classes
Θ	Trainable LSTM parameters (weight matrices W, recurrent matrices U, bias vectors b)
Φ	Hyperparameter set: {L layers, h hidden units, δ dropout, η learning rate, β batch size}
H ∈ ℝⁿˣᵈ	Hidden state output from final LSTM layer; d = output dimensionality
F(Φ)	Multi-objective fitness function for metaheuristic search (Eq. 10)

Mathematical framework of the hybrid metaheuristic-optimized LSTM model

LSTM gate equations

An LSTM cell processes input x_t at each time step t through four gating mechanisms, enabling selective retention and forgetting of information. The complete gate equations are:

1

2

3

4

5

6

where σ(·) is the sigmoid activation, tanh(·) is the hyperbolic tangent, and ⊙ is the Hadamard (element-wise) product. W_*, U_* are input and recurrent weight matrices; b_* are bias vectors. The Long Short-Term Memory (LSTM) network processes sequential input by regulating information flow through gating mechanisms. The input gate controls how much new information is stored in the cell state, as defined in Eq. (1). Similarly, the forget gate determines which previous information should be discarded, as shown in Eq. (2). The candidate cell state computes new potential information to be added, as formulated in Eq. (3).

The overall cell state is then updated by combining retained past information and new candidate values, as described in Eq. (4). The output gate regulates the information passed to the hidden state, as expressed in Eq. (5). Finally, the hidden state output, which serves as the representation for subsequent layers or classification, is computed according to Eq. (6).

Classification and loss function

7

The training objective minimizes the categorical cross-entropy loss with L2 regularization:

8

where 1[y_i = c] is the indicator function, and λ is the L2 regularization weight. The final hidden representation is passed through a classification layer to produce class probabilities using a softmax function, as defined in Eq. (7). The model is trained by minimizing the categorical cross-entropy loss with L2 regularization to prevent overfitting, as given in Eq. (8).

Min-max normalization

9

where min_j and max_j are computed from the training set only. To ensure stable training and prevent feature dominance, all numerical inputs are normalized using Min-Max scaling. This transformation rescales features into the range , as defined in Eq. (9).

Multi-objective fitness function

10

where w₁ = 0.40 (accuracy), w₂ = 0.35 (F1-macro), w₃ = 0.20 (FPR penalty), w₄ = 0.05 (complexity penalty). Fitness is negated so that minimization-oriented optimizers search for maximum quality solutions. The proposed framework employs a multi-objective fitness function that jointly optimizes detection accuracy, F1-score, false positive rate (FPR), and model complexity. This combined objective function is formally defined in Eq. (10), where weighted contributions ensure balanced optimization across performance metrics.

Stage 1: firefly algorithm (FA) — feature selection

FA is applied to select an optimal feature subset s ∈ {0,1}ᶠ. Attractiveness between firefly i and j:

11

Movement update for firefly i toward brighter firefly j:

12

where β₀ = 1.0 (max attractiveness), γ = 1.0 (absorption coefficient), α = 0.5 (randomization), r_{ij} = Euclidean distance between fireflies i and j, rand ~ U(0,1). Continuous values are mapped to binary via a sigmoid transfer function V(x) = 1/(1 + e^{−x}). In the feature selection stage, the Firefly Algorithm (FA) models the attractiveness between candidate solutions based on their brightness (fitness). The attractiveness between two fireflies is computed as a function of their distance, as shown in Eq. (11). Based on this attractiveness, the movement of a firefly toward a better solution is governed by the update rule defined in Eq. (12), which incorporates both deterministic attraction and stochastic exploration.

Stage 2: whale optimization algorithm (WOA) — hyperparameter tuning

WOA optimizes Φ = [L, h, δ, η, β] via encircling prey and spiral bubble-net mechanisms.

Encircling prey (exploitation, |A| < 1):

13

Spiral bubble-net update (probability p = 0.5):

14

Exploration update (|A| ≥ 1):

15

where X* is the best whale position; A = 2a·r − a; C = 2r; r ~ U(0,1); a decreases linearly from 2 to 0; b = 1 (spiral constant); l ~ U(− 1, 1). The Whale Optimization Algorithm (WOA) is employed to optimize LSTM hyperparameters through exploration and exploitation strategies. During exploitation, whales update their positions by encircling the best solution, as described in Eq. (13). Additionally, a spiral bubble-net attacking mechanism is used to simulate the hunting behavior, as formulated in Eq. (14). When exploration is required, whales search for new solutions using the update rule defined in Eq. (15).

Stage 3: grey wolf optimizer (GWO) — ensemble optimization

GWO optimizes ensemble weights w = [w₁,…,w_K] (Σw_k = 1, w_k ≥ 0). GWO maintains three leader positions (α, β, δ):

16

17

18

where A_k = 2a·r_k − a; C_k = 2r_k; r_k ~ U(0,1); a decreases linearly from 2 to 0. The Grey Wolf Optimizer (GWO) simulates the hierarchical leadership structure of wolf packs to optimize ensemble weights. The positions of candidate solutions are updated based on three leading wolves (alpha, beta, and delta). The position updates influenced by these leaders are computed using Eq. (16), Eq. (17), and Eq. (18), respectively. These equations collectively guide the search process toward optimal ensemble configurations.

Final prediction:(19).

19

The final prediction of the ensemble is obtained as a weighted combination of individual model outputs, as defined in Eq. (19), (Table 2 and 3)

Table 2.

Complete metaheuristic parameter settings. All optimizers use population N = 30 and 100 iterations per run over 30 independent runs.

Parameter	Value	Applies To
Population size (N)	30	FA, WOA, GWO
Maximum iterations	100	FA, WOA, GWO
Independent runs	30 (seeds 0–29)	All
FA: randomness α	0.5	FA
FA: max attractiveness β₀	1.0	FA
FA: absorption coefficient γ	1.0	FA
WOA: spiral constant b	1.0	WOA
WOA: a range	2.0 → 0 (linear decay)	WOA
GWO: a range	2.0 → 0 (linear decay)	GWO
LSTM hidden units h	[32, 256]	WOA search space
LSTM layers L	[1, 4]	WOA search space
Learning rate η	[0.0001, 0.01]	WOA search space
Dropout rate δ	[0.1, 0.5]	WOA search space

Table 3.

Optimization statistics across 30 independent runs (NSL-KDD). Baseline standalone LSTM: 94.2% ± 0.47% accuracy.

Optimizer	Metric	Best	Worst	Mean	Std	vs. LSTM
FA+LSTM	Acc (%)	96.41	95.12	95.83	0.31	+ 1.6%
FA+LSTM	F1 (%)	95.90	94.60	95.20	0.33	+ 2.1%
WOA+LSTM	Acc (%)	97.23	96.10	96.71	0.28	+ 2.5%
WOA+LSTM	F1 (%)	96.80	95.50	96.10	0.31	+ 2.9%
GWO+LSTM	Acc (%)	96.95	95.80	96.42	0.29	+ 2.2%
GWO+LSTM	F1 (%)	96.40	95.10	95.80	0.32	+ 2.6%
FA + WOA+GWO+LSTM	Acc (%)	98.98	98.11	98.62	0.21	+ 4.4%
FA + WOA+GWO+LSTM	F1 (%)	98.50	97.60	98.12	0.23	+ 4.9%
FA + WOA+GWO+LSTM	FPR (%)	0.90	1.55	1.20	0.16	−4.1%
FA + WOA+GWO+LSTM	AUC	0.997	0.987	0.992	0.003	+ 0.044

Performance evaluation and benchmarking

Experimental setup

All tests were performed on the next set-up: Intel Core i7-11800 H CPU (2.3 GHz, 8 cores), 32 GB DDR4 RAM, NVIDIA RTX 3060 (6 GB VRAM) graphics. Software Python (version 3.10), TensorFlow (version 2.12), Keras (built-in), PySwarms (version 1.3.0), scikit-learn (version 1.3.0), imbalanced-learn (version 0.11.0). Split of data: 70 training, 15 validation, 15 test (stratified). K-fold cross- validation (k = 5) used in training set to evaluate fitness. Multiple experiments with seeds 029 were repeated 30 times. Upon request, code is availed (Fig. 5 and Fig. 6).

Fig. 5.

Performance comparison chart — proposed model vs. baseline models across key evaluation metrics on NSL-KDD.

Fig. 6.

Convergence curves for the best run of FA, WOA, GWO, and the full FA + WOA+GWO pipeline, showing fitness value vs. iteration number.

Statistical significance testing

To ensure that the proposed model is better than standalone LSTM, two statistical tests have been done on 30 independent runs: Wilcoxon signed-rank test (non-parametric): W = 428, p = less than 0.001 (two-tailed). Large effect size r = 0.78. Makes statistically significant improvement. • Paired t-test (parametric): t(29) = 18.63, p < 0.001. Mean difference in accuracy = 4.49% (98.62% vs. 94.13) proven to be statistically significant (Tables 4, 5 and 6).

Table 4.

Ablation study results (NSL-KDD, 30 runs, mean ± std). ΔAcc = improvement relative to LSTM-only baseline.

Configuration	Acc (%)	F1-macro (%)	FPR (%)	AUC	ΔAcc
(a) LSTM only	94.2 ± 0.47	93.1 ± 0.51	5.3 ± 0.38	0.948	—
(b) FA + LSTM	95.8 ± 0.31	94.7 ± 0.34	3.9 ± 0.28	0.962	+ 1.6%
(c) WOA + LSTM	96.7 ± 0.28	95.6 ± 0.30	2.8 ± 0.22	0.971	+ 2.5%
(d) GWO + LSTM	96.4 ± 0.29	95.3 ± 0.31	3.1 ± 0.25	0.968	+ 2.2%
(e) FA + WOA + LSTM	97.8 ± 0.25	96.9 ± 0.28	1.8 ± 0.19	0.981	+ 3.6%
(f) FA + GWO + LSTM	97.2 ± 0.27	96.1 ± 0.29	2.2 ± 0.21	0.975	+ 3.0%
(g) Full FA + WOA+GWO+LSTM [Proposed]	98.6 ± 0.21	98.1 ± 0.23	1.2 ± 0.16	0.992	+ 4.4%

Table 5.

Comparative performance on NSL-KDD. Non-sequential models (RF, SVM, DT) trained on identical FA-selected features. MCC = Matthews Correlation Coefficient. Proposed model highlighted in green.

Model	Acc (%)	Prec (%)	Recall (%)	F1 (%)	FPR (%)	AUC	MCC
Proposed (FA + WOA+GWO+LSTM)	98.6	98.4	98.9	98.6	1.2	0.992	0.971
WOA+LSTM [12]	96.7	96.1	96.4	96.2	2.8	0.971	0.934
GWO+LSTM [10]	96.4	95.8	96.0	95.9	3.1	0.968	0.929
Standalone LSTM	94.2	92.5	91.8	92.1	5.3	0.948	0.884
CNN-LSTM	96.1	94.8	93.5	94.1	3.1	0.965	0.922
Random Forest (non-seq.)	89.7	88.3	87.6	87.9	8.9	0.921	0.795
SVM (non-sequential)	85.4	83.1	82.0	82.5	12.3	0.891	0.708
Decision Tree (non-seq.)	82.1	80.4	79.8	80.1	15.2	0.862	0.643

Table 6.

Summary of key results and interpretive remarks with section/table cross-references.

Dimension	Key Finding	Remark	Reference
Accuracy	98.62% ± 0.21% (NSL-KDD); 99.1% ± 0.18% (CICIDS2017)	Validates metaheuristic fine-tuning effectiveness	Tables 3 and 5
FPR	1.2% ± 0.16% vs. 5.3% ± 0.38% (LSTM baseline)	4.1% FPR reduction reduces SOC alert fatigue	Table 5
Statistical test	Wilcoxon W = 428 (p < 0.001); t(29) = 18.63 (p < 0.001)	Non-trivial and reproducible improvements	Section 7.3
MCC	0.971 (proposed) vs. 0.884 (LSTM)	Balanced performance including minority classes	Table 5
U2R detection	93.7% — lower than other classes	Class imbalance; future: cost-sensitive learning	Section 7.4
Training time	1,850 s (RTX 3060 — one-time offline cost)	High but justified; inference < 50 ms same HW	Section 7.1
CICIDS2017	99.1% accuracy	Generalization confirmed; CICIoT2023 is future work	Table 6 (Appx)

Results

The presented FA + WOA+GWO+LSTM model shows repeatedly better results on NSL-KDD and CICIDS2017 data sets, confirmed by 30 experimental runs and confirmed by stringent statistical analysis. On NSL-KDD, the model has mean accuracy 98.62% + 0.21% - statistically significant + 4.42% better than standalone LSTM (94.2% + 0.47; W = 428, p < 0.001; paired t(29) = 18.63, p < 0.001). The FPR has been minimized to 1.2% +0.16, which is 4.1% points lower than standalone LSTM, which is essential in reducing alert fatigue in the Security Operations Centers (SOCs). On the CNN-LSTM baseline (0.965), the AUC-ROC of 0.992 is significantly lower, which proves the discriminative power. The Matthews Correlation Coefficient (MCC = 0.971) ensures that the balanced performance is strong in all classes even the minority attacks types.

Super-additive synergy is confirmed by the fact that each of the optimizer stages adds incrementally: FA alone (+ 1.6%), WOA alone (+ 2.5%), GWO alone (+ 2.2%), the three stages of the pipeline altogether (+ 4.4%), and it is super-additive. It is noteworthy that non-sequential baselines (RF: 89.7% SVM: 85.4) with the same subset of features selected by FA prove that the temporal modeling of LSTM does add real value to feature quality by itself.

DoS and Probe have a high detection rate (99.2 and 98.5 respectively) with attack-type analysis. U2R detection stands at 93.7% - worse since the difference in classes is extreme. To some extent, SMOTE-based oversampling addresses this; cost-sensitive learning is a priority in the future. In CICIDS2017, the model gets an accuracy of 99.1% with a ± error of 0.18% which proves the use of reasonable cross-dataset generalization.

With respect to computing properties: 1,850 s training in the given hardware (Intel i7-11800 H, RTX 3060) is a one-time offline expense. The time of the inference is invariably lower than 50 ms per batch on the same machine. It is clearly mentioned that this inference time is platform specific and might vary on resource constrained or embedded deployment platforms. Benchmarking on live-traffic throughput and memory profiling must go through before responsibly making a production deployment claim Fig. 7 and Fig. 8.

Fig. 7.

ROC curves for the proposed model and baselines on NSL-KDD, illustrating AUC-ROC improvements.

Fig. 8.

Confusion matrix for the proposed FA + WOA+GWO+LSTM model on NSL-KDD test set, showing per-class detection performance including U2R (lower right).

Limitations

The limitations of the present research are clearly admitted as follows:

1. Currency of benchmark dataset currency NSL-KDD (19981999 captures) is not representative of current encrypted traffic, zero-day exploits, APTs, and IoT attack patterns. CICIDS2017 is older than cyberthreat of 2020s. Findings might not be applicable to contemporary settings unless they are re-validated on data sets, like CICIoT 2023.

2. NSL-KDD Synthetic sequence construction: The sliding-window method used on NSL-KDD presupposes local temporal locality among neighboring records - not a perfect hypothesis to make about inherently tabular data. Other approaches have not been explored (session-based grouping, graph-based flow ordering).

3. Cost of training computationally: 1,850 s training time can restrict applicable in settings that demand a rapid model retraining as concept drift occurs. Future work is referred to as online/incremental learning strategies.

4. Class imbalance against rare attacks: There is still class imbalance with U2R accuracy (93.7) being worse than other types because the number of extreme classes is extremely small. Generative data augmentation and learning which is cost-sensitive should be tried.

5. Lack of live-traffic testing: There was no experiment of live network traffic; no memory footprint testing, throughput testing, and scalability testing were done. Unless such an evaluation is done, claims of the suitability of real time or production deployment cannot be substantiated.

6. Hardware-specific inference time: The inference latency (less than 50 ms) reported is only applicable to Intel i7-11800 H/RTX 3060 and cannot be replicated in embedded systems or edge devices.

7. Adversarial robustness: The structure has not been tested on adversarily generated traffic that attempts to defeat the IDS. Resistance to gradient-based and mimicry attacks is an unsolved question.

Discussion

The given FA + WOA+GWO+LSTM framework proves that the results of the synergistic combination of three independent complementary metaheuristic optimizers in a stage-separated pipeline are statistically significant and their effect is superior to that of individual-optimizer models as well as to traditional deep learning benchmarks. The ablation experiment (Table 4) makes sure that there are non-redundant, distinct contributions of every optimization phase. The fact that the full pipeline (super-additive 4.4- vs. single-optimizer 2.5-performance) operates better than the strong single-optimizer performance, is evidence that the synergy is real rather than due to combination of redundant processes.

The difference between the FPR of 5.3 to 1.2 is operationally important. In a network with a high throughput of 10,000 connections per second, a 4.1% FPR decrease would mean 410 false alarms/sec, which is a significant enhancement to the efficiency of SOC operation. This improvement is statistically sound as shown by the Wilcoxon signed-rank test (p < 0.001) and the paired t-test (p < 0.001) in 30 independent runs.

The reduced U2R detection rates (93.7) are evidence of the inherent difficulty in extreme class imbalance, and are in line with the rest of the IDS literature^7,13, and prompts further research on cost-sensitive learning. The cross-dataset CICIDS2017 (99.1) results indicate that generalization is reasonable, however, the distribution shift between datasets should be interpreted with care.

The 1,850 s training time is significantly greater than conventional baselines. This is a cost that is incurred once in the offline optimization stage. The latency of inference, of less than 50 ms (measured on Intel i7-11800 H/RTX 3060) is also consistent with near-real-time to classify data in the laboratories. Nevertheless, according to the requirements of the responsible deployment claims, production environment validation must be provided.

Conclusion

The paper has suggested a new three-step hybrid metaheuristic-optimized LSTM architecture to detect advanced cyberthreats, with the firefly algorithm (FA) to select the best feature subset, the whale optimization algorithm (WOA) to optimize LSTM hyperparameters, and the grey wolf optimizer (GWO) to optimize the weight of the ensemble in a stage-based pipeline. The main results are the following: (1) the full FA + WOA+GWO+LSTM pipeline yields 98.62% ± 0.21% mean accuracy on NSL-KDD, and 99.1% ± 0.18% mean accuracy on CICIDS2017, and is statistically significantly greater than all of the baselines using Wilcoxon signed-rank and paired t-tests (p < 0.001) across 30 independent runs; (2) all of the optimizer components have a statistically significant Weaknesses such as age of benchmark dataset, assumptions on synthetic sequence construction, high cost of training and lack of live-traffic testing is explicitly recognized. These will be covered in the future by CICIoT2023 assessment, adversarial robustness, and explainability method combination implementation to deploy production SOC.

Future work

Potential extensions of the proposed framework are: (1) testing on CICIoT2023 and CIC-Bell-DNS-2021 datasets to test its performance against current attack patterns; (2) using Harris Hawks Optimization (HHO) or Particle Swarm Optimization (PSO) to explore the optimization space further; (3) provide explainability with SHAP and LIME to interpret the SOC alerts; (4) edge deployment optimization with model compression and quantization to support IoT/5G edge deployments; (5) adversarial robust.

Author contributions

All Authors contributed equally for the sections mentioned in manuscript.

Funding

The authors extend their appreciation to the Nitte (Deemed to be University).

Data availability

The datasets generated and/or analysed during the current study are available https://www.kaggle.com/datasets/hassan06/nslkdd and https://www.unb.ca/cic/datasets/ids-2017.html. These datasets are public access datasets for research.

Declarations

Competing interests

The authors declare no competing interests.