Reinforcement learning enabled hybrid optimisation for energy-efficient multipath routing in wireless sensor networks

Abstract

Wireless sensor networks have the major challenge of ensuring energy efficiency, scalability, and data delivery. The achievement of these goals becomes challenging when the nodes have limited battery energy, the network topology changes frequently, and the data transfer is not distributed among the nodes. The routing protocols have been found wanting in various ways, as they consume more energy than required, have difficulties selecting the appropriate nodes as cluster heads, and cannot be adapted when the network topology changes frequently. This eventually reduces the lifespan of the network and affects its performance as a whole. To mitigate these problems, this paper presents a Reinforcement Learning-based Multipath Hybrid Whale-Grey Wolf Optimization framework. Deep Reinforcement Learning is incorporated throughout this framework for adaptive sleep scheduling and node activation. The DRL agent learn and schedules tasks based on residual energy, node centrality, and node proximity to the cluster head as primary factors. This research also suggests a new hybrid optimization algorithm, where the global search capability of the Whale Optimization Algorithm (WOA) and the strong convergence and exploitation capabilities of the Grey Wolf Optimizer (GWO) are hybridized for more effective cluster head selection. This hybridization process is done for a more effective and energy-efficient selection of cluster heads. It has also been proposed that a multipath routing technique will be used to develop multiple stable and energy-efficient paths between the cluster heads and the base station. The stability of the paths, the energy left, and the quality of the paths will be considered while establishing the paths using the hybrid WOA–GWO algorithm. The effectiveness of the proposed framework will be validated through extensive simulations, which prove that the proposed model performs well with a packet delivery ratio of 97.8%, total energy consumption of 320 J, and a total throughput at the base station of 72 kbps.

Introduction

Wireless Sensor Networks (WSNs) have emerged as the backbone for the sensing and monitoring requirements of the current era. The applications of WSNs include various real-time data acquisition requirements. Hence, it is essential to focus on achieving efficiency in WSN protocol design¹. In WSNs, the process of routing is responsible for forwarding the information collected from various sensor nodes to the Base Station (BS)². This is done through multi-hop communication. However, it is difficult for traditional routing methods to handle these situations efficiently, leading to energy holes around the Base Station, dying of nodes before their actual time, and a reduction in network life. To avoid these problems, new routing protocols have been proposed for achieving maximum energy efficiency, balancing the load of all the sensor nodes, and ensuring reliable delivery of packets.

Among these routing protocols, cluster-based routing is found to be highly efficient. In this type of routing protocol, a number of sensor nodes are grouped together as clusters and are served by a cluster head that collects and transmits information to the base station. This reduces redundant transmission and saves energy on the network. However, inefficient and poor selection of cluster heads results in faster depletion of energy in the network. Therefore, it is highly important to design adaptive and intelligent cluster head selection protocols that consider residual energy, transmission cost, network density, and distance from the base station³. A Wireless Sensor Network is comprised of low-power sensor nodes that collaborate to sense physical environments and transmit the sensed data to a sink node for processing. This research proposes an enhanced version of the DEEC protocol⁴ using K-Nearest Neighbours and K-Means machine learning algorithms to dynamically select the best cluster head, resulting in DEEC-KNN and DEEC-KM protocols. Simulation results show substantial improvements in terms of energy, stability, and data delivery, showing performance improvements up to 110% compared to the conventional DEEC protocol. The novel machine learning-based clustering protocol is found to perform exceptionally in WSNs, especially in large networks, demonstrating the potential of machine learning in developing energy-efficient sensor networks for the future.

The conventional hierarchical routing protocols, such as Distributed Energy-Efficient Clustering (DEEC), Low Energy Adaptive Clustering Hierarchy (LEACH), Stable Election Protocol (SEP), and Threshold Sensitive Energy Efficient sensor Network (TEEN), have been extensively used to achieve energy efficiency through clustering and aggregation. However, challenges like random selection of CH, non-uniform distribution of CH, and high overhead of re-clustering are still major drawbacks in this approach⁵.

For addressing these problems, various bio-inspired optimization and intelligent decision-making techniques have been proposed for routing optimization in WSNs in recent years. Among these techniques, metaheuristics such as Ant Colony Optimisation (ACO), Particle Swarm Optimisation (PSO), Firefly Algorithm (FA), and Genetic Algorithms (GA) have been used for optimizing the CH selection process. The convergence rate of these techniques is often affected by the problems of slow convergence rate and premature optimization.

Stimulated by these issues, this research presents a novel concept of a Reinforcement Learning-enabled Multipath Hybrid Whale Optimisation Algorithm and Grey Wolf Optimisation Algorithm Framework (RL-MHWO–GWO) to solve the mentioned issues in an energy-efficient way while maintaining balanced node loading and utilizing the dynamic sleep scheduling and node activation using a deep reinforcement learning agent⁶. In order to achieve the most efficient CH selection, this proposed framework integrates the global search ability of the Whale Optimisation Algorithm with the local search ability of the Grey Wolf Optimiser Algorithm. In addition, this proposed framework adopts the multipath routing protocol to develop robust, energy-efficient, and stable paths among CH and BS nodes in the WSN environment⁷. By utilizing the flexibility and hybrid optimization potential of the proposed framework based on the RL and hybrid optimization approach, this proposed framework is more efficient compared to other conventional protocols in establishing scalable, reliable, and energy-efficient communication in the WSN environment.

Novelty

This research novelty is given as follows:

• A novel RL-MHWO–GWO method is suggested for energy-efficient routing in WSNs by combining Deep Reinforcement Learning (DRL), Hybrid Whale–Grey Wolf Optimization (WOA–GWO), and a Multipath routing mechanism.

• The DRL agent is used in the process of dynamic sleep scheduling, which intelligently detects the nodes based on real-time parameters in the network, such as residual energy, node centrality, and distance to the cluster head. It adapts well, saves energy in large quantities, and eliminates redundant messages while utilizing load balancing in the network.

• The research suggested the Hybrid WOA–GWO method for CH selection. Whale Optimisation Algorithm has excellent exploration properties, which enable the optimization technique to avoid local optima and premature convergence. Grey Wolf Optimiser has excellent exploitation properties, which enable the optimization technique to converge quickly to efficient solutions. Thus, the Hybrid method ensures optimal and efficient selection of CH based on residual energy, communication cost, and node density.

• A multipath routing approach develops multiple stable, energy-efficient, and fault-tolerant communication paths between CHs and the base station. In the routing phase, the hybrid approach of the WOA and GWO algorithms is used to assess the candidate paths based on hop count, quality, and energy to avoid congestion, minimize delays in transmitting information, and enhance the performance of WSN communication.

Contribution

The main contributions of this explore are emphasized as follows:

• (i)Energy-aware sleep scheduling using Deep Reinforcement Learning (DRL): In this study, a scheduling method based on DRL is proposed, in which node activities are dynamically managed based on residual energy (RE), node centrality (NC), and distance to the cluster head (DCH). The DRL agent learns a scheduling strategy based on dynamic network conditions, effectively saving energy, balancing network load, and prolonging network lifetime.
• (ii)Cluster Head (CH) selection using Hybrid WOA–GWO: In the proposed RL-MHWO–GWO model, CHs are chosen using a hybridized algorithm that combines Whale Optimisation Algorithm (WOA) and Grey Wolf Optimizer (GWO). WOA improves the global search exploration process, while GWO is responsible for the rapid local exploitation. The hybridization approach adopted in this framework ensures optimal, balanced, and energy-efficient CH selection.
• (iii)Multipath routing with energy-optimized path discovery: A multipath routing scheme based on WOA–GWO is introduced into the system to provide stable, reliable, and energy-efficient paths from CHs to the BS. The paths are assessed based on hop count, residual energy, and link quality. The multipath routing scheme improves routing reliability, reduces congestion and delay, and minimizes total energy consumption during transmission.
• (iv)Performance validation: The proposed RL-MHWO–GWO framework can provide a packet delivery ratio (PDR) of 97.8%, an aggregate throughput of 52 kbps, and an energy consumption of 320 J, while ensuring balanced residual energy in the nodes. The above results validate improvements in energy efficiency, routing, and communication stability.

The remaining sections of this research work are organized as follows: Related works section presents related work on energy-efficient routing in WSNs. Proposed framework section describes the system model and problem formulation. Experiment section explains the proposed RL-MHWO–GWO approach in detail, which includes clustering, CH selection, and routing schemes. Conclusion and future scope section analyses the performance of the proposed approach using simulation results. Section  6 concludes this research work with important findings.

Related works

This section highlights existing work on energy-efficient routing in WSNs and outlines their limitations and research gaps.

Shukla et al.⁸ suggested an EEFTMR scheme for WSNs. The scheme employs an elite-group-based routing strategy for sensor node routing and fault-tolerant CH selection. Fault-tolerant CH tracks the performance of the cluster head and temporarily buffers data until transmission is successful, thereby enhancing fault tolerance. Faulty sensor nodes are identified using hypothesis-based testing at the cluster head node. Additionally, a multipath routing strategy employing a composite performance metric comprising residual energy, hop count, network resilience, and data speed was implemented to ensure fault-tolerant data transmission. Simulation results demonstrated significant improvements in network delay (73.7%), energy efficiency (9.6%), network longevity (9%), and error-free data throughput (61.3%) compared to prior elite-group-based routing methods. Kumar et al.⁹ emphasized power conservation in WSNs using an OEERP. In OEERP, specific sensor nodes—referred to as residual nodes—are bypassed during cluster construction and forward partial information to the next-hop or base station, resulting in wasted power and a shorter network lifetime. To address this, the authors presented an I-OEERP that employs assistant cluster nodes to prevent residual node formation and thus improve network longevity. The protocol integrates CSA and GSA: CSA regulates cluster formation to limit residual nodes, while GSA optimizes routing paths for energy efficiency.

Kumar et al.¹⁰ presented a WAOA for the design of energy-efficient routing in WSNs. WOA is used to identify the optimal cluster heads within a defined search space, whereas ACO is used to determine the shortest paths from source cluster sensors to the cluster head. The issues of cluster head selection and route discovery optimization are formulated as linear programs. Nakas et al.¹¹ presented each class includes a complete description and comparison of protocols based on performance measures, which tells us where they are strong and where they fall short. The research ends with a results discussion and discusses open research issues in energy-efficient WSN routing.

An enhanced Double Deep Q-Learning (DDQN)¹² that is linked to conventional methods, results achieve up to a 35% drop in end-to-end lag, a 28% rise in energy efficiency, and a 20% drop in the number of missed deadlines. Similarly, the HGPSO¹³ can get around these problems by combining the good things. GA’s strong global exploration skills, which keep things interesting and stop hasty convergence, are combined with PSO’s fast local exploration and convergence skills. These results show that HGPSO offers a robust theoretical framework and can be used in large-scale IoT deployments in the real world. The research by¹⁴ suggests a flexible approach that effectively uses available resources. The protocol uses five custom methods to process data, find neighbours, and determine the best approach. Many types of apps can use the protocol, including those that stream large amounts of data in real time. A custom-built sensor node made by our study team uses the protocol. It is decided on a system that lets the monitor devices communicate with each other over their own networks or the Internet, depending on what is needed to prevent resources from running out.

Senthil et al.¹⁵ proposed an HBWCO algorithm to enhance energy-efficient data transmission in wireless sensor networks. The process begins with sensor node initialization and field dimension setup, and proceeds to K-means clustering of nodes. Node energy levels are predicted using a DQN algorithm, and CHs are selected by picking nodes with higher energy levels. The HBWCO scheme achieves 0.948 reliability and 3496 throughput, validating its effectiveness in improving routing reliability and energy savings. Saleem and Saleem¹⁶ proposed EMCL for wireless multimedia sensor networks (WMSNs) to mitigate the limited energy resource issue. Simulation results show that EEMCL enhances energy dissipation, network lifetime, and stability compared with the SEP, SEP-E, and SEPFL protocols. Interestingly, the node death in the last round occurs at round 5833 in EEMCL, surpassing 4027, 2828, and 2325 rounds for SEPFL, SEP-E, and SEP, respectively.

The Wireless Body Sensor Network (WBSN)¹⁷ is a key component for sending critical patient data to remote medical facilities. This study presents a new Q-learning-based clustering routing system, QL-CLUSTER. The Mountain Gazelle Optimiser (MGO)¹⁸ can settle too quickly, is computationally intensive, and can’t be used to solve problems with more dimensions. This review discusses its strengths and weaknesses and shows how it’s becoming increasingly useful across a variety of settings. The research will examine 89 studies published on MGO. These studies can be grouped into four categories: modified, variants, hybrid, and improved. These groups account for 37, 33, and 27% of all studies, respectively. According to research, problems exist¹⁹, such as the high computational demands of real-time apps and the difficulties in handling large amounts of data. To make effective use of resources in Internet of Things systems²⁰, which make use of a wide variety of devices with limited capacity. To counter this, an expanded meta-heuristic approach is recommended as a solution given the issue’s inherent complexity. The RG-ACA algorithm achieved the highest possible ranking across all categories compared to other relevant research currently underway. It achieved 94%, 96 Kbps, and 91% in these three metrics, respectively. Another machine learning-based routing scheme proposed recently referred to as EMO-PEGASIS²¹. This scheme improves the traditional PEGASIS system by adding a two-phase learning process.

Cat Swarm Optimisation (SCSO)²² is a nature-inspired method, and this work provides a comprehensive overview and analysis of the technique. The SCSO algorithm is designed to effectively focus on investigating solution regions inspired by sand cat hearing and on identifying solutions that are both most suited to their hunting behaviour and highly effective. The suggested approach was used in this research to solve 39 functions and transdisciplinary issues. Compared with other chaotic-based metaheuristics and the best-developed SCSO variant, it produced results of 76.3%. The method²³, referred to as Ex-ChOA, considers hybrid models to update search agents’ positions²⁴. It achieved 76.3% better results than the best-developed SCSO version and other chaotic-based metaheuristics tested. Colour satellite images contain a lot of data that let you examine land use, plant growth, and other surface features in greater depth²⁵.

Several studies have investigated hybrid WOA–GWO algorithms for clustering and routing optimization. However, most of these approaches apply static hybrid strategies or focus on single-objective formulations without dynamic topology adaptation. Although EMO-PEGASIS is highly effective at optimizing energy-delay trade-offs through supervised and unsupervised learning, it differs fundamentally from the proposed RL-MHWO–GWO framework. EMO-PEGASIS is based on pre-defined learning phases and chain routing, which does not involve continuous online adaptation or multipath routing. By contrast, the proposed framework uses deep reinforcement learning (DQN) to enable dynamic sleep scheduling based on real-time network feedback, enabling nodes to self-adapt their activity patterns. The above-mentioned variations and distinctions clearly demonstrate that the proposed RL-MHWO–GWO technique is more adaptable and comprehensive for energy-efficient routing in WSNs.

Limitations and research gaps

Several recent studies have integrated DQN-based learning with hybrid metaheuristic clustering strategies. However, most approaches employ reinforcement learning for energy prediction or parameter tuning, while the metaheuristic optimizer operates independently. The interaction between learning and optimization is typically sequential rather than structurally coupled. In the existing routing protocols used in current WSN, the unachieved parameters are energy efficiency, long lifespan, and data transmission reliability. In the existing routing protocols, clustering and heuristic-based optimization techniques are used, which ultimately lead to inefficient selection of cluster heads, unbalanced exhaustion of node energies, and node failures. Additionally, less adaptive and intelligent wake-up scheduling schemes cause wastage of node energies through wakefulness of non-essential nodes, which ultimately affects the performance and lifespan of the network. Besides, dynamic path evaluation is not used in existing routing protocols, which ultimately affects the reliability and fault tolerance of data transmission.

These vacuums are filled by the proposed mechanism, which employs deep reinforcement learning for adaptive sleep scheduling, a hybrid Whale Optimisation Algorithm and Grey Wolf Optimiser algorithm for efficient selection of CHs, and multipath routing. The problem of energy wastage due to vacillation is addressed by using DRL-based scheduling, which is able to learn and adaptively eliminate unnecessary energy consumption.

Proposed framework

The WSN network²⁶ and energy model²⁷ are defined, which demonstrate the routing method among the CHs, SNs, and BS. The proposed framework comprises a DRL agent for adaptive sleep scheduling, a hybrid GWO for enhanced cluster head selection, and multipath routing to form stable links. The hybrid optimization approach strikes a balance between exploration and exploitation to deliver energy-efficient, well-balanced CH selection²⁸. Finally, the multipath routing layer²⁹ determines several reliable, energy-efficient paths from CHs to the BS using dynamic hop counts, residual energy, and link quality. Figure 1 depicts the system model, showing the interactions among SNs, CHs, and the BS in the proposed routing scheme.

Fig. 1.

System model.

WSN’s network design

A WSN is set up consisting of a homogeneous quantity of SNs and a BS, in which the SNs are placed casually and consistently in the specified network area. Any two nodes communicate through single-hop or multi-hop transmission. The SNs are classified as source nodes and normal nodes, each with a unique network ID and a specific location in the network. The base station is placed at a strategic location in the centre of the network area.

All sensor nodes have the same storage, energy, and processing capacities. Each SN can sense its location in the network, while the base station has greater processing capability and is not energy-limited. SNs know their remaining energy and calculate the distance to nearby nodes. SNs collect data periodically and transmit it to their respective CHs, where it is aggregated. Power levels for transmission are dynamically adjusted based on the distance to the receiving node, promoting symmetric announcement links and effective energy use. SNs also continuously sense their surroundings at the maximum sensing rate, and the base station is always within communication range of all SNs in the network.

Energy model

In a WSN, every SN spends energy on transmitting/receiving packets, aggregation, and control (Eqs. 1 & 2). The radio model used in this work is

1

2

Where k is bits per packet, d is transmission distance, is electronics energy, is the threshold distance, and are the amplifier coefficients for free-space and multipath models.

Problem formulation

The primary goal of this investigate is to optimize network lifetime and transmission reliability. Typically, SNs in WSNs operate on batteries with limited capacity, often in isolated or inaccessible locations, making it even more critical to develop methods that minimize unnecessary energy consumption without compromising data transmission effectiveness—the analytical derivation of the objective function, Eq. (3).

3

Reduce overall energy consumption (EC):

Minimize total energy used in sensing/processing and in intra-/inter-cluster communication over all rounds (Eq. 4):

4

Where N is number of node, R is number of communication rounds, is the energy spent by node i.

Enhance network lifetime (NL):

Let, , maximizing NL guarantees sustained network operation and prolongs the network’s functional period. To ensure reliable data transmission, a high packet delivery ratio (PDR) is maintained by establishing trustworthy, energy-efficient routing paths from SNs to the BS. The PDR is computed as:

5

Where, in Eq. (5) denotes the number of data packets from node i that are successfully received at the BS in round t, and is the number of data packets generated/sent by node i in round t. A larger PDR indicates greater communication reliability and robustness.

The energy, lifetime, and reliability are combined as a single scalar minimization objective function using normalized terms, so the goals are commensurate:

6

Where, in Eq. (6) refers to the normalized energy consumption, and is the normalized network lifetime. The suitable references, denote the initial total network energy or the baseline observed value, and denoted the baseline lifetime. The weight factors assigned to EC, NL, and PDR are , , and , respectively. The maximization form is derived from Eq. (6):

7

The minimization was retained using Eqs. (6) and (7) for consistency with later optimization steps.

RL-MHWO–GWO framework

Figure 2 shows the entire working process of the proposed RL-MHWO–GWO framework. The methodology starts with node initialization, in which sensor nodes are randomly deployed within the defined monitoring area. Once the nodes have been deployed, the neighbor discovery process uses periodic HELLO packets to discover the neighborhood topology. After this, the DRL-based scheduler adjusts the duty cycles of the sensor nodes. Next, the hybrid WOA–GWO algorithm selects the cluster heads to achieve energy-efficient, evenly distributed clustering. This process finds the best energy-efficient paths from the CHs to the BS using relay CHs, based on hop count, signal strength, and residual energy. Data is then sent via the best routes to the BS, ensuring secure, guaranteed transmission throughout the network.

Fig. 2.

Overall working process of the proposed RL-MHWO–GWO.

Node initialization

The N sensor nodes are deployed uniformly at random in a 2-D square sensing field . The position of the node is represented using Eq. (8):

8

Each node is initialized with an initial energy , sensing range , and communication range . The Euclidean distance between any two nodes i,j is calculated using Eq. (9):

9

Connectivity is ensured by verifying such that for all i; otherwise, the deployment is regenerated (see Eq. (10)). Each node knows its residual energy and can estimate link qualities to neighbors. The BS is placed either at the field center or at a designated coordinate .

10

Neighbor node discovery

Every node periodically broadcasts short HELLO beacons containing . Upon reception, nodes update a neighbor table with (ID,distance,residualenergy,linkquality). These tables support subsequent CH selection and route construction.

Sleep scheduling using deep reinforcement learning (DRL)

The proposed sleep scheduling mechanism is implemented using a Deep Q-Network (DQN) to approximate the optimal action–value function Q(s,a). Sleep scheduling is modeled as a markov decision process In the proposed DRL framework for sleep scheduling, the environment progresses in discrete communication rounds. A round of the environment represents one complete routing round, comprising the observation of the state by all sensor nodes, making sleep/active decisions, sensing and transmitting data by active nodes, and intra- and inter-cluster communication, culminating in reward calculation. State transitions occur at the end of each round based on changes in residual energies, link qualities, and queue states.

The round-by-round formulation enables coordinated decision-making and aligns with the clustering and multipath routing phases of the RL-MHWO–GWO framework. State for node i at time, the DQN consists of an input layer corresponding to the state vector. The DRL component follows a centralized training–distributed execution architecture in which the DQN model is maintained and trained at the base station. Sensor nodes only transmit lightweight state summaries once per communication round and do not perform neural network inference locally. This design ensures that computational and memory requirements remain within the capabilities of resource-constrained sensor nodes. The state representation in Eq. (11) is designed to capture both node-level and network-level dynamics affecting energy consumption and communication efficiency. The state vector for node at time is given by:

11

where each component has a distinct functional role. The residual energy reflects the node’s remaining lifetime and is critical for energy-aware decision making. The node degree represents local connectivity and helps preserve coverage. The distance to the cluster head d(i,CH) captures communication cost, while the link quality reflects channel reliability. Finally, the queue length models traffic load and congestion. This combination ensures that the DRL agent observes energy, topology, communication, and traffic conditions simultaneously, enabling informed scheduling decisions.

The reward function is constructed as a weighted combination of multiple performance objectives:

12

Where, is energy saved by transitioning to sleep mode, encouraging energy efficiency, is coverage contribution, ensuring that sufficient nodes remain active, is delay or queue penalty, discouraging excessive buffering and latency, and Δ PDR is the improvement in packet delivery ratio, promoting reliable communication; The weighting factors are normalized such that , allowing flexible prioritization between energy conservation and communication performance.

This reward design ensures that the DRL agent does not optimize a single metric in isolation but instead learns a balanced scheduling policy that jointly minimizes energy consumption while maintaining coverage, reducing delay, and improving reliability. This structured state and reward design enables stable and interpretable policy learning, directly aligning the DRL objective with system-level performance goals in WSNs.

We employ a DQN agent with a target network . The temporal-difference target is computed using Eq. (13).

13

Formally, the DRL time index t denotes the routing round index, and the transition is evaluated once per round. A centralized training and distributed execution paradigm is implemented, where knowledge tuples together from sensor nodes is combined at the base station for training. The DQN is trained using experience replay with a buffer and mini-batch gradient descent to minimize the temporal-difference loss defined in Eq. (13). A separate target network is employed and periodically updated to stabilize learning and prevent Q-value overestimation. During performance, each node follows an -greedy policy to balance exploration and exploitation. Once qualified, the learned policy allows sensor nodes to independently agree on their sleep or active states based on real-time network conditions, thereby reducing redundant node activity, conserving energy, and preserving adequate sensing coverage.

Deep Q-network (DQN) architecture

Figure 3 shows the architecture of the proposed Deep Q-Network (DQN) employed for adaptive sleep scheduling is implemented as a lightweight fully connected neural network to accommodate the resource constraints of WSNs.

Fig. 3.

The proposed DQN architecture for adaptive sleep scheduling.

The network architecture involves of an input layer with 5 neurons representing the state vector, Eq. (14). Hidden Layer 1 has 64 neurons with ReLU activation, Hidden Layer 2 has 32 neurons with ReLU activation, and the output layer has 2 neurons corresponding to the Q-values of the actions {ACTIVE, SLEEP}

14

Where, is residual energy, is neighbor degree, d(i,CH) is distance to the cluster head, is link quality, and is the queue length.

The network parameters are optimized using the Adam optimizer with a learning rate of . A discount factor of γ=0.95 is used to balance immediate and long-term rewards. Training is stabilized using an experience replay buffer of size 10,000, and a target network updated every 500 steps. Action selection follows an ε-greedy strategy, with ε linearly decayed from 1.0 to 0.05 during training.

The vital configuration parameters of the DQN employed for adaptive sleep scheduling are summarised in Table 1, as conventionally implemented in DRL solutions for energy-efficient wireless sensor networks. The DRL outputs a binary duty set (active nodes) each round. This DRL block replaces the fuzzy system in the baseline, providing adaptive, learned duty-cycling under dynamic traffic and topology.

Table 1.

DRL Configuration Parameters.

Parameter	Value
RL Model	Deep Q-Network (DQN)
State size	5
Action space	2 (Sleep, Active)
Hidden layers	2 (Fully connected)
Activation	ReLU
Discount factor (γ)	0.95
Replay buffer Size	10,000
Replay batch size	64
epsilon schedule (Action selection)	ε-greedy exploration strategy
ε linear decay	1.0 to 0.05
Target update interval	500 steps
Training mode	Centralized
Execution	Distributed

The proposed DRL framework adopts a centralized training and distributed execution strategy. At the end of each communication round, every active sensor node transmits a compact state-action summary to the BS. This summary includes the node’s residual energy , neighbor degree , average link quality , distance to the cluster head d(i,CH), queue length , selected action , and the immediate reward . Using these reports, the BS constructs experience tuples for all participating nodes and aggregates them into a replay buffer. The centralized learner therefore observes a batch of node-level transitions per round, representing the global network dynamics in a compact and energy-efficient manner. Model updates are performed at the BS, and the updated DQN parameters are periodically broadcast to sensor nodes for distributed execution in subsequent rounds. To limit communication overhead, only active nodes report summaries once per round, and raw packet-level data is never transmitted to the BS for learning purposes.

Cluster-Head (CH) selection using hybrid WOA–GWO

In the proposed Hybrid WOA–GWO algorithm, the number of cluster heads (CHs) selected in each round is regulated using a target CH ratio . Given N deployed sensor nodes, the expected number of CHs is , with admissible bounds .

Whale optimisation algorithm (WOA)

The WOA mimics humpback whales’ encircling and spiral attacks. coefficient vectors, encircling the best, and spiral updating are computed using Eqs. (15–17).

Coefficient vectors, Eq. (15):

15

Encircling the best , Eq. (16):

16

Spiral updating, Eq. (17):

17

Grey wolf optimizer (GWO)

During optimization, candidate solutions, encoded as binary vectors, are evaluated against this constraint. Solutions that violate the CH cardinality bounds incur a penalty term in the fitness function. Additionally, a repair mechanism randomly deactivates or activates nodes with the lowest or highest fitness contribution to restore feasibility. This approach enforces stable cluster formation while preserving the adaptive behaviour of the hybrid optimizer. With wolves guiding the search, Eq. (18):

18

In all experiments, is set to 0.05, resulting in approximately of nodes acting as CHs per ro, consistent with standard clustering-based WSN protocols.

Algorithm 1.

Hybrid WOA–GWO for CH selection

Hybridization and fitness

We encode a solution as a binary vector . A sigmoid binarizer maintains feasibility. Fitness (to minimize) jointly optimizes energy, balance, and coverage, Eq. (19):

19

With weights , normalized energies , link quality LQ, and cluster load . Early iterations use WOA for global exploration; later iterations switch to GWO, Eq. (19) for fast convergence. A mixing probability decreases linearly, Eq. (20):

20

Termination, stop when or .

Hybridization and fitness evaluation

The fitness function used in the hybrid WOA–GWO alg function used in the hybrid WOA–GWO algorithm evaluates each candidate CH set by jointly considering energy efficiency, load balance, and communication cost, while ensuring feasibility. For a binary CH selection vector, Eq. (21).

21

The fitness to be minimized is defined as, Eq. (22):

22

The CH energy cost term, Eq. (23), penalizes the selection of low-energy nodes as cluster heads, Eq. (24), is the normalized residual energy of node i.

23

24

The load balance term, Eq. (25), measures the variance in cluster sizes, Eq. (26).

25

26

is the number of member nodes assigned to CHc. Lower variance implies better load balancing.

27

Equation (27) represents the cumulative cost for all non-CH sensor nodes j to associate with their most suitable CH, based on distance d(i,j) and link quality LQ(i,j). Feasibility penalty term, Eq. (28), ensures that at least one CH is selected by assigning a large penalty if no CH exists.

28

Cluster membership assignment during fitness evaluation for a given candidate CH set is expressed using Eq. (29), the cluster membership is determined inside the fitness evaluation using Eq. (30):

29

Each non- CH node j is assigned to the that minimizes the attachment cost, Eq. (29).

30

Thus, cluster formation is implicitly embedded within the fitness evaluation, and no separate clustering step is required during optimization.

Cluster formation

Non-CH nodes listen to CH-ADV beacons and join the nearest / best-energy CH by sending JOIN-REQ. The attached score for node j to CHi is, Eq. (31).

31

Nodes select the CH that maximizes S. CHs aggregate member data and schedule TDMA.

For relay CH selection (tiered inter-cluster), a rank is computed using Eq. (32):

32

Where is the number of active members.

Multipath routing mechanism for energy-efficient data transmission

Path Disjointness Constraint

In the proposed multipath routing mechanism, a partially node-disjoint constraint is employed. Multiple paths originating from a source cluster head and terminating at the base station are permitted to share the source CH and the BS, while intermediate relay CH s are encouraged to be disjoint across different paths. Let denote the selected set of routing paths. For any two paths and , partial disjointness is enforced by minimizing the intersection of their intermediate node sets, Eq. (33):

33

To promote this behavior during optimization, a node-overlap penalty is added to the path fitness function.

Candidate path encoding

In the proposed WOA–GWO guided multipath routing scheme, a candidate path is encoded as an ordered list of cluster-head identifiers that defines a feasible multi-hop route from a source CH to the BS, Eq. (34):

34

Where is the source is the last relay CH, and each consecutive pair satisfies the communication range and link-quality constraints of the CH-level graph. For optimization, each candidate solution maintained by the proposed WOA–GWO algorithm is initially represented as a continuous priority vector (Eq. 35).

35

Where is the number of CHs in the network. A priority-based decoding procedure converts this vector into a discrete routing path by iteratively selecting, at each hop, the neighboring CH with the lowest priority-weighted cost that has not yet been visited. This decoding ensures loop-free and connected paths. Given the active CH graph , we evaluate candidate paths p from CH to BS using a WOA–GWO fitness, Eq. (36):

36

Where bars denote normalized path metrics (hop count, end-to-end delay, inverse link quality, inverse residual energy). The hybrid optimizer searches K disjoint/partially disjoint paths minimizing . The hop-wise selection and the best path cost over all hops are computated using Eqs. (37) and (38).

37

38

Loop-freeness enforcement and repair mechanism

To ensure valid routing paths, the proposed WOA–GWO-guided multipath routing explicitly enforces loop-free path construction. During candidate path decoding, a visited set is maintained for each path. Once a CH is added to the path, it is inserted into and excluded from future hop selection. At each hop h, the next relay CH is selected only from the feasible neighbor set, Eq. (39):

39

Where denotes the current CH. This constraint prevents revisiting previously traversed nodes and ensures loop-free progression.

If or a loop is detected, a repair procedure is activated:

1. The looping segment of the path is truncated at the last valid node.

2. The algorithm reconnects the path by selecting the nearest feasible unvisited CH with sufficient residual energy.

3. If no such CH exists, the path is directly terminated at the base station.

This repair process guarantees that all candidate paths evaluated by the optimizer are connected, loop-free, and feasible, without discarding potentially good partial solutions.

Link quality computation

In the proposed framework, link curality between two communicating nodes i and j is quantified using an Expected Transmission metric, which reflects the reliability of the wireless link. The link quality LQ(i,j) is defined as, Eq. (40):

40

where and denote the forward and reverse packet delivery ratios, respectively. A higher LQ(i,j) value indicates a more reliable link, and this metric is incorporated into both cluster membership assignment and multipath routing fitness evaluation.

End-to-end delay computation

The end-to-end delay of a routing path p is calculated as the sum of hop-wise delays along the path, Eq. (41):

41

Where is the transmission delay, is the propagation delay with distance and signal propagation speed c, is the queuing delay determined by buffer occupancy, and is the node processing delay.

Convergence and stability analysis

The exploration coefficient in WOA decreases linearly, ensuring diminishing exploration, as shown in Eqs. (42) and (43).

42

Bounded feasible space:

43

Under diminishing exploration and a bounded discrete search space, the hybrid update exhibits probabilistic convergence toward a local optimum.

Algorithm 2.

WOA–GWO-Guided Multipath Routing

Structural interaction between DRL and hybrid optimization

The proposed RL-MHWO–GWO framework does not merely combine DRL and metaheuristic optimization sequentially. Instead, it establishes a hierarchical and dynamically coupled interaction between learning-driven scheduling and hybrid search.

DRL reduces the candidate node set

At each communication round , the DRL agent determines the active node set, Eqs. (44–46).

44

Nodes not selected by the DRL policy are placed in sleep mode:

45

This directly constrains CH selection because only active nodes are eligible candidates:

46

Thus, the DRL policy performs search-space pruning before optimization, reducing computational complexity and preventing low-energy nodes from being selected.

Hybrid optimizer operates on DRL-filtered topology

The WOA–GWO hybrid algorithm performs CH selection over the reduced feasible domain, as defined in Eqs. (47 and 48).

47

Hence, the optimizer does not operate on the full static topology but on a time-varying DRL-filtered network graph:

48

This ensures that CH selection reflects real-time energy and topology conditions rather than fixed deployment assumptions.

Reward and fitness are indirectly coupled

The DRL reward function incorporates residual energy, load variance, and delivery reliability, as defined in Eqs. (49 and 50).

49

Meanwhile, the hybrid optimizer fitness function includes:

50

Since both formulations depend on residual energy distribution and load balancing, the DRL policy gradually reshapes the fitness landscape encountered by the optimizer.

Search space dynamically changes per round

Because DRL decisions are updated each round, the feasible solution space evolves as defined in Eqs. (51 and 52).

51

This means the hybrid optimizer solves a time-varying constrained optimization problem, unlike conventional static WOA–GWO frameworks. The search dimension effectively becomes:

52

This dynamic reduction prevents premature convergence and improves stability under energy depletion.

Phase-adaptive WOA–GWO scheduling

Unlike static hybrids, the proposed process transitions between exploration and exploitation stages based on iteration index, as defined in Eq. (53).

53

Experiment

This section reports the setup, parameters, metrics, and analyses that demonstrate the effectiveness of RL-MHWO–GWO for energy-aware routing.

Experimental setup

All simulations were implemented in MATLAB R2023a on an Intel Core i7-10700 (3.8 GHz) with 16 GB DDR4 RAM, running Windows 10 Pro 64-bit. DRL training uses mini-batch replay (batch size = 64) and updates the target network every 500 steps. The simulation configuration of the proposed framework is given in Table 2. To ensure comprehensive evaluation, the proposed RL-MHWO–GWO framework is tested under multiple network configurations. The number of sensor nodes is varied as N = {100, 150, 200, 250}, and nodes were randomly deployed in a 500 m × 500 m monitoring area. Each node was initialized with 2 J of energy. The simulation ran for up to 10,000 communication rounds, or until the last node depleted its energy. Network lifetime was evaluated in terms of first node death (FND) and last node death (LND). All compared protocols were executed under identical environmental and network parameters to ensure fair benchmarking. Each active node transmits a 12–16 byte state summary (node ID, residual energy, neighbor degree, and link-quality indicator) to the base station once per round, resulting in approximately 3–4 kB control overhead per round for a 250-node network. The radio model parameters are set as nJ/bit, pJ/bit/m² fs.

Table 2.

Simulation configuration.

Parameter	Value
Number of nodes	250
Deployment area	500 m × 500 m
Initial energy	2 J per node
Packet size	512 bytes (better than 1024)
Data rate	250 kbps (IEEE 802.15.4 standard)
Data traffic	Constant Bit Rate (CBR)
	50nJ/bit
MAC layer	TDMA-based clustered
Traffic model	Periodic sensing (1 packet/round)
Communication Rounds	10,000

The optimized parameter values used in the RL-MHWO–GWO framework are: weights α₁ = 0.40, α₂ - α₄ = 0.20, Population size = 40, Max iterations () = 80, Routing weight - energy () = 0.40, Routing weight - Hop count () = 0.30, and Routing weight - link quality () = 0.30.

Evaluation metrics

The RL-MHWO–GWO model is validated using various performance metrics to ensure its effectiveness. These encompass energy consumption, PDR, throughput, end-to-end delay, path loss, network lifetime, number of dead nodes, number of alive nodes, residual energy, scalability, and LBR. These metrics’ definitions with formulas are given as follows:

Energy consumption (EC)

In the network, a node’s energy level or energy required for the process of transmission is evaluated by this metric. Especially, evaluate the energy consumption between the node’s initial energy and the final energy following transmission is calculated using Eq. (54).

54

PDR

The success rate of data packets delivered from the source to the destination is evaluated by PDR using Eq. (55).

55

From the aforementioned Eq. (55), signifies total packets received, and represents total packets sent.

Throughput

The aggregate data rate successfully received at the base station over all multipath routes, expressed in kbps, is computed using Eq. (56).

56

End-to-end delay

End-to-end delay represents the average multi-hop transmission delay, including forwarding, queuing, and processing time across intermediate nodes, calculated using Eq. (57).

57

In the above Eq. (57), signifies the packet arrival time, represents the packet sent time, and denotes the total packets received.

Path loss

Evaluate the path loss at the time of transmission by using Eq. (58). It is the difference between the sending end’s transmitting power and the power at the receiving end when receiving the signal .

58

Network lifetime

The time period the network functions effectively before a significant node failure is measured by this metric using Eq. (59). Number of communication rounds until FND or LND.

59

From the above-mentioned Eq. (59), and N represent the residual energy of node i at round t and the total number of nodes, respectively.

Number of dead nodes

The number of nodes that have fully depleted their energy and are not working is known as the number of dead nodes (see Eq. (60)). A minimal dead node indicates stable energy consumption across the network.

60

Number of alive nodes

The number of nodes that remain with residual energy and can carry out sensing or communication is called alive nodes; an elevated count at any given time indicates superior load balancing and energy-efficient routing. The mathematical formula for calculating the number of alive nodes is given in Eq. (61)

61

Residual energy

The variance between initial energy and consumed energy is called each node’s residual energy. It is given by Eq. (62)

62

Scalability

It shows the capability of the protocol to maintain performance by maximizing network size, including the number of nodes or area.

Load balancing ratio

In the course of transmission and routing processes in WSNs, this metric is employed to assess how uniformly energy is allocated between total SNs. A high value of LBR indicates stabilized resource usage and optimal energy fairness, which is very important for improving network lifetime and stability. The formula for LBR is as follows, Eq. (63)

63

From the aforementioned Eq. (63), the total active node’s average energy consumption is represented as , and the standard deviation related to energy consumption over nodes is denoted as .

Performance analysis

The proposed RL-MHWO–GWO framework achieves a total energy consumption of 320 J, a packet delivery ratio of 97.8%, and an aggregate throughput of 72 kbps measured at the base station, as shown in Table 3. The average multi-hop end-to-end delay is reduced to 38 ms, with minimized jitter under stable traffic conditions. The network lifetime extends up to 9,500 communication rounds (LND), while maintaining balanced energy utilization across nodes, with approximately 222 alive nodes and 28 dead nodes at the observed stability point. The average residual energy per node remains around 0.72 J, demonstrating effective load distribution. Furthermore, the model exhibits strong scalability (≈ 95.4%) and achieves a high load balancing ratio (96.8%). Overall, the proposed framework significantly enhances reliability, energy efficiency, and routing stability under realistic WSN operating conditions, making it well-suited for practical multi-hop sensor network deployments.

Table 3.

Performance analysis results of the proposed model.

Performance metrics	Values
Energy consumption	320 J
PDR	97.8%
Aggregate Throughput (kbps)	72 kbps
Multi-hop Delay (ms)	38 ms
Jitter	0.15 ms
Network Lifetime (LND)	9500 rounds
Number of dead nodes	28
Number of alive nodes	222
Average Residual Energy	0.72 J
Scalability	95.4%
LBR	96.8%

The throughput values correspond to aggregate network throughput at the base station under parallel multipath delivery. All delay measurements reflect realistic multi-hop transmission under clustered routing assumptions. The robustness of the proposed RL-MHWO–GWO framework is evaluated under varying network conditions. Node density: N = 100 to 250 nodes, Traffic load: packet generation rate varied from 0.5 to 2 packets per node per round, Initial energy levels: varied between 1 and 3 J. The results show that the proposed framework maintains stable performance, with PDR remaining above 97.8% and energy consumption increasing linearly with node density. Additionally, the LBR remains above 95% across all scenarios, demonstrating uniform energy distribution and adaptability to dynamic WSN environments.

Convergence behaviour analysis

The convergence behaviour of the proposed framework is influenced by both the hybrid metaheuristic optimization and the reinforcement learning components.

For cluster head selection and multipath routing, the Hybrid WOA–GWO algorithm exhibits fast and stable convergence due to the complementary strengths of its constituent optimizers. In the initial stages, the WOA performs well in the exploration of the solution space, preventing the search process from converging prematurely. As the iterations advance, the GWO takes over the search process, improving the local search and speeding up the convergence towards the near-optimal solution. This causes a smooth decrease in the fitness function value with the advancement of iterations and converges after a few iterations.

For the DRL-based sleep scheduling, the convergence is obtained through the progressive stabilization of the Q-values learned by the DQN agent. During the early stages of training, a higher exploration rate results in varying rewards. As training continues, the ε-greedy policy transitions from exploration to exploitation, leading to stable action choices and energy-saving decisions. This learning process reaches convergence when the average reward per round and the node duty-cycle decisions stabilize over multiple rounds.

The combined convergence of the hybrid optimizer and the DRL agent ensures that the proposed framework attains stable routing and scheduling decisions without any oscillations, thus facilitating reliable and energy-efficient network operation. The DQN model is stored only at the base station (≈ 10.8 MB), while sensor nodes maintain only small state vectors (< 100 bytes). Therefore, the proposed DRL mechanism does not impose significant memory requirements on sensor nodes.

Comparative analysis

In this section, the performance of the proposed RL-MHWO–GWO model is compared with various existing approaches, including LEACH³⁰, ABCACO³¹, AHCS GWO³², IQ-ABC³³, and WAOA³⁴. Table 4 compares the baseline parameters of the proposed model with those of existing models. All comparative protocols were implemented in the same MATLAB environment using identical radio and energy models to eliminate simulation bias. Simulation scripts were executed under controlled conditions to ensure consistent environmental assumptions across all evaluated protocols.

Table 4.

Baseline parameter.

Protocol	Key parameters
LEACH	p = 0.1
WAOA	population = 30, iter = 50
AHCS–GWO	wolves = 25
IQ-ABC	colony size = 40
Proposed	pop = 30, iter = 50

Compared with EMO-PEGASIS²⁷, the proposed RL-MHWO–GWO solution has better performance in all key aspects. The packet delivery ratio is improved from 96.8 to 97.8%, and the average end-to-end delay is decreased from 79.2 to 38.0 ms due to multipath parallel transmission rather than chain routing. Energy consumption is more evenly distributed through DRL-controlled sleep schedules, resulting in an increased network lifetime and residual energy. Moreover, the WOA–GWO hybrid optimization approach can ensure a more stable selection of the cluster-head and avoid congestion hotspots, thus improving the network throughput to 72 kbps. Table 5 shows the overhead summarization of the proposed model. In conclusion, the proposed solution based on reinforcement learning, metaheuristic optimization, and multipath routing has better performance than the two-phase ML architecture of EMO-PEGASIS.

Table 5.

Overhead summarization.

Component	Size	Location
DQN model	~ 10.8 MB	Base station
Node state report	12–16 bytes	Sensor node
Policy broadcast	~ 250 bits	Base station
Per-round control overhead	~ 3–4 kB	Network

Figure 4a–d shows the comparative results of the proposed framework and traditional routing methods in terms of PDR, throughput, end-to-end delay, and network lifetime. The proposed framework achieves a PDR of 97.8%, a throughput of 72 kbps, minimized end-to-end delay, and maximized network lifetime, demonstrating its efficiency for reliable data transmission and fast communication. Moreover, it balances energy consumption and maximizes network sustainability. On the other hand, the traditional methods have decreased performance in terms of delivery ratio, data transfer rate, end-to-end delay, and network lifetime. Therefore, this comparative analysis demonstrates the superiority of the proposed framework for providing energy-efficient, reliable routing in WSNs.

Fig. 4.

Comparative analysis for (a) PDR, (b) Throughput, (c) End-to-End delay, and (d) Network lifetime.

Figure 5a,b show the comparative analysis of energy consumption and residual energy achieved by the proposed model and other existing techniques. Compared to the existing approaches, the proposed model consumes only 36 J of energy and maintains 5 J of residual energy. The relatively high throughput supported by the proposed framework is mainly due to the multipath routing algorithm and intelligent sleep scheduling. By establishing multiple stable, energy-efficient paths between cluster heads and the base station, the proposed method effectively reduces congestion and packet retransmissions. Consequently, data packets are transmitted more reliably and faster, resulting in higher throughput.

Fig. 5.

Comparative analysis for (a) Energy consumption, (b) Residual energy.

Likewise, the extended network lifetime results from balanced energy consumption among sensor nodes. The DRL-based sleep scheduling strategy avoids unnecessary node operations by dynamically turning low-utility nodes into sleep mode, while the hybrid WOA–GWO algorithm chooses cluster heads based on residual energy and load balancing. This integrated strategy avoids premature exhaustion of vital nodes, delays node failures, and substantially extends the network lifetime. These strategies, taken together, explain the high throughput and extended network lifetime observed in the simulation results. In contrast, existing approaches have high energy consumption and low residual energy. This highlights the proposed model’s capability to optimize energy consumption and prolong the overall network’s sustainability.

The comparative results of the performance of the proposed model with the existing methods regarding the number of alive nodes and dead nodes are depicted in Fig. 6a,b. The proposed model sustains around 222 alive nodes and 28 dead nodes, which indicates efficient energy distribution among the sensor nodes. However, the existing methods result in more nodes being killed and fewer alive nodes. Hence, the proposed model enhances network stability and prolongs its functional time.

Fig. 6.

Comparative analysis for (a) Number of alive nodes, (b) Number of dead nodes.

The comparative analysis of LBR and scalability obtained by the proposed model and other existing methods is depicted in Fig. 7a,b. Contrary to the existing methods, the proposed model provides a better LBR of 96.8% and scalability of 95.4%. This is better since it indicates the capacity to distribute network traffic evenly.

Fig. 7.

Comparative analysis for (a) LBR, (b) Scalability.

The comparative analysis of path loss is presented in Fig. 8. From the Fig. 8, the proposed model yields a path loss of 73 dB, whereas the existing models yield higher path loss. The reduced path-loss value of the proposed model increases communication reliability and is a crucial factor in network energy efficiency, as it reduces retransmissions.

Fig. 8.

Comparative analysis for path loss.

While existing studies^30–34 have explored the use of metaheuristic optimization, reinforcement learning, or their combinations for routing in wireless sensor networks, most approaches apply these techniques in a loosely coupled or sequential manner. In such methods, reinforcement learning is typically used for prediction or parameter tuning, while optimization algorithms operate on static or partially updated network states. In contrast, the proposed RL-MHWO–GWO framework introduces a tightly integrated and co-adaptive design, where reinforcement learning, hybrid optimization, and routing decisions interact dynamically across communication rounds.

First, the DRL component is employed for adaptive sleep scheduling, rather than prediction. It directly controls node participation in each round, thereby influencing energy consumption, network topology, and traffic distribution in real time. Second, the hybrid WOA–GWO algorithm is designed as a phase-adaptive optimization strategy, combining global exploration (WOA) and local exploitation (GWO) with a fitness function specifically tailored to WSN constraints, including residual energy, load balance, communication cost, and link quality. Third, the DRL policy and hybrid optimization are mathematically coupled, where the active node set determined by the DRL agent defines the feasible search space for cluster head selection and routing. This dynamic interaction allows the optimization process to adapt continuously to changing network conditions. These combined features distinguish the proposed model from existing approaches by providing a holistic, adaptive, and system-level optimization framework, rather than a simple combination of individual techniques.

Computational complexity analysis

The computational complexity of different approaches for efficient routing in WSNs is shown in Table 6. The proposed model achieves the least training time of 1.33 h, the least control overhead of 19,300 bytes, the least number of parameters of 2.7 M, the least FLOPs of 3.4 G, and the least routing response time of 0.17 s in WSNs. The proposed model is efficient for routing in WSNs because it requires less computational time and uses fewer resources. However, it is observed that existing approaches, such as LEACH, ABCACO, AHCS GWO, IQ ABC, and WAOA, are not efficient for routing in WSNs because they require additional computational and training time.

Table 6.

Computational complexity analysis.

Methods	FLOPs (G)	Parameters (M)	Response time (s)	Computational time (s)	Control overhead (Bytes)	Training time (Hours)
RL-MHWO–GWO (Proposed)	3.4	2.7	0.17	4.4	19,300	1.33
LEACH30	7.8	5.1	1.59	7.8	35,300	2.39
ABCACO31	8.7	5.8	3.46	12.5	40,200	3.7
AHCSGWO 19	9.5	6.3	5.50	10.9	43,700	3.95
IQ-ABC33	12.0	8.1	7.63	13.7	56,100	5.79
WAOA34	11.2	7.5	4.41	8.9	52,000	4.5

Time complexity analysis

Let N denote the number of sensor nodes, P the population size of the optimizer, and the maximum number of iterations. For each iteration, the hybrid WOA–GWO algorithm performs the following operations, Eqs. (64)–(68):

1. Position update (WOA/GWO): Each candidate solution updates an N-dimensional position vector. This step has a time complexity of.

64

• 2.Binarisation and repair: Continuous solutions are converted into binary cluster-head vectors and repaired to satisfy cardinality constraints on cluster heads. Both binarisation and repair require a linear scan over N nodes, resulting in.

65

• 3.Fitness evaluation: Assign each non-cluster-head node to the nearest feasible cluster head, and compute energy and load-balancing terms. In the worst case, this involves comparing each node against all selected cluster heads, leading to a complexity of.

66

Where K is the number of cluster heads. Since K ``N and typically with small , this term is approximated as

67

Combining all steps, the overall time complexity per iteration is

68

Therefore, the total time complexity of the hybrid WOA–GWO algorithm over all iterations is:

69

This linear necessity on the number of nodes varieties the proposed hybrid optimization method computationally scalable and fit for large wireless sensor networks.

Limitations of the proposed framework

Despite the positive developments in performance efficiency gain with the help of the proposed framework, some limitations are identified with the proposed framework. First and foremost, additional control overheads are introduced with the DRL agent’s centralized training approach, which could be a scalability issue in large-scale dynamic topologies. The hybrid approach of WOA–GWO could also introduce increased computational complexity in the base station compared to the traditional heuristic approach. Additionally, it is assumed that accurate residual energy, link quality, and node location estimates are available for the proposed model, which could be impacted by measurement errors and channel changes. Finally, security and robustness are not considered in the proposed framework.

Conclusion and future scope

In the case of RL-MHWO–GWO, intelligent cluster head selection and multipath routing optimization are achieved by integrating reinforcement learning with hybrid metaheuristic optimization. The proposed system ensures the minimization of energy consumption, balanced network loading, and routing efficiency. The proposed system has been validated with large-scale simulation experiments, which prove the effectiveness of the proposed system and the fact that it is better than the existing procedures and improves with the increase of sensor nodes. Although the proposed model for RL-MHWO–GWO is strong, it has a limitation in the level of precision in the measurement of residual energy, signal power, and proximity. In the future, there will be an improvement in the reliability of the decision-making process with the incorporation of state-of-the-art predictive model technology. The results of the simulations show that the proposed model for routing in the network is effective. The proposed model offers a packet delivery ratio of 97.8%, a throughput of 72 kbps, energy consumption of 35 J, and a network lifetime of 9500 rounds with balanced residual energy in the sensor nodes. The proposed model for routing in the network is robust, scalable, and energy-efficient compared to other routing protocols, as indicated by the improvement in all the results. Future work will be carried out to improve upon these limitations and introduce a robust and secure routing protocol. Additionally, since the RL-MHWO–GWO mainly emphasizes the performance and energy efficiency of the system, future research will also include lightweight cryptographic protocols and trust models to enable safe and attack-resistant data transmission for the system. Such protocols will enable the system to handle the emerging requirements for network security and reliability, as well as increase the potential of the system.

Author contributions

R. Nareshkumar: Conceptualisation, Methodology, Writing - Original Draft, Investigation, Validation. Prabu Selvam: Conceptualisation, Methodology, Writing - Original Draft, Supervision, Investigation. Karthikeyan Kaliyaperumal: Methodology, Writing - Original Draft, Investigation, Data Curation, Investigation. A. Hussain: Methodology, Writing - Review and Editing, Project administration, Validation, Funding acquisition. Uma Maheswari: Methodology, Writing - Review and Editing, Project administration, Validation, Project administration.

Funding

The authors received no specific funding for this study.

Data availability

No datasets were generated or analysed during the current study.

Code availability

Not applicable.

Materials availability

Not applicable.

Declarations

Competing interests

The authors declare no competing interests.

Ethics approval and consent to participate

This study did not require ethical approval.

Consent for publication

Not applicable.