A reinforcement learning-enhanced discrete zebra optimization algorithm for solving the traveling salesman problem

Abstract

In this study, a novel hybrid algorithm named the Reinforced Zebra Optimization Algorithm (RZOA) is proposed to solve the Traveling Salesman Problem (TSP). Initially, a discrete version of the Zebra Optimization Algorithm, referred to as DZOA, was developed, in which the continuous relationships of the original ZOA were transformed into a set of discrete operators to effectively update the routes. Subsequently, to enhance the intelligence of the operator selection process and to maintain a dynamic balance between exploration and exploitation, a mechanism based on Deep Reinforcement Learning was designed. Within the framework of the proposed RZOA, each zebra agent adaptively and self-learnedly makes decisions using a Deep Q-Network (DQN). Each agent observes both its own state and the overall population state, employs the DQN to select the optimal operator, and improves its decision-making policy dynamically through an experience replay memory. The state–action–reward structure was carefully designed to ensure an intelligent and balanced learning behavior between local and global search processes. The results of numerical experiments conducted on 42 standard benchmark datasets from the TSPLIB repository demonstrate that the proposed RZOA exhibits significant superiority over the compared algorithms in terms of solution quality, convergence speed, and performance stability. In particular, the proposed method achieves near-optimal performance with an average Percentage Deviation of the Best solution (PDB) below 1% and an average Percentage Deviation of the Average solution (PDA) typically below 0.5% for small and medium-scale instances, while maintaining PDA values within 4–5% for large-scale problems. Moreover, RZOA outperforms competing algorithms in the majority of benchmark cases, demonstrating superior robustness and consistency. Furthermore, the Friedman and Wilcoxon statistical analyses confirm this superiority at a 95% confidence level. Overall, by integrating the global search capability of the Zebra Optimization Algorithm with the adaptive decision-making power of the DQN, the proposed RZOA provides a novel, intelligent, and efficient approach for solving complex combinatorial optimization problems.

Introduction

In recent years, extensive research has been conducted to address various optimization problems, including discrete single-objective, multi-objective, and combinatorial problems^1–8 through the use of metaheuristic algorithms, which have become increasingly popular. Among these approaches, swarm intelligence–based algorithms have attracted significant attention. Such algorithms are generally inspired by collective behaviors and population-based patterns observed in nature. Representative examples include the Particle Swarm Optimization (PSO)⁹, Bee Colony Optimization¹⁰, Gorilla Troops Optimizer (GTO)¹¹, African Vulture Optimization Algorithm (AVOA)¹², and the Grey Wolf Optimizer (GWO)¹³. The key characteristic shared by these algorithms lies in the exchange of information among individuals within the population during the problem-solving process.

One of the most well-known and fundamental combinatorial optimization problems frequently used to evaluate the performance of swarm intelligence algorithms is the Traveling Salesman Problem¹⁴. This problem has broad applications in domains such as robot path planning, telecommunication network design, production scheduling, vehicle routing, and integrated circuit design. In the TSP, the goal is to determine a tour that visits a given set of cities exactly once and returns to the starting city, while minimizing the total travel distance¹⁵. The TSP is theoretically classified as an NP-hard problem, meaning that the computational time required to obtain the optimal solution increases exponentially with the number of cities. Consequently, exact methods such as branch and bound, integer programming, and cutting-plane algorithms are only practical for small-scale instances and become computationally infeasible for medium or large-scale problems due to their excessive time and memory requirements. Therefore, the use of metaheuristic algorithms, capable of producing high-quality near-optimal solutions within acceptable computational time, becomes essential¹⁶. In line with this need, a variety of metaheuristic algorithms based on swarm intelligence have been introduced over the past decades for solving the TSP, such as the Discrete Grey Wolf Optimizer (D-GWO)¹⁷, the Discrete Whale Optimization Algorithm (DWOA)¹⁸, and the Discrete Bat Algorithm (DBA)¹⁹. Despite their relative success, most of these methods rely on fixed rules or random operator selection to explore the solution space. This often leads to an imbalance between exploration and exploitation, causing premature convergence or trapping the algorithm in local optima.

In 2022, Trojovská et al. introduced a new metaheuristic algorithm called the Zebra Optimization Algorithm²⁰, designed for solving both constrained and unconstrained continuous optimization problems. Due to its simple structure, lack of numerous control parameters, and robust performance, ZOA has been successfully applied to various fields, such as PID controller design optimization²¹, 3D path planning based on Bézier curves²², microgrid management and scheduling²³, autonomous vehicle routing²⁴, and DC motor drive system controller design²⁵. Although ZOA has been employed in many optimization tasks, it has not yet been applied to the TSP.

In the proposed framework of this study, the core of the DZOA is integrated with a decision-making layer based on DQN. In this structure, each zebra agent, at each iteration, observes both its internal state and the global state of the population. This combined information is fed into the DQN, which outputs the optimal operator or control parameter to be applied in the next step. This mechanism ensures that operator selection and parameter adjustment are neither static nor random, but are determined dynamically based on the current state of the search process. The resulting algorithm, referred to as the RZOA, therefore combines the global exploration capability of ZOA with the adaptive, intelligent decision-making ability of deep reinforcement learning. To evaluate the performance of the proposed approach, the RZOA was compared against nine representative metaheuristic algorithms from the literature using 42 benchmark instances from the TSPLIB dataset. The results of the experiments and statistical analyses clearly demonstrate the effectiveness and superiority of RZOA.

The main contributions of this research can be summarized as follows:

• A novel discrete formulation of ZOA is proposed for solving combinatorial optimization problems. In this formulation, the continuous search mechanism of ZOA is transformed into a set of effective discrete operators (e.g., Swap, Insertion, 2-opt, and 3-opt), enabling efficient route updates in TSP.

• A reinforcement learning-based operator selection mechanism is developed, in which a DQN network adaptively selects the most appropriate search operator based on the current search state, including solution quality, population diversity, and search progress.

• A novel state–action–reward design is introduced, where both individual-level and population-level features are incorporated into the state representation, and a normalized reward function is defined to ensure stable and scale-independent learning behavior.

• A tightly integrated hybrid framework is proposed, where reinforcement learning is deeply embedded into the optimization process, allowing dynamic and learned balancing between exploration and exploitation rather than relying on predefined rules.

• The proposed framework is generalizable and can be extended to other combinatorial optimization problems beyond TSP, providing a flexible and intelligent paradigm for adaptive operator selection in discrete search spaces.

• Extensive experiments on 42 TSPLIB benchmark instances demonstrate that the proposed RZOA achieves superior performance in terms of solution quality, convergence speed, and stability compared to several state-of-the-art algorithms.

The remainder of this paper is organized as follows: Section 2 presents a review of related works. Section 3 introduces the proposed RZOA algorithm. Section 4 discusses the simulation results obtained from implementing RZOA and comparing its performance with other advanced metaheuristics. Finally, Section 5 concludes the paper and outlines potential directions for future research.

Related work on TSP and ZOA

Research on the meta-heuristic algorithms of TSP

The traveling salesman problem

The Traveling Salesman Problem is defined as follows: Suppose there is a set of n cities represented as N= {1, 2, …, n}, and the distance between any two cities i and j is denoted by d_ij. The decision variable x_ij is defined according to Eq. (1):

1

The objective function aims to minimize the total tour length, as expressed in Eq. (2):

2

subject to the constraint that each city must be entered and left exactly once, as shown in Eq. (3):

3

To prevent the formation of disjoint subtours, the constraint in Eq. (4) is applied:

4

If the coordinates of each city are known, the distance between cities i and j can be calculated using the Euclidean distance formula given in Eq. (5):

5

The goal is to find a permutation of cities that minimizes the value of the objective function L. Figure 1 illustrates an example of the Traveling Salesman Problem with eight cities, where both a random tour and the optimal tour are shown for comparison.

Fig. 1.

Schematic representation of a sample Traveling Salesman Problem with 8 cities.

Related work on TSP

In this section, the conventional algorithms for solving the Traveling Salesman Problem, as well as deep learning–based approaches applied to TSP, are reviewed. Traditional TSP-solving algorithms are generally categorized into three main groups. The first group includes exact methods such as the Branch and Bound algorithm^26,27, which aim to find the deterministic optimal solution to the problem. However, since these methods are non-polynomial in complexity, their execution time increases exponentially with problem size. Therefore, they are impractical and time-consuming for large-scale TSP instances. The second group consists of approximation methods²⁸, which provide near-optimal solutions within shorter computational times by simplifying the model or employing mathematical approximations. Finally, the third group comprises metaheuristic algorithms, which seek to find high-quality approximate solutions in a reasonable amount of time. Owing to their strong ability to explore vast solution spaces, these methods have gained significant attention in recent years. Among the most well-known algorithms in this category are Ant Colony Optimization (ACO)^29–31, Particle Swarm Optimization (PSO)^32–35, Artificial Bee Colony (ABC)^36–39, Discrete JAYA⁴⁰, Bat Algorithm (BA)^41–43, Discrete Sparrow Search Algorithm (DSSA)⁴⁴, Discrete Komodo Algorithm (DKA)⁴⁵, Discrete Differentiated Creative Search (DDCS)⁴⁶, Pathfinder Algorithm (PFA)⁴⁷, Artificial Rat Algorithm (ARA)⁴⁸, Cuckoo Search (CS)^49,50, and Discrete Spider Monkey Optimization (DSMO)⁵¹.

Deep learning–based algorithms developed for solving the Traveling Salesman Problem are typically classified into four main categories⁵². The first category includes end-to-end constructive algorithms, which employ neural networks to generate complete solutions from scratch^53,54. These methods are known for their high computational speed, but they generally produce lower-quality solutions. The second category includes end-to-end improvement algorithms, which iteratively refine and improve initial solutions to achieve higher-quality results, though at the cost of greater computational effort^55,56. The third category encompasses hybrid algorithms based on large language models, which can automatically generate and improve heuristics by leveraging vast amounts of data and pre-trained knowledge. These approaches automate much of the algorithm design process, traditionally dependent on human expertise, in a data-driven and self-learning manner^57–59. Finally, the fourth category, which is the focus of this study, includes direct hybrid algorithms. In these approaches, deep learning is directly integrated into the heuristic search framework of optimization algorithms. In other words, such methods combine the pattern-learning capability of deep neural networks with the systematic search ability of heuristics to identify optimal solutions in the search space⁶⁰. In this model, deep learning is employed to detect underlying features and hidden patterns within TSP data, and the acquired knowledge is then used to guide the search process. Consequently, the algorithm benefits both from the generalization ability of deep learning and the structured exploration mechanism of heuristic methods. Several representative examples of these hybrid algorithms are discussed below.

Xing and Tu integrated Graph Neural Networks (GNNs) with Monte Carlo Tree Search (MCTS)⁶¹. In this approach, the GNN captures the structural dependencies and interactions among graph nodes, generating prior probabilities for node selection. These probabilities guide MCTS to explore more promising search paths. Experimental results demonstrated that the GNN–MCTS framework generalized effectively to large-scale problems with up to 1,000 nodes, even when trained on small instances. Wang et al. proposed a Self-Supervised Reinforcement Learning (SSRL) framework that combines self-supervised learning with the Lin–Kernighan–Helsgaun (LKH) metaheuristic⁶². This model employs a dual-decoder GNN architecture: the edge decoder learns edge scores via self-supervised learning, while the node decoder refines node penalties using reinforcement learning. SSRL successfully reconstructs and enhances the core components of LKH without expert-designed rules. It leverages Self-Supervised Q-Learning and Self-Supervised Actor-Critic methods to improve learning stability and efficiency. Experimental findings confirmed that SSRL outperforms LKH, VSR-LKH, and Neuro LKH in large-scale TSP instances, achieving better generalization and higher computational efficiency.

In⁶³, a method was proposed to enhance the generalization capability of pre-trained Graph Convolutional Networks (GCNs) by integrating them with the LKH algorithm. Large TSP instances were divided into smaller subproblems, each containing a fixed number of cities consistent with the GCN training size. The resulting sub-solutions were combined through an attention-based merging mechanism. The merged probabilistic heatmap was then used to construct a set of candidate edges for LKH. Experimental results demonstrated that this approach significantly improved GCN generalization and achieved superior performance compared with LKH, Neuro LKH, VSR-LKH, and Att-GCN+MCTS across instances ranging from 20 to 10,000 cities. Ye et al. introduced Deep ACO⁶⁴, a GNN-based metaheuristic that integrates learned heuristics into Ant Colony Optimization. The framework consists of two stages: in the first stage, the GNN learns problem-specific heuristic maps; in the second stage, ACO utilizes these maps to guide exploration and perform local optimization. Deep ACO also employs Neural-guided Local Search to escape local optima by alternating between standard improvement steps and heuristic-driven exploration. Experimental results indicated that Deep ACO outperforms classical ACO variants and demonstrates competitive performance with deep learning–based algorithms on random uniform Euclidean instances of up to 1,000 nodes.

Research on ZOA

The basic ZOA

In this section, the Zebra Optimization Algorithm inspired by the natural behavioral patterns of zebras is introduced, and its mathematical model is described. Zebras are members of the equine family that primarily inhabit the eastern and southern regions of Africa. These animals are distinguished by their characteristic black and white striped coats, which not only create a unique appearance but also serve vital biological functions, such as camouflage against predators and protection from biting insects⁶⁵. The social behavior of zebras in the wild generally follows two main patterns: foraging behavior and defensive behavior against predators. During foraging, the leading zebra acts as a pathfinder, navigating through plains and guiding the herd toward grasslands and food sources, while the other zebras follow it resulting in a coordinated and collective movement pattern⁶⁶. Conversely, when faced with threats, zebras exhibit two distinct defense strategies: against predators such as lions, they adopt a zigzag escape pattern to confuse the attacker; however, when facing smaller predators like wild dogs or hyenas, they form protective clusters, moving cohesively to intimidate or disorient the enemy⁶⁵. The mathematical modeling of these two intelligent behaviors forms the foundation of the ZOA.

The ZOA is a population-based optimization algorithm, where each zebra represents a candidate solution to the problem, and its position in the search space corresponds to a set of decision variable values. The optimization process consists of multiple iterations, during which the positions of the population members are updated based on two key behaviors of zebras in nature: (i) foraging, and (ii) defensive strategy against predators. In the foraging phase, the leading zebra representing the best solution in the population guides the other zebras toward its position. This process is mathematically modeled by Eq. (6)^66,67:

6

where X_i​(t) is the position of zebra i at iteration t, P_Z is the position of the leading zebra (the best member of the population), r is a random number uniformly distributed in [0,1], and I is a control coefficient regulating the intensity of following behavior, typically in the range^1,2. This relation ensures that population members gradually move toward the best current solution, thereby exploring more promising regions of the search space.

In the defensive phase, zebra positions are updated by simulating their behavior when attacked by predators. When the predator is a lion, zebras escape in a zigzag pattern, representing the exploration phase⁶⁸; when smaller predators such as hyenas or wild dogs attack, the zebras cluster together and perform coordinated, aggressive movements, forming the exploitation phase⁶⁵. This phase is modeled as follows, using Eq. (7):

7

Here, A_Z represents the position of the current best solution, R is the movement range adjustment coefficient (a small constant, typically 0.001), t is the current iteration number, T is the total number of iterations, and r is a uniformly distributed random number in [0,1]. The algorithm stochastically switches between the two defensive strategies (escape or group attack) to maintain a balance between exploration and exploitation. The position update process continues through these two phases until the algorithm terminates. In each iteration, the fitness of every zebra (solution) is calculated, and the best-performing individual is identified as the current leader. Upon completion, the best candidate solution found throughout the iterations is reported as the final optimal solution.

Related work on ZOA

In recent years, numerous studies have been conducted to improve the performance of ZOA and expand its applications across various domains.

Krithiga et al.⁶⁹ introduced a hybrid version of ZOA combined with Long Short-Term Memory neural networks, termed MLSTM-ZOA. The primary goal of this work was to minimize Total Harmonic Distortion (THD) in multilevel inverters. Experimental results demonstrated that the proposed model outperformed algorithms such as MPA, HHO, MGMPA, and SF-BOA in both convergence speed and solution accuracy, achieving a THD reduction between 41% and 81%. Subsequently, El-Hageen et al.⁷⁰ proposed an enhanced version of ZOA incorporating a Chaotic Map, referred to as CZOA, designed for optimal Cluster Head selection in Wireless Sensor Networks to minimize energy consumption and extend network lifetime. Results indicated that CZOA effectively avoided local optima and improved population diversity, leading to significant energy savings and an increased number of active nodes in the network.

Similarly, Abdelmalek et al.⁷¹ employed ZOA for Maximum Power Point Tracking optimization in photovoltaic systems. Under Partial Shading Conditions and dynamic environments, ZOA achieved a tracking efficiency of 99.95% and a 15% faster convergence speed compared to competing algorithms such as GWO, PSO, FPA, and WOA. Experimental results also revealed improved stability and reduced oscillations at the operating point, demonstrating that ZOA is a robust and reliable approach for solar energy systems. In the power systems domain, Arkwazee and Abttan⁷² developed an improved version called IMZOA to solve the Combined Economic Emission Dispatch problem. IMZOA integrates adaptive search mechanisms and cubic modeling to enhance performance across power systems of different scales (6, 31, and 110-unit test systems). Results showed that IMZOA not only reduced fuel costs by up to 0.8% but also significantly lowered emissions compared to PSO, LM, SA, and other advanced algorithms.

In the field of artificial intelligence, Emine Baş and Şaban Baş⁷³ applied ZOA for the training of Artificial Neural Networks. Their research demonstrated that ZOA can serve as an effective alternative to traditional gradient-based learning methods, improving convergence rate and prediction accuracy. Compared with GOA, PDO, and OOA, the ZOA-trained network exhibited greater stability and lower prediction error. Finally, Ren and Feng²¹ introduced the Modified ZOA (MZOA) for the automatic tuning of PID controller parameters. This version integrates a chaotic map, Householder mirror learning, and a tangent flight strategy to overcome issues such as premature convergence and slow optimization observed in the standard ZOA. Simulation results confirmed that MZOA achieves more accurate and faster PID parameter tuning, yielding a system response curve that closely tracks the reference signal.

The proposed reinforced ZOA for TSP

Discretization process of ZOA and structure of DZOA

In its original form, ZOA was designed for continuous optimization problems, where the positions of agents in the vector space R^d are updated using social alignment and predator avoidance equations²⁰. This property makes ZOA highly effective in solving continuous mathematical functions. However, the permutational nature of combinatorial optimization problems such as TSP does not align with such continuous movements. In these problems, the search space consists of a finite set of permutations, each representing a specific visiting order of cities. Therefore, it is necessary to redesign the ZOA framework so that the agents’ movements are interpreted not as continuous vector shifts but as discrete transformations among permutations. This redesign process is known as the discretization of ZOA (DZOA).

The discretization process of ZOA proposed in this study is based on replacing continuous motion equations with discrete local operators. In the continuous version, an agent X_i moves to a new position in the continuous space at each iteration based on its relative position to other agents. In contrast, in the discrete version, the same agent is represented as a tour T_i = (c₁, c₂, …, c_n), and its movement corresponds to applying a neighborhood operator on that permutation. The set of operators in DZOA is designed to maintain a balance between exploration and exploitation. These operators include: Swap, which exchanges the positions of two cities in the tour (a small, exploitation-oriented movement), Insertion, which removes a city and reinserts it in another position, 2-opt, which reverses a segment of the tour by removing and reconnecting two edges, and 3-opt, which deletes three edges and reconnects the resulting subpaths in multiple possible configurations. Each operator contributes uniquely to improving and diversifying the search process. The following describes each operator mathematically and schematically.

In the Swap operator, two cities are selected, and their positions in the permutation are exchanged^{74, as illustrated in Figure 2}. If the current tour is represented as X= [x₁​, x₂​, ., x_i​, ., x_j​, ., x_n​] then after swapping positions i and j, the new tour becomes X’= [x₁​, ., x_j​, ., x_i​, ., x_n​]. This operator makes minor adjustments and is primarily used in the exploitation phase.

Fig. 2.

Illustration of the Swap operator in the TSP path.

In the Insertion operator, a city is selected and reinserted into a new position in the tour^{75, as shown in Figure 3}. If city x_i is removed and inserted after city x_j, the resulting tour is X’= [x₁, ., x_j, x_i, x_j+1, ., x_n]. This operator can induce relatively large changes and is typically used in the early exploration stages.

Fig. 3.

Illustration of the Insertion operator in the TSP path.

The 2-opt operator is one of the most widely used local search methods for routing problems^{76, and its process is depicted in Figure 4}. It removes two edges from the tour and reverses the intermediate segment to eliminate route intersections. Mathematically, if edges (x_i, x_i+1) and (x_j, x_j+1) are removed and the middle segment is reversed, the new tour becomes: X'= [x₁, ., x_i, x_j, x_j−1, ., x_i+1, x_j+1, ., x_n]. This operator is mainly applied during the exploitation phase to enhance convergence accuracy.

In the 3-opt method, three edges are removed and then reconnected in several possible configurations. Compared with 2-opt, this operator produces more substantial modifications and thus can be used for both exploration and exploitation^{77, as illustrated in Figure 5}. The general formulation can be written as X’= Reconnection (X, {(x_i, x_j), (x_k, x_l), (x_m, x_n)}), where the function Reconnection is responsible for reconnecting the three separated segments.

Fig. 4.

Illustration of the 2-opt operator in the TSP path.

Fig. 5.

Illustration of the 3-opt operator in the TSP path.

Additionally, in the discrete version of ZOA, a new concept called w_i is introduced as a weight coefficient determining the intensity or range of an agent’s movement. It is modeled as the probability of selecting an operator, formulated in Eq. (8):

8

where O_j denotes the j-th operator among m available operators, and α_j weights are more likely to choose exploratory operators such as 3-opt, while those with lower weights tend to prefer exploitation-oriented operators such as Swap. The position of agent i at iteration t is represented by the tour , which is updated according to Eq. (9):

9

where T_i is the permutation of agent i at iteration t, O is the selected operator from the defined operator set, is the agent’s weight at iteration t which determines the probability distribution over the operators and N represents the operator-application function on the permutation. This equation serves as the direct mapping of continuous ZOA equations into the discrete permutation space. The pseudocode for the DZOA algorithm is presented in Algorithm 1, detailing how operators are applied and how permutations are iteratively updated.

Algorithm 1.

DZOA for TSP.

The provided pseudocode outlines the overall procedure of DZOA for solving the TSP. In this algorithm, each zebra agent represents a tour (route) among cities, and the goal is to find the shortest possible path. Initially, a population of agents is randomly generated, where each agent T_i corresponds to a random permutation of cities. Each agent is also assigned a weight W_i, which defines its tendency toward exploratory or exploitative moves. In the second step, the objective function the total tour length is evaluated for each route using the EvaluateTour subroutine. The best initial tour is recorded as BestTour, and its corresponding cost as BestCost. During each iteration, for every agent, the probabilities of selecting discrete operators (Swap, Insertion, 2-opt, and 3-opt) are computed based on its weight W_i using Eq. (8). One operator is then chosen probabilistically and applied to the current tour T_i to generate a new tour T_{i_new} via the subroutine N (T_i, O, W_i). The new tour is evaluated using EvaluateTour; if its total path length is shorter than before (i.e., an improvement is achieved), it replaces the previous tour. If necessary, the agent’s weight is adaptively updated to maintain a balance between exploration and exploitation. After all agents are processed within an iteration, the best current tour in the population is compared with BestTour, and if a superior tour is found, both BestTour and BestCost are updated. Finally, upon algorithm termination, the best tour and its corresponding cost are returned as the final output.

Deep reinforcement learning and the design of state, action, and reward

In classical metaheuristic algorithms, the search mechanism is typically based on static rules or randomized choices. For instance, in many algorithms, the selection of search operators is predefined and non-adaptive meaning that throughout all iterations, the algorithm employs fixed probabilities or simple random schemes to choose operators. This characteristic often causes the algorithm to either overemphasize exploration, wasting computational effort in unpromising regions, or conversely, to overexploit the search space, thereby becoming trapped in local optima⁷⁸. To overcome these limitations, the idea of integrating reinforcement learning (RL) with metaheuristic algorithms has recently emerged as a promising approach⁷⁹. RL is a branch of machine learning in which an agent interacts with a dynamic environment and, through repeated trial and error, learns which actions to take in each state to maximize its long-term cumulative reward. The agent has no prior knowledge of the environment; instead, it gradually learns an optimal decision-making policy purely from experience and feedback. The learning process in RL is defined through four main components:

Environment

the problem the agent aims to solve (here, TSP and the population structure of DZOA).

Agent

the decision-making entity that observes the environment’s state and selects an action according to its current policy.

Reward

a numerical feedback signal from the environment that evaluates the quality of the agent’s action.

Policy

a mapping from states to actions that governs the agent’s behavior over time.

In its simplest form, the agent’s decision-making rule is expressed by the Q-learning update Eq. (10):

10

In this equation which forms the foundation of Q-learning, Q (s_t, a_t) represents the value of taking action a_t in state s_t; r_t is the immediate reward; α is the learning rate; and γ is the discount factor. Over successive iterations, the agent updates its Q-values and ultimately derives its optimal policy according to Eq. (11):

11

In other words, in each state, the agent selects the action with the highest estimated Q-value. While Q-learning is effective for simple problems, it becomes impractical in environments with large state and action spaces, such as combinatorial optimization problems like TSP, where maintaining a full Q-table is computationally infeasible. To address this limitation, Deep Reinforcement Learning (DRL) methods have been proposed, in which a deep neural network is used to approximate the Q-function. This approach known as DQN was introduced by Watkins and colleagues and achieved human-level performance in complex environments such as Atari games⁸⁰. In DQN, the Q-function is represented not as a lookup table but as a parametric neural model with weights θ, expressed as: ⁸¹. The network is trained by minimizing the loss function given in Eq. (12):

12

Where . Here,   denotes the parameters of the target network, which are updated with a delay relative to the main network to enhance learning stability. Additionally, a replay memory buffer is employed to store past experience tuples , allowing random and independent sampling that breaks temporal correlations between consecutive data points.

In the hybrid RZOA framework, each zebra agent acts as an RL agent. In every iteration, the agent observes its own state and the state of the population, and then via the DQN selects the optimal action (i.e., the appropriate operator for generating a new tour). The design of the state, action, and reward components ensures that the agent can dynamically balance exploration and exploitation throughout the search process. The state in this research is modeled as a vector according to Eq. (13):

13

where L_i is the current tour length of agent i, L_best is the best tour found by the population, represents the performance gap between the agent and the best individual, σ is the population diversity index, iter is the current iteration number, and ratio denotes the progress ratio of the algorithm. This state vector is fed into the DQN, enabling the agent to decide which modification to apply to its tour. The action set corresponds to the four discrete operators defined earlier: A = {a₁, a₂, a₃, a₄} = {Swap, Insertion, 2-opt, 3-opt}.

For each state, the DQN outputs the corresponding Q-values for all actions, and the agent selects one action based on the ε-greedy policy, defined in Eq. (14):

14

At the beginning of the algorithm, the value of ε is kept high to encourage exploration; as learning progresses and the agent gains more knowledge of the search space, ε is gradually decreased to promote exploitation of the learned policy. The reward function plays a crucial role in training the DQN, as it determines the learning trajectory. In this study, the reward is defined based on the relative improvement in tour quality, encouraging the agent to continuously refine its path. The reward function is expressed in Eq. (15):

15

To make the agent more sensitive to the degree of improvement, a normalized reward function is also employed, as defined in Eq. (16):

16

This formulation ensures that the reward is proportional to the percentage reduction in tour length rather than its absolute change. As a result, the agent continues to receive meaningful feedback even during the later stages of optimization when improvements become smaller. The architecture of the DQN used in this framework consists of three layers: an input layer with six state features, two hidden layers with nonlinear activation functions (ReLU), and an output layer with four Q-values corresponding to the four possible actions. This design is simple yet efficient, offering fast convergence, stable learning behavior, and effective adaptive control between exploration and exploitation during the RZOA optimization process.

The proposed RZOA

Overview and objective

The hybrid RZOA algorithm is designed to integrate the local search capability of the DZOA with the adaptive decision-making ability of deep reinforcement learning. In this approach, the DZOA core is responsible for generating and refining tours within the discrete search space of the TSP, while the DQN agent dynamically guides the selection of operators or the adjustment of weights. This integration transforms the search process from a static and random mechanism into an intelligent, experience-driven policy that achieves higher stability and solution quality in the long run. The proposed framework comprises three main components: (1) the DZOA core, which executes discrete operators such as Swap, Insertion, 2-opt, and 3-opt, and is responsible for tour refinement; (2) the DQN agent, acting as a centralized or shared decision-maker that receives the states of both agents and the population and selects the most appropriate action; and (3) the state–action–reward structure, which governs the interaction between these two layers. This interaction is reinforced through a solution acceptance policy that maintains a balance between exploration and exploitation during the search process.

Decision-making logic

The decision-making logic in the hybrid RZOA is founded on the dynamic interaction between the agent’s current state, the action selection through the DQN network, and the acceptance or rejection of the newly generated solution. In each iteration, the agent’s state vector is first constructed, consisting of the following features: the ratio of the agent’s tour quality to the best tour found so far, the agent’s weight as an indicator of its tendency toward exploration or exploitation, the population diversity index representing the dispersion of solutions, and the progress ratio of the current iteration to the total number of iterations. This state vector is fed into the DQN, which computes the Q-values corresponding to the possible actions. Using the ε-greedy policy, one action is then selected meaning that with probability ε, an action is chosen randomly to encourage exploration, and with probability 1–ε, the action with the highest Q-value is selected to promote exploitation. The selected action may correspond to one of the neighborhood operators Swap, Insertion, 2-opt, or 3-opt or to a weight adjustment operation (i.e., increasing or decreasing W_i). If a weight adjustment is selected, the value of W_i is dynamically updated to enhance the agent’s flexibility in adapting its exploration–exploitation inclination.

After executing the selected operator or weight modification, a new candidate solution is produced, which must be evaluated through the solution acceptance policy. In the simplest case, only improved solutions are accepted. However, to prevent premature convergence and entrapment in local optima, a Metropolis-based probabilistic acceptance rule is also employed, allowing slightly inferior solutions to be accepted with limited probability. Finally, a reward value proportional to the improvement in tour quality is assigned to the agent, and the complete experience tuple including the current state, selected action, obtained reward, and the next state is stored in the experience replay memory. This memory is periodically sampled to train the DQN using the Double DQN algorithm and mini-batch sampling, ensuring accurate and stable Q-value estimation. Through this process, the agent gradually learns which actions are most effective under specific conditions, developing an intelligent and adaptive policy that guides the search toward better solutions over time.

Comprehensive pseudocode

The hybrid RZOA is designed to enhance metaheuristic search efficiency and to intelligently adapt the operator selection process. In the initialization phase, a population of random tours representing zebra agents is generated and evaluated. Adaptive parameters such as migration weights and alignment coefficients are initialized, and both the learning Q-network and the target network are configured. During each iteration, the global population metrics including the best tour length, average diversity, and algorithmic progress ratio are computed. Each agent constructs a state vector containing local and global features and feeds it into the DQN to select one of the four discrete operators (Swap, Insertion, 2-opt, or 3-opt) according to the ε-greedy policy. The selected operator is then applied to the agent’s tour, and the reward is computed based on the improvement or degradation of the tour length. If the new solution improves the objective function, it is accepted directly; otherwise, acceptance is determined according to the Metropolis criterion, enabling the algorithm to escape local optima.

Throughout the execution, each agent’s experiences consisting of state, action, reward, and next state are stored in the experience replay buffer and periodically used to train the DQN. The network parameters are updated by minimizing the loss between the predicted and target Q-values, and the target network parameters are periodically synchronized with those of the main network. In each generation, the exploration rate (ε) is gradually decreased to maintain a proper balance between exploration and exploitation. Upon completion, the best tour obtained during the entire process is reported as the algorithm’s final output. This hybrid framework enables operator selection to be governed not by randomness but by adaptive learning from accumulated experience. Consequently, the algorithm progressively learns an optimal decision-making policy for improving tour quality and demonstrates faster convergence speed and superior solution accuracy compared to the classical DZOA. The pseudocode of the hybrid RZOA is presented in Algorithm 2.

Algorithm 2.

RZOA for TSP.

Time complexity analysis

The computational complexity of the proposed RZOA algorithm is analyzed based on its main components. Let n denote the number of cities, N the population size, and T the maximum number of iterations. The initialization phase requires O (N × n) operations to generate the initial population. During each iteration, the main computational cost arises from the application of discrete search operators such as Swap, Insertion, 2-opt, and 3-opt, as well as the evaluation of candidate solutions. Among these, the most computationally expensive operations are the 2-opt and 3-opt moves, which require O(n²) time in the worst case. Therefore, the dominant time complexity of the optimization process can be approximated as O (T × N × n²). In addition, the reinforcement learning component based on the Deep Q-Network introduces a relatively small overhead. Since the state vector is low-dimensional and only forward propagation is required during the decision-making process, the complexity of the DQN can be considered O (1) per agent per iteration. Consequently, the overall time complexity of the RZOA algorithm is O (T × N × n²), which is comparable to other metaheuristic algorithms that rely on local search operators.

Experimental results

This section presents a comprehensive experimental evaluation of the proposed RZOA algorithm, organized into six subsections. Section  4.1 describes the experimental settings, including the benchmark datasets, performance evaluation metrics, and parameter configurations employed in the simulations. Section  4.2 provides a comprehensive comparison between RZOA and several advanced discrete metaheuristic algorithms, namely DDCS, DKA, DSSA, D-GWO, DJAYA, and DSMO, in order to assess the effectiveness and robustness of the proposed approach. In Sect.  4.3, the performance of RZOA is further compared with Ant System, CBA-NNM, and GA-MARL, which represent classical and learning-based optimization methods. Section  4.4 presents an ablation study to analyze the contribution of each component of the proposed algorithm to the overall performance. Section  4.5 reports the statistical analysis of the obtained results to evaluate the significance of the observed performance differences. Finally, Sect.  4.6 investigates the learning behavior of the reinforcement learning component to provide deeper insight into the adaptive decision-making process within the proposed framework.

Experimental settings

The datasets used in this study were selected from the TSPLIB benchmark library and include 42 problem instances of varying sizes, ranging from small problems with 30 cities to very large problems with up to 2392 cities⁸². The purpose of this diverse selection is to comprehensively and accurately evaluate the performance of RZOA under different problem scales and complexity levels. To clearly and precisely demonstrate the performance of RZOA, four standard evaluation metrics were employed, each describing a specific aspect of the algorithm’s effectiveness:

Best

Represents the shortest distance obtained across all runs, indicating the maximum potential and capability of the algorithm to achieve near-optimal or optimal solutions.

Average

Calculated as the mean of multiple independent runs of the algorithm, providing a statistically robust measure that minimizes the influence of randomness and presents a general view of the algorithm’s average behavior.

Standard Deviation (SD)

Measures the algorithm’s stability by quantifying the variability of the best solutions obtained in multiple runs. It is computed using Eq. (17). A higher standard deviation indicates greater variability and sensitivity to random factors, implying the need for repeated experiments to ensure reliability.

17

where f_i is the tour length obtained in the i-th run, ​ is the mean of all obtained results, and n is the number of algorithms runs.

Percentage Deviation of the Best solution (PDB)

Indicates the deviation of the best solution obtained by RZOA from the theoretical optimum, calculated using Eq. (18).

18

Percentage Deviation of the Average solution (PDA)

Represents the deviation of the average solution obtained by RZOA from the theoretical optimum, computed using Eq. (19).

19

In Eqs. (18) and (19), Optimal denotes the best-known theoretical solution corresponding to the given TSP instance in the TSPLIB dataset.

In the proposed algorithm, part of the parameters belongs to the metaheuristic core DZOA, and the rest correspond to the reinforcement learning layer (DQN), including the learning rate, discount factor, replay memory size, epsilon decay rate, and target network update frequency. These two parameter sets were tuned in coordination to ensure stable learning while preserving the algorithm’s exploratory power in large search spaces. For the DZOA component, the population size typically ranges between 30 and 80, depending on the problem size. Larger populations increase diversity but also raise computational cost. The maximum number of iterations is set between 200 and 400 for small-scale problems and 400–1000 for larger instances. The weight parameter W_i is randomly initialized within the range [0.2, 0.5] and dynamically updated in each iteration based on migration and alignment rules, allowing more successful agents to receive higher weights.

In the deep reinforcement learning layer, the DQN network parameters were selected to ensure both learning stability and convergence speed. The learning rate (α) is set within the range [0.001, 0.005 larger values may cause oscillations in Q-values. The discount factor (γ) typically ranges from 0.9 to 0.99, preserving the influence of future rewards. The replay memory size (C) is generally between 10,000 and 50,000 to prevent sample correlation. The initial value of ε is 1.0, which decreases linearly or exponentially to a minimum of 0.1, encouraging exploration in the early stages and exploitation in the later phases. The target network update frequency also plays a crucial role in learning stability. Rapid updates can increase oscillations in the objective function, whereas overly slow updates delay convergence. In this study, a hard update every 100 training steps and a soft update rate of τ = 0.01 yielded optimal results. Moreover, the reward function required fine-tuning to effectively guide agent learning. Overly large rewards may lead to instability, while excessively small rewards slow down learning. In this implementation, the normalized reward function, defined in Eq. (16), is employed, where values are constrained to the [-1, + 1] range. This normalization allows the agent to interpret relative improvements independent of problem scale. Finally, based on extensive preliminary experiments conducted on standard TSPLIB datasets, the recommended parameter settings for achieving an appropriate balance between accuracy and convergence speed are summarized in Table 1.

Table 1.

Suggested parameters for the RZOA.

Parameter Type	Parameter Name	Symbol	Recommended Range / Value	Description
DZOA Core	Population size	N	100	Number of agents (Zebra tours)
	Maximum iterations	Tmax	1000	Number of main iterations (depends on problem size)
	Initial Zebra weight	Wi	0.5	Determines the initial balance between exploration and exploitation
	Migration rate	β1	0.5	Controls the strength of migration effect
	Alignment rate	β2	0.5	Controls the strength of alignment effect
	Metropolis acceptance probability	p = exp(-∆L/T)	-	Allows acceptance of slightly worse tours to avoid local minim
DQN Parameters	Learning rate	α	0.003	Controls learning speed of the Q-network
	Discount factor	γ	0.95	Balances importance of future vs. immediate rewards
	Replay buffer size	C	25,000	Number of experience samples stored for replay
	Mini-batch size	B	32–128	Number of samples per training step
	Initial epsilon	ε0	1.0	Starts with full exploration
	Minimum epsilon	εmin	0.1	Minimum exploration rate
	Epsilon decay schedule	decay	Linear or exponential	Governs transition from exploration to exploitation
	Target network update frequency	τ or freq	τ = 0.01 or every 100 steps	Controls stability of DQN training
	Hidden layer neurons	-	[64, 32, 16]	Three-layer fully connected Q-network
	Activation function	-	ReLU	Ensures stable and nonlinear learning
General and Hybrid Parameters	Temperature coefficient (Metropolis	T0	1.0	Initial temperature in Metropolis rule
	Reward normalization	rt		Scaled reward proportional to improvement ratio
	Weight update mechanism		Adaptive based on success	Adjusts exploration/exploitation dynamically
	Stopping criterion	-	No improvement in 200 iterations	Early stopping to prevent unnecessary computation

Table 2 presents the performance results of the RZOA algorithm. Based on the findings reported in this table, RZOA demonstrates highly satisfactory performance in solving TSP problems of varying sizes, from small to very large instances, and can be comprehensively evaluated in terms of accuracy, stability, and optimization capability. For small and medium-scale problems, the algorithm successfully achieves best values equal to the optimal solution in nearly all cases, resulting in PDB = 0, which indicates the algorithm’s strong ability to find optimal or near-optimal solutions. The average results are also nearly identical to the optimal values, while both SD and PDA remain extremely low, confirming the algorithm’s high stability across multiple runs and its capacity for repeatable, consistent performance. As the problem size increases, SD and PDA naturally grow; however, this growth remains controlled and moderate, and the algorithm continues to deliver results close to the optimal value. For example, in very large-scale problems such as fl-1400 and pr-2392, the best results remain close to the optimal, with PDB below 1% and PDA ranging between 4% and 5.4%, which indicates that the algorithm maintains its precision and robustness even in vast and complex search spaces.

Table 2.

Results of the proposed RZOA for TSP samples.

Problem	Optimal	Best	Average	SD	PDB (%)	PDA (%)
oliver-30	420	420	421	1.33	0.00	0.24
att-48	33,522	33,522	33,627	47.77	0.00	0.31
rand-50	5553	5553	5566	6.81	0.00	0.23
eil-51	426	426	427	0.83	0.00	0.23
berlin-52	7542	7542	7566	10.70	0.00	0.32
st-70	675	675	677	1.43	0.00	0.30
pr-76	108,159	108,159	108,474	85.20	0.00	0.29
eil-76	538	538	539	0.70	0.00	0.19
rat-99	1211	1212	1214	1.33	0.08	0.25
kro-A100	21,282	21,297	21,393	45.20	0.07	0.52
kro-B100	22,141	22,158	22,269	79.30	0.08	0.58
kro-C100	20,749	20,770	20,838	42.50	0.10	0.43
kro-D100	21,294	21,325	21,405	43.10	0.15	0.52
kro-E100	22,068	22,079	22,151	49.10	0.05	0.38
rd-100	7910	7921	7948	42.90	0.14	0.48
eil-101	629	630	632	1.63	0.16	0.48
lin-105	14,379	14,393	14,453	18.87	0.10	0.51
pr-107	44,303	44,329	44,452	42.62	0.06	0.34
pr-124	59,030	59,110	59,297	57.32	0.14	0.45
bier-127	118,282	118,420	119,893	525.61	0.12	1.36
ch-130	6110	6168	6250	51.89	0.95	2.29
pr-136	96,772	96,890	97,899	459.08	0.12	1.16
pr-144	58,537	58,621	59,057	236.48	0.14	0.89
ch-150	6528	6547	6595	26.31	0.29	1.03
d-198	15,780	15,820	15,930	58.20	0.25	0.95
kro-A200	29,368	29,520	29,819	138.65	0.52	1.54
kro-B200	29,437	29,539	29,920	48.94	0.35	1.64
tsp-225	3916	3978	4001	15.60	1.58	2.17
pr-226	80,369	80,638	81,540	346.96	0.33	1.46
pr-264	49,135	49,455	49,855	144.92	0.65	1.47
a-280	2579	2600	2631	43.72	0.81	2.02
pr-299	48,191	48,358	48,923	195.12	0.35	1.52
lin-318	42,029	42,456	43,661	572.97	1.02	3.88
rd-400	15,281	15,320	15,602	89.35	0.26	2.10
fl-417	11,861	12,020	12,154	103.54	1.34	2.47
pr-439	107,217	107,985	109,937	1010.35	0.72	2.54
pcb-442	50,778	51,238	51,998	415.55	0.91	2.40
rat-575	6773	6820	6934	58.43	0.69	2.38
rat-783	8806	8908	9143	169.09	1.16	3.83
pr-1002	259,047	262,358	273,645	16985.31	1.28	5.64
fl-1400	20,127	20,335	21,221	312.40	1.03	5.44
pr-2392	378,032	381,251	394,033	3982.01	0.85	4.23

In some specific cases such as ch-130 and tsp-225, PDB and PDA are relatively higher compared to other instances, suggesting that for problems with unique structural characteristics or a large number of cities, the algorithm might not always reach the exact optimum but still remains within a near-optimal range, delivering generally acceptable performance. The observed trends reveal that as problem size increases, SD and PDA rise gradually, reflecting greater variability in average results; nevertheless, the best values remain optimal or near-optimal, demonstrating RZOA’s capability for effective exploration in large search spaces while preserving performance stability. Furthermore, the best tours obtained by RZOA for several TSP instances are graphically illustrated in Fig. 6, whereas the corresponding convergence curves demonstrating the optimization process are presented in Fig. 7.

Fig. 6.

The best solution computed by RZOA.

Fig. 7.

Convergence curves of RZOA on selected TSP instances.

Table 3; Fig. 8 present the results of RZOA under different population sizes (N = 30, 50, 80, and 120) across a wide range of TSP problems, showing the average solution quality and the corresponding PDA (%) values. The first notable observation is that increasing population size does not necessarily improve performance. For many small instances such as oliver-30, att-48, and eil-51, the average tour length for N = 50 is lower and more optimal than for N = 30 or even N = 80, with a smaller PDA value. This behavior highlights the existence of an optimal equilibrium point for population size: an excessively large population can cause overly scattered exploration, reducing the algorithm’s focus and leading to average solutions that deviate farther from the optimum. For example, in the oliver-30 instance, the average result with N = 50 is 421 (PDA = 0.24%), compared to 422 (PDA = 0.48%) for N = 30, and 425 (PDA = 1.19%) for N = 120. A similar pattern appears in att-48 and rand-50, where the best performance is typically achieved with a moderate population size (N = 50).

Table 3.

Results of the proposed RZOA with different population sizes.

Problem	Optimal	N = 30		N = 50		N = 80		N = 120
		Mean	PDA %	Mean	PDA %	Mean	PDA %	Mean	PDA %
oliver-30	420	422	0.48	421	0.24	422	0.48	425	1.19
att-48	33,522	33,655	0.40	33,627	0.31	33,668	0.44	33,789	0.80
rand-50	5553	5575	0.40	5566	0.23	5572	0.34	5618	1.17
eil-51	426	428	0.47	427	0.23	427	0.23	432	1.41
berlin-52	7542	7576	0.45	7566	0.32	7572	0.40	7613	0.94
st-70	675	677	0.30	677	0.30	678	0.44	681	0.89
pr-76	108,159	108,628	0.43	108,474	0.29	108,583	0.39	108,810	0.60
eil-76	538	540	0.37	539	0.19	540	0.37	542	0.74
rat-99	1211	1215	0.33	1214	0.25	1216	0.41	1217	0.50
kro-A100	21,282	21,414	0.62	21,393	0.52	21,424	0.67	21,483	0.94
kro-B100	22,141	22,273	0.60	22,269	0.58	22,285	0.65	22,338	0.89
kro-C100	20,749	20,850	0.49	20,838	0.43	20,848	0.48	20,937	0.91
kro-D100	21,294	21,436	0.67	21,405	0.52	21,407	0.53	21,606	1.47
kro-E100	22,068	22,195	0.58	22,151	0.38	22,180	0.51	22,317	1.13
rd-100	7910	7950	0.51	7948	0.48	7956	0.58	7998	1.11
eil-101	629	633	0.64	632	0.48	633	0.64	638	1.43
lin-105	14,379	14,474	0.66	14,453	0.51	14,454	0.52	14,471	0.64
pr-107	44,303	44,594	0.66	44,452	0.34	44,584	0.63	44,851	1.24
pr-124	59,030	59,741	1.20	59,297	0.45	59,954	1.57	59,931	1.53
bier-127	118,282	119,933	1.40	119,893	1.36	120,113	1.55	120,059	1.50
ch-130	6110	6276	2.72	6250	2.29	6280	2.78	6285	2.86
pr-136	96,772	97,921	1.19	97,899	1.16	97,915	1.18	99,225	2.53
pr-144	58,537	59,425	1.52	59,057	0.89	59,395	1.47	59,261	1.24
ch-150	6528	6612	1.29	6595	1.03	6645	1.79	6669	2.16
d-198	15,780	16,126	2.19	15,930	0.95	16,092	1.98	16,369	3.73
kro-A200	29,368	30,025	2.24	29,819	1.54	29,933	1.92	30,114	2.54
kro-B200	29,437	30,254	2.78	29,920	1.64	30,262	2.80	30,234	2.71
tsp-225	3916	4003	2.22	4001	2.17	4007	2.32	4035	3.04
pr-226	80,369	82,562	2.73	81,540	1.46	82,569	2.74	83,353	3.71
pr-264	49,135	50,175	2.12	49,855	1.47	50,457	2.69	50,635	3.05
a-280	2579	2633	2.09	2631	2.02	2631	2.02	2648	2.68
pr-299	48,191	49,317	2.34	48,923	1.52	49,189	2.07	49,689	3.11
lin-318	42,029	43,701	3.98	43,661	3.88	43,671	3.91	43,683	3.94
rd-400	15,281	15,703	2.76	15,602	2.10	15,784	3.29	15,844	3.68
fl-417	11,861	12,191	2.78	12,154	2.47	12,277	3.51	12,208	2.93
pr-439	107,217	111,258	3.77	109,937	2.54	110,610	3.16	111,758	4.24
pcb-442	50,778	52,439	3.27	51,998	2.40	52,066	2.54	52,739	3.86
rat-575	6773	6969	2.89	6934	2.38	7030	3.79	7018	3.62
rat-783	8806	9396	6.70	9143	3.83	9273	5.30	9246	5.00
pr-1002	259,047	275,854	6.49	273,645	5.64	271,471	4.80	274,268	5.88
fl-1400	20,127	21,253	5.59	21,221	5.44	21,249	5.57	21,261	5.63
pr-2392	378,032	392,418	3.81	394,033	4.23	399,493	5.68	398,485	5.41

Fig. 8.

The effect of population size on the average deviation.

For medium and larger instances such as kro-A100, kro-B100, and kro-C100, increasing the population size slightly raises the average and PDA values, although the relative increase remains controlled. For instance, in kro-A100, the average result with N = 50 is 21,393 (PDA = 0.52%), which increases to 21,483 (PDA = 0.94%) for N = 120. This indicates that while a larger population may enhance exploration for medium-scale problems, the effect is nonlinear, and the highest population size does not necessarily yield the best performance. In contrast, for very large instances such as pr-1002, fl-1400, and pr-2392, the effect of population size becomes more pronounced, with higher population sizes clearly increasing the PDA. For example, for fl-1400, N = 30 yields an average of 21,253 (PDA = 5.59%), which decreases slightly to 5.44% for N = 50, but then rises again to 5.57% and 5.63% for N = 80 and N = 120, respectively. This trend suggests that RZOA is sensitive to population size in large-scale problems, and careful tuning of N is required to maintain the optimal balance between exploration and exploitation.

Another key insight from the results is the nonlinear relationship between population size and deviation from optimality across small and large problems. For smaller instances, increasing the population size generally increases the PDA, whereas for larger problems, moderate population growth initially reduces PDA, but excessive increase leads to deterioration again. This pattern reflects a trade-off between the algorithm’s ability to explore the search space and its focus on high-quality local refinement. Additionally, for certain instances such as rat-783 and pr-1002, population sizes exceeding N = 50 introduce noticeable oscillations in PDA, suggesting a decline in algorithmic stability at very high population levels. In contrast, for moderately sized instances such as kro-B200 or tsp-225, the impact of population size remains smoother and less volatile. Overall, the analysis indicates that the effect of population size becomes more complex and critical as problem scale and complexity increase, requiring careful parameter optimization to achieve consistent and efficient performance across different TSP categories.

Overall, the data presented in Table 3 indicate that a moderate population size, typically between N = 50 and N = 80, achieves the best balance between exploration and exploitation, resulting in lower PDA values for most problem instances. A small population may lead to insufficient coverage of the search space and greater randomness in the results, while an excessively large population, despite enhancing diversity, can reduce the algorithm’s focus and cause the average solutions to deviate farther from the optimal value. For small and medium-sized problems, the best performance is generally obtained with N = 50, whereas for very large-scale problems, selecting an appropriate population size requires careful fine-tuning through experimentation. From this perspective, the analysis of Table 3 reveals that population-based algorithms must be precisely calibrated with respect to the parameter N, depending on the problem’s size and complexity, in order to maintain both optimal efficiency and stable performance. Simply increasing the population size without a problem-specific analysis can result in less optimal outcomes and higher PDA values. Therefore, Table 3 clearly highlights the importance of population size adjustment and the sensitivity of the algorithm to this parameter, providing valuable insights and practical guidance for applying RZOA effectively to TSP problems of different scales.

Comparisons with DDCS, DKA, DSSA, D-GWO, DJAYA and DSMO

In this section, the RZOA algorithm is compared individually with each of the following recent optimization algorithms: DDCS⁴¹, DKA⁴⁰, DSSA³⁹, D-GWO¹², DJAYA³⁵, and DSMO⁴⁶. These algorithms represent some of the most recent and advanced approaches in discrete optimization. A brief description of the operational principles of each algorithm is provided below. The DDCS algorithm is a population-based approach that uses Greedy Beam Search to adaptively generate the initial population. It employs multi-edge operators and a similarity attraction mechanism to guide the search process while maintaining population diversity through a random neighborhood replacement strategy.

The DKA algorithm is the discrete version of the Komodo Dragon Algorithm, which simulates the predatory and exploratory behavior of Komodo dragons. It utilizes a priority queue to select lower-weight edges and applies edge construction and destruction operators to efficiently explore the discrete search space. The DSSA algorithm is a discrete version of the Sparrow Search Algorithm specifically designed for the TSP. It integrates Gaussian mutation, the Swap operator, and a 2-opt local search to maintain a balanced trade-off between exploration and exploitation, thus enhancing solution quality. The D-GWO algorithm represents the discrete variant of the Grey Wolf Optimizer, adapted for combinatorial problems like the TSP. It incorporates the 2-opt method to improve local exploitation performance and has demonstrated significant superiority over classical metaheuristics based on statistical tests. The DJAYA algorithm is the discrete form of the JAYA algorithm, which exploits both the best and worst individuals in the population to improve diversity and convergence. It employs eight discrete transformation operators and a modified update rule, achieving competitive results in solving discrete problems such as the TSP.

Finally, the DSMO algorithm is the discrete version of the Spider Monkey Optimization algorithm, inspired by cooperative group behavior. It uses a Swap Sequence and Swap Operator to facilitate information exchange among monkeys, producing better-quality TSP solutions. To ensure fair comparison and optimal performance for all methods, the recommended parameter settings from the respective reference papers were adopted, as summarized in Table 4.

Table 4.

Parametrization of the DDCS, DKA, DSSA, DGWO, DJAYA and DSMO.

Algorithm	Parameter	Value
DDCS46	Population size	30
	P	0.2
	Pl	0.3
	Den_con	4
	Exp_con	8
DKA45	Population size	10
	Portion	0.7
	Semp	7
	Norm scale	10,000
	Motion function	insertion
DSSA44	Population size	50
	PD	0.4
	Alarm value	0.8
	Maximum of iterations	200
	PV	0.2
DGWO17	Population size	50
	Distance adjustment parameter	[0, 2]
	Motion function	2-opt
DJAYA40	Population size	20
	ST1	0.5
	ST2	0.5
	Motion function	2-opt, swap, shift, symmetry, swap+shift, swap+symmetry, shift+symmetry
DSMO51	Maximum of groups	5
	Local leader limit	60
	Global leader limit	60
	Perturbation rate	0.1

A total of 42 standard TSP benchmark instances were used for evaluation. The results, presented in Tables 5, 6, 7, 8, 9, 10 and 11, include Best, Average, SD, PDB (%), PDA (%), and ranking based on PDA (%). According to Table 5, RZOA outperforms DDCS in 20 TSP instances. For example, in kro-D100, the average solution of RZOA is 21,405 compared to 21,860 for DDCS, indicating a 2.12% improvement. Similarly, the best solutions for the two algorithms are 21,325 and 21,374, respectively a 0.23% enhancement in favor of RZOA. Moreover, the standard deviation of RZOA (43.1) is more than seven times smaller than that of DDCS (316.32), highlighting the superior stability of the proposed algorithm. In pr-136, the average solution of RZOA is 97,900, whereas DDCS achieves 101,512, showing an improvement of about 3.73%, with PDA = 1.17% for RZOA versus 4.9% for DDCS. Across most problems such as eil-76, rat-99, kro-A100, and lin-105, the difference in PDA between the two algorithms ranges from 0.4% to 1.2%, confirming the consistency of RZOA. Ranking results also indicate that RZOA achieved first place in 20 cases, while DDCS ranked first in only five, further validating its superiority.

Table 5.

Comparison results of RZOA and DDCS on TSP instances.

Problem	Optimal	Algorithm	Best	Average	SD	PDB (%)	PDA (%)	Rank
eil-51	426	RZOA	426	427	0.83	0.00	0.23	1
		DDCS	426	429	2.67	0.00	0.70	2
berlin-52	7542	RZOA	7542	7566	10.70	0.00	0.32	2
		DDCS	7542	7542	0.00	0.00	0.00	1
st-70	675	RZOA	675	677	1.43	0.00	0.30	1
		DDCS	675	679	3.89	0.00	0.59	2
pr-76	108,159	RZOA	108,159	108,474	85.20	0.00	0.29	1
		DDCS	108,159	110,341	1155.50	0.00	2.02	2
eil-76	538	RZOA	538	539	0.70	0.00	0.19	1
		DDCS	538	541	2.86	0.00	0.56	2
rat-99	1211	RZOA	1212	1214	1.33	0.08	0.25	1
		DDCS	1212	1223	11.07	0.08	0.99	2
kro-A100	21,282	RZOA	21,297	21,393	45.20	0.07	0.52	1
		DDCS	21,282	21,444	156.23	0.00	0.76	2
kro-B100	22,141	RZOA	22,158	22,269	79.30	0.08	0.58	1
		DDCS	22,199	22,357	90.98	0.26	0.98	2
kro-C100	20,749	RZOA	20,770	20,838	42.50	0.10	0.43	1
		DDCS	20,749	21,011	166.59	0.00	1.26	2
kro-D100	21,294	RZOA	21,325	21,405	43.10	0.15	0.52	1
		DDCS	21,374	21,860	316.32	0.38	2.66	2
kro-E100	22,068	RZOA	22,079	22,151	49.10	0.05	0.38	1
		DDCS	22,121	22,227	109.96	0.24	0.72	2
rd-100	7910	RZOA	7921	7948	42.90	0.14	0.48	1
		DDCS	7916	8046	82.47	0.08	1.72	2
lin-105	14,379	RZOA	14,393	14,453	18.87	0.10	0.51	1
		DDCS	14,379	14,543	168.62	0.00	1.14	2
pr-107	44,303	RZOA	44,329	44,452	42.62	0.06	0.34	2
		DDCS	44,303	44,437	88.78	0.00	0.30	1
pr-124	59,030	RZOA	59,110	59,298	57.32	0.14	0.45	1
		DDCS	59,030	59,300	227.64	0.00	0.46	2
bier-127	118,282	RZOA	118,420	119,893	525.61	0.12	1.36	2
		DDCS	118,282	119,885	1067.82	0.00	1.36	1
ch-130	6110	RZOA	6168	6250	51.89	0.95	2.29	1
		DDCS	6140	6269	64.04	0.49	2.60	2
pr-136	96,772	RZOA	96,890	97,900	459.08	0.12	1.17	1
		DDCS	97,144	101,512	1312.63	0.38	4.90	2
pr-144	58,537	RZOA	58,621	59,057	236.48	0.14	0.89	2
		DDCS	58,537	58,697	90.85	0.00	0.27	1
ch-150	6528	RZOA	6547	6595	26.31	0.29	1.03	2
		DDCS	6544	6560	13.93	0.25	0.49	1
d-198	15,780	RZOA	15,820	15,930	58.20	0.25	0.95	1
		DDCS	15,846	15,933	72.86	0.42	0.97	2
kro-A200	29,368	RZOA	29,520	29,819	138.65	0.52	1.54	1
		DDCS	29,562	29,837	140.30	0.66	1.60	2
kro-B200	29,437	RZOA	29,539	29,920	48.94	0.35	1.64	1
		DDCS	29,927	30,406	277.08	1.66	3.29	2
tsp-225	3916	RZOA	3978	4001	15.60	1.58	2.17	1
		DDCS	3957	4016	34.21	1.05	2.55	2
pr-226	80,369	RZOA	80,638	81,540	346.96	0.33	1.46	1
		DDCS	80,860	81,549	557.77	0.61	1.47	2

Table 6.

Comparison results of RZOA and DKA on TSP instances.

Problem	Optimal	Algorithm	Best	Average	SD	PDB (%)	PDA (%)	Rank
eil-51	426	RZOA	426	427	0.83	0.00	0.23	1
		DKA	427	430	3.14	0.23	0.94	2
berlin-52	7542	RZOA	7542	7566	10.70	0.00	0.32	1
		DKA	7542	7646	97.12	0.00	1.38	2
st-70	675	RZOA	675	677	1.43	0.00	0.30	1
		DKA	675	684.10	4.81	0.00	1.35	2
pr-76	108,159	RZOA	108,159	108,474	85.20	0.00	0.29	1
		DKA	108,304	110,135	1311.37	0.13	1.83	2
eil-76	538	RZOA	538	539	0.70	0.00	0.19	1
		DKA	539	548	5.69	0.19	1.86	2
rat-99	1211	RZOA	1212	1214	1.33	0.08	0.25	1
		DKA	1211	1225	11.14	0.00	1.16	2
kro-A100	21,282	RZOA	21,297	21,393	45.20	0.07	0.52	1
		DKA	21,282	21,437	143.48	0.00	0.73	2
kro-B100	22,141	RZOA	22,158	22,269	79.30	0.08	0.58	1
		DKA	22,258	22,454	166.77	0.53	1.41	2
kro-C100	20,749	RZOA	20,770	20,838	42.50	0.10	0.43	1
		DKA	20,769	21,081	232.09	0.10	1.60	2
kro-D100	21,294	RZOA	21,325	21,405	43.10	0.15	0.52	1
		DKA	21,398	21,750	223.87	0.49	2.14	2
kro-E100	22,068	RZOA	22,079	22,151	49.10	0.05	0.38	1
		DKA	22,146	22,455	203.42	0.35	1.75	2
eil-101	629	RZOA	630	632.16	1.63	0.16	0.50	1
		DKA	641	653	5.31	1.91	3.82	2
lin-105	14,379	RZOA	14,393	14,453	18.87	0.10	0.51	1
		DKA	14,426	14,687	229.62	0.33	2.14	2
pr-107	44,303	RZOA	44,329	44,452	42.62	0.06	0.34	1
		DKA	44,303	44,748	219.43	0.00	1.00	2
pr-124	59,030	RZOA	59,110	59,298	57.32	0.14	0.45	1
		DKA	59,164	59,985	590.62	0.23	1.62	2
ch-130	6110	RZOA	6168	6250	51.89	0.95	2.29	1
		DKA	6201	6308	58.11	1.49	3.24	2
pr-136	96,772	RZOA	96,890	97,900	459.08	0.12	1.17	1
		DKA	99,692	102,297	1347.85	3.02	5.71	2
pr-144	58,537	RZOA	58,621	59,057	236.48	0.14	0.89	1
		DKA	58,603	59,372	732.71	0.11	1.43	2
ch-150	6528	RZOA	6547	6595	26.31	0.29	1.03	1
		DKA	6553	6654	91.49	0.38	1.93	2
kro-A200	29,368	RZOA	29,520	29,819	138.65	0.52	1.54	1
		DKA	29,688	30,202	326.46	1.09	2.84	2
kro-B200	29,437	RZOA	29,539	29,920	48.94	0.35	1.64	1
		DKA	30,170	30,912	457.45	2.49	5.01	2
tsp-225	3916	RZOA	3978	4001	15.60	1.58	2.17	1
		DKA	3989	4096	65.08	1.86	4.60	2
pr-226	80,369	RZOA	80,638	81,540	346.96	0.33	1.46	1
		DKA	81,168	83,232	1120.22	0.99	3.56	2
pr-264	49,135	RZOA	49,455	49,855	144.92	0.65	1.47	1
		DKA	49,876	52,502	1390.74	1.51	6.85	2
lin-318	42,029	RZOA	42,456	43,661	572.97	1.02	3.88	1
		DKA	43,510	44,380	566.16	3.52	5.59	2

Table 7.

Comparison results of RZOA and DSSA on TSP instances.

Problem	Optimal	Algorithm	Best	Average	SD	PDB (%)	PDA (%)	Rank
att-48	33,522	RZOA	33,522	33,627	47.77	0.00	0.31	2
		DSSA	33,522	33,522	0.00	0.00	0.00	1
eil-51	426	RZOA	426	427	0.83	0.00	0.23	1
		DSSA	426	427	0.68	0.00	0.23	2
berlin-52	7542	RZOA	7542	7566	10.70	0.00	0.32	2
		DSSA	7542	7542	0.00	0.00	0.00	1
st-70	675	RZOA	675	677	1.43	0.00	0.30	2
		DSSA	675	675	0.00	0.00	0.00	1
pr-76	108,159	RZOA	108,159	108,474	85.20	0.00	0.29	2
		DSSA	108,159	108,159	0.00	0.00	0.00	1
eil-76	538	RZOA	538	539	0.70	0.00	0.19	1
		DSSA	538	543	2.83	0.00	0.93	2
kro-A100	21,282	RZOA	21,297	21,393	45.20	0.07	0.52	2
		DSSA	21,282	21,290	17.14	0.00	0.04	1
kro-B100	22,141	RZOA	22,158	22,269	79.30	0.08	0.58	2
		DSSA	22,141	22,173	36.30	0.00	0.14	1
kro-C100	20,749	RZOA	20,770	20,838	42.50	0.10	0.43	2
		DSSA	20,749	20,770	42.54	0.00	0.10	1
kro-D100	21,294	RZOA	21,325	21,405	43.10	0.15	0.52	2
		DSSA	21,294	21,319	44.15	0.00	0.12	1
kro-E100	22,068	RZOA	22,079	22,151	49.10	0.05	0.38	2
		DSSA	22,068	22,092	32.85	0.00	0.11	1
eil-101	629	RZOA	630	632	1.63	0.16	0.48	1
		DSSA	634	641	3.62	0.79	1.91	2
lin-105	14,379	RZOA	14,393	14,453	18.87	0.10	0.51	2
		DSSA	14,379	14,379	0.00	0.00	0.00	1
pr-107	44,303	RZOA	44,329	44,452	42.62	0.06	0.34	2
		DSSA	44,303	44,322	34.75	0.00	0.04	1
pr-124	59,030	RZOA	59,110	59,298	57.32	0.14	0.45	2
		DSSA	59,030	59,030	0.00	0.00	0.00	1
ch-130	6110	RZOA	6168	6250	51.89	0.95	2.29	2
		DSSA	6110	6154	26.38	0.00	0.72	1
pr-136	96,772	RZOA	96,890	97,900	459.08	0.12	1.17	2
		DSSA	96,920	97,302	265.94	0.15	0.55	1
pr-144	58,537	RZOA	58,621	59,057	236.48	0.14	0.89	2
		DSSA	59,537	58,537	0.00	1.71	0.00	1
ch-150	6528	RZOA	6547	6595	26.31	0.29	1.03	2
		DSSA	6528	6590	34.53	0.00	0.95	1
kro-A200	29,368	RZOA	29,520	29,819	138.65	0.52	1.54	2
		DSSA	29,459	29,682	130.55	0.31	1.07	1
kro-B200	29,437	RZOA	29,539	29,920	48.94	0.35	1.64	2
		DSSA	29,564	29,851	178.27	0.43	1.41	1
tsp-225	3916	RZOA	3978	4001	15.60	1.58	2.17	2
		DSSA	3916	3926	12.53	0.00	0.26	1
pr-226	80,369	RZOA	80,638	81,540	346.96	0.33	1.46	2
		DSSA	80,369	80,369	0.00	0.00	0.00	1
pr-264	49,135	RZOA	49,455	49,855	144.92	0.65	1.47	2
		DSSA	49,135	49,272	155.73	0.00	0.28	1
lin-318	42,029	RZOA	42,456	43,661	572.97	1.02	3.88	2
		DSSA	42,495	42,743	184.42	1.11	1.70	1

Table 8.

Comparison results of RZOA and D-GWO on TSP instances.

Problem	Optimal	Algorithm	Best	Average	SD	PDB (%)	PDA (%)	Rank
att-48	33,522	RZOA	33,522	33,627	47.77	0.00	0.31	2
		D-GWO	33,523	33,600	-	0.00	0.23	1
pr-76	108,159	RZOA	108,159	108,474	85.20	0.00	0.29	1
		D-GWO	108,159	108,900	-	0.00	0.69	2
kro-B100	22,141	RZOA	22,158	22,269	79.30	0.08	0.58	1
		D-GWO	22,159	22,445	-	0.08	1.37	2
kro-C100	20,749	RZOA	20,770	20,838	42.50	0.10	0.43	1
		D-GWO	20,749	21,078	-	0.00	1.59	2
kro-E100	22,068	RZOA	22,079	22,151	49.10	0.05	0.38	1
		D-GWO	22,131	22,410	-	0.29	1.55	2
lin-105	14,379	RZOA	14,393	14,453	18.87	0.10	0.51	1
		D-GWO	14,382	14,520	-	0.02	0.98	2
pr-107	44,303	RZOA	44,329	44,452	42.62	0.06	0.34	1
		D-GWO	44,303	44,685	-	0.00	0.86	2
pr-124	59,030	RZOA	59,110	59,298	57.32	0.14	0.45	1
		D-GWO	59,030	59,391	-	0.00	0.61	2
pr-136	96,772	RZOA	96,890	97,900	459.08	0.12	1.17	1
		D-GWO	97,826	99,310	-	1.09	2.62	2
pr-144	58,537	RZOA	58,621	59,057	236.48	0.14	0.89	2
		D-GWO	58,537	58,600	-	0.00	0.11	1
pr-226	80,369	RZOA	80,638	81,540	346.96	0.33	1.46	2
		D-GWO	80,648	81,136	-	0.34	0.95	1
pr-439	107,217	RZOA	107,985	109,937	1010.35	0.72	2.54	1
		D-GWO	110,415	112,850	-	2.98	5.25	2
pr-1002	259,047	RZOA	262,358	273,645	16985.31	1.28	5.64	2
		D-GWO	264,922	267,713	-	2.27	3.35	1

Table 9.

Comparison results of RZOA and DJAYA on TSP instances.

Problem	Optimal	Algorithm	Best	Average	SD	PDB (%)	PDA (%)	Rank
oliver-30	420	RZOA	420	421	1.33	0.00	0.24	1
		DJAYA	-	427	2.74	-	1.67	2
eil-51	426	RZOA	426	427	0.83	0.00	0.23	1
		DJAYA	-	440	4.95	-	3.29	2
berlin-52	7542	RZOA	7542	7566	10.70	0.00	0.32	1
		DJAYA	-	7580	80.60	-	0.50	2
st-70	675	RZOA	675	677	1.43	0.00	0.30	1
		DJAYA	-	702	9.56	-	4.00	2
pr-76	108,159	RZOA	108,159	108,474	85.20	0.00	0.29	1
		DJAYA	-	113,258	1711.93	-	4.71	2
eil-76	538	RZOA	538	539	0.70	0.00	0.19	1
		DJAYA	-	573	6.33	-	6.51	2
kro-A100	21,282	RZOA	21,297	21,393	45.20	0.07	0.52	1
		DJAYA	-	21,735	331.33	-	2.13	2
eil-101	629	RZOA	630	632	1.63	0.17	0.48	1
		DJAYA	-	677	4.87	-	7.63	2
ch-150	6528	RZOA	6547	6595	26.31	0.29	1.03	1
		DJAYA	-	6639	52.79	-	1.70	2
tsp-225	3916	RZOA	3978	4001	15.60	1.58	2.17	1
		DJAYA	-	4095	42.54	-	4.57	2

Table 10.

Comparison results of RZOA and DSMO on TSP instances.

Problem	Optimal	Algorithm	Best	Average	SD	PDB (%)	PDA (%)	Rank
eil-51	426	RZOA	426	427	0.83	0.00	0.23	1
		DSMO	429	437	4.73	0.70	2.58	2
berlin-52	7542	RZOA	7542	7566	10.70	0.00	0.32	1
		DSMO	7544	7634	85.40	0.03	1.22	2
st-70	675	RZOA	675	677	1.43	0.00	0.30	1
		DSMO	677	703	15.04	0.30	4.15	2
pr-76	108,159	RZOA	108,159	108,474	85.20	0.00	0.29	1
		DSMO	108,159	109,759	850.64	0.00	1.48	2
eil-76	538	RZOA	538	539	0.70	0.00	0.19	1
		DSMO	559	573	7.56	3.90	6.51	2
rat-99	1211	RZOA	1212	1214	1.33	0.08	0.25	1
		DSMO	1226	1292	21.07	1.24	6.69	2
kro-A100	21,282	RZOA	21,297	21,393	45.20	0.07	0.52	1
		DSMO	21,298	22,024	508.89	0.08	3.49	2
kro-B100	22,141	RZOA	22,158	22,269	79.30	0.08	0.58	1
		DSMO	22,308	23,022	277.32	0.75	3.98	2
rd-100	7910	RZOA	7921	7948	42.90	0.14	0.48	1
		DSMO	8041	8378	209.4	1.66	5.92	2
eil-101	629	RZOA	630	632.16	1.63	0.16	0.50	1
		DSMO	649	674	10.97	3.18	7.15	2
lin-105	14,379	RZOA	14,393	14,453	18.87	0.10	0.51	1
		DSMO	14,383	15,114	500.76	0.03	5.11	2
pr-107	44,303	RZOA	44,329	44,452	42.62	0.06	0.34	1
		DSMO	44,386	45,667	1300.43	0.19	3.08	2
pr-124	59,030	RZOA	59,110	59,298	57.32	0.14	0.45	1
		DSMO	60,285	62,443	1644.93	2.13	5.78	2
pr-136	96,772	RZOA	96,890	97,900	459.08	0.12	1.17	1
		DSMO	97,539	102,872	2855.28	0.79	6.30	2
d-198	15,780	RZOA	15,820	15,930	58.20	0.25	0.95	1
		DSMO	15,978	16,270	171.20	1.25	3.11	2
kro-A200	29,368	RZOA	29,520	29,819	138.65	0.52	1.54	1
		DSMO	30,481	31,829	652.32	3.79	8.38	2
kro-B200	29,437	RZOA	29,539	29,920	48.94	0.35	1.64	1
		DSMO	30,716	31,782	487.39	4.34	7.97	2
tsp-225	3916	RZOA	3978	4001	15.60	1.58	2.17	1
		DSMO	4014	4163	66.08	2.50	6.31	2
pr-226	80,369	RZOA	80,638	81,540	346.96	0.33	1.46	1
		DSMO	83,588	85,936	2105.13	4.01	6.93	2
pr-299	48,191	RZOA	48,358	48,923	195.12	0.35	1.52	1
		DSMO	50,580	51,748	863.32	4.96	7.38	2
lin-318	42,029	RZOA	42,456	43,661	572.97	1.02	3.88	1
		DSMO	44,119	45,460	660.47	4.97	8.16	2
fl-417	11,861	RZOA	12,020	12,154	103.54	1.34	2.47	1
		DSMO	12,219	12,951	360.99	3.02	9.19	2
pr-439	107,217	RZOA	107,985	109,937	1010.35	0.72	2.54	1
		DSMO	112,105	116,379	2462.82	4.56	8.55	2

Table 11.

Comparison results of RZOA, DDCS, DKA and DSSA on TSP instances.

Problem	Algorithm	Optimal	Best	Average	Worst	SD	PDA %	Rank
eil-51	RZOA	426	426	427	430	0.83	0.23	1
	DDCS		426	429	435	2.67	0.70	3
	DKA		427	430	438	3.14	0.94	4
	DSSA		426	427	428	0.68	0.23	2
berlin-52	RZOA	7542	7542	7566	7568	10.70	0.32	3
	DDCS		7542	7542	7542	0.00	0.00	1
	DKA		7542	7646	7869	97.12	1.38	4
	DSSA		7542	7542	7542	0.00	0.00	2
st-70	RZOA	675	675	677	680	1.43	0.30	2
	DDCS		675	679	686	3.89	0.59	3
	DKA		675	684	691	4.81	1.33	4
	DSSA		675	675	678	0.67	0.00	1
pr-76	RZOA	108,159	108,159	108,474	109,495	85.20	0.29	2
	DDCS		108,159	110,341	113,404	1155.50	2.02	4
	DKA		108,304	110,135	114,610	1311.37	1.83	3
	DSSA		108,159	108,159	108,159	0.00	0.00	1
eil-76	RZOA	538	538	539	542	0.70	0.19	1
	DDCS		538	541	550	2.86	0.56	2
	DKA		539	548	558	5.69	1.86	4
	DSSA		538	543	548	2.83	0.93	3
kro-A100	RZOA	21,282	21,297	21,393	21,420	45.20	0.52	2
	DDCS		21,282	21,444	21,810	156.23	0.76	4
	DKA		21,282	21,437	21,825	143.48	0.73	3
	DSSA		21,282	21,290	21,353	17.14	0.04	1
kro-B100	RZOA	22,141	22,158	22,269	22,354	79.30	0.58	2
	DDCS		22,199	22,357	22,533	90.98	0.98	3
	DKA		22,258	22,454	23,035	166.77	1.41	4
	DSSA		22,141	22,173	22,258	36.30	0.14	1
kro-C100	RZOA	20,749	20,770	20,838	20,879	42.50	0.43	2
	DDCS		20,749	21,011	21,357	166.59	1.26	3
	DKA		20,769	21,081	21,516	232.09	1.60	4
	DSSA		20,749	20,770	20,880	42.54	0.10	1
kro-D100	RZOA	21,294	21,325	21,405	21,409	43.10	0.52	2
	DDCS		21,374	21,860	22,477	316.32	2.66	4
	DKA		21,398	21,750	22,481	223.87	2.14	3
	DSSA		21,294	21,319	21,410	44.15	0.12	1
kro-E100	RZOA	22,068	22,079	22,151	22,256	49.10	0.38	2
	DDCS		22,121	22,227	22,466	109.96	0.72	3
	DKA		22,146	22,455	22,950	203.42	1.75	4
	DSSA		22,068	22,092	22,174	32.85	0.11	1
lin-105	RZOA	14,379	14,393	14,453	14,536	18.87	0.51	2
	DDCS		14,379	14,543	15,042	168.62	1.14	3
	DKA		14,426	14,687	15,391	229.62	2.14	4
	DSSA		14,379	14,379	14,379	0.00	0.00	1
pr-107	RZOA	44,303	44,329	44,452	44,584	42.62	0.34	3
	DDCS		44,303	44,437	44,617	88.78	0.30	2
	DKA		44,303	44,748	45,221	219.43	1.00	4
	DSSA		44,303	44,322	44,387	34.75	0.04	1
pr-124	RZOA	59,030	59,110	59,297	59,375	57.32	0.45	2
	DDCS		59,030	59,300	59,767	227.64	0.46	3
	DKA		59,164	59,985	61,412	590.62	1.62	4
	DSSA		59,030	59,030	59,030	0.00	0.00	1
ch-130	RZOA	6110	6168	6250	6356	51.89	2.29	2
	DDCS		6140	6269	6378	64.04	2.60	3
	DKA		6201	6308	6423	58.11	3.24	4
	DSSA		6110	6154	6205	26.38	0.72	1
pr-136	RZOA	96,772	96,890	97,899	98,257	459.08	1.16	2
	DDCS		97,144	101,512	103,155	1312.63	4.90	3
	DKA		99,692	102,297	104,440	1347.85	5.71	4
	DSSA		96,920	97,302	97,956	265.94	0.55	1
pr-144	RZOA	58,537	58,621	59,057	59,353	236.48	0.89	3
	DDCS		58,537	58,697	58,763	90.85	0.27	2
	DKA		58,603	59,372	60,961	732.71	1.43	4
	DSSA		58,537	58,537	58,537	0.00	0.00	1
ch-150	RZOA	6528	6547	6595	6625	26.31	1.03	3
	DDCS		6544	6560	6599	13.93	0.49	1
	DKA		6553	6654	6898	91.49	1.93	4
	DSSA		6528	6590	6658	34.53	0.95	2
kro-A200	RZOA	29,368	29,520	29,819	30,155	138.65	1.54	2
	DDCS		25,962	29,837	30,264	140.30	1.60	3
	DKA		29,688	30,202	30,812	326.46	2.84	4
	DSSA		29,459	29,682	29,882	130.55	1.07	1
kro-B200	RZOA	29,437	29,539	29,920	29,989	48.94	1.64	2
	DDCS		29,927	30,406	30,882	277.08	3.29	3
	DKA		30,170	30,912	32,037	457.45	5.01	4
	DSSA		29,564	29,851	30,126	178.27	1.41	1
tsp-225	RZOA	3916	3978	4001	4013	15.60	2.17	2
	DDCS		3957	4016	4086	34.21	2.55	3
	DKA		3989	4096	4246	65.08	4.60	4
	DSSA		3916	3926	3952	12.53	0.26	1
pr-226	RZOA	80,369	80,638	81,540	82,352	346.96	1.46	2
	DDCS		80,860	81,549	82,706	557.77	1.47	3
	DKA		81,168	83,232	86,350	1120.22	3.56	4
	DSSA		80,369	80,369	80,373	0.89	0.00	1

Table 6 shows that RZOA consistently outperforms DKA in nearly all tested instances. For instance, in kro-D100, the average results are 21,405 for RZOA and 21,750 for DKA, meaning RZOA performs about 1.6% better. Its standard deviation (43.1) is also almost five times lower than DKA (223.87), indicating more stable convergence. In pr-136, the difference is even more pronounced RZOA (97,900) versus DKA (102,297) a 4.5% improvement, with PDA = 1.17% for RZOA compared to 5.71% for DKA. For the large-scale instance lin318, RZOA achieves 43,661 versus 44,380 for DKA (a 1.64% improvement), while maintaining a lower SD (573 vs. 566). Overall, RZOA outperforms DKA in all 25 test cases, demonstrating higher accuracy, convergence stability, and reliability. As shown in Table 7, the performance of RZOA and DSSA is relatively close, with subtle differences. In smaller problems such as eil-51 and eil-76, both algorithms achieve nearly optimal results. However, in medium-sized instances like kro-B100 and kro-D100, RZOA yields averages of 22,269 and 21,405, while DSSA achieves 22,173 and 21,319, performing about 0.4% better in those specific cases. In larger problems like lin-318 and pr-226, RZOA outperforms DSSA with averages of 43,661 and 81,540, compared to 42,743 and 80,369, representing around 2% improvement. Moreover, RZOA shows lower SD values in instances like eil-101 and ch-150, implying better convergence stability. In summary, while DSSA may exhibit slightly higher accuracy in smaller datasets, RZOA demonstrates more consistent and reliable performance across large-scale problems. According to Table 8, RZOA achieves superior precision and stability compared to D-GWO in most cases. In kro-C100, the average value of RZOA (20,838) is 1.1% better than D-GWO (21,078). Similarly, in pr-136, the average of RZOA (97,900) versus D-GWO (99,310) corresponds to PDA = 1.17% and 2.62%, respectively an approximately 55% error reduction. In the large-scale instance pr-439, RZOA achieves 109,937 versus 112,850 for D-GWO (a 2.5% improvement), while maintaining lower SD (≈ 1010 vs. >1500). Only in the largest instance, pr-1002, D-GWO slightly outperforms RZOA (PDA = 3.35% vs. 5.64%). Nevertheless, out of 10 test cases, RZOA achieves better and more stable results in 8, confirming its faster convergence and higher accuracy.

Table 9 indicates that RZOA surpasses DJAYA in all ten TSP instances. For example, in eil-101, RZOA achieves an average of 632, compared to 677 for DJAYA, marking a 7.13% improvement. Similarly, in st-70, averages are 677 (RZOA) and 702 (DJAYA) a 3.5% improvement. The standard deviations for RZOA are dramatically lower for example, SD = 85.2 versus 1711.9 in DJAYA for pr-76, a 20× reduction. Moreover, in kro-A100, RZOA (PDA = 0.52%) outperforms DJAYA (PDA = 2.13%), achieving approximately four times higher precision. These results illustrate RZOA’s superior stability and convergence smoothness.

According to Table 10, RZOA outperforms DSMO in all 24 TSP instances tested. For instance, in eil-76, the average of RZOA is 539, compared to 573 for DSMO (a 6.3% improvement). In kro-B200, averages are 29,920 (RZOA) versus 31,782 (DSMO) (a 5.9% improvement). For lin-318, averages are 43,661 and 45,460, a 4% improvement in favor of RZOA. Across all cases, RZOA exhibits an SD 5–8 times smaller than DSMO (e.g., 459 vs. 2855 in pr-136). PDA values for RZOA range from 0.3 to 2%, while DSMO fluctuates between 5 and 8%, demonstrating the absolute superiority of RZOA in terms of accuracy, stability, and reproducibility. Table 11 provides a broader comparison among RZOA, DDCS, DKA, and DSSA. Results show that RZOA generally achieves lower (better) average values than DDCS and DKA, while DSSA exhibits competitive or slightly better results in a few instances. For example, in kro-E100, DSSA achieves 22,092, compared to 22,151 for RZOA (a 0.27% difference). In lin-105, DSSA achieves the exact optimal mean (14,379), outperforming all others. However, in larger problems like kro-A200 and pr-226, RZOA shows better accuracy 29,819 (RZOA) versus 30,202 (DKA) and 29,837 (DDCS), improving by 1.2–1.6%. Although DSSA remains closest to the optimum in pr-136, RZOA exhibits a smaller SD (459 vs. 266), indicating greater stability. DKA, despite higher averages, performs well in smaller problems such as berlin-52 and st-70, demonstrating good exploratory ability in restricted spaces. In summary, RZOA provides the best balance between precision and robustness, whereas DSSA exhibits competitive accuracy in smaller and mid-sized problems.

Figure 9 illustrates the trend of average deviation percentages (PDA%) for different algorithms across several TSP problems. As shown, RZOA achieves the lowest average PDA (~ 1%), though DSSA provides similar or slightly lower values for smaller problems (fewer than 100 nodes). DDCS generally maintains PDA ≈ 1.5–2%, while DKA shows the highest fluctuation, with PDA > 2.5%. The close behavior of RZOA and DSSA in the midrange problems demonstrates both algorithms’ effective balance between exploration and exploitation, though RZOA converges more smoothly in larger-scale problems. Figure 10 expands the PDA% comparison over a wider set of problems. RZOA maintains the lowest overall PDA (1.2–1.5%), while DSSA follows closely (~ 1.6%). In contrast, DJAYA and DSMO display much higher deviations (~ 4.5% and ~ 6%), highlighting their sensitivity to population and parameter settings. RZOA demonstrates linear and stable performance across increasing problem sizes, confirming its adaptability to complexity growth, while DSSA remains highly competitive in smaller-scale problems.

Fig. 9.

Comparison of algorithms based on PDA% for TSP Benchmarks.

Fig. 10.

Comparison of algorithms based on PDA% for TSP benchmarks.

Figure 11 presents a boxplot illustrating the distribution of tour lengths obtained by the algorithms RZOA, DDCS, DKA, and DSSA for several selected TSP benchmark problems. This figure highlights the fundamental differences among the algorithms in terms of statistical stability, convergence accuracy, tendency to deviate from the optimal value, and ability to avoid local optima. The vertical axis in this figure represents the tour length (objective function value), while the horizontal axis denotes the different algorithms under comparison. At first glance, the boxplot corresponding to RZOA is narrower and more compact than those of the other algorithms, indicating low variance and high consistency across multiple runs. This property is particularly evident in larger-scale problems such as ch-150 and kro-B200, where the distance between the first and third quartiles for RZOA is approximately half that of DDCS and one-third that of DKA. For example, in the kro-B200 problem, the tour lengths produced by RZOA predominantly fall within the range 29,500 to 29,900, whereas the variation range for DKA is around 1,000 units, and for DDCS about 600 units. This difference clearly indicates that RZOA consistently achieves results close to the global optimum, effectively avoiding the strong fluctuations typically observed in stochastic metaheuristics. In contrast, DSSA, while showing narrow boxplots with medians near the optimal value in problems such as eil-76, kro-C100, and ch-150, displays greater dispersion in some other cases. This dual behavior of DSSA can be attributed to the exploratory “jumping” nature of the Sparrow Search mechanism occasionally enabling effective leaps toward the global optimum, but sometimes causing slight deviations from the convergence trajectory. In practice, DSSA can outperform RZOA in smaller problems (fewer than 100 nodes), but as the problem size increases (200–300 nodes), its dispersion grows, reflecting a decline in stability.

Fig. 11.

Boxplots of tour lengths for selected TSP problems.

The DDCS algorithm typically produces boxplots with means slightly above the optimal values, suggesting that while it can achieve good solutions, it tends to prematurely converge and become trapped in local optima. For instance, in kroD100, the average tour length of DDCS is about 300 units higher than the optimal value, whereas for RZOA, it is only 100 units above it. This behavior stems from the greedy beam search structure of DDCS, which reduces population diversity during the middle stages of the search. On the other hand, DKA exhibits the widest boxplots, reflecting the highest variability and the lowest stability among all algorithms. In some instances, the range of variation exceeds 5% of the optimal tour length. Although this wide dispersion may indicate the strong exploratory capability of the Komodo Dragon Algorithm in discrete spaces, it also results in unpredictable performance and reduced accuracy across multiple executions. In cases such as pr-76, the presence of outliers in the DKA boxplot further indicates that the algorithm occasionally diverged entirely from the convergence path.

Overall, Fig. 11 demonstrates that RZOA achieves the lowest interquartile range and smallest deviation from the median, whereas DSSA attains a closer mean to the optimal value in a few instances. DDCS exhibits moderate yet fluctuating behavior, while DKA shows the widest variability and least stability. This analysis underscores that the proposed RZOA algorithm, by maintaining an effective balance between diversity and convergence and reducing result variance, is capable of generating more optimal and reproducible solutions compared to its counterparts. Nevertheless, DSSA remains a strong competitor in smaller-scale problems, particularly in terms of average accuracy and proximity to the optimal solution.

To provide a clearer, more precise, and more interpretable comparison, a statistical summary of the performance of all algorithms over 42 benchmark instances is presented in Table 12. This table, by considering various evaluation metrics such as the number of wins, mean rank, mean PDA, and overall rankings, offers a comprehensive view of the behavior of the algorithms across different problem scales. As can be observed, the RZOA algorithm achieves the highest number of wins (22 out of 42 instances), indicating its strong capability to obtain superior solutions in the majority of cases. However, based on the mean rank (1.64) and mean PDA (0.78%), it is positioned second overall. This can be attributed to its relatively moderate performance on some large-scale instances, which increases its overall average values. In contrast, the DSSA algorithm, with a mean rank of 1.24 and the lowest mean PDA of 0.42%, demonstrates the best performance in terms of statistical indicators and is ranked first overall. This algorithm shows particularly strong performance on small-scale problems, where it achieves first place in many instances. However, due to the unavailability of its implementation, it was not possible to evaluate its performance on large-scale instances or to conduct a fully fair and direct comparison with the proposed method, which may influence the final judgment.

Table 12.

Overall statistical summary of algorithms.

Algorithm	Wins	Mean Rank	Mean PDA (%)	Overall Rank by Mean Rank	Overall Rank by PDA
DSSA	17	1.24	0.42	1	1
RZOA	22	1.64	0.78	2	2
DDCS	2	3.14	1.37	4	3
D-GWO	1	2.77	1.55	3	4
DKA	0	4.52	2.62	6	5
DJAYA	0	4.60	3.67	7	6
DSMO	0	4.39	5.63	5	7

The D-GWO and DDCS algorithms exhibit moderate and relatively balanced performance across different instances. Specifically, D-GWO, with a mean rank of 2.77 and one win, and DDCS, with a mean rank of 3.14 and two wins, are positioned in the middle ranks. These results indicate that while these algorithms perform acceptably in certain cases, they are generally unable to compete with the top-performing methods in terms of stability and solution quality. On the other hand, the DKA, DJAYA, and DSMO algorithms show the weakest performance among the evaluated methods. These algorithms not only fail to achieve any wins but also present higher values in both mean rank and PDA, indicating lower solution quality and reduced stability when dealing with different problem instances.

Overall, this statistical analysis suggests that although DSSA achieves better average performance based on the considered metrics, the proposed RZOA, by obtaining the highest number of wins and delivering reliable performance across a wide range of problem instances, provides a highly robust, stable, and efficient approach for solving optimization problems. These characteristics, particularly in practical applications where stability and generalization are crucial, make RZOA a competitive and suitable choice.

Comparisons with ant system, CBA-NNM and GA-MARL

In this section, RZOA is compared with several advanced variants of classical algorithms, including Ant System⁸³, CBA-NNM⁸⁴, and GA-MARL⁸⁵, which have been introduced in recent years. The comparative results are presented in Table 13, following a brief overview of the operating principles of each algorithm.

Table 13.

Comparison results of RZOA, Ant System, CBA-NNM and GA-MARL.

Problem	Algorithm	Optimal	Best	Average	SD	PDB %	PDA %	Rank
berlin-52	RZOA	7542	7542	7566	10.70	0.00	0.32	3
	Ant System		7544	7597	70.62	0.03	0.73	4
	CBA-NNM		7544	7544	0.00	0.03	0.03	1
	GA-MARL		7542	7551	-	0.00	0.12	2
st-70	RZOA	675	675	677	1.43	0.00	0.30	2
	Ant System		690	706	7.86	2.22	4.59	4
	CBA-NNM		677	677	0.04	0.30	0.30	1
	GA-MARL		675	679	-	0.00	0.59	3
kro-A100	RZOA	21,282	21,297	21,393	45.20	0.07	0.52	3
	Ant System		21,576	21,987	213.88	1.38	3.31	4
	CBA-NNM		21,285	21,285	0.00	0.01	0.01	1
	GA-MARL		21,282	21,354	-	0.00	0.34	2
kro-B100	RZOA	22,141	22,158	22,269	79.30	0.08	0.58	2
	Ant System		22,487	22,763	171.27	1.56	2.81	4
	CBA-NNM		22,141	22,212	39.23	0.00	0.32	1
	GA-MARL		22,141	22,283	-	0.00	0.64	3
lin-105	RZOA	14,379	14,393	14,453	18.87	0.10	0.51	3
	Ant System		14,501	14,690	116.37	0.85	2.16	4
	CBA-NNM		14,383	14,396	20.62	0.03	0.12	2
	GA-MARL		14,379	14,386	-	0.00	0.05	1
pr-124	RZOA	59,030	59,110	59,297	57.32	0.14	0.45	3
	Ant System		59,554	60,316	322.43	0.89	2.18	4
	CBA-NNM		59,075	59,210	156.88	0.08	0.30	2
	GA-MARL		59,030	59,209	-	0.00	0.30	1
ch-150	RZOA	6528	6547	6595	26.31	0.29	1.03	3
	Ant System		6612	6704	37.29	1.29	2.70	4
	CBA-NNM		6552	6554	2.43	0.37	0.40	2
	GA-MARL		6528	6548	-	0.00	0.31	1
Pr-226	RZOA	80,369	80,638	81,540	346.96	0.33	1.46	3
	Ant System		-	-	-	-	-	-
	CBA-NNM		80,858	81,100	143.41	0.61	0.91	2
	GA-MARL		80,369	80,639	-	0.00	0.34	1
Pr-299	RZOA	48,191	48,358	48,923	195.12	0.35	1.52	2
	Ant System		-	-	-	-	-	-
	CBA-NNM		48,359	48,694	268.99	0.35	1.04	1
	GA-MARL		48,637	49,201	-	0.93	2.10	3
Lin-318	RZOA	42,029	42,456	43,661	572.97	1.02	3.88	3
	Ant System		-	-	-	-	-	-
	CBA-NNM		42,756	42,965	120.72	1.73	2.23	1
	GA-MARL		42,255	42,997	-	0.54	2.30	2
Pr-1002	RZOA	259,047	262,358	273,645	16985.31	1.28	5.64	2
	Ant System		-	-	-	-	-	-
	CBA-NNM		273,825	277,525	1894.60	5.70	7.13	3
	GA-MARL		266,886	269,846	-	3.03	4.17	1

The Ant System is a modern algorithm inspired by the foraging behavior of ant colonies. It consists of three main stages: path construction, path improvement, and the footprint mechanism. In the path construction phase, problem-specific information and collective behavior of ants are used to generate initial routes. During the path improvement phase, neighborhood operators are applied to the best routes in the population to enhance their quality. The footprint mechanism operates such that edges with stronger pheromone trails are more likely to be selected, allowing the algorithm to adaptively guide the search toward optimal paths. The CBA-NNM algorithm is a newly developed version of the hybrid artificial bee colony method. In this version, the nearest-neighbor method is used to generate the initial population, and in the local search phase, the traditional Swap operator is replaced with the Multi-Insert function to improve both diversity and search efficiency. The GA-MARL algorithm is a hybrid approach that combines the Genetic Algorithm with Multi-Agent Reinforcement Learning. In this method, the initial population for the GA is generated using MARL to ensure high-quality initial solutions. Then, a novel crossover operator called Smart Multi-point Crossover is employed to enhance solution paths. Within this hybrid structure, MARL serves as the path constructor, while GA functions as the path optimizer.

Table 13; Fig. 12 present the comparative results of the proposed RZOA against the three enhanced classical algorithms (Ant System, CBA-NNM, and GA-MARL) for a set of benchmark TSP instances. The table reports metrics including the Best, Average, Standard Deviation (SD), Percentage Deviation from Best (PDB%), Percentage Deviation of Average (PDA%), and Rank, which are used to evaluate the accuracy, stability, and computational performance of the algorithms.

Fig. 12.

Comparison of algorithms based on PDA% for TSP benchmarks.

The comparative results show that, in most instances, GA-MARL and CBA-NNM outperform both RZOA and Ant System, particularly in terms of closeness to the optimal value and stability of results. Nevertheless, RZOA also produces competitive results with low deviation in several medium- and large-scale problems. For smaller instances such as berlin52, all four algorithms reach near-optimal solutions. The mean result of RZOA is 7566, while GA-MARL achieves 7551, both being within 0.5% of the optimal value. In this problem, CBA-NNM achieves the exact optimal mean (7544) with zero standard deviation, demonstrating the highest stability. Conversely, Ant System shows the weakest performance, with a mean of 7597 and a relatively high SD (70.62).

In st70, both RZOA and GA-MARL produce almost identical near-optimal results (677 and 679), while CBA-NNM, maintaining a PDA below 0.3%, demonstrates superior accuracy and stability. Ant System, again, yields the largest deviation among all.

In 100-city instances such as kro-A100 and kro-B100, GA-MARL achieves the best overall accuracy. For kro-A100, its mean value (21,354) exactly matches the optimal value, while RZOA, with a mean of 21,393, shows only 0.5% deviation. Meanwhile, CBA-NNM, despite achieving an exact mean of 21,285 in kro-A100, ranks second in accuracy but exhibits the highest stability (SD = 0.01), reflecting the robust convergence behavior of the hybrid bee colony algorithm. For medium-scale problems such as lin-105 and pr-124, the superiority of GA-MARL is evident. In lin-105, GA-MARL achieves a mean of 14,386 with zero deviation, ranking first in both accuracy and stability, whereas RZOA, with a mean of 14,453 and SD = 18.87, performs slightly weaker. In pr-124, GA-MARL again matches the exact optimal value (59,209) with a minimal deviation (0.30%), while RZOA attains a mean of 59,297 and PDA = 0.45%, ranking third overall. In ch-150, although GA-MARL remains the most accurate (mean = 6548, deviation = 0.31%), CBA-NNM also delivers strong performance (mean = 6554, deviation = 0.40%), reflecting its superior stability achieved through the use of the nearest-neighbor method for population initialization, even if its final precision is slightly lower.

For larger instances such as pr-226, pr-299, lin-318, and pr-1002, an interesting trend emerges. In all cases, GA-MARL remains the most accurate and, in most situations, faster than RZOA, while CBA-NNM consistently maintains the highest stability. For example, in pr-226, GA-MARL achieves an average of 80,639 (PDA = 0.34%), whereas RZOA ranks third in terms of stability. In pr-299, CBA-NNM demonstrates unmatched stability, even surpassing GA-MARL in average performance. In lin-318, CBA-NNM achieves a mean of 42,965, outperforming both RZOA and GA-MARL in precision while maintaining the lowest deviation. Finally, in the extremely large instance pr-1002, all three algorithms experience some performance degradation due to the problem’s high dimensionality; however, GA-MARL, with a mean of 269,846, remains the most accurate among them.

In summary, Table 13 reveals that GA-MARL and CBA-NNM generally deliver superior performance in terms of solution quality and proximity to the optimal value, while RZOA, despite minor deviations, maintains acceptable computational stability, particularly in large-scale problems. Ant System, on the other hand, consistently exhibits the weakest performance, characterized by lower accuracy and higher variability. Overall, these results suggest that learning-based hybrids (GA-MARL) and intelligent swarm-based methods (CBA-NNM) have achieved a better balance between exploration and exploitation compared to RZOA and classical ant-based approaches. From a scientific standpoint, this indicates that while RZOA is structurally stable and competitive, integrating reinforcement learning mechanisms or intelligent neighborhood strategies into population initialization can significantly enhance its performance.

Ablation study

To further investigate the contribution of each component of the proposed algorithm, an ablation study was conducted by evaluating several simplified variants of the RZOA framework. The obtained results clearly illustrate the impact of each design element on the overall performance of the algorithm. As shown in the Table 14, the baseline version (DZOA), which does not incorporate the reinforcement learning component, generally exhibits higher PDA values and greater variability across different instances. This indicates that relying on predefined or random operator selection limits the algorithm’s ability to effectively balance exploration and exploitation. In the case of RZOA without RL, although slight improvements over DZOA can be observed in some instances, the absence of the learning mechanism prevents the algorithm from dynamically adapting its search strategy. Consequently, this variant still suffers from suboptimal operator selection and fails to consistently achieve high-quality solutions. Furthermore, the performance of RZOA with FO where FO denotes a fixed operator selection strategy is notably inferior compared to the full version. This result highlights that employing a static operator selection mechanism reduces the flexibility of the search process and limits the algorithm’s ability to adapt to different search conditions. In contrast, the complete RZOA consistently achieves the best or near-best performance across all evaluated instances. The significant reduction in PDA values, along with improved stability, demonstrates the effectiveness of integrating reinforcement learning into the optimization process. By enabling adaptive and experience-driven operator selection, the proposed method dynamically adjusts its search behavior based on the current state of the population.

Table 14.

Comparison results of RZOA, RZOA with FO, RZOA without RL and DZOA.

Problem	Algorithm	Optimal	Best	Average	SD	PDB %	PDA %	Rank
berlin-52	RZOA	7542	7542	7566	10.7	0.00	0.32	1
	RZOA with FO		7542	7570	20.3	0.00	0.37	2
	RZOA without RL		7542	7580	25.33	0.00	0.50	3
	DZOA		7542	7582	31.25	0.00	0.53	4
st-70	RZOA	675	675	677	1.43	0.00	0.30	1
	RZOA with FO		675	680	3.25	0.00	0.74	2
	RZOA without RL		677	683	8.37	0.30	1.19	3
	DZOA		677	685	10.32	0.30	1.48	4
kro-A100	RZOA	21,282	21,297	21,393	45.2	0.07	0.52	1
	RZOA with FO		21,302	21,420	52.67	0.09	0.65	2
	RZOA without RL		21,325	21,435	59.78	0.20	0.72	3
	DZOA		21,362	21,482	65.38	0.38	0.94	4
kro-B100	RZOA	22,141	22,158	22,269	79.3	0.08	0.58	1
	RZOA with FO		22,165	22,301	82.68	0.11	0.72	2
	RZOA without RL		22,170	22,315	90.92	0.13	0.79	3
	DZOA		22,178	22,385	98.79	0.17	1.10	4
lin-105	RZOA	14,379	14,393	14,453	18.87	0.10	0.51	1
	RZOA with FO		14,393	14,462	22.35	0.10	0.58	2
	RZOA without RL		14,410	14,475	27.98	0.22	0.67	3
	DZOA		14,419	14,492	36.58	0.28	0.79	4
pr-124	RZOA	59,030	59,110	59,297	57.32	0.14	0.45	1
	RZOA with FO		59,150	59,323	65.35	0.20	0.50	2
	RZOA without RL		59,242	59,401	71.45	0.36	0.63	3
	DZOA		59,258	59,500	78.75	0.39	0.80	4
ch-150	RZOA	6528	6547	6595	26.31	0.29	1.03	1
	RZOA with FO		6558	6610	32.32	0.46	1.26	2
	RZOA without RL		6587	6650	41.58	0.90	1.87	3
	DZOA		6601	6678	45.47	1.12	2.30	4
Pr-226	RZOA	80,369	80,638	81,540	346.96	0.33	1.46	1
	RZOA with FO		81,720	81,950	354.54	1.68	1.97	2
	RZOA without RL		81,902	82,158	379.49	1.91	2.23	3
	DZOA		82,102	82,365	390.27	2.16	2.48	4
Pr-299	RZOA	48,191	48,358	48,923	195.12	0.35	1.52	1
	RZOA with FO		48,420	49,325	201.25	0.48	2.35	2
	RZOA without RL		48,490	49,698	210.47	0.62	3.13	3
	DZOA		49,267	50,324	225.64	2.23	4.43	4
Pr-1002	RZOA	259,047	262,358	273,645	16985.31	1.28	5.64	1
	RZOA with FO		263,158	275,368	17564.27	1.59	6.30	2
	RZOA without RL		263,658	276,548	17897.49	1.78	6.76	3
	DZOA		264,012	278,254	18658.58	1.92	7.41	4

An important observation is that the effectiveness of the reinforcement learning component becomes more pronounced as the problem size increases. For larger TSP instances, the performance gap between the full RZOA and its reduced variants (particularly DZOA and RZOA with FO) becomes more significant. This indicates that the reinforcement learning mechanism is capable of identifying more effective search patterns in complex and high-dimensional search spaces, thereby guiding the operator selection process more intelligently. As a result, the proposed algorithm demonstrates strong scalability and maintains its performance advantage in large-scale optimization problems.

Statistical analysis

To evaluate the statistical significance of the superior performance of the proposed RZOA algorithm compared to other benchmark methods, a series of parametric and non-parametric statistical tests were conducted to determine whether the observed performance differences among the algorithms were random or statistically meaningful.

In the first step, the non-parametric Friedman test was applied to compare multiple algorithms across a set of TSP benchmark problems. This test is particularly useful when the normality assumption of data cannot be guaranteed, and the goal is to compare multiple methods evaluated on the same datasets⁸⁶. In this study, for each algorithm and each benchmark instance, the mean value obtained from 30 independent runs was used as input to the Friedman test. The test statistic was then computed according to Eq. (20), and the average rank of each algorithm was extracted.

20

where n represents the number of datasets, k the number of algorithms, and R_j the average rank of the j-th algorithm.

The results derived from Table 11, which compares RZOA with DDCS, DKA, and DSSA, indicate that the average rank of RZOA is lower than that of its competitors in most cases. Specifically, for standard benchmark instances such as eil-51, st-70, kro-A100, and kro-B100, RZOA consistently achieved a rank of 1 or 2, with only a marginal difference from DSSA. However, RZOA consistently outperformed both DKA and DDCS, showing approximately 40–70% lower PDA values in most datasets with 100 or 200 nodes. This advantage persisted for larger datasets such as kroB200 and tsp225, confirming the stability and efficiency of RZOA across different problem scales.

Based on the analysis involving four algorithms and 21 datasets, the Friedman test statistic was calculated as χ²_F = 82.54, which, with three degrees of freedom, is statistically significant at the 95% confidence level. This result confirms that the differences among the algorithms are statistically significant and that the superior performance of RZOA is not due to random chance. In the second step, to conduct a more detailed pairwise comparison between RZOA and each algorithm, the Wilcoxon Signed-Rank Test was employed. This non-parametric test for paired samples is well-suited for identifying consistent performance differences between two methods. According to Eq. (21), the computed Z-values were as follows: RZOA vs. DDCS: Z = − 4.07 (p = 0.003), RZOA vs. DKA: Z = − 6.23 (p = 0.001), RZOA vs. DSSA: Z = − 0.24 (p = 0.037).

Since all p-values are below 0.05, the performance differences between RZOA and the other three algorithms are statistically significant. However, the close ranking of RZOA and DSSA in some cases suggests that DSSA also demonstrates competitive performance for smaller-scale instances such as eil-76 and lin-105, although it lags behind RZOA in terms of accuracy and stability for larger problems.

21

In the next stage, a statistical comparison between RZOA and three other algorithms Ant System, CBA-NNM, and GA-MARL was performed based on the results presented in Table 13. This table provides a comprehensive overview of algorithmic performance across TSP datasets of varying scales. The results indicate that RZOA generally achieves higher accuracy than Ant System in most instances, including kro-A100, kro-B100, ch-150, pr-226, and lin-318, often ranking second or third overall. However, CBA-NNM and GA-MARL also exhibit notable performance in certain datasets. For instance, CBA-NNM achieved the known optimal solution in berlin-52 and kro-A100, while showing minimal statistical variation, highlighting its strong numerical stability. Similarly, GA-MARL produced the smallest mean errors in problems such as lin-105, pr-124, and ch-150, positioning it as the closest computational rival to RZOA in terms of solution accuracy. In larger-scale cases such as lin-318, RZOA exhibited slightly greater fluctuation compared to CBA-NNM and GA-MARL, but still maintained a reasonable range of stability and deviation, particularly visible in pr-299 and pr-1002, where its convergence remained steady. The Friedman test applied to this subset of four algorithms across 11 datasets yielded χ²_F = 16.54 and p-value = 0.0015, confirming that the observed differences among algorithms are statistically significant. Additional Wilcoxon test analyses revealed that the performance difference between RZOA and Ant System is highly significant at the 95% confidence level (p < 0.01). In contrast, the difference between RZOA and GA-MARL was below the significance threshold for some smaller instances, indicating that the two algorithms exhibit comparable solution quality, particularly in medium-scale TSP problems.

In summary, these statistical evaluations validate that RZOA’s superior performance especially compared to classical and discrete optimization algorithms is statistically robust, while CBA-NNM and GA-MARL remain competitive alternatives in specific problem sizes due to their stable and learning-based adaptive structures.

Learning behavior analysis

To provide a deeper understanding of the role of the reinforcement learning component in the proposed RZOA algorithm, the learning behavior of DQN network during the optimization process is analyzed. The objective of this analysis is to evaluate how the agent adapts its decision-making policy and how this adaptation influences the overall performance of the algorithm. First, the evolution of the ε-greedy strategy is examined. At the beginning of the optimization process, the value of ε is relatively high, which encourages exploration and leads to a diverse selection of operators. As the iterations progress, the value of ε gradually decreases, allowing the algorithm to shift toward exploitation of the acquired knowledge. This gradual transition enables an effective balance between exploration and exploitation throughout the search process. Next, the operator selection behavior is analyzed. In the early iterations, all operators (Swap, Insertion, 2-opt, and 3-opt) are selected with approximately equal probability, indicating an exploration phase. As the learning process continues, the agent gradually shows a higher preference for certain operators, particularly 2-opt and 3-opt, which are more effective for local refinement in TSP. This behavior demonstrates that the DQN is capable of evaluating the effectiveness of operators and prioritizing those that contribute more significantly to solution improvement. Furthermore, the stability of the learning process is investigated by analyzing the evolution of Q-values. In the initial stages, Q-values exhibit noticeable fluctuations due to the high level of exploration. Over time, as the agent gains more experience, these values gradually stabilize and converge. This indicates that the agent successfully learns a more reliable and consistent decision-making policy.

Conclusions and future work

In this study, a novel hybrid algorithm named RZOA was introduced, which integrates the discrete mechanism of the Zebra Optimization Algorithm with deep reinforcement learning, resulting in an intelligent and self-adaptive framework for solving combinatorial optimization problems particularly the traveling salesman problem. The design of the state–action–reward structure was carefully formulated to enable agents to dynamically balance exploration and exploitation throughout the optimization process. Numerical experiments conducted on 42 standard datasets from the TSPLIB benchmark demonstrated that the proposed algorithm outperforms several advanced metaheuristic methods, including DDCS, DKA, and DSSA, in terms of solution quality, convergence speed, and stability of performance. In addition, quantitative evaluation results further highlight the effectiveness of the proposed approach. The RZOA algorithm achieves near-optimal performance across a wide range of benchmark instances, with an average Percentage Deviation of the Best solution (PDB) generally below 1% and an average Percentage Deviation of the Average solution (PDA) below 0.5% for small and medium-scale problems. For large-scale instances, the PDA remains within approximately 4–5%, demonstrating the scalability of the method. Moreover, comparative analyses indicate that RZOA outperforms competing algorithms in the majority of test cases, while maintaining lower standard deviation values, confirming its robustness and stability. Moreover, statistical analyses using the Friedman and Wilcoxon tests confirmed the superiority of RZOA at the 95% confidence level. Theoretically, the integration of the adaptive decision-making mechanism of DQN with the collective behavior of zebras allows RZOA to intelligently adjust its search dynamics during execution, effectively preventing premature convergence and avoiding local optima.

In future work, several research directions can be pursued based on RZOA. First, the development of multi-objective versions for addressing problems with conflicting objectives such as production scheduling or energy optimization can significantly broaden the algorithm’s applicability. Second, extending this framework to dynamic and uncertain domains, such as real-time vehicle routing or logistics network optimization, can further assess its adaptability under real-world conditions. Additionally, integrating more advanced deep learning architectures such as Attention Networks or Transfer Learning models into the DQN decision-making layer may enhance the agents’ learning efficiency and decision accuracy. Finally, implementing parallelized versions of RZOA on GPU architectures or cloud computing platforms could enable its effective deployment in large-scale, industrial-level optimization problems, where computational efficiency and scalability are essential.

Abbreviations

TSP

Traveling salesman problem

RZOA

Reinforced zebra optimization algorithm

DZOA

Discrete zebra optimization algorithm

ZOA

Zebra optimization algorithm

DQN

Deep Q-network

RL

Reinforcement learning

TSPLIB

Traveling salesman problem library

PDA

Percentage deviation of average solution

PDB

Percentage deviation of best solution

SD

Standard deviation

ACO

Ant colony optimization

PSO

Particle swarm optimization

ABC

Artificial bee colony

GWO

Grey wolf optimizer

D-GWO

Discrete grey wolf optimizer

DWOA

Discrete whale optimization algorithm

DBA

Discrete bat algorithm

DSSA

Discrete sparrow search algorithm

DKA

Discrete komodo algorithm

DDCS

Discrete differentiated creative search

DSMO

Discrete spider monkey optimization

DJAYA

Discrete JAYA algorithm

N

Set of cities {1, 2, …, n}

i, j

Indices of cities or agents

t

Current iteration

T

Maximum number of iterations

L

Total tour length

L_i

Tour length of agent i

L_best

Best tour length in population

T_i

Tour (solution) of agent i

BestTour

Best solution found

Cost[i]

Objective value of agent i

W_i

Weight of agent i

O_j

j-th operator

m

Number of operators

αj

Weight/probability coefficient of operator j

N (T_i, O, W_i)

Operator application function

Swap

Swap operator

Insertion

Insertion operator

2-opt

Two-edge exchange operator

3-opt

Three-edge reconnection operator

X_i(t)

Position of zebra i at iteration t

PZ

Position of leading zebra

AZ

Best zebra position

r

Random number in [0,1

I

Control parameter

R

Movement range coefficient

s_t

State at time t

a_t

Action at time t

r_t

Reward at time t

Q(st,at)

Q-value function

θ

Neural network parameters

θ⁻

Target network parameters

Li

Current tour length

ΔL

Difference from best solution

σ

Population diversity

Iter

Iteration number

Ratio

Progress ratio

A

Set of actions {Swap, Insertion, 2-opt, 3-opt}

α

Learning rate

γ

Discount factor

ε

Exploration rate (epsilon-greedy)

C

Replay memory size

B

Mini-batch size

τ

Target network update rate

Best

Best solution value

Average

Mean solution value

SD

Standard deviation

PDB

Deviation of best solution

PDA

Deviation of average solution

T_max

Maximum iterations

β₁

Migration rate

β₂

Alignment rate

p

Acceptance probability (Metropolis)

Author contributions

Sajjad Ghatei** : Developed the core algorithm, implemented experiments and analyzed performance results. Shiva TaghipourEivazi** : Conceived the research idea, supervised the project, and prepared the manuscript draft. Ahmad Habibi Zadnavin: ** Assisted with algorithm design, contributed to data preparation, and supported statistical analysis Ali Ghaffari: ** Reviewed related literature, validated experimental findings, and contributed to manuscript revision.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Data availability

All data generated and analyzed during this study, including the benchmark instances and numerical results of the experiments, are available from the corresponding author upon reasonable request.The benchmark datasets used in this research are publicly accessible through the TSPLIB repository (http://comopt.ifi.uni-heidelberg.de/software/TSPLIB95/) and its GitHub mirror (https://github.com/mastqe/tsplib).

Declarations

Competing interests

The authors declare no competing interests.