Algorithm 1: Psuedocode of the BO Enhancement: Enhance Bayesian Optimization with Metaheuristic Method

Required: An acquisition function 1: Inputs:  Bayesian Optimization process 2: While the stop criteria are not fulfilled, do the following:   3:  Select the next point to evaluate based on an acquisition function (Expected    Improvement), which balances the exploration of new points and the    the exploitation of promising areas of the search space until maximize accuracy    is obtained:

$x^{\mathrm{*}}=\arg \underset{x\in X}{\max }f\left(x\right)$.

4:  Calculation of EI for each set of hyperparameters and fine-tuning layers using the probabilistic model provides:

$EI\left(x\right)=E\left[\mathrm{m}\mathrm{a}\mathrm{x}\left(0,f\left(x\right)-f\left(x^{*}\right)\right)\right].$

5:  Evaluate the mean µ(x) and variance$\sigma \left(x\right)$of the probabilistic model at x.    These values are estimated based on the observed values of the objective    function at previous evaluation points. 6:  Initialize the population of mean µ(x) and variance$\sigma \left(x\right)$of the probabilistic    model at x. 7:  From the set of a population of mean µ(x) and variance$\sigma \left(x\right),$maximize by me taheuristic method (PSO, ABC, HHO and SFO). The goal is to explore the search space and generate a diverse set of hyperparameters and fine-tuned layers solution. 8:  Evaluate the objective function for each new set of hyperparameters and fine-tuned layers solution in the population. 9:  Select the best hyperparameter and fine-tuned layers from the entire population based on the objective function values. 10: End while