Progress Rate Analysis of Evolution Strategies on the Rastrigin Function: First Results

Abstract

A first order progress rate is derived for the intermediate multi-recombinative Evolution Strategy (μ/μ_I, λ)-ES on the highly multimodal Rastrigin test function. The progress is derived within a linearized model applying the method of so-called noisy order statistics. To this end, the mutation-induced variance of the Rastrigin function is determined. The obtained progress approximation is compared to simulations and yields strengths and limitations depending on mutation strength and distance to the optimizer. Furthermore, the progress is iterated using the dynamical systems approach and compared to averaged optimization runs. The property of global convergence within given approximation is discussed. As an outlook, the need of an improved first order progress rate as well as the extension to higher order progress including positional fluctuations is explained.

1. Introduction

Evolution Strategies (ES) [12,13] are well-recognized Evolutionary Algorithms suited for real-valued non-linear optimization. State-of-the-art ES such as the CMA-ES [8] or its simplification [5] are also well-suited for locating global optimizers in highly multimodal fitness landscapes. While the CMA-ES was originally mainly intended for non-differentiable optimization problems, but yet regarded as a locally acting strategy, it was already in [7] observed that using a large population size can make the ES a strategy that is able to locate the global optimizer among a huge number of local optima. This is a surprising observation when considering the ES as a strategy that acts mainly local in the search space following some kind of gradient or natural gradient [3,6,11]. As one can easily check using standard (highly) multimodal test functions such as Rastrigin, Ackley, and Griewank to name a few, this ES property is not intimately related to the covariance matrix adaptation (CMA) ES which generates non-isotropic correlated mutations, but can also be found in (μ/μ_I, λ)-ES with isotropic mutations. Therefore, if one wants to understand the underlying working principles how the ES locates the global optimizer, the analysis of the (μ/μ_I, λ)-ES should be the starting point.

The question regarding why and when optimization algorithms − originally designed for local search − are able to locate global optima has gained attention in the last few years. A recurring idea comes from relaxation procedures that transform the original multimodal optimization problem into a convex optimization problem called Gaussian continuation [9]. Gaussian continuation is nothing else but a convolution of the original optimization problem with a Gaussian kernel. As has been shown in [10], using the right Gaussian, Rastrigin-like functions can be transformed into a convex optimization problem, thus making it accessible to gradient following strategies. However, this raises the question how to perform the convolution efficiently. One road followed in [14] uses high-order Gauss-Hermite integration in conjunction with a gradient descent strategy yielding surprisingly good results. The other road coming to mind is approximating the convolution by Gaussian sampling. This resembles the procedure ES do: starting from a parental state, offspring are generated by Gaussian mutations. The problem is, however, that in order to get a reliable gradient, a huge number of samples, i.e. offspring in ES must be generated in order to get reliable convolution results. The number of offspring needed to get reliable estimates seems much larger than the offspring population size needed in ES experiments conducted in [7] showing approximately a linear relation between problem dimension N and population size for the Rastrigin function. Therefore, understanding the ES performance from viewpoint of Gaussian relaxation does not seem to help much.

The approach followed in this paper will incorporate two main concepts, namely a progress rate analysis as well as its application within the so-called evolution equations modeling the transition dynamics of the ES [2]. The progress rate measure yields the expected positional change in search space between two generations depending on location, strategy and test function parameters. Aiming to investigate and understand the dynamics of globally converging ES runs, the progress rate is an essential quantity to model the expected evolution dynamics over many generations.

This paper provides first results of a scientific program that aims at an analysis of the performance of the (μ/μ_I, λ)-ES on Rastrigin’s test function based on a first order progress rate. After a short introduction of the (μ/μ_I, λ)-ES, the N-dimensional first order progress will be defined and an approximation will be derived resulting in a closed form expression. The predictive power and its limitations will be checked by one-generation experiments. The progress rate will then be used to simulate the ES dynamics on Rastrigin using difference equations. This simulation will be compared with real runs of the (μ/μ_I, λ)-ES. In a concluding section a summary of the results and outlook of the future research will be given.

2. Rastrigin Function and Local Quality Change

The real-valued minimization problem defined for an N-dimensional search vector y = (y₁, …, y_N) is performed on the Rastrigin test function f given by

$$f(y)=\sum _{i=1}^N{f}_i\left(y_i\right)=\sum _{i=1}^N{y}_i^2+A-A\cos \left(\alpha y_i\right),$$ (1)

with A denoting the oscillation amplitude and α = 2π the corresponding frequency. The quadratic term with superimposed oscillations yields a finite number of local minima M for each dimension i, such that the overall number of minima scales exponentially as M^N posing a highly multimodal minimization problem. The global optimizer is at ŷ = 0.

For the progress rate analysis in Sect. 4 the local quality function Q_y(x) at y due to mutation vector x = (x₁, …, x_N) is needed. In order to reuse results from noisy progress rate theory it will be formulated for the maximization case of F(y) = −f(y) with F_i(y_i) = −f_i(y_i), such that local quality change yields

$$Q_{\mathbf{\text{y}}}(\mathbf{\text{x}})=F(\mathbf{\text{y}}+\mathbf{\text{x}})-F(\mathbf{\text{y}})=f(\mathbf{\text{y}})-f(\mathbf{\text{y}}+\mathbf{\text{x}}).$$ (2)

Q_y(x) can be evaluated for each component i independently giving

$$Q_y(\mathbf{\text{x}})=\sum _{i=1}^N{Q}_i\left(x_i\right)=\sum _{i=1}^N{f}_i\left(y_i\right)-f_i\left(y_i+x_i\right)$$ (3)

$$=\sum _{i=1}^N-\left(x_i^2+2y_i{x}_i+A\cos \left(\alpha y_i\right)\left(1-\cos \left(\alpha x_i\right)\right)+A\sin \left(\alpha y_i\right)\sin \left(\alpha x_i\right)\right).$$ (4)

A closed form solution of the progress rate appears to be obtainable only for a linearized expression of Q_i(x_i). A first approach taken in this paper is based on a Taylor expansion for the mutation x_i and discarding higher order terms

$$Q_i\left(x_i\right)=F_i\left(y_i+x_i\right)-F_i\left(y_i\right)=\frac{\partial F_i}{\partial y_i}{x}_i+O\left(x_i^2\right)$$ (5)

$$\approx \left(-2y_i-\alpha A\sin \left(\alpha y_i\right)\right)x_i=:-f_i^{\prime }{x}_i,$$ (6)

using the following derivative terms

$$k_i=2y_i\text{and}{d}_i=\alpha A\sin \left(\alpha y_i\right),\text{such}\mathrm{that}\frac{\partial f_i}{\partial y_i}=f_i^{\prime }=k_i+d_i.$$ (7)

A second approach is to consider only the linear term of Eq. (4) and neglect all non-linear terms denoted by δ(x_i) according to

$$Q_i\left(x_i\right)=-2y_i{x}_i-x_i^2-A\cos \left(\alpha y_i\right)\left(1-\cos \left(\alpha x_i\right)\right)-A\sin \left(\alpha y_i\right)\sin \left(\alpha x_i\right)$$ (8)

$$=-2y_i{x}_i+\delta \left(x_i\right)\approx -2y_i{x}_i=-k_i{x}_i.$$ (9)

The linearization using $f_i^{\prime }$ is a local approximation of the function incorporating oscillation parameters A and α. Using only k_i (setting d_i = 0) discards oscillations by approximating the quadratic term via $k_i=\partial \left(y_i^2\right)/\partial y_i=2y_i$ with negative sign due to maximization. Both approximations will be evaluated later.

3. The (μ/μ_I, λ)-ES with Normalized Mutations

The Evolution Strategy under investigation consists of a population of μ parents and λ offspring (μ < λ) per generation g. Algorithm 1 is presented below and offspring variables are denoted with overset “∼”.

Population variation is achieved by applying an isotropic normally distributed mutation x ∼ σ𝒩(0, 1) with strength σ to the parent recombinant in Lines 6 and 7. The recombinant is obtained using intermediate recombination of all μ parents equally weighted in Line 11. Selection of the m = 1, …, μ best search vectors y_m;λ (out of λ) according to their fitness is performed in Line 10.

Note that the ES in Algorithm 1 operates under constant normalized mutation σ* in Lines 3 and 12 using the spherical normalization

$${\sigma }^{\ast }=\frac{{\sigma }^{(g)}N}{\left|\right|y^{(g)}\left|\right|}=\frac{{\sigma }^{(g)}N}{R^{(g)}}.$$ (10)

This property ensures global convergence of the algorithm as the mutation strength σ^(g) decreases if and only if the residual distance ║y^(g)║ = R^(g) decreases. While σ* is not known during black-box optimizations, it is used here to investigate the dynamical behavior of the ES using the first order progress rate approach to be developed in this paper. Incorporating self-adaptation of σ or cumulative step-size adaptation remains for future research.

Algorithm 1. (μ/μ_I, λ)-ES with constant σ*.

1: g ← 0

2: y⁽⁰⁾ ← y^(init)

3: σ⁽⁰⁾ ← σ* ║y⁽⁰⁾║ /N

4: repeat

5:     for l = 1, …, λ do

6:         ${\tilde{x}}_l\leftarrow {\sigma }^{(g)}{𝒩}_l(0,1)$

7:         ${\tilde{y}}_l\leftarrow y^{(g)}+{\tilde{x}}_l$

8:         ${\tilde{f}}_l\leftarrow f\left({\tilde{y}}_l\right)$

9:     end for

10:    $\left({\tilde{y}}_{1;\lambda },\dots ,{\tilde{y}}_{\mu ;\lambda }\right)\leftarrow \text{sort}(\tilde{y}\text{w}.\text{r}.\text{t}.\text{ascending}\tilde{f})$

11:    $y^{(g+1)}\leftarrow \frac{1}{\mu }{\displaystyle {\sum }_{m=1}^{\mu }{\tilde{y}}_{m;\lambda }}$

12:    σ^(g+1) ← σ* ║y^(g+1)║/N

13:    g ← g + 1

14: until termination criterion

4. Progress Rate

4.1. Definition

Having introduced the Evolution Strategy, we are interested in the expected one-generation progress of the optimization on the Rastrigin function (1) before investigating the dynamics over multiple generations.

A first order progress rate φ_i for the i-th component between two generations g → g + 1 can be defined as the expectation value over the positional difference of the parental components

$${\phi }_i=\text{E}\left[y_i^{(g)}-y_i^{(g+1)}|{\sigma }^{(g)},y^{(g)}\right]=y_i^{(g)}-\text{E}\left[y_i^{(g+1)}|{\sigma }^{(g)},y^{(g)}\right],$$ (11)

given mutation strength σ^(g) and the position y^(g). First, an expression for y^(g+1) is needed, see Algorithm 1, Line 11. It is the result of mutation, selection and recombination of the m = 1, …, μ offspring vectors yielding the highest fitness, such that $y^{(g+1)}=\frac{1}{\mu }{\displaystyle {\sum }_{m=1}^{\mu }{\tilde{y}}_{m;\lambda }}=\frac{1}{\mu }{\displaystyle {\sum }_{m=1}^{\mu }{\left(y^{(g)}+x\right)}_{m;\lambda }}$. Considering the i-th component, noting that y^(g) is the same for all offspring and setting (x_m;λ)_i = x_m;λ one has

$$y_i^{(g+1)}=\frac{1}{\mu }\sum _{m=1}^{\mu }\left(y_i^{(g)}+x_{m;\lambda }\right)=y_i^{(g)}+\frac{1}{\mu }\sum _{m=1}^{\mu }{x}_{m;\lambda }.$$ (12)

Taking the expectation $\text{E}\left[y_i^{(g+1)}\right]$, setting x = σz = σ𝒩(0, 1) and inserting the expression back into (11) yields

$${\phi }_i=-\frac{1}{\mu }\text{E}\left[\sum _{m=1}^{\mu }{x}_{m;\lambda }|{\sigma }^{(g)},y^{(g)}\right]=-\frac{\sigma }{\mu }\text{E}\left[\sum _{m=1}^{\mu }{z}_{m;\lambda }|{\sigma }^{(g)},y^{(g)}\right].$$ (13)

Therefore progress can be evaluated by averaging over the expectations of μ selected mutation contributions. In principle this task can be solved by deriving the induced order statistic density p_m;λ for the m-th best individual and subsequently solving the integration over the i-th component

$${\phi }_i=-\frac{1}{\mu }\sum _{m=1}^{\mu }{\int }_{-\infty }^{\infty }{x}_i{p}_{m;\lambda }\left(x_i|{\sigma }^{(g)},y^{(g)}\right)\text{d}{x}_i.$$ (14)

However, the task of computing expectations of sums of order statistics under noise disturbance has already been discussed and solved by Arnold in [1]. Therefore the problem of Eq. (13) will be reformulated in order to apply the solutions provided by Arnold.

4.2. Expectations of Sums of Noisy Order Statistics

Let z be a random variate with density p_z(z) and zero mean. The density is expanded into a Gram-Charlier series by means of its cumulants κ_i (i ≥ 1) according to [1, p. 138, D.15]

$$p_z(z)=\frac{1}{\sqrt{2\pi {\kappa }_2}}{\text{e}}^{-\frac{z^2}{2{\kappa }_2}}\left(1+\frac{{\gamma }_1}{6}{\text{He}}_3\left(\frac{z}{\sqrt{{\kappa }_2}}\right)+\frac{{\gamma }_2}{24}{\text{He}}_4\left(\frac{z}{\sqrt{{\kappa }_2}}\right)+\dots \right),$$ (15)

with expectation κ₁ = 0, variance κ₂, skewness ${\gamma }_1={\kappa }_3/{\kappa }_2^{3/2}$, excess ${\gamma }_2={\kappa }_4/{\kappa }_2^2$ (higher order terms not shown) and He_k denoting the k-th order probabilist’s Hermite polynomials. For the problem at hand, see Eq. (13), the mutation variate z ∼ 𝒩(0, 1) with κ₂ = 1 and κ_i = 0 for i ≠ 2 yielding a standard normal density.

Furthermore, let $ϵ\sim 𝒩\left(0,{\sigma }_{ϵ}^2\right)$ model additive noise disturbance, such that resulting observed values are v = z + ϵ. Selection of the m-th largest out of λ values yields

$$v_{m;\lambda }=(z+𝒩(0,{\sigma }_{ϵ}^2){)}_{m;\lambda },$$ (16)

and the distribution of selected source terms z_m;λ follows a noisy order statistic with density p_m;λ. Given this definition and a linear relation between z_m;λ and v_m;λ the method of Arnold is applicable.

In our case the i-th mutation component x_m;λ of Eq. (13) is related to selection via the quality change defined in Eq. (3). Maximizing the fitness F_i(y_i + x_i) conforms to maximizing quality Q_i(x_i) with F_i(y_i) being a constant offset.

Aiming at an expression of form (16) and starting with (3), we first isolate component Q_i from the remaining N − 1 components denoted by Σ_j≠i Q_j. Then, approximations are applied to both terms yielding

$$Q_y(\mathbf{\text{x}})=Q_i\left(x_i\right)+\sum _{j\ne i}{Q}_j\left(x_j\right)$$ (17)

$$\approx -f_i^{\prime }{x}_i+𝒩\left(E_i,D_i^2\right),$$ (18)

with linearization (6) applied to Q_i(x_i). Additionally, ${\displaystyle {\sum }_{j\ne i}{Q}_j}\simeq 𝒩\left(E_i,D_i^2\right)$, as the sum of independent random variables asymptotically approaches a normal distribution in the limit N → ∞ due to the Central Limit Theorem. This is ensured by Lyapunov’s condition provided that there are no dominating components within the sum due to largely different values of y_j. The corresponding Rastrigin quality variance $D_i^2=\text{Var}\left[{\displaystyle {\sum }_{j\ne i}{Q}_j(x_j)}\right]$ is calculated in the supplementary material (https://github.com/omam-evo/paper/blob/main/ppsn22/PPSN22_OB22.pdf). As the expectation E_i = E[Σ_j≠i Q_j(x_j)] is only an offset to Q_y(x) it has no influence on the selection and its calculation can be dropped.

Using x_i = σz_i and $f_i^{\prime }=\mathrm{sgn}\left(f_i^{\prime }\right)\left|f_i^{\prime }\right|$, expression (18) is reformulated as

$$Q_y(\mathbf{\text{x}})=-\mathrm{sgn}\left(f_i^{\prime }\right)\left|f_i^{\prime }\right|\sigma z_i+E_i+𝒩\left(0,D_i^2\right)$$ (19)

$$\frac{Q_{\mathbf{\text{y}}}(\mathbf{\text{x}})-E_i}{\left|f_i^{\prime }\right|\sigma }=\mathrm{sgn}\left(-f_i^{\prime }\right)z_i+𝒩\left(0,\frac{D_i^2}{{\left(f_i^{\prime }\sigma \right)}^2}\right).$$ (20)

The decomposition using sign function and absolute value is needed for correct ordering of selected values w.r.t. z_i in (20).

Given result (20), one can define the linearly transformed quality measure $v_i:=\left(Q_y(x)-E_i\right)/\left|f_i^{\prime }\right|\sigma$ and noise variance ${\sigma }_{ϵ}^2:={\left(D_i/f_i^{\prime }\sigma \right)}^2$, such that the selection of mutation component sgn $\left(-f_i^{\prime }\right)z_i$ is disturbed by a noise term due to the remaining N − 1 components. A relation of the form (16) is obtained up to the sign function.

In [1] Arnold calculated the expected value of arbitrary sums S_P of products of noisy ordered variates containing ν factors per summand

$$S_P=\sum _{\left\{n_1,\dots ,n_{\nu }\right\}}{z}_{n_1;\lambda }^{p_1}\cdots z_{n_{\nu };\lambda }^{p_{\nu }},$$ (21)

with random variate z introduced in Eqs. (15) and (16). The vector P = (p₁, …, p_ν) denotes the positive exponents and distinct summation indices are denoted by the set {n₁, …, n_ν}. The generic result for the expectation of (21) is provided in [1, p. 142, D.28] and was adapted to account for the sign difference between (16) and (20) resulting in possible exchanged ordering. Performing simple substitutions in Arnold’s calculations in [1] and recalling that in our case γ₁ = γ₂ = 0, the expected value yields

$$\text{E}\left[S_P\right]=\mathrm{sgn}{\left(-f_i^{\prime }\right)}^{\left|\right|P_1\left|\right|}{\sqrt{{\kappa }_2}}^{\left|\right|P_1\left|\right|}\frac{\mu !}{(\mu -\nu )!}\sum _{n=0}^{\nu }\sum _{k\ge 0}{\zeta }_{n,0}^{(P)}(k)h_{\mu ,\lambda }^{\nu -n,k}.$$ (22)

Note that expression (22) deviates from Arnold’s formula only in the sign in front of $\sqrt{{\kappa }_2}$. The coefficients ${\zeta }_{n,0}^{(P)}(k)$ are defined in terms of a noise coefficient a according to

$$a=\sqrt{\frac{{\kappa }_2}{{\kappa }_2+{\sigma }_{ϵ}^2}}\text{ with }{\zeta }_{n,0}^{(P)}(k)=\text{Polynomial}(a),$$ (23)

for which tabulated results are presented in [1, p. 141]. The coefficients $h_{\mu ,\lambda }^{i,k}$ are numerically obtainable solving

$$h_{\mu ,\lambda }^{i,k}=\frac{\lambda -\mu }{\sqrt{2\pi }}\left(\begin{array}{l}\lambda \hfill \\ \mu \hfill \end{array}\right){\int }_{-\infty }^{\infty }\text{H}{\text{e}}_k(x){\text{e}}^{-\frac{1}{2}{x}^2}[\varphi (x){]}^i[\Phi (x){]}^{\lambda -\mu -1}[1-\Phi (x){]}^{\mu -i}\text{d}x.$$ (24)

Now we are in the position to calculate expectation (13) using (22). Since z ~ 𝒩(0,1), it holds κ₂ = 1. Identifying P = (1), ║P║₁ = 1 and ν = 1 yields

$$\begin{array}{l}\text{E}\left[\sum _{m=1}^{\mu }{z}_{m;\lambda }\right]=\mathrm{sgn}\left(-f_i^{\prime }\right)\frac{\mu !}{(\mu -1)!}\sum _{n=0}^1\sum _{k\ge 0}{\zeta }_{n,0}^{(1)}(k)h_{\mu ,\lambda }^{1-n,k}\hfill \\ =\mathrm{sgn}\left(-f_i^{\prime }\right)\mu {\zeta }_{0,0}^{(1)}(0)h_{\mu ,\lambda }^{1,0}=-\mathrm{sgn}\left(f_i^{\prime }\right)\mu a{c}_{\mu /\mu ,\lambda },\hfill \end{array}$$ (25)

with ${\zeta }_{1,0}^{(1)}(k)=0$ for any k, and ${\zeta }_{0,0}^{(1)}(k)\ne 0$ only for k = 0 yielding a. The expression $h_{\mu ,\lambda }^{1,0}$ is equivalent to the progress coefficient definition c_μ/μ,λ [2, p. 216]. Inserting (25) back into (13), using $a=\sqrt{1/(1+{(D_i/f_i^{\prime }\sigma )}^2)}=\left|f_i^{\prime }\right|\sigma /\sqrt{{(f_i^{\prime }\sigma )}^2+D_i^2}$ with the requirement a > 0, and noting that $f_i^{\prime }=\mathrm{sgn}\left(f_i^{\prime }\right)\left|f_i^{\prime }\right|$ one finally obtains for the i-th component first order progress rate

$${\phi }_i(\sigma ,y)=c_{\mu /\mu ,\lambda }\frac{f_i^{\prime }\left(y_i\right){\sigma }^2}{\sqrt{{\left(f_i^{\prime }\left(y_i\right)\sigma \right)}^2+D_i^2\left(\sigma ,{(y)}_{j\ne i}\right)}}.$$ (26)

The population dependency is given by progress coefficient c_μ/μ,λ. The fitness dependent parameters are contained in $f_i^{\prime }$, see (7), and in $D_i^2$ calculated in the supplementary material (https://github.com/omam-evo/paper/blob/main/ppsn22/PPSN22_OB22.pdf). For better readability the derivative $f_i^{\prime }$ and variance $D_i^2$ are not inserted into (26). An exemplary evaluation of $D_i^2$ as a function of the residual distance R using normalization (10) is also shown in the supplementary material.

4.3. Comparison of Simulation and Approximation

Figure 1 shows an experimentally obtained progress rate compared to the result of (26). Due to large N one exemplary φ_i-graph is shown on the left, and corresponding i = 1, …, N errors are shown on the right.

Fig. 1.

One-generation experiments with (150/150, 300)-ES, N = 100, A = 10 are performed and quantity (11) is measured averaging over 10⁵ runs. Left: φ_i over σ for i = 2 at position y₂ ≈ 1.19, where y was chosen randomly such that ║y║ = R = 10. Right: error measure φ_i − φ_i,sim between (26) and simulation for i = 1, …, N evaluated at σ = {0.1, 1}. The colors are set according to the legend. (Color figure online)

The left plot shows the progress rate over a σ-range of [0, 1]. This magnitude was chosen in order to study the oscillation, as the frequency α = 2π. The initial position was chosen randomly to be on the sphere surface R = 10.

The red dashed curve uses $f_i^{\prime }$ as linearization, while the blue dash-dotted curve assumes $f_i^{\prime }=k_i$ (with d_i = 0), see also (7). As $f_i^{\prime }$ approximates the quality change locally, agreement for the progress is given only for very small mutations σ. For larger σ very large deviation may occur, depending on the local derivative.

The blue curve φ_i(k_i) neglects the oscillation (d_i = 0) and therefore follows the progress of the quadratic function $f(y)={\displaystyle {\sum }_i{y}_i^2}$ for large σ with very good agreement. Due to a linearized form of Q_i(x_i) in (6) neither approximation can reproduce the oscillation for moderately large σ.

To verify the approximation quality, the error between (26) and simulation is displayed on the right side of Fig. 1 for all i = 1, …, N. It was done for small σ = 0.1 and large σ = 1. The deviations are very similar in magnitude for all i, given randomly chosen y_i. Note that for σ = 1 the red points show very large errors compared to blue, which was expected.

Figure 2 shows the progress rate φ_i over σ*, for i = 2 as in Fig. 1, with y randomly on the surface radii R = {100, 10, 1, 0.1}. Using σ* the mutation σ is normalized by the residual distance R with spherical normalization (10). Far from the origin with R = {100, 10} the quadratic terms are dominating giving better results using φ_i(k_i). Reaching R = 1 local minima are more relevant and mixed results are obtained with ${\phi }_i\left(f_i^{\prime }\right)$ better for smaller σ* and φ_i(k_i) for larger σ*. Within the global attractor R = 0.1 the local structure dominates and ${\phi }_i\left(f_i^{\prime }\right)$ yields better results. These observations will be relevant analyzing the dynamics in Fig. 3 where both approximations show strengths and weaknesses.

Fig. 2.

One-generation progress φ_i (i = 2) over normalized mutation σ* for (150/150, 300)-ES, N = 100, A = 1 and R = {100, 10, 1, 0.1}. Simulations are averaged over 10⁵ runs. These experiments are preliminary investigations related to the dynamics shown in Fig. 3 with σ* = 30. Given a constant σ* the approximation quality varies over different magnitudes of R.

Fig. 3.

Comparing average of 100 optimization runs of Algorithm 1 (black, solid) with iterated dynamics from Eq. (27) under constant σ* = 30 for A = 1 and N = 100. Large populations sizes are chosen to ensure global convergence (left: μ = 150; right: μ = 1500; constant μ/λ = 0.5). Iteration using progress (26) is performed for both $f_i^{\prime }=k_i+d_i$ (red/orange dashed) and $f_i^{\prime }\left(d_i=0\right)=k_i$ (blue dash-dotted) using Equations (27) and (28). The orange dashed iteration was initialized with R⁽⁰⁾ = 0.1 and translated to the corresponding position of the simulation for easier comparison. The evaluation of quality variance $D_i^2(R)$ is shown in the supplementary material (https://github.com/omam-evo/paper/blob/main/ppsn22/PPSN22_OB22.pdf). (Color figure online)

5. Evolution Dynamics

As we are interested in the dynamical behavior of the ES, averaged real optimization runs from Algorithm 1 will be compared to the iterated dynamics using progress result (26) by applying the dynamical systems approach [2]. Neglecting fluctuations, i.e., $y_i^{(g+1)}=\text{E}\left[y_i^{(g+1)}\mid {\sigma }^{(g)},y^{(g)}\right]$ the mean value dynamics for the mapping $y_i^{(g)}\to y_i^{(g+1)}$ immediately follows from (11) giving

$$y_i^{(g+1)}=y_i^{(g)}-{\phi }_i\left({\sigma }^{(g)},y^{(g)}\right).$$ (27)

The control scheme of σ^(g) was introduced in Eq. (10) and yields simply

$${\sigma }^{(g)}={\sigma }^{\ast }\left|\right|y^{(g)}\left|\right|/N.$$ (28)

Equations (27) and (28) describe a deterministic iteration in search space and rescaling of mutations according to the residual distance. For a convergence analysis, we are interested in the dynamics of R^(g) = ║y^(g)║ rather than the actual position values y^(g). Hence in Fig. 3 the R^(g)-dynamics of the conducted experiments is shown.

In Fig. 3, all runs of Algorithm 1 exhibit global convergence with the black line showing the average. The left and right plots differ by population size. Iteration φ_i(k_i), blue dash-dotted curve, also converges globally, though very slowly and therefore not shown entirely. The convergence behavior of iteration ${\phi }_i\left(f_i^{\prime }\right)$, red and orange dashed curves, strongly depends on the initialization and is discussed below.

Three phases can be observed for the simulation. It shows linear convergence at first being followed by a slow-down due to local attractors. Reaching the global attractor the convergence speed increases again. Iteration φ_i(k_i) is able to model the first two phases to some degree. Within the global attractor the slope information d_i is missing such that the progress is largely underestimated.

Iteration ${\phi }_i\left(f_i^{\prime }\right)$ converges first, but yields a stationary state with R^st ≈ 20 when the progress φ_i becomes dominated by derivative term d_i. Starting from R⁽⁰⁾ = 10² the stationary $y_i^{st}$ are either fixed or alternating between coordinates depending on σ, D_i, k_i, and d_i. This effect is due to attraction of local minima and due to the deterministic iteration disregarding fluctuations. It occurs also with varying initial positions. Initialized at R⁽⁰⁾ = 10⁻¹ orange iteration ${\phi }_i\left(f_i^{\prime }\right)$ is globally converging.

It turns out that the splitting point of the two approximations in Fig. 3 occurs at a distance R to the global optimizer where the ES approaches the attractor region of the “first” local minima. For the model parameters considered in the experiment this is at about R ≈ 28.2 − the distance of the farest local minimizer to the global optimizer (obtained by numerical analysis).

Plots in Fig. 3 differ by population size. The convergence speed, i.e. the slopes, show better agreement for large populations, which can be attributed to the fluctuations neglected in (27). Investigations on unimodal funtions Sphere [2] and Ellipsoid [4] have shown that progress is decreased by fluctuations due to a loss-term scaling with 1/μ, which agrees with Fig. 3. On the left the iterated progress is faster due to neglected but present fluctuations, while on the right better agreement is observed due to insignificant fluctuations. These observations will be investigated in future research.

6. Summary and Outlook

A first order progress rate φ_i was derived for the (μ/μ_I, λ)-ES by means of noisy order statistics in (26) on the Rastrigin function (1). To this end, the mutation induced variance of the quality change $D_i^2$ is needed. Starting from (4) a derivation yielding $D_i^2$ has been presented in the supplementary material. Furthermore, the approximation quality of φ_i was investigated using Rastrigin and quadratic derivatives $f_i^{\prime }$ and k_i, respectively, by comparing with one-generation experiments.

Linearization $f_i^{\prime }$ shows good agreement for small-scale mutations, but very large deviations for large mutations. Conversely, linearization k_i yields significantly better results for large mutations as the quadratic fitness term dominates. A progress rate modeling the transition between the regimes is yet to be determined. First numerical investigations of (14) including all terms of (4) indicate that nonlinear terms are needed for a better progress rate model, which is an open challenge and part of future research.

The obtained progress rate was used to investigate the dynamics by iterating (27) using (28) and comparing with ES runs. Iteration via $f_i^{\prime }$ only converges globally if initialized close to the optimizer, since local attraction is strongly dominating. Dynamics via k_i converges globally independent of initialization, but the observed rate matches only for the initial phase and for very large populations. This confirms the need for a higher order progress rate modeling the effect of fluctuations, especially when function evaluations are expensive and small populations must be used. Additionally, an advanced progress rate formula is needed combining effects of global and local attraction to model all three phases of the dynamics correctly.

The investigations done so far are a first step towards a full dynamical analysis of the ES on the multimodal Rastrigin function. Future investigations must also include the complete dynamical modeling of the mutation strength control. One aim is the tuning of mutation control parameters such that the global convergence probability is increased while still maintaining search efficiency. Our final goal will be the theoretical analysis of the full evolutionary process yielding also recommendations regarding the choice of the minimal population size needed to converge to the global optimizer with high probability.