Convergence Properties of the (μ/μ_I, λ)-ES on the Rastrigin Function

Abstract

The highly multimodal Rastrigin test function is analyzed by deriving a new aggregated progress rate measure. It is derived as a function of the residual distance to the optimizer by assuming normally distributed positional coordinates around the global optimizer. This assumption is justified for successful ES-runs operating with sufficiently slow step-size adaptation. The measure enables the investigation of further convergence properties. For moderately large mutation strengths a characteristic distance-dependent Rastrigin noise floor is derived. For small mutation strengths local attraction is analyzed and an escape condition is established. Both mutation strength regimes combined pose a major challenge optimizing the Rastrigin function, which can be counteracted by increasing the population size. Hence, a population scaling relation to achieve high global convergence rates is derived which shows good agreement with experimental data.

1. Introduction

Evolution Strategies (ES) have proven to be well suited for the optimization of highly multimodal real-valued fitness functions due to their underlying stochastic nature. Test functions such as the Rastrigin function contain a huge number of local minima scaling exponentially with the search space dimensionality N. Sampling the search space and applying multiple restarts with standard gradient-based optimization algorithms quickly becomes unfeasible. On the other side, ES achieve high success rates for global convergence on certain multimodal functions, if sufficiently large population sizes are chosen. Experimental investigations in [9] indicate a non-exponential population scaling at most of O (N²) for the tested multimodal functions with the Rastrigin function scaling sub-linearly in N. However, there is little understanding of how the ES is exploring the fitness landscape to find the global optimizer without getting trapped in one of the local minima. This paper investigates the convergence properties on the Rastrigin function based on progress rate theory results of [11] and [12]. To this end, a new aggregated progress rate is introduced modeling the progress as a function of the residual distance to the optimizer. The obtained results are compared to real ES-runs operating with isotropic mutations and self-adaptation for step-size control.

In Sec. 2 the ES under investigation is introduced. In Sec. 3 the Rastrigin function is defined and averaging methods are discussed. Then, the method is applied to component-wise progress rate equations in Sec. 4 obtaining an aggregated measure. In Sec. 5 a population sizing relation is derived and compared to experimental results. Local attraction is discussed in Sec. 6 and a characteristic “escape” mutation strength is derived. Finally, in Sec. 7 conclusions and an outlook are provided.

2. The ES-Algorithm

The ES under investigation, see Algorithm 1, consists of μ parents and λ offspring with truncation ratio ϑ = μ/λ (1 ≤ μ < λ). Selection of the m = 1, …, μ best individuals (out of λ) is denoted by subscript "m; λ". Normally distributed isotropic mutations of strength σ are applied. Intermediate multi-recombination with equal weights is used to obtain the parental location ${\displaystyle \mathbf{\text{y}}}(g)=\left[y_1^{(g)},\dots ,y_N^{(g)}\right]$ in the N-dimensional search space for each generation g. For the σ-adaptation (self-adaptation) the offspring mutation strengths are chosen from a log-normal distribution with learning parameter τ. A smaller τ-value therefore yields a slower σ-adaptation, which will be important later. The default choice is $\tau =1/\sqrt{2N}$, see [10], which ensures optimal performance on the sphere in the limit N → ∞. Recombination also applies to selected σ-values.

3. Rastrigin Function and Averaging

The Rastrigin test function f is defined for a real-valued search vector y = [y₁, …, y_N] as

$$f(\mathbf{\text{y}})={\displaystyle \sum _{i=1}^N\left[y_i^2+A\left(1-\text{cos}\left(\alpha y_i\right)\right)\right]},$$ (1)

Algorithm 1. (μ/μ_I, λ)-σSA-ES.

1: g ← 0

2: initialize (y⁽⁰⁾, σ⁽⁰⁾)

3: repeat

4:     for l = 1, …, λ do

5:         ${\tilde{\sigma }}_l\leftarrow {\sigma }^{(g)}{\text{e}}^{\tau {\mathcal{\text{N}}}_l(\mathbf{0},\mathbf{1})}$

6:         ${{\displaystyle \widetilde{\mathbf{\text{x}}}}}_l\leftarrow {\tilde{\sigma }}_l{\mathcal{\text{N}}}_l({\displaystyle \mathbf{0}},{\displaystyle \mathbf{1}})$

7:         ${{\displaystyle \widetilde{\mathbf{\text{y}}}}}_l\leftarrow {{\displaystyle \mathbf{\text{y}}}}^{(g)}+{{\displaystyle \widetilde{\mathbf{\text{x}}}}}_l$

8:         ${\tilde{f}}_l\leftarrow f\left({{\displaystyle \widetilde{\mathbf{\text{y}}}}}_l\right)$

9:     end for

10:     $\left({\tilde{f}}_{1;\lambda },\dots ,{\tilde{f}}_{m;\lambda },\dots ,{\tilde{f}}_{\mu ;\lambda }\right)\leftarrow \text{sort}\left({\tilde{f}}_1,\dots ,{\tilde{f}}_{\lambda }\right)$

11:     ${{\displaystyle \mathbf{\text{y}}}}^{(g+1)}\leftarrow \frac{1}{\mu }{\displaystyle {\sum }_{m=1}^{\mu }{{\displaystyle \widetilde{\mathbf{\text{y}}}}}_{m;\lambda }}$

12:     ${\sigma }^{(g+1)}\leftarrow \frac{1}{\mu }{\displaystyle {\sum }_{m=1}^{\mu }{\tilde{\sigma }}_{m;\lambda }}$

13:     g ← g +1

14: until termination criterion

with oscillation amplitude A and frequency α. The function is minimized and the global minimizer is located at ŷ = 0. All simulations in this paper will be conducted at default α = 2π (unless stated otherwise). Depending on A, the function shows a finite number M of local attractors for each of the N dimensions, such that the function contains M^N − 1 local minima scaling exponentially with N. Note that for |y_i| > αA/2 the derivative $\partial f/\partial y_i\ne 0$ (for any i), such that no further local minima occur.

As an introductory example, real optimization runs are shown in Fig. 1 using Algorithm 1. The global convergence is investigated by evaluating the dynamics of the parental distance to the global optimizer denoted by R^(g) = ‖y^(g)‖. All runs are initialized far away from the local attractor landscape. The black line shows the median of all successful runs (the mean can also be used, but the median is more robust w.r.t. outliers). The global convergence probability is denoted as P_S. After the initial phase, the σSA-ES maintains a nearly constant normalized mutation strength σ* ≈ 30 with a characteristic dip at g ≈ 200. σ* is defined as

$${\sigma }^{\ast }=\frac{\sigma N}{R}.$$ (2)

Figure 1.

Dynamic runs (500 trials) of Algorithm 1 using (100/100_I, 200)-σSA-ES on the Rastrigin function with N = 100 and A = 1 at learning parameter $\tau =1/\sqrt{2N}$. The upper plot shows the R-dynamics, while the lower plot shows the normalized mutation strength (2). The green-yellow color palette depicts globally converging runs, while the cyan-magenta colors show locally converging runs. The black line marks the median of all successful runs. The measured success probability P_S = 0.91.

A constant σ*-level ensures scale-invariance on the sphere function and therefore linear convergence. The observations from Fig. 1 will be investigated in more detail throughout the paper.

The Rastrigin fitness (1) is defined as a function of y = [y₁, …, y_N]. Convergence however is usually measured as a function of the residual distance R, see Fig. 1. Quantity R is therefore an aggregated measure over all components. In general, the convergence properties can be investigated using progress rate equations (see Sec. 4). A component-wise progress rate however, derived and discussed in [11], has the disadvantage of not being an aggregated measure. As an example, positive progress (convergence) between two generations occurs for decreasing R^(g+1) < R^(g) even if some components deteriorate and show negative progress. Furthermore, analytic treatment of N equations is unfeasible for large dimensionalities. Hence, the idea will be to express y-dependent functions as average values over all positions satisfying ∥y∥ = R to obtain aggregated measures. First, the approach is presented on the Rastrigin function and later transferred to its corresponding progress rate in Sec. 4.

The averaging problem to be solved is

$$\overline{f}(R):=\underset{\parallel {\displaystyle \mathbf{\text{y}}}\parallel =R}{\text{average}}[f({\displaystyle \mathbf{\text{y}}})].$$ (3)

An approach to obtain $\overline{f}(R)$ is to integrate the function over the (N − 1)-dimensional sphere surface S_N (R) with radius ∥y∥ = R and normalize by the sphere surface according to

$$\overline{f}(R)=\frac{1}{S_N}{\displaystyle {\displaystyle {\int }_{\parallel {\displaystyle \mathbf{\text{y}}}\parallel =R}f({\displaystyle \mathbf{\text{y}}})\text{d}s,}}$$ (4)

where ds denotes the (hyper-)surface element. The sphere surface area for N ≥ 2 evaluated using the gamma function Γ is given by

$$S_N(R)=\frac{2{\pi }^{N/2}{R}^{N-1}}{\text{Γ}(N/2)}.$$ (5)

Applying (4) to (1), the first two terms can be evaluated easily noting that $R^2={\displaystyle {\displaystyle {\sum }_i{y}_i^2}}$. Integrating over a constant yields S_N. Therefore, one gets the intermediate result

$$\overline{f}(R)=R^2+NA+T(R),$$ (6)

with

$$T(R):=-\frac{A}{S_N}{\displaystyle {\displaystyle {\int }_{\parallel \mathbf{\text{y}}\parallel =R}{\displaystyle \sum _{i=1}^N\text{cos}\left(\alpha y_i\right)\text{d}s}}}.$$ (7)

Closed-form solutions of (7) can be obtained for N = 1 and N = 2. Starting with N = 1 only two discrete points are relevant (no integration necessary) with two possible solutions y₁ = ±R. Averaging over two points therefore yields

$$T(R)=-\frac{A}{2}{\displaystyle \sum }_{y_1=\pm R}\text{cos}\left(\alpha y_1\right)=-A\text{cos}(\alpha R).$$ (8)

For N = 2 one can use polar coordinates (y₁, y₂) = (R cos ϕ, R sin ϕ) with derivative vector $\frac{\text{d}\left(y_1,y_2\right)}{\text{d}\varphi }=(-R\text{sin}\varphi ,R\text{cos}\varphi )$ on ϕ ∈ [0, 2π). Additionally, one has S₂ = 2πR. Therefore, inserting this parametrization into (7) and using path element length $\Vert \frac{\text{d}\left(y_1,y_2\right)}{\text{d}\varphi }\Vert =R$ one has

$$\begin{array}{l}T\left(R\right)=-\frac{A}{2\pi R}{\displaystyle {\int }_0^{2\pi }{\displaystyle \sum _{i=1}^2\text{cos}}\left[\alpha y_i\left(R,\varphi \right)\right]\Vert \frac{\text{d}\left(y_1,y_2\right)}{d\varphi }\Vert }\text{d}\varphi \\ =-\frac{A}{2\pi }{\displaystyle {\int }_0^{2\pi }\left[\text{cos}\left(\alpha R\text{cos}\varphi \right)+\text{cos}\left(\alpha R\text{sin}\varphi \right)\right]}\text{d}\varphi .\end{array}$$ (9)

The integrals obtained in (9) can be solved in terms of Bessel functions of the first kind J_n (x) with n ≥ 0 by applying the integral identity [1, p. 360, 9.1.18]

$$J_0(x)=\frac{1}{\pi }{{\displaystyle \int }}_0^{\pi }\text{cos}(x\text{sin}t)\text{d}t=\frac{1}{\pi }{{\displaystyle \int }}_0^{\pi }\text{cos}(x\text{cos}t)\text{d}t.$$ (10)

Due to the periodicity integrating cos t and sin t over [0, π] yields the same contribution as the integration over [π, 2π]. Thus, one can extend the integral bounds of (10) as

$$2J_0(x)=\frac{1}{\pi }{{\displaystyle \int }}_0^{2\pi }\text{cos}(x\text{sin}t)\text{d}t=\frac{1}{\pi }{{\displaystyle \int }}_0^{2\pi }\text{cos}(x\text{cos}t)\text{d}t.$$ (11)

Comparing (9) with (11) and setting x = αR, the expression (9) is evaluated as

$$T(R)=-\frac{A}{2\pi }\left[2\pi J_0(\alpha R)+2\pi J_0(\alpha R)\right]=-2A{J}_0(\alpha R).$$ (12)

The final result for f (R) is summarized as

$$\begin{array}{l}\overline{f}(R)=R^2+A(1-\text{cos}(\alpha R))\text{for}N=1\end{array}$$ (13)

$$\overline{f}(R)=R^2+2A\left(1-J_0(\alpha R)\right)\text{for}N=2.$$ (14)

Examples of result (13) and (14) are shown in Fig. 2. The analytic equations are compared to sampled results, where for each R random isotropic positions are chosen with ∥y∥ = R and averaged over 10⁴ trials. Excellent agreement can be observed. Furthermore, one notices a decrease of the oscillation effect when N is increased. This will be useful for the subsequent approach. Unfortunately, integral (4) yields analytically exact results only for the cases N < 3. In the context of progress rate theory of Sec. 4 an approach is needed which can be applied to any arbitrary large dimensionality N.

Figure 2.

Average Rastrigin function evaluated for A = 10 as a function of R. The solid black lines show sampled results using Eq. (1) for N = 1 and N = 2, respectively. The overlaid dotted green line shows (13) and the dashed cyan line (14).

The approach presented now will evaluate the average value of f(y) assuming independent, normally distributed random coordinates y_i with zero mean and variance ${\sigma }_y^2$ according to

$$y_i~{\sigma }_y\mathcal{\text{N}}(0,1).$$ (15)

The approach can be considered as averaging by stochastic sampling (assuming large N) instead of analytic integration. For the determination of σ_y from (15) one demands

$$R^2\stackrel{!}{=}\text{E}\left[{\displaystyle {\displaystyle \sum _{i=1}^N{y}_i^2}}\right]={\sigma }_y^2\text{E}\left[{\displaystyle {\displaystyle \sum _{i=1}^N{\mathcal{\text{N}}}_i^2}}(0,1)\right]={\sigma }_y^2\text{E}\left[{\chi }_N^2\right]={\sigma }_y^{2}N.$$ (16)

It was used that the sum over N independent standard normally distributed variables squared is equal to the chi-squared distributed variable ${\chi }_N^2$ with $\text{E}\left[{\chi }_N^2\right]=N$. Solving (16) for σ_y, expression (15) can be rewritten as

$$y_i~\frac{R}{\sqrt{N}}\mathcal{\text{N}}(0,1).$$ (17)

Equation (17) will be useful for averaging sums over trigonometric functions of y_i, where analytic integration is unfeasible. Furthermore, successful ES runs on the Rastrigin function operating under default step-size adaptation also show normally distributed y_i as in (17), see also experiments in Fig. 4. This property is used again in Sec. 4. As y_i is treated as a random variate for the cosine terms, Eq. (1) is now rewritten as

$$f(R,Y)~R^2+NA-AY,$$ (18)

with Y containing the sum over the random terms

$$Y:={\displaystyle {\displaystyle \sum _{i=1}^N\text{cos}\left(\alpha y_i\right)}}.$$ (19)

By the Central Limit Theorem in the limit N → ∞, the sum over i.i.d. variates approaches a normal distribution with $Y~\text{E}[Y]+\sqrt{\text{Var}[Y]}\mathcal{\text{N}}(0,1)$. Additionally, it is shown in Appendix (A.7) that

$$\frac{\sqrt{\text{Var}[Y]}}{\text{E}[Y]}\stackrel{N\to \infty }{\to }0,$$ (20)

with ratio $\sqrt{\text{Var}[Y]}/\text{E}[Y]$ vanishing as $O(1/\sqrt{N})$. In the asymptotic limit the fluctuation term of Y is negligible, which means that the random variate can be replaced by its expected value $\text{E}[Y]=N{\text{e}}^{-\frac{1}{2}\frac{{(\alpha R)}^2}{N}}$ evaluated in Appendix (A.1). Equation (18) therefore yields (overline denoting the average in the limit N → ∞)

$$\overline{f}(R)=R^2+NA-A\text{E}[Y]$$ (21)

$$=R^2+NA\left(1-{\text{e}}^{-\frac{1}{2}\frac{{(\alpha R)}^2}{N}}\right).$$ (22)

Exemplary evaluations of (22) are shown in Fig 3. The derived results match the sampled results well. Smaller deviations are observed for small values N = 3 or N = 10, which was expected. In the limit N → ∞ the deviations are smoothed out. The limits R → 0 and R → ∞ yield R²-dependent functions, i.e., sphere functions. For R → ∞ the exponential vanishes and NA is negligible, while for R → 0 one has ${\text{e}}^{-\frac{1}{2}\frac{{(\alpha R)}^2}{N}}=1-\frac{1}{2}\frac{{(\alpha R)}^2}{N}+O\left(R^4\right)$ giving $f(R)=\left(1+\frac{{\alpha }^{2}A}{2}\right)R^2$.

Figure 3.

Average Rastrigin function evaluated for A = 10 as a function of R. The solid black lines show sampled results of Eq. (1). The overlaid dotted green lines show Eq. (22) for N = {3, 10, 100, 1000}, from bottom to top.

Figure 4.

Distribution of realized y_i -values (i = 1, …, N over 500 trials) for (100/100_I, 200)-ES, N = 100, A = 1, and $\tau =1/\sqrt{2N}$ at R ≈ 100 (left) and R ≈ 1 (right). The red solid curve shows the density of the normal distribution with y_i ~ 𝒩(0, R²/N).

4. Progress Rate

4.1. Derivation

The method introduced in Sec. 3 will now be applied to results for the progress rate on the Rastrigin function. The progress rate (denoted by φ) measures the expected positional change in search space between two generations g → g+1 as a function of fitness and ES parameters. A positive value corresponds to the ES approaching the optimizer and vice versa. The second order component-wise progress rate ${\phi }_i^{\text{II}}$ for the parental location y^(g) is defined as [8]

$${\phi }_i^{\text{II}}:=\text{E}\left[{\left(y_i^{(g)}\right)}^2-{\left(y_i^{(g+1)}\right)}^2\mid {{\displaystyle \text{y}}}^{(g)},{\sigma }^{(g)}\right].$$ (23)

The second order refers to the square of y_i -values which ensures ${\phi }_i^{\text{II}}>0$ for ${\left(y_i^{(g+1)}\right)}^2<{\left(y_i^{(g)}\right)}^2$ independent of the sign of y_i. A second order model is needed for a correct model of convergence involving large mutation strengths. The expected values for the determination of (23) were already evaluated in [11] in the asymptotic limit N, µ, λ → ∞ (ϑ = μ/λ = const). The result yields

$${\phi }_i^{\text{II}}=c_{\vartheta }\frac{{\sigma }^2}{D_Q}\left(4y_i^2+{\text{e}}^{-\frac{1}{2}{(\alpha \sigma )}^2}2\alpha A{y}_i\text{sin}\left(\alpha y_i\right)\right)-\frac{{\sigma }^2}{\mu }.$$ (24)

In (24) the asymptotic progress coefficient [11] is given by

$$c_{\vartheta }=\frac{{\text{e}}^{-\frac{1}{2}{\left[{\text{Φ}}^{-1}(\vartheta )\right]}^2}}{\sqrt{2\pi }\vartheta },$$ (25)

with Φ ⁻¹ (·) denoting the quantile function of the standard normal variate. The c_ϑ is related to the progress coefficient c_μ/μ,λ ≃ c_ϑ (for µ, λ→ ∞ with constant ϑ = μ/λ), see also [6, p. 249]. The quality gain variance $D_Q^2$ at location y given σ was evaluated in [12] giving

$$\begin{array}{l}{D}_Q^2({\displaystyle \mathbf{\text{y}}})={\displaystyle {\displaystyle \sum _{i=1}^N\{4{\sigma }^2{y}_i^2+2{\sigma }^4}}\\ +\frac{A^2}{2}\left[1-{\text{e}}^{-{(\alpha \sigma )}^2}\right]\left[1-\text{cos}\left(2\alpha y_i\right){\text{e}}^{-{(\alpha \sigma )}^2}\right]\\ +2A\alpha {\sigma }^2{\text{e}}^{-\frac{1}{2}{(\alpha \sigma )}^2}\left[\alpha {\sigma }^2\text{cos}\left(\alpha y_i\right)+2y_i\text{sin}\left(\alpha y_i\right)\right]\}.\end{array}$$ (26)

The first term of (24) is usually referred to as the gain term, while the second term is the loss term characteristic for intermediate recombination. A distinct property of the Rastrigin function is that the gain term (y_i -dependent) is not necessarily positive as it is the case for unimodal functions. This property will be discussed later.

The first step to obtain an R-dependent aggregation of expression (23) is to sum over all N components

$$\begin{array}{c}{\displaystyle {\displaystyle \sum _{i=1}^N{\phi }_i^{\text{II}}=\text{E}}}\left[{\displaystyle {\displaystyle \sum _{i=1}^N{\left(y_i^{(g)}\right)}^2}}-{\displaystyle {\displaystyle \sum _{i=1}^N{\left(y_i^{(g+1)}\right)}^2}}\right]\\ =\text{E}\left[{\left(R^{(g)}\right)}^2-{\left(R^{(g+1)}\right)}^2\right],\end{array}$$ (27)

such that one can define the R-dependent progress rate

$${\phi }_R^{\text{II}}:=\text{E}\left[{\left(R^{(g)}\right)}^2-{\left(R^{(g+1)}\right)}^2\mid R^{(g)},{\sigma }^{(g)}\right].$$ (28)

Given the sphere function f_sph (R) = R², one can relate (28) to the sphere quality gain $\text{E}\left[f_{\text{sph}}\left(R^{(g+1)}\right)-f_{\text{sph}}\left(R^{(g)}\right)\right]=-{\phi }_R^{\text{II}}$, such that the quality gain normalization [5, p. 173] is applicable. This yields the normalized R-dependent progress rate (labeled by the asterisk ^“*”)

$${\phi }_R^{\text{II},\ast }:=\frac{N}{2R^2}{\phi }_R^{\text{II}}.$$ (29)

For N → ∞ one has ${\phi }_R^{\text{Π},\ast }\simeq {\phi }_{\text{sph}}^{\ast }$, see [2, p. 16], yielding the normalized sphere progress rate. Expression (29) has two important properties. First, it is an aggregated progress rate measure over all N components, which is new for the Rastrigin function. Second, its relation to the sphere function enables direct comparison of progress rates.

A prerequisite for the further derivation will be the assumption of normally distributed y_i ~ 𝒩 (0, R²/N)N(…), see (17). This property is experimentally confirmed in Fig. 4, using the data of 500 trials shown in Fig. 1, displayed at two residual distances. Good agreement is observed between the expected density (red curve) and the histogram. Each component contributes roughly as $y_i^2\approx R^2/N$ to the overall residual distance R². This concept of “equal contribution” is not new and was investigated in [7] for the quality gain on the ellipsoid. Slightly larger deviations occur at R ≈ 1 (right), where local attraction is more significant, see also later discussion of Fig. 12. At small mutation strengths where local attraction occurs the assumption of course breaks down.

Now ${\phi }_R^{\text{II}}$ is derived starting from (24). Performing the summation one gets

$${\phi }_R^{\text{II}}(R,{\displaystyle \mathbf{\text{y}}})=c_{\vartheta }\frac{{\sigma }^2}{D_Q}\left(4R^2+{\text{e}}^{-\frac{1}{2}{(\alpha \sigma )}^2}2\alpha A{\displaystyle {\displaystyle \sum _{i=1}^N{y}_i\text{sin}\left(\alpha y_i\right)}}\right)-N\frac{{\sigma }^2}{\mu }.$$ (30)

Similarly, the summation of the variance terms in (26) yields

$$\begin{array}{c}\begin{array}{l}{D}_Q^2(R,{\displaystyle \mathbf{\text{y}}})=4{\sigma }^2{R}^2+2N{\sigma }^4\\ +\frac{A^2}{2}\left[1-{\text{e}}^{-{(\alpha \sigma )}^2}\right]{\displaystyle {\displaystyle \sum _{i=1}^N\left[1-\text{cos}\left(2\alpha y_i\right){\text{e}}^{-{(\alpha \sigma )}^2}\right]}}\end{array}\\ +2A\alpha {\sigma }^2{\text{e}}^{-\frac{1}{2}{(\alpha \sigma )}^2}\left[\alpha {\sigma }^2{\displaystyle {\displaystyle \sum _{i=1}^N\text{cos}}}\left(\alpha y_i\right)+2{\displaystyle {\displaystyle \sum _{i=1}^N{y}_i}}\text{sin}\left(\alpha y_i\right)\right].\end{array}$$ (31)

Analogous to (19) and (20), the sums over the y_i -dependent trigonometric terms of (30) and (31) will be replaced by their respective expectation values assuming $y_i~\frac{R}{\sqrt{N}}\mathcal{\text{N}}(0,1)$ and neglecting fluctuations for N → ∞. The needed expected values are derived in Appendix (A.1), (A.2), and (A.3) giving

$$\text{E}\left[{\displaystyle {\displaystyle \sum _{i=1}^N\text{cos}\left(\alpha y_i\right)}}\right]=N{\text{e}}^{-\frac{1}{2}\frac{{(\alpha R)}^2}{N}}$$ (32)

$$\text{E}\left[{\displaystyle {\displaystyle \sum _{i=1}^N\text{cos}\left(2\alpha y_i\right)}}\right]=N{\text{e}}^{-2\frac{{(\alpha R)}^2}{N}}$$ (33)

$$\text{E}\left[{\displaystyle {\displaystyle \sum _{i=1}^N{y}_i\text{sin}\left(\alpha y_i\right)}}\right]=\alpha R^2{\text{e}}^{-\frac{1}{2}\frac{{(\alpha R)}^2}{N}}.$$ (34)

Furthermore, it is shown in Appendix A that $\sqrt{\text{Var}\left[{\displaystyle {\displaystyle {\sum }_i(\cdot )}}\right]}/\text{E}\left[{\displaystyle {\displaystyle {\sum }_i(\cdot )}}\right]\to 0$ for N → ∞ for all three sums. Finally, a fully R-dependent expression can be given for the progress rate

$${\phi }_R^{\text{II}}=c_{\vartheta }\frac{2R^2{\sigma }^2}{D_Q(R)}\left(2+{\alpha }^{2}A{\text{e}}^{-\frac{{\alpha }^2}{2}\left({\sigma }^2+\frac{R^2}{N}\right)}\right)-N\frac{{\sigma }^2}{\mu }.$$ (35)

Analogously, the R-dependent quality gain variance yields

$$\begin{array}{c}{D}_Q^2(R)=4R^2{\sigma }^2+2N{\sigma }^4+\\ +\frac{N{A}^2}{2}\left[1-{\text{e}}^{-{(\alpha \sigma )}^2}\right]\left[1-{\text{e}}^{-{\alpha }^2\left({\sigma }^2+2\frac{R^2}{N}\right)}\right]\\ +2NA{\alpha }^2{\sigma }^2{\text{e}}^{-\frac{{\alpha }^2}{2}\left({\sigma }^2+\frac{R^2}{N}\right)}\left[{\sigma }^2+2\frac{R^2}{N}\right].\end{array}$$ (36)

Result (35) is important as it measures the progress on the Rastrigin function in R-space aggregating the individual progress rates ${\phi }_i^{\text{II}}$. Note that the first term of (35), i.e., the gain term, is now strictly positive, which is in contrast to Eq. (24).

One-generation experiments are conducted in Fig. 5 by performing single optimization steps for given mutation strength and averaging the results of progress rates (23) and (28), respectively, over 10⁴ trials. Furthermore, the simulations are compared to analytic expressions (24) and (35). To this end, two configurations (constant R = 7) are overlaid with one having y fixed, and one randomly sampled y-values with ∥y∥ = R. The values are normalized using (29) and displayed using scale-invariant mutations σ* of Eq. (2). All results are similar for moderate and large σ*-values showing good agreement. Differences emerge at small σ*. The fixed y = [0.7, …, 0.7] was chosen to lie within a local attractor. In this Case ${\displaystyle {\displaystyle {\sum }_i{\phi }_i^{\text{II}}(\mathbf{\text{y}})}}$ correctly predicts negative progress for small σ*, while ${\phi }_R^{\text{II}}$ falsely assumes normally distributed y_i-coordinates and predicts positive progress. This error vanishes for large σ*, i.e., when the ES is searching at larger scales. Therefore, one can conclude that ${\phi }_R^{\text{II}}$ is a suitable aggregated measure of component-wise ${\phi }_i^{\text{II}}$, if sufficiently large mutations are applied. Indeed, real (successful) ES-runs, such as in Fig. 1 or later in Fig. 6, tend to maintain high σ*-levels, such that the normal assumption for y_i stays valid. Local attraction (assuming small σ*) is investigated further in Sec. 6.

Figure 5.

One generation experiments with 10⁴ repetitions for (100 /100_I, 200)-ES, N = 100, A = 1 at constant R = 7. Black circles show experimentally evaluated (23), summed over i, for constant y = [0.7, …, 0.7]. Blue crosses show (28), where y is randomly sampled for each trial, such that ∥y∥ = R. The green dash-dotted line shows (24) (summed over i) and the red dashed line (35). All values are normalized using (29). The error bars are vanishing and not shown.

Figure 6.

Progress rate ${\phi }_R^{\text{II},\ast }$ for (100 /100_I, 200)-ES with N = 100 and A = 1. High progress rate values are shown in yellow and blue values indicate small (negative) progress. The boundary ${\phi }_R^{\text{II},\ast }=0$ is shown in bold white. The black (left) curve is displaying the median dynamics of Fig. 1 ($\tau =1/\sqrt{2N,}$ P_S = 0.91), while the red (right) curve is showing the same ES with $\tau =1/\sqrt{8N}$ and P_S = 0.99.

A few important remarks regarding results (35) and (36) are made now. Given expression (36), the variance can be written more compactly as

$$D_Q^2=D_{\text{sph}}^2+D_{\text{Ras}}^2,$$ (37)

where $D_{\text{sph}}^2=4R^2{\sigma }^2+2N{\sigma }^4$ corresponds to the quality gain variance of the sphere function [6]. The term $D_{\text{Ras}}^2=D_Q^2(R)-D_{\text{sph}}^2$ is Rastrigin-specific. In the limit of vanishing exponential functions (R → ∞), see later in Sec. 5, the term will simplify significantly giving the so-called Rastrigin (maximum) noise strength

$$D_{\text{Ras}}^2\simeq \frac{N{A}^2}{2}=:{\sigma }_{\text{Ras}}^2.$$ (38)

Having derived (35) and (36), the sphere progress rate ${\phi }_{\text{sph}}^{\ast }$ can be recovered as a special case. It can be obtained from ${\phi }_R^{\text{II}}$ in multiple ways. The technical details are not shown since the calculations are simple and straightforward, only the main steps are explained now. As the normalized progress is constant on the sphere for constant scale-invariant mutations σ*, Eqs. (35) and (36) need to be rewritten as ${\phi }_R^{\text{II}}\left({\sigma }^{\ast }\right)$ and D_Q (σ*) by setting σ = σ*R/N via (2). Furthermore, normalization (29) needs to be applied. One way to recover ${\phi }_{\text{sph}}^{\ast }$ is by setting A = 0 or α = 0, which removes all Rastrigin-specific terms. Another way is applying the limit R → ∞, which suppresses the exponential terms. Additionally, the constant term NA²/2 is negligible in (36) for R → ∞, see also Appendix (B.5). The third way is the limit R → 0. All exponentials contain arguments being a function g(R²) after inserting σ = σ*R/N. Performing a Taylor expansion yields ${\text{e}}^{-g\left(R^2\right)}=1-g\left(R^2\right)+O\left(R^4\right)$ with negligible higher order terms. After simplification, all three approaches yield

$${\phi }_R^{\text{II},\ast }={\phi }_{\text{sph}}^{\ast }=\frac{c_{\vartheta }{\sigma }^{\ast }}{\sqrt{1+{\sigma }^{\ast 2}/2N}}-\frac{{\sigma }^{\ast 2}}{2\mu },$$ (39)

and for N → ∞ the well-known asymptotic formula

$${\phi }_{\text{sph}}^{\ast }=c_{\vartheta }{\sigma }^{\ast }-\frac{{\sigma }^{\ast 2}}{2\mu }.$$ (40)

Both (39) and (40) are scale-invariant (R-independent) expressions. As a conclusion, the Rastrigin progress rate yields the sphere progress rate in the limits R → ∞ and R → 0. This result is important and was expected from (1), as $y_i^2$ is dominating at large scales. For y_i → 0 the global attractor is essentially a quadratic function.

An important property of ${\phi }_{\text{sph}}^{\ast }$ is that for sufficiently small σ* one has ${\phi }_{\text{sph}}^{\ast }>0$, while for too large σ*-values the progress rate becomes negative. The second (non-trivial) zero of (39), denoted by ${\sigma }_{{\phi }_0}^{\ast }$, is derived in Appendix B by setting ${\phi }_{\text{sph}}^{\ast }=0$ and yields in (B.8)

$${\sigma }_{{\phi }_0}^{\ast }={\left[{\left(N^2+8N{c}_{\vartheta }^2{\mu }^2\right)}^{1/2}-N\right]}^{1/2}.$$ (41)

Due to the same global (quadratic) structure, result (41) will also be applicable to the Rastrigin function as an upper bound for σ*.

4.2. Progress Landscape

A more detailed analysis of the progress rate (35) is provided now. Given fitness parameters A, α, and N, the expression ${\phi }_R^{\text{II},\ast }\left({\sigma }^{\ast },R\right)$ is essentially a function of only two variables. Therefore, the results will be displayed in a two-dimensional σ*-R-space denoted as the progress landscape. Note that for the sphere function, see Eqs. (39) and (40), the progress rate is constant for all R (given σ* and N).

Figure 6 shows an example progress landscape, evaluated for ${\sigma }^{\ast }\in \left[0,{\sigma }_{end}^{\ast }\right]$ and R ∈ [10⁻¹, 10²]. The value ${\sigma }_{end}^{\ast }$ is chosen slightly larger than ${\sigma }_{{\phi }_0}^{\ast }$, see Eq. (41), as for ${\sigma }^{\ast }>{\sigma }_{{\phi }_0}^{\ast }$ the progress rate gets negative. The R-range was chosen large enough to provide good visibility of the relevant characteristics. Thin black lines display regions of equal progress rate level. For R → ∞ and R → 0 the sphere limit is recovered (vertical lines of constant progress).

The median of real runs (black and red curves) show a characteristic σ*-drop (also visible in Fig. 1), which is directly related to the progress rate zero. The ESs are moving around the progress dip in σ*-R-space. Interestingly, the σ SA-ES with $\tau =1/\sqrt{2N}$ has a global convergence probability P_S = 0.91, while $\tau =1/\sqrt{8N}$ yields P_S = 0.99. Maximizing ${\sigma }^{\ast }⪅{\sigma }_{{\phi }_0}^{\ast }$ therefore maximizes P_S, which is associated with a smaller learning parameter τ. This effect can also be observed for the CSA-ES (cumulative step-size adaptation), where a higher P_S is observed for smaller cumulation constant values (due to a slower change of σ). The downside of large mutation strengths is less efficiency optimizing the sphere limits (the sphere-optimal value for the (100/100_I, 200)-ES, see Fig. 6, is at σ* ≈ 19). The respective median R-dynamics reaches the stopping value R = 10⁻³ at $g\approx 400(\tau =1/\sqrt{2N})$, while g ≈ 1100 for $\tau =1/\sqrt{8N}$.

One observes that ${\phi }_R^{\text{II},\ast }>0$ for sufficiently small σ*. This means that positive progress is expected at any R for arbitrary small σ* > 0, which contradicts experimental observations, see also Fig. 5, as small σ* significantly increases the local convergence probability. Hence, local attraction is not modeled correctly by ${\phi }_R^{\text{II},\ast }$. Furthermore, the progress dip of Fig. 6 is not related to single local attractors. It is a cumulative effect of oscillations in all N dimensions related to Rastrigin noise term (38). This is investigated in the next section.

5. Convergence and Population Sizing

In this section the convergence properties on the Rastrigin function are discussed. Global convergence (in expectation) requires ${\phi }_R^{\text{II},\ast }>0$ for R ∈ (0, ∞). The boundary ${\phi }_R^{\text{II},\ast }=0$, see Fig. 6, is therefore of most interest, especially the progress dip and its location in σ*-R-space. As it is shown in Appendix B, a closed-form solution can only be obtained under certain (simplified) assumptions. An analytical solution for R(σ*), such that ${\phi }_R^{\text{II},\ast }=0$, cannot be given due to the non-linearity of the underlying equations.

In the limit of R → ∞, all exponentials of (35) and (36) vanish. The resulting equation for ${\phi }_R^{\text{II},\ast }\left({\sigma }^{\ast },R\right)=0$ simplifies significantly with $D_Q^2=D_{\text{sph}}^2+N{A}^2/2$, see (37), such that a fourth order polynomial is obtained in Eq. (B.9) as

$${\sigma }^{\ast 4}+2N{\sigma }^{\ast 2}+\frac{N^4{A}^2}{4R^4}-8N{c}_{\vartheta }^2{\mu }^2=0.$$ (42)

Solving (42) for R yields the zero-progress line

$$R_{{\phi }_0}\left({\sigma }^{\ast }\right)={\left(\frac{1}{4}\frac{N^4{A}^2}{8N{c}_{\vartheta }^2{\mu }^2-2N{\sigma }^{\ast 2}-{\sigma }^{\ast 4}}\right)}^{1/4},$$ (43)

which is visualized in Fig. 7 as a black dashed line. An important relation to the noisy sphere model can be made. In [3] the residual location error R_∞ was derived for the (μ/μ_I, λ)-ES assuming a constant noise strength σ_ϵ in the limit σ* → 0 as

$$R_{\infty }\simeq \sqrt{\frac{{\sigma }_{\in }N}{4c_{\mu /\mu ,{\lambda }^{\mu }}}}.$$ (44)

Figure 7.

Progress rate ${\phi }_R^{\text{II},\ast }$ for (100/100_I, 200)-ES with N = 100 and A = 3. The black dashed line shows Eq. (43). Two lines show Eq. (46) with δ = 5 (yellow, top) and δ = 1 (magenta, bottom), respectively. Crosses indicate the intersection points obtained by Eq. (49). Note that the progress dip at A = 3 is significantly larger compared to A = 1 from Fig. 6.

Applying the limit σ* → 0 to Eq. (43), identifying the constant noise strength of the Rastrigin function (for sufficiently large R) as ${\sigma }_{\text{Ras}}^2=N{A}^2/2$ via (38) yields

$${R_{{\phi }_0}|}_{{\sigma }^{\ast }=0}={\left(\frac{N^3{A}^2}{32c_{\vartheta }^2{\mu }^2}\right)}^{1/4}=\sqrt{\frac{{\sigma }_{\text{Ras}}N}{4c_{\vartheta }\mu }},$$ (45)

which corresponds to result (44) with c_μ/μ,λ ≃ c_ϑ and σ_ϵ = σ_Ras.

Results (43) and (45) explain the σ*-decrease observed in Fig. 1 and Fig. 6, occurring for the ES approaching the Rastrigin noise floor. The red curve $(\tau =1/\sqrt{8N})$ decreases σ* to have positive progress at all, while the black curve $(\tau =1/\sqrt{2N})$ exhibits smaller σ*-values keeping a larger distance to the ${\phi }_R^{\text{II},\ast }=0$ boundary. Thus, the latter realizes a larger local progress. This is the result of the faster adaptation of σ (due to the larger τ). As a result, one has smaller mutations which are in turn more prone to be trapped in a local attractor. This is reflected by a lower success probability P_S. The smaller τ, however, yields a larger P_S -value by keeping a higher σ*-level.

In the limit of R → 0, σ* increases again, as the ES reaches the global attractor optimizing a sphere function with constant σ* (same level as for R → ∞). Since there is no closed-form solution of the progress dip location, a different approach is needed to model the transition point. Recalling that $R_{{\phi }_0}$ of (43) was derived in the limit of vanishing exponential terms, a natural extension of this model is to parametrize the point at which the terms are vanishing. This can also be motivated by looking at Eq. (22) and Fig. 3, where the exponential term models the transition between the sphere limits. Hence, a transition relation R_tr(σ*) is introduced. It can be obtained by investigating the characteristic exponential term of ${\phi }_R^{\text{II}}$ in (35), which also occurs in variance (36). Introducing an attenuation factor δ > 0 and setting σ = σ*R_tr/N, one can demand

$$\begin{array}{l}{\text{e}}^{-\delta }\stackrel{!}{=}{\text{e}}^{-\frac{{\left(\alpha R_{\text{tr}}\right)}^2}{2}\left[{\left(\frac{{\sigma }^{\ast }}{N}\right)}^2+\frac{1}{N}\right]},\text{such}\text{that}\\ R_{\text{tr}}\left({\sigma }^{\ast }\right)=\frac{\sqrt{2\delta N}}{\alpha }\frac{1}{\sqrt{1+{\sigma }^{\ast 2}/N}}.\end{array}$$ (46)

It is assumed that δ is independent of the fitness and strategy parameters. Figure 7 shows $R_{{\phi }_0}$ from (43) and R_tr from (46) with two exemplary evaluations δ = 1 and δ = 5. One observes that $R_{{\phi }_0}$ (black dashed line) follows the zero-progress line up until the dip minimum is reached. The dip location along the R-axis is well approximated by the constant noise limit (45) at σ* = 0. The R_tr-curves (magenta and yellow, respectively) follow a characteristic path depending on the chosen attenuation factor δ. The intersection point ${\sigma }_{\text{sec}}^{\ast }$ of both curves, namely

$$R_{{\phi }_0}\left({\sigma }_{\text{sec}}^{\ast }\right)\stackrel{!}{=}{R}_{\text{tr}}\left({\sigma }_{\text{sec}}^{\ast }\right),$$ (47)

is parametrizing the dip location and will give insight on the population scaling μ(N, α, A). Setting $R_{{\phi }_0}=R_{\text{tr}}$, one obtains a fourth order polynomial in σ* as

$${\sigma }^{\ast 4}+2N{\sigma }^{\ast 2}+\frac{N^2\left({\alpha }^4{A}^2-128{\delta }^2{c}_{\vartheta }^2{\mu }^2/N\right)}{{\alpha }^4{A}^2+16{\delta }^2}=0.$$ (48)

The real non-negative solution of (48) yields after simplification the intersection point

$${\sigma }_{\text{sec}}^{\ast }={\left[N\frac{{\left(1+\frac{8c_{\vartheta }^2{\mu }^2}{N}\right)}^{1/2}}{{\left(1+\frac{{\alpha }^4{A}^2}{16{\delta }^2}\right)}^{1/2}}-N\right]}^{1/2},$$ (49)

which is visualized in Fig. 7. Convergence on the sphere requires

$$0<{\sigma }_{sec}^{\ast }<{\sigma }_{{\phi }_0}^{\ast },$$ (50)

with the sphere-zero ${\sigma }_{{\phi }_0}^{\ast }$ given in Eq. (41). The relation ${\sigma }_{\text{sec}}^{\ast }<{\sigma }_{{\phi }_0}^{\ast }$ follows immediately for any A, α, δ > 0 and setting A = 0 or α = 0 yields ${\sigma }_{\text{sec}}^{\ast }={\sigma }_{{\phi }_0}^{\ast }$. Demanding ${\sigma }_{\text{sec}}^{\ast }>0$ in (49) it must hold

$$\frac{8c_{\vartheta }^2{\mu }^2}{N}>\frac{{\alpha }^4{A}^2}{16{\delta }^2}.$$ (51)

Solving (51) for μ one arrives at the important population sizing result

$$\mu >\sqrt{\frac{N}{2}}\frac{{\alpha }^{2}A}{8c_{\vartheta }\delta }.$$ (52)

Expression (52) relates the fitness-dependent parameters to the population size μ. For the subsequent experiments we will investigate the scaling properties of (52) without considering the potential prefactors of Eq. (52). To this end, repeated experiments of Algorithm 1 are performed and the success probability P_S is measured. Then, the necessary population size μ is evaluated to achieve a high success rate of P_S ≥ 0.99. The results of Figs. 8, 9, and 10 show good agreement with the parameter scaling predicted in Eq. (52). The μ(N)-scaling from experimental results is clearly sub-linear (as already observed in [9]) and indicates a scaling slightly larger than N^1/2. Some fluctuations can be observed which is practically inevitable, as very large N and μ are tested posing limits on the available CPU resources. Furthermore, certain deviations of the experiments to prediction (52) are expected to occur as the underlying model is based on an expected value, see (23), without considering possible higher order moments of the y_i -distribution causing fluctuations.

Figure 8.

Population sizing μ(N) using σSA-ES with $\tau =1/\sqrt{2N}$ and α = 2π. The top plot shows ϑ = 1/4 with A = 1, while the bottom plot shows ϑ = 1/2 with A = 10. The dotted cyan lines depict $\mu \propto \sqrt{N}$, dash-dotted green lines μ ∝ N^5/8, and dashed red lines μ ∝ N. The number of evaluated trials, for increasing N, is 2000, 2000, 1000, 700, 500, 400 (top) and 3000, 3000, 1500, 1000, 700, 600 (bottom).

Figure 9.

Population sizing μ (A) using σSA-ES with ϑ = 1/4, N = 100, $\tau =1/\sqrt{2N}$, and α = 2π. The dotted cyan lines show μ ∝ A. For each data point 2000 trials were evaluated.

Figure 10.

Population sizing μ (α) using σSA-ES with ϑ = 1/4, N = 100, $\tau =1/\sqrt{2N}$, and A = 1. The dotted cyan lines show μ ∝ α². For each data point 2000 trials were evaluated.

6. Local Attraction

In this section the limitations of the R-dependent progress rate ${\phi }_R^{\text{II}}$ are discussed by investigating local attraction effects. In case of local convergence one has σ → 0 (equivalently σ* → 0) while R stagnates. In this case the local structure of the fitness landscape is dominating. Hence, the assumption $y_i~\frac{R}{\sqrt{N}}\mathcal{\text{N}}(0,1)$ being normally distributed around the optimizer cannot hold. While the progress landscapes show positive progress for small σ*-values, this does not imply global convergence of real ES runs, see e.g. Fig. 12. It should be intuitively clear that for too small mutation strengths local convergence occurs. This issue was also observed in one-generation experiments in Fig. 5, where negative progress rates are obtained at certain y, if local attraction is present. As the aggregated (R-dependent) formula (35) is not able to model local attraction, a different approach is needed based on the y_i-dependent formula (24). The goal is to derive a σ-condition avoiding local attraction (in expectation). To this end, a characteristic “escape” mutation strength σ_esc is derived. It can serve as an additional stability criterion for the ES.

Starting with ${\phi }_i^{\text{II}}$ of Eq. (24), the gain function G is defined as

$$G\left(y_i,\sigma \right):=4y_i^2+{\text{e}}^{-\frac{1}{2}{(\alpha \sigma )}^2}2\alpha A{y}_i\text{sin}\left(\alpha y_i\right).$$ (53)

Requiring positive progress ${\phi }_i^{\text{Π}}>0$, Eq. (24) yields

$$\frac{c_{\vartheta }}{D_Q}G\left(y_i,\sigma \right)>\frac{1}{\mu }.$$ (54)

At this point the infinite population limit μ → ∞ is assumed in order to obtain closed-form solutions. As 1/μ → 0 it suffices to show that G > 0 for ${\phi }_i^{\text{II}}>0$ to hold. The function G is plotted in Fig. 11 as a function of σ and y_i. One observes local attraction regions for small mutations located at y₀ ≈ {1, 2, 3, 4, 5}. For small σ each of the attractors is a “stable” point, as y₀ + ϵ (with ϵ > 0) yields positive gain (decreasing y_i in expectation), while y₀ − ϵ yields negative gain (increasing y_i). For sufficiently large σ > σ_esc (black dotted line) positive progress can be ensured. The threshold σ_esc is derived now. Starting with (53), requiring $G\stackrel{!}{=}0$ and assuming y_i ≠ 0, G is refactored as

$$G=2y_i\tilde{G},$$ (55)

with $\tilde{G}$ defined as

$$\tilde{G}:=2y_i+{\text{e}}^{-\frac{1}{2}{(\alpha \sigma )}^2}\alpha A\text{sin}\left(\alpha y_i\right)\stackrel{!}{=}0,$$ (56)

yielding a first condition. The second condition $\frac{\partial G}{\partial y_i}=0$ (at σ = σ_esc) can be inferred from Fig. 11, which yields for (55)

$$\frac{\partial G}{\partial y_i}=2\tilde{G}+2y_i\frac{\partial \tilde{G}}{\partial y_i}\stackrel{!}{=}0.$$ (57)

Figure 11.

Gain function (53) visualized for A = 10 and α = 2π. The boundary G = 0 is shown in bold white, enclosing regions of negative progress. σ_esc from (64) is shown as black dotted. Only the first five local attractors are shown (out of 31).

As $\tilde{G}=0$ and y_i ≠ 0, $\frac{\partial G}{\partial y_i}=0$ is equivalent to $\frac{\partial \tilde{G}}{\partial y_i}=0$. Therefore, one has

$$\frac{\partial \tilde{G}}{\partial y_i}=2+{\text{e}}^{-\frac{1}{2}{(\alpha \sigma )}^2}{\alpha }^{2}A\text{cos}\left(\alpha y_i\right)=0,$$ (58)

such that the following condition is obtained

$${\text{e}}^{-\frac{1}{2}{(\alpha \sigma )}^2}\alpha A=-\frac{2}{\alpha \text{cos}\left(\alpha y_i\right)}.$$ (59)

Inserting condition (59) into (56), it follows

$$2y_i-\frac{2}{\alpha }\frac{\text{sin}\left(\alpha y_i\right)}{\text{cos}\left(\alpha y_i\right)}=0.$$ (60)

Introducing the substitution x = αy_i and applying sin x/cos x = tan x yields

$$\frac{2}{\alpha }(x-\tan x)=0.$$ (61)

The first non-trivial solution of (61) is the most interesting, as it corresponds to G = 0 of the first local attractor at y_i ≈ 0.75, see Fig. 11. Furthermore, negative gain contributions are due to the sine term in (53). For small |y_i| < 1 one has $y_i^2<\left|y_i\right|$, such that the first local attractor corresponds to the worst case requiring the largest σ to obtain G = 0. Numerical solving yields the zero of (61) as

$$x_0\approx \mathrm{4.493.}$$ (62)

Multiplying (56) by α, identifying x₀ = αy_i and σ = σ_esc (point of vanishing gain) results in

$$\begin{array}{c}2x_0+{\text{e}}^{-\frac{1}{2}{\left(\alpha {\sigma }_{\text{esc}}\right)}^2}{\alpha }^{2}A\text{sin}{x}_0=0\\ {\text{e}}^{\frac{1}{2}{\left(\alpha {\sigma }_{\text{esc}}\right)}^2}=-\frac{{\alpha }^{2}A\text{sin}{x}_0}{2x_0}.\end{array}$$ (63)

Resolving (63) for σ_esc yields the final result

$$\begin{array}{c}{\sigma }_{\text{esc}}=\frac{1}{\alpha }\sqrt{2\text{ln}\left(-\frac{{\alpha }^{2}A\text{sin}{x}_0}{2x_0}\right)}\\ \approx \frac{1}{\alpha }\sqrt{2\ln \left(0.1086{\alpha }^{2}A\right)}.\end{array}$$ (64)

Figure 12.

Median dynamics of unsuccessful runs (out of 100 trials) for (400/400_I, 800)-σSA-ES, N = 100, A = 10, with σ_esc ≈ 0.436 (red dashed line). The learning parameter was set to $\tau =1/\sqrt{N}$ (blue, left, P_S = 0.01), $\tau =1/\sqrt{2N}$ (black, center, P_S = 0.08), and $\tau =1/\sqrt{8N}$ (magenta, right, P_S = 0.29).

Figure 12 shows experiments of the (400/400_I, 800)-σSA-ES with α = 2π, and relatively large A = 10, such that one has σ_esc ≈ 0.436 from result (64). A constant σ translates to R (σ*) = σN/σ*, see normalization (2), showing a 1/σ* characteristics (red dashed line) in the progress landscape. The median of real unsuccessful runs is shown for different learning parameters τ. The sharp decrease σ* → 0 indicates local convergence, which agrees well with the σ_esc-line. However, dropping below the threshold σ < σ_esc does not imply that local convergence must occur (see success rate P_S > 0 for all τ). Conversely, it is a stability criterion that maintains positive component-wise progress in expectation, if σ > σ_esc is kept large enough. Of course, this can only hold up to the global attractor, at which σ → 0 must be ensured to have convergence.

Figure 13 shows numerically evaluated progress dip locations, see e.g. the dip at σ* ≈ 25 and R ≈ 2 in Fig. 12, for increasing μ-values while keeping ϑ and the fitness parameters constant. It shows how increasing μ shifts the dip location to larger σ*-values and smaller residual distances R. Using larger populations enables the ES to operate at larger mutation strengths and approach the optimizer more closely resulting in higher global convergence probabilities. The σ_esc-line (red dashed) remains constant as it was derived for μ → ∞. A progress dip located below the σ_esc-line is critical, as both noise floor and local attraction effects overlap yielding effectively zero success rates.

Figure 13.

Numerically evaluated progress dip locations (black dots) for ϑ = 1/2, N = 100, and A = 10 with increasing values μ = {10, 50, 100, 200, …, 1000} from left to right. The red dashed curve shows σ_esc (displayed as R = σ_escN/σ*).

The results obtained from Fig. 12 suggest a synthetic explicit σ-control rule for understanding the meaning of σ_esc. This rule uses a constant mutation strength σ for a sufficiently high number of generations until the global attractor is reached, and then decreases σ → 0. This is realized by defining a σ^(g)-schedule being constant for the first g < 9000 generations. For 9000 ≤ g ≤ 10⁴ it is decreased multiplicatively as σ^(g+1) = cσ^(g) (0 < c < 1), such that the stopping criterion σ < 10⁻⁶ is reached at the last generation. The corresponding experiments are conducted in Fig. 14. The single-trial dynamics show that only the run at σ_esc converges globally (repeated experiments shown in Fig. 15). ES-runs with σ < σ_esc tend to converge locally at large R due to the ES getting stuck in the local minima landscape. ES-runs operating at σ > σ_esc are less prone to local attraction and they reach the Rastrigin noise floor at moderately large R (see intersection of red and white line in Fig. 12).

Figure 14.

Single runs using constant σ for (400/400_I, 800)-ES, N = 100, and A = 10. A schedule for σ^(g) was defined being constant during the first 9000 generations and converging exponentially within the last 1000 generations. One has σ = {0.1, 0.2, 0.3, 0.4, σ_esc, 0.5, 0.6}, from top to bottom (see ordering at g = 2000).

Figure 15.

The success probability P_S is evaluated using parameters of Fig. 14 and 500 repetitions for each σ. The peak occurs around σ ≈ σ_esc = 0.436.

In Fig. 15 different σ-values are tested and the success rate P_S is evaluated over 500 repetitions. One observes that P_S is maximized at σ ≈ σ_esc. As expected, values σ < σ_esc are less successful due to local attraction. For σ > σ_esc local attraction is avoided, but the ES fluctuates at a larger residual distance before σ → 0, such that it is more likely to miss the global attractor.

7. Conclusions and Outlook

In this paper results from progress rate theory were applied and extended to investigate the convergence properties on the Rastrigin function. An aggregated residual distance dependent progress rate was obtained assuming normally distributed y_i-locations around the optimizer. The progress rate yields useful insights on the search behavior of the ES, which can be illustrated by recalling Fig. 1. Far away from local attraction the ES is optimizing the sphere keeping a constant scale-invariant mutation strength. Approaching the local attractor landscape leads to a significant reduction of the (normalized) mutation strength compared to the initial level. As the mutation strength σ decreases together with σ*, it may fall below the σ_esc-threshold (see Fig. 12). Having σ ≫ σ_esc (at σ*-levels comparable to sphere-optimal values) the ES is performing a global search. It is not significantly influenced by single local attractors. For σ ⪅ σ_esc the search can be regarded as rather local and individual attractors gain importance, such that local convergence occurs with higher probability. Within the global attractor the sphere function is optimized again. Considering the ES performance a two-fold positive effect of large populations on the success rate can be identified. First, large μ-values decrease the expected residual distance (45) to the global optimizer (similar to optimizing the sphere under constant noise). Second, intermediate recombination reduces the magnitude of the loss term −σ^*2/(2μ) in (39). Large μ and recombination therefore allow the ES to operate at larger σ-levels keeping σ > σ_esc and enabling a global search.

Furthermore, the progress rate analysis enabled the derivation of the population scaling result in (52), which could be experimentally verified. The result can serve to some extent as a guidance for the investigation of other highly multimodal test functions, provided that a global (spherical) structure exists with local perturbations.

There are multiple issues requiring further research. While it is now clear why large populations and mutation strengths are beneficial optimizing Rastrigin, a detailed analysis of the full σSA-ES or CSA-ES including the step-size adaptation is still pending. Additionally, the ES-efficiency in terms of fitness evaluations as a function of population size, truncation ratio, and learning parameter was not yet investigated. As the population size is a crucial parameter, the idea of using dynamic population control methods seems natural, see e.g. [4]. Actually, the theoretical analysis of population size control strategies is an uncharted research field. Furthermore, a probabilistic model would be useful to predict the success rate P_S as a function of fitness and ES parameters. Whether the obtained results can be transferred to other multimodal functions also remains part of future research.