Algorithm 2.

The mth iteration of Simplified Manifold Preconditioner Adaptation.

	Setup and inputs: d-dimensional random vector $X\in 𝒳\subseteq {\mathfrak{R}}^d,X\sim P$ whose density $p(·)>0$ is continuous with respect to the Lebesgue measure, assume $K$ is a compact subset of $𝒳$, fix the constants $D\gg 0,\kappa >0,0<T^{\text{adapt}}<\mathrm{\infty }$, step size $0<\epsilon <D$, denote ${\mathit{g}}_x={\nabla }_x\mathrm{l}\mathrm{o}\mathrm{g}p(\mathit{x})$ ${\tilde{\mathit{g}}}_{\mathit{x}}={\mathit{g}}_{\mathit{x}}\cdot \mathrm{m}\mathrm{i}\mathrm{n}\left\{\frac{D}{{\mathrm{m}\mathrm{a}\mathrm{x}}_i\left\{{\mathit{g}}_{\mathit{x}}\left[i\right]\right\}},1\right\},{\mathit{G}}_{\mathit{x}}^{-1}=-\frac{{\delta }^2}{\delta {\mathit{x}}^2}\mathrm{l}\mathrm{o}\mathrm{g}p(\mathit{x}),{\tilde{\mathit{G}}}_{\mathit{x}}^{-1}={\mathit{G}}_{\mathit{x}}^{-1}\cdot \mathrm{m}\mathrm{i}\mathrm{n}\left\{\frac{D}{{\mathrm{m}\mathrm{a}\mathrm{x}}_i\left\{{\mathit{G}}_{\mathit{x}}[i,i]\right\}},1\right\}$, let the sequence $\left({\gamma }_m,m\in \mathbb{N}\right)$ be such that ${\gamma }_m>0,{\gamma }_m\downarrow 0$.
	function SiMPA:
1:	Sample $z\sim U(0,1),v\sim U(0,1)$, $\mathit{u}\sim N(\mathbf{0},I_d)$.
2:	Let ${\mathit{\mu }}_{(\text{new})}={\mathit{x}}_{(m-1)}+\frac{{\epsilon }^2}{2}{\mathit{M}}_{(m-1)}{\tilde{\mathit{g}}}_{{\mathit{x}}_{(m-1)}}$ and propose ${\mathit{x}}_{(\mathrm{n}\mathrm{e}\mathrm{w})}={\mathit{\mu }}_{(\mathrm{n}\mathrm{e}\mathrm{w})}+\epsilon {\mathit{M}}_{(m-1)}^{\frac{1}{2}}\mathit{u}$.
3:	Let ${\mathit{\mu }}_{(\text{back})}={\mathit{x}}_{(\mathrm{n}\mathrm{e}\mathrm{w})}+\frac{{\epsilon }^2}{2}{\mathit{M}}_{(m-1)}{\tilde{\mathit{g}}}_{\left.{\mathit{x}}_{(\mathrm{n}\mathrm{e}\mathrm{w})}\right)}$.
4:	Compute
	$\alpha =\frac{p\left({\mathit{x}}_{(\text{new})}\right)}{p\left({\mathit{x}}_{(m-1)}\right)}\cdot \frac{N\left({\mathit{x}}_{(m-1)};{\mathit{\mu }}_{(\text{back})},{\epsilon }^2{\mathit{M}}_{(m-1)}\right)}{N\left({\mathit{x}}_{(\text{new})};{\mathit{\mu }}_{(\text{new})},{\epsilon }^2{\mathit{M}}_{(m-1)}\right)}$.
5:	if $\alpha <v$ and $‖x_{(\mathit{new})}-x_{(m)}‖<D$,	⊳ proposal accepted
6:	Set ${\mathit{x}}_{(m)}={\mathit{x}}_{(\mathrm{n}\mathrm{e}\mathrm{w})}$.
7:	else: set ${\mathit{x}}_{(m)}={\mathit{x}}_{(m-1)}$.	⊳ proposal rejected
	if $z<{\gamma }_m$ and $\left({\mathit{x}}_{(m)}\in K\right.$ or $\left.m<T^{\text{adapt}}\right)$:	⊳ adapting
8:	Set $M_{(m)}^{-1}=M_{(m-1)}^{-1}+\kappa ({\tilde{\mathit{G}}}_{{\mathit{x}}_{(m)}}^{-1}-M_{(m-1)}^{-1})$ and compute $M_{(m)}^{\frac{1}{2}}$.
	else:	⊳ not adapting
9:	Set $M_{(m)}=M_{(m-1)}$.