Algorithm 1.

Optimizing hyperparameters with the decoupled Laplace approximation

Require: input carriers g, choices y

Require: initial hyperparameters θ0, subset of hyperparameters to be optimized θOPT

1: repeat

2:  Optimize for w given current θ → wMAP, Hessian of log-posterior Hθ, log-evidence E

3:  Determine Gaussian prior $\mathcal{N}\left(0,C_{\theta }\right)$ and Laplace appx. posterior $\mathcal{N}\left(w_{\text{MAP}},-H_{\theta }^{-1}\right)$

4:  Calculate Gaussian approximation to likelihood $\mathcal{N}\left(w_L,\mathrm{\Gamma }\right)$ using product identity, where ${\mathrm{\Gamma }}^{-1}=-\left(H_{\theta }+C_{\theta }^{-1}\right)$ and wL = −ΓHθwMAP

5:  Optimize E w.r.t. θOPT using closed form update (with sparse operations) $w_{\text{MAP}}=-H_{\theta }^{-1}{\mathrm{\Gamma }}^{-1}{w}_L$

6:  Update best θ and corresponding best E

7: until θ converges

8: return wMAP and θ with best E