Algorithm 1.

Out-of-sample MSE and predictive MSE.

1.	Obtain posterior samples ${\left\{{\theta }^s\right\}}_{s=1}^S\sim p(\theta |y)$;
2.	For each holdout set k = 1, …, K:
	(a) Compute the (log) weights, $\log W_k^s\stackrel{c}{=}-\log p(y_{{\mathcal{I}}_k}|{\theta }^s)=-{\sum }_{i\in {\mathcal{I}}_k}\log p(y_i|{\theta }^s)$ using the (Gaussian) log-likelihood in (1), where $\stackrel{c}{=}$ denotes equality up to a constant;
	(b) Estimate ${\widehat{\beta }}^{-{\mathcal{I}}_k}\approx {\sum }_{s=1}^S{\beta }^s{W}_k^s$ and ${\stackrel{\_}{\omega }}_j={{\sum }_{s=1}^S{W}_k^s|{\beta }_j^s|}^{-\gamma }$;
	(c) Set ${\widehat{y}}_j^{-{\mathcal{I}}_k}={\tilde{x}}_j^{\prime }{\widehat{\beta }}^{-{\mathcal{I}}_k}\text{for}j\notin {\mathcal{I}}_{\text{k}};$
	(d) Compute ${\widehat{\delta }}_{\lambda }^{-{\mathcal{I}}_k}$ by solving (22) for each λ;
	(e) Sample ${\left\{{\tilde{y}}_i^{s*}\right\}}_{s*=1}^{\tilde{S}}$ from $p({\tilde{y}}_i|y^{-{\mathcal{I}}_k})$ using the SIR algorithm for $i\in {\mathcal{I}}_k$;
3.	Compute ${\mathrm{MSE}}_{\lambda }^{\mathit{out}}$ using ${\widehat{\delta }}_{\lambda }^{-{\mathcal{I}}_k}$ and ${\widetilde{\mathrm{MSE}}}_{\lambda }^{\mathit{out}}$ using ${\widehat{\delta }}_{\lambda }^{-I_k}$ and ${\left\{{\tilde{y}}_i^{s*}\right\}}_{s*=1}^{\tilde{S}}$ in (17).