The R2D2 prior for generalized linear mixed models

Abstract

In Bayesian analysis, the selection of a prior distribution is typically done by considering each parameter in the model. While this can be convenient, in many scenarios it may be desirable to place a prior on a summary measure of the model instead. In this work, we propose a prior on the model fit, as measured by a Bayesian coefficient of determination ($R^2$), which then induces a prior on the individual parameters. We achieve this by placing a beta prior on $R^2$ and then deriving the induced prior on the global variance parameter for generalized linear mixed models. We derive closed-form expressions in many scenarios and present several approximation strategies when an analytic form is not possible and/or to allow for easier computation. In these situations, we suggest approximating the prior by using a generalized beta prime distribution and provide a simple default prior construction scheme. This approach is quite flexible and can be easily implemented in standard Bayesian software. Lastly, we demonstrate the performance of the method on simulated and real-world data, where the method particularly shines in high-dimensional settings, as well as modeling random effects.

1. Introduction

As computing power has increased and become more accessible, Bayesian inference has risen to prominence. Researchers are now free to consider complex models with many parameters. An advantage of the Bayesian approach is that it can incorporate prior domain knowledge about some parameters to reduce uncertainty. In the absence of such information, we might select vague prior distributions, i.e., prior distributions with large variance. This, however, can lead to some unintended consequences such as Lindley’s paradox (Lindley, 1957). Vague prior distributions can also lead to poor estimates when the number of parameters is large relative to the sample size. This has led to the recent development of shrinkage prior distributions (George and McCulloch, 1993; Ročková and George, 2018; Park and Casella, 2008; Hans, 2009; Carvalho et al., 2010; Bhadra et al., 2017; Bhattacharya et al., 2015; Zhang et al., 2022).

Typically, prior distributions are selected for individual parameters based on domain expertise and/or using a general paradigm, e.g., shrinkage priors. There are situations, however, where researchers may have meaningful prior information on the model in general as opposed to specific regression coefficients. For example, consider genetic association studies (e.g., Lewis and Knight, 2012) where scientists search for the genes that contribute to a specific disease. There may be good understanding of how much genes affect the disease, but little information about which genes are relevant. In this case, it may make more sense to pick a prior for the overall model fit that then induces prior distributions on the parameters. There has been some previous work towards this end. Hodges and Sargent (2001) use a flat prior distribution on the degrees of freedom in a Gaussian mixed effects model. Simpson et al. (2017) introduce a paradigm that penalizes the complexity of the model as measured by the Kullback-Liebler (KL) divergence between the null and fitted model. This method places a prior on this KL divergence, thus shrinking the entire model instead of the individual parameters. Fuglstad et al. (2020) present a user-friendly approach to prior construction by utilizing prior beliefs to apportion the overall variance between different random effect components. The authors construct a joint prior distribution which considers the entire model structure. For Gaussian linear regression, Zhang et al. (2022) place a prior on the model fit as measured by the coefficient of determination, $R^2$. The authors first derive a Bayesian $R^2$ and show that the prior $R^2\sim \text{Beta}(a,b)$ yields a beta prime prior on the total variance of the regression parameters which is then distributed to each individual parameter through a Dirichlet Decomposition. For sparse high-dimensional regression problems, certain $R^2$ prior choices and a Dirichlet decomposition give posterior consistency. This method is advantageous because $R^2$ is an intuitive measure of model fit and it has excellent shrinkage properties.

In this work, we consider a prior on a summary of model fit by proposing a beta prior on Zhang et al. (2022)’s definition of $R^2$ for generalized linear mixed models. This extends Zhang et al. (2022) beyond linear regression to allow for non-Gaussian responses and random effects. We derive closed-form expressions in multiple scenarios for the prior of the global variance parameter that induces a beta prior on $R^2$. We also present several approximation strategies when an analytic prior distribution is not possible. The main approach we suggest approximates the prior by a generalized beta prime (GBP) distribution. This distribution is quite flexible as it can achieve boundedness at the origin as well as a heavy tail (Perez et al., 2017). The scaled beta prime distribution, a special case of the GBP, has also previously been used as a prior for the variance of the regression coefficients (Klein et al., 2021; Bai and Ghosh, 2021). Our method, like Zhang et al. (2022), differs from these previous approaches in that we place a GBP prior on the global variance which is then further decomposed in the hierarchy to the individual regression parameters. Our approach also provides an intuitive way to construct informative prior distributions as well as an automatic approach. The proposed methods can be applied using the r2d2glmm package available on GitHub at https://github.com/eyanchenko/r2d2glmm.

The remainder of the paper proceeds as follows. In Section 2, we describe the generalized linear mixed model framework and present several specific examples. In Section 3, we precisely define a Bayesian $R^2$ and show how the model-level prior induces prior distributions for the individual model prior parameters. We also present the prior distributions for several specific regression models as well as approximation techniques when a closed-form solution cannot be found. Section 4 applies the proposed method to real-world data and Section 5 concludes with recommendations for default use and next steps.

2. Generalized linear mixed models

For notational simplicity, we follow Simpson et al. (2017) and specify our model for a generalized linear mixed model (GLMM), although the ideas presented here can generalize to other settings. For observations $i\in \{1,\dots ,n\}$, let $Y_i$ be the response, ${\mathbf{X}}_i=(X_{i1},\dots ,X_{ip})$ be the explanatory variables and $\beta ={({\beta }_1,\dots ,{\beta }_p)}^T$ be the corresponding fixed effects. We standardize the explanatory variables such that each column of $\mathbf{X}$ has mean zero and variance one. We also assume that there are $q$ types of random effects, ${\mathbf{u}}_k$, $k\in \{1,\dots ,q\}$ where ${\mathbf{u}}_k={(u_{k1},\dots ,u_{k{L}_k})}^T$ has $L_k$ levels. We let ${\mathbf{g}}_i={(g_{i1},\dots ,g_{iq})}^T$ for $i\in \{1,\dots ,n\}$ be membership vectors such that $g_{ik}$ is the level of random effect $k$ for observation $i$ and where mixed-membership is excluded. The fixed and random effects prior distributions are assumed to be independent and are related to the response via the linear predictor

$${\eta }_i={\beta }_0+{\mathbf{X}}_i\beta +\sum _{k=1}^q{u}_{k{g}_{ik}}$$ (1)

where ${\beta }_0$ is the intercept. The responses are assumed to be conditionally independent given the linear predictor and follow density function $Y_i\mid {\eta }_i,\theta \sim f(y\mid {\eta }_i,\theta )$, where $\theta$ is an additional parameter in the likelihood function (see examples below).

The model for the fixed and random effects is ${\beta }_j\mid {\varphi }_j,W\stackrel{indep}{\sim }\text{Normal}(0,{\varphi }_{j}W)$ and ${\mathbf{u}}_k\mid {\varphi }_{p+k},W\stackrel{indep}{\sim }\text{Normal}(0,{\varphi }_{p+k}W{\mathbf{I}}_{L_k})$ where $W>0$ controls the overall variance of the linear predictor (not the response) and ${\varphi }_j\ge 0$ satisfy $\sum _{j=1}^{p+q}{\varphi }_j=1$ and apportion the variance to the different model components. Thus, $W$ may be interpreted as the total amount of variation in the fixed and random effects, or as a transformation of the total variation of the mean function. In the latter case, the interpretation depends on the link function. Moreover, large values of $W$ encode a model with greater flexibility since large variance in the mean function means that the model can capture more trends in the data. In the limit as $W\to 0$ conversely, we are reduced to the intercept-only model. This interpretation will be important later in this work when we treat the placement of a large prior mass on $W$ near zero as “penalizing” towards the null (intercept-only) model. Additionally, notice that the fixed and random effects are modeled similarly, i.e., with a random variance. Even so, we maintain their differing interpretations. Specifically, if we are interested in effect estimates themselves, then we treat this effect as “fixed,” but if our interest lies in the underlying population of the effect, then it is treated as “random” (Searle et al., 2009). Following this interpretation, we are most interested in the estimates of $\beta$ and ${\varphi }_{j}W$ for $j=p+1,\dots ,p+q$ .

The prior distribution of $R^2$ relies on the distribution of ${\eta }_i$. For the majority of this work, we assume

$${\eta }_i\mid {\beta }_0,W\sim \text{Normal}({\beta }_0,W).$$ (2)

We derive this result in the Supplemental Materials whether ${\mathbf{X}}_i$ is treated as fixed or random. If we treat ${\mathbf{X}}_i$ as random, then ${\eta }_i$ will be approximately normal for moderate $p$ by the Central Limit Theorem. On the other hand, if we consider ${\eta }_i$ conditional on ${\mathbf{X}}_i$, then the distribution of ${\eta }_i$ is exactly normal where the variance is different for each $i$ but the average variance is $W$ due to $\mathbf{X}’\mathrm{s}$ standardization. For either case, we stress that the prior distribution of ${\eta }_i$ is independent of the explanatory variables, resulting in a prior that does not depend on $\mathbf{X}$, similar to the PC prior (Simpson et al., 2017). Alternative distributions are discussed in Sections 3.2.2 - 3.2.3 but the normality of ${\eta }_i$ is assumed for all experiments.

2.1. Variance decomposition of the linear predictor

The variance parameters $\varphi =({\varphi }_1,\dots ,{\varphi }_{p+q})$ determine the relative variance of each component of the model and are restricted to sum to one. These parameters could be fixed, or given prior distributions to add flexibility to the variance decomposition. In the most general case we can assign these parameters a Dirichlet distribution, $\varphi \sim \text{Dirichlet}({\xi }_1,\dots ,{\xi }_{p+q})$. Often times we will take ${\xi }_1=\cdots ={\xi }_{p+q}\equiv {\xi }_0$. The concentration parameter ${\xi }_0>0$ controls the variation of the prior distribution with large ${\xi }_0$ encouraging all the variance components to be roughly equal to $1∕(p+q)$, and small ${\xi }_0$ reflecting prior uncertainty in the variance components. In some cases, the effects will be grouped and the variance across groups will be decomposed using a Dirichlet prior, e.g., all fixed effects assumed to have the same variance. These ideas are illustrated through examples below.

2.2. Examples

To help fix ideas, we present a few specific examples of this prior construction.

Example 1: Gaussian linear regression model:

In the linear regression setting with no random effects, the linear predictor is simply

$${\eta }_i={\beta }_0+{\mathbf{X}}_i\beta$$

and we have $Y_i\mid {\eta }_i,{\sigma }^2\sim \text{Normal}(\eta ,{\sigma }^2)$ so that $\theta ={\sigma }^2$ is the error variance. We then take ${\beta }_j\mid {\varphi }_j,W\sim \text{Normal}(0,{\varphi }_{j}W)$ for $j=1,\dots ,p$ . Zhang et al. (2022) study the theoretical properties of this approach for various prior distributions on $W$ and $\varphi$ . In general, this is a global-local shrinkage prior which has been studied in various contexts (e.g., Carvalho et al., 2010; Polson and Scott, 2012; Polson et al., 2012; Bhattacharya et al., 2015; Zhang and Bondell, 2018).

Example 2: Poisson regression with two-way random effects:

For a mixed effects model with two-way (non-interacting) random effects, the linear predictor is

$${\eta }_i={\beta }_0+{\mathbf{X}}_i\beta +u_{1g_{i1}}+u_{2g_{i2}},$$

and $Y_i\mid {\eta }_i\sim \text{Poisson}\{\text{exp}({\eta }_i)\}$. The membership vectors $g_{i1}$ and $g_{i2}$ indicate the level assigned to observation $i$ for random effects type one and two, respectively. The variance weights given to the fixed and random effects are determined by the Dirichlet parameter $\varphi$ . For example, to allow each fixed effect to have a different variance, we might take $\varphi \sim \text{Dirichlet}({\xi }_1,\dots ,{\xi }_{p+2})$ where ${\xi }_k$ are fixed hyperparameters; on the other hand, for each fixed effect to have the same variance, we might take $\varphi \sim \text{Dirichlet}({\xi }_1,{\xi }_2,{\xi }_3)$ and then let ${\beta }_j\mid {\varphi }_1,W\sim \text{Normal}(0,\frac{1}{p}{\varphi }_{1}W)$ for $j=1,\dots ,p$ and ${\mathbf{u}}_k\sim \text{Normal}(0,{\varphi }_{k}W{\mathbf{I}}_{L_k})$ for $k=2,3$ .

Example 3: Weibull model:

Survival analysis often uses a Weibull model. For simplicity, we consider uncensored data but this could be extended to censored data. Let there be a single random effect so that the linear predictor is

$${\eta }_i={\beta }_0+{\mathbf{X}}_i\beta +u_{g_i}$$

for membership vector $g_i\in \{1.\dots ,L\}$. If $Y_i$ is the survival time, then the model is $Y_i\mid {\eta }_i,\theta \sim \text{Weibull}(e^{{\eta }_i},\theta )$ for shape parameter $\theta$ . If we assume that the fixed effects have equal variance, then $\beta \mid {\varphi }_1,W\sim \text{Normal}(0,\frac{1}{p}{\varphi }_{1}W{\mathbf{I}}_p)$ and $\mathbf{u}\mid {\varphi }_2,W\sim \text{Normal}(0,{\varphi }_{2}W{\mathbf{I}}_L)$ where $\varphi \sim \text{Dirichlet}({\xi }_0,{\xi }_0)$ .

Example 4: Generalized linear regression with spatial random effects:

Consider the scenario where we observe data from $L$ spatial clusters (e.g., cities or villages) at spatial locations ${\mathbf{s}}_1,\dots ,{\mathbf{s}}_L\in {ℛ}^2$. Then let $Y_i$ be the response from location ${\mathbf{s}}_{g_i}\in {ℛ}^2$ where $g_i\in \{1,\dots ,L\}$ is the cluster indicator. Spatial generalized linear models account for correlation between observations at nearby locations by adding spatially-correlated random effects (e.g., Diggle et al., 1998). Let $u_{g_i}$ be the Gaussian random effect for cluster $g_i$. The linear predictor is then ${\eta }_i={\beta }_0+{\mathbf{X}}_i\beta +u_{g_i}$. A stationary and isotropic model assumes $\mathrm{E}(u_i)=0$ and $\text{Var}(u_i)={\sigma }_u^2$ for all $i$ and $\text{Cor}(u_i,u_j)=C(d_{ij})$, where $C$ is a spatial correlation function such as the exponential function $C(d)=\text{exp}(-d∕\rho )$ and $d_{ij}$ is the distance between locations ${\mathbf{s}}_i$ and ${\mathbf{s}}_j$. The covariance structure of the model is determined by the $L\times L$ correlation matrix $\mathbf{C}$ with ($i,j$) element $C(d_{ij})$. The spatial regression model is then in the form of (1) where $\mathbf{u}\mid {\varphi }_{p+1},W,\rho \sim \text{Normal}(0,{\varphi }_{p+1}W\mathbf{C})$ and ${\sigma }_u^2={\varphi }_{p+1}W$. While the covariance matrix of the random effect is no longer diagonal, the derivation of (2) still holds as the different random effect levels have the same variance and the covariance terms do not appear in the derivation.

Example 5: Generalized additive model:

Non-linear regression models can also be written as (1). Assume that $p$ explanatory variables, $x_{i1},\dots ,x_{ip}$, are allowed to have a non-linear relationship with the response variable. The generalized additive model (e.g., Hastie, 2017; Klein et al., 2021) is

$${\eta }_i={\beta }_0+\sum _{j=1}^p{f}_j(x_{ij})$$

for unknown functions $f_1,\dots ,f_p$. A common approach is to model the $f_j’\mathrm{s}$ using a basis expansion

$$f_j(x)=\sum _{l=1}^{L_j}{B}_{jl}(x){\beta }_l^{(k)}$$

where $B_{jl},\dots ,B_{j{L}_j}$ are basis function, e.g., spline functions and ${\beta }^{(k)}$ are “grouped” fixed effects. This model then fits (1) with $\mathbf{X}=({\mathbf{X}}_1,\dots ,{\mathbf{X}}_p)$ where ${\mathbf{X}}_j\in {ℛ}^{n\times L_j}$ is such that ${({\mathbf{X}}_j)}_{ik}=B_{jk}(x_{ij})$, and $\beta ={({\beta }^{(1)T},\dots ,{\beta }^{(p)T})}^T$. Then ${\beta }_k^{(j)}\sim \text{Normal}(0,\frac{1}{L_j}{\varphi }_{j}W)$ for $j\in \{1,\dots ,p\}$ and $k\in \{1,\dots ,L_j\}$ such that ${\varphi }_j$ determines the proportion of the variance allocated to the non-linear effect of $x_{ij}$.

3. Variance Decomposition $R^2$ and the R2D2 prior

Gelman et al. (2019), Gelman and Hill (2006) and Zhang et al. (2022) propose measures of model complexity that we name the Variance Decomposition $R^2$ (VaDeR). For the GLMM in Section 2, define $\mathrm{E}(Y_i\mid {\eta }_i)=\mu ({\eta }_i)$ and $\text{Var}(Y_i\mid {\eta }_i)={\sigma }^2({\eta }_i)$ which relates the linear predictor to the response distribution. Gelman et al. (2019) use the empirical definition of $R^2$, defined as

$$R_n^2=\frac{\mathrm{V}\{\mu ({\eta }_1),\dots ,\mu ({\eta }_n)\mid \mathbf{X},\mathbf{g},\beta ,\mathbf{u}\}}{\mathrm{V}\{\mu ({\eta }_1),\dots ,\mu ({\eta }_n)\mid \mathbf{X},\mathbf{g},\beta ,\mathbf{u}\}+\mathrm{M}\{{\sigma }^2({\eta }_1),\dots ,{\sigma }^2({\eta }_n)\mid \mathbf{X},\mathbf{g},\beta ,\mathbf{u}\}}$$ (3)

where M and V are the sample mean and variance operators, respectively.

In (3), $\mathrm{V}\{\mu ({\eta }_1),\dots ,\mu ({\eta }_n)\mid \mathbf{X},\mathbf{g},\beta ,\mathbf{u}\}$ is the variance of the expectation of future data and $\mathrm{M}\{{\sigma }^2({\eta }_1),\dots ,{\sigma }^2({\eta }_n)\mid \mathbf{X},\mathbf{g},\beta ,\mathbf{u}\}$ is the expected variance of future residuals, both conditioned on the explanatory variables, membership vectors and fixed and random effects. Because of this conditioning, Gelman et al. (2019) propose $R_n^2$ as an a posteriori measure of model fit. In principle, however, if the values of ${\mathbf{X}}_i$ and ${\mathbf{g}}_i$ are known but we had yet to observe the responses $Y_i$, then the prior distributions of the fixed and random effects would induce a prior distribution on $R_n^2$. Then $R_n^2$ is the proportion of variance explained by the model for future data, conditioned on these variables and our prior information for $\beta$ and ${\mathbf{u}}_k$.

While $R_n^2$ is an intuitive measure of the fit of the model to a particular dataset, for the purpose of setting prior distributions we follow Zhang et al. (2022). We measure complexity at the population level and use the marginal version of $R^2$ that averages over variation in both the explanatory variables and random effect levels ($\mathbf{X}$ and $\mathbf{g}$) as well as parameters ($\beta$ and ${\mathbf{u}}_k$). The marginal distribution does not depend on ${\mathbf{X}}_i$ or ${\mathbf{g}}_i$ so the observations are exchangeable. We can then drop the subscript distinguishing them and consider the model for an arbitrary observation $Y$ with $\mathrm{E}(Y\mid \eta )=\mu (\eta )$, $\text{Var}(Y\mid \eta )={\sigma }^2(\eta )$ and $\eta \mid {\beta }_0,W\sim \text{Normal}({\beta }_0,W)$ as in (2). Then $R^2$ becomes

$$R^2({\beta }_0,W)=\frac{\text{Var}\{\mu (\eta )\}}{\text{Var}(Y)}=\frac{\text{Var}\{\mu (\eta )\}}{\text{Var}\{\mu (\eta )\}+\mathrm{E}\{{\sigma }^2(\eta )\}}$$ (4)

where $\mathrm{E}\{{\sigma }^2(\eta )\}$ and $\text{Var}\{\mu (\eta )\}$ are summaries of the distribution of $\eta$ and thus depend on parameters ${\beta }_0$ and $W$. For the sake of simplicity, we suppress the dependence on (${\beta }_0,W$) and write $R^2({\beta }_0,W)=R^2$ for the remainder of the paper. The Supplemental Materials discusses the relationship between $R_n^2$ and $R^2$ and shows that under general conditions, $R_n^2$ will converge to $R^2$ when both the sample size and number of effective parameters increase. We also include a brief discussion comparing $R^2$ and $R_n^2$ with other measures of model fit for GLMMs (e.g. Cox and Snell, 1989; McFadden, 1973). Moreover, we note that the coefficient of determination is not commonly used for GLMMs as a measure of model fit. This could be for several reasons, not least of which being that it is difficult to define and interpret a principled $R^2$ for logistic regression, poisson regression, etc. A major advantage of $R^2$ and $R_n^2$ is that they can easily be extended to GLMMs, while also having intuitive interpretations.

As denoted in (4), the prior distribution of $R^2$ is determined by the joint prior (${\beta }_0,W$). For Gaussian responses the distribution of $R^2$ is invariant to ${\beta }_0$, and so to reduce the problem to matching univariate distributions, we parameterize the prior for (${\beta }_0,W$) as the conditional prior for $W\mid {\beta }_0$ and marginal prior for ${\beta }_0\sim {\pi }_0$. We then select a prior for $W\mid {\beta }_0$ so that $R^2\sim \text{Beta}(a,b)$ . By construction, since $R^2\sim \text{Beta}(a,b)$ conditioned on any ${\beta }_0$, $R^2$ also follows a $\text{Beta}(a,b)$ marginally over the joint prior for (${\beta }_0,W$) for any marginal prior ${\pi }_0$. Combined with the Dirichlet prior distribution on the variance proportions, this defines the $R^2$ Dirichlet decomposition prior (R2D2).

The $\text{Beta}(a,b)$ prior for $R^2$ is our default choice, but in some cases the support of $R^2$ can be restricted to a subspace of $[0,1]$ and a modification is required. Typically, when $W=0$ we also have $\text{Var}\{\mu (\eta )\}=0$ and thus $R^2=0$ assuming the distribution of $Y\mid \eta$ is not degenerate, i.e., ${\sigma }^2(\eta )>0$ . If, however, $\text{Var}\{\mu (\eta )\}>0$ when $W=0$, then the lower bound of $R^2$, $R_{min}^2$, is strictly greater than zero (e.g. Poisson regression with offsets in Supplemental Material). Conversely, for some link functions, $R^2<1$ for all $W$ (e.g., the zero-inflated Poisson model in Supplemental Materials). In general, the upper bound of $R^2$, $R_{max}^2$, is 1 if and only if $\mathrm{E}\{{\sigma }^2(\eta )\}=o(\text{Var}\{\mu (\eta )\})$ as $W\to \infty$. In cases where $R_{min}^2>0$ and/or $R_{max}^2<1$, we use a $\text{Beta}(a,b)$ prior distribution for the shifted and scaled $R^2$, denoted ${\tilde{R}}^2=(R^2-R_{min}^2)∕(R_{max}^2-R_{min}^2)$. This is equivalent to using a four-parameter beta distribution for the prior where $R^2\sim \text{Beta}(a,b,R_{min}^2,R_{max}^2)$ has density function

$$\pi (r^2)=\frac{{(r^2-R_{min}^2)}^{a-1}{(R_{max}^2-r^2)}^{b-1}}{{(R_{max}^2-R_{min}^2)}^{a+b-1}B(a,b)},R_{min}^2\le r^2\le R_{max}^2.$$

In most cases, $R_{min}^2=0$ and $R_{max}^2=1$ so unless otherwise noted we simply denote the prior as $R^2\sim \text{Beta}(a,b)$.

3.1. Special cases with exact expressions

Below we derive the expressions for the prior distribution for $W$ in several special cases where the exact prior distribution is available.

Location-scale models:

The location-scale model is $Y_i={\eta }_i+\sigma {ϵ}_i$, where the errors ${ϵ}_i$ have mean zero and variance one. Then $\mu (\eta )=\eta$ and ${\sigma }^2(\eta )={\sigma }^2$ and thus $R^2=W∕(W+{\sigma }^2)$ . Assuming $R^2$ follows a $\text{Beta}(a,b)$ and $\sigma =1$ (or more generally that ${\sigma }^2$ appears in the prior variance, ${\beta }_j\mid {\sigma }^2,{\varphi }_j,W\sim \text{Normal}(0,{\sigma }^2{\varphi }_{j}W)$), Zhang et al. (2022) show that the induced prior on $W$ is a Beta Prime distribution, denoted $W\sim \text{BP}(a,b)$ with density function

$$\pi (w)=\frac{1}{B(a,b)}\frac{w^{a-1}}{{(1+w)}^{a+b}},w\ge 0,$$ (5)

where $B(\cdot ,\cdot )$ denotes the Beta function. In the left panel of Figure 1, we plot $\pi (w)$ for various values of $a$ and $b$, and we can see that the BP prior distribution for $W$ has heavier tails when the expected $R^2$ is large ($a>b$) versus small ($a<b$).

Figure 1:

Plot of the prior distribution of $W$ to induce $R^2\sim \text{Beta}(a,b)$ with ${\beta }_0=0$. The title of each panel corresponds to the response model. The normal case takes ${\sigma }^2=1$.

For ${\sigma }^2\ne 1$, and not included in the prior variance, i.e., ${\beta }_j\mid {\varphi }_j,W\sim \text{Normal}(0,{\varphi }_{j}W)$, the induced prior distribution for $W$ is a Generalized Beta Prime (GBP) distribution, $W\mid {\sigma }^2\sim \text{GBP}(a,b,1,{\sigma }^2)$. The GBP distribution can be obtained via a transformation of a BP random variable, i.e., if $V\sim \text{BP}(a,b)$ then $W=d{V}^{1∕c}\sim \text{GBP}(a,b,c,d)$ and has density function

$$\pi (w;a,b,c,d)=\frac{c{\left(\frac{w}{d}\right)}^{ac-1}{\left(1+{\left(\frac{w}{d}\right)}^c\right)}^{-a-b}}{dB(a,b)},w\ge 0$$ (6)

for $a,b,c,d>0$. The GBP reduces to the BP if $c=d=1$.

We note a few properties of the GBP distribution. The behavior at the origin is controlled by the value of $ac$, with

$$\underset{w\to 0}{\text{lim}}\pi (w;a,b,c,d)=\{\begin{array}{cc}\infty \hfill & ac<1\hfill \\ \frac{c}{B(a,v)d}\hfill & ac=1.\hfill \\ 0\hfill & ac>1\hfill \end{array}\phantom{\}}$$

The tail behaviour is controlled by $bc$ with valid mean if only if $bc>1$. Also, for any model with $W\sim \text{GBP}(a,b,c,d)$ for the overall variance, then the standard deviation has prior distribution $W^{1∕2}\sim \text{GBP}(a,b,2c,d^{1∕2})$. As another special case of the GBP, if $a=1∕2$, $b=\nu ∕2$, $c=2$ and $d=\sqrt{\nu {\sigma }^2}$, then $W$ is distributed as a half-$t$ distribution with $\nu$ degrees of freedom and scale ${\sigma }^2$. Specifically, if $W\sim \text{GBP}(\frac{1}{2},\frac{1}{2},1,{\sigma }^2)$, then $\sqrt{W}$ follows a half-Cauchy distribution with scale $\sigma$ as in Gelman (2006).

Poisson regression:

The Poisson regression model is $Y\mid \eta \sim \text{Poisson}(e^{\eta })$ and thus $\mu (\eta )={\sigma }^2(\eta )=e^{\eta }$. Since $\eta \mid {\beta }_0,W\sim \text{Normal}({\beta }_0,W)$, $e^{\eta }\mid {\beta }_0,W\sim \text{LogNormal}({\beta }_0,W)$, and thus

$$R^2=\frac{e^W-1}{e^W-1+e^{-{\beta }_0-\frac{1}{2}W}}.$$ (7)

$R^2\sim \text{Beta}(a,b)$ induces (see Supplemental Materials) the prior for $W$ with density

$$\pi (w\mid {\beta }_0;a,b)=\frac{1}{B(a,b)}\frac{{(e^w-1)}^{a-1}{e}^{-b({\beta }_0+w∕2)}(3e^w-1)}{2{(e^w-1+e^{-{\beta }_0-w∕2})}^{a+b}},w\ge 0.$$ (8)

We plot this distribution in Figure 1 and show that shape of the prior looks very similar to that of the location-scale case. We do note, however, that the prior for $W$ has exponential-decaying tails on the scale of $\mathrm{E}(Y\mid \eta )=e^{\eta }$ as seen in (8). But, on the scale of $\log \{\mathrm{E}(Y\mid \eta )\}=\eta$, which is the same scale as $\beta$ and $\mathbf{u}$, the prior has polynomial-decaying tails. The value of the prior at 0 is $\infty$ if $a<1$, $b{e}^{{\beta }_0}$ if $a=1$ and 0 if $a>1$.

In the Supplemental Materials, we also include the exact prior distributions for: Poisson with offsets, negative binomial, zero-inflated Poisson, and the Weibull model.

3.2. Approximate Methods

In some cases, e.g., logistic regression, a closed-form expression for VaDeR is not available, so in this section we discuss alternatives.

3.2.1. Linear approximation

The two components we must compute for VaDeR are $\text{Var}\{\mu (\eta )\}$ and $\mathrm{E}\{{\sigma }^2(\eta )\}$. The simplest approach to approximate these is with a linear approximation. Applying a first-order Taylor series approximation of $\mu (\eta )$ and ${\sigma }^2(\eta )$ around ${\beta }_0$ gives

$$\text{Var}\{\mu (\eta )\}\approx {\{{\mu }^{\prime }({\beta }_0)\}}^{2}W\text{and}\mathrm{E}\{{\sigma }^2(\eta )\}\approx {\sigma }^2({\beta }_0).$$ (9)

Then denoting $s^2({\beta }_0)={\sigma }^2({\beta }_0)∕{\{{\mu }^{\prime }({\beta }_0)\}}^2$ we have

$$R^2\approx \frac{W}{W+s^2({\beta }_0)}.$$ (10)

If $R^2\sim \text{Beta}(a,b)$ , the resulting prior for $W$ is $W\mid {\beta }_0\sim \text{GBP}(a,b,1,s^2({\beta }_0))$. This result does not require any distributional assumptions about ${\eta }_i$ other than a finite mean and variance after transformation by $\mu (\cdot )$ and ${\sigma }^2(\cdot )$ . Additionally, the computational burden is essentially zero.

3.2.2. Quasi-Monte Carlo (QMC)

As we will show, in many cases the linear approximation dose a poor job approximation the true $R^2$ distribution. Therefore, we must turn to other methods. Since finding $R^2$ reduces to computing complicated integrals, we can use integral approximation techniques, like quasi-Monte Carlo (QMC; e.g., Morokoff and Caflisch, 1995). In usual Monte Carlo integration, the integral of interest is approximated by summing over a randomly generated sample of points. QMC is similar except that the points are selected deterministically. To construct the R2D2 prior, we approximate

$$\mathrm{E}\{\mu {(\eta )}^m\}\approx {\tilde{\mu }}_m(W\mid {\beta }_0)=\frac{1}{K-1}\sum _{i=1}^{K-1}\mu {({\beta }_0+z_i\sqrt{W})}^m$$ (11)

and

$$\mathrm{E}\{{\sigma }^2(\eta )\}\approx {\tilde{\sigma }}^2(W\mid {\beta }_0)=\frac{1}{K-1}\sum _{i=1}^{K-1}{\sigma }^2({\beta }_0+z_i\sqrt{W})$$ (12)

where $z_i$ is the $i∕K$ quantile of a standard normal distribution and $m=1,2$ . This gives an approximation of $R^2$ for a given ${\beta }_0$ and $W$, which we denote by

$${\tilde{R}}^2(W\mid {\beta }_0)\approx \frac{{\tilde{\mu }}_2(W\mid {\beta }_0)-{\tilde{\mu }}_1^2(W\mid {\beta }_0)}{{\tilde{\mu }}_2(W\mid {\beta }_0)-{\tilde{\mu }}_1^2(W\mid {\beta }_0)+{\tilde{\sigma }}^2(W\mid {\beta }_0)}.$$ (13)

Assuming $R^2\sim \text{Beta}(a,b)$ , then the prior for $W$ is

$$\pi (w\mid {\beta }_0;a,b)=\frac{1}{B(a,b)}{\{{\tilde{R}}^2(w\mid {\beta }_0)\}}^{a-1}{\{1-{\tilde{R}}^2(w\mid {\beta }_0)\}}^{b-1}\mid \frac{d{\tilde{R}}^2(w\mid {\beta }_0)}{dw}\mid ,w\ge 0.$$ (14)

Since this cannot be represented with elementary operations, in practice, we take a numerical derivative to evaluate the prior at a given value, and the entire calculation is completed virtually immediately.

The results in (11) and (12) make use of the normality of ${\eta }_i$ from (2). The QMC procedure can be modified to account for non-normal ${\eta }_i$. Let $\eta \sim F(\eta \mid {\beta }_0,W)$ for distribution function $F(\eta \mid {\beta }_0,W)$ . Then we approximate

$$\mathrm{E}\{\mu {(\eta )}^m\}\approx {\tilde{\mu }}_m(W\mid {\beta }_0)=\frac{1}{K-1}\sum _{i=1}^{K-1}\mu {\{q_i({\beta }_0,W)\}}^m$$

where $q_i({\beta }_0,W)$ is the $i∕K$ quantile of $F(\eta \mid {\beta }_0,W)$ . A similar result holds for approximating $\mathrm{E}\{{\sigma }^2(\eta )\}$ which then leads to an analogous result to (13). In practice, $F(\eta \mid {\beta }_0,W)$ can be derived analytically if the distribution of ${\mathbf{X}}_i$ is known. A more general strategy is to average over the empirical distribution of $\mathbf{X}$ giving a mixture of normal distributions for $F(\eta \mid {\beta }_0,W)$ .

3.2.3. Generalized beta prime approximation

The GBP distribution provides an exact solution for the location-scale model in Section 3.1, and an approximate solution for the linear approximation in Section 3.2.1. The prior $W\sim \text{GBP}(a,b,c,d)$ also induces the exact $R^2\sim \text{Beta}(a,b)$ prior distribution for any model with link functions $\text{Var}\{\mu (\eta )\}=W^c$ and $\mathrm{E}\{{\sigma }^2(\eta )\}=d^c$. The GBP will not give an exact solution in all cases, but it is a flexible four-parameter model which may often provide a reasonable approximation. Therefore, a general approximation strategy is to find the values of ($a^{\ast },b^{\ast },c^{\ast },d^{\ast }$) so that the prior $W\sim \text{GBP}(a^{\ast },b^{\ast },c^{\ast },d^{\ast })$ gives an approximate $\text{Beta}(a,b)$ distribution for $R^2$.

The optimal values of ($a^{\ast },b^{\ast },c^{\ast },d^{\ast }$) depend on $\mu (\cdot )$ and ${\sigma }^2(\cdot )$ as well as ${\beta }_0$, $a$ and $b$. For given link functions and parameters, let $W\sim \pi (w)$ be the distribution that gives exactly $R^2\sim \text{Beta}(a,b)$ . The GBP parameters are then set to minimize the Pearson ${\chi }^2$-divergence (Rényi, 1961) between the true and approximated PDFs since this metric enforces a close fit at both the origin and in the tails. We found that minimizing this quantity alone, however, led to unstable solutions, i.e., the surface being maximized over is “flat.” This means that vastly different values of ($a^{\ast },b^{\ast },c^{\ast },d^{\ast }$) may lead to GBP distributions that yield roughly the same approximation of $\pi (w)$. Thus, we also add a regularization term to shrink the prior towards a $\text{GBP}(a,b,1,1)$ distribution. We regularize toward this distribution because it gives the exact solution in the location-scale case and can be considered the baseline distribution. This results in the following optimization problem:

$$\begin{gathered} (a^{\ast },b^{\ast },c^{\ast },d^{\ast })=\underset{\alpha ,\beta ,c,d}{\text{argmin}}{\int }_0^{\infty }{\left\{\frac{f_{GBP}(w;\alpha ,\beta ,c,d)-\pi (w)}{\pi (w)}\right\}}^2\pi (w)dw \\ +\lambda \{{(\alpha -a)}^2+{(\beta -b)}^2+{(c-1)}^2+{(d-1)}^2\}, \end{gathered}$$ (15)

where $\lambda >0$ is a tuning parameter. A larger value of $\lambda$ yields a more stable solution but with a worse fit whereas a smaller value of $\lambda$ yields a better fit but with more instability. We found that $\lambda =\frac{1}{4}$ gives a good balance between fit and stability. In practice, the integral is approximated by a sum and $\pi (w)$ is approximated using QMC as in Section 3.2.2, if necessary. Since the GBP approximation may depend on the QMC procedure which can be modified to allow for non-normality in $\eta$, the GBP approach can similarly be adapted to allow for any distribution of $\eta$.

A major advantage of the GBP prior is that it can be easily implemented in standard Bayesian software such as JAGS (Plummer et al., 2016) or Stan (Carpenter et al., 2017). Because the exact prior distributions found in Section 3.1, as well as the resulting distributions from the QMC procedure in Section 3.2.2 do not easily allow for Gibbs sampling, these priors would be difficult for a practitioner to implement. By finding the GBP approximation, however, the R2D2 prior can be easily coded in JAGS and Stan. Example code is available as a vignette in our R package on GitHub. To specify the prior in these packages, we use the relationship that if $R^2\sim \text{Beta}(a,b)$ and $W=d{\{R^2∕(1-R^2)\}}^{1∕c}$, then $W\sim \text{GBP}(a,b,c,d)$ . Because of these features, we recommend this method for general use even in cases when the exact expression is available, and will be the method we consider in the simulation studies.

Since the GBP approximation depends on ${\beta }_0$ (and $\theta$), this approximation should be updated with the unknown parameter ${\beta }_0$. The calculation in (15) takes about half a second, however, so this would be overly time prohibitive to compute during every MCMC iteration. Instead, we find the GBP approximation once at the beginning of the analysis at ${\widehat{\beta }}_0=g(\sum _{i=1}^n{Y}_i∕n)$ for link function $g(\cdot )$ . Thus, to induce $R^2\sim \text{Beta}(a,b)$ , the first step is to find ($a^{\ast },b^{\ast },c^{\ast },d^{\ast }$) as in (15) at ${\widehat{\beta }}_0$ (and ${\widehat{\theta }}_{MLE}$, if necessary, the maximum likelihood estimate of the dispersion parameter). After determining ($a^{\ast },b^{\ast },c^{\ast },d^{\ast }$), ${\beta }_0$ (and $\theta$) are treated as unknown parameters in the subsequent Bayesian analysis. In the Supplemental Materials, we report the GBP approximations for various values of ($a,b$) and different models. In most cases, the best fitting $a^{\ast }$ and $b^{\ast }$ values are not close to ($a,b$) which demonstrates the need for this approximation.

Figure 2 compares the linear and GBP approximations with the true distribution for the Poisson model (Section 3.1). The GBP is nearly a perfect match to the true distribution for each prior. The linear approximation is reasonable when $a=1,b=4$, but very poor when $a=4,b=1$. This example shows that the GBP is a very good approximation to the true prior distribution of $W$.

Figure 2:

Comparison of different approximation methods for the Poisson regression model with ${\beta }_0=0$. The title of each pane indicates the induced prior distribution on $R^2$.

In the Supplemental Materials, we conduct a simulation study to compare the R2D2 prior with a standard vague prior, the Horseshoe prior (Carvalho et al., 2009), and the PC prior (Simpson et al., 2017) while also comparing different combinations of ($a,b$) . The proposed method performs favorably across all settings and particularly well in the high-dimensional regression setting. Indeed, we empirically find that prior distributions with large prior mass near $R^2=0$ yield good shrinkage properties.

4. Real data analysis

The proposed method is now applied to two real-world data sets. The first analysis is focused on inference, while the second looks at prediction in the high-dimensional setting.

4.1. Malaria data

We now analyze the gambia data set (Thomson et al., 1999) from the geoR package (Ribeiro Jr et al., 2007) in R to demonstrate the use of the R2D2 prior in practice. We also consider PC and vague prior distributions. There are $n=2035$ children in this data set with binary response variable $Y_i$ which equals 1 if child $i$ tested positive for malaria and 0 otherwise. There are $p=5$ explanatory variables including age, indicator of using a bed net, indicator of whether the bed net is treated, “greenness” of village and indicator of a health center in the area. These variables are standardized to have mean zero and variance one. There are also the $L=65$ villages where each child lived, along with the spatial location of each village.

We model the village effect as a spatial random effect. As in Example 4 from Section 2.2, the linear predictor is

$$\log \text{it}\{P(Y_i=1\mid {\eta }_i)\}={\eta }_i={\beta }_0+{\mathbf{X}}_i\beta +u_{g_i}$$ (16)

where $g_i\in \{1,\dots ,L\}$ is the village of response $i$. We also assume that $\mathrm{E}(u_i)=0$ and $\text{Var}(u_i)={\sigma }_u^2$ for all $i$ and exponential spatial correlation $C_{ij}=\text{Cor}(u_i,u_j)=e^{-d_{ij}∕\rho }$ where $d_{ij}$ is the distance between village $i$ and $j$ and $\rho >0$ is the spatial range parameter. Then the full prior specification for R2D2 is

$$\begin{gathered} {\beta }_0\sim \text{Normal}({\mu }_0,{\tau }_0^2),\beta \mid {\varphi }_1,W\sim \text{Normal}(0,\frac{1}{5}{\varphi }_{1}W{\mathbf{I}}_5),\mathbf{u}\mid {\varphi }_2,W,\rho \sim \text{Normal}(0,{\varphi }_{2}W\mathbf{C}), \\ \rho \sim \text{Uniform}(0,2r),W\sim \text{GBP}(a^{\ast },b^{\ast },c^{\ast },d^{\ast }),\varphi \sim \text{Dirichlet}({\xi }_1,{\xi }_2) \end{gathered}$$ (17)

for hyper-parameters set to ${\mu }_0=0$, ${\tau }_0^2=3$, ${\xi }_1={\xi }_2=1$ and $r$ is the maximum distance between pairs of villages. Note that ${\sigma }_u^2={\varphi }_{2}W$ in this model. We find ${\widehat{\beta }}_0=-0.59$ and ($a^{\ast },b^{\ast },c^{\ast },d^{\ast }$) are in Table 1 and the resulting prior distributions are plotted in Figure 3.

Table 1:

Generalized Beta Prime approximation parameters for Gambia data with ${\widehat{\beta }}_0=-0.59$.

$a$	$b$	$a^{\ast }$	$b^{\ast }$	$c^{\ast }$	$d^{\ast }$
1	4	1.15	2.08	0.91	2.09
0.5	0.5	0.57	0.29	0.90	1.54
1	1	1.47	0.65	0.79	1.67
4	4	7.45	2.72	0.73	1.63
4	1	7.77	0.71	0.68	1.45

Figure 3:

Prior $R^2$ and global variance parameter for R2D2 prior for Gambia data

For PC prior, the full prior specification is

$$\begin{gathered} {\beta }_0\sim \text{Normal}({\mu }_0,{\tau }_0^2),\beta \sim \text{Normal}(0,{\tau }_1^2{\mathbf{I}}_5),\mathbf{u}\mid {\sigma }_u^2\sim \text{Normal}(0,{\sigma }_u^2\mathbf{C}), \\ \rho \sim \text{Uniform}(0,2r),{\sigma }_u\sim \text{Exp}({\lambda }_0). \end{gathered}$$ (18)

where ${\mu }_0=0,{\tau }_0^2=3,{\tau }_1^2=100$ and ${\lambda }_0=-\log (0.01)∕.968$ . The vague prior has the same form as the PC prior except ${\sigma }_u^2\sim \text{InvGamma}(0.5,0.0005)$ (Spiegelhalter et al., 2003).

We take 105,000 MCMC samples with the first 5,000 discarded as burn-in, where the analysis is performed in JAGS. The results are in Figure 4 and Table 2. We also present trace plots in the Supplemental Materials to check convergence of the MCMC chain, as well as the effective sample size and computation time. We can see that the posterior distributions of $R_n^2$ are very similar across the different methods. The posterior of $W$, however, varies across the different R2D2 priors with the Beta(4,1) and Beta(4,4) having the greatest mean and Beta(0.5,0.5) and Beta(1,1) having the smallest mean. The posterior distributions of $W$ and ${\sigma }_u^2$ are almost identical for the R2D2 priors which means that the vast majority of the global variance mass is shifted on the random effect variance and away from the fixed effect variance. The posterior for ${\sigma }_u^2$ has the smallest mean for the PC prior, which follows from the fact that this prior shrinks the spatial variance toward zero. Lastly, the posterior of $\rho$ varies across the different priors with the PC prior also yielding the smallest posterior mean.

Figure 4:

Posterior $R^2$, global variance and random effect variance and $\rho$ for vague (uninformative) prior distributions, PC and R2D2 for Gambia data with village spatial random effect.

Table 2:

Posterior mean and standard deviation for $R_n^2$, global variance ($W$), random effect variance (${\sigma }_u^2$) and spatial range ($\rho$) for each method for Gambia data considering spatial random effect.

	$R^2$		$W$		${\sigma }_u^2$		$\rho$
Method	Mean	St. Dev	Mean	St. Dev	Mean	St. Dev	Mean	St. Dev
Vague	0.176	0.016	–	–	2.389	1.397	0.851	0.500
PC	0.173	0.016	–	–	1.210	0.515	0.427	0.270
$R^2\sim \text{Beta}(\frac{1}{2},\frac{1}{2})$	0.171	0.015	1.538	0.827	1.511	0.824	0.549	0.388
$R^2\sim \text{Beta}(1,1)$	0.173	0.016	2.023	1.148	1.992	1.143	0.722	0.465
$R^2\sim \text{Beta}(1,4)$	0.171	0.016	1.540	0.796	1.513	0.792	0.556	0.382
$R^2\sim \text{Beta}(4,1)$	0.175	0.016	2.813	1.412	2.780	1.409	0.977	0.488
$R^2\sim \text{Beta}(4,4)$	0.174	0.016	2.675	1.237	2.637	1.227	0.981	0.479

4.2. Genomics data

For our second data analysis, we apply the proposed method to high-dimensional genomics data from human breast tumors (Perou et al., 2000) in the mixOmics R package (Rohart et al., 2017). This data was pre-processed in Pérez-Enciso and Tenenhaus (2003) such that the response, $Y_i$, is 1 if tumor specimen $i$ was analyzed before chemotherapy treatment, and 0 if it was analyzed after chemotherapy, for sample $i\in \{1,\dots ,n\}$ where $n=47$. There are $p=1000$ gene expressions as predictor variables, so $p\gg n$, and we imputed the appropriate column mean of $\mathbf{X}$ for any missing values. We consider the R2D2 prior with $(a,b)\in \{(1,5),(1,10),(1,20),(1,30)\}$, and compare with the Horseshoe prior. We choose these hyper-parameter combinations because we hypothesize that prior distributions with large mass near $R^2=0$ are useful in high-dimensional contexts.

Since our focus of this analysis is prediction, we randomly split the data into train and test sets, where 75% of the data is used for training, and 25% for testing. The model is fit on the training data, and we compute the Brier Score (BS) and binary cross entropy (BCE) loss on the hold-out test data. If $Y_i$ is the true response, and ${\widehat{Y}}_i$ is the predicted response, then $\text{BS}=\frac{1}{n}\sum _{i=1}^n{({\widehat{Y}}_i-Y_i)}^2$ and $\text{BCE}=-\frac{1}{n}\sum _{i=1}^n\{Y_i\log {\widehat{Y}}_i+(1-Y_i)\log (1-{\widehat{Y}}_i)\}$. We compute ${\widehat{Y}}_i={[1+\text{exp}\{-({\widehat{\beta }}_0+{\mathbf{X}}_i^{\prime }\beta )\}]}^{-1}$ where ${\widehat{\beta }}_0$ and $\beta$ are the posterior means of ${\beta }_0$ and $\beta$ , respectively, and ${\mathbf{X}}_i^{\prime }$ are the predictor variables for the $i$th test data. The splitting procedure is repeated 50 times and we find the average of each metric. A good method will have small BS and BCE. The analysis was performed in Stan where we computed 10,000 MCMC samples with an additional 1000 for burn-in.

The results are in Table 3. In the Supplemental Materials, we also report the computation time, and average effective sample size for $\beta$ and the global variance parameter. We can see that the R2D2 prior with $(a,b)=(1,20)$ and Horseshoe perform the best, with smallest BCE and BS, respectively. Indeed, there is a clear trend that the R2D2 priors with more mass near $R^2=0$ yield better predictions. However, there appears to be diminishing returns beyond $R^2\sim \text{Beta}(1,10)$ .

Table 3:

Prediction results for genomics data with standard error in parentheses. BS is Brier Score, and BCE is the binary cross entropy loss. Best results are labeled in bold.

Method	BS	BCE
Horseshoe	0.23 (0.01)	0.79 (0.07)
$R^2\sim \text{Beta}(1,5)$	0.29 (0.01)	0.89 (0.06)
$R^2\sim \text{Beta}(1,10)$	0.26 (0.00)	0.71 (0.01)
$R^2\sim \text{Beta}(1,20)$	0.26 (0.00)	0.70 (0.01)
$R^2\sim \text{Beta}(1,30)$	0.26 (0.00)	0.71 (0.01)

5. Discussion

In this work, we proposed a novel method for choosing informative prior distributions in the generalized linear mixed model setting. The proposed prior is flexible and interpretable in terms of overall model fit as measured by a Bayesian $R^2$. There are many cases where the prior $R^2$ can be induced exactly as well as general approximation strategies when an exact form is not possible. The main approach that we suggest is approximating the global variance prior with a generalized beta prime distribution because of its flexibility and ability to be implemented in standard software. Combined with an initial estimate of the intercept via a method of moments estimator and the GBP approximation in the r2d2glmm package, we provide a simple and intuitive method for setting prior distributions in GLMMs.

If there is domain knowledge available on how well the model is expected to fit the data, then this could be used to inform prior choice for $R^2$. In the absence of any prior information, we suggest $R^2\sim \text{Beta}(1,1)$ as a reasonable default choice. This prior expresses a full range of model fits, but should not be confused with a flat prior for $\beta$ . Choosing $R^2\sim \text{Beta}(1,b)$ for large $b$, or another prior with large mass near 0, is also a good choice, especially when working in a high-dimensional setting. Indeed, substantial prior mass near $R^2=0$ prevents the model from overfitting by shrinking towards the “base model” of $\beta =0_p$.

The proposed approach naturally fits within the global-local shrinkage prior framework where $W$ controls the global shrinkage, and ${\varphi }_j$ the local shrinkage. Our approach does differ from, e.g., Hamura et al. (2022), who studies shrinkage priors for count data. Using the notation of our paper, Hamura et al. (2022) let $Y_i\mid {\lambda }_i\sim \text{Poisson}({\lambda }_i{e}^{{\eta }_i})$ where ${\lambda }_i\sim \text{Gamma}(\alpha ,\beta ∕{\nu }_i)$, ${\nu }_i\sim \pi (\cdot )$, and ${\eta }_i$ is some offset which may depend on covariates, i.e., ${\eta }_i={\beta }_0+{\mathbf{X}}_i\beta$. This paper is primarily interested in the prior distribution for the local component (${\nu }_i$) and estimating the rate parameter (${\lambda }_i$). The R2D2 prior, on the other hand, is derived for the global component ($W$), and is focused on estimating the regression components (${\eta }_i$).

A limitation of the proposed method is that the hierarchical framework only allows for random intercepts and not, for example, random slopes. Additionally, the finite mean and variance requirement precludes applications to some models, e.g., extreme value analysis (Coles et al., 2001). We have also not proven concentration or shrinkage properties which is an avenue for future work. We could also extend the method to allow for other survival analysis settings beyond the uncensored Weibull model and models that are not GLMMs such as Bayesian deep learning. Finally, at present, the method cannot easily be written in INLA (Lindgren and Rue, 2015), so we consider this an important next step.