Testing Random Effects in the Linear Mixed Model Using Approximate Bayes Factors

SUMMARY

Deciding which predictor effects may vary across subjects is a difficult issue. Standard model selection criteria and test procedures are often inappropriate for comparing models with different numbers of random effects due to constraints on the parameter space of the variance components. Testing on the boundary of the parameter space changes the asymptotic distribution of some classical test statistics and causes problems in approximating Bayes factors. We propose a simple approach for testing random effects in the linear mixed model using Bayes factors. We scale each random effect to the residual variance and introduce a parameter that controls the relative contribution of each random effect free of the scale of the data. We integrate out the random effects and the variance components using closed form solutions. The resulting integrals needed to calculate the Bayes factor are low-dimensional integrals lacking variance components and can be efficiently approximated with Laplace’s method. We propose a default prior distribution on the parameter controlling the contribution of each random effect and conduct simulations to show that our method has good properties for model selection problems. Finally, we illustrate our methods on data from a clinical trial of patients with bipolar disorder and on data from an environmental study of water disinfection by-products and male reproductive outcomes.

1. Introduction

The linear mixed model (Laird and Ware, 1982) is a popular method for longitudinal data. It is often of interest to test whether certain random effects should be included, which is equivalent to setting the random effect variance equal to 0. Because this test lies on the boundary of the parameter space, classical procedures such as the likelihood ratio test can break down (Pauler, Wakefield and Kass, 1999; Lin, 1997; Self and Liang, 1987; Stram and Lee, 1994). Tests for a single variance component can be carried out using mixtures of chi-square distributions (Self and Liang, 1987; Stram and Lee, 1994). For multivariate tests, distributions of test statistics are complex and not easily applied (Pauler et al., 1999; Feng and McCulloch, 1992; Shapiro, 1988). Some alternative frequentist methods include score tests (Lin, 1997; Commenges and Jacqmin-Gadda, 1997; Verbeke and Molenberghs, 2003; Molenberghs and Verbeke, 2007; Zhang and Lin, 2008), Wald tests (Molenberghs and Verbeke, 2007; Silvapulle, 1992), and generalized likelihood ratio tests (Crainiceanu and Ruppert, 2004), but these methods are not easily extended for testing multiple variance components.

Some MCMC methods have been suggested to test variance components (Sinharay and Stern, 2001; Chen and Dunson, 2003; Cai and Dunson, 2006; Kinney and Dunson, 2008), but these methods are generally time consuming to implement, require special software, and rely on subjective choice of hyperparameters. The most widely used approximation to the Bayes factor is based on the Laplace approximation (Tierney and Kadane, 1986), resulting in the Bayesian information criterion (BIC) (Schwarz, 1978) under certain assumptions. However, the Laplace approximation can fail when the parameter lies on the boundary (Pauler et al., 1999; Hsiao, 1997; Erkanli, 1994). Pauler et al. (1999) proposed estimating Bayes factors using an importance sampling approach and a boundary Laplace approximation. Their methods are complex and are only applied in the context of simple variance component models.

Because random effects models involve a distinct parameter for every individual, linear mixed models can have a very large number of dimensions. This is problematic in calculating Bayes factors because high dimensional integrals are needed to calculate marginal likelihoods. Generally these integrals are not available in closed form and one must consider approximations. Numerical integration is not useful in high dimensions (Kuonen, 2003). Monte Carlo integration and importance sampling provide alternatives, but these methods lack accuracy and are computationally demanding. The Laplace and BIC approximations also suffer in performance from high-dimensionality (Kass and Raftery, 1995). In addition, it is not clear how to define the penalty for dimensionality in the BIC (Spiegelhalter et al., 2002).

It is well known that Bayes factors can be sensitive to the choice of prior distributions (Kass and Raftery, 1995). This is problematic in situations in which one has no prior information on the parameters and the goal is to choose the best model based on the data. In these situations it is common to use default priors which can be chosen based on the data without subjective inputs and that result in good frequentist and Bayesian operating characteristics. However, one must choose these default priors with care, because as the prior variance increases the Bayes factor will increasingly favor the null model (Bartlett, 1957).

We propose a simple approach for conducting approximate Bayesian inferences on whether to include random effects in the linear mixed model. Our approach involves a re-parameterization of the linear mixed model allowing an accurate Laplace approximation to the Bayes factor. In Section 2 we introduce two motivating examples. In Section 3 we introduce our method in the context of a repeated measures ANOVA model and conduct a simulation study. In Section 4 we generalize our approach to the linear mixed model and in Section 5 we apply the method to the motivating examples. We conclude with a discussion in Section 6.

2. Motivating Examples

2.1 Hamilton Rating Scale for Depression

We first consider a clinical trial of patients with bipolar I disorder (Calabrese et al., 2003), GlaxoSmithKline study SCAB2003. The investigators concluded that lamotrigine treatment significantly delays time to intervention for a depressive episode compared to placebo. Repeated measurements were collected on the Hamilton Rating Scale for Depression (HAMD), a 17-item scale measuring the severity of depression. We wish to determine if lamotrigine is effective in reducing depressive symptoms during the first year after randomization, as measured by the HAMD summary score, using a linear mixed model.

In assessing the impact of lamotrigine on HAMD scores, it is important to assess the heterogeneity among patients with respect to the overall mean and slope over time. One might expect patients to have different patterns of depressive episodes, perhaps resulting from biological mechanisms or unmeasured covariates that cause different individual profiles over time. This leads to the task of testing whether to include random coefficients for the intercept and slope over time in the linear mixed model.

2.2 Exposure of disinfection by-products in drinking water and male fertility

A multi-center study of 229 male patients from 3 sites (Raleigh, NC; Memphis, TN; and Galveston, TX) was conducted to evaluate the effect of disinfection by-products (DBP’s) in drinking water on male reproductive outcomes in presumed fertile men. DBP exposure was measured using water system samples and data collected on individual water usage. Three exposure variables of interest for the outcome percent normal sperm are brominated haloacetic acids (HAA-Br), brominated trihalomethanes (THM-Br), and total organic halides (TOX). Our focus is to evaluate the DBP exposure effects on the response (% normal sperm) using a linear mixed model.

In assessing the impact of DBPs on sperm quality, it is of interest to assess the heterogeneity among study sites with respect to the overall mean of percent normal sperm (i.e. intercept) and each DBP effect (i.e. slope). It may be the case that study site is a surrogate for unmeasured aspects of water quality or other unmeasured factors of interest.

3. Testing a random intercept

3.1 ANOVA model

We start by considering a simple ANOVA model with a random subject effect

$$M_1^{(1)}:Y_{\mathit{\text{ij}}}=\mu +\lambda b_i+{\epsilon }_{\mathit{\text{ij}}},$$ (1)

in which Y_ij is the jth response for subject i, μ is an intercept, b_i ~ N(0, σ²) is a scaled random effect multiplied by a parameter λ > 0, and ε_ij ~ N(0, σ²) for i = 1,…, n and j = 1,…, n_i. This is an ANOVA model with a random effect variance equal to λ²σ². The utility of this decomposition will later become clear. The notation $M_k^{(a)}$ represents parameterization (a) for model k. We distinguish models parameterized in different ways in order to consider the impact of parameterization on the accuracy of the Laplace approximation to the marginal likelihood. Our initial focus is to compare the ANOVA model to a model with no random subject effect,

$$M_0:Y_{\mathit{\text{ij}}}=\mu +{\epsilon }_{\mathit{\text{ij}}},$$ (2)

in which μ is an overall mean and ε_ij ~ N(0, σ²). We are interested in estimating Bayes factors to determine the posterior odds of $M_1^{(a)}$ versus M₀ given equal prior odds, or

$$B_{10}^{(a)}=\frac{p(\mathit{Y}|M_1^{(a)})}{p(\mathit{Y}|M_0)},$$ (3)

in which $\mathit{Y}=({\mathit{y}}_1^{\prime },\dots ,{\mathit{y}}_n^{\prime })\prime$. Estimating the Bayes factor relies on estimates of

$$p(\mathit{Y}|M_k^{(a)})={\displaystyle \int p(\mathit{Y}|{\mathit{\theta }}_k^{(a)},M_k^{(a)})\pi ({\mathit{\theta }}_k^{(a)}|M_k^{(a)})d{\mathit{\theta }}_k^{(a)},}$$ (4)

in which $p(\mathit{Y}|{\mathit{\theta }}_k^{(a)},M_k^{(a)})$ is the data likelihood, ${\mathit{\theta }}_k^{(a)}$ is the vector of model parameters, and $\pi ({\mathit{\theta }}_k^{(a)}|M_k^{(a)})$ is the prior distribution of ${\mathit{\theta }}_k^{(a)}$. Let $M_0^{(a)}=M_0$, as only one parameterization of M₀ will be considered. For $M_1^{(a)}$ and M₀, the marginal likelihoods are generally not available in closed form. Let ${\mathit{\theta }}_1^{(a)}=({\mathit{\zeta }}_1^{\prime (a)},\mathit{b}\prime ,{\sigma }^2)\prime \text{ and }{\mathit{\theta }}_0^{(a)}=({\mathit{\zeta }}_0^{\prime (a)},{\sigma }^2)\prime$, such that ${\mathit{\zeta }}_k^{(a)}$ includes all parameters other than the random effects b and residual variance σ². We specify an inverse gamma prior on σ² with parameters v, w, in which the mean of σ² is w/(v − 1) for v > 1. By marginalizing out b and σ² in $M_1^{(1)}$ and σ² in M₀, it can be shown that $(\mathit{Y}|\mu ,\lambda ,M_1^{(1)})$ and (Y|μ, M₀) follow multivariate t-distributions with

$$p(\mathit{Y}|{\mathit{\zeta }}_k^{(a)},M_k^{(a)})=\frac{\mathrm{\Gamma }\left(\frac{2v+m}{2}\right){\mathrm{\Pi }}_{i=1}^n|\frac{w}{v}{\displaystyle {\mathbf{\sum }}_i{|}^{-1/2}}}{{(\pi 2v)}^{m/2}\mathrm{\Gamma }(2v/2)}\times {\left\{1+\frac{1}{2v}{\displaystyle \mathbf{\sum }_{i=1}^n{({\mathit{y}}_i-{\mathit{\mu }}_i)}^{\prime }{\left(\frac{w}{v}{\mathbf{\Sigma }}_i\right)}^{-1}({\mathit{y}}_i-{\mathit{\mu }}_i)}\right\}}^{-\frac{2v+m}{2}},$$ (5)

in which $m={\displaystyle {\sum }_{i=1}^n{n}_i}$ is the total number of observations. In our ANOVA setup, μ_i = μ1_ni in M₀ and $M_1^{(1)}$, Σ_i = I_ni in M₀, and ${\mathbf{\Sigma }}_i=({\mathit{I}}_{n_i}+{\lambda }^2{\mathbf{1}}_{n_i}{\mathbf{1}}_{n_i}^{\prime })\text{ in }{M}_1^{(1)}$. After specifying a prior on μ, the Laplace method can be used to integrate over (μ, λ) in $M_1^{(1)}$ and μ in M₀. We use the resulting marginal likelihood estimates to estimate the Bayes factor $B_{10}^{(1)}$. For additional details regarding these multivariate t-distributions, see the Web Appendices.

The Laplace approximation is based on a linear Taylor series approximation of $\tilde{l}({\mathit{\zeta }}_k^{(a)})=\text{log}\{p(\mathit{Y}|{\mathit{\zeta }}_k^{(a)},M_k^{(a)})\pi ({\mathit{\zeta }}_k^{(a)}|M_k^{(a)})\}$. The marginal likelihood $p(\mathit{Y}|M_k^{(a)})$ for model k and parameterization (a) is estimated by

$$\widehat{p}(\mathit{Y}|M_k^{(a)})={(2\pi )}^{d/2}|{\tilde{\mathbf{\Sigma }}}_k^{(a)}{|}^{1/2}p(\mathit{Y}|{\tilde{\mathit{\zeta }}}_k^{(a)},M_k^{(a)})\pi ({\tilde{\mathit{\zeta }}}_k^{(a)}|M_k^{(a)}),$$ (6)

in which ${\tilde{\mathbf{\Sigma }}}_k^{(a)}$ is the inverse of the negative Hessian matrix of $\tilde{l}({\mathit{\zeta }}_k^{(a)})$ evaluated at the posterior mode ${\tilde{\mathit{\zeta }}}_k^{(a)}$. Because the Laplace approximation is based on a linear Taylor series approximation, it requires certain regularity conditions. When the posterior mode lies on the boundary of the parameter space these regularity conditions fail. The Laplace method can perform poorly even if the mode is close to the boundary. Estimating the marginal likelihood in $M_1^{(1)}$ via Laplace can be problematic because of the restriction λ > 0, motivating the parameterization

$$M_1^{(2)}:Y_{\mathit{\text{ij}}}=\mu +e^{\varphi }{b}_i+{\epsilon }_{\mathit{\text{ij}}},$$ (7)

in which ϕ = log(λ). Note the parameter space of ϕ is unrestricted, ensuring that the posterior mode is not on the boundary. Hence the estimated marginal likelihoods based on $M_1^{(2)}$ may be more accurate than those based on $M_1^{(1)}$. Following the steps outlined previously, it can be shown that $(\mathit{Y}|\mu ,\varphi ,M_1^{(2)})$ follows a multivariate t-distribution with density (5), with μ_i = μ1_ni and ${\mathbf{\Sigma }}_i=({\mathit{I}}_{n_i}+e^{2\varphi }{\mathbf{1}}_{n_i}{\mathbf{1}}_{n_i}^{\prime })$. We use the Laplace approximation to integrate over (μ, ϕ) and use the resulting estimate of the marginal likelihood to estimate the Bayes factors $B_{10}^{(2)}$.

3.2 Prior choice

It is important to identify default priors that yield robust tests. We have introduced a parameter λ (or ϕ) that controls the contribution of the random effect free of the scale of the data. We propose priors λ ~ log N(κ_λ, τ_λ) and ϕ ~ N(κ_ϕ, τ_ϕ) with κ_ϕ = κ_λ and τ_ϕ = τ_λ set so that the priors for λ and ϕ are “equivalent”, meaning they lead to the same marginal likelihood. Differences in the estimated marginal likelihoods between $M_1^{(1)}\text{ and }{M}_1^{(2)}$ result from differences in the accuracy of the two Laplace approximations. Given that the random effects are scaled to the residual error, we suggest κ_λ = log(0.3) and τ_λ = 2 as reasonable default values. This centers the parameter λ between 0 and 1. Even if the true value of λ is not close to 0.3, the variance ensures that the prior covers most reasonable values of λ. The choice of priors will be discussed further in our simulation studies.

3.3 Simulation study

We conducted a simulation study to evaluate the performance of our method in correctly identifying models with or without random effects. We simulated 250 data sets based on (1) with n = 50, 100, 500, 1000, n_i = 3, σ² = 1, μ = 0, and λ = 0, 0.15, 0.30, 0.45, 0.60. In order to implement the Laplace approximation, we estimated the posterior mode using an algorithm by Nelder and Mead (1965). We used prior distributions μ ~ N(0, 1) and σ² ~ InvGam(1, 1), which are non-informative given the simulation settings. Estimates of the Bayes factors $B_{10}^{(1)}\text{ and }{B}_{10}^{(2)}$ were calculated for each data set and were interpreted according to the scale given by Wasserman (2000) and Jeffreys (1961).

Both parameterizations performed well in favoring the correct model, but accuracy depended on both the sample size and the true value of λ. In general, as λ increased our method increasingly favored $M_1^{(a)}$ over the null model. Figure 1 shows box plots of log ${\widehat{B}}_{10}^{(1)}$ for λ = 0, 0.30. The dotted black line represents a log Bayes factor of 0. As the sample size increased, our method more accurately detected the absence of a random slope for λ = 0 and more accurately detected the presence of a random slope for λ > 0. Additional tables are available in the Web Appendices.

Figure 1.

Box plot of log ${\widehat{B}}_{10}^{(2)}$, by λ

We compared our method to the restricted likelihood ratio test (RLRT) and ANOVA F-test with α = 0.05. The asymptotic distribution of the RLRT follows a 50:50 mixture of a point mass at 0 and a chi-square distribution with 1 degree of freedom. For the ANOVA model, F =MSA/MSE, with MSA the between-group mean square and MSE the within-group mean square, follows an F distribution with (n−1) and (m−n) degrees of freedom, in which n is the number of subjects and m is the total number of observations. For our Bayesian approach, we chose to reject H₀ if $B_{10}^{(a)}>1$. As illustrated in Table 1 (columns 2–6), the power of our approach was competitive with both the ANOVA F-test and restricted likelihood ratio test. Parameterization (2) was somewhat conservative, while parameterization (1) led to an inflated Type I error for n ≥ 100.

Table 1.

Testing a random intercept or random slope, power and Type I error

	Testing a random intercept					Testing a random slope
n	λ	${\widehat{B}}_{10}^{(1)}>1$	${\widehat{B}}_{10}^{(2)}>1$	ANOVA	RLRT	$\sqrt{{\psi }_{22}}$	${\widehat{B}}_{21}^{(1)}>1$	${\widehat{B}}_{22}^{(2)}>1$	RLRT
50	0	3	4	4	3	0	8	3	4
	0.15	12	12	12	12	0.15	14	7	9
	0.3	22	22	22	22	0.3	30	17	20
	0.45	59	60	60	59	0.45	56	57	59
	0.6	92	93	92	92	0.6	75	90	89
100	0	7	4	4	4	0	4	4	6
	0.15	14	7	8	8	0.15	12	8	13
	0.3	47	42	43	42	0.3	38	38	44
	0.45	93	86	87	87	0.45	69	87	88
	0.6	99	98	98	98	0.6	72	99	100
500	0	12	4	5	5	0	3	2	4
	0.15	26	14	17	17	0.15	26	21	30
	0.3	94	90	92	92	0.3	80	85	95
	0.45	100	100	100	100	0.45	97	99	100
	0.6	100	100	100	100	0.6	98	100	100
1000	0	12	6	8	8	0	1	0	3
	0.15	36	28	35	34	0.15	33	32	45
	0.3	100	100	100	100	0.3	93	99	100
	0.45	100	100	100	100	0.45	85	100	100
	0.6	100	100	100	100	0.6	96	100	100

Table gives percent of times the null hypothesis was rejected out of 250 simulations

^*Type I error is given by λ = 0 or $\sqrt{{\psi }_{22}}=0$

To assess the impact of our prior choice, we conducted additional simulations with priors of the form λ ~ logN(h, ζ), with various combinations of h = log(1), log(0.3), log(0.15) and ζ = 1, 2, 3. Additionally, we considered a log t-distribution for λ with 2 and 10 degrees of freedom. Equivalent priors were also assessed for ϕ using parameterization (2). We also considered prior distributions σ² ∝ σ⁻², σ² ~InvGamma(0.1,0.1), σ² ~InvGamma(0.01,0.01), and μ ∝ c in which c is a constant. We found that alternative priors on σ² and μ did not have notable influence on the estimated Bayes factors, but the priors for λ and ϕ did have some influence. More specifically, values of h = log(0.30) and ζ = 2 resulted in power and Type I error rates that closely aligned with the ANOVA F-test and RLRT. Smaller values of h or ζ led to increased Type I error rates and larger values of h or ζ led to more conservative Type I error rates. See the Web Appendices for more details.

4. Testing a random slope

4.1 Linear mixed model

We generalize our approach for testing random effects by considering a linear mixed model

$${\mathit{y}}_i={\mathit{X}}_i\mathit{\beta }+{\mathit{Z}}_i{\mathit{b}}_i+{\epsilon }_i,$$ (8)

in which y_i = (Y_i1,…, Y_ini)′ is a n_i × 1 vector of responses, X_i = (x_i1,…, x_ip) is a n_i × p design matrix, Z_i = (z_i1,…, z_iq) is a n_i × q design matrix, β = (β₁,…, β_p)′ is a p × 1 vector of parameters, and b_i = (b_i1,…, b_iq)′ is a q × 1 vector of random effects. It is assumed that ε_i ~ N(0,R) is independent of b_i ~ N(0, ψ), in which ψ is the q × q covariance matrix of random effects. A popular choice for R is σ²I, which assumes conditional independence given the random effects.

We choose b_ih ~ N(0, σ²) and introduce a parameter λ_h that controls the contribution of the hth random effect. Let $M_k^{(a)}$ refer to model k and parameterization a. Similar to Chen and Dunson (2003), our reparameterized model takes the form

$$M_0^{(1)}:{\mathit{y}}_i={\mathit{X}}_i\mathit{\beta }+{\mathit{Z}}_{0,i}{\mathbf{\Lambda }}_0^{(1)}{\mathbf{\Gamma }}_0{\mathit{b}}_{0,i}+{\epsilon }_i,$$ (9)

in which Z_0,i = (z_i1,…, z_iq), b_0,i = (b_i1,…, b_iq)′, ${\mathbf{\Lambda }}_0^{(1)}=\text{diag}({\mathit{\lambda }}_0^{(1)})=\text{diag}({\lambda }_1^{(1)},\dots ,{\lambda }_q^{(1)})\prime ,\text{ and }{\lambda }_l^{(1)}~\text{ log }N(\text{log}(0.3),2)$ for l = 1,…, q. Let Γ₀ be a lower triangular matrix with 1_q along the diagonal, and lower off-diagonal elements γ₀ which induce correlation between the random effects.

Our focus is to test whether to include an additional random effect b_i(q+1). Let Z_1,i, ${\mathbf{\Lambda }}_1^{(1)}$, Γ₁ and b_1,i be equal to their counterparts from (9), but including the elements corresponding to the additional random effect b_i(q+1). The model including the additional random effect is

$$M_1^{(1)}:{\mathit{y}}_i={\mathit{X}}_i\mathit{\beta }+{\mathit{Z}}_{1,i}{\mathbf{\Lambda }}_1^{(1)}{\mathbf{\Gamma }}_1{\mathit{b}}_{1,i}+{\epsilon }_i,$$ (10)

in which Z_1,i = (z_i1,…, z_i(q+1)), b_1,i = (b_i1,…, b_i(q+1))′, ${\mathbf{\Lambda }}_1^{(1)}=\text{diag}({\mathit{\lambda }}_1^{(1)})=\text{diag}({\lambda }_1^{(1)},\dots ,{\lambda }_{q+1}^{(1)})\prime ,{\lambda }_l^{(1)}~\text{ log }N(\text{log}(0.3),2)$ for l = 1,…, (q + 1), and Γ₁ is a lower triangular matrix with 1_q+1 along the diagonal and lower off-diagonal elements γ₁.

As demonstrated with the ANOVA model, we also consider an alternate parameterization of (9) and (10) by setting ${\mathit{\lambda }}_k^{(2)}=\text{log}{\mathit{\lambda }}_k^{(1)},{\mathbf{\Lambda }}_0^{(2)}=\text{diag}(e^{{\mathit{\lambda }}_0^{(2)}})=\text{diag}(e^{{\lambda }_1^{(2)}},\dots ,e^{{\lambda }_q^{(2)}}),{\mathbf{\Lambda }}_1^{(2)}=\text{diag}(e^{{\mathit{\lambda }}_1^{(2)}})=\text{diag}(e^{{\lambda }_1^{(2)}},\dots ,e^{{\lambda }_{q+1}^{(2)}}),\text{ and }{\lambda }_l^{(2)}~N(\text{log}(0.3),2)$ for l = 1,…, (q + 1). Let $M_0^{(2)}$ denote the reduced model and $M_1^{(2)}$ denote the full model under this parameterization.

4.2 Approximating the marginal likelihoods

In order to implement the Laplace approximation, we first marginalize out b and σ². Let σ² ~ InvGam(v, w). It can be shown that the marginal distribution $p(\mathit{Y}|\mathit{\beta },{\mathit{\lambda }}_k^{(a)},{\gamma }_k,M_k)$ follows a multivariate t-distribution with density (5), in which μ_i = X_iβ and ${\mathbf{\Sigma }}_i=({\mathit{I}}_{n_i}+{\mathit{Z}}_{k,i}{\mathbf{\Lambda }}_k^{(a)}{\mathbf{\Gamma }}_k{\mathbf{\Gamma }}_k^{\prime }{\mathbf{\Lambda }}_k^{\prime (a)}{\mathbf{Z}}_{k,i}^{\prime })$. After specifying priors for β and γ_k, we use the Laplace method to integrate over $(\mathit{\beta },{\mathit{\lambda }}_k^{(a)},{\gamma }_k)$ to approximate the marginal likelihoods $p(\mathit{Y}|M_k^{(a)})$ used to evaluate the Bayes factor $B_{10}^{(a)}$. See the Web Appendices for additional details.

As previously discussed, many of the existing methods for testing variance components are only applicable in simple settings. One major advantage of our approach is we can test multiple random effects simultaneously by modifying equation (10) such that the Z_1,i, ${\mathbf{\Lambda }}_1^{(a)}$, Γ₁, and b_1,i correspond to a model with several additional random effects.

4.3 Simulation study

We conducted a simulation study to test for the presence of a random slope. We defined one predictor based on time, such that x_i = (1, 2,…, J)′ and ${\mathit{x}}_i^{*}$ is centered and scaled by two times the standard deviation of x_i. This standardization puts the regression coefficients on the same scale as binary indicators (Gelman, 2008). In the context of our method, it puts the scale of the λ parameter for the random slope on the same scale as the λ corresponding to the random intercept. Let ${\mathit{X}}_i=(\mathbf{1},{\mathit{x}}_i^{*})$, β = (β₀, β₁)′, M₁ refer to the random intercept model, and M₂ refer to the random intercept and slope model. Letting Z_1,i = 1_J, ${\mathit{\lambda }}_1^{(a)}={\lambda }_0^{(a)}$, and b_1,i = b_i0, we have

$$M_1^{(a)}:y_{\mathit{\text{ij}}}={\beta }_0+{\lambda }_0^{(a)}{b}_{i0}+{\beta }_1{x}_{\mathit{\text{ij}}}^{*}+{\epsilon }_{\mathit{\text{ij}}}$$ (11)

for the random intercept model. Letting ${\mathit{Z}}_{2,i}=({\mathbf{1}}_J,{\mathit{x}}_i^{*}),{\mathit{\lambda }}_2^{(a)}=({\lambda }_0^{(a)},{\lambda }_1^{(a)})\prime$, and b_2,i = (b_i0, b_i1)′, we have

$$M_2^{(a)}:y_{\mathit{\text{ij}}}={\beta }_0+{\lambda }_0^{(a)}{b}_{i0}+{\beta }_1{x}_{\mathit{\text{ij}}}^{*}+{\lambda }_1^{(a)}({\gamma }_{12}{b}_{i0}+b_{i1})x_{\mathit{\text{ij}}}^{*}+{\epsilon }_{\mathit{\text{ij}}}$$ (12)

for the random intercept and slope model. Our focus is to compare model $M_2^{(a)}\text{ to }{M}_1^{(a)}$. After integrating out b and σ² to produce marginal multivariate t-distributions, the integrals needed to calculate the marginal distributions $p(\mathit{Y}|M_1^{(a)})\text{ and }p(\mathit{Y}|M_2^{(a)})$ only have 3 or 5 dimensions, respectively. Hence the Laplace method can effectively be used to integrate over $({\beta }_0,{\beta }_1,{\lambda }_0^{(a)})\text{ in }{M}_1^{(a)}\text{ and }({\beta }_0,{\beta }_1,{\lambda }_0^{(a)},{\lambda }_1^{(a)},{\gamma }_{12})\text{ in }{M}_2^{(a)}$.

We simulated 250 data sets based on a random intercept and slope model, $Y_{\mathit{\text{ij}}}={\beta }_0+b_{i0}+({\beta }_1+b_{i1})x_{\mathit{\text{ij}}}^{*}+{\epsilon }_{\mathit{\text{ij}}}$. We set β₀ = 2.75, β₁ = 3, J = 10, σ² = 1, and b_i ~ N₂(0, ψ) with ${\psi }_{12}=\rho (b_{i0},b_{i1})\sqrt{{\psi }_{11}{\psi }_{22}},{\psi }_{11}=1$, and ρ(b_i0, b_i1) = −0.3. We varied the random slope standard deviation and sample size over $\sqrt{{\psi }_{22}}=0,0.15,0.30,0.45,0.60$ and n = 50, 100, 500, 1000, respectively. We used prior distributions β ~ N(0, 10I), σ² ~ InvGam(1, 1) and γ₁₂ ~ N(0, 1), which are non-informative given the simulation settings.

In general, as the standard deviation of b_i1 increased, our method increasingly favored $M_2^{(a)}\text{ over }{M}_1^{(a)}$. Figure 2 shows box plots of log ${\widehat{B}}_{21}^{(1)}\text{ for }\sqrt{{\psi }_{22}}=0\text{ and }\sqrt{{\psi }_{22}}=0.30$. As the sample size increased, our method more accurately detected the absence of a random slope for $\sqrt{{\psi }_{22}}=0$ and more accurately detected the presence of a random slope for $\sqrt{{\psi }_{22}}>0$. Additional tables of the estimated Bayes factors are available in the Web Appendices.

Figure 2.

Box plot of log ${\widehat{B}}_{21}^{(2)}$, by standard deviation of random slope

We compared our method to the RLRT with α = 0.05. The asymptotic distribution of the RLRT follows a 50:50 mixture of chi-square distributions with 1 and 2 degrees of freedom. Consistent with the ANOVA simulation, we reject H₀ if $B_{21}^{(a)}>1$. As illustrated in Table 1 (columns 7–10), the power of our approach is competitive with the restricted likelihood ratio test. Parameterization (2) is somewhat conservative, while parameterization (1) leads to an inflated Type I error for n = 50. Also, we occasionally ran into numerical problems using parameterization (1).

We conducted additional simulations with the same set of alternative priors given in section 3.3 in order to evaluate the sensitivity of the estimated Bayes factors to the prior distributions. We found similar results and conclude that a prior distribution λ ~ logN(log(0.3), 2) has good frequentist properties for model selection. Additional details are given in the Web Appendices. We also conducted simulations for testing a random intercept and slope simultaneously and found similar performance with respect to power and Type I error (see Web Appendices).

5. Applications

5.1 Hamilton Rating Scale for Depression

We consider 275 patients (160 lamotrigine 200/400 mg/day, 115 placebo) with at least one outcome measurement and complete covariate data. The number of repeated measurements per subject ranges from 1 to 17, and HAMD scores range from 0 to 35 with a mean value of 7. To better approximate normality, we used a square root transformation of HAMD (sqrt-HAMD). We fit a linear mixed model with sqrt-HAMD as the response, predicted by sqrt-HAMD at screening and baseline, time (in years), treatment, gender, age (< 30, 30–40, 40–50, ≥ 50), and the number of depressive or mixed episodes in the last year (1–2 vs. ≥ 3). Screening refers to the time at enrollment and baseline refers to the time of randomization (after stabilization). Investigators wished to learn whether there was individual-specific variation in the time course of depression symptoms, so that models with random intercepts and slopes (M₂), random intercepts only (M₁), and no random effects (M₀) were fit to the data. We center and scale the variable year by two standard deviations. Based on the scale of both the response and the explanatory variables, we use vague priors on the fixed effects and residual variance as β ~ N₉(0, 10I) and σ² ~InvGamma(0.01, 0.01).

The estimated log Bayes factors are log β̂₂₁ = 61.6, log β̂₂₀ = 348.6, and log β̂₁₀ = 287.9. These estimates show strong evidence for M₂ versus the other models, indicating the intercepts and slopes vary significantly by individual. Fitting M₂ using MCMC methods, basing inference on 15,000 samples after discarding a burn-in of 10,000, we plot the predicted mean for a typical subject (45 year old female with 1 depressive episode in past year and average values of HAMD at screening and baseline) in Figure 3, along with predicted individual HAMD scores for 30 random subjects. These predicted individual lines highlight the considerable heterogeneity across subjects. A step-wise RLRT approach rejects H₀ (M₀) versus the alternative H₁ (M₁) and rejects H₁ versus the alternative H₂ (M₂), which agrees with our Bayesian approach in favoring the random slopes model.

Figure 3.

Predicted mean & individual HAMD, with random slope and intercept

Lamotrigine use, age, and sqrt-HAMD at screening and baseline all appear to be significant predictors of the outcome. A one unit increase in sqrt-HAMD at baseline is associated with a 0.63 (95% CI = 0.51, 0.74) increase in mean sqrt-HAMD, and a one unit increase in sqrt-HAMD at screening is associated with a 0.22 (95% CI = 0.02, 0.48) increase in mean sqrt-HAMD. Older patients generally had greater values of sqrt-HAMD than younger patients. As the main association of interest, sqrt-HAMD values for subjects on lamotrigine are on average 0.33 units lower (95% CI = −0.54, −0.10) than sqrt-HAMD values for subjects on placebo. The 95% credible interval does not contain 0, indicating that lamotrigine may be effective at reducing depressive symptoms. These conclusions reinforce the time-to-event analysis of Calabrese et al. (2003).

5.2 Exposure of disinfection by-products in drinking water and male fertility

The three study sites for the disinfection by-product study were chosen due to their different levels of disinfection by-product measures of interest, including THM-Br, HAA-Br, and TOX. However, because the study populations in the three sites differed dramatically by factors that were not well characterized by measured fixed effects, investigators wanted to base inferences on a model that allowed site-specific heterogeneity not only in overall sperm quality but also in the relationships between DBP concentrations and sperm quality. For each DBP, we fit a random intercepts and slopes model (M₂), random intercepts model (M₁), and model with no random effects (M₀), adjusting in all models for age categories, education, and abstinence interval before providing the sample. Each DBP is transformed by subtracting the mean and dividing by two standard deviations. For percent normal sperm, we use a probit transformation multiplied by five so that the transformed response has a range of −10.5 to −1.8, a mean of −5.6, and a variance of 1.8.

Based on the scale of both the response and the explanatory variables, we use vague priors on the fixed effects β and residual variance σ² that accommodate a wide range of reasonable mean values. We define these priors as β ~ N₉(μ, Σ) and σ² ~InvGamma(0.01, 0.01), with $\mathit{\mu }=(-5.5,{\mathbf{0}}_8^{\prime })\prime$ and Σ a diagonal matrix with diagonal elements $(100,10\times {\mathbf{1}}_8^{\prime })$. For HAA-Br and THM-Br, we observe weak evidence for M₁ versus M₂ (B̂₂₁ = 0.66 and B̂₂₁ = 0.31, respectively) and strong evidence for M₁ versus M₀ (B̂₁₀ > 100 for both HAA-Br and THM-Br). For TOX, we observe weak evidence for M₂ versus M₁ (B̂₂₁ = 1.5) and strong evidence for M₂ versus M₀ (B̂₂₀ > 100). We fit M₂ for each DBP using MCMC methods and conduct inference on 40,000 samples after discarding an equal number as a burn-in. We plot the predicted mean response for a 30–35 year old male who has graduated college and has abstained for 2–3 days (Figure 4). One can see that the intercepts appear to vary for all 3 DBPs but the slopes only appear to differ for the TOX exposure. A step-wise RLRT approach rejects H₀ (M₀) versus the alternative H₁ (M₁) and fails to reject H₁ versus the alternative H₂ (M₂) for all three DBP exposures. With the exception of the TOX exposure, this assessment agrees with our Bayesian approach.

Figure 4.

Predicted means of transformed % normal sperm, M₂

Based on M₁, both HAA-Br and THM-Br have posterior distributions centered near 0, indicating little association between these DPB’s and percent normal sperm. Based on M₂, the posterior distribution of TOX tends to be centered below 0 for Galveston but near 0 for Raleigh and Memphis. Hence, increasing TOX exposure may be associated with decreasing values of percent normal sperm among patients in Galveston.

6. Discussion

We recommend our approach as a simple and efficient method for testing random effects in the linear mixed model. Our approach avoids issues with testing on the boundary of the parameter space, uses low-dimensional approximations to the Bayes factor, and incorporates default priors on the random effects. The scaling of the random effects to the residual variance makes the log N(log(0.3) × 1, 2 × I) and N(log(0.3) × 1, 2 × I) distributions reasonable default priors for ${\mathit{\lambda }}_k^{(1)}\text{ and }{\mathit{\lambda }}_k^{(2)}$, respectively. Simulations suggest that these priors have good small sample properties and consistency in large samples. They also have good frequentist properties with respect to Type I error and power. Incorporating reasonable default priors on the fixed effects, our method can be used for comparing a large class of random effects models with varying fixed and random effects.

Alternative procedures for allowing default priors for model selection via Bayes factors are discussed by Berger and Pericchi (1996). These include the authors’ proposed intrinsic Bayes factors, the Schwarz approximation (Schwarz, 1978), and the methods of Jeffreys (1961) and Smith and Spiegelhalter (1980). Gelman (2006) discuss various approaches to default priors specifically for variance components. Common approaches include the uniform prior (e.g. Gelman, 2007), the half-t family of prior distributions, and the inverse-gamma distribution (Spiegelhalter et al., 2003). These prior distributions can encounter difficulties when the variance components are close to 0. Other discussions of selecting default priors on variance components include Natarajan and Kass (2000), Browne and Draper (2006), and Kass and Natarajan (2006).