Bayesian modeling of the dependence in longitudinal data via partial autocorrelations and marginal variances

Summary

Many parameters and positive-definiteness are two major obstacles in estimating and modelling a correlation matrix for longitudinal data. In addition, when longitudinal data is incomplete, incorrectly modelling the correlation matrix often results in bias in estimating mean regression parameters. In this paper, we introduce a flexible and parsimonious class of regression models for a covariance matrix parameterized using marginal variances and partial autocorrelations. The partial autocorrelations can freely vary in the interval (–1, 1) while maintaining positive definiteness of the correlation matrix so the regression parameters in these models will have no constraints. We propose a class of priors for the regression coefficients and examine the importance of correctly modeling the correlation structure on estimation of longitudinal (mean) trajectories and the performance of the DIC in choosing the correct correlation model via simulations. The regression approach is illustrated on data from a longitudinal clinical trial.

1 Introduction

Longitudinal data, measurements on the same subject over time, arise in many areas, from clinical trials to environmental studies. In such studies, to draw valid inferences, the covariance between repeated observations on the same individuals needs to be properly modelled. On top of this, in incomplete longitudinal data, mis-modeling the covariance matrix can result in biased estimates of fixed effect mean parameters (Little and Rubin, 2002; Daniels and Hogan 2008). Two major obstacles for modeling covariance matrices are 1) the number of parameters and 2) positive-definiteness.

Many approaches have been proposed for estimating a covariance matrix more efficiently, whether by shrinking eigenvalues to obtain more stability (Yang and Berger, 1994; Efron and Morris, 1976) or reducing the dimension via structure (Leonard and Hsu, 1992; Chiu et al., 1996; Pourahmadi, 1999; Pourahmadi 2000; Daniels and Zhao, 2003). Some regression approaches to introduce structure and/or unit-specific covariates (Chiu et al., 1996; Hoff and Niu, 2012) can result in difficulty in interpreting regression coefficients due to complex (but unconstrained) parameterizations. This is not our focus here, but the parameterizations in Pourahmadi (1999) and the one given here do not suffer from this issue. There has also been research on shrinkage to introduce stability in structured ways (Daniels and Kass 1999, 2001; Daniels and Pourahmadi 2002) or without structure (Wong et al. 2003; Liechty et al. 2004).

These approaches can often be thought of in terms of specific decompositions of a covariance matrix. Our approach will focus on the variance/correlation decomposition, used recently by Barnard, McCulloch and Meng (2000), which decomposes the covariance matrix Σ as Σ = DRD, where R is correlation matrix and D is diagonal matrix of standard deviations. Our approach here will rely on this decomposition and a further decomposition of the correlation matrix R into partial autocorrelations which we review next.

Consider a p × p correlation matrix R with (j, j + k)th element, the marginal correlation ρ_jj+k ≡ Cor(Y_j, Y_j+k). The matrix R can be re-parameterized using partial autocorrelations, π_jj+k ≡ Cor(Y_j, Y_j+k|Y_l, j < l < j + k). To provide an expression for the partial autocorrelations as a function of the marginal correlations, we first define some notation (similar to Joe (2006)).

Let R^{(j,j+ k)} be the k + 1 × k + 1 submatrix of R which takes elements from the jth row to the (j + k)th row and the jth column to the (j + k)th column. Then we partition R^(j,j+k) as follows,

$$\left(\begin{array}{ccc}\hfill 1\hfill & \hfill {\mathbf{r}}_1(j,k)\hfill & \hfill {\rho }_{j,j+k}\hfill \\ \hfill {\mathbf{r}}_1^T(j,k)\hfill & \hfill R_2(j,k)\hfill & \hfill {\mathbf{r}}_3^T(j,k)\hfill \\ \hfill {\rho }_{j+k,j}\hfill & \hfill {\mathbf{r}}_3(j,k)\hfill & \hfill 1\hfill \end{array}\right),$$

where r₁(j, k) = (ρ_{j, j+1} ,...,ρ_{j, j+ k–1}), r₃(j, k) = (ρ_{j+k, j+1, ..., j+k,j+k–1}), and R₂(j, k) contains the middle k – 1 rows and columns of R^(j,j+k).

The partial autocorrelations have the following form as a function of the marginal correlations,

$$\{\begin{array}{cc}{\pi }_{j,j+1}={\rho }_{j,j+1}\hfill & \hfill \\ {\pi }_{j,j+k}=\frac{{\rho }_{j,j+k}-r_{jk}}{B_{jk}}\hfill & 2\le k\le p-j\hfill \end{array}\phantom{\}}.$$ (1)

where $r_{jk}=r_1(j,k)R_2^{-1}(j,k)r_3^T(j,k)$ and $B_{jk}^2=[1-r_1(j,k)R_2^{-1}(j,k)r_1^T(j,k)][1-r_3(j,k)R_2^{-1}(j,k)r_3^T(j,k)]$. The marginal correlations, ρ_j,j+k can also be written as a simple function of the partial autocorrelations,

$${\rho }_{j,j+k}=r_{jk}+{\pi }_{j,j+k}{B}_{jk}.$$

One of the advantages of this parameterization is that π_jk can vary independently in (–1, 1) while maintaining positive definiteness of R, unlike ρ_jk (see, e.g., Joe 2006). Based on reparameterizing the marginal correlations into partial autocorrelations, Daniels and Pourahmadi (2009) introduce a prior for R induced by independent uniform priors on the partial autocorrelations, i.e., p(π) = 2^{–p(p–1)/2}

After reparameterizing the off-diagonal elements of R = (ρ_jk) in terms of the partial autocorrelations {π_jk}, we transform them to the entire real line using Fisher's z transformation. Moving from the constrained marginal correlations to the unconstrained transformed partial autocorrelations provides a link function framework similar to the theory of generalized linear models in McCullagh and Nelder (1989). The models proposed here will extend recent models from the literature for correlation matrices including the multivariate probit (Czado 2000) and related models (Daniels and Normand 2006).

This article expands on previous work by describing how to do Bayesian inference in the proposed models, including an appropriate choice of priors. Under an improper prior on the mean parameters and monotone (ignorable) missingness, we provide conditions under which the posterior is proper. We also provide simulation results that demonstrate the importance of specifying the correct correlation structure under ignorable missingness and evaluate the ability of the DIC to select the correct model in these situations.

This article is arranged as follows. In Section 2, we introduce regression models for the partial autocorrelations and marginal variances. We derive and investigate priors for the regression parameters for the partial autocorrelation and marginal variance parameters in Section 3. We provide details on posterior computations in Section 4. Results of a simulation study to investigate correlation structure misspecification are given in Section 5. Application of the models to a schizophrenia clinical trial is presented in Section 6. Section 7 provides conclusions and extensions.

2 Models for the covariance matrix

Let Y_i : i = 1, . . . , n be a p × 1 vector of longitudinal responses measured (without loss of generality) at times 1, . . . , p with distribution,

$$Y_i\sim N_p(x_i^T\beta ,{\Sigma }_i),$$ (2)

where β is a vector of (mean) regression parameters with dimension p_β×1, x_i is a p_β×p covariate matrix, and ${\Sigma }_i=D_i{R}_i{D}_i^T$. We build regression models for R_i via the partial autocorrelations and D_i, via the marginal variances in the following subsections.

2.1 Partial autocorrelations

Consider the following regression model for π_i,jk, the jk-th partial autocorrelation for subject i,

$$z\left({\pi }_{i,jk}\right)=w_{i,jk}^{\star }\gamma ,$$ (3)

where z(·) is Fisher's z-transform, $z\left(\pi \right)=\frac{1}{2}\text{log}\frac{1+\pi }{1-\pi }$ and $w_{i,jk}^{\star }$ is a 1 × q vector of covariates to model structure and subject-level covariates; γ is unconstrained in q-dimensional real space ${\mathbb{R}}^q$. Given that the partial autocorrelations are correlations between longitudinal observations, conditional on intermediate ones, we might expect higher order ones to be zero. For example, we might specify $w_{i,jk}^{\star }=I(\mid k-j\mid =1)$ corresponding to an AR(1) structure with all lag partial autocorrelations bigger than one equal to zero. The design vector, $w_{i,jk}^{\star T}=(1,\mid k-j\mid )$ implies that $\tilde{\Pi }$, the corresponding partial autocorrelation matrix, which has 1's on the main diagonal and (j, k)–th element π_jk (for j, k ≠ 1) has a Toeplitz form with the z-transform of the elements on each subdiagonal having a linear relationship in lag. The design vector, $w_{i,jk}^{\star T}=(I\{\mid k-j\mid =1\},I\{\mid k-j\mid =1\}j,I\{\mid k-j\mid >1\},I\{\mid k-j\mid >1\}\mid k-j\mid )$ implies that the vector of π's has nonstationary structure with the z-transform of the lag one correlations a linear function of time and a stationary form for the rest of the matrix, with the z-transform of the elements for each lag (after the first one) having a linear relationship in lag. For related structures for the parameters of the modified Choleski decomposition, see Pourahmadi (1999) and Pourahmadi and Daniels (2002) .

2.2 Marginal Variances

We assume the logarithms of the marginal standard deviations, σ_i,j (i.e., the jth diagonal element of D_i) follow the regression models,

$$\text{log}\left({\sigma }_{i,j}\right)=A_{i,j}\eta ,$$ (4)

where A_i,j is a 1 × q₀ vector of covariates to model structure and unit-level covariates. For example, A(_i,j) = (I(j = 1), I(j > 1)) induces a structure of equal variance except for time 1; A(_i,j) = (1, j) corresponds to the marginal variances that are log linear in time. Verbyla (1993) proposed models for the marginal variances (residual variances) in terms of unit level covariates (i.e., heterogeneity) in the setting of independent responses.

3 Priors for γ

Standard diffuse priors for γ in (3), e.g., an improper uniform prior on ${\mathbb{R}}^q$ or a diffuse normal prior, result in most of the mass for the partial autocorrelations, π_i,jk being put at –1 and +1. This happens in many settings with diffuse priors on transformed spaces, e.g., coefficients in logistic regression (see Agresti and Hitchcock, 2005). These are not sensible prior beliefs. In the next two subsections, we will review a prior proposed for the partial autocorrelations from Daniels and Pourahmadi (2009) and propose alternative ones that both avoid this behavior for an unstructured vector of π's. We then propose a way to use these priors, which are both within the class of independent transformed Beta priors, to construct priors for γ and point out their connections to g-priors. We also construct a similar prior for η in (4).

3.1 Review of priors for unstructured partial autocorrelations

Independent uniform priors on the partial autocorrelations can be expressed as independent Beta priors on the interval (–1, 1), Beta_(–1,1) (a, b), with parameters a = 1 and b = 1. These priors induce desirable behavior for longitudinal (ordered) data by shrinking higher lag marginal correlations toward zero (Daniels and Pourahmadi, 2009) and a priori favoring serial correlation often seen in longitudinal data (Munoz et. al., 1992). The behavior can be understood by examining the determinant of the Jacobian from ρ to π, J(ρ → π) (Joe, 2006),

$$J(\rho \to \pi )=\prod _{k=1}^{p-1}{(1-{\pi }_{k,k+1}^2)}^{-(p-2)∕2}\prod _{k=2}^{p-2}\prod _{j=1}^{p-k}{(1-{\pi }_{j,j+k}^2)}^{-(p-1-k)∕2}.$$

As lag (k) increases, more mass is placed toward zero. This is not surprising since most priors on R(ρ) do not use information on potential ordering of the responses and would induce this prior form on partial correlations to obtain identical marginal priors for the marginal correlations (Barnard et al., 2000). However, the Beta_(–1,1) (1, 1) prior does not favor positive correlations as we typically see in longitudinal data.

3.2 An alternative prior for unstructured partial autocorrelations

Here we introduce a prior on the partial autocorrelations that favors positive correlations over negative correlations. We propose independent priors on π_jk with pdf's, $p\left({\pi }_{jk}\right)=\frac{1+{\pi }_{jk}}{2}$, which is a Beta_(–1,1) (2, 1) distribution; we refer to these as triangular priors given their shape. The implied marginal priors for ρ_jk are given in Figure 1a. The priors have decreasing mass close to 1 as lag (|j – k|) increases. This is consistent with serial correlation often seen in longitudinal data and places more mass on positive correlations than the Beta_(–1,1) (1, 1) priors.

Figure 1.

Marginal priors for the marginal correlations induced by (a) the triangular Prior and (b) the Uniform prior. In (b), the upper triangles are the marginal priors on the ρ's induced by the original priors; the lower triangles are the marginal priors on ρ's after the normal approximation.

In the following section, we will use these two priors as a starting point to construct a prior for the regression coefficients, γ in (3). In the remaining, all Beta priors will be specified on the interval (–1, 1), but we just denote them as Beta(a, b).

3.3 Proposed prior on γ

In the following, when it is not necessary we will drop the subscripts on π_jk. We start by deriving the distribution of z(π_jk) when the π_jk follow independent Beta(1, 1) priors. For this prior on π, $z\left(\pi \right)=z=\frac{1}{2}\text{log}\frac{1+\pi }{1-\pi }$, with pdf,

$$f\left(z\right)=\frac{2e^{2z}}{{(1+e^{2z})}^2},$$

where z ∈ (–∞, +∞). This is the pdf of a logistic distribution, z ~ logistic(0, ½) with variance π/12. It is well known that the logistic distribution can be approximated with a t-distribution (Albert and Chib 1992). However, the easy to use construction of the multivariate t-distribution as a gamma mixture of normals has t-marginals but they are not independent as we required based on our original specification of independent Beta's. As a result, we will use a normal approximation to the logistic distribution, whose multivariate version does have independent marginals, $z\sim N(0,\frac{\pi }{12})$; that is, the random vector $\mathbf{z}\sim N(\underset{¯}{0},\frac{\pi }{12}{I}_{T\times T})$, where $T=\frac{p(p-1)}{2}$.

Figure 1b shows how well the normal prior approximates the original uniform prior on the hypercube (i.e., the independent Beta(1,1) priors) in terms of the marginal correlations. The upper triangular elements represent the marginal priors of ρ_jk from the original uniform prior and the lower triangular elements represent the marginal priors of ρ_jk from the prior based on the normal approximation. The approximate prior appears to behave sufficiently similarly.

Now, we show how this prior can be used to construct a prior for γ in (3). We first focus on the case of z(π_i,jk) = z(π_jk) and for ease of notation, let z_jk = z(π_jk) and z = (z₁₂ , .., z_1p, z₂₃, ...z_2p, ...z_p–1p)^T. Consider the full rank linear transformation z = wγ, where

$$w=\left(w^{\star }{w}^{\perp }\right).$$

w* is a T × q (T ≥ q) full column rank matrix corresponding to the regression in (3). The matrix w^⊥ is a T × (T – q) full column rank matrix such that (w*)^T × (w^⊥) = 0_q×(T–q) and (w^⊥)^T × w^⊥ = I(_{T–q)×(T–q)}.

Therefore

$$\gamma =\left(\begin{array}{c}\hfill {\gamma }^{\star }\hfill \\ \hfill {\gamma }^{\perp }\hfill \end{array}\right)={\left(w^{\star }{w}^{\perp }\right)}^{-1}z,$$

with

$${\left(w^{\star }{w}^{\perp }\right)}^{-1}=\left(\begin{array}{c}\hfill {\left({\left(w^{\star }\right)}^T{w}^{\star }\right)}^{-1}{\left(w^{\star }\right)}^T\hfill \\ \hfill {\left(w^{\perp }\right)}^T\hfill \end{array}\right).$$

We define E(z) = μ and V ar(z) = σ² based on the multivariate normal prior on z. Under the Beta(1,1) prior on π, μ = 0 and σ² = π/12; under the Beta(2,1) prior (triangular prior), μ = ½ and σ² = 0.5722. The corresponding prior for γ is multivariate normal with mean and variance given below,

$$E\left(\gamma \right)=\mu \left(\begin{array}{c}\hfill {\left({\left(w^{\star }\right)}^T{w}^{\star }\right)}^{-1}{\left(w^{\star }\right)}^T\hfill \\ \hfill {\left(w^{\perp }\right)}^T\hfill \end{array}\right)1_{T\times 1}$$

and

$$Var\left(\gamma \right)={\sigma }^2\left(\begin{array}{cc}\hfill {\left({\left(w^{\star }\right)}^T{w}^{\star }\right)}^{-1}\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill I_{(T-q)\times (T-q)}\hfill \end{array}\right).$$

The resulting prior for γ* is also multivariate normal with expectation, μ(w*^Tw*)^–1w*^T1_T×1 and variance, σ²(w*^Tw*)^–1. The dimension reduction from z to γ* results in the prior variance being too small. To see this note that the variance of ith component of z in z = wγ is

$$\begin{array}{cc}\hfill var\left(z_{i\left(\gamma \right)}\right)& =var\left(w_{(i,:)}\gamma \right)\hfill \\ \hfill & ={\sigma }^2(w_{(i,:)}^{\star }{\left({\left(w^{\star }\right)}^T{w}^{\star }\right)}^{-1}{\left(w_{(i,:)}^{\star }\right)}^T+w_{(i,:)}^{\perp }{\left(w_{(i,:)}^{\perp }\right)}^T).\hfill \end{array}$$

The ith component of z = w*γ* has variance

$$\begin{array}{cc}\hfill var\left(z_{i\left({\gamma }^{\star }\right)}\right)& =var\left(w_{(i,:)}^{\star }{\gamma }^{\star }\right)\hfill \\ \hfill & ={\sigma }^2\left(w_{(i,:)}^{\star }{\left({\left(w^{\star }\right)}^T{w}^{\star }\right)}^{-1}{\left(w_{(i,:)}^{\star }\right)}^T\right).\hfill \end{array}$$

Clearly, var(z_i(γ)) > var(z_i(γ*)). It is easy to adjust for this by noting that the average variance of $z\left({\pi }_{jk}\right)=w_{jk}^{\star }{\gamma }^{\star },\overline{var\left(z\right)}$ is

$$\begin{array}{cc}\hfill \overline{var\left(z\right)}& =\frac{1}{T}\sum _{i=1}^{T}var\left(z_i\right)\hfill \\ \hfill & =\frac{q}{T}{\sigma }^2,\hfill \end{array}$$

where σ² is the desired variance. Hence we can inflate var(γ*) by a factor of $\frac{T}{q}$. The resulting prior for γ* is

$${\gamma }^{\star }\sim N(\mu {\left({\left(w^{\star }\right)}^T{w}^{\star }\right)}^{-1}{\left(w^{\star }\right)}^T{1}_{T\times 1},\frac{T}{q}{\sigma }^2{\left({\left(w^{\star }\right)}^T{w}^{\star }\right)}^{-1}).$$

See the supplementary materials for additional details on the above derivations.

3.4 Extension to unit-specific covariates

We can easily extend this prior to unit-specific covariates. Suppose, for i = 1, . . . , n,

$$z\left({\pi }_{i,jk}\right)=w_{i,jk}^{\star }\gamma .$$

Let $w_i^{\star }$ be a p × q matrix such that $z_i=w_i^{\star }\gamma$. We first stack z₁ , ..., z_n and $w_1^{\star },\dots w_n^{\star }$ together, i.e., $z={(z_1^T,z_2^T,\dots ,z_n^T)}^T$, and $w^{\star }={(w_1^{\star T},w_2^{\star T},\dots ,w_n^{\star T})}^T$. So, we have z = w*γ and w* is nT × q full column rank matrix. Similar to the previous case, we obtain

$$\gamma \sim N(\mu {\left(\sum _{i=1}^n{w}_i^{{\star }_T}{w}_i^{\star }\right)}^{-1}{w}^{\star T}{1}_{\mathbf{nT}\times 1},\frac{nT}{q}{\sigma }^2{\left(\sum _{i=1}^n{w}_i^{{\star }_T}{w}_i^{\star }\right)}^{-1}),$$ (5)

which is our recommended prior in the general case.

3.5 Connection to g-priors

Our priors on γ have similar form to the g–priors introduced by Zellner (1986). However, our derivation begins with a prior on an unconstrained parameter space as opposed to Zellner's construction of a prior based on the posterior distribution of imaginary data y₀, y₀ = x^T β + ε where $∊\sim N(0,{\sigma }_0^2{I}_n)$ (with independent priors on β ∝ 1 and ${\sigma }_0\propto \frac{1}{{\sigma }_0}$). The Zellner prior for β|σ₀ has the form $\beta \sim N({\widehat{\beta }}_0,\frac{{\sigma }_0^2}{g}{\left(x{x}^T\right)}^{-1})$, where ${\widehat{\beta }}_0$ is the least squares estimate based on the imaginary data and g is a penalty parameter; in practice, the mean is typically set to zero so no imaginary data is actually required. Our prior has a similar form but it is based on the projection of z(π_i) on $w_i^{\star }$ with weights based on the original prior for π on the unconstrained space (here a hypercube). The ’weights’ based on the prior in (5) come in through μ in the prior mean, ${\left({\sum }_{i=1}^n{w}_i^{{\star }_T}{w}_i^{\star }\right)}^{-1}{w}^{\star T}\mu 1_{\mathbf{nT}\times 1}$ and σ² in the prior variance, σ²(w*^Tw*)^–1. As a result, with these priors, we do not have to deal with the issue of the choice of g (for some discussion, see George and Foster (2000) and Clyde and George (2000)).

3.6 Prior for η

The most commonly used prior on the marginal variances σ² is the inverse gamma prior, which facilitates computations due to conditional conjugacy. Daniels (2006) used a uniform prior on the transformed innovation (IV) parameters with or without structure similar to the model (4) in Section 2.2 for the marginal standard deviations. Barnard et al. (2000) discussed independent normal priors on logarithmic transformed σ. In particular, they proposed the following prior

$$log\left({\sigma }_i\right)\sim N(\xi ,\Lambda ),$$ (6)

with Λ diagonal. We will derive a prior for η similar to that for γ based on Barnard et al.'s prior for the marginal standard deviations. The resulting prior is

$$\eta \sim N(\lambda {\left(\sum _{i=1}^n{A}_i^T{A}_i\right)}^{-1}{A}^T{1}_{\mathbf{np}\times 1},{\tau }^2\frac{np}{q_0}{\left(\sum _{i=1}^n{A}_i^T{A}_i\right)}^{-1})).$$

Note in the derivation, we have assumed ξ = λ1_p×1 and Λ = τ²I_p×p in (6), where λ and τ are fixed a priori.

4 Posterior distribution and computations

The full data likelihood, L(η, β, γ|y) is proportional to

$$\sum _{i=1}^n{\mid D_i\left(\eta \right)R_i\left(\gamma \right)D_i\left(\eta \right)\mid }^{-\frac{1}{2}}\text{exp}\{-\frac{1}{2}tr\left[{\Sigma }_{i=1}^n{R}_i{\left(\gamma \right)}^{-1}{D}_i{\left(\eta \right)}^{-1}(Y_i-x_i\beta ){(Y_i-x_i\beta )}^T{D}_i{\left(\eta \right)}^{-1}\right]\}.$$

We specify the following priors for β, η, and γ,

$$\beta \propto 1$$ (7)

$$\eta \sim N({\lambda }_0{\left(\sum _{i=1}^n{A}_i^T{A}_i\right)}^{-1}{A}^T{1}_{\mathbf{np}\times 1},{\tau }^2\frac{np}{q_0}{\left(\sum _{i=1}^n{A}_i^T{A}_i\right)}^{-1})$$ (8)

$$\gamma \sim N(\mu {\left(\sum _{i=1}^n{w}_i^{\star T}{w}_i^{\star }\right)}^{-1}{w}^{\star T}{1}_{\mathbf{nT}\times 1},\frac{nT}{q}{\sigma }^2{\left(\sum _{i=1}^n{w}_i^{\star T}{w}_i^{\star }\right)}^{-1})$$ (9)

where ${\sum }_{i=1}^n{A}_i^T{A}_i$ and ${\sum }_{i=1}^n{w}_i^{\star T}{w}_i^{\star }$ are non-singular. In the setting of incomplete longitudinal responses, under an assumption of ignorable dropout (monotone missingness), we only need to specify the full data response model and the likelihood of interest is the observed data likelihood, L(β, γ, η|y_obs, x), where the observed data response is y_obs (Daniels and Hogan 2008); the form of the observed data likelihood is given in the supplementary materials.

Since we specify an improper prior on β, we need to prove the posterior distribution of (β, γ, η) is proper. In the next section, we provide a theorem which gives simple sufficient conditions under which the posterior is proper. The supplementary materials contain details on the MCMC algorithm to sample from the posterior distribution.

4.1 Posterior propriety

In the following theorem, we state conditions that are sufficient for the posterior to be proper. First, we need to introduce some notation. Suppose full-data Y_i : i = 1, . . . , n are independently distributed random variables with distribution $Y_i\sim N_p(x_i^T\beta ,{\Sigma }_i)$, where x_i is a p_β × p covariate matrix, β is a p_β×1 (mean) regression parameter vector, ${\Sigma }_i=D_i^{\frac{1}{2}}\left(\eta \right)R_i\left(\gamma \right)D_i^{\frac{1}{2}T}\left(\eta \right)$, and $D_i\left(\eta \right)=diag\left({\sigma }_{i\left(\eta \right)}^2\right)$ with ${\sigma }_{i\eta }^2=({\sigma }_{i1}^2,{\sigma }_{i2}^2,\dots ,{\sigma }_{ip}^2)$ specified by (4) and R_i(γ) specified by (1) and (3); define ${\Im }_{\beta }$, ${\Im }_{\gamma }$, ${\Im }_{\eta }$ to be sample spaces of β, γ, η, respectively. Let (Q_i1 , ..., Q_ip)^T be a vector of observed data indicators, where Q_ik = 1 if Y_ik is observed (0 otherwise). Let $Y_i^{k_i}=\{Y_{ij},j=1,\dots ,k_i\text{where}{k}_i:Q_{i{k}_i}=1,Q_{i,k_i+1}=0\}$ be the vector of observed data (of dimension k_i) for subject i and S_k = {i, Q_iki = 1 and Q_iki+1 = 0, and k_i = k, where 1 ≤ k ≤ p – 1; i = 1, ..n} be the set of subjects with observed data of dimension k.

Theorem 1:

We assume the observed data distribution for the ith subject (i = 1, . . . , n) is $Y_i^{k_i}\sim N_k(x_i^{k_{i}T}\beta ,{\Sigma }_i^{k_i})$, where $x_i^{k_i}=x_{i[:,1:k_i]}$ is a p_β ×k_i submatrix of x_i and ${\Sigma }_i^{k_i}={\Sigma }_{i[1:k_i,1:k_i]}$ is the k_i × k_i principal sub-matrix of Σ_i. We also assume the priors on the parameters are given by (7)-(9) and missingness is monotone and ignorable. Then the posterior of (β, γ, η) will be proper under the following three (easy to check) conditions:

1. ${\sum }_{i\in S_k}{x}_i^{k_i}{x}_i^{k_{i}T}$ is non-singular for all k ∈ {1, 2, . . . , p – 1}.

2. $\sum A_i{A}_i^T$ is non-singular.

3. $\sum w_i^{\star }{w}_i^{\star T}$ is non-singular.

The proof is given in the supplementary materials. Note that the three conditions are conditions for the three design matrices in our model (for the mean, the variance, and the correlations, respectively); the latter two guarantee that the priors (8) and (9) are proper.

5 Simulation

To assess the importance of the correlation structure on estimating the (mean) longitudinal trajectories in incomplete data, we conducted a simulation study. The true model was (2) with p = 6. For each individual, the rows of the mean design matrix were specified as a second order orthogonal polynomial. We set β = (27, –2.3, 0.50)^T. We considered three sample sizes (30, 100, and 400). For each scenario we simulated 200 data sets.

The true models for marginal variances and partial autocorrelation coefficients were given by

$$z\left({\pi }_{jk}\right)={\gamma }_{1}I(\mid k-j\mid =1\bigcap j=1)+{\gamma }_{2}I(\mid k-j\mid =2)+{\gamma }_{3}I(\mid k-j\mid =1\bigcap j>1)$$ (10)

and

$$\text{log}{\sigma }_{jj}={\eta }_{1}I(j=1)+{\eta }_{2}I(j>1),$$

with γ = (0.65, 0.21, 0.85)^T, and η = (150, 200)^T. The structure on π represents a second order model with the lag one partial autocorrelations constant except for time 1, the lag two partial autocorrelations constant over time, and higher lag partial autocorrelations equal to zero. The structure on the variances corresponds to a constant variance over time after time one.

After simulating the complete data, we induce ignorable missingness via the following missing data mechanism,

$$\text{logit}\left\{P(Q_{ik}=1\mid Q_{i,k-1}=1,y_{obs})\right\}={\alpha }_1+{\alpha }_2{y}_{k-1},$$

where Q_jk = I{Y_jk is observed} and α = (3.86, –0.05).

We fit four models to the simulated data. For each model, we use the same true mean and marginal variance models, but different partial autocorrelation models. Our objective is to evaluate the impact of mis-specifying the partial autocorrelation model on inference on the marginal mean regression coefficients, β. Specifically, the models we compare are:

1. True structured model for π given in (10)

2. Independence model, π = 0

3. AR(1) model: z(π_jk) = γ₁I{|k – j| = 1}

4. Unstructured model (no structure on π)

For models 1, 3, and 4, we use a Beta(1,1) distribution in constructing the prior in (5); we note that the simulation results were similar when using a Beta(2,1). For the prior on η, we set (λ, τ² ) = (0, 100). For each model, we compute the DIC (Spiegelhalter et al., 2002) based on the observed data likelihood (Wang and Daniels, 2011).

For each of the simulated datasets, we ran 20, 000 iterations for each of the four models. For each dataset, we compute the DIC for the four correlation models and rank them (1=best to 4=worst) based on their fit (as measured by the DIC). To compare inference on the mean under all four models, we computed the following two quantities: 1) Total MSE, sum of mean squared error of the components of β and 2) Change from Baseline, change of estimated mean responses from time one to time six. We also compare the mean trajectories graphically.

5.1 Results

The simulation results are given in Tables 1 and 2 and Figure 2. As the sample size increases, the estimates for β quickly approach the true value for the true structured correlation model, more slowly for the unstructured correlation model and to the wrong values for the AR(1) and independence correlation models (with the latter with considerable bias) (Table 1). The results in Table 2 are similar with bias in the estimate of change from baseline and larger MSE's for the estimates of the β's for the incorrect models. Graphically, the fitted trajectories can be seen in Figure 2 and illustrate the bias in the fitted trajectory when the correlation structure is incorrect.

Table 1.

Posterior means of β: The values in first row, second row, and third row correspond to posterior means of β₁, β₂, and β₃, respectively. The true value for β is (27.0, –2.0, 0.50).

Sample Size 30				Sample Size 100				Sample Size 400
Unstr	True	AR(1)	Indep.	Unstr	True	AR(1)	Indep.	Unstr	True	AR(1)	Indep.
26.5	26.9	26.7	25.3	26.7	26.9	26.6	25.3	26.9	27.0	26.7	25.4
-2.1	-2.0	-2.1	-2.5	-2.0	-2.0	-2.1	-2.5	-2.0	-2.0	-2.1	-2.5
0.52	0.51	0.50	0.58	0.51	0.51	0.50	0.58	0.50	0.50	0.49	0.57

‘Unstr’, ‘True’, ‘AR(1)’, and ‘Indep.’ represent unstructured model, true model, AR(1) model, and independence model, respectively.

Table 2.

Summary measures from the simulation: The values in rows correspond to sample size 30, 100, and 400, respectively.

Total MSE				Change from Baseline
Unstr	True	AR(1)	Indep.	Unstr	True	AR(1)	Indep.
6.9	6.5	6.6	10.1	12.6	12.0	12.7	15.1
1.8	1.7	2.0	4.8	12.2	12.0	12.7	14.9
0.42	0.41	0.52	3.4	12.1	12.0	12.8	15.0

‘Unstr’, ‘True’, ‘AR(1)’, and ‘Indep.’ represent unstructured model, true model, AR(1) model, and independence model, respectively. ‘Total MSE’ and ‘Change from Baseline’ correspond to the mean square errors of β and the change of the mean responses from the beginning to the end of study (True change is 12.0), respectively.

Figure 2.

Posterior mean of the trajectories for the Unstructured model, True model, AR(1) model, and Independence model with sample size 30, 100, and 400

The DIC chose the true model with high probability, with this probability generally increasing with sample size (see Table 3). For example, for n = 30, 100, 400, the true structured model is chosen by the DIC with probabilities .66, .99, and .95, respectively. It is interesting to note that the main competitor of the true model in the smaller sample sizes (30, 100) is the (parsimonious) AR(1) structure, while for the larger sample size (400) it is the unstructured model. This is the reason that the probability of the true structured model decreases between n = 100 and n = 400. For the larger sample sizes, AR(1) is no longer a reasonable competitor, but the unstructured is (though it is only chosen with probability .06 for n = 400). And note, of course, the unstructured model is correct, but it has more free parameters than needed.

Table 3.

Percentage of times each model is chosen as best (row 1), second best (row 2), third best (row 3), and worst (row 4).

Sample Size 30				Sample Size 100				Sample Size 400
AR(1)	Ind	True	Unstr	AR(1)	Ind	True	Unstr	AR(1)	Ind	True	Unstr
0.35	.00	0.66	.00	.01	.00	.99	.00	.00	.00	.95	.06
0.66	.00	.35	.00	.73	.00	.01	.26	.00	.00	.06	.95
.00	.15	.00	.85	.26	.00	.00	.74	1.00	.00	.00	.00
.00	.85	.00	.15	.00	1.00	.00	.00	.00	1.00	.00	.00

‘AR(1)’, ‘Indep’, ‘True’, and ‘Unstr’ correspond to the true model, AR(1) model, independence model, and the unstructured model, respectively.

6 Data Example: Schizophrenia trial

The data were collected as part of a randomized, double-blind clinical trial for a ”new” pharmacologic treatment of schizophrenia (Lapierre et al. 1990). The trial compared three doses of the ”new” treatment (low, medium, high) to the standard dose of haloperidol, an effective antipsychotic that had known side effects. At the time of the study, the trial was designed to find the appropriate dosing level since the experimental therapy was thought to have similar antipsychotic effectiveness with fewer side effects. Two hundred forty-five patients were enrolled and randomized to one of the four treatment arms. The intended length of follow-up was 6 weeks, with measures taken weekly expect for week 5. Schizophrenia severity was assessed using the Brief Psychiatric Rating Scale (BPRS) a sum of scores of 18 items that reflect behaviors, mode, and feelings. The scores ranged from 0 to 108 with higher scores indicating higher severity. To enter the study, the BPRS score had to be no less than 20. We will illustrate our approach using only the medium dose arm. Of main inferential interest is the change in BPRS from the beginning to the end of the study.

The dropout rate on the medium dose arm was high, with only 40 out of 61 (about 66%) participants having a measurement at week 7 (the sixth measurement time). Reasons for dropout included adverse events (e.g., side effects), lack of treatment effect, and withdrawal for unspecified reasons. The trajectories of completers vs. non-completers is shown in Figure 3a. Clearly those dropping out were doing worse prior to dropping out (higher BPRS).

Figure 3.

Trajectories of the observed data and posterior mean of the trajectories for the models considered.

6.1 Models

Let the longitudinal vector of outcomes for subject i be Y_i = (Y_i1 , ..., Y_i6)^T, measured at weeks t = (t₁ , . . . , t₆ ) = (1, 2, 3, 4, 5, 7). We assume Y_i follows (2) with mean.

$$E\left(Y_{ij}\right)={\beta }_0+{\beta }_1{x}_1^j+{\beta }_2{x}_2^j,$$

where $x_1^j=(t_j-\overline{t})$ and $x_2^j={(t_j-\overline{t})}^2-\frac{{\sum }_{k=1}^6{(t_k-\overline{t})}^3}{{\sum }_{k=1}^6{(t_k-\overline{t})}^2}(t_j-\overline{t})-\frac{{\sum }_{k=1}^6{(t_k-\overline{t})}^2}{6}$, i.e., an orthogonal quadratic polynomial. We assume missingness is ignorable.

We fit the partial autocorrelation models given below:

Independence Model: z(π_jj+k) = 0, log(σ_j) = I(j = 1)η₁ + I(j > 1)η₂ .

AR(1) Model: z(π_jj+k) = I(k = 1)γ₁, log(σ_j) = I(j = 1)η₁ + I(j > 1)η₂.

Unstructured Covariance Model: $z\left({\pi }_{jj+k}\right)={\gamma }_{jj+k}^{(+)}(\text{with}\gamma =({\gamma }_{12}^{(+)},{\gamma }_{13}^{(+)},{\gamma }_{p-1,p}^{(+)})),\text{log}\left({\sigma }_j\right)={\eta }_j$.

Structured Model 1:

$$z\left({\pi }_{jj+k}\right)=I(k=1\cap j<2){\gamma }_1+I(k=1\cap j>1){\gamma }_2+I(k=2){\gamma }_3,log\left({\sigma }_j\right)=I(j=1){\eta }_1+I(j>1){\eta }_2.$$

Structured Model 2:

$$z\left({\pi }_{jj+k}\right)=I(k=1\cap j<2){\gamma }_1+I(k=1\cap j>1){\gamma }_2+I(k=2\cap j<3){\gamma }_3+I(k=2\cap j>2){\gamma }_4+I(k=3){\gamma }_5+I(k=4){\gamma }_6+I(k=5){\gamma }_7,log\left({\sigma }_j\right)=I(j=1){\eta }_1+I(j>1){\eta }_2.$$

Structured Model 1 is the same model as the one considered in the simulation. Stuctured Model 2 is more flexible than Structured Model 1 for the partial autocorrelations allowing nonstationary lag one and lag two autocorrelations and stationary lag three, four and five (with no structural zeros). The structure on the variances is the same as Structured Model 1. The marginal variance structure for all the models and the partial autocorrelation structures for Structured Models 1 and 2 were chosen after examining the unstructured covariance matrix in Table 4.

Table 4.

MLE of variances (on main diagonal), partial autocorrelations (in upper triangle), and Fisher's z-transformation of partial autocorrelations (lower triangle).

126.25	0.6578	-0.0738	0.0804	-0.0253	-0.5230
0.7889	210.35	0.8543	-0.0593	-0.3328	0.0292
-0.0740	1.2718	224.42	0.8559	0.2648	0.4375
0.0806	-0.0594	1.2779	240.84	0.8961	0.3506
-0.0253	-0.3460	0.2713	1.4522	221.98	0.8433
-0.5805	0.0292	0.4692	0.3661	1.2325	243.08

We use priors specified in (7), (8), and (9) for β, η, γ, respectively. For (9), we consider both the Beta(1, 1) (uniform) and Beta(2, 1) (triangular) for the specification of (μ, σ²). For (8), we set λ = 0 and τ² = 100.

6.2 Results

For all models, we ran 200,000 iterations with minimal burn-in since the chains converged after a few iterations. The plot of all fitted mean trajectories is given in Figure 3b (only shown under the triangular prior). The mean BPRS initially decreased but started to go back up by week 5. This is related to those dropping out doing more poorly than those staying in the study. Table 5 contains the posterior mean of β, the change from baseline to week 7, their 95% credible intervals, and the DIC based on the observed data likelihood. This table shows that models under uniform and triangular prior fit similarly. The changes from baseline in all models were negative with 95% credible interval excluding 0, showing that Medium-dose reduced the BPRS score significantly, which agrees with earlier analysis done in Daniels and Hogan (2008). The changes from baseline varied from –14 to –11 based on the covariance model chosen. According to the DIC, Structured Model 2 with the triangular prior provided the best fit. The change from baseline in Structured Model 2 was almost a full point different from the unstructured model.

Table 5.

Posterior summaries of the models for the schizophrenia trial.

	(β0, β1, β2)	Changes from Baseline (95% CI)	DIC
Independent	(25.6, -2.35, 0.68)	-14.1 (-18.7, -9.4)	1924.6
Unstructured	(26.9, -2.03, 0.62)	-12.2 (-16.3, -7.9)	1663.0
AR(1) (Uniform)	(27.5, -1.97, 0.69)	-11.8 (-15.8, -7.9)	1681.3
AR(1) (Triangular)	(27.5, -1.97, 0.69)	-11.8 (-15.7, -7.9)	1680.6
Structured Model 1 (Uniform)	(27.9, -1.80, 0.58)	-10.8 (-14.6, -7.0)	1669.6
Structured Model 1(Triangular)	(27.9, -1.81, 0.58)	-10.8 (-14.7, -7.0)	1670.4
Structured Model 2(Uniform)	(27.9, -1.89, 0.54)	-11.3 (-15.3, -7.3)	1660.4
Structured Model 2(Triangular)	(27.9, -1.89, 0.55)	-11.3 (-15.4, -7.3)	1658.1

7 Discussion

In this paper, we first extended the priors in Daniels and Pourahmadi (2009) for partial autocorrelations for the unstructured case by introducing a set of (triangular) priors which favor positive marginal correlations. Using Fisher's z-transformation on the partial autocorrelations, we introduced a regression framework to induce structure and/or unit-specific covariates in the correlation matrix. Based on priors proposed for the partial autocorrelations in the non-regression setting, we introduced a prior for the coefficients in the partial autocorrelation regressions (and for the coefficients of the marginal variance regressions). We conducted simulations that illustrated the importance of correct specification of the correlation structure in the setting of ignorable missingness in longitudinal data and show the ability of the DIC to choose the true correlation model. We also fit the models to data from a longitudinal schizophrenia clinical trial.

There are a variety of extensions to the modeling proposed here. Clearly, it can be difficult to ‘find’ a good parametric model that imposes structure on the correlation matrix. Thus extending approaches developed under different parameterizations (Smith and Kohn, 2002; Wong, Carter, and Kohn, 2003) to our setting is an important extension. Correlation matrices (instead of covariance matrices) arise commonly in models for longitudinal data modeled using Gaussian copulas (Nelsen, 1999); efficient computations using the partial autocorrelation in these settings will be a challenging problem due to the lack of conjugacy. However, the partial autocorrelation models provide an opportunity for flexible dependence in longitudinal categorical data via multivariate probit models. To offer some robustness to a selected model for the correlation structure, an alternative would be to shrink the partial autocorrelations to the structure using independent Beta priors as has been done previously using normal priors on other parameterizations of a covariance matrix (Daniels and Kass, 2001; Daniels and Pourahmadi, 2002). Finally, we are considering extensions to irregular longitudinal data using a partial autocorrelation function and to time-dependent covariates.