Modeling a mixture of ordinal and continuous repeated measures

Abstract

We study the correlation structure for a mixture of ordinal and continuous repeated measures using a Bayesian approach. We assume a multivariate probit model for the ordinal variables and a normal linear regression for the continuous variables, where latent normal variables underlying the ordinal data are correlated with continuous variables in the model. Due to the probit model assumption, we are required to sample a covariance matrix with some of the diagonal elements equal to one. The key computational idea is to use parameter-extended data augmentation, which involves applying the Metropolis-Hastings algorithm to get a sample from the posterior distribution of the covariance matrix incorporating the relevant restrictions. The methodology is illustrated through a simulated example and through an application to data from the UCLA Brain Injury Research Center.

1. Introduction

Longitudinal and repeated measures data are extremely common in nearly every field of science. Quite often, a portion of the data vector for an individual consists of ordinal measures. Repeated ordinal data can be treated using a multivariate version of the probit regression model, which assumes that each ordinal measure arises as a discretized version of a latent continuous score and that these latent continuous scores have a multivariate normal distribution with the dependence described by a correlation matrix. This model was studied from a Bayesian standpoint [1]. Two of the biggest difficulties that arise are: (i) choosing a prior distribution for the correlation matrix $R$ of the latent continuous measures and (ii) finding an appropriate sampling step for $R$ in an MCMC algorithm. Chib and Greenberg tackle these problems through the use of a truncated normal prior distribution on the vector of unique elements of $R$ and carefully-tuned proposal distributions for the elements of $R$. Other authors [2, 3] have proposed solutions based on reparametrization. In previous work, [4], we have described a method that allows for unified prior modeling and sampling of the correlation matrix. The method is motivated by the idea of parameter expansion [5] and uses an artificial augmentation of $R$ into a general covariance matrix.

In this paper, we seek to extend this model to data that are a mixture of ordinal and continuous components. Schafer [6] considers the related problem of modeling a mixture of nominal categorical and continuous measure by use of the general location model. The model specifies that the categorical measures follow a log-linear (contingency table) model and that, conditional on the cell of the table, the continuous data have a multivariate normal distribution with different parameters in each cell of the table. In practice, the covariance matrix of the multivariate normal is usually assumed to be constant across cells. Also, the general location model does not perform well when there are sparse cells in the contingency table of the categorical data. The approach that we outline in this paper is quite different as it explicitly establishes a joint model for the set of measures and can allow for arbitrary covariate specifications for both ordinal and continuous data.

The dependence of our multiple measures is modeled through a multivariate normal distribution for latent continuous scores (corresponding to the ordinal measures) and the continuous measures. A general theoretical framework for this modeling approach is called copula modeling. A good recent reference is [7], which studies multivariate Gaussian copula models in an exponential family setting. Inferential theory is described under a generalized estimating equation approach.

Section 2 gives the notation and details for our model. In section 3, we describe the treatment of the covariance structure for the repeated measures. We present a simulated example in section 4 and an application to data on long-term outcome for severe head trauma patients in section 5. Section 6 discusses an extension of this model to allow for longitudinal modeling of the measures. Finally, we summarize the paper and future research directions in section 7.

2. Generalized multivariate probit model

We assume that the data vector $T_i^{\text{T}}=\left(v_i^{\text{T}},c_i^{\text{T}}\right)$ for $i=1,\dots ,N$ consists of a continuous portion $v_i^{\text{T}}=\left(v_{i1},v_{i2},\dots ,v_{i{p}_1}\right)$, with length $p_1$ and an ordinal portion $c_i^{\text{T}}=\left(c_{i1},c_{i2},\dots ,c_{i{p}_2}\right)$, with length $p_2\left(p_1+p_2=p\right)$.

Suppose $v_i$ follows a multivariate normal distribution with mean equal to $X_i^{(1)}\beta$, where $X_i^{(1)}$ is a $p_1$ by $k$ design matrix and $\beta$ is the normal regression parameter column vector with $k$ dimensions, and covariance matrix equal to ${\Sigma }_{vv}$.

We treat the ordinal data in the multivariate probit model framework [1, 8], we assume there is an underlying normal latent variable $z_i$ for each $c_i$, for $i=1,\dots ,N$. The element $c_{ij}$ takes values on the discrete set $0,1,\dots ,J_i-1$, and $c_{ij}=l$ if and only if the latent variable $z_{ij}$ is in the range $\left({\gamma }_{j,l-1},{\gamma }_{j,l}\right]$ where ${\gamma }_{j,l}$ are the set of cut-points, for $j=1,\dots ,p_2$ and $l=0,\dots ,J_j-1$. We set ${\gamma }_{j,0}=-\infty ,{\gamma }_{j,J_j-1}=+\infty$ for notational simplicity, and ${\gamma }_{j,1}=0$ for identifiability of the cut-points. The vector $c_i$ follows a multivariate probit model if $z_i~N_{p_2}\left(X_i^{(2)}\beta ,R_{zz}\right)$, where $X_i^{(2)}$ is the $p_2$ by $k$ design matrix for the linear model and $R_{zz}$ is a $p_2$ by $p_2$ correlation matrix.

Let $\Lambda$ denote the covariance matrix of the $y_i^{\text{T}}=\left(v_i^{\text{T}},z_i^{\text{T}}\right)$ vectors

$$\Lambda =\left(\begin{array}{cc}{\Sigma }_{vv}& {\Sigma }_{vz}\\ {\Sigma }_{vz}^T& R_{zz}\end{array}\right)=\left(\begin{array}{cc}{D}_{vv}^{1/2}{R}_{vv}{D}_{vv}^{1/2}& D_{vv}^{1/2}{R}_{vz}\\ R_{vz}^T{D}_{vv}^{1/2}& R_{vv}\end{array}\right),$$ (1)

where $R_{vv}$ is the corresponding correlation matrix for ${\Sigma }_{vv}$ and $D_{vv}$ is the diagonal matrix of variances of ${\Sigma }_{vv}$.

To model the $v_i$ and the $z_i$ simultaneously, we assume that given the latent vector $z_i$, the vector $y_i^{\text{T}}=\left(v_i^{\text{T}},z_i^{\text{T}}\right)~N_p\left({\text{X}}_i\beta ,\Lambda \right)$, where ${\text{X}}_i$ is vertically partitioned as ${\text{X}}_i^{(1)}$ and ${\text{X}}_i^{(2)}$, and $\Lambda$ is the covariance matrix of the vector ($v_i,z_i$) as defined in equation (1). In a Bayesian implementation of this model, we require a joint prior distribution for $\beta ,\Lambda$, and the cutpoints ${\gamma }_{j,l}$. For simplicity, we will assume the three components are independent a priori. The prior distribution for the regression parameters, $p(\beta )$, will be assumed normal given fixed hyperparameters, i.e., $\beta ~N_k\left(\beta \mid b_0,B_0\right)$. For the ragged array $\left\{{\gamma }_{j,l},j=1,\dots ,p_2,l=\right.\left.2,\dots ,J_j-1\right\}$, we typically assume a non-informative prior $p\left({\gamma }_{j,l}\right)\propto 1$. The prior distribution for $\Lambda$ will be discussed in section 3.

Let $Z=\left(z_1,\dots ,z_N\right),V=\left(v_1,\dots ,v_N\right)$ and $C=\left(c_1,\dots ,c_N\right)$. Then, the joint posterior density of $\beta ,\Lambda ,\gamma ,Z$ is

$$\begin{gathered} p(\beta ,\Lambda ,\gamma ,Z\mid V,C)\propto p(\beta ,\Lambda ,\gamma )p(Z,V\mid \beta ,\Lambda ,\gamma )p(C\mid \beta ,\Lambda ,\gamma ,Z,V) \\ =p(\beta ,\Lambda ,\gamma )p(Z,V\mid \beta ,\Lambda ,\gamma )p(C\mid Z) \\ =p\left(\beta \right)p\left(\Lambda \right)p\left(\gamma \right)\prod _{i=1}^N\left[I_i\times N\left(y_i\mid X_i\beta ,\Lambda \right)\right], \end{gathered}$$

where the indicator function $I_i$ indicates the compatibility of $c_i$ and $z_i$. To implement the Gibbs sampler, the full conditional distributions can be described as follows:

• For the regression parameters, standard Bayesian linear model results imply that

  $$\beta \mid \Lambda ,\gamma ,Z,V,C~N\left(\hat{\beta },V_{\beta }\right),$$
  where

  $$V_{\beta }={\left(\left(\sum _{i=1}^N\left(X_i^T{\Lambda }^{-1}{X}_i\right)\right)+B_0^{-1}\right)}^{-1}\text{and}\hat{\beta }=V_{\beta }\left(\sum _{i=1}^N\left(X_i^T{\Lambda }^{-1}{y}_i\right)+B_0^{-1}{b}_0\right).$$
• For the latent variables, $z_{ij}\mid \beta ,\Lambda ,\gamma ,v_i,c_i,z_{ik},k\ne j$ have interval truncated normal distributions. The truncation is chosen so that $z_{ij}$ is constrained to lie between the two cut-points compatible with $c_{ij}$.

  $$p\left(z_{ij}\mid \beta ,\Lambda ,\gamma ,v_i,c_i,z_{i,k},k\ne j\right)\propto I_{ij}\times N\left(z_{ij}\mid \widetilde{{\mu }_{ij}},\widetilde{{\lambda }_{ij}}\right),$$
  where $I_{ij}={\sum }_{k=1}^J{1}_{\left(c_{ij}=k\right)}{1}_{\left({\gamma }_j(k-1)<z_{ij}<{\gamma }_{jk}\right)},\widetilde{{\mu }_{ij}}$ and $\widetilde{{\lambda }_{ij}}$ conditional mean and variance of $z_{ij}$ given $v_i,z_{ik},k\ne j$. To give explicit formulas for these, recall that the vector $y_i=\left(v_i,z_i\right)$, and thus the element $z_{ij}$ is the $j^{\prime }\equiv \left(p_1+j\right)$th element of $y_i$. Let $x_{i{j}^{\prime }}$ be the $k$ by 1 covariate vector for the $i$th subject’s $j$ th ordinal measure. We have

  $$\begin{gathered} \widetilde{{\mu }_{ij}}=x_{i{j}^{\prime }}^T\beta +{\Lambda }_{j^{\prime },-j^{\prime }}{\Lambda }_{-j^{\prime },-j^{\prime }}^{-1}\left(y_{i,-j^{\prime }}-X_{i,-j^{\prime }}\beta \right) \\ \widetilde{{\lambda }_{ij}}={\Lambda }_{j^{\prime },j^{\prime }}-{\Lambda }_{j^{\prime },-j^{\prime }}{\Lambda }_{-j^{\prime },-j^{\prime }}^{-1}{\Lambda }_{-j^{\prime },j^{\prime }}, \end{gathered}$$

  where ${\Lambda }_{i,-i}$ means the vector of the $i$th row of $\Lambda$ without its $i$th column element, ${\Lambda }_{-i,i}$ is the transpose of ${\Lambda }_{i,-i},{\Lambda }_{-i,-i}$ is the $\Lambda$ matrix without its $i$th row and $i$th column, $X_{i,-j^{\prime }}$ is the matrix $X_i$ without the $j$th row, and $y_{i,-j}$ is the vector $y_i$ without its $j$th element.

• For the cut-points of the latent variables, the full conditional distribution for ${\gamma }_{j,l},j=1,\dots ,p_2,l=2,\dots ,J_j-1$ is found by

  $$\begin{gathered} p\left({\gamma }_{j,l}\mid \beta ,\Lambda ,Z,V,C,{\gamma }_{j,k},k\ne l\right)=p\left({\gamma }_{j,l}\mid z_{ij},c_{ij},i=1,\dots ,N,{\gamma }_{j,k},k\ne l\right)\propto \prod _{i=1}^N\left[1_{\left(c_{ij}=l\right)}{1}_{\left({\gamma }_{j,l-1}<z_{ij}<{\gamma }_{j,l}\right)}+1_{\left(c_{ij}=l+1\right)}{1}_{\left({\gamma }_{j,i}<z_{ij}<{\gamma }_{j,l+1}\right)}\right] \\ =U\left({\gamma }_{j,l}\mid \text{max}\left\{\text{max}\left\{z_{ij}:c_{ij}=l\right\},{\gamma }_{j,l-1}\right\},\text{min}\left\{\text{min}\left\{z_{ij}:c_{ij}=l+1\right\},{\gamma }_{j,l+1}\right\}\right). \end{gathered}$$

  In other words, the cut-point is uniformly sampled from the widest possible interval that is compatible with other cut-points and the ordinal data.

• For the covariance matrix of the latent variables, $p(\Lambda \mid \beta ,\gamma ,Z,V,C)$ is proportional to $p(\Lambda ){\prod }_{i=1}^{N}N\left(y_i\mid X_i\beta ,\Lambda \right)$. Since $\Lambda$ is a covariance matrix with the restriction that some of the diagonal elements are equal to 1, we cannot use either methods for sampling an unrestricted covariance matrix nor methods for sampling a true correlation matrix. In the following section, we will generalize the parameter-extended Metropolis-Hastings algorithm [4] to sample $\Lambda$.

3. Bayesian analysis for mixture of continuous and ordinal repeated measures

3.1. Parameter-extended Metropolis-Hastings (PX-MH) algorithm

We first review the results of Zhang et al. [4] for the case where we have only the ordinal data (so that $p_1=0$ and $\Lambda =R_{zz}\equiv R$).

3.1.1. PX-MH algorithm

1. Set initial value of ($R^{(0)},D^{(0)}$) through setting $W^{(0)}=D^{(0)(1/2)}{R}^{(0)}{D}^{(0)(1/2)}$ to an initial covariance matrix.

   $$\text{For}i=1,\dots ,M:$$

2. Generate ($R^{*},D^{*}$) by generating $W^{*}=D^{*1/2}R{D}^{*1/2}$ from Wishart $p\left(m,W^{(i)}\right)$.

3. Take

   $$\left(R^{(i+1)},D^{(i+1)}\right)=\left\{\begin{array}{ll}\left(R^{*},D^{*}\right)& \text{with}\text{probability}\alpha ,\\ \left(R^{(i)},D^{(i)}\right)& \text{otherwise},\end{array}\right.$$

   where $\alpha =\text{min}\left\{\left(p\left(R^{*},D^{*}\mid Y\right)q\left(W^{(i)}\mid W^{*}\right)\right)/\left(p\left(R^{(i)},D^{(i)}\mid Y\right)q\left(W^{*}\mid W^{(i)}\right)\right),1\right\},q\left(W^{*}\mid W^{(i)}\right)$ is the proposal density function, which is equal to Jacobian $(W\to R,D)$ Wishart $p\left(W^{*}\mid m_0,W^{(i)}\right)$ (i.e., the Wishart density function times the Jacobian term for the transformation from $W$ to $(R,D)$), and $p\left(R^{(i)},D^{(i)}\mid Y\right)$ is the posterior density of $(R,D)$. Alternatively, we could use an inverse-Wishart jumping rule for this step. We name the density function implied on $(R,D)$ by placing a Wishart $(\nu ,\Omega )$ density on $W$, the PXW ($\nu ,\Omega$) density (PXIW if $W$ has an inverse-Wishart density).

Back in the context of the multivariate probit model, we add the artificial variance components $D$ to the sampler. The joint posterior density is now

$$\begin{gathered} p(\beta ,R,Z,D\mid C)\propto p(C\mid \beta ,R,Z,D)p(Z\mid \beta ,R,D)p(\beta ,R,D) \\ =p(C\mid Z)p(Z\mid \beta ,R)p(\beta )p(R,D). \end{gathered}$$

The full conditionals for $\beta$ and $Z$ do not depend on $D$, so they are the same as in section 2. The joint posterior density of $R$ and $D$ given $Z,\beta ,C$ is proportional to $p(Z\mid \beta ,R)p(R,D)$. The PX-MH algorithm can then be used to sample the pair $(R,D)$.

Besides being a convenient proposal density, the PXW distribution is intuitive to use as a prior distribution for ($R,D$). The prior scale $\Omega$ can be set to $R_0/\nu$, where $R_0$ is a prior guess for $R$. Choosing an appropriate prior distribution for the covariance structure in our model is similar to the choice of an appropriate parametric dependence structure when fitting repeated measures models in standard software packages. In both situations, an analyst should be guided by a combination of experience and common sense. In parametric selection, competing models can be compared using the AIC or BIC model criteria. For the PXW distribution, Bayesian model fit criteria can certainly be computed, however a big advantage of our approach compared with the parametric covariance modeling is that the posterior inference for the covariance matrix is not constrained to belong to any particular parametric family. Rather, a sensible prior centre coupled with a small value for the prior degrees of freedom anchors the covariance inference in a useful neighbourhood.

As a natural extension, it is possible to use a hierarchical model for the prior centre [9], by allowing $\Omega (\theta )$ to be a parametric family of correlation matrices parametrized by $\theta$. The model would then be completed with a prior distribution on $\theta$.

3.2. Generalized parameter-extended Metropolis-Hastings (GPX-MH) algorithm

The parameter-extended random-walk algorithm of the previous section can be generalized to the situation where only a portion of the covariance matrix contains restrictions. We denote the generalized algorithm as GPX-MH. To sample $\Lambda$, we expand the parametrization of $\Lambda$ to include a diagonal scale matrix $D_{zz}$ for $z_i$. We call the expanded matrix $\Sigma$, which can be written

$$\Sigma =\left(\begin{array}{cc}{\Sigma }_{vv}& {\Sigma }_{vz}{D}_{zz}^{1/2}\\ D_{zz}^{1/2}{\Sigma }_{vz}^{\text{T}}& D_{zz}^{1/2}{R}_{zz}{D}_{zz}^{1/2}\end{array}\right)=\left(\begin{array}{ll}{D}_{vv}^{1/2}{R}_{vv}{D}_{vv}^{1/2}& D_{vv}^{1/2}{R}_{vv}{D}_{zz}^{1/2}\\ D_{zz}^{1/2}{R}_{vz}^{\text{T}}{D}_{vv}^{1/2}& D_{zz}^{1/2}{R}_{zz}{D}_{zz}^{1/2}\end{array}\right).$$

Therefore, we can write $\Sigma =D^{1/2}R{D}^{1/2}$, where

$$D=\left(\begin{array}{cc}{D}_{vv}& 0\\ 0& D_{zz}\end{array}\right)\text{and}R=\left(\begin{array}{cc}{R}_{vv}& R_{vz}\\ R_{vz}^{\text{T}}& R_{zz}\end{array}\right).$$

Let $Z=\left(z_1,\dots ,z_N\right),V=\left(v_1,\dots ,v_N\right)$ and $C=\left(c_1,\dots ,c_N\right)$. Then the joint posterior density is

$$\begin{gathered} p(\beta ,R,D,\gamma ,Z\mid V,C)\propto p(\beta ,R,D,\gamma )p(Z,V\mid \beta ,R,D,\gamma )p(C\mid \beta ,R,D,\gamma ,Z,V) \\ =p(\beta ,R,D,\gamma )p(Z,V\mid \beta ,R,D,\gamma )p(C\mid Z) \\ =p\left(\beta \right)p\left(R,D\right)p\left(\gamma \right)\prod _{i=1}^N\left[I_i\times N\left(y_i\mid X_i\beta ,\Lambda \right)\right]. \end{gathered}$$

The full conditional distributions $\beta ,z_{ij}$ and ${\gamma }_{mm}$ are the same as in section 2. Based on the above joint posterior density, we have $p(R,D\mid \beta ,\gamma ,Z,V,C)$ is proportional to $p(R,D){\prod }_{i=1}^{N}N\left(y_i\mid X_i\beta ,\Lambda \right)$. As above, we can use the PXW distribution to define a prior density on $(R,D)$. The density function will be the product of the Jacobian for transforming $\Sigma$ to $(R,D)$ times a Wishart density for $\Sigma$. More specifically, we give $(R,D)$ a PXW $\left(m_0,\Omega \right)$ prior distribution, where $\Omega /m_0$ is a prior guess for $\Lambda$. We emphasize that $D_{zz}$ are the extended parameters and $D_{vv}$ are the intrinsically meaningful variance components in $\Lambda$. Then, we use the parameter-extended Metropolis–Hastings algorithm to sample $R$ and $D$ together.

3.3. Gibbs sampling with missing outcomes

Suppose $v_i$ has missing components denoted by $v_i^m$ and observed components denoted by $v_i^o$, and that without loss of generality, the $v_i$ vector is ordered as $v_i=\left(v_i^o,v_i^m\right)$ (we omit the transpose symbols in the following sections for notational simplicity). Similarly, $c_i=\left(c_i^o,c_i^m\right)$. Corresponding to $c_i$, we write $z_i=\left(z_i^o,z_i^m\right)$ although both $z_i^o$ and $z_i^m$ are unobserved latent variables.

Let $V^m=\left(v_1^m,\dots ,v_N^m\right),V^o=\left(v_1^o,\dots ,v_N^o\right)$, and $C^o=\left(c_1^o,\dots ,c_N^o\right)$. Then the joint posterior density of $\beta ,\gamma ,R,D,Z,V^m$ is

$$\begin{gathered} p\left(\beta ,R,D,\gamma ,Z,V^m\mid V^o,C^o\right)\propto p(\beta ,R,D,\gamma )p(Z,V\mid \beta ,R,D,\gamma )p\left(C^o\mid \beta ,R,D,\gamma ,Z\right) \\ =p(\beta ,R,D,\gamma )p(Z,V\mid \beta ,\Sigma ,\gamma )p\left(C^o\mid Z^o\right) \\ =p(\beta )p(R,D)p(\gamma )\prod _{i=1}^N\left[I_i\times N\left(y_i\mid X_i\beta ,\Lambda \right)\right]. \end{gathered}$$

The MCMC sampling algorithm is the same as in section 2 except

• $z_{ij}^o\mid \beta ,\gamma ,R,D,c_{ij}^o,v_i,z_i^m,z_{ik}^o,k\ne j$ have univariate normal distributions truncated at the interval $\left({\gamma }_{j,k-1},{\gamma }_{j,k}\right)$, if $c_{ij}^o=k$ with the conditional mean and variance given in section 2.

• $z_{ij}^m\mid \beta ,\gamma ,R,D,v_i,z_i^o,z_{ik}^m,k\ne j$ have univariate normal distributions having the usual conditional mean and variance without truncation.

• $v_{ij}^m\mid \beta ,\gamma ,R,D,z_i,v_i^o,v_{ik}^m,k\ne j$ have univariate normal distributions having the usual conditional mean and variance without truncation.

4. A simulated example

We now investigate the behaviour of this model for a small data set, for which we expect that the choice of prior distribution for $\Lambda$ may affect our inference to a large extent. We use a total sample size of $N=50$ with $p=5$ repeated measures. We assume the multivariate probit model for repeated categorical outcomes $c_i=\left(c_{i1},c_{i2}\right)$ for $i=1,\dots ,N$. We let each $c_{ij}$ take one of 4 ordered categories, 0, 1, 2, 3 for each $i=1,\dots ,N$, and $j=\mathrm{1,2}$. For simplicity, we set the parameter vector $\left({\beta }_0,{\beta }_1\right)=(\mathrm{0,0})$ and $x_{ij}=(1,j-3{)}^T$ to represent a linear trajectory design matrix with zero intercept and slope.

We assume the following covariance matrix $\Lambda$ for $y_i=\left(v_i,z_i\right)$:

For $i=1,\dots ,N$ we generate a multivariate normal variable $y_i=\left(y_{i1},\dots ,y_{ip}\right)$ with mean vector equal to 0 and covariance matrix equal to $\Lambda$. We set $v_{ij}=y_{ij}$ for $i=1,\dots ,N$, and $j=\mathrm{1,2},3$ and $c_{ij}=0$ if $y_{ij}<0,c_{ij}=1$ if $0<y_{ij}<0.5,c_{ij}=2$ if $0.5<y_{ij}<1$, and $c_{ij}=3$ if $y_{ij}>1$ for $i=1,\dots ,N$ and $j=\mathrm{4,5}$. In other words, the categorical data are created from $y_{ij}$ by discretizing at the the cut-points $\left({\gamma }_{j1},{\gamma }_{j2},{\gamma }_{j3},{\gamma }_{j4},{\gamma }_{j5}\right)=(-\infty ,\mathrm{0,0.5,1.0},\infty )$ for $j=\mathrm{4,5}$.

We now compare the estimates of the regression coefficients, cutpoints, variances, and correlation parameters under two different prior distributions for $R$ and $D$: the $\text{PXW}(m_0=8,I)$ prior distribution, which assumes a prior mean with zero correlation and the $\text{PXW}\left(m_0=\right.8,\text{AR}(1)(0.5))$ prior distribution which assumes a prior mean with moderate AR(1) correlations (see figure 1). The rationale for comparing these specific choices is that (i) the autoregressive model is somewhat but not entirely appropriate (i.e., the correlations in $\Lambda$ are largest near the diagonal and smallest in the corners, but they do not decay in powerlaw fashion) and (ii) the identity model is not appropriate as all the correlation elements are non-zero. We considered both a complete data set, and a data set which had 20% of the $T_{ij}$ values missing completely at random. We ran the MCMC sampler for 20,000 iterations. The Metropolis-Hastings step used a jumping degrees of freedom of $v_0=200$ and achieved an acceptance rate of ~18%. Although this is below the value of 23% recommended by Gelman et al. [10], we found that higher values for $v_0$ substantially increased the autocorrelations. We calculated Gelman and Rubin’s potential scale reduction factor, $\sqrt{\hat{R}}$ for five dispersed chains with the first 1,000 iterations discarded as burn-in [11]. All diagnostics indicated excellent convergence of the algorithm. The values of $\sqrt{\hat{R}}$ for the two regression parameters, ten correlations, three variances, and four ordinal cutpoints were all below 1.04 after 10,000 iterations and declined consistently through a further 10,000 iterations. The multivariate potential scale reduction factor [12] for these 21 parameters was 1.06 after 10,000 iterations, improving to 1.02 at 20,000 iterations. This is a fairly large number of parameters for which, to calculate the MPSRF, and a value this low is an indicator of excellent convergence in this example.

Figure 1.

The density plots under the $\text{PXW}\left(m_0=8,\text{AR}(1)(0.5)\right)$ prior distribution for each correlation parameter (in the complete case data set), showing the extent of information contained in a small data set about $R$. The solid lines are the posterior density plots and the dashed lines are the prior density plots.

The posterior estimated regression parameters and the standard deviations are shown in table 1. We can see that the estimated means for these parameters are quite close to their true values. Neither the choice of prior distribution nor the presence of missingness has a great effect on our inference for $\beta$.

Table 1.

Posterior means and standard deviations for the elements of regression parameters with and without missing values under two different priors.

Q-	${\mathit{\beta }}_0$	${\mathit{\beta }}_1$
True	0.0	0.0
MComI	0.03	0.04
(Std)	(0.12)	(0.06)
MComA	0.04	0.04
(Std)	(0.13)	(0.06)
MMisI	−0.02	0.05
(Std)	(0.13)	(0.06)
MMisA	−0.01	0.04
(Std)	(0.13)	(0.06)

MComI means the $\text{PXW}(m_0=8,I)$ prior without missing data and MComA means the $\text{PXW}(m_0=8,\text{AR}(1))$ prior without missing data. MMisI and MMisA are the corresponding results for the data with missing values.

Table 2 contains the posterior estimated cut-points and their standard deviations. In general, for all of the different situations, we see that the estimated cut-points are close to their true values and that the standard deviations range from 0.13 to 0.21 for the complete data case and 0.13 to 0.23 for the incomplete data case.

Table 2.

Posterior means and standard deviations for the elements of bin boundaries with and without missing values under two different priors.

Q-	${\mathit{\gamma }}_{42}$	${\mathit{\gamma }}_{43}$	${\mathit{\gamma }}_{52}$	${\mathit{\gamma }}_{53}$
True	0.5	1.0	0.5	1.0
MComI	0.58	1.0	0.44	1.14
(Std)	(0.14)	(0.19)	(0.13)	(0.21)
MComA	0.54	0.96	0.43	1.12
(Std)	(0.14)	(0.18)	(0.13)	(0.21)
MMisI	0.65	1.13	0.44	1.19
(Std)	(0.17)	(0.23)	(0.14)	(0.23)
MMisA	0.61	1.08	0.42	1.17
(Std)	(0.17)	(0.23)	(0.13)	(0.23)

MComI means the $\text{PXW}(m_0=8,I)$ prior without missing data and MComA means the $\text{PXW}(m_0=8,\text{AR}(1))$ prior without missing data. MMisI and MMisA are the the corresponding results for the data with missing values.

Table 3 presents the posterior means of the correlation matrix elements and their posterior standard deviations for both complete and incomplete simulated data sets. We see that the estimated posterior correlations are pulled toward zero under the $\text{PXW}(m_0=8,I)$ prior for both the complete and missing data sets. Not surprisingly, the $\text{PXW}(m_0=8,AR(1)(0.5))$ prior gives much better estimates for some elements of the correlation matrix than the $\text{PXW}\left(m_0=8,I\right)$ prior (see figure 2). For example, for the complete data set, the estimated value for $r_{34}$ is equal to 0.36 under the $\text{PXW}\left(m_0=8,I\right)$ prior and 0.50 under $\text{PXW}\left(m_0=8,\text{AR}(1)(0.5)\right)$ prior; the true value of $r_{34}$ is equal to 0.5. We see similar behaviour for $r_{12}$ and $r_{13}$ in the incomplete data set. However, there are some elements of the correlation matrix, which are estimated better under the $\text{PXW}\left(m_0=8,I\right)$ prior, such as $r_{15}$ and $r_{35}$.

Table 3.

Posterior means and standard deviations for the elements of $R$ with and without missing values under two different priors.

Q-	$r_{12}$	$r_{13}$	$r_{14}$	$r_{15}$	$r_{23}$
True	0.7	0.63	0.32	0.09	0.9
MComI	0.55	0.27	0.36	0.07	0.83
(Std)	(0.24)	(0.26)	(0.18)	(0.17)	(0.30)
MComA	0.64	0.40	0.42	0.12	0.86
(Std)	(0.30)	(0.30)	(0.17)	(0.19)	(0.38)
MMisI	0.56	0.31	0.27	0.15	0.82
(Std)	(0.24)	(0.27)	(0.18)	(0.18)	(0.34)
MMisA	0.65	0.42	0.35	0.17	0.85
(Std)	(0.30)	(0.31)	(0.19)	(0.20)	(0.41)
Q-	$r_{24}$	$r_{25}$	$r_{34}$	$r_{35}$	$r_{45}$
True	0.45	0.14	0.5	0.15	0.3
MComI	0.40	0.17	0.36	0.18	0.37
(Std)	(0.16)	(0.17)	(0.19)	(0.20)	(0.15)
MComA	0.51	0.23	0.50	0.25	0.45
(Std)	(0.18)	(0.19)	(0.21)	(0.22)	(0.12)
MMisI	0.29	0.12	0.31	0.17	0.39
(Std)	(0.20)	(0.19)	(0.22)	(0.21)	(0.14)
MMisA	0.42	0.19	0.45	0.23	0.49
(Std)	(0.21)	(0.22)	(0.23)	(0.25)	(0.13)

MComI means the $\text{PXW}(m_0=8,I)$ prior without missing data and MComA means the $\text{PXW}(m_0=8,\text{AR}(1))$ prior without missing data. MMisI and MMisA are the corresponding results for the data with missing values.

Figure 2.

The density plots under the $\text{PXW}\left(m_0=8,I\right)$ prior distribution for each correlation parameter (in the complete case data set), showing the extent of information contained in a small data set about $R$. The solid lines are the posterior density plots and the dashed lines are the prior density plots.

As it is difficult to assess the closeness of a matrix of items to their target values, we make a matrix-wise comparison of the estimated and true matrices. To do this, we compute a risk function attributed to Stein [13], namely $L(\hat{R},R)=\text{tr}(\hat{R},R)-\text{log}\left|\hat{R}{R}^{-1}\right|-p$, where $\hat{R}$ is the estimated posterior correlation matrix and $R$ represents the true correlation matrix. We find that the value for $L(R,\hat{R})$ is equal to 0.55 for the $\text{PXW}\left(m_0=8,I\right)$ prior and 0.36 for the $\text{PXW}(m_0=8,AR(1)(\rho =0.5))$ prior. For the same simulated data with 20% missing values, the value for Stein’s loss function is equal to 0.54 for the $\text{PXW}\left(m_0=8,I\right)$ prior and 0.32 for the $\text{PXW}(m_0=8,AR(1)(\rho =0.5))$ prior. Thus, a more appropriate prior density may lead to better inference with or without missing values. We have investigated additional simulated data sets under the same conditions and have found that the results are reasonably stable. Furthermore, the two main conclusions that posterior inference is centred around the true underlying values and that a more appropriate prior distribution leads to a smaller loss function held for several other choices of $\Lambda$ that we tried (results not shown).

5. Application: metabolic predictors of multiple long-term outcome measures in severe head trauma patients

Traumatic brain injury (TBI) is an enormous health problem in the United States and globally. The UCLA Brain Injury Research Center (BIRC) focuses its research on post-injury brain metabolism and its relationship to a long-term outcome in severe head trauma patients [14]. We apply the model to a mixture of data from the UCLA BIRC on $n=51$ subjects. For each subject we use four ordinal measures: (i) Pupils is 0, 1, or 2, indicating the number of abnormal pupils; (ii) Improve is 0, 1, or 2, indicating whether the subject’s Glasgow Coma Scale score improved, stayed the same, or worsened in the early hours post-injury; and (iii) GL6 and GL12 are 0/1/2/3 ordinal where higher means worse global outcome. We also include eight ‘continuous’ variables: (i) NRS6 and NRS12 give the total score at 6 and 12 months post-injury on the Neurobehavioral Rating Scale with higher scores indicating poorer condition; (ii) Peg6 and Peg12 are related to the time required at 6 and 12 months post-injury to complete a test that involves putting the pegs in a pegboard. The two Peg variables were created by taking the average of the logarithm of the left hand time and that of the right hand time. This transformation was chosen to give approximate normality; (iii) LAC, OXY and GLC are measures of brain metabolism for lactate, oxygen, and glucose, respectively, during first week post-injury. The measures are scaled so that higher values correlate with poorer condition; and (iv) AGE is the age of the subject in years.

We chose a compound symmetry structure for $\Omega (\theta )$ with $\theta$ fixed at an intraclass correlation value of 0.5. The degrees of freedom parameter was set to $v=10$. This prior distribution reflects the clinical belief that these measures should all be positively correlated as they are all scaled in the same direction, but the low value for $v$ allows for significant departures from the actual correlation value of 0.5. We ran the MCMC sampler for 100,000 iterations. The Metropolis-Hastings step used a jumping degrees of freedom of ${\nu }_0=800$ and achieved an acceptance rate of ~15%. Although this is below the value of 23% recommended by Gelman et al. [10], we found that higher values for ${\nu }_0$ substantially increased autocorrelations. We calculated Gelman and Rubin’s potential scale reduction factor, $\sqrt{\hat{R}}$, for five dispersed chains with the first 1000 iterations discarded as burn-in [11]. All diagnostics indicated good convergence of the algorithm. The values of $\sqrt{\hat{R}}$ for the 12 regression and eight variance parameters were all below 1.1 by 30,000 iterations. The correlations and cutpoints demonstrated slightly slower convergence: at 30,000 iterations 52 of the 66 correlations and three of the six ordinal cut-points had PSRF values below 1.1, but by 100,000 iterations $\sqrt{\hat{R}}$ was below 1.1 for all parameters. The multivariate potential scale reduction factor for the cut-point parameters was 1.46 after 30,000 iterations, improving to 1.17 at 100,000 iterations.

Table 4 gives the posterior mean of the correlation matrix $R$ shown in the upper triangular portion of the matrix. The lower triangular portion gives the posterior standard deviations of the correlation matrix $R$. Here we see that the measures of metabolism LAC, OXY and GLC, all have at least moderate correlation with some of the 6-month measures (NRS6, Peg6, and GL6) and 12-month measures (NRS12, Peg12, and GL12). Therefore, we can partially predict short-term and long-term outcomes via early metabolic measures.

Table 4.

Above diagonal: posterior means of the tetrachoric correlation matrix for the BIRC neuropsych measures. Below diagonal: posterior standard deviations of the tetrachoric correlation matrix.

	NR6	NR12	PG6	PG12	LAC	OXY	GLC	AGE	PUP	IMP	GL6	GL12
NR6	1.00	0.75	0.61	0.42	0.37	0.32	0.20	0.16	0.28	0.09	0.66	0.67
NR12	0.07	1.00	0.41	0.36	0.28	0.39	0.26	0.15	0.15	0.15	0.54	0.69
PG6	0.09	0.11	1.00	0.79	0.47	0.30	0.20	0.40	0.24	0.22	0.54	0.55
PG12	0.11	0.11	0.06	1.00	0.32	0.22	0.10	0.42	0.22	0.09	0.56	0.52
LAC	0.11	0.12	0.110	0.13	0.10	0.29	0.08	0.07	0.18	0.15	0.19	0.17
OXY	0.11	0.11	0.11	0.13	0.12	1.00	0.21	0.28	0.20	0.12	0.33	0.37
GLC	0.12	0.11	0.11	0.13	0.13	0.12	1.00	0.13	−0.01	0.17	0.17	0.48
AGE	0.14	0.14	0.12	0.12	0.14	0.13	0.13	1.00	−0.15	0.10	0.27	0.28
PUP	0.13	0.13	0.13	0.14	0.15	0.15	0.14	0.13	1.00	−0.01	0.24	0.31
IMP	0.13	0.13	0.12	0.13	0.14	0.14	0.14	0.14	0.16	1.00	−0.08	0.01
GL6	0.09	0.130	0.130	0.130	0.13	0.13	0.12	0.13	0.14	0.14	1.00	0.85
GL12	0.08	0.08	0.09	0.10	0.12	0.11	0.12	0.11	0.12	0.15	0.06	1.00

Bold faced correlations are those between metabolic measures and long term measures.

Pairwise correlation of a metabolic measurement and a long-term outcome may not be clinically important if this relationship can be explained by other known predictors of outcome such as age, pupil abnormality, and early improvement. Thus, to investigate the correlation structure more carefully, we examined the posterior distribution of the partial correlations in $R$. The partial covariance matrix of variables $X$ controlling for variables $Y$ is

$$S_{X\cdot Y}=S_{XX}-S_{XY}\left(S_{YY}^{-1}\right)S_{YX},$$

where

$$S=\left(\begin{array}{ll}{S}_{XX}& S_{XY}\\ S_{YX}& S_{YY}\end{array}\right)$$

is the covariance matrix of the full vector. The partial correlation matrix of variables $X$ controlling for variables $Y,R_{X\cdot Y}$, is then found by standardizing $S_{X\cdot Y}$.

Table 5 presents the posterior means and standard deviations of the partial correlations for the six long-term outcomes with each of the three metabolic measures controlling for Age, Pupils, Improve and the other two metabolic measures. We see that each metabolic measure seems to give some unique information about long term outcome. The posterior mean of the partial correlation between GLC and GL12 is 0.475 with standard deviation equal to 0.131, suggesting that glucose metabolism is predictive of global 12-month outcome even after controlling for lactate and oxygen metabolism as well as age, pupils, and early improvement. The partial correlations between LAC and NRSR6, and both PEG6 and PEG12, are also fairly strong. This suggests that lactate can predict NRSR6 and peg test times even after controlling for GLC and OXY and the other variables. There is a moderate partial correlation between OXY and NRSR12 after controlling for lactate and glucose metabolism and the other predictors.

Table 5.

Posterior means and standard deviations of partial correlations between brain metabolic measures and the six long-term outcomes while controlling for Age, Pupils, Improve and the other two metabolic measures.

	NRSR6	NRSR12	PEG6	PEG12	GL6	GL12
GLC(Mean)	0.154	0.197	0.130	0.042	0.135	0.475
(std)	0.128	0.120	0.126	0.145	0.134	0.131
LAC (Mean)	0.287	0.171	0.438	0.288	0.099	0.040
(std)	0.128	0.129	0.122	0.143	0.145	0.155
OXY(Mean)	0.143	0.273	0.004	−0.048	0.197	0.174
(std)	0.132	0.117	0.144	0.151	0.152	0.157

GLC is predictive of GL12 outcome even after controlling for lactate and oxygen; LAC seems predictive of NRSR6 and both peg times even after controlling for GLC and OXY, and OXY seems predictive of NRSR12 after controlling for LAC and GLC.

6. Hierarchical models for mixture of continuous and ordinal repeated measures

The metabolic measures used in the analysis of section 5 were, in reality, themselves longitudinally recorded. Each subject has between one and ten measurements potentially taken every 12 hours during the first five days in the intensive care unit. We could directly model these in our vector $v_i$, but a more interesting approach might be to use a separate hierarchical model for these longitudinal data (e.g., random intercept or random trajectory model) and use the subject-specific coefficients from this model (e.g., intercepts or intercepts and slopes) as the (now latent) continuous data that go into the $v_i$ vector. For the analysis just presented, we did a 2-step version of this: first the repeated metabolic measures were analysed in a mixed effects regression package, and then the estimated patient specific means were used as part of the $v_i$ vector.

A more integrated approach would proceed as follows. Let the observed data be $T_i=\left(v_i,c_i\right)$. Suppose that $v_i~N\left(H_i{a}_i,M_i\right)$, where $H_i$ is the design matrix for the longitudinal data $v_i,M_i$ is the covariance matrix of the disturbances in $v_i$ and $a_i$ are the subject-specific parameters for the $i$th subject. We assume for simplicity in this section that $H_i$ and $M_i$ are known. Additionally, we assume that the subject-specific parameters come from a multivariate normal distribution with known mean vector and covariance matrix, i.e., $p\left(a_i\right)=N\left(a_i\mid u,w\right)$. These restrictions can be easily relaxed and the additional computations are easily handled in an MCMC setting. We make only the point here that the $a_i$ latent vectors can be treated via standard regression results. The model is completed by assuming that $y_i=\left(a_i,z_i\right)~N\left(X_i\beta ,\Lambda \right)$.

Define $A=\left(a_1,\dots ,a_N\right)$. Then the joint posterior density can be found as

$$\begin{gathered} p(\beta ,\Lambda ,\gamma ,A,Z\mid V,C)\propto p(\beta ,\Lambda ,\gamma )p(V,C\mid \beta ,\Lambda ,\gamma ,A,Z)p(A,Z\mid \beta ,\Lambda ,\gamma ) \\ =p(\beta ,\Lambda ,\gamma )p(A,Z\mid \beta ,\Lambda ,\gamma )p(C\mid \beta ,\Lambda ,\gamma ,V,A,Z)p(V\mid \beta ,\Lambda ,\gamma ,A,Z) \\ =p\left(\beta \right)p\left(\Lambda \right)p\left(\gamma \right)\prod _{i=1}^{N}N\left(a_i\mid u,w\right)\times \prod _{i=1}^N\left[I_i\times N\left(y_i\mid X_i\beta ,\Lambda \right)\right]\prod _{i=1}^N\left[N\left(v_i\mid H_i{a}_i,M_i\right)\right], \end{gathered}$$

where the indicator function $I_i$ indicates the compatibility of $c_i$ and $z_i$. To implement the Gibbs sampler, all steps are as above with one additional step for the $a_i$. The subject-specific hierarchical parameters have full conditional multivariate normal distributions:

$$p\left(a_i\mid \beta ,\Lambda ,\gamma ,z_i,v_i,c_i\right)\propto N\left(v_i\mid H_i{a}_i,M_i\right)N\left(a_i\mid u,w\right)\propto N\left(a_i\mid {\tilde{u}}_i,{\tilde{w}}_i\right)$$

where ${\tilde{u}}_i$ and ${\tilde{w}}_i$ are the conditional mean and variance of $a_i$ given $z_i$ (section 2). Using standard Bayesian linear model results,

$${\tilde{w}}_i={\left(A_i^{\text{T}}{M}_i^{-1}{A}_i+w^{-1}\right)}^{-1}\text{and}{\tilde{u}}_i={\tilde{w}}_i\left(A_i^{\text{T}}{M}_i^{-1}{v}_i+w^{-1}u\right).$$

7. Discussion

In this manuscript, we generalized the multivariate probit model to a mixture of continuous and ordinal repeated measures and applied Gibbs sampler to do the analysis. We extended our previous work on modeling and sampling, a correlation matrix to modeling and sampling, a covariance matrix with some of the diagonal elements restricted to be equal to 1. The analysis allows for missing values in the outcomes with little additional computational complexity. The practical value of this model was illustrated with a simulation example and through the application to a data set from brain injury research. Natural extensions of the proposed approach would include incorporating additional types of variables (e.g., nominal, Poisson, semi-continuous) and generalizing these models to a hierarchical setting for longitudinal data. In manuscripts in progress, we discuss the use of a multivariate multinomial probit model for analysing repeated nominal measures, and the analyses of mixtures of nominal, ordinal, and continuous measures.