Bayesian flexible hierarchical skew heavy-tailed multivariate meta regression models for individual patient data with applications

Abstract

A flexible class of multivariate meta-regression models are proposed for Individual Patient Data (IPD). The methodology is well motivated from 26 pivotal Merck clinical trials that compare statins (cholesterol lowering drugs) in combination with ezetimibe and statins alone on treatment-naïve patients and those continuing on statins at baseline. The research goal is to jointly analyze the multivariate outcomes, Low Density Lipoprotein Cholesterol (LDL-C), High Density Lipoprotein Cholesterol (HDL-C), and Triglycerides (TG). These three continuous outcome measures are correlated and shed much light on a subject’s lipid status. The proposed multivariate meta-regression models allow for different skewness parameters and different degrees of freedom for the multivariate outcomes from different trials under a general class of skew t-distributions. The theoretical properties of the proposed models are examined and an efficient Markov chain Monte Carlo (MCMC) sampling algorithm is developed for carrying out Bayesian inference under the proposed multivariate meta-regression model. In addition, the Conditional Predictive Ordinates (CPOs) are computed via an efficient Monte Carlo method. Consequently, the logarithm of the pseudo marginal likelihood and Bayesian residuals are obtained for model comparison and assessment, respectively. A detailed analysis of the IPD meta data from the 26 Merck clinical trials is carried out to demonstrate the usefulness of the proposed methodology.

1. INTRODUCTION

According to the National Vital Statistics Reports (Heron 2019) cardiovascular disease (CVD) continues to be the leading cause of death for both men and women. This is the case in the U.S. and worldwide. More than half of all people who die due to heart disease are men. It has been confirmed that increased low density lipoprotein cholesterol (LDL-C) is an independent risk factor for CVD. LDL-C lowering has been consistently shown to reduce the risk of CVD. One large meta-analysis (Baigent et al. 2010) of statin clinical trials shows a progressive reduction in risk of major CVD events with lower on-treatment LDL-C levels. Although LDL-C is a primary cause of CVD, other risk factors contribute, as well, for example, high-density lipoprotein cholesterol (HDL-C) and triglycerides (TG). Large cohort studies show a strong and inverse relationship of HDL-C levels with the risk of incident CVD independent of other lipids. HDL-C is positively associated with a decreased risk of coronary heart disease (CHD). As defined by the US National Cholesterol Education Program Adult Treatment Panel III guidelines (ATP III 2001), an HDL-C level of 60 mg/dL or greater is a negative risk factor. A long-standing association exists between elevated triglyceride levels and CVD (Austin et al. 1998; Sarwar et al. 2007). In a meta-analysis of 17 studies, increased triglyceride levels were associated with increased coronary disease risk in both men and women, after adjustment for HDL-C and other risk factors (Hokanson and Austin 1996). A randomized, controlled clinical trial REDUCE-IT (Bhatt et al. 2019) has demonstrated that intervention to low triglyceride level is associated with reduced CVD events. Among lipid-lowering drugs, statins are the cornerstone of therapy. Ezetimibe is the most commonly used nonstatin agent. It lowers LDL-C levels by 13% to 20% and has a low incidence of side effects (Cannon et al. 2015; Kashani et al. 2008).

Meta-regression (MR) of individual patient data (IPD) is an effective modeling tool for explaining heterogeneity between trials, synthesizing evidence across studies, investigating individual-level interactions, or identifying subgroups (Simmonds and Higgins 2007; Ritz et al. 2008; Kim et al. 2013; Riley et al. 2015; Burke et al. 2017; Belias et al. 2019; Ibrahim et al. 2019). In dealing with IPD multivariate meta-data, it is often the case that the data may be highly skewed and or have heavy-tailed and non-normal distributions to properly model certain response variables which may have skew and/or heavy-tailed distributions.

The modeling framework proposed here is motivated from multivariate IPD data from 26 Merck clinical trials for cholesterol lowering drugs. In our application, we consider a three-dimensional continuous response, in which some components of the response variable are heavy-tailed and/or skew distributions, and some components may have symmetric and/or light tailed distributions. We do not know which are which in advance and our hope is to develop a model that accommodates this flexibility. Thus, in these settings, one needs more complex models than the traditional linear mixed model. There is abundant literature on using skew and/or heavy tailed distributions for modeling univariate and or multivariate data (Chen et al. 1999; Branco and Dey 2001; Azzalini and Capitanio 2003; Sahu et al. 2003; Genton 2004; Adcock 2004; Kim et al. 2008; Arellano-Valle and Genton 2010; Chang and Zimmerman 2016), but sparse in the multivariate MR setting (Kim et al. 2013; Ibrahim et al. 2019). One of the challenges of modeling skew and heavy tailed distributions in the MR setting is that one needs to develop a flexible class of models that allow certain components of the multivariate response to have skew and/or heavy-tailed while allowing for other components to have symmetric and/or light-tailed distributions, while at the same time, capturing heterogeneity between trials via appropriate random effects. Since the skewness and heaviness of the tails will not be known in advance, one needs to also model the skew parameters and scale parameters appropriately to correctly capture the data structure in this complex multivariate MR setting.

Instead of using a Box-Cox transformation on the multivariate response variables as in Kim et al. (2013), we extend the multivariate skew MR models of Ibrahim et al. (2019) to develop a flexible class of multivariate MR models that accommodate skewness and heavy tailed distributions for multivariate meta-data. Under the proposed models, we first assume that the skewness parameters, the covariance matrices of the multivariate responses, and the degrees of freedom in the multivariate t distributions for the error terms are different across trials at the first stage, and then assume hierarchial priors for the multivariate random effects, the skewness parameters, the covariance matrices, and the degrees of freedom at the second stage. The proposed models are very flexible and general. As empirically shown in Section 5, the proposed model leads to a substantial gain in the goodness-of-fit of the multivariate IPD data from the 26 Merck clinical trials. Due to the complexity and computational challenge of the proposed models, an efficient Markov chain Monte Carlo (MCMC) sampling algorithm is developed for sampling from the posterior distribution of the model parameters. Moreover, one needs to develop and use model assessment tools to examine and find the best fitting models. In this paper, we consider the Conditional Predictive Ordinate (CPO) and develop its computational implementation for the proposed model. It is shown that CPO along with our flexible modeling framework identifies a more suitable model than a traditional MR model.

The rest of the paper is organized as follows. In Section 2, we discuss the Merck cholesterol data in detail. Sections 3.1–3.2 lay out the modeling details of our proposed multivariate skew heavy-tailed random effects meta-regression model, the prior distributions, properties of the proposed model as well as a flow diagram explaining all of the components of the model. Section 3.3 develops the likelihood function and joint posterior distribution of all the parameters. Section 4.1 develops new analytic and computational results for CPO, and Section 4.2 gives details of the MCMC computational development. Section 5 presents a detailed analysis of the cholesterol data showing that our proposed model provides a better interpretation and fit over the existing MR model. We close the paper with a discussion in Section 6.

2. THE CHOLESTEROL DATA

We consider the individual patient data (IPD) from 26 Merck-sponsored double-blind, randomized, active or placebo-controlled clinical trials on adult patients with primary hypercholesterolemia, which were analyzed by Kim et al. (2013) and Ibrahim et al. (2019). The IPD data considered in our analyses are a subset of the meta-data published in Leiter et al. (2011). The citations of the primary papers published in clinical journals for these 26 trials can be found in Leiter et al. (2011). A detailed summary of the covariates was provided in Tables 1 and 2 of Kim et al. (2013).

Table 1.

Values of LPML Measure under Various Models for the Cholesterol Data

Flexible skew t model
	τ = 5	τ = 10	τ = 20	τ = 30	random va & τ
va = 20	−261324.67	−261391.54	−261423.48	−261455.18	−261321.15
va = 25	−261322.38	−261346.60	−261413.49	−261448.32
va = 30	−261323.05	−261358.94	−261415.45	−261450.02
skew t model with τ = 30
−262517.61

Table 2.

Posterior Estimates under the flexible skew t model with random ν_a and τ for LDL-C

	Par.	Mean	SD	95% HPD
bl_ldlc	β1,1	−0.031	0.003	(−0.037, −0.025)
bl_hdlc	β1,2	0.013	0.009	(−0.005, 0.032)
bl_tg	β1,3	0.007	0.001	(0.004, 0.010)
BMI	β1,4	0.069	0.017	(0.035, 0.102)
age	β1,5	−0.144	0.009	(−0.162, −0.126)
duration	β1,6	0.338	0.282	(−0.217, 0.895)
Female	β1,7	−1.239	0.206	(−1.621, −0.812)
DM	β1,8	−2.315	0.278	(−2.853, −1.769)
CHD	β1,9	0.258	0.270	(−0.301, 0.763)
potency2	β1,10	−7.319	0.327	(−7.954, −6.669)
potency3	β1,11	−15.738	0.375	(−16.438, −14.968)
Black	β1,12	2.220	0.394	(1.433, 2.975)
Hispanic	β1,13	0.397	0.456	(−0.519, 1.269)
Other	β1,14	−2.077	0.490	(−3.046, −1.146)
onstatin × bl_ldlc	β1,15	−0.058	0.007	(−0.072, −0.045)
onstatin × bl_hdlc	β1,16	−0.074	0.017	(−0.109, −0.041)
onstatin × bl_tg	β1,17	−0.005	0.003	(−0.011, 0.001)
onstatin × BMI	β1,18	−0.068	0.034	(−0.135, −0.004)
onstatin × age	β1,19	0.108	0.018	(0.074, 0.145)
onstatin × duration	β1,20	0.155	0.633	(−1.087, 1.398)
onstatin × Female	β1,21	2.722	0.396	(1.956, 3.497)
onstatin × DM	β1,22	−0.654	0.478	(−1.598, 0.281)
onstatin × CHD	β1,23	−0.862	0.504	(−1.855, 0.106)
onstatin × potency2	β1,24	6.144	0.655	(4.873, 7.431)
onstatin × potency3	β1,25	13.924	0.835	(12.336, 15.597)
onstatin × Black	β1,26	−2.630	0.851	(−4.272, −0.953)
onstatin × Hispanic	β1,27	−2.094	1.020	(−4.151, −0.139)
onstatin × Other	β1,28	3.108	0.962	(1.213, 4.981)

The primary goal of these clinical trials was to evaluate the LDL-C lowering effects of ezetimibe (which works in the digestive tract) in combination with statin (which works in the liver) in comparison with statin alone on treatment-naïve patients at baseline (on a first-line therapy) or patients who underwent washout of previous lipid-modifying therapy at baseline (on a second-line therapy). In our analyses, different statins and their doses are combined to form the “statin” and “statin + ezetimibe” treatment groups. Ezetimibe (EZE) is available at only one dose of 10 mg, and the statins used in these trials included simvastatin, atorvastatin, lovastatin, rosuvastatin, pravastatin, and fluvastatin. Statin potency describes the chemical/medicinal strength of the statin, and it was categorized into three potency classes (low, medium, and high). The covariates include treatment (trt: 0 = Statin and 1 = Statin + EZE), baseline LDL-C (bl-ldlc), baseline HDL-C (bl-hdlc), baseline TG (bl-tg), age, race (White (reference), Black, Hispanic, and Other), gender (Female: 0 = male (reference), 1 = female), diabetes (DM: 0 = No, 1 = Yes), coronary heart disease (CHD: 0 = No, 1 = Yes), body mass index (BMI), statin potency (low (reference), med (potency2), and high (potency3)), and trial duration (duration) (6–12 weeks). Tables 1 and 2 of Kim et al. (2013) show a considerable amount of heterogeneity in the covariates across the trials. Therefore, to examine the treatment effects, there is a need to adjust for these covariates. We consider three primary outcome variables including percent changes from baseline in LDL-C, HDL-C, and TG. For ease of presentation, we simply denote these three outcome variables by LDL-C, HDL-C, and TG. As empirically shown in Kim et al. (2013) and Ibrahim et al. (2019), skew and heavy-tailed distributions are needed for modeling these three primary outcome variables.

3. MULTIVARIATE META REGRESSION MODELS

Consider K randomized trials, where each trial has two treatment arms (“Statin” or “Statin + EZE”), and patients in each trial were either all on statin or all not on statin prior to the trial. The sample size of the individual patient data for the k^th trial is n_k. Let y_ik = (y_i1k,…, y_iJk)′ denote a J-dimensional vector of the responses for the i^th patient in the k^th trial. Also let trt_ik = 1 if the i^th patient received “Statin + EZE” and 0 if “Statin” alone, and onstatin_k = 1 if patients were on statin and 0 if not on statin prior to the trial. Furthermore, let x_ijk denote a p_j-dimensional vector of covariates for the j^th response corresponding to the i^th patient, and ${\beta }_j={\left({\beta }_{j1},\dots ,{\beta }_{j{p}_j}\right)}^{\prime }$ is the vector of fixed effects regression coefficients corresponding to the p_j covariates. Let w_ijk denote a q_j-dimensional vector for the random effects.

3.1. Preliminary

The multivariate skew meta-regression model proposed by Ibrahim et al. (2019) is given by

$$y_{ijk}=\left[{\gamma }_{jk0}+{\gamma }_{jk1}{\text{trt}}_{ik}\right]\left(1-{\text{ onstatin}}_k\right)+\left[{\gamma }_{jk2}+{\gamma }_{jk3}{\text{trt}}_{ik}\right]{\text{onstatin}}_k+{\delta }_j\left(z_{ijk}-E\left[z_{ijk}\right]\right)+x_{ijk}^{\prime }{\beta }_j+{ϵ}_{ijk},{\left({ϵ}_{i1k},\dots ,{ϵ}_{iJk}\right)}^{\prime }\mid {\lambda }_{ik}~N\left(0,{\lambda }_{ik}^{-1}\Sigma \right),{\lambda }_{ik}~\text{Gamma}(\nu /2,\nu /2),$$ (3.1)

where z_ijk\ψ_ik follows an exponential distribution with mean $\frac{1}{{\psi }_{ik}}$, ψ_ik ~ Gamma(τ + 1, τ), where Gamma(a, b) denotes the gamma distribution with mean a/b and variance a/b², and γ_jk = (γ_jk0, γ_jk1, γ_jk2, γ_jk3)′ ~ N₄(γ_j, Ω_j) with γ_j = (γ_j0, γ_j1, γ_j2, γ_j3)′. The model in (3.1) assumed that z_i1k, …, z_iJk are dependent as well as the same covariance matrices, skewness parameters, and degrees of freedom across K trials. Let ${\lambda }_k={\sum }_{i=1}^{n_k}{\lambda }_{ik}$. Following Ibrahim et al. (2019), we define

$${\lambda }_k^{*}=\left({\lambda }_k-n_k\right)/{\left(n_k/(\nu /2)\right)}^{1/2}.$$ (3.2)

Then, we have $E\left[{\lambda }_k^{*}\right]=0$ and $\text{Var}\left({\lambda }_k^{*}\right)=1$ a priori, where the expectation and variance are taken with respect to Gamma(n_kv/2, v/2), and consequently, ${\lambda }_k^{*}$ approximately follows the standard N(0, 1) distribution. Figure 4 of Ibrahim et al. (2019) shows the boxplots of the ${\lambda }_k^{*}\text{'}\text{s}$ from the posterior distribution under the skew t model in (3.1) with τ = 30 and an unstructured Σ for the cholesterol data. From these boxplots, it was found that the posterior distributions of the ${\lambda }_k^{*}\text{'}\text{s}$ substantially depart from their prior distributions for three trials. The reasons for such a discrepancy between the prior and posterior distributions of ${\lambda }_k^{*}$ could be two-fold: (i) outlying trials or (ii) lack of fit due to the use of the same degrees of freedom v across all trials.

Figure 4.

Plots of the posterior estimates (mean and 95% HPD interval) for ν_k under the flexible skew model with random ν_a and τ for the cholesterol data.

3.2. Hierarchical skew heavy-tailed multivariate meta regression models

We propose the following flexible hierarchical skew heavy-tailed multivariate meta regression models as follows.

Stage 1: Model for Multivariate Responses

$$y_{ijk}=x_{ijk}^{\prime }{\beta }_j+w_{ik}^{\prime }{\gamma }_{jk}+{\delta }_{jk}\left(z_{ijk}-E\left[z_{ijk}\right]\right)+{ϵ}_{ijk},$$ (3.3)

where ${\gamma }_{jk}={\left({\gamma }_{j1k},\dots ,{\gamma }_{j{q}_{j}k}\right)}^{\prime }$ represents the vector of q_j - dimensional random effects for the j^th response, δ_jk is a skewness parameter for the j^th response in the kth trial, z_ijk is the skewness latent variable with the expected value E[z_ijk], and ϵ_ik = (ϵ_i1k, …, ϵ_iJk)′ is the vector of error terms. We assume

$${ϵ}_{ik}\mid {\lambda }_{ik}~N\left(0,{\lambda }_{ik}^{-1}{\Sigma }_k\right)\text{ and }{\lambda }_{ik}~\text{Gamma}\left({\nu }_k/2,{\nu }_k/2\right),$$ (3.4)

where Σ_k is a J × J positive definite covariance matrix and v_k > 0 is an unknown parameter. In (3.4), v_k corresponds the degrees of freedom and the variance of ϵ_ik is finite for all v_k > 2. Also, in (3.3), we assume that

$$z_{ijk}\mid {\psi }_{ik}~\mathcal{E}\left({\psi }_{ik}\right)\text{ and }{\psi }_{ik}~\text{Gamma}(\tau +1,\tau ),$$ (3.5)

where ε(ψ_ik) denotes an exponential distribution with mean $\frac{1}{{\psi }_{ik}}$ and Gamma(τ + 1, τ) denotes a Gamma distribution with mean $\frac{\tau +1}{\tau }$ and variance $\frac{\tau +1}{{\tau }^2}$. Under this assumption, E[z_ijk] = 1 and $\text{Var}\left(z_{ijk}\right)=\frac{\tau +1}{\tau -1}$ for τ > 1. Let $z_{ik}^{\delta }={\left(z_{i1k}^{\delta },\dots ,z_{iJk}^{\delta }\right)}^{\prime }=({\delta }_{1k}(z_{i1k}-{E\left[z_{i1k}\right]),\dots ,{\delta }_{Jk}\left(z_{iJk}-E\left[z_{iJk}\right]\right))}^{\prime }$. The covariance matrix of $z_{ik}^{\delta }$ is given by

$$\text{Var}\left(z_{ik}^{\delta }\right)=\frac{\tau }{\tau -1}\text{diag}\left({\delta }_{1k}^2,\dots ,{\delta }_{Jk}^2\right)+\frac{1}{\tau -1}{\delta }_k{\delta }_k^{\prime },$$ (3.6)

where δ_k = (δ_1k, …, δ_Jk)′ for k = 1, …, K. From (3.6), we note that the correlations of the responses, y_ijk, depend on the kth skewness parameter δ_jk, and the z_ijk’s are independent when τ → ∞. We further assume that γ_jk, z_ijk, and ϵ_ik are independent.

At Stage 2, models for the random effects, covariance matrices, skewness parameters, and degrees of freedom are specified as follows.

Stage 2a: Model for Random Effects

$${\gamma }_{jk}~N\left({\gamma }_j,{\Omega }_j\right),$$ (3.7)

where ${\gamma }_j={\left({\gamma }_{j1},\dots ,{\gamma }_{j{q}_j}\right)}^{\prime }$ is the overall mean vector of γ_jk and Ω_j is a q_j × q_j covariance matrix of the random effects γ_jk.

Stage 2b: Model for Covariance Matrices

$${\Sigma }_k^{-1}~{\text{Wishart}}_J(v,(v-J-1)\Sigma ),$$ (3.8)

where Σ is a J × J overall covariance matrix and υ > J + 1. Also Σ_k has prior expectation E[Σ_k|Σ] = Σ when υ > J + 1. The model (3.8) is attractive as it allows for “borrowing of strength” across trials through the common second-level covariance matrix Σ and it also accounts for the heterogeneity of the within-study covariance matrices among different trials at the same time. The parameter υ in (3.8) controls the amount of borrowing across trials. The larger the value of υ, the more the within-trial covariance matrices borrow strength from different trials. Note that Wishart_J (d₀, S₀) denotes the Wishart distribution with mean d₀S₀. That is, $\pi \left({\Sigma }_k^{-1}\mid d_0,S_0\right)\propto {|{\Sigma }_k^{-1}|}^{\left(d_0-J-1\right)/2}\text{exp}\left(-\frac{1}{2}\text{tr}\left(S_0^{-1}{\Sigma }_k^{-1}\right)\right)$.

Stage 2c: Model for Skewness Parameters

$${\delta }_{jk}~N\left({\delta }_j,{\sigma }_{{\delta }_j}^2\right),$$ (3.9)

where −∞ < δ_j < ∞ is the overall skewness parameter for the j^th response and ${\sigma }_{{\delta }_j}^2>0$ is the variance parameter, controlling the amount of “borrowing” across trials for within-trial skewness parameters for the j^th response.

Stage 2d: Model for Degrees of Freedom

$${\nu }_k~\text{Gamma}\left({\nu }_a,{\nu }_a/{\nu }_b\right).$$ (3.10)

where v_a > 0 controls the amount of borrowing across trials and v_b > 0 is the overall degrees of freedom. Under (3.10), the prior mean of v_k is E[v_k] = v_b.

At Stage 3, the prior distributions of the hyperparameters for the random effects, covariance matrices, skewness parameters, and degrees of freedom, which are proposed at Stage 2, as well as the regression coefficients, are specified as follows. Let δ = (δ₁, …, δ_J)′ and ${\sigma }_{\delta }^2={\left({\sigma }_{{\delta }_1}^2,\dots ,{\sigma }_{{\delta }_I}^2\right)}^{\prime }$. We assume that β, γ, δ*, Σ*, v*, τ, and Ω are independent a priori.

Stage 3a: Prior distributions of the Hyperparameters for Random Effects

$$\gamma ~N_{qJ}\left(0,c_1{I}_{qJ}\right)\text{ and }{\Omega }_j^{-1}~{\text{Wishart}}_{q_j}\left(d_1,S_1\right),$$ (3.11)

where $q={\sum }_{j=1}^J{q}_j$.

Stage 3b: Prior distributions of the Hyperparameters for Covariance Matrices

$$v~\text{Gamma}\left(a_1,b_1\right)\text{ and }\Sigma ~{\text{Wishart}}_J\left(d_2,S_2\right).$$ (3.12)

Stage 3c: Prior distributions of the Hyperparameters for Skewness Parameters and Latent Variables

$$\delta ~N_J\left(0,c_2{I}_J\right),{\sigma }_{{\delta }_j}^2~\text{IGamma}\left(a_2,b_2\right),\text{ and }\tau ~\text{Gamma}\left(a_3,b_3\right)1(\tau >1),$$ (3.13)

where IGamma(a, b) denotes the inverse gamma distribution with mean b/(a − 1) when a > 1 and variance b²/[(a − 1)²(a − 2)] when a > 2.

Stage 3d: Prior distributions of the Hyperparameters for Degrees of Freedom

$${\nu }_a~\text{Gamma}\left(a_4,b_4\right)\text{ and }{\nu }_b~\text{IGamma}\left(a_5,b_5\right).$$ (3.14)

Stage 3e: Prior distribution for Fixed Effects Regression Coefficients

$$\beta ~N_p\left(0,c_3{I}_p\right),$$ (3.15)

where $p={\sum }_{j=1}^J{p}_j$.

The multivariate meta-regression model defined in (3.3), (3.4), (3.6), (3.7), (3.9), and (3.10) is very general and flexible, and it includes as special cases the multivariate normal meta-regression model, the multivariate t meta-regression model, and the multivariate skew t meta-regression model. Furthermore, this proposed model also incorporates the different covariance matrices, skewness parameters, and degrees of freedom across the K trials. In the analysis, the hyperparameters of the prior distribution at Stage 3 were specified as c₁ = 100, c₂ = 100, c₃ = 100, d₁ = q_j + 0.1, S₁ = 0.1, d₂ = J + 0.1, S₂ = 0.1, a₁ = 1, b₁ = 0.1, a₂ = 0.1, b₂ = 0.1, a₃ = 1, b₃ = 0.1, a₄ = 1, b₄ = 0.1, a₅ = 0.1, and b₅ = 0.1. These choices of the hyperparameters lead to noninformative priors. The flow diagram of the proposed model is given in Figure 1. Simultaneous estimation of all parameters is not easy and requires a sophisticated and computationally intensive MCMC sampling algorithm.

Figure 1.

Flow Diagram of the Proposed Model.

3.3. The likelihood function and posterior distribution

Let $X_{ik}=\text{diag}\left(x_{i1k}^{\prime },\dots ,x_{iJk}^{\prime }\right)$, $W_{ik}=\text{diag}(w_{ik}^{\prime },\dots ,w_{ik}^{\prime })$, $\beta ={\left({\beta }_1^{\prime },\dots ,{\beta }_J^{\prime }\right)}^{\prime }$, $\gamma ={\left({\gamma }_1^{\prime },\dots ,{\gamma }_J^{\prime }\right)}^{\prime }$, ${\delta }^{*}=\left({\delta }_1^{\prime },\dots ,{\delta }_K^{\prime }\right)$, Σ* = (Σ₁,…, Σ_K), v* = (v₁,…, v_K)′, ${\gamma }_k^R={\left({\gamma }_{1k}^{\prime },\dots ,{\gamma }_{Jk}^{\prime }\right)}^{\prime }$, and $\psi ={\left({\psi }_{11},\dots ,{\psi }_{n_{K}K}\right)}^{\prime }$. Also let y_ik = (y_i1k,…, y_iJk)′, $y={\left(y_{11}^{\prime },\dots ,y_{n_{K}K}^{\prime }\right)}^{\prime }$, $W=\left(W_{11},\dots ,W_{n_{K}K}\right)$, and $X=\left(X_{11},\dots ,X_{n_{K}K}\right)$. Furthermore, we let D_obs = (y, X, W) denote the observed data. Then the complete-data likelihood function is given by

$$L\left(\beta ,\gamma ,{\delta }^{*},{\Sigma }^{*},{\nu }^{*},\tau ,\Omega ,{\gamma }^R,z,\psi ,\lambda \mid D_{\text{obs}}\right)=\prod _{k=1}^K\prod _{i=1}^{n_k}f\left(y_{ik}\mid \beta ,{\delta }_k,{\Sigma }_k,{\gamma }_k^R,z_{ik},{\lambda }_{ik},X_{ik},W_{ik}\right)\times \frac{{\left(\frac{{\nu }_k}{2}\right)}^{\frac{{\nu }_k}{2}}}{\Gamma \left(\frac{{\nu }_k}{2}\right)}{\lambda }_{ik}^{\frac{{\nu }_k}{2}-1}\text{exp}\left\{-\frac{{\nu }_k}{2}{\lambda }_{ik}\right\}\times \prod _{k=1}^K\prod _{i=1}^{n_k}\left[\prod _{j=1}^J{\psi }_{ik}\text{exp}\left(-{\psi }_{ik}{z}_{ijk}\right)\right]\times \frac{{\tau }^{\tau +1}}{\Gamma (\tau +1)}{\psi }_{ik}^{\tau }\text{exp}\left\{-\tau {\psi }_{ik}\right\}\times \prod _{k=1}^K\prod _{j=1}^J{(2\pi )}^{-\frac{4}{2}}{\left|{\Omega }_j\right|}^{-\frac{1}{2}}\times \text{exp}\left\{-\frac{1}{2}{\left({\gamma }_{jk}-{\gamma }_j\right)}^{\prime }{\Omega }_j^{-1}\left({\gamma }_{jk}-{\gamma }_j\right)\right\},$$ (3.16)

where $f\left(y_{ik}\mid \beta ,{\Delta }_k,{\Sigma }_k,{\gamma }_k^R,z_{ik},{\lambda }_{ik},X_{ik},W_{ik}\right)=\frac{{\left|{\Sigma }_k\right|}^{-1/2}}{{(2\pi )}^{J/2}}{\lambda }_{ik}^{J/2}\text{exp}\{-\frac{{\lambda }_{ik}}{2}{\left(y_{ik}-X_{ik}\beta -W_{ik}{\gamma }_k^R-{\Delta }_k{z}_{ik}^{*}\right)}^{\prime }{\Sigma }_k^{-1}(y_{ik}-X_{ik}\beta -W_{ik}{\gamma }_k^R-{\Delta }_k{z}_{ik}^{*})\}$ with $z_{ik}^{*}=z_{ik}-E\left[z_{ik}\right]$, Δ_k = diag(δ_1k,…, δ_Jk), Ω = diag(Ω₁,…,Ω_J), $z=(z_{11}^{\prime },\dots ,{z_{n_{K}K}^{\prime })}^{\prime }$, ${\gamma }^R={\left({\left({\gamma }_1^R\right)}^{\prime },\dots ,{\left({\gamma }_K^R\right)}^{\prime }\right)}^{\prime }$, and $\lambda =({\lambda }_{11},\dots ,{\lambda }_{n_{K}K})$. Then using the complete-data likelihood function in (3.16) and prior distributions in Section 3.2, the joint posterior distribution of all the parameters is given by

$$\pi \left(\beta ,\gamma ,{\delta }^{*},\delta ,{\sigma }_{\delta }^2,{\Sigma }^{*},v,\Sigma ,{\nu }^{*},{\nu }_a,{\nu }_b,\tau ,\Omega ,{\gamma }^R,z,\psi ,\lambda \mid D_{\text{obs}}\right)\propto L\left(\beta ,\gamma ,{\delta }^{*},{\Sigma }^{*},{\nu }^{*},\tau ,\Omega ,{\gamma }^R,z,\psi ,\lambda \mid D_{\text{obs}}\right)\times \pi (\beta )\pi (\gamma ),\pi \left({\delta }^{*}\mid \delta ,{\sigma }_{\delta }^2\right)\pi (\delta )\pi \left({\sigma }_{\delta }^2\right)\times \pi \left({\Sigma }^{*}\mid v,\Sigma \right)\pi (v)\pi (\Sigma )\pi (\Omega )\times \pi \left({\nu }^{*}\mid {\nu }_a,{\nu }_b\right)\pi \left({\nu }_a\right)\pi \left({\nu }_b\right)\pi (\tau )$$ (3.17)

4. BAYESIAN INFERENCE

4.1. Bayesian model comparison via CPO’s

Given the rich specification of our proposed model, it is of interest to compare the performance of various special cases of the general multivariate skew meta-regression model proposed in Section 3.2. That is, we need methods for checking whether a skew and/or heavy-tailed distribution is needed for modeling the y_ik’s. Also, we need to investigate whether the skewness, variance, and degrees of freedom are varying across trials. To this end, we carry out the model comparison using the logarithm of pseudo-marginal likelihood (LPML) proposed by Ibrahim et al. (2001). The LPML is a well-established Bayesian model comparison criterion based on the conditional predictive ordinate (CPO) statistic. The CPO statistic for the ith subject in the kth trial is the marginal posterior predictive density of y_ik. As suggested in Ibrahim et al. (2001), a natural summary statistic of the CPO_ik’s is the LPML defined as

$$\text{LPML}=\sum _k^K\sum _i^I\text{log}\left({\text{CPO}}_{ik}\right).$$ (4.1)

We use LPML as a criterion-based measure for model selection. The larger the LPML, the better the fit of a given model. From (3.16), the marginal likelihood function is given by

$$f\left(y_{ik}\mid \beta ,{\delta }_k,{\Sigma }_k,{\gamma }_k^R,\tau ,{\nu }_k,X_{ik},W_{ik}\right)={\int }_{{\lambda }_{ik}\in R_{+}}{\int }_{{\psi }_{ik}\in R_{+}}{\int }_{z_{ik}\in R_{+}^J}f\left(y_{ik}\mid \beta ,{\delta }_k,{\Sigma }_k,{\gamma }_k^R,z_{ik},{\lambda }_{ik},X_{ik},W_{ik}\right)\times \prod _{j=1}^J[{\psi }_{ik}\text{exp}\left\{-{\psi }_{ik}{z}_{ijk}\right\}d{z}_{ik}\times \frac{{\tau }^{\tau +1}}{\Gamma (\tau +1)}{\psi }_{ik}^{\tau }\text{exp}\left\{-\tau {\psi }_{ik}\right\}d{\psi }_{ik}]\times \frac{{\left(\frac{{\nu }_k}{2}\right)}^{\frac{{\nu }_k}{2}}}{\Gamma \left(\frac{{\nu }_k}{2}\right)}{\lambda }_{ik}^{\frac{{\nu }_k}{2}-1}\text{exp}\left\{-\frac{{\nu }_k}{2}{\lambda }_{ik}\right\}d{\lambda }_{ik}=\frac{{\left|{\Sigma }_k\right|}^{-\frac{1}{2}}}{{\pi }^{\frac{J}{2}}}\frac{\Gamma \left(\frac{{\nu }_k+J}{2}\right)}{\Gamma \left(\frac{{\nu }_k}{2}\right)}{\nu }_k^{\frac{{\nu }_k}{2}}\frac{\Gamma (\tau +J+1)}{\Gamma (\tau +1)}{\tau }^{\tau +1}\times {\int }_{z_{ik}\in R_{+}^J}{\left(A_{ik}+{\nu }_k\right)}^{-\frac{{\nu }_k+J}{2}}{\left(\sum _{j=1}^J{z}_{ijk}+\tau \right)}^{-(J+\tau +1)}d{z}_{ik},$$ (4.2)

where $A_{ik}={\left(\begin{array}{llllllllll}{y}_{ik}\hfill & -\hfill & X_{ik}\hfill & \beta \hfill & -\hfill & W_{ik}\hfill & {\gamma }_k^R\hfill & -\hfill & {\Delta }_k\hfill & z_{ik}^{*}\hfill \end{array}\right)}^{\prime }{\Sigma }_k^{-1}\left(y_{ik}-X_{ik}\beta -W_{ik}\begin{array}{llll}{\gamma }_k^R\hfill & -\hfill & {\Delta }_k\hfill & z_{ik}^{*}\hfill \end{array}\right)$. Let $\theta =\left(\beta ,\gamma ,{\delta }^{*},\delta ,{\sigma }_{\delta }^2,{\Sigma }^{*},v,\Sigma ,{\nu }^{*},{\nu }_a,{\nu }_b,\tau ,\Omega ,{\gamma }^R\right)$ denote the collection of parameters and the random effects. Let $D_{\text{obs}}^{(-ik)}$ denote the observed data with the ith patient in the kth trial deleted. Let Θ and $\mathcal{Z}$ be the parameter spaces corresponding to θ and z, respectively. Also let $\pi \left(\theta \mid D_{\text{obs}}^{(-ik)}\right)$ denote the posterior of θ given $D_{\text{obs}}^{(-ik)}$. The following proposition gives a computational form of the CPO statistic.

Proposition 4.1. Let h_ik (z_ik) be a normalized weight function satisfying ${\int }_{R_{+}^J}{h}_{ik}\left(z_{ik}\right)d{z}_{ik}=1$. For the ith patient in the kth trial, CPO_ik can be written as

$$\begin{gathered} CP{O}_{ik}={\int }_{\Theta }f\left(y_{ik}\mid \beta ,{\delta }_k,{\Sigma }_k,{\gamma }_k^R,\tau ,{\nu }_k,X_{ik},W_{ik}\right)\times \pi \left(\theta \mid D_v^{(-ik)}\right)d\theta \\ ={\left[{\int }_{\Theta }{\int }_{\mathcal{Z}}{f}_{y_{ik},z_{ik}}\times \pi \left(\theta ,z\mid D_{\text{obs}}\right)dzd\theta \right]}^{-1}, \end{gathered}$$ (4.3)

where $f_{y_{ik},z_{ik}}=\frac{h_{ik}\left(z_{ik}\right)}{f\left(y_{ik},z_{ik}\mid \beta ,{\delta }_k,{\Sigma }_k,{\gamma }_k^R,\tau ,{\nu }_k,X_{ik},W_{ik}\right)}$ and $f(y_{ik}{z}_{ik}\mid \beta ,{\delta }_k,{\Sigma }_k,{\gamma }_k^R,\tau ,{\nu }_k,X_{ik},W_{ik})={\left(A_{ik}+{\nu }_k\right)}^{-\frac{{\nu }_k+J}{2}}{\left({\sum }_{j=1}^J{z}_{ijk}+\tau \right)}^{-(J+\tau +1)}\frac{{\left|{\Sigma }_k\right|}^{-\frac{1}{2}}}{{\pi }^{\frac{J}{2}}}\frac{\Gamma \left(\frac{{\nu }_k+J}{2}\right)}{\Gamma \left(\frac{{\nu }_k}{2}\right)}{\nu }_k^{\frac{{\nu }_k}{2}}\frac{\Gamma (\tau +J+1)}{\Gamma (\tau +1)}{\tau }^{\tau +1}$.

The proof of this proposition is given in Appendix A.

Remark 4.1. Using the CPO Identity I in Zhang et al. (2017), CPO_ik can be written as

$${\text{CPO}}_{ik}=\frac{1}{{\int }_{\Theta }\frac{1}{f\left(y_{ik}\mid \beta ,{\delta }_k,{\Sigma }_k,{\gamma }_k^R,\tau ,{\nu }_k,X_{ik},W_{ik}\right)}\pi \left(\theta \mid D_{\text{obs}}\right)d\theta },$$

where f(y_ik | β, δ_k, Σ_k, ${\gamma }_k^R$, τ, v_k, X_ik, W_ik) is given in (4.2). This identity requires a J-dimensional integration over $R_{+}^J$. Compared to the CPO identity I, (4.3) is more computationally attractive.

Remark 4.2. The CPO_ik given in (4.3) uses the CPO Identity II in Zhang et al. (2017). Now, let {(θ^(b), z^(b)), b = 1, …, B} denote a Gibbs sample of (θ, z) from π(θ, z | D_obs). Using Proposition 4.1, a Monte Carlo estimator of CPO_ik in (4.3) is given by

$${\widehat{\text{CPO}}}_{ik}=B{\left[\sum _{b=1}^B{f}_{y_{ik},z_{ik}}^{(b)}\right]}^{-1},$$ (4.4)

where $f_{yik}^{(b)},z_{ik}=\frac{h_{ik}\left(z_{ik}^{(b)}\right)}{f\left(y_{ik},z_{ik}^{(b)}\mid {\beta }^{(b)},{\delta }_k^{(b)},{\Sigma }_k^{(b)},{\left({\gamma }_k^R\right)}^{(b)},{\tau }^{(b)},{\nu }_k^{(b)},X_{ik},W_{ik}\right)}$.

The Monte Carlo error of ${\widehat{\text{CPO}}}_{ik}$ given in (4.4) depends on the choice of the weight function h_ik(z_ik). Following Theorem 1 in Zhang et al. (2017), the optimal weight function is defined as the weight function minimizing the variance of the Monte Carlo estimator. Here, we have

$$h_{ik,\text{opt}}\left(z_{ik}\right)=\frac{f\left(y_{ik},z_{ik}\mid \beta ,{\delta }_k,{\Sigma }_k,{\gamma }_k^R,\tau ,{\nu }_k,X_{ik},W_{ik}\right)}{f\left(y_{ik}\mid \beta ,{\delta }_k,{\Sigma }_k,{\gamma }_k^R,\tau ,{\nu }_k,X_{ik},W_{ik}\right)}.$$ (4.5)

However, the optimal weight function is not analytically available and difficult to compute since the denominator involves a J-dimensional integration. We can, instead, use a multivariate normal density as a possible choice of h_ik(z_ik), which is constructed via the Laplace approximation to the joint density f(y_ik, z_ik | β, δ_k, Σ_k, ${\gamma }_k^R$, τ, v_k, X_ik, W_ik). Let u_ik = log(z_ik) for i = 1, …, n_k, k = 1, …, K, then z_ik = exp(u_ik). Here, log and exp functions applied to a vector means that the operations are applied to every element of the vector. Denote $g\left(u_{ik}\right)=\text{log}\{f(y_{ik},\text{exp}\left(u_{ik}\right)\mid \beta ,{\delta }_k,{\Sigma }_k,{\gamma }_k^R,\tau ,{\nu }_k,X_{ik},W_{ik}){\prod }_{j=1}^J\text{exp}\left(u_{ijk}\right)\}$. Let $u_{ik}^0$ be the stationary point, where ∇g(u_ik) = 0, and $B=-{{\nabla }^{2}g\left(u_{ik}\right)|}_{u=u_{ik}^0}$. Then, the optimal weight function h_ik,opt(z_ik) is approximately given by $\frac{f\left(\text{log}\left(z_{ik}\right)\mid u_{ik}^0,B^{-1}\right)}{{\prod }_{j=1}^J{z}_{ijk}}$, where $f\left(\cdot \mid u_{ik}^0,B^{-1}\right)$ denotes the density of a $N\left(u_{ik}^0,B^{-1}\right)$ distribution. A detailed derivation of this approximated optimal weight function is given in Appendix B.

4.2. Computational development

We consider the following one-to-one transformations: ${\gamma }_k^{*R}={\left({\gamma }_{1k}^{*\prime },{\gamma }_{2k}^{*\prime },\dots ,{\gamma }_{Jk}^{*\prime }\right)}^{\prime }={\gamma }_k^R-\gamma$ and ${\delta }_k^{*}={\left({\delta }_{1k}^{*\prime },{\delta }_{2k}^{*\prime },\dots ,{\delta }_{Jk}^{*\prime }\right)}^{\prime }={\delta }_k-\delta$ for k = 1,…, K. Thus, ${\gamma }_{jk}^{*}={\gamma }_{jk}-{\gamma }_j$ and ${\delta }_{jk}^{*}={\delta }_{jk}-{\delta }_j$ for j = 1,…, J and k = 1,…,K. Write ${\gamma }^{*R}={\left({\left({\gamma }_1^{*R}\right)}^{\prime },\dots ,{\left({\gamma }_K^{*R}\right)}^{\prime }\right)}^{\prime }$ and ${\delta }^{**}=\left({\delta }_1^{*\prime },\dots ,{\delta }_K^{*\prime }\right)$. Let ${\Delta }_k^{*}=\text{diag}\left({\delta }_{1k}^{*},\dots ,{\delta }_{Jk}^{*}\right)$ and Δ = diag(δ₁, …, δ_J). Also, let $X_{ik}^{*}=\left(X_{ik},W_{ik}\right)$ for i = 1,…, n and k = 1,…, K and θ = (β′, γ′)′. We present a detailed development of the MCMC sampling algorithm. Although the analytic evaluation of the joint posterior distribution of (θ, δ**, δ, ${\sigma }_{\delta }^2$, Σ*,υ, Σ, v*, v_a, v_b, τ, Ω, γ*^R, z, ψ, λ) based on the observed data D_obs given in Equation (3.17) is not possible, the proposed model allows us to develop an efficient MCMC sampling algorithm to sample from (3.17). The MCMC sampling algorithm requires sampling from the following full conditional distributions in turn: (i) [γ*^R, θ, δ**, δ, ${\sigma }_{\delta }^2\mid {\Sigma }^{*}$, υ, Σ, ν*, ν_a, ν_b, τ, Ω, z, ψ, λ, D_obs]; (ii) [λ, ν*, ν_a, ν_b | θ, δ**, δ, ${\sigma }_{\delta }^2$, Σ*, υ, Σ, τ, Ω, γ*^R, z, ψ, D_obs]; (iii) [Σ*, υ, Σ | θ, δ**, δ, ${\sigma }_{\delta }^2$, ν*, ν_a, ν_b, τ, Ω, γ*^R, z, ψ, λ, D_obs]; (iv) [z | θ, δ**, δ, ${\sigma }_{\delta }^2$, Σ*, υ, Σ, ν*, ν_a, ν_b, τ, Ω, γ*^R, ψ, λ, D_obs]; (v) [ψ, τ | θ, δ**, δ, ${\sigma }_{\delta }^2$, Σ*, υ, Σ, ν*, ν_a, ν_b, Ω, γ*^R, z, λ, D_obs]; and (vi) [Ω | θ, δ**, δ, ${\sigma }_{\delta }^2$, Σ*, υ, Σ, ν*, ν_a, ν_a, τ, γ*^R, z, ψ, λ, D_obs]. For (i), we apply the collapsed Gibbs technique of Liu (1994) and Chen et al. (2000) through the identity

$$\left[{\gamma }^{*R},\theta ,{\delta }^{**},\delta ,{\sigma }_{\delta }^2\mid {\Sigma }^{*},v,\Sigma ,{\nu }^{*},{\nu }_a,{\nu }_b,\tau ,\Omega ,z,\psi ,\lambda ,D_{\text{obs}}\right]=\left[{\gamma }^{*R}\mid \theta ,{\delta }^{**},\delta ,{\sigma }_{\delta }^2,{\Sigma }^{*},v,\Sigma ,{\nu }^{*},{\nu }_a,{\nu }_b,\tau ,\Omega ,z,\psi ,\lambda ,D_{\text{obs}}\right]\times \left[{\delta }^{**}\mid \theta ,\delta ,{\sigma }_{\delta }^2,{\Sigma }^{*},v,\Sigma ,{\nu }^{*},{\nu }_a,{\nu }_b,\tau ,\Omega ,z,\psi ,\lambda ,D_{\text{obs}}\right]\times \left[\theta ,\delta ,{\sigma }_{\delta }^2\mid {\Sigma }^{*},v,\Sigma ,{\nu }^{*},{\nu }_a,{\nu }_b,\tau ,\Omega ,z,\psi ,\lambda ,D_{\text{obs}}\right].$$ (4.6)

That is, we sample δ** after collapsing out γ*^R, and also sample θ, δ, and ${\sigma }_{\delta }^2$ after collapsing out γ*^R and δ**. For (ii), we apply the collapsed Gibbs technique of Liu (1994) and Chen et al. (2000) through the identity

$$\left[\lambda ,{\nu }^{*},{\nu }_a,{\nu }_b\mid \theta ,{\delta }^{**},\delta ,{\sigma }_{\delta }^2,{\Sigma }^{*},v,\Sigma ,\tau ,\Omega ,{\gamma }^{*R},z,\psi ,D_{\text{obs}}\right]=\left[\lambda \mid \theta ,{\delta }^{**},\delta ,{\sigma }_{\delta }^2,{\Sigma }^{*},v,\Sigma ,{\nu }^{*},{\nu }_a,{\nu }_b,\tau ,\Omega ,{\gamma }^{*R},z,\psi ,D_{\text{obs}}\right]\times \left[{\nu }_b\mid \theta ,{\delta }^{**},\delta ,{\sigma }_{\delta }^2,{\Sigma }^{*},v,\Sigma ,{\nu }^{*},{\nu }_a,\tau ,\Omega ,{\gamma }^{*R},z,\psi ,D_{\text{obs}}\right]\times \left[{\nu }^{*},{\nu }_a\mid \theta ,{\delta }^{**},\delta ,{\sigma }_{\delta }^2,{\Sigma }^{*},v,\Sigma ,\tau ,\Omega ,{\gamma }^{*R},z,\psi ,D_{\text{obs}}\right].$$ (4.7)

That is, we sample ν_b after collapsing out λ, and also sample ν* and ν_a after collapsing out λ and ν_b. For (iii), we apply the collapsed Gibbs technique of Liu (1994) and Chen et al. (2000) through the identity

$$\left[{\Sigma }^{*},v,\Sigma \mid \theta ,{\delta }^{**},\delta ,{\sigma }_{\delta }^2,{\nu }^{*},{\nu }_a,{\nu }_b,\tau ,\Omega ,{\gamma }^{*R},z,\psi ,\lambda ,D_{\text{obs}}\right]=\left[{\Sigma }^{*}\mid \theta ,{\delta }^{**},\delta ,{\sigma }_{\delta }^2,v,\Sigma ,{\nu }^{*},{\nu }_a,{\nu }_b,\tau ,\Omega ,{\gamma }^{*R},z,\psi ,\lambda ,D_{\text{obs}}\right]\times \left[v,\Sigma \mid \theta ,{\delta }^{**},\delta ,{\sigma }_{\delta }^2,{\nu }^{*},{\nu }_a,{\nu }_b,\tau ,\Omega ,{\gamma }^{*R},z,\psi ,\lambda ,D_{\text{obs}}\right].$$ (4.8)

That is, we sample υ and Σ after collapsing out Σ*. For (v), we apply the collapsed Gibbs technique of Liu (1994) and Chen et al. (2000) through the identity

$$\left[\psi ,\tau \mid \theta ,{\delta }^{**},\delta ,{\sigma }_{\delta }^2,{\Sigma }^{*},v,\Sigma ,{\nu }^{*},{\nu }_a,{\nu }_b,\Omega ,{\gamma }^{*R},z,\lambda ,D_{\text{obs}}\right]=\left[\psi \mid \theta ,{\delta }^{**},\delta ,{\sigma }_{\delta }^2,{\Sigma }^{*},v,\Sigma ,{\nu }^{*},{\nu }_a,{\nu }_b,\tau ,\Omega ,{\gamma }^{*R},z,\lambda ,D_{\text{obs}}\right]\times [\tau \mid \theta ,{\delta }^{**},\delta ,{\sigma }_{\delta }^2,{\Sigma }^{*},v,\Sigma ,{\nu }^{*},{\nu }_a,{\nu }_b,\Omega ,{\gamma }^{*R},\psi ,z,\lambda ,D_{\text{obs}}].$$ (4.9)

All the full conditional distributions discussed above are presented in Section S1 of the Supplementary Materials (http://intlpress.com/site/pub/files/_supp/sii/2020/0013/0004/SII-2020-0013-0004-s004.pdf).

5. ANALYSIS OF THE CHOLESTEROL DATA

We re-analyze the cholesterol data discussed in Section 2. In (3.3), x_ijk consists of 14 covariates, including bl_ldlc, bl_hdlc, bl_tg, BMI, age, duration, Female, DM, CHD, potency2, potency3, black, hispanic, and other, as well as 14 interaction terms between the 14 covariates and onstatin as in Ibrahim et al. (2019). We model these three outcome variables LDL-C, HDL-C, and TG jointly via (3.3), (3.4), (3.5), and (3.6) with J = 3 and K = 26 in conjunction with the models specified at Stage 2 and the priors specified at Stage 3. We standardized all the fourteen covariates for numerical stability in the posterior computations.

As shown in Ibrahim et al. (2019), the multivariate skew meta-regression model with an unstructured covariance matrix for the multivariate outcome variables outperformed the symmetric normal and t models as well as the skew models with a diagonal covariance matrix. Thus, for the cholesterol data, we only fit the flexible multivariate meta-regression models defined by (3.3), (3.4), (3.5), and (3.6) with different fixed values of τ in (3.6) and random τ with prior specified in (3.13). We also consider different fixed values of ν_a in (3.10) and random ν_a with a prior specified in (3.14). In total, we consider 14 different models, including the one, which was the best model considered in Ibrahim et al. (2019), and the values of LPML are reported in Table 1. We see, from Table 1, that the LPML values under the proposed flexible skew t models are greater than the one (LPML = −262517.61) under the skew t model with τ = 30 (Ibrahim et al. 2019). The best flexible skew t model is the one with random ν_a and τ, which has LPML = −261321.15 while the second best model is the one with ν_a = 25 and τ = 5. The values of LPML for these models are −261321.15 and −261322.38, which are very close.

We extend ${\lambda }_k^{*}$ in (3.2) to

$${\lambda }_k^{*}=\left({\lambda }_k-n_k\right)/{\left(n_k/\left({\nu }_k/2\right)\right)}^{1/2}$$ (5.1)

to account for different degrees of freedom for the kth trial under the proposed flexible skew meta-regression model. Figure 2 shows the boxplots of the ${\lambda }_k^{*}\text{'}\text{s}$ defined in (5.1) under the flexible skew t model random ν_a and τ. The boxplots of the ${\lambda }_k^{*}\text{'}\text{s}$ under the flexible skew t model with ν_a = 25 and τ = 5 are shown in Figure S.1 of the Supplementary Materials. We see from Figure 2 and Figure S.1 that all of these boxplots had a median close to zero and no obvious outlying trials were found from these two figures. These two figures are quite different than Figure 2 of Ibrahim et al. (2019), in which boxplots corresponding to trials 8 and 25 were quite different than the rest of the 24 boxplots and these boxplots were much more heterogenous than those in Figures 2 and Figure S.1 under the proposed models. Thus, the outlying trials identified in Ibrahim et al. (2019) could be due to a lack of fit.

Figure 2.

Boxplots of the ${\lambda }_k^{*}\text{'}\text{s}$ from the posterior distribution under flexible skew model with random ϕ and τ for the cholesterol data.

The posterior estimates, including posterior means, posterior standard deviations (SDs), and 95% highest posterior density (HPD) intervals of the parameters under the flexible skew t model with random ν_a and τ are reported in Tables 2–6 and Tables S.1–S.3. Those posterior estimates under the flexible skew t model with ν_a = 25 and τ = 5 are reported in Tables S.4–S.11 of the Supplementary Materials. The posterior means and the 95% HPD intervals of the 28 regression coefficients in Tables 2–4 (or Tables S.4–S.6) under the proposed flexible skew models and the skew t model with τ = 30 (Ibrahim et al., 2019) are also plotted in Figure 3 and Figure S.2 of the Supplementary Materials. We call a posterior estimate “statistically significant at a significance level of 0.05” if the corresponding 95% HPD interval does not contain 0. Under this notion, we see from Figures 2 and S.2 that significant posterior estimates are consistent between the proposed model and the model of Ibrahim et al. (2019) for most of the regression coefficients except for a few coefficients. For example, for LDL-C, onstatin × DM (β_1,22) was not significant with 95% HPD intervals (−1.598, 0.281) and (−1.553, 0.322) under the proposed models with random ν_a and τ (Table 2) and with ν_a = 25 and τ = 5 (Table S.4), respectively, while it was significant with 95% HPD interval (−2.088, −0.225) under the skew t model with τ = 30 (Table 7 of Ibrahim et al. (2019)). For HDL-C, onstatin × potency2 (β_2,24) was nearly significant with HPD intervals (−2.094, −0.029) and (−2.053, 0.000) under the proposed models with random ν_a and τ (Table 3) and with ν_a = 25 and τ = 5 (Table S.5), respectively, while it was not significant with 95% HPD interval (−1.955, 0.068) under the skew t model with τ = 30 (Table 8 of Ibrahim et al. (2019)). For TG, onstatin × duration (β_3,20) was not significant with HPD intervals (−0.242, 1.380) and (−0.215, 1.390) under the proposed models with random ν_a and τ (Table 4) and with ν_a = 25 and τ = 5 (Table S.6), respectively, while it was significant with 95% HPD interval (0.227, 1.539) under the skew t model with τ = 30 (Table 9 of Ibrahim et al. (2019)). Therefore, different models may identify different sets of significant covariates. Since the proposed flexible skew model fits the cholesterol data much better than the skew t model with τ = 30, the latter model may potentially incorrectly identify the association between the outcome variables and covariates, yielding a misleading conclusion in terms of the clinical importance of covariates.

Table 6.

Posterior Estimates under the flexible skew t model with random ν_a and τ

Par.	Mean	SD	95% HPD	Par.	Mean	SD	95% HPD
v1	7.789	1.189	(5.593, 10.179)	v14	7.208	0.961	(5.508, 9.191)
v2	8.694	1.554	(5.998, 11.880)	v15	5.408	0.619	(4.258, 6.645)
v3	7.511	1.514	(4.649, 10.531)	v16	8.761	1.418	(6.285, 11.631)
v4	8.520	1.328	(6.243, 11.313)	v17	8.418	1.772	(5.182, 11.890)
v5	9.176	1.536	(6.449, 12.285)	v18	9.272	1.730	(6.292, 12.851)
v6	8.560	1.419	(5.989, 11.384)	v19	10.479	2.113	(6.847, 14.652)
v7	9.728	1.521	(6.958, 12.749)	v20	8.379	1.506	(5.662, 11.390)
v8	7.000	0.581	(5.900, 8.141)	v21	6.626	1.067	(4.680, 8.731)
v9	8.150	0.906	(6.412, 9.894)	v22	6.671	1.198	(4.527, 9.110)
v10	10.103	1.113	(8.024, 12.286)	v23	6.417	1.093	(4.403, 8.575)
v11	7.910	0.978	(6.087, 9.844)	v24	7.593	1.362	(5.128, 10.317)
v12	5.809	1.173	(3.609, 8.091)	v25	6.825	1.003	(4.937, 8.806)
v13	6.838	1.114	(4.860, 9.160)	v26	6.235	0.923	(4.537, 8.080)

Table 4.

Posterior Estimates under the flexible skew t model with random ν_a and τ for TG

	Par.	Mean	SD	95% HPD
bl_ldlc	β3,1	−0.002	0.006	(−0.013, 0.009)
bl_hdlc	β3,2	0.030	0.017	(−0.002, 0.063)
bl_tg	β3,3	−0.093	0.003	(−0.098, −0.088)
BMI	β3,4	0.237	0.029	(0.181, 0.295)
age	β3,5	−0.032	0.016	(−0.064, 0.000)
duration	β3,6	−0.186	0.182	(−0.566, 0.140)
Female	β3,7	2.570	0.361	(1.879, 3.279)
DM	β3,8	−0.549	0.480	(−1.465, 0.417)
CHD	β3,9	0.680	0.471	(−0.243, 1.591)
potency2	β3,10	−4.303	0.572	(−5.422, −3.179)
potency3	β3,11	−9.458	0.666	(−10.775, −8.178)
Black	β3,12	−1.997	0.665	(−3.334, −0.744)
Hispanic	β3,13	1.045	0.805	(−0.512, 2.615)
Other	β3,14	−1.012	0.838	(−2.697, 0.573)
onstatin × bl_ldlc	β3,15	0.015	0.010	(−0.004, 0.036)
onstatin × bl_hdlc	β3,16	−0.093	0.028	(−0.150, −0.039)
onstatin × bl_tg	β3,17	−0.026	0.005	(−0.036, −0.017)
onstatin × BMI	β3,18	0.038	0.054	(−0.064, 0.146)
onstatin × age	β3,19	0.096	0.029	(0.040, 0.153)
onstatin × duration	β3,20	0.605	0.413	(−0.242, 1.380)
onstatin × Female	β3,21	0.262	0.639	(−0.975, 1.513)
onstatin × DM	β3,22	2.221	0.766	(0.715, 3.684)
onstatin × CHD	β3,23	0.922	0.804	(−0.690, 2.482)
onstatin × potency2	β3,24	3.468	1.063	(1.384, 5.549)
onstatin × potency3	β3,25	6.237	1.302	(3.623, 8.732)
onstatin × Black	β3,26	−1.101	1.316	(−3.662, 1.482)
onstatin × Hispanic	β3,27	0.896	1.603	(−2.240, 4.007)
onstatin × Other	β3,28	3.854	1.458	(1.039, 6.738)

Figure 3.

Plots of the relative posterior estimates (mean/SD and 95% HPD interval/SD) for β under the skew t model with τ = 30 (blue) and flexible skew model with random ν_a and τ (red) for the cholesterol data.

Table 3.

Posterior Estimates under the flexible skew t model with random ν_a and τ for HDL-C

	Par.	Mean	SD	95% HPD
bl_ldlc	β2,1	0.002	0.003	(−0.004, 0.008)
bl_hdlc	β2,2	−0.197	0.010	(−0.217, −0.178)
bl_tg	β2,3	0.024	0.001	(0.021, 0.027)
BMI	β2,4	−0.145	0.017	(−0.179, −0.112)
age	β2,5	0.075	0.009	(0.057, 0.094)
duration	β2,6	0.065	0.063	(−0.057, 0.190)
Female	β2,7	0.523	0.212	(0.114, 0.941)
DM	β2,8	−2.270	0.278	(−2.819, −1.728)
CHD	β2,9	−0.985	0.273	(−1.536, −0.465)
potency2	β2,10	0.790	0.317	(0.176, 1.424)
potency3	β2,11	0.256	0.367	(−0.445, 0.989)
Black	β2,12	−2.448	0.371	(−3.187, −1.730)
Hispanic	β2,13	−1.097	0.449	(−1.972, −0.211)
Other	β2,14	−0.050	0.504	(−1.051, 0.920)
onstatin × bl_ldlc	β2,15	−0.007	0.005	(−0.017, 0.003)
onstatin × bl_hdlc	β2,16	−0.006	0.014	(−0.035, 0.021)
onstatin × bl_tg	β2,17	−0.016	0.002	(−0.020, −0.011)
onstatin × BMI	β2,18	0.027	0.027	(−0.026, 0.081)
onstatin × age	β2,19	−0.059	0.014	(−0.088, −0.031)
onstatin × duration	β2,20	−0.245	0.194	(−0.638, 0.128)
onstatin × Female	β2,21	0.786	0.324	(0.161, 1.427)
onstatin × DM	β2,22	1.719	0.390	(0.914, 2.462)
onstatin × CHD	β2,23	0.464	0.395	(−0.299, 1.248)
onstatin × potency2	β2,24	−1.052	0.526	(−2.094, −0.029)
onstatin × potency3	β2,25	−0.946	0.643	(−2.176, 0.338)
onstatin × Black	β2,26	2.571	0.634	(1.328, 3.819)
onstatin × Hispanic	β2,27	−0.436	0.769	(−1.967, 1.038)
onstatin × Other	β2,28	−0.895	0.742	(−2.300, 0.579)

The results shown in Table 5 under the flexible skew model with random ν_a and τ, Table S.7 under the flexible skew model with ν_a = 25 and τ = 5, and Table 6 of Ibrahim et al. (2019) indicate that patients on “statin + EZE” had significantly more percent changes from baseline in all three outcome variables (LDL-C, HDL-C, and TG) than those on statin alone for both the first-line and second-line therapies. For first-line therapy, the 95% HPD intervals were (−15.771, −12.492), (−15.748, −12.455), and (−15.662, −12.454) for the percent change from baseline in LDL-C (γ_1,1); (1.265, 2.956), (1.302, 2.952) and (1.285, 2.870) for the percent change from baseline in HDL-C (γ_2,1); and (−7.304, −4.523), (−7.357, −4.548), and (−7.316, −4.369) for the percent change from baseline in TG (γ_3,1), respectively, under the flexible skew model with random ν_a and τ, the flexible skew model with ν_a = 25 and τ = 5, and the skew model with τ = 30. For second-line therapy, these 95% HPD intervals were (−21.849, −16.293), (−21.840, −16.260), and (−21.586, −15.974) for the percent change from baseline in LDL-C (γ_1,3); (0.763, 1.773), (0.740, 1.751), and (0.726, 1.736) for the percent change from baseline in HDL-C (γ_2,3); and (−9.373, −6.448), (−9.306, −6.452), and (−9.213, −5.868) for the percent change from baseline in TG (γ_3,3), respectively, under the above three models.

Table 5.

Posterior Estimates under the flexible skew t model with random ν_a and τ

Par.	Mean	SD	95% HPD	Par.	Mean	SD	95% HPD
γ1,0	−22.112	3.034	(−28.094, −16.127)	Ω1,0,0	20.133	10.345	(6.526, 39.943)
γ1,1	−14.129	0.827	(−15.748, −12.455)	Ω1,0,1	−9.158	5.367	(−19.859, −1.405)
γ1,2	6.229	4.290	(−2.232, 14.651)	Ω1,1,1	8.197	3.900	(2.791, 15.742)
γ1,3	−19.107	1.410	(−21.849, −16.293)	Ω1,2,2	43.791	21.636	(13.970, 84.722)
γ2,0	9.918	1.354	(7.291, 12.594)	Ω1,2,3	−26.458	14.770	(−54.154, −5.612)
γ2,1	2.111	0.430	(1.265, 2.956)	Ω1,3,3	24.157	12.438	(7.482, 47.977)
γ2,2	13.850	1.818	(10.345, 17.446)	Ω2,0,0	1.957	1.085	(0.530, 4.033)
γ2,3	1.256	0.260	(0.763, 1.773)	Ω2,0,1	−1.734	0.990	(−3.616, −0.395)
γ3,0	−1.626	2.679	(−6.795, 3.708)	Ω2,1,1	1.680	0.992	(0.344, 3.587)
γ3,1	−5.918	0.704	(−7.304, −4.523)	Ω2,2,2	1.182	0.817	(0.121, 2.657)
γ3,2	4.180	3.665	(−2.717, 11.540)	Ω2,2,3	−0.188	0.355	(−0.930, 0.366)
γ3,3	−7.923	0.738	(−9.373, −6.448)	Ω2,3,3	0.137	0.161	(0.007, 0.419)
Σ11	72.921	4.189	(64.870, 81.231)	Ω3,0,0	9.561	6.304	(1.245, 21.757)
Σ12	13.629	3.067	(7.729, 19.587)	Ω3,0,1	−6.229	4.040	(−14.125, −0.545)
Σ13	22.812	4.459	(14.018, 31.508)	Ω3,1,1	4.463	3.107	(0.218, 10.459)
Σ22	96.721	5.193	(86.786, 107.018)	Ω3,2,2	9.301	5.919	(1.423, 20.324)
Σ23	−37.165	5.046	(−47.266, −27.514)	Ω3,2,3	−5.427	4.046	(−13.380, −0.181)
Σ33	182.385	10.700	(161.945, 203.932)	Ω3,3,3	3.574	3.221	(0.017, 9.468)
δ1	9.651	0.612	(8.438, 10.865)	${\sigma }_{{\delta }_1}^2$	8.242	3.181	(3.345, 14.545)
δ2	−1.522	0.189	(−1.907, −1.166)	${\sigma }_{\delta 2}^2$	0.154	0.133	(0.016, 0.403)
δ3	19.112	0.820	(17.518, 20.752)	${\sigma }_{{\delta }_3}^2$	13.696	5.151	(5.273, 23.948)
ϕ	19.672	8.486	(6.197, 36.399)	v0	33.921	4.472	(25.751, 43.207)
υ	7.863	0.523	(6.886, 8.926)	τ	4.350	0.290	(3.787, 4.914)

Tables 6 and S.8 as well as Figures 4 and S.3 show that the posterior estimates of the degrees of freedom, ν_k’s, vary across trials, with the posterior estimates from 5.408 to 10.479 (Table 6) and 5.452 to 9.778 (Table S.8). Tables S.1 and S.9 and Figures 5 and S.4 indicate that the skewness parameters, δ_k,j’s, are very heterogenous for outcome variables LDL-C and TG and relatively homogenous for HDL-C among the 26 trials. Finally, we see from Tables S.2, S.3, S.10, and S.11 that the magnitudes of the covariances and the variances are different across these 26 trials, however, interestingly, the signs of the correlations among the three outcome variables (LDL-C, HDL-C, and TG) are consistent across trials. These posterior estimates suggest that the skewness parameters, the covariance matrices of the multivariate responses, and the degrees of freedom in the multivariate t distributions for the error terms should be different across trials, which further empirically confirms the finding according to the LPML criterion that the flexible skew model did fit the cholesterol data better than the skew t model with τ = 30.

Figure 5.

Plots of the posterior estimates (mean and 95% HPD interval) for δ_k under the flexible skew model with random ν_a and τ for the cholesterol data.

To compute posterior estimates, including posterior means, posterior SDs, 95% HPD intervals, LPMLs, and boxplots of ${\lambda }_k^{*}\text{'}\text{s}$, we used 20,000 MCMC samples, which were taken from every fifth iteration, after a burn-in of 20,000 iterations. The convergence of the MCMC sampling algorithm was checked using several diagnostic procedures discussed in Chen et al. (2000). The HPD intervals were computed via the Monte Carlo method developed by Chen and Shao (1999). Computer code was written for the FORTRAN 95 compiler using IMSL subroutines with double precision accuracy. The FORTRAN code is available from the authors upon request.

6. DISCUSSION

In this paper, we have proposed a general and flexible multivariate skew and heavy-tailed meta-regression model for modeling individual level patient meta-data, which is a novel extension of the multivariate skew meta-regression model of Ibrahim et al. (2019). Due to the complexity of the proposed model, we have also developed an efficient MCMC sampling algorithm using the collapsed Gibbs technique of Liu (1994) and Chen et al. (2000) to carry out challenging posterior computations due to the large size of the meta-data and the high-dimensions of the random effects. In addition, we have proposed the logarithm of the pseudo marginal likelihood for model comparison. As was seen from the analysis of the (LDL-C, HDL-C, TG) data, the proposed model has substantially improved goodness-of-fit over the one developed in Ibrahim et al. (2019).

An extension to network meta-regression (NMR) of the proposed model for individual level patient network meta-data can be developed. Under the network regression setting, multiple treatments are compared using both direct comparisons of interventions within randomized controlled trials and indirect comparisons across trials based on a common comparator, accounting for covariates. Such an extension is potentially useful and significant in comparing and assessing the effects of cholesterol lowering drugs. A detailed development of this extension is beyond the scope of the current paper.

Supplementary materials available on the journal website consist of the full conditional distributions, additional tables (Tables S.1–S.3) of the posterior estimates under the flexible skewed model with random ν_a and τ for the cholesterol data, and the posterior estimates (Table S.4–S.11; Figure S.1–S.4) under the flexible skewed model with ν_a = 25 and τ = 5.

APPENDIX

Appendix A. Proof of Proposition 4.1

The CPO_ik statistic for the ith patient in the kth trial is defined as

$$\begin{gathered} {\text{CPO}}_{ik}={\int }_{\Theta }f\left(y_{ik}\mid X_{ik},W_{ik},\theta \right)\pi \left(\theta \mid D_{\text{obs}}^{(-ik)}\right)d\theta \\ ={\int }_{\Theta }f\left(y_{ik}\mid X_{ik},W_{ik},\theta \right)\times \frac{f\left(y_{-ik}\mid X_{-ik},W_{-ik},\theta \right)\pi (\theta )}{{\int }_{\Theta }f\left(y_{-ik}\mid X_{-ik},W_{-ik},\theta \right)\pi (\theta )d\theta }d\theta \\ ={\int }_{\Theta }{\int }_{\mathcal{Z}}\frac{\left[\prod _{k=1}^K\prod _{i=1}^{n_k}{f}_{y_{ik},z_{ik}}^A\right]\pi (\theta )}{{\int }_{\Theta }f\left(y_{-ik}\mid X_{-ik},W_{-ik},\theta \right)\pi (\theta )d\theta }dzd\theta , \end{gathered}$$ (A.1)

where $f_{y_{ik},z_{ik}}^A=f\left(y_{ik}\mid X_{ik},W_{ik},\theta ,z_{ik}\right)\pi \left(z_{ik}\mid \tau \right)$.

Let ${\mathcal{Z}}_{-ik}$ denote the integration region of z_-ih. Note that

$$\begin{gathered} {\int }_{\Theta }f\left(y_{-ik}\mid X_{-ik},W_{-ik},\theta \right)\pi (\theta )d\theta ={\int }_{\Theta }{\int }_{{\mathcal{Z}}_{-ik}}f\left(y_{-ik}\mid X_{-ik},W_{-ik},\theta ,z_{-ik}\right)\times \pi \left(z_{-ik}\mid \tau \right)\pi (\theta )d{z}_{-ik}d\theta \\ ={\int }_{\Theta }{\int }_{\mathcal{Z}}{h}_{ik}\left(z_{ik}\right)f\left(y_{-ik}\mid X_{-ik},W_{-ik},\theta ,z_{-ik}\right)\times \pi \left(z_{-ik}\mid \tau \right)\pi (\theta )dzd\theta , \end{gathered}$$ (A.2)

where h_ik (z_ik) is any weight density function such that ${\int }_{z_{ik}}{h}_{ik}\left(z_{ik}\right)d{z}_{ik}=1$, and

$$f\left(y_{-ik}\mid X_{-ik},W_{-ik},\theta ,z_{-ik}\right)\pi \left(z_{-ik}\mid \tau \right)=\frac{1}{f\left(y_{ik}\mid X_{ik},W_{ik},\theta ,z_{ik}\right)\pi \left(z_{ik}\mid \tau \right)}\times \left[\prod _{k=1}^K\prod _{i=1}^{n_k}f\left(y_{ik}\mid X_{ik},W_{ik},\theta ,z_{ik}\right)\pi \left(z_{ik}\mid \tau \right)\right].$$ (A.3)

Thus, combining (A.1), (A.2), and (A.3) yields (4.3).

Appendix B. Derivation of an approximation of the optimal weight function

The optimal weight function is given by

$$\begin{gathered} h_{ik,opt}\left(z_{ik}\right)=f\left(z_{ik}\mid \beta ,{\delta }_k,{\Sigma }_k,{\gamma }_k^R,\tau ,{\nu }_k,y_{ik},X_{ik},W_{ik}\right) \\ =\frac{f\left(y_{ik},z_{ik}\mid \beta ,{\delta }_k,{\Sigma }_k,{\gamma }_k^R,\tau ,{\nu }_k,X_{ik},W_{ik}\right)}{f\left(y_{ik}\mid \beta ,{\delta }_k,{\Sigma }_k,{\gamma }_k^R,\tau ,{\nu }_k,X_{ik},W_{ik}\right)} \\ =\frac{f\left(y_{ik},z_{ik}\mid \beta ,{\delta }_k,{\Sigma }_k,{\gamma }_k^R,\tau ,{\nu }_k,X_{ik},W_{ik}\right)}{f^M\left(y_{ik},z_{ik}\mid \beta ,{\delta }_k,{\Sigma }_k,{\gamma }_k^R,\tau ,{\nu }_k,X_{ik},W_{ik}\right)}, \end{gathered}$$

where $f^M\left(y_{ik},z_{ik}\mid \beta ,{\delta }_k,{\Sigma }_k,{\gamma }_k^R,\tau ,{\nu }_k,X_{ik},W_{ik}\right)={\int }_{z_{ik}\in R_{+}^J}f\left(y_{ik},z_{ik}\mid \beta ,{\delta }_k,{\Sigma }_k,{\gamma }_k^R,\tau ,{\nu }_k,X_{ik},W_{ik}\right)d{z}_{ik}$. Let u_ik = log(z_ih) for i = 1,…, n_k, k = 1,…, K, then z_ik = exp(u_ik). We note that the log and exp functions applied to a vector means that the operations are applied to every element of the vector. Then, we have

$$h_{ik,opt}\left(z_{ik}\right)=\frac{f_{y_{ik},u_{ik}}^A}{{\int }_{u_{ik}\in R^J}{f}_{y_{ik},u_{ik}}^A\times \prod _{j=1}^J\text{exp}\left(u_{ijk}\right)d{u}_{ik}},$$

where $f_{y_{ik},u_{ik}}^A=f(y_{ik},\text{exp}\left(u_{ik}\right)\mid \beta ,{\delta }_k,{\Sigma }_k,{\gamma }_k^R,\tau ,{\nu }_k,X_{ik},W_{ik})$ and

$$h_{ik,opt}\left(z_{ik}\right)\prod _{j=1}^J\text{exp}\left(u_{ijk}\right)=\frac{f_{y_{ik},u_{ik}}^A\times \prod _{j=1}^J\text{exp}\left(u_{ijk}\right)}{{\int }_{u_{ik}\in R^J}{f}_{y_{ik},u_{ik}}^A\times \prod _{j=1}^J\text{exp}\left(u_{ijk}\right)d{u}_{ik}}.$$ (A.4)

The right hand side of (A.4) can be approximated by a multivariate normal density which is constructed via the Laplace approximation to the joint density $f(y_{ik}\text{exp}\left(u_{ik}\right)\mid \beta ,{\delta }_k,{\Sigma }_k,{\gamma }_k^R,\tau ,{\nu }_k,X_{ik},W_{ik}){\prod }_{j=1}^J\text{exp}\left(u_{ijk}\right)$. Denote $g\left(u_{ik}\right)=\text{log}\left\{f(y_{ik},\text{exp}\left(u_{ik}\right)\mid \beta ,{\delta }_k,{\Sigma }_k,{\gamma }_k^R,\tau ,{\nu }_k,X_{ik},W_{ik}){\prod }_{j=1}^J\text{exp}\left(u_{ijk}\right)\right\}$. Letting $u_{ik}^0$ be the stationary point where ∇g(u_ik) = 0, and $B=-{{\nabla }^{2}g\left(u_{ik}\right)|}_{u=u_{ik}^0}$, the function h_ik,opt (z_ik) ${\prod }_{j=1}^J\text{exp}\left(u_{ijk}\right)$ can be approximated by $\varphi \left(\text{log}\left(z_{ik}\right)\mid u_{ik}^0,B^{-1}\right)$, where $\varphi \left(\text{log}\left(z_{ik}\right)\mid u_{ik}^0,B^{-1}\right)$ denotes the density function of $N(u_{ik}\mid u_{ik}^0,B^{-1})$. Thus, h_ik,opt (z_ik) is approximated by the function $\frac{\varphi \left(\text{log}\left(z_{ik}\right)\mid u_{ik}^0,B^{-1}\right)}{{\prod }_{j=1}^J{z}_{ijk}}$. From equation (4.2), we have

$$g\left(u_{ik}\right)=\text{log}\left\{f_{y_{ik},u_{ik}}^A\times \prod _{j=1}^J\text{exp}\left(u_{ijk}\right)\right\}\propto -\frac{{\nu }_k+J}{2}\text{log}\left(A_{ik}\left(u_{ik}\right)+{\nu }_k\right)-(J+\tau +1)\text{log}\left(\sum _{j=1}^J\text{exp}\left(u_{ijk}\right)+\tau \right)+\sum _{j=1}^J{u}_{ijk},$$ (A.5)

where $A_{ik}\left(u_{ik}\right)=(y_{ik}-X_{ik}\beta -W_{ik}{\gamma }_k^R-{\Delta }_k(\text{exp}\left(u_{ik}\right){-E{z}_{ik}))}^{\prime }{\Sigma }_k^{-1}(y_{ik}-X_{ik}\beta -W_{ik}{\gamma }_k^R-{\Delta }_k(\text{exp}\left(u_{ik}\right)-E{z}_{ik}))$. Using g(u_ik) in (A.5), we obtain ∇g(u_ik) and ∇²g(u_ik) after some algebra.