Nonparametric Bayes testing of changes in a response distribution with an ordinal predictor

Summary

In certain biomedical studies, one may anticipate changes in the shape of a response distribution across the levels of an ordinal predictor. For instance, in toxicology studies, skewness and modality might change as dose increases. To address this issue, we propose a Bayesian nonparametric method for testing for distribution changes across an ordinal predictor. Using a dynamic mixture of Dirichlet processes, we allow the response distribution to change flexibly at each level of the predictor. In addition, by assigning mixture priors to the hyperparameters, we can obtain posterior probabilities of no effect of the predictor and identify the lowest dose level for which there is an appreciable change in distribution. The method also provides a natural framework for performing tests across multiple outcomes. We apply our method to data from a genotoxicity experiment.

1. Introduction

In many biomedical studies, there is interest in evaluating changes in a response distribution across an ordinal predictor. For example, toxicologists are often interested in assessing whether several biological responses vary in distribution with dose. In such settings, normality assumptions are typically not justified and there are biological reasons to expect changes in not only the location but also the variance and shape with dose. In particular, when there is substantial heterogeneity amongst subjects in their biological response to treatment due to variation in gene expression, timing of the cell cycle, and other factors, changes in variance, skewness, and even modality are natural.

Several frequentist methods have been proposed for testing for trends in non-Gaussian responses. Univariate approaches, such as Jonckheere's test (Terpstra, 1952; Jonckheere, 1954), are used most frequently, though other authors have proposed related methods for multivariate responses (e.g., Dietz and Killeen, 1981; Dietz, 1989; O'Brien, 1984; Huang et al., 2005). Unfortunately, these methods are most sensitive to changes in the location of a distribution and may ignore important trends in shape. In contrast, k-sample tests based on empirical distribution functions (e.g, Kiefer, 1959; Ahmad, 1976) are sensitive to changes in distribution shape, but not changes in location and scale.

Bayesian nonparametric approaches have several advantages over frequentist alternatives, including the ability to incorporate prior information (e.g., from historical controls) and exact inferences on unknown distributions. Although there is an extensive literature on estimation (for a review see Müller and Quintana, 2004) and a growing literature on goodness of fit testing (e.g., Berger and Guglielmi, 2001; Carota and Parmigiani, 1996; Verdinelli and Wasserman, 1998), there is a paucity of methods for nonparametric testing across multiple groups. Gopalan and Berry (1998) use a nonparametric Dirichlet process prior (DPP; Ferguson, 1973, 1974) to perform multiple comparisons on distribution means. Recently, Basu and Chib (2003) proposed a less restrictive method which compares semiparametric models using Bayes factors. This method may be promising when there are only 2−3 levels of an ordinal predictor. However, in toxicology studies with several dose groups, this approach would be unwieldy when one considers each possible contrast between dose groups.

In analyzing data from multiple groups, one would typically be interested not only in testing but also in estimation of group-specific distributions. Such estimation can potentially be accomplished by fitting a DP mixture (DPM; Antoniak, 1974) model separately to the different dose groups. For some references on applications of DPMs, refer to West et al. (1994), Escobar and West (1995), Bush and MacEachern (1996), Kleinman and Ibrahim (1998), and Mukhopadhyay and Gelfand (1997). The disadvantage of such an approach is its inability to model trends and borrow information across the different dose groups, which is particularly important in applications having a modest number of subjects per group. Note that applications of the Basu and Chib (2003) procedure to test differences between groups would also have this disadvantage.

An alternative to using separate DPMs is to consider the dose group-specific distributions as a collection of dependent unknown distributions, which are assigned a dependent nonparametric prior. Dependent nonparametric priors have been the focus of a growing body of literature. In the early work of Cifarelli and Regazzini (1978), dependence was induced through a regression model within the parametric base measure of the DP (for related work see Mira and Petrone, 1996; Tomlinson and Escobar, 1999; Carota and Parmigiani, 2002; Giudici et al., 2003). Although the method is straightforward, it has limited flexibility. Other authors proposed inducing nonparametric dependence by parameterizing random measures as products of DP distributed factors, though within a different framework than ours (see, e.g., Gelfand and Kottas, 2001; Pennell and Dunson, 2006). MacEachern (1999; 2000) proposed a dependent Dirichlet process which characterizes dependency by defining a stochastic process for the atoms in the Sethuraman (1994) stick-breaking characterization of the DP. This approach has been successfully applied in ANOVA (De Iorio et al., 2004) and spatial (Gelfand et al., 2004) applications. A limitation of these applications is the assumption of fixed weights on the atoms, which does not allow new features to appear as dose increases. To solve this problem, Dunson (2006) proposed a dynamic mixture of DPs (DMDP), which is related to a mixture structure originally proposed by Müller et al. (2004) to combine inferences across related nonparametric models. More flexible dependent nonparametric priors for continuous predictors have been proposed by Griffin and Steel (2006), Dunson et al. (2007), and Duan et al. (2005).

Motivated by tractability in addressing the problem of nonparametric Bayes testing of changes with dose, we focus here on generalizing the Dunson (2006) approach. Our goal is to obtain posterior probabilities for local and global null hypotheses corresponding to equivalence in an unknown distribution between groups. Instead of requiring exact equivalence, we treat the distributions in adjacent groups as effectively equivalent if the total variation norm between their probability measures is less than ∊. We use a hierarchical modification of the DMDP to borrow information across groups while assigning probabilities to the local null and alternative hypotheses. Using an MCMC implementation, we obtain posterior model probabilities for the local and global hypotheses from a single run, while also producing posterior distributions for thresholds and estimates of group-specific distributions. Multiple response data can be accommodated without complication.

In Section 2, we describe our generalization of the DMDP and hyperprior structure. In Section 3, we outline an MCMC methodology for testing and estimation. In Section 4, we evaluate our approach using simulated data. In Section 5 we apply our method to data from a genotoxicity experiment. In Section 6 we discuss the results and future work.

2. Nonparametric Model and Prior Structure

2.1 General Framework

Let y_hi be a vector of q continuous outcomes measured on subject i (i = 1, . . . , n_h) in group h (h = 1, . . . , d) and X_h be a score for group h, where X_h−1 < X_h < X_h+1. In a toxicology study, X_h is the dose administered to each animal in group h with X₁ = 0 representing the control group. To relax distribution assumptions, we assume that y_hi ∼ F_h which has the density function

$$f_h\left({\mathbf{y}}_{hi}\right)=\int \mathrm{N}({\mathbf{y}}_{hi};{\mu }_{hi},{\Sigma }_{hi})d{G}_h({\mu }_{hi},{\Sigma }_{hi}),$$ (1)

where G_h is unknown. Hence, the density of y in group h is an infinite mixture of multivariate Gaussian densities, with the mixture distribution varying across groups. As the number of components increases without bound, a mixture of Gaussians converges uniformly to any smooth density function (see Sorenson and Alspach, 1971).

It is clear that differences in the mixture distributions, G_h and G_h+1, imply differences in the outcome distributions, F_h and F_h+1. Hence, we focus on testing for changes in the mixture distributions. In particular, our focus is on local null hypotheses characterizing differences in adjacent groups and on global hypotheses representing intersections of these local hypotheses. As in the Kolmogorov-Smirnov test, we could specify a point null hypothesis which requires G₁, . . . , G_d to match. However, many have argued that point nulls are artificial and are rarely observed in practice (see, e.g., Berger and Delampady, 1987; Nickerson, 2000). In this paper, we use a more realistic null hypothesis under which the group-specific distributions effectively match. We formalize this hypothesis below.

Let G₁, . . . , G_d denote probability measures corresponding to the mixture distributions for the different groups. In addition, let B denote any Borel set such that $B\in \mathcal{B}$, with $B\subset {\Re }^q$ and $\mathcal{B}$ the Borel sigma-algebra of subsets of ${\Re }^q$, and let the total variation norm of G_h+1 − G_h be defined as

$${\Vert G_{h+1}-G_h\Vert }_{TV}=\underset{B\in \mathcal{B}}{max}\mid G_{h+1}\left(B\right)-G_h\left(B\right)\mid .$$

When ||G_h+1 − G_h||_TV = 0 we say that G_h+1 and G_h match. However, we are interested in local nulls which only require G_h+1 and G_h to effectively match which we specify as the following:

$$H_{0h}:{\Vert G_{h+1}-G_h\Vert }_{TV}\le ∊$$ (2)

for h = 1, . . . , d − 1, where ∊ is some small constant such that when H_0h holds, there is no appreciable difference in the mixture distributions across groups h and h + 1. The global null of no changes in the response distribution across groups (H₀) corresponds to the intersection of these local nulls. Note that in using the total variation norm to define our hypothesis tests, we treat differences across all sets as being potentially important, in terms of the application. This approach avoids the need to specify a response range which is of particular interest. Such a range may be diffcult to choose objectively, particularly in toxicology studies as it is typically not known how to exactly equate a response value in an animal to a corresponding value in humans.

2.2 DMDP Model

In addition to conducting hypothesis tests, we would like to estimate the density in each dose group. As mentioned in Section 1, a promising approach would be to assign a dependent nonparametric process to G₁, . . . , G_d thereby allowing us to borrow information across groups. Using the method of Müller et al. (2004), we could let $G_h=\pi G_0+(1-\pi )G_h^{\ast }$ for h = 1, . . . , d, where G₀ is a global probability measure and $G_h^{\ast }$ is an innovation measure specific to group h, with $G_0,G_1^{\ast },\dots ,G_d^{\ast }$ assigned independent nonparametric priors. However, this approach results in an over-specified model in that d+1 random measures are incorporated to characterize d unknown distributions. Teh et al. (2006) proposed an alternative model which assumes that the G_h are drawn from DPPs with a common DP-distributed base measure G₀. However, these hierarchical priors treat the groups as exchangeable. To incorporate information on ordering in the groups, Dunson (2006) proposed a dynamic mixture of Dirichlet processes (DMDP), which is conceptually related to the Müller et al. (2004) approach, but avoids the over-specification problem.

We first consider the case of two groups (d = 2). To induce dependency, we model the mixture distributions using the following DMDP:

$$\begin{array}{c}\hfill G_1\sim \mathrm{DP}\left({\alpha }_0{G}_0\right)\hfill \\ \hfill G_2=(1-{\pi }_1)G_1+{\pi }_1{G}_1^{\ast }{G}_1^{\ast }\sim \mathrm{DP}\left({\alpha }_1{G}_{01}\right),\hfill \end{array}$$ (3)

where $0\le {\pi }_1\le 1,G_1^{\ast }$ is an innovation distribution that characterizes changes in the mixture distribution of y caused by increasing the group score from X₁ (e.g., dose = 0) to X₂ (e.g., dose > 0), and G₀₁ is a known distribution. Note that the DMDP implies the following densities for y in groups 1 and 2:

$$\begin{array}{cc}\hfill f_1\left(\mathbf{y}\right)& =\int \mathrm{N}(\mathbf{y};\mu ,\Sigma )d{G}_1(\mu ,\Sigma )\hfill \\ \hfill f_2\left(\mathbf{y}\right)& =(1-{\pi }_1)f_1\left(\mathbf{y}\right)+{\pi }_1\int \mathrm{N}(\mathbf{y};\mu ,\Sigma )d{G}_1^{\ast }(\mu ,\Sigma ).\hfill \end{array}$$ (4)

Hence, the distribution of y in group 2 is a mixture of the distribution in group 1 and a different DP mixture of normals characterized by mixture distribution $G_1^{\ast }$. This is a natural model for toxicology data since we would expect that the distribution in the treatment group, F₂, shares features with the distribution in the control group, F₁, but that innovations may have occurred due to stress induced by the test chemical. The amount of borrowing of information from the control control group and the level of innovation are represented by the weights (1 − π₁) and π₁ respectively.

In the above model, it is convenient to choose G₀ ≡ G₀₁, with these base measures having a conjugate normal-inverse Wishart form: $G_0(\mu ,\Sigma )=\mathrm{N}(\mu ;{\mu }_0,\kappa \Sigma )\cdot \mathrm{W}({\Sigma }^{-1};v_0,v_0^{-1}{\mathbf{V}}_0^{-1})$, where $\mathrm{W}(\cdot ;v_0,v_0^{-1}{\mathbf{V}}_0^{-1})$, denotes the Wishart density with degrees of freedom v₀ and mean ${\mathbf{V}}_0^{-1}$. In this case, our nonparametric prior for F₁ is centered on a multivariate t-distribution, while the prior for F₂ is centered on a mixture of multivariate t-distributions. As α₀ and α₁ increase, we place more weight on this parametric base model for the unknown densities. By choosing hyperprior densities for α₀ and α₁ (as we describe in Section 2.4), the method can adapt flexibly to accommodate lack of fit in the multivariate t components.

Extending our model to d groups, for h = 2, . . . , d we let

$$\begin{array}{cc}\hfill G_h& =(1-{\pi }_{h-1})G_{h-1}+{\pi }_{h-1}{G}_{h-1}^{\ast }\hfill \\ \hfill & =\left\{\prod _{l=1}^{h-1}(1-{\pi }_l)\right\}{G}_1+\sum _{l=1}^{h-2}\left\{\prod _{m=l+1}^{h-1}(1-{\pi }_m)\right\}{\pi }_l{G}_l^{\ast }+{\pi }_{h-1}{G}_{h-1}^{\ast }\hfill \\ \hfill & ={\omega }_{h1}{G}_1+\sum _{l=1}^{h-1}{\omega }_{h,l+1}{G}_l^{\ast }\hfill \\ \hfill G_l^{\ast }& \sim DP\left({\alpha }_l{G}_0\right),\text{for}l=1\dots ,h-1,\hfill \end{array}$$ (5)

where ${\omega }_{hl}={\pi }_{l-1}{\prod }_{m=1}^{h-1}(1-{\pi }_m)$ for l = 1, . . . , h − 1 and ω_hl = π_h−1 for l = h with π₀ = 1 and ω_h = (ω_h1, . . . , ω_hh)^′ are group-specific probability weights on the different components in the mixture. Note that the above DMDP is not exchangeable on (G₁, . . . , G_d). As we move from predictor level h to h + 1 we decrease the weight assigned to $G_1,G_1^{\ast },\dots ,G_{h-1}^{\ast }$ and introduce a new unknown distribution to our mixture, $G_h^{\ast }$. As mentioned previously, this is a natural model for toxicology data; with every increase in dose level, h to h+1, individuals are subjected to higher levels of stress resulting in changes in the response distribution characterized by $G_h^{\ast }$. Hence, we would expect the response distributions at adjacent doses to be more similar than dose groups that are several levels apart, as is the case in our model.

Another important feature of our model is that it imposes a type of order restriction across groups. For instance, if the mixture distribution changes when we increase the predictor from the group h to the group h + 1 level, all later groups (h + 2, . . . , d) will also differ from group h. This is due to the following theorem:

THEOREM 1

For any G_h, G_h+1, and G_h+j+1 distributed according to (5) (h = 1, . . . , d − 2; j = 1, . . . , d − h − 1)

$$\mathrm{Pr}\left({\Vert G_{h+j+1}-G_h\Vert }_{TV}\le ∊\mid {\Vert G_{h+1}-G_h\Vert }_{TV}>∊\right)=0.$$

A proof of this theorem is provided in Web Appendix A. While it is possible, conceptually, for the distribution in group h + j + 1 to shift back toward the distribution in group h, it is unlikely in any application for groups h and h + j + 1 to effectively match when groups h and h + 1 differ. For instance, in a toxicology experiment it is possible for a feedback mechanism to activate at a dose greater than h + 1, causing G_h+j+1 to resemble G_h. However, the feedback should not negate the effect of increasing dose from the group h to the group h + 1 level. Biological plausibility motivated our formulation of the model hypotheses in this order-dependent manner.

2.3 Model Space Prior

In the two sample case of expression (3),

$$\begin{array}{cc}\hfill H_0:{\Vert G_2-G_1\Vert }_{TV}& ={\max }_{B\in \mathcal{B}}\mid G_2\left(B\right)-G_1\left(B\right)\mid \hfill \\ \hfill & ={\max }_{B\in \mathcal{B}}\mid (1-{\pi }_1)G_1\left(B\right)+{\pi }_1{G}_1^{\ast }\left(B\right)-G_1\left(B\right)\mid \hfill \\ \hfill & ={\pi }_1{\Vert G_1^{\ast }-G_1\Vert }_{TV}\le ∊.\hfill \end{array}$$ (6)

Note that H₀ holds for any fixed ∊ if: 1.) π₁ ≤ ∊, regardless of G₁ and $G_1^{\ast }$ or 2.) ${\Vert G_1^{\ast }-G_1\Vert }_{TV}\le ∊$, regardless of π₁. Hence, the value of π₁ and the independent DPPs on G₁ and $G_1^{\ast }$ control the prior probability allocated to H₀. If probability close to zero is allocated to H₀, then we may expect the posterior probability of H₀ to be small unless the sample size is large, clearly an unappealing property in most applications. Thus, we want to be able to control the prior probability allocated to H₀.

We first consider the possibility of controlling Pr(H₀) through the priors for G₁ and $G_1^{\ast }$. The DPPs G₁ ∼ DP(α₀G₀) and $G_1^{\ast }\sim \mathrm{DP}\left({\alpha }_1{G}_0\right)$ imply that

$${\Vert G_1-G_1^{\ast }\Vert }_{TV}=\underset{B\in \mathcal{B}}{\max }\mid \sum _{h=1}^{\infty }{V}_{1h}\prod _{l<h}(1-V_{1l}){\delta }_{{\theta }_{1h}}\left(B\right)-\sum _{h=1}^{\infty }{V}_{2h}\prod _{l<h}(1-V_{2l}){\delta }_{{\theta }_{2h}}\left(B\right)\mid ,$$

where θ_1h and θ_2h ∼ G₀, V_1l ∼ Be(1, α₀), V_2l ∼ Be(1, α₁), and δ_θ denotes a degenerate distribution with mass at θ (Sethuraman, 1994). Suppose that we choose the Borel set B to contain (θ_1h, h = 1, . . . ,∞) but not (θ_2h, h = 1, . . . , ∞). Then, for a set B satisfying these conditions, ${\delta }_{{\theta }_{1h}}\left(B\right)=1\text{for}h=1,\dots ,\infty$ and ${\delta }_{{\theta }_{2h}}\left(B\right)=0\text{for}h=1,\dots ,\infty$. Hence, the above absolute value reduces to

$$\mid \sum _{h=1}^{\infty }{V}_{1h}\prod _{l<h}(1-V_{1l})\mid =1.$$

The theorem below follows directly:

THEOREM 2

Let G₀ be a continuous base measure. For any G₁ ∼ DP(α₀G₀) independent of $G_1^{\ast }\sim DP\left({\alpha }_1{G}_0\right)$

$${\Vert G_1-G_1^{\ast }\Vert }_{TV}=1.$$

From Theorem 2 we have $\mathrm{Pr}({\Vert G_1^{\ast }-G_1\Vert }_{TV}<∊)=0$. Thus zero probability is allocated to H₀ by the total variation component, which implies that Pr(H₀) is entirely controlled by π₁. Hence, one can effectively replace H₀ in (6) with π₁ ≤ ∊* , where ∊* is also close to zero. Extending this rationale to the d-group setting, we let

$$H_{0h}:{\pi }_h\le {∊}^{\ast }.$$ (7)

To control the prior probability allocated to H_0h, we propose a prior for π_h which is a mixture of its distribution under H_0h and H_1h : π_h > ∊* . Let ζ_h be a latent variable that equals 1 when the local null H_0h is true and 0 otherwise and let Pr(ζ_h = 1) = p_0h. Given ζ_h, our prior for π_h has the following form:

$${\pi }_h\mid {\zeta }_h\sim {\zeta }_h\cdot \mathrm{Unif}(0,{∊}^{\ast })+(1-{\zeta }_h)\cdot \mathrm{Unif}({∊}^{\ast },1),h=1,\dots ,d-1.$$ (8)

In the above mixture, p₀ = (p₀₁, . . . , p_0h)^′ is chosen to reflect prior knowledge about the chances of H₀ being true. For example, to assign equal prior weight to H₀ and H₁, p₀ should be chosen to satisfy $\mathrm{Pr}\left(H_0\right)={\prod }_{h=1}^{d-1}{p}_{0h}=0.5$. Under the Bayesian Bonferroni method of Westfall et al. (1997), this may be achieved by setting p₀₁ = · · · = p_0,d−1 = 0.5^1/(d−1). However for large d, the probability of each local null hypothesis is nearly 1, which can make this approach overly conservative. We instead use a model space prior described by Hans and Dunson (2005). This approach induces dependency in the local null hypotheses by assuming p_0h = p₀ for h = 1, . . . , d − 1 where p₀ ∼ Be(a_p, b_p). As a consequence of this prior, $\mathrm{Pr}(H_{0h}\mid \{{\zeta }_j:j\ne h\})=(a_p+{\sum }_{j\ne h}{\zeta }_j)∕(a_p+b_p+d-2)$. Hence when several H_0j(j ≠ h) are true, we shrink toward no change between groups h and h+1, which is conceptually appealing in dose-response studies.

As in Hans and Dunson (2005), a_p and b_p are chosen so that on average Pr(H₀)=0.5, i.e., $\mathrm{E}\left(p_0^{d-1}\right)$ = 0.5. This implies that a_p and b_p must satisfy the following:

$$\begin{array}{cc}\hfill 0.5& ={\int }_0^1{p}_0^{d-1}\frac{\Gamma (a_p+b_p)}{\Gamma \left(a_p\right)\Gamma \left(b_p\right)}{p}_0^{a_p-1}{(1-p_0)}^{b_p-1}d{p}_0\hfill \\ \hfill & =\frac{\Gamma (a_p+b_p)\Gamma (a_p+d-1)}{\Gamma \left(a_p\right)\Gamma (a_p+b_p+d-1)}.\hfill \end{array}$$ (9)

We then impose the constraint a_p+b_p = 1 to represent unit information in the prior and solve (9) numerically for a_p and b_p.

2.4 Hyperpriors for DP Parameters

To decrease the sensitivity of our model to subjectively chosen hyperparameters, we specify the following hyperpriors:

$$\begin{array}{cc}\hfill & {\kappa }^{-1}\sim \mathrm{Ga}(a_{\kappa },b_{\kappa }){\mu }_0\sim \mathrm{N}(\beta ,\Omega )\hfill \\ \hfill {\alpha }_h\sim & \mathrm{Ga}(a_{\gamma h},b_{\gamma h})\text{for}h=0,1,\dots ,d-1.\hfill \end{array}$$

In practice, one may favor priors which assign high prior probability to low values of α_h since this implies that the number of mixture components is small (Antoniak, 1974) and a very flexible density estimator can be produced with few mixture components. Hence, a reasonable choice is a_γh = a_γ = 1 and b_γh = b_γ = 1. We also recommend following Escobar and West (1995) by choosing diffuse priors for κ⁻¹ and μ₀, with a_κ/b_κ small and, when possible, β chosen to reflect the mean of y in previous studies. Potentially, hyperpriors could also be assigned to V₀ and v₀, though we choose to fix these parameters to avoid over-parameterization. We recommending choosing V₀ so that a N(β, V₀) distribution sufficiently covers an expected range of values of y and choosing v₀ to reflect one's confidence in this choice. When there is little prior information on V₀, we strongly recommend performing a sensitivity analysis on v₀ as small values can often result in convergence or numerical problems, particularly in the case of a multivariate response.

3. Posterior Computation

3.1 Gibbs Sampling Methodology

We sample from the joint posterior of our model parameters using a Gibbs sampling methodology which is related to the Bush and MacEachern (1996) and West et al. (1994) approaches for the Dirichlet process mixture. Details may be found in Web Appendix B.

3.2 Hypothesis Testing

The global null may be formally evaluated using Rao-Blackwellization (Gelfand and Smith, 1990). For T iterations of the MCMC with a burn-in of T_b iterations, we compute

$$\begin{array}{cc}\hfill \widehat{\mathrm{Pr}}(H_0\mid \text{Data})& =\frac{1}{T-T_b}\sum _{t=T_b+1}^T\mathrm{Pr}({\zeta }_1=\cdots ={\zeta }_{d-1}=1\mid p_0^{\left(t\right)},{\mathbf{M}}^{\left(t\right)})\hfill \\ \hfill & =\frac{1}{T-T_b}\sum _{t=T_b+1}^T\prod _{h=1}^{d-1}{p}_{0h}^{\ast \left(t\right)},\hfill \end{array}$$ (10)

where the superscript (t) denotes a value sampled at iteration t, ${\mathbf{M}}^{\left(t\right)}=\{M_{hi}^{\left(t\right)};h=1,\dots ,d;i=1,\dots ,n_h\},M_{hi}\in \{1,\dots ,h\}$ is the mixture indicator for subject i in group h, and $p_{0h}^{\ast \left(t\right)}=\mathrm{Pr}({\zeta }_h=1\mid p_0^{\left(t\right)},{\mathbf{M}}^{\left(t\right)})$. The explicit form for $p_{0h}^{\ast }$ is given in Web Appendix B.

A common approach is to reject H₀ if (10) is less than some small pre-determined value (e.g., 0.05). However, it is appealing in a Bayesian analysis to report the posterior null hypothesis probability, or alternatively the Bayes factor, as a weight of evidence, avoiding the choice of an arbitrary cuto for determining significance. Hence, we also recommend reporting that there is evidence against H₀ when the posterior probabilities are only moderately small (i.e., between 0.05 and 0.1).

An attractive feature of our methodology is that we can test local hypotheses within the same Markov chain used to test H₀. Since we properly calibrate our prior for p₀ to give equal prior probability to H₀ and H₁, the posterior probabilities of the local nulls do not need to be adjusted (see Westfall et al., 1997). For instance, the posterior probability of a distribution change between groups h and h + 1 would be the average value of $p_{0h}^{\ast }$ over T − T_b iterations. In toxicology studies, there is also interest in comparing each dose group to control (group 1). The null hypothesis of no difference in the distributions of groups 1 and h is

$$H_{0,h-1}^{\ast }:{\pi }_j\le {∊}^{\ast }j=1,\dots ,h-1$$ (11)

and we estimate its posterior probability as

$$\widehat{\mathrm{Pr}}(H_{0,h-1}^{\ast }\mid \text{Data})=\frac{1}{T-T_b}\sum _{t=T_b+1}^T\prod _{j=1}^{h-1}{p}_{0j}^{\ast \left(t\right)}.$$ (12)

3.3 Density Estimation

The predictive density of a future ${\varphi }_{h,n_h+1}=\{{\mu }_{h,n_h+1},{\Sigma }_{h,n_h+1}\}$ can be obtained at each iteration of the MCMC:

$$\pi \left({\varphi }_{h,n_h+1}\right)=\sum _{l=1}^h\left\{\frac{{\alpha }_{l-1}}{{\alpha }_{l-1}+m_l}\cdot d{G}_0\left({\varphi }_{h,n_h+1}\right)+\sum _{r=1}^{k_l}\left(\frac{m_{lr}}{{\alpha }_{l-1}+m_l}\right){\delta }_{{\theta }_{lr}}\right\},$$ (13)

where, as explained in Web Appendix B, m_l and k_l are the number of subjects and the number of unique means and covariances, respectively, in mixture component l, ${\theta }_{lr}=\{{\mu }_{lr}^{\ast },{\Sigma }_{lr}^{\ast }\}$, and m_lr is the number of subjects whose mean and covariance equals θ_lr. This implies that the predictive density of ${\mathbf{y}}_{h,n_h+1}$ is

$$f_{h,n_h+1}\left(\mathbf{y}\right)={\omega }_0\cdot \mathrm{t}\left(\mathbf{y};v_0-q+1,{\mu }_0,\frac{v_0(1+\kappa )}{v_0-q+1}{\mathbf{V}}_0\right)+\sum _{l=1}^h\sum _{r=1}^{k_l}{\omega }_{hlr}\cdot \mathrm{N}(\mathbf{y};{\mu }_{lr}^{\ast },{\Sigma }_{lr}^{\ast }),$$ (14)

where the multivariate t-density results from integrating $\mathrm{N}(\cdot ;{\mu }_{h,n_h+1},{\Sigma }_{h,n_h+1})$ over G₀ and ω₀ and ω_hlr are the respective multipliers on $d{G}_0\left({\varphi }_{h,n_h+1}\right)$ and ${\delta }_{{\theta }_{lr}}$ in (13). Following T iterations, the Rao-Blackwellized estimate,

$${\stackrel{‒}{f}}_{h,n_h+1}\left(\mathbf{y}\right)=\frac{1}{T-T_b}\sum _{t=T_b+1}^T{f}_{h,n_h+1}^{\left(t\right)}\left(\mathbf{y}\right),$$ (15)

may be computed over a grid of values.

In a toxicology study, one would also be interested in the posterior density of the lowest observed adverse e ects level (LOAEL). Assuming that any change in the distribution relative to group 1 is an adverse effect, we can estimate the posterior density of the LOAEL as follows:

$$\begin{array}{cc}\hfill \widehat{\mathrm{Pr}}(\mathrm{LOAEL}=h\mid \text{Data})& =\frac{1}{T-T_b}\sum _{t=T_b+1}^T(1-p_{0,h-1}^{\ast \left(t\right)})\prod _{j=1}^{h-2}{p}_{0j}^{\ast \left(t\right)}h=2,\dots ,d\hfill \\ \hfill \widehat{\mathrm{Pr}}(\mathrm{LOAEL}=d+1\mid \text{Data})& =\frac{1}{T-T_b}\sum _{t=T_b+1}^T\prod _{j=1}^{d-1}{p}_{0j}^{\ast \left(t\right)},\hfill \end{array}$$ (16)

where a LOAEL of d+1 implies that the LOAEL is greater than the largest dose considered in the study. Hans and Dunson (2005) use a similar approach to estimate the posterior density of the LOAEL under umbrella orderings in parametric models.

4. Simulation Studies

4.1 Description of Data

We performed three simulation studies to evaluate the performance of our methodology. In each simulation case, we generated a vector of three responses for each subject, y_hi = (y_hi1, y_hi2, y_hi3)^′, from a mixture of five multivariate normal distributions:

$$f_h\left({\mathbf{y}}_{hi}\right)=\sum _{j=1}^5{w}_{hj}\mathrm{N}({\mathbf{y}}_{hi};{\mu }_j,{\tau }_j\cdot {\mathbf{I}}_3),$$

where μ_j = (μ_j1, μ_j2, μ_j3)^′ and w_hj ∝ exp(γ_j0 + (h − 1) γ_j1) for h = 1, . . . , 6 and i = 1, . . . , 30. For each mixture component, μ_jk = x_j + k − 1 where x_j is a constant.

The parameter values used in the simulations are provided in Table 1. Cases 1 and 2 differed only in their values of the mixture probabilities; in Case 1 we let γ_j0 = 1 and γ_j1 = 0 for j = 1 . . . , 5 to simulate under a null model, while in Case 2 the mixture probabilities varied with predictor level. As seen in Figure 1, the right tails of the marginal densities of y₁ become heavier with predictor level in Case 2 while, in Case 3, the modality of the distribution changes across h. The latter case may arise in toxicology experiments in which gene×dose interactions cause sub-populations to respond differently to treatment. The marginal densities of y₂ and y₃ follow a similar trend with dose, though their location parameters differ. Five data sets were simulated under each case.

Table 1.

Parameter values for the mixture components in simulation cases 1−3.

	Case 1
	j
Parameter	1	2	3	4	5
xj	−0.6	−0.5	−0.4	0.5	1
τj	0.16	0.25	0.36	0.903	1.323
γj0	1	1	1	1	1
γj1	0	0	0	0	0
	Case 2
xj	−0.6	−0.5	−0.4	0.5	1
τj	0.16	0.25	0.36	0.903	1.323
γj0	1	1	1	−3	−3
γj1	−1	−0.25	−0.25	0.25	0.75
	Case 3
xj	−2	−1	0	1	2
τj	0.16	0.25	0.36	0.903	1.323
γj0	−3	−3	1	−3	−3
γj1	1	0.5	−1	0.5	1

Figure 1.

Marginal density of y₁ at each predictor level in simulation cases 1−3.

4.2 Univariate Analyses

We first performed a univariate analysis on y₁ in order to compare our methodology to standard frequentist methods. In each case, we assumed μ₀ ∼ N(0, 100), κ⁻¹ ∼ (0.5, 50), and α_h ∼ Ga(1, 1) for h = 1, . . . , d a priori and let v₀ = 10 and V₀ = 1. In our mixture priors for π₁, . . . , π₅, we let ∊* = 0.05 and p₀ ∼ Be(0.725, 0.275) to assign equal prior probability to H₀ and H₁. We ran our MCMC for a total 55,000 iterations, discarding the first 5,000 as a burn-in, and saved every 10th iteration to thin the chain. For the purpose of comparing our method to the frequentist approaches, we decided to reject $H_0\text{if}\widehat{\mathrm{Pr}}(H_0\mid \text{Data})\le 0.05$.

Our method correctly assigned high probability to H₀ in Case 1 (> 0.914 for each data set) and very low posterior probability to H₀ in Cases 2 and 3 (as seen in Table 2). These results were consistent with the p-values from a k-sample Kolmogorov-Smirnov test (Kiefer, 1959), while both Jonckheere's test and the Kruskal-Wallis test failed to reject the null in 5/5 Case 3 simulations. Since stochastic ordering was violated in Case 3, it is not surprising that Jonckheere's test was insignificant. Being sensitive to location but not shape changes, the Kruskal-Wallis test could detect distribution changes in case 2 but not 3, while our approach was sensitive to changes in each case.

Table 2.

Posterior probabilities of global and local null hypotheses for the univariate analysis of y₁ in simulation cases 1−3. The letters in superscript indicate that the following frequentist tests were significant at the 0.05 level: J=Jonckheere, KW=Kruskal-Wallis, KS=Kolmogorov-Smirnov, S=Shirley, D=Dunn. Since Shirley's test assumes a monotone trend in location, it was not performed when Jonckheere's test was insignificant.

Case 1
Data set	H0	$H_{01}^{\ast }$	$H_{02}^{\ast }$	$H_{03}^{\ast }$	$H_{04}^{\ast }$	$H_{05}^{\ast }$
1	0.914	0.987	0.927	0.925	0.918	0.914
2	0.977	0.998	0.996	0.989	0.986	0.977
3	0.979	0.998	0.995	0.990	0.988	0.979
4	0.971	0.988	0.980	0.978	0.976	0.971
5	0.971	0.987	0.983	0.981	0.978	0.971
Case 2
1	0.066	0.987	0.980	0.952	0.080	0.066
2	< 0.001J,KW,KS	0.990	0.945	0.888	< 0.001S,D	< 0.001S,D,KS
3	0.001J,KW,KS	0.990	0.944	0.075	0.004S	0.001S,D,KS
4	< 0.001J,KW,KS	0.994	0.959	0.253	< 0.001S,D	< 0.001S,D
5	0.002J,KW,KS	0.963	0.914	0.843	0.202	0.002S,D,KS
Case 3
1	< 0.001KS	0.260	0.193	< 0.001KS	< 0.001KS	< 0.001KS
2	< 0.001KS	0.799	0.027	< 0.001KS	< 0.001KS	< 0.001KS
3	< 0.001KS	0.838	< 0.001KS	< 0.001	< 0.001KS	< 0.001KS
4	< 0.001KS	0.637	0.097	< 0.001	< 0.001KS	< 0.001
5	< 0.001KS	0.864	0.048	< 0.001KS	< 0.001KS	< 0.001KS

In addition to assessing evidence of any change across the groups, it is of interest to identify dose groups that differ from control. We compared our approach to methods commonly used for non-Gaussian toxicology data, Dunn's method (1964) and Williams' (1986) modification of Shirley's method (1977). We also compared our method to (d − 1) 2-sample Kolmogorov-Smirnov tests performed under a Bonferroni correction. As seen in Table 2, our method found clear evidence that groups 5 and 6 differed from group 1 in 3/5 and 4/5 Case 2 simulations, respectively. These results were consistent with Shirley's method, while Dunn's method found a significant difference between groups 1 and 5 in only 2/5 simulations and the Kolmogorov-Smirnov tests could only identify differences between groups 1 and 6. In Case 3, our method revealed that groups 4−6 were all clearly different from group 1 and found significant differences between groups 1 and 3 in 3/5 simulations. The Kolmogorov-Smirnov tests were more conservative and suggested group-related trends that were inconsistent with the simulated data; in two data sets a lower dose group was significantly different from group 1, while a higher dose group was not found to differ from baseline.

4.3 Multivariate Analyses

We next repeated our analyses using a multivariate approach. In addition to testing for a predictor effect on the distribution of y, in Cases 2 and 3 we also tested for an effect on the joint distribution of y₁, z₁, z₂ and y₁, y₂, z₁, where (z_hi1, z_hi2) ∼ N(0, I₂) for h = 1, . . . , 5 and i = 1 . . . , 30 independently of y_hi. The purpose of these latter tests was to evaluate the performance of our multivariate method under varying numbers of a ected outcomes. The implementation was very similar to that of the univariate analyses, with μ₀ ∼ N(0, 100 · I₃) a priori and V₀ = I₃.

Figure 2 summarizes the results. As in the univariate analyses, our method gave high posterior probability to H₀ in Case 1; the posterior probability was greater than 0.989 in each of the five cases. In Case 2, the posterior probabilities of $H_0,H_{04}^{\ast }$, and $H_{05}^{\ast }$ were small in each analysis. However, in most cases, the posterior probabilities of $H_{01}^{\ast },H_{02}^{\ast }\text{and}{H}_{03}^{\ast }$ increased as we decreased the number of y's (the affected outcomes) in our model. Thus our method tends to be conservative when only one outcome is moderately affected, which is a desirable feature for a multivariate method. A similar trend was seen in the posterior probabilities of $H_{01}^{\ast }$ and $H_{02}^{\ast }$ in Case 3.

Figure 2.

Posterior probabilities of global and local null hypotheses for the multivariate analyses in simulation cases 1−3. Labels for y-axes denote the different outcomes used in each analysis. Data points correspond to results from data sets 1 (+), 2 (o), 3 (*), 4 (·), and 5 (×). As a point of reference for evaluating each posterior probability, we have provided a dashed line at 0.1.

4.4 Sensitivity Analyses

As recommended in Section 2.4 we performed sensitivity analyses on the choice of v₀. The results are provided in Web Figures 1 and 2. The results of our univariate analyses were relatively unaffected by an increase (v₀ = 20) and decrease (v₀ = 5) in the degrees of freedom. In our multivariate analyses of y, increasing v₀ to 20 substantially changed the posterior probability of $H_{01}^{\ast }$ and $H_{02}^{\ast }$ in 2/5 Case 3 simulations, while all other results were fairly robust. In one of these data sets, the MCMC mixed poorly which could be reflective of a poor choice of V₀. When v₀ = 5, the prior is quite diffuse and computational instabilities arise due to difficulties in inversion of ill-conditioned covariance matrices, arising in some of the MCMC samples.

Web Figure 1.

Sensitivity analysis of v₀ in our univariate analysis of y₁. Values on the vertical axis correspond to posterior probabilities of the different null hypotheses. Data points correspond to results from data sets 1 (+), 2 (o), 3 (*), 4 (·) and 5 (×).

Web Figure 2.

Sensitivity analysis of v₀ in our multivariate analysis of y. Values on the vertical axis correspond to posterior probabilities of the different null hypotheses. Data points correspond to results from data sets 1 (+), 2 (o), 3 (*), 4 (·), and 5(×). Mixing was poor for Case 3, data set 2 when v₀ = 20.

5. Genotoxicity Example

5.1 Data and Methods

We considered data from a genotoxicity study analyzed previously by Dunson (2006), Dunson et al. (2003), and Dunson and Taylor (2005). The study assessed the effect of oxidative stress, induced by H₂O₂, on the frequency of DNA strand breaks using the comet assay (single cell electrophoresis). Human lymphoblast cells were exposed to 5 different concentrations of hydrogen peroxide (0, 5, 20, 50, and 100 μM H₂O₂). Several surrogate measures of DNA damage were obtained for 100 cells in each group, though we focused on the two best surrogates: y₁ = % tail DNA and y₂ = Olive tail moment (OTM). More information on these surrogates, including diagnostics demonstrating non-normality, may be found in Duez et al. (2003) and Dunson et al. (2003).

Our implementation was identical to that described in the simulation studies, with a few exceptions. Based on the range of surrogate values reported in Duez et al. (2003), we chose β = (50, 10)^′, V₀₁₁ = 181, V₀₂₂ = 10, and V₀₁₂ = V₀₂₁ = 38.2, where V_0ij denotes element (i, j) of V₀. Note that we chose a large value for the covariance because % tail DNA and OTM are likely to be highly correlated. However, since we are uncertain about how well these mean and variance values represent the current data, we chose a relatively diffuse prior for μ₀, N(β, 6 · V₀), and let v₀ = 10. Also, using the guidelines in Section 2.3, we assumed p₀ ∼ Be(0.7, 0.3) a priori. We ran our MCMC for 105,000 iterations, otherwise implementing as in Section 4.

5.2 Results

Our method provided strong evidence in favor of an effect of H₂O₂ on the frequency of DNA strand breaks as $\widehat{\mathrm{Pr}}(H_0\mid \text{Data})<0.001$. As demonstrated by the posterior predictive densities in Figure 3, treatment with H₂O₂ changes the joint distribution of the % tail DNA and OTM from a unimodal distribution favoring small values to a multi-modal distribution supporting large levels of DNA damage. While genotoxic effects are evident at the smallest dose of H₂O₂ (the posterior probability that 5 μM is the LOAEL is > 0.999), they appear to level o at the higher doses; the outcome distribution clearly does not differ between the 50 and 100 μM groups $(\widehat{\mathrm{Pr}}(\text{No difference}50-100\mu \mathrm{M}\mid \text{Data})=0.990)$ and there is little evidence that the distribution changes at doses higher than 5 μM $(\widehat{\mathrm{Pr}}(\text{No difference}5-100\mu \mathrm{M}\mid \text{Data})=0.281)$. Figure 3 demonstrates that these densities are in cagreement with the observed data, supporting goodness of fit.

Figure 3.

Posterior predictive density of % tail DNA and OTM in each dose group. ‘·’ denotes an observation.

Although previous analyses of these data have also demonstrated a relationship between H₂O₂ and the frequency of DNA strand breaks, we have provided several new pieces of information. For instance, neither the latent response models fit by Dunson et al. (2003) and Dunson (2006) nor the quantile regression approach of Dunson and Taylor (2005) provided a formal method for testing for changes across dose groups and for estimating threshold doses. In genetic epidemiology studies of DNA repair it is important to use minimal doses of the test agent to avoid high levels of cell death, which make the comet assay unreliable. Hence our finding that 5 μM is a potential threshold level for DNA damage has important implications for future experiments. Another advantage of our method is our ability to obtain smooth, nonparametric estimates of the joint density of the surrogates.

5.3 Sensitivity Analyses

As in Section 4.4, we performed a sensitivity analysis on our choice of v₀. Increasing the degrees of freedom to v₀ = 20 did not change the posterior probability of H₀ or the LOAEL, but did increase the probability of no change across doses 5−100 μM to 0.826. When v₀ = 5 mixing was poor and the model did not converge after 105,000 iterations.

6. Discussion

We have proposed a nonparametric method which models distribution changes across an ordinal predictor and provides a formal approach for Bayesian testing of local and global changes across groups. In addition, we can perform inferences on multivariate distributions and identify thresholds. Our method provides more informative results than many frequentist k-sample tests and, as demonstrated in simulation studies, may be more sensitive to changes in distributional shape. We acknowledge, however, that these results are based on a small number data sets and larger simulation studies are needed to adequately characterize the sensitivity of our method.

Although our method is relatively simple to program, the computation can be intensive for large data sets especially those with several responses and dose groups. For example, it took approximately 33 hours to complete the MCMC for our analysis of the comet assay data using a Matlab (version 7.1) program compiled in C and run on a Dell Optiplex SX270 (2.8 GHz Pentium 4 processor). Thus, a more efficient computational method would likely be necessary for massive, high dimensional data sets. One promising approach may be to develop a Variational Bayes (VB) implementation (see Blei and Jordan, 2006).

Some extensions of our method may also serve as exciting areas of future research. For example, a formal method of incorporating historical data would be useful in toxicology studies with extensive historical control databases. It may also be of interest to incorporate order restrictions into our method which ensure that the the quantiles of the response are non-decreasing with dose. This extension would be related to the nonparametric approaches for stochastic ordering described by Gelfand and Kottas (2001) and Hoff (2003).

Web Appendix A: Proof of Theorem 1

Note that

$${\Vert G_{h+j+1}-G_h\Vert }_{TV}=\underset{B\in \mathcal{B}}{\max }\mid \prod _{l=h}^{h+j}(1-{\pi }_l)G_h\left(B\right)+\sum _{l=h}^{h+j-1}{\pi }_l\prod _{m=l+1}^{h+j}(1-{\pi }_m)G_l^{\ast }\left(B\right)+{\pi }_{h+j}{G}_{h+j}^{\ast }\left(B\right)-G_h\left(B\right)\mid .$$

Under Sethuraman (1994) stick-breaking representation of the Dirichlet process

$$\begin{array}{cc}\hfill G_1\left(B\right)& =\sum _{j=1}^{\infty }{V}_{1j}\prod _{k<j}(1-V_{1k}){\delta }_{{\theta }_{1k}}\left(B\right),\hfill \\ \hfill G_l^{\ast }\left(B\right)& =\sum _{j=1}^{\infty }{V}_{l+1,j}\prod _{k<j}(1-V_{l+1,k}){\delta }_{{\theta }_{l+1,k}}\left(B\right)l=1,\dots ,h+j,\hfill \end{array}$$

${\theta }_{lk}\stackrel{iid}{\sim }{G}_0,V_{lj}\stackrel{ind}{\sim }\mathrm{Be}(1,{\alpha }_{l-1})$, and δ_θ denotes a point mass at θ. Suppose we chose B to contain (θ_lk, k = 1, . . . , ∞) for l = 1, . . . , h but not for l > h. Then, for a set B satisfying these conditions, G_h(B) = 1 and $G_l^{\ast }\left(B\right)=0\text{for}l=h,h+1,\dots ,h+j$ which causes the absolute value to reduce to

$$\begin{array}{cc}\hfill \mid \prod _{l=h}^{h+j}(1-{\pi }_h)-1\mid & =1-\prod _{l=h}^{h+j}(1-{\pi }_l)\hfill \\ \hfill & ={\pi }_h+(1-{\pi }_h)\sum _{l=h+1}^{h+j-1}{\pi }_l\prod _{k=l+1}^{h+j}(1-{\pi }_k)+{\pi }_{h+j}(1-{\pi }_h)\hfill \\ \hfill & \ge {\pi }_h\hfill \\ \hfill & >∊,\hfill \end{array}$$

since π_h > ∊ when ||G_h+1 − G_h||_TV > ∊. Hence

$$\mathrm{Pr}\left({\Vert G_{h+j+1}-G_h\Vert }_{TV}\le ∊\mid {\Vert G_{h+1}-G_h\Vert }_{TV}>∊\right)=0.$$

Web Appendix B: Gibbs Sampling Methodology

To simplify notation, let ϕ_hi = {μ_hi, Σ_hi}. As in Dunson (2006), the derivation of our full conditional posterior distributions involves re-writing the model in (5) as

$$\begin{array}{cc}\hfill {\varphi }_{hi}& \sim \sum _{l=1}^{h}1(M_{hi}=l)G_l^{\ast \ast }\hfill \\ \hfill M_{hi}& \sim \text{Multinomial}(1,\dots ,h;{\omega }_{h1},\dots ,{\omega }_{hh}),\text{for}i=1,\dots ,n_h\hfill \\ \hfill G_l^{\ast \ast }& \sim \mathrm{DP}\left({\alpha }_{l-1}{G}_0\right),\text{for}l=1,\dots ,h,\hfill \end{array}$$ (B-1)

where $G_l^{\ast \ast }=G_1\text{for}l=1$ and $G_l^{\ast \ast }=G_{l-1}^{\ast }\text{for}l=2,\dots ,h$. As mentioned in Section 2.2, ω_hl is distinct for each h. For l = 1, . . . , d, let ϕ_l = {ϕ_hi : M_hi = l; h = l, . . . , d, i = 1, . . . , n_h} and $m_l={\sum }_{h=l}^d{\sum }_{i=1}^{n_h}1(M_{hi}=l)$. Due to the discrete nature of the DP measure $G_l^{\ast \ast }$, the elements of ϕ_l are allocated to k_l ≤ m_l distinct values (or clusters) ${\theta }_l={({\theta }_{l1}^{\prime },\dots ,{\theta }_{l{k}_l}^{\prime })}^{\prime }$ where ${\theta }_{lr}=\{{\mu }_{lr}^{\ast };{\Sigma }_{lr}^{\ast }\}$ and we let $m_{lr}=\sum {\sum }_{h,i:M_{hi=l}}1({\varphi }_{hi}={\theta }_{lr})$.

Let ${\varphi }_l^{\left(hi\right)}$ be the subset of ϕ_l excluding subject h,i (l = 1, . . . , d). The full conditional prior of ϕ_hi given ${\varphi }^{\left(hi\right)}=\{{\varphi }_1^{\left(hi\right)},\dots ,{\varphi }_d^{\left(hi\right)}\}$ is

$$\sum _{l=1}^h{\omega }_{hl}\left\{\left(\frac{{\alpha }_{l-1}}{{\alpha }_{l-1}+m_l^{\left(hi\right)}}\right)G_0+\sum _{r=1}^{k_l^{\left(hi\right)}}\left(\frac{m_{lr}^{\left(hi\right)}}{{\alpha }_{l-1}+m_l^{\left(hi\right)}}\right){\delta }_{{\theta }_{lr}^{\left(hi\right)}}\right\},$$ (B-2)

where $m_l^{\left(hi\right)}$ is the number of latent ϕ_h′,i′ (h′, i′ ≠ h, i) in ${\varphi }_l^{\left(hi\right)}$ whose unique values are ${\theta }_l^{\left(hi\right)}=({\theta }_{lr}^{\left(hi\right)},r=1,\dots ,k_l^{\left(hi\right)}$ and $m_{lr}^{\left(hi\right)}$ denotes the number of ϕ_h′,i′ having value ${\theta }_{lr}^{\left(hi\right)}$. To simplify notation we will henceforth let

$${\omega }_{hl0}^{\left(hi\right)}=\frac{{\omega }_{hl}\cdot {\alpha }_{l-1}}{{\alpha }_{l-1}+m_l^{\left(hi\right)}}$$

and ${\omega }_{hlr}^{\left(hi\right)}$ denote the multiplier on ${\delta }_{{\theta }_{lr}^{\left(hi\right)}}$ in (B-2). Updating the conditional prior with the data, we have the following full conditional posterior for ϕ_hi:

$$\sum _{l=1}^h\left\{{\tilde{\omega }}_{hl0}^{\left(hi\right)}{G}_{0hi}+\sum _{r=1}^{k_l^{\left(hi\right)}}{\tilde{\omega }}_{hlr}^{\left(hi\right)}{\delta }_{{\theta }_{lr}^{\left(hi\right)}}\right\},$$ (B-3)

where

$$\begin{array}{cc}\hfill {\tilde{\omega }}_{hl0}^{\left(hi\right)}& =c\cdot {\omega }_{hl0}^{\left(hi\right)}\cdot \mathrm{t}({\mathbf{y}}_{hi};v_0-q+1,{\mu }_0,{\mathbf{V}}_0^{\ast })\hfill \\ \hfill {\mathbf{V}}_0^{\ast }& =\frac{v_0(1+\kappa )}{v_0-q+1}\cdot {\mathbf{V}}_0\hfill \\ \hfill {\tilde{\omega }}_{hlr}^{\left(hi\right)}& =c\cdot {\omega }_{hlr}^{\left(hi\right)}\cdot \mathrm{N}({\mathbf{y}}_{hi};{\mu }_{lr}^{\ast \left(hi\right)},{\Sigma }_{lr}^{\ast \left(hi\right)})\hfill \\ \hfill G_{0hi}\left({\varphi }_{hi}\right)& =\mathrm{W}\left({\Sigma }_{hi}^{-1};v_0+1,{\mathbf{V}}_{0hi}^{-1}\right)\cdot \mathrm{N}\left({\mu }_{hi};\frac{\kappa }{\kappa +1}({\mathbf{y}}_{hi}+{\kappa }^{-1}{\mu }_0),\frac{\kappa }{\kappa +1}{\Sigma }_{hi}\right)\hfill \\ \hfill {\mathbf{V}}_{0hi}& =v_0\cdot {\mathbf{V}}_0+\frac{1}{\kappa +1}({\mathbf{y}}_{hi}-{\mu }_0){({\mathbf{y}}_{hi}-{\mu }_0)}^{\prime }\hfill \end{array}$$

and t(·; v, μ, V) denotes the multivariate t density with v degrees of freedom, location parameter μ, and scale V.

Gibbs sampling may proceed by sampling directly directly from (B-3). However to improve mixing, we apply an algorithm which parallels those described by Bush and MacEachern (1996) and West et al. (1994). Let S_hi = (0, l) if ϕ_hi is allocated to a new cluster in mixture component l and S_hi = (r, l) if ${\varphi }_{hi}={\theta }_{rl}^{\left(hi\right)}$. At each iteration of our MCMC, we update the parameters in our model using the following steps:

1. Sample each S_hi from its full conditional posterior distribution which is multinomial with $\mathrm{Pr}(S_{hi}=(r,l)\mid \cdot )={\tilde{\omega }}_{hlr}$ for $l=1,\dots ,h,r=0,1,\dots ,k_l^{\left(hi\right)}$. Whenever S_hi = (0, l) sample a new value for ϕ_hi from G_0hi and assign subject h, i to their own cluster in component l.

2. Given the updated values of S = {S_hi, h = 1, . . . , d; i = 1, . . . , n_h}, for l = 1, . . . , d and r = 1, . . . , k_l sample θ_lr from

$$\mathrm{W}\left({\Sigma }_{lr}^{\ast -1};v_0+m_{lr},{\mathbf{V}}_{0lr}^{\ast -1}\right)\cdot \mathrm{N}\left({\mu }_{lr}^{\ast };{\kappa }^{\ast }({\kappa }^{-1}{\mu }_0+m_{lr}{\stackrel{‒}{\mathbf{y}}}_{lr}),{\kappa }^{\ast }{\Sigma }_{lr}^{\ast }\right),$$ (B-4)

where

$${\mathbf{V}}_{0lr}^{\ast }=v_0{\mathbf{V}}_0+\sum _{h=l}^d\sum _{i:S_{hi}=(l,r)}({\mathbf{y}}_{hi}-{\stackrel{‒}{\mathbf{y}}}_{lr}){({\mathbf{y}}_{hi}-{\stackrel{‒}{\mathbf{y}}}_{lr})}^{\prime }+\frac{m_{lr}}{\kappa \cdot m_{lr}+1}({\mu }_0-{\stackrel{‒}{\mathbf{y}}}_{lr}){({\mu }_0-{\stackrel{‒}{\mathbf{y}}}_{lr})}^{\prime },$$

κ* = κ/(1 + κ · m_lr), and ${\stackrel{‒}{\mathbf{y}}}_{lr}$ is the mean response vector in cluster (l, r).

The priors for our remaining model parameters are conjugate, and thus may be updated by adding the following Gibbs steps to our MCMC algorithm:

3. For h = 1, . . . , d − 1 sample:

a.) ζ_h from $\mathrm{Bin}(1,p_{0h}^{\ast })$ where

$$p_{0h}^{\ast }=\frac{p_0\cdot {\Phi }_h\left({∊}^{\ast }\right)}{p_0\cdot {\Phi }_h\left({∊}^{\ast }\right)+\frac{{∊}^{\ast }}{1-{∊}^{\ast }}(1-p_0)(1-{\Phi }_h\left({∊}^{\ast }\right))},$$

with

$$\begin{array}{cc}\hfill {\Phi }_h\left({∊}^{\ast }\right)& ={\int }_0^{{∊}^{\ast }}\mathrm{Be}(x;a_h^{\ast },b_h^{\ast })dx\hfill \\ \hfill a_h^{\ast }& =1+\sum _{j=h+1}^d\sum _{i=1}^{n_j}1(M_{ji}=h+1)\hfill \\ \hfill b_h^{\ast }& =1+\sum _{j=h+1}^d\sum _{i=1}^{n_j}1(M_{ji}<h+1).\hfill \end{array}$$

b.) π_h from

$$\pi ({\pi }_h\mid \cdot )={\zeta }_h\cdot \frac{\mathrm{Be}({\pi }_h;a_h^{\ast },b_h^{\ast })}{{\Phi }_h\left({∊}^{\ast }\right)}1(0\le {\pi }_h\le {∊}^{\ast })+(1-{\zeta }_h)\cdot \frac{\mathrm{Be}({\pi }_h;a_h^{\ast },b_h^{\ast })}{1-{\Phi }_h\left({∊}^{\ast }\right)}1({∊}^{\ast }<{\pi }_h\le 1),$$ (B-5)

where the notation a|· denotes a given all other variables.

4. Sample p₀ from

$$\pi (p_0\mid \cdot )=\mathrm{Be}\left(p_0;a_p+\sum _{h=1}^{d-1}{\zeta }_h,b_p+d-1-\sum _{h=1}^{d-1}{\zeta }_h\right).$$ (B-6)

5. For h = 0, 1, . . . , d − 1 sample a latent variable, u_h, from π(u_h|α_h, m_h+1) = Be(u_h; α_h + 1, m_h+1) and then sample α_h from its full conditional posterior

$$\pi ({\alpha }_h\mid u_h,k_{h+1},m_{h+1})=p_{uh}\mathrm{Ga}({\alpha }_h;a_{{\gamma }_h}+k_{h+1},b_{{\gamma }_h}-log\left(u_h\right))+(1-p_{uh})\mathrm{Ga}({\alpha }_h;a_{{\gamma }_h}+k_{h+1}-1,b_{{\gamma }_h}-log\left(u_h\right)),$$ (B-7)

where

$$\frac{p_{uh}}{1-p_{uh}}=\frac{a_{{\gamma }_h}+k_{h+1}-1}{m_{h+1}(b_{{\gamma }_h}-log\left(u_h\right))}.$$

6. Sample μ₀ from π(μ₀|·) = N(μ₀; β*,Ω*) where

$$\begin{array}{cc}\hfill {\beta }^{\ast }& ={\Omega }^{\ast }\left({\Omega }^{-1}\beta +\frac{1}{\kappa }\sum _{h-1}^d\sum _{r=1}^{k_h}{\Sigma }_{hr}^{\ast -1}{\mu }_{hr}^{\ast }\right)\hfill \\ \hfill & {\Omega }^{\ast }={\left\{{\Omega }^{-1}+\frac{1}{\kappa }\sum _{h=1}^d\sum _{r=1}^{k_h}{\Sigma }_{hr}^{\ast -1}\right\}}^{-1}.\hfill \end{array}$$

7. Sample κ⁻¹ from

$$\pi ({\kappa }^{-1}\mid \cdot )=\mathrm{Ga}\left({\kappa }^{-1};a_{\kappa }+\frac{q}{2}\sum _{h=1}^d{k}_h,b_{\kappa }+\frac{1}{2}\sum _{h=1}^d\sum _{r=1}^{k_h}{({\mu }_{hr}^{\ast }-{\mu }_0)}^{\prime }{\Sigma }_{hr}^{\ast -1}({\mu }_{hr}^{\ast }-{\mu }_0)\right).$$ (B-8)