Bayesian nonparametric inference on stochastic ordering

Summary

This article considers Bayesian inference about collections of unknown distributions subject to a partial stochastic ordering. To address problems in testing of equalities between groups and estimation of group-specific distributions, we propose classes of restricted dependent Dirichlet process priors. These priors have full support in the space of stochastically ordered distributions, and can be used for collections of unknown mixture distributions to obtain a flexible class of mixture models. Theoretical properties are discussed, efficient methods are developed for posterior computation using Markov chain Monte Carlo, and the methods are illustrated using data from a study of DNA damage and repair.

1. Introduction

Our focus is inference on K group-specific distributions. For example, in toxicology studies, the groups may correspond to different doses of a potentially adverse chemical exposure. In such settings, it is of interest to assess whether or not the response distribution changes across groups, while also estimating the group-specific distributions. Although parametric assumptions may be difficult to justify, it is common to have prior knowledge that the magnitude of a response for a particular experimental unit would not decrease if that unit had been exposed to a higher dose. This implies stochastic ordering in the response distributions.

Focusing on the two-sample case, Arjas & Gasbarra (1996) proposed nonparametric Bayes methods for ordered hazard functions, while Gelfand & Kottas (2001) induced priors on stochastically ordered distributions through products of independent Dirichlet process (Ferguson, 1973, 1974) components. More recently, Hoff (2003b) developed general methods for estimating probability measures constrained to belong to a convex set, considering applications to mean, mode, quantile and stochastic ordering constraints. The problem of Bayesian estimation of probability measures subject to a partial stochastic order was further considered by Hoff (2003a), relying on the theory in Hoff (2003b) to develop latent variable and rejection sampling methods for posterior computation.

It can be difficult to implement these methods routinely, particularly for moderate to large K. In addition, interest often focuses not only on estimation of the group specific distributions but also on testing hypotheses of equalities between groups against stochastically ordered alternatives. There has been surprisingly little work on nonparametric Bayes hypothesis testing and model selection: Gutierrez-Pena & Walker (2005) imbed parametric model selection problems within a Bayesian nonparametric framework; Berger & Guglielmi (2001) test the fit of a parametric model through comparison to a nonparametric alternative; Dass & Lee (2004) study consistency of Bayes factors for testing point nulls versus nonparametric alternatives; and Basu & Chib (2003) develop methods for calculating Bayes factors for comparing Dirichlet process mixture models.

There has been recent interest in Bayesian methods for unconstrained collections of probability measures. Much of this work relies on extending the Dirichlet process. Suppose P is an unknown probability measure on $(\mathcal{X},\mathcal{B}(\mathcal{X}))$, with $\mathcal{X}\subset \Re$ a Borel subset of Euclidean space and $\mathcal{B}(\mathcal{X})$ the Borel σ-algebra of subsets of $\mathcal{X}$. Then P ~ DP(αP₀) denotes that P is assigned a Dirichlet process prior with precision α and base measure P₀. Sethuraman (1994) showed that this is equivalent to the stick-breaking representation:

$$P(\cdot )=\sum _{h=1}^{\infty }{\pi }_h{\delta }_{{\Theta }_h}(\cdot ),{\pi }_h=V_h\prod _{l<h}\left(1-V_l\right),$$ (1)

where V_h ~ Be(1, α) and Θ_h ~ P₀, independently for h = 1, …, ∞. Here, δ_Θ is a probability measure concentrated at the atom Θ, V = {V_h, h = 1, …, ∞} is an infinite sequence of stick-breaking weights, and Θ = {Θ_h, h = 1, …, ∞} is an infinite sequence of atoms, with V and Θ mutually independent.

In order to generalize the Dirichlet process to place a prior on a collection of probability measures (P₁, …, P_K), MacEachern (1999, 2001) proposed a dependent Dirichlet process, which represents each P_k as in (1), but incorporates dependence through dependent stick-breaking weights and/or dependent atoms. De Iorio et al. (2004) used the dependent Dirichlet process to create an analysis-of-variance-like dependence structure for random measures, assuming fixed stick-breaking weights but dependent atoms. Gelfand et al. (2005) developed a related approach for spatially-dependent probability measures. More flexible methods that also allow the stick-breaking weights to vary, with some computational expense, have been proposed by Griffin & Steel (2006) and Duan et al. (2007). Alternative strategies for modelling dependent collections of random measures through convex combinations of independent Dirichlet processes have been proposed by Müller et al. (2004), Pennell & Dunson (2006) and Dunson et al. (2007).

These methods are appealing in allowing flexible borrowing of information across groups. Our contribution is the development of a framework for testing of equalities in distributions between groups against stochastically ordered alternatives, while also allowing estimation of smooth, stochastically ordered densities. Hoff (2003a) assumes the distributions are discrete, noting that Dirichlet process mixtures of parametric models can be used to address this problem, but without developing an approach for inference in such a case.

2. Stochastically ordered random probability measures

2·1. Formulation and background

Let $\left(P_1,\dots ,P_K\right)\in {\mathcal{P}}^K$, with ${\mathcal{P}}^K$ the set of K × 1 collections of probability measures on $(\mathcal{X},\mathcal{B}(\mathcal{X}))$. In addition, define the following convex subset of ${\mathcal{P}}^K$:

$$C_E=\left\{\left(P_1,\dots ,P_K\right)\subset {\mathcal{P}}^K:P_i\preccurlyeq P_j\text{ for all }(i,j)\in E\right\},$$

where E ⊂ (1, …, K)² is a prespecified K × K matrix defining a partial ordering. Here, P_i ≼ P_j if and only if P_i(x, ∞) ≤ P_j(x, ∞) for all x, so that P_j is stochastically larger than P_i. The collections of probability measures belonging to C_E satisfy the partial ordering defined by E.

As shown by Hoff (2003a,b), C_E is a weakly closed convex set with extreme points $\left\{\left({\delta }_{s_1},\dots ,{\delta }_{s_K}\right):s\in {\mathcal{S}}_E\right\}$, where ${\mathcal{S}}_E=\left\{\left(s_1,\dots ,s_K\right)\in {\mathcal{X}}^K:s_i\le s_j\text{ for all }(i,j)\in E\right\}$. Using a corollary to Choquet’s theorem, Hoff (2003a,b) shows that every (P₁, …, P_K) ∈ C_E can be represented as a mixture over the extreme points. Hence, by placing an unconstrained prior on the mixing measure, one can obtain a prior for collections of stochastically ordered random measures. Let x = (x₁, …, x_n)′ denote the observed data, with $x_i~P_{a_i}$, where a_i ∈ {1, …, K} is a group index. Then Hoff (2003a) induces a prior on (P₁, …, P_K) with weak support on C_E by letting $x_i=s_{i,a_i}$, with s_i ~ Q, where Q ~ DP(αQ₀) and Q₀ is a K-variate Borel probability measure on S_E.

2·2. Restricted dependent Dirichlet process

As a direct consequence of applying the Sethuraman (1994) formulation to Q, we obtain the following specification for (P₁, …, P_K):

$$P_k(\cdot )=\sum _{h=1}^{\infty }{\pi }_h{\delta }_{{\Theta }_{hk}}(\cdot ),{\pi }_h=V_h\prod _{l<h}\left(1-V_l\right),k=1,\dots ,K,$$ (2)

where V_h ~ Be(1, α) and Θ_h = (Θ_h1, …, Θ_hK)′ ~ Q₀, for h = 1, …, ∞, with $V={\left\{V_h\right\}}_{h=1}^{\infty }$ and $\Theta ={\left\{{\Theta }_h\right\}}_{h=1}^{\infty }$ mutually independent sequences of stick-breaking random variables and atoms, respectively. We refer to (2) as a restricted dependent Dirichlet process, since the prior implies a dependent Dirichlet process for the collection (P₁, …, P_K), with restrictions incorporated through the atoms.

Although the prior on P = (P₁, …, P_K) is equivalent to that induced by the Hoff (2003a) latent variable specification, so that his theoretical results apply, the reparameterization in (2) allows one to obtain additional insight into properties and to develop methods for hypothesis testing and posterior computation, as will be clear in §3 and 4.

In considering choice of Q₀ in (2), it is important to keep in mind that, unless Q₀ is chosen to assign probability mass to boundaries of S_E, pr(Θ_hi < Θ_hj, h = 1, …, ∞,for all (i, j) ∈ E) = 1. The strict order restriction on the atoms for the different groups can lead to a tendency to overestimate group differences, particularly when the true difference is small, sample sizes are small to moderate, and the number of groups is moderate to large. Similar issues arise in much simpler settings involving estimation of normal means or regression parameters subject to order restrictions (Dunson & Neelon, 2003). By choosing Q₀ to allow probability mass on the boundary of S_E, we allow atoms to be identical in the different groups with positive probability.

In considering choice of Q₀ and the impact on borrowing of information across groups, we focus on the two group case in which E = {(1, 2)}, so that P₁ ≼ P₂. Data consist of $x_i~P_{a_i}$ independently, where a_i = 1 for i = 1, …, n₁ and a_i = 2 for i = n₁ + 1, …, n. We specify Q₀ as

$$f\left({\Theta }_1,{\Theta }_2\right)=f_1\left({\Theta }_1\right)\left\{{\pi }_0{\delta }_0\left({\Theta }_2-{\Theta }_1\right)+\left(1-{\pi }_0\right)f_2\left({\Theta }_2-{\Theta }_1\right)\right\},$$ (3)

where $\mathcal{X}=\Re ,f_1(\cdot )$ is a density on $\Re$, such as Gaussian, 0 ≤ π₀ ≤ 1 is the prior probability of Θ₁ = Θ₂, and f₂(·) is a density with support on ${\Re }^{+}$, such as truncated Gaussian.

It is clear from (3) that borrowing of information between Groups 1 and 2 is controlled by the probability π₀ placed on identical atoms, and by the choice of f₂, which has an impact on the magnitude of the differences in the subset of paired atoms that are not identical. In order to assess how borrowing of information occurs in prediction, after updating the prior with data from the two groups, we calculate the conditional predictive distribution of x_n+1 for a new subject in Group 2.

By a generalization of the Blackwell & MacQueen (1973) Pólya urn scheme, the conditional distribution of x_n+1 given a_n+1 = 2, x₁ and x₂ is

$$\left(x_{n+1}\in \cdot |x_2,x_1\right)~\left(\frac{\alpha }{\alpha +n}\right)f_2^{*}(\cdot )+\sum _{h=1}^{n_1}\left\{\left(\frac{{\pi }_0}{\alpha +n}\right){\delta }_{x_h}(\cdot )+\left(\frac{1-{\pi }_0}{\alpha +n}\right)f_2\left(\cdot ;x_h\right)\right\}+\sum _{h=n_1+1}^n\left(\frac{1}{\alpha +n}\right){\delta }_{x_h}(\cdot ),$$ (4)

where $f_2^{*}=f_1\oplus f_2$ is the convolution of f₁ and f₂, and $f_2\left(x_h\right)={\delta }_{x_h}\oplus f_2$. Hence, a new subject in Group 2 has probability (n − n₁)/(α + n) of being assigned to the same x value as a subject in Group 2 of the dataset, and probability π₀n₁/(α + n) of being assigned to the same value as a subject in Group 1. The probability of being assigned to a new value, x_n+1 ∉ {x₁, x₂}, is {α + n₁(1 − π₀)}/(α + n).

2·3. Restricted dependent Dirichlet process mixtures

Expression (2) implicitly assumes almost sure discreteness of each P_k, so does not allow modelling of continuous distributions subject to a stochastic order constraint. As a broader class of restricted dependent Dirichlet process mixture models, we propose

$$\begin{gathered} x_i~{\mathcal{K}}_{{\sigma }_i}\left(\cdot ,{\mu }_i\right), \\ \left({\mu }_i,{\sigma }_i\right)~P_{a_i},i=1,\dots ,n, \\ {P}_k=\sum _{h=1}^{\infty }{\pi }_h{\delta }_{{\Psi }_h},k=1,\dots ,K, \end{gathered}$$ (5)

where ${\mathcal{K}}_{\sigma }:\mathcal{D}\times \mathcal{X}\to {\Re }^{+}$ is a kernel that depends on the scale parameter σ, $\mathcal{D}$ is the domain of the data, ${\Psi }_h=\left\{{\Theta }_h,{\Gamma }_h\right\}~Q_0\otimes R_0$, ${\left\{{\pi }_h\right\}}_{h=1}^{\infty }$, ${\left\{{\Theta }_h\right\}}_{h=1}^{\infty }$ and Q₀ are as defined in (2), ${\Psi }_h\in {\Re }^{+}$, R₀ is a base probability measure on $\left({\Re }^{+},\mathcal{B}\left({\Re }^{+}\right)\right)$, and Q₀ ⊗ R₀ denotes the product measure.

Letting $g_k(x)=\int {\mathcal{K}}_{\sigma }(x,\mu )P_k(d\mu ,d\sigma )$, for k = 1, …, K and all $x\in \mathcal{D}$, the prior for (P₁, …, P_K) in (5) induces a prior for the collection $g=\left\{g_1,\dots ,g_k\right\}\in {\mathcal{L}}_E$. Suppose for all $\sigma \in {\Re }^{+}$ and $\mu \in \mathcal{X},{\mathcal{K}}_{\sigma }(\cdot ,\mu )$ is a nondegenerate density on $\mathcal{D}$ satisfying the monotone stochastic ordering condition

$${\int }_{\mathcal{D}(z)}{\mathcal{K}}_{\sigma }(x,\mu )dx\ge {\int }_{\mathcal{D}(z)}{\mathcal{K}}_{\sigma }(x,\mu +\Delta )dx,\text{\hspace{1em}for\hspace{0.17em}all\hspace{0.17em}}\Delta >0,z\in \mathcal{D},$$

with $\mathcal{D}(z)$ the subset of $\mathcal{D}$ excluding values greater than z. Then ${\mathcal{L}}_E$ is a set of densities satisfying partial stochastic ordering E from the following result:

LEMMA 1. For all $\left(g_1,\dots ,g_K\right)\in {\mathcal{L}}_E$,

$${\int }_{\mathcal{D}(z)}{g}_k(x)dx\ge {\int }_{\mathcal{D}(z)}{g}_l(x)dx,\text{\hspace{1em}for\hspace{0.17em}all\hspace{0.17em}}z\in \mathcal{D}and(k,l)\in E.$$

Lemma 1 follows from the monotone stochastic ordering condition after noting that

$${\int }_{\mathcal{D}(z)}{g}_k(x)dx=\sum _{h=1}^{\infty }{\pi }_h{\int }_{\mathcal{D}(z)}{\mathcal{K}}_{{\sigma }_h}\left(x,{\mu }_{hk}\right)dx,k=1,\dots ,K,$$

and μ_hk ≤ μ_hl for all (k, l) ∈ E.

Note that ${\mathcal{L}}_E$ is a convex set with extreme points

$$\left[\left\{{\mathcal{K}}_{\sigma }\left(\cdot ,{\mu }_1\right),\dots ,{\mathcal{K}}_{\sigma }\left(\cdot ,{\mu }_K\right)\right\}:\sigma \in {\Re }^{+},\mu \in {\mathcal{S}}_E\right],$$

assuming that ${\mathcal{K}}_{\sigma }\left(\cdot ,\mu \right)$ cannot be expressed as a convex combination of kernels of the same form but with different μ, σ. Every $g\in {\mathcal{L}}_E$ can be represented as a mixture over the extreme points, and our prior in (5) corresponds to placing a Dirichlet process prior on the mixing measure. Consider the special case in which ${\mathcal{K}}_{\sigma }(x,\mu )=\mathcal{K}((x-\mu )/\sigma )/\sigma$, with $\mathcal{K}(\cdot )$ an arbitrary non-degenerate density on $\Re$, such as the standard normal. In this case, $g_k(x)=\int \mathcal{K}((x-\mu )/\sigma )/\sigma P_k(d\mu ,d\sigma )$, with P_k having a marginal Dirichlet process prior on $\left(\Re \times {\Re }^{+},\mathcal{B}(\Re )\otimes \text{B}\left({\Re }^{+}\right)\right)$. From Lo (1984) (refer to expression 3.1) it follows that the support of the prior for g_k contains all densities on $\Re$ with respect to Lesbesgue measure in its closure.

Note that expression (5) induces a Dirichlet process location-scale mixture on each of the densities g_k. A convenient special case corresponds to location-scale mixtures of normal densities. However, it is well known that one can approximate any density on $\Re$ with respect to Lesbesque measure through use of a Dirichlet process location mixture of normals having a single unknown variance (Ghosal et al., 1999, Lijoi et al., 2005). Hence, for simplicity in applications, we focus the remainder of the article on the location mixture

$$x_i~{\mathcal{K}}_{\sigma }\left(\cdot ,{\mu }_i\right),{\mu }_i~P_{a_i},i=1,\dots ,n,$$ (6)

with the prior for (P₁, …, P_K) specified as in (2) and with σ an unknown scale.

3. Properties and hypothesis testing

3·1. Two-group case

One of our primary goals is to develop an approach for hypothesis testing of equalities in distributions between groups against stochastically ordered alternatives. We focus on the model defined in expressions (2) and (6). Under this model, differences between groups in (g₁, …, g_K) are controlled through differences in the mixture distributions. Hence, we base inferences about differences among (g₁, …, g_K) on tests of differences among (P₁, …, P_K).

In developing the methods, we initially consider the two group case in which P₁ ≼ P₂. Instead of requiring the distributions to be exactly identical in the two groups, interval null hypotheses are formulated based on bounding a distance metric. In particular, we focus on the total variation distance,

$$d_{12}=\underset{B\in \mathcal{B}(\mathcal{X})}{\text{max}}\left|P_2(B)-P_1(B)\right|.$$ (7)

LEMMA 2. Under the prior defined in (2) and (3),

$$d_{12}=\sum _{h=1}^{\infty }{\pi }_{h}1\left({\beta }_h>0\right)~\text{Be}\left\{\alpha \left(1-{\pi }_0\right),\alpha {\pi }_0\right\},$$

The proof is in the Appendix. Lemma 2 implies that the distance between P₁ and P₂ stochastically increases with 1−π₀. In addition, E(d₁₂) = 1−π₀, with α controlling uncertainty in this prior expectation.

Using the distance metric in (7), we formalize the null and alternative hypotheses as

$$H_0:d_{12}\le ϵ,H_1:d_{12}>ϵ,$$ (8)

where ϵ > 0 is a small positive constant. In the limit as π₀ → 0, d₁₂ → δ₁ in distribution, so that pr(d₁₂ < ϵ) →0, for any ϵ →1

Theorem 1 provides justification for using hypotheses on the mixing measures as a basis for inferences on G₁ and G₂.

THEOREM 1. Specify H₀ and H₁ as in (8) and let G_k(B) = ∫_B g_k(x)dx, k = 1, 2, with $g_k(x)=\int \mathcal{K}(x,s)d{P}_k(s)$. Then H₀ implies that

$$\underset{x\in (\mathcal{X})}{\text{max}}\left|G_2(x,\infty )-G_1(x,\infty )\right|<ϵ.$$

3·2. Multiple-group case

It is straightforward to generalize the approach in §3.1 to the multiple-group setting. When there are K groups subject to partial stochastic ordering indexed by E, it is first necessary to specify P₀. For most orderings of interest in applications, one can express the hth atom in group k as ${\Theta }_{hk}=w_k^{\prime }{\beta }_h^{*}$, where w_k is a fixed K × 1 vector of 0s and 1s, and ${\beta }_h^{*}={\left({\beta }_{h1}^{*},\dots ,{\beta }_{hK}^{*}\right)}^{\prime }$ is a parameter vector having a subset of elements constrained to be non-negative or nonpositive.

For example, when there is a simple stochastic ordering: P₁ ≼ P₂ ≼ …, P_K we let $w_k={\left(1_k^{\prime },0_{K-k}^{\prime }\right)}^{\prime }$, which implies that ${\beta }_{hk}^{*}={\Theta }_{hk}-{\Theta }_{h,k-1}$, for k = 2, …, K. Hence, constraining ${\beta }_{hk}^{*}\ge 0$, for k = 2, …, K, induces a P₀ with appropriate support. Generalizing prior (3), we let

$$f\left({\beta }_h^{*}\right)=f_1\left({\beta }_{h1}^{*}\right)\prod _{k=2}^K\left\{{\pi }_{0k}{\delta }_0\left({\beta }_{hk}^{*}\right)+\left(1-{\pi }_{0k}\right)f_k\left({\beta }_{hk}^{*}\right)\right\},$$ (9)

where π_0k = pr(Θ_hk = Θ_{h, k−1}) and f_k is a density with support on ${\Re }^{+}$, for k = 2, …, K. To modify the prior to accommodate tree stochastic order, i.e. P₁ ≼ P_k, k = 2, …, K, we can use the same P₀ but with w_k = (1, 0_k−2, 1, 0_K−k)′. Umbrella orderings, with location of the peak known, and cases involving multiple factors, discussed in §5, can also be accommodated by changing W = (w₁, …, w_K)′.

For ease in exposition, we focus our discussion of multiple group hypothesis testing on the simple ordering case, though modifications to other cases are automatic. Generalizing the hypotheses in (8), we let

$$H_{0k}:d_{k,k+1}\le ϵ,H_{1k}:d_{k,k+1}>ϵ,$$ (10)

for k = 1, …, K − 1, with $d_{k,k+1}={\text{max}}_{x\in \mathcal{X}}|P_{k+1}(x,\infty )-P_k(x,\infty )|$. We refer to H_0k as the local null hypothesis of near-equivalence in Groups k and k + 1. One can also consider the global null hypothesis, H₀ d_M ≤ ϵ, where

$$d_M=\underset{(j,l)\in \left\{1,\dots ,K\right\}}{\text{max}}\underset{B\in \mathcal{B}(\mathcal{X})}{\text{max}}\left|P_l(B)-P_j(B)\right|.$$

Hence, H₀ implies near-equivalence in all K groups. It is straightforward to show that d_M ~ Be{α(1 − Π₀),αΠ₀}, with ${\Pi }_0={\prod }_{k=1}^{K-1}{\pi }_{0k}$. Using the Gibbs sampler of §4, we can calculate local and global hypothesis probabilities from a single chain, while also obtaining model-averaged group-specific density estimates.

4. Posterior computation

4·1. Model and background

In describing algorithms for posterior computation, we focus on the following case:

$$\begin{gathered} x_i~N\left(w_{a_i}^{\prime }{\beta }_i,{\tau }^{-1}\right) \\ {\beta }_i~P=\sum _{h=1}^{\infty }{V}_h\prod _{l<h}\left(1-V_l\right){\delta }_{{\beta }_k},V_h~\text{Be}(1,\alpha ),{\beta }_h^{*}~P_0, \end{gathered}$$ (11)

where a_i = k, for i ∈ {n_k−1 + 1, …, n_k−1 + n_k}, k = 1, …, K, with n₀ = 0 and n_k the number of subjects in group k, w_k chosen as discussed in §3.2, and β_i = (β_i1,…,β_iK)′. Note that this expression is a special case of (5) with $\mathcal{K}$ corresponding to a normal kernel and with ${\mu }_i=w_{a_i}^{\prime }{\beta }_i$. Assuming prior (9), we let $f_1\left({\beta }_{h1}^{*}\right)=N\left({\beta }_{h1}^{*};{\mu }_0,{\sigma }_0^2\right)$ and $f_k\left({\beta }_{hk}^{*}\right)=N_{+}\left({\beta }_{hk}^{*};0,{\kappa }^{-1}\right)$, for k = 2, …, K, with N₊(·) denoting the normal density truncated to be positive.

As a result of the reparameterization used in expression (11), we effectively have a typical Dirichlet process mixture of normal linear regression models, with a constrained mixture structure used in the base measure, P₀. Hence, we can use standard Markov chain Monte Carlo algorithms for posterior computation in Dirichlet process mixture models. Since the structure of the base measure creates some difficulties in implementing Pólya urn-based algorithms, we focus on a blocked Gibbs sampler (Ishwaran & James, 2001). This algorithm is based on updating the random weights and atoms in a truncation approximation to the infinite stick-breaking representation.

In particular, let $P={\sum }_{h=1}^N{V}_h{\prod }_{l<h}\left(1-V_l\right){\delta }_{{\beta }_k^{*}}$, with the components defined as in (11) but with V_N = 1 so that terms N + 1, …, ∞ can be excluded. Following the approach of Ishwaran & James (2001), one can show that this approximation tends to be accurate for moderate N, such as N = 20, particularly if α ≤ 1. One can assess whether or not N is sufficiently large by monitoring the maximum index for an occupied cluster: N_O = max_i ζ_i, with ζ_i = h if ${\beta }_i={\beta }_h^{*}$. If the posterior probability assigned to N_O values close to N is not close to zero, N should be increased.

4·2. Prior specification and posterior inference

We recommend a default prior specification. To avoid sensitivity to the measurement scale, standardize the data by subtracting the Group-1 mean and dividing by the Group-1 standard deviation. Then μ₀ = 0 and ${\sigma }_0^2=1$ are chosen for the mean and variance of the Group-1 atoms. To assign high probability to a wide range of mild to moderate shifts in the response density, let $\kappa ~\text{Ga}(1/2,1/2)$ to induce a Cauchy prior on ${\beta }_{hk}^{*}$, for k > 1. The Cauchy is often used as a robust default in parametric model selection (Jeffreys, 1961). A diffuse prior can be chosen for the error precision, τ⁻¹, which is common to all the models.

In addition, we choose a Ga(a_α, b_α) prior for α, with the gamma distribution parameterized to have mean a_α/b_α and variance $a_{\alpha }/b_{\alpha }^2$. There is substantial information in the data about α, so a diffuse prior could be used, but we recommend letting a_α = 1 and b_α = 1 to favour a small-to-moderate number of clusters. Letting ϵ = 0.05 and fixing α = 1, one can assign pr(H_0k) = pr(H_1k) = 1/2 by choosing π_0k = 0.792, the solution to the equation $0.5={\int }_0^{0.05}\text{Be}\left(\pi ;1-{\pi }_{0k},{\pi }_{0k}\right)d\pi$. Letting a_π + b_π = 1 to correspond to a unit information prior, we obtain a hyperprior for π_0k with mean 0.792 by letting a_π = 0.792, b_π = 0.208. In the two group case, the induced prior on d₁₂ is U-shaped, having a mean of 0.208 and high density near 0 and 1.

With the prior specification complete, the Gibbs sampler described in the Appendix can be run to obtain draws from the posterior distribution. These draws can be used to obtain group-specific density estimates and posterior hypothesis probabilities. The posterior distribution of the distance between any two groups can also be obtained.

5. Simulation examples

We considered two simulation cases. In both cases, K = 2 and data in Group 1 were simulated from the following mixture of three normals:

$$f(y)=0.2N\left(y;-2.5,{\tau }^{-1}\right)+0.7N\left(y;0,{\tau }^{-1}\right)+0.1N\left(y;1.5,{\tau }^{-1}\right),$$

with τ = 3. In Case 1 data in Group 2 were simulated from this same density, while in Case 2 we increased the component-specific means slightly from (−2.5, 0, 1.5) to (−2.4, 0.4, 2.2), resulting in a distance of d₁₂ = 1 as each of the means varied.

For each case, we simulated 100 datasets under three sample sizes, n₀ = 10, 25 and 100, with n₀ denoting the sample size per group. Each dataset was analyzed using the Gibbs sampler of §4, with 2000 iterations collected after a 500 iteration burn-in. Apparent convergence was rapid and mixing was good. The set of simulations in the n₀ = 100 case was run using Matlab on a Mac PowerBook G4 laptop in approximately 10 hours.

Figure 1 plots the results in Case 1, showing the group-specific Bayesian density estimates for each of the 100 simulated datasets in the three sample sizes under consideration, along with a histogram of the d₁₂ distance measure. For a small sample size of 10/group, the estimates were surprisingly good on average, though the primary or secondary mode was substantially overestimated in a small proportion of simulations. In addition, in many of the simulations, the density estimate exhibited shrinkage towards the normal base measure, with the primary peak and dip in the left-hand tail underestimated. The performance improved dramatically for n₀ = 25, with most of the density estimates close to the truth. The n₀ = 100 estimates were all very close to the truth. The distribution of the posterior mean of d₁₂ across the simulations was increasingly concentrated at the true value of 0 as the sample size increased. Figure 2 plots the results in Case 2. The performance was similar to that described in case 1, but with the distribution of d₁₂ concentrated increasingly away from 0 as the sample size increased.

Figure 1:

Results from simulation study Case 1. The rows correspond to the sample sizes of 10, 25 and 100 per group. The first two columns show the posterior mean density estimates for Groups 1 and 2 from each simulation in dotted curves, with the true densities as solid curves. The third column shows a histogram of the posterior means of the distance d₁₂ between the groups, with vertical lines for the prior mean.

Figure 2:

Results from simulation study Case 2. The rows correspond to the sample sizes of 10, 25 and 100 per group. The first two columns show the posterior mean density estimates for groups 1 and 2 from each simulation in dotted curves, with the true densities as solid curves. The third column shows a histogram of the posterior means of the distance d₁₂ between groups, with vertical lines for the prior mean.

Summary statistics of d₁₂ across the simulations are shown in Table 1. The posterior probability of H₀ increased as the sample size increased under Case 1, and decreased as the sample size increased under Case 2. For the two small sample sizes, there was not strong evidence in favour of either hypothesis. This is as expected, given that we have purposely chosen a simulation case in which the difference between the groups is subtle. Repeating the simulation in a case with a larger difference, results not shown, showed that we can obtain strong evidence in favour of H₁ even in sample sizes of n₀ = 10. Table 1 illustrates that the posterior hypothesis probabilities are somewhat robust to ϵ, with changes of less than 0.10 even with ϵ varying from 0.01 to 0.1. To assess sensitivity of hypothesis testing to the hyperprior on π₀, we repeated simulation Cases 1 and 2 for sample size n₀ = 100 with (i) π₀ fixed at 0.5 and (ii) a_π = b_π = 1. In each case, density estimation performance was the same as that shown in Figs 1 and 2. However, when π₀ = 0.5, there was a tendency to overestimate d₁₂, leading to substantially lower posterior probabilities of H₀ in Case 1 than those shown in Table 1. This is as expected given that fixing π₀ does not allow borrowing of information across the atoms in the restricted dependent Dirichlet process specification. Although prior (ii) was also centred on π₀ = 0.5, there was less of a tendency to overestimate d₁₂ under H₀.

Table 1.

Simulation study. Summary statistics of d₁₂ across the simulations for each sample size. Results shown are means with 95% empirical confidence limits in parentheses.

Case	Common sample size	d12	Pr(d12 < ϵ | data)
			ϵ = 0.01	ϵ = 0.05	ϵ = 0.1
1	10	0.140 (0.028, 0.614)	0.67 (0.17, 0.86)	0.72 (0.18, 0.91)	0.75 (0.22, 0.93)
1	25	0.085 (0.011, 0.299)	0.72 (0.19, 0.89)	0.78 (0.22, 0.94)	0.81 (0.28, 0.96)
1	100	0.044 (0.003, 0.215)	0.79 (0.28, 0.94)	0.85 (0.30, 0.99)	0.89 (0.42, 1.00)
2	10	0.233 (0.029, 0.660)	0.55 (0.10, 0.82)	0.60 (0.10, 0.91)	0.63 (0.10, 0.93)
2	25	0.278 (0.032, 0.765)	0.46 (0.02, 0.84)	0.51 (0.02, 0.87)	0.54 (0.06, 0.91)
2	100	0.560 (0.091, 0.884)	0.13 (0.00, 0.74)	0.15 (0.00, 0.79)	0.18 (0.00, 0.83)

6. Genotoxicology application

6·1. Data structure and the scientific problem

We applied the approach to data from a study of DNA damage and repair. Batches of cells were exposed to 0, 5, 20, 50 or 100 micromoles H₂O₂ and DNA damage was then measured in individual cells after allowing a repair time of 0, 60 or 90 minutes. With i = 1, …, n indexing the cells under study, the measured response x_i for cell i was the Olive tail moment, which is a surrogate of the frequency of DNA strand-breaks obtained using the comet assay.

The goal of the study is to assess the sensitivity of the comet assay to detecting damage induced by the known genotoxic agent H₂O₂, while also investigating how rapidly damage is repaired. Let a_i ∈ {1, …, K} be a group index denoting the level of H₂O₂ and repair time for cell i. The value of a_i for each dose × repair time value is shown in Fig. 3. The total sample size is 1400, with 100 cells per group except for groups 9 and 13, which had 50.

Figure 3:

Genotoxicity application. Directed graph illustrating order restriction. Arrows point towards stochastically larger groups. Posterior probabilities of H_1k are shown.

Among cells with zero repair time, DNA damage should be nondecreasing with dose of H₂O₂. In addition, within a given dose level, DNA damage should be nonincreasing with repair time. Hence, we make the ordering assumption illustrated in Fig. 3 using a directed graph, with arrows pointing towards stochastically larger groups. We wish to assess whether or not DNA damage continues to increase at higher levels of H₂O₂ exposure and to investigate whether or not damage is significantly reduced across each increment of the repair time.

6·2. Analysis and results

Previous authors analyzed the data in Groups 1–5 using a dynamic mixture of Dirichlet processes (Dunson, 2006; Pennell & Dunson, 2006), which allows for dependence in the distributions within adjacent dose groups but does not enforce stochastic ordering restrictions. In addition, that model does not allow the incorporation of both dose of H₂O₂ and repair time, though extensions are possible.

We implemented the approach described in §4. The Gibbs sampler was run for 20, 000 iterations after a 1, 000-iteration burn-in. As in the simulation study, the chain appeared to converge rapidly and mix efficiently based on standard diagnostics. In fact, 20, 000 iterations was really more than sufficient, and we obtained essentially indistinguishable results based on a 500-iteration burn-in and a 2, 000-iteration collection interval, as used in the simulation studies.

Figure 4 plots group-specific Bayesian density estimates and 95% pointwise credible intervals. Although the Bayes estimates have been constrained to follow a stochastic ordering, there is no evidence of systematic deviations from frequentist kernel density estimates obtained using only the data in a given group. Figure 3 provides posterior probabilities of local alternative hypotheses for different group comparisons. The results suggest highly significant increases in DNA damage between the 0, 5 and 20 micromole H₂O₂ dose groups given a repair time of 0 minutes, with the evidence of further increases at higher doses less clear. As expected, there is no evidence of a change in the distribution between Groups 1, 6 and 11, since there was no induced damage to be repaired. However, there were highly significant decreases in DNA damage in each of the exposed groups after a repair time of 60 minutes. Allowing an additional 30 minutes of repair did not significantly alter the distribution. These conclusions were the same for ϵ = 0.01 and ϵ = 0.10

Figure 4:

Genotoxicity application. Estimated densities of the Olive tail moment in a subset of the H₂O₂ dose × repair-time groups. Solid curves are posterior mean density estimates and dashed curves provide 95% pointwise credible intervals.

These results are consistent with the raw data and plots shown in Fig. 4, and are both scientifically reasonable and interesting. It is known that the type of damage induced by hydrogen peroxide can be repaired quickly by base excision and repair mechanisms. The result that there is no further improvement after 60 minutes suggests that it may be unnecessary to collect data for repair times exceeding an hour in future molecular epidemiology studies using the comet assay to identify genotypes predictive of DNA repair rates.

To assess sensitivity, we repeated analyses using a variety of alternative hyperprior settings. In particular, we tried the cases a_π + b_π = 5 instead of 1 to correspond to a more informative prior, a_α = 1, b_α = 1 to favour more clusters, κ = 1, and κ = 2. The estimated densities did not change noticeably across these analyses and the conclusions were robust.

7. Discussion

The proposed approach for posterior computation focused on order restrictions that can be expressed in terms of sign constraints on a K × 1 vector of regression coefficients specific to each Dirichlet process component. This encompasses simple order, tree orders, umbrella orders with known changepoints, and certain multi-factor cases, such as that considered in Section 6. However, orders having more than K − 1 restrictions cannot be accommodated. For example, considering the order restriction illustrated graphically in Fig. 3, a different computational approach is needed if the graph were modified to include additional arrows, e.g., from Groups 6 to 7.

Although we have focused on a relatively simple setting, it is straightforward to imbed our prior for stochastically ordered mixture distributions within a larger hierarchical model. For example, one can incorporate a parametric adjustment for covariates within the kernel. In addition, one can allow for stochastically ordered latent variable distributions. Ongoing work focuses on the extension to incorporate continuous predictors, which can conceptually be accomplished by replacing the atoms with nondecreasing stochastic processes.

Appendix

Technical details

Proof of Lemma 2.

We first show that

$$d_{12}=\underset{B\in \mathcal{B}(\mathcal{X})}{\text{max}}\left\{\sum _{h=1}^{\infty }{\pi }_{h}1\left({\Theta }_{h1}\in B,{\Theta }_{h2}\notin B\right)\right\}=\sum _{h=1}^{\infty }{\pi }_{h}1\left({\beta }_h>0\right).$$

This follows by first noting that the expression in {·} is maximized when B is chosen as the union of small neighbourhoods around the atoms ${\left\{{\Theta }_{h1}\right\}}_{h=1}^{\infty }$, with these neighbourhoods sufficiently small to exclude all elements of ${\left\{{\Theta }_{h2}\right\}}_{h=1}^{\infty }$ that do not belong to ${\left\{{\Theta }_{h1}\right\}}_{h=1}^{\infty }$. Then, under (3), it is clear that, for this choice of B, $1\left({\Theta }_{h1}\in B,{\Theta }_{h2}\notin B\right)=1\left({\beta }_h>0\right)$. The final part of Lemma 1 follows from noting that G ~ DP(αG₀) implies

$$G(B)=\sum _{h=1}^{\infty }{\pi }_{h}1\left({\Theta }_h\in B\right)~\text{Be}\left[\alpha G_0(B),\alpha \left\{1-G_0(B)\right\}\right],\text{\hspace{0.17em}f}orallB\in \mathcal{B}(\mathcal{X}),$$

where V_h ~ Be(1, α), Θ_h ~ G₀, h = 1, …, ∞. In addition, 1(Θ_h ∈ B) ~ Ber{G₀(B)}.

Proof of Theorem 1.

The conditions in Theorem 1 imply that

$$G_1(x,\infty )=\sum _{h=1}^{\infty }{\pi }_h{\mathcal{K}}^{*}\left(x,{\Theta }_{h1}\right)\text{\hspace{1em}and\hspace{1em}}{G}_2(x,\infty )=\sum _{h=1}^{\infty }{\pi }_h{\mathcal{K}}^{*}\left(x,{\Theta }_{h1}+{\beta }_h\right),\forall x\in \mathcal{X},$$

where $\mathcal{K}*(x,\Theta )=\underset{x}{\overset{\infty }{\int }}\mathcal{K}(z,\Theta )dz$. Letting ${\mathcal{H}}_0=\left\{h:{\beta }_h=0,h=1,2,\dots ,\infty \right\}$ and ${\overline{\mathcal{H}}}_0=\left\{1,2,\dots ,\infty \right\}\backslash {\mathcal{H}}_0$, we have

$$G_2(x,\infty )-G_1(x,\infty )=\sum _{h\in {\overline{\mathcal{H}}}_0}{\pi }_h\left\{{\mathcal{K}}^{*}\left(x,{\Theta }_{h1}+{\beta }_h\right)-{\mathcal{K}}^{*}\left(x,{\Theta }_{h1}\right)\right\}.$$

Since ${\sum }_{h\in {\overline{\mathcal{H}}}_0}{\pi }_h<ϵ$ under H₀ and, for any fixed x, $0\le {\mathcal{K}}^{*}(x,\Theta )\le 1$ is a monotone increasing function of Θ, Theorem 1 follows directly.

Gibbs sampling steps.

Let ζ_i = h denote that individual i is sampled from component h, so that ${\beta }_i={\beta }_h^{*}$, for h = 1, …, T. Then the blocked Gibbs sampler proceeds through the following steps.

Step1. Update ζ_i, for i = 1, …, n, by sampling from the multinomial conditional distribution with

$$\text{pr}\left({\zeta }_i=h\right)=\frac{{\pi }_{h}N\left(x_i;w_{a_i}^{\prime }{\beta }_h^{*},{\tau }^{-1}\right)}{{\sum }_{l=1}^N{\pi }_{l}N\left(x_i;w_{a_i}^{\prime }{\beta }_l^{*},{\tau }^{-1}\right)},h=1,\dots ,N.$$

Step 2. Assuming τ ~ gamma(a_τ, b_τ), update τ by sampling from the full conditional,

$$\text{gamma}\left(a_{\tau }+\frac{n}{2},b_{\tau }+\frac{1}{2}\sum _{i=1}^n{\left(x_i-w_{a_i}^{\prime }{\beta }_i\right)}^2\right).$$

Step 3. Update V_h, for h = 1, …, N − 1, by sampling from the full conditional distribution,

$$\text{Be}\left(1+\sum _{i=1}^{n}1\left({\zeta }_i=h\right),\alpha +\sum _{i=1}^{n}1\left({\zeta }_i>h\right)\right).$$

Step 4. Update ${\beta }_h^{*}$, for h = 1, …, N, via the following Gibbs sub-steps:

1. update ${\beta }_{h1}^{*}$ from N(E_h1, V_h1), where

   $$E_{h1}=V_{h1}\left\{{\sigma }_0^{-2}{\mu }_0+\tau \sum _{i:{\zeta }_i=h}\left(x_i-\sum _{k=2}^K{w}_{a_i,k}{\beta }_{ik}\right)\right\},V_{h1}={\left\{{\sigma }_0^{-2}+\tau \sum _{i=1}^{n}1\left({\zeta }_i=h\right)\right\}}^{-1}$$
2. update ${\beta }_{hk}^{*}$, for k = 2, …, K, from the full conditional, ${\widehat{\pi }}_{hk}{\delta }_0+\left(1-{\widehat{\pi }}_{hk}\right)N_{+}\left(E_{hk},V_{hk}\right)$, where the conditional probability of ${\beta }_{hk}^{*}=0$ is

   $${\widehat{\pi }}_{hk}={\pi }_{0k}{\left\{{\pi }_{0k}+\left(1-{\pi }_{0k}\right)\frac{\sqrt{2\kappa }{\int }_0^{\infty }N\left(z;E_{hk},V_{hk}\right)dz}{\sqrt{\pi }N\left(0;E_{hk},V_{hk}\right)}\right\}}^{-1},$$

   and the mean and variance in the normal component are, respectively,

   $$E_{hk}=V_{hk}\left\{\tau \sum _{i:{\zeta }_i=h}\left(x_i-\sum _{l\ne k}{w}_{a_i,i}{\beta }_{il}\right)\right\},V_{hk}={\left\{\kappa +\tau \sum _{i:{\zeta }_i=h}^n{w}_{a_i,k}^2\right\}}^{-1}.$$

Step 5. Assuming α ~ Ga(a_α, b_α), update α from its full conditional distribution,

$$\text{Ga}\left(a_{\alpha }+N-1,b_{\alpha }-\sum _{h=1}^{N-1}\text{log}\left(1-V_h\right)\right).$$

Step 6. Assuming π_0k ~Be(a_π, b_π), for k = 2, …, K, update π_0k from its full conditional distribution,

$$\text{Be}\left(a_{\pi }+\sum _{h=1}^{N}1\left({\beta }_{hk}^{*}=0\right),b_{\pi }+\sum _{h=1}^{N}1\left({\beta }_{hk}^{*}\ne 0\right)\right).$$

Step 7. Assuming κ ~ Ga(a_κ, b_κ), update κ from its full conditional distribution,

$$\text{Ga}\left(a_{\kappa }+\frac{1}{2}\sum _{h=1}^N\sum _{k=2}^{K}1\left({\beta }_{hk}^{*}\ne 0\right),b_{\kappa }+\frac{1}{2}\sum _{h=1}^N\sum _{k=2}^K{\left({\beta }_{hk}^{*}\right)}^2\right).$$