BAREB: A Bayesian repulsive biclustering model for periodontal data

Summary

Preventing periodontal diseases (PD) and maintaining the structure and function of teeth are important goals for personal oral care. To understand the heterogeneity in patients with diverse PD patterns, we develop BAREB, a Bayesian repulsive biclustering method that can simultaneously cluster the PD patients and their tooth sites after taking the patient- and site-level covariates into consideration. BAREB uses the determinantal point process (DPP) prior to induce diversity among different biclusters to facilitate parsimony and interpretability. Since PD progression is hypothesized to be spatially-referenced, BAREB factors in the spatial dependence among tooth sites. In addition, since PD is the leading cause for tooth loss, the missing data mechanism is non-ignorable. Such nonrandom missingness is incorporated into BAREB. For the posterior inference, we design an efficient reversible jump Markov chain Monte Carlo sampler. Simulation studies show that BAREB is able to accurately estimate the biclusters, and compares favorably to alternatives. For real world application, we apply BAREB to a dataset from a clinical PD study, and obtain desirable and interpretable results. A major contribution of this paper is the Rcpp implementation of our methodology, available in the R package BAREB.

1 |. INTRODUCTION

Periodontal disease (PD), a chronic widespread inflammatory disease, can damage the soft tissues and bones that support the teeth, leading to loosening and eventual loss of teeth. In addition to impacting the quality of life,¹ PD has been linked to a number of systemic diseases, such as heart diseases and diabetes.^2,3 Therefore, preventing PD, and maintaining the structure and function of teeth are important goals of personal oral care. The most popular clinical biomarker quantifying the progression of PD is the clinical attachment level (CAL), defined as the depth (in mm) from the cementoenamel junction to the base of a tooth. In clinical studies, the CAL is measured at six pre-specified tooth-sites (excluding the four third molars) using a periodontal probe, which leads to 168 measurements for a full mouth with no missing teeth (for illustration, see Figure F1 in the Supplementary Materials).

PD data are complex and multi-level (patient-, tooth-, site-level), and traditional summary-based statistical approaches, such as mean, sum scores, or maximum site-level values when applied to patient-level evaluation lead to imminent loss of information.^4,5,6 To mitigate this and other salient features of PD data such as non-random missingness, spatial association, and non-Gaussianity of responses, Reich and colleagues^7,8 proposed Bayesian inference under the desired mixed-effects modeling framework. However, in estimating covariate effects on the CAL response, the authors assumed all patients share the same coefficient. This assumption is questionable, as the rate of PD progression can be very different among patients, with possible clustering of patients according to PD incidence.

The motivation for this work comes from a clinical study of oral PD assessment for Gullah-speaking African-American Diabetic patients (henceforth, GAAD study) residing in the coastal South Carolina sea-islands.³ To evaluate the heterogeneity of PD incidence among patients, available one-dimensional clustering methods focus on grouping either the patients, or their toothsites, separately,⁹ according to disease status. While useful, such clustering techniques cannot identify co-localized tooth sites that are important in inferring patient-level clustering. Furthermore, clustering of tooth sites should depend on which subgroup of patients we focus upon, given that different subgroups may partition the tooth sites in different ways, indicating different PD patterns. Therefore, it is desirable to learn whether there exist subgroups of patients, such that within each subgroup, the PD incidence of some tooth sites are different from others, and also different from patients in other subgroups. However, this inferential framework is further complicated in presence of missing data. In the GAAD dataset, a considerable proportion of patients (around 95%) have missing teeth, with an average of 32% teeth missing for a patient. This missingness is often assumed non-ignorable in PD studies,⁷ since PD is a major cause for tooth-loss, and estimating the counterfactual CAL values the missing tooth-sites (from a missing tooth) would have had if the tooth was not missing can facilitate subgroup identification. This is important, in conjunction to spatial associations observed in PD progression studies, i.e., proximally located tooth/tooth-sites may exhibit similar PD patterns compared to the distally located ones, because a missing/failed tooth is predictive of higher PD status, and often expected to be surrounded by teeth with high CAL.¹⁰ We aim to fill these gaps in the existing PD literature by developing a probabilistic biclustering, or two-dimensional clustering method.¹¹

The current literature on biclustering is considerably rich,^12,13,14,15 with applications to genomics and other fields. However, all these biclustering methods focus on the mean in each bicluster, and fails to incorporate important covariate information (such as age, gender), and aforementioned data characteristics typical to PD. Also, from a Bayesian standpoint, utilizing independent priors on the bicluster-specific parameters continues to remain popular due to their computational convenience and flexibility.^13,15,16 However, such an approach could cause over-fitting issues and redundant biclusters, leading to inferences that are hard to interpret.¹⁷ For example, the NoB-LCP biclustering method of Xu et al.¹⁶ used the Dirichlet process (DP) priors¹⁸, where the atoms are independent and identically distributed (i.i.d.) from a base distribution, to infer clustering of histone modifications and genomic locations. Due to the properties of DP, NoB-LCP inferred a large number of small clusters with very few genomic locations, leading to unnecessarily complex models and poor interpretability.

To this end, we develop BAREB, a BAyesian REpulsive Biclustering model to study the heterogeneity in patients with diverse PD patterns. Our contributions are three fold. First, under a matrix formulation of our CAL responses (with rows as patients and columns as tooth-sites), the proposed method produces simultaneous clustering of the study patients and their tooth sites, taking into account the spatial association among tooth sites and non-random missingness patterns, and provides model-based posterior probabilities for these random partitions. These biclusters are defined via consistent associations between CAL values and covariates among a subset of tooth sites for a subgroup of patients. In other words, our proposed model will cluster any two patients together, if they give rise to the same partition of tooth sites. Second, to address the issues with independent priors, we make use of a repulsive prior – the determinantal point process, or DPP¹⁹ on the random partitions, which encourages diversity in PD patterns among different biclusters. Bayesian inference using the DPP priors have proved to be extremely effective in facilitating parsimony and interpretability in a variety of mixture and latent feature allocation models with biomedical applications to infer clinically meaningful subpopulations.^20,21 Third, BAREB introduces a (latent) shared-parameter framework to deal with the non-ignorable missingness, with the resulting marginal mixture density accommodating the non-Gaussianity of CAL responses. In addition, integrated R and C++ codes for implementing BAREB in other application domains are available via the R package BAREB.²²

The rest of the paper proceeds as follows. In Section 2, we present the statistical formulation for BAREB and develop the Bayesian inferential framework, with the associated choice of priors, joint likelihood, posteriors, and model comparison measures. In Section 3, we evaluate the finite sample performance of BAREB, and the advantages of using the repulsive DPP over plausible alternatives via a simulation study. Application of our method to the motivating GAAD data is presented in Section 4. Finally, we conclude, with a discussion in Section 5.

2 |. STATISTICAL MODEL AND INFERENCE

2.1 |. Formulating BAREB

We use an N × J matrix Y = [y_ij] to represent the observed CAL values, with y_ij denoting the CAL for patient i at tooth site j, where i = 1, …, N; j = 1, …, J. From the motivating GAAD study, we consider patients with at least one tooth present, and with complete set of covariates. Missing CAL values are denoted as y_ij = NA. Note, a tooth-site is missing, if and only if the corresponding tooth is missing. Hence, we consider the missingness indicator δ_i(t) at the tooth-level, i.e., δ_i(t) = 1 if tooth t of patient i is missing, otherwise δ_i(t) = 0, i = 1, …, N; t = 1, …, T.

BAREB clusters any two patients together if they have the same partition of tooth sites after accounting for patient-level covariates. Since the clustering of tooth sites are nested within clusters of patients, we start the model construction with a random partition of patients {1, …, N} by denoting the vector e = (e₁, …, e_N) as the patient cluster membership indicator. Denote S to be the number of patient clusters, where e_i = s indicates patient i belongs to patient cluster s, s = 1, …, S. We propose a categorical distribution prior for e, such that

$$e_i\stackrel{ \text{i}.\text{i}.\text{d}. }{~}\text{Categorical }(w),i=1,\dots ,N,$$ (1)

where w = (w₁, …, w_S) with ${\sum }_{s=1}^S{w}_s=1$. We assume a Dirichlet distribution prior on w, such that w ~ Dirichlet(α), where α = (α₁, …, α_S).

Next, we consider clustering of tooth sites within each of the S patient clusters. Recall, the partition of tooth sites is nested within patient clusters, i.e., site clusters can be different for different patient clusters. Let D_s be the number of tooth site clusters for the s-th patient cluster. Define r_s = (r_s1, …, r_sJ) the vector of clustering labels r_sj ∈ {1, …, D_s} that describe the partition of tooth sites corresponding to the s-th patient cluster, where r_sj = d denotes that tooth site j is assigned to site cluster d in patient cluster s. Letting r = (r₁, …, r_S), we assume independent categorical priors for each r_s, given by

$$p(r|e)=\prod _{s=1}^{S}p\left(r_s\right)\text{ and }{\mathit{r}}_{sj}\stackrel{ \text{i}.\text{i}.\text{d}. }{~}\text{ Categorical}\left({\mathit{\varphi }}_s\right),j=1,\dots ,J$$ (2)

where ϕ_s ~ Dirichlet(${\alpha }_s^{\mathit{\varphi }}$), with ${\alpha }_s^{\mathit{\varphi }}=\left({\alpha }_{s1}^{\varphi },\dots ,{\alpha }_{s{D}_s}^{\varphi }\right)$. The prior probability models on the biclustering in (1) and (2) can be characterized as a partition of patients and a nested partition of tooth sites, nested within each cluster of patients. These biclusters will provide valuable information on the periodontal decay patterns of teeth and heterogeneity among PD patients for developing subsequent prevention and treatment strategies.

Given e and r, we construct a sampling model for the CAL as:

$$y_{ij}=x_i{\beta }_i+z_j{\gamma }_{ij}+v_{ij}+{ϵ}_{ij}.$$ (3)

Here, x_i is the vector of patient-level covariates (e.g., age) for patient i with the corresponding regression parameter β_i; z_j is the vector of tooth site-level covariates (e.g., jaw indicator) including an intercept term with the corresponding regression parameter γ_ij; v_ij models the spatial dependence among tooth sites; and ϵ_ij are independent errors: ${ϵ}_{ij}\stackrel{ \text{i}.\text{i}.\text{d}. }{~}N\left(0,{\sigma }^2\right)$.

The prior probability models for β_i and γ_ij make use of the biclustering. We define p(β_i, γ_ij | e, r) as follows. If e_i = s and r_sj = d, we assume all sites in the same site cluster d in patient cluster s share the parameter ${\tilde{\gamma }}_{sd}$, and all patients in the same patient cluster s share the parameter ${\tilde{\beta }}_s$, i.e., ${\beta }_i={\tilde{\beta }}_s$ for all i with e_i = s and ${\gamma }_{ij}={\tilde{\gamma }}_{sd}$ for all i and j, with e_i = s and r_sj = d. Figure 1 presents a graphical illustration of the proposed BAREB model with 8 patients and 6 tooth sites. Here, we assume three patient clusters, with cluster # 1 having two site clusters, cluster # 2 having three site clusters, and cluster # 3 having two site clusters. Varying colors indicate the different biclusters with the corresponding parameters. Within each patient cluster, all patients share the same ${\tilde{\beta }}_s$, but different ${\tilde{\gamma }}_{sd}’\text{s}$ across different site clusters, with d = 1, …, D_s; s = 1, …, S. We will discuss the priors for ${\tilde{\beta }}_s$ and ${\tilde{\gamma }}_{sd}$ in Section 2.2.

FIGURE 1.

An illustration of the proposed BAREB model with 8 patients and 6 tooth sites. Here, we assume 3 patient clusters.

To account for non-randomly missing data, we introduce a latent variable g_i(t), such that δ_i(t) = I(g_i(t) > 0), t = 1, …, T. We model g_i(t) as

$$g_i(t)~N\left({\mu }_i^{*}(t),1\right)\text{ and }{\mu }_i^{*}(t)=c_0+c_1{R}_t{\mu }_i,$$ (4)

where μ_i = (μ_i1, …, μ_iJ) with each element μ_ij = x_iβ_i + z_jγ_ij + v_ij, R_t is a J-dimensional vector with $R_t(j)=\frac{1}{6}$ if site j is on tooth t, and 0 otherwise, and c = (c₀, c₁) is the unknown (estimable) parameter that controls the relationship between the CAL response and the probability of missing tooth. Under this shared-parameter joint modeling framework⁷, the mean μ_i is shared between the two regression models, with the mean of g_i(t) specified as the average CAL value corresponding to the six locations of tooth t for patient i. If c₁ > 0, higher CAL values will more likely result into increased probability of missing tooth, and vice versa. From (4), we can easily derive $p\left({\delta }_i(t)=1\right)=\mathbf{\Phi }\left({\mu }_i^{*}(t)\right)$ after integrating out g_i(t), where Φ(·) is the cumulative distribution function of the standard normal distribution.

2.2 |. Priors and joint likelihood

In this subsection, we discuss the priors on the linear coefficients ${\tilde{\beta }}_s’\text{s}$ and ${\tilde{\gamma }}_{sd}’\text{s}$, and the spatial term v_ij. In practice, independent priors on biclustering-specific parameters ${\tilde{\beta }}_s’\text{s}$ and ${\tilde{\gamma }}_{sd}’\text{s}$ are preferred due to their computational tractability. However, such an approach could cause redundant clusters, resulting in inferences that are hard to interpret as biologically/clinically meaningful subpopulations.^15,16 Xu etal.²¹ proposed to use the DPP as a prior to induce repulsiveness among component-specific parameters in the context of Gaussian mixture models, and showed that the DPP prior yields more parsimonious and interpretable inference compared to independent priors. Now, we extend the use of the DPP in our biclustering setup to encourage repulsive coefficients ${\tilde{\beta }}_s’\text{s}$ and ${\tilde{\gamma }}_{sd}’\text{s}$, thereby inducing diverse PD patterns among different biclusters.

Let C^β denote an S×S positive semidefinite matrix constructed through a covariance function $C_{s{s}^{\prime }}^{\beta }=C^{\beta }\left({\beta }_s,{\beta }_{s^{\prime }}\right)$. We modify the DPP prior used in Xu et al.²¹ by defining a prior on $\left({\tilde{\beta }}_1,\dots ,{\tilde{\beta }}_S\right)$, with respect to the S-dimensional Lebesgue measure on ${ℝ}^S$ as

$$p\left({\tilde{\beta }}_1,\dots ,{\tilde{\beta }}_S|S\right)=\frac{1}{Z_S}\text{det}\left[C^{\beta }\right]\left({\tilde{\beta }}_1,\dots ,{\tilde{\beta }}_S\right),$$ (5)

where Z_S is the normalizing constant, and det[C^β](${\tilde{\beta }}_1,\dots ,{\tilde{\beta }}_S$) is the determinant of the matrix ${\left[C_{s{s}^{\prime }}^{\beta }\right]}_{S\times S}$. Geometrically, the determinant can be interpreted as the volume of a parallelotope spanned by the column vectors of C^β. Therefore, the prior in (5) defines a repulsive point process, since equal or similar column vectors span smaller volume than very diverse ones. Specifically, $p\left({\tilde{\beta }}_1,\dots ,{\tilde{\beta }}_S|S\right)=0$, whenever ${\tilde{\beta }}_s={\tilde{\beta }}_{s^{\prime }}$ for some s ≠ s^′. In other words, it assigns vanishing values to the density at the point configurations that have replicate(s) within themselves. In this paper, we use $C^{\beta }\left({\tilde{\beta }}_s,{\tilde{\beta }}_{s^{\prime }}\right)=\psi \left({\tilde{\beta }}_s;0,{\sigma }_{\beta }^{2}I\right)\text{exp}\left\{-\frac{{\Vert {\tilde{\beta }}_s-{\tilde{\beta }}_s,\Vert }_2^2}{{\theta }_{\beta }^2}\right\}\psi \left({\tilde{\beta }}_{s^{\prime }};0,{\sigma }_{\beta }^{2}I\right)$, where θ_β is an unknown parameter that controls how repulsive the prior is; and ψ(·;μ, Σ) is the density function of a multivariate normal distribution with mean μ and variance Σ. If desired, alternative covariance functions can be implemented here without complicating the model. Similarly, we define the repulsive prior on ${\tilde{\gamma }}_{sd}$, s = 1, …, S, as follows:

$$p\left({\tilde{\gamma }}_{s1},\dots ,{\tilde{\gamma }}_{s{D}_s}|D_s\right)=\frac{1}{Z_{{\gamma }_s}}\text{det}\left[C^{{\gamma }_s}\right]\left({\tilde{\gamma }}_{s1},\dots ,{\tilde{\gamma }}_{s{D}_s}\right),$$ (6)

where $C^{{\gamma }_s}\left({\tilde{\gamma }}_{sd},{\tilde{\gamma }}_{s{d}^{\prime }}\right)=\psi \left({\tilde{\gamma }}_{sd};0,{\sigma }_{\gamma }^{2}I\right)\text{exp}\left\{-\frac{{\Vert {\tilde{\gamma }}_{sd}-{\tilde{\gamma }}_{s{d}^{\prime }}\Vert }_2^2}{{\theta }_{{\gamma }_s}^2}\right\}\psi \left({\tilde{\gamma }}_{s{d}^{\prime }};0,{\sigma }_{\gamma }^{2}I\right)$. We further assume the priors $p\left(D_s\right)\propto Z_{{\gamma }_s}/D_s!$, $p\left({\theta }_{\beta }\right)=N\left(0,{\sigma }_{{\theta }_{\beta }}^2\right),p\left({\theta }_{{\gamma }_s}\right)=N\left(0,{\sigma }_{{\theta }_{\gamma }}^2\right)$, s = 1, …, S.

We assume the priors ${\sigma }_{sp}^2~\text{IG}\left(a_{sp},b_{sp}\right)$ and ρ ~ Unif(a_ρ, b_ρ), where IG(·, ·) and Unif(·, ·) denote the Inverse Gamma and Uniform densities, respectively. For the variance of the error ϵ_ij, we assume a conjugate prior $p\left({\sigma }^2\right)~\text{IG}\left(a_{{\sigma }^2},b_{{\sigma }^2}\right)$. The elements of the parameter vector c determining non-random missingness are assigned conjugate normal priors, i.e., $p\left(c_i\right)=N\left(0,{\sigma }_{c_0}^2\right)$, i = 0, 1. Note, we do not assume a prior on the number of patient clusters S, as it complicates posterior computation if we allow both S and D_s, s = 1, …, S to be random. We will discuss how to choose S in Section 2.3.

Finally, we complete the model construction by assigning a prior to the spatial term v_ij. Let v_i = (v_i1, …, v_iJ)^′. For each patient i, we assume

$$v_i\stackrel{ \text{i}.\text{i}.\text{d}. }{~}\text{MVN}(0,\Sigma ),$$ (7)

where MVN(·, ·) denotes a multivariate normal density. Following Besag’s work,²³ we model the spatial effect by considering a conditional autoregressive (CAR) prior: $\Sigma ={\sigma }_{sp}^{2}G{(\rho )}^{-1}$, where G(ρ) = B − ρW. Here B is a diagonal matrix with the jth diagonal entry being the number of neighbors at jth site and W denotes the adjacency matrix for 168 tooth sites in the mouth structure. To construct the adjacency matrix, we consider the adjacent sites on the same tooth and sites that share a gap between teeth as “neighbors”. The adjacency structure considered in both data analysis and simulation studies is presented in Figure F2 in the Supplementary Materials (see Type I & II neighbors).

In summary, the joint model of BAREB factors as

$$\underset{(3)}{\underbrace{p\left(Y|{\left\{{\tilde{\beta }}_s\right\}}_{s=1}^S,{\left\{{\tilde{\gamma }}_{sd}\right\}}_{d=1,s=1}^{D_s,S},{\left\{v_i\right\}}_{i=1}^N,e,r,{\sigma }^2\right)}}\underset{(1)}{\underbrace{p(e|S,w)}}\underset{(2)}{\underbrace{p\left(r|e,S,{\left\{{\mathit{\varphi }}_s\right\}}_{s=1}^S\right)}}\underset{(4)}{\underbrace{p\left({\delta }_i(t)|c,{\mu }_i^{*}(t)\right)}}\times \underset{(5)}{\underbrace{p\left({\left\{{\tilde{\beta }}_s\right\}}_{s=1}^S\right)}}p\left({\sigma }^2\right)p(c)p\left({\theta }_{\beta }\right)p(S){\displaystyle \prod _{s=1}^S\underset{(6)}{\underbrace{p\left({\left\{{\tilde{\gamma }}_{sd}\right\}}_{d=1}^{D_s}\right)}}}p\left(D_s\right)p\left({\theta }_{{\gamma }_s}\right)\underset{(7)}{\underbrace{p\left({\left\{v_i\right\}}_{i=1}^N|{\sigma }_{sp}^2,\rho \right)}}p\left({\sigma }_{sp}^2\right)p(\rho ).$$

2.3 |. Posterior inference

We carry out Markov chain Monte Carlo (MCMC) simulations for posterior inference. One challenging step is to update the number of site clusters D_s nested within each patient cluster. We design a reversible jump MCMC (RJMCMC) sampler²⁴ that allows random D_s within the DPP prior for (${\tilde{\gamma }}_{s1},\dots ,{\tilde{\gamma }}_{s{D}_s}$) using the moment-matching principle²⁵ in a multivariate setting. Details of the sampling procedures are described in Appendix B of the Supplementary Materials.

To determine the patient cluster cardinality S, we use a model selection procedure based on the Watanabe-Akaike information criterion, WAIC,²⁶ instead of designing a RJMCMC sampler for computational efficiency. Compared to other popular model selection methods such as AIC, BIC, and DIC,²⁷ the WAIC based on point-wise predictive density is fast, computationally convenient, and fully Bayesian using the full posterior distribution rather than a point estimate. WAIC has been widely used for model comparison and estimating the number of mixtures in mixture models^28,29,30. WAIC estimates the expected log point-wise predictive density ($\widehat{\mathit{\text{elppd}}}$) as the measurement of model performance, defined as $\text{WAIC}=-2\widehat{\mathit{\text{elppd}}}$, where

$$\widehat{\mathit{\text{elppd}}}=\sum _{i=1}^N\left\{\text{log}\left(\frac{1}{B}\sum _{b=1}^{B}p\left(y_i|{\theta }^{(b)}\right)\right)-V_B\left(\text{log}p\left(y_i|{\theta }^{(b)}\right)\right)\right\}.$$ (8)

Here y_i = (y_i1, …, y_iJ); B is the number of post burn-in MCMC posterior samples; θ^(b) is the posterior draw of the parameter vector from the b-th iteration; and V_B represents the sample variance denoted by $V_B\left(a_b\right)=\frac{1}{B-1}{\sum }_{b=1}^B{\left(a_b-\overline{a}\right)}^2$, where $\overline{a}={\sum }_{b=1}^B{a}_b$. The first term within the braces in (8) is the log point-wise predictive density for non-missing data, which can be considered as the goodness of fit; the second term is the estimated effective number of parameters, which can be considered as the penalty term determining model complexity. We run BAREB for a set of different S values, and choose the optimal S that yields the smallest WAIC.

Another challenge in implementing BAREB is to summarize a distribution over random partitions. We follow Dahl’s approach,³¹ and report a point estimate of the biclustering. Consider an N × N matrix H, where the element $H_{i_1,i_2}=\mathbb{P}\left(e_{i_1}=e_{i_2}|\mathit{\text{data}}\right)$ represents the estimated posterior probability of patient pairs clustered together, with $\mathbb{P}(\cdot )$ being the empirical posterior mean computed based on MCMC samples. Within each MCMC iteration, the posterior sample of the patient clustering indicator e defines an N ×N clustering matrix V^e, with the element $V_{i_1,i_2}^e=I\left(e_{i_1}=e_{i_2}\right)$ defined as an indicator that patient i₁ is clustered with patient i₂. With this, we propose a least-square (LS) summary for patient clustering by minimizing the Frobenius distance between V^e and H of the posterior pairwise co-clustering probabilities, given as $e^{LS}=\text{arg\hspace{0.17em}}{\text{min}}_e\Vert V^e-H{\Vert }^2$, which is a point estimate of the patient clustering. Conditional on e^LS, we extract site-level clustering r_s from MCMC iterations in which the patient clustering indicator e is the same as e^LS. Then we compute the LS summary of the site-level clustering $r_s^{LS}$ for each patient level cluster s, through the same formulation.

The LS summary also plays a crucial role in handling the label switching problem typical to any RJMCMC implementation.³² Here, we relabel the cluster membership indicator at each iteration to match the LS summary in a post-processing step after the MCMC runs. For example, consider patient-level clustering, and let e^b be the patient cluster membership indicator drawn from the bth iteration. We assign the relabelling of e^b as $e_{\text{new}}^b$, obtained by minimizing ${\text{min}}_{e_{\text{new }}^b\in \mathcal{A}\left(e^b\right)}\text{dist}\left(e^{LS},e_{\text{new }}^b\right)$. Here, the distance dist(·, ·) is defined as the number of elements that are dissimilar in the two input vectors, and $\mathcal{A}\left(e^b\right)$ denotes the set consisting of S^b! possible relabelings for e^b, where S^b denotes the number of patient-level clusters in the bth iteration.

3 |. SIMULATION STUDY

In this section, we conduct simulation studies to evaluate the performance of BAREB by comparing the posterior inference to the simulation truth. Furthermore, to elucidate the advantages of using the repulsive DPP prior that encourages the linear coefficients in different biclusters to be diverse, we compared BAREB to an alternative model that uses independent priors on these linear coefficients.

We simulated a data matrix Y with N = 80 patients and J = 168 tooth sites, with the true number of patient clusters S₀ = 3. All 80 patients have the same probability to be assigned to the three clusters. We assumed the three patient clusters partitioned the tooth sites into (D₁₀, D₂₀, D₃₀) = (2, 3, 4) site clusters, respectively, where each tooth site was equally assigned to the two site clusters in patient cluster 1, the three site clusters in patient cluster 2, and the four site clusters in patient cluster 3. Figure 2(a) illustrates the simulated true biclustering scheme, with rows representing patients, and columns representing tooth sites. We generated three patient-level covariates, x_i = (x_i1, x_i2, x_i3) with two continuous covariates x_i1 and x_i2 generated from N(0, 3²), and one binary x_i3 generated from Bernoulli(0.5). The site-level covariate vector z_j = (1, z_j1, z_j2) was generated as z_j1 from N(0, 3²), and z_j2 from Binomial(5, 0.5). For the linear coefficients ${\left\{{\tilde{\beta }}_s\right\}}_{s=1}^{S_0}$ and ${\left\{{\tilde{\gamma }}_{sd}\right\}}_{d=1,s=1}^{D_{s0}, S_0}$, we fixed them as in Table T1 (Supplementary Materials). Conditional on e_i = s and r_sj = d, the observed response y_ij was generated from y_ij | e_i = s, $r_{sj}=d~N\left(x_i{\tilde{\beta }}_s+z_j{\tilde{\gamma }}_{sd}+v_{ij},{\sigma }_0^2\right)$, where σ₀ = 1 and v_i was generated from the CAR model with parameters ${\sigma }_{sp}^2=4$ and ρ = 0.96. In the missing model (4), we assumed c = (c₀, c₁) = (0.1, 0.2), leading to about 20% missing teeth in the simulated data.

FIGURE 2.

Simulation Illustration of biclustering: (panel a) The heatmap of biclustering in the simulation truth, (panel b) The estimated biclustering under BAREB, (panel c) the estimated biclustering under the Indep model. Different colors represent different biclusters.

We applied the proposed BAREB to the simulated dataset. The hyperparameters were set to be α = (1, …, 1), ${\alpha }_s^{\mathit{\varphi }}=(1,\dots ,1)$ for s = 1, …, S, $a_{{\sigma }^2}=b_{{\sigma }^2}=1/2$, ${\sigma }_{{\theta }_{\beta }}^2=100$, a_sp = 1, b_sp = 1, a_ρ = 0.95, b_ρ = 1, ${\sigma }_{{\theta }_{\gamma }}^2=100$, ${\sigma }_{\beta }^2={\sigma }_{\gamma }^2={10}^5$, and ${\sigma }_{c0}^2=100$. We considered S ∈ {2, …, 10}. The RJMCMC sampler was implemented with an initial burn-in of 3000 iterations, followed by B = 2000 post-burn-in iterations. A laptop computer with 2 GHz Intel Core i5 processor with 8 GB memory took around 45 minutes to run 5000 iterations. Convergence diagnostics assessed using R package coda revealed no issues. Figure F3, panel a (Supplementary Materials) plots WAIC versus S, indicating $\widehat{S}=3$, which is the same as the simulation truth. To further investigate the effectiveness of WAIC in model selection (question raised by a reviewer), we also simulated data under the ground truth S = 4. We again observe that WAIC is the least when $\widehat{S}=4$. Next, the LS summary of the posterior on e was calculated. Then, conditional on e^LS, we calculated the LS estimates of site clusters r^LS. Figure 2(b) plots the LS summary of the posterior on e and r, showing that the BAREB correctly assigns the patients to the three patient clusters. For site clusters in each patient cluster, BAREB identified ${\widehat{D}}_1=2$, ${\widehat{D}}_2=3$, and ${\widehat{D}}_3=4$, which also matches the simulation truth. As shown in Figure 2(b), BAREB assigns most of the tooth sites to their simulated true site clusters. For instance, in patient cluster 1, only four sites in site cluster 1 were misclassified; in patient cluster 3, site clusters 1 and 4 were correctly identified, while only one site in cluster 2 and two sites in cluster 3 were misclassified. Table T1 (Supplementary Materials) reports the posterior mean and the mean squared error (MSE) of the estimated coefficients, where MSE is computed as the mean squared differences between the posterior samples and the simulated true values across post-burn-in iterations. Compared to the simulation truth, BAREB can accurately estimate these coefficients with small MSE. We also plot the 95% estimated credible intervals (CI) of the coefficients in Figure 3, where the black dots represent the simulated true values. We observe that the 95% CIs are centered around the simulated true values.

FIGURE 3.

Simulation study: Posterior mean and 95% CIs for the estimated ${\left\{{\tilde{\beta }}_s\right\}}_{s=1}^{\widehat{S}}$ and ${\left\{{\tilde{\gamma }}_{sd}\right\}}_{d=1,s=1}^{{\widehat{D}}_s,\widehat{S}}$, the patient- and site-level parameters under BAREB. Black dots represent the simulated true values.

Furthermore, we conducted sensitivity analyses of the proposed BAREB model with respect to both hyperparameters and the covariance function used in the DPP prior. For the hyperparameters, we considered ${\sigma }_{{\theta }_{\beta }}^2\in \left\{50,100,200\right\}$ and ${\sigma }_{{\theta }_{\beta }}^2\in \left\{50,100,200\right\}$. For the covariance function, we applied the proposed BAREB by replacing the exponential covariance function used in generating the simulated dataset with the rational quadratic covariance function: $C_{\alpha ,k}\left(x_1,x_2\right)={\left(1+\frac{\Vert x_1-x_2{\Vert }_2^2}{2\alpha k^2}\right)}^{-\alpha }$ for both $C^{\beta }\left({\tilde{\beta }}_s,{\tilde{\beta }}_{s^{\prime }}\right)$ and $C^{{\gamma }_s}\left({\tilde{\gamma }}_{sd},{\tilde{\gamma }}_{s{d}^{\prime }}\right)$. Figure F6 (Supplementary Materials) plots the estimated LS summary of the posterior on both patient clustering and site clustering for these sensitivity analyses. We observe that the BAREB still correctly assigns the patients to the three patient clusters and most of the tooth sites to their simulated true site clusters, irrespective of the choice of the hyperparameter values, and covariance functions. We also summarized the corresponding MSEs for patient-level and site-level coefficients in Table T2 (Supplementary Materials). These values are similar across the various choices of hyperparameters and covariance functions, revealing the robustness of BAREB to the choices of hyperparameters and the covariance functions used in the DPP prior.

Next, to emphasize the advantage of the repulsive DPP prior, we conducted another study by replacing the DPP priors on ${\tilde{\beta }}_s’\text{s}$ and ${\tilde{\gamma }}_{sd}’\text{s}$ with the independent multivariate normal (MVN) priors. To be precise, for the patient-level ${\tilde{\beta }}_s,s=1,\dots ,S$, we assume independent MVN priors. For the site cluster membership indicator r_s in patient cluster s, we consider a Pólya urn prior $p\left(r_s\right)\propto {\alpha }_1^{D_s}{\prod }_{d=1}^{D^s}\Gamma \left(n_{sd}\right)$, where α₁ is the total mass parameter of the Pólya urn scheme, and n_sd is the number of tooth sites in site Patient–level coefficients Site–level coefficients in patient cluster 2 cluster d of patient cluster s. Conditional on e and r, we assume independent MVN priors on ${\tilde{\gamma }}_{sd}$, d = 1, …, D_s; s = 1, …, S. We coin this as the ‘Indep’ model. Based on the WAIC criteria, Indep identified $\widehat{S}=3$ patient clusters, which agrees with the truth. Figure 2(c) plots the LS summary of the posterior on e and r under this Indep model. We observe that although Indep assigns patients to their simulated true patient clusters, it fails to identify tooth-site clusters within the patient clusters.

Figure F4 (Supplementary Materials) compares the histograms of the posterior number of site-clusters within each patient cluster under the BAREB and Indep models. Clearly, BAREB recovers the ground truth, while the Indep overestimates the number of site clusters with substantial probability. This is a well-known phenomenon of applying Bayesian nonparametric priors to clustering¹⁶. Since Indep correctly estimates the number of patient clusters but incorrectly estimates the number of site clusters within each patient cluster, the posterior estimated cluster-specific linear coefficients ${\tilde{\beta }}_s$ and ${\tilde{\gamma }}_{sd}$ cannot be compared directly between the BAREB and Indep. Considering ${\beta }_i={\tilde{\beta }}_s$ for all i with e_i = s and ${\gamma }_{ij}={\tilde{\gamma }}_{sd}$ for all i and j, with e_i = s and r_sj = d, we computed the mean squared errors (MSE) between the posterior estimated and the simulated true linear patientand site-level coefficients, respectively, under the BAREB and Indep models. For patient-level coefficients β_i’s, the MSE’s are 7.66e-3 (BAREB) and 8.50e-3 (Indep); for the site-level coefficients γ_ij’s, the MSEs are 0.278 (BAREB) and 0.467 (Indep). Thus, we observe that the DPP is advantageous over independent priors in biclustering scenarios.

4 |. APPLICATION: GAAD DATA

The GAAD study³ was primarily designed to explore the relationship between PD and diabetes status, determined by the glycosylated haemoglobin (HbA1c) level. Excluding patients with all teeth missing, we have N = 288 patients in the dataset. We considered several patient-level covariates as potential risk factors of PD, which includes Age (in years), Gender (female=1, male=0), Smoking indicator (smoker=1, non-smoker=0), and HbA1c (high level=1, controlled=0). We also considered a site-level covariate, the jaw indicator (upper jaw=1, low jaw=0). Table T3 (Supplementary Materials) lists the patient characteristics from the dataset.

We applied BAREB to the PD dataset, considering S = {2, ⋯, 10}. The hyperparameters were set to be α = (1, …, 1), ${\alpha }_s^{\varphi }=(1,\dots ,1)$ for s = 1, …, S, $a_{c^2}=b_{o^2}=1/2$, ${\sigma }_{{\theta }_1}^2=100$, a_sp = 1, b_sp = 1, a_ρ = 0.8, b_ρ = 1, ${\sigma }_{{\theta }_1}^2=100$, and ${\sigma }_{c_0}^2=100$. For each S, we used 5,000 post burn-in samples after 10,000 iterations to compute our posterior estimates. WAIC identified $\widehat{S}=4$ patient clusters. We then computed the LS estimates e^LS and r^LS to summarize the posterior inference for biclustering. The four patient clusters had cluster sizes of 3, 174, 80, and 31 patients, respectively. The numbers of site clusters within patient clusters are 3, 2, 2, and 1. To examine the goodness-of-fit of the proposed BAREB model to the observed GAAD data, we used the posterior predictive measure of Gelman et al.,³³ with the likelihood as the discrepancy function. The resulting p-value of 0.494 indicates decent model adequacy.

Figure 4 summarizes the posterior mean and 95% credible intervals of the covariates within the four estimated patient clusters. Note, we separately plot the parameters for patient cluster 1 for visualization, since its scale is very different from the other clusters. Also, due to the small cluster size (n₁ = 3) in patient cluster 1, the estimated parameters may not be very reliable, with the 95% CIs much wider than those observed in other patient clusters, as expected. Further looking into patient cluster 1 reveals these 4 patients have at least half of the teeth missing, with an average missingness rate being 69%. Therefore, we excluded the patient cluster 1 from the following clinical interpretations. As shown in Figure 4, the effects of the patient-level covariates are quite distinct among different patient clusters. Panel (a) shows periodontal health deteriorates with age, since CIs of Age in all clusters are positive, and exclude 0. Similarly, a significant positive association was found between PD and smoking, a factor that has been believed to increase the risk of PD.³⁴ Panel (b) shows that males are more likely to have severe PD than females, confirming previous findings,³⁵ while panel (c) reveals smoking to be an important determinant of PD, mostly for patient clusters 3 and 4. From panel (d), we observe that PD has a positive correlation with diabetes (HbA1c), revealing that diabetes is possibly an important risk factor of PD.³⁶ The plots for the site-level Jaw Indicator in panel (e) implies that teeth in the upper jaw (maxilla) are more likely to develop PD. Although this finding is inconclusive, previous studies^37,38 seem to indicate that some maxillary teeth, such as the left and right central incisors and the first premolar, experience a higher rate of missingness (due to PD), than the mandibular (lower jaw) teeth. In the GAAD dataset, the tooth missingness rates in the maxilla and mandible are, respectively, 37% and 28%. Among the non-missing teeth, the mean CAL values (combining all teeth) are 2.03 and 1.83, respectively, for the maxilla and mandible.

FIGURE 4.

Posterior mean and 95% credible intervals for the parameters corresponding to the patient-level covariates: (a) Age, (b)Gender, (c) Smoking status indicator, (d) HbA1c, and site-level covariate (e) Jaw indicator. In panel (e), P1S1 denotes site cluster 1 in patient cluster 1, P1S2 denotes site cluster 2 in patient cluster 1, etc.

Next, we report site clustering in patient clusters 2 and 3. Patient cluster 1 is excluded owing to its small cluster size, while patient cluster 4 is excluded as there is only one site cluster from the LS estimate. Figure 5 (panels a and c) display the ordinal CAL values from two randomly selected patients in patient clusters 2 and 3, respectively, based on the classification of the American Association of Periodontology.³⁹ The five ordinal categories are (i) no PD (CAL 0–1 mm), (ii) slight PD (CAL 1–2 mm), (iii) moderate PD (CAL 3–4 mm), (iv) severe PD (CAL 5 ≥ 5mm), and (v) missing (if the tooth is missing). Figure 5 (panels b and d) plot the heatmaps of the estimated posterior probability of each pair of sites being clustered together, as described in subsection 2.3. BAREB estimates distinct site clustering patterns in different patient clusters. In particular, for the randomly selected patient in patient cluster 2, two maxillary molars and four mandibular molars are missing. From the corner dark squares in Figure 5 (panel b), these missing molar sites are clustered together. For example, the missing maxillary molar sites 1–6 and 79–84 are clustered, along with the mandibular missing sites 157–168. Also, the four rectangular black patches reveal tooth sites of the same type are more likely to be clustered together. On the contrary, for the randomly selected patient from patient cluster 3, the corresponding plot (panel d) produces a checkerboard pattern, where the missing sites at different site locations tend to cluster with a high probability.

FIGURE 5.

Site-level clustering in GAAD data. Panels (a) and (b) plot the ordinal CAL levels, and the heatmap of the posterior clustering probability of tooth-sites for a randomly selected patient from patient cluster 2. Panels (c) and (d) plot the same, respectively, now for a randomly selected patient from patient cluster 3.

Then, in Figure 6, we report the posterior predictive probabilities of the aforementioned ordinal CAL categories. For illustration, we consider a specific tooth-site (# 120) from the mandibular incisor of a hypothetical patient with mean age 55.27 years old, under all possible combinations of gender, smoking and HbA1c levels, where F, M, N, S, L, and H denote female, male, non-smoker, smoker, controlled HbA1c, and high HbA1c, respectively. Considering a tooth-site with no, or slight PD as ‘healthy’, the probability being healthy is 0.561 for a female non-smoker with controlled HbA1c, while the probability is 0 for a male smoker with high HbA1c. Specifically, females have higher probability of having healthy teeth than males (for example, FNL = 0.561, versus MNL = 0.461); smokers have lower probability than nonsmokers (FSL = 0.542 versus FNL = 0.561); controlled HbA1c have higher probability of having no PD than high HbA1c (FNL = 0.423 versus MNH = 0.000), and so on. Figure F5 (Supplementary Materials) presents the density histogram plot of the CAL response, overlaid with the fitted curve generated from the marginal posteriors and Silverman’s rule-of-thumb smoothing bandwidth. The proposed BAREB model induces a marginal density capable of accommodating the possible non-Gaussian features of the CAL. Thus, in lieu of non-Gaussian parametric (say, skew-t), and semiparametric assumptions for the error term ϵ_ij that may induce computational issues, we consider our N(0, σ²) assumption to be adequate. We also present the posterior means and 95% credible intervals for other remaining parameters in Table T4 (Supplementary Materials). It is worth noting that estimate of c₁ is positive and significant, which unsurprisingly implies that higher values of CAL leads to higher probability of missing teeth. The estimate of ρ, the spatial association parameter in the CAR model, is 0.843, which corresponds to moderate spatial correlation based on the calibration of ρ.⁴⁰

FIGURE 6.

Posterior predictive probabilities of ordinal CAL categories for site # 120 of a mandibular incisor from a hypothetical patient with age 55.27 years, under various combinations of gender, smoking and HbA1c levels.

5 |. CONCLUSION

Our proposed BAREB can detect simultaneous clustering patterns among PD study patients and tooth sites, factoring in both patient- and site-level covariates, the spatial dependence among tooth sites, and possible nonrandom missingness patterns. That way, our proposal improves upon available clustering techniques into learning the heterogeneity in PD among patients with distinct disease patterns. We also demonstrate the advantages of using the DPP prior over independent priors for quantifying diversity among various biclusters, balancing parsimony and interpretation. In addition, BAREB is readily implementable via R, and can be a welcome addition to a user’s toolbox.

Although motivated from an oral health application, BAREB provides a general framework for inference on biclustering in many other applications involving a data matrix and covariates. For example, in gene expression data (where rows and columns represent genes and patient samples, respectively), BAREB can discover functionally related genes under different subsets of patients, factoring in various clinical covariates. Furthermore, the choice of a parametric selection model to tackle non-random missingness was also due to computational reasons, leaving us no opportunity to introduce sensitivity parameters for assessing non-random missingness.⁴¹ All these are viable areas for future research, and will be pursued elsewhere.