Bayesian Clustering via Fusing of Localized Densities

Abstract

Bayesian clustering typically relies on mixture models, with each component interpreted as a different cluster. After defining a prior for the component parameters and weights, Markov chain Monte Carlo (MCMC) algorithms are commonly used to produce samples from the posterior distribution of the component labels. The data are then clustered by minimizing the expectation of a clustering loss function that favors similarity to the component labels. Unfortunately, although these approaches are routinely implemented, clustering results are highly sensitive to kernel misspecification. For example, if Gaussian kernels are used but the true density of data within a cluster is even slightly non-Gaussian, then clusters will be broken into multiple Gaussian components. To address this problem, we develop Fusing of Localized Densities (FOLD), a novel clustering method that melds components together using the posterior of the kernels. FOLD has a fully Bayesian decision theoretic justification, naturally leads to uncertainty quantification, can be easily implemented as an add-on to MCMC algorithms for mixtures, and favors a small number of distinct clusters. We provide theoretical support for FOLD including clustering optimality under kernel misspecification. In simulated experiments and real data, FOLD outperforms competitors by minimizing the number of clusters while inferring meaningful group structure. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.

1. Introduction

Clustering data into groups of relatively similar observations is a canonical task in exploratory data analysis. Algorithmic clustering methods, such as k-means, k-medoids, and hierarchical clustering, rely on dissimilarity metrics, an approach which is often heuristic but may perform well in practice; see Hastie, Tibshirani, and Friedman (2009), Jain (2010), and Kiselev, Andrews, and Hemberg (2019) for an overview. In comparison, model-based clustering methods use mixtures of probability kernels to cluster data, ordinarily by inferring the component labels (Fraley and Raftery 2002). Choices of kernel depend on the type of data being considered, with the Gaussian mixture model (GMM) particularly popular for continuous data. A conceptual advantage of model-based methods is the ability to express uncertainty in clustering. For example, the expectation-maximization (EM) algorithm uses maximum likelihood estimates of the component-specific parameters and weights to calculate each observation’s posterior component allocation probabilities, which are interpreted as a measure of clustering uncertainty (Bensmail et al. 1997). As estimation of the weights and component-specific parameters is crucial for model-based clustering, there is a rich literature on quantifying rates of convergence for various mixture models, which is usually expressed in terms of the mixing measure (see Guha, Ho, and Nguyen 2021 and references therein).

Bayesian mixture models have received increased attention as a clustering method in recent years. The Bayesian framework can account for uncertainty in the number of components and incorporate prior information on the component-specific parameters, with Markov chain Monte Carlo (MCMC) algorithms employed to generate posterior samples for the mixture weights, kernel parameters, and component labels for each data point. Based on posterior samples of the component labels, one can obtain Monte Carlo approximations to Bayes clustering estimators. For $p$-dimensional data $\mathit{X}=\left(X_1,\dots ,X_n\right)$, a Bayes estimator ${\mathit{c}}^{*}$ is the minimizer of an expected loss function conditional on $\mathit{X}$ over all possible clusterings $\mathit{c}=\left(c_1,\dots ,c_n\right)$. There are several popular choices of loss, including Binder’s (Binder 1978) and the Variation of Information (VI) (Meilă 2007), which are distance metrics over the space of partitions and favor clusterings that are similar to the component labels. Further details on set partitions and their role in Bayesian clustering can be found in Meilă (2007), Wade and Ghahramani (2018), and Paganin et al. (2021). Several measures of uncertainty in clustering with Bayesian mixtures exist, including the posterior similarity matrix and credible balls of partitions (Wade and Ghahramani 2018).

However, model-based clustering approaches, including Bayesian implementations, can be brittle in applications due to unavoidable impacts of model misspecification. While the data are assumed to come from a model $X_i~f={\sum }_{k=1}^K{a}_{k}g\left(\cdot ;{\tilde{\theta }}_k\right)$, in reality, data are generated via a true process $f^0$, which may or may not be in the support of the prior for $f$. A specific example is kernel misspecification, which occurs when $f^0$ is in fact a mixture model whose components are not contained in the kernel family mixed over by the fitted model. Mixtures will often induce over-clustering in this setting by approximating a single component in $f^0$ with many components from $f$. An example of this phenomena is shown in Figure 1, where a 10 component Bayesian GMM is fit to data generated from a mixture of bivariate skew Gaussian distributions. Despite using a concentration parameter of 1/10 in the symmetric Dirichlet prior to induce a small number of clusters (Rousseau and Mengersen 2011), the GMM allocates the data into 10 poorly defined groups. A group of observations in the bottom third of the sample space is split into two, while several observations are placed into their own groups despite being near dense collections of data.

Figure 1.

A Bayesian GMM with 10 components and Dirichlet prior concentration parameter equal to 1/10 is fit to data generated from a mixture of skew Gaussian distributions.

Several approaches have been proposed to address the issue of over-clustering due to kernel misspecification. A natural solution is to define a flexible class of kernels, exemplified by the mixtures in Karlis and Santourian (2009), O’Hagan et al. (2016), and Dang et al. (2023). To increase flexibility further, Rodríguez and Walker (2014) propose a mixture of nonparametric unimodal kernels. Similarly, Bartolucci (2005), Li (2005), Di Zio, Guarnera, and Rocci (2007), and Malsiner-Walli, Frühwirth-Schnatter, and Grün (2017) use carefully chosen mixtures of Gaussians to characterize the data within each cluster. However, there is an unfortunate pitfall with the general strategy of using flexible families of kernels. In particular, as the flexibility of the kernel increases, identifiability and optimal estimation rates for inferring the mixing measure tend to weaken, especially when the true number of mixture components is unknown (Nguyen 2013; Ho and Nguyen 2016a; Heinrich and Kahn 2018). Even the transition from location Gaussian kernels with known covariance to location-scale Gaussian kernels can have substantial consequences on the convergence rate of the component means (Manole and Ho 2022). Such problems motivated Ho, Nguyen, and Ritov (2020) to propose an alternative estimator of the mixing measure that is more robust than maximum likelihood and Bayesian approaches and can achieve optimal convergence rates.

Alternatively, one can develop generalized Bayesian methods of clustering that avoid defining a fully generative probabilistic model for the data. For example, Duan and Dunson (2021) propose to conduct model-based clustering based on a pairwise distance matrix instead of the data directly to reduce sensitivity to kernel misspecification. Rigon, Herring, and Dunson (2023) instead define a Gibbs posterior for clustering, incorporating a clustering loss function in place of a likelihood function, completely bypassing modeling of the data. An alternative that maintains a fully generative model while robustifying inferences to misspecification is to use a coarsened posterior (Miller and Dunson 2019; Gorsky, Chan, and Ma 2024). This coarsening approach often has good practical performance in reducing over-clustering due to kernel misspecification.

Unfortunately, these approaches for accommodating kernel misspecification have various drawbacks. Along with slower convergence rates, flexible kernels typically require a large number of parameters, worsening the already burdensome computational cost of Bayesian clustering. The generalized Bayes approaches can perform well in certain settings. However, both Gibbs posteriors and coarsened posteriors are highly sensitive to key tuning parameters, which can be difficult to choose objectively in practice.

A key theoretical advance is given in Aragam et al. (2020), which attempts to solve the problem of clustering based on a mixture model by merging components in a Gaussian mixture. They rely on a two-stage procedure that lacks partition uncertainty quantification and assumes that the true number of kernels is known. However, the approach of viewing clusters as arising from merging closely overlapping kernels is promising. Related merging ideas have been implemented in both frequentist and Bayesian approaches in a variety of settings, and several algorithms exist for deciding how and when to combine components together (Chan et al. 2008; Baudry et al. 2010; Hennig 2010; Melnykov 2016; Guha, Ho, and Nguyen 2021; Manole and Khalili 2021).

In this article, we propose a novel decision theoretic method for Bayesian clustering that mitigates the effects of model misspecification. Suppose we model the data with a Bayesian mixture model, with $K$ components, component labels $\mathit{s}=\left(s_1,\dots ,s_n\right)$, component-specific atoms ${\tilde{\theta }}_1,\dots ,{\tilde{\theta }}_K$, and kernels $g\left(\cdot ;{\tilde{\theta }}_1\right),\dots ,g\left(\cdot ;{\tilde{\theta }}_K\right)$. Rather than focusing on $\mathit{s}$, we compute clusters with the localized densities ${\left\{g\left(\cdot ;{\theta }_i\right)\right\}}_{i=1}^n$, defined by ${\theta }_i={\tilde{\theta }}_{s_i}$. We define a loss function for any clustering $\widehat{\mathit{c}}$ that favors allocating $i$ and $j$ to the same cluster when the statistical distance between $g\left(\cdot ;{\theta }_i\right)$ and $g\left(\cdot ;{\theta }_j\right)$ is small, encouraging grouping observations with overlapping component kernels. We cluster the data with a Bayes estimator, interpreted as a Fusing of Localized Densities (FOLD). Our method has a fully decision theoretic justification, leads to interpretable uncertainty quantification, and can be readily implemented using the output of existing MCMC algorithms for mixtures.

Though previous methods have used merging kernels to account for kernel misspecification, to our knowledge none have a formal Bayesian decision theoretic justification. Although FOLD requires combinatorial optimization when obtaining the point estimate, we suggest two reasonable approximations: one in which we reduce the optimization space to a smaller set of candidate clusterings, and another in which we use existing greedy algorithms to compute a locally-optimal solution. We also provide concrete theoretical guarantees for our procedure, including the result that as the sample size grows, the clustering point estimate converges to an oracle clustering rule. In addition, we list intuitive examples of oracle rules and empirically validate our asymptotic theory using a simple example.

In Section 2, we explain our clustering method from a Bayesian decision theoretic perspective, provide a framework for uncertainty quantification, and demonstrate how to implement FOLD in practice. In Section 3, we show asymptotic concentration of FOLD for misspecified and well-specified regimes. We apply FOLD to a cell line dataset in Section 4, showing excellent performance. Finally, we provide concluding remarks and some extensions in Section 5. The supplementary material contains additional simulations and real-data examples, as well as proofs for all theoretical results.

2. Clustering with Localized Densities

2.1. Notation and Setup

Let $X_i^0=\left(X_{i1}^0,\dots ,X_{ip}^0\right)\in {\mathbb{R}}^p$ be multivariate observations, collected into ${\mathit{X}}^0=\left(X_1^0,\dots ,X_n^0\right)$. Assume that $X_1^0,\dots ,X_n^0$ are random and generated from an unknown mixture model: $X_i^0~f^0$, where $f^0={\sum }_{m=1}^{M^0}{a}_m^0{g}_m^0(\cdot ),a_m^0>0$ for $m=1,\dots ,M^0,{\sum }_{m=1}^{M^0}{a}_m^0=1$, and $M^0<\infty$. Since $f^0$ is a mixture, the data-generating process can be equivalently stated with the addition of latent variables $s_i^0\in \left\{1,\dots ,M^0\right\}$, so that $\left(X_i^0\mid s_i^0=m\right)~g_m^0(\cdot )$ for all $i=1,\dots ,n$. ${\mathbb{P}}^0$ refers to probability and convergence statements with respect to the true data generating process, with ${\mathbb{P}}^0\left(A\right)={\int }_A{f}^0\left(x\right)dx$ for any set $A$.

Throughout this article, we will focus on the setting in which ${\mathit{X}}^0$ is modeled with a mixture. Let $X_i=\left(X_{i1},\dots ,X_{ip}\right)\in {\mathbb{R}}^p$, where $\mathit{X}=\left(X_1,\dots ,X_n\right)$. A Bayesian mixture model assumes that $\mathit{X}$ are generated via

$$\lambda ~{\pi }_{\Lambda },{\theta }_i\left|\lambda ~\lambda ,X_i\right|{\theta }_i~g\left(X_i;{\theta }_i\right);$$ (1)

for all $i=1,\dots ,n$. Here, $\lambda$ is a probability distribution over a parameter space $\Theta$ and known as the mixing measure, whereas $𝒢=\left\{g\right(\cdot ;\theta ):\theta \in \Theta \}$ denotes a family of probability densities with support on ${\mathbb{R}}^p$. We will impose more conditions on both $\Theta$ and $𝒢$ when we develop our theoretical results in Section 3. ${\pi }_{\Lambda }$ is the prior distribution for the mixing measure with support $\Lambda$, and examples of such priors are implemented in Bayesian finite mixture models, mixtures of finite mixtures (MFMs) (Miller and Harrison 2018), Dirichlet process mixtures (DPMs) (Ferguson 1973; Antoniak 1974), and Pitman-Yor process mixtures (Ishwaran and James 2001, 2003). Generally, realizations from ${\pi }_{\Lambda }$ are almost-surely discrete (e.g., as in Sethuraman 1994) and can be represented as $\lambda =\sum _{k=1}^K{a}_k{\delta }_{{\tilde{\theta }}_k}$, where $1\le K\le \infty ,{\tilde{\theta }}_k\in \Theta ,0<a_k<1$, and $\sum _{k=1}^K{a}_k=1$. Hence, sampling ${\theta }_i$ from (1) results in ties amongst the atoms $\mathit{\theta }=\left({\theta }_1,\dots ,{\theta }_n\right)$, which are represented by unique values $\tilde{\mathit{\theta }}=\left({\tilde{\theta }}_1,\dots ,{\tilde{\theta }}_{K_n}\right)$ for some $1\le K_n\le n$. From these ties we can construct a latent partition $S=\left(S_1,\dots ,S_{K_n}\right)$ of the integers $\left[n\right]=\{1,\dots ,n\}$, where $S_k=\left\{i\in \left[n\right]:{\theta }_i={\tilde{\theta }}_k\right\}$. The latent partition may also be represented with labels $\mathit{s}=\left(s_1,\dots ,s_n\right)$, where $s_i=k⟺i\in S_k$. We will assume that the prior for $\tilde{\mathit{\theta }}$ is non-atomic, so the allocation of two observations to the same component is equivalent to equality in their atoms, that is $s_i=s_j⟺{\theta }_i={\theta }_j$.

Ultimately, our goal is to cluster the data ${\mathit{X}}^0$ using the mixture model outlined in (1). The typical approach is to infer the conditional distribution of $S$ after observing the data. More formally, by assuming that $\lambda ~{\pi }_{\Lambda }$, we have induced a prior distribution for $S$, $\Pi \left(S\right)$. We then cluster the data by inferring $\Pi (S\mid \mathit{X}={\mathit{X}}^0$), which we will refer to for the remainder of this article as $\Pi \left(S\mid {\mathit{X}}^0\right)$. To construct a point estimate of $S$, we minimize the posterior expectation of a clustering loss function that favors similarity to $S$. The decision theoretic approach to Bayesian clustering was first proposed in Binder (1978), and was advocated for more recently in Wade and Ghahramani (2018) and Dahl, Johnson, and Müller (2022). However, Bayesian methods can lead to homogenous, tiny, and often uninterpretable clusters in practice. These issues often arise in the presence of kernel misspecification. Even well-separated true clusters in ${\mathit{X}}^0$ will be broken into multiple components from $𝒢$ under a minor degree of kernel misspecification. To address this issue, we propose a loss function focused on simplifying the overfitted kernels via merging rather than estimating the component labels.

2.2. Decision Theory Formulation

We aim to estimate a clustering $\widehat{C}=\left\{{\hat{C}}_1,\dots ,{\hat{C}}_{{\hat{K}}_n}\right\}$ of ${\mathit{X}}^0$, or labels $\widehat{\mathit{c}}=\left({\hat{c}}_1,\dots ,{\hat{c}}_n\right)$, based on merging components from the joint posterior that have similar kernels $g\left(\cdot ;{\tilde{\theta }}_k\right)$. The motivation here is to remedy the cluster splitting that occurs in using multiple parametric kernels $g\left(\cdot ;{\tilde{\theta }}_k\right)$ to represent each “true” kernel $g_m^0(\cdot )$. We define the localized density of observation $i$ to be $g\left(\cdot ;{\theta }_i\right)$, a random probability density that takes values $g\left(x;{\theta }_i\right)$ for all $x\in {\mathbb{R}}^p$, where ${\theta }_i$ is the atom of observation $i$ as defined in (1). Observe that the localized density is the distribution for $X_i$ under the fitted mixture model given ${\theta }_i$, that is $\left(X_i\mid {\theta }_i={\tilde{\theta }}_k\right)~g\left(\cdot ;{\tilde{\theta }}_k\right)$. Prior variation in the localized densities is modeled by sampling $\lambda ~{\pi }_{\lambda }$, and then sampling ${\theta }_i~\lambda$, for $i=1,\dots ,n$.

To counteract cluster splitting, we define the loss of assigning two observations into the same or different clusters as a function of the statistical distance between their localized densities. Figure 2 shows the behavior of the posterior distributions of the localized densities when the model $f\left(x\right)=\sum _{k=1}^{30}{a}_k{𝒩}_2\left(x;{\tilde{\theta }}_k,0.02I\right)$ is fit to a version of the moons data. The red points plot ${\mathbb{E}}_{\Pi }\left({\theta }_i\mid {\mathit{X}}^0\right)$ for each $i=1,\dots ,n$, with gray circles representing the 95% high density regions of a ${𝒩}_2\left({\mathbb{E}}_{\Pi }\left({\theta }_i\mid {\mathit{X}}^0\right),0.02I\right)$ distribution. The crescent clusters are split into multiple overlapping Gaussian kernels, but are recovered by fusing these kernels together.

Figure 2.

A Bayesian location Gaussian mixture is fit to a version of the moons dataset. The localized densities are inferred, then merged together to recover the crescents.

We define the loss of any clustering $\widehat{\mathit{c}}$ resulting from the model in (1) to be

$$ℒ(\widehat{\mathit{c}},\mathit{\theta })=\sum _{i<j}\left\{{\mathbf{1}}_{{\hat{c}}_i={\hat{c}}_j}{𝒟}_{ij}+\omega {\mathbf{1}}_{{\hat{c}}_i\ne {\hat{c}}_j}\left(1-{𝒟}_{ij}\right)\right\},$$ (2)

where $\omega >0,{𝒟}_{ij}=d\left\{g\left(\cdot ;{\theta }_i\right),g\left(\cdot ;{\theta }_j\right)\right\}$, and $d$ is any distance between probability distributions assumed to be bounded in the unit interval, that is $0\le d\left\{P\right(\cdot ),Q(\cdot \left)\right\}\le 1$ for all measures $P$ and $Q$, and with the property that $d\left\{P\right(\cdot ),Q(\cdot \left)\right\}=1$ if and only if $\text{supp}\left(P\right)\cap \text{supp}\left(Q\right)=\varnothing$. A key example is when $d$ is chosen to be the Hellinger distance, but this need not be the case in general. Observe that $ℒ(\widehat{\mathit{c}},\mathit{\theta })$ is nonnegative for any $\widehat{\mathit{c}}$. The loss of assigning ${\hat{c}}_i={\hat{c}}_j$ is ${𝒟}_{ij}$. If $s_i=s_j$, then ${\theta }_i={\theta }_j$ and hence there is no loss incurred. When $s_i\ne s_j$, then ${\theta }_i\ne {\theta }_j$ but the loss will remain small when the kernels $g\left(\cdot ;{\theta }_i\right),g\left(\cdot ;{\theta }_j\right)$ are similar under $d$. Conversely, allocating $i$ and $j$ to different clusters results in a loss of $\omega \left(1-{𝒟}_{ij}\right)$. If ${𝒟}_{ij}=1$, that is if the supports of $g\left(\cdot ;{\theta }_i\right)$ and $g\left(\cdot ;{\theta }_j\right)$ are disjoint, then setting ${\hat{c}}_i\ne {\hat{c}}_j$ accumulates zero loss. Otherwise, the loss depends on the separation in $d$ between the localized densities and the loss parameter $\omega$.

Our loss $ℒ(\widehat{\mathit{c}},\mathit{\theta })$ is invariant to permutations of the data indices and of the labels in either $\widehat{\mathit{c}}$ or $\mathit{s}$, which is desirable for clustering losses (Binder 1978). In addition, $ℒ(\widehat{\mathit{c}},\mathit{\theta })$ is a continuous relaxation of Binder’s loss function (Binder 1978; Lau and Green 2007),

$${ℒ}_{\mathrm{B}}(\widehat{\mathit{c}},\mathit{s})=\sum _{i<j}\left\{{\mathbf{1}}_{{\hat{c}}_i={\hat{c}}_j}{\mathbf{1}}_{s_i\ne s_j}+\omega {\mathbf{1}}_{{\hat{c}}_i\ne {\hat{c}}_j}{\mathbf{1}}_{s_i=s_j}\right\}.$$ (3)

A key property of Binder’s loss is that it is a quasimetric over the space of partitions (Wade and Ghahramani 2018; Dahl, Johnson, and Müller 2022). $ℒ(\widehat{\mathit{c}},\mathit{\theta })$ is not a quasimetric, but instead aims to mitigate the problem of cluster splitting arising under kernel misspecification. Note the difference in notation in the arguments of (3) and (2): we omit $\mathit{s}$ from the notion for $ℒ(\widehat{c},\mathit{\theta })$ to reflect that we will incorporate information from the atoms in clustering ${\mathit{X}}^0$, not just the latent partition. However, we can also state (3) in terms of $\mathit{\theta }$ by recalling that $s_i=s_j⟺{\theta }_i={\theta }_j$. One can show that $ℒ(\widehat{\mathit{c}},\mathit{\theta })$ can be rewritten as the sum of ${ℒ}_{\mathrm{B}}(\widehat{\mathit{c}},\mathit{s})$ and a remainder term $ℬ(\widehat{\mathit{c}},\tilde{\mathit{\theta }})$ that depends on the unique values $\tilde{\mathit{\theta }}$, and that $ℒ(\mathit{s},\mathit{\theta })=0$ only when the components of $f$ are completely separated under $d$. The closed form of the remainder $ℬ(\widehat{\mathit{c}},\tilde{\mathit{\theta }})$ is given in the supplement, and is interpreted as an added cost to Binder’s loss that encourages merging components.

Proposition 1. $ℒ(\widehat{\mathit{c}},\mathit{\theta })={ℒ}_{\mathrm{B}}(\widehat{\mathit{c}},\mathit{s})+ℬ(\widehat{\mathit{c}},\tilde{\mathit{\theta }})$, where $ℬ(\mathit{s},\tilde{\mathit{\theta }})=0$ if and only if $d\left\{g\left(\cdot ;{\tilde{\theta }}_k\right),g\left(\cdot ;{\tilde{\theta }}_{k^{\prime }}\right)\right\}=1$ for all component pairs $k\ne k^{\prime }=1,\dots ,K$.

The parameter $\omega$ calibrates separation of the clusters. For example, suppose we compare the clustering ${\widehat{\mathit{c}}}_1$, which includes clusters ${\hat{C}}_h$ and ${\hat{C}}_{h^{\prime }}$, with the clustering ${\widehat{\mathit{c}}}_2$, which is equivalent to ${\widehat{\mathit{c}}}_1$ but now contains the merged cluster ${\hat{C}}_h\cup {\hat{C}}_{h^{\prime }}$. The difference in their losses is $ℒ\left({\widehat{\mathit{c}}}_1,\mathit{\theta }\right)-ℒ\left({\widehat{\mathit{c}}}_2,\mathit{\theta }\right)=\omega \sum _{i\in {\hat{C}}_{h,j}\in {\hat{C}}_{h^{\prime }}}\left(1-{𝒟}_{ij}\right)-\sum _{i\in {\hat{C}}_h,j\in {\hat{C}}_{h^{\prime }}}{𝒟}_{ij}$. The loss of ${\widehat{\mathit{c}}}_2$ is less than that of ${\widehat{\mathit{c}}}_1$ when

$$\frac{1}{\left|{\hat{C}}_h\right|\left|{\hat{C}}_{h^{\prime }}\right|}\sum _{i\in {\hat{C}}_h,j\in {\hat{C}}_{h^{\prime }}}{𝒟}_{ij}<\gamma :=\frac{\omega }{1+\omega }.$$ (4)

This implies that large values of $\omega$ promote fusing clusters, while smaller values lead to more clusters having less between-cluster heterogeneity. When ${\widehat{\mathit{c}}}_1=\mathit{s}$, it is clear that a smaller loss can be attained by merging components with average pairwise statistical distance less than $\gamma$. Trivial clusterings are favored when $\omega$ is taken to its lower and upper limits, similar to both Binder’s loss and the VI loss (Wade and Ghahramani 2018; Dahl, Johnson, and Müller 2022). As $\omega \to 0$, our loss is minimized by placing each observation in its own cluster, and, as $\omega \to \infty$, all observations are placed in a single cluster. Binder’s loss exhibits a similar interpretation, and one can show that, under the same settings outlined above, ${ℒ}_{\mathrm{B}}\left({\widehat{\mathit{c}}}_2,\mathit{s}\right)<{ℒ}_{\mathrm{B}}\left({\widehat{\mathit{c}}}_1,\mathit{s}\right)$ when ${\left|{\hat{C}}_h\right|}^{-1}{\left|{\hat{C}}_{h^{\prime }}\right|}^{-1}\sum _{i\in {\hat{C}}_h,j\in {\hat{C}}_{h^{\prime }}}{1}_{s_i\ne s_j}={\left|{\hat{C}}_h\right|}^{-1}{\left|{\hat{C}}_{h^{\prime }}\right|}^{-\mathbf{1}}{m}_{h{h}^{\prime }}<\gamma$, where $m_{h{h}^{\prime }}$ counts the number of pairs across clusters $h$ and $h^{\prime }$ that are allocated to different mixture components. However, the loss is always minimized when $\widehat{\mathit{c}}=\mathit{s}$.

The risk of any clustering $\widehat{\mathit{c}}$ is the posterior expectation of its loss, which integrates over the uncertainty in $\mathit{\theta }$ after observing the data;

$$\begin{gathered} ℛ\left(\widehat{\mathit{c}}\right)={\mathbb{E}}_{\Pi }\left\{ℒ(\widehat{\mathit{c}},\mathit{\theta })\mid {\mathit{X}}^0\right\} \\ =\sum _{i<j}\left\{{\mathbf{1}}_{{\hat{c}}_i={\hat{c}}_j}{\Delta }_{ij}+\omega {\mathbf{1}}_{{\hat{c}}_i\ne {\hat{c}}_j}\left(1-{\Delta }_{ij}\right)\right\}, \end{gathered}$$ (5)

where ${\Delta }_{ij}={\mathbb{E}}_{\Pi }\left({𝒟}_{ij}\mid {\mathit{X}}^0\right)$ is the posterior expected distance between the localized densities $g\left(\cdot ;{\theta }_i\right)$ and $g\left(\cdot ;{\theta }_j\right)$ for observations $i$ and $j$, respectively. The point estimator of the clustering of ${\mathit{X}}^0$ is given by a Bayes estimator, denoted ${\mathit{c}}_{\text{FOLD}}$, which is the minimizer of (5) over the space of all possible cluster allocations, that is ${\mathit{c}}_{\text{FOLD}}=\underset{\widehat{\mathit{c}}}{\text{argmin}}ℛ\left(\mathit{c}\right)$. The expected statistical distance terms in the risk (5) exhibit several bounded and metric properties.

Proposition 2. Let $\Delta ={\left({\Delta }_{ij}\right)}_{i,j}$, where ${\Delta }_{ij}={\mathbb{E}}_{\Pi }\left({𝒟}_{ij}\mid {\mathit{X}}^0\right)$. Then $\Delta$ satisfies the following with ${\mathbb{P}}^0$-probability equal to 1 for all $i,j,l=1,\dots ,n$: (a) $0\le {\Delta }_{ij}\le 1$; (b) ${\Delta }_{ii}=0,{\Delta }_{ji}={\Delta }_{ij}$, and ${\Delta }_{il}\le {\Delta }_{ij}+{\Delta }_{jl}$; and (c) ${\Delta }_{ij}\le \Pi \left(s_i\ne s_j\mid {\mathit{X}}^0\right)$.

Properties (a) and (b) follow naturally from the assumptions we have made regarding the statistical distance $d$. To show (c), recall that, conditional on $\lambda =\sum _{k=1}^K{a}_k{\delta }_{{\tilde{\theta }}_k}$, variation in ${\theta }_i$ and ${\theta }_j$ is explained by sampling from $\lambda$ in (1). This implies that ${\mathbb{E}}_{\Pi }\left({𝒟}_{ij}\mid {\mathit{X}}^0,\lambda \right)=\sum _{k\ne k^{\prime }}d\left\{g\left(\cdot ;{\tilde{\theta }}_k\right),g\left(\cdot ;{\tilde{\theta }}_{k^{\prime }}\right)\right\}\Pi \left(s_i=k,s_j=k^{\prime }\mid {\mathit{X}}^0,\lambda \right)$. Since $0\le d(\cdot ,\cdot )\le 1$, we have that ${\mathbb{E}}_{\Pi }\left({𝒟}_{ij}\mid {\mathit{X}}^0,\lambda \right)\le \sum _{k\ne k^{\prime }}\Pi \left(s_i=k,s_j=k^{\prime }\mid {\mathit{X}}^0,\lambda \right)=\Pi \left(s_i\ne s_j\mid {\mathit{X}}^0,\lambda \right)$. Taking the expectation with respect to the posterior of $\lambda$ on both sides of the inequality results in (c). Properties (a) and (b) imply that $ℛ\left(\widehat{\mathit{c}}\right)$ is nonnegative for any clustering and that the sum in (5) may be taken over the indices $i<j$. Property (c) states that FOLD will generally favor a fewer number of clusters than Binder’s loss. One can show that Binder’s loss prefers assigning $i$ and $j$ to the same cluster when $\Pi \left(s_i\ne s_j\mid {\mathit{X}}^0\right)<\gamma$. If ${\Delta }_{ij}<\gamma <\Pi \left(s_i\ne s_j\mid {\mathit{X}}^0\right)$, FOLD will disagree with Binder’s loss, with FOLD preferring to merge $i$ and $j$ into the same cluster, implying that ${\mathit{c}}_{\text{FOLD}}$ is often more coarse then ${\mathit{c}}_{\mathrm{B}}=\underset{\widehat{\mathit{c}}}{\text{argmin}}{\mathbb{E}}_{\Pi }\left\{{ℒ}_{\mathrm{B}}(\widehat{\mathit{c}},\mathit{s})\mid {\mathit{X}}^0\right\}$.

Recently, the Variation of Information (VI) (Meilă 2007) has attracted attention in the literature for use as a clustering loss function (Wade and Ghahramani 2018; De Iorio et al. 2023; Denti et al. 2023), and we compare FOLD to the VI from a methodological point of view in the Supplement. The VI is primarily motivated from an information theoretic perspective, but it, like Binder’s loss, is a metric on the partition space (Meilă 2007; Dahl, Johnson, and Müller 2022) and encourages similarity to the component labels when used in Bayesian clustering. While the VI has been observed to yield fewer clusters than Binder’s loss empirically on synthetic and real data (Wade and Ghahramani 2018), we show in a simulation study in the Supplementary and an application in Section 4 that ${\mathit{c}}_{\text{VI}}$ is still vulnerable to the over-clustering problem, albeit usually to a lesser degree than ${\mathit{c}}_{\mathrm{B}}$, whereas ${\mathit{c}}_{\text{FOLD}}$ is more robust.

2.3. Uncertainty Quantification

In applications of Bayesian clustering to real data, we often find that there is substantial uncertainty in the component labels a posteriori. To avoid overstating the significance of ${\mathit{c}}_{\text{FOLD}}$, it is thus important to express this uncertainty in a clear and interpretable manner. We focus on the clustering that minimizes (2), that is ${\mathit{c}}_{\mathit{\theta }}=\underset{\widehat{\mathit{c}}}{\text{argmin}}ℒ(\widehat{\mathit{c}},\mathit{\theta })$.

Observe that ${\mathit{c}}_{\mathit{\theta }}$ depends on $\omega$ and may not be equal to $\mathit{s}$. To see this, we recall (4), which shows that we can acquire a clustering with loss smaller than $s$ by merging any two components with statistical distance below $\gamma$. Thus, we formulate our measures of uncertainty with respect to the FOLD posterior $\Pi \left({\mathit{c}}_{\mathit{\theta }}\mid {\mathit{X}}^0\right)$, instead of $\Pi \left(\mathit{s}\mid {\mathit{X}}^0\right)$.

Accordingly, we adapt the notion of a credible ball for Bayesian clustering estimators (Wade and Ghahramani 2018) to our procedure. The 95% credible ball around ${\mathit{c}}_{\text{FOLD}}$ is defined as $B\left({\mathit{c}}_{\text{FOLD}}\right)=\left\{\mathit{c}:\mathbb{D}\left({\mathit{c}}_{\text{FOLD}},\mathit{c}\right)\le {ϵ}_{\text{FOLD}}\right\}$, where ${ϵ}_{\text{FOLD}}\ge 0$ is the smallest radius such that $\Pi \left\{\mathbb{D}\left({\mathit{c}}_{\text{FOLD}},{\mathit{c}}_{\mathit{\theta }}\right)\le {ϵ}_{\text{FOLD}}\mid {\mathit{X}}^0\right\}\ge 0.95$. We interpret $B\left({\mathit{c}}_{\text{FOLD}}\right)$ as a neighborhood of clusterings centered at ${\mathit{c}}_{\text{FOLD}}$ with posterior probability mass of at least 0.95. Larger ${ϵ}_{\text{FOLD}}$ means that the FOLD posterior distributes its mass across a larger set around ${\mathit{c}}_{\text{FOLD}}$, implying that we are more uncertain about the cluster allocations of the point estimator. In practice, we characterize the credible ball with bounds that give a sense of the clusterings contained within it (Wade and Ghahramani 2018), much like we would for an interval on the real line. The horizontal bounds of the credible ball consist of the clusterings $\mathit{c}\in B\left({\mathit{c}}_{\text{FOLD}}\right)$ for which $\mathbb{D}\left(\cdot ,{\mathit{c}}_{\text{FOLD}}\right)$ attains its maximum value. We can also impose further restrictions on our bounds, such as requiring that they contain the minimum or maximum number of clusters over $B\left({\mathit{c}}_{\text{FOLD}}\right)$ while also maximizing $\mathbb{D}\left(\cdot ,{\mathit{c}}_{\text{FOLD}}\right)$ amongst groupings with the same number of clusters, giving rise to the notions of vertical upper and vertical lower bounds, respectively. Alternatively, one can display the posterior similarity matrix (PSM) for ${\mathit{c}}_{\mathit{\theta }},{𝒫}_{\text{FOLD}}={\left(\Pi \left\{c_{i\mathit{\theta }}=c_{j\mathit{\theta }}\mid {\mathit{X}}^0\right\}\right)}_{i,j}$, as a heatmap, with the entries ordered by ${\mathit{c}}_{\text{FOLD}}$, as is common practice for both Binder’s loss and the VI.

2.4. Implementation with MCMC Output

Our methodology relies on Markov chain Monte Carlo (MCMC) samples from $\Pi \left(\mathit{\theta }\mid {\mathit{X}}^0\right)$, which are used to estimate pairwise distances between the localized densities, ${\Delta }_{ij}\approx (1/T)\sum _{t=1}^T{𝒟}_{ij}^{\left(t\right)}$, where ${𝒟}_{ij}^{\left(t\right)}=d\left\{g\left(\cdot ;{\theta }_i^{\left(t\right)}\right),g\left(\cdot ;{\theta }_j^{\left(t\right)}\right)\right\}$, and ${\theta }_i^{\left(t\right)},{\theta }_j^{\left(t\right)}$ can be computed directly from samples of the components ${\mathit{s}}^{\left(t\right)}$ and unique values ${\tilde{\mathit{\theta }}}^{\left(t\right)}$ via ${\theta }_i^{\left(t\right)}={\tilde{\theta }}_k^{\left(t\right)}⟺s_i^{\left(t\right)}=k$. When $d$ is chosen to be the Hellinger distance, many families of kernels, such as the Gaussian family, admit a closed form of ${𝒟}_{ij}$. The label-switching problem, which results from the inherent non-identifiability of mixture models with any permutation of the labels yielding the same likelihood (Redner and Walker 1984), has no impact on FOLD, as ${\Delta }_{ij}$ is invariant to labeling. FOLD can be easily applied to virtually any mixture model for which samples from $\Pi \left(\mathit{\theta }\mid {\mathit{X}}^0\right)$ can be obtained and pairwise statistical distances can be estimated.

In practice, the minimization of (5) must be approximated due to the enormous size of the partition space for even modest $n$. We discuss two methods for obtaining an approximate minimizer. The first is to simply reduce the set of partitions over which we minimize by only considering a tree of clusterings produced by hierarchical clustering on $\Delta$. We suggest generating candidates using average linkage due to the appearance of the average linkage dissimilarity metric as a criterion for merging clusters in (4). Figure 3 displays candidate clusterings of the moons data resulting from average linkage clustering on $\Delta$. The true grouping of the data is present amongst the candidates. Since these groupings are hierarchical, Figure 3 can be interpreted as an enumeration of clusterings favored by $ℛ\left(\widehat{\mathit{c}}\right)$, arranged by increasing values of $\omega$. A similar method with single-linkage is defined in Aragam et al. (2020), and Medvedovic and Sivaganesan (2002) and Fritsch and Ickstadt (2009) have previously used hierarchical clustering on $1_n{1}_n^T-𝒫$ to compute clustering point estimators, where $𝒫={\left(\Pi \left\{s_i=s_j\mid {\mathit{X}}^0\right\}\right)}_{i,j}$.

Figure 3.

Candidate clusterings of the moons data under a location Gaussian mixture model, generated from average linkage hierarchical clustering on $\Delta$.

Recent developments for minimizing risk functions have focused on greedy algorithms that take an existing candidate partition and make minor, but locally-optimal updates, typically by reallocation of objects to clusters. Examples include a method motivated by the Hasse diagram in Wade and Ghahramani (2018), the greedy algorithm in Rastelli and Friel (2018), and the SALSO algorithm of Dahl, Johnson, and Müller (2022). SALSO provides a point estimate by first initializing a clustering, then reallocating observations to clusters and, finally, breaking existing clusters and then reallocating observations again over several parallel runs. The final point estimate is the partition that exhibits the smallest risk across all the runs. In our simulations and application, we implement FOLD with both the hierarchical clustering heuristic and SALSO, and find that they generally agree on the optimal clustering.

To express uncertainty in ${\mathit{c}}_{\text{FOLD}}$, we rely on samples from $\Pi \left({\mathit{c}}_{\mathit{\theta }}\mid {\mathit{X}}^0\right)$, defined by

$${\mathit{c}}_{\mathit{\theta }}^{\left(t\right)}=\underset{\widehat{\mathit{c}}}{\text{argmin}}\sum _{i<j}\left\{{\mathbf{1}}_{{\hat{c}}_i={\hat{c}}_j}{𝒟}_{ij}^{\left(t\right)}+\omega {\mathbf{1}}_{{\hat{c}}_i\ne {\hat{c}}_j}\left(1-{𝒟}_{ij}^{\left(t\right)}\right)\right\}.$$ (6)

As in the point estimation case, ${\mathit{c}}_{\mathit{\theta }}^{\left(t\right)}$ cannot be computed exactly and so we approximate the minimization in (6) either using a set of candidates generated from hierarchical clusterings of ${𝒟}^{\left(t\right)}={\left({𝒟}_{ij}^{\left(t\right)}\right)}_{i,j}$ or via a greedy algorithm such as SALSO. The posterior probability of a credible ball is approximated by $\Pi \left\{\mathbb{D}\left({\mathit{c}}_{\mathit{\theta }},{\mathit{c}}_{\text{FOLD}}\right)\le ϵ\mid {\mathit{X}}^0\right\}\approx (1/T)\sum _{t=1}^T{\mathbf{1}}_{\mathbb{D}\left({\mathit{c}}_{\mathit{\theta }}^{\left(t\right)},{\mathit{c}}_{\text{FOLD}}\right)\le ϵ}$ and FOLD computes ${ϵ}_{\text{FOLD}}$ by incrementally increasing the value of $ϵ$ over a grid. The entries of ${𝒫}_{\text{FOLD}}$ are estimated via $\Pi \left(c_{i\mathit{\theta }}=c_{j\mathit{\theta }}\mid {\mathit{X}}^0\right)\approx (1/T)\sum _{t=1}^T{\mathbf{1}}_{c_{i\mathit{\theta }}^{\left(t\right)}=c_{j\mathit{\theta }}^{\left(t\right)}}$.

2.5. Selection of $\omega$

The loss parameter $\omega$ controls the separation among the inferred clusters in ${\mathit{c}}_{\text{FOLD}}$. This parameter is also influential in ${\mathit{c}}_{\text{VI}}$ and ${\mathit{c}}_{\mathrm{B}}$ (Dahl, Johnson, and Müller 2022), though there has been little discussion in the literature on how to select an appropriate $\omega$ for these loss functions. The formulation of (2) in terms of statistical distances leads to two sensible methods for determining the loss parameter. First, similarly to common practice in choosing key tuning parameters in algorithmic clustering methods such as k-means and DBSCAN, we propose an elbow plot diagnostic to select $\omega$. At a grid of possible $\omega$ values, we calculate $r_{\omega }=\left(\sum _{h=1}^{K_{\omega }^{*}}{r}_{\omega h}\right)/\sum _{i<j}{\Delta }_{ij}$, where $r_{\omega h}=\sum _{i,j\in C_{\omega h}^{*}}{\Delta }_{ij},{\mathit{c}}_w^{*}$ is the FOLD point estimator for that specific $\omega$, and $C_{\omega }^{*}=\left\{C_{\omega 1}^{*},\dots ,C_{\omega K_w^{*}}^{*}\right\}$ is the partition associated with ${\mathit{c}}_{\omega }^{*}$. The numerator in $r_w$ sums over $r_{\omega h}={\mathbb{E}}_{\Pi }\left\{\sum _{i,j\in C_{\omega h}^{*}}{𝒟}_{ij}\mid {\mathit{X}}^0\right\}$, or the expected statistical distance between all localized densities associated with the $h$th cluster in ${\mathit{c}}_{\omega }^{*}$, and the denominator normalizes so that $r_{\omega }\in \left[\mathrm{0,\; 1}\right]$ for all $\omega$.

To build some intuition about the utility of the elbow plot, recall that as $\omega \to \infty$, (5) is minimized by placing all observations in a single cluster, that is $C_{\infty }^{*}=\left\{\right[n\left]\right\}$ and $r_{\infty }=1$. In many cases, such as the data in Figure 3, decreasing $\omega$ (and subsequently adding more clusters in $C_{\omega }^{*}$) will decrease $r_{\omega }$, for example because ${\Delta }_{ij}\approx 1$ for any $i,j$ pairs in different crescents. Of course, at the other extreme, if we set $\omega$ to be very small, we place all objects in their own cluster, making $C_0^{*}=\left\{\right\{1\},\dots ,\{n\left\}\right\}$ and $r_0=0$. We construct the elbow plot to quantify the amount of improvement each time we increment $\omega$ across this grid. At some point, the improvement in $r_{\omega }$ becomes negligible as we decrease $\omega$ because we begin to cut dense clusters into smaller and smaller sub-clusters. On an elbow plot, this causes the curve of $r_{\omega }$ to bend, creating the so-called “elbow”. It is at this threshold that we fix $\omega$. The threshold is easier to identify when the elbow plot is monotonic, and we verify this is the case if one uses hierarchical clustering to generate candidates.

Proposition 3. If candidate clusterings are selected using average linkage hierarchical clustering on $\Delta$, then there is a positive relationship between $\omega$ and $r_{\omega }$.

A corollary of Proposition 3 is that any jump in the elbow plot corresponds to decreasing the number of clusters by one, meaning that there is a negative relationship between $K_{\omega }^{*}$ and $r_{\omega }$. Therefore, one can relabel the x-axis of the elbow plot in terms of the number of clusters rather than $\omega$, leading to a more interpretable figure. An example of using an elbow plot to select $\omega$ is given in Section 4.

Alternatively, one can use the default value ${\omega }^{\text{AVG}}={\gamma }^{\text{AVG}}/\left(1-{\gamma }^{\text{AVG}}\right)$, with ${\gamma }^{\text{AVG}}={\left(\genfrac{}{}{0pt}{}{n}{2}\right)}^{-1}\sum _{i<j}{\Delta }_{ij}$. Under this choice of $\omega$, candidate clusters are combined when the average of ${\Delta }_{ij}$ between the clusters is smaller than the average of ${\Delta }_{ij}$ across the entire sample. ${\gamma }^{\text{AVG}}$ estimates $\overline{𝒟}={\left(\genfrac{}{}{0pt}{}{n}{2}\right)}^{-1}\sum _{i<j}{𝒟}_{ij}=\sum _{k<k^{\prime }}\left(\left|S_k\right|\left|S_{k^{\prime }}\right|/\left(\genfrac{}{}{0pt}{}{n}{2}\right)\right)d\left\{g\left(\cdot ;{\tilde{\theta }}_k\right),g\left(\cdot ,{\tilde{\theta }}_{k^{\prime }}\right)\right\}$, a weighted pairwise sum across the components. If we use $\overline{𝒟}/(1-\overline{𝒟})$ in place of $\omega$ in (2), (4) implies that FOLD will favor merging mixture components when $d\left\{g\left(\cdot ;{\tilde{\theta }}_k\right),g\left(\cdot ;{\tilde{\theta }}_{k^{\prime }}\right)\right\}<\overline{𝒟}$. Importantly, the decision to merge components will depend on how separated they are from the others and their sizes. To see this, consider the following example, in which we fit a mixture with $K=3$ components, where $d\left\{g\left(\cdot ;{\tilde{\theta }}_1\right),g\left(\cdot ;{\tilde{\theta }}_2\right)\right\}=ϵ>0$, $d\left\{g\left(\cdot ;{\tilde{\theta }}_1\right),g\left(\cdot ;{\tilde{\theta }}_3\right)\right\}=d\left\{g\left(\cdot ;{\tilde{\theta }}_2\right),g\left(\cdot ;{\tilde{\theta }}_3\right)\right\}=\delta >0$, and $\left|S_1\right|=\left|S_2\right|=\left|S_3\right|=n/3$. Then, as $n$ grows large, $\overline{𝒟}\to (2/9)ϵ+(4/9)\delta$, and so FOLD will favor merging $S_1$ and $S_2$ into one cluster when $ϵ<(4/7)\delta$. The smaller $\delta$ is, the smaller $ϵ$ must be in order to merge $S_1$ and $S_2$. Hence, ${\omega }^{\text{AVG}}$ excels at problems in which $f^0$ is composed of well-separated kernels that are approximated by multiple components in $f$. We provide simulation studies in the supplement that show that ${\omega }^{\text{AVG}}$ performs very well in this setting.

3. Asymptotic Analysis

Though there is a rich literature on asymptotic properties of Bayesian mixture models for estimating the density (Ghosal, Ghosh, and Ramamoorthi 1999; Ghosal and van der Vaart 2007), number of components (Rousseau and Mengersen 2011; Miller and Harrison 2014; Cai et al. 2021; Ascolani et al. 2023), and the mixing measure (Nguyen 2013; Ho and Nguyen 2016a, 2016b; Guha, Ho, and Nguyen 2021), little attention has been given to the large sample behavior of clustering estimators. Rajkowski (2019) focused on the maximum a posteriori (MAP) estimator for $\mathit{s}$ when data are generated from a DPM with Gaussian components and the kernels are correctly specified. They show multiple properties of the MAP estimator, including the key result that the intersection between the convex hulls of two clusters is at most a single observation. In this section, we show convergence of FOLD toward the oracle clustering procedure in which the objects are partitioned into groups using knowledge of the true dating generating process. We take a different approach than Rajkowski (2019) by focusing on posterior contraction of the mixing measure $\lambda$, which allows us to consider misspecifed models, either in the kernels or number of components.

3.1. Assumptions

We first require that the parameter space $\Theta \subset {\mathbb{R}}^p$ is compact and $f$, defined formally as $f\left(x\right)=\int g(x;\theta )\lambda \left(d\theta \right)$, is identifiable with respect to the mixing measure, that is, $\int g(x;\theta ){\lambda }_1\left(d\theta \right)=\int g(x;\theta ){\lambda }_2\left(d\theta \right)$ for all $x\in \chi$ if and only if ${\lambda }_1={\lambda }_2$ (Teicher 1961). Identifiability of the mixing measure is satisfied by location and location-scale GMMs, as well as various exponential family mixtures (Barndorff-Nielsen 1965) and location family mixtures (Teicher 1961). More specifically, our results rely on contraction of the mixing measure $\lambda$ to some oracle measure ${\lambda }^{*}$, which implies convergence of $f$ to an oracle model $f^{*}=\int g(\cdot ;\theta ){\lambda }^{*}\left(d\theta \right)$. An oracle measure is defined as any Kullback-Liebler (KL) divergence minimizer between our class of models and the true data generating process, that is ${\lambda }^{*}\in \underset{\lambda \in \Lambda }{\text{argmin}}\text{KL}\left(f^0,f\right)$, where recall $\Lambda =\text{supp}\left({\pi }_{\Lambda }\right)$. We now present the general assumptions required for our results.

Assumption 1. Suppose that the following conditions hold.

• (A1) There exists an $L>0$ such that $\left|\text{supp}\right(\lambda \left)\right|\le L$ for all $\lambda \in \Lambda$.

• (A2) The KL minimizer ${\lambda }^{*}=\sum _{m=1}^{M^{*}}{a}_m^{*}{\delta }_{{\theta }_m^{*}}$ exists and is unique.

• (A3) $f^{*}$ and $f^0$ are such that $\text{supp}\left(f^0\right)\subseteq \text{supp}\left(f^{*}\right)$.

• (A4) $𝒢=\left\{\tilde{g}\right(x-\theta ):\theta \in \Theta \}$ for some bounded probability density function $\tilde{g}(\cdot )$, $\tilde{g}(\cdot )$ is $\zeta$-Hölder continuous for some $\zeta >0$, and for any fixed ${\theta }^{\prime },D_g\left(\theta ,{\theta }^{\prime }\right)=d\left\{g(\cdot ;\theta ),g\left(\cdot ;{\theta }^{\prime }\right)\right\}$ is continuous in $\theta$.

• (A5) There exists a nonnegative sequence ${ϵ}_n$ so that ${ϵ}_n\to 0$ and

$${\rho }_n\left({\mathit{X}}^0\right)=\Pi \left\{\lambda \in \Lambda :W_2\left(\lambda ,{\lambda }^{*}\right)\gtrsim {ϵ}_n\mid {\mathit{X}}^0\right\}\stackrel{{\mathbb{P}}^0}{\to }0.$$ (7)

(A1) is satisfied by any mixture model in which the number of components is bounded, and examples include a finite mixture model, a truncated MFM, or truncated DPM. Assumption (A2) ensures that the limiting values of the clustering procedure are well-defined and exist. A simple example of when (A2) holds is when the kernels are correctly specified and $\Pi \left(K=M^0\right)>0$, in which case ${\lambda }^{*}={\lambda }^0$ and $f^{*}=f^0$. We refer to the case in which $\Pi \left(K=M^0\right)=1$ as the exact-fitted and well-specified regime, which is permitted under assumptions (A1)–(A2). Under any misspecification, (A2) holds when ${\text{min}}_{\lambda \in \Lambda }\text{KL}\left(f^0,f\right)={\text{min}}_{\lambda \in \Omega }\text{KL}\left(f^0,f\right)$, where $\Omega$ is the space of all probability measures over $\Theta$ (Guha, Ho, and Nguyen 2021). That is, assumption (A2) is satisfied in the misspecified regime when the minimum KL divergence is attained over $\Lambda$. This still allows for an exact-fitted and misspecified regime, in which $\Pi \left(K=M^{*}\right)=1$. (A3) is a technical assumption to ensure there are no regions in the sample space under which $f^0$ gives positive probability but $f^{*}$ does not. Assumption (A4) restricts our results to location families, and the continuity conditions are satisfied by a variety of models including the location GMM with fixed covariance. Assumption (A5) is a crucial statement on the estimation of parameters in a mixture model. (A5) will hold under various Lipschitz and strong identifiability conditions on $𝒢$ (Nguyen 2013; Ho and Nguyen 2016a; Guha, Ho, and Nguyen 2021) that are satisfied by the location GMM (Manole and Ho 2022), and we discuss these conditions in detail in the Supplementary Material. (A5) will hold with ${ϵ}_n=(\text{log}n/n{)}^{1/4}$ for second-order identifiable families, and first-order identifiable families with ${ϵ}_n=(\text{log}n/n{)}^{1/2}$ if the kernels are correctly specified and $\Pi \left(K=M^0\right)=1$ (Guha, Ho, and Nguyen 2021); see Ho and Nguyen (2016a, 2016b) for a characterization of first and second-order identifiable kernels.

3.2. Convergence to The Oracle Rule

Synonymous with the oracle mixing measure is the oracle clustering procedure or rule. We define the oracle clustering procedure to be the minimizer of the oracle risk function ${ℛ}^{*}\left(\widehat{\mathit{c}}\right)={\mathbb{E}}_{\Pi }\left[ℒ(\widehat{\mathit{c}},\mathit{\theta })\mid {\lambda }^{*},{\mathit{X}}^0\right]$,

$$\begin{gathered} {\mathit{c}}_{\text{FOLD}}^{*}=\underset{\widehat{\mathit{c}}}{\text{argmin}}{ℛ}^{*}\left(\widehat{\mathit{c}}\right) \\ =\underset{\widehat{\mathit{c}}}{\text{argmin}}\sum _{i<j}\left\{{\mathbf{1}}_{{\hat{c}}_i={\hat{c}}_j}{\Delta }_{ij}^{*}+\omega {\mathbf{1}}_{{\hat{c}}_i\ne {\hat{c}}_j}\left(1-{\Delta }_{ij}^{*}\right)\right\}, \end{gathered}$$ (8)

where ${\Delta }_{ij}^{*}={\mathbb{E}}_{\Pi }\left({𝒟}_{ij}\mid {\lambda }^{*},{\mathit{X}}^0\right)=\sum _{m<m^{\prime }}d\left\{g\left(\cdot ;{\theta }_m^{*}\right),g\left(\cdot ;{\theta }_{m^{\prime }}^{*}\right)\right\}{q}_{ij}^{m{m}^{\prime }*}$, with $q_{ij}^{m{m}^{\prime }*}=\Pi \left(s_i=m,s_j=m^{\prime }\mid {\mathit{X}}^0,{\lambda }^{*}\right)+\Pi \left(s_i=m^{\prime },s_j=m\mid {\mathit{X}}^0,{\lambda }^{*}\right)$, and

$$\Pi \left(s_i=m,s_j=m^{\prime }\mid {\mathit{X}}^0,{\lambda }^{*}\right)=\frac{a_m^{*}{a}_{m^{\prime }}^{*}g\left(X_i^0;{\theta }_m^{*}\right)g\left(X_j^0;{\theta }_{m^{\prime }}^{*}\right)}{f^{*}\left(X_i^0\right)f^{*}\left(X_j^0\right)}.$$

Observe that ${\Delta }_{ij}^{*}$ is a weighted sum of the total statistical distance between the oracle components, where the weight of each component pair is given by the conditional probability that $s_i=m$ and $s_j=m^{\prime }$ (or vice versa) given $\mathit{X}={\mathit{X}}^0$ and $\lambda ={\lambda }^{*}$. We interpret the oracle clustering procedure as a rule for grouping the observations in ${\mathit{X}}^0$ based on the knowledge of the optimal parameter values in the mixture model. We can construct two simple examples of oracle rules which follow from the definition in (8).

Proposition 4. Instances of the oracle clustering procedure include:

1. If $d\left\{g\left(\cdot ;{\theta }_m^{*}\right),g\left(\cdot ;{\theta }_{m^{\prime }}^{*}\right)\right\}=1$ for all $m,m^{\prime }$, then ${\mathit{c}}_{\text{FOLD}}^{*}={\mathit{s}}^{*}$, where $s_i^{*}=s_j^{*}$ if and only if there exists a unique $m\in \left[M^{*}\right]$ such that $g\left(X_i^0;{\theta }_m^{*}\right)g\left(X_j^0;{\theta }_m^{*}\right)>0$.

2. If $M^{*}=2$ and $d\left\{g\left(\cdot ;{\theta }_1^{*}\right),g\left(\cdot ;{\theta }_2^{*}\right)\right\}<\gamma$, then ${\mathit{c}}_{\text{FOLD}}^{*}={\mathit{c}}_0$, where $c_{0i}=c_{0j}$ for all $i,j$.

We refer to the procedure on Proposition 4(a) as the “match rule” in that it allocates each observation to a mixture component with compatible support, and call the procedure in Proposition 4(b) the “merge rule”, that is, as the statistical distance between oracle components becomes smaller, we are more likely to combine them and subsequently place all observations in one cluster. A similar quantity exists for Binder’s loss function, denoted ${\mathit{c}}_{\mathrm{B}}^{*}=\underset{\widehat{\mathit{c}}}{\text{argmin}}{\mathbb{E}}_{\Pi }\left[{ℒ}_{\mathrm{B}}(\widehat{\mathit{c}},\mathit{s})\mid {\lambda }^{*},{\mathit{X}}^0\right]$. Under the settings of Proposition 4(a), ${\mathit{c}}_{\mathrm{B}}^{*}={\mathit{c}}_{\text{FOLD}}^{*}={\mathit{s}}^{*}$, meaning that the match rule is the optimal clustering procedure under both FOLD and Binder’s loss when the oracle components are perfectly separated. Intuitively, this is because these settings reflect an ideal scenario in which all mixture components are perfectly separated and no merging is required. Otherwise, say in the setting of Proposition 4(b), the two methods can differ. A sufficient condition for ${\mathit{c}}_{\mathrm{B}}^{*}={\mathit{c}}_0$ is $q_{ij}^{12}<\omega \left(1-q_{ij}^{12}\right)$ for all pairs $i,j$. Any observations in the tails of the two oracle components will blow up the value of $q_{ij}^{12}$, hence, it is possible that ${\mathit{c}}_{\mathrm{B}}^{*}\ne {\mathit{c}}_0$ even for relatively large $\omega$. This implies that a concrete advantage of our approach is robustness to outliers when the oracle components are not well separated. We now state our primary theoretical result on the relationship between ${\mathit{c}}_{\text{FOLD}}$ and ${\mathit{c}}_{\text{FOLD}}^{*}$.

Theorem 1. Fix $0<\delta <1$. Then under assumptions (A1)–(A5), with ${\mathbb{P}}^0$-probability tending to 1,

$${\left(\genfrac{}{}{0pt}{}{n}{2}\right)}^{-1}\left|ℛ\left({\mathit{c}}_{\text{FOLD}}\right)-{ℛ}^{*}\left({\mathit{c}}_{\text{FOLD}}^{*}\right)\right|\lesssim \text{max}(\omega ,1)\left\{{\rho }_n\left({\mathit{X}}^0\right){\bar{\Delta }}^{*}+𝒬\left({\mathit{X}}^0\right)\right\},$$ (9)

where ${\bar{\Delta }}^{*}={\left(\genfrac{}{}{0pt}{}{n}{2}\right)}^{-1}\sum _{i<j}{\Delta }_{ij}^{*},𝒬\left({\mathit{X}}^0\right)={\left(\genfrac{}{}{0pt}{}{n}{2}\right)}^{-1}\sum _{i<j}{\tau }_n^{*}\left(X_i^0,X_j^0\right)$, and ${\tau }_n^{*}\left(X_i^0,X_j^0\right)$ is a function of $\left(X_i^0,X_j^0,{ϵ}_n,{\rho }_n\left({\mathit{X}}^0\right),{\lambda }^{*},\zeta ,\delta \right)$ so that ${\tau }_n^{*}\left(X_i,X_j\right)\stackrel{{\mathbb{P}}^0}{\to }0$.

To prove Theorem 1, we first note that we can write ${\Delta }_{ij}$ as the posterior expectation of a quantity that only depends on $\lambda =\sum _{k=1}^K{a}_k{\delta }_{{\tilde{\theta }}_k}$,

$${\Delta }_{ij}={\int }_{\lambda \in \Lambda }\Pi \left(d\lambda \mid {\mathit{X}}^0\right)\sum _{k<k^{\prime }}{D}_g\left({\tilde{\theta }}_k,{\tilde{\theta }}_{k^{\prime }}\right)a_k{a}_{k^{\prime }}\times \frac{g\left(X_i^0;{\tilde{\theta }}_k\right)g\left(X_j^0;{\tilde{\theta }}_{k^{\prime }}\right)+g\left(X_j^0;{\tilde{\theta }}_k\right)g\left(X_i^0;{\tilde{\theta }}_{k^{\prime }}\right)}{f\left(X_i^0\right)f\left(X_j^0\right)}.$$ (10)

Next, we decompose $\Lambda =B_{{ϵ}_n}\left({\lambda }^{*}\right)\cup \left(\Lambda \setminus B_{{ϵ}_n}\left({\lambda }^{*}\right)\right)$, where $B_{{ϵ}_n}\left({\lambda }^{*}\right)=\left\{\lambda \in \Lambda :W_2\left(\lambda ,{\lambda }^{*}\right)\lesssim {ϵ}_n\right\}$, and split (10) into a sum of integrals over these domains. Since $0\le D_g\left(\theta ,{\theta }^{\prime }\right)\le 1$, the integral over $\Lambda \setminus B_{{ϵ}_n}(\lambda {)}^{*}$ is bounded above by ${\rho }_n\left({\mathit{X}}^0\right)$, which goes to 0 in probability by (A5). Then, using a similar proof technique given in Nguyen (2013), we show that there exists a finite $N$ so that for all $n\ge N$ and $\lambda \in B_{{ϵ}_n}\left({\lambda }^{*}\right),\lambda$ has at least $M^{*}$ support points and for any $m\in \left[M^{*}\right]$ there exists a set of indices ${ℐ}_m^{\left(\lambda \right)}\subset \left[K\right]$ so that ${\text{max}}_{k\in {ℐ}_m^{\left(\lambda \right)}}‖{\tilde{\theta }}_k-{\theta }_m^{*}‖\lesssim {ϵ}_n$ and $\left|\sum _{k\in {ℐ}_m^{\left(\lambda \right)}}{a}_k-a_m^{*}\right|\lesssim \text{max}\left({ϵ}_n,{ϵ}_n^{2\delta }\right)$. Furthermore, if we denote ${ℐ}^{\left(\lambda \right)}=\bigcup _{m=1}^{M^{*}}{ℐ}_m^{\left(\lambda \right)}$, (A5) also implies that $\sum _{k\notin {ℐ}^{\left(\lambda \right)}}{a}_k\lesssim {ϵ}_n^{2\delta }$. By (A4), these bounds translate into similar upper and lower bounds for ${\Delta }_{ij}$. We then sum over all $i,j$ pairs and show that ${\left(\genfrac{}{}{0pt}{}{n}{2}\right)}^{-1}\left|ℛ\left(\widehat{\mathit{c}}\right)-{ℛ}^{*}\left(\widehat{\mathit{c}}\right)\right|$ is uniformly bounded by the right hand side of (9) for all $\widehat{\mathit{c}}$. That is, the rate of convergence of $ℛ\left(\widehat{\mathit{c}}\right)$ to ${ℛ}^{*}\left(\widehat{\mathit{c}}\right)$ does not depend on the clustering estimator itself. Finally, we show that this implies convergence of the minimizers at the same rate.

The bound in (9) is split into two remainder terms. It follows from (A5) that ${\rho }_n\left({\mathit{X}}^0\right){\bar{\Delta }}^{*}\stackrel{{\mathbb{P}}^0}{\to }0$ since $0\le {\bar{\Delta }}^{*}\le 1$. Therefore, smaller values of ${\bar{\Delta }}^{*}$ will cause this term to diminish faster. This can arise when $f^{*}$ is comprised of many closely overlapping components. Hence, convergence to the oracle rule will actually benefit from overfitting. $𝒬\left({\mathit{X}}^0\right)$ is a more general remainder term that results from translating the posterior contraction of ${\tilde{\theta }}_k$ and $a_k$ to the sum in (10). In particular, this remainder term will depend on $\delta$, which controls the rate at which the irrelevant mixture components are emptied, the Hölder constant $\zeta$, which controls convergence of $g\left(X_i^0;{\tilde{\theta }}_k\right)$ to $g\left(X_i^0;{\theta }_m^{*}\right)$, and $L-M^{*}$, or the degree to which the number of components is misspecified, with larger $L$ implying a slower rate of convergence.

In the supplementary material we empirically validate Theorem 1 on a simple simulated example. We simulate data from $M^0=4$ Gaussian components with covariance equal to $0.25I$, then fit a 4-component Bayesian GMM, meaning that $f^0\left(x\right)=f^{*}\left(x\right)=\sum _{m=1}^4{a}_m^0𝒩\left(x;{\mu }_m^0,0.25I\right)$. We compute ${\mathit{c}}_{\text{FOLD}},{\mathit{c}}_{\text{FOLD}}^{*},{\mathit{c}}_{\mathrm{B}}$, and ${\mathit{c}}_{\mathrm{B}}^{*}$, the latter of which denotes the oracle clusters corresponding to Binder’s loss, for increasing sample size. We show that both $ℛ\left(\widehat{\mathit{c}}\right)\to {ℛ}^{*}\left(\widehat{\mathit{c}}\right)$ and ${\mathit{c}}_{\text{FOLD}}\to {\mathit{c}}_{\text{FOLD}}^{*}$ as $n\to \infty$. Additionally, there are substantial differences between ${\mathit{c}}_{\text{FOLD}}^{*}$ and ${\mathit{c}}_{\mathrm{B}}^{*}$ because oracle FOLD corresponds to the merge rule for some of the Gaussian components, while oracle Binder prefers estimation of each individual component.

4. Example: GSE81861 Cell Line Dataset

We apply FOLD to the GSE81861 cell line dataset (Li et al. 2017), which measures single cell counts in 57,241 genes from 630 single-cell transcriptomes. There are seven distinct cell lines present in the dataset, so we compare FOLD and other model-based clustering methods to the true cell line labels as a performance benchmark. First, we apply routine pre-processing steps for RNA sequence analysis (as in Chandra, Canale, and Dunson 2023). We discard cells with low read counts, giving $n=519$ total cells. We then normalize the data using SCRAN (Lun, Bach, and Marioni 2016) and select informative genes with M3Drop (Andrews and Hemberg 2019). We use principal component analysis (PCA) for dimension reduction by taking $X^0$ to be the projection of the normalized cell counts onto the first $p=5$ principal components, then scale the projections to have zero mean and unit variance.

We fit a $p$-dimensional Bayesian Gaussian finite mixture to ${\mathit{X}}^0$ with $K=50$ components, and simulate 25,000 posterior samples after a burn-in of 1000. Every fourth iteration is discarded, leaving 6000 samples remaining. Along with ${\mathit{c}}_{\text{FOLD}}$, we calculate model-based clusterings with the VI loss, Binder’s loss, and mclust (Scrucca et al. 2016), the latter of which implements the EM algorithm and chooses the number of clusters via the Bayesian information criterion (BIC). We also compare to algorithmic clustering methods, including average linkage hierarchical clustering (HCA), k-means, DBSCAN, and spectral clustering. Details on hyperparameter choices for all methods are given in the supplement. For FOLD, we create candidate clusterings by average-linkage clustering on $\Delta$, then choose $\omega$ by consulting the elbow plot in Figure 4. The plot suggests 6 clusters, which corresponds to $\omega =25$. We also implement SALSO over 150 independent runs to minimize (5). However, over repeated replications we find that the SALSO estimate is either equal to or has a higher risk than the point estimate given by hierarchical clustering on $\Delta$. Binder’s loss and the VI loss are implemented with $\omega =1$ (the default value in mcclust (Fritsch 2022)), which results in ${\mathit{c}}_{\text{VI}}={\mathit{c}}_{\mathrm{B}}$.

Figure 4.

Elbow plot for choosing the number of clusters in ${\mathit{c}}_{\text{FOLD}}$ with the cell line dataset.

Figure 5 shows the UMAP plots (McInnes et al. 2018) of the original normalized count data along with colors indicating the true cell types and the clusterings from the three model-based methods. The adjusted Rand index (ARI) (Hubert and Arabie 1985) with the true cell types and number of clusters $\left({\hat{K}}_n\right)$ for ${\mathit{c}}_{\text{FOLD}}$ and all other clusterings are given in Table 1. First, observe that the Bayesian clustering methods outperform the competitors in the ARI, indicating that the overfitted Gaussian components are accurately approximating the distribution within each type. However, FOLD notably excels over the other Bayesian methods with an ARI of 0.995, only misclassifying six observations. Note that the VI, Binder’s loss, and mclust split the GM12878 and H1 cell types into two clusters each, while FOLD correctly identifies both types, keeping each as single clusters. The splitting of types is not unexpected in this application since the GM12878 and H1 types were each sequenced in two separate batches (Li et al. 2017), indicating that FOLD is more robust to the batch effect than other Bayesian methods. However, ${\mathit{c}}_{\text{FOLD}}$ underestimates the number of groups by combining the H1437 and IMR90 cell types. The other model-based methods merge these types as well, with the VI and Binder’s loss producing nine clusters and mclust giving seven clusters. Of the algorithmic methods, k-means performs the best, estimating the correct number of types while achieving an ARI of 0.904. However, k-means splits the GM12878 and A549 types into two clusters each, the latter of which is always kept as one cluster by the model-based methods.

Figure 5.

UMAP plots of the cell line dataset with colors corresponding to the true cell types, ${\mathit{c}}_{\text{FOLD}}$, VI/Binder, and mclust, respectively.

Table 1.

Adjusted Rand index (ARI) with the true cell lines and the number of clusters (${\hat{K}}_n$) for FOLD and competitors on the cell line dataset.

	FOLD	VI	Binder	Mclust	HCA	K-Means	DBSCAN	Spectral
ARI	0.995	0.915	0.915	0.854	0.622	0.904	0.679	0.620
${\hat{K}}_n$	6	9	9	7	7	7	5	5

The 95% credible ball around ${\mathit{c}}_{\text{FOLD}}$ is displayed in Figure 6. The credible ball communicates that there is substantial uncertainty in cell types where batch effects occur. The horizontal and vertical upper bounds effectively merge the GM12878 type with the H1437 cell type, which is likely due to the proximity of these two types in the sample space. Conversely, the vertical lower bound splits the GM12878 cell type into two clusters. We are also uncertain in our classification of the H1 cell type, which is similarly split into multiple clusters by the vertical lower bound because of separate batching. Interestingly, in all bounds, the H1437 and IMR90 cell types are allocated to the same cluster. This type merging could be explained by the fact that IMR90 consists of a small number of cells or that both types are isolated from the lung (Li et al. 2017). In the supplement, we further evaluate our methodology by applying FOLD and competitors to six clustering datasets.

Figure 6.

The clustering ${\mathit{c}}_{\text{FOLD}}$ along with the horizontal, vertical lower, and vertical upper bounds for the cell line dataset. Here, $\mathbb{D}(\cdot ,\cdot )$ is the VI and the bounds are unique.

5. Discussion

Fusing of Localized Densities (FOLD) is a Bayesian method for cluster analysis that characterizes clusters as possibly containing multiple mixture components. We first detailed the decision theoretic justification behind our approach, in which we obtain a point estimate of the clustering by minimizing a novel loss function. Our loss function has several appealing properties, such as favoring the merging of overlapping kernels, simplification to Binder’s loss when all the mixture components are well separated, and invariance to permutations of the labels. Uncertainty in cluster allocations is expressed with a credible ball and posterior similarity matrix. We have given concrete guidance on tuning the loss parameter, including an elbow plot method and default value that performs excellently on simulated examples.

Throughout the article, we have primarily focused on the Gaussian mixture model because of its ubiquity in the literature and useful theoretical properties. However, FOLD can be applied to any parametric mixture in which a bounded statistical distance between kernels is simple to compute. For the Hellinger distance, this includes the beta, exponential, gamma, and Weibull families. A near identical approach can be applied to discrete families, where localized mass functions would replace the role of the localized densities.

We have shown that under regularity conditions, our clustering point estimator converges to the oracle clustering procedure as $n\to \infty$ for both misspecified and well-specified kernel regimes. The oracle rule categorizes observations into clusters using knowledge about the KL-minimizer. We gave two concrete examples of oracle rules, including the match rule and the merge rule. Our findings imply similar behavior for Binder’s loss, meaning that analogous rules could be potentially derived for other methods such as the VI. These results would provide valuable insight into the overall asymptotic performance of Bayesian clustering. In addition, our results could be extended by focusing on contraction of the mixing measure in terms of the Voronoi loss function (Manole and Ho 2022), a loss specifically intended for strongly identifiable families that has been shown to contract at a faster rate.

Though we implemented FOLD with the Hellinger distance in our simulations and application, other distribution metrics could be used instead. For a general statistical distance $d$, one could set ${𝒟}_{ij}=1-\text{exp}\left\{-d\left(g\left(\cdot ;{\theta }_i\right),g\left(\cdot ;{\theta }_j\right)\right)\right\}$. The loss of co-clustering $i$ and $j$ is $1-\text{exp}\left\{-d\left(g\left(\cdot ;{\theta }_i\right),g\left(\cdot ;{\theta }_j\right)\right)\right\}$. This implies that we would favor co-clustering $i$ and $j$ when $d\left(g\left(\cdot ;{\theta }_i\right),g\left(\cdot ;{\theta }_j\right)\right)<-\text{log}(1-\gamma )$. For ${\theta }_i\in {\mathbb{R}}^p$, an even simpler variant is to choose ${𝒟}_{ij}=1-\text{exp}\left\{-\rho {‖{\theta }_i-{\theta }_j‖}^2\right\}$, where $\rho >0$ is some fixed bandwidth, which would promote merging components with similar atoms. Alternatively, one can view $\Delta$ as a stochastic distance matrix and employ k-medoids, hierarchical clustering, or spectral clustering to cluster the data. Finally, the general notion of specifying loss functions for Bayesian clustering using $\mathit{\theta }$ and not just $\mathit{s}$ provides a promising direction for future research.