Bayesian Pairwise Comparison of High-Dimensional Images

Abstract

A fundamental task in the automated analysis of images is the development of effective image pair comparison techniques. For two high-dimensional images, a statistical method must automatically label them as “similar” or “different” depending on whether random error and spatial dependencies could account for the pixel-wise differences. We develop a Bayesian strategy by constructing a novel extension of Dirichlet processes called the spatial random partition model (sRPM). The process groups spatially proximal image pixels with similar intensities into clusters, thereby achieving dimension reduction in the large number of pixels. Next, we apply the sRPM-based analytical procedure to compare two images. The image comparison problem is formulated as a hypothesis test involving a univariate metric adaptive to spatial correlations and robust to random variability in the pixel intensities. To handle the computational burden, we foster a two-stage technique for MCMC analysis and hypothesis testing of image pairs. A simulation study analyzes artificial datasets and finds compelling evidence for the high accuracy of sRPM in image comparison. We demonstrate the effectiveness of the technique by statistically analyzing satellite image data.

1. Introduction

Due to the rapid advancements in image acquisition techniques, images have emerged as one of the most versatile and information-rich data resources across diverse industries and scientific disciplines. For example, in the manufacturing industry, images play a pivotal role in quality control processes due to their ease of acquisition and cost-effectiveness. Similarly, in the natural sciences, satellite images are indispensable for analyzing Earth’s surface and environment across disciplines such as forest science, climate science, agriculture, forecasting, ecology, and fire science (Qiu, 2005; Gonzalez, 2009).

In many of these contexts, a series of images is often collected over time from a longitudinal process—such as images acquired from a rolling process in metalworking—where a critical goal is to monitor for temporal changes and detect potential “change points.” Identifying such change points is essential as they indicate moments where the process diverges beyond acceptable limits of variation, potentially signaling anomalies or shifts in underlying dynamics. This task is closely related to statistical process control (SPC) (Qiu, 2014), but existing SPC methodologies are insufficient for high-dimensional image data due to unrealistic model assumptions and limited practical applicability.

A foundational task underpinning this monitoring process is image pair comparison. By developing effective techniques for comparing high-dimensional images, we can systematically classify pairs of images as “similar” or “different” based on whether observed pixel-wise differences can be attributed to random error. Successful image comparison is often the first step toward more complex objectives, such as inferring the nature of detected changes, isolating regions of interest, or predicting future behaviors of a system under monitoring. For example, in quality control, identifying a shift might guide interventions to rectify the process, while in satellite image analysis, detected changes might signal environmental shifts requiring further study. Thus, image comparison lays the groundwork for meaningful downstream analyses and decision-making in these applications.

In response to the growing need for sophisticated methodologies that combine robust modeling with efficient computation, this paper aims to address the challenges of high-dimensional image comparison and provide a practical framework for advancing SPC techniques tailored to modern image data. To be useful, image comparison techniques must have (i) high sensitivity and specificity, (ii) the ability to handle high dimensionality and spatial dependencies through model-based dimension reduction, (iii) flexibility, in the sense that they do not make restrictive parametric assumptions limiting the applicability of the models in real settings, and (iv) computational efficiency, for example, being able to harness the power of efficient computation to analyze high-dimensional images. This paper addresses these critical challenges, providing a practical framework for advancing SPC techniques tailored to modern image data.

To motivate the methodological development, consider three images presented in Figure 1 and consisting of 100×100 pixels each. We denote these images by ${\mathfrak{I}}_1,{\mathfrak{I}}_2$ and ${\mathfrak{I}}_3$, respectively. Although the images may look identical at the first glance, in reality, all three images are different to varying degrees. A visual inspection reveals that differences between images ${\mathfrak{I}}_1$ and ${\mathfrak{I}}_2$ are relatively subtle compared to differences between ${\mathfrak{I}}_1$ and ${\mathfrak{I}}_3$. Intuitively, it appears that random variation could possibly account for the differences between ${\mathfrak{I}}_1$ and ${\mathfrak{I}}_2$. However, although ${\mathfrak{I}}_1$ and ${\mathfrak{I}}_3$ are very similar in most places, there are systematic differences, e.g., the dark area on the lower right side of ${\mathfrak{J}}_1$ has a slightly different shape.

Figure 1:

Are images ${\mathfrak{I}}_1$ and ${\mathfrak{I}}_2$ significantly different? Are images ${\mathfrak{I}}_1$ and ${\mathfrak{J}}_3$ significantly different?

Relevant background and present state of knowledge

There is only limited existing research on monitoring image pairs or sequences, mainly in the chemical and industrial engineering literature, where images have been widely used in recent years (Megahed et al., 2011; Prats-Montalban and Ferrer, 2014; Yan et al., 2015). Existing methods for image comparison often proceed in two main steps. For instance, the methods first extract some features from each image using techniques such as principal component analysis (PCA), and then monitor the extracted features by a conventional control chart (Duchesne et al., 2012; Lin et al., 2008). Some other methods focus on certain prespecified regions in individual images called regions of interest (ROIs), and then monitor the images by a control chart constructed using a summary statistic of the ROIs, e.g., the average image intensity (Jiang et al., 2011; Megahed et al., 2012). The first type of method completely ignores the spatial structure of the images while the second type considers the spatial structure only within the prespecified ROIs. Most of these methods fail to take into account image edges and complicated correlation structures of the spatial image intensities. Consequently, they do not provide a reliable tool for image monitoring applications; see Qiu (2020) for a related discussion. Notable exceptions include Wang and Ye (2010), who compare nonparametric curves under a heteroscedastic model with spatially correlated errors; Koosha et al. (2017), who use nonparametric wavelet basis functions to extract key features and detect change points and fault locations simultaneously; and Roy and Mukherjee (2024), who propose a feature-based image comparison method in which edges and jump points are considered as the primary features. As a side note, in the literatures of image processing, fMRI, and machine learning, there exist methods or algorithms for analyzing a set of images obtained in a given time interval (Feng and Qiu, 2018; Guo and Zhang, 2015; Guo et al., 2018; Julea et al., 2011; Lindquist, 2008). However, these methods are retrospective and cannot be used effectively for prospective monitoring of spatiotemporally correlated image sequences.

There are relatively few Bayesian techniques for image comparison. Nonparametric Bayes techniques that are potentially applicable to this problem typically involve extensions of Dirichlet processes (Ferguson, 1973; Ghosal et al., 1999) to spatial settings; see Reich and Fuentes (2015) for a recent review. These strategies include Gelfand et al. (2005), Duan et al. (2007), Dunson and Park (2008), Griffin and Steel (2006), Rodriguez et al. (2010), and Reich and Bondell (2011). Another class of Bayesian methods (e.g., Na et al., 2013; Chatzis and Tsechpenakis, 2010) applies infinite-dimensional spatial extensions of hidden Markov models to the problem of image segmentation. However, these models have certain limitations. One notable challenge is their sensitivity to the predefined definition of a ‘neighborhood,’ which is assumed to be uniform across the image and known in advance. This assumption can limit the model’s flexibility, as the Markov assumption treats pixels in different neighborhoods as independent. As a result, these models may find it difficult to capture more complex or unknown spatial correlation structures beyond first-order dependencies, including medium- or long-range interactions that can vary across the image space.

Moreover, these Bayesian methods are either broadly applicable spatial models or are primarily designed for the analysis of single images. As a result, they are not well-suited for image comparison, which is the central focus of this paper. Specifically, they lack the ability to account for spatial correlation structures between non-random pixel locations, identify and transfer relevant information across images using unsupervised, model-based dimensionality reduction techniques, or are inefficient at detecting significant deviations from quality standards while considering the spatial nature of images.

Motivated by these challenges, this article develops, in stages, a Bayesian strategy for comparison of high-dimensional image pairs. Section 2 invents a new nonparametric approach for analyzing a single image with multiple intensity values at each of $p$ pixels or voxels (e.g. a colored image). We construct a novel extension of Dirichlet processes, called the spatial random partition model (sRPM), that groups proximal image pixels with similar intensities into the same cluster. The infinite mixture model adapts the general Bayesian frameworks of random partition models and density estimation in the presence of arbitrary covariates (e.g. Müller et al., 2011) to spatial settings and image analysis in particular. The proposed sRPM model favors a small number of relatively large clusters, thereby achieving dimension reduction in the large number of pixels in a manner that accounts for spatial correlations. A key benefit of this dimension reduction is that the extracted image information is useful for answering inferential questions in image sequence monitoring applications.

Section 3 extends the sRPM-based analytical procedure to compare two images that share the same pixel locations but possess possibly different intensity values at multiple pixels. Image comparison is achieved by an intuitively appealing univariate metric for measuring image differences. This metric is not only robust to random response variability in the large number of pixels but also adaptive to spatial correlations. The image comparison problem is then formulated as a single-parameter hypothesis test. This comparison procedure utilizes only cluster-related information previously extracted from the first image, as developed in Section 2. Depending on the degree to which the two images differ, the marginal posterior of this metric is used to call the two images “similar” or “different” based on the posterior probabilities of competing hypotheses. To handle the computational burden in analyzing high-dimensional images, Section 3.1 fosters a two-stage technique for MCMC analysis and hypothesis testing of image pairs. Section 4 analyzes artificial datasets using the proposed technique and finds compelling evidence for the high accuracy of sRPM in image comparison. Section 5 applies the sRPM technique to detect differences between pairs of satellite images. Finally, several remarks conclude the article in Section 6.

2. Bayesian analysis of a single image

For $d\ge 2$, imagine that we have a single $d$-dimensional image, denoted by ${\mathfrak{I}}_1$ and consisting of $p$ pixels. In Figure 1, $d=2$ because ${\mathfrak{I}}_1$ is two-dimensional. However, we are equally interested in analyzing three-dimensional images, such as brain images used in Alzheimer’s disease research. We wish to achieve dimension reduction in the large number of pixels in a manner that accounts for spatial correlation. The extracted information should be subsequently useful for comparisons between ${\mathfrak{I}}_1$ and other images in image sequence monitoring.

2.1. Spatial random partition model (sRPM)

The proposed sRPM defines a coherent process, rather than simply a model. Specifically, the distribution of the responses for any subset of pixels in image ${\mathfrak{I}}_1$ can be obtained from the distribution for a larger set of pixels by marginalization. The common domain of the images is assumed to be a design set denoted by ${\Delta }_p$ containing $p$ regularly or irregularly arranged $d$-dimensional pixels. This design set is assumed to belong to a compact convex set in ${ℛ}^d$ denoted by $𝒟$. For pixel $i=1,\dots ,p$, where $p$ is large, and for image indexed by $t=\mathrm{1,2},\dots$, the $i^{\mathit{\text{th}}}$ pixel is located at ${\mathit{s}}_i={\left(x_i,y_i\right)}^{\prime }\in {\Delta }_p$ for two-dimensional images (i.e., $d=2$) and is associated with a pixel-specific intensity vector ${\mathit{z}}_{it}$ of length $r$. Color images can be represented by triplets, ${\mathit{z}}_{it}=\left(z_{itR},z_{itG},z_{itB}\right)\in [\mathrm{0,1}{]}^3$, corresponding to RGB values. Monochrome images correspond to scalar intensities, so that $z_{it}\in [\mathrm{0,1}]$.

Non-random pixel locations.

We regard the pixel locations as non-random design points and the associated intensities as random, image-specific responses. To achieve dimension reduction in the large number of pixels, $p$, one possibility is to utilize the sparsity-inducing property of Dirichlet processes to group a set of proximal pixels with similar intensities in image ${\mathfrak{J}}_1$ into $q_1$ latent clusters, where $q_1$ is unknown but typically much smaller than $p$. This would drastically reduce the necessary memory storage space from $O\left(p\right)$ to include only the parameters related to the cluster characteristics. The use of Dirichlet processes to achieve dimension reduction has precedence in the literature; see Kim et al. (2006) and Dunson and Park (2008).

However, standard Dirichlet processes are inadequate for image analysis because they treat all $p$ pixels as exchangeable, and they would allocate the pixels to latent clusters regardless of spatial proximity. To accommodate the intrinsic spatial nature of images, we propose extending the Dirichlet process to a spatial random partition model (sRPM) so that only proximal pixels with similar intensities are a posteriori allocated to the same cluster. Applying density estimation regressed on covariates (e.g. Müller et al., 2011) and random partition models, the sRPM process provides a novel framework for image analysis that flexibly adapts to the spatial correlations among the pixels.

Allocation variables ${\mathit{c}}_1=\left({\mathit{c}}_{11},\dots ,{\mathit{c}}_{\mathit{p}1}\right).$

Let the event $\left[c_{i1}=k\right]$ represent pixel $i$ of image ${\mathfrak{I}}_1$ belonging to cluster $k$ in a spatial random partition model, which we will formulate after introducing key concepts and definitions. In the eventual inferential procedure, we will find that an MCMC point estimate of allocation vector ${\mathit{c}}_1$ can be computed if desired.

Aggregated attraction function.

Each cluster is associated with a latent spatial knot and a cluster-specific bandwidth, with individual pixels in image ${\mathfrak{I}}_1$ stochastically allocated to a cluster depending on the pixel-cluster attraction. More specifically, for cluster $k$, the spatial knot is a $d$-dimensional vector ${\mathit{\mu }}_k$, and the bandwidth is a $d\times d$ positive definite matrix, ${\mathbf{\Sigma }}_k$. Since the spatial knots and bandwidths are unknown, we assume normal-inverse Wishart (NIW) priors for these cluster-specific quantities: $\left({\mathit{\mu }}_k,{\mathbf{\Sigma }}_k\right)\stackrel{\mathit{\text{iid}}}{~}{𝒲}_d\left({\mathit{\phi }}_0\right),k=1,\dots ,q_1$, for hyperparameter ${\mathit{\phi }}_0=\left({\mathit{\mu }}_0,{\kappa }_0,{\mathbf{\Sigma }}_0,v_0\right)$ consisting of positive hyperparameters ${\nu }_0$ and ${\kappa }_0,d$-dimensional vector ${\mathit{\mu }}_0$, and $d\times d$ positive-definite matrix ${\mathbf{\Sigma }}_0$.

The pixel-cluster attraction is determined by the pixel’s position relative to the spatial knot ${\mathit{\mu }}_k$, with the relative distance calibrated by cluster bandwidth ${\mathbf{\Sigma }}_k$. In particular, the attraction between pixel $i$ and cluster $k$ is assumed to be equal to the multivariate normal density, $N_d\left({\mathit{s}}_i\mid {\mathit{\mu }}_k,{\mathbf{\Sigma }}_k\right)$. This ensures that the pixel-cluster attraction is greater for pixels near the spatial knot and for clusters with smaller bandwidths. The concept of pixel-cluster attraction is illustrated in Figure 2 for $q=6$ clusters belonging to a rectangular design space in a single image. The areas and orientations of the ellipses are determined by the bandwidths quantified by the positive-definite matrices ${\mathbf{\Sigma }}_k$. The sizes of the dots marking the spatial knots are proportional to the attraction between pixel ${\mathit{s}}_i$ and the $k\text{th}$ cluster.

Figure 2:

Illustration of pixel-cluster attraction for a pixel located at ${\mathit{s}}_i$.

In image ${\mathfrak{I}}_1$, using the pixel-cluster allocation variables, let ${\mathit{s}}_{k1}^{*}=\left\{{\mathit{s}}_i:c_{i1}=k,i=1,\dots ,p\right\}$ denote the pixel locations assigned to the $k\text{th}$ cluster. The spatial compactness of cluster $k$ in image ${\mathfrak{J}}_1$ is quantified by an aggregated attraction function denoted by $g\left({\mathit{s}}_{k1}^{*}\right)$. Larger values of aggregated attraction are associated with tighter spatial clusters, i.e., nearby pixels grouped into smaller clusters. Marginalizing over the unknown cluster-specific knots and bandwidths, and regarding the pixel locations as non-random design points, the aggregated attraction function for cluster $k$ in image ${\mathfrak{J}}_1$ is obtained as follows:

$$g\left({\mathit{s}}_{k1}^{*}\right)=\iint \prod _{i:c_{i1}=k}{N}_d\left({\mathit{s}}_i\mid {\mathit{\mu }}_k,{\mathbf{\Sigma }}_k\right){𝒲}_d\left({\mathit{\phi }}_0\right)d{\mathit{\mu }}_{k}d{\mathbf{\Sigma }}_k=\frac{{\Gamma }_d\left(v_{k1}/2\right)}{{\pi }^{m_{k1}d/2}{\Gamma }_d\left(v_0/2\right)}\frac{{\left|{\mathbf{\Sigma }}_0\right|}^{v_0/2}}{{\left|{\mathbf{\Sigma }}_{k1}\right|}^{v_{k1}/2}}{\left(\frac{{\kappa }_0}{{\kappa }_{k1}}\right)}^{d/2},$$ (1)

where ${\Gamma }_d(\cdot )$ denotes the multivariate gamma function. The new quantities appearing in equation (1) have the following definition. Based on the pixels assigned to the $k\text{th}$ cluster, the updated NIW prior has hyperparameter ${\mathit{\phi }}_{k1}=\left({\mathit{\mu }}_{k1},{\kappa }_{k1},{\mathbf{\Sigma }}_{k1},v_{k1}\right)$, where

$$\begin{gathered} {\mathit{\mu }}_{k1}=\frac{{\kappa }_0}{{\kappa }_0+m_{k1}}{\mathit{\mu }}_0+\frac{m_{k1}}{{\kappa }_0+m_{k1}}{\overline{\mathit{s}}}_{k1},v_{k1}=v_0+m_{k1},{\kappa }_{k1}={\kappa }_0+m_{k1}, \\ {\mathbf{\Sigma }}_{k1}={\mathbf{\Sigma }}_0+{\mathit{V}}_{k1}+\frac{{\kappa }_0{m}_{k1}}{{\kappa }_0+m_{k1}}\left({\overline{\mathit{s}}}_{k1}-{\mathit{\mu }}_0\right){\left({\overline{\mathit{s}}}_{k1}-{\mathit{\mu }}_0\right)}^{\prime }. \end{gathered}$$ (2)

These quantities rely on the following cluster-related summaries for image ${\mathfrak{I}}_1$ : (i) number of pixels, $m_{k1}$, allocated to cluster $k$ by the allocation vector ${\mathit{c}}_1$, (ii) pixel location averages, ${\overline{\mathit{s}}}_{k1}=\sum _{i:c_{i1}=k}{\mathit{s}}_i/m_{k1}$, and (iii) matrix error sum of squares for the pixel locations, ${\mathit{V}}_{k1}=\sum _{i:c_{i1}=k}\left({\mathit{s}}_i-{\overline{\mathit{s}}}_{k1}\right){\left({\mathit{s}}_i-{\overline{\mathit{s}}}_{k1}\right)}^{\prime }$, for $k=1,\dots ,q_1$. Notice that these quantities rely only on the pixel locations but not on the image intensities.

We emphasize that the pixel locations are conceived as non-random design points. By construction, the correlation induced by the spatial knots and bandwidths of the clusters implies that larger values of the attraction function are a priori associated with tighter spatial clusters, i.e., proximal pixels allocated to clusters with smaller areas or volumes, i.e., Lebesgue measures in ${ℛ}^d$.

Spatial random partition model for images.

As mentioned, standard Dirichlet processes are inadequate because they ignore the spatiotemporal nature of the images and regard the intensities at the $p$ pixels as a priori exchangeable. Using the cluster-specific aggregated attraction functions in (1), we extend the standard Dirichlet process to a spatial random partition model (sRPM) for the pixel-cluster allocations in image ${\mathfrak{I}}_1$ :

$$\left[{\mathit{c}}_1\right]\propto {\alpha }^{q_1}\prod _{k=1}^{q_1}g\left({\mathit{s}}_{k1}^{*}\right)\Gamma \left(m_{k1}\right),{\mathit{c}}_1\in {ℬ}_p,$$ (3)

where $[\cdot ]$ denotes density functions, ${ℬ}_p$ is the set of possible partitions of $p$ pixels, and $\alpha >0$ is the mass parameter.

In other words, allocation vector ${\mathit{c}}_1=\left(c_{11},\dots ,c_{p1}\right)$ is a priori distributed as the partitions induced on the $p$ image pixels by sRPM. Furthermore, attraction function (1) ensures that a pixel is a priori more likely to join clusters containing a larger number of spatially proximal pixels. Summing over all possible partitions of the $p$ pixels in image ${\mathfrak{I}}_1$, the normalization constant for density (3) is $\sum _{{\mathit{c}}_1\in {ℬ}_p}{\alpha }^{q_1}\prod _{k=1}^{q_1}g\left({\mathit{s}}_{k1}^{*}\right)\Gamma \left(m_{k1}\right)$.

Intensities.

Finally, we specify the sampling distribution of the intensity or response vector ${\mathit{z}}_{i1}=\left(z_{i11}\right.,\dots ,z_{ir1})$, for $i=1,\dots ,p$. Conditional on the pixel-cluster allocations of image ${\mathfrak{I}}_1$, the intensity vectors of the member pixels in cluster $k$ are modeled as a shared cluster-specific mean vector, ${\mathit{v}}_k=\left(v_{k1},\dots ,v_{kr}\right)$, plus Gaussian white noise. More specifically, conditional on the event that the $i^{\mathit{\text{th}}}$ pixel belongs to the $k\text{th}$ cluster, ${\mathit{z}}_{i1}\mid \left\{c_{i1}=k\right\}\stackrel{iid}{~}{N}_r\left({\mathit{v}}_k,{\sigma }^2\mathit{I}\right)$, with ${\sigma }^2$ assigned an inverse chi-square prior. Small values of $\sigma$ guarantee that the clusters consist of pixels with similar intensities. Consequently, the posterior distribution of sRPM tends to group proximal image pixels with similar intensities into the same cluster, and tends to favor a small number of relatively large clusters, thereby achieving dimension reduction.

For cluster $k=1,\dots ,q_1$, let mean vectors ${\mathit{v}}_k\stackrel{\mathit{\text{iid}}}{~}{N}_r\left({\mathit{v}}_0,{\mathbf{\Sigma }}_0\right)$ in the prior. Standard conditionally conjugate priors are imposed on all the remaining hyperparameters to complete the model specification. Notice that a common $\sigma$ parameter for the $r$ intensity components presupposes that the intensities have comparable scales; this may require standardizing each intensity component before analysis. Despite the apparently strong parametric assumption of homoscedastic Gaussian errors in the sRPM model, Dirichlet processes and many of their extensions, including sRPM, are intrinsically nonparametric and capable of flexibly adapting to complex-structured regression surfaces (Lijoi and Prünster, 2010).

2.2. Posterior inferences for image ${\mathfrak{I}}_1$

The following Proposition establishes the posterior equivalence of two models with very different interpretations. The proof is presented in Supplementary Material. Applying well-established sampling procedures, the sRPM parameters are then updated in a relatively straightforward manner.

Proposition 1.

Consider an alternative model in which (i) sampling distribution $\left({\mathit{s}}_i,{\mathit{z}}_{i1}\right)\mid \left\{c_{i1}=k\right\}\stackrel{\mathit{\text{indep}}}{~}{N}_d\left({\mathit{\mu }}_k,{\mathbf{\Sigma }}_k\right)\times N_r\left({\mathit{v}}_k,{\sigma }^2\mathit{I}\right)$, and (ii) allocation vector ${\mathit{c}}_1=\left(c_{11},\dots ,c_{p1}\right)$ is distributed as the partitions induced by a regular Dirichlet process prior with mass parameter $\alpha$ and base distribution ${𝒲}_d\left({\mathit{\phi }}_0\right)\times N_r\left({\mathit{\nu }}_0,{\mathbf{\Sigma }}_0\right)$. This model regards the pixel locations ${\mathit{s}}_1,\dots ,{\mathit{s}}_p$ as random, unlike sRPM, for which ${\Delta }_p$ is the design set of pixel locations. Nevertheless, for every $p$, given the responses and set ${\Delta }_p$, the posterior distribution of the remaining parameters of sRPM and the alternative model are identical.

Intuitively, posterior inferences about the key parameters of interest are equivalent under the alternative model due to the high degree of separability between the pixel locations and image intensities in the two models. Starting with ad hoc initial values, the sRPM model parameters for image ${\mathfrak{I}}_1$ are then iteratively generated by Gibbs sampling steps from their full conditionals. The post-burn-in MCMC sample is used for posterior inference. We briefly outline the Gibbs sampling steps below for some of the parameters:

1. Allocation vector ${\mathit{c}}_1$. For the allocation vector of the alternative posterior distribution described in Proposition 1, this parameter vector is updated using the Gibbs sampler for finite-dimensional Dirchlet process representation of Ishwaran and James (2001).

2. Spatial knots and bandwidths. Given the data and the current values of the remaining model parameters including allocation vector ${\mathit{c}}_1$, the spatial knots and bandwidths have independent NIW distributions: $\left({\mathit{\mu }}_k,{\mathbf{\Sigma }}_k\right)\stackrel{\mathit{\text{indep}}}{~}{𝒲}_d\left({\mathit{\phi }}_{k1}\right)$, for $k=1,\dots ,q_1$, with the updated NIW hyperparameter ${\mathit{\phi }}_{k1}$ described after equation (1).

Cluster-related posterior inferences.

By using the MCMC sample and applying the least-squares method of Dahl (2006), we can compute ${\mathit{c}}_1$, an estimate of the pixel-cluster allocation vector for image ${\mathfrak{I}}_1$. Intuitively, this procedure first uses the MCMC sample to estimate $\mathit{A}$, the “adjacency matrix” whose elements represent the estimated posterior probabilities that each pixel pair belongs to a common cluster. Next, using an MCMC sample, estimate ${\mathit{c}}_1$ is chosen to minimize the Frobenius norm of the difference between $\stackrel{\mathit{ˆ}}{\mathit{A}}$ and the adjacency matrix associated with ${\mathit{c}}_1$. Since they are straightforward deterministic functions of ${\mathit{c}}_1$, the estimated number of clusters, ${\hat{q}}_1$, and the estimated numbers of pixels allocated to the clusters, ${\hat{m}}_{11},\dots ,{\hat{m}}_{1{\hat{q}}_1}$, are immediately available. Furthermore, estimates of the spatial knots, ${\mathit{\mu }}_1,\dots ,{\mathit{\mu }}_{{\hat{q}}_1}$, and bandwidths, ${\mathbf{\Sigma }}_1,\dots ,{\mathbf{\Sigma }}_{{\hat{q}}_1}$, can be computed. For example, for image ${\mathfrak{I}}_1$ of Figure 1, the detected cluster characteristics, along with image ${\mathfrak{I}}_1$ for comparison, are graphically presented in Figure 2. sRPM detected ${\hat{q}}_1=57$ spatial clusters, achieving impressive dimension reduction in the 10, 000 image pixels.

3. Comparison of image pairs

We will extend the sRPM-based analytical procedure of Section 2 to the comparison of two images. Suppose that the images, denoted by ${\mathfrak{I}}_1$ and ${\mathfrak{I}}_2$, share the same pixel locations but possess possibly different intensities at multiple pixel locations. Thus, design set ${\Delta }_p\subset {ℛ}^d$ consists of $p$ regularly or irregularly arranged pixels that are common to both images. Image comparison is achieved by an intuitively appealing measure for image differences that is not only robust to random response variability of the large number of pixels but is also adaptive to spatial correlation. Furthermore, the comparisons utilize only cluster-related information extracted from ${\mathfrak{J}}_1$, as described in Section 2. Depending on the degree to which the two images are different, the marginal posterior of this measure is used to call the images “similar” or “different” using the posterior probabilities of competing hypotheses.

Propagation of information from image ${\mathfrak{I}}_1$ to ${\mathfrak{I}}_2$.

One of the key features of the Bayesian paradigm is its natural ability to borrow strength from comparable or somewhat similar data. Applied to the image comparison problem, this implies that conditional on image ${\mathfrak{I}}_1$, the posterior of ${\mathfrak{I}}_1$ plays the role of the prior for ${\mathfrak{I}}_2$. For example, the pixels in image ${\mathfrak{I}}_2$ may join any of the $q_1$ clusters already discovered in image ${\mathfrak{J}}_1$ (if the pixels belong to regions of design space ${\Delta }_p$ where the two images are somewhat similar) or may join a new set of clusters unique to image ${\mathfrak{I}}_2$ (if the pixels belong to regions where the two images are very different). If ${\mathfrak{I}}_2$ is associated with $q_2$ clusters, then $q_2\ge q_1$ because we condition on the $q_1$ clusters previously discovered in ${\mathfrak{J}}_1$. The first $q_1$ of these clusters are shared with ${\mathfrak{J}}_1$ whereas the remaining $\left(q_2-q_1\right)$ clusters are unique to ${\mathfrak{J}}_2$. In the extreme situation where the images are very different, $q_2>q_1$, and none of the $q_1$ clusters of ${\mathfrak{I}}_1$ are occupied in ${\mathfrak{I}}_2$. At the other extreme, when the two images are nearly identical, $q_2=q_1$, and only the $q_1$ clusters of ${\mathfrak{I}}_1$ are occupied by the $p$ pixels of ${\mathfrak{I}}_2$. Consequently, the latent spatial knots and bandwidths of the image ${\mathfrak{I}}_2$ clusters have the following independent normal-inverse Wishart (NIW) conditional priors:

$$\left({\mathit{\mu }}_k,{\Sigma }_k\right)|{\Im }_1\stackrel{\mathit{\text{indep}}}{~}\{\begin{array}{ll}{𝒲}_d\left({\mathit{\phi }}_{k1}\right),& k=1,\dots ,q_1,\\ {𝒲}_d\left({\mathit{\phi }}_0\right),& k=\left(q_1+1\right),\dots ,q_2,\end{array}$$ (4)

where the components of the NIW hyperparameter ${\mathit{\phi }}_{k1}$ is described in equation (2) and estimated using the Gibbs sampler for image ${\mathfrak{I}}_1$. Extending the earlier notation for allocation variables, for $i=1,\dots ,p$, let $\left[c_{i2}=k\right]$ represent the event that pixel $i$ of image ${\mathfrak{I}}_2$ belongs to cluster $k$, and let ${\mathit{s}}_{k2}^{*}=\left\{s_i:c_{i2}=k\right\}$ be the set of pixel locations belonging to cluster $k$. It can be shown that the conditional prior (4) gives the following expression for the aggregated attraction function:

$$g\left({\mathit{s}}_{k2}^{*}\right)=\frac{1}{{\pi }^{m_{k2}d/2}}\times \{\begin{array}{ll}\frac{{\Gamma }_d\left(v_{k2}/2\right)}{{\Gamma }_d\left(v_{k1}/2\right)}\frac{{\left|{\Sigma }_{k1}\right|}^{V_{k1}/2}}{{\left|{\Sigma }_{k2}\right|}^{V_{k2}/2}}{\left(\frac{{\kappa }_{k1}}{{\kappa }_{k2}}\right)}^{d/2},& k=1,\dots ,q_1,\\ \text{\%\%}\frac{{\Gamma }_d\left(v_{k2}/2\right)}{{\Gamma }_d\left(v_0/2\right)}\frac{{\left|{\Sigma }_0\right|}^{V_0/2}}{{\left|{\Sigma }_{k2}\right|}^{V_{k2}/2}}{\left(\frac{{\kappa }_0}{{\kappa }_{k2}}\right)}^{d/2},& k=\left(q_1+1\right),\dots ,q_2.\end{array}$$ (5)

The new quantities in equation (5) are defined as follows. Based on the assigned pixels, the updated NIW prior has hyperparameters ${\mathit{\phi }}_{k2}=\left({\mathit{\mu }}_{k2},{\mathit{\kappa }}_{k2},{\mathbf{\Sigma }}_{k2},v_{k2}\right)$ with components

$$\begin{gathered} {\mathit{\mu }}_{k2}=\frac{m_{k1}}{m_{k1}+m_{k2}}{\mathit{\mu }}_{k1}+\frac{m_{k2}}{m_{k1}+m_{k2}}{\overline{\mathit{s}}}_{k2},v_{k2}=v_{k1}+m_{k2},{\kappa }_{k2}={\kappa }_{k1}+m_{k2}, \\ {\mathbf{\Sigma }}_{k2}={\mathbf{\Sigma }}_{k1}+{\mathit{V}}_{k2}+\frac{{\kappa }_{k1}{m}_{k2}}{{\kappa }_{k1}+m_{k2}}\left({\overline{\mathit{s}}}_{k2}-{\mathit{\mu }}_{k2}\right){\left({\overline{\mathit{s}}}_{k2}-{\mathit{\mu }}_{k2}\right)}^{\prime }. \end{gathered}$$ (6)

These quantities rely on the following cluster-related summaries from image ${\mathfrak{I}}_2$ : (i) number of pixels, $m_{k2}$, allocated to cluster $k$ by the allocation vector ${\mathit{c}}_2$, (ii) pixel location averages, ${\overline{\mathit{s}}}_{k2}=\sum _{i:C_{i2}=k}{\mathit{s}}_i/m_{k2}$, and (iii) matrix error sum of squares for the pixel locations, ${\mathit{V}}_{k2}=\sum _{i:c_{i2}=k}\left({\mathit{s}}_i-{\overline{\mathit{s}}}_{k2}\right){\left({\mathit{s}}_i-{\overline{\mathit{s}}}_{k2}\right)}^{\prime }$, for $k=1,\dots ,q_2$. Then, the conditional sRPM for the pixel-cluster allocations of image ${\mathfrak{I}}_2$ becomes

$$\left[{\mathit{c}}_2\mid {\mathit{c}}_1\right]\propto {\alpha }^{q_2}\prod _{k=1}^{q_2}g\left({\mathit{s}}_{k2}^{*}\right)\Gamma \left(m_{k1}+m_{k2}\right),{\mathit{c}}_2\in {ℬ}_p,$$ (7)

where $m_{k1}=0$ for $k>q_1$, i.e., the new clusters of ${\mathfrak{I}}_2$ were empty in ${\mathfrak{J}}_1$. Analogously as in image ${\mathfrak{J}}_1$, the intensity or response vector ${\mathit{z}}_{i2}=\left(z_{i12},\dots ,z_{ir2}\right)$ is distributed as ${\mathit{z}}_{i2}\mid \left\{c_{i2}=k\right\}\stackrel{\mathit{\text{iid}}}{~}{N}_r\left({\mathit{v}}_k,{\sigma }^2\mathit{I}\right)$. For $k=1,\dots ,q_1$, the prior for mean vector ${\mathit{v}}_k$ coincides with its posterior from image ${\mathfrak{I}}_1$. For the new clusters indexed by $k=\left(q_1+1\right),\dots ,q_2$, the mean vectors have the previously specified no-data prior, ${\mathit{v}}_k\stackrel{\mathit{\text{iid}}}{~}{N}_r\left({\mathit{v}}_0,{\mathbf{\Sigma }}_0\right)$.

A quantitative measure for image comparisons.

Using the parameters of the conditional model for image ${\mathfrak{I}}_2$, specifically, the random number of pixels of ${\mathfrak{I}}_2$ allocated to the spatial clusters, we define the differential pixel proportion of image ${\mathfrak{I}}_2$ to be

$$\psi =\frac{1}{p}\sum _{i=1}^pℐ\left(c_{i1}\ne c_{i2}\right),$$ (8)

where $ℐ(\cdot )$ is the indicator function. This quantity measures the relative area or volume of the regions in ${ℛ}^d$ where images ${\mathfrak{I}}_1$ and ${\mathfrak{I}}_2$ are different; $\psi$ values near 0 (1) are indicative of similar (dissimilar) image pairs. Since the pixel-cluster allocations are a denoised version of the image after accounting for spatial correlation, this advantage is inherited by the measure $\psi$. Image comparison problem can be formulated as a single-parameter hypothesis test:

$$H_0:\psi \le {\psi }^{*}\left(“\text{similar}\text{images}”\right)\text{versus}{\mathrm{H}}_1:\psi >{\psi }^{*}\left(“\text{different}\text{images}”\right),$$ (9)

for some prespecified threshold ${\psi }^{*}\in \left(\mathrm{0,1}\right)$. For image comparisons, the key parameter of interest is $P\left(\psi >{\psi }^{*}\mid {\mathfrak{I}}_1,{\mathfrak{I}}_2\right)$, i.e. the posterior probability of hypothesis $H_1$. The two images are declared different (similar) if this posterior probability does (not) exceed 0.5.

3.1. Posterior inferences for differential proportion $\psi$

Due to the computational costs of analyzing high-dimensional images, we perform combined MCMC analysis and hypothesis testing of the two images in separate stages:

• Stage 1: First, image ${\mathfrak{I}}_1$ is analyzed using the computationally efficient MCMC strategy described in Section 2.2. In this manner, we obtain ${\mathit{c}}_1$, an estimate of the pixel-cluster allocation vector of ${\mathfrak{I}}_1$. From the estimated allocations, we compute the estimated number of clusters ${\hat{q}}_1$ and the estimated cluster sizes ${\hat{m}}_{11},\dots ,{\hat{m}}_{1{\hat{q}}_1}$. For estimating these important parameters, a detailed description is presented in Section 2.2.

• Stage 2a: Assuming all parameters specific to image ${\mathfrak{I}}_1$ to be equal to their posterior estimates, image ${\mathfrak{I}}_2$ is analyzed using the Gibbs sampler of Ishwaran and James (2001) applied to the conditional sRPM (7). The Monte Carlo scheme is analogous to that for ${\mathfrak{I}}_1$, and the computational costs of the two MCMC samplers are identical.

• Stage 2b: For the $l^{\mathit{\text{th}}}$ Gibbs sampling update of the ${\mathfrak{J}}_2$ parameters, where $l=1,\dots ,L$, let $m_{12}^{\left(l\right)},\dots ,m_{q_2^{\left(l\right)}2}^{\left(l\right)}$ denote the sizes of the $q_2^{\left(l\right)}$ clusters. Then, ${\psi }^{\left(l\right)}=\sum _{k={\hat{q}}_1+1}^{q_2^{\left(l\right)}}{m}_{k2}^{\left(l\right)}/p$ represents the $l^{\left(th\right)}$ MCMC draw for the parameter $\psi$. The values ${\psi }^{\left(1\right)},\dots ,{\psi }^{\left(L\right)}$ estimate the marginal posterior density of $\psi$ and the estimated posterior probability $\hat{P}\left(\psi >{\psi }^{*}\mid {\mathfrak{I}}_1,{\mathfrak{I}}_2\right)=\frac{1}{L}\sum _{l=1}^{L}I\left({\psi }^{\left(l\right)}>{\psi }^{*}\right)$, is immediately available. Images ${\mathfrak{I}}_1$ and ${\mathfrak{I}}_2$ are called different or similar depending on whether or not $\hat{P}\left(\psi >{\psi }^{*}\mid {\mathfrak{I}}_1,{\mathfrak{I}}_2\right)$ exceeds 0.5 .

4. Simulation studies

To investigate the accuracy of sRPM procedure in labeling images as “similar” or “different,” we generated 500 sets of truly similar and truly different image pairs (i.e. 1,000 images in total), with each monochrome image consisting of 100×100 pixels each. For each image, the $p=\mathrm{10,000}$ pixel signals were generated from an adaptation of the Potts model (Hurn et al., 2003) with $q^{*}=45$ hidden states on the square lattice. A Potts model induces spatial correlation in the pixel intensities in a very different manner than sRPM. For hidden states $v_1,\dots ,v_p\in {\left\{1,\dots ,q^{*}\right\}}^p$, the model induces spatial correlation in the hidden states by borrowing information from neighboring pixels. Let ${𝒩}_i$ be the pixel indices surrounding pixel $i$, so that ${𝒩}_i$ has 3, 5, and 8 elements depending on whether $i$ belongs to the image interior, edge, or corner. Specifically, $\left[v_1,\dots ,v_p\right]=\frac{1}{𝒞\left(\mathbf{\Lambda }\right)}\text{exp}\left[\sum _{i=1}^p\sum _{u\in {𝒩}_i}{\lambda }_{v_u{v}_i}\right]$, where $v_i\in \left\{1,\dots ,q^{*}\right\},i=1,\dots ,p$, with ${\lambda }_{v_u{v}_i}>0$ being a similarity measure between hidden states $v_u$ and $v_i$ in a diagonally dominant $q^{*}\times q^{*}$ matrix, $\mathbf{\Lambda }=\left(\left({\lambda }_{v_i{v}_i}\right)\right)$. The similarity measures were related to the pairwise distances of $q$ randomly generated points on the square lattice. The normalizing constant is denoted by $𝒞\left(\lambda \right)$. Due to its intractability, we generated a sample from the Potts density as the last draw of the corresponding Gibbs sampler.

The $q^{*}$ hidden signals associated with the hidden states were generated i.i.d. from the standard normal distribution. Finally, zero-mean Gaussian noise was added to the pixel-specific hidden signals to obtain the continuous image intensities, which were then standardized to belong to the interval [0, 1]. Truly similar images shared the same hidden signals but differed in the pixel-specific noise. Truly different images not only differed in pixel-specific noise, but also had slightly different similarity measures in the Potts densities.

Following image generation, the posterior inference strategy outlined in Section 3 was implemented to analyze each image pair using the sRPM technique and call each image pair as similar or different. Averaging over the 500 datasets, the blue line in Figure 4 displays the receiver operating characteristic (ROC) curve of accurate calls, along with 95% confidence bands, as the threshold similarity ${\psi }^{*}$ was increased from 0 to 1. As is well known, the area under the curve (AUC) close to 1 is indicative of a method’s reliability. The estimated AUC for sRPM was 0.928 and a 95% confidence interval was (0.914, 0.942).

Figure 4:

Estimated ROC curves of the sRPM and Feng and Qiu (2018) approaches for the 500 artificial datasets of the simulation study. The dotted lines represent 95% confidence bands for the ROC curves.

Minimizing the distance from the ROC curve to the upper left corner of the square, we obtained the optimal threshold value of ${\psi }^{*}=0.183$. The percentage of correct and incorrect calls for the optimal ${\psi }^{*}$ is displayed in Table 1. image comparison. In extensive simulation studies, we have obtained reliable inferences for threshold values belonging to the interval, [0.1, 0.2]. For image comparison in real datasets, we recommend selecting the threshold ${\psi }^{*}$ of hypothesis test (9) in this range. For comparison, we analyzed the same datasets using the image comparison approach based on continuity regions (Feng and Qiu, 2018). Averaging over the 500 datasets, the red line of Figure 4 displays the ROC curve with 95% confidence bands. For the Feng and Qiu (2018) method, the estimated AUC was 0.801, with a 95% confidence interval of (0.770, 0.831), demonstrating the significantly greater effectiveness of sRPM in image comparisons. Minimizing the distance of the competing method’s ROC curve to the upper left corner of the square in Figure 4, Table 1 displays the optimal percentage of correct and incorrect image pairs calls for the Feng and Qiu (2018) approach. These results reveal that the sRPM method has approximately balanced levels of sensitivity (i.e., ability to detect differences) and specificity (i.e., ability to detect similarities). In contrast, the Feng and Qiu (2018) strategy has significantly lower sensitivity and specificity. These findings reveal that, at least for the types of images investigated here, the sRPM method is more conservative than the Feng and Qiu (2018) strategy in rejecting the null hypothesis of image similarity.

Table 1:

Averaging over the 500 artificial image pairs of the simulation study, the optimal detection accuracies of the sRPM and Feng and Qiu (2018) methods for image comparison. See the text for further explanation. Shown in parentheses are standard errors.

sRPM
		Truth
		Similar	Different
Detected	Similar	86.4% (1.5%)	13.6% (1.5%)
	Different	14.0% (1.6%)	86.0% (1.6%)
Feng and Qiu (2018)
		Truth
		Similar	Different
Detected	Similar	78.4% (1.8%)	21.6% (1.8%)
	Different	29.8% (2.0%)	70.2% (2.0%)

5. Analysis of satellite images

Chicago area images

Figure 5 displays monochrome satellite images of the Chicago area taken in 1990 (image ${\mathfrak{I}}_1$) and in 1999 (image ${\mathfrak{J}}_2$). The data are available at available at https://users.phhp.ufl.edu/pqiu/research/book/data/index.html. From an informal visual comparison, we find that the two images have a few differences. Following the 9-year interval between the images, there appear to be more dark spots in image ${\mathfrak{I}}_2$. These changes may be due to environmental changes or new construction. Some dark spots in image ${\mathfrak{I}}_1$ also appear in image ${\mathfrak{I}}_2$, although their sizes have changed, e.g., in the lower portions of the images. Most of the differences between the images are fairly small in magnitude, difficult to describe using a parametric model, and are mostly local, affecting only small portions of the images.

Figure 5:

Reference satellite image ${\mathfrak{J}}_1$ was taken in 1990 in the Chicago area. Satellite image ${\mathfrak{I}}_2$ of the same area taken in 1999.

For calling the two images similar or different, we focused on the differential pixel proportion, $\psi$, defined in equation (8), and performed hypothesis test (9) to obtain an MCMC estimate of posterior probability $P\left(\psi >{\psi }^{*}\mid {\mathfrak{I}}_1,{\mathfrak{I}}_2\right)$. As mentioned earlier, extensive simulation studies have suggested a threshold value ${\psi }^{*}$ belonging to the interval [0.1, 0.2] to obtain reliable inferences. In general, two images were declared to different or similar depending on whether or not the estimate $\hat{P}\left(\psi >{\psi }^{*}\mid {\mathfrak{I}}_1,{\mathfrak{I}}_2\right)$ exceeded 0.5.

For the Chicago area images, the posterior inference procedure outlined in Section 3.1 was implemented to obtain the marginal posterior of differential parameter $\psi$. An MCMC estimate of the marginal posterior is shown in the left panel of Figure 6. The right panel of Figure 6 plots the estimated posterior probability $\hat{P}\left(\psi >{\psi }^{*}\mid {\mathfrak{I}}_1,{\mathfrak{I}}_2\right)$ as ${\psi }^{*}$ varies over the range [0, 0.5]. We find that as ${\psi }^{*}$ increases, the posterior probability stays level at approximately 1 before falling sharply as ${\psi }^{*}$ approaches 0.5. That is, irrespective of the prespecified threshold ${\psi }^{*}\in [0.1,0.2]$, the estimated posterior probability exceeds 0.99, which is a Bayes factor greater than 100 if we assign equal prior probabilities to the competing hypotheses. This is decisive evidence that the Chicago area satellite images are different.

Figure 6:

For the Chicago area images, the left panel displays an MCMC estimate of the marginal posterior of differential parameter $\psi$. The right panel displays an estimate of the posterior probability $P\left(\psi >{\psi }^{*}\mid {\mathfrak{I}}_1,{\mathfrak{I}}_2\right)$ for different values of the threshold ${\psi }^{*}$.

Bay Area images

Figure 7 displays two satellite images of the San Francisco Bay Area in 1990 and 1999. Figure 1 of Supplementary Material displays the estimated marginal posterior of differential parameter $\psi$. We find negligible posterior mass assigned to $\psi$ smaller than 0.48. Consequently, the estimated probability $\hat{P}\left(\psi >{\psi }^{*}\mid {\mathfrak{I}}_1,{\mathfrak{I}}_2\right)$ was equal to 1 for any prespecified threshold ${\psi }^{*}\in [\mathrm{0.1,0.2}]$, and is overwhelming evidence that the two satellite images are different.

Figure 7:

In the left panel, satellite image ${\mathfrak{J}}_1$ of the San Francisco bay area, taken in 1990. In the right panel, satellite image ${\mathfrak{I}}_2$ of the same region, taken in 1999.

6. Discussion

There is a critical need for statistical techniques that can reliably identify “similar” or “different” image pairs depending on whether random measurement error is able to account for pixel-wise differences between the images. We have developed a Bayesian strategy relying on a novel extension process called the spatial random partition model (sRPM). This process achieves dimension reduction while accounting for spatial correlation by grouping proximal image pixels with similar intensities into clusters. The extracted image information is then utilized for image comparison via a single-parameter hypothesis test based on a univariate metric that is adaptive to pixel intensity variation and spatial correlation. We foster a computationally efficient two-stage MCMC technique. A simulation study analyzes artificial datasets and finds compelling evidence for the high sensitivity and specificity of sRPM. The success of sRPM is demonstrated by analyzing satellite image data. R code implementing the method is available at https://github.com/sguha-lab/sRPM.

The nonparametric nature of sRPM makes it highly effective in analyzing images with diffuse spatially differentiated regions. However, the method may be challenged by images with sharply demarcated features (e.g., text images) or jagged images with steep slopes. In applications where a sequence of multiple images is available from a longitudinal process, interest often focuses on detecting temporal change points, if any, at which the longitudinal process diverges beyond acceptable limits of variation. We are currently developing natural extensions of the proposed sRPM analytical framework in such a manner that, at any point in the image sequence, only cluster-related information that has been progressively extracted from the preceding images is used to determine whether the current time point is a change point. It is often known that the first few images of the sequence correspond to an “in-control” condition of the longitudinal process. In these situations, we are devising strategies that exploit this information to further improve the inferential accuracy of monitoring image sequences. The MCMC strategy of Ishwaran and James (2001) is adequate for the image sizes of this paper. However, for ultra high-dimensional images, these algorithms are computationally intensive. We are exploring whether the Metropolis-Hastings algorithm proposed by Guha (2010) could potentially be applied to significantly reduce computational times while ensuring that the posterior mixing rates and ESS in the sense of Turek et al. (2017) are satisfactory.

Figure 3:

Image ${\mathfrak{J}}_1$ in Figure 1 and its detected cluster characteristics by the sRPM technique.