Point Divergence Gain and Multidimensional Data Sequences Analysis

Abstract

We introduce novel information-entropic variables—a Point Divergence Gain (${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$), a Point Divergence Gain Entropy ($I_{\alpha }$), and a Point Divergence Gain Entropy Density ($P_{\alpha }$)—which are derived from the Rényi entropy and describe spatio-temporal changes between two consecutive discrete multidimensional distributions. The behavior of ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$ is simulated for typical distributions and, together with $I_{\alpha }$ and $P_{\alpha }$, applied in analysis and characterization of series of multidimensional datasets of computer-based and real images.

1. Introduction

Extracting the information from raw data obtained from, e.g., a set of experiments, is a challenging task. Quantifying the information gained by a single point of a time series, a pixel in an image, or a single measurement is important in understanding which points bring the most information about the underlying system. This task is especially delicate in case of time-series and image processing because the information is not only stored in the elements, but also in the interactions between successive points in a time series. Similar, when extracting information from an image, not all pixels have the same information content. This type of information is sometimes called local information because the information depends not only on the frequency of the phenomenon but also on the position of the element in the structure. The most important task is to identify the sources of information and to quantify them. Naturally, it is possible to use standard data-processing techniques based on quantities from information theory like, e.g., Kullback–Leibler divergence. On the other hand, the mathematical rigorousness is typically compensated by an increased computational complexity. For this end, a simple quantity called Point Information Gain and its relative macroscopic variables—a Point Information Gain Entropy and a Point Information Gain Entropy Density—were introduced in [1]. In [2], mathematical properties of the Point Information Gain were extensively discussed and applications to real-image data processing were pointed out. From the mathematical point of view, the Point Information Gain represents a change of information after removing an element of a particular phenomena from a distribution. The method is based on the Rényi entropy, which has been already extensively used in multifractal analysis and data processing (see e.g., Refs. [2,3,4,5] and references therein).

In this article, we introduce an analogous variable to the Point Information Gain. This new variable locally determines an information change after an exchange of a given element in a discrete set. We use a simple concept of entropy difference between the original set and the set with the exchanged element. The resulting value is called Point Divergence Gain ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$ [6,7]. The main idea is to describe the importance of changes in the series of images (typically representing a video record from an experiment) and extract the most important information from it. Similar to the Point Information Gain Entropy and the Point Information Gain Entropy Density, the macroscopic variables called a Point Divergence Gain Entropy $I_{\alpha }$ and a Point Divergence Gain Entropy Density $P_{\alpha }$ are defined to characterize subsequent changes in a multidimensional discrete distribution by one number. The goal of this article is to examine and demonstrate some properties of these variables and use them for examination of time-spatial changes of information in sets of discrete multidimensional data, namely series of images in image processing and analysis, after the exchange of a pixel of a particular intensity for a pixel at the same position in the consecutive image. The main reason for choosing the Point Divergence Gain as the relevant quantity for the analysis of spatio-temporal changes is the fact that it represents an information gain of each pixel change. One can also consider model-based approaches based on the theory of random-fields, which can be more predictive in some cases. On the other hand, the model-free approach based on entropy gives us typically more relevant information for real data, where it is typically difficult to find an appropriate model. For the overview of model-based approaches in the random field theory, one can consult, e.g., Refs. [8,9,10].

The paper is organized as follows: in Section 2, we define the main quantity of the paper, i.e., the Point Divergence Gain and the related quantities and discuss its theoretical properties. In Section 3, we show applications of the Point Divergence Gain to image processing for both computer-based and real sequences of images. We show that the Point Divergence Gain can be used as a measure of difference for clustering methods and detects the most prominent behaviour of a system. In Section 4, we explain the presented methods and finer technical details necessary for the analysis including algorithms. Section 5 is dedicated to conclusions. All image data, scripts for histogram processing, and Image Info Extractor Professional software for image processing are available via sftp://160.217.215.193:13332/pdg (user: anonymous; password: anonymous.).

2. Basic Properties of Point Divergence Gain and Derived Quantities

2.1. Point Divergence Gain

Recently, a quantity called Point Information Gain (PIG, ${\Gamma }_{\alpha }^{\left(i\right)}$) [6,7] and its generalization based on the Rényi entropy [2] have been introduced. We show how to apply the concept of PIG to sequence of multidimensional data frames.

Let us assume a set of variables with k possible outcomes (e.g., possible colours of each pixel). The ${\Gamma }_{\alpha }^{\left(i\right)}$ is a simple variable based on entropy difference and enables us to quantify an information gain of each phenomenon. It is simply defined as a difference between entropy of an original discrete distribution

$$P={\left\{p_j\right\}}_{j=1}^k=\left\{\frac{n_1}{n},\dots ,\frac{n_k}{n}\right\},$$ (1)

which typically describes a frequency histogram of possible outcomes. Let us also define a distribution, where one occurrence of the i-th phenomenon is omitted, i.e.,

$$P^{\left(i\right)}={\left\{p_j^{\left(i\right)}\right\}}_{j=1}^k=\left\{\frac{n_1}{n-1},\dots ,\frac{n_i-1}{n-1},\dots ,\frac{n_k}{n-1}\right\}.$$ (2)

Thus, the Point Information Gain is defined as

$${\Gamma }_{\alpha }^{\left(i\right)}\equiv {\Gamma }_{\alpha }^{\left(i\right)}\left(P\right)={\mathcal{H}}_{\alpha }\left(P^{\left(i\right)}\right)-{\mathcal{H}}_{\alpha }\left(P\right),$$ (3)

where ${\mathcal{H}}_{\alpha }$ is the Rényi entropy (Despite all computer implementations being calculated as ${log}_2$, the following derivations are written in natural logarithm, i.e., ln.)

$${\mathcal{H}}_{\alpha }\left(P\right)=\frac{1}{\alpha -1}ln\sum _i{p}_i^{\alpha }.$$ (4)

The Rényi entropy represents a one-parametric class of information quantities tightly related to multifractal dynamics and enables us to focus on certain parts of the distribution [11]. Unlike the typically used Rényi’s relative entropy [3,4,11,12,13,14,15,16,17], the Point Information Gain ${\Gamma }_{\alpha }^{\left(i\right)}$ is a simple, computationally tractable quantity. Its mathematical properties have been extensively discussed in [2]. On the same basis, we can define a Point Divergence Gain (PDG, ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$), where a discrete distribution $P^{\left(i\right)}$ is replaced by a distribution

$$P^{(l\to m)}={\left\{p_j^{(l\to m)}\right\}}_{j=1}^k=\left\{\frac{n_1}{n},\dots ,\frac{n_l-1}{n},\dots ,\frac{n_m+1}{n},\dots ,\frac{n_k}{n}\right\},$$ (5)

which can be obtained from the original distribution P, where the occurrence of the examined l-th phenomenon ($n_l\in {\mathbb{N}}^{+}$) is removed and supplied by a point of the occurrence of the m-th phenomenon ($n_m\in {\mathbb{N}}_0$). The main idea behind the definition is to quantify the information change in the subsequent image, if only one point is changed. Analogous to the Point Information Gain ${\Gamma }_{\alpha }^{\left(i\right)}$, the Point Divergence Gain can be defined as

$${\mathsf{\Omega }}_{\alpha }^{(l\to m)}\equiv {\mathsf{\Omega }}_{\alpha }^{(l\to m)}\left(P\right)={\mathcal{H}}_{\alpha }\left(P^{(l\to m)}\right)-{\mathcal{H}}_{\alpha }\left(P\right).$$ (6)

Let us first show its connection to the Point Information Gain ${\Gamma }_{\alpha }^{\left(i\right)}$. Since $P^{\left(l\right)}=P^{(l\to m,m)}$, it is possible to express the Point Divergence Gain as

$${\mathsf{\Omega }}_{\alpha }^{(l\to m)}\left(P\right)={\mathcal{H}}_{\alpha }\left(P^{(l\to m)}\right)-{\mathcal{H}}_{\alpha }\left(P^{(l\to m,m)}\right)+{\mathcal{H}}_{\alpha }\left(P^{\left(l\right)}\right)-{\mathcal{H}}_{\alpha }\left(P\right)={\Gamma }_{\alpha }^{\left(l\right)}\left(P\right)-{\Gamma }_{\alpha }^{\left(m\right)}(P^{(l\to m)}).$$ (7)

Let us investigate mathematical properties of the PDG. The ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$ can be rewritten as

$$\begin{array}{cc}{\mathsf{\Omega }}_{\alpha }^{(l\to m)}\hfill & ={\mathcal{H}}_{\alpha }\left(P^{(l\to m)}\right)-{\mathcal{H}}_{\alpha }\left(P\right)\hfill \\ & =\frac{1}{1-\alpha }ln\left(\sum _{j=1}^k{\left(p_j^{(l\to m)}\right)}^{\alpha }\right)-\frac{1}{1-\alpha }ln\left(\sum _{j=1}^k{p}_j^{\alpha }\right)=\frac{1}{1-\alpha }ln\left(\frac{{\sum }_{j=1}^k{\left(p_j^{(l\to m)}\right)}^{\alpha }}{{\sum }_{i=1}^k{p}_j^{\alpha }}\right).\hfill \end{array}$$ (8)

By plugging the relative frequencies from Equations (1) and (5) into Equation (8), we obtain

$$\begin{array}{cc}\hfill {\mathsf{\Omega }}_{\alpha }^{(l\to m)}& =\frac{1}{1-\alpha }ln\left[\frac{{(n_l-1)}^{\alpha }+{(n_m+1)}^{\alpha }+{\sum }_{j=1,j\ne l,m}^k{n}_j^{\alpha }}{{\sum }_{j=1}^k{n}_j^{\alpha }}\right]\hfill \\ & =\frac{1}{1-\alpha }ln\left[\frac{{(n_l-1)}^{\alpha }+{(n_m+1)}^{\alpha }+{\sum }_{j=1}^k{n}_j^{\alpha }-n_l^{\alpha }-n_m^{\alpha }}{{\sum }_{j=1}^k{n}_j^{\alpha }}\right]\hfill \\ & =\frac{1}{1-\alpha }ln\left[\frac{{(n_l-1)}^{\alpha }-n_l^{\alpha }+{(n_m+1)}^{\alpha }-n_m^{\alpha }}{{\sum }_{j=1}^k{n}_j^{\alpha }}+1\right].\hfill \end{array}$$ (9)

As seen in Equation (9), the variable ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$ does not depend (contrary to the ${\Gamma }_{\alpha }^{\left(i\right)}$) on n but depends only on the number of elements of each phenomenon j. In Equation (9), let us design the nominator ${\sum }_{j=1}^k{n}_j^{\alpha }$, which is constant and related to the original distribution (histogram) of elements and to the parameter $\alpha$, as ${\mathcal{C}}_{\alpha }$. It gives us the final form

$${\mathsf{\Omega }}_{\alpha }^{(l\to m)}=\frac{1}{1-\alpha }ln\left[\frac{{(n_l-1)}^{\alpha }-n_l^{\alpha }+{(n_m+1)}^{\alpha }-n_m^{\alpha }}{{\mathcal{C}}_{\alpha }}+1\right].$$ (10)

Equation (10) demonstrates that, for a particular distribution, ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$ is a function only of the parameter $\alpha$ and frequencies of occurrences of the phenomena $n_l$ and $n_m$ in the original distribution, between which the exchange of the element occurs. Equation (10) further shows that if the exchange of the element occurs between phenomena l and m of the same (similar) frequencies of occurrence (i.e., $n_l\approx n_m$), the value of ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$ equals 0. If we remove a rare point and supply it by a high-frequency point (i.e., $n_l\ll n_m$), the value of ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$ is negative, and vice versa. Low values of parameter $\alpha$ separate low-frequency events as ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}=0$, whereas high $\alpha$ emphasize high-frequency events as ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}\gg 0$ or ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}\ll 0$ and merge rare events into ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}=0$. With respect to the previous discussion and practical utilization of this notion, we emphasize that, for real systems with large n, the ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$ are rather small numbers.

In the 3D plots of Figure 1, we demonstrate ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$-transformations of four thoroughly studied distributions—the Cauchy, Gauss (symmetrical), Lévy, and Rayleigh distribution (asymmetric; all specified in Section 4.1)—for $\alpha =\{0.5;1.0;2.0;4.0\}$, where each point presents the exchange of the element between bins l and m (Algorithm 1). In this case, the (a)symmetry of the distribution is always maintained.

Figure 1.

The ${\mathsf{\Omega }}_{\alpha }$-transformations of the discrete (a) Cauchy; (b) Gauss; (c) Lévy; and (d) Rayleigh distribution for $\alpha$ = $\{0.5;1.0;2.0;4.0\}$ (Section 4.1).

Algorithm 1: Calculation of a point divergence gain matrix (${\mathsf{\Omega }}_{\alpha }$) for typical histograms.

Now we will consider the specific case $\alpha =2$ (collision entropy) for which Equation (10) can be simplified to

$${\mathsf{\Omega }}_2^{(l\to m)}=-ln\left[\frac{2}{{\mathcal{C}}_2}(n_m-n_l+1)+1\right]=-ln\left[\frac{2}{{\mathcal{C}}_2}(\Delta n^{(l\to m)}+1)+1\right].$$ (11)

For a specific difference $\Delta n^{(x\to y)}=D$, Equation (11) can be approximated by the 1st-order Taylor sequence

$$\begin{array}{cc}\hfill {\mathsf{\Omega }}_2^{(l\to m)}& \approx -ln\left[\frac{2}{{\mathcal{C}}_2}(D+1)+1\right]-\frac{2}{2(D+1)+{\mathcal{C}}_2}(\Delta n^{(l\to m)}-D)\hfill \\ & =-\frac{2}{2D+2+{\mathcal{C}}_2}\Delta n^{(l\to m)}+\frac{2D}{2D+2+{\mathcal{C}}_2}-ln\left[\frac{2D}{{\mathcal{C}}_2}+{\mathcal{C}}_2+1\right].\hfill \end{array}$$ (12)

Equations (11) and (12) show that, for each unique $\Delta n^{(x\to y)}$, the ${\mathsf{\Omega }}_2^{(l\to m)}$ depends only on the difference between the bins l and m, which the exchange of the element occurs between, and this dependence is almost linear. In other words, this explains why, for all distributions in Figure 2, the dependencies ${\mathsf{\Omega }}_2^{(l\to m)}=f(n_m,n_m-n_l)$ are planes.

Figure 2.

The dependencies ${\mathsf{\Omega }}_{\alpha }=f(n_m,n_m-n_l)$ for the discrete (a) Cauchy; (b) Gauss; (c) Lévy; and (d) Rayleigh distribution at $\alpha$ = $\{0.5;1.0;2.0;4.0\}$ (Section 4.1).

For $\alpha \to 1$, the Rényi entropy becomes the ordinary Shannon entropy [18] and we obtain (cf. Equation (4))

$$\begin{array}{c}\hfill {\mathcal{H}}_1\left(P\right)=-\sum _{j=1}^k{p}_{j}ln{p}_j=-\sum _{j=1}^k\frac{n_j}{n}ln\frac{n_j}{n}=-\sum _{j=1,j\ne l,m}^k\frac{n_j}{n}ln\frac{n_j}{n}-\frac{n_m}{n}ln\frac{n_m}{n}-\frac{n_l}{n}ln\frac{n_l}{n}\end{array}$$ (13)

and

$$\begin{array}{c}\hfill {\mathcal{H}}_1(P^{(l\to m)})=-\frac{n_m+1}{n}ln\frac{n_m+1}{n}-\frac{n_l-1}{n}ln\frac{n_l-1}{n}-\sum _{j=1,j\ne l,m}^k\frac{n_j}{n}ln\frac{n_j}{n}.\end{array}$$ (14)

The difference of these entropies (cf. Equation (9)) is gradually giving

$$\begin{array}{cc}\hfill {\mathsf{\Omega }}_1^{(l\to m)}& =-\frac{n_m+1}{n}ln\frac{n_m+1}{n}-\frac{n_l-1}{n}ln\frac{n_l-1}{n}+\frac{n_m}{n}ln\frac{n_m}{n}+\frac{n_l}{n}ln\frac{n_l}{n}\hfill \\ & =-\frac{n_m+1}{n}ln(n_m+1)+\frac{n_m+1}{n}lnn-\frac{n_l-1}{n}ln(n_l-1)+\frac{n_l-1}{n}lnn+\frac{n_m}{n}ln{n}_m\hfill \\ & \hspace{1em}-\frac{n_m}{n}lnn+\frac{n_l}{n}ln{n}_l-\frac{n_l}{n}lnn\hfill \\ & =\underset{=0}{\underbrace{(\frac{n_m+1}{n}+\frac{n_l-1}{n}-\frac{n_m}{n}-\frac{n_l}{n})}}lnn-\frac{n_m}{n}ln(n_m+1)-\frac{1}{n}ln(n_m+1)-\frac{n_l}{n}ln(n_l-1)\hfill \\ & \hspace{1em}+\frac{1}{n}ln(n_l-1)+\frac{n_m}{n}ln{n}_m+\frac{n_l}{n}ln{n}_l\hfill \\ & =\frac{1}{n}(n_{m}ln\frac{n_m}{n_m+1}+n_{l}ln\frac{n_l}{n_l-1}+ln\frac{n_l-1}{n_m+1}).\hfill \end{array}$$ (15)

One can see that relation (15) is defined for $n_l\in \mathbb{N}\backslash \{0,1\}$ and $n_m\in {\mathbb{N}}^{+}$ and is approximately equal to 0 for $n_l,n_m\gg 0$ (the Cauchy and Rayleigh distribution for $\alpha =1$ in Figure 3).

Figure 3.

The dependencies ${\mathsf{\Omega }}_{\alpha }=f(n_l,n_m)$ for the discrete (a) Cauchy; (b) Gauss; (c) Lévy; and (d) Rayleigh distribution at $\alpha$ = $\{0.5;1.0;2.0;4.0\}$ (Section 4.1).

For $n_l\in {\mathbb{N}}^{+}$ and $n_m\in {\mathbb{N}}_0$, from Equation (10), further implies:

1. If $\alpha =0$, then ${\mathsf{\Omega }}_0^{(l\to m)}=0$.

2. If $\alpha \to \infty$, then ${\mathsf{\Omega }}_{\infty }^{(l\to m)}\to 0$.

2.2. Point Divergence Gain Entropy and Point Divergence Gain Entropy Density

In this section, we introduce two new variables that help us to investigate changes between two (typically consecutive) points of time series. A typical example can be provided by video processing, where each element of a time or spatial series is represented by a frame. Let us have two data frames ${\mathcal{I}}_b=\{a_1,\dots ,a_n\}$ and ${\mathcal{I}}_b=\{b_1,\dots ,b_n\}$ (For simplicity, we use only one index which corresponds to a one-dimensional frame. In case of images, we have typically two-dimensional frames and the elements are described by two indexes, e.g., x and y positions.). At each position $i\in \{1,\dots ,n\}$, it is possible to replace the value $a_i$ by the value of the following frame, i.e., $b_i$. The resulting ${\mathsf{\Omega }}_{\alpha }^{(a_i\to b_i)}$ then quantifies how much information is gained/lost, when, at the i-th position, we replace the value $a_i$ for the value $b_i$. A Point Divergence Gain Entropy (PDGE, $I_{\alpha }$) is defined as a sum of absolute values of all PDGs for all pixels, i.e.,

$$I_{\alpha }({\mathcal{I}}_a;{\mathcal{I}}_b)=\sum _{i=1}^n|{\mathsf{\Omega }}_{\alpha }^{(a_i\to b_i)}|=\sum _{l=1}^k\sum _{m=1}^k{n}_{lm}|{\mathsf{\Omega }}_{\alpha }^{(l\to m)}|,$$ (16)

where $n_{lm}$ denotes the number of present substitutions $l\to m$, when we transform ${\mathcal{I}}_a\to {\mathcal{I}}_b$. The absolute value ensures that the contribution of the transformation of a rare point to a frequent point (negative ${\mathsf{\Omega }}_{\alpha }$) and a frequent point to a rare point (positive ${\mathsf{\Omega }}_{\alpha }$) do not cancel each other and both contribute to the resulting PDGE. Typically, appearance or disappearance of a rare point (and replacement by a frequent value—typically background colour) carries important information about the experiment. The PDGE can be understood as an absolute information change.

Moreover, it is possible to introduce other macroscopic quantity—a Point Divergence Gain Entropy Density (PDGED, $P_{\alpha }$), where we do not sum over all pixels, but only over all realized transitions $l\to m$. Thus, the PDGED can be defined as

$$P_{\alpha }({\mathcal{I}}_a;{\mathcal{I}}_b)=\sum _{l=1}^k\sum _{m=1}^k{\chi }_{lm}|{\mathsf{\Omega }}_{\alpha }^{(l\to m)}|,$$ (17)

where

$${\chi }_{lm}=\left\{\begin{array}{cc}1,\hfill & n_{lm}\ge 1,\hfill \\ 0,\hfill & n_{lm}=0.\hfill \end{array}\right.$$ (18)

Let us emphasize that two transitions $a_1\to b_1$ and $a_2\to b_2$, where the frequencies of the occurrences of the phenomena $a_1$ and $a_2$ are equal and of the phenomena $b_1$ and $b_2$ are equal as well, give two unique values of the ${\mathsf{\Omega }}_{\alpha }^{(a_i\to b_i)}$. In the computation of the PDGED, this is arranged by a hash function (Algorithm 2). We can understand the quantity PDGED as an absolute information change of all realized transitions of phenomena $m\to l$.

Algorithm 2: Calculation of a point information gain matrix (${\mathsf{\Omega }}_{\alpha }$) and values $P_{\alpha }$ and $I_{\alpha }$ for two consecutive images of a time-spatial series.

If the aim is to assess the influence of elements of a high occurrence on the time-spatial changes in the image series, it is recommended to use PDGE where each element is weighted by its number of occurrences. If the aim is to suppress the influence of these extreme values, it is better to compute PDGED.

Let us consider a time-series $\mathcal{V}$, where each time step contains one frame, so $\mathcal{V}=\{{\mathcal{I}}_1,{\mathcal{I}}_2,\dots \}$. The series $\mathcal{V}$ can be, e.g., a sequence of images (a video) obtained from some experiment, etc. For each time step, it is possible to calculate $I_{\alpha }\left(t\right)=I_{\alpha }({\mathcal{I}}_t;{\mathcal{I}}_{t+s})$, resp. $P_{\alpha }\left(t\right)=P_{\alpha }({\mathcal{I}}_t;{\mathcal{I}}_{t+s})$, where s is the time lag. Typically, we assume $s=1$, i.e., consecutive frames with a constant time step.

3. Application of Point Divergence Gain and Its Entropies in Image Processing

The generalized Point Divergence Gain ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$ in Equation (10) was originally used for characterization of dynamic changes in image series, namely in z-stacks of raw RGB data of unmodified live cells obtained via scanning along the z-axis using video-enhanced digital bright-field transmission microscopy [6,7]. In these two references, this new mathematical approach utilizes 8- and 12-bit intensity histograms of two consecutive images for pixel-by-pixel intensity weighted (parameterized) subtraction of these images to suppress the camera-based noise and to enhance the image contrast (In case of calibrated digital camera-based images, where the value of each point of the image reflects a number of incident photons, or, in case of computer-based images, it can be sufficient to use a simple subtraction for evaluation of time-spatial changes in the image series.).

For this paper, we chose other (grayscale) digital image series (Table 1) in order to demonstrate other applications of the PDG mathematical approach in image processing and analysis. Moreover, we newly introduce applications of the additive macroscopic variables Point Divergence Gain Entropy $I_{\alpha }$ and Point Divergence Gain Entropy Density $P_{\alpha }$.

Table 1.

Specifications of image series.

Series	Source	Bit-Depth	Number of Img.	Resolution	Origin
Toy Vehicle	[19]	8-bit	10	512 × 512	camera
Walter Cronkite	[19]	8-bit	16	256 × 256	camera
Simulated BZ	[20,21,22]	8-bit	10,521	1001 × 1001	computer-based ${}^a$
Ring-fluorescence		12-bit	1058	548 × 720	experimental ${}^b$
Ring-diffraction		8-bit ${}^c$	1242	252 × 280	experimental ${}^b$

${}^a$ A set of a noisy hotch-potch machine simulation of the Belousov–Zhabotinsky reaction [20,21,22] at 200 achievable states with the internal excitation of 10, and phase transition, internal excitation, and external neighbourhood kind of noise of 0, 0.25, and 0.15, respectively. ${}^b$ The microscopic series of a 6-$\mathsf{\mu }$m standard microring (FocalCheck^TM, cat. No. F36909, Life Technologies^TM (Eugene, OR, USA)) were acquired using the CellObserver microscope (Zeiss, Oberkochen, Germany) at the EMBL (Heidelberg, Germany). For both light processes, the green region of the visible spectrum was selected using an emission and transmission optical filter, respectively. In case of the diffraction, the point spread function was separated and the background intensities was disposed using Algorithm 1 in [7]. ${}^c$ The 12-bit depth was reduced using a Least Information Lost algorithm [23], which, by shifting the intensity bins, filled all empty bins in the histogram obtained from the whole data series up and rescaled these intensities between their minimal and maximal value.

3.1. Image Origin and Specification

Owing to the relation of the ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$ to the Rényi entropy, the $I_{\alpha }$ and $P_{\alpha }$ as macroscopic variables can determine a fractal origin of images by plotting $I_{\alpha }=f_I\left(\alpha \right)$ and $P_{\alpha }=f_P\left(\alpha \right)$ spectra. If we deal with an image multifractality, the dependency $I_{\alpha }=f_I\left(\alpha \right)$ or the dependency $P_{\alpha }=f_P\left(\alpha \right)$ shows a peak. In case of a unifractality, these dependences are monotonous. It is demonstrated in Figure 4 and Figure 5. There can be no doubts that the origin of the simulated Belousov–Zhabotinsky reaction (Figure 4) is multifractal. This statement is further strengthened by the courses of the dependencies $I_{\alpha }=f_I\left(\alpha \right)$ and $P_{\alpha }=f_P\left(\alpha \right)$, where we can see peaks with maxima at $\alpha \in (1,2)$. On the contrary, a pair of images in Figure 5 (moving toys of cars) is a mixture of the objects of different fractal origin. In this case, whereas the course of $f_I\left(\alpha \right)$ is monotonous and thus shows a unifractal characteristics, the dependence $f_P\left(\alpha \right)$ has a maximum at $\alpha =0.6$ and thus demonstrates some multifractal features in the image. This is due to the fact that, since each information contribution is counted only once, the $P_{\alpha }$ is more sensitive to the phenomena, which occur less frequently in the image. The monotonic course of the $P_{\alpha }$ would be achieved only when a sequence of time-evolved Euclidian objects was transformed into the values ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$.

Figure 4.

The $I_{\alpha }$, $P_{\alpha }$, and ${\mathsf{\Omega }}_{\alpha }$ for a pair of multifractal grayscale images. I. The $I_{\alpha }$ and $P_{\alpha }$ spectra, II. 8-bit visualization of ${\mathsf{\Omega }}_{\alpha }$-values for $\alpha =\{0.99;2.0\}$.

Figure 5.

The $I_{\alpha }$, $P_{\alpha }$, and ${\mathsf{\Omega }}_{\alpha }$ for a pair of real-life grayscale images. I. the $I_{\alpha }$ and $P_{\alpha }$ spectra; II. 8-bit visualization of ${\mathsf{\Omega }}_{\alpha }$-values for $\alpha =\{0.99;2.0\}$.

As mentioned in Section 2.2, the variables $I_{\alpha }$ and $P_{\alpha }$ measure absolute information change between a pair of images and characterize a similarity between these images. Therefore, these variables can find a practical utilization in auto-focusing in both light and electron digital microscopy. The in-focus object can be defined as an image with the global extreme of $I_{\alpha }$ or $P_{\alpha }$. In other characteristics, this image fulfils the Nijboer–Zernike definition [24]: it is the smallest and darkest image in light or electron diffraction or the smallest and brightest image in light fluorescence (Section 3.3).

3.2. Image Filtering and Segmentation

Segmentation is a type of filtering of specific features in an image. The parameter $\alpha$ and the related value of ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$ enable us to filter the parts of two consecutive images, which are either stable or differently variable in time. This can be employed in a 3D image reconstruction by thresholding and joining ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$ = 0 from two consecutive images or in image tracking via thresholding of the highest and lowest ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$ in a first image and the following image, respectively.

This is illustrated using simple examples in Figure 4 and Figure 5 where the highest (red-coded) and lowest (blue-coded) values of the ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$ show the position of the object in the second and the first image of the image sequence, respectively. Compared with the ${\mathsf{\Omega }}_{0.99}^{(l\to m)}$, the variance between the extremes of the ${\mathsf{\Omega }}_{2.00}^{(l\to m)}$ is wider and the number of points ${\mathsf{\Omega }}_{2.00}^{(l\to m)}=0$ is lower.

In digital light transmission microscopy, this mathematical method enabled us to find time stable intracellular objects inside live mammalian cells from consecutive pixels that fulfilled the equality ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}=0$ for $\alpha =4.00$ [6] or $\alpha =5.00$ [7]. In these cases, the high value of $\alpha$ ensured merging rare points in the image, suppressing the camera noise that was reflected in the images and, thus, modelling the shape of organelles. The rest of image escaped the observation. In the next paper [25], this method was extended to widefield fluorescent data.

As in the case of the Point Information Gain [2], the process of image segmentation of objects of a certain shape can be further improved by usage of the surroundings of this shape from which the intensity histogram is created for each pixel in the image.

3.3. Clustering of Image Sets

Finally, we used the Point Divergence Gain to detect the most relevant information contained in a sequence of images, capturing, e.g., an experiment. For this end, we used $I_{\alpha }$ or $P_{\alpha }$ as quantities of information change in the consecutive images and applied the clustering methods on them. The values of $I_{\alpha }$ or $P_{\alpha }$ are small numbers (Section 2.1). Due to the computation rounding of small numbers of the $I_{\alpha }$ and the $P_{\alpha }$ and for a better characterization of the image multifractality, in clustering, we use $\alpha$-dependent spectra of these variables than a sole number at one $\alpha$.

The dependence of the label of the cluster on the order of the image in the series is the smoothest for joint vectors $[I_{\alpha },P_{\alpha }]$. The similarity of these vectors (and thus images as well) is described in a space of principal components, e.g., [26], and classified by standard clustering algorithms such as k-means++ algorithm [27]. In comparison to the entropies and entropy densities related to the ${\Gamma }_{\alpha }^{\left(i\right)}$, the clustering using the $I_{\alpha }$ and the $P_{\alpha }$ is more sensitive to changes in the patterns (intensities) and does not require other specification of images by local entropies computed from a specific type of surroundings around each pixel.

The described clustering method was examined on z-stacks obtained using light microscopy. The z-stacks were classified into 2–6 clusters (groups) when patterns of each image was described by 26 numbers, i.e., by vectors $[I_{\alpha },P_{\alpha }]$ at 13 $\alpha$ (Figure 6a and Figure 7a). These clusters were evaluated on the basis of the sizes of intensity changes between images. These five classification graphs of the gradually splitting clusters (Figure 6a and Figure 7a, middle) further demonstrate the mutual similarity among the micrographs in each data series. The typical (middle) image of each cluster is shown in Figure 6b and Figure 7b.

Figure 6.

The results of clustering of a z-stack of grayscale microscopic images of a microring obtained using a fluorescence mode. (a) the dependencies of (upper) the $P_{\alpha }$ and (lower) the $I_{\alpha }$ vs. order of the image in the z-stack for $\alpha =\{0.5;0.99;2.0;4.0\}$ and (middle) clustering (k-means, squared Euclidian distance, 2–6 groups) of the z-stack using connected spectra [$I_{\alpha }$, $P_{\alpha }$] for $\alpha =\{0.1;0.3;0.5;0.7;0.99;1.3;1.5;1.7;2.0;2.5;3.0;3.5;4.0\}$; (b) the typical (middle) group’s images for clustering into five groups (in (a), middle). The original 12-bit images are visualized in 8 bits using the Least Information Loss conversion [23].

Figure 7.

The results of clustering of a z-stack of grayscale microscopic images of a microring obtained using a diffraction mode. (a) the dependencies of (upper) the $P_{\alpha }$ and (lower) the $I_{\alpha }$ vs. order of the image in the z-stack for $\alpha =\{0.5;0.99;2.0;4.0\}$ and (middle) clustering (k-means, squared Euclidian distance, 2–6 groups) of the z-stack using connected spectra [$I_{\alpha }$, $P_{\alpha }$] for $\alpha =\{0.1;0.3;0.5;0.7;0.99;1.3;1.5;1.7;2.0;2.5;3.0;3.5;4.0\}$; (b) the typical (middle) group’s images for clustering into 5 groups (in (a), middle). The original 12-bit images are visualized in 8 bits using the Least Information Loss conversion [23].

Firstly, we shall deal with a z-stack with 1057 images of a microring obtained using a widefield fluorescent microscope. The results of clustering illustrate a canonically repetitive properties of the so-called point spread function as the image of the observed object goes to and from its focus. In this case, the image group containing the real focus of the maximal $I_{\alpha }$ and $P_{\alpha }$ at low $\alpha$ (Section 3.1) is successfully determined by clustering into two clusters (Figure 6a). However, we will aim for a description of the results for five clusters. The central Cluster 5 (94 images) can be called an object’s focal region with image levels where parts of the object have their own focus. The in-focus cluster is asymmetrically surrounded by Cluster 4 (131 and 53 images below and above Cluster 5, respectively), which was set on the basis of the occurrence of the lower peaks of $I_{\alpha }$ and $P_{\alpha }$ at low $\alpha$. Cluster 3 (190 and 150 images below and above the focus, respectively) is typical of constant $I_{\alpha }$ and $P_{\alpha }$ for all $\alpha$. Cluster 2 contains img. 176–214 and the last 126 images. These images are characteristic of constant $I_{\alpha }$ and decreasing/increasing $P_{\alpha }$ at $\alpha \ge 2$. Cluster 1 (the first 175 images) is prevalently dominated by increasing $I_{\alpha }$ and decreasing $P_{\alpha }$ at high $\alpha$.

Before the calculation of the $I_{\alpha }$ and $P_{\alpha }$, the undesirable background intensities were removed from the images obtained using optical transmission microscopy. The rest of each image was rescaled into 8 bits (Section 4.2). The results of clustering of these images (Figure 7a) are similar to fluorescent data (Figure 6a). The light transmission point spread function is symmetrical around its focus as well but the pixels at the same $x,y$-positions below and above the focus have opposite, dark vs. bright, intensities. Furthermore, the transitional regions between the clusters are longer than for the fluorescent data. The central, in-focus, part of the z-stack (img. 427–561 in Cluster 4) with the highest peaks of $I_{\alpha }$ and $P_{\alpha }$ is unambiguously separated using four clusters. The focus itself lies at the 505th image. This central part of the z-stack is surrounded by eight groups of images which were, due to their similarity, objectively classified into three clusters. Cluster 1 was formed by images 1–78, 376–426, and 562–661. These images show peaks of middle values of the $I_{\alpha }$ and $P_{\alpha }$. Images 79–153, 292–375, and 662–703 were classified into Cluster 2 (dominated by the local minimum of the $I_{\alpha }$ at $\alpha <1$). Cluster 3 is related to the images with the lowest values of the $I_{\alpha }$ together with the lowest values and local peaks of the $P_{\alpha }$ for $\alpha <1$ and for $\alpha <1$, respectively. This cluster contains images 154–291 and the last 537 images of the series.

Let us mention that, in the clustering process, the $I_{\alpha }$ and $P_{\alpha }$ can recognize outliers such as incorrectly saved images or images with illumination artifacts.

4. Materials and Methods

4.1. Processing of Typical Histograms

For the Cauchy, Lévy, Gauss, and Rayleigh distributions, dependences of the ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$ on the number of elements in bins l and m were calculated for $\alpha$ = {0.1, 0.3, 0.5, 0.7, 0.99, 1.3, 1.5, 1.7, 2.0, 2.5, 3.0, 3.5, 4.0} using a pdg_histograms.m Matlab${}^{\circledR }$ 2014 script (Mathworks, Natick, MA, USA). The following probability density functions $f\left(x\right)$ were studied:

1. Lévy distribution:

   $$f\left(x\right)=\mathrm{round}\left[{10}^c\frac{exp\left(-\frac{1}{2x}\right)}{\sqrt{2\pi x^3}}\right],x\in \mathbb{N},\left\{\begin{array}{c}x\in [1,256],c\in \{5,7\},\hfill \\ x\in [1,85],c=3,\hfill \end{array}\right.$$ (19)
2. Cauchy distribution:

   $$f\left(x\right)=\mathrm{round}\left[{10}^c\frac{1}{\pi \left(1+x^2\right)}\right],x\in \mathbb{Z},\left\{\begin{array}{c}x\in [-127,127],c=7,\hfill \\ x\in [-44,44],c=3.5,\hfill \end{array}\right.$$ (20)
3. Gauss distribution:

   $$f\left(x\right)=\mathrm{round}\left[{10}^c\frac{exp\left(-\frac{x^2}{2{\sigma }^2}\right)}{\sigma \sqrt{2\pi }}\right],x\in \mathbb{Z},\left\{\begin{array}{c}x\in [-4,4],c=4,\sigma =1,\hfill \\ x\in [-29,29],c=3,\sigma =10,\hfill \\ x\in [-36,36],c=4,\sigma =10,\hfill \\ x\in [-64,64],c=10,\sigma =10,\hfill \end{array}\right.$$ (21)
4. Rayleigh distribution:

   $$f\left(x\right)=\mathrm{round}\left[{10}^c\frac{x}{b^2}exp\left(-\frac{x^2}{2b^2}\right)\right],x\in \mathbb{N},x\in [1,108],c=10,b=16.$$ (22)

In Figure 1, the Cauchy and Lévy distributions at c = 7 and the Gauss distribution at parameters c = 10 and $\sigma$ = 10 are depicted.

4.2. Image Processing and Analysis

Image analysis based on calculation of the ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$, $I_{\alpha }$, and $P_{\alpha }$ is demonstrated on five standard grayscale multi-image series (Table 1). All images were processed using Whole Image mode in an Image Info Extractor Professional software (Institute of Complex Systems, FFPW, USB, Nové Hrady, Czech Republic). A pair of images 5000–5001 of a simulated Belousov–Zhabotinsky (BZ) reaction and a pair of images motion01.512–motion02.512 were recalculated for 40 values of $\alpha$ = {0.1, 0.2, ..., 0.9, 0.99, 1.1, 1.2, ..., 4.0}. The rest of series were processed for 13 values of $\alpha$ = {0.1, 0.3, 0.5, 0.7, 0.99, 1.3, 1.5, 1.7, 2.0, 2.5, 3.0, 3.5, 4.0}. The transformation at 13 $\alpha$ was followed by clustering of the matrices [$P_{\alpha }$, $I_{\alpha }$] vs. Img. by k-means method (squared Euclidian distance metrics). Due to a high data variance in the BZ simulation, the clustering was preceded by the z-score standardization of the matrices over $\alpha$. The resulted indices of clusters were reclassified to be consecutive (i.e., the first image of the series and the first image of the following group are classified into gr. 1 and 2, respectively, etc.).

5. Conclusions

In this paper, we derived novel variables from the Rényi entropy—a Point Divergence Gain ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$, a Point Divergence Gain Entropy $I_{\alpha }$, and a Point Divergence Gain Entropy Density $P_{\alpha }$. We have discussed their theoretical properties and made a brief comparison with the related quantity called Point Information Gain ${\Gamma }_{\alpha }^i$ [2]. Moreover, we have shown that the ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$ and related quantities can find their applications in multidimensional data analysis, particularly in video processing. However, due to element-by-element computation, we can characterize time-spatial (4-D) changes much more sensitively than using, e.g., the previously derived ${\Gamma }_{\alpha }^i$. The ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$ can be considered as a microstate of the information changes in the space-time. However, the ${\mathsf{\Omega }}_{\alpha }^{(l\to m)}$, $I_{\alpha }$, and $P_{\alpha }$ show a property that is similar to the ${\Gamma }_{\alpha }^i$ and its relative macroscopic variables. Due to the derivation from the Rényi entropy, they are good descriptors of multifractility. Therefore, they can be utilized to characterize patterns in datasets and to classify the (sub)data into groups of similar properties. This has been successfully utilized in clustering of multi-image sets, image filtration, and image segmentation, namely in microscopic digital imaging.

Author Contributions

Renata Rychtáriková is the main author of the text and tested the algorithms; Jan Korbel is responsible for the theoretical part of the article; Petr Macháček is the developer of the Image Info Extractor Professional software; Dalibor Štys is the group leader who derived the point divergence gain. All authors have read and approved the final manuscript.

Conflicts of Interest

The authors declare no conflict of interest.