Enhancing sparse representation of color images by cross channel transformation

Abstract

Transformations for enhancing sparsity in the approximation of color images by 2D atomic decomposition are discussed. The sparsity is firstly considered with respect to the most significant coefficients in the wavelet decomposition of the color image. The discrete cosine transform is singled out as an effective 3 point transformation for this purpose. The enhanced feature is further exploited by approximating the transformed arrays using an effective greedy strategy with a separable highly redundant dictionary. The relevance of the achieved sparsity is illustrated by a simple encoding procedure. On typical test images the compression at high quality recovery is shown to significantly improve upon JPEG and WebP formats.

1 Introduction

In the signal processing field sparse representation usually refers to the approximation of a signal as superposition of elementary components, called atoms, which are members of a large redundant set, called a dictionary [1]. The superposition, termed atomic decomposition, aims at approximating the signal involving as few atoms as possible [1–4]. Sparsity is also relevant to data collection. Within the emerging theory of sampling known as compressive sensing (CS) this property is key for reconstructing signals from a reduced number of measures [5–7]. In particular, distributed compressive sensing (DCS) algorithms exploit inter signal correlation structure for multiple signal recovery [8].

Sparse image representation using redundant dictionaries has been considered in numerous works e.g. [9–11] and in the context of applications such as image restoration [12–14], feature extraction [15–18] and super resolution [19–22].

The sparsity property of some types of 3D images benefits from 3D processing [23–26]. In particular most RGB (Red-Green-Blue) images admit a sparser atomic decomposition if approximated by 3D atoms [27]. Within the 3D framework the gain in sparsity comes at expenses of computational cost though.

The purpose of this work is to show that the application of a transformation across the direction of the colors improves sparsity in the wavelet domain representation of the 2D color channels. The relevance of this feature is demonstrated by a simple encoding procedure rendering good compression results in comparison to commonly used image compression standards.

The work is organized as follows: Sec. 2 introduces the mathematical notation. Sec. 3 compares several cross color transformations enhancing sparsity. A numerical example on a large data set is used to illustrate the suitability of the dct for that purpose. Sec. 4 demonstrates the gain in sparsity obtained by the atomic decomposition of color images when the dct is applied across the RGB channels. Sec. 5 illustrates the relevance of the approach to image compression with high quality recovery. The conclusions are summarized in Sec. 6.

2 Mathematical notation

Throughout the paper we use the following notational convention. $\mathbb{R}$ represents the set of real numbers. Boldface letters indicate Euclidean vectors, 2D and 3D arrays. Standard mathematical fonts indicate components, e.g., $\mathbf{d}\in {\mathbb{R}}^N$ is a vector of components $d\left(i\right)\in \mathbb{R},i=1,\dots ,N$. The elements of a 2D array $\mathbf{I}\in {\mathbb{R}}^{L_x\times L_y}$ are indicated as I(i, j), i = 1, …, L_x, j = 1, …, L_y and the color channels of $\mathbf{I}\in {\mathbb{R}}^{L_x\times L_y\times 3}$ as I(:, :, z), z = 1, 2, 3. The transpose of a matrix, G say, is indicated as G^⊤. A set of, say M, color images is represented by the arrays $\mathbf{I}\left\{m\right\}\in {\mathbb{R}}^{L_x\times L_y\times 3},m=1,\dots ,M.$

The inner product between arrays in ${\mathbb{R}}^{L_x\times L_y}$ is given by the Frobenius inner product 〈⋅, ⋅〉_F as

$${⟨\mathbf{G},\mathbf{I}⟩}_{\mathrm{F}}=\sum _{i=1}^{L_x}\sum _{j=1}^{L_y}G(i,j)I(i,j).$$

Consequently, the Frobenius norm ‖⋅‖_F is calculated as

$${\parallel \mathbf{G}\parallel }_{\mathrm{F}}=\sqrt{\sum _{i=1}^{L_x}\sum _{j=1}^{L_y}G{(i,j)}^2}.$$

The inner produce between arrays in ${\mathbb{R}}^N$ is given by the Euclidean inner product 〈⋅, ⋅〉 as $\langle \mathbf{d},\mathbf{g}\rangle ={\sum }_{i=1}^{N}d\left(i\right)g\left(i\right).$

3 Cross color transformations

Given a color image I(i, j, z), i = 1, …, L_x, j = 1, …, L_y, z = 1, 2, 3 the processing of the 3 channels can be realized either in the pixel/intensity or in the wavelet domain. Since the representation of most images is sparser in the wavelet domain [27–30] we approximate in that domain and reconstruct the approximated image by the inverse wavelet transform. Thus, using a 3 × 3 matrix T, we construct the transformed arrays $\mathbf{U}\in {\mathbb{R}}^{L_x\times L_y\times 3}$ and $\mathbf{W}\in {\mathbb{R}}^{L_x\times L_y\times 3}$ as follows

$$\begin{array}{ccc}\hfill U(:,:,z)& =& \sum _{l=1}^{3}I(:,:,l)T(l,z),z=1,2,3.\hfill \end{array}$$ (1)

$$\begin{array}{ccc}\hfill W(:,:,z)& =& \mathbf{\text{dwt}}\left(U\right(:,:,z\left)\right),z=1,2,3,\hfill \end{array}$$ (2)

where dwt indicates the 2D wavelet transform. For the transformation T we consider the following cases

• (i)The dct.
• (ii)The reversible YCbCr color space transform.
• (iii)The principal components (PC) transform.
• (iv)A transformation learned from an independent set of images.

The dct is given by the matrix

$$\left(\begin{array}{ccc}\frac{1}{\sqrt{3}}& \frac{\sqrt{2}}{\sqrt{3}}\text{cos}\left(\frac{\pi }{6}\right)& \frac{\sqrt{2}}{\sqrt{3}}\text{cos}\left(\frac{\pi }{3}\right)\\ \frac{1}{\sqrt{3}}& 0& \frac{\sqrt{2}}{\sqrt{3}}\text{cos}\left(\pi \right)\\ \frac{1}{\sqrt{3}}& \frac{\sqrt{2}}{\sqrt{3}}\text{cos}\left(\frac{5\pi }{6}\right)& \frac{\sqrt{2}}{\sqrt{3}}\text{cos}\left(\frac{5\pi }{3}\right)\end{array}\right),$$

whereas the YCbCr color space transform is given by the matrix below [31]

$$\left(\begin{array}{ccc}0.299& -0.169& 0.5\\ 0.587& -0.331& -0.419\\ 0.114& 0.5& -0.0813\end{array}\right).$$

The columns of the principal components transform are the normalized eigenvectors of covariance matrix of the RGB pixels. Thus, the transformation is image dependent and optimal with respect to decorrelating the color channels.

As a first test, the approximation of the transformed channels is realized by keeping a fixed number of the largest absolute value entries, and setting the others equal to zero. In relation to this, for an image of size L_x × L_y × 3 we define the Sparsity Ratio (SR) as follows

$$\begin{array}{c}\hfill \text{SR}=\frac{L_x·L_y·3}{\text{Number}\text{of}\text{nonzero}\text{entries}\text{in}\text{the}\text{three}\text{channels}}.\end{array}$$ (3)

The quality of a reconstructed image $\tilde{\mathbf{I}}$, with respect to the original 8-bit image I, is compared using the Peak Signal-to-Noise Ratio (PSNR)

$$\text{PSNR}=10{\text{log}}_{10}\left(\frac{{255}^2}{\text{MSE}}\right),\text{with}\text{MSE}=\frac{1}{L_x·L_y·3}\sum _{i,j,z=1}^{L_x,L_y,3}{(I(i,j,z)-\tilde{I}(i,j,z))}^2.$$

For the numerical examples below the transformation corresponding to case (iv) is learned from a set of images I{m}, m = 1, …, M all of the same size. Starting from an invertible 3 × 3 matrix T^k, with k = 1, the learning algorithm proceeds through the following instructions.

1. Use T^k to transform the arrays I{m}→U^k{m}→W^k{m} as in (1) and (2).

2. Approximate each transformed array W^k{m} to obtain ${\tilde{\mathbf{W}}}^k\left\{m\right\}$ by keeping the largest K absolute value entries.

3. Apply the inverse 2D wavelet transform idwt to reconstruct the approximated arrays ${\tilde{\mathbf{U}}}^k\left\{m\right\},m=1,\dots ,M$ as

   $$\begin{array}{ccc}\hfill {\tilde{U}}^k\left\{m\right\}(:,:,z)& =& \mathbf{\text{idwt}}\left({\tilde{W}}^k\left\{m\right\}(:,:,z)\right),z=1,2,3.\hfill \end{array}$$
4. Use the original images I{m}, m = 1, …, M to find G = T⁻¹ by least squares fitting, i.e

   $${\mathbf{G}}^{✱}=\underset{\begin{array}{c}G(z,l)\\ z,l=1,2,3\end{array}}{\text{arg}\text{min}}\mathcal{E}\left(\mathbf{G}\right),$$

   where

   $$\mathcal{E}\left(\mathbf{G}\right)=\sum _{m=1}^M\sum _{i,j,l=1}^{L_x,L_y,3}{(I\left\{m\right\}(i,j,l)-\tilde{I}\left\{m\right\}(i,j,l))}^2$$

   with

   $$\tilde{I}\left\{m\right\}(i,j,l))=\sum _{z=1}^3{\tilde{U}}^k\left\{m\right\}(i,j,z)G(z,l).$$

5. While $\mathcal{E}$ decreases, or the maximum number of allowed iterations has not been reached, set k → k + 1, T^k = (G*)⁻¹ and repeat steps 1)–5).

Given the arrays ${\tilde{\mathbf{U}}}^k\left\{m\right\},m=1,\dots ,M$ the least squares problem for determining the transformation T^k has a unique solution. However, the joint optimization with respect to the arrays ${\tilde{\mathbf{U}}}^k\left\{m\right\},m=1,\dots ,M$ and the transformation T^k is not convex. Hence, the algorithm’s outcome depends on the initial value.

The transformation (iv) used in the numerical examples of Secs. 3.1 and 4.1 has been learned from M = 100 images, all of size 384 × 512 × 3, from the UCID data set [32], which contains images of buildings, places and cars. The learning curves for two random orthonormal initializations is shown in Fig 1.

Fig 1. Learning curves for the transformation (iv) corresponding to two different random orthogonal transforms initializing the process.

The mean value PSNR with respect to the 100 images in the training set corresponds to SR = 15 for all the images.

It is worth mentioning that, as shown in Fig 1, learning is richer when starting comparatively distant from a local minimum (left graph in Fig 1). However, since the convergence is to a local minimum other random initializations, even if generating less learning, may produce better results (right graph in Fig 1).

The aim of the next numerical test is to demonstrate the effect on the SR (c.f. (3)) produced by the above (i)–(iv) transformations across the color channels.

3.1 Numerical test I

Using the whole Berkeley data set [33], consisting of 300 images all of size 321 × 481 × 3 we proceed with each image as in (1) and (2). The dwt corresponds to the Cohen-Daubechies-Feauveau 9/7 (CDF97) wavelet family. Fig 2 shows the transformed channels of the image in Fig 3, including the dct transformation across channels (right graph) and without T transformation (left graph). As noticeable in the figure, the effect of the dct is to re-distribute the intensity in the color channels by transferring values between channels.

Fig 2. Magnitude of the entries in the array W constructed as in (2) from the image of Fig 3, with T the dct (right graph) and without T (left graph).

Fig 3. One of the RGB images in the Berkeley’s data set.

For the numerical test the approximations are realized fixing SR = 20 and SR = 10. The reconstructed images are obtained for the approximated arrays $\tilde{W}$ as

$$\begin{array}{ccc}\hfill \tilde{U}(:,:,z)& =& \mathbf{\text{idwt}}\left(\tilde{W}(:,:,z)\right),z=1,2,3\hfill \end{array}$$ (4)

$$\begin{array}{ccc}\hfill \tilde{I}(:,:,z)& =& \sum _{l=1}^3\tilde{U}(:,:,l)\left(T^{-1}(z,l)\right),z=1,2,3\hfill \end{array}$$ (5)

where T⁻¹ is the inverse of T. When no transformation T is considered the image is reconstructed directly from (4).

The 2nd and 4th columns of Table 1 show the mean value PSNR ($\overline{\text{PSNR}}$), with respect to the whole data set, for SR = 20 and SR = 10 respectively, corresponding to the transformations T listed in the 1st column of the table. The 3rd and 5th columns give the standard deviations (std).

Table 1. Mean value PSNR, with respect to the 300 images in the Berkeley data set.

The approximation of each image is realized by setting the least significant entries in the arrays W{m} = 1, …, 300 equal zero, in order to obtain SR = 20 for all the images (2nd column) and SR = 10 for all the images (4th column).

Transf.	$\overline{\text{PSNR}}$	std	$\overline{\text{PSNR}}$	std
(i) dct	33.9	5.0	38.9	5.1
(ii) YCbCr	33.7	5.0	38.7	5.1
(iii) PC	33.7	4.9	38.4	5.0
(iv) Learned	33.7	4.9	38.7	5.0
(v) No transf.	29.7	4.8	32.9	5.2

While Table 1 shows that all (i)–(iv) transformations render equivalent superior results, with respect to not applying a transformation (case (v)), the dct slightly exceeds the others. Case (iv) refers to the best result achieved when initializing the learning algorithm with 500 different random orthonormal transformations. When initializing the algorithm with transformations (i) and (ii) it appears that each of these transformations is close to a local minimizer of the method. This stems from the fact that such initializations do not generate significant learning.

The common feature of most of the 300 images in the data set used in this numerical example is the correlation property of the three color channels. This property is assessed by the correlation coefficients

$$\begin{array}{c}\hfill r\left(z\right)=\frac{{\sum }_{i=1}^{L_x}{\sum }_{j=1}^{L_y}\Gamma (i,j,z)\Gamma (i,j,z+1)}{\sigma \left(z\right)\sigma (z+1)},z=1,2,r\left(3\right)=\frac{{\sum }_{i=1}^{L_x}{\sum }_{j=1}^{L_y}\Gamma (i,j,1)\Gamma (i,j,3)}{\sigma \left(1\right)\sigma \left(3\right)},\end{array}$$

where

$$\begin{array}{c}\hfill \Gamma (i,j,z)=(I(i,j,z)-\overline{I(:,:,z)}),\sigma \left(z\right)=\sqrt{\sum _{i=1}^{L_x}\sum _{j=1}^{L_y}{(I(i,j,z)-\overline{I(:,:,z)})}^2},\end{array}$$ (8)

and $\overline{I(:,:,z)}$ indicates the mean value of channel z.

The range of the correlation coefficient r(z), z = 1, 2, 3 for the images in the data set can be better estimated using the Fisher transform [34, 35] which is defined as follows

$$\zeta \left(z\right)=\frac{1}{2}\text{ln}\left(\frac{r\left(z\right)+1}{r\left(z\right)-1}\right),z=1,2,3.$$

As seen in the graphs of Fig 4, the histograms of the transformed coefficients ζ(z), z = 1, 2, 3 resemble normal distributions. Hence, the confidence intervals can be well estimated in this domain. Once that is done, the intervals for the correlation coefficients are retrieved through the inverse transformation

$$r\left(z\right)=\frac{e^{2\zeta \left(z\right)}-1}{e^{2\zeta \left(z\right)}+1},z=1,2,3.$$

Fig 4. Histograms of the transformed correlation coefficients between the RGB channels: ζ(1) (top graph), ζ(2) (middle graph), and ζ(3) (bottom graph).

Table 2 gives the range of the correlation coefficients concerning approximately 68% and 95% of the images in the data set.

Table 2. Intervals for the correlation coefficients between R and G channels, r(1), G and B channels, r(2), and R and B channels, r(3), involving approximately 68% and 95% of the images in the data set.

Percentage	r(1)-interval	r(2)-interval	r(3)-interval
68%	(0.8821, 0.9882)	(0.8559, 0.9854)	(0.6881, 0.9854)
95%	(0.6612, 0.9964)	(0.5968, 0.9955)	(0.2617, 0.9884)

In view of the high correlation between the color channels for most images in the data set it is surprising than the PC transformation (iii), which completely decorrelates the channels, does not overperform the other transformations, on the contrary. This feature has also been noticed in the context of bit allocation for subband color image compression [36].

4 Approximations by atomic decomposition

We have seen (c.f. Table 1) that by transformation of channels it is possible to gain quality when reducing nonzero entries in the channels. Now we discuss how to improve quality further by approximating the 2D arrays (2) by an atomic decomposition, other than just by neglecting their less significant entries. For the approximation to be successful it is important to use an appropriate dictionary. To this end, one possibility could be to learn the dictionary from training data [37–40]. However as demonstrated in previous works [27, 29, 30] a separable dictionary, which is easy to construct, is well suited for the purposes of achieving sparsity and delivers a fast implementation of the approach. Since we use that dictionary in the numerical examples, below we describe the method for constructing the atomic decomposition of the array W considering specifically a separable dictionary.

Firstly we concatenate the 3 planes W(:, :, z), z = 1, 2, 3 into an extended 2D array ${\mathbf{W}}^{\prime }\in {\mathbb{R}}^{3L_x\times L_y}$ and divide this array in small blocks ${\mathbf{W}}_q^{\prime },q=1,\dots ,Q$. Without loss of generality the blocks are assumed to be square of size N_b × N_b say, and are approximated using separable dictionaries ${\mathcal{D}}^x=\{{\mathbf{d}}_n^x\in {\mathbb{R}}^{N_b},\parallel {\mathbf{d}}_n^x{{\parallel }_2=1\}}_{n=1}^{M_b}$ and ${\mathcal{D}}^y=\{{\mathbf{d}}_m^y\in {\mathbb{R}}^{N_b},\parallel {\mathbf{d}}_n^y{{\parallel }_2=1\}}_{m=1}^{M_b}$.

For q = 1, …, Q every element ${{\mathbf{W}}^{\prime }}_q\in {\mathbb{R}}^{N_b\times N_b}$ is approximated by an atomic decomposition as below:

$$\begin{array}{c}\hfill {{\mathbf{W}}^{\prime }}_q^{k_q}=\sum _{n=1}^{k_q}{c}^{k_q,q}\left(n\right){\mathbf{d}}_{{\ell }_n^{x,q}}^x{\left({\mathbf{d}}_{{\ell }_n^{y_{,}q}}^y\right)}^T,\end{array}$$ (6)

where ${\ell }_n^{y_{,}q}$ is the index in the set {1, 2, …, M_b} corresponding to the label of the atom in the dictionary ${\mathcal{D}}^y$ contributing to the n-th term in the approximation of the q-th block. The index ${\ell }_n^{x_{,}q}$ has the equivalent description. The assembling of the approximated blocks gives rise to the approximated array ${{\mathbf{W}}^{\prime }}^a={\widehat{\text{J}}}_{q=1}^Q{{\mathbf{W}}^{\prime }}_q^{k_q}$, where $\widehat{\text{J}}$ represents the assembling operation, i.e. the operation that retrieves the approximation ${{\mathbf{W}}^{\prime }}^a\in {\mathbb{R}}^{3L_x\times L_y}$ of the array ${\mathbf{W}}^{\prime }\in {\mathbb{R}}^{3L_x\times L_y}$ from the approximation of the blocks in the partition. The approximated array ${{\mathbf{W}}^{\prime }}^a\in {\mathbb{R}}^{3L_x\times L_y}$ is reshaped back into 3 planes, $W^a(:,:,z)\in {\mathbb{R}}^{L_x\times L_y},z=1,2,3$, to be converted back to the approximated RGB intensity image as in (4) and (5).

The approximation of the partition ${{\mathbf{W}}^{\prime }}_q\in {\mathbb{R}}^{N_b\times N_b},q=1,\dots ,Q$ is carried out iteratively as a two step process which selects i) the atoms in the atomic decomposition (6) and ii) the sequence in which the blocks in the partition are approximated. The procedure is called Hierarchized Block Wise (HBW) implementation of greedy pursuit strategies [28, 41]. For the selection of the atoms we apply the Orthogonal Matching Pursuit (OMP) approach [42] dedicated to 2D with separable dictionaries (OMP2D) [43]. Thus, the whole algorithm is termed HBW-OMP2D [28]. The method iterates as described below.

On setting k_q = 0 and ${\mathbf{R}}_q^0={{\mathbf{W}}^{\prime }}_q\in {\mathbb{R}}^{N_b\times N_b}$ at iteration k_q + 1 the algorithm selects the indices ${\ell }_{k_q+1}^{x,q}$ and ${\ell }_{k_q+1}^{y,q}$, as follows:

$$\begin{array}{c}\hfill {\ell }_{k_q+1}^{x,q},{\ell }_{k_q+1}^{y,q}=\underset{\begin{array}{c}n=1,\dots ,M_b\\ m=1,\dots ,M_b\end{array}}{\text{arg}\text{max}}\left|{⟨{\mathbf{d}}_n^x,{\mathbf{R}}_q^{k_q}{\mathbf{d}}_m^y⟩}_F\right|,q=1,\dots ,Q,\end{array}$$ (7)

where ${\mathbf{R}}_q^{k_q}={{\mathbf{W}}^{\prime }}_q-{{\mathbf{W}}^{\prime }}_q^{k_q}$. The calculation of ${{\mathbf{W}}^{\prime }}_q^{k_q}$ is realized in order to minimize $\parallel {\mathbf{R}}_q^{k_q}{\parallel }_F$, which is equivalent to finding the orthogonal projection onto the subspace spanned by the selected atoms ${\{{\mathbf{A}}_n={\mathbf{d}}_{{\ell }_n^{x,q}}^x{\left({\mathbf{d}}_{{\ell }_n^{y,q}}^y\right)}^T\in {\mathbb{R}}^{N_b\times N_b}\}}_{n=1}^{k_q}$. In our implementation, the calculation of the coefficients c^q(n), n = 1, …, k_q in (6) is realized as

$$\begin{array}{c}\hfill c^{k_q,q}\left(n\right)={⟨{\mathbf{B}}_n^k,{{\mathbf{W}}^{\prime }}_q^{k_q}⟩}_{\mathrm{F}},n=1,\dots ,k_q,\end{array}$$ (8)

where the set ${\{{\mathbf{B}}_n^{k_q}\in {\mathbb{R}}^{N_b\times N_b}\}}_{n=1}^{k_q}$ is biorthogonal to the set ${\{{\mathbf{A}}_n\in {\mathbb{R}}^{N_b\times N_b}\}}_{n=1}^{k_q}$ and needs to be upgraded and updated to account for each newly selected atom. Starting from ${\mathbf{B}}_1^1={\mathbf{Q}}_1={\mathbf{A}}_1={\mathbf{d}}_{{\ell }_1^x}^x{\left({\mathbf{d}}_{{\ell }_1^y}^y\right)}^{\top }$ the updating and upgrading is realized through the recursive equations [43, 44]:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill {\mathbf{B}}_n^{k_q+1}& ={\mathbf{B}}_n^{k_q}-{\mathbf{B}}_{k_q+1}^{k_q+1}{⟨{\mathbf{A}}_{k_q+1},{\mathbf{B}}_n^k⟩}_{\mathrm{F}},n=1,\dots ,k_q\hfill \\ \hfill {\mathbf{B}}_{k_q+1}^{k_q+1}& ={\mathbf{Q}}_{k_q+1}/{\parallel {\mathbf{Q}}_{k_q+1}\parallel }_{\mathrm{F}}^2,\text{where}\hfill \\ \hfill {\mathbf{Q}}_{k_q+1}& ={\mathbf{A}}_{k_q+1}-\sum _{n=1}^{k_q}\frac{{\mathbf{Q}}_n}{\parallel {\mathbf{Q}}_n{\parallel }_{\mathrm{F}}^2}{⟨{\mathbf{Q}}_n,{\mathbf{A}}_{k_q+1}⟩}_{\mathrm{F}},\hfill \end{array}\end{array}$$ (9)

with the additional re-orthogonalization step

$$\begin{array}{c}\hfill {\mathbf{Q}}_{k_q+1}\leftarrow {\mathbf{Q}}_{k_q+1}-\sum _{n=1}^{k_q}\frac{{\mathbf{Q}}_n}{\parallel {\mathbf{Q}}_n{\parallel }_{\mathrm{F}}^2}.{⟨{\mathbf{Q}}_n,{\mathbf{Q}}_{k_q+1}⟩}_{\mathrm{F}}.\end{array}$$ (10)

As discussed in [28, 41], for ${\ell }_{k_q+1}^{x,q}$ and ${\ell }_{k_q+1}^{y,q},q=1,\dots ,Q$ the indices resulting from (7), the block to be approximated in the next iteration corresponds to the value q^⋆ such that

$$q^{\star }=\underset{q=1,\dots ,Q}{\text{arg}\text{max}}|⟨{\mathbf{d}}_{{\ell }_{k_q+1}^{x,q}}^x,{\mathbf{R}}_q^{k_q}{\mathbf{d}}_{{\ell }_{k_q+1}^{y,q}}^y⟩|.$$

The algorithm stops when the required total number of $K={\sum }_{q=1}^Q{k}_q$ atoms has been selected. This number can be fixed using the SR, which is now calculated as

$$\begin{array}{c}\hfill \text{SR}=\frac{L_x·L_y·3}{K}.\end{array}$$ (11)

Remark 1. The above described implementation of HBW-OMP2D is very effective in terms of speed, but demanding in terms of memory (the partial outputs corresponding to all the blocks in the partition need to be stored at every iteration). An alternative implementation, termed HBW Self Projected Matching Pursuit (HBW-SPMP) [29, 45], would enable the application of the identical strategy to much larger images than the ones considered in this work.

4.1 Numerical example II

For this and the next numerical example, we use a mixed dictionary consisting of two classes of sub-dictionaries of different nature:

• I)
  The trigonometric dictionaries ${\mathcal{D}}_C^x$ and ${\mathcal{D}}_S^x$, defined below, for i = 1…, N_b

  $${\mathcal{D}}_C^x={\left\{w_c\left(n\right)\text{cos}\frac{\pi (2i-1)(n-1)}{2M}\right\}}_{n=1}^{M_x},{\mathcal{D}}_S^x={\left\{w_s\left(n\right)\text{sin}\frac{\pi (2i-1)\left(n\right)}{2M}q\right\}}_{n=1}^{M_x}.$$

  w_c(n) and w_s(n), n = 1, …, M_x are normalization factors.
• II)The dictionary ${\mathcal{D}}_L^x$, which is constructed by translation of the prototype atoms in Fig 5.

Fig 5. Prototypes (each in different color) that generate the dictionaries ${\mathcal{D}}_L^x$ by sequential translations of one point.

The mixed dictionary ${\mathcal{D}}^x$ is built as ${\mathcal{D}}^x={\mathcal{D}}_C^x\cup {\mathcal{D}}_S^x\cup {\mathcal{D}}_L^x$ and ${\mathcal{D}}^y={\mathcal{D}}^x$.

Table 3 shows the improvement in $\overline{\text{PSNR}}$ achieved by atomic decompositions using the mixed dictionary for SR = 20 and SR = 10.

Table 3. Mean value PSNR, with respect to 300 images in the Berkeley data set, produced by 2D atomic decomposition of the arrays W{m} = 1, …, 300 in order to obtain SR = 20 (2nd and 6th column) and SR = 10 (4th and 8th column).

Block size	8 × 8				16 × 16
Transf.	$\overline{\text{PSNR}}$	std	$\overline{\text{PSNR}}$	std	$\overline{\text{PSNR}}$	std	$\overline{\text{PSNR}}$	std
(i) dct	40.5	5.0	48.1	4.4	40.8	4.9	48.6	4.2
(ii) YCbCr	40.3	5.0	47.8	4.4	40.6	4.9	48.3	4.2
(iii) PC	39.6	4.8	46.3	4.5	39.9	4.8	46.7	4.5
(iv) Learned	40.3	4.9	47.4	4.3	40.5	4.9	47.8	4.2
(v) No transf.	34.0	5.2	39.1	5.6	34.1	5.2	39.3	5.5

Notice that while case (v), which does not include any T transformation, gives superior results than by disregarding entries (c.f. Table 1) when applying any of the transformations (i)–(iv) the results improve further. The PC transform, however, appears significantly less effective than the others. In addition to rendering the best results, the dct brings along the additional advantage of being orthonormal. Consequently, it does not magnify errors at the inversion step. Because of this, we single out the dct as the most convenient cross color transformation out of the four considered here.

5 Application to image compression

In order achieve compression by filing an atomic decomposition we need to address two issues. Namely, the quantization of the coefficients c_q(n)n = 1, …, k_q, q = 1, …, Q in (6) and the storage of the indices $({\ell }_n^{x,q},{\ell }_n^{y,q}),n=1,\dots ,k_q,q=1,\dots ,Q$. We tackle the matters by simple but effective procedures [30].

For q = 1, …, Q the absolute value coefficients |c_q(n)|, n = 1, …, k_q are converted to integers through uniform quantization as follows

$$\begin{array}{c}\hfill c_q^{\Delta }\left(n\right)=\{\begin{array}{cc}\lceil \frac{|c_q\left(n\right)|-\theta }{\Delta }\rceil ,\hfill & \text{if}|c_q\left(n\right)|\ge \theta \hfill \\ & 0\text{otherwise}.\hfill \end{array}\end{array}$$ (12)

The signs of the coefficient are encoded separately as a vector, s_q, using a binary alphabet. Each pair of indices $({\ell }_n^{x,q},{\ell }_n^{y,q})$ corresponding to the atoms in the decompositions of the block ${\mathbf{W}}_q^{\prime }$ is mapped into a single index o_q(n). The set o_q(1), …, o_q(k_q) is sorted in ascending order $o_q\left(n\right)\to {\tilde{o}}_q\left(n\right),n=1,\dots ,k_q$ to take the differences ${\delta }_q\left(n\right)={\tilde{o}}_q\left(n\right)-{\tilde{o}}_q(n-1),n=2,\dots ,k_q$ and construct the string of non-negative numbers ${\tilde{o}}_q\left(1\right),{\delta }_q\left(2\right),\dots ,{\delta }_q\left(k_q\right)$. The order of the set ${\tilde{o}}_q\left(n\right),n=1,\dots ,k_q$ induces order in the unsigned coefficients, $c_q^{\Delta }\left(n\right)\to {\tilde{c}}_q^{\Delta }\left(n\right)$, and in the corresponding signs $s_q\left(n\right)\to {\tilde{s}}_q\left(n\right)$.

For each q the number 0 is added at the end of the indices ${\tilde{o}}_q\left(n\right),n=1,\dots ,k_q$ before concatenation, to be able to separate strings corresponding to different blocks. Each sequence of strings corresponding to q = 1, …, Q is concatenated and encoded using the off-the-shelf MATLAB function Huff06 [46], which implements Huffman coding.

The compression rate is given in bits-per-pixel (bpp) which is defined as

$$\text{bpp}=\frac{\text{Size}\text{of}\text{the}\text{file}\text{in}\text{bits}}{\text{Number}\text{of}\text{intensity}\text{pixels}\text{in}\text{a}\text{single}\text{channel}}.$$

At the reconstruction stage the indices $({\tilde{\ell }}_n^{x,q},{\tilde{\ell }}_n^{y,q}),n=1\dots k_q$ are recovered from the string of differences δ_q(n), n = 2, …, k_q. The signs of the coefficients are read from the binary string. The quantized unsigned coefficients are read and transformed into real numbers as:

$$|{\tilde{c}}_q^{\mathrm{r}}\left(n\right)|=\Delta ·{\tilde{c}}_q^{\Delta }\left(n\right)+(\theta -\Delta /2),n=1\dots k_q.$$

The codec for reproducing the examples in the next sections has been made available on [47].

5.1 Numerical example III

The relevance to image compression of the achieved sparsity by dct cross color transformation is illustrated in this section by comparison with results yielded by the compression standards JPEG, and WebP, on the 15 images in Table 4. These are typical images, used for compression tests, available in ppm or png format. The first 9 images are classic test images taken from [48]. The last six images are portions of 1024 × 1024 × 3 pixels shown in Fig 6 from very large high resolution images available on [49].

Table 4. Test images.

The last column gives the approximation times to produce the results in Table 5.

No	Image	Size	time (secs)
1	Lenna	512 × 512 × 3	1.8
2	Goldhill	576 × 720 × 3	3.8
3	Barbara	576 × 720 × 3	3.2
4	Baboon	512 × 512 × 3	2.7
5	Zelda	576 × 784 × 3	2.5
6	Sailboat	512 × 512 × 3	1.8
7	Boy	512 × 768 × 3	3.8
8	Jupiter	1072 × 1376 × 3	2.9
9	Saturn	1200 × 1488 × 3	2.6
10	Building	1024 × 1024 × 3	9.5
11	Cathedral	1024 × 1024 × 3	7.6
12	Flower	1024 × 1024 × 3	3.6
13	Spider-web	1024 × 1024 × 3	3.8
14	Bridge	1024 × 1024 × 3	8.5
15	Deer	1024 × 1024 × 3	5.8

Fig 6. Illustration of the 1024 × 1024 × 3 panels from the six high resolution test images (No 10–15) listed in Table 4.

All the results have been obtained in the MATLAB environment (version R2019a), using a machine with CPU Intel(R) Core(TM) i7–3520M RAM 8GB CPU @ 2.90GHz. For the image approximation the HBW-OMP2D method was implemented with a C++ MEX file. All the channels and all the images were partitioned into blocks of size 16 × 16. The approximation times to produce the results in Table 5 are displayed in the last column of Table 4.

Table 5. Compression rate (bpp) corresponding to JPEG (bjp), WebP (bwb) and the proposed sparse representation (bsr), for the values of PSNR given in the 2nd column.

The corresponding values of MSSIM are given in the 3rd to 5th columns. ssjp, sswb, and sssr indicate the MSSIM produced by JPEG, WebP and the sparse representation codec respectively.

I	dB	ssjp	sswb	sssr	bjp	bwb	bsr
1	35.9	0.98	0.98	0.97	3.28	2.21	1.37
2	36.6	0.98	0.98	0.98	3.63	2.47	2.01
3	37.2	0.98	0.99	0.99	3.67	2.80	1.77
4	28.8	0.97	0.97	0.96	5.80	4.66	2.48
5	39.3	0.98	0.98	0.95	2.61	1.81	1.07
6	30.9	0.98	0.98	0.95	4.49	3.21	1.51
7	32.6	0.97	0.98	0.97	4.34	3.07	2.22
8	48.2	0.99	0.99	0.99	0.60	0.51	0.20
9	49.0	0.99	0.99	0.99	0.34	0.36	0.12
10	37.4	0.99	0.99	0.99	3.41	2.35	1.75
11	38.5	0.99	0.99	0.99	2.84	1.83	1.20
12	41.5	0.99	0.99	0.99	1.78	1.08	0.53
13	45.0	0.99	0.99	0.99	1.55	1.10	0.57
14	34.5	0.99	0.99	0.99	3.57	2.48	1.48
15	30.9	0.97	0.97	0.96	3.76	2.59	0.90

For realizing the comparison we proceed as follows: we set the required value of PSNR as that produced by JPEG at quality = 95 and tune compression with the other methods to produce the same PSNR. In our codec the tuning is realized by approximating the image up to PSNR_o = 1.025 ⋅ PSNR (where PSNR is the targeted quality) and setting the quantization parameter Δ so as to reproduce the targeted value of PSNR.

For compression with JPEG we use the MATLAB imwrite command. The compression with WebP was realized using the software for Ubuntu distributed on [50]. All the approaches were tuned for producing the same value of PSNR as JPEG for quality 95. The Mean Structural SIMilarity (MSSIM) index [51] was then calculated with the approximation corresponding to those values of PSNR.

6 Conclusions

The application of a cross color transformation for enhancing sparsity in the atomic decomposition of RGB images has been proposed. It was demonstrated that the effect of the transformation is to re-distribute the most significant values in the dwt of the 2D channels. As a result, when approximating the arrays by disregarding the less significant entries, the quality of the reconstructed image improves with respect to disabling the cross color transformation. Four transformations have been considered: (i) a 3 point dct, (ii) the reversible YCbCr color space transform, (iii) the PC transform, (iv) a transformation learned from an independent set of images.

The quality of the image approximation was improved further by approximating the transformed arrays by an atomic decomposition using a separable dictionary and the greedy pursuit strategy HBW-OMP2D. The dct was singled out as the most convenient cross color transformation for approximating RGB color images in the wavelet domain.

The approximation approach has been shown to be relevant for image compression. By means of a simple coding strategy the achieved compression for typical test images considerably improves upon the most commonly used compression standards, namely JPEG and WebP.

Supporting information

S1 Data

(TXT)

Data Availability

The data are available in the Aston University repository (DOI: 10.17036/researchdata.aston.ac.uk.00000590).

Funding Statement

The author(s) received no specific funding for this work.