Increasing the quality of reconstructed signal in compressive sensing utilizing Kronecker technique

Abstract

Quality of reconstruction of signals sampled using compressive sensing (CS) algorithm depends on the compression factor and the length of the measurement. A simple method to pre-process data before reconstruction of compressively sampled signals using Kronecker technique that improves the quality of recovery is proposed. This technique reduces the mutual coherence between the projection matrix and the sparsifying basis, leading to improved reconstruction of the compressed signal. This pre-processing method changes the dimension of the sensing matrix via the Kronecker product and sparsity basis accordingly. A theoretical proof for decrease in mutual coherence using the proposed technique is also presented. The decrease of mutual coherence has been tested with different projection matrices and the proposed recovery technique has been tested on an ECG signal from MIT Arrhythmia database. Traditional CS recovery algorithms has been applied with and without the proposed technique on the ECG signal to demonstrate increase in quality of reconstruction technique using the new recovery technique. In order to reduce the computational burden for devices with limited capabilities, sensing is carried out with limited samples to obtain a measurement vector. As recovery is generally outsourced, limitations due to computations do not exist and recovery can be done using multiple measurement vectors, thereby increasing the dimension of the projection matrix via the Kronecker product. The proposed technique can be used with any CS recovery algorithm and be regarded as simple pre-processing technique during reconstruction process.

Introduction

Compressive sensing (CS), a new method of representing signal by fewer samples, is used in various applications. CS, was introduced for generating samples well below the Nyquist rate, and has been used in signal processing, image processing, data compression and encryption [1–3] and in biomedical engineering applications such MRI [4] and Echocardiographic Images [5].

Basically, CS has two main phases; a sensing or compression phase and a reconstruction or recovery phase. In compression phase, a rectangular matrix multiplies a vector, that is the signal to be compressed. This phase is very simple, fast and energy efficient. Efficient and fast compressors based on CS are already available, [6–9]. The effectiveness of different compression algorithms is compared using compression ratio (CR). Compression ratio (CR) is defined as follows:

$$CR(\%)=\left(1-\frac{\text{samples}\text{in}\text{compressed signal}}{\text{samples}\text{in}\text{the}\text{original}\text{signal}}\right)\times 100$$ 1

As the process of compression maps the original signal from a higher dimension space into a lower dimension space, compressive sensing can also be assumed as an encryption system [10, 11]. For instance, CS has been used for simultaneous compression and encryption of medical images [12, 13]. The secrecy of CS-based cryptosystems is high, because without knowing the key (projection matrix) reconstruction is impossible [14]. CS shifts the computational burden from the compression phase to the recovery phase. In the recovery phase, solution for an underdetermined system of equations is carried out. Solving such equations to obtain sparsest response requires computationally intense approaches. Nevertheless, in many applications of CS, the burden of computation in recovery phase is transferred to the third party, i.e. to the cloud environment. In addition, in remote sensing scenario, remote sensors are considered to have limited storage and computational capability while the remote centers have large storage and unlimited computational capabilities. Hence such remote centers generally can handle the recovery phase.

Compressive sensing

Compressive sensing (CS) is a sampling model for acquiring and reconstructing a sparse signal, by solving an underdetermined equation system, [2]. For CS, signal $\text{x}\in {\mathbb{R}}^{\text{N}}\text{needs}\text{to}\text{be}\text{either}$ sparse or compressible. Signal $\text{s}$ is called $\text{k}$-sparse if it has only $\text{k}$ nonzero coefficients and $(\text{N}-\text{k})$ zero coefficients. In the sparse domain most practical signals have $\text{k}$ significant nonzero coefficients and $(\text{N}-\text{k})$ negligible coefficients, Such signals are called compressible signals. Orthonormal projection matrices such as DCT, Fourier or wavelet basis is used to map $\text{x}$ to its sparse domain, i.e. $\text{x}=\mathrm{\psi }\text{s}$. The product of a projection matrix ${\mathrm{\Phi }}_{\text{M}\times \text{N}},\text{M}\ll \text{N}$ and a $\text{k}$-compressible signal $\text{x}\in {\mathbb{R}}^{\text{N}}$ will generate the measurement vector $\text{y}\in {\mathbb{R}}^{\text{M}}$, i.e.$\text{y}=\mathrm{\Phi }\text{x}=\mathrm{\Phi }\mathrm{\psi }\text{s}$, where $\mathrm{\psi }$ is a known sparsifying basis and for brevity we set $\text{A}=\mathrm{\Phi }\mathrm{\psi }$. In the recovery phase, for noiseless case, if $\text{s}$ is sparse $(\text{k}\ll \text{N})$, then it can be reconstructed from M measurements by solving the ${\ell }_1\text{minimization}$ problem,

$$\text{min}{∥\text{s}∥}_1\text{subject to}\text{As}=\text{y}$$ 2

And also for noisy case,

$${min}_{\widehat{\text{s}}\in {\mathbb{R}}^{\text{N}}}\Vert \widehat{\text{s}}{\Vert }_1\text{subject}\text{to}\Vert \text{y}-\text{A}\widehat{\text{s}}{\Vert }_2<ϵ$$ 3

where ${∥.∥}_2$ is Euclidian norm which provides the energy of signal $\text{s}$, ${∥\text{s}∥}_2=\sqrt[2]{{\text{s}}_1^2+{\text{s}}_2^2+\dots +{\text{s}}_{\text{N}}^2}$, and ${∥.∥}_1$ is $1-\text{norm}$ of a vector, $ϵ$ is the upper bound on the Euclidean norm of the contributing noise, and $\text{s}$ and $\widehat{\text{s}}$ are recovered sparse signal in noiseless and noisy case, respectively. For stable and exact recovery, Restricted Isometry Property (RIP) is a sufficient condition for exact reconstruction of sparse signal,

$$(1-{\mathrm{\delta }}_{\text{k}})\Vert \text{s}{\Vert }_2^2\le \Vert \text{As}{\Vert }_2^2\le (1+{\mathrm{\delta }}_{\text{k}})\Vert \text{s}{\Vert }_2^2$$ 4

where ${\mathrm{\delta }}_{\text{k}}$ is isometry constant of a matrix $\text{A}$, and its value belongs to the interval of zero to one, [15]. Unfortunately checking the RIP of a matrix or calculating its isometry constant is practically impossible. But, if $\text{M}\ge \text{Cklog}(\text{N}/\text{k})$, for some large enough $\text{M}$ and $\text{N}$, and a constant $\text{C}$ as a function of $\text{M}/\text{N}$. with overwhelming probability, matrices with entries of i.i.d. Gaussian (zero mean and variance $1/\text{M}$) preserve RIP condition [15]. In addition, sub-Gaussian random matrices, namely, Bernoulli distribution with independent entries taking $\pm 1/\sqrt{\text{M}}$ also yields also preserves RIP condition [16].

Besides these random projection matrices, there are other ways to choose a projection matrix. For example, a projection matrix can be obtained from orthonormal bases. Candès et al. [16] proved that if $\text{U}$ is an orthonormal matrix, and $\text{M}$ rows of this matrix at random are selected as a projection matrix, and by renormalizing its columns it can be used as a projection matrix provided

$$\text{k}\le \text{C}.\frac{1}{\mathrm{\mu }{\left(\mathrm{\Phi },\mathrm{\psi }\right)}^2}.\frac{\text{M}}{{(log\text{N})}^6},\mathrm{\mu }\left(\mathrm{\Phi },\mathrm{\psi }\right)=\sqrt{\text{N}}.\text{max}|<{\mathrm{\Phi }}_{\text{i}},{\mathrm{\psi }}_{\text{j}}>|,{\mathrm{\Phi }}_{\text{i}}\in \mathrm{\Phi },{\mathrm{\psi }}_{\text{j}}\in \mathrm{\psi },$$ 5

where $\text{k}$ is the order of sparsity, $\mathrm{\mu }\left(\mathrm{\Phi },\mathrm{\psi }\right)$ is mutual coherence between $\mathrm{\Phi }$ and $\mathrm{\psi }$ that measures their similarity. For instance, if we choose the rows of $\mathrm{\Phi }$ from Fourier bases, and $\mathrm{\psi }$ as an identity matrix, then $\mathrm{\mu }$ = 1. Matrices that provide a smaller μ are desirable for sensing. Also, maximum coherence of a matrix, can be used to verify the RIP condition. By definition, maximum coherence of a matrix ${\text{A}}_{\text{M}\times \text{N}}$ is

$$\mathrm{\mu }=\underset{i\ne j}{max}\frac{\left|<A_i,A_j>\right|}{{∥A_i∥}_2{∥A_j∥}_2}1\le i,j\le N$$ 6

where ${\text{A}}_{\text{i}}$ and ${\text{A}}_{\text{j}}$ denote columns of $A$. Welch bound [17] may be used to determine the smallest μ. Welch bound states that any arbitrary ${\text{A}}_{\text{M}\times \text{N}}$ has $\mathrm{\mu }\ge \sqrt{\text{(N}-\text{M)}/\text{M}(\text{N}-1)}$. Unfortunately, a general method to generate matrices that meet this bound has not been introduced yet. Given the matrix $\text{A}$ with its $\mathrm{\mu }$, it can preserve RIP condition of order $2\text{k}$ with ${\mathrm{\delta }}_{2\text{k}}<\mu (2k-1)$ while $2\text{k}<1/\mathrm{\mu }+1$, [18]. Once the appropriate matrices are designed or chosen, then sensing can be performed. After the sensing phase, in the recovery phase, we need an algorithm to recover the initial signal. There are numerous algorithms to reconstruct initial signal, including interior-point algorithms [19, 20], gradient projection [21], greedy approaches such as orthogonal matching pursuit (OMP) [22, 23], and smooth- zero norm [24].

Length of sparse signal in CS

The length of sparse signal plays a significant role in the compression. The length of sparse signal has direct relation with the size of projection matrix, the compression ratio, the computation complexity, and the order of sparsity. Given ${\text{y}}_{\text{m}\times 1}={\mathrm{\Phi }}_{\text{m}\times \text{n}}{\text{x}}_{\text{n}\times 1}$, length of $\text{x}$ is equal to the number of columns in $\mathrm{\Phi }$; as the former increases, the latter increases accordingly. Then, $\mathrm{\Phi }$ has more elements and require much storage space. In addition, the product of $\mathrm{\Phi }$ by $\text{x}$ requires $\text{m}\times \text{n}$ multiplications, and $\text{m}\times (\text{n}-1)$ additions. For instance, given $\text{CR}=50\text{\%}$ or $\text{m}=\text{n}/2$, for sensing 256 samples, if we choose $\text{n}=256$, we need $32768$ multiplication operations, while if we divide it into two separate segments of $\text{n}=128$ samples, then we need 8192 × 2 = 16384 multiplication operations. Thereby, it will be more efficient to sense signals in smaller lengths. This observation will be more relevant, when we use CS in devices such as a wearable ECG recorders that have limited power supply, memory for storage and computational capability. Wearable ECG recorders, collect ECG signals for several days and nights, and transfer them to a remote medical centre for further processing. In such devices with limited resources, CS should be applied in a fast and energy-efficient manner. When sensors acquire ECG signal in smaller lengths, fewer multiplication and addition operations are required during the compressive phase to obtain the measurement vector. As order of sparsity and number of measurements are related to the length of the signal, the choice of length of the signal has implications during the recovery phase. In this paper, we propose a Kronecker-based recovery of compressively sampled signals. During the sensing phase, signals are sensed with limited number of samples while during the recovery phase we concatenate measurement vectors to do recovery in larger size, and utilizing Kronecker technique to increase the recovery quality.

Kronecker product

If $\text{B}$ is a $\text{p}\times \text{q}$ matrix and $\text{A}$ is an $\text{m}\times \text{n}$ matrix, then the Kronecker product of $\text{B}$ and $\text{A}$ is the $\text{pm}\times \text{qn}$ matrix

$${\text{B}}_{\text{p}\times \text{q}}\otimes A_{\text{m}\times \text{n}}=\left[\begin{array}{cccc}{b}_{11}A& b_{12}A& \cdots & b_{1q}A\\ b_{21}A& b_{22}A& \cdots & b_{2q}A\\ ⋮& ⋮& \ddots & ⋮\\ b_{p1}A& b_{p2}A& \cdots & b_{\mathit{pq}}A\end{array}\right]$$ 7

For a special case, when $\text{B}$ is an identity matrix ${\text{I}}_{\text{p}\times \text{p}}$—a square matrix with ones on the main diagonal and zeros elsewhere—the Kronecker product is

$${\text{I}}_{\text{p}\times \text{p}}\otimes A_{\text{m}\times \text{n}}=\left[\begin{array}{cccc}A& 0& \cdots & 0\\ 0& A& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & A\end{array}\right]$$ 8

Orthonormal matrices such as Identity matrix have RIP constant ${\mathrm{\delta }}_{\text{K}}=0$. Duarte et al. have proved that RIP constant for new measurement matrix of Kronecker product obeys the following inequality

$${\mathrm{\delta }}_{\text{K}}\left({\text{A}}_1\otimes {\text{A}}_2\otimes \dots \otimes {\text{A}}_{\text{I}}\right)\le \prod _{\text{i}=1}^{\text{I}}\left(1+{\mathrm{\delta }}_{\text{K}}\left({\text{A}}_{\text{i}}\right)\right)-1$$ 9

where ${\mathrm{\delta }}_{\text{K}}\left({\text{A}}_{\text{i}}\right)$ is the RIP constant of matrix ${\text{A}}_{\text{i}}$, [25, 26]. If RIP constant is smaller, then the matrix is more efficient for sensing. In (10), the output RIP constant is not larger than that of the main matrix. Although we cannot certainly say that Kronecker product will improve the RIP constant, but we can assert that it will not have a negative effect on RIP constant of Kronecker product.

$$\left\{\begin{array}{c}{\mathrm{\delta }}_{\text{K}}\left(\text{I}\otimes \text{A}\right)\le \left(\left(1+{\mathrm{\delta }}_{\text{K}}\left(\text{I}\right)\right)\left(1+{\mathrm{\delta }}_{\text{K}}\left(\text{A}\right)\right)\right)-1\hfill \\ {\mathrm{\delta }}_{\text{K}}\left(\text{I}\otimes \text{A}\right)\le \left(1+{\mathrm{\delta }}_{\text{K}}\left(\text{A}\right)\right)-1\hfill \\ {\mathrm{\delta }}_{\text{K}}\left(\text{I}\otimes \text{A}\right)\le {\mathrm{\delta }}_{\text{K}}\left(\text{A}\right)\hfill \end{array}\right.$$ 10

Proposed Kronecker technique

In the recovery phase, the measurement vector and the product of projection matrix and sparsifying basis are assumed to be available. Given these pairs (${\text{y}}_{\text{m}\times 1},{\mathrm{\Phi }}_{\text{m}\times \text{n}}{\mathrm{\psi }}_{\text{n}\times \text{n}}$), the initial sparse signal ${\text{s}}_{\text{n}\times 1}$ can be reconstructed. For simplicity, we denote the output of recovery algorithms by $\mathcal{F}$

$${\text{s}}_{\text{n}\times 1}=\mathcal{F}({\text{y}}_{\text{m}\times 1},{\mathrm{\Phi }}_{\text{m}\times \text{n}}{\mathrm{\psi }}_{\text{n}\times \text{n}})$$ 11

In many CS applications, due to the limited capability of the device or due to the need for fast compression, sensing phase is done in piecewise fashion. The signal instead of being sensed as a single vector of length N, it is sensed as shorter signals of length less than N leading to measurement vectors of a number of consecutive segments. Let $\text{Y}=\{{\text{y}}_{\text{m}\times 1}^1,{\text{y}}_{\text{m}\times 1}^2,\dots ,{\text{y}}_{\text{m}\times 1}^{\text{l}}\}$ as a set of consecutive measurement vectors where ${\text{y}}_{\text{m}\times 1}^{\text{i}}$ stands for $\text{ith}$ measurement vector and $\text{S}=\{{\text{s}}_{\text{n}\times 1}^1,{\text{s}}_{\text{n}\times 1}^2,\dots ,{\text{s}}_{\text{n}\times 1}^{\text{l}}\}$ is a set of corresponding sparse vectors.

Let us consider an orthonormal dictionary as the sparsifying dictionary. Without loss of generality, let us consider the orthonormal DCT dictionary as the sparsifying dictionary as it is most commonly used for compression in CS. Let us assume that we have a sensed a sparse signal of length $2n$, which may also be viewed as either one long sensed signal or a two consecutively sensed sparse signals each of equal lengths n leading to measurement vectors of length $2m$ and m respectively. Recovery may be attempted in one of the three methods given below, of which the third method (Method C) is the proposed method.

Method A

This method refers to the common CS recovery methods available currently in the literature in which each measurement vector is considered separately for reconstruction.

Method B

In Method B, instead of using just one long measurement vector, the measurement vector may be segmented into several consecutive measurements of equal lengths. Alternatively, we may view this as a sensor that is producing several smaller equal length compressively sensed measurements. Without loss of generality, let us assume that we have divided the single measurement vector into two equal length measurement vectors. If we concatenate two consecutive measurement vectors, ${\text{y}}_{2\text{m}\times 1}=[{\text{y}}_{\text{m}\times 1}^1,{\text{y}}_{\text{m}\times 1}^2]$, and then via ${\widehat{\text{A}}}_{2\text{m}\times 2\text{n}}={\text{I}}_{2\times 2}\otimes ({\mathrm{\Phi }}_{\text{m}\times \text{n}}{\mathrm{\psi }}_{\text{n}\times \text{n}})$, two consecutive sparse vectors ${\text{s}}_{2\text{n}\times 1}=[{\text{s}}_{\text{n}\times 1}^1,{\text{s}}_{\text{n}\times 1}^2]=\mathcal{F}({\text{y}}_{2\text{m}\times 1},{\widehat{\text{A}}}_{2\text{m}\times 2\text{n}})$ can be reconstructed. Instead of applying recovery algorithms two times for two measurement vectors, we concatenate measurement vectors and then apply the recovery algorithm at once. In this case, the new matrix is

$$\begin{array}{c}\hfill {\widehat{A}}_{2\text{m}\times 2\text{n}}={\text{I}}_{2\times 2}\otimes ({\mathrm{\Phi }}_{\text{m}\times \text{n}}{\mathrm{\psi }}_{\text{n}\times \text{n}})=\\ \hfill ({\text{I}}_{2\times 2}\otimes {\mathrm{\Phi }}_{m\times n})({\text{I}}_{2\times 2}\otimes {\mathrm{\psi }}_{\text{n}\times \text{n}})=\\ \hfill \left(\begin{array}{cc}{\mathrm{\Phi }}_{\text{m}\times \text{n}}{\mathrm{\psi }}_{\text{n}\times \text{n}}\hfill & 0\hfill \\ 0\hfill & {\mathrm{\Phi }}_{\text{m}\times \text{n}}{\mathrm{\psi }}_{\text{n}\times \text{n}}\hfill \end{array}\right)=\left(\begin{array}{cc}{\mathrm{\Phi }}_{\text{m}\times \text{n}}& 0\\ 0& {\mathrm{\Phi }}_{\text{m}\times \text{n}}\end{array}\right)\left(\begin{array}{cc}{\mathrm{\psi }}_{\text{n}\times \text{n}}\hfill & 0\hfill \\ 0\hfill & {\mathrm{\psi }}_{\text{n}\times \text{n}}\hfill \end{array}\right)\\ \hfill \end{array}$$ 12

For simplicity we exemplified concatenating two measurement vectors, but this procedure can be done for larger number of measurement vectors. For concatenating $\text{t}$ consecutive measurement vectors, we have ${\text{y}}_{\text{tm}\times 1}^{\text{i}}=[{\text{y}}_{\text{m}\times 1}^{\text{i}},{\text{y}}_{\text{m}\times 1}^{\text{i}+1},\dots ,{\text{y}}_{\text{m}\times 1}^{\text{i}+\text{t}-1}]$, where i = 0,1,…,$\frac{l}{t}$ and l is the total number of segments and is a multiple of t and via ${\widehat{\text{A}}}_{\text{tm}\times \text{tn}}={\text{I}}_{\text{t}\times \text{t}}\otimes ({\mathrm{\Phi }}_{\text{m}\times \text{n}}{\mathrm{\psi }}_{\text{n}\times \text{n}})$, one can reconstruct $\text{t}$ corresponding sparse vectors

$${\text{s}}_{\text{tn}\times 1}^{\text{i}}=[{\text{s}}_{\text{n}\times 1}^{\text{i}},{\text{s}}_{\text{n}\times 1}^{\text{i}+1},\dots ,{\text{s}}_{\text{n}\times 1}^{\text{i}+\text{t}-1}]=\mathcal{F}({\text{y}}_{\text{tm}\times 1}^{\text{i}},{\widehat{\text{A}}}_{\text{tm}\times \text{tn}})$$ 13

Since in this paper, the factor $\text{t}$ repetitively is used, we call this factor as Kronecker factor.

Method C

In (12), the projection matrix $\mathrm{\Phi }$ is fixed and its elements are unchangeable. But, $\mathrm{\psi }$ can be changed; hence, we can choose the best one. Instead of using matrix (12) we propose (14), and we will show that this matrix can improve the quality of reconstruction.

$$\left(\begin{array}{cc}{\mathrm{\Phi }}_{\text{m}\times \text{n}}& 0\\ 0& {\mathrm{\Phi }}_{\text{m}\times \text{n}}\end{array}\right).\left({\mathrm{\psi }}_{2\text{n}\times 2\text{n}}\right)$$ 14

In (14), we considered two consecutive measurement vectors, however, we can increase the number of consecutive measurement vectors. For $\text{t}>2$ the sparsifying basis will be ${\mathrm{\psi }}_{\text{tn}\times \text{tn}}$.

We assert that our proposed technique decreases the mutual coherence factor, thereby improving the signal quality. As aforementioned, in the recovery phase lower mutual coherence leads to better reconstruction. For a given measurement vector and a projection matrix, we try to apply the sparsifying basis in an efficient manner. In CS applications, we try to find a projection matrix to be incoherent enough with sparsifying basis; but in our work, we assert that for a chosen projection matrix and sparsifying basis if we apply our Kronecker technique, we decrease the mutual coherence. Hence, we do not change the nature of sparsifying basis or projection matrix. For simplicity, we use $\mathrm{\Phi }=({\text{I}}_{\text{t}\times \text{t}}\otimes {\mathrm{\Phi }}_{\text{m}\times \text{n}})$, ${\mathrm{\psi }}_{\text{s}}=({\text{I}}_{\text{t}\times \text{t}}\otimes {\mathrm{\psi }}_{\text{n}\times \text{n}})$, and ${\mathrm{\psi }}_{\text{l}}={\mathrm{\psi }}_{\text{tn}\times \text{tn}}$ as symbols for projection matrix denoting, small sparsifying basis, and large sparsifying basis, respectively. The coefficients of the sparsifying dictionary, which is orthonormal DCT in this paper ${\mathrm{\psi }}_{\text{n}\times \text{n}}$, are shown in (15),

$${\mathrm{\psi }}_{\text{ij}}=\text{w}\left(\text{j}\right)cos\frac{\mathrm{\pi }\left(2\text{i}-1\right)\left(\text{j}-1\right)}{2\text{n}},\text{i}=1,2,\dots ,\text{n}$$

$$\text{w}\left(\text{j}\right)=\left\{\begin{array}{cc}\sqrt{\frac{1}{\text{n}}}& \text{j}=1\\ \sqrt{\frac{2}{\text{n}}}& 2\le \text{j}\le n\end{array}\right.$$ 15

where ${\mathrm{\psi }}_{\mathit{ij}}$ corresponds to the ith row and jth column of the dictionary. Equation (15) shows that the maximum value of the element in a DCT dictionary is $1/\sqrt{\text{n}}$, where $\text{n}$ is the number of rows or columns; therefore, the maximum element in ${\mathrm{\psi }}_{\text{s}}$ $({\mathrm{\psi }}_{\text{n}\times \text{n}})$ is $\sqrt{\text{t}}$ times greater than that of the ${\mathrm{\psi }}_{\text{l}}({\mathrm{\psi }}_{\text{tn}\times \text{tn}})$. In other words, if we increase the size of a normalized DCT dictionary, the maximum element in dictionary will be decreased, thereby having smaller coefficients. Because of zero matrices in projection matrix, in (14), just 50% of DCT coefficients are used in each multiplication and others do not contribute. Since each ${\mathrm{\psi }}_{\text{l}}$ is normalized, if we use 50% of its coefficients, its Euclidian norm or energy will be smaller. On the contrary, in Method B, all coefficients in ${\mathrm{\psi }}_{\text{s}}$ is used in multiplication and its energy is equal to one. For instance, consider two DCT dictionaries with different sizes, one with 64 columns and other with 512 columns. Since both of them are orthonormal, for the dictionary with 64 columns, the normalized energy of each column is shared equally by 64 elements; on the other hand, for the dictionary with 512 columns, it will be shared equally by 512 elements. The larger DCT matrix has smaller elements than that of the smaller DCT matrix. Hence, the mutual coherence of a large DCT matrix is lower than small DCT matrix. Projection matrices such as random matrices, theoretically, are assumed to be incoherent with any sparsifying dictionary. By choosing any sparsifying matrix according to our proposed method, Method C, a reduced mutual coherence between the projection matrix and the sparsifying matrix may be obtained. This is shown by the equation below

$$\mathrm{\mu }\left(\mathrm{\Phi },{\mathrm{\psi }}_{\text{l}}\right)<\mu \left(\mathrm{\Phi },{\mathrm{\psi }}_{\text{s}}\right)$$ 16

Figure 1 shows the mutual coherence of different projection matrices, namely, random Gaussian distribution $\mathcal{N}(0,1/m)$, symmetric Bernoulli distribution $(P\left({\mathrm{\Phi }}_{\mathit{ij}}=\pm 1/\sqrt{m}\right)=0.5)$, general orthogonal measurement ensembles which in [16] was introduced as a reliable way to generate projection matrices, and selecting $m$ rows of Fourier basis at random. The mutual coherence of these mentioned projection matrices has been checked by simulation using Method B and our proposed approach. In this simulation, $n=32$, $m=16$, $t=4$. Therefore, ${\mathrm{\psi }}_s=({\text{I}}_{4\times 4}\otimes {\mathrm{\psi }}_{32\times 32})$, and ${\mathrm{\psi }}_l={\mathrm{\psi }}_{128\times 128}$. The simulation is run 1000 times. Also, simulations (omitted for lack of space) for different values of $(\text{n},\text{m},\text{t})$ substantiated this reduction in mutual coherence. In Fig. 1, the horizontal axis shows the number of iterations. For instance, a value 100 on the x axis means for $100\text{th}$ time random matrices, Gaussian $\mathcal{N}(0,1/16)$, and symmetric Bernoulli distribution $\text{P}\left({\mathrm{\Phi }}_{\text{ij}}=\pm 1/\sqrt{16}\right)=0.5$ were generated to check their mutual coherence with DCT bases in both Methods B and C. In addition, projection matrices based on orthogonal ensembles and Fourier basis were generated, i.e. selecting $\text{m}$ rows of these matrices at random. We did not show the ensemble average results of our simulation just to emphasis the fact that for all 1000 times, the proposed Method C provided better mutual coherence than Method B. Figure 1 shows that for DCT bases, if we use our proposed Kronecker technique, the mutual coherence will be decreased.

Fig. 1.

Comparing the mutual coherence of different projection matrices with two sparsifying bases, according to Method B and Method C

Experimental results and discussion

To demonstrate the applicability of our proposed method for improvement of quality of reconstructed signal, we carried out reconstruction of compressively sampled ECG signals. We chose to demonstrate our proposed method for ECG signals as they are highly compressible. We chose the ECG signal, record no.100 from standard MIT-BIH Arrhythmia Database, [27]. We conducted our simulations in MATLAB with an Intel core 2 Due CPU, with a 2.00-GHz clock frequency and a 3.00-GHz RAM module. We used SNR to measure the quality of reconstructed signal as follows

$$\text{SNR}=20{log}_{10}\frac{{∥\text{x}∥}_2}{{∥\text{x}-\widehat{\text{x}}∥}_2}$$ 17

where x is the original ECG signal and $\widehat{\text{x}}$ is the recovered ECG signal. Although other researchers have used percentage root mean square difference (PRD) [28], for quality measure of compression, inference obtained using SNR will be the same as PRD. In our experiment, $1024$ samples of an ECG signal were chosen, and for different CR, namely, 50, 75 and 87%, we checked the effectiveness of our technique, Method C.

In Fig. 2, “Ordinary recovery” refers to Method B and “Proposed recovery” refers to Method C. In Fig. 2, the results of a comprehensive experiment on 1024 samples of the ECG signal are shown. Theoretically, it is possible to sense whole 1024 samples at once; however, as aforementioned, in practice piecewise sensing is more efficient. To do the piecewise sensing process, we divide the $1024$ ECG samples into a number of segments of $16$, $32$, $64$, $128$, $256$, and $512$ samples. For example, $1024=8\times 128$, we sense whole signal in eight $128$-sample blocks. Similarly, as $1024=32\times 32$, we use thirty-two $32$-sample blocks, and so on. For CR = 50%, CR = 75%, and CR = 87%, each 128-sample block is mapped into $64$, $32$, $16$ measurements, respectively. We used three types of projection matrices normally used in CS: Gaussian $\mathcal{N}(0,1/\text{m})$, and symmetric Bernoulli distribution $\text{P}\left({\mathrm{\Phi }}_{\text{ij}}=\pm 1/\sqrt{\text{m}}\right)=0.5$, and selecting $\text{m}$ rows of an orthonormal basis at random. We used SL0 (smoothed L0 norm) [24] to reconstruct the signal.

Fig. 2.

Comparing the quality of reconstruction for ECG signal. Projection matrices: a Gaussian distribution, b Bernoulli distribution, c General Ensemble according to [16]. The results are based on SNR scale, and each one is obtained from the mean of 500 tests for CR = [50%, 75%, 87%], and $\text{n}=16,32,64,128,512$. Ordinary method is recovery based on Method B, and proposed method is based on Method C

For each $\left(\text{n},\text{CR}\right)$ combination, we ran 500 times recovery via Methods B and C, and we have presented the averaged result. In Fig. 2, (a) is the result of Gaussian distribution, (b) is the result based on Bernoulli distribution and (c) is the result that was obtained from a measurement matrix whose rows have been selected at random from an orthonormal basis. In Method C, 512 samples were considered as a reference of Kronecker Factor, ${\text{t}}_{\text{n}}=\frac{512}{\text{n}}$. For instance, given $\text{n}=16$, ${\text{t}}_{16}=\frac{512}{16}=32,{\mathrm{\psi }}_{\text{s}}=({\text{I}}_{32\times 32}\otimes {\mathrm{\psi }}_{16\times 16})$, ${\mathrm{\psi }}_{\text{s}}=({\text{I}}_{32\times 32}\otimes {\mathrm{\psi }}_{16\times 16})$, and ${\mathrm{\psi }}_{\text{l}}={\mathrm{\psi }}_{512\times 512}$. With 32 segments concatenated, we need to do recovery twice as we have 64 segments with 16 samples each. As it can be seen, when $\text{n}=512$, ${\text{t}}_{512}=\frac{512}{512}=1$, ${\mathrm{\psi }}_{\text{s}}=({\text{I}}_{1\times 1}\otimes {\mathrm{\psi }}_{512\times 512})$, and ${\mathrm{\psi }}_{\text{l}}={\mathrm{\psi }}_{512\times 512}$. Since ${\mathrm{\psi }}_{\text{s}}={\mathrm{\psi }}_{\text{l}}$, both methods have same SNR. After choosing CR and projection matrices, we ran the recovery of ECG signal using smoothed L0 norm (SL0) algorithm. From Fig. 2, it can be seen that proposed technique increases the quality of reconstructed signal. As the CR increases, the quality of reconstruction decreases. Once sensing is done, we cannot change the projection matrix or increase the number of measurements. However, for a given measurements and a projection matrix, through Kronecker technique, we can increase the quality of reconstructed signal.

To show effect of the proposed method on quality of the reconstructed ECG signal quality, 1024 sample of ECG signal was chosen and compressed in a piecewise manner, with $\text{n}=64$, $\text{CR}=50\text{\%}$, Kronecker factor at $8$, a Gaussian projection matrix $\mathcal{N}(0,1/32)$ and accordingly ${\mathrm{\psi }}_{\text{s}}=({\text{I}}_{8\times 8}\otimes {\mathrm{\psi }}_{64\times 64})$, and ${\mathrm{\psi }}_{\text{l}}={\mathrm{\psi }}_{512\times 512}$. Figure 3 shows the difference in the quality of reconstruction between our proposed Kronecker technique ($43.5\text{dB}$) and Method B ($37.1\text{dB}$). The proposed method, in this case, delivers an improvement of $6\text{dB}$ and thus a better quality reconstructed signal.

Fig. 3.

CS-recovery ECG signal, Method C (upper graph) and Method B (lower graph). SNR of Method C $(43.5\text{dB}$) which is the proposed method and the SNR of ordinary CS recovery, Method B $\left(37.1\text{dB}\right)$

Comparison of Methods A and B

The quality of reconstructed signal in Method A is equal to that of Method B. Kronecker product increases the dimension, but the content of the reconstructed signal does not change. This claim is proved in the Appendix. In the following paragraph, we will demonstrate the quality of reconstruction is the same for both the methods. We will use OMP as the reconstruction algorithm for discussion sake.

When we have two measurement vectors,$({\text{y}}^1,{\text{y}}^2)$, if we reconstruct using Method $\text{A}$, OMP algorithm must be run twice. Suppose after the first run, we obtain the coefficients of the first sparse vector $[{\text{s}}_1^1,{\text{s}}_2^1,\dots ,{\text{s}}_{\text{n}}^1]$, and after second run we obtain the coefficients of second sparse vector $[{\text{s}}_1^2,{\text{s}}_2^2,\dots ,{\text{s}}_{\text{n}}^2]$. OMP in first iteration tries to find the most correlated column with the measurement vector. OMP projects measurement vector to that column and then starts to find the most correlated column for obtained residue. In each iteration, OMP maps residue to the set of obtained columns and it prevents choosing the same column repetitively and this leads to faster convergence. If we run Method B, the coefficients will be exactly augmented as $[{\text{s}}_1^1,{\text{s}}_2^1,\dots ,{\text{s}}_{\text{n}}^1,{\text{s}}_1^2,{\text{s}}_2^2,\dots ,{\text{s}}_{\text{n}}^2]$. In Method B, the position and the value of coefficients are fixed, however, the order of finding coefficients will be different. If we have strictly sparse vectors—exactly $\text{k}$ nonzero—the results of Methods A and B are the same. However, when we deal with noisy and compressible signals, the results of A and B will be approximately equal to each other; the difference is less than the contributed noise energy and generally the difference is negligible.

Conclusion

We have presented a novel way of improving the quality of reconstruction of compressively sampled signals through Kronecker technique. Through Kronecker technique during recovery, we have introduced a simple pre-processing in the recovery phase before applying the recovery algorithm. Although, we increased the dimension of the signal under recovery, instead of multiple recoveries, we have proposed a “one-shot” recovery. We believe that our proposed method has no restriction on the choice of sparsifying dictionaries and will perform equally well with both fixed and adaptive dictionaries. The proposed method also works for almost all well-known projection matrices such as random matrices, or deterministic matrices. We have demonstrated a practical application of our proposed technique through compression of ECG signal using CS. This proposed method is viable for wearable devices that have limited storing and processing capabilities. Such wearable devices like wearable ECG recorders, may compress the data in an energy efficient manner and send them to a remote processing centre where it can be recovered on demand. Using the proposed method, we were able to decrease the length of input signal in the compression phase, thereby decreasing the computation and the processing time. However, decreasing the length of input vector affects the order of sparsity and quality of signal. Quality of signal is then recovered through the Kronecker technique during the recovery process.

Appendix

To show that Methods A and B have equal quality, we start with the definition of CS recovery in (18)

$${\text{min}}_{\text{s}\in {\mathbb{R}}^{\text{n}}}{∥\text{s}∥}_1\text{s}.\text{t}.\text{y}=(\mathrm{\Phi }\mathrm{\psi })\text{s}$$

$$\text{A}=\mathrm{\Phi }\mathrm{\psi }:{\text{min}}_{\text{s}\in {\mathbb{R}}^{\text{n}}}{∥\text{s}∥}_1\text{s}.\text{t}.\text{y}=\text{As}$$

$${\text{min}}_{\text{s}\in {\mathbb{R}}^{\text{n}}}\sum _{\text{i}=1}^{\text{n}}\left|{\text{s}}_{\text{i}}\right|$$

$$\text{s}.\text{t}{\text{y}}_{\text{i}}=\sum _{\text{j}=1}^{\text{n}}{\text{a}}_{\text{ij}}{\text{s}}_{\text{j}}\text{i}=1,2,\dots ,\text{m}$$ 18

For two measurement vectors, $\{{\text{y}}^1,{\text{y}}^2\}\in {\mathbb{R}}^{\text{m}}$, that have been sensed by the same projection matrix ${\mathrm{\Phi }}_{\text{m}\times \text{n}}$, according to Method A, (18) must be solved twice:

$$\left\{\begin{array}{ccc}{\text{min}}_{{\text{s}}^1\in {\mathbb{R}}^{\text{n}}}\sum _{\text{i}=1}^{\text{n}}\left|{\text{s}}_{\text{i}}^1\right|\hfill & \text{s}.\text{t}{\text{y}}_{\text{i}}^1=\sum _{\text{j}=1}^{\text{n}}{\text{a}}_{\text{ij}}{\text{s}}_{\text{j}}^1& \text{i}=1,2,\dots ,\text{m}\\ {\text{min}}_{{\text{s}}^2\in {\mathbb{R}}^{\text{n}}}\sum _{\text{i}=1}^{\text{n}}\left|{\text{s}}_{\text{i}}^2\right|\hfill & \text{s}.\text{t}{\text{y}}_{\text{i}}^2=\sum _{\text{j}=1}^{\text{n}}{\text{a}}_{\text{ij}}{\text{s}}_{\text{j}}^2& \text{i}=1,2,\dots ,\text{m}\end{array}\right.$$ 19

In Method B, we concatenate the two measurement vectors into one measurement vector, $\tilde{\text{y}}=[{\text{y}}^1,{\text{y}}^2]\in {\mathbb{R}}^{2\text{m}}$, and via ${\tilde{\text{A}}}_{2\text{m}\times 2\text{n}}={\text{I}}_{2\times 2}\otimes {\text{A}}_{\text{m}\times \text{n}}={\text{I}}_{2\times 2}\otimes \left({\mathrm{\Phi }}_{\text{m}\times \text{n}}{\mathrm{\psi }}_{\text{n}\times \text{n}}\right)$ to do the recovery:

$${\text{min}}_{\tilde{\text{s}}\in {\mathbb{R}}^{\text{n}}}\sum _{\text{i}=1}^{2\text{n}}\left|{\tilde{\text{s}}}_{\text{i}}\right|\text{s}.\text{t}{\tilde{\text{y}}}_{\text{i}}=\sum _{\text{j}=1}^{2\text{n}}{\tilde{\text{a}}}_{\text{ij}}{\tilde{\text{s}}}_{\text{j}}\text{i}=1,2,\dots ,2\text{m}$$

$${\text{min}}_{\tilde{\text{s}}\in {\mathbb{R}}^{\text{n}}}\left(\sum _{\text{i}=1}^{\text{n}}\left|{\tilde{\text{s}}}_{\text{i}}\right|+\sum _{\text{i}=\text{n}+1}^{2\text{n}}\left|{\tilde{\text{s}}}_{\text{i}}\right|\right)$$

$$\text{s}.\text{t}{\tilde{\text{y}}}_{\text{i}}=\sum _{\text{j}=1}^{2\text{n}}{\tilde{\text{a}}}_{\text{ij}}{\tilde{\text{s}}}_{\text{j}}\text{i}=1,2,\dots ,2\text{m}$$

Because of zero matrices in $\tilde{\text{A}}$,

$${\text{min}}_{\tilde{\text{s}}\in {\mathbb{R}}^{\text{n}}}\left(\sum _{\text{i}=1}^{\text{n}}\left|{\tilde{\text{s}}}_{\text{i}}\right|+\sum _{\text{i}=\text{n}+1}^{2\text{n}}\left|{\tilde{\text{s}}}_{\text{i}}\right|\right)$$

$$\text{s}.\text{t}\left\{\begin{array}{cc}{\tilde{\text{y}}}_{\text{i}}=\sum _{\text{j}=1}^{\text{n}}{\tilde{\text{a}}}_{\text{ij}}{\tilde{\text{s}}}_{\text{j}}& \text{i}=1,2,\dots ,\text{m}\\ {\tilde{\text{y}}}_{\text{i}}=\sum _{\text{j}=\text{n}+1}^{2\text{n}}{\tilde{\text{a}}}_{\text{ij}}{\tilde{\text{s}}}_{\text{j}}& \text{i}=m+1,\dots ,2\text{m}\end{array}\right.$$

${\tilde{\text{y}}}_{\text{i}},\text{i}=1,2,\dots ,\text{m}$ contain exactly the elements of ${\text{y}}^1$, and ${\tilde{\text{y}}}_{\text{i}},\text{i}=\text{m}+1,\dots ,2\text{m}$ contain elements of ${\text{y}}^2$. Also, ${\tilde{\text{a}}}_{\text{ij}}\left\{\begin{array}{c}\text{i}=1,2,\dots ,\text{m}\\ \text{j}=1,2,\dots ,\text{n}\end{array}\right.$ and ${\tilde{\text{a}}}_{\text{ij}}\left\{\begin{array}{c}\text{i}=\text{m}+1,\dots ,2\text{m}\\ \text{j}=\text{n}+1,\dots ,2\text{n}\end{array}\right.$ contain the elements of matrix $\text{A}$. Hence,

$${\text{min}}_{\tilde{\text{s}}\in {\mathbb{R}}^{\text{n}}}\left(\sum _{\text{i}=1}^{\text{n}}\left|{\tilde{\text{s}}}_{\text{i}}\right|+\sum _{\text{i}=\text{n}+1}^{2\text{n}}\left|{\tilde{\text{s}}}_{\text{i}}\right|\right)$$

$$\text{s}.\text{t}\left\{\begin{array}{cc}{\text{y}}_{\text{i}}^1=\sum _{\text{j}=1}^{\text{n}}{\text{a}}_{\text{ij}}{\tilde{\text{s}}}_{\text{j}}& \text{i}=1,2,\dots ,\text{m}\\ {\text{y}}_{\text{i}}^2=\sum _{\text{j}=\text{n}+1}^{2\text{n}}{\text{a}}_{\text{ij}}{\tilde{\text{s}}}_{\text{j}}& \text{i}=1,2,\dots ,\text{m}\end{array}\right.$$ 20

In (20), two independent vectors are minimized via independent conditions. Let ${\tilde{\text{s}}}^1\in {\mathbb{R}}^{\text{n}}$ and ${\tilde{\text{s}}}^2\in {\mathbb{R}}^{\text{n}}$ comprise the first and second $\text{n}$ components of $\tilde{\text{s}}\in {\mathbb{R}}^{2\text{n}}$ . Therefore, we can rewrite (20) into two independent minimization equations,

$$\left\{\begin{array}{ccc}{\text{min}}_{{\tilde{\text{s}}}^1\in {\mathbb{R}}^{\text{n}}}\sum _{\text{i}=1}^{\text{n}}\left|{\tilde{\text{s}}}_{\text{i}}^1\right|\hfill & \text{s}.\text{t}{\text{y}}_{\text{i}}^1=\sum _{\text{j}=1}^{\text{n}}{\text{a}}_{\text{ij}}{\tilde{\text{s}}}_{\text{j}}^1& \text{i}=1,2,\dots ,\text{m}\\ {\text{min}}_{{\tilde{\text{s}}}^2\in {\mathbb{R}}^{\text{n}}}\sum _{\text{i}=1}^{\text{n}}\left|{\tilde{\text{s}}}_{\text{i}}^2\right|\hfill & \text{s}.\text{t}{\text{y}}_{\text{i}}^2=\sum _{\text{j}=1}^{\text{n}}{\text{a}}_{\text{ij}}{\tilde{\text{s}}}_{\text{j}}^2& \text{i}=1,2,\dots ,\text{m}\end{array}\right.$$ 21

Equation (21) is equal with (19), therefore Method A and B have same results. This procedure can be extended to more than two measurements.

Conflict of interest

There is no conflict of interests among authors.

Ethical approval

Authors used the data available in [27] for their study and did not collect data from any human participant or animal.