Sparse sampling: theory, methods and an application in neuroscience

Abstract

The current methods used to convert analogue signals into discrete-time sequences have been deeply influenced by the classical Shannon–Whittaker–Kotelnikov sampling theorem. This approach restricts the class of signals that can be sampled and perfectly reconstructed to bandlimited signals. During the last few years, a new framework has emerged that overcomes these limitations and extends sampling theory to a broader class of signals named signals with finite rate of innovation (FRI). Instead of characterising a signal by its frequency content, FRI theory describes it in terms of the innovation parameters per unit of time. Bandlimited signals are thus a subset of this more general definition. In this paper, we provide an overview of this new framework and present the tools required to apply this theory in neuroscience. Specifically, we show how to monitor and infer the spiking activity of individual neurons from two-photon imaging of calcium signals. In this scenario, the problem is reduced to reconstructing a stream of decaying exponentials.

Introduction

The world is analogue, but computation is digital. The process that bridges this gap is known as the sampling process and has been instrumental to the digital revolution of the past 60 years. Without the sampling process, we could not convert real-life signals in digital form, and without digital samples, we could not use computers for digital computation. The sampling process is also ubiquitous in that it is present in any mobile phone or digital camera but also in sophisticated medical devices like MRI or ultrasound machines, in sensor networks and in digital microscopes just to name a few examples.

Over the last six decades, our understanding of the conversion of continuous-time signal in discrete form has been heavily influenced by the Shannon–Whittaker–Kotelnikov sampling theorem (Shannon 1949; Whittaker 1929; Kotelnikov 1933; Unser 2000) which showed that the sampling and perfect reconstruction of signals are possible when the Fourier bandwidth or spectrum of the signal is finite. In this case, the signal is said to be bandlimited and must be sampled at a rate (Nyquist rate) at least twice its maximum nonzero frequency in order to reconstruct it without error.

We are so used to this approach that we tend to forget that it comes with many strings attached. First of all, there are no natural phenomena that are exactly bandlimited (Slepian 1976). Moreover, we tend to forget that the Shannon sampling theorem provides sufficient but not necessary conditions for perfect reconstruction. In other words, this theorem does not claim that it is not possible to sample and reconstruct non-bandlimited signals. It is therefore incorrect to assume that the bandwidth of a signal is related to its information content. Consider for instance the function shown in Fig. 1a. This is a stream of short pulses and appears in many applications including bio-imaging, seismic signals and spread-spectrum communication. If the pulse shape is known a priori, the signal is completely determined by the amplitude and location of such pulses. If there are at most $K$ pulses in a unit interval, then the signal is completely specified by the knowledge of these $2K$ parameters per unit of time. Assume now that the duration of the pulses is reduced but that the average number of pulses per unit interval stays the same. Clearly, the information content of the signal is not changing (still $2K$ parameters per unit of time); however, its bandwidth is increasing (bandwidth increases when the support of a function decreases).

Fig. 1.

Examples of signals with FRI. When the shape of the pulse is known, the signal depends only on the amplitude and location of such pulses

Consider, as second example, the signal shown in Fig. 2c. This is given by the sum of a bandlimited signal with a step function. Clearly, the step function has only two degrees of freedom: the discontinuity location and its amplitude. So, its information content is finite. The bandlimited function has a finite number of degrees of freedom per unit of time since it is fully determined by its samples at points spaced by the sampling period (given by the inverse of the Nyquist rate). We thus say that they both have a finite rate of innovation. However, the combination of these two functions leads to a signal with infinite bandwidth (see Fig. 2d). Now, if we were to relate the information content of the signal to its bandwidth, we would conclude incorrectly that this signal has an infinite rate of information since it requires an infinite sampling rate for perfect reconstruction. Therefore, bandwidth and information content are not always synonyms.

Fig. 2.

Nyquist rate versus information rate. The signal $x_1(t)$ depicted in part a is bandlimited as shown in part b. The sum of $x_1(t)$ with a step function lead to a signal $x_2(t)$ with infinite bandwidth as shown in part c and d

A first attempt to reconcile these two notions: sampling rate and information content was made in Vetterli et al. (2002). Here, they introduced a new class of signals called signals with finite rate of innovation (FRI) which includes both bandlimited signals and the non-bandlimited functions discussed so far. They went on showing that classes of FRI signals can be sampled and perfectly reconstructed using an appropriate acquisition device. These results have then be extended to include more classes of acquisition devices (Dragotti et al. 2007; Seelamantula and Unser 2008; Asl et al. 2010; Tur et al. 2011; Urigüen et al. 2013) and more classes of signals (Maravić and Vetterli 2005; Berent et al. 2010; Chen et al. 2012). FRI sampling theory has also had impact in various applications (Baboulaz and Dragotti 2009; Poh and Marziliano 2010; Tur et al. 2011; Kandaswamy et al. 2013) and here we focus on an application in neuroscience.

The paper is organised as follows. In the next section, we define FRI signals and give some examples. Section 3 presents the framework for sampling and reconstructing some classes of FRI signals. Specifically, we show how to sample and perfectly reconstruct a stream of Diracs and what are the conditions that the acquisition device has to satisfy. We also extend this framework to the case of streams of decaying exponentials and present some denoising strategies. Section 4 presents an algorithm to reconstruct streaming signals where there is no clear separation between consecutive bursts of spikes. Section 5 describes an application of this theory to monitor neural activity from two-photon calcium images. Finally, conclusions are drawn in Sect. 6.

Notations

For $f(t)\in {\mathit{L}}^{\mathbf{2}}(\mathbb{R})$, where ${\mathit{L}}^{\mathbf{2}}(\mathbb{R})$ is the Hilbert space of finite-energy functions, the Fourier transform of $f(t)$ is denoted by $\widehat{f}(\mathit{\omega })$ and is given by $\widehat{f}(\mathit{\omega })=\mathcal{F}\{f(t)\}={\int }_{-\infty }^{+\infty }f(t)e^{-i\mathit{\omega }t}\text{d}t$. If $f(t)$ is complex-valued, $f^{\ast }(t)$ denotes its complex conjugate. The Hermitian inner product is $〈f,g〉={\int }_{-\infty }^{+\infty }f(t)g^{\ast }(t)\text{d}t$. The indicator function is denoted by ${\mathbf{1}}_A(t)$ and is given by ${\mathbf{1}}_A(t)=1$ if $t\in A$, and ${\mathbf{1}}_A(t)=0$ if $t\notin A$. ${\mathit{\delta }}_{i,j}$ denotes the Kronecker delta, which is defined as ${\mathit{\delta }}_{i,j}=1$ if $i=j$ and 0 otherwise. $\lfloor ·\rfloor$ and $\lceil ·\rceil$ denote the floor and ceil functions.

Finite rate of innovation signals

Classical sampling theorems state that any bandlimited function $x(t)$ such that $\widehat{x}(\mathit{\omega })=0,\forall \mathit{\omega }>{\mathit{\omega }}_{max}$, can be perfectly recovered from its samples $x_n{=x(t)|}_{t=nT}$ if the sampling rate $2\mathit{\pi }/T$ is greater than or equal to twice the highest frequency component of $x(t)$, that is, $2\mathit{\pi }/T\ge {\mathit{\omega }}_{max}$. Moreover, the original signal can be perfectly reconstructed as follows:

$$\begin{array}{c}\hfill x(t)=\sum _{n=-\infty }^{\infty }{x}_n\text{sinc}(t/T-n),\end{array}$$ 1

where $\text{sinc}(t)=sin(\mathit{\pi }t)/\mathit{\pi }t$. If $x(t)$ is not bandlimited, sampling with an ideal lowpass filter ($h(t)=\text{sinc}(t/T)$) and reconstruction applying (1) provides a lowpass approximation of $x(t)$. This is the best approximation in the least square sense of $x(t)$ in the space spanned by ${\{\text{sinc}(t/T-n)\}}_{n\in \mathbb{Z}}$ (Unser 2000). However, it is an approximation, and perfect reconstruction of the original signal is not achieved. We also note that signals defined as in (1) are completely specified by the knowledge of a new parameter $x_n$ every $T$ seconds.

Based on this observation, consider now a new class of signals that extend the one in (1) (Vetterli et al. 2002):

$$\begin{array}{c}\hfill x(t)=\sum _{k\in \mathbb{Z}}\sum _{r=0}^R{a}_{r,k}{g}_r(t-t_k),\end{array}$$ 2

where ${\{g_r(t)\}}_{r=0}^R$ is a set of known functions. We note that, since $g_r(t)$ are known, signals in (2) are uniquely determined by the set of parameters $a_{r,k}$ and $t_k$. Introducing a counting function $C_x(t_a,t_b)$ that counts the number of degrees of freedom in $x(t)$ over the interval $\left[t_a,t_b\right]$, we define the rate of innovation $\mathit{\rho }$ as follows (Vetterli et al. 2002; Dragotti et al. 2007; Blu et al. 2008; Urigüen et al. 2013):

$$\begin{array}{c}\hfill \mathit{\rho }=\underset{\mathit{\tau }\to \infty }{lim}\frac{1}{\mathit{\tau }}{C}_x\left(-\frac{\mathit{\tau }}{2},\frac{\mathit{\tau }}{2}\right)\end{array}$$ 3

and signals with a finite $\mathit{\rho }$ are called signals with a finite rate of innovation (FRI).

It is of interest to note that bandlimited signals fall under this definition. Therefore, one possible interpretation is that it is possible to sample them because they have a finite rate of innovation (rather than because they are bandlimited). Examples of FRI signals which are not bandlimited and which are of interest to us include

• Stream of pulses: $x(t)={\sum }_k{a}_{k}p(t-t_k)$. For instance, stream of decaying exponentials:

  $$\begin{array}{c}\hfill x(t)=\sum _k{a}_k{e}^{-(t-t_k)/\mathit{\tau }}{\mathbf{1}}_{t\ge t_k},\end{array}$$ 4

  which are a good fit for calcium transient signals induced by neural activity in two-photon calcium imaging. Figure 1a, b are examples of such signals.
• Piecewise sinusoidal signals (see Fig. 1c):

  $$\begin{array}{c}\hfill x(t)=\sum _k\sum _r{a}_{k,r}{e}^{i({\mathit{\omega }}_{k,r}t+{\mathit{\varphi }}_{k,r})}{\mathbf{1}}_{[t_k,t_{k+1})}(t).\end{array}$$ 5
• Stream of Diracs (see Fig. 1d):

  $$\begin{array}{c}\hfill x(t)=\sum _k{a}_k\mathit{\delta }(t-t_k).\end{array}$$ 6

Sampling scheme

Consider the typical acquisition process as shown in Fig. 3. This is usually modelled as a filtering stage followed by a sampling stage. The filter accounts for the modifications that the analogue signal $x(t)$ experiences before being sampled. It may model an anti-aliasing filter or it might be due to the distortion introduced by the acquisition device, for example, in the case of a digital camera the distortion due to the lens. Filtering signal $x(t)$ with $h(t)=\mathit{\phi }(-t/T)$ and retrieving samples at instants of time $t=nT$ is equivalent to computing the inner product between $x(t)$ and $\mathit{\phi }(t/T-n)$. Specifically, the filtered signal is given by

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill y(t)& =x(t)\ast h(t)\hfill \\ \hfill & ={\int }_{-\infty }^{+\infty }x(\mathit{\tau })h(t-\mathit{\tau })\text{d}\mathit{\tau }\hfill \\ \hfill & ={\int }_{-\infty }^{+\infty }x(\mathit{\tau })\mathit{\phi }\left(-\frac{t-\mathit{\tau }}{T}\right)\text{d}\mathit{\tau }.\hfill \end{array}\end{array}$$ 7

Moreover, sampling $y(t)$ at regular intervals of time $t=nT$ leads to

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill y_n& {=y(t)|}_{t=nT}\hfill \\ \hfill & ={\int }_{-\infty }^{+\infty }x(\mathit{\tau })\mathit{\phi }(\frac{\mathit{\tau }}{T}-n)\text{d}\mathit{\tau }\hfill \\ \hfill & =〈x(t),\mathit{\phi }(\frac{t}{T}-n)〉.\hfill \end{array}\end{array}$$ 8

Fig. 3.

Acquisition process

The function $\mathit{\phi }(t)$ is called the sampling kernel. In order to guarantee perfect reconstruction of the signal $x(t)$, the sampling kernel and the input signal have to satisfy some conditions. The literature presents a variety of kernels that can be used to achieve perfect reconstruction of FRI signals. Here, we will focus on exponential reproducing kernels since they offer the best flexibility and resilience to noise.

• Exponential reproducing property: Any function $\mathit{\phi }(t)$ that together with its shifted versions can reproduce exponential functions of the form $e^{{\mathit{\alpha }}_{m}t}$ with ${\mathit{\alpha }}_m\in \mathbb{C}$ and $m=0,1,\dots ,P$:

  $$\begin{array}{c}\hfill \sum _{n\in \mathbb{Z}}{c}_{m,n}\mathit{\phi }(t-n)=e^{{\mathit{\alpha }}_{m}t},m=0,1,\dots ,P.\end{array}$$ 9

The exponential reproduction property is illustrated in Fig. 4 for two different kernels that reproduce different exponentials. In both cases, the kernels are of compact support. The advantage of such kernels is that the summation in (9) can be truncated and still have a region in time where the exponential functions are perfectly reproduced. In general, the exponentials $e^{{\mathit{\alpha }}_{m}t}$ are perfectly reproduced when the summation is computed for $n\in \mathbb{Z}$. Let $t\in [0,L)$ be the support of $\mathit{\phi }(t)$, that is, $\mathit{\phi }(t)=0$ for $t\notin [0,L)$. If the summation is truncated to $n=n_0,\dots ,n_f$, it follows that the perfect reproduction of the exponential functions holds for $t\in [n_0-1+L,n_f+1)$.

Fig. 4.

Two different kernels that reproduce exactly different exponentials. The first row illustrates the reproduction of two real exponential functions with the kernel ${\mathit{\phi }}_1(t)$, shown in a. The second row illustrates the reproduction of two complex exponentials with the kernel ${\mathit{\phi }}_2(t)$, shown in d. In e and f, only the real part of the exponentials is shown ($e^{{\mathit{\alpha }}_{m}t}=e^{i{\mathit{\omega }}_{m}t}=cos({\mathit{\omega }}_{m}t)+isin({\mathit{\omega }}_{m}t)$). The thin lines represent the shifted and weighted kernels, the thick line represents their sum and the dashed line the true exponential. Note that both kernels are of compact support. The summation in (9) is truncated which leads to the border effects in b, c, e and f

Exponential reproducing kernels

For the sake of clarity, in what follows, we restrict the analysis to the case where the parameter ${\mathit{\alpha }}_m$ in (9) is purely imaginary, that is ${\mathit{\alpha }}_m=i{\mathit{\omega }}_m$ for $m=0,1,\dots ,P$, where ${\mathit{\omega }}_m\in \mathbb{R}$. This analysis can easily be extended to the more general case where ${\mathit{\alpha }}_m$ has nonzero real and imaginary parts, or is purely real.

A function $\mathit{\phi }(t)$ together with a linear combination of its shifted versions reproduces the exponentials ${\{e^{i{\mathit{\omega }}_{m}t}\}}_{m=0}^P$ as in (9) if and only if it satisfies the generalised Strang-Fix conditions:

$$\begin{array}{c}\hfill \widehat{\mathit{\phi }}({\mathit{\omega }}_m)\ne 0\text{and}\widehat{\mathit{\phi }}({\mathit{\omega }}_m+2\mathit{\pi }l)=0,\end{array}$$ 10

where $m=0,1,\dots ,P$, $l\in \mathbb{Z}\backslash \{0\}$ and $\widehat{\mathit{\phi }}(\mathit{\omega })$ is the Fourier transform of $\mathit{\phi }(t)$ (Strang and Fix 1971; Unser and Blu 2005; Urigüen et al. 2013). A family of functions that satisfy these conditions are the exponential B-splines, also named E-splines. These functions are constructed through the convolution of elementary zero order E-splines, where each elementary function reproduces a particular exponential $e^{i{\mathit{\omega }}_{m}t}$. The Fourier transform of a zero order E-spline that reproduces the exponential $e^{\mathit{\alpha }t}$ is given by

$$\begin{array}{c}\hfill {\widehat{\mathit{\beta }}}_{\mathit{\alpha }}(\mathit{\omega })=\frac{1-e^{\mathit{\alpha }-i\mathit{\omega }}}{i\mathit{\omega }-\mathit{\alpha }}.\end{array}$$ 11

Figure 5 illustrates the Fourier transform of zero order E-splines for two different values of the parameter $\mathit{\alpha }$.

Fig. 5.

Absolute value of the Fourier transform of zero order E-splines given by (11). If the parameter $\mathit{\alpha }$ is equal to zero, the E-spline corresponds to the sinc function. For $\mathit{\alpha }=i{\mathit{\omega }}_0$ purely imaginary, the Fourier transform of the E-spline is a shifted version of the sinc function

The corresponding E-spline that reproduces the set of exponentials ${\{e^{{\mathit{\alpha }}_{m}t}\}}_{m=0}^P$ is obtained as follows

$$\begin{array}{c}\hfill {\mathit{\beta }}_{\mathit{\alpha }}(t)=\left({\mathit{\beta }}_{{\mathit{\alpha }}_0}\ast {\mathit{\beta }}_{{\mathit{\alpha }}_1}\ast \cdots \ast {\mathit{\beta }}_{{\mathit{\alpha }}_P}\right)(t),\end{array}$$ 12

where $\mathit{\alpha }=\left({\mathit{\alpha }}_0,{\mathit{\alpha }}_1,\dots ,{\mathit{\alpha }}_P\right)$. Thus, the Fourier transform of ${\mathit{\beta }}_{\mathit{\alpha }}(t)$ is given by

$$\begin{array}{c}\hfill {\widehat{\mathit{\beta }}}_{\mathit{\alpha }}(\mathit{\omega })=\prod _{m=0}^P\left(\frac{1-e^{{\mathit{\alpha }}_m-i\mathit{\omega }}}{i\mathit{\omega }-{\mathit{\alpha }}_m}\right).\end{array}$$ 13

E-splines have compact support $P+1$ and have $P-1$ continuous derivatives. It can be shown that any function that reproduces the set of exponentials ${\{e^{{\mathit{\alpha }}_{m}t}\}}_{m=0}^P$ can be expressed as the convolution of another function $\mathit{\gamma }(t)$ with the corresponding E-spline that reproduces these exponentials, that is, $\mathit{\phi }(t)=\mathit{\gamma }(t)\ast {\mathit{\beta }}_{\mathit{\alpha }}(t)$ and $\mathit{\gamma }(t)$ satisfies ${\int }_{-\infty }^{+\infty }{e}^{-{\mathit{\alpha }}_{m}t}\mathit{\gamma }(t)\text{d}t\ne 0$ for all ${\mathit{\alpha }}_m$(Unser and Blu 2005; Delgado-Gonzalo et al. 2012). It is also true that if $\mathit{\phi }(t)$ reproduces a set of exponentials, this property is preserved through convolution. Let

$$\begin{array}{c}\hfill \mathit{\psi }(t)=\mathit{\phi }(t)\ast \mathit{\rho }(t),\end{array}$$ 14

for $\mathit{\rho }(t)$ such that ${\int }_{-\infty }^{+\infty }{e}^{-{\mathit{\alpha }}_{m}t}\mathit{\rho }(t)\text{d}t\ne 0$. The function $\mathit{\psi }(t)$ also reproduces the same set of exponentials. This is easy to verify since $\mathit{\psi }(t)$ also satisfies the Strang-Fix conditions.

Sampling with an exponential reproducing kernel

The choice of purely imaginary parameters ${\mathit{\alpha }}_m=i{\mathit{\omega }}_m$ leads to an important family of sampling kernels. These design parameters directly determine the information of the input analogue signal $x(t)$ that we acquire and allow us to perfectly reconstruct the input signal from the discrete samples $y_n$ for some classes of signals. Specifically, the different ${\mathit{\omega }}_m$ correspond to the frequencies of the Fourier transform of $x(t)$ that we are able to retrieve from the only knowledge of samples $y_n$. It can be shown that if parameters ${\mathit{\alpha }}_m$ are real or appear in complex conjugate pairs, the corresponding E-spline is real. We thus impose that for all ${\mathit{\alpha }}_m$ that are nonzero, their complex conjugates are also present in $\mathit{\alpha }$. If parameters ${\mathit{\alpha }}_m=i{\mathit{\omega }}_m$ in vector $\mathit{\alpha }$ are sorted in increasing order of ${\mathit{\omega }}_m$, we have that ${\mathit{\alpha }}_m^{\ast }={\mathit{\alpha }}_{P-m}$.

Let us assume that function $x(t)$ is localised in time and thus only $N$ samples $y_n$ are nonzero. Let ${\left(s_m\right)}_{m=0}^P$ be the sequence obtained by linearly combining samples $y_n$ with the coefficients $c_{m,n}$ from (9), that is, $s_m={\sum }_{n=1}^N{c}_{m,n}{y}_n$. We have that

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill s_m& \stackrel{(a)}{=}\sum _{n=1}^N{c}_{m,n}〈x(t),\mathit{\phi }(t/T-n)〉\hfill \\ \hfill & \stackrel{(b)}{=}{\int }_{-\infty }^{+\infty }x(t)\sum _{n=1}^N{c}_{m,n}\mathit{\phi }(t/T-n)\text{d}t\hfill \\ \hfill & \stackrel{(c)}{=}{\int }_{-\infty }^{+\infty }x(t)e^{i{\mathit{\omega }}_{m}t/T}\text{d}t=\widehat{x}(-{\mathit{\omega }}_m/T),\hfill \end{array}\end{array}$$ 15

where $(a)$ follows from (8), $(b)$ from the linearity of the inner product and $(c)$ from the exponential reproduction property. The quantity $s_m$ therefore corresponds to the Fourier transform of $x(t)$ evaluated at $\mathit{\omega }=-{\mathit{\omega }}_m/T$. Since we have imposed $-{\mathit{\omega }}_m={\mathit{\omega }}_{P-m}$, we also have that $s_{P-m}=\widehat{x}({\mathit{\omega }}_m/T)$.

Computation of $c_{m,n}$ coefficients

We have established the properties that a function $\mathit{\phi }(t)$ has to satisfy in order to reproduce exponentials, which are given by the Strang-Fix conditions. Moreover, we have seen the importance of the E-splines since they allow us to obtain samples of the Fourier transform of the input signal. We now show how to obtain the coefficients $c_{m,n}$ in (9) required to reproduce the exponential functions ${\{e^{i{\mathit{\omega }}_{m}t}\}}_{m=0}^P$, and that are used to obtain the sequence $s_m$ in (15). These coefficients are given by

$$\begin{array}{c}\hfill c_{m,n}={\int }_{-\infty }^{\infty }{e}^{i{\mathit{\omega }}_{m}t}\tilde{\mathit{\phi }}(t-n)\text{d}t,\end{array}$$ 16

where $\tilde{\mathit{\phi }}(t)$ is chosen to form with $\mathit{\phi }(t)$ a quasibiorthonormal set (Dragotti et al. 2007). This includes the particular case where $\tilde{\mathit{\phi }}(t)$ is the dual of $\mathit{\phi }(t)$, that is, $〈\tilde{\mathit{\phi }}(t-n),\mathit{\phi }(t-m)〉={\mathit{\delta }}_{n,m}$. The introduction of $\tilde{\mathit{\phi }}(t)$ is a technicality that is needed in order to show where the coefficients $c_{m,n}$ come from, but we do not need to work with this function. From (16), we can express $c_{m,n}$ in terms of $c_{m,0}$ by applying a change of variable $t^{\prime }=t-n$:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill c_{m,n}& =e^{i{\mathit{\omega }}_{m}n}{\int }_{-\infty }^{\infty }{e}^{i{\mathit{\omega }}_{m}t}\tilde{\mathit{\phi }}(t)\text{d}t\hfill \\ \hfill & =e^{i{\mathit{\omega }}_{m}n}{c}_{m,0}.\hfill \end{array}\end{array}$$ 17

If we plug this expression in (9), we can derive an expression to compute $c_{m,0}$ for each $m=0,\dots ,P$:

$$\begin{array}{c}\hfill c_{m,0}={\left(\sum _{n\in \mathbb{Z}}{e}^{-i{\mathit{\omega }}_m(t-n)}\mathit{\phi }(t-n)\right)}^{-1},m=0,1,\dots ,P,\end{array}$$ 18

which is valid for any value of $t$. Let $\mathit{\psi }(t):=e^{-i{\mathit{\omega }}_{m}t}\mathit{\phi }(t)$, we have that

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \sum _{n\in \mathbb{Z}}{e}^{-i{\mathit{\omega }}_m(t-n)}\mathit{\phi }(t-n)& =\sum _{n\in \mathbb{Z}}\mathit{\psi }(t-n)\hfill \\ \hfill & \stackrel{(a)}{=}\sum _{k\in \mathbb{Z}}\widehat{\mathit{\psi }}(2\mathit{\pi }k)e^{i2\mathit{\pi }kt}\hfill \\ \hfill & \stackrel{(b)}{=}\sum _{k\in \mathbb{Z}}\widehat{\mathit{\phi }}({\mathit{\omega }}_m+2\mathit{\pi }k)e^{i2\mathit{\pi }kt},\hfill \end{array}\end{array}$$ 19

where $(a)$ follows from the Possion summation formula1 and $(b)$ from the fact that the Fourier transform of $\mathit{\psi }(t)$ is equal to the Fourier transform of $\mathit{\phi }(t)$ shifted by ${\mathit{\omega }}_m$. Since $\widehat{\mathit{\phi }}(\mathit{\omega })$ satisfies the Strang-Fix conditions, from (18) and (19) it follows that

$$\begin{array}{c}\hfill c_{m,0}={\left[\widehat{\mathit{\phi }}({\mathit{\omega }}_m)\right]}^{-1}.\end{array}$$ 20

The dots in Fig. 6b illustrate the values $\widehat{\mathit{\phi }}({\mathit{\omega }}_m)$ that are used in the computation of the different $c_{m,0}$ for an E-spline with $P=6$. Note that the generalised Strang-Fix conditions (10) impose some constraints on the choice of ${\mathit{\omega }}_m$ since we have to guarantee that $\widehat{\mathit{\phi }}({\mathit{\omega }}_m)\ne 0$. From (11) and Fig. 5, it is clear that each ${\mathit{\omega }}_m$ introduces zeros at locations ${\mathit{\omega }}_m+2\mathit{\pi }l$, where $l\in \mathbb{Z}\backslash \{0\}$, we thus have to guarantee that for all pairs of distinct $m,n$ we have ${\mathit{\omega }}_m-{\mathit{\omega }}_n\ne 2\mathit{\pi }l$. In Fig. 6b, it can be appreciated that $\widehat{\mathit{\phi }}(\mathit{\omega })$ is nonzero for all $\mathit{\omega }={\mathit{\omega }}_m$, and that the locations ${\mathit{\omega }}_m+2\mathit{\pi }$ and ${\mathit{\omega }}_m-2\mathit{\pi }$ are zero since the curve in dB tends to $-\infty$.

Fig. 6.

E-spline of order $P=6$ and $\mathit{\alpha }=\left[-i\mathit{\pi }/2,-i\mathit{\pi }/3,-i\mathit{\pi }/6,0,i\mathit{\pi }/6,i\mathit{\pi }/3,i\mathit{\pi }/2\right]$. Note that the support in time is equal to $P+1$ and quickly decays to zero. This E-spline reproduces the exponentials in Fig. 4 among others. Parameters ${\mathit{\alpha }}_m$ are purely imaginary or equal to zero; purely imaginary ${\mathit{\alpha }}_m$ appear in complex conjugate pairs. a illustrates the shape of the E-spline in time and b in frequency (expressed in dB). The dots in b represent the locations at which the Fourier transform of $\mathit{\phi }(t)$ is sampled in order to compute the $c_{m,0}$ coefficients as in (20). c is a representation of the complex values $e^{{\mathit{\alpha }}_m}$

From (20) and (17), we can compute the $c_{m,n}$ coefficients for our choice of ${\left({\mathit{\alpha }}_m\right)}_{m=0}^P$ and any value of $n\in \mathbb{Z}$. By combining these coefficients with ${\{\mathit{\phi }(t-n)\}}_{n\in \mathbb{Z}}$, the exponentials ${\{e^{{\mathit{\alpha }}_{m}t}\}}_{m=0}^P$ are perfectly reproduced as shown in Fig. 4.

Approximate reproduction of exponentials

The generalised Strang-Fix conditions (10) impose restrictive constraints on the sampling kernel. This becomes a problem when we do not have control or flexibility over the design of the acquisition device. Recent publications (Urigüen et al. 2013; Dragotti et al. 2013) show that these conditions can be relaxed and still have a very accurate exponential reproduction, which is the property we require in order to reconstruct the analogue input signal. The first part of the Strang-Fix conditions, that is $\widehat{\mathit{\phi }}({\mathit{\omega }}_m)\ne 0$, is easy to achieve, but the second part is harder to guarantee when we do not have control over the sampling device.

If the sampling kernel does not satisfy the generalised Strang-Fix conditions, the exponential reproduction property (9) cannot be satisfied exactly. We thus have to find the coefficients $c_{m,n}$ that better approximate the different exponentials $e^{i{\mathit{\omega }}_{m}t}$:

$$\begin{array}{c}\hfill \sum _{n\in \mathbb{Z}}{c}_{m,n}\mathit{\phi }(t-n)\simeq e^{i{\mathit{\omega }}_{m}t}.\end{array}$$ 21

There are various options to compute these coefficients, but a good and stable approximation is obtained with the constant least squares approach (Urigüen et al. 2013). If the Fourier transform of the sampling kernel is sufficiently small at $\mathit{\omega }={\mathit{\omega }}_m+2\mathit{\pi }l$, $l\ne 0$, the $c_{m,n}$ coefficients are given by

$$\begin{array}{c}\hfill c_{m,n}=\widehat{\mathit{\phi }}({\mathit{\omega }}_m)e^{i{\mathit{\omega }}_{m}n}.\end{array}$$ 22

Gaussian filters are good candidates for this approach since they are smooth and the shape in time is very similar to the E-splines (see Fig. 7a). The Fourier transform of such filters is given by

$$\begin{array}{c}\hfill \mathit{\phi }(t)=\frac{1}{\sqrt{2\mathit{\pi }{\mathit{\sigma }}^2}}{e}^{-t^2/2{\mathit{\sigma }}^2}\stackrel{\mathcal{F}}{⟶}\widehat{\mathit{\phi }}(\mathit{\omega })=e^{-{\mathit{\omega }}^2{\mathit{\sigma }}^2/2}.\end{array}$$ 23

It is clear that the filter is nonzero at $\mathit{\omega }={\mathit{\omega }}_m+2\mathit{\pi }l$, $l\ne 0$, however, as can be appreciated from Fig. 7a, the attenuation at these frequencies is very strong. This makes the exponential reproduction very accurate as illustrated in Fig. 7b, c.

Fig. 7.

Gaussian sampling filter with $\mathit{\sigma }=1$ and approximate exponential reproduction. a Fourier transform of Gaussian function compared to a first order E-spline that reproduces frequencies $-0.1\mathit{\pi }$ and $0.1\mathit{\pi }$. The zoom-in box shows the region where the E-spline is zero ($\mathit{\omega }={\mathit{\omega }}_m+2\mathit{\pi }l$); the Gaussian function has a negligible amplitude. b–d are examples of exponential reproduction with the Gaussian filter. As in Fig. 4, the thin lines represent the shifted and weighted versions of $\mathit{\phi }(t)$ (which in this case is the Gaussian function), the thick line their sum, and the dashed line the true exponentials

In the case of the Gaussian filter, we can easily obtain the $c_{m,n}$ coefficients of the exponentials to be reproduced since we have an analytical expression for its Fourier transform. When an analytic expression is unknown, we can still apply this approach since we only need knowledge of the transfer function of the acquisition device at frequencies $\mathit{\omega }={\mathit{\omega }}_m$. The $c_{m,n}$ coefficients are then given by (22).

The approximate Strang-Fix framework is therefore very attractive since it allows us to use the theory discussed so far with any acquisition device.

Perfect reconstruction of FRI signals

In the previous section, we have seen some properties of exponential reproducing kernels. We have also seen that if the sampling kernel satisfies the exponential reproducing property, we can obtain some samples of the Fourier transform of the input analogue signal from the measurements ${\left(y_n\right)}_{n=1}^N$ that result from the sampling process. We now show how this partial knowledge of the Fourier transform can be used to perfectly reconstruct some classes of band unlimited signals.

Perfect reconstruction of a stream of Diracs

We assume that the input signal is a stream of Diracs: $x(t)={\sum }_{k=1}^K{a}_k\mathit{\delta }(t-t_k)$, and that the sampling kernel $\mathit{\phi }(t)$ satisfies the exponential reproduction property for a choice of $\mathit{\alpha }={\left({\mathit{\alpha }}_m\right)}_{m=0}^P$ such that ${\mathit{\alpha }}_m=i{\mathit{\omega }}_m$, where ${\mathit{\omega }}_m\in \mathbb{R}$ for $m=0,1,\dots ,P$. We further impose the frequencies ${\mathit{\omega }}_m$ to be equispaced, that is ${\mathit{\omega }}_{m+1}-{\mathit{\omega }}_m=\mathit{\lambda }$, and to be symmetric, that is ${\mathit{\omega }}_m=-{\mathit{\omega }}_{P-m}$. We thus have ${\mathit{\omega }}_m={\mathit{\omega }}_0+\mathit{\lambda }m$ and ${\mathit{\omega }}_P=-{\mathit{\omega }}_0$.

Since $x(t)$ is a sum of Diracs, we have that the Fourier transform is given by a sum of exponentials:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill \widehat{x}(\mathit{\omega })& ={\int }_{-\infty }^{+\infty }\sum _{k=1}^K{a}_k\mathit{\delta }(t-t_k)e^{-i\mathit{\omega }t}\text{d}t\hfill \\ \hfill & =\sum _{k=1}^K{a}_k{e}^{-i\mathit{\omega }{t}_k}.\hfill \end{array}\end{array}$$ 24

This is clearly a band unlimited signal. We now consider the sequence $s_m$ that is obtained by linearly combining samples $y_n$ with the coefficients $c_{m,n}$ from the exponential reproducing property (9). From (15), we have that $s_m=\widehat{x}(-{\mathit{\omega }}_m/T)$ and therefore:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill s_m& =\sum _{k=1}^K{a}_k{e}^{i{\mathit{\omega }}_m{t}_k/T}\hfill \\ \hfill & =\sum _{k=1}^K\underset{b_k}{\underbrace{a_k{e}^{i{\mathit{\omega }}_0{t}_k/T}}}{\left(\underset{u_k}{\underbrace{e^{i\mathit{\lambda }{t}_k/T}}}\right)}^m\hfill \\ \hfill & =\sum _{k=1}^K{b}_k{u}_k^m,\hfill \end{array}\end{array}$$ 25

where $b_k:=a_k{e}^{i{\mathit{\omega }}_0{t}_k/T}$ and $u_k:=e^{i\mathit{\lambda }{t}_k/T}$. Note that we have also applied the fact that the frequencies can be expressed as ${\mathit{\omega }}_m={\mathit{\omega }}_0+\mathit{\lambda }m$. The perfect recovery of the original stream of Diracs, that is, the estimation of the locations $t_k$ and the amplitudes $a_k$ of the $K$ Diracs, is now recast as the estimation of parameters $b_k$ and $u_k$ from the knowledge of values $s_m$. The problem of estimating the parameters of a sum of exponentials from a set of samples arises in a variety of fields and has been analysed for several years by the spectral estimation community (Pisarenko 1973; Paulraj et al. 1985; Schmidt 1986). One way to solve it is by realising that the sequence $s_m$ given as in (25) is the solution to the following linear homogeneous recurrence relation

$$\begin{array}{c}\hfill h_K{s}_{m-K}+\cdots +h_1{s}_{m-1}+s_m=0.\end{array}$$ 26

See section “Linear homogeneous recurrence relations with constant coefficients” of Appendix for a description of this type of homogeneous systems and their solutions. Note that coefficients $h_1,\dots ,h_K$ are unknown, but can be obtained from the following linear system of $K$ equations:

$$\begin{array}{c}\hfill \left[\begin{array}{cccc}{s}_{K-1}& s_{K-2}& \dots & s_0\\ s_K& s_{K-1}& \dots & s_1\\ ⋮& ⋮& \ddots & ⋮\\ s_{2K-2}& s_{2K-3}& \dots & s_{K-1}\end{array}\right]·\left[\begin{array}{c}{h}_1\\ h_2\\ ⋮\\ h_K\end{array}\right]=-\left[\begin{array}{c}{s}_K\\ s_{K+1}\\ ⋮\\ s_{2K-1}\end{array}\right].\end{array}$$ 27

It can be shown that, if the $K$ parameters $u_k$ in (25) are distinct, which is a direct consequence of the fact that all the delays $t_k$ are different, the Toeplitz matrix in the left-hand side of (27) is of rank $K$, and therefore, the solution is unique (see section “Rank deficiency of Toeplitz matrix” of Appendix for a proof on the rank of this matrix). As shown in section “Linear homogeneous recurrence relations with constant coefficients” of Appendix, the parameters $u_k$ are obtained from the roots of the polynomial $H(z)=h_K{z}^{-K}+\cdots +h_1{z}^{-1}+1$. Once the parameters $u_k$ have been obtained, the amplitudes $b_k$ of the sum of exponentials can be directly retrieved from (25) by solving the associated least squares problem. From $u_k$ and $b_k$, we can then compute $t_k$ and $a_k$. The stream of Diracs is thus perfectly recovered. In the literature, this approach is known as Prony’s method or the annihilating filter method (Stoica and Moses 2005).

The system of equations (27) requires at least $2K$ consecutive values $s_m$. Recall that the sequence $s_m$ is obtained as follows $s_m={\sum }_{n=1}^N{c}_{m,n}{y}_n$, with $m=0,1,\dots ,P$, where $P+1$ is the number of exponentials reproduced by the sampling kernel. We thus have a lower bound on the number of exponentials that the sampling kernel has to reproduce: $P+1\ge 2K$. The perfect reconstruction of a stream of Diracs is summarised in the following theorem.

Theorem 1

Consider a stream $x(t)$ of K Diracs: $x(t)={\sum }_{k=1}^K{a}_k\mathit{\delta }(t-t_k)$, and a sampling kernel $\mathit{\phi }(t)$ that can reproduce exponentials $e^{i({\mathit{\omega }}_0+\mathit{\lambda }m)t}$, with $m=0,1,\dots ,P$, and $P+1\ge 2K$. Then, the samples defined by $y_n=〈x(t),\mathit{\phi }(t/T-n)〉$ are sufficient to characterise $x(t)$ uniquely.

Figure 8 illustrates the entire sampling process. Note that, since the sampling kernel is of compact support and the stream of Diracs is localised in time, there are only a small number of samples $y_n$ that are nonzero. From Fig. 8e, it is clear that the signal is not bandlimited. Furthermore, in the classical sampling setup, in order to sample a continuous-time signal at rate $T^{-1}$ Hz, an anti-aliasing filter that sets to zero $\widehat{x}(\mathit{\omega })$ for $|\mathit{\omega }|\ge \mathit{\pi }/T$ has to be applied before acquisition. The FRI framework does not impose this stringent condition since the sampling kernel is not necessarily equal to zero for all $|\mathit{\omega }|\ge \mathit{\pi }/T$.

Fig. 8.

Sampling of a stream of Diracs and perfect reconstruction. $K=4$ Diracs sampled with an E-spline of order $P=7$ which corresponds to the critical sampling rate ($P+1=2K$). a is the continuous-time stream of Diracs, b the sampling kernel $h(t)=\mathit{\phi }(-t/T)$ where $\mathit{\phi }(t)$ is an E-spline of order $P=7$ that reproduces the exponentials illustrated in c. d is the continuous-time signal $y(t)=x(t)\ast h(t)$ and the corresponding discrete samples $y_n{=y(t)|}_{t=nT}$. In e, $|\widehat{x}(\mathit{\omega })|$ is obtained from (24) and $|s_m|$ from samples $y_n$ linearly combined with coefficients $c_{m,n}$. f is the reconstructed stream of Diracs from the sequence $s_m$. The original signal is perfectly reconstructed

Perfect reconstruction of a stream of decaying exponentials

Streams of Diracs are an idealisation of streams of pulses. Although this example may seem limited, the framework presented so far can be applied to other classes of functions that model a variety of signals. For instance, calcium concentration measurements obtained from two-photon imaging to track the activity of individual neurons can be modelled with a stream of decaying exponentials. In this model, the time delays correspond to the activation time of the tracked neuron, that is, the action potentials (AP).

Let $x(t)$ be a stream of $K$ decaying exponentials, that is

$$\begin{array}{c}\hfill x(t)=\sum _{k=1}^K{a}_k{e}^{-\mathit{\alpha }(t-t_k)}{\mathbf{1}}_{t\ge t_k}=\sum _{k=1}^K{a}_k{\mathit{\rho }}_{\mathit{\alpha }}(t-t_k),\end{array}$$ 28

where ${\mathit{\rho }}_{\mathit{\alpha }}(t):=e^{-\mathit{\alpha }t}{\mathbf{1}}_{t\ge 0}$. See Fig. 9a for an example of such signal. This is also an FRI signal since $x(t)$ is perfectly determined by a finite number of parameters: ${\{(t_k,a_k)\}}_{k=1}^K$. Let us assume that $x(t)$ is sampled with the acquisition device described in Sect. 3.2.1, that is, an exponential reproducing kernel $h(t)=\mathit{\phi }(-t/T)$, followed by a sampling stage. We thus have that $\mathit{\phi }(t)$ satisfies (9), and the resulting samples $y_n$ can be expressed as the inner product between $x(t)$ and $\mathit{\phi }(t/T-n)$ as in (8).

Fig. 9.

Sampling of a stream of decaying exponentials and perfect reconstruction. Since $x(t)$ is an infinite duration signal, samples $y_n$ are nonzero for $n\ge n_0$, for some $n_0$ that depends on the location of the first decaying exponential. However, if the number of decaying exponentials is finite, the number of nonzero samples $z_n=y_n-y_{n-1}{e}^{-\mathit{\alpha }T}$ is also finite since they are equivalent to sampling a stream of Diracs with a compact support kernel. a Input signal, $x(t)$, b filtered and sampled signal, c reconstructed signal

Let us also assume that the reproduced exponentials $e^{i{\mathit{\omega }}_{m}t}$ can be expressed as $e^{i({\mathit{\omega }}_0+\mathit{\lambda }m)t}$, with $m=0,1,\dots ,P$. It can be shown that sampling the signal in (28) with $\mathit{\phi }(-t/T)$ and computing the following finite differences

$$\begin{array}{c}\hfill z_n=y_n-y_{n-1}{e}^{-\mathit{\alpha }T},\end{array}$$ 29

is equivalent to the sequence that would result from sampling the stream of Diracs $s(t)={\sum }_{k=1}^K{a}_k\mathit{\delta }(t-t_k)$ with the following kernel

$$\begin{array}{c}\hfill \mathit{\psi }(t)={\mathit{\beta }}_{\mathit{\alpha }T}(-t)\ast \mathit{\phi }(t)\end{array}$$ 30

where ${\mathit{\beta }}_{\mathit{\alpha }T}(-t)$ is a zero order E-spline with parameter $\mathit{\alpha }T$ (Oñativia et al. 2013a). Note that $\mathit{\alpha }$ is the exponent in (28). We thus have that

$$\begin{array}{c}\hfill z_n=〈s(t),\mathit{\psi }(t/T-n)〉.\end{array}$$ 31

Since convolution preserves the exponential reproduction property, $\mathit{\psi }(t)$ reproduces the same exponentials as $\mathit{\phi }(t)$. Thus, we can find the coefficients $d_{m,n}$ such that

$$\begin{array}{c}\hfill \sum _{n\in \mathbb{Z}}{d}_{m,n}\mathit{\psi }(t-n)=e^{i{\mathit{\omega }}_{m}t},m=0,1,\dots ,P.\end{array}$$ 32

We now have all the elements to perfectly reconstruct the stream of decaying exponentials $x(t)$ from samples $y_n$, that is, estimate the set of pairs of parameters ${\{(t_k,a_k)\}}_{k=1}^K$. By combining the sequence $z_n$ with coefficients $d_{m,n}$, we obtain exactly the same measurements $s_m$ as in (25):

$$\begin{array}{c}\hfill s_m=\sum _{n=1}^N{d}_{m,n}{z}_n=\sum _{k=1}^K{b}_k{u}_k^m,\end{array}$$ 33

where $b_k=a_k{e}^{i{\mathit{\omega }}_0{t}_k/T}$ and $u_k=e^{i\mathit{\lambda }{t}_k/T}$. We can therefore apply Prony’s method to this sequence and obtain the parameters of interest. Figure 9 illustrates the perfect reconstruction of a stream of $K=4$ decaying exponentials.

FRI signals with noise

The acquisition process inevitably introduces noise making the solutions described so far only ideal. Perturbations may arise in the analogue and digital domain. We model the noise of the acquisition process as a white Gaussian process that is added to the ideal samples. The noisy samples are therefore given by

$$\begin{array}{c}\hfill {\tilde{y}}_n=y_n+{\mathit{\epsilon }}_n,\end{array}$$ 34

where $y_n$ are the ideal noiseless samples from (8) and ${\mathit{\epsilon }}_n$ are i.i.d. Gaussian random variables with zero mean and variance ${\mathit{\sigma }}_{\mathit{\epsilon }}^2$. In order to have a more robust reconstruction, we increase the number of samples $s_m$ by making the order $P$ larger than the critical rate $2K-1$.

The denoising strategies that can be applied to improve the performance of the reconstruction process come from the spectral analysis community, where the problem of finding sinusoids in noise has been extensively studied. There are two main approaches. The first, named Cadzow denoising algorithm, is an iterative procedure applied to the Toeplitz matrix constructed from samples $s_m$ as in (27). Let us denote by $\mathit{S}$ this matrix. By construction, this matrix is Toeplitz, and in the noiseless case, it is of rank $K$. The presence of noise makes this matrix be full rank. The Cadzow algorithm (Cadzow 1988) looks for the closest rank deficient matrix which is Toeplitz. At each step, we force matrix $\mathit{S}$ to be of rank $K$ by computing the singular value decomposition (SVD) and only keeping the $K$ largest singular values and setting the rest to zero. This new matrix is not Toeplitz anymore, we thus compute a new Toeplitz matrix by averaging the diagonal elements. This last matrix might not be rank deficient, and we can thus iterate again. The next step is to solve equation (27). This is done computing the total least squares solution that minimises ${||\mathit{S}\mathit{h}||}^2$ subject to ${||\mathit{h}||}^2=1$, where $\mathit{h}$ is an extended version of the vector in (27) and has length $K+1$. If this vector is normalised with respect to the first element, we have that the following $K$ elements correspond to the coefficients $h_k$ in (26). This approach has successfully been applied in the FRI setup in (Blu et al. 2008).

The second approach is based on subspace techniques for estimating generalised eigenvalues of matrix pencils (Hua and Sarkar 1990, 1991). Such approach has also been applied in the FRI framework (Maravić and Vetterli 2005). This method is based on the particular structure of the matrix $\mathit{S}$, which is Toeplitz and each element is given by a sum of exponentials. Let ${\mathit{S}}_0$ be the matrix constructed from $\mathit{S}$ by dropping the first row and ${\mathit{S}}_1$ the matrix constructed from $\mathit{S}$ by dropping the last row. It can be shown that in the matrix pencil ${\mathit{S}}_0-\mathit{\mu }{\mathit{S}}_1$ the parameters ${\left\{u_k\right\}}_{k=1}^K$ from (25) are rank reducing numbers, that is, the matrix ${\mathit{S}}_0-\mathit{\mu }{\mathit{S}}_1$ has rank $K-1$ for $\mathit{\mu }=u_k$ and rank $K$ otherwise. The parameters ${\left\{u_k\right\}}_{k=1}^K$ are thus given by the eigenvalues of the generalised eigenvalue problem $({\mathit{S}}_0-\mathit{\mu }{\mathit{S}}_1)\mathit{v}=0$.

Further variations of these two fundamental approaches have been proposed recently. See for example Tan and Goyal (2008), Erdozain and Crespo (2011), Hirabayashi et al. (2013).

Sampling streaming FRI signals

In the previous section, we have seen how to sample and reconstruct a set of $K$ Diracs. We now consider the case where we have a streaming signal:

$$\begin{array}{c}\hfill x(t)=\sum _{k\in \mathbb{Z}}{a}_k\mathit{\delta }(t-t_k).\end{array}$$ 35

If the stream is made of clearly separable bursts, we can apply the previously described strategy by assuming that each burst has a maximum number of spikes. However, when this separation cannot be made because of the presence of noise, or due to the nature of the signal itself, this strategy is not valid. The infinite stream presents an obvious constraint due the number of parameters that have to be recovered. We have seen that the order of the sampling kernel, $P$, and its support are directly related to the number of parameters to be estimated. However, we cannot increase $P$ indefinitely. In order to handle this type of signals, we thus consider a sequential and local approach (Oñativia et al. 2013b).

Sliding window approach

We assume that $x(t)$ has a bounded local rate of innovation of $2K/\mathit{\tau }$, that is, for any time window of duration $\mathit{\tau }$ there are at most $K$ Diracs within the window. Since each Dirac has two degrees of freedom, location and amplitude, the rate of innovation is $2K/\mathit{\tau }$. We analyse sequentially the infinite stream with a sliding window that progresses in time by steps equal to the sampling interval $T$. Let the $i$-th window cover the following temporal interval

$$\begin{array}{c}\hfill t\in (n_{i}T,n_{i}T+\mathit{\tau }],\end{array}$$ 36

where $\mathit{\tau }=NT$ and $N$ is the number of samples that are processed for each position of the sliding window. The acquisition device is the same as in the previous section: the sampling kernel is given by $h(t)=\mathit{\phi }(-t/T)$ and $y_n=〈x(t),\mathit{\phi }(t/T-n)〉$. In order to have a causal filter $h(t)$, that is $h(t)=0$ for $t<0$, we impose the support of $\mathit{\phi }(t)$ to be $t\in (-L,0]$, where $L=P+1$ if $\mathit{\phi }(t)$ is an E-spline of order $P$. The support of $\mathit{\phi }(t/T-n)$ is therefore $t\in ((n-L)T,nT]$. Consequently, a Dirac located at $t=t_k$ influences $L$ samples $y_n$. The indices corresponding to these samples are given by

$$\begin{array}{c}\hfill \lceil t_k/T\rceil \le n<\lceil t_k/T\rceil +L.\end{array}$$ 37

When we process the stream sequentially, there are border effects due to the fact that we only process $N$ samples at a time. Diracs located just before the sliding window influence samples within the window, and the Diracs inside the observation window which are close to the right border influence samples outside the window. These effects are illustrated in Fig. 10. However, if the sliding window is big enough, there are a good number of positions of the sliding window that will fully capture each individual Dirac and therefore lead to a good estimate of its amplitude and location. In the noiseless case, we can detect if we are in the presence of these border effects or if there is no border effect and therefore the reconstruction can be exact. Nonetheless, in the presence of noise, we cannot guarantee perfect reconstruction.

Fig. 10.

Border effects with the sliding window approach. In this example, $N=16$ and $T=1/4$. a A nearby Dirac located before the observation window $\mathit{\tau }$ influences samples $y_n$ of the window. b A Dirac inside the window but close to the right border generates nonzero samples outside the window

For this reason, the sequential algorithm works in two steps: first, it estimates the locations for each position of the sliding window; second, it analyses the consistency of the retrieved locations among different windows. The $i$-th window processes samples ${\left({\tilde{y}}_n\right)}_{n=n_i+1}^{n_i+N}$. Let $\{{\widehat{t}}_k^{(i)}\}$ be the set of estimated locations within the $i$-th window. When the observation window is at position $t=n_{i}T$, we know that Diracs located at $t<(n_i-L)/T$ cannot have any influence on the current samples. We can therefore analyse the consistency of the locations up to $(n_i-L)/T$. Figure 11a shows the retrieved locations for different positions of the sliding window, where the horizontal axis corresponds to the window index, $n_i$, and the vertical axis to the locations in time, that is, for a given window index, each dot corresponds to an estimate of the set $\{{\widehat{t}}_k^{(i)}\}$. Consistent locations among different windows appear as horizontally aligned dots. The shaded area represents the evolution in time of the observation window: for a given index $n_i$, the vertical cross section of the shaded area represents the time interval $\mathit{\tau }$ that is seen by this window. This consistency can be analysed by building a histogram of all the estimated locations up to a given time. This is illustrated in Fig. 11b. The Diracs are then estimated from the peaks of this histogram.

Fig. 11.

Noisy scenario with SNR = 15 dB, $N=50$ and $T=1/16$. The maximum rate of innovation of the streaming signal is $2K/\mathit{\tau }=3.2(K=5)$. a Plot of the sequentially estimated locations, the horizontal axis indicates the index of the sliding window and the vertical axis the location in time. b Histogram of the locations shown in a. Horizontally aligned dots in a lead to peaks in the histogram in b

Application to neuroscience

To understand how neurons process information, neuroscientists need accurate information about the firing of action potentials (APs of spikes) by individual neurons. We thus need techniques that allow to monitor large areas of the brain with a spatial resolution that distinguishes single neurons and with a temporal resolution that resolves APs. Of the currently available techniques, only multiphoton calcium imaging (Denk et al. 1990, 1994; Svoboda et al. 1999; Stosiek et al. 2003) and multielectrode array electrophysiology (Csicsvari et al. 2003; Blanche et al. 2005; Du et al. 2009) offer this capability. Of these, only multiphoton calcium imaging currently allows precise three-dimensional localisation of each individual monitored neuron within the region of tissue studied, in the intact brain. Populations of neurons are simultaneously labelled with a fluorescent indicator, acetoxy-methyl (AM) ester calcium dyes (Stosiek et al. 2003). This allows simultaneous monitoring of action potential-induced calcium signals in a plane (Ohki et al. 2005) or volume (Göbel and Helmchen 2007) of tissue. The calcium concentration is measured with a laser-scanning two-photon imaging system.

For a given region of interest (ROI) where a neuron is located, the calcium concentration is obtained by averaging the value of the pixels of the ROI for each frame. The result is a one-dimensional fluorescence sequence. We assume that when a neuron is activated, the calcium concentration jumps instantaneously, and each jump has the same amplitude $A$. The concentration then decays exponentially, with time constant $\mathit{\tau }$, to a baseline concentration. The one-dimensional fluorescence signal can therefore be characterised by convolving the spike train with a decaying exponential and adding noise:

$$\begin{array}{c}\hfill \begin{array}{cc}\hfill c(t)& =A\sum _k{e}^{-(t-t_k)/\mathit{\tau }}{\mathbf{1}}_{t\ge t_k}+{\mathit{\epsilon }}_t\hfill \\ \hfill & =A\sum _k\mathit{\delta }(t-t_k)\ast e^{-t/\mathit{\tau }}{\mathbf{1}}_{t\ge 0}+{\mathit{\epsilon }}_t,\hfill \end{array}\end{array}$$ 38

where the index $k$ represents different spikes and the different $t_k$ their occurrence times. Hence, the goal of spike detection algorithms is to obtain the values $t_k$.

A number of methods have previously been used to detect spike trains from calcium imaging data, including thresholding the first derivative of the calcium signal (Smetters et al. 1999), and the application of template-matching algorithms based on either fixed exponential (Kerr et al. 2005, 2007; Greenberg et al. 2008) or data-derived (Schultz et al. 2009; Ozden et al. 2008) templates. Machine learning techniques (Sasaki et al. 2008) and probabilistic methods based on sequential Monte Carlo framework (Vogelstein et al. 2009) or fast deconvolution (Vogelstein et al. 2010) have also been proposed. Some broadly used methods such as template matching or derivative-thresholding have the disadvantage that they do not deal well with multiple events occurring within a time period comparable to the sampling interval. Our spike detection algorithm is based on connecting the calcium transient estimation problem to the theory of FRI signals. The calcium concentration model in (38) is clearly a FRI signal, we can thus apply the techniques presented in the previous sections.

Spike inference algorithm

The spike inference algorithm is based on applying the sliding window approach presented in Sect. 4.1 combined with the reconstruction of streams of decaying exponentials presented in Sect. 3.2.2. One major issue of the framework presented so far is that we have assumed the number $K$ of spikes within a time window to be known a priori. In practice, this is a value that has to be estimated.

In the noiseless case, the number of spikes can be recovered from the rank of the Toeplitz matrix constructed from samples $s_m$:

$$\begin{array}{c}\hfill \mathit{S}=\left[\begin{array}{ccc}{s}_{\lceil P/2\rceil }& \dots & s_0\\ ⋮& \ddots & ⋮\\ s_P& \dots & s_{P-\lceil P/2\rceil }\end{array}\right].\end{array}$$ 39

In the noisy case, matrix $\mathit{S}$ becomes full rank. An estimate of $K$ can still be obtained by thresholding the normalised singular values of $\mathit{S}$. Let ${\mathit{\mu }}_1\ge {\mathit{\mu }}_2\ge \dots {\mathit{\mu }}_{\lfloor P/2\rfloor +1}$ be the singular values of $\mathit{S}$ sorted in decreasing order. We can estimate $K$ as the number of singular values that satisfy ${\mathit{\mu }}_i/{\mathit{\mu }}_1\ge {\mathit{\mu }}_0$. Where $0<{\mathit{\mu }}_0<1$ is adjusted depending on the level of noise. This approach tends to overestimate $K$. Moreover, we never detect the $K=0$ case since when noise is present we always have ${\mathit{\mu }}_1\ne 1$.

To overcome these inaccuracies, we make the algorithm more robust by applying a double consistency approach. We run the sliding window approach presented in Sect. 4.1 twice. First, with a sufficiently big window where we estimate $K$ from the singular values of $\mathit{S}$. Second, with a smaller window where we assume that we only capture one spike and therefore we always set $K=1$. We then build a joint histogram out of all the locations retrieved from both approaches and estimate the spikes from the peaks of the histogram. This approach is illustrated in Figs. 12 and 13 with real data.

Fig. 12.

Double consistency spike search with real data. a and b show the detected locations with dots and the original spikes with horizontal lines for two different window sizes. In a, the algorithm runs estimating the number of spikes within the sliding window. In b, the algorithm runs assuming a fixed number of spikes equal to one for each position of the sliding window. c shows the joint histogram of the detected locations

Fig. 13.

Fluorescence signal and detected spikes using the double consistency approach. The spikes are detected from the peaks of the histogram in Fig. 12c

This technique is fast and robust in high noise and low temporal resolution scenarios. It is able to achieve a detection rate of 84 % of electrically confirmed AP with real data (Oñativia et al. 2013a), outperforming other state of the art real-time approaches. Due to its low complexity, tens of streams can be processed in parallel with a commercial off-the-shelf computer.

Conclusions

We have presented a framework to sample and reconstruct signals with finite rate of innovation. We have shown that it is possible to sample and perfectly reconstruct streams of Diracs, and more importantly, streams of decaying exponentials. The latter offer a perfect fit for calcium transients induced by the spiking activity of neurons. The presented approach is sequential, and the reconstruction is local. These two features make the overall algorithm resilient to noise and have low complexity offering real-time capabilities.

The theoretical framework, where perfect reconstruction can be achieved, is also extended to the more realistic case where we do not have full control over the sampling kernel. In this case, perfect reconstruction cannot be guaranteed, but we can still reconstruct the underlying analogue signal with high precision if the sampling kernel can reproduce exponentials approximately.

Appendix

Linear homogeneous recurrence relations with constant coefficients

Let $L_K\left[·\right]$ be the linear operator with constant coefficients that establishes the following recurrence relation of order up to $K$ when applied to a sequence $y_n$:

$$\begin{array}{c}\hfill L_K\left[y_n\right]=h_K{y}_{n-K}+\cdots +h_1{y}_{n-1}+y_n.\end{array}$$ 40

The corresponding homogeneous system is given by

$$\begin{array}{c}\hfill L_K\left[y_n\right]=0.\end{array}$$ 41

This is the discrete-time version of a homogeneous linear differential equation given by

$$\begin{array}{c}\hfill h_K\frac{d^{K}y(t)}{t^K}+\cdots +h_1\frac{\text{d}y(t)}{\text{d}t}+y(t)=0.\end{array}$$ 42

Both, the linear homogeneous differential equation (42) and the linear homogeneous recurrence relation (41) have equivalent solutions. In the continuous-time case, the functions that satisfy the homogeneous equation have the form of exponential functions. Similarly, the solution to the discrete-time version has the form of exponential sequences. The solution to (41) is not unique, but all the solutions have the form $z^n$, where $z\in \mathbb{C}$. Thus, to solve (41) we set $y_n=z^n$, leading to

$$\begin{array}{c}\hfill h_K{z}^{-K}{z}^n+\cdots +h_1{z}^{-1}{z}^n+z^n=0.\end{array}$$ 43

Division by $z^n$ gives the $K$th order polynomial

$$\begin{array}{c}\hfill H(z)=h_K{z}^{-K}+\cdots +h_1{z}^{-1}+1.\end{array}$$ 44

$H(z)$ is the characteristic polynomial of the homogeneous system. The roots of $H(z)$, that is, the values $z_1,z_2,\dots ,z_K$ that satisfy $H(z_k)=0$, determine the solution to (41). We have that $L_K\left[z_k^n\right]=0$. If the $K$ roots are distinct, the solution to the homogeneous recurrence relation is given by any linear combination of the sequences constructed from the different roots:

$$\begin{array}{c}\hfill y_n=\sum _{k=1}^K{a}_k{z}_k^n⟺L_K\left[y_n\right]=0,\end{array}$$ 45

since $L_K\left[z_k^n\right]=0$ and $L_K\left[{\sum }_{k=1}^K{a}_k{z}_k^n\right]={\sum }_{k=1}^K{a}_k{L}_K\left[z_k^n\right]$.

Rank deficiency of Toeplitz matrix

Let $\mathit{S}$ be the following $(P-M+1)\times (M+1)$ Toeplitz matrix:

$$\begin{array}{c}\hfill \mathit{S}=\left[\begin{array}{cccc}{s}_M& s_{M-1}& \dots & s_0\\ s_{M+1}& s_M& \dots & s_1\\ ⋮& ⋮& \ddots & ⋮\\ s_P& s_{P-1}& \dots & s_{P-M}\end{array}\right],\end{array}$$ 46

where $(P-M+1)\ge K$, $(M+1)\ge K$ and each element of the matrix is given by $s_m={\sum }_{k=1}^K{b}_k{u}_k^m$, with all $b_k$ nonzero and all $u_k$ distinct. The matrix $\mathit{S}$ can be decomposed as follows:

$$\begin{array}{c}\hfill \mathit{S}=\underset{\mathit{B}}{\underbrace{\left[\begin{array}{ccc}1& \dots & 1\\ u_1& \dots & u_K\\ ⋮& \ddots & ⋮\\ u_1^{P-M}& \dots & u_K^{P-M}\\ \end{array}\right]}}\underset{\mathit{A}}{\underbrace{\left[\begin{array}{ccc}{b}_1& \dots & 0\\ ⋮& \ddots & ⋮\\ 0& \dots & b_K\end{array}\right]}}\underset{\mathit{C}}{\underbrace{\left[\begin{array}{cccc}{u}_1^M& u_1^{M-1}& \dots & 1\\ ⋮& ⋮& \ddots & ⋮\\ u_K^M& u_K^{M-1}& \dots & 1\\ \end{array}\right]}}.\end{array}$$ 47

Since $\mathit{B}$ and $\mathit{C}$ are Vandermonde matrices with distinct elements, both are of rank $K$. Therefore, if elements $b_1,b_2,\dots ,b_K$ are nonzero, matrix $\mathit{S}$ has rank $K$.