Fast and Stable Signal Deconvolution via Compressible State-Space Models

Abstract

Objective

Common biological measurements are in the form of noisy convolutions of signals of interest with possibly unknown and transient blurring kernels. Examples include EEG and calcium imaging data. Thus, signal deconvolution of these measurements is crucial in understanding the underlying biological processes. The objective of this paper is to develop fast and stable solutions for signal deconvolution from noisy, blurred and undersampled data, where the signals are in the form of discrete events distributed in time and space.

Methods

We introduce compressible state-space models as a framework to model and estimate such discrete events. These state-space models admit abrupt changes in the states and have a convergent transition matrix, and are coupled with compressive linear measurements. We consider a dynamic compressive sensing optimization problem and develop a fast solution, using two nested Expectation Maximization algorithms, to jointly estimate the states as well as their transition matrices. Under suitable sparsity assumptions on the dynamics, we prove optimal stability guarantees for the recovery of the states and present a method for the identification of the underlying discrete events with precise confidence bounds.

Results

We present simulation studies as well as application to calcium deconvolution and sleep spindle detection, which verify our theoretical results and show significant improvement over existing techniques.

Conclusion

Our results show that by explicitly modeling the dynamics of the underlying signals, it is possible to construct signal deconvolution solutions that are scalable, statistically robust, and achieve high temporal resolution.

Significance

Our proposed methodology provides a framework for modeling and deconvolution of noisy, blurred, and undersampled measurements in a fast and stable fashion, with potential application to a wide range of biological data.

I. Introduction

In many signal processing applications such as estimation of brain activity from magnetoencephalography (MEG) time-series [2], estimation of time-varying networks [3], electroencephalogram (EEG) analysis [4], calcium imaging [5], functional magnetic resonance imaging (fMRI) [6], and video compression [7], the signals often exhibit abrupt changes that are blurred through convolution with unknown kernels due to intrinsic measurement constraints. Extracting the underlying signals from blurred and noisy measurements is often referred to as signal deconvolution. Traditionally, state-space models have been used for such signal deconvolution problems, where the states correspond to the unobservable signals. Gaussian state-space models in particular are widely used to model smooth state transitions. Under normality assumptions, posterior mean filters and smoothers are optimal estimators, where the analytical solution is given respectively by the Kalman filter and the fixed interval smoother [8], [9].

When applied to observations from abruptly changing states, Gaussian state-space models exhibit poor performance in recovering sharp transitions of the states due to their underlying smoothing property. Although filtering and smoothing recursions can be obtained in principle for non-Gaussian state-space models, exact calculations are no longer possible. Apart from crude approximations like the extended Kalman filter, several methods have been proposed including numerical methods for low-dimensional states [10], Monte Carlo filters [10], [11], posterior mode estimation [12], [13], and fully Bayesian smoothing using Markov chain Monte Carlo simulation [14], [15]. In order to exploit sparsity, several dynamic compressed sensing (CS) techniques, such as the Kalman filtered CS algorithm, have been proposed that typically assume partial information about the sparse support or estimate it in a greedy and online fashion [16], [17], [18], [19], [20]. However, little is known about the theoretical performance guarantees of these algorithms.

In this paper, we consider the problem of estimating state dynamics from noisy and undersampled observations, where the state transitions are governed by autoregressive models with compressible innovations. Motivated by the theory of CS, we employ an objective function formed by the ℓ₁-norm of the state innovations [21]. Unlike the traditional compressed sensing setting, the sparsity is associated with the dynamics and not the states themselves. In the absence of observation noise, the CS recovery guarantees are shown to extend to this problem [21]. However, in a realistic setting in the presence of observation noise, it is unclear how the CS recovery guarantees generalize to this estimation problem.

We will present stability guarantees for this estimator under a convergent state transition matrix, which confirm that the CS recovery guarantees can be extended to this problem. The corresponding optimization problem in its Lagrangian form is akin to the MAP estimator of the states in a linear state-space model where the innovations are Laplace distributed. This allows us to integrate methods from Expectation-Maximization (EM) theory and Gaussian state-space estimation to derive efficient algorithms for the estimation of states as well as the state transition matrix, which is usually unknown in practice. To this end, we construct two nested EM algorithms in order to jointly estimate the states and the transition matrix. The outer EM algorithm for state estimation is akin to the fixed interval smoother, and the inner EM algorithm uses the state estimates to update the state transition matrix [22]. The resulting EM algorithm is recursive in time, which makes the computational complexity of our method scale linearly with temporal dimension of the problem. This provides an advantage over existing methods based on convex optimization, which typically scale super-linearly with the temporal dimension.

Our results are related to parallel applications in spectral estimation, source localization, and channel equalization [23], where the measurements are of the form Y = AX + N, with Y is the observation matrix, X denotes the unknown parameters, A is the measurement matrix, and N is the additive noise. These problems are referred to as Multiple Measurement Vectors (MMV) [24] and Multivariate Regression [25]. In these applications, solutions with row sparsity in X are desired. Recovery of sparse signals with Gaussian innovations is studied in [26]. Several recovery algorithms including the ℓ₁ − ℓ_q minimization methods, subspace methods and greedy pursuit algorithms [27] have been proposed for support union recovery in this setup. Our contributions are distinct in that we directly model the state innovations as a compressible sequence, for recovery of which we present both sharp theoretical guarantees as well as fast algorithms from state-space estimation.

Finally, we provide simulation results as well as applications to two experimentally-acquired data sets: calcium imaging recordings of neuronal activity, and EEG data during sleep. In the former, the deconvolution problem concerns estimating the location of spikes given the temporally blurred calcium fluorescence, and in the latter, the objective is to detect the occurrence and onset of sleep spindles. Our simulation studies confirm our theoretical predictions on the performance gain obtained by compressible state-space estimation over those obtained by traditional estimators such as the basis pursuit denoising. Our real data analyses reveal that our compressible state-space modeling and estimation framework outperforms two of the commonly-used methods for calcium deconvolution and sleep spindle detection. In the spirit of easing reproducibility, we have made MATLAB implementations of our codes publicly available [28].

II. Methods

In this section, we establish our problem formulation and notational conventions and present our main theoretical analysis and algorithm development. In the interest of space, the description of the experimental procedures is given in Section I of the Supplementary Material.

A. Problem Formulation and Theoretical Analysis

Throughout the paper we use bold lower and upper case letters for denoting vectors and matrices, respectively. We denote the support of a vector x_t ∈ ℝ^p by supp(x_t) and its jth element by (x_t)_j. We consider the linear compressible state-space model given by

$${\mathbf{x}}_t=\mathbf{\Theta }{\mathbf{x}}_{t-1}+{\mathbf{w}}_t,{\mathbf{y}}_t={\mathbf{A}}_t{\mathbf{x}}_t+{\mathbf{v}}_t,$$ (1)

where ${({\mathbf{x}}_t)}_{t=1}^T\in {ℝ}^p$ denote the sequence of unobservable states, Θ is the state transition matrix satisfying ||Θ||< 1, w_t ∈ ℝ^p is the state innovation sequence, ${({\mathbf{y}}_t)}_{t=1}^T\in {ℝ}^{n_t}$ are the linear observations, A_t ∈ ℝ^n_t×p denotes the measurement matrix, and v_t ∈ ℝ^n_t denotes the measurement noise. The main problem is to estimate the unobserved sequence ${({\mathbf{x}}_t)}_{t=1}^T$ (and possibly Θ), given the sequence of observations ${({\mathbf{y}}_t)}_{t=1}^T$. This problem is in general ill-posed, when n_t < p, for some t. We therefore need to make additional assumptions in order to seek a stable solution.

Given a sparsity level s and a vector x, we denote the set of its s largest magnitude entries by S, and its best s-term approximation error by σ_s(x) = ||x − x_S||₁. When σ_s(x) ~ 𝒪^{(1/2 − ξ)} for some ξ ≥ 0, we refer to x as (s, ξ)– compressible. We assume that the state innovations are sparse (resp. compressible), i.e. x_t − Θx_t−1 is s_t-sparse (resp. (s_t, ξ)- compressible) with s₁ ≫ s_t for t ∈ [T]\{1}. Our theoretical analysis pertain to the compressed sensing regime where 1 ≪ s_t < n_t ≪ p. We refer to the regime of n_t > p as the denoising regime.

For simplicity of notation, we define x₀ to be the all-zero vector in ℝ^p. For a matrix A, we denote restriction of A to its first n rows by (A)_n. We say that the matrix A ∈ ℝ^n×p satisfies the restricted isometry property (RIP) [29] of order s, if for all s-sparse x ∈ ℝ^p, we have

$$(1-{\delta }_s){\Vert \mathbf{x}\Vert }_2^2\le {\Vert \mathbf{Ax}\Vert }_2^2\le (1+{\delta }_s){\Vert \mathbf{x}\Vert }_2^2,$$ (2)

where δ_s ∈ (0, 1) is the smallest constant for which Eq. (2) holds [30]. We assume that the rows of A_t are a subset of the rows of A₁, i.e. A_t = (A₁)_nt, and define ${\stackrel{\sim }{\mathbf{A}}}_t=\sqrt{\frac{n_1}{n_t}}{\mathbf{A}}_t$. Other than its technical usefulness, the latter assumption helps avoid prohibitive storage of all the measurement matrices. In order to promote sparsity of the state innovations, we consider the dynamic ℓ₁-regularization (dynamic CS from now on) problem defined as

$$\underset{{({\mathbf{x}}_t)}_{t=1}^T,\mathbf{\Theta }}{\text{minimize}}\sum _{t=1}^T\frac{{\Vert {\mathbf{x}}_t-\mathbf{\Theta }{\mathbf{x}}_{t-1}\Vert }_1}{\sqrt{s_t}}\text{s.t.}{\Vert {\mathbf{y}}_t-{\mathbf{A}}_t{\mathbf{x}}_t\Vert }_2\le \sqrt{\frac{n_t}{n_1}}\epsilon .$$ (3)

where ε is an upper bound on the observation noise, i.e., ||v_t||₂ ≤ ε for all t. Note that this problem is a variant of the dynamic CS problem introduced in [21]. We also consider the modified Lagrangian form of (3) given by

$$\underset{{({\mathbf{x}}_t)}_{t=1}^T,\mathbf{\Theta }}{\text{minimize}}\lambda \sum _{t=1}^T\frac{{\Vert {\mathbf{x}}_t-\mathbf{\Theta }{\mathbf{x}}_{t-1}\Vert }_1}{\sqrt{s_t}}+\frac{1}{n_t}\frac{{\Vert {\mathbf{y}}_t-{\mathbf{A}}_t{\mathbf{x}}_t\Vert }_2^2}{2{\sigma }^2}.$$ (4)

for some constants ${\sigma }_2^2$ and λ ≥ 0. Note that if v_t ~ 𝒩(0, n_tσ²I), then Eq. (4) is algebraically similar to the maximum a posteriori (MAP) estimator of the states in (1), assuming that the state innovations were independent Laplace random variables with respective parameters $\lambda /\sqrt{s_t}$. We will later use this analogy to derive fast solutions to the optimization problem in (4).

Uniqueness and exact recovery of the sequence ${({\mathbf{x}}_t)}_{t=1}^T$ in the absence of noise was proved in [21] for Θ = I, by an inductive construction of dual certificates. The special case Θ = I can be considered as a generalization of the total variation (TV) minimization problem [31]. Our main result on stability of the solution of (3) is the following:

Theorem 1 (Stable Recovery of Activity in the Presence of Noise)

Let ${({\mathbf{x}}_t)}_{t=1}^T\in {ℝ}^p$ be a sequence of states with a known transition matrix Θ = θI, where |θ|< 1 and Ã_t, t ≥ 1 satisfies RIP of order 4s with δ_4s < 1/3. Suppose that n₁ > n₂ = n₃ = ⋯ = n_T. Then, the solution ${({\widehat{\mathbf{x}}}_t)}_{t=1}^T$ to the dynamic CS problem (3) satisfies

$$\frac{1}{T}\sum _{t=1}^T{\Vert {\mathbf{x}}_t-{\widehat{\mathbf{x}}}_t\Vert }_2\le \frac{1-{\theta }^T}{1-\theta }\left(12.6\left(1+\frac{1}{T}\sqrt{\frac{n_1}{n_2}}-\frac{1}{T}\right)\epsilon +\frac{3}{T}\sum _{t=1}^T\frac{{\sigma }_{s_t}({\mathbf{x}}_t-\mathbf{\Theta }{\mathbf{x}}_{t-1})}{\sqrt{s_t}}\right).$$ (5)

Proof Sketch

The proof of Theorem 1 is based on establishing a modified cone and tube constraint for the dynamic CS problem (3) and using the boundedness of the Frobenius norm of the inverse first-order differencing operator. Details of the proof are given in Appendix VII.

Remark 1

The first term on the right hand side of Theorem 1 implies that the average reconstruction error of the sequence ${({\mathbf{x}}_t)}_{t=1}^T$ is upper bounded proportional to the noise level ε, which implies the stability of the estimate. The second term is a measure of compressibility of the innovation sequence and vanishes when the sparsity condition is exactly met.

Remark 2

Similar to the general compressive sensing setup, the result of Theorem 1 holds even if the compressibility assumption on the innovation sequence is violated. In this case, the second term on the right hand side of Eq. (5) will be at most of the order of a constant. In addition, in the absence of any assumptions on the sparsity, i.e., s_t ~ 𝒪(p), the result of Theorem 1 provides a valid performance bound in the denoising regime. However, the result of Theorem 1 is the sharpest when the underlying innovation sequence is compressible, i.e., the second term in the bound is negligible compared to the first term. Even though we expect that the compressibility assumption holds for the innovation sequences pertaining to the biological signals of our interest (e.g., two-photon calcium traces of neuronal spiking and EEG sleep spindles), in order to take into account the possible violation of this assumption in our forthcoming algorithm development, we will work with the modified Lagrangian formulation of Eq. 4. The Lagrangian formulation allows us to adaptively select the sparsity level in a data-driven fashion: by choosing the trade-off parameter λ via cross-validation, the sparsity level of the reconstructed innovation sequence is adapted to the observed data.

B. Fast Iterative Solution via the EM Algorithm

Due to the high dimensional nature of the state estimation problem, algorithms with polynomial complexity exhibit poor scalability. Moreover, when the state transition matrix is not known, the dynamic CS optimization problem (4) is not convex in ( ${({\mathbf{x}}_t)}_{t=1}^T$, Θ). Therefore standard convex optimization solvers cannot be directly applied. This problem can be addressed by employing the Expectation-Maximization algorithm [22]. A related existing result considers weighted ℓ₁-regularization to adaptively capture the state dynamics [32]. Our approach is distinct in that we derive a fast solution to (4) via two nested EM algorithms, in order to jointly estimate the states and their transition matrix. The outer EM algorithm converts the estimation problem to a form suitable for the usage of the traditional Fixed Interval Smoothing (FIS) by invoking the EM interpretation of the Iterative Re-weighted Least Squares (IRLS) algorithms [33]. The inner EM algorithm performs state and parameter estimation efficiently using the FIS. We refer to our estimated as the Fast Compressible State- Space (FCSS) estimator.

In order to employ the EM theory, we first note that the problem of Eq. (4) can be interpreted as a MAP problem: the first term corresponds to the state-space prior −log p(x_t|x_t−1, Θ) = −log p_st (x_t − Θx_t−1), where $p_{s_t}(\mathbf{x})~exp(-\lambda {\Vert \mathbf{x}\Vert }_1/\sqrt{s_t})$ denoting the Laplace distribution; the second term is the negative log-likelihood of the data given the state, assuming a zero-mean Gaussian observation noise with covariance σ²I.

The choice of the ℓ₁-norm in our formulation is motivated by the superior performance of ℓ₁-regularized least-squares solutions in the compressive sensing tradition, as highlighted by Theorem 1. However, in the foregoing analogy to a MAP estimation problem, caution should be taken in interpreting the compressibility assumption in a statistical sense: we do not make the assumption that the innovation sequence ${\left\{{\mathbf{w}}_t\right\}}_{t=1}^T$ is Laplace distributed, but will exploit the algebraic similarity of the ℓ₁-regularized least-squares problem to the MAP estimator of a Laplace distributed linear state-space model in order to develop fast algorithms. In fact, it is well-known that samples from the Laplace distribution are not compressible [34]. There exist several other choices of regularization schemes for promoting compressibility, including the weighted ℓ₁-norm, the ℓ₀-quasi norm, and the ℓ_ν-norm for 0 < ν < 1. Theorem 1 suggests that the ℓ₁-norm provides a suitable regularization scheme in our setting, as the first term in the error bound is within a constant factor of that of the best sparse estimator, i.e., an estimator that has access to the location of the non-zero components of the innovation sequence. Nevertheless, the forthcoming algorithm development can be carried out using the weighted ℓ₁-norm or ℓ_ν-norm regularization, with no assumption on the statistics of the innovation sequence.

The outer EM loop of FCSS

Consider the ε-perturbed ℓ₁-norm as

$${\Vert \mathbf{x}\Vert }_{1,\epsilon }=\sqrt{x_1^2+{\epsilon }^2}+\sqrt{x_2^2+{\epsilon }^2}+\cdots +\sqrt{x_p^2+{\epsilon }^2}.$$ (6)

Note that for ε = 0, ||x||_1,ε coincides with the usual ℓ₁-norm. We define the ε-perturbed version of the dual problem (4) by

$$\underset{{({\mathbf{x}}_t)}_{t=1}^T,\mathbf{\Theta }}{\text{minimize}}\lambda \sum _{t=1}^T\frac{{\Vert {\mathbf{x}}_t-\mathbf{\Theta }{\mathbf{x}}_{t-1}\Vert }_{1,\epsilon }}{\sqrt{s_t}}+\frac{1}{n_t}\frac{{\Vert {\mathbf{y}}_t-{\mathbf{A}}_t{\mathbf{x}}_t\Vert }_2^2}{2{\sigma }^2}.$$ (7)

As it will become evident shortly, this slight modification is carried out for the sake of numerical stability. The ε-perturbation only adds a term of the order 𝒪(εp) to the estimation error bound of Theorem 1, which is negligible for small enough ε [33].

If instead of the ℓ_1,ε-norm, we had the square ℓ₂ norm, then the above problem could be efficiently solved using the FIS. The outer EM algorithm indeed transforms the problem of Eq. (7) into this form, by invoking the equivalence of the IRLS algorithm as an instance of the EM algorithm for solving ℓ_1,ε-minimization problems via the Normal/Independent (N/I) characterization of the ε-perturbed Laplace distribution [33].

Suppose that at the end of the l^th iteration, the estimates ${({\widehat{\mathbf{x}}}_t^{(l)})}_{t=1}^T$, Θ̂^(l) are obtained, given the observations ${({\mathbf{y}}_t)}_{t=1}^T$. As it is shown in Section II of the Supplementary Material, the outer EM algorithm transforms the optimization problem to:

$$\underset{{({\mathbf{x}}_t)}_{t=1}^T,\mathbf{\Theta }}{\text{minimize}}\frac{\lambda }{2}\sum _{j=1}^p\sum _{t=1}^T\frac{{({\mathbf{x}}_t-\mathbf{\Theta }{\mathbf{x}}_{t-1})}_j^2+{\epsilon }^2}{\sqrt{s_t}\sqrt{{\left({\widehat{\mathbf{x}}}_t^{(l)}-{\widehat{\mathbf{\Theta }}}^{(l)}{\widehat{\mathbf{x}}}_{t-1}^{(l)}\right)}_j^2+{\epsilon }^2}}+\sum _{t=1}^T\frac{1}{n_t}\frac{{\Vert {\mathbf{y}}_t-{\mathbf{A}}_t{\mathbf{x}}_t\Vert }_2^2}{2{\sigma }^2}.$$ (8)

in order to find ${({\widehat{\mathbf{x}}}_t^{(l+1)})}_{t=1}^T$, Θ̂^(l+1). Under mild conditions, convergence of the solution of (8) to that of (4) was established in [33]. The objective function of (8) is still not jointly convex in ( ${({\mathbf{x}}_t)}_{t=1}^T$, Θ). Therefore, to carry out the optimization, i.e. the outer M step, we will employ another instance of the EM algorithm, which we call the inner EM algorithm, to alternate between estimating of ${({\mathbf{x}}_t)}_{t=1}^T$ and Θ.

The inner EM loop of FCSS

Let ${\mathbf{W}}_t^{(l)}$ be a diagonal matrix such that

$${({\mathbf{W}}_t^{(l)})}_{j,j}={s_t}^{-1/2}{\left\{{\left({\widehat{\mathbf{x}}}_t^{(l)}-{\widehat{\mathbf{\Theta }}}^{(l)}{\widehat{\mathbf{x}}}_{t-1}^{(l)}\right)}_j^2+{\epsilon }^2\right\}}^{-1/2}.$$

Consider an estimate Θ̂^(l,m), corresponding to the m^th iteration of the inner EM algorithm within the l^th M-step of the outer EM. In this case, Eq. (8) can be thought of the MAP estimate of the Gaussian state-space model given by:

$$\begin{gathered} {\mathbf{x}}_t={\widehat{\mathbf{\Theta }}}^{(l,m)}{\mathbf{x}}_{t-1}+{\mathbf{w}}_t,{\mathbf{w}}_t~\mathcal{N}\left(\mathbf{0},\frac{1}{\lambda }{\mathbf{W}}_t^{{(l)}^{-1}}\right) \\ {\mathbf{y}}_t={\mathbf{A}}_t{\mathbf{x}}_t+{\mathbf{v}}_t,{\mathbf{v}}_t~\mathcal{N}(\mathbf{0},n_t{\sigma }^{2}I), \end{gathered}$$ (9)

In order to obtain the inner E step, one needs to find the density of ${({\mathbf{x}}_t)}_{t=1}^T$ given ${({\mathbf{y}}_t)}_{t=1}^T$ and Θ̂^(l,m)}. Given the Gaussian nature of the state-space in Eq. (9), this density is a multivariate Gaussian density, whose means and covariances can be efficiently computed using the FIS. For all t ∈ [T], the FIS performs a forward Kalman filter and a backward smoother to generate [35], [8]:

$$\begin{gathered} {\mathbf{x}}_{t\mid T}^{(l,m+1)}:=\mathbb{E}\left\{{\mathbf{x}}_t\mid {({\mathbf{y}}_t)}_{t=1}^T,{\widehat{\mathbf{\Theta }}}^{(l,m)}\right\}, \\ {\mathbf{\sum }}_{t\mid T}^{(l,m+1)}:=\mathbb{E}\left\{{\mathbf{x}}_t{\mathbf{x}}_t^{\prime }\mid {({\mathbf{y}}_t)}_{t=1}^T,{\widehat{\mathbf{\Theta }}}^{(l,m)}\right\},\text{and} \\ {\mathbf{\sum }}_{t-1,t\mid T}^{(l,m+1)}={\mathbf{\sum }}_{t,t-1\mid T}^{(l,m+1)}=\mathbb{E}\left\{{\mathbf{x}}_{t-1}{\mathbf{x}}_t^{\prime }\mid {({\mathbf{y}}_t)}_{t=1}^T,{\widehat{\mathbf{\Theta }}}^{(l,m)}\right\}. \end{gathered}$$

Note that due to the quadratic nature of all the terms involving ${({\mathbf{x}}_t)}_{t=1}^T$, the outputs of the FIS suffice to compute the expectation of the objective function in Eq. (8), i.e., the inner E step, which results in:

$$\underset{\mathbf{\Theta }}{\text{maximize}}-\frac{\lambda }{2}\left(\mathbf{\Theta }\left(\sum _{t=1}^T{\mathbf{W}}_t^{(l)}\left({\mathbf{x}}_{t-1\mid T}^{(l,m+1)}{{\mathbf{x}}^{\prime }}_{t-1\mid T}^{(l,m+1)}+{\mathbf{\sum }}_{t-1\mid T}^{(l,m+1)}\right)\right){\mathbf{\Theta }}^T\right)+\frac{\lambda }{2}\text{Tr}\left(\mathbf{\Theta }\left(\sum _{t=1}^T{\mathbf{W}}_t^{(l)}\left({\mathbf{x}}_{t-1\mid T}^{(l,m+1)}{{\mathbf{x}}^{\prime }}_{t\mid T}^{(l,m+1)}+{\mathbf{x}}_{t\mid T}^{(l,m+1)}{{\mathbf{x}}^{\prime }}_{t-1\mid T}^{(l,m+1)}+2{\mathbf{\sum }}_{t-1,t\mid T}^{(l,m+1)}\right)\right)\right),$$ (10)

to obtain Θ̂^(l,m). The solution has a closed-form given by:

$${\widehat{\mathbf{\Theta }}}^{(l,m+1)}={\left(\sum _{t=1}^{T}2{\mathbf{W}}_t^{(l)}\left({\mathbf{x}}_{t-1\mid T}^{(l,m+1)}{{\mathbf{x}}^{\prime }}_{t-1\mid T}^{(l,m+1)}+{\mathbf{\sum }}_{t-1\mid T}^{(l,m+1)}\right)\right)}^{-1}\left(\sum _{t=1}^T{\mathbf{W}}_t^{(l)}\left({\mathbf{x}}_{t-1\mid T}^{(l,m+1)}{{\mathbf{x}}^{\prime }}_{t\mid T}^{(l,m+1)}+{\mathbf{x}}_{t\mid T}^{(l,m+1)}{{\mathbf{x}}^{\prime }}_{t-1\mid T}^{(l,m+1)}+2{\mathbf{\sum }}_{t-1,t\mid T}^{(l,m+1)}\right)\right).$$ (11)

This process is repeated for M iterations for the inner EM and L iterations for the outer EM, until a convergence criterion is met. Algorithm 1 summarizes the FCSS algorithm.

Algorithm 1.

The Fast Compressible State-Space (FCSS) Estimator

1:	procedure FCSS
2:	Initialize: Θ̂(0) = 0, ${({\widehat{\mathbf{x}}}_t^{(0)}=\mathbf{0})}_{t=1}^T,{({\mathbf{W}}_t^{(0)}=\mathbf{0})}_{t=1}^T$.
3:	repeat
4:	l = 0.
5:	Outer E-step:
6:	${\mathbf{W}}_t^{(l)}=\text{diag}{\left\{\frac{1}{\sqrt{s_t}\sqrt{{\left({\widehat{\mathbf{x}}}_t^{(l)}-{\widehat{\mathbf{\Theta }}}^{(l)}{\widehat{\mathbf{x}}}_{t-1}^{(l)}\right)}_j^2+{\epsilon }^2}}\right\}}_{j=1}^p$.
7:	Outer M-step:
8:	repeat
9:	m = 0.
10:	Inner E-Step: Find the smoothed estimates ${\mathbf{x}}_{t\mid T}^{(l,m+1)},{\mathbf{\sum }}_{t\mid T}^{(l,m+1)}$ and ${\mathbf{\sum }}_{t-1,t\mid T}^{(l,m+1)}$ using a Fixed Interval Smoother for $$\{\begin{array}{l}{\mathbf{x}}_t={\widehat{\mathbf{\Theta }}}^{(l,m)}{\mathbf{x}}_{t-1}+{\mathbf{w}}_t,{\mathbf{w}}_t~\mathcal{N}\left(\mathbf{0},\frac{1}{\lambda }{\mathbf{W}}_t^{{(l)}^{-1}}\right)\hfill \\ {\mathbf{y}}_t={\mathbf{A}}_t{\mathbf{x}}_t+{\mathbf{v}}_t,{\mathbf{v}}_t~\mathcal{N}(\mathbf{0},n_t{\sigma }^{2}I)\hfill \end{array}.$$
11:	Inner M-Step: Update Θ̂(l,m+1) via Eq. (11).
12:	m ← m + 1.
13:	until convergence criteria met
14:	Update the estimates: $${\widehat{\mathbf{\Theta }}}^{(l+1)}\leftarrow {\widehat{\mathbf{\Theta }}}^{(l,m)},{\left({\widehat{\mathbf{x}}}_t^{(l+1)}\right)}_{t=1}^T\leftarrow {\left({\mathbf{x}}_{t\mid T}^{(l,m)}\right)}_{t=1}^T$$
15:	l ← l + 1.
16:	until convergence criteria met $$\widehat{\mathbf{\Theta }}\leftarrow {\widehat{\mathbf{\Theta }}}^{(l)}.{({\widehat{\mathbf{x}}}_t)}_{t=1}^T\leftarrow {\left({\widehat{\mathbf{x}}}_t^{(l)}\right)}_{t=1}^T$$
17:	end procedure

Remark 1

In order to quantify the complexity of the FCSS algorithm, we first consider the complexity of EM iterations. By virtue of the FIS procedure, the complexity of the FCSS algorithm is linear in T, i.e., the observation duration. Each iteration of the FIS algorithm requires two matrix inversions, each with a complexity of 𝒪(p³). Similarly, updating Θ requires an additional inversion which is of complexity 𝒪(p³). Altogether, the complexity of the FCSS algorithm amounts to 𝒪(p³T). As we will show in Section III, the linear dependence of the computational complexity on T makes the FCSS algorithm scale favorably when applied to long data sets. For the applications of interest in this paper, we make additional assumptions on the structure of Θ which mitigate the cubic dependence on p. In particular, we assume a diagonal structure for the calcium deconvolution problem and a two-parameter matrix form for the sleep spindle detection. Hence, the complexity reduces to O(p²T), where the quadratic dependence on p arises from matrix multiplication. Finally, the EM algorithm is known to converge linearly in the number of iterations [36]. Our numerical experiment show that in the applications of interest in this paper, very few EM iterations suffice for achieving stable estimates.

Remark 2

In order to update Θ in the inner M-step given by E. (11), we have not specifically enforced the condition ||Θ||< 1 in the maximization step. This condition is required to obtain a convergent state transition matrix which results in the stability of the state dynamics. It is easy to verify that the set of matrices Θ satisfying ||Θ||< 1 − η, is a closed convex set for small positive η, and hence one can perform the maximization in (11) by projection onto this closed convex set. Alternatively, matrix optimization methods with operator norm constraints can be used [37]. We have avoided this technicality by first finding the global minimum and examining the largest eigenvalue. In the applications of interest in this paper which follow next, the largest eigenvalue has always been found to be less than 1.

III. Results

In this section, we study the performance of the FCSS estimator on simulated data as well real data from two-photon calcium imaging recordings of neuronal activity and sleep spindle detection from EEG.

A. Application to Simulated Data

We first apply the FCSS algorithm to simulated data and compare its performance with the Basis Pursuit Denoising (BPDN) algorithm. The parameters are chosen as p = 200, T = 200, s₁ = 8, s₂ = 4, ε = 10⁻¹⁰, and Θ = 0.95I. We define the quantity 1 − n/p as the compression ratio. We refer to the case of n_t = p, i.e., no compression, as the denoising setting. The measurement matrix A is an n_t×p i.i.d. Gaussian random matrix, where n_t is chosen such that $\frac{s_t}{n_t}$ is a fixed ratio. An initial choice of $\lambda \ge \frac{2\sqrt{2}}{\sigma }\sqrt{\frac{logp}{n_t}}$ is made inspired by the theory of LASSO [38], which is further tuned using two-fold cross-validation.

Figures 1–(a) and 1–(b) show the estimated states as well as the innovations for different compression ratios for one sample component. In the denoising regime, all the innovations (including the two closely-spaced components) are exactly recovered. As we take fewer measurements, the performance of the algorithm degrades as expected. However, the overall structure of the innovation sequence is captured even for highly compressed measurements.

Fig. 1.

Reconstruction results of FCSS on simulated data vs. compression levels. (a) reconstructed states, (b) reconstructed spikes. The FCSS estimates degrade gracefully as the compression level increases.

Figure 2 shows the MSE comparison of the FCSS vs. BPDN, where the MSE is defined as $10{log}_{10}\frac{1}{T}{\sum }_{t=1}^T{\Vert {\widehat{\mathbf{x}}}_t-{\mathbf{x}}_t\Vert }_2^2$. The FCSS algorithm significantly outperforms BPDN, especially at high SNR values. Figure 3 compares the performance of FCSS and BPDN on a sample component at a compression level of n/p = 2/3, in order to visualize the performance gain implied by Figure 3.

Fig. 2.

MSE vs. SNR comparison betwee FCSS and BPDN. The FCSS significantly outperforms the BPDN, even for moderate SNR levels.

Fig. 3.

Example of state reconstruction results for FCSS and BPDN. Top: true states, Middle: FCSS state estimates, Bottom: BPDN estimates. The FCSS reconstruction closely follows the true state evolution, while the BPDN fails to capture the state dynamics.

Finally, Figure 4 shows the comparison of the estimated states for the entire simulated data in the denoising regime. As can be observed from the figure, the sparsity pattern of the states and innovations are captured while significantly denoising the observed states.

Fig. 4.

Raster plots of the observed and estimated states via FCSS. Left: noisy observations, Right: FCSS estimates. The FCSS significantly denoises the observed states.

B. Application to Calcium Signal Deconvolution

Calcium imaging takes advantage of intracellular calcium flux to directly visualize calcium signaling in living neurons. This is done by using calcium indicators, which are fluorescent molecules that can respond to the binding of calcium ions by changing their fluorescence properties and using a fluorescence or two-photon microscope and a CCD camera to capture the visual patterns [39], [40]. Since spikes are believed to be the units of neuronal computation, inferring spiking activity from calcium recordings, referred to as calcium deconvolution, is an important problem in neural data analysis. Several approaches to calcium deconvolution have been proposed in the neuroscience literature, including model-free approaches such as sequential Monte Carlo methods [41] and model-based approaches such as non-negative deconvolution methods [5], [42]. These approaches require solving convex optimization problems, which do not scale well with the temporal dimension of the data. In addition, they lack theoretical performance guarantees and do not provide clear measures for assessing the statistical significance of the detected spikes.

In order to construct confidence bounds for our estimates, we employ recent results from high-dimensional statistics [43]. We first compute the confidence intervals around the outputs of the FCSS estimates using the node-wise regression procedure of [43], at a confidence level of $1-\frac{\alpha }{2}$. We perform the node-wise regression separately for each time t. For an estimate x̂_t, we obtain ${\widehat{\mathbf{x}}}_t^{\mathrm{u}}$ and ${\widehat{\mathbf{x}}}_t^{\mathrm{l}}$ as the upper and lower confidence bounds, respectively. Next, we partition the estimates into small segments, starting with a local minimum (trough) and ending in a local maximum (peak). For the i^th component of the estimate, let t_min and t_max denote the time index corresponding to two such consecutive troughs and peaks. If the difference ${({\widehat{\mathbf{x}}}_{t_{max}}^{\mathrm{l}})}_i-{({\widehat{\mathbf{x}}}_{t_{min}}^{\mathrm{u}})}_i$ is positive, the detected innovation component is declared significant (i.e., spike) at a confidence level of 1 − α, otherwise it is discarded (i.e., no spike). We refer to this procedure as Pruned-FCSS (PFCSS).

We first apply the FCSS algorithm for calcium deconvolution in a scenario where the ground-truth spiking is recorded in vitro through simultaneous electrophysiology (cell-attached patch clamp) and two-photon calcium imaging. The calcium trace as well as the ground-truth spikes are shown for a sample neuron in Figure 5–(a). The FCSS denoised estimate of the states (black) and the detected spikes (blue) using 95% confidence intervals (orange hulls) and the corresponding quantities for the constrained f-oopsi algorithm [42] are shown in Figures 5–(b) and –(c), respectively. Both algorithms detect the large dynamic changes in the data, corresponding to the spikes, which can also be visually captured in this case. However, in doing so, the f-oopsi algorithm incurs a high rate of false positive errors, manifested as clustered spikes around the ground truth events. Similar to f-oopsi, most state-of-the-art model-based methods suffer from high false positive rate, which makes the inferred spike estimates unreliable. Thanks to the aforementioned pruning process based on the confidence bounds, the PFCSS is capable of rejecting the insignificant innovations, and hence achieve a lower false positive rate. One factor responsible for this performance gap can be attributed to the underestimation of the calcium decay rate in the transition matrix estimation step of f-oopsi. However, we believe the performance gain achieved by FCSS is mainly due to the explicit modeling of the sparse nature of the spiking activity by going beyond the Gaussian state-space modeling paradigm.

Fig. 5.

Ground-truth performance comparison between PFCSS and constrained f-oopsi. Top: the observed calcium traces (black) and ground-truth electrophysiology data (blue), Middle: PFCSS state estimates (black) with 95% confidence intervals (orange) and the detected spikes (blue), Bottom: the constrained f-oopsi state estimates (black) and the detected spikes (blue). The FCSS spike estimates closely match the ground-truth spikes with only a few false detections, while the constrained f-oopsi estimates contain significant clustered false detections.

Next, we apply the FCSS algorithm to large-scale in vivo calcium imaging recordings, for which the ground-truth is not available due to measurement constraints. The data used in our analysis was recorded from 219 spontaneously active neurons in mouse auditory cortex. The two-photon microscope operates at a rate of 30 frames per second. We chose T = 2000 samples corresponding to 1 minute for the analysis. We chose p = 108 well-separated neurons visually. We estimate the measurement noise variance by appropriate re-scaling of the power spectral density in the high frequency bands where the signal is absent. We chose a value of ε = 10^{− 10}. It is important to note that estimation of the measurement noise variance is critical, since it affects the width of the confidence intervals and hence the detected spikes. Moreover, we estimate the baseline fluorescence by averaging the signal over values within a factor of 3 standard deviations of the noise. By inspecting Eq. (4), one can see a trade-off between the choice of λ and the estimate of the observation noise variance σ². We have done our analysis in both the compression regime, with a compression ratio of 1/3 (n/p = 2/3), and the denoising regime. The measurements in the compressive regime were obtained from applying i.i.d. Gaussian random matrices to the observed calcium traces. The latter is done to motivate the use of compressive imaging, as opposed to full sampling of the field of view.

Figure 6–(a) shows the observed traces for four selected neurons. The reconstructed states using FCSS in the compressive and denoising regimes are shown in Figures 6–(b) and –(c), respectively. The 90% confidence bounds are shown as orange hulls. The FCSS state estimates are significantly denoised while preserving the calcium dynamics. Figure 7 shows the detected spikes using constrained f-oopsi and PFCSS in both the compressive and denoising regimes. Finally, Figure 8 shows the corresponding raster plots of the reconstructed spikes for the entire ensemble of neurons. Similar to the preceding application on ground-truth date, the f-oopsi algorithm detects clusters of spikes, whereas the PFCSS procedure results in sparser spike detection. This results in the detection of seemingly more active neurons in the raster plot. However, motivated by the foregoing ground-truth analysis, we believe that a large fraction of these detected spikes may be due to false positive errors. Strikingly, even with a compression ratio of 1/3 the performance of the PFCSS is similar to the denoising case. The latter observation corroborates the feasibility of compressed two-photon imaging, in which only a random fraction of the field of view is imaged, which in turn can result in higher acquisition rates.

Fig. 6.

Performance of FCSS on large-scale calcium imaging data. Top, middle and bottom rows correspond to three selected neurons labeled as Neuron 1, 2, and 3, respectively. (a) raw calcium traces, (b) FCSS reconstruction with no compression, (c) FCSS reconstruction with 2/3 compression ratio. Orange hulls show 95% confidence intervals. The FCSS significantly denoises the observed traces in both the uncompressed and compressed settings.

Fig. 7.

Reconstructed spikes of PFCSS and constrained f-oopsi from large-scale calcium imaging data. Top, middle and bottom rows correspond to three selected neurons labeled as Neuron 1, 2, and 3, respectively. (a) constrained f-oopsi spike estimates, (b) PFCSS spike estimates with no compression, (c) PFCSS spike estimates with 2/3 compression ratio. The PFCSS estimates in both the uncompressed and compressed settings are sparse in time, whereas the constrained f-oopsi estimates are in the form of clustered spikes.

Fig. 8.

Raster plot of the estimated spikes from large-scale calcium imaging data. Left: the constrained f-oopsi estimates, Middle: PFCSS estimates with no compression, Right: PFCSS estimates with a $\frac{1}{3}$ compression ratio. The PFCSS estimates are spatiotemporally sparse, whereas the constrained f-oopsi outputs temporally clustered spike estimates.

In addition to the foregoing discussion on the comparisons in Figures 5, 6, 7, and 8, two remarks are in order. First, the iterative solution at the core of FCSS is linear in the observation length and hence significantly faster than the batch-mode optimization procedure used for constrained f-oopsi. Our comparisons suggest that the FCSS reconstruction is at least 3 times faster than f-oopsi for moderate data sizes of the order of tens of minutes. Moreover, the vector formulation of FCSS allows for easy parallelization (without the need for GPU implementations), which allows simultaneous processing of ROI’s without losing speed. As a numerical example, the results of Figure 6–(b) took an average of 60–70 seconds for all the p = 108 ROI’s and T = 2000 frames on an Apple Macintosh desktop computer. Second, using only about two-thirds of the measurements achieves similar results by FCSS as using the full measurements.

C. Application to Sleep Spindle Detection

In this section we use compressible state-space models in order to model and detect sleep spindles. A sleep spindle is a burst of oscillatory brain activity manifested in the EEG that occurs during stage 2 non-rapid eye movement (NREM) sleep. It consists of stereotypical 12–14 Hz wave packets that last for at least 0.5 seconds [44]. The spindles occur with a rate of 2–5% in time, which makes their generation an appropriate candidate for compressible dynamics. Therefore, we hypothesize that the spindles can be modeled using a combination of few echoes of the response of a second order compressible state-space model. As a result, the spindles can be decomposed as sums of modulated sine waves.

In order to model the oscillatory nature of the spindles, we consider a second order autoregressive (AR) model where the pole locations are given by $a{e}^{-j2\pi \frac{f}{f_s}}$ and $a{e}^{+j2\pi \frac{f}{f_s}}$, where 0 < a < 1 is a positive constant controlling the effective duration of the impulse response, f_s is the sampling frequency and f is a putative frequency accounting for the dominant spindle frequency. The equivalent state-space model for which the MAP estimation admits the FCSS solution is therefore:

$$\begin{gathered} {\mathbf{x}}_t=2acos\left(2\pi \frac{f}{f_s}\right){\mathbf{x}}_{t-1}-a^2{\mathbf{x}}_{t-2}+{\mathbf{w}}_t, \\ {\mathbf{y}}_t={\mathbf{A}}_t{\mathbf{x}}_t+{\mathbf{v}}_t. \end{gathered}$$ (12)

Note that impulse response of the state-space dynamics is given by $h_n=a^{n}cos\left(2\pi \frac{f}{f_s}n\right)u_n$, which is a stereotypical decaying sinusoid. By defining the augmented state ${\stackrel{\sim }{\mathbf{x}}}_t={[{\mathbf{x}}_t^{\prime },{\mathbf{x}}_{t-1}^{\prime }]}^{\prime }$, Eq. (12) can be expressed in the canonical form:

$${\stackrel{\sim }{\mathbf{x}}}_t=\stackrel{\sim }{\mathbf{\Theta }}{\stackrel{\sim }{\mathbf{x}}}_{t-1}+{\stackrel{\sim }{\mathbf{w}}}_t,{\mathbf{y}}_t={\stackrel{\sim }{\mathbf{A}}}_t{\stackrel{\sim }{\mathbf{x}}}_t+{\mathbf{v}}_t,$$ (13)

where ${\stackrel{\sim }{\mathbf{w}}}_t:={[{\mathbf{w}}_t^{\prime },{\mathbf{0}}^{\prime }]}^{\prime },{\stackrel{\sim }{\mathbf{A}}}_t=[\begin{array}{ll}{\mathbf{A}}_t\hfill & \mathbf{0}\hfill \end{array}]$ and

$$\stackrel{\sim }{\mathbf{\Theta }}=\left[\begin{array}{cc}2acos\left(2\pi \frac{f}{f_s}\right)\mathbf{I}& -a^2\mathbf{I}\\ \mathbf{I}& \mathbf{0}\end{array}\right].$$

Eq. (10) can be used to update Θ̃ in the M step of the FCSS algorithm. However, Θ̃ has a specific structure in this case, determined by a and f, which needs to be taken into account in the optimization step. Let $\varphi =:2acos\left(2\pi \frac{f}{f_s}\right)$ and ψ = a², and let

$$\begin{gathered} \sum _{t=1}^T{\stackrel{\sim }{\mathbf{W}}}_t^{(l)}\left({\mathbf{x}}_{t-1\mid T}^{(l,m+1)}{{\stackrel{\sim }{\mathbf{x}}}^{\prime }}_{t-1\mid T}^{(l,m+1)}+{\mathbf{\sum }^{\sim }}_{t-1\mid T}^{(l,m+1)}\right)=:\left[\begin{array}{cc}\mathbf{A}& \mathbf{B}\\ \mathbf{C}& \mathbf{D}\end{array}\right], \\ \sum _{t=1}^T{\stackrel{\sim }{\mathbf{W}}}_t^{(l)}\left({\mathbf{x}}_{t-1\mid T}^{(l,m+1)}{{\stackrel{\sim }{\mathbf{x}}}^{\prime }}_{t\mid T}^{(l,m+1)}+{\stackrel{\sim }{\mathbf{x}}}_{t\mid T}^{(l,m+1)}{{\stackrel{\sim }{\mathbf{x}}}^{\prime }}_{t-1\mid T}^{(l,m+1)}+2{\mathbf{\sum }^{\sim }}_{t-1,t\mid T}^{(l,m+1)}\right)=:\left[\begin{array}{cc}\mathbf{E}& \mathbf{F}\\ \mathbf{G}& \mathbf{H}\end{array}\right]. \end{gathered}$$

Then, Eq. (10) is equivalent to maximizing

$$\underset{\varphi ,\psi }{\text{maximize}}\frac{\lambda }{2}({\varphi }^2\text{Tr}(\mathbf{A})-\varphi \psi \text{Tr}(\mathbf{B}+\mathbf{C})+{\psi }^2\text{Tr}(\mathbf{D})+\text{Tr}(\mathbf{A}))-\frac{\lambda }{2}(\varphi \text{Tr}(\mathbf{E})-\psi \text{Tr}(\mathbf{G})+\text{Tr}(\mathbf{F})).$$ (14)

subject to 0 ≤ ψ ≤ 1 and ϕ² ≤ 4ψ, which can be solved using interior point methods. In our implementation, we have imposed additional priors of f ~ Uniform(12, 14) Hz and a ~ Uniform(0.95, 0.99), which simplifies the constraints on ϕ and ψ to

$${0.95}^2\le \psi \le {0.99}^2,\text{and}4\psi {cos}^2\left(2\pi \frac{14}{f_s}\right)\le {\varphi }^2\le 4\psi {cos}^2\left(2\pi \frac{12}{f_s}\right).$$

Given the convexity of the cost function in (14), one can conclude from the KKT conditions that if the global minimum is not achieved inside the region of interest it must be achieved on the boundaries.

Figure 9, top panel, shows two instances of simulated spindles (black traces) with parameters f_s = 200 Hz, f = 13 Hz and a = 0.95, with the ground truth events generating the wave packets shown in red. The middle panel shows the noisy version of the data with an SNR of − 7.5 dB. The noise was chosen as white Gaussian noise plus slowly varying (2 Hz) oscillations to resemble the slow oscillations in real EEG data. As can be observed the simulated signal exhibits visual resemblance to real spindles, which verifies the hypothesis that spindles can be decomposed into few combinations of wave packets generated by a second order AR model. The third panel, shows the denoised data using FCSS, which not only is successful in detecting the the ground-truth dynamics (red bars), but also significantly denoises the data.

Fig. 9.

Performance of FCSS on simulated spindles. Top: simulated clean data (black) and ground-truth spindle events (red), Middle: simulated noisy data, Bottom: the denoised signal (black) and deconvolved spindle events (red). The FCSS estimates are significantly denoised and closely match the ground-truth data shown in the top panel.

We next apply FCSS to real EEG recordings from stage-2 NREM sleep. Manual scoring of sleep spindles can be very time-consuming, and achieving accurate manual scoring on a long-term recording is a highly demanding task with the associated risk of decreased diagnosis. Although automatic spindle detection would be attractive, most available algorithms sensitive to variations in spindle amplitude and frequency that occur between both subjects and derivations, reducing their effectiveness [45], [46]. Moreover most of these algorithms require significant pre- and post-processing and manual tuning. Examples include algorithms based on Empirical Mode Decomposition (EMD) [47], [48], [49], data-driven Bayesian methods [50], and machine learning approaches [51], [52]. Unlike our approach, none of the existing methods consider modeling the generative dynamics of spindles, as transient sparse events in time, in the detection procedure.

The data used in our analysis is part of the recordings in the DREAMS project [53], recorded using a 32-channel polysomnograpgh. We have used the EEG channels in our analysis. The data was recorded at a rate of f_s = 200 Hz for 30 minutes. The data was scored for sleep spindles independently by two experts. We have used expert annotations to separate regions which include spindles for visualization purposes. For comparison purposes, we use a bandpass filtered version of the data within 12–14 Hz, which is the basis of several spindle detection algorithms [46], [54], [55], hallmarked by the widely-used bandpass filtered root-mean-square (RMS) method [46].

Figure 10 shows the detection results along with the bandpass filtered version of the data for two of the EEG channels. The red bars show the expert markings of the onset and offset of the spindle events. The FCSS simultaneously captures the spindle events and suppressed the activity elsewhere, whereas the bandpass filtered data produces significant activity in the 12–14 Hz throughout the observation window, resulting in high false positives. To quantify this observation, we have computed the ROC curves of the FCSS and bandpass filtering followed by root mean square (RMS) computation in Figure 11, which confirms the superior performance of the FCSS algorithm over the data set. The annotations of one of the experts has been used for as the ground truth benchmark.

Fig. 10.

Performance comparison between FCSS and band-pass filtered EEG data. Left and right panels correspond to two selected electrodes labeled as 1 and 2, respectively. The orange blocks show the extent of the detected spindles by the expert. Top: raw EEG data, Middle: band-pass filtered EEG data in the 12–14 Hz band, Bottom: FCSS spindle estimates. The FCSS estimates closely match the expert annotations, while the band-pass filtered data contains significant signal components outside of the orange blocks.

Fig. 11.

The ROC curves for FCSS (solid red) and bandpass-filtered RMS (dashed black). The FCSS outperforms the widely-used band-pass filtered RMS method as indicated by the ROC curves.

IV. Discussion

In this section, we discuss the implication of our techniques in regard to the application domains as well as existing methods.

A. Connection to existing literature in sparse estimation

Contrary to the traditional compressive sensing, our linear measurement operator does not satisfy the RIP [21], despite the fact that A_t’s satisfy the RIP. Nevertheless, we have extended the near-optimal recovery guarantees of CS to our compressible state-space estimation problem via Theorem 1. Closely related problems to our setup are the super-resolution and sparse spike deconvolution problems [56], [57], in which abrupt changes with minimum separation in time are resolved in fine scales using coarse (lowpass filtered) frequency information, which is akin to working in the compressive regime.

Theoretical guarantees of CS require the number of measurements to be roughly proportional to the sparsity level for stable recovery [38]. These results do not readily generalize to the cases where the sparsity lies in the dynamics, not the states per se. Most of the dynamic compressive sensing techniques such as Kalman filtered compressed sensing, assume partial information about the support or estimate them in a greedy and often ad-hoc fashion [16], [17], [18], [19], [20]. As another example, the problem of recovering discrete signals which are approximately sparse in their gradients using compressive measurements, has been studied in the literature using Total Variation (TV) minimization techniques [58], [31]. For one-dimensional signals, since the gradient operator is not orthonormal, the Frobenius operator norm of its inverse grows linearly with the discretization level [58]. Therefore, stability results of TV minimization scale poorly with respect to discretization level. In higher dimensions, however, the fast decay of Haar coefficients allow for near-optimal theoretical guarantees [59]. A major difference of our setup with those of CS for TV-minimization is the structured and causal measurements, which unlike the non-causal measurements in [58], do not result in an overall measurement matrix satisfying RIP. We have considered dynamics with convergent transition matrices, in order to generalize the TV minimization approach. To this end, we showed that using the state-space dynamics one can infer temporally global information from local and causal measurements. Another closely related problem is the fused lasso [60] in which sparsity is promoted both on the covariates and their differences.

B. Application to calcium deconvolution

In addition to scalability and the ability to detect abrupt transitions in the states governed by discrete events in time (i.e., spikes), our method provides several other benefits compared to other spike deconvolution methods based on state-space models, such as the constrained f-oopsi algorithm. First, our sampling-complexity trade-offs are known to be optimal from the theory of compressive sensing, whereas no performance guarantee exists for constrained f-oopsi. Second, we are able to construct precise confidence intervals on the estimated states, whereas constrained f-oopsi does not produce confidence intervals over the detected spikes. A direct consequence of these confidence intervals is estimation of spikes with high fidelity and low false alarm. Third, our comparisons suggest that the FCSS reconstruction is at least 3 times faster than f-oopsi for moderate data sizes of the order of tens of minutes. Finally, our results corroborate the possibility of using compressive measurement for reconstruction and denoising of calcium traces. From a practical point of view, a compressive calcium imaging setup can lead to higher scanning rate as well as better reconstructions, which allows monitoring of larger neuronal populations [61]. Due to the structured nature of our sampling and reconstruction schemes, we can avoid prohibitive storage problems and benefit from parallel implementations.

C. Application to sleep spindle detection

Another novel application of our modeling and estimating framework is to case sleep spindle generation as a second-order dynamical system governed by compressive innovations, for which FCSS can be efficiently used to denoise and detect the spindle events. Our modeling framework suggest that spectrotemporal spindle dynamics cannot be fully captured by just pure sinusoids via bandpass filtering, as the data consistently contains significant 12–14 Hz oscillations almost everywhere (See Figure 10). Therefore, using the bandpass filtered data for further analysis purposes clearly degrades the performance of the resulting spindle detection and scoring algorithms. The FCSS provides a robust alternative to bandpass filtering in the form of model-based denoising.

In contrast to state-of-the-art methods for spindle detection, our spindle detection procedure requires minimal pre- and post-processing steps. We expect similar properties for higher order AR dynamics, which form a useful generalization of our methods for deconvolution of other transient neural signals. In particular, K-complexes during the stage 2 NREM sleep form another class of transient signals with high variability. A potential generalization of our method using higher order models can be developed for simultaneous detection of K-complexes and spindles.

V. Conclusion

In this paper, we considered estimation of compressible state-space models, where the state innovations consist of compressible discrete events. For dynamics with convergent state transition dynamics, using theory of compressed sensing we provided an optimal error bound and stability guarantees for the dynamic ℓ₁-regularization algorithm which is akin to the MAP estimator for a Laplace state-space model. We also developed a fast and low-complexity iterative algorithm, namely FCSS, for estimation of the states as well as their transition matrix. We further verified the validity of our theoretical results through simulation studies as well as application to spike deconvolution from calcium traces and detection of sleep spindles from EEG data. Our methodology has two unique major advantages: first, we have proven theoretically why our algorithm performs well, and characterized its error performance. Second, we have developed a fast algorithm, with guaranteed convergence to a solution of the deconvolution problem, which for instance, is ~ 3 times faster than the widely-used f-oopsi algorithm in calcium deconvolution applications.

While we focused on two specific application domains, our modeling and estimation techniques can be generalized to apply to broader classes of signal deconvolution problems: we have provided a framework to model transient phenomena which are driven by sparse generators in time domain, and whose event onsets are of importance. Examples include heart beat dynamics and rapid changes in the covariance structure of neural data (e.g., epileptic seizures). In the spirit of easing reproducibility, we have made a MATLAB implementation of our algorithm publicly available [28].

VII. Appendix: Proof of Theorem 1

The main idea behind the proof is establishing appropriate cone and tube constraints [30]. In order to avoid unnecessary complications we assume s₁ ≫ s₂ = ⋯ = s_T and n₁ ≫ n₂ = ⋯ = n_T. Let x̂_t = x_t + g_t, t ∈ [T] be an arbitrary solution to the primal form (3). We define z_t = x_t − θx_t−1 and h_t = z_t − ẑ_t for t ∈ [T]. For a positive integer p, let [p] := {1, 2, ⋯, p}. For an arbitrary set V ⊂ [p], x_V denotes the vector x restricted to the indices in V, i.e. all the components outside of V set to zero. We can decompose ${({\mathbf{h}}_t)}_{S_t^c}={({\mathbf{h}}_t)}_{I_1}+{({\mathbf{h}}_t)}_{I_2}+\cdots +{({\mathbf{h}}_t)}_{I_{r_t}}$, where r_t = ⌊p/4s_t⌋, and (h_t)_I1 is the 4s_t-sparse vector corresponding to 4s_t largest-magnitude entries remaining in ${({\mathbf{h}}_t)}_{S_t^c}$, (h_t)_I2 is the 4s_t-sparse vector corresponding to 4s largest-magnitude entries remaining in ${({\mathbf{h}}_t)}_{S_t^c}-{({\mathbf{h}}_t)}_{I_1}$ and so on. By the optimality of ${({\widehat{\mathbf{z}}}_t)}_{t=1}^T$ we have

$$\sum _{t=1}^T\frac{{\Vert {\mathbf{z}}_t+{\mathbf{h}}_t\Vert }_1}{\sqrt{s_t}}\le \sum _{t=1}^T\frac{{\Vert {\mathbf{z}}_t\Vert }_1}{\sqrt{s_t}}.$$

Using several instances of triangle inequality we have

$$\sum _{t=1}^T\frac{-{\sigma }_{s_t}({\mathbf{z}}_t)+{\Vert {({\mathbf{h}}_t)}_{S_t^c}\Vert }_1-{\Vert {({\mathbf{h}}_t)}_{S_t}\Vert }_1+{\Vert {({\mathbf{z}}_t)}_{S_t}\Vert }_1}{\sqrt{s_t}}\le \sum _{t=1}^T\frac{{\Vert {\mathbf{z}}_t+{\mathbf{h}}_t\Vert }_1}{\sqrt{s_t}}\le \sum _{t=1}^T\frac{{\Vert {\mathbf{z}}_t\Vert }_1}{\sqrt{s_t}}=\sum _{t=1}^T\frac{{\Vert {({\mathbf{z}}_t)}_{S_t}\Vert }_1+{\sigma }_{s_t}({\mathbf{z}}_t)}{\sqrt{s_t}},$$

which after re-arrangement yields the cone condition given by

$$\sum _{t=1}^T\frac{{\Vert {({\mathbf{h}}_t)}_{S_t^c}\Vert }_1}{\sqrt{s_t}}\le \sum _{t=1}^T\frac{{\Vert {({\mathbf{h}}_t)}_{S_t}\Vert }_1+{\Vert {({\mathbf{z}}_t)}_{S_t^c}\Vert }_1}{\sqrt{s_t}}.$$ (15)

Also, by the definition of partitions ${(I_j)}_{j=1}^{r_t}$ we have

$$\begin{gathered} \sum _{t=1}^T\sum _{j=2}^{r_t}{\Vert {({\mathbf{h}}_t)}_{I_j}\Vert }_2\le \sum _{t=1}^T\sum _{j=2}^{r_t}2\sqrt{s_t}{\Vert {({\mathbf{h}}_t)}_{I_j}\Vert }_{\infty } \\ \le \sum _{t=1}^T\sum _{j=2}^{r_t}\frac{{\Vert {({\mathbf{h}}_t)}_{I_{j-1}}\Vert }_1}{2\sqrt{s_t}}=\sum _{t=1}^T\frac{{\Vert {({\mathbf{h}}_t)}_{S_t^c}\Vert }_1}{2\sqrt{s_t}} \\ \le \sum _{t=1}^T\frac{{\Vert {({\mathbf{h}}_t)}_{S_t}\Vert }_1+{\sigma }_{s_t}({\mathbf{z}}_t)}{2\sqrt{s_t}}\le \sum _{t=1}^T\frac{{\Vert {({\mathbf{h}}_t)}_{S_t}\Vert }_2}{2}+\frac{{\sigma }_{s_t}({\mathbf{z}}_t)}{2\sqrt{s_t}}. \end{gathered}$$ (16)

Moreover, using the feasibility of both x_t and x̂_t we have the tube constraints

$$\begin{gathered} {\Vert {\mathbf{y}}_1-{\mathbf{A}}_1{\mathbf{x}}_1\Vert }_2\le \epsilon \Rightarrow {\Vert \theta {({\mathbf{y}}_1)}_{[n_2]}-\theta {\mathbf{A}}_2{\mathbf{x}}_1\Vert }_2\le \theta \epsilon , \\ {\Vert {\mathbf{y}}_2-{\mathbf{A}}_2{\mathbf{x}}_2\Vert }_2\le \sqrt{\frac{n_2}{n_1}}\epsilon , \end{gathered}$$

from which we conclude ||y₂ − θ (y₁)_[n2] − A₂z₂||₂≤ (1+θ)ε. Similarly ||y₂ − θ (y₁)_[n2] − A₂ẑ₂||₂≤ (1+θ)ε. Therefore the triangle inequality yields ||A₂h₂||₂≤ 2(1 + θ)ε. Similarly for all t ∈ [T]\{2}, we have the tighter bound

$${\Vert {\mathbf{A}}_t{\mathbf{h}}_t\Vert }_2\le 2(1+\theta )\sqrt{\frac{n_t}{n_1}}\epsilon ,$$ (17)

which is a consequence of having fewer measurements for t ∈ [T]\{2}. In conjunction, (15), (16), and (17) yield

$$\begin{gathered} 2(1+\theta )\left(T+\sqrt{\frac{n_1}{n_2}}-1\right)\epsilon \ge {\Vert {\mathbf{A}}_1{\mathbf{h}}_1\Vert }_2+\sum _{t=2}^T\sqrt{\frac{n_1}{n_t}}{\Vert {\mathbf{A}}_t{\mathbf{h}}_t\Vert }_2 \\ \ge \sum _{t=1}^T{\Vert {\stackrel{\sim }{\mathbf{A}}}_t{({\mathbf{h}}_t)}_{S_t\cup I_1}\Vert }_2-\sum _{t=1}^T\sum _{j=2}^{r_t}{\Vert {\stackrel{\sim }{\mathbf{A}}}_t{({\mathbf{h}}_t)}_{I_j}\Vert }_2 \\ \ge \sqrt{1-{\delta }_{4s}}\sum _{t=1}^T{\Vert {({\mathbf{h}}_t)}_{S_t\cup I_1}\Vert }_2-\frac{\sqrt{1+{\delta }_{4s}}}{2}\sum _{t=1}^T\sum _{j=2}^{r_t}{\Vert ({\mathbf{h}}_{I_j,t})\Vert }_2 \\ \ge \sqrt{1-{\delta }_{4s}}\sum _{t=1}^T{\Vert {({\mathbf{h}}_t)}_{S_t\cup I_1}\Vert }_2-\frac{\sqrt{1+{\delta }_{4s}}}{2}\sum _{t=1}^T{\Vert {({\mathbf{h}}_t)}_{S_t\cup I_1}\Vert }_2+\frac{{\sigma }_{s_t}\left({\mathbf{z}}_t\right)}{\sqrt{s_t}} \\ \ge \left(\sqrt{1-{\delta }_{4s}}-\sqrt{\frac{1+{\delta }_{4s}}{2}}\right)\sum _{t=1}^T{\Vert {({\mathbf{h}}_t)}_{S_t\cup I_1}\Vert }_2-\frac{\sqrt{1+{\delta }_{4s}}}{2}\sum _{t=1}^T\frac{{\sigma }_{s_t}\left({\mathbf{z}}_t\right)}{\sqrt{s_t}}. \end{gathered}$$

Therefore after rearrangement for δ_4s < 1/3

$$\sum _{t=1}^T{\Vert {({\mathbf{h}}_t)}_{S_t\cup I_1}\Vert }_2\le 8.37(1+\theta )\left(T+\sqrt{\frac{n_1}{n_2}}-1\right)\epsilon +\frac{5}{2}\sum _{t=1}^T\frac{{\sigma }_{s_t}\left({\mathbf{z}}_t\right)}{\sqrt{s_t}}.$$

Next, using (16) yields

$$\begin{gathered} \sum _{t=1}^T{\Vert {\mathbf{h}}_t\Vert }_2\le \sum _{t=1}^T\sum _{j=2}^{r_t}{\Vert {({\mathbf{h}}_t)}_{I_j}\Vert }_2+{\Vert {({\mathbf{h}}_t)}_{S_t\cup I_1}\Vert }_2 \\ \le 12.55\left(T+\sqrt{\frac{n_1}{n_2}}-1\right)\epsilon +3\sum _{t=1}^T\frac{{\sigma }_{s_t}\left({\mathbf{z}}_t\right)}{\sqrt{s_t}}. \end{gathered}$$ (18)

By definition we have h_t = g_t − θg_t−1 for t ∈ [T] with g₀ = 0. Therefore by induction we have ${\mathbf{g}}_t={\sum }_{j=1}^t{\theta }^{t-j}{\mathbf{h}}_j$ or in matrix form

$$\mathcal{G}:=\left[\begin{array}{l}{\mathbf{g}}_1\hfill \\ {\mathbf{g}}_2\hfill \\ ⋮\hfill \\ {\mathbf{g}}_t\hfill \end{array}\right]=\underset{\mathcal{A}}{\underbrace{\left[\begin{array}{llll}\mathbf{I}\hfill & 0\hfill & \cdots \hfill & 0\hfill \\ \theta \mathbf{I}\hfill & \mathbf{I}\hfill & \cdots \hfill & 0\hfill \\ {\theta }^2\mathbf{I}\hfill & \theta \mathbf{I}\hfill & \cdots \hfill & 0\hfill \\ ⋮\hfill & ⋮\hfill & \ddots \hfill & ⋮\hfill \\ {\theta }^{T-1}\mathbf{I}\hfill & {\theta }^{T-2}\mathbf{I}\hfill & \cdots \hfill & \mathbf{I}\hfill \end{array}\right]}}\left[\begin{array}{l}{\mathbf{h}}_1\hfill \\ {\mathbf{h}}_2\hfill \\ ⋮\hfill \\ {\mathbf{h}}_t\hfill \end{array}\right]=:\mathcal{A}\mathcal{H}.$$

Using several instances of the triangle inequality we get:

$$\begin{gathered} \sum _{t=1}^T{\Vert {\mathbf{g}}_t\Vert }_2=\sum _{t=1}^T{\Vert \sum _{j=1}^t{\theta }^{t-j}{\mathbf{h}}_j\Vert }_2\le \sum _{t=1}^T\sum _{j=1}^t{\theta }^{t-j}{\Vert {\mathbf{h}}_j\Vert }_2 \\ \le \sum _{j=1}^{T-1}{\theta }^j\sum _{t=1}^T{\Vert {\mathbf{h}}_t\Vert }_2=\frac{1-{\theta }^T}{1-\theta }\sum _{t=1}^T{\Vert {\mathbf{h}}_t\Vert }_2, \end{gathered}$$

which in conjunction with (18) completes the proof.