A new look at state-space models for neural data

Abstract

State space methods have proven indispensable in neural data analysis. However, common methods for performing inference in state-space models with non-Gaussian observations rely on certain approximations which are not always accurate. Here we review direct optimization methods that avoid these approximations, but that nonetheless retain the computational efficiency of the approximate methods. We discuss a variety of examples, applying these direct optimization techniques to problems in spike train smoothing, stimulus decoding, parameter estimation, and inference of synaptic properties. Along the way, we point out connections to some related standard statistical methods, including spline smoothing and isotonic regression. Finally, we note that the computational methods reviewed here do not in fact depend on the state-space setting at all; instead, the key property we are exploiting involves the bandedness of certain matrices. We close by discussing some applications of this more general point of view, including Markov chain Monte Carlo methods for neural decoding and efficient estimation of spatially-varying firing rates.

1 Introduction; forward-backward methods for inference in state-space models

A wide variety of neuroscientific data analysis problems may be attacked fruitfully within the framework of hidden Markov (“state-space”) models. The basic idea is that the underlying system may be described as a stochastic dynamical process: a (possibly multidimensional) state variable q_t evolves through time according to some Markovian dynamics p(q_t|q_t−1, θ), as specified by a few model parameters θ. Now in many situations we do not observe the state variable q_t directly (this Markovian variable is “hidden”); instead, our observations y_t are a noisy, subsampled version of q_t, summarized by an observation distribution p(y_t|q_t).

Methods for performing optimal inference and estimation in these hidden Markov models are very well-developed in the statistics and engineering literature [80, 24, 23]. For example, to compute the conditional distribution p(q_t|Y_1:T) of the state variable q_t given all the observed data on the time interval (0, T], we need only apply two straightforward recursions: a forward recursion that computes the conditional distribution of q_t given only the observed data up to time t,

$$p(q_t\mid Y_{1:t})\propto p(y_t\mid q_t)\int p(q_t\mid q_{t-1})p(q_{t-1}\mid Y_{1:t-1}){dq}_{t-1},\text{for}t=1,2,\dots ,T$$ (1)

and then a backward recursion that computes the desired distribution p(q_t|Y_1:T),

$$p(q_t\mid Y_{1:T})=p(q_t\mid Y_{1:t})\int \frac{p(q_{t+1}\mid Y_{1:T})p(q_{t+1}\mid q_t)}{p(q_{t+1}\mid q_t)p(q_t\mid Y_{1:t}){dq}_t}{dq}_{t+1},\text{for}t=T-1,T-2,\dots ,1,0.$$ (2)

Each of these recursions may be derived easily from the Markov structure of the state-space model. In the classical settings, where the state variable q is discrete [80, 32, 35, 40, 47, 36, 27], or the dynamics p(q_t|q_t−1) and observations p(y_t|q_t) are linear and Gaussian, these recursions may be computed exactly and efficiently: note that a full forward-backward sweep requires computation time which scales just linearly in the data length T, and is therefore quite tractable even for large T. In the linear-Gaussian case, this forward-backward recursion is known as the Kalman filter-smoother [85, 24, 75, 94].

Unfortunately, the integrals in equations (1) and (2) are not analytically tractable in general; in particular, for neural applications we are interested in cases where the observations y_t are point processes (e.g., spike trains, or behavioral event times), and in this case the recursions must be solved approximately. One straightforward idea is to approximate the conditional distributions appearing in (1) and (2) as Gaussian; since we can compute Gaussian integrals analytically (as in the Kalman filter), this simple approximation provides a computationally tractable, natural extension of the Kalman filter to non-Gaussian observations. Many versions of this recursive Gaussian approximation idea (with varying degrees of accuracy versus computational expediency) have been introduced in the statistics and neuroscience literature [28, 29, 5, 49, 111, 41, 9, 96, 115, 25, 119].

These methods have proven extremely useful in a wide variety of neural applications. Recursive estimation methods are especially critical in online applications, where estimates must be updated in real time as new information is observed. For example, state-space techniques achieve state-of-the-art performance decoding multineuronal spike train data from motor cortex [112, 102, 113] and parietal cortex [119, 47], and these methods therefore hold great promise for the design of motor neuroprosthetic devices [22]. In this setting, the hidden variable q_t corresponds to the desired position of the subject’s hand, or a cursor on a computer screen, at time t; y_t is the vector of observed spikes at time t, binned at some predetermined temporal resolution; the conditional probability p(y_t|q_t) is given by an “encoding” model that describes how the position information q_t is represented in the spike trains y_t; and p(q_t|Y_1:t+s) is the desired fixed-lag decoding distribution, summarizing our knowledge about the current position q_t given all of the observed spike train data Y from time 1 up to t + s, where s is a short allowed time lag (on the order of 100 ms or so in motor prosthetic applications). In this setting, the conditional expectation E(q_t|Y_1:t+s) is typically used as the optimal (minimum mean-square) estimator for q_t, while the posterior covariance Cov(q_t|Y_1:t+s) quantifies our uncertainty about the position q_t, given the observed data; both of these quantities are computed most efficiently using the forward-backward recursions (1–2). These forward-backward methods can also easily incorporate target or endpoint goal information in these online decoding tasks [100, 118, 53, 113].

State-space models have also been applied successfully to track nonstationary neuron tuning properties [11, 30, 25, 17, 81]. In this case, the hidden state variable q_t represents a parameter vector which determines the neuron’s stimulus-response function. [54] discusses an application of these recursive methods to perform optimal online experimental design —i.e., to choose the stimulus at time t which will give us as much information as possible about the observed neuron’s response properties, given all the observed stimulus-response data from time 1 to t.

A number of offline applications have appeared as well: state-space methods have been applied to perform optimal decoding of rat position given multiple hippocampal spike trains [9, 120, 25], and to model behavioral learning experiments [96, 97, 98, 101]; in the latter case, q_t represents the subject’s certainty about the behavioral task, which is not directly observable and which changes systematically over the course of the experiment. In addition, we should note that the forward-backward idea is of fundamental importance in the setting of sequential Monte Carlo (“particle-filtering”) methods [23, 8, 46, 33, 93, 26, 106, 38], though we will not focus on these applications here.

However, the forward-backward approach is not always directly applicable. For example, in many cases the dynamics p(q_t|q_t−1) or observation density p(y_t|q_t) may be non-smooth (e.g., the state variable q may be constrained to be nonnegative, leading to a discontinuity in log p(q_t|q_t−1) at q_t = 0). In these cases the forward distribution p(q_t|Y_1:t) may be highly non-Gaussian, and the basic forward-backward Gaussian approximation methods described above may break down¹. In this paper, we will review more general direct optimization methods for performing inference in state-space models. We discuss this approach in section 2 below. This direct optimization approach also leads to more efficient methods for estimating the model parameters θ (section 3). Finally, the state-space model turns out to be a special case of a richer, more general framework involving banded matrix computations, as we discuss at more length in section 4.

2 A direct optimization approach for computing the maximum a posteriori path in state-space models

2.1 A direct optimization interpretation of the classical Kalman filter

We begin by taking another look at the classical Kalman filter-smoother [24, 112, 94]. The primary goal of the smoother is to compute the conditional expectation E(Q|Y) of the hidden state path Q given the observations Y. (Throughout this paper, we will use Q and Y to denote the full collection of the hidden state variables {q_t} and observations {y_t}, respectively.) Due to the linear-Gaussian structure of the Kalman model, (Q, Y) forms a jointly Gaussian random vector, and therefore p(Q|Y) is itself Gaussian. Since the mean and mode of a Gaussian distribution coincide, this implies that E(Q|Y) is equal to the maximum a posteriori (MAP) solution, the maximizer of the posterior p(Q|Y). If we write out the linear-Gaussian Kalman model more explicitly,

$$\begin{array}{l}{q}_t={Aq}_{t-1}+{\epsilon }_t,{\epsilon }_t~\mathcal{N}(0,C_q);q_1~\mathcal{N}({\mu }_1,C_1)\\ y_t={Bq}_t+{\eta }_t,{\eta }_t~\mathcal{N}(0,C_y)\end{array}$$

(where (0, C) denotes the Gaussian density with mean 0 and covariance C), we can gain some insight into the the analytical form of this maximizer:

$$\begin{array}{l}E(Q\mid Y)=arg\underset{Q}{max}p(Q\mid Y)\\ =arg\underset{Q}{max}logp(Q,Y)\\ =arg\underset{Q}{max}\left(logp(q_1)+\sum _{t=2}^{T}logp(q_t\mid q_{t-1})+\sum _{t=1}^{T}logp(y_t\mid q_t)\right)\\ =arg\underset{Q}{max}\left[-\frac{1}{2}\left({(q_1-{\mu }_1)}^T{C}_1^{-1}(q_1-{\mu }_1)+\sum _{t=2}^T{(q_t-{Aq}_{t-1})}^T{C}_q^{-1}(q_t-{Aq}_{t-1})+\sum _{t=1}^T{(y_t-{Bq}_t)}^T{C}_y^{-1}(y_t-{Bq}_t)\right)\right].\end{array}$$ (3)

The right-hand-side here is a simple quadratic function of Q (as expected, since p(Q|Y) is Gaussian, i.e., log p(Q|Y) is quadratic), and therefore E(Q|Y) may be computed by solving an unconstrained quadratic program in Q; we thus obtain

$$\widehat{Q}=arg\underset{Q}{max}logp(Q\mid Y)=arg\underset{Q}{max}\left[\frac{1}{2}{Q}^{T}HQ+{\nabla }^{T}Q\right]=-H^{-1}\nabla ,$$

where we have abbreviated the Hessian and gradient of log p(Q|Y):

$$\begin{array}{l}\nabla ={\nabla }_{Q}logp(Q{\mid Y)\mid }_{Q=0}\\ H=\nabla {\nabla }_{Q}logp(Q\mid Y){\mid }_{Q=0}.\end{array}$$

The next key point to note is that the Hessian matrix H is block-tridiagonal, since log p(Q|Y) is a sum of simple one-point potentials (log p(q_t) and log p(y_t|q_t)) and nearest-neighbor two-point potentials (log p(q_t, q_t−1)). More explicitly, we may write

$$H=\left(\begin{array}{cccccc}{D}_1& R_{1,2}^T& 0& \cdots & & 0\\ R_{1,2}& D_2& R_{2,3}^T& 0& & ⋮\\ 0& R_{2,3}& D_3& R_{3,4}& \ddots & \\ ⋮& \ddots & & \ddots & \ddots & 0\\ & & & & D_{N-1}& R_{N-1,N}^T\\ 0& \cdots & & 0& R_{N-1,N}& D_N\end{array}\right)$$ (4)

where

$$D_i=\frac{{\partial }^2}{\partial q_i^2}logp(y_i\mid q_i)+\frac{{\partial }^2}{\partial q_i^2}logp(q_i\mid q_{i-1})+\frac{{\partial }^2}{\partial q_i^2}logp(q_{i+1}\mid q_i),$$ (5)

and

$$R_{i,i+1}=\frac{{\partial }^2}{\partial q_i\partial q_{i+1}}logp(q_{i+1}\mid q_i)$$ (6)

for 1 < i < N. These quantities may be computed as simple functions of the Kalmanm del parameters; for example, $R_{i,i+1}=C_q^{-1}A$.

This block-tridiagonal form of H implies that the linear equation Q̂ = H⁻¹∇ may be solved in O(T) time (e.g., by block-Gaussian elimination [78]; note that we never need to compute H⁻¹ explicitly). Thus this matrix formulation of the Kalman smoother is equivalent both mathematically and in terms of computational complexity to the forward-backward method. In fact, the matrix formulation is often easier to implement; for example, if H is sparse and banded, the standard Matlab backslash command Q̂ = H\∇ calls the O(T) algorithm automatically — Kalman smoothing in just one line of code.

We should also note that a second key application of the Kalman filter is to compute the posterior state covariance Cov(q_t|Y) and also the nearest-neighbor second moments $E(q_t{q}_{t+1}^T\mid Y)$; the posterior covariance is required for computing confidence intervals around the smoothed estimates E(q_t|Y), while the second moments $E(q_t{q}_{t+1}^T\mid Y)$ are necessary to compute the sufficient statistics in the expectation-maximization (EM) algorithm for estimating the Kalman model parameters (see, e.g., [94] for details). These quantities may easily be computed in O(T) time in the matrix formulation. For example, since the matrix H represents the inverse posterior covariance matrix of our Gaussian vector Q given Y, Cov(q_t|Y) is given by the (t, t)-th block of H⁻¹, and it is well-known that the diagonal and off-diagonal blocks of the inverse of a block-tridiagonal matrix can be computed in O(T) time; again, the full inverse H⁻¹ (which requires O(T²) time in the block-tridiagonal case) is not required [86, 87, 4].

2.2 Extending the direct optimization method to non-Gaussian models

From here it is straightforward to extend this approach to directly compute Q̂_MAP in non-Gaussian models of interest in neuroscience. In this paper we will focus on the case that log p(q_t+1|q_t) is a concave function of Q; in addition, we will assume that the initial density log p(q₀) is concave and also that the observation density log p(y_t|q_t) is concave in q_t. Then it is easy to see that the log-posterior

$$logp(Q\mid Y)=logp(q_0)+\sum _{t}logp(y_t\mid q_t)+\sum _{t}logp(q_{t+1}\mid q_t)+\mathit{const}.$$

is concave in Q, and therefore computing the MAP path Q̂ is a concave problem. Further, if log p(q₀), log p(y_t|q_t), and log p(q_t+1|q_t) are all smooth functions of Q, then we may apply standard approaches such as Newton’s algorithm to solve this concave optimization.

To apply Newton’s method here, we simply iteratively solve the linear equation²

$${\widehat{Q}}^{(i+1)}={\widehat{Q}}^{(i)}-H^{-1}\nabla ,$$

where we have again abbreviated the Hessian and gradient of the objective function log p(Q|Y):

$$\begin{array}{c}\nabla ={\nabla }_{Q}logp(Q\mid Y){\mid }_{Q={\widehat{Q}}^{(i)}}\\ H=\nabla {\nabla }_{Q}logp(Q\mid Y){\mid }_{Q={\widehat{Q}}^{(i)}}.\end{array}$$

Clearly, the only difference between the general non-Gaussian case here and the special Kalman case described above is that the Hessian H and gradient ∇ must be recomputed at each iteration Q̂⁽ⁱ⁾; in the Kalman case, again, log p(Q|Y) is a quadratic function, and therefore the Hessian H is constant, and one iteration of Newton’s method suffices to compute the optimizer Q̂.

In practice, this Newton algorithm converges within a few iterations for all of the applications discussed here. Thus we may compute the MAP path exactly using this direct method, in time comparable to that required to obtain the approximate MAP path computed by the recursive approximate smoothing algorithm discussed in section 1. This close connection between the Kalman filter and the Newton-based computation of the MAP path in more general state-space models is well-known in the statistics and applied math literature (though apparently less so in the neuroscience literature). See [28, 29, 5, 18, 42] for further discussion from a statistical point of view, and [51] for applications to the integrate-and-fire model for spiking data. In addition, [117] previously applied a related direct optimization approach in the context of neural decoding (though note that the conjugate gradients approach utilized there requires O(T³) time if the banded structure of the Hessian is not exploited).

2.3 Example: inferring common input effects in multineuronal spike train recordings

Recent developments in multi-electrode recording technology [63, 55] and fluorescence microscopy [14, 67, 64] enable the simultaneous measurement of the spiking activity of many neurons. Analysis of such multineuronal data is one of the key challenges in computational neuroscience today [10], and a variety of models for these data have been introduced [12, 103, 56, 39, 90, 72, 102, 65, 89, 92, 77, 91]. Most of these models include stimulus-dependence terms and “direct coupling” terms representing the influence that the activity of an observed cell might have on the other recorded neurons. These coupling terms are often interpreted in terms of “functional connectivity” between the observed neurons; the major question now is how accurately this inferred functional connectivity actually reflects the true underlying anatomical connectivity in the circuit.

Fewer models, however, have attempted to include the effects of the population of neurons which are not directly observed during the experiment [65, 66, 52]. Since we can directly record from only a small fraction of neurons in any physiological preparation, such unmeasured neurons might have a large collective impact on the dynamics and coding properties of the observed neural population, and may bias the inferred functional connectivity away from the true anatomical connectivity, complicating the interpretation of these multineuronal analyses. For example, while [77] found that neighboring parasol retinal ganglion cells (RGCs) in the macaque are functionally coupled — indeed, incorporating this functional connectivity in an optimal Bayesian decoder significantly amplifies the information we can extract about the visual stimulus from the observed spiking activity of large ensembles of RGCs — [48] recently demonstrated, via simultaneous pairwise intracellular recordings, that RGCs receive a significant amount of strongly correlated common input, with weak direct anatomical coupling between RGCs. Thus the strong functional connectivity observed in this circuit is in fact largely driven by common input, not direct anatomical connectivity.

Therefore it is natural to ask if it is possible to correctly infer the degree of common input versus direct coupling in partially-observed neuronal circuits, given only multineuronal spike train data (i.e., we do not want to rely on multiple simultaneous intracellular recordings, which are orders of magnitude more difficult to obtain than extracellular recordings). To this end, [52] introduced a state-space model in which the firing rates depend not only on the stimulus history and the spiking history of the observed neurons but also on common input effects (Fig. 1). In this model, the conditional firing intensity, λ_i(t), of the i-th observed neuron is:

$${\lambda }_i(t)=exp\left({\mathbf{k}}_i·\mathbf{x}(t)+{\mathbf{h}}_i·{\mathbf{y}}_i(t)+\left(\sum _{i\ne j}{\mathbf{l}}_{ij}·{\mathbf{y}}_j(t)\right)+{\mu }_i+q_i(t)\right),$$ (7)

where x is the spatiotemporal visual stimulus, y_i is cell i’s own spike-train history, μ_i is the cell’s baseline log-firing rate, y_j are the spike-train histories of other cells at time t, k_i is the cell’s spatiotemporal stimulus filter, h_i is the post-spike temporal filter accounting for past spike dependencies within cell i, and l_ij are direct coupling temporal filters, which capture the dependence of cell i’s activity on the recent spiking of other cells j. The term q_i(t), the hidden common input at time t, is modeled as a Gauss-Markov autoregressive process, with some correlation between different cells i which we must infer from the data. In addition, we enforce a nonzero delay in the direct coupling terms, so that the effects of a spike in one neuron on other neurons are temporally strictly causal.

Figure 1.

Schematic illustration of the common-input model described by equation (7); adapted from [52].

In statistical language, this common-input model is a multivariate version of a Cox process, also known as a doubly-stochastic point process [15, 99, 59]; the state-space models applied in [96, 102, 17] are mathematically very similar. See also [116] for discussion of a related model in the context of motor planning and intention.

As an example of the direct optimization methods developed in the preceding subsection, we reanalyzed the data from [77] with this common input model [104]. We estimated the model parameters θ = (k_i, h_i, l_ij, μ_i) from the spiking data by maximum marginal likelihood, as described in [51] (see also section 3 below, for a brief summary); the correlation time of Q was set to ~ 5 ms, to be consistent with the results of [48]. We found that this common-input model explained the observed cross-correlations quite well (data not shown), and the inferred direct-coupling weights were set to be relatively small (Fig. 2); in fact, the quality of the fits in our preliminary experiments is indistinguishable from those described in [77], where a model with strong direct-coupling terms and no common-input effects was used.

Figure 2.

Single-trial inference of the relative contribution of common, stimulus, direct coupling, and self inputs in a pair of retinal ganglion ON cells [104]; data from [77]. Top panel: Inferred linear common input, Q̂: red trace shows a sample from the posterior distribution p(Q|Y), black trace shows the conditional expectation E(Q|Y), and shaded region indicates ±1 posterior standard deviation about E(Q|Y), computed from the diagonal of the inverse log-posterior Hessian H. 2^nd panel: Direct coupling input from the other cell, l_j · y_j. (The first two panels are plotted on the same scale to facilitate comparison of the magnitudes of these effects.) Blue trace indicates cell 1; green indicates cell 2. 3^rd panel: The stimulus input, k · x. 4^th panel: Refractory input, h_i · y_i. Note that this term is strong but quite short-lived following each spike. All units are in log-firing rate, as in equation (7). Bottom: Observed paired spike trains Y on this single trial. Note the large magnitude of the estimated common input term q̂(t), relative to the direct coupling contribution l_j · y_j.

Given the estimated model parameters θ, we used the direct optimization method to estimate the sub-threshold common input effects q(t), on a single-trial basis (Fig. 2). The observation likelihood p(y_t|q_t) here was given by the standard point-process likelihood [99]:

$$logp(Y\mid Q)=\sum _{it}{y}_{it}log{\lambda }_i(t)-{\lambda }_i(t)dt,$$ (8)

where y_it denotes the number of spikes observed in time bin t from neuron i; dt denotes the temporal binwidth. We see in Fig. 2 that the inferred common input effect is strong relative to the direct coupling effects, in agreement with the intracellular recordings described in [48]. We are currently working to quantify these common input effects q_i(t) inferred from the full observed RGC population, rather than just the pairwise analysis shown here, in order to investigate the relevance of this strong common input effect on the coding and correlation properties of the full network of parasol cells. See also [113] for applications of similar common-input state-space models to improve decoding of population neural activity in motor cortex.

2.4 Constrained optimization problems may be handled easily via the log-barrier method

So far we have assumed that the MAP path may be computed via an unconstrained smooth optimization. In many examples of interest we have to deal with constrained optimization problems instead. In particular, nonnegativity constraints arise frequently on physical grounds; as emphasized in the introduction, forward-backward methods based on Gaussian approximations for the forward distribution p(q_t|Y_0:t) typically do not accurately incorporate these constraints. To handle these constrained problems while exploiting the fast tridiagonal techniques discussed above, we can employ standard interior-point (aka “barrier”) methods [7, 51]. The idea is to replace the constrained concave problem

$${\widehat{Q}}_{\mathit{MAP}}=arg\underset{Q:q_t\ge 0}{max}logp(Q\mid Y)$$

with a sequence of unconstrained concave problems

$${\widehat{Q}}_{\epsilon }=arg\underset{Q}{max}logp(Q\mid Y)+\epsilon \sum _{t}log{q}_t;$$

clearly, Q̂_ε satisfies the nonnegativity constraint, since log u → −∞ as u → 0. (We have specialized to the nonnegative case for concreteness, but the idea may be generalized easily to any convex constraint set; see [7] for details.) Furthermore, it is easy to show that if Q̂_MAP is unique, then Q̂_ε converges to Q̂_MAP as ε → 0.

Now the key point is that the Hessian of the objective function log p(Q|Y) + ε Σ_t log q_t retains the block-tridiagonal properties of the original objective log p(Q|Y), since the barrier term contributes a simple diagonal term to H. Therefore we may use the O(T) Newton iteration to obtain Q̂_ε, for any ε, and then sequentially decrease ε (in an outer loop) to obtain Q̂. Note that the approximation arg max_Q p(Q|Y) ≈ E(Q|Y) will typically not hold in this constrained case, since the mean of a truncated Gaussian distribution will typically not coincide with the mode (unless the mode is sufficiently far from the nonnegativity constraint).

We give a few applications of this barrier approach below. See also [51] for a detailed discussion of an application to the integrate-and-fire model, and [105] for applications to the problem of efficiently deconvolving slow, noisy calcium fluorescence traces to obtain nonnegative estimates of spiking times. In addition, see [16] for an application of the log-barrier method to infer firing rates given point process observations in a closely-related Gaussian process model [82]; these authors considered a slightly more general class of covariance functions on the latent stochastic process q_t, but the computation time of the resulting method scales superlinearly³ with T.

2.4.1 Example: point process smoothing under Lipschitz or monotonicity constraints on the intensity function

A standard problem in neural data analysis is to smooth point process observations; that is, to estimate the underlying firing rate λ(t) given single or multiple observations of a spike train [44]. One simple approach to this problem is to model the firing rate as λ(t) = f(q_t), where f(.) is a convex, log-concave, monotonically increasing nonlinearity [69] and q_t is an unobserved function of time we would like to estimate from data. Of course, if q_t is an arbitrary function, we need to contend with overfitting effects; the “maximum likelihood” Q̂ here would simply set f(q_t) to zero when no spikes are observed (by making −q_t very large) and f(q_t) to be very large when spikes are observed (by making q_t very large).

A simple way to counteract this overfitting effect is to include a penalizing prior; for example, if we model q_t as a linear-Gaussian autoregressive process

$$q_{t+dt}=q_t+{\epsilon }_t,{\epsilon }_t~\mathcal{N}(0,{\sigma }^{2}dt),$$

then computing Q̂_MAP leads to a tridiagonal optimization, as discussed above. (The resulting model, again, is mathematically equivalent to those applied in [96, 102, 52, 17, 104].) Here 1/σ² acts as a regularization parameter: if σ² is small, the inferred Q̂_MAP will be very smooth (since large fluctuations are penalized by the Gaussian autoregressive prior), whereas if σ² is large, then the prior term will be weak and Q̂_MAP will fit the observed data more closely.

A different method for regularizing Q was introduced by [13]. The idea is to impose hard Lipschitz constraints on Q, instead of the soft quadratic penalties imposed in the Gaussian state-space setting: we assume

$$\mid q_t-q_s\mid <K\mid t-s\mid$$

for all (s, t), for some finite constant K. (If q_t is a differentiable function of t, this is equivalent to the assumption that the maximum absolute value of the derivative of Q is bounded by K.) The space of all such Lipschitz Q is convex, and so optimizing the concave loglikelihood function under this convex constraint remains tractable. [13] presented a powerful method for solving this optimization problem (their solution involved a dual formulation of the problem and an application of specialized fast min-cut optimization methods). In this one-dimensional temporal smoothing case, we may solve this problem in a somewhat simpler way, without any loss of efficiency, using the tridiagonal log-barrier methods described above. We just need to rewrite the constrained problem

$$maxlogp(Q\mid Y)s.t.\mid q_t-q_s\mid <K\mid t-s\mid \forall s,t$$

as the unconstrained problem

$$maxlogp(Q\mid Y)+\sum _{t}b\left(\frac{q_t-q_{t+dt}}{dt}\right),$$

with dt some arbitrarily small constant and the hard barrier function b(.) defined as

$$b(u)=\{\begin{array}{ll}0\hfill & \mid u\mid <K\hfill \\ -\infty \hfill & \mathit{otherwise}.\hfill \end{array}$$

The resulting concave objective function is non-smooth, but may be optimized stably, again, via the log-barrier method, with efficient tridiagonal Newton updates. (In this case, the Hessian of the first term log p(Q|Y) with respect to Q is diagonal and the Hessian of the penalty term involving the barrier function is tridiagonal, since b(.) contributes a two-point potential here.) We recover the standard state-space approach if we replace the hard-threshold penalty function b(.) with a quadratic function; conversely, we may obtain sharper estimates of sudden changes in q_t if we use a concave penalty b(.) which grows less steeply than a quadratic function (so as to not penalize large changes in q_t as strongly), as discussed by [31]. Finally, it is interesting to note that we may also easily enforce monotonicity constraints on q_t, by choosing the penalty function b(u) to apply a barrier at u = 0; this is a form of isotonic regression [95], and is useful in cases where we believe that a cell’s firing rate should be monotonically increasing or decreasing throughout the course of a behavioral trial, or as a function of the magnitude of an applied stimulus.

2.4.2 Example: inferring presynaptic inputs given postsynaptic voltage recordings

To further illustrate the flexibility of this method, let’s look at a multidimensional example. Consider the problem of identifying the synaptic inputs a neuron is receiving: given voltage recordings at a postsynaptic site, is it possible to recover the time course of the presynaptic conductance inputs? This question has received a great deal of experimental and analytical attention [6, 74, 110, 79, 61, 37, 108, 114, 71], due to the importance of understanding the dynamic balance between excitation and inhibition underlying sensory information processing.

We may begin by writing down a simple state-space model for the evolution of the post-synaptic voltage and conductance:

$$\begin{array}{l}{V}_{t+dt}=V_t+dt[g^L(V^L-V_t)+g_t^E(V^E-V_t)+g_t^I(V^I-V_t)]+{\epsilon }_t\\ g_{t+dt}^E=g_t^E-\frac{g_t^E}{{\tau }_E}dt+N_t^E\\ g_{t+dt}^I=g_t^I-\frac{g_t^I}{{\tau }_I}dt+N_t^I.\end{array}$$ (9)

Here $g_t^E$ denotes the excitatory presynaptic conductance at time t, and $g_t^I$ the inhibitory conductance; V^L, V^E, and V^I denote the leak, excitatory, and inhibitory reversal potentials, respectively. Finally, ε_t denotes an unobserved i.i.d. current noise with a log-concave density, and $N_t^E$ and $N_t^I$ denote the presynaptic excitatory and inhibitory inputs (which must be nonnegative on physical grounds); we assume these inputs also have a log-concave density.

Assume V_t is observed noiselessly for simplicity. Then let our observed variable y_t = V_t+dt − V_t and our state variable $q_t={(g_t^E{g}_t^I)}^T$. Now, since $g_t^I$ and $g_t^E$ are linear functions of $N_t^I$ and $N_t^E$ (for example, $g_t^I$ is given by the convolution $g_t^I=N_t^I\ast exp(-t/{\tau }_I)$, the log-posterior may be written as

$$\begin{array}{l}logp(Q\mid Y)=logp(Y\mid Q)+logp(N_t^I,N_t^E)+\mathit{const}.\\ =logp(Y\mid Q)+\sum _{t=1}^{T}logp(N_t^E)+\sum _{t=1}^{T}logp(N_t^I)+\mathit{const}.,N_t^E,N_t^I\ge 0;\end{array}$$

in the case of white Gaussian current noise ε_t with variance σ²dt, for example⁴, we have

$$logp(Y\mid Q)=-\frac{1}{2{\sigma }^{2}dt}\sum _{t=2}^T{\left[V_{t+dt}-\left(V_t+dt(g^L(V^L-V_t)+g_t^I(V^I-V_t)+g_t^E(V^E-V_t))\right)\right]}^2+\mathit{const}.;$$

this is a quadratic function of Q.

Now applying the O(T) log-barrier method is straightforward; the Hessian of the log-posterior log p(Q|Y) in this case is block-tridiagonal, with blocks of size two (since our state variable q_t is two-dimensional). The observation term log p(Y|Q) contributes a block-diagonal term to the Hessian; in particular, each observation y_t contributes a rank-1 matrix of size 2×2 to the t-th diagonal block of H. (The low-rank nature of this observation matrix reflects the fact that we are attempting to extract two variables — the excitatory and inhibitory conductances at each time step — given just a single voltage observation per time step.)

Some simulated results are shown in Fig. 3. We generated Poisson spike trains from both inhibitory and excitatory presynaptic neurons, then formed the postsynaptic current signal I_t by contaminating the summed synaptic and leak currents with white Gaussian noise as in equation (9), and then used the O(T) log-barrier method to simultaneously infer the presynaptic conductances from the observed current I_t. The current was recorded at 1 KHz (1 ms bins), and we reconstructed the presynaptic activity at the same time resolution. We see that the estimated Q̂ here does a good job extracting both excitatory and inhibitory synaptic activity given a single trace of observed somatic current; there is no need to average over multiple trials. It is worth emphasizing that we are inferring two presynaptic signals here given just one observed postsynaptic current signal, with limited “overfitting” artifacts; this is made possible by the sparse, nonnegatively constrained nature of the inferred presynaptic signals. For simplicity, we assumed that the membrane leak, noise variance, and synaptic time constants τ_E and τ_I were known here; we used exponential (sparsening) priors $p(N_t^E)$ and $p(N_t^I)$, but the results are relatively robust to the details of these priors (data not shown). See [37, 38, 71] for further details and extensions, including methods for inferring the membrane parameters directly from the observed data.

Figure 3.

Inferring presynaptic inputs given simulated postsynaptic voltage recordings. Top: true simulated conductance input (green indicates inhibitory conductance; blue excitatory). Middle: observed noisy current trace from which we will attempt to infer the input conductance. Bottom: Conductance inferred by nonnegative MAP technique. Note that inferred conductance is shrunk in magnitude compared to true conductance, due to the effects of the prior $p(N_t^E)$ and $p(N_t^I)$, both of which peak at zero here; shrinkage is more evident in the inferred inhibitory conductance, due to the smaller driving force (the holding potential in this experiment was −62 mV, which is quite close to the inhibitory reversal potential; as a result, the likelihood term is much weaker for the inhibitory conductance than for the excitatory term). Inference here required about one second on a laptop computer per second of data (i.e., real time), at a sampling rate of 1 KHz.

3 Parameter estimation

In the previous sections we have discussed the inference of the hidden state path Q in the case that the system parameters are known. However, in most applications, we need to estimate the system parameters as well. As discussed in [45], standard modern methods for parameter estimation are based on the likelihood p(Y|θ) of the observed data Y given the parameters θ. In the state-space setting, we need to compute the marginal likelihood by integrating out the hidden state path Q⁵:

$$p(Y\mid \theta )=\int p(Q,Y\mid \theta )dQ=\int p(Y\mid Q,\theta )p(Q\mid \theta )dQ.$$ (10)

This marginal likelihood is a unimodal function of the parameters θ in many important cases [70], making maximum likelihood estimation feasible in principle. However, this high-dimensional integral can not be computed exactly in general, and so many approximate techniques have been developed, including Monte Carlo methods [84, 18, 42, 1] and expectation propagation [58, 115, 119, 51].

In the neural applications reviewed in section 1, the most common method for maximizing the loglikelihood is the approximate Expectation-Maximization (EM) algorithm introduced in [96], in which the required expectations are approximated using the recursive Gaussian forward-backward method. This EM algorithm can be readily modified to optimize an approximate log-posterior p(θ|Y), if a useful prior distribution p(θ) is available [52]. While the EM algorithm does not require us to compute the likelihood p(Y|θ) explicitly, we may read this likelihood off of the final forward density approximation p(q_T, y_1:T) by simply marginalizing out the final state variable q_T. All of these computations are recursive, and may therefore be computed in O(T) time.

The direct global optimization approach discussed in the preceding section suggests a slightly different strategy. Instead of making T local Gaussian approximations recursively — once for each forward density p(q_t, y_1:t) — we make a single global Gaussian approximation for the full joint posterior:

$$logp(Q\mid Y,\theta )\approx -\frac{T}{2}log(2\pi )-\frac{1}{2}log\mid -H_{\theta }\mid +\frac{1}{2}{(Q-{\widehat{Q}}_{\theta })}^T{H}_{\theta }(Q-{\widehat{Q}}_{\theta }),$$ (11)

where the Hessian H_θ is defined as

$$H_{\theta }={\nabla {\nabla }_Q(logp(Q\mid \theta )+logp(Y\mid Q,\theta ))|}_{Q={\widehat{Q}}_{\theta }};$$

note the implicit dependence on θ through

$${\widehat{Q}}_{\theta }\equiv arg\underset{Q}{max}[logp(Q\mid \theta )+logp(Y\mid Q,\theta )].$$

Equation (11) corresponds to nothing more than a second-order Taylor expansion of log p(Q|Y, θ) about the optimizer Q̂_θ.

Plugging this Gaussian approximation into equation (10), we obtain the standard “Laplace” approximation [43, 18, 117] for the marginal likelihood,

$$logp(Y\mid \theta )\approx logp(Y\mid {\widehat{Q}}_{\theta },\theta )+logp({\widehat{Q}}_{\theta }\mid \theta )-\frac{1}{2}log\mid -H_{\theta }\mid +\mathit{const}.$$ (12)

Clearly the first two terms here can be computed in O(T), since we have already demonstrated that we can obtain Q̂_θ in O(T) time, and evaluating log p(Y|Q, θ) + log p(Q|θ) at Q = Q̂_θ is relatively easy. We may also compute log |− H_θ| stably and in O(T) time, via the Cholesky decomposition for banded matrices [18, 51]. In fact, we can go further: [51] show how to compute the gradient of equation (12) with respect to θ in O(T) time, which makes direct optimization of this approximate likelihood feasible via conjugate gradient methods⁶.

It is natural to compare this direct optimization approach to the EM method; again, see [51] for details on the connections between these two approaches. It is well-known that EM can converge slowly, especially in cases in which the so-called “ratio of missing information” is large; see [19, 57, 88, 68] for details. In practice, we have found that direct gradient ascent of expression (12) is significantly more efficient than the EM approach in the models discussed in this paper; for example, we used the direct approach to perform parameter estimation in the retinal example discussed above in section 2.3. One important advantage of the direct ascent approach is that in some special cases, the optimization of (12) can be performed in a single step (as opposed to multiple EM steps). We illustrate this idea with a simple example below.

3.1 Example: detecting the location of a synapse given noisy, intermittent voltage observations

Imagine we making noisy observations from a dendritic tree (for example, via voltage-sensitive imaging methods [21]) which is receiving synaptic inputs from another neuron. We do not know the strength or location of these inputs, but we do have complete access to the spike times of the presynaptic neuron (for example, we may be stimulating the presynaptic neuron electrically or via photo-uncaging of glutamate near the presynaptic cell [3, 64]). How can we determine if there is a synapse between the two cells, and if so, how strong the synapse is and where it is located on the dendritic tree?

To model this experiment, we assume that the neuron is in a stable, subthreshold regime, i.e., the spatiotemporal voltage dynamics are adequately approximated by the linear cable equation

$${\overrightarrow{V}}_{t+dt}={\overrightarrow{V}}_t+dt(A{\overrightarrow{V}}_t+\theta U_t)+{\epsilon }_t,{\epsilon }_t~\mathcal{N}(0,C_{V}dt).$$ (13)

Here the dynamics matrix A includes both leak terms and intercompartmental coupling effects: for example, in the special case of a linear dendrite segment wih N compartments, with constant leak conductance g and intercompartmental conductance a, A is given by the tridiagonal matrix

$$A=-gI+{aD}^2,$$

with D² denoting the second-difference operator. For simplicity, we assume that U_t is a known signal:

$$U_t=h(t)\ast \sum _i\delta (t-t_i);$$

h(t) is a known synaptic post-synaptic (PSC) current shape (e.g., an α-function [50]), * denotes convolution, and Σ_i δ(t − t_i) denotes the presynaptic spike train. The weight vector θ is the unknown parameter we want to infer: θ_i is the synaptic weight at the i-th dendritic compartment. Thus, to summarize, we have assumed that each synapse fires deterministically, with a known PSC shape (only the magnitude is unknown) at a known latency, with no synaptic depression or facilitation. (All of these assumptions may be relaxed significantly [73].)

Now we would like to estimate the synaptic weight vector θ, given U and noisy observations of the spatiotemporal voltage V. V is not observed directly here, and therefore plays the role of our hidden variable Q. For concreteness, we further assume that the observations Y are approximately linear and Gaussian:

$$y_t=B{\overrightarrow{V}}_t+{\eta }_t,{\eta }_t~\mathcal{N}(0,C_y).$$

In this case the voltage V and observed data Y are jointly Gaussian given U and the parameter θ, and furthermore V depends on θ linearly, so estimating θ can be seen as a rather standard linear-Gaussian estimation problem. There are many ways to solve this problem: we could, for example, use EM to alternatively estimate V given θ and Y, then estimate θ given our estimated V, alternating these maximizations until convergence. However, a more efficient approach (and one which generalizes nicely to the nonlinear case [51]) is to optimize equation (12) directly. Note that this Laplace approximation is in fact exact in this case, since the posterior p(Y|θ) is Gaussian. Furthermore, the log-determinant term log | − H_θ| is constant in θ (since the Hessian is constant in this Gaussian model), and so we can drop this term from the optimization. Thus we are left with

$$\begin{array}{l}arg\underset{\theta }{max}logp(Y\mid \theta )=arg\underset{\theta }{max}\{logp(Y\mid {\widehat{V}}_{\theta },\theta )+logp({\widehat{V}}_{\theta }\mid \theta )\}\\ =arg\underset{\theta }{max}\{logp(Y\mid {\widehat{V}}_{\theta })+logp({\widehat{V}}_{\theta }\mid \theta )\}\\ =arg\underset{\theta }{max}\underset{V}{max}\{logp(Y\mid V)+logp(V\mid \theta )\};\end{array}$$ (14)

i.e., optimization of the marginal likelihood p(Y|θ) reduces here to joint optimization of the function log p(Y|V)+log p(V|θ) in (V, θ). Since this function is jointly quadratic and negative semidefinite in (V, θ), we need only apply a single step of Newton’s method. Now if we examine the Hessian H of this objective function, we see that it is of block form:

$$H=\nabla {\nabla }_{(V,\theta )}[logp(Y\mid V)+logp(V\mid \theta )]=\left(\begin{array}{cc}{H}_{VV}& H_{V\theta }^T\\ H_{\theta V}& H_{\theta \theta }\end{array}\right),$$

here the H_VV block is itself block-tridiagonal, with T blocks of size n (where n is the number of compartments in our observed dendrite) and the H_θθ block is of size n × n. If we apply the Schur complement to this block H and then exploit the fast methods for solving linear equations involving H_VV, it is easy to see that solving for θ̂ can be done in a single O(T) step; see [51] for details.

Figure 4 shows a simulated example of this inferred θ in which U⃗ (t) is chosen to be two-dimensional (corresponding to inputs from two presynaptic cells, one excitatory and one inhibitory); given only half a second of intermittently-sampled, noisy data, the posterior p(θ|Y) is quite accurately concentrated about the true underlying value of θ.

Figure 4.

Estimating the location and strength of synapses on a simulated dendritic branch. Left: simulation schematic. By observing a noisy, subsampled spatiotemporal voltage signal on the dendritic tree, we can infer the strength of a given presynaptic cell’s inputs at each location on the postsynaptic cell’s dendritic tree. Right: illustration of the method applied to simulated data [73]. We simulated two presynapic inputs: one excitatory (red) and one inhibitory (blue). The (known) presynaptic spike trains are illustrated in the top panel, convolved with exponential filters of τ = 3 ms (excitatory) and τ = 2 ms (inhibitory) to form U_t. Second panel: True (unobserved) voltage (mV), generated from the cable equation (13). Note that each presynaptic spike leads to a post-synaptic potential of rather small magnitude (at most ≈ 1 mV), relative to the voltage noise level. In this case the excitatory presynaptic neuron synapses twice on the neuron, on compartment 12 and and a smaller synapse on compartment 8, while the inhibitory neuron synapses on compartment 3. Third panel: Observed (raster-scanned) voltage. The true voltage was not observed directly; instead, we only observed a noisy, spatially-rastered (linescanned), subsampled version of this signal. Note the very low effective SNR here. Bottom panel: True and inferred synaptic weights. The true weight of each synapse is indicated by an asterisk (*) and the errorbar shows the posterior mean E(θ_i|Y) and standard deviation $\sqrt{\mathit{Var}({\theta }_i\mid Y)}$ of the synaptic weight given the observed data. Note that inference is quite accurate, despite the noisiness and the brief duration (500 ms) of the data.

The form of the joint optimization in equation (14) sheds a good deal of light on the relatively slow behavior of the EM algorithm here. The E step here involves inferring V given Y and θ; in this Gaussian model, the mean and mode of the posterior p(V|θ, Y) are equal, and so we see that

$$\widehat{V}=arg\underset{V}{max}logp(V\mid \theta ,Y)=arg\underset{V}{max}\{logp(Y\mid V)+logp(V\mid \theta )\}.$$

Similarly, in the M-step, we compute arg max_θ log p(V̂|θ). So we see that EM here is simply coordinate ascent on the objective function in equation (14): the E step ascends in the V direction, and the M step ascends in the θ direction. (In fact, it is well-known that EM may be written as a coordinate ascent algorithm much more generally; see [62] for details.) Coordinate ascent requires more steps than Newton’s method in general, due to the “zigzag” nature of the solution path in the coordinate ascent algorithm [78], and this is exactly our experience in practice⁷.

As emphasized above, the linear-Gaussian case we have treated here is special, because the Hessian H is constant, and therefore the log |H| term in equation (12) can be neglected. However, in some cases we can apply a similar method even when the observations are non-Gaussian; see [51, 104] for examples and further details.

4 Generalizing the state-space method: exploiting banded and sparse Hessian matrices

Above we have discussed a variety of applications of the direct O(T) optimization idea. It is natural to ask whether we can generalize this method usefully beyond the state space setting. Let’s look more closely at the assumptions we have been exploiting so far. We have restricted our attention to problems where the log-posterior p(Q|Y) is log-concave, with a Hessian H such that the solution of the linear equation ∇ = HQ can be computed much more quickly than the standard O(dim(Q)³) required by a generic linear equation. In this section we discuss a couple of examples that do not fit gracefully in the state-space framework, but where nonetheless we can solve ∇ = HQ quickly, and therefore very efficient inference methods are available.

4.1 Example: banded matrices and fast optimal stimulus decoding

The neural decoding problem is a fundamental question in computational neuroscience [83]: given the observed spike trains of a population of cells whose responses are related to the state of some behaviorally-relevant signal x(t), how can we estimate, or “decode,” x(t)? Solving this problem experimentally is of basic importance both for our understanding of neural coding [77] and for the design of neural prosthetic devices [22]. Accordingly, a rather large literature now exists on developing and applying decoding methods to spike train data, both in single cell- and population recordings; see [76, 1] for a recent review.

We focus our attention here on a specific example. Let’s assume that the stimulus x(t) is one-dimensional, with a jointly log-concave prior p(X), and that the Hessian of this log-prior is banded at every point X. Let’s also assume that the observed neural population whose responses we are attempting to decode may be well-modeled by the generalized linear model framework applied in [77]:

$${\lambda }_i(t)=exp\left({\mathbf{k}}_i\ast x(t)+{\mathbf{h}}_i·{\mathbf{y}}_i+\left(\sum _{i\ne j}{\mathbf{l}}_{ij}·{\mathbf{y}}_j\right)+{\mu }_i\right).$$

Here * denotes the temporal convolution of the filter k_i against the stimulus x(t). This model is equivalent to equation (7), but we have dropped the common-input term q(t) for simplicity here.

Now it is easy to see that the loglikelihood log p(Y|X) is concave, with a banded Hessian, with bandwidth equal to the length of the longest stimulus filter k_i [76]. Therefore, Newton’s method applied to the log-posterior log p(X|Y) requires just O(T) time, and optimal Bayesian decoding here runs in time comparable to standard linear decoding [109]. Thus we see that this stimulus decoding problem is a rather straightforward extension of the methods we have discussed above: instead of dealing with block-tridiagonal Hessians, we are now simply exploiting the slightly more general case of a banded Hessian.

The ability to quickly decode the input stimulus x(t) leads to some interesting applications. For example, we can perform perturbation analyses with the decoder: by jittering the observed spike times (or adding or removing spikes) and quantifying how sensitive the decoded stimulus is to these perturbations, we may gain insight into the importance of precise spike timing and correlations in this multineuronal spike train data [2]. Such analyses would be formidably slow with a less computationally-efficient decoder.

See [1] for further applications to fully-Bayesian Markov chain Monte Carlo (MCMC) decoding methods; bandedness can be exploited quite fruitfully for the design of fast preconditioned Langevin-type algorithms [84]. It is also worth noting that very similar banded matrix computations arise naturally in spline models [107, 34], which are at the heart of the powerful BARS method for neural smoothing [20, 44]; see [1] for a discussion of how to exploit bandedness in the BARS context.

4.2 Smoothness regularization and fast estimation of spatial tuning fields

The applications discussed above all involve state-space models which evolve through time. However, these ideas are also quite useful in the context of spatial models [60]. Imagine we would like to estimate some two-dimensional rate function from point process observations. [81] discuss a number of distinct cases of this problem, including the estimation of place fields in hippocampus [9] or of tuning functions in motor cortex [72]; for concreteness, we focus here on the setting considered in [17]. These authors analyzed repeated observations of a spike train whose mean rate function changed gradually from trial to trial; the goal of the analysis here is to infer the firing rate λ(t, i), where t denotes the time within a trial and i denotes the trial number.

One convenient approach to this problem is to model λ(t, i) as

$$\lambda (t,i)=f[q(t,i)],$$

and then to discretize (t, i) into two-dimensional bins in which q(t, i) may be estimated by maximum likelihood in each bin. However, as emphasized in [44, 96, 45, 17], this crude histogram approach can lead to highly variable, unreliable estimates of the firing rate λ(t, i) if the histogram bins are taken to be too small; conversely, if the bins are too large, then we may overcoarsen our estimates and lose valuable information about rapid changes in the firing rate as a function of time t or trial number i.

A better approach is to use a fine binwidth to estimate λ, but to regularize our estimate so that our inferred λ̂ is not overly noisy. One simple approach is to compute a penalized maximum likelihood estimate

$$\widehat{Q}=arg\underset{Q}{max}\left[logp(Y\mid Q)-c_1\sum _{it}{[q(t,i)-q(t-dt,i)]}^2-c_2\sum _{it}{[q(t,i)-q(t,i-1)]}^2\right];$$ (15)

the observation likelihood p(Y|Q) is given by the standard point-process log likelihood

$$logp(Y\mid Q)=\sum _{it}{y}_{it}logf[q(t,i)]-f[q(t,i)]dt,$$

where dt denotes the temporal binwidth; y_it here denotes the number of spikes observed in time bin t during trial i (c.f. equation (8)). The constants c₁ and c₂ serve as regularizer weights: if c₁ is large, then we penalize strongly for fluctuations in q(t, i) along the t-axis, whereas conversely c₂ sets the smoothness along the i-axis.

This quadratic penalty has a natural Bayesian interpretation: Q̂ is the MAP estimate under a “smoothing” Gaussian prior of the form

$$logp(Q)=-c_1\sum _{it}{[q(t,i)-q(t-dt,i)]}^2-c_2\sum _{it}{[q(t,i)-q(t,i-1)]}^2+\mathit{const}.$$ (16)

(Note that this specific Gaussian prior is improper, since the quadratic form in equation (16) does not have full rank — the sums in log p(Q) evaluate to zero for any constant Q — and the prior is therefore not integrable. This can be corrected by adding an additional penalty term, but in practice, given enough data, the posterior is always integrable and therefore this improper prior does not pose any serious difficulty.)

Now the key problem is to compute Q̂ efficiently. We proceed as before and simply apply Newton’s method to optimize equation (15). If we represent the unknown Q as a long vector formed by appending the columns of the matrix q(t, i), then the Hessian with respect to Q still has a block-tridiagonal form, but now the size of the blocks scales with the number of trials observed, and so direct Gaussian elimination scales more slowly than O(N), where N is the dimensionality (i.e., the total number of pixels) of q(t, i). Nonetheless, efficient methods have been developed to handle this type of sparse, banded matrix (which arises, for example, in applied physics applications requiring discretized implementations of Laplace’s or Poisson’s equation); for example, in this case Matlab’s built in H\∇ code computes the solution to Hx = ∇ in O(N^3/2) time, which makes estimation of spatial tuning functions with N ~ 10⁴ easily feasible (on the order of seconds on a laptop). See Fig. 6 for an example of an estimated two-dimensional firing rate λ(t, i), and [81] for further details.

Figure 6.

An example of the fast spatial estimation method applied to data from [17], Fig. 3. Top panel: observed spike train data. Note that the firing rate qualitatively changes both as a function of time t and trial number i; 50 trials total are observed here. Bottom: λ(t, i) estimated using the fast regularized methods described in [81]. See [81] for further comparisons, e.g. to linear kernel smoothing methods.

5 Conclusion

Since the groundbreaking work of [9], state-space methods have been recognized as a key paradigm in neuroscience data analysis. These methods have been particularly dominant in the context of online decoding analyses [9, 8, 102, 93, 112, 100, 53] and in the analysis of plasticity and nonstationary tuning properties [11, 30, 25, 97, 17, 54, 81], where the need for statistically rigorous and computationally efficient methods for tracking a dynamic “moving target” given noisy, indirect spike train observations has been particularly acute.

The direct optimization viewpoint discussed here (and previously in [28, 29, 5, 18, 42, 51]) opens up a number of additional interesting applications in neuroscience, and has a couple advantages, as we have emphasized. Pragmatically, this method is perhaps a bit conceptually simpler and easier to code, thanks to the efficient sparse matrix methods built into Matlab and other modern numerical software packages. The joint optimization approach makes the important extension to problems of constrained optimization quite transparent, as we saw in section 2.4. We also saw that the direct techniques outlined in section 3 can provide a much more efficient algorithm for parameter estimation than the standard Expectation-Maximization strategy. In addition, the direct optimization approach makes the connections to other standard statistical methods (spline smoothing, penalized maximum likelihood, isotonic regression, etc.) quite clear, and can also serve as a quick initialization for more computationally-intensive methods that might require fewer model assumptions (e.g., on the concavity of the loglikelihoods p(y_t|q_t)). Finally, this direct optimization setting may be generalized significantly: we have mentioned extensions of the basic idea to constrained problems, MCMC methods, and spatial smoothing applications, all of which amply illustrate the flexibility of this approach. We anticipate that these state-space techniques will continue to develop in the near future, with widespread and diverse applications to the analysis of neural data.