TIME-VARYING ℓ₀ OPTIMIZATION FOR SPIKE INFERENCE FROM MULTI-TRIAL CALCIUM RECORDINGS

Abstract

Optical imaging of genetically encoded calcium indicators is a powerful tool to record the activity of a large number of neurons simultaneously over a long period of time from freely behaving animals. However, determining the exact time at which a neuron spikes and estimating the underlying firing rate from calcium fluorescence data remains challenging, especially for calcium imaging data obtained from a longitudinal study. We propose a multi-trial time-varying ${\ell }_0$ penalized method to jointly detect spikes and estimate firing rates by robustly integrating evolving neural dynamics across trials. Our simulation study shows that the proposed method performs well in both spike detection and firing rate estimation. We demonstrate the usefulness of our method on calcium fluorescence trace data from two studies, with the first study showing differential firing rate functions between two behaviors and the second study showing evolving firing rate functions across trials due to learning.

1. Introduction.

The brain is the most complicated organ in a vertebrate’s body. Consequently, despite the tremendous efforts to study the brain, our knowledge is still far from complete. A variety of techniques have been developed to measure brain activity. For example, the two most common non-invasive methods for measuring human brain activities are functional magnetic resonance imaging (fMRI) and electroencephalography (EEG). fMRI technology is an imaging method that measures brain activity by detecting the blood oxygenation level dependent (BOLD) signal, as blood flow increases with increased neuronal activity. Although fMRI data has excellent spatial resolution, its poor temporal resolution is a limitation in studying the temporal dynamics of neuronal activity. In comparison, EEG has poor spatial but excellent temporal resolution and can capture the electrical activity on the cortex using multiple electrodes attached to the scalp.

The measurement methods in animal brains are typically invasive and able to provide greater details. For example, single-unit recordings measure the current flowing in and out of a neuron, thus allowing for the precise recording of individual action potentials (also known as “spikes”), the electrical events through which neurons communicate with each other. The neural signals can also be indirectly measured by quantifying the influx of Ca²⁺ ions that accompany the electrical activities of a neuron. In the past few years, calcium imaging in freely behaving animals, named “the Method of the Year 2018” by Nature Methods (Editorial, 2019), has been increasingly adopted in neuroscience research of animals since it allows simultaneous measurement of the activity of a large population of neurons at the single-neuron resolution over weeks using optical imaging of living animals. For example, Kappel et al. (2022) imaged neural activities of the visual areas of zebrafish to understand neuronal selectivity, and Yadav et al. (2022) recorded neuron activities in the hippocampus when mice performed a behavioral task. More importantly, because the activity of individual neurons can now be tracked over a long period of time (e.g., many days or weeks), it is now possible to understand the dynamic reorganization of neurons over time. In our recent study, we conducted a longitudinal investigation of the neural ensemble dynamics of contextual discrimination by recording mice’s calcium imaging in the hippocampus for about 60 days (Johnston et al., 2022) using appropriate genetically encoded calcium indicators (Tian et al., 2009) and miniature fluorescence microscopes (Ghosh et al., 2011). As detailed in Section 3.2.2, the longitudinal recording over a large number of days allows researchers to understand the neuronal and behavioral dynamics and how neuronal activities change over time.

While the technical advancements bring great flexibility to neuroscience research, they also create significant challenges to every aspect of data analysis - from storing the large amount of video recordings to downstream statistical modeling and inference (Pnevmatikakis, 2019). In longitudinal recordings of freely behavinga nimals, after motion correction and registration of detected neurons across multiple sessions (Giovannucci et al., 2019), fluorescence traces of individual neurons can be extracted, for example, using independent component analysis (Mukamel, Nimmerjahn and Schnitzer, 2009) or non-negative matrix factorization methods (Maruyama et al., 2014; Pnevmatikakis et al., 2016; Zhou et al., 2018). The output of the pre-processing steps is the fluorescence trace, i.e., a time series of fluorescence intensities, for each neuron during each experimental trial, which lasts for a few seconds to minutes. For example, the top panel of Figure 1 visualizes a simulated fluorescence trace. This trace is a noisy proxy, rather than a direct observation of the underlying spike train (shown as the black vertical bars), i.e., a temporal sequence of action potentials (spikes). The quantitative but noisy fluorescence traces are frequently used for further statistical analysis including visualization (heat maps of the fluorescence); clustering of neurons that share similar neural activity profiles; comparison of neural activity levels between various experimental conditions; and studying neural encoding and decoding. For many important questions, such as those involving the analysis of the precise timing of neural activity in response to stimuli, it is essential to estimate the underlying spike train from a noisy fluorescence trace. As reported in our recent work (Shen et al., 2022), the estimated spike data often provide more an accurate estimation of neuron clusters and earlier latency, defined as the time reaching 70% decoding accuracy. Therefore, one critical step is to estimate the underlying spike data from calcium traces.

Figure 1:

An example trial from the static firing rate simulation. Top: traces and spike times. Bottom: firing rate functions. Black: the true firing function or spike times; Red: estimates based on constant penalty; Green: estimates based on time-varying penalty estimated from one trial; Blue: estimates based on time-varying penalty estimated from all trials.

Several approaches have been developed including linear deconvolution (Yaksi and Friedrich, 2006) or nonnegative convolutions (Vogelstein et al., 2010). Fully Bayesian methods have also been developed to obtain posterior distributions of spike trains, thus allowing uncertainty quantification of the estimates of spikes (Pnevmatikakis et al., 2013). In Theis et al. (2016), a supervised learning approach is proposed based on a probabilistic relationship between fluorescence and spikes. This algorithm is trained on data where spike times are known. Research has shown that nonnegative deconvolution outperforms supervised algorithms and is more robust to the shape of calcium fluorescence responses (Pachitariu, Stringer and Harris, 2018). In Jewell and Witten (2018), upon which our proposed method builds, an autoregressive model with ${\ell }_0$ optimization to was proposed to analyze calcium imaging data. This approach detects spikes by framing the problem as a changepoint detection task and solving it using a dynamic programming algorithm, which significantly improves computational efficiency and spike detection accuracy. In their follow-up paper (Jewell et al., 2020), they further enhanced the algorithm by introducing functional pruning techniques, which further reduce the computational complexity, making the method feasible for large-scale data sets. However, one limitation of these methods is that these spike detection methods analyze only one trial at a time; thus, shared information across trials is largely ignored and hence not efficient. In a longitudinal study, neural activities are measured in multiple trials or sessions. In these settings, aggregating information across trials could increase the accuracy of spike detection. In this paper, we will develop a novel method that overcomes these limitations by combining information across trials in a longitudinal study.

There has been little work on how to utilize the information from multiple trials to improve accuracy in spike detection. Among the few multi-trial methods we identified, Picardo et al. (2016) assumed that repeated trials share the same burst time but have trial-specific magnitude, baseline fluorescence, and noise level. Conceptually, integrating multiple trials should be beneficial if the trial-to-trial variation is mainly due to randomness. In reality, however, as pointed out in Deneux et al. (2016), the gain by naively combining trials may be limited - it is likely to bring improvement for some trials but might perform worse in others when the neural activity in some trials does not follow the marginal pattern across all trials, or even produce misleading results when these recordings are not properly temporally aligned (i.e., non-constant burst time). During a learning process or when adjusting to a new environment, neurons constantly reorganize and show plasticity, leading to varying neural dynamics across trials. Recently, D’Angelo et al. (2022) proposed a Bayesian multi-trial method on using neuronal activities measured by calcium imaging data to predict the class of image inputs.

In this paper, we develop a robust multi-trial spike inference method to address the challenges in analyzing longitudinal calcium imaging data. Our approach both integrates the commonality and accounts for evolving neural dynamics across trials. Efficiency is achieved by aggregating information from temporally adjacent trials into a dynamic ${\ell }_0$ penalization framework, whereas robustness is guaranteed by using unique parameters for each trial.

2. Methods.

In this section, we describe our proposed multi-trial time-varying penalized auto-regression (MTV-PAR) method. We first introduce a time-varying ${\ell }_0$ penalized framework based on a pre-specified penalization function (Section 2.1). Next, we detail the procedure for estimating a firing rate function and use it to obtain a reasonable penalization function (Section 2.2). We then provide details on how to jointly make inference of spikes and firing rate by alternating a spike detection step and a firing rate estimation step (Section 2.3). Finally, we explain how the smoothing parameters in estimating firing rate function and sparsity penalty in detecting spikes are chosen in Section 2.4.

2.1. Spike detection using a time-varying ${\ell }_0$ penalized auto-regressive model.

2.1.1. Time-varying ${\ell }_0$ penalized auto-regressive model.

Let $y_r\left(t\right)$ be the fluorescence recorded at time point $t$ of trial $r$, where $t=1,\dots T$, $r=1,\cdots ,R$. In practice, multiple pre-processing steps, including normalization, (Vogelstein et al., 2010), are typically implemented to obtain $y_r\left(t\right)$. The calcium fluorescence trace $y_t\left(t\right)$, $t=1,\cdots ,T$ is often modeled using the following first-order auto-regressive model

$$\begin{array}{l}{y}_r\left(t\right)=c_r\left(t\right)+{ϵ}_r\left(t\right),{ϵ}_r\left(t\right)\sim N\left(0,{\sigma }_r^2\right),\\ c_r\left(t\right)={\gamma }_r{c}_r\left(t-1\right)+s_r\left(t\right),\text{s}.\text{b}.\text{t}.s_r\left(t\right)\ge 0\end{array}$$ (1)

where $c_r\left(t\right)$ and ${ϵ}_r\left(t\right)$ denote the underlying calcium concentration and noise at time $t$ in the $r$th trial, respectively. Thus, the first equation in the model describes the random deviation due to noise of the observed fluorescence $y_r\left(t\right)$ from the underlying theoretical calcium concentration $c_r\left(t\right)$ at time $t$ in the $r$th trial. The second equation makes two assumptions. First, in the absence of an action potential (neural firing/spike), i.e., $s_r\left(t\right)=0$, the calcium concentration decays exponentially with a decay rate of $0<{\gamma }_r<1$. Note that this decay rate parameter depends on multiple factors such as the sampling frame, the cell types, and the kinetics of genetically encoded indicators. Second, there is a sudden increase in calcium concentration by an amount of $s_r\left(t\right)>0$ when there is at least one neuronal spike at time $t$. Note that $s_r\left(t\right)$ is a function of time, and there might be multiple neural spikes within a time frame. The main goal of spike detection to is locate the time points with a positive $s_r\left(t\right)$.

Here, we propose to regularize the inference on spike detection by introducing a time-varying penalty function ${\lambda }_r\left(t\right)$ on the number of spikes, as the spikes tend to be sparse but not evenly distributed over time:

$$\underset{c_r\left(1\right),\dots ,c_r\left(T\right)}{\min }\left\{\frac{1}{2}\sum _{t=1}^T{\left(y_r\left(t\right)-c_r\left(t\right)\right)}^2+\sum _{t=2}^T{\lambda }_r\left(t\right){\mathrm{𝟙}}_{s_r\left(t\right)>0}\right\}.$$ (2)

Note that the penalty function ${\lambda }_r\left(t\right)$ is unknown. We provide a procedure to estimate ${\lambda }_r\left(t\right)$ in Section 2.2. Essentially, ${\lambda }_r\left(t\right)$ will be specified to be a decreasing function of the instantaneous firing rate $f_r\left(t\right)$, which is smooth over time within trials and changes slowly over trials $r$, $r=1,\cdots ,R$. The proposed approach provides a balance of efficiency and robustness because the parameters in the AR(1) model are trial-specific (in order to capture the trial-to-trial variation in fluorescence signal parameters) and the firing rate functions across trials are assumed to change slowly to capture the evolving neural dynamics.

2.1.2. A dynamic programming algorithm for time-varying ${\ell }_0$ penalized auto-regressive model.

In our proposed time-varying ${\ell }_0$ penalized auto-regressive model, a calcium fluorescence trace at the $r$th trial $y_r\left(t\right)$, $t=1,\cdots ,T$, is modeled by a first-order auto-regressive model, as given in Equation (1). As previously stated, $c_r\left(t\right)$ denotes the underlying calcium concentration and a positive value of $s_r\left(t\right)$ implies that a spike occurs at time $t$. Since spikes tend to be sparse, ideally, one should penalize the number of spikes by introducing an ${\ell }_0$ penalty. However, the ${\ell }_0$ penalization is computationally intractable; therefore, existing methods often impose an ${\ell }_1$ penalization (Vogelstein et al., 2010; Friedrich and Paninski, 2016; Friedrich, Zhou and Paninski, 2017). Recently, Jewell and Witten (2018) found that, for spike detection, an ${\ell }_0$ penalty brings substantial improvements over an ${\ell }_1$ penalty. They also showed that relaxing the nonnegative constraint of $s_r\left(t\right)$ has a negligible effect on the results; more importantly, the corresponding ${\ell }_0$ optimization problem is equivalent to a changepoint detection problem whose solution can be obtained efficiently using a dynamic programming algorithm. Using a similar strategy, we prove that the following time-varying penalization problem can also be solved by a dynamic programming algorithm:

$$\underset{c_r\left(1\right),\dots ,c_r\left(T\right)}{\min }\left\{\frac{1}{2}\sum _{t=1}^T{\left(y_r\left(t\right)-c_r\left(t\right)\right)}^2+\sum _{t=2}^T{\lambda }_r\left(t\right){\mathrm{𝟙}}_{s_r\left(t\right)\ne 0}\right\}.$$ (3)

Specifically, we find that the time-varying ${\ell }_0$ optimization problem is equivalent to the following changepoint problem whose solution can be efficiently identified using a dynamic programming algorithm. For ease of presentation, we drop the trial index $r$ and focus on the fluorescence trace of a cell at a given trial. The equivalent changepoint problem can be shown as (See Appendix A.1 for proof):

$$\underset{0={\tau }_0<{\tau }_1<\dots <{\tau }_k<{\tau }_{k+1}=T,k}{\min }\left\{\sum _{j=0}^k\left[𝒟\left(y_{\left({\tau }_j+1\right):{\tau }_{j+1}}\right)+\lambda \left({\tau }_j\right)\right]\right\},$$ (4)

where ${\tau }_1,\dots {\tau }_k$ are $k$ changepoints, i.e., the points satisfying $c\left(t\right)-\gamma c\left(t-1\right)\ne 0$ and

$$𝒟\left(y_{a:b}\right)=\underset{\begin{array}{c}c\left(a\right),c\left(t\right)=\gamma c\left(t-1\right)\\ t=a+1,\dots ,b\end{array}}{\min }\left\{\frac{1}{2}\sum _{t=a}^b{\left(y\left(t\right)-c\left(t\right)\right)}^2\right\},$$ (5)

which leads to the following closed-form solution:

$$𝒟\left(y_{a:b}\right)=\frac{1}{2}\sum _{t=a}^b{\left(y\left(t\right)-{\gamma }^{t-a}\frac{{\sum }_{t=a}^{b}y\left(t\right){\gamma }^{t-a}}{{\sum }_{t=a}^b{\gamma }^{2\left(t-a\right)}}\right)}^2.$$ (6)

Note that the parameter $\gamma$, which measures the speed at which the calcium concentration decays, is not estimated. The value of $\gamma$ is usually close to 1 (Vogelstein et al., 2010; Yaksi and Friedrich, 2006), as a somatic calcium transient caused by an action potential is often characterized by an almost instantaneous rise but a slow decay. For computational feasibility, rather than estimating $\gamma$ iteratively, similar to (Pnevmatikakis et al., 2016; Friedrich, Zhou and Paninski, 2017), we set the estimate to be the empirical auto-correlation at lag 1. Finally, as shown in Appendix A.2, the optimization problem (3) can be solved by computing $F\left(t\right)$ recursively:

$$F\left(t\right)=\underset{0\le \tau <t}{\text{min}}\left\{F\left(\tau \right)+𝒟\left(y_{\left(\tau +1\right):t}\right)+\lambda \left(\tau \right)\right\},\text{for}t=1,2,\cdots ,T.$$ (7)

The equation (7) is a vanilla dynamic programming version with a time complexity of $O\left(T^2\right)$ (Jackson et al., 2005). Although previous study has demonstrated that the computational cost can be reduced from quadratic to linear time complexity using a pruned dynamic programming mechanism (Killick, Fearnhead and Eckley, 2012), this improvement is only applicable when the penalty function is a constant. Therefore, we directly employ the vanilla dynamic programming method (Jewell and Witten, 2018), as shown in Algorithm 1. We implement this algorithm in Rcpp so that the computational speed can be improved drastically.

$$\underset{¯}{\overline{\begin{array}{l}\mathbf{\text{Algorithm}}\mathbf{1}\text{Dynamic}\text{programming}\text{algorithm}\text{to}\text{detect}\text{spikes}\text{with}\text{a}\text{time-varying}\text{penalization}\\ \underset{¯}{\text{function}}\\ 1:\mathbf{\text{Input}}\mathbf{:}\text{Time-varying}\text{penalty}\text{function}\lambda \left(t\right),\text{single-trial}\text{fluorescence}y\left(t\right)\text{with}T\text{time}\text{points}.\\ 2:\mathbf{\text{Initialize}}:F\left(0\right)=-{\lambda }_0,spikeset=\varnothing \\ 3:\mathbf{\text{for}}t=1,2,\dots T\mathbf{\text{do}}\\ 4:\text{Calculate}F\left(t\right)=\underset{0\le \tau <t}{\text{min}}\left\{F\left(\tau \right)+\mathcal{D}\left(y_{\left(\tau +1\right):t}\right)+\lambda \left(\tau \right)\right\}\\ 5:\text{Find}{t}^{\ast }=\underset{0\le \tau <t}{\text{argmin}}\left\{F\left(\tau \right)+\mathcal{D}\left(y_{\left(\tau +1\right):t}\right)+\lambda \left(\tau \right)\right\}\\ 6:\mathbf{\text{if}}{t}^{\ast }\notin spikeset\mathbf{\text{then}}\\ 7:Add{t}^{\ast }tospikeset\\ 8:\mathbf{\text{end}}\mathbf{\text{if}}\\ 9:\mathbf{\text{end}}\mathbf{\text{for}}\\ 10:\text{Return}spikeset\end{array}}}$$

2.2. Firing rate estimation and time-varying penalty.

2.2.1. Estimating the firing rate function based on multi-trial spike data.

Estimating the firing rate function is a crucial task in the analysis of spike train data (Cunningham et al., 2009). One commonly used descriptive approach involves building the peristimulus time histogram (PSTH), where the spike counts are averaged from multiple trials within each time bin (Gerstein and Kiang, 1960). Kernel methods are often applied to achieve smoothness (Cunningham et al., 2008). Bayesian methods have also been considered, e.g. by proposing Gaussian processes (Cunningham et al., 2008; Shahbaba, Behseta and Vandenberg-Rodes, 2015) and Bayesian Adaptive Regression Splines (DiMatteo, Genovese and Kass, 2001; Olson et al., 2000; Kass, Ventura and Cai, 2003; Kass, Ventura and Brown, 2005; Behseta and Kass, 2005).

The PSTH approach and the other smoothing methods assume that the underlying firing rate function does not change over trials - which is a serious limitation. The estimated firing rates are then compared between different groups to capture the association between firing rate and animal behavior (Jog et al., 1999; Wise and Murray, 1999; Wirth et al., 2003). However, recent evidence suggests the need to move beyond the independent and identically distributed trial assumption and regard both neural and behavioral dynamics as smooth and continuous (Huk, Bonnen and He, 2018; Ombao et al., 2018; Fiecas and Ombao, 2016). Thus, it is essential to model the between-trial dynamics. To account for the between-trial dynamics, Czanner et al. (2008); Paninski et al. (2010) proposed a state-space model. In the state-space framework, the spike train is characterized by a point process model (Brown et al., 2003; Brown, 2005; Kass, Ventura and Brown, 2005; Daley and Vere-Jones, 2007) for the underlying fire rate. In this paper, to make it feasible to jointly estimate the spike trains and the firing rate function from the observed multi-trial fluorescence data, we consider instead a computationally less demanding two-dimensional Gaussian-boxcar kernel smoothing function $G\left(r,t\right)$, which is formulated as follows

$$G\left(r,t\right)=\frac{1}{\sqrt{2\pi \sigma }}\exp \left(-\frac{t^2}{2{\sigma }^2}\right)I\left(\left|r\right|<B/2\right)$$ (8)

where $\sigma$ denotes the within-trial kernel bandwidth in the Gaussian kernel, $I\left(\cdot \right)$ is the indicator function, and $B$ is bandwidth for the between-trial sliding windows in the boxcar kernel. The Gaussian kernel is chosen for its properties of smoothness and rapid decay, which are suitable for capturing the temporal dynamics within each trial. The boxcar kernel is used to integrate information across trials within a local neighborhood, ensuring that the firing rate estimates adapt smoothly over trials. Let $f_r\left(t\right)$ denote the firing rate, i.e., the expected number of spikes per second, at time $t$ in trial $r$. Our estimate of $f_r\left(t\right)$ is given by

$${\widehat{f}}_r\left(t\right)\propto \sum _{r\prime ,t\prime }G\left(r-r\prime ,t-t\prime \right)p_{r\prime }\left(t\prime \right),$$

where $p_{r\prime }\left(t\prime \right)$ is the estimated number of spikes per second in a small time bin centered at time $t\prime$ in trial $r\prime$.

While the Gaussian kernel is a proper density function, the combined kernel $G\left(r,t\right)$ does not integrate to 1 over the entire domain because it is intended to apply a localized smoothing effect rather than representing a probability density function. The primary goal is to achieve a balance between smoothing the data within trials and across trials without the necessity for $G\left(r,t\right)$ to be a proper density function.

2.2.2. Time-varying penalty.

Choosing the appropriate type of regularization plays a critical role in studying brain connectivity (Monti et al., 2017; Hu, Fortin and Ombao, 2019; Gao et al., 2021; Ting et al., 2021). In spike detection, sparsity has been enforced via penalization or introducing appropriate prior distributions. Existing approaches usually adopt a tuning parameter uniformly for the whole time series of a fluorescence trace. Within a trial, the firing rate right after each stimulus is expected to be higher than baseline, which is defined as a short period right before the presence of a cue or stimulus. Thus, using a constant penalty may not be optimal. Our proposed non-constant penalization is inspired by prior work in the literature. For example, time-varying penalization was used for analyzing multivariate time series data in Yu et al. (2017) and Zbonakova, Härdle and Wang (2016) studied the dynamics of the penalty term in a Lasso framework for the analysis of interdependences in the stock markets.

Here, we expand on those approaches and consider a decreasing function of the firing rate function for the time-varying penalty, motivated by the fact that spikes are expected to be less frequent in regions with low firing rates than in regions with high firing rates. Ideally, the penalty should be always non-negative and small in locations with higher firing rates. Hence, we use a negative exponential function to achieve adaptive regularization. Specifically, the time-varying function is chosen as

$${\lambda }_r\left(t\right)\propto e^{-a{\widehat{f}}_r\left(t\right)}$$

where ${\widehat{f}}_r\left(t\right)$ is the estimated firing rate (See Algorithm 2). Here the value $a$ controls how much the penalty function should depend on the firing rate function. In particular, $a=0$ reverts to a constant penalty. The choice of a negative exponential function is similar in spirit to imposing a prior distribution on ${\lambda }_r\left(t\right)$, as done in Zeng, Thomas and Lewinger (2021) and the value $a$ can be considered as a hyperparameter. To avoid extremely large or small penalties, we scale the estimated firing rate function of each trial by dividing by its maximum value so that it ranges between 0 and 1. This implies that the default value $a=1$ in our analysis leads to mildly time-varying penalties - within a trial, the penalty at the highest firing rate is about 37% of the penalty at a firing rate value of 0. In our experience with both simulated and real data, the results are not sensitive to the choice of $a$. For example, both $a=0.5$ and $a=1$ give similar results.

In addition, to facilitate comparison with the case of a constant penalty, the following formula is used in each trial to scale the penalty function ${\lambda }_r\left(t\right)$ to have a mean value $\lambda$,

$${\lambda }_r\left(t\right)=\lambda T\times \frac{\exp \left\{-a\frac{{\widehat{f}}_r\left(t\right)}{\max \left({\widehat{f}}_r\left(t\right)\right)}\right\}}{{\sum }_{t\prime =1}^T\exp \left\{-a\frac{{\widehat{f}}_r\left(t\prime \right)}{\max \left({\widehat{f}}_r\left(t\prime \right)\right)}\right\}}.$$ (9)

2.3. Multi-Trial time-varying ${\ell }_0$ penalized auto-regressive model (MTV-PAR) - an iterative algorithm.

Thus far, we have presented our solutions to two problems separately. The first problem is to detect spikes from a single calcium fluorescence trace using a time-varying ${\ell }_0$ penalized approach under the assumption that the time-varying penalty function ${\lambda }_r\left(t\right)$ is already known. The second problem is to use multi-trial spike data to estimate the firing rate function $f_r\left(t\right)$ for $r=1,\cdots ,R$ and $t=1,\cdots ,T$ using a Gaussian-boxcar smoothing method. The firing rate function is then used to obtain a time-varying penalty function. In practice, neither the firing rate functions nor the spike locations are known. We therefore propose to alternate between the spike detection and firing rate estimation steps until the spike indicator function converges to a stable value, as shown in Algorithm 2. Note that, different from the E step of an expectation-maximization algorithm, the spike detection step performs hard assignments rather than calculating conditional probabilities to reduce computational time. Another thing we would like to point out is that the results do not change much after the second iteration. For this reason, although we leave Algorithm 2 as an iterative algorithm, in practice, we only used two iterations.

$$\underset{¯}{\overline{\begin{array}{l}\underset{¯}{\mathbf{\text{Algorithm}}\mathbf{2}\text{Simultaneous}\text{spike}\text{detection}\text{and}\text{firing}\text{rate}\text{estimation}}\\ 1:\mathbf{\text{Input}}\mathbf{:}\text{Multi-trial}\text{fluorescence}\text{data}y{\left(t\right)}^{R\times T}\text{with}R\text{trials}\text{and}T\text{time}\text{points}.\text{Penalty}\text{term}\lambda .\text{Maximum}\\ \text{number}\text{of}\text{iterations}{N}_{\text{max}}.\text{Convergence}\text{tolerance}ϵ.\\ 2:\mathbf{\text{Initialize}}:\text{Penalty}\text{function}{\lambda }_r\left(t\right)=\lambda .\text{Previous}\text{changepoints}{x}_{\text{prev}}=\infty .\text{Iteration}\text{counter}n=0.\\ 3:\text{Set}\text{the}\text{binary}\text{spike}\text{indicator}x{\left(r,t\right)}^{R\times T}=0.\\ 4:\mathbf{\text{while}}‖x\left(r,t\right)-x_{\text{prev}}\left(r,t\right)‖/\left[RT\right]>ϵ\text{and}n<N_{\text{max}}\mathbf{\text{do}}\\ 5:x_{\text{prev}}=x\left(r,t\right).\\ 6:n=n+1.\\ 7:\mathbf{\text{Spike}}\mathbf{\text{detection}}\mathbf{\text{step}}:\\ 8:\mathbf{\text{for}}r=1,2,\dots R\mathbf{\text{do}}\\ 9:\text{Apply}\text{Algorithm}1\text{to}\text{detect}\text{spikes}\text{and}\text{let}spikeset\text{denote}\text{the}\text{times}\text{at}\text{which}\text{spikes}\text{were}\text{detected}.\\ 10:\text{Set}x\left(r,spikeset\right)=1.\\ 11:\mathbf{\text{end}}\mathbf{\text{for}}\\ 12:\mathbf{\text{Firing}}\mathbf{\text{rate}}\mathbf{\text{estimation}}\mathbf{\text{step}}:\\ 13:\text{Estimate}\text{the}\text{firing}\text{rate}{\widehat{f}}_r\left(t\right),\text{for}r=1,\cdots ,R\text{and}t=1,\cdots ,T\text{using}\text{Gaussian-Sliding-Window}\text{kernel}\\ \text{smoothing}.\\ 14:\text{Calculate}\text{the}\text{weight}\text{function}\\ w_r\left(t\right)=e^{-a\frac{{\widehat{f}}_r\left(t\right)}{\max \left({\widehat{f}}_r\left(t\right)\right)}}\\ 15:\text{Update}\text{the}\text{time-varying}\text{penalty}\text{function}{\lambda }_r\left(t\right)=\lambda \cdot T\cdot w_r\left(t\right)/{\sum }_{t\prime }{w}_r\left(t\prime \right)\\ 16:\mathbf{\text{end}}\mathbf{\text{while}}\\ 17:\mathbf{\text{Output}}:\text{Estimated}\text{spikes}x\left(r,t\right)\text{and}\text{firing}\text{rate}\text{functions}{\widehat{f}}_r\left(t\right).\end{array}}}$$

2.4. Practical issues: choosing smoothing parameters for the firing rate function and the sparsity parameter for spike detection.

Spike detection methods often involve multiple tuning parameters. In our proposed time-varying ${\ell }_0$ penalized method, the practical issues are how to choose an ${\ell }_0$ penalty (which determines the sparsity of spike events) and the smoothing parameters in estimating the firing rate functions (which determine the smoothness of the estimated firing rate functions). These tuning parameters are best calibrated using benchmark data. However, benchmark data are often not available, especially for the increasingly popular longitudinally measured calcium imaging data. In the following paragraphs, we summarize common practices and outline our strategy to choose these tuning parameters.

In Bayesian methods, sparse priors have been considered (Vogelstein et al., 2010). While Pnevmatikakis et al. (2016); Friedrich, Zhou and Paninski (2017) proposed to use the sparsest signal (i.e., largest penalty) that explains a certain amount of variation of the observed fluorescence, Pachitariu, Stringer and Harris (2018) found their results are robust to the choice of the penalty parameter and recommend no penalty. One interesting idea is the pseudo cross-validation method of Jewell and Witten (2018), which treats the odd time points as the training set and the even as the testing, or vice versa. We found that a more computationally feasible approach is to choose the minimal mean penalty $\lambda$ (see Section 2.2.2) such that the baseline firing rate estimated from the constant penalty approach is no greater than 0.1 spikes per second. This choice seems to work well in real data, as spikes are often successfully detected in the presence of surges of calcium transient due to neurons’ responses to stimuli.

We also proposed the Gaussian-boxcar method to ensure that the estimated firing function is smooth both within-trial (Gaussian smoothing) and across-trials (boxcar averaging). For PSTH based on spike train data, common choices of the Gaussian bandwidth are around 50–150 ms (Cunningham et al., 2009); Bayesian adaptive methods have also been developed (DiMatteo, Genovese and Kass, 2001; Olson et al., 2000; Kass, Ventura and Cai, 2003; Kass, Ventura and Brown, 2005; Behseta and Kass, 2005). Another factor is the kinetic properties of the calcium indicator used in a calcium imaging study. One major class of indicators is genetically encoded calcium indicators, which are used to detect in vivo neuronal firing activity by measuring intracellular calcium concentration through imaging the fluorescent molecules that bind to calcium ions (Ca²⁺). Ali and Kwan (2019) reviewed the kinetics of calcium transients of several fluorescent indicators, including two commonly used indicators: GCaMP6f has a rise time of 42 ms and decay time of 142 ms, and GCaMP6s has a rise time of 179 ms and decay time of 550 ms; a time-bin of 200 ms is suggested in one of their analyses. For ease in implementation, a “plug-in” technique of Sheather and Jones (1991); Jones, Marron and Sheather (1996) is adopted. Our numerical experiments indicate that this approach provides reasonably better performances than using several bin sizes recommended for PSTH or based on the kinetic properties.

Another important quantity (a “smoothing” parameter) is the window size of the trials, which allows for borrowing information across neighboring trials. In the traditional setup, a few dozen trials are conducted within a short period of time, such as during a single day. Thus, the neural behaviors across trials are expected to be stable, despite the presence of noise (trial-to-trial variability). In Example 1 (Section 3.2.1), the trials were collected within a few days after the animal was trained. Therefore, for estimating the firing function, we stratify the trials by the stimulus and response types and treat all trials under the same type as independent replicates. On the contrary, in the example in Section 3.2.2, 21 trials were recorded during a six-week study. In this example, the independence assumption may not hold anymore. Our approach is supported by Driscoll et al. (2017), which showed that the behaviors of individual neurons might be stable within one single day or a few days but not over weeks. In more general settings, a sensitivity analysis may be required, by varying the window size. As an illustration, we conduct simulation studies in Section 3.1 to examine whether the result using a reasonable or ideal number of window size is substantially different from that using the smallest size, i.e., using only one trial (the current trial).

3. Results.

3.1. Simulation Studies.

In this section, we will use simulation studies to evaluate the performance of our MTV-PAR method and the ${\ell }_0$ approach by Jewell and Witten (2018). Among the many competing methods, the ${\ell }_0$ approach was selected because its performance has been shown to be superior to other methods using both simulated and benchmark data. Our MTV-PAR approach integrates information across trials in a longitudinal study by using a time-varying penalty function, thereby improving spike detection accuracy. In contrast, the ${\ell }_0$ approach by Jewell and Witten (2018) treats trials independently and adopts a constant penalty function. In the following subsections, we will detail the metrics used for quantifying accuracy and the results of simulations conducted under both static and dynamic firing rate functions to evaluate the performance of MTV-PAR.

3.1.1. Metrics for quantifying accuracy.

Here, we will consider two metrics to summarize the performance of our MTV-PAR methods and the ${\ell }_0$ approach by Jewell and Witten (2018). The first metric focuses on spike detection, and the second focuses on the estimation of firing rates.

When comparing estimated spike trains with the ground truth, the Victor-Purpura (VP) distance (Victor and Purpura, 1996, 1997) was implemented, which has been used for comparing spike trains. It is defined as the minimum total cost required to transform one spike train into another using the following three basic operations:

• Insert a spike into a spike train. (Cost = 1)

• Delete a spike. (Cost = 1)

• Shift a spike by an interval $\text{Δ}t$. $\left(\text{Cost}=q\text{Δ}t\right)$ A large $q$ makes the distance more sensitive to fine timing differences. We use the default value $q=1$.

Since the proposed MTV-PAR method gives an estimate of the firing rate function together with the spikes, its performance on firing rate estimation was also evaluated in these simulations. The approach in Jewell and Witten (2018) does not estimate firing rates; hence, for a fair comparison, we apply the same Gaussian-boxcar kernel smoothing in (8) to the spikes estimated using Jewell and Witten (2018) in order to estimate the firing rates. The accuracy of firing rate estimation is calibrated by the ${\ell }_2$ norm (Adams, Murray and MacKay, 2009) of the difference between an estimated firing rate function and the true function:

$${‖f\left(t\right)-\widehat{f}\left(t\right)‖}_2={\left(\frac{1}{\int m\left(t\right)dt}\int {\left|f\left(t\right)-\widehat{f}\left(t\right)\right|}^{2}m\left(t\right)dt\right)}^{1/2}$$

where $m\left(t\right)$ is the weight at $t$ and its default value is a constant.

3.1.2. Simulation under a static firing rate function.

In this simulation setting, the trials are treated as repeated trials, i.e., the trials share the same underlying firing rate function, and each trial is an independent realization of an inhomogeneous Poisson process. Several forms of firing rate functions have been considered in previous work. For example, Kass, Ventura and Cai (2003); Behseta and Kass (2005) assumed bell-shaped firing rate functions. Pachitariu, Stringer and Harris (2018) used a piecewise constant stimulus rate. Firing rate functions with multiple peaks from exponential stimuli functions have also been considered (Reynaud-Bouret et al., 2014). In our simulation, we chose the following bi-modal firing rate function

$$f\left(t\right)=0.5+9.5\ast \sum _{t_0}\text{exp}\left(-{\left(t-t_0\right)}^2/d^2\right)\left(t=1/50,2/50,\dots ,T/50\right)$$ (10)

where $d=1$ and the stimuli peaks are at time $t_0=6$ second (s) and 14 s. Thus, the baseline firing rate is 0.5 spikes per second and $f\left(t\right)$ reaches its maximum value (10 spikes per second) at time points 6 s and 14 s, as shown in the top panel of Figure 1.

To generate spike trains for multiple trials under an inhomogeneous Poisson process with the firing rate function in (10), the ideas in Adams, Murray and MacKay (2009) were adopted here. In each trial, the spikes were first randomly drawn from a homogeneous Poisson process. A thinning process was then applied to create a realization from the desired inhomogeneous Poisson process (see Appendix B for details). After obtaining the simulated spike trains, calcium fluorescence traces were generated using the auto-regressive model (1). When generating simulated data, we choose realistic parameters, i.e., parameters similar to what are estimated from data generated in Xiangmin Xu’s lab. Specifically, the following parameters were used in the simulations: $T=1000$, $\gamma =0.96$, $\sigma =0.3$ and $R=50$ trials in total. According to Vogelstein et al. (2010), the parameters above correspond to a sampling rate of 50 Hz and a length of 20 seconds per trial. For each simulation setting, 100 data sets were generated.

We implemented the constant penalty ${\ell }_0$ penalized method and our MTV-PAR methods at two window lengths of trials. Because the trials are assumed to have the same underlying firing rate function, using all trials (TV-all) is well justified. To understand how sensitive the results are to the choice of window length, we also examined the performance of TV-1, in which the ${\ell }_0$ penalty function of a trial is calculated using an initial estimate of spikes of the current trial. In this simulation study, since the firing rate function is static for all trials, we averaged the firing rate estimates across trials to measure the accuracy. As summarized in Figure 2 by the VP distance between true and estimate spikes and the ${\ell }_2$ norm for true and estimated firing rate functions, the two MTV-PAR methods provide noticeably more accurate estimates of spike events and firing rate for most values of $\lambda$, the parameter that controls the average penalty on the number of spikes. The comparison also indicates that using multiple trials brings small but consistent improvements. It is a little bit surprising that TV-1 works almost as well as aggregating multiple trials to obtain a penalty function. This is probably due to our conservative choice of the penalty function, such as re-scaling the firing function between 0 and 1. Thus, we made two conclusion that (1) the results are insensitive to the choice of B and (2) the improvement is mainly due to the time-varying penalty.

Figure 2:

Performance of three methods under a static firing rate function. Presented are the box plots based on 100 simulated data sets. (a): VP distance between the spike train and the estimated spike trains; (b): ${\ell }_2$ norm of firing rates between the true and estimated firing rate function.

To further illustrate the performances of the MTV-PAR, we showcase an example trial in full detail in Figure 1. The lower panel shows that the firing rate functions estimated from the two MTV-PAR methods are closer to the true underlying function; specifically, although both MTV-PAR and the constant-penalty methods underestimate the peak firing rate, the MTV-PAR methods are less biased. To understand this difference, we compared the estimated spikes. As indicated by the upper panel of Figure 1, compared to the two MTV-PAR methods, the constant-penalty method tends to produce extra spike events in low firing rate regions but miss spike events in higher firing rate regions. As a result, adopting the proposed time-varying penalty improves both spike detection and firing rate estimation.

We also simulated data with a larger noise level $\left(\sigma =0.3\right)$, longer series $\left(T=2,000\right)$ and higher decay rate $\left(\gamma =0.98\right)$. In all scenarios, we observed that our proposed MTV-PAR increased accuracy in both spike detection and firing rate estimation when compared to using a constant penalty function.

3.1.3. Simulation under a dynamic firing rate function.

We also simulated data under dynamic firing rate functions across trials. Figure 3(a) shows a dynamic firing rate function with two peaks within each trial; across trials, the peak values first increase and then decrease. The two-dimensional firing rate function at time point $t$ and trial $r$ is as follows:

$$f_r\left(t\right)=0.5+9.5\left(\exp \left\{-\frac{{\left(t-6\right)}^2}{d^2}\right\}+\exp \left\{-\frac{{\left(t-14\right)}^2}{d^2}\right\}\right)\ast \exp \left\{-\frac{{\left(r-R/2\right)}^2}{1000}\right\}$$ (11)

where $d=1$, $t=1/50,2/50,\dots ,T/50$ (second) and $r=1,2,\dots ,R$.

Figure 3:

Performance of the three methods under a dynamic firing rate function (a): Heat map of the true dynamic firing rate function for the simulated data, with each trial lasting 20 seconds across 50 trials. (b-c): VP distances of spike trains and ${\ell }_2$ norm of firing rates between the truth and the estimates. (d): for each of the three methods, the heat map of the difference between the true firing rate function and the estimated functions.

The other simulation parameters are the same as those of the static firing rate function: $T=1,000$, $\gamma =0.96$, $\sigma =0.3$ and $R=50$. Figure 3 summarizes the simulation results of the constant-penalty method and two MTV-PAR approaches, with the window length being $B=1$ and $B=10$, respectively. Similar to the simulation in Section 3.1.2, the MTV-PAR methods perform much better than using a constant penalty with reduced VP distance between true and estimated spike events and reduced ${\ell }_2$ norm between the true and estimated firing rate functions (Figure 3 (b) and (c)). Figure 3 (d) shows the heat maps of the difference between the true and estimated firing rate functions for each of the trials. Note that the larger differences observed in regions of higher firing rates are consistent with the properties of the Poisson distribution, where the variance increases with the mean. This is not indicative of a bias in the algorithm but rather a reflection of the inherent variability in the data. Again, these simulations demonstrate that the two MTV-PAR methods were able to better characterize the variation in the firing rate within each trial and across trials.

3.1.4. Simulation under dynamic constant firing rates.

Here, we refer to “dynamic” as across trials and “constant” as within trials. Specifically, we generated 50 trials: the first 25 trials had a constant firing rate of 5 spikes per second (approximately 100 spikes over 20 seconds), while the latter 25 trials had a constant firing rate of 2 spikes per second (approximately 40 spikes over 20 seconds). This is a scenario in which the Jewell-Witten approach is likely to outperform our proposed methods (TV-1 and TV-10).

The results in Figure 4 showed that the Jewell-Witten approach performed slightly better but very similarly to our methods under this condition. This indicates that when the firing rate is constant within trials, our proposed methods are as effective as the Jewell-Witten method in estimating spike locations and firing rates.

Figure 4.

Performance of the three methods under “dynamic constant firing rates”. (a): VP distance between the spike train and the estimated spike trains; (b): ${\ell }_2$ norm of firing rates between the true and estimated firing rate function.

3.2. Analysis of the Calcium Imaging Data.

We now pursue the scientific investigations on the neural activities in two longitudinal mice studies in which both calcium imaging data and behavioral data were recorded. In this first data set, calcium imaging data were collected within a few days after the participating mice had been trained for a discrimination task; thus, it is reasonable to treat trials with the same behavior outcome as repeated trials. As a comparison, in the second study, calcium recording started in the first learning trial and lasted for a few weeks; therefore, neural dynamics are expected as a result of the learning process and neural plasticity. For this reason, the proposed model which incorporates neuronal dynamics in firing rate estimation is likely to improve spike detection and firing rate estimation.

3.2.1. Mouse task data I.

The activity of neurons (labeled with GCaMP6s) from the mouse’s anterior lateral motor cortex (ALM) was recorded with two-photon calcium imaging during a head-fixed whisker-based discrimination task (Li et al., 2015). In the experiment, mice were supposed to discriminate the pole locations using their whiskers and report the perceived pole position by licking. Each trial is composed of three epochs: sample epoch (mice presented with a vertical pole), delay epoch (the pole was removed), and response epoch (mice cued to give a response). If a mouse licked the correct lick port, it was rewarded with liquid.

The data we present here is from one mouse with 73 trials over a few days. Each trial consists of $T=100$ data points at 15 Hz. The fluorescence data were obtained after typical pre-processing procedures such as correction for neuropil contamination and $\text{Δ}F/F_0$ transformation where $F_0$ is the baseline calcium fluorescence. The baseline fluorescence level is often estimated by the average of calcium fluorescence during several seconds period prior to the onset of a cue/stimulus (Shemesh et al., 2020). For example, in this experiment, the authors used the averaged fluorescence within a 0.5s period right before the start of each trial. There were four possible behavioral outcomes in the experiment: correct/incorrect lick left/right. Because most of the trials were either “correct lick left” or “correct lick right”, we combined the two incorrect groups as a single group. Figure 5 shows the calcium fluorescence traces of a pyramidal tract neuron in 73 trials, including 31 trials of “correct lick left”, 21 trials of “correct lick right”, and 21 trials of “incorrect lick” .

Figure 5:

An example cell from the water lick experiment (Li et al., 2015). Top Left: The calcium fluorescence trace under correctly lick left outcome (black dots: estimated spikes). Top Right: The calcium fluorescence trace under correctly lick right outcome. Bottom Left: The calcium fluorescence trace under incorrect lick outcomes. Bottom Right: Estimated firing rate functions of under different conditions. Vertical dashed line: the start of response epoch.

One interesting question is whether the neuron responded differently for different outcomes. Therefore, we conducted a stratified analysis for each outcome. Since the mouse has been well trained before calcium imaging recording, firing rates across trials under the same outcome group are relatively stable, which was confirmed by available 2D visualization (data not shown). Thus, it is sensible to combine all the trials within an outcome type when estimating firing rate.

As indicated in Figure 5, in “correct lick left” trials, the neuron fired right after the cue time (when the mouse was cued to make decisions); however, there was almost no neural activity under the other two outcome groups. The estimated firing rate function also confirmed this difference. It is known that the ALM brain region of mice is involved in planning directed licking (Guo et al., 2014). The estimated spikes and firing rate functions provide convincing evidence that this neuron is likely to show neural selectivity and play a critical role in the cognitive process of making the correct decision of licking the left pole.

3.2.2. Mouse task data II.

The second data set is from our long-term contextual discrimination experiment, in which mice were trained to recognize two contexts via fear conditioning (foot shock) (?). The research goal was to understand the behavior-associated hippocampal neural ensemble dynamics at single-cell resolution. Viral injections were administered into mice brains to introduce a genetically encoded calcium indicator (GCaMP6f). Fluorescence signals from hippocampal CA1 excitatory neurons were then optically recorded using one photon head-mounted miniscopes from freely moving mice (Figure 6, upper left). There were four stages in the experiment: habituation (mice freely exploring the environment), learning (learning to freeze in a stimulus context with foot shock), extinction (no foot shock), and relearning (stimulus reinstated).

Figure 6:

An example neuron from the foot shock study. Top Left: Calcium imaging of mouse hippocampus using a miniaturized scope and the four stages of the experimental design: habituation, learning, extinction, and relearning (?). Top Right: Calcium fluorescence traces and estimated spikes of a sample neuron. The red and blue traces were obtained during the learning and relearning stages, respectively. The long dashed vertical line denotes the time when a foot shock was applied, and the short black vertical lines are the time locations at which spikes were detected. Lower Left: the estimated firing rate function using MTV-PAR. Lower Right: the estimated firing rate function using stratified analysis. Vertical dashed line: the start time of foot shock.

Here, activity of 141 neurons for this particular mouse was recorded for several weeks. We chose the 21 foot shock sessions with the first 11 sessions (each with one trial) in the learning stage and the remaining 10 sessions (each with one trial) in the relearning stage. In each shock session, a foot shock was administered and the fluorescence trace was recorded at 15 Hz for 2 minutes. Figure 6 (upper right panel) shows an example neuron. The calcium fluorescence traces from different trials were temporally aligned by the start of shock time. As justified in Section 2.4, the Gaussian-boxcar kernel smoothing was used for each trial to ensure a smooth estimate of firing rate function. Note that the total number of trials in this data set is 21; given this relatively small number of trials, we presented the results based on a window length of 5, which is about 25% of the total number of trials. Although our simulations indicate that the results are not insensitive to the choice of window sizes and a window size of 1 might also work, we found that the estimated firing rate function based a window size of 1 is bumpy.

It is worth noting that treating the trials as independent realizations of the same underlying process will lead to undesired results. When assuming the trials as random samples from the same underlying distribution, the estimated peak firing time is misleading. As shown in Figure 6 (lower right panel), although the stratified estimates from the two stages showed that neuronal firing took place sooner and was more frequent in the relearning stage than in the learning stage in response to the foot shock stimulus, the results based on stratified analysis were not able to completely characterize the intrinsic evolution of neural firing during the cognitive learning process. As a comparison, our MTV-PAR provides not only estimated spikes but also an estimated firing rate function for each trial, with the between-trial change assumed to be smooth. The 3D firing rate plot in Figure 6 (lower left) suggests that the neuron is more synchronous to the stimulus during the relearning stage as compared to the learning stage. This very interesting finding may reflect the evolving learning-related neuronal dynamics.

We also applied the constant penalty method to both of the two real studies. To make sure that the results are comparable, we first estimated spike events using each of the two methods and then applied the same smoothing parameter to estimate firing rate. Our simulation in Section 3.1.2 and Figure 1 indicated that MTV-PAR provides less biased (downward) peak firing rates; consistent with the simulation results, the peak firing rates of the real data estimated by the constant penalty method are lower those by MTV-PAR (data not shown).

4. Discussion.

In this paper, we proposed MTV-PAR, a time-varying ${\ell }_0$ penalized method to simultaneously conduct spike detection and firing rate estimation from longitudinal calcium fluorescence trace data. One novel aspect of the proposed method is to introduce a time-varying penalty, as neurons tend to fire at certain conditions, such as shortly after the presence of stimuli. Another distinct advantage of the MTV-PAR method is the ability to account for the intrinsic neural dynamics across trials, which provides a flexible framework analyze longitudinal calcium recordings of neuronal activities.

In our MTV-PAR, each iteration consists of two steps: the spike detection step using time-varying ${\ell }_0$ regularization based on the current estimate of the firing rate functions and the firing rate estimation step based on the currently detected spikes. Here we used a time-varying ${\ell }_0$ penalization in the spike detection step and a Gaussian-boxcar smoothing in the firing rate estimation step. In some situations, other types of regularization and smoothing methods might be preferred or more appropriate; our strategy of integrating information from multiple trials can be adopted in a similar manner.

Like other existing methods, we aim to estimate the time of a neural spike or a burst of spike events. Bounded by the temporal resolution of calcium imaging data, when there are multiple spikes within the same time frame, we are not able to zoom in to estimate the times of individual spikes. In several works, the change in fluorescence intensities due to spike events at time $t$, i.e., $s\left(t\right)$, was initially motivated to model spike counts (Vogelstein et al., 2010; Friedrich and Paninski, 2016; Friedrich, Zhou and Paninski, 2017); following this line of thought, it is reasonable to assume that $s\left(t\right)$ is positively correlated with the spike counts for re-scaled fluorescence traces. In other work (Pnevmatikakis et al., 2013; Jewell and Witten, 2018), the sign of $s\left(t\right)$ was the focus. The unknown and nonlinear relationship between spike counts and fluorescence transient measured based upon genetically encoded indicators (Vogelstein et al., 2010; Lütcke et al., 2013; Rose et al., 2014) complicates the accurate interpretation and modeling of fluorescence transient data. In the current version of MTV-PAR, the magnitude of $s\left(t\right)$ was not used in the firing rate estimation. It is worth conducting future research to examine how to best utilize $s\left(t\right)$ in analyzing calcium fluorescence data.

APPENDIX A: PROOFS FOR SECTION 2.3

A.1. Equivalence between Problem (3) and Problem (4).

$$\underset{c\left(1\right),\dots ,c\left(T\right)}{\min }\left\{\frac{1}{2}\sum _{t=1}^T{\left(y\left(t\right)-c\left(t\right)\right)}^2+\sum _{t=2}^T\lambda \left(t\right)1_{\left(c\left(t\right)-\gamma c\left(t-1\right)\ne 0\right)}\right\}$$

$$\iff \underset{\begin{array}{c}0={\tau }_0<{\tau }_1<\dots <{\tau }_k<{\tau }_{k+1}=T,k\\ c\left(1\right),\dots ,c\left(T\right)\end{array}}{\min }\left\{\sum _{j=0}^k\frac{1}{2}\sum _{t={\tau }_j+1}^{{\tau }_{j+1}}{\left(y\left(t\right)-c\left(t\right)\right)}^2+\sum _{t=2}^T\lambda \left(t\right)1_{\left(c\left(t\right)-\gamma c\left(t-1\right)\ne 0\right)}\right\}$$

$$\iff \underset{\begin{array}{c}0={\tau }_0<{\tau }_1<\dots <{\tau }_k<{\tau }_{k+1}=T,k\\ c\left(1\right),\dots ,c\left(T\right)\end{array}}{\min }\left\{\sum _{j=0}^k\frac{1}{2}\sum _{t={\tau }_j+1}^{{\tau }_{j+1}}{\left(y\left(t\right)-c\left(t\right)\right)}^2+\sum _{j=0}^k\lambda \left({\tau }_j\right)\right\}-\lambda \left(0\right)$$

$$\iff \underset{0={\tau }_0<{\tau }_1<\dots <{\tau }_k<{\tau }_{k+1}=T,k}{\min }\left\{\sum _{j=0}^k\frac{1}{2}\sum _{\begin{array}{c}t={\tau }_j+1\dots {\tau }_{j+1}\\ \left(c\left(t\right)=\gamma c\left(t-1\right)\right)\end{array}}{\left(y\left(t\right)-c\left(t\right)\right)}^2+\sum _{j=0}^k\lambda \left({\tau }_j\right)\right\}$$

$$\iff \underset{0={\tau }_0<{\tau }_1<\dots <{\tau }_k<{\tau }_{k+1}=T,k}{\min }\left\{\sum _{j=0}^k\left[𝒟\left(y_{\left({\tau }_j+1\right):{\tau }_{j+1}}\right)+\lambda \left({\tau }_j\right)\right]\right\}$$

A.2. Equivalence between Problem (3) and Problem (7).

Based on the result in Appendix A.1, to show that Problems (3) and (7) are equivalent, we only need to show that Problems (4) and (7) are equivalent.

$$\underset{0={\tau }_0<{\tau }_1<\dots <{\tau }_k<{\tau }_{k+1}=t,k}{\min }\left\{\sum _{j=0}^k\left[𝒟\left(y_{\left({\tau }_j+1\right):{\tau }_{j+1}}\right)+\lambda \left({\tau }_j\right)\right]\right\}$$

$$\iff \underset{0={\tau }_0<{\tau }_1<\dots <{\tau }_k<{\tau }_{k+1}=t,k}{\min }\left\{\sum _{j=0}^{k-1}\left[𝒟\left(y_{\left({\tau }_j+1\right):{\tau }_{j+1}}\right)+\lambda \left({\tau }_j\right)+𝒟\left(y_{\left({\tau }_k+1\right):{\tau }_{k+1}}\right)+\lambda \left({\tau }_k\right)\right]\right\}$$

$$\iff \underset{0<{\tau }_k<{\tau }_{k+1}=t}{\min }\left\{\underset{0={\tau }_0<{\tau }_1<\dots <{\tau }_{k\prime }<{\tau }_{k\prime +1}={\tau }_k,k\prime }{\min }\left\{\sum _{j=0}^{k\prime }\left[𝒟\left(y_{\left({\tau }_j+1\right):{\tau }_{j+1}}\right)+\lambda \left({\tau }_j\right)\right]\right\}+𝒟\left(y_{\left({\tau }_k+1\right):{\tau }_{k+1}}\right)+\lambda \left({\tau }_k\right)\right\}$$

$$\iff \underset{0\le \tau <t}{\min }\left\{F\left(\tau \right)+𝒟\left(y_{\left(\tau +1\right):t}\right)+\lambda \left(\tau \right)\right\}$$

APPENDIX B: ALGORITHM FOR SIMULATING SPIKE TRAINS FROM AN INHOMOGENEOUS POISSON PROCESS

$$\underset{¯}{\begin{array}{l}\overline{\underset{¯}{\begin{array}{l}\mathbf{\text{Algorithm}}\text{Simulate}\text{Spike}\text{Trains}\text{from}\text{an}\text{Inhomogeneous}\text{Poisson}\text{Process}\left(\text{Modified}\text{from}\right.\\ \text{Algorithm}1\text{of}\text{Adams},\text{Murray}\text{and}\text{MacKay}\left(\text{2009}\right)\end{array}}}\\ 1:\mathbf{\text{Input}}\text{Length}\text{of}\text{data}n,\text{upper}\text{bound}\text{of}\text{rate}\text{intensity}{f}^{\ast },\text{firing}\text{rate}\text{function}f\left(t\right)\\ 2:\text{Draw}\text{the}\text{number}\text{of}\text{spikes}:S\sim Poisson\left(n{f}^{\ast }\right)\\ 3:\text{Uniformly}\text{distribute}\text{the}\text{spikes}:{ℰ}_S={\left\{s_i\right\}}_{i=1}^S~\text{Uniform}\left(n\right)\\ 4:\mathbf{\text{for}}i=1,2,\dots S\mathbf{\text{do}}\\ 5:\mathbf{\text{if}}uniform\left(0,1\right)\ge \sigma \left(f\left(s_i\right)\right)\mathbf{\text{then}}\\ 6:\text{exclude}{s}_i\text{from}{ℰ}_S\\ 7:\mathbf{\text{end}}\mathbf{\text{if}}\\ 8:\mathbf{\text{end}}\mathbf{\text{for}}\\ 9:\text{Return}\text{the}\text{set}\text{of}\text{spikes}{ℰ}_S\end{array}}$$