Information-Theoretic Approach and Fundamental Limits of Resolving Two Closely-Timed Neuronal Spikes in Mouse Brain Calcium Imaging

Abstract

Objective:

Although optical imaging of neurons using fluorescent genetically encoded calcium sensors has enabled large-scale in vivo experiments, the sensors’ slow dynamics often blur closely-timed action potentials into indistinguishable transients. While several previous approaches have been proposed to estimate the timing of individual spikes, they have overlooked the important and practical problem of estimating inter-spike-interval (ISI) for overlapping transients.

Methods:

We use statistical detection theory to find the minimum detectable ISI under different levels of signal-to-noise ratio (SNR), model complexity, and recording speed. We also derive the Cramer-Rao lower bounds (CRBs) for the problem of ISI estimation. We use Monte-Carlo simulations with biologically derived parameters to numerically obtain the minimum detectable ISI and evaluate the performance of our estimators. Furthermore, we apply our detector to distinguish overlapping transients from experimentally-obtained calcium imaging data.

Results:

Experiments based on simulated and real data across different SNR levels and recording speeds show that our algorithms can accurately distinguish two fluorescence signals with ISI on the order of tens of milliseconds, shorter than the waveform’s rise time. Our study shows that the statistically optimal ISI estimators closely approached the CRBs.

Conclusion:

Our work suggests that full analysis using recording speed, sensor kinetics, SNR, and the sensor’s stochastically distributed response to action potentials can accurately resolve ISIs much smaller than the fluorescence waveform’s rise time in modern calcium imaging experiments.

Significance:

Such analysis aids not only in future spike detection methods, but also in future experimental design when choosing sensors of neuronal activity.

I. Introduction

Calcium imaging relies on the sudden change of intracellular calcium ion concentration during neuronal activity, called action potentials (APs) [1, 2]. Genetically encoded calcium indicators (GECIs) are popular tools used to image intracellular calcium dynamics and therefore, track neuronal activity [1, 2]. These indicators consist of a calcium binding domain connected to a fluorescent protein. Calcium binding to the indicator causes the protein to undergo conformational changes, increasing the fluorescence brightness during times of AP [1].

Recent advances in optical microscopy and GECIs have increased the use of these tools in large scale in vivo recording of neuronal populations [2–5]. Accurate extraction of neuronal activities from the optical recordings is expected to give insight into how neuronal circuitry process information. Therefore, to fully understand how stimuli are processed and transmitted among neurons, spike extraction approaches with high accuracy and precision are needed. However, during periods of rapid activity, closely timed AP induced fluorescence transients accumulate, making the detection and separation of individual spikes a challenge (Fig. 1 (a)).

Fig. 1.

Closely timed AP induced fluorescence transients accumulate, making the detection of spike counts a challenge. (a) Simultaneous optical and electrical recording from a neuron. Red markers in the electrical recording correspond to spike times. Data are from [1]. (b) Illustration of the null hypothesis with one AP evoked signal (top), and the alternative hypothesis of two closely timed fluorescence signals with sub-second ISI (bottom).

Over the past years, several groups have tackled the paramount problem of firing rate inference or spike train extraction from the observed fluorescence signals. Methods that estimate firing rate or spiking probabilities include particle filtering [6], fast nonnegative deconvolution [7], supervised learning with probabilistic models [8], and the Markov Chain Monto-Carlo methods [9]. Although these approaches are advantageous for assessing the uncertainty in the estimations, they might not be best suitable for temporal coding studies where a single spike train with spike time estimates is necessary [21]. Methods that estimate spike trains include nonnegative deconvolution [10, 11], sparsity-based reconstruction [12–15], template matching [16–18], finite rate of innovation [19, 20], and Bayesian methods [21]. Several of these studies behave well in reconstructing neuronal bursting activities [8, 13–15, 20, 21]. However, few of these studies have conducted theoretical performance limit analysis. Such analysis can aid in resolving experimental design issues for optimal spike detectability, such as sensor kinetics, recording speed, and photon counts [17]. Among the above-mentioned studies, only [17] and [20] have compared their single spike time estimation method with the optimal performance of any unbiased estimator through the Chapman-Robbins and Cramer-Rao lower bounds (CRB), respectively.

In this paper, we extend the application of previous studies [17, 20] which considered only isolated spikes, by investigating the case of temporally overlapping waveforms. In parallel to other computational spike extraction methods [10–21], we quantify 1) resolution from the statistical point-of-view and 2) the theoretical bound on the precision of estimating the inter-spike-interval (ISI). Our work is based on the statistical and information theoretic tools developed in the past two decades for estimating the fundamental resolution of optical systems [22–26]. As the symmetric point spread function (PSF) considered in the numerical results of these optically oriented papers does not match our problem, we extend this framework for the ISI estimation and study possible consequences of asymmetric waveforms.

The organization of the paper is as follows. In section II, the statistical description of the acquired data, the detection framework for finding the minimum detectable ISI, and the derivation of the Fisher information matrix related to the asymptotic performance of the estimation problem is described. Section III presents the numerical results on simulated and experimentally-obtained datasets. Finally, discussion and concluding remarks are presented in sections IV and V, respectively.

II. METHODS

A. Action Potential Evoked Fluorescence Signal Model

In response to an AP, the intracellular calcium concentration rises rapidly, which is followed by a slow decay to its baseline value [27]. As validated by experiments in [1], we assume that the measured fluorescence signal is linearly related to the intracellular calcium concentration. Given samples at time points t_k (k = 1, 2…, K), the mean fluorescence signal generated in response to a single AP at time t = 0 with normalized peak amplitude θ₀ = A is expressed as [16]

$$s_0\left(t_k;{\theta }_0\right)=A{F}_{0}h\left(t_k\right)+F_0,$$ (1)

where the change in the fluorescence signal is modeled as

$$h\left(t_k\right)=a\left(1-\exp \left(-t_k/{\tau }_{on}\right)\right)\exp \left(-t_k/{\tau }_d\right)u\left(t_k\right).$$ (2)

In (1), F₀ is the baseline photon rate due to the neuron’s resting state fluorescence, autofluorescence from cellular structures, and fluorescence from the extracellular space [17]. In (2), u(t_k) is the unit step function, a is a normalization factor, and τ_on and τ_d are known rise and decay time constants, respectively.

Assuming the optical technique used for measurement has negligible read out noise, the recordings are photon shot noise limited. Therefore, the K-element measurement vector y is distributed according to Poisson statistics with a time-varying mean s₀(t_k;θ₀).

B. Statistical Analysis of Resolution

In this section, we explain the tools from statistical theory used in the detection problem. Our study is the continuation of a previous work by Shahram and Milanfar [24], generalized by considering asymmetric waveforms and Poisson statistics.

We test the hypothesis of whether one or two spikes are present at an observation window of length K points, as illustrated in Fig. 1 (b). The null hypothesis H₀ denotes the case where there is one spike present as described in the previous section. The alternative hypothesis H₁ refers to the case where we have two spikes with ISI ≠ 0.The peak amplitude of the signal generated by the two spikes in H₁ should be comparable to the peak amplitude of a single spike under the H₀ hypothesis. Thus, in the case where the neuron spikes twice at times d₁ and –d₂ (ISI = d₁ + d₂ =d) with normalized peak amplitudes α and β (where A = α + β), we define the accumulated mean signal as

$$s_1\left(t_k;{\mathit{\theta }}_1\right)=\alpha F_{0}h\left(t_k-d_1\right)+\beta F_{0}h\left(t_k+d_2\right)+F_0,$$

where θ₁ = [d₁, d₂, α, β] is the parameter vector defining the signal.

The probability distribution of the set of photon measurements y under H_j (j ∈{0,1}) is then modelled as

$$\begin{gathered} p(\mathit{y}|H_j)=\prod _{k=1}^K\mathrm{Poisson}\left(s_j\left(t_k;{\mathit{\theta }}_j\right)\right) \\ =\prod _{k=1}^K\exp \left(-s_j\left(t_k;{\mathit{\theta }}_j\right)\right)\frac{s_j^{y(t_k)}\left(t_k;{\mathit{\theta }}_j\right)}{y\left(t_k\right)!}. \end{gathered}$$

The Log-likelihood Ratio Test (LRT) can be used to choose between the two hypotheses [28] for a given set of measurements:

$$\begin{gathered} L(\mathit{y})=\ln (p(\mathit{y}|H_1)/p(\mathit{y}|H_0)) \\ =\sum _{k=1}^{K}y\left(t_k\right)\ln \left(s_1\left(t_k;{\mathit{\theta }}_1\right)/s_0\left(t_k;{\theta }_0\right)\right)-\sum _{k=1}^K(s_1\left(t_k;{\mathit{\theta }}_1\right)-s_0\left(t_k;{\theta }_0\right)). \end{gathered}$$

For any given dataset, H₁ is selected as the more likely hypothesis if the log-likelihood ratio exceeds a predefined threshold. The choice of threshold depends on the desirable value for the probability of detection, P_D, or the tolerable value for the probability of false positive, P_F [28].

Parameters (τ_on, τ_d, and F₀) for a particular calcium probe in a particular biological system can be systematically characterized and thus are considered to be known quantities. However, the model parameters ({θ₀, θ₁}) in the above LRT are unknown in general. To address this composite hypothesis problem, we use the Generalized Likelihood Ratio Test (GLRT) to simultaneously assess the existence of two spikes and estimate the ISI between them. GLRT uses the maximum likelihood (ML) estimates of the unknown parameters to form the Neyman-Pearson detector [28]. The ML estimates of the unknown parameters in θ _j are found by maximizing the loglikelihood function of the data under H_j (j ∈ {0,1}):

$$\begin{gathered} {\widehat{\mathit{\theta }}}_j=\underset{{\mathit{\theta }}_j}{\mathrm{argmax}}\{\ln (p(\mathit{y}|H_j))\} \\ =\underset{{\mathit{\theta }}_j}{\mathrm{argmax}}\sum _{k=1}^K\left[y\left(t_k\right)\ln \left(s_j\left(t_k;{\mathit{\theta }}_j\right)\right)-s_j\left(t_k;{\mathit{\theta }}_j\right)\right], \end{gathered}$$ (3)

where we have kept only the parameter dependent parts. We numerically solve the above nonlinear maximization problem using MATLAB’s optimization toolbox. Note that without loss of generality, we have set the single spike model characterized in (1) to start at t = 0 In the case of aiming to detect the timing of single spikes, modifying the signal model in (1) to include the unknown time shift and using this model in the maximum likelihood equation will solve the problem. Before addressing the general case of detecting spikes with fully unknown parameters, we consider the more intuitive case of detecting spikes with known amplitudes, as in [24].

1). Spikes with Known Amplitudes

The hypotheses for differentiating the case of one large spike starting at the test origin (defined as time zero), versus the case of two smaller amplitude spikes located around the test origin are expressed as

$$\begin{gathered} H_0:s_0\left(t_k\right)=(\alpha +\beta )F_{0}h\left(t_k\right)+F_0, \\ {H}_1:s_1\left(t_k;{\mathit{\theta }}_1\right)=\alpha F_{0}h\left(t_k-d_1\right)+\beta F_{0}h\left(t_k+d_2\right)+F_0, \\ {\mathit{\theta }}_1=\left[d_1,d_2\right]. \end{gathered}$$

Note that while the amplitudes (α, β) are assumed to be known, their values can be equal or different. Also, the spikes in the H₁ case can be symmetrically (d₁ = d₂ = d/2) or asymmetrically (d₁ ≠ d₂ ) distributed around the test origin.

The minimum detectable distance between two spikes that can be distinguished from a single spike is modified by how the time origin of the test is defined. Conceptually, the most challenging problem set up has high temporal overlap between s₁(t_k; θ₁) and s₀(t_k). Numerically, such a set up can be attained by finding the maximum point of cross-correlation between s₁(t_k; θ₁) and s₀(t_k) [24]. This setup, using the Taylor expansion, then defines the test time origin τ as

$$|\tau |=\frac{\alpha d_1-\beta d_2}{\alpha +\beta }.$$

Therefore, fixing the location of the test origin to τ = 0 leads to αd₁ = βd₂, equivalently,

$$d_1=\frac{\beta }{(\alpha +\beta )}d\text{and}{d}_2=\frac{\alpha }{(\alpha +\beta )}d.$$ (4)

This suggests that, for α = β, the “best” (i.e. most challenging) location to carry out the hypothesis test is in the middle of the two transients and for α ≠ β, the point should be closer to the larger signal. The condition of whether the spikes are symmetrically or asymmetrically located according to (4) around the test origin is studied to investigate the effect of defining the test origin, or equivalently the H₀ hypothesis, in quantifying the resolution limit.

2). Spikes with Random Amplitudes

Calcium ion influx through calcium channels and calcium binding to the sensor are stochastic processes that can lead to variations in the calcium signal response and thus, the fluorescence signal. Therefore, the signal peak value of a single spike can change from time to time and even drastically from one neuron to another. To encompass these variabilities, we consider the more general case of differentiating spikes with unknown intensities, by treating the peak amplitudes as random variables. The H₁ hypothesis is described by (1), and the H₀ hypothesis under the condition in (4) is expressed as

$$\begin{gathered} H_1:s_1\left(t_k;{\mathit{\theta }}_1\right)=\alpha F_{0}h\left(t_k-d_1\right)+\beta F_{0}h\left(t_k+d_2\right)+F_0, \\ {\mathit{\theta }}_1=[d,\alpha ,\beta ]. \end{gathered}$$

This is a generalization of the previous work, in which the amplitudes of unknown signals were assumed to be deterministic [24]. Since a Bayesian hypothesis testing approach to combine observation data and a priori information about the peak amplitude distribution involves integrations that are not analytically solvable, we used the GLRT based conditional ML estimation technique [29]. We incorporate the prior information in quantifying the performance of the detector through computing the expected value of P_D (and P_F) over p(α, β) (the joint probability distribution of the amplitudes [29].

The prior probability distribution of the single spike amplitude has not been previously investigated. Therefore, we set to find the best probability model from a set of measurements.

C. Extraction of Single and Double AP Evoked Fluorescence Transients

We used the publicly available experimental dataset, provided by the Svoboda lab [1] as reference. The dataset contains simultaneous optical imaging and loose-seal cell-attached recording of nine GCaMP6s and eleven GCaMP6f (types of GECIs) expressing neurons. We extract single AP induced transients to find the best probability model for peak fluorescence response. The outline of the processing steps is highlighted in Fig. 2. First, we identify single fluorescence transients using the electrophysiological data and extract them from the optical recordings. To ensure accurate estimation of single AP evoked fluorescence peak values, we discard spikes with ISI values less than twice the fluorescence half decay time constant (2τ_1/2, approximately 1s for GCaMP6s and 0.3 s for GCaMP6f [1]). We also discard cases with high neuropil contamination. Second, the background signal F₀ for each spike is calculated by averaging the baseline near the onset time in periods with no neuronal activity. Next, we use a twostep nonlinear least square procedure to fit a double exponential model as in (2) to the extracted spike transients. The least square curve fitting method finds the best fit of the model to the data, y_i, by minimizing ${\chi }^2={\sum }_{i=1}^n{\left(y_i-f_i\right)}^2/{\sigma }_i^2$, where f is the set of estimated values and ${\sigma }_i^2$ is the standard deviation (std) of each data point [30]. Since the data has Poisson statistics, in the first step, we use the data itself as an estimate of ${\sigma }_i^2$. We repeat the fitting for a second time to reduce the overemphasis of data points with lower variance [31]. In this step, we use the fitted values, f_i, as the estimates of ${\sigma }_i^2$. Lastly, we evaluate the goodness of fit by calculating the p-value associated with the final χ² value. Signals that have a poor fit (p-value<0.05) to the model, are discarded from further analysis.

Fig. 2.

Flowchart of the single spike waveform extraction and curve fitting for characterizing the prior probability model.

We use the fitted results to obtain the distribution of normalized peak values for each neuron. We test fifteen different one-sided distributions listed in Table I on all neurons separately. We determine the best distribution model among all neurons using a two-step procedure. First, for each neuron, we calculate the ML estimates of each model’s parameters. We then use Pearson’s χ² goodness of fit test for each fitted model. Models that result in p-values<0.05 are discarded from the set of possible probability models. In the second step, we choose the best probability model among the remaining models using the Akaike Information Criterion (AIC), defined as [32]

$$\text{AIC}=-2\ln (f(x|\widehat{\mathit{\theta }}))+2k,$$

where $\widehat{\mathit{\theta }}$ is the ML estimates of the model’s k parameters based on the observations, x. For a single dataset, the model resulting in the smallest AIC score is the best model that represents the data [32]. We select the probability model with the lowest sum of AIC score across all neurons as the model that best describes the dataset (among the models considered in this paper).

TABLE I.

LIST OFONE SIDED DISTRIBUTIONS USED IN MODEL FITTING.

#	Distribution Name	#	Distribution Name
1	Rayleigh	9	Log-Normal
2	Birnbaum-Saunders	10	Nakagami
3	Extreme Value	11	Normal
4	Gamma	12	Rician
5	Half Normal	13	Weibull
6	Inverse Gaussian	14	Burr
7	Logistic	15	T Location Scale
8	Log-Logistic

We also extract visually indistinguishable double spike cases to demonstrate the detection performance of our framework on experimentally-obtained data. Double spike signals are defined as cases with two closely-timed spikes without any other spike occurring within 2τ_1/2 time interval around them. Further, we discarded cases in which the two spikes were visually distinguishable. We centered the two spike signals such that time t = 0 is in the middle of the two waveforms. Examples of one and two spike signals are illustrated in Fig. 1 (a).

D. The Cramer-Rao Lower Bound

In this section, we utilize the Cramer-Rao based lower bounds as reference to study the limits of attainable precision in the estimation of the AP evoked fluorescence transient peak amplitudes and ISI under H₁ hypothesis. The covariance matrix C of any unbiased estimator of the p-parameter vector θ₁ is a p × p matrix that satisfies [33]

$${\mathbf{C}}_{{\widehat{\mathit{\theta }}}_1}\ge {\mathbf{I}}_F^{-1},$$

where I_F is the p × p Fisher information matrix. The elements of I_F for data with Poisson statistics are calculated as

$$\begin{gathered} {\mathbf{I}}_{F_{ij}}=-\text{E}\left\{\frac{{\partial }^{2}p\left(\mathit{y};{\mathit{\theta }}_1\right)}{\partial {\mathit{\theta }}_{1i}\partial {\mathit{\theta }}_{1j}}\right\} \\ =\sum _{k=1}^K\frac{1}{s_1\left(t_k;{\mathit{\theta }}_1\right)}\frac{\partial s_1\left(t_k;{\mathit{\theta }}_1\right)}{\partial {\mathit{\theta }}_{1i}}\frac{\partial s_1\left(t_k;{\mathit{\theta }}_1\right)}{\partial {\mathit{\theta }}_{1j}}. \end{gathered}$$ (5)

In the following sections, the CRBs for estimating ISI, α, and β are derived for the problems described in sections II.B.1 and II.B.2.

1). CRB for Known Amplitude Signals

For this case, under the assumption of (4), the only unknown parameter is d. Therefore, the Fisher information matrix reduces to a scalar calculated as

$$I_F=-\text{E}\left\{\frac{{\partial }^{2}p(\mathit{y};d)}{\partial d^2}\right\}=\sum _{k=1}^K\frac{1}{s_1\left(t_k;d\right)}{\left(\frac{\partial s_1\left(t_k;d\right)}{\partial d}\right)}^2.$$

Thus, the lower bound for the unbiased estimation of d is $\text{var}(\widehat{d})\ge I_F^{-1}$. We refer the reader to Supplementary Materials for the full derivation of the above quantity.

2). Hybrid CRB for the Random Amplitude Signals

To address the more challenging case of random spike amplitudes combined with unknown deterministic ISI, we estimate the unknown parameters through a joint ML and maximum a posteriori (MAP) estimator. This optimization problem involves the simultaneous ML estimation of ISI (or d) and MAP estimation of the normalized peak amplitudes [29] (α and β):

$${\widehat{\mathit{\theta }}}_1=\left[\begin{array}{c}{\widehat{d}}_{\text{ML}}\\ {\widehat{\alpha }}_{\text{MAP}}\\ {\widehat{\beta }}_{\text{MAP}}\end{array}\right]=\underset{d,\alpha ,\beta }{\mathrm{argmax}}\left\{\ln p_{\mathit{y}|{\mathit{\theta }}_1}(\mathit{y}|{\mathit{\theta }}_1)+\ln p_{\alpha ,\beta |d}(\alpha ,\beta |d)\right\},$$

where p_α,β|d(α,β|d) is the conditional joint prior distribution of the amplitudes. For this hybrid problem, we utilize the more general Hybrid CRB (HCRB) [29] method, which is defined as

$$\text{HCRB}\triangleq {\mathbf{I}}_H^{-1}.$$

I_H is called the Hybrid information matrix, which defines the lower limit on the mean square error (MSE) of any estimator. It is a 3×3 matrix for the problem in section II.B.2 (θ₁ = [d,α,β]), expressed as the sum

$${\mathbf{I}}_H={\mathbf{I}}_D+{\mathbf{I}}_P,$$

where

$${\mathbf{I}}_D(d)={\text{E}}_{{\mathit{\theta }}_r|d}\left\{{\mathbf{I}}_F\left(d,{\mathit{\theta }}_r\right)\right\},$$

and

$$\begin{gathered} {\mathbf{I}}_{P_{ij}}=-{\text{E}}_{{\mathit{\theta }}_r|d}\left\{\frac{{\partial }^2\ln \left(p\left({\mathit{\theta }}_r|d\right)\right)}{\partial {\mathit{\theta }}_{ri}\partial {\mathit{\theta }}_{rj}}\right\}, \\ {\mathit{\theta }}_r=[\alpha ,\beta ]. \end{gathered}$$ (6)

The elements of the Fisher information matrix are calculated according to (5), in which the derivatives of the mean signal s₁(t_k; θ₁) relative to the amplitudes are derived in Supplementary Materials. To attain I_D , we calculate the expectation of I_F with respect to α and β. Note that the amplitudes of the two spikes are independent and identically distributed (i.i.d) random variables and independent from d i.e. p(𝛼,𝛽|d) = p(α)p(β). This integral is numerically solved using MATLAB. I_P on the other hand, can be attained analytically, which is derived in section III.C based on the best model match for the prior distribution of α and β.

III. RESULTS

In this section, we present numerical analysis of the minimum detectable ISI and the CRBs formulized in the previous sections using biologically plausible simulations. We also present the detection results of applying the formulated detector on the experimental dataset described in section II.C. Our simulations are parameterized based on the experimental results in [1, 16] for two different calcium sensors: GCaMP6s and GCaMP6f. Since multiple existing scanning techniques have different imaging speeds, we consider multiple frame rates (f_s) in our simulations as well. Acousto-optical deflector (AOD) based two-photon microscopes have allowed high speed imaging of neuronal activities up to 500 Hz [16], enabling millisecond precision spike time estimations. Resonant scanning methods are more widely used, achieving 30 Hz for a 512×512 pixel field-of-view, or 60 Hz for a smaller area such that the laser dwell time per neuron is approximately kept the same. Without loss of generality, we consider the case in which the dwell time per neuron is constant across different recording speeds for the comparison between their resolution limits and theoretical lower bounds. Table II lists the values of parameters used in the simulations. We determine the dwell time by considering a 15μm diameter neuron imaged by systems with 1 μm pixel size.

TABLE II.

LIST OF CHOSEN MEAN AND STANDARD DEVIATIONS OF ΔF/F₀ PRIOR DISTRIBUTIONS.

Parameter	GCaMP6s	GCaMP6f
Mean	0.23	0.19
Std	0.03	0.06

A. The Gamma Distribution Characterizes the Peak Amplitude

The data extraction pipeline described in section II.C resulted in n = 44, 10, 13, 51, 48, 30, 61, 10, and 13 waveforms per GCaMP6s labeled neurons and n = 100, 63, 88, 39, 14, 60, 99, 283, 54, 38, and 93 waveforms per GCaMP6f labeled neurons. The χ² test eliminated distribution numbers 1, 10, 11, 12, 13, 14, and 15 for GCaMP6s neurons and 1, 2, 3, 5, 6, 8, 9, and 15 for GCamp6f neurons from Table I. Among remaining models, the Gamma distribution resulted in the minimum sum of AIC score for both calcium sensors. The Gamma probability distribution with parameters k and c is defined as [34]

$$f(x;k,c)=c^{-k}{x}^{k-1}\exp (-x/c)/\mathrm{\Gamma }(k),x>0,$$ (7)

where Γ(k) is the gamma function with argument k. The mean and variance of this distribution are kc and kc², respectively [34]. Fig. 3 (a) illustrates the empirical ΔF/F histogram of one neuron from the GCaMP6f dataset, with the best fit Gamma distribution overlaid on it.

Fig. 3.

Resolution limit is quantified by the smallest detectable ISI (ISI_min).ISI_min smaller than the fluorescence waveform’s rise time can be achieved under certain experimental conditions. (a) Experimental probability distribution of peak ΔF/F from the GCaMP6f dataset is best described by the Gamma distribution (n = 283 samples). (b) GLRT achieves similar ISI_min values to LRT. (c) ISI_min versus combined SNR and ratio between signal amplitudes for (lower curves) d₁ = d₂ and (upper curves) αd₁ = βd₂ Detecting two unequally bright transients that are symmetrically located around the test origin gives better detection results. (d) Prior knowledge about the probability distribution of randomly distributed amplitudes results in similar detection performance to the equal known amplitude case. (e) ISI_min calculated for different recording speeds. Under the same dwell time and photon emission rate by the neuron, faster recordings detect smaller ISIs. All results (b)-(e) were obtained with 2000 Monte-Carlo simulation and at detection performance point of P_D = 0.99 and P_F = 0.017.

Similar to previous calcium imaging studies [16], we define signal-to-noise ratio (SNR) as

$$\text{SNR}=\left(\mathrm{\Delta }F/F_0\right)\sqrt{F_0},$$

where ΔF is the change in fluorescence of one AP evoked calcium transient at its peak amplitude, equal to AF₀ in (1). Noting that the mean and variance of the Gamma distribution are dependent, we carry out simulations with different levels of SNR by fixing k and c (thus fixing the mean and variance) while changing the baseline photon rate F₀. Based on the mean and standard deviation of ΔF/F₀ values from all neurons in each dataset, we selected the mean and standard deviation of both sensors’ ΔF/F₀ prior distributions as listed in Table III. To be consistent in simulations between the known and unknown amplitude cases, we carried out the simulations related to section II.B.1 with α + β = 0.46 for GCaMP6s and α + β = 0.38 for GCaMP6f.

TABLE III.

LIST OF VALUES USED FOR THE KNOWN PARAMETERS IN SIMULATIONS.

Parameter	GCaMP6s	GCaMP6f
τon	72 ms [1]	18 ms [1]
τd	793.5 ms [1]	204.9 ms [1]
fs	500 Hz (AOD), 60 Hz and 30 Hz (Resonant)	500 Hz (AOD), 60 Hz and 30 Hz (Resonant)
Dwell time	25 μs	25 μs

B. The Detector Distinguishes Two Fluorescence Transients with ISI on the Order of Tens of Milliseconds

1). Performance Characterized Through Data Simulation

Due to the asymmetry of the transients, ISI values greater than t_rise (the time when the fluorescence transient h(t) reaches its maximum) result in visually distinguishable transients. Therefore, in this paper we are interested in the range of values ISI< t_rise. For the bi-exponential model described in (2) and according to GCaMP6s parameters in Table II, we have

$$t_{rise}={\tau }_{on}\ln \left(1+{\tau }_d/{\tau }_{on}\right)\cong 0.2s.$$

Numerical evaluation of the smallest detectable ISI depends on the selection of P_D and P_F. We set the number of false positives to be equal to the number of misses, relating P_D and P_F through

$$P_F=p\left(H_1\right)/\left(1-p\left(H_1\right)\right)\left(1-P_D\right),$$ (8)

where p(H₁) is the probability of the H₁ hypothesis. Assuming a Gamma distribution for ISI values [35], p(H₁) is the probability of ISI < t_rise, calculated by

$$p\left(H_1\right)=\underset{x=0}{\overset{0.2}{\int }}\mathrm{Gamma}(k,c)dx,$$ (9)

were the Gamma distribution is defined in (7). We determined the parameters k and c using the dataset from section II.C. Since this dataset was obtained from anesthetized mice (which include very large ISIs not observed in awake state), we used only ISI values less than 1 ms for estimating biologically plausible values for the Gamma distribution in awake mice. Fitting Gamma distributions to the ISI values of individual neurons, we estimated a mean value of k = 1 and c = 0.2. Substituting these values in (9) and considering a high detection threshold of P_D = 0.99 for (8) result in P_F = 0.017; the same values of P_D and P_F are used for analyzing the resolution limits of GCaMP6s and GCaMP6f.

It is illuminating to see how well the GLRT detector performs compared to the best optimal detector (LRT), in which all the parameters are known. Fig. 3 (b) shows the smallest detectable ISI (ISI_min) of both sensors for the symmetrically located spikes with equal known amplitudes versus SNR for an AOD scanner operating at f_s = 500 Hz. The results were obtained by generating receiver operating characteristic (ROC) curves from 2000 Monte-Carlo simulations at each SNR and ISI sampled with 0.5 ms spacing. ISI_min was determined by the smallest ISI value for which the corresponding ROC curve satisfied P_D = 0.99 and P_F = 0.017. Comparing the two detectors of each sensor, Fig. 3 (b) suggests that GLRT performs very close to the optimal detector. It also shows that we can accurately resolve ISI values much smaller than the fluorescence waveforms’ rise times (t_rise ≅ 200 ms and 45 ms for GCaMP6s and GCaMP6f, respectively) at different levels of SNR. In particular, at SNR = 3 obtained from the GCaMP6s dataset, the detector distinguishes fluorescence waveforms with ISI as small as about 60 ms. Similarly, for SNR = 2 obtained from GCaMP6f dataset, we can distinguish waveforms with ISI as small as 40 ms.

Fig. 3 (c) compares two cases of the known amplitudes, namely, d₁ = d₂ and αd₁ = βd₂, for different combined SNR levels (i.e., sum of the two transients’ SNRs) and amplitude ratios between the waveforms. The α ≠ β case gives better detection performance under the d₁ = d₂ condition, suggesting that at a fixed SNR level, we can resolve smaller ISIs compared to the equal amplitudes case. This result was also reported in [24] for a symmetric PSF. As explained in section II.B.1, for the case of α ≠ β with d₁ = d₂, the H₀ hypothesis is not located in the most challenging distance between two signals of the H₁ hypothesis, making the detection problem easier. However, when the test is conducted according to (4), the α ≠ β case is a more challenging problem compared to α = β. That is, with the same SNR level, the detector can resolve a larger ISI. This result emphasizes the importance of the H₀ hypothesis in the performance of the detector.

For the case of unknown amplitudes with prior probability distribution, as explained in section II.B.2, the performance of the detector is characterized by averaging P_D and P_F. Since a closed form expression is not available for P_D and P_F relating them to α and β, Monte-Carlo simulation with f_s = 500 Hz was used to numerically solve the problem. At each SNR value, 200 independent values of α and β were drawn from their prior distribution. For each draw at each SNR and ISI sampled with 0.5 ms spacing, 2000 simulations were executed, and the results are shown in Fig. 3 (d). Comparing the result of this problem with the known amplitude case for both sensors, we note that the prior knowledge about the amplitudes in the random case has resulted in a performance very close to the known case, with the latter slightly outperforming the former especially at the low SNR = 2. In all cases, the utilized detector can distinguish the presence of two spikes at ISIs much smaller than the fluorescence waveforms’ rise times. At the SNR levels of the GCaMP6s and GCamp6f datasets (SNR = 3 and 2, respectively), the detector for the general case of random amplitudes, on average, detects two fluorescence waveforms that are about 70 ms and 60 ms apart.

We compare the detection performance of different recording speeds under equal dwell time and baseline photon emission rates in Fig. 3 (e). Results from this analysis indicate that higher recording speeds can resolve significantly smaller ISI values for both calcium sensors at low photon emission rates. Nonetheless, experimentalists equipped with a conventional recording system can attain resolution limits smaller than the fluorescence waveform’s rise time by imaging a smaller field-of-view and thus, increasing dwell time and SNR. Overall, when designing experiments, sensor properties, SNR, and frame rate should all be considered to achieve the desirable spike detection performance.

2). Detector Performance on Experimental Dataset

We applied the formulated detector under the unknown amplitudes case on the experimental data described in section II.C. The data extraction pipeline resulted in 82 and 258 two spike samples for GCaMP6s and GCaMP6f expressing neurons, respectively. In analyzing experimental data, the test origin needs to be determined first. This can be done by finding the maximum point of cross-correlation between individual signals and s₀(t_k). To avoid erroneous calculation of the time origin due to noise, we used the true spike times to center the extracted signals on t = 0

We determined the detection threshold based on a desired value for P_F common between all SNR values. This can be done because the probability distribution of the log-likelihood ratio under the H₀ hypothesis is independent from the true values of parameters defining the model under H₀ [27]. As an illustrative example, we performed the detection problem by setting P_F = 0.3. Fig. 4 (a) illustrates examples of one and two spike fluorescent signals that were either correctly or incorrectly labeled by the detector. Fixing the detection threshold at the desired P_F level, different theoretical P_D values are derived from the ROC curves of 2000 simulated data for each different SNR and ISI pair values (Fig. 4 (b) illustrates the case for GCaMP6f). The detector achieved total detection rates of 0.74 and 0.87 for GCaMP6s and GCaMP6f datasets, respectively. It also resulted 0.26 in (GCaMP6s) and 0.37 (GCaMP6f) total false positive rates, which are approximately the expected values from setting P_F = 0.3.

Fig. 4.

Two spike detection results from the experimental dataset. (a) Examples of true positive (TP), false negative (FN), false positive (FP), and true negative (TN) from the GCaMP6f dataset. Vertical red lines correspond to spike times in the two spike cases. (b) GCaMP6f two spike samples overlaid on heat map of P_D versus SNR and ISI at a fixed P_F. Green circles are true positive samples, whereas false negatives are shown with navy (n = 258 samples). The detector obtained 0.87 true positive rate and 0.37 false positive rate. (c) The detector approximately achieves the expected performance calculated through theoretical analysis. Error bars indicate 68% confidence intervals. Gray shaded areas denote discretized intervals. Number of samples in each interval is written al the corresponding interval. All results are obtained at fixed P_F = 0.3

To further compare the detector’s expected performance through theoretical analysis to the observed performance on experimental data, we took the following steps. First, we grouped the two spike data points based on their theoretical P_D values (P_D,Expected). Since P_D is a continuous variable, we discretized the values by rounding to obtain the sample groups. Next, real P_D value (P_D,Real) for each group was calculated as the percentage of samples correctly detected as two spikes in each group. We utilized the binomial confidence interval to assess the P_D,Real values [36]. Since the number of samples in each group was relatively small, we used the 68% confidence interval corresponding to distribution of data within one standard deviation of the mean to assess whether our detector attained performance close to the theoretically predicted performance. Our analysis revealed that the detector’s detection performance on experimental data was indeed close to that predicted in theory (Fig. 4 (c)), as the P_D,Expected values fall in the confidence intervals of P_D,Real

C. Prior Knowledge about Signal Amplitudes Yields Theoretically Equal ISI Estimation Performance to the Known Case

In this subsection, we present the CRB and HCRB for the known and the random amplitude cases, respectively. Fig. 5 (a) illustrates the effect of amplitude ratios on $\sqrt{\text{CRB}}$ for the α ≠ β case under the two conditions, d₁ = d₂ and αd₁ = βd₂, with fixed ISI = 60 ms and combined SNR = 8 for GCaMP6s. As the ratio between the amplitudes diverges from one, CRB gets larger for the αd₁ = βd₂ case, whereas it decreases in d₁ = d₂ (similarly for GCaMP6f illustrated in Supplementary Fig. 1 (a)). This result is similar to the result in Fig. 3, where we emphasized on the effect of defining the H₀ hypothesis.

Fig. 5.

Lower bounds on ISI estimation for GCaMP6s at f_s = 500 Hz. (a) $\sqrt{\text{CRB}}$ for two cases of known and unequal amplitudes versus the amplitude ratio at combined SNR = 8 and ISI = 60 ms. Estimating ISI from two unequally bright transients that are symmetrically located around the test origin gives better precision. (b) $\sqrt{\text{CRB}}$ and $\sqrt{\text{HCRB}}$ for the case of known and random amplitude cases, respectively, versus (left) ISI at combined SNR = 8, and (right) SNR at ISI = 60 ms (αd₁ = βd₂). An optimized ISI estimator with a random but known prior distribution about the amplitudes asymptotically performs similar to an optimized unbiased estimator of ISI with known α = β.

We complete the derivation of the HCRB described in section II.D.2 by calculating I_P according to (6). Based on the i.i.d assumption of α and β, and their independence from ISI, only the second and third diagonal elements of I_P corresponding to α and β are none-zero and equal. Referring that these α,β ∼ Gamma(k,c),these two elements are derived as

$${\mathbf{I}}_{P2,2}={\mathbf{I}}_{P3,3}=-E\left\{\frac{{\partial }^2\ln (p(\alpha ))}{\partial {\alpha }^2}\right\}=\text{E}\left\{(k-1)/{\alpha }^2\right\}.$$

Note that $x\triangleq 1/\alpha \sim \mathrm{InvGamma}\left(k,c^{-1}\right)$ with the probability distribution function defined as

$$f\left(x;k,c^{-1}\right)=c^{-k}{x}^{-k-1}\exp \left(-c^{-1}/x\right)/\mathrm{\Gamma }(k),x>0.$$

The mean and variance of this distribution are c⁻¹/(k−1) (for k>1) and c⁻²/[(k−1)²(k−2)] (for k>2), respectively [34]. Thus, we arrive at

$$\begin{gathered} {\mathbf{I}}_{P2,2}={\mathbf{I}}_{P3,3}=\text{E}\left\{(k-1)/{\alpha }^2\right\} \\ =(k-1)\left[\mathrm{var}(1/\alpha )+{(\text{E}(1/\alpha ))}^2\right]=c^{-2}/(k-2), \end{gathered}$$

for k>2. Thus, I_P is attained as

$${\mathbf{I}}_P=\left[\begin{array}{ccc}0& 0& 0\\ 0& c^{-2}/(k-2)& 0\\ 0& 0& c^{-2}/(k-2)\end{array}\right].$$

We compare the $\sqrt{\text{HCRB}}$ of the random amplitude case with $\sqrt{\text{CRB}}$ of the known and equal amplitude case at combined SNR = 8 and ISI = 60 ms in Fig. 5 (b) for GCaMP6s; It is seen that the two bounds are nearly identical (mean±std difference of 0.2±0.05 ms (right) and 0.05±0.07 ms (left)). Similar results are obtained for GCaMP6f lower bounds, as illustrated in Supplementary Fig. 1 (b) (mean±std difference of 0.24±0.04 ms (right) and 0.05±0.09 ms (left)). We thus conclude that an optimized unbiased estimator of ISI with known α = β asymptotically performs similar to an optimized ISI estimator with a random but known prior distribution about the amplitudes.

D. Maximum Likelihood and Maximum a Posteriori Estimators Closely Approach the Theoretical Bounds

In this section, we compare the performance of ML and MAP estimators with their corresponding lower bounds. Fig. 6 (a) and (b) show the comparison of bias and standard deviation of GCaMP6s ISI estimation (through 5000 Monte-Carlo simulations) to the $\sqrt{\text{CRB}}$ limit, assuming symmetrically located spikes with known amplitudes fixed at combined SNR = 20. Except for very small ISI values in Fig. 6 (a), the results show that the ML estimator is unbiased and its standard deviation is very close to the lower limit, emphasizing its ability to achieve the theoretically best possible precision. However, for small values of ISI in the α = β problem, the standard deviation of ISI estimations becomes smaller than the lower limit. Similar results are obtained for GCaMP6f as illustrated in Supplementary Fig. 2 (a) and (b)). Note that in the α = β problem, the maximization problem in (3) has two answers: ISI = d and ISI = -d. The ML estimator achieves asymptotic consistency and efficiency under certain conditions; one is that the maximum of (3) should be unique [37]. For large ISI values, where the two peaks are relatively far from each other, iterative optimization methods used to numerically solve the maximization problem converge to one of the two peaks depending on the starting point. Therefore, we may assume a “unique” peak at the local region around either one of the maximums where the consistency and efficiency properties hold. However, as ISI gets smaller and the precision of estimation decreases, observation noise will deviate the peak locations of the log-likelihood function towards ISI = 0. In the presence of such bias, the comparison of ML variance to CRB is theoretically invalid. We specify the boundary of the valid region for ML and CRB comparison based on the simulation results presented in Supplementary Fig. 3 for GCaMP6s and Supplementary Fig. 4 for GCaMP6f. The histograms of numerically calculated ISI values at combined SNR = 20 show that for small values of ISI, a discernible peak appears at ISI = 0. This results from the estimator being trapped around zero.

Fig. 6.

ML and MAP estimators nearly achieve the information theoretic bounds. Results obtained from 5000 Monte-Carlo simulations at combined SNR = 20 for GCaMP6s. Standard deviation (std) and bias of ISI estimation compared to $\sqrt{\text{CRB}}$ for known values of (a) α = β and (b) β = 3α with d₁ = d₂ at f_s = 500 Hz. RMSE of (c, left) ISI and (c, right) α and β estimations compared to their $\sqrt{\text{HCRB}}$, with αd₁ = βd₂ at f_s = 500 Hz. Dashed-dot vertical line shows the smallest ISI for which the CRB comparison is valid. (d) Standard deviation of estimating ISI from simulated dataset compared to $\sqrt{\text{CRB}}$ for the case of equal amplitudes with different recording rates. Higher frame rates bring us closer to the information theoretic lower bound compared to slower speeds. Only valid regions are depicted.

At ISI > 8 ms (GCaMP6s) and ISI > 4 ms (GCaMP6f) the peak at zero becomes less prominent (peak height becomes smaller than half of the height at true value), reducing the bias. Therefore, we determine ISI = 8 ms and 4 ms as the boundary of the valid region of ML and CRB comparison for GCaMP6s and GCaMP6f with 500 Hz recording speed, respectively. These boundaries are illustrated in Fig. 6 (a) and Supplementary Fig. 2 (a) for the equal amplitude cases. The estimations to the left of these boundaries have considerable bias, therefore making the comparison of the standard deviation to the $\sqrt{\text{CRB}}$ invalid.

Fig. 6 (c) compares the root-mean-squared error (RMSE) of the GCaMP6s parameter estimations to the $\sqrt{\text{HCRB}}$ limits versus ISI for the case of random amplitudes with αd₁ = βd₂ and combined SNR = 20 and f_s = 500 Hz. The results suggest that the simultaneous ML and MAP estimation of ISI and the amplitudes achieves very close performance to the asymptotic limit, especially for α and β. In the small ISI region, bias in the amplitude estimations towards α = β leads to the previous problem of non-unique solution for ISI. Therefore, we included the valid boundary as in Fig. 6 (a) for completeness. The comparison of the results on the left of this line to the boundary is not valid. Similar results are obtained for GCaMP6f as illustrated in Supplementary Fig. 2 (c).

Finally, we analyze the information theoretic lower bound for different recording systems. Under the equal amplitude case, Fig. 6 (d) compares the calculated standard deviation of ISI estimation through 5000 Monte-Carlo simulations to the lower bounds for the GCaMP6s sensor at the fixed combined SNR = 20. At the same SNR level, the CRB for ISI estimation using higher recording rates is smaller compared to lower recording rates. More importantly, the very high 500 Hz recording speed comes very close to achieving its estimation lower bound, as can be seen from the small distance between the calculated standard deviation and the theoretical lower bound. Similar results are obtained for GCaMP6f as shown in Supplementary Fig. 2 (d).

IV. DISCUSSION

Calcium sensor kinetics and SNR significantly impact spike detectability and precision of spike time estimation [17]. Thus, there is a need for understanding the theoretical resolution limits of detecting closely-timed neuronal spikes from fluorescence signals. Similar to studies that have shown resolutions beyond the Rayleigh limit is possible in optical imaging systems [22–26], this study showed that by using the statistical approach, attaining resolution finer than the peak time of the indicator is possible. While this result was expected from a previous algorithm conducted on OGB-1 labeled neurons which assumed uniform calcium spike responses [16], our detector achieved equal performance considering randomly distributed calcium responses. The latter scenario better matches the true, stochastic response of calcium indicators during live animal experiments. The CRB lower bounds on the variance of ISI estimation further verified the results of the detection framework.

Our detection theoretic framework assumed no definite knowledge about the peak value of a single spike, which is beneficial for modeling experiments with no ground truth available. This was a particularly challenging case since the peak amplitude of the single spike in the H₀ hypothesis is comparable to the peak amplitude of the signal generated by two spikes in the H₁ hypothesis. Thus, a simple decision between H₀ and H₁ based on amplitude alone, especially in low SNRs, cannot provide accurate results. We showed that utilizing the signal’s temporal information, as modeled through s₀(t;θ₀), and s₁(t;θ₁) enabled accurate detection.

The resolution limits and estimation bounds were estimated based on a set of experimentally derived parameters. We determined the detection criteria, i.e. P_D and P_F, by relating them through the prior probabilities of the two hypotheses, which were derived using the available dataset with ground truth spike times. The prior probabilities derived in our work are applicable to this specific data and need to be recomputed for any new experiment. In general, accurate information about the spiking behavior of the neurons might not be available. In such events, experimentalists can use any desirable values for P_D and P_F to derive the resolution limits of detecting temporally overlapping fluorescence waveforms. A good performing detector is one with very high P_D (usually above 0.9) and low P_F (such as 0.01). In general, a very high P_D along with a very low P_F value will make the detection of two spikes a harder problem, resulting in larger resolution limits (i.e. ISI_min).

Our model was based on Poisson statistics of the signals, which is generally true for shot-noise limited recordings. Importantly, we have demonstrated that our formulated detector performs as expected on experimentally-obtained datasets. However, under certain conditions this assumption can be violated. For example, some signal extraction methods are based on the weighted average of multiple pixel values, which would generate signals that are not purely Poissonian. Another case is when neuropil contamination is removed by subtracting the average pixel values around the neuron soma. Nevertheless, our formalism should allow incorporation of other noise models in future work.

We have assumed linear relationship between calcium dynamics and fluorescence response. In general, this relationship is non-linear and sensor saturation occurs at very high neuronal firing rates. This effect is especially pronounced for past generations of protein calcium sensors with high dissociation constant. For the case of GCaMP6 sensors considered in our work, which are currently the best GECIs due to their favorable properties, the fluorescence response is linear in the low spike regime [1]. Therefore, the non-linear dynamics and saturation assumptions are not necessary for the work presented in this paper, which deals with one and two spike cases. Much like our discussion of extraneous noise sources, we could potentially incorporate such non-linear models of sensor response into our models as well.

This work is the first step in the continuum research to utilize detection theoretic tools to set the optimal resolution limits for temporally overlapping fluorescence signals. Future work will extend the current framework to the more general case of more than two spikes. Such analysis should take into account the non-linearity and saturation effect [38, 39].

V. CONCLUSION

In this paper, we addressed the problem of accurately detecting two AP evoked fluorescence transients from a statistical viewpoint. We employed a previously introduced hypothesis testing framework to tackle the resolution problem. We also presented the asymptotic performance of the parameter estimators using the traditional and Hybrid CRBs.

Through simulation and experimental data across different SNR levels and recording speeds, we showed that the detectors can accurately distinguish two signals with ISI on the order of tens of milliseconds, shorter than the waveform’s rise time. Our work could also better inform the choice of calcium indicator used in neuroscience experiments. The latest GCaMP6 calcium sensors present a trade-off between sensor kinetics and SNR: GCaMP6s is significantly slower than other sensors such as GCaMP6f, but has superior ΔF/F response. Choice between these two sensors, for example, has previously relied on the emphasis of one of the two metrics. Our work suggests that instead of assessing the kinetics of the sensor as the sole metric of temporal resolution, additional analysis using recording speed, sensor kinetics, SNR, and the sensor’s stochastically distributed response to action potentials can accurately resolve ISI values much smaller than the fluorescence waveform’s rise time. Such analysis aids not only in future spike detection, but also in future experimental design when choosing sensors of neuronal activity. Even as GECIs continue improving, our work should maintain relevance by assessing the achievable temporal resolution of an imaging experiment.