Methods for Assessment of Memory Reactivation

Abstract

It has been suggested that reactivation of previously acquired experiences or stored information in declarative memories in the hippocampus and neocortex contributes to memory consolidation and learning. Understanding memory consolidation depends crucially on the development of robust statistical methods for assessing memory reactivation. To date, several statistical methods have seen established for assessing memory reactivation based on bursts of ensemble neural spiking activity during off-line states. Using population-decoding methods, we propose a new statistical metric known as the “weighted distance correlation” to assess hippocampal memory reactivation (i.e., “spatial memory replay”) during quiet wakefulness and slow wave sleep. The new metric can be combined with an unsupervised population decoding analysis such that it can detect statistical dependency beyond linearity in the memory traces and is invariant to latent state labeling. We validate the new metric using two rat hippocampal recordings in spatial navigation tasks. Our proposed analysis framework may have a broader impact on assessing memory reactivations in other brain regions under different behavioral tasks.

1. Introduction

Memory consolidation is a fundamental process for transferring previously acquired experiences into long-term memory, and has been implicated in various forms of learning and memory, including episodic, declarative, procedural and emotional memories. Many brain areas, such as the hippocampus, neocortex, and amygdala, contribute to memory consolidation (McGaugh, 2000, 2015). A hallmark of memory consolidation is memory reactivation—reactivation of spatiotemporal neuronal spike patterns during off-line states, such as quiet wakefulness (e.g., Kudrimoti et al., 1999; Nadasdy et al., 1999; Foster & Wilson, 2001; Davidson et al., 2009; Carr et al., 2011; Kitamura et al., 2017) and sleep (e.g., Wilson & McNaughton, 1994; Skaggs & McNaughton, 1996; Louise & Wilson, 2001; Lee & Wilson, 2002; Rothschild et al., 2017).

In a standard WAKE/SLEEP experimental paradigm (Fig. 1A), animals are trained to perform a specific task behavior during WAKE. Before and right after the task, animals are put into a sleep box. During the whole period, neuronal ensemble activity, including spikes and local field potentials (LFPs), are recorded from the target brain areas. The scientific question of memory reactivation is that how to assess the memory reactivation based on the recorded ensemble neural spiking activity in WAKE and SLEEP alone?

Figure 1:

(A) General paradigm of in WAKE/SLEEP experiments. The scientific question is to assess the “memory codes” during sleep. (B) Graphical overview of the method for assessment of memory reactivation. Step 1: Neural spike trains of a population are fit by a hidden Markov model (HMM) with Bayesian inference. Upon completion of inference, a state transition matrix and firing rate matrix are derived. Step 2: From the candidate event of memory reactivation, bursts of ensemble neural spiking activity are analyzed by a maximum likelihood decoding estimator to infer a state sequence. The derived state sequence is assessed by specific statistical metrics (e.g., weighted distance correlation) followed by a significance test from randomly shuffled data.

To assess memory reactivation in the hippocampus and neocortex, various statistical methods have been developed, including correlation analysis (Wilson & McNaughton, 1994), template matching (Nadasdy et al., 1999; Louie & Wilson, 2001; Euston et al., 2007; Wilber et al., 2017), sequence matching (Lee & Wilson, 2002, 2004), principal component analysis (PCA) (Peyrache et al., 2009, 2010; Gulati et al., 2014; Ramanthan et al., 2015), independent component analysis (ICA) (Lopes-dos-Santos et al., 2013; van den Ven et al., 2016), and population decoding method (Davidson et al., 2009; Wu & Foster, 2014). Typically, for most of methods, the goal is to establish a structure or template in a behavioral (WAKE) state and to seek the “similarity” (by correlation or matching) between the preexistent structure and the tested candidate events during off-line states, such as slow wave sleep (SWS). However, it is noted that except for the population decoding method, other methods cannot identify the content of reactivated memory other than merely establishing significant similarity of spike activities between two states (e.g., WAKE vs. SLEEP). In other words, they can reveal the presence of memory replay but not necessarily the content of replay. A detailed discussion of these methods is referred to (Chen & Wilson, 2017).

In this paper, we proposed an unsupervised population decoding framework and associated statistical methods for assessing memory reactivation of population codes in the hippocampus and neocortex. We focus on the population-decoding method for assessing hippocampal or neocortical memory reactivation. One type of population decoding methods is supervised and based on neuronal receptive fields (RFs), another type is unsupervised and requires no RF assumption. Both methods have been used for identifying rodent hippocampal memory reactivations (e.g., Davidson et al., 2009; Wu & Foster, 2014; Grosmark & Buzsáki, 2016; Chen et al., 2016). We make critical observations on the existing statistical measures and further propose a new statistical measure for assessing memory reactivation. The new measures are motivated to resolve two issues. First, the current correlation-based measure is restricted by the linearity assumption, which can pose challenges when linearizing a complex spatial environment, such as open field, T-maze, and radial maze. In other words, spatial trajectory between two neighboring points in space can be discontinuous due to an arbitrary choice of linearization. The second motivation is to study memory reactivation in non-spatial tasks, in which it is difficult to define linear or nonlinear structure of the abstract event episode. As we will discuss in the remaining paper, our proposed measure is capable of capturing statistical dependency of temporal trajectory points beyond the linearity assumption and is also invariant to latent state labeling or permutation. In developing these methods, we have used hippocampal memory reactivation as a working example. We validate our methods using both synthetic examples and rat hippocampal recordings during spatial navigation and sleep.

2. Assessment of Memory Reactivation

2.1. Population decoding methods

Supervised RF-based population decoding.

The standard population decoding paradigm consists of encoding and decoding phases. In the encoding phase (where spikes and behavioral measures are collected), neuronal RF models are established (Zhang et al., 1998; Brown et al., 1998; Zemel et al., 1998; Barbieri et al., 2004; Agarwal et al., 2016). In the decoding phase, maximum likelihood or Bayesian methods are used to decode the stimulus based on the ensemble neural spiking activity and the established RFs (termed Decode_wRF). The encoding phase is to pre-identify the structure or referential content of spikes (i.e., neuronal RF), and the decoding phase is to reconstruct the memory.

Unsupervised RF-free population decoding.

The second population decoding method is unsupervised and requires no neuronal RF information or explicit behavioral measures (termed Decode_woRF). This is achieved by constructing a hidden Markov model (HMM) and associating spatiotemporal spiking patterns with unique latent states without defining the stimulus-specific content of those states a priori. Take rodent navigation as an example, we assume that the latent states follow Markovian or semi-Markovian transition dynamics in a finite state space $S=\left\{S_i\right\}$ (where $S_i\in \left\{1,2,\dots ,m\right\}$ are categorical variables) (Chen et al., 2012, 2014; Chen, Linderman & Wilson, 2016). Animal’s run trajectories across spatial locations (“states”) are associated with consistent hippocampal ensemble spiking patterns, which are characterized by a stationary state transition matrix defining $p\left(S_t|S_{t-1}\right)$. The observed spike count data, y_t, is assumed to follow a state-dependent Poisson probability distribution $p\left({\mathbf{y}}_t|S_t\right)$. The goal of inference is to estimate the maximum a posteriori (MAP) state sequences and the unknown state transition matrix P and firing rate parameters at each state. To estimate the number of latent states m directly from data, we have developed a Bayesian nonparametric version of the HMM: hierarchical Dirichlet process (HDP)-HMM, combined with advanced Markov chain Monte Carlo (MCMC) inference algorithms (Linderman et al., 2016).

In a general term, the problem of assessing memory reactivation can be described as follows: Given ensemble neural spike recordings of hippocampal or neocortical circuits during a specific (either spatial or non-spatial) task and during offline states (QW or sleep), how do we assess the structure or content of memory reactivation during QW or sleep? Overall, our procedure of unsupervised learning for assessing memory reactivations consists of two steps: (i) Apply the unsupervised population decoding analysis to infer latent structures of population codes from the hippocampus or neocortex. (ii) Apply a memoryless maximum likelihood (ML) decoding analysis to extract “memory traces” during the candidate events; apply statistical metrics to assess the fidelity or goodness-of-fit of the memory traces, and test their statistical significance. These two steps are graphically illustrated in Fig. 1B. We will address the second step in the following subsections.

Statistical assumptions.

Our unsupervised population decoding analysis for assessing memory reactivation is built upon a few important statistical assumptions. Without the behavioral or RF measures, we assume that the spike trains from neuronal populations observed in WAKE and SLEEP (or quiet wakefulness) share a similar data structure in their state firing rate map, except that they are subject to a proper time compression or scaling. Furthermore, when only a subset of neurons subset of neurons participate in memory reactivation, events of “sparsified spike trains” can be viewed as random realizations of the wake experiences subject to time compression or velocity variation. Finally, the memory replay events have a similar state-transition matrix as real events, but the timing of these state transitions is not required to be consistent with constant velocity transitions. These assumptions also imply that our analysis will hold for both WAKE→SLEEP and SLEEP→WAKE directions.

In addition, we assume that there are adequate behavioral sampling and sufficient number of recorded neurons that warrant the proposed unsupervised decoding analysis.

2.2. Statistical measures

Linear line fit.

In one-dimensional (1D) spatial navigation task, rodent hippocampal memory activation is expressed by a trajectory across time. The quality of the trajectory (known as “replay score”) can be determined by goodness of the best linear fit. For instance, the replay score may correspond to the mean estimated likelihood that the animal is on the specified trajectory. The test of statistical significance for the observed replay score is compared with sample distributions of score obtained after randomly shuffling the spike data (Davidson et al., 2009).

Weighted correlation.

Correlation measures have been proposed for measuring the hippocampal memory replay. Since the estimated spatial position is associated with a unimodal or multimodal nonnegative posterior probability (Zhang et al., 1998; Zemel et al., 1998), one can use these weights to quantify the strength of correlation between the changes in probability values across time and reconstructed spatial position (Wu & Foster, 2014; Grosmark & Buzsáki, 2016; Chen et al., 2016).

Given a set of paired scalar random variables $\left\{u_i,v_i\right\}$, assume that the observations to be correlated have different degrees of weights $\left\{w_i\right\}$ (where $0\le w_i\le 1$), the weighted mean $m_w$, weighted covariance $Co{v}_w$ and weighted correlation R_w are defined respectively as follows:

$$m_w\left(u\right)=\frac{{\sum }_i{w}_i{u}_i}{{\sum }_i{w}_i}$$ (2.1)

$$Co{v}_w\left(u,v\right)=\frac{{\sum }_i{w}_i\left(u_i-m_w\left(u\right)\right)\left(v_i-m_w\left(v\right)\right)}{{\sum }_i{w}_i}$$ (2.2)

$$R_w\left(u,v\right)=\frac{Co{v}_w\left(u,v\right)}{\sqrt{Co{v}_w\left(u,u\right)Co{v}_w\left(v,v\right)}}$$ (2.3)

where $-1\le R_w\le 1$. However, similar to Pearson’s correlation, the weighted correlation R_w is subject to linear assumption between random variables u and v. In the hippocampal replay example, u corresponds to time (horizontal axis), v corresponds to the animal’s position. Due to the linearity assumption, this metric is insufficient for assessing the quality of a discontinuous or curved trajectory. The discontinuity may be induced by arbitrary linearization of the environment, such as the circular track (e.g., Grosmark & Buzsáki, 2016) or Y-maze (e.g., Wu & Foster, 2014). The nonlinear continuity may be induced by time warping (e.g., induced by varying velocity at different time bins), yielding a sigmoid-shape trajectory (unpublished data, M.A. Wilson, personal communication).

Distance correlation.

The distance correlation was initially developed to quantify statistical dependence between two random variables (or two random vectors of arbitrary dimensions) (Székely & Rizzo, 2009). The distance correlation is derived from distance variance $Co{v}_d\left(u,v\right)$ and distance covariance $Co{v}_d\left(u,v\right)$ between random variables u and v:

$$R_d\left(u,v\right)=\frac{Co{v}_d\left(u,v\right)}{\sqrt{Co{v}_d\left(u,u\right)Co{v}_d\left(v,v\right)}}$$ (2.4)

where $0\le R_d\le 1$. In theory, R_d is zero if and only if u and v are statistically independent; and R_d = 1 is the subspaces spanned by U and V are almost surely equal. In addition, R_d is invariant to scaling and translation and has theoretic links to kernel-based distance metric (Sejdinovic et al., 2013).

To compute the distance covariance for (u_i, v_i) (i = 1,…,n), we need to define two distance metrics, dist_u and dist_v, for all pairwise distances: dist_u(u_i, u_j) and dist_v(v_i, v_j) (Appendix A). A common choice of the distance metrics can be Euclidean distance, chi-squared distance, Earth Mover’s distance, and L₁ distance (e.g., MATLAB function “pdist2”). Moreover, the distance metrics used for u and v may be either identical or different. We can also define a custom distance measure as long as it satisfies all four axioms: (i) dist(u, u) ≥ 0; (ii) dist(u, v) = 0 ⟺ u = v; (iii) dist(u, v) = dist(v, u); and (iv) dist(u, z) ≤ dist(u, v) + dist(v, z).

Notably, the estimation of R_d based on finite samples is statistically biased. Therefore, a bias-corrected distance correlation has been used in practice (Székely & Rizzo, 2013). Given n > 3 samples, the new bias-corrected measure is given by

$$R_d^{\ast }\left(u,v\right)=\{\begin{array}{cc}\frac{Co{v}_d^{\ast }\left(u,v\right)}{\sqrt{Co{v}_d^{\ast }\left(u,u\right)Co{v}_d^{\ast }\left(v,v\right)}},& \mathrm{if}Co{v}_d^{\ast }\left(u,u\right)Co{v}_d^{\ast }\left(v,v\right)\ne 0\\ 0,& \mathrm{if}Co{v}_d^{\ast }\left(u,u\right)Co{v}_d^{\ast }\left(v,v\right)=0\end{array}$$ (2.5)

where $Co{v}_d^{\ast }\left(u,v\right)$ denotes the modified distance covariance (Appendix A). From now on, we use the notation $R_d^{\ast }$ to represent the bias-corrected R_d.

Distance metrics.

For assessing a spatial memory reactivation event, one variable is time t (horizontal axis), another variable is spatial position (vertical axis), and n corresponds to the number of time bins. For the time variable, we consider two empirical distance metrics. The first one is based on absolute time lag between two points:

$$d_{ij}\left(t_i,t_j\right)=\left|t_i-t_j\right|,$$ (2.6)

which is a linear measure. The second one is based on a bounded piecewise distance

$$d_{ij}\left(t_i,t_j\right)=\{\begin{array}{ll}\frac{d_{\max }}{3}\left|i-j\right|,& \mathrm{if}0\le \left|i-j\right|\le 3\\ d_{\max },& \mathrm{if}\left|i-j\right|>3\end{array}$$ (2.7)

which only considers the neighboring time bins (less than 4 bins apart, Fig. 2A). The constant d_max controls the maximum distance and the slope.

Figure 2:

(A) Temporal distance metric d_ij(t_i, t_j) in equation 2.7. (B) Distance metric d_ij(S_i, S_j) in equation 2.9 with P_max = 0.5 (solid) and P_max = 1 (dashed). (C) Distance metric d_ij(S_i, S_j) in equation 2.10 with varying values of scaling parameter α. Here, d_max = 2.

For the spatial position variable, we use different distance metrics for Decode_wRF and Decode_woRF. Since the decoded position is known in Decode_wRF, we may compute the actual distance between two positions v_i and v_j:

$$d_{ij}\left(v_i,v_j\right)=\left|v_i-v_j\right|.$$ (2.8)

In the standard distance correlation, distance metrics of equations 2.6 and 2.8 are used (Appendix A).

For Decode_woRF, the decoded position is unknown, and we only have the inferred states $\left\{S_i\right\}\in \left\{1,\dots ,m\right\}$ (which are subject to permutation ambiguity). However, from the estimated $m\times m$ state transition $\mathrm{matrix}P=\left\{P_{ij}\right\}$, we may define a distance matrix using a piecewise linear transformation:

$$d_{ij}\left(S_i,S_j\right)=\{\begin{array}{cc}\frac{d_{\max }}{2P_{\max }}\left(2P_{\max }-{\rho }_{ij}\right),& \mathrm{if}{\rho }_{ij}\le P_{\max }\mathrm{and}{S}_i\ne S_j\hfill \\ 0,& \mathrm{if}{\rho }_{ij}>P_{\max }\mathrm{or}{S}_i=S_j\hfill \end{array}$$ (2.9)

where $P_{\max }={\max }_{ij}\left\{{\tilde{P}}_{ij}\right\}$ denotes the maximum value of matrix $\tilde{P}$, and $\tilde{P}$ is obtained by removing self-transition probability $\left\{P_{ii}\right\}$ in the diagonal and renormalized in each row. The parameter ${\rho }_{ij}=\frac{{\tilde{P}}_{ij}+{\tilde{P}}_{ji}}{2}$ denotes the average state transition probability between state i and state j (such that $\left\{{\rho }_{ij}\right\}$ and $\left\{d_{ij}\left(S_i,S_j\right)\right\}$ are symmetric). We can prove that distance metric (2.9) satisfies all four axioms (Appendix B). In addition, d_ij(S_i, S_j) is invariant to the permutation of state label.

Equations 2.8 and 2.9 fulfill the distance triangle inequality. When the distance triangle inequality is not strictly imposed,¹ we may employ an empirical nonlinear transformation between ρ_ij and d_ij:

$$d_{ij}\left(S_i,S_j\right)=\{\begin{array}{cc}{d}_{\max }\exp \left(\frac{\alpha {\rho }_{ij}}{{\rho }_{ij}-1}\right),& \mathrm{if}{S}_i\ne S_j\\ 0,& \mathrm{if}{S}_i=S_j\end{array}$$ (2.10)

where the scale parameter α controls the shape of the curve (Fig. 2C) and the effective support, which plays a similar role of P_max in equation 2.9. Depending on the data range and distribution, we choose the scale parameter to maximize the discrepancy between two distinct inputs. When α = 1, the curve is close to a piecewise linear function.

Note that the average transition probability ${\rho }_{ij}=\frac{P_{ij}+P_{ji}}{2}$ is estimated from the ensemble spike data alone using Decode_woRF. However, if we estimate the transition matrix estimated from the actual behavioral data (namely, $P_{ij}=\mathrm{Pr}\left(v_i\left(t\right)|v_j\left(t-1\right)\right)$, where v(t) denotes the animal’s position at time t), Decode_wRF can also employ equation 2.9 or 2.10 to compute $d_{ij}\left(v_i,v_j\right)$.

Weighted distance correlation.

Motivated from weighted correlation R_w and distance correlation R_d, we further propose a new statistical metric called weighted distance correlation R_wd, which integrates a weighting strategy (as in R_w) in computing the distance metric (Appendix C). Using a bias correction procedure (Appendix A), we also derive a bias-corrected estimate of R_wd, denoted as $R_{wd}^{\ast }$.

Toy examples.

To gain some insight into the difference between three correlation metrics {$R_w,R_d^{\ast },R_{wd}^{\ast }$}, we construct six synthetic trajectories (arranged in the form of 57 spatial bins and 15-20 temporal bins) and compute three correlation metrics (shown in Fig. 3). Specifically, Figure 3A,C,D show linear or piecewise linear trajectories, each with a constant or varying slope; Figure 3B shows an example of two disconnected trajectories. Figures 3E and 3F are “noisy” negative examples.

Figure 3:

Comparison of R_w, $R_d^{\ast }$ and $R_{wd}^{\ast }$ metrics for six synthetic trajectory examples. In the title of top panels, three numbers represent [R_w, $R_d^{\ast }$, $R_{wd}^{\ast }$]. In the bottom panels, horizontal lines represent the result obtained from using equation 2.9 as the distance metric, whereas curves with markers represent the result obtained from using equation 2.10 with varying α as the distance metric. In Fig. 3D, dashed lines represent the results using bimodal weighting. For computing R_wd, we assume a known state transition matrix estimated directly from the animal’s run position.

It is noted that R_w fails to detect two disconnected patterns (Fig. 3B), or underestimates the correlation due to outliers (Fig. 3C). Sometimes, R_w also produces a spurious value for noise (Fig. 3F). The traditional distance correlation metric $R_{d1}^{\ast }$ (equations 2.6 and 2.8) produces a similar value as R_w in some cases (Figs. 3A, D, E), whereas $R_{d2}^{\ast }$ based on a “local” distance metric (equation 2.7) is more insensitive to global patterns (Figs. 3B and 3C) and focuses on local continuity (Fig. 3E). In contrast, $R_{wd}^{\ast }$ combines the features of $R_{d2}^{\ast }$ and $R_w^{\ast }$ and produces a better quantification for “nonlinear curves”. In addition, the linear (equation 2.9) and nonlinear (equation 2.10, α = 1) distance metric $R_{wd}^{\ast }$ produce nearly identical results. Finally, it is found that the weighting produces a more accurate characterization for multi-modal trajectories (Fig. 3D).

2.3. Assessment of significance

All statistical analyses are performed on the “candidate events” of memory reactivation. A candidate event can be determined by one or multiple criteria based on neural recordings, such as multi-unit activity burst or LFPs (e.g., hippocampal sharp-wave ripples or neocortical spindles or slow waves) (e.g., Ji & Wilson, 2007; Davidson et al., 2009).

An event is claimed to be a “statistically meaningful” memory reactivation event if it meets three established criteria for statistical significance:

• The |R_w| is greater than a threshold of 0.5 (Wu & Foster, 2014); no threshold is set for R_wd or $R_{wd}^{\ast }$.²

• The time length is at least 5 temporal bins with spike(s) in each bin (i.e., 100 ms for QW or SWS epochs) (Wu & Foster, 2014). Therefore, shorter candidate events are not considered in the current study, although they might also contribute to memory reactivation.

• In addition, we generate shuffled candidate events from each pre-identified candidate event, and compute the R_shuffle from randomly shuffled ensemble neural spiking activity. Two types of shuffling operations are considered: temporal shuffling and cell shuffling. Algebraically, the spike count matrix is subject to both row (temporal) and column (cell) shuffle operations. A total of 1000 shuffled samples are constructed for each shuffling operation. From the raw and shuffled statistics, we compute the Z-score for R as follows:

  $$Z=\frac{R-\mathrm{mean}of{R}_{\mathrm{shuffle}}}{\mathrm{SD\; of}{R}_{\mathrm{shuffle}}}.$$ (2.11)

  The Z-score of R_w, denoted as Z_w, is based on the computation of |R_w|. The Z-score of R_wd, denoted as Z_wd, is based on the uncorrected version of R_wd. In Appendix D, we show that Z_wd is unbiased due to bias cancellation.

  Assuming the null distribution of shuffle statistics is normally distributed, then the event is significant only if the Z-score of R is greater than a threshold 1.65 (which is equivalent to setting a one-tailed P-value 0.05, assuming the null distribution is normally distributed). However, when the null distribution is not normally distributed (e.g., having an asymmetric support), we use the Monte Carlo P-value to identify statistical significance. In reporting the number of statistical events, we use the Monte Carlo P-value criterion. For comparing group statistics or visualization, we also use the Z-score statistics regardless of the shape of null distribution. A high positive Z-score indicates that the raw data statistic is much greater than those obtained by randomly shuffled data (null hypothesis), and therefore is highly significant in a statistical sense. We consider a candidate event statistically significant when two Monte Carlo P-values are both smaller than 0.05 under two independent shuffling operations.

These three criteria are chosen to be consistent with previous published studies (Wu & Foster, 2014; Grosmark & Buzsáki, 2016; Chen et al., 2016). Modifying individual criterion on the length or significance level (e.g., P < 0.01) will inevitably change the statistic number, but our proposed analysis should serve as a standard guideline for similar behavior tasks.

3. Results

Software for all analyses is written in Python and MATLAB. The Python source code for Decode_woRF is available online (https://github.com/slinderman/pyhsmm_spiketrains). The MATLAB code for computing three correlation metrics and shuffle operation is also available (https://github.com/szl/memory). In our analyses, we have tested various distance metrics. Although the exact quantitative statistics may be slightly different, their trends are similar. In all results reported below, we have used equations 2.7 and 2.9 for computing the distance metric.

3.1. Experimental data

We test our methods on two rat hippocampal recordings. The data are publicly available (https://crcns.org/data-sets/hc/hc-11/). The recording session consisted of a long (~4 hour) pre-RUN sleep epoch home-cage recording performed in a familiar room, followed by a RUN epoch (~45 minutes) in a novel circular maze (1 m diameter, Dataset 1, 77 place cells) or in a linear track (1.6 m long, Dataset 2, 79 place cells). After the RUN epoch the animal was transferred back to its home cage in the familiar room where another long (~4 hour) post-RUN sleep was recorded. Details of experimental protocols and data have been published (Grosmark & Buzsáki, 2016; Grosmark et al., 2016; Chen et al., 2016).

In each dataset, from the RUN epoch, we extract place RFs from all putative hippocampal CA1 pyramidal cells. For each selected cell, we compute the spatial information rate (bits/s) as follows (Skaggs et al., 1993): $I_c=\int {\lambda }_c\left(S\right){\log }_2\frac{{\lambda }_c\left(\mathit{S}\right)}{{\lambda }_c}p\left(S\right)dS$, where ${\lambda }_c\left(S\right)$ denotes the mean firing rate of the c-th hippocampal neuron at spatial location S, and ${\lambda }_c=\int {\lambda }_c\left(S\right)p\left(S\right)dS$ denotes the total average firing rate (spikes/s). To account for the total firing rate effect, we compute the normalized information rate, $\frac{I_c}{{\lambda }_c}$, measured by bits/spike.

3.2. Decoding analysis setup

For Decode_wRF, we assume a Poisson firing rate (Zhang et al., 1998; Davidson et al., 2009). In Dataset 1, we use a 5 cm spatial bin for the linearized circular track, resulting in 57 spatial bins (excluding the reward location). From the RUN period, we use a velocity threshold (>10 cm/s) to identify the RUN period, resulting a total of 3376 temporal bins (bin size: 250 ms). We use 90% RUN spike data for training the model. Based on the constructed place fields, we compute the decoding accuracy on all RUN data (median decoding error: 4.5 cm, mean decoding error: 12.0 cm).

For Decode_woRF, we use a state-of-the-art Bayesian inference algorithm for the HDP-HMM (Linderman et al., 2016; Chen et al., 2016), which accommodates automatic model selection for identifying the number of latent states. The configuration for the hyperparameters is as follows (Linderman et al., 2016): ${\alpha }_0\sim \mathrm{Gamma}\left(a_{\alpha },1\right)$, $\gamma \sim \mathrm{Gamma}\left(a_{\gamma },1\right)$ (for HDP priors) and ${\lambda }_c\sim \mathrm{Gamma}\left(a_c,b_c\right)$ (for firing rate parameters). The choice of the hyperparameter priors {a_α, a_γ, a_c, b_c} is determined by the predictive likelihood of test data over various configurations, each with 10 Monte Carlo runs. We select the optimal hyperparameters among a total of 50 runs for the final evaluation. From the ensemble neural spiking activity alone during RUN, we can infer the state space map and state-transition matrix (Fig. 4).

Figure 4:

(A) State space map. (B) Spatial bins have non-uniform state occupancy distribution shown by the histogram. (C) Inferred state transition matrix inferred from Decode_woRF using spike data alone. (D) Transition matrix inferred from RUN behavior.

The optimal median and mean decoding error from Decode_woRF are 5.6 cm and 13.0 cm, respectively—a performance comparable to those derived from Decode_wRF. The inferred number of states varies between 95 and 100, which is relatively consistent for various hyperparameter choices. The inferred state transition matrix (Fig. 4C) also bears a good resemblance with the “ground truth” transition matrix that is estimated directly from animal’s run position (Fig. 4D). Note that Decode_woRF employs an adaptive spatial bin size, allowing non-even spacing within the environment according to the sampling occupancy or spiking frequency (Fig. 4B).

Here, to assess rat hippocampal memory reactivations during QW or SWS, in light of empirical observations (e.g., Davidson et al., 2009; Grosmark & Buzsáki, 2016), we assume that the time compression factor is around 12 during memory reactivation (i.e., 20 ms, as compared to 250 ms in RUN behaviors). In practice, the speedup factor is unknown, and may range from 10 to 33 (Wu & Foster, 2014; Deng et al., 2016). Changing the speedup factor may affect the results reported below. Notably, our previous analyses (Chen et al., 2016) have also shown the inferred state transition structure and decoding error during RUN behaviors is robust across a wide range of temporal bin sizes (150-250 ms).

3.3. Testing the reliability of our method for pattern detection

Before testing our method on real memory reactivation events during QW and SWS—where the ground truth of a “replay” is absent, we first create two “synthetic” memory reactivation events (Fig. 5A) to test the reliability of our method for detecting patterns with known ground truth. The assumption here is that the decoded patterns from the RUN data are similar to those decoded during memory reactivation events.

Figure 5:

(A) Two representative events of run trajectories. (B) Comparison of scatter plots of Z-score statistics between Z_wd (derived from R_wd) and Z_w (derived from R_w). Each point represents the result from one Monte Carlo run (n = 1000). Lower left corner within the dashed box indicates the non-significant region.

We select actual animal’s run trajectories and the associated ensemble neural spiking activity. Each run trajectory lasts 20 temporal bins (i.e., 5 s duration, 250 ms bin size). The first trajectory consists of two disconnected lines (similar to Fig. 3B) induced by linearization of the circular track. The second trajectory is a full path covering the complete track. Similar to the strategy used in (Chen et al., 2016), we manipulate the ensemble neural spiking activity by systematically changing the active cell percentage, varying from 10%, 15%, 20%, to 25%. At each percentage, we randomly select subsets of cells and remove their spikes. Based on the “sparsified” spike trains, we run the decoding analysis and assess the reconstructed trajectory by Z_wd and Z_w. We repeat the procedure 1000 times for each event. The results on scatter plots and detection ratios are shown in Fig. 5B and Table 1, respectively. As seen from these two examples, even with an active cell percentage as low as 10-15%, our method is still able to identify the pattern embedded in the rat hippocampal ensemble spiking activity. In all experimental analyses, we use the distance metrics 2.7 and 2.9 for computing R_wd and the respective Z_wd. In general, as the percentage of active cells increases, the number of detected signifiant events also increases; but the number of significant events is greater using Z_wd than using Z_w. This effect is more pronounced in Event 1 than Event 2.

Table 1:

Detection ratios for the “simulated” data on two run trajectories. All results are derived based on Decode_woRF using Z_wd.

Cell percentage	Event 1	Event 2
10%	0.593	0.788
15%	0.726	0.884
20%	0.771	0.955
25%	0.828	0.972

The interpretation of these simulation results is based on the assumption that the replay events can be viewed as “sparsified” spike trains with the same structure as the ones observed in run behavior (subject to additional time compression, which is not simulated here). To test whether our observations also hold for shorter sequences, we also test examples with 5 or 10 temporal bins (250 ms bin size). Similar trends are found between Z_wd and Z_w.

3.4. Testing the similarity assumption of RUN vs. SLEEP structures

We further test the assumption that the decoded patterns from the RUN data are similar to those candidate events of memory reactivation during SLEEP. Specifically, in parallel with the unsupervised population decoding analysis on the RUN→LEEP paradigm, we reverse the direction and apply the analysis to the SLEEP→RUN paradigm. Specifically, we apply Decode_woRF using n = 117 candidate events (during post-SWS) as the training data. We infer the firing rate matrix from SLEEP epochs and then further infer the state sequence during RUN.

Next, we compare the inferred state sequence during RUN with the observed animal’s position (“ground truth”) to assess the state space map and decoding error (median error: 0.16 m). The results are shown in Fig. 6. Notably, despite the fact that the sleep-associated ensemble neural spiking activity is sparse and fragmental, our method is still capable of extracting meaningful latent structures of “place codes”. Importantly, our method works equally well for both RUN→LEEP and SLEEP→RUN directions, supporting our assumption on the similarity between RUN and SLEEP structures.

Figure 6:

Inferred state space maps. (A) Using rat hippocampal ensemble neural spiking activity from RUN epochs alone (effective number of states: 100) and testing on RUN data (median decoding error: 0.05 m)—replotting Fig. 4A. (B) Using rat hippocampal ensemble neural spiking activity from post-SWS epochs alone (effective number of states: 34) and testing on RUN data (median decoding error: 0.16 m).

3.5. Testing on QW and SWS data

Based on the hippocampal LFP, we compute the delta/theta power ratio and the ripple band (150-300 Hz) power. Combining these features with EMG (electromyography), we classify the sleep stages into WAKE, SWS, and REM states while the animal stay in the sleep box (see Supplementary Fig. S1 in Grosmark & Buzsáki, 2016).

We also identify the quiet wakefulness (QW) periods, when the animal was in an immobile wake state (speed <2 cm/s). For screening the candidate events for hippocampal memory reactivation during QW or SWS, we use hippocampal LFP ripple band power combined with hippocampal multi-unit activity (threshold>mean+3SD). We also impose a minimum temporal bin criterion (≥5 bins consisting of spikes) and a minimum cell activation criterion (at least 10% of cell population). By “active”, we consider two options in which each cell fire at least 1 (termed option 1) or 2 (termed option 2) spikes during the duration of interest. We analyze candidate events from five distinct periods in time: pre-QW, maze-QW, post-QW, pre-SWS, and post-SWS, and report the results in Table 2 (Dataset 1; supplementary table for Dataset 2).

Table 2:

Summary statistics of hippocampal memory replay candidate events from Dataset 1 (ℓ: total number of events; ℓ₁: the number of events that had at least 5 temporal bins with spikes and at least 10% active cells (option 1 & option 2); ℓ₂: the number of events in which R_w meets the significance criterion (derived from Decode_wRF); ℓ₃: the number of events in which R_wd meets the significance criterion (derived from Decode_wRF) ℓ₄: the number of events in which R_wd meets the significance criterion (derived from Decode_woRF with a HDP-HMM). ℓ₅: the number of events in which R_wd meets the significance criterion (derived from Decode_woRF with a finite-state HMM).

Dataset 1
	ℓ	ℓ1	ℓ2	ℓ3	ℓ4	ℓ5
pre-QW	338	271 (option 1)	11	10	10	15
		164 (option 2)	7	9	7	12
maze-QW	136	93 (option 1)	16	21	18	18
		51 (option 2)	14	15	15	16
post-QW	1619	1328 (option 1)	150	216	189	189
		944 (option 2)	126	193	160	162
pre-SWS	984	767 (option 1)	38	23	35	23
		454 (option 2)	28	12	17	15
post-SWS	1519	1108 (option 1)	91	136	123	123
		741 (option 2)	69	117	108	97

Comparison between R_w and $R_{wd}^{\ast }$.

We first compare statistics ℓ₂ and ℓ₃ in Table 2, which employs the supervised population-decoding algorithm Decode_WRF. During maze-QW and post-QW, the fraction of significant events in option 1 is $\frac{16+150}{93+1328}=11.7\%$ using Z_w, and $\frac{21+216}{93+1328}=16.7\%$ using Z_wd. During pre-QW and pre-SWS, the ratio of significant events is smaller, suggesting that memory reactivation is stronger following right after RUN experiences. During pre-SWS, the fraction of significant memory reactivation events is 5.0% using Z_w and 3.0% using Z_wd. During post-SWS, the fraction of significant memory reactivation events is 8.2% using Z_w and 12.2% using Z_wd. Therefore in this dataset, Z_wd achieves a higher detection significance rate in post-QW or post-SWS and lower significance rate in pre-QW or pre-SWS. This is consistent with our “simulation” studies in section 3.3. This trend is also similar in option 2.

To investigate whether the increase of significance rate is due to the decrease in false negative (FN) or increase in FP, we further compare the scatterplot of Z_w and Z_wd (Fig. 7). It is found that there are a large percentage of candidate events that Z_w and Z_wd do not agree on. Because of the lack of ground truth, it is impossible to define exactly the outcome that Z_w and Z_wd disagree upon. For instance, Figure 8 lists four examples that two criteria do not reach a consensus. In Fig. 8A, we see an instance that Z_w cannot capture the nonlinearity pattern; whereas in Fig. 8D, we see another instance that Z_wd fails because the neighboring temporal bins are coupled with spatially separated bins (e.g., the positions at the 5th and 6th time bins are 0.5 m apart), which implies a large d_ij or small P_ij inferred from RUN epochs. There are also other ambiguous instances with relatively short events (< 10 time bins, Figs. 8B and 8C).

Figure 7:

Scatter plot comparison of Z_wd and Z_w derived from Decode_wRF for pre-QW, maze-QW, post-QW, pre-SWS and post-SWS candidate events. Lower left corner within the dashed box indicates the non-significant region.

Figure 8:

Discrepancy examples of memory reactivation traces detected by Z_wd and Z_w derived from Decode_wRF. X-axis denotes time bin (bin size: 20 ms).

Due to the lack of growth truth of “memory traces”, it is difficult to define a replay event in a statistical sense. One possible way is to assess the candidate events using multiple independent methods and compare with our proposed statistical measures. For instance, for a spatial task, we can compare with the template matching (Nadasdy et al., 1999; Louie & Wilson, 2001; Euston et al., 2007; Wilber et al., 2017) and line fit (Davidson et al., 2009) methods. In Figs. 8A–D, employing a line fit and assessing the Z-score of its linear correlation yields: …. and …. respectively. However, it should be stressed that the uncertainty of ground truth still remains even when all measures agree with each other in statistical significance. The outstanding question of defining a statistical criterion for memory replay events still remains, especially in the presence of non-spatial tasks.

Comparison between Decode_wRF and Decode_woRF.

We further compare the detection performance using Z_wd between Decode_wRF and Decode_woRF. For Decode_wRF, the transition matrix is computed from the RUN behavior (Fig. 4D). For Decode_woRF, the transition matrix is inferred from the ensemble spiking data alone (Fig. 4C).

It is found that the derived R_wd from Decode_wRF and from Decode_woRF are correlated, but their Z_wd statistics do not completely agree with each other (Fig. 9). Specifically, we observe that (i) the number of significant events derived from Decode_woRF is lower than that derived from Decode_wRF (ℓ₃ and ℓ₄ statistics in Table 2); (ii) the Z-score value in Decode_woRF is often greater than in Decode_wRF (see examples in Fig. 10). The significance disagreement between Decode_wRF and Decode_woRF (e.g., Fig. 11) may be induced by two factors: (i) the inferred state transition matrix and (ii) adaptive spatial sampling as shown in the state space map. Therefore, the inferred number of states are 100, which is much higher than the number of spatial bins (i.e., 57). They both affect the measure of R_wd as well as Z_wd.

Figure 9:

Scatter plot comparison of Z_wd between Decode_wRF and Decode_woRF for pre-QW, maze-QW, post-QW, pre-SWS and post-SWS candidate events. Lower left corner within the dashed box indicates the non-significant region.

Figure 10:

Examples of significant memory replays detected using Z_wd but failed using Z_w. (A) Results from Decode_wRF. (B) Results from Decode_woRF. The title shows the Z-score. X-axis denotes time bin (bin size: 20 ms). Y-axis denotes the spatial position in panel A and denotes the inferred latent state in panel B.

Figure 11:

Examples of disagreement in detected memory traces from Decode_wRF (A) and Decode_woRF (B). The title shows the Z_wd statistic. X-axis denotes time bin (bin size: 20 ms). Y-axis denotes the spatial position in panel A and denotes the inferred latent state in panel B.

3.6. Information coding and memory reactivation

Finally, we examine the relationship between information coding and hippocampal memory reactivation during post-QW and post-SWS. For all post-QW and post-SWS candidate events that meet our selection criterion, we compute two statistics: n₁—the percentage of active cells among 77 neurons, and n₂—the percentage of high spatial information cells among active cells. Neither n₁ nor n₂ is different between post-QW and post-SWS candidate events (p > 0.5, rank-sum test). However, n₁ is positively correlated with Z_wd among all (or significant only) candidate events during post-QW (Pearson’s correlation ρ = 0.34, p < 10⁻³), and post-SWS (ρ = 0.30, p < 10⁻³), suggesting that the number of active cells is critical for detecting memory replays (Chen et al., 2016; Grosmark & Buzsáki, 2016). In addition, based on the normalized information rate (bits/spike), we compute the active information content (AIC, unit: bits) from each candidate event:

$$AIC=\sum _{c\in ActiveCells}\frac{I_c}{{\lambda }_c}\times \#\mathrm{Spikes\; from\; the}c\mathrm{-th\; cell}$$ (3.1)

assuming mutual independence between all spikes. It is also found that the active information content of candidate events is positively correlated with their Z-scores (ρ = 0.37 for post-QW; ρ = 0.35 for post-SWS, p < 10⁻³).

4. Discussion

4.1. Limitation of our method

Detecting reactivated memory traces in the hippocampus and neocortex is an important step to study memory consolidation. We have proposed a weighted distance correlation metric (R_wd) for assessing memory reactivation based on rat ensemble neural spiking activity. The new metric goes beyond the linearity assumption of the weighted correlation R_w, and combines the merits of weighting and distance correlation. This metric can be used in both Decode_wRF and Decode_woRF.

However, the new metric has also some limitations, as evidenced in our empirical observations from experimental data. First, R_wd may not capture the “parts” in the whole pattern (e.g., Fig. 3C); in other words, it cannot identify only a subset of the pattern. This may contribute to erroneous negatives. Second, R_wd focuses on local distance and ignores the global pattern, and therefore misidentifies certain spurious trajectories, such as the low amplitude zigzag curve (e.g., Fig. 3E). This will contribute to erroneous positives. Third, while combining R_wd and Decode_woRF, our method also relies on d_ij or P_ij, the latter of which is estimated from the spike activity of WAKE behavior. When a reactivated sequence is associated with low P_ij values, the event may be assigned with a low score (e.g., Fig. 8D). This will also contribute to erroneous negatives. How to improve these issues will be the subject of our future investigation.

4.2. Detection sensitivity on the number of states

In Decode_woRF, the number of states is inferred directly from the data. One important question is whether the number of states affects R_wd or Z_wd. In practice, changing the sample size has a stronger impact on the inferred number of states m. The more the data samples, the larger is the dimensionality m. There is also variability in m by changing hyperparameters, but the effect is relatively smaller.

Different dimensionality of m will affect not only the estimates in the state transition matrix and firing rate matrix, but also the distance metric. To determine whether the choice of a smaller m would affect Z_wd, we repeat the unsupervised population decoding analysis using a prefixed 57-state HMM (Chen et al., 2014; Linderman et al., 2016), where m = 57 is chosen to match the spatial RF resolution in Decode_wRF. The decoding accuracy of the 57-state HMM is comparable with that of HDP-HMM, with median and mean prediction error as 6.0 cm and 14.6 cm, respectively. The detection results derived from the 57-state HMM are shown in Table 2 (ℓ₅ column). By comparing ℓ₄ and ℓ₅ columns, it is found that the statistics are highly comparable and their Z_wd are also correlated (ρ = 0.64, p < 10⁻¹⁰), suggesting that our method is robust with respect to the model dimensionality.

In a general setup, an automatic model selection of dimensionality m is preferred for unbiased assessment. From a model representation perspective, a larger number of latent states would imply a finer spatial representation. Therefore, the posterior weights may be either uniformly distributed or multimodal, which may produce certain biases when using only two dominant modes. To alleviate this issue, first, we can use more posterior modes (Appendix C). Second, we can zero threshold the small transition probabilities P_ij to “denoise” the state transition probability matrix before computing the distance metric (see equation B.3).

4.3. Statistical criterion of memory replay

As the goal of our method is to assess the memory reactivation from candidate events, it is impossible to determine a replay event in advance without the ground truth. In principle, one can integrate all available information, such as spikes, LFPs and animal’s behavioral measures (e.g., velocity), to construct a statistical model of replay. In a related work reported in (Deng et al., 2016), a real-time population-decoding method was developed to compute the posterior probability of the replay feature and to classify the discrete replay content of hippocampal memory reactivation candidate events during QW. In their W-maze spatial navigation task, four potential replay contents are “outbound, forward”, “outbound, reverse”, “inbound, forward” and “inbound, reverse”. When assessing or classifying discrete contents of such constrained memory replays, a good statistical criterion can be determined. However, if the contents are drawn from a continuous set or an infinite discrete set, then difficulty will arise. In addition, their approach is based on a supervised population-decoding analysis using animal’s position measures during the encoding phase. How to generalize this statistical measure into an unsupervised decoding analysis setting or extend statistical assessment of replay contents to a non-spatial task remains unresolved.

In practice, we recommend to apply multiple independent or complementary methods for assessing memory reactivation events (see review in Chen & Wilson, 2017). It might be possible to integrate the information from these methods to establish a probabilistic criterion for memory replay. However, it would require to establish some benchmark datasets and conduct systematic investigations.

4.4. Extension to other behavioral tasks

In this paper, we have tested our method using the rodent 1D spatial (i.e., circular track) navigation task as an example. In the presence of complex mazes with multiple choice points (e.g., T-maze or radial maze), the run trajectory discontinuity induced by linearization will occur, especially when the trajectory covers two disjoint arms (e.g., Wu & Foster, 2014). This problem becomes even more challenging for a 2D spatial navigation task. To assess hippocampal memory reactivation in those tasks, the unsupervised population decoding analysis is appealing. Furthermore, R_wd is only dependent on the state transition matrix and is invariant to the space linearization or state permutation. We are planning to extend our investigation to 2D spatial memory reactivation in the rat hippocampus (Kloosterman & Wilson, 2009). In other hippocampal non-spatial tasks (e.g., Allen et al., 2016; Shan et al., 2016), the latent states may accommodate non-spatial features of experiences or distinct behavioral patterns that cannot be defined or measured directly.

In principle, Decode_woRF can be applied to the ensemble neural spiking activity from neocortical or subcortical structures, such as the visual cortex (Ji & Wilson, 2007; Haggerty & Ji, 2015), motor cortex (Gulati et al., 2014; Ramanthan et al., 2015), striatum (Ahmed et al., 2008) and amygdala (McGaugh, 2004). For either spatial or non-spatial tasks, we may assume that the dynamic population firing rates are driven by a latent state sequence, where the content of the latent state is not defined a priori. During SWS or QW epochs, we can adapt a similar strategy to detect the patterns of memory reactivation from ensemble neural spiking activity. In this case, the content-based method has a clear advantage compared to the subspace (e.g., PCA and ICA) methods, since the latter analysis might lead to false statement (see Appendix E). It is hoped that our proposed method will be widely tested in the future practice of assessing memory reactivation in neural data analysis.

4.5. Beyond SWS

In contrast to SWS that has been strongly implicated in memory reactivation and consolidation, the exact function of REM sleep remains elusive. One hypothesis of REM sleep is that the hippocampal readout for declarative memories is attenuated, leading to a functional isolation of limbic/paralimbic structures and the neocortex, resulting in memory reorganization; whereas the amygdala and other networks of the limbic system that are responsible for emotional processing show high reactivation (Landmann et al., 2015). Accumulating evidence has suggested that REM sleep is essential for consolidating contextual or emotional memories (Boyce et al., 2016; Menz et al., 2016; Li et al., 2017). Unlike SWS, there is no population synchrony (“UP state”) associated with hippocampal sharp wave ripples during REM sleep, yet there is a prevalence of theta oscillations in rodents (Louie & Wilson, 2001; Grosmark et al., 2012). As a result, it remains a challenge to assess hippocampal-neocortical memory reactivation during REM sleep in either animals or humans (Louie & Wilson, 2001; Horikawa et al., 2007; Schönauer et al., 2017; Siclari et al., 2017). Here, we have focused on SWS epochs using population decoding analyses. How to apply these methods to REM sleep epochs, such as identifying potential candidate events and proper analysis timescale, remains unsettled. Without a principled guideline, any quantitative measure (e.g., the R_wd statistic derived from population decoding, or the reactivation strengths derived from PCA/ICA, Appendix E) can be misleading or lead to false hypotheses. Therefore, studying memory reactivation during REM sleep requires extra care in future investigations.

Figure 12:

Examples of ambiguous candidate events derived from Decode_wRF. In these 4 examples, both Z_w and Z_wd are above the significance level. X-axis denotes time bin (bin size: 20 ms).

Appendix A: Bias correction for distance covariance

For self-contained purpose, we show the formula to correct the sample bias in distance covariance (Székely & Rizzo, 2013). Let (a_ij} and (b_ij} denote two matrices that contain all pairwise distances between random samples u_i and u_j and between v_i and v_j, respectively:

$$a_{ij}=dis{t}_u\left(u_i,u_j\right)$$

$$b_{ij}=dis{t}_v\left(v_i,v_j\right)$$

where dist_u and dist_v are two custom-defined distance metrics for variables u and v, respectively. Further, the sample distance covariance is defined by (Székely & Rizzo, 2009):

$$Co{v}_d\left(u,v\right)=\frac{1}{n}\sqrt{\sum _{i,j=1}^n{A}_{i,j}{B}_{i,j}}$$ (A.1)

where $A_{i,j}=a_{ij}-{\overline{a}}_i-{\overline{a}}_j+\overline{a}$, $a_{ij}=\left|u_i-u_j\right|$, ${\overline{a}}_i=\frac{1}{n}{\sum }_{k=1}^n{a}_{ik}$, ${\overline{a}}_j=\frac{1}{n}{\sum }_{k=1}^n{a}_{kj}$ and $\overline{b}=\frac{1}{n^2}{\sum }_{i,j=1}^n{a}_{ij}$ (similarly defined for $B_{i,j},{\overline{b}}_i,{\overline{b}}_j,\overline{b}$).

For bias correction, we compute the modified terms of A_i,j and B_i,j as follows:

$$A_{i,j}^{\ast }=\{\begin{array}{ll}\frac{n}{n-1}\left(A_{i,j}-\frac{a_{ij}}{n}\right),& i\ne j\\ \frac{n}{n-1}\left({\overline{a}}_i-\overline{a}\right),& i=j\end{array}$$ (A.2)

and

$$B_{i,j}^{\ast }=\{\begin{array}{ll}\frac{n}{n-1}\left(B_{i,j}-\frac{b_{ij}}{n}\right),& i\ne j\\ \frac{n}{n-1}\left({\overline{b}}_i-\overline{b}\right),& i=j\end{array}$$ (A.3)

For sample size n > 3, the distance covariance is corrected as

$$Co{v}_d^{\ast }\left(u,v\right)=\frac{1}{n\left(n-3\right)}\left\{\sum _{i,j=1}^n{A}_{i,j}^{\ast }{B}_{i,j}^{\ast }-\frac{2}{n-2}\sum _{i=1}^n{A}_{i,j}^{\ast }{B}_{i,j}^{\ast }\right\}$$ (A.4)

Mathematically, the expected value of $Co{v}_d^{\ast }\left(u,v\right)$ is unbiased (Székely & Rizzo, 2013):

$$\mathbb{E}\left[Co{v}_d^{\ast }\left(u,v\right)\right]=Co{v}_d\left(u,v\right)$$ (A.5)

Therefore, the bias-corrected distance correlation is given by

$$R_d^{\ast }\left(u,v\right)=\frac{Co{v}_d^{\ast }\left(u,v\right)}{\sqrt{Co{v}_d^{\ast }\left(u,u\right)Co{v}_d^{\ast }\left(v,v\right)}}$$ (A.6)

when $Co{v}_d^{\ast }\left(u,u\right)Co{v}_d^{\ast }\left(v,v\right)\ne 0$. In theory, the distance covariance and distance correlation are nonnegative and unique (Székely & Rizzo, 2012). Although $0\le R_d\le 1$, $R_d^{\ast }$ can take negative values subject to $\left|R_d^{\ast }\right|\le 1$.

Appendix B: Proof

For simplicity, let d_max = 2P_max, then equation 2.9 is rewritten as

$$d_{ij}\left(S_i,S_j\right)=\{\begin{array}{cc}2P_{\max }-{\rho }_{ij},& \mathrm{if}{\rho }_{ij}\le P_{\max }\mathrm{and}{S}_i\ne S_j\hfill \\ 0,& \mathrm{if}{\rho }_{ij}>P_{\max }\mathrm{or}{S}_i=S_j\hfill \end{array}$$ (B.1)

where $P_{\max }={\max }_{ij}\left\{P_{ij}\right\}$.

By definition $d_{ij}\left(S_i,S_i\right)=0$, and it is easy to prove $d_{ij}\left(S_i,S_j\right)=d_{ji}\left(S_j,S_i\right)$, the last thing we need to prove is the triangle inequality: $d_{ij}\left(S_i,S_j\right)+d_{jk}\left(S_j,S_k\right)\ge d_{ik}\left(S_i,S_k\right)$.

Note that for i ≠ j, i ≠ k, j ≠ k, we have

$$\begin{array}{l}{d}_{ij}\left(S_i,S_j\right)+d_{jk}\left(S_j,S_k\right)-d_{ik}\left(S_i,S_k\right)\\ =2P_{\max }-{\rho }_{ij}-{\rho }_{jk}+{\rho }_{ik}\\ =2P_{\max }-\frac{P_{ij}+P_{ji}}{2}-\frac{P_{jk}+P_{kj}}{2}+\frac{P_{ik}+P_{ki}}{2}\\ \ge 0\end{array}$$ (B.2)

because of $P_{\max }\ge \frac{P_{ij}+P_{ji}}{2}$, $P_{\max }\ge \frac{P_{jk}+P_{kj}}{2}$, and $P_{ik}+P_{ki}\ge 0$, and it completes the proof.

To minimize the impact of small transition probability events (5/total number of states, below which is considered a chance level), we can modify the distance criterion (B.1) as follows

$$d_{ij}\left(S_i,S_j\right)=\{\begin{array}{cc}2P_{\max }-{\rho }_{ij},& \mathrm{if}{\rho }_{ij}\le P_{\max }\mathrm{and}{S}_i\ne S_j\hfill \\ 0,& \mathrm{if}{\rho }_{ij}>P_{\max }\mathrm{or}{S}_i=S_j\hfill \\ 2P_{\max },& \mathrm{if}{\rho }_{ij}<5/\mathrm{total\; number\; of\; states}\hfill \end{array}$$ (B.3)

From equation B.2, it is easy to prove that this modified version still satisfies the triangle inequality.

In addition, we can incorporate a “time local neighborhood” constraint into equation B.1

$$d_{ij}\left(S_i\left(t_1\right),S_j\left(t_2\right)\right)=\{\begin{array}{cc}2P_{\max }-{\rho }_{ij},& \mathrm{if}{\rho }_{ij}\le P_{\max }\mathrm{and}{S}_i\ne S_j\mathrm{and}1\le \left|t_1-t_2\right|\le 3\\ 0,& \mathrm{if}{\rho }_{ij}>P_{\max }\mathrm{or}{S}_i=S_j\hfill \\ d_{\max },& \mathrm{if}\left|t_1-t_2\right|>3\hfill \end{array}$$ (B.4)

In general, equation B.4 may violate the triangle inequality.

Appendix C: Weighted distance

Note that the weighted correlation R_w has taken into account the uncertainty of the estimated posterior probability (i.e., multi-modal distribution), but it fails to capture the nonlinearity or discontinuity of a trajectory. In contrast, the distance correlation R_d can capture more complex trajectory shapes and is invariant to the row permutation. To combine the desirable features of R_w and R_d, we propose a new metric called weighted distance correlation, R_wd. The concept of the weighted distance correlation is to compute a weighted distance based on both weighting and a predefined distance metric.

For simplicity, we first extend the unimodal (i.e., the MAP estimate in the y-axis) to bimodal scenario. Let us extract the first two greatest posterior probabilities in each column (i.e., temporal bin). Assume that we compute the weighted distance correlation between two neighboring temporal bins. The two posterior probabilities are normalized. Without loss of generality, we assume that the two normalized posterior probabilities are a and 1 − a in the first temporal bin, associated with the row indices (i.e., spatial bin) i and j, respectively. Similarly, assume that the two normalized posterior probabilities are b and 1 − b in the second temporal bin, with respective row indices k and l.

Next, we define a new “weighted distance” metric, d_w, as follows

$$d_w=ab{d}_{ik}+a\left(1-b\right)d_{il}+\left(1-a\right)b{d}_{jk}+\left(1-a\right)\left(1-b\right)d_{kl}$$ (C.1)

where d_ik, d_il, d_jk and d_kl are the computed distance measures (e.g., equations 2.8, 2.9 or 2.10), and the sum of those weighted coefficient is equal to 1, namely $ab+a\left(1-b\right)+\left(1-a\right)b+\left(1-a\right)\left(1-b\right)=1$. In the special case when a = 0 (or 1) and b = 0 (or 1), d_w reduces to the standard distance metric without weighting. Therefore, we may use the new metric in the place of the standard distance metric for computing the weighted distance correlation.

More generally, we can extend the above bimodal 2-to-2 setting to a multimodal m-to-m setting, which produces a sum of m² combinatorial terms. Let $w=\left[w_1,\dots ,w_m\right]$ and $v=\left[v_1,\dots ,v_m\right]$ denote two weight matrices with nonnegative elements, where ${\sum }_{i=1}^m{w}_i=1$, ${\sum }_{j=1}^m{v}_j=1$. Then we have

$$d_w=\sum _{i=1}^m\sum _{j=1}^m{w}_i{v}_j{d}_{ij},\mathrm{where}\sum _{i=1}^m\sum _{j=1}^m{w}_j{v}_j=1$$ (C.2)

In practice, we find that it is sufficient to use 2 to 4 dominant modes for computing the weighted distance d_w.

Appendix D: Unbiased Z-score of distance correlation

Let us consider computing the Z-score of distance correlation R_d as follows

$$Z_d=\frac{R_d-\mathrm{mean\; of}{R}_{d,\mathrm{shuffle}}}{\mathrm{SD\; of}{R}_{d,\mathrm{shuffle}}}.$$ (D.1)

In the presence of a small sample size n, R_d is over-estimated (which is always nonnegative). Without loss of generality, we assume that estimate of R_d with n samples are ${\widehat{R}}_{d,n}=R_d+\delta$, where δ ≥ 0 denotes the estimation bias. Next, we assume that with the same sample size n, the estimation bias δ is statistically identical for different random shuffles. Under this assumption, the bias of $\mathbb{E}\left[R_{d,\mathrm{shuffle}}\right]$ is still δ, and the expected bias of $R_d-\mathbb{E}\left[R_{d,\mathrm{shuffle}}\right]$ is 0 due to bias cancellation. In addition, the standard deviation of R_d,shuffle is independent of the bias constant. Therefore, under these assumptions, Z_d will be unbiased even when ${\widehat{R}}_{d,n}$ is potentially biased for a small n. This has been empirically confirmed by a computer simulation experiment (Fig. 13).

Figure 13:

Computer simulations for assessing R_d (equation 2.8) for two independent samples under various sample size n. (A) The ground truth of R_d is 0. Solid and dashed curves denote the uncorrected estimate ${\widehat{R}}_{d,n}$ and its bias-corrected estimate, respectively. Error bar denotes the SEM from 10 Monte Carlo random simulations. (B) Z_d (using uncorrected R_d) based on 1000 random shuffles. Error bar denotes the SEM. None of Z_d estimate reaches the statistical significant level, and Z_d does not change monotonically with respect to the sample size.

The same reasoning also applies to Z_wd for weighted distance correlation R_wd.

Appendix E: Subspace methods: PCA and ICA

Subspace methods are derived from second-order correlation statistics (e.g. PCA) or high-order moment statistics (e.g. ICA). However, these methods have several limitations. First, they are linear methods, therefore they are insufficient to capture statistical dependency with nonlinear transformation. Second, a stationary correlation statistic is assumed during the complete TEMPLATE or MATCH period, which is untrue in the presence of distinct or complex behaviors that drive the state-dependent neuronal responses. Third, the derived reactivation strength (RS) from these methods does not identify the content of memory; instead, RS is positively correlated with the quadratic power of temporal firing rate of individual neurons.

To illustrate the third point, let’s consider the PCA method as an example. Specifically, the method consists of three steps (Peyrache et al., 2009): (i) Compute a C × C correlation matrix C from a Z-scored C × T population firing rate matrix collected from C neurons during a TEMPLATE epoch (e.g., during WAKE behavior); (ii) Conduct PCA and extract the dominant eigenvalue λ₁ and eigen-subspace P₁ such that $C={\sum }_{i=1}^C{\lambda }_i{P}_i\approx {\lambda }_1{P}_1$, where $P_1=u_1{u}_1^{\top }$ and u₁ is the principal eigenvector; (iii) In the testing phase, project the instantaneous C-dimensional firing rate vector v_t (Z-scored) onto the dominant subspace and compute the time-varying RS

$$\begin{array}{l}R{S}_t=v_t^{\top }{P}_1{v}_t=\mathrm{trace}\left\{P_1\left(v_t{v}_t^{\top }\right)\right\}\\ =\mathrm{trace}\{\mathrm{diag}\left\{P_1\left(v_t{v}_t^{\top }\right)\right\}+\mathrm{offdiag}\left\{P_1\left(v_t{v}_t^{\top }\right)\right\}\}\\ =\mathrm{trace}\left\{\mathrm{diag}\left\{P_1\left(v_t{v}_t^{\top }\right)\right\}\right\}+\mathrm{trace}\left\{\mathrm{offdiag}\left\{P_1\left(v_t{v}_t^{\top }\right)\right\}\right\}\\ \ge 0\end{array}$$ (E.1)

where ^⊤ denotes the transpose operator. Note that the second term of the third equation line (off-diagonal elements) characterizes the second-order interaction between pairwise neurons, whereas the first term of the third equation line (diagonal elements) characterizes quadratic power of the firing rate projected onto the dominant eigenspace. Although the first term and the sum of these two terms is nonnegative, the exact value of each term is dependent on the actual values of P₁ and v_t (i.e., the second off-diagonal term can be a negative value). Mathematically, $\mathrm{trace}\left\{\mathrm{diag}\left\{P_1\left(v_t{v}_t^{\top }\right)\right\}\right\}={\sum }_{c=1}^C{v}_{c,t}^2{P}_{cc}$ (where P_cc denotes the c-th diagonal term of P₁)—if the value of a single element v_c,t is sufficiently large, its contribution may significantly bias the value of RS_t and dominate the value of off-diagonal term. Therefore, RS_t can be very sensitive to an outlier in the firing rate vector v_t for several obvious reasons: (i) biological neuronal firing rates are non-Gaussian distributed and highly skewed (Buzsáki & Mizusek, 2014); (ii) neurons can have intrinsic bursting; (iii) neuronal spike can be corrupted by noise. All these factors would make “outliers” appear more frequently than expected.

For instance, in the presence of two neurons, consider the following two scenarios

$$\mathrm{Scenario}1:P_1=\left[\begin{array}{cc}0.832& 0.555\\ 0.555& 0.832\end{array}\right],v_t=\left[\begin{array}{c}0.2\\ 0.2\end{array}\right],v_t^{\top }{P}_1{v}_t=0.113$$

$$\mathrm{Scenario}2:P_1=\left[\begin{array}{cc}0.995& 0.095\\ 0.095& 0.995\end{array}\right],v_t=\left[\begin{array}{c}0.8\\ 0.1\end{array}\right],v_t^{\top }{P}_1{v}_t=0.662$$

where two P₁ matrices are orthonormal and positive definite. Note that in both scenarios, the off-diagonal term P₁₂ or P₂₁ characterizes the pairwise neuronal interaction. However, even P₁₂ is much smaller in Scenario 2 (implying their correlation is not significant) than in Scenario 1, the instantaneous RS value can be still very large due to the choice of v_t.

More importantly, the RS measure cannot reveal the content of memory reactivation. A high or low value in RS is relative (depending on relative change in population firing rate) and is only meaningful for specific recordings. In other words, the same RS quantity may have different interpretations or implications depending on specific neuronal ensembles. Associating high RS values with memory reactivation may lead to a false hypothesis. This has been confirmed by our empirical observations while working on rodent hippocampal spike data.