Modeling multiscale causal interactions between spiking and field potential signals during behavior

Abstract

Objective.

Brain recordings exhibit dynamics at multiple spatiotemporal scales, which are measured with spike trains and larger-scale field potential signals. To study neural processes, it is important to identify and model causal interactions not only at a single scale of activity, but also across multiple scales, i.e. between spike trains and field potential signals. Standard causality measures are not directly applicable here because spike trains are binary-valued but field potentials are continuous-valued. It is thus important to develop computational tools to recover multiscale neural causality during behavior, assess their performance on neural datasets, and study whether modeling multiscale causalities can improve the prediction of neural signals beyond what is possible with single-scale causality.

Approach.

We design a multiscale model-based Granger-like causality method based on directed information and evaluate its success both in realistic biophysical spike-field simulations and in motor cortical datasets from two non-human primates (NHP) performing a motor behavior. To compute multiscale causality, we learn point-process generalized linear models that predict the spike events at a given time based on the history of both spike trains and field potential signals. We also learn linear Gaussian models that predict the field potential signals at a given time based on their own history as well as either the history of binary spike events or that of latent firing rates.

Main results.

We find that our method reveals the true multiscale causality network structure in biophysical simulations despite the presence of model mismatch. Further, models with the identified multiscale causalities in the NHP neural datasets lead to better prediction of both spike trains and field potential signals compared to just modeling single-scale causalities. Finally, we find that latent firing rates are better predictors of field potential signals compared with the binary spike events in the NHP datasets.

Significance.

This multiscale causality method can reveal the directed functional interactions across spatiotemporal scales of brain activity to inform basic science investigations and neurotechnologies.

1. INTRODUCTION

Identifying the functional causality structure in brain networks is important for our understanding of neural mechanisms underlying brain functions and for designing neurotechnologies to decode and modulate brain states [1–11]. Further, many behaviors involve multiple spatiotemporal scales of neural activity, spanning not only neural spiking activity but also larger-scale neural population activity measured through local field potentials (LFP) [12–28]. Thus, identifying the causality structure can be important not only at a single scale of activity but also across different activity scales [2,12,29,30]. As many modern neurophysiological datasets provide measurements of the brain at multiple scales through simultaneous recordings of spike trains and field potentials [31,32], statistical tools that can assess multiscale causality are important to develop and validate. Further, it is important to study whether modeling the causality structure across different scales can lead to more accurate models of neural activity beyond what is possible through identifying a single-scale causality structure. Finally, neurophysiology experiments can record from only a subset of neurons in the brain and as such, functional causality is generally evaluated by Granger-like causality methods in which causality is statistically measured within the recorded data only. Regardless of the method used, a fundamental limitation of such Granger-like causality evaluation is that identified causal connections are not conditioned on the unrecorded activity (see also Discussion). To improve accuracy of Granger-like causality assessment, it is beneficial to consider as much data as possible within a given experimental recording. As spiking and field potential activity provide measurements of the brain at multiple scales [31, 32], considering both of these multiscale signals simultaneously could lead to more accurate causality analyses.

Identifying multiscale causality introduces additional challenges because of the signal differences in the spike train and field potential modalities. Spike trains consist of fast action potential events that take a binary 0–1 value while field potentials are continuous-valued signals [8,33–36]. Some prior multiscale studies focus on undirectional connectivity using correlation-based measures such as spike-field coherence [18,37–40] or on developing encoding models that fit correlation terms between signals [8,41,42]. But these studies do not focus on assessing causality and directional interactions between spike trains and field potentials. There are also studies on spike-field coherence that measures consistent neural spiking at a specific phase of LFP [43] and on model-free measures such as spike-triggered average of the LFP (stLFP) that assess interactions from spike trains to field potentials, but these measures are not always causal [30]. To assess causality, traditionally, Granger causality [44] is used for continuous signals like LFP and electroencephalography (EEG)/magnetoencephalography (MEG) [45–47]. Also, although Granger causality is not directly applicable for discrete signals that are not Gaussian processes [33], it has been used for filtered continuous signals generated from spike trains [47–49]. But assessing multiscale causality between spike trains and field potentials requires methods that can work on a mixture of both discrete and continuous signal modalities.

For binary spike trains alone, other methods for computing causality have also been developed beyond Granger causality [47,50–58]. Some methods use a generalized linear model (GLM) framework to model the spike trains as point processes [53, 54, 56, 57, 59]. Doing so, causality for spike trains can be computed using an information-theoretic measure termed directed information [53–56, 60], which is a generalization of Granger causality. But these methods are applicable within only spike trains rather than across spike trains and field potentials.

In general, methods for computing causality for spike trains or field potentials are either model-based or model-free. Model-based methods can explicitly incorporate the influence of various factors such as behavior on neural activity but depend on the accuracy of the constructed models. Despite this challenge, these methods have been shown to be useful tools for analyzing causality statistically and can reveal neurophyisological insights as shown in prior work [47, 56]. However, to ensure the accuracy of the estimated causality, these methods need to be carefully validated by assessing the accuracy of the underlying models in predicting the neural signals.

To measure causality in spike-field networks, one prior approach applied Granger causality by operating on the power spectrum of the spike trains computed using multitaper techniques in sliding bins [61]. This method is non-parametric and model-free. Thus, new methods are needed to produce an encoding model of spike-field network interactions as encoding models are often desired. Further, computing the spectrum required stationarity and did not consider the behavior. As a dynamic behavior such as movement makes the spectrum change over time, new methods are needed when a dynamic behavior is being performed to take it into account. Another prior approach computes a joint probability density across all signals [62]. But because computing this density is complex, this approach has focused on smaller networks, for example consisting of 3 signals or nodes [62]. As modern datasets contain recordings from tens or hundreds of electrodes, there is a need to develop and validate methods that can assess multiscale causality across larger networks. Further, as the predictor for field potentials, this prior approach directly used the binary spike trains [62]. However, compared with the measured binary spike trains, the firing rates of neurons may be a better predictor of field potentials [30,63]; but these firing rates are not measurable and are instead latent. Thus, firing rates are challenging to incorporate in a model of field potentials.

To address the above challenges in computing multiscale causality between binary and continuous spike-field signals, we recently developed a model-based Granger-like theoretical framework and tested it within numerical Monte Carlo simulations [64]. In this model-based Granger-like framework, a spike train is modeled as a point process GLM in which the instantaneous firing rate of each neuron is modeled as a function of the history of both the spike trains and the field potentials across the network. Further, in addition to their own history, field potentials can be modeled either as a function of the history of the measurable binary spike trains or the history of the latent firing rates. This framework resolves the above challenges through three main ideas. First, to enable assessing causality across networks consisting of several nodes/signals, it assumes that conditioned on the history of all signals—i.e. all spike trains and field potential time-series—, spike trains and field potentials at a given time are conditionally independent and thus their models can be separately estimated. Second, to enable using the latent firing rates of neurons as predictors of field potentials, the framework consists of a sequential maximum-likelihood estimation approach. It first fits the GLM model of the spike trains to estimate the latent firing rates and then uses these estimated latent firing rates to fit the model of field potentials. Third, time-varying dynamic behavior is modeled as covariates in both spike and field models. Thus, a stable measure of causality during behavior can be estimated. Together, these ideas resolve the challenge of estimating a large number of parameters, allow for firing rates to be predictors of field potentials, consider behavior, and generate a network encoding model. The two expanded parametric models can then be used to compute a multiscale causality measure in terms of directed information.

While this multiscale causality framework was shown effective within Monte Carlo simulations [64], its statistical tests to assess causality to field potential signals required repetitively splitting the data, which resulted in reduced data efficiency and redundant computation. To make the method suitable for real neural datasets, especially when assessing causality for a large number of signals in modern datasets that contain recordings from tens of electrodes, it is important to improve the data efficiency of the method because real datasets are length-limited and contain a large number of electrodes. Moreover, the framework was only tested in Monte Carlo simulations, and so it remained unclear whether multiscale causality can be recovered in more realistic biophysical simulations—where there exits a model mismatch between the fitted model and the ground truth network—or in real neural datasets. Further, it is not known whether multiscale spike-field network causalities exist in real neural data, and whether modeling these multiscale causalities would improve the prediction of neural signals beyond what is possible with single-scale causality. Finally, it is not known whether field potentials are better predicted from the history of the binary spike trains or that of the latent firing rates.

Here, we address the above problems by developing a new data-efficient statistical test procedure within our multiscale network causality method based on the Wald test [65], and by then demonstrating the multiscale network causality method on both biophysical simulations and experimental data collected from two non-human primates (NHP) during an arm reaching behavior. We refer to the network model with the identified causal connections as the multiscale spike-field causality graph. First, we show that our new test procedure is more data-efficient and can thus perform better with the same amount of training data. Second, we show that the method can identify multiscale causality in more realistic biophysical simulations of spike-field network activity in which the ground-truth network structure is known. Then, we apply the method to NHP spike-field data during motor behavior and show that given its data-efficiency, the method can perform causality tests within high-dimensional recordings from over 100 electrodes. Also, our results on NHP data suggest the existence of multiscale spike-field network causality. Further, we find that the multiscale causality method leads to models that improve the prediction of both spike trains and field potentials in NHP data. Finally, in the NHP datasets, we find that latent firing rates are better predictors of field potentials and explore the properties of different directed connections, such as their density as a function of distance or their density within vs. across brain regions.

2. METHODS

In this section, we first describe the spike-field causality graph identification problem. Then we describe the multiscale encoding model and the causality graph identification procedure. Finally we describe the experimental data during a motor task and the model evaluation measures.

2.1. Multiscale spike-field causality graph

We define a multiscale network as a collection of spike trains and field potentials, which we refer to as multiscale signals. The multiscale signals consist of the binary spike trains from $C$ neurons and $D$ continuous-valued field potential time-series. We denote the sets of $C$ spike trains and $D$ field potentials by $𝒩=\left\{N^1,N^2,\dots ,N^C\right\}$ and $𝒴=\left\{y^1,y^2,\dots ,y^D\right\}$, respectively. To allow for the possibility that the timescale of spike trains is faster than field potentials, our model can use a smaller sample time for spike trains than that of field potentials if needed. In the model, spike trains are observed at time $t$ and field potentials are observed at time $mt$, where $m$ is the ratio between the sampling frequencies of field potentials and spike trains as a positive integer. Then the value of the $q$-th spike train $N^q$ at time $t$ can be denoted by $N_t^q$, and the value of the $q$-th field potential $y^q$ at time $mt$ can be denoted by $y_{mt}^q$ (where the $q$-th signal could be the one recorded from the $q$-th recording channel for example).

We compute the causality between spike trains and field potentials by developing a multiscale model and using it to compute a directed information measure [60, 64], as we now describe. Assume that we have a recording length of $n$, i.e. $t\in [1,2,\dots ,n]$. We denote the likelihood function of the field potential $y_{mt}^q$, causally conditioned on all signals in the network up to time $mt-1$, by $f_t(y_{mt}^q||𝒩\cup 𝒴\backslash \{y^q\})$. Here $𝓐\cup 𝓑$ indicates the union of sets $𝓐$ and $𝓑$ and $𝓐\backslash 𝓑$ indicates the relative complement of set $𝓑$ in set $𝓐$, i.e. the elements in set $𝓐$ that do not belong to $𝓑$. Similarly, we denote the likelihood function of a spike train $N_{mt}^q$, causally conditioned on all signals in the network up to time $mt-1$, by $f_t(N_{mt}^q||𝒩\cup 𝒴\backslash \{N^q\})$. Thus the conditional likelihood function of the $n$ samples of $y^q$ and $N^q$ can be denoted by $f_n(y^q||𝒩\cup 𝒴\backslash \{y^q\})$ and $f_n(N^q||𝒩\cup 𝒴\backslash \{N^q\})$, respectively. We have

$$f_n(y^q||𝒩\cup 𝒴\backslash \{y^q\})=\prod _{t=1}^n{f}_t(y_{mt}^q||𝒩\cup 𝒴\backslash \{y^q\})$$ (1)

$$f_n(N^q||𝒩\cup 𝒴\backslash \{N^q\})=\prod _{t=1}^n{f}_t(N_{mt}^q||𝒩\cup 𝒴\backslash \{N^q\})$$ (2)

We now define the causality measure. Consider a signal $z^i$ and a signal $w^j$, which can each be either spike train or field potential. Directed information [53–56,60,64] as a measure of causality from $z^i$ to $w^j$ is defined as the log-likelihood ratio

$$\begin{array}{l}{I}_n(z^i\to w^j||𝒩\cup 𝒴\backslash \{z^i,w^j\})=\\ \text{ln}\frac{f_n(w^j||𝒩\cup 𝒴\backslash \{w^j\})}{f_n(w^j||𝒩\cup 𝒴\backslash \{w^j,z^i\})}=\\ \text{ln}{f}_n(w^j||𝒩\cup 𝒴\backslash \{w^j\})-\text{ln}{f}_n(w^j||𝒩\cup 𝒴\backslash \{w^j,z^i\}).\end{array}$$ (3)

Given this definition, there is no causality from $z^i$ to $w^j$ if and only if

$$f_n(w^j||𝒩\cup 𝒴\backslash \{w^j\})=f_n(w^j||𝒩\cup 𝒴\backslash \{w^j,z^i\}).$$ (4)

Intuitively, having no causality means that knowing the signal $z^i$ is not helpful in predicting the signal $w^j$ if one already knows all signals other than $z^i$ and $w^j$.

We can also define a directed information rate as

$$\begin{array}{c}I(z^i\to w^j||𝒩\cup 𝒴\backslash \{z^i,w^j\})=\\ \frac{1}{n}[\text{ln}{f}_n(w^j||𝒩\cup 𝒴\backslash \{w^j\})\\ -\text{ln}{f}_n(w^j||𝒩\cup 𝒴\backslash \{w^j,z^i\})].\end{array}$$ (5)

Note that the history of all signals in the network are considered in Equation (4) to provide a measure of conditional causality. However, we can also condition on only the history of a subset of the network. In a special case, we can condition on no other network nodes to study unconditional causality. In this case, we can say that there is no unconditional causality from $z^i$ to $w^j$ if and only if

$$f_n(w^j||\{z^i\})=f_n(w^j).$$ (6)

There are four possible kinds of causality in a multiscale network: spike-spike, spike-field, field-spike, and field-field. We can represent these causalities using an unweighted directed graph named causality graph. In this graph, each vertex represents a spike train or a field potential and each directed edge represents causality between two vertices.

2.2. Causality graph identification

Having defined the causality measure in the multiscale spike-field network, we need to estimate the likelihood models in Equation (1) and Equation (2) to compute it and develop a statistical test to assess the significance of causality. This estimation is based on our proposed multiscale parametric models defined above as we expand on below. We also devise the statistical tests for causality assessments below.

2.2.1. Spike train model and causality to spike trains.

A binary spike train can be modeled as a point-process GLM that depends on various covariates [33, 64], including the history of all observed spike trains, field potentials and the current behavioral states. Here we denote the behavioral states at time $mt$ by ${\mathit{u}}_{mt}$. For each spike train $q$ at time $mt$, the model is fully specified by the instantaneous firing rate function ${\text{λ}}_{mt}^q$.

Given any subset of spike trains ${𝒩}_{\text{sub}}\subseteq 𝒩\backslash \left\{N^q\right\}$ and any subset of field potentials ${𝒴}_{\text{sub}}\subseteq 𝒴$ we write the logarithm of the firing rate of the $q\text{th}$ spike train $N^q\in 𝒩$ within the GLM framework as

$$\begin{gathered} \text{ln}{\text{λ}}_{mt}^q({𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}})= \\ {\beta }_0^q+{\sum }_{N^j\in {𝒩}_{\text{sub}}\cup \{N^q\}}{\mathit{\beta }}_1^{q{j}^{\prime }}{\mathit{N}}_{mt-1,mt-m{K}_{\text{s}}^q}^j \\ +{\sum }_{y^j\in {𝒴}_{\text{sub}}}{\mathit{\beta }}_2^{q{j}^{\prime }}{\mathit{y}}_{m(t-1,t-K_{\text{s}}^q)}^j+{\mathit{\beta }}_3^{q^{\prime }}{\mathit{u}}_{mt} \end{gathered}$$ (7)

where $K_{\text{s}}^q$ is a non-negative integer representing the length of history. Here $\beta ’\text{s}$ with subscripts and superscripts are parameters to be estimated. All $\mathit{\beta }’\text{s}$ are vectors except ${\beta }_0^q$. ${\mathit{A}}_{a,b}$ denotes the vector $\left[A_a,A_{a+1},\dots ,A_b\right]$ and ${\mathit{A}}_{m(a,b)}$ denotes the vector $\left[A_{ma},A_{ma+m},\dots ,A_{mb}\right]$. We note that all samples of spike trains from time $mt-m{K}_{\text{s}}^q$ to time $mt-1$ are included as covariates in the GLM model above as can be seen in the first summation on the right side of Equation (7).

We denote the set of all unknown parameters (all ${\beta }^{’}\text{s}$) in Equation (7) by $\theta$. Elements in $\theta$ need to be estimated using maximum likelihood estimation (MLE) by maximizing the point process likelihood function of $N_{mt}^q$ written as

$$\begin{gathered} f_t(N_{mt}^q||{𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}})= \\ {(\text{Δ}{\text{λ}}_{mt}^q({𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}}))}^{N_{mt}^q}{\text{e}}^{-\text{Δ}{\text{λ}}_{mt}^q({𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}})} \end{gathered}$$ (8)

where $\text{Δ}$ is the time-step used to bin the spikes [33,64]. Note that the likelihood of spike trains is evaluated at times $mt$ because the field potential signal is only available at these times. However, the firing rates at time $mt$ are evaluated using all samples of spike trains at every time-step from time $mt-m{K}_{\text{s}}^q$ to time $mt-1$ as seen from Equation (7).

The MLE is given by

$$\widehat{\theta }=\underset{\theta }{\text{arg min}}[-\text{ln}{f}_n(N^q||{𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}})].$$ (9)

To find the set of signals that cause $N^q$, we need to select the smallest sets ${𝒩}_{\text{sub}}$ and ${𝒴}_{\text{sub}}$ such that $f_n(N^q||{𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}})=f_n(N^q||𝒩\cup 𝒴\backslash \left\{N^q\right\})$. This would mean that conditioning on signals other than the ones in ${𝒩}_{\text{sub}}$ and ${𝒴}_{\text{sub}}$ does not improve the likelihood function and thus all and only signals in ${𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}}$ cause $N^q$. To select the hyperparameters ${𝒩}_{\text{sub}}$, ${𝒴}_{\text{sub}}$, and $K_{\text{s}}^q$ (i.e. history length), we devise a model selection procedure based on comparing $f_n(N^q||{𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}})$ with different hyperparameters [64]. We use the Akaike information criterion (AIC) for this selection [66] and thus for a given ${𝒩}_{\text{sub}}$ and ${𝒴}_{\text{sub}}$ we pick $K_{\text{s}}^q$ as follows:

$${\widehat{K}}_{\text{s}}^q=\underset{K_{\text{s}}^q}{\text{arg min}}\{\underset{\theta }{\text{min}}[-\text{ln}{f}_n(N^q||{𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}})+\delta K_{\text{s}}^q]\}$$ (10)

where $\delta K_{\text{s}}^q$ is the number of parameters with $\delta =m(|{𝒩}_{\text{sub}}|+1)+|{𝒴}_{\text{sub}}|$ and with $|\cdot |$ being the operator counting the number of elements in a given set. To determine whether a signal $z\in {𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}}$ causes $N^q$, by Wilk’s theorem [67], under the hypothesis that $z$ does not cause $N^q$, we have

$$2I_n(z\to N^q||{𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}}\backslash \{z\})\sim {\chi }^2(c{K}_{\text{s}}^q)$$ (11)

where ${\chi }^2(k)$ denotes a chi-squared distribution with degree of freedom $k$, $c=1$ if $z$ is a spike train and $c=m$ if $z$ is a field potential. If the p-value in this test is smaller than a desired threshold, we say $z$ significantly causes $N^q$. This enables us to remove any signal in a given set ${𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}}$ that does not significantly cause the modeled spike train. We will further explain the model selection procedure in Section 2.2.3. For tests on spike train models, the threshold of p-value is set to 0.05 and we also control false discovery rate (FDR) using Benjamini-Hochberg procedure [68].

2.2.2. Field potential model and causality to field potentials.

We model a field potential using a linear Gaussian model. We assume a field potential $y^q\in 𝒴$ is Gaussian distributed with time-variant mean ${\mu }_{mt}^q$ at time $mt$ and fixed variance ${\sigma }_q^2$. The mean of the field potential depends on various covariates, including the history of observed field potentials, the history of spike firing rates (which need to be estimated) and the current behavioral states. Instead of the history of spike firing rates, it is easy to use the history of the observed binary spike trains as predictors as well because these spike trains are directly observed. We can thus study whether the binary events or the firing rates are better predictors of the field potentials. Given any subset of spike trains ${𝒩}_{\text{sub}}\subseteq 𝒩$ and any subset of field potentials ${𝒴}_{\text{sub}}\subseteq 𝒴\backslash \left\{y^q\right\}$, we can write the mean of the field potential as

$$\begin{gathered} {\mu }_{mt}^q({𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}})= \\ \begin{array}{c}{\alpha }_0^q+{\sum }_{N^j\in {𝒩}_{\text{sub}}}{\mathit{\alpha }}_1^{q{j}^{\prime }}\text{ln}{\mathbf{\text{λ}}}_{m(t-1,t-K_{\text{f}}^q)}^j\\ +{\sum }_{y^j\in {𝒴}_{\text{sub}}\cup \{y^q\}}{\mathit{\alpha }}_2^{q{j}^{\prime }}{\mathit{y}}_{m(t-1,t-K_{\text{f}}^q)}^j+{\mathit{\alpha }}_3^{q^{\prime }}{\mathit{u}}_{mt}\end{array} \end{gathered}$$ (12)

where $K_{\text{f}}^q$ is the history length for spike-field and field-field causality, which is a non-negative integer. Here $\alpha ’\text{s}$ with subscripts and superscripts are parameters to be estimated. Thus, the Gaussian likelihood function of the field potential is given by

$$\begin{gathered} f_t(y_{mt}^q||{𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}})= \\ {(\sqrt{2\pi }{\sigma }_q)}^{-1}{\text{e}}^{-{[y_{mt}^q-{\mu }_{mt}^q({𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}})]}^2{(2{\sigma }_q^2)}^{-1}}. \end{gathered}$$ (13)

We denote the set of all remaining unknown parameters by $\theta$ including all $\alpha ’\text{s}$ in Equation (12) and $\sigma$ in Equation (13). Elements in $\theta$ still need to be estimated using MLE by maximizing the Gaussian likelihood function as

$$\widehat{\theta }=\underset{\theta }{\text{arg min}}[-\text{ln}{f}_n(y^q||{𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}})].$$ (14)

We now have to perform model selection and statistical tests of causality for the field potential models. In our previous work, this was done through a data-expensive and redundant empirical data-splitting process [64]. Here, we improve on the data efficiency of this method by devising an alternative analytical approach as described in the next paragraphs. The new method is better suited for neural data analyses as real datasets are length-limited and can contain recordings from many electrodes, thus introducing a large number of parameters and signals.

The new analytical method is devised as follows. Once $\alpha ’\text{s}$ are estimated, the covariance matrix of these estimates – i.e., covariance of $\widehat{\alpha }’\text{s}$ – denoted by $V$ plays an important role in model selection procedure and statistical tests for causality as we will show below. Finding this covariance matrix is not straightforward. This is because we use the history of spike firing rates as predictors of field potentials and thus the spike train models have to be estimated before the field potential models can be estimated, resulting in a 2-step MLE problem for the field potential models. This means that in the second step of MLE, there are nuisance parameters $\widehat{\beta }’\text{s}$ in the spike models that are estimated from the first step. The latent spike firing rates computed using these nuisance parameters are different from the true values, and thus $V$ is also related to the variances of $\widehat{\beta }’\text{s}$.

We thus need to estimate $V$, which is the covariance of the $\widehat{\alpha }$ parameters of the field potential model in the presence of nuisance parameters. To do so, we leverage a method developed for this covariance estimation in the presence of nuisance parameters [65]. Let’s use ${\nabla }_{\zeta }$ to denote the first order derivatives of a function w.r.t. $\zeta$ and ${\nabla }_{\zeta \eta }^2$ to denote the second order derivative of a function w.r.t. $\zeta$ and $\eta$. We define $G_{\alpha }=-n^{-1}{\nabla }_{\alpha \alpha }^2\text{ln}{f}_n\left(y^q\right)$ as the second order derivative of $-n^{-1}\text{ln}{f}_n\left(y^q\right)$ w.r.t. $\alpha ’\text{s}$. Let $G_{\beta }=-n^{-1}{\nabla }_{\alpha \beta }^2\text{ln}{f}_n\left(y^q\right)$ be the second order derivative of $-n^{-1}\text{ln}{f}_n\left(y^q\right)$ w.r.t. ${\beta }^{’}\text{s}$ then $\alpha ’\text{s}$, and $M=-n^{-1}{\nabla }_{\beta \beta }^2\text{ln}{f}_n(N)$ with $-n^{-1}\text{ln}{f}_n(N)=-n^{-1}[\text{ln}{f}_n\left(N^1\right)+\text{ln}{f}_n\left(N^2\right)+\dots +\text{ln}{f}_n\left(N^C\right)]$. Also let ${\text{Ω}}_{\alpha }=n^{-1}{\sum }_{t=1}^n{\nabla }_{\alpha }\text{ln}{f}_t\left(y_{mt}^q\right){\nabla }_{\alpha }\text{ln}{f}_t{\left(y_{mt}^q\right)}^{\prime }$, and ${\text{Ω}}_{\beta }=n^{-1}{\sum }_{t=1}^n{\nabla }_{\beta }\text{ln}{f}_t\left(N_{mt}\right){\nabla }_{\beta }\text{ln}{f}_t{\left(N_{mt}\right)}^{\prime }$ where $f_t\left(N_{mt}\right)={\left[f_t\left(N_{mt}^1\right),\dots ,f_t\left(N_{mt}^C\right)\right]}^{\prime }$ (see Equations (1), (2)). It can be shown that [65]

$$\begin{gathered} V=n^{-1}{G}_{\alpha }^{-1}{\text{Ω}}_{\alpha }{G}_{\alpha }^{-1} \\ +n^{-1}{G}_{\alpha }^{-1}{G}_{\beta }{M}^{-1}{\text{Ω}}_{\beta }{M}^{-1}{G}_{\beta }^{\prime }{G}_{\alpha }^{-1}. \end{gathered}$$ (15)

All variables in Equation (15) are evaluated at the estimated value of the parameters, i.e., at $\widehat{\alpha }$ and $\widehat{\beta }$. The first term in $V$ above is the same as the typical covariance in a 1-step MLE. The second term in $V$ is contributed by the uncertainty of the nuisance parameters $\widehat{\beta }’\text{s}$ due to the 2-step nature of the MLE here that affects the field potential model. Now that we have an estimate of $V$, we can formulate the model selection procedure and statistical tests for assessing causality to field potential signals.

A model selection procedure is required when ${𝒩}_{\text{sub}}$, ${𝒴}_{\text{sub}}$, or $K_{\text{f}}^q$ are undetermined. For given ${𝒩}_{\text{sub}}$ and ${𝒴}_{\text{sub}}$, we select the history length $K_{\text{f}}^q$ using an information criterion. For the multiscale model, there is uncertainty in $\widehat{\alpha }’\text{s}$ contributed by $\widehat{\beta }’\text{s}$, i.e., the second term in $V$ in Equation (15) and so AIC is not directly applicable. However, we can apply AIC to a model with only field potential signals. Thus, taking the history of spike-field connections to be no longer than the history of field-field connections in this case, we select the length of history in the multiscale model as:

$${\widehat{K}}_{\text{f}}^q=\underset{K_{\text{f}}^q}{\text{arg min}}\{\underset{\theta }{\text{min}}[-\text{ln}{f}_n(y^q||{𝒴}_{\text{sub}})+\delta K_{\text{f}}^q]\}$$ (16)

where $\delta K_{\text{f}}^q$ is the number of parameters with $\delta =|{𝒴}_{\text{sub}}|+1$.

The estimated $V$ is then applied to statistical tests for assessing causality. We use the Wald test [65] to test whether a sub-vector of $\alpha$ is 0 in Equation (12), and thus to determine whether a signal $z\in {𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}}$ causes $y^q$. The Wald statistic follows a ${\chi }^2$ distribution under the null hypothesis ${\alpha }_s=0$ and is given by

$$W={\alpha }_s^{\prime }{V}_s^{-1}{\alpha }_s\sim {\chi }^2(r)$$ (17)

where ${\alpha }_s$ is any sub-vector of $\alpha$, $V_s$ is a sub-matrix of $V$ with selected rows and columns corresponding to ${\alpha }_s$ and the degree of freedom $r$ is the length of ${\alpha }_s$. To assess causality from a signal $z$ to a field potential $y^q$, we consider the ${\alpha }_s$ that only contains parameters related to $z$; we then say $z$ does not cause $y^q$ when the p-value corresponding to the above test is larger than a threshold. We use a threshold of 0.05 for statistical tests and do separate FDR controls on spike-field and field-field connections in the following sections.

2.2.3. Model selection and causality assessment.

To find the set of signals that cause a spike train or a field potential, we use the above statistical tests described in Section 2.2.1 and 2.2.2. We now describe how we assess causality to a spike train and to a field potential signal, respectively.

For a spike train $N^q$ and another specific signal, we first model $N^q$ using its own history and the history of that other signal to decide if that specific signal unconditionally causes $N^q$. We then construct the set ${𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}}$ to consist of all signals that significantly unconditionally cause $N^q$. This allows us to narrow down the number of signals for which we need to compute the conditional causality to $N^q$. We then proceed to testing the conditional causality within this set ${𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}}$ using Equation (11). In particular, we determine if each signal $z$ in ${𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}}$ significantly conditionally causes the modeled signal (i.e. when knowing the history of all other signals in set ${𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}}$, does knowing $z$ still help the prediction of $N^q$). At a given significance level, we eliminate every insignificant $z$ from the set after all tests are done. This procedure provides the set of signals that significantly cause a spike train.

For a field potential signal $y^q$, we first take ${𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}}$ to be $𝒩\cup 𝒴\backslash \left\{y^q\right\}$ i.e. all of the signals being recorded except $y^q$ itself. However, because these sets are high-dimensional, to reduce the computation complexity of model selection and make the method applicable to large channel counts, we devise a preprocessing procedure to reduce the number of signals that need to be considered in model-selection, i.e. the number of signals in ${𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}}$. First, we fit a single-scale model with only field-field connections and remove the insignificant connections after FDR control from ${𝒴}_{\text{sub}}$. The lengths of history are also selected using Equation (16). We then compute a residual for each field signal at each time point by subtracting the single-scale one-step-ahead prediction of the signal from its observed value (predicting the signal at the current time based on all past values of field potentials, see Section 2.4). This residual indicates the residual part of the field potential signal that is not predicted by the other field potential signals. Second, we consider each pairwise spike-field connection by fitting a pairwise field model for each residual series. Spike-field connections with p-values larger than threshold 0.05 are removed from ${𝒩}_{\text{sub}}$. Together, these preprocessing steps provide us with a subset of signals ${𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}}$ before we do model selection for the multiscale model, thus reducing the complexity and allowing for causality assessment in large channel-sets.

Having obtained the ${𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}}$, we fit the multiscale model and look at the p-values computed by Equation (17) for the causality of spike trains to the field potential signal and remove the spike train whose p-value is the largest and also larger than a relatively large threshold of 0.95 from the set and then fit the model again. This threshold means that the removed spike train has less than 5% chance of having a causal connection to the field potential and thus helps avoid false negatives. We repeat this removal and refitting procedure until all p-values corresponding to spike trains are smaller than the threshold. Then the remaining signals in ${𝒩}_{\text{sub}}\cup {𝒴}_{\text{sub}}$ are the candidates that may cause $y^q$. Now, the p-values computed by Equation (17) are used to find all significant causalities to field potential signals (see also Section 2.2.2).

Based on prior evidence about the length of history for spike and field potential dynamics and for computational tractability, we consider a history length of up to 100ms for a spike train [33] and up to 300ms for a field potential [30] in our model selection procedure. History lengths around these values are generally considered long enough for spike train and field potential dynamics [30,33]. This means that we allow $K_{\text{s}}^q$ to be up to 10 and $K_{\text{f}}^q$ to be up to 30 as the NHP data is sampled at 100 Hz, i.e. at a time-step of $\text{Δ}=10\text{ms}$.

2.3. Motor task and neural data recording

We use multiscale recordings from motor cortical areas of two Rhesus macaques performing a motor task [69]. The NHPs reached to objects located at different 3D locations for a liquid reward and then returned to the resting position before initiating another reach as described in [69]. For the first NHP J, 7 sessions of recordings were obtained from the dorsal premotor cortex (PMd), ventral premotor cortex (PMv), primary motor cortex (M1), and prefrontal cortex (PFC) contralateral to the arm using an array of 137 electrodes (Gray Matter Research, USA) where neighboring electrodes were spaced 1.5 mm apart in X and Y directions and the depth in Z direction was adjusted by the experimenter. For the second NHP C, 4 sessions of recordings were obtained from left PMd, right PMd, left PMv and right PMv using 128 electrodes in total from 4 arrays (Gray Matter Research, USA). Arm movements were tracked by reflective body markers attached to the subject’s skin. Marker locations were captured by near infrared cameras with sampling frequency of 100 frames per second (Motion Analysis Corp., USA). Behavioral covariates $u_t$ were taken as the 7 joint angle time-series corresponding to the shoulder, elbow and wrist movements (shoulder elevation, elevation angle, shoulder rotation, elbow flexion, pro supination, wrist flexion, and wrist deviation) [8]. Neural data was originally recorded with a sampling rate of 30 kHz. All surgical and experimental procedures were performed in compliance with the National Institute of Health Guide for Care and Use of Laboratory Animals and were approved by the New York University Institutional Animal Care and Use Committee.

As described in [69], local field potentials (LFPs) were extracted by applying a low-pass filter with 400 Hz cut-off frequency on the raw signals. Spike trains were obtained by band-pass filtering the raw signal from 0.3 to 6.6 kHz and then identifying the threshold crossings below the mean filtered signal. The threshold was set to 3.5 standard deviations [69]. We fit models both for all available channels in each monkey and for selected higher-quality channels. In the latter case, we eliminated low-quality noisy spike trains by keeping the 10 spike trains that can be best predicted using only behavioral states as predictors in GLM models for each session [8]. We then also used the field potentials recorded from these channels in our network causality analyses. Here we use the same sampling rate for spikes and LFPs of 100 Hz (i.e. $m=1$).

2.4. Prediction power

In the biophysical simulations, we can know the ground truth causality graph and compare that to the estimated causality graph. However, this is not the case for real neural data. As such, we evaluate the estimated causality graph in real data by asking how well the multiscale model with that causality graph can predict the spike trains and field potentials. In other words, we take the fitted multiscale model corresponding to an identified causality graph. We then use this fitted model to predict the signals one-step-ahead into the future. This means that we assess how well a signal can be predicted by the history of those signals that are identified as causing it within a fitted multiscale model. This way we can compare different causality graphs by comparing the prediction of their corresponding multiscale models. We refer to this one-step-ahead prediction capability as prediction power (PP) and define it for spike trains and field potentials as follows.

For each spike train, the one-step-ahead prediction of the continuous valued firing rates ${\text{λ}}^q$ in Equation (7) can be computed from a model based on any given causality graph. Then given any threshold, we can predict the binary spike $N_{mt}^q$ to be 1 if the one-step-ahead predicted firing rate exceeds the threshold and to be 0 otherwise. We can then construct a receiver operating characteristic (ROC) curve by sweeping the threshold and compute the area under the curve (AUC) of the ROC. We define the predictive power (PP) as PP=2×AUC-1 [8,13,42]. The range of PP is [0,1] in the training set but PP could be smaller than 0 in the test set. A perfect predictor achieves a PP score of 1 and a random predictor achieves a PP score of 0.

For each field potential, we find the one-step-ahead prediction of the field potential $y^q$ by computing ${\mu }^q$ from the multiscale model in Equation (12) based on the history of the signals that are identified as causing it in the identified causality graph. We define the normalized root mean-squared error (NRMSE) (the root mean-squared error normalized by the standard deviation of $y^q$ [70]) as

$$\text{NRMSE}=\sqrt{\sum _t{({\mu }_{mt}^q-y_{mt}^q)}^2/\sum _t{({\overline{y}}^q-y_{mt}^q)}^2}$$

where ${\overline{y}}^q$ is the mean of $y^q$ and the prediction ${\mu }_{mt}^q$ is given in Equation (12). Given the NRMSE, for LFPs, we define PP=1-NRMSE. Thus the range of PP is also [0,1] in the training set but PP could be smaller than 0 in the test set. A perfect predictor achieves a PP score of 1 and a chance-level predictor achieves a PP of 0.

Our statistical tests are on functions of PP, for example on the PP increase ratio defined in Figure 3 that shows the ratio of improvement in PP between different cases (e.g., multiscale vs. single-scale). Further, we use cross-validation to evaluate PPs and assess statistical significance in tests on functions of PP.

Figure 3.

(a–f) Multiscale causalities are successfully detected, thus improving the prediction of spike trains and field potentials. Signals—whether spike train signals in (a–c) or field potential signals in (d–f)—are sorted by the 95% confidence bound of PP increase ratios (these ratios are defined in the legends). PP increase ratios are significantly above 0 for signals to the right of the black vertical lines (p-value<0.05, one-sided t-test). Box plots to the right of each panel show the statistics of significant PP increase ratios, where the central mark is the median, the edges of the box are the 25th and 75th percentiles and the whiskers extend to the most extreme data points that are not outliers. Outliers are shown by red crosses and are points whose distance to the box is larger than 1.5 times of the box height. P-values are shown for comparing the average to 0 by one-sided t-tests. (a–c) Multiscale causalities improve the prediction of spike trains. (a) PP increase ratio from baseline to multiscale. Signals from all 11 sessions with multiscale connections are shown. (b) PP increase ratio from single-scale to multiscale. (c) PP increase ratio from cross-scale to multiscale. (d–f) Multiscale causalities improve the prediction of field potentials. (d) PP increase ratio from baseline to multiscale. Signals from all 11 sessions with multiscale connections are shown. (e) PP increase ratio from single-scale to multiscale. (f) PP increase ratio from cross-scale to multiscale.

Note we report p-values using the scientific notation used in MATLAB in which $xe-n$ denotes $x\times {10}^{-n}$.

3. RESULTS

In this section, we first show, using Monte-Carlo simulations, that the new statistical test on causality to field potentials in our multiscale causality method introduced in Section 2.2.2 works successfully and is data-efficient. We then use biophysical simulations with a known ground-truth causality graph to show that our method can identify the ground truth multiscale causality graph in realistic simulations of multiscale spike-field signals. Finally we use our method to examine multiscale causality in NHP spike-field neural data during a motor behavior.

3.1. Simulation showing data-efficiency of the newcausality method

To show data-efficiency of the statistical test for assessing causality to field potentials, we compare the new test to our original test in [64] using the same simulated data from [64]. In [64], 100 randomly generated 10-node networks with 16 connections were used to evaluate the performance of our original multiscale causality method. Note in our prior work, we only performed Monte Carlo simulations to validate the approach [64]. We apply our new statistical test to the 100 random networks and evaluate the AUC from data lasting for different lengths of 3-min to 24-min.

Figure 1 shows the comparison of AUCs of the two tests. We can observe that for any given length of data shown in Figure 1a, the new method has larger AUC than the original method used in [64]. At 3 min, the overall AUC using the new method is significantly larger than the original AUC by 2.1% (p-value=1.6e-2, one-sided paired t-test). At 12 min, the overall AUC using the new method is again significantly larger than the original AUC by 9.0% (p-value=8e-20, one-sided paired t-test). Furthermore, the new AUC achieved using 6-min data is larger than the original AUC using even 12-min data (p-value=0.008, one-sided paired t-test). These results show that the new method performs as well or better than the original method using much less training data and thus is more data-efficient than the original method.

Figure 1.

Comparison of new and original test procedures on causality to field potentials shows that the new method is more data-efficient. (a) AUC of field-field and spike-field connections with errorbar showing mean ± sem. (b) AUC of field-field connections with errorbar showing mean ± sem. (c) AUC of spike-field connections with errorbar showing mean ±sem. The new method can perform as well or better than the original method using much less training data.

3.2. Biophysical simulation

We use the Virtual Electrode Recording Tool for EXtracellular potentials (VERTEX) [71] to generate data from biophysical simulations. Given the position of electrodes, properties of neurons and tissues, VERTEX can calculate extracellular LFPs using a forward modeling approach [72].

3.2.1. Simulation settings.

The purpose of this simulation is to show that our method can identify the ground truth causality graph in a realistically simulated multiscale spike-field network. We simulate 10 sessions of spike-field data, with each session lasting 200 s. The placement of neurons are randomized for each session. For generality, each random simulated network includes all 4 types of causality: spike-field, field-field, spike-spike, field-spike. The overall simulated network structures are the same in each session but neurons are randomly placed in 4 different 3-D regions (A-D squares in the x-y plane in Figure 2a). The size of each region is 1 mm × 1 mm × 0.2 mm. The minimum distance between two connected regions is 6 mm. Neurons in a region are randomly placed in each session while electrodes recording LFPs are placed as shown in x-y plane with height $z=0.1\text{mm}$.

Figure 2.

Multiscale causality identification method can find the directed connections within spike-field networks in biophysical simulations. (a) Simulated network structure consists of 6120 neurons as detailed in Section 3.2.1. The figure shows 60 neurons (orange circles) and 20 electrodes (blue crosses) in 4 different 3-D regions (A–D squares in x-y plane). The size of each region is 1 mm × 1 mm × 0.2 mm. The minimum distance between two connected regions is 6 mm. Neurons in a region are randomly placed in each session while electrodes recording LFPs are placed as shown in x-y plane with height $z=0.1\text{mm}$. A spike train is observed from each neuron. Connections between spike trains are shown in black arrows and are all from left to right. To have a clear ground truth between regions, 20 spike trains from regions A, D and 10 LFPs from regions B, C are modeled. (b–e) Ground truth and identified network between signals. The ground truth (b, d) and identified network (c, e) averaged across 10 sessions are shown. For (c, e), the colors indicate the ratio of sessions for which a positive connection was identified and the scale is shown by the color bar. (f–i) Receiver operating characteristic (ROC) curves of connectivity with shaded 95% confidence interval computed using the ground truth and identified networks. Our method can successfully identify (f) field-field (g) spike-field (h) field-spike and (i) spike-spike connections in the multiscale spike-field networks.

Each simulated network contains 6120 neurons. We chose our simulated network and its parameters such that we can have realistic firing rates with a reasonable number of simulated neurons and a known ground truth causality to which we can compare. Among all neurons, a network of 60 neurons (orange circles) and 20 electrodes (blue crosses) are shown in Figure 2a. The weight of the synapses is 1100 pA for every connection between these neurons. Each of the 60 displayed neurons has an input from another neuron not shown in the figure through a synapse with a weight of 4000 pA. This input neuron fires with Poisson statistics at a rate of 10 Hz if connected to a displayed neuron without other inputs or 5 Hz otherwise. Each of the 30 displayed neurons with no displayed input from the same region also has an output to another 200 neurons within the same region through synapses with a weight of 2200 pA. All neurons are excitatory pyramidal neurons. Synapses are single exponential current-based and the axon arbor is uniformly distributed within the region. Other synapse properties are specified in Table 1. The structural and passive properties are the same as the layer 2/3 pyramidal neuron model described in [71]. The 60 neurons shown in Figure 2a and the 6000 neurons driven by them follow the same adaptive exponential (AdEx) model [73] with parameters specified in Table 1. These neurons also have random current input following Ornstein Uhlenbeck process with parameters also specified in Table 1.

Table 1.

Parameters of neurons, synapses and input currents in the biophysical simulation. AHP here refers to after-hyperpolarization.

Name	Value
spike generation threshold	−50 mV
spike steepness parameter	1 mV
scale factor of AHP current	2.6 nS
AHP current time constant	25 ms
AHP current instantaneous change	600 pA
reset membrane potential	−60 mV
cutoff membrane potential	−45 mV
synapse exponential decay time constant	2 ms
axon conduction speed	0.3 m/s
synapse release delay	0.5 ms
input current mean	100 pA
input current std	30 pA
input current time constant	2 ms

The raw output from VERTEX includes spike timings and LFPs. The original sampling rate of LFPs is 1 kHz. Under the above simulation settings, neurons rarely spike more than once every 5 ms as expected. The maximum $K$ considered here is also 30 for field potentials and 10 for spikes. For spikes, we generate spike trains from spike timings. To obtain the spike trains, we generate time-series of 0’s and 1’s by binning the spikes in 5ms time bins. If there is a spike in a time bin, we set the value of the spike train at that time to 1 and if there is no spike in a time bin, we set the value of the spike train at that time to 0. For LFPs, we down-sample the LFP signals at 100Hz sample rate and standardize them to have zero mean and unit variance. Example simulated spike and LFP signals are shown in the Appendix.

3.2.2. Multiscale causal connections can be identified in biophysical simulations.

To have a clear ground truth between regions that we can then use for assessment, we select 20 spike trains from regions A and D and 10 LFPs from regions B and C as shown in Figure 2a. All our subsequent analyses will focus on these signals. The ground truth causality graph for these signals is shown in Figure 2b, d, with former showing the causality to the spike trains in area A and D and the latter showing the causality to LFPs in regions B and C. For spike-spike connections, the ground truth is the same as synapse connections and causal connections are shown with yellow. For the other 3 types of connections, each LFP in a region is caused by every direct synaptic input to that region and also causes every signal in that region and every signal in the next region with direct synaptic connections. Thus for these 3 types of connections, the true causal connections are shown between regions (rather than being at the level of synapse connections) in yellow.

We apply the multiscale causality algorithm to each session of the simulated data. A causality from one signal to another is defined as positive if the first signal causes the second. A positive causality in an identified network is true positive if the ground truth is positive. It is false positive if the ground truth is negative. Then we can also compute the true positive rate and false positive rate by dividing the number of true positives and false positives by the number of positives and negatives in ground truth, respectively.

The ratio of sessions for which a positive causality was identified from each signal to another signal is shown by the color in Figure 2c, e. Comparing with the ground-truth causal connections in Figure 2b, d, this figure suggests that the method is successful in identifying causal connections.

To quantify this performance, we plot the receiver operating characteristic (ROC) curves for different kinds of connections by sweeping the threshold of p-values from 0 to 1 (p-values used for causality detection). The average ROC curves across 10 sessions with shaded 95% confidence are shown in Figure 2f–i. We find the area under the ROC curve (AUC) in each case. The chance-level AUC is 0.5, which we compare to. As we can observe from Figure 2i, true spike-spike causalities are accurately detected and the associated AUC is 0.999 ± 0.001 (mean ± std), which is significantly larger than chance level of 0.5 (p-value=9e-26, one-sample one-sided t-test). In addition, for field-field, spike-field and field-spike causality, the AUCs are shown in Figure 2f–h and are 0.95 ± 0.04, 0.79 ± 0.10 and 0.91 ± 0.14, respectively. All these AUCs are also significantly larger than the chance level of 0.5 (p-values=2e-11, 5e-6 and 3e-6, respectively; one-sample one-sided t-test).

3.3. Multiscale causal connections improve prediction of spike trains and field potentials in NHP datasets

Having established that the multiscale causality graph can be significantly identified in biophysical simulations, we next examined whether multiscale causal connections were also identified in real NHP motor datasets and if so whether their identification improved the prediction of spike-field activity. To investigate these questions, we define four types of PP to be compared: baseline, single-scale, cross-scale, and multiscale. Multiscale PP is the PP of full models that contain all kinds of connections including multiscale causal connections between spike-field and field-spike, as well as self-history for each signal. Baseline PP is the PP of a model acquired by removing causal connections from the full models and considers only self-history in prediction of a given signal. Single-scale PP is the PP of models acquired by removing spike-field and field-spike connections from the full models. Cross-scale PP is the PP of models acquired by removing spike-spike and field-field connections except self-history from the full models. All PP’s are computed in cross-validation as described in Methods. As a first validation and for easier visualization, in Sections 3.3.1 and 3.3.2, we focus on the causality analyses with the higher-quality 10-channel sets. In later sections, we repeat these analyses when using all channels and further provide all the other results by including all of the channels in the analyses. All conclusions are similar in the 10-channel sets and the all-channel set analyses.

3.3.1. Spike train prediction.

Here we examine spike trains and find that the identified multiscale causalities improve the prediction of spike trains in cross-validation. To show a meaningful comparison, we compute these improvements for spike trains with at least one single-scale (spike-spike) and one cross-scale (field-spike) connection in all 5 cross-validation folds. In this section we focus on a high-quality 10-channel set for each session for easier visualization and validation, and then we confirm that our conclusions for all-channel analyses are similar in Section 3.3.3.

Figure 3a shows that multiscale PP is significantly larger than baseline PP for all signals with multiscale connections (p-value<0.05 by one-sided t-test). We then asked whether the PP improvement over baseline was due to single-scale or cross-scale causal connections being identified. To answer this question, we found the normalized improvement of multiscale PP vs. both single-scale PP (p-value = 2e-11, one-sided t-test) and cross-scale PP (p-value = 0.045, one-sided t-test) as in Figures 3b and 3c, respectively. We can see when considering both connection types instead of just spike-spike or field-spike connections, most of the PPs (97.0% and 83.6% of PPs) for a spike train are significantly improved (p-value<0.05 by one-sided t-test). Figure 3b shows the normalized PP improvement contributed by field-spike connections was on average 66.0% for the significant signals, while Figure 3c shows the normalized PP improvement contributed by spike-spike connections was on average 24.7% for the significant signals. This result suggests that both spike-spike and field-spike causal connections are successfully identified, leading to improved prediction of spikes. Further, this result suggests that multiscale connections (from field potentials to spikes) exist and their identification helps the prediction of spike trains.

Interestingly, overall on average across signals, cross-scale connections were more important compared to single-scale connections in improving PP of spike trains (p-value=8e-20 by one-sided Wilcoxon rank sum test). This result suggests that the unique contribution from field potentials to spike train prediction is more than that from the spike trains.

3.3.2. Field potential prediction.

We next consider field potentials and find that the identified multiscale connections also improve the prediction of field potentials. Here we focus on the high-quality 10-channel sets for easier visualization and validation and then confirm that our conclusions for all-channel analyses are similar in Section 3.3.3. Similar to Figure 3a–c, Figure 3d–f show the PP improvements for field potentials that had at least one field-field connection and one spike-field connection identified in all 5 folds. Multiscale PP significantly improved compared with baseline PP for all signals with multiscale connections (p-value<0.05 by one-sided t-test). We then asked whether the improvement over baseline was due to single-scale or cross-scale causal connections being identified. To answer this question, we found the normalized improvement of multiscale PP vs. both single-scale PP (p-value = 2e-4, one-sided t-test) and cross-scale PP (p-value = 8e-37, one-sided t-test) as in Figure 3e and 3f, respectively. We can see that when considering both spike-field and field-field connections instead of just one kind of connection, 73.3% and 97.8% of the PPs were significantly improved, respectively (p-value<0.05 by one-sided t-test). Figure 3f shows that the normalized PP improvement contributed by single-scale (field-field) connections was on average 88.4%, while Figure 3e shows that the normalized PP improvement contributed by cross-scale (spike-field) connections was on average 10.3%. This result suggests that both spike-field and field-field causal connections are successfully identified, leading to improved prediction of field potentials. Further, this result suggests that multiscale connections (from spike to fields) exist and that is why their identification helps the prediction of field potentials.

Comparing the single-scale and cross-scale contribution to PP’s reveals that for field potential prediction, single-scale causal connections are more important than cross-scale ones. On average, the normalized PP improvement contributed by single-scale (field-field) connections (88.4%) shown by Figure 3f is much larger than that by cross-scale (spike-field) connections (10.3%) shown by Figure 3e (p-value 3e-16 by one-sided Wilcoxon rank sum test). This result shows that the unique contribution to field potential prediction from field potentials is much larger than that from spike trains.

3.3.3. Prediction considering all signals.

We now consider all channels and study how much multiscale connections improve the prediction of signals by considering all observed signals. The models are fitted using the first 80% of samples and the PPs are computed using the last 20% of samples as test set. Similar to 10-channel sets, we find that the identified multiscale causal connections improve the prediction of both spikes and field potentials. For signals with multiscale connections, the mean of PP increase ratios of spikes from baseline to multiscale model was 58% (p-value 0.004; one-sided t-test). The mean of PP increase ratios of fields from baseline to multiscale model was 6.4% (p-value 2e-47; one-sided t-test). Compared with the 10-channel sets, the improvement from baseline to multiscale is larger for all-channel sets because there are more connections being modeled in this latter case. Also, the associated averaged PP increase ratios across sessions from baseline, single-scale or cross-scale to multiscale models are summarized in Table 2. Similar to 10-channel set results, more PP improvement is contributed by field-spike or field-field connections than spike-spike or spike-field connections. All analyses that follow are done by considering all channels together.

Table 2.

Mean and sem of averaged PP increase ratio across sessions from baseline to multiscale, and from single-scale or cross-scale to multiscale models of all signals with multiscale connections. P-values are computed by one-sided paired t-tests.

	PP increase ratio	monkey	mean	sem	p-value
Spike	From Baseline to Multiscale:	J	71.5%	37.3%	0.03
	(multiscale PP - baseline PP)/baseline PP	C	38.7%	3.4%	2e-19
	From Single-scale to Multiscale:	J	76.3%	5.0%	4e-31
	(multiscale PP - single-scale PP)/(multiscale PP - baseline PP)	C	58.3%	3.7%	4e-28
	From Cross-scale to Multiscale:	J	13.9%	6.0%	0.01
	(multiscale PP - cross-scale PP)/(multiscale PP - baseline PP)	C	26.2%	3.7%	1e-10
Field	From Baseline to Multiscale:	J	7.8%	1.0%	4e-8
	(multiscale PP - baseline PP)/baseline PP	C	6.2%	0.31%	2e-41
	From Single-scale to Multiscale:	J	2.7%	0.9%	3e-3
	(multiscale PP - single-scale PP)/(multiscale PP - baseline PP)	C	0.51%	0.16%	8e-4
	From Cross-scale to Multiscale:	J	91.6%	1.0%	7e-28
	(multiscale PP - cross-scale PP)/(multiscale PP - baseline PP)	C	97.6%	0.41%	5e-168

3.4. Latent firing rates predict field potentials betterthan binary spike trains

In addition to handling the case of large networks, one advantage of our multiscale causality identification framework is that it allows our multiscale models to use either the latent spike firing rates or the observed spike events as field potential predictors. This can then allow us to test the hypothesis that latent firing rates may be better predictors of field potentials as could be implied from biophysical models [63]. We thus used our framework to test this hypothesis within the NHP datasets. As a comparison, we trained an alternative multiscale model with the same length of history. Figure 4 shows the multiscale PP difference between two models. We find that using the latent firing rates predicts the field potentials better than using the observed binary spike trains (p-value= 8.5e-49 for monkey J and 0.003 for monkey C, one-sided paired t-test). This result indicates that the latent firing rates contain more information than binary spike trains about field potentials. This result is consistent with the prediction in biophysical studies suggesting firing rate can be an LFP proxy [63].

Figure 4.

Our method using latent firing rates predicts field potentials better than using binary spike trains. Panels (a) and (b) show the probability density function (PDF) of PP difference using firing rates vs. using spike trains for monkeys J and C, respectively. PP differences greater than 0 are shown in red and the rest are shown in blue. In most cases, PP differences are positive indicating that latent firing rates are better predictors of field potentials overall. The PPs are computed for all field potentials. The p-values are computed with one-sided paired t-test and indicate that PPs are larger when using latent firing rates compared to using binary spike trains as predictors.

3.5. Properties of causal connections

3.5.1. Density of Connections and correlations to distance.

We next examined the density of connections and whether there was a relationship between the physical distance from a given signal and the density of connections from that signal. To do so, we evenly cut the sorted distances into 50 groups. Density in each group is defined by the ratio between the number of identified connections and all possible connections in that group. The distance representing each group is defined as the average of lower and upper bound of the distances in the group. Figure 5 shows the results of the correlation analyses between the distance of causal connections and their density for (a) field-field, (b) spike-field, (c) field-spike, and (d) spike-spike connections.

Figure 5.

Causal connection density from a signal is negatively correlated to physical distance from that signal. Pearson’s correlation between the physical distance (mm) from a signal and the corresponding density of connections from that signal is shown for (a) field-field (b) spike-field (c) field-spike and (d) spike-spike connections in monkey J’s dataset where we have 3-D coordinates of electrodes. There is a significant negative correlation between distance and density of connections for all connection types.

We find that there was a significant negative correlation between distance and density for all types of connections (p-value=2e-20, 2e-4, 5e-6 and 2e-6, respectively, Pearson’s correlation test). This result suggests that the density of the directed functional connectivity represented by directed information is negatively correlated with physical distance for connections from field potentials and also from spike trains.

3.5.2. Strength of causality

We then examined the strength of causality between two signals. From section 3.5.1, the density of field-field connections is much larger than that of other kinds of connections. This may be due to field potentials being aggregate network-level signals that contain information from groups of neurons nearby. Here we examined the average strength of individual connections measured by directed information across 11 sessions shown in figure 6. The strength of field-spike connections was significantly larger than the strength of spike-spike connections (p-value = 0.008, paired one-sided t-test) but the strength of field-field connections was not significantly different from the strength of spike-field connections (p-value=0.46, two sided t-test). This latter result suggests that field-field causality improves the prediction power of field potentials more than spike-field causality (see section 3.3.2) mainly because of the larger density of field-field connections as the individual field-field and spike-field connection strengths are comparable.

Figure 6.

Average individual-connection strength of four types of causal connections measured by directed information (DI) with errorbar (mean ± sem) across sessions.

3.5.3. Causal connections within vs. across regions.

We also compared the density of causal connections across regions with that within regions. The electrode arrays were placed in different cortical regions via an MRI guided stereotax [69]. The regions for the two NHPs are specified in Section 2.3. Figure 7 shows the density of within-region and cross-region connections in each session. We can observe that the method identifies that there are more within-region connections than cross-region connections for each kind of connection (p-values: 9e-10, 0.1, 2e-9 and 1e-5, respectively; one-sided paired t-test).

Figure 7.

Within-region and cross-region density of four kinds of connections in each session.

4. DISCUSSION

Here we explore the causality structure in multiscale spike-field networks and demonstrate a multiscale spike-field causality graph identification method based on directed information in realistic biophysical simulations and in motor cortical spike-LFP data during motor behavior from two NHPs. We show that the method is data-efficient and can successfully identify the multiscale causal connections in biophysical simulations. Further, our method reveals that multiscale spike-field network causality exists in NHP spike-field motor cortical data during arm movements and can be identified; indeed, models that included the identified multiscale causal connections better predicted both spike trains and field potentials compared to models that just included either single-scale or cross-scale connections. Finally, compared to using the binary spike trains as predictors, our method better predicted the LFPs by uniquely allowing for latent firing rates to be used as LFP predictors. These results show how causality can be computed in multiscale binary-continuous data reflected in spike-field measurements and how multiscale causality identification improves the modeling of spike-field network activity.

Prior directed information measures of causality have been developed for a single scale of activity, e.g. spikes [53–56, 60]. Our method extends and demonstrates the directed information measure for estimation of multiscale causality in mixed binary-continuous spike-field signals. The multiscale directed information measure is a causal directional measure, and thus is distinct from prior non-causal measures to assess spike-field connectivity including spike-field coherence [18,37–40] or measures in encoding models that use the LFP features at a given time to predict the spikes at the same time [8,42] or to study correlations [41].

Our method is a model-based Granger-like causality method. There are some general limitations for these methods [56,74]. First, these methods depend on building statistical models of the phenomena, and thus depend on the accuracy of these models. Indeed, that is why as our main measure of how accurately causality is estimated in NHP data, we consider the model with the identified causality network, and find this model’s predictions for the measured spiking and LFP signals. As these signals are observations from the latent causality network, their better prediction can imply a better causality identification by our method. By comparing these predictions for models with the identified single-scale causality vs. models with the identified multiscale causality, we find that multiscale causality identification improves the prediction of both spike trains and LFP signals. Second, a fundamental challenge for assessing causality is that factors that are not measured in a given experiment cannot be considered or conditioned on in any causality method. Thus, the causality graph identified from the neural signals measured in an experiment provides a measure of network interactions within the measured network only and without conditioning on the unmeasured elements. Despite this challenge, model-based Granger-like causality is still a useful tool for analyzing functional connectivity statistically and could reveal insights about the neurophysiology as shown in prior work [47, 56], for example for activity during sleep or during visuomotor integration. Our work on modeling multiscale causality fills an important gap for such analyses because it allows us to model all observed signals in a given experiment for better causality assessment by including both field potentials and spike trains rather than just one or the other. As such the method enables including more factors/neural elements when assessing causality.

The method demonstrated here has several unique features. First, it allows multiscale causality to be assessed for large networks, which is needed for modern electrophysiology datasets that record from tens of electrodes. We demonstrated this capability by showing that the method can compute multiscale connections across a large number of spike and LFP signals in NHP data and in biophysical simulations. Second, our method allows the use of latent firing rates as field potential predictors. The use of latent firing rates was motivated by biophysical suggestions that firing rates may be better LFP predictors compared to binary spike events [63]. Our results confirmed this biophysical conclusion. We found that in our NHP data, the estimated history of firing rates predicted the LFPs better than the history of binary spike trains. Also, our new multiscale causality test procedure based on the Wald test was more accurate for a given training data length, and thus was data-efficient. This is key for assessing causality in high-dimensional data recorded across large-scale brain networks. Finally, our method generates an encoding model of spike-field network activity during behavior that is more predictive of neural data compared with models that only use single-scale causality.

It is also important to note that by explicitly modeling a dynamic behavior as a covariate in the spike model in Equation (7) and field model in Equation (12), the method can take into account non-stationarities that are due to a changing behavioral signal that modulates the neural signals. Having modeled the changing behavior, the method then assumes that spike and field potential models are stationary within the typical time-lengths of a dataset. If the data time-length is much longer, it may be that the models change for example due to plasticity. Future work can thus explore refitting the models intermittently or adaptively for more accurate causality assessment.

We found that the density of causal connections is negatively correlated with physical distance for connections in the NHP motor cortical data. This is consistent with a prior study [8] which has shown that the strength of non-causal correlations from LFP to spike trains drops over distance. It is also consistent with prior studies showing spike-LFP coherence drops over distance [29,75].

One future direction would be to explore whether the model selection procedure in this work may be further improved using regularization methods. Regularization methods like ridge or LASSO [76, 77] can help reduce over-fitting and can potentially be integrated into our method. However, because of the two-step model fitting procedure in our method and the fact that the distribution of regularized parameters within a regularization method are hard to find, it would be hard to statistically test the significance of causal connections with such a regularization procedure [78]. Nevertheless, a future direction could be to solve this challenge and assess whether regularization can further improve our method of multiscale causality identification.

The multiscale spike-field causality graph identification method demonstrated here can also be applied to other brain regions and behaviors to discover the multiscale causal interactions in spike-field networks that drive these behaviors. One can also use this method to explore the dependence of multiscale causality in a given brain region on behavioral and task context, or to study how the causal network topology may change as a result of adaptation, learning or plasticity [79–83]. This method can also inform the design of future closed-loop neural control systems such as brain stimulation systems [1, 3, 4, 84–98]. For example, a major question in developing deep brain stimulation (DBS) therapies for mental disorders such as treatment-resistant depression is which site within the brain should be stimulated [1,4,92]. This method can provide one approach to identify candidate sites by finding the brain site that has the strongest causal connection to the cortico-limbic regions involved in mental disorders. This method can thus help develop future closed-loop DBS systems for mental disorders that aim to causally modulate symptom-related neural activity using the decoded symptom as feedback [1,3,4,87,92–94,96]. More generally, this method can facilitate the investigation of causal directed interactions in large-scale brain networks, whose findings can guide the development of more effective neurotechnologies.

Appendix

Here we show an example of simulated data from networks shown in Figure 2a. As mentioned in Section 3.2, we use VERTEX [71] to generate the simulated data. The raw output from VERTEX includes spike timings and LFPs. The sampling rate is 1 kHz. Then, we generate the binary spike trains using 5-ms time bins. If there is a spike in a time bin, the corresponding value of the spike train is 1. If there is no spike in a time bin, the corresponding spike train value is 0. LFPs are downsampled to 100 Hz and standardized to zero mean and unit variance. Figure 8 shows 20 example spike trains and 10 example LFPs during a 10-second period.

Figure 8.

We show 20 example spike trains and 10 example LFPs observed from a network shown in Figure 2a during a 10-second period. Sampling rates are 100 Hz for LFPs and 200 Hz for spikes (i.e., 5-ms spike bins). The regions to which the LFP and spike train signals belong are indicated by A-D. These regions as well as the LFP and spike train numberings are the same as in Figure 2a.