Estimating a Dynamic State to Relate Neural Spiking Activity to Behavioral Signals during Cognitive Tasks

Abstract

An important question in neuroscience is understanding the relationship between high-dimensional electrophysiological data and complex, dynamic behavioral data. One general strategy to address this problem is to define a low-dimensional representation of essential cognitive features describing this relationship. Here we describe a general state-space method to model and fit a low-dimensional cognitive state process that allows us to relate behavioral outcomes of various tasks to simultaneously recorded neural activity across multiple brain areas. In particular, we apply this model to data recorded in the lateral prefrontal cortex (PFC) and caudate nucleus of non-human primates as they perform learning and adaptation in a rule-switching task. First, we define a model for a cognitive state process related to learning, and estimate the progression of this learning state through the experiments. Next, we formulate a point process generalized linear model to relate the spiking activity of each PFC and caudate neuron to the stimated learning state. Then, we compute the posterior densities of the cognitive state using a recursive Bayesian decoding algorithm. We demonstrate that accurate decoding of a learning state is possible with a simple point process model of population spiking. Our analyses also allow us to compare decoding accuracy across neural populations in the PFC and caudate nucleus.

I. INTRODUCTION

Demonstrating the existence of meaningful relationships between behavior and neural activity is essential to our understanding of the brain and has been a subject of intensive investigation in neuroscience. Behavioral and cognitive neuroscientists aim to understand how the brain uses neural activity to integrate sensory inputs, control movements, facilitate learning and memory, activate and express emotions, etc. Neuroengineers focus on how to decode and stimulate neural activity to assist, supplement or suppress behavior.

Investigations of these questions have led to new multi-faceted experimental design and have generated behavioral data with growing complexity. In addition, technological advances now allow for recording of large quantities of information from the brain at multiple spatial and temporal scales, includes multi-channel electrode arrays, EEG, MEG and fMRI. Access to this type of high-dimensional data, both behavioral and neural, has also presented a challenge for statistical data analysis and modeling: What is an adequate representation of the relation between features of the behavioral task and structures in the neural activity?

Traditionally, studies of neural coding focus on the relation between electrophyiological data and directly observable biological or behavioral signals. For example, place-field models describe spiking activity in hippocampus as a function of an animal’s position in its environment. Recently, there has been increasing interest in models relating neural activity to more general variables that influence multiple aspects of behavior and cognitive function. Understanding the structure of such cognitive variables may be essential to the study of multiple neural disease processes. For example, deficits in cognitive flexibility have been linked to autism, obsessive-compulsive disorders and schizophrenia. However, cognitive flexibility is only observable through its influence on behavior, and therefore difficult to link to neural activity directly. Other examples of cognitive states that may provide meaningful links between behavior and neural activity include features like attention, affective response tendency, and approach-avoidance level. Finally, such cognitive variables are often dynamic, leading to changing behavioral outcome to stimuli through time. An important statistical challenge is to understand neural representations of these cognitive state processes and to estimate their dynamics through time.

One approach that has been successful in linking neural data to dynamic, unobserved signals is state-space modeling [1]–[8]. For example, state-space modeling has been used to predict the movements of a rat from ensemble place-cell activity [9], [10]. However, such methods require the behavioral signal to be estimated to be low-dimensional and directly observable during a first, encoding step. This would not be possible for the abstract, cognitive state processes, described above.

Here we present one possible solution: a general state-space paradigm to model and fit a low-dimensional cognitive state process that allows us to relate outcomes of various behavioral tasks to simultaneously recorded neural activity across multiple brain areas. The paradigm consists of three steps. First, we estimate the dynamics of a cognitive state variable using previous knowledge of its influence on observed behavioral signals. Second, we construct models that use the estimated state and relevant covariates related to behavior to describe the statistical structure of neural activity. Last, we estimate the dynamic state again, this time using only the neural activity. We illustrate the proposed paradigm with an application to data recorded in the lateral prefrontal cortex (PFC) and caudate nucleus of non-human primates as they perform learning and adaptation in a rule-switching task.

II. COGNITIVE STATE-SPACE DECODING PARADIGM

In this section, we first construct a low-dimensional cognitive state process to relate behavior and neural activity. We then describe a general three-model paradigm to estimate the cognitive state in three steps, summarized in Fig. 1.

Fig. 1.

Schematic representation of the general three-model cognitive state-space paradigm (Lower) in comparison with the classical two-model state-space paradigm (Upper).

A. Model Framework

We model the underlying cognitive function (signal) during a task as a stochastic and dynamic process. The abstract state variable x_k evolves through time according to some dynamics:

$$x_k\mid x_{k-1}\sim f(x_k\mid x_{k-1}).$$ (1)

In most situations, if not all, we do not observe the cognitive state. In other words, x_k is “hidden”. Some examples of this unobserved abstract cognitive state are reward motivation level, susceptibility to fear, flexibility of learning, etc.

Instead, we obtain some behavioral data ${\overrightarrow{z}}_k$ related to the task, parameterized by θ_z:

$${\overrightarrow{z}}_k\mid x_k\sim g({\overrightarrow{z}}_k\mid x_k;{\theta }_z).$$ (2)

${\overrightarrow{z}}_k$ can be a vector with components from distinct distributions, both discrete and continuous. For example, ${\overrightarrow{z}}_k$ can includes a binary component of correct vs. incorrect choices and a continuous component of reaction times.

We also record, in addition to the behavioral data, some electrophysiological activity ${\overrightarrow{y}}_k$, parameterized by θ_y:

$${\overrightarrow{y}}_k\mid x_k\sim h({\overrightarrow{y}}_k\mid x_k;{\theta }_y).$$ (3)

Similarly, ${\overrightarrow{y}}_k$ can also include both discrete neural signals such as single unit spiking activity and continuous neural signals such as local field potentials.

B. Estimating State Dynamics From Behavior

Our paradigm for estimating the cognitive state x_k involves three steps. In the first step, we estimate the state dynamics, $p(x_k\mid {\overrightarrow{z}}_{1:k})$, using previous knowledge of the behavioral task structure $g({\overrightarrow{z}}_k\mid x_k;{\theta }_z)$ and some smoothing constraints f(x_k∣x_k–1) on the state process dynamics. Assuming the function g in (2) is known, we use this known structure of g to estimate the state dynamics during behavioral experimental tasks:

$$p(x_k\mid {\overrightarrow{z}}_{1:k})\propto p({\overrightarrow{z}}_k\mid x_k)\int p(x_k\mid x_{k-1})p(x_{k-1}\mid {\overrightarrow{z}}_{1:k-1}){\mathit{dx}}_{k-1}.$$ (4)

The integral on the right hand side of (4) is the one-step prediction density $p(x_k\mid {\overrightarrow{z}}_{1:k-1})$ defined by the Chapman-Kolmogorov equation. Here we have assumed that given the past state value, x_k–1, the distribution of the current state does not depend on the past behavior. The integral in (4) typically cannot be solved analytically, but multiple numerical and approximation methods are available to compute its value at each time point. One approach is to apply some kind of filtering algorithm such as Kalman filters, which compute the distribution of the state given parameter estimates $\widehat{\theta }$. Extensions of these methods, such as Expectation-Maximization (EM) and sequential Monte Carlo, simultaneously optimize the model parameter estimates $\widehat{\theta }$ and the unobserved cognitive state process.

C. Neural Encoding

Second, we characterize the relationship between the cognitive state and features of neural activity. In this encoding step, we use an estimate of x_k from the previous step to identify models for function $h({\overrightarrow{y}}_k\mid x_k;{\theta }_y)$ with unknown structures in (3).

An example of $h({\overrightarrow{y}}_k\mid x_k;{\theta }_y)$ can be a point process model with condition intensity function λ(x_k), when ${\overrightarrow{y}}_t$ is single unit neural spiking activity:

$$p({\overrightarrow{y}}_k\mid x_k)={[\lambda (x_k;{\theta }_y)\Delta ]}^{{\overrightarrow{y}}_k}\text{exp}[\lambda (x_k;{\theta }_y)\Delta ],$$ (5)

where λ(x_k;θ_y) can be estimated by parametric models of generalized linear model form.

Another example of $h({\overrightarrow{y}}_k\mid x_k;{\theta }_y)$ can be multivariate Gamma models with mean μ(x_k;θ_y), when ${\overrightarrow{y}}_t$ is vector of power estimates in local field potential at specific frequencies.

In any case, we treat the estimated state process from step 1 as known, with some uncertainty, and use model fitting methods, such as maximum likelihood to estimate the unknown model parameters.

D. Neural Decoding

Third, we estimate the dynamic state x_k from a new dataset that includes both neural and behavioral activity. More specifically, in this “decoding” step, we compute the posterior distribution of the state process conditioned on the observed neural activity up until the current time:

$$\begin{array}{cc}\hfill p(x_k\mid {\overrightarrow{y}}_k,{\overrightarrow{z}}_k)& \propto p({\overrightarrow{y}}_k,{\overrightarrow{z}}_k\mid x_k)\hfill \\ \hfill & \int p(x_k\mid x_{k-1})p(x_{k-1}\mid {\overrightarrow{y}}_{1:k-1},{\overrightarrow{z}}_{1:k-1}){\mathit{dx}}_{k-1}.\hfill \end{array}$$ (6)

If we assume that conditioning on the state, behavior and neural activity are independent, then

$$p({\overrightarrow{y}}_k,{\overrightarrow{z}}_k\mid x_k)=p({\overrightarrow{y}}_k\mid x_k)p({\overrightarrow{z}}_k\mid x_k).$$ (7)

If we choose to decode x_k during the structured behavioral task, then we use both known g and identified models for h to estimate x_k. If we choose to decode x_k outside of the structured task, we use identified h to estimate x_k.

III. APPLICATION: DECODING LEARNING STATE FROM SPIKING ACTIVITY IN MONKEY PFC AND CAUDATE

A. Experimental Data

We illustrate the application of the proposed paradigm with an example study. The behavioral and neural data are obtained from two monkeys performing a temporally delayed, on-line learning task in which they had to determine by trial-and-error which of four picture cues or spatial locations was currently rewarded within a learning block. Individual blocks followed either a “spatial” or an “object” rule. In the “spatial” rule, the animal was required to choose the target in the same location on every trial (e.g., always upper right). In the “object” rule, the correct action was to choose a picture that matched a picture cue (e.g., always a blue sailboat). The “spatial” rule is substantially easier and rewards perseverative behavior, while the “object” rule rewards flexibility. Details of the behavioral paradigm, data acquisition, and previous analyses of this experimental data are discussed in [11].

In this specific example, the behavioral data is whether the monkey chose the correct location on each trial. The neural data is spiking activity recorded in the lateral prefrontal cortex (PFC) and caudate nucleus of the monkeys. The cognitive state is whether the subject has learned the rule of the task.

B. Estimate State Dynamics From Behavior

We take advantage of previous development of a dynamic approach to analyzing learning experiments with binary responses [11]–[15]. We use a state-space model of learning in which a Bernoulli probability model describes behavioral task responses and a Gaussian state equation describes the hidden state process.

In other words, $p({\overrightarrow{z}}_k\mid x_k)$ in (4) is expressed as the Bernoulli probability mass function:

$$p({\overrightarrow{z}}_k\mid x_k)=q_k^{{\overrightarrow{z}}_k}{(1-q_k)}^{1-{\overrightarrow{z}}_k},$$ (8)

where q_k is defined by the logistic equation:

$$q_k=\frac{\text{exp}(\mu +x_k)}{1+\text{exp}(\mu +x_k)},$$ (9)

and μ is determined by the probability of a correct response by chance in the absence of learning or experience. Here x_k defines the learning state of the animal at trial k in the experiment. The unobservable state process x_k∣x_k 1 is defined as a random walk:

$$x_k=x_{k-1}+{\epsilon }_k,$$ (10)

where the ε_k are independent Gaussian random variables with mean 0 and variance ${\sigma }_{\epsilon }^2$. The one-step prediction density $p(x_k\mid {\overrightarrow{z}}_{1:k-1})$, or learning curve, is the probability of a correct response as a function of the state process and is calculated using an EM algorithm:

$$\begin{array}{cc}\hfill & f(q\mid \mu ,x_{k\mid k},{\sigma }_{k\mid k}^2)={[{\left(2\pi {\sigma }_{k\mid k}^2\right)}^{1∕2}q(1-q)]}^{-1}\hfill \\ \hfill & \text{exp}\left(-\frac{1}{2{\sigma }_{k\mid k}^2}{\left[\log \right[q{\left[\right(1-q\left)\text{exp}\right(\mu \left)\right]}^{-1}]-x_{k\mid k}]}^2\right).\hfill \end{array}$$ (11)

Detailed estimation methods are referred to in [12].

Fig. 2 shows two examples of the learning curves estimated by the EM algorithm in two learning blocks in the rule-switching behavioral task. The correct and incorrect responses are shown, respectively, by black and gray marks above the panels. Neglecting the possibility of behavioral preferences or other biases, the probability of a response occurring by chance is shown as a horizontal line at 0.25. Solid blue lines are the learning curve estimates, and the dotted blue lines are the associated 95% confidence intervals. The lower confidence bounds for the learning trial estimates remained above 0.25 after trial 14 and 63, which are, respectively the learning trials for the two learning block examples shown here.

Fig. 2.

Two examples of the EM algorithm applied in the analysis of the state dynamics in a rule-switching task. The correct and incorrect responses are shown, respectively, by black and gray marks above the panels. The probability of a correct response occurring by chance is 0.25 in horizontal line. Solid blue lines are the learning curve estimates, and the dotted blue lines are the associated 95% confidence intervals.

For simplicity, we further dichotomize the trials within each learning block to be “learned” trials if the lower bound of the learning state estimate remains above 0.25 for the remainder fo the trial block or “not learned” trials if otherwise.

C. Neural Encoding

Because the neural data in our example is spiking activity, we present a point process generalized linear model (GLM) approach [3], [16] for constructing a conditional intensity function that characterizes the spiking activity of PFC and caudate neurons. The conditional intensity function [17], [18] relates spiking probability simultaneously to the temporal features of the behavioral task.

In this case, the conditional intensity model is defined as follows:

$$\log {\lambda }^c(t)=\sum _{j=1}^2\sum _{i=1}^N{\alpha }_{i,j}^c{B}_{i,j}^c(t).$$ (12)

Here c = 1, ⋯ C is the index of the neuron. j = 1,2 is the binary indicator of the behavioral outcome of the trial, where j = 1 and j = 2 are “learned” and “not learned” states, respectively. $B_{i,j}^c\left(t\right)$ is a basis function for a cardinal spline for neuron c, trial type j. Cardinal splines are locally defined third-order polynomial functions that flexibly approximate arbitrary smooth functions using a small number of basis functions [19]. Here, we use spline functions to capture the firing probability as a function of time relative to the picture cue. N is the number of spling control points used to fit the data. Here we chose N = 16 control points. $\theta =\left[{\left\{{\alpha }_{i,j}^c\right\}}_{i=1}^N\right]$ is a set of unknown parameters which relate the temporal features of the behavioral task to instantaneous spike rate.

It follows from the definition of the conditional intensity function that the probability of a spike from neuron c in a small time interval [t, t + Δ) is approximately:

$$\text{Pr}\left(\text{Spike from neuron}c\text{in}\right[t,t+\Delta )\mid \theta )\approx {\lambda }^c(t\mid \theta )\Delta .$$ (13)

This spiking intensity function describes a GLM for the spike train data. Such GLMs have a number of nice properties, including convexity of the likelihood surface and asymptotic normality of the parameter estimates, which allow us to compute maximum likelihood estimates for the model parameters in a straightforward manner. We fit these GLMs using the estimated learning state from the behavioral data. We examine the model fits to the data from 500 ms before picture cue to 2500 ms after picture cue.

Fig. 3 shows the model parameters and their uncertainty for the maximum likelihood fit to four example neurons in this spiking data. Each panel shows the spline estimates, in solid lines, and 95% confidence bounds in dashed lines, as a function of time relative to picture cue, represented by the vertical line in cyan. The times of go cue, feedback, and start of inter-trial interval are identified as vertical lines in green, yellow and black, respectively. The estimated intensity and 95% confidence bounds for the learned state and not-learned state are plotted in red and blue, respectively.

Fig. 3.

Four examples of the estimated conditional intensity with 95% confidence bounds as a function of the temporal features in the task. In each panel, blue and red solid line is the estimated conditional intensity for the “not learned” and “learned” trials, respectively. The vertical lines in cyan, green, yellow and black are the time for picture cue, go cue, feedback, and start of the inter-trial interval, respectively.

Top two and the lower-left panels in Fig. 3 show the model fit for three neurons in the caudate nucleus. For the neuron plotted in the top-left, at around 1000 ms after the picture cue and right before the go cue, the estimated intensity for learned trials in red is significantly higher than the estimated intensity for not-learned trials in blue. For the neuron plotted in the top-right, at around 2000 ms after the picture cue and within a 300 ms lag of the feedback, the estimated intensity for not-learned trials is significantly higher than that of the learned trials. For the neuron plotted in the lower-left, at around 800 ms after the picture cue, the estimated intensity for not-learned trials is significantly higher than that of the learned trials.

The lower-right panel in Fig. 3 shows the model fit for a neuron in the PFC. The 95% confidence bounds for the learned and not-learned trials are always overlapping, which means that the temporal spiking properties during the observation interval are not significantly different for the two learning states.

To assess the goodness-of-fit of the two-state model, we constructed Kolmogorov-Smirnov (KS) plots of time-rescaled inter-spike intervals (ISIs) [20]. The time-rescaling theorem produces a set of rescaled ISIs that are independent with an exponential distribution with mean 1 if the proposed model accurately describes the structure in the observed spiking activity. To construct the KS plot, we plot the empirical cumulative distribution of the rescaled ISIs against the theoretical cumulative distribution of the Exponential(1) distribution. The better the quality of the model fit, the closer the K-S plot should be to a 45 degree line [21]. The left panel in Fig. 4 shows an example K-S plot for the model fit. The model and empirical CDFs demonstrate a good overall fit, with some evidence of misfit in the smaller rescaled ISIs. This may suggest some model misfit related to our assumptions of a dichotomized cognitive state and a lack of spiking history dependence structure in the point process model. More accurate modeling would likely lead to improvements in the oeverall goodness-of-fit and the resulting decoded state estimates.

Fig. 4.

Left: An example Kolmogorov-Smirnov (KS) plot of time-rescaled inter-spike intervals for the estimated point process neural encoding model. It plots the empirical cumulative distribution of the rescaled ISIs against the theoretical cumulative distribution of the Exponential(1) distribution. Right: An example receiver-operating characteristic (ROC) curve. It plots sensitivity of the cut-off, the probability of rejecting the null hypothesis when it is false, versus significance level, the probability of rejecting a null hypothesis when it is true. ROC curves using decoding results from the neural ensemble in PFC and caudate are plotted in solid blue lines and dashed red line, respectively.

D. Neural Decoding

The previous subsection focused on the construction of neural spiking models, which uses relevant covariates related to behavior to describe the statistical structure of neural spiking activity. In this subsection we present a simple recursive Bayesian algorithm to decode the dynamic state from the spiking activity. For each trial k, we compute the posterior distribution of the monkey’s learning state given the combined spiking activity of the neural ensemble within the observation interval, [0,T]:

$$p({\overrightarrow{x}}_k\mid \Delta N_{1:T})\propto p(\Delta N_{1:T}\mid {\overrightarrow{x}}_k)p\left({\overrightarrow{x}}_k\right).$$ (14)

Here $p\left({\overrightarrow{x}}_k\right)$ is the prior distribution of the state. In this case, we choose a uniform prior for the binary state. The observation model, or likelihood, is given by

$$p(\Delta N_{1:T}\mid {\overrightarrow{x}}_k)\propto \prod _{t=1}^T\prod _{c=1}^C{[{\lambda }^c({\overrightarrow{x}}_k)\Delta ]}^{\Delta N_t}\text{exp}(-{\lambda }^c({\overrightarrow{x}}_k)\Delta ).$$ (15)

We perform a classification procedure by thresholding the posterior of the state at various cut-off probabilities to determine whether the monkey is in a “learned” or a “not learned” state during a particular trial. The right panel in Fig. 4 shows the receiver-operating characteristic (ROC) curve [22], [23]. It plots the sensitivity of the cut-off, the probability of rejecting the null hypothesis when it is false, versus significance level, the probability of rejecting a null hypothesis when it is true. ROC curves using decoding results from the neural ensemble in PFC and caudate are plotted as a solid blue line and a dashed red line, respectively. The ROC curve based on spiking activity in the caudate is consistently above the ROC curve based on spiking activity in PFC. It shows that neural activity in the caudate provides a better decoding of the learning state than PFC, which corroborates previous findings in the literature that the caudate contributes more closely to learning [24]–[27].

IV. DISCUSSION

The classical two-model state-space paradigm has been successfully applied to relate behavior and neural activity directly in low-dimensional, directly observable data. However, when both the behavioral and neural data become high-dimensional and multi-faceted, this direct approach becomes computationally challenging. Here we proposed a new three-model paradigm to characterize the relationship between behavior and the neural activity. We first introduced a cognitive state process whose dynamics can be estimated from behavior. We then used the state and relevant covariates related to behavior to describe the neural activity. Lastly, we estimated the dynamic state from a combination of behavioral and neural data.s

We illustrated our paradigm with a specific example of two monkeys performing a temporally delayed, on-line learning task. We demonstrated that accurate decoding of the learning state is possible with a simple point process model of population spiking. Our analyses also allowed us to compare decoding accuracy across neural population in the PFC and caudate nucleus.

Immediate extensions to the application of the paradigm shown here are under active development. First, more accurate statistical descriptions of the behavioral data hopefully will lead to a more accurately estimated learning curve. Second, instead of working with the simplified, dichotomized learning state process, the neural encoding and decoding steps can deal directly with a continuous state process. Last, to improve the quality of fit, the point process models used for neural encoding can be expanded to include spiking history.

The essential goal of the proposed paradigm is to demonstrate the existence of meaningful relationships between complex behavior and high-dimensional neural activity. We achieve dimensionality-reduction by using hidden cognitive state processes to represent the relationship. In principle, identifying low-dimensional states that provide meaningful links between behavior and neural data is a major challenge that is specific to particular neural processing tasks. In many cases, neuroscientists may have prior conceptions of cognitive features that could represent such meaningful links. Future work may allow us to develop principled methodologies to identify cognitive states directly from data. Furthermore, by assigning some cognitive meaning to the hidden state, we can design experiments to determine the effect of manipulations of neural activity on cognitive influences of behavior. For the specific example shown here, the cognitive state can be thought of as a measure of learning flexibility, and we can modulate it to facilitate learning in the monkeys.

We envision the proposed paradigm to play a future role in the development of new types of closed-loop experiments, aiming to characterize causal relationships between neural activity and the behavior they encode. The proposed algorithm can allow investigators to identify and manipulate a low-dimensional correlate of cognitive influence in a content-specific way, altering neural activity related to certain cognitive features to modulate behavior. This may be an important step in treating mental diseases such as post-traumatic stress disorder and obsessive-compulsive disorders clinically.