ASSESSMENT OF SYNCHRONY IN MULTIPLE NEURAL SPIKE TRAINS USING LOGLINEAR POINT PROCESS MODELS

Abstract

Neural spike trains, which are sequences of very brief jumps in voltage across the cell membrane, were one of the motivating applications for the development of point process methodology. Early work required the assumption of stationarity, but contemporary experiments often use time-varying stimuli and produce time-varying neural responses. More recently, many statistical methods have been developed for nonstationary neural point process data. There has also been much interest in identifying synchrony, meaning events across two or more neurons that are nearly simultaneous at the time scale of the recordings. A natural statistical approach is to discretize time, using short time bins, and to introduce loglinear models for dependency among neurons, but previous use of loglinear modeling technology has assumed stationarity. We introduce a succinct yet powerful class of time-varying loglinear models by (a) allowing individual-neuron effects (main effects) to involve time-varying intensities; (b) also allowing the individual-neuron effects to involve autocovariation effects (history effects) due to past spiking, (c) assuming excess synchrony effects (interaction effects) do not depend on history, and (d) assuming all effects vary smoothly across time. Using data from primary visual cortex of an anesthetized monkey we give two examples in which the rate of synchronous spiking can not be explained by stimulus-related changes in individual-neuron effects. In one example, the excess synchrony disappears when slow-wave “up” states are taken into account as history effects, while in the second example it does not. Standard point process theory explicitly rules out synchronous events. To justify our use of continuous-time methodology we introduce a framework that incorporates synchronous events and provides continuous-time loglinear point process approximations to discrete-time loglinear models.

1 Introduction

One of the most important techniques in learning about the functioning of the brain has involved examining neuronal activity in laboratory animals under varying experimental conditions. Neural information is represented and communicated through series of action potentials, or spike trains, and the central scientific issue in many studies concerns the physiological significance that should be attached to a particular neuron firing/spiking pattern in a particular part of the brain. In addition, a major relatively new effort in neurophysiology involves the use of multielectrode recording, in which responses from dozens of neurons are recorded simultaneously. Much current research focuses on the information that may be contained in the interactions among neurons. Of particular interest are spiking events that occur across neurons in close temporal proximity, within or near the typical one millisecond accuracy of the recording devices. In this paper we provide a point process framework for analyzing such nearly synchronous events.

The use of point processes to describe and analyze spike train data has been one of the major contributions of statistics to neuroscience. On the one hand, the observation that individual point processes may be considered, approximately, to be binary time series allows methods associated with generalized linear models to be applied [cf. Brillinger (1988), (1992)]. On the other hand, basic point process methodology coming from the continuous-time representation is important both conceptually and in deriving data-analytic techniques [e.g., the time-rescaling theorem may be used for goodness-of-fit and efficient spike train simulation; see Brown, et al. (2001)]. The ability to go back and forth between continuous time, where neuroscience and statistical theory reside, and discrete time, where measurements are made and data are analyzed, is central to statistical analysis of spike trains. From the discrete-time perspective, when multiple spike trains are considered simultaneously it becomes natural to introduce loglinear models [cf. Martignon, et al. (2000)] and a widely read and hotly debated report by Schneidman, et al. (2006) examined the extent to which pairwise dependence among neurons can capture stimulus-related information. A fundamental limitation of much of the work in this direction, however, is its reliance on stationarity. The main purpose of the framework described below is to handle the non-stationarity inherent in stimulus-response experiments by introducing appropriate loglinear models while also allowing passage to a continuous-time limit. The methods laid out here are in the spirit of Ventura, Cai, and Kass (2005), who proposed a bootstrap test of time-varying synchrony, but our methods are different in detail and our framework is much more general.

Statistical modeling of point process data focuses on intensity functions, which represent the rate at which the events occur, and often involve covariates [cf. Brown, et al. (2003); Kass, Ventura and Brown (2005); Paninski, et al. (2009); and references therein]. A basic distinction is that of conditional versus marginal intensities: the conditional intensity determines the event rate for a given realization of the process, while the marginal intensity is the expectation of the conditional intensity across realizations. In neurophysiological experiments stimuli are often presented repeatedly across many trials, resulting in many replications of the multiple sequences of spike trains. This is the situation we concern ourselves with here, and it is illustrated in Figure 1, part A, where the responses of a single neuron for 120 trials are displayed: each line of the raster plot shows a single spike train, which is the neural response on a single trial. The experiment that generated these data is described in Section 1.1. The bottom panel in part A of Figure 1 displays a smoothed peristimulus time histogram (PSTH), which summarizes the trial-averaged response by pooling across trials. As we explain in greater detail in Section 1.2, scientific questions and statistical analyses may concern either within-trial responses (conditional intensities) or trial-averaged responses (marginal intensities).

Figure 1.

Neural spike train raster plots for repeated presentations of a drifting sine wave grating stimulus. (A): Single cell responses to 120 repeats of a 10 second movie. At the top is a raster corresponding to the spike times, and below is a peri-stimulus time histogram (PSTH) for the same data. Portions of the stimulus eliciting firing are apparent. (B): The same plots as in (A), for a different cell. (C): Population responses to the same stimulus, for 5 repeats. Each block, corresponding to a single trial, is the population raster for ν = 128 units. On each trial there are several dark bands, which constitute bursts of network activity sometimes called “up states.” Up state epochs vary across trials, indicating they are not locked to the stimulus.

A point process evolves in continuous time but, as we have noted, it is convenient for many statistical purposes to consider a discretized version. Decomposing time into bins of width δ, we may define a binary time series to be 1 for every time bin in which an event occurs, and 0 for every bin in which an event does not occur. It is not hard to show that, under the usual regularity condition that events occur discretely (i.e. no two events occur at the same time), the likelihood function of the binary time series approximates the likelihood of the point process as δ → 0. For a pair of point processes, the discretized process is a time series of 2×2 polytomous variables indicating, in each time bin, whether an event of the first process occurred, an event of the second process occurred, or both, or neither. This suggests analyzing nearly synchronous events based on a loglinear model with cell probabilities that vary across time. Intuitive as such procedures may be, their point process justification is subtle: the standard regularity condition forbids two processes having synchronous events, so it is not obvious how we might obtain convergence to a point process (as δ → 0) for discrete-process likelihoods that incorporate synchrony.

One way out of this impasse is to introduce a marked point process framework in which each event/mark could be of three distinct types: first process, second process, or both. The standard marked point process requires modification, however, because it fails to accommodate independence as a special case. Under independence, the discretized events for each process occur with probability of order O(δ) while the synchronous events occur with probability of order O(δ²) as δ → 0. We refer to this as a sparsity condition, and the generalization to multiple processes involves a hierarchical sparsity condition. Once we introduce a family of marked point processes indexed by δ, we can guarantee hierarchical sparsity. Not only does this allow, as it must, the special case of independence models, but it also makes the conditional intensity for neuron i depend only on the history for neuron i, asymptotically (as δ → 0). This in turn avoids confounding the dependence described by the loglinear model and greatly reduces the dimensionality of the problem. We require two very natural regularity conditions based on well-known neurophysiology: the existence of a refractory period, during which the neuron cannot spike again, and smoothness of the conditional intensity across time. It would be possible, and sometimes advantageous, instead to model dependence through the individual-neuron conditional intensity functions. The loglinear modeling approach used here avoids this step.

1.1 A motivating example

In a series of experiments performed by one of us (Kelly, together with Dr. Matthew Smith), visual images were displayed at resolution 1024 × 768 pixels on a computer monitor while the neural responses in primary visual cortex of an anesthetized monkey were recorded. Each of 98 distinct images consisted of a sinusoidal grating that drifted in a particular direction for 300 milliseconds, and each was repeated 120 times. Each repetition of the complete sequence of stimuli lasted approximately 30 seconds. This kind of stimulus has been known to drive cells in primary visual cortex since the Nobel prize-winning work of Hubel and Wiesel in the 1960s. With improved technology and advanced analytical strategies, much more precise descriptions of neural response are now possible. A small portion of the data from 5 repetitions of many stimuli is shown in part C of Figure 1.

The details of the experiment and recording technique are reported in Kelly, et al. (2007). A total of 125 neural “units” were obtained, which included about 60 well-isolated individual neurons; the remainder were of undetermined origin (some mix of 1 or more neurons). The goal was to discover the interactions among these units in response to the stimuli. Each neuron will have its own consistent pattern of responses to stimuli, as illustrated in parts A and B of Figure 1. Synchronous spiking across neurons is relatively rare. However, in each of the 5 blocks within part C of Figure 1 (each block corresponding to a single trial) several dark bands of activity across most neurons may be seen during the trial. These bands correspond to what are often called network “up” states, and are often seen under anesthesia. For discussion and references see Kelly, et al. (2009). It would be of interest to separate the effects of such network activity from other synchronous activity, especially stimulus-related synchronous activity. The framework in this paper provides a foundation for statistical methods that can solve such problems.

1.2 Overview of approach

We begin with some notation. Suppose we observe the activity of an ensemble of ν neurons labeled 1 to ν over a time interval [0, T), where T > 0 is a constant. Let $N_T^i$ denote the total number of spikes produced by neuron i on [0, T) where i = 1, …, ν. The resulting (stochastic) sequence of spike times is written as $0\le s_1^i<\cdots <s_{N_T^i}^i<T$. For the moment we focus on the case ν = 3, but other values of ν are of interest and with contemporary recording technology ν ≈ 100 is not uncommon, as in the experiment in Section 1.1. Let δ > 0 be a constant such that T is a multiple of δ (for simplicity). We divide the time interval into bins of width δ. Define Xⁱ(t) = 1 if neuron i has a spike in the time bin [t, t + δ) and 0 otherwise. Because of the existence of a refractory period for each neuron, there can be at most 1 spike in [t, t + δ) from the same neuron if δ is sufficiently small. Then writing

$$P_{a,b,c}^{1,2,3}(t)=P(X^1(t)=a,X^2(t)=b,X^3(t)=c),\forall a,b,c\in \{0,1\},$$

the data would involve spike counts across trials [e.g., the number of trials on which (X¹(t), X²(t), X³(t)) = (1, 1, 1)]. The obvious statistical tool for analyzing spiking dependence is loglinear modeling and associated methodology.

Three complications make the problem challenging, at least in principle. First, there is nonstationarity: the probabilities vary across time. The data thus form a sequence of 2^ν contingency tables. Second, absent from the above notation is a possible dependence on spiking history. Such dependence is the rule rather than the exception. Let ${\overline{\mathscr{H}}}_t^i$ denote the set of values of Xⁱ(s), where s < t, and s, t are multiples of δ. Thus ${\overline{\mathscr{H}}}_t=({\overline{\mathscr{H}}}_t^1,\dots ,{\overline{\mathscr{H}}}_t^{\nu })$ is the history of the binned spike train up to time t. We may wish to consider conditional probabilities such as

$$P_{a,b,c}^{1,2,3}(t|{\overline{\mathscr{H}}}_t)=P(X^1(t)=a,X^2(t)=b,X^3(t)=c|{\overline{\mathscr{H}}}_t)$$

for a, b, c ∈ {0, 1}. Third, there is the possibility of precisely timed lagged dependence (or time-delayed synchrony): e.g., we may want to consider the probability

$$P_{1,1,1}^{1,2,3}(s,t,u)=P(X^1(s)=1,X^2(t)=1,X^3(u)=1),$$ (1)

where s, t, u may be distinct. Similarly, we might consider the conditional probability

$$P_{1,1,1}^{1,2,3}(s,t,u|{\overline{\mathscr{H}}}_s^1,{\overline{\mathscr{H}}}_t^2,{\overline{\mathscr{H}}}_u^3)=P(X^1(s)=1,X^2(t)=1,X^3(u)=1|{\overline{\mathscr{H}}}_s^1,{\overline{\mathscr{H}}}_t^2,{\overline{\mathscr{H}}}_u^3).$$

In principle we would want to consider all possible combinations of lags. Even for ν = 3 neurons, but especially as we contemplate ν ≫ 3, strong restrictions must be imposed in order to have any hope of estimating all these probabilities from relatively sparse data in a small number of repeated trials. To reduce model dimensionality we suggest four seemingly reasonable tactics: (i) considering models with only low-order interactions, (ii) assuming the probabilities $P_{a,b,c}^{1,2,3}(t)\text{ or }{P}_{a,b,c}^{1,2,3}(t|{\overline{\mathscr{H}}}_t)$ vary smoothly across time t, (iii) restricting history effects to those that modify a neuron’s spiking behavior based on its own past spiking, and then (iv) applying analogues to standard loglinear model methodology. Combining these, we obtain tractable models for multiple binary time series to which standard methodology, such as maximum likelihood and smoothing, may be applied. In modeling synchronous spiking events as loglinear time series, however, it would be highly desirable to have a continuous-time representation, where binning becomes an acknowledged approximation. We therefore also provide a theoretical point process foundation for the discrete multivariate methods proposed here.

It is important to distinguish the probabilities $P_{a,b,c}^{1,2,3}(t)\text{ and }{P}_{a,b,c}^{1,2,3}(t|{\overline{\mathscr{H}}}_t)$. The former are trial-averaged or marginal probabilities, while the latter are within-trial or conditional probabilities. Both might be of interest but they quantify different things. As an extreme example suppose, as sometimes is observed, each of two neurons has highly rhythmic spiking at an approximately constant phase relationship with an oscillatory potential produced by some large network of cells. Marginally these neurons will show strongly dependent spiking. On the other hand, after taking account of the oscillatory rhythm by conditioning on each neuron’s spiking history and/or a suitable within-trial time-varying covariate, that dependence may vanish. Such a finding would be informative, as it would clearly indicate the nature of the dependence between the neurons. In Section 3 we give a less dramatic but similar example taken from the data described in Section 1.1.

We treat marginal and conditional analyses separately. Our use of two distinct frameworks is a consequence of the way time resolution will affect continuous-time approximations. We might begin by imagining the situation in which event times could be determined with infinite precision. In this case it is natural to assume, as is common in the point process literature, that no two processes have simultaneous events. As we indicate, this conception may be applied to marginal analysis. However, the event times are necessarily recorded to fixed accuracy, which becomes the minimal value of δ, and δ may be sufficiently large that simultaneous events become a practical possibility. Many recording devices, for example, store neural spike event times with an accuracy of 1 millisecond. Furthermore, the time scale of physiological synchrony—the proximity of spike events thought to be physiologically meaningful—is often considered to be on the order of δ = 5 milliseconds [cf. Grün, Diesmann and Aertsen (2002a), (2002b) and Grün (2009)]. For within-trial analyses of synchrony, the theoretical conception of simultaneous (or synchronous) spikes across multiple trials therefore becomes important and leads us to the formalism detailed below. The framework we consider here provides one way of capturing the notion that events within δ milliseconds of each other are essentially synchronous.

The rest of this article is organized as follows. Section 2 presents the methodology in three subsections: Sections 2.1 and 2.2 introduce marginal and conditional methods in the simplest case while Section 2.3 discusses the use of loglinear models and associated methodology for analyzing spiking dependence. In Section 3 we illustrate the methodology by returning to the example of Section 1.1. The main purpose of our approach is to allow covariates to take account of such things as the irregular network rhythm displayed in Figure 1, so that synchrony can be understood as either related to the network effects or unrelated. Figure 2 displays synchronous spiking events for two different pairs of neurons, together with accompanying fits from continuous-time loglinear models. For both pairs the independence model fails to account for synchronous spiking. However, for one pair the apparent excess synchrony disappears when history effects are included in the loglinear model, while in the other pair they do not, leading to the conclusion that in the second case the excess synchrony must have some other source. Theory is presented in Sections 4–6. We add some discussion in Section 7. All proofs in this article are deferred to the Appendix.

Figure 2.

Synchronous spike analysis for two pairs of neurons. Results for one pair shown on left, in parts (A)–(D) and for the other pair on the right in parts (E)–(H). Part (A): Response of a cell to repetitions of a 1 second drifting grating stimulus. The raster plot is shown above and the smoothed PSTH below. Part (B): Response from a second cell, as in (A). In both (A) and (B), spikes that are synchronous between the pair are circled. Part (C): Correct joint spike predictions from model, shown as circles (as in parts (A) and (B)), when false positive rate is set at 10%. In top plot the joint spikes are from the history-independent model, as in (13), while in the bottom plot they are as in (16), including the network covariate in the history term. Part (D): ROC curves for the models in part (C). Parts (E), (F), (G), and (H) are similar to Parts (A), (B), (C), and (D) but for the second pair of neurons.

2 Methodology

In this section we present our continuous-time loglinear modeling methodology. We begin with the simplest case of ν = 2 neurons, presenting the main ideas in Sections 2.1 and 2.2 for the marginal and conditional cases, respectively, in terms of the probabilities $P_a^1(t)=P(X^1(t)=a),P_{a,b}^{1,2}(t)=P(X^1(t)=a,X^2(t)=b)$, etc., for all a, b ∈ {0, 1}. We show how we wish to pass to the continuous-time limit, thereby introducing point process technology and making sense of continuous-time smoothing, which is an essential feature of our approach. In Section 2.3 we reformulate using loglinear models, and then give continuous-time loglinear models for ν = 3. Our analyses in Section 3 are confined to ν = 2 and ν = 3 because of the paucity of higher-order synchronous spikes in our data. Our explicit models for ν = 3 should make clear how higher-order models are created. We give general recursive formulas in Sections 5 and 6.

2.1 Marginal methods for ν = 2

The null hypothesis

$$H_0:P_{1,1}^{1,2}(t)=P_1^1(t)P_1^2(t),\forall t\in 𝒯=\{0,\delta ,2\delta ,\dots ,T-\delta \},$$ (2)

is a statement that both neurons spike in the interval [t, t + δ), on the average, at the rate determined by independence. Defining ζ(t) by

$$P_{1,1}^{1,2}(t)=P_1^1(t)P_1^2(t)\zeta (t),\forall t\in 𝒯,$$ (3)

we may rewrite (2) as

$$H_0:\zeta (t)=1,\forall t\in 𝒯.$$ (4)

As in Ventura, et al. (2005) to assess H₀ the general strategy we follow is to (i) smooth the observed-frequency estimates of $P_1^1(t),P_1^2(t)\text{ and }{P}_{1,1}^{1,2}(t)$ across time t, and then (ii) form a suitable test statistic and compute a p-value using a bootstrap procedure. We may deal with time-lagged hypotheses similarly, e.g., for a lag h > 0, we write

$$P_{1,1}^{1,2}(t,t+\delta h)=P(X^1(t)=1,X^2(t+\delta h)=1)=P_1^1(t)P_1^2(t+\delta h)\zeta (t,t+\delta h),$$ (5)

then smooth the observed-frequency estimates for $P_{1,1}^{1,2}(t,t+\delta h)$ as a function of t, form an analogous test statistic and find a p-value.

To formalize this approach we consider counting processes $N_t^i$ corresponding to the point processes $s_1^i,s_2^i,\dots ,s_{N_t^i}^i$, i = 1, 2 (as in Section 1.2 with ν = 2). Under regularity conditions, the following limits exist:

$$\begin{array}{cc}\hfill {\lambda }^i(t)& =\underset{\delta \to 0}{\text{lim}}{\delta }^{-1}P(N_{t+\delta }^i-N_t^i=1),\hfill \\ \hfill {\lambda }^{1,2}(t)& =\underset{\delta \to 0}{\text{lim}}{\delta }^{-2}P((N_{t+\delta }^1-N_t^1)(N_{t+\delta }^2-N_t^2)=1),\hfill \\ \hfill \xi (t)& =\underset{\delta \to 0}{\text{lim}}\zeta (t).\hfill \end{array}$$ (6)

Consequently, for small δ, we have

$$P_1^i(t)\approx {\lambda }^i(t)\delta ,P_{1,1}^{1,2}(t)\approx {\lambda }^{1,2}(t){\delta }^2.$$

The smoothing of the observed-frequency estimates for $P_{1,1}^{1,2}(t)$ may be understood as producing an estimate λ̂^1,2(t) for λ^1,2(t). The null hypothesis in (2) becomes, in the limit as δ → 0,

$$H_0:{\lambda }^{1,2}(t)={\lambda }^1(t){\lambda }^2(t),\forall t\in [0,T),$$

or equivalently,

$$H_0:\xi (t)=1,\forall t\in [0,T).$$ (7)

The lag h case is treatly similarly. Under mild conditions, Theorems 1 and 2 of Section 5 show that the above heuristic arguments hold for a continuous-time regular marked point process. This in turn gives a rigorous asymptotic justification (as δ → 0) for estimation and testing procedures such as those in steps (i) and (ii) mentioned above, following (4), and illustrated in Section 3.

2.2 Conditional methods for ν = 2

To deal with history effects, Equation (3) is replaced with

$$P_{1,1}^{1,2}(t|{\overline{\mathscr{H}}}_t)=P_1^1(t|{\overline{\mathscr{H}}}_t^1)P_1^2(t|{\overline{\mathscr{H}}}_t^2)\zeta (t),\forall t\in 𝒯,$$ (8)

where ${\overline{\mathscr{H}}}_t^i$, i = 1, 2, i = 1, 2, are, as in Section 1, the binned spiking histories of neurons 1 and 2, respectively, on the interval [0, t). Analogous to (4), the null hypothesis is

$$H_0:\zeta (t)=1,\forall t\in 𝒯.$$

We note that there are two substantial simplifications in (8). First, $P_1^i(t|{\overline{\mathscr{H}}}_t)=P_1^i(t|{\overline{\mathscr{H}}}_t^i)$, which says that neuron i’s own history ${\overline{\mathscr{H}}}_t^i$ is relevant in modifying its spiking probability (but not the other neuron’s history). Second, ζ(t) does not depend on the spiking history ℋ̄_t. This is important for what it claims about the physiology, for the way it simplifies statistical analysis, and for the constraint it places on the point process framework. Physiologically, it decomposes excess spiking into history-related effects and stimulus-related effects, which allows the kind of interpretation alluded to in Section 1 and presented in our data analysis in Section 3. Statistically, it improves power because tests of H₀ effectively pool information across trials, thereby increasing the effective sample size.

Consider counting processes $N_t^i$, i = 1, 2, as in Section 2.1. Under regularity conditions, the following limits exist for t ∈ [0, T):

$$\begin{array}{cc}\hfill {\lambda }^i(t|{\mathscr{H}}_t^i)& =\underset{\delta \to 0}{\text{lim}}{\delta }^{-1}P(N_{t+\delta }^i-N_t^i=1|{\overline{\mathscr{H}}}_t^i),i=1,2,\hfill \\ \hfill {\lambda }^{1,2}(t|{\mathscr{H}}_t)& =\underset{\delta \to 0}{\text{lim}}{\delta }^{-2}P((N_{t+\delta }^1-N_t^1)(N_{t+\delta }^2-N_t^2)=1|{\overline{\mathscr{H}}}_t),\hfill \\ \hfill \xi (t)& =\underset{\delta \to 0}{\text{lim}}\zeta (t),\hfill \end{array}$$ (9)

where ℋ_t = lim_δ→0 ℋ̄_t and ${\mathscr{H}}_t^i={\text{lim}}_{\delta \to 0}{\overline{\mathscr{H}}}_t^i$, i = 1, 2. For sufficiently small δ, we have

$$P_1^i(t|{\overline{\mathscr{H}}}_t^i)\approx {\lambda }^i(t|{\mathscr{H}}_t^i)\delta ,i=1,2,\text{ and }{P}_{1,1}^{1,2}(t|{\overline{\mathscr{H}}}_t)\approx {\lambda }^{1,2}(t|{\mathscr{H}}_t){\delta }^2,$$ (10)

for all t ∈ 𝒯. Again following Ventura, et al. (2005), we may smooth the observed-frequency estimates of $P_{1,1}^{1,2}(t|{\overline{\mathscr{H}}}_t)$ to produce an estimate of λ^1,2(t|ℋ_t), and smooth the observed-frequency estimates of $P_1^i(t|{\overline{\mathscr{H}}}_t^i)$ to produce estimates of ${\lambda }^i(t|{\mathscr{H}}_t^i)$, i = 1, 2. Letting δ → 0 in (8), we obtain

$${\lambda }^{(1,2)}(t|{\mathscr{H}}_t)=\xi (t){\lambda }^1(t|{\mathscr{H}}_t^1){\lambda }^2(t|{\mathscr{H}}_t^2),\forall t\in [0,T).$$ (11)

Consequently, for sufficiently small δ, a conditional test of H₀ : ζ(t) = 1 for all t becomes a test of the null hypothesis $H_0:{\lambda }^{1,2}(t|{\mathscr{H}}_t)={\lambda }^1(t|{\mathscr{H}}_t^1){\lambda }^2(t|{\mathscr{H}}_t^2)$ for all t or, equivalently, in this conditional case we have the same null hypothetical statement as (7).

In attempting to make Equation (9) rigorous, a difficulty arises: for a regular marked point process, the function ξ need not be independent of the spiking history. This would create a fundamental mismatch between the discrete data-analytical method and its continuous-time limit. The key to avoiding this problem is to enforce the sparsity condition (10). Specifically, the probabilities $P_1^i(t|{\overline{\mathscr{H}}}_t^i)$ are of order O(δ) while the probabilities $P_{1,1}^{1,2}(t|{\overline{\mathscr{H}}}_t)$ are of order O(δ²). This also allows independence models within the marked point process framework. Section 6 proposes a class of marked point process models indexed by δ and provides results that validate the heuristics above.

2.3 Loglinear models

We now reformulate in terms of loglinear models the procedures sketched in Sections 2.1 and 2.2 for ν = 2 neurons, and then indicate the way generalizations proceed when ν ≥ 3.

In the marginal case of Section 2.1, it is convenient to define

$$\begin{array}{cc}{\tilde{P}}_{0,0}^{1,2}(t)\hfill & =1,\hfill \\ {\tilde{P}}_{1,0}^{1,2}(t)\hfill & =P_1^1(t),\hfill \\ {\tilde{P}}_{0,1}^{1,2}(t)\hfill & =P_1^2(t),\hfill \\ {\tilde{P}}_{1,1}^{1,2}(t)\hfill & =P_{1,1}^{1,2}(t),\forall t\in 𝒯.\hfill \end{array}$$

Equation (3) implies that

$$\text{log}[{\tilde{P}}_{a,b}^{1,2}(t)]=a\text{log}[P_1^1(t)]+b\text{log}[P_1^2(t)]+\mathit{\text{ab}}\text{log}[\zeta (t)]$$ (12)

for all a, b ∈ {0, 1} and t ∈ 𝒯 and in the continuous-time limit, using (6), we write

$$\text{log}{\lambda }^{1,2}(t)=\text{log}{\lambda }^1(t)+\text{log}{\lambda }^2(t)+\text{log}[\xi (t)]$$ (13)

for t ∈ [0, T). The null hypothesis may then be written as

$$H_0:\text{log}[\xi (t)]=0,\forall t\in [0,T).$$ (14)

In the conditional case of Section 2.2, we similarly define

$$\begin{array}{cc}{\tilde{P}}_{0,0}^{1,2}(t|{\overline{\mathscr{H}}}_t)\hfill & =1,\hfill \\ {\tilde{P}}_{1,0}^{1,2}(t|{\overline{\mathscr{H}}}_t)\hfill & =P_1^1(t|{\overline{\mathscr{H}}}_t),\hfill \\ {\tilde{P}}_{0,1}^{1,2}(t|{\overline{\mathscr{H}}}_t)\hfill & =P_1^2(t|{\overline{\mathscr{H}}}_t),\hfill \\ {\tilde{P}}_{1,1}^{1,2}(t|{\overline{\mathscr{H}}}_t)\hfill & =P_{1,1}^{1,2}(t|{\overline{\mathscr{H}}}_t),\forall t\in 𝒯,\hfill \end{array}$$

we may rewrite (8) as the loglinear model

$$\text{log}[{\tilde{P}}_{a,b}^{1,2}(t|{\overline{\mathscr{H}}}_t)]=a\text{log}[P_1^1(t|{\overline{\mathscr{H}}}_t^1)]+b\text{log}[P_1^2(t|{\overline{\mathscr{H}}}_t^2)]+\mathit{\text{ab}}\text{log}[\zeta (t)],$$ (15)

for all a, b ∈ {0, 1} and t ∈ 𝒯 and in the continuous-time limit we rewrite (11) in the form

$$\text{log}{\lambda }^{1,2}(t|{\mathscr{H}}_t)=\text{log}{\lambda }^1(t|{\mathscr{H}}_t^1)+\text{log}{\lambda }^2(t|{\mathscr{H}}_t^2)+\text{log}[\xi (t)]$$ (16)

for t ∈ [0, T). The null hypothesis may again be written as in (14).

Rewriting the model in loglinear forms (12), (13), (15), and (16) allows us to generalize to ν ≥ 3 neurons. For example, with the obvious extensions of the previous definitions, for ν = 3 neurons the two-way interaction model in the continuous-time marginal case becomes

$$\text{log}[{\tilde{P}}_{a,b,c}^{1,2,3}(t)]=\text{log}[P_1^1(t)]+\text{log}[P_1^2(t)]+\text{log}[P_1^3(t)]+\mathit{\text{ab}}\text{log}[{\zeta }_{\{1,2\}}(t)]+\mathit{\text{ac}}\text{log}[{\zeta }_{\{1,3\}}(t)+\mathit{\text{bc}}\text{log}[{\zeta }_{\{2,3\}}(t)]$$ (17)

for all a, b, c ∈ {0, 1} and t ∈ 𝒯, and

$$\text{log}[{\lambda }^{1,2,3}(t)]=\text{log}[{\lambda }^1(t)]+\text{log}[{\lambda }^2(t)]+\text{log}[{\lambda }^3(t)]+\text{log}[{\xi }_{\{1,2\}}(t)]+\text{log}[{\xi }_{\{1,3\}}(t)]+\text{log}[{\xi }_{\{2,3\}}(t)]$$

for all t ∈ (0, T]. The general form of (17) is given by Equation (28) in Section 5. In the conditional case, the two-way interaction model becomes

$$\text{log}[{\tilde{P}}_{a,b,c}^{1,2,3}(t|{\overline{\mathscr{H}}}_t)]=a\text{log}[P_1^1(t|{\overline{\mathscr{H}}}_t)]+b\text{log}[P_1^2(t|{\overline{\mathscr{H}}}_t)]+c\text{log}[P_1^3(t|{\overline{\mathscr{H}}}_t)]+\mathit{\text{ab}}\text{log}[{\zeta }_{\{1,2\}}(t)]+\mathit{\text{ac}}\text{log}[{\zeta }_{\{1,3\}}(t)]+\mathit{\text{bc}}\text{log}[{\zeta }_{\{2,3\}}(t)]$$ (18)

for all a, b, c ∈ {0, 1} and t ∈ 𝒯 and in continuous time,

$$\text{log}[{\lambda }^{1,2,3}(t|{\mathscr{H}}_t)]=\text{log}[{\lambda }^1(t|{\mathscr{H}}_t)]+\text{log}[{\lambda }^2(t|{\mathscr{H}}_t)]+\text{log}[{\lambda }^3(t|{\mathscr{H}}_t)]+\text{log}[{\xi }_{\{1,2\}}(t)]+\text{log}[{\xi }_{\{1,3\}}(t)]+\text{log}[{\xi }_{\{2,3\}}(t)]$$

for all t ∈ (0, T]. In either the marginal or conditional case, the null hypothesis of independence may be written as

$$H_0:\text{log}[{\xi }_{\{i,j\}}(t)]=0,\forall t\in (0,T],1\le i<j\le 3.$$ (19)

On the other hand, we could include the additional term abc log[ξ_{1,2,3}(t)] and use the null hypothesis of no three-way interaction

$$H_0:\text{log}[{\xi }_{\{1,2,3\}}(t)]=0,\forall t\in (0,T].$$ (20)

These loglinear models offer a simple and powerful way for studying dependence among neurons when spiking history is taken into account. It has an important dimensionality reduction property in that the higher order terms are asymptotically independent of history. In practice this provides a huge advantage: the synchronous spikes are relatively rare; in assessing excess synchronous spiking with this model, the data may be pooled over different histories leading to a much larger effective sample size. The general conditional model in Equation (34) retains this structure. An additional feature of these loglinear models is that time-varying covariates may be included without introducing new complications. In Section 3 we use a covariate to characterize the network up states, which are visible in part C of Figure 1, simply by including it in calculating each of the individual-neuron conditional intensities ${\lambda }^1(t|{\mathscr{H}}_t^1)\text{ and }{\lambda }^2(t|{\mathscr{H}}_t^2)$ in (16).

Sometimes, as in the data we analyze here, the synchronous events are too sparse to allow estimation of time-varying excess synchrony and we must assume it to be constant, ζ(t) = ζ for all t. Thus, for ν = 2, the models of (12) or (15) take simplified forms in which ζ(t) is replaced by the constant ζ and we would use different test statistics to test the null hypothesis H₀ : ζ = 1. To distinguish the marginal and conditional cases we replace ζ(t) by ζ_H in (15) and then also write H₀ : ζ_H = 1. Moving to continuous time, which is simpler computationally, we write ξ(t) = ξ, estimate ξ and ξ_H, and test H₀ : ξ = 1 and H₀ : ξ_H = 1. Specifically, we apply the loglinear models (12), (13), (15), and (16) in two steps. First, we smooth the respective PSTHs to produce smoothed curves λ̂ⁱ(t), as in parts A and B of Figure 1. Second, ignoring estimation uncertainty and taking λⁱ(t) = λ̂ⁱ(t), we estimate the constant ζ. Using the point process representation of joint spiking (justified by the results in Sections 5 and 6) we may then write

$$\text{log}L(\xi )=-{\displaystyle \int \lambda (t)\mathit{\text{dt}}}+{\displaystyle \sum \text{log}\lambda (t_i)}$$

where the sum is over the joint spike times t_i and λ(t) is replaced by the right-hand side of (13), in the marginal case, or (16), in the conditional case. It is easy to maximize the likelihood L(ξ) analytically: setting the left-hand side to ℓ(ξ), in the marginal case we have

$$\ell \prime (\xi )=-{\displaystyle \int {\lambda }^1(t){\lambda }^2(t)\mathit{\text{dt}}}+\frac{N}{\xi }$$

where N is the number of joint (synchronous) spikes (the number of terms in the sum), while in the conditional case we have the analogous formula

$$\ell \prime ({\xi }_H)=-{\displaystyle \int {\lambda }^1(t|{\mathscr{H}}_t^1){\lambda }^2(t|{\mathscr{H}}_t^2)\mathit{\text{dt}}}+\frac{N}{{\xi }_H}$$

and setting to 0 and solving gives

$$\widehat{\xi }=\frac{N}{{\displaystyle \int {\lambda }_1^1(t){\lambda }_1^2(t)\mathit{\text{dt}}}}$$ (21)

and

$${\widehat{\xi }}_H=\frac{N}{{\displaystyle \int {\lambda }_1^1(t|{\mathscr{H}}_t^1){\lambda }^2(t|{\mathscr{H}}_t^2)\mathit{\text{dt}}}}$$ (22)

which, in both cases, is the ratio of the number of observed joint spikes to the number expected under independence.

We apply (21) and (22) in Section 3. To test H₀ : ξ = 1 and H₀ : ξ_H = 1 we use a bootstrap procedure in which we generate spike trains under the relevant null-hypothetical model. This is carried out in discrete time, and requires all 4 cell probabilities ${\tilde{P}}_{a,b}^{1,2}(t)\text{ or }{\tilde{P}}_{a,b}^{1,2}(t|{\overline{\mathscr{H}}}_t)$ at every time t ∈ 𝒯. These are easily obtained by subtraction using $P_1^1(t)={\lambda }^1(t)\delta ,P_1^2(t)={\lambda }^2(t)\delta$ and ζ̂ = ξ̂ or, in the conditional case, $P_1^1(t|{\overline{\mathscr{H}}}_t^1)={\lambda }^1(t|{\mathscr{H}}_t^1)\delta ,P_1^2(t|{\overline{\mathscr{H}}}_t^2)={\lambda }^2(t|{\mathscr{H}}_t^2)\delta$, and ζ̂_H = ξ̂_H. As we said above, λⁱ(t) = λ̂ⁱ(t) is obtained from the preliminary step of smoothing the PSTH. Similarly, the conditional intensities ${\lambda }^i(t|{\mathscr{H}}_t^i)={\widehat{\lambda }}^i(t|{\mathscr{H}}_t^i)$ are obtained from smooth history-dependent intensity models such as those discussed in Kass, et al. (2005). In the analyses reported here we have used fixed-knot splines to describe variation across time t.

In the case of 3 or more neurons the analogous estimates and cell probabilities must, in general, be obtained by a version of iterative proportional fitting. For ν = 3, to test the null hypothesis (20) we follow the steps leading to (21) and (22). Under the assumption of constant ζ₁₂₃ we obtain

$${\widehat{\xi }}_{123}=\frac{N}{{\displaystyle \int {\lambda }_1^1(t){\lambda }_1^2(t){\lambda }^3(t){\xi }_{12}(t){\xi }_{13}(t){\xi }_{23}(t)\mathit{\text{dt}}}}$$ (23)

and

$${\widehat{\xi }}_{123,H}=\frac{N}{{\displaystyle \int {\lambda }_1^1(t|{\mathscr{H}}_t^1){\lambda }^2(t|{\mathscr{H}}_t^2){\lambda }^3(t|H_t){\xi }_{12,H}(t){\xi }_{13,H}(t){\xi }_{23,H}(t)\mathit{\text{dt}}}}.$$ (24)

In Section 3 we fit (17) and report a bootstrap test of the hypothesis (20) using the test statistic ξ̂₁₂₃ in (23).

3 Data Analysis

We applied the methods of Section 2.3 to a subset of the data described in Section 1.1 and present the results here. We plan to report a more comprehensive analysis elsewhere.

We took δ = 5 milliseconds (ms), which is a commonly-used window width in studies of synchronous spiking. Raster plots of spike trains across repeated trials from a pair of neurons are shown in Parts A and B of Figure 2, with synchronous events indicated by circles. Below each raster plot is a smoothed PSTH, that is, the two plots show smoothed estimates λ̂¹(t) and λ̂²(t) of λ¹(t) and λ²(t) in (6), the units being spikes per second. Smoothing was performed by fitting a generalized additive model using cubic splines with knots spaced 100ms apart. Specifically, we applied Poisson regression to the count data resulting from pooling the binary spike indicators across trials: for each time bin the count was the number of trials on which a spike occurred. To test H₀ under the model in (13) we applied (21) to find log ξ̂. We then computed a parametric bootstrap standard error of log ξ̂ by generating pseudo-data from model (13) assuming H₀ : log ξ = 0. We generated 1000 such trials, giving 1000 pseudo-data values of log ξ̂, and computed the standard deviation of those values as a standard error, to obtain an observed z-ratio test statistic of 3.03 (p = .0012 according to asymptotic normality).

The highly significant z ratio shows that there is excess sychronous spiking beyond what would be expected from the varying firing rates of the two neurons under independence. However, it does not address the source of the excess synchronous spiking. The excess synchronous spiking could depend on the stimulus or, alternatively, it might be due to the slow waves of population activity evident in part (C) of Figure 1, the time of which vary from trial to trial and therefore do not depend on the stimulus. To examine the latter possibility we applied a within-trial loglinear model as in (16) except that we incorporated into the history effect not only the history of each neuron but also a covariate representing the population effect. Specifically, for neuron i (i = 1, 2) we used the same generalized additive model as before, but with two additional variables. The first was a variable that, for each time bin, was equal to the number of neuron i spikes that had occurred in the previous 100ms. The second was a variable that, for each time bin, was equal to the number of spikes that occurred in the previous 100ms across the whole population of neurons, other than neurons 1 and 2. We thereby obtained fitted estimates ${\widehat{\lambda }}^1(t|{\mathscr{H}}_t^1)\text{ and }{\widehat{\lambda }}^2(t|{\mathscr{H}}_t^2)$ of ${\lambda }^1(t|{\mathscr{H}}_t^1)\text{ and }{\lambda }^2(t|{\mathscr{H}}_t^2)$. Note that the fits for the independence model, defined by applying (7) to (11), now vary from trial to trial due to the history effects. Applying (22) we found log ξ̂_H, and then again computed a bootstrap standard error of log ξ̂ by creating 1000 trials of pseudo-data, giving a z-ratio of .39, which is clearly not significant.

Raster plots for a different pair of neurons are shown in parts (E) and (F) of Figure 2. The same procedures were applied to this pair. Here, the z-ratio for testing H₀ under the marginal model was 3.77 (p < .0001) while that for testing H₀ under the conditional model remained highly significant at 3.57 (p = .0002). In other words, using the loglinear model methodology we have discovered two pairs of V1 neurons with quite different behavior. For the first pair, synchrony can be explained entirely by network effects, while for the second pair it can not; this suggests that, instead, for the second pair, some of the excess synchrony may be stimulus-related.

We also compared the marginal and conditional models (13) and (16) using ROC curves. Specifically, for the binary joint spiking data we used each model to predict a spike whenever the intensity was larger than a given constant: for the marginal case whenever log λ^1,2(t) > c_marginal, and for the conditional case whenever log λ^1,2(t|ℋ_t) > c_conditional. The choice of constants c_marginal and c_conditional reflect trade-offs between false positive and true positive rates (analogous to type I error and power) and as we vary the constants the plot of true vs. false positive rates forms the ROC curve. To determine the true and false positive rates we performed ten-fold cross-validation, repeatedly fitting from 90% of the trials and predicting from the remaining 10% of the trials. The two resulting ROC curves are shown in part D of Figure 2, labeled as “no history” and “history,” respectively. To be clear, in the two cases we included the terms corresponding, respectively, to ξ and ξ_H, and in the history case we included both the auto-history and the network history variables specified above. The ROC curve for the conditional model strongly dominates that for the marginal model, indicating far better predictive power. In part C of Figure 2 we display the true positive joint spike predictions when the false-positive rate was held at 10%. These correctly-predicted joint spikes may be compared to the complete set displayed in parts A and B of the figure. The top display in part C, labeled “no history,” shows that only a few joint spikes were correctly predicted by the marginal model, while the large majority were correctly predicted by the conditional model. Furthermore, the correctly predicted joint spikes are spread fairly evenly across time. In contrast, the ROC curves for the second pair of neurons, shown in part (G) of Figure 2, are close to each other: inclusion of the history effects (which were statistically significant) did not greatly improve predictive power. Furthermore, the correctly predicted synchronous spikes are clustered in time, with the main cluster occurring near a peak in the individual-neuron firing-rate functions shown in the two smoothed PSTHs in parts (E) and (F).

Taking all of the results together, our analysis suggests that the first pair of neurons produced excess synchronous spikes solely in conjunction with network effects, which are unrelated to the stimulus, while for the second pair of neurons some of the excess synchronous spikes occurred separately from the network activity and were, instead, often time-locked and, therefore, stimulus-related.

We also tried to assess whether 2-way interactions were sufficient to explain observed 3-way events by fitting the no-3-way interaction model given by (17), and then testing the null hypothesis in (20). We did this for a particular set of 3 neurons, whose joint spikes are displayed in Figure 3. The method is analogous to that carried out above for pairs of neurons, in the sense that the test statistic was ξ̂₁₂₃ given by (23) and a parametric bootstrap procedure, based on the fit of (17), was used to compute an approximate p-value. Fitting of (17) required an iterative proportional fitting procedure, which we will describe in detail elsewhere. We obtained p = .16, indicating no significant 3-way interaction. In other words, for these three neurons, 2-way excess joint spiking appears able to explain the occurrence of the 3-way joint spikes. However, as may be seen in Figure 3, there are very few 3-way spikes in the data. We mention this issue again in our discussion.

Figure 3.

Plots of synchronous spiking events for 3 neurons. Each of the three plots displays all joint spikes (as circles) for a particular pair of neurons. The dark circles in each plot indicate the 3-way joint spikes.

4 A marked point process framework

In this section, a class of marked point processes for modeling neural spike trains is briefly surveyed. These models take into account the possibility of two or more neurons firing in synchrony (i.e. at the same time). Consider an ensemble of ν neurons labeled 1 to ν. For T > 0, let N_T denote the total number of spikes produced by this ensemble on the time interval [0, T) and let 0 ≤ s₁ < ⋯ < s_NT < T denote the specific spike times. For each j = 1, …, N_T, we write (s_j, (i₁, …, i_k)) to denote the event that a spike was fired synchronously) at time s_j by (and only by) the i₁, …, i_k neurons. We observe that

$${\mathscr{H}}_T=\{(s_1,{\kappa }_1),\dots ,(S_{N_T},{\kappa }_{N_T}):{\kappa }_j\in 𝒦,j=1,\dots ,N_T\}$$ (25)

forms a marked point process on the interval [0, T) with 𝒦 as the mark space satisfying

$$𝒦\subseteq \{(i_1,\dots ,i_k):1\le i_1<\cdots <i_k\le \nu ,k=1,\dots ,\nu \}.$$

We follow Daley and Vere-Jones (2002), page 249, and define the conditional intensity function of ℋ_T as

$$\begin{array}{cc}\lambda (t,\kappa |{\mathscr{H}}_t)\hfill & =\lambda (t,\kappa |\{(s_1,{\kappa }_1),\dots ,(s_{N_t,}{\kappa }_{N_t})\})\hfill \\ \hfill & =\{\begin{array}{c}{h}_1(t)f_1(\kappa |t)\forall 0\le t\le s_1,\hfill \\ h_{\mathrm{i}}(t|(s_1,{\kappa }_1),\dots ,(s_{i-1},{\kappa }_{i-1}))f_i(\kappa |t;(s_1,{\kappa }_1),\dots ,(s_{i-1},{\kappa }_{i-1}))\forall s_{i-1}<t\le s_i,i=2,\dots ,N_T,\hfill \\ h_{N_T+1}(t|(s_1,{\kappa }_1),\dots ,(s_{N_t},{\kappa }_{N_T}))f_{N_T+1}(\kappa |t;(s_1,{\kappa }_1),\dots ,(s_{N_T},{\kappa }_{N_T}))\forall s_{N_T}<t<T,\hfill \end{array}\hfill \end{array}$$

where h₁(․) is the hazard function for the location of the first spike s₁, h₂(․|(s₁, κ₁)) the hazard function for the location of the second spike s₂ conditioned by (s₁, κ₁), and so on, while f₁(․|t) is the conditional probability mass function of κ₁ given s₁ = t, and so on. It is also convenient to write λ(t, κ|∅) = λ(t, κ|ℋ_t) for all t < s₁. The following proposition and its proof can be found in Daley and Vere-Jones (2002), page 251.

Proposition 1. Let ℋ_T be as in (25). Then the density of ℋ_T is given by

$$p_{\lambda }({\mathscr{H}}_T)=p_{\lambda }(\{(s_1,{\kappa }_1),\dots ,(s_{N_T},{\kappa }_{N_T})\})=[{\displaystyle \prod _{i=1}^{N_T}}\lambda (s_i,{\kappa }_i|{\mathscr{H}}_{s_i})]\text{exp}[-{\displaystyle \sum _{\kappa \in 𝒦}}{\displaystyle {\int }_0^T}\lambda (t,\kappa |{\mathscr{H}}_t)\mathit{\text{dt}}].$$

5 Theoretical results: marginal methods

In this section we (i) provide a justification of the limiting statements in (6) and (ii) generalize to higher-order models. We also note that lagged dependence can be accommodated within our framework, treating the case ν = 2.

5.1 Regular marked point process and loglinear modeling

In this subsection, we prove that the heuristic arguments of Section 2 for marginal methods hold under mild conditions. Consider ν ≥ 1 neurons labeled 1 to ν. For T > 0, let N_T denote the total number of spikes produced by these ν neurons on the time interval [0, T) and let 0 ≤ s₁ < ⋯ < s_NT < T denote the specific spike times. For each j = 1, …, N_T, we write (s_j, (i_j)) to represent the event that a spike was fired at time s_j by neuron i_j where i_j ∈ {1, …, ν}. We observe from Section 4 that

$${\mathscr{H}}_T=\{(s_1,(i_1)),\dots ,(s_{N_T},(i_{N_T}))\}$$

forms a marked point process on the interval [0, T) with mark space 𝒦 = {(1), …, (ν)}. Following the notation of Section 4, let λ(t, (i)|ℋ_t) denote the conditional intensity function of the point process ℋ_T. We assume that the following two conditions hold.

Condition (I) There exists a strictly positive refractory period for each neuron in that there exists a constant θ > 0 such that λ(t, (i)|ℋ_t) = 0 if there exists some (s,(i)) ∈ ℋ_t such that t − s ≤ θ, i ∈ {1, …, ν}.

Condition (II) For each k ∈ {0, …, 2⌈T/θ⌉−1} and i, i₁, …, i_k ∈ {1, …, ν}, the conditional intensity function λ(t, (i)|{(s₁, (i₁)), …, (s_k, (i_k))}) is a continuously differentiable function in (s₁, …, s_k, t) over the simplex 0 ≤ s₁ ≤ ⋯ ≤ s_k ≤ t ≤ T.

If δ < θ, then Condition (I) implies that there is at most 1 spike from each neuron in a bin of width δ. Conditions (I) and (II) also imply that the marked point process is regular in that (exactly) synchronous spikes occur only with probability 0. Theorem 1 below gives the limiting relationship between the bin probabilities of the induced discrete-time process and the conditional intensities of the underlying continuous-time marked point process.

Theorem 1. Suppose that Conditions (I) and (II) hold, 1 ≤ i₁ < ⋯ i_k ≤ ν and 1 ≤ k ≤ ν. Then

$$\underset{\delta \to 0}{\text{lim}}{\delta }^{-k}{P}_{1,\dots ,1}^{i_1,\dots ,i_k}(t_m)=\frac{1}{k!}{\displaystyle \sum _{j_1,\dots ,j_k:\{j_1,\dots ,j_k\}=\{i_1,\dots ,i_k\}}}E{\displaystyle \prod _{l=1}^k}\left[\lambda \right(t,(j_k)|\{(t,(j_1))\}\cup \dots \cup \{(t,(j_{k-1}))\}\cup {\mathscr{H}}_t)],$$

where t_m = mδ → t as δ → 0 and λ(t, (i₂)|{(t, (i₁))} ∪ ℋ_t) = lim_t*→t− λ(t, (i₂)|{(t*, (i₁))} ∪ ℋ_t), etc. Here the expectation is taken with respect to ℋ_t.

Theorem 1 validates the heuristics stated in (6) where ν = 2,

$$\begin{array}{cc}\hfill {\lambda }^i(t)& =E[\lambda (t,(i)|{\mathscr{H}}_t)],\hfill \\ \hfill {\lambda }^{1,2}(t)& =\frac{1}{2}{\displaystyle \sum _{1\le i_1\ne i_2\le 2}}E\left[\lambda \right(t,(i_2)|\{(t,(i_1))\}\cup {\mathscr{H}}_t)\lambda (t,(i_1)|{\mathscr{H}}_t)].\hfill \end{array}$$

Next we construct the discrete-time loglinear model induced by the above marked point process. First define recursively for t_m = mδ,

$$\begin{array}{cc}\hfill {\zeta }_{\{i_1\}}(t_m)& ={\delta }^{-1}{P}_1^{i_1}(t_m),\hspace{1em}\forall i_1=1,\dots ,\nu ,\hfill \\ \hfill {\zeta }_{\{i_1,i_2\}}(t_m)& =\frac{{\delta }^{-2}{P}_{1,1}^{i_1,i_2}(t_m)}{{\zeta }_{\{i_1\}}(t_m){\zeta }_{\{i_2\}}(t_m)},\hspace{1em}\forall 1\le i_1<i_2\le \nu ,\hfill \\ \hfill \dots & \hfill \\ \hfill {\zeta }_{\{i_1,\dots ,i_k\}}(t_m)& =\frac{{\delta }^{-k}{P}_{1,\dots ,1}^{i_1,\dots ,i_k}(t_m)}{{\displaystyle {\prod }_{\mathrm{\Xi }⊊\{i_1,\dots ,i_k\}}{\zeta }_{\mathrm{\Xi }}(t_m)}},\hspace{1em}\forall 1\le i_1<\cdots <i_k\le \nu ,2\le k\le \nu .\hfill \end{array}$$ (26)

We further define

$${\xi }_{\{i_1,\dots ,i_k\}}(t)=\underset{\delta \to 0}{\text{lim}}{\zeta }_{\{i_1,\dots ,i_k\}}(t_m)\forall 1\le i_1<\dots <i_k\le \nu ,1\le k\le \nu ,$$ (27)

where lim_δ→0 t_m → t, whenever the expression on the right hand side of (27) is well-defined. The following is an immediate corollary of Theorem 1.

Corollary 1. Let ξ_{i1}(t) and ξ_{i1,…,ik}(t) be as in (27). Then with the notation and assumptions of Theorem 1, we have

$$\begin{array}{cc}\hfill {\xi }_{\{i_1\}}(t)& =E[\lambda (t,(i_1)|{\mathscr{H}}_t)],\hfill \\ \hfill {\xi }_{\{i_1,\dots ,i_k\}}(t)& ={[k!{\displaystyle \prod _{\mathrm{\Xi }⊊\{i_1,\dots ,i_k\}}}{\zeta }_{\mathrm{\Xi }}(t)]}^{-1}\times {\displaystyle \sum _{\{j_1,\dots ,j_k\}=\{i_1,\dots ,i_k\}}}E{\displaystyle \prod _{l=1}^k}\left[\lambda \left(t,(j_k)|\{(t,(j_1))\}\cup \dots \cup \{(t,(j_{k-1}))\}\cup {\mathscr{H}}_t\right)\right],\hfill \end{array}$$

whenever the right hand sides are well-defined.

It is convenient to define ${\tilde{P}}_{0,\dots ,0}^{1,\dots ,\nu }(t_m)=1$. For a₁, …, a_ν ∈ {0, 1} and not all 0, define ${\tilde{P}}_{a_1,\dots ,a_{\nu }}^{1,\dots ,\nu }(t_m)=P_{1,\dots ,1}^{i_1,\dots ,i_k}(t_m)$ where i ∈ {i₁, …, i_k} if and only if a_i = 1. Then using the notation of (26), the corresponding loglinear model induced by the above marked point process is

$$\text{log}[{\tilde{P}}_{a_1,\dots ,a_{\nu }}^{1,\dots ,\nu }(t_m)]=\text{log}[P_{1,\dots ,1}^{i_1,\dots ,i_k}(t_m)]={\displaystyle \sum _{i=1}^{\nu }}{a}_i\text{log}[P_1^i(t_m)]+{\displaystyle \sum _{\mathrm{\Xi }\subseteq \{1,\dots ,\nu \}:|\mathrm{\Xi }|\ge 2}}({\displaystyle \prod _{j\in \mathrm{\Xi }}}{a}_j)\text{log}[{\zeta }_{\mathrm{\Xi }}(t_m)],$$ (28)

for all a₁, …, a_ν ∈ {0, 1}. Under Conditions (I) and (II), Corollary 1 shows that ξ_Ξ(t) = lim_δ→0 ζ_Ξ(t_m) is continuously differentiable. This gives an asymptotic justification for smoothing the estimates of ζ_Ξ, Ξ ⊆ {1, …, ν}.

5.2 Case of ν = 2 neurons with lag h

This subsection considers the lag h case for two neurons labeled 1 and 2. Let h, m be integers such that 0 ≤ m ≤ m + h ≤ Tδ⁻¹ − 1. As in (1), we write

$$P_{a,b}^{1,2}(t_m,t_{m+h})=P[X^2(t_{m+h})=b,X^1(t_m)=a],\forall a,b\in \{0,1\},$$

where t_m = mδ and t_{m + h} = (m + h)δ. Analogous to Theorem 1, we have the following results for the lag case.

Theorem 2. Suppose Conditions (I) and (II) hold. Then

$$\underset{\delta \to 0}{\text{lim}}\frac{P_{1,1}^{1,2}(t_m,t_{m+h})}{{\delta }^2}=E\left[\lambda (t+\tau ,(2)|\{(t,(1))\}\cup {\mathscr{H}}_{t+\tau })\lambda (t,(1)|{\mathscr{H}}_t)\right],$$

where t_m+h → t + τ and t_m → t as δ → 0 for some constant τ > 0. Here the expectation is taken with respect to ℋ_t+τ (and hence also ℋ_t).

Corollary 2. Let ζ(t_m, t_m+h) be defined as in (5). Then with the notation and assumptions of Theorem 2, we have

$$\underset{\delta \to o}{\text{lim}}\zeta (t_m,t_{m+h})=\frac{E[\lambda (t+\tau ,(2)|\{(t,(1))\}\cup {\mathscr{H}}_{t+\tau })\lambda (t,(1)|{\mathscr{H}}_t)]}{E[\lambda (t+\tau ,(2)|{\mathscr{H}}_{t+\tau })]E[\lambda (t,(1)|{\mathscr{H}}_t)]},\forall 0\le t<T-\tau ,$$ (29)

whenever the right hand side is well-defined.

We observe from Conditions (I) and (II) that the right hand side of (29) is continuously differentiable in t. Again this provides an asymptotic justification for smoothing the estimate of ζ(t, t + τ), with respect to t, when δ is small.

6 Theoretical results: conditional methods

This section is analogous to Section 5, but treats the conditional case. We (i) provide a justification of the limiting statements in (9) and (ii) generalize to higher-order models. We again also note that lagged dependence can be accommodated within our framework, treating the case ν = 2.

6.1 Synchrony and loglinear modeling

This subsection considers ν ≥ 1 neurons labeled 1 to ν. We model the spike trains generated by these neurons on [0, T) by a marked point process ℋ_T with mark space

$$𝒦=\{(i_1,\dots ,i_k):1\le i_1<\cdots <i_k\le \nu ,k=1,\dots ,\nu \}.$$

Here, for example, the mark (i₁) denotes the event that neuron i₁ (and only this neuron) spikes, (i₁, i₂) denotes the event that neuron i₁ and neuron i₂ (and only these two neurons) spike in synchrony (i.e. at the same time), and the mark (1, …, ν) denotes the event that all ν neurons spike in synchrony.

Let N_t denote the total number of spikes produced by these neurons on [0, t), 0 < t ≤ T, and let 0 ≤ s₁ < ⋯ < s_NT < T denote the specific spike times. For each j = 1, …, N_T, let κ_j ∈ 𝒦 be the mark associated with s_j. Then ℋ_T can be expressed as

$${\mathscr{H}}_T=\{(s_1,{\kappa }_1),\dots ,(s_{N_T},{\kappa }_{N_T})\}.$$ (30)

Given ℋ_t, we write

$${\mathscr{H}}_t^i=\left\{s:(s,\kappa )\in {\mathscr{H}}_t\text{ for some }\kappa =(i_1,\dots ,i_k)\text{ such that }i\in \{i_1,\dots ,i_k\}\right\},\forall i=1,\dots ,\nu .$$

${\mathscr{H}}_t^i$ denotes the spiking history of neuron i on [0, t). The conditional intensity function λ(t, κ|ℋ_t), t ∈ [0, T) and κ ∈ 𝒦, of the marked point process ℋ_T is defined to be

$$\begin{array}{cc}\hfill \lambda (t,(i)|{\mathscr{H}}_t)& ={\lambda }^i(t|{\mathscr{H}}_t^i),\forall t\in [0,T),\hfill \\ \hfill \lambda (t,(i_1,\dots ,i_k)|{\mathscr{H}}_t)& ={\delta }^{k-1}{\gamma }_{\{i_1,\dots ,i_k\}}(t){\displaystyle \prod _{j=1}^k}{\lambda }^{i_j}(t|{\mathscr{H}}_t^{i_j}),\forall t\in [0,T),\hfill \end{array}$$ (31)

where δ > 0 is a constant, γ_{i1,…,ik}(t)’s are functions depending only on t and the ${\lambda }^i(t|{\mathscr{H}}_t^i)’s$ are conditional intensity functions depending only on the spiking history of neuron i. We take γ_{i}(t) to be identically equal to 1.

From (31), we note that the above marked point process model is not a single marked point process but rather a family of marked point processes indexed by δ. In the sequel, we let δ → 0. We further assume that the following two conditions hold.

Condition (III) There exists a strictly positive refractory period for each neuron in that there is a constant θ > 0 such that for i = 1, …, ν and t ∈ [0, T),

$${\lambda }^i(t|{\mathscr{H}}_t^i)=0,\text{ if there exists some }s\in {\mathscr{H}}_t^i\text{ such that }t-s\le \theta .$$

Condition (IV) For each k ∈ {0, …, ⌈T/θ⌉ − 1} and i ∈ {1, …, ν}, λⁱ(t|{s₁, …, s_k}) is a continuously differentiable function in (s₁, …, s_k, t) over the simplex 0 ≤ s₁ ≤ ⋯ ≤ s_k ≤ t ≤ T.

Following Section 2.2, we divide the time interval [0, T) into bins of width δ. For simplicity, we assume that T is a multiple of δ. Let t_m = mδ and Xⁱ(t_m), m = 0, …, Tδ⁻¹ − 1, be as in Section 1. If Xⁱ(t_l) = 1 for all l ∈ {l₁, …, l_k}, and Xⁱ(t_l) = 0 otherwise, for some subset 0 ≤ l₁ < ⋯ < l_k ≤ m − 1, we write

$${\overline{\mathscr{H}}}_{t_m}^i=\{t_{l_1},\dots ,t_{l_k}\},{\overline{\mathscr{H}}}_{t_m}=({\overline{\mathscr{H}}}_{t_m}^1,\dots ,{\overline{\mathscr{H}}}_{t_m}^{\nu }).$$ (32)

It should be observed that although the above definitions of ${\mathscr{H}}_{t_m}^i\text{ and }{\overline{\mathscr{H}}}_{t_m}$ differ from those given in Section 1.2, they are equivalent. We note that the conditional intensity functions λ(t, (i₁, …, i_k)|ℋ_t) in (31) depend on the bin width δ. This is necessary in order to preserve the natural hierarchical sparsity conditions given by

$$\underset{m=0,\dots ,T{\delta }^{-1}-1}{\text{sup}}P[X^{i_1}(t_m)=1,\dots ,X^{i_k}(t_m)=1]=O({\delta }^k),$$

as δ → 0 for all 1 ≤ i₁ < ⋯ < i_k ≤ ν, 1 ≤ k ≤ ν.

Theorem 3. Consider the marked point process ℋ_T as in (30) with conditional intensity function satisfying (31). Then under Conditions (III) and (IV), we have

$$\begin{array}{cc}\hfill P_1^i(t_m|{\overline{\mathscr{H}}}_{t_m})& =\delta {\lambda }^i(t_m|{\overline{\mathscr{H}}}_{t_m}^i)+O({\delta }^2),\hfill \\ P_{1,1}^{i_1,i_2}(t_m|{\overline{\mathscr{H}}}_{t_m})\hfill & ={\delta }^2[1+{\gamma }_{\{i_1,i_2\}}(t_m)]{\displaystyle \prod _{j=1}^2}[{\lambda }^{i_j}(t_m|{\overline{\mathscr{H}}}_{t_m}^{i_j})]+O({\delta }^3),\hfill \end{array}$$

and in general,

$$P_{1,\dots ,1}^{i_1,\dots ,i_k}(t_m|{\overline{\mathscr{H}}}_{t_m})={\delta }^k\{{\displaystyle \sum _{j=1}^k}{\displaystyle \sum _{{\mathrm{\Xi }}_1,\dots ,{\mathrm{\Xi }}_j:\mathit{\text{all disjoint and nonempty}},\cup {\mathrm{\Xi }}_j=\{i_1,\dots ,i_k\}}}{\displaystyle \prod _{l=1}^j}{\gamma }_{{\mathrm{\Xi }}_l}(t_m)\}{\displaystyle \prod _{\ell =1}^k}[{\lambda }^{i_{\ell }}(t_m|{\overline{\mathscr{H}}}_{t_m}^{i_{\ell }})]+O({\delta }^{k+1}),$$

for sufficiently small δ where ℋ̄_tm and ${\overline{\mathscr{H}}}_{t_m}^i$ are defined by (32).

The following is an immediate corollary of Theorem 3. It gives an asymptotic justification for Equation (8) in Section 2.2.

Corollary 3. With the notation and assumptions of Theorem 3, we have for ν = 2,

$$P_{1,1}^{1,2}(t_m|{\overline{\mathscr{H}}}_{t_m})=\zeta (t_m)P_1^1(t_m|{\overline{\mathscr{H}}}_{t_m}^1)P_1^2(t_m|{\overline{\mathscr{H}}}_{t_m}^2)+O({\delta }^3),$$

for sufficiently small δ > 0 uniformly over ${\overline{\mathscr{H}}}_{t_m}^i$, m = 0, …, Tδ⁻¹ − 1 where ζ(t_m) = 1 + γ_{1,2}(t_m).

We now use Theorem 3 to construct a loglinear model (for the above spike train data) whose higher order coefficients are asymptotically independent of past spiking history. First define recursively,

$$\begin{array}{cc}\hfill {\zeta }_{\{i_1\}}(t_m|{\overline{\mathscr{H}}}_{t_m})& ={\delta }^1{P}_1^{i_1}(t_m|{\overline{\mathscr{H}}}_{t_m}),\hspace{1em}\forall i_1=1,\dots ,\nu ,\hfill \\ \hfill {\zeta }_{\{i_1,i_2\}}(t_m|{\overline{\mathscr{H}}}_{t_m})& =\frac{{\delta }^{-2}{P}_{1,1}^{i_1,i_2}(t_m|{\overline{\mathscr{H}}}_{t_m})}{{\zeta }_{\{i_1\}}(t_m|{\overline{\mathscr{H}}}_{t_m}){\zeta }_{\{i_2\}}(t_m|{\overline{\mathscr{H}}}_{t_m})},\hspace{1em}{\forall }_1\le i_1<i_2\le \nu ,\hfill \\ \hfill \dots & \hfill \\ \hfill {\zeta }_{\{i_1,\dots ,i_k\}}(t_m|{\overline{\mathscr{H}}}_{t_m})& \frac{={\delta }^{-k}{P}_{1,\dots ,1}^{i_1,\dots ,i_k}(t_m|{\overline{\mathscr{H}}}_{t_m})}{{\displaystyle {\prod }_{\mathrm{\Xi }⊊\{i_1,\dots ,i_k\}}{\zeta }_{\mathrm{\Xi }}(t_m|{\overline{\mathscr{H}}}_{t_m})}},\hspace{1em}\forall 1\le i_1<\cdots <i_k\le \nu .\hfill \end{array}$$ (33)

It follows from Theorem 3 and (33) that for sufficiently small δ,

$$\begin{array}{cc}\hfill {\zeta }_{\{i_1\}}(t_m|{\overline{\mathscr{H}}}_{t_m})& ={\mathrm{\lambda }}^{i_1}(t_m|{\overline{\mathscr{H}}}_{t_m}^{i_1})+O(\delta ),\hspace{1em}\forall i_1=1,\dots ,\nu ,\hfill \\ \hfill {\zeta }_{\{i_1,i_2\}}(t_m|{\overline{\mathscr{H}}}_{t_m})& =1+{\gamma }_{\{i_1,i_2\}}(t_m)+O(\delta ),\hspace{1em}\forall 1\le i_1<i_2\le \nu ,\hfill \\ \hfill \dots & \hfill \\ \hfill {\zeta }_{\{i_1,\dots ,i_k\}}(t_m|{\overline{\mathscr{H}}}_{t_m})& =\frac{{\displaystyle {\sum }_{j=1}^k{\displaystyle {\sum }_{{\mathrm{\Xi }}_1,\dots ,{\mathrm{\Xi }}_j:\text{all}\text{disjoint}\text{and}\text{nonempty},\cup {\mathrm{\Xi }}_j=\{i_1,\dots ,i_k\}}{\displaystyle {\prod }_{l=1}^j{\gamma }_{{\mathrm{\Xi }}_l(t_m)}}}}}{{\displaystyle {\prod }_{\mathrm{\Xi }⊊\{i_1,\dots ,i_k\}:|\mathrm{\Xi }|\ge 2}{\zeta }_{\mathrm{\Xi }}(t_m|{\overline{\mathscr{H}}}_{t_m})}}+O(\delta )\hfill \end{array}$$

whenever 1 ≤ i₁ < ⋯ < i_k ≤ ν and k ≥ 2, assuming that terms on the right hand side are well-defined. The practical importance of these results lies in the fact that the coefficients ζ_{i1,…,ik}(t_m|ℋ̄_tm) with k ≥ 2 are asymptotically (as δ → 0) independent of ℋ̄_tm, the spiking history of the neurons. It is convenient to define ${\tilde{P}}_{0,\dots ,0}^{1,\dots ,\nu }(t_m|{\overline{\mathscr{H}}}_{t_m})=1$. For a₁, …, a_ν ∈ {0, 1} and not all 0, define ${\tilde{P}}_{a_1,\dots ,a_{\nu }}^{1,\dots ,\nu }(t_m|{\overline{\mathscr{H}}}_{t_m})=P_{1,\dots ,1}^{i_{1,\dots ,i_k}}(t_m|{\overline{\mathscr{H}}}_{t_m})$ where i ∈ {i₁, …, i_k} if and only if a_i = 1. Then the induced loglinear model is

$$\text{log}[{\tilde{P}}_{a_1,\dots ,a_{\nu }}^{1,\dots ,\nu }(t_m|{\overline{\mathscr{H}}}_{t_m})]={\displaystyle \sum _{i=1}^{\nu }}{a}_i\text{log}[P_1^i(t_m|{\overline{\mathscr{H}}}_{t_m})]+{\displaystyle \sum _{\mathrm{\Xi }\subseteq \{1,\dots ,\nu \}:|\mathrm{\Xi }|\ge 2}}({\displaystyle \prod _{j\in \mathrm{\Xi }}}{a}_j)\text{log}[{\zeta }_{\mathrm{\Xi }}(t_m|{\overline{\mathscr{H}}}_{t_m})],$$ (34)

for all a₁, …, a_ν ∈ {0, 1} where the second term on the right hand side of (34) is asymptotically (as δ → 0) independent of the spiking history ℋ̄_tm.

6.2 ν = 2 neurons with time-delayed synchrony

This subsection considers ν = 2 neurons labeled 1, 2 and let τ > 0 be a constant denoting the spike lag. We model the spike train generated by the two neurons on [0, T) by a marked point process ℋ_T as in (25) with mark space 𝒦 = {(1), (2), (1, 2)}. The marks (1), (2) are interpreted as before as isolated (i.e. non-synchronous) spikes. However now, (1, 2) is interpreted to be neuron 1 spiking first and then neuron 2 spiking second after a delay of τ time units. The mark (1, 2) is used to model a precise time-delayed synchronous spiking of lag τ between the 2 neurons.

Let N_T denote the number of times the three marks occur on [0, T) and s₁ < ⋯ < s_NT be the specific spike times. For each j = 1, …, N_T, let κ_j ∈ 𝒦 be the mark associated with s_j. Then ℋ_t can be decomposed into $({\mathscr{H}}_t^1,{\mathscr{H}}_{t+\tau }^2)$ where

$$\begin{array}{cc}\hfill {\mathscr{H}}_t^1& =\{s:(s,\kappa )\in {\mathscr{H}}_t\text{ where }\kappa \in \{(1),(1,2)\}\},\hfill \\ \hfill {\mathscr{H}}_{t+\tau }^2& =\{s:(s,\kappa )\in {\mathscr{H}}_{t+\tau }\text{ where }\kappa \in \{(2),(1,2)\}\}.\hfill \end{array}$$ (35)

To be definite, (s, κ) = (s, (1, 2)) means neuron 1 spikes at time s and neuron 2 spikes at time s + τ. The conditional intensity function λ(t, κ|ℋ_t), t ∈ [0, T) and κ ∈ 𝒦, of the marked point process ℋ_T is defined to be

$$\begin{array}{cc}\hfill \lambda (t,(i)|{\mathscr{H}}_t)& ={\lambda }^i(t|{\mathscr{H}}_t^i),\forall i=1,2,t\in [0,T),\hfill \\ \hfill \lambda (t,(1,2)|{\mathscr{H}}_t)& =\delta \gamma (t,t+\tau ){\lambda }^1(t|{\mathscr{H}}_t^1){\lambda }^2(t+\tau |{\mathscr{H}}_{t+\tau }^2),\forall t\in [0,T),\hfill \end{array}$$ (36)

where δ > 0 is a constant, γ(t, t + τ) is a continuously differentiable function in t on the interval 0 ≤ t ≤ T − τ, and ${\lambda }^1(t|{\mathscr{H}}_t^1),{\lambda }^2(t+\tau |{\mathscr{H}}_{t+\tau }^2)$ are conditional intensity functions depending only on the spiking history of neuron 1 up to time t and on the spiking history of neuron 2 up to time t + τ respectively.

As in Section 1, we divide the time interval [0, T) into bins of width δ > 0. Let ${\overline{\mathscr{H}}}_{t_m}=({\overline{\mathscr{H}}}_{t_m}^1,{\overline{\mathscr{H}}}_{t_{m+h}}^2)$ be as in (32) where, for simplicity, we assume that δ is chosen such that m, h are integers satisfying 0 ≤ m ≤ m + h ≤ Tδ⁻¹ − 1 and t_{m + h} = t_m + τ. Recall that by definition,

$$\begin{array}{cc}\hfill P_1^i(t_m|{\overline{\mathscr{H}}}_{t_m}^i)& =P(X^i(t_m)=1|{\overline{\mathscr{H}}}_{t_m}^i),\hfill \\ \hfill P_{1,1}^{1,2}(t_m,t_{m+h}|{\overline{\mathscr{H}}}_{t_m})& =P(X^1(t_m)=1,X^2(t_{m+h})=1|{\overline{\mathscr{H}}}_{t_m}).\hfill \end{array}$$

Theorem 4. Consider the marked point process ℋ_T as in (35) with conditional intensity function satisfying (36). Let m, h be integers satisfying 0 ≤ m ≤ m + h ≤ Tδ⁻¹ − 1 and t_{m + h} = t_m + τ. Then under Conditions (III) and (IV), we have

$$P_{1,1}^{1,2}(t_m,t_{m+h}|{\overline{\mathscr{H}}}_{t_m})={\delta }^2[\gamma (t_m,t_{m+h})+1]P_1^1(t_m|{\overline{\mathscr{H}}}_{t_m}^1)P_1^2(t_{m+h}|{\overline{\mathscr{H}}}_{t_{m+h}}^2)+O({\delta }^3),$$

for sufficiently small δ.

The practical significance of Theorem 4 is that γ(t_m, t_{m + h}) does not depend on the spiking history of the 2 neurons. This implies that a statistic based on γ can be constructed to test the null hypothesis H₀ that there is no time-delayed spiking synchrony at lag τ between the 2 neurons.

7 Discussion

We have described an approach to assessing spike train synchrony using loglinear models for multiple binary time series. We tried to motivate the application of loglinear modeling technology in Section 1, emphasizing two features of individual neural response: stimulus-induced nonstationarity that remains time-locked across trials, and within-trial effects that are history-dependent, with timing that varies across trials. These were incorporated into the models by including for individual neurons both time-varying marginal effects, which stay the same across trials, and history-dependent terms; interaction terms were treated separately. In Section 3 we presented results for two pairs of neurons. For both pairs there was evidence of excess synchronous spiking beyond that explained by stimulus-induced changes in individual-neuron firing rates. In one pair, network activity, represented as history dependence, was sufficient to account for excess synchronous spiking, but the other pair displayed excess synchronous spiking that remained highly statistically significant even after network effects were incorporated, indicating stimulus-related synchrony. Our theoretical results provided a continuous-time point process foundation for the methods, justifying both our use of smoothing and our derivation of the excess-synchrony estimators ζ̂ and ζ̂_H.

Assessment of synchrony via continuous-time loglinear models is closely related to the unitary-event analysis of Grün, et al. (2002a), (2002b). Unitary event analysis assumes each neuron follows a locally-stationary Poisson processes, which has been shown to be somewhat conservative in the sense of providing inflated p-values in the presence of non-Poisson history dependence. Its main purpose is to identify stimulus-locked excess synchrony. Because the loglinear models could be viewed as generalizations of locally-stationary Poisson models, they could extend unitary-event analysis to cases in which it seems desirable to account more explicitly for stimulus and history effects. This is a topic for future research.

We also provided an example of testing for 3-way interaction. The results we gave in Section 3 for a particular triple of neurons indicated no evidence of excess 3-way joint spiking above that explained by 2-way joint spiking. A systematic finding along these lines, examining large numbers of neurons, would be consistent with findings of Schneidman, et al. (2006). However, as may be seen from Figure 3, 3-way joint spikes are very sparse. A careful study of the power to detect 3-way joint spiking in contexts like the one considered here could be quite helpful. We plan to carry out such a study and report it elsewhere.

We have restricted history effects to individual neurons by assuming, first, that each neuron’s history excludes past spiking of the other neurons under consideration and, second, that the interaction effects are independent of history. This greatly simplifies the modeling and avoids confounding the interaction effects with cross-neuron effects. While it would be possible, in principle (by modifying the hierarchical sparsity condition), to allow history-dependence within interaction effects, we see no practical benefit of doing so. With the two additional, highly plausible assumptions used here we get both tractable discrete-time methods and a sense in which the methods may be understood in continuous time. It is somewhat inelegant to have a sequence of marked processes (indexed by δ) but this appears to be the best that can be achieved by starting with very natural discrete-time loglinear models. An alternative would be to use more standard point process models with short time-scale cross-neuron effects. Presumably similar results could be obtained, but the relationship between these different approaches is also a subject for future research. A quite different technology involves permutation-style assessment via “dithering” or “jittering” of individual spike times [cf. Geman, et al. (2010), Grün (2009)]. Synchrony is one of the deep topics in computational neuroscience and its statistical identification is subtle for many reasons, including inaccuracies in reconstruction of spike timing from the complicated mixture of neural signals picked up by the recording electrodes [e.g., Harris, et al. (2000), Ventura (2009)]. It is likely that multiple approaches will be needed to grapple with varying neurophsyiological circumstances.

Appendix

Proof of Theorem 1. For simplicity, we shall consider only the case for ν = 2. The proof for other values of ν is similar though more tedious. We observe from Proposition 1 that

$$\frac{P_{1,0}^{1,2}(t_m)}{\delta }=\frac{1}{\delta }{\displaystyle \sum _{k=0}^{2\lceil T/\theta \rceil }}{\displaystyle \sum _{i_1,\dots ,i_k\in \{1,2\}}}{\displaystyle {\int }_0^{t_m}}\dots {\displaystyle {\int }_{s_{k-1}}^{t_m}}{\displaystyle {\int }_{t_m}^{t_{m+1}}}\lambda (s_{k+1},(1)|\{(s_1,(i_1)),\dots ,(s_k,(i_k))\})\times [{\displaystyle \prod _{l=1}^k}\lambda (s_l,(i_l)|\{(s_1,(i_1)),\dots ,(s_{l-1},(i_{l-1}))\})]\times e^{-{\displaystyle {\sum }_{j=1}^2}{\displaystyle {\int }_0^{t_{m+1}}}\lambda (w,(j)|{\mathscr{H}}_w)\mathit{\text{dw}}}{\mathit{\text{ds}}}_{k+1}{\mathit{\text{ds}}}_k\dots {\mathit{\text{ds}}}_1.$$ (37)

Condition (I) implies that the summations Σ_kΣ_{i1, … , ik∈{1,2}} in (37) contain a finite number of summands. Hence letting δ → 0, the right hand side of (37) equals

$${\displaystyle \sum _{k=0}^{2\lceil T/\theta \rceil }}{\displaystyle \sum _{i_1,\dots ,i_k\in \{1,2\}}}\underset{\delta \to 0}{\text{lim}}\frac{1}{\delta }{\displaystyle {\int }_0^{t_m}}\dots {\displaystyle {\int }_{s_{k-1}}^{t_m}}{\displaystyle {\int }_{t_m}^{t_{m+1}}}\lambda (s_{k+1},(1)|\{(s_1,(i_1)),\dots ,(s_k,(i_k))\})\times [{\displaystyle \prod _{l=1}^k}\lambda (s_l,(i_l)|\{(s_1,(i_1)),\dots ,(s_{l-1},(i_{l-1}))\})]\times e^{-{\displaystyle {\sum }_{j=1}^2}{\displaystyle {\int }_0^{t_{m+1}}}\lambda (w,(j)|{\mathscr{H}}_w)\mathit{\text{dw}}}{\mathit{\text{ds}}}_{k+1}{\mathit{\text{ds}}}_k\dots {\mathit{\text{ds}}}_1.$$ (38)

Using Condition (II) and Taylor expansion, we have

$$\begin{array}{cc}\hfill & \lambda (s_{k+1},(1)|\{(s_1,(i_1)),\dots ,(s_k,(i_k))\})\hfill \\ =\hfill & \lambda (t_m,(1)|\{(s_1,(i_1)),\dots ,(s_k,(i_k))\})+O(\delta ),\hfill \end{array}$$

uniformly over t_m ≤ s_k+1 ≤ t_m+1, 0 ≤ s₁ ≤ ⋯ ≤ s_k ≤ t_m. Consequently (38) equals

$${\displaystyle \sum _{k=0}^{2\lceil T/\theta \rceil }}{\displaystyle \sum _{i_1,\dots ,i_k\in \{1,2\}}}\underset{\delta \to 0}{\text{lim}}\{{\displaystyle {\int }_0^{t_m}}\dots {\displaystyle {\int }_{s_{k-1}}^{t_m}}\lambda (t_m,(1)|\{(s_1,(i_1)),\dots ,(s_k,(i_k))\})\times [{\displaystyle \prod _{l=1}^k}\lambda (s_l,(i_l)|\{(s_1,(i_1)),\dots ,(s_{l-1},(i_{l-1}))\})]\times e^{-{\displaystyle {\sum }_{j=1}^2}{\displaystyle {\int }_0^{t_m}}\lambda (w,(j)|{\mathscr{H}}_w)\mathit{\text{dw}}}{\mathit{\text{ds}}}_{k+1}{\mathit{\text{ds}}}_k\dots {\mathit{\text{ds}}}_1+O(\delta )\}=E[\lambda (t,(1)|{\mathscr{H}}_t],$$

since t_m+1 − t_m = δ and lim_δ→0 t_m = t. Using a similar argument, we have

$$\frac{P_{1,1}^{1,2}(t_m)}{{\delta }^2}={\displaystyle \sum _{1\le {\ell }_1\ne {\ell }_2\le 2}\frac{1}{{\delta }^2}}{\displaystyle \sum _{k=0}^{2\lceil T/\theta \rceil }{\displaystyle \sum _{i_1,\dots ,i_k\in \{1,2\}}{\displaystyle {\int }_0^{t_m}\dots }}{\displaystyle {\int }_{s_{k-1}}^{t_m}{\displaystyle {\int }_{t_m}^{t_{m+1}}{\displaystyle {\int }_{s_{k+1}}^{t_{m+1}}\times }\lambda (s_{k+2},(2)|\{(s_1,(i_1)),\dots ,(s_k,(i_k)),(s_{k+1},(1))\})\times \lambda (s_{k+1},(1)|\{(s_1,(i_1)),\dots ,(s_k,(i_k))\})}}}\times [{\displaystyle \prod _{l=1}^k\lambda (s_l,(i_l)|\{(s_1,(i_1)),\dots ,(s_{l-1},(i_{l-1}))\})]\times e^{-{\displaystyle {\sum }_{j=1}^2{\displaystyle {\int }_0^{t_{m+1}}\lambda (w,(j)|{\mathscr{H}}_w)\mathit{\text{dw}}}}}{\mathit{\text{ds}}}_{k+2}{\mathit{\text{ds}}}_{k+1}{\mathit{\text{ds}}}_k\dots {\mathit{\text{ds}}}_1\to \frac{1}{2}\left\{E\right[\lambda (t,(2)|\{(t,(1))\}\cup {\mathscr{H}}_t)\lambda (t,(1)|{\mathscr{H}}_t)]+E[\lambda (t,(1)|\{(t,(2))\}\cup {\mathscr{H}}_t)\lambda (t,(2)|{\mathscr{H}}_t)\left]\right\}}$$

and ${\delta }^{-1}{P}_1^1(t_m)={\delta }^{-1}[P_{1,0}^{1,2}(t_m)+P_{1,1}^{1,2}(t_m)]\to E[\lambda (t,(1)|{\mathscr{H}}_t]$ as δ → 0.

Proof of Theorem 2. For simplicity, define

$$P_{a,b;c,d}^{1,2;1,2}(t_m,t_{m+h})=P\left(X^1(t_m)=a,X^2(t_m)=b,X^1(t_{m+h})=c,X^2(t_{m+h})=d\right),$$

for all a, b, c, d, ∈ {0, 1}. We observe from Proposition 1 that

$${\delta }^{-2}{P}_{1,0;0;1}^{1,2;1,2}(t_m,t_{m+h})=\frac{1}{{\delta }^2}{\displaystyle \sum _{j=0}^{2\lceil T/\delta \rceil }}{\displaystyle \sum _{k=0}^{2\lceil T/\theta \rceil }}{\displaystyle \sum _{i_1,\dots ,i_j,i_{j+2},\dots ,i_{j+k+1}\in \{1,2\}}}{\displaystyle {\int }_0^{t_m}}\cdots {\displaystyle {\int }_{s_{j-1}}^{t_m}}{\displaystyle {\int }_{t_m}^{t_{m+1}}}{\displaystyle {\int }_{s_{j+1}}^{t_{m+h}}}\cdots {\displaystyle {\int }_{s_{j+k}}^{t_{m+h}}}{\displaystyle {\int }_{t_{m+h}}^{t_{m+h+1}}}\times \lambda \left(s_{j+k+2},(2)|\{(s_1,(i_1)),\dots ,(s_j,(i_j)),(s_{j+1},(1)),(s_{j+2},(i_{j+2})),\dots ,(s_{j+k+1},(i_{j+k+1}))\}\right)\times \left[{\displaystyle \prod _{l=j+2}^{j+k+1}}\lambda \left(s_l,(i_l)|\left\{(s_1,(i_1)),\dots ,(s_j,(i_j)),(s_{j+1},(1)),(s_{j+2},(i_{j+2})),\dots ,(s_{l-1},(i_{l-1}))\right\}\right)\right]\lambda \left(s_{j+1},(1)|\{(s_1,(i_1)),\dots ,(s_j,(i_j))\}\right)\times \left[{\displaystyle \prod _{l=1}^j}\lambda \left(s_l,(i_l)|\{(s_1,(i_1)),\dots ,(s_{l-1},(i_{l-1}))\}\right)\right]\times e^{-{\displaystyle {\sum }_{\ell =1}^2}{\displaystyle {\int }_0^{t_{m+1}}}\lambda (w,(\ell )|{\mathscr{H}}_w)dw}{\mathit{\text{ds}}}_{j+k+2}{\mathit{\text{ds}}}_{j+k+1}\dots {\mathit{\text{ds}}}_{j+2}{\mathit{\text{ds}}}_{j+1}{\mathit{\text{ds}}}_j\dots {\mathit{\text{ds}}}_1\to E\left[\lambda \left(t+\tau ,(2)|\{(t,(1))\}\cup {\mathscr{H}}_t\right)\lambda (t,(1)|{\mathscr{H}}_t)\right]$$

as δ → 0. Theorem 2 follows since $P_{1,1}^{1,2}(t_m,t_{m+h})\sim P_{1,0;0;1}^{1,2;1,2}(t_m,t_{m+h})$ as δ → 0.

Proof of Theorem 3. For simplicity, we only consider the case ν = 2. Let ${\overline{\mathscr{H}}}_{t_m}^i=\{t_{l_{i,1},\dots ,}{t}_{l_{i,k_i}}\}\text{ and }{\displaystyle {\cup }_{i=1}^2\{l_{i,1},\dots ,l_{i,k_i}\}}=\{l_1,\dots ,l_k\}$ where 0 ≤ l₁ < ⋯ < l_k ≤ m − 1. We observe from (31) that $P_1^1(t_m|{\overline{\mathscr{H}}}_{t_m})=P_{1,0}^{1,2}(t_m|{\overline{\mathscr{H}}}_{t_m})+O({\delta }^2)$, and

$$\begin{array}{cc}\frac{P_{1,0}^{1,2}(t_m|{\overline{\mathscr{H}}}_{t_m})P({\overline{\mathscr{H}}}_{t_m})}{\delta }\hfill & =\frac{1+O(\delta )}{\delta }{\displaystyle {\int }_{t_{l_1}}^{t_{l_1+1}}}\cdots {\displaystyle {\int }_{t_{l_k}}^{t_{l_k+1}}}{\displaystyle {\int }_{t_m}^{t_{m+1}}}{\lambda }^1(s_{k+1}|\{s_{l_{1,1}},\dots ,s_{l_{1,k_1}}\}){\mathit{\text{ds}}}_{k+1}\mathit{\text{dF}}(s_{l_1},\dots ,s_{l_k})\hfill \\ \hfill & ={\lambda }^1(t_m|{\overline{\mathscr{H}}}_{t_m}^1)P({\overline{\mathscr{H}}}_{t_m})+O(\delta )P({\overline{\mathscr{H}}}_{t_m}),\hfill \end{array}$$

for sufficiently small δ where F denotes the distribution function of s_l1, …, s_lk. Here s_l1 ∈ [t_l1, t_l1+1), s_l1,1 ∈ [t_l1,1, t_l1,1+1), etc. This proves the first statement of Theorem 3. Next we observe that

$$\begin{array}{cc}\frac{P_{1,1}^{1,2}(t_m|{\overline{\mathscr{H}}}_{t_m}P({\overline{\mathscr{H}}}_{t_m})}{{\delta }^2}\hfill & =\frac{1+O(\delta )}{{\delta }^2}{\displaystyle {\int }_{t_{l_1}}^{t_{l_1+1}}\cdots {\displaystyle {\int }_{t_{l_k}}^{t_{l_k+1}}{\displaystyle {\int }_{t_m}^{t_{m+1}}\delta \gamma (s_{k+1}){\mathrm{\lambda }}^1(s_{k+1}|\{s_{l_{1,1}},\dots ,s_{l_{1,k_1}}\})\times {\mathrm{\lambda }}^2(s_{k+1}|\{s_{l_{2,1}},\dots ,s_{l_2,k_2}\})d{s}_{k+1}d}F(s_{{{}^l}_1},\dots ,s_{l_k})+\frac{1+O(\delta )}{{\delta }^2}{\displaystyle {\int }_{t_{l_1}}^{t_{l_1+1}}\cdots }{\displaystyle {\int }_{t_{l_k}}^{t_{l_k+1}}{\displaystyle {\int }_{t_m}^{t_{m+1}}{\displaystyle {\int }_{t_m}^{t_{m+1}}{\mathrm{\lambda }}^1(s_{k+2}|\{s_{l_{1,1}},\dots ,s_{l_{1,k_1}}\})\times {\mathrm{\lambda }}^2(s_{k+1}|\{s_{l_{2,1}},\dots ,s_{l_{2,k_2}}\})d{s}_{k+2}d{s}_{k+1}dF(s_{l_1},\dots ,s_{l_k})}}}}}\hfill \\ & =[1+\gamma (t_m)]{\mathrm{\lambda }}^1(t_m|{\overline{\mathscr{H}}}_{t_m}^1){\mathrm{\lambda }}^2(t_m|{\overline{\mathscr{H}}}_{t_m}^2)P({\overline{\mathscr{H}}}_{t_m})+O(\delta )P({\overline{\mathscr{H}}}_{t_m}),\hfill \end{array}$$

for sufficiently small δ. This proves Theorem 3.

Proof of Theorem 4. We observe that ℋ_tm → ℋ̄_tm is a many-to-one mapping and from (36) that

$$P(X^1(t_m)=1,X^2(t_{m+h})=1|{\mathscr{H}}_{t_m})={\delta }^2\gamma (t_m,t_{m+h}){\lambda }^1(t_m|{\mathscr{H}}_{t_m}^1){\lambda }^2(t_{m+h}|{\mathscr{H}}_{t_{m+h}}^2)+{\delta }^2{\lambda }^1(t_m|{\mathscr{H}}_{t_m}^1){\lambda }^2(t_{m+h}|{\mathscr{H}}_{t_{m+h}}^2)+O({\delta }^3),$$ (39)

for sufficiently small δ. We further observe that

$$\begin{array}{ccc}\hfill P_1^1(t_m|{\mathscr{H}}_{t_m}^1)& =\hfill & P(X^1(t_m)=1,X^2(t_{m+h})=0|{\mathscr{H}}_{t_m}^1)+O({\delta }^2)\hfill \\ \hfill & =\hfill & \delta {\lambda }^1(t_m|{\mathscr{H}}_{t_m}^1)+O({\delta }^2)\hfill \\ \hfill & =\hfill & \delta {\lambda }^1(t_m|{\overline{\mathscr{H}}}_{t_m}^1)+O({\delta }^2),\hfill \\ \hfill P_1^2(t_{m+h}|{\mathscr{H}}_{t_{m+h}}^2)& =\hfill & P(X^1(t_m)=0,X^2(t_{m+h})=1|{\mathscr{H}}_{t_{m+h}}^2)+O({\delta }^2)\hfill \\ \hfill & =\hfill & \delta {\lambda }^2(t_{m+h}|{\mathscr{H}}_{t_{m+h}}^2)+O({\delta }^2)\hfill \\ \hfill & =\hfill & \delta {\lambda }^2(t_{m+h}|{\overline{\mathscr{H}}}_{t_{m+h}}^2)+O({\delta }^2).\hfill \end{array}$$ (40)

Thus it follows from (39) and (40) that

$$\begin{array}{cc}P\left(X^1(t_m)=1,X^2(t_{m+h})=1|{\overline{\mathscr{H}}}_{t_m}\right)\hfill & =E\left[P\left(X^1(t_m)=1,X^2(t_{m+h})=1|{\mathscr{H}}_{t_m}\right)|{\overline{\mathscr{H}}}_{t_m}\right]\hfill \\ \hfill & ={\delta }^2[\gamma (t_m,t_{m+h})+1]P_1^1(t_m|{\overline{\mathscr{H}}}_{t_m}^1)P_1^2(t_{m+h}|{\overline{\mathscr{H}}}_{t_{m+h}}^2)+O({\delta }^3),\hfill \end{array}$$

for sufficiently small δ. This proves Theorem 4.