Within-Subject Reaction Time Variability: Role of Cortical Networks and Underlying Neurophysiological Mechanisms

Abstract

Variations in reaction time are a ubiquitous characteristic of human behavior. Extensively documented, they have been successfully modeled using parameters of the subject or the task, but the neural basis of behavioral reaction time that varies within the same subject and the same task has been minimally studied. In this paper, we investigate behavioral reaction time variance using 28 datasets of direct cortical recordings in humans who engaged in four different types of simple sensory-motor reaction time tasks. Using a previously described technique that can identify the onset of population-level cortical activity and a novel functional connectivity algorithm described herein, we show that the cumulative latency difference of population-level neural activity across the task-related cortical network can explain up to 41% of the trial-by-trial variance in reaction time. Furthermore, we show that reaction time variance may primarily be due to the latencies in specific brain regions and demonstrate that behavioral latency variance is accumulated across the whole task-related cortical network. Our results suggest that population-level neural activity monotonically increases prior to movement execution, and that trial-by-trial changes in that increase are, in part, accounted for by inhibitory activity indexed by low-frequency oscillations. This pre-movement neural activity explains 19% of the measured variance in neural latencies in our data. Thus, our study provides a mechanistic explanation for a sizable fraction of behavioral reaction time when the subject’s task is the same from trial to trial.

Graphical Abstract

1. Introduction

A fundamental goal of cognitive neuroscience is to understand how thoughts translate into actions. For over a century, reaction time (RT) experiments have been used to probe the neural mechanisms underlying aspects of this process (Galton, 1890). In these experiments, subjects may be presented with a sensory stimulus (e.g., a sound or cross appearing on a screen) and instructed to respond with a motor action (e.g., a button press) as quickly as possible. Many studies have shown that reaction time varies from trial to trial (Hanes and Schall, 1996; Schall, 2001, 2003; Gold and Shadlen, 2007) and that this variation can be as large as the reaction time itself (Arieli et al., 1996). A rich body of literature has attributed the subject-to-subject variability in reaction time to factors such as age (Weiss, 1965; Fozard, 1994; Kramer et al., 1999), gender (Blough and Slavin, 1987), intelligence (Carlson et al., 1983), exercise (Kramer et al., 1999), and anxiety (Liotti et al., 1991). Other studies demonstrate that within-subject variability in reaction time can be accounted for by variations in task demands (e.g., deciding between different stimuli or response options) (Stone, 1960; Schall, 2003; Gerson et al., 2005; Gold and Shadlen, 2007; Dean et al., 2011). Some of those changes have been described by the popular drift-diffusion model (Edwards, 1965; Ratcliff, 2002; Ratcliff and McKoon, 2008; Bitzer et al., 2014; Kubanek et al., 2013), which relates the increase in reaction time to increases in task difficulty in simple two-choice decision tasks.

1.1. Reaction Time Variance in Tasks that do not Require a Decision

The neural basis for reaction time variance is much less clear for behaviors where the subject does not have to make a decision, i.e., when the stimulus and the subject’s response are the same across trials (Arieli et al., 1996; Schall, 2003; Gold and Shadlen, 2007). Earlier work has treated RT variance as random additive biological noise (Dean et al., 2011), but increasing evidence points to physiological sources (Hanes and Schall, 1996; Lakatos et al., 2008; Jensen and Mazaheri, 2010; Coon et al., 2016; Lee et al., 2016). At the same time, prior research has had difficulty explaining more than a very small fraction of RT variance (e.g., Womelsdorf et al., 2006). Several studies have provided theoretical evidence that RT variance may be due to latency variations in different areas of the brain, e.g., areas involved in sensory perception, sensory-motor transformation, or motor execution (Sternberg, 1969; Ulrich and Stapf, 1984; Rizzolatti and Luppino, 2001; Schall, 2003), and there is increasing recognition that behaviors, in general, result from neuronal population activity across large-scale cortical networks and not just from the neural activity at individual locations (Bressler and Menon, 2010).

In our study, we sought to shed experimental and mechanistic light on this important question by linking variations in behavioral reaction time to variations in the onset of neural activity, and then identifying neural markers that predicted those neural activity onset variations. To do this, we recorded 28 datasets of electrocorticographic (ECoG) activity in 12 human subjects (S1–S12) who engaged in four different reaction time tasks labeled Attention, Auditory, Tactile, and Visual. Each of the tasks required the subject to rapidly respond to a sensory stimulus with a button press. In each trial, we evaluated the modulation of neural activity in the broadband gamma (70–170 Hz) range. Broadband gamma activity is widely recognized as a direct reflection of the population-level activity of neurons directly underneath the electrode (Miller et al., 2009; Whittingstall and Logothetis, 2009; Manning et al., 2009; Ray and Maunsell, 2011). Although it may have a somewhat more complex origin (Leszczynski et al., 2019), broadband gamma modulation has been documented during motor behavior (Crone et al., 1998; Miller et al., 2007), language production (Pei et al., 2011), vibrotactile stimulation (Wahnoun et al., 2015), visual perception (Ray and Maunsell, 2011), auditory function (Potes et al., 2014), and attention (Gunduz et al., 2012). Moreover, studies have shown that the neuronal processes that translate sensory stimuli to motor outputs manifest themself as an ordered sequence of broadband gamma burst across different areas of cortex (Coon et al., 2016).

To investigate network behavior, we detected the cortical locations at which broadband gamma activity increased during reaction time tasks and established the timing and sequence of that activation. To link the timing of population-level activity to the timing of behavior, we measured the cumulative broadband onset latency differences across the task-related cortical network and related it to behavioral reaction time.

1.2. Physiological Mechanisms Explaining Latencies

In addition to investigating the relationship between the onset times of neuronal populations with reaction time, we also sought to determine its generating physiological mechanisms. The magnitude and variability of reaction time are still somewhat unexplained, possibly because they cannot be adequately accounted for by transduction or neural transmission delays (Sabatini and Regehr, 1996; Schall, 2001) or by communication through different neural networks (Coon et al., 2016). A possible mechanism can be derived from the observation that behavior is often produced only with a certain level of activity in a local population of neurons (Hanes and Schall, 1996; Gold and Shadlen, 2007; Coon and Schalk, 2016). Thus, the time to ramp up to a threshold of activity may vary from trial to trial. Indeed, prior research found evidence that the firing rate of individual neurons can progressively increase prior to behavioral onset (Hanes and Schall, 1996; Chen and Hallet, 1999), that these firing rate modulations can precede behavior by hundreds of milliseconds (Luschei et al., 1968; Hatsopoulos and Suminski, 2011), and that faster increases in that rate can be linked to faster initiation of behavior (Hanes and Schall, 1996; Roitman and Shadlen, 2002).

We hypothesize that the rate of increase in subthreshold population-level activity is a general determinant of the latency of the above threshold population responses that, in turn, generate a motor response. Because population-level activity is determined not only by excitatory but also by inhibitory inputs, we also hypothesize that this rate increase can be explained in part by variations in cortical excitability induced by low-frequency oscillations. Low-frequency oscillatory activity, particularly in the alpha (8–12 Hz) frequency band, has been linked to rhythmic variations in cortical excitability (Fries, 2005; Lakatos et al., 2008; Haegens et al., 2011; Bollimunta et al., 2011; Spaak et al., 2012; Coon et al., 2016; Schalk et al., 2017; Peterson and Voytek, 2017) and has been suggested to provide a general mechanism that underlies dynamic changes in brain function (Fries, 2005; Jensen and Mazaheri, 2010; Schalk, 2015). Thus, a transient state of low excitability of a particular neuronal population could delay the time that this population can be excited by afferent input (e.g., from lower-level sensory areas in our reaction-time tasks), and hence delay the time of excitation of all neuronal populations connected to it.

2. Materials and Methods

2.1. Subjects

Twelve human subjects (S1–S12) participated in this study. They included six males (S3, S4, S6, S7, S10, and S12) and six females (S1, S2, S5, S8, S9, and S11), with ages ranging from 24 to 57 years. All subjects were patients with intractable epilepsy at Albany Medical College who underwent temporary placement of subdural electrode arrays to localize seizure foci prior to surgical resection. They provided informed consent to participate in the study, which was approved by the Institutional Review Board of Albany Medical College.

The implanted electrode grids were approved for human use (Ad-Tech Medical Corp., Racine, WI; and PMT Corp., Chanhassen, MN). They consisted of platinum-iridium discs that had a diameter of 4 mm (2.3–3.0 mm exposed), were embedded in silicone, spaced 6–10 mm apart, and implanted subdurally over the left hemisphere in seven subjects (S1, S2, S3, S5, S7, S10, and S11) and the right hemisphere in five subjects (S4, S6, S8, S9, and S12). Following the placement of the grids, each subject had postoperative computer tomography (CT) scans that were co-registered with magnetic resonance (MR) images. CT images gave the location of the electrodes; MR images gave three-dimensional subject-specific cortical models (Coon et al., 2016).

2.2. Data Collection

We recorded ECoG signals from the subjects at their bedside using the general-purpose BCI2000 software (Schalk et al., 2004). BCI2000 interfaced with eight 16-channel g.USBamp biosignal acquisition devices or one g.HIamp biosignal acquisition device (g.tec., Graz, Austria) to amplify and digitize the signals at a sampling rate of 1200 Hz. To ensure uninterrupted clinical monitoring during the experiments, a splitter box connected the signals from the patient to the clinical monitoring system and, simultaneously, to the g.tec biosignal acquisition device(s). The location of reference and ground varied across subjects. Specifically, electrodes distant from predicted epileptic foci and task-related areas were selected for reference and ground. We used visual inspection to identify channels that were contaminated with noise. This procedure removed 98 channels (0–24 in each subject, 7 on average) from a total of 1327; the remaining 1229 (51–136 in each subject, 88 on average) were submitted to further analysis.

2.3. Experimental Tasks

The Attention task (Supplementary Fig. 1A), performed by subjects S1–S4, implemented a modified Posner task that was originally designed to study visual-spatial attention. A full description of the task can be found in Gunduz et al. (2012). In brief, at the start of each trial, a cross and an arrow were presented at the center of the screen. After 2 s, a cosine grating (the cue) appeared on the screen at the location indicated by the arrow. At a random interval (1.5–2.5 s) after the onset of the cue presentation, the cosine grating changed its contrast. We here refer to this event as the stimulus, and we focused on the 1.5 s periods before, and after stimulus onset (“baseline” and “task” periods, respectively).

During the Auditory (Supplementary Fig. 1B), Tactile (Supplementary Fig. 1C), and Visual (Supplementary Fig. 1D) tasks, the subjects (S5–S8) were presented with a salient sensory stimulus (a 880 Hz tone, vibration of a vibrotactile stimulator placed on the chin contralaterally to the implant, or a circle on the screen, respectively). At the start of each trial, a cross appeared at the center of the screen. After 1.5 s, a cue appeared on the screen (an ear symbol for the auditory stimulus; a hand symbol for the tactile stimulus; a symbol containing a pair of eyes for the visual stimulus; or a depiction of the combination of the three symbols (catch trials)). The stimulus was presented at a random interval (1–3 s) after the cue. We focused on the 1 s periods before, and the stimulus onset and labeled those periods as “baseline” and “task,” respectively.

In all tasks, the subjects were asked to rapidly respond to the stimulus with a button press. After the subject’s response, we provided verbal feedback (Attention task) or visual feedback (Auditory, Tactile, and Visual tasks) about the subject’s performance to facilitate behavioral compliance. Together, the data that are reported in this paper are based on a total of 28 ECoG datasets (one dataset per subject and task).

Because we were interested in studying the neural basis of reaction time, we only included those trials that resulted in a button press (134–580 trials, 280 trials on average). In each trial, we measured the reaction time (RT) as the time that elapsed between the stimulus and the button press. In line with previous findings (Arieli et al., 1996; Hanes and Schall, 1996; Schall, 2001, 2003; Gold and Shadlen, 2007), we observed a large variance in reaction time, both within and across tasks (911 ± 364 ms (Attention), 265 ± 111 ms (Auditory), 263 ± 116 ms (Tactile), and 34 ± 175 ms (Visual)). Reaction time mean and standard deviation for each subject and task are presented in Supplementary Table 1.

2.4. Signal Processing

To remove slow signal drifts allowed by the DC-coupled amplifiers, we first applied a high-pass filter at 0.5 Hz (zero-phase 2^nd order Butterworth filter). We then re-referenced the signals to a common average reference (CAR) montage (Brunner et al., 2009; Liu et al., 2015), and downsampled the signals to 400 Hz after applying an antialiasing FIR lowpass filter with delay compensation.

2.4.1. Alpha power

To extract alpha oscillations, we bandpass-filtered (zero-phase 3^rd order Butterworth filter) the downsampled signals between 8 and 12 Hz. To extract alpha power, we computed the magnitude of the Hilbert transform of alpha oscillations. To allow for comparisons across electrode locations and subjects, we normalized alpha power at each location using the z-score transformation ($z=\frac{x-\mu }{\sigma }$, where x is the signal, μ the mean, and σ the standard deviation).

2.4.2. Broadband gamma amplitude

To extract broadband gamma amplitude, we bandpass-filtered (zero-phase 3^rd order Butterworth filter) the downsampled signals between 70 and 170 Hz. We then computed the magnitude of the Hilbert transform of the filtered signals. We normalized broadband amplitude measurements at each location using the z-score transformation.

2.4.3. NEO-transformed broadband gamma amplitude

For the detection of task-related locations and onset times only, we processed broadband gamma amplitude using a nonlinear energy operator (NEO) (Kaiser, 1990). NEO enhances signals where both amplitude and frequency are higher than for background noise, thus enhancing neural signals against biological and systematic noise. It is commonly used in detection of single-neuron spiking activity (Gibson et al., 2008; Eftekhar et al., 2010), EMG onset detection (Solnik et al., 2010), and EEG classification (Kaleem et al., 2010), and has shown to also be useful for ECoG processing in preliminary testing.

The nonlinear energy operator (NEO) can be expressed as $y_n=x_n^2-x_{n+1}\cdot x_{n-1}$, where y_n is the output signal, x_n the input signal and n the sample point (Kaiser, 1990). Finally, we normalized the NEO broadband amplitude measurements at each location using the z-score transformation.

2.5. Electrode Selection

We determined the cortical locations whose NEO-processed broadband gamma activity was significantly modulated by the task. To do this, for each location, we computed the median broadband amplitude time course of the task (Median_task) and baseline (Median_baseline) periods, as in Fig. 1C. Then, for each location, we calculated its signal-to-noise ratio (SNR) as $SNR=\frac{\mathit{\text{var}}\left({\mathit{\text{Median}}}_{\mathit{\text{task}}}\right)}{\mathit{\text{var}}({\mathit{\text{Median}}}_{\mathit{\text{baseline}}})}$. To identify locations with statistically significant SNR values, we applied a bootstrap randomization test (Efron and Tibshirani, 1993) in which we randomly scrambled the task and baseline labels from all locations 10,000 times, and computed one random SNR value for each such iteration. We then calculated the p-value for each electrode location as the fraction of randomized SNR values that were larger than the computed SNR. Finally, we labeled all those electrode locations as “task-related” whose p-value was smaller than 0.05. This procedure identified 5–9 task-related locations for the Attention task, 4–12 for the Auditory task, 4–13 for the Tactile task and 4–9 for the Visual task, across the different subjects (Supplementary Figs. 2, 3, and 4).

Figure 1: Detection of onset times.

(A) Electrode locations of Subject S1 (small grey circles) on the three-dimensional cortical template provided by the Montreal Neurological Institute (MNI brain model). Stars mark locations at which broadband gamma amplitude significantly increased during the Attention task. (B) Time course of broadband gamma in a single trial for the motor cortical location marked by the green star in (A). A black arrow indicates the onset time (OnT) that was detected by our algorithm in that trial. The dashed vertical line indicates the time of stimulus delivery (time 0). (C) Time course of broadband gamma for the same location as in (B), but averaged across trials. The grey histogram shows the distribution of onset times for all trials. The large variance in onset times indicated by the histogram suggests that the relatively broad peak in the green average broadband gamma trace is mainly due to variations in single-trial reaction times. (D) Averaged broadband gamma trace for locations 1–5. The dotted vertical line annotates the median reaction time (RT). In contrast to panel C, data are time-aligned to the onset of broadband gamma at the respective location and each trial, which sharpens the average broadband response and reveals a clear temporal progression of broadband activity across locations, from sensory to motor cortices.

2.6. Onset Time Detection

For all task-related locations, we then identified the time of onset of broadband gamma in each trial using a modified version of the detection algorithm described in full in Coon and Schalk (2016), using the NEO-processed broadband amplitude. We searched for the onset time in the entire task period (not time-limited by reaction time).

We first computed an optimal detection threshold for each location. To do this, we varied the detection threshold between 1 and 6 z-scores (with 0.1 increment); for each varying threshold level, we computed the number of trials in which broadband amplitude crossed the threshold at least once during the task period (N_task) and during the baseline period (N_baseline). The threshold that maximized the difference between N_task minus N_baseline became the optimal (location-specific) detection threshold. Given this threshold, the time of onset of broadband amplitude was then determined by identifying the time of the first peak in broadband gamma following the first threshold crossing during the task period. In 14.96% of all trials, broadband amplitude did not exceed the threshold value during the task period; hence, no onset time was detected using this procedure.

To identify a time of onset even for this subset of 14.96% of trials, we defined a trial-optimized threshold. For each trial, the broadband amplitude in the baseline and task periods was converted into two binary sequences x_baseline and x_task. The x_baseline and x_task value at each data point could be either 1 or 0, depending on whether the broadband amplitude was above or below the varying threshold level. We then defined the optimal threshold as the one that minimized the p-value of a Wilcoxon rank-sum test computed between x_baseline and x_task, with the constraint that sum(x_task) > sum(x_baseline). Just as before, the onset time was the time of the first peak after the first threshold crossing during the task period. In 1.74% of all trials, no sample exceeded this threshold, and no onset was detected. To define a time of onset even for those trials, we defined the onset time as the time of the maximum of the broadband amplitude signal during the task period.

Finally, the onset time for each trial was time-shifted to the nearest peak of the broadband gamma amplitude in the non-NEO processed data.

2.7. Functional Connectivity Algorithm

Based on the times of onset of broadband gamma activity in each trial and each task-related location, we then identified the functional connectivity between the task-related locations. We implemented an algorithm inspired by techniques for natural language processing (such as n-gram (Brown et al., 1992) and skip-gram (Mikolov et al., 2013)), which determine the probability that pairs of words appear in a text. Unlike other popular methods such as Granger Causality (Bullmore and Sporns, 2009; He et al., 2011; Seth et al., 2015), our algorithm allows for the variability of latencies between trials that have been demonstrated in ECoG data in our prior study (Coon and Schalk, 2016). For each location, the algorithm identifies the successive location in the trajectory of activation in seven steps:

1. Given all task-related locations E_i, we determined all possible connections E_i → E_j between them. For example, for three task-related locations all possible connections would be: E₁ → E₂, E₁ → E₃, E₂ → E₁, E₂ → E₃, E₃ → E₁, E₃ → E₂.

2. Based on the time of onset of a particular E_i, we then determined, in each trial, the sequence of activation of all E_i. For example, for three task-related locations the activation could be E₁ → E₂ → E₃ in one trial, or E₃ → E₁ → E₃ in another trial, or assume any directional combination of the three locations.

3. For each trial and each possible connection E_i → E_j, we calculated the corresponding trial transition probability $\begin{array}{c}{P}_{\mathit{\text{trial}}}\\ E_i\to E_j\end{array}$ as 1/n⁴, where n + 1 is the number of locations in the sequence between E_i and E_j. For example, in a trial where the activation sequence is E₁ → E₂ → E₃, the transitions probabilities $\begin{array}{c}{P}_{\mathit{\text{trial}}}\\ E_1\to E_2\end{array}$ and $\begin{array}{c}{P}_{\mathit{\text{trial}}}\\ E_1\to E_3\end{array}$ would be 1 and 1/2⁴, respectively.

4. The overall transition probability, considering all trials, is $\underset{E_i\to E_j}{P}=\frac{1}{m}\cdot \sum _1^m\underset{E_i\to E_j}{P_{\mathit{\text{trial}}}}$, where m is the number of trials.

5. We then sought to eliminate all connections that were not statistically significant. To do this, we applied a bootstrap randomization test in which we computed the random connection probability for each connection E_i → E_j using 10,000 random activation sequences. We discarded all connections E_i → E_j with a probability less than the probability computed by the randomization algorithm.

6. For each location pair E_i and E_j, we identified the most likely direction of information propagation (i.e., E_i → E_j or E_j → E_i). To do this, we compared the transitions probabilities $\underset{E_i\to E_j}{P}$ and $\underset{E_j\to E_i}{P}$. If $\underset{E_i\to E_j}{P}$ was larger than $\underset{E_j\to E_i}{P}$, we did not consider the direction E_j → E_i in the following step. Likewise, if $\underset{E_i\to E_j}{P}$ was smaller than $\underset{E_j\to E_i}{P}$, we did not consider the direction E_i → E_j in the following step. In case $\underset{E_i\to E_j}{P}$ and $\underset{E_j\to E_i}{P}$ were equal, we computed the median onset time of locations E_i and E_j. We discarded the direction that was originating from the location with the larger median onset time.

7. Finally, we selected the directional location pairs to form the activation trajectory. Each location was paired with the location that showed the highest transition probability.

Supplementary Fig. 5 shows the computed transition probabilities for all task-related electrode pairs for Subjects S1 (Attention task), S10 (Auditory task), and S12 (Auditory task). The transition probabilities for each possible directional location pair are represented by grayscale (darker gray shows the highest probability). For each source location, we selected the most likely destination location pair according to the highest transition probability and annotated this with a colored square. The selection of highest transition probability results in the formation of simple trajectories (such as those shown in Fig. 2) or more complex trajectories that may suggest some degree of parallel processing (Mesulam 1990; Bressler 1995; Supplementary Fig. 6).

Figure 2: Example of the trajectory of population-level cortical activity and latencies between locations.

(A) Three-dimensional model of the cortical surface for subject S 10. Electrode locations are indicated with small grey circles. Locations whose broadband gamma amplitude increased during the Auditory task are marked by stars; arrows show the trajectory of that activation. (B) Single-trial broadband gamma traces and onset times (black dots) for all task-related locations. (C) Mean and standard error for transmission times (TTs) between the locations shown in A. The average cumulative cortical transmission time (mean CTT) approximates this subject’s average reaction time (mean RT). (D) In individual trials, the cumulative cortical transmission time CTT strongly predicts behavioral reaction time RT (Spearman’s correlation r=0.64).

2.8. Network Contribution to Reaction Times

2.8.1. Effect of cortical network coverage

Throughout this paper, we refer to the single-trial latency differences between locations as transmission times (TTs), the sum of these latency differences across all cortical locations as the cortical transmission times (CTTs), and the latency of the button press as reaction time (RT) (Fig. 2). To calculate the effect of the number of network connections (i.e., a pair of task-related locations) included in the computation of the correlation between CTTs and RTs, we separated, for each subject and task, the location pairs in four quartiles. To do this, we followed these five steps:

1. We identified the cortical locations in the activation trajectory that only serve as destination locations (i.e., locations to which activity only propagates to but does not originate from). We will refer to these locations as final destination locations. For example, for an activation trajectory with connected location pairs E₁ → E₃ and E₂ → E₃, the final destination location was E₃.

2. For each of the final destinations’ locations, we identified the cortical locations from which activity originates (i.e., source locations). For the same example activation trajectory as in 1, the source locations are E₁ and E₂.

3. We then relabeled the source locations into destination locations and identified the source locations for the relabeled destination locations. We repeated this step until all the network locations were included.

4. At each step, we calculated the percentage of network covered (${\mathit{\text{Net}}}_{\mathit{\text{prc}}}=100\cdot \frac{N^o\mathit{\text{IncludedConnections}}}{N^o\mathit{\text{TotalConnections}}}$, and the percentage of correlation between CTTs and RTs ($r_{prc}=100\cdot \frac{r\mathit{\text{NetCovered}}}{r\mathit{\text{NetTotal}}}$). To compute CTTs, we accumulated latencies only across the covered network.

5. Finally, we grouped Net_prc and r_prc values according to Net_prc’s quartile (i.e., 0–25%, 25–50%, 50–75%, or 75–100%).

2.8.2. Effect of electrode placement

To quantify the effect of the placement of electrode grids on the ability to predict reaction times, we defined two concepts: temporal and spatial coverage. We defined temporal coverage for each subject as the standard deviation of the median onset times (one median onset time computed per task-related location). We defined spatial coverage for each subject as $\frac{1}{N}\cdot \sum _1^N{\left(x_i-\mu \right)}^2$, where x_i were the coordinates of location i, and N is the number of task-related electrodes. We excluded locations in prefrontal cortex from this calculation (10% of task-related electrodes), as we sought to only focus on the perception to execution aspect of the task.

3. Results

3.1. Detection of Cortical Activation

The task-related locations (i.e., the cortical locations at which the task significantly modulated population-level activity) were widely distributed across the cortex — from sensory to motor cortices. See Fig. 1A for an example of the task-related locations for Subject S1 executing the Attention task. For each of these cortical locations, we then identified the onset time (OnT) of broadband gamma activity (i.e., the first time at which population-level activity increased above baseline during the task) in individuals trials. In Fig. 1A, locations are ranked from 1–5 according to their respective median onset time. Fig. 1B depicts population-level broadband gamma amplitude at the motor cortical location marked by a green star in (A) and the onset time OnT in a single trial. Fig. 1C shows the average of that activity and a histogram of onset times across trials (grey bars). The duration of cortical activity in individual trials is very brief, and the broad shape of the average cortical activity is accounted for mostly by the large variability in trial-by-trial onset times: the standard deviation of onset times is large (161 ± 87 ms) and increases with increasing mean onset time (r² = 0.7, Supplementary Fig. 7A). Fig. 1D shows the time courses of average broadband gamma activity for each of these five locations, aligned to the time of activity onset in each trial. As could be expected in this visuomotor task, the spatial location and onset timing of broadband gamma activity suggest an ordered progression of cortical activity from visual to motor cortices.

3.2. Accumulated Cortical Latencies Explain Reaction Time Variance

In the previous section, we determined the times of onset of population-level activity at different locations and for each trial. In this section, we proceed to determine whether the CTT (i.e., the sum of the latency differences between successively activated cortical locations) can explain a substantial fraction of trial-by-trial reaction time variance. To do this, we first determined which locations were successively activated, i.e., the trajectory of activation. The output of this analysis is shown in Fig. 2 for Subject S10 executing the Auditory task. Fig. 2A highlights the four task-related locations for which the Auditory task modulated population-level activity. Each arrow suggests that cortical activation proceeds from one location (source) to another (destination); together, the arrows indicate the cortical trajectory of that activation. Similar to Fig. 1, the trajectory of activity begins in sensory cortex and ends in motor cortex. For some subjects/tasks, the trajectory was simply a linear sequence of locations, whereas, for others, it was more complex (see Supplementary Fig. 6 for an example).

We then computed the broadband latency differences between the task-related locations by subtracting the onset time of the source location from the onset time of the destination location in each trial. This yielded one transmission time (TT) per source/destination location pair, as shown in Fig. 2B. Accumulation of these TTs results in the CTT. Fig. 2C shows the averages of TT, CTT, and RT, calculated across trials. Most notable, in the context of our primary hypothesis, the average cumulative cortical transmission time CTT (297.10 ms, Fig. 2C) closely approximates the average RT (297.25 ms) in this example.

CTT also strongly correlated with RT across all trials (Spearman’s correlation r = 0.64, Fig. 2D), which indicates that onset differences of population-level activity across large areas of the brain predict a large fraction of reaction time variance. The results presented in Table 1 document that this finding generalizes to all subjects and all tasks. The magnitude of the correlation varied across subjects and tasks (r=0.13–0.64, mean r=0.32), and was statistically significant (p ≤ 0.05) in all cases. To correct for multiple comparisons, the critical p-value was adjusted from 0.05 to 0.0388 by using the False Discovery Rate (FDR) method (Benjamini and Hochberg, 1995).

Table 1: Cumulative cortical transmission times explain reaction times.

Spearman’s correlation (r) between reaction times (RTs) and cortical transmission times (CTTs) for each subject/task, and the p-value (p) of that relationship.

	TASKS
	Attention		Auditory		Tactile		Visual
Subject	r	P	r	P	r	P	r	P
S1	0.59	<0.0001	-	-	-	-	-	-
S2	0.51	<0.0001	-	-	-	-	-	-
S3	0.35	<0.0001	-	-	-	-	-	-
S4	0.17	0.0390	-	-	-	-	-	-
S5	-	-	0.39	<0.0001	0.37	<0.0001	0.49	<0.0001
S6	-	-	0.15	0.0003	0.24	<0.0001	0.13	0.0010
S7	-	-	0.34	<0.0001	0.25	<0.0001	0.41	<0.0001
S8	-	-	0.52	<0.0001	0.31	<0.0001	0.13	0.0100
S9	-	-	0.27	0.0001	0.20	0.0047	0.16	0.0020
S10	-	-	0.64	<0.0001	0.52	<0.0001	0.24	0.0010
S11	-	-	0.27	0.0001	0.17	0.0140	0.18	0.0110
S12	-	-	0.43	<0.0001	0.30	0.0001	0.16	0.0400

3.3. Reaction Time is a Function of the Whole Cortical Network

The neural basis for variations in reaction time cannot be attributed to a single location/location pair. Fig. 3A illustrates (for the same dataset shown in Fig. 2) that the correlation between CTTs and RTs greatly diminishes when we only consider TT3 (r = 0.11, left bar) or TT2 and TT3 together (r = 0.28, middle bar) compared to when we consider latency differences throughout the network (TT1–TT3) (r = 0.64, right bar). This finding generalizes to all subjects and tasks (Fig. 3B). Location pairs are separately grouped in four quartiles (from smallest to largest network coverage) for each subject. The bars show the mean and standard error of the correlation, normalized by the correlation computed from the whole task-related network.

Figure 3: Reaction time cannot be attributed to a single location.

(A) For Subject S 10 and the Auditory task, Spearman’s correlation between accumulated transmission times TTs (i.e., cortical transmission times (CTTs)) and reaction times (RTs) increases with inclusion of increasing number of pairs of functionally connected locations. (B) This finding generalizes to all subjects and tasks: increasing the number of location pairs (given in % of all location pairs in each subject and task) in the computation improves the Spearman’s correlation (given in % of the total correlation in each subject and task) between the cortical transmission times (CTTs) and reaction time (RT).

3.4. Subthreshold Population-Level Activity Predicts Onset Time

After showing the trial-by-trial variations in the onset of population-level activity that predict variations in reaction time, we sought to determine a physiological origin. We previously hypothesized that a certain level of population-level activity is necessary to produce behavior and that the time to reach this threshold activity may be variable. The results presented in Fig. 4 show evidence that this is the case. Panel A shows the time course of broadband gamma, averaged across all subjects and locations, for each reaction time quartile. Population-level activity (measured by broadband gamma) increases prior to the onset of the full population response (OnT, shown here as time 0), and the rate of increase is fastest for the trials with the fastest reaction time. Panel B quantifies this relationship between pre-onset broadband amplitude and OnT. This relationship is best when we consider broadband gamma amplitude during the 300 ms prior to OnT. Together, these data suggest that behavior is indeed contingent on a specific level of population-level activity and that, hence, the time to reach this threshold influences the timing of behavior.

Figure 4: Broadband gamma amplitude prior to onset time varies with onset time.

(A) The time course of broadband gamma, averaged across all task-related locations, subjects, and tasks. For each location, trials are grouped into four quartiles from fastest (dark green) to slowest (dark red trace), and broadband gamma is aligned to onset time (time 0). (B) Mean and standard error of Pearson’s correlation (r) between onset time and average pre-onset broadband gamma for different time periods. Broadband gamma during the 300 ms prior to onset time yields the highest correlation and is highlighted in red.

The activity of a local cortical population is determined not only by afferent input (e.g., from lower-level sensory areas), but also by rhythmic inhibition from low-frequency oscillations, primarily in the alpha (8–12 Hz) band. Thus, we hypothesized that broadband gamma amplitude just prior to onset of the population-level response is modulated by alpha activity. Our results suggest that this is the case: the relationship of broadband activity with onset time depends on pre-onset alpha activity (Fig. 5A), and this dependency is strongest for the 75 ms period prior to onset. Ancillary analyses demonstrate that broadband gamma for the 300–75 ms period is considered sufficiently independent from the 75–0 ms period (see Discussion). Thus, we here consider these two neural components (broadband activity in the 75–0 ms and the 300–75 ms periods) as f1 and f2, respectively.

Figure 5: Physiological phenomena explain variable latency in population activation time.

(A) Mean and standard error (across all task-related locations, subjects and tasks) of Pearson’s correlation between f₁ (average broadband gamma power 75 ms prior to onset time) and onset time, separated for high and low alpha power (statistically significantly different, Wilcoxon rank-sum test, ****: p ≪ 0.0001). The correlations for high and low alpha are also statistically significantly different (with considerably smaller difference: *: p < 0.05) when evaluating the broadband gamma power 150 to 75 ms prior to onset time. (B) Mean and standard error (across all task-related locations, subjects and tasks) of Pearson’s correlation between onset time and features f₁, f₂, and the linear combination of f₁ and f₂. (C) Averaged broadband amplitude for 25% fastest (dashed green trace) and slowest (red trace) trials, for two groups of channels in Subject S7. Cortical locations are separated to 25% earlier and later activated (according to mean onset time in each location). Each location group is aligned to the mean onset time of the slowest trials, and the black arrows indicate the average onset time of the fastest trials. The stimulus occurs at time 0.

These two components of pre-onset population-level activity strongly predict the onset time (OnT) of a local population in single trials: across all locations, subjects, and tasks, the correlation r of f1 with OnT is 0.26, of f2 with OnT is 0.35, and of f1 and f2 together with OnT is 0.41 (Fig. 5B). For these three cases, the correlations were statistically significant (p < 0.0388) for 86%, 93%, and 94% of all task-related locations, respectively. To correct for multiple comparisons, the critical p-value was adjusted from 0.05 to 0.0388 by using the FDR method. For more information on these correlation and their statistical significance for each subject and task, see Supplementary Tables 2 and 3.

Furthermore, our data suggest that population-level activity begins to increase at approximately the same time in fast and in slow trials: Fig. 5C shows the difference in broadband gamma amplitude prior to onset time between the 25% fastest and slowest trials (green and red traces, respectively) for both the 25% earliest and latest responding cortical locations (bottom and top traces, respectively). The data for the fastest trials (green traces) are time-aligned with the data for the slowest trials (red traces), and the black arrows give the median onset time of the fastest trials. These data indicate that broadband gamma amplitude begins to increase around the time of the full population-level response in the fastest trials at those locations. Together, these data suggest that in slow trials, only a fraction of the neuronal population of any cortical area becomes active at the same time.

3.5. Population-Level Activity Occurs Within Large-scale Cortical Networks

In Section 3.3, we showed that the ability to predict reaction time depends on information across the whole cortical network. Specifically, we found that our ability to infer reaction time from network activity varied across the four tasks (Fig. 6A) and that both temporal as well as spatial coverage of the electrodes predicted this variation. Figs. 6B and C show the relationship between average coverage (temporal and spatial) values and our ability to predict reaction time from network activity.

Figure 6: Influence of cortical coverage on prediction of reaction time.

(A) Mean and standard errors of Spearman’s correlations between cortical transmission times (CTTs) and reaction times (RTs), grouped by task. (B) Variations in “temporal coverage” are related to differences in predictive performance across the different tasks shown in A. The temporal coverage increases, as onset times across different electrodes are spaced out across a larger time period. (C) Variations in “spatial coverage” of the cortex are also related to those differences in performance. Broader spatial coverage indicates that the task-related electrodes cover a larger cortical area.

4. Discussion

4.1. Predictions from Network Activity

Our observation in Section 3.5 is in alignment with an increasing recognition that behavior is supported by functional interactions between nodes in large-scale cortical networks, and may not solely be defined by spatially confined brain dynamics (Bressler and Menon, 2010). Studies on the role of other cortical activity (e.g., event-related potentials (Gerson et al., 2005)) and physiological responses (e.g., saccades (Dean et al., 2011)) on reaction-time also support this observation.

Thus, it is likely that our ability to observe the neural basis of a significant portion of reaction time variance rested primarily on our ability to detect the onset of population-level activity across large areas of the cortex, and may explain why important previous efforts that investigated neural activity in smaller areas of the brain reported markedly smaller effect sizes (e.g., a maximum r² of 0.01 as in Womelsdorf et al. (2006)). Our results support the body of literature that broadband gamma activity is widely recognized as a direct measure of the population-level activity (Crone et al., 1998; Ray and Maunsell, 2011).

4.2. Modulation of Cortical Activity by Low-Frequency Oscillatory Activity

A previous model (Coon et al., 2016) proposed that any increase in population-level activity (as assessed by broadband gamma) can only occur when cortical excitability (as assessed by low-frequency oscillatory amplitude) exceeds a certain threshold (Fig. 7A). Our data are consistent with the hypothesis that broadband gamma increases occur immediately with the arrival of afferent input, but that the degree of that increase is modulated by low-frequency activity (Fig. 7B). Establishing the details of this relationship will require further studies that separately control for afferent and oscillatory inputs.

Figure 7: Alpha oscillations rhythmically modulate broadband gamma.

If afferent input arrives during a period of low cortical excitability, the full population response is delayed (red traces) compared to when it arrives during a period of high excitability (green traces). (A) Model suggested in Coon et al. (2016). The neuronal population does not respond until a specific excitability threshold is reached. (B) Our data are consistent with the hypothesis that the neuronal population begins to respond as soon as afferent input arrives, and the degree of the response is proportional to cortical excitability.

It is worth noting the distinction between two fundamentally different neural phenomena that low frequency oscillatory signal components and high frequency broadband signal components reflect in the EEG: (1) truly periodic, oscillatory processes with specific timing and center frequencies (typically tied to modulatory or gating functions); and (2) non-oscillatory, broadband processes (typically reflecting local computation), that nevertheless can be extracted more readily in some frequency bands than in others. The functional significance of the former has been recognized since the early days of EEG, when Hans Berger and others observed that closing one’s eyes elicits an approximately 10 Hz oscillation in the scalp EEG signal recorded from posterior areas of the head, over visual cortex (Adrian and Matthews, 1934). Since then a vast literature on band-limited low-frequency oscillations has emerged relating rhythmic brain activity to nearly every aspect of brain function, including learning & memory (cortical and hippocampal “theta”, O’Keefe and Recce 1993; Sauseng et al. 2009), spatial navigation (theta, Ekstrom et al. 2005), attention (alpha; Voytek et al. 2010; Gunduz et al. 2011), perception (canonical “gamma”; Fries 2005; Womelsdorf et al. 2006), and motor processing. For example, “beta” rhythms (18–25 Hz), much like the alpha we investigate here, have been implicated in the gating of cortical processing that governs motor output (Miller et al., 2007; Mäki and Ilmoniemi, 2010) through GABA-mediated inhibition/dis-inhibition (Porjesz et al., 2002; Gaetz et al., 2011), and together with alpha comprise the sensorimotor “mu” rhythm (so called because the double peak in the power spectrum evokes the form of this letter of the Greek alphabet). Across nearly all of these domains, these oscillations are viewed as manifestations of neuro-modulatory processes whose timing is critical to their function (Buzsáki and Draghun, 2004; Klimesch et al., 2007; Hanslmayr et al., 2013), such as biasing the excitability of populations of cells and coordinating their timing across multiple brain regions to e.g., bind perceptual features into a unified percept (Engel et al., 2001; Womelsdorf et al., 2007) or facilitate multimodal (ex. visuomotor) processing (Schoffelen et al., 2005; van Dijk et al., 2008; Voytek et al., 2010; Coon et al., 2016). In contrast, high frequency broad band fluctuations (broadband “high gamma”; 70–170 Hz) capture non-oscillatory changes in the mean spike rate of populations of cells (Miller et al., 2009; Ray and Maunsell, 2011). Hence broadband high gamma, although measurable through spectral decomposition, is not truly an oscillatory phenomenon per se but rather a reflection of the energy injected across all frequencies of the spectrum that results from changes in mean firing rates. While present in all frequency bands, the effect is obscured in low frequency bands by the overpowering presence of true harmonic oscillations (alpha, theta, etc.) yet can be cleanly recovered preferentially in the high gamma band precisely because no other oscillations overlap in this frequency band to interfere with its measurement. And because broadband high gamma reflects excitation (mean firing rate) more than excitability (a predisposition to respond to afferent stimulation), it represents a more direct measure of cortical population activity than low frequency oscillations alone. Our results illustrate the utility of analyzing broadband high gamma changes in conjunction with traditional oscillatory activity, allowing first for the trajectory of population activity to be charted across brain regions with precise temporal resolution, and then, in turn, enabling the study of the underlying phenomena (ex. oscillations) that explain variability in its timing and consequent changes in behavior.

As previously reported by Coon et al. in 2016 and supported by preliminary findings here (Supplementary Fig. 11), onset times preferentially occur during the alpha trough when cortical excitability is high (Buzsáki and Draghun, 2004; Lakatos et al., 2008). The phase of alpha oscillations at the time that the afferent input arrives is not measurable as that time is unknown. However, in a separate study, we are investigating the effect of stimulus-induced changes in the prestimulus phase of ongoing oscillations on the spatio-temporal properties of cortical activation. Prestimulus oscillatory phase has been shown to influence stimulus perception and cortical information flow (Hanslmayr et al., 2013; Milton and Pleydell-Pearce, 2016; Kaiser et al., 2019). In addition, the interplay between gamma, alpha phase, and cortical excitability in transcranial magnetic stimulation (TMS) opens up new avenues for investigation (Wagner et al., 2019).

Cortical oscillations have been shown to contribute to information transmission throughout the cortex by modulating cortical excitability (Ward, 2003; Schnitzler and Gross, 2005). The effect of these oscillations on cortical excitability has been evaluated using oscillatory power (Pfurtscheller et al., 1996; Jensen and Mazaheri, 2010; de Pesters et al., 2016) and phase (Fries, 2005; Haegens et al., 2011). In this work, we investigated only the relationship between alpha power and onset times. In future work, other frequency bands that have been reportedly modulating cortical excitability (such as theta (Hirase et al., 1999; Howard and Poeppel, 2012)) should be explored.

4.3. Feedforward vs. Feedback Loops

In the implementation of the functional connectivity algorithm, we chose to consider only feedforward connections between cortical locations. Our methodology may also prove useful to assess feedback loops, such as those from the motor to primary somatosensory cortex (Zagha et al., 2013).

4.4. Future Work

In this study, we utilized different datasets recorded in different subjects, which adds several sources of variance. Because our results simply report certain properties of information transmission within the cortex, we did not have to resolve these different sources of variance. We performed several within-subject/task analyses (see Supplementary Tables 2, 3 and 4, and Supplementary Fig. 12). These analyses showed that our results generalize across different subjects and entirely different tasks, which is testament to the important role of these properties in the brain’s internal communication. Future studies with specific experimental setups and/or electrode placements may unravel additional insights about the role of cortical information transmission in specific tasks or at specific locations.

5. Conclusions

In this paper, we demonstrated that accumulating latency differences of population-level activity across task-related locations in the cortical network explained a sizable fraction (up to 41%) of the reaction time variance. We documented this effect in 28 datasets from 12 human subjects executing four different tasks, which suggests that our findings generalize across subjects and tasks. We also demonstrate that temporal variations of neural activity can be found across the cortex, and thus our results cannot be attributed to a specific location. Finally, we provided evidence that the variations in onset time that lead to variations in behavior are caused by differential ramping-up of sub-threshold population-level activity. These differences explain 19% of the latency variance across all cortical locations. In summary, in this paper, we provide a mechanistic explanation for a sizable fraction of behavioral reaction time when the subject’s task is the same from trial to trial and thereby provide a path for future investigations into the physiological underpinnings of reaction time variance. Because many neurological disorders are accompanied by changes in reaction time or its variance (Richardson et al., 2011; Karalunas et al., 2014; Kochan et al., 2017), a better understanding of the neural basis for reaction time variance may ultimately lead to new approaches for rectifying deficits in important nervous system disorders (Firbank et al., 2017).

Highlights.

• Behavioral reaction-time task elicits electrocorticographic broadband gamma response

• Functional connectivity analysis reveals cortical response trajectories

• Subthreshold broadband gamma activity predicts cortical onset time

• Onset times accumulated across trajectories explain behavioral reaction-time variance

Data Availability

Comprehensive data-sets may be provided to interested researchers upon reasonable request to the corresponding author and approval of a data-sharing agreement by the Institutional Review Board of Albany Medical College.