Working Memory Enhances Cortical Representations via Spatially-specific Coordination of Spike Times

Summary

The online maintenance and manipulation of information in working memory (WM) is essential for guiding behavior based on our goals. Understanding how WM alters sensory processing in pursuit of different behavioral objectives is then crucial to establish the neural basis of our goal-directed behavior. Here we show that in the middle temporal (MT) area of rhesus monkeys, the power of the local field potentials in the αβ band (8–25Hz) increases, reflecting the remembered location and the animal’s performance. Moreover, the content of WM determines how coherently MT sites oscillate and how synchronized spikes are relative to these oscillations. These changes in spike timing are not only sufficient to carry sensory and memory information, they can also account for WM-induced sensory enhancement. These results provide a mechanistic-level understanding of how WM alters sensory processing by coordinating the timing of spikes across the neuronal population, enhancing the sensory representation of WM targets.

Introduction

Working memory (WM) is an integral part of our daily life, determining how we interact with the world around us based on our goals (D’Esposito and Postle, 2015). WM helps us guide our actions based on current goals, in part by altering the processing of sensory information (Postle, 2005). Attention and WM share overlapping mechanisms: items held in WM are better processed and attention facilitates the entrance of items into WM (Awh and Jonides, 2001). Understanding the neural basis and circuitry by which WM engages sensory areas and alters representation within them is then crucial to establish the basis of our goal-directed behavior. Sensory areas have been recently shown to receive a WM-rich signal from the Frontal Eye Field (FEF) part of prefrontal cortex (Merrikhi et al., 2017). In the presence of this top-down WM signal, the ability of neurons in visual areas, including V4 and middle temporal (MT) cortex, to represent and discriminate between stimuli appearing near the location held in memory increases. Considering the mounting evidence regarding the FEF’s role in attention (reviewed in Clark et al., 2014), the WM-rich signal sent from the FEF to visual areas corroborates the idea that one purpose of WM signals is to alter neural processing within sensory areas based on goals held in WM (Gazzaley and Nobre, 2012; Postle, 2006).

Despite receiving a robust WM signal from prefrontal cortex, neurons in visual areas such as V4 or MT show almost no change in their firing rate during the delay period of WM tasks (Lee et al., 2005; Mendoza-Halliday et al., 2014; Zaksas and Pasternak, 2006). The finding that a WM signal fails to alter the firing rate in visual areas, yet enhances the processing of visual signals there (Merrikhi et al., 2017), suggests that a WM signal drives subthreshold modulations. Indeed previous studies have shown changes in LFP signals in extrastriate cortex, including MT, during maintenance of WM (Lee, et al., 2005; Mendoza-Halliday, Torres, & Martinez-Trujillo, 2014). Most models of attention presume a WM module to alter sensory processing based on current goals, yet little is known about how WM contributes to visual processing and visual attention. How this subthreshold input results in enhanced visual processing is crucial for understanding the mechanism by which WM alters sensory processing—in other words, the neural basis for the interdependence of attention and WM. We specifically assessed the subthreshold and supra-threshold effects of WM on visual areas by simultaneously recording the spiking activity and local field potentials (LFPs) from multiple sites within MT, where WM-related enhancement of location representation has already been shown during a spatial WM task (Merrikhi et al., 2017). We quantified the receptive fields (RFs) in MT and altered the location held in WM relative to each RF to study the effect of a top-down WM signal on oscillations and spike timing in MT, both alone and in the presence of visual stimuli. This systematic investigation revealed a robust increase in the αβ power of LFPs within MT when remembering a location near the MT RF. We found that not only did these changes in LFP power correlate with WM performance, they also altered the temporal pattern of spiking activity in MT, even though the overall firing rate remained unchanged. These WM-induced changes in oscillatory power and spike timing were specific to the αβ band, and were sufficient to enhance the ability of MT neurons to encode visual stimuli. Furthermore, the WM-induced enhancement of the visual representation depended upon these oscillatory changes. These results provide a mechanistic understanding of how by altering the oscillatory activity within visual areas, a spatially-specific WM signal is capable of enhancing the efficacy of a visual sensory signal, privileging the processing of WM targets. The significance of these results for better understanding the neural circuitry involved in the control of visual signals and visual attention are discussed.

Results

We simultaneously recorded spiking activity and LFP signals from the MT cortex of two monkeys using linear array electrodes (216 neurons, 480 LFP channels). Our primary goal was to examine how memorizing a location inside a neuron’s RF alters LFP oscillations and spike synchrony. In order to assess the neurons’ RFs we measured the firing rate of each neuron in response to a 7*7 matrix of probes presented during fixation (Figure S1, STAR Methods). For the same neurons, we measured the effects of spatial working memory using a memory-guided saccade (MGS) task (Figure 1A). During this task the monkey needs to maintain spatial information throughout a brief delay, and the animal’s spatial WM performance is tested by its ability to make an eye movement to the remembered location after the delay. The location held in WM could vary relative to the neuron’s RF (Figure 1A). Since neurons with different RFs were recorded simultaneously using linear array electrodes, Memory IN refers to the condition at which the memory location corresponded to the RF of the neuron being analyzed (e.g. the red circle in Figure 1A), while memory OUT was always a location in the opposite visual hemifield from the neuron’s RF (e.g. the blue circle in Figure 1A).

Figure 1. WM modulates LFP power in the αβ band.

A) Schematic of the MGS task. The monkey fixates on a central fixation point (FP), and a cue stimulus appears in one of six positions arranged around the neuron’s RF location (right). The cue stimulus disappears, and the monkey maintains fixation throughout a blank delay period. Following the disappearance of the fixation point, the monkey saccades to the remembered location to receive a reward. B) The bottom plot shows the normalized firing rate of 107 MT neurons across the course of the MGS task, when the memorized location is inside (IN, red) and outside (OUT, blue) of the neurons’ RFs. Data are smoothed within a window of 30 ms and represented as mean ± SEM. The upper plot shows the proportion of neurons showing a statistically significant increase in firing rate during the memory IN compared to the OUT condition at each point in time measured a bin size of 100 ms with step size of 50 ms. C) The scatter plot of raw firing rates during the last 500 ms of the memory period. Black dots show neurons with statistically significant changes in firing rate and empty circles show neurons with no significant firing rate modulation. The diagonal histogram shows the distribution of firing rate changes. D) The average LFP power spectrum during the memory period across frequencies (n=480 LFP recordings), for memory IN (red) and memory OUT (blue) condition. Shading shows the standard error across recorded LFP signals. E) The scatter plot of αβ power (8–25Hz) during memory IN vs. OUT. Black dots show LFP sites with a significant change in power, empty circles show LFP sites with no significant power modulation. The diagonal histogram shows the distribution of differences in αβ power for all LFPs. See also Figure S1.

In the absence of firing rate changes, LFP αβ oscillations in MT predict WM performance

We confirmed that the MT firing rates across population during the delay period do not reflect the location held in working memory. Population average and percentage of neurons with significant firing rate changes over the course of a trial are shown in Figure 1B. Out of 107 neurons, 6% (7 neurons) showed a significantly decreased, and 14% (15 neurons) showed a significantly increased firing rate during the delay period of the IN condition compared to OUT (p<0.05, Wilcoxon rank-sum test, STAR Methods; Figure 1C). Overall, we did not find a significant difference between the firing rate of neurons in the IN vs. OUT conditions (n=107, Δ firing rate=0.191±0.148, p=0.225, Wilcoxon signed-rank test), consistent with previous reports on MT and other visual areas (Lee et al., 2005; Mendoza-Halliday et al., 2014; Zaksas and Pasternak, 2006). The fact that visual areas lack delay-period firing rate changes, despite receiving a strong WM signal from prefrontal cortex (Merrikhi et al., 2017), begs the question of whether and how this top-down input affects visual areas. We studied the subthreshold changes in MT during spatial WM. We specifically looked at LFPs, as they have been suggested to reflect the input to an area (Buzsáki et al., 2012). Using the fast Fourier transform technique, we compared the power spectrum density of ongoing oscillations when the monkey remembered a location inside vs. outside the MT site’s RF (memory IN vs OUT; see STAR Methods). We found that whereas spatial WM fails to alter the firing rate in visual area MT, it drastically enhances LFP power specifically in the αβ frequency range (Figure S1). As shown in figure 1D–E, the power in the αβ range is significantly greater in the memory IN compared to the OUT condition (n=480, Δ power=0.0006±0.00005, p<10⁻³, Wilcoxon signed-rank test). A similar enhancement was also observed in the session-averaged data (Figure S1). Out of 480 recorded LFPs, 297 sites (62%) showed a significant increase in their αβ power (Wilcoxon rank-sum test, STAR Methods). The proportion of neurons significantly changing their firing rate in IN vs. OUT condition did not depend on whether the LFP recorded from that channel showed a significant WM modulation or not (df=3, p=0.223, Chi-square test). Within the significantly modulated neurons, there was no correlation between the magnitude of changes in firing rate and magnitude of change in αβ LFP power at the same site (n=22, r=0.096, p=0.673, Pearson correlation). This set of analysis show how strongly WM affects LFPs within visual area MT, yet with minimal impact on its firing rate.

We found that not only does the αβ power reflect the remembered location, it is also predictive of the animal’s WM performance. We specifically quantified the performance in terms of the accuracy, precision, and speed of memory-guided eye movements. Figure 2A illustrates the monkey’s saccade landing points relative to the visual target location in memory IN trials for a sample session. The trials are split into high power (33% of trials with the highest αβ power) and low power groups (bottom third of trials) based on the LFP recording on a single channel. Saccade error, defined as the distance between the saccade landing point and the visual target location, was significantly reduced in high power trials compared to low power ones (n=20 trials in each group, Δ saccade-error=−0.419±0.132 degrees of visual angle, dva, p=0.010, Wilcoxon signed-rank test), indicating that a greater αβ power was associated with greater saccade accuracy. Consistently, we found a significant trial-by-trial correlation between the saccade accuracy and the αβ power during the memory period (r_error&power=−0.300, p=0.019, Pearson correlation). For the same sample, saccades on high power trials were also more precise: saccade scatter, defined as the average distance between pairs of saccade landing points, was significantly lower for high power trials compared to low power ones (Figure 2A bottom, Δ saccade-scatter=−0.586±0.070 dva, p<10⁻³, Wilcoxon signed-ranked test). Performing the same analyses on a population of 153 channels exhibiting memory-selective αβ power modulation (see STAR Methods), confirmed that the αβ power is predictive of the saccade accuracy and precision. We observed an approximately 9% increase in saccade accuracy and 5.5% increase in the saccade precision for high power trials relative to low power ones (Δ saccade-scatter=−0.015±0.004, p<10⁻³; Δ saccade-error=−0.030±0.004, p<10⁻³; non-normalized values: Δ saccade-scatter=−0.042±0.026 dva, p<10⁻³; Δ saccade-error=−0.079±0.016 dva, p<10⁻³, Wilcoxon signed-rank test; Figure 2B & C, see STAR Methods). The trial-by-trial correlation between saccade accuracy and αβ power was also observed across the population (r_error&power=−0.068±0.006, p<10⁻³, Pearson correlation). To quantify the speed of memory-guided responses we measured the saccade velocity, which also showed a significant trial-by-trial correlation with the αβ power (r_{power&velocity=}0.036±0.009, p<10⁻³, Pearson correlation). Saccades were 7% faster in high power trials compared to low power ones; (Δ velocity=2.183±1.085 dva/s, P=0.007, Wilcoxon signed-rank test, Figure 2D). All behavioral correlations except one were significant for both m1 and m2 animals (Δ saccade-error_m1=−0.040±0.004, p<10⁻³, Δ saccade-error_m2=−0.011±0.007, p=0.034; Δ saccade-scatter_m1=−0.012±0.004, p=0.003, Δ saccade-scatter_m2=−0.020±0.009, p=0.036; Δ velocity_m1=3.894±1.246 dva/s, p<10⁻³, Δ velocity_m2=−0.696±1.974 dva/s, p=0.741, Wilcoxon signed-rank test). We also confirmed that the observed behavioral correlations during WM are specific to the αβ frequency range (Figure S2).

Figure 2. Memory period αβ power correlates with behavioral performance.

A) The αβ power of a sample LFP signal during the memory period is predictive of the monkey’s behavior. (Top) Saccade landing points for high αβ power trials and low αβ power trials are shown in light and dark colors, respectively; the cross is the fixation point and the black circle is the remembered target location. The distance between saccadic landing points and the remembered target location (saccade error) is smaller for high αβ power trials compared to low αβ power trials in this session. The histogram shows the distributions of pairwise distances between saccadic landing points (saccade scatter) for high (green) and low (purple) αβ power trials in this session. B–D) A comparison of saccade scatter, error and velocity for high αβ power trials vs. low αβ power trials, across all memory-selective LFP signals (n=169). Scatter plots show the mean saccade scatter, error, and velocity values for high vs. low αβ power trials for each memory-selective LFP signal. Diagonal histograms show the distribution of differences between conditions for each performance measure. The data from m1 is represented by rectangles and circles indicate the data from m2. See also Figure S2.

WM-induced αβ oscillations alter the timing of spikes in MT

We found that αβ oscillations in area MT reflect the content of WM and correlate with performance. In spite of the fact that during the memory period the average firing rate in MT does not change, if spike times are locked to these WM-induced αβ oscillations, such synchronized spiking activity could potentially benefit visual areas by driving postsynaptic neurons more effectively, resulting in stronger signals (Womelsdorf and Fries, 2007). Figure 3A illustrates the spiking activity for a memory IN trial during an example recording session. The simultaneously recorded LFP signal, filtered at five different frequencies, is also shown. Whereas spikes were clustered around a specific phase at 8–12, 11–15 and 17–21 Hz, a similar temporal relationship between the spike times and LFP phase was not present at higher frequencies. The phase at which each spike occurred for all memory IN trials is represented by an individual vector on the polar plot (n=226 spikes). The angle of each line indicates the phase at which that spike occurred, and is illustrated separately for five different frequency bands. The αβ-specific locking between the LFP phase and the spike times of this sample neuron is manifested in the clustering of vectors in the upper polar plots but not the lower ones. Using the average vector in these polar coordinates, we quantified the degree to which spikes were locked to a specific LFP phase. The value of spike-phase locking (SPL) can in principle vary from zero to one, reflecting no locking to completely phase-locked spikes respectively (see STAR Methods). This neuron’s SPL values across various frequencies are illustrated in figure 3B, indicating an ~2-fold increase in the locking of spikes to αβ phase for memory IN vs. OUT. Figure 3C shows the same analysis for all neuron-LFP pairs. The SPL in the αβ band was significantly greater for the memory IN condition compared to OUT (n=1605 neuron-LFP pairs, Δ SPL=0.009±0.001, p<10⁻³, Wilcoxon signed-rank test). This effect was also significant when SPLs for an individual neuron were averaged across simultaneously recorded LFP channels (n=107 neurons, ΔSPL=0.010±0.004, p<10⁻³, Wilcoxon signed-rank test; Figure 3C inset). Despite the absence of a significant difference in firing rate between the memory IN and OUT conditions, we performed an additional quantification of phase locking using the phase consistency method to control for any differences in the rate or the number of trials between conditions, and still found a significant increase in phase locking for memory IN (Figure S4). The observed changes in the SPL were restricted to the αβ-band and were not due to memory-induced changes in the αβ power or firing rate (Figure S4). This confirms that in the absence of firing rate changes, WM’s influence on visual areas (reflected in their αβ LFP band) is capable of changing the firing pattern of neurons in these areas.

Figure 3. Spike-phase locking (SPL) in the αβ band increases when remembering a location in the MT RF.

A) The LFP and spiking data of a sample neuron-LFP pair, over 300 ms of the delay period of a memory IN trial. Under the spike train are the simultaneously recorded filtered LFP signals centered at 5 different frequencies (10, 13, 19, 42 and 55 Hz; bandwidth ±2 Hz). The polar plots show the phases of the ongoing filtered signal at the time when spikes occurred (each plotted as a vector with a constant amplitude), over all trials for this sample pair. B) The magnitude of the vector average across all spikes is the spike-phase locking magnitude (SPL) for that frequency. The plot shows the SPL for the sample pair across all frequencies, for a memory location inside the MT RF (red) and outside the MT RF (blue). C) The SPL for memory IN (red) and memory OUT (blue), across frequencies for all pairs of neurons and simultaneously recorded LFPs (n=1605 neuron-LFP pairs). The SPL is measured using a frequency bandwidth of 4 Hz and step of 1Hz, and is smoothed with a window of 2Hz. Data are represented as mean ± SEM. Inset scatter plot shows the SPL at αβ for memory IN compared to memory OUT, with the SPL values for multiple simultaneously recorded LFPs averaged for each neuron (n=107 neurons). Error bars on each point indicate standard error of the SPL values for that neuron across multiple simultaneously recorded LFPs. For illustration purposes, one outlier data point is excluded from the scatter plot but not from the analysis. See also Figure S4.

WM alters the strength of visual signals

In order to guide our behavior, WM affects the processing of sensory signals and the execution of motor commands according to our goals. Keeping a location or object in WM improves the processing of that item (Downing, 2000; Olivers et al., 2006; Zhang et al., 2011). These benefits of WM could be provided by changing the processing within sensory areas (e.g. Merrikhi et al., 2017). Knowing that WM is capable of changing the timing of spikes, the question is whether such a mechanism can also be used during visual processing to bring about the benefits of WM for visual perception. In order to uncover the mechanism whereby the WM signal improves visual processing, we developed a modified version of the MGS task. In this version, the animal had to maintain a location in WM while a set of visual probes were presented at and around the MT RF. We recorded the firing rates of 109 MT neurons during this task. The visual probes were presented in a 7×7 grid during both the fixation and memory periods (see Figure 4A and STAR Methods). This paradigm enables us to directly examine the interaction between WM and visual signals at the level of MT. We found that although WM does not alter the delay period spiking activity in MT neurons (Figure 1C), neurons’ sensitivity to incoming visual signals increased when remembering a location in their RF. Figure 4B illustrates the visually evoked spiking activity for each neuron (firing rate 30–230ms after the onset of the optimal probe) during memory IN and fixation. We observed an ~20% increase in the visually evoked spiking activity of MT neurons when the probe was presented during the memory IN condition compared to fixation (n=109, firing rate_Fixation=12.105±1.201 spikes/sec, firing rate_Memory=14.537±1.183 spikes/sec, p<10⁻³, Wilcoxon signed-rank test, Figure 4B). Therefore, although WM fails to alter delay period spiking activity in the absence of visual information, it is capable of changing the strength of visual signals. About 9% of trials in this new paradigm were presented without any probe. This gave us the opportunity to confirm the differing effects of WM in the presence and absence of visual input. Analyzing these no-probe trials, we did not find a significant change in the delay period firing rate of MT neurons (n=109, Δfiring rate=0.400±0.277 spikes/sec, p=0.342, Wilcoxon signed-rank test, Figure 4C, abscissa histogram). Notice that these are the same neurons exhibiting a 20% increase in visually evoked spiking activity when presented with visual probes. For a direct comparison, the WM-induced change in delay period spiking activity (no-probe trials) is plotted against the change in visually evoked spiking activity (probe-trials) in Figure 4C for each individual neuron. The change in visually evoked activity was significantly greater than the change in delay period activity (n=109, Δ firing rate_Visual - Δ firing rate_Delay=2.032±0.435 spikes/sec, p<10⁻³, Wilcoxon signed-rank test).

Figure 4. WM modulates the firing rate of neurons in the presence of visual stimuli.

A) The animal performed an MGS task in which visual probes appeared during the 1s fixation period and the 1s delay period. The probe (white circle) was a brief (200 ms) small (~ 1dva) visual stimulus presented in a 7×7 grid of possible locations (open white circles, shown here for illustration only and not present on the screen). On each trial, four probes were presented in succession during both the fixation and memory periods, with an inter-probe interval of 200 ms. This 7×7 grid of probes was positioned to overlap with the RF of the recorded neuron based on the preliminary RF mapping. The location of the remembered target could vary with respect to the RF of recorded neurons. On ~9% of trials, no probes were presented. B) Visually evoked firing rates increased during the memory period. The scatter plot shows visually evoked spiking activity for the optimal probe of each neuron in the memory period vs. fixation. The diagonal histogram shows the distribution of changes in the visually evoked spiking activity between memory and fixation across 109 neurons. Two outlier neurons (>70 spikes/s) are excluded from the scatter plot but not from the analysis. C) During the memory period, visually evoked activity increased but delay activity in the absence of probes was unchanged. Scatter plot of WM-induced changes in the visually evoked spiking activity (using probe trials) against WM-induced changes in the delay period activity (using no-probe trials). Top histogram indicates the WM-induced change in delay period activity represented by the distribution of firing rate change during memory vs. fixation; right histogram indicates the WM-induced change in visually evoked activity and is similar to the diagonal histogram in Figure 4B. Diagonal histogram illustrates the difference between the effects of WM on firing rates in the presence and absence of visual probes. For illustration purposes, one outlier data point is excluded from the scatter plot but not from the analysis.

WM-induced changes in spike timing enhance MT neurons’ encoding of visual stimuli

These observed effects of spatial WM on the strength of visual signals and its effects on RF changes in visual areas MT and V4 have been previously reported by our group (Merrikhi et al., 2017). Considering that we have found spike timing to reflect the content of WM, we were specifically interested to know whether these changes in spike timing are sufficient to serve as the neural basis for the benefits of WM for sensory processing as well. First, we verified that spike timing is capable of carrying visual information. We found that by knowing the phases at which spikes occur, one can predict the stimulus location. For each of 49 probes presented during the fixation, we determined the phase of the αβ oscillation at which these probes generate spikes. After removing the sampling bias (see STAR Methods), we measured the mutual information (MI) between the 49 stimulus locations and phase of spikes. We found a significant encoding of information by spike phase (n=109 neurons, MI_Fixation=0.095±0.001, p<10⁻³, Wilcoxon signed-rank test, Figure 5A top histogram). Applying a similar analysis during WM revealed a significant increase in MI for probes presented during memory compared to fixation (n=109, ΔMI _{Memory vs. Fixation}=0.015±0.004, p<10⁻³, Wilcoxon signed-rank test, Figure 5A). As separately marked in figure 5A, during memory, 42% of neurons showed an individually significant increase in MI, and 8% significantly decreased (Wilcoxon signed-rank test, STAR Methods). Therefore, not only can spike timing encode the content of WM and visual information, but WM-induced changes in this timing can enhance the visual information encoded in this form. We further studied the spatial profile of this information enhancement by performing the same analysis on pairs of probes, instead of all 49. For each pair of probes, we measured their distance from the memorized location and the neuron’s RF. We quantified the change in the amount of visual information induced by WM as ΔMI=100×(MI_Memory-MI_Fixation)/MI_Fixation. Figure 5B shows how this information changes as a function of distance from the RF and the locus of WM. The information enhancement was greater for probes presented close to the WM locus and the neuron’s RF (r_{WM-distance&ΔMI}=−0.019, p=0.045; r_{RF-distance& ΔMI=}−0.017, p=0.024, Pearson correlation). We also found a significant interaction between the effect of the distance to the WM locus and the neuron’s RF, with a regression coefficient of 0.001 (p<10⁻³). As evident in Figure 5B, the greatest increase in information occurs for probes within 8.5 dva and 4.5 dva of the RF center and the locus of WM, respectively assessed by showing a significant increase of MI during memory compared to fixation at a p<0.01 using the Wilcoxon signed-rank test. Figure 5C specifically illustrates the ~28% increase in information for probes presented in this region (MI_Memory=0.051±0.0003, MI_Fixation=0.040±0.0003, p<10⁻³, Wilcoxon signed-rank test). During memory, 39% of neurons showed an individually significant increase in MI, and 13% significantly decreased (Wilcoxon signed-rank test, STAR Methods). Comparing the MI between the two memory conditions also showed more information about probe location encoded in spike phases during memory IN than memory OUT (n=109, MI_IN=0.051±0.003, MI_OUT=0.034±0.003, p<10⁻³, Wilcoxon signed-rank test, Figure 5D). Assessing the statistical significance for each individual neuron, we found 40% of neurons had a significant increase in MI, and 15% a significantly decrease (Wilcoxon signed-rank test, STAR Methods). These results indicate that the changes in spike timing which happen during WM convey information about visual stimuli. This information enhancement could serve as the neural basis for behavioral benefits of WM and we found this enhancement to be specific only to αβ band oscillation (Figure S5).

Figure 5. WM increased the information about incoming visual stimuli conveyed via the αβ-phase at spike times.

A) Across 109 MT neurons, the mutual information between the 49 different probe stimuli and the phases of ongoing αβ oscillation at the time of spikes showed a significant increase during memory IN compared to fixation. Scatter plot shows the MI for each MT neuron during fixation (x-axis) and during memory (y-axis). Fill indicates significance of changes in individual neurons (filled blue, p<0.05; open, p>0.05). Top histogram shows the distribution of MI values during fixation; diagonal histogram shows the distribution of differences in MI between memory and fixation. B) The increase in mutual information during the memory period depends on the distance between the probes and the RF center or memory location. Color scale shows the change in mutual information (memory – fixation) between the spikes’ phases (αβ) and probe location for pairs of probes. The change is plotted as a function of the probes’ distance from the RF center (y-axis) and distance from the memory location (x-axis). The geometric mean of the two probe positions was used to calculate distances. C&D) Change in mutual information for probe pairs < 8.5 dva from the RF center and < 4.5 dva from the memorized location. The scatter plot illustrates the average pairwise mutual information (between pairs of probe locations and αβ phase at the time of spikes) for the memory IN condition vs. fixation (C) or memory OUT (D) for individual neurons. Blue dots indicate neurons showing a statistically significant MI change, and empty circles indicate non-significant ones. Diagonal histogram shows the distribution of differences in MI between memory IN vs. fixation (C) or vs. memory OUT (D). See also Figure S5.

WM enhances visually evoked firing rate in a phase-dependent manner

We showed that a WM signal is capable of altering the pattern of spiking in visual areas. Considering that the timing of spikes has been suggested to alter the efficacy of signals (Fries, 2015), we hypothesized that inducing these phase-dependent changes in spiking activity is the mechanism by which WM enhances the efficacy of visual signals. Clearly, causally inducing such phase-dependent changes in spike timing and comparing the efficacy of visual signals with and without these changes is not currently within the realm of experimental capabilities. In the absence of such causal techniques to directly test this hypothesis, we traced the correlative interdependence between WM-induced changes in signal efficacy and spike timing. As measures of signal efficacy, we specifically examined the gain of the visually evoked firing rate to a single visual stimulus and the ability to discriminate between pairs of stimuli based on the neuronal firing rate. The gain was measured as the average firing rate in response to all probes in the RF, and was normalized by the firing rate of the neuron in response to its optimal probe during fixation. The discriminability was measured using a d′ index, based on the area under the curve in the receiver operating characteristic method (See STAR Methods; Macmillan and Creelman, 2005). We found that WM enhances the gain of spiking responses by 11%, consistent with previous findings (Merrikhi et al., 2017). As would be anticipated based on the gain increase, we found that when the d′ index is measured using the firing rate, the discriminability of probe locations does increase during memory IN compared to fixation (n=109, d′_Fixation=0.343±0.018; d′_Memory=0.417±0.016, p<10⁻³, Wilcoxon signed-rank test, Figure 6A). Quantifying the d′ index solely using the spike phases also revealed a significant increase in discriminability during memory compared to fixation (n=109, d′_Fixation=0.236±0.009, d′_Memory=0.348±0.011, p<10⁻³, Wilcoxon signed-rank test, Figure 6B). This WM-induced change in discriminability measured based on spike phase was significantly greater than the discriminability enhancement measured using the firing rate (n=109, Δ d′_{Firing rate=}0.074±0.008, Δ d′_{Spike phase=}0.112±0.005, p<10⁻³, Wilcoxon rank-sum test, see Figure S6 for a sample size control). Precisely measuring the capacity of a phase code to carry information above and beyond the rate code requires more sophisticated information theoretic measures and a larger number of trials (see Kayser et al., 2009). However, when we measured the amount of information (MI) between the 49 stimulus locations and firing rates we found only 2% increase in the amount of information during memory IN compared to fixation, which was significantly lower than the 16% increase in mutual information measured based on the spike phase (n=109, phase MI_Fixation=0.095±0.0005, phase MI_Memory=0.110±0.0006, firing rate MI_Fixation=0.182±0.0009, firing rate MI_Memory=0.186±0.0009; P _{ΔPhase MI vs. Δfiring rate MI}=0.0037, Wilcoxon signed-rank test). The above results highlight the significance of observed spike phase changes in the representation of sensory stimuli and the effects of WM on this representation.

Figure 6. WM produces αβ-phase dependent enhancement of the visual signal.

A) Discriminability for the location of a visual probe based on the spike count is enhanced during memory IN compared to fixation. The scatter plot shows the average ability of individual MT neurons to discriminate probe locations solely based on their firing rate during memory and fixation periods. The diagonal histogram shows the difference in firing rate discriminability between memory and fixation. B) Discriminability for the location of a visual probe based on the αβ phases at spike times is also enhanced during memory compared to fixation. The scatter plot shows the average ability of individual MT neurons to discriminate probe locations solely based on the timing of their spikes (i.e. the phase of αβ oscillation at which each spike occurs) during memory and fixation periods. The histogram shows the difference in phase discriminability between memory and fixation. C) The WM-induced change in gain (memory – fixation) is greater during the preferred αβ phase than the anti-preferred phase across 109 MT neurons. The scatter plot shows the change in gain at the preferred phase (y-axis) and anti-preferred phase (x-axis) for each neuron. The histogram shows the distribution of differences between the change in gain for preferred vs. anti-preferred αβ phase. D) The WM-induced change in discriminability (memory – fixation) is greater during the preferred αβ phase than the anti-preferred phase across 109 MT neurons. The scatter plot shows the change in discriminability at the preferred phase (y-axis) and anti-preferred phase (x-axis) for each neuron. The histogram shows the distribution of differences between the discriminability change for the preferred vs. anti-preferred phase. E) The phase dependence of the change in gain and discriminability is strongest for locations near the locus of WM. Plots show the mean value of the phase-dependent gain and discriminability modulations as a function of visual probe distance from the location held in WM. Thick lines indicate distance bins whose mean is significantly different from zero. Shading indicates standard error for data in each bin. See also Figure S6.

In order to study the link between spike phase changes with gain and discriminability modulations, we separately quantified the gain and discriminability for spikes happening at or opposite the preferred αβ phase induced by WM (see STAR Methods). We found that whereas both gain and discriminability were enhanced even for spikes at the anti-preferred phase, up to 61% and 75% of WM-induced discriminability and gain enhancement, respectively, were accounted for by spikes occurring during the preferred phase (n=109, Δ gain_Preferred=0.004±0.001, Δ gain_{Anti-preferred=}0.002±0.001, p<10⁻³, Figure 6C; Δ d′_Preferred=0.068±0.006, Δ d′_{Anti-preferred=} 0.042±0.006, p<10⁻³, Wilcoxon signed-rank test, Figure 6D; see Figure S6 for a control for the effects of rate changes on discriminability improvement). These results not only verify that WM enhances the efficacy of signals in visual areas, they also demonstrate the phase dependence of these signal enhancements. The finding that the enhancement in neuronal firing rate and stimulus representation is greater at certain phases of the αβ oscillation suggests a potential neural mechanism by which WM-induced phase modulations enhance the efficacy of visual signals, facilitating the processing of sensory stimuli matching the content of WM.

WM-driven changes in the visual response follow the locus of WM

The finding that representation of visual information is enhanced during the memory period compared to fixation suggests that the WM signal received by visual areas could be utilized for selective enhancement of items presented at the locus of WM. The distinction between a selective vs. generalized enhancement of sensory information is important since the former is often linked to selective attention, whereas the latter could arise from global factors such as arousal or motivation. The finding that a spatially-specific WM signal is sent to visual areas (Merrikhi et al., 2017), along with our current findings that αβ modulations are selective for the WM location (memory IN vs. OUT in figures 1 and 3), support the idea that the observed changes will selectively benefit the representation of targets spatially matching the locus of WM. Evidence for a spatially specific enhancement is found in the capacity of a phase code to represent a visual target: the information encoded in spike timing is greater in memory IN than OUT, and the magnitude of this enhancement is greater for probes closer to the locus of WM (figure 5B&D). Analyses of firing rate and phase discriminability also corroborated the spatial specificity of WM-induced effects. The discriminability of probe locations is greater during memory IN than OUT, both for d′ measured with firing rate (n=109, d′_IN=0.417±0.016; d′_OUT= 0.375±0.018, p=0.008, Wilcoxon signed-rank test, Figure S6C) and for d′ based on phase at spike times (n=109, d′_IN=0.348±0.011, d′_OUT=0.285±0.006, p<10⁻³, Wilcoxon signed-rank test, Figure S6D). Moreover, as shown in figure 6E, when we quantified the phase-dependent modulation of gain and discriminability we found both these effects to be significant primarily for locations close to the locus of WM (Figure 6E). Whereas general arousal or motivation are expected to globally improve neural processing, the greater sensory representation improvements at the WM location indicate that WM-driven changes are spatially specific, maximizing the benefit for stimuli presented closer to the WM locus.

MT sites sharing the WM locus exhibit greater synchrony and coherence

At the population level, changes in the spike timing of different neurons relative to a common oscillation have the potential to turn into firing rate changes in downstream neurons (Fries, 2005; Womelsdorf et al., 2007). For clarity, Figure S7 illustrates a case in which the firing rates of the input neurons are unchanged by WM; however, the same effect of changes in spike timing could occur at multiple stages of processing, including those which also exhibit changes in firing rate. The net effect of WM-induced changes in oscillations and spike timing in multiple neurons with RFs corresponding to the WM location is to increase the chance that their inputs will sum in the postsynaptic neuron and trigger an action potential—this effect could be subthreshold in the absence of sensory input, and only alter the firing rates of downstream neurons in the presence of a visual stimulus. Changes in both spike timing and firing rate could occur simultaneously in the input neurons during visual stimulation, resulting in a synergistic effect of these changes on the firing rate of downstream neurons. These effects could each occur at multiple stages of processing and pool across multiple levels of the visual hierarchy, with both rate and spike timing changes in each area contributing to an increase in firing rate in the next downstream area. Alternatively, rather than occurring across many areas, these changes could be state dependent, or specific to a subnetwork of neurons within a given brain area. For such changes in spike timing to ultimately alter firing rates, there must be a common oscillation coordinating activity across the population of neurons whose RFs correspond with the locus of WM. Having the RFs of simultaneously-recorded MT neurons precisely mapped, we tested whether WM induces a coordinated oscillation between pairs of sites with overlapping RFs.

In order to test the degree of coherence and synchrony between MT sites we measured the phase of the αβ LFP oscillation at each site and assessed whether pairs of MT sites maintain a fixed relative phase, i.e. their phases are locked. We looked for changes in the magnitude of this phase-locking as a result of memory maintenance, and as a function of the RF overlap between sites. Figure 7A shows five trials of a sample pair of αβ LFP signals (8–25 Hz band pass filtered) for the memory IN and memory OUT conditions. During the memory IN condition, the oscillations in both channels are in-phase throughout the trial, whereas for the memory OUT condition the same two sites drift in and out of phase. The polar plot shows the distribution of phase differences for the same pair across all trials. Phase-phase locking (PPL) quantifies the coherence by testing whether the two LFPs are at a consistent phase lag. Zero-phase locking (ZPL) tests whether that lag is zero, as a measure of oscillatory synchrony (See STAR Methods).

Figure 7. Maintenance of spatial information alters the correlated activity of LFP pairs based on the overlap between receptive fields.

A) Filtered LFP signals (αβ range, center frequency 10 Hz, bandwidth±2 Hz) for a sample pair of channels (solid and dashed lines) with similar RFs, for 5 memory IN trials (left, red) and 5 memory OUT trials (right, blue). The polar plot demonstrates the histogram of phase differences between the two LFPs across all trials for memory IN (red) and memory OUT (blue). B) The amplitude of the phase-phase locking (PPL) is plotted as a function of RF overlap for memory IN (red) and memory OUT (blue), with PPL increasing with greater RF overlap. The inset shows the relationship between PPL differences (PPL IN - OUT) as a function of the shared RF ratio. Error bars indicate standard error across LFP pairs in each range of RF overlap. C) The PPL between αβ oscillations on channels with similar RFs was greater when the memory location was inside the MT RF. The scatter plot shows αβ PPL for 43 pairs of LFP recordings during memory IN and OUT conditions. The diagonal histogram shows the distribution of change in PPL (IN – OUT) across LFP pairs. D) The proportions of phase-phase locked trials with zero phase lag was greater when the memory location was inside the MT RF. The scatter plot shows αβ ZPL for 43 pairs of LFP recordings during memory IN and OUT conditions. The histogram shows the distribution of change in ZPL (IN – OUT) across LFP pairs. See also Figures S1 & S3 & S4 & S7.

Based on the ratio of the RF shared between two neurons we quantified the RF overlap (See STAR Methods). We found a robust relationship between how coherently two channels oscillate (PPL) and the degree to which the RFs of neurons from these sites overlap in visual space. This relationship even in the absence of a memory signal indicates that a shared bottom-up signal contributes to the coherence of oscillations (Figure 7B; correlation between PPL and RF overlap in Memory OUT (r_OUT)=0.449, p<10⁻³, Pearson correlation). However, a memory signal caused MT sites to oscillate more coherently for the memory IN condition compared to OUT (n=1092, r_IN=0.461, p<10⁻³, Pearson correlation; ΔPPL=0.006±0.001, p<10⁻³, Wilcoxon signed-rank test). ΔPPL was statistically significant in 25% of LFP pairs (Wilcoxon rank-sum test, STAR Methods). The magnitude of this increase in coherence was not constant across sites, but instead depended on the extent of RF overlap between the sites, indicating that the coherence depends upon a combination of bottom-up and top-down factors (Figure 7B, inset; r _{Overlap & ΔPPL=}0.114, p<10⁻³, Pearson correlation). We also verified that the effect of RF overlap is not due to the distance between electrode contacts (Figures S3 and S4).

We were interested in the effect of the top-down memory signal on visual processing, therefore we examined the WM-induced coherence and synchrony for 43 pairs of sites with similar RFs (RF overlap > 0.5). We found that both synchrony and coherence between αβ rhythms at separate sites increased when remembering a location inside the RFs vs. outside the RFs (Figure 7C & D, ΔPPL=0.010±0.001, p<10⁻³; ΔZPL=0.015±0.002, p<10⁻³, Wilcoxon signed-rank test). ΔPPL was significant in 37% of similar-RF LFP pairs (Wilcoxon rank-sum test, STAR Methods). This suggests that the top-down memory signal is capable of coordinating oscillatory activity across visual sites, allowing phase-locked spiking activity to more effectively drive postsynaptic neurons, resulting in stronger signals. This is consistent with the idea that WM-induced coordinated oscillations across spatially similar sites, combined with phase-locked spiking activity, can more effectively drive postsynaptic neurons in downstream visual areas and enhance the representation of sensory signals based on the content of WM.

Discussion

We studied the representation of location information in MT cortex, a dorsal stream area often associated with the processing of motion and location information (Goodale and Milner, 1992; Mishkin and Ungerleider, 1982), and where visual representations are known to be enhanced during WM (Merrikhi et al., 2017). We found that the content of WM produced behaviorally correlated changes in the αβ LFP power in this area, yet did not significantly alter the firing rate of its neurons (Figures 1&2). A more careful analysis revealed the timing of spikes to be locked to the phase of these αβ oscillations during WM (Figure 3). In addition, various sensory signals (probe locations) also generate differentiable oscillatory patterns of spikes in the same frequency range, and during WM these oscillatory patterns become more distinguishable (Figure 5). Since spiking activity is widely assumed to be the principal means for the neuronal encoding of information, and indeed WM enhances the spiking activity in the presence of visual information (Figure 4), we hypothesized that inducing these oscillatory changes in spiking activity might be the mechanism by which WM enhances the efficacy of behaviorally-relevant signals. We found that when presented with visual information during the memory period, the gain and discriminability of spiking activity increases, and these changes depend upon the phase of αβ oscillations. Interestingly, we found that at the level of MT, the encoding of visual signals via spike timing shows greater improvement during WM compared to the encoding of the same signal by firing rate (Figure 6). These changes in spike patterns, capable of encoding sensory signals and their WM-induced modulations at the level of single neurons, are locked to αβ LFP oscillations, which are themselves synchronized in a spatially-specific manner across the population (Figure 7). The spatial specificity and sensory sensitivity of these modulations suggest a potential mechanism by which WM benefits perception: an enhanced efficacy of visual signal transmission via the spatially-specific coordination of spike times, which could operate in conjunction with changes in firing rate across multiple levels of the visual hierarchy. Considering that ventral stream areas also receive spatial working memory information (Merrikhi et al., 2017), and the segregation of information in the dorsal and ventral streams may not be as extreme as sometimes suggested (Milner and Goodale, 2008), the degree to which similar changes in oscillatory power and spike timing contribute to visual information processing in other dorsal and ventral areas during spatial and object WM tasks remains to be studied.

Attention is the means by which the processing of selected sensory information is enhanced at the expense of ignoring irrelevant distracters. The maintenance of information in WM is an essential part of attention. The exact mechanism by which a top-down attentional signal alters processing of sensory information has been the subject of meticulous research (Clark et al., 2014). Our results provide a basis for how a top-down prefrontal signal contributes to the enhancement of sensory processing. Prefrontal areas send a signal directly to visual areas carrying the content of WM (Merrikhi et al., 2017)—although it is not yet known whether the content of WM is present in the comparatively weak prefrontal projections to MT (Markov et al., 2014a). Here we show that, in the presence of such a top-down signal, neurons in visual areas change their pattern of spiking in a spatially-specific manner. Even without changing the input-output relationship of a single neuron, such WM-induced changes in phase-locked spiking activity across a population of neurons could be sufficient to enhance the gain of visual signals. Our experimental data also confirms that the changes in spike timing are associated with a phase-dependent enhancement of visual signals and a stronger representation of visual stimuli.

LFPs reflect a mixture of feedforward, local, and feedback signals. The modulation of low frequency rhythms in many tasks involving top-down feedback suggests that the changes we observed may be an effect of feedback (Bastos et al., 2015a; van Kerkoerle et al., 2014a; Michalareas et al., 2016; Richter et al., 2017; see also STAR Methods for more details). The joint change in low-frequency rhythms and signal gain reported here suggest that similar changes in low frequency rhythms and spike synchrony could be employed to generate attentional benefits. However, α power is decreased and no specific β modulation has been reported in V4 during attention tasks with ostensibly similar top-down inputs (Buffalo et al., 2011; Fries et al., 2008). Increased αβ synchrony between areas appears to be a consistent signature of top-down influence; however, the effect of this inter-area oscillatory synchrony on local power and synchrony may differ, increasing αβ range SPL and PPL in MT and inferotemporal cortex (Tallon-Baudry et al., 2004), while suppressing α power in V4 (Buffalo et al., 2011). Whether these differences in local αβ effects are attributable to the specific task being performed, the brain area, or the specific layer (Buffalo et al., 2011) recorded from remains to be seen.

The mechanism by which the content of WM is preserved has also been the subject of extensive research. Sensory recruitment models posit a contribution of sensory areas to memory maintenance (Harrison and Tong, 2009; Lee et al., 2005; Pasternak and Greenlee, 2005). Our set of results provides an alternative view about how sensory areas are recruited by WM: rather than maintaining the content of WM in sensory areas, a purpose of the WM signal sent to these areas, and the WM-induced modulations there, could be to bring about the benefits of WM for sensory processing. Thus, regardless of the mechanism by which the content of WM is maintained, the processing of incoming signals in sensory areas is altered based on the WM content, providing a potential mechanism for the tight link between attention and WM often observed at both behavioral and neurophysiological levels (e.g. Awh et al., 1998; Ikkai and Curtis, 2011; Smyth, 1996).

Establishing whether and how the network of brain regions involved in WM incorporates sensory areas is essential for understanding the circuitry underlying the interdependence of attention and WM (Gazzaley and Nobre, 2012). We know that the FEF part of the prefrontal cortex sends direct projections to visual areas including MT (Markov et al., 2014a; Stanton et al., 1995), these projections synapse on dendritic spines of pyramidal neurons (Anderson et al., 2011) and carry the content of WM (Merrikhi et al., 2017). Indeed, manipulation of persistent activity within the FEF (using dopamine D1-receptor antagonists) alters the strength of signals in visual areas (Noudoost and Moore, 2011). These results strongly implicate the FEF as the area that drives the WM-induced oscillatory changes and the accompanying representational enhancement reported in this study. Ultimately, however, only causal experiments can definitively untangle the underlying neural circuitry.

STAR Methods

CONTACT FOR REAGENT AND RESOURCE SHARING

Further information and requests for resources and reagents should be directed to and will be fulfilled by the Lead Contact, Behrad Noudoost (behrad.noudoost@utah.edu).

EXPERIMENTAL MODEL AND SUBJECT DETAILS

General & surgical procedures

Two adult male rhesus monkeys (Macaca mulatta) were used in this study. All experimental procedures were in accordance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals, the Society for Neuroscience Guidelines and Policies. The protocols for all experimental, surgical, and behavioral procedures were approved by the Montana State University Institutional Animal Care and Use Committee. All surgical procedures were carried out under Isoflourane anesthesia and strict aseptic conditions. Prior to undergoing behavioral training, each animal was implanted with a stainless steel headpost (Gray Matter Research, Bozeman MT), attached to the skull using orthopedic titanium screws and dental acrylic. Following behavioral training, custom-made PEEK recording chambers (interior 22×22 mm) were mounted on the skull and affixed with dental acrylic. Within the chambers two 22×22 mm craniotomies were performed above the prefrontal and extrastriate visual areas (prefrontal chambers were centered at 42 mm A/P, 23 mm M/L and 28 mm A/P, 23 mm M/L; extrastriate craniotomies were centered at −6 mm A/P, 23 mm M/L and −13 mm A/P, 23 mm M/L).

Behavioral monitoring

Animals were seated in a custom-made primate chair, with their head restrained and a tube to deliver juice rewards placed in their mouth. Eye position was monitored with an infrared optical eye tracking system (EyeLink 1000 Plus Eye Tracker, SR Research Ltd, Ottawa CA), with a resolution of <0.01º RMS; eye position was monitored and stored at 2 KHz. The EyeLink PM-910 Illuminator Module and EyeLink 1000 Plus Camera (SR Research Ltd, Ottawa CA) were mounted above the monkey’s head, and captured eye movements via an angled infrared mirror. Juice was delivered via a syringe pump and the Syringe PumpPro software (NE-450 1L-X2, New Era Pump Systems, Inc., Farmingdale NY). Stimulus presentation and juice delivery were controlled using custom software, written in MATLAB using the MonkeyLogic toolbox (Asaad et al., 2013). Visual stimuli were presented on an LED-lit monitor (Asus VG248QE: 24in, resolution 1920×1080, 144 Hz refresh rate), positioned 28.5 cm in front of the animal’s eyes. A photodiode (OSRAM Opto Semiconductors, Sunnyvale CA) was used to record the actual time of stimulus appearance on the monitor, with a continuous signal sampled and stored at 32 KHz.

Behavioral tasks

Eye calibration

The fixation point, a ~1 dva white circle, appeared in the center of the screen, and the monkey maintained fixation within a ±1.5 dva window for 1.5 s. For eye calibration the fixation point could appear either centrally or offset by 10 dva in the vertical or horizontal axis. The monkey was rewarded for maintaining fixation.

Preliminary RF mapping

Preliminary RF mapping was conducted by having the monkey fixate within a ±1.5 dva window around the central fixation point, while ~2.5×4 dva white bars swept in 8 directions (4 orientations) across the approximate location of the neuron’s RF. Responses from the recording site were monitored audibly and visually by the experimenter, and the approximate boundaries of the RF were noted for the positioning of stimuli in subsequent behavioral tasks.

RF mapping

RF mapping occurred during a passive fixation task. Monkeys fixated within a ±1.5 dva window around the central fixation point. After 200 ms of fixation, RFs of neurons were mapped by presenting brief (200 ms) visual probes (~1 dva white circles) in a 7×7 dva grid of locations in 1–2.5 dva intervals. Eight probes were presented in succession, in pseudo-random order, with an inter-probe interval of 200 ms. this 7×7 grid of probes was positioned to overlap with the RF of the recorded neuron based on the preliminary RF mapping described above. Monkeys maintained fixation for 1800 ms to receive a reward.

Visually guided saccade task

Motion direction selectivity was measured during a visually guided saccade task. Monkeys fixated within a ±1.5 dva window around the central fixation point. After 500 ms of fixation, a moving grating appeared in the RF of the neuron being recorded. After 1000 ms, the fixation point disappeared. On 50% of trials, a saccade target (1 dva black circle) appeared opposite the RF simultaneous with the offset of the fixation point; on these ‘saccade away’ trials, the monkey saccaded away from the RF to the target to receive a reward. On the remaining 50% of ‘saccade toward’ trials, no target appeared and the monkey saccaded to the moving grating to receive a reward. Gratings were 2.5 dva, 100% contrast, 1.5 degree/cycle and moved in one of eight directions. This task was used in order to measure the directional tuning properties of recorded neurons and confirm that the recorded neurons are within MT cortex.

Memory guided saccade task

Monkeys fixated within a ±1.5 dva window around the central fixation point. After 1 s of fixation, a 1.35 dva square target was presented and remained onscreen for 1 s. The animal then remembered the target location while maintaining fixation for 1–2 s (memory period) before the central fixation point was removed. (In some sessions the memory period was a fixed 1 s in length; in other sessions the delay period varied randomly from 1.5 s to 2 s.) The animal then had 500 ms to shift its gaze to a ±4 dva window around the previous target location, and remain fixating there for 200 ms to receive a reward. This task was performed with six potential target locations, located in 45 degree increments from −90 to +90 degrees, and at 180 degrees relative to the estimated RF center. Locations for LFP recordings are labeled based on the RFs of neurons recorded on the same channel.

Memory guided saccade task with visual probes

Monkeys fixated within a ±1.5 dva window around the central fixation point. After 1 s of fixation, a 1.35 dva square target was presented and remained onscreen for 1 s. The animal then remembered the target location while maintaining fixation for 1 s (memory period) before the central fixation point was removed. The animal then had 500 ms to move his eyes to a ±4 dva window around the previous target location, and remain fixating there for 200 ms to receive a reward. The visual stimulus consisted of a series of brief (200 ms) visual probes (~1 dva white circles) in a 7×7 dva grid of locations in 1–2.5 dva intervals, both before target presentation (fixation RF mapping) and during the memory period (memory period RF mapping). Four probes were presented in succession, with an inter-probe interval of 200 ms. This 7×7 grid of probes was positioned to overlap with the RF of the recorded neuron based on the preliminary RF mapping described above. The location of the remembered target could vary with respect to the RF of recorded neurons.

Neurophysiological recording

The electrode was mounted on the recording chamber and positioned within the craniotomy area using a Narishige two-axis platform allowing continuous adjustment of the electrode position. For single electrode recordings, a 28-gauge guide tube was lowered to contact or just penetrate the dura, using a manual oil hydraulic micromanipulator (Narishige, Tokyo, Japan). Then a varnish-coated tungsten microelectrode (FHC, Bowdoinham, ME), shank diameter 200–250 μm, impedance 0.2–1 MΩ (measured at 1 kHz), was advanced into the brain for the extracellular recording of neuronal activity. Single-electrode recordings used a Plexon pre-amplifier and AM Systems amplifier, filtering from 300 Hz - 5 KHz. For array electrode recordings a 28-gauge guide tube was lowered as described, and the 16-channel linear array electrode (V-probe, Plexon, Inc., Dallas, TX; distance between electrode contacts was 150μm) was advanced into the brain using the hydraulic microdrive. The array electrode was connected to a headstage pre-amplifier (Neuralynx, Inc., Bozeman MT). Neuralynx Digital Lynx SX and associated software were used for data acquisition. Spike waveforms and continuous data were digitized and stored at 32 kHz for offline spike sorting and data analysis. Spike waveforms were sorted manually, and the quality of isolations for simultaneously recorded neurons confirmed using a support vector machine classifier (Figure S3). Area MT was identified based on stereotaxic location, position relative to nearby sulci, patterns of gray and white matter, and response properties of units encountered. The location of brain areas within the recording chamber was verified via single-electrode exploration prior to beginning data collection with the electrode arrays.

METHOD DETAILS

Data Analysis

All analyses were carried out using MATLAB. The LFP signal was resampled at 1000 Hz and normalized by subtracting the mean waveform and dividing the result by the σ over trials. We also removed the evoked response by subtracting the average stimulus-aligned LFP response across all trials. All population statistics are reported as mean±SEM (standard error of mean). Nonparametric statistical tests (Wilcoxon signed-rank and rank-sum) tests are used throughout for calculating p-values, unless mentioned otherwise.

Power spectrum estimation

To analyze the amplitude spectrum, we discarded stimulus-evoked components of the oscillations by subtracting the (evoked) mean waveform across trials from single trials. We calculated the power spectral density (PSD) of each signal by multiplying the signal with a single Slepian function and taking the absolute value of Fourier transform applied to it (Jarvis and Mitra, 2001; Mitra and Pesaran, 1999). The PSDs were averaged across the trials of each channel. Next, we replaced those power components corresponding to the 60 Hz (a window of 57–63 Hz) line noise by the mean power of the two neighboring frequency components. We obtained fixation and visual power spectra by calculating the amplitude spectra in a 1000 ms window preceding the stimulus (fixation response), and in a 1000 ms window during the stimulus presentation (visual response). The window used for calculating the power spectrum of the memory period was 500 ms long, but the starting time differed between variable and fixed delay sessions: for variable delay sessions, it was the window from 1000–1500 ms after stimulus offset, and for fixed delay sessions it was the window from 500–1000 ms after stimulus offset. The memory time windows were chosen to ensure that the frequency content was not contaminated by stimulus-evoked activity. For analysis based on the responses to visual probes (including RF mapping, discrimination, and mutual information), the response window was from 30–230 ms after probe onset (selected based on previously reported visual latencies in MT and the latencies observed in our own data). For all analyses, unless otherwise stated, the same time windows were used for the fixation, visual and memory periods. The memory time windows were chosen to ensure that the frequency content was not contaminated by stimulus-evoked activity. For all analyses, unless otherwise stated, the same time windows were used for the fixation, visual and memory periods. The frequency bands used for subsequent analysis were chosen based on a combination of terms applied in previous research (Buzsáki, 2006) and the specific frequencies in which our own data showed modulations. Frequencies with a significant increase in power (p<0.05) during stimulus presentation inside the RF vs. outside the RF across the population were used to define the limits of the gamma band at 40–90 Hz. Other frequency bands are defined as 1–7 Hz (δθ) and 8–25 Hz (αβ) based on the terminology used in previous studies (Buzsáki, 2006).

In this paper most of the focus was on αβ oscillations. Recently, evidence has mounted that feedback within the visual system is mediated by αβ oscillations. It began with the observation that spike-field coherence in the different frequency bands differs greatly for superficial vs. deep layers of visual cortex: gamma synchrony is dominant in the superficial layers, and alpha synchrony in the deep layers (Buffalo et al., 2011). The effects of attention on synchrony were also layer-dependent, with attention enhancing gamma synchrony in the superficial layers and reducing alpha synchrony in the deep layers. Since the superficial layers are the source of feedforward projections, while deeper layers send feedback projections (Felleman and Van Essen, 1991; Markov et al., 2014b), these findings suggested that gamma synchrony may be primarily related to feedforward pathways, and alpha to feedback. Bastos and colleagues evaluated the oscillatory relationships between numerous visual, prefrontal, and parietal areas during selective attention, and compared these interactions with the relative positions of these areas in an anatomically-based hierarchy (Bastos et al., 2015b). They found greater Granger causal influences in the beta band for anatomically identified feedback projections, but greater gamma-band effects for feedforward projections. Human magnetoencephalography verifies the predominance of gamma-band influences in feedforward projections and αβ in feedback projections across the visual hierarchy (Michalareas et al., 2016). Consistent with this hypothesis, microstimulation of V1 can drive gamma oscillations in V4, while V4 microstimulation drives alpha oscillations in V1 (van Kerkoerle et al., 2014b). Recent work provides evidence that the lower frequency feedback effects help drive the feedforward changes in gamma synchrony, with the strength of feedback beta influences from parietal to visual cortex predicting the strength of feedforward gamma effects within visual cortex (Richter et al., 2017). Similarly, in humans performing a cued shape discrimination, alpha fluctuations associated with prefrontal feedback preceded stimulus evoked gamma band activity (Popov et al., 2017). Beta power has previously been associated with the delay period of motor tasks in parietal cortex (Pesaran et al., 2002; Scherberger et al., 2005), and beta band synchrony between areas or sites has been reported in visual cortex during visual working memory (Tallon-Baudry et al., 2001, 2004). Therefore, increased αβ rhythms within and between areas appears to be a consistent signature of top-down influence.

Behavioral correlation

In order to determine whether behavior varied based on trial-to-trial fluctuations in the power spectrum of specific frequency bands, the PSD values were averaged across the frequency band (8–25 Hz for alpha-beta and 40–90 Hz for gamma) during the visual or memory period for each trial. Behavioral correlations were performed only for selective LFP channels, defined as the channels with power in the relevant frequency band significantly (p<0.05) greater during the memory period than during the fixation period for the memory IN condition (defined here as the memory condition with the maximum power in the relevant frequency band for each channel). The Pearson correlation coefficient was calculated for the band-averaged PSD and the behavioral measurements (saccade error, saccade scatter, and saccade velocity) across trials for the memory IN condition. Finally, the mean of the distribution of the correlation coefficients across all selective channels is calculated. The saccade error was the Euclidean distance between the saccadic landing point and the target position for each trial. The saccade scatter was the mean of pairwise distances between all pairs of saccadic landing points. The saccade velocity is defined as the distance between the saccade landing point and the initial eye position divided by duration (from saccade initiation till saccade landing). Saccade error and saccade scatter values were normalized within each session by subtracting the minimum of all values within the session and then dividing by the difference between the maximum and minimum of all values within that session. The reaction time was the period of time between the go-cue and saccade initiation for each trial. We also examined the relationship between oscillatory power and behavior by selecting trials with low or high power in the relevant frequency band, and comparing their behavioral performance between these two sets of trials (Figure 2A–D); the trials were sorted based on the average band PSD, then the top 33% were labeled high-power trials and the bottom 33% were considered low-power trials. Only the sessions with at least 10 correct trials for each of the high- and low-power groups were included in this analysis.

Spike-phase locking

To investigate the spike-phase locking, we excluded LFP-unit pairs recorded at the same electrode channel and obtained 1605 spike-LFP pairs in both monkeys. The LFP for each trial was normalized by subtracting its mean and dividing by its σ. We filtered the LFPs into different frequency bands each with an equal bandwidth of 4 Hz and a frequency center of 2, 4, …, 100 Hz. In order to avoid edge effects at the beginning and end of an LFP time segment, we removed 50 ms of data from both ends of the filtered LFP. Next, we used a Hilbert transform to calculate the analytic signal of all components. Instantaneous phases of each component were quantified by calculating the angles corresponding to the analytic signal. For each spike, we made a vector with the instantaneous phase at the spike time and an amplitude of one. The spikes from all trials were pooled together, and then we used the vector averaging method to calculate the spike-phase locking magnitude. In order to control for any effects of potentially different numbers of spikes & trials between the two conditions, we considered a fixed window of 30 spikes and measured the spike-phase locking magnitude for the spikes of each window.

Phase-phase locking

We quantified phase locking between individual pairs of LFP channels (m, n) by computing the phase-phase locking value across the selected time window. The LFP for each trial was normalized by subtracting its mean and dividing by its σ. We filtered the LFPs into different frequency bands each with an equal bandwidth of 4 Hz and a frequency center of 2, 4, …, 100 Hz. In order to avoid edge effects at the beginning and end of an LFP time segment, we removed 50 ms of data from both ends of the filtered LFP. Next, we used a Hilbert transform to calculate the analytic signal of all components. Instantaneous phases of each component were quantified by calculating the angles corresponding to the analytic signal. Instantaneous amplitudes of each component were quantified by calculating the absolute value of the analytic signal. The phase-phase locking between two recorded channels m and n at a particular center frequency f is defined as

$$\mathit{PPL}(f)=\frac{1}{N}\mid \sum _{t=1}^N(Z_m^f(t)-Z_n^f(t))\mid$$

Where

$$Z_m^f(t)=1\ast exp(i{P}_m^f(t))$$

Where ${\mathit{P}}_{\mathit{m}}^{\mathit{f}}(\mathit{t})$ is the instantaneous phase at time t in the frequency center f for channel m. N is the number of samples in the trial. The PPL was calculated for each trial, then the values were averaged across trials in order to estimate the phase-phase locking magnitude for each pair of recorded channels. We also measured the proportion of trials with a phase difference of zero between the two channels, known as zero-phase locking (ZPL), according to the formula:

$$\begin{gathered} \mathit{ZPL}(f)=\frac{1}{N}\mid \sum _{t=1}^{N}Cp(\angle (Z_m^f(t)-Z_n^f(t)))\mid \\ {Z}_m^f(t)=1\ast exp(i{P}_m^f(t)) \\ Cp(p)=\{\begin{array}{cc}1& p\cong 0\\ 0& p\ne 0\end{array} \end{gathered}$$

Where ${\mathit{P}}_{\mathit{m}}^{\mathit{f}}(\mathit{t})$ is defined as in the previous equation. The ZPL was calculated across all trials of a particular condition. We used the limit of ±10 degree phase difference as an approximation of zero.

RF overlap

Neuronal firing rate (spikes/second) to the probes were measured in the window 30–230 ms after probe onset. For each neuron, the 7×7 matrix generated based on the firing rate generated by the presentation of 7×7 probes were then linearly interpolated to a 769×769 grid. The RF profile was constructed based on locations in which probe-evoked activity was at least 50% of the maximum probe-evoked response. An example of the high-resolution RF estimated by this method is shown in supplementary figure 1. Knowing the location of probes then we quantified the RF area in terms of degrees of visual angle for each neuron. RF overlap was the area of overlap between the two RF profiles.

$$\text{RF}\text{overlap}=\sqrt{R1\ast R2},\text{where}\mathrm{R}1=\frac{\mathit{shared}\text{RF}\text{area}}{{\mathit{neuron}}_1\text{RF}\text{area}}\text{and}\mathrm{R}2=\frac{\mathit{shared}\text{RF}\text{area}}{{\mathit{neuron}}_2\text{RF}\text{area}}$$

In Figure 7, the RF of each LFP signal is considered as the RF of the recorded neuron from that channel. Pairs were considered to have overlapping RFs if the shared RF ratio was greater than 0.5. Memory locations were described relative to the RF profiles of pairs based on their responses at those locations during the RF mapping task; for pairs of neurons with overlapping RFs, memory IN locations were locations with responses >70% of maximum in both RFs.

Mutual information

Using the MGS task with probes described above, we calculated the mutual information between probe location and the phase of ongoing alpha-beta oscillations at the time of visually evoked spikes during the fixation period and during spatial memory. The mutual information between stimulus and response quantifies the ability to discriminate between different stimuli (in this case, different probe locations) based on the response (in this case, based on the alpha-beta phase the time of spikes). The response window consisted of the 30–230 ms after probe onset, and only spike and LFP data from this period was used in calculations. Oscillation phase was calculated as described in the ‘Spike-phase locking’ section above. We divided the phase values from –pi to pi into 4 bins of size pi/2 (Panzeri et al., 2007). In order to quantify the mutual information, the first step is measuring the response variability. The most common method for measuring response variability is Shannon entropy (Shannon, 1948). The entropy of the response is

$$H(R)=-\sum _{r}P(r){log}_{2}P(r)$$

Where P(r) is the probability of observing response r to any stimulus across all trials. The next step is a calculation of the conditional response variability as follows:

$$H(R\mid S)=-\sum _r\sum _{s}P(s)P(r\mid s){log}_{2}P(r\mid s)$$

Where P(s) is the probability of presentation of stimulus s and P(r|s) is the probability of observing response r given presentation of stimulus s. Finally, the mutual information is the difference between response entropy and conditional entropy (Panzeri et al., 2007).

$$I(S;R)=H(R)-H(R\mid S)=\sum _{r,s}P(s)P(r\mid s){log}_2\frac{P(r\mid s)}{P(r)}$$

Where I(S; R) is the mutual information between the stimulus space and the response space. Calculations were done either in a space of 49 stimuli (for all 49 probe locations; Figure. 3A), or in a space of 2 stimuli (for pairs of probe locations; Figures. 3B, C). For the shuffling procedure, the spike trains and LFP signals were shuffled across trials and the mutual information is calculated as above.

Two-point discrimination

To quantify the two-point discrimination between two probes, we used the d′ index, defined as

$$d^{\prime }=\sqrt{2}\ast Z(\mathit{AUC})$$

Where AUC is the area under ROC curve and Z is the inverse of the cumulative distribution function of the Gaussian distribution (Macmillan and Creelman, 2005). A higher d′ value indicates that the two probe locations can be differentiated more readily. In the MGS task with probe presentation, 9% of trials had no probes during the memory and fixation periods. In order to have a bias-free measure of the neuron’s preferred phase, we calculated the preferred phase for each neuron using the average vector of the αβ phase at spike times during the delay period of those no-probe trials, and the anti-preferred phase was defined as 180 degrees opposite. We then calculated gain and d′ modulation, based on only the spikes occurring during the preferred or anti-preferred αβ phase across trials with probe presentation.

Individual statistical significance

Firing rate

For each neuron the spike counts are calculated for each trial. Then we used the Wilcoxon rank-sum test between the spike count values of the two conditions (IN & OUT), p<0.05.

Power

For each site the power across frequencies is computed for each trial. Then the power values in the desired range (8–25 Hz) are pooled and then we used the Wilcoxon rank-sum test the difference between the power values of the two conditions (IN & OUT), p<0.05.

MI

There is a single value of MI for each neuron across all trials for the difference between two conditions (memory & fixation), so we compared the actual value to 1000 MI values computed for shuffled trials using the Wilcoxon signed-rank test, p<0.05.

PPL

For each pair of sites the PPL is calculated across frequencies for each trial. Then the PPL values are compared between the two conditions using Wilcoxon rank-sum test between two conditions (IN & OUT), p<0.05.

QUANTIFICATION AND STATISTICAL ANALYSIS

All population statistics are reported as mean±SE (standard error). Nonparametric statistical tests including Wilcoxon signed-rank and rank-sum are used throughout for calculating p-values, and the statistical test is mentioned in each instance.