Spatial Patterns of Persistent Neural Activity Vary with the Behavioral Context of Short-Term Memory

Summary

A short-term memory can be evoked by different inputs and control separate targets in different behavioral contexts. To address the circuit mechanisms underlying context-dependent memory function, we determined through optical imaging how memory is encoded at the whole-network level in two behavioral settings. Persistent neural activity maintaining a memory of desired eye position was imaged throughout the oculomotor integrator after saccadic or optokinetic stimulation. While eye position was encoded by the amplitude of network activity, the spatial patterns of firing were context-dependent: cells located caudally generally were most persistent following saccadic input, whereas cells located rostrally were most persistent following optokinetic input. To explain these data, we computationally identified four independent modes of network activity and found these were differentially accessed by saccadic and optokinetic inputs. These results show how a circuit can simultaneously encode memory value and behavioral context, respectively, in its amplitude and spatial pattern of persistent firing.

Introduction

A short-term memory circuit can accumulate over time information arising from different senses or brain regions and provide outputs to separate targets in order to meet the varied processing demands that arise within different memory contexts. During short-term memory, information is stored as a pattern of persistent activity across a neuronal population (Major and Tank, 2004), as evidenced by work in the spinal cord (Prut and Fetz, 1999), hindbrain (Lopez-Barneo et al., 1982), midbrain (Glimcher and Sparks, 1992), and forebrain (Fuster and Alexander, 1971). The memory centers in these brain regions often act as hubs with multiple input sources and disparate output targets. At these centers, different input-output combinations are engaged for different behavioral contexts, placing context-specific processing demands on short-term memory circuits (Baddeley, 2000). Although significant progress has been made in understanding the cellular and circuit mechanisms underlying persistent firing (Fisher et al., 2013; Wang et al., 2006), relatively little is known about how short-term memory circuits achieve the flexibility needed to generate the different input-output relationships required for context-dependent function.

A major challenge in addressing the neural basis of context-dependent processing during memory behavior is to visualize activity throughout a memory circuit under different behavioral contexts. Here we overcome this challenge by using optical imaging at the whole-circuit scale to directly observe context-dependent persistent activity in the oculomotor velocity-to-position neural integrator (VPNI) of behaving zebrafish. The VPNI for horizontal eye movements is a conserved vertebrate brain region that mathematically integrates transient inputs encoding desired eye velocity to generate persistent signals that encode a memory of desired eye position in the absence of further input (Lopez-Barneo et al., 1982; McCrea and Horn, 2006; McFarland and Fuchs, 1992). The VPNI can integrate inputs from upstream saccadic, visual, and vestibular afferents (Kaneko, 1999; McFarland and Fuchs, 1992), and send its output to multiple targets including the oculomotor nuclei, cerebellum, superior colliculus, and thalamus (McCrea and Baker, 1985; Prevosto et al., 2009). Although the essential memory-generating function of this system is preserved across different oculomotor tasks, the specific transformations that occur within the VPNI may be context dependent in order to handle the particular requirements of different eye movement behaviors such as gaze-shifting saccades, optokinetic tracking, or compensating for head movements through the vestibulo-ocular reflex.

Recent findings on VPNI physiology offer important clues as to how context-dependent processing might occur in this and other memory systems. Mechanistically, maintenance of information in the VPNI depends in large part on positive feedback generated through recurrent excitatory synaptic interactions (Aksay et al., 2001, 2007; Fisher et al., 2013). Traditionally, it has been assumed (Seung, 1996; Seung et al., 2000) that such positive feedback tightly couples memory neuron firing rates so that there exists, in all behavioral contexts, a single pattern of activity across the network specified by the relative firing rates of the constitutive neurons. That is, when a system responds to stimuli of different amplitudes, the overall level of persistent firing differs, but the pattern of activity is maintained (Figure 1a). In the language of dynamical systems, this pattern is referred to as a mode of network activity, and systems with a single persistent mode are referred to as line attractors. However, while the concept of a line attractor has played a prominent role in understanding graded short-term memory in the VPNI and other brain regions (Brody et al., 2003; Kiani et al., 2013; Major and Tank, 2004; Wang, 2001), recent studies have revealed considerable heterogeneity in the dynamics of persistent firing within a memory circuit that does not fully agree with this picture (Bernacchia et al., 2011; Joshua et al., 2013; Jun et al., 2010; Machens et al., 2010; Miri et al., 2011a). This heterogeneity suggests instead that the circuit dynamics in these systems are established by multiple modes of activity, where each mode is defined by a different pattern of relative firing rates and may be associated with a different timescale of persistent firing (Figure 1b).

Figure 1. A potential network mechanism for short-term memory storage and temporal integration in different contexts.

(a) Persistent activity for three cells (top) in a line-attractor network defined by a single pattern, or mode, of activity (bottom). The value of the memory variable is stored in the amplitude of activity (top) along this mode. The pattern of relative firing rates of the constitutive neurons in the memory network, constant across different memory values, defines the mode (bottom). (b) Heterogeneity in the time course of persistent activity (blue) of the same three cells (top) when two modes are active (bottom). The activity of each cell is given as a linear combination of the firing rate components associated with each mode. These two components are displayed in the brown and yellow traces. (c) If the integrating network is capable of supporting multiple modes of activity, then different behavioral contexts could use different sets of modes. In this schematic, the two modes of panel b are accessed during one context (context A, top), and a different combination of modes is accessed in a second context (context B, bottom). At the single cell level (top), a signature of such context-dependent processing in a graded short-term memory circuit or integrator would be firing rates that differ across contexts in both amplitude and rate of drift back to baseline. (d) Schematic of a multi-functional memory network. Such a network supports multiple, spatially and dynamically diverse modes of activity that can be utilized by separate inputs and outputs (red, green, blue) for different behaviors or behavioral contexts.

The ability of a single memory network to generate and maintain multiple, distinct patterns of activity offers a possible solution to the challenge of context-dependent short-term memory storage. Because each mode can be accessed independently and effect a specific response in downstream targets, different modes (or combinations of modes) can be associated with different contexts (Figure 1c). Furthermore, because the modes represent dynamically organized groups of neurons, no special switching circuitry is required to rapidly change contextual processing. Rather, within the circuit established by a fixed set of synaptic connections, external inputs associated with a particular context automatically trigger the appropriate context-dependent response when they stimulate the network with a specific functional pattern of input. In this manner, a single memory network would potentially be able to generate numerous different input-output relationships for generalized context-dependent processing (Figure 1d).

To test if different modes of activity in the VPNI are used for contextual processing, we recorded the dynamics of a large population of integrator neurons during the gaze-holding period following both spontaneous saccadic and visually evoked optokinetic eye movements. We report three findings of general relevance to context-dependent memory storage. First, while the overall magnitude of persistent firing represented what value of the memory parameter was stored, the pattern of persistent firing across the circuit represented how the memory signals were initiated, i.e. the behavioral context. Second, a switch in behavioral context was coupled to a rapid transition between underlying modes of activity, resulting in a reversal across the circuit in a spatial gradient of the decay time constants of neuronal persistent firing. Third, computational modeling revealed how these data can be explained by a mechanism based upon activation of different patterns of recurrent interactions by different sets of inputs associated with each behavioral context. Together, these results demonstrate how a single short-term memory circuit can generate multiple, context-dependent input-output functions.

Results

To investigate how contextual information is represented during short-term memory, activity in the velocity-to-position neural integrator (VPNI) was assessed through recording and analysis of calcium-sensitive fluorescence responses during both spontaneous saccadic and optokinetic behavior. In the following, we identified for each of these contexts the patterns of firing underlying the maintenance of eye fixations following the cessation of upstream command signals, when the dynamics of eye position are dependent on persistent activity generated within the VPNI (Cannon and Robinson, 1987). We then compared across behaviors the dynamics and spatial organization of firing patterns in the network to quantify the impact that context has on persistent activity. Finally, we constructed a model of the VPNI that incorporates a range of experimental results in this system (Aksay et al., 2001, 2003, 2007) and explains how context-dependent processing of different short-term memory inputs can occur in a single circuit.

Persistent activity after saccadic and optokinetic stimulation

We measured the activity of neurons in the horizontal VPNI of larval zebrafish using two-photon calcium imaging (Figure 2a) and directly compared the dynamics of firing after saccadic eye movements to the dynamics after eye movements driven by the optokinetic reflex (OKR). Integrator neuron activity was imaged in horizontal planes after bolus-delivery to the caudal hindbrain of Oregon-Green Bapta-1 AM (Figure 2b), a calcium-sensitive indicator which reliably reports firing rate changes in these cells (Miri et al., 2011b). VPNI somata were identified using a modified version of a previously described method to find pixel clusters in which fluorescence was significantly correlated to eye position (Miri et al., 2011b). In total, 204 integrator cells from 13 zebrafish were included in the analysis. All cells were recorded during both saccadic and optokinetic behavior.

Figure 2. Experimental setup for exploring the context dependence of persistent activity.

(a) Schematic of experimental set-up used for synchronous two-photon calcium imaging, behavioral control, and eye-tracking (DM: dichroic mirror, F: filter, IR: infrared, L: lens, PMT: photomultiplier tube, OBJ: objective, CMOS: CMOS camera). (b) Average image of one plane in the caudal hindbrain where VPNI neurons were located. The Mauthner axons are visible on either side of the midline. Outlines indicate cells with eye position correlated activity; blue outlines indicate those that are plotted in (c). (c) Eye position and stimulus velocity (top), and fluorescence time series of individual cells (bottom) during saccadic and optokinetic eye movements. Dashed lines indicate the baseline level of fluorescence for each cell. Colored bars indicate post-stimulus fixation regions where gaze stability is dependent on persistent firing generated within the VPNI.

Our experimental paradigm was designed to test if network-level activity in the VPNI differed with stimulus context. Activity was measured during fixations at ipsilateral, eccentric values of eye position whose maintenance are known to depend upon the firing of VPNI neurons both in the dark (Miri et al., 2011b) and in the light (Miri et al., 2011a). Fixations were initiated in one of two ways. First, zebrafish spontaneously generated saccades to eccentric positions while viewing a stationary optokinetic stimulus (Figure 2c, left), subsequent fixations depended upon temporal integration by the VPNI of velocity signals from the saccadic burst generator (Scudder et al., 2002). Second, zebrafish tracked transient movement of an optokinetic stimulus to eccentric positions, at which point stimulus movement was stopped and fixation was initiated (Figure 2c, right). Because larval zebrafish do not display post-rotatory after-nystagmus (Beck et al., 2004), optokinetic velocity-storage signals were not present after cessation of stimulus movement; hence, these fixations depended upon temporal integration by the VPNI of velocity signals from pretectal pathways (Mustari and Fuchs, 1990; Mustari et al., 1994). Fixation positions did not differ significantly between conditions: the mean eye position during “post-saccadic fixation” was 9.1+0.4° (s.e.m), while the mean eye position during “post-optokinetic fixation” was 8.9 + 0.4° (p=0.35); furthermore, the average difference between left and right eye position during post-saccadic (0.42 + 0.40°) and post-optokinetic fixation (0.31 + 0.55°) also did not differ significantly (p=0.88). Thus, during both types of fixation, the VPNI had to represent what position the eye was held at, and also may have represented contextual information, namely whether the fixation was initiated through saccadic or optokinetic stimulation. We expected that if the VPNI was able to retain information about context, then during fixation the patterns of persistent firing for the two behaviors should differ significantly (Figure 1c).

We next briefly describe the qualitative features of the neural activity during these two behaviors (Figure 2c). During ipsiversive saccades (those directed ipsilaterally), integrator cells uniformly showed a rapid rise in activity. During the subsequent fixation (shaded blue), fluorescence levels either remained elevated relative to baseline (e.g. cell 4) or relaxed back towards initial values (e.g. cell 6). During contraversive saccades (those directed contralaterally), activity levels rapidly fell back to baseline if they were still elevated at the end of the fixation. During constant ipsiversive movement of the eyes driven by a constant-velocity optokinetic stimulus, fluorescence values increased in a ramp-like or exponential fashion; during the subsequent fixation (shaded red), fluorescence levels either remained elevated (e.g. cell 1) or relaxed back toward baseline (e.g. cell 8). Eye position and neuronal activity were reset either by contraversive saccades or contraversive stimulus motion (at right).

Modes of integrator activity during post-saccadic and post-optokinetic fixations

To facilitate a quantitative comparison of the dynamics and patterns of persistent firing in the two behavioral contexts, we first determined the persistent firing modes activated during fixations under each context. To do so, we employed the singular value decomposition (SVD) to analyze the stimulus-triggered average responses of the VPNI population during eye fixations and the preceding stimulus period (Experimental Procedures). The SVD is a factorization procedure that allowed us to identify, in the state space defined by the activity of all recorded neurons, those directions, or modes, which explain the greatest amount of variation in the data. For each context, we identified in two steps those directions that captured behaviorally related variation. First, we analyzed the results of SVDs of both the fluorescence data and estimated noise during each context to determine a subspace in which relevant signal resided, effectively smoothing away those dimensions that carried only noise. Second, we compensated for the slow intrinsic dynamics of calcium imaging to determine the firing rate trajectory in this subspace, and then found the modes needed to explain persistent firing present during the fixation period.

For each context, we found that fluorescence activity at the population level could be understood by considering the contributions primarily along three modes. Fluorescence averages were determined over a period that included both the fixation (Figure 3a, solid) and a brief period immediately preceding it (Figure 3a, dashed). After taking a SVD of the population fluorescence data for each response type separately, we plotted the singular values, or root-sum-squared (rss) amplitudes, of the data projections (also termed “components”), along the resulting fluorescence modes (Figure 3b, black). To understand how many of these modes were needed to capture the signal in our data, we next computed a SVD of noise estimated from inter-trial differences (Experimental Procedures) to determine the spectrum of noise singular values (Figure 3b, gray). In each context, the first two data modes had rss amplitudes with confidence intervals that did not overlap with any noise modes. Furthermore, the cumulative signal captured along the first three modes, determined from a comparison of the data and noise singular values (Experimental Procedures), together explained greater than 99.5% of the total signal variation (Figure 3b, points). Given this result, in order to minimize loss of signal while still removing noise, we kept in our analysis the first three fluorescence modes for each context.

Figure 3. Mode analysis of responses during post-saccadic (top) or post-optokinetic (bottom) fixations.

(a) Stimulus-triggered average fluorescence responses (ΔF/F) for three cells (numbered 1 through 3) during saccadic and optokinetic eye movements. Time zero corresponds to the end of the stimulus. Solid lines represent the analyzed portion of the fixation period. (b) Singular values of the stimulus-averaged responses (black), which give the root-sum-square (rss) amplitudes of each fluorescence component. These are compared to the average rss amplitudes from noise (gray). Error bars give the 90% confidence intervals. Colored dots indicate the cumulative sum of the fractional signal variation explained (c) Projection of population firing rates (normalized) along the primary (brown) and secondary (yellow) firing rate modes (S1, S2, K1 and K2). Dashed lines show projected firing at high (black) and low (gray) eye positions. (d) Deconvolution of average fluorescence data from a gives firing rates corrupted by noise (light lines, smoothed with a 1 second moving median filter, MATLAB medfilt1), while deconvolution after using the singular value decomposition to separate signal from noise gives firing rates that better capture the underlying dynamics (thick lines).

Next, we applied a deconvolution procedure accounting for the dynamics of calcium buffering in these cells to determine, from the first three fluorescence components, three firing rate modes (Experimental Procedures). For each context, we found that during the fixation period, which was defined by the window between 1 and 7 seconds after termination of the movement command, only two of these modes were required to explain greater than 95% of the mean and variance of the firing rate; firing along the third mode was primarily restricted to the period preceding the fixation (Figure S1). During fixations, the primary saccadic and optokinetic firing modes, S1 and K1, were associated with slow relaxation times (Figure 3c, brown traces), and the secondary firing modes, S2 and K2, with faster dynamics (Figure 3c, yellow traces). For each context, a linear combination of the contributions along the two persistent firing modes accurately reflected the trends underlying the observed relaxation in firing rate during fixations (Figure 3d, thick lines; Figure S2). We also confirmed using simulated data with known firing rates the accuracy of this analysis procedure in providing a reliable estimate of the mode structure underlying persistent activity (Figure S3).

Finally, we examined the relationship between activation along these modes and changes in eye position. We found that changes in eye position were largely encoded by changes in firing along the primary mode (Figure 3c, dashed lines). An increase in average fixation position by 44.5% during post-saccadic (⊖_sacc=7.3° to ⊖_sacc=10.5°) and 30.2% during post-optokinetic (⊖_okr=8.5° to ⊖_okr=11.1°) fixations resulted in 35.5% and 40.4% percent increases in firing along modes S1 and K1, but only 1.4% and 15.2% increases along S2 and K2. In summary, these results suggest that persistent activity for each context is supported predominantly by two firing rate modes, one with faster and one with slower relaxation times, and that eye position is largely encoded by the amplitude of firing along the slow mode.

Heterogeneity in persistent firing

Next, to see if there were differences between behavioral contexts in the way that the VPNI stored a memory of desired eye position, we examined the pattern of single-neuron firing rates across the population following saccadic versus following optokinetic eye movements. This was done by characterizing the fixation-period firing (e.g. Figure 3d, thick lines) using a traditional measure of persistent activity, the persistence time constant τ, which quantifies the relaxation of firing rate back to its baseline level after stimulation.

We observed a wide range of time constant values for both behavioral contexts. Heterogeneity at the population level was visualized for each context separately first by using a heat map to display the activities of all neurons (Figure 4a and 4b, left), and second by averaging the activity of subsets of neurons (Figure 4a and 4b, middle). While the firing rates of some neurons were maintained at approximately the same elevated level throughout the fixation window (top of each panel), the firing rates of other neurons showed moderate relaxation back to baseline (middle), and the firing rates of yet others showed a fast relaxation to baseline (bottom). During post-saccadic fixations, the mean τ_sacc was 21.0 seconds, with the lowest, middle, and highest 20% of data exhibiting time constants of 2.9, 7.7 and 76.1 seconds respectively (Figure 4a, right; 6.5 fixations per cell on average). During post-optokinetic fixations, the mean τ_okr was 6.7 seconds, with lowest, middle, and highest 20% values of 1.2, 4.0 and 19.6 seconds (Figure 4b, right; 6.4 fixations per cell on average). These results suggest that across the VPNI population there is pronounced heterogeneity in the time constants of persistent firing during both contexts, with a population mean that is significantly lower during post-optokinetic fixations as compared to post-saccadic fixations (p=2.6×10^-10; t-test).

Figure 4. Dynamics of persistent firing during (a) post-saccadic and (b) post-optokinetic fixations.

(Left) Normalized firing rate responses of individual cells. Cells were sorted separately for each context according to relaxation time constant. (Middle) Firing rates of the least (blue), middle (green) and most (red) persistent 20% of cells. (Right) Distribution of the time constants of persistent firing for the population. (c) A cell-by-cell comparison of relaxation time constants from post-optokinetic (τ_optokinetic) vs. post-saccadic (τ_saccadic) fixations. Note that, for ease of visualization of the majority of the data, time constants outside the range 1-100 seconds were plotted at the closest corresponding edge.

We next directly compared the values of the time constants between the two fixation conditions on a cell-by-cell basis and found that the measures of persistent activity for the two behavioral contexts were uncorrelated (Figure 4c; Spearman rank correlation (src=0.11; p=0.12). We tested whether this lack of correlation was simply due to intertrial noise by assessing the correlation in the data within a context. We found that time constants between randomly-selected sets of trials from the same context were highly correlated for both post-saccadic (src=0.68; p=4.9×10^-29) and post-optokinetic (src=0.63; p=5.4×10^-24) fixations. Thus, the heterogeneity in persistent firing dynamics during post-saccadic and post-optokinetic fixations must have been associated with different patterns of persistent firing across the network for the two contexts.

Spatial patterns of persistent firing

Next we sought to determine if the differences between the two fixation conditions in the time constants of persistent firing could be related to different spatial patterns of firing across the integrator network. To do this, we assessed two metrics. The first was a contextual different measure (D; Experimental Procedures) that indicated on a unit scale how much longer or shorter τ_okr was relative to τ_sacc for a given cell. The second, a pairwise difference measure (Δ; Experimental Procedures), indicated for a given context how persistent one cell was in comparison to another and was used to assess spatial gradients in persistent activity.

We found significant differences along the rostro-caudal axis: while during post-saccadic fixations the most persistent cells were those located caudally, during post-optokinetic fixations the most persistent cells were instead located rostrally; in other words, there was a reversal in the spatial gradient of persistent firing along this axis (Figure 5, Figure S4 and Figure S5). A plot of the ROI for the VPNI population, color coded by the contextual difference measure, qualitatively revealed a trend towards greater τ_okr relative to τ_sacc for cells at rostral and medial locations (Figure 5a). Assessing these trends quantitatively (Figure 5b; 3,884 pairs), we found that, for every 100 micrometers of displacement in the caudal direction, in the post-saccadic context persistence values rose by an average of 6.8% (src=0.12, p=3.7×10^-7), but in the post-optokinetic context they fell by an average of 6.5% (src=-0.12, p=5.0×10^-8). We also found that the gradients along the dorso-ventral and medio-lateral axes differed across the two behavioral contexts, but without the same degree of statistical certainty (Figure S5). In summary, these data show that there exist differing spatial patterns of firing across the network for the two behavioral contexts, with a significant reversal of gradients along the rostro-caudal direction.

Figure 5. Context-dependent reversal in spatial gradients of persistent firing.

(a) Map of VPNI neurons from all animals aligned to the rhombomere 6/7 border (Figure S4). Neurons are color-coded according to the fractional difference between their time constant during post-optokinetic and post-saccadic fixations $(D=({\tau }_{\text{okr}}-{\tau }_{\text{sacc}})/\frac{1}{2}({\tau }_{\text{okr}}+{\tau }_{\text{sacc}}))$, with red indicating longer time constants during post-optokinetic fixations. (b) For the entire population, gradients in persistent firing rate along the rostro-caudal direction assessed by measuring pairwise differences in time constant, normalized by the pairwise mean time constant. Confidence intervals indicate the standard error of the estimate of the mean.

Degree of overlap between post-saccadic and post-optokinetic modes

The differences in the spatial patterns of activity described above suggest that the two firing rate modes active during post-saccadic fixations were not the same as the two modes active during post-optokinetic fixations. In this section we extend our SVD analysis to quantify the degree to which the modes differed.

First, to find a new set of firing rate modes that could explain memory-related activity under both contexts, we performed an SVD on the firing rate signal jointly captured by the modes S1, S2, K1, and K2 during the fixation period (Experimental Procedures). This identified a set of four orthogonal modes, labeled J1-J4, that were together required to account for at least 95% of both the mean and variance of the firing rate signal. The firing rate components associated with modes J1 and J2 contained information about the slow relaxation of activity in each context, whereas those associated with modes J3 and J4 captured the relatively fast dynamics (Figure S6a).

Next, to investigated the degree of separation between the fixation signals in the two contexts, we graphically represented their time-varying firing rate trajectories during the fixation period within the new space defined by modes J1 to J4 (Figure 6a; also see quantification in Figure S6b). Network activity during post-saccadic (blue) and post-optokinetic (red) fixations were plotted in the J1-J2 (top) and J3-J4 (bottom) planes. In the J1-J2 plane, changes in time of the post-saccadic signal were primarily along the J1 direction, while changes in time of the post-optokinetic signal were along both the J1 and J2 modes. Furthermore, the component along the J2 mode was opposite in sign for post-saccadic versus post-saccadic fixations. A difference also was evident in the J3-J4 plane, with the post-saccadic signal following a trajectory nearly perpendicular to the post-optokinetic trajectory. Thus, there was a clear separation of signals in the two contexts, despite the fact that in each case the eye position being represented was highly similar.

Figure 6. Context-dependent utilization of VPNI modes.

(a) Illustration of the different trajectories of post-saccadic (blue) and post-optokinetic (red) firing rates (arbitrary units). To visualize the four-dimensional trajectories, trajectories were projected separately along the dimensions defined by the two slower firing rate modes J1 and J2 (top), and the two faster firing rate modes J3 and J4 (bottom). J1* through J4* specify a rotated and reflected set of joint firing rate modes that align with the post-saccadic trajectory. (b) Root-sum-square (rss) amplitudes of firing rate projections along modes J1* through J4*. Dashes indicate rss amplitudes of projected activity at high (black) and low (gray) eye positions (high and low positions are the same as in Figure 3c).

We next quantified this separation between the post-saccadic and post-optokinetic signals. To facilitate this comparison, the modes J1-J4 were rotated and reflected so that the post-saccadic trajectory was aligned with new modes J1* and J3* (Experimental Procedures; Figure 6b, dashed axes). We then calculated the root-sum-square (rss) amplitudes of the two signals along the new set of modes. As shown in Figure 6b, while the post-saccadic signal was almost entirely contained in modes J1* and J3*, and virtually absent from modes J2* and J4*, the post-optokinetic signal was distributed across three modes, J1*, J2*, and J4*. Furthermore, while nearly all of the variation in firing associated with post-saccadic eye position was along J1*, a sizeable proportion of the firing variation associated with post-optokinetic eye position existed along both J1* and J2* (Figure 6b, dashes, Figure S6c). In summary, this segregation of post-saccadic and post-optokinetic signals into spatially distinct modes of the network demonstrates that the VPNI represents eye position signals differently depending upon the behavioral context.

Potential mechanism of context-dependent persistent activity

We next explored if and how the current framework for understanding persistent activity in the VPNI memory network could be modified to incorporate our results on context-dependent firing patterns. Previous results from intracellular recordings (Aksay et al., 2001), multi-unit recordings (Miri et al., 2011a), and pharmacological perturbations (Aksay et al., 2007) have suggested that persistent activity and temporal integration in the VPNI is critically dependent on positive feedback generated through local recurrent excitatory connections (Aksay et al., 2007; Fisher et al., 2013). Hence, we expect that the patterns of persistent activity observed across the population are intimately linked to the patterns of synaptic feedback in the circuit (Fisher et al., 2013; Seung, 1996). In the following, we show how the structure of the recurrent synaptic interactions can be arranged to support persistent firing modes that match those found experimentally. We also show that a simple way to get context-dependent access to these modes is through spatial pattering of the different inputs: by having stronger saccadic inputs at rostral locations and stronger optokinetic inputs at caudal locations, we find that the multi-mode VPNI model is able to generate heterogeneity in dynamics and context-specific gradients in activity consistent with our experiments.

The VPNI model we constructed was focused on the generation of persistent firing at eccentric eye positions like those exhibited in the data. The model consisted of 100 recurrently connected neurons, in line with previous estimates of the VPNI population size (Aksay et al., 2000). We specified the recurrent connection weights so that four modes were present (Experimental Procedures), with modes 1 and 2 associated with relatively slow dynamics comparable to the above modes J1* and J2*, and modes 3 and 4 with relatively fast dynamics comparable to modes J3* and J4*. In addition, we used the spatial gradients of modes J1*-J4* (Figure S7) to specify a rostro-caudal axis for the model (Experimental Procedures). Saccadic inputs in the model consisted of 200 millisecond long bursts, and optokinetic inputs consisted of 7-second long steps of constant stimulation, consistent with the experiments (Figure 2c). In addition, we chose the input strengths so that the simulated responses approximated the response amplitude distribution in the data (Figure 6b).

Context-dependent stimulation of this multimodal model of the VPNI network was able to generate the heterogeneity and gradients of persistence time constant observed experimentally. Context-dependent stimulation was produced by activating modes 1 and 3 with saccadic inputs, and modes 1, 2 and 4 with optokinetic inputs. At the population level, activity for both contexts was composed of cells with long, intermediate, and short persistence time scales (Figure 7a). As in the experiments, the cell-by-cell comparison of time constants exhibited little correlation (Figure 7b). The spatial gradients of persistent firing along the rostro-caudal axis exhibited similar trends and a similar reversal between contexts (Figure 7c). We found that persistence gradients arose due to the spatial organization of the network's modes (Figure 7d), coupled with differential activation of these modes by saccadic and optokinetic inputs that were oppositely graded along the rostro-caudal axis (Figure 7d; top bars).

Figure 7. Multimode model of the VPNI that captures the experimental results.

(a) The normalized firing rates during fixations following saccadic (top) or optokinetic (bottom) stimulation in a model of the VPNI with multiple modes of persistent activity. Cell indexing was arranged by persistence time constant in each context. (b) Direct cell-by-cell comparison of the time constants of persistent firing, τ_saccadic and τ_optokinetic. (c) Gradients along the rostro-caudal axis of the simulation. (d) Smoothed values of the entries for modes 1 through 4 (rows 1 through 4) arranged according to the rostro-caudal axis of the stimulation. For each mode, smoothing was done by computing averages over 15 bins moving along the rostro-caudal axis. The normalized strengths of the inputs are indicated by the bar heights at the top of the panel, and the value of a mode entry is indicated by the color intensity (red: receives optokinetic input only; blue, receives saccadic input only; purple, receives both optokinetic and saccadic input).

We summarize the findings in this work as follows. Our experimental results suggest that the VPNI memory network exhibits multiple modes of persistent activity that can be accessed in a context-specific manner. The modeling results supplement these findings by showing how the existing framework for understanding persistent firing in the VPNI can be extended to incorporate the new data, predicting that differential targeting of VPNI modes is achieved via anatomical gradients in input strength. Together, these results suggest how selective targeting of distinct dynamical modes of a single network provides a general strategy for the simultaneous maintenance of stimulus value and stimulus context during short-term memory behaviors.

Discussion

We have determined a neural basis for context-dependent processing in a short-term memory network. Like other memory systems, the oculomotor integrator is responsible for processing inputs from multiple brain areas and projecting memory output to numerous targets. We addressed how this multi-functionality is accomplished by optically imaging the dynamics of large populations of integrator neurons during related behavioral contexts. We found that activity in the oculomotor integrator during fixations following either spontaneous saccades or optokinetic ramps was explained by one quickly relaxing and one slowly relaxing mode of persistent firing. However, a comparison of the modes activated during post-saccadic versus post-optokinetic fixations revealed them to be quite different, and this difference was manifest most prominently as a reversal in a spatial gradient of the relaxation time constant. Taken together, these results provide an example of how context-dependent processing in a short-term memory network can be accomplished through the utilization of different modes of persistent neural activity.

This work has shown that in a short-term memory circuit both the overall amplitude of persistent firing and the pattern of relative firing rates can be used to represent information. Graded persistent activity has historically been studied at the single-neuron level, where the main focus has been on characterizing the relationship between a single memory variable and the amplitude of firing. The presence of this relationship has been seen as evidence that persistent firing is used for storing what value the stimulus encoded. Here we show that, in addition to storing stimulus value, the VPNI network retains information about how a memory has been activated by using distinct patterns of persistent activity to represent different contexts. We suggest that this strategy of selectively targeting different patterns, or modes, of a network could be important for memory systems in general, where, in addition to retaining stimulus value, a network may need to simultaneously encode how the stored information is to be used (Cui and Andersen, 2007; Mante et al., 2013; Rigotti et al., 2013; Stokes et al., 2013), when a behavior is to be performed (Machens et al., 2010), or where a stimulus was located (Rao et al., 1997). On a broader level, utilization of multiple modes to generate different input-output relationships also could be applicable to non-memory settings requiring multi-sensory computations (Cullen, 2012; Fetsch et al., 2012; Pouget et al., 2002).

The activation of different modes of persistent activity in different behavioral contexts was associated with a rapid directional switch in a spatial gradient of the time constants of persistent firing. Spatial gradients in the functional properties of cells have been observed in a number of settings, including in the frequency tuning of locomotor neurons in the zebrafish hindbrain (Kinkhabwala et al., 2011) and in the size and spacing of grid cell fields in rat entorhinal cortex (Hafting et al., 2005). Typically, such gradients have been interpreted as being static or fixed. However, the reversal in rostro-caudal gradient observed here upon a rapid switch in behavioral context demonstrates that functional gradients can be dynamic, varying with the processing needs associated with different behaviors. As discussed below, our finding that spatial patterns of persistent firing are dynamic also has mechanistic implications for temporal integration in the VPNI system.

Context-specific patterns of persistent activity in the VPNI may be used to achieve context-specific control of downstream targets. In the VPNI, one reason why such control may be desirable lies in the division of labor at the level of the motoneurons and muscles of the oculomotor system. Studies in the tadpole (Dieringer and Precht, 1986; Straka and Dieringer, 2004) and primate (Büttner-Ennever et al., 2001; Büttner-Ennever, 2006) suggest that quickly fatiguing twitch fibers receive input from motoneurons with large somatic and axonal diameters, while fatigue-resistant, slow non–twitch fibers receive inputs from smaller motoneurons. Because saccadic and optokinetic eye movements are performed at vastly different speeds, optimal control of eye movements might require the recruitment of a larger proportion of twitch fibers during saccades and non-twitch fibers during OKR. If so, the oculomotor plant could have different dynamical properties after each of these two different stimuli (Sklavos et al., 2006), which would require differential neural control for seemingly identical fixations. Considering the large and small motoneuron populations as separate targets of the VPNI, differential control of these targets could be achieved by activating different subsets of persistent firing modes. Likewise, given the wide range of brain areas that receive efference copy signals from the VPNI, including the cerebellum, thalamus (McCrea and Baker, 1985), and posterior parietal cortex (Prevosto et al., 2009), different input-output relationships in the VPNI could be used to separately alter different aspects of a motor plan (Lisberger, 2009; Sommer and Wurtz, 2008).

Having multiple modes in a memory circuit also might be useful for the control of learning. If the set of different persistent firing patterns generated by the circuit is sufficiently diverse, then modifications of behavior could be achieved through a relatively simple re-weighting of the contributions made by each firing pattern. In the VPNI, the time constant of oculomotor integration can be modified by visual feedback, a process mediated by the cerebellum (O. Debowy and R. Baker, unpublished observations) that results in changes in the firing of VPNI neurons (Major et al., 2004). The cerebellum could help control the behavioral time constant of integration purely by adjusting the relative strengths of saccadic or optokinetic inputs onto different activity modes in the VPNI. Such an approach may be easier to implement than the traditionally proposed mechanism of VPNI plasticity involving retuning of order N² recurrent synaptic interactions (Arnold and Robinson, 1991; MacNeil and Eliasmith, 2011), as it would only require the modification of order N feedforward input connections.

The mode analysis presented here revealed a clear separation between the fixation signals following saccadic input and those following an optokinetic stimulus ramp. We did observe significant overlap along one mode, J1*. In principle, this overlap could be related to the fact that, for both contexts, VPNI fixation activity was measured while the stationary optokinetic stripe pattern was present. This was done to ensure that post-saccadic and post-optokinetic signals were monitored under the same conditions. This did, however, generate a source for common signal into the VPNI during fixations, retinal slip. Although we did not attempt to quantify how much of the overlap along mode J1* was related to this common signal, given the overall stability of fixations and the similarities between dark (Miri et al., 2011b) and light (Miri et al., 2011a) conditions in fixation dynamics both before and after VPNI inactivation, it is likely that the influence of this signal upon integrator firing during fixations, and therefore any contribution to the overlap, was quite small.

Our experimental results can be understood within an attractor network framework that has a long history in the oculomotor integrator (Cannon et al., 1983; Fisher et al., 2013; Seung et al., 2000) and other short-term memory settings (Brody et al., 2003; Burak and Fiete, 2009; Miller et al., 2003; Wang, 2001). In this framework, patterns of persistent activity across the network are established by the patterns of synaptic feedback interactions between constitutive neurons, as determined both by synaptic strengths and by synaptic nonlinearities such as those associated with short-term synaptic plasticity (Mongillo et al., 2008) and intrinsic cellular response properties (Aksay et al., 2007; Fisher et al., 2013; Koulakov et al., 2002; Lisman et al., 1998). Previous experimental and theoretical studies of persistent firing in the VPNI – including intracellular voltage measurements (Aksay et al., 2001), simultaneous paired neuronal recordings (Aksay et al., 2003), and localized pharmacological inactivations (Aksay et al., 2007) – have supported a prominent role for synaptic feedback through local recurrent excitatory interactions. These studies and others have been interpreted as being consistent with a “line attractor” hypothesis of VPNI function (Seung, 1996), which posits that a single mode of activity is able to explain persistent firing during fixations. However, this previous work was performed within one behavioral context only. Our new findings on variation with context in the time constants and spatial patterns of persistent firing indicate instead that the VPNI uses a “multimodal attractor” with at least four persistent firing modes. The model of Figure 7, which is also consistent with the above experimental results, suggests how these modes can be generated easily through specification of distinct patterns of connectivity within the network, and shows how context-dependent processing in this system can be initiated simply through opposing gradients in the transient saccadic and optokinetic inputs. Furthermore, the input patterning proposed by this mechanism appears to align with the efferent topography observed at a key component of the oculomotor neural integrator in mammals, the nucleus prepositus hypoglossi (NPH): specifically, inputs from the paramedian pontine reticular formation, the location of many saccadic burst neurons, distribute primarily in the rostral regions of the NPH, whereas second-order vestibular inputs, which also carry optokinetic information, present in a more distributed manner, with stronger input to caudal ventromedial portions of the NPH (Graybiel, 1977; McCrea, 1988; McCrea and Horn, 2006).

While it may be possible to explain these data with other models, especially those involving a context-dependent modification of the functional connectivity within the circuit (Fisher et al., 2013; Mante et al., 2013; Mongillo et al., 2008), experimental constraints suggest challenges to such alternative proposals. First, the patterns and dynamics of the persistent activity we observed were largely independent of input amplitude (Figures 3c and 6b). Second, we observed contextual differences in firing patterns even during the fixation period, when all external stimuli were absent – thus, contextual influences must be remembered on the tens of seconds time scale. Third, transitions between the different behavioral contexts and associated persistent activity modes generally occurred within a few seconds. Thus, any mechanism dependent on modifications to functional connectivity must exhibit a sustained and consistent effect that can be rapidly activated and deactivated in a manner independent of stimulus amplitude. These constraint apply to both elements mediating feedback, synaptic efficacies and intrinsic cellular properties, which in zebrafish locomotor (Kinkhabwala et al., 2011; McLean et al., 2007) and rodent grid-cell (Giocomo and Hasselmo, 2009) systems have been associated with a gradient in neuronal tuning properties, at least during a single behavioral context. The model proposed in Figure 7 based on differential targeting of activity modes defined by a fixed pattern of network connectivity incorporates all of these constraints, and represents a parsimonious hypothesis consistent with current and previous (Aksay et al., 2001, 2003, 2007) experiments in this system.

In summary, our results suggest a general framework for context-dependent processing by short-term memory networks. In this framework, different spatial patterns of firing across the circuit represent the manner by which, and context within which, a memory was activated; the overall amplitude of the context-specific firing pattern represents the value of the quantity stored in memory. Although here we have only probed the storage of memory in two settings, it is possible that additional contexts could be represented by activating additional modes of network firing (Figure 1d). Thus, by associating each context with a separate pattern of network activity, the proposed mechanism offers a scalable solution to the challenge of context-dependent information processing in short-term memory networks.

Experimental Procedures

Two-photon calcium imaging during fixation behavior

All experimental procedures were approved by Weill Cornell Medical College's Institutional Animal Care and Use Committees. Larvae were embedded in low-temperature agarose and the calcium sensitive dye Oregon Green 488 BAPTA-1 AM (5 mM, in DMSO with 5% pluronic) was injected into the caudal hindbrain (Supplementary Experimental Procedures). Two-photon imaging and eye-tracking were performed using custom-designed instrumentation; most features of this setup have been previously described (Miri et al., 2011b). Briefly, laser power control was achieved using an electro-optical modulator (Conoptics 350-50UV, driven using Conoptics 302 RM amplifier). Excitation light (790 nm) was provided through a 40× objective to the caudal hindbrain, while illumination of the eyes was achieved using an infrared LED (850 nm, Thorlabs 851L) aimed at the back of the same objective. Eye movements were monitored using a sub-stage CMOS camera (Allied Vision Technologies, Guppy FireWire Camera) at a sampling rate of approximately 13 Hz. Eye position calculations were performed using a custom Matlab program.

An optokinetic stimulus of vertical stripes was provided via a small projector (3M, MPro110). The projected light was passed through a bandpass (Thorlabs FGB25, 315-445 nm) and short-pass filter (Thorlabs FES0450, 450 nm) so that it would not interfere with fluorescence collection or eye tracking. The resulting output of black stripes on a blue background was displayed on a screen placed roughly 2 cm in front of the fish; the screen consisted of a 10 cm × 1 cm piece of diffusion film (Novatron 4008 Diffusion Sheets) anchored to the microscope stage using wax (Wiki Stix). A projected stripe measured roughly 1 cm in width on the screen, corresponding to approximately 30 degrees of the animal's field of view. Custom Matlab software was used for stimulus control. The microscope and above instrumentation were located within a light-tight enclosure.

Fluorescence imaging was performed during the period following either a saccadic or optokinetic eye movement while the fish viewed the stationary visual stimulus. For each animal, we imaged a stack of 3-7 horizontal (184 × 184 μm) planes separated by 10-15 μm for 10 sequential minutes each at approximately 1 Hz (512 × 512 pixels, 2 ms/line). For each identified integrator neuron in rhombomeres 7 and 8 (Figure S4), separate trial-averaged responses were computed for both post-saccadic and post-optokinetic fixations (Supplementary Experimental Procedures).

Determination of mode structure through the Singular Value Decomposition

The singular value decomposition (SVD) was used to identify the state-space trajectories traversed by the VPNI during post-saccadic and post-optokinetic fixations. Here we summarize our methodology; complete details are provided in the Supplementary Experimental Procedures. As a first step, we performed an SVD on the fluorescence data encompassing both the fixation period and the stimulus period immediately preceding it. For a population of N neurons recorded across time, the SVD decomposes the population fluorescence signals into N orthogonal spatial patterns of fluorescence (modes) V_n, where n=1 denotes the pattern that captures the most variation in the data and the subsequent n denote the spatial firing patterns that progressively capture the next most variation. Associated with each mode V_n is a temporal pattern of fluorescence changes C_n that is defined as the projection (dot product) of the fluorescence data F onto mode V_n through the equation $C_{t,n}={\sum }_{i=1}^N{F}_{t,i}{V}_{i,n}$, where F_t,i is the fluorescence of cell i at time point t and V_i,n is the entry for cell i in the n^th mode. Single-cell fluorescence responses then were smoothed by including only the N_sig,f largest (most significant) modes. Significance was determined by comparing the SVD of the signal to the SVD of an estimate of the noise in our data (Supplementary Experimental Procedures). Next, we deconvolved $F_{t,i}^{\mathit{\text{smooth}}}$ using a kernel representing the dynamics of calcium buffering in these cells (Miri et al., 2011b), thereby obtaining an estimate of the firing rate changes associated with the stimulus and fixation periods. We then performed a second SVD on the deconvolved fluorescence data to identify the smallest number (N_sig,r) of firing rate modes $V_{i,n}^r$ required to explain at least 95% of the total persistent firing rate variation during the post-stimulus fixation period. The fixation period was chosen to begin 1 second after the cessation of movement to avoid the inclusion of any stimulus-related signals in our analysis.

Note that fluorescence responses, rather than firing rate estimates, were used at the initial stages of this analysis because deconvolving fluorescence results in amplification of high frequency noise, making assessment of the contribution of noise to the dimensionality of the data more difficult. Results from performing an SVD on simulated data revealed that the dimensions identified from this approach should be nearly identical to those present in the underlying firing rates (Figure S3).

Assessment of persistent firing

The relaxation dynamics of the firing rate $r_{t,i}^{\mathit{\text{fix}}}$ and firing rate components $C_{t,n}^r$ were characterized using the persistence time constant τ. In Figure 3c, we compared the relaxation dynamics towards baseline of the firing rate components associated with the primary and secondary modes of network activity. This was done by fitting the first component to the function ae^−t/τ , and the second component to the function ae^−t/τ + c . For illustrative purposes only, the two fits were aligned by their baseline value in this figure panel. For Figure 4 and Figure 5, persistent firing rates during fixations were quantified by determining the relaxation time constant resulting from fits of $r_{t,i}^{\mathit{\text{fix}}}$ to the form ae^−t/τ .

To examine contextual differences in persistent firing, we first generated a spatial map of a measure that, for each cell, characterizes the relative difference in the dynamics of relaxation during post-saccadic and post-optokinetic fixations. This map was facilitated by registering data from each fish to the rostro-caudal location of the border between rhombomeres 6 and 7 (Figure S4). Cells were then color-coded based on the fractional difference between their post-optokinetic and post-saccadic time constants, $D=({\tau }_{\text{okr}}-{\tau }_{\text{sacc}})/\frac{1}{2}({\tau }_{\text{okr}}+{\tau }_{\text{sacc}})$; if multiple cells were present at a particular location, the average value D for all cells at that location was displayed.

Second, contextual differences were also examined by assessing differences in the gradients of persistent firing. Gradients were calculated by first taking the pairwise difference τ_a − τ_b between the relaxation time constants of cell a and cell b and dividing by the pair's average time constant to yield the percentage difference, Δ, such that

$$\Delta =\frac{{\tau }_a-{\tau }_b}{\frac{1}{2}({\tau }_a+{\tau }_b)}\times 100$$

This quantity was then plotted against the pairwise distance between the two cells, x_a − x_b, along the rostro-caudal, medio-lateral, and dorso-ventral axes (differences in rostro-caudal position were computed without reference to the location of the rhombomere 6/7 border). Axes were defined such that more positive values corresponded to more caudal, lateral, or dorsal positions for cell a. Pairwise measurements were made only for those cells on the same side of the brain in a given fish; statistics were calculated by pooling all pairs from all fish. Gradient plots were made by sorting data points in order of increasing pairwise distance and then separating into bins containing an equal number of pairs. Error bars were determined as the standard error of the estimate of the mean.

Overlap between post-saccadic and post-optokinetic modes

To identify a region in state-space that contained the post-saccadic and post-optokinetic signals together, we performed a joint SVD on the fixation-related firing rates identified from the analysis of Figure 3. First, the post-saccadic and post-optokinetic fixation firing rate responses were concatenated to form a 2T^fix × N matrix, where T^fix represents the number of time points in the trial-averaged saccadic and optokinetic fixations. An SVD of this joint signal matrix identified four modes of importance, labeled J1-J4 (Figure S6a-b). To quantify the separation between the firing trajectories during post-saccadic and post-optokinetic fixations within this 4-D space, modes J were then rotated to a basis J* where two of the modes were aligned closely with the primary and secondary post-saccadic modes identified in Figure 3. The sign of mode J4* was reversed to ensure that the sign of the initial post-saccadic firing rate projection was positive. In the resultant basis defined by modes J1* through J4*, J1* was almost perfectly aligned with the primary saccadic mode S1, and J3* with the secondary mode S2. By then measuring the size of the post-saccadic and post-optokinetic signals' projections along these new modes, we were able to assess the separation between the signals associated with each context (Figure 6b). To compare the encoding of eye position under these two behavioral conditions, we projected the responses at high and low eye positions (Figure 3c) onto modes J1*-J4* and calculated the root-sum-square (rss) amplitude along each (Figure 6b, dashes; Figure S6c).

Model simulations of a multi-modal integrator

To generate a network capable of reproducing the main features of our data, we used a linear model of the form:

$${\tau }_{\text{cell}}\frac{{\mathit{\text{dr}}}_i(t)}{\mathit{\text{dt}}}=-r_i(t)+\sum _{j=1}^N{W}_{\mathit{\text{ij}}}{r}_j(t)+h_i^{\mathit{\text{sac}}}{I}^{\mathit{\text{sac}}}(t)+h_i^{\mathit{\text{okr}}}{I}^{\mathit{\text{okr}}}(t),$$

where r_i(t) is the firing rate of neuron i, W_ij is the connection from neuron j onto neuron i, τ_cell is the cellular time constant, which was taken to be 100 milliseconds, I^sac(t) and I^okr(t) are the functional forms of the saccadic and optokinetic inputs to the network, and $h_i^{\mathit{\text{sac}}}$ and $h_i^{\mathit{\text{okr}}}$ are their strengths onto neuron i. The network consisted of 100 neurons, in line with experimental estimates of the number of cells in this system (Aksay et al., 2000).

The connectivity W was specified by determining a rank-4 matrix in which two modes supported persistent activity with a ten second time constant and the other two modes supported activity with a one second time constant, consistent with our joint SVD analysis (Figure 6). Additionally, a rostro-caudal axis was constructed so that the model network's modes would have spatial gradients similar to the experimentally-measured joint SVD modes J1* to J4* (Figure S7; Supplementary Experimental Procedures).