SUMMARY
We show that when a moving object suddenly reverses direction, there is a brief, synchronous burst of firing within a population of retinal ganglion cells. This burst can be driven by either the leading or trailing edge of the object. The latency is constant for movement at different speeds, objects of different size, and bright versus dark contrasts. The same ganglion cells that signal a motion reversal also respond to smooth motion. We show that the brain can build a pure reversal detector using only a linear filter that reads out synchrony from a group of ganglion cells. These results indicate that not only can the retina anticipate the location of a smoothly moving object, but that it can also signal violations in its own prediction. We show that the reversal response cannot be explained by models of the classical receptive field and suggest that nonlinear receptive field subunits may be responsible.
INTRODUCTION
In order to initiate coordinated movements, the brain must compensate for both delays in the responses of neurons and delays in the movement of limbs to their intended targets. In principle, the only way to compensate for these delays is to use the past trajectory of an object’s motion to make predictions about its future location. Such predictions are commonplace in everyday life, and even more salient examples come from athletics, such as a batter hitting a baseball or a wide receiver acrobatically catching a pass. Evidence of motion extrapolation exists for a variety of tasks: a batter tracking a fast-moving pitch (De Lucia and Cochran, 1985; Land and McLeod, 2000), human subjects hitting a moving object (Brouwer et al., 2003; Smeets and Brenner, 1995), extrapolating a trajectory (Pavel, 1990; Pavel et al., 1992), or tracking an object with smooth-pursuit eye movements (Barnes and Asselman, 1991; Becker and Fuchs, 1985; Robinson, 1965). Animals also exhibit forms of motion extrapolation both in their eye movements (Klam et al., 2001; Medina et al., 2005) and in predictive firing of neurons in cortex and cerebellum (Fukushima et al., 2002; Heinen and Liu, 1997; Kettner et al., 1997; MacAvoy et al., 1991; Suh et al., 2000). Previous work has even shown that the retina makes a contribution to predicting the location of a moving object (Berry et al., 1999).
What happens, though, when the trajectory of a moving object suddenly changes in an unpredictable fashion? In sports, this leads to errors in motor coordination, such as a batter’s swing missing a curveball when a fastball was expected, or a football player bobbling a fumbled football despite its slow movement. During smooth pursuit, sudden deviations in the trajectory lead to pursuit errors, and large localization errors often trigger catchup saccades (Carpenter, 1988; Krauzlis and Lisberger, 1994; Medina et al., 2005; Robinson, 1965). One of the most profound motion discontinuities is a reversal of direction, which makes any previous extrapolation of the object’s trajectory misleading. Motion reversal has long been known to evoke a characteristic change in EEG recordings from human subjects (Clarke, 1972; MacKay and Rietveld, 1968). These visually evoked potentials show peaks at 135–170 ms and ~260 ms after a reversal, and their source has been localized to the MT+ region of human cortex (Ahlfors et al., 1999). One experiment in monkeys suggested that single cells in areas MT and LIP might respond to motion reversals (Maimon and Assad, 2006). However, there has been no evidence of a specific response to motion reversal in earlier cortex, let alone subcortical structures.
Here, we show that motion reversal triggers an extra burst of firing in retinal ganglion cells. This firing occurs with a constant latency (~250 ms in salamander, ~190 ms in mouse), regardless of cell type or the location of the object in a cell’s receptive field. As a result, the reversal evokes synchronized firing from a large population of ganglion cells, and this synchrony can uniquely distinguish the firing event from the response of the same cells to smooth motion. We suggest that this synchronized burst signals a violation in the retina’s ongoing motion prediction. Such an error signal could be used by the brain to redirect attention or to reset central mechanisms of motion extrapolation. We also find that the synchronized burst of firing helps the retinal representation of a moving object’s location rapidly catch up to the object’s true position. This acceleration is a consequence of the retinal representation switching from one edge of the object to the other after the reversal.
RESULTS
We used a multielectrode array to record spikes extracellularly from retinal ganglion cells in salamander and mouse (Segev et al., 2004) while presenting a variety of moving objects as stimuli (see Experimental Procedures). A moving bar (110 μm wide, traveling at 1.32 mm/s) evoked a strong, temporally distributed response as it moved through a ganglion cell’s receptive field (Figure 1, top). However, when the bar reversed direction near the cell’s receptive field (pictogram to the right), an additional sharp peak of firing was observed (Figure 1, bottom). While the cell’s initial, distributed response was timed to the arrival of the bar on the receptive field center (solid arrowheads), the sharp burst of firing following the reversal was not timed to the reentry of the bar on the receptive field center (gray arrowheads). Rather, the response occurred at a fixed latency (~250 ms in salamander, ~190 ms in mouse) after the bar reversed, regardless of reversal position. We therefore conclude that this response was triggered by the reversal itself. This response was seen in a large proportion of cells in both species (salamander: 278/745 = 37%; mouse: 17/39 = 43%). Many cells, especially those in the mouse, also fired spikes when the bar moved out of the surround (asterisk). The strength of this “shift effect” varied considerably within the population and did not correlate with the response to motion reversal (Barlow et al., 1977; McIlwain, 1966).
The reversal response appeared at a fixed latency not only for different reversal positions around a single cell, but also across different cells. Figure 2 shows spike time rasters for five cells in salamander and six in mouse responding to a bar reversing at one position (see pictograms). Because each cell had a different receptive field center location, the initial response began at a different time. However, the reversal response was synchronized across all cells. The synchrony was even more apparent in the entire population (Figures 2C and 2D). When we aligned responses to the reversal time and calculated the average firing rate for a set of cells responding to different reversal locations, the distribution of initial firing due to the bar entering the receptive field was smeared, but the reversal response remained a sharp peak. Even large changes in the stimulus parameters failed to change the latency of the reversal response. Reversing bars of opposite contrast, varying widths, and different speeds all elicited a response peaked at roughly the same latency (Figures 2E and 2F).
We next studied how the strength of the reversal response depended on the location at which the bar reversed on a cell’s receptive field. When we measured the location of the bar’s leading edge at the time of reversal, we found that reversals occurring before the cell’s center coordinate had a strength that roughly followed the spatial profile of the receptive field center. However, reversals occurring beyond the center coordinate elicited responses from up to four center radii away (Figure 3A, left). The observation that both light and dark bars could trigger a reversal response in the same ganglion cell made us wonder if both the leading and trailing edge of the bar could drive firing after a motion reversal. If we instead considered the position of the bar’s trailing edge, we found that responses for reversals beyond the center coincided with the spatial profile of the center (Figure 3A, right).
We investigated this effect further by using wider moving bars. Considered individually, many ganglion cells had a reversal response whose strength peaked around two different locations, corresponding to reversals of the leading and trailing edges. Figure 3B shows a cell with a peak in its firing rate at a fixed latency of ~250 ms after the reversal. This reversal response was elicited by the leading edge (top four traces) or trailing edge (bottom two traces), but no response occurred for reversals at intermediate positions. We repeated the analysis of Figure 3A for wider bars (220 and 440 μm) and found the same asymmetry in the strength of the reversal response as function of the position of the leading edge at the time of reversal. However, when a single-contrast edge reversed its direction of motion, the strength of the reversal response corresponded well with the spatial profile of the receptive field center (Figure 3C).
We also probed the retina with moving squares instead of extended bars. Motion reversal of a square object triggered a synchronous burst of firing in the ganglion cell population, just as for a reversing bar (Figure 4A). Because a square has a limited extent in the direction perpendicular to its motion, some reversals occurred in the surround above or below the center. These motion reversals did not trigger any firing (Figure 4B). For some reversal locations, the entire square was contained within the receptive field center and still triggered a burst of firing, indicating that stimulation of the surround is not necessary for a reversal response. Together, these observations suggest that synchronized firing can be generated when either the leading or trailing edge of a moving object reverses direction on the receptive field center but not on the surround.
Identifying Motion Reversals
Since the ganglion cells that respond to a reversing bar are not a small, specialized class, but rather a large fraction of the entire population, the brain faces a serious challenge in interpreting this message. Most spikes from one of these cells signal the smooth motion of an object across the cell’s receptive field, while other spikes from the same cell have a very different meaning: an object has reversed direction. This ambiguity must be resolved by looking at the population of ganglion cells. With this idea in mind, we constructed a decoder which used the spikes from a group of cells to distinguish reversals from smooth motion. We can think of the decoder as a “reversal-detector” cell in a subsequent neural circuit, which receives input from many retinal ganglion cells. We model this detector with a linear filter f(t) that acts on all input spikes {ti}, and a threshold, θ. Whenever the contribution summed over all inputs exceeds the threshold
the detector decides that a reversal has occurred.
After computing the optimal filter from the data (see Experimental Procedures), we evaluated the performance of the decoder by detecting reversals in single-trial spike trains (Figure 5). We randomly selected many groups out of 31 cells recorded in one experiment, and applied the filter to all the spikes in the ganglion cell population for 120 trials each of smooth motion and motion reversals. Different values of the threshold θ result in a tradeoff between more misses versus more false alarms; we chose many values of θ and quantified the total error using the area under the receiver operator characteristic (ROC) curve (Green and Swets, 1966). Performance depended strongly on the spatial separation of ganglion cell receptive fields in each group (Figure 5D). This effect occurs because reversal synchrony was easier to detect when smooth motion triggered the cells to fire at different times. For widely separated groups of 16 ganglion cells, the total error rate was 4.2%.
Can this simple decoder achieve even better performance with more cells? Because reversals anywhere up to two radii beyond the center coordinate in either the horizontal or vertical direction can trigger a burst of firing, we estimate that over 275 cells are available to the decoder and at least 100 of them have significant reversal responses (see Experimental Procedures). To explore the performance possible with larger populations, we pooled over reversal locations, treating data from the same cell recorded at multiple reversal locations as multiple cells with receptive fields at different distances from point of reversal. Using 71 such cells with reversal responses, the decoder achieved 100% correct detection on all 120 reversal trials and 0% false alarms on all 240 nonreversal trials. While this analysis does not demonstrate that the brain actually does identify reversals in this manner, such identification is clearly possible using a biologically plausible mechanism and pooling over a realistic number of retinal ganglion cells.
Excellent detection is made possible not just by the synchrony of ganglion cell firing, but also by its temporal profile. If instead of filtering the spike trains with f(t), we merely used the spike count in a single time window, the performance was very poor. For time windows between 5 and 40 ms, the spike count following the reversal exceeded all other time bins in no more than 26% of the trials. This means that error rates lower than 50% are not possible with a decoder that simply counts spikes in a single time window. The typical response of a ganglion cell involves initial firing due to smooth motion, followed by a pause, and then a sharp burst. The decoder’s temporal filter has a shape resembling a second-derivative in time, which matches well to this temporal profile of ganglion cell firing. Of course, it may be possible to recognize a motion reversal using the pattern of cells that fire together in a small time window rather than just the total number of cells by performing a nonlinear operation on retinal spike trains, although such a decoding mechanism would be more biophysically elaborate than the form we propose.
Neural Image of a Reversing Object
We can gain more insight into the manner in which the retina tracks a moving object that reverses direction by considering the “neural image” of ganglion cell activity. The neural image is the spatial pattern of firing in the ganglion cell population as a function of time (Berry et al., 1999). We calculated the neural image for each point in time by first plotting the (normalized) firing rate of all the recorded ganglion cells as a function of the distance between the location of the moving bar and the cell’s receptive field center coordinate (Figure 6A). Because we only recorded from a small fraction of all of the ganglion cells that respond to the bar’s motion, we included data taken from the same cells at multiple reversal locations. These data were then smoothed to make our best estimate of the spatial pattern of activity in the larger population of ganglion cells (see Experimental Procedures).
Long before motion reversal, the neural image travels along with the moving bar, despite the response latency of ganglion cells (Figure 6B). This spatial shift in the neural image results from motion anticipation, which is a simple form of prediction that the retina makes about the future location of a smoothly moving object (Berry et al., 1999). Immediately after the reversal, the neural image over-shoots the reversal location. This is a manifestation of the predictive nature of retinal motion processing: because of its response delay, the retina does not yet know that a reversal has occurred and its previous anticipation of the object’s location still operates. When information about the reversal becomes available, 100 ms later, the retinal response drops off. Then, at 250 ms after the reversal, the sharp, synchronous burst of firing occurs. Interestingly, the neural image rapidly accelerates in the new direction of motion at this time, nearly catching up to the bar’s location. Retinal firing then briefly drops off, and finally, at 400 ms after the reversal, a smooth motion response emerges with correct anticipation of the bar’s location. Again, the long delay before correct anticipation emerges is consistent with the interpretation that anticipation is a form of prediction that requires extensive integration over the object’s past trajectory.
Why does the neural image catch up at the time of the synchronous burst of firing? This can be seen as a consequence of the spatial asymmetry of reversal locations that drive a response (Figure 3). When the leading edge of an object moves over a ganglion cell but the trailing edge is still within the receptive field center, the cell participates in the synchronous burst of firing. However, when the leading edge of an object reverses before it has reached the receptive field center, the cell does not fire. Therefore, when an object reverses, ganglion cells far away in the new direction of motion will fire but those far away in the old direction of motion will not fire. As a result, the neural image is significantly shifted in the new direction of motion, helping to catch up to the object’s location. Another way of looking at the same data is to observe that during smooth motion, contrast gain control tends to localize ganglion cell firing near the leading edge of the object (Berry et al., 1999). But then, the reversal response can be triggered by either the leading or trailing edge. Thus, it helps to switch the location of the neural image to what will be the new leading edge of the object when it is moving in the other direction.
Generality of the Synchronized Response
Next, we asked whether this synchronized retinal response is unique to motion reversals or whether the retina conveys a more general signal of motion acceleration. To explore this question, we tested several other kinds of motion discontinuity. For both moderate acceleration and deceleration in the same direction of motion, there was no extra burst of firing (Figure 7A), nor was there any extra firing for an object that abruptly stopped its motion (Figure 7B, middle). These data indicate that the response is not a generic signal of acceleration or motion discontinuity.
However, when a stationary object suddenly began moving, there was a synchronized burst of firing in the retinal population (Figure 7B, lower). While this firing pattern resembles the response to motion reversal, there are several salient differences: the latency is much shorter (140 versus 260 ms), the peak firing rate is roughly twice as large, and the latency depends systematically on the speed (data not shown). Furthermore, different sets of ganglion cells exhibit responses to motion onset and motion reversal. These facts suggest that different circuit mechanisms may be responsible for the response to motion onset, although a more unified picture may emerge from further studies. We also found that bars that moved at a constant velocity and crossed elicited a response that was nearly identical to that for a reversing bar (Figure 7C). So, a synchronized burst of firing is not unique to motion reversal. However, the synchronized firing patterns elicited by both motion onset and crossing motion possess important differences that distinguish them from the reversal response (see below).
Receptive Field Dynamics
In order to gain greater insight about the reversal response, we asked whether the spatiotemporal dynamics of a ganglion cell’s receptive field could account for the phenomenon. Previous work has shown that ganglion cell responses to a smoothly moving bar are well predicted by a linear-nonlinear (LN) model with gain control (Berry et al., 1999; Shapley and Victor, 1981). The linear part of the model describes the basic spatial and temporal filtering of the receptive field through a convolution operation (Rodieck and Stone, 1965). A nonlinearity then truncates negative values to produce a firing rate as a function of time. Adding gain control to this model enables it to mimic the ability of ganglion cells to anticipate the leading edge of a moving bar (Berry et al., 1999).
For each cell, the model was fit to firing rate traces for a bar which reversed far enough away from the receptive field center such that each pass of the bar elicited its own smooth motion response (Figure 8A, lower). The same parameters were used for all other reversal locations (see legend). The model provided a good fit for responses to smooth motion, but failed to predict the reversal response for any reversal position (Figure 8B). The failure of the LN model lies in the initial spatiotemporal filtering, as this convolution often had a negative or zero value at the peak of the reversal response (Figure 8B). Across the population (n = 85 cells at 267 total reversal positions), the convolution value at the time of the reversal peak varied widely and on average was not significantly different than zero (p = 0.87). This analysis indicates that more sophisticated models of receptive field dynamics are needed to explain the reversal response (see below).
DISCUSSION
We have found that an abrupt reversal of motion evokes an extra burst of firing in a large fraction of all retinal ganglion cells in the salamander and mouse. This firing occurs with the same latency when either the leading or trailing edge of an object reverses on a cell’s receptive field. Because the latency is constant for all cells and all reversal locations, and because either edge can drive a response, motion reversal evokes a synchronous burst of firing in a population of roughly 100 ganglion cells. To our knowledge, this mode of firing has not previously been described.
Circuit Mechanism
As shown in Figure 8, the reversal response cannot be explained by models of the classical receptive field, even if they include gain control. However, the reversal response embodies several kinds of invariance to properties of the moving object, such as polarity, size, and speed. These invariances are not only useful for the neural code, but also have implications for the circuit mechanisms responsible. The fact that both bright and dark objects can evoke a reversal response suggests that the retinal interneurons involved in this processing have ON-OFF response characteristics, as has been described in many types of amacrine cells (Miller et al., 2006; Pang et al., 2002; Sakai and Naka, 1987a, 1987b), or perhaps come in analogous ON and OFF populations, as found for bipolar cells (Ghosh et al., 2004; Pang et al., 2004) and starburst amacrine cells (Famiglietti, 1991; Zhang and Wu, 2001). We also found that the reversal response can be triggered entirely by a single-contrast edge. Presumably, these two facts are related: the trailing edge of a dark moving object locally resembles a bright moving edge. Thus, if the reversal response can be triggered separately and similarly for either edge, the latency would be independent of the object’s size.
The fact that the response latency is roughly constant as a function of the speed of the moving object is challenging to explain, and places strong constraints on the circuit mechanism. One possibility is that the detection of a motion reversal involves the re-excitation of a retinal interneuron during the object’s return path at a fixed time delay with respect to when it was first excited by the initial path of the moving object. If such interneurons tile the retina and ganglion cells pool over many such subunits, excitation could follow a motion reversal at the same latency for a variety of speeds, even if driven by a different set of subunits at each speed. Furthermore, the subset of ganglion cells that receive input from these interneurons would fire synchronously, as we observe. An appealing candidate for such an interneuron is the bipolar cell, or perhaps even the axonal terminal of the bipolar cell, as these cells are likely to serve as motion-sensitive subunits in the receptive fields of Y-type ganglion cells (Demb et al., 2001; Shapley and Victor, 1979; Victor, 1988). Why might the bipolar cell be re-excited at a fixed time delay? One speculation is that motion in the initial direction leaves inhibition in its wake, and that the interneuron requires a roughly fixed amount of time to recover from this inhibition before it can be excited again (Dong and Werblin, 1998; Roska et al., 1998).
The case of crossing bars is important because the spatiotemporal pattern of photoreceptor activation on either side of the crossing point, taken alone, is identical to that of reversing bars. The observation that crossing bars elicit nearly the same response as a reversing bar (Figure 7C) therefore implies that the computation of motion reversal must be a fairly localized operation, consistent with the re-excitation model. Our experiments involving moving bars of different widths have implications for the spatial scale of the reversal computation. Both the leading and trailing edges of a relatively narrow moving bar can drive a reversal response (110 μm width, compared with a typical receptive field diameter of ~250 μm). But smooth motion of the same bar does not elicit separate responses from each edge. These data indicate that the receptive field of the reversal-sensitive subunits must be significantly smaller than the ganglion cell dendritic field and suggest that their size is less than ~100 μm. Again, these observations point to individual bipolar cells as a possible locus of reversal detection, although sharp excitation could also be generated by a narrow-field amacrine cell inhibiting a sustained amacrine cell and thereby transiently removing tonic inhibition from the ganglion cell.
The model of re-excitation of a retinal interneuron after recovery from inhibition also has implications for the locations that can trigger a reversal response. In this model, it is not the location at the time of reversal that determines the response. Instead, it is location of an edge at the time that interneurons recover from inhibition. Because of this additional delay, reversal locations somewhat beyond the boundary of the receptive field center can trigger a response via this mechanism. This may explain why we sometimes see quite strong responses at a distance of over two radii from the center coordinate. What is clear from our data is that the surround cannot directly generate a reversal response by itself, as there are several conditions that are not effective: (1) motion reversals that occur before reaching the center; (2) reversals with the trailing edge more than one surround radius (which is four to five center radii) past the center; and (3) reversals of a square above or below the center. Thus, all of our data is consistent with the idea that reversals are generated by the center mechanism, but possibly with a delay relative to the time of reversal.
Synchrony in the Neural Code
There are many ways that the ganglion cell population might encode a motion reversal. We find that the retina, rather than simply staying silent, sends a positive signal to the brain. The retina could contain a specialized class of ganglion cells that signal motion reversals. Instead, it employs the same ganglion cells that anticipate smooth motion and uses a distinct firing mode within the population: namely, a synchronized burst, which can be readily detected by subsequent neural circuits. Synchronized firing has been proposed as a special event in many neural populations (Abeles, 1991; Hatsopoulos et al., 1998; Meister, 1996; Vaadia et al., 1995), although the role of synchrony for population codes is an ongoing topic of research. This retinal reversal response is one of the few examples in which synchrony encodes a qualitatively different property of the world (Ishikane et al., 2005; Neuenschwander and Singer, 1996). By multiplexing the reversal signal onto optic nerve fibers using a synchrony code, the retina can send different kinds of visual messages using fewer optic nerve fibers (Meister, 1996).
What is the purpose of the retina’s response to a motion reversal? During smooth motion, anticipation corrects for the retina’s response latency, and as a result, the peak firing rate of a ganglion cell represents the object’s true, current location (Berry et al., 1999). Immediately after a reversal of motion, the retina cannot anticipate the object’s location, which can lead to large localization errors. The synchronized burst of ganglion cell activity can therefore serve to identify a violation in the retina’s ongoing prediction of a moving object’s location.
Why might the brain need to know that retinal motion prediction has been violated? One intriguing possibility comes from a consideration of how an animal coordinates its motor output with moving objects in the environment. Motion anticipation corrects for neural delays in the retina itself, but this degree of anticipation is not sufficient for coordinated movement. There are additional neural delays in sensory-motor pathways downstream of the retina, and there are motor delays for limbs to reach their intended locations. Thus, subsequent neural circuits in the brain may need to perform additional extrapolation of a moving object’s trajectory. Evidence from human psychophysics indicates that in many behavioral contexts, both hand and eye movements embody such extrapolations (Barnes and Asselman, 1991; Brouwer et al., 2003; Land and McLeod, 2000; Pavel et al., 1992; Smeets and Brenner, 1995).
Although the neural mechanisms that underlie central motion extrapolation are not known, the only way, in principle, to make such an extrapolation is to use the past trajectory of an object’s motion. However, when there is a sudden reversal of motion, the entire preceding trajectory will be not just irrelevant, but actually misleading about the future location of the moving object. As a result, the central mechanism of motion extrapolation would benefit from being “initialized,” so that it can begin to accumulate a prediction of the object’s future location that uses only the relevant portion of the trajectory. Perhaps the synchronous burst produced by the retina following a motion reversal may play a role in initializing central motion extrapolation.
Different Kinds of Synchronized Firing
Why might the retina respond to motion onset and reversal but not to other accelerations at the speeds we tested? One possibility is that less severe accelerations lead to a small enough error in the retinal representation of the object’s location such that no explicit violation signal is needed. For instance, when motion suddenly stops, the neural image overshoots the actual position of the object (Figure 6), but this overshoot is less than 100 μm, which is smaller than the receptive field size of an individual ganglion cell. In contrast, when the object reverses its direction of motion, the localization error at the time of the reversal response is more than 250 μm compared with the center of the bar. As the size of the retina’s localization error should depend on the speed of motion as well as the magnitude of the acceleration, it will be fruitful to analyze the neural image following a variety of different motion discontinuities.
Another related possibility is that synchronized firing is only needed to initialize mechanisms of central motion extrapolation. In the case of motion onset, such central mechanisms would not be engaged prior to the discontinuity, but for other kinds of mild acceleration, they would already be operating. In this context, it is important to consider how the brain might specifically recognize the appropriate motion discontinuities. What the brain needs is a detector that is triggered by either motion onset or motion reversal, but not by smooth motion or other mild accelerations. In fact, if retinal spike trains are imported into to the same reversal detector as described above (Figure 5), we get excellent discrimination between motion onset and smooth motion (0 errors in 240 trials using all 71 cells). The decoder achieves the same performance for discrimination between motion that suddenly starts (detection) and motion that suddenly stops (no detection). Such a discontinuity detector could thus serve as the trigger for initializing central motion extrapolation mechanisms.
At the same time, this detector is unable to reliably distinguish motion onset from motion reversal. However, if we use the same form of decoding algorithm, but with a different linear filter that is optimized for distinguishing start from reversal, we can again achieve excellent performance (0 errors in 240 trials). This demonstrates that the ganglion cell population does in fact convey information about which kind of motion discontinuity has occurred, and it suggests that a more sophisticated decoding process, using either two successive simple stages or a single complex mechanism, can perform this discrimination.
A similar issue arises in how the brain might interpret the response to crossing bars. Although this response seems very similar to that for a reversing bar, the retina still provides information that can distinguish between crossing and reversing bars: the crossing bar stimulus will generate synchronized firing in two populations of ganglion cells on either side of the crossing point, while a simple reversal will only generate synchronized firing in a single group of cells offset from the reversal point in the new direction of motion. Thus, the brain can discriminate between reversal and crossing by pooling over an even larger population of ganglion cells.
These analyses indicate that synchronized firing in the population of retinal ganglion cells has the potential to encode qualitatively different events in the visual world. While the connection between synchronized firing and central mechanisms of motion extrapolation is highly speculative, it may help to make sense of this surprising aspect of the retinal code as well as motivate further experiments.
EXPERIMENTAL PROCEDURES
Recording
Pieces of retina obtained from larval tiger salamanders (Ambystoma tigrinum) and mice (strain C57/BL6) were perfused continuously with oxygenated Ringer’s medium. Ganglion cell spikes were recorded extracellularly from a multielectrode array. Salamander recordings were made at room temperature, and mouse recordings were made at 36°C. Details of the recording and spike sorting are described elsewhere (Segev et al., 2004).
Visual Stimulation
Visual stimuli were presented on a computer monitor running at 120 Hz (Puchalla et al., 2005). Moving bars were presented on a gray background and were 110 μm wide, traveling at 1.32 mm/s unless otherwise noted. Bars reversed at 9–15 locations near the receptive fields of the ganglion cells with 55 or 110 μm separations between locations. Parameters of other stimuli are noted in figure legends. Moving squares were 165 × 165 μm in size and also traveled at 1.32 mm/s. Squares reversed at a grid of 7 × 7 locations over the multielectrode array.
Receptive Fields
Spatiotemporal receptive fields were measured by reverse correlation to random flicker presented at 60 Hz. In experiments with moving bars, we mapped receptive fields with flickering black and white strips 22 μm wide and oriented parallel to the moving bar stimuli. In experiments with moving squares, we mapped receptive fields with flickering squares 55 μm on a side. A 1D or 2D Gaussian was fit to the spatial profile of the center to identify a center coordinate xi and center radius σi for each cell i [29]. The uncertainty in the center coordinate ranged from 0.95 to 7.6 μm with an average of 2.3 μm; the uncertainty in the center radius averaged 3.3 μm. Thus, retinal locations measured in units of center radii had a combined uncertainty of ~3% (Figure 3 and Figure 4). The spread of receptive fields in a group of cells, Δ, was defined as the standard deviation of the set of center coordinates, {xi}. For making pictograms, we displayed a circle with a size given by the boundary of the receptive field center. The boundary was defined as the point at which the spatial profile changed polarity.
Reversal Response
Cells were classified as reversal responsive if they had a peak in their firing rate, as measured by the peristimulus time histogram (PSTH), that was greater than 10 Hz between 200 ms and 300 ms after the reversal in the salamander and between 150 ms and 250 ms in the mouse. We chose this relatively conservative criterion to exclude cells with low firing rates and cells with receptive fields too far from the location of motion reversals. We also required that the latency of the reversal response remained constant for at least three different reversal positions. This was to make sure that we did not mistake a smooth motion response that happened to come at roughly the right time for a reversal response. Our criteria are likely to disqualify some cells that actually respond to motion reversal, mostly because we only sample a discrete number of reversal locations, so our estimate of the fraction of cells participating is a lower bound. All population averages and decoding analysis include only reversal responsive cells; figure legends indicate whether population averages include all tested locations for reversal responsive cells (Figure 2) or only locations with a firing rate greater than 10 Hz (Figure 4).
Decoder
Reversals were detected using a decoder consisting of a linear filter applied to spikes from all the ganglion cells, followed by summation and a threshold operation. The optimal filter f(t) is a function of time that, when convolved with the average population firing rate accumulated over all trials, r(t), most closely resembles a target function φ(t), which represents the detection of the reversal event following the reversal response. The target function was chosen to have a value of one for a 100 ms window beginning 350 ms after the reversal, and zero everywhere else (Figure 5C). We solved for this condition in the frequency domain:
where
is the Fourier transform of f(t).
For correct detection, the decoder had to exceed threshold within the time window defined by the target function. All other threshold crossings were counted as errors (false alarms). In addition, any threshold crossings that occurred during other stimulus trials, such as smooth motion, were counted as false alarms. Because we measured error rates on a per trial basis, we allowed no more than one false alarm per trial. As false alarms could occur anywhere over a 1 s period in each trial (except for the 100 ms wide target region), the error rate for a decoder that selected random times would be 90% for reversal trials only and 95% for reversal and smooth motion trials combined. For purposes of cross-validation, the optimal filter was constructed using half of the data, then tested on the other half, and vice versa.
Pooling over Many Cells
If we assume that reversal responses can arise from locations up to two radii away on either side of the center coordinate in either the horizontal or vertical direction (Figure 3 and Figure 4), then there is a circular region of approximately two center radii that can sense the reversal of even a very small object. Assuming a center radius of 125 μm and a density of 1400 cells/mm2 in salamander (Segev et al., 2004), there are ~275 ganglion cells in this area. If 278/745 = 37% of them have a reversal response, then the decoder can pool over ~100 cells. Reversal of larger objects would engage even more ganglion cells.
Neural Image
We first calculated the PSTH over 75 stimulus trials for each ganglion cell in 25 ms bins. Then, for each time step, we plotted each cell’s firing rate versus the distance between its receptive field center coordinate, xi, and the location at which the moving bar reversed direction. The firing rate was normalized by each cell’s peak firing rate in response to smooth motion. Data were pooled over 9–15 different reversal locations and smoothed with a Gaussian filter that had a width of 20 μm (Figure 6A). For each point in time, the location and amplitude of the peak activity of the neural image was found (Figure 6B). In these experiments, the neural image was found to be a unimodal distribution at all points in time, so that its peak was unambiguous.
Acknowledgments
We thank R.A. da Silveira for useful discussions and S. Palmer for comments on the manuscript. This work was supported by the National Eye Institute (R01 EY14196).
References
- Abeles M. Corticonics: Neural Circuits of the Cerebral Cortex. New York: Cambridge University Press; 1991. [Google Scholar]
- Ahlfors SP, Simpson GV, Dale AM, Belliveau JW, Liu AK, Korvenoja A, Virtanen J, Huotilainen M, Tootell RB, Aronen HJ, Ilmoniemi RJ. Spatiotemporal activity of a cortical network for processing visual motion revealed by MEG and fMRI. J Neurophysiol. 1999;82:2545–2555. doi: 10.1152/jn.1999.82.5.2545. [DOI] [PubMed] [Google Scholar]
- Barlow HB, Derrington AM, Harris LR, Lennie P. The effects of remote retinal stimulation on the responses of cat retinal ganglion cells. J Physiol. 1977;269:177–194. doi: 10.1113/jphysiol.1977.sp011898. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Barnes GR, Asselman PT. The mechanism of prediction in human smooth pursuit eye movements. J Physiol. 1991;439:439–461. doi: 10.1113/jphysiol.1991.sp018675. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Becker W, Fuchs AF. Prediction in the oculomotor system: smooth pursuit during transient disappearance of a visual target. Exp Brain Res. 1985;57:562–575. doi: 10.1007/BF00237843. [DOI] [PubMed] [Google Scholar]
- Berry MJ, II, Brivanlou IH, Jordan TA, Meister M. Anticipation of moving stimuli by the retina. Nature. 1999;398:334–338. doi: 10.1038/18678. [DOI] [PubMed] [Google Scholar]
- Brouwer AM, Middelburg T, Smeets JB, Brenner E. Hitting moving targets: a dissociation between the use of the target’s speed and direction of motion. Exp Brain Res. 2003;152:368–375. doi: 10.1007/s00221-003-1556-8. [DOI] [PubMed] [Google Scholar]
- Carpenter RHS. Movements of the Eyes. London: Pion; 1988. [Google Scholar]
- Clarke PG. Visual evoked potentials to sudden reversal of the motion of a pattern. Brain Res. 1972;36:453–458. doi: 10.1016/0006-8993(72)90754-8. [DOI] [PubMed] [Google Scholar]
- De Lucia PR, Cochran EL. Perceptual information for batting can be extracted throughout a ball’s trajectory. Percept Mot Skills. 1985;61:143–150. doi: 10.2466/pms.1985.61.1.143. [DOI] [PubMed] [Google Scholar]
- Demb JB, Zaghloul K, Haarsma L, Sterling P. Bipolar cells contribute to nonlinear spatial summation in the brisk-transient (Y) ganglion cell in mammalian retina. J Neurosci. 2001;21:7447–7454. doi: 10.1523/JNEUROSCI.21-19-07447.2001. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Dong CJ, Werblin FS. Temporal contrast enhancement via GABAC feedback at bipolar terminals in the tiger salamander retina. J Neurophysiol. 1998;79:2171–2180. doi: 10.1152/jn.1998.79.4.2171. [DOI] [PubMed] [Google Scholar]
- Famiglietti EV. Synaptic organization of starburst amacrine cells in rabbit retina: analysis of serial thin sections by electron microscopy and graphic reconstruction. J Comp Neurol. 1991;309:40–70. doi: 10.1002/cne.903090105. [DOI] [PubMed] [Google Scholar]
- Fukushima K, Yamanobe T, Shinmei Y, Fukushima J. Predictive responses of periarcuate pursuit neurons to visual target motion. Exp Brain Res. 2002;145:104–120. doi: 10.1007/s00221-002-1088-7. [DOI] [PubMed] [Google Scholar]
- Ghosh KK, Bujan S, Haverkamp S, Feigenspan A, Wassle H. Types of bipolar cells in the mouse retina. J Comp Neurol. 2004;469:70–82. doi: 10.1002/cne.10985. [DOI] [PubMed] [Google Scholar]
- Green DM, Swets JA. Signal Detection Theory and Psychophysics. New York: Wiley; 1966. [Google Scholar]
- Hatsopoulos NG, Ojakangas CL, Paninski L, Donoghue JP. Information about movement direction obtained from synchronous activity of motor cortical neurons. Proc Natl Acad Sci USA. 1998;95:15706–15711. doi: 10.1073/pnas.95.26.15706. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Heinen SJ, Liu M. Single-neuron activity in the dorsomedial frontal cortex during smooth-pursuit eye movements to predictable target motion. Vis Neurosci. 1997;14:853–865. doi: 10.1017/s0952523800011597. [DOI] [PubMed] [Google Scholar]
- Ishikane H, Gangi M, Honda S, Tachibana M. Synchronized retinal oscillations encode essential information for escape behavior in frogs. Nat Neurosci. 2005;8:1087–1095. doi: 10.1038/nn1497. [DOI] [PubMed] [Google Scholar]
- Kettner RE, Mahamud S, Leung HC, Sitkoff N, Houk JC, Peterson BW, Barto AG. Prediction of complex two-dimensional trajectories by a cerebellar model of smooth pursuit eye movement. J Neurophysiol. 1997;77:2115–2130. doi: 10.1152/jn.1997.77.4.2115. [DOI] [PubMed] [Google Scholar]
- Klam F, Petit J, Grantyn A, Berthoz A. Predictive elements in ocular interception and tracking of a moving target by untrained cats. Exp Brain Res. 2001;139:233–247. doi: 10.1007/s002210100759. [DOI] [PubMed] [Google Scholar]
- Krauzlis RJ, Lisberger SG. Temporal properties of visual motion signals for the initiation of smooth pursuit eye movements in monkeys. J Neurophysiol. 1994;72:150–162. doi: 10.1152/jn.1994.72.1.150. [DOI] [PubMed] [Google Scholar]
- Land MF, McLeod P. From eye movements to actions: how batsmen hit the ball. Nat Neurosci. 2000;3:1340–1345. doi: 10.1038/81887. [DOI] [PubMed] [Google Scholar]
- MacAvoy MG, Gottlieb JP, Bruce CJ. Smooth-pursuit eye movement representation in the primate frontal eye field. Cereb Cortex. 1991;1:95–102. doi: 10.1093/cercor/1.1.95. [DOI] [PubMed] [Google Scholar]
- MacKay DM, Rietveld WJ. Electroencephalogram potentials evoked by accelerated visual motion. Nature. 1968;217:677–678. doi: 10.1038/217677a0. [DOI] [PubMed] [Google Scholar]
- Maimon G, Assad JA. A cognitive signal for the proactive timing of action in macaque LIP. Nat Neurosci. 2006;9:948–955. doi: 10.1038/nn1716. [DOI] [PubMed] [Google Scholar]
- McIlwain JT. Some evidence concerning the physiological basis of the periphery effect in the cat’s retina. Exp Brain Res. 1966;1:265–271. doi: 10.1007/BF00234346. [DOI] [PubMed] [Google Scholar]
- Medina JF, Carey MR, Lisberger SG. The representation of time for motor learning. Neuron. 2005;45:157–167. doi: 10.1016/j.neuron.2004.12.017. [DOI] [PubMed] [Google Scholar]
- Meister M. Multineuronal codes in retinal signaling. Proc Natl Acad Sci USA. 1996;93:609–614. doi: 10.1073/pnas.93.2.609. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Miller RF, Staff NP, Velte TJ. Form and function of ON-OFF amacrine cells in the amphibian retina. J Neurophysiol. 2006;95:3171–3190. doi: 10.1152/jn.00090.2005. [DOI] [PubMed] [Google Scholar]
- Neuenschwander S, Singer W. Long-range synchronization of oscillatory light responses in the cat retina and lateral geniculate nucleus. Nature. 1996;379:728–732. doi: 10.1038/379728a0. [DOI] [PubMed] [Google Scholar]
- Pang JJ, Gao F, Wu SM. Segregation and integration of visual channels: layer-by-layer computation of ON-OFF signals by amacrine cell dendrites. J Neurosci. 2002;22:4693–4701. doi: 10.1523/JNEUROSCI.22-11-04693.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pang JJ, Gao F, Wu SM. Stratum-by-stratum projection of light response attributes by retinal bipolar cells of Ambystoma. J Physiol. 2004;558:249–262. doi: 10.1113/jphysiol.2004.063503. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Pavel M. Predictive control of eye movement. Amsterdam: Elsevier; 1990. [PubMed] [Google Scholar]
- Pavel M, Cunningham H, Stone V. Extrapolation of linear motion. Vision Res. 1992;32:2177–2186. doi: 10.1016/0042-6989(92)90078-w. [DOI] [PubMed] [Google Scholar]
- Puchalla JL, Schneidman E, Harris RA, Berry MJ. Redundancy in the population code of the retina. Neuron. 2005;46:493–504. doi: 10.1016/j.neuron.2005.03.026. [DOI] [PubMed] [Google Scholar]
- Robinson DA. The mechanics of human smooth pursuit eye movement. J Physiol. 1965;180:569–591. doi: 10.1113/jphysiol.1965.sp007718. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rodieck RW, Stone J. Response of cat retinal ganglion cells to moving visual patterns. J Neurophysiol. 1965;28:819–832. doi: 10.1152/jn.1965.28.5.819. [DOI] [PubMed] [Google Scholar]
- Roska B, Nemeth E, Werblin FS. Response to change is facilitated by a three-neuron disinhibitory pathway in the tiger salamander retina. J Neurosci. 1998;18:3451–3459. doi: 10.1523/JNEUROSCI.18-09-03451.1998. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sakai HM, Naka K. Signal transmission in the catfish retina. IV. Transmission to ganglion cells. J Neurophysiol. 1987a;58:1307–1328. doi: 10.1152/jn.1987.58.6.1307. [DOI] [PubMed] [Google Scholar]
- Sakai HM, Naka K. Signal transmission in the catfish retina. V. Sensitivity and circuit. J Neurophysiol. 1987b;58:1329–1350. doi: 10.1152/jn.1987.58.6.1329. [DOI] [PubMed] [Google Scholar]
- Segev R, Goodhouse J, Puchalla J, Berry MJ., 2nd Recording spikes from a large fraction of the ganglion cells in a retinal patch. Nat Neurosci. 2004;7:1154–1161. doi: 10.1038/nn1323. [DOI] [PubMed] [Google Scholar]
- Shapley RM, Victor JD. Nonlinear spatial summation and the contrast gain control of cat retinal ganglion cells. J Physiol. 1979;290:141–161. doi: 10.1113/jphysiol.1979.sp012765. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Shapley RM, Victor JD. How the contrast gain control modifies the frequency responses of cat retinal ganglion cells. J Physiol. 1981;318:161–179. doi: 10.1113/jphysiol.1981.sp013856. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Smeets JB, Brenner E. Prediction of a moving target’s position in fast goal-directed action. Biol Cybern. 1995;73:519–528. doi: 10.1007/BF00199544. [DOI] [PubMed] [Google Scholar]
- Suh M, Leung HC, Kettner RE. Cerebellar flocculus and ventral paraflocculus Purkinje cell activity during predictive and visually driven pursuit in monkey. J Neurophysiol. 2000;84:1835–1850. doi: 10.1152/jn.2000.84.4.1835. [DOI] [PubMed] [Google Scholar]
- Vaadia E, Haalman I, Abeles M, Bergman H, Prut Y, Slovin H, Aertsen A. Dynamics of neuronal interactions in monkey cortex in relation to behavioural events. Nature. 1995;373:515–518. doi: 10.1038/373515a0. [DOI] [PubMed] [Google Scholar]
- Victor JD. The dynamics of the cat retinal Y cell subunit. J Physiol. 1988;405:289–320. doi: 10.1113/jphysiol.1988.sp017334. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Zhang J, Wu SM. Immunocytochemical analysis of cholinergic amacrine cells in the tiger salamander retina. Neuroreport. 2001;12:1371–1375. doi: 10.1097/00001756-200105250-00017. [DOI] [PubMed] [Google Scholar]