Sophisticated Temporal Pattern Recognition in Retinal Ganglion Cells

Abstract

Pattern recognition is one of the most important tasks of the visual system, and uncovering the neural mechanisms underlying recognition phenomena has been a focus of researchers for decades. Surprisingly, at the earliest stages of vision, the retina is capable of highly sophisticated temporal pattern recognition. We stimulated the retina of tiger salamander (Ambystoma tigrinum) with periodic dark flash sequences and found that retinal ganglion cells had a wide variety of different responses to a periodic flash sequence with many firing when a flash was omitted. The timing of the omitted stimulus response (OSR) depended on the period, with individual cells tracking the stimulus period down to increments of 5 ms. When flashes occurred earlier than expected, cells updated their expectation of the next flash time by as much as 50 ms. When flashes occurred later than expected, cells fired an OSR and reset their temporal expectation to the average time interval between flashes. Using pharmacology to investigate the retinal circuitry involved, we found that inhibitory transmission from amacrine cells was not required, but on bipolar cells were required. The results suggest a mechanism in which the intrinsic resonance of on bipolars leads to the OSR in ganglion cells. We discuss the implications of retinal pattern recognition on the neural code of the retina and visual processing in general.

INTRODUCTION

In a number of sensory systems, it has been suggested that resonant oscillations in single neurons play a role in pattern recognition (Hutcheon and Yarom 2000). Resonance has been shown to be involved in bat echolocation (Galazyuk and Feng 2001), insect call recognition (Bush and Schul 2006), tuning of hair cells in the turtle cochlea (Art and Fettiplace 1987; Jones et al. 1999), and temporal filtering in the electric fish (Fortune and Rose 2003). Izhikevich (2001) described a model of how such “resonate-and-fire” neurons could provide sensitivity to fine temporal structure.

Here we describe a surprisingly sophisticated form of temporal pattern recognition in the retina and suggest that resonant oscillations play a role in this system as well. Our previous work (Schwartz et al. 2007) has shown that retinal ganglion cells fire to the omission of a flash in a periodic sequence over a range of stimulus frequencies. This firing event is called an omitted stimulus response (OSR). The current findings extend this result in a number of ways.

First, we offer a detailed characterization of the diversity of responses in a large population of ganglion cells (n = 434) to a simple periodic sequence. Many cells show evidence of resonant responses both during the periodic sequence and after the sequence ends. Next, we consider flash sequences that deviate from perfect periodicity. Not only are the pattern recognition capabilities of ganglion cells robust to these deviations, but cells are also able to change their predictions dynamically according to estimates of the last flash interval as well as the recent average of flash intervals. Using pharmacology to dissect the circuitry of the retina, we find that inhibitory transmission from amacrine cells is not required for the OSR, but on bipolar cells are required. In light of these results, we describe how a circuit involving intrinsic resonance in bipolar cells may be able to explain the OSR.

METHODS

Recording

Pieces of retina obtained from larval tiger salamanders (Ambystoma tigrinum) were perfused continuously with oxygenated Ringer's medium. Ganglion cell spikes were recorded extracellularly from a multielectrode array at room temperature. Details of the recording and spike sorting are described elsewhere (Segev et al. 2004).

Visual stimulation

Visual stimuli were presented on a white light-emitting diode mounted directly underneath the retina. Mean light levels were 30 lux. At 625 nm, the peak sensitivity of the salamander L cone, this corresponds to 2.20 × 10⁵ photons·μm^–2·s^–1. Normal flash sequences contained 16 or 32 flashes presented at 12 Hz with 1 s between trials. Variations of this stimulus pattern are described in the text. All firing rate histograms include ≥50 trials. For jittered flash sequences (Fig. 7), the time of each flash (except the first and last) was jittered by a random time shift taken from a Gaussian distribution with SD of 5 ms. Ordering of the trials for the histograms in Fig. 7 is post hoc.

FIG. 7.

Jittered flash sequences elicit different types of OSRs. A–D: data are plotted as spike rasters with flash times in blue, spike times in red, and the time of the missing flash (assuming mean flash interval of 83 ms) in green for 4 different cells (A–D). Flashes are presented with a mean interval of 83 ms and SD of 5 ms (see methods). Rasters are aligned to the time of the last flash and ordered by the time of the second to last flash. Yellow, orange, and pink highlights and purple crosses are spike time predictions (see text).

Pharmacology

All pharmacological agents were dissolved into a separate batch of Ringer's medium and washed in through the perfusion system after control recordings were made. Picrotoxin (50–100 μM) and strychnine (20–100 μM) were mixed together and washed in for 40 min before data were collected. The same technique was used for recordings with (1,2,5,6-tetrahydropyridin-4-yl)methylphosphinic acid (TPMPA, 200 μM), SR95531 (gabazine, 100 μM), atropine (10 μM), d-tubocurarine (100 μM), (RS)-α-cyclopropyl-4-phosphonophenylglycine (CPPG, 200 μM), 6-cyano-7-nitroquinoxaline-2,3-dione (CNQX, 100 μM), and d-2-amino-7-phosphonoheptanoic acid (d-AP7, 180 μM). Amacrine blocking experiments were also done with baclofen (100 μM) and phaclofen (500 μM). 2-Amino-4-phosphonobutyric acid (APB) was used in either the dl form (100 μM) or the l form (10–20 μM) and washed in for 25 min. All chemicals were from Sigma (St. Louis, MO), except l-APB, CPPG, and d-AP7 from Tocris Bioscience (Ellisville, MO).

RESULTS

Variety of responses to flash trains

We recorded from a total of 434 cells from 18 experiments with identical flash trains consisting of 16 flashes of 40-ms duration presented at 12/s. Responses to this stimulus varied dramatically among the cells (Fig. 1). To describe the variety of responses, we separated the response time into three epochs: the first three flashes (start response), the remaining 13 flashes (sustained response), and the period following the first missing flash (omitted stimulus response or OSR).

FIG. 1.

Retinal ganglion cells show a variety of responses to a flash train. Flashes are shown in solid blue, missing flash is dotted, and the missing flash time is shown in green. Poststimulus time histograms (PSTHs) of ganglion cells are shown in red. A: types of responses to the first 3 flashes in the sequence include (from top to bottom) single peak, decaying, no response, complex, and facilitating. B: the remaining flashes in the sequence also elicit 5 different response types (from top to bottom): regular, decaying, no response, harmonic, and facilitating. After the missing flash, cells have one of 5 omitted stimulus response (OSR) types (not in order): no response, single peak, weak response, double peak, or ringing.

Start responses (first epoch) were divided into five types (Fig. 1A): a single peak of firing with no other responses, firing decaying steadily during the first three flashes, no response at all, complex responses, and firing facilitating during the first three flashes. Cells with more than three peaks of firing or with firing that changed in an irregular fashion during the first epoch were classified as complex. Sustained responses (second epoch) were also divided into five types (Fig. 1B): regular firing to every flash, steadily decaying firing, no response (or very weak response), irregular firing with period-doubling or more complex harmonics, and steadily facilitating firing. It is important to note that the ganglion cell population did not form clearly distinct clusters according to these types and that the boundaries between our types were often somewhat arbitrary. Instead, these categories are helpful for a qualitative description of the diversity of firing profiles we observed.

Firing after the end of the flash sequence, during the OSR period, was classified as single peak, weak (<10-Hz peak firing rate), no response, double peak, or ringing. Examples of these types are shown in both panels of Fig. 1. The strength of the OSR had a broad distribution with a tail of very large values (Fig. 2). Cells classified as having a significant OSR (with either single or multiple peaks) had a mean count of 1.29 ± 1.57 spikes/trial (SD, n = 146 cells) in the 120 ms following the omitted flash; the median value was 0.84 spike/trial and 90% of the values were in the range 0.16–4.28 spikes/trial. The ratio of spikes elicited by the omitted flash to those elicited by the first flash also formed a broad distribution with some cells having an OSR six- to eightfold stronger than the response to a single flash. The mean ratio was 1.81 ± 1.6 (SD) with a median of 1.21 and 90% of the values in the range 0.40–6.15. Similar results were found if we characterized the strength of the OSR using a cell's peak firing rate rather than the integrated number of spikes per stimulus trial (data not shown).

FIG. 2.

Flash train response and OSR size statistics. A and B: histograms of OSR size for cells characterized as having single-, multiple-, or double-peaked OSRs measured in spikes per trial (A) or normalized by the mean response to the first 3 flashes (B).

The number of cells having each kind of response type is summarized in Table 1. Cells with no start response were very rare (2/434). This might seem surprising because about 15% of ganglion cells are on-type, measured by their reverse correlation during random flicker stimulation (Segev et al. 2006), and are thus not expected to respond to a dark flash. However, we found that most on-type cells responded in some way in the first epoch of the flash sequence, often exhibiting complex start responses. Ganglion cells with facilitating responses either in epoch one or in epoch two were also uncommon. To compare with categories described in our previous study (Schwartz et al. 2007), we note that we did not previously distinguish between single peak and complex start responses, nor did we distinguish between regular, decaying, facilitating, or harmonic sustained responses. By these definitions, we would have n = 96 “start-end” cells and n = 89 “sustained” cells. “End-only” cells were more common in the mouse than in the salamander.

TABLE 1.

Classification of cells by firing pattern

Start Type	Number of Cells	Sustained Type	Number of Cells	OSR Type	Number of Cells
Single Peak	190	Regular	46	Single Peak	110
Decaying	85	Decaying	64	Weak Response	105
No Response	2	No Response	201	No Response	183
Complex	142	Harmonic	106	Double Peak	20
Facilitating	15	Facilitating	17	Ringing	16

Cells are classified by firing patterns in the start epoch (first 3 flashes), the sustained epoch (subsequent 13 flashes), and the OSR epoch (period following the missing flash).

Harmonic sustained responses were reasonably common (106/434 = 24%). This category itself included considerable diversity. Some cells showed period-doubling (Fig. 3A). This phenomenon is well known for nonlinear oscillators (Guckenheimer and Holmes 1983) and its presence in ganglion cell spike trains may give clues about the circuit mechanism (Crevier and Meister 1998). Cells with period-doubled sustained responses could have any kind of OSR: single peak, multiple peak, or none. Other cells exhibited a period-tripled response, with either enhancement or suppression of the firing every third flash, or a significantly enhanced response to one particular flash late in the sequence (Fig. 3B). Both of these kinds of responses are more exotic in terms of their underlying dynamics. About a quarter of all cells with a strong OSR had multiple peaks (double peak or ringing: 36/146 = 25%). This phenomenon similarly suggests that retinal circuitry is capable of resonating in response to a periodic input. We will return to this idea in the discussion.

FIG. 3.

Retinal ganglion cells show a variety of harmonic sustained responses. Flashes are shown in solid blue, missing flash is dotted, and the missing flash time is shown in green. PSTHs of ganglion cells are shown in red. A: cells exhibiting period-doubling during their sustained response included examples with (from top to bottom) sudden onset during the flash sequence, irregular period-doubling, and late onset from regular firing. B: cells with more complex harmonic responses included (from top to bottom) enhancement of the response every 3rd flash, suppression every 3rd flash, and enhancement on the 13th flash. Cells with harmonic sustained responses could have any OSR type (shown on the right).

Table 2 shows the number of cells displaying each combination of response type in the three epochs. For a given response type in one epoch, a wide distribution of behaviors was observed in the other two epochs. However, a cell's Start Type did have a significant correlation with its OSR Type (chi-square test of independence, P < 10^–3). If the Start Type was single peak or decaying, then an OSR was less likely and those that occurred were less likely to have multiple peaks. In contrast, if the Start Type was complex, then an OSR was more likely and those that occurred were more likely to be multi-peaked. The Start Type had an even more significant correlation with the Sustained Type (P < 10^–4). If the Start Type was decaying, then decaying or no sustained responses were more common and regular or harmonic sustained responses were less common. In contrast, if the Start Type was complex, then decaying or absent sustained responses were less common and regular or harmonic sustained responses were more common. There was no significant overall relationship between Sustained Type and OSR Type (P = 0.22). However, when we looked only at whether cells had an OSR, we found that an OSR was more likely for cells with regular sustained responses and less likely for cells with no sustained firing (P < 0.01).

TABLE 2.

Relationship between firing patterns in different epochs

A. Start versus OSR Type
Start Type	Number of Cells	Single Peak	Weak Response	No Response	Double Peak	Ringing
Single Peak	190	23%	23%	50%	1.1%	3.2%
Decaying	85	24%	24%	48%	3.5%	1.2%
Complex	142	29%	25%	31%	9.9%	4.9%
Facilitating	15	33%	33%	20%	0.0%	13.0%
Total	434	25%	24%	42%	4.6%	3.9%

B. Start versus Sustained Type
Start Type	Number of Cells	Regular	Decaying	No Response	Harmonic	Facilitating
Single Peak	190	11%	14%	51%	21%	3.2%
Decaying	85	4.7%	21%	60%	11%	3.5%
Complex	142	15%	9.2%	34%	37%	4.9%
Facilitating	15	0%	40%	20%	33%	7.0%
Total	434	11%	15%	46%	24%	3.9%

C. Sustained versus OSR Type
Sustained Type	Number of Cells	Single Peak	Weak Response	No Response	Double Peak	Ringing
Regular	46	28%	22%	33%	13.0%	4.3%
Decaying	64	28%	25%	41%	3.1%	3.1%
No Response	201	24%	24%	48%	2.5%	2.0%
Harmonic	106	25%	25%	39%	4.7%	6.6%
Facilitating	17	29%	24%	29%	12.0%	6.0%
Total	434	25%	24%	42%	4.6%	3.9%

None of the response types we measured had any correspondence to basic response properties of ganglion cells like receptive field size, spike-triggered average, flash response latency, or on–off index. The response to a single flash could not predict the response to a flash train or the presence or the absence of an OSR, but a number of trends emerged from this comparison. Cells characterized as complex Start Type had a very similar burst following a single flash (Fig. 4, A and B). Other cells showed a double response to a single flash. This second response was cancelled when the flash train was presented. Both cells that were silent during the flash train (Fig. 4, C and D) and those with a sustained or decaying response (Fig. 4, E and F) showed the same cancellation of the double flash response. We will return to the interpretation of this cancellation in the discussion.

FIG. 4.

Comparison of responses to flash trains and single flashes. Each panel shows the response of an individual ganglion cell to a flash train (top) and a single flash (bottom). A, C, D, and F: 18-Hz flash trains. Flashes are 20 ms in duration. B and E: 12-Hz flash trains. A and B: complex start type cells retain their firing patterns after a single flash. C–F: silent, decaying, and sustained cells that have double responses to a single flash have this second response suppressed when a flash train is presented.

Precise timing

Although previous work has shown that the OSR latency shifts predictively when averaged over a population of cells (Schwartz et al. 2007), we wanted to see whether individual ganglion cells could track small changes in the stimulus period. Flash trains were presented with interflash intervals of 63 to 83 ms in increments of 5 ms. The firing rate of one ganglion cell to these flash trains showed that the OSR latency shifted consistently for each interval (Fig. 5A). Plotting OSR latency versus stimulus period showed a linear relationship with a slope near one for the three individual cells tested in this experiment (Fig. 5B). This means that the OSR latency shifted predictively as a function of the stimulus period, such that the latency was constant with respect to the time of the missing flash with an accuracy of a few milliseconds (Fig. 5C).

FIG. 5.

Single cells shift OSR timing with stimulus frequency. A: response of a single cell to periodic flash sequences with intervals of either 83, 73, or 63 ms (top to bottom). Traces are aligned to the time of the last real flash; missing flash is shown in a dotted line; OSR is indicated with an arrow. B: latency of OSR firing peak vs. stimulus period for cell in A (cell A) and 2 other cells. C: response latency for the same 3 cells plotted relative to the time of the missing flash minus the latency to the first flash response.

If a ganglion cell can track changes in a stimulus period as small as 5 ms, how long does it take to update its representation when the stimulus period suddenly changes? We explored this question by changing only the final interval in a sequence of 16 flashes. Figure 6A shows the response of one ganglion cell to a flash train at 12 Hz (top) and to trains in which the last flash was 21 ms earlier (middle) or 21 ms later (bottom) than established by the previous periodicity. In the purely periodic flash train condition, we observed an OSR 144 ms after the last flash (61 ms after the omitted flash). When the last flash was shifted earlier, the OSR shifted as well. In this condition, the latency of the OSR from the last flash was 123 ms. This was precisely 21 ms shorter than the latency in the normal condition. Equivalently, the OSR latency with respect to the missing flash was the same (61 ms) if the retina expected the next flash at 62 ms after the last flash rather than 83 ms. Thus one could describe this cell as having perfectly updated its expectation of the time or the next flash based on the final time interval between flashes. We observed this form of updated prediction when the last flash was shifted 3–30 ms early (Fig. 6B). The peak firing rate of the OSR was not affected by changing the last interval between flashes (data not shown).

FIG. 6.

A cell's prediction can be updated in a single flash interval. A: responses of a ganglion cell to a normal 12-Hz flash train (top), and trains with the last flash shifted 21 ms early (middle) or 21 ms late (bottom). OSR latencies are shown by arrows. B: OSR latency from last flash vs. time shift of the last flash for early flashes (n = 5 cells). Dotted line is unity slope. C: peak firing rate of period-early OSR (see methods) vs. time shift of the last flash (n = 4 cells). Zero time shift point (square) represents baseline activity. Open circles are nonsignificant OSRs; closed circles are significant (P < 0.05).

Although an earlier final flash caused an earlier OSR, a later last flash did not change the OSR latency in the same way. In this condition, we observed two separate bursts of firing, both with the normal OSR latency. In Fig. 6A (bottom) when the last flash was shifted 21 ms later, there were large peaks in firing following the second to last flash and the last flash, respectively. The first firing event can be interpreted as a “period-early” OSR because an extra long time interval between flashes is like the omission of a flash at the expected time. However, there was also an OSR at the same latency from the last flash, as in the case of a purely periodic flash sequence. When the last flash was shifted later by a small amount, there was no period-early OSR, but for shifts of ≥18 ms, we saw a significant enhancement of firing, which increased with larger intervals (Fig. 6C). Neither the latency of the period-early OSR nor that of the final (normal) OSR depended on the final flash interval (data not shown).

The data from the flash shift experiment suggest that a ganglion cell can update its frequency prediction after a single time interval between flashes, as when the last flash was earlier than for a purely periodic sequence (Fig. 6A, middle), while at the same time it can retain a representation of the past stimulus frequency after such a change, as we observed with the normal OSR following the late final flash (Fig. 6A, bottom). To study this dual representation more carefully, we presented a set of “jittered” flash sequences in which each of the 16 flash intervals was chosen from a normal distribution (mean = 83 ms; SD = 5 ms). Figure 7 shows examples of four different cells responding to this stimulus. The data are plotted as spike rasters with stimuli aligned to the time of the last flash. Additionally, the flash trains are ordered according to the last flash interval with short intervals toward the bottom of the raster and long intervals toward the top.

In each of these cells, we can characterize the spiking events by their latency from actual and missing flashes. The ganglion cell in Fig. 7A had no OSR and simply fired in response to each flash with a latency of about 65 ms. This latency following the last two flashes is highlighted in yellow; it is given by the time T_flash = T_n + τ_flash, where T_n is the time of the nth flash and τ_flash = 65 ms. The cell in Fig. 7B had an OSR when the last flash interval was sufficiently short. The latency of the OSR shifted according to this final flash interval (as in Fig. 6B). We predicted the timing of this OSR by a simple model in which the cell's estimate of the frequency of the periodic sequence is updated by the time interval between flashes: T_OSR = T_n + (T_n – T_n–1), where T_OSR is the time of the OSR, T_n and T_n–1 are the times of the last and second-to-last flashes, respectively, and τ_OSR = 80 ms is a constant delay that corresponds to the OSR latency for a purely periodic flash sequence. The orange highlight shows that the prediction of this model fits the spike times quite well.

For longer flash intervals, however, this ganglion cell no longer had an OSR following the end of the flash sequence but instead had an OSR one period early. The period-early OSRs were predicted by a similar model, shifted back by one stimulus period: T_OSR2 = T_n–1 + (T_n–1 – T_n–2) + τ_OSR. Purple crosses are spike time predictions from this model with the same OSR latency as before, τ_OSR = 80 ms. Note the deviation from the yellow highlight (prediction of simple flash response) near the top of the raster. These spikes are a period-early OSR, not a response to the last flash, because their timing is affected by the time of the second to last flash.

As we move up the raster from short to long final flash intervals, there is a dividing line where the normal OSR ends and the period-early OSR begins. Interestingly, the division between these two behaviors occurs near the average flash interval of 83 ms (trial 120). Figure 7C shows a ganglion cell that had an OSR for shorter than average time intervals between the last two flashes, with a timing explained by the model based on this last interflash interval (orange highlight). For longer than average interflash intervals, the cell had a period-early OSR (prediction as before shown by the purple crosses) but also had an additional OSR at the end of the sequence following the omitted flash. These spike times were predicted using the average interval between flashes (Δ = 83 ms) with the analogous model, T′_OSR = T_n + Δ + τ_OSR, where again τ_OSR = 80 ms (pink highlight). As in Fig. 6A, this cell seemed to contain representations of both the final flash interval and the average interval during the jittered sequence. Figure 7D shows another ganglion cell with similar characteristics but with a multiple-peak “ringing” OSR. Spike time predictions are highlighted as before, with the same OSR latency, τ_OSR = 80 ms.

The data in Figs. 6 and 7 present a consistent picture of the timing of the OSR, in which there are two limits. When a flash comes earlier than the average time interval between flashes, the OSR latency shifts proportionally earlier, with shifts as large as ≥50 ms seen in individual ganglion cells. However, when the flash comes later than the average time interval, an OSR is produced and the cell's expectation is reset to the average stimulus period. Thus the retina seems to maintain simultaneous representations of both the last time interval between flashes and the average time interval in the recent past.

Identifying the required retinal circuitry

To begin to understand how the retina generates an omitted stimulus response, we used pharmacological agents to dissect its circuitry. Amacrine cells, inhibitory third-order neurons, exhibit extreme morphological diversity and are capable of local dendritic processing that has not been well characterized, making them a candidate site for the generation of the OSR. We used a variety of pharmacological agents to block the action of amacrine cells. The neurotransmitters that mediate the inhibitory effects of amacrine cells are γ-aminobutyric acid (GABA) and glycine. Picrotoxin (GABA_A and GABA_C antagonist) and strychnine (glycine antagonist) block receptors for both neurotransmitters. As expected, firing rates were significantly higher after blocking these sources of inhibition (13.8 vs. 8.2 spikes/trial; P < 0.02). Rather than abolishing the OSR, this treatment increased its size in all cells that we tested (Fig. 8, A and D; n = 13 cells). Some cells with no OSR in the control condition even developed one after the addition of these amacrine cell blockers (Fig. 8B). The metabotropic GABA_B antagonist phaclofen and agonist baclofen also had no effect on the OSR (data not shown).

FIG. 8.

The OSR does not require inhibitory neurotransmission from amacrine cells. A: responses of a ganglion cell to a 12-Hz flash train before and after application of picrotoxin and strychnine. B: spike raster from a different cell as picrotoxin and strychnine are washed in. Wash-in is complete near dotted line. C: response of a cell before and after application of an amacrine blocking cocktail containing SR9591, (1,2,5,6-tetrahydropyridin-4-yl)methylphosphinic acid (TPMPA), tubocurarine, and atropine. D: OSR size (spikes per trial) before and after application of amacrine blocking drugs. Picrotoxin and strychnine, n = 11 cells; SR9591, TPMPA, tubocurarine, and atropine, n = 13 cells.

More evidence arguing against the involvement of amacrine cell inhibition in the OSR came from subsequent experiments in which we again blocked GABA receptors with different agents (SR95531 to block GABA_A and TPMPA to block GABA_C) and blocked acetylcholine receptors, another known target of amacrine signaling, with tubocurarine and atropine. Again, the OSR size was never reduced and increased significantly for most cells (Fig. 8, C and D). These data cannot completely rule out some role for amacrine cells in contributing to the omitted stimulus response. Some amacrines are known to signal using dopamine, nitric oxide, somatostatin, or a host of different neuropeptides that we did not block (Kim et al. 2000; Thermos 2003). However, such signals are generally considered slow and modulatory and would be unlikely to cause the large burst of firing we see after flash sequences of around1sin duration. In particular, the OSR is present even when the mean light level is held constant over each flash period (Schwartz et al. 2007), suggesting that dopamine, which is known to mediate many network-level effects of light adaptation (Dowling 1987), is unlikely to be necessary for the OSR. Alternatively, gap-junctional coupling of amacrine cells to bipolar cells or ganglion cells was still present under all of our pharmacological manipulations (Brivanlou et al. 1998; Sakai and Naka 1990) and may play a role in generating the OSR.

If inhibition from amacrine cells is not needed for an omitted stimulus response, could bipolar cells be responsible? We blocked light responses in on bipolar cells using the selective metabotropic glutamate receptor 6 (mGluR6) agonist APB. As reported, this treatment eliminated the ganglion cell response to isolated on flashes, while leaving the off response intact (data not shown) (Slaughter and Miller 1981). However, APB also completely eliminated both the OSR and all responses to flashes after the first (Fig. 9; n = 15 cells). The modulation of responses to dark flashes by on bipolar agonists was a surprising result that has implications for mechanism (see discussion).

FIG. 9.

on bipolars are required for the OSR. A and B: responses of 2 different ganglion cells to a 12-Hz flash train before application of 2-amino-4-phosphonobutyric acid (APB), after application, and after wash-out. Notice that both the sustained response and the OSR are blocked by APB. (Cell in A is the same cell as shown in Fig. 4E.) C: the response of a ganglion cell to the addition of (RS)-α-cyclopropyl-4-phosphonophenylglycine (CPPG). D: OSR size (spikes per trial) before and after application of APB (n = 13 cells) or CPPG (n = 5 cells).

The mGluR6 antagonist CPPG constitutively depolarizes on bipolars (Jane et al. 1996). As with APB, this treatment abolished the OSR, but CPPG did not eliminate the response to flashes during the sequence. The OSR was recovered after washout of APB or CPPG, and the sustained flash response in cells that had such a response was recovered after APB washout. Experiments with ionotropic glutamate receptor antagonists were performed to gain information about their roles in the OSR. The α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid (AMPA)/kainate antagonist CNQX eliminated the OSR along with all other spiking, whereas the N-methyl-d-aspartate (NMDA) antagonist d-AP7 had no effect on the OSR (data not shown), suggesting that AMPA and kainite receptors are involved and that NMDA receptors have no special role in this phenomenon.

DISCUSSION

Pattern recognition is a ubiquitous task in sensory systems. Thus discovering the neural mechanisms underlying pattern recognition has been the subject of considerable effort in recent years (Abarbanel and Talathi 2006; Delorme and Thorpe 2001; Gutierrez-Galvez and Gutierrez-Osuna 2003; Oyamada et al. 2000; Serre et al. 2007). The omitted stimulus response in the retina is an example of highly sophisticated pattern recognition taking place in a potentially tractable neural circuit.

The sophistication of the OSR lies in the ability of ganglion cells to adapt to changes in the stimulus pattern. The latency of the OSR is not a fixed property of a cell, but rather changes dynamically with stimulus frequency (Fig. 5) even after a single flash interval (Figs. 6 and 7). Remarkably, this computation does not seem to involve a complex network of interneurons. Our pharmacology results indicate that GABA, glycine, and acetylcholine signaling from amacrine cells are not involved in the OSR (Fig. 8) and that on bipolar cells are required (Fig. 9). Although we are not able to rule out all involvement of amacrine cells, it is quite possible that the feedforward circuit of photoreceptors to bipolar cells to ganglion cell is sufficient for generating this dynamic pattern recognition system.

Oscillations as a possible mechanism

If the OSR computation is really performed in this small feedforward network in the retina, what sort of mechanism is involved? An important clue comes from the work of Burrone and Lagnado (1997) who found that dissociated on bipolar cells from the goldfish can spontaneously oscillate due to the action of voltage-gated calcium channels coupled with calcium-gated potassium and potassium A channels. A component of this oscillation was observed at 5–10 Hz, a frequency at which we find an OSR (Schwartz et al. 2007). Similarly, Protti et al. (2000) found that bipolar cells in the goldfish slice preparation exhibited a spontaneous oscillation with frequency components at about 10 Hz, which was especially prominent in the light-adapted condition. Although we cannot identify the ion channels involved in the OSR phenomenon or even the site of such an oscillation, the concept of an electrical resonance in the retina is consistent with our data.

Ganglion cells recognize patterns after only a few flashes (Schwartz et al. 2007), and update their predictions in a single flash interval (Figs. 6 and 7). Such rapid recognition cannot be accomplished by “learning” in a neural circuit in the form of changing synaptic weights via known mechanisms of long-term potentiation or depression, because it is too fast. Rather than altering the connections between cells, a periodic stimulus could entrain an intrinsic resonance within a single cell. Additionally, the firing patterns of certain cells are highly suggestive of an oscillator. In a number of cases, we observed cells that generated two OSR peaks or even had ringing behavior following a flash train. Many cells had complex harmonic firing patterns during the flash sequence (Figs. 1 and 3), some of which exhibited period-doubling (Fig. 3A) and others which approximated period-tripling or other beat patterns (Fig. 3B).

These observations along with our pharmacology results (Figs. 8 and 9) support the idea that oscillations in on bipolars are the source of both the OSR and many of the interesting firing patterns we see during the sequence of dark flashes. During a periodic sequence of flashes, we propose that on bipolar cells oscillate resonantly at the stimulus frequency, but this response is partially cancelled by regular, nonresonating responses from off bipolars. Although both on and off bipolars release glutamate onto ganglion cells, on bipolars have a relatively high sustained rate of glutamate release (Zaghloul et al. 2003) and suppression of this release rate by the visual stimulus or inhibition from off bipolars (through amacrine cells or even gap junctions) onto on bipolar terminals could reduce this release rate and confer a net inhibitory signal onto the ganglion cell.

When the stimulus ends, resonating on bipolar cells would “ring” for one or more periods, delivering excitation to the ganglion cell one stimulus period later, whereas nonresonating off bipolars would be silent. Thus we propose that this excitation from on bipolar cells, which is not cancelled by any activity from off bipolars, may result in the omitted stimulus response. Firing during the flash sequence presumably also arises from on bipolars, since it is blocked by APB (Fig. 9). The synapse from photoreceptors to off bipolars is known to undergo rapid and prolonged desensitization (DeVries and Schwartz 1999). If the off bipolars are desensitized after the first few flashes, firing during the middle of the train may, in fact, be caused entirely by on bipolars. The different varieties of sustained responses we observe could be due in part to the ratio of on and off bipolars synapsing onto the ganglion cell and the time course of the off bipolar desensitization (DeVries 2000).

Consistent with our pharmacology results, generation of the OSR in this model does not require amacrine inhibition. We speculate that amacrine inhibition might play a role in the cancellation of on responses during the flash sequence. Such cancellation is apparent when single flash responses are compared with the flash train response (Fig. 4, C–F). Although we do see some cells in which the amacrine block reveals more sustained firing (Fig. 8, B and C), there seem to be a number of unexplained mechanisms contributing to the complexity of the response during the flash train.

The oscillator idea is only a general framework in which to consider a more detailed mechanistic explanation of the OSR. Considerable challenges remain in explaining our data in terms of such a model. First, is the issue of entraining to different stimulus frequencies (Fig. 5). It is possible that the electrical resonance of bipolar cells can be tuned by the stimulus frequency using a second messenger, like calcium, that could vary with stimulus period (J. Gao, G. Schwartz, M.J. Berry 2nd, and P. Holmes, unpublished observations). Alternatively, ganglion cells might receive input from a “bank” of oscillating bipolars, each tuned to a slightly different frequency, such that over a range of stimulus frequencies, there is always one bipolar cell capable of resonating. In the turtle cochlea, cells are tuned to different frequencies because of different splice variants of K⁺ channels (Jones et al. 1999).

Not only can the retina recognize a particular frequency, but it can also estimate the average stimulus period in considerable noise, switching its temporal expectation from the interval between the last two flashes to the average interval after an OSR occurs (Figs. 6 and 7). It is difficult to speculate on how such abilities may arise, but it is possible that the mechanism involves the interaction of multiple oscillators at different frequencies aided by electrical coupling between bipolar cells. Such coupling between bipolar cells has been observed in teleost fish using dye tracing (Umino et al. 1994) and can be inferred from the synchronization of period-doubled light responses (Crevier and Meister 1998). Future work combining intracellular recordings of input currents to ganglion cells as well as photoreceptor and bipolar recordings and biophysically realistic modeling of the ion channels in these cells should help elucidate the mechanism of the OSR.

Representing patterns in the neural code of the retina

Presenting a very simple pattern, 16 dark flashes at 12 Hz, we observed a wide variety of responses in the population of retinal ganglion cells (Figs. 1–3) and very slight variations in the pattern caused large changes in firing (Figs. 5–7). What does this mean for the manner in which patterns are represented in the neural code of the retina?

This extensive diversity of response characteristics may help the retina resolve a fundamental ambiguity: almost all cells that fire after a pattern violation also fire after the presentation of an isolated flash, so how can the brain know whether the stimulus was a real flash or the absence of an expected flash? This is not possible at the level of individual ganglion cells, but may be possible using combinations of firing among the population of ganglion cells. For instance, if the brain receives spikes from ganglion cells that have an OSR, whereas ganglion cells with a start-only response type (e.g., Fig. 1A, top) are silent, then it is likely that there was a pattern violation rather than a real flash. The diversity in the population is great enough to support even more pattern selectivity. For instance, if one subset of cells fire after a sequence of 100-ms flash intervals is followed by 80 ms, whereas a different subset fire when the next flash interval is 120 ms, then the brain may be able to distinguish among different kinds of pattern violations as well. Furthermore, the extensive diversity of sustained responses may also allow the brain to distinguish the second from the third flash within a periodic sequence (and more generally determine the order of flashes) using just the instantaneous firing state of the retinal population. Of course, these speculations can be confirmed only by carrying out explicit decoding analysis using spike trains from populations of ganglion cells.

Such a combinatorial code may play an important role in hierarchical pattern detection. A cell signals not the presence of a particular, low-level visual feature, such as a single flash of light, but instead signals the presence of the entire, composite pattern. A similar phenomenon has long been known to exist in on–off cells, which signal rapid changes in light intensity without conveying the polarity of the change. Although the composite feature signaled by an on–off cell is relatively simple, our data show that more complex patterns can be grouped together in this fashion by ganglion cells. As a result, the firing of such a ganglion cell can serve as a “pointer” to the presence of the composite pattern, which concentrates information about the temporal sequence into spiking at a single point in time. This representation allows subsequent neural circuits to manipulate this information in the same fashion as they would handle lower-level visual features, such as a single flash of light. Such hierarchical models have been proposed for object recognition (Riesenhuber and Poggio 2002) and motion detection (Clifford and Ibbotson 2002), but these results suggest that, especially for temporal pattern recognition, the retina plays a much larger role than previously assumed.

In thinking about the purpose of retinal pattern detection, it is tempting to view start-end cells as “efficient encoders,” representing only the changes in a temporal pattern and not wasting spikes on expected stimuli. However, we believe that the purpose of periodic pattern detection is not related to coding efficiency. Ganglion cells have extensively overlapping receptive fields (Segev et al. 2006) and a large fraction of ganglion cells respond to each flash in a periodic sequence. Thus the population as a whole is very likely to encode the presence of each and every flash, making the entire population highly redundant (Puchalla et al. 2005), even though individual cells may appear efficient.

Instead, we think that the formation of temporal expectations in the retina and the representation of these expectations via the firing pattern of combinations of ganglion cells is a mechanism that not only conveys basic information about events in the visual world, but also explicitly labels these events as expected or surprising. This division of sensory information is important because the behavioral consequences of the same visual information can be quite different depending on whether that state of the external world was expected. Another significant aspect of this division is that expectation and surprise are represented explicitly by instantaneous firing combinations of ganglion cells, rather than by patterns of spikes distributed across time. This allows the brain to use fast and simple biophysical mechanisms, such as coincidence detection and feedforward inhibition, to determine whether stimuli were expected. Thus this representation may also be favorable for hierarchical pattern detection in subsequent neural circuits.