Synaptic rectification controls nonlinear spatial integration of natural visual inputs

Summary

A central goal in the study of any sensory system is to predict neural responses to complex inputs, especially those encountered during natural stimulation. Nowhere is the transformation from stimulus to response better understood than the vertebrate retina. Nevertheless, descriptions of retinal computation are largely based on stimulation using artificial visual stimuli, and it is unclear how these descriptions map onto the encoding of natural stimuli. We demonstrate that nonlinear spatial integration, a common feature of retinal ganglion cell (RGC) processing, shapes neural responses to natural visual stimuli in primate Off parasol RGCs, whereas On parasol RGCs exhibit surprisingly linear spatial integration. Despite this asymmetry, both cell types show strong nonlinear integration when presented with artificial stimuli. We show that nonlinear integration of natural stimuli is a consequence of rectified excitatory synaptic input, and that accounting for nonlinear spatial integration substantially improves models that predict RGC responses to natural images.

Introduction

Our understanding of the computations performed by sensory neurons has benefitted from many decades of study using artificial stimuli designed to probe particular features of a sensory receptive field (RF). For example, the RF structure of retinal ganglion cells (RGCs) has been well characterized using artificial visual stimuli, beginning with spots and annuli to reveal center-surround organization (Kuffler, 1953; Barlow et al., 1957). Other studies have used moving stimuli to measure direction selectivity (Barlow & Levick, 1965), spatiotemporal white noise to characterize linear feature selectivity and nonlinear processing (Meister et al., 1995; Chichilnisky, 2001; Pillow et al., 2008), and fine gratings to describe nonlinear spatial integration (Enroth-Cugell & Robson, 1966; Hochstein & Shapley, 1976a; Hochstein & Shapley, 1976b). These studies reveal a variety of non-classical RF features that can be exercised using appropriately designed stimuli (for review see Kastner & Baccus, 2014; Gollisch & Meister, 2010), but understanding the roles of these RF features in visual function requires knowing how they shape the encoding of natural inputs.

A well-known failure of classical RF models is the inability to capture nonlinear integration of signals across space (Enroth-Cugell & Robson, 1966; Stone & Pinto, 1993; Demb et al., 1999; Petrusca et al., 2007; Bolinger & Gollisch, 2012). Nonlinear integration is mediated by subunits in the RF that provide rectified input to the RGC (Hochstein & Shapley, 1976b; Victor & Shapley, 1979; Demb et al., 1999; Crook et al., 2008; Schwartz et al., 2012). In short, this means that each subunit can effectively transmit preferred contrast inputs but that the suppressive impact of non preferred contrast inputs will be minimal. Such subunit structure limits cancelation of light and dark regions in a scene and thus shapes RGC responses to spatially structured stimuli - e.g. causing the RGC response to depend on the spatial distribution of light, not just the total amount of light, within the RF center. At least in some cases, bipolar cells presynaptic to the RGC are the physiological basis of the rectified RF subunits (Demb et al., 2001; Schwartz et al., 2012; Freeman et al., 2015; Kuo et al., 2016).

Thus, an impressive body of work demonstrates the ubiquity of nonlinear spatial integration, the circuit and synaptic mechanisms that give rise to a nonlinear RF, and some of the computations that a nonlinear RF can support (Olveczky et al., 2003; Gollisch & Meister, 2008; Schwartz et al., 2012; Kuo et al., 2016, for review see Gollisch & Meister, 2010; Schwartz & Rieke, 2011). Beyond the retina, the classical energy model of complex cells in V1 also relies on nonlinear transformation of multiple linear filter outputs before integration (Adelson & Bergen, 1985), and more recent evidence suggests that a nonlinear subunit RF structure is appropriate for both simple and complex cells in primary visual cortex (Rust et al., 2005; Vintch et al. 2012; Vintch et al., 2015).

Nonlinear spatial integration is absent from most predictive models for RGC responses. Current models can accurately predict responses to the same sort of visual stimuli on which they were trained (Keat et al., 2001; Pillow et al., 2005). Yet such models are much less successful in generalizing to predict responses to natural stimuli (Heitman et al., 2016; for review, see Carandini et al., 2005). Most predictive RGC models incorporate an initial linear filtering step that accounts for feature selectivity in time and/or space (i.e. a linear RF). But the many studies demonstrating nonlinear spatial integration suggest that a linear spatial RF may not be an appropriate description. It is unclear, however, whether nonlinear responses are primarily elicited by artificial stimuli designed to test spatial linearity or are instead elicited by natural stimuli. The aim of this study is to help bridge the gap between this classical understanding of nonlinear spatial integration and modern computational models.

Our central goals were to answer three related questions about the importance of nonlinear spatial integration in RGC encoding of natural inputs: (1) Does nonlinear spatial integration affect RGC responses to natural stimuli? (2) What circuit or synaptic mechanisms control spatial integration of natural scenes? And, (3) does incorporation of nonlinear RF structure substantially improve models that predict responses to natural stimuli? We will answer these questions for responses of On and Off-center parasol (magnocellular-projecting) RGCs in the monkey retina. Parasol RGCs provide key signals for visually-guided behavior and exhibit pronounced spatial nonlinearities (Petrusca et al., 2007; Crook et al., 2008, Cafaro & Rieke, 2013). This makes them an ideal class of RGC in which to answer these questions.

Results

We used a combination of in vitro recording and RF modeling to explore whether and how nonlinear spatial integration in the RF center shapes parasol RGC responses to natural visual stimuli. We focused on spatial integration within the RF center for two reasons. First, the RF center of parasol RGCs can show strong spatial nonlinearity when probed with artificial stimuli (Petrusca et al., 2007; Crook et al., 2008, Cafaro & Rieke, 2013; see also Fig. 5), but the impact on coding of natural stimuli is unknown. Second, understanding coding in the RF center is a tractable but essential step towards the broader question of natural stimulus encoding.

Fig. 5. On & Off parasol RGC receptive fields contain small nonlinear subunits.

(A) Area summation curve from an example On parasol RGC. Points indicate mean ± S.E.M., curve indicates the best fit difference-of-Gaussians model (see methods). Dashed vertical line indicates the RF center size of this cell. (B) Same as (A) for an example Off parasol cell. (C) For a grating stimulus centered over the RF, a LN RF produces no response to contrast modulation, while a nonlinear subunit RF produces a frequency-doubled (F2) response. (D) Cycle average responses in an example On parasol RGC. A contrast reversing grating stimulus, centered over the RF, produces robust F2 responses in spike output (top), excitatory input (middle) and inhibitory input (bottom) across a range of spatial frequencies. Different colored traces correspond to different bar widths of the grating stimulus. (E) On parasol RGC population summary data showing the nonlinear F2 response amplitude as a function of bar width. For each sampled bar width, the F2 response amplitude was calculated from 20–60 cycles of the stimulus. Within each cell, the bar width was normalized by the size of the RF center (measured as in A), to control for cell-type and eccentricity based differences in RF center size. Gray points indicate individual measurements from the example cell in (D), and colored squares indicate mean ± S.E.M. for equally populated bins. Vertical dashed lines indicate the (normalized) bar width that drove half-maximal F2 responses, on average. Top: spike responses, 11 cells. Middle: Excitatory synaptic currents, 7 cells. Bottom: Inhibitory currents, 8 cells. (F–G) Same as (D–E) for Off parasol RGCs. These cells also show small-scale nonlinear RF structure in their spike output (top, 8 cells) as well as their excitatory (middle, 7 cells) and inhibitory synaptic inputs (bottom, 6 cells).

We start by showing that Off, but not On, parasol RGCs nonlinearly integrate natural visual inputs, although both cell types exhibit strong spatial nonlinearities when probed with artificial stimuli. This On/Off asymmetry persists across a range of light levels. We then provide evidence that the asymmetry in natural stimulus integration results in part from differences in rectification of the cells’ excitatory synaptic inputs. Finally, we show that including rectified subunits in a RF model can substantially improve predictions of Off parasol RGC responses to natural images.

Spatial integration of natural retinal inputs

To begin, we designed a stimulus to test whether a linear model of spatial integration could account for responses of parasol RGCs to natural stimuli. This stimulus was based on an online database of natural scenes and measured eye movements called the Database Of Visual Eye movementS (DOVES, van der Linde et al., 2009, live.ece.utexas.edu/research/doves). The DOVES database contains eye tracking data measured from human observers freely viewing grayscale images from the van Hateren natural image database (van Hateren & van der Schaaf, 1998). Using these data, we reconstructed the image impinging on the retina during these periods of natural viewing (Fig. 1A,B). Our DOVES-based movie stimuli capture several important features of natural retinal input, including the statistics of natural images, fixation lengths, and fixational eye movements (see Supplemental Experimental Procedures for details).

Fig. 1. Natural retinal inputs drive nonlinear responses in Off but not On parasol RGCs.

(A) A natural image and associated eye movement trajectory from the DOVES database. White dots indicate fixation points, green and red dots indicate the first and last fixations. (B) Top: Schematic illustrating the DOVES movie, restricted to the RF center. Bottom: A linear equivalent disc stimulus movie was generated using a circular Gaussian RF sampling each frame in the natural movie. (C) Example On parasol RGC spike responses to these stimuli. Vertical dashed lines indicate frames at which a change of fixation (“saccade”) occurred. Numbers above indicate fixations. (D) On parasol RGC population data showing mean spike count during each fixation (94 total fixations from 15 RGCs) in response to DOVES and associated linear equivalent disc movies. Open circles correspond to one fixation measured in one cell, green circles indicate the eight fixations from the example cell in (C). (E–F) Same as (C–D) for an example Off parasol RGC and population data (66 fixations from 10 RGCs). (G) Cumulative histogram of the nonlinearity index (NLI, see text) for every fixation response measured in On (gray trace) and Off (black trace) parasol RGCs. (H) For each cell, we averaged the NLI over all presented fixations. Open circles indicate cell-average NLIs for individual cells, closed circles show population mean ± S.E.M.

To probe integration in the RF center with minimal activation of the surround, we presented DOVES movies with an aperture that restricted the image to the RF center. We scaled the DOVES movies to a mean luminance level of ~4,000 cone opsin isomerizations (R*)/M or L cone/s), which corresponds to low-photopic light levels. During presentation of DOVES stimuli, we set the mean monitor intensity (i.e. the mean gray level of the aperture around the image and of the blank screen between trials) to 4000 R*/cone/s.

We designed a second stimulus to explicitly test linear spatial integration of the DOVES movies. This stimulus was a uniform disc whose intensity was equal to the average intensity of the corresponding frame in the original movie weighted by the cell’s RF center; we refer to this stimulus as the linear equivalent disc movie. To construct the linear equivalent disc movie, we sampled each frame of the original movie with a circular Gaussian function and linearly integrated the (weighted) pixel values to yield a single intensity value for each frame (Fig. 1B, bottom). The size of the Gaussian RF center was determined using the area-summation curve generated by presenting spots, centered over the RF, of various sizes (e.g. see Fig. 5). The key property of this manipulation is that a RGC that integrates spatial inputs according to a linear Gaussian RF will produce the same response to the original movie (“Natural Image”) and the sequence of linearly integrated intensity values (“Linear Equivalent Disc”).

For each cell, we presented one of ten reconstructed DOVES stimuli (each movie was generated using a different natural image and/or measured eye movement trajectory). For many fixations, On parasol RGCs generated near-identical responses to the original DOVES movie and the linear equivalent disc movie (Fig. 1C,D). Spike responses to these two stimuli did not differ systematically across all measured fixations (Fig. 1D; p = 0.65). However, the same stimuli often produced large differences in the spike responses of Off parasol RGCs (Fig. 1E,F). The linear equivalent disc and the natural movie produced approximately equivalent responses for some fixations (points along unity in Fig. 1F), but responses to other fixations differed substantially (points below unity in Fig. 1F). On average, Off parasol RGCs responded more strongly to the DOVES movie than to the linear equivalent disc movie (Fig. 1F; p = 8.7×10⁻⁶).

To quantify the deviations from linear integration, we defined a nonlinearity index (NLI) as

$$\mathit{NLI}=\frac{r_{\mathit{image}}-r_{\mathit{disc}}}{\left|r_{\mathit{image}}\right|+\left|r_{\mathit{disc}}\right|}$$ (1)

where, for each fixation, r_image and r_disc are the average (over repeated trials) spike count in response to the original DOVES movie and to the linear equivalent disc movie. Thus a NLI near 0 indicates near-linear spatial integration, negative NLIs indicate stronger responses to the linear equivalent disc movie and positive NLIs indicate stronger responses to the DOVES movie. The absolute value operators in the denominator ensure that this sign relationship holds for responses that can take negative values (e.g. synaptic current responses in Fig. 2).

Fig. 2. Nonlinear synaptic current responses to natural visual stimuli.

(A) Mean synaptic current responses—in the same example On parasol RGC as in Fig. 1—to DOVES and linear equivalent disc movies. (B) Same as A, for an example Off parasol RGC’s synaptic inputs. (C) Analogous to Fig. 1D. Average excitatory charge transfer (integrated current) in response to the DOVES movie and associated linear equivalent disc movie; each point corresponds to one fixation (44 fixations from 7 RGCs). Colored points correspond to the example On parasol RGC in (A). Left: excitatory charge transfer, right: inhibitory charge transfer. (D) Same as (C) for Off parasol RGC synaptic inputs (48 fixations from 7 RGCs). Colored points indicate example Off parasol RGC in (B). (E) Cumulative histograms of NLI values for all fixations measured in On and Off parasol RGC excitatory and inhibitory currents. (F) Cell-average NLI values, 7 cells in each case.

Figure 1G shows the cumulative distributions of NLI values for all fixations and all recorded On and Off parasol RGCs. NLI values for both cell types varied considerably across fixations; NLI values in Off parasol RGCs were shifted to positive values relative to On parasol RGCs (median NLI of 0 in On versus 0.12 in Off parasol RGCs, p = 6×10⁻⁶, Wilcoxon rank sum test on the distributions in Fig. 1G). We averaged NLIs across fixations for each cell to yield the cell-average NLI (Fig. 1H). On parasol RGCs had an average NLI that was not significantly different than zero (p = 0.54), whereas the average NLI for Off parasol RGCs was positive and significantly larger than that for On parasol RGCs (p = 0.002). Thus, the linear equivalent disc effectively mimicked natural stimuli for On parasol RGCs, but the same linear prediction systematically underestimated responses of Off parasol RGCs to natural stimuli.

Off parasol RGC spike responses were typically more transient than On parasol RGC responses to the same stimuli (e.g. compare Fig. 1C and E). This was also true for artificial stimuli like gratings (Fig. 5) and spots (Fig. 6). Despite this difference, we saw no obvious history effects in responses to DOVES movies that could account for the On/Off asymmetry in spatial integration. Moreover, as we show below, the asymmetry in spatial integration is present in responses to flashed natural images from a mean gray background (Fig. 3).

Fig. 6. Sharp Off pathway rectification drives strongly nonlinear responses.

(A) Left: Mean excitatory current responses in a representative On parasol RGC presented with spots of varying contrasts. Gray traces denote negative contrast steps and black traces indicate positive contrasts (responses to ± 12.5%, 25%, 50% contrast are shown). Shaded region indicates time over which charge was integrated to yield the contrast response functions. Right: Mean ± S.E.M. contrast response function for On parasol RGC excitatory currents, normalized within each cell (7 cells). (B) Contrast step responses (left) and the mean contrast response function (right) for On parasol RGC inhibitory input (9 cells). (C–D) Same as (A–B) but for Off parasol cell excitatory (C, 6 cells) and inhibitory (D, 6 cells) synaptic currents. (E) We modified our LN and nonlinear subunit models to use the excitatory contrast response function from either On or Off parasol cells in place of the simple piecewise-linear nonlinearity. (F) We presented each model with 1,000 randomly selected patches from a natural image and compared the two model outputs. Left: responses when the models include the On parasol RGC excitatory nonlinearity (inset); right: responses when the models instead use the Off parasol RGC excitatory nonlinearity. (G) We repeated the analysis in (F) for all 101 natural images in the DOVES database and measured the median nonlinearity index across all sampled patches. Each point corresponds to one image. Green point indicates the example image used in (F).

Fig. 3. Failures of linear integration in Off parasol RGCs is due to nonlinear subunit RF structure.

(A) Example of a natural image used in the modeling and experiments below. Diagram shows the spatial arrangement of subunits used in the RF models outlined in (B). (B) In both models, the RF center is composed of subunits. The linear-nonlinear (LN) RF model (left) takes the linear sum of subunit outputs and passes that sum through a rectified-linear output nonlinearity to yield the response. The nonlinear subunit RF model (right, “subunit”) applies the rectified-linear nonlinearity at the output of each subunit before summation at the ganglion cell. (C) Model outputs for 10,000 randomly-selected patches (gray points) from the image in (A). Black line indicates unity. (D) Histogram of the model response differences (subunit model response minus LN model response) for randomly-selected image patches (gray line). For the following experiments, we sampled these image patches to uniformly span the range of model differences (black line). (E) Spike rasters from an example On parasol RGC in response to a flashed stationary natural image patch (top) and its linear equivalent disc (bottom). Colored outlines indicate the region of the scene in (A) from which this patch was drawn. (F) Same as (E) for an example Off parasol RGC. (G) Mean spike counts (over five presentations of each patch) in response to 40 different patches from a natural image, for the example On parasol RGC (top) and Off parasol RGC (bottom). Dashed line indicates unity. (H) We subtracted the response to each linear equivalent disc from the response to its associated image, giving us a measure of the strength of nonlinear integration (“measured difference”) and compared that to the difference in model outputs for each image patch presented. Solid lines show linear fits, with linear correlation coefficient indicated at top left. (I) Population data showing the coefficients of variation (standard deviation between image and disc responses divided by average image response) for On and Off parasol RGCs. Open symbols correspond to individual cells, closed symbols denote mean ± S.E.M. (10 On parasol RGCs, 12 Off parasol RGCs). (J) Population data for On and Off parasol RGCs showing linear correlation coefficients between experimentally measured differences and model response differences.

Each cell was presented with a small number of fixations (typically 5–7 in a single DOVES movie), whereas the variability in spatial structure within and across natural images is quite large. This means that it is difficult to identify features of the DOVES movies that drive deviations from linearity in Off parasol RGC responses. Another consequence of this limited sampling is that much of the variability in cell-average NLIs (Fig. 1H) is due to variability across natural image stimuli rather than biological variability in the parasol RGC population. This is illustrated in Fig. S1, which shows that NLI values for a single cell can vary considerably across movies. We will return later to the question of what features of an image produce deviations from linear spatial integration.

Could a failure to accurately capture the linear RF account for the differences between the responses to DOVES and linear equivalent disc movies? Three observations indicate that this is not the case. First, the systematic undershoot of the linear model across fixations is characteristic of a failure of (linear) cancelation of positive and negative contrasts and is not predicted by errors in linear RF estimation. Second, there is no reason to suspect that our linear RF estimation is systematically wrong for Off but not On parasol RGCs. Third, we can account for most of the failures of linear integration in the context of natural scenes using a simple nonlinear subunit model (Figs. 3, 4, 7).

Fig. 4. Mean light-level dependence of nonlinear spatial integration of natural images.

(A) Contrast response functions measured in an example On parasol RGC at three luminance levels: high (~50,000 R*/cone/s), medium (~5,000 R*/cone/s) and low (~500 R*/cone/s). (B) At the same three light levels, we measured the spike responses to flashed natural image patches and their corresponding linear equivalent discs, as in Fig. 3. (C,D) Same as (A,B) for an example Off parasol RGC. (E) Population data showing the coefficient of variation (CV) between image and disc responses (as in Fig. 3I) across these three luminance levels. (F) The linear correlation coefficient between measured differences and modeled differences (as in Fig. 3J) across luminance levels. For On parasol RGCs: n = 6, 7, 7 (low, medium, high); for Off parasol RGCs: n = 6, 6, 6 (low, medium, high).

Fig. 7. Inclusion of nonlinear excitatory subunits improves a predictive model of natural image responses.

(A) We modified our nonlinear subunit and LN RF models (Fig. 2B) to include the measured contrast response function for Off parasol RGC excitatory inputs as well as a linear inhibitory channel. The nonlinear subunit model applies the measured nonlinearity before spatial integration (bottom), while the LN model applies this nonlinear transform to excitatory inputs after spatial integration (top). In both models inhibitory input is purely cross-over (on-tuned) and linear. Excitatory and inhibitory inputs are linearly combined and passed through a threshold-linear input-output function to yield spike output. (B) We measured an Off parasol RGC’s mean excitatory charge transfer in response to presentation of 40 randomly selected patches from a natural image. We then passed those image patches through each model and measured the net excitatory input for each stimulus. The subunit model (bottom) provided a better prediction of the excitatory input than the LN model (top), which generally underestimated the excitatory input. (C) Population data for the experiments in (B), showing that the fraction of explained variance is higher for the nonlinear subunit model than the LN model (9 cells). (D) We compared the spike output of each of our models to measured Off parasol RGC spike responses. As with the excitatory inputs to these cells, the nonlinear subunit model more accurately predicted Off parasol RGC spike outputs. (E) Population data showing that the nonlinear subunit model outperforms the LN model (13 cells).

A spatially linear RF allows responses to light and dark regions in a scene to cancel. For a spatially nonlinear RF, however, such cancelation is incomplete. This difference is a consequence of rectified subunits within the RF: because of rectification, subunits that are suppressed by non-preferred contrast do not effectively transmit a suppressive signal to the RGC, while subunits that are activated transmit a strong positive signal. Thus the smaller Off parasol RGC responses to the linear equivalent disc compared to the structured image patches are consistent with an impact of rectified subunits in the RF center.

If the nonlinear responses are due to presynaptic cells acting as rectified subunits, we should see nonlinear responses in the synaptic inputs to parasol RGCs and such nonlinearities should differ for On and Off RGCs. To test these hypotheses, we presented DOVES movies and corresponding linear equivalent disc stimuli while measuring synaptic currents in whole-cell voltage-clamp recordings (Fig. 2). We isolated inhibitory and excitatory synaptic currents using holding potentials near the cation and chloride reversal potentials (see Experimental Procedures). We then compared the charge transfer (integrated current) in response to DOVES and linear equivalent disc movies. The excitatory synaptic inputs to On parasol RGCs show relatively small, but significant, deviations from linear integration (Fig. 2A,C; p = 0.02), whereas the inhibitory inputs to the same cells show strong nonlinear integration (Fig. 2A,C; p = 1.8×10⁻⁵). For Off parasol RGCs, both excitatory and inhibitory inputs are larger in response to the original natural movie (Fig. 2B,D; excitatory charge transfer: p = 8×10⁻⁵, inhibitory charge transfer: p = 3×10⁻⁵). Fig. 2E shows an asymmetry in the degree of nonlinear spatial integration in these different inputs: NLIs for Off parasol RGC excitatory inputs were strongly skewed, with 88% of fixations producing positive NLIs, whereas NLIs for On parasol RGC excitatory inputs were more symmetrical, with positive NLIs for 64% of fixations. Cell-average NLIs were significantly more positive for Off than On RGC excitatory inputs (Fig. 2F; on average ~5 times greater for Off than On RGCs, p = 0.01).

These data show that spatial integration in parasol RGC spike outputs qualitatively resembles that in the cells’ excitatory synaptic inputs (see also Crook, 2014). While there is an asymmetry in the strength of nonlinear integration in the excitatory inputs to On and Off parasol RGCs, the excitatory inputs to both cell types deviate substantially from linear spatial integration. Indeed, neither the excitatory nor inhibitory synaptic inputs to On parasol RGCs are as linear as their spike outputs. Moreover, the inhibitory input to On and Off parasol RGCs is differentially driven by these natural stimuli. In particular, the inhibitory input to On parasol RGCs is stronger than inhibition to Off parasol RGCs (compare the example traces in Fig. 2A and B as well as the population data in Fig. 2C and D). These observations suggest that synaptic integration and intrinsic cell properties shape spatial encoding by On parasol RGCs. We will return to this issue in the Discussion.

A nonlinear subunit model predicts failures of linear integration

Our experiments with DOVES stimuli (Fig. 1,2) show that a spatially linear RF model fails to predict responses of Off parasol RGCs to natural inputs. To test the ability of simple models to capture these failures of linear integration, we turned to a stimulus paradigm that does not include eye movements. These stimuli were stationary image patches selected from a larger natural image (Fig. 3A; van der Linde et al., 2009; van Hateren & van der Schaaf, 1998). This simpler stimulus allowed us to focus on spatial integration in isolation, without history dependence created by interactions between eye movements and the spatial structure of natural scenes (Kuang et al., 2012; Rucci & Victor, 2015).

We compared how well two models captured failures of linear integration: a linear-nonlinear (LN) RF model and a nonlinear subunit RF model (Fig. 3B; see Supplemental Experimental Procedures for details). Image patches were passed through a square grid of RF subunits (Fig. 3A, inset), each modeled as a circular Gaussian. The subunit models shown in Fig. 3B are illustrated with On (positive contrast-preferring) subunits, but models with Off (negative contrast-preferring) subunits produced similar results.

In the LN model, the output of each subunit was weighted by a larger Gaussian representing sampling in the RF center (Fig. 3A, inset). The subunit outputs were linearly summed and passed through a rectifying output nonlinearity. In the nonlinear subunit model, the output of each subunit was passed through the same rectifying nonlinearity before being weighted and summed with the other (rectified, weighted) subunit outputs. These models were purposefully designed to be simplistic and minimal, with the key difference being the site of nonlinear transformation: before (nonlinear subunit model) or after (LN model) integration by the RGC. Models were not fit directly to RGC responses. Instead, we set each subunit diameter to be 1/6 of the RF center diameter, consistent with the subunits sizes measured in Fig. 5 below. The behavior of the model was similar as long as the subunits were ~1/2 the width of the RF center or smaller (See Fig. S2).

Outputs of each RF model were computed for 10,000 randomly-selected patches from a natural image. The nonlinear subunit model always produced greater or equal responses than the LN model, with some image patches producing much stronger responses in the nonlinear subunit model than the LN model (Fig. 3C). This can also be seen in a histogram of model response differences (nonlinear subunit minus LN, Fig. 3D, gray trace). Relatively homogeneous patches drive the models similarly while patches with both positive and negative contrast regions produce stronger responses in the subunit model than the LN model (Fig. 3C, insets). This is because rectification of subunit outputs prevents cancelation of positive and negative contrast regions.

If nonlinear subunit RF organization underlies failures of linear integration, we should be able to predict which image patches produce nonlinear responses in parasol RGCs. To test this, we selected patches from natural images and presented them to On and Off parasol RGCs while measuring spike responses. Image patches were not randomly drawn for these experiments (unlike Fig. 3C and the gray trace in Fig. 3D); instead one patch was drawn from each bin of the full histogram such that the patches chosen uniformly span the observed range of differences in model outputs (black trace in Fig. 3D, note that the displayed bin size has been increased for clarity). This allowed us to explore the entire range of nonlinear response strengths within a single experiment.

We presented each patch, restricted to the RF center, for 200 ms. We then presented the corresponding linear equivalent disc for each image patch (Fig. 3E,F). As in Fig. 1, On parasol RGC responses to the linear equivalent disc were similar to responses to the corresponding natural image (Fig. 3G, top). However, for many patches, Off parasol RGCs responded more weakly to the linear equivalent disc than to the image (Fig. 3G, bottom). Responses that deviated from linearity tended to be those of weaker or intermediate response strengths. This behavior is seen in the responses to DOVES movies (Fig. 1F) and is predicted by the model (Fig. 3C). The stronger deviations for intermediate response strengths are likely because the patches that strongly drive the cell are often uniformly the preferred contrast, and thus there is little opportunity for cancelation of light and dark regions of the scene. On average the difference between the responses to the flashed image patches and linear equivalent disc stimuli was greater in Off than On parasol RGCs (Fig. 3I, p = 2×10⁻⁴).

For each image patch, we calculated the difference between the spike response to the image and to the linear equivalent disc, and compared this to the difference in subunit and LN model outputs to the same patch. For On parasol RGCs, the measured differences were near zero, and there was little correlation between measured response differences and model response differences (Fig. 3H, top). Off parasol RGCs, however, showed a strong correlation between measured and modeled response differences (Fig. 3H, bottom, Fig. 3J). Thus, the same image patches that produced nonlinear responses in Off parasol RGCs produced stronger responses in the nonlinear subunit model than in the LN model.

The experiments in Fig. 3 were performed at low-photopic light levels of ~4,000 R*/cone/sec. Previous work has shown that mean luminance can change the spatial processing properties of RGCs (Barlow et al., 1957), including nonlinear spatial integration (Grimes et al., 2014). These observations suggest two questions: (1) Does the observed On/Off asymmetry persist when we change the mean light level? (2) Do changes in mean light level change the spatial RF structure in ways that impact natural image encoding? To answer these questions, we repeated the experiments and analysis shown in Fig. 3 at “low” (~500 R*/cone/s), “medium” (~5,000 R*/cone/s, similar to the data in Fig. 3), and “high” (~50,000 R*/cone/s) mean luminance levels. The correlation between measured response differences and model response differences was larger for Off compared to On parasol RGCs at the two highest light levels (p<0.005 for both medium & high), but not at the lowest level (Fig. 4F). For Off parasol RGCs, this correlation value was smaller at the lowest light level compared to the higher light levels (p < 0.005 in each case). Thus, nonlinear spatial integration contributes less to Off parasol RGC responses to natural scenes at lower light levels.

These experiments revealed an additional change in spatial processing in the On parasol RGCs. In particular, at the highest light levels, there was a tendency for the (relatively small) deviations from linear integration to negatively correlate with our subunit model predictions (Fig. 4F). This suggests a light-level dependent contribution from an additional nonlinear mechanism, operating at a small spatial scale—for example, adaptation to luminance or contrast in the photoreceptors.

On and Off parasol RGC receptive fields contain small nonlinear subunits

The asymmetry in the impact of nonlinear spatial integration on On and Off parasol RGC responses to natural images is puzzling given that both cell types exhibit nonlinear responses to high spatial frequency grating stimuli (Petrusca et al., 2007; Crook et al., 2008). Could a difference in nonlinear RF structure between On and Off parasol RGCs underlie the difference in spatial integration revealed using natural stimuli? To answer this question, we used grating stimuli to probe the spatial scale and contrast dependence of nonlinear RF subunits.

As in our experiments with DOVES movies and natural image stimuli, we started each recording by finding the center of the RF and measuring its size (Fig. 5A,B). To test for the presence of small nonlinear subunits in the RF, we centered a contrast reversing grating stimulus over the RF. A spatially linear RF will produce no response to the temporal modulation of this stimulus (Fig. 5C, left), but a RF composed of subunits with rectified output will produce a response at every half cycle of the temporal modulation (Fig. 5C, right). This frequency-doubled (F2) response is a classical signature of nonlinear spatial integration (Enroth-Cugell & Robson, 1966; Hochstein & Shapley, 1976a).

We used stationary, contrast-reversing gratings modulated at 4 Hz to characterize the spatial properties of nonlinear RF subunits of On and Off parasol RGCs. For both cell types, spike outputs and excitatory and inhibitory synaptic inputs showed strong F2 responses to this stimulus (Fig. 5D–G). Varying the width of the bars composing the grating revealed that the F2 response in all cases was present at a spatial scale much smaller than the RF center size (Fig. 5D–G). This indicates that nonlinear RF subunits are several times smaller than the RF center. F2 responses of excitatory and inhibitory inputs peaked for bar widths ~1/3rd of the RF center size and declined for larger bar widths. This characteristic dependence on bar width can be explained by a center-surround organization of the subunits. These experiments show that both On and Off parasol RGCs exhibit nonlinear responses at a spatial scale substantially smaller than the RF center.

To test the contrast sensitivity of the nonlinear response, we presented sinusoidally-modulated uniform or split-field grating stimuli over the RF center at a range of contrasts (Fig. S3). Spike and synaptic current responses in both cell types showed nonlinear F2 responses across a range of contrasts. Off parasol RGCs showed 1.5- to 2-fold stronger F2 responses in spike output and excitatory synaptic input than On parasol RGCs. The reverse was true for inhibitory inputs to these cells. It is not clear, however, how these differences between On and Off parasol cell responses can explain the large asymmetry in spatial integration of natural images.

Asymmetric synaptic rectification in On and Off parasol RGC pathways

Figure 5 shows that grating stimuli reveal small nonlinear subunits in RFs of both On and Off parasol RGCs. If deviations from linear integration are the result of nonlinear subunit RF structure (Fig. 3,4), what then accounts for the On/Off asymmetry in spatial integration of natural images? A key property of nonlinear subunit RF models—including the one we present above—is rectification of non-preferred contrasts. Previous work has identified subunit rectification as an important regulator of integration of spatially structured visual inputs in mouse RGCs (Schwartz et al., 2012; Grimes et al., 2014).

To test for differences in rectification between On and Off parasol RGCs, we measured proxies for the subunit nonlinearities in each of these cells. Because nonlinear subunit RF structure is present at the level of synaptic currents (Fig. 5), the relevant nonlinearity will be reflected in these inputs. We used voltage clamp recordings to measure the contrast-response functions of synaptic currents in On and Off parasol RGCs for uniform discs. In the simplified case where a population of homogeneous subunits conveys the synaptic input, the shape of the contrast response function is identical to that of the subunit output nonlinearity. More generally, the contrast response function constrains the shape of subunit nonlinearities, and the degree of rectification correlates with the degree of subunit output rectification.

Excitatory currents in On parasol RGCs were approximately linear for low contrasts and became rectified for stronger negative contrasts (Fig. 6A). Inhibitory input to the same cells was very sharply rectified at non-preferred (positive) contrasts. On parasol RGCs received a small amount of feedforward (i.e. On-derived) inhibitory input (Fig. 6B), indicated by the increase in inhibitory input at high positive contrasts.

Excitatory input to Off parasol RGCs was sharply rectified even for low contrasts, and correspondingly the cells received very little tonic excitatory input (Fig. 6C). Off parasol RGCs receive a sizable feedforward (Off-derived) inhibitory input at high negative contrasts (Fig. 6D). The asymmetry in rectification of the excitatory inputs to On and Off parasol RGCs is consistent with previous reports of stronger rectification of Off parasol RGC spike responses (Chichilnisky & Kalmar, 2002). Rectification of responses to white noise stimuli (Fig. S4; see also Trong & Rieke, 2008) showed a similar asymmetry.

We modified our LN and nonlinear subunit models to use a smooth function fit to the average excitatory contrast-response functions for either On or Off parasol RGCs (Fig. 6E). We presented these models with randomly-selected patches from a natural image (Fig. 6F). The differences in model responses were stronger with the Off parasol nonlinearity than with the On parasol nonlinearity (Fig. 6F). For each of the 101 natural images in the DOVES database, we computed the mean NLI for subunit and LN model outputs across all patches drawn from that image (Fig. 6G). For both the On and Off parasol RGC nonlinearity, the mean NLI was positive for every image, indicating that the subunit model tended to produce stronger responses than the LN model. For every image, the Off nonlinearity produced a more positive NLI value than did the On nonlinearity. We saw similar results when we assigned the more rectified Off nonlinearity to the On cell model and vice-versa (by reflecting the nonlinearities across the vertical axis, not shown), indicating that the shape of the nonlinearity produced these differences rather than some feature of negative contrast regions of natural images.

The stronger nonlinear behavior seen with the Off nonlinearity is consistent with the stronger nonlinearities seen in Off parasol RGC excitatory responses to DOVES stimuli and image patches. The decrease in Off parasol rectification and NLI with decreasing light level (Fig. 4) further supports the importance of rectification in nonlinear spatial integration. These models of the excitatory inputs to On and Off parasol RGCs can also account for the cells responses to grating stimuli (Fig. S5). The NLIs predicted by the On parasol nonlinearity are stronger than those observed in the spike output of these cells (see Fig. 1). This suggests that other mechanisms shape spatial (non)linearity in On parasol RGCs, for example the large, nonlinear inhibitory inputs to these cells (Fig. 2).

These models show that the degree of subunit rectification determines the degree to which a nonlinear RF can be approximated as linear in the context of natural visual stimulation. A similar effect of subunit rectification can be seen using a simplified piecewise-linear nonlinearity (Fig. S6), which shows that this is a general consequence of rectified subunit nonlinearities and does not depend on the precise shape of the nonlinearity.

A nonlinear subunit model improves predictions of responses to natural images

To determine the extent to which incorporating a nonlinear RF structure can improve predictive models of Off parasol RGC responses, we used our excitatory subunit and LN models from above (Fig. 6E–G) to build simple predictive models of Off parasol RGC excitatory synaptic current and spike output responses (Fig. 7A). We start by predicting excitatory synaptic inputs in response to natural image patches, and then expand the model to predict spike responses.

For each cell, we used the measured RF center size (based on area-summation curves, e.g. see Fig. 5A,B), as the RF size in the model, and set the subunit size to be 1/6 of this measured RF center size. Models incorporated the measured nonlinear contrast-response function for excitatory synaptic inputs to Off parasol RGCs (from Fig. 6C). As with the modeling above, performance was similar for a range of subunit and RF center sizes, as long as the RF center was several times larger than a subunit (Fig. S2).

To test how well these models predict excitatory current responses to natural stimuli, we voltage clamped Off parasol RGCs and measured the excitatory charge transfer in response to presentation of 40 randomly-selected patches from a natural image. We normalized the excitatory responses to control for across-cell differences in response strength. We then compared the measured excitatory responses to those predicted by each of the two models (Fig. 7A). The LN model underestimated the excitatory current response for many image patches, and the nonlinear subunit model gave a better prediction (Fig. 7B,C, p=3.9×10⁻³). On average, the subunit model explained 92% of the measured variance, while the LN model accounted for 81% (Fig. 7C).

To test whether a nonlinear excitatory RF structure could also improve predictive models of spike output, we expanded the excitatory models above to include simplified forms of inhibitory input and spike generation. In both models the inhibitory input is purely linear and exclusively cross-over (On-tuned). The excitatory and inhibitory inputs are combined linearly (with a scaling of excitatory inputs to account for the approximately 3-fold difference in driving force acting on excitatory conductances at spike threshold) and passed through a rectifying nonlinear function to yield the spike output of the cell.

Even with these simplified forms of inhibition and spike generation, the subunit model improves the spike count prediction (Fig. 7D,E; p=2.4×10⁻⁴). On average, the subunit model accounted for 82% of the response variance while the LN model captured only 57%, showing that most of the errors made by an LN model in predicting spike responses to natural images can be corrected by incorporating nonlinear excitatory subunits into the RF. This improvement in prediction with the excitatory subunit model is robust to changes in the form of inhibitory input. For example, using a rectified (rather than linear) cross-over inhibition or even removing inhibition altogether did not qualitatively change the results shown here. This suggests that the improvement in prediction of Off parasol RGC spike responses stems from the improved predictions of the cell’s excitatory inputs.

There are many aspects of retinal circuitry and intrinsic RGC computation that these models exclude. Notable examples include local adaptational mechanisms (Dunn et al., 2007; Kastner & Baccus, 2013), heterogeneous/non-Gaussian linear RF structure (Field et al., 2010; Schwartz et al., 2012), and realistic inhibitory inputs and spike generation mechanisms. These and other features will likely be important to include in models that predict parasol cell responses, especially when presented with time-varying stimuli (like the DOVES stimuli in Fig. 1 & 2). Indeed, the observation that an On parasol RGC’s synaptic inputs can be spatially nonlinear, while its spike output is linear (when probed with natural images), implies that additional mechanisms, beyond rectification of excitatory subunits, shape spatial integration at the level of spike outputs in On parasol RGCs. Nevertheless, the substantial improvement in predictions of Off parasol RGC responses to natural images suggests that excitatory nonlinear subunits will be an essential part—indeed perhaps the most important of the mechanisms listed above—of full predictive models for some RGC types.

Discussion

We investigated the extent to which nonlinear spatial integration affects RGC responses to natural visual stimuli. We found that responses of On but not Off parasol RGCs were well approximated using a spatially linear RF, despite both cell types showing nonlinear responses to grating stimuli designed to probe spatial nonlinearity. Sharply rectified excitatory subunits in the RF center mediate the nonlinear responses of Off parasol RGCs. Finally, we found that a simple RF model that includes rectified subunits accurately predicts responses to natural images, capturing approximately 80–90% of the response variance; this represents a substantial improvement over models with linear spatial integration.

Synaptic rectification controls nonlinear integration

The sharply rectified excitatory subunits in the Off pathway mediate the nonlinear spatial integration of Off parasol RGCs; excitatory inputs to On parasol RGCs exhibited substantially milder rectification. Excitatory inputs to brisk transient RGCs in guinea pig show a similar On/Off asymmetry of rectification (Demb et al. 2001). In addition to differences among cell types, synaptic rectification can differ within the same cell type as lighting conditions change (Grimes et al., 2014). The degree of synaptic rectification may be a common control point for retinal circuits that can change with cell type and stimulus conditions. Indeed, Off parasol RGCs behaved more linearly at lower light levels (Fig. 4).

On parasol RGC responses to natural images were more linear than expected given the degree of excitatory subunit rectification and the nonlinear responses to contrast-reversing grating stimuli. Thus other mechanisms appear to linearize On parasol RGC responses. Two lines of evidence support this conclusion.

First, the excitatory synaptic inputs to On parasol RGCs in response to natural inputs showed only modest deviations from linear spatial integration (Fig. 2). This is surprising considering the clear nonlinear responses elicited by low-contrast grating stimuli (Fig. S3). The grating stimuli used here, and in past work, were designed to elicit nonlinear (F2) responses while minimizing linear (F1) response components. This was achieved by carefully centering the stimulus over the RF center and/or using high spatial frequency gratings. When presented with natural images, however, very rarely is the linear response component so effectively nulled. Most natural image responses contain both linear and nonlinear response components, and it is not known whether these response components interact in a straightforward fashion or if, for instance, the linear response component can suppress nonlinear responses. Consistent with this hypothesis, the On parasol RGC excitatory subunit model (Fig. 6) could reproduce observed grating responses (Fig. S5), but over-estimated the NLI in response to natural images.

Second, the small deviations from linearity in the On excitatory inputs (Fig. 2) are absent at the level of spike output. Nonlinear cross-over inhibitory input to On parasol RGCs could further “linearize” responses to images (Werblin, 2010; Cafaro & Rieke, 2013). The images that drive strong nonlinear responses in the excitatory input would also be expected to drive nonlinear responses in inhibitory input, resulting in weaker nonlinear responses in spike output compared to excitatory synaptic inputs.

A nonlinear receptive field enhances responses to spatial structure in natural images

Nonlinear responses to natural images depended strongly on the particular region of the image presented. This was true for the DOVES movies (Fig. 1), the stationary natural images (Fig. 3), and the nonlinear RF model responses (Fig. 3, 4, 6, 7). What is it about these images that drive strong deviations from linear integration? The mechanism of nonlinear integration—rectification of non-preferred contrasts and thus incomplete cancelation of light and dark regions of a scene—suggests an answer. In particular, homogeneous images that drive most of the subunits in the same direction (e.g. a population of On subunits all subjected to strong positive contrast) will produce the same output in a LN or a comparable nonlinear subunit model. However, images containing a wide distribution of contrasts strongly activate some subunits and deactivate others, causing a linear and a nonlinear RF to produce very different responses.

To explore this intuition, we compared the difference in model outputs for randomly sampled natural image patches (as in Fig. 3) with the normalized variance of each image patch (variance in pixel intensities divided by the average intensity of the patch). These two values were positively correlated (0.43 ± 0.18 mean ± standard deviation, linear correlation coefficient) for 10,000 patches from each of 101 natural images. Thus heterogeneous natural images more strongly drive nonlinear spatial integration than homogeneous images.

Where in natural scenes are the relatively “nonlinear” patches located? To illustrate this, we passed each RF model over every location within several natural images and computed the model output difference (nonlinear subunit minus LN model). Figure 8 shows example heat maps of model differences and the original images from which they were generated. For some images, it is clear that spatial inhomogeneity drives nonlinear responses (Fig. 8A,B), whereas regions of the scene that are relatively homogeneous at the scale of the RF center (e.g. the tree trunk or sky in Fig. 8C) activate both models approximately equally. Some images show strong nonlinear responses clustered along boundaries or edges within the scene (Fig. 8A,C).

Fig. 8. Rectified subunit RF structure enhances responses to spatial structure in natural images.

(A–C) Left: example natural images were passed through each of the models in Fig. 4, with each point in the image sampled. Right: for each image, we constructed a heat map of model differences (subunit minus LN model) to illustrate the locations within the scene where the two models differ. For scale, the RF center size is indicated in panel (C). Color scale in (C, right) applies to all heat maps.

Thus, a nonlinear RF composed of rectified subunits enhances responses to regions of scene with a large degree of spatial inhomogeneity, including edge structure. Whether and how downstream circuits use this information is unknown. Psychophysical evidence suggests that during free-viewing, new fixation locations are not randomly dispersed through the visual field, but that saccades are typically directed towards regions of the scene with edges (Reinagel & Zador, 1999; Baddeley & Tatler, 2006). The highly contrast sensitive and nonlinearly integrating peripheral Off parasol RGCs would be well-suited to detect these regions of a natural scene.

Another consequence of the enhanced sensitivity to spatial contrast in Off parasol RGCs is that some natural visual stimuli will activate both On and Off parasol RGCs simultaneously. In particular, this will occur when a saccade is made to an image patch that contains high spatial contrast (e.g. an edge) but is, on average, brighter than the previous fixation. For example, see the spike responses to fixation number 5 in Fig. 1C,E. This suggests that during natural vision, the standard division of labor in which On cells signal increases in luminance and Off cells signal decreases in luminance is too simplistic. In reality, RGCs encode multiple features of the scene (here, for example, mean luminance and spatial contrast), and RGC responses are shaped by a combination of sensitivities to these features.

Implications for constructing generalizable models

A linear RF is a common feature of predictive models in the visual system, but its ability to capture responses to natural visual stimuli has been largely untested (see Heitman et al., 2016 for an exception). Our observations suggest that to predict responses to natural visual inputs, models of some RGC types will need to account for nonlinear RF structure, while for other RGC types a linear approximation suffices. In the case of Off parasol RGCs, we found that the precise subunit organization was not as important as simply having several rectified subunits that are smaller than the RF center. The same model may apply quite directly to other classes of RGC that contain rectified subunits.

Which RGCs can be approximated using a linear RF is not clear from responses to typical artificial stimuli such as gratings. The use of natural stimuli to test models of neural computation engages a variety of mechanisms in ways that are not easily predicted from responses to artificial stimuli. Artificial stimuli probe only a very small region of stimulus space, which makes them ideal for dissecting a particular mechanism of interest (e.g. grating stimuli to probe nonlinear spatial integration), but poorly suited when attempting to place circuit mechanisms in their broader functional context. A similar approach may enhance understanding of the coding impact of several other mechanisms, including interactions between the center and surround, photoreceptor adaptation in the context of spatially structured stimuli, and integration of spatially nonlinear synaptic inputs.

While the results presented here suggest that spatially nonlinear models will be required to capture natural scene responses for some classes of visual neurons, further work is needed to develop methods of constructing such models and fitting them to data. Already some important progress has been made on this front (e.g. see Freeman et al., 2015; Vintch et al., 2012; Vintch et al., 2015). We hope that the work presented here will guide the development of predictive models by highlighting specific mechanisms that substantially shape neural responses to natural stimuli.

Experimental Procedures

Tissue preparation

Retinal tissue was obtained from terminally anesthetized Macaque monkeys (M. nemestrina, M. mulatta, or M. fascicularis) of either sex via the tissue distribution program at the Washington National Primate Research Center. All procedures were approved by the Institutional Animal Care and Use Committee at the University of Washington. After enucleation, the eye was hemisected and the vitreous humor was removed mechanically. In some cases, the eye cup was treated for ~15 minutes with human plasmin (~50 μg/mL, Sigma or Haematologic Technologies Inc.) to aid in removal of vitreous. We observed no differences in retinal health or sensitivity after plasmin treatment. The retina was dark adapted for ~1 hr, and all subsequent procedures were performed under infrared light using night-vision goggles. The retina and pigment epithelium were separated from the sclera and stored in oxygenated (95%O₂/5%CO₂) Ames bicarbonate solution (Sigma) in a light-tight container. Retinal mounts (~2–3 mm on a side) were removed from the pigment epithelium and laid flat, photoreceptor-side down, onto a poly-D-lysine coated coverslip (BD biosciences) before being placed in a recording dish that was continuously perfused at 7–9 mL/min with Ames solution at 31–35ºC. During recording, the tissue was visualized using infrared illumination and DIC optics.

Visual stimulation

Visual stimuli were presented on an OLED microdisplay monitor (eMagin, Bellevue, WA) focused, via a microscope condenser, onto the photoreceptors. The 800×600 monitor display was presented with a resolution of 1.2 μm/pixel at the retina. Stimuli were presented and data acquired using custom written stimulation and acquisition software packages Stage (stage-vss.github.io) and Symphony (symphony-das.github.io). Monitor outputs were linearized by gamma correction. Stimuli were calibrated using monitor power outputs, the spectral content of the monitor, macaque photoreceptor spectral sensitivity (Baylor et al., 1987), and a collecting area of 0.37 μm² for cones (Schnapf et al., 1990) and 1 μm² for rods. Unless otherwise noted (i.e. Fig. 4), mean light levels produced ~4,000 isomerizations (R*)/M or L-cone/s, ~1,000 R*/Scone/s and ~8,000 R*/rod/s.

Patch recordings

Electrophysiological recordings were performed using a Multiclamp 700B amplifier (Molecular Devices). Spike responses were measured using extracellular or loose-patch recordings with an Ames-filled pipette (tip resistance ~2–6 MΩ). For voltage clamp recordings, we used low-resistance pipettes (tip resistance ~1.5–4 MΩ) filled with a Cs-based internal solution (containing, in mM: 105 CsCH₃SO₃, 10 TEA-Cl, 20 HEPES, 10 EGTA, 5 Mg-ATP, 0.5 Tris-GTP, and 2 QX-314, pH 7.3, ~280 mOsm). We compensated for access resistance (~4–12 MΩ) online by 50–75%. Reported voltages have been corrected for an approximately −10 mV liquid junction potential.

We isolated excitatory (inhibitory) synaptic inputs by holding at the reversal potential for inhibitory (excitatory) synaptic currents. Reversal potentials were estimated for each cell by delivering light steps near the expected reversal potentials (approximately 0 mV for excitatory inputs and −60 mV for inhibitory currents) and adjusting the holding potential to minimize the appropriate current.

Cell identification and selection

On and Off parasol RGCs were first identified under DIC optics by the size and morphology of their soma, which was generally a very reliable indicator of cell type. We confirmed cell type by delivering light stimuli over the RF center to observe the transient spike responses and high contrast sensitivity characteristic of parasol RGCs. The overall health and sensitivity of the retina was confirmed by delivering a uniform, 5% contrast, 4 Hz modulated stimulus, which produces a robust spike response in On parasol RGCs in sufficiently sensitive tissue. Sensitivity was continuously monitored (typically before each recording) in this way. Mean spike rates in healthy, sensitive tissue at a mean light level of ~4,000 R*/cone/s were 12.8 ± 1.5 Hz for On parasol RGCs and 2.2 ± 0.6 Hz (mean ± S.E.M.) for Off parasol RGCs.