Parallel processing of rod and cone signals: retinal function and human perception

Abstract

We know a good deal about the operation of the retina when either rod or cone photoreceptors provide the dominant input (i.e. very dim or very bright conditions). However, we know much less about how the retina operates when rods and cones are co-active (intermediate lighting conditions—e.g. dusk). Such mesopic conditions span 20–30% of the light levels over which vision operates and encompass many situations in which vision is essential—e.g. driving at night. These lighting conditions are challenging because rod and cone signals differ substantially: rod responses are nearing saturation, while cone responses are weak and noisy. A rich history of perceptual studies can guide our investigation of how the retina operates under mesopic conditions, and in doing so provide a powerful opportunity to link general issues about parallel processing in neural circuits with computation and perception. We review some of the successes and challenges in understanding the retinal basis of perceptual rod-cone interactions.

Introduction

At any given moment, roughly half of our spinning planet is bathed in light from the sun, while the other half is dimly lit by stars and sunlight reflecting off the moon. These two visual environments exhibit vastly different properties and place correspondingly very different demands on the visual system. A striking aspect of vision is its ability to work seamlessly across these dramatic changes in input. This seamlessness belies the substantial changes required in the operation of the retinal circuits that support vision during each diurnal cycle.

At the extremes—starlight (scotopic conditions) and sunlight (photopic conditions)— vision relies on signals generated by either the rod or cone photoreceptors. Rods, cones, and the associated retinal circuits exhibit several specializations matched to the differences in the conditions under which they operate. In starlight, rods generate discernable responses to individual absorbed photons (Baylor et al., 1979; Baylor et al., 1984), and the resulting signals are protected from noise by several synaptic specializations as they are transmitted through the retina (Field and Rieke, 2002; Sampath and Rieke, 2004; Ala-Laurila and Rieke, 2014; Grimes et al., 2015). As a result, vision under these conditions is limited as much by the division of light into discrete photons as it is by biological noise (reviewed by (Donner, 1992; Field et al., 2005; Rieke, 2008)). Cones have faster responses than rods and their responses are processed in parallel by multiple distinct retinal circuits (reviewed by (Field and Chichilnisky, 2007; Masland, 2012)). Cone vision supports the high spatial and chromatic acuity that dominates our daytime visual experience.

But separate rod and cone vision is not the whole story. We spend a good deal of each day at intermediate (mesopic) light levels, where both rods and cones are active. Mesopic vision operates at light levels from moonlight to dawn or dusk, and includes most artificially lit nighttime environments (e.g. night driving). Vision under these conditions is challenging largely because the rod and cone photoreceptor signals differ substantially. Rod responses approach saturation, and hence adaptive gain control mechanisms in the rods and post-rod retinal circuits are essential for reducing the gain of rod signaling to match the prevailing inputs. On the other hand, cone responses are emerging and weak, and hence amplifying these nascent signals and protecting them from noise is essential. Likely because of the challenges these transitional conditions pose, deficits in mesopic vision can provide early indications of the onset of visual diseases (Petzold and Plant, 2006; Arden and Hogg, 1985).

Perceptual rod-cone interactions have been studied for more than a century. This rich history of work highlights interactions between signals generated in the rod and cone photoreceptors that influence the spatial, temporal and chromatic sensitivity of human vision under mesopic conditions (reviewed by (Buck, 2004; Buck, 2014; Stockman and Sharpe, 2006)). Because signals from rods and cones converge within retinal circuits to modulate the responses of a common set of retinal ganglion cells, retinal circuits have been implicated in several perceptually-relevant interactions between rod and cone signals. This provides a unique and powerful opportunity to understand how parallel processing of distinct inputs shapes perception.

We begin by reviewing some key aspects of the retinal circuitry and the routing of signals through it that are relevant to rod-cone interactions. We then describe several well-studied types of perceptual rod-cone interactions, with a specific emphasis on opportunities to connect visual perception to retinal mechanisms. Lastly, we present a kinetic model that describes the retinal integration of rod and cone signals for a range of mesopic stimuli.

Retinal circuitry and potential sites of rod-cone interactions

Decades of beautiful anatomical work, culminating in modern electron microscopy, has led to a near complete list of retinal cell types and an emerging understanding of their connectivity (reviewed by (Wu, 2010; Masland, 2012; Dunn and Wong, 2014)). These studies provide a foundation for considering the possible sites at which rod and cone signals could interact within retinal circuits and contribute to perceptual rod-cone interactions.

Rod and cone photoreceptors in mammalian retinas are tightly packed and highly intermingled in the outermost retinal layer, which ensures that they sample much of same visual space. An exception is the primate fovea (representing <1% of the total retinal area), which lacks rod photoreceptors and contains specialized circuitry for high acuity cone-mediated vision. The lack of rods in the fovea is clear when trying to perceive small dim objects, like distant stars. Instead of looking at such objects directly, maximal sensitivity is achieved when the object is 5–10 degrees away from the center of gaze, thus projecting the object on a region of the retina that contains a high density of rod photoreceptors.

Rod-cone interactions are shaped by differences in the relative densities of rod and cone photoreceptors across the retina. Thus, rod-cone interactions are absent in the fovea since it lacks rods. The density of rods relative to cones increases with distance from the fovea. Likely as a direct result of this increasing rod density, mesopic vision operates at higher light levels for peripheral vision, compared to near-foveal vision. The effect is large – with the midpoint of mesopic vision differing by as much as a factor of 10 (Raphael and MacLeod, 2011). This means that even within a single visual scene, signaling in different retinal regions can be dominated by inputs from different photoreceptor types.

Convergence is an important feature of peripheral retinal circuits, which are driven by both rods and cones. While RGCs in the fovea receive (indirect) input from single cones, RGCs in the peripheral retina receive input from tens to hundreds of cones and hundreds to thousands of rods. This high convergence in the peripheral retina enhances sensitivity, enables a faster and more reliable encoding of stimuli and creates ample opportunities for rod-cone interactions.

Rod and cone signals intermingle at multiple points within the retinal circuits. This begins in the outer retina, where rods and cones are connected via gap junctions (Figure 1A). These electrical synapses allow rod signals to be conveyed directly to cones (Nelson, 1977; Schneeweis and Schnapf, 1995a; Schneeweis and Schnapf, 1999). Rod and cone signals can also interact in the outer retina through horizontal cells (Figure 1A). Mammalian horizontal cells create lateral interactions between rods and cones by receiving synaptic input from cones and providing feedback synaptic output to both rods and cones (Kolb and Famiglietti, 1974; Boycott et al., 1987; Mariani, 1984). Horizontal cell feedback opposes signals generated in the photoreceptor outer segments, and in doing so contributes to center-surround receptive fields and to the transient nature of retinal responses to steps in light intensity (Dacey et al., 2000; Mangel, 1991).

Figure 1:

Potential sites of retinal rod-cone interactions. A. Sites in the outer retina include (1) gap junctions between rod and cones and (2) horizontal cells, which receive input from cones and provide feedback to rods and cones. B. In the inner retina, rod and cone signals are combined in the cone bipolar synaptic terminal. In the depicted On pathway, signals from the rod bipolar cell are conveyed by gap junctions to cone bipolar terminals, where they mix with cone signals originating from input to the cone bipolar cell dendrites.

Rod and cone signals are transmitted from the outer retina to the inner retina via ten or more subtypes of bipolar cells (reviewed by (Masland, 2012; Dunn and Wong, 2014; Euler et al., 2014)). One bipolar type, the rod bipolar cell, preferentially contacts rods, while the remaining types (cone bipolar cells) preferentially contact cones. Electron microscopy of the outer retinas of non-human primates and mice reveals substantial species differences in the selectivity of photoreceptor-to-bipolar synaptic connectivity. In mouse, some rod bipolar cell dendrites contact cones, and some cone bipolar cells contact rods (Behrens et al., 2016; Hack et al., 1999; Tsukamoto et al., 2001); physiological studies confirm these connections (Pang et al., 2010). These aberrant connections are much less prevalent in primate retina (Tsukamoto and Omi, 2014; Tsukamoto and Omi, 2016); most notably, rod bipolar cells appear to exclusively contact rods. This clearer anatomical segregation of rod and cone signals in primate compared to mouse is an important consideration for the mechanistic basis of rod-cone interactions. We revisit this issue below.

In the inner retina, cone bipolar cells make direct synaptic contacts with retinal ganglion cells (RGCs), whereas rod bipolar cells do not. To reach the RGCs, signals routed through dedicated rod bipolar cells are transmitted to AII amacrine cells via conventional glutamatergic synapses, and are subsequently transmitted to On cone bipolar cells through gap junctions and to Off cone bipolar cells through glycinergic synapses (On circuits illustrated in Figure 1B).

Of particular importance to rod-cone interactions, rod and cone signals are combined in cone bipolar cells before being transmitted synaptically to RGCs (Gouras and Link, 1966; Enroth-Cugell et al., 1977; Field et al., 2009). Due to this convergence, RGCs transmit combined rod and cone signals to downstream visual areas. At the same time, different RGC types process cone signals differently. Notably, midget (parvocellular-projecting) RGCs respond strongly to chromatic stimuli, whereas parasol (magnocellular-projecting) RGCs respond strongly to luminance (reviewed by (Dacey and Packer, 2003; Solomon and Lennie, 2007)). Thus, for some stimuli cone signals can be carried preferentially by one RGC type, while another RGC type may dominate rod signals. This is another important consideration for where and how rod-cone interactions can occur (Sun et al., 2001).

Routing of rod and cone signals through retinal circuits

The circuitry described above provides a map of the possible routes that rod and cone signals can take through the retina and the possible sites of interactions between them. Where and how rod and cone signals interact will depend in large part on which of these routes are active under specific conditions. This routing appears to depend on both species and mean luminance.

Work from cat and mouse has revealed three distinct pathways through which rod signals can traverse the retina and modulate RGC responses (Figure 2; reviewed by (Field et al., 2005; Demb and Pugh, 2002)). In the ‘primary’ or rod bipolar pathway (Figure 2A), rod signals are transmitted through rod bipolar cells and AII amacrine cells to cone bipolar cells and hence to RGCs (Nelson and Kolb, 1984). In the ‘secondary’ pathway (Figure 2B), rod signals are transmitted directly to cones through gap junctions (Kolb, 1977; Tsukamoto et al., 2001; Schneeweis and Schnapf, 1995b). In the ‘tertiary’ pathway (Figure 2C), rod signals are transmitted directly to a subset of cone bipolar cells (Tsukamoto et al., 2001; Soucy et al., 1998; Cowan et al., 2016).

Figure 2:

Pathways conveying rod signals through the retina. A. Primary or rod bipolar pathway, in which rod signals are transmitted through the dedicated rod bipolar cell and AII amacrine cell. B. Secondary or rod-cone pathway, in which rod signals are conveyed to cones via gap junctions. C. Tertiary pathway, in which rods provide direct input to a subset of cone bipolar cells, such as the Off cone bipolar cell depicted here.

A prominent hypothesis about the importance of these different pathways is that vision meets the competing demands associated with operating in starlight and moonlight using different circuits with unique specializations (Tsukamoto et al., 2001; Deans et al., 2002). The rod bipolar pathway is the dominant route that rod signals take across the retina at low scotopic light levels (Deans et al., 2002; Murphy and Rieke, 2006). Several aspects of this pathway are well suited for reliably transmitting single photon responses in the rod photoreceptors, hence supporting vision in starlight (reviewed by (Field et al., 2005)). Particularly important among these specializations is amplification, which permits a RGC to respond when photons are absorbed in only a few of the thousand or so rods from which it receives converging input (Barlow et al., 1971; Mastronarde, 1983; Ala-Laurila and Rieke, 2014).

The high gain associated with detecting sparse photons, if maintained, would lead to saturation of retinal responses as light levels increase. One hypothesis about how such saturation is avoided is that rod signals begin to traverse the retina through the secondary and tertiary pathways at intermediate light levels. Work in rodent retina supports this hypothesis. Evidence for this change in routing includes a combination of genetic and pharmacological manipulations and recordings from key components of each pathway (Soucy et al., 1998; Deans et al., 2002; Trexler et al., 2005; Grimes et al., 2014; Ke et al., 2014). Collectively, these experiments indicate that a significant proportion (but not all) of the rod-mediated modulation of RGC responses comes from the secondary and tertiary pathways under low mesopic conditions in rodents.

The rod bipolar pathway itself, however, also has strong adaptational mechanisms that reduce gain and hence can maintain sensitivity at intermediate light levels (Dunn and Rieke, 2008; Oesch and Diamond, 2011; Jarsky et al., 2011). Indeed, several pieces of evidence suggest that the routing of rod signals in primate may differ from that in rodent. First, as described in the previous section, bipolar connections with rods and cones appear to be better segregated in primate retina compared to rodent retina. Second, rod signals in the middle of the mesopic range are weak in key elements of the secondary pathway (cones and horizontal cells) compared to rod signals in ganglion cells (Grimes et al., 2015). This suggests that most of the rod signals reaching the RGC are routed through the primary pathway under these conditions. If correct, this would imply that rod and cone signals are largely independent as they traverse the primate retina, and are combined in the cone bipolar synaptic terminal before being transmitted to RGCs. This in turn would significantly constrain the possible sites of rod-cone interactions in the retina.

Prospects for relating perceptual and retinal mechanisms

The anatomical and physiological findings summarized above indicate that rod and cone signals are combined prior to transmission down the optic nerve; this has led to the hypothesis that perceptual interactions between rod and cone signals could originate in the retina. Nonetheless, there are inherent difficulties in connecting human perceptual results to biophysical mechanisms. First, perceptual interactions could arise anywhere along the visual pathway, and separating the contributions of different locations can be difficult. Second, while work in rodents provides direct access into many mechanistic underpinnings of the visual pathways, insights from these models may not be directly applicable to the human visual system. With these caveats in mind, we next discuss three types of rod-cone interactions and their possible connections with retinal circuits: (1) spatial interactions and independence of adaptation, (2) chromatic interactions and hue shifts, and (3) temporal interactions.

Adaptation

Does the adaptational state of rod signals influence cone signals? What about the converse – does cone adaptation affect rod signals? Adaptation of rod- and cone-derived signals, when studied independently, depends on multiple mechanisms (reviewed by (Burns and Baylor, 2001; Dunn and Rieke, 2006; Demb and Singer, 2015)). These mechanisms include adaptation in the photoreceptors themselves and in post-photoreceptor circuits in retina and cortex. Adaptation in the photoreceptors will naturally be independent for rods and cones. For post-photoreceptor mechanisms, a key consideration is location relative to the point of convergence of rod- and cone-derived signals: mechanisms located prior to convergence have the capability of acting independently on rod and cone signals, while those operating post-convergence do not.

Psychophysically, the codependence or independence of rod and cone adaptation under mesopic conditions has been studied using test and background stimuli with a wide range of sizes, wavelengths, and strengths. These experiments reveal a diverse set of interactions and highlight some of the challenges in relating perception to retinal mechanisms.

Steady rod backgrounds can increase perceptual thresholds (i.e. decrease sensitivity) for small well-centered cone test flashes (Buck, 1985; Latch and Lennie, 1977; Temme and Frumkes, 1977), with a maximal impact for backgrounds with an extent of ~1°. Larger backgrounds minimally affect, or sometimes lower, thresholds for cone test flashes (Stiles, 1939; Westheimer, 1970; Blick and MacLeod, 1978). Cone backgrounds similarly have several effects on the detection of rod test stimuli. Spatially small cone backgrounds produce sustained increases in rod thresholds, whereas larger cone backgrounds produce smaller and more transient changes. Rod sensitivity is also more affected by long-wavelength backgrounds (i.e. backgrounds that increase activity in both rods and cones) than by rod-matched short-wavelength backgrounds (Sharpe et al., 1989; Latch and Lennie, 1977; Frumkes and Temme, 1977).

The rod-cone interactions described above emphasize lateral interactions – i.e. interactions between different regions of space. One early proposal, supported by direct recordings in amphibian retina, is that horizontal cells might mediate these interactions (Eysteinsson and Frumkes, 1989). Mammalian horizontal cells, however, receive input only from cones, and hence provide a mechanism for cone adaptation to influence rods but not the converse. The impact of cones on rod thresholds is eliminated or at least substantially weakened when the cone background and rod test stimulus are presented to different eyes (Buck and Pulos, 1987), suggesting an origin in the retina or other early visual areas that get monocular input.

Lateral interactions could occur at any location in the visual system that receives spatially-separated rod and cone signals, and the diversity of observed rod-cone spatial interactions could be due to experimental conditions that favor distinct mechanisms and/or adaption sites (Latch and Lennie, 1977; Blick and MacLeod, 1978). Specifically, central mechanisms may be engaged more strongly for small backgrounds than large backgrounds since low spatial frequencies are generally filtered out as signals traverse the visual hierarchy. In this proposal, the lateral interactions would occur in cortical circuits. However, interactions could differ for spatially coextensive stimuli – when the relevant rod and cone signals are more likely combined within retinal circuits and not available separately in cortex.

Hue shifts in mesopic vision

Humans are trichromats, normally expressing three types of cone photoreceptors with distinct absorption spectra (Figure 3). Cone absorption efficiency depends on wavelength, but once a photon is absorbed it triggers the same electrical response regardless of wavelength (Baylor et al., 1987). This means that intensity and wavelength are ambiguous from the responses of a single cone, and hence that color vision relies on comparing the relative activity of the three cone types (Rushton, 1972). At typical daytime light levels, perceived colors are well-predicted by relative cone activations (Stiles and Burch, 1959; Baylor et al., 1987; Wandell, 1995).

Figure 3:

Impact of rod signals on color appearance. A. Spectral sensitivities of rods (R) and short-(S), middle- (M) and long- (L) wavelength sensitive cones. B. (left) Spectral sensitivity of hypothetical RGC receiving input in the receptive field center from L cones and in the surround from combination of L and M cones with and without rod input. Rod signals are assumed to add identically to all cone signals. (right) As in left panel, but for a RGC receiving center input from S cones.

Color perception under photopic conditions can be remarkably constant—e.g. a reflecting object seen as yellow maintains its apparent color across a large range of mean light levels. But perceived colors change under mesopic conditions. A classic example is the Purkinje effect (Purkinje, 1825), which describes a shift in perceptual sensitivity towards short wavelengths at intermediate (dawn or dusk) compared to bright light levels. Probed systematically, hue shifts can exhibit a complex dependence on wavelength (see (Buck, 2014) for a more complete review).

How might rod signals impact color vision, and in particular can they explain the hue shifts that occur under mesopic conditions? The absorption spectrum of rod photoreceptors is distinct from the spectra of the three cone types (Figure 3A). All ganglion cells outside the fovea appear to receive both rod and cone input, and rods and cones produce qualitatively similar modulations in ganglion cell responses – e.g. responses of On cells are increased for light increments whether those increments are sensed by rods, cones or both (Gouras and Link, 1966; Enroth-Cugell et al., 1977). The combination of rod and cone signals with the same polarity allows rod activity to shift the wavelength sensitivity of the ganglion cell responses towards that of the rods. The effect of this shift on perception will depend on how those signals are interpreted by subsequent visual circuits. For example, a cell receiving predominant input in its receptive field center from long-wavelength sensitive cones will shift its sensitivity towards shorter wavelengths with rod input (Figure 3B). Based on this cell alone, additional rod activity might be expected to elicit a perception of increased redness (i.e. increased activity in the example cell is interpreted perceptually as an increase in the intensity of long wavelength light). Similarly, a cell receiving predominant input from short-wavelength sensitive cones will shift its sensitivity towards longer wavelengths with rod input (Figure 3C).

These considerations suggest a simple picture in which hue shifts can be predicted by each cone receiving identical rod input—e.g. through rod-cone gap junctions. Although qualitatively correct, this cannot account for the subtlety of rod mediated hue shifts. For example, identical rod inputs to long- and middle-wavelength sensitive cones might be expected to cancel if the difference between signals in these cone types determines red vs green perception. However, rod input creates a green bias, suggesting stronger rod input to middle-wavelength sensitive cones than long-wavelength sensitive cones ((Cao et al., 2005), reviewed in (Buck, 2014)). It is not clear where or how such cone-type-specific rod input occurs.

Rodents are dichromats, but unlike primates the spectrally-distinct cone types are not coexpressed throughout the retina. Instead, short and middle wavelength cones are largely segregated to the ventral and dorsal halves of the retina (Lyubarsky et al., 1999), whereas rods are expressed at high density across the entire retina. This suggests that rods can acts as a second, spectrally-distinct photoreceptor under mesopic conditions. This raises the possibility that mice exhibit enhanced color discrimination under mesopic conditions, and recent work indicates that horizontal cell-mediated interactions between rods and cones can give rise to color opponency in mice (Joesch and Meister, 2016). These same mechanisms could play a role in creating blue shifts in human perception in the mesopic regime (Joesch and Meister, 2016).

Rod-cone interactions over time

Several general features of neural circuits dictate how the integration of time-varying signals from parallel pathways controls circuit output, including: (1) differential shaping of response kinetics in parallel pathways, (2) differences in the distribution of common inputs into parallel pathways, and (3) the location of nonlinear circuit elements relative to the point of integration of signals from different parallel pathways. These issues figure prominently in how rod and cone signals interact over time.

Rod-cone flicker

Temporal interactions between flickering rod and cone signals can break the normal seamlessness of vision, revealing properties of the underlying mechanisms that are normally hidden; this is analogous to optical illusions, which often highlight specific properties of adaptation (reviewed by (Kingdom, 2011)). A well-studied example is the destructive interference of rod and cone signals as measured perceptually (MacLeod, 1972). MacLeod demonstrated this effect by monitoring the sensitivity of human observers to a high contrast flickering yellow light across a range of mean light levels and backgrounds (Figure 4). At low light levels, 7.5 Hz flicker is invisible due to the slow kinetics of signals from dark-adapted rods. The flicker transitions from invisible to visible as its intensity increases (i.e. moving vertically in Figure 4B). This transition occurs at purely scotopic light levels, and reflects an increase in rod sensitivity to flickering stimuli. As intensity continues to increase into the mesopic range, the flicker, unexpectedly, becomes invisible again. Further increases in intensity cause the flicker to again become visible as cones begin to dominate vision. The surprise is the anomalous transition from visible to invisible at low mesopic light levels—i.e. as cones begin to contribute to retinal signals, the stimulus becomes invisible. This transition is eliminated by the presence of a rod-adapting background (moving horizontally in Figure 4B); hence rod activity is required.

Figure 4:

Perceptual cancelation of rod and cone signals in mesopic vision (modified from Stockman and Sharpe (2006); see also MacLeod (1972)). A. The stimulus consists of a high-contrast flickering yellow spot superimposed on a large blue background. B. Detectability of the flicker in the yellow spot as a function of both spot (y-axis) and background (x-axis) intensity. Hashed regions show combinations of spot and background intensity for which flicker is invisible. C. Mean spike response in an On Parasol RGC to rod and cone flicker, presented individually and simultaneously.

MacLeod hypothesized that the disappearance of flicker at intermediate light levels was caused by a delay of rod signals relative to cone signals by one half cycle of the stimulus. Such a delay would cause rod and cone responses to the same flickering input to be precisely out of phase with each other, and hence to cancel when the two signals are combined. Cancelation is strongest at 7.5 Hz, requiring a ~65 ms delay of rod signals relative to cone signals. MacLeod tested this interpretation by replacing the yellow flicker with separate stimulation of rods (with a short wavelength stimulus) and cones (long wavelength stimulus) and introducing a delay in the cone stimulus relative to the rod stimulus. Consistent with a delay of rod signals leading to cancelation, the interaction between rod and cone stimuli changes from destructive to constructive when the cone signal is delayed by one half cycle. Rod signals in RGCs are indeed delayed relative to cone signals, and the two exhibit cancelation consistent with MacLeod’s hypothesis (Figure 4C).

Several other groups have repeated similar experiments using a range of experimental conditions (van den Berg and Spekreijse, 1977; Kilavik and Kremers, 2006; Sun et al., 2001). Some of these experiments find a shorter delay—20–30 ms—between rod and cone signals. Several aspects of the experiments differ – notably the mean and contrast of both the rod and cone stimuli. It is not clear why these specific changes in stimuli lead to different apparent delays. Quantitative analysis also reveals deviations from the predictions from models based on summation of phase shifted responses, as might be expected if several retinal pathways with different rod and cone weights contribute (van den Berg and Spekreijse, 1977). Similarly, while models based on a single fixed delay between rod and cone signals can account for signal cancelation across a range of temporal frequencies, those models fail at low frequencies (< 2Hz; (Sun et al., 2001; Kilavik and Kremers, 2006)). This suggests a shift in the location of rod-cone flicker interactions at low frequencies, with one suggestion being that rod and cone signals are predominantly carried by different RGC types with different inherent delays at low frequencies (Sun et al., 2001).

Rod-cone interactions are often quantitatively analyzed using models described as vector and probability summation. Vector summation refers to a model in which rod and cone signals are summed as vectors (e.g. a phase shift between rod and cone signals could cause vectors to point in different directions) and the combined signal is compared to a criterion level to determine perceptual threshold; MacLeod’s flicker cancelation hypothesis is an example. Probability summation refers to a model in which rod and cone signals each have a separate probability of exceeding a criterion for perception, and perceptual detection of the combined signal reflects the probability that one or both signals exceed the criterion. Mechanistically, vector summation suggests the combination of (potentially phase-shifted) rod and cone signals in a single pathway, while probability summation suggests that the signals remain distinct.

Different types of cone stimulation can lead to interactions better described as vector or probability summation. When long- and middle-wavelength cones experience similar temporal modulations, the combination of the resulting signals follows expectations for vector summation (Sun et al., 2001; Cao et al., 2010), except at high contrast where sublinear summation is apparent. Parasol ganglion cells exhibit similar behavior, and are the likely perceptual substrate for these stimuli (Cao et al., 2010). Chromatic cone stimulation (e.g. red-green flicker), however, generates signals that combine with rods in a manner more consistent with probability summation (Sun et al., 2001), suggesting that rod and cone signals are conveyed by different RGC types, likely midget and parasol RGCs, for such stimulation.

Kinetics and routing of rod signals

The flicker cancelation experiments considered collectively suggest that the kinetics of rod signals relative to cone signals may change substantially with experimental conditions, particularly mean light level. Indeed, direct measurement of the perceptual delay of rod signals relative to cone signals shows a speeding of 30–40 ms between low and high mesopic conditions (Figure 5A) (Sharpe et al., 1989). These kinetic differences, and their dependence on mean light level, figure prominently in the interpretation of many perceptual rod-cone interactions. Rod-mediated responses of primate parasol ganglion cells exhibit a change in kinetics similar to that observed perceptually (Figure 5B). Thus, retinal mechanisms appear to contribute strongly to this kinetic change.

Figure 5:

Change in kinetics of rod signals across the mesopic range. A. Perceptual delay in rod signals relative to cone signals in response to 8 Hz flicker across a range of mean light levels (modified from Sharpe and Stockman, 1999). This delay was measured as the phase advance applied to a rod stimuli required to create maximal cancelation between superimposed rod and cone stimuli. B. On parasol spike responses to an 8 Hz flickering spot (gray at top) across a range of mean light levels. 1 scotopic troland is approximately 4 R*/rod/sec.

What retinal mechanisms could account for such a change in kinetics? As detailed in the Routing section above (see Figure 2), a prominent hypothesis is that the dominant route that rod-derived signals take through the retina depends on mean light level (Sharpe and Stockman, 1999; Bloomfield and Dacheux, 2001; Deans et al., 2002). Thus, at low light levels, rod signals are hypothesized to traverse the retina largely through the primary rod pathway, while at higher light levels the secondary and tertiary pathways are proposed to convey much of the rod response. Further, the primary pathway is presumed to introduce considerably larger delays in rod signals than the (assumed) faster secondary and tertiary pathways. The shift in kinetics can then be ascribed to a shift from the slow primary pathway to the faster secondary and tertiary pathways (Figure 5A). Rod signals as measured perceptually can also exhibit self-cancelation (Sharpe et al., 1989; Sharpe and Stockman, 1999); this could be explained if under some conditions rod signals traverse the retina through several pathways with different kinetics, and these signals can cancel in a manner similar to rod-cone flicker cancelation.

The hypothesized change in routing of rod signals with light level is invoked to interpret a number of rod-cone interactions, but it has not been tested directly in primate. Several pieces of evidence (see Routing section) suggest that it may not be correct. Resolving how rod signals traverse the primate retina, and specifically whether this routing changes with light level, will be central to interpreting rod-cone interactions.

Paired flash interactions

Perceptual interactions between rod and cone signals have also been studied using pairs of flashes. These experiments generally fall into two categories.

Responses to simultaneous flashes, chosen to elicit similar responses in rods and cones, are consistent with sublinear summation of responses generated by the two photoreceptor types. These experiments usually start by measuring detection thresholds for separate rod and cone preferring flashes. Threshold for the combined flashes is then measured, with the ratio of the rod and cone preferring flashes fixed by their independent thresholds. These experiments reveal thresholds that are higher than expected for vector summation models based on a linear combination of rod- and cone-mediated signals, but lower than expected for probability summation models in which perceptual threshold is separately evaluated for rod and cone signals (Drum, 1982; Benimoff et al., 1982; Sun et al., 2001). Thus, these experiments suggest that rod- and cone-mediated signals are combined sublinearly.

A similar experimental design has also been used to measure interactions between supra-threshold rod and cone stimuli. In this case, brightness matching rather than detection threshold is used to evaluate interactions. Results from these experiments similarly support a sublinear combination of rod and cone signals (Benimoff et al., 1982).

Paired flash interactions have also been measured across a range of temporal offsets between rod- and cone-preferring flashes. In this case, the protocol is to deliver an initial rod or cone-preferring adapting flash, followed after a delay by a test flash probing signals from the other photoreceptor type. The adapting flash is typically a factor of 10 or more above threshold, and the adapting and test flashes generally overlap spatially.

The task is to either detect the test flash or to judge its apparent brightness, and this determination is compared for trials with and without the adapting flash.

Paired flash experiments allow comparison of the impact of rod signals on cone sensitivity (rod adapting flash and cone test flash) and vice versa. Test flash thresholds are elevated when the flashes are close together in time (within 200–300 ms), and this elevation occurs for rod-cone (rod adapting and cone test flashes) and cone-rod interactions (Frumkes et al., 1973). The dependence on temporal offset suggests a delay of rod signals relative to cone signals, consistent with the flicker experiments described above. One complication in these experiments is that with these short intervals, the responses to the two flashes are often perceptually fused so that they appear as a single flash. Thus the app,earance of the test flash, and likely the judgment the observer is making, varies as a function of temporal offset (Frumkes et al., 1973). Electrophysiological measurements of interactions between simultaneous or near-simultaneous rod and cone-preferring flashes generally show either linear summation or suppression, in broad agreement with the perceptual results (Gouras and Link, 1966; Enroth-Cugell et al., 1977).

With longer temporal offsets between the adapting and test flashes, the test flash elicits a reliably separate percept. Brightness matching experiments under these conditions (i.e. test flashes with and without a prior adapting flash are matched in apparent brightness) reveal an asymmetry in rod-cone vs cone-rod interactions: rod adapting flashes produce a stronger suppression of cone test flashes than vice-versa (Grimes et al., 2015).

Where do these asymmetric rod-cone interactions originate? Delivery of adapting and test flashes to different eyes reduced interactions, indicating at least a partial origin in monocular visual pathways (Grimes et al., 2015). Consistent with this observation, spike outputs of parasol RGCs show similar asymmetric interactions. Direct recordings from cells across the retinal circuitry show that interactions are absent in responses of bipolar cells but present in excitatory synaptic inputs to ganglion cells. This suggests an origin at the bipolar output synapse.

Likely synaptic mechanisms include synaptic depression due to depletion of available vesicles, which is prominent at rod bipolar synapses (Singer and Diamond, 2003; Oesch and Diamond, 2011; Jarsky et al., 2011), or synaptic rectification combined with dynamic changes in synaptic operating point (Figure 6A). Vesicle depletion makes three testable predictions about paired-flash interactions: (1) all presynaptic signals that elicit equal transmitter release will produce similar paired-flash interactions because they will invoke similar synaptic depression, (2) nonlinear interactions will be maximal for short time offsets and monotonically decay as the time offset increases, and (3) increasing flash strength will not overcome paired-flash interactions, as depression will affect all responses traversing the synapse. None of these predictions hold. First, rod adapting flashes produce considerably more suppression than cone adapting flashes, even when the synaptic output in response to the two adapting flashes is similar (Figure 7E). Second, suppression is weak for near-simultaneous flashes, and peaks 200–300 ms after the adapting flash (Figure 6B). Third, suppression can be overcome by increasing the strength of the test flash (Figure 6C).

Figure 6:

Paired flash interactions are more consistent with a change in operating point than with synaptic depression. A. Schematic of how vesicle depletion (middle) and changes in operating point (right) are expected to change synaptic input-output relation. Top depicts complement of vesicles in a cone bipolar synaptic terminal, and release of vesicles in response to a depolarizing stimulus. Bottom shows relation between release and presynaptic voltage. Vesicle depletion compresses the entire input-output curve (middle) and hence changes gain. Changes in operating point do not alter the input-output curve (right) but change gain by shifting the synapse to a location with a different local slope. B. Time course of rod-cone interactions measured from rod adapting flashes (blue arrow) and cone test flashes (red arrows). Bottom shows index of suppression across a range of delays between rod and cone flashes (adapted from (Grimes et al., 2015)). The suppression index is a normalized measure of the ratio of the responses to the second of the paired flashes with and without the first flash. C. Increasing flash strength overcomes impact of rod adapting flash on response to cone test flash. Bottom shows index of suppression as a function of cone test flash strength.

Figure 7:

Modeling the retinal integration and processing of rod and cone signals with a linear-nonlinear cascade model. A. Linear and nonlinear model components are derived from recordings of RGC responses to independent rod- and cone-preferring noise (0–40 Hz). B. Linear filters (top) and nonlinearities (bottom) derived from rod (blue)- and cone (red)-preferring noise stimuli. C. Model for predicting excitatory synaptic input to On parasol RGCs for arbitrary rod and cone stimuli. The final step of the model uses a common nonlinearity that reflects the average of the rod and cone derived nonlinearities. D. Model output in response to paired rod and cone flashes with a 0.2 s time offset. Gray lines are the linear sum of responses to separate rod and cone flashes, black lines are responses to the paired rod and cones flashes. E. Paired flash responses recorded from the same On parasol RGC used for noise experiments (A-D). Gray and black lines are the same as in D. Adapted from (Grimes et al., 2015).

These observations point to a working model, formalized in the next section, in which dynamic changes in operating point and synaptic rectification effectively alter synaptic gain and hence sensitivity to modest amplitude test stimuli.

Model for retinal rod-cone interactions

A full understanding of mesopic vision would entail developing comprehensive models that accurately predict retinal outputs to arbitrary stimuli and relate the mathematical transformation of visual signals to the biophysical operation of real retinal mechanisms (e.g. synapses). Such models, once thoroughly tested, could permit real visual inputs to be transformed into neural signals, a prerequisite for most prosthetic approaches to visual restoration. In this section we describe recent progress towards one such model.

Neural circuits are composed of many linear and nonlinear elements (e.g. active membrane conductances and synapses) that shape signals before transmission to downstream targets. Fully characterizing these elements is a huge undertaking. Empirical models that package together linear and nonlinear components provide an alternative (Chichilnisky, 2001). Such models are almost certainly oversimplified, but also can capture key signaling properties.

Linear-nonlinear cascade models estimate the transformation of stimuli into neural responses via a 1) single linear filter, which shapes the time course of neural signals, and 2) a static nonlinearity, which can rectify the model’s output. Figure 7 shows how such a model can be used to probe the basis of rod-cone paired flash interactions (Grimes et al., 2015).

Linear and nonlinear model components were estimated from responses of On parasol RGCs to Gaussian noise stimuli that preferentially activated either rods (blue) or cones (red) (Figure 7A). The resulting linear filters differ in kinetics: the cone filter has a shorter time-to-peak and is less biphasic than the rod filter (Figure 7B, top). Nonlinearities for both stimuli exhibited a similar rectified shape (Figure 7B, bottom). These model components are consistent with a framework in which rod and cone signals are shaped distinctly before passing through a shared nonlinearity, likely located at the cone bipolar output synapse.

These empirically-derived components can be used to predict On parasol responses to stimuli that activate both rods and cones. Such predictions are formed in three steps (Figure 7C): 1) independent linear filtering of rod and cone stimuli through their respective linear filters, 2) linear summation of the filtered signals, and 3) nonlinear transformation of the combined signal. Figure 7D shows the model’s predictions for RGC responses to paired rod-cone flash experiments; measured responses to the same stimuli from the same cell that the model components were derived from are shown in Figure 7E. This model shows that a combination of distinct kinetics of rod- and cone-mediated responses and a shared nonlinearity can account for asymmetric rod-cone interactions.

Concluding remarks

As described above, interactions between signals generated in the rod and cone photoreceptors strongly impact how we see across a broad range of light levels. These rod-cone interactions have been well studied in controlled perceptual experiments; in a few cases, directly comparable physiological experiments permit comparison of retinal mechanisms with perception. This background provides a glimpse of the opportunities and challenges that lie ahead.

From a mechanistic perspective, rod-cone interactions in the retina provide an excellent opportunity to understand how parallel processing impacts perceptually-relevant circuit computation. In part, that is because we can build on decades of beautiful anatomical work about cell types and connectivity. In part, it is because we can study parallel processing in the context of physiological stimuli. And in part it is because of the clear and well-defined role of the retina in perception. A challenge in this regard involves designing perceptual experiments that emphasize retinal mechanisms over cortical mechanisms and understanding which RGC types provide the substrate for perception of specific stimuli.

From a functional perspective, an immediate question about rod-cone interactions is whether they are a bug or a critical feature of how vision works. The convergence of rod and cone signals in retinal circuits may be driven by a need to minimize the number of axons in the optic nerve and hence permit rapid eye movements. Such convergence will inevitably create interactions between rod and cone signals. But mesopic vision may also be enhanced by rod-cone interactions. Under mesopic conditions, rods and cones are at very different points in their operating range, and sensitivity may be improved by combining the relatively slow but sensitive rod signals with faster but less sensitive cone signals. Rod-cone interactions have been studied almost exclusively using controlled artificial stimuli, therefore an understanding of retinal function will require extending these studies to include natural inputs.

Summary points:

• Studies of rod-cone interactions provide a rare opportunity to understand how parallel processing affects perceptually-relevant circuit function.

• The mechanisms mediating interactions between rod and cone signals are poorly understood, yet these interactions strongly impact vision across a broad range of light levels.

• Interactions between rod and cone signals can break the normal seamlessness of vision and in doing so provide guidance in investigating the underlying mechanisms.

• Some types of rod-cone interaction can be largely accounted for by known retinal mechanisms

Future Issues:

• Are rod-cone interactions an unavoidable consequence of combining rod and cone signals in the same retinal circuits, or do they enhance visual sensitivity?

• What are the retinal and cortical contributions to perceptual rod-cone interactions?

• What is the relationship between known aspects of rod-cone interactions and deficits in mesopic vision?