Visual computation in the cephalopod brain

Abstract

Computational models of visual system function are largely based on the mammalian visual hierarchy, as the exemplar of a large complex visual system. However, the cephalopod visual system provides an intriguing alternative model for vision; it is relatively similar in size, complexity, and acuity to that of mammals, but has a fundamentally different neural architecture. While this system has been largely overlooked relative to more standard model species, renewed interest in the diversity of neural computations across species, along with technical developments allowing for further investigation, have led to insights into this evolutionarily distinct visual system. Here we provide a brief overview of the unique architecture of the cephalopod visual system, and review recent advances in understanding its neural circuitry, visual coding, and potential computational mechanisms. Throughout, we highlight how investigating cephalopod visual processing provides the opportunity to identify both shared and novel principles for visual computations and their neural implementation.

Introduction

Vision is a critical sense that many animals throughout the animal kingdom rely on to survive in their local environment. To use visual input, the brain must extract useful information from the pattern of light gathered by the eyes. Decades of research has established various computations that transform the activation of photoreceptors into neural representations that support perception, cognition, and behavior. This has elucidated a number of computational mechanisms including topographic organization, parallel pathways, lateral inhibition, hierarchical feature selectivity, and excitatory-inhibitory balance [1–4], as well as potential high-level principles including efficient coding, predictive processing, and Bayesian inference [5–7]. This work has also inspired artificial network models such as deep convolutional neural networks [8]. However, most of these findings are based on a small subset of species, particularly mammals, limiting our understanding of how neural circuitry can solve visual scene analysis.

This raises a thought experiment: if one were to discover a completely independent instantiation of a large, high resolution visual system, which aspects would be the same as the mammalian visual system and/or current neural networks? Which would be different? From a computational perspective, what would we learn by exploring the diversity of algorithms and architecture of this new visual system? Fortuitously, evolution has already created such a visual system in cephalopods.

Coleoid cephalopods, which include squids, cuttlefishes, and octopuses, are a compelling model to investigate visual processing [9–11]. Their large and complex brains evolved completely independently from those of vertebrates, with our last common ancestor having only a simple nervous system roughly 600 million years ago [12]. In size, the cephalopod brain is orders of magnitude larger than any other invertebrate, with the size of an adult octopus brain being similar to that of many mammals, including mice [13]. The resolution of their vision is equally impressive, with visual acuity in some species similar to that of a cat at ~5 cyc/deg [14]. This independent evolution has thus resulted in a visual system that rivals many mammals, but has a radically different neural organization with no direct homology corresponding to thalamus, cortex, or other vertebrate brain regions.

In addition, their unique visual system facilitates elaborate behaviors. These include behaviors that are common throughout the animal kingdom, such as prey capture, navigation, and identifying conspecifics [15], as well as others that are relatively unique to cephalopods. Perhaps most notable is their remarkable ability to camouflage based on their environment (Figure 1a), [16,17], which requires vision and is influenced by visual cues [18,19]. These behavioral capabilities raise intriguing questions regarding the underlying visual processing, such as: Do similar circuits and computations underlie their shared behaviors, despite the independent evolution and different embodiment? How is a novel capability, such as rapid camouflage, implemented through transformations of visual input? Answering such questions has the potential to provide fundamental insights into the diversity of visual system designs and computation.

Figure 1. Cephalopod visual system organization and neural circuitry.

a) Cephalopods camouflage, as illustrated by an octopus emerging from a hidden position. From Ref. [16]. b) Image of the eye of Octopus rubescens, courtesy of Bella Zedona. c) Organization of the octopus eye, optic lobe, and brain, based on DAPI staining of all cell nuclei (blue) along with cell type markers. Courtesy of Denise Piscopo. d) Organization of the octopus optic lobe layers, based on DAPI (green) for cell nuclei and phalloidin (magenta) for neuropil. Courtesy of Abbigale Koenigsmark. e) Schematic of optic lobe circuitry based on cellular anatomy, delineating the primary morphological cell types and their processes. Based on [36–38]. f) Localization of the four main neurotransmitter-defined cell types found in the optic lobe. Based on [39–41].

Despite this potential, much less is known about visual processing in cephalopods relative to other model species, in part due to the lack of genetic tools and standard recording techniques. Indeed, to date there have only been two dedicated studies of neural coding in the central visual system of cephalopods [21,22]. However, recent studies have provided new insight into their neural circuitry and visual processing. Here we review these findings relative to well-studied computational mechanisms in standard model systems, with the caveat that much is still unknown, especially in terms of inferring high-level theoretical principles (Table 1).

Table 1.

Selected computational mechanisms and principles identified in mammalian visual systems, some of which have been found in cephalopods, many of which have not been tested.

Computational mechanism/principle	Present in cephalopods?
Retinotopic organization	Yes [21,22]
Orientation/direction selectivity	Yes [22]
Hierarchical processing	Yes [21,22]
Lateral inhibition	Yes [31,46]
Parallel pathways	Yes [21,22]
Stereopsis	Yes [51]
Size selectivity	Yes, with ON/OFF asymmetry [21]
Excitatory-inhibitory balance	Unknown
Invariant tuning	Unknown
Gain control/normalization	Unknown
Efficient coding	Unknown
Predictive coding	Unknown
Bayesian inference	Unknown

Overview of cephalopod visual system organization

Although octopuses, squid, and cuttlefish represent a vast diversity within cephalopods, having diverged from each other up to 200 million years ago, they share a similar overall visual system organization [23]. Notably, the details of this general organization can differ across species, correlated with habitat, suggesting adaptations to different scene statistics and behavioral demands [24].

Cephalopods have camera-type eyes (Figure 1b) that are remarkably similar to vertebrates’, despite evolving independently [25]. This includes a pupil that restricts light, and a cornea and lens that together focus light onto the retina [14]. However, beyond these optical elements, the cephalopod and vertebrate visual systems are structured fundamentally differently [26]. The only neurons in the cephalopod retina are photoreceptors that are oriented towards the incoming light. Most cephalopods express only one type of opsin in their photoreceptors, rendering them color-blind, at least at the retinal level [27]. However, the arrangement of their rhabdomeric photoreceptors enable the detection of the polarization of light [28], which mammals cannot generally see. The photoreceptors project directly into the brain via optic nerve bundles that enter the visual processing centers, termed the optic lobes (Figure 1c), without any elaborate intermediate circuitry as seen in the vertebrate retina [29]. Interestingly, the optic lobes also send major projections back to the retina itself [30,31].

The two optic lobes make up nearly 2/3rd of the cephalopod central brain and project to many other brain regions [23] including the vertical lobe complex, which facilitates learning and memory [32,33]; the peduncle lobe, a visuomotor center [32,34]; and the basal lobe complex, which mediates motor control [35,36]. Since the optic lobes receive direct photoreceptor input and project directly to these cognitive and motor centers, most of the visual processing is thought to happen within the optic lobe, thereby corresponding to the entire visual processing hierarchy found in mammals.

Neural circuit architecture

The cephalopod optic lobe is made up of four distinct layers that together form a neural architecture unlike any other brain structure in the animal kingdom (Figure 1d,e). The most superficial layer, the outer granular layer (OGL), is made up of neurons that morphologically resemble amacrine cells of the vertebrate retina [35]. Below the OGL lies the plexiform layer, a neuropil layer that includes the terminals of photoreceptor axons. The third layer is the inner granular layer (IGL), a cell body layer with a wide diversity of anatomical cell types, including neurons reminiscent of bipolar cells in the vertebrate retina [36,37]. These first three layers are often described as the ‘deep retina’ due to their anatomical similarities to the vertebrate retina [37], although there is no shared evolutionary origin. The core and majority of the optic lobe is termed the medulla, which provides the major output from the optic lobe to other brain regions. It is made up of neurons with diverse morphologies, with cell bodies that are clustered in a curious tree-like branching fashion surrounded by neuropil. Generally, the optic lobe gives the appearance of a fusion of a layered, retina-like outer structure (OGL and IGL) with a matrix of intermingled cell body clusters and neuropil in the interior (medulla).

The intricate architecture of the optic lobe (Figure 1e) raises both hypotheses and questions regarding the circuit basis of the computations it may perform. For example, the horizontal morphology of OGL amacrine cells is well suited for lateral processing of photoreceptor input and distribution to downstream IGL neurons, though this has not been directly tested. Strikingly, early anatomical studies demonstrated a tiered organization of these amacrine cells with different arbor sizes [29], and a recent transcriptomics study showed that these tiers express different neuropeptides [39], suggesting distinct neuropeptide channels may be responsible for processing different spatial scales [22]. The IGL contains neurons with diverse morphologies, including bipolar cells, and it is unknown if these different cell types extract distinct features from the visual scene. In contrast to the superficial similarity of the OGL/IGL to the vertebrate retina, the striking architecture of the medulla appears unique. At a global scale, the branched, tree-like organization of the cell bodies (Figure 1d) suggests a substrate for performing hierarchical processing, integrating across increasingly larger regions of visual space consistent with increased receptive field size and tuning properties described below [22].

In contrast to the detailed morphological characterization of optic lobe neurons, very little was known about their molecular or functional identity, even at the level of neurotransmitter usage, until recently. Single cell RNA sequencing studies in octopus and squid [39–41] revealed three main classes of neurons in the optic lobes based on neurotransmitter usage - dopamine, glutamate, or acetylcholine - and a fourth class that coexpresses dopamine and glutamate. In addition, smaller subsets of neurons were defined by their expression of neuropeptides, as well as a distinct population that expresses octopamine. These neurotransmitter-defined cell types also occupy distinct locations in the optic lobe (Figure 1f). For example, the amacrine cells in the OGL are largely dopaminergic, while the medulla consists of intermingled glutamatergic and cholinergic neurons [39–41]. Strikingly, there are almost no GABAergic neurons in the optic lobe, although they are present in other brain regions [39–41]. However, acetylcholine has recently been shown to be inhibitory in the optic lobe [42], and therefore the glutamatergic and cholinergic neurons may be serving the role of excitatory-inhibitory balance [43] in the medulla.

This knowledge of neurotransmitter usage provided initial insight into computational circuit motifs in the optic lobe in a recent study that investigated local circuitry by combining calcium imaging with pharmacology in optic lobe slices [42]. This identified a potential circuit based on novel ionotropic excitation by dopamine from OGL amacrine neurons onto IGL, and reciprocal ionotropic inhibition by acetylcholine from IGL back onto the OGL. Together, this reciprocal excitation-inhibition motif was hypothesized to implement lateral inhibition in the optic lobe, similar to amacrine and bipolar cells in the vertebrate retina.

Functional organization and neural coding

Understanding of the functional properties of the cephalopod visual system has lagged behind that of anatomical understanding due to limited recording techniques. However, available recordings demonstrate that some computational principles are shared with other visual systems, though often via distinct neural implementation, while others show clear cephalopod specialization.

In the isolated retina, electrophysiological recordings demonstrated that the photoreceptors depolarize to light onset [44], consistent with invertebrate phototransduction. They have predominantly ON-center receptive fields [45], and display lateral inhibition [31,46], a standard computational motif in both vertebrate and invertebrate visual systems that serves to enhance contrast and suppress redundant signals. The mechanism for this lateral inhibition in cephalopods is not immediately apparent, given that there are no other neurons in their retina to mediate it, but evidence suggests that it might result from the centrifugal projection of optic lobe neurons back into the retina [45]. Interestingly, while photoreceptors are predominantly ON, they can also acquire OFF responses depending on stimulus history, such as baseline luminance [45]. While this is not typical in the mammalian retina, it is similar to a recent finding in the zebrafish retina showing that the degree of ON versus OFF responses is under inhibitory control [47].

In cephalopods, the photoreceptors provide glutamatergic excitatory input into the optic lobe [40,48]. This excitatory input has also been shown to be gated by a pre-synaptic cholinergic inhibitory shunt, which has been proposed to serve as a temporal filter that reduces the response to static stimuli [48]. This temporal filtering via pre-synaptic shunt could thereby represent a novel mechanism for implementing segmentation of moving objects relative to the background, acting right as signals enter the optic lobe.

Within the optic lobe, calcium imaging in the octopus has provided initial insight into the functional organization of visual response properties [21]. Reverse correlation with a sparse noise stimulus demonstrated the existence of spatially localized ON (light) and OFF (dark) population receptive fields. ON responses were initiated in the plexiform layer, consistent with the termination of photoreceptor axons, and propagated throughout the deeper layers. Computationally, these ON signals need to go through a sign inversion to give rise to OFF responses, which are observed in the IGL and span throughout the medulla. This is reminiscent of the sign inversion that occurs at the vertebrate photoreceptor to bipolar cell synapse resulting in parallel ON and OFF pathways, although the mechanism driving this inversion in cephalopods is unknown. Notably, spatially localized receptive fields were retinotopically organized, demonstrating that there is a map of visual space across the optic lobe. This is a prevalent organization found throughout the animal kingdom, suggesting this may be a fundamental characteristic of complex visual systems, though notable exceptions exist [49].

While these findings demonstrated shared principles of ON and OFF pathways and retinotopic organization, they also revealed differences in terms of spatial integration [21]. Responses to ON stimuli demonstrated size selectivity, with the response decreasing to stimuli of larger size, while responses to OFF stimuli continued to increase for spots of increasing size. This illustrates distinct differences in the mechanisms that process light and dark, potentially implementing efficient coding based on the statistics of the underwater scene, similar to how ON/OFF asymmetries in mammals reflect the terrestrial scene [50].

A recent study in head-fixed squid provided a more in-depth understanding of neural coding through the circuitry of the optic lobe [22]. Calcium imaging in the superficial layers (OGL/IGL) revealed that individual neurons responded to distinct temporal patterns of increments and decrements of full-field illumination, allowing delineation of functional cell types. Intriguingly, response properties were organized into distinct sub-laminae, and receptive field sizes increased from OGL to IGL, suggesting integration of information. Furthermore, a small fraction of neurons in the OGL and IGL demonstrated complex response properties, including orientation and direction selectivity. Neuropixel recordings through the depth of the medulla demonstrated that both the receptive field size and the fraction of orientation/direction selective cells continued to increase. This progression is consistent with hierarchical processing through the optic lobe [22], which may recapitulate the hierarchical processing observed through the mammalian retino-geniculo-cortical pathway. Additionally, this study examined the integration of polarization information, which we discuss more fully below.

Toward computational principles underlying behavior

These initial investigations of visual system circuitry and function open up many intriguing areas for further inquiry. Here we highlight key computational questions surrounding the cephalopod visual system and its roles in behavior, along with emerging results on potential algorithms and neural mechanisms that drive these roles.

A key computational challenge in interacting with the visual environment is estimating distance to objects. In animals with forward-facing eyes, a primary mechanism is stereopsis, based on comparing retinal images between the two eyes. To test for stereopsis in cephalopods, a recent study in cuttlefish used colored glasses placed over the two eyes to present images of virtual prey with different disparity between the two eyes [51]. Strikingly, the animals targeted their attacks at the virtual location defined by disparity, demonstrating the role of stereopsis. However, in contrast to mammalian stereopsis in the central visual field, which results from alignment of pixels with the same luminance, cuttlefish could accurately strike when the images in the two eyes had opposite luminance values as well. This implies that the general computation of stereopsis evolved in cephalopods, but works equally on correlation and anti-correlation of luminance between the two eyes. Notably, a recent study demonstrated that humans can detect anti-correlation in the peripheral visual field, but this results in a perception of reversed depth [52]. Furthermore, another invertebrate species, the praying mantis, uses yet another algorithm for stereopsis, based on the correspondence of dynamic changes rather than static correlation [53].

In contrast, stereopsis likely does not apply to octopuses, which do not have significant binocular overlap and therefore likely depend on monocular cues. Intriguingly, it has been reported that octopuses bob their head vertically before striking prey [54], similar to mammals estimating distance across a gap [55,56], which can generate cues for computing distance based on motion parallax. As yet another possible mechanism, in squid this head bobbing behavior is complemented by a ‘retinal bump’ that creates a discontinuity in focus matched to the typical prey strike distance; scanning this bump across a target thereby provides a measure of whether the animal is at the appropriate distance [57].

Although these behavioral studies suggest underlying algorithms for distance estimation, there are not yet neural recordings to determine how this information is actually encoded and processed in the brain.

The underwater world poses unique visual challenges, including turbidity and decreased color information. One way cephalopods overcome these challenges is by utilizing polarized light information. While cephalopods are known behaviorally to detect light polarization and use it to analyze form and recognize objects [29,58,59], the mechanisms of polarization processing are not as well established. One possibility is that polarization information is processed in a similar way to how other visual systems process color, with channels dedicated to specific polarizations that are combined through an opponent process to reconstruct a polarization-based image of the entire visual scene. On the other hand, rather than full reconstruction, the visual system may only extract specific trigger features that are defined by differences in polarization, such as the presence of prey. Support for the latter model comes from recent recordings from the squid optic lobe, which demonstrated that while neurons in the OGL and IGL are tuned to the angle of polarization, in the medulla this information is integrated to create responses to decrements in the degree of polarization, and is combined additively with luminance information [22]. This is consistent with a role in increasing the effective contrast of nearby underwater targets, further supported by field studies of natural-scene polarization content [22].

Perhaps the most remarkable visually driven behavior possessed by cephalopods is their ability to camouflage. As a cephalopod camouflages to match its visual background, it is clear that they do not simply replicate the visual scene on their skin pixel-for-pixel, but rather survey the visual environment to match visual statistics of the scene [17,20,60]. This process is mediated by a pathway from the optic lobe to the lateral basal lobes, a premotor region, which then projects to the chromatophore lobes that directly control chromatophores in the skin [61]. While the underlying neural circuitry that transforms visual input into a camouflaged skin patterning is unknown, recent behavioral studies have begun to reveal potential algorithms.

Traditionally, studies classified camouflage body patterns based on a set of distinct basis patterns that could be rendered in combinations to create the full range of patterns observed. This approach demonstrated that some visual features evoke a small number of these basis patterns, suggesting that they are visually triggered in a roughly one-to-one manner [62]. This implies that the visual system must extract these different features separately before routing them to different motor outputs. Recent high-resolution analysis of camouflage at the chromatophore level provided further insight, revealing that the patterning at this microscopic level had high dimensionality, with different chromatophore-level patterns resulting in macroscopic patterns that appeared similar and correlated with the animal’s visual background [63]. This suggests that the visual system may also be extracting a corresponding high dimensional basis set, supporting texture metamers similar to those described in the primate visual system [64]. The same study revealed that the camouflage process was highly dynamic, moving through multiple steps that iteratively approach the final pattern, suggesting camouflage is an active search process involving feedback and potentially visual input [63].

The observations that camouflage patterns represent the underlying structure of the visual scene, rather than directly recreating the scene, have motivated the hypothesis that the camouflage pathway acts as an autoencoder. In this model, the optic lobe represents high-dimensional visual information, and the lateral basal lobe reduces this to a low-dimensional representation that can then be decoded and expanded back into the high-dimensional space of the chromatophores [20,60]. Directly testing such models will require neural recordings during camouflage behavior in order to determine how visual information is transformed along the pathway to motor output. To complement this, a recent field study generated a database of natural underwater scenes, along with behavioral video of cuttlefish camouflaging within these scenes, which can provide a foundation for quantitative analysis of visual scene statistics and corresponding camouflage strategies [65].

Conclusion

As described here, recent technical advances have provided new glimpses into the cephalopod visual system, revealing that despite independent evolution and divergent anatomical organization, many low-level computational mechanisms observed in vertebrates and other invertebrates are conserved in cephalopods (Table 1). These include retinotopic organization, lateral inhibition, and orientation and direction selectivity, suggesting that such mechanisms are key components of a large and complex visual system. In some cases, while these properties are observed, they have unique variations, such as the asymmetry in ON/OFF size selectivity, and the ability to use both correlation and anti-correlation equally in stereopsis.

Nevertheless, much remains unknown about this visual system. In particular, so little is understood about neural coding that we cannot yet begin to deduce high-level computational principles, such as efficient coding and predictive processing that provide a theoretical framework for vertebrate vision. Furthermore, the study of cephalopod vision remains largely at the level of the first few synapses, examined with a limited repertoire of visual stimuli. Expanding investigations deeper into the brain, with richer stimuli and in the context of naturalistic behavior, will illuminate the full extent of similarities and differences, with the potential to reveal computational novelties unique to cephalopod vision.

Data availability

No data was used for the research described in the article.