Escaping the nocturnal bottleneck, and the evolution of the dorsal and ventral streams of visual processing in primates

Abstract

Early mammals were small and nocturnal. Their visual systems had regressed and they had poor vision. After the extinction of the dinosaurs 66 mya, some but not all escaped the ‘nocturnal bottleneck’ by recovering high-acuity vision. By contrast, early primates escaped the bottleneck within the age of dinosaurs by having large forward-facing eyes and acute vision while remaining nocturnal. We propose that these primates differed from other mammals by changing the balance between two sources of visual information to cortex. Thus, cortical processing became less dependent on a relay of information from the superior colliculus (SC) to temporal cortex and more dependent on information distributed from primary visual cortex (V1). In addition, the two major classes of visual information from the retina became highly segregated into magnocellular (M cell) projections from V1 to the primate-specific temporal visual area (MT), and parvocellular-dominated projections to the dorsolateral visual area (DL or V4). The greatly expanded P cell inputs from V1 informed the ventral stream of cortical processing involving temporal and frontal cortex. The M cell pathways from V1 and the SC informed the dorsal stream of cortical processing involving MT, surrounding temporal cortex, and parietal–frontal sensorimotor domains.

This article is part of the theme issue ‘Systems neuroscience through the lens of evolutionary theory’.

1. Introduction

In 1942, Gordon Walls [1] published his book, ‘The vertebrate eye and its adaptive radiation.’ Walls stressed the ‘simplicity’ of the eyes of most mammals in comparison to those of reptiles and birds, noting the loss of features of eyes for high-acuity colour vision as early mammals adapted to a long period of nocturnal life. After millions of years of nocturnal life, some diurnal mammals reinvented new cones and other features of eyes that serve diurnal vision and mediate colour vision with high acuity, but most did not. Thus, Walls concluded that many mammals never escaped from their ancestral ‘bottleneck’ or ‘knot-hole’ to recover good daytime vision. Overall, the nocturnal bottleneck theory for the persistence of regressive features of the retina and eye in general, reflecting the reduced importance of vision in early mammals, has been supported by further studies [2–6]. Judging from present-day mammals, and the fossil record, early mammals had small eyes with small optic nerves. Cone receptors were reduced from 4 types to 2, and coloured oil droplets over cones were lost. The retinal image was poorly magnified, and there was no fovea. Photo-pigments were lost in the pineal eye and hypothalamus.

The mammaliaform ancestors of early mammals emerged after a volcano-triggered mass extinction event some 265 million years ago [3,7] (figure 1). Those leading to mammals were small and likely nocturnal to avoid the emerging diurnal dinosaurs. Early mammals remained small and nocturnal to continue to avoid predatory dinosaurs [8] until the mass destruction event from an asteroid impact some 66 million years ago resulted in the extinction of the non-avian dinosaurs and much of other life [9]. Early mammals adapted to dim light by relying less on vision, greatly expanding the olfactory system [10,11], developing high-frequency hearing [12–14] and using long tactile whiskers and body hairs for near body touch [15]. The proportion of the forebrain used for sensory and motor functions became larger, suggesting further processing of this sensory information [16]. Having fur and metabolic thermoregulation were adaptations to the colder night-time temperatures [17–20].

Figure 1.

Mammals evolved from mammal-like therapsids around 230 million years ago between the major Permian–Triassic (P/Tr) extinction 265 million years ago and the major Cretaceous–Paleogene (K/Pg) extinction 66 million years ago. Early mammals were nocturnal to escape predation by diurnal dinosaurs and largely remained nocturnal until the extinction of dinosaurs in the K/Pg. The eyes and vision of nocturnal mammals greatly regressed during the period of forced nocturnality and remained regressed for many mammals after the extinction of dinosaurs. As a clear exception, ancestral primates emerged with large eyes and frontal vision while remaining nocturnal before the K/Pg extinction. Most primate lines became diurnal after the K/Pg extinction. Nearly all dinosaurs and early birds were diurnal. This figure is a simplified version of figure 1 of Gerkema et al. [3]. (Online version in colour.)

Surprisingly, most mammals did not adapt their eyes and visual systems to see better in dim light. Both nocturnal birds and nocturnal primates have large, forward-facing eyes with large corneas that collect more light and project a large image on the retina [21]. In some nocturnal mammals, a reflecting tapetum behind the retina gets more out of the available light. Such modifications indicate that it is possible to escape the nocturnal bottleneck even while remaining nocturnal. And that is exactly what early primates did as they emerged 80–90 mya shortly before the extinction of the dinosaurs [22]. The obvious question is how did ancestral primates escape from the bottleneck while remaining nocturnal, and return to vision as an important source of sensory information? In addition, how was the central visual system modified to process the newly important visual information? And what further changes occurred in visual systems in those primates that became diurnal, and in diurnal primates that then adapted again to nocturnal life (owl monkeys and tarsiers)? As all extant primates depend highly on cortical processing of visual information in two functionally distinct dorsal and ventral streams of cortical areas and modules [23,24], it is useful to first consider the beginnings of the dorsal and ventral streams as reflected in non-primate vertebrates.

2. Two visual pathways in vertebrates

Fish, amphibians, reptiles, birds and mammals all have been long recognized as having two major visual pathways. One is from the retina to the optic tectum (the superior colliculus (SC) of mammals), and then to the dorsal thalamus (nucleus rotundus of reptiles and birds and the posterior pulvinar of mammals) and then to the pallium (the striatum, dorsal ventricular ridge or temporal cortex). The other pathway is from the retina to the dorsal thalamus, the dorsal lateral geniculate nucleus (LGN) and then to the pallium (the dorsal cortex, Wulst or primary visual cortex). While the terms for structures vary for historical reasons and histological differences, the basic components of pathways from retina to forebrain appear to have been established early in vertebrate evolution [25] and variously expanded and differentiated in especially mammals and birds. The dorsal cortex of reptiles has been widely recognized as homologous to neocortex of mammals (e.g. [11,26–29]) and part of the striatum or dorsal ventricular ridge may be homologous to lateral neocortex of mammals (e.g. [30]).

For fish, amphibians, reptiles and birds, the general consensus has been that the optic tectum is the main visual centre, and with its connections, the tectum is responsible for most visual behaviour. In fish, the optic tectum was seen as critical, in part because it is ‘larger than the telencephalon or cerebellum in most teleosts’ [31]. However, even in jawless fish (lampreys), retinal projections reach the thalamus and are relayed to the lateral pallium or cortex [25]. Ingle [32] suggested that fish have two different processing systems, one for orienting toward moving objects and one for identifying the object. While tectal ablations in fish appear to produce permanent blindness, some abilities such as discrimination of white from black remain in sharks [33]. Frogs with a unilateral lesion of the optic tectum ‘totally ignored prey moving within the monocular field contralateral to the damaged side’, and failed to jump away from looming dark objects [34]. However, not all of the visual behaviour was lost, and such frogs could avoid a stationary barrier using the deprived eye. In a widely recognized report, Lettvin et al. [35] concluded that different classes of retinal ganglion cells projecting to the optic tectum were already specialized in ways to help mediate the basic visual behaviours. Thus, one class of cells with small receptive fields served as bug detectors, while another class, receptive to larger looming objects, may signal predators. In frogs, the other main target of the retina is the LGN of the dorsal thalamus [36]. Muntz [37] found that the projections of the retina to the LGN of frogs had ‘completely different properties than the tectal projection’ as the projection to the LGN was from on-fibres with small receptive fields that were highly sensitive to blue light, while receiving none of the axons from the four types of detectors projecting to the tectum. The blue-sensitive retinal projections to the LGN were thought to help frogs jump toward water when startled [38].

The two visual pathways in reptiles have been perhaps most studied in turtles. In turtles, removal of the dorsal cortex, the target of the LGN [39], appears to impact only on visual habituation. After lesions of the visual dorsal cortex, turtles seemed to have normal vision but failed to habituate to a repeated but harmless visual threat [40,41]. By contrast, after lesions of the optic tectum, turtles were unable to learn a pattern discrimination, and acted as if blind [42]. Birds, as flying dinosaurs, are similar to reptiles in that most retinal projections terminate in a large, highly laminated optic tectum, which sends a massive projection to nucleus rotundus of the dorsal thalamus [43]. Rotundus in turn projects to a lateral part of the pallium [44], a part of the dorsal ventricular ridge, which appears to be homologous or at least analogous to the temporal cortex of mammals [28,30]. Alternatively, connections from the tectum to the thalamus, and then to the dorsal ventricular ridge [45], may have extended to innervate the nearby dorsolateral cortex in the ancestors of mammals. In pigeons, large lesions of the tectum or rotundus produced severe impairments on all visual pattern discrimination tasks [46], while lesions to the geniculate to Wulst system do not produce such impairments [47].

Results from mammals follow and extend this vertebrate pattern [48]. All mammals, except possibly a few nearly blind species with degenerate visual systems, have two visual pathways to the cortex. The first pathway is from retina to the LGN of the dorsal thalamus, which then projects to primary visual cortex, V1 or area 17. A second major pathway is from the retina to the SC, which projects to a posterior nucleus of the pulvinar complex, which projects to temporal neocortex [49]. This variously named posterior nucleus is now easily recognized by its dense expression of the vesicular glutamate transporter (VGLUT2) as a result of VGLUT2 mRNA expression in neurons of the sublayer of the SC, which projects to the posterior pulvinar [49]. These two pathways are the ‘two visual systems' of mammals that have been long recognized (e.g. [50]). Early studies in non-primate mammals found that lesions of the SC have a much greater impact on visual behaviour than lesions of the LGN or primary visual cortex (V1). In part, this may partly reflect the problem of SC lesions typically damaging the descending motor outputs of the deeper layer of the tectum, as well as the ascending projections to the posterior pulvinar. Nevertheless, lesions of the SC produce mammals that appear to be blind and have few remaining visual abilities [48]. By contrast, lesions of V1 result in highly reversible impairments in the discrimination of visual patterns and objects [50–55]. In reviewing such studies on tree shrews, which are closely related to primates, Ware et al. [56] concluded that complete lesions of V1 in tree shrews produced ‘little or no deficit in visual discrimination’. By contrast, V1 lesions in humans and other primates produce such major visual impairments that the few remaining abilities have been called ‘blindsight’ [57,58]. However, lesions of the SC in primates produce rather minor impairments [59]. Nevertheless, as pointed out by Goodale [60], results from such studies have differed such that ‘the literature abounds with contradictory results'. This may be in part because of experimental variables, but also because of the variable ways in which visual information is distributed and used in different species.

Overall, across most vertebrates, two major visual pathways to the forebrain exist, one via the optic tectum or SC, to the thalamus and then to the pallium, and the other to the thalamus (LGN) and to primary visual cortex, dorsal cortex or pallium. The two pathways appear to be somewhat different in the information they provide to pallial processing, and they may interact after activating different pallial targets. The two pathways seem to have emerged in early vertebrates [25] and have been modified and extended in various ways over 500 million years or more of branching vertebrate evolution. But, an important change occurred with the emergence of early primates that appears to make primates unique. In brief, we propose that projections from primary visual cortex invaded part of the cortical territory of the posterior pulvinar projections and out-competed the SC to pulvinar to cortex inputs to create a unique cortical area for the dorsal stream visual processing, the middle temporal visual area MT (figure 2), which is dominated by a retina to LGN to V1 to MT pathway [49,61,62]. If so, how did this happen, and what are the consequences for visual processing in primates?

Figure 2.

(a) Lateral view of the brain of small nocturnal galago (a strepsirrhine primate) that likely reflects much of the brain organization already present in ancestral primates. Arrows indicate major connections. In primates, both dorsal and ventral streams of visual processing start in primary visual cortex (V1). The middle temporal visual area (MT) of the dorsal stream gets visual inputs directly from V1, and from the second visual area (V2) and the third visual area (V3), areas activated directly from V1. MT gets visual information from the magnocellular (M cell) pathway from V1 and band-like modules in V2. These bands are not well documented in galagos, but they are present in all studied anthropoid primates. The other three types of bands in V2 mainly receive parvocellular (P cell) information from V1 and project to the dorsal lateral (DL or V4) visual area, which has rostral (DLr) and caudal (DLc) divisions. MT is interconnected with adjoining visual areas, the MT crescent (MTc), the middle superior temporal area (MST) and the fundal area of the superior temporal sulcus (FST), which has dorsal (FSTd) and ventral (FSTv) divisions. The dorsal medial area (DM) also contributes to dorsal stream visual processing. Posterior parietal cortex is divided into rostral (PPCr) and caudal (PPCc) regions. Auditory cortex included primary (A1) and rostral areas (R). Somatosensory cortex includes areas 3a, 3b (S1), 1 and 2, S2 and PV. Frontal motor areas include primary motor cortex (M1) and premotor areas PMV (ventral premotor area), PMD (dorsal premotor area) and SMA (supplementary motor area). (b) In anthropoid primates, V2 is divided into histologically visible thick and thin stripes or bands that are either pale or dark. The thick bands get inputs from the M cell-dominated layer of V1, while the thin bands and the pale bands receive inputs from the P cell-dominated layers of V1.

3. The recovery of good vision in early nocturnal primates

A major consequence of the nocturnal bottleneck of over 200 million years was that vision became less important for early mammals, and the eyes and central visual system structures regressed. This regression of both of the two visual systems perhaps reduced the relative dominance of the tectal-to-pallium-pathway, thereby making it possible for later emerging mammals and their descendants to rely proportionally more on the more direct retina-to-LGN-to-visual cortex pathway. Multisensory activity in neocortex could be facilitated by olfactory inputs to orbital frontal cortex, somatosensory inputs to parietal cortex and auditory inputs to temporal cortex [63,64]. The areal organization of cortex was more suitable for expansion and modification than the retinotopic laminar organization of the SC. Of the taxa of mammals that emerged before the great extinction event 66 million years ago [65], only the early primates, then nocturnal, regained a high degree of reliance on vision, and for them, the LGN to primary visual cortex pathway became proportionately greater and more important than the tectum to cortex pathway.

The early primates that emerged before the extinction of dinosaurs were small and nocturnal [66]. The persistence of small body size in early mammals, including primates, has been attributed to a reliance on insects and other small invertebrates that provide a sustaining diet for only mammals of small size [67]. Just before the extinction of dinosaurs, early primates evolved large forward-facing eyes that allowed effective vision in dim light while they remained nocturnal. These primates evolved a better visual system for nocturnal vision as they adapted for foraging for insects [68,69] and possibly plant food [70] in the fine branches of trees. The large forward-facing eyes not only collected more light, but the large overlapping visual field of the two eyes increased effective light capture due to binocular summation [71]. The forward-facing eyes were directed toward insect prey and other small invertebrates. This forward placement of the eyes meant that the relevant part of the retinal image in front of the face was based on the centre of the lens, thereby reducing image distortions from the lens that increase with distance from the optical axis [72]. In addition, the forward-facing eyes enlarged the binocular field and the field of stereoscopic vision, perhaps allowing camouflaged insect prey that matched their background to stand out more clearly in depth [68,73]. Another possible advantage of a large binocular visual field is to use the two different images to see around visual clutter (the X-ray hypothesis; [74]). Importantly, humans and likely other primates perform better in reaching and grasping tasks during binocular compared to monocular viewing [75]. As vision was more important for these early primates, retinal inputs had increased [76] and the forebrain was proportionately larger in comparison to other mammals [16,76], suggesting an expanded role in cortical processing for vision. The olfactory bulbs and olfactory cortex were reduced in comparison to neocortex, as olfaction became less important, perhaps because scent marking in the fine branch environment was less productive than visual recognition of conspecifics, predators and prey.

The reduction of the protruding jaw in early primates may be another adaptation that provided improved vision at close range directly in front of the face. As a consequence, capturing the resisting insect prey with the reduced jaw would place the eyes in some danger, and it was better and safer to reach and grasp prey, and bring prey to the jaw after they are subdued. Forelimbs and hindlimbs were already adapted for grasping thin branches for support, and early primates were distinguished from other mammals by having nails on all or most digits and toes rather than claws, thereby providing mechanical support for the pads on the tips of long fingers and toes [77,78]. The hands and feet also had soft pads and other features that were adaptations for gripping thin branches of trees and small prey [79]. The powerful grasp of present-day primates likely evolved with the first primates [80]. Present-day strepsirrhine primates either retained or re-evolved a grooming claw on the second toe, and it is now ‘widely accepted’ [69] that the long sharp claws on the digits and all but the first toe of marmosets were redeveloped as they became smaller and squirrel-like and more dependent on vertical climbing. Vision would have been used both to identify objects, but also to guide behaviour. For ancestral primates, the use of the forearm to skillfully grasp thin branches for support, reach and grasp prey and other items of food, and bring food to the mouth are all behaviours likely guided and controlled mostly by the dorsal stream of visual processing [81–83]. The ventral stream would have been most important for object identification, and for social primates, recognition of individuals [23,24].

We propose that the ventral stream in primates is an elaboration of the retina to LGN to primary visual cortex pathway, and that the dorsal stream is an elaboration of retina to SC to the posterior pulvinar to temporal cortex pathway of early mammals, and that the basics of these two pathways can be traced back to early vertebrates. In especially social primates, and perhaps others, visual information originating in V1 is processed over several areas to activate a series of areas or domains in successively more ventral sectors of the temporal lobe with a number of domains devoted to the difficult task of recognizing large numbers of individual faces [84], as well as other classes of objects [85,86]. By contrast, visual information from the SC activates neurons in the posterior pulvinar that project to satellite areas [87] of the MT complex (MST, MTc, FSTd and FSTv; figure 2) in temporal cortex. These areas around MT in turn project to posterior parietal cortex (PPC) [49,61,82,88]. These areas and visual areas MT and DM with projections from V1, directly or indirectly (see below), activate action-specific domains in a strip of PPC (figure 2, [82]).

In most non-primate mammals, the relay of visual information from the SC to the temporal cortex remains a major source of information for further processing and use in other cortical areas that are likely species-variable areas of the dorsal stream. Importantly, almost all retinal ganglion cells in rodents and rabbits project to the SC [89–91]. With some exceptions (e.g. cats), the SC likely gets projections from nearly all retinal ganglion cells in most non-primate mammals, and the colliculus remains the most important source of visual information for the dorsal stream of visual processing. Thus, most of these mammals remain capable of considerable object discrimination after lesions of primary visual cortex [50,53,92]. Quite differently, as many as 80% of ganglion cells in diurnal primates project only to the LGN [93], and the middle temporal visual area, MT—a dorsal stream area that is unique to primates [94]—gets direct and indirect visual information from V1 in all primates (e.g. [62,95]). After further processing, MT distributes this information to other areas of the MT complex [87,96], and to the caudal part of PPC [97]. Another visual area that may be unique to primates, the dorsomedial visual area, DM, also receives direct inputs from V1 [98] and contributes to dorsal stream areas [97,99,100].

4. Invading the diurnal niche: squirrels and tree shrews

Early mammals likely remained nocturnal for many millions of years to avoid predation by diurnal dinosaurs [16]. The long period of forced nocturnal life for mammals came to a sudden end 66 million years ago when a two mile-wide asteroid plunged into the sea of Mexico with an explosion that darkened the sky, caused fires and high temperatures, and then cooled the darkened earth for a period of years [9,101]. The result was the extinction of not only the dinosaurs, but of most plant and animal life [102]. The 1000 or more species of dinosaurs were lost, but some species of monotremes, marsupials and eutherian mammals, including nocturnal primates, survived and diverged and expanded due to the new opportunities to occupy new environments, including diurnal environments. Free of the extensive feeding by large dinosaur herbivores, neotropical rainforests became dominated by flowering plants that formed dense, closed canopies that provided fruit and leaves that fed early primates [102] and allowed them to become bigger. Many mammals remained nocturnal. Others became cathemeral or crepusular, while some became diurnal [103]. Some strepsirrhine (prosimian) primates became diurnal, while early haplorhine (anthropoid) primates were diurnal, with tarsiers and owl monkeys reverting back to being nocturnal.

In considering adaptations to diurnal life, common tree shrews and most tree squirrels are prime examples of mammals that most fully adapted to daytime vision. They resemble each other in appearance and in adaptations to a mainly arboreal life. Both evolved as separate clades millions of years after the extinction of dinosaurs, with tree shrews of Southeast Asia representing the Order (Scandentia), which is more closely related to primates than squirrels (Rodentia) [104]. The fossil record indicates that tree shrews have changed little over at least 34 million years, and tree shrews have been considered a ‘living model of an ancestral primate’ [105]. Tree squirrels emerged more recently about 14 million years ago in a part of North America [106]. Squirrels and tree shrews look so much alike that tree shrews have been considered as a type of squirrel [107]. Tree shrews have been intensively studied, largely due to Le Gros Clark [108], who considered tree shrews as the most primitive primate. While there are nocturnal species of squirrels and tree shrews, the diurnal species have more expansive visual systems. Both diurnal tree shrews and squirrels have been described as having an all cone retina. While this is nearly true, small numbers of rod receptors have been reported in subsequent studies [109–111]. As humans have proportionately more rods, we see better in dim light than do squirrels and tree shrews.

Squirrels and tree shrews are clades in the Superorder, Euarchontoglires, with primates, but their visual systems differ from those of primates in similar ways. Squirrels and tree shrews resemble primates in having large eyes with retinas that are densely packed with cones, in having a laminated LGN and a distinctive area 17 (V1), and in having a large portion of neocortex devoted to vision [112–115]. They both have a very large SC with more pronounced lamination than found in most mammals, including primates [116]. Their SC is roughly 10 times larger than the SC of a nocturnal rodent of a similar size, such as a laboratory rat [117]. The SC also differs from those of primates by receiving inputs from nearly all of the ganglion cells of the contralateral eye [118,119], and by representing the complete retina of the contralateral eye [120]. The SC projects densely onto the posterior nucleus of the pulvinar, PIp (figure 3), which relays to temporal visual cortex. Thus, the SC remains an important and possibly a major source of visual information to cortex in squirrels [121] and tree shrews [119]. These highly visual diurnal mammals have also escaped the nocturnal bottleneck by upgrading both the retina to geniculate and retina to SC systems, while primates are much more dependent on the retina to geniculate system. As a result, tree shrews [56] and squirrels [54] preserve much of their functional vision after V1 lesions, while monkeys and humans have ‘cortical blindness' after V1 lesions [58,122].

Figure 3.

In mammals, temporal cortex receives a relay of visual information from the SC via VGLUT 2-positive projections to the posterior nucleus of the inferior pulvinar (PIp). (a) Proposed functional areas and regions of neocortex on a lateral view of a squirrel brain. The temporal posterior region receives activating inputs from PIp. (b) The PIp nucleus of the pulvinar and the adjoining LGN of a squirrel. The frontal brain section was processed for the vesicular glutamate transporter isoform VGLUT 2. The high-level expression in PIp reflects activating inputs from the SC, while the reactivity in the LGN reflects activating inputs from the retina. (c) Cortical areas on a lateral view of the brain of a tree shrew. The temporal posterior region gets activating inputs from PIp. (d) The PIp nucleus of the pulvinar and the adjoining LGN of a tree shrew. Both are VGLUT2 positive. In (a) and (c), area 17 is V1, while area 18 is V2. For other architectonic divisions of squirrel and tree shrew brains, see Wong & Kaas [114,115]. In monkeys, PIp is divided into two nuclei. For comparisons with galagos and monkeys, see Baldwin et al. [112]. (Online version in colour.)

5. Visual systems in present-day nocturnal and diurnal primates

Early primates were nocturnal, but after the extinction of the dinosaurs, most became diurnal. The early haplorhines or anthropoids all were diurnal, although the lines leading to present-day tarsiers and owl monkeys reverted to become nocturnal again. On the island of Madagascar, some strepsirrhines or prosimians, there free of competition from diurnal monkeys, became diurnal or crepuscular [123–125]. Nocturnal primates largely fed on insects and small invertebrates, and this food supply restricted them to having smaller body sizes and brains. The diurnal primates depended more on fruit and other plant food, and generally became larger. Those that ate fruit had larger brains with more neocortex than those of a similar body size that ate leaves. The formation of social groups by many diurnal primates offered protection from predators and the defense of home territories. Primate visual systems responded to these new opportunities. As over 350 present-day species evolved, major differences in body and brain size, as well as behaviour, emerged. Yet, some consistencies remained. Most notably, neocortex is a larger part of the brain and is more densely packed with neurons in all extant primates than in other mammals [126], and more of this cortex is devoted to vision [63,72].

As for other mammals, early primates had eyes with both rods and cones. The cones were sensitive to short wavelengths (blue or S cones) or medium to long wavelengths (green/red or M/L cones). The S cones were not that critical in all primates and they were lost in nocturnal lorises and galagos, and in nocturnal owl monkeys [127]. The emerging anthropoid branch of the primate radiation of diurnal primates retained the ancestral blue (S) cones and the medium to long (M/L) cones in greatly increased numbers. These early monkeys occupied parts of Africa and somehow a few rafted to Central America at least 35 million years ago to populate Central and South America [128,129]. Somewhat earlier, several anthropoid groups had colonized Africa from Asia [130]. Present-day Old World monkeys and apes, as well as humans, have fully trichromatic vision, which would be very important in being able to judge the ripeness of fruit. Most species of New World monkeys have a weak form of trichromatic vision as alleles for slightly different pigments in the M/L cones on the X chromosome emerged. Thus, those females with different alleles on each of their two X chromosomes have a weak form of trichromatic vision, of likely some use, while all males with only one X chromosome do not. However, one branch of New World monkeys, the howler monkeys, evolved a retina with three cones (S, M and L) by duplicating the ancestral M/L gene [131]. Overall, trichromatic vision appears to have evolved in some anthropoid primates well after they became diurnal, along with a great increase in cone numbers. These cone outputs via retinal ganglion cells not only mediate colour vision, but also high visual acuity [132]. The manifestation of a fovea in the retina allowed for an even higher visual acuity.

The mammalian ancestors of primates had three major classes of ganglion cells of the retina that provided outputs to the LGN and the SC, the midget cells providing the parvocellular (P) pathway, the larger ganglion cells for the magnocellular pathway and the small ganglion cells for the koniocellular pathway [133,134]. These pathways have been called the X, Y and W pathways in non-primate mammals, and sometimes in primates [135], suggesting that these three classes appeared early in mammalian evolution. The P pathway provides visual detail and colour to the ventral stream. The M pathway is sensitive to stimulus change and thus is important in signalling stimulus motion and change in the dorsal stream. The K pathway is more varied, with many K cells carrying blue cone signals. The colour-coding K cells in the LGN of marmosets have been most studied and the K on/off cells respond transiently and are thought to likely contribute to dorsal stream processing [136]. In contrast with the rapidly conducting M ganglion cells, the K ganglion cells have thin, slowly conducting axons.

In diurnal anthropoids, the P pathway is much more important, and it is this pathway that provides the dominant inputs to the cortical ventral stream. The M pathway contributes most to the dorsal stream, while the small K pathway, which is reduced in diurnal primates, contributes weakly to both cortical streams. Thus, the two P layers of the LGN in nocturnal strepsirrhine primates (galagos, lorises and lemurs) are thin and undivided, while P layers are thick and subdivided into four or more sub-layers in diurnal monkeys, chimps and humans [137]. In macaque monkeys, and likely other anthropoids, P ganglion cells project only to the lateral geniculate P layers and constitute nearly 80% of the retinal ganglion cells [93]. Similar proportions of P ganglion cells appear to project only to the LGN of marmosets [138]. By contrast, both the M and K ganglion cells of the retina project to both the LGN and the SC. Thus, the P cell information reaches the cortex only via the geniculate projection to V1, and P cells provide the major input to the ventral stream of cortical processing, while the cortical dorsal stream in primates gets M and K information from both the SC and V1.

6. Tarsiers and owl monkeys: nocturnal anthropoid primates that evolved from diurnal ancestors

Tarsiers and owl monkeys are the only nocturnal anthropoid primates. Tarsiers were previously classified as prosimian primates, which are largely nocturnal, but now are recognized as an early branch of the anthropoid radiation [139]. Moreover, present-day tarsiers appear to have evolved from diurnal ancestors that subsequently reverted back to nocturnal life [140]. Thus, their visual systems have a mixture of traits with some reflecting adaptations to daytime vision of their diurnal ancestors and some reflecting their second adaption to night-time vision. The diurnal ancestors of nocturnal tarsiers had large forward-facing eyes without a reflecting tapetum. They had a fovea and likely a cone-dominated retina and a retina to LGN projection that was dominated by the P cell pathway. Present-day tarsiers have even larger eyes than their diurnal ancestors [141] that capture more light as a compensation for the loss of the tapetum. They have a retina dominated by rods, small numbers of both S and M/L cones and a degenerated and variable fovea [142]. Their LGN reflects the anthropoid pattern of layers, but the two parvocellular layers are thin and without sub-layers [143,144]. However, their LGN has a thick K cell region between the pairs of P an M layers [145] as an apparent adaptation to dim light. Importantly, the K cells do not form two distinct K cell layers as in strepsirrhine primates [137]. Their SC is not large, while the visual pulvinar conforms to the anthropoid pattern with two nuclei, PIcm and PIp, that relay SC inputs to the surrounding region of visual area MT. Primary visual cortex is extremely large, occupying over 20% of neocortex [144]. Thus, V1 mediates a high level of visual acuity without the help of an all cone fovea. The apparent cost of this exceptionally large V1 is that there is little room for higher visual areas. Areas V2 and MT are clearly present, and temporal cortex is large enough to contain several ventral stream visual areas, but PPC is relatively small. Tarsiers are likely highly dependent on V1 and less on higher levels of visual processing than other anthropoids.

In a similar manner, owl monkeys evolved from a line of diurnal New World monkeys, and they are closely related to diurnal titi monkeys [69]. As adaptations to nocturnal life, owl monkeys have much larger eyes than their diurnal relatives and more like other nocturnal primates. Owl monkeys also have a high ratio of rods to cones in their retina, and the cones are only of the M/L type, having lost the S type [146,147]. The ganglion cell densities of the retina are lower than in diurnal primates, and they only have a variable and rudimentary fovea [148]. Their eyes lack a reflecting tapetum. The LGN has two thin P cell layers that do not subdivide, as in other nocturnal primates, while lacking the distinct K cell layers as in diurnal monkeys and other anthropoids [137]. However, the zone between the pairs of P cell and M cell layers is packed with K cells [149], consistent with the premise that the K cell pathway is more important in nocturnal primates. As expected for nocturnal primates, V1 in owl monkeys has neurons with larger receptive fields and less of V1 is devoted to central vision than in diurnal primates [150], reflecting more receptor summation in the retina at the cost of visual acuity. A number of visual areas have been identified in owl monkeys, including some of those involved in dorsal and ventral stream processing (e.g. [72]). Thus, in becoming nocturnal, and possibly doing so to reduce competition from diurnal relatives, tarsiers and owl monkeys improved vision in dim light by evolving big eyes that are dominated by rods, and having more convergence of receptors to ganglion cells, at the cost of having only traces of a fovea and reduced acuity. For uncertain reasons, owl monkeys also lost their S cones, suggesting that S cones were not an important contributor to the K cell pathway. Consistent with the finding of [151] that the magnocellular layers of the LGN are relatively larger in nocturnal than diurnal primates, the M layers are larger in tarsiers and owl monkeys.

Several other terrestrial mammals that became nocturnal have also lost their S cones [152]. They all are nocturnal and appear to have evolved from diurnal ancestors. Most notably, flying squirrels evolved from highly diurnal tree squirrels with nearly all cone retinas, including S cones. Flying squirrels are nocturnal and have a retina that is nearly all rods. The retina has no S cones as a result of the gene for the violet-sensitive pigment becoming non-functional. While most nocturnal mammals have a functional gene for the S cone pigment, those that evolved from highly diurnal ancestors do not seem to need it.

7. The emergence of MT as a primate-specific visual area

We propose that MT was once part of a larger region of temporal cortex that was responsive to activation by a relay of visual information from the SC to the pulvinar. In early primates, MT became a distinct area in this temporal cortex zone of visual activation by the SC as MT became dominated by visual inputs from V1 and adjoining areas, V2 and V3, while losing its activating inputs from the SC via the inferior pulvinar [49,61,153]. Thus, MT emerged as an area dependent on V1, and V1 lesions deactivate MT [154–156]. As MT became a major contributor to the dorsal stream of visual processing visual, dorsal stream functions became more impaired by V1 lesions.

MT receives activating inputs directly from V1 and indirectly from modules in V2 and V3 that are activated by projections from V1 [98]. These inputs to MT depend on the M cell inputs to V1 as M cell and P cell inputs are highly segregated in different input and output sub-layers of V1. The M cell-dominated output layers of V1 then activate modules in V2, and likely in V3, that project to MT [157]. Thus, MT neurons appear to reflect only M cell activity [158,159]. Yet, M and P ganglion cells in the retina are responsive to both rods and cones. The critical difference is that the cone cell inputs to P ganglion cells result in colour opponent responses, while cone inputs to M ganglion cells are summed, making them highly insensitive to colour change. The V1 outputs to other areas and modules of areas are dominated by P cell inputs from the LGN to V1. In part, this includes blue-sensitive S cone responses in the P cell pathway and possibly in some of the K cell inputs. Other contributions of the K cell pathway are largely unknown [133,134].

These major changes in the sources of visual information for the dorsal stream of processing in primates suggest that the dorsal stream has become more elaborated and functionally important in primates, compared to non-primate relatives, tree shrews and rodents. As the dorsal stream became highly dependent on inputs from V1, V1 lesions impacted greatly on the functions of both the dorsal and ventral streams in primates. Thus, in macaques, Kluver [160] concluded that every visual aspect of objects is totally abolished after the removal of V1 in monkeys. Such monkeys were considered to be cortically blind. However, some functions for guiding motor actions with visual information based on the SC and pulvinar remained, but largely below visual awareness. Hence, these abilities were called blindsight [58,122].

8. Other visual pathways to the MT region

Other studies have reported that neurons in MT are responsive to visual stimuli after lesions or inactivation of V1 (e.g. [161]; also see [162] for an extensive review of pathways to MT). While there is some evidence that MT does receive some activating inputs from the SC via two nuclei in the pulvinar [163], these nuclei are now thought to project to only the areas surrounding MT and not to MT [164,165]. One other potential source of visual responses is the spread of activation from parts of MT with preserved inputs from V1 to deprived parts of MT via intrinsic horizontal networks [143,166]. Another possibility is that the recordings presumed to have been from MT have been from cortex surrounding MT, as these areas (MTc, MST and FST) have activating inputs from the SC via pulvinar nuclei [88]. As the areas surrounding MT are interconnected with MT, these connections could be a source of activation of neurons in MT, especially after long post-lesion recovery times. In addition, a small number of K neurons in the LGN project to MT [165] and do not degenerate after V1 lesions [167]. However, these neurons have response properties [133] that are quite different from those of MT neurons, and they would appear to be a poor and weak replacement. Finally, a small number of retinal ganglion cells project to the pulvinar nucleus PIm, which projects to MT [168]. However, their numbers are very small, and they tend to disappear with postnatal maturation. In the end, the behavioural evidence is clear: V1 lesions have a major impact on MT and on dorsal stream functions in primates. For this reason, Leopold [162] has referred to V1 in primates as the ‘bottleneck of visual signals’ to extrastriate cortex. V1 lesions in other mammals do not have the same impact because they do not have an MT that makes V1 critically important in dorsal stream functions.

9. MT-like areas in cats and megabats

MT-like areas or regions of cortex have been proposed for domestic cats and the highly visual fruit-eating bats. In these two cases, a cladistic analysis of MT features across taxa suggested that these MT-like areas are examples of convergent or parallel evolution [169]. The larger fruit-eating megabats are of special interest because they were once proposed to be flying primates because of primate-like features of their well-developed visual system [170]. These larger and more visual bats were called megabats to distinguish them from the smaller echo-locating microbats. Molecular and related studies have since provided compelling evidence that all bats are closely related and are members of the Superorder, Laurasiatheria. Bats are not primates or even in the Superorder, Euarchontoglires, with primates [171].

Megabats resemble primates in having large eyes that are somewhat more forward facing than in most mammals, but not as forward facing as in primates [94]. As in early primates, megabats are nocturnal and use vision to find fruit in dim light. For bats, flying in the daytime is dangerous because predatory birds are much better flyers. The retinal ganglion cell densities in megabats are high, but not as high as is diurnal tree shrews or even nocturnal galagos. The LGN is laminated, but in a different pattern from in primates or tree shrews. The SC is large, but contrary to previous assumptions, megabats do not have a SC that represents only the contralateral visual hemifield [172], as this feature appears to be unique to primates [94,173]. In a study of visual cortex, Krubitzer & Calford [174] found that V1 in megabats projects to V2 and a small MT-like area on the rostral border of V2 (see fig. 1 of [174]). Subsequently, Rosa [175] mapped extrastriate cortex in megabats and found evidence for a narrow V3 between V2 and this MT-like area, which he called the occipitoparietal area, OP. However, OP is not positioned in temporal cortex or separated from V1 by a visual area like V4 (DL), as in primates. OP's roles in dorsal and ventral stream processing are unknown.

Domestic cats are also special in that, for a time, their visual system was the most intensively studied. As a result, as many as 17 cortical visual areas have been proposed for domestic cats [176]. Cats are highly visual, and their eyes are large and forward-facing in this semi-nocturnal predator. The X (P) cells of the retina project almost exclusively to the LGN, while the Y (M) cells project to both the LGN and the SC, as in primates [177]. The X cells of the LGN project, as expected, to V1, and the Y (M) cells project to V1 and V2, while the W cells project to V1, V2 and the lateral suprasylvian region, LS and 21a. As in other mammals, the SC projects to a part of the pulvinar that then projects to temporal cortex, which is the LS (lateral sylvian) region in cats [178]. As both areas 17 and 18 receive dense inputs from the LGN in cats, lesions of V1 alone do not impact greatly on visual processing, but such lesions do impair the perception of higher spatial frequency stimuli [179]. Lesions that include both areas 17 and 18 do impact on visual acuity and discriminations of stimulus orientation [180]. However, these impairments are far less than the cortical blindness produced by V1 lesions in primates. Lesions of the SC remove the dorsal stream relay of visual information via the pulvinar, and this impairs the learning of visual form discriminations, but does not abolish them (e.g. [181]). Neurons in LS cortex are activated both by inputs from areas 17 and 18 and from the SC to pulvinar to LS pathway, and they seem at least partially to, compensate for each other after lesions [182]. Thus, the cortical inputs from V1 and V2 to LS do not play the critical role in cats as they do to MT in primates. Overall, it is clear that the visual system, and especially visual cortex, is well developed in cats, and that thalamic and cortical elements of the dorsal and ventral streams of processing exist. However, it is uncertain if these inputs remain isolated from each other or interact in temporal cortex, and if an area with the critical MT functions and significance exists. The similarities in relative position of LS and MT, their connections and other features of the visual system of cats and primates, such as ‘blobs’ in V1, appear to be results of convergent evolution.

10. Dorsal stream domains in parietal and frontal cortex of primates

The term ‘domain’ is used to distinguish small regions of cortex in parietal and frontal cortex that appear to be highly devoted to different specific classes of motor behaviour (figure 4). These domains are smaller than most sensory and motor areas, and at least in motor and premotor cortex (PMC), parts of areas. These action-specific domains include those for head and eye protection, looking, grasping, reaching, body defense and locomotion. Visually responsive areas or regions in the caudal half of PPC in primates [184] project to more anterior parts of PPC where domains for action-specific behaviour exist [82]. Other visual information comes more directly to these domains from MT and areas of the MT complex [87]. The action-specific domains in PPC activate functionally matched sets of domains in PMC and in primary motor cortex. These domains are present in strepsirrhine galagos, New World owl and squirrel monkeys [185], and at least to a large extent in the less fully examined macaque monkeys [81,186–188]. Also, there is accumulating evidence for such domains in humans (e.g. [189]). A much less complex organization exists in the less expansive PPC of tree shrews and squirrels [190,191].

Figure 4.

Action-specific domains in posterior parietal and frontal cortex that have been identified in primates. (a) Small regions (domains) where electrical stimulation evokes specific action have been most studied in galagos and squirrel monkeys, and less extensively in macaque monkeys. Functionally matched domains exist in PPC, primary motor cortex (M1) and premotor cortex (PMC). Matched domains are interconnected, but other connections differ for domains in each region. (b) The functional domains on a squirrel monkey brain shown at high magnification. CS, central sulcus; FSa, anterior frontal sulcus; FSp, posterior frontal sulcus; IPS, interior parietal sulcus; LS, lateral sulcus; STS, superior temporal sulcus. For details, see Kaas & Stepniewska [183]. (Online version in colour.)

In primates, these sensory–motor domains are most clearly revealed by electrical microstimulation, which evokes the domain-specific actions [192]. Studies based on electrical stimulation indicate that galagos and monkeys have at least eight functionally distinct domains in the rostral half of PPC [183]. These domains form a roughly somatotopic sequence, with evoked face movements most lateral followed by successively more medial sites for arm and hand movements, and finally hindlimb movements. They include domains for aggressive and protective face movements, eye movements, reaching, grasping and bringing the hand to the mouth. Most medial domains are for forelimb and hindlimb combinations of movements or for hindlimb movements alone [193]. The domains in PPC are at least bimodal as they receive both somatosensory and visual information (eg. [81]). They are likely to vary in organization and number according to the specific behaviours of the primates studied, but this has not been well studied yet. We expect a considerable elaboration of types of domains in humans, as there is evidence for dorsal stream domains for speaking [194,195] and for tool use [196,197]. A major point here is that the basic domains for arm and hand use take up much of the domain territories in PPC, PMC and motor cortex in all primates. Thus, when the ancestors of present-day primates emerged, they were highly dependent on visual guidance for the control of the forearm and hand [68–70,77]. Given the repeating sets of action-specific domains in PPC, PMC and motor cortex, large amounts of cortex were devoted to visually guided reaching, grasping, bringing food to the mouth and defensive movements of the arm. While these three replications of domains in PPC, motor cortex and PMC may seem redundant, the domains in each region appear to interact in ways that promote a specific action such as a reach to an object, while competing domains are suppressed. As each of the three main regions receives different types of information from other cortical areas and subcortical structures, the decision-making process becomes more complex and involves subcortical projections to the basal ganglia [198,199]. Nearly 40 years ago, Goodale [60] proposed that ‘in the monkey’, ‘there are likely to be relatively independent visuomotor networks mediating the different patterns of visually guided movement underlying locomotion, posture, reaching, climbing, grasping objects, catching insects, avoiding obstacles and escaping from predators'. But the locations of these networks were unknown. Now we know that there are substreams of goal-related domains in parietal, motor and PMC that compete with each other to mediate specific movements, while sometimes cooperating to produce combined movements [200]. Additionally, these substreams of the dorsal stream are multisensory [100] and functionally matched domains project to overlapping zones in the striatum. As Cisek [201] proposed, the dorsal stream areas ‘process sensory information to specify, in parallel, several potential actions’ ‘which compete against each other within the fronto-parietal cortex’ using ‘a variety of biasing influences’.

11. Ventral stream domains in temporal cortex of primates

The higher levels of ventral stream processing in the temporal lobe of primates have been most extensively studied in macaque monkeys and humans. In early pioneering studies in macaques, recordings with microelectrodes revealed neurons that were best activated by objects in their rather large receptive fields [202]. Since that time, many studies have revealed that small regions of temporal cortex, now called domains, are selectively responsive to faces, bodies or other objects. Much research has focused on the several domains that are sensitive to faces (see [85] for review). In at least some primates, these face-sensitive domains are in somewhat thicker cortex and can be anatomically identified as small ‘bumps'. These ‘bumps' are in places where neurons have been shown to be face-selective in macaques. Similar ‘bumps' have been identified anatomically in other primates, including other monkeys, baboons, apes and humans [203]. These domains appear to exist in fairly consistent locations and numbers within and even across species. Thus, it is likely that domains for faces and other objects exist in many or most anthropoid primates, and in their early anthropoid ancestors. Importantly, it is also clear that the selectivity of domains for faces and other objects depends on postnatal experience. Thus, monkeys reared without seeing faces develop domains sensitive to other body parts [204] and human face domains can become responsive to cars or birds in humans who are experts on cars or birds [205]. The consistent locations of these domains in temporal cortex have been attributed to the existence of protomaps or proto-structures that are present before birth and are postnatally modified by experience [85].

Presently, very little is known about the temporal cortex of strepsirrhine primates. There is only limited evidence for connections of visual areas DL (V4) and V2 with temporal cortex in galagos [206–208], and temporal areas project to frontal cortex, as in other primates [209]. But we don't yet know if strepsirrhine primates have functional domains in the temporal cortex that are similar to those in monkeys, and thus the organization of the ventral stream of visual processing in the ancestral primates remains uncertain.

12. Analogous dorsal and ventral streams in somatosensory and auditory systems in primates

While evolutionary change is often portrayed as a slow and steady process, changes sometimes occur in spurts that are followed by long periods of slow change with adjustments to the spurts [210]. Innovations in core systems tend to be preserved, but further changes that improve core functions are likely to evolve [211]. The innovative changes in the dorsal and ventral streams of visual processing that likely occurred in early primates have persisted as core processes for well over 66 million years, despite many changes in these streams and in the visual systems of primates.

Quite obviously, the core functions of the dorsal and ventral streams of visual processing have been improved by multisensory processing that includes auditory and somatosensory information. In addition, the functions of auditory and somatosensory systems in identifying or locating objects by sounds or touch would likely be potentiated by having two specialized cortical subsystems, one for each type of task. In support of this view, the cortical connections of auditory and somatosensory areas have been described in terms of dorsal and ventral specialized streams that also become more multisensory at higher levels [212–215].

13. Dorsal and ventral streams of visual processing in mice and rats

In view of the widely accepted evidence for both dorsal and ventral streams of processing in primates, two such streams are now being proposed for mice and rats, rodents that are included with primates as one of the clades of the Euarchontoglires. It is not a question about the existence of dorsal and ventral streams, as different subcortical visual pathways to temporal cortex and V1, via the pulvinar and the LGN, are recognized in rats and mice, as in other mammals (e.g. [216]). Rather, it has to do with the now prevailing assumption that both the ventral and dorsal streams start with projections to adjoining cortex from V1, as this was proposed for the dorsal steam of primates [24]. Given the common use of rats and now mice in neuroscience research [217], it is often tempting to make these model mammals primate-­like. This tendency is perhaps reflected in the proposals for large numbers of extrastriate visual areas in rats and mice and assigning them to dorsal and ventral streams [218–221]. These proposals differ from the evolutionary history proposed here, and from the now classical ‘two visual systems' of Schneider [50] (1969). It is also relevant that a considerable number of nine visual areas have been proposed as directly bordering V1 in rats and mice, while the proposed visual areas that border V1 in rats and mice have been alternatively interpreted as just two areas, V2 and a medial area, prostriata [222], as in primates, in the more visual rodents, squirrels [113,223], and perhaps most mammals [224]. Here, we propose that the dorsal stream of visual processing in rats and mice starts in temporal cortex as a result of a relay of SC inputs via the pulvinar to temporal cortex [225]. A cortical zone in temporal cortex (postrhinal cortex, POR) is dependent on inputs from the SC via the pulvinar for activation [226]. This is not a new idea, but for rodents, it relates back to the ‘two visual systems' of Schneider [50]. While the ventral stream of cortical processing starts in primary visual cortex, the dorsal stream, in our view, does not for most mammals. It is only in primates, we propose, that a major input from V1 to dorsal stream processing emerges, while the ancestral source of inputs to the dorsal stream remains significant only in providing ‘blind sight’. V1 lesions in rodents have less impact on cortically mediated vision than in primates because V1 does not contribute to the dorsal stream in a dominant way. In a recent study, response properties of proposed visual areas bordering or near V1 were studied after lesions of V1 or the SC in mice [227]. Surprisingly, ‘visual response properties' of even the bordering extrastriate areas ‘were preserved after V1 ablation’. Responses to visual stimuli were also preserved after SC lesions, although the tuning profiles of the higher visual areas were altered. This suggests a mixing of dorsal and ventral stream sources of activation in these visual areas or parts of visual areas. In a more visual rodent, grey squirrels, the SC sends driving inputs to the posterior nucleus of the pulvinar, which projects to a region of temporal cortex that does not border V1 [190]. Other nuclei of the visual pulvinar receive modulating inputs from the SC and project to V1 and visual areas bordering or close to V1. In a similar manner, modulating SC inputs to nuclei of the pulvinar in primates with projections to early visual areas may function to modulate visual activity in these areas.

14. Summary and conclusion

We propose that primates are unusual mammals in that primates evolved good vision during a long period of nocturnal life, while other mammals did not. Thus, they escaped the ‘nocturnal bottleneck’. This escape depended on adaptations for nocturnal vision by their eyes, but also by the central visual system. Most importantly, vision was processed in a greatly expanded sheet of visual cortex of many areas as parts of functionally distinct dorsal and ventral streams. Visual cortex became highly dependent on V1 as a source of visual information for both the dorsal and ventral streams. Further elaborations of the ventral stream for object vision allowed humans to recognize large numbers of individual humans and promote the formation of large social groups. Further elaborations of the dorsal stream led to the skillful use of tools and the production of language in our ancestors. There were many factors of importance in this long path of evolution. Stephen Jay Gould [228] was perhaps right when he suggested that the evolution of humans from early mammals in other times, circumstances and places would be nearly impossible.

A number of conclusions seem supported by the evidence.

• (1) The two cortical visual systems that have been proposed for primates have antecedents that extend back to the beginnings of vertebrate evolution. One pathway is from the retina to the LGN of the dorsal thalamus, or its forerunner. This nucleus or part of the thalamus projects to a part of the pallium that becomes primary visual cortex of mammals. The other major pathway is from the retina to the optic tectum (SC), which projects to a nucleus or region in the dorsal thalamus that is termed the rotundus or posterior pulvinar. This nucleus or region projects to a lateral part of the pallium that becomes part of temporal cortex of mammals.
• (2) In most vertebrates, the projections from the optic tectum to the thalamus provide the most critical visual information to the pallium. Thus, lesions of this system impact more on visual behaviour than do lesions of the retina to geniculate to pallium pathway.
• (3) Early mammals were small, rapidly maturing and nocturnal. These adaptations allowed them to avoid predators long enough to reproduce their numbers. Their adaptations to a long period of nocturnal life lasting 200 million years or more included less dependence on vision, a regressed visual system and a greater reliance on olfaction, hearing and touch. The eyes were small, classes of cones were lost or reduced in number, rods predominated, output ganglion cells were reduced in number and the SC target of retinal projections became less prominent. Cortical processing of sensory information became more important, as did the more direct retina to geniculate to cortex pathway.
• (4) Early primates remained nocturnal until the extinction of dinosaurs, while relying greatly on vision in dim light as they searched for insects and small vertebrate prey in the thin branches of tropical trees. Success depended on enhanced vision, and they adapted by evolving large forward-facing eyes with more retinal projections, and likely a reflecting tapetum backing the retina. The major targets of the retinal projections, the lateral geniculate and the SC, became larger and more differentiated, as did visual areas of cortex, which increased in number. The snout was reduced in order to block less of the frontal visual field, and prey was grasped with the hand rather than the jaw to avoid risk of damage to the large, forward-facing eyes.
• (5) A major change in visual processing in ancestral primates occurred when visual projections from V1, and those dependent on V1, captured part of the temporal lobe territory that had been activated by the SC via the pulvinar in the ancestors of primates. The result was an emergence of dorsal stream visual area unique to primates, area MT. This change allowed the dorsal stream processing to become much more dependent on information relayed from V1, and less dependent on visual information relayed from the SC. Thus, V1 became the most important source of visual information for both the dorsal and ventral streams of visual processing.
• (6) After the demise of the dinosaurs 66 million years ago, primates and other mammals rapidly evolved and branched into various lines of change to take advantage of the new and especially the diurnal opportunities. Many mammals became bigger and dependent on new sources of food. Often the eyes did not change very much, even for mammals that became diurnal (the bottleneck effect). The persisting strepsirrhine primates largely remained small and nocturnal, perhaps to avoid competition with the emerging anthropoid monkeys. Those nocturnal lemurs that made it by rafting to Madagascar avoided this competition. Thus, some species of lemurs became diurnal.
• (7) The newly diurnal anthropoid monkeys acquired features that promoted high-acuity colour vision, including a fovea of densely packed cones and a cone-dominated retina. About 80% of the retinal output was via cone sensitive P cells that projected to only the LGN and related to the expanded ventral stream of visual processing.
• (8) An early branch of the anthropoid radiation, tarsiers, and a later-evolving branch of New World monkeys, owl monkeys, returned to nocturnal life, and had to readapt to vision in dim light. They did this by evolving large light-collecting eyes with retinas composed of nearly all rods, and having more receptors converge onto a reduced number of retinal ganglion cells at the cost of lower visual acuity. Central nuclei and cortical areas reduced their representations of central vision, as the fovea was degenerate. The reflecting tapetum did not reappear.
• (9) Whether diurnal or nocturnal, all primates have an expanded dorsal stream of visual processing, which includes MT, areas of the MT complex in the temporal lobe, visual areas of the occipital–parietal region and action-specific domains in parietal cortex that activate the action-specific domains in premotor and motor cortex. Much of this sensorimotor cortex is concerned with arm and hand use.
• (10) Several non-primate mammals have MT-like areas, but these areas do not appear to be key dorsal stream areas that are highly dependent on V1.
• (11) An expanded ventral stream of visual processing appears to exist in all extant primates and was likely present in ancestral primates.

Data accessibility

This article has no additional data.

Authors' contributions

All authors contributed to the writing of the paper and construction of figures.

Competing interests

We declare we have no competing interests.

Funding

Our research has been supported by National Institutes of Health Grant no. RO1 EY02686.