A new neural framework for visuospatial processing

Abstract

The division of cortical visual processing into distinct dorsal and ventral streams is a key framework that has guided visual neuroscience. The characterization of the ventral stream as a ‘What’ pathway is relatively uncontroversial, but the nature of dorsal stream processing is less clear. Originally proposed as mediating spatial perception (‘Where’), more recent accounts suggest it primarily serves non-conscious visually guided action (‘How’). Here, we identify three pathways emerging from the dorsal stream that consist of projections to the prefrontal and premotor cortices, and a major projection to the medial temporal lobe that courses both directly and indirectly through the posterior cingulate and retrosplenial cortices. These three pathways support both conscious and non-conscious visuospatial processing, including spatial working memory, visually guided action and navigation, respectively.

The dorsal and ventral processing streams were originally identified in monkeys as two anatomically and functionally distinct pathways that originate in the striate cortex. The ventral stream was described as coursing through the occipitotemporal cortex to its anterior temporal target (area TE), and the dorsal stream as coursing through the occipitoparietal cortex to the posterior half of the inferior parietal lobule (area PG)^1,2 (FIG. 1a). These streams were soon extended — from area TE into the ventrolateral prefrontal cortex and from area PG into the dorsolateral prefrontal cortex³ (FIG. 1a). Lesions of the ventral and dorsal streams in monkeys produced selective deficits in object vision and spatial vision, respectively, leading to their characterization as ‘What’ and ‘Where’ pathways^1–3. Later, a patient with agnosia (D.F.), who has a large bilateral lesion of the occipitotemporal cortex and a small left-sided lesion of the occipitoparietal cortex (FIG. 1b)^4,5, was found to have impaired perception of objects but intact ability to reach to objects, including shaping her grasping hand to reflect the size, shape and orientation of the object. Similarly, patient D.F. could not accurately adjust the orientation of her hand to match the orientation of a distant slot but was accurate in orienting her hand when reaching toward that same slot⁶. These findings, combined with the dense interconnections between the posterior parietal and frontal premotor areas⁷, led to the proposal that the dorsal stream was more appropriately characterized as a ‘How’ than as a Where pathway⁸ (FIG. 1b). Further, the ability of patient D.F. to reach accurately for an object to which she could not consciously orient led to the hypothesis that the dorsal stream was concerned with automatic, non-conscious, visually guided action rather than with spatial perception; according to this view, only the ventral stream produces representations accessible to consciousness^8,9. Notably, however, patient D.F. is capable of making conscious judgments of absolute depth¹⁰, which probably depend on the dorsal stream.

Figure 1. Frameworks of visuospatial processing.

a | The original formulation^1,2,3 of the dorsal and ventral streams in the macaque monkey. The ventral stream is a multisynaptic pathway projecting from the striate cortex (area OC) to area TE in the inferior temporal cortex, with a further projection from area TE to ventral prefrontal region FDv. The dorsal stream is a multisynaptic pathway projecting from striate cortex to area PG in the inferior parietal lobule, with a further projection from area PG to dorsolateral prefrontal region FDΔ. On the basis of the effects of lesions in monkeys, the ventral stream was termed a ‘What’ pathway supporting object vision, whereas the dorsal stream was labelled a ‘Where’ pathway supporting spatial vision. b | The top panel depicts the location of the lesions in patient D.F. (shown in blue and indicated by white arrows) that led to impairment in object perception but not in the accuracy of orienting her hand when reaching to the same objects. This pattern of results led to the proposal^8,90, depicted in the bottom panel, that the dorsal stream is more accurately characterized as a motoric ‘How’ pathway supporting visually guided action than as a perceptual ‘Where’ pathway. c | The new neural framework for dorsal stream function that is proposed in this Review. At least three distinct pathways emanate from the posterior parietal cortex. One pathway targets the prefrontal cortex (shown by a dashed green arrow; see also part a) and supports spatial working memory (the parieto–prefrontal pathway); a second pathway targets the premotor cortex (shown by a dashed red arrow) and supports visually-guided actions (the parieto–premotor pathway); and the third targets the medial temporal lobe, both directly and through the posterior cingulate and retrosplenial areas (shown by a dashed blue arrow), and supports navigation (the parieto–medial temporal pathway). PCC, posterior cingulate cortex; RSC, retrosplenial cortex; TE, rostral inferior temporal cortex; TEO, posterior inferior temporal cortex; V1, visual area 1 (also known as primary visual cortex). Part b, top panel is modified, with permission, from REF. 5 © (2003) Oxford Journals.

Since the findings in patient D.F., our knowledge of both the anatomical connections and functional properties of the dorsal stream has vastly increased, and this necessitates a new neural framework for visuospatial processing. We review anatomical and functional evidence in primates indicating that the dorsal stream actually gives rise to three distinct, major pathways; a parieto–prefrontal pathway, a parieto–premotor pathway and a parieto–medial temporal pathway (FIG. 1c), which primarily support spatial working memory, visually guided action and spatial navigation, respectively. The parieto–medial temporal pathway, which is the main focus of this paper, has not been well-characterized previously in primates (but see REFS 11–14 for related proposals). This pathway courses medially through the posterior cingulate cortex (PCC) and retrosplenial cortex (RSC), providing spatial information to the medial temporal lobe (MTL) (FIG. 1c). The existence of these multiple pathways, each mediating a different class of visuospatial functions, suggests that earlier characterizations of the dorsal stream as being either a perceptual Where or a motoric How pathway were insufficient to adequately capture the diversity of its visuospatial functions.

Here, we review anatomical evidence, primarily from monkeys, in order to update the circuitry within the classical dorsal visual stream, after which we show that there is a trifurcation of this stream beyond the parietal cortex. We then review the functional evidence that the parietal cortex has properties consistent with it serving as the source for each of the three proposed pathways, and we briefly describe the first two pathways — the parieto–prefrontal and parieto–premotor pathways. We next focus on the parieto–medial temporal pathway and show that in both humans and monkeys, the two intermediate areas along this pathway (PCC and RSC) have the requisite properties to relay spatial information to the MTL. In addition, we show that damage to the parieto–medial temporal pathway in humans results in various forms of topographic disorientation¹⁵ that differ predictably depending on where along the pathway the damage has occurred. We conclude that the dorsal stream, which was originally defined as the pathway between the striate cortex and the posterior part of the inferior parietal lobule, actually comprises a widespread visuospatial processing system that contributes to both spatial perception and non-conscious spatial processing across numerous cortical areas in the frontal, temporal and limbic lobes.

Anatomy of the three pathways

We begin with an update to the original anatomical definition of the dorsal stream. This stream is now known to consist of a set of projections from early visual cortical areas to posterior regions of the parietal cortex — the latter including medial portions of the superior parietal lobule (SPL) — together with interconnections among those parietal regions. This occipito–parietal circuit is the common anatomical antecedent of the separate parieto– prefrontal, parieto–premotor and parieto–medial temporal pathways discussed below.

Occipito–parietal circuit

This circuit is shown in FIG. 2a. Portions of the primary visual cortex (V1) that represent the peripheral as well as the central visual field project to area V6 (part of parietooccipital area PO), a retinotopically organized and functionally distinct visual area in the anterior wall of the parietooccipital sulcus^16–18. Area V6 also receives strong projections from visual areas V2, V3 and V3A in the preoccipital region¹⁶. Recent anatomical studies have reported two main projections from area V6 to the parietal lobe. One of these remains medial, projecting to the bimodal (visual and somatosensory) areas V6A, medial intraparietal area (MIP) and ventral intraprietal area (VIP); the other courses laterally to the lateral intraparietal area (LIP), middle temporal area (MT) and medial superior temporal area (MST)¹⁶ (FIG. 2a). The portions of V1 that represent the central as well as peripheral visual field are strongly connected with area MT through V2, V3 and V4. All of these parietal and caudal superior temporal areas (V6A, MIP, VIP, LIP, MT and MST) are strongly interconnected with each other and with the caudal and rostral portions of the inferior parietal lobule (cIPL and rIPL, respectively; see the IPL subdivisions in FIG. 2a; inset)^19,20. This occipito–parietal circuit constitutes the common origin of three distinct pathways (FIG. 2b).

Figure 2. Anatomy of the three pathways.

a | The complexity of the occipito–parietal connections (shown by black arrows) on standard medial and lateral views of a rhesus monkey brain. The parts of visual area 1 (V1; also known as the primary visual cortex) that represent the central as well as peripheral visual fields are strongly connected with middle temporal area (MT) through visual areas V2, V3 and V4. By contrast, the parts of V1 that represent both the central and peripheral visual fields project through visual areas V2, V3 and V3A to a retinotopically organized and functionally distinct area V6 on the rostral bank of the parieto-occipital sulcus (pos). The visual information from area V6 reaches the parietal lobe through two main channels: one projecting medially to the bimodal (visual and somatosensory) area V6A and medial intraparietal area (MIP), which are located on the rostral bank of pos and the medial bank of the caudal intraparietal sulcus (ips), respectively; and the other projecting laterally to lateral intraparietal area (LIP) and ventral intraparietal area (VIP) in the ips and to areas MT and MST in the caudal superior temporal sulcus (sts). All of these posterior parietal areas are strongly connected with each other and with the surface cortex of the inferior parietal lobule (IPL). Feed-forward projections from lower- to higher-level processing areas (shown by single-ended arrows in the main figure) are usually reciprocated by feedback projections (not shown) from higher to lower areas; connections between areas at the same hierarchical level are shown by double-ended arrows in the main figure. The inset, which illustrates the findings from a study by Rozzi et al.¹⁹, depicts connections of IPL subdivisions with the posterior cingulate cortex (PCC) and ventral premotor area F5. Note that areas Opt and PG (subdivisions of the caudal IPL (cIPL)), which are primarily visual, appear to have stronger reciprocal connections with the PCC than with F5 (thick versus thin lines). The reverse holds for areas PFG and PF (subdivisions of the rostral IPL (rIPL)), which are primarily somatosensory. b | The sources and targets of the three pathways that emerge from the parietal component of the occipito–parietal circuit (also known as the dorsal stream). The parieto–prefrontal pathway (shown by green arrows) links areas LIP, VIP and MT/MST with a pre-arcuate region (area 8A; the frontal eye-field) and the caudal part of the principal sulcus in the lateral prefrontal cortex (area 46) — targets that serve eye movement control and spatial working memory, respectively. The parieto–premotor pathway (shown by red arrows) links areas V6A and MIP with the dorsal premotor cortex (areas F2 and F7), and also links area VIP with the ventral premotor cortex (areas F4 and F5) — targets that serve visually guided eye movements, reaching and grasping. The parieto–medial temporal pathway (shown by blue arrows; thick, thin and dashed lines represent dense, moderate and light projections, respectively) originates in the cIPL — that is, areas Opt and PG (see part a, inset) — and projects to subdivisions of the hippocampus (CA1/prosubiculum (proS), and presubiculum (preS)/parasubiculum (paraS)), both directly and indirectly via the PCC (areas 31 and 23), retrosplenial cortex (RSC; areas 29 and 30) and the posterior parahippocampal cortex (areas TF and TH in the rostral portion, and area TFO in the caudal portion) — targets that enable navigation and route learning. 23v, ventral subregion of the posterior cingulate; 28, entorhinal cortex; 35 and 36, perirhinal cortex; as, arcuate sulcus; cas, calcarine sulcus; CC, corpus callosum; cis, cingulate sulcus; cs, central sulcus; ios, inferior occipital sulcus; ls, lateral sulcus; ots, occipitotemporal sulcus; PGm, medial parietal area (also known as 7m); ps, principal sulcus; TE, rostral inferior temporal cortex; TEav, anterior ventral subregion of TE; TEOv, ventral subregion of TEO; TEpv, posterior ventral subregion of TE.

Parieto–prefrontal pathway

This pathway, which has its strongest sources in areas LIP, VIP, MT and MST, links the occipito–parietal circuit with two areas — a pre-arcuate region and the caudal portions of the banks of the principal sulcus in the prefrontal cortex (areas 8A and 46, respectively)^21,22 (FIG. 2b). The former target is strongly involved in the top-down control of eye movements, and the latter target is involved in spatial working memory (for example, REFS 23–25).

Parieto–premotor pathway

This pathway (FIG. 2b) is actually comprised of two parallel projections. One has its major source in areas V6A and MIP, and targets the dorsal premotor cortex (areas F2 and F7)^26,27. The other arises primarily from area VIP (as well as the somatosensory-related rIPL (areas PFG and PF) (FIG. 2 inset) and projects to the ventral premotor cortex (areas F4 and F5)¹⁹. This parieto–premotor pathway mediates eye movements²⁸, reaching and grasping^16,29–37, among numerous other forms of visually guided action.

Parieto–medial temporal pathway

The third pathway (FIG. 2b) is the most complex of the three. It links the cIPL (which includes areas Opt and PG in FIG. 2) with the MTL — including the hippocampus — through both direct and indirect projections (FIG. 2b). One set of efferents runs from the cIPL directly: first, to a small cytoarchitectonic zone located between the subiculum and CA1 (CA1/prosubiculum); second, to the pre- and parasubicular subdivisions of the hippocampal formation^38,39 (see also REFS 40,41); and third, to the posterior parahippocampal areas TF and TH⁴¹. In addition, there is a set of indirect projections from the same source to the same targets, relayed through a pair of serially connected caudal limbic areas — the PCC (areas 23 and 31) and the RSC (areas 29 and 30)^42–46. One of the two major outputs of these caudal limbic areas is to the presubiculum and parasubiculum, the same hippocampal subdivisions that also receive input directly from the cIPL. Their other major output is to the posterior parahippocampal cortex (areas TF, TH and TFO), which projects in turn to the CA1/prosubicular subdivisions of the hippocampus. The ultimate target of this complex pathway — the hippocampal formation — is implicated in both monkeys^47,48 and humans^49,50 in the complex spatial processing required for navigating through the environment. Indeed, as described below, processing throughout this pathway is focused on navigationally relevant information, from distant-space perception and different visuospatial frames of reference through whole-body motion and head direction to route learning and spatial long-term memory.

The tract-tracing evidence in support of a parieto–medial temporal pathway in monkeys is reinforced by evidence for similar connections in humans. Margulies and colleagues⁵¹ evaluated the functional (or effective) connectivity between medial parietal areas and the rest of the brain in humans and monkeys and found evidence for much of this circuitry. In particular, they showed that the time course of functional MRI signal changes in the regions that constitute the occipito–parietal circuit were strongly correlated, even at rest, suggesting a close connection between the human homologue of the inferior parietal lobule (that is, the angular and supramarginal gyri)⁵² and area PO (also known as V6; FIG. 2b) (FIG. 3). Strong correlations were also observed between time courses of fMRI responses in regions that constitute the parieto–medial temporal pathway — that is, the IPL, PCC and RSC — and the parahippocampal cortex (FIG. 3; see REF. 53 for similar results in monkey and see REF. 54 for related results in human connectivity). These findings in humans provide converging support for the proposal that a general occipito–parietal visuospatial system gives rise to a parieto–medial temporal pathway, among others.

Figure 3. Parieto–medial temporal pathway in humans.

This figure is based on resting-state MRI functional connectivity of the precuneus in humans. Medial parietal area PGm (also known as 7m) and area V6 (part of parieto-occipital area PO) in the caudal part of the medial surface show strong functional connectivity (black lines) with the angular gyrus, the likely human homologue of the caudal inferior parietal lobule (cIPL). V6 also shows strong connectivity with early visual areas in the region of the calcarine sulcus (cs), reflecting a network that is the presumptive human homologue of the occipito–parietal network observed in monkeys (FIG. 2a). Similarly, the posterior cingulate cortex (PCC, areas 23 and 31) and the retrosplenial cortex (RSC, areas 29 and 30), on the medial surface, show strong functional connectivity (shown by blue lines) with both cIPL and the parahippocampal gyrus in the medial temporal lobe, reflecting a network that is the presumptive human homologue of the parieto–medial temporal pathway observed in monkeys (FIG. 2b). ips, intraparietal sulcus; pos, parieto–occipital sulcus; SPL, superior parietal lobule. Figure is modified, with permission, from REF. 51 © (2009) National Academy of Sciences.

In the following sections we first consider the functional evidence for the role of the posterior parietal cortex in supporting spatial processing in all three pathways. We then review functional evidence from each of the regions along the parieto–medial pathway, from cIPL to MTL, and demonstrate that the functional properties of each area in the pathway are consistent with a role in navigation.

Functions of the three pathways

The posterior parietal cortex is one of the most heavily studied areas of the primate brain and it is active under a wide variety of conditions (for example, REF. 55), so by necessity this Review focuses only on the subregions and findings necessary to establish its functional role in the three proposed pathways.

Occipito–parietal circuit

The circuit that serves as the source of all three pathways seems to integrate information equally from central and peripheral visual fields⁵⁶ and to represent space largely, though not exclusively, in egocentric frames of reference (FIG. 2a). Although initial visual representations are entirely retinotopic, this circuit transforms those representations into additional reference frames relative to parts of the body as well as the eye. In the monkey, parietal neurons provide information about many egocentric aspects of vision, including optic flow (for example, REFS 57–59) and stimulus depth (for example, REFS 60–62). In humans, egocentric hemi-spatial neglect most commonly arises from damage to the IPL^63–65, whereas allocentric neglect (that is, hemi-spatial neglect relative to objects) is primarily associated with damage to ventral cortical areas, including the MTL^63,64. The egocentric maps of space formed in the occipito–parietal circuit are the functional antecedents of the three proposed pathways. Given the strong interconnectivity among the regions of the occipito–parietal circuit, it is not surprising that evidence for the functions of each of the three proposed pathways is distributed across multiple posterior parietal subdivisions.

Parieto–prefrontal pathway

The subregions of the occipito–parietal circuit that give rise to this pathway (LIP, VIP, MT and MST) are strongly involved in the initiation and control of eye movements (for example, REF. 66) and are crucial for spatial working memory (see REF. 67 for a review). This pathway (FIG. 2b) provides the input to prefrontal cortex that is necessary for top-down executive control of visuospatial processing. In particular, in monkeys, cortex in the intraparietal sulcus (IPS) (areas LIP and VIP) and posterolateral prefrontal areas (areas 8A and 46) are tightly coupled during spatial working memory tasks, showing co-activation⁶⁸, very similar firing characteristics⁶⁹ and reciprocal effects of inactivation⁷⁰. In humans, the activity of these subdivisions is related to spatial working memory^71–73. Further, lesions of the posterior parietal cortex in humans are associated with deficits in spatial working memory^74–76 and the top-down guidance of eye movements⁷⁷.

Parieto–premotor pathway

The posterior parietal subregions associated with this pathway (V6A, MIP, and VIP), and those that give rise to the parieto–prefrontal pathway (LIP, VIP, MT and MST), maintain coordinated maps of space and body position (FIG. 2b). In particular, all of these subregions maintain the continuously aligned representations of visual coordinates relative to the location of body parts that are necessary for visually guided action in peripersonal space⁷⁸.

In monkeys, LIP neurons modulate their response based on the position of a visual stimulus relative to the body⁷⁹. Area VIP in monkeys³² and the superior parietal lobule (SPL) in humans⁸⁰ contain head-centred maps of somatosensory and visual space. A recent study in monkeys reported that both LIP and MIP receive inputs from cerebellar regions that provide vestibular information about arm, eye and head position⁸¹, perhaps supporting the realignment of visual maps with body position during movement. Area 5 (the SPL) in monkeys⁸² and regions of the IPS in humans⁸³ have been found to maintain hand- and arm-centred visual maps of peripersonal space. MT and MST neurons can also integrate changes in visual information caused by observer motion^58,59. Further, all of these regions in both monkeys^{34,36,37,84,85} and humans^86,87 have been directly implicated in reaching and grasping objects, and some anterior regions of the parietal lobe contain three-dimensional representations of objects suited for guiding grip (see REFS 88,89 for reviews; BOX 1). Moreover, posterior parietal damage can lead to profound deficits in visually guided reaching and grasping (for example, REF. 90), which is part of the evidence that led to the characterization of the dorsal stream as a How pathway.

Box 1. Contributions of the posterior parietal cortex to multimodal spatial processing.

In limiting this Review to visuospatial processing, we recognize that we have largely neglected the role that the posterior parietal cortex plays in spatial processing in other modalities. We did note the role of the region’s somatosensory inputs (for example, REF. 96) in the creation of visual somatotopic maps⁸⁰. But there are also regions of the parietal cortex involved in the translation of purely visual information into representations suitable for guiding complex actions (see REFS 88,89 for reviews). In particular, it has been shown in monkeys that the anterior intraparietal area (AIP) contains representations of the three-dimensional shapes of objects suited for guiding grasping^189–191, and adjacent regions of the rostral portion of the inferior parietal lobule (rIPL) contain complex multimodal representations of space relative to the arm and hand⁹⁷.

The posterior parietal cortex also receives input from auditory cortex (for example, REF. 19), and posterior parietal neurons have been shown to encode the location of auditory targets¹⁹². Bilateral damage to the posterior parietal region in humans can result in substantial impairments of both auditory and visual spatial localization¹⁹³ and in the auditory homologue of visual hemispatial neglect, in which sounds from one half of space are ignored (see REF. 194 for a review). The importance of the posterior parietal cortex in spatial coordination across modalities is most evident in cases of cross-modal extinction. In this disorder, patients with unilateral parietal damage will neglect a contralesional stimulus in one sensory modality when an ipsilesional stimulus is present in a different modality (for example, REF. 195).

The posterior parietal region has also been broadly implicated in the long-term memory and retrieval of spatial information both within and across modalities. Bilateral parietal damage in humans can lead to impairments in recall¹⁹⁶ and imagination¹⁹⁷ of events (particularly their spatial aspects) and to deficits in mental rotation (for example, REF. 198). In addition, in monkeys, responses of regions of the posterior parietal cortex are modulated by the animals’ learned use of tools (see REF. 199 for a review). In humans, the anterior portion of the IPL and the intraparietal sulcus show evidence of selective representations of static tool stimuli^200,201. Further, damage to the posterior parietal cortex can lead to impairment in conscious knowledge of tool use²⁰², a finding that has led to the hypothesis that the posterior parietal region contributes multimodal spatial information about tools to a distributed network that is responsible for representing semantic tool knowledge²⁰³.

Within the IPL however, only the rIPL (also known as area 7b, and composed of areas PFG and PF) is specialized for guiding action in peripersonal space. Specifically, neurons in the rIPL maintain visual somatotopic maps⁹¹, are responsive to observed and executed actions⁹², and are selective for particular components of observed actions⁹³, just as neurons in the caudal regions from which the parieto–premotor pathway for vision originates. Furthermore, the rIPL receives vestibular input from cerebellum^94,95 and is strongly connected with somatosensory areas⁹⁶. A recent survey of the functional properties of IPL found that many PFG neurons were multimodal, showing sensitivity to both somatosensation, motor activity and visual stimuli in peripersonal space⁹⁷.

Parieto–medial temporal pathway

The same survey⁹⁷ reported that a subregion of cIPL (area PG), whose projections contribute to the parieto–medial temporal pathway (FIG. 2b), contained fewer somatosensory or motor neurons than rIPL and was more responsive to changes in eye position towards extrapersonal loci. This is consistent with the suggestion that cIPL is specialized for processing distant space^94,98 and less involved in guiding actions of the body. Indeed, one recent study reported no activation in the cIPL (specifically, area Opt) in response to observed or executed actions⁹². Further, some cIPL neurons encode space in world-⁷⁹ or object-centred^99–101 reference frames, which are potentially useful for navigation and encoding landmarks but are of limited utility for guiding the action of body parts. These neurons are also very sensitive to the speed of optic flow⁵⁷, which is useful for updating position during navigation⁵⁸. There are also reports that cIPL neurons code direction during mental navigation of mazes^102,103.

Differentiation among posterior parietal areas in humans is difficult, but fMRI signal changes have been observed in both parietal cortex and hippocampus during navigation¹⁰⁴, and this activity predicts the accuracy of chosen heading directions¹⁰⁵. One recent study has also reported that the response of the posterior parietal cortex during navigation of a virtual environment may be consistent with a representation of absolute distance¹⁰⁶. The representation of egocentric depth seems to involve V3A, the most caudal subregions of the posterior parietal cortex V6 and V6A, and the IPS. There is a large amount of information about depth throughout the IPS^60,61,107, and this information may contribute to the coding of three-dimensional shape⁶². Posterior parietal lesions can also lead to egocentric disorientation¹⁵, a form of topographic disorientation typified by striking deficits in navigation, and to impaired memory for landmarks¹⁰⁸. Patients with such lesions are unable to orient themselves within real or imagined environments and therefore can neither navigate nor describe routes between familiar locations^15,109,110. This deficit suggests that posterior parietal cortex is the source of the egocentric information needed for navigation.

In summary, there is strong functional evidence in both monkeys and humans for the participation of the posterior parietal cortex in different conscious and non-conscious forms of visuospatial processing. Different forms of spatial representation are widely distributed across the posterior parietal cortex, resulting in considerable overlap among the regions that are important for spatial working memory, visually guided action and navigation. This functional overlap is unsurprising given the strong interconnectivity within the occipito–parietal circuit. Unfortunately, there has been little research that directly compares the relative importance of these different functional properties (for example, peripersonal versus extrapersonal coding) within the same region. Such research is essential if we are to determine whether functional biases exist within the posterior parietal cortex in line with the proposed anatomical pathways originating in this cortex.

Nonetheless, as indicated in FIG. 2b, the three pathways clearly have differently weighted parietal inputs, with the parieto–medial temporal pathway being the most dependent on the cIPL. Next, we turn to evaluating the functional properties of the other regions along this complex pathway.

Functional properties of the PCC

The PCC (areas 31 and 23 in FIG. 2b) is often treated together with the RSC, despite its distinct cytoarchitecture¹⁴. However, the functional properties that are uniquely attributable to the PCC are closely related to those of the cIPL, which is consistent with the direct interconnectivity of the two regions. Here, we briefly review the role of the PCC in eye movements, attention and navigation.

In monkeys, the PCC is heavily connected with the supplementary eye field (SEF)¹¹¹, and PCC neurons are strongly modulated by the onset of behaviourally relevant targets, saccades made to those targets^112,113 and the motivational value of those targets^114,115. In humans, the PCC is activated during eye movement tasks^116,117. Furthermore, Dean and colleagues¹¹⁸ reported that some PCC neurons in monkeys also encode the location of saccades and their targets in allocentric coordinates, responding to the same allocentric location after whole-body rotations, both before and after a saccade (FIG. 4a). Consistent with its position in the parieto–medial temporal pathway, the PCC has also been proposed as an area that contributes to the translation between the egocentric representations of space in the posterior parietal cortex and the allocentric representations in the MTL (for example, REF. 119).

Figure 4. Functional evidence from PCC and RSC.

a | Design and results of a study by Dean and Platt¹¹⁸. Monkeys fixated the centre of the screen, after which a target appeared in one of ten positions across the upper visual field (top panel). After a delay, the central fixation cross disappeared and monkeys made an eye movement to the target. To determine whether a neuron encoded the position of the target in allocentric (that is, world- or screen-centred coordinates) or egocentric coordinates, monkeys’ heads were rotated, thus changing the egocentric but not allocentric position of the target (middle panel). Neurons in the posterior cingulate cortex (PCC) encoded target location in both allocentric and egocentric coordinates with a bias towards allocentric coding (bottom panel). b | Design and results of a study by Hashimoto, Tanaka and Nakano¹³⁸. Three patients with lesions of the retrosplenial cortex (RSC) were tested for their ability to coordinate allocentric and egocentric representations. Patients were placed in the middle of a 3 × 3 grid with three objects arrayed around them. After a study period, the patients closed their eyes, the objects were removed and the patients then had to recreate the array (middle panel). When patients were rotated before recreating the array, their performance was significantly impaired compared with the control situation (bottom panel), suggesting a deficit in coordinating egocentric and allocentric representations after changes in egocentric position. Part a is reproduced, with permission, from REF. 118 © (2006) Society for Neuroscience. Part b, data in the bottom panel are from REF. 138.

In humans, the PCC has been shown to participate in shifting spatial attention. The region responds strongly during top-down shifts of attention in response to cues¹²⁰. Activity in the region is also correlated with the speed of target detection¹²¹ and the increase in that speed for cued targets¹²², suggesting that it has a role in the allocation of visual attention throughout a display in response to cues. Consistent with this proposal, PCC activity increases with demand on spatial attentional selection¹²³.

Finally, the PCC has been directly implicated in navigation in both humans and monkeys. In monkeys, neurons have been reported that respond to particular places¹²⁴, and inactivation of the PCC neurons can lead to deficits in following previously learned routes¹²⁵. In humans, the PCC is active when moving dots simulate optic flow of a type that would be consistent with the observer moving forward along an unambiguous heading direction¹²⁶.

Functional properties of the RSC

The RSC (areas 29 and 30 in FIG. 2b) has been implicated in spatial memory, imagination and planning (see REFS 14,127,128 for reviews). These attributes are in keeping with the position of the RSC as a complex connectional hub between the posterior parietal cortex, posterior parahippocampal cortex and hippocampus. Here, we focus on the aspects of RSC function that suggest that the RSC plays a key part in the type of spatial processing required for navigation.

Although little is known about its function in monkeys, the RSC in humans seems to be directly involved in coordinating and translating between egocentric and allocentric reference frames^129–131 (see REF. 12 for a computational model of this process), enabling individuals to orient themselves with respect to their environment. Specifically, RSC lesions in humans lead to heading disorientation, a form of topographic disorientation^15,132 in which patients are unable to orient themselves with respect to landmarks in the environment. These patients are able to recognize landmarks but cannot extract directional information from them (for example, turn right at the light)^133,134. Importantly, in some cases, patients can draw detailed maps of familiar locations but still cannot describe routes through those maps¹³⁵, suggesting that the RSC may have a particular role in the coordination of egocentric heading and allocentric representations of the environment. In line with this suggestion, RSC is active when heading direction is calculated from optic flow¹³⁶ and is modulated by learned heading directions¹³⁷. Further, one recent study reported that RSC damage causes deficits in the coordination of egocentric and allocentric reference frames following body rotations¹³⁸ (FIG. 4b).

In human neuroimaging studies, a region that responds more strongly to scenes than to objects (and which overlaps both the RSC and some portions of the PCC) has been identified and labelled the retrosplenial complex¹³¹. The retrosplenial complex responds strongly to visual landmarks^139–141 and to the recall of familiar locations¹⁴². Furthermore, retrosplenial complex activity is stronger when participants make judgments about the spatial aspects rather than the non-spatial aspects of scenes and when they are exposed to familiar rather than unfamiliar scenes¹⁴³ (FIG. 5b). Retrosplenial complex activity is also greater when participants identify particular scenes than when they are asked to make more general judgments of scene category or meaning¹⁴⁴. The retrosplenial complex may also participate in coding the location of objects in scenes in relation to the locations of navigationally relevant landmarks¹⁴⁵. In rats, RSC damage leads to poorer coding by thalamic neurons of head direction relative to landmarks¹⁴⁶, further emphasizing its importance in integrating landmarks and egocentric orientation.

Figure 5. Functional evidence from retrosplenial complex and medial temporal lobe.

a | Location of the retrosplenial complex averaged across 38 participants. Notably, the location of this area is similar to that of the lesions described in FIG. 4b. b | Functional MRI response magnitude of the retrosplenial complex to different aspects of visual scenes. Retrosplenial complex responses were high when participants were asked to judge whether the depicted familiar scene was located to the east or west of a reference point (‘location’) and, separately, whether the image was taken facing to the east or west (‘orientation’). Retrosplenial complex responses were comparatively lower when participants made familiarity judgments about scenes, with the greatest reduction occurring in response to unfamiliar, as compared to familiar, scenes. c | Location of the parahippocampal place area (PPA) in the medial temporal lobe (MTL) averaged across 38 participants. d | fMRI response magnitude of the PPA to five different visual stimuli. PPA responses were far higher for scenes than for either single or multiple objects, and were equally high for both furnished and empty rooms. Parts a and c are reproduced, with permission, from REF. 131 © (2008) Cell Press. Part b is modified, with permission, from REF. 143© (2007) Society for Neuroscience. Part d is modified, with permission, from REF. 165 © Macmillan Publishers Ltd. All rights reserved.

When human participants are presented with an image of a part of a larger scene, the retrosplenial complex shows evidence of generating representations that extrapolate beyond the borders of the image^147,148. Furthermore, it seems that retrosplenial complex representations may be partly allocentric¹⁴⁹, with some degree of viewpoint independence¹⁵⁰, and activity in this region correlates with learning of allocentric layouts¹⁵¹. Taken together, these results suggest that RSC representations place the current view of the observer into a larger spatial context.

Functional properties of the MTL

Similar to the parietal cortex, the MTL, and the hippocampus in particular, is among the most heavily studied areas in primates, and all the pertinent findings cannot be covered in this Review. Here, we focus briefly on data that are directly relevant to navigation and that are most likely to be related to the interconnections of this region with the other components of the parieto–medial temporal pathway.

In monkeys, the posterior parahippocampal cortex — which is located between the ventral part of visual area 4 (V4v) and the perirhinal and entorhinal cortices — can be divided into a caudal subregion (area TFO) and a rostral subregion (areas TF and TH)¹⁵². Relatively little is known about the functional properties of TFO in monkeys, but damage to its likely human homologue within the lingual gyrus¹³¹ leads to a form of topographic disorientation known as landmark agnosia¹⁵. Patients with this type of agnosia have no difficulty in orienting themselves within an environment or in producing maps, however, they cannot recognize prominent, navigationally relevant landmarks (for example, REFS 153–155).

Ablation of areas TF and TH in monkeys (FIG. 2b) leads to deficits in location and object–place associative memory^156–158. Consistent with functions of a region concerned with processing spatial context, neurons from these areas (TF/TH neurons) in monkeys show relatively weak selectivity for individual objects and have wide visual receptive fields with only a weak bias towards the fovea¹⁵⁹. Furthermore, in keeping with its strong connections to the hippocampus, damage to the presumed human homologue of areas TF and TH leads to anterograde topographic disorientation^15,160,161, in which patients can no longer form representations of new environments. In contrast to the heading disorientation observed with RSC damage, these patients have intact orienting but cannot learn new routes through unfamiliar settings (for example, REFS 162,163).

There is an extensive human imaging literature on the parahippocampal place area (PPA; FIG. 5c), a region that spans the presumptive human homologues of TFO and TF/TH^164,165. The findings indicate that the PPA responds selectively to scenes more than to objects (FIG. 5d) and, consistent with the proposed role of the TFO in representing landmarks, that the PPA shows strong fMRI responses during the retrieval of landmarks from memory^140,166 and to navigationally relevant objects^167,168. Finally, in line with the role of TF/TH in creating spatial representations of complex environments, the PPA responds strongly during topographic and spatial learning (for example, REFS 169–171) and is more sensitive to the spatial layout of complex scenes than to their semantic content^165,172 (see REFS 173,174 for opposing views): it shows equivalent activation in response to seeing furnished rooms containing many objects and the same rooms when emptied of all objects¹⁶⁵ (FIG. 5d).

Each of the two indirect routes through which the proposed parieto–medial temporal pathway connects to the hippocampus (FIG. 2b) implicates a different aspect of the hippocampal contribution to navigation. First, in both rats and monkeys, the presubicular and parasubicular targets of the projections from cIPL and RSC contain a preponderance of head-direction cells^48,175, which respond selectively depending on the particular direction the animal is facing. These hippocampal subdivisions may also be part of a larger circuit that includes the anterior thalamic nuclei and mammillary bodies, and that represents and updates head direction¹⁷⁶. Through bidirectional connections, these cells could provide heading information to the RSC and be influenced by the descending spatial information from the cIPL.

Second, again in both rats and monkeys^48,177, the CA1 and prosubicular targets of the projections from the cIPL and areas TF/TH (FIG. 2b) contain place cells, which become active whenever the animal is in, or transits through, a particular part of the environment. In monkeys, these hippocampal subdivisions also contain allocentric spatial view cells^178,179, which respond whenever the animal looks at a particular part of the environment, regardless of distance. Importantly, the response of hippocampal place cells can be modulated by changing a landmark’s position relative to the boundaries of the space^180–182. This reflects the dense projections these cells receive from the posterior parahippocampal cortex, a region responsible for placing landmarks within a spatial context. Finally, there is evidence from human neuroimaging that the hippocampus is active during topographic route learning and navigation (for example, REFS 183,184).

Conclusion

We propose a new neural framework for visuospatial processing, in which the parietal component of the original dorsal stream serves as a neural nexus of visuospatial function, giving rise to a diverse set of processing pathways that mediate both spatial perception and visually guided action across multiple cortical areas within the frontal, temporal and limbic lobes. In short, this framework fundamentally alters the characterization of the dorsal stream from a single-purpose system to a multifaceted one. In this final section, we highlight the empirical and theoretical considerations that arise from this new framework, both for research within each of the three proposed pathways and for gaining a deeper conceptual understanding of visuospatial functions generally.

Two pathways originating in the dorsal stream consist of relatively simple, direct projections from the posterior parietal cortex to prefrontal and premotor areas that participate in spatial working memory and visually directed actions, respectively. The existence of a parieto–prefrontal pathway was noted early in the conceptual formulation of the dorsal and ventral visual processing streams³; its perceptual Where functions, particularly as they relate to spatial working memory, were investigated extensively by Patricia Goldman-Rakic and colleagues (for example, REF. 68). By contrast, the parieto–premotor pathway has functions consistent with those of a motoric How pathway supporting visually guided action, such as reaching for and grasping an object. Although this visually guided motoric contribution of the dorsal stream was also noted early¹, the relative importance of this contribution in humans — and its characterization as a largely automatic, non-conscious process (as opposed to the conscious correlates of human perception) — were not fully appreciated until the seminal research of Milner and Goodale^4–6,8.

The parieto–medial temporal pathway is the most complex of the three pathways as it consists of both direct and multisynaptic projections from the cIPL to the posterior parahippocampal cortex and hippocampus. The posterior parietal region seems to contain predominantly egocentric representations of space, whereas the MTL contains spatial representations of scenes that are mainly allocentric. The multisynaptic projections pass through the PCC and RSC, each of which seems to contribute to the coordination of egocentric and allocentric frames of reference, although it is still unclear how these transformations between reference frames are brought about. This pathway’s functional properties — uncovered in neuroimaging and lesion studies — clearly indicate that it is crucial for navigation, although such evidence does not exclude the possibility that it may contribute to other forms of visuospatial function as well.

To account for how these three pathways support such distinctly different visuospatial functions — conscious perception and nonconscious action — and yet emerge from overlapping sources within the posterior parietal cortex, we suggest that their differential cortical targets are largely responsible. It may not be a coincidence that the targets of the parieto–prefrontal and parieto–medial temporal (‘perception’) pathways are six-layered cortex that contain a granular lamina IV, a type of cortex that is common to all cortical sensory systems; indeed, the targets of the primary thalamocortical projection of each sensory modality contains koniocortex in lamina IV, which has the finest neuronal granulation of all. By contrast, the target of the parieto–premotor (‘action’) projection is five-layered cortex, labelled agranular because it lacks a lamina IV — the same agranular type as the primary motor cortex in the precentral gyrus. Whether or not these differences in the makeup of their cortical targets help to account for the striking differences in the functional characteristics — perception versus action — of the three pathways is a matter for future research.

In this Review, we have examined the functional properties of each area along the parieto–medial temporal pathway separately, but this approach is purely didactic. In fact, we assume that there are gradients of function within and between these areas, including their posterior parietal and medial temporal endpoints. A major premise in this perspective is that the function of a region emerges from its particular pattern of connectivity, and that densely and reciprocally interconnected regions must share at least some functional properties. This view makes it possible to account for the presence of some neurons with allocentric spatial representations in the posterior parietal cortex and the presence of head direction cells in the presubiculum and parasubiculum of the hippocampus.

This new framework also highlights the paucity of empirical data regarding the intermediate areas in the parieto–medial temporal pathway. We still know very little about the PCC and RSC in monkeys, and only a few studies have examined these areas in humans. We hope this Review will encourage further research into the functions of these posterior limbic areas in both species. In particular, more studies are needed to determine the relative dominance of egocentric versus allocentric representations (for example, REF. 118) at each area along the pathway and — perhaps even more importantly — to identify the neural mechanisms in each that result in the transformation of one frame of reference into the other.

In proposing a trifurcation in the extension of the dorsal stream, we do not mean to imply that there are three pathways only, for there may well be others. For example, there is some evidence for a processing stream that includes area MT and regions of the superior temporal sulcus, that is devoted to motion and form processing^56,185. We also do not mean to suggest that the three proposed pathways are entirely divergent. Ultimately, the purpose of all visuospatial processing is the guidance of adaptive behaviour. So, whereas the parieto–premotor pathway contributes to motor output directly, the perceptual processing that occurs in the parieto–prefrontal and parieto–medial temporal pathways must also eventually influence motor output, though indirectly. For example, the ‘perception’ pathways can converge through a set of projections between the prefrontal cortex and hippocampus that pass through the cingulum bundle¹⁸⁶. This set of connections could enable the coordination of spatial working memory and navigation based on long-term memory, as well as associative retrieval of the contents of one form of memory by the other. To guide behaviour, such an interaction would then need to be translated into a set of motor actions, perhaps even through the parieto–premotor system.

Finally, although this Review has focused exclusively on the visuospatial functions of the dorsal stream and its extensions, it also has implications for the object-vision functions of the ventral stream. Just as the three visuospatial pathways are presumed to interact at some point along their routes or at their terminations, all three pathways are available for interaction with the ventral stream (see also REF. 187 for a direct connection between the dorsal and ventral stream). The article in which the ventral and dorsal streams were first proposed¹ ended with the question of how the object and spatial information carried in these two separated systems are subsequently integrated into a unified visual percept. Our Review suggests, as one potential answer, that a major point of convergence and perceptual integration is within the MTL, to which both the ventral stream and the parieto–medial temporal pathway project¹⁸⁸. In addition to the convergence of the two streams in the hippocampus, it is likely that the ventral stream also provides the parahippocampal cortex with the form of information needed for the representations of landmarks, whereas the parieto–medial temporal pathway provides the spatial information required for marking their navigational relevance. On a more general note, it is entirely possible that a future review of the ventral stream will propose that this stream too contains multiple cortical pathways out of a central neural nexus, similarly complicating its characterization as a What pathway.

Glossary

Retinotopic

An organization or map in visual cortex that reflects the spatial organization of visual stimuli as they appear on the retina

Egocentric

An umbrella term for maps and/or patterns of modulation that can be defined in relation to some point on the observer (for example, head- or eye-centred maps)

Optic flow

The apparent motion of the environment caused by relative motion between the observer and the visual surround. During navigation, it can be a source of information about the observer’s movement

Neglect

A deficit resulting from cortical lesions that causes the observer to ignore part of visual space. This deficit can be egocentric, as in hemispatial neglect (in which one half of the visual field is ignored) or allocentric (for example, when the left side of all objects is ignored)

Allocentric

An umbrella term for maps and/or patterns of modulation that are defined in relation to an object exterior to the observer

Somatotopic map

A map (or a pattern of neural modulation) based on distance from some body part. For example, a cell might increase its firing with decreasing distance of a stimulus from the face or hand