A brief comparative review of primate posterior parietal cortex: a novel hypothesis on the human toolmaker

Abstract

The primate visual system contains two major cortical pathways: a ventral-temporal pathway that has been associated with object processing and recognition, and a dorsal-parietal pathway that has been associated with spatial processing and action guidance. Our understanding of the role of the dorsal pathway, in particular, has greatly evolved within the framework of the two-pathway hypothesis since its original conception. Here, we present a comparative review of the primate dorsal pathway in humans and monkeys based on electrophysiological, neuroimaging, neuropsychological, and neuroanatomical studies. We consider similarities and differences across species in terms of the topographic representation of visual space; specificity for eye, reaching, or grasping movements; multi-modal response properties; and the representation of objects and tools. We also review the relative anatomical location of functionally- and topographically-defined regions of the posterior parietal cortex. An emerging theme from this comparative analysis is that non-spatial information is represented to a greater degree, and with increased complexity, in the human dorsal visual system. We propose that non-spatial information in the primate parietal cortex contributes to the perception-to-action system aimed at manipulating objects in peripersonal space. In humans, this network has expanded in multiple ways, including the development of a dorsal object vision system mirroring the complexity of the ventral stream, the integration of object information with parietal working memory systems, and the emergence of tool-specific object representations in the anterior intraparietal sulcus and regions of the inferior parietal lobe. We propose that these evolutionary changes have enabled the emergence of human-specific behaviors, such as the sophisticated use of tools.

Introduction

The notion of two functionally segregated visual pathways has significantly shaped our conception of primate vision over the last few decades. The initial conception of the two pathway hypothesis, based largely on empirical studies of the macaque monkey, stated that the ventral-temporal pathway serves object vision, whereas the dorsal-parietal pathway serves spatial vision (Mishkin & Ungerleider, 1982). In accordance with this hypothesis, early monkey physiology studies showed shape- and object-selective responses in neurons of inferior temporal (IT) cortex (Desimone, Albright, Gross, & Bruce, 1984; Gross, Rocha-Miranda, & Bender, 1972; Logothetis & Sheinberg, 1996). In contrast, neurons in the posterior parietal cortex (PPC) transform visual information into different spatial reference frames with respect to the eyes, body, or world (Andersen, Snyder, Bradley, & Xing, 1997), and this information is further integrated with spatial information from the auditory and somatosensory systems.

Observations of the human visual system are also consistent with the two-pathway hypothesis. For example, a large swath of ventral temporal and lateral occipital cortex, referred to as the lateral occipital complex (LOC), responds more strongly to pictures of objects than to their scrambled counterparts (Grill-Spector, et al., 1999; Kourtzi & Kanwisher, 2001; Malach, et al., 1995). Both IT and LOC represent objects independent of the precise physical cues that define them (Grill-Spector, et al., 1998) and regardless of changes in external viewing conditions that affect an object’s appearance, but not its identity (Grill-Spector, et al., 1999; James, et al., 2002; Sawamura, et al., 2005; Vuilleumier, et al., 2002). Hence, IT and LOC display response properties that characterize an effective object recognition system subserving perceptual object constancy. With respect to spatial cognition, lesions of human PPC can lead to the neglect of contralateral space with respect to different spatial reference frames (e.g., spatial neglect, Corbetta & Shulman, 2011) or visuomotor deficits (e.g., optic ataxia, Perenin & Vighetto, 1988). PPC is also an important node in the spatial attention network, which underlines its more general role in spatial cognition (Szczepanski, Konen, & Kastner, 2010).

However, other observations from human subjects suggested that this original framework was incomplete. For example, some patients (most notably patient D.F.) with ventral stream lesions show severe forms of visual agnosia, yet can perform visually-guided reaching and grasping movements towards those same objects (Goodale, Milner, Jakobson, & Carey, 1991; James, et al., 2002). Such observations led to a reformulation of the two-pathway hypothesis, with an emphasis on vision for perception for the ventral pathway versus vision for action for the dorsal pathway (Goodale & Milner, 1992). In this influential framework, spatial cognition is one among other aspects of the dorsal visual representation. Notably, there is an assumption that there is also a representation of object information in the dorsal stream, which is specifically utilized for visually-guided movement.

In the past few decades, continued study of the human and macaque visual systems (reviewed below) has provided support for the broad framework of the action-perception two-pathway hypothesis. However, recent studies have revealed important differences across the two species, in particular with respect to dorsal object representations. As we review in detail below, the object vision system of the human dorsal pathway is much more developed and refined that that in the macaque. Indeed, the complexity of the human dorsal object representations appears to mirror the complexity of those found in ventral representations (Konen & Kastner, 2008a).

Here, we review and compare some of the literature on the dorsal pathway in humans and monkeys with particular emphasis on PPC. We show that many of the disparate findings are due to species differences in the functional organization of the dorsal pathway. Finally, we propose a novel hypothesis as to why the human dorsal pathway may have undergone significant functional transformations relative to other species in the primate lineage.

Topographic organization

Phase-encoded retinotopic mapping along the polar angle and eccentricity dimensions using functional magnetic resonance imaging (fMRI) has been widely used to reveal topographic organization within the human (Arcaro, et al., 2009; Brewer, Liu, Wade, & Wandell, 2005; Engel, Glover, & Wandell, 1997; Hagler & Sereno, 2006; Kastner, et al., 2007; Konen & Kastner, 2008a; Larsson & Heeger, 2006; Schneider, Richter, & Kastner, 2004; Sereno, Dale, Reppas, & Kwong, 1995; Swisher, Halko, Merabet, McMains, & Somers, 2007) and also the macaque monkey visual system (Brewer, Press, Logothetis, & Wandell, 2002; Kolster, et al., 2009). Specifically, such topographic mapping enables the simultaneous investigation of visuotopic organization across a large region of cortex, thereby allowing the visuospatial map of an individual area to be anchored in the framework of the topographic organization across multiple surrounding areas. Thus, comparing the topographic large-scale organization of primate PPC at the level of maps serves as a first useful approach in our comparative review.

In macaque monkeys, multiple visual areas within and adjacent to the intraparietal sulcus (IPS) have been identified in PPC (Andersen, Bracewell, Barash, Gnadt, & Fogassi, 1990; Pandya & Seltzer, 1982; Van Essen, 2004). Visual areas within PPC have been distinguished based on their cyto- and myeloarchitecture, as well as their connectivity patterns, including the ventral and dorsal lateral intraparietal areas (LIPv/d), and two cortical zones, dorsal prelunate (DP) and lateral occipital parietal (LOP), also referred to as caudal intraparietal (CIP). Arcaro et al. (2011) studied the visuotopic organization of these regions. They identified two visual field maps anterior to area V3A within caudal PPC (named CIP-1 and CIP-2). In the polar angle dimension, the representation in CIP-1 extended from regions near the upper vertical meridian (that is the shared border with V3A) to those within the lower visual field (that is the shared border with CIP-2), thus spanning a contralateral hemifield. The polar angle representation in CIP-2 was found to be a mirror reversal of the CIP-1 representation. Anterior to CIP-2, a third polar angle representation was found within LIP (visuotopic LIP, LIP_vt), thereby corroborating previous findings (Ben Hamed, Duhamel, Bremmer, & Graf, 2001; Blatt, Andersen, & Stoner, 1990; Patel, et al., 2010). The polar angle representation in LIP extended from regions near the upper vertical meridian (that is the shared border with CIP-2) to those near the lower vertical meridian (Fig. 1A).

Figure 1. Topographic organization of primate PPC.

Comparison of the fMRI-defined retinotopic organization of dorsal occipital and parietal cortex in both monkeys (A) and humans (B). Lines denote areal boundaries formed by phase angles at or close to the upper (red, dotted) or lower (blue, dashed) vertical meridian. Surfaces are rendered with anterior (A) up, posterior (P) down, medial (M) left, and lateral (L) right. Major sulci are labeled in each view: cs, central sulcus; pcs, postcentral sulcus; ips, intraparietal sulcus; ls, lunate sulcus; pos, parieto-occipital sulcus; tos, transverse occipital sulcus. (After: Arcaro, et al., 2011)

In humans, seven topographically organized parietal areas have been described: six of these areas form a contiguous band along the intraparietal sulcus, and one area branches off into the superior parietal lobule (SPL) (Konen & Kastner, 2008b; Schluppeck, Glimcher, & Heeger, 2005; Sereno, Pitzalis, & Martinez, 2001; Silver, Ress, & Heeger, 2005; Swisher, et al., 2007). Each of these topographic areas contains a continuous representation of the contralateral visual field, and boundaries between these areas correspond to alternating representations of the upper and lower vertical meridian. The most posterior area, referred to as IPS0 (sometimes also as V7), is located anterior to area V3A at the intersection of the transverse occipital sulcus and the IPS. IPS1 and IPS2 are in the posterior part of the IPS. More anterior, IPS3 and IPS4 are located in the anterior/lateral branch of the IPS (Fig. 1B), whereas the most anterior area, IPS5, typically extends into the intersection between the IPS and the postcentral sulcus (not shown in Fig. 1B, see Fig. 2B). An additional representation of the contralateral visual field, referred to as SPL1, typically branches off the most superior IPS areas and extends medially into the SPL (not shown in Fig. 1B, see Fig. 2B).

Figure 2. Functional organization of primate PPC.

Schematic outline of functional regions with respect to effector responses within monkey (A) and human (B) parietal cortex. In monkeys, anatomically-defined areas CIP, LIP, AIP, VIP, and MIP are located along the IPS. The functionally-defined PRR (parietal reach region) overlaps with and extends beyond MIP. In humans, topographically-defined areas IPS0–5 are located along the IPS, with SPL1 extending into the superior parietal lobe. In both species, preferences for different effectors are indicated by the yellow (saccade-related responses), green (reaching-related responses), and blue (grasping-related responses) color patches. In monkeys, these regions are largely confined within the IPS. In humans, reaching-related responses (hPRR, human parietal reach region) are typically found outside the IPS, medially in SPL, and grasping-related responses (hPGR, human parietal grasp region) are located laterally at the intersection of the IPS and the postcentral sulcus. A human-specific region in the left hemisphere in the inferior parietal lobule shows tool-related responses (red). Labels for surface orientation and major sulci are as in Fig. 1.

The topographic organization of PPC in the macaque and human can be compared based on polar angle (Konen & Kastner, 2008b) and representations of central space (Konen & Kastner 2008, unpublished observations; also see Swisher, et al., 2007). The topographic organization of V3A in the macaque is similar to area V3A in the human (Brewer, et al., 2002; Fize, et al., 2003; Tsao, et al., 2003; Van Essen, et al., 2001). The topographic organization and location in relation to surrounding topographic regions of CIP-1 and CIP-2 in the macaque is similar to areas IPS0 and IPS1 in the human. For both monkey and human topographies, the representations of central space are located on the lateral border (denoted by asterisks in Fig. 1), with more peripheral representations located medially. From a topographic perspective, LIP_vt in the macaque appears similar to IPS2 in the human. In both species, there is a posterior-to-anterior progression from the upper to the lower vertical meridian representations with a representation of central space on the lateral border (Fig. 1).

Topographic organization within the IPS in humans consists of additional areas, IPS3/4/5, extending further anterior. The IPS also extends anterior from LIP_vt in the macaque. However, consistent topographic organization has not been identified beyond LIP_vt. It is important to note that our comparison of the topographic organization of the dorsal pathway in humans and monkeys is not aimed at establishing homology between areas, but instead, it is aimed at revealing similarities and differences in large-scale brain topography as a first starting point to ultimately identify regions that show similar functional response profiles across primate species. We review functional response properties of IPS areas in the following sections.

Functional response properties: Effector specificity and multi-modal responses

A different way to look at the functional parcellation of primate PPC is based on the specificity of responses related to motor effectors for eye, grasping, or reaching movements. Macaque area LIP, which is located in the lateral bank of macaque IPS, has been shown to be involved in the encoding of saccadic eye movements (Andersen, et al., 1990). About half of LIP neurons exhibit pre- and peri-saccadic responses (Barash, Bracewell, Fogassi, Gnadt, & Andersen, 1991). In contrast, little evidence for saccade-related activity has been found in the ventral intraparietal area (VIP), which is located anterior to LIP in the fundus of the IPS (Schaafsma & Duysens, 1996); but see (Thier & Andersen, 1998). The majority of VIP neurons respond during smooth pursuit eye movements (53%) (Schlack, Hoffmann, & Bremmer, 2003), which is a higher proportion than has been reported in LIP (39%) (Bremmer, Distler, & Hoffmann, 1997; Colby, Duhamel, & Goldberg, 1993b). In humans, preferential responses for saccadic or smooth pursuit eye movements change gradually across areas of the IPS with IPS1/2 and the medial SPL1 preferring saccadic eye movements and IPS3/4/5 preferring smooth pursuit eye movements (Konen & Kastner, 2008b, schematically depicted in Fig. 2B). Interestingly, areas in close anatomical proximity such as IPS1/2 and SPL1 in the posterior/medial PPC and IPS3/4/5 in the anterior/lateral PPC show similar response characteristics with respect to effector specificity.

In macaque monkeys (Fig. 2A), neurons in the anterior intraparietal area (AIP), which is located in the anterior-lateral part of the IPS, play a key role in preshaping of the hand for grasping (Murata, Gallese, Luppino, Kaseda, & Sakata, 2000; Sakata, Taira, Kusunoki, Murata, & Tanaka, 1997; Sakata, Taira, Mine, & Murata, 1992; Taira, Mine, Georgopoulos, Murata, & Sakata, 1990). These neurons exhibit selectivity for specific grasping movements, as well as for the shape, size and orientation of three-dimensional visual objects, regardless of the object’s position in visual space (Murata, et al., 2000). Moreover, inactivation of AIP disrupts a monkey’s ability to utilize visual information for preshaping the contralateral hand (Gallese, Murata, Kaseda, Niki, & Sakata, 1994). In humans, grasping-specific activation is typically found in the left anterior IPS, partially overlapping with IPS5, and extending into the postcentral sulcus (Fig. 2B, schematically outlined in blue; Konen, Mruczek, Montoya, & Kastner, 2013). We refer to this area as the human parietal grasp region (hPGR) to emphasize the functional definition of these areas, while implying that there may be functional homology, or at least strong functional similarity between hPGR and macaque AIP. This area appears to be critical for grasping movements, since patients with lesions in the anterior-lateral IPS show deficits in finger coordination (Binkofski, et al., 1998; Jeannerod, Decety, & Michel, 1994), and transcranial magnetic stimulation of the anterior IPS, but not more posterior regions, disrupts grasping movements (Rice, Tunik, & Grafton, 2006; Tunik, Frey, & Grafton, 2005).

In macaques, neurons in the parietal reach region (PRR) have been associated with the planning and execution of reaching movements. PRR is located on the posterior-medial bank of the IPS (Fig. 2A). It is a functionally defined area (and thus may extend medially beyond the borders of the IPS) and likely overlaps with the medial intraparietal area (MIP) and area V6A (Andersen & Buneo, 2002; Snyder, Batista, & Andersen, 1997). PRR appears to code the targets for a reach in visual coordinates relative to the direction of gaze (Buneo & Andersen, 2006; Buneo, Jarvis, Batista, & Andersen, 2002; Pesaran, Nelson, & Andersen, 2006). Moreover, PRR exhibits intention-related activation reflecting the plan for a reaching movement (for review see Andersen & Buneo, 2002). In humans, reaching-specific activation is typically found medially in the left precuneus and SPL, partially overlapping with IPS1/2 and SPL1, in an area that we refer to as the human parietal reach region (hPRR) (Fig. 2B, schematically outlined in green).

In monkeys, neurons in VIP, LIP and PRR have also been shown to have bi- or even tri-modal sensory responses, to visual, auditory, and/or somatosensory stimuli (Fig. 3) (Mullette-Gillman, Cohen, & Groh, 2005, 2009; Schlack, Sterbing-D’Angelo, Hartung, Hoffmann, & Bremmer, 2005) suggesting an important function in multisensory integration. In that respect, a particularly critical area appears to be VIP. Colby and colleagues were the first to show tactile responses in VIP: multisensory VIP neurons showed large visual receptive fields with spatially restricted, aligned somatosensory receptive fields on the face and shoulders (Colby, Duhamel, & Goldberg, 1993a; Duhamel, Colby, & Goldberg, 1998). For example, a VIP neuron with a visual receptive field in the upper right quadrant of the visual field also typically has a somatosensory receptive field located on the upper right part of the forehead (Fig. 3A). In addition to the tactile responses, bimodal auditory and visual neurons were found to code comparable regions of space (Mullette-Gillman, et al., 2005, 2009; Schlack, et al., 2005). In most cases, the auditory responses were spatially tuned, and the auditory and visual receptive field locations were typically found to be spatially congruent (Fig. 3B; Schlack, et al., 2005). Similar results were obtained in LIP and PRR. Given their multi-modal response properties, regions in the macaque PPC likely integrate sensory information from multiple modalities. The finding that different regions of primate PPC are also closely linked to specific motor effectors indicates their functional significance in integrating sensory information with motor plans.

Figure 3. Multimodal responses in primate PPC.

(A) Tactile (left) and visual (right) stimuli converge onto similar directional selectivity and receptive field location in a multisensory neuron of area VIP. Upper row: non-preferred direction. Lower row: preferred direction. Reproduced with permission from Duhamel, et al. (1998). (B) Spatially congruent visual (left) and auditory (middle) receptive fields (RFs) in a VIP neuron. The RFs are superimposed in the right panel. Color bars indicate spike rates relative to spontaneous activity. Reproduced with permission from Schlack, et al. (2005). (C) Overlapping retinotopic (upper panels) and somatotopic (lower panels) maps in human anterior parietal cortex. The color contours in the lower panels show spherically aligned somatotopic whole body mapping data superimposed on the visual maps. Reproduced with permission from Sereno and Huang (2014). (D) Activation in the human parietal lobule during spatial and temporal auditory processing. Reproduced with permission from Michalka, et al. (2016).

In humans, a multisensory area that has a representation of retinotopic visual space as well as a somatotopic map has been revealed by somatosensory stimulation of the face in the form of air puffs (Sereno & Huang, 2006, 2014). This area is located in the anterior IPS and overlaps or is identical with IPS5. Furthermore, by combining full body air-puff mapping with wide field visual stimulation, bimodal (somatosensory and visual) fMRI mapping experiments revealed that visual and somatosensory responses overlapped in the parietal cortex extending beyond IPS5 to include a medial region concentrating on the lower body and peripheral lower visual field (Fig. 3C; Huang, Chen, Tran, Holstein, & Sereno, 2012). In contrast to visual and somatosensory responses, the evidence for auditory responses in human PPC is sparse. A recent fMRI study suggests that, depending on auditory task demands (spatial vs. temporal processing), auditory stimulation may recruit visuotopic regions within the PPC (Fig. 3D; Michalka, Rosen, Kong, Shinn-Cunningham, & Somers, 2016). Furthermore, transcranial magnetic stimulation (TMS) of PPC near the junction of the anterior intraparietal sulcus and the postcentral sulcus eliminates the facilitatory influence of vision (Pasalar, Ro, & Beauchamp, 2010) or audition (Serino, Canzoneri, & Avenanti, 2011) on tactile perception near the hand, suggesting a causal role of these parts of human PPC in multisensory integration.

The location of hPGR and hPRR relative to topographically organized cortex indicates that over the course of evolution, functionally similar areas that are now located within the IPS in non-human primates were partially relocated outside of the IPS in humans (Fig. 2), possibly in order to accommodate human-specific areas in PPC. The macaque IPS contains areas LIP, VIP, AIP, and portions of PRR. A number of studies have indicated functional similarities between human IPS5 and area VIP in monkeys. Both contain a co-registered, multimodal representation of tactile and visual space, especially near the face (Duhamel, et al., 1998; Sereno & Huang, 2006). Additionally, both show a preference for smooth pursuit eye movements compared to saccades (Konen & Kastner, 2008b; Schlack, et al., 2003), as well as selectivity for motion including optic flow patterns (Konen & Kastner, 2008b; Schaafsma, Duysens, & Gielen, 1997). In monkeys, area AIP is located anterior to VIP (Colby, et al., 1993b; Taira, et al., 1990). Similarly, in humans, hPGR is located anterior to IPS5 (Konen, et al., 2013; Mruczek, von Loga, & Kastner, 2013), suggesting that the spatial relationship known in monkeys between AIP and VIP is preserved. However, hPGR is located outside of the IPS, at the junction of the IPS and the post-central sulcus.

Previous studies have also suggested a functional correspondence between macaque LIP and human IPS1/2. For example, both in humans and monkeys, these IPS regions show activity related to spatial attention (Colby & Goldberg, 1999; Lauritzen, D’Esposito, Heeger, & Silver, 2009; Silver, et al., 2005; Szczepanski, et al., 2010), saccadic eye movements (Konen & Kastner, 2008b; Snyder, Batista, & Andersen, 2000), working memory (Gnadt & Andersen, 1988; Killebrew, Mruczek, & Berryhill, 2015; Schluppeck, Curtis, Glimcher, & Heeger, 2006; Sheremata, Bettencourt, & Somers, 2010), and object stimuli (Janssen, Srivastava, Ombelet, & Orban, 2008; Konen & Kastner, 2008a; Mruczek, et al., 2013; Sereno & Maunsell, 1998), as will be elaborated in the next section. Additionally, human IPS2 and the ventral portion of macaque LIP show similar topographic representations of visual space relative to more posterior regions (Arcaro, et al., 2011; Ben Hamed, et al., 2001). In humans, IPS1/2 span across both banks of the IPS, whereas monkey LIP is confined to the lateral bank of the IPS (see also, Koyama, et al., 2004). In monkeys, area MIP and V6A form the functionally defined PRR and are located posterior and medial to LIP (Andersen & Buneo, 2002). In humans, the relative position of PRR to LIP is preserved, but hPRR has been relocated outside of the IPS and into the SPL. Consequently, hPRR, similar to hPGR, appears to be relatively distinct from the topographic regions in the IPS (Konen, et al., 2013).

The comparison of macaque and human IPS suggests that IPS3/4 may be candidate areas that contribute to the large increase in human parietal cortex relative to the macaque (Brodmann, 1909; Van Essen, et al., 2001). These areas may be human-specific; that is, there may be no functional equivalent of these areas in the macaque monkey. Little is known about their functional response properties. Simon et al. (2002) found that calculation and language-related tasks activated the lateral bank of the IPS and that this activation was independent from that observed for grasping, pointing, saccades or attentional shifts. Thus, these IPS areas may support advanced visuo-spatial skills that are associated with human-specific abilities such as abstract number representation and calculation (Dastjerdi, Ozker, Foster, Rangarajan, & Parvizi, 2013; Dehaene, Dehaene-Lambertz, & Cohen, 1998; Nieder & Dehaene, 2009; Piazza, Pinel, Le Bihan, & Dehaene, 2007). However, it is not yet clear to what extent regions for number representation and calculation overlap with the topographic maps, leaving open the possibility that IPS3/4 are not newly evolved areas, but instead that during human evolution other regions of the IPS split into multiple subregions with different functions. These alternative possibilities need to be tested in future studies.

In summary, it appears that, over the course of evolution, the organization and functional contributions of the IPS has diverged in macaques and humans since their last common ancestor. Specifically, putatively homologous regions in macaque posterior-medial (LIP, PRR/MIP/V6A) and anterior-lateral (AIP, VIP) IPS are spatially separated in the human PPC, presumably to accommodate expanses in parietal cortex that support human-specific functions. Notably, in the human, unique networks have emerged that are associated with human-specific abilities, such as the tool network (Mruczek, et al., 2013), that do not have counterparts in macaques. However, before discussing the tool network, we first review the representation of object information in primate PPC more generally.

Representation of object information

In monkeys, object-related activity has been typically found in the context of visually-guided actions such as grasping, specifically in AIP (Murata, et al., 2000; Tsutsui, Jiang, Sakata, & Taira, 2003). However, object-related activity has also been reported under passive viewing conditions, thus independent of action planning. For example, Sereno and Maunsell (1998) were the first to report 2D shape selectivity in area LIP (Sereno & Maunsell, 1998). However, shape-selective neurons in LIP rarely exhibited size- and position-invariance (Janssen, et al., 2008), suggesting that the shape representation in the dorsal stream is fundamentally distinct from that in the ventral visual stream. In addition to LIP, neurons in AIP also show significant selectivity for the contours of 2D objects (Romero, Van Dromme, & Janssen, 2012) and to simple shape features or fragments of objects (Romero, Pani, & Janssen, 2014; Romero, Van Dromme, & Janssen, 2013). Similar to LIP, the fragment selectivity in AIP does not show tolerance to stimulus position (Romero, et al., 2014).

In addition to 2D shape selectivity in LIP and AIP, a series of single-cell and fMRI studies in monkeys (Durand, et al., 2007; Joly, Vanduffel, & Orban, 2009) as well as imaging experiments in humans (Georgieva, Peeters, Kolster, Todd, & Orban, 2009) have suggested that CIP (see Fig. 1A) is sensitive to the 3D depth structure of objects. In monkeys, CIP neurons can signal the 3D orientation of large planar surfaces when either disparity or texture gradients are used as depth cues (Tsutsui, Sakata, Naganuma, & Taira, 2002). CIP neurons can also be selective for concave and convex surfaces defined by disparity (Katsuyama, et al., 2010), and jointly encode the tilt and slant of large planar surfaces (Rosenberg, Cowan, & Angelaki, 2013). In addition to CIP, AIP neurons in monkeys also respond selectively to concave and convex surfaces that have identical contours (Srivastava, Orban, De Maziere, & Janssen, 2009). It has been proposed that depth structure information of both objects and the environment is first extracted from disparity and texture in CIP, and is then transmitted to AIP (Nakamura, et al., 2001; Orban, 2011; Sakata, Tsutsui, & Taira, 2005; Tsutsui, Taira, & Sakata, 2005). In humans, fMRI experiments using identical stimuli and paradigms as those used in the monkey studies revealed that the 3D shape network is very similar to that in monkeys (Georgieva, et al., 2009). Depth perception in humans consistently evokes strong BOLD responses in dorsal visual areas, particularly V3A, IPS0/V7, and PPC.

Taken together, the functions of 2D and 3D object information in the monkey dorsal pathway are still poorly understood. One might argue that LIP serves as a relay conveying mainly 2D object information to AIP, whereas CIP serves as a source relaying 3D object information to AIP. This object information then is utilized to mediate visuo-motor transformations of AIP neurons involved with grasping movements.

In humans, there is mounting evidence for the representation of considerably more complex object information in dorsal extrastriate cortex and PPC (Jeong & Xu, 2016; Konen & Kastner, 2008a; Xu & Jeong, 2015). Konen et al. (2008a) used a parametric fMRI adaptation paradigm to investigate neural representations of different types of object stimuli such as 2D- and 3D-objects, line drawings of objects and tools, as well as size- and viewpoint-invariance of these object categories in areas of the dorsal pathway. Furthermore, they compared responses directly across both the ventral and dorsal pathways in the same study participants. Adaptation is a robust phenomenon in which repeated presentations of the same visual stimulus leads to gradual response reductions as a function of repetition frequency, thereby providing an indirect measure of an area’s selectivity for certain visual stimuli. This measure can then be quantified by an adaptation index, with positive values indicating selectivity for a given stimulus (see Fig. 4 for a replication of these results from a follow-up study, which is described in detail below).

Figure 4. Object selectivity in the human and monkey visual systems.

In a cross-species comparison study (Kastner, et al., 2009), an adaptation index was used to quantify the degree of object selectivity in each area. Positive index values indicate stronger adaptation and thus greater object selectivity; values around zero indicate the absence of adaptation effects/object selectivity. Error bars denote SEM. Asterisks (*) indicates that the adaptation index value significantly different from zero. For both humans (A), replicating the results of Konen and Kastner (2008a) and monkeys (B), left panels show results from experiments probing object selectivity and right panels show results from experiments probing size- and viewpoint-invariance. Human ROIs were topographically and/or functionally defined using standard criteria. Monkey ROIs were anatomically defined using the Saleem and Logothetis (2007) atlas. Both species showed significant object-selective and viewpoint-invariant responses in ventral stream regions (e.g., LOC in humans; TEO and TE in monkeys). Both species also showed significant object-selective responses in dorsal stream regions (e.g., IPS0–2 in humans; LIP and VIP in monkeys). However, high-level, viewpoint-invariance responses in the dorsal stream were only observed in humans (e.g., IPS1–2 in humans).

In the human ventral pathway, Konen et al. (2008a) found evidence corroborating earlier reports showing object-selectivity in LOC for stimulus types such as photographs, line drawings, and 3D-objects (see also Fig. 4A, left; Grill-Spector, et al., 1999; Kourtzi, Erb, Grodd, & Bulthoff, 2003; Kourtzi & Kanwisher, 2001; Malach, et al., 1995). They were also able to replicate the finding that the LOC response is invariant to image transformations such as size, position, and viewpoint (see also Fig. 4A, right; Grill-Spector, et al., 1999; Kourtzi, et al., 2003; Sawamura, et al., 2005; Vuilleumier, et al., 2002) suggesting that this area is part of a recognition system that subserves perceptual object constancy (Riesenhuber & Poggio, 2002). In contrast, at intermediate stages of processing within the ventral pathway, in area V4, a different response pattern was observed. V4 responded selectively to a variety of object stimuli, but these responses depended specifically on object size and viewpoint (see also Fig. 4A). The dissociation of neural responses with respect to viewpoint and size between V4 and LOC is in agreement with studies in monkeys showing that neurons at successive stages of the ventral pathway encode progressively more complex attributes of an object, and exhibit response properties invariant to image transformations in IT (Booth & Rolls, 1998; Desimone, et al., 1984; Gross, et al., 1972; Sawamura, et al., 2005). These findings therefore provided further evidence to support the notion of a hierarchical organization of neural representations related to object information in the ventral pathway of primates (Riesenhuber & Poggio, 2002).

Surprisingly, Konen et al. (2008a) found a similar hierarchically organized system of object representations in the human dorsal pathway. As in the ventral pathway, areas at intermediate stages of cortical processing such as V3A, MT, and IPS0/V7 showed size- and viewpoint-specific responses, whereas high-level areas in PPC such as IPS1/2 showed size- and viewpoint-invariant responses (see also Fig. 4A). These object-related neural representations appeared to generalize across multiple types of stimuli including 2D- and 3D-objects, and line drawings of objects and tools. Additionally, these areas did not distinguish general objects from tools, but responded similarly to them (see also Mruczek, et al., 2013). No object-selective responses were found in the anterior/lateral IPS (IPS3/4) implicating functional differences across topographically organized areas of PPC. Overall, object representations in the dorsal system appeared to mirror those of the well-known ventral system in terms of basic response properties such as category selectivity and invariance to image transformations.

A number of characteristics of object representations in the dorsal pathway are worth noting. First, the object responses were found for abstract 2D images, under passive viewing conditions, and even when attention was drawn away from the stimuli. Thus, these object representations are active even when the behavioral context does not require concurrent action planning. Rather, these data provide strong evidence that object information is represented in the human dorsal pathway independent of action planning, although this information would likely be utilized for such purposes, when required. Second, the dorsal object-related activity appeared indistinguishable from that in the ventral pathway and was not specific to changes of viewpoint and size but rather generalized across changing viewing conditions. While it is widely agreed upon that a successful object recognition system requires generalization across changing viewing conditions (Riesenhuber & Poggio, 2002), one would predict that a system that uses object information for the control of skilled actions would preserve the specificity of object appearance with regard to shape, size and orientation (Craighero, Fadiga, Umilta, & Rizzolatti, 1996). For example, the same object when presented from different viewpoints will require different hand postures during grasping (James, et al., 2002). The finding that object-related activity generalizes across image transformations, therefore, does not support this view. Alternatively, view-independent object information in the dorsal pathway may serve more sophisticated forms of object manipulation than simple sensory-motor transformations in visually-guided behavior. For example, such representations may be critical in guiding the placement of the hand and fingers on occluded regions of an object or predicting how an object could be reoriented during manipulation.

The finding of an elaborate second dorsal object vision system in humans did not seem compatible with existing monkey physiology literature, as reviewed above. These findings were therefore met with considerable skepticism. One possibility is that the results were not compatible due to differences in approach. For example, studies in humans and monkeys typically use different methods – mainly intracranial electrophysiology in monkeys and functional brain imaging in humans – and experimental designs that vary considerably. Furthermore, monkey neurophysiology often focuses on differences within a small number of subjects, whereas human neuroimaging often focuses on group-averaged data. Finally, studies of object vision often utilize stimuli drawn from object categories that carry considerable meaning to humans (e.g., tools, clothing, vehicles), even if the specific exemplars and images used are novel.

To address these concerns, Kastner, Konen and colleagues performed a follow-up study in which an identical experimental approach and design was used in the two primate species (Kastner, Pinsk, Arcaro, Li, & Konen, 2009). Specifically, they used the same fMRI adaptation paradigm as in Konen et al. (2008a) while acquiring fMRI data in a second group of human participants and in two monkeys trained to maintain fixation over extended periods of times (i.e., for several minutes) in the scanner environment. In addition to using a similar data collection technique for both species, the study used a region-of-interest analysis that focuses on effects in single-subjects. Finally, they used a subset of the stimulus types previously used in Konen et al. (2008a) that had no semantic or categorical attributes, specifically the abstract 2D and 3D objects..

In humans, Kastner, Konen and colleagues replicated the pattern of results regarding the neural representation of object information in the ventral and dorsal pathway described above (Fig. 4A; Konen & Kastner, 2008a). In contrast, in monkeys (Fig. 4B), they found that while the ventral pathway was similarly organized in representing 2D- and 3D-objects independent of viewing conditions, the organization of the dorsal pathway was species-specific: areas at intermediate levels of visual processing (e.g., MT, V3A) did not carry object information, whereas areas at advanced levels of processing such as LIP and VIP showed shape selectivity that reflected viewpoint-specific, low-level features of objects. These studies are among the very few that demonstrate a difference in the functional organization of a particular response property in the primate brain. Furthermore, these results not only reconcile a wealth of knowledge from monkey electrophysiology, but also indicate that from an evolutionary perspective, the human dorsal object vision system appears to be a recent development and may have evolved to enable human-specific behaviors related to PPC such as sophisticated tool use. We elaborate on this hypothesis in the final section of this review. But first, we take a closer look at the human tool network.

Uniquely human functions: The tool network

Thus far, we have reviewed the neural representation of object information in the dorsal visual pathway and PPC in general terms. However, with respect to human parietal cortex function, there is one object category that deserves special consideration: tools. While tool use and even tool making are not unique to humans – apes and crows use sticks to forage for insects, to give just two examples (see Shumaker, Walkup, & Beck, 2011 for many more) – humans appear to be the only sophisticated tool users and makers in the animal kingdom. One important distinction is that humans thoroughly understand the causal impact that an object used as a tool has on a different object or on the environment. Remarkably, humans also seamlessly transform ordinary objects into tools. For example, when a ball rolls under a cabinet, thereby getting out of reach for a two year old child, he or she will use a stick or any other elongated object to extend the radius of the arm to reach the ball. This is a remarkable ability that illustrates the potential and behavioral flexibility of the ‘human toolmaker’ from early development onwards.

Human tool use ability is reflected in the evolution of a dedicated network that represents information about tools. In parietal cortex, neuroimaging studies have shown that viewing pictures of tools, which have strong and specific action associations, activates a left-lateralized network of brain regions including the anterior IPS and regions in the inferior parietal lobe, along with regions in frontal and temporal cortex (Chao & Martin, 2000; Chouinard & Goodale, 2012; Lewis, 2006; Valyear, Cavina-Pratesi, Stiglick, & Culham, 2007). The same left-lateralized network has been associated with viewing, hearing, executing, planning and pantomiming tool use (Lewis, 2006). Importantly, a functionally equivalent tool region is not found in macaques, even after extensive training with tool usage (Peeters, et al., 2009).

Tool-related activity has been consistently found lateral, anterior, and inferior to IPS5 and lateral and inferior to hPGR, with some partial overlap of all three regions (Mruczek, et al., 2013; Valyear, et al., 2007). Importantly, this activity appears to be specific to tools; graspable objects other than tools do not activate these regions as strongly. Tool-evoked activity is found throughout most regions of the IPS, especially in the left hemisphere. However, the response evoked by the presentation of graspable objects and tools is similar in posterior regions (Konen & Kastner, 2008a; Mruczek, et al., 2013), thus not discriminating between these two categories of objects. Posterior IPS regions also do not discriminate between action-related or non-action related objects. In contrast, responses to non-action related objects are absent in anterior IPS (Fig. 4A), and graspable objects evoke a weaker response than tools in anterior regions (Mruczek, et al., 2013). One interpretation of this pattern of results is that the posterior IPS encodes basic visual features common to any graspable object (of which tools form a subset), whereas the anterior IPS integrates this information with stored knowledge of action associations, affordances, and goals based on one’s personal and evolutionary experience with tools.

Such an account is also corroborated by studies in patients with lesions in regions of the parietal tool network. Lesions of the left posterior parietal cortex can lead to ideomotor apraxia (Buxbaum, Johnson-Frey, & Bartlett-Williams, 2005; Buxbaum, Kyle, & Menon, 2005; Johnson-Frey, 2004), in which patients maintain the ability to grasp and manipulate objects, but fail to grasp tools in a manner that is consistent with their function (Buxbaum, Sirigu, Schwartz, & Klatzky, 2003; Goldenberg & Hagmann, 1998; Sirigu, et al., 1995). Consistent with the presence of the tool-specific region identified by neuroimaging studies, this indicates a dissociation between the neural mechanisms that support grasping an object (i.e., sensorimotor transformations) and those that support using an object (Binkofski & Buxbaum, 2013; Johnson-Frey, 2004). In some cases, patients fail to demonstrate (i.e., pantomime) appropriate tool usage, even though they maintain the ability to recognize tools and describe their function (Daprati & Sirigu, 2006; Johnson-Frey, 2004). Based on a quantitative analysis of lesion sites and behavioral deficits, Goldenberg and Spatt (2009) concluded that left posterior parietal cortex is necessary for understanding general principles of tool usage and mechanical interactions between objects, rather than prototypical uses or specific grips or movements associated with tools.

A hypothesis on brain adaptations for the ‘human toolmaker’

Based on a comparative review of the response properties of monkey and human PPC, as detailed above, in this final section we propose a hypothesis regarding the evolution of the human-specific tool network. The most extensively studied primate brain model that we used as a basis for discussion in this review, the macaque monkey, is not a tool user in natural environments. Although monkeys can be trained on tool use (Ishibashi, Hihara, & Iriki, 2000; Ishibashi, et al., 2002), they will not transfer or generalize the learned use to similar object manipulations. However, it is conceivable to reason that the non-spatial and viewpoint-specific object information in monkey PPC is utilized for informing visually-guided object manipulations. The circuit that subserves such function in monkey PPC can be outlined as follows. Both area LIP, which carries information for 2D aspects of objects, and area CIP, which carries information about 3D aspects of objects, project to area AIP. The object representations in these areas may provide necessary information for the manipulation and grasping of objects executed by a parieto-frontal motor network that includes AIP. Importantly, area LIP has also been implicated in a wide variety of cognitive functions including spatial attention, decision making, categorization, and working memory (e.g., Colby & Goldberg, 1999; Freedman & Assad, 2006; Kusunoki, Gottlieb, & Goldberg, 2000; Shadlen & Newsome, 1996, 2001). In this respect, LIP can be considered a cognitive hub that may integrate object information with other cognitive processes relevant to ongoing behavior and provide such integrated object information to area AIP in order to be implemented with motor plans and ultimately to mediate visually-guided behavior. Importantly, LIP also projects directly to VIP, where sensory responses have a bias for objects that are near or approaching the body and stimulation of VIP results in defensive-like movements (Cooke, Taylor, Moore, & Graziano, 2003; Graziano & Cooke, 2006). A major function of area VIP is to maintain a margin of safety around the body and to coordinate action that defend the body surface (Graziano & Cooke, 2006). In this respect, object information in VIP may be useful to represent objects in peripersonal space and to integrate them into the body schema (Ishibashi, et al., 2002; Obayashi, et al., 2001). For example, physiology studies have shown that VIP neurons may expand their receptive fields to include manipulable objects extending from the hand (Obayashi, et al., 2001). Taken together, we propose that non-spatial object information in areas LIP, CIP and VIP of the macaque constitutes an integral part of a fronto-parietal perception-to-action system aimed at manipulating objects in peripersonal space.

This account is consistent with the role of the dorsal pathway in the framework of the perception-action hypothesis of primate vision. How did this network for perception-to-action become transformed in humans in order to support the human toolmaker? We propose four adaptations, as detailed below.

1. Formation of a second dorsal object vision system

While viewpoint-specific representations of 2D- and 3D-aspects of objects may be sufficient for object manipulations involving the fronto-parietal motor system such as grasping a cup or a piece of food, they may not be sufficient to support the ability to manipulate tools flexibly and, importantly, to build them. The sophistication that is observed in human tool use may require highly flexible object representations that can get transformed in varying behavioral context. While the ventral object vision pathway is directed at integrating such object information with memory systems for recognition, the human dorsal object vision pathway may have emerged to integrate such information for flexible tool use. Importantly, these object representations emerge in similar regions as in the macaque suggesting that an expansion of functional properties in human IPS1/2, an area similar to monkey LIP, may be one of the prerequisites for developing a specific tool network. In our account, these areas provide direct input to IPS5 and the tool area (see Fig. 2). Importantly, this input system would need to represent high-level object information in flexible ways. However, it should not be specialized for representing tool information, so that other object- and cognitive-based information can be integrated. As reviewed above, the response properties of human posterior PPC indeed reflect these predictions.

2. Integration of object information with cognitive information, particularly related to working memory

The ability to interact with tools and particularly to build them requires that information about action sequences is not only integrated with information about object properties, but also that it can be maintained over some period of time. Accordingly, one would predict that the same areas that advance functionality to represent object information would also advance cognitive function, particularly with respect to working memory. Indeed, much evidence has demonstrated parietal mechanisms important for visual working memory ¹. In monkeys, spatial working memory, in particular, has been readily demonstrated in LIP neurons. For example, spatially-specific delay activity has been found in an oculomotor delayed response task in which monkeys were trained to make a saccade to a remembered location after a delay (Fig. 5A–B; Chafee & Goldman-Rakic, 1998; Gnadt & Andersen, 1988). While spatial working memory function has also been demonstrated in human IPS (Courtney, Ungerleider, Keil, & Haxby, 1996; Jonides, et al., 1993; Killebrew, et al., 2015; Schluppeck, et al., 2006; Sheremata, et al., 2010), the posterior IPS regions, possibly including IPS1/2, appear to be one, if not the, major hub for working memory more generally in the human brain. This is best exemplified by the seminal work of Todd and Marois (2004). Using fMRI they found a strong correlation between fMRI activity in posterior IPS and the behavioral working memory capacity to encode visual items of individual participants (Fig. 5C–D). Such correlation was not found in an iconic memory task that did not require maintenance of visual information and not in other parts of the brain (notably not in PFC) suggesting that posterior IPS activity reflects working memory storage and is a major hub for this cognitive function in the human brain.

Figure 5. Working memory in primate parietal cortex.

(A) Oculomotor delayed response (ODR) task: Monkeys were required to maintain fixation and encode the location of a cue presented in the periphery. After fixation offset, monkeys made a saccade to the remembered location. (B) Delay period activity of an example LIP neuron in the ODR task. The center panel indicates eight locations at which a cue could appear and each raster plot indicates the resulting neural activity in response to the cue (C) and during the delay period (D). The LIP neuron showed elevated activity during the cue and delay period at the preferred receptive field location (90 degree). Panels A and B reproduced with permission from Chafee and Goldman-Rakic (1998). (C) Visual working memory task: Participants encoded colors in a sample display while rehearsing two digits. After a delay, participants responded whether the color of the probe item matched the remembered colors. (D) Correlation between behavioral working memory capacity and fMRI signals in IPS. The black and red solid lines indicate behavioral working memory capacity (Cohen’s K) and fMRI response amplitudes during a visual short-term memory task (VSTM) at each set size, respectively. FMRI signals and behavioral working memory capacity were strongly correlated. However, an iconic memory task (IM) that used the same sample display without working memory requirement did not show such correlation (indicated by the red dotted line). Panels C and D reproduced with permission from Todd and Marois (2004).

Berryhill and Olson (2008) extended these results by demonstrating causal evidence that PPC is involved in working memory. They found that patients with right PPC damage showed impaired performance in both spatial and object visual working memory tasks. Taken together, these findings support strong evidence for advanced cognitive functionality with respect to visual working memory in the human posterior PPC. Interestingly, this functionality is observed in the regions that also carry advanced object representations, thereby allowing for integration of perceptual and cognitive function in posterior PPC. This may be another prerequisite for the formation of an efficient tool network. According to our model, information relevant for tool use and manipulation will be input from the posterior IPS to IPS5 and the human tool area. Together with hPGR (and the more extended fronto-parietal action system), these regions form an integrated large-scale circuit supporting tool use and manipulation (see Fig. 2).

3. Tool-specific responses in IPS5

Despite the tool area, the most anterior region in the IPS, IPS5, also exhibits strong and selective responses to viewing tools (Mruczek, et al., 2013). As mentioned earlier, IPS5 is located just posterior to hPGR (Konen, et al., 2013) and posterior, medial and superior to the tool-specific region (Mruczek, et al., 2013). However there is some partial overlap between all of these regions and they may act as a coordinated network to guide sophisticated object interactions.

We alluded earlier to the functional correspondence between human IPS5 and area VIP in monkeys (Konen & Kastner, 2008b; Silver & Kastner, 2009). The tool specificity in IPS5 is consistent with the idea that neural processes in macaque VIP provided the evolutionary foundation for the tool-specific circuit in humans. Interestingly, evidence from monkey physiology (Iriki, Tanaka, & Iwamura, 1996), human psychophysics (Maravita, Spence, Kennett, & Driver, 2002) and human neuropsychology (Berti & Frassinetti, 2000) shows that tool usage dynamically modifies the brain’s internal “body schema”, a representation of posture and positioning in space that is hypothesized to be located in the PPC. Tools extend the internal body schema to include the spatial regions around or reachable by the tool, which suggests an integration of the tool as an effector for actions (Graziano & Botvinick, 2002; Maravita & Iriki, 2004).

4. The formation of category-specific tool regions

We have reviewed the evidence of tool-specific regions in left parietal cortex in a previous section. In our account, the tool-specific regions of the anterior IPS integrate object and cognitive information from the posterior IPS and information from IPS5 with respect to the body schema with experience-dependent knowledge of action associations, affordances, and goals, which are uniquely linked to tools.

The anterior parietal tool-specific region is an integral part of two other cortical networks. First, the human tool network includes regions of inferior temporal and medial temporal cortex. These regions have been implicated in conceptual and semantic knowledge related to tools and actions (Lewis, 2006). Regions of the ventral tool network likely interact with the parietal tool region (Lewis, 2006) and may provide one way in which object information from the dorsal and ventral streams interact. Second is the fronto-parietal action-to-perception system, of which hPGR is a part as well. Importantly, parts of this integrated network will also serve action recognition. The formation of an action recognition system may be another principal stepping stone towards the formation of a tool network. However, reviewing the extensive literature on this topic is beyond the scope of this review.

Conclusion

The primate parietal cortex underwent significant transformations during evolution, anatomically as well as functionally. While we are only at a beginning understanding of the functional transformations in the primate lineage, it appears that the significance of spatial processing that involves the dorsal visual pathway including PPC is preserved in human and non-human primates. In contrast, non-spatial information appears to be represented with greater complexity in humans. For example, extrastriate dorsal regions exhibit viewpoint- and size-dependent shape and object selectivity, whereas higher-order areas in PPC represent object information regardless of changes in viewpoint and size in humans, but not in monkeys. Thus, the human dorsal vision system shows a response profile indicative of a flexible object recognition system with the potential to contribute to a variety of sophisticated, visually-guided, object-directed behaviors. This expansion in functional repertoire of human posterior PPC may be paralleled by an expansion of cognitive function as well. An example for such expanded functionality is working memory. Working memory may be instrumental when deriving motor and action sequences directed at complex interactions with objects in the world such as tools. Remarkably, the evolutionarily novel and uniquely human tool network is located to a large extent in parietal cortex. Based on a comparative analysis of monkey and human PPC literature, we propose that the functional expansion of PPC, in its contribution to the representation of non-spatial information and enhanced cognitive function, reflect prerequisites for developing a highly refined tool network.

Highlights.

• The dorsal object system in humans and monkeys supports action guidance.

• The human dorsal object system mirrors the complexity of the ventral stream.

• The human dorsal object system is integrated with working memory systems.

• The human dorsal object system contains unique tool-specific representations.

• These may support the human toolmaker’s sophisticated object-directed behavior.