Reciprocal Interactions between Parietal and Occipito-Temporal Representations Support Everyday Object-Directed Actions

Abstract

Everyday interactions with common manipulable objects require the integration of conceptual knowledge about objects and actions with real-time sensory information about the position, orientation and volumetric structure of the grasp target. The ability to successfully interact with everyday objects involves analysis of visual form and shape, surface texture, material properties, conceptual attributes such as identity, function and typical context, and visuomotor processing supporting hand transport, grasp form, and object manipulation. Functionally separable brain regions across the dorsal and ventral visual pathways support the processing of these different object properties and, in cohort, are necessary for functional object use. Object-directed grasps display end-state-comfort: they anticipate in form and force the shape and material properties of the grasp target, and how the object will be manipulated after it is grasped. End-state-comfort is the default for everyday interactions with manipulable objects and implies integration of information across the ventral and dorsal visual pathways. We propose a model of how visuomotor and action representations in parietal cortex interact with object representations in ventral and lateral occipito-temporal cortex. One pathway, from the supramarginal gyrus to the middle and inferior temporal gyrus, supports the integration of action-related information, including hand and limb position (supramarginal gyrus) with conceptual attributes and an appreciation of the action goal (middle temporal gyrus). A second pathway, from posterior IPS to the fusiform gyrus and collateral sulcus supports the integration of grasp parameters (IPS) with the surface texture and material properties (e.g., weight distribution) of the grasp target. Reciprocal interactions among these regions are part of a broader network of regions that support everyday functional object interactions.

Introduction

Perception and action iteratively work together to support activities of daily living. Perception-action interactions occur at multiple time scales—from the scale of ‘driving to work’ or ‘making dinner’, to the intermediate scale of complex actions such as ‘opening a door’ or ‘taking a bite of pasta’, to the fine-grain scale of calibrating grasp parameters (‘grasping the handle of the fork’) (Botvinick, 2008; Pezzulo & Cisek, 2016). Here we focus on perception-action iterations that unfold at the intermediate and fine-grained levels: the time scale of milliseconds to seconds during individual object-directed actions. This covers the period from when an action goal is first represented, through when visual information about a potential grasp target enters the eye, to the moment the object is first contacted with the hand in a grasp, through the subsequent manipulation that is applied to the object to meet the action goal (e.g., grasping a fork to take a bite of food).

Consider the following everyday scenario of grasping one’s fork at the dinner table. That grasp will be constrained by the shape of the fork, the identity of the object itself (i.e., that it is a fork and not a knife), the material the object is made of (i.e., a metal or plastic fork) by the presence of other obstacles (e.g., a wine glass on the table), and importantly by what is to be done with it once it is in hand. If one is grasping the fork to pass it to someone, the initial grasp is very different than if the fork is being grasped to bring food to the mouth. Object-directed grasps anticipate (in location, hand posture, and grip force) what will be done with the object after it will have been grasped. Object directed grasps also anticipate properties of the object that is to be grasped, such as its temperature, how slippery it might look, and its expected weight distribution. The calibration of initial object-directed grasps to object representations and action goals is ‘end-state-comfort’ (Creem & Proffitt, 2001; Rosenbaum, Vaughan, Barnes, Marchak, & Slotta, 1990).

Here we refer to grasps that display end-state-comfort as ‘functional’ grasps: what makes a grasp a functional grasp is that the form and force of the grasp is constrained by object properties in interaction with the broader action goal. Viewed in this way, in naturalistic or everyday behavior, just about all object-directed actions are functional and display end-state comfort (Creem & Proffitt, 2001). When was the last time you grasped an object with no intention to do anything with it after it was grasped? Even grasping a handle to ‘hold on’ implies grasping it in a way that will support ‘holding on’—one does not use a precision grasp to ‘hold on’ to a bus handle.

These considerations frame the following question: How does a semantically rich interpretation of the visual scene and an agent’s action goals interact with the systems that bring the hand from its current position to the correct part of the target object with the appropriate grip posture and strength? We sketch an answer to this question within a framework for thinking about interactions between parietal and occipito-temporal regions. Our argument has the following form:

• The default situation ‘in the wild’ is that objects are selected as action targets because of how their functional use can facilitate broader behavioral goals. What will be done with the object, once it is ‘in-hand,’ depends on the broader action goal driving behavior and by perceptual and conceptual information about the target object that is processed by the ventral stream. Action goals and conceptual representations of objects and their properties are represented prior to the deployment of the initial grasp.

• Representation of object properties having to do with surface texture, material composition and weight are supported by ventral occipito-temporal regions (collateral sulcus, fusiform gyrus, lingual gyrus). The center of mass of an object and the anticipated coefficient of friction at the grasp location drive where and how firmly to grasp an object (M. Davare, Kraskov, Rothwell, & Lemon, 2011). Processing those perceptual properties is a part of anticipating how an action will feel throughout its execution—which informs a forward model that supports motor planning. We frame the hypothesis that interactions between visuomotor grasp processes in the dorsal visual pathway (IPS) with surface and object material properties (collateral sulcus) allow the system to calibrate the form and force of a grasp to the correct part of the target object. This interaction also gives rise to neural specificity for manipulable objects in the medial ventral stream.

• The supramarginal gyrus of the left inferior parietal lobule supports praxis representations (what is done with the object, once it is ‘in-hand’ (Gonzalez Rothi, Ochipa, & Heilman, 1991; Johnson-Frey, 2004; Tarhan, Watson, & Buxbaum, 2015). Those praxis representations are a pre-requisite for grasps to display end-state-comfort: what will be done with the object once it is in hand constrains the initial grasp. But what will be done with the object once it is in hand depends on the identity of the object to be grasped, its perceptual properties (material, texture, object parts, etc), as well as the broader goal of the action. If, as we argue, dorsal stream regions represent grasp targets in the absence of a conceptual interpretation of what the object is, accessing the correct praxis representations in the supramarginal gyrus must depend on temporal lobe inputs that convey object information. We frame the hypothesis that interactions between praxis representations (supramarginal gyrus) and conceptual representations of objects and actions (posterior middle temporal gyrus) allow the system to access the correct praxis representation for the correct object.

This approach for thinking about the neural systems that support object directed actions centers connectivity and interaction among dissociable regions in parietal and occipito-temporal cortex in support of everyday actions. The model we frame draws on prior discussions in the literature that have emphasized interactions between object and action representations in support of goal-directed behavior (e.g., Borra et al., 2008; Gallivan and Culham, 2015; Hutchison and Gallivan, 2013; Goodale et al., 1994(Binkofski & Buxbaum, 2013; Buxbaum, Schwartz, & Carew, 1997)). We hope to motivate several new constraints that apply to this network, and to frame new empirical predictions about the time course of reciprocal interactions within this network.

End-state-comfort as an index of dorsal and ventral visual stream interactions

The seminal work of Ungerleider and Mishkin (1982) and Goodale and Milner (Goodale & Milner, 1992) framed the proposal that visual processing can be separated into dorsal and ventral visual cortical pathways (see also (Livingstone & Hubel, 1988; Merigan & Maunsell, 1993)). The ventral object processing stream supports the derivation of stable object properties from visual input such as visual form, texture and color, and interfaces visual information with a semantic interpretation. The dorsal stream processes visual inputs without access to a semantic interpretation of the visual scene, translating real-time visual inputs (location, orientation, structure) into motor parameters in egocentric reference frames, as relevant to current action. The ventral visual stream projects anatomically from primary visual cortex (V1) to ventral occipito-temporal regions, whereas the dorsal visual stream projects into dorsal occipital and posterior parietal cortex via geniculo-striate, geniculo-extrastriate (Koniocellular) (Sincich, Park, Wohlgemuth, & Horton, 2004) and extra-geniculostriate (Magnocellular) pathways (Lyon, Nassi, & Callaway, 2010).

Within the ventral stream, an important gross distinction is between the ventral surface of occipito-temporal cortex and the lateral surface of occipitotemporal cortex. Posterior ventral occipito-temporal areas (medial fusiform gyrus, collateral sulcus, and lingual gyrus) support computations about color and surface texture (Cant, Arnott, & Goodale, 2009; Cant & Goodale, 2007, 2009; Miceli et al., 2001; Simmons et al., 2007; Stasenko, Garcea, Dombovy, & Mahon, 2014) while more anterior regions along the collateral sulcus support computations about material composition and the weight of objects (Gallivan, Cant, Goodale, & Flanagan, 2014); for work in non-human primates, see (Koteles, De Maziere, Van Hulle, Orban, & Vogels, 2008). Lesions that involve the lingual gyrus and collateral sulcus are associated with achromatopsia when they involve more posterior segments and color agnosia when they involve more anterior aspects (Miceli et al., 2001; Stasenko et al, 2014; for convergence from fMRI, see Cant & Goodale, 2007; Simmons et al., 2007). Patients with achromatopsia have difficulty with color perception and naming but do not necessarily have difficulty with knowledge of the typical colors of things (e.g., knowing that the sky is blue). By contrast, patients with color agnosia can be spared for color perception but impaired for knowledge that (for instance) the grass is green, the sky is blue, or that a watermelon is a different color on the inside than on the outside (see also Siuda-Krzywicka et al., 2019). One well-studied patient with a lesion involving the collateral sulcus demonstrated an impairment for processing surface-texture properties with spared processing of visual form (Cavina-Pratesi et al., 2010).

By contrast, lateral occipito-temporal regions, including the lateral occipital area (LO), support analysis of visual form. LO is classically identified with functional MRI by contrasting activity for intact objects to scrambled versions of those same stimuli (Grill-Spector & Malach, 2004). Damage to lateral occipital-temporal cortex can cause visual form agnosia, leading to profound deficits in visual object recognition and any computations that depend on object form processing, while sparing early visual processing, haptic shape recognition, object grasping, general object knowledge, and processing of color and surface texture (Goodale, Milner, Jakobson, & Carey, 1991; Snow, Goodale, & Culham, 2015).

Within lateral occipito-temporal cortex more broadly, there is specialization for separable visual categories, including body parts, manipulable objects, and of relevance—hands (S. Bracci, Cavina-Pratesi, Ietswaart, Caramazza, & Peelen, 2012; S Bracci & Peelen, 2013); for convergent findings with TMS, see (Pitcher, Charles, Devlin, Walsh, & Duchaine, 2009). Gallivan and Culham (2015) suggest that inputs from parietal action representations to lateral occipital-temporal regions, potentially via the Vertical Occipital Fasciculus (Yeatman et al., 2014) may support a forward model that allows the system to maintain real-time guidance of object-directed actions as they unfold.

Anterior to LO, regions in lateral occipito-temporal cortex, sometimes referred to as posterior middle temporal gyrus, support conceptual processing of actions and events, as well as objects that are the targets of actions (i.e., ‘tools’; (Brambati et al., 2006; Buxbaum, Shapiro, & Coslett, 2014; Kalenine, Buxbaum, & Coslett, 2010; BZ. Mahon et al., 2007; (Lingnau & Downing, 2015; Wurm, Caramazza, & Lingnau, 2017) Tranel, Kemmerer, Adolphs, Damasio, & Damasio, 2003 (Wurm & Caramazza, 2022).

Regions at the junction of dorsal occipital cortex and posterior parietal cortex, including V3A and V6A (Fattori et al., 2010; Georgieva, Peeters, Kolster, Todd, & Orban, 2009; Monaco et al., 2011; Pitzalis et al., 2013) through posterior and superior parietal areas in each hemisphere, support analysis of visual object structure, such as analysis of an object’s principal axis, in the service of object-directed reaching and grasping (J. Chen, Snow, Culham, & Goodale, 2018; Culham et al., 2003; Fabbri, Stubbs, Cusack, & Culham, 2016; Pisella et al., 2000). This includes processing of the graspable status of the stimulus, as well as stimulus elongation and orientation (Almeida, Mahon, & Caramazza, 2010; Almeida, Mahon, Nakayama, & Caramazza, 2008b; Almeida, Mahon, Zapater-Raberov, Dziuba, Cabaco, et al., 2014; J. Chen et al., 2018; Culham et al., 2003; Fabbri et al., 2016; Fang & He, 2005). More medial/superior and posterior parietal regions (SPOC; Gallivan and Culham, 2015 (Rossit, McAdam, McLean, Goodale, & Culham, 2013)) have been argued to support the transport phase of the reach-to-grasp action, also corresponding to control of the arm. By contrast, anterior Intra-parietal Sulcus, aIPS, supports object grasping, corresponding to control of the hand (Binkofski et al., 1998; Culham et al., 2003). Disruption of dorsal stream processing can cause optic ataxia, leading to a deficit in the ability to use visual inputs in real time to drive ballistic object-directed grasping or pointing actions. Optic ataxia is generally more pronounced for the contralesional hand grasping targets in the contralesional hemifield (Desmurget et al., 1999; Jakobson, Archibald, Carey, & Goodale, 1991; Pisella, Binkofski, Lasek, Toni, & Rossetti, 2006). Perenin and Vighetto (1988) reported that unilateral right hemisphere damage disrupts object directed grasping, with the effect particularly pronounced for visual targets located in the left visual field. By contrast, unilateral left hemisphere posterior-parietal lesions can result in a deficit for grasping targets in the right visual field, and can also disrupt accurate grasping with the right hand regardless of the visual field location of the grasp target (see (Bálint, 1909; M. T. Perenin & Vighetto, 1988).

Of particular relevance to the motivation for the framework developed herein, the regions across parietal and temporal-occipital cortex that are individually and collectively necessary for functional object-directed actions are automatically engaged when participants view manipulable objects. Figure 1 shows a univariate contrast map obtained in a group of healthy participants who were sub-vocally naming images of common objects, animals, faces and places. The pattern of findings summarized in Figure 1 is observed in the absence of any intention to act (across naming, perceptual n-back, and incidental fixation tasks).The contrast map highlights regions that exhibit differential neural responses to the visual presentation of manipulable objects compared to a range of baseline categories (faces, animals, places; Chao and Martin, 2000; (Chao, Haxby, & Martin, 1999; Chao & Martin, 2000; Downing, Chan, Peelen, Dodds, & Kanwisher, 2006; Mahon et al., 2007; Noppeney, Price, Penny, & Friston, 2006). The posterior middle (and inferior) temporal gyrus supports object and action-related conceptual representations (Brambati et al., 2006; Tranel, Kemmerer, Adolphs, Damasio, & Damasio, 2003). Left parietal regions, extending along IPS, and involving the supramarginal gyrus, are differentially active when viewing manipulable objects compared to baseline categories (Chao and Martin, 2000). Thus, parietal regions supporting object directed action are engaged by visual presentation of manipulable objects even in the absence of an overt action task. Finally, the contrast map also highlights that manipulable objects lead to differential activity in the anterior collateral sulcus and the medial fusiform gyrus, typically bilaterally, but often stronger in the left (Chao et al., 1999; Mahon et al., 2007).

Figure 1. A network for processing object-directed actions.

Functional neuroimaging and neuropsychological studies of patients with acquired brain lesions support a neurocognitive framework about the anatomical and functional organization of visual object recognition, grasping, and use. The functional MRI contrast map shown in the figure was obtained from healthy participants viewing images of ‘tools’ compared to faces, animals, and places (Q. Chen et al., 2017; F. Garcea et al., 2018). The first description of this functional network was by Alex Martin and colleagues (Chao et al., 1999; Chao & Martin, 2000).

The color-coding in the contrast map aligns the regions within this domain-specific network with proposed local representational types. It is important to emphasize that the regions highlighted in this map, with the exception of lateral occipital-temporal cortex (LO), are differentially engaged when naming tools compared to baseline categories (faces, animals). LO, which supports processing of object form, is defined in the map by contrasting all stimuli (tools, animals, faces, places) against scrambled image baselines. Note that regions involved in linguistic aspects of naming are not highlighted in the map (e.g. left hemisphere perisylvian naming centers). Reproduced from Mahon B (2020). The representation of tools in the human brain. In: D Poeppel, M Gazzaniga (Eds.), The new cognitive neurosciences, sixth edition, MIT Press, Cambridge, MA.

Dorsal stream regions are automatically engaged by graspable stimuli, independently of processing in the ventral stream (Fang and He, 2005). The visuomotor transformations computed by the dorsal stream about grasp targets lack a conceptual interpretation of the target stimulus (Almeida et al, 2014 (Carey, Harvey, & Milner, 1996)). Thus, posterior dorsal stream regions, on their own, compute action-related volumetric properties of objects in support of what could be called ‘afunctional grasps’ – i.e., grasps that (merely) respect volumetric and biomechanical constraints of the target stimuli and the position of the body relative to the grasp target. Visual processing within the dorsal stream, while perceptually rich and perhaps even invariant to some surface transformations (E. Freud, Plaut, & Behrmann, 2016; Konen & Kastner, 2008b; Vaziri-Pashkam & Xu, 2018), is agnostic as to the identity of the object and its function. Shaping the hand in a manner that anticipates subsequent object use implies that object use information is represented prior to the grasp. Furthermore, accessing the correct object use representation presupposes that the object has been identified—which is to say, functional object-grasps that display end-state comfort imply that the visual information about an object has been processed by the ventral stream. In this way, end-state-comfort implies interactions between the dorsal and ventral visual pathways (Creem & Proffitt, 2001; Danckert & Rossetti, 2005; M Davare, Kraskov, Rothwell, & RN, 2011; Goodale, Jakobson, & Keillor, 1994; Pisella et al., 2006; van Polanen & Davare, 2015).

An important source of evidence for the role of the ventral stream in supporting functional object grasps is the study of end-state comfort in patient DF, who had bilateral LO lesions and a dense visual form agnosia. Despite her visual form agnosia, DF can grasp objects in a volumetrically appropriate manner (Goodale et al 1991). Interestingly, when a biomechanically comfortable and volumetrically appropriate grasp is in conflict with a functional grasp, she does not display end-state comfort (Carey et al., 1996). For instance, in a situation in which the handle of a tool is oriented away from a participant, a healthy participant will ‘reach around’ to grasp the object by its handle, even though that grasp is biomechanically uncomfortable. Patient DF fails to display such effects of end-state comfort in those situations. The consequence of DFs lesion is to disrupt her ability to access meaning from vision—which is to say, to process a visual stimulus through her ventral stream.

Functional object-directed grasps require a deep representation of the object that is the target of the action—its identity, function, context of use, and material properties (and so on). None of those properties are represented by the dorsal stream. Another line of neuropsychological evidence that converges with this conclusion is the observation that the calibration of grip aperture in a patient with optic ataxia after bilateral posterior parietal lesions was very poor when grasping unfamiliar novel objects, and more accurate when grasping familiar objects (Jeannerod, Decety, & Michel, 1994). The implication is that the patient’s intact ventral visual pathway was able to provide useful inputs about the expected size of the grasp target, compensating for the functional deficit in bottom-up derivation of grip size from visual input caused by the bilateral posterior parietal lesions.

The Dorsal Visual Stream is one of Several Parietal-Based Processes That Support Functional Object Use

The ‘dorsal visual pathway’, as attributed to Goodale and Milner (1992) and Ungerleider and Miskin’s (1983) classic papers, is anatomically just a sub-part of parietal cortex broadly. Thus, neural responses in parietal regions to visual presentation of objects are not monolithically due to visual processing in the dorsal pathway. For instance, posterior superior parietal regions are involved in adaptive and flexible manipulation of visual representations (Gauthier et al., 2002; Y. Xu, 2018). The anterior supramarginal gyrus, which supports the representation of praxis or object use, is automatically engaged in a manner that is specific to the use of the object being seen (Q. Chen, Garcea, Jacobs, & Mahon, 2018). This region is just posterior to secondary somatosensory area SII (Gallivan and Culham, 2015), and as discussed below, heavily interconnected with lateral temporal cortex (see Figure 5). Supramarginal representations of object-associated praxis may not be visual, or visuo-motor, or even motor-relevant representations—but rather proprioceptive representations of how the action will feel¹ (Desmurget et al., 2009; Sirigu et al., 1996). Such anticipatory proprioceptive representations of how the action will feel can drive motor planning, and within a forward model, support online corrections as the action unfolds (Hutchison & Gallivan, 2018). Indeed, how the hand will wind up at the end of an action is arguably a core input around which motor actions are planned (Graziano, Aflalo, & Cooke, 2005).

Figure 5. Schematic of regions connected by known structural connections supporting functional object grasping.

Figure 5 represents a simple schematic representation of the principal anatomical pathways that form a network of regions that supports functional object directed action. The figure distinguishes between the core regions of this network (black outlines) which have been the focus of this review, as well as additional systems (grey outlines) that are known to be critical for functional object directed actions.

A range of findings support the broad generalization that the visual structure of objects is computed independently by dorsal and ventral stream processes. The observation that grip aperture can be spared in patients with ventral stream lesions suggests that the dorsal visual pathway computes its own proprietary visual representations of object volumetric structure, orientation, and distance (Goodale et al., 1991; Snow et al., 2015). One potential difference between dorsal and ventral visual representations is that dorsal visual object representations may be (more) viewpoint-specific, while ventral visual object representations are (more) view-point invariant (James, Humphrey, Gati, Menon, & Goodale, 2002) (Gallivan & Culham, 2015; Schenk, 2006).

There are also arguments that visual object representations do achieve perceptual invariance. Konen and Kastner (Konen & Kastner, 2008b) used fMRI in healthy participants to show size and viewpoint invariance in lateral occipital cortex (LOC), as well as in subregions of posterior IPS (specifically, IPS1 and IPS2). Freud and colleagues (E. Freud, Culham, Plaut, & Behrmann, 2017) found that posterior parietal and dorsal occipital cortex activity related to shape processing was correlated with visual recognition performance. Furthermore, neural signatures of visual shape representations in posterior parietal cortex were found to be present in individuals with ventral stream lesions and deficits for visual shape perception (E. Freud, Ganel, et al., 2017). Those findings support the argument that posterior parietal 3D shape representations are not dependent on ventral stream processing (E. Freud et al., 2016). Konen and Kastner (2008a) argued that parietal cortex (like the ventral stream) contains a hierarchically organized processing stream. They further argued that the fact that visual information is represented, at least to some extent, in retinotopic coordinates in parietal cortex, suggests that retinotopic coordinates could be a common substrate for the integration of information between posterior parietal cortex and the ventral stream (see discussion in (E Freud, Behrmann, & Snow, 2021).

A complementary perspective is offered by Xu (Y Xu, 2018), who argued that while ventral stream representations achieve invariance in order to provide a stable basis for perception, the role of posterior parietal cortex is to process visual representations in a manner that is tuned to the current task—’adaptive’ visual processing. Indeed, bilateral posterior superior parietal regions are involved in flexible and adaptive manipulation of visual representations (even in the absence of perception, such as in visual mental imagery and working memory (Gauthier et al., 2002; Jeong & Xu, 2016; Konen & Kastner, 2008a; Y. Xu, 2018). Importantly in this regard, a core aspect of Xu’s proposal is that abstract visual object representations in posterior parietal cortex are not the result of processing in the dorsal pathway—they are dependent on inputs from ventral stream processing. As reviewed below (see Figure 5), there are at least two vertical anatomical pathways could support integration between posterior parietal cortex and the ventral stream (Jitsuishi & Yamaguchi, 2020; Yeatman et al., 2014). Those pathways connect posterior parietal regions to ventral temporal-occipital regions.

Stepping back, there are three broad possibilities about how visual object representations are computed across the dorsal and ventral visual pathways: i) some parietal representations of visual object structure depend on inputs from the ventral stream (Y. Xu, 2018), ii) some ventral stream representations depend on inputs from the dorsal stream (Ayzenberg & Behrmann, 2023a, 2023b), or iii) the dorsal and ventral pathways independently compute visual object structure and independently achieve invariance to surface transformations in the input (Konen & Kastner, 2008b). In the end, and key take away for present purposes, is that regardless of how abstract or viewpoint invariant are the visual object representations that are native to the dorsal stream, there is no evidence that those representations interface with conceptual representations. Those visual representations are proprietary to action. Ultimately, the fact that naturalistic action regularly displays end-state-comfort, indicates that functional object grasping and use is dependent on a ventral pathway mediated interfaces with a conceptual interpretation of the scene in relation to standing action goals.

Connectivity between Parietal and Occipital-Temporal Regions

The framework we have sketched emphasizes computations that are supported by reciprocal interactions between parietal and occipito-temporal regions. What is the evidence for privileged functional and structural connectivity among the relevant regions? A core expectation is that the connectivity of parietal regions into ventral and lateral occipito-temporal cortex should align with relevant functional parcellations of parietal and occipito-temporal regions.

One distinction that is central to parietal lobe organization is the distinction between the dorsal-dorsal and ventral-dorsal streams (Binkofski & Buxbaum, 2013; Gallivan & Culham, 2015; Rizzolatti & Matelli, 2003). The dorsal-dorsal stream involves the interaction of superior parietal regions and dorsal premotor regions in frontal cortex, while the ventral-dorsal pathway involves the interaction of inferior parietal and ventral pre-motor frontal regions. The dorsal-dorsal pathway supports real-time visuomotor updating and is critical for the transport and real-time corrections involved in object grasping, while the ventral-dorsal stream integrates actions with semantic representations and supports functional grasps and object use.

There is privileged functional connectivity, as measured at rest and during task-based fMRI, between parietal action areas and ventral-medial structures of the temporal lobe (fusiform gyrus), as well as lateral temporal regions (the posterior middle temporal gyrus; the extra-striate body area (Almeida, Fintzi, & Mahon, 2013; Q. Chen, Garcea, Almeida, & Mahon, 2017; F. Garcea & Mahon, 2014; Mahon, Kumar, & Almeida, 2013; Mahon et al., 2007; Stevens, Tessler, Peng, & Martin, 2015; Walbrin & Almeida, 2021). Bracci et al. Importantly, functional connectivity between ventral stream regions and parietal action areas is modulated according to whether the task involves action or perception (F. E. Garcea, Q. Chen, R. Vargas, D. A. Narayan, & B. Z. Mahon, 2018; Hutchison & Gallivan, 2018). Furthermore, the multivoxel pattern of resting functional connectivity to parietal actions areas in the ventral stream predicts (positively) the amplitude of stimulus-driven responses to tools (but not to faces, places or animals; (Q. Chen et al., 2017; Walbrin & Almeida, 2021). The voxel-wise alignment of resting functional connectivity to parietal cortex with category-preferences for tools points to an important role of functional connectivity in shaping stimulus preferences in the ventral stream (Mahon, 2022).

Beyond functional connectivity among parietal and occipito-temporal regions, direct structural projections have been described. Borra and colleagues found that macaque aIPS has a direct projection into inferior temporal cortex, and argued that object grasping is influenced by representations of object identity computed by the temporal lobe (Borra et al., 2008). Subsequent work has shown a distinction between the projections of the anterior inferior parietal lobe (supramarginal gyrus) and more posterior and superior parietal regions along IPS. There is direct structural connectivity of the supramarginal gyrus into lateral temporal cortex, including the superior, middle, and inferior temporal gyrus (Bullock et al., 2019; Caspers et al., 2011; Jung, Cloutman, Binney, & Lambon Ralph, 2017; Ruschel et al., 2014). Integration of the supramarginal gyrus with lateral temporal regions is generally discussed in the context of integrating phonological processing (supramarginal gyrus) with linguistic representations (middle temporal gyrus). There may be a meaningful homology between phonological representations and object-directed action representations (supramarginal gyrus) and their mapping onto abstract representations of words and objects in the middle and inferior temporal gyrus (for discussion, see Caglar et al, under review). Potentially relevant in this regard is a left/right asymmetry: connectivity of the left supramarginal gyrus to the left middle temporal gyrus is stronger than the connectivity of the right supramarginal gyrus to the right middle temporal gyrus (Caspers et al., 2011; Ruschel et al., 2014). An important question for future research is the degree to which praxis and phonological processing are co-lateralized across brains, or whether variability in the laterality of structural connectivity is related to functional lateralization of language or object-directed action.

A distinct vertical integration pathway between more posterior and superior parietal regions and ventral temporal-occipital cortex has recently been described (Jitsuishi & Yamaguchi, 2020); see also findings and discussion in (Bullock et al., 2019; Caspers et al., 2011; Jung et al., 2017). That pathway, termed the IPS-Fusiform Gyrus (or IPS-FG) is a direct pathway that projects from the posterior medial bank of IPS1 (superior parietal lobule) into the fusiform gyrus along its anterior and medial bank (i.e., the collateral sulcus). Anatomically, IPS-FG is anterior to the Vertical Occipital Fasciculus (Yeatman et al., 2014) and posterior to the descending segment of the arcuate fasciculus (see discussion in (Bullock et al., 2019)). There may be overlap between what has been referred to as the Temporal-Superior Parietal projection (TP-SPL) and the IPS-FG, which will have to be explored in future work.

The physiological tuning profiles of the cortical projections of the IPS-FG remain to be mapped, but based on the anatomical location of the pathway it is excellently positioned to integrate grasp- and transport-related processes (IPS) with representations of surface texture and material composition in the medial fusiform gyrus. Jitsuishi and Yamaguchi (2020) note a high level of inter-subject variability in left-right asymmetries in the projection pattern, as measured with deterministic tractography in DTI data. It is also the case there is substantial inter-subject variability in the degree to which differential neural activity for manipulable objects compared to non-manipulable object baselines is bilateral or stronger in the left than in the right. In particular, the supramarginal gyrus, middle temporal gyrus, and to some degree aIPS tend to be more left lateralized, whereas posterior IPS and medial fusiform/collateral sulcus activity tends to be more bilateral, with substantial variability across participants. An important target for future empirical work will be to test whether inter-subject variability in laterality differences in structural integration between IPS and FG are aligned with inter-subject variability in the fMRI derived measure of cortical information processing.

In summary, there are clear hypotheses about the neuroanatomical substrate of a functional model that emphasizes the integration of temporal lobe object representations with parietal lobe action representations. At the most general level, more anterior and inferior parietal regions (supramarginal gyrus) are directly connected with posterior aspects of the middle temporal gyrus, whereas more superior and posterior aspects of parietal cortex have direct projections into inferior and ventral regions of temporal lobe. Importantly, those parietal-temporal interfaces are in the setting of direct projections aong the fusiform and middle and inferior temporal gyrus, as well as the supramarginal gyrus with ventral premotor regions (Rizzolatti & Matelli, 2003; Rushworth, Behrens, & Johansen-Berg, 2006) (see discussion below, and Figure 5).

Bottom Up from the Top Down: Parietal representations shape visual object representations in the temporal lobe

In an influential series of papers, Bar and colleagues (Bar et al., 2006; Kveraga, Boshyan, & Bar, 2007) argued that orbitofrontal regions support a fast magnocellular-based analysis of visual inputs that guides more detailed (parvocellular dominated) perceptual analysis in the ventral stream. Bar and colleagues’ proposal represented an important instance where a region well outside of the classic ventral visual hierarchy (Riesenhuber & Poggio, 1999) provides guides visual processing. That proposal emphasized inputs to the ventral stream that were fundamentally in the service of visual perception and recognition. Here we propose that interactions with regions of the dorsal visual pathway bias processing in the ventral visual pathway to extract specific object properties in support of functional object directed action.

As noted above, and shown in Figure 1, there is a consistent pattern across individuals for the collateral sulcus and medial fusiform to be differentially engaged when viewing manipulable objects compared to a range of baselines (faces, animals). Why would a region of ventral temporal cortex that supports analysis of surface texture and material properties of objects prefer manipulable objects over other categories of stimuli? We have previously suggested that functional specialization for manipulable objects, or ‘tools,’ in the collateral sulcus and medial fusiform gyrus is driven by computational demands imposed by parietal action systems, via long-range connectivity-based constraints (Mahon et al., 2007; (Mahon and Caramazza, 2011; Mahon, 2019; 2020; Garcea et al., 2018; Lee et al., 2019); for broader discussion of connectivity constrained approaches, see Martin, 2016; Riesenhuber, 2007; Bouhali et al., 2014; (Amaral, Bergstrom, & Almeida, 2021; Walbrin & Almeida, 2021) Bracci & Peelen, 2013; (Amaral et al., 2022) Op de Beeck et al., 2019; Osher et al., 2016; Saygin et al., 2016); for alternative perspectives, see (Haxby et al., 2001; Levy, Hasson, Avidan, Hendler, & Malach, 2001; Nasr, Echavarria, & Tootell, 2014).

The long-range connectivity constraints from parietal cortex that shape neural specificity for manipulable objects in the collateral sulcus and medial fusiform gyrus could be correspond to the direct white matter pathway from posterior IPS into the fusiform gyrus and collateral sulcus reviewed (Jitsuishi & Yamaguchi, 2020). Alternatively, connectivity from the supramarginal gyrus (responsible for praxis representation) into the collateral sulcus and medial fusiform could be mediated by the left posterior middle temporal gyrus, which does have direct connectivity both to the supramarginal gyrus and to the fusiform gyrus. But what is the functional evidence that parietal action-relevant processing modulates responses in ventral temporal-occipital cortex?

Several studies have reported that action-relevant variables associated with manipulable objects modulate neural responses in temporal-occipital areas (Q. Chen et al., 2017; Gallivan, McLean, Valyear, & Culham, 2013; F. Garcea, Q. Chen, R. Vargas, D. Narayan, & B. Mahon, 2018; Mahon et al., 2007; Valyear & Culham, 2010). For instance, even within ‘graspable’ objects, the relation of manner of manipulation to structure and function modulates neural responses in medial ventral temporal-occipital regions (Mahon et al., 2007). Similarly, it has been found that computation of functional grasps depends on processing in ventral temporal-occipital cortical regions (Valyear & Culham, 2010). In one particularly direct demonstration of the role of ventral temporal regions in action, Gallivan and colleagues found that upcoming actions (pointing versus grasping versus reaching) could be decoded, before the action, from multi-voxel patterns in the ventral stream (Gallivan, Chapman, McLean, Flanagan, & Culham, 2013; Gallivan, McLean, et al., 2013). Moreover, Chen and colleagues separated elongation from ‘tool-ness’ and used dynamic causal modeling of fMRI time series data to look at direction-specific modulations between dorsal and ventral stream regions. They showed that the direction of causal interaction was dependent on the kind of stimulus presented: from parietal to temporo-occipital regions for non-tool elongated stimuli (cucumber), and from ventral to dorsal stream regions for tools, regardless of whether they were elongated (J. Chen et al., 2018). A recent study using extracranial EEG found that (putative) parietal sources distinguish among the visual categories of graspable objects, insects and birds, prior to ventral sources; those findings are consistent with the view that the dorsal stream provides inputs to ventral stream analyses (Ayzenberg, Simmons, & Behrmann, 2023).

Decisive evidence for modulation of ventral stream responses by parietal regions comes from two studies that tested how causal modulation of parietal regions affects responses in the medial fusiform and collateral sulcus, and specifically for processing manipulable objects. Across both studies it was found that a) modulation of processing in parietal areas affects medial fusiform and collateral sulcus regions more so than neighboring ventral stream regions (i.e., is anatomically specific), and b) that modulation was only for graspable and manipulable objects, and not for other categories of visual stimuli, such as animals or faces (i.e., it is content specific). Garcea and colleagues (F. E. Garcea et al., 2019) found that lesions (brain tumors) in parietal cortex, and specifically the supramarginal gyrus and aIPS, disrupt neural responses to ‘tool’ (but not place) stimuli in medial ventral stream regions, as well as in the left posterior middle temporal gyrus. Lee and Colleagues (Lee, Mahon, & Almeida, 2019) found that tDCS to the left inferior parietal cortex distorted representational similarity and pattern discriminability for tools in the medial fusiform region. Furthermore, tDCS polarity (cathodal vs. anodal stimulation) modulated connectivity between the medial aspects of the fusiform and left inferior parietal cortex. Importantly that effect of tDCS polarity on functional connectivity was observed to be differential when ‘tool’ stimuli were being processed compared to when place stimuli were being processed, indicating an interaction of connectivity with representational content.

The causal evidence from Garcea and colleagues (2019) and Lee and colleagues (2019) shows that neural responses in regions of the ventral stream that exhibit differential neural responses to ‘tools’ depend, at least in part, on the integrity of processing in parietal cortex (Figure 3). This supports our core hypothesis that parietal action regions modulate perceptual analyses of surface texture and material properties in ventral temporal regions in support of functional object directed action.

Figure 3. Interference with parietal action representations causes domain- and region-specific changes in medial ventral temporal cortex.

Two studies provide causal evidence for the hypothesis that parietal action representations modulate ventral occipital-temporal processing of manipulable objects. Garcea and colleagues (F. E. Garcea et al., 2019) found that lesions involving aIPS were associated with reduced responses to tool stimuli in medial ventral occipital-temporal areas, but were not associated with reduced responses to place stimuli in the same regions (Panel A). A whole-brain analysis that searched for where BOLD activity was inversely related to the probability of a lesion to aIPS, identified the medial fusiform gyrus and collateral sulcus (Panel B; this analysis also identified the posterior inferior/middle temporal gyrus). This was despite the fact that place stimuli elicit stronger activity in the medial fusiform gyrus and collateral sulcus than do tools (Downing et al., 2006; Mahon et al., 2007). In a separate study, in healthy participants, Lee and colleagues (Lee et al., 2019) found that cathodal tDCS to left parietal cortex disrupted voxel-wise pattern discriminability between tools and animals (Panel C) in medial ventral temporo-occipital cortex, but not in lateral aspects of the fusiform gyrus (fusiform face area). Neural similarity among tool stimuli was increased after excitatory (anodal) stimulation of left parietal cortex, and reduced after inhibitory (cathodal) stimulation of left parietal cortex. Moreover, as shown in Panel D, cathodal versus anodal stimulation modulated effective functional connectivity between SMG and the medial fusiform gyrus for tool, but not for place stimuli. These findings indicate that parietal representations of object-directed action causally modulate, online, visual object processing of graspable objects in occipital-temporal cortex. Reproduced with permission from (Mahon, 2023)

Feeding the forward model: Parietal regions supporting functional object use take inputs from occipito-temporal regions representing object properties

In the course of functional object use, it is often the case that an action plan is represented prior to perception of the object—an object is needed because it will be useful to fulfill a goal. However, for the situation in which action representations must be accessed from a visually presented object, in the absence of a standing action goal, a key question is which parietal action representations are accessed through a purely dorsal stream analysis, and which are dependent on processing of the visual input by the ventral stream.

One way to test whether (and how) processing in parietal regions depends on inputs from the ventral stream is to psychophysically manipulate visual attributes of stimuli so as to bias processing of those stimuli toward a ventral (as opposed to a dorsal) visual pathway analysis. Asymmetries in the projections of parvocellular and magnocellular channels into the ventral and dorsal streams provides the leverage needed to accomplish this. The parvocellular pathway differentially projects to the ventral compared to the dorsal visual pathway (Livingstone & Hubel, 1988; Merigan & Maunsell, 1993) and shows greater sensitivity for high spatial frequencies, low temporal frequencies, and heterochromatic red/green isoluminant boundaries (Figure 4). By comparison, there are magnocellular projections into both the ventral and dorsal visual pathways.

Figure 4. Access to parietal regions supporting functional grasps from visual presentation of an object depends on processing in the ventral visual pathway.

Images of tools (and of the baseline category, animals) were titrated so as to be defined by visual dimensions that by bias processing toward parvocellular channels, and thus away from processing in the dorsal visual pathway (e.g., high spatial frequency, red/green isoluminant contrast). Regions of parietal cortex that continue to exhibit tool preferences for stimuli biased to be processed by parvocellular pathways receive their inputs from the ventral (rather than the dorsal) stream. We found that, under those conditions, tool preferences were restricted to aIPS and the supramarginal gyrus. This was the case when stimuli contained only high spatial frequencies (Panel A; (Mahon et al., 2013)), were broad band in terms of spatial frequencies but presented at a low temporal frequency (Panel B, (Kristensen et al., 2016)), or were defined by red/green isoluminant color contrast (Panel C) (Almeida et al., 2013). In a fourth study, we reasoned that parietal responses to tools that depend on ventral stream processing will exhibit resilience to the visual field in which the stimulus is presented. While posterior and superior parietal areas showed strong biases for stimuli presented in the contralateral visual field, the left inferior parietal cortex exhibited tool preferences that were not modulated by the side of presentation of the stimuli (Panel D) (F. Garcea et al., 2016). These findings show that neural responses to visually presented ‘tools’ in left inferior parietal areas depend on inputs from the ventral visual pathway.

In a series of four independent studies, we found that stimuli biased toward processing by parvocellular channels, and by inference the ventral stream, continue to activate anterior/inferior parietal regions (supramarginal gyrus, aIPS) but not posterior superior parietal regions (posterior IPS, superior parietal regions, dorsal occipital cortex). In one study (Mahon et al., 2013), participants saw images of tools and animals that had been filtered to contain only high or only low spatial frequencies. The stimuli containing only high-spatial frequencies would be biased toward analysis by parvocellular channels, and hence within the ventral visual pathway. Because the dorsal pathway has little to no parvocellular input, it has low sensitivity for resolving information contained in high spatial frequencies. Thus, any voxels that continue to exhibit tool preferences even for stimuli defined entirely by high-spatial frequencies must be supported by inputs from the ventral visual pathway. To test that, whole-brain contrasts were computed to identify voxels exhibiting increased BOLD for ‘tool’ compared to animal stimuli, separately for the low and for the high spatial frequency stimuli. It was found that tool preferences for stimuli defined by high-spatial frequencies (Figure 4) no longer activated regions along posterior IPS, superior parietal cortex, and dorsal occipital cortex—that is, the regions of parietal cortex that are associated with the ‘dorsal’ visual pathway, and which are routinely shown to be differentially engaged by graspable objects compared to various baselines (Chao & Martin, 2000; Fang & He, 2005; Mahon et al., 2007; Noppeney et al., 2006) see Fig1). Rather, stimuli defined by high-spatial frequency information produced ‘tool’ preferences that were restricted to anterior and inferior parietal regions, including aIPS and the supramarginal gyrus (Mahon et al., 2013).

We then sought to replicate that core dissociation using a different psychophysical dimension that also distinguishes between magnocellular and parvocellular channels. Kristensen and colleagues varied the temporal frequency (Kristensen, Garcea, Mahon, & Almeida, 2016) at which broad-band images of tools and animals were presented, and observed the same dissociation: tool-preferences for stimuli presented at low-temporal frequencies were restricted to anterior and inferior parietal regions centered on aIPS. That pattern was replicated by a third study that used red/green isoluminant color contrast (Almeida et al., 2013) to bias visual processing toward parvocellular analyses, and thus the ventral stream (Figure 4): Specifically, ‘tool’ preferences were observed in selective inferior parietal regions for stimuli defined by red/green heterochromatic isoluminant contrast.

In a fourth, and conceptually independent test (F. Garcea, Kristensen, Almeida, & Mahon, 2016), we asked whether parietal object representations that depend on inputs from the ventral stream exhibit resilience to the visual hemifield in which the stimuli were presented. That prediction is based on the premise that outputs of ventral stream processing will be abstracted away from the retinotopic (visual field) location at which the stimulus was initially presented. We found that responses to manipulable objects in the left inferior parietal lobe were not modulated according to whether the stimuli were presented in the left or the right visual field (Figure 4). This was in contrast to neural responses in superior and posterior parietal areas, where tool responses were strongly modulated by hemifield of presentation (always stronger contralateral to the hemifield of presentation).

The four independent tests briefly reviewed demonstrate that neural responses to manipulable objects in the anterior inferior parietal lobe, and in particular the supramarginal gyrus and aIPS, depend on processing in the ventral object processing stream. These findings support the hypothesis that parietal processes supporting functional object-directed grasping and use are driven by inputs that come by way of the ventral visual pathway. This is conceptually aligned with the argument sketched above based on end-state-comfort: because the dorsal stream does not support access to a conceptual interpretation of the visual input, and because functional object grasps presuppose a conceptual interpretation, it follows that a critical input to functional grasping are object representations computed by the ventral visual pathway.

Reciprocal Interactions between Parietal and Occipito-Temporal Representations

We have presented evidence for two directions of influence between parietal action representations and occipito-temporal object representations. The goal is to test hypotheses about the conditions under which a particular direction of causal influence is active, and at a particular period of time during cycles of perception-action interaction.

One direction of influence is from action-relevant parietal regions on specific regions of the ventral stream to extract relevant object properties that are relevant to functional action. We have focused on interactions supporting processing of surface texture, material composition, weight distribution, and conceptual attributes such as object function—as these are all attributes of objects that are known to be processed by the ventral and not by the dorsal visual pathway.

Modulation of neural responses in occipito-temporal regions by parietal-based action representations could be considered a form of input from parietal cortex to the temporal lobe—akin to the orbitofrontal input hypothesis of Bar and colleagues in regard to visual shape and form (Bar et al., 2006). By hypothesis, the action-normative influences of parietal areas on ventral temporal cortex will be observed for visual stimuli that the dorsal visual pathway independently registers—i.e., visual objects supporting graspability. That assumption is supported by findings indicating that graspable objects are processed ‘asemantically’ by the dorsal stream (that is, without access to information about object identity or object function; (Almeida, Mahon, Nakayama, & Caramazza, 2008a; Almeida, Mahon, Zapater-Raberov, Dziuba, Cabaço, et al., 2014; J. Chen et al., 2018; Fang & He, 2005; Handy, Grafton, Shroff, Ketay, & Gazzaniga, 2003; Vincent, Laliberte, Morris, & Wiemann, 1984).

The other direction of influence is from temporal lobe conceptual representations of objects and actions (posterior lateral temporal regions, such as pMTG) on parietal representations of object praxis (supramarginal gyrus). In order to access the correct representations of object use the object must be identified, and its function represented. The findings summarized herein demonstrating signatures of parvocellular (i.e. ventral stream) on responses in inferior parietal regions may represent this direction of influence (Almeida et al. 2013, Mahon et al 2013, Kristensen et al, 2016 Garcea et al, 2016). Furthermore, and by hypothesis, the impaired performance of Patient DF when functional object grasps are mis-aligned with what would be biomechanically most comfortable (Carey et al., 1996) is due to the disruption of ventral stream inputs to parietal action systems.

Reciprocal interactions between parietal and dorsal stream regions with ventral and lateral occipito-temporal regions support everyday actions that display end-state-comfort. According to the goal of the action, and the anticipated grasp location on an object, specific information about the surface texture and material properties of the object become relevant. Where does a hot cup of coffee need to be grasped so it won’t be too hot to hold? How much grip force will be required to support a stable lift of an object, according to the anticipated surface friction at the anticipated grasp point as well as the expected weight distribution around that grasp point? Which tool, of those available in the workspace, would support a given action goal, where the action goal is represented independently of perceptual input. For instance, if one needs to hold a door open, should one select a sponge or a screwdriver? Reciprocal action-perception interactions drive differentiated analysis of object properties, in the service of the fulfilling the action goal. Importantly, while those interactions may drive processing in the ventral visual pathway, that processing in the ventral pathway is fundamentally in the service of action, rather than in the service of perception, communication, or other downstream functions that are also supported by the ventral stream.

The temporal patterning of reciprocity in interactions between parietal and temporal-occipital regions is likely modulated by the demands of the current task goals. An example of a task difference is whether or not an action goal is represented prior to perception of the target of the action. For instance, when shown a visual object, out of context and in the absence of a standing action goal, the first direction of influence may be from dorsal to ventral regions. This situation is representative of many cognitive neuroscience studies in which the system is ‘surprised,’ on each trial and by each stimulus. That experimental strategy has the merit of protecting observed activity patterns from participants’ strategic anticipation of upcoming stimuli. In this case, it may be that the dorsal stream generates a range of potential object-directed grasps, and then uses information computed in ventral stream regions to winnow the set of possible grasps down to those that match perceptual object properties (Schubotz, Wurm, Wittmann, & von Cramon, 2014). On the other hand, ‘in the wild’, object-directed actions unfold in situ in an environment, and in the context of standing desires and action goals. As such, perception of an object that is the target of an action often does not precede the representation of an action goal—the object is sought because of the action goal. Consider the last time you retrieved an object from the fridge, cabinet, the other room, or simply just out of sight on your desk. In that situation, the dynamics of interactions between parietal and occipito-temporal cortex begin well before visual perception of the object.

Much remains to be determined about how to think about the nature of reciprocal interactions between parietal and ventral and lateral occipito-temporal regions. One idea would be that ‘inputs’ from parietal regions change the gain on occipito-temporal perceptual regions, in a manner that amplifies that type of perceptual inputs that are demanded of successful functional object grasps. Another possibility is that neural responses in occipito-temporal regions could track the degree to which the outputs from the ventral stream are successfully processed by downstream systems supporting object-directed actions. On this view, when parietal action representations are disrupted (e.g., transient or chronic lesion), the collateral sulcus and posterior-middle temporal gyrus are ‘backed up’ (by analogy, as to how when a valve is closed on a pipe, that slows the flow-rate upstream of the valve).

The Broader Context

The network of cortical regions, and their putative connectivity, summarized in Figure 1, represents, in our view, part of a domain-specific network that is dedicated to solving a specific computation: building a forward model of how first-person actions will change the state of the world and the body (Fischer & Mahon, 2021). That network is fundamentally concerned with transforming sensory signals of the body, and the objects and layout of peri personal space into specific actions, driven by a conceptually rich action goal. Action goals are not ‘driven’ by perception; action goals are abstract (get a drink of water, go to class)—and must be registered to the state of the world and of the body (Botvinick, 2008). The representation of abstract action goals can drive processing in higher order and modal regions, and inputs from those modal regions in turn constrain action goals. The dorsal visual pathway delivers to the system a continuously updated visual representation of the current layout of the world, in egocentric reference frames that are natively aligned with independent sensory evidence about the state of the body. The ventral stream delivers a perceptual and conceptual interpretation of the visual scene that aligns with an abstract action goal.

Of course, dyadic interactions among specific parietal and occipital -temporal regions would not be sufficient to support functional object directed actions. A number of other systems are involved, including systems that support selection among multiple representations (inferior frontal gyrus: (Kan & Thompson-Schill, 2004; Roy, Riesenhuber, Poggio, & Miller, 2010), flexible manipulation of visual object representations to plan actions (posterior superior parietal cortex; (Y. Xu, 2018)), object-specific processing and integration of diverse object-related information including object function (anterior temporal lobe; (Anzellotti, Mahon, Schwarzbach, & Caramazza, 2011; Canessa et al., 2008; Patterson, Nestor, & Rogers, 2007), planning of motor-relevant aspects of actions given anticipated end-states and a space of biomechanically possible actions (e.g., ventral premotor cortex, (Schubotz et al., 2014)), and linearizing hierarchic action plans (pre/SMA, (Botvinick, 2008; Picard & Strick, 1996, 1997)). Attention to functionally relevant object properties is implemented via the reciprocal parietal-occipito-temporal connectivity described herein, and may be a direct reflection of the role of the dorsal attention network in guiding purposeful behavior (Corbetta, Patel, & Shulman, 2008; Corbetta & Shulman, 2002).

Our project here has been to frame some ideas for thinking about how perception and action iteratively work together to support activities of daily living, and to motivate testable proposals about causal dependencies in processing between parietal and ventral stream regions as a function of task demands. We believe that studies of how transient and chronic lesions affect processing distal from the site of the lesion, in particular with methods that afford high temporal resolution, hold great potential to confirm or reject the core ideas we have proposed about processing dependencies among parietal and ventral stream regions.

Figure 2. Non-Geniculostriate pathways are sufficient to support wrist orientation during object grasping.

Some aspects of object grasping (grip scaling, wrist orienting) can be preserved in the setting of ventral stream lesions (Goodale et al., 1991) and cortical blindness (M T Perenin & Rossetti, 1996; Prentiss, Schneider, Williams, Sahin, & Mahon, 2018). The figure demonstrates preservation of wrist-orientation when a subject with cortical blindness (Panels A & B) grasps a handle in the hemianopic field (Panel F) even though the subject cannot perform a perceptual matching task with the same visual information (Panel D). Reproduced from Mahon B (2020). The representation of tools in the human brain. In: D Poeppel, M Gazzaniga (Eds.), The new cognitive neurosciences, sixth edition, MIT Press, Cambridge, MA.