The posterior parietal cortex in adaptive visual processing

Abstract

Although the primate posterior parietal cortex (PPC) has been largely associated with space, attention, and action-related processing, a growing number of studies have reported the direct representation of a diverse array of action-independent non-spatial visual information in the PPC during both perception and visual working memory. By describing the distinctions and the close interactions of visual representation with space, attention, and action-related processing in the PPC, here I propose that we may understand these diverse PPC functions together through PPC’s unique contribution to adaptive visual processing and form a more integrated and structured view of PPC’s role in vision, cognition and action.

The parietal puzzle

Decades of monkey neurophysiology and human patient and imaging research has led to three prominent views regarding the function of the primate posterior parietal cortex (PPC) centered around the intra-parietal sulcus (IPS). The views focus on either the representation of space, attention, or action. Each view is supported by a wealth of evidence, yet at the same time is substantially different from the other views. Complicating this range of views, over the last two decades, a growing number of studies have reported the direct representations of action-independent non-spatial visual information in the PPC, both in perception and during visual working memory (VWM) delay, ranging from simple visual features such as colors, orientations and shapes to complex and abstract ones including object identity and categories [1–3]. The presence of these visual representations cannot be directly accounted for by PPC’s existing role in space, attention, and action-related processing. It appears that the more we learn about the PPC, the less certain we are of its precise function.

Here I examine the existence of action-independent non-spatial visual representation in the PPC and its relationship with the other major PPC functions from both monkey and human research with an emphasis on human imaging studies. I first describe the latest findings supporting the direct representation of a diverse array of such visual information in the PPC. I then describe, based on functional and anatomical evidence, how action-independent non-spatial visual representation in the PPC is both distinct and at the same time closely interacts with PPC’s role in space, attention and action-related processing. Given PPC’s involvement in multiple distinctive, yet interconnected mental operations, I propose that we may understand these diverse PPC functions together through PPC’s unique contribution to adaptive visual processing. Under this framework, the close connections and interactions between an effective attention gating mechanism, a flexible representation center and a motor output system, all at the same time interacting with both a space-and a non-space based representation, allow incoming visual information to be selected, represented and sustained to guide thoughts, solve problems, and, if needed, support the execution of appropriate actions. Understanding the functions of the PPC through its unique contribution to adaptive visual processing both accommodates each distinctive PPC function and allows us to form an integrated and structured view regarding PPC’s overall role in vision, cognition and action.

The PPC in space, attention, and action-related processing

Decades of neuroscientific research has identified three major functions related to the PPC centered around the IPS, focusing on space, attention, and action-related processing, respectively. The space view describes the PPC’s role in spatial location representation and is featured prominently in the ventral-what and dorsal-where two-pathway view of visual information processing in the primate brain [3], The presence of multiple spatial reference frames in the PPC (e.g., eye-centered or head- centered) [5] and the existence of topographic maps along the IPS [6] provide additional support for this view. Further research has delineated three pathways in which visuospatial information from parietal cortex may be relayed to PFC, premotor areas, and medial temporal lobe to accomplish tasks such as spatial working memory, motor control and navigation, respectively [7].

Meanwhile, the attention view of the PPC highlights its role in top-down and bottom-up attention-related processing [8–11]. By responding to salient and task-relevant stimuli, the PPC is believed to sort locations, features, objects, semantic and reward information according to their behavioral relevance to form a spatially organized priority map; such a map can guide the visual system to prioritize processing and the motor system to perform action towards the prioritized locations, visual features, or objects. In the human brain, bottom-up attention is controlled by a ventral circuit including the right temporal parietal junction (TPJ) and inferior frontal gyrus (IFG), whereas top-down attention is controlled by a dorsal circuit including prefrontal regions, IPS and superior parietal lobule (SPL) [8,12].

The action view of the PPC focuses on its role in sensorimotor transformation needed in the planning and control of visually guided movements. Under this view, different IPS areas are associated with different types of movement planning and execution, such as anterior IPS in grasping, medial IPS and superior parietal lobule in reaching, and macaque LIP and a similar human area in eye movement [13–19]. Based on neuropsychological observations from patient DF, Goodale and Milner [20; see also 21] proposed that the dorsal pathway contains visual object information necessary for the control of action.

Although each of these three views is strongly supported by a wealth of corresponding empirical evidence, none can replace the other views. Despite the prominent representation of spatial location in the PPC and its ability to provide guidance to space-based attention selection and action, the location view by itself cannot easily account for non-spatial attentional effects and the intricacies of attention- and action-related PPC activities. Similarly, the attention view does not fully capture details of spatial and action representations. Likewise, the action view is limited in explaining attentional effects that do not require a motor output. Space, attention and action-related processing thus appear to be three distinctive functions associated with the PPC.

The existence of action-independent non-spatial visual representation in the PPC

Besides the three major views described above, a growing number of studies over the last two decades have also reported the direct representations of a diverse array of action-independent non-spatial visual information in the PPC in both perception and VWM [1–3, and Table 1].

Table 1.

The PPC regions involved in action-independent non-spatial visual information representation, spatial processing, attention control, and action planning. Although the encoding of visual representation has been shown in multiple PPC regions, the regions listed here show the greatest action-independence, tolerance to low-level image transformations, and/or distractor resistance. Likewise, although spatial signal is present throughout the PPC, the spatial regions listed here are primarily involved in space representation. Numbers refer to the relevant references.

Action Independent Non-Spatial Visual Information
Macaque LIP	Color: 23,26–28 Motion: 24,25,27,28,31 Shape: 22,32,91 Abstract Category: 29–31,33
Human PPC(strongest in VWM capacity-defined superior IPS and topographic areas IPS1/IPS2)	Color: 37,56,62,63 Motion: 37,57 Shape (including orientation): 35,36,38–41,44,46,55,60,61,63,64 Object Identity: 43,58 Abstract Category: 42,44,45,47–49
Predominantly Spatial Information
Macaque CIP	71,72
Human Inferior IPS	52,67,73,74,77,78,
Attention and Task Set Switching
Macaque 7a and DP	88,89
Human SPL and right TPJ	9,12,85–87
Action Planning
Macaque LIP, AIP, MIP, and parietal reach region	13,16–19
Human IPS1/IPS2, AIP, medial IPS, SPL	14–16

Sereno and Maunsell [22] were the first to report shape selectivity, a non- spatial feature, in macaque monkey’s lateral intra-parietal (LIP) neurons during both perception and VWM delay period (Figure 1A). LIP neurons also exhibit selectivity to directions of motion, color, and abstract features such as the learned motion category and shape-associations during perception and/or VWM [23–31] (Figures 1B, 1C, 2B). Although VWM-related representations exist in the prefrontal cortex (PFC), several studies have shown that, at least for non-spatial features, VWM-related representations appear to arise earlier in the PPC than PFC [32–34]. This indicates that VWM representations in the PPC are not triggered by feedback originated from the PFC, but may by themselves play a critical role in VWM storage (see also [2]).

Figure 1.

The representations of shape, color and motion in the macaque LIP and the human PPC. (A) A macaque LIP neuron exhibits increased firing to the presentation of a preferred shape (adapted from [22]). (B) Average activity from color-selective macaque LIP neurons shows increased firing to a task relevant color compared to a task irrelevant color (adapted from [27]). (C) Motion direction classification of macaque LIP neurons shows significant representation of motion direction during the encoding period of both a delayed-match-to-sample (DMS) and a delayed-match-to-category (DMC) task. The three dashed, vertical lines represent the start of the sample period, the end of the sample period and the end of the delay period (adapted from [31]). (D) In human IPS, VWM representation of the cued color or orientation was found in IPS using fMRI and an inverted encoding model when participants were asked to remember either color or orientation of one of the two colored gratings (adapted from [63]). (E) Using fMRI adaptation, size and viewpoint tolerant object shape representation was found in human topographic areas IPS1 and IPS2, comparable to that found in LOC, a ventral object representation area (adapted from [36]). (F) Using fMRI MVPA, degraded novel shapes could be successfully decoded in the human PPC when they were attended but not when they were shown as distractors and unattended. Performance is reported here as both d’ on the y-axis and as decoding performance on the x-axis (chance decoding is 50%) (adapted from [40]). (G) The human superior IPS (as defined in a VWM task) exhibited shape selectivity for two real-world object shapes (measured as the difference in fMRI response pattern correlation for a pair of identical objects and that for two different objects). This shape selectivity was found when object shape was task relevant in a shape repetition detection task but not in a motion jitter detection task. In neither task did superior IPS showed selectivity for object location, which was either above or below the central fixation (adapted from [39]).

Figure 2.

The representations of real-world object identities, abstract shape and motion categories, location and size-tolerant natural object categories, and semantic and action categories in the macaque LIP and the human PPC. (A) fMRI response patterns in the human superior IPS (as defined in a VWM task) showed higher correlations for two sets of well-known face images sharing the same rather than different identities, and for two sets of well-known car images sharing the same rather than different identities (adapted from [43]). (B) Firing rates from two macaque LIP neurons showing differential responses to the different shape and motion categories during both the encoding and delay period of a DMC task (adapted from [30]). (C) Significant fMRI MVPA decoding of 8 natural and manmade object categories (with exemplars varying in viewpoint and pose) were found in topographically and functionally defined human occipital and parietal regions. In both the PPC, ventral occipito-temporal (VOT) and lateral occipito-temporal (LOT) regions (but not in early visual areas), these category representations show significant tolerance to changes in image spatial location and size such that a classifier trained to discriminate categories at one location/size could also discriminate those at the other location/size significantly above chance (chance was 0.5). The actual images used were equated in luminance, contrast, and spatial frequency to remove the contribution of low-level image differences among the categories (adapted from [49]). (D) The human PPC showed significant decoding of the semantic category and action associated with a set of manmade objects only when such information was task relevant. The decoding was measure as the correlation of fMRI response pattern similarity and semantic or action similarity among the different objects (adapted from [45]).

The representation of non-spatial feature has also been obtained in fMRI studies, with both the macaque LIP and the human PPC exhibiting size tolerant shape selective fMRI adaptation to gray-scaled object images, comparable to that found in the macaque IT cortex and the human lateral occipital cortex (LOC) object selective regions [35]. Likewise, topographically defined human IPS areas V3A, and IPSO to IPS2 also exhibit fMRI adaptation to novel and familiar 2D and 3D objects, with IPS1 and IPS2 additionally showing tolerance to changes in size and viewpoint comparable to that found in LOC [36] (Figure 1E).

The advancement of fMRI pattern decoding has provided further evidence showing that the PPC can hold distinctive representations for both low to mid-level visual features such as colors, motion directions, shapes, and abstract and high-level features, such as viewpoint invariant object identity, object category, semantic information, and target/non-target distinction [37–49] (Figures 1D–1G, 2A, 2C, 2D, and 4C). In one decoding study, we found that not only could view-invariant face identity representations be held in superior IPS, but that these representations closely tracked behaviorally perceived face identity similarity [43] (Figure 2A). In another decoding study, we observed object category representation in the PPC that was tolerant to changes in size and location, similar to that found in ventral object processing regions [49] (Figure 4C).

Figure 4.

Non-spatial visual representation and its interaction with spatial and attention-related processing in the human PPC. (A) By using both simple and complex shapes and varying display setsize in a VWM task, fMRI response amplitudes in the VWM-defined human superior IPS were shown to track behavioral capacity, exhibiting increased responses with increasing display setsize and plateauing at the maximum behavioral VWM capacity. Those in the human inferior IPS, however, increased with display setsize and plateaued at setsize 4 regardless of the complexity of the shapes encoded (adapted from [52]). Further work showed that the inferior IPS responses tracked object locations [52,77]. (B) Lesion sites of a patient with simultanagnosia showing bilateral lesions to large portions of the PPC (reprinted with permission from [80]). Patients with simultanagnosia suffer a debilitating visual impairment: they can perceive a single complex objects but are able to see only one object when confronted with two or more objects. (C) Attention and task differentially modulated object category representation in the human occipito-temporal and PPC regions (the same set of brain regions as those described in Figure 2C) (adapted from [47]). Here participants attended to either object shapes or colors of the images from 8 natural and manmade categories (the same 8 as shown in Figure 2C). The strength of color and object shape integration varied from partially overlapping (top), to overlapping but on separate objects (middle), to being fully integrated (bottom). fMRI MVPA decoding revealed strong bottom-up and top- down effects in the PPC. These effects could be best seen when the two dimensions that captured most of the representational variance among the 8 categories in superior IPS were projected onto a 2D surface using multi-dimensional scaling (MDS). Here the distance between each pair of categories on this surface reflects the similarity between them. In these MDS plots, the spread of object categories was much greater when shape, rather than color, was attended to and when the two features were not fully integrated. Additionally, object category representations were completely separated by task. The object category representational structure in superior IPS thus captures the differences among the object categories, the degree of feature integration, and the goal of visual processing. (FA - faces, BD - bodies, CT - cats, EL - elephants, SC - scissors, CH - chairs, CR - cars, and HO - houses)

Similar to monkey neurophysiology findings, visual representation held in human PPC can be sustained through VWM delay. Using fMRI response amplitude measures, a region in the superior part of the human IPS (here forth referred to as superior IPS for simplicity) has been shown to track the amount of visual information stored in VWM [50–53; see also 54]. fMRI pattern decoding further revealed the direct representation of VWM content in this brain region during the delay period [55]. Together, robust VWM decoding of color, orientation, simple shape, and viewinvariant abstract shape has been reported in the PPC [55–64] (e.g., Figure 1D). There is a close correlation between the VWM representation held in the PPC in fMRI decoding studies and behavior VWM performance as well as a causal connection between PPC damage and VWM performance [55,62,65].

Collectively, in both macaques and humans, across a variety of different measurements and behavioral paradigms, the PPC has been shown to be capable of representing a diverse array of visual features in both perception and VWM. These representations exhibit tolerance to image transformations such as size, spatial location and viewpoint, similar to those found in ventral visual cortex. These representations are unrelated to action-based processing in the PPC. In the studies described above, a motor response was either absent (e.g., in a passive viewing task or during the delay period of a VWM task) or was identical for the different stimulus conditions (e.g., in a 1-back repetition detection task).

In human studies, some have examined the entire PPC as a single brain region whereas others have reported action-independent non-spatial visual representation from multiple IPS regions. Nevertheless, only representations in IPS1 and IPS2 have been found to exhibit the most tolerance to changes in low-level visual features and those in the VWM capacity defined superior IPS have been shown to exhibit greater task modulation and resistance to distraction than other IPS regions [36,55,66]. Given the overlap between superior IPS and IPS1/IPS2 [67], it appears that action-independent non-spatial visual representation may be the most robust and converge in IPS1/IPS2/superior IPS in the human brain. In macaque studies, action- independent non-spatial visual representations are largely reported from area LIP. With the human IPS1/IPS2 being the homolog of macaque LIP [6], the same action- independent non-spatial visual processing converging zone thus appears to exist in both the human and macaque PPC (Figure 3).

Figure 3.

In both the macaque (left) and human (right) PPC, there is a posterior space-dominated processing zone and an anterior and medial action-dominated processing zone surrounding a visual processing converging zone that carries actionindependent non-spatial visual representations. The latter is additionally flanked laterally by regions involved in attention and cognitive control. Although the encoding of visual representation has been shown in multiple PPC regions, the regions illustrated here show the greatest action-independence, tolerance to low-level image transformations, and/or distractor resistance in visual representation. Likewise, although spatial signal is present throughout the PPC, caudal IPS in macaques and inferior IPS in humans appear to be primarily involved in space representation. (Brain drawings courtesy of M. Vaziri-Pashkam)

PPC’s ability to represent action-independent non-spatial visual information cannot be explained by its role in space, attention, and action-related processing. This ability also questions some of the previous evidence supporting PPC’s involvement in attention and action. Nevertheless, such visual processing in the PPC does not exist in isolation. Rather, it closely interacts with the other PPC functions.

Spatial and action-independent non-spatial visual processing in the PPC

The existence of action-independent non-spatial visual representation in the PPC is at odds with the original ventral-what and dorsal-where two-pathway framework and a space-centered view of the PPC function [4,7] as these only consider the representation of space in the PPC.

Anatomically, there appears to be some separation between spatial and action-independent non-spatial visual processing in the PPC. In the macaque LIP, despite the presence of strong spatial signals [24,68; see also 5], distinct and independent spatial and non-spatial signals coexist at both the single-neuron and population levels [68]. At the network level, topography maps in macaques co-localized quite poorly with the IPS regions defined by Lewis and Van Essen [69] according to architectonic subdivisions, such that area LIP only partially overlapped with one of the topographic maps [70]. The representation of space, especially 3D space, has been more prominently associated with responses in caudal intra-parietal (CIP) area in macaques and the homologous region in the inferior part of the human IPS (V3AA/3B/IPS0) [71–74], In a series of human fMRI studies, we further showed that the human inferior IPS can track the locations of up to four objects regardless of their complexity. This is quite different from the response of superior IPS described earlier, which tracks four or fewer objects depending on their complexity [52,53,75–77] (Figure 4A). This inferior IPS region overlaps more substantially with IPS topographic areas and exhibits greater spatial processing characteristics, such as hemispheric asymmetry and bias, than superior IPS [67,78]. Overall human inferior and medial PPC exhibits a stronger bias in space-based processing while superior and lateral PPC a stronger bias in non-space-based processing [67]. This results in the action-independent non-spatial PPC visual processing converging zone (i.e., superior IPS/IPS1/IPS2) to be bordered inferiorly/posteriorly by the PPC region predominantly involved in space-based visual processing (Figure 3 right). A similar anatomical arrangement can also be seen in the macaque PPC (Figure 3 left).

Despite the distinction between spatial and action-independent non-spatial visual processing in the PPC, these two types of processing are nevertheless tightly connected. The space-based attentional priority map has long been known to guide visual processing and representation in the PPC, with spatial processing deficits such as neglect and extinction severely impairing visual processing capacity [5,8–11]. In human patient studies, bilateral parietal lesions can often result in simultanagnosia, a debilitating visual impairment in which patients can only perceive a single object when confronted with two or more objects [79–81] (Figure 4B). Based on the response differences between inferior and superior IPS in the normal human brain, we proposed that when our visual system is confronted with a large number of items competing for representation, inferior IPS selects and individuates a fixed number of about four items by their spatial locations; superior IPS then encodes a variable number of the selected items depending on their complexity [82]. A predominantly space-based PPC mechanism is thus necessary for normal visual perception and closely interacts with the PPC mechanism involved in the representation of actionindependent non-spatial information. Together, they support the representation of multiple visual objects in visual perception.

Attention and action-independent non-spatial visual representation in the PPC

Detailed representations of action-independent non-spatial visual information in the PPC are not anticipated by a purely attentional account: Such an account would assume the PPC to be a control center and modulate representations formed in sensory regions, rather than directly carrying visual representations. The existence of action-independent non-spatial visual representation in the PPC also raises the possibility that some of the past evidence depicting attentional signals in the PPC could in fact reflect visual representation. For example, elevated LIP signals could reflect the encoding and representation of salient or behaviorally relevant visual stimuli rather than attention [83], and enhanced human IPS activation with increased visual processing load could likewise reflect an increase in visual representation rather than increased attentional demand [84]. Although the macaque LIP and the human IPS have been considered as key attentional areas [8,9], the existence of action-independent non-spatial visual representations in these areas suggests that they may be better considered as areas involved in representation but can at the same time be modulated by attention. This is similar to ventral visual areas whose responses can be modulated by attention but are not considered as attentional areas. One may argue that IPS could simply contain an attentional template tracking salient and behaviorally relevant visual events rather than directly carrying visual representation. However, as such a template would have to be specific for each distinct visual event or feature value, it is in essence no different from the definition of a representation. As Andersen and Cui [13] once argued: overextending the concept of attention to include a large variety of different neural processes could only weaken the concept and undercut our ability to understand other cognitive functions associated with the PPC.

Anatomically, the human IPS region involved in action-independent non-spatial visual representation is flanked by SPL and TPJ (TPJ on the right hemisphere only), regions predominately involved in the control of top-down and bottom-up visual attention and the shift of cognitive sets [9,12,85–87]. In macaques, although PPC regions involved in the shift of attention have not been fully established [12], areas lateral to IPS (areas 7a and DP) have been shown to exhibit transient activity during cognitive set shifts [88,89]. Thus, a similar anatomical separation may exist in both the macaque and human PPC regarding action-independent non-spatial visual representation and attentional control (Figure 3).

Despite the functional and anatomical distinctions between action-independent non-spatial visual representation and attention, they are nevertheless tightly connected, such that both top-down task demand and bottom-up saliency cues can significantly modulate visual representation in the PPC. In macaque LIP, neurons exhibited selectivity to features such as color and motion/shape category when such information was task relevant and attended [23,27,29,30]. Likewise, in the human PPC, fMRI pattern decoding of color, orientation, shape, motion, and category were obtained in some studies when these features were task relevant and attended [37– 39,45,63]. The human PPC is also under greater attention and top-down control and contains stronger visual responses and more robust distractor-resistant visual representations in VWM tasks than occipo-temporal regions [52,55,64,66,90].

Aside from top-down attentional factors, saliency - a bottom-up attentional feature - also impacts PPC visual responses, with salient and novel visual events greatly enhancing LIP responses [83]. This may account for the encoding of shape information in the macaque LIP and the human PPC under passive viewing [22,35,36,91], and the representation of task-irrelevant distractors in human superior IPS during VWM delay [55]. It is interesting to note that highly salient information is virtually impossible to ignore even when it is task-irrelevant, especially when its processing poses no measurable cost and/or when an observer is not fully occupied. In everyday vision, salient visual events are often information rich with biological significance and may become highly relevant at the next moment (e.g., a ball flying towards you). Perhaps for this reason, and because of our curiosity to continuously seek new information from the environment [92], it is advantageous and adaptive for the PPC to encode salient external visual events even when they are irrelevant to the task at hand. Besides saliency, a task-irrelevant feature may also be encoded in the PPC when it interacts with the task-relevant feature through location- and object- based attention selection mechanisms [41,47,48].

Bottom-up saliency and top-down task demand, as well as other attentional effects, are not mutually exclusive; rather, there are ample occasions in which they may jointly affect PPC visual responses. This can be best seen in recent macaque LIP studies in which top-down and bottom-up, as well as space- and feature-based, attention jointly modulated color and motion direction encoding in LIP neurons [27,28; see also 93]. Likewise, in a human fMRI decoding study, we found that bottom-up stimulus effect (i.e., the similarity among object categories and the integration between color and shape features) and top-down task demand (i.e., whether shape or color was task relevant) can both modulate the strength of object category representation in the PPC [47; see also 41] (Figure 4C).

Overall, action-independent non-spatial visual representations in the PPC are highly gated by attentional factors, making them dynamic and adaptive to the behavioral context of the visual input.

Action-related processing and action-independent non-spatial visual representation in the PPC

As discussed earlier, the representation of action-independent non-spatial visual information in the PPC is separable and independent from action-related processing. The existence of such visual representation in the PPC also raises the possibility that some previously reported action-related PPC responses may reflect goal-directed visual processing needed for, but not uniquely associated with, action. This can be seen in the delayed movement task in which a prolonged (often blank) delay period is imposed between the initial viewing of the visual target and the subsequent action execution, a task that necessarily engages visual representation and, when the delay period is blank, VWM. In a human fMRI decoding study using this paradigm, multiple parietal and frontal regions were shown to exhibit preparatory brain activity related to action intentions towards visual objects [94]. Some of these responses likely reflected the encoding of the task-relevant object information independent of action planning. Some of the PPC activations reported in other studies involving the encoding of location information in action planning could likewise reflect the representation of spatial information needed for, but independent of, action planning [95,96]. Thus, there exists an asymmetrical relationship between visual representation and action in the PPC: While the representation of visual information in the PPC can be independent of its role in action, the execution of visually guided actions necessarily engages the processing of relevant visual information.

Anatomically, the human PPC areas involved in action-independent non- spatial visual representation (i.e., superior IPS/IPS1/IPS2) are bordered anteriorly by the PPC regions involved in eye movement, grasping and reaching-related action processing [14–16] (Figure 3). Likewise, in macaques, LIP is bordered anteriorly and medially by AIP, MIP and the parietal reach region associated with grasping and reach [17,18]. Although it has been argued that human IPS1/IPS2 and macaque LIP may contain saccades-related signals [17,19], LIP inactivation studies have produced very mild impairment in saccade generation [93], It is argued that LIP does not directly control the saccadic eye movements, but exerts some influence through its connections with regions such as the frontal eye field and superior colliculus [93].

The PPC in adaptive visual processing

By analyzing action-independent non-spatial visual representation in the PPC in detail, it is clear that this PPC function is both distinctive from the other PPC functions, and at the same time closely interacting with them. Given that each PPC function taps into a basic and fundamental aspect of mental operation, how should we understand these diverse PPC functions with respect to each other? Can we form a framework that accommodates each distinctive function while at the same time provides us with an integrated and structured view of the PPC?

Based on the functional and anatomical connections between action- independent non-spatial visual representation in the PPC and the PPC’s other major functions as presented above, here I propose that perhaps we can bring together these diverse PPC functions through PPC’s unique contribution to adaptive visual processing. Linder this framework, the PPC contributes to adaptive visual processing by selecting, representing and sustaining salient or task-driven visual information (both spatial and non-spatial) to guide thoughts, solve problems, and, if needed, support the execution of appropriate actions (Figure 5). Within this framework, attention selection through both spatial and non-spatial mechanisms is a critical gating mechanism: by closely interacting with information inflow, it determines what may or may not be selected for further processing. PPC’s ability to directly and flexibly represent a diverse array of visual information provides the needed buffer to retain information that has been selected during both perception and VWM. Finally, action planning, involving both spatial and motor-related processing, utilizes the information that has been selected, represented and maintained in the PPC to guide the execution of timely and appropriate motor acts, serving as one of the end goals of adaptive visual processing. This framework partially echoes a recent analysis focusing primarily on the macaque LIP [93]. The present framework complements that analysis by considering multiple PPC subregions together and by examining its application to the human brain.

Figure 5.

The PPC in adaptive visual processing. Given the close functional and anatomical connections between action-independent non-spatial visual representation in the PPC and PPC’s role in attention, action, and space-related processing, these diverse PPC functions may be brought together through PPC’s unique contribution to adaptive visual processing. Within this framework, attention selection through both spatial and non-spatial mechanisms is a critical gating mechanism that determines what may or may not be selected for further processing. PPC’s ability to directly and flexibly represent a diverse array of visual information provides the needed buffer to retain information that has been selected during both perception and VWM. Action planning, involving both spatial and motor-related processing, utilizes the information that has been selected, represented and maintained in the PPC to guide the execution of timely and appropriate motor acts, serving as one of the end goals of adaptive visual processing. Visual information may be further processed and manipulated in the PFC and feedback signals from the PFC may also modulate processing in the PPC.

As task-driven visual processing often leads to the execution of an appropriate action, it is advantageous for action and visual representation to be tightly connected in the PPC. Indeed, previous analysis has focused primarily on PPC’s role in sensorimotor transformation [13,20,21,97]. Action, however, is just one possible outcome of adaptive visual processing, as not all visual processing necessarily leads to an immediate or overt action. As humans, we think much more than we act. In fact, the complex information processing occurring prior to motor planning is one of the factors that distinguish us from the other primates. The present framework thus departs from previous frameworks by emphasizing the direct representation of rich visual information in the PPC in both perception and VWM, and the dissociation between such visual processing in the PPC and sensorimotor transformation.

Does the adaptive visual processing framework provide an appropriate synthesis of the diverse functions associated with the PPC, or is it merely a restatement of the list of the PPC functions? Given that each existing PPC function taps into a basic and fundamental aspect of mental operation, a framework that overextends one function to encompass all or replaces all existing functions with something entirely different would lose utility. Thus, any synthesized view of the PPC would have to include its diverse functions. The present framework does not just list the different PPC functions, but describes how they may be configured to uniquely support adaptive visual processing based on the functional and anatomical evidence reviewed here: it tells us what the “whole” may look like by describing how the “parts” can fit together, thereby providing a more integrated and structured understanding of the functions of the PPC.

Is adaptive visual processing the right overarching framework to unite the different PPC functions? Visual processing needs to be both invariant, to provide us with a stable representation of the visual environment, and adaptive, to allow us to interact flexibly and efficiently with the external world. Direct comparisons of non- spatial visual processing between the occipito-temporal cortex (OTC) and the PPC reveal that while the content of visual representations is largely invariant to attention and task in the OTC, it is under greater task/attentional control in the PPC [3, see also 45,47,98]. Anatomically, the OTC is relatively isolated from attention and task regions, enabling representations formed in the OTC to more faithfully track the quality of the visual input that is general as well as context and task invariant [99]. In contrast, as reviewed above (Figure 3), within both the human and macaque PPC, there is a posterior space-dominated processing zone and an anterior and medial action-dominated processing zone surrounding a visual processing converging zone that carries robust action-independent non-spatial visual representations. The latter is additionally flanked laterally by regions involved in attention and cognitive control. This anatomical configuration enables the gating, representation and storage of salient and task-relevant visual information, and the subsequent utilization of such information to guide thoughts and behavior. These PPC functions are key characteristics of an adaptive visual processing system. As such, an adaptive visual processing framework, I would argue, provides a useful summary description of the diverse functions associated with the PPC.

The PPC, together with the PFC, forms the task-positive [100] or multiple demand (MD) network [101], a set of brain regions long known to be involved in the performance of a variety of tasks including abstract problem solving beyond sensory- motor transformation. As such, regions in PFC may also be involved in adaptive visual processing through functions such as cognitive control, task rule interpretation and maintenance, and complex informative processing and decision-making. The PPC thus is not the only brain region involved in adaptive visual processing. Nonetheless, through the distinctive functions it carries and the interactions among these functions, the PPC makes its unique and specific contribution to adaptive visual processing unlike any other region in the primate brain.

Compared to the adaptive visual processing framework, the task-positive/MD framework provides a broader and more general and global summery of a set of brain regions without detailing the functions of specific PPC subregions and their interactions. For this reason, the task-positive/MD framework, although necessarily including adaptive visual processing, is limited in helping us understand and integrate the diverse functions associated with the PPC at the functional and anatomical resolutions examined in the present Review. On the other hand, however, a detailed understanding of how adaptive visual processing is supported by the PPC can greatly enrich our knowledge of the task-positive/MD framework.

To the extent that a vast majority of the studies in the literature have used visual stimulation in their examinations of PPC functions, adaptive visual processing provides an appropriate framework for integrating findings from these studies. Nevertheless, PPC’s involvement in adaptive visual processing does not imply that adaptive visual processing is the only function of the PPC. In addition to visual information, online processing of non-visual information, such as auditory and sematic information, has also been documented in the PPC [102,103]. This suggests that the PPC may serve as an adaptive information processing center beyond vision. It is presently unknown whether or not the diverse PPC functions described here interact equally well with other input modalities as they do with visual input. Some of the PPC functions described here and their anatomical locations are likely input modality dependent while others may not. Future research is needed to address this thoroughly.

Aside from the PPC functions described here, a few other PPC functions have also been documented, such as its representations of number and proportion [104] and confidence [105]. These are features that are not directly available from the sensory input but require the accumulation and abstraction of such input. Perhaps PPC’s ability to accumulate information in VWM gives rise to its ability to represent these more abstract features. There also exists an intriguing parallel between the IPS regions involved in processing externally generated visual information and lateral PPC regions involved in processing information retrieved from long-term memory (Box 1). Future explorations and refinements of the framework proposed here to accommodate additional findings associated with the PPC and its interaction with the PFC through both feedforward and feedback processing (see Figure 5, not discussed here due to space limitations) will undoubtedly stimulate new areas of research and bring a more complete understanding of the function of the PPC (see Outstanding Questions).

Box 1. The PPC’s involvement in long-term memory retrieval/recognition.

Areas in lateral IPS and angular gyrus (AnG) have been shown to be involved in long-term memory (LTM) retrieval and recognition related processes [106,107]. It is thought that lateral IPS tracks differences in item memory strength or “familiarity”, whereas AnG tracks explicit recollection. Hutchinson et al. [108] proposed that, during recognition memory decisions, lateral IPS could serve as a “mnemonic accumulator” of signals from the medial temporal lobe to facilitate memory retrieval- related decision process (see also [106]). Such an accumulator function echoes the VWM function associated with superior IPS. Using fMRI pattern decoding, the strength of AnG activity patterns during encoding has been shown to reflect the content and quality of memory encoding and could predict the success of later memory retrieval [109]. AnG has also been shown to contain detailed representations of the retrieved information from LTM [110], Anatomically, lateral IPS and AnG appear to be at least nearby regions to, if not partially overlapping with, superior IPS. Could these regions be part of the same network mediating the representation of task-relevant visual information, with some receiving and retaining externally generated input and others receiving and retaining internally generated or retrieved information? This is an intriguing possibility that awaits further investigation and will provide further insights regarding how task relevant information, whether from external or internal sources, may be represented in the PPC.

Outstanding Questions.

• Does PPC’s ability to carry visual representation in both perception and VWM give rise to its ability to represent number/proportion and confidence?

• Can PPC serve as an adaptive information processing center for the online processing of non-visual information, such as auditory and semantic information?

• Do other input modalities interact with the different PPC subregions similarly as visual input does? Which PPC subregions process input modality dependent or independent information?

• Can we extend the framework proposed here to include lateral PPC regions to account for the processing of both externally generated visual information and that retrieved from long-term memory?

• Given the close interaction between the PFC and the PPC during attentional control, action planning, as well as processing and manipulation of visual information, how does the PPC interact with the PFC during adaptive visual processing?

Concluding Remarks

Given the diverse set of PPC findings accumulated over the years, understanding the precise function of the PPC has become more elusive. Flere by examining the existence of action-independent non-spatial visual representation in the PPC and its relationship with PPC’s role on space, attention and action-related processing, I propose that we may incorporate these diverse PPC functions into a single coherent and overarching framework through PPC’s unique contribution to adaptive visual processing. This allows us to both better appreciate each distinctive PPC function and enable us to form a more integrated and structured view regarding PPC’s overall role in vision, cognition and action. The approach taken here could help guide how PPC functions should be examined and interpreted in future studies.

Highlights.

• The primate posterior parietal cortex (PPC) was traditionally viewed as being involved in spatial, attention and action-related processing. A growing number of studies, however, have reported the direct representation of a diverse array of action-independent non-spatial visual information in the PPC during both perception and visual working memory.

• Functional and anatomical evidence shows that PPC’s direct representation of action-independent non-spatial visual information is both distinct and at the same time closely interacts with PPC’s role in space, attention and action-related processing.

• Given PPC’s involvement in multiple distinctive, yet interconnected mental operations, here I propose that we may understand these diverse PPC functions together through PPC’s unique contribution to adaptive visual processing.

• Understanding the functions of PPC through its contribution to adaptive visual processing both accommodates each distinctive PPC function and allows us to form a more integrated and structured view regarding PPC’s overall role in vision, cognition and action.