Visual Short-Term Memory Activity in Parietal Lobe Reflects Cognitive Processes beyond Attentional Selection

Abstract

Visual short-term memory (VSTM) and attention are distinct yet interrelated processes. While both require selection of information across the visual field, memory additionally requires the maintenance of information across time and distraction. VSTM recruits areas within human (male and female) dorsal and ventral parietal cortex that are also implicated in spatial selection; therefore, it is important to determine whether overlapping activation might reflect shared attentional demands. Here, identical stimuli and controlled sustained attention across both tasks were used to ask whether fMRI signal amplitude, functional connectivity, and contralateral visual field bias reflect memory-specific task demands. While attention and VSTM activated similar cortical areas, BOLD amplitude and functional connectivity in parietal cortex differentiated the two tasks. Relative to attention, VSTM increased BOLD amplitude in dorsal parietal cortex and decreased BOLD amplitude in the angular gyrus. Additionally, the tasks differentially modulated parietal functional connectivity. Contrasting VSTM and attention, intraparietal sulcus (IPS) 1–2 were more strongly connected with anterior frontoparietal areas and more weakly connected with posterior regions. This divergence between tasks demonstrates that parietal activation reflects memory-specific functions and consequently modulates functional connectivity across the cortex. In contrast, both tasks demonstrated hemispheric asymmetries for spatial processing, exhibiting a stronger contralateral visual field bias in the left versus the right hemisphere across tasks, suggesting that asymmetries are characteristic of a shared selection process in IPS. These results demonstrate that parietal activity and patterns of functional connectivity distinguish VSTM from more general attention processes, establishing a central role of the parietal cortex in maintaining visual information.

SIGNIFICANCE STATEMENT Visual short-term memory (VSTM) and attention are distinct yet interrelated processes. Cognitive mechanisms and neural activity underlying these tasks show a large degree of overlap. To examine whether activity within the posterior parietal cortex (PPC) reflects object maintenance across distraction or sustained attention per se, it is necessary to control for attentional demands inherent in VSTM tasks. We demonstrate that activity in PPC reflects VSTM demands even after controlling for attention; remembering items across distraction modulates relationships between parietal and other areas differently than during periods of sustained attention. Our study fills a gap in the literature by directly comparing and controlling for overlap between visual attention and VSTM tasks.

Introduction

The intraparietal sulcus (IPS) of posterior parietal cortex (PPC) has been implicated in a wide variety of cognitive tasks including visual attention (Corbetta and Shulman, 2002), visual short-term memory (VSTM; Todd and Marois, 2004; Xu and Chun, 2006), enumeration (Dehaene et al., 2003), perceptual decision-making (Freedman and Assad, 2006), long-term memory retrieval (Sestieri et al., 2017), and motor planning (Oristaglio et al., 2006). Disentangling discrete cognitive mechanisms is rendered difficult by numerous cognitive functions subserved by the PPC.

Studies of VSTM offer a clear example of the difficulty in isolating the relevant cognitive mechanisms within the IPS. Early studies indicated a central role of IPS in encoding and maintaining VSTM representations (Todd and Marois, 2004; Xu and Chun, 2006), demonstrating that IPS activity reflects the number of items held in memory. Recent investigations, however, show that multiple cortical areas carry information about remembered items (Harrison and Tong, 2009; Serences et al., 2009; Riggall and Postle, 2012; Ester et al., 2015; Bettencourt and Xu, 2016). Furthermore, change detection tasks, often used to study VSTM, rely upon attentional selection and evoke brain activation similar to attentional modulation (Huettel et al., 2001; Gazzaley et al., 2007). Even stimulus-specific delay period activity, arguably a VSTM function, might reflect sustained attention. Together, the distributed nature of VSTM and its overlap with visual attention processes calls into question whether VSTM activity in IPS reflects the maintenance or sustained attention necessary to hold objects in memory, or both.

Functional analyses of PPC can be impeded by the fact that spatially proximal areas may be functionally disparate. IPS contains multiple retinotopic maps of the visual field (Sereno et al., 2001; Schluppeck et al., 2005; Silver et al., 2005; Swisher et al., 2007; Silver and Kastner, 2009) that partially overlap areas subserving visual attention (Silver et al., 2005; Szczepanski et al., 2010) and VSTM (Sheremata et al., 2010). Memory load-dependent increases in IPS activation are generally accompanied by decreased activation in the angular gyrus, revealing its involvement in VSTM (Todd et al., 2005). Resting-state functional connectivity demonstrates negative correlations between the default mode network, including angular gyrus, and dorsal attention areas, including IPS (Fox et al., 2005). Therefore, retinotopically defined IPS, used as a seed in functional connectivity, can define the angular gyrus within individuals. Here, we retinotopically define IPS (IPS0-2) and visual cortical areas (V1–V3, V3A/B) and use functional connectivity patterns to identify the angular gyrus, frontal eye fields, and antieror IPS to objectively measure activity during VSTM and attention.

VSTM and attention also show representational similarities within the parietal cortex. Under passive viewing, right hemisphere (RH) and left hemisphere (LH) IPS regions exhibit symmetric contralateral visual field representations; however, during attention and VSTM tasks RH shows a hemispheric asymmetry by responding across the visual field, thus losing its contralateral bias (Sheremata et al., 2010; Szczepanski et al., 2010; Rosen et al., 2015; Sheremata and Silver, 2015; Jeong and Xu, 2016). It is unclear, however, whether this emergent hemispheric asymmetry and the resulting visual field asymmetries (Sheremata and Shomstein, 2014, 2017) reflect VSTM-specific processes beyond common attention processes.

To isolate memory-specific VSTM processes beyond sustained attention, it is essential to control for visual stimulation, spatial selection, and, importantly, sustained aspects of selection. Here, we directly measure activity and functional connectivity for visual attention and VSTM tasks with equivalent visual stimulation, selection, and sustained attention demands within retinotopically mapped IPS0-2. We hypothesize that if parietal activity during VSTM reflects memory-specific processes, then BOLD signal and functional connectivity should differentiate maintenance from sustained attention. To foreshadow, VSTM demonstrated activation in IPS1-2 and deactivation in the angular gyrus driven by factors beyond selection or sustained attention. VSTM also altered functional connectivity patterns in PPC relative to attention; IPS1-2 demonstrated greater connectivity with anterior portions of the dorsal attention network and the angular gyrus, but decreased connectivity with posterior regions of interest. These findings demonstrate VSTM-specific processing, beyond general attention demands, within PPC.

Materials and Methods

Participants

Twelve adults (5 males; age range, 21–36 years) with normal or corrected-to-normal vision and no history of neurological problems participated in the fMRI study. One author (S.L.S.) served as a participant. All participants gave informed consent, and the study was approved by the Institutional Review Boards of the George Washington University and Georgetown University. Each participant in the fMRI study was trained outside the scanner and then participated in a single 1.5 h fMRI session. Data from one participant were discarded because of incomplete cortical coverage of the regions of interest (ROIs).

Experimental design and statistical analysis

Stimuli and tasks.

Stimuli were generated using the PsychoPy software package (RRID:SCR_006571; Peirce, 2009) and were displayed using an LCD projector on a screen mounted at the back of the scanner bore. Participants viewed the stimuli via a mirror angled at 45°. All stimuli were presented on a mean luminance background. At least 1 d before the MRI session, participants participated in a training session for the localizer, attention, and VSTM tasks. Participants performed each task until they verbally indicated that they understood the task and performed at a level of 75% or above. On the day of the MRI session, participants again practiced each task just before entering the scanner bore.

Retinotopic localizer.

Circular patches of global motion were presented in opposite quadrants, upper left (UL)/lower right (LR) and upper right (UR)/lower left (LL), in an alternating block design. Each patch was centered at 6° eccentricity and subtended a 2° visual angle. Each block began with a 2 s cue in which the outline of the two patches appeared in either the UL and LR quadrants or the UR and LL quadrants. After the cue, participants covertly attended the two patches to determine whether a change in motion direction occurred in the left or the right visual hemifield for each 2 s trial, followed by a 1 s intertrial interval. After five trials (20 s blocks), the locations of the patches switched to the opposite visual quadrants. Stimuli were presented in each configuration (UL/LR and UR/LL) once per block, each block was presented four times, and each participant performed two runs of the retinotopic localizer. Retinotopically defined areas of the occipital [early visual cortex (EVC), including V1, V2, V3, and V3A/B] and parietal cortex (IPS0, IPS1, and IPS2) were defined by contrasting time points in which stimuli appeared in the first stimulus location (UL/LR) and in the second stimulus condition (UR/LL).

Main experiment.

The two tasks, attention and VSTM, were designed to rely on exactly the same visual stimuli while engaging two different cognitive mechanisms. Stimulus displays (Fig. 1) consisted of 14 bars presented along the perimeter of an imaginary circle with an eccentricity of 6° visual angle. The bars were presented against a mean luminance gray background surrounding a central fixation cross, and the position of the bars in each hemifield was offset 2.3° from the vertical meridian such that seven of the bars were presented in each visual field.

Figure 1.

Stimuli and paradigms for visual attention and VSTM tasks. Identical stimuli and attentional selection demands were used across the two tasks. In the VSTM paradigm (green labels), participants were asked to remember the orientation of target bars (here in yellow) while ignoring distractor bars (here in blue). During the VSTM delay, a series of brief stimulus presentations before the memory probe and response served as a dynamic mask. In the visual attention paradigm (blue labels), participants were asked to select three target bars (yellow) and ignore distractor bars (blue). During the brief presentations, participants had to monitor target bars for when all selected yellow targets were vertical (e.g., displays 1 and 4), and report the number of these presentations in the target interval (1–2 or 3–4; in this example, the response would be 1–2 as there were 2 targets). Targets frames are indicated by red arrows.

In both the attention and VSTM conditions, the visual stimuli were identical, requiring selection of visual information as well as sustained attention. On each trial, three of the bars were presented in a distinct color, yellow or blue, while the remaining bars were presented in the alternate color (i.e., yellow targets with blue distractors) and participants had to maintain spatial attention at those three spatial locations. After 500 ms, the bars disappeared followed by a quick succession of five stimulus presentations (150 ms each followed by a 150 ms blank screen). Afterward, the stimuli were again presented for 750 ms, during which time the participants were asked to make a response. Each participant performed six runs, three of VSTM and three of visual attention. Each run consisted of four blocks with six trials per block.

Because retinotopic IPS areas are sensitive to visual drive (Swisher et al., 2007), the number of visual stimuli was kept constant for all conditions. Each run (5 min, 10 s) consisted of two task-active blocks in each visual field and two passive-viewing blocks in each visual field, pseudorandomly chosen for each run. Each block (38 s) began with a cue (6 s) instructing the participants whether to perform the task or to passively view the stimuli and the location of the target stimuli (left or right). In the “left” blocks, the targets were presented to the left, whereas all of the stimuli on the right were distractors. In the “right” blocks, the targets were presented to the right, whereas all of the stimuli on the left were distractors.

VSTM task and attention task.

In the VSTM task (Fig. 1, green labels), participants were asked to perform an orientation change detection task. Upon presentation of the initial stimuli (“sample”), participants were asked to attend to the locations of the target stimuli and maintain their orientation in visual short-term memory while ignoring the rapid serial visual presentation of stimuli (“dynamic mask”) during the retention period. Upon the final stimulus presentation (“memory probe”), participants were asked to report whether the stimuli were identical to the first presentation (button 1) or there was a change in the orientation of one of the bars (button 2).

In the attention task (Fig. 1, blue labels), participants were asked to maintain fixation at the central fixation cross while directing attention to the locations indicated by the target color bars presented in the initial stimulus (“spatial cue”). When the rapid serial visual presentation stream appeared (“target interval”), they were asked to monitor how often all of the target bars were presented vertically and to respond when the final stimulus (“response cue”) appeared (button 1 if one or two all-vertical presentations, button 2 if three or four all-vertical presentations). Participants were instructed to exclude the spatial cue or the response cue in their response, but to direct their attention toward the target stimuli during the spatial cue.

A key aspect of the experimental design for the two tasks is that the paradigm ensured the following: (1) that visual stimulation was identical between the VSTM and attention tasks; (2) that in the sample display of VSTM or the spatial cue display of the attention task, participants spatially selected the same stimuli in a behaviorally relevant manner (Fig. 1, three yellow bars); (3) that during the dynamic mask set of displays for the VSTM task or the target interval set of displays for the attention task participants had to sustain attention to either selected items in a memory array or the items in the visual display; and (4) in the memory probe display for VSTM task or the response cue display of the attention task participants responded with either one of the two buttons. The differentiating aspect of the two tasks was the cognitive processes required during the dynamic mask and the target interval, such that during the dynamic mask, in addition to engaging sustained attention, participants actively maintained the remembered items across distraction.

Behavioral analysis.

Participants' performance for VSTM was measured in terms of capacity. We used Pashler's K (Pashler, 1988) for measuring capacity, which is defined as K = SS × (HR − FA)/(1 − FA), where SS is the set size, HR is the hit rate, and FA is the false-alarm rate.

fMRI acquisition and analysis.

MRI scanning was performed on a Siemens Tim Trio 3 tesla scanner equipped with a 12-channel head coil at the Center for Functional and Molecular Imaging at the Georgetown University Medical Center. High-resolution (1.0 × 1.0 × 1.0 mm) T1-weighted images were obtained for surface reconstruction. In the IPS localizer and in the main experiment, T2*-weighted, gradient echo, echoplanar images (repetition time = 2 s; echo time = 30 ms; voxel size = 3 mm isotropic voxels; 20% skip, slices = 38; duration: IPS localizer = 2 min, 54 s; main experiment = 5 min, 10 s) were acquired. Functional scans were acquired using automated motion correction performed by prospective acquisition correction (Thesen et al., 2000).

fMRI general linear model analysis.

Intensity normalization was performed before signal averaging [Dale et al., 1999; Fischl et al., 1999; Freesurfer (RRID:SCR_001622)]. Single-participant VSTM fMRI data were analyzed voxel-by-voxel using a general linear model (GLM) that included a predictor for each condition (e.g., task, visual field, and passive viewing). A t test was performed on each voxel to compare activation differences between conditions, and significance values were projected onto flattened cortical hemisphere representations. For the voxel-by-voxel analysis, the BOLD signal was modeled as a linear time-invariant system; a γ response function was assumed for each stimulus condition, with a delay of δ = 2.25 s and a decay time constant of τ = 1.25. An estimated response was generated by convolving the response function with the stimulus time course (i.e., each block) and minimizing the residual error (FS-FAST, Cortech). A random-effects group analysis was performed using surface-based averaging techniques (Fischl et al., 1999). For the ROI analysis, the percentage signal change data were extracted (from all time points for a block) and averaged by condition, across runs, to construct time course data for all voxels with a functionally defined ROI.

ROI definition.

Retinotopically defined areas of the IPS (IPS0-2) and occipital area V3A/B were identified using the reversal of quadrant preference using the IPS localizer within the intraparietal sulcus. Because our localizer did not demonstrate horizontal meridian reversals, we defined EVC (V1–V3) as any area activated by the retinotopic localizer posterior and inferior to V3A/B on the dorsal surface and inferior to V4 on the ventral surface constrained to the occipital cortex. More anterior parietal and frontal areas were not reliably identified with this retinotopic localizer.

Frontal eye fields (FEF), anterior IPS, and the angular gyrus were defined in individual participants using functional connectivity with an IPS1-2 seed region in each hemisphere, using the data across all attention and VSTM runs (Table 1). FEF ROIs were defined by finding an area of contiguous positive partial correlation, anatomically constrained by the intersection of the superior frontal gyrus and precentral sulcus. The anterior IPS was defined as a contiguous area in the parietal cortex anterior and lateral to IPS2. The angular gyrus was defined as a contiguous area in inferior parietal lobule that showed negative partial correlation and was bounded by the intraparietal sulcus, the supramarginal gyrus, and the superior temporal sulcus.

Table 1.

MNI coordinates for all functionally defined regions

ROI	MNI coordinates
	Mean	SD
V1-3
LH	−10, −90, 1	±2, 2, 6
RH	14, −86, 5	±1, 3, 4
V3AB
LH	−20, −91, 17	±4, 4, 5
RH	21, −87, 22	±4, 5, 5
IPS0
LH	−22, −82, 26	±2, 5, 5
RH	26, −79, 29	±3, 5, 4
IPS1
LH	−21,−74, 38	±2, 2, 6
RH	22, −72, 41	±4, 7, 5
IPS2
LH	−20, −68, 46	±4, 6, 6
RH	20, −69, 49	±5, 5, 4
Anterior IPS
LH	−27, −51, 53	±3, 3, 2
RH	25, −51, 55	±2, 4, 3
FEF
LH	−31, −4, 51	±6, 5, 4
RH	32, −1, 51	±5, 3, 2
Angular gyrus
LH	−43, −66, 28	±3, 3, 4
RH	48, −59, 29	±4, 5, 8

Functional connectivity analysis.

Functional connectivity was measured across each run (attention and passive viewing blocks during the attention task, VSTM and passive viewing blocks during the VSTM task). Following preprocessing, the data underwent multiple regression with nuisance regressors, including the average white matter signal, the average signal from the ventricular regions of interest, whole-brain signal, and 12 motion regressors (6 motion parameters from Freesurfer motion correction and their 6 temporal derivatives). We removed motion time points and applied a bandpass filter with 0.01 < f < 0.08 Hz. We then calculated the average time course for retinotopically defined IPS1-2, combined, to create a seed region for functional connectivity analyses. We collapsed across attention and VSTM runs to define the FEFs, anterior IPS, and angular gyrus ROIs. The FEF was defined by the intersection of the superior frontal sulcus and the precentral gyrus, which showed a positive correlation with IPS1-2. The angular gyrus was defined as a contiguous area in the inferior parietal lobule that was negatively correlated with IPS1-2 across both tasks.

Results

Visual short-term memory and attention fMRI scans were performed varying the visual field location of memory targets (left and right visual field), while maintaining the overall pattern of visual stimulation (targets and distractors). Each block began with a cue indicating the visual field of the targets (left/right) and whether to perform the task or passively view the stimuli (Fig. 1). In the memory runs, participants performed an orientation change detection task while ignoring a dynamic mask. In the attention runs, participants deployed attention to the target bars and then kept track of how many intervals contained all vertical targets. Importantly, across VSTM and attention runs, stimuli and spatial selection demands were controlled.

Behavioral performance

VSTM behavioral performance was assessed using percentage correct (see Materials and Methods). A 2 × 2 factor ANOVA with task (attention, VSTM) and visual field (left, right) as within-subject factors and accuracy as a dependent measure demonstrated a main effect of task (F_(1,10) = 13.48, p = 0.0043, ANOVA), reflecting that the attention task (mean = 78 ± 11%) was more difficult than the VSTM task (mean = 91 ± 5%). This result was later replicated in a follow-up eye-tracking experiment (see below). Therefore, any brain activation that is greater during the VSTM condition than the attention condition is not attributable to greater task difficulty. There was neither a significant effect of target location (F_(1,10) = 9.34, p = 0.98, ANOVA) nor an interaction between task and visual field (F_(1,10) = 3.11, p = 0.11, ANOVA), demonstrating that neither attention nor memory performance was different across the visual hemifields.

Dorsal posterior parietal cortex

Using an external localizer, we were able to localize early visual cortex (V1–V3), as well as cortical area V3A/B, IPS0, IPS1, and IPS2 (Table 1). Consistent with greater map reliability in posterior IPS areas compared with more anterior areas of the dorsal attention network, the retinotopic localizer used did not reliably identify maps for IPS3–5 or FEF in all participants (Swisher et al., 2007), requiring us to use functional connectivity to identify the more anterior regions anterior IPS and FEF. IPS0-2 are highly consistent with reports from previous retinotopic mapping studies, and the center of mass for anterior IPS is centered near IPS5, extending from IPS3, on the inferior and medial border to areas more anterior and lateral than IPS5 (Kastner et al., 2007; Swisher et al., 2007; Szczepanski et al., 2010). No map structure was evident in FEF using our localizer, but importantly the coordinates were consistent with those reported previously (Kastner et al., 2007).

To assess whether dorsal posterior parietal cortex plays a role in VSTM beyond task demands that can be attributed to visual attention processing, we conducted a four-way ANOVA for ROIs in dorsal parietal cortex with factors of task (VSTM/ attention), ROI (IPS0/IPS1/IPS2/anterior IPS), hemisphere (left/right), and visual field (ipsilateral/ contralateral). Our results revealed a significant main effect of task (F_(1,10) = 5.96, p = 0.035, ANOVA) showing a higher BOLD response for the VSTM task compared with the visual attention task (Fig. 2a).

Figure 2.

Task modulation for attention and VSTM tasks. a, Across IPS there was an increase in modulation for VSTM compared with visual attention. b, In early visual cortex, greater modulation for attention than for VSTM, though not significant, argues against load differences between the tasks. Error bars reflect the SE of the difference of the means.

It is possible that differences between the tasks beyond specific VSTM task demands (i.e., task difficulty) led to overall greater activation across cortical regions for VSTM compared with visual attention. There are two arguments against this possibility. As mentioned above, if one uses accuracy as a measure of task difficulty, then the attention task was more difficult, yet activity in IPS was greater for VSTM compared with attention. Additionally, we tested whether early visual cortical areas also showed an increase for VSTM compared with visual attention (Fig. 2b). In contrast to dorsal parietal cortex, early visual cortex (V1–V3) showed similar activation for both tasks, with a weak trend toward higher activity for visual attention rather than VSTM, while V3A/B showed no effect (EVC, t₍₁₀₎ = 1.82, p = 0.098, paired t test; V3A/B, t₍₁₀₎ = 1.07, p = 0.31, paired t test). These results demonstrate that increased activity in posterior parietal cortex cannot be accounted for by generalized higher activity across visual cortical areas or by a simple task-difficulty explanation.

Across tasks, there was a main effect of ROI (F_(3,30) = 12.89, p = 0.000014, ANOVA). Paired comparisons demonstrated that the effect of ROI was due to a lower percentage signal change in posterior compared with anterior IPS regions (IPS0 < IPS1: t₍₁₀₎ = 5.53, p = 0.0083, paired t test; IPS0 < IPS2: t₍₁₀₎ = 4.32, p = 0.0015, paired t test; IPS0 < anterior IPS: t₍₁₀₎ = 4.47, p = 0.0012, paired t test; corrected for multiple comparisons). While there was no significant effect of hemisphere (F_(1,10) = 2.94, p = 0.12, ANOVA), there was a significant interaction between hemisphere and ROI (F_(3,30) = 3.71, p = 0.022, ANOVA), reflecting a trend for lower signal in the RH for IPS0 and IPS1, and a trend toward higher signal in the RH for IPS2 and anterior IPS. There was no significant difference in any area after correcting for multiple comparisons.

Importantly, there was a significant interaction between hemisphere and visual field (F_(1,10) = 5.40, p = 0.043, ANOVA), confirming previous reports of hemispheric asymmetries in dorsal parietal cortex during both visual attention (Szczepanski et al., 2010; Rosen et al., 2015; Sheremata and Silver, 2015) and VSTM (Sheremata et al., 2010; Jeong and Xu, 2016; Fig. 3); LH IPS exhibited strongly contralateral responses, while right hemisphere IPS exhibited bilateral responses. A significant interaction between ROI and visual field (F_(3,30) = 7.87, p = 0.00051, ANOVA) demonstrated a stronger contralateral bias in IPS0 compared with IPS2 (t₍₁₀₎ = 3.84, p = 0.0038, paired t test, corrected for multiple comparisons). Visual field showed no significant interaction with task, demonstrating that the pattern of hemispheric asymmetries (i.e., representational similarities discussed in the introduction) was similar in both VSTM and visual attention.

Figure 3.

Contralateral bias collapsed across attention and VSTM. The left, but not right, parietal cortex demonstrated a consistent contralateral bias. Error bars represent the SEM.

Eye-tracking control

To ensure that the difference in activation profiles between the VSTM and attention tasks was not driven by potentially different patterns of eye movements during the dynamic mask and “target period,” we collected behavioral and eye-tracking data for an additional six participants outside the scanner. This follow-up not only served to examine eye-movement patterns during the VSTM and attention tasks, but also allowed an opportunity to replicate the basic behavioral findings. A mixed-design ANOVA was conducted with condition (attention, VSTM) as a within-subject variable and session (fMRI, eye-tracking) as a between-subject variable, with accuarcy as the dependent measure. A main effect of task (F_(1,15) = 15.22, p = 0.001, ANOVA) was observed pointing to higher accuracies in the VSTM task (VSTM: VSTM: 88 ± 3%; attention: 83 ± 2%). Importantly, neither the effect of session (F = 0.0096, p = 0.92, ANOVA) nor the interaction with session (F = 2.39, p = 0.14, ANOVA) reached significance, suggesting that the original finding was replicated, albeit with a much smaller sample. Focusing on eye movements, participants held fixation well in both tasks and the pattern of eye movements did not differ across tasks (attention task, SD = 0.53°; memory task, SD = 0.58°; t₍₅₎ = 0.60, p = 0.58, paired t test). This result strongly suggests that greater activation of posterior parietal cortex during VSTM was not due to a difference in eye movement patterns between tasks.

Angular gyrus

In addition to the role of the dorsal parietal cortex in VSTM, deactivation of the angular gyrus has been shown to reflect the number of items held in memory (Todd et al., 2005). If deactivation in the angular gyrus is due to memory-specific processes, then greater deactivation should be seen for the VSTM compared with the visual attention condition. To determine whether deactivation in the angular gyrus simply reflects the attention demands required during VSTM, we ran an ANOVA for activity within the angular gyrus with factors of task, hemisphere, and visual field. A marginally significant main effect of task (F_(1,10) = 4.52, p = 0.059, ANOVA) reflected greater deactivation for VSTM compared with the visual attention task (Fig. 4). Because selective attention demands were controlled across the attention and VSTM task, deactivation during VSTM cannot be accounted for by attention-specific demands inherent in change detection tasks.

Figure 4.

Deactivation in angular gyrus. While both VSTM and visual attention led to deactivation of the angular gyrus, this modulation was stronger in the VSTM task. Error bars represent the SE of the difference of the means.

In addition, the ANOVA showed a significant main effect of hemisphere (F_(1,10) = 34.42, p = 0.00016, ANOVA), with greater deactivation in the left than right hemisphere. While this hemispheric difference is reversed compared with a previous report (Todd et al., 2005), VSTM significantly deactivated ventral parietal cortex across the left and right hemispheres in both studies. In the discussion, we delineate possible differences between the studies that could account for the difference.

Functional connectivity

Because areas of the dorsal parietal cortex showed a greater response for VSTM than could be accounted for by visual attention task demands alone, we asked whether parietal areas show a change in connectivity with other areas of the dorsal attention network. We hypothesize that if activity in IPS1-2 is integral to VSTM maintenance, then differences between attention and VSTM should be apparent in the functional connectivity of IPS regions with higher-order cognitive regions that lie more anteriorly within the dorsal attention network and/or with lower-order perceptual regions in occipital cortex. To examine network changes, we chose a seed defined by the union of retinotopically mapped areas IPS1-2. These areas were chosen to represent dorsal parietal cortex, while IPS0 was excluded because the more dorsal areas showed a significantly different response during VSTM compared with attention.

To determine whether memory and attention demands differentially affect bottom-up and top-down patterns of connectivity, we divided ROIs into occipital areas posterior to IPS1-2 [early visual cortex (V1-3), V3A/B] and more anterior regions of the dorsal frontoparietal network (anterior IPS, FEF). A three-way ANOVA with factors of hemisphere, task, and direction (posterior, anterior) was performed to determine the effects of task demands on partial correlation. Across tasks, there was a significant main effect of direction (F_(1,10) = 54.07, p = 0.000024, ANOVA), demonstrating greater connectivity between IPS1-2 and more anterior dorsal attention network ROIs compared with more posterior ROIs. A significant interaction between the direction of connectivity and task (F_(1,10) = 5.33, p = 0.044, ANOVA; Fig. 5a) demonstrated that this was driven by an interaction between weaker connectivity with posterior areas during VSTM compared with attention (VSTM = 0.21; attention = 0.23; t₍₁₀₎ = 1.22, p = 0.25, paired t test) and stronger connectivity with anterior areas during VSTM compared with attention (VSTM = 0.28; attention = 0.27; t₍₁₀₎ = 2.09, p = 0.08, paired t test).

Figure 5.

Functional connectivity using IPS1-2 seed. VSTM showed a different pattern of functional connectivity compared with visual attention. Specifically, there was decreased connectivity with more posterior ROIs and increased connectivity with more anterior ROIs. Furthermore, VSTM led to an increase in inverse correlation with the angular gyrus, an area of the default mode network. Error bars reflect the SE of the difference of the means.

Finally, a significant interaction among hemisphere, task, and direction of connectivity (F_(1,10) = 7.36, p = 0.022, ANOVA) reflected greater connectivity with anterior areas compared with posterior areas for VSTM in the left hemisphere (t₍₁₀₎ = 4.97, p = 0.00042, paired t test) but not the right hemisphere (t₍₁₀₎ = 0.73, p = 0.48, paired t test).

Although IPS0 did not show a significant difference between the attention and VSTM tasks, retinotopic mapping studies have shown that IPS0-2 shares a foveal confluence. It is therefore possible that IPS0 shares functional similarities to IPS1-2 and that including it in the seed might strengthen the patterns of functional connectivity. We therefore repeated the functional connectivity analysis using IPS0 as a seed. Only the main effect of direction of connectivity (posterior/anterior) reached significance (F_(1,10) = 13.29, p = 0.0044, ANOVA), indicating greater connectivity with occipital cortical areas compared with anterior areas of the dorsal attention network. The pattern of connectivity with IPS0 was not modulated by task (F_(1,10) = 0.19, p = 0.67, ANOVA), an interaction between task and direction (F_(1,10) = 0.0013, p = 0.97, ANOVA) or task and hemisphere (F_(1,10) = 0.25, p = 0.63, ANOVA). Therefore, the pattern of functional connectivity for IPS0 did not show any task modulation seen with IPS1-2, further validating our choice of excluding it from the functional connectivity analysis.

Functional connectivity between networks

GLM analyses presented above corroborate studies demonstrating a role for both IPS and areas of the ventral parietal cortex implicated in the default mode network (Fig. 5b). Functional connectivity can further our understanding by demonstrating whether these networks interact to support VSTM by determining the degree to which activity in IPS and the angular gyrus is inversely correlated. We therefore used the same IPS1-2 seed to investigate how activity in dorsal parietal cortex varies in relation to activity in the angular gyrus. To this end, we performed a two-way ANOVA with hemisphere and task factors. A main effect of task emerged (F_(1,10) = 14.16, p = 0.0037, ANOVA), demonstrating greater inverse correlation between these areas during VSTM (partial correlation = −0.06) compared with visual attention (partial correlation = −0.03). These findings demonstrate that VSTM task demands modulate interactions between IPS and the angular gyrus to support memory-related processes.

Discussion

Here, we compared VSTM with sustained attention processing in human PPC under conditions that equated visual stimulation and controlled for spatial attention. VSTM-specific activation was observed in parietal areas IPS1, IPS2, and anterior IPS, and VSTM-specific deactivation was observed in angular gyrus, a region of the default mode network. Although prior studies have reported activation in IPS (Todd and Marois, 2004; Xu and Chun, 2006; Sheremata et al., 2010; Sprague et al., 2014; Ester et al., 2015; Bettencourt and Xu, 2016) and deactivation in angular gyrus (Todd et al., 2005) during VSTM tasks, this is the first study that can confidently argue that corresponding IPS and angular gyrus activity is specific to VSTM cognitive processes that go beyond sustained attention. This VSTM-specific activation did not extend to visual cortical areas, with early visual cortex exhibiting a trend for greater activation during sustained attention. Functional connectivity analysis focused on IPS1-2 revealed a relative shift toward anterior connectivity during VSTM and posterior connectivity during the attention task; this observation is consistent with engagement of higher-order cognitive processes during working memory and greater interaction with perceptual mechanisms during selective attention. The present findings also replicate prior work demonstrating a hemispheric asymmetry within parietal lobe visual areas in which left hemisphere responses are strongly selective for the contralateral visual field, while right hemisphere responses are driven across the entire visual field (Sheremata et al., 2010; Szczepanski et al., 2010; Rosen et al., 2015; Sheremata and Silver, 2015; Jeong and Xu, 2016). Here, we observed this hemispheric asymmetry equally in both tasks. This indicates that the hemispheric asymmetry reflects attentional selection processes in both tasks and does not reflect any VSTM-specific effects.

Disentangling the processes underlying VSTM in IPS is crucial for understanding the role of the parietal cortex in cognition. Prior research has persuasively argued that visual spatial short-term memory involves attention-based rehearsal (Awh and Jonides, 2001; Postle et al., 2004; Chun, 2011) and selective filtering of distractors (Bettencourt et al., 2011); therefore, a complete dissociation of VSTM and attention processes is not expected. Moreover, prior work has demonstrated that IPS and superior parietal lobule more broadly are recruited by both VSTM and attention (Postle et al., 2004; Todd and Marois, 2004; Xu and Chun, 2006; Sheremata et al., 2010; Chun, 2011; Sprague et al., 2014; Ester et al., 2015; Bettencourt and Xu, 2016). Our study is the first to directly compare VSTM and visual attention, controlling for attentional selection processes (i.e., the number and location of targets selected amid distractors) essential for maintaining remembered items. By controlling for attention selection demands, we address a major gap in the literature, demonstrating that maintaining information in the absence of a visual stimulus requires cognitive processes subserved by the parietal cortex. Although our findings do not reveal any fine-scale regions within parietal cortex that are specific to either VSTM or attention, our results demonstrate a divergence in BOLD signal and functional connectivity that demonstrates functional biases at a broad scale with stronger VSTM recruitment in IPS and a shift toward attentional processing in occipital lobe regions.

Tuning of both individual neurons (Nieder et al., 2006) and MRI voxels (Harvey and Dumoulin, 2011; Harvey et al., 2013) has been shown to underlie enumeration or the estimation of the number of items in a set. Given that participants were asked to count the number of targets during the attention task, it is possible that our attention task uses the same underlying resources. However, it is unlikely that this component of the attention task can explain the difference between attention and VSTM found here. In addition, our attention task required participants to remember how often all of the targets were presented vertically and update the number mentally, two components associated with working memory (Baddeley, 2003). Because enumeration-related or working memory processes would increase the overall activation level in IPS, any additional processes occurring during the attention task would increase activity during attention and thus reducing, the overall difference between the two tasks.

While activation in IPS reflects the number of items stored in memory (Todd and Marois, 2004; Xu and Chun, 2006; Sheremata et al., 2010), recent studies have argued for a central role of the occipital cortex in VSTM (Harrison and Tong, 2009; Serences et al., 2009). These studies demonstrate that activity patterns in occipital cortex can be used to decode the contents of memory even while activation cannot be demonstrated above the baseline BOLD signal. The results of the present study establish that connectivity between IPS and more posterior cortical areas, including early visual cortex, is decreased during VSTM compared with visual attention. This suggests that VSTM maintenance, per se, does not produce an increase in connectivity with visual cortex beyond that due to attention demands; in fact, these areas show greater communication when attention is directed to physically presented stimuli compared with remembered stimuli. Therefore, it remains an open question as to the mechanism by which memory maintenance at the level of visual cortex communicates with IPS.

In contrast to the memory-specific processes in IPS, differences in contralateral visual field bias across the hemispheres in IPS did not distinguish attention and VSTM tasks. This is consistent with a recent study that demonstrated asymmetries only when participants were cued in advance to remember stimuli in the left or right visual field (Jeong and Xu, 2016). Therefore, hemispheric asymmetries are a defining characteristic of attentional selection in posterior parietal cortex. While it is well established that this attentional selection modulates visual processing in the occipital cortex, asymmetries are not apparent in the overall BOLD signal in occipital cortex (Sheremata and Silver, 2015). Additional analyses are needed to assess whether asymmetries in occipital cortex can be revealed by different analyses, such as forward encoding models. If no asymmetries are found in occipital cortex, theories of attentional modulation will need to account for the discrepant contralateral biases in parietal and occipital cortices.

Our results also demonstrate that VSTM task demands modulate the relationship between regions of the dorsal attention and default mode networks. Greater inverse connectivity was seen between IPS1-2 and the angular gyrus during VSTM compared with visual attention, demonstrating that IPS activity was interdependent with deactivation in the angular gyrus. These networks show inverse correlation during both resting-state and functional connectivity (Fox et al., 2005). Our findings suggest that load-dependent deactivation in the angular gyrus (Todd et al., 2005; Xu and Chun, 2006) is driven by VSTM load-dependent activation in IPS (Todd and Marois, 2005) rather than being coincidentally related. These findings point to dynamically interdependent networks whose communications are modulated by specific task demands, in this case VSTM maintenance.

The exact role of the default-mode network in cognition is one of the outstanding fundamental questions in cognitive neuroscience. While this network has been proposed to be deactivated during the performance of any task that requires external objects or events (Raichle, 2015), the task differences reported here argue for a more nuanced role specifically within the angular gyrus. Consistently, fMRI and brain stimulation studies have found a role for the right angular gyrus during attention tasks such as target detection (Shulman et al., 2003) and feature binding (Koivisto and Silvanto, 2012), respectively. Importantly, a neurophysiological study (Popa et al., 2009) demonstrated that local field potential power in the dorsal attention network and the default mode network shows both anticorrelations and correlations during visual attention tasks, suggesting that both antagonistic and cooperative interactions occur between these networks. Our findings, in conjunction with these previous studies, suggest that distinct signals originate in the angular gyrus and highlight the importance of teasing apart different functions within this area.

In conclusion, experiments reported here reveal important functional effects in PPC specific to the maintenance of visual information. Namely, retinotopic regions of IPS are recruited by VSTM processes beyond what can be attributed to spatial selection and sustained attention, to task difficulty, or to bottom-up stimulus drive. A complementary set of VSTM influences were also observed in the default mode network region in the angular gyrus, an area thought to show consistently decreased activity across tasks requiring monitoring of external objects or events. Cognitive mechanisms distinct to VSTM result in a shift toward interactions of frontoparietal interactions at a cost for interactions with occipital cortex. The present study, along with prior work indicating stimulus-specific delay period activity in IPS (Bettencourt and Xu, 2016), provide converging evidence in favor of a VSTM-specific role for IPS. The findings and vital controls of these studies support the early, but recently questioned, claims of a central role for IPS in visual short-term memory (Todd and Marois, 2004; Xu and Chun, 2006).