Abstract
Coordinated reach-to-grasp movements are often accompanied by rapid eye movements (saccades) that displace the desired object image relative to the retina. Parietal cortex compensates for this by updating reach goals relative to current gaze direction, but its role in the integration of oculomotor and visual orientation signals for updating grasp plans is unknown. Based on a recent perceptual experiment, we hypothesized that inferior parietal cortex (specifically supramarginal gyrus [SMG]) integrates saccade and visual signals to update grasp plans in additional intraparietal/superior parietal regions. To test this hypothesis in humans (7 females, 6 males), we used a functional magnetic resonance paradigm, where saccades sometimes interrupted grasp preparation toward a briefly presented object that later reappeared (with the same/different orientation) just before movement. Right SMG and several parietal grasp regions, namely, left anterior intraparietal sulcus and bilateral superior parietal lobule, met our criteria for transsaccadic orientation integration: they showed task-dependent saccade modulations and, during grasp execution, they were specifically sensitive to changes in object orientation that followed saccades. Finally, SMG showed enhanced functional connectivity with both prefrontal saccade regions (consistent with oculomotor input) and anterior intraparietal sulcus/superior parietal lobule (consistent with sensorimotor output). These results support the general role of parietal cortex for the integration of visuospatial perturbations, and provide specific cortical modules for the integration of oculomotor and visual signals for grasp updating.
SIGNIFICANCE STATEMENT How does the brain simultaneously compensate for both external and internally driven changes in visual input? For example, how do we grasp an unstable object while eye movements are simultaneously changing its retinal location? Here, we used fMRI to identify a group of inferior parietal (supramarginal gyrus) and superior parietal (intraparietal and superior parietal) regions that show saccade-specific modulations during unexpected changes in object/grasp orientation, and functional connectivity with frontal cortex saccade centers. This provides a network, complementary to the reach goal updater, that integrates visuospatial updating into grasp plans, and may help to explain some of the more complex symptoms associated with parietal damage, such as constructional ataxia.
Keywords: fMRI, functional connectivity, grasp, parietal cortex, saccades, transsaccadic integration
Introduction
We inhabit a dynamic visual environment, where the brain must simultaneously compensate for both afferent (externally driven) and reafferent (internally driven) sensory events, often using internal efference copies of our own motion (Sherrington, 1908; Sperry, 1950; von Holst and Mittelstaedt, 1950; Helmholtz, 1867). For example, parietal cortex plays an important role in updating reach goals in response to both unexpected changes in object location (Pisella et al., 2000) and internally driven changes in eye position (Batista et al., 1999; Khan et al., 2005). This internal compensation, likely using saccade efference copies, allows more precise aiming (Vaziri et al., 2006; Dash et al., 2016) and reaches toward targets that are no longer visible (Henriques et al., 1998; Fiehler et al., 2011). However, successful object manipulation also requires grasping: shaping of the hand to fit specific object attributes, such as shape and orientation (Jeannerod, 1984; Desmurget and Prablanc, 1997; Fabbri et al., 2016). In order to successfully coordinate reach transport and grasp (Marotta et al., 2003; Castiello, 2005), intended grasp location and orientation must remain linked and updated during saccades (Crawford et al., 2004; Fan et al., 2006). However, to date, the cortical mechanisms for transsaccadic grasp updating have not been studied.
Transsaccadic grasp updating could recruit the mechanisms for transsaccadic perception: the comparison and integration of visual information across visual fixations (Irwin, 1996; Prime et al., 2007; Melcher and Colby, 2008). Transcranial magnetic stimulation studies suggest that the frontal eye field (FEF) provides the saccade efference copy for transsaccadic integration in posterior parietal cortex (PPC) (Prime et al., 2010, 2011). A recent fMRI study showed that inferior parietal cortex (specifically, the supramarginal gyrus [SMG]) is sensitive to transsaccadic changes in visual stimulus orientation (Dunkley et al., 2016). Human SMG may be an expansion of primate lateral intraparietal cortex, which contains saccade, visual feature, and spatial updating signals (Gnadt and Andersen, 1988; Subramanian and Colby, 2014). Since the inferior parietal cortex is thought to mediate perception and action (Goodale and Milner, 1992; Rizzolatti and Matelli, 2003), we hypothesized that SMG might also play a role in transsaccadic updating of grasp orientation, using efference copy input from the FEF.
Successful grasp also requires the updating of sensorimotor plans. Several parietal regions have been implicated in the visuomotor transformations for grasp, including the anterior intraparietal sulcus (aIPS) (Murata et al., 2000; Monaco et al., 2010), superior parietal cortex (SPL) (Culham et al., 2003; Filimon et al., 2009), and superior parieto-occipital cortex (SPOC) (Gallivan et al., 2011; Rossit et al., 2013). Transcranial magnetic stimulation experiments suggest that SPOC is involved in early visuomotor transformations for reach goals (Vesia and Crawford, 2012; Monaco et al., 2014). Of these regions, the more anterior intraparietal regions have specifically been implicated in updating grasp in response to external visual perturbations (Glover et al., 2005; Le et al., 2014; Janssen and Scherberger, 2015). However, it is not known whether any part or all of these regions are specifically involved in transsaccadic grasp updating.
Based on this literature, we hypothesized that SMG and the parietal grasp network provide the visuomotor coupling for transsaccadic grasp updating, by integrating visual features with saccade signals from FEF. To test this model, we merged two previous event-related fMRI paradigms for transsaccadic integration (Dunkley et al., 2016) and grasp planning (Monaco et al., 2014) (see Fig. 1B). We then applied a specific set of criteria to identify regions involved in the integration of eye position and visual orientation signals for grasp updating: (1) these regions should be specifically sensitive to transsaccadic changes in required grasp orientation plans (see Fig. 1C1); (2) they should show saccade modulations during grasp preparation (see Fig. 1C2); and (3) these modulations should become progressively more grasp task-specific as the sensorimotor transformation advances (see Fig. 1C3). Finally, during grasp updating, these regions should show stronger functional connectivity, both with each other and the cortical saccade generator, during saccades compared with fixation.
Materials and Methods
Participants
To determine the appropriate number of participants (human) required for a sufficient effect size/level of power in this study, we reviewed the most relevant previous literature and then did a power analysis. The current experimental design was based on our previous fMRI studies of transsaccadic memory (Dunkley et al., 2016) and grasp orientation (Monaco et al., 2014). Thirteen participants were analyzed in the Dunkley et al. (2016) study and 14 in the Monaco et al. (2014) study. We have found previously that an additional motor response in the experiment increases the cortical activation in the posterior parietal ROIs for this study (Chen et al., 2014; Cappadocia et al., 2018), so we based our power analysis on the Monaco et al. (2011) study and chose their ROI that was closest to the posterior parietal activation that we expected. Specifically, we used the effect size (1.27, Cohen's d) from their left posterior intraparietal sulcus activation, along with the following parameters: (1) two-tailed t test option, (2) an α value of 0.05 (we had planned for one contrast), and (3) a high power value (0.98). Using these values in G*Power (Faul et al., 2009), we calculated that 13 participants would provide a sufficient actual power value (0.987).
In order to obtain a reliable dataset of 13 participants (after the exclusion criteria described in Analysis below), we had to test 17 participants. These were all graduate students from York University (Toronto), experienced with performing visual experiments and with no known neurologic disorders and normal or corrected-to-normal vision. These participants (7 females, 6 males) were all right-handed and were 26.5 ± 3.7 years of age (range 22–32 years). All participants provided written consent and were compensated financially for their time. The York University Human Participants Review Subcommittee approved all experiments.
Experimental setup and stimuli
Participants were asked to fill out an MRI screening form. Upon passing MRI screening, participants were informed about the task. Once they felt comfortable with what the experiment entailed, they were asked to assume a supine position on the MRI table, with their head in a six-channel coil tilted forward at a 20° angle (to allow for direct visibility of the objects) (Monaco et al., 2014). To obtain a complete signal, we also placed a four-channel coil anteriorly on the head (Monaco et al., 2014).
This experiment was conducted in complete darkness. In our setup, we had red fixation LEDs for participants to focus on during the entire duration of a trial. A fixation LED was placed to the left and right of the central stimulus (between 10° and 12° from the center of the stimulus to each LED) (Monaco et al., 2014). There was also a white LED that was used to illuminate the stimulus only when participants would grasp at a particular time point in each trial (see Fig. 1A,B). These LEDs were mounted onto a rotatable platform that was placed above each participant's pelvis. LEDs were held in place by MRI-compatible rigid tubes (which were made of many units to allow for movement of the overall tube to position the LEDs accordingly).
The stimulus that participants had to grasp was a 6°-long bar with rounded ends (see Fig. 1A,B) and centered on the platform. The bar could be rotated, but two MRI-compatible pins were placed in the surrounding area to ensure that the bar could only be oriented horizontally (0°) or obliquely (135°).
For each participant, right eye position was recorded using an infrared camera affixed to the right side of the MRI table (see Fig. 1A). Eye movement signals were recorded using iViewX software (SensoMotoric Instruments) for offline analysis. We recorded, using a hand camera (see Fig. 1A), the reaching and grasping movements of participants during each trial of every run.
Last, to reduce any motion artifacts in the imaging data, participants' upper arm and shoulder were immobilized using an MRI-compatible belt that was strapped down across their torso. Participants reached with their right hand and pivoted only from their elbow joint, with only the minimal rotation of the shoulder joint. Their right arm was supported with foam padding and sand bags to provide a comfortable height from which the arm could reach and grasp the object for the duration of the experiment. We also made sure that the addition of the padding was appropriate and allowed participants to reach and grasp the object appropriately.
General paradigm/procedure
Experiment
We used an event-related fMRI design to identify cortical regions involved in updating grasp orientation across saccadic eye movements. Specifically, we developed a behavioral paradigm that combined elements of (1) a transsaccadic orientation memory study, where participants viewed a briefly presented sine-wave grating, made a saccade, and then had to judge whether a second visual stimulus had the same or different orientation (Dunkley et al., 2016); and (2) a grasp orientation study, where the orientation of a grasp stimulus could remain the same or change just before the reach (Monaco et al., 2011). First, participants were placed in the MRI bore, and a comfortable reach distance was determined for placement of the grasp stimulus by moving the platform along the MRI table. Participants were then trained to reach and grasp this bar stimulus (see Experimental setup and stimuli) in response to a specific “go” signal (see Fig. 1A,B). At the beginning of each reach trial, participants rested their arm, bent at the elbow, on their abdomen in a position that was within comfortable reaching distance of the stimulus. Participants were instructed to use all digits of their right hand to grasp the center of the object (see Fig. 1A). Upon completing the grasp, participants returned their arm to the same resting position as before the reach.
Each trial started with the illumination of one of the two LEDs to the right and left of the central target. Then, the central target was illuminated for 250 ms. The target could be oriented at 0° or 135° (pseudorandomized and counterbalanced within and across runs). Participants were required to keep fixating for another 1.75 s. This first 2 s phase was referred to as the Stimulus Presentation phase (see Fig. 1B). After this period, participants kept fixating on the same LED for another 1.75 s (Fixate condition) or made a saccade to the other LED, which would be illuminated while the previous LED would be extinguished (Saccade condition). (Fixation occurred for the entirety of a trial only for Fixation trials, which were intermingled with Saccade trials.) Following this 1.75 s period, the object was illuminated for 250 ms. This was referred to as the Action Preparation phase (see Fig. 1B), when participants were expected to retain stimulus location and orientation information, and use this to prepare for a movement (Monaco et al., 2011; Chen et al., 2014; Cappadocia et al., 2018). The object could now be oriented in the same orientation as in the first illumination/presentation (Same Orientation condition, e.g., 0° orientation first and then, another 0° orientation; same for the 135° orientation) or a different orientation compared with the first (Different Orientation condition, e.g., 0° orientation first, followed by a 135° orientation, and vice versa). Participants were then given 4 s to reach out to grasp the object in its final orientation as described above (Action Execution phase) while still fixating the illuminated LED. Following this phase, the LED was set up for the next trial and participants had 16 s to rest while maintaining fixation (intertrial interval), so as to allow the BOLD signal to return to baseline. The illumination of the stimulus marked the beginning of each trial, whereas the end of the 16 s period of relaxation marked the end of the trial.
In order to create the different orientation conditions, one experimenter rotated the stimulus as needed in the scanner room, but out of the participant's view and in complete darkness. To reduce the possibility of participants predicting Different versus Same orientation based on sound feedback, the experimenter moved the stimulus away and back to its required orientation (also during Same Orientation conditions).
The design of the experiment consisted of a 2 (Gaze Position: Fixate or Saccade) × 2 (Gaze Fixation Location: Left or Right) × 2 (Object Orientation: 0° or 135°) design. This produced eight condition types, which were repeated 4 times within one run. There were six runs in total. As mentioned previously, the condition types were pseudorandomized and intermingled within each run and across runs.
Compared with our previous study (Dunkley et al., 2016), we used a shorter stimulus period (total of 2 s for each stimulus presentation) to match an acquisition time of 2 s, and to ensure a reasonably long run/experiment (given that a long intertrial interval is needed to allow the BOLD to return to baseline). Recent studies have suggested that this transsaccadic integration can occur on the order of tens of milliseconds (Prime et al., 2008; Dunkley et al., 2016; Stewart and Schütz, 2019). In addition, we chose a fixed intertrial interval (no jitter) because we did not investigate response timing in this study, and we wished to maximize our statistical power to detect transsaccadic integration signals (Dunkley et al., 2016).
Saccade localizer
To determine which regions are involved in the production of saccadic eye movements, we used a localizer that had a sequence similar to that of the experimental runs. This localizer comprised alternating periods of fixation and saccadic eye movements. First, a baseline of activity would be established as a result of participants fixating the illuminated LED for 18 s (two runs total of data were collected, where participants fixated the left LED first and right LED second, or vice versa). Then, every second for 6 s, the LEDs would alternate in illumination, resulting in saccades. After this, participants would then fixate the initial LED for 16 s. This fixation-saccade sequence was repeated 8 times in the localizer run. There was a last fixation period of 18 s.
Imaging parameters
We used a 3T Siemens Magnetom TIM Trio MRI scanner. The functional experimental data were acquired using an EPI sequence (TR = 2000 ms; TE = 30 ms; flip angle [FA]= 90 degrees; FOV = 192 × 192 mm, matrix size = 64 × 64 with an in-slice resolution of 3 mm × 3 mm; slice thickness = 3.5 mm, no gap) for all six functional runs in an ascending and interleaved manner. For the saccade localizer, an EPI sequence was also used to acquire the data sequences (TR = 2000 ms; TE = 30 ms; FA = 90 degrees; FOV = 192 × 192 mm, matrix size = 64 × 64 with an in-slice resolution of 3 mm × 3 mm; slice thickness = 3.5 mm, no gap). Along with functional data, a T1-weighted anatomic reference volume was acquired using an MPRAGE sequence (TR = 1900 ms, FA = 256 mm × 256 mm; voxel size = 1 × 1 × 1 mm3). For each volume of anatomic data obtained, 192 slices were acquired. For the experimental task, we collected 395 volumes of functional data for the experimental runs, where each volume comprised 35 slices. For the saccade localizer, we collected 98 volumes of functional data, where each volume comprised 35 slices.
Analysis
Behavioral data
We monitored eye position during the experiment and analyzed it offline, to verify that participants fixated on the appropriate LED and did not make any additional, unnecessary saccades during trials. Any trials showing inappropriate fixation or saccades were removed from additional analysis. Similarly, video data were analyzed offline to determine whether the participant grasped the object at the required time. Any trials during which any anomaly in grasping occurred (i.e., participant grasped the object too early or too late, etc.) were removed from further analysis by being designated as confound predictors in the GLM (see below). On this basis, eight trials across all participants (0.003%) were removed from the entire dataset (two trials each were excluded from 2 participants and one trial from another 4 participants).
Functional imaging data: experimental
A GLM was created for each run for each participant. A predictor was used as a baseline for the period of fixation at the beginning and the end of each run (Baseline), accounting for the first 18 s of each run and the 16 s intertrial interval. The initial 2 s (Stimulus Presentation phase) during which the object was illuminated and participants had to fixate an LED was assigned a predictor that indicated the location of the fixation LED (Adapt_LVF and Adapt_RVF if the fixation was on the right LED or left LED, respectively; left and right visual field for LVF and RVF, respectively). The subsequent Action Preparation phase (2 s) was assigned one of four predictors: Sacc_DiffFeature, Sacc_SameFeature, Fix_DiffFeature, or Fix_SameFeature for when participants made a saccade or fixated and, for each of these, whether the orientation of the object was the same (Same Orientation condition) or different (Different Orientation condition). The Action Execution phase was divided in two 2 s phases. There were four predictors for the first 2 s of the grasp event. These predictors were based on the direction of the preceding saccade and on whether the orientation of the object that was being grasped was the same in the Action Preparation phase as in the Stimulus Presentation phase (Same Orientation condition) or different (Different Orientation condition). Thus, the four predictors were as follows: Motor Execution_Sacc_DiffFeature, Motor Execution_Sacc_SameFeature, Motor Execution_Fix_DiffFeature, and Motor Execution_Fix_SameFeature. The following 2 s of the Action Execution phase were provided a Motor Execution predictor. These predictors comprised each GLM for each participant (BrainVoyager QX 2.8, Brain Innovation). Each predictor variable was convolved with a hemodynamic response function (standard 2-γ function model). GLMs were modified through the addition of confound predictors for eye movement or hand movement errors. If a GLM had >50% of the trials being modeled in the confound predictor, the GLM for that run was not included in the overall population level GLM (random effects GLM, RFX GLM).
Additionally, functional data for all runs across all participants were preprocessed (slice time correction: cubic spline, temporal filtering: <2 cycles/run, and 3D motion correction: trilinear/sinc). Data for runs that had abrupt motion of >2 mm were excluded from the RFX GLM and additional analysis. As a result, the data from 4 participants were excluded because more than half of the runs were unusable due to abrupt, excessive head motion of >2 mm. From the remaining 13 participants, 8 additional runs were removed (i.e., 256 trials of a possible 2496 across the 13 participants, or 10.3%). The anatomic data were transformed to a Talairach template (Talairach and Tournoux, 1988), and the functional data from the remaining 13 participants were coregistered using gradient-based affine alignment (translation, rotation, scale affine transformation) to raw anatomic data. Functional data were smoothed using an FWHM of 8 mm.
Functional imaging data: saccade localizer
For the preprocessing of functional data for the localizer, see Functional imaging data: experimental. On the basis of excessive head motion (>2 mm), one person's data were completely excluded, half of the data were excluded for a second person (runs where initial fixation was on the left), and half were excluded for a third person (runs where initial fixation was on the right). Using the remaining functional data, we ran an RFX GLM on the data for each of the localizers. For the saccade localizers, we had three predictors: a 9 s Baseline predictor, a 16 s fixation Fix predictor, and a 6 s saccade Sacc predictor. The results of the saccade localizer were used to identify which areas are involved in saccade production in our task specifically (see Figs. 2B, 4B).
Experimental design and statistical analysis
BrainVoyager (QX 2.8, Brain Innovation) was used for the analysis in this study. For each analysis, we derived data from the most appropriate experimental phase (i.e., Action Preparation phase, Action Execution phase), that is, when the relevant brain events would be expected. This included volumetric map contrasts for general grasp and saccade activation (Grasp Fixation – Baseline; Grasp Saccade – Fixation) from the Action Preparation phase (see Fig. 2), as well as the more specific hypothesis tests described below. This experiment was not designed to temporally separate BOLD signals from our task phases, so they could be influenced by other task phases, but these additional signals should cancel in our specific hypothesis tests.
In the above contrasts, the volumetric maps had p values Bonferroni-corrected according to the number of contrasts that were applied to the same dataset (i.e., p < 0.25 for two contrasts, corrected from p < 0.05). in addition, cluster threshold correction was applied to these data using the plugin provided by BrainVoyager that implements Monte Carlo simulations (Forman et al., 1995). In order to qualitatively visualize the data (see Figs. 2–5), we superimposed the surviving clusters onto the “inflated brain rendering of an example participant” for each analysis. Since this process often results in small anatomic distortions, we sometimes also include transverse slice renderings below in key points (see Fig. 3A). These data are only provided for visualization purposes: the following describes the objective procedures that we used for anatomic localization and hypothesis testing.
Localization of sites of interest
We hypothesized that (1) several specific cortical areas are involved in transsaccadic updating of grasp plans (see Introduction), and (2) to qualify for this role, they must pass three specific predictions (see Fig. 1C). In order to apply these predictions to specific sites, we first used the contrast [(Grasp Saccade Different Orientation – Grasp Saccade Same Orientation) – (Grasp Fixation Different Orientation – Grasp Fixation Same Orientation)], with a t statistic of 2.2 (p < 0.048), as implemented in BrainVoyager, to the Action Execution phase. This was designed to provide a preliminary dataset related to cortical signals that are feature modulated in a saccade-specific manner. Again, cluster threshold correction was applied to these data. In Results, we refer to the regions that survive as “clusters of activation” (i.e., see Fig. 3A).
Next, to localize specific anatomic sites within these clusters, we decreased the p values applied to the data until only peak voxels of activation remained within each of the clusters (Frost and Goebel, 2012; Lührs et al., 2017). These peak voxels were then used to determine the Talairach coordinates shown in Table 1. These coordinates were then fed into BrainVoyager Brain Tutor (Brain Tutor 2.5, Brain Innovation) to provide an initial estimate of the anatomic name of each “site,” which were then confirmed against previous conventions in the literature (see Table 1). We adjusted the thresholds to select active voxels within a maximum 1000 mm3 cubic area surrounding the peak voxel(s). The selected areas were then used to test our specific predictions (see Hypothesis testing).
Table 1.
Site name | Talairach coordinates |
Active voxels (n) | References | |||||
---|---|---|---|---|---|---|---|---|
x (mean) | y (mean) | z (mean) | x (SD) | y (SD) | z (SD) | |||
Left aIPS | −38 | −41 | 53 | 2.3 | 1.6 | 1.9 | 217 | Tunik et al., 2008; Singhal et al., 2013 |
Left mSPL | −13 | −51 | 54 | 2.4 | 2.3 | 2.7 | 642 | Tunik et al., 2008; Filimon et al., 2009 |
Right SMG | 49 | −40 | 48 | 2.0 | 2.2 | 2.1 | 361 | Dunkley et al., 2016 |
Right lSPL | 35 | −49 | 53 | 2.8 | 2.8 | 2.7 | 875 | Tunik et al., 2008; Filimon et al., 2009 |
Hypothesis testing
Once the sites of interest were determined, they were used to test our specific predictions (see Fig. 1C). Here, we used BrainVoyager to select a subcluster of activation around each peak voxel and extract the corresponding β weights. We then plotted these β weights in “bar graph” format that allowed direct visual comparisons to our predictions. For each prediction, the p value (0.05) was Bonferroni-corrected for the number of t tests conducted (i.e., relative to how many sites were tested).
First, we tested whether the overall statistical results of the volumetric map contrast described above [(Grasp Saccade Different Orientation – Grasp Saccade Same Orientation) – (Grasp Fixation Different Orientation – Grasp Fixation Same Orientation)] from the Action Execution phase held up for the specific anatomic coordinates selected as our sites of interest, using their β weights for direct comparison to Prediction 1 (see Fig. 1C1). In other words, we confirmed whether these specific sites of interest showed the same saccade-specific stimulus orientation modulations as the entire cluster. We then used their active voxels to extract β weights from two additional Preparatory Phase contrasts (when saccades occurred) to test Predictions 2 and 3: saccade-related activation (Grasp Saccade – Grasp Fixation) to test Prediction 2 (see Fig. 1C2), and task-specific saccade-sensitivity [(Grasp Saccade – Grasp Fixation) – (Localizer Saccade – Localizer Fixation)] to test Prediction 3. In other words, we tested whether these sites were modulated by saccades in our task, and whether those modulations were task-specific. Last, for each test of our predictions, t and p values, as well as effect size (calculated Cohen's d, using G*Power) (Faul et al., 2009) are provided.
Functional connectivity: psychophysiological interaction analysis
Finally, to determine the network of cortical regions that interact to update saccade signals during the grasp preparation, we conducted psychophysiological interaction analysis (Friston et al., 1997; McLaren et al., 2012; O'Reilly et al., 2012) on data derived from right SMG (seed region) from the Action Preparation phase. We used three predictors: (1) physiological component (z-normalized time courses obtained from the seed regions for each participant for each included run), (2) psychological component (predictors of the model were convolved with a hemodynamic response function), and (3) psychophysiological interaction component (multiplication of z-normalized time courses with task model in a volume-by-volume manner). For the task model produced for the psychological component, the Saccade predictors were set to a value of 1, whereas the Fixation predictors were set to a value of −1 all other and baseline predictors were set to a value of 0. Single design matrices were created for each participant for each included run. These were subsequently included in an RFX GLM (Friston et al., 1997) to determine functional connectivity between right SMG with each of these and associated sites.
Results
Task-related grasp and saccade modulations
Various studies have shown that humans can remember stimulus properties for several seconds, and use these to plan action until a Go signal is provided (Chen et al., 2014; Cappadocia et al., 2018). Our goal here was to examine the influence of a saccade on these signals, especially when it interrupts a change in the external world. To test this, we used a task (Fig. 1B) with three key phases: Stimulus Presentation (which begins with the original grasp stimulus orientation), Action Preparation (which included a saccade in 50% of trials, and ends with a Different or Same stimulus orientation that also acts as a Go signal), and Action Execution (where the actual reach and grasp occurs). By design, we expected brain activation to be dominated by: (1) visual signals during the Stimulus Presentation phase, (2) grasp preparation, saccade, and spatial updating signals during the Action Preparation phase, and (3) grasp motor signals and (in the case of Different stimulus orientations) grasp orientation updating during the Action Execution phase of this task. As noted in Materials and Methods, our experiment was not designed to distinguish the temporal events in this sequence (Cappadocia et al., 2017), but for each analysis we maximized the relevant signal by deriving data from the most appropriate task phase.
We begin with an overview of the activation derived from the Action Preparation phase (between first and second stimulus; Fig. 1B), where one might expect to find events most closely related to the saccade-related updating of the original grasp stimulus. First, we derived the overall task-related activity from this phase, by contrasting Grasp Fixation trials against their baseline activity (Fig. 2A). This revealed activation in a parietofrontal network, including right SMG and several well-established reach/grasp regions: aIPS, SPL, precentral gyrus (PCG; corresponding to primary motor cortex), and dorsal/ventral precentral sulcus (PCSd/PCSv; likely portions of these regions corresponding to dorsal and ventral premotor cortex, respectively) (Culham et al., 2003; Galletti et al., 2003; Castiello, 2005). In short, the initial stimulus (and likely subsequent events) evoked massive activity in the grasp network.
To detect whether these task-related signals were also modulated by saccades, we compared Grasp Saccade trials with Grasp Fixation trials derived from the Action Preparation phase (Fig. 2B, sky blue regions), and compared this with activity from our saccade localizer task (Fig. 2B, fuchsia regions). These two contrasts produced overlap in some cortical regions (e.g., right frontal cortex and SMG), but saccades also produced extensive superior parietal and occipital modulations in the grasp task, including aIPS and adjacent portions of SPL (Murata et al., 2000; Culham et al., 2003; Filimon et al., 2009; Monaco et al., 2010). However, these additional modulations could be related to various functions, such as updating reach goals (Batista et al., 1999; Khan et al., 2005), general aspects of eye-hand coordination (Vesia and Crawford, 2012), or expected sensory feedback (Culham and Valyear, 2006). To identify specific transsaccadic grasp updating activity, we used our a priori predictions (Fig. 1C1–C3) to localize and test specific sites of interest, as described in Materials and Methods and shown in the following analyses.
Interactions between saccade and orientation sensitivity
If our participants incorporated original object orientation into short-term memory and used this for grasp planning (Monaco et al., 2011; Chen et al., 2014), their brains should (1) update this information across saccades (Melcher and Colby, 2008; Dunkley et al., 2016), and (2) update this again when they saw the final object orientation (Wolf and Schütz, 2015; Fornaciai et al., 2018). Thus, the cortical response to the second stimulus should be modulated by the orientation of the first stimulus (Monaco et al., 2011), and some of these modulations should depend on changes in eye position. Specifically, we predicted that these sites should show an increased response to orientation changes in the Grasp Saccade condition and little or no increase in the Grasp Fixation condition (Fig. 1C1). Alternatively, if participants ignored the initial stimulus and waited for the final stimulus to plan the grasp, these modulations should not occur.
Based on previous literature, we hypothesized that this might involve both right SMG (Dunkley et al., 2016) and the intra/superior parietal grasp network (Glover et al., 2005; Le et al., 2014). To test this, we had to localize specific sites of interest and apply our three predictions (Fig. 1C). As a first step, we identified cortical regions that were sensitive to changes in grasp orientation (Different vs Same Orientation) that follow saccades. Specifically, we used a voxelwise contrast applied to the trials wherein a saccade or fixation occurred: that is, (Grasp Saccade Different Orientation – Grasp Saccade Same Orientation) – (Grasp Fixation Different Orientation – Grasp Fixation Same Orientation). Then, as described in Materials and Methods, we applied a cluster thresholding algorithm to isolate specific clusters of activation. This yielded four separate clusters of activation, spanning left and right anterior PPC (Fig. 3A). This suggests that several regions within anterior PPC show saccade-specific sensitivity to grasp stimulus orientation changes, but does not yet provide the anatomic specificity required to test our hypotheses.
Site localization and anatomic coordinates
To localize specific anatomic sites for our subsequent analysis, we identified the peak voxels of PPC clusters described in the previous section, and then extracted their Talairach coordinates (Table 1). According to BrainVoyager Brain Tutor 2.5 (Brain Innovation), these peak voxels correspond to right SMG, left aIPS, and left/right SPL, which we further identified as left medial superior parietal lobule (mSPL) (a medial portion of left SPL), and right lSPL (a lateral portion of right SPL, slightly posterolateral to right aIPS). This was confirmed against previous literature (Tunik et al., 2008; Singhal et al., 2013; Dunkley et al., 2016). These sites have been indicated as black/white dots superimposed on the voxel clusters in Figure 3A, and will henceforth be referred to as “putative transsaccadic grasp updating sites.” In the following two hypothesis sections, it is only the active voxels immediately surrounding these sites (within a 1000 mm3 cubic area) that were analyzed (see Materials and Methods).
Prediction 1: saccade-specific orientation sensitivity
Prediction 1 (Fig. 1C1) was that our putative transsaccadic grasp updating sites should be more sensitive to changes in grasp stimulus orientation, which occur across a saccade, as opposed to fixation. [We have already shown this for the voxel clusters used to localize these sites (Fig. 3A), so here we are simply confirming this for data derived from these specific anatomic coordinates (right SMG, left aIPS, left mSPL, and right lSPL), and converting the data into a bar-graph format for direct comparison with our prediction (Fig. 1C1).] To do this, we extracted β weights from the active voxels, including and surrounding the peak voxel(s) of these sites, again from the Action Execution phase. Figure 3B shows the orientation change sensitivity of these variables (Different – Same) for each of our four sites, contrasting the Saccade condition against the Fixation condition.
As expected, each followed the predicted pattern: higher orientation change sensitivity following saccades versus fixation. For statistical analysis of these data, we used an α value of 0.05; we tested Prediction 1 for our four transsaccadic grasp updater sites, so resulting p values were adjusted for multiple comparisons and assessed against a Bonferroni p value of 0.0125 (0.05/4) for statistical significance. All four sites showed significant saccade-specific responses to changes in stimulus orientation (t(12)SMG = 3.30, pSMG = 0.0032, effect size = 0.92; t(12)aIPS = 2.34, paIPS = 0.019, effect size = 0.65; t(12)lSPL = 5.49, plSPL = 0.000069, effect size = 1.52; t(12)mSPL = 3.04, pmSPL = 0.0051, effect size = 0.84). These p values remained significant after correction for multiple comparisons, with the exception of one site: aIPS. However, the original cluster associated with aIPS (Fig. 3A) did survive cluster threshold correction, and this was a key component of our hypothesis, so we retained this site for further analysis.
Predictions 2 and 3: site-specific saccade modulations and task specificity
To examine the influence of saccades on our putative transsaccadic grasp updating sites, we performed additional contrasts. Figure 4 shows (1) the overall activity over baseline during the Fixation condition (Fig. 4A), and (2) saccade modulations in both our task and saccade localizer (Fig. 4B), derived as in Figure 2B. The location of our putative transsaccadic grasp updating sites is indicated by the four black dots superimposed on the contrasts. All four sites (right SMG, left aIPS, and bilateral SPL) fell within these task-related regions of activation, as well as within or bordering on, regions of saccade modulation in the localizer task (Fig. 4B). We then used the sites of interest defined in Figure 3A to extract β weights from the latter data, to directly test Prediction 2 (greater activation during saccades compared with fixation) and Prediction 3 (greater modulation during the grasp task than during saccades alone, i.e., task-specific saccade modulations).
Figure 4C shows the application of Prediction 2 on β weights extracted from grasp-related activity in Figure 4B. All four sites showed significantly higher activity in the presence of saccades (t test statistics were assessed against a Bonferroni-corrected p value of 0.0125 [0.05/4 = 0.0125] for multiple comparisons for the four transsaccadic sites separately for each of Predictions 2 and 3), although SMG did not survive correction for multiple comparisons (t(12)SMG = 2.08, pSMG = 0.030, effect size = 0.58; t(12)aIPS = 3.85, paIPS = 0.0011, effect size = 1.07; t(12)lSPL = 4.53, plSPL = 0.00034, effect size = 1.26; t(12)mSPL = 7.27, pmSPL = 0.0000050, effect size = 2.02).
To test the task specificity of these modulations, we applied Prediction 3; that is, we tested whether our putative updating sites showed saccade modulations during the grasp task, but not during saccades alone (Fig. 4D). In this case, only aIPS and bilateral SPL showed significant task specificity (t(12)SMG = 1.34, pSMG = 0.11, effect size = 0.43; t(12)aIPS = 3.58, paIPS = 0.0030, effect size = 1.14; t(12)lSPL = 3.44, plSPL= 0.0037, effect size = 1.09; t(12)mSPL = 3.12, pmSPL = 0.0062, effect size = 0.99). This suggests a progression of grasp task specificity from SMG to the more superior motor regions.
Functional connectivity of SMG with saccade and grasp sites
Our analyses so far have confirmed our perceptual updating result for SMG (Dunkley et al., 2016), and extended this function to sensorimotor updating in aIPS and SPL for grasp; but do these sites participate in a coherent functional network for grasp updating? Based on our previous finding that right SMG is active for perceptual orientation updating (Dunkley et al., 2016), and its reappearance in the current grasp task, we hypothesized that SMG is a key hub for updating visual orientation across saccades, and that it would communicate with both saccade regions (for signal input) and grasp regions (for signal output) during our grasp task. To do this, we identified a seed site within the right SMG using our independent saccade localizer data, and performed a psychophysiological interaction analysis to examine which sites showed increased functional connectivity for Saccade compared with Fixation trials with SMG derived from data aligned with the Action Preparation phase (Fig. 5A-C). This resulted in three sites that survived cluster threshold correction: right PCSd (likely a portion corresponding to FEF), left medial, superior frontal gyrus (likely the supplementary eye field [SEF]), and SPL (including a cluster that overlaps with aIPS).
Discussion
In this study, we set out to identify the cortical regions associated with updating grasp plans during changes in gaze direction and/or object orientation. We reasoned that, to perform this function, the brain would have to integrate saccade signals in regions sensitive to visual orientation and/or grasp orientation updating. To identify these sites, we applied three specific criteria: specific transsaccadic sensitivity to orientation changes, sensitivity to intervening saccades versus fixation, and task specificity in these saccade modulations, at least in the more superior parietal grasp motor sites. We found four sites that met these criteria: right SMG, a site previously implicated in transsaccadic orientation perception (Dunkley et al., 2016), and three more dorsal sites that are associated with grasp correction (Prime et al., 2008; Vesia et al., 2010). Finally, with the use of task-related functional connectivity analysis with seed site SMG, we identified a putative network for saccades that includes parietal and prefrontal regions.
Transsaccadic updating of object orientation for grasp
Here, we hypothesized that SMG (Dunkley et al., 2016) would contribute to feature updating for grasp execution, whereas some part of other regions involved planning/updating grasp orientation (Murata et al., 2000; Monaco et al., 2014, 2015) would also be involved in the transsaccadic updating of orientation for grasp preparation. To test this, we compared orientation change specificity for saccades versus fixation during Action Execution, and found four sites (right SMG, left aIPS, and bilateral SPL) that fit this criterion and passed our standard statistical criteria. (Our right lSPL site was similar to left aIPS, but positioned more laterally and posterior.) We further found that all of these sites were modulated by saccades, although the motor task specificity of these modulations was clearer in aIPS and SPL. Finally, the laterality of these responses was consistent with our hypothesis (i.e., right SMG being consistent with the general role of right parietal cortex in spatial awareness) (Perry and Zeki, 2000), whereas left aIPS was opposite to the motor effector uses (the right hand). This supports a general-purpose role for right SMG in the transsaccadic updating of object orientation, and adds a more unique role for aIPS and SPL in updating grasp orientation.
SMG is a region that has largely been implicated in perception tasks, such as those requiring spatial processing of orientation (Kheradmand et al., 2015) and visual search (Eimer et al., 2011), or those requiring cross-modal spatial attention (Macaluso et al., 2000). In contrast, SPL possesses both saccade and grasp-preshaping signals (Filimon et al., 2009; Gallivan et al., 2011), making this an ideal site to update grasp plans. Our anterior SPL grasp updating sites excluded more posterior grasp areas, such as SPOC (Gallivan et al., 2011; Rossit et al., 2013), consistent with the idea that the latter is concerned with setting initial reach goals (Vesia et al., 2010; Vesia and Crawford, 2012), whereas the former anterior areas are involved in updating those goals (Glover et al., 2005; Le et al., 2014; Janssen and Scherberger, 2015). These updated signals might then be relayed to PMd (Tanné-Gariépy et al., 2002; Davare et al., 2006), which possesses both reach-only and intermingled saccade-reach populations of neurons (Filimon et al., 2009). Finally, aIPS is sensitive to object orientation information for grasp (Murata et al., 2000; Brouwer et al., 2009; Glover et al., 2012; Vesia et al., 2017). aIPS appeared twice in our analysis: first, in Figure 3, near the coordinates provided in some previous studies (Medendorp et al., 2003; Gallivan et al., 2011; Monaco et al., 2011) and second, clustered with SPL in our network analysis (Fig. 5). It is thought that populations of neurons in aIPS may process object features, such as its orientation to ultimately shape and orient the hand to match the object's shape and orientation (Monaco et al., 2014). Information related to grasping is then proposed to travel to PMv to engage specific reach/grasp-related neuronal populations to generate motor commands (Davare et al., 2006, 2009; Filimon et al., 2009). Thus, our result appears to be consistent with the known functions of these regions, and extends our understanding of how these functions might be linked to update grasp signals in the presence of saccades.
A putative network for transsaccadic updating of grasp plans
An important goal for this study was to understand how distributed cortical regions might work as a network to update grasp plans during saccades. Based on the computational requirements of this function, we hypothesized that such a network should involve: (1) regions specific to transsaccadic updating of orientation features, (2) saccade regions for oculomotor input, and (3) and grasp updating regions for motor output. Given our previous (Dunkley et al., 2016) and current results, we hypothesized that right SMG would play the first role (i.e., here, it would update object features across saccades during the Action Preparation phase so that these could be spatially integrated with new visual information for Action Execution), and chose this as the seed site for our functional connectivity analysis. As described in the Introduction, we expected prefrontal saccade regions to play the second role, and parietal grasp regions to provide the final role (based on our current results, aIPS/SPL). Indeed, this analysis revealed a functional network for saccades versus fixation involving right SMG, right SPL, right aIPS, right PCSd, and the left superior frontal gyrus. Together with the overlapping sites that fit the previous three criteria, this suggests a saccade-dependent network with the specific properties needed for updating grasp orientation.
Psychophysiological interaction analysis does not provide directionality; but based on the functional requirements of the task and known physiology of these regions, we conceptualized this network as shown in Figure 5D. PCSd likely corresponds to the right FEF (Luna et al., 1998; Krauzlis, 2005). The FEF is a key component of the cortical saccade generator (Krauzlis, 2005), and is known to provide reentrant feedback to earlier visual areas (Moore and Armstrong, 2003; Hamker, 2010). The superior frontal gyrus likely corresponds to the SEF (Grosbras et al., 1999; Krauzlis, 2005), which has reciprocal connections with FEF. Thus, FEF/SEF could be the source of saccade signals for SMG and the entire network. As discussed above, aIPS (Gallivan et al., 2011) and SPL are implicated in grasp planning/corrections, show saccade signals (Filimon et al., 2009; Filimon, 2010), and of course were already implicated in transsaccadic grasp updating in our other analyses. Thus, this putative network appears to possess all of the signals and characteristics that one would expect to find in a transsaccadic updating circuit during grasp preparation.
Eye-hand coordination is relatively understood in terms of the transport component of reach, but little is known about the integration of saccade and visual signals for updating grasp configuration across eye movements. We set out to identify a putative human grasp updater and found a remarkably consistent cluster of regions, including SMG and aIPS/SPL, (likely) receiving oculomotor inputs from prefrontal eye fields. This network provides the necessary neural machinery to integrate object features and saccade signals, and thus ensure grasp plans remain updated and coordinated with gaze-centered reach transport plans (Batista et al., 1999; Khan et al., 2005). These new findings have several general implications: First, this circuit might explain some of the various symptoms of constructional apraxia resulting from damage to the PPC (Heilman et al., 1986; Sirigu et al., 1996). Constructional apraxia is a disorder affecting complex manual tasks that involve the coding and updating of multiple objects (Smith and Gilchrist, 2005; Russell et al., 2010). Second, the role of the inferior parietal cortex in both transsaccadic perception (Dunkley et al., 2016) and grasp updating supports the notion that inferior parietal cortex (a very late phylogenetic development) has high-level visuospatial functions for both ventral and dorsal stream vision (Goodale and Milner, 1992). Finally, the various roles of specific parietal modules in spatial updating (Klier and Angelaki, 2008), visual feedback corrections (Medendorp et al., 2003), and (here) a combination of the two for action updating support a general role for parietal cortex for detecting, differentiating, and compensating for internally- and externally-induced spatial perturbations.
Footnotes
The authors declare no competing financial interests.
This work was supported by a Natural Sciences and Engineering Research Council Discovery Grant. B.R.B. was supported by the Natural Sciences and Engineering Research Council Brain-in-Action CREATE Program, and the Ontario Graduate Scholarship (Queen Elizabeth II Graduate Scholarships in Science and Technology). S.M. was supported by the Canadian Institutes of Health Research during experiments and later a Marie Curie Fellowship. J.D.C. was supported by the Canada Research Chair Program. We thank Joy Williams for help in collecting the data as the MRI technologist; and Dr. Xiaogang Yan and Saihong Sun for technical and programming support.
References
- Batista AP, Buneo CA, Snyder LH, Andersen RA (1999) Reach plans in eye-centered coordinates. Science 285:257–260. 10.1126/science.285.5425.257 [DOI] [PubMed] [Google Scholar]
- Brouwer AM, Franz VH, Gegenfurtner KR (2009) Differences in fixations between grasping and viewing objects. J Vis 9:1–24. [DOI] [PubMed] [Google Scholar]
- Cappadocia DC, Monaco S, Chen Y, Blohm G, Crawford JD (2017) Temporal evolution of target representation, movement direction planning, and reach execution in occipital-parietal-frontal cortex: an fMRI study. Cereb Cortex 27:5242–5260. 10.1093/cercor/bhw304 [DOI] [PubMed] [Google Scholar]
- Cappadocia AD, Chen Y, Douglas J (2018) Cortical mechanisms for reaches versus saccades: progression of effector-specificity through target memory to movement planning and execution. bioRxiv 415562 Available at 10.1101/415562. [DOI]
- Castiello U. (2005) The neuroscience of grasping. Nat Rev Neurosci 6:726–736. 10.1038/nrn1744 [DOI] [PubMed] [Google Scholar]
- Chen Y, Monaco S, Byrne P, Yan X, Henriques DY, Crawford JD (2014) Allocentric versus egocentric representation of remembered reach targets in human cortex. J Neurosci 34:12515–12526. 10.1523/JNEUROSCI.1445-14.2014 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Crawford JD, Medendorp WP, Marotta JJ (2004) Spatial transformations for eye and hand coordination. J Neurophysiol 92:10–19. 10.1152/jn.00117.2004 [DOI] [PubMed] [Google Scholar]
- Culham JC, Valyear KF (2006) Human parietal cortex in action. Curr Opin Neurobiol 16:205–212. 10.1016/j.conb.2006.03.005 [DOI] [PubMed] [Google Scholar]
- Culham JC, Danckert SL, DeSouza JF, Gati JS, Menon RS, Goodale MA (2003) Visually guided grasping produces fMRI activation in dorsal but not ventral stream brain areas. Exp Brain Res 153:180–189. 10.1007/s00221-003-1591-5 [DOI] [PubMed] [Google Scholar]
- Dash S, Nazari SA, Wang H, Yan X, Crawford JD (2016) Superior colliculus responses to attended, unattended, and remembered saccade targets during smooth pursuit eye movements. Front Syst Neurosci 10:1–12. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Davare M, Andres M, Cosnard G, Thonnard JL, Olivier E (2006) Dissociating the role of ventral and dorsal premotor cortex in precision grasping. J Neurosci 26:2260–2268. 10.1523/JNEUROSCI.3386-05.2006 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Davare M, Montague K, Olivier E, Rothwell JC, Lemon RN (2009) Ventral premotor to primary motor cortical interactions during object-driven grasp in humans. Cortex 45:1050–1057. 10.1016/j.cortex.2009.02.011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Desmurget M, Prablanc C (1997) Postural control of three-dimensional prehension movements. J Neurophysiol 77:452–464. 10.1152/jn.1997.77.1.452 [DOI] [PubMed] [Google Scholar]
- Dunkley BT, Baltaretu B, Crawford JD (2016) Trans-saccadic interactions in human parietal and occipital cortex during the retention and comparison of object orientation. Cortex 82:263–276. 10.1016/j.cortex.2016.06.012 [DOI] [PubMed] [Google Scholar]
- Eimer M, Muggleton NG, Taylor PC, Walsh V, Kalla R (2011) TMS of the right angular gyrus modulates priming of pop-out in visual search: combined TMS-ERP evidence. J Neurophysiol 106:3001–3009. 10.1152/jn.00121.2011 [DOI] [PubMed] [Google Scholar]
- Fabbri S, Stubbs KM, Cusack R, Culham JC (2016) Disentangling representations of object and grasp properties in the human brain. J Neurosci 36:7648–7662. 10.1523/JNEUROSCI.0313-16.2016 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Fan J, He J, Tillery SI (2006) Control of hand orientation and arm movement during reach and grasp. Exp Brain Res 171:283–296. 10.1007/s00221-005-0277-6 [DOI] [PubMed] [Google Scholar]
- Faul F, Erdfelder E, Buchner A, Lang AG (2009) Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. J Behav Res Methods 41:1149–1160. [DOI] [PubMed] [Google Scholar]
- Fiehler K, Schütz I, Henriques DY (2011) Gaze-centered spatial updating of reach targets across different memory delays. Vision Res 51:890–897. 10.1016/j.visres.2010.12.015 [DOI] [PubMed] [Google Scholar]
- Filimon F. (2010) Human cortical control of hand movements: parietofrontal networks for reaching, grasping, and pointing. Neuroscientist 16:388–407. 10.1177/1073858410375468 [DOI] [PubMed] [Google Scholar]
- Filimon F, Nelson JD, Huang RS, Sereno MI (2009) Multiple parietal reach regions in humans: cortical representations for visual and proprioceptive feedback during on-line reaching. J Neurosci 29:2961–2971. 10.1523/JNEUROSCI.3211-08.2009 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Forman SD, Cohen JD, Fitzgerald M, Eddy WF, Mintun MA, Noll DC (1995) Improved assessment of significant activation in functional magnetic resonance imaging (fMRI): use of a cluster-size threshold. Magn Reson Med 33:636–647. 10.1002/mrm.1910330508 [DOI] [PubMed] [Google Scholar]
- Fornaciai M, Binda P, Cicchini GM (2018) Trans-saccadic integration of orientation information. J Vis 18:1–11. [DOI] [PubMed] [Google Scholar]
- Friston KJ, Buechel C, Fink GR, Morris J, Rolls E, Dolan RJ (1997) Psychophysiological and modulatory interactions in neuroimaging. Neuroimage 6:218–229. 10.1006/nimg.1997.0291 [DOI] [PubMed] [Google Scholar]
- Frost MA, Goebel R (2012) Measuring structural-functional correspondence: spatial variability of specialised brain regions after macro-anatomical alignment. Neuroimage 59:1369–1381. 10.1016/j.neuroimage.2011.08.035 [DOI] [PubMed] [Google Scholar]
- Galletti C, Kutz DF, Gamberini M, Breveglieri R, Fattori P (2003) Role of the medial parieto-occipital cortex in the control of reaching and grasping movements. Exp Brain Res 153:158–170. 10.1007/s00221-003-1589-z [DOI] [PubMed] [Google Scholar]
- Gallivan JP, McLean DA, Valyear KF, Pettypiece CE, Culham JC (2011) Decoding action intentions from preparatory brain activity in human parieto-frontal networks. J Neurosci 31:9599–9610. 10.1523/JNEUROSCI.0080-11.2011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Glover S, Miall RC, Rushworth MF (2005) Parietal rTMS disrupts the initiation but not the execution of on-line adjustments to a perturbation of object size. J Cogn Neurosci 17:124–136. 10.1162/0898929052880066 [DOI] [PubMed] [Google Scholar]
- Glover S, Wall MB, Smith AT (2012) Distinct cortical networks support the planning and online control of reaching-to-grasp in humans. Eur J Neurosci 35:909–915. 10.1111/j.1460-9568.2012.08018.x [DOI] [PubMed] [Google Scholar]
- Gnadt JW, Andersen RA (1988) Memory related motor planning activity in posterior parietal cortex of macaque. Exp Brain Res 70:216–220. 10.1007/bf00271862 [DOI] [PubMed] [Google Scholar]
- Goodale MA, Milner AD (1992) Separate visual pathways for perception and action. Trends Neurosci 15:20–25. 10.1016/0166-2236(92)90344-8 [DOI] [PubMed] [Google Scholar]
- Grosbras MH, Lobel E, Van de Moortele PF, LeBihan D, Berthoz A (1999) An anatomical landmark for the supplementary eye fields in human revealed with functional magnetic resonance imaging. Cereb Cortex 9:705–711. 10.1093/cercor/9.7.705 [DOI] [PubMed] [Google Scholar]
- Hamker FH. (2010) The reentry hypothesis: linking eye movements to visual perception. J Vis 3:808–816. [DOI] [PubMed] [Google Scholar]
- Heilman KM, Rothi LG, Mack L, Feinberg T, Watson RT (1986) Apraxia after a superior parietal lesion. Cortex 22:141–150. 10.1016/s0010-9452(86)80038-7 [DOI] [PubMed] [Google Scholar]
- Helmholtz H von. (1867) Handbook of Physiological Optics. In: Southall JPCS. (ed), London: Dover reprint, 1963. [Google Scholar]
- Henriques DY, Klier EM, Smith MA, Lowy D, Crawford JD (1998) Gaze-centered remapping of remembered visual space in an open-loop pointing task. J Neurosci 18:1583–1594. 10.1523/JNEUROSCI.18-04-01583.1998 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Irwin DE. (1996) Integrating information across saccadic eye movements. Curr Dir Psychol Sci 5:94–100. 10.1111/1467-8721.ep10772833 [DOI] [Google Scholar]
- Janssen P, Scherberger H (2015) Visual guidance in control of grasping. Annu Rev Neurosci 38:69–86. 10.1146/annurev-neuro-071714-034028 [DOI] [PubMed] [Google Scholar]
- Jeannerod M. (1984) The timing of natural prehension movements. J Mot Behav 16:235–254. 10.1080/00222895.1984.10735319 [DOI] [PubMed] [Google Scholar]
- Khan AZ, Pisella L, Rossetti Y, Vighetto A, Crawford JD (2005) Impairment of gaze-centered updating of reach targets in bilateral parietal-occipital damaged patients. Cereb Cortex 15:1547–1560. 10.1093/cercor/bhi033 [DOI] [PubMed] [Google Scholar]
- Kheradmand A, Lasker A, Zee DS (2015) Transcranial magnetic stimulation (TMS) of the supramarginal gyrus: a window to perception of upright. Cereb Cortex 25:765–771. 10.1093/cercor/bht267 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Klier EM, Angelaki DE (2008) Spatial updating and the maintenance of visual constancy. Neuroscience 156:801–818. 10.1016/j.neuroscience.2008.07.079 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Krauzlis RJ. (2005) The control of voluntary eye movements: new perspectives. Neuroscientist 11:124–137. 10.1177/1073858404271196 [DOI] [PubMed] [Google Scholar]
- Le A, Vesia M, Yan X, Niemeier M, Crawford JD (2014) The right anterior intraparietal sulcus is critical for bimanual grasping: a TMS study. Cereb Cortex 24:2591–2603. 10.1093/cercor/bht115 [DOI] [PubMed] [Google Scholar]
- Lührs M, Sorger B, Goebel R, Esposito F (2017) Automated selection of brain regions for real-time fMRI brain-computer interfaces. J Neural Eng 14:016004–016014. 10.1088/1741-2560/14/1/016004 [DOI] [PubMed] [Google Scholar]
- Luna B, Thulborn KR, Strojwas MH, McCurtain BJ, Berman RA, Genovese CR, Sweeney JA (1998) Dorsal cortical regions subserving visually guided saccades in humans: an fMRI study. Cereb Cortex 8:40–47. 10.1093/cercor/8.1.40 [DOI] [PubMed] [Google Scholar]
- Macaluso E, Frith CD, Driver J (2000) Modulation of human visual cortex by cross-modal spatial attention. Science 289:1206–1208. 10.1126/science.289.5482.1206 [DOI] [PubMed] [Google Scholar]
- Marotta JJ, Medendorp WP, Crawford JD (2003) Kinematic rules for upper and lower arm contributions to grasp orientation. J Neurophysiol 90:3816–3827. 10.1152/jn.00418.2003 [DOI] [PubMed] [Google Scholar]
- McLaren DG, Ries ML, Xu G, Johnson SC (2012) A generalized form of context-dependent psychophysiological interactions (gPPI): a comparison to standard approaches. Neuroimage 61:1277–1286. 10.1016/j.neuroimage.2012.03.068 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Medendorp WP, Goltz HC, Vilis T, Crawford JD (2003) Gaze-centered updating of visual space in human parietal cortex. J Neurosci 23:6209–6214. 10.1523/JNEUROSCI.23-15-06209.2003 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Melcher D, Colby CL (2008) Trans-saccadic perception. Trends Cogn Sci 12:466–473. 10.1016/j.tics.2008.09.003 [DOI] [PubMed] [Google Scholar]
- Monaco S, Chen Y, Medendorp WP, Crawford JD, Fiehler K, Henriques DY (2014) Functional magnetic resonance imaging adaptation reveals the cortical networks for processing grasp-relevant object properties. Cereb Cortex 24:1540–1554. 10.1093/cercor/bht006 [DOI] [PubMed] [Google Scholar]
- Monaco S, Fattori P, Galletti C, Cavina-Pratesi C, Sedda A, Culham JC (2011) Functional magnetic resonance adaptation reveals the involvement of the dorsomedial stream in hand orientation for grasping. J Neurophysiol 106:2248–2263. 10.1152/jn.01069.2010 [DOI] [PubMed] [Google Scholar]
- Monaco S, Króliczak G, Quinlan DJ, Fattori P, Galletti C, Goodale MA, Culham JC (2010) Contribution of visual and proprioceptive information to the precision of reaching movements. Exp Brain Res 202:15–32. 10.1007/s00221-009-2106-9 [DOI] [PubMed] [Google Scholar]
- Monaco S, Sedda A, Cavina-Pratesi C, Culham JC (2015) Neural correlates of object size and object location during grasping actions. Eur J Neurosci 41:454–465. 10.1111/ejn.12786 [DOI] [PubMed] [Google Scholar]
- Moore T, Armstrong KM (2003) Selective gating of visual signals by microstimulation of frontal cortex. Nature 421:370–373. 10.1038/nature01341 [DOI] [PubMed] [Google Scholar]
- Murata A, Gallese V, Luppino G, Kaseda M, Sakata H (2000) Selectivity for the shape, size, and orientation of objects for grasping in neurons of monkey parietal area AIP. J Neurophysiol 83:2580–2601. 10.1152/jn.2000.83.5.2580 [DOI] [PubMed] [Google Scholar]
- O'Reilly JX, Woolrich MW, Behrens TE, Smith SM, Johansen-Berg H (2012) Tools of the trade: psychophysiological interactions and functional connectivity. Soc Cogn Affect Neurosci 7:604–609. 10.1093/scan/nss055 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Perry RJ, Zeki S (2000) The neurology of saccades and covert shifts in spatial attention: an event-related fMRI study. Brain 123:2273–2288. 10.1093/brain/123.11.2273 [DOI] [PubMed] [Google Scholar]
- Pisella L, Gréa H, Tilikete C, Vighetto A, Desmurget M, Rode G, Boisson D, Rossetti Y (2000) An “automatic pilot” for the hand in human posterior parietal cortex: toward reinterpreting optic ataxia. Nat Neurosci 3:729–736. 10.1038/76694 [DOI] [PubMed] [Google Scholar]
- Prime SL, Tsotsos L, Keith GP, Crawford JD (2007) Visual memory capacity in transsaccadic integration. Exp Brain Res 180:609–628. 10.1007/s00221-007-0885-4 [DOI] [PubMed] [Google Scholar]
- Prime SL, Vesia M, Crawford JD (2008) Transcranial magnetic stimulation over posterior parietal cortex disrupts transsaccadic memory of multiple objects. J Neurosci 28:6938–6949. 10.1523/JNEUROSCI.0542-08.2008 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Prime SL, Vesia M, Crawford JD (2010) TMS over human frontal eye fields disrupts trans-saccadic memory of multiple objects. Cereb Cortex 20:759–772. 10.1093/cercor/bhp148 [DOI] [PubMed] [Google Scholar]
- Prime SL, Vesia M, Crawford JD (2011) Cortical mechanisms for trans-saccadic memory and integration of multiple object features. Philos Trans R Soc Lond B Biol Sci 366:540–553. 10.1098/rstb.2010.0184 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Rizzolatti G, Matelli M (2003) Two different streams form the dorsal visual system: anatomy and functions. Exp Brain Res 153:146–157. 10.1007/s00221-003-1588-0 [DOI] [PubMed] [Google Scholar]
- Rossit S, McAdam T, McLean DA, Goodale MA, Culham JC (2013) fMRI reveals a lower visual field preference for hand actions in human superior parieto-occipital cortex (SPOC) and precuneus. Cortex 49:2525–2541. 10.1016/j.cortex.2012.12.014 [DOI] [PubMed] [Google Scholar]
- Russell C, Deidda C, Malhotra P, Crinion JT, Merola S, Husain M (2010) A deficit of spatial remapping in constructional apraxia after right-hemisphere stroke. Brain 133:1239–1251. 10.1093/brain/awq052 [DOI] [PubMed] [Google Scholar]
- Sherrington CS. (1908) Reciprocal innervation of antagonistic muscles. Thirteenth note. -On the antagonism between reflex inhibition and reflex excitation. Proc R Soc Lond 80:565–578. [Google Scholar]
- Singhal A, Monaco S, Kaufman LD, Culham JC (2013) Human fMRI reveals that delayed action re-recruits visual perception. PLoS One 8:e73629 10.1371/journal.pone.0073629 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Sirigu A, Duhamel JR, Cohen L, Pillon B, Dubois B, Agid Y (1996) The mental representation of hand movements after parietal cortex damage. Science 273:1564–1568. 10.1126/science.273.5281.1564 [DOI] [PubMed] [Google Scholar]
- Smith AD, Gilchrist ID (2005) Drawing from childhood experience: constructional apraxia and the production of oblique lines. Cortex 41:195–205. [DOI] [PubMed] [Google Scholar]
- Sperry RW. (1950) Neural basis of the spontaneous optokinetic response produced by visual inversion. J Comp Physiol Psycho 43:482–489. [DOI] [PubMed] [Google Scholar]
- Stewart EE, Schütz AC (2019) Transsaccadic integration is dominated by early, independent noise. J Vis 19:1–19. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Subramanian J, Colby CL (2014) Shape selectivity and remapping in dorsal stream visual area LIP. J Neurophysiol 111:613–627. 10.1152/jn.00841.2011 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Talairach J, Tournoux P (1988) Co-planar stereotaxic atlas of the human brain (1988) Thieme Medical Publishers, New York. [Google Scholar]
- Tanné-Gariépy J, Rouiller EM, Boussaoud D (2002) Parietal inputs to dorsal versus ventral premotor areas in the macaque monkey: evidence for largely segregated visuomotor pathways. Exp Brain Res 145:91–103. 10.1007/s00221-002-1078-9 [DOI] [PubMed] [Google Scholar]
- Tunik E, Ortigue S, Adamovich SV, Grafton ST (2008) Differential recruitment of anterior intraparietal sulcus and superior parietal lobule during visually guided grasping revealed by electrical neuroimaging. J Neurosci 28:13615–13620. 10.1523/JNEUROSCI.3303-08.2008 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Vaziri S, Diedrichsen J, Shadmehr R (2006) Why does the brain predict sensory consequences of oculomotor commands? Optimal integration of the predicted and the actual sensory feedback. J Neurosci 26:4188–4197. 10.1523/JNEUROSCI.4747-05.2006 [DOI] [PMC free article] [PubMed] [Google Scholar]
- Vesia M, Barnett-Cowan M, Elahi B, Jegatheeswaran G, Isayama R, Neva JL, Davare M, Staines WR, Culham JC, Chen R (2017) Human dorsomedial parieto-motor circuit specifies grasp during the planning of goal-directed hand actions. Cortex 92:175–186. 10.1016/j.cortex.2017.04.007 [DOI] [PubMed] [Google Scholar]
- Vesia M, Crawford JD (2012) Specialization of reach function in human posterior parietal cortex. Exp Brain Res 221:1–18. 10.1007/s00221-012-3158-9 [DOI] [PubMed] [Google Scholar]
- Vesia M, Prime SL, Yan X, Crawford JD, Sergio LE (2010) Specificity of human parietal saccade and reach regions during transcranial magnetic stimulation. J Neurosci 30:13053–13065. 10.1523/JNEUROSCI.1644-10.2010 [DOI] [PMC free article] [PubMed] [Google Scholar]
- von Holst E, Mittelstaedt H (1950) Das reafferenzprinzip. Sci Nat 37:464–476. [Google Scholar]
- Wolf C, Schütz AC (2015) Trans-saccadic integration of peripheral and foveal feature information is close to optimal. J Vis 15:1–18. 10.1167/15.16.1 [DOI] [PubMed] [Google Scholar]