Reach preparation enhances visual performance and appearance

Abstract

We investigated the impact of the preparation of reach movements on visual perception by simultaneously quantifying both an objective measure of visual sensitivity and the subjective experience of apparent contrast. Using a two-by-two alternative forced choice task, observers compared the orientation (clockwise or counterclockwise) and the contrast (higher or lower) of a Standard Gabor and a Test Gabor, the latter of which was presented during reach preparation, at the reach target location or the opposite location. Discrimination performance was better overall at the reach target than at the opposite location. Perceived contrast increased continuously at the target relative to the opposite location during reach preparation, that is, after the onset of the cue indicating the reach target. The finding that performance and appearance do not evolve in parallel during reach preparation points to a distinction with saccade preparation, for which we have shown previously there is a parallel temporal evolution of performance and appearance. Yet akin to saccade preparation, this study reveals that overall reach preparation enhances both visual performance and appearance.

1. Introduction

The visual brain prioritizes the processing of regions of the visual scene that are most salient (i.e. physically distinctive from the background; e.g. a ‘no trespassing’ sign on a palisade) and/or immediately relevant for behaviour (e.g. the next rung on the ladder as you climb over). In the acting observer, priority maps—which combine stimulus-driven salience and goal-driven relevance in a common signal—are thought to govern visual attention and the selection of appropriate actions [1–5].

The evidence for a link between goal-directed actions and visual attention is compelling for the case of saccadic eye movements. A number of saccade-related brain regions, including the lateral intraparietal area (LIP), the frontal eye fields (FEF) and the superior colliculus (SC), are thought to represent priority maps and all of them have been causally linked to attention-related changes in visual performance [6–12]. Moreover, saccade preparation results in an obligatory shift of attention to its target [13–19]. Indeed, microstimulation of neurons in FEF not only improves visual performance at the location corresponding to the stimulated site, but also increases the gain of visual responses of V4 neurons with overlapping receptive fields [20,21]. These findings suggest that a feedback signal originating in FEF (or other saccade-related areas) triggers presaccadic attention shifts by modulating the gain of visual responses in extrastriate visual cortex [22]. In agreement with this idea, we recently found a rapid enhancement of the apparent contrast of stimuli presented at the target of an upcoming saccade, concomitant with an improvement in visual performance [23]. These findings demonstrate that saccade preparation modulates appearance, much like covert attention [24–28], by changing the effective contrast of a stimulus. Perceived contrast, thus, could be considered a perceptual correlate of priority, as it results from bottom-up salience and behavioural relevance [24–28].

Saccades are a tractable model for the study of goal-directed movements, but their direct consequences for retinal input link them inextricably to visual processing. In this study, we tested whether and how goal-directed reaches, during which retinal input of the target does not change, alter visual performance and appearance.

Neurophysiological studies have investigated the link between attention and reach preparation by studying the effector specificity of areas known to underlie presaccadic attention shifts. The SC, for instance, has been implicated in visual target selection for both saccades and reaches [29]. By contrast, in the intraparietal sulcus, separate regions encode saccades (LIP) and reach movements (parietal reach region) [30,31]. Moreover, FEF visual neurons allocate attention to remembered target locations for saccades but not for reaches [32]. These results suggest that either attention is not allocated to reach targets or a distinct attentional mechanism underlies this allocation.

Behavioural studies have revealed clear attentional consequences of reach preparation [33–38] similar to those during saccade preparation [13–19,23]. Specifically, the identification of a target letter embedded in an array of distractor letters is best if the target's location coincides with the reach goal [33,36]. Importantly, given that in these two studies, the reach goal was not systematically related to the test stimulus location, their results imply that reach preparation causes an obligatory shift of attention to the target. However, such performance benefits may draw on different attentional resources than saccades, as the concurrent preparation of a saccade leaves the shift of attention to the reach target largely unaffected ([37], but see [38]). Moreover, varying the time between a movement cue and the presentation of the test array demonstrated that attention can be withdrawn from the target once the reach movement has been prepared, unlike for saccades [34]. Thus, it remains an open question whether reach preparation affects visual processing similarly to saccade preparation. In particular, given that presaccadic attention alters not only performance but also appearance [23], here we examined whether attention alters appearance in a similar way prior to reaches.

This is the first study to concurrently assess performance and appearance before visually guided reaching movements. We adapted the procedure of our recent study examining changes in visual performance and appearance before saccades [23]. In that study, observers compared the orientation and contrast of a test stimulus, appearing at the saccade target at different times before saccade onset, to a standard stimulus, presented before the saccade goal was cued. In this study, on each trial, observers placed their right index finger just below a fixation point at a touch screen's centre (figure 1). Two identical standard stimuli then flashed on either side of fixation, and following a cue to initiate a reach movement to one of the locations, a test stimulus appeared with equal probability either at the reach target or at the opposite location (figure 2a). Following the movement, observers reported the test's orientation (measuring performance) and perceived contrast (measuring appearance) relative to the standard stimulus, with a single touch of the screen (figure 2b). Gaze fixation was monitored throughout the trial. Using this procedure, we were able to investigate spatially selective changes in visual performance and appearance at different stages of reach preparation. Moreover, unlike in previous studies [34,35,38], the location of the test stimulus was unpredictable, allowing us to isolate the perceptual consequences of reach preparation from any strategic deployment of attention. A continuous increase in perceived contrast at the reach target would suggest that the movement preparation for saccades and reaches has similar consequences for both aspects of visual processing— performance and appearance.

Figure 1.

Reach set-up. Observers sat with their head stabilized and their arms comfortably positioned on an elbow cup. An eye tracker constantly monitored fixation, and a touch screen registered the finger's reach position.

Figure 2.

Experimental procedure, reach and test stimulus timing. (a) Sequence of events in each trial. As soon as gaze and reach were detected near their respective fixation marks, standard stimuli briefly appeared within two placeholders, 7.5° to the left and to the right of fixation, followed by a movement cue and, after a variable delay, a test stimulus that appeared unpredictably at either of the two stimulus locations. In the reach condition, the cue indicated the reach target (one circle below each stimulus location), and observers executed the reach quickly. In this example, the test stimulus is presented at the reach target. In the neutral condition, the cue pointed in both directions and observers maintained reach fixation. Times indicate durations of the depicted frames. (b) Perceptual report. When the observer executed an appropriate reach, a response screen appeared, asking observers to report in a single touch of the screen the orientation and contrast of the test stimulus (a thick outline post-cued its location) relative to the standard. As a result of touching the screen, the selected button turned white. (c) Density plot of reach latencies, stacked for the seven observers tested. Markers and error bars show individual means and standard deviations. (d) Stacked individual density plot of test offset times. We divided the distribution in four time windows before reach onset. The earliest bin collapses all trials in which the test stimulus disappeared earlier than 150 ms before the reach onset.

2. Results

(a). Reach performance

We defined the reach onset as the time at which the finger was lifted from the touch screen display and the landing site as the position at which the finger first touched the screen after the reach. Reaches were accurate, landing 1.25°±0.08° (mean±s.e.m.; Euclidean distance from the reach target) away from the reach target, undershooting the target (7.5° eccentricity) by 0.52°±0.11° in the horizontal dimension. Figure 2c shows density functions of individual reach latencies (time between cue onset and reach onset), stacked to emphasize the overall distribution of reach latencies, along with each observer's mean and s.d. The average reach latency was 278±13 ms.

Reach performance was either independent of or only mildly affected by the location, contrast and timing of the test stimulus. The average landing errors were similar for all 70 factorial combinations of the two test positions, five cue-test intervals (CTIs) and seven test contrasts, ranging from 1.12° to 1.35° (figure 3a). There were no main effects of test location, CTI or test contrast on landing errors (Fs < 2.15, ps > 0.10), but there was a significant interaction between test location and test contrast (F_6,24 = 2.92, p = 0.020, ) that emerged from non-monotonic differences. There were no other significant interactions: Fs < 1.25, ps > 0.2.

Figure 3.

Reach performance and test stimulus timing. (a) Average landing site error, (b) reach latency and (c) test timing relative to reach onset, plotted for each CTI, test location relative to the reach target and test contrast. Error bars are s.e.m. (Online version in colour.)

Overall, the average reach latencies were also similar for all 70 combinations of test positions, CTIs and test contrasts, ranging from 265 to 289 ms (figure 3b). A repeated three-way ANOVA on latencies yielded small but significant effects of test location (279 ms versus 276 ms at the reach location versus the opposite location; F_1,6 = 9.54, p = 0.021, ) and CTI (F_4,24 = 5.50, p = 0.003,), but no main effect of test contrast and no two-way interactions (Fs < 1.8, ps > 0.17). There was a three-way interaction (F_24,144 = 2.10, p = 0.004, ) owing to a (marginal) interaction of test location and test contrast at CTI of 106 ms (F_6,36 = 3.12, p = 0.015, ) but not for shorter or longer CTIs (Fs < 1.95, ps > 0.09). In sum, reach onset relative to the offset time of the test stimuli varied mainly as a function of CTI (figure 3c). In the reach-locked analyses of perceptual reports (see §2b,c), we accounted for all differences in test timing by devising neutral baselines for each time window before reach onset (defined in figure 2d). For each of these baselines, each combination of test stimulus properties (location, CTI and contrast) had the same statistical weight as in the corresponding reach condition (see §4e).

(b). Orientation discrimination performance

We estimated observers' objective visual performance in the orientation discrimination task, expressed as sensitivity d′. Figure 4a(i) shows performance, averaged across observers, as a function of test location and the CTI. The dashed black line shows the data from the neutral condition in which no reach was planned or initiated. A general comparison of conditions collapsed across time is presented in the grey shaded area (‘all data’). Figure 4a(ii) shows the difference of the two reach conditions from the neutral baseline; filled symbols highlight significant deviations. Horizontal bars between the two panels highlight significant differences between sensitivity at the reach target and the opposite location.

Figure 4.

Orientation discrimination performance, expressed as d′, as a function of test stimulus location relative to the reach target (target or opposite) and time of test stimulus presentation, relative to both (a) cue onset and (b) reach onset. In a(i) and b(i) we show the average data with s.e.m. In a(ii) and b(ii) we re-plot the data from panels a(i) and b(i) as the difference of the two reach conditions from their neutral baseline, with 95% CIs. We constrained all analyses to trials in which the test stimulus presentation had been completed before the reach (§4d). In (a) therefore, the last time bin (greater than or equal to 153 ms) pools the 153 and 200 ms CTIs, because observers with short reach latencies had very few trials long after the cue. In ‘all data’, we computed d′ by pooling trials from all CTIs. In (b), neutral baselines were computed separately for the target and opposite conditions, and plotted in the corresponding colours (dashed lines). Filled symbols highlight significant deviations from baseline. Lines in between (i) and (ii) highlight significant differences between the two reach conditions (target versus opposite). (Online version in colour.)

Across all trials, observers' sensitivity was higher at the reach target (blue) than at the opposite (red) location (mean±95% CI of Δd′ = 0.22±0.13, p < 10⁻³). This difference was largely owing to a cost in performance at the opposite location compared with the neutral condition (Δd′ = −0.16±0.12, p = 0.008). The performance difference emerged early after cue onset (12 ms CTI: Δd′ = 0.35±0.27, p = 0.010). A repeated-measures ANOVA conducted on the baseline-corrected data (figure 4a(ii)) confirmed that test location affected performance (target versus opposite; F_1,6 = 12.37, p = 0.013, ), but that neither performance per se (F_3,18 = 1.63, p = 0.22) nor the difference between the two locations (F_3,18 = 1.80, p = 0.11) evolved as a function of CTI.

We also analysed performance as a function of the time of test offset relative to the reach onset (figure 4b). To this end, we determined performance for test stimuli presented in one of four pre-reach time windows (figure 2d) and constructed separate neutral baselines for each reach condition (see §4e). A repeated-measures ANOVA conducted on the baseline-corrected data (figure 4b(ii)) confirmed an effect of test location (F_1,6 = 7.64, p = 0.033, ), showing better performance at the reach target than at the opposite location, and yielded a marginal effect of time window (F_3,18 = 2.97, p = 0.059, ) indicating an overall increase in performance relative to the baseline; there was no interaction (F_3,18 = 0.59, p = 0.78).

(c). Contrast appearance

In addition to orientation, observers also compared the contrast of the test stimulus to the standard stimulus, which was presented prior to the reach cue. Figure 5 shows the dynamics of perceived contrast as a function of test location and time, expressed as the point of subjective equality (PSE), the test contrast that the observer perceives to be identical to the contrast of the standard (50% ‘higher’ response). As in previous studies [23–25,28], the PSE will be used to index subjective experience of contrast (lower PSEs indicate higher perceived contrast of the test stimulus). To compute PSEs, we modelled the relation between test contrast and the proportion of ‘test higher than standard’ responses with cumulative Gaussian psychometric functions.

Figure 5.

Perceived contrast, expressed as PSEs, as a function of test stimulus location relative to the reach target and time of test stimulus presentation relative to both (a) cue onset, and (b) reach onset. Conventions as in figure 4. (Online version in colour.)

Across all trials, there was a clear effect of reach location on perceived contrast (ΔPSE = −0.072±0.019, p < 10⁻¹²). Immediately after cue onset, perceived contrast was higher than the standard, and this effect decreased gradually with time for the neutral condition and at the target location. Note that this might be a general phenomenon in two-interval tasks [23,39] and possibly result from a fading trace for the stimulus presented in the first interval (standard stimulus). More importantly for this study, the average PSE (figure 5a, ‘all data’) was significantly lower for the reach target than for the neutral baseline (ΔPSE = −0.043±0.018, p < 10⁻⁵); for the opposite location, it was significantly higher (ΔPSE = 0.029±0.016, p < 10⁻³). This indicates that the Gabor at the location of the reach target appeared to have higher contrast than that of the opposite location. A repeated-measures ANOVA conducted on the baseline-corrected data (figure 5a(ii)) confirmed an effect of test location (F_1,6 = 10.76, p = 0.017, ), which increased significantly with longer CTIs (interaction: F_3,18 = 7.32, p = 0.002, ); there was no main effect of CTI (F < 1). We conducted the same analysis for the reach-locked data (figure 5b(ii)) by computing separate neutral baselines for each reach condition (as we did for performance). We obtained similar main effects (test location: F_1,6 = 7.95, p = 0.030, ; time window: F < 1) but not the interaction (F_3,18 = 1.59, p = 0.17), hinting at a less consistent evolution of perceived contrast across observers, when analysed with respect to reach onset rather than relative to the onset of the movement cue. The magnitude of the PSE difference reached a maximum in the longest CTIs (153 ms or later, ΔPSE = −0.111±0.033, p < 10⁻¹⁰; figure 5a) and in the last time window (50–0 ms) before reach onset (ΔPSE = −0.107±0.042, p < 10⁻⁶, figure 5b).

3. Discussion

We investigated the impact of reach movement preparation on visual perception by simultaneously quantifying both an objective measure of visual sensitivity and the subjective experience of apparent contrast. Observers had better visual discrimination performance at the reach target than at the opposite location, consistent with earlier findings [33–38]. Remarkably, during reach preparation, the physical test contrast necessary to subjectively match the standard contrast decreased at the reach target relative to the opposite location. Taken together, these results indicate that the preparation of a reach enhances both performance and appearance at the reach target relative to other locations.

In this study, we observed better visual performance at the reach target than at the opposite location. This effect occurred immediately after cue onset—possibly owing to a relative advantage for the visual memory for the location that was part of the movement plan—and varied little as a function of time. Note that other studies have also found very early effects before both reaches [34,35] and saccadic eye movements [16,17,23]. In particular, our results are consistent with a previous study [35] in which performance was better at the target of a reach than elsewhere, independent of CTI (which varied between 80 and 320 ms), but differ from a previous study [37] that reported a monotonic increase in visual performance time locked to the reach onset. These authors used large arrays of stimuli (12 objects, 10 of which were tested), possibly increasing competition for attentional resources, and the discrimination target appeared at the cued movement goal in 50% of all trials, that is nine times more often than in any other test location. This correlation between movement cue and test location may have implicitly encouraged observers to shift voluntary covert attention to the movement goal, and the increase in visual performance could have reflected the dynamics of voluntary covert attention [25,40–42]. By contrast, in our study, there was no advantage in attending to the reach target more than to the opposite location for the purposes of judging the test, as the test location was not predicted by the cue. The effects we measured were therefore automatic consequences of reach preparation, and given that the relative benefits at the reach location emerged within 100 ms after cue onset, they outpaced the time course of voluntary attention shifts [25,40–42].

Perceived contrast exhibited a very clear temporal pattern. During reach preparation, the apparent contrast of stimuli at the movement goal increased relative to the opposite location. This difference was largest for long CTIs and just before movement onset, when the perceived contrast at the reach target was greater than that of the neutral condition (in which no movement was planned or initiated), and perceived contrast at the opposite location was lower than the neutral condition.

An intriguing explanation of the different dynamics of performance and appearance before manual reaches is that both measures may reflect complementary aspects of visual processing. Specifically, we propose that visual performance benefits accompany movement preparation (or intention) and may dissipate once the movement has been prepared (in less than 300 ms; [33]). By contrast, visual appearance is a correlate of priority, a combination of stimulus salience and behavioural relevance, which need not be withdrawn before the movement is executed. These two aspects of visual perception—performance and appearance—are likely implemented by overlapping but not identical neural mechanisms, one involving feedback from areas involved in reach planning (or intention; see also [31]) and the other originating in an area encoding priority in general, irrespective of a particular effector. By contrast, for saccades, the temporal dynamics of performance and appearance are highly correlated, suggesting a common underlying mechanism [23]. But akin to saccade preparation, this study reveals that reach preparation enhances both visual performance and appearance.

Goal-directed movements are crucial for our interaction with the world—giving priority to the processing of their targets will facilitate visually guided behaviour. This study, in combination with our previous research [23–25,28], reveals that attention, intention and priority leave notable traces in our subjective visual experience and objective visual performance. Investigating the mechanisms underlying these connections between vision and movement will be an exciting endeavour for the next decade as it promises to uncover more general principles of how action shapes perception and perception shapes action.

4. Material and methods

(a). Participants

Seven observers (age 20–42 years, four males; one author) participated in the experiment. Except for the author (B.L.), all observers were naive about the experimental hypotheses. All observers had normal or corrected-to-normal vision and signed a consent form before participation. The NYU Institutional Review Board approved the experimental protocol, and we performed the experiment in agreement with the Helsinki declaration.

(b). Set-up

Participants sat in a dimly lit, sound-attenuated room, with chin and head stabilized and right arm positioned comfortably on an elbow cup (figure 1). We presented stimuli on a gamma linearized 19-inch Dell (Round Rock, TX, USA) M992 screen (1280 × 1024 pixels, 85 Hz vertical refresh) at a distance of 38.5 cm. A ViewPoint eye tracker (Arrington Research, Scottsdale, AZ, USA) monitored observers' gaze position throughout a trial (60 Hz sampling rate). A resistive touch screen (KEYTEC, Garland, TX, USA) mounted on the computer screen registered the position of the right index finger. Small adhesive rubber pads attached to the tip of the finger strongly improved the reliability of the touch screen data. A personal computer running Matlab (MathWorks, Natick, MA, USA) using PsychToolbox extensions [43] controlled stimulus presentation and data collection.

(c). Procedure

Before each trial, two fixation stimuli, one for gaze (a red dot, 0.2° in diameter centred within a black annulus, 0.7° diameter, at screen centre) and the other for reach (black dot in a black annulus, positioned 2° below gaze fixation) appeared against a uniform grey background.

When observers achieved gaze and reach fixation (within a 3.75° and a 1.5° radius, respectively), the gaze fixation dot turned white and the trial started with the presentation of two placeholders (dashed circles 3° in diameter, 7.5° left and right of fixation) centred on potential stimulus locations. Two reach targets (identical to, and horizontally aligned with, the reach fixation mark) appeared simultaneously, centred on potential reach locations (figure 2a). This layout ensured that the hand would reach towards the stimulus locations without ever occluding them.

After an interval of 247–1000 ms, randomly drawn from a uniform distribution, identical standard stimuli (see description below) appeared simultaneously for 71 ms at the two stimulus locations. A cue appeared 506 ms after the onset of the standard stimuli. On reach trials, the cue (a 0.5° line pointing to the left or right of gaze fixation) signalled both the initiation and direction of the movement. On neutral trials, the cue pointed to the potential test locations (two 0.5° lines). At a variable time after the cue onset (12, 59, 106, 153, 200 ms), we flashed a test stimulus for 71 ms with equal probability at either of the two stimulus locations. That is, the cue was completely non-predictive of the location of the test stimulus—we informed observers explicitly of that fact. Following a delay and a total of 1 s after cue onset, an array of four squircle-shaped (defined by x⁴ + y⁴ = r⁴) buttons appeared at the centre of the screen. In a two-by-two alternative forced-choice task, observers reported the relative orientation of the test stimulus (clockwise or counterclockwise of the standard) and contrast (higher or lower than the standard) with a single touch of one of the four buttons (figure 2b). Observers received auditory feedback on performance in the orientation discrimination task, but no feedback regarding the contrast judgement.

On reach trials, observers were required to initiate (within 400 ms of the cue) and complete (within less than 300 ms of initiation) a movement as quickly and accurately (landing 2.5° or less from the reach target location) as possible, while maintaining gaze fixation (within 3.75°). On neutral trials, we required observers to also maintain reach fixation (within 1.5°). If the gaze/reach fixation or movement latency/accuracy criteria were violated, the trial was aborted immediately, verbal feedback specified the mistake at the screen centre, and an identical trial was repeated in random order at the end of the block of trials (70.5±3.5% were successfully completed in the first attempt).

Standard Gabors had 22.4% contrast, were vertically oriented, with a Gaussian envelope of 0.5° s.d. The test Gabor was identical to the standard Gabors on any given trial, except for its orientation (rotated clockwise or counterclockwise of the standard) and contrast (range of 0.9 log units in seven equidistant steps around standard contrast). To avoid adaptation to luminance contrast and negative after-effects, we randomized the spatial frequency (1 or 1.5 cpd) and phase (range of 2π) of the standard and test Gabors.

Before the first session and several times during the study, observers completed a 3-up/1-down staircase procedure (starting value: 15° or 5°; adaptive step-size: 2°, 1° or 0.5°) to obtain the 79% orientation discrimination threshold for test stimuli presented before the onset of a reach. With the exception of the changes in test stimulus orientation and the absence of neutral blocks, the staircase procedure was identical to the main experiment. For the initial session, the average of two repetitions of the threshold procedure was used to estimate an observer's threshold. Across observers, the average orientation difference between test and standard stimulus was 3.2°±0.3° (mean±s.e.m.).

After a practice training session, each observer completed at least 1890 trials across multiple 1-h sessions. Because of variable reach-latency distributions, we tested each observer so that we obtained 100 trials per test location in each pre-reach time window (mean±s.e.m.: 182±10 trials); owing to consistently fast reaches, observer B.L., our most experienced observer in reaching tasks, had only 51 trials per location for the earliest time window (less than −150 ms) before the reach. We include her data in the analysis, but note that the pattern of results was the same without her data.

Reach and neutral trials were blocked (70 trials per block), and observers completed runs of three blocks (each run consisting of two reach blocks and one neutral block, randomly ordered).

(d). Data analysis

We excluded 7.9±2.0% of all trials from the analyses if either reach latency was shorter than 100 ms (2.1±1.9%) or saccades were detected offline (5.9±1.2%). We supplemented the online detection of fixation breaks with this offline saccade detection to ensure that perceptual effects were not confounded by the execution of eye movements. Eye position data, sampled at 60 Hz, were low-pass filtered, and saccades were detected offline using velocity criteria. Specifically, to be considered a saccade, the velocity of the initial data point was required to exceed, and remain above, 50° s⁻¹ and reach a peak velocity greater than 75° s⁻¹. Velocities greater than 800° s⁻¹ were considered to be the result of a blink and therefore were not considered saccades. Trials containing any saccades in the reach interval (i.e. within 400 ms of the onset of the movement cue) were excluded.

We determined observers' sensitivity in the orientation discrimination task, d′ = z(hit rate) − z(false alarm rate). We arbitrarily defined a clockwise response to a clockwise stimulus as a hit and a clockwise response to a counterclockwise stimulus as a false alarm. To determine PSEs, we fit cumulative Gaussians functions with four parameters (mean μ, standard deviation δ, lower and upper asymptote, γ and λ) to each observer's contrast reports, using maximum-likelihood estimation with no prior assumptions about parameter values [44]. For the analyses of performance and appearance, we only included trials in which the stimulus presentation was completed by the time of reach onset (90.7±2.7% of all reach trials).

We bootstrapped each observer's perceptual report data 1000 times by resampling from the binomial distribution with the given number of trials and probability (hit/false alarm rates or proportion of higher contrast reports) as parameters [45]. We then computed the variable of interest (d′ or PSE) for each of these bootstrap runs and derived s.e.m. from the distribution of means across observers and 95% CIs from the differences between these means. We derived p-values from these CIs by assuming normally distributed differences.

(e). Deriving the neutral baseline

The neutral condition was designed to match all bottom-up aspects of the stimulation protocol in the reach conditions (except, of course, the difference in cue shape). Therefore, it offers a valid baseline to compare the temporal evolution of appearance and performance in the reach conditions as a function of different CTIs (cue-locked analyses; figures 4a and 5a); that is, for each CTI, there was a corresponding neutral baseline. The situation is different for the analyses of perceptual changes with respect to the reach onset (reach-locked analyses; figures 4b and 5b), because there is no reach in the neutral condition. One way to deal with this is to collapse the data from all neutral trials and use that as a baseline for all time windows relative to reach onset (defined in figure 2d) and both test locations. This would be a fair strategy if performance and appearance were not affected by the timing and contrast of the test stimulus. However, performance and perceived contrast increased with CTI in the neutral condition (figures 4a and 5a), possibly because visual information (about the neutral cue) is accumulated across time [46,47]. In addition, the time between test offset and reach onset strongly covaried with CTI (figure 3c) and (to a much lesser extent) depended on test contrast and location. As a consequence of these issues, collapsing the data from all neutral trials to obtain a single neutral baseline could have biased the results.

In the reach-locked analyses of perceptual reports, therefore, we derived neutral baselines for each time window before a reach ensuring identical test stimulus parameters as in the corresponding reach condition. Following Rolfs & Carrasco [23], we computed baselines separately for each observer and each reach-locked time window in three steps. First, for each combination of CTI, i and test contrast, c, we determined the number of trials available in both the reach and the neutral condition, resulting in matrices of numbers of trials R_i,c and N_i,c, respectively, as well as a matrix P_i,c, containing the number of a certain perceptual report in the neutral condition (correct or ‘higher’, for performance and perceived contrast, respectively). Next, we divided R_i,c by its maximum value to obtain a matrix of weights, W_i,c. Finally, we multiplied W_i,c elementwise with both N_i,c and P_i,c and rounded all elements to the nearest integer, resulting in matrices , respectively. These matrices represent the neutral condition such that, for each reach-locked temporal bin, the prevalence of test stimuli with different combinations of CTIs and test contrasts is identical to that in the corresponding reach condition. were then used to compute the neutral baseline values of d′ and PSE. Note that these control data do not themselves evolve across time relative to a (simulated) reach; instead, they are baselines derived to match each temporal bin of the reach conditions for physical stimulus parameters. The relevant comparison is the difference between two conditions (e.g. reach target versus neutral for target) and whether and how that difference evolves across time.

Funding statement

This research was supported by a Marie Curie IOF (235625) of EU's 7th FP and a DFG Emmy Noether grant (RO 3579/2–1) to M.R., and an NIH grant no. (R01 EY016200) to M.C.