Electrophysiological measurement of information flow during visual search

Abstract

The temporal relationship between different stages of cognitive processing is long-debated. This debate is ongoing, primarily because it is often difficult to measure the time course of multiple cognitive processes simultaneously. We employed a manipulation that allowed us to isolate ERP components related to perceptual processing, working memory, and response preparation, and then examined the temporal relationship between these components while observers performed a visual search task. We found that when response speed and accuracy were equally stressed, our index of perceptual processing ended before both the transfer of information into working memory and response preparation began. However, when we stressed speed over accuracy response preparation began before the completion of perceptual processing or transfer of information into working memory on trials with the fastest reaction times. These findings show that individuals can control the flow of information transmission between stages, either waiting for perceptual processing to be completed before preparing a response or configuring these stages to overlap in time.

Introduction

One of the oldest debates in psychology centers on the temporal relationship between cognitive operations. For example, it has been hypothesized that responding appropriately to an object that we encounter requires information to be processed in a sequence of discrete stages in which one stage must finish before the next can begin (Donders, 1868/1969; Purcell et al., 2010; Sternberg, 1969). Competing models have shown that behavioral reaction time (RT) is equally well accounted for by a continuous information processing architecture in which the computations performed at different processing stages overlap in time (McClelland, 1979; Usher & McClelland, 2001). Results of studies attempting to settle this debate with neuroscientific evidence have been mixed, primarily because these studies have not measured neural activity indexing different processing stages simultaneously (Coles, Smid, Scheffers, & Otten, 1995; Meyer, Osman, Irwin, & Yantis, 1988; Mouret & Hasbroucq, 2000; Renault, Ragot, Lesevre, & Remond, 1982; Schall, 2003; Woodman, Kang, Thompson, & Schall, 2008).

The goal of the current study was to determine whether evidence for continuous information flow can be observed during visual search, and if so, to specify precisely which cognitive operations can be configured to overlap with one another in time using event-related potentials (ERPs). The ERP technique is uniquely suited to address these questions because discrete ERP components have been shown to measure discrete aspects of cognition and are temporally precise, indexing the earliest and latest time points at which the underlying cognitive processes are operative (Luck, 2005). However, it can be difficult to determine when one ERP component ends and another begins because separate components often bleed into one another. This component overlap problem has been difficult to address with statistical and mathematical analysis techniques alone (Luck, 2005; Rugg & Coles, 1995), and as a result the majority of previous electrophysiological studies examining the temporal relationship between processing stages have inferred the relationship between processing stages while measuring a single ERP component (Miller & Hackley, 1992; Osman et al., 1992).

To overcome this problem and enable the ability to directly compare components related to discrete processing stages, we examined the time course of two components that can be distinguished by their lateralized distributions, the perceptual attention-related N2pc and the response-related LRP, alongside a non-lateralized measure of the transfer of information into working memory, the P3b. The goal was to directly examine the temporal relationship between these components in order to provide a window into the temporal unfolding of cognitive processes from perceptual processing through response preparation in a typical visual search task

The N2pc is lateralized on the scalp relative to the locus of spatial attention, and previous research in visual search tasks has demonstrated that the onset of the N2pc can be used to track when perceptual-level attention is deployed to an object. Critically, during search the N2pc is directly followed by the onset of a lateralized positivity (the Pd) which signals the termination of perceptual attention, suggesting that the offset of the N2pc provides a sensitive measure of the completion of perceptual processing (Sawaki, Geng, & Luck, 2012; Sawaki & Luck, 2013; Woodman & Luck, 2003b). Traditional models of cognitive information flow also posit that following the completion of perceptual processing information is transferred into working memory, where it can be used to drive post-perceptual processes (cf. Bundesen, 1990; Duncan, 1996). Previous work has demonstrated that the centro-parietal sub-component of the P3, the P3b, measures this transfer of task-relevant information into working memory (Donchin & Coles, 1988; Vogel & Luck, 1998, 2002; Polich, 2007). Thus, in the context of the current work we would expect that in a discrete processing architecture, the onset of the P3b would coincide with the offset of the N2pc, indicating a completion of early perceptual processing and transfer of these visual representations into working memory.

In contrast to the perceptual selectivity of the N2pc, the stimulus-locked LRP is lateralized on the scalp contralateral to the hand that will be used to make an upcoming response, is maximal over motor cortex (Coles, 1989; Eimer, 1998), and is defined by a negative deviation from baseline as information regarding the hand required to make the response becomes available (Kutas & Donchin, 1980). Thus, the LRP is selectively related to response selection and preparation prior to the execution of a response (Smulders & Miller, 2012).

Because the majority of ERP components are not lateralized when bilateral arrays of visual stimuli are presented, we were able to isolate separate measures of perceptual (N2pc) and response-stage (LRP) processing against the background of other electrical activity in the brain. Then, by counterbalancing the visual field of the task-relevant stimuli and the hand used to respond, we were able to simultaneously measure when these components began and also when they ended. This approach allowed us to track the flow of information while subjects performed a standard visual search task. Specifically, we compared the temporal relationship between the end of perceptual processing, as measured with the offset of the N2pc, with the onset of preparation, as measured by the onset of the LRP.

In Experiment 1, we established the viability of our method by demonstrating that we could reliably isolate each component while subjects performed a simple visual search task, and we provided evidence that when speed and accuracy are equally stressed the flow of information from perceptual processing to working memory and response preparation in our task is serial in nature, but that buffering in working memory and response preparation temporally overlap with one another. In Experiment 2, we used an identical task and showed that speed stress had no influence on the latency of the N2pc, but influenced the onset of the LRP, leading to an overlap with the offset of the N2pc and suggesting continuous processing between perception and response preparation.

Experiment 1

In Experiment 1, subjects performed a visual search task responding with either their left or right hands using a videogame controller, as shown in Figure 1. The search task required subjects to press one button when a target shape was absent, and another when the target shape was present. Throughout the experiment the target shape was a Landolt square with a gap to the top and the nontargets were Landolt squares with gaps to the left, right, and bottoms of the squares. The possible target was always marked by the task relevant color for that block of trials and the black items were always task-irrelevant distractors. For example, one subject would begin by looking for red Landolt squares with a gap to their top for the first block of trials. This subject would press one button if a red Landolt square with a gap to the top was present in the array and a different button if the red square had a gap to the left, right, or bottom (i.e., target absent). This subject would then switch to looking for green Landolt squares with gaps to their top on the next block of trials. Half of the subjects completed the task in this order, and half in the reverse order.

Figure 1.

Experimental setup and grand average waveforms from Experiment 1. (A) The visual search stimuli showing a right visual field red target (B) Stimulus-locked grand average waveforms from electrode sites contralateral (red) or ipsilateral (black) to the possible target object, showing the N2pc at electrodes OL and OR (C) Stimulus-locked grand average waveforms for electrode sites contralateral (blue) or ipsilateral (black) to the response hand showing the LRP at electrode C3 and C4. (D) Stimulus-locked grand average waveforms for electrode site Pz on target present and target absent trials

At the same time, this target color counterbalancing was crossed with counterbalancing which hands were used to make the target present and absent responses leading to four unique target color/response conditions. Half of the subjects began by pressing a button with their right index finger to indicate that the target was present in the search array and using their left index finger to indicate that the target was absent. An equal number had this initial response mapping reversed. By counterbalancing the hand used to respond and the target color, we presented visual search arrays with the same sensory stimulation, but eliciting different lateralization of the N2pc and LRP. We then used these measurements, alongside the non-lateralized P3b, to directly examine the overlap between perceptual-attention, working memory, and response-related processes. This allowed us to distinguish between competing predictions of models that propose a serial versus temporally overlapping progression of processing stages.

If the flow of information from perception to response during visual search is discrete in nature we should see that the N2pc to the target ends before the P3 or LRP begin. In contrast, if the transmission of information during visual search is continuous, then we should find temporal overlap between the offset of the N2pc and the onset of either or both the P3 and LRP.

Method

Participants

Twenty volunteers (18–35 years old, normal color vision and acuity) participated for monetary compensation after informed consent was obtained.

Stimuli

The participants viewed the stimuli (Figure 1a) at a distance of 114 cm on a white background (90.0 cd/m²). A black fixation cross (<0.01 cd/m², 0.4° × 0.4°) was visible throughout each trial. The visual search arrays were composed of outlined Landolt squares (0.7° × 0.7°, line segments 0.1° thick) with a gap (0.5°) on the left, right, top, or bottom. The search array contained one red and one green Landolt-square stimulus and 10 black distractor stimuli arranged similar to the number locations on a clock face (centered 4.4° from fixation). In addition to this middle ring of items, an inner and an outer circle of 17 black Landolt squares each were presented at 1.4° and 7.4°, respectively, with the squares scaled according to the cortical magnification factor (Woodman & Luck, 2003b). These additional distractor rings never contained the targets, but serve to increase the amplitude of the N2pc (Luck, Girelli, McDermott, & Ford, 1997).

Procedure

Each trial began with the presentation of a fixation cross 1000–1400 ms before the onset of the visual search array (randomly jittered with a rectangular distribution). The search array was then presented for 1500 ms. Target presence (present or absent) and location were randomly selected. The final event on each trial was the offset of the fixation point and search array marking the beginning of the 1000-ms inter-trial interval during which participants were allowed to blink.

For half of the trial blocks, participants were instructed to respond to the search array by pressing a button with their right index finger for target present arrays and a button with the left index finger for target absent arrays on a hand-held game pad. On the other half of blocks this response mapping was reversed. Subjects were instructed to respond as accurately and quickly as possible.

Each participant performed 24 practice trials and then 4 blocks of 200 trials with short breaks every 66 trials (with the final block of trials containing 74 trials). Across blocks, observers switched between task-relevant red and green items to rule out physical stimulus explanations of the lateralized ERP effects. This counterbalancing was crossed with the counterbalancing of the response mapping.

Electrophysiology

The electroencephalogram (EEG) was recorded using tin electrodes situated within an elastic cap, and were amplified by an SA Instrumentation amplifier with a gain of 20,000 and a bandpass of 0.01–100 Hz. The amplified signals were digitized at 250 Hz by a PC and averaged offline. A subset of 10/20 sites were used (Fz, Cz, Pz, F3, F4, C3, C4, P3, P4, PO3, PO4, T3, T4, T5, T6, O1, and O2) as well as nonstandard sites OL (halfway between O1 and T5) and OR (halfway between O2 and T6). The EEG was referenced online to the right mastoid and then re-referenced offline to the average of the left and right mastoids. The electrooculogram (EOG) was recorded using electrodes placed 1 cm lateral to the external canthi to measure horizontal eye movements and by using an electrode placed beneath the left eye, referenced to the right mastoid, to measure vertical eye movements and blinks.

Data were averaged across correct trials to derive ERPs using our standard methods (Carlisle, Arita, Pardo, & Woodman, 2011). We used a two-step procedure for ocular artifact rejection (Woodman & Luck, 2003b). Trials with amplifier blocking, blinks, or eye movement artifacts were rejected prior to averaging. For blocking artifacts, all channels with voltage deviations >100 μV within a moving window of 200ms on any channel were rejected. Blinks and eye movements were detected using a step function with a moving window voltage criterion of 30μV for eye movements and 80μV for blinks. This procedure led to the rejection of 6.6% of trials per participant (with a single participant maximum of 24.6%). Averaged horizontal EOG (HEOG) waveforms were then used to reject any subjects with significant systematic unrejected eye movements (described in further detail in Woodman & Luck, 2003b). Four participants were replaced due to excessive eye movements (either >25% of individual trials rejected or residual systematic eye movements that resulted in HEOG voltage deflections >3.2 μV, corresponding to an ocular deviation of ±0.1°) and 2 participants with no measurable LRP or N2pc, which invalidated the fractional area latency measures.

The N2pc was measured as the difference between electrodes contralateral versus ipsilateral to the possible target, during the period 150–350 ms after the onset of the search array at electrodes OL and OR. The P3 component was measured as the difference in the waveforms elicited by target present versus target absent trials at electrode Pz, where the P3b is maximal, with a window spanning 150–600 ms post-stimulus. The LRP was measured as the difference between electrode sites contralateral versus ipsilateral to the response hand at electrodes C3/4, during the period 250–700 ms after the onset of the visual search display. The measurement windows for each component were set liberally to include the entire interval that the components could have been observed across all subjects (Luck, 2005).

The factional-area latency method was used to measure the onset and offset of the components (Luck, 2005; Woodman, 2010a). This method avoids the problems with high-frequency noise that are inherent when trying to measure the instantaneous voltage at any single time point, or the arbitrary setting of criteria necessary to avoid false positives with sliding-window analyses. The offset of the N2pc was measured as the time point that 95% of the area under the curve of the contralateral-minus-ipsilateral waveforms occurred for each subject. The onset of the P3 was measured as the time point that 5% of the area of the target-present-minus-target-absent waveforms occurred for each subject’s averaged waveforms, and similarly the LRP onset was measured conservatively as the time point that 5% of the area under the curve of the averaged contralateral-minus-ipsilateral waveforms occurred for each subject. Analyses of variance (ANOVAs) were used for all statistical tests and the Greenhouse-Geisser epsilon correction for non-sphericity was used where appropriate (Jennings & Wood, 1976).

We collapsed across color because we found no significant effects of target color in the ERP or behavioral analyses (F < 1.0). Consistent with previous work, responses were significantly faster in target present than target absent trials (see Table 1), reflecting the fact that the search process is terminated differently in each type of trial (see Chun & Wolfe, 1996, for a discussion). As a result, we considered target present and target absent trials separately in our fractional area analyses. Also evident in Table 1, is a slight stimulus-response compatibility effect in Experiment 1 (i.e., a ‘Simon Effect’, Simon, 1969). We note that this effect could produce artifactual overlap of the N2pc and LRP due to more trials coming from the situation in which the possible target and the response hand were on the same side of space. To rule out contamination from these same-side trials being approximately 4% greater than that of different-side trials, we threw out 4% of the same-side trials (randomly selected). These analyses yielded the same results as those without identical numbers of observations in each cell, due to the small number of trials involved. For follow-up median split analyses in both experiments, we sorted the trials into fast and slow halves within each trial type (i.e., stimulus and response on same side versus different sides) so that these analyses were immune to explanation as a simple stimulus-response compatibility effect.

Table 1.

Mean accuracy (percent correct ± SEM) and reaction time (RT; mean ± SEM) across Experiments 1 and 2 for target present versus absent and for the task-relevant item and response on the same side of space versus different sides.

	Target Present		Target Absent
	Accuracy	RT	Accuracy	RT
Experiment 1	95.5 ± 1.1%	563.2 ± 20.8 ms	96.1 ± 1.2%	604.1 ± 20.4 ms
Experiment 2	78.8 ± 2.2%	470.1 ± 12.4 ms	85.4 ± 1.5%	465.3 ± 10.3ms

	Different Sides		Same Sides
	Accuracy	RT	Accuracy	RT
Experiment 1	94.8 ± 1.4%	587.6 ± 19.9 ms	96.5 ± 0.8%	580.4 ± 20.9 ms
Experiment 2	77.9 ± 2.7%	444.6 ± 10.8 ms	85.6 ± 1.9%	449.2 ± 11.3 ms

Results and Discussion

In Experiment 1, participants’ behavioral responses to the visual arrays were both fast and accurate (see Table 1). Figure 1 shows search-locked waveforms collapsed across posterior electrode sites (panel B) and C3/C4 (panel C). As can be seen in Figure 2, we observed that the N2pc to the possible target object in the visual scene (i.e., the red item during blocks of searching for a red Landolt square with a gap up) ended before the LRP began, resulting in a significantly earlier N2pc offset than the LRP onset in both target absent (M = 311.2 ms and 337.1 ms, respectively, F(1,19)=2.25, p<0.05) and target present trials (M = 301.0 and 324.2 ms, respectively, F(1,19)=2.28, p<0.05). The finding that the N2pc offset earlier and the LRP onset earlier in target present relative to target absent trials parallels the finding of faster RTs on target present relative to target absent trials (F(1,19)=5.77, p<0.05), providing converging evidence that the changes in the timing of these components are related to changes in search performance.

Figure 2.

(A) Grand average N2pc, P3, and LRP difference waves. Mean RT shown with black vertical line, along with the measurement windows marking the significant onset and offset for the N2pc and LRP shown in the gray area under the curves. N2pc and LRP difference waves for target absent (B) and target present (C) trials, with the measurement windows marking the significant onset and offset for the N2pc and LRP shown in the gray area under the curves.

The significant gap between the offset of the N2pc and the onset of the LRP suggests that the output of perception was held in memory while the response was programmed, as predicted by models of visual attention (Bundesen, 1990; Duncan, 1996). To explicitly test this explanation and validate the N2pc offset measure of the completion of perceptual processing we analyzed the onset of the P3 component. Figure 2A shows that the onset of the P3 coincides with the offset of the N2pc (M = 286.5 and M = 308.2, respectively; F(1,19)=1.68, p=0.11) indicating that as the task-relevant object was perceived, its representation was uploaded into working memory and buffered for approximately 30–40ms before the appropriate behavioral response began being prepared.

We performed two additional analyses to rule out alternative explanations and determine the robustness of our observations. First, it is possible that eye movements undetectable in our artifact rejection procedures might have contaminated our lateralized measures of visual processing and response selection. Participants could have first made small eye movements in the direction of the target object, and then in the direction of the hand they prepared to use. If this were the case, it would not be surprising to observe two discrete phases of lateralized activity. However, the waveforms in Figure 3 shows that this pattern of eye-movement activity was not found. Second, we wondered if overlap between perception and response selection might be observed on the trials with fast responses but not slow responses. We performed a median split of trials based on RT. Despite significantly faster mean RT on the fast relative to slow trials in both target absent (514.8 ms versus 750.4 ms, F(1,19)=53.34, p<0.001) and target present (484.2 ms versus 663.6 ms, F(1,19)=21.09 p<0.001) conditions, we found no significant overlap between the N2pc and LRP in fast trials for either condition (target absent, F<1, ns, target present F(1,19)=1.90 p>0.2).

Figure 3.

The horizontal electrooculogram (HEOG) from trials in which the target hemifield and response hand were on the same versus different sides of space showing that the sequence of lateralized effects cannot be explained by a sequence of eye movements to the task-relevant item and then the hand used to respond.

These results validate our approach, and indicate that perceptual processing as indexed by the offset of the N2pc was completed before response selection indexed by the LRP began, regardless of reaction time. In addition, the offset of the N2pc coincided with the onset of the P3 (i.e., the P3b given its scalp distribution and larger amplitude for target presence vs. absence). This is consistent with our proposal that the N2pc offset provides a useful measure of the end of perceptual processing as the P3 onset has been interpreted as a chronometric marker of the end of perceptual analysis and updating of working memory (Coles et al., 1995). This concurrent offset of the N2pc and onset of the P3 in Experiment 1 provides empirical validation of the N2pc offset as a measure of the end of perceptual attentional processing in our task, consistent with previous proposals that the N2pc offsets concurrently with the withdrawal of perceptual-level attention from a stimulus (Sawaki et al., 2012).

At the same time, others have argued that perceptual processing may continue beyond the N2pc in some cases, for example in certain search tasks that generate a robust Sustained Posterior Contralateral Negativity (SPCN; Mazza et al., 2007) or Contralateral Delay Activity (CDA; Luria & Vogel, 2011; Tollner, Conci, Rusch, & Muller, 2013). For example, Tollner et al., (2013) have shown that an arguably perceptual manipulation (target contrast) influences the magnitude of the CDA, while leaving the N2pc unaltered. On this basis, they have argued that the N2pc provides a selective measure of feedforward perceptual processing, whereas the CDA represents a selective marker of feedback/recurrent perceptual processing on the contents of visual working memory (VWM). In the context of the current work, this raises the possibility that the LRP begins after feedforward perceptual processes, but overlaps with feedback processes that operate over representations in VWM. Thus, Experiment 1 may only provide evidence that the initial perceptual sweep must complete before a response can begin to be programmed, with partial information being passed to response-selection processes even when perceptual processing is ongoing.

However, given that we did not observe a CDA in our task, such an argument seems only to apply to more demanding search tasks in which perceptual processing is extended in time, such as that used by Tollner et al. (2013). One may ask why we did not observe a strong CDA (or SPCN) in our task, especially given its similarity to previous studies that used compound search/discrimination tasks and elicited these components (e.g., Mazza et al., 2007, Tollner et al. 2013). Aside from obvious differences in the stimuli themselves, we would argue that the search task employed in Experiment 1 was not a true compound search/discrimination task, and thus may not have taxed later perceptual/VWM processes associated with the CDA/SPCN (e.g., as in Tollner et al. 2013; Luria & Vogel, 2011).

Specifically, in our task subjects performed a target present/absent judgment on a target that remained consistent in color and orientation across a block of 200 trials, which may have minimized the involvement of working memory processes reflected in the CDA/SPCN. For example, we have recently shown that under circumstances in which the search target remains constant, WM begins to give up control to LTM within approximately 3–5 trials (Carlisle et al., 2011; Reinhart & Woodman, 2012). In addition, it is possible given this consistency of the target template across trials subjects were able to make the present/absent judgment using, e.g., a less taxing template matching process rather than a trial-by-trial discrimination process per se. As a result, it is likely that the lack of a CDA/SPCN in the current work reflects differences in the demands required for completing our search task relative to previous work (Mazza et al., 2007; Luria & Vogel, 2011; Tollner et al., 2013).

Thus, the results of Experiment 1 suggest that in our task, when speed and accuracy are equally stressed, perceptual processing completes before a response is prepared or information is transferred into working memory. This finding is consistent with serial processing accounts of response preparation (Meyer, Yantis, Osman, & Smith, 1984; Miller, 1988; Purcell et al., 2010; Sternberg, 1969, 2001). In Experiment 2, we see whether such a serial processing mechanism operates under conditions in which speed is stressed over accuracy, which may be more likely to drive parallel processing of stimulus and response information.

Experiment 2

Although the results of Experiment 1 support a serial account of perceptual processing and response preparation, it is not clear whether these processes occur in an obligatorily serial manner. Specifically, in Experiment 1 subjects were instructed to complete the task both quickly and accurately (the modal instruction in cognitive psychology and neuroscience experiments), which may be more likely to drive serial processing. However, it is possible that perceptual processing and response preparation may be configured to overlap when subjects are pressured to trade accuracy for speed of responses, whereby the cost of configuring information to flow in a more continuous or cascaded manner between stages is paid in accuracy. To test this hypothesis, we invited the same subjects from Experiment 1 to perform the same visual search task, but this time we instructed them to sacrifice accuracy for speed.

Method

The method we used in Experiment 2 was identical to that of Experiment 1 with the following exceptions. In Experiment 2, subjects were instructed to respond as quickly as possible regardless of accuracy. This was reinforced by presenting feedback indicating the RT on the last block of trials relative to the mean RT across the entire experiment, including instructions to respond even faster on the next block of trials.

Results and Discussion

The effectiveness of our instruction to trade accuracy for speed in Experiment 2 was evidenced by a significant decrease in RT (M difference = 136 ms, F(1,19)=29.2, p<0.001), and increased error rates (M difference = 13.7%, F(1,19)=37.30, p<0.001), but the N2pc offset concurrently with the onset of the LRP in target present trials (M = 296.7 ms and 313.1 ms, respectively, F(1,19)=1.46 p<0.16) as well as target absent trials (M = 315.0 ms and 313.5 ms, respectively, F<1, ns).

However, when we split the trials with correct responses based on each subject’s median RT (target absent trial Ms = 408.8 ms versus 523.2 ms, F(1,19)=60.94 p<0.001; target present trial Ms = 414.6 ms versus 526.6 ms, F(1,19)=26.27 p<0.001 ), we found a significant overlap of the N2pc offset (286.4 ms) and LRP onset (246.5 ms) during the trials with the fastest RTs (overlap = 39.9ms; F(1,19)= 6.24 p<0.05, Figure 4), but in trials with the slowest RTs there was a significant gap (75.6 ms) between the offset of the N2pc (292.1 ms) and the onset of the LRP (367.7 ms; F(1,19)=29.36 p<0.001).

Figure 4.

Grand average difference waves from fast and slow trials showing the N2pc, LRP, and P3 in Experiment 2, in which subjects performed the visual search task under speed stress. The visual search stimuli and analysis methods were identical to Experiment 1.

In fast trials (Figure 5), overlap was observed for both target absent trials (41.8 ms overlap, M = 288.3 ms N2pc offset and 246.5 ms LRP onset, F(1,19)= 4.54, p < 0.05) and target present trials (59.7 ms overlap, M = 285.0 ms N2pc offset and 220.3 ms LRP onset, F(1,19)= 7.61, p < 0.01). In contrast, in slow trials (Figure 6) the N2pc offset significantly earlier than the onset of the LRP in both target absent (M = 287.3 ms N2pc offset and 385.2 ms LRP onset, F(1,19)= 29.38, p < 0.001) and target present trials (M = 298.2 ms N2pc offset and 348.8 ms LRP onset, F(1,19)= 18.66 p < 0.01). Thus, in fast trials overlap occurred almost exclusively due to latency shifts in the LRP, a finding consistent with previous work showing that speed stress has greater influence on post-perceptual response processes indexed by the LRP (Rinkenauer et al., 2004).

Figure 5.

N2pc and LRP difference waves from trials with fast responses in the median split analysis performed in Experiment 2, showing target absent and target present trials. The offset of the N2pc significantly overlapped with the onset of the LRP in both conditions (indicated with dark gray shading).

Figure 6.

N2pc and LRP difference waves from trials with slow responses in the median split analysis performed in Experiment 2, showing target absent and target present trials. On slow trials, we observed a pattern of results similar to Experiment 1, with no significant overlap between the offset of the N2pc and the onset of the LRP.

As in Experiment 1, we also examined the temporal relationship between the N2pc and the P3. In contrast to Experiment 1, we observed a significant gap between the offset of the N2pc and the onset of the P3 in both target absent (M = 87ms, F(1,19)= 29.20, p < 0.001) and target present (M = 90ms, F(1,19)= 36.02, p < 0.001). This gap was driven by the presence of a non-lateralized negativity appearing between 200 and 350ms that was absent in Experiment 1, leading to a later onset of the temporally overlapping P3 wave. This negativity, typically referred to as the ‘selection negativity’ or SN (Hillyard & Anllo-Vento, 1998), appears in response to stimuli containing target-defining features. Interestingly, the amplitude of the SN was significantly larger in fast trials (M = 0.74 μV) than in slow trials (M = 0.47 μV; F(1,19)= 5.88 p < 0.05).

This appearance of the SN under the speed stress in Experiment 2, coupled with the larger SN amplitude in trials with the fastest RTs, suggests that changes in feature-based attention processes indexed by the SN may be at least partially responsible for the facilitation of search RTs when subjects sacrifice accuracy for speed. Whether these changes in feature-based attention occur in addition to or are responsible for the earlier onset of response preparation processes is unclear on the basis of the current data. However, the fact that a significant SN is present even on slow trials suggests that changes in feature-based processing alone are not sufficient to drive speed RTs, and raises the possibility that changes in feature-based processing and response preparation combine to speed search.

Finally, we feel that it is necessary to note that the same group of subjects participated in both Experiment 1 and Experiment 2. This step was taken to control for individual differences in the presence or absence of the ERP components of interest and to ensure comparability between waveforms across speed and accuracy conditions. This raises the possibility that the observed changes in ERP components were due to learning across the two sessions, but we note that RT speeding during visual search typically occurs during the first 10–20 trials of the task (Carlisle, Arita, Pardo, & Woodman, 2011; Logan, 1988; Schneider & Shiffrin, 1977; Woodman, Carlisle, & Reinhart, 2013). Indeed, during two recent studies we found that RTs began leveling off within approximately 6 trials (Carlisle et al., 2011; Reinhart & Woodman, 2014). Further, it is not clear why, if driven by a learning effect, the overlap between the perceptual processing and response preparation was only observed when examining median split data and not in the overall data. Thus, it seems most likely that speed stress drove the results obtained in Experiment 2, supporting the hypothesis that individuals can control the flow of information between processing stages. This provides neuroscientific evidence for theoretical proposals that the configuration of the temporal relationship between cognitive stages is flexible (Logan, 1980). We believe that it remains to be seen whether learning effects in this and other tasks might involve reconfiguring stages to overlap as task proficiency increases.

General Discussion

Our results have important theoretical implications, allowing us to rule out theories proposing cognitive architectures in which information flows in a purely continuous or cascaded manner between the stages of perceptual processing and response preparation (Eriksen & Schultz, 1979; McClelland, 1979). Our findings also call into doubt models in which RT effects are explained by the accumulation of evidence during a single, slowly evolving decision stage that accounts for all RT variability (Ratcliff, 2006). When participants maintained high levels of accuracy, our results conformed to the predictions of models in which information flows in a discrete manner between stages (Meyer, Yantis, Osman, & Smith, 1984; Miller, 1988; Purcell et al., 2010; Sternberg, 2001).

However, when speed was stressed our results provided evidence of continuous processing, demonstrating the ability to change the nature of information transmission given different task demands. Furthermore, the appearance of the SN only under speed stress suggests that different cognitive processes may come online when sacrificing search accuracy for speed. Together, suggests that a source of speed-accuracy tradeoffs in task performance may be the reconfiguration of cognitive processing to operate in a continuous mode of information flow between stages. The observation that people can configure processing stages to overlap supports proposals that preparing a behavioral response can begin before perception is complete (Logan, 1980; McElree & Carrasco, 1999; Tombu & Jolicoeur, 2003; Woodman & Luck, 2003b).

Finally, it is notable that across both experiments the onset of the P3 always followed the offset of the N2pc, suggesting that perceptual processing must be completed before information regarding the target identity can be transferred into working memory. Likewise, in both experiments the P3 and LRP temporally overlapped with one another, and in the fastest trials of Experiment 2 the P3 even began after response preparation. This suggests that following perceptual processing, buffering of information in working memory and response preparation can be carried out in parallel and possibly independently, and future studies using the approach outlined here could be used to shed light on the nature of these effects.

Together, our results are consistent with conclusions drawn in previous behavioral studies that instructions can change the temporal relationship between stages of processing. Specifically, previous studies of the psychological refractory period (i.e., PRP) paradigm have found patterns of interference consistent with temporal overlap between perceptual processing and response selection. Lehle and Hübner (2009) reported that when subjects were instructed to perform two tasks at the same time there was a significant increase in the amount of crosstalk between the two tasks that were performed. These findings were interpreted using a modeling framework in which perceptual and response selection changes can be configured to overlap in time (Tombu & Jolicoeur, 2003) in the PRP paradigm. Similar evidence for the concurrent processing of two pieces of information has been found examining semantic priming of information retrieved from memory (Logan & Schulkind, 2000).

Of note, our findings suggest that the overlap between perceptual processing and response preparation occurs primarily as the result of shifts in the timing of response-related processes indexed by the LRP and not the allocation of perceptual-level attention indexed by the N2pc. This is consistent with a recent study by Tollner et al. (2012), who showed that the task set employed during a visual search task influenced the timing of the LRP without influencing the N2pc/PCN. Although they did not formally analyze the temporal relationship between the LRP and N2pc/PCN across task sets, the data reported in this work clearly show that task sets associated with faster reaction times led to an earlier LRP onset and greater overlap between components (Tollner et al., 2012). Thus, it is possible that the faster RTs observed in particular types of search task (e.g., localization vs. detection; Tollner 2012) result from the ability to concurrently process stimulus information and program a response.

Given results such as these showing that factors related to top-down task set primarily influence the latency of post-perceptual processes, in future work it would be interesting to use our approach coupled with bottom-up, perceptual manipulations of, e.g., target-distractor similarity or stimulus homogeneity to see if these such manipulations are more likely to influence early perceptual processes and whether such changes could also lead to overlap between processes. Taken together, the present electrophysiological findings build on previous work by showing that task-dependent modulations of response preparation can cause stages of cognitive processing to be flexibly configured to overlap within a single task, in addition to driving fundamental changes in the processes employed to complete the task. This suggests that the presence of specific cognitive operations and the temporal relationship between them are not a fixed structural limitations of the brain, but instead are adjustable parameters that can be used to conform to changing task demands.