Electrophysiological markers of visual dimension changes and response changes

Abstract

In cross-dimensional visual search tasks, target discrimination is faster when the previous trial contained a target defined in the same visual dimension as the current trial. The ‘dimension-weighting’ account (DWA; Found & Müller, 1996) explains this intertrial facilitation by assuming that visual dimensions are weighted at an early perceptual stage of processing. Recently, this view has been challenged by models claiming that intertrial facilitation effects are generated at later stages that follow attentional target selection (Mortier et al., 2005). To determine whether intertrial facilitation is generated at a perceptual stage, at the response selection stage, or both, we focused on specific ERP components (directly linkable to perceptual and response-related processing) during a compound search task. Visual dimension repetitions were mirrored by shorter latencies and enhanced amplitudes of the N2pc suggesting a facilitated allocation of attentional resources to the target. Response repetitions and changes systematically modulated the LRP amplitude suggesting a benefit from residual activations of the previous trial biasing the correct response. Overall, the present findings strengthen the DWA indicating a perceptual origin of dimension change costs in visual search.

Introduction

Over the last decade, there have been a growing number of reports of intertrial facilitation effects on the performance in visual search tasks. Such effects are found even in ‘pop-out’ search tasks, in which the target is a singleton element defined by a simple feature difference relative to the distractor elements in the search display: responses to a singleton target on a given trial n are faster when target-defining attributes are the same as on the preceding trial n-1 (or, more generally, n-i, where i>1 – though the strongest effect is typically found for i=1).¹ These attributes include, besides target position (e.g., Maljkovic & Nakayama, 1996), the target-defining feature (e.g., when, variably across trials, the target was either red amongst green distractors or green amongst red distractors; Maljkovic & Nakayama, 1994), and the target-defining dimension (e.g., when the target was variably different in color or different in orientation from the distractors; Müller, Heller, & Ziegler, 1995). Intertrial facilitation effects have been found both in standard visual search tasks, in which observers had to make a ‘target-present/absent’ decision (e.g. Müller et al., 1995), and in so-called ‘compound’ search tasks (Duncan, 1985), in which the target-defining feature differs from the feature that determines response selection (e.g., when the target is singled out by being the only red element in the display, while the response is determined by a shape aspect of the target; e.g., Maljkovic & Nakayama, 1994). Müller and colleagues have argued in favor of a primary role for the target-defining dimension in generating such effects: under comparable conditions (target-present/absent task, constant distractor definition), intertrial facilitation was larger for a dimension repetition versus a change, compared to a feature repetition versus change within the same dimension (e.g., Found & Müller, 1996; Müller, Krummenacher, & Heller, 2004; Müller, Reimann, & Krummenacher, 2003; see also Olivers & Meeter, 2006, for a systematic comparison). Note that Müller and colleagues also found feature repetition effects – though, generally, these were robust only for the color dimension. Despite this primacy of dimensions, dimension-specific intertrial facilitation has tended to be weak, if at all present, in compound tasks, at least under conditions in which the target was highly salient (e.g., Chan & Hayword, 2007; Cohen & Magen, 1999; Krummenacher, Müller, & Heller, 2002; Kumada, 2001; Mortier, Theeuwes, & Starreveld, 2005; Theeuwes, Reimann, & Mortier, 2006).² Based on this and other dissociations with visual search tasks requiring a simple target-present/absent decision, several authors have recently proposed that target detection relies on different mechanisms in compound, relative to simple, visual search tasks (Chan & Hayword, 2007; Mortier, van Zoest, Meeter, & Theeuwes, 2007). On this background, the present study was designed to investigate (i) why RT intertrial facilitation is overall reduced under compound-task conditions in efficient visual search and (ii) whether any effects observable arise at an early perceptual and/or a later response-related stage of processing.

Perceptual and response-based accounts of intertrial facilitation

Several accounts of the origin of intertrial facilitation effects have been proposed, which may be classified as either ‘perceptually based’ or ‘response-based’ (see Meeter & Olivers, 2006, and Olivers & Meeter, 2006, for a systematic discussion). Perceptual accounts (e.g., Müller et al., 1995; Müller et al., 2003; Wolfe, Butcher, Lee, & Hyle, 2003) assume that repetition of target-defining attributes on successive trials facilitates the early sensory coding of critical attributes – which, in efficient visual search, is assumed to occur pre-attentively and in parallel across the search display. In contrast, response-based accounts assume that intertrial facilitation originates at a processing stage after focal-attentional selection of the target, at which the target is attentionally analyzed and translated into an appropriate response (e.g., Cohen & Magen, 1999; Mortier et al., 2005; Theeuwes et al., 2006).

There have been other attempts to explain intertrial effects in terms of the retrieval of task-relevant episodic memories. On one such account, proposed by Huang, Holcombe, and Pashler (2004), the translation from stimulus to response involves a process in which memories of previous episodes with similar stimuli and associated responses are automatically retrieved. If retrieved and currently required responses match, the current response is expedited; if they do not match, the current response is delayed (e.g., see Logan, ​1990, ​2002; Neill, 1997; Waszak, Hommel, & Allport, 2003). Thus, this hypothesis is essentially a variant of the response-based account. An alternative episodic-retrieval account proposed by Hillstrom (2000) assumes that retrieval of earlier trial episodes re-establishes the attentional priorization settings that had led to the detection of the previous targets. However, as noted by Meeter and Olivers (2006), “this idea is very difficult to distinguish from a [perceptually-based] view, in which the priorization settings are more directly altered by the preceding trial” (p. 218). Overall, the relevant episodic-memory retrieval notions can be subsumed under either perceptually- or response-based accounts.

Although some theorists (e.g., Cohen & Magen, 1999; Mortier et al., 2005; Theeuwes et al., 2006) have tended to treat perceptually- or response-based accounts as mutually exclusive, they are at least logically compatible with each other. Intertrial facilitation may operate at both pre-attentive perceptual and post-selective response-related stages of processing, as has been explicitly acknowledged by Müller et al. (2003) as well as by Meeter and Olivers (2006). Nevertheless, it remains an open issue whether, in a particular task, intertrial facilitation arises at perceptual, at response-related, or at both stages of processing. The present study was designed to address this issue in relation to dimension-based intertrial facilitation in compound-search tasks under conditions of high target saliency (i.e., low target ambiguity).

Dimension-specific intertrial facilitation in compound search tasks

As noted above, the detection of search targets defined by a singleton feature in dimensions such as color and orientation (with the critical dimension varying randomly across trials) is faster when the target-defining dimension remains the same across consecutive trials, and this effect is largely unaffected by whether or not the target feature is also repeated. To explain this reaction time (RT) pattern, Müller and colleagues proposed a ‘dimension-weighting’ account (DWA; e.g., Found & Müller, 1996; Müller et al., 1995), which is essentially an extension of the Guided Search model proposed by Wolfe and colleagues (e.g., Wolfe, 1994). The DWA assumes that attentional weight can be allocated to various basic visual dimensions (such as orientation, color, motion), with the total weight being limited. Preferential weighting of one dimension leads to faster detection of singleton feature targets defined in this dimension, relative to targets defined in other dimensions. This facilitation results from enhanced coding of feature contrast (saliency) signals within the weighted dimension and/or amplified transmission of dimension-specific feature contrast signals onto an overall-saliency map of the visual display, which determines the allocation of focal selective attention. The delay in target detection observed when the target dimension changes across trials may have two causes. It is possible that sufficient attentional weight must be shifted from the old to the new target-defining dimension as a pre-condition for target detection (i.e., to sufficiently amplify the feature contrast signal at the overall-saliency map level). Alternatively, the target is processed and eventually selected based on the relatively low weight allocated to its defining dimension, and the weight shift follows target detection (e.g., see Gramann, Töllner, Krummenacher, Eimer, & Müller, 2007). In either case, there is a weight shift to the new target-defining dimension, which influences the processing of any subsequent target. While this weight shift is largely bottom-up controlled by the presence of a feature contrast signal in a given dimension, it can to some extent be top-down modulated when a target is expected to be defined in another dimension (see Müller et al., 2003). Importantly, the DWA interprets weighting effects to be pre-attentive (‘perceptual’) in nature, modulating signal strength prior to the selective-attention stage, which operates based on the overall-saliency map (Müller & Krummenacher, 2006; see also Folk & Remington, 1998).

Recently, this view has been challenged by models which assume that the ‘weighting’ effects described by Müller and his colleagues are post-selective in nature, arising at a stage following focal-attentional selection (which is itself top-down impenetrable), at which detected targets are translated into responses (e.g., Cohen & Magen, 1999; Mortier et al., 2005; Theeuwes et al., 2006). This challenge has been based in part on findings in compound search tasks in which the detection-relevant target attribute is independent of the response-relevant attribute. One example is illustrated in Figure 1: The target is defined by a unique shape, while the response is determined by the vertical or horizontal orientation of a grating within the target object. In such compound tasks, dimension-specific intertrial effects are greatly reduced, if at all present, relative to simple detection tasks in which observers are instructed to make a target-present/absent response (e.g., intertrial effects of 9 vs. 34 ms in the study of Theeuwes et al., 2006; see also Krummenacher et al., 2002, and Kumada, 2001), which is not easily explained in terms of the DWA. Instead, the fact that such effects are scarcely evident in compound tasks has been taken as evidence that dimension repetition/change “modulates the speed with which one can give a response after the target has been detected”; for example, on a dimension repetition trial, “after entering the second [attentional] stage of processing, less sensory evidence is required to decide whether an item is a target or a distractor” (paraphrase of Theeuwes, personal communication, 30 October, 2001, and Theeuwes, 1992, p. 605)³.

Figure 1.

Example for the visual search array with a vertically oriented target defined in the shape dimension. Search arrays consisted of 8 stimuli in a circular array against a black background, each presented equidistant from a white central fixation cross. Distractors were blue circles and targets were defined in the color dimension (red) or shape dimension (square). Each stimulus was either horizontally or vertically oriented. Participants were asked to discriminate the orientation of the singleton target as fast and accurately as possible.

However, Müller and Krummenacher (2006) have recently proposed that the central assumption underlying this argument – that the processes of target selection (assumed to be pre-attentive) and response selection (assumed to be post-selective) are independent in compound tasks – may not be tenable. They reanalyzed various sets of compound-task data (Krummenacher et al., 2002; Müller & Krummenacher, 2006; Pollmann, Weidner, Müller, Maertens, & von Cramon, 2006) to examine whether and how the effect of a change in the target-defining dimension was contingent on a change in the response, that is, in the target attribute that determined the response hand (change vs. no-change). For all data sets, an identical pattern of results was observed: An intertrial dimension change effect was present only when the response (attribute) was repeated, in which case RTs were significantly faster with a dimension repetition as compared to a change. In contrast, no such effect was evident when the response (attribute) changed. Essentially, with any change, whether in dimension and/or response, RTs were equally slow. A similar, albeit non-significant, interactive pattern can also be seen in Figure 7 of Olivers and Meeter (2006; see also Figure 5 of Chan & Hayword, 2007). Müller and Krummenacher took this pattern to suggest that, although, statistically, there was no correlation between the two types of change (target-defining dimension, response attribute), the system ‘assumes’ there is one (see also, e.g., Kingstone, 1992⁴). If the target dimension (the task attribute that becomes available fastest) remains unchanged, the system implicitly assumes that the attribute on which the response will be based will also be unchanged; that is, the unchanged response is facilitated, and there is a cost if the response attribute actually changes. In contrast, if the dimension changes, the system may cancel any prior assumptions as to the response attribute to be expected and start processing from scratch. Whatever the explanation, dimension-specific RT intertrial effects are overall reduced in compound tasks because they are evident only in the absence of a response change. Therefore, behavioral effects observed in such compound tasks may not permit the dissociation of perceptual and response-related processes associated with dimension changes/repetitions in a simple and straightforward manner.

Rationale of the present study

The present EEG study was designed to overcome this limitation by examining not only response times in a compound-search task, but also event-related brain potentials (ERPs) associated with dimension and response repetitions versus changes. In the present task, participants had to first search for a singleton target uniquely defined in either the color or the shape dimension, before they could select the appropriate response, which was determined by the orientation of a grating within the target object (horizontal vs. vertical). In this way, a target defined in a changed dimension could be associated with either the same (e.g., a horizontal color target preceded by a horizontal shape target) or a different response (e.g. a horizontal color target preceded by a vertical shape target) as the preceding target, as could be a target defined in a dimension that was repeated across successive trials. This resulted in four experimental conditions: same dimension – same response (sDsR), same dimension – different response (sDdR), different dimension – same response (dDsR), and different dimension – different response (dDdR). A similar paradigm was employed in an event-related fMRI study by Pollmann et al. (2006). The behavioral results revealed the interactive pattern of dimension and response change effects described above. At the neuronal level, dimension changes were associated with activations primarily in posterior visual areas, whereas response changes elicited activations primarily in motor-related areas of the parietal and frontal cortices.

To gain further insights into the time course of pre-attentive perceptual and post-selective response-related processes in cross-dimensional search, the present study focused on two specific components of the ERP, which can be directly linked to perceptual-related and response-related stages of information processing, respectively. The first component, the N2pc, is a negative-going deflection with a maximum over visual areas of the hemisphere contralateral to the location of an attended stimulus. The N2pc has been observed in numerous previous visual search experiments, typically between 175 and 300 ms after the onset of the search array. It is interpreted as reflecting the attentional selection of target among non-target stimuli, based on target-defining perceptual attributes (e.g., Eimer, 1996; Woodman & Luck, 1999; Hopf, Boelmans, Schoenfeld, Heinze, & Luck, 2002). Thus, the onset of the N2pc can be interpreted as a marker of the transition from the pre-attentive perceptual coding of the whole search array to the focal-attentional processing of selected – target – stimuli. Factors that facilitate the perceptual analysis of visual features should also facilitate subsequent feature-based attentional target selection processes, and this should result in an earlier onset and possibly also enhanced amplitude of the N2pc component. In the present study, we measured the N2pc in order to examine whether the intertrial facilitation effect in cross-dimensional visual search tasks is linked to the focal-attentional selection of targets. If this effect arises from enhanced perceptual processing within the target dimension on dimension repetition trials, resulting in more efficient attentional target selection (as assumed by the DWA), the N2pc elicited on such trials should be triggered earlier and/or be more pronounced than that on dimension change trials. In contrast to the RT effects, where intertrial facilitation effects are also dependent on response repetition (see above), this N2pc modulation should be observed irrespective of whether the response is repeated or changed. Alternatively, if the intertrial facilitation effect arises exclusively at a post-selective response selection stage, the N2pc should not differ between dimension repetition and dimension change trials.

The second component examined in the present study was the lateralized readiness potential (LRP). This component, typically observed over the motor area contralateral to the side of a unimanual response, is linked to the activation and execution of motor responses (e.g., Hackley & Valle-Inclán, 2003). To extract this component from the ERP, waveforms recorded from electrodes ipsilateral to the side of a response are subtracted from contralateral ERPs (Hackley & Valle-Inclán, 2003; see also Eimer, 1998, and Eimer & Coles, 2003, for methodological details about the derivation and interpretation of the LRP). LRP onset marks the start of effector-specific response activation and execution processes that occur after response selection has been completed. When measured relative to stimulus onset (stimulus-locked LRP), LRP onset differences across task conditions therefore reflect differences in the time demands of processing stages that occur prior to response activation. When measured relative to response onset (response-locked LRP), LRP differences across task conditions indicate differences in response activation and execution processes.

In the present study, both stimulus-locked and response-locked LRPs were measured to investigate whether and how changes versus repetitions of target dimensions and responses across successive trials affect response-related processing stages. Response-locked LRP waveforms were computed to assess any effects on response activation and execution stages. Response-locked LRPs were expected to be differentially affected by response repetitions versus alternations; the critical question was whether these LRPs would also be modulated by dimension changes. This should not be the case if dimension-specific intertrial effects in visual search only affect perceptual-attentional stages prior to response-related stages, as postulated by the DWA. In addition, stimulus-locked LRPs were computed to further investigate how dimension and response changes versus repetitions affect processing stages that precede response activation and execution. Because stimulus-locked LRP latencies are determined both by the time it takes to attentionally select and analyze the target and by the time required to select an appropriate response, these latencies may allow insights into the time demands of response selection processes that are intermediate between attentional target selection (indexed by the N2pc) and response production (indexed by the response-locked LRP).⁵ More specifically, we investigated whether processes at this stage might be responsible for the interaction between dimension and response changes previously observed for behavioral intertrial facilitation effects in compound tasks (Müller & Krummenacher, 2006). In contrast, the hypothesis that dimension-specific intertrial effects are based solely on response selection processes, as suggested by Mortier and colleagues (e.g., Mortier et al., 2005), would be consistent with systematic stimulus-locked LRP differences between dimension repetition and dimension change trials.

Method

Participants

Thirteen observers (8 female) took part in the Experiment. Their ages ranged from 21 to 36 years (mean age 28.5 years; SD = 6.5 years). Observers were either paid or received course credit for participating. All observers were right-handed, had normal or corrected-to-normal vision, and reported no history of neurological disorder. One observer had to be excluded from the analyses due to excessive eye-blink artifacts.

Stimuli and task

As illustrated in Figure1, the visual search display consisted of eight colored shape stimuli presented in a circular array against a black background, each presented equidistant (3.0° of visual angle) from a white central fixation cross. Each stimulus array contained one singleton which was equally likely defined in the color dimension (red circle, 1.2° radius) or the shape dimension (square, 2.4° × 2.4°) among seven distracters (blue circles, 1.2° radius). The position of the singleton was selected randomly from one of the six lateral positions. Each single stimulus contained a grating that was oriented either vertically or horizontally. The gratings consisted of three black bars (0.4° × 2.4°) separated by two gaps (0.3° × 2.4°). Observers were instructed to maintain central fixation throughout the experiment and to give a speeded forced-choice response indicating the grating orientation of the singleton target, using their left index finger or right index finger, respectively.

Procedure

Observers were seated in a dimly lit experimental chamber, with response buttons under their left and right index fingers. The positions of the response buttons were vertically aligned to avoid spatial stimulus-response compatibility effects. Stimuli were presented on a 17″ computer screen placed at a viewing distance of approximately 55 cm. Twenty experimental blocks of 72 trials were run. Each trial started with a white fixation cross for 500 ms, followed by the search array for 200 ms. The trial was terminated by the observer’s response or after a maximum duration of 1000 ms. During the intertrial interval, a central white fixation cross was shown for a variable duration of 950, 1000, or 1050 ms. Trials on which singletons were defined in terms of either color or shape, and trials on which target gratings were horizontal or vertical in orientation were presented in random order and with equal probability, thus resulting in an equal proportion of each of the four experimental trial conditions: same dimension – same response (sDsR), same dimension – different response (sDdR), different dimension – same response (dDsR), different dimension – different response (dDdR). Observers with odd participant numbers started with their left index finger on the upper button and their right index finger on the lower button, and vice versa for observers with even participant numbers. These response button assignments were changed in the second experimental half after ten experimental blocks. Prior to the start of each experimental half, observers performed at least one block of practice trials.

EEG recording and data analysis

EEG was recorded with Ag-AgCl electrodes mounted in an elastic cap (Falk Minow Service, Munich) referenced to linked earlobes. Electrode positions were a subset of the international 10/10 system sites (FPz, F7, F3, Fz, F4, F8, FC5, FC6, T7, C3, Cz, C4, T8, CP5, CP6, P7, P3 Pz, P4, P8, PO7, PO3, PO4, PO8, O1, Oz, and O2). The horizontal electrooculogram (HEOG) was recorded from the outer canthi of both eyes. Data was recorded with a BrainAmp amplifier (Brain Products, Munich) using an analog bandpass from 0.1 to 40 Hz and a digitization rate of 500 Hz. All electrode impedances were kept below 5 kΩ. Prior to epoching the EEG, Independent Component Analysis (ICA), as implemented in the software package Brain Vision Analyzer (Brain Products, Munich), was performed to eliminate blinks and horizontal eye movements from the EEG. Only trials with correct responses during the current and the preceding trial were selected for further analyses. Trials with signals exceeding +/− 60 μV on any recording channel were excluded from further analysis before the ERPs were averaged.

For the N2pc analysis, EEG data were epoched off-line into 1200 ms periods with a 200-ms pre-stimulus baseline that was used for the baseline correction. The N2pc was computed by subtracting ERPs obtained at lateral posterior electrodes PO7/PO8 ipsilateral to the side of the singleton stimulus in the visual search display from contralateral ERPs. Statistical analyses were conducted for N2pc peak latencies (latency of maximal negative amplitude in N2pc waveform between 190 and 270 ms post-stimulus) and mean amplitudes (obtained in the 190-270 ms post-stimulus latency window where the N2pc is maximal).

For the LRP analysis, response- and stimulus-locked waveforms were extracted from the EEG data. To obtain the response-locked LRP, EEG was epoched into 1200-ms periods that ranged from 800 ms before to 400 ms after response onset. No baseline correction was applied prior to artifact rejection and averaging. The stimulus-locked LRP was measured within a 1000 ms period after the onset of the search display, relative to a 200 ms pre-stimulus baseline. Both LRP waveforms were computed separately for all four trial conditions. This was done by subtracting the waveforms at electrodes C3/C4 ipsilateral to the side of the manual response from contralateral ERPs (used formula: (C4(left)-C3(left) + C3(right)-C4(right)) / 2). To determine the onset latencies of stimulus- and response-locked LRPs, we used the jackknife-based scoring method proposed by Ulrich and Miller (2001; see also Miller et al., 1998), which defines the LRP onset as the point in time where LRP amplitudes reach a specific criterion value relative to the pre-stimulus baseline. According to Miller et al. (1998) we used 50% and 90% of maximum LRP amplitude as an optimal criterion for determining stimulus-locked and response-locked LRP onset latencies, respectively. Statistical analyses were performed on stimulus- and response-locked LRP latencies, as well as on mean response-locked LRP amplitudes (obtained in the 100–20 ms interval prior to response onset).

Results

Behavioral data

Trials on which observers made an incorrect response (7.53% of all trials), trials on which the reaction time was excessively slow (> 1000 ms; 0.89%), and trials for which the response on the previous trial was incorrect (6.65%) were excluded from analysis (15.07% of all trials). Figure 2 displays the error rates and reaction times obtained in the remaining trials separately for each of the four experimental conditions. Reaction times were analyzed by a repeated-measures ANOVA with the factors dimension change (same dimension, different dimension) and response change (same response, different response). Both factors (dimension change: F(1,11) = 41.486, p<.001; η² = .790); response change: F(1,11) = 8.909, p<.012; η² = .447), as well as their interaction (F(1,11) = 57.73, p<.001; η² = .840) were significant. Further analysis (post-hoc contrasts, Tukey HSD) confirmed that RTs were significantly faster (p<.001) on trials on which neither the dimension nor the response changed relative to each of the other three trial conditions. There were no significant RT differences among trials on which either the dimension, or the response, or both factors changed (see Figure 2). This interactive pattern of effects mirror that observed in previous studies (Krummenacher et al., 2002; Müller & Krummenacher, 2006; Pollmann et al., 2006).

Figure 2.

Reaction times (lines) and errors rates (bars) as a function of dimension and response changes (sD = same dimension; dD = different dimension).

Error rates were examined by an analogous ANOVA, which revealed a main effect of dimension change (F(1,11) = 6.102, p<.031; η² = .357) as well as a significant dimension change × response change interaction (F(1,11) = 19.306, p<.001; η² = .637). Further analyses (post-hoc contrasts, Tukey HSD) revealed that, when the target-defining dimension stayed the same, more errors (p<.01) were made when the response changed than when it was repeated (i.e., observers tended to respond ‘same’). In contrast, when the dimension changed, slightly more errors (p<.11) were made when the response was repeated rather than changed (i.e, there was tendency to respond ‘different’).

Electrophysiological data

N2pc

Figure 3A shows the ERPs obtained at PO7/PO8 contralateral and ipsilateral to the side of a singleton target, collapsed across all four experimental conditions. As expected, an N2pc component was clearly visible. As can be seen from Figure 3B, search arrays that were preceded by same target-defining dimension elicited enhanced N2pc amplitudes as compared to arrays preceded by a different dimension (−2.25 μV (± 1.47) vs. −1.95 μV (± 1.29)). This effect was observed independently of repetitions or changes in the manual response.

Figure 3A.

Grand-averaged ERPs collapsed across all for experimental conditions at electrodes PO7/PO8. The solid line indicates ipsilateral activity and the dashed line contralateral activity in response to the singleton target.

Figure 3B.

N2pc. Averaged difference waves (contralateral activity minus ipsilateral activity) of the N2pc component for each of the four experimental conditions at electrodes PO7/PO8. Dark grey lines indicate dimension repetitions, light grey lines indicate dimension changes in consecutive trials. Solid lines indicate response repetitions and dashed lines response changes. The analyzed time window ranged from 190 to 270 ms poststimulus.

To formally assess the effects of dimension changes and response changes on this component, the N2pc was quantified by computing difference waves (contralateral activity minus ipsilateral activity) for each of the four experimental conditions, and repeated-measures ANOVAs were conducted for the mean N2pc amplitude obtained between 190 and 270 ms post stimulus. To test whether the N2pc was reliably elicited, we initially compared ERP mean amplitudes obtained during the baseline period and during the N2pc time window in a repeated-measure ANOVA for the factor period (baseline versus N2pc time window). A highly significant main effect of period (F(1,11) = 32.161, p<.001; η² = .745) confirmed the presence of the N2pc. Next, we conducted an ANOVA on mean N2pc amplitudes for the factors dimension change and response change that revealed a significant main effect of dimension change (F(1,11) = 5.984, p<.032; η² = .352). In contrast, there was no effect of response change (F(1,11) = 0.471, p<.507; η² = .041), and no interaction between the two factors (F(1,11) = 0.001, p<.977; η² = .000). An analogous repeated-measures ANOVA was used to examine N2pc peak latencies. As for the mean amplitude analysis, only the dimension change effect was significant (F(1,11) = 17.498, p<.002; η² = .614), with earlier peak latencies for trials on which the target-defining dimension was repeated relative to dimension change trials (243 ms (± 16) vs. 251 ms (± 17)). Again, no significant effect for response change (F(1,11) = 1.479, p<.249; η² = .119) and no significant interaction (F(1,11) = 0.364, p<.558; η² = .032) were obtained, indicating that the dimension change effect was manifest independently of the required response.⁶

LRP

Figure 4 presents the response-locked LRP waveforms for all four experimental conditions at C3/C4. There were no systematic onset latency differences between conditions. A dimension change × response change repeated-measures ANOVA of the response-locked LRP onset latencies (determined by the jackknife method of Ulrich & Miller, 2001) revealed no significant effects (dimension change, F(1,11) = 1.533; response change, F(1,11) = 1.913; interaction, F(1,11) = 0.014)⁷. However, there were systematic response-locked LRP amplitude differences: conditions in which the response on the current trial differed from that on the preceding trial exhibited more negative-going deflections prior to response onset (see Figure 4). For statistical examination, the LRP mean amplitudes obtained in the 100-20-ms window preceding response onset were subjected to a repeated-measures ANOVA with the factors dimension change and response change. In marked contrast to the N2pc, only the response change effect (F(1,11) = 7.115, p<.022; η² = .393) was significant, reflecting enhanced response-locked LRP amplitudes for response change trials (−1.76 μV (± 1.19) vs. −1.31 μV (± 1.13)). In contrast, the dimension change effect (F(1,11) = 0.464, p<.51; η² = .040) and the interaction between the two factors (F(1,11) = 2.142, p<.171; η² = .163) were non-significant. Hence, response-locked LRP amplitude was affected by response change only, independently of repetitions or changes in the target-defining dimension.

Figure 4.

Lateralized readiness potential. Response-locked averages for each of the four experimental conditions at electrodes C3/C4. Solid lines indicate response repetitions and dashed lines indicate response changes. Dark grey lines indicate dimension repetitions and light grey lines indicate dimension changes in consecutive trials. The analyzed time window ranged from −100 to −20 ms pre-response.

Figure 5 presents the stimulus-locked LRP waveforms obtained at C3/4, for all four experimental conditions. A dimension change × response change repeated-measures ANOVA was performed on the stimulus-locked LRP onset latencies (determined by the jackknife method of Miller et al., 1998). The fastest onset latencies were found for sDsR trials (341 ms (± 8)), followed by the latencies for dDdR (357 ms (± 4)) and sDdR trials (372 ms (± 5)). Stimulus-locked LRP onsets were most delayed for dDsR trials (407 ms (± 6)). The ANOVA revealed the main effect for dimension change (F(1,11) = 10.513, p>.008) as well as the interaction between dimension change and response change to be significant (F(1,11) = 14.232, p>.003), while the main effects for response change (F(1,11) = 0.262, p>.62) was not significant. The interaction was further examined by a series of pairwise comparisons using a Bonferroni p-level correction (as suggested by Miller et al., 1998). These comparisons revealed significant stimulus-locked LRP onset latency differences between all experimental conditions (p<.001).

Figure 5.

Lateralized readiness potential. Stimulus-locked averages for each of the four experimental conditions at electrodes C3/C4 for the 800-ms post-stimulus time interval relative to a 200-ms pre-stimulus baseline. Solid lines indicate response repetitions and dashed lines indicate response changes. Dark grey lines indicate dimension repetitions and light grey lines indicate dimension changes in consecutive trials.

Discussion

The present study was designed to investigate the mechanisms underlying dimension-specific intertrial effects in cross-dimensional visual search tasks. Specifically, the aim was to resolve the question whether the intertrial effects can be attributed to a single information processing stage, either a pre-attentive ‘perceptual’ or a post-selective ‘response selection’ stage, or whether both stages are responsible for some aspect of these effects. To address this issue, different ERP components which can be directly linked to different stages of information processing were examined: the N2pc, which reflects the allocation of focal attention to task-relevant stimuli based on perceptual attributes (Eimer, 1996; Woodman & Luck, 1999), and the LRP, which reflects the activation and execution of uni-manual motor responses (Hackley & Valle-Inclán, 2003; Eimer & Coles, 2003). These components were measured in a ‘compound’ task in which a dimension change across consecutive trials could occur independently of a response change, and vice versa. This task required observers to detect a color- or, alternatively, a shape-defined singleton target and then to select the appropriate left- or right-hand response which was determined by the horizontal or vertical orientation of a grating within the target object.

Effects of dimension change

Repetitions of the target-defining dimension on consecutive trials were associated with both shorter peak latencies and enhanced amplitudes of the N2pc component. In line with previous work on the N2pc (Eimer, 1996; Woodman & Luck, 1999), this pattern of effects can be interpreted in terms of a more efficient and faster allocation of focal attention to the current (repeated) target. Importantly, this effect was independent of repetitions or changes in the manual response, indicating that the efficiency of focal-attentional selection is solely determined by repetitions versus changes of the target-defining dimension across trials, and is not affected by concurrent repetitions versus changes in response-related attributes.

This systematic effect of visual dimension change on N2pc peak latencies is in line with the predictions of the DWA. According to this account, repeating the target-defining dimension on consecutive trials implies that the critical dimension is attentionally weighted on the current trial, thereby facilitating the emergence of the target’s saliency signal at the level of the overall-saliency map which guides the allocation of focal attention. By contrast, changes of the target dimension on consecutive trials lead to the engagement of a time-consuming ‘weight-shifting’ process. This process transfers attentional weight from the old to the new target-defining dimension, so as to amplify the target’s saliency signal above the detection threshold at the overall-saliency map level. The delayed peak latencies of the N2pc component for dimension change versus repetition trials may be interpreted as reflecting this weight-shifting process. It should be noted that the size of this N2pc latency shift (8 ms) was substantially smaller than the RT difference observed between sDsR trials and the other three trial types. This suggests that weight-shifting processes alone cannot account for this RT effect, but that other post-selective processing stages are also involved (see below). In addition, due to inter-individual and inter-trial variability of N2pc onsets, which will inevitably result in some ‘temporal smearing’ of this component, the observed onset latency differences are likely to underestimate the real contribution of dimension changes to the onset of the N2pc. Nevertheless, the fact that a significant delay of N2pc latencies on dimension change versus repetition trials was obtained demonstrates unequivocally that this factor did affect the timing of processes involved in attentional target selection.

In addition, target dimension changes also affected the amplitudes of the N2pc component. However, since the paradigm used in the present study does not provide a baseline measure, it is not clear whether the observed N2pc modulation represents an amplitude enhancement on dimension repetition trials, an amplitude reduction on dimension change trials, or both. From the DWA perspective, the N2pc modulation may be interpreted as reflecting both. That is, if the pre-attentive perceptual processing of task-relevant dimensions is facilitated on dimension repetition trials, preferential weighting of a given visual dimension is assumed to give rise to increased activation, or synchronized firing, of groups of neurons processing feature contrast signals defined in this dimension, thus resulting in more efficient allocation of focal attention compared to dimension change trials, and in increased N2pc amplitudes.

Taken together, the present N2pc results provide clear evidence in favor of visual-dimension weighting as conceived by the DWA, and against alternative accounts which assume that dimension-specific intertrial effects in visual search are exclusively generated at post-selective processing stages, such as response selection (Cohen et al., 1999; Mortier et al. 2005; Theeuwes et al., 2006). The N2pc differences between dimension change and repetition trials started to emerge as early as around 180 ms post-stimulus. This makes it extremely unlikely that this effect is in any way related to the motor response, especially when considering that the average response latency was around 570 ms. Furthermore, the present findings are in agreement with the study of Pollmann et al. (2006), who identified activations primarily in posterior visual areas in response to dimension changes. The spatial overlap between the areas described by Pollmann et al. and the lateral parieto-occipital electrode positions analyzed in the present study suggests common neural generators involved in processes of visual-dimension weighting (see also Hopf et al., 2002, for an MEG analysis of the cortical generators underlying the N2pc component).

Effects of response change

While changes versus repetitions of the required response across trials had no impact on the N2pc amplitudes and latencies, this factor affected the amplitudes (but not the onset latencies) of response-locked LRP waveforms. LRP amplitudes measured immediately prior to response onset were enhanced on trials on which the response hand changed relative to trials on which it remained the same as on the preceding trial. These response-locked LRP amplitude modulations related to response change were completely independent of repetitions and changes in the visual dimension of the target (see Figure 4).

Experimental manipulations of factors affecting response-locked LRPs usually result in onset latency differences, with earlier response-locked LRP onsets for conditions where the duration of response activation and execution processes is prolonged (see Eimer & Coles, 2003, for more details). However, no such latency shifts were observed in the present study, where the difference between response alternation and repetition trials was reflected instead by response-locked LRP amplitude differences. Several previous experiments have already found modulations of response-locked LRP amplitudes under conditions where the demands on response-related processing stages were varied. For example, Miller and Low (2001) measured LRPs in a simple RT task where the response to a target stimulus was specified in advance by a cue, and in a choice RT task where the response remained uncertain until the target was presented. In the simple RT task, where the cued response could be fully prepared during the cue-target interval, reaction times were almost 100 ms faster and response-locked LRP amplitudes were significantly reduced relative to the choice RT task. Similar response-locked LRP amplitude modulations have also been reported in a recent task switching study (Karayanidis, Nicholson, Schall, Meem, Fulham, & Michie, 2006).

These earlier findings, and the response-locked LRP amplitude modulations observed in the present experiment, suggest that these amplitude measures might reflect weight-shifting processes in response activation and execution that could be analogous to the process postulated for dimension changes. When the response (e.g., left index finger) remains the same on consecutive trials, some partial activation of the required response is carried over from the preceding trial and can thus facilitate the accrual of activation initiated by the new response signal, leading to faster reactions. As a result of the pre-existing response activation in the motor system, less additional activation is required to reach the motor threshold on response repetition trials, and this is reflected by reduced response-locked LRP amplitudes relative to trials on which the response hand had to be changed. On the latter trials, activation of the correct response involves an additional time-consuming shift of motor activation across hemispheres, prolonging the time required for the response activation process to be completed. It should be noted that, although response repetition and response change trials may have differed with respect to pre-existing response activation levels, response-locked LRP onset latencies were not modulated by response change (see Figure 4), suggesting that this factor did not systematically affect the time demands of response execution processes.

Interactions between dimension change and response change

The electrophysiological results discussed so far (N2pc and response-locked LRP) provide evidence that effects of dimension changes and response changes in visual search are generated at separate perceptual-attentional and response-related processing stages. However, the observation that the RT data did not show an additive pattern of dimension change and response change effects appears to be at variance with this conclusion. Recall that the observed RT pattern revealed fastest reactions when both the target-defining dimension and the required response remained the same on consecutive trials. Changes of the dimension, the response, or both, all slowed down RTs to a similar level. This interactive pattern of RT effects resembles that observed in previous studies (Krummenacher & Müller, 2002; Müller & Krummenacher, 2006; Pollmann et al., 2006; see also Olivers & Meeter, 2006, Figure 7; though some of the earlier studies had revealed marginal RT differences between conditions with at least one change, and, in Olivers and Meeter’s meta-analysis of five compound-task experiments, the interaction was not statistically reliable⁸). Thus, the present RT findings may be taken as supporting an interpretation along the lines suggested by Pollmann et al. (2006), namely, that repetitions of the target-defining dimension facilitate unchanged responses, whereas a dimension change disrupts any pre-set stimulus-response links, so that response selection and programming must start from scratch.

However, the present electrophysiological findings suggest a somewhat different account of the interactive pattern of RT effects. On this account, a heuristic version of which is illustrated in Figure 6, the interaction arises at a processing stage intermediate between focal-attentional selection and the response production, that is: stimulus-to-response translation or ‘response selection’. This account assumes that the observed effects on the N2pc and the response-locked LRP can be interpreted in terms of facilitated processing (resulting in faster processing times) at perceptual and response production stages, respectively, and takes into consideration the latencies and topographies of the N2pc (around 250 ms; extrastriate cortex) and the response-locked LRP (around 460 ms post-stimulus, i.e., 100 ms prior to response; primary motor cortex). Thus, as is illustrated in Figure 6, the early stage of focal-attentional selection is facilitated when the target-defining dimension remains unchanged, reflected in the present study by the effect of dimension change on N2pc amplitudes and latencies. The late stage of response production is facilitated when the response remains unchanged, and this was reflected (albeit indirectly) by the effect of response change on response-locked LRP amplitudes.

Figure 6.

Schematic illustration of the inferred processing times (black and grey lines) required by successive processing stages involved in performing a compound search task, for each experimental (dimension change × response change) condition. The summed processing times of the three stages yield the overall reaction time for a given condition. Black lines indicate processing times derived from interpreting the ERP results (N2pc: prolonged processing times for dimension changes; response-locked LRP: prolonged processing times for response changes). Grey lines represent inferred processing times derived by subtracting black lines from the overall reaction times.

Given this pattern of electrophysiological results, and the overall RTs in the four (dimension change × response change) conditions, it is possible to make inferences about the processing time required by the intermediate response selection stage. As illustrated in Figure 6, the assumption is that the duration of this intermediate stage is shorter when either both the dimension and the response remain the same or when both change; in contrast, it is prolonged when either the dimension or the response changes. This may be explained by postulating that the response selection stage assumes a correlation between the two types of change, even though dimension and response changes occurred independently of each other in the event statistics. That is, if focal-attentional analysis confirms the target dimension to be the same as on the preceding trial, the response selection system implicitly assumes that the response (and/or the attribute on which the response is based) will also remain the same, thus facilitating the selection of an unchanged response. By contrast, if the target dimension changes, the system assumes that the response (attribute) will change, too, thus facilitating the selection of a changed response. Note that the error pattern is consistent with such a linking of dimension and response ‘expectancies’. This linking may occur because it is easier for the system to change both expectancies than to change just one (see also Kingstone, 1992, who showed that such linkages may operate even when the relevant attributes are negatively correlated, rather than just uncorrelated). Note that, although phrased in terms of ‘response selection’, this account is neutral with respect to whether the linked expectancies exist between search-critical stimulus attributes and motor responses as such, or between search-critical and response-critical stimulus attributes (i.e., target-defining dimension and grating orientation).

Evidence in favor of the account illustrated in Figure 6 is provided by the pattern of stimulus-locked LRP onset latencies, which mark the transition between response selection and response production stages. The onset of response production is determined both by the duration of perceptual-attentional processes as well as by the duration of response selection. As demonstrated by the current N2pc results, perception and subsequent attentional selection are fast on trials on which the target-defining dimension is repeated (sDsR, sDdR), and slow on dimension change trials (dDsR, dDdR). Response selection is assumed to be fast on trials on which target dimension and response are both repeated or both changed (sDsR, dDdR), and slow when only one of them is changed (sDdR, dDsR). Thus, stimulus-locked LRP onset latencies should be fastest on sDsR trials, slowest on dDsR trials, and intermediate on dDdR and sDdR trials (see Figure 6).

This predicted pattern of stimulus-locked LRP onset latencies (sDsR < dDdR = sDdR < dDsR) was almost exactly matched by the observed data (sDsR < dDdR < sDdR < dDsR). The only exception was that onset latencies were 15 ms faster for dDdR trials relative to sDdR trials, whereas the model shown in Figure 6 predicts no latency difference between these two conditions. However, this prediction is based on the simplifying assumption that the effects of dimension change on the duration of perceptual-attentional stages, and of linked expectancies regarding stimulus and response changes on the duration of response selection stages are of exactly the same magnitude, which need not be the case. The earlier stimulus-locked LRP onset for dDdR relative to sDdR trials can easily be explained by assuming that the impact of linked expectancies on the time demands of response selection is more pronounced than the impact of dimension change on perceptual-attentional processing. In addition, it is conceivable that any delay in detecting the target in the changed dimension may make the response selection system tend towards a changed response – similar to a target-present/absent search task, where a delay in detecting the target makes the response system tend towards an ‘absent’ decision (see, e.g., Chun & Wolfe, 1996). This could have further shortened the duration of response selection on dDdR trials, resulting in an earlier stimulus-locked LRP onset. Whatever the exact explanation for the earlier LRP onset on dDdR trials, the more general and more important conclusion is that the observed stimulus-locked LRP onset latencies support the pattern derived from the proposed account.

Considering the pattern of stimulus-locked LRP effects together with the N2pc effects provides answers to the two questions addressed in the present study: (i) why is RT intertrial facilitation overall reduced in compound-search tasks and (ii) do these intertrial facilitation effects arise at an early perceptual and/or a later response-related stage of processing? The answer to the first question is that the overall RT intertrial facilitation effects are reduced because they are masked under response-change conditions by system-immanent linkages between stimulus and response. The answer to the second question is that RT intertrial facilitation effects originate at both (pre-attentive) perceptual and (post-selective) response selection-related stages of processing. Recall that the only effect evident in the RT data was the advantage for sDsR relative to dDsR trials (see Figure 2). According to our model, this advantage arises because of both faster perceptual processing and faster response selection on sD relative to dD trials. In contrast, there was no advantage for sDdR versus dDdR trials. According to our model, the lack on an effect is due to faster perceptual processing being counteracted by slower response selection on sD trials, with the reverse pattern on dD trials. In any case, where RT intertrial facilitation is observed, the N2pc latency advantage for dimension repetition trials (around 10 ms) is unlikely to account for the whole RT intertrial facilitation (of some 50 ms); rather, the effect is due to both expedited perceptual processing and expedited response selection.

Conclusion

In summary, the present study provides new insights into the mechanisms underlying dimension-specific intertrial effects in visual search tasks under conditions of high target saliency (low target ambiguity). Visual dimension changes and response changes elicited differential activation patterns affecting distinct ERP components. Dimension repetitions versus changes were reflected in the N2pc, indicating facilitated allocation of focal attention to targets defined in a repeated dimension. That is, at least part of the RT intertrial facilitation effect arises at a perceptual processing stage prior to focal-attentional selection. The observed response-locked LRP effects indicate that, with response repetitions on consecutive trials, the required response was pre-activated (‘weighted’) by the motor system. Concerning processes between focal-attentional selection and motor-response generation, the stimulus-locked LRP effects taken together with the N2pc effects suggest that another part of the RT intertrial facilitation effect arises at the response selection stage. This pattern of effects provides strong support for the dimension-weighting account, and appears inconsistent with views that dimension-specific intertrial effects are generated exclusively at post-selective response-related stages of processing.