What does the dot-probe task measure? A reverse correlation analysis of electrocortical activity

Abstract

The dot-probe task is considered a gold standard for assessing the intrinsic attentive selection of one of two lateralized visual cues, measured by the response time to a subsequent, lateralized response probe. However, this task has recently been associated with poor reliability and conflicting results. To resolve these discrepancies, we test the underlying assumption of the dot-probe task—that fast probe responses index heightened cue selection—using an electrophysiological measure of selective attention. Specifically, we used a reverse correlation approach in combination with frequency-tagged steady-state visual potentials (SSVEPs). Twenty-one participants completed a modified dot-probe task in which each member of a pair of lateralized face cues, varying in emotional expression (angry-angry, neutral-angry, or neutral-neutral), flickered at one of two frequencies (15 or 20 Hz), to evoke SSVEPs. One cue was then replaced by a response probe, and participants indicated the probe orientation (0° or 90°). We analyzed the SSVEP evoked by the cues as a function of response speed to the subsequent probe (i.e. a reverse correlation analysis). Electrophysiological measures of cue processing varied with probe hemifield location: Faster responses to left probes were associated with weak amplification of the preceding left cue, apparent only in a median split analysis. By contrast, faster responses to right probes were systematically and parametrically predicted by diminished visuo-cortical selection of the preceding right cue. Together, these findings highlight the poor validity of the dot-probe task, in terms of quantifying intrinsic, nondirected attentive selection irrespective of probe/cue location.

1. Introduction

One of the most fundamental aspects of human cognition is the attentive selection of sensory information for in-depth processing. Accordingly, the processes mediating selective attention have been extensively studied in the laboratory using a wide range of experimental paradigms and dependent measures. These studies have converged to show that selective attention is best conceptualized as a set of processes that act to resolve competition between concurrent sensory representations, memories, cognitive, and motor programs (Mangun & Hillyard, 1995; Posner & Rothbart, 1998; Reynolds & Heeger, 2009). One of these processes—of particular interest to applied and clinical scientists—has been the selection of motivationally or emotionally salient sensory information without explicit instruction by an experimenter (Bradley, 2009). A large body of work has shown that emotionally arousing sensory stimuli are detected more rapidly and accurately (Anderson, 2005; Öhman, Flykt, & Esteves, 2001), are perceived more vividly (Markovic, Anderson, & Todd, 2014), are remembered better (Bradley, Greenwald, Petry, & Lang, 1992), and interfere more strongly with competing tasks than neutral stimuli (Müller, Andersen, & Keil, 2008; Pessoa & Ungerleider, 2004).

Central to the present report, aversive or threatening cues presented away from an observer’s fixation capture and hold spatial attention as measured by behavioral (Calvo & Lang, 2005), electrophysiological (Keil, Moratti, Sabatinelli, Bradley, & Lang, 2005), and hemodynamic neuroimaging indices (Armony & Dolan, 2002). This phenomenon has interested clinical researchers, because it may serve as a laboratory model of hypervigilance to threat, a symptom of anxiety and post traumatic disorders (Bogels & Mansell, 2004). The term dysfunctional attention bias denotes excessive, maladaptive attentional orienting toward a class of stimuli, such as phobic cues (e.g. spiders, angry faces), trauma-relevant cues (e.g., combat noises) or cues consistent with depressive cognitions (e.g. rejection, devaluation). Many theoretical accounts of fear, anxiety, and mood disorders emphasize the potential causal role of these dysfunctional attention biases both for etiology and for maintenance of these syndromes (Bar-Haim, Lamy, Pergamin, Bakermans-Kranenburg, & Van, 2007; Bogels & Mansell, 2004; Elsesser, Sartory, & Tackenberg, 2005). In addition, intervention research has examined the benefits of changing dysfunctional biases through training as a potential treatment approach (Amir et al., 2009; Cardi et al., 2015; Hakamata et al., 2010). Moreover, given heightened attention allocation to threat cues away from fixation, incorporating spatial location in the task demand could a quantitative measure of content-specific biases in spatial attention allocation.

1.1 The dot-probe task

The most widely used task for assessing, and more recently for altering (Amir et al., 2009), emotional attention biases is the dot-probe task (MacLeod, Mathews, & Tata, 1986; Yiend & Mathews, 2005). The standard dot-probe paradigm presents two lateralized cues, one of which is subsequently replaced by a probe, which is typically two dots, oriented vertically or horizontally. Observers are instructed to keep central fixation during the cue period, and respond to the orientation of the dot-probe as quickly as possible. Fast responses to the probes are taken to index selective attention to the previous congruent (i.e. same location) cue. The rationale underlying this manipulation is that the covert allocation of spatial attention to one of the two lateralized cues will result in facilitation of the response to a subsequent probe that appears at the congruent (“attended”) location. Accordingly, with the neutral-threat cue pairs typically used, researchers compare response times for trials in which the probe replaces a threat cue and trials in which the probe replaces a neutral cue. Relatively faster responses to probes replacing threat cues are interpreted as an attention bias towards the threatening stimulus (see Yiend, Barnicot, & Koster, 2013 for a review).

Early conceptual criticism of this reasoning was based on findings observed with cued spatial attention tasks that involve explicit instruction cues, such as arrows instructing observers to “attend left”, instead of “implicit” cueing by threat or emotional cues (Posner, 1980). In these tasks, extending the cue duration or the cue-probe intervals often delays choice response times in the attended (cued) hemifield (Posner, Rafal, Choate, & Vaughan, 1985). This so-called inhibition of return effect (Tipper, Grison, & Kessler, 2003) and similar attention dynamics make the interpretation of response time differences in the dot-probe task difficult: The onset time of the implicit cueing effect exerted by a threat cue is not known, and the resulting attention dynamics may vary qualitatively across stimulus types, and observers (McTeague, Shumen, Wieser, Lang, & Keil, 2011). In addition to conceptual concerns, recent work has pointed to psychometric and methodological issues, finding low reliability of the dot-probe task (Kappenman, Farrens, Luck, & Proudfit, 2014; Schmukle, 2005; Waechter, Nelson, Wright, Hyatt, & Oakman, 2014), along with low external validity against robust electrophysiological measure of emotional attention (Kappenman et al., 2014), and a lack of consistent evidence that the dot-probe task captures automatic attention towards emotional stimuli (Puls & Rothermund, 2017). In the same vein, meta-analyses and reviews of the literature have found small effect sizes (Schoth, Nunes, & Liossi, 2012), absence of effects for specific clinical populations (Bantin, Stevens, Gerlach, & Hermann, 2016), as well as absence of effects in healthy observers (Bar-Haim et al., 2007).

Many studies also report conflicting results with the dot-probe task in clinical or sub-clinical (e.g., high-anxious) samples: Findings of faster probe responses at threat-cue locations (e.g., (Gilboa-Schechtman, Foa, & Amir, 1999; Bradley, Mogg, Falla, & Hamilton, 1998; Bradley, Mogg, White, Groom, & Bono, 1999; Mogg & Bradley, 1999) appear along with studies reporting faster probe response times at neutral-cue locations (De Ruiter & Brosschot, 1994; Mansell, Ehlers, Clark, & Chen, 2002; Chen, Ehlers, Clark, & Mansell, 2002). These opposing findings have amplified conceptual uncertainty regarding the processes mediating performance in the dot-probe task. For example, Koster and colleagues (Koster, Crombez, Verschuere, & De Houwer, 2004) noted that the standard result—faster responses to probes at threat-cue locations—may result from attention vigilance to threat, or from difficulty disengaging spatial attention from a once attended (threat cue) location, as predicted by the well-known model of spatial attention (Posner, 1980).

Basic research into the temporal dynamics of selective attention points to additional conceptual issues: Animal and human studies show that mechanisms of feature-based and spatial attention act largely independently (Andersen, Müller, & Hillyard, 2009; McMains, Fehd, Emmanouil, & Kastner, 2007). Importantly, they possess different time dynamics and may interact only at late stages of processing (Andersen, Fuchs, & Müller, 2009). These properties of selective attention mechanisms do not support a simple additive model of a global “attention” deployment that results in broad facilitation of threat cue processing, their location, and the subsequent probe stimuli alike (Reynolds & Heeger, 2009). Viewing attention as a unitary phenomenon is also at odds with recent behavioral (Herrmann, Montaser-Kouhsari, Carrasco, & Heeger, 2010; Ling & Carrasco, 2006) and neurophysiological studies (Frank & Sabatinelli, 2017; Andersen, Fuchs, et al., 2009), which have demonstrated that mechanisms of attentive selection vary with the to-be-attended stimulus dimension, showing a high degree of task-specificity. This complexity is illustrated by findings that responses to targets following a motivationally relevant stimulus are slowed, relative to those following a neutral stimulus (Heim, Ihssen, Hasselhorn, & Keil, 2012; Ihssen, Heim, & Keil, 2007), and that maintaining attention to a given location over several hundred milliseconds is accompanied by poor, as opposed to improved, performance (Ling & Carrasco, 2006).

Based on this literature, two hypotheses can be formed regarding the processes mediating dot-probe performance: First, one may hypothesize that faster dot-probe responses reflect some form of heightened attention to the preceding cue, deployed in ways that facilitate responding to a subsequent probe. Alternatively, one may hypothesize that slower responses reflect heightened attentional engagement with the preceding cue, leading to competition and interference with the processing of a different stimulus appearing at the same location. Given the ambiguous nature of the dot-probe task, it has not been possible to rigorously test these alternative views based on behavioral measures alone. Together, these considerations raise the question regarding the neurocognitive processes associated with performance (response speed) in the dot-probe task. Specifically, are fast versus slow probe responses in this task related to heightened versus diminished electrocortical facilitation of the preceding cues? The current study addressed this question, using a version of the dot-probe task that accommodated steady-state visual evoked potential (SSVEP) recordings.

1.2 The present study

The methodology used in this study is a reverse correlation approach, which refers to analyses where one measure (typically a behavioral response) is used as a grouping variable applied to instantiations of either the sensory input, or the neural responses (Ringach & Shapley, 2004). In a straightforward version of reverse correlation analysis, the neural responses belonging to one group (e.g., fast behavioral responses) are then averaged and compared with the neural response average based on the other group of trials (e.g., slow behavioral responses). Here, we use the response time to the probe as a grouping variable of the SSVEP response to the preceding cues. This quantifies the temporal dynamics of spatial selective electrocortical cue processing that is specifically related to slow versus fast probe responses in the dot-probe task. Using this objective and direct electrocortical metric of attentive selection, we investigate the validity of the assumption that fast probe responses occur when probes appear at spatially attended cue locations.

Steady-state visual evoked potential frequency-tagging is a powerful technique which possesses the properties necessary for a reverse correlation analysis of the dot-probe task: The SSVEP is evoked by a visual stimulus that is periodically and rapidly (> 4 Hz) modulated in terms of luminance or contrast (Norcia, Appelbaum, Ales, Cottereau, & Rossion, 2015). The ssVEP is extracted from scalp EEG signals as a robust oscillatory response at the exact frequency of the driving stimulus, primarily originating in peri-calcarine regions of the visual cortex (Di Russo et al., 2007). Of interest to the present study, the SSVEP signal and the tagging technique enable researchers to separately measure the cortical engagement associated with concurrently presented stimuli (Ding, Sperling, & Srinivasan, 2006; Wang, Clementz, & Keil, 2007). For example, two fully overlapping stimuli flickering at different temporal rates evoke different electrophysiological response trains that can be separated as two distinct, narrow, peaks in the frequency spectrum of the electroencephalogram (EEG) recordings (Appelbaum, Wade, Vildavski, Pettet, & Norcia, 2006). The SSVEP frequency-tagging technique can also be applied to naturalistic nonoverlapping stimuli in complex arrays (Wieser, Miskovic, & Keil, 2016), as used in the present study.

It is well established that the amplitude of the SSVEP is heightened when elicited by a stimulus that is selectively attended based on its spatial location (Müller et al., 1998; Clementz, Wang, & Keil, 2008) or because of its feature composition (Andersen, Hillyard, & Müller, 2008; Müller et al., 2006). Further, SSVEP amplitude is reduced for stimuli competing with task-relevant (Müller & Hübner, 2002; Wang et al., 2007) or emotionally salient visual stimuli (Müller et al., 2008; Attar & Müller, 2012). The time-varying SSVEP amplitude (the envelope of the SSVEP signal at the frequency of interest) to a given stimulus in an array has therefore been regarded a continuous metric of selective attention to a stimulus, under competition (Müller, Teder-Salejarvi, & Hillyard, 1998). Here, we implemented a version of the dot-probe that enabled leveraging this property: Two frequency-tagged faces served as cues, one in each hemifield, flickering at different temporal rates. Four different cue pairs were used: (1) neutral-neutral, (2) neutral-angry, (3) angry-neutral, and (4) angry-angry. A fully balanced design was used to ensure that each participant saw the same amount of neutral and angry faces across the course of the experimental session, thus facilitating reverse correlation analyses that were balanced for visual input. Faces were followed by an oriented (0° or 90°) sinusoidal gratings (Gabor patches) shown in the same location, which served as the probe, and observers were instructed to make a choice response to their orientation. Gabor patches were used because they evoke robust event-related potentials (ERPs), allowing extraction of additional neural indices of spatial attention to separate hemifields: P1 and N1 amplitudes elicited by the probes.

This experimental design allows for averaging of the electrocortical signal evoked by each cue individually as a function of probe response time, i.e., a reverse correlation approach (Ringach & Shapley, 2004). Given the present low-anxiety student sample, we did not expect effects of facial expression on response time (Bar-Haim et al., 2007), or on the SSVEP (McTeague, Shumen, Wieser, Lang, & Keil, 2011; Wieser, McTeague, & Keil, 2012). The vigilance interpretation of the dot-probe task would be supported by findings of heightened cue-SSVEP signal in trials with fast responses to the probe appearing in the same hemifield as the tagged cue. Interference or competition interpretations would be supported by findings of relatively smaller SSVEP amplitude for fast, compared to slow probe response trials in the cue-congruent hemifield.

2. Method

2.1 Participants

Twenty-one undergraduate students (15 females, M age = 19 years; SD = .9) participated for psychology course credit at the University of Florida. All participants reported right-hand dominance, normal or corrected-to-normal vision, and no personal or family history of epilepsy or photic seizures. They reported low levels of anxiety as measured by the trait form of the State-Trait Anxiety Inventory (Spielberger, Gorsuch, Lushene, Vagg, & Jacobs, 1983) prior to the experiment (mean = 33.4, SD = 6.4, range = 20–46). In young adults, recent normative studies with the STAI found that scores above 50 tend to separate the top 30% trait anxious individuals from low-anxiety raters (Crawford, Cayley, Lovibond, Wilson, & Hartley, 2011), illustrating the low levels of trait anxiety in the present sample. The study was conducted in accordance with the Declaration of Helsinki and all procedures were approved by the institutional review board of the University of Florida.

Stimuli

Visual stimuli comprised two probes and 80 picture cues. The two probes were Gabor patches, each created by filtering a grayscale sinusoidal grating with a Gaussian envelope. The task-relevant feature of the Gabor patches was implemented by manipulating their orientation to a vertical axis to either 0° (vertical grating) or 90° (horizontal grating). The resulting Gabor patch probes subtended a horizontal and vertical visual angle of 5.19° and had a spatial frequency of .50 cycles per degree visual angle. They had a Michelson contrast of .99 and were presented against a gray background with a luminance 31 cd/m², measured with a Gossen MavoSpot luminance meter.

Eighty pictures were selected from the Karolinska Directed Emotional Faces (KDEF) database to serve as cues. Each picture contained the portrait of one of forty actors (23 male, 17 female) with forward gaze. Each actor provided two pictures: one neutral and one angry emotional expression (Lundqvist, Flykt, & Öhman, 1998). Pictures were converted to grayscale, adjusted for average luminance (56.8 cd/m²) using the Matlab image processing toolbox, and subsequently resized to 200 × 200 pixels (occupying 5.71° visual angle). During each trial of the experiment, two faces of different identities were presented simultaneously to the left and right hemifield. The expression of each face was pseudo-randomized throughout the experiment, with equal numbers of trials in each of four expression conditions: neutral-neutral, neutral-angry, angry-neutral, or angry-angry. Each face was seen 4 times during the experiment. The neutral-neutral and angry-angry served as potential control conditions for disambiguating vigilance from difficulty disengaging from the cue, if a bias was found towards angry faces (Koster et al., 2004). They also ensured that participants saw identical numbers of angry and neutral faces across the experimental fashion, helping to prevent imbalances in the visual input contained in trials entering the reverse correlation analyses.

To quantify the selective processing of each of the two concurrent face cues, EEG was recorded while the two faces flickered rapidly on and off at two different frequencies (frequency-tagging). This stimulation resulted in steady-state visually evoked potentials (SSVEPs) evoked by each face, separable in the frequency domain. Flickering was accomplished by alternating between the gray background and the image: one at 15 Hz (66.67 ms flicker cycle) and the other at 20 Hz (50 ms flicker cycle). To achieve precise flickering at the 15 and 20 Hz tagging frequencies, the face tagged at 15 Hz was turned on for 4 refresh intervals and off for 4 refresh intervals of the 120-Hz monitor (refreshes occurring every 8.33 ms); the face tagged at 20 Hz was turned on and off every 3 retrace intervals. These face-specific on-off modulations were repeated for a total of 240 refresh intervals, resulting in a duration of the frequency tagging display of 240 * 8.33 ms = 2000 ms. Thus, a 15 Hz and a 20 Hz SSVEP were elicited in visual cortex, each corresponding to the face flickered at that frequency. Tagging frequencies were fully counterbalanced across participants, such that each location and face was tagged equally often with each of the two tagging frequencies.

2.2 Experimental procedure

After informed consent, participants were seated in a small, dimly lit room approximately 55 cm from a 23” LED monitor (refresh rate: 120 Hz) connected to a PC. EEG sensors were then applied to the scalp, and impedances were adjusted to the desired level (see below). Once participants were instructed on the experimental procedures, asked to maintain fixation on a central fixation cross throughout the entire experiment, and instructed to keep head and eye movements to a minimum, the experiment began (stimulus presentation: PsychToolbox 3.0.10; Brainard, 1997).

Each trial (Figure 1) consisted of a central black fixation circle (its diameter occupying 0.52° of visual angle) presented for 1.5 +/− 5 seconds (random rectangular distribution). Subsequently, two flickering faces (always of different identities) appeared to the left and right of the fixation circle (1.04° of visual angle deviation) for 2 seconds. Then, one of the faces was immediately replaced by a probe (Gabor patch), which remained on the screen until participants responded with a button press. The location of the probe was pseudo-randomized, such that the probe replaced each category of picture (i.e. either angry or neural faces) an equal number of times. Participants indicated the orientation of the probe as quickly as possible by pressing the L key to indicate a horizontal probe and the A key to indicate a vertical probe. Participants used the left hand for “A” and the right for “L”. The orientation associated with the L and A keys was counterbalanced between subjects. After the offset of the probe, participants viewed a blank gray screen for 1.5 +/− 5 seconds (random rectangular distribution). Participants completed 160 total trials (40 in each of the cue conditions), which lasted about 20 minutes.

Figure 1.

Task and Conditions. Each trial began with a 1–2 second fixation circle, followed by either a neutral or angry face on either side of the fixation circle. Four cue conditions were implemented: neutral – neutral (N-N), neutral – angry (N-A), angry – neutral (A-N), and angry- angry (A-A). In all conditions, both faces flickered for 2000 ms at different frequencies, to elicit separable SSVEP signals. Finally, a probe was presented on either the left or right side of the fixation circle, until participants responded to the orientation of the probe (either vertical or horizontal), in which case the probe disappeared from the screen. Note that these images are not drawn to scale.

2.3 EEG Acquisition and Processing

EEG was recorded continuously with a HCGSN 129 channel system by Electrical Geodesic (EGI). Electrode impedances were kept below 40 kΩ and the vertex electrode (Cz) was used as the recording reference. All channels were digitized at a rate of 500 Hz, and filtered online using a Butterworth on-line low pass filter with a 3-dB point (cut-off) at 200 Hz. All further data processing was done offline.

EEG was digitally filtered offline using a 2nd order Butterworth high-pass filter with a 1 dB point at .05 Hz, and a 12th order Butterworth low-pass filter with a 3 dB point at 40 Hz. Eye movement artifacts were then detected and corrected using an artifact linear regression correction method implemented in the Biosig suite of Matlab functions (Vidaurre, Sander, & Schlögl, 2011). Segments were then extracted from the continuous EEG, with each segment having a duration of 2800 ms (200 ms before and 2600 ms after face stimulus onset).

These segments were then submitted to a semi-automated artifact detection procedure designed for multi-channel electrophysiology, which is based on distributions of trial and channel statistics (Junghofer, Elbert, Tucker, & Rockstroh, 2000; Peyk, DeCesarei, & Junghöfer, 2011). First, quality of channels across all trials was assessed based on the distribution of three quality indices in relation to the recording reference (i.e., Cz): the absolute amplitude, the standard deviation, and the maximum of the first temporal derivative. Channels 5 standard deviations above the median of these three distributions were then replaced by spherical spline interpolation from the full channel set (Junghofer et al., 2000).

Next, data were converted arithmetically to the average reference, and distinct sensors from individual trials were also excluded and interpolated when located in the tails (5 standard deviation above the median) of the distribution of one of the three quality indices mentioned above. Trials in which interpolated channels were clustered in one scalp region and quantified as described in (Peyk et al., 2011) were also discarded. The median number of interpolated channels was 10 (SD 4.8, range 3 to 15).

Artifact-free trials were averaged in two different ways. First, cue-based averages were calculated by averaging across the trials belonging to the same cue (face pair) condition (i.e., neutral – neutral, neutral – angry, angry – neutral, and angry- angry). These averages served to examine any effects of facial expression on the SSVEP, and subsequent probe (Gabor patch) evoked ERPs (P1 and N1).

To address the main question of the present study, we then grouped trials by response time speed and probe hemifield, enabling reverse correlation analysis of the SSVEP and ERP. For each participant, trials in which the probe was presented in the left (80 trials) and right hemifield (80 trials) were separately sorted by the participant’s response time. Then, for each hemifield probe (left and right) separately, artifact-free trials above the probe response time median (slow probe response trials) and below the median (fast response trials) were averaged into 4 new condition averages: left-probe-fast, left-probe-slow; right-probe-fast, and right-probe-slow, irrespective of cue expression.

2.4 Analysis of Eye-Tracking data

Eye-tracking data were continuously sampled at 500 Hz using an EyeLink 1000plus system. This system tracked the right pupil using a video camera and an infrared light source located approximately 50 cm in front of the participant, just below the presentation monitor. Eye-tracking data were used to assure participants fixated on the central fixation circle during trials and did not move their eyes toward either the faces or probes. Trials were removed in which such eye movements were made. After artifact rejection, both from noisy EEG data (described above) and removal of trials were eye movements were made, a median of 25 trials per participant were kept in each of the 4 conditions (SD = 5.3, range = 12–37).

2.5 Response time and accuracy

Here, response time refers to the time between probe onset and the button press indicating the orientation of the probe. Outliers were removed from single trial response times to the probes by excluding trials with responses with latencies smaller than 180 ms or greater than 2.5 standard deviations above the median of each probe hemifield condition. Response times on correct trials for each participant were then averaged across trials separately for each cue condition and probe location. Response times were assessed with a 2×2×2 repeated measures ANOVA with a factor of probe location (levels: left or right), a factor of left-cue expression (levels: neutral or angry), and a factor of right-cue expression (levels: neutral or angry). Accuracy was calculated as the percent of button presses that correctly indicated the orientation of the probe. Accuracy was then evaluated statistically in the same fashion as response time, with a 2×2×2 repeated measures ANOVA having the same factors.

2.6 SSVEP as related to response time

2.6.1 Median Split Analysis

To test the hypothesis that fast probe responses are preceded by heightened selective attention to the preceding cue in the same location, a reverse correlation analysis was conducted. Two analyses assessed the degree to which fast versus slow response times to the probes were associated with previous visuocortical selection of a cue (i.e. a face).

In a first step, the reverse correlation analysis was conducted by averaging the EEG data relative to the speed of the left versus right probe trials, determined by within-subject median split within each probe hemifield condition. This resulted in four SSVEP averages time-locked to the cue onset, but organized by the subsequent probe response speed (faster versus slower than each hemifield’s median, determined for each participant). The ssVEP power was obtained by extracting the power at the tagging frequency. For SSVEP analysis at each tagging frequency, artifact free epochs of the voltage data were averaged by response time condition (above vs below the median) and the resulting time domain averages were submitted to a Hilbert transform, which estimates the time-varying amplitude of a stationary, narrow-band signal (Wieser et al., 2016). To this end, averaged data were filtered with a 10^th-order Butterworth band-pass filter having a width of .5 Hz around the center frequencies of 15 and 20 Hz, respectively. Then the time-varying amplitude was extracted as the complex conjugate of the band-pass filtered signal and the Hilbert-transformed analytic signal, for each time point. Dependent variables for statistical analysis (see below) were then obtained by averaging the time varying amplitude of the cue-evoked SSVEP in an early time segment (200–1000 ms post onset) and a late time segment (1000 – 1800 ms), across sensor clusters of the 5 sensors just to the left and right of Oz (depicted in Figure 2a).

Figure 2.

The SSVEP Signal. Panel A depicts the grand mean (N=21) frequency spectrum across all participants and conditions, taken from a sensor just right of Oz, across the time window highlighted in grey in Panel B (.5 – 2 seconds). Panel B depicts the grand mean across time from the same sensor. Time 0 is the onset of the frequency-tagged face cues (15 and 20 Hz tags), and the 2-second mark indicates the onset of the probe.

For the main statistical analysis, we focused on the congruent cue-probe pairs, which refers to one lateral dot-probe, and the SSVEP power representing the face cue previously in the same location, defined both by tagging frequency and scalp location (contralateral; Figure 4a). SSVEP power was extracted for the congruent cue-probe pairs (i.e., left cue - left probe, and right cue - right probe) separately for slow and fast responses. The resulting four SSVEP power values were submitted to repeated measures ANOVA with the within-subject factors of response speed (levels: fast versus slow probe response trials), and probe location (levels: left versus right probe—and thus also cue—location) entered the ANOVA. Follow-up t-tests were conducted where appropriate.

Figure 4.

SSVEP as Related to Response Time. Panel A depicts the time-varying SSVEP power at the tagging frequency of the right face (either 15 or 20 Hz) and the SSVEP power at the tagging frequency of the left face (either 15 or 20 Hz), during the cue presentation. The power of the tagging frequency is quantified as the mean across the contralateral sensor cluster, shown in red. The power is shown separately for trials where participants later responded either fast or slow to the subsequent probe. Panel B depicts the mean power across the scalp at the tagging frequency of the face cue, during the late cue window (1 – 2 seconds after onset of the cue). Topographies are shown separately by the tagging frequency of either the right or left face cue, for subsequent fast versus slow responses to the probe, and by whether the probe appeared on the left or the right.

A number of control analyses were conducted to test alternative hypotheses. To test if incongruent probes had a different effect compared to congruent cue-probe pairs, a repeated measures ANOVA with the same factors (response speed and probe location) was conducted, with the dependent measure being the SSVEP amplitude at the frequency tag of the respective incongruent cue, relative to the probe. To test the specificity of the topography of these effects, both these previous congruent and incongruent ANOVAs were repeated with the dependent measure being taken from the ipsilateral (rather than contralateral) EEG sensors, relative to the cue location. To assess whether the facial expression of the cues had an impact on the SSVEP amplitude, a repeated measures ANOVA was conducted with factors of tagging frequency (2 levels: left cue tagging-frequency and right cue tagging-frequency) and cue condition (4 levels: neutral-neutral, neutral-angry, angry-neutral, and angry-angry). To assess the temporal specificity of these effects, all analyses were conducted on both late and early windows within the cue period as described above. As a final method check to test if the frequency tagging of the hemifield cues—despite counterbalancing, see above—resulted in any residual differences between hemisphere/hemifield conditions, we also conducted a t-test on the ssVEP power from the left hemisphere tag compared to the right.

2.6.2 Quartile Split Analysis

To examine the parametric co-variation of probe response time and electrocortical processing, we next implemented a reverse correlation analysis with quartiles, rather than based on a median split. Trials were again divided into left and right probe trials, and then into quartiles (fastest, fast, slow, and slowest trials) within each participants and probe location, based on probe response time. The EEG data were then averaged for these eight trial groups. This resulted in lower signal-to-noise ratio of the EEG data, but higher resolution for examining the relationship between the ssVEP power and the behavioral data. Based on the time information available from the median-based analysis, signal-to-noise ratio was amplified in the quartile analysis by implementing a moving window average procedure across the late interval (1000 – 1800 ms), immediately preceding the probe before extracting power via a Discrete Fourier transform (Keil et al., 2008). This step enabled reliable estimation of the SSVEP amplitude for quartiles, at relatively low trials counts, i.e. a mean of 11.9 (SD = 2.2) trials per quartile. To quantify the parametric change of the cue-specific SSVEP with response time quartile, a linear F-test was conducted with weights [3 1 −1 −3], across the quartiles from fastest to slowest, reflecting the hypothesis that faster probe responses were preceded by larger SSVEP power.

2.7 ERP Components

Early components P1 and N1 of the event-related potential (ERP) in response to the probe were used as additional indices of selective processing of the probe (Luck, Fan, & Hillyard, 1993). To quantify the P1 amplitude, voltage was averaged across a time window 118 – 146 ms post stimulus onset (identified based on the grand mean peak latency), in a right sensor cluster centered around PO4, PO8 and their 16 nearest neighbors, and a mirrored left sensor cluster centered around PO3, PO7, and their 16 nearest neighbors. To quantify the N1 amplitude, voltage was averaged across a time window 218 – 258 ms post stimulus (probe) onset, across the same two sensor clusters. Again, this time range was identified based on the grand mean ERP waveform. Within each group, the P1 and N1 to the probe were averaged separately at both sensor clusters across trials based on the location of the probe (either left or right) and response speed. P1 and N1 values were then submitted to a 2×2×2 repeated measures ANOVA containing within-subject factors of electrode cluster, probe location, and response time grouping.

2.8 Statistical analysis: Summary

Five dependent measures were analyzed. To simultaneously quantify selective processing of each face cue, frequency-tagged SSVEPs were analyzed. To assess responses to the subsequent probes, two behavioral measures were analyzed, response time and accuracy, and two electrophysiological measures were analyzed, the P1 and N1 ERP components. The analysis of each dependent variable was performed with a repeated measures ANOVA. All significant (p < .05) effects in the omnibus ANOVA were followed by simple main effects analyses (Schabenberger, Jr, & Kong, 2000). These were conducted within hemisphere and within hemifield (were appropriate), to directly address the hypotheses of the present study, and corrected for multiple (two) comparisons using the Bonferroni method. Deviations from sphericity were addressed by using F and p statistics corrected by means of the Greenhouse-Geisser method (Greenhouse & Geisser, 1959).

3. Results

3.1 Behavioral data

The repeated measures ANOVA on response time indicated a main effect of probe location, F(1,20) = 6.00, p = .02, η_p² = .23, such that participants responded slightly faster to right probes (M = 802 ms, SD = 143) compared to left probes (M = 815 ms, SD = 155). There was no main effect of left cue expression or right cue expression (Figure 3), and no interactions. The repeated measures ANOVA on accuracy resulted in no main effects or interactions (see Table 1).

Figure 3.

Individual participants’ response speed. Each line represents the mean response speed of an individual participant across the four facial expression conditions: neutral-neutral, angry-neutral, neutral-angry, and angry-angry. No statistical differences were detected across the facial expression conditions.

Table 1.

Behavioral Data. Median probe response latency (seconds) and accuracy (% correct) by cue condition. Participants responded faster to right probes compared to left probes, but there were no significant differences between facial expression cue conditions.

	Response Latency (SD)		Accuracy (SD)
Cue Condition	Left probe	Right probe	Left probe	Right probe
Neutral - Neutral	.78 (.14)	.76 (.17)	95.5 (6.4)	96.7 (6.9)
Neutral - Angry	.76 (.17)	.75 (.12)	97.3 (5.2)	98.7 (3.5)
Angry - Neutral	.77 (.17)	.76 (.17)	97.6 (5.6)	97.8 (4.1)
Angry - Angry	.78 (.16)	.77 (.14)	97.0 (6.1)	98.2 (4.9)

3.2 Steady-state visually evoked potentials

The SSVEP signal

The steady-state visually evoked potential (SSVEP) elicited by the two flickering faces elicited an increase in power at the two tagging frequencies: 15 and 20 Hz (Figure 2A). In the time domain, a robust SSVEP waveform was observed in all participants (Figure 2B) during the 2-second interval when the faces were presented. A clear event-related potential (ERP) was elicited to both the presentation of the face cues and the probe onset.

SSVEP as related to response time

The congruent cue-probe ANOVA in the contralateral hemisphere during the late portion of the cue had no main effect of response speed or probe location, but did show a significant interaction between response speed and probe location, F(1, 20) = 12.779, p = .002. This interaction was followed up by examining a simple main effect of response speed in each visual field. For the left cue, SSVEP power of trials containing fast response times (M = .204 µV, SD = .131) were associated with a significantly larger SSVEP amplitude compared to the SSVEP power of trials containing slow response times (M = .159 µV, SD = .095), F(20) = 2.598, p = .017. In the right cue location, the SSVEP power of the fast response group (M = .120 µV, SD = .127) was significantly smaller compared to the slow response group of trials (M = .156 µV, SD = .110), F(20) = 3.0, p = .007, (Figure 4). These tests were both significant after correcting for multiple comparisons using the Bonferroni method, which yielded a significance threshold of p = .025.

In a more parametric approach, a reverse correlation analysis was conducted on quartiles instead of the median split, and a linear F-contrast test was conducted in each hemisphere. This resulted in a significant linear contrast for probes shown in the right visual field, F(63) = 5.741, p = .019, but no significant linear contrast for probes shown in the left visual field. Specifically, the pattern that relatively faster responses to right-hemifield probes were preceded by relatively lower right-cue-evoked SSVEP power was robust (Figure 5). The pattern that relatively faster responses to left-hemifield probes were preceded by relatively enhanced right-cue-evoked SSVEP power did not persist in this more fine-grained analysis.

Figure 5.

SSVEP as Related to Response Time: Quartile analysis. The SSVEP power of the congruent cue, taken from the one second cue interval just preceding the dot-probe onset. EEG trials were split into quartiles by dot-probe response time, separately for each visual field of the probe (Left and Right), to examine the hypothesis that faster response times are preceded by larger SSVEP power, or more selective attention to the previous cue.

Control analyses

None of the control analyses to test alternative hypotheses had any significant main effects or interactions (no F values above 1). There were no overall significant differences between the left and right tagging frequencies. There were no significant effects found in the ANOVAs conducted in the early cue window, and the following ANOVAs conducted in the late cue window: incongruent cue-probe ANOVA in the contralateral hemisphere, the congruent cue-probe ANOVA in the ipsilateral hemisphere, the incongruent cue-probe ANOVA in the ipsilateral hemisphere. As expected, the ANOVA assessing effects of the facial expression in the cues showed no main effect of cue condition and no interactions between cue condition and tagging frequency.

3.3 ERP Components

The 2×2×2 repeated measures ANOVA on the P1 amplitude showed no main effects for probe location, electrode cluster, or response time grouping, and no interactions between response time grouping and either of the other factors. These results suggest that response latency was not related to P1 amplitude, and that P1 was not biased toward a probe location. Voltage increased on the contralateral scalp relative to the visual field of the probe, reflected in a significant interaction of electrode cluster by probe location F(1,20) = 4.76, p < .001, η_p² = .19. Follow-up simple main effects were both significant, such that for probes in the left visual field, the P1 was larger at the right electrode cluster, F(20) = 2.12, p = .046, η_p² = 0.69, and vice versa, such that probes in the right visual field were associated with larger P1 amplitudes over left electrodes F(20) = 2.15, p = .044, η_p² = 0.68. These results reflect the contralateral nature of brain activity relative to stimulated visual field, and were only significant at the trend level after the Bonferroni correction for multiple comparisons, which yielded a significance threshold of p = .025.

The 2×2×2 repeated measures ANOVA on the N1 amplitude indicated a 2-way interaction between probe location and electrode cluster, F(1,20) = 22.26, p < .001, η_p² = .53, reflecting contralateral brain activity relative to stimulated visual field, and a 3-way interaction of probe location by electrode cluster by response time grouping, F(1,20) = 10.96, p < .001, η_p² = .35. There was a marginal main effect of response time grouping, F(1,20) = 4.24 p = .053, η_p² = .175. Follow-up ANOVAs by hemifield were conducted separately for the left and right electrode clusters, with factors probe location and response time grouping. In the right electrode cluster, only a main effect of probe location was found F(1,20) = 12.227, p = .002, η_p² = .379. Again, this reflects the contralateral nature of brain activity relative to stimulated visual field, and suggests that the N1 was not related to response time. However, in the left electrode cluster, a main effect of probe location was observed, F(1,20) = 21.617, p < .001, η_p² = .519, along with an interaction between probe location and response time grouping, F(1,20) = 6.302, p = .022, η_p² = .236. Follow-up simple main effects of response speed were conducted separately by probe location. When the probe was presented in the right visual field, the N1 amplitude was larger for fast RTs compared to slow RTs, F(20) = −2.605, p = .017, η_p² = .80, only in the left cluster. This was significant after the Bonferroni correction for multiple comparisons, which resulted in a significance threshold at p = .025. RTs in response to probes in the left visual field were not significantly related to N1 amplitude (Table 2).

Table 2.

N1 component. The grand mean (N=21) of the N1 amplitude (µV) to the onset of the probe stimulus (Gabor patch) is given for the left and right sensor clusters. N1 amplitudes are shown by probe location (left or right visual field) and by response speed (within-subject median split of fast versus slow trials).

	Mean (SD) N1 Amplitude
Condition	Left cluster	Right cluster
Fast RT to Left Probe	0.11 (1.1)	−1.1 (1.7)
Slow RT to Left Probe	0.43 (0.8)	−0.8 (1.2)
Fast RT to Right Probe	−1.2 (1.6)	−0.1 (1.2)
Slow RT to Right Probe	−0.4 (0.9)	−0.2 (1.0)

4. Discussion

The current study investigated the neural dynamics of selective attention that might explain the discrepancies in the dot-probe literature and reports of its poor reliability (Kappenman et al., 2014). The dot-probe task is based on the assumption that faster probe responses reflect the selective deployment of covert spatial attention allocation to a cue preceding that probe at the same location. We examined this assumption by quantifying selection dynamics at the electrocortical level, using frequency-tagged SSVEPs in a reverse correlation approach. Findings did not support an interpretation of fast probe responses as indexing selective electrocortical facilitation of the respective hemifield. Instead, we found that SSVEP amplitude changes prior to the probe demonstrated heightened facilitation in fast probe trials for cues presented in the left visual field. By contrast, fast responses to right visual field probes were associated with relatively diminished SSVEP amplitude evoked by right visual field cues, suggesting less selection, or relative suppression, of the right visual field cue in fast probe-response trials, compared to slow-response trials. Furthermore, both effects occurred late, but not early, during a 2-second cue presentation period. This pattern of results is inconsistent with using the speeded response to the probe as a putative index of a monolithic attention construct, assumed to be sustained throughout the cue and probe periods.

The present findings can be interpreted in the context of the extensive literature on hemispheric asymmetries in spatial selective attention tasks (Vossel, Geng, & Fink, 2014). Classical theoretical accounts of hemispheric asymmetries in spatial attention have emphasized the role of the right hemisphere in visual orienting, vigilance and cued attention allocation (Posner & Petersen, 1990, Corbetta et al., 1998, Hellige, Laeng, & Michimata, 2010). Seminal studies in this field have demonstrated that spatial selection of visual stimuli in the left visual field is often reflected in robust right hemispheric amplification, seen at parietal (Szczepanski, Konen, & Kastner, 2010) and occipital locations (Keil et al., 2005). By contrast, selection of a right visual field stimulus has often been associated with bilateral engagement, measured by means of hemodynamic (Kim et al., 1999) as well as electrophysiological measures (Kelly, Lalor, Reilly, & Foxe, 2006), and consistent with lesion work (Mangun et al., 1994). Traditionally, this has been taken to indicate the over-representation of cortical tissue mediating spatial selectivity in the right hemisphere (Corbetta et al., 1998). This traditional view has been refined and qualified based on more recent work (Szczepanski et al., 2010), leading to the conclusion that hemispheric asymmetries in spatial attention to a large extent depend on the nature of the task or paradigm (Reuter-Lorenz, Kinsbourne, & Moscovitch, 1990), as well as the dependent variables used (Foxe & Snyder, 2011; Foxe, McCourt, & Javitt, 2003). Given this important role of the paradigm and dependent variable, asymmetries in SSVEP studies of spatial attention are of interest here. These studies have consistently shown focal posterior amplification of visuo-cortical neural mass activity, contralateral to the attended visual field, for both right and left visual fields/hemispheres (Keil et al., 2005; Morgan, Hansen, & Hillyard, 1996; Müller & Hillyard, 2000). Thus, the present finding of specific SSVEP amplitude reduction only for right visual field cues in fast right probe RT trials suggests that inattention to the right visual field cue enabled faster subsequent probe responses. This hemisphere bias is particularly pronounced in studies using face cues, which may be further explained by evidence suggesting the right hemisphere processes faces faster and more deeply (Kanwisher, McDermott, & Chun, 1997; Eimer, 2011). Research on attention competition across time may thus provide additional clues toward the interpretation of these effects, and is considered in the following paragraph.

A pattern of initial, sustained attentional selection, followed by lapses in performance to subsequent targets is predicted by a body of work in the field of temporal attention dynamics (Ling & Carrasco, 2006; Wieser & Keil, 2011). Specifically, emotionally engaging, but task-irrelevant cues prompt sustained subsequent interference effects on the performance of a subsequent, secondary task, even when cue prompts are no longer present (Ihssen et al., 2007; Most, Chun, Widders, & Zald, 2005). These interference effects are particularly pronounced for stimuli presented in the left visual field (Hartikainen, Ogawa, & Knight, 2000), which supports the notion that the right hemisphere is particularly sensitive to capture effects induced by briefly presented salient cues (Junghöfer, Bradley, Elbert, & Lang, 2001; A. Keil et al., 2005). To the extent that the present study found efficient probe discrimination in the left visual field after sustained facilitation of the same visual field, the literature reviewed in the previous paragraphs strongly suggests that probe performance does not reflect the temporal extension of an initial capture effect exerted by the cue, but rather hemispheric asymmetries in how spatial selection extended over different time periods affects subsequent probe processing. Depending on cue duration and cue-probe interval, the dot-probe paradigm may tap into a complex nexus of facilitation effects, interference effects, and time-varying effects of sustained selection as described above, potentially affecting performance in opposite directions. For example, the present response times were longer than typically observed in dot-probe studies using shorter cue durations (Puls & Rothermund, 2017; Bradley et al., 1998), which may be differentially affected compared to studies cue durations compared to the ones used here (Bradley, Field, Mogg, & De Houwer, 2004).

The present study suggests that rapid changes in left visual field are more readily detected by the right hemisphere, with performance outcome depending on enhanced cortical engagement immediately prior to probe onset. However, and in line with some classical models of spatial attention (Corbetta et al., 1995), for right visual field cue-probe presentations, the left hemisphere may be at a disadvantage and potentially requires additional cross-hemispheric processing for probe detection. In the case of right visual field presentation, attentional disengagement may in fact be advantageous to the slower, left visual hemisphere. This is further supported by our more fine-grained reverse correlation analysis using quartiles, given that the right visual field probe response times were more likely to be associated with large SSVEP response on slow trials, relative to fast trials, but the left visual field probe response times did not show a significant linear relation with electrocortical facilitation. These results suggest that selectively attending to a right-hemifield cue is associated with slower response times to a subsequent probe in the right visual field. By contrast, the facilitation of left visual field probe response times associated with attending a preceding left visual field cue appears less robust and awaits further investigation, potentially using different cue durations.

The present study was limited to one cue duration, selected to be relatively long, to enable robust SSVEP time series estimation. At 2000 ms, the present cue duration was longer than most studies using the dot-probe task to quantify attentional biases to social threat cues (Bantin et al., 2016). In past work on sequence processing with emotional distractors, interference effects exerted by initial emotional distractors on subsequent target detection performance lasted for several hundreds of milliseconds (Hartikainen et al., 2000; Heim & Keil, 2012). This raises questions regarding how the selection dynamics identified by reverse correlation vary with different cue-probe intervals. Future research may use the SSVEP-reverse correlation approach with shorter cue durations to address this question. However, the N2pc, another electrocortical index of selective attention, was found in a dot-probe task with 500 ms cues to also exhibit lateralization effects, such that shorter N2pc latencies were found for left angry faces, but this effect was not observed in the right visual field (Reutter, Hewig, Wieser, & Osinsky, 2017). Taken together, this suggests that this effect is relatively robust with respect to cue duration, given that this effect holds true with relatively shorter (i.e. 500 ms) or longer (i.e. 2000 ms) cue durations.

As expected (Yiend & Mathews, 2005), probe response time analyses comparing the effects of facial expression cues did not show differences related to cue content, in the present low-anxiety sample. Similarly, SSVEP amplitudes were not affected by facial expressions, replicating earlier studies in low-anxiety samples (McTeague et al., 2011; Wieser, McTeague, & Keil, 2011). Instead, a hemifield response time bias was observed, such that probes presented in the right visual field elicited faster response times. A post hoc analysis compared vertical and horizontal probes as well as response hand, for left and right visual field probe RTs. This analysis showed that the right visual field advantage, traditionally observed with nonverbal, symbolic cues (Geffen & Wallace, 1971), was driven solely by vertical, but not horizontal probe orientations. This effect highlights previously identified differences in discriminatory sensitivity between cardinal and noncardinal orientations (Furmanski & Engel, 2000), as well as between vertical and horizontal orientations (Rovamo, Virsu, Laurinen, & Hyvärinen, 1982). Although probe orientations are typically counterbalanced across cue conditions in dot-probe tasks, as was done in the present study, comparison of vertical versus horizontal probes appears to introduce unwanted variability. Researchers may therefore wish to use noncardinal orientations in studies where orientation (e.g., upright colon symbol vs. horizontal colon symbol) is used to prompt discriminatory responses.

We also investigated ERPs to probe-onset as an ancillary metric of electrocortical probe processing. The use of the ERP data was constrained by the fact that limited trial counts were available, because the study was designed to provide SSVEP data. Despite their less than satisfactory signal quality (Thigpen, Kappenman, & Keil, 2017), ERPs collected in the present study supported the validity of key manipulations and analytic steps taken in this study: The P1 amplitude demonstrated amplification contralateral to the probe location, for both visual fields, with no sensitivity to fast or slow selective responding to the probe feature, replicating electrophysiological studies on feature-selective target selection (Luck, Heinze, Mangun, & Hillyard, 1990). By contrast, the subsequent N1, known to be sensitive to discrimination processes (Vogel & Luck, 2000), showed heightened amplitudes in fast RT trials compared to slow RT trials, for left posterior sensors in response to right visual field probes. The fact that this expected difference was confined to the right visual field may be a result of insufficient trial counts, or may reflect cerebral asymmetries as observed with SSVEPs.

In summary, the present study provides an initial demonstration of the reverse correlation approach with SSVEPs applied to a widely used translational task. Specifically, we quantified the electrocortical selection processes underlying the response time differences observed when participants respond to a probe following a set of lateralized cues. Replicating and extending previous findings that cast doubt on the external validity of this task (Kappenman et al., 2014), we found that fast probe responses were related to heightened visuocortical selection in one hemisphere, but not the other. The present results warrant replication and extension—for example to include a range of different cue durations. They also raise substantial doubt regarding the interpretation of fast versus slow probe responses as indicative of a unitary, temporally sustained attentional amplification mechanism, the location of which can be prompted by aversive cues and remains in place after the cues disappear. Instead, the present study suggests that a complex interplay of temporal interference, sequence effects, cueing, and stimulus visual field, result in a net outcome (response time) that is difficult to interpret. These limitations may explain the growing reports of unsatisfactory reliability and validity associated with the dot-probe task.