Behavioral-Based Sorting of Single-Trial EEG Data Reveals Evidence for ERP Correlates of Facilitation and Interference in the Picture–Word Interference Task

Abstract

A paradigm that has been widely used in cognitive science to examine questions about language production has produced an as-yet-unresolved conundrum in the findings of several event-related potential (ERP) studies. Using the picture–word interference (PWI) task, these studies have found behavioral evidence of interference effects (i.e., slower and/or less accurate responses) but ERP evidence of typical priming (i.e., smaller N400s) when to-be-named pictures are presented with semantically related words. To address these seemingly contradictory results, we performed a semantic PWI experiment with two modifications from the typical design. First, we implemented a novel behavior-contingent analysis of semantically related word–picture pairs in which single-trial ERP data were reaveraged for word–picture pairs that either showed a facilitation pattern or an inhibition pattern (faster or slower RTs for related than unrelated pairs). Second, we extended the ERP analysis epoch to latencies not typically analyzed in previous studies due to concerns about speech artifact. In the behavioral contingent averaged ERPs, the “facilitation bins” produced typical priming in early (200–400 ms) and late (400–600 ms) N400 epochs, while the “inhibition bins” produced reversed priming (greater negativity for related than unrelated prime words) in the late N400 epoch. In addition, after correcting for speech artifact, we observed reversed priming for the semantic condition in a later epoch (700–1200 ms). Together, these results help resolve the apparent conflict between neural and behavioral responses in the PWI paradigm and suggest a way forward in using response-contingent averaging of ERP data in language production.

A review from Sá-Leite et al. (2023) found that at least 192 studies have reported results using the picture–word interference (PWI) task. In the standard version of this task, participants are instructed to rapidly name common objects presented as pictures. However, in addition to the picture, a written word that is either unrelated or is in some way related to the name of the depicted object is also presented. In the semantic version of this task, the meaning of the written word is related to the name of the pictured word. For example, a picture of a DOG is presented along with a written word like cat or some other neutral, unrelated word like pan. Participants are typically told to ignore the printed word and name the depicted object as quickly as possible. In most studies (e.g., Alario et al. 2000; Caramazza and Costa 2000; Glaser and Düngelhoff 1984; Lupker 1979; Schriefers et al. 1990; but see, e.g., Costa et al. 2005), participants are slower and/or less accurate in naming objects when the word (cat) is semantically related to the name of the depicted object (DOG) compared to when the word is unrelated to the object (pan). There are a variety of explanations that have been proposed for this result (e.g., Dhooge and Hartsuiker 2010; Finkbeiner and Caramazza 2006; Mahon et al. 2007; Nozari and Hepner 2019; Rahman and Melinger 2007, 2009; Riès et al. 2013; Roelofs, Piai and Schriefers 2013; Roelofs 1992; Roelofs 2018), but what all of these accounts have in common is the notion that, at some level in the language production system, there is increased competition between semantically related items compared to semantically unrelated items.

Consistent with the notion of increased competition for semantic pairs is the finding of naming facilitation (faster RTs and more accurate responses) with so-called identity pairs. In this case, the word paired with the picture is the actual name of the depicted object (e.g., the word dog presented with a picture of a DOG). Like the case of semantically-related pairs, identity pairs are contrasted with word/picture pairs that are unrelated (e.g., pan-DOG—see Glaser and Düngelhoff 1984; Roelofs et al. 2016).

Because behavioral data tend to aggregate the variability of the processes of interest to a point after the relevant neurocognitive processes, they are not ideal for making inferences regarding the time course of the component processes during language production. More recently, a number of studies have turned to electrophysiological techniques to help shed more light on the processing dynamics involved in the patterns of facilitation and interference in the PWI task. Event-related potentials (ERPs), because of their continuous nature and differential sensitivity to the time course of processing, are arguably a more sensitive approach to studying language processing, especially when combined with behavioral measures (see Luck, 2014). Of most interest here is a curious finding coming from a handful of previous PWI studies utilizing ERPs, which have reported an apparent dissociation between modulations of language-sensitive ERP components and naming responses in the PWI task (Blackford et al. 2012; Roelofs et al. 2016; Rose et al. 2019; Anderson et al. 2022; Emmorey, Mott, et al. 2021). In one such study, Blackford et al. (2012) reported that the N400 component was attenuated, but picture-naming times were longer (i.e., the classic interference effect) when participants named a picture (e.g., SHORTS) preceded 80 ms earlier by a semantically related word (e.g., pants) compared to when the word was semantically unrelated to the picture (e.g., mouse). Blackford et al. (2012) also found faster naming and greater N400 amplitude attenuation for identity pairs (e.g., horse—HORSE) compared to unrelated pairs (e.g., chair—HORSE). The N400 is a frequently reported measure that is thought to reflect lexico-semantic processing (see Grainger and Holcomb 2009). It tends to be larger in amplitude when a target word (or picture) is not preceded by a supporting context and is usually attenuated when preceded by a semantically related context. The finding that the amplitude of the N400 was smaller for both pictures that were semantically related to or had the same name as the distractor word suggests that at least through the N400 epoch, a pattern of automatic priming and not interference dominates processing in the PWI task. Such N400 priming effects (i.e., smaller N400s for contextually related items) are the norm in studies of word and picture comprehension (see Kutas and Federmeier 2011 for a review of the N400 literature).

The same pattern of N400 and behavioral effects reported in Blackford et al. (2012) has been replicated in several subsequent semantic PWI studies using electrophysiological methods (Anderson et al. 2022; Rose et al. 2019; Emmorey et al. 2020; Piai et al. 2014; Roelofs et al. 2016; Wong et al. 2017). Using a PWI cross-language task, Roelofs et al. (2016) found a similar pattern of N400 attenuation and RT interference for semantic pairs and an even larger N400 attenuation but RT facilitation for identity pairs. In their study, pictures were named in one language, but words were presented in a second language.

Curiously, neither the Roelofs et al. (2016) cross-language study nor any of the other previous PWI studies using ERPs have reported clear evidence for an ERP effect that is consistent with the RT interference effect. Although infrequently reported, there have been cases where ERP negativities to language stimuli in priming tasks other than the PWI have resulted in a reversal of the typical N400 priming pattern. In other words, a larger negativity for targets with related primes (Emmorey, Holcomb, Midgley 2021). While it might be tempting to refer to such an ERP pattern as an interference effect, because interference and facilitation are defined behaviorally, we will take the more neutral stance of referring to such atypical ERP effects (larger negativities to related than unrelated pictures) as reversed priming. We will also refer to the more standard direction of priming (larger negativities to unrelated than related) as typical priming. Typical N400 priming is generally associated with response time (RT) facilitation from related prime words, while reversed N400 priming is more consistent with RT interference.

1 |. The Present Study

The goal of the present study was to investigate several potential sources of the apparent dissociation between modulations of the N400 amplitude and naming responses in the PWI task. We recorded naming latency and ERPs as monolingual English speakers performed a PWI task with word primes, followed 200 ms later by to-be-named line drawings of common objects. Written word primes were either the name of the depicted object (Identity condition), semantically related to the object (Semantic condition), or unrelated to the object (Unrelated control condition).

Following previous PWI studies (e.g., Blackford et al. 2012; Emmorey, Mott, et al. 2021; Roelofs et al. 2016), we predicted that picture naming RT and N400 amplitude would be reduced for pictures primed with the identical word name for the target picture (e.g., a picture of a COW preceded by the word cow) compared to the same pictures when preceded by unrelated control words (e.g., COW preceded by the word pan). On the other hand, following these same previous studies, we predicted that picture-naming latencies would be increased, but that N400 amplitude would be reduced for pictures primed by semantically related words (e.g., a picture of a DOG preceded by the word cat) compared to the same pictures when preceded by semantically unrelated control words (e.g., a picture of a DOG preceded by the word pan). As reviewed above, this divergence in the pattern of RT and ERP effects for semantically related pictures has been reported in several previous PWI studies and has been interpreted by most as ERPs being primarily sensitive to cross-domain lexico-semantic overlap between the word and picture, similar to N400 semantic priming effects reported in previous language and picture comprehension studies (e.g., Chwilla et al. 1995; McPherson and Holcomb 1999). Importantly, as mentioned above, in most comprehension studies, behavioral responses typically go in the same direction as the ERP effects, with faster RTs and/or greater accuracy being associated with smaller N400s (i.e., a pattern of facilitation/priming).

Here, we investigate several possibilities for prior failures to find ERP correlates of semantic interference (i.e., reversed priming) in the PWI task. One reason ERPs may not show clear evidence of interference effects is that time-locking to picture onset might result in greater timing variability of interference effects that in turn results in attenuated or washed-out ERP reversed priming effects. The results of two recent PWI studies, one using EEG (Anderson et al. 2022) and the other using MEG (Piai et al. 2014), are consistent with this possibility. Both studies reported the typical pattern of reduced N400/M400s for semantically related pictures (i.e., a typical priming effect), and neither the ERP data of Anderson et al. (2022) nor the phase-locked MEG data for Piai et al. (2014) showed evidence of reversed priming. Interestingly, both studies did report evidence of what we are characterizing here as a reversed priming effect, but only when the raw EEG data were analyzed using either a current source density (CSD) transformation (Anderson et al. 2022) or the non-phase-locked MEG activity (Piai et al. 2014). In both cases, a reversed priming effect was present but only at left anterior electrode/sensor sites. Since the methods used in both of these studies have greater spatial resolution than standard ERP averaging, it seems reasonable to speculate that reversed priming in the PWI task may not be easily visualized with standard ERP methodology.

There are at least two other possibilities for previous failures to find robust evidence of reversed ERP priming effects, which we explored in the current study. First, it could be that the methodological demands of EEG studies, specifically the requirement for artifact-free signals, obscure any direct evidence of interference-linked processes. In particular, any such effect that is relatively close to the point when the speech output process(es) becomes active could be missed. Indeed, many ERP language production studies terminate EEG processing just before the onset of the fastest oral naming responses to avoid the influence of this substantial source of EEG artifact. For example, in Blackford et al. (2012) response latencies ranged from as early as 600 ms to as long as 1200 ms, so ERP recording was cut off at 600 ms to avoid articulatory artifact in the average ERP wave forms. If reversed priming effects are somewhat delayed compared to typical priming, cutting off recording too soon might exclude any reversed priming EEG activity.

At the time of the Blackford et al. (2012) study, techniques for reliably removing speech artifact were just becoming available but were still not widely in use. Today, there are several well-established techniques for removing a variety of artifact sources (e.g., Riès et al. 2011, 2013; Winkler et al. 2011), allowing trials and EEG epochs beyond the point of articulation to be examined. Here, we implemented one such procedure, ICA with Multiple Artifact Rejection Algorithm (MARA—Winkler et al. 2011; Winkler et al. 2014), which allowed us to recover many trials that otherwise would have been removed from analysis due to articulatory artifacts. This approach allowed us to extend the epoch of ERP analysis from 600 ms to over 1 s postpicture onset.

Using this approach, we hypothesized that in addition to the typical N400 priming effects reported in previous studies, we would also be able to determine if later portions of the picture-locked ERP response produce a reversed priming effect. A study by Janssen et al. (2015) using a different picture naming paradigm reported a late (~750 ms) effect that they linked with late cognitive control processes. Also, the Piai et al. (2014) MEG study found an effect in the response-locked frequency domain MEG data that could reflect a reversed priming effect in the −400 to −200 ms epoch prior to the naming response. In typical ERP PWI paradigms, an important test of any post-N400 reverse priming effect and its relationship with behavioral interference would be finding such an effect in the Semantic but not the Identity condition, since the Identity condition should be exclusively associated with typical priming/facilitation. Crucially, we do not currently know what pattern of ERPs might be associated with any post-N400 priming effects.

A second possibility for previous failures to find ERP evidence of reversed priming in the PWI task is variability in the pattern of facilitation and interference across items and participants. In many studies, the size of the RT interference effect is quite small. For example, in the original PWI study by Lupker (1979), the semantic interference effect across three experiments ranged from 24 to 32 ms. Such small effects could occur because different semantically related word–picture pairs result in varying amounts of interference or, in some cases, might actually have resulted in facilitation (e.g., Alario et al. 2000). A recent proposal from Nozari and Hepner (2019) argues for just such a flexible criterion system where task goals and the state of the production system determine whether a competitive or a noncompetitive selection pattern of results arises.

Accordingly, in prior ERP studies, trial and participant variability might have resulted in a mixture of typical and reversed priming effects which, in the average ERP, obscured any evidence of reversed priming. Such a mixture might also have obscured evidence of stronger typical priming for some items. Here, we explored this possibility by using the pattern of RTs for each specific pair of related and unrelated word–picture combinations to sort single trial ERPs into separate facilitation and interference bins. Because each word and picture were presented twice, once as related and once as unrelated, if the naming RTs for a specific picture were faster for the related as compared to the matched unrelated condition, these two single trial ERPs were sorted into the facilitation bins (i.e., one in the related facilitation bin and the other in the matched unrelated facilitation bin). However, if the RT was slower for the related compared to the unrelated condition, the single trial ERPs from this picture were placed in the comparable interference bins. Evidence that variability in the direction of priming effects contributes to weak or absent ERP correlates of interference effects would be finding clear differences in the size and/or polarity of relatedness effects for the facilitation and interference bins.

Since ERP’s typical priming effects have usually been associated with larger negativities for unrelated compared to related word/picture pairs (e.g., Chwilla et al. 1995), it seems reasonable to predict that any ERP effects associated with behavioral interference would be of the reversed priming variety. Consequently, given previous ERP semantic PWI results, we predicted that evidence for reversed priming would show up in the N400 epoch for interference bins. Specifically, we predicted such effects could begin in the early phase of ERP priming (200–300 ms) but would be largest in the late phase of the N400 epoch (400–600 ms), where the bulk of lexico-semantic processing occurs (Grainger and Holcomb 2009). In this case, there should be a transition to a reversed N400 pattern for items sorted into the interference bins. Conversely, consistent with a dominant typical priming (facilitation) pattern, we predicted that the RT-sorted facilitation bins would produce larger early (200–400 ms) and late (400–600 ms) negativities for unrelated compared to related word-picture pairs (i.e., typical N400 priming effects). Finally, it is possible that all or some of the reversed priming/interference pattern would show up in a later epoch (700–1200 ms) that is closer in time to the naming response but which has not been previously examined in ERP studies using the PWI task.

2 |. Method

2.1 |. Participants

Participants were 29 native English speakers from the San Diego State University community (20 females; M age = 24 years, SD = 4.6). All participants were right-handed and had normal or corrected-to-normal vision and typical neurological profiles. One additional participant was run in the experiment but was excluded from the analysis due to high (nonspeech) artifact rejection rates even after ICA. Participants were given informed consent in accordance with San Diego State University IRB guidelines and compensated monetarily for their time.

2.2 |. Stimuli

Picture stimuli were 100 simple black on a white background line drawings selected from a variety of sources (e.g., Snodgrass and Vanderwart, 1980; internet searches) and previously used in another PWI study from our lab (Emmorey, Mott, et al. 2021). Pictures were presented centered on an LCD video monitor at a viewing distance of 150 cm and at horizontal and vertical visual angles between 2 and 3. These viewing parameters ensured that participants did not have to make eye movements to fully perceive each picture.

The Identity and Semantic sets each contained 50 unique pictures. Each picture was presented twice throughout the experiment. The 50 pictures in the Identity set were presented once in the related condition (e.g., the word bag followed by a picture of a BAG) and once after the presentation of an unrelated word chosen from other words included in the Identity set (e.g., the word pen followed by a picture of a BAG). The 50 other pictures in the Semantic set were presented once in the semantically related condition (e.g., the word truck followed by a picture of a CAR) and once after an unrelated word chosen from the other words included in the Semantic set (e.g., the word mouth followed by a picture of a CAR). In both conditions, lists were counterbalanced so that any target item was presented first in the related condition to half of the participants and first in the unrelated condition to the other half of the participants. Participants were not familiarized with the pictures prior to the experiment, and related and unrelated trials using the same items were separated by at least 50 intervening trials. None of the items in the semantic pairs or unrelated pairs (semantic or identity) had significant phonological, orthographic, or morphological overlap. The semantic pairs were tested for their semantic relatedness in 10 participants not included in the ERP study. The 50 semantic word–picture pairs were rated with an average semantic relatedness of 4.25 (range 3.3–4.9) on a scale of 1 (not related) to 5 (highly related).

For the Identity condition, the mean ZipF frequency (1–7 scale) for the English prime words was 4.36 (SD = 0.56); for the Semantic condition, the mean ZipF frequency for the English prime words was 4.43 (SD = 0.62) (Van Heuven et al. 2014).

2.3 |. Procedure

Each trial began with a fixation cross presented in the middle of the screen for 800 ms, which was followed by a 500-ms blank screen (see Figure 1 for an example trial). The blank screen was followed by a visually presented English word in upper-case Arial font in the center of the screen for 200 ms. A picture then replaced the English word and remained on the screen for 2000 ms. The picture was followed by an 800-ms blank screen and a final fixation cross, which remained on the screen until the participant pressed a button on a game pad resting in their lap. The next trial then began. Participants were instructed to name the picture presented, regardless of the word preceding it. Participants were asked to produce the name as quickly and accurately as possible, without hesitating. They were also instructed to otherwise remain still and not blink during the trial, but they could blink and rest in between trials while the final fixation cross was on the screen. Participants wore a lightweight headset with a microphone positioned directly in front of their mouth. Naming responses were digitized (16-bit resolution), time-locked to the onset of each picture, and stored as WAV files for later offline analysis.

FIGURE 1 |.

Sample trial in the semantically related condition.

2.4 |. EEG Recording

Participants sat in a comfortable chair in a darkened and sound-attenuated room as EEG was continually recorded from a 32-channel tin electrode cap (Electro-cap International Inc., Eaton, OH; see Figure 2 for electrode montage). The EEG signal was amplified by a SynAmpsRT amplifier (Neuroscan-Compumedics, Charlotte, NC), using a bandpass of DC to 200 Hz and sampled continuously at 500 Hz. Electrodes were placed under the left eye and on the outer canthus of the right eye to monitor for eye artifacts. The online recording reference electrode was placed on the left mastoid, and an electrode was placed on the right mastoid to monitor differential mastoid activity. After data collection, the raw EEG data were re-referenced to the average of the two mastoids. All electrode impedances were kept below 2.5 kΩ.

FIGURE 2 |.

Montage of electrodes used for EEG recording, including the four interpolated sites used in data analyses, formed by averaging each of the three electrodes connected by dashed lines.

2.5 |. Data Analysis

Reaction times (RTs) collected during the experiment were measured offline by a computer algorithm that detected the onset envelope of the naming response referenced to the onset of the picture stimulus. Accuracy was computed by counting as errors trials in which participants produced an incorrect or unexpected picture name or when participants hesitated or did not respond. These error trials and trials with RTs below 400 or above 2500 ms were excluded from the ERP and RT analyses (6.3% of the data). RT data were analyzed in R using linear mixed effects models (lme4). Error data were analyzed with logit mixed models as recommended by Jaeger (2008). In both cases, we first conducted omnibus analyses contrasting relatedness Type (Identity vs. Semantic) and prime-target Relatedness (related vs. unrelated pairs) in two factor models. Significant interactions of Relatedness and Type were followed up with analyses examining the main effect of Relatedness separately for the Identity and Semantic conditions.

2.6 |. EEG Analysis

The continuous raw EEG data were imported into EEGLab (Delorme et al. 2006) and filtered between 0.1 and 30 Hz and submitted to an ICA analysis. We used the MARA algorithm (Multiple Artifact Rejection Algorithm—Winkler et al. 2014) to automatically select artifactual components. Components identified by MARA with a probability of being artifactual greater than 0.8 were removed from each participant’s raw EEG data (between 3 and 10 components per participant) prior to ERP averaging. We compared ICAed ERPs with non-ICA ERPs over the first 600 ms postpicture onset to make sure that the ICA procedure did not fundamentally alter the expected early ERP response seen in previous studies. The ICA processed EEG was used to form averaged ERPs time-locked to a point 100-ms preprime word onset and continuing on for 1500 ms (1200 ms postpicture onset). Activity in the 100-ms preprime onset period (−300 to −200 ms prepicture onset) was used as a baseline. As with the RT analysis, trials with incorrect responses or with hesitations were removed from the analysis. Any trials with residual artifacts between −300 and 1200 ms were also excluded from the analyses (< 1% of trials).

In order to keep the number of factors manageable for analysis purposes while balancing the need to accurately capture the pattern of effects in the data, we reduced the number of analyzed scalp sites to four regions of interest (ROIs). To compute each ROI, we averaged EEG data from three adjacent electrodes in each of four quadrants across the scalp: Left frontal (LF), right frontal (RF), left posterior (LP), and right posterior (RP—see Figure 2 for the sites used to compute each ROI). We divided up the four ROIs into two variables: Laterality (left vs. right) and Anteriority (frontal vs. posterior—see Figure 2). In order to understand the time course of processing, separate ANOVAs were performed for the following time epochs: 200–400 ms and 400–600 ms, which are similar to the epochs used in previous PWI studies with similar stimuli (e.g., Emmorey, Mott, et al. 2021), as well as comprehension priming studies examining both prelexical (N250) and lexico-semantic (N400) sensitive regions of the ERP (see Grainger and Holcomb 2009). However, while these latencies were originally chosen to avoid EEG contamination by articulatory artifact, here we also include a later measurement window: 700–1200 ms to capture processing not typically explored in ERP PWI studies. This window was selected to correspond to the timeframe of the majority of the behavioral responses in the Semantic condition, which we operationalized as the mean, +/− 2 standard deviations of the naming RT rounded to the nearest 100 ms boundary in the Semantic condition (mean = 907 ms, SD = 130 ms). The resulting ANOVA design was a 2 relatedness Type (Identity vs. Semantic) ×2 prime-picture Relatedness (related vs. unrelated) ×2 Laterality (left vs. right) ×2 Anteriority (Frontal vs. Posterior) design. However, since the focus of the study is on the differential effects of relatedness, only effects involving the Relatedness and Type interaction were considered (i.e., Relatedness × Type and Relatedness × Type and scalp distribution factors) in omnibus ANOVAs. When such interactions were found, they were followed up with ANOVAs contrasting the Relatedness factor separately for Identity and Semantic conditions.

Finally, to explore the relationship between behavioral measures of facilitation and interference and ERP relatedness effects, we also used a novel approach that involved a more fine-grained analysis of the Semantic condition. We did this by separately averaging trials where there was facilitation for the semantically related condition (i.e., faster RTs for the related than matched unrelated item) and trials where there was evidence of interference (i.e., slower RTs for the related than matched unrelated item). The same measurement windows mentioned above were used for these analyses. This variable is referred to as RT-Type and involves comparison of facilitation bins versus interference bins. Because there were not sufficient interference effects in the Identity condition (fewer than 33% of trials), we did not include any fine-grained analyses for these data.

3 |. Results

3.1 |. Behavioral Results

The mean RTs and accuracy results are presented in Table 1. For RTs, there was a main effect of Relatedness (t(19) = 7.97, p < 0.0001) and an interaction between Relatedness and Type (t(29) = 10.98, p < 0.0001). Participants named pictures 99 ms faster when the prime word was the same as the picture name but were 14 ms slower to name pictures preceded by semantically related prime words compared to the same pictures preceded by an unrelated prime. Follow-up analyses indicated that the 99-ms facilitation effect was significant for the Identity condition (t(29) = 14.35, p < 0.0001), but that the smaller interference effect for the semantic condition was also significant (t(29) = −2.00, p = 0.046).

TABLE 1 |.

Mean response time (RT) and accuracy (percent correct) in each condition. Standard deviations are given in parentheses.

	Identity condition		Semantic condition
	Related prime	Unrelated prime	Related prime	Unrelated prime
RT (SD)	784 (134)	883 (127)	914 (123)	900 (136)
Error % (SD)	1.6 (7.1)	5.5 (6.9)	9.9 (5.8)	8.1 (6.9)

For accuracy, participants made 4% fewer naming errors for Identity pictures compared to unrelated pictures, but they made 1.8% more naming errors for pictures preceded by a semantically related prime word compared to the same pictures preceded by a semantically unrelated word. There was a main effect of Relatedness (z = 4.0, p = 0.002) and an interaction between Relatedness × Type (z = 5.99, p < 0.0001). Both the increased accuracy for Identity-related picture naming and the decreased accuracy for semantically related picture naming were statistically significant (Identity Relatedness effect: z = 5.73, p = 0.0001; Semantic Relatedness effect: z = −2.05, p = 0.04).

3.2 |. ERP Results

Plotted in Figures 3 and 4 are the grand mean ERPs from unrelated (solid line) and related (dashed line) trials in the Identity condition (Figure 3) and the Semantic condition (Figure 4). As can be seen, there is a large central-anterior negativity (N1) that peaks between 100 and 150 ms postpicture onset. This early exogenous component is followed by a central-posterior P2 peaking between 200 and 250 ms and then a smaller anterior negativity peaking around 300 ms. This later peak has a similar scalp distribution to both the N250 seen in previous word priming studies (see Grainger and Holcomb 2009) and the N300 reported in previous studies with picture stimuli (e.g., Sitnikova et al. 2006). The N250/N300 is followed by a large posterior P3-like positivity, which peaks around 500 ms. The ERPs return toward baseline following the P3. Importantly, the waves in both figures do not appear to contain any significant contamination from speech artifacts.

FIGURE 3 |.

Grand mean ERP waveforms from the Identity condition from four interpolated scalp sites, comparing responses to picture targets preceded by unrelated (solid black) versus related (dashed red) word primes. Note that the prime word and target picture onset, as well as the three measurement windows, are indicated on the time-course legend.

FIGURE 4 |.

Grand mean ERP waveforms for the Semantic condition from four interpolated scalp sites, comparing responses to picture targets preceded by unrelated (solid black) versus semantically related (dashed red) word primes. Note that the prime word and target picture onset, as well as the three measurement windows, are indicated on the time-course legend.

3.2.1 |. 200–400 ms

In this first epoch, there was a main effect of relatedness (F(1, 28) = 6.54, p = 0.016) and interactions between Type, Relatedness, and Laterality (F(1, 28) = 24.46, p < 0.001). As can be seen in the voltage maps for this epoch (Figure 5), both relatedness types produced larger negativities for unrelated than related pictures, although the effect was larger for the Identity condition, especially at right hemisphere sites. Separate follow-up analyses for the two types of priming revealed significant Relatedness by Laterality interactions for both types (Identity Relatedness × Laterality: F(1, 28) = 27.67, p < 0.001; Semantic Relatedness × Laterality: F(1, 28) = 4.32, p = 0.047).

FIGURE 5 |.

Voltage maps for mean amplitude in the three measurement epochs for the Identity (A) and Semantic (B) priming effect (unrelated–related). Blue areas are due to greater negativity in the unrelated condition (or greater positivity in the related), which is associated with a typical priming effect, especially in the N400 timeframe (200–600 ms). Red areas are due to greater positivity in the unrelated (or greater negativity in the related) condition. Here, we refer to such effects as reversed priming.

3.2.2 |. 400–600 ms

Like the previous epoch, there was an interaction between Relatedness, Type, and Laterality (F(1, 28) = 4.90, p = 0.035), indicating that the N400 effect (unrelated vs. related) was larger for the Identity condition, especially at right hemisphere sites (see Figures 3, 4 and Figure 5b middle). Separate follow-up analyses confirmed that the Relatedness effect by Laterality effect was significant for the Identity condition (Relatedness × Laterality: F(1, 28) = 6.10, p = 0.02). For the Semantic pairs, there was a Relatedness × Anteriority interaction (F(1, 28) = 6.02, p = 0.019).

3.2.3 |. 700–1200 ms

In the last epoch, there was not a main effect of Relatedness, but there was an interaction between Relatedness, Type, and Anteriority (F(1,28) = 5.09, p = 0.032). Separate follow-up analyses for the Identity and Semantic types revealed that only the Semantic condition showed a significant Relatedness effect (main effect: F(1,28) = 9.75, p = 0.004) with related pictures producing greater negativity than unrelated (i.e., reversed priming). The Identity condition showed a significant gradient of priming with greater typical priming, but primarily at right hemisphere sites (Identity: Relatedness × Laterality interaction, F(1,28) = 22.68, p < 0.001).

3.2.4 |. Interim Summary

There were clear effects of the Relatedness manipulation for pictures in the Identity and Semantic conditions across all three ERP time epochs, as well as in the behavioral naming responses. In the case of the Identity condition, the pattern was consistent; across the three ERP epochs and naming latency/accuracy, there was a pattern of typical ERP priming and behavioral facilitation with more negative-going ERPs and slower/less accurate responses for target pictures following unrelated prime words compared to identity prime words. A similar but somewhat weaker pattern of ERP effects was seen in the Semantic condition across the first two ERP epochs. However, in the final ERP epoch (700–1200 ms) and for response accuracy, the semantic manipulation produced a reversed pattern. There were more errors and greater ERP negativities for target pictures following semantically related prime words compared to unrelated words (i.e., reversed priming and behavioral interference).

A close examination of item-level RTs revealed that for the Semantic condition, there were roughly equal numbers of pictures that showed a facilitation pattern (46% of trials on average) and an interference pattern (54%). On the other hand, in the Identity condition, there were very few trials (fewer than one-third on average) that produced an interference pattern.

3.3 |. Analysis of Semantic Effects as a Function of RT Facilitation/Interference

Because of the nearly equal proportions of trials in the Semantic condition, where participants were either faster or slower on related than unrelated items, we sorted single-trial ERPs from the Semantic pairs on a participant-by-participant basis using these RT criteria. Of note, there were substantial individual differences in which specific items showed these two kinds of effects (see Table S1 in Supporting Information, which report the distribution of specific semantic items sorted into facilitation and interference bins by participant). Perhaps surprisingly, no item was sorted into either bin type for all participants. The semantic item that ended up most often in the RT sorted interference bins (related—GUN) also ended up being included in four participants’ facilitation bins. Likewise, the most frequently sorted item going into the facilitation bins (related—CAT) was also sorted into eight participants’ interference bins.

ERPs formed using this RT constrained sorting scheme are depicted in Figures 6 and 7 with voltage maps for the three analysis epochs shown in Figure 8. We analyzed the RT-sorted ERP data using the same epochs and ANOVA design used for the Identity-Semantic analyses; only now, we refer to the facilitation versus interference contrast as the RT-Type variable. As mentioned above, there were relatively few trials in the Identity condition where RTs were slower on related trials, so we did not attempt to reaverage ERPs from this condition.

FIGURE 6 |.

Grand mean ERP waveforms for the Semantic condition trials with facilitated RTs (trials with slower RTs to unrelated than related), contrasting pictures following related and unrelated word primes. Note that the three measurement windows are indicated on the time-course legend.

FIGURE 7 |.

Grand mean ERP waveforms for the Semantic condition trials with inference RTs (slower RTs to related than unrelated), contrasting pictures following related and unrelated word primes. Note that the three measurement windows are indicated on the time-course legend.

FIGURE 8 |.

Voltage maps for three epochs for the ERP data sorted by RT. RT sorted Facilitation (A) and interference (B) priming effects (unrelated–related) are shown in the Semantic Condition.

3.3.1 |. 200–400 ms

In this epoch, there was not a main effect of Relatedness, but there was an interaction between Relatedness and RT-Type (F(1,28) = 10.39, p = 0.003). As can be seen in Figure 8 and verified in follow-up analyses, there was a clear typical priming effect (unrelated more negative than related) in the RT facilitation condition (main effect of Relatedness: F(1,28) = 76.98, p = 0.013; Figure 8A left) and virtually no evidence of a relatedness effect in the RT interference condition (all p > 0.34; Figure 8B left).

3.3.2 |. 400–600 ms

In this second epoch, there was again a significant Relatedness × RT-Type interaction (F(1,28) = 13.29, p = 0.001). Follow-up analyses indicated that the facilitation condition showed a large, widely distributed typical N400 response (main effect of Relatedness: F(1,28) = 8.93, p = 0.006; see Figure 8A middle) while the interference condition showed a reversed priming effect with unrelated pictures showing more positive-going ERPs than related pictures (main effect of Relatedness: F(1, 29) = 10.5, p = 0.004), especially at the posterior sites (Relatedness × Anteriority interaction: F(1, 28) = 10.52, p = 0.003; see Figure 8B middle).

3.3.3 |. 700–1200 ms

There was a significant interaction between Relatedness, RT-Type, and Anteriority (F(1, 28) = 4.69, p = 0.039) with clear evidence of reversed priming (more positive ERPs for unrelated than related pictures) in both the facilitation and interference conditions, although with somewhat different scalp distributions. Follow-up analyses revealed a significant Relatedness by Laterality interaction in the interference condition (F(1, 28) = 4.83, p = 0.037), indicating a larger reversed priming effect over the left hemisphere (see Figure 8b right). For facilitation trials, there was a significant Relatedness by Anteriority interaction (F(1, 28) = 8.38, p = 0.007) with greater reversed priming over posterior sites (see Figure 8A right) and a small typical priming effect over anterior sites.

4 |. Discussion

In a picture–word interference paradigm, monolingual English speakers saw printed word primes immediately followed by simple line drawings that were to be named. The word primes were either identical to the name of the following picture, semantically related to the picture name, or semantically unrelated to the following picture. Paralleling results from previous PWI studies, we found significantly faster and more accurate naming responses when the picture was preceded by the word naming the picture compared to the same picture when it was paired with a completely unrelated word—a pattern consistent with a robust identity facilitation effect. For semantically related pairs, we also found evidence for interference in both the RT and accuracy data. Participants tended to be slower and less accurate in naming semantically related than unrelated pictures. Thus, the behavioral results confirm the oft-reported semantic interference effect in the PWI task. However, we note that the RTs in this study were longer than a previous study using similar timing (870 ms here and 770 ms in Anderson et al. 2022). This difference might have been because we used a much longer stimulus duration in the current study (2000 ms vs. 300 ms in Anderson et al. 2022) or alternatively because we used line drawings while Anderson et al. (2022) used color pictures of real objects, which are less abstract.

For the Identity condition, the ERP results indicated greater amplitude negativities to unrelated compared to same-name target pictures as early as 200–300 ms after picture onset. This early epoch has been associated with the onset of the N400 but also has been shown to reflect differences in at least two other ERP components (the N250 and N300), especially when the prime and target stimuli are presented in close temporal proximity (200 ms in the present case). The N250 has been shown to be sensitive to the mapping of prelexical orthographic features onto word-level representations (see Grainger and Holcomb 2009), and the N300 has been argued to reflect a similar process in the case of object stimuli (i.e., object features involved in object recognition, e.g., Eddy et al. 2006; Rugg 1985). The more frontal distribution of this early ERP effect is also consistent with previous studies using picture targets and might also reflect, in part, the overlap in prelexical imagistic features activated by the prime word and target picture (e.g., McPherson and Holcomb 1999). Accordingly, the lower the feature overlap, the less of a typical priming effect and the larger the negativity, as in the case with unrelated word-picture pairs. Here, we interpret this early ERP response as reflecting a process whereby prelexical orthographic and perhaps imagistic features activated by the prime word are mapped onto a combination of word and object-level representations activated by the target picture. Consistent with this interpretation, the 200–300 ms effect was weaker for the Semantic condition (see Figure 5), presumably because target pictures that are semantically related to the prior prime words share a smaller subset of prelexical orthographic and/or imagistic features (e.g., “round shape” for apple <picture of LEMON>; “four legs” for dog <picture of CAT>) compared to the nearly complete overlap of features in the case of prime words that name the picture (deer <picture of DEER>).

In the next epoch (400–600 ms), the ERP effects shifted to a more central-posterior distribution, a pattern typically associated with the peak and declining phase of the N400 component. This epoch has been shown in numerous previous studies to reflect an interactive process whereby whole-word as well as object representations are mapped onto concepts (see Grainger and Holcomb 2009). This priming effect (greater amplitude negativity for unrelated than related primes) was most apparent for the Identity condition and was significantly smaller for the Semantic condition where it could only be seen at more anterior sites (see Figures 3–5). We interpret this pattern of N400 effects as reflecting interactive conceptual-semantic priming of the pictured object and object name in both the Identity and Semantic conditions. As in several previous ERP studies using the PWI paradigm we could find no evidence of a reversed priming effect in either of the first two epochs in the overall average ERP data.

Finally, in the temporal window following the N400 (700–1200 ms), the typical ERP priming effect seen in the previous epochs had largely dissipated in both the Identity and Semantic conditions (see Figure 5A,B right side). However, of particular interest is what might be a reversed priming effect in the Semantic condition, most notably at left posterior sites. We suggested in the Introduction that this interpretation rests on not finding a similar pattern in the Identity condition since no interference should occur in that case. Consistent with this prediction, the Identity condition showed small but significant evidence of typical ERP priming at posterior sites (see the blue area in Figure 5A right side) and no evidence of a reversed effect. The reversed priming effect in the Semantic condition is consistent with the hypothesis that ERPs are indeed sensitive to a later post-N400 process(es) that differs for identity and semantic pairs. We suggest that previous PWI studies using ERPs may have failed to find evidence of interference/reversed priming effects because they did not examine later epochs in order to avoid speech artifacts. In the current study, we used rigorous artifact correction techniques to help remove such artifacts and found clear evidence of an ERP correlate of the behavioral interference effect.

Such a post-N400 process could reflect competition that occurs after earlier lexico-semantic processing, as suggested in multiple accounts of behavioral naming interference effects (e.g., Dhooge and Hartsuiker 2010; Finkbeiner and Caramazza 2006; Pinet and Nozari 2023; Roelofs 2021). Although previous ERP studies using the PWI task have not reported a similar effect, one study by Janssen et al. (2015) using a different picture naming task reported a post-N400 positivity that was tightly correlated with behavioral measures of competition. They suggested such an effect could reflect postselection cognitive control factors responsible for resolving between competing representations. Such an explanation could also account for the pattern of effects we are reporting here.

Alternatively, the late positivity could underlie differential activity in a self-monitoring process (see Gauvin and Hartsuiker 2020; Roelofs 2020). The idea is that because speakers frequently make speech errors, a system to catch and correct errors is continuously active during language production. The post-N400 effect found in the current study could reflect such a process, which would presumably intervene more often with semantic pairs when the name of the prime word might be, at least initially, selected for production.

4.1 |. Response-Contingent ERP Averaging

Examination of Figure 5B (center) suggests that there was not a typical priming effect at posterior sites in the late phase of the N400 (400–600 ms) in the Semantic condition, even though there was a small effect at anterior sites (as revealed by the significant Relatedness × Anteriority interaction in this epoch). This distribution is curious because the N400 effect typically has a more posterior distribution (Kutas and Federmeier 2011). One possible reason why more robust evidence for late N400 amplitude modulation as well as RT effects was not observed in the above analyses is that such effects were masked or attenuated by a mixture of typical and reversed priming across items/trials. In other words, because our ERP measures were made on averages across items and participants, it is possible that variability in the direction of priming attenuated any clear evidence of effects in either direction. Because all participants were presented with each prime word and picture twice (once as a related pair and once as an unrelated pair), we were able to calculate for each participant the direction of the RT effect for every picture. Consistent with the possibility of a mixture of facilitation and interference in the Semantic condition, the direction of the difference in RTs was approximately equally distributed between a pattern of facilitation (related faster than unrelated) and a pattern of interference (unrelated faster than related). This was not the case for the Identity condition, where there were many more facilitation pairs. Using these RT data, we reaveraged the Semantic condition ERPs for each participant, forming separate related and unrelated facilitation and interference bins.

For facilitation bins, there were robust typical priming effects (unrelated more negative than related) in both the 200–300 ms and 300–600 ms windows (see Figures 6 and 8A). However, for the interference bins, there were null priming effects in the early epoch 200–400 ms window but clear reversed priming effects in the later 400–600 ms window (see Figures 7 and 8B). Our tentative interpretation of the mixed pattern of priming we observed in the ERP results for the first two epochs (200–300 and 300–600 ms) is that interactions between prime word and picture representations reflect the sum of both typical priming (facilitation) and reversed priming (interference) across multiple levels of processing. This interpretation is consistent with previous characterizations of priming effects in comprehension studies with words and pictures (Chwilla et al. 1995; Eddy et al. 2006; Holcomb 1993; McPherson and Holcomb 1999). For example, Grainger and Holcomb (2009) argued that ERPs to words and pictures are sensitive to manipulations at the level of prelexical/image form as well as whole-word/object and conceptual level representations. In the current study, it seems likely that at least some of the observed ERP effects in the first two epochs of the facilitation bins were due to processing across these levels.

Note that earlier studies using other electrophysiological measures (CSD and MEG; Anderson et al. 2022; Piai et al. 2014) have shown a similar pattern to the interference bins reported here. In addition, the computational modeling work by Roelofs and colleagues is consistent with the mixed pattern of ERP responses we are reporting here (Piai et al. 2014; Roelofs et al. 2016). In fact, the latency of the Piai et al. (2014) reversed priming result (350–650 ms) is almost identical to the epoch where we find a reversed semantic priming effect (400–600 ms), albeit with different electrophysiological measures (frequency domain MEG data in the Piai et al. study). This correspondence of effects is consistent with the hypothesis raised in the Introduction that ERPs might not be as directly sensitive to a reversed semantic effect and either more spatially sensitive techniques (e.g., MEG) or require data processing methods like the sorting of the specific trials that actually produce the strongest evidence of interference in order to show the effect.

Interestingly, the final ERP epoch (700–1200 ms) continued to show clear reversed priming effects for both facilitation and interference bins, although with a somewhat different scalp distribution (see Figure 8; left side). One possibility for why facilitation trials could show reversed priming in this latency range is that even these trials (or some of these trials) might be producing residual competition at the level of what is likely a serial output system—we can only say one word at a time. Bimodal bilinguals who are fluent in both a spoken and signed language are the exception in this case, as a sign and a word can be produced at the same time (Emmorey et al. 2008), and thus late interference effects would not be predicted in a cross-language PWI paradigm. If bimodal bilinguals show the late reversed priming effect, this might indicate that this effect is more likely due to the alternative self-monitoring explanation mentioned above. We are currently testing these two possibilities with the data from (Emmorey, Holcomb, and Midgley 2021), which used the same stimuli and presentation parameters as used here.

Future ERP priming studies that use response-contingent data to conditionally assign single trial EEG data to various conditions will be necessary to further characterize the pattern of results observed here, as well as other reported priming effects. For example, if we had had more trials, it might have been possible to create response-contingent ERP bins with finer granularity. Item pairs with the clearest divergence of RTs might then have shown the clearest evidence of typical and reversed ERP priming. Another promising future direction would be to examine ERP data in bilingual participants using the RT sorting approach in order to determine whether cross-language priming effects parallel those seen here with within-language items. Yet another promising direction would be to use this approach in other production paradigms like the blocked-cyclic picture naming task (e.g., Lin et al. 2022) or other types of picture naming tasks where competitive processes are likely (e.g., Pinet and Nozari 2023). Finally, another promising direction we are currently pursuing derives from models that predict that the nature of the word/picture semantic relationship might have an impact on the balance of typical and reversed priming. Several prior studies have suggested that associated word/picture pairs (honey—BEE) are more likely to produce a facilitation dominant pattern compared to the pattern generated by categorically related pairs (e.g., dog—CAT) which constituted the majority of items in the current experiment.

Overall, the results of this study point to a promising new way of analyzing single-trial EEG data based on behavior-contingent oral responses to more accurately reveal the time course of linguistic and cognitive processes. This approach would seem to hold promise in language production studies beyond the PWI task.

Supplementary Material

Supporting Information

Additional supporting information can be found online in the Supporting Information section. Table S1: This table shows which pictures were included in the facilitation (Positive 1) and interference (Minus 1) bins for each participant. To get a +1 score, the picture naming latency had to be faster in the related condition than in the unrelated condition for that participant. A Minus 1 score required a faster unrelated than related RT. Missing values (blanks) were either due to equal naming times in the two conditions or naming errors. These trials were not included in the reaveraged data.

Data Availability Statement

The data that support the findings of this study are openly available in ERP_PWI_NCL at https://openneuro.org/.