Does Discourse Congruence Influence Spoken Language Comprehension before Lexical Association? Evidence from Event-Related Potentials

Abstract

The goal of this study was to examine how lexical association and discourse congruence affect the time course of processing incoming words in spoken discourse. In an ERP norming study, we presented prime-target pairs in the absence of a sentence context to obtain a baseline measure of lexical priming. We observed a typical N400 effect when participants heard critical associated and unassociated target words in word pairs. In a subsequent experiment, we presented the same word pairs in spoken discourse contexts. Target words were always consistent with the local sentence context, but were congruent or not with the global discourse (e.g., “Luckily Ben had picked up some salt and pepper/basil”, preceded by a context in which Ben was preparing marinara sauce (congruent) or dealing with an icy walkway (incongruent). ERP effects of global discourse congruence preceded those of local lexical association, suggesting an early influence of the global discourse representation on lexical processing, even in locally congruent contexts. Furthermore, effects of lexical association occurred earlier in the congruent than incongruent condition. These results differ from those that have been obtained in studies of reading, suggesting that the effects may be unique to spoken word recognition.

Spoken language comprehension depends critically on a listener’s ability to use context in the rapid creation of a cohesive and coherent discourse representation. Processing of incoming words in spoken discourses can be facilitated by the meaning of words in the immediately preceding local context, and by the meaning of the global discourse context. Results from previous studies suggest a rapid and robust effect of global discourse representations on local lexical-semantic processing. However, the results from studies of local lexical association on the incremental processing of incoming words in the immediate context are less clear-cut. In the current study, we used event-related potentials (ERPs) to examine the influence and timing of lexical association and discourse congruence on the processing of incoming words during spoken language comprehension.

In the following sections, we introduce different models of discourse comprehension and discuss evidence that the discourse context has a rapid influence on lexical-semantic processing. We then review studies that have examined how meaning relations among individual words in the immediately preceding context influence processing of incoming words in discourse. Finally, we introduce the current study.

Models of Discourse Comprehension

Discourse comprehension involves constructing a global, message-level representation incrementally; that is, each incoming word in discourse is understood in light of information from the immediately preceding context (e.g., word meanings, syntactic structure) and information from preceding message-level content and world knowledge (e.g., the discourse context).

Traditionally, models of discourse comprehension have not made predictions about the time-course by which local and global sources of information become available as words are processed in a discourse context. Instead, most models have focused on the nature of the representations that influence how a discourse model is constructed (e.g., Sanford & Garrod, 1998; McKoon & Ratcliff, 1992). These models predict that incremental processing of incoming words in the discourse is driven by an interaction between semantic information that is activated by incoming words and knowledge that is relevant to the situation described in a text (e.g., knowledge about ordering food, asking for the bill, etc. is relevant to a story about a restaurant). Sanford and colleagues have shown that comprehenders sometimes process discourse at a relatively shallow level, especially when common scenarios are described. For example, readers often fail to detect anomalies such as “victim” in the following text: “Child abuse cases are being reported much more frequently these days. In a recent trial, a 10-year sentence was given to the victim, but this was subsequently appealed” (Sanford, Leuthold, Bohan & Sanford, 2010). In such cases, meaning relations among individual words in the context appear to dominate the resulting representation (e.g., Sanford & Garrod, 1998; Sanford et al., 2010).

Other models make predictions about how local and global sources of information are activated and integrated, and these models differ in their assumptions about the role of top-down processing. According to the memory-based text processing framework, activation of concepts from LTM is relatively automatic and operates according to general principles of memory (e.g., Kintsch, 1988; Albrecht & O’Brien, 1993; Myers & O’Brien, 1998; McKoon & Ratcliff, 1998). Incoming words and combinations of words activate related information in memory by means of recognition priming processes, and the activated concepts are then integrated into a representation that is coherent at the local level (i.e., the level at which readers and listeners understand that entities and events in adjacent clauses are related) and at the global level (i.e., the level at which readers understand the situation that is described in a text or the spoken input). Integration is primarily an automatic process of convergence of ideas in the text. This type of model makes no predictions about the time course by which the stored meaning of individual words and the meaning of the overall discourse representation influence incoming words in the local context.

The constructionist theory proposes a similar account of the activation process, but views integration as a “search after meaning” (see Graesser, Singer, & Trabasso, 1994), involving the controlled supervision of coherence at both the local and global levels. This model predicts that the time course by which local and global sources of information are processed will be influenced by how effortful it is to activate and integrate the different sources of information. For example, a deliberate search of information from the preceding context would lead to the later integration of this information into the discourse model than the integration of information that has been primed from memory.

In contrast to two-stage models, van Berkum and colleagues have recently proposed a one stage model of discourse comprehension, the Multiple-cause Intensified Retrieval (MIR) hypothesis (Van Berkum, 2009a). According to the MIR hypothesis, a range of factors can facilitate retrieval of a given word in the discourse context, and the context in which a word appears is a “mixed-bag” of factors that may contribute to retrieving stored knowledge about a given word. This includes the presence of semantically related words, scenario-based world knowledge, and the current representation of the discourse (2009a). In addition, “non-linguistic” information such as knowledge about the speaker is also proposed to form part of the context in which words are interpreted. This account is generally consistent with models of language comprehension that have a parallel architecture and assume that word-by-word processing in context is influenced both by the constraints of lexico-semantic and syntactic information (Tanenhaus, Spivey-Knowlton, Eberhard, & Sedivy, 1995; MacDonald, Pearlmutter, & Seidenberg, 1994; Altmann & Steedman, 1998; Jackendoff, 2002; 2007; Kuperberg, 2007). These models predict that the time course by which semantic and syntactic sources of information influence incremental processing of incoming words depends on the relative strength of their constraints. Van Berkum’s MIR model expands on these types of models by suggesting that discourse-related information and other contextual cues can take precedence over lexical semantic processing of incoming words in the discourse context.

More recently, accounts of sentence and discourse comprehension have considered the possibility that prediction of upcoming lexical items as a function of the meaning of the preceding context plays a major role in discourse comprehension; several studies have yielded evidence that is consistent with these accounts (Federmeier, 2007; Delong, Urbach & Kutas, 2005; van Berkum, Brown, Zwitserlood, Kooijman, & Hagoort, 2005). Further, Otten and van Berkum (2008) have shown that prediction of upcoming words in a sentence as a function of context cannot be attributed to effects of priming alone. However, their study used highly constrained scenario-like passages. In the current experiment, the passages were coherent, but they did not describe well-known scenarios; moreover, predictability of the critical word from the context was limited. This made it possible to assess the potential interplay of local and global sources of information as a discourse model is constructed in real-time.

Effects of Discourse Context

Recent evidence from both behavioral and ERP studies is most consistent with one stage models of discourse comprehension. These studies have shown that discourse context rapidly influences the processing of incoming words (e.g. van Berkum, Hagoort & Brown, 1999; Federmeier & Kutas, 1999; van Berkum, Brown, Hagoort & Zwitserlood, 2003; Camblin et al., 2007; Kuperberg, Paczynski & Ditman, in press). Specifically, ERP studies of discourse-context effects on lexical-semantic processing have found a discourse-dependent N400 effect (e.g. van Berkum et al., 1999; 2003; Federmeier & Kutas, 1999). The N400 effect is sensitive to semantic aspects of linguistic input, and is typically maximal over centro-parietal electrode sites. Its amplitude is modulated by manipulations of relatedness in word priming paradigms (e.g. Bentin, Kutas & Hillyard, 1985; Holcomb, 1988; Kutas & Hillyard, 1989), by manipulations of congruency, cloze probability, and expectancy in sentence contexts (e.g. Kutas & Hillyard, 1984; van Petten, 1993), and by manipulations of congruency in discourse contexts (e.g. Federmeier & Kutas, 1999; van Berkum et al., 1999; for a review, see Swaab, Ledoux, Camblin & Boudewyn, in press, see also van Berkum 2009a for an overview of results that show pragmatic and non-linguistic manipulations can affect the amplitude of the N400). Finally, recent studies using highly constraining contexts suggest that the amplitude of the N400 is modulated by semantic predictions that are evident to words preceding a critical word (Federmeier, 2007; Delong et al., 2005; van Berkum et al., 2005). In sum, some evidence suggests that the amplitude of the N400 is modulated as a function of semantic prediction, and all results of previous work show that the amplitude of the N400 is reduced when the meaning of a word can be easily retrieved and integrated into the preceding context.

In a series of studies, van Berkum and colleagues showed that congruency with the discourse context immediately affects lexical-semantic processing of critical words, even when these words are consistent with the local sentence context (van Berkum et al., 1999; van Berkum et al., 2003). The discourse-dependent N400 effects did not differ in time-course, distribution or morphology from the canonical N400 effect. Van Berkum and colleagues have suggested that this indicates that word processing as reflected by the N400 is immediately sensitive to message-level context. Moreover, it has been shown that the discourse representation may facilitate local lexical-semantic processing independently from the effect of meaning relations between individual words in the preceding discourse context. Otten and van Berkum (2008) manipulated the expectancy of critical words as a function of constraints in the preceding discourse context. As illustrated in the following example from their study (translated from Dutch), in the “Predictive Context”, a specific word (“tip”) was highly expected given the context of the preceding discourse, but not in the “Prime Control Context”. This word was replaced by an anomalous word in both conditions (in bold in the example):

Predictive Context

The woman was very satisfied with the waiter’s service. So she gave him a maniac on top of the bill to show her appreciation.

Prime control context

The woman was not very satisfied with the waiter’s service. So she gave him a maniac on top of the bill to elevate his mood.

In the example above, the word “tip” was highly expected (but never presented) in the predictive context but not in the prime control context. Importantly, as can be seen in the example, the critical anomalous words were preceded by the same content words in both context conditions. Comparing the critical anomalous words in the predictive and prime control condition yielded an ERP effect with a frontal distribution, which is not typical for the N400. This latter finding suggests that the meaning of individual un-integrated words in a discourse context affects processing of upcoming words in sentences, but that this effect may be independent from the effects of the global discourse.

Other findings suggest that the effects of discourse context can rapidly override semantic information that is typically activated by individual words outside of a meaningful context. Nieuwland and van Berkum (2006) measured ERPs to critical words that violated the animacy constraints of a directly preceding word in the local context of a sentence, as in: “the peanut was falling in love”. The discourse context was manipulated such that the violation of animacy would, or would not, be expected given the context. For example, the peanut was animated in a “cartoon-like” manner such that it was plausible for the peanut to be happy, dancing and in love. The authors found a decrease in the amplitude of the N400 to words that violated the animacy constraints but were story-consistent relative to words that were consistent with the animacy constraints, but story-inconsistent (e.g., salted).

Overall, the results of these studies suggest an immediate impact of the discourse representation on local lexical-semantic processing. However, there is also evidence that comprehenders sometimes build a less precise representation of the gist of discourse via the activation of related semantic or world knowledge stored in long-term memory (e.g. Barton & Sanford, 1993). Some studies have shown that this can lead to a failure to detect semantic anomalies in a text (“semantic illusions”; see Ferreira, Bailey & Ferraro, 2002 for a review), especially when the anomalous word is strongly associated with the discourse scenario (e.g., “The authorities had to decide where to bury the survivors” after a story about a plane crash), or when it is semantically related to the intended word (e.g., “How many animals of each kind did Moses put on the ark?”, in which Moses is semantically related to Noah) (Barton & Sanford, 1993; Nieuwland and van Berkum, 2005).

Thus, stored semantic information that is activated by individual words along with discourse-level information may have separable influences on the processing of incoming words. However, an important unresolved question concerns when these different sources of information come on-line during discourse comprehension. This question has been addressed in studies that have investigated the influence of lexical association on the processing of words in sentence or discourse contexts; we discuss these studies in the next section.

Effects of lexical association

Lexical associations have a robust influence on word processing across a wide variety of tasks and methodologies. Lexical-semantic processing is typically facilitated when words are preceded by primes that are associated in meaning (e.g. salt-pepper). This facilitative effect of associative relations is robust when words are presented in pairs or in lists (e.g., Andruski, Blumstein, & Burton, 1994; Bölte & Coenen, 2002; Marslen-Wilson & Zwitserlood, 1989; Swinney, Onifer, Prather, & Hirshkowitz, 1979; Williams, 1988). In ERP studies, it results in modulation of the N400 amplitude, i.e. a reduced N400 is found to critical words that are preceded by an associated word relative to an unassociated one (e.g. Domalski, Smith, & Halgren, 1991; Holcomb, Anderson, & Grainger, 2005; Joyce et al., 1999; Rugg et al., 1993).

The impact of associative relations between individual words in sentence and discourse contexts is less consistent. When associated words are embedded in sentences, effects of association are not always observed: Effects of lexical association are not found when the critical words are presented in different clauses (Carroll & Slowiacczek, 1986), when a preceding associated word is in an unfocused clause (Morris & Folk, 1998), or when the critical word is inconsistent with the context (e.g., “The gory details of what he had done convinced everyone that he deserved life in death”), in which death is lexically associated with life but incongruent with the meaning of the sentence (Van Petten, 1993; Van Petten, Weckerly, McIsaac & Kutas, 1997). But other studies have found effects of lexical association when critical words are at least moderately consistent with the meaning of a sentence (Camblin et al., 2007; Sereno & Rayner, 1992). Furthermore, a number of recent ERP studies have shown that associative relations between words may directly influence thematic role assignment, even in syntactically unambiguous sentences (Kuperberg, Sitnikova, Caplan, & Holcomb, 2003; Kolk, Chwilla, van Herten & Oor, 2003; Kim & Osterhout, 2005; Nakano, Saron & Swaab, 2010). The results of these latter studies suggest that lexical-semantic associations between individual words in a local context may indeed have a separable and important influence on processing of words in sentences (Kuperberg, 2007).

The influence of lexical-semantic associations on word processing in spoken rather than written contexts has thus far only been examined in studies using cross-modal presentation in which auditorily presented sentence or discourse contexts are followed by visually presented target words (e.g. Hess, Foss & Caroll, 1995). These studies, like those using visual-visual presentation, have produced varying results. Williams (1988) and Tabossi (1988a) found effects of lexical association only when the sentence context and lexical association converged on the same conceptual representation -- even though reliable effects of lexical association were found when the same critical words were preceded by lexical associates in random word lists. For example, Tabossi (1988) found reliable cross-modal priming when a visual target (“fat”) was consistent with the meaning of a preceding sentence (“To follow her diet, the woman eliminated the use of butter”), but found no priming when the visual target was inconsistent with the sentence meaning (e.g., “To soften it, the woman heated a piece of butter”). A second factor that influences the effects of lexical association on critical words in spoken sentences is the saliency of the prime; facilitation is found when associated words are presented at the end of sentences, at clause boundaries, or have been accented prosodically (Norris et al, 2006).

The cross-modal paradigm has also been used to study the effects of lexical association in discourse contexts. Hess and colleagues (1995), for example, asked participants to name visually presented target words following spoken discourse contexts. Participants were faster at naming the target words only when the targets were consistent with the overall context, regardless of whether or not they were preceded by an associated word (Hess et al., 1995). In contrast, associations have facilitated processing for related critical words in discourse contexts when the task was lexical decision rather than naming (Blutner & Sommer, 1988; Swinney, 1979, Experiment 1). The discrepancy in findings may be attributable, in part, to differences in the tasks. Lexical decision is slower than naming, making it more vulnerable to strategic processing. Additionally, the cross-modal presentation itself may, in some ways, have interfered with normal spoken language comprehension in these studies. In sum, the results of studies thus far do not unequivocally support an independent role of lexical association on real-time processing of words in sentence and discourse contexts.

The Current Study

The goal of the current study was to use ERPs to determine whether listeners benefit from local meaning relations between words in spoken discourse contexts, and whether potential effects of local association precede or follow effects of discourse congruence. Although several studies have shown that discourse-level information has a strong and rapid influence on word processing (van Berkum et al., 1999; Federmeier & Kutas, 1999; van Berkum et al., 2003; Camblin et al., 2007), few studies have directly examined the relative contributions of discourse congruency and lexical association. In addition, most previous studies have focused on reading comprehension. Results may differ between reading and listening comprehension for several reasons. Listeners, unlike readers, have no control over the rate of input, nor can they “re-experience” parts of the speech signal that are unclear. In addition, words in a written text are clearly separated by blank spaces, whereas coarticulation across word boundaries means that the beginnings and ends of words are not obviously marked in the acoustic stimulus. Listeners can make use of cues in the speech input that are not available to readers (e.g., prosody, hesitations). Moreover, many studies of spoken language processing show that word recognition in passages or sentences occurs before the word can be unambiguously identified given the speech signal (Cutler & Norris, 1979; Marslen-Wilson & Tyler, 1980; Marslen-Wilson, 1985; Zwitserlood, 1989). Such inherent differences between understanding speech and understanding written text may influence if and when lexical association and discourse congruence affect word processing in discourse contexts. Before performing our discourse experiment, we conducted an ERP norming study, in which we examined lexical association in the absence of discourse, presenting primes and targets in pairs. This was done to establish a baseline N400 effect of lexical association for the items to be used in the experiment, in which we embedded the pairs in discourse contexts and manipulated discourse congruence. In the following, we report the ERP norming study and then introduce and report the results of the discourse experiment.

ERP Norming Study

In many tasks, lexical-semantic processing is facilitated when words are preceded by “prime” words that are associated in meaning (e.g., salt-pepper). This facilitation is extremely robust when words are presented in pairs or in random lists of words (e.g., Meyer & Schvaneveldt, 1971; Holcomb & Neville, 1990). Previous studies on the influence of lexical association on word processing have primarily involved word pairs or random word lists and visual or cross-modal presentation in which a spoken prime word or word fragment is followed by a visual target (e.g., Andruski, Blumstein, & Burton, 1994; Bölte & Coenen, 2002; Marslen-Wilson & Zwitserlood, 1989; Swinney, Onifer, Prather, & Hirshkowitz, 1979; Williams, 1988). Behaviorally, facilitation by means of lexical association has been observed as reduced lexical-decision and naming times for related compared to unrelated words (e.g. Norris et al., 2007; Neely, 1991). Electrophysiological studies using event-related potentials (ERPs) have also found robust effects of lexical association on word processing with uni-modal visual and cross-modal presentation of primes and targets, which is reflected in the modulation of the N400 component.

Although quite a few ERP studies have assessed effects of rhyme in the auditory modality (e.g. Praamstra & Stegeman, 1993; Praamstra, Meyer & Levelt, 1994), few have studied facilitation when lexical associates are presented in the auditory modality (Hagoort, Brown & Swaab, 1996). Hagoort, Brown and Swaab (1996) examined lexical-semantic priming in patients with aphasia who were unable to read, and elderly control subjects who were matched with respect to age and education with the patients. In this study, both prime and target words were presented in the auditory modality. The results showed N400 priming effects for the elderly control subjects and patients with mild comprehension deficits. Interestingly, N400 priming effects were also found for patients with severe comprehension deficits, suggesting that their impaired comprehension could not be attributed to a loss of lexical-semantic representations alone (e.g., Hagoort et al., 1996) This finding also suggests that lexical-semantic priming can occur separately and in the absence of overall comprehension. In order to establish that our materials would elicit a canonical N400 priming effect, we performed a unimodal auditory priming experiment in which the primes and targets were presented in pairs without context.

Methods

Participants

Fourteen undergraduates from the University of California, Davis gave informed consent and participated in the study; they were compensated with course credit. All were right-handed, native speakers of English, with no reported hearing loss or psychiatric/neurological disorders.

Stimuli

Associated prime-target word pairs were culled from the Edinburgh Associative Thesaurus (Kiss, Armstrong, Milroy & Piper, 1973) and from association pre-tests. There were a total of 132 experimental and 162 filler trials. Filler pairs were all unrelated in meaning and this resulted in 22% associated word pairs in the experiment. The low ratio of related to unrelated trials was chosen to make it less likely that participants would develop a strategy in which they tried to predict a target word after receiving a prime. All associated word pairs had an association strength of at least 20%, with an average rating of 39.8% (range: 20–90%). Unassociated word pairs were created by using control words that were not associated, or were only mildly associated, with the prime (average association score of 0.2%, range 0–4%). The associated and unassociated target words were matched on word frequency using Francis and Kucera (1982) word counts (associated: M = 83.71 per million, SD = 136; unassociated: M = 83.86 per million, SD = 182; t < 1). The two types of targets were also matched on spoken duration, with associated targets having an average duration of 776 ms (range 444 ms – 1119 ms), and unassociated targets having an average duration of 805 ms (range 563 ms – 1092 ms); (t = −1.328; p = .189).

All words were randomized for recording purposes and read by a female, native speaker of American English with natural intonation and at a normal speaking rate. The words were digitally recorded using a Schoeps MK2 microphone and Sound Devices USBPre A/D (44,100 Hz, 16 bit). Speech onset and offset of each word was determined by visual inspection of the speech waveform and by listening to the words using speech editing software (Audacity, by Soundforge). The average duration of prime words was 729 milliseconds (ms); target words had an average duration of 791ms. The time between onset of prime and onset of target (Stimulus Onset Asynchrony (SOA)) was held constant at 1312ms. This was achieved by varying the interval of silence (ISI) between the prime and the target words; the prime with the longest duration was identified (1212 ms) and then a 100 ms interval of silence was added. The interval of silence for all other stimuli was calculated by subtracting the duration of the prime words from the SOA (1312 ms), and ranged from 100 ms to 1075 ms.

Procedure

Participants sat in a comfortable chair in an electrically shielded, sound-attenuating booth. The stimuli were presented through Beyer dynamic headphones using Presentation software. Participants were instructed to listen to the stimuli for comprehension; participants had no other task. Trials began with a white fixation cross appearing against a black background in the center of a computer screen approximately 100 cm in front of the subjects. The white fixation cross was presented 1000 ms prior to the onset of the prime word and remained visible during the auditory presentation of both prime and target words, and for an additional 1000 ms after the offset of the target word. The white cross was then replaced by a green cross that was presented for 1000 ms, followed by the sentence “Press for Next”.

Participants were asked to keep their eyes fixated on the white cross and to refrain from blinking or moving as long as the white cross was visible. This was done to minimize subject-generated artifacts in the EEG signal. When the white fixation cross was replaced with the green one, subjects were told that they could blink and move their eyes until they pressed for next. Condition-specific stimulus codes were sent at the onset of the critical words and these codes were used for later off-line averaging of the EEG.

ERP recording and data reduction

EEG was recorded from 29 tin electrodes, mounted in an elastic cap (ElectroCap International). Additional electrodes were placed on the outer canthi and below the left eye in order to monitor eye movements and blinks. The right mastoid electrode was used as the recording reference. The left mastoid was also recorded for later off-line algebraic re-referencing. The EEG signal was amplified with band pass cutoffs at .01 and 30 Hz, and digitized on-line at a sampling rate of 250 Hz (Neuroscan Synamp I). EEG was digitized continuously along with accompanying stimulus codes used for subsequent averaging. Impedances were kept below 5 kΩ.

Prior to off-line averaging, all single-trial waveforms were screened for amplifier blocking, muscle artifacts, horizontal eye movements and blinks over epochs of 1200 ms, starting 200 ms before the onset of the critical target words. This led to an average rejection rate of 7.4% of the trials. Average ERPs were computed over artifact-free trials in the related and unrelated conditions. All ERPs were filtered off-line with a Gaussian low-pass filter with a 25 Hz half-amplitude cutoff. Statistical analyses were conducted on the filtered data.

Results and Discussion

As can be seen in Figure 1, unassociated targets elicited more negative waveforms than their associated counterparts; this is particularly evident over the central-posterior electrode sites.

Figure 1.

ERP norming study: Effects of lexical association. Displayed on the left are ERP waveforms to the target words in the unrelated (red) and the related (blue) conditions. In this figure and all other figures, waveforms are shown for 9 electrodes over central, parietal and posterior electrode sites. Negative voltage is plotted upward. Displayed on the right is a topographic map showing the scalp distribution of the N400 effect (unrelated – related) in the 300-500ms time window.

A repeated-measures ANOVA was performed on the mean amplitude of the N400 in a typical 300–500ms epoch. A Greenhouse-Geisser correction was applied to F tests with more than one degree of freedom in the numerator. Association (Associated vs. Unassociated) and Electrode Site (29 sites) were within-subjects factors. No main effect of association was observed (F(1,13) = 1.35; p = 0.267), but a significant Association by Electrode Site was found (F(28,364) = 2.98; p = 0.0432). As a follow up to this interaction, a subset analysis was conducted over 9 electrode sites for which the N400 effect is found to be maximal (subset analysis; CZ, CP1, CP2, PZ, POZ, P3, P4, O1 and O2). A main effect of Association was found in the subset analysis (F(1,13) = 5.2; p = 0.0401). In order to determine the onset of this effect, we conducted repeated measures ANOVAs over 100ms epochs between 150–800ms, with an overlap of 50ms for each time window (i.e. 150–250 ms, 200–300 ms, 250–350 ms, 300–400 ms, 350–450 ms, 400–500 ms, 450–550 ms, 500–600 ms, 650–750 ms, 700–800 ms), across the same nine electrodes and with the same factors as the previous analysis (Association (Associated vs. Unassociated and Electrode Site (the subset of 9 electrodes). Results for these analyses are reported in Table 1.

Table 1.

Results of lexical association from repeated measures ANOVAs across nine centro-posterior electrodes (CZ, CP1, CP2, PZ, POZ, P3, P4, O1 and O2) for the ERP norming study. The degrees of freedom for all F values were (1, 13).

	Association
Epoch	F	p
150– 250ms	1.9	0.19
200– 300ms	1.76	0.2
250– 350ms	2.88	0.11
300– 400ms	3.08	0.1
350– 450ms	3.54	0.08
400– 500ms	7.39	0.0176
450– 550ms	10.93	0.0057
500– 600ms	18.68	0.0008
550– 650ms	23.83	0.0003
600– 700ms	14.2	0.0023
650– 750ms	15.83	0.0016
700– 800ms	13.79	0.0026

As can be seen in Table 1, the onset analyses showed significant effects of association for all time windows between 400– 800ms, with a trend emerging between 350–450ms.

In sum, associative relations between words that were presented in pairs elicited an N400 effect with the typical distribution of the canonical auditory N400, but had a later onset and a longer duration than what is typically observed for visual N400 effects (i.e., 300–500ms). This discrepancy between the onset and duration of the auditory and visual N400 effects between words might be attributed to differences in the time it takes to perceive and recognize the word stimuli. Prime and target words in visual studies are usually presented for about 200ms, and studies of text comprehension suggest that this is also the average time by which a word is recognized. In addition, as has been shown in many previous visual studies, the meaning of the prime word has a rapid influence on processing the meaning of the target word, resulting in an N400 effect between 300–500ms. In the current study, however, the average duration of the prime and target words was much longer (791ms; range 444 ms – 1119 ms); even though listeners typically do not need to hear the entire speech signal to uniquely identify a word, it is reasonable to conclude that perception and access to lexical-semantic information in our auditory priming study would be delayed for both prime and target words relative to a typical visual priming study. This extended time for lexical access could explain the delay in the onset of N400 effect relative to N400 effects of lexical association in the visual modality. The longer duration of the N400 effect in the auditory modality could also be explained as a function of the duration of the target word; perception and recognition of words with a long duration would result in later N400 effects than for words with a short duration, resulting in an average N400 effect that is extended over a longer period of time. A late onset of the N400 in the auditory modality has also been observed by Diaz and Swaab (2007), in response to semantically consistent compared to inconsistent words in lists (e.g. dog vs. desk following a list of animals). Relevant to the current study is that this relatively late N400 effect was elicited by spoken words in the absence of linguistic structure.

Overall, our study showed significant N400 effects of lexical association on target word processing when both prime and target words were presented in the auditory modality. This is important because it establishes that the prime-target pairs to be used in our discourse experiment affect processing of words when they are presented in pairs in the auditory modality.

ERP Experiment of Association in Discourse Context

In the current experiment, the same word pairs that were tested in the ERP norming study were embedded in discourse contexts (see Table 2 and the Appendix for examples of the stimuli). Critical words appeared in the third and final sentence of the passages and varied in terms of discourse congruence (congruent, incongruent) and lexical association (related, unrelated) with a preceding word in the immediate context (e.g., “Luckily Ben had picked up some salt and pepper/basil”) The critical sentence was preceded by a context in which Ben was either preparing marinara sauce (congruent discourse) or dealing with an icy walkway (incongruent discourse). Within a stimulus set, the final sentences were identical up to the critical word. Moreover, the critical words were always congruent in their local sentence contexts (i.e. the preceding discourse context is what made the critical word congruous or incongruous). ERPs were time-locked to the target words in the four conditions: (1) Congruent/ Associated, (2) Congruent/ Unassociated, (3) Incongruent/ Associated, and (4) Incongruent/ Unassociated.

Table 2.

Example stimulus sets showing each of the four conditions described in the text. For clarification, the primes are shown in italics and target words are capitalized; during the experiment these words were not specifically emphasized.

Condition	Context	Target Sentence
Associated Congruent	Keith was arranging furniture in the lobby of the hotel. He wanted to make sure the room appeared comfortable, but not overly cluttered.	He was very specific about the placement of the chairs and TABLES.
Unassociated Congruent	Keith was arranging furniture in the lobby of the hotel. He wanted to make sure the room appeared comfortable, but not overly cluttered.	He was very specific about the placement of the chairs and LAMPS.
Associated Incongruent	Keith wanted the arrangements for his outdoor wedding to be very simple. There would only be seats for the guests and a simple archway.	He was very specific about the placement of the chairs and TABLES.
Unassociated Incongruent	Keith wanted the arrangements for his outdoor wedding to be very simple. There would only be seats for the guests and a simple archway.	He was very specific about the placement of the chairs and LAMPS.
Associated Congruent	Although he tried very hard, Ben’s cooking skills were pathetic at best. His latest attempt at making marinara sauce was particularly bland and unappetizing.	Luckily he had picked up some salt and PEPPER.
Unassociated Congruent	Although he tried very hard, Ben’s cooking skills were pathetic at best. His latest attempt at making marinara sauce was particularly bland and unappetizing.	Luckily he had picked up some salt and BASIL.
Associated Incongruent	Todd slipped on a large patch of ice near his front step. He wanted to be sure the ice melted before anyone else took a fall.	Luckily he had picked up some salt and PEPPER.
Unassociated Incongruent	Todd slipped on a large patch of ice near his front step. He wanted to be sure the ice melted before anyone else took a fall.	Luckily he had picked up some salt and BASIL.

As discussed previously, rapid effects of global discourse on word processing have been observed in recent studies, both during reading (e.g. van Berkum et al., 1999) and listening (e.g. van Berkum et al., 2003). On the basis of these findings, we predicted that the discourse congruency manipulation would result in an early and robust N400 effect, with reduced amplitude for congruent relative to incongruent critical words. The basis for predictions about lexical association are far less clear, however, since most previous studies of language comprehension have not presented critical words in the auditory modality. Furthermore, studies using either unimodal visual or cross-modal presentations have shown mixed results with respect to lexical association in discourse contexts. In a reading study, Camblin and colleagues showed lexical association effects on N400 and eye-tracking measures that were both later and less robust than the effects of discourse congruence (Camblin et al., 2007). In cross-modal studies in which participants listened to discourse and received visual target words, some studies have shown effects of association in short passages (Blutner & Sommer, 1988; Swinney, 1979).

A main effect of lexical association in the current study would manifest as a reduced N400 to targets in the associated relative to unassociated condition. The timing of an N400 effect of lexical association and discourse congruence will indicate whether lexical processing is delayed relative to discourse processing, as was found in reading by Camblin et al. (2007). Alternatively, the absence of an N400 effect of lexical association would suggest that the meaning relations between individual words do not immediately contribute to the processing of incoming words during comprehension of spoken discourse. Finally, although Camblin et al. (2007) found that lexical association and discourse congruence did not interact, Hess et al.’s (2005) findings using a cross-modal paradigm suggest that an interaction may be observed in listening comprehension such that the association effects depend on the congruency of the prime with the global discourse context. However, Hess and colleagues (2005) manipulated the prime words in their study such that they were consistent or not with the preceding discourse. In contrast, in our study, prime words were always consistent with the global context and our manipulation of discourse congruence and lexical association was for target words only. This makes it possible to determine if any effect of association is restricted to the congruent discourse condition, as observed by Hess and colleagues. Further, this design makes it possible to determine whether discourse congruence effects are restricted to cases in which targets are associates of lexical primes.

The goals of this experiment were to assess (1) if lexical association plays a role in the real-time processing of incoming words that are either congruent or incongruent with the preceding discourse and (2) the time course by which discourse congruence and lexical association influence word processing. Thus, the results of the current study may further our understanding of the contributions of meaning relations between individual words and the representation of the discourse context to on-line processing of words in short spoken discourse.

Method

Participants

Informed consent was obtained from sixteen students at the University of California, Davis, who received course credit for their participation. All participants were right-handed, native speakers of English and reported no psychiatric or neurological disorder; no subjects had participated in the ERP norming study.

Stimuli

The materials consisted of 72 experimental story sets (from Camblin et al, 2007), in which discourse congruence (congruent or incongruent) and lexical association (associated or unassociated) was orthogonally varied to produce four conditions (See Table 2): Congruent-Associated; Congruent-Unassociated; Incongruent-Associated; Incongruent-Unassociated. For a detailed description of the pre-tests and construction of the stimuli we refer the reader to Camblin et al. (2007); the most relevant aspects of the stimuli are summarized here. Pre-testing of the congruence of the passages on a 5 point scale (1= completely congruent; 5=completely incongruent) showed that incongruent stories were rated as significantly less congruent than congruent stories in both associated and unassociated conditions (Congruent/Associated passages, M = 1.44; Congruent/Unassociated, M = 1.5; Incongruent/Associated, M = 4.31; Incongruent/Unassociated, M = 4.32). Latent semantic analysis (LSA) of the stories showed that conditions did not differ in coherence, with LSA values ranging from .21 to.22 across conditions; LSA values represent the similarity in meaning between words, sentences or passages (see Landauer, Foltz & Laham, 1998). Furthermore, story cloze probability was low (< 33%) and did not affect the pattern of results in a previous reading study (Camblin et al., 2007) or in the current study¹. The target words were identical to those in the ERP norming study; associated and unassociated target words were matched for length and frequency

As can be seen in Table 2 and the Appendix, the first two sentences of each three-sentence passage established a discourse context and the third (final) sentence contained the associated or unassociated target word. The target was either congruent or incongruent with the discourse. All but 22 of the prime and target pairs were presented with one intervening word; the remaining stimuli had either two intervening words (8), or three (14). The target was always congruent with the meaning of the final sentence. That is, the target word made sense in its context when the sentence was heard in isolation. Sentence sets were divided into four lists and counterbalanced such that the critical words and the two sentences preceding the final sentence were not repeated within subjects. Each list contained 144 experimental stories, 36 in each condition. An additional forty filler stories were included; 20 were congruent and 20 ended with a word that was anomalous at both the discourse and sentence level.

All story sets and fillers were spoken by a female speaker (the same speaker as in the priming experiment), with natural inflection and at a natural speaking rate. Recording was done using the same procedure and apparatus as in the first experiment. The discourse context (first two sentences of the passage) and the critical sentence were recorded separately. Speech onset and offset of each critical word in all conditions was determined by visual inspection of the speech waveform and by listening to the words using speech-editing software (Audacity, by Soundforge). The average duration of the stories was 8951 ms (ranging from 7917–9359 ms), and the average duration of the critical words was 568 ms (ranging from 293–861ms). The duration of the critical words did not differ between conditions (t<1). The duration between prime and target was the same for the congruent/incongruent associated and for the congruent/incongruent unassociated conditions because the same final sentence was used for the two associated conditions and the two unassociated conditions. There were no statistically significant differences in duration of the onset of the prime to the onset of the target between the associated and unassociated conditions (p=0.43). The average duration of the target words was much shorter in this experiment than in the ERP norming study. This is due to the fact that the target words in this experiment coarticulated with the preceding speech input. This did not affect the comparison between effects of priming and discourse congruence in this study because association and discourse congruence were compared within subjects.

A one-second silence was inserted between the second and third sentences using Presentation software²; this allowed for the same two-sentence contexts to be used in creating the congruent-associated and congruent-unassociated condition. The same procedure was used to create the incongruent-associated and incongruent-unassociated conditions. The target sentences were the same in all conditions up to the sentence-final target word.

Comprehension questions did not focus on the prime or the target but rather on the context of the discourse (see Appendix for examples). The same true/false question was asked in the congruent associated/unassociated and incongruent associated/unassociated conditions; this was necessary because the content of the discourse varied as a function of congruency. Half of the questions required a true response, and half required a false response.

Procedure

The procedure was the same as in the ERP norming study, except for the following differences. The discourse trials began with a white fixation cross at the center of the screen, approximately 100 cm in front of the participants. The fixation cross was present from 900 ms before onset of the stimuli and during presentation of the entire passage until the offset of the final word. The fixation cross was then replaced by a visually presented comprehension question about the preceding discourse. Subjects were asked to make a true/false response by pressing a yes or no button with the index and middle fingers of their right hand, respectively. The comprehension question remained on the screen until the subject made a response.

Each experimental session began with a practice block consisting of filler passages, after which two of the four counterbalanced lists were presented in random order, each containing both experimental trials and fillers in a pseudorandom order. Each list was divided into four blocks for presentation purposes such that each participant listened to eight blocks of stimuli (two lists). The order of blocks was counterbalanced.

ERP recording and data reduction

EEG recording and data reduction were performed as described in the ERP norming study.

Results

Behavior

Participants, on average, scored 93% percent correct (range: 83–99%) on the true/false comprehension questions. Accuracy did not significantly differ between conditions, with participants answering 93%, 92%, 92% and 91% of the questions correctly for the congruent/ associated, incongruent/ associated, congruent/ unassociated and incongruent/ unassociated conditions, respectively.

ERPs

Figure 2 shows the N400 effect of priming as a function of congruency at central, parietal, and posterior electrode sites. Figure 3 shows the N400 effect of congruency as a function of association. Figure 4 (top) shows the difference waveforms for association (unassociated - associated) in the congruent and incongruent conditions and Figure 4 (bottom) shows differences for congruence (incongruent - congruent) in the associated and unassociated conditions. Figure 5 shows topographic maps of the scalp distribution of the N400 effects for the four pair-wise comparisons in 100ms epochs.

Figure 2.

Effects of association in the discourse experiment. ERP waveforms are shown for the associated (blue lines) and unassociated (red lines) target words in the congruent (left side) and incongruent (right side) conditions. Examples of prime and target words in each condition are displayed separately, target words are underlined. Also shown are examples of discourses that were congruent or incongruent with the meaning of the critical word.

Figure 3.

Effects of discourse congruence in the discourse experiment. ERP waveforms are shown for the congruent (blue lines) and incongruent (red lines) target words in the associated (left side) and unassociated (right side) conditions. Examples of prime and target words in each condition are displayed separately, target words are underlined. Also shown are examples of discourses that were congruent or incongruent with the meaning of the critical word.

Figure 4.

The top part of this figure shows the difference waveforms for Unassociated-Associated targets in the Congruent and Incongruent conditions. The bottom part shows the difference waveforms for Incongruent-Congruent targets in the Associated and Unassociated conditions.

Figure 5.

Topographic maps showing the scalp distribution of the N400 effects for the four pair-wise comparisons in 100ms epochs from 250– 600ms. Significance values are denoted as follows: ***p<.001, **p<.001, *p<.05, ^p<.09.

As in the ERP norming study, a repeated measures ANOVA was performed on the mean amplitude of the N400 in the typical 300–500ms epoch, with Association (Associated vs. Unassociated), Discourse Congruence (Congruent, Incongruent) and Electrode Site (29 sites) as within-subjects factors. A Greenhouse-Geisser correction was applied to F tests with more than one degree of freedom in the numerator for all relevant analyses reported in this paper. Interactions with electrode site were followed-up with an ANOVA on a subset of 9 electrode sites for which the N400 effect has been found to be maximal (subset analysis; CZ, CP1, CP2, PZ, POZ, P3, P4, O1 and O2). In order to determine the onset of the effects, we again conducted repeated measures ANOVAs over 100ms epochs between 150–800ms, with an overlap of 50ms for each time window (i.e. 150–250 ms, 200–300 ms, 250–350 ms, 300–400 ms, 350–450 ms, 400–500 ms, 450–550 ms, 500–600 ms, 650–750 ms, 700–800 ms). These onset analyses were conducted on the subset of nine electrodes and included the same factors as in the previous analyses (i.e., discourse congruence, association, and electrode site (9 electrodes). Results for these analyses are reported in Table 3.

Table 3.

Results of discourse congruence and lexical association from repeated measures ANOVAs across nine centro-posterior electrodes (CZ, CP1, CP2, PZ, POZ, P3, P4, O1 and O2) for the discourse experiment. The degrees of freedom for all F values were (1, 15).

	Congruence		Association		Interaction C × A
Epoch	F	p	F	p	F	p
150– 250ms	2.24	0.16	<1		1.19	0.29
200– 300ms	5.25	0.04	1.71	0.21	1.93	0.18
250– 350ms	12.13	0.003	3.4	0.09	6.48	0.02
300– 400ms	12.56	0.003	5.08	0.04	3.86	0.07
350– 450ms	13	0.003	4.97	0.04	1.88	0.19
400– 500ms	30.94	0.0001	5.59	0.03	<1
450– 550ms	31.75	0.0000	5.93	0.03	<1
500– 600ms	11.21	0.004	4.36	0.05	<1
550– 650ms	5.55	0.03	2.07	0.17	<1
600– 700ms	6.68	0.02	1.01	0.33	<1
650– 750ms	8.02	0.01	1.66	0.22	<1
700– 800ms	7.4	0.02	2.33	0.15	<1

The ANOVA in the 300–500 ms epoch revealed a significant main effect of discourse congruence (F(1,15) = 26.61; p = 0.0001); the amplitude of the N400 to critical words in the congruent condition was reduced relative to the incongruent condition. The interaction of congruence by electrode was significant (F(28,420) = 2.77; p = 0.0354), indicating that the effect was maximal over centro-parietal electrode sites as is typical of the N400 (see subset analysis below). There was no main effect of association, but a significant interaction of association with Electrode Site F(28,420) = 2.57; p = 0.0469), indicating the N400 effect of priming (reduced N400 in the associated relative to unassociated condition) was restricted to centro-parietal electrode sites (which was confirmed in the ANOVA on the subset of centro-parietal electrodes reported below). Finally, we found no congruence by association interaction (F(1,15) = 1.38; p = 0.259) and no significant three-way interaction (F<1).

The interaction of association and discourse congruence by electrode site was followed-up with an ANOVA on the subset of 9 centro-parietal electrode sites; this analysis revealed main effects of discourse congruence (F(1,15) = 23.75; p = 0.0002) and association F(1,15) = 5.74; p = 0.03).

As can be seen in Table 3, the onset analyses showed significant effects of discourse congruence in all time windows between 200–800 ms and main effects of association were found in the epochs between 300–600 ms. A significant interaction between discourse and association was found in the 250–350 ms time window and was marginally significant in the 300–400 ms time window.

In order to identify the source of the interaction, we performed pair-wise comparisons in the same 100ms overlapping time windows described above. These analyses were conducted on the subset of nine electrodes as in the previous analyses. Results for these analyses are reported in Table 4. A significant effect of congruence for associated words was obtained in all time windows between 150– 600ms; in comparison, the effect of congruence for unassociated words was significant in all time windows from 400– 650ms, with a trend toward significance beginning at 300ms. A significant effect of association for congruent words was found between 200– 500ms, and a trend toward a significant effect of association for incongruent words was found between 500– 600ms.

Table 4.

F values for each of the pair-wise comparisons from repeated measures ANOVAs across nine centro-posterior electrodes (CZ, CP1, CP2, PZ, POZ, P3, P4, O1 and O2) for the discourse experiment. The degrees of freedom for all F values were (1, 15).

	Congruence		Association
EPOCH	Associated	Unassociated	Congruent	Incongruent
150– 250ms	5.54*	<1	1.44	<1
200– 300ms	12.19**	<1	4.25^	<1
250– 350ms	21.49***	1.16	9.64**	<1
300– 400ms	16.90***	3.33^	8.79**	<1
350– 450ms	13.55**	3.66^	5.96*	<1
400– 500ms	18.15***	8.91**	5.09*	1.26
450– 550ms	14.44**	10.22**	2.66	2.74
500– 600ms	4.78*	5.97*	1.26	3.35^
550– 650ms	1.73	3.96^	<1	2.28
600– 700ms	1.68	3.58^	<1	1.48
650– 750ms	1.02	3.55^	<1	2.12
700– 800ms	<1	2.98	<1	2.21

^{^}p<.09

^*p<.05

^**p<.01

^***p<.001

In sum, the effect of discourse congruence had an earlier onset and lasted longer than the effect of association. Moreover, an interaction effect was found between discourse congruence and lexical association from 250– 400ms; pair-wise comparisons showed that the congruence effect emerged earlier in the associated than in the unassociated condition and the association effect emerged earlier in the congruent than in the incongruent condition.

General Discussion

The goal of this study was to examine the time-course by which local lexical semantic relations between individual words and global discourse congruence affect the processing of incoming words in discourse. Specifically, we were interested in identifying if and when stored semantic representations of individual words interact with the episodic representation of discourse contexts during lexical-semantic processing. In an ERP norming study, we first established that a set of prime-target word pairs, subsequently used in the discourse experiment, elicited reliable N400 modulation as a function of lexical association. In the discourse experiment, these same prime-target pairs were embedded in discourse. The target words were either associated or not to a locally preceding “prime” word, and were congruent or not with the overall discourse passage. ERPs were recorded from healthy adults as they listened to these short stories. Both the lexical association and discourse congruence manipulations resulted in reliable N400 effects. However, interactions between these factors were observed on the onset and duration of the N400 effect. The largest and earliest N400 effects on the critical words were obtained for the effects of congruence in the associated condition, and for the effects of association in the congruent condition; that is, the largest effects occurred in those conditions in which the global discourse representation and the local association converged on the same meaning. Effects of congruence were observed for unassociated items, but this effect emerged later and had a shorter duration than in the other two comparisons. Finally, in incongruent discourses, association had a very small and late effect on the N400. The results also show that the effects of association were not always smaller than those of discourse congruence. The effect of association for the congruent items started earlier (250ms) and lasted longer (a total of 250ms) than the effects of congruence for the unassociated items (onset at 400 ms, duration 200ms).

The onset of the discourse-congruence effect (200ms) occurred well before the offset of the target words, which had an average duration of 568ms ms, i.e., before the whole word could have been perceived by the listener. Consistent with previous findings, our study shows very early effects of the overall discourse representation on local lexical-semantic processing of incoming words (e.g. van Berkum et al., 1999; 2003; 2005; Federmeier & Kutas, 1999; Nieuwland & van Berkum, 2006; Otten & van Berkum, 2008; Camblin et al, 2007; Kuperberg et al, in press).

The effects of lexical association in our study differ from the findings of behavioral studies that have examined the effects of global discourse and local association with a cross- modal priming paradigm (Blutner & Sommer, 1988; Hess et al, 1995; Swinney, 1979, Experiment 1). When a lexical decision task was used, reduced lexical decision latencies were found to visually presented target words that were consistent with the global context, and this facilitation increased when the target words were also associated to a lexical prime in the local context. However, those studies found no effects of lexical association for target words that were inconsistent with the global contexts. When cross-modal priming was paired with naming rather than lexical decision, naming latencies to a visual target word were reduced when they were consistent with the overall context, but were not further facilitated by an associated prime word in the local context (Hess et al., 1995). In our study, effects of lexical association were not restricted to the congruent discourse conditions: We obtained a small and late N400 effect as a function of lexical association in the incongruent condition as well. Further, in contrast to the naming study by Hess and colleagues, the earliest effect of congruency was obtained when the critical word was also associated to a preceding prime word in the local context. The discrepancy of our results with those using the cross-modal paradigm may be, in part, due to differences in the task and the way in which the stimuli were presented. In cross-modal studies, participants are asked to name or to make a lexical decision to a target word that is presented in the visual modality, whereas, in our study, both primes and target words were embedded in the speech stream. Moreover, no decision was made to the target word. A cross-modal presentation of context and target words along with a secondary task may interrupt comprehension in a manner that did not occur in our study. Additionally, it is possible that the latency of lexical decisions and naming do not reflect contributions of both global discourse and local association, but are instead based on which source of information is available first.

The results of the current study also differ from previous results by Camblin et al. (2007). They pitted the effects of global discourse congruency and lexical association on word processing during reading comprehension (Camblin et al, 2007). They found independent effects of discourse congruence and lexical association, whereas we found an interaction between these factors in listening comprehension. As discussed before, in the current study the largest and earliest N400 effects were found when discourse congruence and lexical association converged on the same target word. The effect of congruence was also observed for unassociated targets, however this effect emerged later. Finally, the effect of lexical association for target words in incongruent discourse was relatively small and delayed. Overall, this pattern of results suggests that during spoken language comprehension, local lexical associations and overall discourse congruence may each exert their own influence on processing of incoming words in discourse; however, when these two sources of information are consistent with one another, their effects are additive, leading to an additional processing benefit for targets that are both associated and congruent.

The difference in findings for reading and listening comprehension may result from any one of many differences between spoken and written discourse. As we mentioned previously, listening and reading differ in several crucial ways. In contrast to written input, speech contains prosodic contours and stress patterns. Prosody has been shown to affect the salience and facilitate the integration of concepts in the developing sentence representation (Norris et al., 2006) and prosody also affects the semantic and syntactic interpretations of sentences (Akker & Cutler, 2003; Steinhauer, Alter & Friederici, 1999). Further, coarticulation can facilitate perception of incoming words during listening but not during reading comprehension. But perhaps the most important difference between written and the spoken input is that the speech signal disappears, but the written text remains available. Readers can always go back in the text to make sure that they understood the input, but listeners must interpret and comprehend the speech-signal as it unfolds in real-time. Future studies that directly compare reading and listening comprehension are needed to further assess if and how these differences between written and spoken input lead to differences in the time course by which stored lexical-semantic information and the overall discourse representation influence processing of incoming words in context.

The results of our study are consistent with models of language comprehension that assume a parallel architecture, and constraint-based processing. That is, multiple sources of information, including the global discourse representation, rapidly facilitate the retrieval of incoming words in discourse, and the strongest cues in the context have the earliest and most pronounced effect (e.g., van Berkum, 2009a). However, our results do not rule out the possibility that this rapid effect occurs during the integration and interpretation of the critical word within its context, which is consistent with two- stage models (e.g., Graesser, Singer, & Trabasso, 1994)

Our results can also be easily accommodated by a model that was proposed by Kuperberg (2007). She describes a model of language comprehension in which (at least) two processing streams act in parallel. The semantic memory-based stream computes semantic features and relations among words in the sentence. Processing difficulties are primarily reflected in the N400 component. The combinatorial stream is sensitive to linguistic constraints, including constraints of morphosyntax and thematic role relations. When the two streams provide contradictory output (that is, when the semantic interpretation that is computed by the first stream contradicts morphosyntactic or thematic information in the sentence), continued analysis must be undertaken to resolve the inconsistency. Kuperberg suggests that this is reflected in the P600 component. For example, Kuperberg and colleagues found a P600 to “eat” in sentences like “At breakfast the eggs would eat…” (Kuperberg et al., 2003; see also: Kolk, Chwilla, van Herten & Oor, 2003; van Herten, Chwilla & Kolk, 2006, but see Bornkessel-Schlesewsky & Schlesewsky, 2008 for a different interpretation). In our study, we found an N400, but not a P600, effect to conflicting information from local association and global discourse in the associated/ incongruent condition and in the unassociated/ congruent condition³. Thus, consistent with predictions from Kuperberg’s model, the P600 is not elicited when conflict during language processing results from mismatching semantic information generated by the integrated representation of the discourse context and semantic information from the individual words in the local context.

Our congruence manipulation involved passages that were not highly constraining, and for which the meaning representation had to be constructed in real-time by the listener, without much facilitation from stored discourse scenarios. Nevertheless, our results show a very early and robust N400 effect of discourse congruence on the processing of incoming words in the discourse context. We therefore suggest that the semantic stream as proposed by Kuperberg may additionally include meaning representations that are not stored in semantic memory. Recent findings by Kuperberg and colleagues (in press) are consistent with this idea. They showed immediate effects of inferencing on the N400 to incoming words, in stories for which meaning associations between words were held constant.

Our results may appear to contradict studies that have found a P600 effect to semantic illusions in discourse contexts. Nieuwland and van Berkum (2005) showed a P600 to critical words that were anomalous in discourse, but were highly consistent with the discourse scenario (for example when a “tourist” was replaced by a “suitcase” in an airport scenario). However, semantic illusions typically arise from highly schematic scenarios, which essentially serve as primes for certain words (e.g. the scenario of an airport serves as a prime for the word suitcase, leading to the kind of semantic illusion effect for suitcase described above). Our stories were not based on scenarios, but rather depicted novel utterances whose meaning had to be constructed “on the fly” by the listener.

In conclusion, the results reported here confirm that discourse-level factors have an early influence on the incremental processing of incoming words during spoken language comprehension. Our study also shows that incremental processing of incoming words in discourse is influenced by lexical association in the immediately preceding context. Finally, earlier N400 effects were observed when critical words were both associated and congruent with the overall discourse context than when they were unassociated and incongruent. We did not observe this interaction in a previous study that examined the effects of overall discourse congruence and associative priming during reading (Camblin et al, 2007). This suggests that global and local contexts may affect comprehension differently as a function of input modality.

Appendix

Example stimulus sets showing each of the four conditions. For clarification, the primes are shown in italics and target words are capitalized; during the experiment these words were not specifically emphasized.

1. Congruent-Associated/Unassociated

   • Lynn had gotten a sunburn at the beach. Nothing she tried would help her dry and irritated skin. She couldn’t stop scratching her arms and LEGS/NOSE.

   • True/False: Lynn stayed in the sun too long

   • Incongruent- Associated/Unassociated

   • Mary’s wool sweater was uncomfortable and itchy. She fidgeted as the rough material irritated her skin. She couldn’t stop scratching her arms and LEGS/NOSE.

   • True/False: Mary was very cozy in her warm sweater.

2. Congruent-Associated/Unassociated

   • Rachel hated being away from home during the school year. She loved visiting her family during the semester breaks. She looked forward to the summer and WINTER/CHRISTMAS.

   • True/False: Rachel lives with her family.

   • Incongruent- Associated/Unassociated

   • Kara hated the cold and complained bitterly when it snowed. Her mood always improved when the weather turned warm. She looked forward to the summer and WINTER/CHRISTMAS.

   • True/False: Kara prefers warm weather.

3. Congruent-Associated/Unassociated

   • Lisa enjoyed working at the pet store. She didn’t think there could be a better job for her. She loved all of the cats and DOGS/FISH.

   • True/False: Lisa hates animals.

   • Incongruent- Associated/Unassociated

   • Karen enjoyed going to the zoo. Her favorite enclosures held the tigers, lions, grizzlies and polar bears. She loved all of the cats and DOGS/FISH.

   • True/False: Karen loves animals.

4. Congruent- Associated/Unassociated

   • The comedic and lighthearted movie was greatly anticipated by Disney fans. People of all ages could enjoy the theme of the story and the charismatic characters. The story was definitely appreciated by adults and CHILDREN/TODDLERS.

   • True/False: Everyone hated the movie.

   • Incongruent- Associated/Unassociated

   • The history channel ran a human interest series on Holocaust survivors. It was far too disturbing for young people, but older audiences found it inspirational. The story was definitely appreciated by adults and CHILDREN/TODDLERS.

   • True/False: Everyone loved the movie.

5. Congruent- Associated/Unassociated

   • Cordelia’s parking space was in front of a large plot of lilies. Every spring it would coat her car in fine yellow dust. She was very annoyed with the pollen from the FLOWERS/BLOOMS.

   • True/False: Cordelia’s car was covered in pollen.

   • Incongruent- Associated/Unassociated

   • Sheila suffered from watery eyes and a constant sneeze every spring. She had pinpointed her allergy to the needles from the pine trees. Sheila was very annoyed with the pollen from the FLOWERS/BLOOMS.

   • True/False: Sheila is not allergic to anything.

6. Congruent- Associated/Unassociated

   • Charles wanted to make a light breakfast for his visiting guests. He was going to serve toast with a variety of spreads. Unfortunately he ran out of bread and BUTTER/JAM.

   • True/False: Charles wanted to make his guests dinner.

   • Incongruent- Associated/Unassociated

   • Garth sat by the pond and fed some crumbs to the nearby ducks. He enjoyed watching the ducks as they flocked around the food. Unfortunately he ran out of bread and BUTTER/JAM.

   • True/False: Garth enjoyed feeding the ducks.

7. Congruent- Associated/Unassociated

   • Darren knew his girlfriend wanted jewelry for their anniversary. He was relieved when he found the perfect gift at a nearby jewelry store. He got a good deal on the charm and BRACELET/RING.

   • True/False: Darren got a good deal on his girlfriend’s anniversary present.

   • Incongruent- Associated/Unassociated

   • Tommy got his mother a gold necklace for her birthday. Attached to the necklace was a trinket that read “World’s Best Mom.” He got a good deal on the charm and BRACELET/RING.

   • True/False: Tommy and his mom don’t get along.

8. Congruent- Associated/Unassociated

   • Fred bought a novel to read during his plane ride. He was so enthralled with the novel that the flight seemed relatively brief. He really enjoyed the story and the BOOK/WRITING.

   • True/False: Fred liked reading the novel.

   • Incongruent- Associated/Unassociated

   • Before Nathan’s bedtime his mom told him a tale. She talked to him about his grandparent’s journey to America many years ago. He really enjoyed the story and the BOOK/WRITING.

   • True/False: Nathan’s grandparents are originally from America.

9. Congruent- Associated/Unassociated

   • Noreen easily decided on the wedding gifts for her daughter and son-in-law. They would need better cookware and some new serving plates. She purchased some nice pots and PANS/DISHES.

   • True/False: Noreen’s son is getting married.

   • Incongruent- Associated/Unassociated

   • Jody found some beautiful ceramic planters to put in her sunroom. She also had her eye on some flowering cactus plants on sale. She purchases some nice pots and PANS/DISHES.

   • True/False: Jody is decorating her bedroom.

10. Congruent- Associated/Unassociated

    • Garrett was unaware that his sister had submitted his poem in the prestigious contest. He was shocked when he won the award and the hefty cash prize. He was not prepared for the fame and FORTUNE/PRAISE.

    • True/False: Garrett was given a public service award on the news.

    • Incongruent- Associated/Unassociated

    • Rick was mortified when the videotape of his arrest was shown on the news. After the news show aired, he was ridiculed by the entire neighborhood. He was not prepared for the fame and FORTUNE/PRAISE.

    • True/False: Rick was embarrassed about the footage.