Parafoveal N400 effect during sentence reading

Abstract

Research has suggested that during reading, parafoveal information pertaining to the next word in a line might be, at least partially, processed. We used event-related brain potentials (ERPs) to examine this issue. Volunteers read sentences presented word-by-word at fixation with each word flanked bilaterally on its right by the next word in the sentence and on its left by the preceding word. Infrequently, the right flanker was replaced by a word that was semantically incongruous with the ongoing sentence context. N400 amplitudes to the critical triads were smaller when the right flanker was contextually congruent than incongruent, indicating that parafoveal information was extracted and quickly and incrementally integrated within the evolving sentence representation.

INTRODUCTION

A consistent finding in behavioral studies is that linguistic stimuli can be at least partially processed when perceived outside the foveal visual field, usually defined as the central two degrees of the visual field around fixation [5, 6]. Eye-tracking research also has indicated that parafoveal perception may play an important role in reading. It seems that information in words can be acquired before they are foveally fixated, thereby determining saccade programming and facilitating their subsequent processing when they are fixated (see review in [15]). The impact of parafoveal information on the processing of a fixated word is known as the parafoveal-on-foveal effect. Not all researchers, however, have interpreted this type of effect as a genuine effect of parafoveal information on foveal processing, proposing instead that it results from misallocated fixations (i.e., unintended fixations on parafovealwords). This interpretive divide is theoretically important as these data are taken to support parallel versus serial models of attentional allocation during reading, respectively [4, 10].

Electrophysiological research on parafoveal perception in reading, which could be relevant to this issue, is scarce. A few studies have used event related brain potentials (ERPs) to analyze the processing of word pairs, with one word appearing at fixation and the other in the parafovea. Baccino & Manunta [1], for example, combined eye tracking and EEG recordings to obtain Eye-Fixation-Related Potentials (EFRP) associated to stimuli presented in this way. They reported effects on early sensory components (N1 and P140) contingent on the lexical status (word/non-word) of the parafoveal stimulus and a later effect on the P2 component varying with the associative relationship between the foveally and parafoveally presented words. A similar manipulation in the right and left visual hemifield showed similar ERP effects but only for words flashed in the right hemifield [16]. A hemifield asymmetry also has been observed in a magnetoencephalographic study of repetition priming (50 ms prime-to-target delay), in which prime words presented in the right but not in the left parafovea affected the processing of foveal target words, yielding left-lateralized activity in the occipito-temporal cortex and superior temporal cortex [14].

Obtaining electrophysiological correlates of parafoveally perceived words in sentence contexts is methodologically challenging because ERP recordings are very sensitive to artifacts caused by eye-movements. For this reason, in ERP studies of reading, sentences are presented one word after another, at a single point (usually fixation), with readers asked to avoid eye movements (e.g. [11]). In a study related to the present one, Barber at al. [2] observed small but reliable parafoveal ERP effects during sentence reading using a slight modification of this standard rapid visual serial presentation (RSVP) procedure. Short sentences were presented word-by-word on a computer screen at fixation, flanked two degrees bilaterally by letter strings. All but the third words in the sentences were flanked by two pseudowords. The third word of each sentence was flanked either by two pseudowords, or by a pseudoword/word combination. In the latter case, the word randomly appeared either as the left flanker or right flanker, and was either semantically congruent or incongruent with the sentence context. Parafoveal effects were examined in two languages, English and Hebrew, with different orthographic systems and reading directions. Relative to congruent flankers, incongruent words flankers (i.e., parafoveal words) elicited larger P2 amplitudes between 175 and 375 ms after triad onset. Critically, in English the parafoveal effect was observed only when the flanker words were positioned to the right of the fixated word, while the opposite pattern of flanker asymmetry was observed in Hebrew reading. The observed interaction between congruency, flanker position, and language is consistent with an attentional asymmetry determined by scanning directionality observed during reading [14, 15, 16]. These findings indicate that, at least under some circumstances, linguistic information acquired parafoveally is processed (at least to some degree), and demonstrate that ERPs are a suitable tool for examining questions of this type. However, the extent or depth of word processing in the parafovea during reading remains an open question. Words might be processed fully including their meaning, or alternatively only information about word form (e.g. length or orthographic regularities) may be extracted and related to the overall context. Kretzschmar et al. [13] examined saccadic-locked ERPS to sentence final words and reported a dissociation between foveal and parafoveal processes: predictability affected the foveal processing of the final word while context congruency modulated the responses to the previous word. On the other hand, eye-tracking studies of reading have consistently failed to demonstrate semantic effects in the parafovea [15].

The current study was designed to investigate some of these issues. German sentences were presented word-by-word at the central position, and each word (n) was flanked two degrees bilaterally by the next word in the sentence (n+1) to the right, and the previous word of the sentence (n-1) to the left (see Figure 1). Critical words occurred randomly at a variable middle position (words 2 to 6) in each sentence. The right flankers consisted either of the upcoming congruent word or a contextual incongruent word, yielding two different experimental conditions:

Figure 1.

Sentence presentation procedure (see text for description).

1. Congruent condition: in which flankers were n-1 on the left and n+1 on the right, just as in all the other triads of the sentence.

2. Incongruent condition: in which the right flanker was substituted by a word that was semantically incongruent with the overall sentence context.

For example, in the sentence Tino konnte ohne Brille nicht gut lesen (Tino could not read well without glasses), the critical word ohne (without) was flanked in the congruent condition by the word konnte (could) to its left, and the word Brille (glasses) to its right. Whereas in the incongruent condition, the same critical word was flanked to its right by the word Mäuse (mice).

EEG was recorded while the participants read sentences at fixation for meaning and ERPs were calculated separately for critical triads flanked by semantically congruent versus incongruent words. We predicted that the flanker manipulation would modulate ERPs time-locked to the presentation of the critical triad. Additionally, and due to the flanker types in the current study, we expected some involvement of semantic integration processes. Flanker congruency thus was expected to modulate the amplitude of the centro-parietal N400 component, a negative-going component that typically correlates with how well a word fits with a prior context – be it a word, a picture, a sentence, or a discourse [12]; the better the fit, the smaller the N400 amplitude.

METHOD

All procedures were cleared by the ethical review board of Otto-von-Guericke University.

Participants

Eighteen neurologically healthy students of the Otto-von-Guericke University took part in the experiment (mean age: 24.2, SD: 3.3; range: 19–30; 16 women). Participants were all right-handed, German native speakers with normal or corrected to normal vision.

Stimuli

A total of 360 German sentences were created for the experiment, ranging from 6 to 11 words in length, and presented according to the procedure and experimental conditions described above. To control for attentional biases to the right flanker, we also included a condition of incongruent left flankers, where n-1 was substituted by a word that was semantically incongruent with the overall sentence context (and n+1 was presented as right flanker). This left flanker condition, however, was not analyzed for a number of reasons. For one, in this case, contextual congruency effects are confounded with repetition priming effects because the left flankers in the congruent condition are the repetition of the n-1 word previously presented at central position (see Figure 1). Additionally, n-1 words belonged to different grammatical categories and their substitutions resulted in many cases in grammatical class violations. Different sentence lists were constructed to counterbalance the experimental conditions. No participant saw any given sentence more than once, however, across participants each sentence was presented in both conditions. In order to use the same words across participants, the n+1 word of each sentence was used as an incongruent flanker in a different sentence. Sentences were presented in a different random order for each participant.

Procedure

All stimuli were presented on a monitor positioned at eye level 70 cm in front of the participant, and displayed in yellow characters against a dark grey background. The distance between fixation and the center-most letter of the flankers was 2º. A single trial began with the presentation of the fixation cross for 1000 ms, followed by a 500 ms blank interval. After that the sentence was presented serially in 6 to 11 displays, each consisting of three words (word triads) as described above exposed for a duration of 100 ms, with an inter-display duration of 370 ms (see Figure 1). Participants were asked to read the sentences silently so as to be ready to answer comprehension questions related to sentence meaning, which required a yes/no answer via one of two button presses. The mean percentage of correct answers to the comprehension questions was 90%, showing that participant understood most of the sentences. Although participants were not explicitly asked to ignore flankers, they were told that lateral information was irrelevant for their task performance and were asked to maintain focus on the center of the screen and to avoid eye movements. The short presentation time of the triads (100 ms) prevented that flankers were foveally perceived after lateral eye movements. At the same time, the 470 ms SOA allowed us to analyze most of the standard N400 time window without contamination of the following triad processing. At this presentation rate, lateral movements to fixate irrelevant flankers while missing relevant central information is very disruptive and makes reading difficult. Foveal perception of the flankers would require at least some kind of accurate prediction on when (in which triad) and where (in which flanker) the manipulation is going to take place. However, flanker words were randomly manipulated only in 2/3 of the sentences, and only one flanker was manipulated per sentence at different word positions across sentences. Therefore, it was not possible for participants to predict either when or where the incongruent flanker would appear.

EEG recording and Analysis

EEG was recorded via 29 tin electrodes mounted in an elastic cap and arranged following the 10-10 International System (Fz, Cz, Pz, Fp1/2, F3/4, F7/8, Fc1/2, Fc5/6, C3/4, T7/8, Cp1/2, Cp5/6, P3/4, P7/8, Po3/4, O1/2). These electrodes were referenced on-line to an electrode placed on the left mastoid, and subsequently re-referenced offline to the average activity of both mastoids. Electrooculogram (EOG) was monitored by means of two bipolar montages of electrodes placed at the sub- and supraorbital ridges of the left eye, and on the external canthus of each eye. Impedances were kept below 5 kΩ for mastoid and scalp electrodes, and below 10 kΩ for EOG electrodes. Data were acquired at a sampling rate of 250 Hz. The EEG signal was filtered off-line with a bandpass filter (0.1–30 Hz). Epochs of interest were selected time-locked to the onsets of the word triad presentations (−100 to 1100 ms). Ocular correction for horizontal eye movements was made using the method described by Gratton et al. [8]. Artifacts were removed semi-automatically; segments with electrical activity exceeding ±100 μV at any electrode site were automatically detected, and then confirmed as artifacts by visual inspection. This procedure resulted in an average of 113,3 segments for the congruent condition and 113,4 for the incongruent condition. The participant with the lowest number of segments had 98 in the congruent condition and 99 in the incongruent condition. Finally, after baseline correction (−100 to 0 ms), single-subject ERPs for each condition and electrode were calculated, and were used to obtain grand-average ERPs across participants. The Brain Vision software (www.brainproducts.com) was used for the recording and analysis of the EEG signal, and the R software for all statistical analyses (http://www.r-project.org).

RESULTS

Figure 2 shows grand average ERP waveforms time-locked to the onset of the critical word triad, separately for congruent and incongruent right flanker trials in three right-central-parietal electrodes (C4, CP2, P4). Negative and positive peaks identified as N1, P2 and N400 components were observed for the critical triad and started around 470 ms for the next one. Differences between the congruent and incongruent condition started around 300 ms post-triad onset and lasted for about 200 ms (i.e., the N400 time window). In this time window, incongruent flankers were associated with more negative amplitudes than congruent flankers. The topographical map (figure 3), obtained from the point-by-point subtraction of the ERP activity of the triads including incorrect right flankers from that of the corresponding congruent flanker condition, reveals a right central maximum. In addition, between 300 and 500 ms after the onset of the following word triad (i.e., 770–970 ms after the onset of the critical triad), voltage amplitudes were slightly more positive for the incongruent relative to the congruent condition, especially over left hemisphere sites.

Figure 2.

ERPs at three representative electrodes, and horizontal EOG (uncorrected), showing congruent and incongruent right flankers.

Figure 3.

Topographical distribution of the congruent and incongruent conditions, and the congruent vs incongruent contrast in the analyzed time window: 300–500 ms after the onset critical word-triad.

A subset of 20 electrodes was used for an initial statistical analysis (Fp1/2, PO3/4, O1/2 and midline electrodes were excluded). The mean amplitudes between 300 and 500 ms relative to the onset of the critical triad were subjected to omnibus analyses of variance (ANOVAs) with the factor CONGRUENCY (congruent vs incongruent flanker), and three topographic factors: HEMISPHERE [left (F7, F3, FC5, FC1, T7, C3, CP5, CP1, P7, P3), right (F8, F4, FC6, FC2, T8, C4, CP6, CP2, P8, P4)], LATERALITY [medial (F3, F4, FC1, FC2, C3, C4, CP1, CP2, P3, P4), lateral (F7, F8, FC5, FC6, T7, T8, CP5, CP6, P7, P8)], and ANTERIORITY [frontal (F7, F3, F4, F8), frontal-central (FC5, FC1, FC2, FC6), central (T7, C3, C4, T8), central-parietal (CP5, CP1, CP2, CP6), parietal (P7, P3, P4, P8)]. This analysis revealed a two-way interaction between CONGRUENCY and HEMISPHERE [F(1,17)=12.22; P<0.01], a two-way interaction between CONGRUENCY and LATERALITY [F(1,17)=6.3; P<0.05], and a three-way interaction between HEMISPHERE, CONGRUENCY and LATERALITY [F(1,17)=5.94; P<0.05]. Post-hoc Hochberg comparisons showed significant differences between conditions at right-medial electrodes [t(1,17)=2.24; P<0.05], as well as at right-lateral electrodes [t(1,17)=2.22; P<0.05]. No interactions with the factor CONGRUENCY involving the factor ANTERIORITY were found. However, separate ANOVAs for each electrode (over all the electrodes) showed significant effects of the factor CONGRUENCY mainly in central and posterior electrodes of the right hemisphere (see Table 1), confirming the right central-parietal distribution of the effect, which is consistent with the standard distribution of the N400 component to word reading. In the 770–970 ms time window (i.e., the N400 time window of the following triad), analyses revealed neither a main effect of CONGRUENCY [F(1,17)= 0.74; P=0.39], nor any reliable interactions between CONGRUENCY and any topographic factors (Fs<1). Statistical analyses thus confirmed the central-parietal and right lateralized distribution of the congruency effect for critical triads in the time-window related to the appearance of the N400, but do not support effects in the following triad.

Table 1.

Electrodes for which ANOVAs showed significant effects of congruency in the N400 time window of the critical triad.

		Congruent		Incongruent
	F1,17	mean	SD	mean	SD
Cz	6.08 *	2.07	(1.3)	1.22	(1.45)
C4	4.93 *	1.49	(1.1)	0.76	(1.1)
CP2	6.94 *	1.78	(1.21)	0.91	(1.23)
CP6	8.75 **	0.85	(1.04)	0.23	(1.15)
P4	7.24 *	1.52	(1.08)	0.76	(0.99)
P8	14.21 **	0.96	(0.81)	0.34	(0.61)
PO4	7.48 *	1.39	(1.04)	0.67	(0.86)
O2	7.29 *	1.02	(1.16)	0.37	(0.78)

^*P< .05;

^**P<.01.

Horizontal EOG is also plotted in Figure 2. Differences between the two experimental conditions seen in the lateral eye channels began after the onset of the N400 effect. The onset of the differences between conditions in each electrode was statistically determined by means of point-by-point ANOVAs. One-way ANOVAS were performed for each temporal sample and each electrode separately comparing the congruent and incongruent conditions. The onset of the effects was defined as the first point in a sequence of at least ten consecutive points showing significant differences (P<0.05). While in some posterior electrodes (e.g. P8 and PO4) the onset of the N400 effect was found at 286 ms, significant differences between conditions in the HEOG did not start until 312 ms after triad onset. These data confirmed that differences in the HEOG started simultaneously or even later than the onset of the N400 effect (maybe triggered by the parafoveal information), and much later than the offset of the triad (=100 ms). Our experimental effects thus cannot be attributed to foveal perception of the flankers.

DISCUSSION

The aim of the current experiment was to assess ERP effects associated with parafoveal perception of words when participants read sentences presented one word at a time at fixation (foveally). We found that congruent and incongruent words displayed in the parafovea of the right visual field yielded reliably different ERP responses. In particular, ERP amplitudes were statistically different between 300 and 500 ms after critical triad onsets, with greater negativity for incongruent flankers relative to congruent flankers. Maximum differences were located at central and parietal electrode sites. The characteristics of this effect are consistent with a modulation of the N400 component of the ERP. The N400 is a negative-going component that peaks around 400 ms after the onset of a word or any other potentially meaningful stimulus. N400 amplitude to words within sentences is a linear function of how predictable those words are based on the sentence context up to that point (where predictability is measured by off-line cloze probability measures); the more predictable a word, the smaller the N400 elicited. N400 amplitude thus is often considered as a general index of how well word meaning fits with the ongoing context – word, sentence, or discourse, although is not an index of plausibility per se (for review, see [12]). The largest N400 or priming effects are observed for semantically primed (vs. unprimed) words in the focus of attention, but there is evidence of small but reliable N400 effects even in the absence of attention and/or awareness, as long as the priming stimulus is attended (reviewed in [3]). In the present experiment, congruent right flankers were an acceptable continuation of the sentence up to that point whereas incongruent flankers were not, making them difficult to fit with the meaning of the sentence. These data thus suggest that information extracted from right flankers in the parafovea was available for ongoing sentential semantic level processes, even though they were two degrees lateral to fixation (also see [9]).

In a somewhat similar study, manipulation of the semantic relationship between word pairs presented simultaneously (one word at fixation and the other in the right parafovea) affected ERP amplitudes in the time range of the P2 (200–230 ms) component [1]. Similarly, in a related study by Barber et al. [2] in which all but an occasional target word were flanked by pseudowords, the semantic manipulation of flanker words also modulated P2 amplitudes. Semantic effects have been reported in both the P2 and N400 time windows (e.g., with the lateralized presentation of a target after a foveally presented sentence context; e.g. [7]); the functional distinction between these effects is not clear, though Federmeier et al. [7] interpreted both as a sign of contextually-based prediction – the P2 at the level of visual form and the N400 at the level of meaning.

Taken as a whole, our results are consistent with some involvement of semantic processes under circumstances that resemble aspects of natural reading [see also 13]. Additional research is needed to determine if words identified in the parafovea are fully processed or if, alternatively, only formal features (e.g. length or initial letters) are used to confirm contextually-anticipated items. This issue is closely related to the absence of any reliable ERP effects to the triad occurring immediately after the critical triad, where eye-tracking studies have typically reported facilitation of fixated words after parafoveal preview [15].

In summary, our study provides ERP evidence for parafoveal perception during word-by-word sentence reading in the absence of contaminating eye movements. Semantically incongruent (with respect to the ongoing sentence context) right flanker words were associated with more negative amplitudes in the time range of the N400 component of the ERP than when these same words served as semantically congruent right flankers. These data show that at least during this mode of reading some linguistic information can be extracted from the parafovea and quite quickly integrated with the evolving representation of the sentence meaning.