Abstract
During natural reading, parafoveal information is processed to some degree. Although isolated words can be fully processed in the parafovea, not all sentence reading experiments have found evidence of semantic processing in the parafovea. We suggest a possible reconciliation for these mixed results via two ERP studies in which volunteers read sentences presented word by word at fixation, flanked bilaterally by the next word to its right and the previous word to its left. Half the words in the right parafovea of critical triads and in the fovea for the subsequent triad were semantically incongruent. The conditions under which parafoveal words elicit canonical visual N400 congruity effects suggest that they are processed in parallel with foveal words, but that the extraction of semantic information parafoveally is a function of contextual constraint and presentation rate, most likely under high contextual constraint and at slower rates.
Keywords: Reading, parafovea, parafoveal-on-foveal effect, preview benefit, ERPs, N400
Introduction
Scanning printed text involves eye movements called saccades, but words are perceived only during fixations — a period of relative steadiness between saccades when a reader's gaze lands on a specific word. However, not every word in a text is fixated; many words (especially short ones, and/or function) are skipped (e.g., only 23% of 2-3 letter words and 35% of function words are fixated), and thus presumably perceived only parafoveally (Brysbaert, Drieghe & Vitu, 2005). Indeed, reading is reliably slower (20-40 ms/word) when parafoveal information is not available than when it is (Sereno & Rayner, 2003). This raises questions about the nature of information extracted from the parafovea and integrated with foveal information during any given fixation.
Parafoveal perception seems to influence reading in different ways (see review in Schotter, Angele & Rayner, 2012). Information in upcoming words not-yet-fixated can guide saccadic programming (e.g., where to move the eyes next and which words to skip). When a word is fixated, if the upcoming word in the parafovea is perceived, and then subsequently fixated at the end of the next saccade, that word's (now foveal) processing might be facilitated due to its parafoveal preprocessing; this is known as a preview benefit. A more controversial effect of parafoveal perception is the putative parafoveal influence (of the upcoming word) on the processing of the currently fixated word; this is known as the parafoveal-on-foveal effect. As words carry information at different levels of representation, a question pertinent to a full understanding of parafoveal perception in reading is the extent to which the various representational levels of a word perceived in the parafovea can be processed.
Many studies have shown that isolated words can be fully processed even when they appear in the parafovea (Bouma, 1970), demonstrating that reduced visual acuity in the parafovea per se does not prevent word processing. The situation during normal reading is somewhat different, however. During sentence reading, foveal and parafoveal information are simultaneously available and word recognition occurs along with higher-level linguistic operations (e.g. semantic and syntactic integration) as well as eye movement control. Research based on the monitoring of eye movements has provided valuable information on what kind of information is processed under these circumstances. Parafoveal preview benefits, for instance, have been explained in terms of the preprocessing of orthographic and phonological codes, but, at least in alphabetic languages, semantic information does not seem to contribute to these effects (Rayner, Balota, and Pollatsek, 1986; Altarriba, Kambe, Pollatsek, and Rayner, 2001; but see Yan, Richter, Shu & Kliegl, 2009; and Yan, Zhou, Shu, Kliegl, 2012 for evidence in Chinese). Hohenstein et al. (Hohenstein, Laubrock & Kliegl, 2010), however, observed semantic preview effects when limiting the time available for parafoveal preview (i.e. 125 ms). They proposed that unlimited preview presentations (as has been the case in most experiments) could interfere with lexical access of the target word. Evidence of semantic parafoveal-on-foveal effects has been obtained mainly with artificial tasks (e.g., Murray, 1998). Similar results from natural reading (e.g., Inhoff, Radach, Starr, and Greenberg, 2000) have been proven difficult to replicate (see reviews in Rayner, White, Kambe, Miller, & Liversedge, 2003; and Drieghe, 2011). Thus, although words in the parafovea can sometimes be processed at a semantic and/or pragmatic level, it is an open question if this happens in normal reading at all, or if so, under what circumstances.
Although electrophysiological research on this topic is relatively scarce, there are a few relevant reports. As in the behavioral and eye movement literatures, ERP experiments have consistently shown that isolated words presented in the parafovea (albeit in the context of a prime word or sentence presented word by word at the fovea) can be semantically processed and their meanings quickly integrated with the preceding context (Federmeier & Kutas, 1999; Coulson, Federmeier, Van Petten & Kutas, 2005). When parafoveal stimuli co-occur with foveal stimuli, the conclusion from ERPs is less clear. One approach to examine this issue has been to simultaneously record eye-movements and ERPs to obtain fixation-related potentials or FRPs (reviewed in Dimigen, Sommer, Hohlfeld, Jacobs, & Kliegl 2011). Baccino and Manunta (2005), for instance, recorded FRPs to word pairs, one at fixation and the other in the right parafovea, and found effects contingent on the lexical status (word/non-word) of the parafoveal stimulus, as well as the associative relationship between the two words. Using a similar design, Simola and colleagues (Simola, Holmqvist, & Lindgren, 2009) also obtained the lexical effects for target words in the right parafovea but not the semantic effects. Kretzschmar et al. (Kretzschmar, Bornkessel-Schlesewsky, & Schlesewsky, 2009) examined FRPs to sentence final words and reported a dissociation between foveal and parafoveal processes: predictability affected the foveal processing of the critical word while semantic relatedness with the prior context modulated the responses to the last fixation before the critical word (parafoveal effect). The interpretation of this parafoveal effect must be viewed with caution, however, as there was no procedure for disentangling it from early foveal effects associated to the last fixation. This limitation was overcome in a recent study in which parafoveal previews were manipulated via a gaze-contingent display change (Dimigen, Kliegl, & Sommer, 2012). In this study participants read five nouns from left to right as in normal reading. Target nouns were preceded by parafoveal previews that were either: a) identical, b) semantically related but orthographically unrelated or c) completely unrelated to the target nouns. FRPs were recorded to target and pre-targets words. While no parafoveal-on-foveal effects were observed, a temporo-occipital positivity between 200 and 280 ms differentiated the identity preview from the two other conditions. Moreover, no differences were observed between the two orthographically unrelated conditions suggesting that word meaning did not contribute to the preview effects.
In our own work, we have recorded ERPs to sentences presented word-by-word at fixation, each flanked two degrees bilaterally by letter strings or a word (Barber, Doñamayor, Kutas & Münte, 2010; Barber, Shir, Bentin, & Kutas, 2011). In one study, each fixated word was flanked to its right by the next word in the sentence to its right, and to its left by the previous word (Barber et al., 2010). Occasionally, one of the flanker words was semantically incongruous with the sentence context. Triads with semantically congruent flankers elicited smaller N400s than triads with incongruent flankers. As N400 amplitude modulations index lexico-semantic processing (see review in Kutas & Federmeier, 2011), we took these results as evidence that, under these circumstances, some parafoveal information was incrementally integrated with the sentence representation and influenced processing of word at fixation. Obviously, reading in this modified version of the rapid serial visual presentation (RSVP) differs in several ways from normal reading. It does, however, allow us to minimize the influence of some factors (e.g. eye movement control) and keep that of others constant (e.g., location of the parafoveal information), while we manipulate various factors that could modulate parafoveal word processing.
To sum up, both eye-tracking and electrophysiological data provide some evidence of semantic level analysis of parafoveal stimuli, at least under favorable conditions. What remains unclear, however, is which variables render parafoveal semantic effects more or less elusive during natural reading. As reading is a highly complex skill involving many inter-linked components unfolding over time, we hypothesized that the nature and extent of parafoveal processing may be a function of the availability of cognitive resources and the time to use them, and that it is variation in these resources that would help to explain the seemingly contradictory findings in the literature. There is some empirical evidence consistent with this general claim. Parafoveal processing can be modulated by the difficulty of the concurrent foveal processing. Preview benefits, for example, are less when the fixed word is infrequent compared to frequent (Henderson & Ferreira, 1990; Schroyens, Vitu, Brysbaert, & d'Ydewalle, 1999). In our experiments we take a different tack to examining how cognitive load might modulate parafoveal word processing.
Using a bilateral flanker procedure (Barber et al. 2010, 2011), we have asked participants to read sentences comprised of the central words of triads, appearing at a relatively slow rate (470 ms stimulus-onset-asynchrony, or SOA from the beginning of one triad to the next). This rate clearly is not representative of natural reading rates. We reasoned that such long intervals between bouts of word processing can lessen cognitive load, enabling cognitive operations (or strategies) that might (or could) not otherwise have been deployed under the more stringent time constraints of natural reading. Likewise, we reasoned that word predictability within a given context also could modulate cognitive load and in turn impact parafoveal processing. In principle, a word's predictability (contextual expectancy) might influence the requisite cognitive resources. Word recognition occurs more readily and quickly for highly predictable words, thereby potentially freeing up resources for other operations (such as parafoveal processing). During normal reading contextually-predictable words are not only fixated for shorter durations (Ehrlich & Rayner, 1981) but are more often skipped altogether than contextually less predictable words (Balota, Pollatsek, & Rayner, 1985). It was these sorts of considerations that led us to conduct two ERP experiments aimed at assessing the potential roles of reading rate and word predictability on parafoveal-on-foveal effects in the context of this word triad paradigm.
We used an RSVP with bilateral flankers (i.e., word triad) procedure: words that were semantically congruent or semantically incongruent within the ongoing sentential context appeared either as the right flanker of a critical triad, and/or in the center position (at fixation) of the subsequent triad (see Figure 1 in the Method section). Half the target words were highly contextually predictable and half were less predictable (sentential constraint and word predictability were both measured via offline cloze probability ratings). Triad presentations was either relatively slow (SOA=450 ms, Expt. 1), as in our prior experiments, or approximated normal reading rates (SOA=250 ms, Expt. 2). We predicted that under more optimal conditions (long SOAs and high contextual constraint), words in the parafovea would be semantically processed and integrated with the sentential context, as indexed by an N400 congruity effect. Conversely, when cognitive resources are more limited (such as with shorter SOAs and/or weaker contextual constraint), we predicted that parafoveal processing effects would be diminished if not absent.
Method
Two nearly identical experiments, varying only in SOAs were conducted as described below.
Participants
Twenty-seven undergraduates of the University La Laguna (15 female, 18-38 years of age, mean age 24) participated in Experiment 1, and twenty-three (13 female, 17-29 years of age, mean age 22) in Experiment 2, for course credits or monetary compensation. Non-native Spanish speakers and individuals with a history of neurological or psychiatric impairment were excluded. All participants were right-handed according to an abridged Spanish version of the Edinburgh Handedness Inventory (Oldfield, 1971; LQ>50).
Materials
Stimuli consisted of 304 Spanish sentences, each 9-18 words in length. Target words in half the sentences were preceded by high contextual constraint, and in the other half by low contextual constraint, based on a normative study with 50 University La Laguna students who did not participate in the ERP experiments. Target word cloze probability was based on the percentage of individuals who continued the context with that word: low cloze words had probability of 0.4 or less (mean=10.3; SD=15.5) and high cloze probability words had a cloze probability of 0.7 or more (mean=89.7; SD=9.3). Target words appeared somewhere between position 6 to 15 in the sentence and were followed by an additional 3 to 7 words. Half the target words were replaced with words that were semantically incongruent in the ongoing context. Across consecutive triads, each word of the sentence was presented in three different positions: left parafovea, centre (fixation), or right parafovea. The congruity manipulation was implemented in the right parafovea, in the centre position, or in both, yielding four experimental conditions (see also Figure 1):
Parafovea congruent – Fovea congruent (PC-FC)
Parafovea congruent – Fovea incongruent (PC-FI)
Parafovea Incongruent – Fovea congruent (PI-FC)
Parafovea Incongruent – Fovea incongruent (PI-FI)
The number of letters of the targets words ranged from 4 to 9 letters in all conditions. Congruent and incongruent target words in high and low constraint sentences were matched on multiple lexical features using the LEXESP database (Sebastián-Gallés, Martí, Carreiras, & Cuetos, 2000). A one-way ANOVA conducted on the two constraint lists confirmed no reliable differences on these lexical factors for all congruent and incongruent sentences (see Table 1). As we wanted to facilitate parafoveal processing, foveal load was kept low in all conditions by using short (2-7 letters) and relatively frequent words in the pre-target position [Length: Low constraint mean=3.08 (SD=0.28; range=2-7), High constraint mean=3,64 (SD= 1.36; range= 2-7); Lexical frequency: Low constraint mean=8244,19 (SD=5153,7); High constraint mean=6821,42 (SD=4175,69]. Target words were all nouns, whereas the grammatical class of the pre-target words included determiners, adverbs, pronouns and adjectives (equally distributed across conditions). Each participant read each sentence only once. Across participants all sentences appeared in all four conditions counterbalanced in four different lists. Sentences appeared in a different random presentation order for each participant.
Table 1.
Target words | lexical frequency* | orthographic neighborhood | word length | imagineability | concreteness | bigram frequency* |
---|---|---|---|---|---|---|
High cloze - Congruent | 21.7 (37.6) | 3.1 (4.1) | 5.8 (1.4) | 4.1 (2.8) | 4.1 (2.7) | 738.1 (999.2) |
High cloze - Incongruent | 21.0 (35.5) | 3.0 (3.8) | 5.8 (1.4) | 3.9 (2.5) | 3.9 (2.5) | 626.6 (708.8) |
Low cloze - Congruent | 24.5 (57.2) | 1.5 (2.4) | 6.7 (1.6) | 2.9 (3.1) | 2.8 (2.9) | 610.8 (572.0) |
Low cloze - Incongruent | 25.2 (54.8) | 1.3 (2.2) | 6.7 (1.6) | 2.8 (3.0) | 2.8 (2.9) | 515.7 (703.6) |
= frequency per million.
Procedure
Sentences were presented as a series of word triads, with the (fixated) word at the center of the screen flanked 2 degrees bilaterally by the next word in the sentence to its right, and the previous word in the sentence to its left (see Figure 1). Words were presented in Courier New, a non-proportional font in which each letter occupies the same amount of space. Since 3 characters corresponded to 1 degree of visual angle, all the target words in the parafovea were displayed between 2 and 5 degrees. Participants were seated in an electrically shielded room 120 cm in front of a 17-inch computer screen; word triads were presented in white lowercase letters against a black background via Presentation (Version 0.70, http://www.neurobs.com). Each trial began with a fixation cross (+) displayed for 1500 ms, followed by a 300 ms blank interval, followed by the word triad for 100 ms. The short duration of the word triad presentation was intended to force participants to fixate the center word and to eliminate lateral eye movements1. The two experiments differed in triad presentation rate: the inter-stimulus interval (ISI) was 350 ms in Experiment 1 (SOA=450 ms) and 150 ms (SOA=250 ms) in Experiment 2. The interval between sentences varied randomly between 250 and 750 ms. Participants were asked to silently read the sentences and to answer yes-no questions about their meaning. In Experiment 1, half the sentences were questioned; in Experiment 2, all sentences were questioned to confirm comprehension given the fast presentation rate. In both experiments half of the questions were related to the target word. For example, the sentence “Mary bought her new bike last week because it was on sale.” (María compró su nueva bicicleta la semana pasada porque estaba rebajada.), in which “bike” (bicicleta) is the target word, was followed by the question: “Did Mary buy a new jacket?”; the sentence “The twin brothers watched the television comfortably seated on the living room couch.” (Los gemelos miraban la televisión cómodamente sentados en el sofá de la sala.), in which the target word is “couch” (sofá), was followed by the question: “Were the twin brothers sitting on the carpet?”. The remaining questions queried other parts of the sentence, words before or after the target. For example the sentence “After that rainy day, the gardener had to ask for other boots and throw the old ones away.” (Después de aquel día de lluvia, el jardinero tuvo que pedir otras botas y tirar las viejas.), in which “boots” (botas) is the target word, was followed by the question: “Were John's boots ruined in the snow?”). The mean percentage of correct answers was 87% (SD=10.8) in Experiment 1, and 91% (SD=5.71) in Experiment 2. Participants were asked to avoid muscle and eye movement and blinking whenever word triads were on screen. The experiments started with 12 practice trials and were divided into four blocks of 76 sentences each, with a short rest period between blocks.
EEG Recording
EEG was recorded via 27 Ag/AgCl electrodes (Figure 2) in an elastic electrode cap (Easycap www.easycap.de) referenced to the left mastoid. Eye movements and blinks were monitored via four additional external electrodes, two at the outer canthus of each eye and two at the infra-orbital and supraorbital regions of the right eye, thereby providing bipolar recordings of the horizontal and vertical electro-oculogram (EOG). All electrical activity was recorded and amplified with a bandwidth of 0.01–100 Hz, and sampled at 250 Hz. Impedance was 5 kΩ or for all electrodes except the EOG channels (kept below 10 kΩ). EEG was recorded, stored and processed using BrainVision software (www.brainproducts.com). The recorded data were filtered offline with a bandpass 0.1-30 Hz, and re-referenced to the algebraic mean of the activity at the two mastoids. Artifacts were removed semi-automatically with rejection values adjusted for each participant (excluding epochs with amplitudes of more than 70-100 μV in a 200 ms period). This procedure led to the exclusion of ∼12% of the trials (evenly distributed across experimental conditions) in Experiment 1, and 8% in Experiment 2. The recordings of 5 participants in Experiment 1 and 8 participants in Experiment 2 were corrected for blinks (Gratton, Coles, & Donchin, 1983).
ERP analyses
Epochs of interest were time-locked to the critical word-triad presentations, from -100 to 1100 ms for Experiment 1, and from -100 to 900 ms for Experiment 2 and averaged for each condition, electrode, and participant. Triads from sentences for which the comprehension questions were answered incorrectly were not included in the average ERPs or in the statistical analyses. N400 amplitudes were measured for the critical triad ERPs to the parafoveal (Experiment 1: 300-550 ms; Experiment 2: 350-450 ms) and foveal responses, i.e., the following triad (Experiment 1: 750 to 1000 ms; Experiment 2: 600 to 800 ms) 2. Additional analyses encompassed the interval between the two N400 time windows (Experiment 1: 550-750 ms; Experiment 2: 450-600 ms). ERP mean amplitudes for the parafoveal and foveal conditions were subjected to separate ANOVAs. ANOVAs of the N400 for the parafoveal manipulation included Congruity-parafovea (congruent, incongruent), Constraint (high, low) and three topographical factors (see Figure 2): Hemisphere (left, right), Laterality (medial, lateral), and Anteriority (frontal, fronto-central, central, centro-parietal, parietal). Third time window analyses included an additional Congruity-fovea (congruent, incongruent) factor. Statistical analyses were performed with R (http://www.r-project.org). Reported results include the original degrees of freedom and p-values corrected for violation of sphericity using the Greenhouse-Geisser epsilon (ε). For post-hoc testing of the significant interactions with the critical factors Constraint, Congruity-parafovea or Congruity-fovea we report the Hochberg p-value.
Results
Figure 3 (upper row) shows grand average congruent versus incongruent ERPs for the parafoveal (right) and foveal (left) conditions at CP2, a right centro-parietal site where the visual N400 effect is typically largest. As expected, the congruence manipulation in the fovea resulted in a N400 effect starting ∼300 ms after stimulus onset (∼750 ms in the ERPs shown). Likewise, ERP differences associated with the semantic incongruence manipulation (in the right parafovea) are manifest in the N400 time window to the critical triad (300-550 ms after the triad onset). This effect continues until 750 ms, i.e., 300 ms after the onset of the subsequent triad. The foveal-based and parafoveal-based N400 effects are both evident under high and low contextual constraint; the N400 effect seems slightly larger with the foveal manipulation (see Figure 3). The ERPs in the high contextual constraint conditions of Experiment 2 (Figure 4, middle row) resemble those in Experiment 1: there are N400 congruity effects for both the parafoveal and the foveal manipulations, starting ∼50 ms later and slightly smaller in size than at the slower presentation rate of Experiment 1. By contrast, in the low contextual constraint conditions of Experiment 2 (bottom row), there is an N400 congruity effect only to the foveal, not the parafoveal congruence manipulation (this is also clear in the difference ERPs for both experiments in Figure 5). Both foveal-based and parafoveal-based N400 effects are characterized by the standard visual N400 centro-parietal topography (Figure 6). As evident in Figure 7, there is no apparent effect of the parafoveal preview on the subsequent foveal N400 effect at either presentation rate. Finally, Figure 8 shows the mean activity of the horizontal EOG by condition and experiment. No reliable differences between conditions are observed within the time interval during which the critical triads were displayed (i.e., the first 100 ms after stimulus onset).
Experiment 1
Since the parafoveal effects overlapped with the onset of the foveal N400 to the next triad, we statistically evaluated the effects in three time windows for Experiment 1:
300 to 550 ms time window
This ANOVA (Congruity-parafovea × Constraint × Hemisphere × Laterality × Anteriority) yielded a main effect of Congruity-parafovea (F(1,27)=23.9; p<0.001), two-way interactions of Congruity-parafovea with Hemisphere (F(1,27)=5.7; p<0.05) and Laterality (F(1,27)=21.2; p<0.001), and a three-way interaction of Congruency-parafovea, Hemisphere and Laterality (F(1,27)=5.7; p<0.05). Post-hoc tests showed that the congruity effect was significant at all locations (p<0.05). Overall, the congruity manipulation produced a large N400 effect with a widespread distribution extending to the outer areas of the scalp.
550 to 750 ms time window
This ANOVA yielded main effects of Congruity-parafovea (F(1,27)=12.5; p<0.01) and Constraint (F(1,27=5.5; p<0.05). There were also two-way interactions of Congruity-parafovea with Hemisphere (F(1,27)=6.8; p<0.05), Laterality (F(1,27)=14.3; p<0.001) and Anteriority (F(4,108)=4.1; p<0.01; ε=0.31), a three-way interaction of Congruity-parafovea and Hemisphere with Laterality (F(1,27)=9.9; p<0.01). Post-hoc tests revealed differences between congruent and incongruent conditions over both hemispheres; the effect was significant bilaterally for sites close to the midline (p<0.01), but only over the right lateral sites (p<0.01). Constraint also interacted with Laterality (F(1,27)=4.5; p<0.05), with contextual constraint differences most pronounced at more medial sites (p<0.05). The statistical data confirm that in this time window there was an effect of constraint with a narrow central distribution, and a continuing congruity effect with a central right distribution. This effect spans the lateral right frontal and parietal sites.
750 to 1000 ms time window
This ANOVA (Congruity-fovea × Congruity-parafovea × Constraint × Hemisphere × Laterality × Anteriority) yielded main effects of Congruity-fovea (F(1,27)=30.5; p<0.001) and Constraint (F(1,27)=5.3; p<0.05). There were two-way interactions of Congruity-fovea with Constraint (F(1,27)=10.5; p<0.01), Congruity-fovea with Hemisphere (F(1,27)=5.3; p<0.05), Congruity-fovea with Laterality (F(1,27) =20.0; p<0.001), and Congruity-fovea with Anteriority (F(4,108)=7.4; p<0.001; ε=0.39). There also were four three-way interactions of Congruity-fovea with Constraint and Laterality (F(1,27)=6.35; p<0.05), Congruity-fovea with Hemisphere and Laterality (F(1,27)=10.15; p<0.01), Congruity-fovea with Hemisphere and Anteriority (F(4,108)=3,45; p<0.05; ε=0.45), and Congruity-fovea with Laterality and Anteriority (F(4,108)=4.83; p<0.01; ε=0.62). Post-hoc tests of these interactions revealed significant congruity effects at all electrodes (p<0.001). The congruity effect was statistically reliable for both constraint conditions (p<0.001), although numerically larger for high (2.8 μV) than low constraint (1.5 μV). For congruent conditions, the constraint effect was reliably larger for high than low constraint conditions (p<0.01). Recapitulating, the congruity manipulation yielded a large N400 effect with a widespread distribution across the scalp for low as well as high constraint sentences, albeit larger for the latter. This difference reflected a more positive going waveform to the congruent words of the high constraint condition.
N400 effect onset analysis
Consecutive t-tests (every 4 ms) comparing congruent versus incongruent ERPs between −100 to 1100 ms were used to determine the statistical onset of the N400 effect. Defined as a significant difference at three consecutive data-points, the parafoveal N400 effect began around 312 ms, and the foveal N400 effect began at 302 ms (752 ms, in the critical triad ERP).
Experiment 2
Statistical analyses paralleled those in Experiment 1, but were delayed by 50 ms, and shorter in duration given the faster presentation rate.
350 to 450 ms time window
This ANOVA (Congruity-parafovea × Constraint × Hemisphere × Laterality × Anteriority) yielded a two-way interaction of Congruity-parafovea with Hemisphere (F(1,22)=7.95; p<0.01) and a three-way interaction of Congruity-parafovea with Hemisphere and Laterality (F(1,22)=8.41; p<0.01). Post-hoc tests revealed significant congruity effects at all right lateral sites (p<0.05). The Congruity-parafovea by Constraint interaction was marginal (F(1,22)=4.08; p=0.056). We thus conducted separate ANOVAs (Congruity-parafovea × Hemisphere × Laterality × Anteriority) for the high and low constraint conditions. For high constraint conditions, this yielded a main effect of Congruity-parafovea (F(1,22)=5.71; p<0.05), an interaction of Congruity-parafovea with Laterality (F(1,22)=4.94; p<0.05), and a three-way interaction of Congruity-parafovea with Hemisphere and Laterality(F(1,22)=5.66; p<0.05). The N400 congruity effect under high constraint showed a broad distribution, but larger at right-central areas. By contrast, for low constraint conditions there were no significant effects including the Congruency-parafovea factor. There was no main effect of Congruity-parafovea (F(1,22)=0.02; p=0.90); the three-way interaction of the factors Congruity-parafovea, Hemisphere and Laterality came the closest to a significant result (F(1,22)=3.68; p=0.07), with the rest of the interactions involving the factor Congruity showing F's < 2.99 and p's > 0.098.
450 to 600 ms time window
This ANOVA yielded a main effect of Congruity-parafovea (F(1,22)=6.99; p<0.05) and Constraint (F(1,22)=4.89; p<0.05), and an interaction between them (F(1,22)=6.88; p<0.05), as well as of each with Laterality (F(1,22)=5.36; p<0.05; and F(1,22)=9.36; p<0.01 respectively). There also were three-way interactions of Congruity-parafovea with Constraint and Laterality (F(1,22)=4.45; p<0.05), and Congruity-parafovea with Constraint and Anteriority (F(4,88)=5.40; p<0.001; ε=0.41). Post-hoc testing revealed a significant congruity effect for the high constraint conditions at centro-parietal (p<0.05), and more medial sites (p<0.05). In this time window there was there was a reliable effect of constraint with a right-central distribution extending to the right frontal and right central parietal sites, and a centro-parietal congruity effect for high constraint sentences.
600 to 800 ms time window
This ANOVA (Congruity-fovea × Congruity-parafovea × Constraint × Hemisphere × Laterality × Anteriority) yielded main effects of Constraint (F(1,22)=12.04; p<0.01), and Congruity-fovea (F(1,22)=68.35; p<0.001), and an interaction between them (F(1,22)=4.4; p<0.05). There were two-way interactions of Constraint with Laterality (F(1,22)=11.76; p<0.01), Congruity-fovea with Laterality (F(1,22)=27.19; p<0.001), and Congruity-fovea with Anteriority (F(4,88)=17.15; p<0.001; ε=0.34). There were three-way interactions of Constraint with Hemisphere and Laterality (F(1,22)=11.06; p<0.01), and Congruity-fovea with Laterality and Anteriority (F(4,88)=9.82; p<0.001; ε=0.72). Post-hoc tests showed widespread congruity effects under both high constraint (p<0.001) and low constraint (p<0.001), with a significant difference between them only for the congruent responses (p<0.01). The effect of contextual constraint was visible at medial sites bilaterally (p<0.01) and at lateral sites over the right hemisphere (p<0.01). Recapitulating, the constraint manipulation produced an N400 effect with a canonical right centro-parietal distribution. As in experiment 1, the significantly more positive going waveforms in the congruent condition under high constraint led to larger N400 amplitude differences for the high than low constraint conditions.
Discussion
We conducted two experiments to determine whether or not a word in the right parafovea (2-5 degrees to the right of a fixated word) can be processed at a semantic level as a word at fixation also is being processed, and if so, whether their processing is modulated by cognitive load. To that end, we recorded electrical brain activity as participants read sentences presented word by word at fixation (foveal), flanked to the left by the previous word in the sentence and to the right by the next word (parafoveal) in the sentence. Critical words were occasionally semantically incongruous either in the right parafovea, at the fovea (the subsequent word), or both. Our index of semantic processing was the N400 congruity effect (relatively larger centro-parietal negativity ∼300-600 ms post word onset region in the ERP to contextually incongruous versus congruent words) triggered by a congruity manipulation at fixation and/or parafoveally. Given that experiments on relatively normal reading and those using more artificial reading tasks have come down on different sides of this issue, we varied two factors known to influence the ease or difficulty of word recognition during reading that we thought might facilitate a rapprochement: contextual constraint and word presentation rate (i.e., stimulus onset asynchrony or SOA). Contextual constraint was high or low and sentence words appeared at a fast rate approximating normal reading (250 ms SOA in Experiment 2) or at a slower rate (450 ms SOA in Experiment 1).
Our results are clear. As expected, the congruity manipulation at the fovea elicited an N400 effect (greater negativity for incongruous than congruous words) at both presentation rates regardless of contextual constraint. Consistent with the extant ERP literature, the N400 congruity effect was (numerically) larger for high than low contextual constraint, due to greater positivity in the N400 time window to higher (versus lower) cloze probability words in the congruent condition. This pattern of N400 amplitude modulation is evidence of lexico-semantic analysis and contextual integration of the critical word at fixation (i.e., foveally).
At issue, however, was whether parafoveal words are similarly semantically processed and integrated with the ongoing message. Based on the same logic with respect to the pattern of parafoveal N400 effects, the answer is a qualified yes, for reasons that we believe might help to explain the mixed results in the preview literature. N400 effects to the parafoveal congruity manipulation were observed at the slow presentation rate under both low and high sentential constraint and at the fast presentation rate only under high (but not low) constraint. In sum, under the right conditions, words in the right parafovea can be semantically integrated into the sentence representation; under the wrong conditions, they seem not to be. In two experiments, we demonstrate that presentation rate (especially slow) and degree of contextual constraint (especially high) both contribute to creating the right conditions, at least in this type of experimental reading setup.
Parafoveal semantic effects can emerge from semantic processing of the parafoveal information or from the use of formal features (e.g. length or initial letters) to corroborate contextually-anticipated items. In Experiment 1, N400 parafoveal effects under low constraint conditions indicate that parafoveal words were semantically processed, their meaning activated apparently with little support from the preceding context. In addition, the larger size of the congruity effect under high constraint suggests that lexical predictions likely contributed. The pattern of results in Experiment 2 indicates that less information is extracted from the parafovea when time is relatively more limited. The presence of an N400 parafoveal effect only under high constraint could reflect a) contextual facilitation of the extraction of lexical information from the parafovea, and/or b) the use of partial word information to confirm lexical predictions.
It is noteworthy that the foveal and parafoveal N400 congruity effects are similar in morphology, topography, and onset latency, differing only in overall amplitude (larger for foveal than parafoveal congruity). Our N400 results thus provide no evidence that the qualitative nature of the semantic analyses at fixation and parafoveally differs in any way – quite the opposite. Moreover, whereas the faster rate of word presentation delays the N400 effect by approximately 50 ms, the foveal versus parafoveal congruity manipulation has no statistically reliable effect on the onset of the N400 congruity effect. At least under some circumstances, then, a word at fixation and another in the right parafovea - artificially farther laterally than in most natural reading situations - can nonetheless both be processed at the semantic level seemingly in parallel.
Overall, our results demonstrate that extracting semantic information from the parafovea is not limited either by (reduced) visual acuity or the scope of visuo-spatial attentional processes per se. The limiting resources are apparently cognitive in nature. Given sufficient perceptual/cognitive resources, readers seem able to process not only the fixated word but another in the right parafovea at a semantic level, and integrate both with the ongoing sentence context. Concurrent evidence supporting this idea comes from eye-tracking studies showing that increasing the difficulty of foveal processing reduces the amount of parafoveal processing (Henderson & Ferreira, 1990; Schroyens et al., 1999). Whereas we show that high contextual constraint and more time between words both allow semantic level analysis of parafoveal content (at least in this paradigm), we expect other variables known to facilitate word processing to similarly enable parafoveal semantic analysis, either alone or in combination. Among the likely candidates that future research could assess are grammatical class, word length, lexical frequency, syntactic predictions, semantic association, etc.
At a practical level, our hypothesis that the extent to which a parafoveal word is processed semantically is a function of the available processing resources may offer grounds for reconciliation of contrary results in the literature. According to our hypothesis, parafoveal-on-foveal semantic effects in eye movement experiments with normal reading have not been particularly robust and have proven difficult to replicate (Rayner et al., 2003) because fewer resources were available to do so than in more artificial reading tasks including ERP studies with long inter-word intervals. Results from Experiment 1 are consistent with reports showing that the meaning of a word to the right of fixation influences how long readers look at a fixated word (e.g., Murray, 1998), as well as with ERP findings for isolated word pairs (Baccino & Manunta, 2005). Also consistent with this hypothesis is Simola et al.'s (2009) failure to replicate the semantic effects of Baccino & Manunta in a slightly more complex reading task (i.e. parafoveal words appearing either in the left or in the right parafovea). A similar logic might explain the absence of semantic effects in Dimigen et al. (2012), who used a resource demanding task which more closely approximated normal reading.
It goes without saying that in our experiments did not involve natural reading. Indeed, there are important differences between the way in which our participants “read” sentences and natural reading. In our procedure, for example, sentences were read absent eye movements at a fixed presentation rate, the parafoveal was not followed (in space) by another word, and the distance between the word at fixation and that in the parafovea was not only variable but sometimes unusually large, thereby less subject to lateral masking. However, our experimental procedure allows us to isolate a basic processing mechanism that we believe may be operating but obfuscated or hidden during the processing complexities and overlap of natural reading. At least within the present paradigm we have shown that parafoveal processing at a semantic level under relatively optimal conditions is modulated by contextual constraint/word predictability as well as by the time available for processing. We suggest that it is reasonable to propose that these variables (in combination with other factors not yet empirically tested) also may influence the nature and extent of parafoveal processing effects during natural reading.
We conclude with some final observations and a speculation. In these two experiments, the N400 effect elicited by the congruity manipulation at fixation was not measurably affected by the congruity manipulation of the immediately preceding word (in the parafovea). Even though the parafoveal word was semantically apprehended, it did not appear to impact the amplitude N400 congruity effect when that same (previously parafoveal) word re-appeared or not at fixation, regardless of its contextual fit. At the same time, however, ERP effects due to the parafoveal congruity manipulation were long-lasting, and did overlap with the beginning of the subsequent triad's processing (albeit not reliably modulating its N400). More specifically there was no reliable repetition effect when the same word (congruous or incongruous) was repeated at the fovea – i.e., no preview benefit. This is at odds with the robustness of orthographic preview benefit effects in the eye-tracking literature (Schotter et al., 2012), and with a recent study reporting FRPs consistent with this type of parafoveal-foveal facilitation during normal reading (Dimigen, et al., 2012). There are a number of possible explanations for this discrepancy. It is well established that priming effects are quite sensitive to the SOAs between a prime and target. A MEG study, for instance, using word pairs in which the target word appeared at fixation and the prime word in the parafovea reported repetition priming effects with a 237 ms SOA (even shorter than our 250 ms SOA), but not with a longer SOA of 287 ms (Pernet, Uusvuori & Salmelin, 2007). Among other potential explanations, the fixed presentation rate and/or the SOAs we used could account for why we did observe any preview effects.
Acknowledgments
HAB was funded by the grant PSI2010-19767 of the Spanish Ministry of Science. MK was supported by the grant HD22614 from the US National Institute of Child Health and Human Development, and the grant AG08313 from the National Institute of Aging. We thank Javier López and Samara Muñoz for their help in data collection.
Footnotes
Although eye movements were not monitored via an eye tracker, and thus we cannot totally dismiss the possibility that participants made a saccade from the center to the right periphery and/or chose to fixate between the foveal and right parafoveal word, we believe that this was unlikely. First, fixating on the word the centre of the screen in this procedure is encouraged both by the location (central and bilaterally peripheral at 2 degrees) of the word stimuli and the short duration of the stimulus presentation (100 ms). Independent testing in our lab showed that 100 ms was insufficient for participants to be able to read the word in the fovea, and then saccade to the word in right periphery and read it. Moreover, when participants were queried post experiment about the goals of the experiment (i.e., what do you think we were manipulating in this experiment? What do you think is the goal of this study?), none of them mentioned the parafoveal manipulation. In fact, if anything it seemed that they were most concerned about avoiding the distracting influence of the words in the periphery for the task at hand. In short, they had no reason to fixate in any location other than center.
In Experiment 2, N400 effects were delayed by ∼50 ms, consistent with Kutas' (1987) report of delayed N400 effects at a presentation rate of 10 words/s. We thus analyzed the N400 of the critical triad in a shorter time window (100 ms) to avoid overlap with the semantic processing of the following triad.
References
- Altarriba J, Kambe G, Pollatsek A, Rayner K. Semantic codes are not used in integrating information across eye fixations in reading: Evidence from fluent Spanish English bilinguals. Perception & Psychophysics. 2001;63:875–890. doi: 10.3758/BF03194444. [DOI] [PubMed] [Google Scholar]
- Baccino T, Manunta Y. Eye-Fixation-Related Potentials: Insight into Parafoveal Processing. Journal of Psychophysiology. 2005;19(3):204–215. doi: 10.1027/0269-8803.19.3.204. [DOI] [Google Scholar]
- Balota DA, Pollatsek A, Rayner K. The interaction of contextual constraints and parafoveal visual information in reading. Cognitive Psychology. 1985;17:364–390. doi: 10.1016/0010-0285(85)90013-1. 10.1016/0010-0285(85) 90013-1. [DOI] [PubMed] [Google Scholar]
- Barber HA, Doñamayor N, Kutas M, Münte T. Parafoveal N400 effect during sentence reading. Neuroscience Letters. 2010;479(2):152–156. doi: 10.1016/j.neulet.2010.05.053. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Barber HA, Shir B, Bentin S, Kutas M. Parafoveal perception during sentence reading?: An ERP paradigm using rapid serial visual presentation (RSVP) with flankers. Psychophysiology. 2011;48:523–531. doi: 10.1111/j.1469-8986.2010.01082.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Bouma H. Interaction effects in parafoveal letter recognition. Nature. 1970;226:177–178. doi: 10.1038/226177a0. [DOI] [PubMed] [Google Scholar]
- Brysbaert M, Drieghe D, Vitu F. Word skipping: Implications for theories of eye movement control in reading. In: Underwood G, editor. Cognitive processes in eye guidance. Oxford: Oxford University Press; 2005. pp. 53–77. [DOI] [Google Scholar]
- Coulson S, Federmeier K, Van Petten C, Kutas M. Right hemisphere sensitivity to word and sentence-level context. Journal of Experimental Psychology: Learning, Memory, and Cognition. 2005;31:129–147. doi: 10.1037/0278-7393.31.1.129. [DOI] [PubMed] [Google Scholar]
- Dimigen O, Kliegl R, Sommer W. Trans-saccadic parafoveal preview benefits in fluent reading: A study with fixation-related brain potentials. Neuroimage. 2012;62:381–393. doi: 10.1016/j.neuroimage.2012.04.006. [DOI] [PubMed] [Google Scholar]
- Dimigen O, Sommer W, Hohlfeld A, Jacobs A, Kliegl R. Coregistration of eye movements and EEG in natural reading: Analyses & Review. Journal of Experimental Psychology: General. 2011;140(4):552–572. doi: 10.1037/a0023885. [DOI] [PubMed] [Google Scholar]
- Drieghe D. Parafoveal-on-foveal effects in eye movements during reading. In: Liversedge SP, Gilchrist ID, Everling S, editors. Oxford handbook on eye movements. Oxford: Oxford University Press; 2011. pp. 839–855. [Google Scholar]
- Ehrlich SF, Rayner K. Contextual effects on word perception and eye movements during reading. Journal of Verbal Learning and Verbal Behavior. 1981;20:641–655. doi: 10.1016/S0022-5371(81)90220-6. [DOI] [Google Scholar]
- Federmeier KD, Kutas M. Right words and left words: Electrophysiological evidence for hemispheric differences in meaning processing. Cognitive Brain Research. 1999;8(3):373–392. doi: 10.1016/S0926-6410(99)00036-1. [DOI] [PubMed] [Google Scholar]
- Gratton G, Coles MGH, Donchin E. A new method for the off-line removal of ocular artifact. Electroencephalography and Clinical Neurophysiology. 1983;55:468–484. doi: 10.1016/2F0013-4694(88)90135-9. [DOI] [PubMed] [Google Scholar]
- Henderson JM, Ferreira F. Effects of foveal processing difficulty on the perceptual span in reading: Implications for attention and eye movement control. Journal of Experimental Psychology: Learning, Memory and Cognition. 1990;16:417–429. doi: 10.1037/0278-7393.16.3.417. [DOI] [PubMed] [Google Scholar]
- Hohenstein S, Laubrock J, Kliegl R. Semantic preview benefit in eye movements during reading: A parafoveal fast-priming study. Journal of Experimental Psychology: Learning, Memory, and Cognition. 2010;36:1150–1170. doi: 10.1037/a0020233. [DOI] [PubMed] [Google Scholar]
- Inhoff AW, Radach R, Starr M, Greenberg S. Allocation of visuo-spatial attention and saccade programming during reading. In: Kennedy A, Radach R, Heller D, Pynte J, editors. Reading as a perceptual process. Oxford: Elsevier; 2000. pp. 221–246. [DOI] [Google Scholar]
- Kutas M. Current trends in Event-Related Potential Research (EEG Suppl 40) Elsevier Science Publishers B.V.; 1987. Event-related brain potentials (ERPs) elicited during rapid serial visual presentation of congruous and incongruous sentences; pp. 406–411. [PubMed] [Google Scholar]
- Kutas M, Federmeier KD. Thirty years and counting: Finding meaning in the N400 component of the event related brain potential (ERP) Annual Review of Psychology. 2011;62:621–647. doi: 10.1146/annurev.psych.093008.131123. [DOI] [PMC free article] [PubMed] [Google Scholar]
- Kretzschmar F, Bornkessel-Schlesewsky I, Schlesewsky M. Parafoveal versus foveal N400s dissociate spreading activation from contextual fit. Neuroreport. 2009;20(18):1613–8. doi: 10.1097/WNR.0b013e328332c4f4. [DOI] [PubMed] [Google Scholar]
- Murray WS. Parafoveal pragmatics. In: Underwood G, editor. Eye guidance in reading and scene perception. Oxford, England: Elsevier; 1998. pp. 181–200. [DOI] [Google Scholar]
- Oldfield RC. The assessment and analysis of handedness: The Edinburgh Inventory. Neuropsychologia. 1971;9:97–113. doi: 10.1016/0028-3932(71)90067-4. [DOI] [PubMed] [Google Scholar]
- Pernet C, Uusvuori J, Salmelin R. Parafoveal-on-foveal and foveal word priming are different processes. Behavioral and neurophysiological evidence NeuroImage. 2007;38:321–330. doi: 10.1016/j.neuroimage.2007.07.035. [DOI] [PubMed] [Google Scholar]
- Rayner K. Eye movements in reading and information processing: 20 years of research. Psychological Bulletin. 1998;124:372–422. doi: 10.1037/0033-2909.124.3.372. [DOI] [PubMed] [Google Scholar]
- Rayner K, Balota DA, Pollatsek A. Against parafoveal semantic preprocessing during eye fixations in reading. Canadian Journal of Psychology. 1986;40:473–483. doi: 10.1037/h0080111. [DOI] [PubMed] [Google Scholar]
- Rayner K, White SJ, Kambe G, Miller B, Liversedge SP. On the processing of meaning from parafoveal vision during eye fixations in reading. In: Hyönä J, Radach R, Deubel H, editors. The mind's eye: Cognitive and applied aspects of eye movement research. Amsterdam: Elsevier Science; 2003. pp. 213–234. [DOI] [Google Scholar]
- Schotter ER, Angele B, Rayner K. Parafoveal processing in reading. Attention, Perception, & Psychophysics. 2012;74:5–35. doi: 10.3758/s13414-011-0219-2. [DOI] [PubMed] [Google Scholar]
- Schroyens W, Vitu F, Brysbaert M, d'Ydewalle G. Eye movement control during reading: Foveal load and parafoveal processing. Quarterly Journal of Experimental Psychology. 1999;52:1021–1046. doi: 10.1080/027249899390909. [DOI] [PubMed] [Google Scholar]
- Sebastián-Gallés N, Martí A, Carreiras M, Cuetos F. LEXESP: Una base de datos informatizada del español. Barcelona, Spain: Ed. Universitat de Barcelona; 2000. [Google Scholar]
- Sereno SC, Rayner K. Measuring word recognition in reading: Eye movements and event-related potentials. Trends in Cognitive Sciences. 2003;7(11):489–493. doi: 10.1016/j.tics.2003.09.010. [DOI] [PubMed] [Google Scholar]
- Simola J, Holmqvist K, Lindgren M. Right visual field advantage in parafoveal processing: evidence from eye-fixation-related potentials. Brain and Language. 2009;111(2):101–113. doi: 10.1016/j.bandl.2009.08.004. [DOI] [PubMed] [Google Scholar]
- Yan M, Richter EM, Shu H, Kliegl R. Readers of Chinese extract semantic information from parafoveal words. Psychonomic Bulletin & Review. 2009;16:561–566. doi: 10.3758/PBR.16.3.561. [DOI] [PubMed] [Google Scholar]
- Yan M, Zhou W, Shu H, Kliegl R. Lexical and sublexical semantic preview benefits in Chinese reading. Journal of Experimental Psychology: Learning, Memory, and Cognition. 2012;38(4):1069–1075. doi: 10.1037/a0026935. [DOI] [PubMed] [Google Scholar]