Event-related brain potentials reveal how multiple aspects of semantic processing unfold across parafoveal and foveal vision during sentence reading

Abstract

Recent event-related brain potential (ERP) experiments have demonstrated parafoveal N400 expectancy and congruity effects, showing that semantic information can be accessed from words in parafoveal vision (a conclusion also supported by some eyetracking work). At the same time, it is unclear how higher-order integrative aspects of language comprehension unfold across the visual field during reading. In the current study, we recorded ERPs in a parafoveal flanker paradigm while readers were instructed to read passively for comprehension or to judge the plausibility of sentences in which target words varied in their semantic expectancy and congruity. We directly replicated prior work showing graded N400 effects for parafoveal viewing, which are then not duplicated when the target words are processed foveally. Critically, although N400 effects were not modulated by task goals, a posteriorly-distributed late positive component thought to reflect semantic integration processes was observed to semantic incongruities only in the plausibility judgment task. However, this effect was observed at a considerable delay, appearing only once words had moved into foveal vision. Our findings thus suggest that semantic access can be initiated in parafoveal vision, whereas central foveal vision may be necessary to enact higher-order (and task-dependent) integrative processing.

1. Introduction

Unlike in speech processing, wherein the stimulus stream unfolds in a linear and serial manner that is (typically) outside of the control of the comprehender, reading entails the active control of attention across a visual field comprising multiple words. In English, this span ranges from about four characters to the left of fixation to upwards of 15 characters to the right of fixation, such that attention is asymmetrically distributed to extract information from upcoming words in parafoveal vision (McConkie & Rayner, 1975). Research on parafoveal processing during reading has largely focused on understanding the kind of information (e.g., orthographic, phonological, semantic) that can be extracted from the word to the right of fixation, and how that parafoveal information is dynamically integrated with subsequent foveal word processing (see Schotter et al., 2012 for a review). This research base has largely come from eye-tracking methodologies. More recently, event-related brain potential (ERP) methods have emerged as a useful tool in providing both converging and unique information on the time-course and neural underpinnings of parafoveal word processing.

For example, ERP experiments have demonstrated that the N400, a component strongly linked to the access of information from long-term semantic memory, shows graded sensitivity to lexico-semantic expectancy and semantic congruity/plausibility in parafoveal vision (Barber et al., 2013; Stites et al., 2017; Payne et al., 2017; see Schotter, 2018 for a recent review). ERP studies of word by word sentence processing have also frequently reported positivities following the N400 (see review by Van Petten & Luka, 2012). These “post-N400 positivities” (PNPs) have been shown to be sensitive to both violations of semantic plausibility (as seen on what is often termed the the late positive complex (LPC), which has a posterior scalp distribution) and plausible prediction violations (as seen on more frontally distributed PNPs), suggesting that they reflect higher-order integrative and revision processes following initial semantic access. Although widely reported in RSVP ERP studies, PNP effects have not been explicitly tested for nor reported in ERP studies of parafoveal processing. Given that recent eye tracking findings (reviewed in more detail below) have presented evidence for parafoveal detection of plausibility violations in eye-movement behavior during natural reading (e.g., Schotter & Jia, 2016; Veldre & Andrews, 2016), it remains an open question whether the LPC, which has been linked to plausibility-related integration failures (Van Petten & Luka, 2012; Browuer et al., 2017; DeLong et al., 2014), can be elicited from parafoveal stimulation. Thus, in the current study, our goal was to elucidate the nature and time-course of higher-level integrative processing across parafoveal and foveal vision by studying the posterior LPC component of the ERP during sentence reading in a flanker-RSVP paradigm that permits the measurement of ERP effects elicited by words in parafoveal vision.

1.1. Parafoveal Semantic Processing in Eye-Tracking and E/FRPs.

Our understanding of parafoveal processing largely comes from studies recording eye-movements during reading (see reviews by Clifton et al., 2016; Rayner, 2009; Schotter et al., 2012), and especially from experiments that have employed gaze-contingent display change paradigms to explore the types of information that can be extracted from parafoveal vision and rapidly integrated with foveal vision. For instance, orthographically and phonologically related masks produce reliable parafoveal preview benefit effects (see Rayner, 2009 for a review), suggesting that readers can extract initial letter and abstract phonological information from word n+1. However, evidence that semantic information can be extracted from word n+1 in parafoveal vision has been more controversial, and there is an ongoing debate regarding the nature and scope of semantic extraction from parafoveal vision (e.g., Rayner et al., 2014; Clifton et al., 2016; Dimigen et al., 2012; Johnson & Dunne, 2012; Snell et al., 2016).

Early behavioral and eye-tracking experiments suggested that semantic information was not extracted from parafoveal vision in English (Rayner et al., 1986; Bertram & Hyönä, 2007; Rayner et al., 2014). Other work, however, has found evidence for the extraction of semantic features from parafoveal vision. For example, semantic parafoveal preview benefits have been attested in morphologically rich languages such as Mandarin (Yan, Richter, Shu, & Kliegl, 2009) and German (Hohenstein, Laubrock, & Kliegl, 2010). Moreover, parafoveal semantic effects have also been reported in certain cases in English, when previews are available only for brief durations (Hohenstein et al., 2010), when the previews are synonymous with the target word (Schotter, 2013), or when previews are embedded within constraining sentence contexts (Schotter et al., 2015; Schotter & Jia, 2016; Veldre & Andrews, 2016). Indeed, these findings are consistent with recent ERP studies of parafoveal semantic processing, which have provided converging evidence that high-level semantic information can be activated during parafoveal perception, particularly in the presence of supportive context information (Barber et al., 2011, 2013; Kornrumpf et al., 2016; Lopez-Perez et al., 2016; Kretzschmar et al., 2009; Li et al., 2015; Stites et al., 2017; Payne & Federmeier, 2017).

One approach to using ERPs to assess parafoveal processing in reading has been to co-register electroencephalogram (EEG) and eye-tracking data within the same experiment to construct fixation-related potentials (Dimigen et al., 2011, 2012; Kornrumpf et al., 2016; Lopez-Perez et al., 2016). An alternative approach, developed by Barber, Kutas, and colleagues (Barber et al., 2010, 2011, 2013), has been to modify the traditional ERP–RSVP paradigm to afford the perception of parafoveal targets. In the flanker-RSVP paradigm, a centrally presented word (word n) is flanked in the left and right visual hemi-fields by the preceding and following words in the sentence (word n + 1 and n – 1), respectively. Each word is separated by 2° of visual angle from central fixation so that the flankers appear in parafoveal vision. Participants maintain central fixation, and each word is shifted to the left across trials, such that the sentence appears to progress successively across the screen.

Although this paradigm is non-naturalistic, the flanker ERP paradigm has proven to be a useful tool for examining parafoveal processing during reading, providing a bridge between eye movement and ERP studies of language processing while simultaneously maintaining the experimental and EEG artifact control that is a strength of traditional ERP studies, including control of the rate, timing, and location of visual input across successive words and the reduction of eye-movement artifacts. In addition ERPs in the flanker paradigm can be analyzed over long multi-word epochs, providing a continuous measure of neural activity that spans the time from prior to parafoveal word onset until after foveal processing has completed. Related to this, the flanker paradigm allows for the examination of “long-latency” ERP components ( > 400ms) that have been implicated in high-level semantic processing (Van Petten & Luka, 2012). Such multi-word and long-latency ERPs cannot be easily recorded from fixation-related potentials, which are constrained to short time-windows due to trial-to-trial (and subject-to-subject) variation in the length of fixation durations, variation in subsequent saccade targets, and regressions (see Payne et al., 2016, 2017 for discussions).

Studies using the flanker-word ERP paradigm have consistently found evidence for semantic modulations of the N400 to words presented in parafoveal vision (Barber et al., 2010, 2013; Li et al., 2015; Stites et al., 2017; Payne & Federmeier, 2017). For example, Barber and colleagues (2011, 2013) observed the classic N400 semantic congruity effect to words presented in parafoveal vision, such that semantically incongruent parafoveal words elicited larger N400 amplitudes than did congruent words, and showed that these effects were larger for semantic anomalies in highly constraining compared to unconstraining sentences. In contrast, findings from FRP studies are more variable, with some studies finding evidence consistent with parafoveal semantic processing (Baccino & Manunta, 2005; Lopez-Perez et al., 2016; Kretzschmar et al., 2009), but others finding no clear effects of semantic manipulations in parafoveal vision (i.e., Dimigen et al., 2012; Li et al., 2015; see Schotter, 2018 for an extensive review).

Recently, Stites, Payne, and Federmeier (2017) used the flanker paradigm to examine whether the parafoveal N400 responds in a manner that is graded with respect to semantic expectancy (as is observed for the foveal N400) and to directly examine the effects of parafoveal processing on subsequent foveal viewing. Young participants read highly constraining sentence contexts wherein a target word was expected, unexpected but plausible, or incongruent, as well as neutral (low constraint) sentences in which the target word was weakly expected. When words first appeared in parafoveal vision, findings revealed an N400 effect that was continuously graded with increasing expectancy, replicating the graded N400 pattern that is typically found during foveal-only presentation. Importantly, when parafoveal targets were subsequently presented (again) in foveal vision, N400 amplitudes were reduced overall (see also Ni et al., 2015; Dimigen et al., 2012), and the relationship between foveal N400 amplitude and both expectancy and congruity were substantially mitigated, suggesting that parafoveal semantic processing facilitated subsequent foveal viewing.

1.2. Electrophysiological Indices of Multiple Stages of Context Processing.

A number of ERP experiments have revealed multiple neural mechanisms by which semantic information is assessed and updated, including effects on ERP components following the N400 (Federmeier et al., 2007; Wlotko et al., 2012; DeLong et al., 2014; Payne & Federmeier, 2017; Van Petten & Luka, 2012). For example, violations of thematic-role assignment (e.g., the eggs would eat) do not modulate the N400 but instead produce a late positive potential with a similar timing and topography to the P600 effect that is commonly observed to morphosyntactic violations (Kuperberg et al., 2003); this effect has been termed the “semantic P600.” Similarly, many studies have reported bi-phasic ERP responses to semantic congruity violations, such that large N400 amplitudes to semantically anomalous words are often immediately followed by a posterior LPC, with a similar morphology and distribution to the (semantic) P600. (Van Petten & Luka, 2012; Leckey & Federmeier, 2019). Indeed, Brouwer and colleagues (2014, 2017) have recently proposed a neurocomputational model accounting for the bi-phasic N400/LPC complex in terms of a single-stream “Retrieval-Integration” (RI) cycle, in which the N400 reflects semantic memory retrieval and the LPC reflects a process where the activated memory representation is integrated into an updated message-level representation. A major goal of the current study was to examine whether the LPC to plausibility violations can be initiated in parafoveal vision. One previously successful way of probing late positive components has been to examine the effects of comprehension goals on the magnitude of these potentials.

1.3. Task and Goal Effects on Comprehension

Although some studies have shown that participants’ task goals can influence the magnitude of the N400 (Chwilla et al., 1995, Schacht et al., 2014; Lau et al., 2009), most evidence suggests that the N400 is largely automatic and obligatory, as neither task relevancy nor conscious awareness is required to elicit N400 effects (see Kutas & Federmeier, 2011, for a review). N400 priming is observed for masked targets that cannot be categorized or even reported (Stenberg et al., 2000), during the attentional blink (Vogel et al., 1998), and even in minimally conscious states (e.g., sleep, coma; Steppacher et al., 2013). In contrast, language effects following the N400 are more notably modulated by controlled attention and task demands. For instance, Batterink & Neville (2013), using a cross-modal distraction task, showed that the P600 to syntactic violations was only evident to violations that were consciously reported. Similarly, larger positivities to overt syntactic violations (Gunter et al., 1997; Osterhout & Mobley, 1995) and selectional restriction violations (Kolk et al., 2003) have been observed when the task involves making active acceptability or plausibility judgements following each sentence (see also Brouwer et al., 2012). This patten of task-and attention-dependent effects has led some to argue that the syntactic P600 may be part of the larger P3b family of responses (Coulson et al., 1998; Hahne & Friederici, 2002), given their similar sensitivity to probability, task relevancy, and RT alignment (Sassenhagen et al., 2014), defining features of P3b activity that are not present for other language-related components such as the N400 (Payne & Federmeier, 2017).

Like the pattern seen for the syntactic P600, the LPC to semantic plausibility violations has been shown to be modulated by task. Generally, experiments that require a plausibility judgement manifest larger and more robust LPCs to semantically anomalous words (Kolk et al., 2003; Geyer et al., 2006; but see Stroud & Phillips, 2012), suggesting that these effects may be attentionally-dependent to some degree. However, to our knowledge, few if any studies have directly examined task effects on LPC responses to semantic anomalies within subjects in a single study. Moreover, very little is known about the possibility of task effects on the frontal positivity to plausible but expected words (Brothers et al., 2017). Most importantly for the present study, no work to date has examined whether and how task goals modulate neural indices of semantic access and integration across parafoveal and foveal vision.

1.4. Current Study

In the current study, we examined the impact of comprehension task goals on two neural indices of semantic processing across parafoveal and foveal vision: the N400 and the LPC. In Stites et al. (2017), readers showed modulations of N400 activity in parafoveal vision, but there was no LPC (or frontal positivity) following either parafoveal or foveal viewing during passive reading. Here, to further probe post-N400 semantic processing across parafoveal and foveal vision, we introduced a plausibility task manipulation within the experimental paradigm used in Stites et al. (2017). To the extent that the LPC is more task-dependent than N400 effects, we reasoned that shifting the goal from passive comprehension to the overt assessment of the plausibility of sentences should leave the N400 unchanged but increase the magnitude of the LPC. We can also assess what impact, if any, this kind of task demand might have on the frontal positivity. The central question, then, is when such higher-level integrative effects might be observed. If plausibility information can be rapidly integrated with the ongoing semantic context in parafoveal vision (Veldre & Andrews, 2016), we would expect to observe a bi-phasic LPC following the parafoveal N400 effect (e.g., Van Petten & Luka, 2016; Brouwer et al., 2017). If, however, integrating the target word into its prior context requires direct fixation (Kretzschmar et al., 2009), such LPC effects may be delayed until the target word is directly fixated.

2. Method

2.1. Participants.

Twenty-four college-aged adults (M = 20; Range: 18–23, 16 female) participated in the experiment and were compensated with course credit. All participants were right-handed as assessed by the Edinburgh inventory (Oldfield, 1971) and had an average of 13.9 years of education (Range: 12–16 years). Participants reported no visual impairments and had normal (corrected or uncorrected) visual acuity. All readers were monolingual speakers of English, with no consistent exposure to other languages before the age of five. Participants had no history of neurological or psychiatric disorders or brain damage. Each participant signed informed consent documents approved by the University of Illinois Institutional Review Board, and this work was carried out in accordance with the Code of Ethics of the World Medical Association (Declaration of Helsinki).

2.2. Materials.

Experimental sentences included 180 highly constraining sentences and 60 low constraint control sentences that have been used in previous work (Payne et al., 2016; Payne & Federmeier, 2017; Stites et al., 2017). Results from a full norming study on these stimuli in younger adults are described in Stites et al., (2017). Note that, all items were counterbalanced across the critical task manipulation (described in more detail below), such that item differences cannot account for the critical goal-manipulation findings reported below.

For the highly constraining sentence contexts, sentences were identical up to the critical parafoveal target word across all conditions. These highly constraining contexts were continued with either the most expected word (mean cloze = .87), an unexpected but plausible word of the same grammatical class (mean cloze = .06), or a semantically incongruent word. For example,

Maria remembered to shut the front window and lock the back…

• (1)door (expected)
• (2)room (unexpected but plausible)
• (3)note (incongruent)

…before leaving for vacation.

Semantically incongrous words were drawn from the same grammatical class as the unexpected target words and were matched on perceived concreteness and imageability, based on norms collected from the MRC Psycholinguistics Database (Wilson, 1988), as well as on frequency measures derived from the Corpus of Contemporary American English (CoCa, Davies, 2008). The expected words were slightly shorter than the incongruent words, which, in turn, were slightly shorter than the unexpected words. Note, however, that because the parafoveal manipulation is defined based on a definition of degrees of visual angle to the nearest edge (instead of word adjacency, as in the eye tracking literature), differences in the length of the pre-target or target word from item to item do not contribute to differences in parafoveal eccentricity of the initial letters of the following or preceeding words. Note also that a single-trial analysis using the same stimuli on a different sample, reported in Stites et al., (2017) revealed that length effects played no role in the magnitude of expectancy and anomaly effects on the parafoveal N400. Additionally, there were 60 neutral low-constraint experimental sentence frames that always contained their most expected target word (mean cloze = .23) (e.g., During years of drudgery, the man had derived comfort from a single book that he enjoyed again and again). Including these items allows us to examine the parafoveal N400 to low-to-moderately expected continuations and also ensures that half of the sentences presented to subjects did not violate readers’ predictions about upcoming targets, and contained no unexpected or anomalous words. Expected words in the low constraint sentences also did not differ from the high constraint expected words in terms of frequency, concreteness, or imageability, although they were longer than the targets in the high constraint sentences, t(59) = −2.76, p < .01 (which again, does not contribute to differences in the location in the visual field of the surrounding words). In each of the three high constraint conditions, the parafoveal target word was preceded by an identical pre-target word (mean length: 3.5 characters, SD = 1.8), which, in 121/180 sentences, was a short function word or pronoun (i.e., the, to, his). The targets in the low constraint sentences were preceded by similar short pre-target function words or pronouns in 42/60 sentences (mean length: 3.0 characters, SD = 1.4). The majority (85%) of the immediately post-target words were high frequency function/closed class words (e.g., after, because, with), and the rest of the post-target words were distributed across nouns (2%), verbs (2%), adjectives (4%), and adverbs (7%).

2.3. Procedure.

Participants were seated 85 cm from a 21” CRT computer monitor in a dim, quiet testing room. As in our prior flanker ERP studies, sentences were presented serially in triads, with the target word appearing at central fixation, flanked bilaterally by the upcoming word in the sentence to the right, and the preceding word to the left. Participants were informed that multiple words and symbols would appear to the left and right of the central word but to keep their fixation on the word presented at the center of the screen and read each sentence for comprehension. At the viewing distance, 3.5 letters subtended one degree of visual angle. The beginning letter of the right parafoveal word and the final letter of the left parafoveal word were anchored so as to appear at 2° of visual angle from the center of the screen. Each trial began with a series of fixation crosses (“+++++”) that remained on the screen for a duration that was jittered from 500–1500 ms. Each triad was visible on the screen for 100ms, with an inter-stimulus interval of 350 ms, during which no masking was used. Because it takes approximately 180–200 ms to program and execute an eye movement (Becker & Jürgens, 1979; Rayner, Slowiaczek, Clifton, & Bertera, 1983; Reichle et al., 2003), this short stimulus duration minimized the possibility that participants could program and execute a saccade to the parafoveal word within the amount of time that it was visible on the screen. The experiment began with a practice block of nine sentences to allow participants to become accustomed to the experimental procedure. Participants were continually given feedback about reducing eye movements throughout the experiment as needed. The 240 sentences were divided into 6 blocks of 40 sentences each.. In the first three blocks, participants were instructed to read for comprehension. In the last three blocks, participants were instructed to make yes/no plausibility judgements following each sentence. Following the end of each sentence in blocks 3–6, there was a 1000ms SOA before a prompt reading “Plausible?” was presented on the screen. The prompt remained on the screen until participants selected whether they thought the sentence was plausible or not. Items and response hand were counterbalanced across subjects.

Note that the plausibility manipulation was always the final three blocks. This was an intentional design choice that was driven by a concern for potential carry-over effects of the plausibility manipulation. Carry-over effects of the type we were concerned about have been previously reported in studies aiming to manipulate task instructions. For example, Fallon et al. (2006) showed evidence for task instruction carryover effects on on-line sentence processing in a listening comprehension task. When listening under instructions for comprehension versus recall, participants showed different effects based on whether the recall instructions occurred in the first or second block. The pattern of effects reflected task carryover, such that effects during listening for comprehension looked more like effects during listening for recall when passive comprehension was in the second block, suggesting that instructions to revert to passive comprehension were not effective. Thus, we opted to maintain a consistent task order, with passive comprehension blocks always coming first.

2.4. EEG Recording and Processing.

EEG was recorded from twenty-six evenly-spaced silver-silver chloride electrodes embedded in an Electro-Cap (following the same montage as in Federmeier et al., 2007). Electrodes were referenced on-line to the left mastoid and re-referenced off-line to the average of the right and left mastoids. In addition, one electrode (referenced to the left mastoid) was placed on the left infraorbital ridge to monitor for vertical eye movements and blinks, and another two electrodes (referenced to one another) were placed on the outer canthus of each eye to monitor for horizontal eye movements. Electrode impedances were kept below 5 kΩ. The continuous EEG was amplified through a bandpass filter of .02–100 Hz and recorded to hard disk at a sampling rate of 250 Hz.

Epochs of EEG data were taken from 100 ms before stimulus onset to 1400 ms post-stimulus onset, such that the epoch contained target words from when they first appeared in parafoveal vision through when they appeared in foveal vision. This long-average time window allows for us to examine ERPs to multiple concurrent words. Notably, this window allows for us to examine ERP components time-locked to parafoveal onset, but also examine the subsequent foveal N400 and LPC canonical time window, up to 950ms post foveal stimulus onset.

EEG epochs were examined for artifacts (amplifier blocking, signal drift, eye movements, eye blinks, or muscle activity), which were rejected off-line before averaging, using individually selected thresholds through visual inspection of the data. We took particular care to ensure that data contained as few eye movements as possible to make sure that our findings represent circumstances in which readers were directly fixating the central word. Individualized eye-movement thresholds were separately determined for each participant through visual inspection of the ERPs to identify trials with eye movements in the bipolar HEOG; these thresholds were then applied to every trial for that participant, such that all trials with eye movements were removed prior to performing data analysis. Following this, average residual eye movements in right and left EOG within the stimulus presentation window (100ms following stimulus onset) were calculated. Residual EOG was less than 3μV for the congruity violation effect for all subjects, corresponding to an eye rotation of less than ± 0.1° (see Woodman & Luck, 2003; Lins, Picton, Berg, & Scherg, 1993). Two subjects were removed from further analyses following artifact detection for an excessive number of eye-movement artifacts ( > 50%), leaving N = 22 adults for analysis. Artifact-free ERPs were averaged separately for each subject by stimulus type after subtraction of the 100ms pre-stimulus baseline. On average, a total of 12% (SD = 11) of trials were marked with artifacts and not included in subsequent analyses. Prior to statistical analyses, ERPs were digitally filtered with a low-pass filter of 30Hz.

3. Results

3.1. Plausibility Judgements.

Plausibility responses for the second half (last 3 blocks) of the experiment showed that both low constraint (96%) and expected sentences (97%) elicited “plausible” responses most of the time (not different from one another; t > 1). Unexpected items, which were plausible by design, were also largely judged as plausible (76%), albeit to a lower degree than either the expected (t = 10.85, p <.001) or the low constraint sentences (t = 10.07, p <.001). Finally, incongruent items were largely judged as implausible (9%), and garnered lower rates of “plausible” responses than all other conditions (t_{[Incon-Expected]} = −20.23, p <.001; t_{[Incon-Unexp]} = −14.28, p <.001; t_{[Incon – Low Const]} = −19.91, p <.001). Thus, participants were successfully discriminating between the plausibility of sentence types in the judgement task.

3.2. Event-Related Brain Potentials.

Figure 1 shows long-average ERPs at midline prefrontal, central, and posterior electrode sites. The ERPs are time-locked to the onset of the critical word triad for all four target word conditions (initially in parafoveal vision). ERPs are presented separately for the passive comprehension block (no judgement; replicating Stites et al., 2017) and the plausibility judgement block. Typical electrophysiological responses were observed for visual stimulation. Following early sensory components, a late negative-going wave— the N400—was observed. Replicating prior work on the parafoveal N400 with younger adults (Stites, Payne, & Federmeier, 2017), the N400 to the parafoveal presentation of the target word was graded in amplitude based on the expectancy of the parafoveal target. At foveal presentation (450ms following parafoveal onset), the parafoveal target moved into foveal vision. Sensory potentials to the foveal target (most notably the posterior-occipital P2) were observed following the offset of the parafoveal N400 effect. Again, replicating prior work (Stites et al., 2017; Barber et al., 2011, 2013; Payne & Federmeier, 2017), the foveal N400 was reduced in magnitude (more positive), and showed a reduced impact of expectancy (Stites et al., 2017) for both the passive comprehension and plausibility judgement conditions. Finally, only in the plausibility judgement condition, a broadly distributed parietal/posterior positivity (DeLong et al., 2014; Federmeier et al., 2007, 2010; Kuperberg et al., 2013; Van Petten & Luka, 2012) clearly followed the foveal N400 to incongruent target words.

Figure 1.

Grand average event-related brain potentials at representative prefrontal, central, and posterior midline electrode sites. The time window encompasses both parafoveal and foveal viewing for expected, unexpected, incongruent and low constraint target words. Time 0 indicates when the critical target appeared in parafoveal vision. Plots are presented separately for passive reading (left) and plausibility judgement (right) blocks. At 450 ms the target word shifts from parafoveal to foveal vision, marked in the diagram as “foveal onset” (negative voltage plotted up).

3.3. Parafoveal and Foveal N400.

Analyses of N400 effects in parafoveal and foveal vision were conducted across an average of the eight centro-parietal electrode sites (LMCe, RMCe, LDCe, RDCe, LDPa, RDPa, MiCe, MiPa), where N400 effects are typically largest¹. Time-window selection of the foveal and parafoveal N400 was based on a priori time-windows for the canonical N400 effect (300–500ms post stimulus onset), as in Stites et al., (2017). Subject-level mean amplitudes for the parafoveal and foveal N400 effect were submitted to separate linear mixed-effects models using the lme4 package (v. 1.1.19; Bates et al., 2010) and the afex package (v. 0.19.1) in the R language for statistical computing (v. 3.4.3). Note that, because analyses are conducted on subject-mean amplitudes, as in traditional ERP experiments, these models are analogous to repeated-measures ANOVAs, but do not require a sphericity-violation correction because the homogeneity of effect variances is modeled in the random-effects structure (see Barr et al., 2013 for more discussion). Random intercepts were defined for subjects, and a maximal random effects structure for the within-subject factors (Context and Task) was fit across subjects, excluding covariance terms between random intercepts and slopes (Barr et al., 2013; Bates et al., 2016; Matushek et al., 2017) to achieve model convergence. The top two panels of Table 1 present the results of the omnibus likelihood ratio tests of the Context x Task effects on the parafoveal and foveal N400 effects. All fixed effects were deviance coded. Approximate degrees of freedom for follow-up tests are calculated via a Satterthwaite approximation.

Table 1.

Results of Omnibus Likelihood Ratio Tests of Context and Task Effects on Parafoveal N400, Foveal N400, and Foveal Late Positive Component.

	Chi-Sq	df	p-value
Parafoveal N400
Context	45.61	3	<.001
Task	1.09	1	.30
Context x Task	0.29	3	.96
Foveal N400
Context	11.03	3	.01
Task	2.64	1	.10
Context x Task	7.03	3	.07
Foveal LPC
Context	6.93	3	.07
Task	1.82	1	.18
Context x Task	17.37	3	<.001

As can be seen in Table 1 there was a reliable main effect of Context for the parafoveal N400 (replicating Stites et al., 2017 and Payne & Federmeier, 2017), but there was no significant effect of Task and no Task x Context interaction present, suggesting that the parafoveal N400 was not influenced by comprehension goals. Follow-up pairwise tests between conditions found, as predicted, that N400 amplitudes were facilitated in the high constraint expected condition relative to all of the other conditions, Expected vs. LC: t(38.38) = 2.18, p <.05 ; Expected vs. Unexpected: t(33.36) = 4.98, p <.001; Expected vs. Incongruent: t(35.56) = 5.19, p <.001. The N400 amplitude elicited by the low constraint words was also significantly facilitated relative to the incongruent, t(70.50) = 2.27, p <.05, and unexpected but plausible words, t(69.42) = 2.03, p <.05. Unexpected and incongruent words did not significantly differ, t(66.11) = .28.

In foveal vision, there was a reliable omnibus effect of context, though the magnitude of the effect was smaller in foveal compared to parafoveal vision, replicating Stites et al., (2017). Despite the significant omnibus likelihood ratio test, none of the pairwise contrasts between sentence conditions yielded statistically significant differences (all t’s < 1), consistent with the claim that foveal N400 effects are reduced in the presence of a valid parafoveal semantic preview (see Stites et al., 2017). As described in more detail below, this small and non-significant interaction was driven largely by an LPC in the plausibility judgement condition beginning within the later portion of the foveal N400 window (see Figure 1). Taken together, these findings indicate weak effects of expectancy or congruency on the foveal N400 when parafoveal preview had been available.

3.4. Posterior Late Positive Component.

Figure 1 reveals a clear positivity following the N400 to the foveally presented target word only in the plausibility judgement condition, with no clear positivity in the passive comprehension condition. A time-window of 600–800ms post target word foveal onset was chosen a priori based on review of the prior literature (Kos et al., 2012; Delong et al., 2014). The same eight central posterior channels were used, as these also encompass much of the typical LPC distribution. As can be seen in the bottom portion of Table 1, for the posterior LPC, there was a reliable Task x Context interaction. This interaction is graphically depicted in Figure 2, showing a clear increase in LPC amplitude to incongruent target words in the plausibility judgement task, but not in the passive comprehension task. Targeted pairwise tests of the incongruity effect on the LPC (incongruent vs. unexpected but plausible) showed a reliable LPC in the plausibility judgement task, t(34.95) = 3.19, p <.005, but not in passive comprehension, t(22.36) = .51, p =.61. These findings suggest that the late positive component following foveal presentation was strongly sensitive to modulations of task demand, consistent with the claim that it is functionally similar to other late posterior positivities indexing high-level semantic integration processes (Van Petten & Luka, 2012).

Figure 2.

Foveal LPC amplitudes for expected, unexpected, incongruent, and low constraint target words for passive reading (no judgement) and plausibility judgement conditions.

To isolate the impact of congruity violation on LPC activity timecourse, a difference wave was constructed via point-wise subtraction of the ERP waveforms of the incongruent condition and the unexpected but plausible condition, separately for the plausibility judgement and passive comprehension conditions. Figure 3a plots these difference waves at the midline parietal channel and Figure 3b shows the scalp distribution of the LPC for both task conditions. This clearly demonstrates a robust congruity violation effect following foveal viewing of the target in the plausibility condition relative to passive comprehension.

Figure 3.

A. Difference wave of the canonical LPC effect (incongruent-unexpected) overlaid separately for passive comprehension and plausibility judgement blocks. B. Scalp topography of the foveal LPC effect.

Because latency measures are nonlinear and exhibit a high degree of measurement error at the single-subject level, we adopted a jackknife approach to assessing onset latency of the LPC (Kiesel et al., 2008; Ulrich and Miller, 2001). The 50% peak latency was measured separately for each jackknife sub-sample, within a temporal ROI from 400 to 1400 ms post parafoveal word onset. The mean latency of the LPC was 1286 ms post parafoveal onset (836 ms post foveal onset) in the passive comprehension blocks and 1031 ms post parafoveal onset (581 ms post foveal onset) in the plausibility judgement blocks. Thus, the observed late positivity began in the canonical time window of the LPC relative to foveal word onset in both conditions. A jackknife corrected t-test (see Ulrich and Miller, 2001, Kiesel et al., 2008) was conducted to compare the onset latencies of the LPC difference wave between the passive comprehension and plausibility judgement tasks. Onset latency did not significantly differ between the two conditions (t_corrected = 1.22).

3.5. Frontal Positivity.

As can be seen in Figure 1, there was a tendency for responses to be more positive to unexpected but plausible items over frontal electrode sites, particularly in the plausibility judgement condition. We measured the frontal positivity over an apriori selection of 11 frontal and prefrontal electrode sites (MiPf, LLPf, RLPf, LMPf, RMPf, LDFr, RDFr, LMFr, RMFr, LLFr, RLFr; see Federmeier et al., 2007; Payne & Federmeier, 2017), using the same 600–800ms time window as the posterior LPC. There was a significant likelihood ratio test for the Task x Context interaction on the frontal positivity. This omnibus interaction was driven by a significant difference in the direction and magnitude of the Unexpected vs. Expected contrast between the passive comprehension block (−.86 uV, SE = .57, t = 1.52) and the plausibility judgement block (.96, uV, SE = .97, t < 1), (b = 1.58, se = .63, t (22.81) = 2.51, p <.05). Note however that neither of these simple contrasts were significantly different from zero in either the passive comprehension or plausibility judgement blocks (Gelman & Stern, 2006), suggesting only very weak modulation of the frontal positivity by task demands in foveal vision. In original reports of the frontal positivity, it was measured relative to an equally unexpected word that was not a prediction violation (Federmeier et al., 2007). Although it has sometimes been characterized relative to a strongly expected condition (as in Delong et al., 2014; Thornhill & Van Petten, 2012), this risks confounding the frontal positivity with post-N400 frontal negativities (Wlotko & Federmeier, 2012). However, the contrasts between the unexpected but plausible and and low constraint or anomalous items were not statistically significant.

4. Discussion

The aim of the current study was to examine the timecourse of semantic processing as a function of task goals in the parafoveal flanker paradigm. Our findings were clear. First, we replicated findings from Stites et al., (2017) showing parafoveal N400 effects that were graded by the degree of semantic expectancy and congruity. Critically, these effects were identical in both passive comprehension (as in Stites et al., 2017) as well as under a task manipulation wherein participants were instructed to judge the plausibility of the sentences. In contrast, the LPC was selectively elicted only during the plausibility judgement task. Interestingly, the LPC appeared at a substantial delay following the parafoveal N400, coming online only after foveal viewing of the target word. The implications of these findings for parafoveal processing in reading and the electrophysiology of semantic processing are discussed below.

4.1. Parafoveal N400.

The N400 findings from the current study directly replicated Stites et al. (2017), showing contextually graded N400 effects in parafoveal vision. These findings are consistent with a growing ERP and eye tracking literature indicating that there are cases in which semantic features of words can be accessed during parafoveal exposure. Moreover, we further replicated Stites et al. (2017) in showing that the magnitude of the N400 effect was reduced in foveal vision following parafoveal semantic processing of the target word. Drawing on theories of parafoveal processing from models of eye-movement control during reading that posit a stage of trans-saccadic parafoveal-to-foveal integration of word representations (Rayner et al., 2009), we have previously argued (Stites et al., 2017; Payne & Federmeier, 2018) that the reduced N400 effect in foveal vision following parafoveal processing of the same stimulus may reflect one such mechanism. That is, semantic memory is activated parafoveally and this lingering activation facilitates semantic access when the word is subsequently repeated in foveal vision (Stites, Payne, & Federmeier, 2017). These findings are also consistent with those from Grainger and colleagues (2016), who observed foveal-to-parafoveal lexical priming effects on the N400 in a trans-saccadic repetition priming paradigm, further suggesting that semantic information can be rapidly primed across visuo-spatially distinct stimuli.

In the present experiment, the item at fixation during parafoveal processing of the target word was generally a short function word. If this allowed particularly extensive processing of the parafoveal target, then the subsequent foveal processing of that target might resemble cases that have been previously described as as forced fixations (Schotter, 2018), in which the eyes fixate on a word that has already been semantically processed. If so, then the forced fixation account would predict that there is little to no additional processing of the fixated word (consistent with the lack of robust N400 effects), and attention is shifted entirely to the following word. In this case, we would expect that not only N400 effects, but also post-N400 integrative effects could be initiated and even completed during parafoveal exposure alone. Testing whether, during parafoveal processing, any further integrative processing of the target words occurs beyond N400 effects, for example on the LPC, was thus a major goal of the current study.

4.2. Post-N400 Positivities.

As seen in prior work (including Stites et al., 2017) we did not observe reliable post N400 positivites following either prediction violations or plausibility violations in passive comprehension. The precise mechanisms underlying these effects are not well understood. For example, factors relating to both individual differences and task demands appear to be important for eliciting the LPC (Tanner et al., 2014; Kos et al., 2012). Here, we systematically examined whether such effects are modulated by task in a within-subject manipulation across parafoveal and foveal vision. Indeed, our plausibility judgement task had a substantial influence on the neural response to semantically incongruent targets. In fact, whereas no clear LPC was observed to incongruent targets in the passive comprehension task, a large posterior positivity was observed in the plausibility judgement task.

Notably, this LPC was not initiated directly following the parafoveal N400, resulting in a substantial deviation from the timing of the typical bi-phasic N400-LPC pattern attested in prior work (e.g., DeLong et al., 2011, 2014; Hoeks et al., 2004; Brouwer & Crocker, 2017). We conducted a detailed onset latency analysis of the LPC difference wave (incongruent – unexpected) in the plausibility judgement condition, showing that the effect onset was well outside of the window typical of the LPC (which is canonically measured from ~500–800 ms post-stimulus onset). In contrast, the timing of effect onset relative to subsequent foveal viewing of the target (581ms) was fully consistent with typical LPC patterns seen in word by word reading paradigms. Thus, the latency data suggest that, despite initial semantic access of the target word taking place in parafoveal vision (as indexed by the parafoveal N400), the LPC may not be initiated until foveal viewing. More complex designs would be needed to completely rule out the possibility that this pattern could reflect a type of ‘spillover’ effect, wherein the canonical LPC is somehow delayed considerably when it is triggered parafoveally. Nevertheless, we believe the more parsimonious account is that we are measuring an LPC with typical timing properties that is only elicited once the word is processed in foveally. This, in turn, would imply that foveal viewing may be necessary for the kind of integrative processing reflected in the LPC. One other study, by Kretzschmar and colleagues (2009) has argued for similar visual-field dissociations between semantic activation, which was initiated in parafoveal vision, and contextual integration, which requires foveal processing. Our findings complement and extend these results by showing that during sentence processing, in the context of active judgment tasks, plausibility violations elicit robust LPC effects at temporal delays more consistent with foveal than with parafoveal processing.

The functional dissociation of the N400 and LPC, with the former indexing relatively automatic stages of semantic access and the later reflecting higher-level processes of integration and control, is predicted by a number of accounts (Brouwer et al., 2017; Van Petten & Luka, 2012), yet, for semantic violations, these are often observed as bi-phasic activity (increased N400 and LPC to incongruent items). Here, we were able to demonstrate that these effects can be dissociated across the visual field for the same target word. Our findings are consistent with staged models arguing some degree of separation between the initial activation of semantic features and their subsequent integration in sentence contexts (e.g., Staub, 2011; Brouwer et al., 2017) and further suggest that visual attention allocation may be one critical factor contributing to such a dissociation. Taken together, these findings suggest that meaning activation can be initiated parafoveally in advance of direct fixation. However, the later, task-dependent LPC activity is delayed and follows foveal viewing of the target. These differences in the timing and distribution of the N400 and LPC effects across the visual field have interesting implications for mapping onto the timing of similar effects on the eye movement record during reading. Some effects of overt plausibility violations can be detected rapidly in the eye movement record (e.g., Warren, 2011). At the same time, the effects of plausibility constraints persist for considerable time as well, even impacting foveal processing of the word following the plausibility violation (see e.g., Payne, Stites, & Federmeier, 2016). Recent work examining fixation-related potentials suggest that the LPC/P600 to syntactic violations is associated with increased rates of regressive eye movements (Metzner et al., 2016). Whether similar regressive effects would be observed with the foveal LPC to semantic violations is an open question.

Although our design included items that were unexpected but plausible, which have been associated with frontally distributed PNPs (e.g., Federmeier et al., 2007), we did not observe a reliable frontal positivity to prediction violations in parafoveal or foveal vision (see also Stites et al., 2017) in either task. One reason such effects might not manifest in the flanker RSVP design is that the presence of parafoveal information prior to foveal viewing reduces the impact of prediction violations, since at least certain semantic features of the parafoveal word can be activated in parafoveal vision. To the extent that the frontal positivity indexes retroactive revision of processing in the face of a disconfirmed bottom-up signal (Federmeier, 2007; Payne & Federmeier, 2017), it may be that the initial parafoveal semantic preview acts to attenuate the need to engage in active revision once the prediction violation is foveally viewed. Another possibility is that the frontal positivity is task-dependent, but that focusing readers’ attention to semantic plausibility instead of prediction would not specifically modulate the frontal positivity. For instance, Brothers and colleagues (2017), have shown that instructing participants to actively and explicitly predict sentence-final target words results in large frontal positivities to non-predicted words.

4.3. Caveats and Limitations.

There are important costs and benefits to consider with respect to the flanker ERP paradigm. This paradigm has now been adopted quite widely in the literature, and its relative strengths and weaknesses have been addressed across a now substantial number of prior studies (Barber et al., 2011, 2013, 2015; Li et al., 2015; Declerk et al., 2018; Payne et al., 2016; Payne & Federmeier, 2018; Stites et al., 2017; Grainger, 2017; Kornrumpf et al.; Zhang et al., 2015; Snell et al., 2017; Niefind & Dimigen, 2016; Snell et al., 2018a,b,c; Snell & Grainger, 2017, 2018). Nevertheless, there are some important considerations with respect to the current findings that merit review.

In assessing the generalizeability of our findings, it is important to note that most of our sentences were of high contextual constraint, with a very predictable parafoveal target and a short pre-target word. In normal reading, such conditions yield high skipping rates of the pre-target word. In the flanker paradigm, because readers cannot skip these words, readers may therefore allocate disproportiately more attention to the parafoveal target, because they do not need to process the short pre-target word that they would have otherwise skipped. However, it is precisely because the conditions of our experiment are likely to produce greater-than-typical parafoveal processing (consistent with the eye tracking literature, e.g., Kennedy & Pynte, 2005; Schotter et al., 2015), that the lack of an LPC following the parafoveal N400 effect in the judgment task is notable. Our data make clear that even in a case wherein foveal load is especially low, and thus parafoveal processing of the target word should be especially extensive (indeed, where the target word might have even tended to be skipped), LPC effects are not observed until a considerable delay, once the target word has been fixated.

Importantly, although the flanker ERP paradigm is non-naturalistic, it provides a method for continuously tracking brain activity during, between, and after multiple stages of word processing across parafoveal and foveal vision and allowing for the measurement of long-latency and sustained neural responses that play a critical role in higher-order language processing. The ability to examine such late slow potentials across parafoveal and foveal viewing is particularly important given that these late post-N400 potentials provide insights into the neural basis of key aspects of higher-order language processing (Van Petten and Luka, 2012). Thus, despite the limitations, the flanker ERP paradigm offers a window into the neural dynamics underlying the interaction between parafoveal and foveal word processing during sentence comprehension, effects that are not easily observed with other behavioral or neuroimaging methods.

Finally, it is important to note that we decided not to use a cross-over design for the plausibility manipulation. This was an intentional design choice driven by concerns that task instructions tend to carry over from active to passive blocks (e.g., Fallon et al., 2006) such that, after performing a judgment task, participants do not read “passively” in the same way they would have before an explicit task. However, this means that the LPC pattern could have been influenced by the fact that the judgment task always occurred second. For example, it is possible that LPC effects were augmented by the subject’s increased experience with the materials and the flanker paradigm. The fact that the participants were more familiar/practiced however, makes the observed delay of the LPC in the current study even more striking.

4.4. Conclusion.

Studies measuring eye-movement control in natural reading and event-related brain potentials during RSVP reading have converged on the finding that information extraction and integration across parafoveal and foveal vision during reading is a temporally extended process involving the interaction between attention, vision, and higher-order language comprehension mechanisms. The current study adds to this literature by reporting the first examination of higher-order semantic integration processes in the parafoveal flanker paradigm and revealing task-and visual field dissociations of semantic access (reflected by the N400) and later semantic integration (reflected in the LPC). In particular, replicating prior work, we find that semantic access is initiated parafoveally and that it facilitates subsequent foveal processing of the word. Importantly, we here show for the first time that later, task-dependent and more integrative aspects of processing, are not elicited with their usual time course for words in parafoveal preview. Instead, the timecourse of these effects suggests they are only initiated once the word is processed foveally. This pattern is consistent with the literature tracking eye movements during natural reading, which finds evidence for some degree of parafoveal semantic processing but also suggests limitations on the impact of that semantic processing for immediate reading behavior (Dambacher et al., 2007; Schotter et al., 2015; Veldre & Andrews, 2015).