Anticipating syntax during reading: Evidence from the boundary change paradigm

Abstract

Previous evidence suggests that grammatical constraints have a rapid influence during language comprehension, particularly at the level of word categories (noun, verb, preposition). These findings are in conflict with a recent study from Angele, Laishley, Rayner & Liversedge (2014), in which sentential fit had no early influence on word skipping rates during reading. In the present study, we used a gaze-contingent boundary change paradigm to manipulate the syntactic congruity of an upcoming noun or verb outside of participants’ awareness. Across three experiments (total N=148) we observed higher skipping rates for syntactically valid previews (The admiral would not confess…), when compared to violation previews (The admiral would not surgeon…). Readers were less likely to skip an ungrammatical continuation, even when that word was repeated within the same sentence (The admiral would not admiral…), suggesting that word-class constraints can take precedence over lexical repetition effects. To our knowledge, these results provide the first evidence for an influence of syntactic context during parafoveal word recognition. Based on the early time-course of this effect, we argue that readers can use grammatical constraints to generate syntactic expectations for upcoming words.

During sentence comprehension, skilled readers can process language with remarkable speed, even in the face of processing challenges such as lexical ambiguity and degraded visual input. To cope with these challenges, readers can rely on a set of probabilistic constraints - learned over a lifetime of language exposure - to help anticipate and organize incoming perceptual information. By combining multiple sources of information from the surrounding context, anticipatory constraints help prepare the language comprehension system to process upcoming material both quickly and accurately.

For languages with a strict word order preference like English and Mandarin, the grammatical constraints of a preceding sentence can provide an important cue for identifying the syntactic category of an upcoming word (e.g. noun, verb, preposition, determiner). These word-class constraints differ in many ways from semantic constraints, which are usually discussed in terms of lexical predictability or cloze probability (Taylor, 1953). While semantic constraints are often specific, selecting a small number of lexical items for pre-activation (We could tell he was angry from the tone of his… voice), syntactic cues instead provide weaker constraints, allowing for a wider scope of pre-activation (Bill wanted to … [verb]; Jane bought the … [noun]). Additionally, while semantic constraints are somewhat sparse – often occurring at the ends of clauses or in highly stereotyped phrases - word-class cues are more pervasive, with each word in a sentence providing expectations for the next word’s syntactic category. While a wide range of behavioral and neuroimaging evidence suggests that semantic constraints play a critical role in language comprehension (Ehrlich & Rayner, 1981; Rayner & Well; 1996; Kutas & Federmeier, 2000; Staub, 2015), it is less certain whether anticipatory word-class constraints are also computed during natural reading.

Some of the earliest evidence for the use of word-class constraints comes from the lexical decision task. In these studies, participants are faster to identify words when they appear at the end of syntactically congruent sentence contexts (If your bicycle is stolen you must formulate/batteries), and this occurs even in the absence of constraining semantic information (Wright & Garrett, 1984). These results have been replicated frequently using both lexical decision and word naming tasks (West and Stanovich, 1986). Word naming - because of its relative speed – is less likely to be influenced by delayed, post-lexical integration effects (Seidenberg, Waters, Sanders & Langer, 1984). Because syntactic congruity manipulations also have an influence on naming times, this suggests that word-class information may influence very early stages of lexical processing.

Additional support for the rapid accessibility of word-class constraints comes from the speed-accuracy tradeoff task (McElree & Griffith, 1995). In this task, participants make speeded acceptability judgments to different types of linguistic violations. By tracking the accuracy of these acceptability judgments over a range of response deadlines (10ms-3000ms), it is possible to generate speed-accuracy trade-off functions to calculate the earliest time-point when readers were sensitive to a particular type of violation. These functions show that readers begin accumulating information about word-class and subcategorization constraints (Some students commonly laugh/exams) approximately 100 milliseconds before they begin accumulating information about semantic and thematic acceptability (Some students love/infuriate exams).

This timing difference is also consistent with neurophysiological studies of word-class processing. In studies recording event-related potentials, participants show a rapid neural response to violations of word-class constraints. This early, left-anterior negativity (ELAN) has been observed within 100ms post-stimulus onset (Neville, Nicol, Barss, Forster & Garrett 1991; Friederici, Pfeifer & Hahne 1993; Hasting & Kotz, 2008), and precedes the neural indices for other types of linguistic violations such as semantic mismatches (N400) or gender agreement violations (P600); although see Steinhauer and Drury (2012) for conflicting views. Word-class constraints also appear to have a special functional primacy during sentence processing. After encountering a break in a sentence’s ongoing phrase structure, readers show diminished or “blocked” semantic processing for an ungrammatical continuation (Friederici, Steinhauer & Frisch, 1999; Frisch, Hahne & Friederici, 2004). For example, in German, semantic violations produce a large N400 effect (The priest was built), but this N400 difference is eliminated if the critical verb also produces a word-class violation (The priest was by the built). These “syntactic blocking” effects are not typically observed for other types of grammatical violations (Hagoort, 2003, Martín-Loeches, et. al., 2006), again suggesting a special role for word-class.

In addition to arguing for the rapid integration of word-class information, some models of sentence processing go a step further, suggesting that readers also generate syntactic predictions about upcoming language input (Bohnage, Meuller, Friedrici & Fiebach, 2015; MacDonald, Pearlmutter & Seidenberg, 1994; Hale, 2001; Levy, 2008; Chang, Dell, Bock, 2006; see Traxler 2014 for a review). According to these models, readers can combine statistical regularities of the language and features of the current context to anticipate the likely syntactic category of upcoming words. The success or failure of these predictions has been hypothesized as a primary source of bottom-up processing difficulty during reading comprehension (Levy, 2008), and it has been argued that syntactic prediction errors are critical for both language learning and syntactic priming (Chang, Dell & Bock, 2006).

One critical method for investigating the use of anticipatory word-class constraints is eye-tracking during reading. During natural reading tasks, word skipping rates are one of the earliest measures known to be sensitive to lexical and contextual factors (Driegh, Rayner & Pollatsek, 2005; Schotter, Angele & Rayner, 2012). The likelihood that a reader will skip over a word without any first-pass fixations is known to be influenced by both low-level variables such as word length, as well as higher-level variables like word frequency, repetition, and word predictability. Word skipping is thought to occur when an early stage of lexical access has been completed for a parafoveal target word, causing a reader to direct their gaze further along in the text (Reichle, Pollatsek & Rayner, 2006). If word-class constraints are computed in an anticipatory fashion, and if this information can influence lexical access, then word-class congruity effects should be observed on early eye-tracking measures such as word skipping.

Using this methodology, Angele and colleagues (2014) produced a surprising set of results, suggesting that readers may not use syntactic information during the initial stages of lexical processing. This study used a combination of eye-tracking and a gaze-contingent boundary change paradigm (Rayner, 1975) in which readers are given an incorrect preview of an upcoming word in a sentence. This preview remains on the screen until the subject’s eyes cross an invisible boundary, at which point the preview is replaced with the correct target word. In their study, Angele and colleagues presented participants with high frequency (dog) or low frequency (dim) previews that could produce either a valid or an invalid sentence continuation (The increasingly dim/dog light… The excitable dog/dim was…). As in previous studies, subjects were more likely to skip higher frequency previews than low frequency previews, suggesting an early lexical facilitation effect (Rayner, Sereno & Raney, 1996). Critically though, skipping rates in this study were not affected by sentential fit. In other words, subjects were more likely to skip “dog” than “dim” in the sentence “The increasingly dog/dim…” even though this word constituted a grammatical violation. Based on this result, the authors concluded that early oculomotor decisions of when and where to move the eyes during reading are not typically influenced by grammatical constraints.

If this is the case, these results have profound implications for theories of anticipatory sentence processing. A critical question for these theories has been at what level of representation - what “grain size” - readers make predictions about an upcoming message. If factors like lexical predictability have a robust influence on early eye movement measures (Ehrlich & Rayner, 1981; Rayner, Slattery, Drieghe & Liversedge, 2011) but word-class constraints do not, then the answer to this question may be clear: Readers can generate predictions at the lexical level but do not make syntactic predictions at the level of word-class categories.

Before accepting this strong conclusion, some features of this previous study should be discussed in detail. Rather than focusing on grammatical constraints specifically, Angele and colleagues manipulated the general “sentential fit” between a word and its context. The authors reported that 40% of their sentence frames used semantic rather than syntactic constraints to alter the acceptability of an upcoming target word. In addition, this study only used three-letter previews and critical words. While this may have increased the amount of parafoveal preview available to participants, it may have also increased the likelihood that subjects could become aware of the upcoming violation prior to triggering the boundary change. Indeed, participants in this study showed significant skipping differences on a pre-target word (n-1) as a function of syntactic fit, but these early differences were not discussed in detail.

Considering the theoretical importance of this question, we wished to expand upon this previous work with a new set of experiments. In these experiments, we employed a boundary change technique to manipulate the syntactic fit of an upcoming target word in a fully-crossed design. Our first goal was to specifically isolate the influence of word-class constraints during natural reading to determine whether these constraints can come online rapidly enough to influence skipping behavior. Second, we employed a wide range of word lengths (4–7 characters) to determine if previous findings would apply to a more representative sample of words. Finally, we wished to determine whether low-level lexical priming could influence skipping for an upcoming target word (Gordon, Plummer & Choi, 2013), even if this primed word was syntactically invalid (see Angele & Rayner, 2013 and Friederici, Steinhauer & Frisch, 1999 for contrasting predictions).

EXPERIMENT 1

In the current experiment, participants read a series of sentences that required either a verb or noun in a particular sentence position¹. Participants received one of three preview types: a Valid parafoveal preview that was consistent with the grammatical constraints of the sentence (The admiral would not confess…), a Violation preview of the same length and word frequency that violated these constraints (The admiral would not surgeon…), or a Repetition preview that was an ungrammatical repetition of a previously encountered word in the sentence (The admiral would not admiral...).

If early stages of lexical processing are blind to word-class information, we would predict no differences in skipping rates for Valid or Violation preview types (would not confess/surgeon….). In addition, because word repetition has been shown to facilitate early lexical processing (Traxler, Foss, Seely, Kaup & Morris, 2000; Gordon, Plummer & Choi, 2013), a syntax-blind account would also predict the highest skipping rates for ungrammatical repetitions (The admiral would not admiral…). In contrast, if readers do generate predictions for upcoming syntactic categories, we should expect a very different pattern of results. Under this scenario, participants should show higher skipping rates for grammatically Valid previews, and little to no repetition benefit for words that violate word-class constraints.

Participants

Seventy-two UC Davis undergraduates participated in the study for course credit. All participants were native English speakers with normal or corrected-to-normal vision and no history of reading impairments. None of these subjects had previously participated in any norming studies for these materials.

Stimuli

For this experiment, we prepared a set of 54 noun-verb quadruplets (admiral - confess - surgeon - provide). These words were all 4–7 letters in length (mean = 5.4, SD = 1.1) with an average SUBTLEX-US frequency of 34 per million (range= 3 – 165, SD = 36). These items were selected to be either word-class unambiguous (e.g. insist) or strongly biased toward one dominant usage (e.g. sell)¹. Within each quadruplet, all words were identical in length and closely matched in word frequency. We also developed a set of 216 sentence frames that each contained a noun and a verb from one of the quadruplets (The admiral would not confess to any of the charges). The initial prime word (admiral) and the critical target word (confess) were always separated by one or two intervening words (average length = 1.4 words, 6.9 characters). These sentence frames were constructed to be syntactically constraining while keeping the critical words unpredictable in context. To verify that this set of critical words was low in cloze probability, we performed a sentence-completion task with a separate group of 120 subjects. Each participant received a set of 108 sentence frames leading up to the critical target word, and they were asked to fill in the first sentence completion that came to mind. On average, participants produced the critical word less than 1% of the time (average cloze = 0.9%, SD = 3.4%).²

During the main eye-tracking experiment, the initial prime word of each sentence (admiral) always had a valid preview with no boundary change. For the critical target word (confess), participants could receive one of three previews: a Valid preview that was identical to the target word, a word-class Violation preview that was inconsistent with the sentence’s local syntax, or a Repetition preview that was a grammatically unlicensed repetition of the preceding prime word. Following a display change, the critical preview was always replaced with the syntactically correct continuation. To eliminate any lexical or contextual differences across conditions, we fully crossed all preview words and all sentence frames in a Latin-square design. This counterbalancing scheme resulted in a total of 12 experimental lists, which were randomly assigned to participants.

critical noun/verb: (admiral - surgeon - confess - provide)

• 1aThe admiral would not (confess/surgeon/admiral) to any of the charges.
• 1bThey had to confess that the (admiral/provide/confess) had a drinking problem.
• 1cHopefully, the surgeon will (provide/admiral/surgeon) them with more information.
• 1dShe was willing to provide the (surgeon/confess/provide) with all her medical history.

Each participant read 54 experimental sentences with 18 items in each preview condition. These critical sentences were interspersed with 126 filler sentences of varying lengths and grammatical structures, none of which contained boundary changes. Forty-eight of these fillers closely resembled the items used in Gordon, Plummer, and Choi (2013). These stimuli contained either a New or Repeated proper name, with 24 items appearing in each condition: “While looking for Erin and (Ronnie/Calvin), we finally found Calvin under the stairs”. These sentences were included as a control condition to verify that lexical repetition would influence skipping rates within a felicitous sentence context. On average, the critical names (Ronnie/Calvin) were 5.7 characters in length (range = 4–7) and were closely matched in frequency within-items.

Procedure

Eye movements were monitored from the right eye once per millisecond using an SR Research Eyelink 1000 Plus. At the beginning of each session, the eye-tracker was calibrated using a 9-point grid. Tracker accuracy was monitored throughout the experiment, and re-calibrations were performed when calibration error exceeded 0.3 degrees of visual angle. The sentence stimuli were displayed in Consolas font using a Viewsonic P220f monitor. This monitor had a resolution of 1024 x 768 and a refresh rate of 132HZ. Subjects were seated approximately 80cm from the monitor with their chin resting comfortably on a chin rest. At this viewing distance, three characters corresponded to approximately 1° of visual angle.

During the eye-tracking task, participants were asked to read each sentence carefully for comprehension. After one quarter of the trials, subjects were presented with a comprehension question about the preceding sentence. For trials with boundary changes, an invisible, gaze-contingent boundary was placed to the left of the space preceding the critical target word. On average, display changes were completed 7ms after the participants’ eyes crossed this boundary and 5ms prior to the onset of the next fixation. At the end of the experimental session, participants were asked whether they detected any “flashing or flickering” or “any words changing on the screen” while they read. If they responded “yes” to this question, the participant was also asked to estimate the number of times that they detected a boundary change.

Results

Participants’ comprehension question accuracy was uniformly high (mean = 96%, range = 86% – 100). All fixations less than 80ms in duration were either merged with an adjacent fixation within 1 character or else they were discarded. Fixations longer than 1000ms were replaced with this cutoff value (less than 1% of the data). In addition, we excluded any trials where a boundary change was completed more than 10ms after the onset of the next fixation (less than 3% of trials). For our analyses, we calculated a number of early eye-movement measures for both the critical target region, and the pre-target region that separated the prime and target words. We calculated skipping rate (the probability that a region received no first-pass fixations), first-fixation duration (the length in milliseconds of the initial, first-pass fixation falling within a scoring region), and gaze duration (the sum of all first-pass fixations that occurred before exiting a region). Means and standard deviations for these measures can be seen in in Table 1.³

Table 1.

Eye-Tracking Measures from Experiment 1

		Pre-Target	Target	Spillover
Skipping Rate (%)	Valid	28.5 (17.0)	15.9 (11.6)	9.9 (10.6)
	Violation	29.6 (16.2)	12.2 (10.0)	8.5 (9.0)
	Repetition	27.3 (16.3)	13.2 (11.0)	10.4 (11.4)
Regression Rate (%)	Valid	10.3 (9.1)	13.3 (11.5)	11.4 (13.9)
	Violation	8.8 (9.1)	19.1 (13.8)	18.6 (13.0)
	Repetition	9.6 (9.5)	19.6 (15.0)	15.7 (12.9)
First Fixation Duration (FFD)	Valid	222 (40)	228 (34)	230 (32)
	Violation	221 (39)	242 (36)	227 (36)
	Repetition	226 (41)	245 (38)	229 (38)
Gaze Duration (GZD)	Valid	273 (58)	253 (42)	324 (67)
	Violation	278 (73)	276 (52)	316 (68)
	Repetition	287 (73)	280 (46)	322 (82)
Total Time (TT)	Valid	327 (83)	292 (65)	397 (94)
	Violation	370 (124)	334 (80)	408 (103)
	Repetition	371 (116)	342 (83)	401 (104)

Statistical analyses were performed using linear mixed effect models, using the lme4 package in R (Bates, Mäckler, Bolker & Walker, 2015). For each contrast of interest, a maximal mixed-effects model was fit to the data with crossed random slopes and intercepts for both subjects and items. For skipping rate data, we used binomial general linear mixed-effect models with a logit link function. All of these models converged successfully. Reported p-values were obtained using likelihood ratio tests for the reading time data, and Wald Z tests for the skipping rate data.

Skipping Rates

In the pre-target region there were no significant differences in skipping rate across conditions. In contrast, skipping rates on the target word were significantly influenced by preview type (see Figure 1). Skipping rates were higher for grammatically acceptable Valid previews (15.9%) than for either Violation previews (12.2%) or grammatically unlicensed Repetition previews (13.2%). Mixed-effect analyses showed a significant difference between Valid and Violation previews (z = −2.74, p = 0.006) and between Valid and Repetition previews (z = −2.46, p = 0.014). The 1% skipping rate difference between the Violation and Repetition previews did not approach significance (z = 0.34, p = 0.74).

Figure 1.

Average skipping rates for different previews conditions across the three experiments. Valid and Control preview strings were congruent with the syntactic constraints of the preceding context (The admiral would not confess…). In contrast, Violation (The admiral would not surgeon…), and ungrammatical Repetition previews (The admiral would not admiral… ) were of the incorrect word-class. Error bars represent ±1 standard error of the mean, calculated within-subjects (Morey, 2008).

While there was no clear benefit of word repetition for grammatically incorrect previews, we did observe repetition effects for New vs Repeated proper names in the set of control sentences. Skipping rates were higher for Repeated names (14.6%) than for New names (9.4%), and this difference was highly significant (z = −4.4, p < 0.001).

Reading Times

In the pre-target region, there were no significant differences on first fixation time. For gaze durations some significant differences were observed, with longer reading times in the pre-target region for Repetition previews (287ms) than Valid previews (273ms), b = 16ms, t = 2.34, p = 0.022. Reading times in the Violation preview condition (278ms) fell roughly in between the other two means, with neither contrast reaching significance. While this pattern of results may suggest a possible parafoveal-on-foveal effect – with longer parafoveal reading times when an upcoming target is infelicitous – it is unlikely that these effects are syntactic in nature considering the absence of a parafoveal-on-foveal effect when comparing Valid and Violation previews (ts < 1).

On the target region, reading times showed a standard “preview benefit” effect (Rayner, 1975; Inhoff & Rayner, 1986). Target words with Valid previews were read significantly faster than those with Violation previews (first fixation: b = 14ms, t = 3.92, p < 0.001, gaze duration: b = 23ms, t = 4.51, p < 0.001) or Repetition previews (first fixation: b = 18ms, t = 4.87, p < 0.001, gaze duration: b = 27ms, t = 5.53, p < 0.001). Reading times in the Violation and Repetition preview conditions were quite similar, with no significant differences for first fixation (242ms vs 245ms) or gaze duration measures (276ms vs 280ms), all ts < 1.

Finally, the control sentences showed substantial reading time differences between New and Repeated names. Replicating the results of Gordon, Plummer, and Choi (2013), Repeated names showed shorter first fixation (b = 23ms, t = 7.04, p < 0.001) and gaze durations (b = 31ms, t = 6.17, p < 0.001) when compared to New names.

Discussion

In the current study, participants read sentences for comprehension that contained grammatically valid or invalid parafoveal previews. The goal of this study was to assess the online use of syntactic constraints during reading and to determine whether these constraints can influence early stages of lexical access. Contrary to the predictions of a syntax-blind account, we observed higher skipping rates for grammatically valid previews. Moreover, while syntax-blind accounts would predict the highest skipping rates for Repetition previews (The admiral would not admiral…), instead we observed an effect in the opposite direction. Ungrammatical repetitions were skipped less often than novel continuations of the correct word-class (The admiral would not confess…).

Why might we have observed no repetition benefit for ungrammatical nouns and verbs while observing robust skipping differences for New vs Repeated proper names (5%)? Clearly, these two repetition effects should be compared with caution considering that they were generated by different critical words in different sentence frames. Nonetheless, in a combined analysis, we observed a significant interaction between sentence-type (Proper Names vs Noun/Verb) and repetition status (New vs Repeated), z = 2.88, p = 0.004.

There are several possible explanations for this pattern of results. Considering that nouns and verbs have shown clear repetition effects in previous studies (5% skipping difference, Choi & Gordon, 2013), it is unlikely that these effects were driven by inherent lexical differences between proper names and nouns/verbs. A more plausible explanation is that anticipatory word-class constraints were able to override or “block” lexical priming for parafoveal words of the wrong word-class. Under this account, word-class constraints would act as a filter, reducing activation for inconsistent syntactic categories, and biasing the word identification system to identify only syntactically correct continuations.

Alternately, these repetition differences could also be interpreted as a difference in lexical predictability. While repeated names likely have a higher cloze probability than novel names, the same could not be said for ungrammatical Repetition previews. In our cloze norms, both Violation and Repetition previews had cloze probabilities of 0%. Under this account, readers would combine both semantic and syntactic information when generating lexical predictions, and only lexical items that are fully consistent with both sets of constraints would be selected for pre-activation. In many ways the “predictability” and “syntactic blocking” accounts are quite similar; in both cases syntactic constraints are available rapidly and can take precedence over simple word-to-word priming.

Independent of these word repetition effects, we also observed clear differences in skipping rates for grammatical and ungrammatical previews. To our knowledge this is the first demonstration of a syntactic context effect during parafoveal lexical processing. While this result is consistent with previous research on the rapid availability of word-class information, it is also inconsistent with the results of Angele and colleagues (2014) who found no differences in skipping rate as a function of sentential fit. Before discussing these results further, we thought it was important to provide additional empirical evidence for this effect, using a new set of subjects and items.

EXPERIMENT 2

In previous studies, only a small number of factors have been shown to reliably influence skipping rates, including word length, word frequency, predictability, and repetition. In contrast, manipulations of plausibility or syntactic fit are often observed on later “integrative” eye-tracking measures such as first-pass time or regression-path duration (Abbott & Staub, 2015; see Clifton, Staub & Rayner, 2007 for a review). Based on this previous literature of relatively late syntactic effects, the primary goal of Experiment 2 was to provide additional evidence for an early influence of word-class constraints during parafoveal lexical processing. In addition, while Experiment 1 provided no clear signs of a syntactic parafoveal-on-foveal effect, we wished to investigate this question further while avoiding any potential biases from the ungrammatical Repetition condition. It is possible that repetition violations may have attracted additional attention in the parafovea, and we therefore included only Valid and Violation previews in Experiment 2, while substantially increasing the number of items per condition.

Participants

Twenty-eight UC Davis undergraduates participated in this study for course credit. All participants were native English speakers with normal or corrected-to-normal vision and no history of reading impairments. None of these subjects previously participated in any norming studies for these materials.

Stimuli

We prepared a set of 120 noun-verb pairs (food-lose), many of which were included as critical targets in Experiment 1 (100/120). Again, these critical words were all either word-class unambiguous or strongly biased toward one dominant usage. The words in each pair were always identical in length (mean = 5.4 characters, range = 4–7) and closely matched in frequency (mean = 43 per million, range = 2–354 per million). These targets were embedded in a new set of 240 sentence frames, each containing a single critical noun or verb. To ensure that these targets were not predictable in context, we conducted another sentence-completion task with a group of 80 participants. On average, the critical target word was produced less than 1% of the time (average cloze = 0.4%, SD = 1.2%).

• 2aThere’s a chance Henry will (lose/food) in the final lap of the race.
• 2bOnce the generator died, all the (food/lose) in the fridge began to spoil.

During the main experiment, participants could receive one of two previews: either a Valid preview that was identical to the upcoming target word (There’s a chance Henry will lose…), or a Violation preview that did not match the word-class constraints of the preceding context (There’s a chance Henry will food…). Over four counterbalanced lists, all preview words and all sentence frames were fully crossed to generate items in the two conditions. Participants saw 120 sentences in the Valid preview condition, 60 sentences in the Violation preview condition, and 40 fillers of various types which contained no boundary changes. Participants saw an unequal number of trials in each condition in order to reduce the overall number of boundary changes. Trials were presented in a unique randomized order for each subject.

Procedure

All experimental procedures were identical to those employed in Experiment 1. Skipping and reading time results were analyzed for the critical target word, as well as a two-word pre-target region (see Table 3).

Table 3.

Eye-Tracking Measures from Experiment 2

		Pre-Target	Target	Spillover
Skipping Rate (%)	Valid	6.7 (4.3)	15.1 (8.8)	9.8 (7.6)
	Violation	7.0 (5.9)	11.8 (6.8)	8.7 (8.6)
Regression Rate (%)	Valid	7.9 (4.1)	9.1 (6.8)	8.4 (4.4)
	Violation	8.5 (5.2)	16.2 (10.6)	12.6 (7.1)
First Fixation Duration (FFD)	Valid	214 (24)	223 (24)	223 (25)
	Violation	212 (26)	240 (26)	225 (27)
Gaze Duration (GZD)	Valid	293 (50)	248 (35)	311 (47)
	Violation	294 (55)	271 (36)	316 (53)
Total Time (TT)	Valid	357 (78)	283 (44)	365 (62)
	Violation	385 (88)	316 (48)	391 (68)

Results

Subjects’ comprehension accuracy was quite high (mean = 94%, range = 88% – 100%). All data trimming procedures and analysis methods were identical to Experiment 1. Again, we used mixed effects models with a maximal random effects structure, and none of the models showed issues with convergence.

Skipping Rates

In the pre-target region there were no significant differences in skipping rate across conditions. In the critical target region, we observed a syntactic congruity effect that closely replicated the results of Experiment 1. Participants were more likely to skip Valid previews (15.7%) than Violation previews (12.3%), and this difference was significant (z = −3.28, p = 0.001).

Reading Times

In the pre-target region we again observed no evidence of a syntactic parafoveal-on-foveal effect. Reading times in this region were essentially identical across conditions, with no differences in first fixation (Valid: 214ms, Violation: 212ms) or gaze durations (Valid: 293ms, Violation: 294ms), ts < 1. In the target region we observed a standard preview benefit effect, with shorter reading times for Valid previews (first fixation: b = 17ms, t = 5.15, p < 0.001, gaze duration: b = 23ms, t = 6.12, p < 0.001).

Discussion

The results of Experiment 2 replicated the finding that readers are more likely to skip grammatically valid sentence continuations that are congruent with the preceding syntactic context. This provides additional evidence that readers can rapidly compute word-class constraints during online sentence processing and use these constraints to influence parafoveal word recognition. One potential caveat to these results is that, in both experiments, syntactic Violation previews were always followed by a boundary change, while no boundary changes occurred in the Valid preview condition.⁴ While previous research suggests that the presence of an upcoming boundary change does not influence skipping decisions (Risse & Kliegl, 2014), nonetheless, we wished to rule out this potential confound in Experiment 3 by comparing skipping rates to a grammatically-valid, boundary change Control condition. If the skipping differences observed in Experiments 1 and 2 were caused by the presence of a boundary change, we would expect to see no skipping differences between Violation previews and Control previews which both trigger boundary changes. In contrast, if skipping rates are indeed influenced by the syntactic constraints of the preceding sentence, then we should see reduced skipping rates for Violation previews relative to both Valid and Control preview strings.

EXPERIMENT 3

Experiment 3 was essentially a direct replication of Experiment 2 with the addition of a grammatically-valid, boundary change Control condition (see Appendix B for examples). Participants read sentences for comprehension with three types of previews: Valid previews that were identical to the upcoming target word (She was angry because the coffee…), Control previews that were grammatically correct but did not match the upcoming target (She was angry because the ladies…), and Violation previews that violated the word-class constraints of the preceding context (She was angry because the expect…). Following a boundary change, participants always saw a grammatically correct target word (coffee).

Participants

Forty-eight UC Davis undergraduates participated in this study for course credit.

Materials and Procedure

Some of the sentence frames from Experiment 2 had to be rewritten so that two critical words from each quadruplet (coffee-ladies-expect-follow) could plausibly continue the same sentence frame (e.g. She was angry because the coffee/ladies… - Her younger brother would always expect /follow…). Otherwise, the critical target words and sentence frames were identical to those used in Experiment 2. Over twelve counterbalanced lists, all preview words and sentence frames were fully crossed to generate items in the three conditions. During the experiment, participants saw 80 Valid previews, 40 Control previews, and 40 Violation previews, which were randomly interspersed with 44 filler sentences containing no boundary changes. The experimental procedures were identical to the previous experiments.

Results

Participants answered 95% of all comprehension question correctly (range = 83% - 100%). Data trimming criteria were identical to experiments 1 and 2, and they affected fewer than 3% of trials.

Skipping Rates

No significant differences in skipping rate were observed in the pre-target region (See Table 4). On the critical target word, skipping rates were higher for grammatically Valid previews (15.3%) and Control previews (14.3%) relative to Violation previews (11.4%). Mixed-effect analyses showed a significant difference between Valid and Violation previews (z = −4.10, p < 0.0001) and between Control and Violation previews (z = −3.30, p < 0.001). The presence of a boundary change alone (Valid vs Control) resulted in a 1% skipping rate difference which did not reach significance (z = −1.44, p = 0.15). This pattern of results suggests that the skipping rate effects observed in Experiments 1 and 2 were not a byproduct of the boundary change itself, but were caused by the match or mismatch between syntactic constraints of the preceding context and the syntactic category of an upcoming target word.

Table 4.

Eye-Tracking Measures from Experiment 3

		Pre-Target	Target	Spillover
Skipping Rate (%)	Valid	6.9 (6.7)	15.3 (9.8)	10.7 (9.1)
	Control	7.0 (7.1)	14.3 (9.9)	9.2 (9.3)
	Violation	6.8 (7.6)	11.4 (10.2)	10.2 (9.7)
Regression Rate (%)	Valid	11.0 (8.1)	14.4 (11.9)	11.6 (7.9)
	Control	13.1 (11.6)	17.2 (13.8)	19.3 (10.7)
	Violation	12.4 (9.5)	22.4 (15.0)	18.5 (13.3)
First Fixation Duration (FFD)	Valid	213 (27)	225 (26)	224 (27)
	Control	213 (29)	234 (36)	227 (29)
	Violation	215 (28)	235 (36)	224 (29)
Gaze Duration (GZD)	Valid	304 (63)	251 (34)	330 (64)
	Control	301 (62)	270 (47)	330 (57)
	Violation	306 (73)	272 (46)	326 (65)
Total Time (TT)	Valid	403 (133)	306 (77)	416 (112)
	Control	423 (147)	353 (92)	448 (123)
	Violation	439 (147)	342 (87)	440 (134)

Reading Times

As before, no reading time differences were observed in the pre-target region. On the target region there were consistent reading time benefits for Valid previews relative to both conditions containing boundary changes (Valid vs Control, first fixation duration: b = 9ms, t = 3.22, p = 0.002, gaze duration: b = 20ms, t = 5.06, p < 0.001; Valid vs Violation, first fixation duration: b = 11ms, t = 3.08, p = 0.003, gaze duration: b = 22ms, t = 4.79, p < 0.001). Reading times did not differ between the Control and Violation conditions (all ts < 1). Based on previous boundary studies investigating semantic and phonological overlap (Schotter, 2013; Pollatsek, Lesch, Morris & Rayner, 1992) one may have expected that shared word category information between previews and targets would have reduced reading times in the Control condition. Based on the current data, this does not appear to be the case. It is possible that, in a constraining syntactic context, word-class information was already highly pre-activated, and therefore congruent word-class information in the parafovea did not lead to additional facilitation in the Control condition.⁵

Combined Data

Across three experiments, we observed consistent evidence for the rapid use of word-class constraints during online sentence comprehension. One critical question is whether this syntactic skipping effect is pre-lexical or post-lexical in nature. In other words, does syntactic context exert its influence only after a word in the parafovea has been completely identified? To investigate this, we performed a follow-up analysis to determine whether lexical characteristics of the critical preview (word length and word frequency) would influence the magnitude of the syntactic skipping effect.

In this analysis, we combined skipping rate data from Valid and Violation conditions across all three experiments, for a total of 148 participants and 10978 critical trials. Although we did not manipulate lexical characteristics directly in these tasks, the critical target words spanned a representative range of frequency and length values, and - by design - these two lexical variables were not significantly correlated, r(254) = −0.02, p = 0.7. Skipping rates for the critical target region were entered into a maximal, binomial general linear mixed-effect model, with word length, word frequency, syntactic congruity, and their higher order interactions entered as predictor variables. All of these predictors were mean-centered prior to analysis.

Consistent with previous findings, readers were more likely to skip shorter words (z = −11.97, p = 2 × 10⁻¹⁶) and words with higher frequencies (z = 5.16, p = 2.5 × 10⁻⁷). There was also a highly significant main effect of syntactic congruity (z = −5.83, p = 5.6 × 10⁻⁹; average effect: 3.7%, 95% CI: [2.2% – 5.2%]). Interestingly, we observed no significant two-way or three-way interactions (zs < 1.5). Based on this result, it appears that these three factors can operate independently during early lexical processing. While suggestive, these results are nonetheless preliminary, and we believe this question should be investigated more closely in future work using a broader range of length and frequency values (e.g. see Rayner, Sereno & Raney, 1996 for evidence of a length by frequency interaction on skipping rates).

We also ran an additional set of analyses to investigate two additional factors: 1) the cloze probability of the parafoveal preview, and 2) whether subjects detected any boundary changes during the course of the experiment. Cloze probability values were obtained from our set of offline sentence completion norms. Valid previews had an average probability of 0.6% (SD = 2.5%), while Violation previews all had a cloze probability of 0%. In the post-experiment debriefing, 41% of participants responded “yes” to the question “Did you detect anything flashing or flickering, or any words changing on the screen as you read?” On average, this subset of participants reported detecting a small number of boundary changes overall (3.2 trials, range = 1–8).

In these analyses, cloze probability had no significant influence on skipping decisions (z = 0.40, p = 0.69) likely due to the very narrow range of cloze values in the present stimulus set. Skipping rates also did not differ between participants who did or did not detect any of the boundary changes (z = 0.60, p = 0.55). In both analyses, syntactic validity had a significant effect on skipping rates (all zs > 6, ps < 0.0001), which did not differ across groups (z < 1). This suggests that detecting a small number of boundary changes does not influence a participants’ skipping behavior or the size of the syntactic congruity effect. Moreover, it appears that very small differences in cloze probability (<1%) cannot explain the skipping rate differences between syntactically correct and incorrect previews.

General Discussion

Previous studies have shown that a number of factors can influence parafoveal word identification, including orthographic, phonological, morphological and semantic information (see Schotter, Angele & Rayner, 2012, for a review). Critically, syntactic constraints and word-class information in general have not been included in eye-movement models as mediators of online reading behavior. To the contrary, the present results suggest that word-class information plays a critical role in online reading comprehension and can influence even the earliest stages of word recognition.

In the present study, readers were more likely to skip words that were syntactically congruent with the preceding context. While these early syntactic effects are consistent with previous studies of word-class processing, the present experiments also go a step further by combining natural reading with the boundary change technique. A serious critique of previous behavioral and ERP studies investigating this topic is that participants were often exposed to a large number of overt, word-class violations. Because of this, participants in these tasks may have developed specific strategies for detecting violations that would not be used during typical reading comprehension (see McElree & Griffith, 1995 for a discussion). The present experiments avoided this issue by only presenting word-class violations in the parafovea and rapidly triggering a boundary change before the critical words were directly fixated. Because subjects were not consciously aware of these grammatical errors, the present results provide even stronger evidence that readers naturally pre-activate word-class information during reading.

Anticipation or Integration?

While the present syntactic skipping effect was observed on a very early eye-movement measure, it is still important to consider whether these effects are truly anticipatory in nature, or whether they could also be explained by a post-lexical integration mechanism.

Under an anticipatory account, readers would use a preceding sentence context to pre-activate word-class information for upcoming target words. These syntactic constraints could then be used to narrow the search space for potential word candidates during lexical access, and congruent words with the correct syntactic features could be recognized more rapidly. Because syntactic constraints can provide facilitation during lexical access in the parafovea, this should result in increased word skipping rates for syntactically congruent continuations.

In contrast, an integrative explanation would unfold quite differently. Under this account, a reader would partially identify a word in the parafovea and program a saccade to skip it. While programming this saccade, the reader would then complete lexical access (identifying the word’s syntactic category) and begin integrating this word into the preceding context (Reichle, Warren & McConnell, 2009). For an ungrammatical continuation, this integration process would fail, and the reader would cancel their current skipping saccade in order to directly fixate the parafoveal target. In this way, rapid integration failures could also account for a difference in skipping rates, as a result of aborted skipping saccades.

One obvious critique of this integration account is that there is simply not enough time during natural reading for sentence integration mechanisms to influence skipping decisions. In most theories and computational models of eye-movements, saccade programs can only be cancelled during an early labile stage. After this window has passed, a ballistic eye-movement cannot be cancelled or interrupted (Hanes & Carpenter, 1999; Pollatsek, Reichle & Rayner, 2006). Based on the inherent timing constraints of the oculomotor system, is unlikely that both full identification and contextual integration of a parafoveal word could be completed within this labile window.

To formally investigate this question, Staub (2011) performed a series of simulations using the E-Z Reader 10 model of eye-movement control which incorporates a post-lexical integration stage (Reichle, Warren & McConnell, 2009, see also Abbott & Staub, 2015). In these simulations, integration difficulty was shown to have an influence on first fixation times on critical target words, which is consistent with prior work on semantic plausibility (Warren & McConnell, 2007). Critically, these simulations showed that integration difficulty had little to no influence on skipping rates - except for very short target words which could be rapidly identified in the parafovea. The anticipated null-effect of semantic plausibility on skipping rates was later confirmed experimentally by Abbott and Staub (2015) in a large sample of readers.

If word-class constraints also operate during a post-lexical integration stage, then we would expect no skipping differences across conditions, similar to Abbott and Staub’s semantic plausibility manipulation (The man angered the stapler…). In contrast, syntactic congruity produced robust differences in skipping in the current experiments, and these differences remained relatively constant across a range or word lengths (4–7 characters, see Figure 2). A similar pattern has also been observed in studies investigating lexical predictability during reading. In these studies, the effects of a predictive context on skipping also remained constant across different levels of word length and frequency (Rayner, Ashby, Pollatsek & Reichle, 2004; Rayner, Slattery, Drieghe & Liversedge, 2011). Based on these null interactions, it has been suggested that lexical predictability operates at a distinct, pre-lexical processing stage (see Staub, 2015 for an in-depth discussion). If the same holds true for word-class constraint effects – as was observed in our combined regression analysis – it may suggest that these syntactic effects are also anticipatory in nature.

Figure 2.

The effects of syntactic congruity (The admiral would not confess/surgeon…) on skipping rates as a function of word length. Data was combined across all three experiments. Error bars represent ±1 standard error of the mean, calculated within-subjects.

There is one additional piece of evidence from the current data that is inconsistent with an integration account. If early syntactic skipping differences were the result of aborted skipping saccades, then grammatical violations should also produce inflated reading times in the pre-target region (reflecting the extra time needed to plan and execute a new saccade program; Reichle & Drieghe, 2013). In the present experiments, we observed no reading time differences in the pre-target region between Valid and Violation previews, suggesting that cancelled saccades were not the cause of the syntactic congruity effect.⁶ In sum, the current data align most closely with an anticipatory account in which readers 1) pre-activate syntactic constraints for upcoming words, and 2) use these constraints immediately to facilitate early stages of lexical identification.

The case of “the” skipping

The present results are also relevant to prior studies investigating skipping rates for the letter string “t-h-e” in felicitous and infelicitous contexts (Angele & Rayner, 2013). In sentences like the following (She knew that she would the/ace…) readers are more likely to skip an infelicitous preview of the word “the” than a correct preview like “ace”. While these results have been interpreted to suggest that readers are initially insensitive to syntactic cues, there are several alternate variables which may have produced this skipping effect, including the large frequency differences between “the” and other word types, and differences in skipping decisions for content and function words. Word length also may play a role, with syntactic manipulations on extremely short three-letter words resulting in syntactic skipping differences on even earlier, pre-target regions (Angele & Rayner, 2013; Angele, Laishley, Rayner & Liversedge, 2014).

One way to reconcile the present results with these previous findings is a model in which both lexical familiarity and word-class constraints can jointly influence word skipping decisions. While the effects of word frequency appear to take precedence for skipping decisions on relatively short function words, in other circumstances syntactic constraint effects are more pronounced and can even override word familiarity effects (such as the lexical repetition effect in Experiment 1). Clearly, more work is necessary to determine all the factors that can influence skipping decisions for short and highly familiar function words, particularly when these different cues come into conflict.

Syntactic Constraints and Comprehension

A final important question is why readers generate anticipatory word-class constraints in the first place - especially considering that word-class violations are exceedingly rare in natural speech and text? What online processing role do they serve?

One important role for word-class constraints is lexical disambiguation. Languages with fixed word orders such as English also have a large number of word-class ambiguities at the lexical level (Hawkins, 2012). Ambiguities of this type are only possible because they can be easily resolved using syntactic context (he liked to play, he liked the play). By pre-activating appropriate syntactic categories during comprehension, readers can avoid activating inappropriate meanings of words. For example, Folk and Morris (2003) showed slower reading times for noun-noun ambiguities (Ella saw the pitcher…), but no costs for noun-verb ambiguities (park) that appeared in a syntactically disambiguating context (Bill said that the park…). By using anticipatory word-class constraints, languages with fixed word-orders can afford higher degrees of cross-class lexical ambiguity. This in turn allows for more word senses and a denser, more expressive vocabulary, without the addition of online processing costs.

Another potential benefit of word-class constraints is the rapid detection and repair of garden path sentences. While processing a garden path sentence, readers can initially commit to one incremental parse that they later realize is incorrect. In a sentence like “the florist sent the flowers was pleased”, readers can rapidly detect that their initial parse is incorrect after encountering the disambiguating word “was” (Frazier & Rayner, 1982; see also Pickering, Traxler & Crocker’s, 2000, testability preference principle). The detection of this ambiguity can occur quickly, likely because this word produces an immediate word-class violation. By providing an early warning sign of structural difficulty, the detection of this temporary violation can trigger regressive saccades to allow for rapid re-analysis.

Finally, word recognition itself is often a noisy and error-prone process (Gibson, Bergen & Piantodosi, 2013; Levy, Bicknell, Slattery & Rayner, 2009). This is especially true for parafoveal word recognition when visual acuity is degraded (Brysbeart, Vitu & Schroyens, 1996). For example, words with a large number of orthographic neighbors trigger more regressive saccades and longer re-reading times, likely due to an incorrect initial interpretation (reading “birch” as “birth”; Slattery, 2009). In the face of noisy bottom-up input, selecting between two competing lexical candidates (sing vs song) can often be enhanced by incorporating anticipatory word-class information. By narrowing the scope of possible word candidates, even during the earliest stages of visual word identification, reading can proceed more quickly and accurately. Based on the current results, we suggest that word category information and anticipatory syntactic constraints are critical features of reading comprehension. Therefore, any comprehensive theory of eye-movement control during reading should take these factors into account.

Table 2.

Eye-Tracking Measures from control sentences (novel vs repeated names) in Experiment 1

		Pre-Target	Target
Skipping Rate (%)	Novel	12.7 (10.5)	9.4 (9.2)
	Repeated	11.7 (10.6)	14.6 (10.3)
Regression Rate (%)	Novel	11.7 (7.9)	20.2 (12.7)
	Repeated	13.2 (10.2)	18.0 (13.1)
First Fixation Duration (FFD)	Novel	250 (38)	237 (36)
	Repeated	249 (38)	214 (31)
Gaze Duration (GZD)	Novel	294 (59)	263 (51)
	Repeated	295 (60)	231 (35)
Total Time (TT)	Novel	407 (131)	351 (104)
	Repeated	378 (107)	288 (65)

Appendix A

Example sentences from Experiment 1. Each subject saw only one sentence frame from each quadruplet, paired with one of the three potential previews (Valid/Violation/Repetition). Following a boundary change, incorrect preview strings were always replaced by the grammatically correct critical word. The full set of materials is available from the first author upon request.

• 1aAfter the meeting, our client had to (accept/cousin/client) the new contract.
• 1bThey were willing to accept the (client/decide/accept) because he was rich.
• 1cAfter dinner, my cousin couldn’t (decide/client/cousin) if he wanted more pudding.
• 1dI can’t decide whether my (cousin/accept/decide) is reckless or just stupid.
• 2aSometimes the valve would (seize/shops/valve) up because it was rusty.
• 2bThe woman tried to seize the (valve/adopt/seize) and twist with both hands.
• 2cThe woman who owned the shops wanted to (adopt/valve/shops) a puppy to attract customers.
• 2dThe couple decided to adopt after their (shops/seize/adopt) started turning a profit.
• 3aThe men who repaired the hull won’t (dine/nuns/hull) on the deck of the ship.
• 3bThe sailors would dine in the (hull/undo/dine) of the ship when it was raining.
• 3cAfterward, the nuns tried to (undo/hull/nuns) all the damage caused by sister Kennedy.
• 3dIt was difficult to undo what the (nuns/dine/undo) had caused that fateful afternoon.
• 4aThe man holding the beer tried to (grab/song/beer) a bottle opener from his friend.
• 4bI made sure to grab another (beer/join/grab) from the cooler on the porch.
• 4cThe writer of the song wanted to (join/beer/song) our band, but we rejected him.
• 4dThe band asked me to join after their (song/grab/join) appeared on the radio.
• 5aIf you visit the airport, I would (suggest/bedroom/airport) arriving two hours early.
• 5bSome legislators suggest that the (airport/prepare/suggest) should be closed.
• 5cThe kids in the bedroom should (prepare/airport/bedroom) the beds for the guests.
• 5dWe needed to prepare the (bedroom/suggest/prepare) for the visiting guests.

Appendix B

Example sentences used in Experiment 2 and 3. Each sentence is presented with the three possible preview conditions (Valid/Control/Violation). In Experiment 2 participants saw only Valid and Violation previews, and in Experiment 3 they saw all three preview types. Previews always changed to the first critical word following a boundary change.

• 1aWithout asking permission, the man tried to (grab/join/beer) the phone from my hands.
• 1bWe didn't want Jack and Linda to (join/grab/song) our secret club in the treehouse.
• 1cThe official knew that the (beer/song/grab) and liquor would be searched at the border.
• 1dThe author really liked that (song/beer/join) because it had a catchy beat.
• 2aThe Egyptian government failed to (approve/include/workers) a budget for the next fiscal year.
• 2bIn the end, they didn't (include/approve/luggage) her photo on the cover of the magazine.
• 2cJan heard that the (workers/luggage/approve) had been rescued from the bottom of the mine.
• 2dWe thought it was exciting that the (luggage/workers/include) had an anti-theft alarm.
• 3aThe prince had started to (drown/bleed/mummy) and none of us were able to help him.
• 3bWhen Andrew started to (bleed/drown/mummy) on the mattress, we knew his wound was serious.
• 3cThe women were disappointed that the (comic/mummy/drown) did not have any new material.
• 3dThere was very little chance that the famous (mummy/comic/bleed) would ever be recovered.
• 4aThe detective was sure that no one could (confirm/replace/schools) my father's alibi.
• 4bIt took several graduate students to (replace/confirm/sweater) the light bulb in the hallway.
• 4cMy grandmother thought that the (schools/sweater/confirm) would be open on Labor Day.
• 4dThey spent so much money on the (sweater/schools/replace) that they couldn't buy the mittens.
• 5aHe was going to (deny/bury/maid) the allegations on prime time television.
• 5bThe convict started to (bury/deny/wolf) the body in the garden out back.
• 5cWhen we first saw the (maid/wolf/deny) we thought she had an honest face.
• 5dJohn was afraid that the (wolf/maid/bury) would continue to approach the tent.