Investigating effects of selectional restriction violations and plausibility violation severity on eye-movements in reading

Abstract

This paper presents a study investigating whether and how different kinds of knowledge affect the detection of plausibility and possibility violations. Readers’ eye-movements were monitored while reading sentences describing impossible events cued by selectional restriction violations, extremely implausible events without selectional restriction violations, and plausible events, in order to determine whether the time course of disruption is determined by overall implausibility/unlikelihood, or whether impossibility cued by selectional restriction violations additionally affects disruption. Both early and late fixation measures showed stronger disruption in the impossible/selectional restriction violation condition. However, measures indexing regressive eye-movements showed similar disruption in both extremely implausible conditions. This suggests that the magnitude and latency of disruption to possibility and plausibility violations is not a simple function of the overall implausibility/unlikelihood of the resulting event, but that selectional restriction violations influence the early and late time course of disruption.

In order to understand reading comprehension, it is important to know when different kinds of knowledge are used during reading. Many kinds of knowledge influence comprehension, including information from: syntax, the lexicon, schemas, and the local context. The timing of availability and use of these kinds of knowledge has been relatively well studied, as it has ramifications for the architecture of the language processing system (Cook & Myers, 2004; Garrod & Terras, 2000; Grodner, Gibson & Watson, 2005; Rayner, Carlson, & Frazier, 1983; Trueswell, Tanenhaus & Garnsey, 1994).

During comprehension, these multiple sources of information interact, allowing the computation of higher-order knowledge. For example, knowledge of an event's possibility or impossibility can be informed by linguistic, contextual, and/or world knowledge. An event such as John inflated the carrots may be judged impossible because lexical knowledge indicates that the verb inflated requires an inflatable object, because the context restricts carrots to refer to non-inflatable vegetables, and/or because carrot-inflating events are never encountered. Plausibility is similarly determined by the interaction of multiple sources of information, and investigators have begun to disentangle the relative effects of these sources of information on judgments (Connell & Keane, 2004), and to determine their time courses of use (Cook & Myers, 2004; Garrod & Terras, 2000).

The current experiment investigates whether the latency and amount of disruption caused by a plausibility/possibility violation is determined by the unlikelihood of the resulting event given all available knowledge, or whether there are kinds of knowledge that are privileged in violation detection. One candidate for such privileged knowledge is a predicate’s selectional restrictions, which may be represented internal to the lexicon, and thus be available earlier than non-lexical sources of information (Katz & Fodor, 1963). Selectional restriction violations lead to impossibility, as in John drank a tree, in which the verb drink requires a liquid object but a tree is not liquid. If such information is among the earliest available, then selectional restriction violations should be detected earlier than violations cued by contextual or world knowledge. However, many researchers question the existence of boundaries between lexical, contextual, and world knowledge (e.g. Jackendoff, 2002). If all of these sorts of knowledge are equally accessible, they could jointly affect impossibility/implausibility detection through a general measure like the unlikelihood of an event. In this case, implausibility and impossibility might represent different points on a scale of unlikelihood or unexpectedness rather than separate constructions.

Experiments by Marslen-Wilson, Brown, and Tyler (1988) and Hagoort, Hald, Bastiaansen, and Petersson (2004) are relevant to these questions. Marslen-Wilson et al. (1988) had participants listen to sentences while monitoring for a target word. In the relevant conditions, the target word was an argument, which either violated a verb’s selectional restrictions (e.g. guitar in John drank the guitar) or was pragmatically unlikely (e.g. guitar in John buried the guitar). Participants were slower to detect the target word in the restriction-violating condition, but this result is confounded by the fact that this condition was also the most implausible/unlikely. Hagoort et al. (2004) used ERP and fMRI methods to compare the processing of true sentences (yellow below), false sentences (white), and sentences with a violation of a predicate’s selectional restrictions (sour).

1) The Dutch trains are {yellow/white/sour} and very crowded.

Hagoort et al. measured N400s to the critical predicate adjective and found no differences between the anomalous versus false conditions, as well as no differences in fMRI profiles. If N400s reflect the difficulty of integrating a word into a semantic representation, these results suggest that selectional restriction and world knowledge violations cause similar integration difficulty. This finding is consistent with the hypothesis that a single construct like unlikelihood or violation of expectancy predicts disruption, because the false and anomalous statements described equally unexpected states. However, if the N400 indexes lexical mismatches independent of a statement’s semantics (Fischler, Bloom, Childers, Roucos, & Perry, 1983), then these results may not bear on the issue of knowledge processing. Additionally, these studies’ high percentage (50%) of false or anomalous sentences and slow presentation rates may have disrupted natural reading. Overall, the results of Marslen-Wilson et al. (1988) and Hagoort et al. (2004) are inconclusive as to whether selectional restriction knowledge is privileged in impossibility/implausibility detection, or whether detection might instead be determined by a single construct like unlikelihood.

Rayner, Warren, Juhasz & Liversedge (2004) investigated eye-movements to sentences with varying degrees of implausibility (like 2–4 below) to determine how event unlikelihood influenced eye-movement patterns.

2) The woman used a brush to apply the thick mascara in the morning.

3) The woman used a rag to apply the thick mascara in the morning.

4) The woman used a pitchfork to carry the thick mascara in the morning.

Rayner et al. (2004) reported that first pass reading times on the target word (mascara above) were longer in the most implausible condition (4) than in the other conditions. The moderately implausible condition (3) did not differ from baseline (2) until after the target word. These results show that detection latency decreased and disruption severity increased as unlikelihood increased, however approximately half of the stimuli in the most implausible condition included a selectional restriction violation. The results therefore do not distinguish between accounts under which selectional restrictions are privileged knowledge and ones in which disruption is driven by the global unlikelihood of an event or state.

The current experiment investigates processing disruption to impossible events cued by selectional restriction violations as compared to extremely implausible events without selectional restriction violations, but with similar unlikelihood. Eye-tracking was used because it provides detailed information about the time course of language interpretation during normal reading (Cook & Myers, 2004; Ehrlich & Rayner, 1981; Rayner, 1998; Rayner et al., 1983; Rayner & Duffy, 1986; Rayner et al., 2004; Trueswell et al., 1994). Eye-movements were examined across sentences describing three kinds of events: (1) a plausible event (possible-plausible condition), (2) an extremely implausible but possible event with no selectional restriction violation (possible-implausible condition), and (3) an extremely implausible event that violated a verb’s selectional restrictions and was therefore impossible (impossible-implausible condition). Although a fully crossed design would have been preferable, it was not possible. The missing condition would be a selectional restriction-violating impossible event that was plausible, but impossible events necessarily violate plausibility.¹ If selectional restriction information is privileged in cueing violations because it is available or used earlier than other information, disruption should appear earlier in the comparison between the conditions that are similarly implausible but differ by a selectional restriction violation (the possible-implausible and impossible-implausible conditions) than between the conditions without selectional restriction violations but with different plausibilities (the possible-plausible and possible-implausible conditions). Alternatively, if unlikelihood drives disruption, disruption should be evident in the comparison between the possible-plausible and possible-implausible conditions, but not in the comparison between the possible-implausible and impossible-implausible conditions.

Methods

Participants

54 undergraduates at the University of Pittsburgh participated for course credit.² All were native English speakers with normal or corrected-to-normal vision.

Apparatus

A Fourward Technologies Dual-Purkinje Image Generation VI eye-tracker monitored the gaze location of participants’ right eyes during reading. The eye tracker has a spatial resolution of 10-min arc and samples gaze location every millisecond. Participants viewed the stimuli binocularly on a monitor 61 cm from their eyes.

Materials

The experiment tested thirty items with three conditions: possible-plausible, possible-implausible, and impossible-implausible, where impossible indicated a selectional restriction violation and implausible indicated a contextual/world knowledge violation.

Possible-plausible: The man used a strainer to drain the thin spaghetti yesterday evening.

Possible-implausible: The man used a blow-dryer to dry the thin spaghetti yesterday evening.

Impossible-implausible: The man used a photo to blackmail the thin spaghetti yesterday evening.

Every item began with a definite noun phrase followed by the verb used, an instrument (strainer in the possible-plausible condition above), an infinitival verb (to drain in the possible-plausible condition above), an adjectival noun phrase (the thin spaghetti), and then a final adjunct phrase (yesterday evening). Note that all words following the infinitival verb were the same across conditions. The target word was the noun of the adjectival noun phrase (spaghetti above), and the locus of the violation in the possible-implausible and impossible-implausible conditions. To increase the chance it would be fixated, the target word was a minimum of 5 characters long. In the possible-plausible condition, the target noun was a natural participant in the event; e.g. it is natural to drain spaghetti with a strainer. In the possible-implausible condition, the target noun satisfied the selectional restrictions of the infinitival verb, but was an unlikely participant in the event; e.g. it is possible, but strange to dry spaghetti with a blow-dryer. In the impossible-implausible condition, the target noun violated the verb’s selectional restrictions; e.g. spaghetti, being insentient, cannot be blackmailed.

Items were designed to be equally possible in the two possible conditions and equally implausible in the two implausible conditions. To check this, possibility and plausibility ratings were gathered from 37 and 51 University of Pittsburgh students respectively, who did not participate in the main experiment. Separate ratings for plausibility and possibility presented stimuli up through the target word, without the final adjunct phrase. Conditions were counterbalanced across three presentation lists. A few participants completed both surveys, but always in different presentation lists. In the plausibility ratings, participants used a scale of 1 (very likely) – 7 (very unlikely) to rate the event’s likelihood in the real world. In the possibility ratings, participants indicated, via a binary choice, whether the event described in the sentence was possible (recorded as a score of 1) or impossible (recorded as a score of 0). Idealized and actual mean possibility and plausibility ratings appear in Table 1. The qualitative pattern of the actual ratings was similar to the idealized ratings, however all pair-wise comparisons between conditions for both the plausibility and the possibility ratings were significant (ps<.05). Effect sizes for the comparisons are provided in Table 2.

Table 1.

Idealized and actual mean possibility and plausibility ratings (standard deviations) per condition.

	Possible-Plausible		Possible-Implausible		Impossible-Implausible
	Ideal	Actual	Ideal	Actual	Ideal	Actual
Possibility	1	0.97 (.04)	1	0.74 (.16)	0	0.10 (.10)
Plausibility	1	1.63 (.49)	7	5.84 (.75)	7	6.74 (.28)

Table 2.

Effect sizes (Cohen’s d) for norming ratings of plausibility and possibility, for each condition comparison.

	Possible-Plausible vs. Possible-Implausible	Possible-Implausible vs. Impossible-Implausible
Possibility	1.9	4.7
Plausibility	6.7	1.6

The 30 experimental items were combined with 60 filler items. Four filler items were implausible or impossible, meaning that approximately 16% of the sentences a participant read were impossible and a fully overlapping 27% were implausible. Five filler items had the main verb used, but continued with a syntactic structure different from the one in the experimental items. Conditions were counterbalanced across three presentation lists using a Latin square design. After one quarter of the sentences, participants answered a yes/no comprehension question. Half of these required a “yes” response. Questions were asked only about the possible-plausible condition of the experimental items.

Procedure

The experiment lasted between 30 and 55 minutes. A forehead rest and bite bar minimized head movements. Participants were asked to read normally, for comprehension, and were told that after some sentences they would need to answer a yes/no comprehension question. After the participant was seated at the eye tracker and had been instructed as to the format of the experiment, the tracker was aligned and calibrated. A screen of boxes appeared after each item as a calibration check. The tracker was recalibrated as necessary.

Results

Three regions of interest were defined for the purposes of analysis. The pre-target region consisted of the determiner and adjective (the thin) preceding the target noun. The target region was the target noun (spaghetti). The post-target region was the five characters (including spaces) immediately following the target word. Comprehension rates were high (Mean 91.7%, SD 4.1%). All participants scored higher than 80%. Track losses and missing trials accounted for a loss of 12% of the data, and were distributed relatively evenly across conditions. Approximately 89% of all regions were fixated during the first pass in all three conditions. Fixations shorter than 80ms which were not within 3 characters of a previous or subsequent fixation were eliminated. Additionally, data points more than 3 standard deviations from the mean of a condition were eliminated. This excluded less than 2% of the remaining data.

Six standard eye-movement measures were computed for all three regions of interest (Rayner, 1998). First fixation duration is the duration of the first fixation on a region or word during first pass reading. Single fixation duration is the duration of the first fixation on a region or word during first pass reading, conditional upon that word or region only receiving one fixation. Gaze duration is the sum of all fixations from first entering a region during first pass reading until leaving it. Regression path duration (sometimes called go-past time) is the sum of all fixations from first entering a region during first pass reading until leaving it to the right, including regressive fixations. Regressions out is the proportion of times a regression was launched from a region during first pass reading. Finally, total time is the sum of all fixations on a region.

Data were subjected to analyses of variance (ANOVAs) using participants (F1) and items (F2) as random factors. Contrasts between the possible-plausible and possible-implausible conditions measured effects of the plausibility manipulation, while contrasts between the possible-implausible and impossible-implausible conditions measured effects of the event possibility/selectional restriction violation manipulation. Interpretation of these effects is qualified by the fact that there were small but significant differences between the off-line ratings for the factors assumed to be held constant in these comparisons. Unfortunately, the off-line ratings were so highly correlated that it was not possible to statistically control for these small differences. We will return to this issue in the discussion.

Pre-target Region

Table 3 presents means for each eye-movement measure in the pre-target region. The only fully reliable effects in this region were in the total time measure, which includes rereading initiated later in the sentence. Possibility and plausibility violations both increased total reading time (Possibility: F1(1,53)=7.65, p=.008; F2 (1,29)=6.68, p=.015; Plausibility: F1(1,53)=5.76, p = .02; F2 (1,29)=4.03, p=.054). One measure of early processing, regression path duration, showed an effect of possibility that was reliable by participants (F1(1,53)=6.41, p=.014; F2(1,29)=2.82, p=.104). However, the effect may not truly reflect possibility, as the possible-plausible condition was 19ms longer than the possible-implausible condition and did not differ from the impossible-implausible condition.

Table 3.

Average fixation times (standard deviations) on the pre-target region in milliseconds.

	FFD	Single	Last-3	Gaze	Reg Path	% Reg out	Total
Possible-Plausible	238 (35)	257 (43)	228 (51)	342 (72)	392 (92)	8.1 (9.5)	423 (104)
Possible-Implausible	240 (37)	259 (47)	238 (62)	330 (71)	373 (90)	9.2 (10.5)	454 (128)
Impossible-Implausible	244 (39)	266 (65)	243 (65)	345 (80)	417 (133)	9.0 (10.7)	499 (143)

Following Rayner et al. (2004), we analyzed fixations on the three characters immediately preceding the target region (see Table 3) to determine whether there was any evidence that readers semantically pre-processed the upcoming word. Although mean durations were numerically longer in conditions with violations, there was no hint of a possibility or plausibility effect (ps > .2). The absence of effects in measures indexing first pass processing during the pre-target region suggests that conditions were processed similarly prior to the target word.

Target Region

Table 4 shows the means for the target region. First fixation durations and single fixation durations showed an effect of possibility (FF: F1(1,53)=5.55, p=.022; F2(1,29)=5.42, p=.027, SF: F1(1,53)=4.27, p=.044; F2(1,29)=5.37, p=.028). There was no effect of plausibility; in fact the possible-plausible and possible-implausible means differed by only one millisecond.

Table 4.

Average fixation times (standard deviations) on the target region in milliseconds.

	FFD	Single	Gaze	Reg Path	% Reg out	Total
Possible-Plausible	259 (39)	266 (45)	309 (61)	340 (71)	7.9 (8.5)	360 (80)
Possible-Implausible	260 (42)	267 (52)	325 (80)	373 (110)	10.8 (12)	402 (106)
Impossible-Implausible	273 (45)	280 (47)	322 (60)	386 (99)	11.1 (12)	433 (120)

Gaze duration on the target showed no effect of event possibility or plausibility. This may have been due to increased regressions out in the implausible conditions (F1(1,53)=4.03, p=.05; F2(1,29)=1.67, p=.207), which contributed to a fully reliable effect of plausibility in the regression path duration measure (Plausibility: F1(1,53)=10.16, p=.002; F2(1,29)=7.18, p=.012; Possibility: Fs<2, ps>.2). Total time showed effects of both event possibility and plausibility (Possibility: F1(1,53)=7.39, p=.009; F2(1,29)=5.60, p=.032; Plausibility: F1(1,53)=10.33, p=.002; F2(1,29)=9.42, p=.005). Overall, patterns on the target word suggest that event possibility violations affected the earliest eye-movement measures while both event possibility and plausibility violations affected later measures.

Post-target Region

Eye-movement measures in the post-target region are shown in Table 5. There were fully reliable effects of event possibility in first fixation duration, single fixation duration, gaze duration, and regression path duration (FF: F1(1,53)=6.36, p=.015; F2(1,29)=4.47, p=.043, SF: F1(1,53)=4.14, p=.047; F2(1,29)=6.84, p=.014; Gaze: F1(1,53)=4.55, p=.038; F2(1,29)=6.09, p=.020); RegPath: F1(1,53)=17.65, p<.001; F2(1,29)=8.14, p=.008). The plausibility manipulation had no reliable effects on any measure in this region.

Table 5.

Average fixation times (standard deviations) on the post-target region in milliseconds.

	FFD	Single	Gaze	Reg Path	% Reg out	Total
Possible-Plausible	264 (43)	278 (53)	365 (83)	454 (129)	12.6 (11)	439 (105)
Possible-Implausible	268 (41)	276 (59)	355 (91)	475 (162)	16.2 (14)	464 (126)
Impossible-Implausible	288 (62)	301 (81)	389 (99)	584 (194)	19.8 (16)	480 (118)

Discussion

ANOVAs indicated an earlier onset of processing disruption associated with impossibility (first and single fixation duration on the target word) than severe implausibility (regression path duration on the target word). Interestingly, measures during first pass processing on the target word that included regressions (i.e. regressions out and regression path duration) patterned with plausibility and did not show possibility effects. However, in the subsequent region and in measures including second pass reading, there were either effects of possibility, or both possibility and plausibility.

The interpretation of these results is complicated by the fact that there were small but reliable differences between the factors assumed to be held constant in the ANOVAs. As possibility is categorical, it is not clear whether to be concerned about the small difference between possibility means. However, the small plausibility difference is a concern. We cannot rule out the possibility that unlikelihood may have played a part in the disruption that we have been attributing to impossibility/the presence of a selectional restriction violation. The amount of disruption that unlikelihood may have contributed depends on the nature of the relationship between disruption and unlikelihood. If the relationship were linearly increasing, then implausibility could have played only a small part in the effects we attribute to impossibility, because a plausibility difference four times greater showed no effect in the same measures. Alternatively, if the relationship were non-linear such that a ranking of maximal unlikelihood was associated with much more disruption than a ranking one point lower on a seven point scale, then implausibility may have strongly contributed to the effects attributed to impossibility. However, such a relationship did not hold for the measures indexing regressive eye-movements, which showed the opposite pattern of effects: reliable effects corresponding to the large difference spanning the majority of the unlikelihood scale, but none corresponding to the small difference at the upper end of the scale. These considerations suggest that the speed of detection and the magnitude of processing disruption associated with a plausibility/possibility violation do not simply monotonically increase with the unlikelihood of the resulting event or state, a possible interpretation of Hagoort et al. (2004)’s and Rayner et al. (2004)’s results. It is either the case that readers are sensitive to both likelihood and possibility, or unlikelihood and disruption have an extremely complex relationship.

The finding of earlier processing disruption in conditions with a selectional restriction violation than in conditions without such violations is consistent with the hypothesis that information about a verb’s selectional restrictions is privileged over other kinds of knowledge in comprehension. This privilege may be because selectional restriction knowledge is represented in the lexicon and is available earlier than world/contextual knowledge. However, experiments using the visual world paradigm have demonstrated extremely early effects of contextual knowledge (e.g. Sedivy, Tanenhaus, Chambers, & Carlson, 1999), and the current data cannot rule out the possibility that unlikelihood/implausibility had a small effect at the earliest stages of interpretation. A more likely alternative, based on the idea that semantic interpretation begins with a coarse-grained analysis that is subsequently refined (Sanford & Garrod, 2005), is that the knowledge involved in the semantic/thematic fit between a noun and verb may be exactly the kind of coarse grained knowledge recruited during initial interpretation. World or contextual knowledge about the likelihood of a multi-participant event may generally be recruited later, when the initial coarse interpretation is refined.

One simple explanation of the pattern of effects is that plausibility/possibility violation detection occurs most quickly in the presence of a selectional restriction violation, but contextual and world knowledge soon come into play and often spur the reader to look back in the text. Rereading is a potentially useful and cheap response to any unexpected violation, which could explain why the single dimension of unlikelihood seems to account for the pattern in regressive eye-movements during first pass reading. However, subsequent processing showed effects of possibility or both possibility and plausibility. The reemergence of possibility as an influence on later processing may occur as readers attempt to create a coherent discourse model for a sentence. Possible-implausible stimuli inspire peculiar, but imaginable discourse models. Impossible-implausible stimuli, being semantically uninterpretable, fail to inspire any coherent discourse model at all.

It is an open question whether the pattern in the impossible-implausible condition was due to general processing differences between impossibility and implausibility or the presence of a selectional restriction violation. We think that the strength and clarity of the cue to impossibility is a more likely determinant of detection latency than the final judgment of impossibility, because such judgments can differ within and between people and be difficult to compute (e.g. in preliminary norms some subsequently discarded sentences were rated possible and impossible by equal numbers of participants, and at times our own judgments changed upon reflection). Selectional restriction violations, being clashes between the core meanings of a predicate and one of its arguments, are likely to be detected during the initial phases of semantic interpretation.