The Impact of Linguistic Prediction Violations on Downstream Recognition Memory and Sentence Recall

Ryan J Hubbard; Kara D Federmeier

doi:10.1162/jocn_a_02078

. 2024 Jan 1;36(1):1–23. doi: 10.1162/jocn_a_02078

The Impact of Linguistic Prediction Violations on Downstream Recognition Memory and Sentence Recall

Ryan J Hubbard ^1,^✉, Kara D Federmeier ¹

PMCID: PMC10864033 PMID: 37902591

Abstract

Predicting upcoming words during language comprehension not only affects processing in the moment but also has consequences for memory, although the source of these memory effects (e.g., whether driven by lingering pre-activations, re-analysis following prediction violations, or other mechanisms) remains underspecified. Here, we investigated downstream impacts of prediction on memory in two experiments. First, we recorded EEG as participants read strongly and weakly constraining sentences with expected, unexpected but plausible, or semantically anomalous endings (“He made a holster for his gun / father / train”) and were tested on their recognition memory for the sentence endings. Participants showed similar rates of false alarms for predicted but never presented sentence endings whether the prediction violation was plausible or anomalous, suggesting that these arise from pre-activation of the expected words during reading. During sentence reading, especially in strongly constraining sentences, plausible prediction violations elicited an anterior positivity; anomalous endings instead elicited a posterior positivity, whose amplitude was predictive of later memory for those anomalous words. ERP patterns at the time of recognition differentiated plausible and anomalous sentence endings: Words that had been plausible prediction violations elicited enhanced late positive complex amplitudes, suggesting greater episodic recollection, whereas anomalous sentence endings elicited greater N1 amplitudes, suggesting attentional tagging. In a follow-up behavioral study, a separate group of participants read the same sentence stimuli and were tested for sentence-level recall. We found that recall of full sentences was impaired when sentences ended with a prediction violation. Taken together, the results suggest that prediction violations draw attention and affect encoding of the violating word, in a manner that depends on plausibility, and that this, in turn, may impair future memory of the gist of the sentence.

INTRODUCTION

Numerous psychological and neuroscientific studies have identified that the brain generates predictions of features of upcoming stimuli and environmental states to more efficiently process the rapid input of information we constantly receive (De Lange, Heilbron, & Kok, 2018; Bar, 2009; Friston, 2005). Prediction appears to play an important role in several areas of perceptual and cognitive processing (Den Ouden, Kok, & De Lange, 2012). For instance, it is now widely accepted that language comprehension involves a combination of integrative processing to build a message-level meaning, as well as engagement of predictive mechanisms to facilitate rapid comprehension (Federmeier, 2007, 2022; Pickering & Gambi, 2018; Kuperberg & Jaeger, 2016; Kutas, DeLong, & Smith, 2011; Altmann & Mirković, 2009). The benefits of contextual information and predictive processing during language comprehension have been observed in behavioral studies examining RTs to lexical decisions and word naming times (Simpson, Peterson, Casteel, & Burgess, 1989; Schwanenflugel & LaCount, 1988; Schuberth, Spoehr, & Lane, 1981; Fischler & Bloom, 1979), as well as in eye-tracking studies during passive reading (Staub, 2015; Rayner, Slattery, Drieghe, & Liversedge, 2011; Frisson, Rayner, & Pickering, 2005; Ehrlich & Rayner, 1981) and visual world paradigms (Kamide, 2008; Altmann & Kamide, 1999). In addition, cognitive neuroscientific studies have identified evidence of facilitation through prediction by examining how neural signals related to stimulus processing vary based on predictability. For instance, the N400 component of the ERP, a centroparietal negativity reflecting access of semantic long-term memory (Federmeier, 2022; Kutas & Federmeier, 2000, 2011), is reduced in a graded manner following words that are more predictable (Szewczyk & Federmeier, 2022; Wlotko & Federmeier, 2012; Federmeier, Wlotko, De Ochoa-Dewald, & Kutas, 2007; Federmeier & Kutas, 1999). In addition, recent work using multivariate analysis approaches have demonstrated that features of an upcoming word are neurally pre-activated before the word's occurrence (Hubbard & Federmeier, 2021a; Wang, Kuperberg, & Jensen, 2018). Thus, there is ample evidence that individuals can engage predictive processing mechanisms during comprehension of language and that the engagement of these mechanisms can facilitate processing.

Facilitation in terms of language comprehension often refers to speed of processing; language information accrues rapidly, and prediction allows comprehenders to better keep up with that rapid input. However, pre-activation of upcoming information could have unintended negative consequences as well. Comprehenders may process stimuli that were predicted in a “top–down verification mode” (Van Berkum, 2010), such that, if the encountered stimulus matches what was predicted, the bottom–up stimulus input is then processed less deeply or fully. For instance, in one study, predictable sentence-ending words that were then repeated later in a different context elicited reduced ERP repetition effects compared with (similarly repeated) unpredictable words, suggesting that less information had been taken in and retained about those predictable words (Rommers & Federmeier, 2018a). This effect was not observed for unexpected words in the same sentence contexts (Lai, Rommers, & Federmeier, 2021), indicating that prediction played a crucial role in creating the reduced processing.

On the other hand, when comprehenders are predicting and then encounter inputs that do not match what they predicted, the (erroneously) pre-activated features of the stimulus that was not encountered can linger. This lingering pre-activation can result in repetition-like ERP effects for predicted words that were never actually observed (Rommers & Federmeier, 2018b). In addition, these lingering activations can affect later memory judgments: When individuals are tested on their memory for previously read words, they are more likely to falsely remember seeing predictable words that they did not actually read compared with unpredictable words they did not read (Haeuser & Kray, 2022; Höltje & Mecklinger, 2022; Hubbard, Rommers, Jacobs, & Federmeier, 2019; Smith, Hasinski, & Sederberg, 2013), and they are slower to correctly reject predictable words when tested immediately after reading a sentence (Rich & Harris, 2021). Thus, prediction may essentially generate a representation of the upcoming expected stimulus, which can in some ways facilitate processing of the expected stimulus in the moment, but can also have downstream consequences for how that stimulus is then processed in future encounters.

Prediction can also serve a second purpose—namely, prediction errors, or instances in which the encountered stimulus does not match the predicted stimulus, can be potentially useful signals for learning and updating of internal models (Dell & Chang, 2014; Chang, Dell, & Bock, 2006) and may critically impact memory (Sinclair & Barense, 2019). Prediction errors can arise when the context provided by a sentence leads to the prediction of a particular word (e.g., “The rude waiter was not given a tip”), but a different word than what was predicted is encountered (e.g., “The rude waiter was not given a tray”). Although eye-tracking studies of natural reading behavior have reported little evidence of a processing penalty in reading or fixation times following unexpected words (Frisson, Harvey, & Staub, 2017; Luke & Christianson, 2016), studies employing electrophysiological measurements have identified unique neural responses elicited by words that violate predictions (Van Petten & Luka, 2012). These effects have three important characteristics: First, they are observed most robustly when a particular word is highly expected in the context (i.e., the sentence context is strongly constraining [SC] toward that word) and are less likely to be observed when there is no highly predictable word (i.e., the sentence context is weakly constraining [WC]), suggesting elicitation requires there to be a prediction violation (Federmeier et al., 2007). Second, they are observed at a processing stage later in time with respect to the N400, suggesting they reflect engagement of mechanisms that arise after initial semantic processing of the incoming word. Last, the pattern of ERP responses to prediction-violating words depends on plausibility, suggesting these post-N400 signals may reflect some sort of updating or revision of the previously built message-level representation.

More specifically, unexpected words that violate predictions but are still plausible given the preceding sentence context (e.g., “The rude waiter was not given a tray”) elicit an extended positivity, observed roughly 600–1000 msec poststimulus, with an anterior maximum over the scalp. This contrasts with the response to semantically anomalous words (e.g., “The rude waiter was not given a cabin”), which often (although not always) elicit a positivity with a posterior scalp distribution in a similar time-frame to the anterior positivity (Brothers, Wlotko, Warnke, & Kuperberg, 2020; DeLong & Kutas, 2020; Kuperberg, Brothers, & Wlotko, 2020; Brothers, Swaab, & Traxler, 2015; DeLong, Quante, & Kutas, 2014; Paczynski & Kuperberg, 2012; Thornhill & Van Petten, 2012; Delong, Urbach, Groppe, & Kutas, 2011; Van De Meerendonk, Kolk, Vissers, & Chwilla, 2010; Otten & Van Berkum, 2008; Federmeier et al., 2007). Importantly, recent MEG work found that separable neural sources produced these different responses to the different types of unexpected words, suggesting they reflect the engagement of different processes (Wang et al., 2023), although the exact mechanisms engaged is currently a debated topic.

One recent account (Kuperberg et al., 2020) posits a hierarchical generative framework in which the anterior positivity following plausible violations reflects successful updating of the individual's internally constructed situation model to fit with the unexpected information, whereas the posterior positivity reflects an initial (and potentially continued) failure to update the situation model because of the semantically anomalous nature of the information. Others have posited that the anterior positivity reflects engagement of the frontal cortex to inhibit the predicted word, because of the appearance of a plausible alternative (Ness & Meltzer-Asscher, 2018; Kutas, 1993). The posterior positivity has often been compared with the P600, an ERP component initially associated with the processing of syntactic anomalies (Hagoort, 2003; Gunter, Stowe, & Mulder, 1997; Osterhout & Holcomb, 1992, 1993) and in recent accounts linked to difficulty in semantic integration (Brouwer, Crocker, Venhuizen, & Hoeks, 2017; Brouwer, Fitz, & Hoeks, 2012; Kuperberg, 2007). Thus, the posterior positivity could index integration success, whereas the anterior positivity reflects suppression of the predicted word.

These differing accounts raise questions regarding the locus of the previously discussed effects of prediction on future memory. Individuals may falsely remember seeing predictable words they never actually read because the pre-activated representations linger in memory; however, it may also be the case that prediction violations lead to re-analysis of the originally constructed situation model, causing some level of encoding of the predictable target word. For instance, reading “The rude waiter was not given a tray” may cause the reader to re-evaluate the scenario involving the waiter—potentially that the waiter is not given as many tables to work—but still may lead to the inference that the waiter is not given tips for their work or will receive less tips because of not being given as many tables. Critically, then, the impacts on false memory for predictable words may differ based on the type of prediction violation that is read instead of the predictable word. Returning to the previous example, reading “the rude waiter was not given a cabin” may lead the reader to consider a completely different scenario, or to simply not engage the same re-evaluation of the scenario, leading to a lack of encoding of the predictable word “tip” into memory at all. In this case, false recognition of predictable words would only occur following unexpected but plausible prediction violations, as only these violations lead to some level of encoding of the predictable word.

The mechanisms engaged to deal with different violation types may also lead to differences in memory for the prediction violations themselves. In our previous work (Hubbard et al., 2019), we found that unexpected but plausible sentence endings elicited larger N1 and late positive complex (LPC) amplitudes during a recognition test compared with expected sentence endings. The increased N1 amplitude may have been caused by participants performing some internal target discrimination of predictable and unpredictable words (Curran, Tanaka, & Weiskopf, 2002; Hopf, Vogel, Woodman, Heinze, & Luck, 2002; Vogel & Luck, 2000), whereas the increased LPC amplitude likely reflected deeper encoding of the unexpected words, leading to greater episodic detail associated with these words during recognition (Yu & Rugg, 2010; Woodruff, Hayama, & Rugg, 2006; Rugg et al., 1998). Unpredictable sentence endings might draw more attention than expected endings, leading to greater depth of encoding (Röer, Bell, Körner, & Buchner, 2019; Craik & Tulving, 1975; Craik & Lockhart, 1972; Wallace, 1965; Von Restorff, 1933). However, semantically anomalous sentence endings were not included in this study, and it is unclear if these words would draw even greater attention than plausible unexpected words, leading to deeper encoding.

Recent work has compared false recognition for predicted words, as well as recognition of prediction violations, following plausible versus anomalous violations (Haeuser & Kray, 2022, 2023). Participants self-paced their reading of sentences and then were tested on their memory for predictable sentence endings and prediction violations, predictable but unseen words, and unpredictable unseen (new) words. Increased luring was observed for predictable but unseen words, and this luring did not differ based on the type of prediction violation that was read. In addition, a slight benefit to recognition memory was found, specifically for semantically anomalous violations compared with plausible violations or expected endings. However, there were important caveats to this study. First, although self-paced reading paradigms are likely more ecologically valid and closer to actual reading than the rapid-serial visual presentation (RSVP) paradigms used in ERP studies (Ditman, Holcomb, & Kuperberg, 2007), they allow for individual variability in reading rates of critical items, which could influence recognition results. Indeed, participants in this study devoted longer reading times to prediction violations, which could have influenced their recognition memory for these items (Tullis & Benjamin, 2011; Son & Metcalfe, 2000; Mazzoni & Cornoldi, 1993). Second, EEG was not recorded in this experiment, making it difficult to directly examine how processes engaged during reading are related to the observed pattern of recognition memory results. Last, only highly predictable (SC) sentences were read in this study, and thus the effect of constraint could not be tested. Including WC sentences serves as an important baseline comparison, because postviolation ERPs are more robustly observed in SC contexts, and thus the impacts of prediction violations on memory may differ when constraint is low.

In the current study, we expanded upon the previous results of Haeuser and Kray (2022) and Hubbard and colleagues (2019) in two experiments examining the impact of prediction violations on downstream memory. The design of the first experiment largely replicated the design of Hubbard and colleagues (2019), but we now added semantically anomalous endings, in which the real-word plausibility of the scenario described by the sentence was violated, as well as the unexpected but plausible sentence endings. With this design, we could evaluate the impact of different types of prediction violations, as well as how these effects interact with sentential constraint, on downstream luring effects and recognition memory. By recording EEG while sentences were read, we could more directly examine how engagement of processes following prediction violations during reading might influence downstream memory, by relating the magnitude of post-N400 ERP responses to later recognition. This design also allowed us to examine the processes engaged during recognition of the sentence final words themselves, by examining LPC and N1 ERP responses elicited by sentence final words during the recognition test to better understand how prediction errors impact the formation of memories.

In the second experiment, we attempted to provide a fuller understanding of how predictive processing influences downstream memory by testing individuals' free recall of sentences they had read. Prediction violations may lead to message-level revision to incorporate the unexpected information, which in turn may impact memory for the sentence as a whole. Few studies have investigated the impact of predictability on sentence recall; although classic psychological research does suggest that more predictable sentences are more readily recalled from memory (Holmes & Murray, 1974), “predictability” in that work was defined based on the real-life plausibility of the events of the sentence, not quantitatively based on the cloze probability of the words within the sentence. Recent work from our laboratory examining the impact of value-driven strategies on sentence recall found that different strategies influenced sentence recall and that plausible final-word prediction violations tended to reduce sentence recall (Chung & Federmeier, 2023). Other work has shown that sentences containing more highly associated words tend to be recalled more easily (Vanevery & Rosenberg, 1970; Rosenberg, 1968, 1969). Thus, it stands to reason that unpredictable or semantically anomalous words may disrupt the semantic associations between words of the sentence, leading to an impairment in sentence recall. Alternatively, given that predictability was manipulated only on the single, sentence-ending word, whether that final word was predictable could end up having little impact on the overall gist-level representation of the sentence in memory, which may be more important than the specific lexical units (Potter & Lombardi, 1990; Graesser, 1978).

Thus, we were able to test a set of hypotheses across the two experiments. We expected that predictions for sentence ending words would linger, leading to more false alarms for lure items compared with new items; of interest, then, was whether the preceding sentence context or the type of prediction violation would have any effect on the false alarm rate. We also hypothesized that predictable sentence endings would not be recognized as easily as unpredictable endings, but that sentences with expected endings would be recalled more easily from memory. We expected that anomalous sentence endings would draw the most attention and thus likely elicit larger N1s than plausible unexpected endings, but that this would also create a greater detriment on recall of the sentences containing the anomalies. Finally, we expected that the magnitude of the post-N400 positivities at the time of sentence reading would be related to downstream memory for the prediction violations, but that the magnitude of this relationship would differ based on the type of prediction violation.

EXPERIMENT 1

Materials and Methods

Participants

Forty-five individuals from the Champaign–Urbana area participated in the experiment in exchange for cash. The sample size was chosen based on the previously conducted study (Hubbard et al., 2019); we increased the sample size for this study, as an additional condition was included in the current study. All participants were right-handed, reported normal or corrected-to-normal vision, were native English speakers, and had no history of any neurological or psychiatric disorder. Following data collection, three participants were removed because of excessive noise or artifacts in the EEG data, leaving 42 participants in the reported data. Mean age was 20.2 years (range: 18–30 years), and 25 of the participants were women. The study was approved by the institutional review board at University of Illinois, Urbana-Champaign (UIUC), and all participants provided written informed consent and were debriefed following participation. Typical methods of conducting power analyses are not appropriate for mixed-effects model analyses (Faul, Erdfelder, Lang, & Buchner, 2007). Therefore, we used a modern simulation-based approach to estimate power (Kumle, Võ, & Draschkow, 2021). We used the ERP data from Hubbard and colleagues (2019) to estimate the power of detecting an anterior positivity ERP effect, as this effect is typically smaller than other ERP effects and likely more difficult to detect. A linear mixed-effects model was constructed predicting anterior positivity amplitude, with a fixed effect of condition, random intercepts for participants and items, and a random slope of condition for participants. We then used the mixedpower package in R to estimate the power of detecting this effect with an N of 42. This analysis suggested the power to detect this effect was 0.89; thus, the study was adequately powered to detect the effects of interest.

Materials

The stimuli were composed of 240 English sentences, a subset of the stimuli originally used in Federmeier and colleagues (2007). Half of the sentences (120) were SC toward a particular ending word (final word cloze > 0.68, mean final word cloze = 0.83), whereas the other half were WC (final word cloze < 0.42, mean final word cloze = 0.28). A third of the sentences (80) ended with the highest cloze probability sentence ending (i.e., “expected”), a third of the sentences ended with an unexpected but plausible sentence ending (cloze probability approximately 0), and the final third of the sentences ended with a semantically anomalous ending word. Anomalous sentence endings were never produced during the cloze norming and thus had a cloze probability of 0. These words were not normed for plausibility, but were highly implausible given the scenario of the preceding sentence and thus determined to be semantically incongruous by experimenter judgment. Note that although an anomalous ending is also unexpected by the participant, we use the term “unexpected” here to refer specifically to the unexpected but plausible sentence endings and not the semantically anomalous endings. Thus, participants read 40 SC sentences with expected endings (strongly constrained expected [SCE]), 40 with unexpected but plausible endings (SCU), and 40 SC sentences with anomalous endings (SCA); this was also the case for the WC sentences (40 weakly constrained expected [WCE], 40 WCU, 40 WCA). These stimuli were evenly split into 10 blocks (four of each condition in each block). The lexical properties (word frequency, concreteness, imageability, familiarity) of sentence ending words were controlled such that there were no significant differences across these variables between the experimental conditions.

Participants were tested on their memory for sentence ending words in the recognition memory blocks of the experiment. In each block, participants were presented with sentence ending words, new words, and sentence medial fillers to ensure participants attended to and read the entire sentence. Sentence ending words were either matches, in which the word presented at test matched what the participant read during encoding, or lures, in which the word presented at test was the expected ending to a sentence with an unexpected but plausible ending, or an anomalous ending, at encoding. As an example of a lure item, a participant may read the sentence “Shuffle the cards before you forget” (an unexpected but plausible ending) during the encoding period, and be tested on the word “deal” (the expected ending) during the memory test. New words were stimuli that were never presented during the course of the encoding period, and were selected to match the stimulus characteristics of the lure items. Over the course of the experiment, participants were tested on 20 items in each condition, along with 40 new items to ensure an equal number of old and new items were tested. Thus, in each of the 10 recognition blocks, participants were tested on two items from each condition, as well as four new items and two sentence medial words, resulting in 26 items per test block. Table 1 provides an overview and examples of the different types of test items.

Table 1. .

Examples of Stimuli Presented during Encoding and Test Blocks during the Experiment

Encoding				Test
Constraint	Sentence	Ending Type	Ending	Test Item Type	Test Item
SC	Tim threw a rock and broke the	Expected (E)	window	Match	window
		Unexpected (U)	camera	Match	camera
		Unexpected (U)	camera	Lure	window
		Anomalous (A)	novel	Match	novel
		Anomalous (A)	novel	Lure	window
WC	His ring fell into a hole in the	Expected (E)	sink	Match	sink
		Unexpected (U)	couch	Match	couch
		Unexpected (U)	couch	Lure	sink
		Anomalous (A)	banana	Match	banana
		Anomalous (A)	banana	Lure	sink

Open in a new tab

Lexical properties (word frequency, concreteness, imageability, familiarity) were mostly controlled across test items; however, unexpected but plausible test items did significantly differ from expected test items in word frequency (E = 107, U = 85, t = 1.98, p = .05), obtained from Kučera and Francis (1967). Anomalous test items did not significantly differ from expected or unexpected but plausible stimuli in word frequency. Table 2 summarizes the lexical properties of the stimuli used in the recognition memory test. To account for the potential differences in word frequency between conditions, frequency was included as a covariate in statistical analyses.

Table 2. .

Lexical Properties of Test Stimuli

Condition	Frequency	Concreteness	Imageability	Familiarity	Word Length
SCE match	4.14	519	567.56	533,94	5.05
SCU match	3.34	455.83	552.31	491.77	6.15
SCA match	4.10	560	547.65	558.25	5.5
SCU lure	4.05	494.59	564.42	515.26	5.45
SCA lure	3.73	554.06	572.38	567.44	4.65
WCE match	4.13	492.77	582.33	533.85	5.35
WCU match	3.10	463.2	553.6	498.27	5.85
WCA match	3.68	561.55	556.9	556.9	5.3
WCU lure	3.34	511.21	578.4	535.27	5.3
WCA lure	2.93	516.7	566.91	512.36	5.45
New	3.80	493.68	563.13	518.53	5.65

Open in a new tab

Values represent means across items. Frequency values are log transformed and obtained from Kučera and Francis (1967). Concreteness, imageability, and familiarity values are obtained from the MRC psycholinguistic database.

Similar to Hubbard and colleagues (2019), the stimuli were presented in a pseudorandomized order to prevent issues of stimulus repetition during the memory test. Namely, a sentence ending word used as a test item (“Shuffle the cards before you deal”) could appear in the middle of another sentence (“He learned to deal with it”), and this repetition could influence recognition memory. To avoid this issue, presentation order was set up to avoid participants reading both sentences before being tested; namely, any sentence containing a critical test item in the middle of it was presented only after the item had already been tested. All participants read the same list of stimuli; although the order of presentation of each stimulus within blocks was randomized, the order of presentation of the blocks was not.

Procedure

Participants were seated approximately 100 cm from a CRT computer monitor in a sound-attenuated and electrically shielded recording booth. Participants were given an explanation of the experimental procedure, as well as a short practice session to familiarize them with the task. The experiment was divided into 10 encoding-test blocks. During each encoding phase, participants read sentences word by word in a RSVP format and were instructed to try to remember what they read, as their memory would be tested later. Each word appeared in the center of the screen for 200 msec, followed by a 300-msec interstimulus interval (a blank screen). After the last word of the sentence was presented, a blank screen was presented for 500 msec, followed by a fixation cross for 1000 msec. Participants were instructed to try not to blink when they were reading the sentence and to blink and rest their eyes once the fixation cross appeared. Following each encoding phase, participants were given math problems to complete for 30 sec as a distractor between the encoding and test phases.

Following each encoding phase, participants were tested on their memory in the test phase. Each trial of the test phase began with a fixation cross in the center of the screen for 1000 msec, which was then replaced by a test item (a sentence ending word, a new word, or a sentence medial filler). After 1000 msec, a confidence scale appeared underneath the test word, at which point participants could make their response. Upon making a response, the trial would end and the next trial would begin. The confidence scale consisted of 4 points—“Sure New,” “Maybe New,” “Maybe Old,” and “Sure Old.” Participants were instructed to try not to blink during the initial presentation of the word, but once the confidence scale appeared and they could make their response, they could blink. The test phase was self-paced, in that participants could take as long as they needed to respond.

EEG Recording and Preprocessing

EEG data were recorded from 26 Ag/AgCl electrodes embedded into a flexible elastic cap and distributed over the scalp in an equidistant arrangement. Additional facial electrodes were attached for monitoring of EOG artifacts, including one adjacent to the outer canthus of each eye and one below the lower eyelid of the left eye. Electrode impedances were kept below 5 kΩ. Signals were amplified by a BrainVision amplifier with a 16-bit A/D converter, an input impedance of 10 MΩ, an online bandpass filter of 0.016–250 Hz, and a sampling rate of 1 kHz. The left mastoid electrode was used as a reference for online recording; offline, the average of the left and right mastoid electrodes was used as a reference.

Preprocessing of the EEG data was completed using functions from the EEGLAB (Delorme & Makeig, 2004) and ERPLAB (Lopez-Calderon & Luck, 2014) toolboxes in the MATLAB programming environment. Following data collection, each raw EEG time series was passed through a 0.1- to 30-Hz bandpass Butterworth filter with a 12-dB/oct roll-off. The signal then was segmented into epochs from −200 msec prestimulus to 1000 msec poststimulus, relative to the onset of each sentence ending word during encoding and each test item during the test phase. The 200 msec prestimulus was used as a baseline period and was averaged and subtracted from the poststimulus data. Ocular artifacts were corrected using the same procedure as in Hubbard and colleagues (2019). For participants' data containing a large number of blinks, the data were decomposed into independent components with Adaptive Mixture Independent Components Analysis (Hsu et al., 2018; Palmer, Kreutz-Delgado, & Makeig, 2012), and the correlation between each independent component timecourse and the bipolar VEOG channel (lower eye channel - left prefrontal channel) was calculated to find the component(s) containing blinks. Components with a high correlation were removed from trials marked as containing blinks. This ocular artifact correction process was performed for 36 of the 42 participants' data, and, on average, two components were removed. The remaining components were then recombined to reconstruct the EEG data, which were then scanned with an additional sliding window amplitude threshold (300-msec sliding time window, 50-msec step size, 90-μV threshold), and finally manually checked by the experimenter for any additional artifacts. In total, an average of 6% of trials were removed, with a range of 2%–16% across participants. Following these preprocessing steps, epochs were averaged together to create an ERP for each participant, and grand average ERPs were created by averaging participant ERPs. The grand average plotted ERPs were filtered with a 10-Hz lowpass filter for clarity of visualization and were only calculated for visualizing effects, not statistical measurements.

Statistical Analyses

Statistical analyses were conducted at the trial level to account for variance because of individual items. Both the behavioral and electrophysiological data were statistically analyzed using linear mixed-effects models, using the lme4 package in R (Bates, Mächler, Bolker, & Walker, 2015). Recognition memory performance was analyzed with generalized mixed-effects logistic regression models (Jaeger, 2008), predicting whether participants made a correct or incorrect recognition response (0 = incorrect, 1 = correct) on trial-level behavioral data. Fixed effects predicting accuracy for each analysis are detailed in the Results section, and the numerical coding of these fixed effects (i.e., dummy coding, contrast coding) are explicitly stated (Brehm & Alday, 2022).

Our analytic approach was to conduct statistical tests of specific condition contrasts when replicating effects from prior studies where the effects were known (e.g., when testing sentence-final ERP effects), and to use an omnibus ANOVA-like approach for conducting tests when the outcomes were less studied and our goal was to test main effects and interactions before specific condition contrasts (e.g., when examining the recognition memory results). In cases where a variable with three levels (i.e., expectancy) was tested, the procedure outlined by Levy (2014) was used, in which two numerical coding variables were entered into the model (e.g., X1, Level 1 = 0, Level 2 = 0.5, Level 3 = −0.5; X2, Level 1 = 0.5, Level 2 = −0.5, Level 3 = 0). To then test for significance of main effects and interactions, likelihood-ratio tests between mixed-effects models, differing only in the presence or absence of a fixed main effect or interaction, were conducted. Random effects were the same between models, and included intercepts for items and slopes and intercepts for participants, with slopes for the fixed effects of the model. When comparing different conditions of interest for testing simple effects, the relevel function in R was used to change the reference condition level in the mixed-effects model, and the model was recomputed. Note that no other aspects of the model were changed in these cases, and this process only changes the reference level to conduct specific tests (Linck, 2016). Word frequency of the target word was included as a fixed effect, because of the potential differences in frequency between conditions that could explain effects of interest (Sassenhagen & Alday, 2016), and because of the long-standing literature suggesting that word frequency can have significant effects on recognition memory (Glanzer & Adams, 1985, 1990). Word frequency values were log-transformed, scaled, and centered. When testing for simple effects (e.g., differences between specific conditions), Wald's z scores were computed for the coefficients of interest. Recognition performance data were plotted as bar plots, with error bars representing 95% within-subject confidence intervals calculated using the Cousineau–Morey method (Morey, 2008; Cousineau, 2005).

For the ERP analyses, statistical analyses were performed on trial-level measurements extracted from averaged activity in specified time windows and channel clusters, chosen to correspond with the previous study (Hubbard et al., 2019). For the N400, measurements were extracted from a central-posterior cluster of six channels (shown in Figure 2), from 250 to 500 msec. For the anterior positivity, a frontal cluster of six channels and a time window of 700–1000 msec was used (shown in Figure 2). For the posterior positivity, an occipital cluster of three channels and a time window of 700–1000 msec was used (shown in Figure 2; DeLong et al., 2014). For the LPC, a posterior cluster of seven channels and a time window of 500–800 msec was used (shown in Figure 4), and for the N1, a posterior cluster of five channels and a time window of 50–150 msec was used, based on the results of the cluster analysis in the previous study (shown in Figure 4). Trial-level amplitudes were predicted with linear mixed-effects models, with random effects including intercepts for items and slopes and intercepts for participants. Significance testing of main effects and interactions was conducted with likelihood-ratio tests between mixed-effects models, and testing of simple effects was conducted with t tests using the Satterthwaite approximation method in the lmerTest package in R (Kuznetsova, Brockhoff, & Christensen, 2017).

Figure 2. — ERPs time-locked to sentence final words during the encoding phase. Top: Time-course of ERPs at three different channel ROIs, highlighting different ERP components (the N400, anterior positivity, and posterior positivity). Dotted lines show the averaged time window for the topography plots. Bottom: Topography plots of the ERP effects in the top plot. The condition difference and time window are shown below each topography plot. The bolded channels depict the averaged channel ROIs for the top ERP plots.

Figure 4. — ERPs time-locked to match items during the recognition phase. Middle: Time-course of ERPs at the posterior channel cluster, with different ERP components (the N1 and LPC) labeled. Dotted lines show the averaged time window for the topography plots. Left: Topography plot of the N1 ERP effect, and bar plots of condition differences in amplitude. Right: Topography plot of the LPC ERP effect, and bar plots of condition differences in amplitude. Error bars depict 95% within-subject confidence intervals calculated using the Cousineau–Morey method.

Brain–Behavior Correlations

Engagement of the post-N400 processes following prediction violations (the anterior and posterior positivities) may impact downstream recognition memory for the prediction violations, or the predicted lures that the prediction violations replaced. To directly test this hypothesis, we conducted brain–behavior correlation analyses to relate the magnitude of the ERP responses during encoding to later recognition memory. Isolating the processes indexed by ERPs generally involves measuring a difference in amplitude between conditions, which renders relating single trial ERP measurements to behavioral outcomes difficult (Klawohn, Meyer, Weinberg, & Hajcak, 2020; Meyer, Lerner, De Los Reyes, Laird, & Hajcak, 2017). In addition, correlations of ERP measurements with performance within a single condition of the recognition memory test may be contaminated by participants' overall response bias during the test (Rotello & Macmillan, 2007). Therefore, correlations were made at the participant level, rather than the trial level, and partial correlations were used to control for overall memory performance. In addition, Spearman partial correlations were used to relate variables, as this method is more robust to outliers than Pearson correlations (Pernet, Wilcox, & Rousselet, 2013; Rousselet & Pernet, 2012).

For each participant, anterior positivity amplitudes were calculated as the difference between the average anterior positivity in the SCU condition and the average anterior positivity in the WCE condition (where no engagement of the anterior positivity, or other frontal effects that have sometimes been observed in this paradigm, was expected; see Hubbard & Federmeier, 2021b), and posterior positivity amplitudes were calculated as the difference between the average posterior positivity in the SCA condition and the average posterior positivity in the WCE condition. Partial correlation analyses entailed correlating subject-level anterior positivity amplitudes with average “old” responses for SCU matches, controlling for average old responses across all match conditions (thus controlling for response bias). Anterior positivity amplitudes were also correlated with average old responses for SCU lures, controlling for average old responses across all lure conditions. The same process was conducted for posterior positivities, SCA matches, and SCA lures. Partial correlation analyses were conducted with the ppcor package in R (Kim, 2015), which also provided t and p values for testing significance of the correlations.

Results

Recognition Memory Performance

Overall recognition performance across all of the test item types, plotted as proportion old response, is presented in Figure 1A. Participants were successfully able to discriminate old items (matches) from new items (lures and new words). There appeared to be an increase in false alarms for lure items compared with new items, as was observed previously (Hubbard et al., 2019). To differentiate conditions, we labeled expected ending lures that were replaced by unexpected but plausible words as unexpected lure (UL), whereas expected ending lures that were replaced by semantically anomalous words as anomalous lure (AL).

To assess the differences between conditions statistically, mixed-effects logistic regression models predicting accuracy were used to compare conditions, with word frequency included to account for this variance. First, we examined the false alarms to lure items by comparing the four lure conditions (SC-UL, WC-UL, SC-AL, and WC-AL) to the new item condition. The first analysis predicted recognition rates with a fixed effect of condition consisting of the four lure conditions combined, compared with new items (contrast coded, lure = 0.5, new = −0.5), as well as a fixed effect of confidence (contrast coded, sure = 0.5, maybe = −0.5). This analysis resulted in significant main effects of Condition, χ²( 1) = 22.69, p < .01, and Confidence, χ²(1) = 16.41, p < .01, but no significant interaction, χ²(1) = 1.41, p = .24. To assess if each of the lure conditions differed in false alarms, an analysis was conducted with a fixed effect of condition that was dummy coded, such that each of the four lure conditions were compared with the new item condition (i.e., simple effects). Every lure condition significantly differed in accuracy to new items (SC-UL, β = 0.66, z = 2.91, p < .01; WC-UL, β = 0.94, z = 4.05, p < .01; SC-AL, β = 0.88, z = 3.84, p < .01; WC-AL, β = 0.75, z = 3.09, p < .01), demonstrating that false alarms to lures were observed for all conditions. Word frequency also predicted false alarms in the expected direction, with higher frequency words leading to greater false alarms (β = 0.36, z = 6.65, p < .01). Thus, participants made more false alarms and were more confident in their judgments to lure items compared with new items.

We next tested whether there were differences in false alarms between the lure conditions. Fixed effects of Constraint (two levels, SC and WC; contrasted coded, SC = 0.5, WC = −0.5), Confidence (two levels, sure and maybe; contrast coded, sure = 0.5, maybe = −0.5), and Expectancy (two levels, UL and AL; contrasted coded, AL = 0.5, UL = −0.5), as well as the interaction between these effects, were included in a mixed-effects logistic regression model predicting accuracy. There were no significant main effects of Constraint or Expectancy, Constraint, χ²(1) = 0.06, p = .80; Expectancy, χ²(1) = 0.001, p = .97, and the interaction between these variables was not significant, χ²(1) = 1.40, p = .24. However, there was a significant main effect of Confidence, χ²(1) = 11.80, p < .01, as well as a significant interaction of Confidence and Constraint, χ²(1) = 6.42, p = .01. This is observable in Figure 1B; namely, participants were more confident in their false alarms when the lure was from a SC sentence. Thus, increased false alarms were observed for each of the four lure conditions, but the rate of false alarms did not differ between conditions, although confidence judgments were higher for SC lures.

The next analyses focused on the match items. We tested whether accuracy differed between match conditions with a mixed-effects logistic regression model including fixed effects of Constraint (two levels, SC and WC; contrasted coded, SC = 0.5, WC = −0.5), Expectancy (three levels, E, U, and A; contrasted coded with two variables), Confidence (contrast coded, sure = 0.5, maybe = −0.5), the interaction of these factors, and word frequency. The model reported significant main effects of Constraint, χ²(1) = 9.15, p < .01, and Expectancy, χ²(2) = 12.91, p < .01, as well as a significant interaction of these factors, χ²(2) = 7.42, p = .02. There was a significant main effect of Confidence, χ²(1) = 63.75, p < .01, but Confidence did not significantly interact with any other factor. In addition, word frequency significantly impacted recognition in the expected direction, with lower frequency words leading to more hits (β = −0.10, z = −2.02, p = .04).

Follow-up simple effects tests were conducted to unpack the interaction between Constraint and Expectancy. Recognition performance did not significantly differ between SC-E match items and WC-E match items (β = 0.03, z = 0.15, p = .88), and accuracy for WC-E match items did not significantly differ from WC-U match items (β = 0.21, z = 1.02, p = .31). However, accuracy was significantly higher for SC-U match items compared with SC-E match items (β = 0.48, z = 2.22, p = .03), as well as compared with WC-U items (β = 0.66, z = 3.06, p < .01). Finally, performance for WC-A match items was greater than WC-U match items (β = 0.43, z = 2.07, p = .04), and accuracy for SC-A match items showed a similar (but nonsignificant) pattern compared with SC-U match items (β = 0.46, z = 1.92, p = .06). Thus, there was an overall pattern for better recognition memory for prediction violation test items, and this pattern was larger for violations of SC sentences compared with WC sentences.

To summarize the behavioral results, participants were more likely to false alarm to lure test items compared with new test items, and this effect did not differ based on sentence constraint or sentence ending type (unexpected or anomalous), although individuals were more confident in their memory judgments when the false alarms were from SC sentences. On the other hand, sentence constraint and ending type did have an impact when examining recognition memory for match test items. Prediction violations were better recognized than expected endings, and this effect was larger when constraint was high.

Sentence Final ERPs

Grand average ERPs to sentence final words during the encoding phase are plotted in Figure 2. ERPs were statistically analyzed to determine if the prior effects on ERPs of interest seen with these materials (e.g., Federmeier et al., 2007) were replicated.

First, N400 amplitudes were analyzed to determine if the effects of sentence constraint and expectancy on the N400 were replicated. N400 amplitudes were compared between WCE endings and SCE endings, as well as between WCE and unexpected (U) endings (collapsed across constraint) and anomalous (A) endings (collapsed across constraint), because previous research has demonstrated that N400s to prediction violations largely do not differ based on constraint. There were significant differences in N400 amplitude between WCE and U endings (β = −1.64, t = −4.66, p < .01), as well as between WCE and A endings (β = −2.87, t = −6.97, p < .01). Surprisingly, the difference in N400 amplitudes between SCE and WCE endings was not statistically significant (β = 0.54, t = 1.37, p = .17), possibly because of post-N400 differences in the measurement window.¹ Thus, the graded N400 effect was largely replicated in this experiment.

Next, we examined anterior positivity amplitudes to determine if SCU sentence endings elicited greater anterior positivity ERPs. Previous work has operationalized the anterior positivity effect as a significant difference between SCU and WCU endings (Federmeier et al., 2007), or a difference between expected (E, collapsed across constraint) endings and SCU endings (DeLong et al., 2014). However, frontally distributed negativity responses have also been observed following expected endings to more highly constraining sentences (Wlotko & Federmeier, 2012). Therefore, the WCE condition, where neither a positivity nor a negativity is expected to be elicited, was used as baseline comparison condition. The difference between SCU endings and anomalous (A) endings was tested as well. The difference in anterior positivity amplitudes between SCU and WCU endings was not significant (β = −0.75, t = −1.55, p = .13). In contrast, the difference in anterior positivity amplitudes elicited by SCU and WCE endings was significant (β = −1.24, t = −2.59, p = .01), as was the difference between SCU and A endings (β = −1.43, t = −3.11, p < .01). To determine the specificity of the anterior positivity effect, an additional analysis compared anterior positivity amplitudes elicited by SCA endings to WCE endings; this comparison was not significant (β = 0.08, t = 0.14, p = .89). Thus, unexpected but plausible endings to SC sentences elicited an anterior positivity response.

Last, we examined posterior positivity amplitudes to determine if SCA sentence endings elicited greater posterior positivity ERPs. We tested the difference between SCA and WCA endings, as well as the difference between SCA endings and WCE endings, and SCA and unexpected (U) endings. All three comparisons were significant; specifically, posterior positivity amplitudes elicited by SCA endings were more positive compared with WCA endings (β = −1.07, t = −2.70, p = .01), WCE endings (β = −1.22, t = −2.93, p < .01), and U endings (β = −1.24, t = −3.43, p < .01). To determine the specificity of the posterior positivity effect, an additional analysis compared posterior positivity amplitudes elicited by SCU endings to WCE endings, which we used as a baseline condition for consistency with the anterior positivity analysis. This analysis produced a nonsignificant result (β = 0.58, t = 1.62, p = .11). Thus, semantically anomalous endings to SC sentences elicited a posterior positivity response.

To summarize the sentence-final ERP results, we replicated effects that were observed in prior studies. Namely, N400 amplitudes varied based on sentential constraint and expectancy, anterior positivites were elicited by unexpected endings of SC sentences, and posterior positivites were elicited by anomalous endings of SC sentences. Participants elicited neural responses that were indicative of engagement of anticipatory processing during reading, and likely did not engage in radically different processing during sentence reading, allowing us to link responses during sentence reading to future behavior on the recognition test.

Brain–Behavior Correlations

Partial correlation analyses were conducted to relate the magnitude of post-N400 ERPs (the anterior and posterior positivities) to downstream memory for matches and lures. These analyses related ERP amplitudes to recognition performance at the participant level, while controlling for overall memory performance. The results of these partial correlations are plotted in Figure 3.

Figure 3. — Partial correlations between post-N400 ERP magnitudes (the anterior and posterior positivities) and downstream memory for matches and lures. ERP amplitudes were calculated as average measurements of the difference of the condition of interest (SCU for anterior positivities, SCA for posterior positivities) and the WCE condition. Accuracy reflects average “Old” response for the specific condition of interest, controlling for overall memory performance.

At the participant level, anterior positivity amplitudes elicited by SCU endings at the time of encoding were not correlated with successful recognition memory for SCU match items (r = .01, t = 0.04, p = .97), or for false recognition of predicted lures that were replaced by unexpected but plausible words (r = −.01, t = −0.06, p = .96). In contrast, although posterior positivity amplitudes elicited by SCA endings at the time of encoding were not correlated with false recognition of predicted lures that were replaced by anomalous words (r = .09, t = 0.53, p = .60), posterior positivity amplitudes were significantly correlated with recognition of SCA match items (r = .35, t = 2.33, p = .02). Thus, the magnitude of elicitation of posterior positivities was related to future memory for semantically anomalous endings. In summary, elicitation of the anterior positivity to unexpected endings during sentence reading was unrelated to recognition of matches or lures during the recognition test, whereas elicitation of the posterior positivity to anomalous endings during sentence reading was related to downstream recognition of these endings, although not to predictable lures, during the memory test.

Recognition ERPs

Grand average ERPs to match items during the recognition memory test are plotted in Figure 4. Only correct responses were included. ERP components were statistically analyzed to assess differences in recognition memory processes between the different types of items.

The first analysis examined differences in N400 amplitudes between conditions. N400 amplitudes elicited by match items were predicted in a mixed-effects linear regression model including fixed effects of Constraint (two levels, SC and WC; contrasted coded, SC = 0.5, WC = −0.5), Expectancy (three levels, E, U, and A; contrasted coded with two variables), and the interaction of these factors. The model reported no significant main effect of Constraint, χ²(1) = 1.95, p = .16, or Expectancy, χ²(2) = 4.57, p = .10, as well as no significant interaction of these factors, χ²(2) = 3.65, p = .16. Thus, there were no statistical differences in N400 amplitudes between match item conditions during the recognition test.

The next analysis examined differences in LPC amplitudes between conditions. A mixed-effects linear regression model analysis with the same fixed effects as previously described was conducted. The model reported no significant main effect of Constraint, χ²(1) = 0.15, p = .70; however, the effect of Expectancy was significant, χ²(2) = 9.04, p = .01. The interaction of these factors was not statistically significant, χ²(2) = 0.12, p = .94. Follow-up simple effects tests were conducted to unpack the significant effect of Expectancy. Amplitudes of LPCs elicited by unexpected (U) items were significantly more positive than for expected (E) items (β = −1.49, t = −2.99, p < .01), as well as for anomalous (A) items (β = −1.17, t = −2.15, p = .03). There was no significant difference in LPC amplitudes between E and A items (β = −0.32, t = −0.62, p = .54). In summary, match items that were previously unexpected but plausible sentence endings elicited larger LPCs than previously expected or semantically anomalous endings.

The final analysis examined differences in N1 amplitudes between conditions. A mixed-effects linear regression model analysis with the same fixed effects as previously described was conducted. The model reported no significant main effect of Constraint, χ²(1) = 0.46, p = .50, whereas the effect of Expectancy was significant, χ²(2) = 12.80, p < .01. The interaction of these factors was not statistically significant, χ²(2) = 1.47, p = .48. Follow-up simple effects tests were conducted to unpack the significant effect of Expectancy. Amplitudes of N1s elicited by anomalous (A) items were significantly more negative than for expected (E) items (β = 0.86, t = 2.99, p < .01), as well as for unexpected (U) items (β = 0.88, t = 3.10, p < .01). There was no significant difference in N1 amplitudes between E and U items (β = 0.02, t = 0.06, p = .95). In summary, match items that were previously semantically anomalous sentence endings elicited larger N1s than previously expected or unexpected but plausible endings.

To summarize the recognition test ERP results, we found that unexpected test items elicited larger LPC amplitudes than both expected and anomalous test items. In addition, we found that anomalous test items elicited greater amplitude N1 responses compared with expected and unexpected test items.