Co-Registration of EEG and Eye-Tracking in Psycholinguistics and Bilingualism Research

Abstract

Eye-tracking and electroencephalogram (EEG) recordings provide a window into the human mind. Combining both methods through co-registration offers a powerful and innovative approach for examining how cognitive and neural processes unfold in real time. While both EEG and eye-tracking have independently advanced our understanding of bilingual language processing, co-registration research in the field of bilingualism remains scarce. Given the potential of co-registration methodology to integrate multiple sources of information, this article provides a theoretically and empirically informed perspective on how it can be applied to bilingualism research. Drawing on findings from EEG and eye-tracking studies on bilingual language processing, as well as co-registration research in reading, we identify key areas of bilingualism, such as crosslinguistic activation and inference revision, where co-registration can help address existing gaps. We also discuss several methodological challenges associated with co-registration and offer practical recommendations for its effective implementation in future studies. We conclude that co-registration can move research on bilingual language processing towards a more integrated perspective, one that better captures the dynamic interplay between language and cognition.

1. Introduction

Imagine trying to fully appreciate a symphony by only listening to the violins or percussion instruments separately, rather than listening to how they interact over time. The melody might still be recognizable, but the full depth, aesthetic power, and emotion of the composition would be lost. Similarly, research on bilingual language processing has benefited from behavioral, eye-tracking, and EEG approaches, each offering complementary insights into bilingualism and cognition. Behavioral paradigms involving language switching, picture naming, or priming (e.g., [1-3]) have been instrumental in characterizing performance outcomes, typically indexed by response times and accuracy rates [4]. However, behavioral outcomes primarily reflect the end products of language processing and not the underlying mechanisms that produce them. To move beyond the limitations of behavioral measures, psycholinguists have widely implemented non-invasive techniques such as eye-tracking and electroencephalography (EEG). Eye-tracking records eye movements and pupil dilation while participants process words or visual scenes, allowing researchers to track, with millisecond precision, gaze direction and duration in real time. From eye-movement recordings, researchers can extract a range of metrics that act as indicators of different stages of linguistic and cognitive processing, from early lexical access (e.g., via first fixation duration) to later processing difficulty (e.g., via regression path duration, pupil dilation). Eye-tracking thus offers a window into how individuals anticipate, process, and integrate linguistic information [5]. EEG, on the other hand, records patterns of electrical brain activity through electrodes placed on the scalp. With its high temporal resolution, EEG allows researchers to track the brain processes that underpin language comprehension and production [6]. Event-related potentials (ERPs), derived from EEG recordings, serve as markers of specific cognitive processes and provide precise temporal information about when and how language-related mechanisms are engaged [7].

Eye-tracking and EEG have each significantly advanced the field of psycholinguistics, improving methodological rigor and precision in the study of language processing (see [8] for a review on EEG; and [9] for a review on eye-tracking). Yet, both methods have traditionally been employed in isolation (see Figure 1). Combining EEG and eye-tracking through co-registration allows researchers to leverage the methodological strengths of both techniques to more effectively investigate the effects of language processing on broader cognitive functioning and to investigate the implications of these effects for theoretical models of bilingual language processing. Co-registration improves rigor of experimental research in several ways. First, when EEG and eye-tracking are combined, the information obtained is more than the sum of the individual measures, creating a dataset in which interactions between gaze patterns and brain activity can be examined. Moreover, eye-tracking and EEG provide complementary sources of information: eye-tracking indexes gaze patterns that reflect visual processing, whereas EEG captures patterns of brain activity in response to stimulus processing. By integrating the two methods, researchers can leverage their combined strengths to monitor where attention is directed and for how long, identify the stages of cognitive processing involved, and capture the associated neural responses. In psycholinguistics, co-registration is particularly useful because it enables researchers to record the brain activity of participants during natural viewing tasks, such as reading and scene viewing [10, 11]. As a result, co-registration has the potential to provide fine-grained and ecologically valid insights into the mechanisms underlying language and cognitive processing.

Figure 1.

Number of publications over the last 25 years in psycholinguistics research using either EEG, eye-tracking, or co-registration of EEG and eye-tracking from Web of Science (WoS).

By combining neural and eye-tracking data, co-registration can help evaluate and improve theoretical accounts of bilingual language processing. For instance, experimental studies in bilingualism suggest that during comprehension, both languages are simultaneously activated (i.e., a non-selective view; [12-14]). These findings have prompted the development of several connectionist and computational models of word comprehension (e.g., [15,16]), production (e.g., [17]), or both (e.g., [18]). In the present review, we adopt the Bilingual Language Interaction Network for Comprehension of Speech (BLINCS; [19]) as a framework for illustrating how model-based predictions can be evaluated using co-registration.

BLINCS is a contemporary computational model that builds on earlier monolingual and bilingual interactive frameworks to capture the dynamic nature of bilingual language processing. The model assumes parallel activation of both languages and emphasizes crosslinguistic competition across multiple levels of representation (i.e., phonological, phono-lexical, ortho-lexical and semantic) during spoken word comprehension. BLINCS also incorporates the influence of visual information on language processing. Through bidirectional interactions across levels, the model can be used to generate time-sensitive predictions about how bilinguals allocate attentional resources during speech comprehension, making BLINCS particularly well suited for evaluation using both neural and eye-tracking techniques [20-21]. A central feature of BLINCS is its focus on variability within the bilingual language system. The model incorporates both long-term experiential factors (e.g., age of acquisition, proficiency, language dominance) and short-term contextual factors (e.g., language exposure), as well as lexical properties known to influence language processing (e.g., word frequency, neighborhood density). By linking these sources of variability to dynamic interactions across levels of representation, BLINCS provides a principled framework for testing hypotheses about bilingual language processing. In this context, co-registration allows for a precise examination of the temporal relationship between visual attention and neural processing, capturing in real time the dynamics of crosslinguistic activation during language comprehension.

Despite its strengths, relatively few psycholinguistic studies have employed co-registration. Recently, however, technological improvements have reduced previous limitations (for a review see [22]), making co-registration a promising new methodology for use in psycholinguistics research. The present article highlights the potential of combining brain activity and eye movement data to address questions in bilingual language processing research. We first outline key findings from bilingualism research that have employed EEG and eye-tracking independently. We then briefly review a sample of studies on co-registration in psycholinguistics and identify theoretical and practical gaps in bilingualism research that could benefit from this approach. Finally, we discuss methodological challenges of integrating EEG and eye-tracking and propose strategies to overcome these barriers to advance psycholinguistic research with bilingual populations.

2. EEG insights into bilingual language processing: Current evidence from psycholinguistic research

EEG is widely used in cognitive and linguistic research due to its ability to capture the brain’s rapid response to a variety of stimuli while remaining relatively easy to implement and cost-effective [23]. Nevertheless, EEG signals are inherently complex, requiring careful preprocessing to isolate the neural patterns associated with specific experimental task conditions. Event-related potentials (ERPs) address this challenge by capturing systematic, time-locked changes in the EEG signal in response to specific events, such as hearing a word, seeing a picture, or making a decision. By averaging EEG responses across events, researchers can identify consistent peaks and valleys (i.e., ERP components) in brain activity that reflect underlying neural processes elicited by each event.

Research using ERPs in language processing has revealed several “canonical” components that are consistently observed across psycholinguistic studies [8]. While some ERP components are associated with early, perceptual stages of language processing, those emerging in later time windows (200-600ms) are often sensitive to experimental manipulations that affect semantic processing and the need for reanalysis during comprehension. Late ERP components are especially relevant to research on bilingualism, as they reflect the cognitive mechanisms that underlie and support language. In the following subsections, we briefly review four ERP components: N400 (semantic processing; [7]), P600 (syntactic processing and reanalysis; [24]), N2 (cognitive control and conflict monitoring; [25]), and P300 (updating and working memory; [25]), all of which have been shown to be shaped by bilingualism (see [26] for a review) and are relevant for understanding the potential of co-registration. For each ERP component, we describe its functional characteristics (see Table 1) and review key empirical findings related to bilingual language processing. Finally, we also consider recent findings in bilingualism research that leverage neural oscillatory analyses (see [27]).

Table 1.

Key ERP Components Associated with Bilingual Language Processing and Control

MEASURE	DESCRIPTION	ASSOCIATED PROCESSES
N400	~400 ms; centro-parietal (modality-dependent)	Semantic processing, lexical access, and prediction
P600	~500–800 ms; centro-parietal / sometimes frontal	Syntactic reanalysis, conflict monitoring, and integration
N2	~200–350 ms; fronto-central	Conflict detection, inhibition, and early control
P300	~300–600 ms; centro-parietal	Working memory, context updating, and proactive control

2.1. N400

The N400 ERP component is a negative-going waveform that typically peaks around 400 ms. First reported by Kutas and Hillyard [28], the N400 has been widely interpreted as a neural marker of semantic processing and integration. For example, incongruent words in semantically constraining sentences (e.g., “He spread his toast with socks”) elicit larger N400 amplitudes than congruent words (e.g., “He spread his toast with butter”). The N400 is not limited to linguistic input and has been observed across different modalities [23]. For instance, semantically incongruent facial expressions (e.g., [29]) evoke similar N400 effects as incongruent words, indicating that the component is not limited to linguistic processing. This finding supports the view that the N400 is a domain-general marker of semantic processing (for a review, see [7]). The N400 has also been linked to predictive processing, as the brain uses context to anticipate upcoming information [30, 31]. The N400, therefore, serves as an index of semantic processing, sensitive to both context and prediction across modalities.

In bilinguals, the N400 tends to be smaller in amplitude and delayed in the second language (L2) compared to the first language (L1) (e.g., [32-34]), indicating that processing semantic information in the L2 is generally less automatic and requires greater cognitive effort. The N400 is also sensitive to crosslinguistic activation, with reduced amplitudes for target words overlapping phonologically or semantically with their translation equivalents in the non-target language [35-37]. Furthermore, bilinguals perceive unrelated concepts as more semantically related than monolinguals do, as shown in a smaller N400 effect (i.e., reduced N400 amplitudes for related compared to unrelated pictures) (e.g., [38, 39]). N400 effects are also modulated by individual differences in language experience such as proficiency, age of acquisition, and daily exposure (e.g., [40, 41]). In sum, the N400 reveals how semantic information is processed across languages and different bilingual experiences.

2.2. P600

The P600, first reported by Osterhout and Holcomb [42], is a positive-going waveform that typically peaks around 500 ms post-stimulus onset and can persist for several hundred milliseconds. This positivity appears in response to ungrammatical sentences and other anomalies [43]. The P600 is elicited across both written and spoken modalities by a variety of syntactic violations, such as those found in garden-path sentences [44], and has also been linked to thematic role anomalies, discourse integration, and pragmatic processing [44-46].

In studies on bilingual language processing, the P600 has been shown to be smaller in amplitude or delayed in the L2 compared to the L1 [33, 47]. Greater P600 amplitudes have also been observed in response to less frequent but grammatical L2 structures [48]. In both cases, researchers have suggested that these patterns reflect increased processing effort and less engagement of control mechanisms during syntactic reanalysis in L2. These P600 effects vary by age of acquisition and proficiency, with early and highly proficient bilinguals showing neural responses that closely resemble those of monolingual native speakers [49,50]. Similarly, P600 effects have been found during code-switching [51], reflecting the engagement of cognitive control processes, as bilinguals must manage competition between two active languages. Together, these findings suggest that P600 modulations can index either syntactic processing difficulty (e.g., when processing in the L2) or the engagement of cognitive control processes (e.g., during code-switching).

2.3. N2

The N2 is a negative-going waveform peaking around 200–350 ms post-stimulus onset and reflects early engagement of cognitive control mechanisms [52]. For instance, in Go/No-Go paradigms participants respond to go trials and withhold their response to no-go trials. Compared to go trials, infrequent no-go trials elicit a larger N2, reflecting the recruitment of inhibitory control and conflict monitoring [53, 54]. Therefore, the N2 serves as a neural marker of cognitive control, which is relevant for studying how bilinguals manage competition between simultaneously activated languages [55].

Research on bilingual language processing has shown that the N2 is sensitive to language-switching demands. For example, Jackson et al. [56] observed larger N2 amplitudes during language switches, suggesting the involvement of control processes in language selection. Subsequent studies confirmed this switch effect, linking the N2 to greater demands in conflict monitoring [57]. Training in language switching has been shown to enhance conflict monitoring efficiency. This is evidenced by shorter N2 latencies [58-59] and increased N2 amplitudes for non-trained executive control tasks [60], suggesting transfer effects of language training to broader cognitive control functions.

N2 modulations in bilinguals have also been observed in non-linguistic tasks [61]. Compared to monolinguals, bilinguals often show larger N2 amplitudes and earlier latencies in Go/No-Go and Flanker tasks, suggesting enhanced domain-general conflict monitoring abilities [62, 63]. However, language group differences are not always observed. A meta-analysis by Kousaie and Phillips [65] revealed that differences in N2 amplitudes between monolinguals and bilinguals are often task-dependent and tend to emerge most clearly under conditions of high conflict (see [66] for similar effects in the P300 component).

Finally, factors such as L2 proficiency and age of second language acquisition modulate N2 responses. Higher proficiency has been linked to larger N2 amplitudes in auditory no-go trials [67, 68], while both early L2 learners [69] and bilingual children [63] show earlier N2 onset latencies than monolinguals, suggesting more efficient conflict detection. Overall, the N2 emerges as a reliable marker of early control processes in bilinguals, reflecting adaptation to the demands of managing two active languages.

2.4. P300

Studies exploring the N2 component frequently report effects on the P300 component, which is a positive deflection emerging between 300-600 ms after stimulus onset [56]. The P300 is commonly divided into P3a and P3b [70]. The P3a, with a frontal distribution and shorter latency, is generally associated with attentional capture and response inhibition [71, 72]. The P3b, with a centro-parietal distribution, appears later and is closely linked to updating and working memory processes [73].

Studies using nonverbal executive control tasks have consistently shown that bilinguals exhibit larger P300 amplitudes and shorter latencies than monolinguals, particularly in high-conflict conditions (e.g., Go/No-Go or Flanker tasks [74, 67, 75]). These findings have been interpreted as more efficient stimulus categorization and faster engagement of attentional control in bilinguals. Moreover, bilinguals immersed in dual-language contexts show increased P3b amplitudes during memory tasks [75], suggesting greater engagement of context monitoring and updating mechanisms under highly demanding conditions. However, tasks involving linguistic conflict (e.g., Stroop) sometimes show greater P300 amplitudes for monolinguals, possibly reflecting differences in neural recruitment strategies [66]. Together, these findings highlight the P300 as a valuable neural marker for tracking how bilinguals allocate control resources when managing two language systems and adapting to varying task demands [65].

2.5. Neural oscillations in bilingualism research

EEG recordings not only capture ERPs, but they also allow the analysis of neural oscillations, offering complementary information about brain activity [76]. Neural oscillations are rhythmic patterns of neural activity that support communication between different brain regions and cognitive systems. Through time-frequency or synchronization analyses, neural oscillations are typically characterized in terms of frequency bands (e.g., theta, beta, gamma) and their associated power (i.e., the magnitude of oscillatory activity within a given frequency band). Importantly, neuroscience research has shown that distinct patterns of brain oscillations are linked to different cognitive processes [77]. For example, theta oscillations have been associated with memory processes, whereas alpha oscillations are commonly linked to attentional allocation and control mechanisms.

In terms of language processing, theta-band activity has been consistently linked to word learning, lexical access, and memory consolidation [78,79]. Previous studies have shown that newly learned words elicit increased theta power, suggesting a role of theta oscillations in memory encoding [81,82]. Similar theta modulations have been observed in bilingual contexts, such as during word translation, in which theta oscillations are associated with the activation of semantic representations [82]. Beyond word-level processing, neural oscillations also capture higher-level processing. While decreases in alpha and beta power are associated with increased syntactic and semantic demands, increases in theta and delta activity are associated with the coordination of working memory processes during semantic processing [83]. Importantly, bilinguals often show reduced alpha and beta modulations compared to monolinguals despite similar behavioral performance, which has been interpreted as more efficient neural processing during sentence comprehension [84].

Neural oscillations are also sensitive to individual differences and task demands during bilingual language processing (e.g., [85]). For example, Blanco-Elorrieta et al. [86] found that during sentence processing, delta activity was modulated by language proficiency, with more proficient L2 speakers being better able to track sentences under more demanding noise conditions. Other studies have highlighted how oscillatory dynamics capture language-related differences in broader cognitive domains, such as creativity. For instance, Jonczyk et al. [87] found lower alpha power when bilinguals completed a creative task in L1 compared to L2, along with greater beta desynchronization in L2 than in L1, suggesting that operating in an L2 is associated with reduced interference from competing mental representations. Another commonly used methodological approach to analyze neural oscillations involves resting-state recordings [88]. In psycholinguistics research, resting-state recordings have already been examined in a growing number of studies exploring the impact of prior bilingual experience on neural oscillations (e.g., [89,90]). For instance, Amorouso and colleagues [91] found that delta and beta power can distinguish bilinguals from monolinguals, pointing to experience-related changes in brain organization. Complementing these findings, resting-state EEG work has shown that age of acquisition and patterns of language use are associated not only with changes in oscillatory power (beta, gamma), but also with connectivity measures across theta, alpha, and gamma bands [92]. Overall, these studies provide evidence that bilingualism reorganizes neural networks.

Taken together, EEG research provides converging evidence that bilingual language processing engages a set of language-relevant and domain-general neural mechanisms that unfold over time and are shaped by language experience. ERP components capture distinct stages of semantic processing, syntactic reanalysis, conflict monitoring, and context updating, while oscillatory analyses reveal complementary frequency-based dynamics related to memory, attention, and control. These studies highlight the value of EEG for creating theoretically grounded predictions that can be further tested using EEG and eye-tracking co-registration. Note that in this paper, we will focus on ERP measures when formulating predictions for open questions in the field of bilingualism (refer to Sections 4.1 and 4.2), as ERPs have been more widely used in previous co-registration work in psycholinguistic research. The potential contribution of neural oscillations is discussed in the section on future directions (refer to Section 5). Next, we move from EEG to eye tracking and briefly review a sample of eye-tracking research that has advanced our understanding of bilingual language processing.

3. Eye-tracking insights into bilingual language processing: Current evidence from psycholinguistic research

Eye-tracking captures, with millisecond precision, where and for how long individuals fixate on specific stimuli. Because eye-movements are tightly linked to attentional and other cognitive processes, eye-tracking offers a window into the mechanisms that shape human cognition and language processing (i.e., the eye-mind hypothesis [93]). For example, by mapping eye-movement patterns during reading, researchers can determine whether and when a word is being considered, as well as the cognitive effort required [94, 95]. This link between gaze and cognition has made eye-tracking a widely used method in psycholinguistics, particularly suited to examining the rapid and transient mechanisms that unfold during language processing [5, 96]. Eye-tracking is also a non-invasive method that offers strong ecological validity, allowing participants to engage with tasks that closely resemble everyday activities such as reading a book or inspecting a visual scene [97]. In doing so, eye-tracking bridges laboratory rigor with real-world language use, making it an especially useful tool for investigating bilingualism and its impact on cognition.

To translate gaze behavior into quantitative data, researchers rely on a set of measures that serve as indices of attention and cognitive effort. The three most basic eye-tracking measures analyzed in psycholinguistic studies are fixations (i.e., when the eyes stop on a stimulus), saccades (i.e., rapid eye movements between fixations), and regressions (i.e., backward movements in the text) (see Table 2). These gaze events are informative for understanding both auditory and written comprehension processes at multiple linguistic levels, including at the phonological, morpho-syntactic, and discourse levels (see [96] for a review of the auditory modality; see [98] for a review of the written modality). From these key measures, additional metrics can be calculated to investigate different stages of language processing [9]. Broadly, these metrics fall into two categories: early measures (e.g., first fixation duration, first-pass reading time), which are generally associated with processes of word recognition and lexical access [99], and late measures [100], which often capture processes of integration and reanalysis.

Table 2.

Eye-Tracking Measures by Type: Basic Movements, Early, and Late Fixation

MEASURE	DESCRIPTION	ASSOCIATED TASKS
BASIC MEASURES
Saccade	Rapid movement of the eye.	Visual Scene Perception; Reading
Fixation	Pause of the eye on a specific location in the visual display.	Visual Scene Perception; Reading
Regression	Eye movement that returns to a previously viewed location in the visual display (i.e., backward saccade).	Visual Scene Perception; Reading
EARLY MEASURES
First fixation duration	The duration of the first fixation on an area of interest (AOI).	Visual Scene Perception; Reading
First pass reading time	The duration of all fixations on an area of interest (AOI) before the gaze exits.	Reading
LATE MEASURES
Total fixation duration	The duration of all fixations on an area of interest (AOI) during a trial.	Visual Scene Perception
Total reading time	The duration of all fixations on an area of interest (AOI) during a trial.	Reading
Fixation count	Sum of all fixations on an area of interest (AOI) during a trial.	Visual Scene Perception; Reading
Second pass reading time	Sum of all fixations on an area of interest (AOI) after it has been exited for the first time.	Reading
Regression-path duration	Sum of all fixations, including regressions to earlier parts of a sentence, before moving forward to the next region in the text.	Reading

In the following sections, we review studies that applied eye-tracking to investigate bilingual language processing across different psycholinguistics paradigms. Specifically, we examine how eye-tracking measures have been interpreted in studies employing the visual world paradigm and reading tasks. As with EEG, outlining these measures is essential for understanding how eye-tracking can be effectively combined with electrophysiological recordings in co-registration paradigms.

3.1. Crosslinguistic lexical activation and competition: Insights from the Visual World Paradigm

The Visual World Paradigm (VWP; [101]) has become a valuable method for studying spoken language comprehension. In a typical VWP experiment, participants listen to auditory stimuli while viewing a display containing a set of objects that vary in their relation to the speech input. For example, in studies examining spoken word recognition, the display may include: (1) a target item (e.g., beaker), whose label appears in the speech signal; (2) a competitor item (e.g., beetle), whose label overlaps with the target’s word form or meaning; and (3) control items (e.g., carriage), whose labels are unrelated to the speech input and serve as baselines [102]. The objects are often presented as line drawings, photographs, or realistic visual scenes on a computer screen (see [96] for a review).

Crucially, adding eye-tracking to the VWP has allowed researchers to address different questions in language processing (e.g., [103-105]). By manipulating the relationship between what participants see and hear, this approach can be used to examine how eye movements are influenced by the timing and location of visual input relative to speech [106]. As speech unfolds, listeners may shift their gaze from one object to another (e.g., from the competitor to the target), revealing the dynamics of real-time language processing. This interaction between gaze and the unfolding speech signal enables users of the VWP to investigate the time course of language processing, including mechanisms such as lexical activation, competition, and prediction. In the VWP, common early measures include fixation proportions, total fixation time, and the number of fixations and saccades time-locked to key points in the auditory input (refer to [5] as well as [9] for reviews).

Applied to bilingualism, the VWP has been particularly useful for uncovering parallel activation and competition across languages. Marian and Spivey [107] discovered that when Russian-English bilinguals heard an English word like marker, they showed a higher proportion of eye movements to a visual competitor whose Russian label (marka, “stamp”) was phonologically related, even though the task was conducted entirely in English. This finding, replicated across modalities and in a variety of language pairs, provides compelling evidence that bilinguals activate lexical candidates in both of their languages (e.g., [108-110, 21]). Subsequent visual world studies have shown that crosslinguistic activation varies with individual differences in proficiency, language dominance, and cognitive control (e.g., [111, 112]). For instance, Blumenfeld and Marian [113] found that bilinguals’ fixation patterns to between-language competitors (e.g., pool – pulgar, “thumb” in Spanish) differed systematically by language proficiency. Bilinguals with higher proficiency in a second language directed more fixations to crosslinguistic competitors, indicating greater parallel activation across languages. Similarly, Sarret et al. [114] showed that target fixation latencies were delayed when crosslinguistic competitors were present, highlighting the cost of resolving lexical competition for bilinguals. These findings underscore how the VWP’s temporal precision allows researchers to trace not only when crosslinguistic competition emerges but also how individual differences influence its resolution.

The VWP has also been used to study other cognitive processes in bilinguals including memory. Fernandez-Duque and colleagues [115] showed that Spanish-English bilinguals with high Spanish proficiency made more fixations to Spanish phonological competitors and subsequently remembered them better than control items, suggesting that heightened crosslinguistic competition can enhance both visual attention and memory for competing items. These findings highlight how eye movements can reveal not only the dynamics of crosslinguistic competition, but also the long-term effects of dual-language processing in shaping broader cognitive function.

3.2. Semantic processing and comprehension: Insights from reading tasks

Eye-tracking has also played a central role in uncovering the cognitive mechanisms underlying reading in bilinguals [116]. During reading, fixations are essential for recognizing words and integrating them into the overall meaning of the text [117]. Consequently, the temporal precision of eye-tracking has become instrumental in studies on reading for differentiating (early) word identification from (late) integration processes.

The distinction between early word identification versus late integration processes has been particularly useful for addressing a central question in bilingual reading research: whether bilinguals activate only one language when reading (selective access) or whether both languages are activated in parallel (non-selective access). Evidence strongly supports the non-selective view, indicating that bilinguals’ two languages interact during visual word recognition. For example, interlingual homographs (e.g., pie in English vs. pie “foot” in Spanish) elicit longer first-pass reading times than control words, reflecting between-language competition (e.g., [118, 119]). In contrast, cognates (e.g., piano in English and Spanish) yield shorter first fixation durations, indicating facilitation due to shared form and meaning (e.g., [120, 121]). Early fixation measures are also sensitive to lexical frequency, age of acquisition, and language proficiency, revealing how both individual and linguistic factors shape visual word processing [122-125]. Late measures, on the other hand, capture the cognitive demands involved in reading, including the detection of processing difficulty and the resolution of linguistic and contextual ambiguity (for a review see [126]). For instance, regression-path duration and total reading times increase when bilinguals encounter ambiguous syntactic structures, reflecting the costs of integration [127, 125]. These measures are thus highly informative for understanding morphosyntactic processing in bilinguals (e.g., [128, 129]). Examining both early and late measures provides detailed information about the different stages of bilingual language processing. Whereas early measures reveal how bilinguals manage crosslinguistic activation and competition during word recognition and lexical access, late measures capture the integration of syntactic and contextual information.

Across a variety of paradigms, eye-movement measures index both early stages of word recognition and later stages of semantic and syntactic processing, revealing robust evidence for parallel crosslinguistic activation in bilingual individuals ([19]). By testing theoretical predictions with real-time measures of visual attention during spoken and written language comprehension, eye-tracking provides a methodological bridge between model-based assumptions and empirical observations. Importantly, the temporal precision of eye-tracking closely matches that of EEG, making it possible to directly link eye-tracking measures to ERP components during naturalistic tasks. In the next section, we review studies that have applied co-registration of eye movements and ERPs to study language processing and discuss the methodological strength of co-registration for bilingualism research.

4. Co-registration in psycholinguistics and bilingualism research: Current knowledge and open questions

Co-registration with EEG and eye-tracking allows researchers to study how neural activity and gaze behavior unfold over time during language processing. A key advantage of this approach is that it enables the measurement of saccade- and fixation-related potentials (SRPs and FRPs), which are EEG signals time-locked to the onset of individual saccades and fixations. These measures make it possible to isolate neural responses associated with specific eye movements during experimental tasks, including those that require natural reading or scene viewing [10]. By combining EEG and eye-tracking measures, co-registration preserves ecological validity while providing the temporal precision needed to track how linguistic information is processed in real time.

Despite the potential of co-registration to offer new insights into the processes that are involved in human cognition (e.g., [11, 130]), relatively few psycholinguistic studies have adopted this approach. Most studies to date have focused on monolingual populations, primarily within the domain of reading, examining how readers process words when directly fixated in the foveal region as well as when previewing upcoming words in the parafoveal region (i.e., the area outside the fixation point that can be previewed without direct gaze). For example, Kretzschmar et al. [131] demonstrated that readers begin to semantically evaluate words even before they are directly fixated, as evidenced by parafoveal N400 effects. In contrast, Dimigen et al. [132] found that the N400 only emerged when semantically related words were fixated, highlighting the limits of parafoveal semantic access. Their findings also revealed an early positive component (200–280 ms) when readers previewed an identical word parafoveally. Beyond reading, co-registration has also been applied to other tasks involving scene viewing [133] and spoken language comprehension in monolingual populations [134], further highlighting the method’s utility. Nevertheless, despite developments in monolingual language processing, to date, co-registration research to study bilingual language processing remains extremely limited.

To our knowledge, only two studies have used co-registration of eye movements and brain activity with bilinguals. The first study, conducted by Antúnez et al. [135], examined whether bilinguals extract and integrate semantic information from words in the parafovea during reading. In this study, Basque-Spanish bilinguals read words in Spanish while previewing words in Basque that were either translations or unrelated words. The combined use of EEG and eye-tracking allowed the authors to derive FRPs time-locked to eye movements on the target word, revealing greater N400 effects for semantically unrelated words in the parafovea. The findings show not only the early stages of semantic processing, but also how activation extends across languages, underscoring the value of co-registration for studying lexico-semantic access.

A second study by Egurtzegi et al. [136] used co-registration to look at gaze patterns and EEG band power oscillations during sentence planning in bilinguals. The authors found that Basque-German speakers began organizing sentence structures earlier in Basque, as indicated by longer fixations on the agent and increased frequency synchronization in the theta and alpha bands. In contrast, when speaking German, sentence planning was delayed, as indicated by greater desynchronization in these power bands. However, while both EEG and eye-tracking data were simultaneously recorded, Egurtzegi et al. [136] analyzed the eye tracking and EEG data independently, rather than using co-registration to integrate the two measures (e.g., fixation-related oscillations, [137]).

Next, we provide examples of research domains (e.g., crosslinguistic co-activation and inference revision) where co-registration can help address current gaps in bilingualism. In particular, we discuss the implementation of co-registration in the Visual World Paradigm [101] to study the influence of co-activation in higher-order cognitive domains, such as memory, decision making, and semantic processing. We also consider other areas where co-registration can be applied in research with bilinguals, such as the Situation Model Revision Task [138]. Critically, these research domains represent questions that cannot be fully addressed using EEG or eye-tracking in isolation. While each technique provides valuable information, co-registration allows researchers to determine how neural activity and gaze patterns unfold during language processing, offering theoretical power beyond either method alone.

4.1. Investigating crosslinguistic co-activation through co-registration

One promising application of co-registration lies in its potential to extend findings of parallel crosslinguistic activation derived from studies using the VWP (e.g., [114, 139, 140]). Because eye-tracking provides a precise index of visual attention and perception during language processing, integrating eye-tracking with EEG in the VWP makes it possible to time-lock shifts in attention with corresponding neural responses.

A clear benefit of combining the two techniques in the VWP is that they provide complementary information on how task stimuli are processed. In studies of crosslinguistic co-activation, for example, a bilingual Spanish-English speaker listening to the auditory target word cow (/kaʊ/) may look not only at the target picture of a cow, but also to a crosslinguistic phonological competitor (e.g., helmet, in Spanish “casco”, /'kasko/). These gaze patterns reveal attentional engagement, as well as lexical access and competition through gaze behavior. The addition of EEG allows researchers to determine whether the phonological competitor was processed semantically (e.g., via an N400), inhibited (e.g., via a P300), or reanalyzed (e.g., via a P600; see illustration in Figure 2). The combined approach can also be valuable because the activation of the non-target language may not always impact gaze behavior or lead to significant differences between fixations on target and competitor items. Instead, co-registration may allow researchers to capture the brain activity elicited by both types of items, revealing neural responses to language co-activation not evident in eye movement metrics. This constitutes a clear case where neither EEG nor eye-tracking alone can determine whether co-activation engaged automatic or controlled processes.

Figure 2. Illustration of co-registration output in the Visual World Paradigm.

In this task, participants listened to a spoken word (e.g., cow) and were asked to select the corresponding image from a visual display. The lower panel shows an eye-tracking heatmap on the left with fixation distributions for the crosslinguistic phonological competition condition. On the right, the ERP figure displays fixation-related neural responses (FRPs) for control (cow – doll) versus crosslinguistic phonological competitor items (e.g., cow /kaʊ/ - helmet, “casco” /'kasko/ in Spanish). The enhanced P600 for competitor items indicates increased reanalysis or integration costs, reflecting the additional cognitive effort required to resolve crosslinguistic interference during spoken word recognition in bilinguals.

Similarly, as the field moves toward understanding how individual differences in bilingual language experience shape cognition ([141]), co-registration offers the opportunity to disentangle overlapping individual-difference effects in crosslinguistic co-activation. Specifically, bilinguals with different profiles (e.g., proficiency, age of acquisition, language use and exposure) may show similar gaze patterns while differing in their underlying neural responses (i.e., neural–gaze decoupling; see [142] for a similar pattern with behavioural outcomes), or conversely, comparable neural responses accompanied by distinct attentional strategies. When using co-registration in the VWP, there is the possibility that low-proficiency L2 speakers may show eye movements comparable to high-proficiency bilinguals when resolving crosslinguistic competition. However, the low-proficiency bilingual group may still exhibit delayed or enhanced ERP responses (e.g., larger N400 or P600 amplitudes) compared to the high-proficiency group, reflecting increased processing effort or reduced efficiency. In contrast, highly proficient bilinguals may show a tighter coupling between gaze behavior and neural indices of competition, reflecting more efficient coordination between visual strategies and neural processes.

Despite its potential, using co-registration with the VWP presents several methodological challenges. Unlike visual stimuli, an auditory stimulus unfolds over time, making it difficult to define a clear onset for time-locking ERPs. Additionally, due to the high number of ocular artifacts, spiked potentials associated with saccades could overlap with ERP components of interest and contaminate the data. To address these concerns, researchers can extract FRPs at multiple key timepoints [10], which helps identify different brain responses elicited as the word is gradually presented. For example, first fixations to a phonological or semantic competitor may reflect early stages of lexical access and semantic processing. In the time window anchored to this fixation, researchers could expect modulations in the N400, which is sensitive to semantic processing and the ease of connecting auditory input with visual information [143]. In contrast, longer fixation durations preceding a behavioral response (e.g., mouse click) may indicate later processes, such as conflict resolution or reanalysis, which are likely going to be associated with components such as the P600. By comparing EEG activity across different gaze events, co-registration can help clarify when and how bilinguals undergo shifts in neurocognitive processing.

Another key challenge that researchers face when combining EEG and eye-tracking in the VWP is the large number of eye movements it elicits. As in other visual search tasks (e.g., [144]), participants explore the visual display freely, often directing multiple fixations to different areas in the display before and after hearing the target word. Eye movements introduce muscular ocular artifacts into the EEG signal, compromising the quality of the signal and making it difficult to isolate ERPs [145]. Several strategies can help mitigate this issue. One common approach is to apply independent component analysis (ICA) to identify and remove artifacts related to eye movements [146]. This process is often guided by signals from electrooculogram (EOG) electrodes, which are placed above and below the eye (for vertical movements) and at the outer corners of the eyes (for horizontal movements). Collecting data from the EOG channels improves the accuracy of artifact removal and helps preserve the underlying neural signal. Recent developments have optimized ICA for free viewing by adjusting preprocessing parameters and using eye-tracking data to guide component rejection ([147]; OPTICAT code available). There are also useful tutorials and resources online showing how eye tracking data can guide ocular artifact detection and correction [145]. As noted previously in Section 4, another effective solution is to analyze FRPs, which allows researchers to isolate time windows of brain activity that have been previously cleaned from blinks and saccadic eye movements (i.e., during periods of gaze stability) [147]. Furthermore, researchers can simplify the visual complexity of the task. For instance, presenting two items instead of four reduces the number of eye movements needed to inspect the display, while still eliciting meaningful competition between referents. Arranging stimuli closer to the screen center can also minimize peripheral scanning.

Beyond advancing our understanding of how bilinguals manage the parallel activation of both languages, co-registration holds promise for elucidating complex cognitive processes such as prediction and updating during comprehension.

4.2. Investigating prediction and revision processes through co-registration

Research on high-level comprehension processes in bilinguals have employed reading comprehension tasks based on situation model theories [148, 149] to investigate how bilinguals manage inferential processing across their two languages. In the Situation Model Revision task [138, 150], participants build an initial mental representation of a narrative and subsequently update it when new, conflicting information arises. The Situation Model Revision task captures bilinguals’ ability to monitor and update mental representations in both their L1 and L2. By aligning eye movements with brain activity elicited during the task, we can gain a better understanding of the monitoring and updating processes involved in bilingual comprehension during inference revision. For instance, previous studies have found that when readers encounter a word that contradicts their initial inference (e.g., “violin” after having inferred “guitar”), they often show longer fixation durations and regressions to earlier parts of the text [151]. These eye-movement patterns reflect processing difficulty and reanalysis. With co-registration, researchers can examine whether such fixations and regressions reflect successful semantic processing. For example, a larger N400 time-locked to the first fixation on the unexpected word would indicate a semantic mismatch with the current mental model [152, 153]. However, if that word is subsequently fixated after rereading the sentence, and the N400 is significantly reduced, this may suggest successful integration of the updated inference. Therefore, co-registration can help distinguish between automatic inference updating and controlled processing strategies during comprehension, beyond what can be inferred from behavioral or neural measures alone.

Regarding the impact of different bilingual experiences on comprehension, co-registration can also provide insightful findings. Bilinguals with higher proficiency or earlier age of acquisition may show a better temporal correspondence between eye-movement indices of reanalysis (e.g., fewer regressions and shorter fixation durations on the critical word) and neural markers of inference updating (e.g., reduced N400 amplitudes), reflecting a more automatized revision process [154, 155]. In contrast, less proficient bilinguals may exhibit longer or more frequent regressions while still showing typical N400 or P600 responses, suggesting the engagement of compensatory mechanisms to achieve successful comprehension [138]. By capturing both convergent and divergent neural–gaze patterns, co-registration allows psycholinguistics to disentangle how prior bilingual experience modulates the engagement of automatic and controlled processes during inference revision and comprehension.Additionally, given the role of the P3b in updating processes [70, 156], examining whether and when it emerges after an unexpected word is encountered can shed light on how bilinguals update their mental representations. Correlating P3b amplitude with regression patterns and fixation durations on the updated sentence could help disentangle the cognitive cost of revising information during reading across both languages. Similarly, P600 effects associated with reanalysis may appear in sentences with longer total reading times and more regressions, indicating effortful suppression of the initial inference and the construction of new situation models [45].To implement co-registration effectively in an inference revision task, several methodological challenges need to be addressed. As with the VWP (see section 4.2), ocular artifacts and neural signals need to be aligned with precise task events. However, a more specific concern about this task is its similarity to naturalistic reading paradigms, where participants read continuous text at their own pace [157]. The self-paced reading procedure introduces variability in fixation times and regression patterns, complicating the alignment of eye-movement measures with ERP components elicited by specific task events. This issue is especially problematic when multiple regressions or fixations occur around critical regions of interest. Recent co-registration studies in psycholinguistics using natural reading tasks offer some solutions. For instance, FRPs time-locked to fixations on target words can be used to observe the brain activity evoked when the item is specifically fixated, as well as when processing begins. Moreover, strategies such as correlating regression patterns with specific ERP components allow researchers to distinguish different processing phases during reading [137, 147, 158].

While the natural flow of reading introduces complexity, the growing literature on co-registration in reading provides tools and frameworks that can be adapted to bilingual comprehension paradigms. There are also specific strengths that make inference-revision tasks suitable for co-registration. For example, texts in inference revision tasks are often presented sentence-by-sentence or word-by-word, which makes them more appropriate for EEG and eye-tracking than free-reading paradigms while still maintaining some degree of ecological validity. Researchers can take advantage of this presentation format by introducing triggers that mark sentence or word onset to improve the time-locking of brain activity and its synchronization with gaze data. In addition, synchronization can be further improved by using shared event markers between EEG and eye-tracking recordings. Thus, co-registration can enhance the interpretation of results regarding the revision of the mental model and how the revision process is modulated when reading in L2.

5. Future directions and methodological considerations for the application of co-registration in bilingualism research

Co-registration of EEG and eye-tracking holds considerable promise for advancing research on bilingual language processing and its impact on cognition more broadly. In the field of memory, previous studies have examined the role of language as a contextual cue that can either hinder or facilitate the encoding and retrieval of information in bilinguals (e.g., [159]). Co-registration could clarify whether the effects of language on memory result from shifts in attention allocation during the task (captured via eye-tracking measures) or from differential engagement of semantic and updating processes (e.g., reflected in N400, P3b, P600). Additionally, in the domain of moral decision-making, research has shown that bilinguals often make different moral judgments in their native language (L1) versus their foreign language (L2) (e.g., [160, 161]), frequently favoring more utilitarian responses in their L2. Co-registration could clarify whether these patterns of utilitarian responses reflect changes in emotional engagement (e.g., indexed by pupil size) or in controlled evaluation (e.g., P600). Similarly, in emotion perception tasks, bilinguals sometimes show reduced sensitivity to emotional cues in L2 (e.g., [162]). Co-registration could help disentangle whether these differences in emotionality arise from reduced visual attention to emotional features or from attenuated emotional engagement when working in L2, as reflected in ERP responses like the N300 (e.g., [163]). In sum, numerous areas of bilingual language processing research may benefit from combining brain activity and eye-movement recordings.

In this paper, we have highlighted specific topics in bilingualism research that can benefit from the implementation of co-registration. More broadly, however, the potential of this method for the field lies in its ability to address wider questions, such as how underlying cognitive mechanisms are reflected in observable behavior during language processing. The ability to map attentional behavior onto underlying neural processes represents a qualitative advance over single-method approaches. Co-registration enables researchers to ask which mechanisms support successful performance, when they are engaged, and how they vary across conditions. As such, co-registration can contribute to integrating theoretical models of bilingual language processing with empirical measures of both brain and gaze behavior.

Our approach to guiding the implementation of EEG and eye-tracking co-registration in bilingualism research is to build on existing knowledge from previous co-registration studies with monolingual populations (see Section 4) and extend it to future studies exploring bilingualism and cognition. Accordingly, we consider that studies exploring ERPs are the natural first step, as they allow clearer predictions and more robust interpretations. Alternatively, or as a complementary approach, neural oscillations can also be analyzed to understand the brain dynamics of bilingual language processing and how they interact with gaze behavior. Similar to eye-tracking, neural oscillations provide a continuous tracking of neural activity. This correspondence makes their joint analyses highly informative to understand the brain networks engaged during bilingual language processing and how these networks modulate visual attention. Based on previous literature, we suggest that oscillatory activity in the theta and beta bands would be particularly informative for examining crosslinguistic co-activation [80, 81], whereas delta and alpha-beta bands may be especially relevant for sentence-level inferences and comprehension [84, 86, 87].

Despite the promise of co-registration, its application with bilingual populations involves several practical and interpretive challenges. Addressing these challenges is essential to fully realize the potential of co-registration and advance the field of psycholinguistics. Eye-tracking and EEG provide distinct types of information. While eye movements offer indices of attentional shifts, EEG captures neural activity that may precede, parallel, or follow those attentional shifts. As a result, it is possible to observe mismatches between gaze behavior and ERP components. For instance, a fixation pattern suggesting re-evaluation may not be accompanied by P3b or N400 effects. Researchers must therefore interpret combined data cautiously, grounding their analyses in well-specified models of bilingualism to avoid misalignment or misinterpretation of combined measures. Beyond these interpretive considerations, the practical implementation of co-registration also introduces several technical and analytical challenges.

EEG and eye-tracking systems differ in their sampling rates, temporal resolution, and event-logging methods, requiring accurate cross-system alignment and calibration. Additionally, pre-processing each modality involves separate workflows: EEG demands procedures such as ICA, filtering, and artifact correction, while eye-tracking requires fixation parsing, saccade identification, downsampling, processing, and segmentation by area of interest. Therefore, co-registration requires dual expertise that may not yet be common among psycholinguistic researchers. To address this, the field would benefit from the development and dissemination of open-source pipelines and documentation, as well as shared standards for data formatting and analysis. Tools such as the EYE-EEG toolbox [10] provide a starting point, yet community-driven validation and adaptation for bilingualism research remain limited.

While most studies in psycholinguistics using EEG and eye-tracking co-registration have been conducted with adult participants, co-registration is well-suited for examining developmental changes in language processing. Co-registration could also help determine how early co-activation emerges and which linguistic factors modulate its appearance (e.g., age of second language acquisition, type of bilingual acquisition). Similarly, research on bilingual children’s comprehension [164] requires the adoption of online techniques to obtain more direct measures of the underlying cognitive mechanisms involved in bilingual language processing. Despite co-registration’s potential, there are methodological challenges that arise when using this approach with infants and children. A major challenge is that infants and children are constantly moving, which introduces higher levels of noise and artifacts into the EEG recordings and makes stable calibration for eye-tracking difficult. Second, ICA algorithms are typically developed for adults, reducing their accuracy in detecting signal artifacts in infants and children [165]. Third, infants’ eyes tend to be more reflective, further hindering the eye-tracker’s ability to track eye movement patterns [166]. Beyond these technical issues, there are also practical constraints that must be considered. For example, EEG and eye-tracking setup requires a considerable amount of time, and with infants, this time should be minimized to avoid shortening the experimental task. Despite these challenges, advances in child- and infant-friendly methodologies are increasingly enabling reliable measurement of neural and visual processes across early development (see [167] a comprehensive review).

Advancing co-registration in bilingual language processing also depends on the development of shared interpretative theoretical frameworks. The field lacks consensus on how to map cognitive constructs (e.g., interference, inhibitory control, updating) onto combined neural and eye-movement measures. For example, should successful control of crosslinguistic co-activation be identified through reduced N400 amplitudes, increased P600 responses, or a specific sequence of fixations and regressions? Each combination may reflect different cognitive mechanisms engaged, and these mechanisms may shift across languages or with individual differences (e.g., age of L2 acquisition, proficiency, etc.). Moving forward, research in bilingualism must aim to link theoretical constructs with combined neural and gaze-behavioral patterns, creating a framework that supports integration across studies.

A first approach to advance this goal could be to implement co-registration in empirical studies in order to test theoretical models of bilingual language processing. For example, in the BLINCS model, interactivity between levels of representation is mediated by bidirectional excitatory connections shaped by Hebbian learning [168]. According to this principle, units that are repeatedly co-activated strengthen their connections over time. Following this view, in BLINCS, representations that are closely mapped tend to be simultaneously activated, strengthening their connections. The model predicts that repeated crosslinguistic co-activation, especially among phonologically overlapping items, can reorganize the bilingual language system through experience-dependent plasticity [19]. Empirical support for this prediction comes from studies using electrophysiological measures. Specifically, Ning and colleagues [39] reported that bilinguals rated semantically unrelated word pairs as more related than monolinguals and showed a reduced N400 difference between related and unrelated words, suggesting that bilinguals may possess a more interconnected semantic network than monolinguals. Although these findings are consistent with BLINCS’ prediction, they do not directly reveal what role crosslinguistic activation plays in the restructuring of bilingual semantic networks. Co-registration has the potential to address this gap by linking neural indices of semantic activation with concurrent eye-tracking indices of lexical access and cognitive effort. Examining the temporal alignment between neural responses (e.g., N400 amplitudes) and eye-movement patterns during bilingual language processing can therefore provide direct insight into how crosslinguistic co-activation shapes the organization of bilingual semantic networks. Together, these developments pave the way for co-registration to become a valuable cutting-edge tool for bilingualism research in both a theoretical and methodological way. The present paper advances current efforts in psycholinguistics to refine and expand methodological tools for studying bilingual language processing. By highlighting the potential of co-registering brain activity and eye-movement data, we emphasize how this approach can advance our understanding of how managing two languages shapes cognitive functioning. As the field of bilingualism research progresses, co-registration provides a powerful means for investigating the dynamic interplay between language and cognition, offering a lens through which the complexity of the human mind can be more fully captured and better understood.