Cortical cognitive processing during reading captured using functional-near infrared spectroscopy

Abstract

Neuroimaging studies using functional magnetic resonance imaging (fMRI) have provided unparalleled insights into the fundamental neural mechanisms underlying human cognitive processing, such as high-level linguistic processes during reading. Here, we build upon this prior work to capture sentence reading comprehension outside the MRI scanner using functional near infra-red spectroscopy (fNIRS) in a large sample of participants (n = 82). We observed increased task-related hemodynamic responses in prefrontal and temporal cortical regions during sentence-level reading relative to the control condition (a list of non-words), replicating prior fMRI work on cortical recruitment associated with high-level linguistic processing during reading comprehension. These results lay the groundwork towards developing adaptive systems to support novice readers and language learners by targeting the underlying cognitive processes. This work also contributes to bridging the gap between laboratory findings and more real-world applications in the realm of cognitive neuroscience.

Introduction

Over the last three decades, functional magnetic resonance imaging (fMRI) has offered unprecedented insights into the fundamental neural mechanisms underlying human cognition, including complex high-level cognitive processes. Reading provides a unique context to investigate brain activity during complex processing because it involves a carefully choreographed, dynamic interaction between the reader, the text, and context¹. Specifically, reading comprehension entails translating a series of abstract symbols (i.e., letters) into associated units (words), interpreting the strings of words (i.e., sentences) based on semantic context and syntax^4–6, establishing connections within the text (e.g., connecting a pronoun to a previously seen noun⁷), and incorporating prior knowledge, inferences, and predictions^2,3, such as linking textual representations to information stored in long-term-memory (e.g., retrieving a memory associated with an event discussed in the text⁸).

Even though reading relies on a diverse set of computations implemented across distributed brain systems, including those involved in domain-general mental processes (i.e., cognitive control, perception, memory), a distinct set of brain regions (the ‘language network’), selective for core high-level linguistic processes compared to more domain-general processes can be isolated using fMRI^9–24, as shown using the Language Localizer Task¹⁷. This task was designed to robustly define (‘localize’) language-sensitive functional brain regions pertaining to different aspects of reading (e.g., identifying words, parsing sentences) distinct from those supporting non-linguistic processes. The boundaries of the language network have been further defined using group-constrained participant-specific analyses^17,26 which consider individual (reader-level) variation in neural recruitment that might deviate from the group-level network. This variation is important to consider because of the highly distributed nature of activations covering adjacent brain regions and because reading is a highly individualized process¹ and thus readers might engage different processes and consequently different brain regions.

Subsequent studies have found the language network to be activated across the majority of participants engaged in high-level linguistic processing, to be replicable within participants over time, to be robust to variations in language stimuli and presentation (e.g., visual, audio), and to be consistent across many of the world’s spoken languages^{9,10,16,17,25–28}. Together, this body of research builds the necessary foundation for investigating reading higher-level processes in the brain – at the group level—as well as considering individual variability in reading comprehension. However, this work has been conducted in the fMRI scanner, which is not suited for measurement of brain function in more naturalistic settings²⁹ and has limited utility as a real-time measurement system that could be integrated with adaptive technology to facilitate reading in real-world scenarios.

Conversely, functional near-infrared spectroscopy (fNIRS) has emerged as a non-invasive brain measurement technique that holds great potential for measurement of higher-level cognitive processes in the human cortex in naturalistic settings^30–33. FNIRS is a popular tool for investigating language development and processing in pediatric (reviewed in^34,35) and other populations where fMRI is difficult or impossible to conduct (e.g., in people with cochlear implants). In the realm of reading, fNIRS has been used to capture cortical processes related to specific aspects of linguistic processing, such as language localization in Broca’s and Wernicke’s areas^36–38, prefrontal language lateralization^39–42, associative memory judgements⁴³, word frequency and predictability^44,45, processing of semantic and syntactic anomalies⁴⁶, and speech articulation effects on reading⁴⁷, often in small studies (n ≤ 10) and/or with localized recordings from one or very few cortical areas of interest. In addition, fNIRS has been used to capture cortical processing related to reading single words or word pairs^48–53, which largely captures ‘lower-level’ processes such as word identification but does not assess higher-level processes crucial to reading comprehension such as sentence comprehension, inferencing, and integration⁵⁴. Thus, fNIRS has yet to be validated as a reliable measure of the underlying higher-level linguistic neural processes implicated in reading comprehension. As a step in this direction, in this study we used the Language Localizer Task together with multi-channel fNIRS to record widespread bilateral cortical brain activity related to sentence-level reading comprehension. Based on prior fMRI work, we formed two main hypotheses to be tested in the current study. First, we expected to observe increased hemodynamic response associated with sentence reading comprehension relative to the control condition in areas corresponding to the functional language network. Second, we expected to observe high between-subject reliability in cortical activations within the language network. We show in a large sample of 82 healthy young adults that we can robustly capture cortical higher-level reading-related processes, replicating prior work with fMRI, and extending prior fNIRS work. In addition, we show relatively high between-subject reliability in captured cortical activations, particularly in prefrontal aspects of the language network. Our study therefore validates fNIRS as a viable approach to measure complex brain processes implicated in reading comprehension outside the scanner, opening the door for more basic research and real-time personalized interventions to enhance reading outcomes (e.g.,⁵⁸).

Methods

Design

We administered the Language Localizer Task to a group of individuals equipped with fNIRS sensors to record cortical brain activity during high-level linguistic processes involved in sentence-level reading in a more naturalistic setting.

Participants

Ninety-five young adults participated in this study. The institutional review board of the University of Colorado Boulder approved the study, and all participants provided written consent. Twelve participants were excluded from data analysis due to technical issues with fNIRS data collection (unrecoverable missing trigger values), for a final analyzed sample of 82 participants (age mean (SD) 22.1 years (4.6); 28 male, 52 female, 2 other; 63 White, 9 Asian, 6 Hispanic, 4 Other). All methods were carried out in accordance with relevant guidelines and regulations. The institutional review board (IRB) of the University of Colorado Boulder approved the study. Written informed consent was obtained from all participants.

Language Localizer Task

We administered the Language Localizer Task, developed by Fedorenko et al.¹⁷ to localize language processing in the brain using visual text stimuli. The task comprised four conditions manipulating word-level and sentence-level processing (Fig. 1a), presented in a block design of 1 block per condition, 20 text stimuli per block separated by 10 s, in randomized order. The stimulus presentation timing and duration was the same for all participants, with each text stimulus consisting of 12 words (or non-words, see below) successively presented at fixation for 156 ms, with 2 s between sentences. Participants were asked to read silently and presented with the following instruction: "In this task you will be asked to read a sequence of sentences. Each sentence will be presented in the center of the screen one word at a time. Between each sentence you will view a fixation point (displayed as a ' + ') in the center of the screen. The sentences will continue to advance without the need for any input. After all the sentences have been displayed you will be given a brief rest period before the next task begins."

Figure 1.

Study design and fNIRS montage. (a) Experimental design for the Language Localizer Task. Each participant was presented with 4 types of conditions (sentence, word list, jabberwocky, non-word lists), presented in a random order. Each condition consisted of 20 text stimuli, with examples shown in the figure. (b) The 42-channel fNIRS montage. Each channel consisted of a source-detector pair, represented by a colored line. Green circles represent EEG channels, not analyzed in this study.

The four conditions included Sentence, Word list (generated by shuffling words in sentences), Jabberwocky sentences (sentences where all the content words were replaced by pronounceable nonwords, like, “sulrist”) and Non-word list (shuffled jabberwocky sentences). The Sentence condition involves accessing the meaning of individual words (lexical semantic processing), relating the words to one another (syntactic processing), and then deriving the meaning of the entire sentence (sentence-level semantic processing). The Word list condition engages semantic lexical processing, but not syntactic processing, because individual lexical items (words) cannot be combined into more complex representations to form a sentence. The Jabberwocky condition engages syntactic processing as it follows the rules of English syntax to form a sentence, but not semantic lexical processing, as it includes nonsense words. The Non-word list condition contains a nonsense string of pronounceable non-words and therefore does not engage syntactic nor sentence-level semantic processing.

To identify brain regions involved in sentence comprehension, i.e., those that are sensitive to word- and sentence-level meaning and sentence structure, our contrast of interest (and the main contrast investigated in the original language localizer study¹⁷) was [Sentence – Non-word list]. This contrast captures higher-level linguistic processes, such as those that underlie reading of texts, because it involves retrieving the meanings of individual words and combining these lexical meanings into more complex structural representations (sentences and texts). In addition, this contrast does not target phonological or morphological processing, because the relevant properties are matched across conditions.

fNIRS data acquisition

We collected fNIRS data using the channel montage in Fig. 1(b) as part of a larger study (not analyzed here). Data collection was conducted using the NirSport 16 × 16 system (NIRx, Berlin, Germany) with a sampling rate of 10.2 Hz.

The fNIRS montage was designed to target brain areas involved in reading comprehension and language processing. The channel montage was guided by the functional language network, defined initially using the Language Localizer Task^10,17 and refined in other studies^{11,12,15,16,22,23,28,59}, and more generally by meta-analytic functional brain maps associated with the terms ‘language’, ‘reading’ and ‘reading comprehension’ derived from Neurosynth (neurosynth.org), as well as regions associated with attention, integration and executive functioning. Specifically, the montage of 42 channels spanned prefrontal, parietal and temporal cortices, including bilaterally the inferior frontal gyrus (IFG), and orbital IFG (IFGorb), middle frontal gyrus (MFG), superior frontal gyrus (SFG), angular gyrus (AngG), and anterior and posterior temporal gyri (AntTemp, PostTemp).

fNIRS data processing

All fNIRS preprocessing was conducted in the NIRS Brain AnalyzIR Toolbox (version 837)⁶⁰ in MATLAB (version 2021a). Briefly, the raw voltage data were resampled to 5.1 Hz, converted to optical density, and then converted to changes in oxygenated (HbO) and deoxygenated (HbR) hemoglobin using the modified Beer-Lambert Law⁶¹, followed by a bandpass filter (0.01- 0.5 Hz) to remove systemic artifacts including heart rate, respiration, and blood pressure. HbO and HbR timeseries data were used to calculate subject-level (first-level) statistics using a General Linear Model (GLM) with an autoregressive pre-whitening filter to account for serially correlated errors and downweigh outliers due to motion artifacts⁶². Specifically, each source-detector pair (channel) was analyzed with an individual GLM. The independent variable was created via the canonical hemodynamic response function (HRF or “double gamma function”) of the stimuli, which models a hemodynamic response with a default undershoot period of 16 s and a default peak period of 4 s. Given this independent variable, as well as the coefficient of that stimulus for that channel (β) and an error term (ε), the resulting dependent variable (Y) was the measured ∆HbO or ∆HbR time series data. The first-level GLM thus estimated the [sentences – nonwords] contrast per participant averaged across 20 text stimuli.

Group-level analysis

The resulting first-level statistic contains the subject-level regression coefficients (beta values for the contrast) as well as their corresponding error-covariance matrices. Using the output from the first-level analysis, a second-level (group-level) model was calculated, using the full covariance from the first-level models, to perform a weighted least-squares regression and generate group level statistics on the [sentences – nonwords]. To control for multiple comparisons, false discovery rate (FDR) correction was used with the significance level set at 0.05 (q < 0.05)⁶³. Analysis results are t-values resulting from the contrasts.

Individual variability and correspondence with fMRI-derived functional language localizer

Because individual differences can attenuate the effects of localizers in group-level statistics¹⁷, we next computed for each fNIRS channel the number of participants with a significant (q < 0.05) activation in this channel. We defined the top 10% channels as regions with high between-subject reliability. The bottom 10% were defined as regions showing low between-subject reliability. This served to test whether the [sentence-nonword] contrast reliably engages the same regions across participants.

To assess the degree of spatial overlap with the functional language network we mapped the top 10% and bottom 10% channels onto the current functional language localizer map downloaded from evlab.mit.edu//funcloc/ (and encompassing bilaterally the IFG, IFGorb, MFG, AntTemp, MidTemp and PostTemp, but not the SFG included in the original localizer¹⁷). The MNI coordinates corresponding to the fNIRS channels were overlaid onto the localizer map on top of a standard anatomical MRI template in MNI space. We expected the fNIRS channels with high between-subject reliability to have high anatomical overlap with the localizer. As a control, we expected the bottom 10% (low inter-subject reliability) channels to show weaker (or no) overlap with the localizer.

Results

Group results

Group results are depicted in Fig. 2. HbO activations have positive t-values (red in Fig. 2a) while HbR activations have negative t-values (blue in Fig. 2b). Table 1 lists these group results, with only significant activations of HbO and HbR included.

Figure 2.

fNIRS group results for the [sentence-nonwords] contrast. Both directions of the contrast (sentences – nonwords) and its inverse (nonwords – sentences) are overlaid over a brain, with nasion (Nz) and inion (Iz) locations added for reference. Only significant channels (q < 0.05) are shown. (a) For HbO, positive t-values (red) correspond to relatively larger activity for the first term in the contrast, and negative t-values (blue) correspond to larger activity for the second term. (b) For HbR contrasts, negative t-values (blue) correspond to larger activation in that region.

Table 1.

fNIRS group results for the [sentence-nonwords] contrast.

s	d	brain region	FLL	L / R	beta (se)	t	p	q
Significant HbO activations
S2	D4	BA 45 – IFG, Broca	IFGorb	L	4.61 (1.12)	4.11	< 0.001	< 0.001
S9	D1	BA 09 – MFG BA 10 – frontal pole	SFG*	mid	7.47 (2.88)	2.59	0.010	0.028
S9	D3	BA 09 – MFG BA 08 – SFG	SFG*	L	5.92 (2.51)	2.36	0.018	0.048
S9	D10	BA 09 – MFG BA 08 – SFG	SFG*	R	7.44 (2.56)	2.90	0.004	0.013
S10	D11	BA 45 – IFG, Broca	IFGorb	R	4.46 (1.35)	3.29	< 0.001	0.004
S13	D12	BA 38 – AntTemp	AntTemp	R	5.32 (1.91)	2.79	0.005	0.016
Significant HbR activations
S4	D4	BA 45 – IFG, Broca BA 46 – MFG	IFG	L	− 2.46 (1.02)	− 2.40	0.016	0.043
S5	D5	BA 38 – AntTemp	AntTemp	L	− 4.86 (1.09)	− 4.47	< 0.001	< 0.001
S12	D11	BA 45 – IFG, Broca BA 46 – MFG	IFG	R	− 0.85 (0.27)	− 3.14	0.002	0.007

Only significant activations are included (significant HbO positive t-value and significant HbR negative t-values). All regions mapped onto the original group-level functional language localizer (FLL; [sentences > nonword lists] ¹⁷).

*SFG was included in the original language localizer but is not included in the current version of the language localizer.

Dfe = 5120.29. s, source; d = detector; BA, Brodman area; L/R, left/right hemisphere; se, standard error; t, t-value; p, p-value; q, FDR-corrected p-value; I/M/SFG, inferior/middle/superior frontal gyrus.

(a) ITG, inferior temporal gryus. AntTemp = anterior temporal gyrus (temporal pole).

These group level results align well with the group-level functional language network¹⁷. Specifically, all 9 significant group-level results mapped onto Fedorenko’s language localizer parcellation, with four significant fNIRS activations (i.e., channels) mapping onto the IFG bilaterally, three onto MFG bilaterally and two onto the anterior temporal gyrus (AntTemp) bilaterally. Fedorenko’s group level contrast [sentences > nonwords list] resulted in activations in the left IFG, left IFGorb, left MFG, left SFG, bilateral AntTemp, bilateral Mid- and PostTemp, left AngG and to a lesser extent in the cerebellum¹⁷.

Individual results and correspondence with fMRI-derived functional language localizer

As individual differences can attenuate the effects of localizers in group level statistics^17,27, we next generated statistical results per participant on the contrast of interest ([sentences – nonwords]). This resulted in 681 total channels across 77 participants showing significant (q < 0.05) HbO or HbR activation (either positive HbO t-values or negative HbR t-values). On average this resulted in just over 8 significant channels per participant (mean (SD), 8.8 (6.9), range: 1 – 39 channels).

Channels with the highest between-subject reliability (defined as those showing significant increases in HbO or HbR response in most participants (the first four bars in the chart in Fig. 3(a)) all mapped onto inferior frontal gyri (including the left-lateralized Broca’s area; Fig. 3(a), red spheres 1–4; right BA38/45, left BA45, right BA48/6, left BA38/45). Across the four channels showing the highest between-subject reliability, 77% of participants had significant activation in at least one of the four channels (Fig. 3b). These four channels showed high anatomical overlap with bilateral inferior frontal areas in the functional language localizer map. Of note, the next four highest-reliability channels (top 20% percent; bars 5–8 in Fig. 3a) also mapped onto IFG, IFGorb, and additionally onto bilateral AntTemp. The control channels with low inter-subject reliability (defined as the bottom 10%) showed less anatomical overlap with the functional language localizer map or showed overlap with brain regions less reliably activated in language studies. For example, the SFG (blue sphere 38 in Fig. 3a) is an area that was included in the initia¹⁷, but not in subsequent versions of the localizer (e.g.^16,28) and that has shown lower group-level activation in the original localizer experiments¹⁷, as well as low reliability in replication and extension studies with new participants and new language comprehension paradigms^{9,10,16,17,25,26}. Of note, the AngG (blue sphere 37 in Fig. 3a) also typically shows less activation than frontal and temporal areas in studies of language processing^10,11,17,28.

Figure 3.

Between-subject reliability in significant fNIRS activations. (a) fNIRS channels—labelled with corresponding Brodmann anatomical areas—are shown in the chart with the number of participants with significant activation (i.e. increased HbO or HbR response) in that channel. The fMRI-based functional language localizer map (downloaded from evlab.mit.edu//funcloc/) is shown in yellow on the cortical surface of a standard MRI template in MNI space. Channels with high (top 10%) and low (bottom 10%) between-subject reliability are highlighted in red and blue respectively in the chart and on the brain map. Regions (black arrows) are named using the convention used in¹⁷. (b) Significant channels per participant ID are displayed for the high between-subject reliability (i.e.,top 10%) channels (HbR channels in green, HbO channels in black). MFG, middle frontal gyrus; IFG, inferior frontal gyrus; IFGorb, orbital IFG; Ant / Post Temp, anterior / posterior temporal gyrus; AngG, angular gyrus.

To test which fNIRS signal type contributes more to the high between-subject reliability in activations, we also investigated the HbO and HbR signals separately. Although the channels with the highest between-subject reliability channels mapped to the same or similar anatomical location as the channels derived when considering the activation across signal types (HbR or HbO in Fig. 3(a): right BA38/45, left BA45, right BA48/6, left BA38/45; HbO only: right BA48/6, right BA38/45, left BA38/47; left BA38/45; HbR only: right BA38/45, left BA45, left BA38/45, left BA38/20), the HbR signal contributed more to the indiviual-level results (green dots in Fig. 3b). The HbR signal therefore seems to be the more reliable signal component across participants, consistent with prior fNIRS work showing the HbR signal to more reliably reflect neuronal activation than the HbO signal^33,64,65.

Discussion

We administered an established language localizer task¹⁷ with individuals equipped with fNIRS sensors to extend prior fMRI work on higher-level linguistic processing in the human brain. Our results, obtained outside the fMRI scanner, demonstrate the convergent validity of fNIRS in identifying cortical brain regions activated during linguistic processes involved in reading and underscore the potential of fNIRS for real-world measurement of reading processes. This is the largest study to date to use fNIRS to capture reading-related cortical activation, and the only one using comprehensive brain coverage, therefore providing a robust replication of prior fMRI work on reading-related language processing. Group-level and participant-level analyses validate the utility of fNIRS for capturing reading processes with the ultimate goal of tracking and enhancing reading comprehension in an individualized manner.

Our group-level results aligned with the functional brain network defined using the language localizer task¹⁷ by showing activations in the inferior and mid prefrontal cortices and the anterior temporal lobe. At the participant-level, significant fNIRS activations common to most participants (i.e., having high between-subject reliability) mapped bilaterally onto the inferior prefrontal cortices and—to lesser extent—onto the bilateral anterior temporal cortices. These results are thus in line with contemporary fMRI studies of language processing which show most reliable and robust activations in inferior prefrontal and temporal regions (especially in the left hemisphere)^{10,12,13,16,17,22,23,25,59,66,67}. The fNIRS activations showing the most variability across participants (i.e., low inter-subject reliability) also showed less overlap with the language network, by either having less anatomical overlap with the established functional language localizer map or showing anatomical overlap with superior frontal and posterior temporal brain areas less reliably activated in language studies^{9,10,16,17,25,26}.

The significance of this work lies in showcasing our ability to measure neural correlates of higher-level language processing related to reading outside the fMRI scanner. This has implications for the domains of education and human–computer interaction, and applications such as usability testing of reading comprehension assessments, design of adaptive learning systems, and medical diagnostics related to reading. By leveraging foundational fMRI studies, our research takes a crucial step towards developing adaptive systems capable of assisting and enhancing reading outcomes, thereby bridging the gap between laboratory findings and real-world applications in cognitive neuroscience. Moroever, the finding that the majority of participants (771%) showed activation in the bilateral inferior frontal cortex, suggests that wearable, low-cost versions of fNIRS – with a few channels concentrated onto the prefrontal cortex—might be sufficient to capture brain correlates of complex linguistic processing during reading comprehension in ecological settings. This finding also provides support for ongoing efforts to develop cost-effective ultra-wearable and ultra-unobtrusive fNIRS devices to support human functioning across domains. Additionally, both fNIRS signal types (increase in the HbO as well as decrease in the HbR hemodynamic response) contributed to these results, though the HbR component showed less variability across participants. This aligns with previous research that have found HbR to be a more reliable signal, being it is less susceptible to drift and systemic artifacts, especially when studying complex tasks^33,64,65, and could be relevant for example when selecting which signals to use in potential real-time applications (i.e., as features for real-time machine learning models).

This work has several limitations. First, while we followed standard procedures for removing motion and physiological artifacts from the fNIRS signal, we did not additionally regress out short-channel signal –which might have improved brain activity estimates by removing systemic noise from the signal of interest⁶⁸. Second, the population used in this study (a homogenous sample of young, predominantly white, adults—mostly undergraduate students) is a limitation towards the generalizability of our findings.

Future work can build on these findings to use fNIRS to measure complex reading-related and other cognitive processes in real-time, including mind wandering⁶⁹, inferencing⁸, and error correction⁷⁰. This can be followed by using more naturalistic reading paradigms such as reading of long connected texts⁷¹ with the goals of learning and retention, rather than individual sentences that the bulk of fMRI work on language processing is based on. There is also a rich literature base on eye movements and reading (e.g.,^71,72), which can be complemented with real-time fNIRS measurement to help disambiguate particular eye movements (e.g.,²¹). A longer-term goal is to use multimodal measurement of eye movements and fNIRS to select real-time intelligent adaptations to support individuals in personalized ways during reading. By demonstrating for the first time that higher-order cortical neural processes implicated in reading can be reliably measured outside the MRI scanner, we open the door to new research opportunities and application areas.

Author contributions

Conceptualization: S.D.,L.H., data collection: E.D., R.S., analysis and interpretation. M.C., L.H., E.D., R.S., writing—original draft, M.C., L.H..; writing—review & editing, M.C., L.H., E.D., R.S., S.D.

Data availability

Source data and code needed to reproduce the results presented in Figs. 2, 3 and Table 1 will be available at https://github.com/emotive-computing/EML-localizer upon publication.

Competing interests

The authors declare no competing interests.