Measuring cognitive load in multitasking using mobile fNIRS

Abstract

Cognitive load, or the mental effort required to process and retain information, is a critical factor in high-stakes environments where task demands often exceed working memory capacity, leading to performance declines and errors. However, most cognitive load research has relied on controlled, single-task paradigms, limiting its applicability to real-world multitasking situations. Addressing this gap, we used a mobile, two-channel functional near-infrared spectroscopy (fNIRS) device to measure cognitive load in a complex multitasking environment, simulating real-world cognitive demands. Thirty-one undergraduate participants engaged in single-task and multitask conditions to simulate real-world cognitive demands. Results showed that subjective cognitive load ratings were higher, performance scores were lower, and error rates increased in the multitask condition compared to the single-task condition. However, contrary to expectations, prefrontal cortex activation did not increase in the multitask condition, suggesting a "cognitive disengagement" effect, where the brain limits engagement to manage overload. This finding challenges the traditional one-to-one association between cognitive load and prefrontal activation, as seen in simpler validation studies. Our study highlights the value of mobile fNIRS for assessing cognitive load in ecologically valid settings and provides insights that could inform strategies for optimizing performance in high-stakes environments like aviation and healthcare.

1. Introduction

Cognitive load refers to the mental effort required to process and retain information during task performance (Sweller, 1988). This concept centers on the limitations of working memory, a function largely associated with the prefrontal cortex (Miller and Cohen, 2001). Working memory serves as the brain's short-term system for holding and managing information needed for complex tasks, such as learning, problem-solving, and reasoning (Baddeley, 2003). Working memory has a limited capacity, typically handling only five to seven items at a time (Miller, 1956), though individual capacity can vary based on factors like experience, training, and innate cognitive abilities (Alloway and Alloway, 2010). When task demands exceed this capacity, cognitive overload occurs, impairing an individual's ability to process information effectively and increasing the likelihood of errors (Sweller et al., 2011). This issue is especially critical in high-stakes environments, where cognitive overload can have severe consequences. For example, a pilot managing multiple controls and indicators during a turbulent storm must coordinate several tasks under intense pressure. In such scenarios, the heightened cognitive load increases the risk of errors, which can compromise safety by leading to navigation mistakes, misinterpretation of instrument readings, or delayed reactions to critical events. Therefore, it is essential to develop reliable methods for monitoring cognitive load in real-world scenarios.

Traditionally, cognitive load has been measured using subjective methods, such as self-reported mental effort ratings (Paas, 1992; Paas and van Merriënboer, 1994). In recent years, objective methods have been developed to capture cognitive load more accurately, including heart rate variability (Thayer et al., 2009), eye-tracking (Marshall, 2007), and neural imaging techniques like EEG, fNIRS, and fMRI (Antonenko et al., 2010; Ayaz et al., 2012; Miri & Mohammad Reza Daliri, 2023). Among these techniques, functional near-infrared spectroscopy (fNIRS) offers a unique advantage due to its non-invasive monitoring of brain activity through changes in cerebral oxygenation, making it both practical and adaptable for diverse settings (Pinti et al., 2020). This technique operates by emitting near-infrared light into the scalp, which penetrates the skull and detects cerebral blood flow changes associated with neural activity (Jobsis, 1977). Studies consistently link higher cognitive load with increased prefrontal cortex activation (Boere et al., 2024; Causse et al., 2017; Fishburn et al., 2014; Hirshfield et al., 2023; Saikia et al., 2021; Xu et al., 2024). Further, the portability and ease of use of mobile fNIRS devices has enabled researchers to assess cognitive load in ecologically valid environments, such as classrooms, workplaces, and high-stress professional settings, where real-time data are particularly valuable.

Despite the growing body of literature on cognitive load using fNIRS, most studies have been conducted in controlled laboratory settings, often focusing on single-task or simple dual-task scenarios (Ayaz et al., 2012; Boere et al., 2024, Hirshfield et al., 2023; Saikia et al., 2021; Xu et al., 2024). While these findings provide valuable insights into how cognitive load operates under idealized conditions, they may not fully capture the complex and dynamic demands of real-world multitasking environments, where individuals frequently need to shift attention between tasks or manage competing demands. For instance, in high-stakes fields such as air traffic control or healthcare, professionals are required to monitor multiple sources of information, make quick decisions, and switch between tasks in real time. The limitations of current cognitive load research become particularly evident in such settings, where the effects of multitasking and task-switching on cognitive resources are not yet fully understood. These constraints underscore a significant gap in the literature: a need for practical, real-time methods to measure cognitive load in dynamic, multitasking contexts.

To address this gap, the current study uses a mobile fNIRS device to measure cognitive load across single-task and multitask conditions. The study employs a dynamic task paradigm to simulate natural working memory and attention demands, enhancing ecological validity. We hypothesize that participants' subjective cognitive load ratings will be higher in multitasking conditions, consistent with prior research showing that cognitive load increases with task complexity (Paas et al., 1992; Sweller et al., 2011). We also expect fNIRS measurements to reveal elevated prefrontal cortex activation during multitasking, aligning with findings that link prefrontal activity to cognitive load levels (Boere et al., 2024; Fishburn et al., 2014; Causse et al., 2017). Additionally, we hypothesize that participants’ task performance will be lower, and error rates will be higher, in multitasking conditions compared to single-task conditions. This prediction is based on established evidence that cognitive overload impairs performance accuracy and increases errors, particularly when individuals are required to manage multiple concurrent demands (Paas et al., 2003; Sweller et al., 2011). By deepening our understanding of cognitive demands in multitasking scenarios, this research offers practical insights for fields like aviation, where monitoring and managing cognitive load is essential for safety.

2. Methods

2.1. Participants

Thirty-one undergraduate students from the University of Victoria participated in this study (M_age = 22 [99% CI: 20, 23]; 20 female; 8 male; 2 non-binary; 4 left-handed). We computed a sample size power analysis assuming an effect size of .5 (Boere et al., 2024), a significance level of .05, and a power of .95, revealing a prospective sample size of 26 participants. To avoid conducting underpowered research, our lab follows a protocol where we continue to collect data until we have 30 clean data participants, corresponding to a power of .99. As such, we collected thirty-one participants, and one was removed due to bad data quality. All participants had a normal or corrected-to-normal vision and volunteered for extra course credit in a psychology course. Participants all provided written and informed consent approved by the Human Research Ethics Board at the University of Victoria.

2.2. Experimental design

Participants were seated in a sound-dampened room, viewed stimuli on multiple 12′ iPad Pros, and responded using the touch screen of these devices. The researcher explained the subjective cognitive load rating scale (Paas et al., 1992), which participants used to rate their self-perceived cognitive load from 1 to 9 on a Likert scale, ranging from "very, very low" to "very, very high".

The experimental task was the classic Tetris® game (PlayStudios Inc, 2023), which allows players to rotate falling blocks to strategically clear levels by filling them. Clearing lines accumulate points. As time progresses, the speed of the falling blocks increases, simultaneously making the game more challenging. All games started on Level 1. Participants played a 1-min practice round to ensure the game was familiar. Participants completed four 5-min blocks of Tetris in total. Condition One had participants play Tetris on one iPad. Condition Two introduced multiple tasks by having participants play three Tetris games simultaneously on three iPads horizontally aligned side-by-side. During game-play for either condition, the score was recorded if the participant lost the game, and a new iPad was immediately swapped in, starting on Level 1. Cumulative game scores per block were combined for the final block score. The order of conditions was randomized to control fatigue effects and ensure that any measurable impact was not dependent on a cumulative effect.

Participants were asked to provide their cognitive load rating at the following time points throughout each 5-min block: pregame (baseline), 1:30, 3:30 and 4:50 to ensure the collection of accurate ratings. That was done as multiple ratings have been recommended to maximize scale reliability (Ouwehand et al., 2021). The time of 4:50 was chosen instead of 5:00 to ensure this timing was still during gameplay and not post. Participants were given a minimum 1-min break between games to allow for the return to baseline.

2.3. Data acquisition

We used Mendi's portable headband fNIRS system to measure the hemodynamic response signal (Mendi®, Sweden, 2020) – previously validated in our lab for its ability to detect differences in oxyhemoglobin fluctuations between varying task loads in the prefrontal cortex (Boere et al., 2024). The system uses wavelengths 765 and 856 nm and outputs concentration changes of oxyhemoglobin (HbO) and deoxyhemoglobin (HbR). The sampling rate was 31.23 Hz and there is a hardware low-pass filter before sampling at 2.5 kHz. The optodes of the Mendi equipment measure activity in the prefrontal Brodmann area 10, one for the left and one for the right hemisphere. After the recording, all data were transported via Bluetooth to the Mendi IOS application.

2.4. Data processing & analysis

The data analysis was performed after the raw data were converted into optical density and concentration changes using the modified Beer-Lambert Law (Jobsis, 1977). All data processing was completed in MATLAB 2022a (Version 9.12, The MathWorks, Natick, MA, USA). Data were then filtered through a hamming bandpass (.05 Hz–2 Hz) to remove physiological noises (cardiac, respiratory, and Mayer waves) and signal drift (Pinti et al., 2020).

Typical hemodynamic responses increase for HbO with neural activity in a specific region and return to baseline afterward. In HbR, signals typically behave oppositely, decreasing upon stimulus onset and increasing back to baseline after the end of the stimulus. Here, we measured the mean value of the signal in a 5-min time window. Specifically, each participant completed four 5-min blocks, two of each condition in random order. Condition one played Tetris on a single iPad, and condition two played Tetris on three iPads concurrently. Then, the mean HbO values of each participant's game conditions were averaged separately – creating an overall average activity. This procedure was repeated for participants' game performance scores, errors made, and subjective cognitive load ratings.

All statistics were conducted in R (Version 4.0.0, the R Foundation, Vienna, Austria; R Team, 2016) using RStudio (Version 1.1.463, RStudio Inc., Boston, U.S.A.). All figures were created in GraphPad Prism (Version 9.2.1). Paired sample t-tests were used to compared high and low load conditions for neural and behavioral measures. Statistical assumptions for normality were tested using the Shapiro-Wilk test. When assumptions were violated, Wilcoxon signed-rank test was used. Spearman rank correlations were used to examine relationships between cognitive and behavioural measures. An alpha level of .05 was used for significance testing, and Holm's corrections were applied to adjust for multiple comparisons. Cohen's D effect size was computed for each comparison.

3. Results

3.1. Behavioral performance

We tested whether performance scores differed between the one-game and three-game conditions. A significant difference was observed, with one-game scores (M = 7995, 95% CI [4237, 11752]) higher than three-game scores (M = 3576, 95% CI [2396, 4756]), t(29) = 3.36, p = .002 (See Fig. 1).

Fig. 1.

Behavioural data for single (1 iPad) and multi-game (3 iPads) task conditions. Left panel – Mean Tetris scores. Right panel – Mean errors. All error bars represent 95% confidence intervals.

3.2. Errors

A significant difference in errors was found between conditions, with the one-game condition (M = .53, 95% CI [.34, .73]) resulting in fewer errors than the three-game condition (M = 1.52, 95% CI [.98, 2.10]), t(29) = 4.79, p < .001 (See Fig. 1).

3.3. Self-perceived cognitive load

Self-rated cognitive load also differed significantly, with lower ratings in the one-game condition (M = 4.12, 95% CI [3.71, 4.54]) compared to the three-game condition (M = 5.79, 95% CI [5.42, 6.20]), t(29) = 10.50, p < .001, Cohen's d = 1.62 (See Fig. 2).

Fig. 2.

Cognitive load ratings between single (1 iPad) and multi-game (3 iPads) conditions. Left panel - Change in oxyhemoglobin (HbO) concentrations recorded by Mendi compared to baseline. Right panel - Self-Reported Ratings. All error bars represent 95% confidence intervals.

3.4. Neural (HbO concentration)

HbO concentration differed significantly between conditions, with higher levels in the one-game condition (M = .0069 x 10^-6 mol/L, 95% CI [.89, 1.60]) than in the three-game condition (M = .0018 x 10^-6 mol/L, 95% CI [.52, 1.15]), t(29) = 3.71, p = .001, Cohen's d = .68 (See Fig. 2).

3.5. Correlations

After correcting for multiple comparisons, a moderate positive correlation was found between self-perceived cognitive load and errors, indicating that as cognitive load increased, participants made more errors. Conversely, there was a weak negative correlation between self-perceived cognitive load and task performance, suggesting that higher cognitive load was associated with lower task performance (see Table 1).

Table 1.

Spearman correlations.

	r-value	95% CI	p-value	CST
HbO & Errors	−.11	[-.36, .16]	.419	.025
HbO & Performance	.15	[-.12, .39]	.265	.0167
HbO & Cognitive Load	−.11	[-.36, .16]	.426	.050
Cognitive Load & Errors	.43	[.12, .62]	.001	.010
Cognitive Load & Performance	−.33	[-.54, −.07]	.011	.0125

Note. 95% CI = confidence interval. CST = corrected significance threshold. Bolded p-values represent significant associations after correcting for multiple comparisons.

4. Discussion

The goal of this study was to examine cognitive load differences in single-task versus multi-task conditions using a two-channel fNIRS device with a complex, ecologically valid task paradigm that simulates real-world working memory and attention demands. We hypothesized that participants would report higher subjective cognitive load, show lower performance scores, and make more errors in the multi-task condition compared to the single-task condition. Additionally, we hypothesized that fNIRS measurements would reveal increased prefrontal cortex activation (HbO concentration) in the multi-task condition, reflecting greater cognitive demands. The results largely validated our hypotheses regarding self-reported cognitive load, performance, and error rates. However, contrary to our expectations, the neural data showed a surprising decrease in prefrontal activation under multi-task conditions.

Participants demonstrated significantly lower scores and more frequent errors in the three-game multitask condition compared to the single-game single-task condition, illustrating how managing multiple tasks simultaneously can impair performance. According to cognitive load theory, this deterioration likely results from task demands exceeding working memory capacity (Sweller, 2011). These performance declines are supported by subjective ratings, which showed significantly higher self-reported cognitive load in the multitask condition, indicating that participants were aware of the additional mental demands. Together, these findings confirm that multitasking can lead to cognitive overload, impairing both accuracy and overall effectiveness.

Contrary to our initial hypothesis, HbO concentration in the prefrontal cortex was lower in the multitask condition than in the single-task condition. This result does not align with the standard findings from previous fNIRS studies using simpler tasks, where higher cognitive load generally corresponds to increased prefrontal activation (Boere et al., 2024; Fishburn et al., 2014; Causse et al., 2017). Instead, our findings suggest a "cognitive disengagement" or "neural overload" effect, where the brain reduces engagement in response to extreme demands. Similar patterns have been observed in other studies. For example, Durantin et al. (2014) reported decreased prefrontal activation at high levels of task difficulty, suggesting that the brain reduces engagement as a response to mental overload. Likewise, Mandrick et al. (2013) found stable or reduced prefrontal responses in dual-task conditions, indicating that performance begins to degrade as cognitive capacity is exceeded. Together, these studies suggest that the prefrontal cortex, which typically supports attention and working memory, may downregulate activity as a protective adaptation to prevent further cognitive strain. This lack of prefrontal activation and associated decline in performance also aligns with the bottleneck theory, which posits that the brain has limited capacity to process concurrent information streams (Pashler, 1994). In complex multitasking scenarios, this bottleneck likely compels the brain to adopt less resource-intensive processing strategies, reflecting an adaptive response to avoid overload when cognitive resources are maxed out.

This study's use of mobile fNIRS in a complex, multitasking environment represents an important step forward in cognitive load research, addressing the limitations of standard lab-based tasks. Most previous studies have relied on simplified working memory paradigms to validate fNIRS responses, yet controlled setups often lack the ecological validity required to reflect real-world multitasking demands. By applying mobile fNIRS to a dynamic, high-load task, our study broadens the scope of cognitive load research and provides valuable insights for high-stakes environments, including aviation, emergency response, and complex industrial operations. Findings indicating that prefrontal activation may not linearly increase under extreme cognitive load suggest a threshold at which cognitive systems shift toward prioritizing stability over sustained engagement. Recognizing that the brain may perform differently under unique tasks demands, highlights the importance of practical strategies, such as structured cognitive breaks or task rotations, to manage demands effectively in demanding settings. Broadening research to include more ecologically valid tasks will ultimately deepen understanding of cognitive load in applied contexts and inform interventions to enhance performance and safety.

While our findings contribute a novel perspective on cognitive load research, it is still important to acknowledge the study's limitations. Our research was primarily conducted with undergraduate students, averaging 21 years of age, which limits the generalizability of our conclusions. Although participants' baseline familiarity with Tetris was considered and accommodated with practice rounds, the study did not adjust for individual levels of expertise or practice beyond the lab setting. Additionally, individual differences in executive functioning and working memory capacity may interact with the effects of multitasking on cognitive load and performance.

5. Conclusion

Our study addresses a critical gap in cognitive load research by using mobile fNIRS to examine multitasking in a complex, real-world scenario, moving beyond the simpler, controlled tasks typical of past studies. By doing so, we found that managing multiple tasks can overwhelm mental resources, leading to declines in accuracy and increases in errors. Notably, prefrontal cortex activation did not increase with higher task demands, suggesting that the brain may “downshift” or limit additional resource allocation when cognitive load becomes excessive. This unexpected result challenges the standard one-to-one relationship observed in simpler paradigms and has significant implications for high-stakes fields like aviation and healthcare, where understanding cognitive limits is crucial. By advancing cognitive load research into more realistic settings, our study provides insights that could inform strategies to optimize performance and safety in cognitively demanding environments.

CRediT authorship contribution statement

Katherine Boere: Writing – review & editing, Writing – original draft, Visualization, Validation, Supervision, Methodology, Investigation, Formal analysis, Conceptualization. Francesca Anderson: Writing – original draft, Project administration, Data curation, Conceptualization. Kent G. Hecker: Writing – review & editing. Olav E. Krigolson: Writing – review & editing, Validation, Supervision, Methodology.

Declaration of competing interest

The authors declare that they have no known competing interests or personal relationships that could have appeared to influence the work reported in this paper.

Data availability

All processing scripts can be found at https://github.com/Neuro-Tools. In addition, the data supporting this study's findings are available fromhttps://osf.io/nj56y/.