Figure - PMC

Skip to main content

An official website of the United States government

Here's how you know

Here's how you know

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

View full-text article in PMC

. 2023 Dec 15;17:1283206. doi: 10.3389/fnhum.2023.1283206

Search in PMC
Search in PubMed
View in NLM Catalog
Add to search

Copyright © 2023 Ahmed, Nidiffer and Lalor.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

PMC Copyright notice

Experimental design. (A) General outline of both experiments. Participants were presented with two audiovisual speakers, one on the left and another on the right (faces covered in the figure to protect identity of the speaker). The deep magenta arrow indicates the speaker (left or right) they had to attend while the blue, green and orange arrows indicate their gaze behavior. They either attended to the audio-only (A), visual-only (V) or audiovisual (AV) modality of the speaker indicated by the text cue. Both attended and unattended AV and (A + V) decoder models were derived from their EEG recordings. As described in the text, these decoders sought to reconstruct an estimate of the speech envelope from the corresponding EEG. Multisensory integration was then quantified as the difference between reconstruction accuracy (Pearson’s r between the actual and the reconstructed envelope of the audiovisual speaker) using the AV decoder and the (A + V) decoder. (B) Experimental conditions in detail. The deep magenta arrow placed on top of the panels corresponds to attentional cue, indicating that participants were attending to the left stimulus in this example. In experiment 1, participants fixated on a white crosshair placed in the middle of the computer screen (crosshair condition, first panel). In experiment 2, participants either looked at the left speaker that also coincided with the cued attentional side (direct looking condition, middle panel) or at the right speaker marked by the white ellipse while still attending to the left speaker (eavesdropping condition, third panel). Left and right attentional sides were counterbalanced. (C) Attentional side was counterbalanced. Another example is shown where participants are cued to attend to the right speaker, indicated by the deep magenta arrow. Once again, the blue arrow indicates crosshair fixation, while the green arrow indicates directly looking at the to-be-attended speaker (this time the right speaker) and the orange arrow indicates directly looking at the to-be-ignored speaker (this time the left speaker).