Cross-Modal Capture: Preliminary Evidence of Inefficient Filtering in Children with Autism Spectrum Disorder

Abstract

This study investigates how task-irrelevant auditory information is processed in children with autism spectrum disorder (ASD). Eighteen children with ASD and 19 age- and IQ-matched typically developing (TD) children were presented with semantically-congruent and incongruent picture-sound pairs, and in separate tasks were instructed to attend to only visual or both audio-visual sensory channels. Preliminary results showed that when required to attend to both modalities, both groups were equally slowed for semantically-incongruent compared to congruent pairs. However, when asked to attend to only visual information, children with ASD were disproportionally slowed by incongruent auditory information, suggesting that they may have more difficulty filtering task-irrelevant cross-modal information. Correlational analyses showed that this inefficient cross-modal attentional filtering was related to greater sociocommunicative impairment.

Introduction

Successfully navigating one’s environment requires integrating information across multiple sensory modalities. In the laboratory, however, selective attention tasks generally focus on participants’ ability to select task-relevant and/or suppress task-irrelevant information within a single modality. While studies that have examined attention and perception in autism spectrum disorder (ASD) also commonly focus on one sensory channel (i.e., only vision or audition), more recent research has begun to focus on interactions across modalities. Typically, these studies examine how information from both visual and auditory systems are integrated into one percept. For example, a number of studies have investigated how task-relevant multisensory (i.e., audio-visual) signals enhance speech perception (Beker et al., 2018). However, cross-modal interactions may also serve to distract us from a to-be-attended sensory channel (e.g., watching the road while driving with noisy children). The present study investigates the latter phenomenon, and examines how task-irrelevant auditory information is processed while children with ASD and their TD peers attend and respond to information within the visual modality.

Prior reports have demonstrated that ASD is associated with the inability to suppress or fdter task-irrelevant information (e.g., Burack, 1994; Keehn et al., 2016; Murphy et al., 2014). For example, a large body of studies has examined distractor suppression using the Eriksen flanker task (Eriksen & Eriksen, 1974), and have shown that individuals with ASD are slower and less accurate compared to their typically developing (TD) peers when the target is flanked by incongruent distractors (e.g., Adams & Jarrold, 2012). Likewise, impaired suppression of task-irrelevant sounds while attending to auditory information has also been shown in ASD (teder-Salejarvi et al., 2005). These and other studies provide insight into impaired within-modality filtering; however, less is known about how task-irrelevant information may be fdtered across modality. Murphy and colleagues (2014) used a cued auditory-visual paradigm and showed significant reductions in accuracy and slower responses in ASD, as well as atypical modulation of EEG alpha-band activity when task-irrelevant sensory information was present. Similarly, results from an audio-visual oddball paradigm by Ciesielski et al. (1990) showed increased false alarms to to-be-ignored targets in the unattended sensory channel and reduced amplitude of electrophysiological indices of selective attention in adults with ASD. While results from intersensory filtering remain limited, together with findings from unimodal studies, they suggest that ASD may be associated with obligatory processing of task-irrelevant stimuli (Belmonte, 2017).

The aim of the present study was to further examine performance of children and adolescents with ASD when they were instructed to attend to either one modality (visual) or integrate across two modalities (visual and auditory) when information in each of those sensory channels was either semantically congruent or incongruent. We hypothesized that when obligated to attend to both modalities, children with ASD would perform similar to their TD peers. However, when instructed to attend and respond to a single modality (i.e., visual), atypical attentional filtering would disproportionally slow children with ASD. Finally, because behavioral (Faja et al., 2016) and electrophysiological (Larson et al., 2012) indices of distractor inhibition have been shown to be associated with sociocommunicative impairment, we examined the relationship between our behavioral measures of filtering and ASD symptomatology.

Methods and Materials

Participants

Nineteen children with ASD and 20 TD children participated in the study. One child with ASD and one TD child were unable to complete the experimental paradigms and were thus excluded, resulting in a final sample of 18 children with ASD and 19 age- and nonverbal IQ-matched TD children (Table 1). Clinical diagnoses were confirmed using the Autism Diagnostic Interview – Revised (ADI-R; Rutter et al., 2003), the Autism Diagnostic Observation Schedule (ADOS; Lord et al., 1999), and expert clinical judgment according to DSM-IV criteria. Children with ASD-related medical conditions (e.g., Fragile-X syndrome, tuberous sclerosis) were excluded. Participants in the TD group had no reported family history of ASD and were confirmed via parent report to be free of ASD-related symptoms or any other neurological or psychiatric conditions. Informed assent and consent were obtained from all participants and their caregivers in accordance with the university Institutional Review Board.

Table 1.

Participant Characteristics

	ASD	TD	t-value	P
n (M:F)	18 (17:1)	19 (18:1)	-	-
Age (years)	14.3 (3.0);	13.5 (3.2);	0.86	0.4
	8.7–19.9	8.9–18.8
Verbal IQ	106 (18);	111 (11);	1.01	0.32
	79–147	87–134
Nonverbal IQ	109 (18);	114 (10);	0.90	0.37
	70–140	96–132
SRS Total score	82 (15);	41 (5);	9.35	< .001
	56–112	35–47
ADOS  Communication	3 (2);	-	-	-
	0–5
Social Interaction	8 (3);	-	-	-
	3-13
Repetitive Behavior	2 (1);	-	-	-
	0–5

IQ determined using the Wechsler Abbreviated Scale of Intelligence (Wechsler, 1999). Mean (SD); range.

Experimental Paradigms

Visual Attention (VA) Task.

Stimuli consisted of pictures and sounds of animals (i.e., cat, cow, crow, dog, duck, frog, lion, rooster, sheep) and musical instruments (i.e., banjo, bell, cymbals, drum, flute, piano, saxophone, trumpet, violin). Centrally-presented pictures subtended between 5.1–8.0° × 5.2–8.6° visual angle. For each trial, a picture and a sound were presented, which were either congruent or incongruent. The auditory stimuli were presented simultaneously with as well as after the onset of the visual stimuli at 11 different stimulus onset asynchronies (SOA; 0, 25, 50, 75, 100, 125, 150, 175, 200, 225, and 250ms) relative to the onset of the visual stimulus. For congruent trials, the sound matched the picture (e.g., see a dog, hear a “bark”), and for incongruent 4 trials the sound was randomly selected from the opposite category and did not match the picture (e.g., see a dog, hear a violin). As illustrated in Figure 1, the trial began with a blue fixation cross which was presented for 500ms. Next, the visual stimulus was presented and remained on the screen until a response was made or until 1500ms had elapsed. The intertrial interval was 1000ms, during which a white fixation cross was presented. Participants were asked to indicate via a dominant-hand button-box response whether the picture was an animal or an instrument, and were instructed to respond as quickly as possible without making mistakes. Participants were also explicitly informed that they should focus only on the pictures and to ignore the sounds.

Figure 1.

Illustration of an incongruent trial. Pictures and sounds were identical for both visual attention (VA) and integrated attention (IA) tasks. For VA task, the sound was presented at 11 different stimulus onset asynchronies relative to picture onset, whereas for the IA task the onset of the picture and the sound were always simultaneous.

The task included 552 trials, half of which included an animal picture and half of which included an instrument picture. For each visual stimulus type, half the auditory stimuli were congruent and half were incongruent, which resulted in four picture-sound pairings: animal-animal, instrument-instrument, animal-instrument, and instrument-animal. Stimulus onset asynchrony was counterbalanced across each pairing, resulting in 12 trials per SOA for each condition. Baseline trials, which did not include a sound, were also presented for each visual stimulus type (animal, instrument). The task was divided into three blocks of 184 trials, allowing short breaks between blocks. Visual stimuli (animal, instrument), sound stimuli (animal, instrument), and SOA (0, 25, 50, 75, 100, 125, 150, 175, 200, 225, 250ms) were counterbalanced across block and were presented in a pseudorandom order.

Integrated Attention (IA) Task.

Stimuli and trial presentation were identical to the visual attention task with the exception that visual and auditory were always presented simultaneously. Importantly, the instructions provided to the participants were different. Participants were asked to indicate via a dominant-hand button-box response whether the sound and picture matched, and were instructed to respond as quickly as possible without making mistakes. Given the nature of the judgment participants were required to make for the IA task, which required attention to both auditory and visual information, the sound and picture were only presented at the same time rather than at varying SOAs (as in the VA task).

The task included 200 trials, half of which included an animal picture and half of which included an instrument picture. For each visual stimulus type, half the auditory stimuli were congruent and half were incongruent, which resulted in four picture-sound pairings: animal-animal, instrument-instrument, animal-instrument, and instrument-animal. The task was divided into two blocks of 100 trials, allowing short breaks between blocks. Visual stimuli (animal, instrument) and sound stimuli (animal, instrument) were counterbalanced across block and were presented in a pseudorandom order. Prior research has shown that ASD is associated with impairments in set shifting (Hill, 2004). Therefore, the IA task was always presented after the VA task in order to eliminate the potential confound associated with set shifting in the interpretation of our hypothesized group difference for the VA task.

Data Analysis

For both VA and IA paradigms, picture conditions (animal, instrument) were collapsed into congruent and incongruent trial types based on associated sound stimuli. For the VA task, mean accuracy rates and response times (RT) for correct trials were entered into a 2 (group: ASD, TD) × 2 (congruency: congruent, incongruent) × 11 (SOA: 0, 25, 50, 75, 100, 125, 150, 175, 200, 225, 250ms) mixed-model repeated measures ANOVA. For the IA task, mean accuracy rates and RTs for correct trials were entered into a 2 (group: ASD, TD) × 2 (congruency: congruent, incongruent) mixed-model repeated measures ANOVA. Crucially, in order to investigate differences between VA and IA experiments, RTs were entered into a 2 (group: ASD, TD) × 2 (experiment: VA, IA) × 2 (congruency: congruent, incongruent) mixed-model repeated measures ANOVA. Lastly, we examined the relationship between experimental measures and ASD symptomatology. Summary scores from the ADOS diagnostic algorithm were used as symptom measures, with higher ADOS scores reflecting greater severity. Difference scores were calculated for VA and IA experiments by subtracting RT of the congruent from the incongruent condition. Pearson correlations between ADOS scores and difference scores were conducted.

Results

Visual Attention Task

There were no significant differences between groups in overall accuracy (ASD = 91%; TD = 92%) and there were no significant differences in accuracy as a function of congruency or SOA, nor were there significant interactions between group and any of these factors (all p > .13) with the exception of group and SOA, which was marginally significant, F(10,350) = 1.65, p = .09, ƞ_p² = .05. Follow-up paired-samples t-tests showed that TD children did not differ in accuracy from the first (0ms) to last (250ms) SOA (first = 92%, last = 92%), t(18) = −0.11, p = .91, whereas children with ASD showed significantly worse performance for the first (M = 88%) compared to the last (M = 92%) SOA, t(17) = 2.35, p = .03. For RT, similarly, there were no significant differences between groups (ASD = 611ms; TD =599ms). However, there were significant main effects of congruency, F(1,35) = 22.86, p < .001, ƞ_p² = .37, and SOA, F(10,350) = 3.15, P= .001, ƞ_p² =.08, reflecting faster RT for congruent (M = 597ms) compared to incongruent (M = 613ms) conditions and slower RTs for longer SOAs (first [0ms] = 598ms; last [250ms] = 619ms). There was a marginally significant interaction between group and congruency, F(1, 35) = 3.77, p = .06, ƞ_p² = 10, as the ASD group was disproportionally slowed by incongruent distractors, and a marginally significant 3-way interaction between group, congruency, and SOA, F(10, 350) = 1.66, p = .09. To further examine this, difference scores were created by subtracting the RT of the congruent from the incongruent conditions at each SOA (Figure 2a). Follow-up t-tests revealed significantly greater difference scores (i.e., slower RTs to incongruent relative to congruent trials) for the ASD group at 50ms as well as 175ms SOAs compared to the TD group (all p < .05). There were no other significant interactions between congruency and SOA or SOA and group (all p > .53)

Figure 2.

Difference scores (incongruent – congruent) for visual attention (VA) at each SOA for TD (solid line) and the ASD (dashed line) group (a). Difference scores (incongruent – congruent) for VA and integrated attention (IA) tasks (b). For the ASD group only, scatter plot shows correlation between VA and IA difference scores and ADOS Total score (c). Dashed line, Xs for IA task; Solid line, circles for VA task. Error bars in (a) and (b) reflect ± 1 SEM.

Integrated Attention Task

For accuracy rates, there was a significant main effect of congruency, F(1, 35) =12.21, p < .001, ƞ_p² = .26, reflecting reduced accuracy for incongruent compared to congruent trials. However, there was no difference in accuracy between groups (ASD = 84%; TD = 84%), F(1, 35) = 0.00, p = .95, ƞ_p² = .00, and no significant interaction between group and congruency, F(1, 35) = 0.18, p = .67, ƞ_p² = .01. The pattern of results were identical for RT, as there was a significant main effect of congruency, F(1, 35) =70.68, p < .001, ƞ_p² = .67, reflecting faster RT congruent compared to incongruent trials. However, there was no difference in RT between groups,F(1, 35) = 1.42, P = .241, ƞ_p² = .04, and no significant interaction between group and congruency, F(1, 35) = 0.00. p = .99, ƞ_p² = .00.

Between-Task Comparison

There were significant main effects of experiment and congruency, as RTs were faster for the VA compared to the IA task and for congruent compared to incongruent conditions. Again, groups did not differ in their overall RT, F(1, 35) =0.02, p =.88. ƞ_p² = .00, however, there was a significant three-way interaction between group, experiment, and congruency, F(1, 35) =5.08. p = .03, ƞ_p² = .13. To further examine this interaction, difference scores were calculated by subtracting congruent from incongruent trials for each task (Figure 2b). Follow-up paired-samples t-tests showed that the difference score was greater for the IA compared to the VA task for the TD group, t(18) = −4.72, p < .001. However, difference scores between VA and IA tasks were not significantly different for the ASD group,t(17) = −1.75, p = .10. Further, independent-samples t-tests showed that difference scores were not different between groups for the IA task, t(35) = −1.19, p = .24, whereas they were different for the VA task, t(35) = 1.86, p = .07.

Relationship with ASD Symptomatology

As illustrated in Figure 2c, higher ADOS Total score was associated with VA difference score, r(17) = .49, p = .048, but not IA difference score, r(17) = −.05, p = .836. No other significant correlations existed for VA or IA difference scores.

Discussion

Using identical auditory and visual stimuli and similar paradigms we investigated audio-visual information processing in children and adolescents with ASD while they were instructed to attend to either a single visual or both audio-visual sensory channels. When required to attend to both modalities (“do the sound and the picture match”) performance for ASD and TD groups was equivalent; children in both groups were equally slowed for semantically incongruent compared to congruent trials. However, when asked to attend to only the visual information, children with ASD were disproportionally slowed by semantically incongruent auditory information, suggesting that they that they have more difficulty filtering task-irrelevant cross-modal information. Further, correlational analyses showed that this inefficient cross-modal attentional filtering may be related to increased sociocommunicative impairment in ASD.

Results from the integrated attention task are consistent with previous studies that have demonstrated equivalent semantic processing in ASD. For example, prior reports have shown equivalent semantic priming in ASD (Toichi & Kamio, 2001). Additionally, using stimuli similar to the present study, McCleery et al. (2010) used event-related potentials (ERP) and showed similar automatic non-verbal semantic integration in ASD and TD children in a passive paradigm. Importantly, results from the IA task demonstrate that children with ASD show similar slowing to semantically-incongruent information when required to attend to both visual and auditory information.

In contrast, for the VA task, in which participants were instructed to respond to visual and ignore auditory information, children with ASD were slower to respond than TD peers when incongruent auditory stimuli were presented. Difficulties filtering irrelevant sounds, as evidenced by slower RTs to incongruent compared to congruent trials in ASD, occurred early at shorter SOAs and persisted to longer SOAs. Further, whereas accuracy for TD children did not vary based on the timing of irrelevant auditory stimuli, children with ASD were significantly worse for simultaneous compared to late auditory information. These findings are in agreement with prior studies investigating resistance to distraction and filtering of irrelevant information more generally, and for inter-sensory stimuli specifically (Ciesielski et al., 1990; Murphy et al., 2014). Electrophysiological evidence from both Ciesielski et al. (1990) and Murphy et al. (2014) indicates that mechanisms of selective attention, which are associated with selection of task-relevant information and suppression of task-irrelevant information may contribute to impaired performance. Furthermore, together with previous behavioral (Faja et al., 2016) and ERP (Larson et al., 2012) results from unimodal flanker tasks, the present results suggest that inefficient filtering of task-irrelevant inter-sensory information is associated with social communication impairment in ASD.

Alternatively, as has been highlighted by others (Adams & Jarrold, 2012), impaired filtering in ASD may result from increased perceptual capacity (e.g., Remington et al., 2009). More recently, Tillmann and colleagues (Tillmann et al., 2015; Tillmann & Swettenham, 2017) have shown that increased perceptual capacity in ASD results in greater capture by task-irrelevant auditory information. Thus, in the present study, larger perceptual capacity in children with ASD may have resulted in processing of task-irrelevant auditory information, and thus slowing of RT to semantically incongruent sounds in the VA condition.

The current study is not without limitations. In particular, the interpretation of our results remains limited and should be considered preliminary given our sample size. In addition, while the stimuli included for each task were identical, task length and some experimental manipulations did differ. Therefore, the findings of the present study should be interpreted with caution. Nevertheless, the results add to the growing body of evidence that suggests that children with ASD have difficulty filtering task-irrelevant information, and add further support that impaired distractor suppression in ASD exists across discrete sensory channels. While multimodal tasks more closely mimic real-world environments, cross-modal paradigms have only examined processing of task-irrelevant information for a single source (i.e., computer also presenting visual stimuli). However, inter-sensory capture is more likely to occur from distinct sources. Thus, further work should attempt to investigate a more ecologically valid index of impaired distractor suppression in ASD in an effort to understand how impaired filtering may contribute to the sociocommunicative, academic, and other challenges faced by individuals with ASD.