Sensory processes modulate differences in multi‐component behavior and cognitive control between childhood and adulthood

Abstract

Many everyday tasks require executive functions to achieve a certain goal. Quite often, this requires the integration of information derived from different sensory modalities. Children are less likely to integrate information from different modalities and, at the same time, also do not command fully developed executive functions, as compared to adults. Yet still, the role of developmental age‐related effects on multisensory integration processes has not been examined within the context of multicomponent behavior until now (i.e., the concatenation of different executive subprocesses). This is problematic because differences in multisensory integration might actually explain a significant amount of the developmental effects that have traditionally been attributed to changes in executive functioning. In a system, neurophysiological approach combining electroencephaloram (EEG) recordings and source localization analyses, we therefore examined this question. The results show that differences in how children and adults accomplish multicomponent behavior do not solely depend on developmental differences in executive functioning. Instead, the observed developmental differences in response selection processes (reflected by the P3 ERP) were largely dependent on the complexity of integrating temporally separated stimuli from different modalities. This effect was related to activation differences in medial frontal and inferior parietal cortices. Primary perceptual gating or attentional selection processes (P1 and N1 ERPs) were not affected. The results show that differences in multisensory integration explain parts of transformations in cognitive processes between childhood and adulthood that have traditionally been attributed to changes in executive functioning, especially when these require the integration of multiple modalities during response selection. Hum Brain Mapp 38:4933–4945, 2017. © 2017 Wiley Periodicals, Inc.

INTRODUCTION

Many of our everyday tasks require complex sensorimotor integration processes, like talking on the phone while typing on a keyboard at the same time. Such “multitasking” situations require the ability to generate, process, prioritize, and cascade multiple actions in a rapid succession to execute multicomponent behavior [Duncan 2010], which is also known as action cascading [Dippel and Beste 2015; Duncan 2010]. It is defined as the ability to generate, process, and execute separate task goals and responses in an expedient temporal order to be able to display efficient goal‐directed behavior [Dippel and Beste, 2015; Duncan, 2010; Mückschel et al., 2014, 2015; Stock et al., 2014, 2015]. Thus, action cascading and multicomponent behavior refer to the concatenation of different executive functions or subprocesses; for example, stop an ongoing action and turn to another required action. This often requires the integration of information derived from different sensory modalities [Gohil et al. 2015, 2016]. For successful sensorimotor integration, the temporal contiguity of stimuli is a fundamental requirement, especially given that even simultaneous stimulus input does not necessarily reach primary sensory cortices (e.g., visual and auditory areas) at the same time [Hanson et al. 2009; Kayser et al. 2015; Meredith and Stein 1985; Stein and Meredith 1993]. In this context, the concept of a critical “temporal window of integration” has been put forward by several neurophysiological and behavioral studies [e.g., Meredith 2002; Stein and Meredith 1993]. These studies provide evidence that there is a critical “window” of time within which multisensory stimuli are very likely to become integrated [Conrey and Pisoni 2006; Diederich and Colonius 2009; Meredith 2002; van Wassenhove et al. 2007]. Yet, this temporal window of integration has been shown to undergo strong developmental changes and gradually becomes longer during childhood and adolescence [Gori et al. 2008; Hahn et al. 2014; Hillock‐Dunn et al. 2016; Nardini et al. 2008]. This means that compared to adults, children are less likely/able to integrate information from different modalities, especially when there is a temporal gap/stimulus onset asynchrony (SOA).

Yet still, the role of developmental, age‐related effects on multisensory processes during multicomponent behavior has not been examined until now. This is problematic because differences in multisensory integration might actually explain some of the developmental effects that have traditionally been attributed to changes in executive functioning, especially when this requires the integration of different sensory modalities. Executive functions include conflict detection, cue/context monitoring, attention orientation, target processing, response selection, and performance monitoring [Diamond 2013]. Especially regarding response selection processes as one important instance of executive functions this may be critical to consider, because this aspect of executive function is central for action cascading and multicomponent behavior, in which the integration of multiple sensory inputs has been shown to play an important role [Gohil et al. 2015, 2016].

One of the reasons is that multicomponent behavior depends on extended fronto‐parietal cortical [Duncan, 2010] and fronto‐striatal networks [Beste and Saft 2015; Beste et al. 2015; Stock et al. 2014, 2016; Yildiz et al. 2014], which are well‐known to show a protracted development in children and adolescents [Fair et al. 2009; Giorgio et al. 2010; Gogtay et al. 2004; Hämmerer et al. 2014; Sowell et al. 2003]. Yet, those networks are also known to be involved in multisensory integration processes [Bizley et al. 2016; Redgrave and Gurney 2006; Yau et al. 2015]. If developmental changes in the critical time window for multisensory integration played a significant role for multicomponent behavior, one would expect those effects to be limited to cases of multimodal (as compared to unimodal) stimulus presentation. The reason for this hypothesis is that only situations with multimodal stimulus input require the integration of stimuli from different modalities. Developmental effects in multicomponent behavior should furthermore be most evident in case of a temporal gap between stimuli signaling different actions while situations with simultaneous stimulus presentation should be significantly less affected. In the case that developmental differences in multicomponent behavior are mainly due to processes affecting the temporal window of multisensory information integration, there should be no difference between children and adults when task performance does not require integrating information from different sensory modalities. Importantly, this would imply that we might have overestimated general developmental effects on the ability of children and adults to accomplish multicomponent behavior. Such a finding would challenge current thinking about subprocesses involved in executive functions, which are generally assumed to undergo strong modulations from childhood to adulthood [Diamond 2013].

These hypotheses were tested in a neurophysiological study employing EEG recordings and source localization methods in children and adults. Both groups were examined with a purely visual and an auditory‐visual version of a Stop‐Change (SC) paradigm to vary the need for multisensory information integration (see Material and Methods section for details). Within the paradigm, the temporal delay between a visual STOP and visual or auditory CHANGE stimulus was furthermore varied to manipulate the temporal window of integration. Several lines of evidence have shown that processes mediating between stimulus evaluation and response execution, that is, processes at the response selection stage that are reflected by the P3 event‐related potential (ERP) [Twomey et al. 2015; Verleger et al. 2005], are most important for performance in this task [Beste et al. 2015; Stock et al. 2014, 2016; Yildiz et al. 2014]. Modulations in the P3 have been shown to be associated with modulations in medial frontal and inferior parietal cortices including the temporo‐parietal junction [Mückschel et al. 2014]. Therefore, and because inferior parietal cortices are important for sensory integration processes across sensory modalities [Ionta et al. 2011; Jakobs et al. 2012; Parker Jones et al. 2014], it is likely that the P3 as well as associated functional neuroanatomical structures show an interaction between age group, number of sensory modalities involved, and the temporal spacing of stimuli signaling different executed actions.

Yet, it cannot be excluded that processes related to automatic, bottom‐up guided allocation of perceptual and attentional resources, as well as top‐down guided discrimination process selectively allocating attention to relevant stimulus features are also modulated by the aforementioned experimental factors. These processes have been shown to be reflected by the P1 [Brisson et al. 2007; Buzzell et al. 2013; Luck et al. 2000] and N1 ERP component [e.g., Hopf et al. 2002; Luck et al. 1990; Vogel and Luck 2000], respectively. However, previous results have shown that performance in the applied experimental paradigm is not affected by the processes reflected by the P1 and N1 [Gohil et al. 2015; Mückschel et al. 2017]. It is therefore unlikely that P1 and N1 processes reflect the expected interactive effects between age group, number of sensory modalities involved and the temporal spacing of stimuli signaling different to be executed actions.

MATERIAL AND METHODS

Sample

Our sample consisted of N = 58 healthy right‐handed participants N = 28 of which were adults (aged 22–30; mean age = 24.78) and N = 30 of which were children (aged 8–12; mean age = 11.71). All of the participants stated to be right‐handed and had no history of psychiatric or neurologic diseases. Each participant performed two experiments (visual or visual‐auditory version of the task) in two different, counterbalanced appointments. Each adult participant or the children's parents gave written informed consent before beginning the experiment. After the experiment, each participant was reimbursed with 20 €. The study was approved by the ethics committee of the Faculty of Medicine of the TU Dresden and conducted in accordance with the declaration of Helsinki.

General experimental paradigm

A modified version of the SC paradigm introduced by Verbruggen et al. [2008] was used in this study (see Fig. 1A for illustration) and this version of the experiment was further modified for children (see Fig. 1B for illustration) [Mückschel et al. 2015].

Figure 1.

Schematical illustration of the experimental setup for adults (A) and children (B). The experiments presented a visual GO signal (white circle) at the beginning of all trials. In GO trials, the subjects needed to respond with the right hand (middle finger = “above” response, index finger = “below” response). In SC trials, the GO stimulus was followed by a visual STOP stimulus (red rectangle, see middle) after a variable and individually adjusted stop‐signal delay (SSD). The CHANGE stimulus was either presented with a stimulus onset asynchrony (SOA)/stop‐signal delay (SCD) of 0 ms or of 300 ms after the STOP stimulus. In the unimodal task, the CHANGE stimulus was a bold yellow line. By contrast, the bimodal task used 200 ms sine tones (1,300 Hz, 900 Hz, and 500 Hz) as CHANGE stimuli. [Color figure can be viewed at http://wileyonlinelibrary.com]

In a sound‐attenuated room, all participants were comfortably seated in front of a 17 inch CRT computer monitor at a distance of 57 cm. A regular computer keyboard was placed in front of the participants and they were instructed to press one out of four different keys (“S,” “C,” “N,” and “K”) in each trial to correctly respond. The participants were asked to respond with both hands (i.e., “S”/“C” using the left hand and “N”/“K” using their right hand). The adult participants performed the task with 864 trials. Two thirds were “GO” trials and the rest were “SC” trials. The trial order was pseudo‐randomized for these conditions. The task array was presented on a black background (0.4 cd/m²). As shown in Figure 1, it consisted of a white bordered rectangle (55 × 16 mm) which contained four vertically arranged circles with white borders, which were separated by three white horizontal lines (width 13 mm, line thickness 1 mm). This empty array was presented at the beginning of each trial for 250 ms before one of the four circles (diameter of 7 mm) was filled in with white color (120.1 cd/m²), thus becoming the target stimulus in the GO condition. The participants were instructed to press one of the two keys on the keyboard with their right hand to report the location of this GO stimulus in relation to the middle line. Of note, the GO stimulus remained on the screen until the end of the trial. Participants had to respond with their right middle finger (“K” key) when the target was located above the middle white line, and to respond with their right index finger (“N” key) when the target was located below the middle line. If the participants failed to respond within 1000 ms after the target stimulus onset, a sign asking to speed up responses (the German word “Schneller!” which translates to “Faster!”) appeared above the stimulus array. This sign stayed on the screen until the trial was ended by a button press.

SC trials occurred with a likelihood of 33% (i.e., constituted one‐third of the trials). These SC trials also began with the empty array followed by the GO stimulus. During the SC trials, a STOP stimulus (the white rectangle border turned red, see Fig. 1) was presented after the GO stimulus with a variable Stop‐signal delay (SSD). This STOP stimulus as well as the GO stimulus remained on the screen until the end of the SC trial because the position of the target/GO stimulus in relation to one of the three horizontal lines had to be re‐evaluated upon the presentation of the CHANGE stimulus. The SSD was adjusted to each participant's individual task performance using a staircase algorithm throughout the experiment [cf. Verbruggen et al., 2008]: The SSD was initially set to 250 ms. When the participant did not press a key before the presentation of the STOP stimulus and correctly responded to the CHANGE stimulus as described below during an SC trial, the SSD was increased by 50 ms. In contrast to this, any incorrect responses (i.e., responses within the SSD/before the CHANGE stimulus as well as wrong responses to the CHANGE stimulus) decreased the SSD by 50 ms. As a result, the staircase procedure produced a 50% probability of successfully performed SC trials in case participants correctly followed the task instructions. The SSD variation was restricted to range from 50 to 1,000 ms to keep the trial duration within reasonable limits.

A CHANGE stimulus was presented after the STOP stimulus and instead of the right hand, it required the participants to respond with their left hand. The reaction time (RT) was measured relative to the presentation of this CHANGE stimulus. There were two SC conditions. In the first condition, there was no Stop‐Change delay (SCD0) so that STOP and CHANGE stimuli were presented at the same time. The second SC condition had a stimulus onset asynchrony of 300ms (SCD300) so that the CHANGE stimulus always followed the onset of the STOP stimulus after 300 ms. The versions of the experiments either used visual or auditory CHANGE stimuli (as explained below). Regardless of the stimulus modality, the CHANGE stimuli would point out one of the three lines to the participants who were instructed to spatially relate the target (i.e. the white circle) to the new reference line. Participants were instructed to respond with their left hand middle finger (“S” key) when the target was located above the newly set reference line, and to respond with their left hand index finger (“C” key) when the target was located below the newly set reference line. If participants did not respond within 2,000 ms after the onset of the CHANGE stimulus, the speed up sign appeared above the stimulus array and stayed on the screen until the trial was terminated by a button press. The dominant hand was always used to respond to the GO stimuli because it responds faster than the non‐dominant hand, which makes stopping the initial GO response a bit more demanding (although this aspect is minimized due to the applied staircase procedure). The reason why the other hand was used for CHANGE stimuli is that it makes the CHANGE more demanding.

Visual and Auditory Experiments

All participants completed a visual and a visual‐auditory version of the SC paradigm. In the visual version of the experiment, the CHANGE stimuli were bold yellow bars presented for 200 ms replacing one of the three white lines, thus turning it into the new reference line.

In the visual‐auditory version of the experiment, a sine tone (200 ms duration), which was presented via headphones to both ears, was used as the CHANGE stimulus. In each SC trial, one of three differently pitched tones (low/500 Hz, middle/900 Hz, and high/1200 Hz) was presented at a 75 dB sound pressure level. Prior to the testing we ensured that each of the different pitches could be discriminated with at least 95% accuracy. Importantly, this included that not only the pitch could be distinguished but that also the association with the different lines of the visual array (i.e., the bordered rectangle which contained four vertically arranged circles with white borders, which were separated by three white horizontal lines) as outlined below. This required a training of the association in adults and children.

Each tone represented one of the three horizontal lines (i.e., the high 1,200 Hz tone represented the high reference line, the middle 900 Hz tone represented the middle reference line, and the low 500 Hz tone represented the low reference line). Irrespective of the modality in which the CHANGE stimulus was presented, participants were asked to indicate the spatial relation of the target stimulus to the reference line indicated by the respective CHANGE stimulus. Importantly, the auditory CHANGE required the subjects to take both visual information and auditory information into account to perform a correct response. For example, a filled white circle right below the middle line required an “above” judgment when the CHANGE stimulus was the low tone (500 Hz), but a “below” judgment whenever the CHANGE stimulus was the high tone (1,200 Hz). This should illustrate that neither information alone (i.e., auditory or visual) was sufficient to come up with the right response. Instead, both the visual and the auditory information need to be integrated to reach a valid decision which response to execute. Importantly, the visual GO stimulus was still displayed, when the auditory CHANGE stimulus was presented. The concept of the temporal window of integration refers to the window in which concurrent multisensory information can be integrated into one percept or can exert a crossmodal influence. This is the case in the paradigm applied.

Due to ethical considerations, the procedure employed for the children was slightly modified: first, the children were given 576 trials instead of the 864 administered to the adults in order do to decrease fatigue and ensure high task compliance. Second, instead of four vertically arranged circles with white border, the children's task version only comprised two vertically arranged circles with white border (as shown in the Fig. 1B). These circles were separated by three horizontal white lines as done in the adults' task version. While this reduced the complexity of the visual array, both groups received identical instructions and virtually performed the same task as all of them had to spatially relate the target stimulus to one of three reference lines (above vs. below—judgement).

EEG Recording and Analysis

A QuickAmp amplifier (Brain Products, Inc.) connected to 60 sintered Ag/AgCl ring electrodes located at equidistant scalp positions (customized BrainCap Fast‘n Easy sub‐inion model EEG caps) using the 10/10 system nomenclature was used to acquire a high‐density EEG recording. Fpz was set as the reference electrode. All electrode impedances were kept below 5 kΩ and the data were recorded with an initial sampling rate of 500 Hz. It was later (offline) down‐sampled to 256 Hz before applying an IIR band‐pass filter ranging from 0.5 to 20 Hz (with a slope of 48 db/oct each) using the BrainVision Analyzer 2 software package. Next, technical as well as rare motor artifacts such as sneezing or jaw‐clenching were removed during a manual raw data inspection. Additionally, an independent component analysis (ICA) using the Infomax algorithm was applied to remove the regular/recurring artifacts like eye blinks or saccades. The number of ICs removed varied between 3 and 7 (mean = 4.1 ± 2). Then, one more manual raw data inspection was applied to remove any residual artifacts. Subsequently, single‐trial segments locked to the onset of the visual STOP stimulus were formed for the two SCD conditions [compare to Mückschel et al. 2014]. Each of the segments started −800 ms before the onset of the STOP stimulus (set to time point zero) and ended 1,000 ms thereafter. An automated artifact rejection procedure was applied to these segmented data. The rejection criteria were a value difference of more than 150 μV in a 250 ms interval, activity below 0.1 μV in a 200 ms interval, or a maximum voltage step of >60 μV/ms. The artifact rejection procedure eliminated ∼1.5% (± 0.9) of trials no differing between groups and experimental conditions. To eliminate the reference potential, a current source density (CSD, (order of splines = 4, maximum degree of Legendre polynomials = 10, default lambda = 1 e –5) transformation was applied [Nunez and Pilgreen 1991]. The CSD works as a spatial filter [Nunez and Pilgreen 1991], which reduces the impact of volume conduction. This helps identify the electrodes that best reflect neurophysiological (EEG) activity related to specific cognitive processes. Furthermore, the CSD interpolation reduces artifacts of opposite polarity which may otherwise be observed around the electrodes reflecting large ERPs. A baseline correction was then set to the time window from −800 to −700 ms to obtain a prestimulus baseline. Lastly, the single trial segments were individually averaged for each SC condition. Based on this procedure, the P1, N1, and P3 ERPs' mean amplitudes were quantified.

Electrodes and components were chosen based on a visual inspection of the scalp topography of the grand average over participants and conditions. Based on this, the visual P1 and N1 ERPs (i.e., mean amplitudes) on the STOP stimulus were quantified at electrodes P7 and P8 (P1:110–130 ms and N1: 190–230 ms) in the visual‐auditory experiment and the visual‐visual experiment. The auditory N1 (i.e., mean amplitudes) on the CHANGE stimulus (in case of the visual‐auditory experiment) was quantified at electrodes C5 and C6 (SCD0: 200–230 ms, SCD300: 420–510 ms). The visual N1 on the CHANGE stimulus (i.e. mean amplitudes) (in case of the visual‐visual experiment) was quantified at electrodes P7 and P8 (SCD0: 190–230 ms, SCD300: 420–510 ms). The P3 (i.e., mean amplitudes) was quantified at electrode Cz (SCD0: 270–330 ms, SCD300: 540–580 ms) in the visual‐auditory experiment and the visual‐visual experiment. This choice of electrodes and time intervals used to quantify the ERP was validated by a procedure described in Mückschel et al. [2014]: In this procedure, the above time intervals were taken and the mean amplitude within the defined search intervals was determined for each of the 60 electrode positions. This was performed only after CSD transformation of the data which emphasizes scalp topography [Nunez and Pilgreen 1991]. Then, to compare each electrode against an average of all other electrodes, Bonferroni correction for multiple comparisons (critical threshold, P = 0.0007) was used. Only electrodes, which displayed significantly larger mean amplitudes (i.e., negative for the N‐ potentials and positive for the P‐potentials) when compared to other electrodes were chosen. This procedure revealed the same electrodes as previously chosen by visual inspection. The quantification of all ERPs was made at the single subject level. Latencies are given relative to the onset of the stop signal (time point 0), and amplitudes were quantified relative to the pre‐stimulus baseline.

To obtain an estimate about the reliability of the neurophysiological data in the groups and the experimental conditions, we calculate the signal‐to‐noise (SNR) as implemented in the Brain Vision Analyzer II software package (BrainProducts Inc.).

sLORETA

For source localization analyses, standardized low resolution brain electromagnetic tomography (sLORETA) [Pascual‐Marqui 2002] was used, which provides a single solution to the inverse problem [Marco‐Pallarés, 2005; Pascual‐Marqui, 2002; Sekihara et al., 2005]. For sLORETA, the intracerebral volume is partitioned into 6,239 voxels at 5 mm spatial resolution. Then, the standardized current density at each voxel is calculated in a realistic head based on the MNI152 template [Mazziotta et al. 2001]. It has been mathematically proven that sLORETA provides reliable results without a localization bias [Sekihara et al. 2005]. Moreover, there is evidence from EEG/fMRI and neuronavigated EEG/TMS studies underlining the validity of the sources estimated using sLORETA [Dippel and Beste 2015; Sekihara et al. 2005]. The voxel‐based sLORETA images were compared between groups and experimental conditions using the sLORETA‐built‐in voxel‐wise randomization tests with 2,000 permutations, based on statistical nonparametric mapping (SnPM). Voxels with significant differences (P < .01, corrected for multiple comparisons) between contrasted conditions and groups were located in the MNI‐brain.

Statistics

To analyze behavioral and ERP data, separate mixed effects analyses of variance (ANOVAs) were used. The factors “condition” (SCD0 vs. SCD300) and “stimulus modality” (unimodal vs. bimodal SC stimuli) were used as within‐subject factors. “Group” (children vs. adults) was used as a between‐subjects factor. All reported values underwent Greenhouse‐Geisser correction and Bonferroni correction, whenever necessary. Kolmogorov–Smirnov tests indicated that all variables used for the analysis were normally distributed (all z < 0.5; P > .4). For all descriptive statistics, the standard error of the mean (SEM) was used as a measure of variability.

RESULTS

Behavioral Data

Reaction time data

The analysis of reaction times (RTs) in GO trials using the “group” as the between‐subject factor and “stimulus modality” as the within‐subject factor revealed no main effect of “stimulus modality” (F(1,57) = 1.96; P = .167) or “group” (F(1,57) = 2.71; P = .105). There was also no interaction “Stimulus modality × group” (F(1,57) = 2.16; P = .147). The mean RT was 459ms (± 25) and the mean SSD was 260 ms (±32). Both parameters are well‐comparable to the study by Verbruggen et al. [2008] introducing the applied paradigm.

Concerning the CHANGE RTs in SC trials, a mixed effects ANOVA using the within‐subject factors “SCD interval” and “stimulus modality” and the between‐subject factor “group” revealed a main effect of “SCD interval” (F(1,57) = 310.40; P < .001; η _p ² = .845) indicating that RTs were longer in SCD0 trials (929 ms ± 21) than in SCD300 trials (755 ms ± 25). Also, there was a main effect of “group” (F(1,57) = 54.15; P < .001; η _p ² = .487) showing that children were generally slower (1013 ms ± 31) than adults (672 ms ± 33). There was a main effect of “stimulus modality” (F(1,57) = 80.15; P < .001; η _p ² = .584) indicating that RTs were longer when bimodal stimuli (i.e., auditory CHANGE stimuli) were presented (972 ms ± 27.01) compared to the unimodal stimulus presentation (i.e. visual CHANGE stimuli; 713 ms ± 27.66). No interaction effects were significant (all F < 1.2; P > .2). The SSRT did not differ between groups for the unimodal and bimodal stimulus modality (F(1,57) = 2.10; P = .152; η _p ² = .036). No other interactions were significant (all F < 2.23; P > .141).

Accuracy data

To account for the trial difference between the groups in the experimental design, accuracy was measured using the percentage of correct responses. The analysis of GO trials revealed no main or interaction effects (all F < 2.7 P > .15). The accuracy to STOP the response cannot differ because the staircase procedure was applied to assess SSRTs. Yet, the accuracy can be modulated for the response on the CHANGE stimuli. There, the main effect of “SCD interval” (F(1,57) = 1012.24; P < .001; η _p ² = .947) showed that the percentage of correct responses were higher in the SCD300 condition (85.3% ± 1) than in the SCD0 condition (58.6% ± 0.6). There was also a main effect of “stimulus modality” (F(1,57) = 11.77; P = .001; η _p ² = .171) indicating that there were more correct responses for unimodal STOP and CHANGE stimuli (74.1% ± 0.7) than for bimodal STOP and CHANGE stimuli (69.9% ± 1.2). The main effect of group (F(1,57) = 29.05; P < .001; η _p ² = .338) revealed that adults responded more correctly (75.9% ± 1.1) than children (68.1% ± 1.0). There were interactions of “stimulus modality × group” (F(1,57) = 8.29; P = .006; η _p ² = .127), “SCD interval × group” (F(1,57) = 20.93; P < .001; η _p ² = .269) and “stimulus modality × SCD interval” (F(1,57) = 54.75; P < .001; η _p ² = .49), but an interaction of “stimulus modality × SCD interval × group” was also found (F(1,57) = 5.84; P = .019; η _p ² = .093) (Fig. 2). Post‐hoc independent samples t‐tests revealed that in the SCD300 condition, the accuracy difference between the unimodal and the bimodal experiment versions was higher in children (14.5% ± 2.8) than in adults (4.1 ± 1.5) (t57 = −3.15; P = .003). No such difference was evident in the SCD0 condition (t57 = −0.69; P = .213).

Figure 2.

Illustration of the interaction “stimulus modality × SCD interval × group” obtained for the accuracy data (rate of correct hits). The upper plot shows the rate of correct hits for the different conditions (visual‐visual and visual‐auditory) in adults (black squares) and children (white circles) for the SCD0 condition. The plot at the bottom shows the rate of correct hits for the different conditions (visual‐visual and visual‐auditory) in adults (black squares) and children (white circles) for the SCD300 condition. The mean and standard error of the mean are given.

Neurophysiological Data

P1 and N1

The P1 and N1 ERPs at electrodes P7 and P8 are shown in Figure 3. The P1 amplitudes were analyzed in a mixed effects ANOVA using the factors “SCD interval” and “stimulus modality” as within‐subject factors and “group” as between‐subject factor. The factor “electrode” was not modelled because the auditory N1 could only be quantified in the bimodal task version so that any effect of “electrode” would have been be confounded by the stimulus modalities.

Figure 3.

Event‐related potentials (ERPs) showing the P1 and N1 ERP components pooled for electrodes P7 and P8 (upper two plots) for the adult and children group and the SCD0 and SCD300 condition. Time point 0 denotes the time point of STOP signal presentation. The baseline is shown from −800 ms to −600 ms before STOP signal presentation. The scalp topography maps show the scalp topography at the maximum of each respective ERP component. Blue colors denote negativity, red color positivity. The first plot shows the visual P1 and N1 in the visual‐visual condition, the second plot in the visual‐auditory condition. The third plot at the bottom show the P1 and N1 for the auditory change stimuli pooled for electrode C5 and C6. Note the x‐axis is truncated at 800 ms post STOP stimulus presentation. [Color figure can be viewed at http://wileyonlinelibrary.com]

The main effect of “SCD interval” (F(1,57) = 30.171; P < .001; η _p ² = .346) showed that the P1 amplitude was larger in the SCD0 (46.8 μV/m² ± 3.2) than in the SCD300 (41.5 μV/m² ± 3.0) condition. The main effect of “stimulus modality” (F(1,57) = 4.85; P = .032; η _p ² = .078) showed that the P1 amplitude was larger for bimodal stimuli (47 μV/m² ± 3.8) than for unimodal stimuli (41.4 μV/m² ± 2.8). The main effect of “group” (F(1,57) = 31.15; P < .001; η _p ² = .353) showed that the P1 amplitude was larger in children (61.25 μV/m² ± 4.22) compared to adults (27.1 μV/m² ± 4.5). There was an interaction of “SCD interval” × “stimulus modality” (F(1,57) = 12.34; P = .001; η _p ² = .178). Post‐hoc independent samples t‐tests revealed that the P1 ERP differed between unimodal and bimodal stimuli in the SCD0 condition (t57 = 3.52; P = .001) where the mean P1 amplitude was larger for bimodal stimuli (52.9 μV/m² ± 4.6) compares to unimodal stimuli (42.5 μV/m² ± 3.68). This difference could not be found in the SCD300 condition (t57 = 0.37; P = .713). There were no interactions with the factor “group” (all F < 9.26; P > .23).

In the N1 ERP, there was main effect of “SCD interval” (F(1,57) = 65.51; P < .001; η _p ² = .535) showing that the N1 amplitude was larger (i.e., more negative) in the SCD0 condition (–39.7 μV/m² ± 2.4) than in the SCD300 condition (–20.4 μV/m² ± 1.8). The main effect of “stimulus modality” (F(1,57) = 27.96; P < .001; η _p ² = .329) showed that the N1 amplitude was larger for bimodal stimuli (–38.8 μV/m² ± 2.2) than for unimodal stimuli (–21.2 μV/m² ± 2.6). The main effect of “group” (F(1,57) = 13.67; P < .001; η _p ² = .193) showed that the N1 amplitude was larger in children (–36.5 μV/m² ± 2.4) compared to adults (–23.6 μV/m² ± 2.5). There was an interaction of “stimulus modality” × “group” (F(1,57) = 14.48; P < .001; η _p ² = .33). Post‐hoc independent samples t‐tests revealed that N1 amplitudes differed between the groups in case of bimodal stimuli (t57 = 5.88; P < .001) where the mean N1 amplitude was larger in children (–51.6 ± 3.8) compared to adults (–26 ± 1.9). This was not the case for unimodal stimuli (t57 = .06; P = .952). There was an interaction of “SCD interval” × “stimulus modality” (F(1,57) = 20.36; P < .001; η _p ² = .263). Post‐hoc independent samples t‐tests revealed that N1 amplitudes differed between unimodal and bimodal stimuli in the SCD300 condition (t57 = −6.27; P < .001), where the mean N1 amplitude was larger for bimodal stimuli (–36 ± 2.9) compared to unimodal stimuli (–5.3 ± 3.3). This was not the case in the SCD0 condition (t57 = −1.34; P = .185). There were no other interactions with the factor “group” (all F < 0.71; P > .3).

The analysis of the SNR‐data revealed no main or interaction effects (all F < 0.95; P > .4), showing that the results obtained are not biased by the SNR level.

To examine whether the slow potentials prior to the STOP stimulus onset bias the results, we quantified the mean amplitude of these potentials in the time period between −500 ms and −100 ms and used these values as covariates in the analyses. None of these analyses revealed an effect of these covariates (all F < 0.5; P > .4) and also the main or interaction effects remained the same.

P3

The P3 at electrode Cz is shown in Figure 4. For the P3 amplitudes, the main effect of “SCD interval” (F(1,57) = 69.44; P < .001; η _p ² = .549) showed that the P3 amplitude was larger in the SCD0 condition (57 μV/m² ± 4.2) than in the SCD300 condition (36.8 μV/m² ± 2.6). The main effect of “stimulus modality” (F(1,57) = 20.73; P < .001; η _p ² = .267) showed that the P3 amplitude was larger for bimodal stimuli (53.99 μV/m² ± 4.04) than for unimodal stimuli (39.76 μV/m² ± 3.17). The main effect of “group” (F(1,57) = 15.34; p< .001; η _p ² = .212) showed that the P3 amplitude was larger in children (59.7 μV/m² ± 4.5) compared to adults (34 μV/m² ± 4.8). There was also an interaction of “stimulus modality × SCD interval × group” (F(1,57) = 4.55; P = .037; η _p ² = .074), which is shown in Figure 4. Post‐hoc independent samples t‐tests revealed that in the SCD300 condition, the P3 amplitude was significantly modulated by groups and modality (t57 = −2.05; P = .044). The mean P3 amplitude difference between unimodal and bimodal stimuli (i.e., bimodal minus unimodal) was higher in children (13.71 μV/m²± 5.13) than in adults (0.17 μV/m²± 3.95). There was no such group difference in the SCD0 condition (t57 = 1.20; P = .233). For the SCD300 condition, the sLORETA analysis compared the difference between unimodal and bimodal stimuli in children to the difference between unimodal and bimodal stimuli in adults. This comparison revealed modulations in the anterior cingulate cortex (ACC; BA24) and the inferior parietal cortex including the temporo‐parietal junction (BA40). In these two areas, the depicted activation is larger in children than in adults, showing that the activation difference between unimodal and bimodal stimuli was larger in children than adults.

Figure 4.

Event‐related potentials (ERPs) showing the P3 ERP components at electrode Cz for the children and the adult group. Time point 0 denotes the time point of STOP signal presentation. The baseline is shown from −800 ms to −600 ms before STOP signal presentation. The upper row shows the P3 ERP component in the SCD0 condition (left) and the descriptive values for the P3 ERP component in the different experimental conditions and age groups (right). The lower row shows the P3 ERP component in the SCD300 condition (left) and the descriptive values for the P3 ERP component in the different experimental conditions and age groups (right). The sLORETA plots show the sources in the medial frontal cortex (BA24) and the left inferior parietal cortex (BA40), corrected for multiple comparisons. The sLORETA color scale denotes critical t‐values. The scalp topography maps show the scalp topography at the maximum of each respective ERP component. Blue colors denote negativity, red color positivity. [Color figure can be viewed at http://wileyonlinelibrary.com]

The analysis of the SNR‐data revealed no main or interaction effects (all F < 1.11; P > .3), showing that the results obtained are unbiased with respect to the SNR.

To examine whether the slow potentials prior to STOP stimulus onset biased the results, we quantified the mean amplitude of these potentials in the time window from −500 ms to −100 ms and used these values as covariates in the analyses. None of these analyses revealed an effect of these covariates (all F < 0.2; P > .8) and also the main or interaction effects remained the same.

DISCUSSION

In this study, we examined whether developmental changes in the time window, within which we are able to perform multisensory and sensorimotor information integration, could explain some of the age‐related modulations of multicomponent behavior.

We predicted that if there was a significant contribution of the critical time window to developmental changes in multicomponent behavior, its effects should be limited to cases of multimodal stimulus presentation (as compared to unimodal stimulus presentation) and should furthermore be more pronounced in case of a temporal gap (as compared to simultaneous input). To investigate this question, we manipulated stimulus input modalities (uni‐ vs bimodal) and temporal stimulus spacing (SCD0/simultaneous presentation vs. SCD300/delayed stimulus presentation) in a SC task which was used to assess multicomponent behavior in children and adults.

While children seemed to generally perform worse than adults, we also found a distinct interaction of age group, the number of sensory modalities, and the temporal spacing of stimuli. This supports our a priori hypothesis that some of the age‐related differences in multicomponent behavior can actually be attributed to developmental changes in the critical time window underlying multisensory and sensorimotor integration processes. For both behavioral and neurophysiological measures, we found that children were worse in integrating multisensory information only when there was a temporal gap between the onset of two task‐relevant stimuli (i.e., in the SCD300 condition). More precisely, children revealed a larger decline in accuracy (percentage of correct responses) than adults when STOP and CHANGE stimuli were presented in different sensory modalities. Since there were no differential effects in the reaction time data and children performed worse than adults with respect to both accuracy and hit RTs, a speed‐accuracy‐trade‐off can be ruled out. While the main effect of group in hit RTs suggests that there might still be a developmental effect on executive functioning/multicomponent behavior, the accuracy data clearly suggests that developmental changes in the critical time window underlying multisensory information integration are an important factor which may strongly contribute to age‐dependent differences in multicomponent behavior. This interpretation was underlined by the ERP and source localization results, showing very specific effects:

The P1 and N1 ERP components showed no effects reflecting the interaction between age group, number of involved sensory modalities, and the temporal spacing of stimuli requiring different actions. This suggests that the automatic, bottom‐up guided allocation of perceptual and attentional resources [Brisson et al. 2007; Buzzell et al. 2013; Luck et al. 2000], as well as top‐down guided discrimination process selectively allocating attention to relevant stimulus features [Hopf et al. 2002; Luck et al. 1990; Vogel and Luck 2000] are not modulated by developmental changes in the critical integration time window. Previous studies have also shown that these mechanisms are less important for multicomponent behavior [Gohil et al. 2015; Mückschel et al. 2016]. The main effects observed for the P1 and N1 mainly reflect known multisensory enhancement effects described before [Gohil et al. 2015], as well as phase resetting that may influence early sensory processing between modalities [Lakatos et al. 2007]. One might of course argue that the age group differences in P1 amplitudes might also potentially reflect some developmental differences in the attentional resources invested in multicomponent behavior and therefore be connected to the slower responses in the children. Yet, it cannot be fully excluded that the augmentation in P1 amplitudes seen in the children was partially caused by the changes in stimulus array size and complexity, which was needed to allow the children to properly understand and execute the task.

Importantly, and like the accuracy data, the P3 amplitude reflected interactive effects between age group, number of involved sensory modalities, and the temporal spacing of stimuli requiring different actions. Processes reflected by the P3 ERP have previously been shown to determine performance in multicomponent behavior [Mückschel et al. 2014; Dippel and Beste 2015]. As such, the results suggest that processes crucial for multicomponent behavior undergo developmental changes between childhood and adulthood. Even though several interpretation on the functional relevance of the P3 have been put forward [Polich 2007], the P3 has been suggested to reflect decision or selection mechanisms between stimulus evaluation and responding [Twomey et al. 2015; Verleger et al. 2005], or the allocation of processing resources [Polich 2007]. The finding that children had stronger P3 amplitude modulations between unimodal vs. bimodal stimulus input in the SCD300 condition than adults suggests that response selection mechanisms are more prone to be influenced by the complexity of sensory integration processes during multicomponent behavior than during preceding attentional processes (refer P1 and N1 effects). In adults, these mechanisms seem to be more stable. This difference could be explained by the fact that adults are typically endowed with longer critical time intervals within which multisensory and sensorimotor information integration may take place. Hence, adults should have been better able to integrate the multimodal stimulus input in the simultaneous SCD0 condition as well as the temporally spaced SCD300 condition because their critical window should have allowed them to process both types of input rather similarly. In contrast to this, children's critical time windows for information integration are typically shorter so that they should have had no major problems in integrating information in the simultaneous SCD0 condition but should have experienced grater difficulties than adults in integrating information in the temporally spaced SCD300 condition. As a result, they displayed larger input differences only in the SCD300 condition. It is therefore likely that the timing properties of bimodal stimuli signaling STOP and CHANGE processes in the SCD300 condition fall out of the temporal window of integration making response selection processes more difficult with the outcome that performance declines. Since the temporal window of integration is only important for multisensory stimuli [Conrey and Pisoni 2006; Diederich and Colonius 2009; van Wassenhove et al. 2007], it is irrelevant (i.e., should have no effect) in the unimodal condition [Gori et al. 2008; Hahn et al. 2014; Hillock‐Dunn et al. 2016; ; Nardini et al. 2008]. Interestingly, the sLORETA analysis shows that modulations in the P3 were related to activation differences in the inferior parietal cortex including the temporo‐parietal junction (TPJ, BA40) and the anterior cingulate cortex (BA24). The anterior cingulate and medial frontal regions are involved in conflict monitoring and response selection processes [Rushworth 2008; Rushworth et al. 2007], and, most likely due to the hub‐like connectivity pattern of this cortical structure [Vogt 2016], are modulated by multisensory stimuli. The TPJ has previously been reported to be related to modulations in the P3 component [Verleger et al. 1994], including the paradigm applied in this study [Mückschel et al. 2014]. The TPJ has furthermore been shown to be involved in multisensory processing [Ionta et al. 2011; Jakobs et al. 2012; Parker Jones et al. 2014], but has also been shown to play in role in the chaining of actions during response selection [Chersi et al. 2011; Karch et al. 2010], as it sustains executive control [Collette et al. 2005]. It is possible that because the chaining of actions in the SCD300 condition and the necessary integration of bimodal stimuli both demand processing resources of probably the same functional neuroanatomical structures, these may rather easily become overstrained in children. It may be speculated that this is not the case in adults, due to their completed brain maturation. In line with this, results by Gogtay et al. [2004] show that parietal areas, together with prefrontal areas, mature late. As to the effects observed in the anterior cingulate cortex (ACC), the same mechanisms are possible. At this point it needs to be noted that the nature of cues being used to trigger cognitive flexibility processes can have profound effects on age‐related (developmental) differences in performance between children and adults [Chavelier, 2015]. Several studies show that “transparent cues” (i.e., that are not abstract and easy to translate into a task goal) improve performance in task‐switching mechanisms [e.g., Blaye and Chavelier, 2011; Chavelier and Blaye, 2009; Chavelier et al., 2011], which are similar to the processes induced by the unimodal or bimodal CHANGE stimuli. In the bimodal condition, this study used non‐transparent cues (i.e., abstract pitches in the bimodal condition), while the cues were transparent in the unimodal condition (i.e., each reference line was clearly marked). Importantly, the relation of abstract (non‐transparent) auditory cues to the different reference lines of the visual stimulus was trained in children and adults (refer methods section), which minimizes this possible confound of that study. It is well‐known that training of cues can increase otherwise deficient task performance [Chavelier et al., 2014]. Nevertheless, future studies shall examine in how far sensory processes that are shown in this study to modulate differences in multicomponent behavior between children and adults are further subjects to effects of cue transparency. This is important, since goal setting and activation processes play a major in role in the processes examined [Mückschel et al., 2014; Verbruggen et al., 2008] and are strongly modulated by developmental effects [Chavelier and Blaye, 2009].

The cross‐sectional design is a limitation of this study, and future studies may use longitudinal study designs to rigorously address the developmental aspects behind interactive effects of multimodal stimulus input and the temporal spacing of stimuli requiring different actions during multicomponent behavior. These studies should also use a similar number of trials in the different age group, which is also a limitation of the study. In this regard, it may also be interesting to examine aging processes in later stages of life as well. This is because ageing has been shown to affect multicomponent behavior, but only when there is no temporal spacing between the different stimuli [Stock et al. 2016]. It therefore seems that there may be different developmental trajectories having specific effects on the framework of multicomponent behavior. The study has important clinical implications. Occupational therapists are always looking at the exact questions raised in this study; that is, what conditions lead to declines in behavioral control. This study suggests that the number of sensory modalities, which need to be integrated, is an important aspect to consider concerning multicomponent behavior which is highly relevant for daily life functioning. Especially for patients with attention‐deficit hyperactivity disorder, it may thus be useful to determined individual “thresholds” beyond which these patients lose control over multicomponent behavior in the face of different sensory information.

In summary, the study shows that differences in how children and adults accomplish multicomponent behavior do not solely depend on developmental differences in response selection processes per se. Instead, age differences in multicomponent behavior could be shown to largely depend on the complexity arising from the requirement to integrate temporally separated stimuli from different modalities during response selection processes as reflected by the P3 ERP. These differences were due to modulations of activity in medial frontal and inferior parietal cortices. Primary perceptual gating or attentional selection processes (P1 and N1 ERPs) were not affected. The results show that differences in multisensory integration explain parts of transformations in cognitive processes between childhood and adulthood that have traditionally been attributed to changes in executive functioning, especially when these require the integration of multiple modalities during response selection.