Skip to main content
NCBI home page
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2013 Aug 19.
Published in final edited form as: J Alzheimers Dis. 2012;32(2):317–328. doi: 10.3233/JAD-2012-111070

Delayed Audiovisual Integration of Patients with Mild Cognitive Impairment and Alzheimer’s Disease Compared with Normal Aged Controls

Jinglong Wu a,*, Jiajia Yang a, Yinghua Yu a, Qi Li b, Naoya Nakamura a, Yong Shen c,e, Yasuyuki Ohta d, Shengyuan Yu f,*, Koji Abe d
PMCID: PMC3746512  NIHMSID: NIHMS465110  PMID: 22810093

Abstract

The human brain can anatomically combine task-relevant information from different sensory pathways to form a unified perception; this process is called multisensory integration. The aim of the present study was to test whether the multisensory integration abilities of patients with mild cognitive impairment (MCI) and Alzheimer’s disease (AD) differed from those of normal aged controls (NC). A total of 64 subjects were divided into three groups: NC individuals (n = 24), MCI patients (n = 19), and probable AD patients (n = 21). All of the subjects were asked to perform three separate audiovisual integration tasks and were instructed to press the response key associated with the auditory, visual, or audiovisual stimuli in the three tasks. The accuracy and response time (RT) of each task were measured, and the RTs were analyzed using cumulative distribution functions to observe the audiovisual integration. Our results suggest that the mean RT of patients with AD was significantly longer than those of patients with MCI and NC individuals. Interestingly, we found that patients with both MCI and AD exhibited adequate audiovisual integration, and a greater peak (time bin with the highest percentage of benefit) and broader temporal window (time duration of benefit) of multisensory enhancement were observed. However, the onset time and peak benefit of audiovisual integration in MCI and AD patients occurred significantly later than did those of the NC. This finding indicates that the cognitive functional deficits of patients with MCI and AD contribute to the differences in performance enhancements of audiovisual integration compared with NC.

Keywords: Alzheimer’s disease, audiovisual integration, cognitive functional deficits, mild cognitive impairment, performance enhancement

INTRODUCTION

Alzheimer’s disease (AD) is one of the most severe degenerative diseases in older people. Patients with AD show a multiplicity of cognitive deficits in such domains as memory, language, and attention [1–3]. Amnestic mild cognitive impairment (MCI) is a clinical disorder that afflicts elderly individuals and is characterized by isolated memory impairment that is more severe than that of normal aging, while other cognitive functions remain normal [4–6]. MCI is a transitional zone between age-related memory loss and AD and is a major risk factor for the development of AD [7]. Recently, many MCI and AD studies have focused on the memory-related deficits associated with this disease [3, 8–10]. These studies have suggested that the best cognitive predictor of the development of dementia is low memory performance [8]. Although memory deficits are the salient feature of AD, several other cognitive functional deficits, such as those in multi-sensory integration, are also considered to influence the everyday lives of patients with AD.

Humans are constantly bombarded with information from multiple sensory organs. For instance, when driving a car, we are surrounded by visual (e.g., road, roadside billboard), auditory (e.g., car engine, music), and somatosensory (e.g., feeling the steering wheel) information. To focus on relevant information and ignore what is irrelevant, the human brain is equipped with a selection mechanism executed by the cognitive function of attention. The attention system allows us to dynamically select and enhance the processing of those objects and events that are most relevant at each moment. The brain can then combine task-relevant information from anatomically different sensory pathways to form a unified perception, known as multisensory integration (see [11] for review).

Many recent studies on multisensory audiovisual integration have investigated healthy subjects, using behavioral, event-related potential (ERP), and neuroimaging measures [12–18]. The behavioral results from these studies have shown that responses to audiovisual stimuli are more rapid and accurate than responses to either a unimodal visual or auditory stimulus [12, 14–16]. Some studies have found audiovisual interaction effects not only in sensory-specific visual and auditory cortices but also in nonspecific cortices [12–15]. Moreover, our recent studies [19, 20] also indicated that responses to audiovisual targets were faster than responses to unimodal visual or auditory targets, and the ERPs measured during responses to the multisensory targets differed from those of the unimodal targets.

Aging is a condition characterized by a general decline in many types of physical and psychological performance. Recently, a neuropsychological study examined the speed of discrimination responses to the presentation of visual, auditory or combined audiovisual stimuli in aged and young individuals [21, 22]. All of the subjects performed with high accuracy, but the elderly subjects were significantly slower than the young subjects. However, although a great deal is known about audiovisual integration in younger and older adults, whether audiovisual integration in patients with MCI and AD differs from older adults remains unclear.

The aims of the present study were to investigate whether MCI and AD patients exhibit audiovisual integration differently compared with aged-matched normal control (NC) individuals and to identify the different patterns among the three groups (MCI, AD, and NC). Three subject groups (NC, MCI, and AD) performed audiovisual integration tasks. We recorded the response times (RT) and accuracy for all the tasks, and cumulative probability distributions (CDF) [23, 24] were used to the compare RTs across conditions to investigate enhancements in audiovisual integration. The main finding of the present study was that patients with both MCI and AD showed clear evidence of audiovisual integration, but the onset times and peaks of the performance enhancements in these subjects were later than those of the NC individuals.

MATERIALS AND METHODS

Subjects

Three groups of right-handed subjects (NC, MCI, and AD) consented to participate in this study. All of the MCI and AD patients were recruited from the Okayama University Hospital, Japan (21 patients) and Chinese PLA General Hospital (19 patients). Demographic information on the subjects is provided in Table 1. All of the subjects possessed normal or corrected-to-normal vision and normal hearing capability. No subjects had remarkable findings in motor and sensory systems or deep tendon reflexes. The experimental protocol was approved by the ethics committee of Okayama University and Chinese PLA General Hospital.

Table 1.

Demographic information

NC MCI AD
Sample size, no. 24 19 21
Male/female 9/15 10/9 9/12
Age 70.8 ± 1.2 74.4 ± 0.6 76.3 ± 1.9
Education (years) 13.4 ± 0.5 11.1 ± 0.1 11.2 ± 0.5
CDR score (out of 3) 0 ± 0 0.5 ± 0** 1.5 ± 0.5**
MMSE score (out of 30) 29.0 ± 0.2 28.0 ± 0.8 21.3 ± 1.0**
Open in a new tab

Data are presented as the means ± standard error of the mean (SEM).

**

Statistically significant compared with all other groups (p < 0.01).

AD, Alzheimer’s disease; CDR, Clinical Dementia Rating; MCI, mild cognitive impairment; MMSE, Mini-Mental State Examination; NC, normal controls.

The NC group consisted of 24 subjects between the ages of 59 and 81. The NC individuals were designated as “cognitively normal” when they presented no cognitively based limitations in daily living activities. NC individuals were defined by a Mini-Mental State Examination (MMSE) score [25] of 27 or greater and a Clinical Dementia Rating (CDR) [26] of 0. None of the NC subjects had any history of neurological or psychiatric disease, and none of them was taking any medication that affected the central nervous system at the time of testing.

The MCI group consisted of 19 subjects between the ages of 56 and 87. Patients with amnestic MCI were diagnosed using the Petersen criteria [7]. In addition, all patients with amnestic MCI had MMSE scores of 24 or greater and CDR scale scores of 0.5. These individuals underwent magnetic resonance imaging (MRI) of the brain to confirm that they did not have a focal lesion affecting memory-sensitive substrates (the medial thalamus, septal/hypothalamic region, or medial temporal lobe). As assessed by the cognitive battery of the Consortium to Establish a Registry for Alzheimer’s Disease [27], MCI patients typically show memory performances that are 1.5 reference deviations below the age-adjusted average. The areas tested by the battery include verbal learning, recognition and recall tests, global cognitive function, and impairment of daily living.

The AD group consisted of 21 subjects between the ages of 58 and 86. The diagnosis of AD was made in accordance with the criteria of the National Institute of Neurological and Communicative Disorders and Stroke (NINCDS) and the Alzheimer’s Disease and Related Disorders Association (ADRDA) [27]. All patients with AD had MMSE scores of between 15 and 26 and a CDR score of 1 or 2, which corresponded to what is known as mild-to-moderate AD.

In addition, all MCI and AD patients had a confirmed Rosen-modified Hachinski Ischemic Score of at least 4 [28]. Patients were excluded if they had a clinically significant neurological disease other than MCI or AD or a major psychiatric disorder. Psychiatric co-morbidity was excluded by history, clinical examination, and a Composite International Diagnostic Interview [29].

Stimuli and tasks

The experiment contained three stimulus types, including unimodal visual (V) stimuli, unimodal auditory (A) stimuli, and bimodal audiovisual (AV) stimuli. The unimodal V_stimuli included a checkerboard subtending at a 5-degree visual angle that was presented against a black background. These V_stimuli were presented unilaterally to lateral locations on either the left or the right of the display at a 12-degree visual angle below 5 degrees in the vertical direction relative to the fixation point in the horizontal direction (see Fig. 1). The duration of the stimulus was 150 ms. The unimodal A_stimuli consisted of 1,600-Hz tones with linear rise and fall times of 5 ms, intensities of 65 dB, and durations of 150 ms. These A_stimuli were presented through headphones. The bimodal AV_stimuli consisted of a combination of the unimodal auditory and visual stimuli. Presenting the visual and auditory stimuli simultaneously created the subjective impression of a single bimodal audiovisual object.

Fig. 1.

Fig. 1

Open in a new tab

An example showing the time sequence and stimuli. The default setting of the visual screen consisted of a fixation cross and a task illustration at the center (i.e., an image of two eyes for visual selective attention, an image of two ears for auditory selective attention and an image of one eye and one ear for audiovisual divided attention). The visual stimulus included a checkerboard subtending at a 5-degree visual angle presented against a black background. The stimulus was presented unilaterally to lateral locations on either the left or the right side of the display at a 12-degree visual angle below 5 degrees in the vertical direction relative to the fixation point in the horizontal direction. The auditory stimulus was a 1,600-Hz tone with an intensity of 65 dB and was presented via headphones. The subjects were asked to keep their eyes focused on the fixation cross and to covertly direct their attention to a designated subset of presented objects. Each subject was required to press a button with their right or left hand to respond to the stimulus on the corresponding side.

Each subject participated in three separate sessions, and the subjects were given three types of attention instructions for each session. However, the subjects were instructed to keep their eyes focused on the fixation cross and to covertly direct their attention to a designated subset of presented objects in all cases. In the auditory selective attention session, the subjects were instructed to pay attention to the unimodal A_stimuli and only the auditory components of the bimodal AV_stimuli. In the visual selective attention session, the subjects were instructed to pay attention to the unimodal V_stimuli and only the visual component of the bimodal AV_stimuli. In the audiovisual divided attention session, the subjects were instructed to pay attention to all visual, auditory, and audiovisual stimuli. In all sessions, each subject was required to press a button with the left index finger when identifying a stimulus on the left side and a button with the right index finger when identifying a stimulus on the right side. Moreover, all three sessions were conducted in the same day with an adequate rest (i.e., from 10 min to 30 min).

The interstimulus interval (ISI) of the stimuli varied randomly from 3,000 to 4,000 ms (the mean ISI was 3,500 ms) (Fig. 1). In each session, 72 unimodal V_stimuli, 72 unimodal A_stimuli, and 72 bimodal AV_stimuli were presented. Of each set of 72 stimuli, 36 were presented on the left side, and 36 were presented on the right side. All stimuli were randomly presented in each session.

Data processing and analysis

The mean accuracy was computed for each subject under each condition. RT data were first analyzed to remove outlying data points (incorrect responses were not eliminated), which were defined as responses occurring faster than 200 ms or slower than 2 standard deviations from the mean RT for each subject. Differences in accuracy and RT of the three subject groups were analyzed using a repeated measures analysis of variance (ANOVA). The level of significance was fixed at p < 0.05. The Bonferroni test (α = 0.05) was performed to detect differences between the subject groups.

To control for the redundant nature of the multisensory condition, RTs were analyzed using CDFs, and multisensory data were compared with statistical facilitation using a CDF of the total probability of the visual and auditory responses. This model is often referred to as the independent race model [23, 24]. This model allows for a comparison of the multisensory condition with the joint probability of the unimodal conditions. Each subject’s unimodal CDFs were then used to calculate the race distribution, using the following formula at each time bin: (P[A] + P[V]) − (P[A] × P[V]). In this formula, P(A) is the probability of responding within a given time in a unimodal auditory trial, and P(V) is the probability of responding within a given time in a unimodal visual trial. If the probability of response to the multisensory stimulus is significantly greater than that predicted by the summed probability of the unimodal stimuli, it is said to violate the race model, a result that likely suggests neural integration of the two unimodal inputs [23, 24]. To perform this analysis, each subject’s data were processed to generate subject-specific CDFs for each condition using 10-ms time bins. Next, the CDFs from all the subjects in each group were averaged to generate group CDFs. This analysis procedure ensured that the results reflected a group pattern and were not skewed by one or a few outlying subjects. As was done for each of the sensory conditions, the race model CDF was generated for each subject and averaged at each time bin to generate a group CDF.

Significant deviations from the race model were determined by subtracting the predicted summed probability from the multisensory probability at each time bin for each subject, thereby creating a difference curve for each subject. A one-sample t-test was performed at each time bin within each group to compare this difference curve with zero, and significant (p < 0.05) deviations were then identified. All statistical analyses were performed using SPSS version 12.0 j software (SPSS, Tokyo, Japan).

RESULTS

Although the subjects were recruited from Japan and China (for detail, see subjects section), both the accuracies and RTs showed the similar trend as shown in Tables 2 and 3. Thus, the following mean accuracy and RT included the subjects’ data from Japan and China for each group (i.e., NC, MCI, and AD).

Table 2.

Mean accuracies and response times of the subjects from Japan

Experimental conditions Accuracy %
RTs ms
NC MCI AD NC MCI AD
Auditoiy Attention
 A_Stimuli 99.4 (0.2) 99.1 (0.4) 93.3 (1.8) 466.5 (12.8) 518.7 (16.1) 623.1 (35.1)
 AV_Stimuli 98.4 (0.6) 97.9 (0.7) 97.0 (1.2) 460.5 (14.1) 494.8 (15.5) 623.3 (41.4)
Visual Attention
 V_Stimuli 96.4 (1.1) 95.8 (1.3) 93.5 (1.6) 471.1 (10.3) 510.9 (24.1) 588.6 (34.1)
 AV_Stimuli 98.6 (0.4) 98.5 (0.5) 97.2 (0.9) 431.7 (9.7) 452.3 (17.6) 542.6 (29.7)
Divided Attention
 A_Stimuli 98.6 (0.6) 97.5 (1.1) 92.9 (1.8) 480.4 (8.5) 510.6 (16.6) 631.6 (41.4)
 V_Stimuli 96.7 (0.9) 93.8 (1.5) 95.9 (0.9) 454.9 (10.5) 481.6 (13.8) 594.7 (41.4)
 AVStimuli 98.2 (0.5) 98.3 (0.6) 94.8 (1.4) 387.7 (7.6) 411.9 (11.8) 508.9 (33.1)
Open in a new tab

Data are presented as the means and SEM. AD, Alzheimer’s disease; MCI, mild cognitive impairment; NC, normal controls; RT, response times.

Table 3.

Mean accuracies and response times of the subjects from China

Experimental conditions Accuracy %
RTs ms
NC MCI AD NC MCI AD
Auditoiy attention
 A_Stimuli 94.2 (0.9) 90.8 (2.4) 84.4 (2.6) 501.8 (10.6) 451.9 (17.9) 575.9 (38.1)
 AV_Stimuli 95.5 (0.6) 90.0 (1.6) 85.8 (3.3) 489.5 (11.4) 440.9 (16.4) 537.2 (42.9)
Visual attention
 V_Stimuli 94.4 (0.9) 91.2 (1.5) 85.5 (3.3) 496.6 (16.9) 497.9 (20.6) 627.5 (72.4)
 AV_Stimuli 94.4 (1.1) 90.3 (1.9) 88.8 (2.3) 449.1 (12.1) 443.6 (16.8) 530.5 (41.2)
Divided attention
 A_Stimuli 93.5 (0.6) 85.1 (3.8) 83.7 (2.7) 526.6 (24.2) 470.2 (29.8) 574.6 (40.8)
 V_Stimuli 91.9 (2.3) 87.9 (2.9) 82.6 (5.2) 499.0 (16.1) 477.4 (19.1) 673.6 (69.7)
 AV_Stimuli 95.6 (0.6) 88.9 (3.6) 87.8 (3.5) 431.3 (15.4) 394.6 (15.6) 484.0 (38.1)
Open in a new tab

Data are presented as the means and SEM. AD, Alzheimer’s disease; MCI, mild cognitive impairment; NC, normal controls; RT, response times.

Accuracy

The mean accuracy was greater than 80% under the visual, auditory, and multisensory conditions for all of the subject groups, as shown in Fig. 2a–c. We performed a group (three groups) × condition (seven conditions) repeated measures ANOVA on the mean accuracies. We did not find a significant main effect of condition [F(6, 366) = 1.21, p > 0.05] and did not find a significant interaction between group and condition [F(12, 366) = 1.24, p = 0.254]. In contrast, the analysis revealed a significant main group effect [F(2, 61) = 31.00, p < 0.001]. A multiple comparison using the Bonferroni correction (α = 0.05) revealed that the mean accuracy of the AD group was significantly lower than those of the MCI (p = 0.05) and NC groups (p = 0.002) under auditory selective attention with A_stimuli (see Fig. 2a). In addition, the same group differences were found in the audiovisual divided attention session with A_stimuli (p < 0.05 for all), as shown in Fig. 2c. However, there were no significant group differences under the other conditions (p > 0.05 for all).

Fig. 2.

Fig. 2

Open in a new tab

Mean response times (RTs) and accuracy of (a) auditory attention, (b) visual attention and (c) divided attention conditions for each stimulus type. The bar graphs represent the mean RTs, and the line graphs represent the mean accuracy under each condition. The RTs are presented as means ± SEM. NC, normal aged control; MCI, mild cognitive impairment; AD, Alzheimer’s disease; A, auditory; V, visual; AV, audiovisual; *p < 0.05; **p < 0.01; ***p < 0.001.

Response time

The mean RTs for all the conditions are also presented in Fig. 2. We performed a group (three groups) × condition (seven conditions) repeated measures ANOVA and a multiple comparison using the Bonferroni correction (α = 0.05) on the mean RTs. The repeated measures ANOVA revealed significant primary effects of mean RTs for both conditions [F(6, 336) = 19.19, p < 0.001] and groups [F(2, 61) = 8.71, p < 0.001]. However, we did not find a significant interaction between group and condition [F(12, 336) = 0.55, p = 0.88]. These results indicate that, under both divided and visual selective attention conditions, the RTs for the bimodal AV_stimuli were significantly faster (p < 0.01 for all) than the RTs for the unimodal V_stimuli or A_stimuli (Fig. 2b, c). The multiple comparison revealed no significant differences between the RTs for the bimodal AV_stimuli and unimodal A_stimuli under the auditory selective attention conditions (see Fig. 2a; p > 0.05 for all). In addition, the analysis revealed that the mean RTs of the AD group were significantly longer than those of the MCI group (the differences ranged from 92 to 137 ms; p < 0.05 for all) and the NC group (the differences ranged from 98 to 143 ms; p < 0.05 for all) under all conditions (Fig. 2). However, there were no significant differences between the MCI and NC groups under any conditions (p > 0.05 for all).

Cumulative distribution function

The race model was used to analyze the RTs to determine if the observed bimodal AV_stimuli behavioral enhancements were greater than those predicted by the statistical summation for the unimodal V_stimuli and A_stimuli. This analysis is able to determine if the multisensory response is faster than predicted by statistical facilitation associated with redundant sensory stimuli. To account for the increased probability of faster responses to the bimodal AV_stimuli, the distributions of the bimodal responses were compared with the race model, a summed probability of the unimodal responses. As shown in Fig. 3a, the responses to the bimodal stimuli under divided attention condition was compared with the summed probability of response to the unimodal auditory and visual stimuli (race model distribution) under the divided attention condition of the NC group. Moreover, divided attention condition contains both an auditory and a visual component, while the visual and auditory selective attention conditions contain only one of these components. Therefore, we used responses to the bimodal stimuli under auditory selective attention and visual selective attention conditions to calculate a selective attention race model distribution of the NC group (Fig. 3b). Furthermore, the difference in response probability between the bimodal trials and the race model under all conditions for the NC group are shown in Fig. 3c. We performed the same comparison for the MCI (Fig. 4a–c) and AD (Fig. 5a–c) groups.

Fig. 3.

Fig. 3

Open in a new tab

Cumulative distribution functions (CDFs) for responses in the unimodal A_stimuli and V_stimuli and bimodal AV_stimuli trials under divided and selective attention conditions in the NC group. a) For trials with divided attention, the CDFs for the response times are depicted for the unimodal auditory stimuli (A_stimuli (AV)), unimodal visual stimuli (V_stimuli (AV)), and bimodal audiovisual stimuli (AV_stimuli (AV)). The response probabilities predicted by summing the unimodal response probabilities (race model) are depicted by the thick, solid curve. Note that faster response times were more likely for the bimodal AV_stimuli than predicted by the race model in several time bins. b) CDFs are depicted for the response times to the unimodal auditory stimuli (A_stimuli (A)) and bimodal audiovisual stimuli (AV_stimuli (A)) under auditory selective attention conditions and for the unimodal visual (V_stimuli (V)) and bimodal audiovisual stimuli (AV_stimuli (V)) under visual selective attention conditions. The thick, solid curve depicts the race model predictions based on the total probability of the unimodal responses under selective attention conditions. c) The differences in response probability between the bimodal trials and the race model predictions under divided attention, auditory selective attention and visual selective attention conditions illustrate that bimodal audiovisual enhancements are present under the divided attention and visual selective attention conditions but not the auditory selective attention conditions. The gray fields represent the SEM.

Fig. 4.

Fig. 4

Open in a new tab

Cumulative distribution functions (CDFs) for responses to the unimodal A_stimuli and V_stimuli and bimodal AV_stimuli trials under divided and selective attention conditions in the MCI group. a) Trials under divided attention conditions. b) Trials under visual and auditory selective attention conditions in the MCI group. c) The differences in response probability between the bimodal trials and race model predictions under divided attention and auditory selective attention and visual selective attention conditions in the MCI group. The gray fields represent the SEM.

Fig. 5.

Fig. 5

Open in a new tab

Cumulative distribution functions (CDFs) for the responses to the unimodal A_stimuli and V_stimuli and the bimodal AV_stimuli trials under divided and selective attention conditions in the AD group. a) Trials under divided attention conditions. b) Trials under visual and auditory selective attention conditions in the AD group. c) The differences in response probability between the bimodal trials and the race model predictions under divided attention, auditory selective attention and visual selective attention conditions in the AD group. The gray fields represent the SEM.

The positive deflections in the difference curves shown in Figs. 3c, 4c, and 5c for each subject group reflect the time bins in which the observed responses to the bimodal stimuli were faster than those predicted by the correspondence race model. One-sample -tests t were performed across these distributions to determine whether these multisensory enhancements were significantly above zero. Significant enhancements were found for all subject groups (NC, MCI, and AD) under the visual selective attention and divided attention conditions but not the auditory selective attention condition. Table 4 details the statistical significance of the values for the three subject groups under all experimental conditions.

Table 4.

Temporal representation of statistical significance (p values) for all groups

graphic file with name nihms465110f6.jpg
Open in a new tab

This table represents the statistical significance of a one-sample t-test at each time bin within each group to compare the CDF curves to zero, as shown in Figs. 3c, 4c and 5c. The gray and black blocks show the different statistical significance values. The dotted arrows represent the onset time bin. The *represents the time bin in which peak behavioral enhancements were observed for each subject group. NC, normal aged control; MCI, mild cognitive impairment; AD, Alzheimer’s disease; A, auditory; V, visual; AV, audiovisual; n.s., no significance; RT, response time.

As shown in Table 4, the NC subjects showed evidence of multisensory integration under the divided attention and visual selective attention conditions. This result was seen from 290 to 570 ms (with a peak of 22.01% at 390 ms) under the divided attention conditions and from 320 to 390 ms (with a peak of 4.11% at 340 ms) under the visual selective attention conditions following the stimulus onset. Evidence of multisensory integration in MCI subjects was seen from 310 to 580 ms (with a peak of 12.81% at 400 ms) and from 340 to 380 ms (with a peak of 4.12% at 350 ms) under the divided attention and visual selective attention conditions, respectively. The AD subjects showed evidence of multisensory integration from 330 to 590 ms (with a peak of 14.35% at 430 ms) and from 340 to 440 ms (with a peak of 4.36% at 430 ms) under the divided attention and visual selective attention conditions, respectively.

DISCUSSION

The present work contrasted audiovisual integration performance under different attention conditions in MCI and AD patients and normal aged controls (NC). These results indicated that the mean response time of the patients with AD was significantly longer than that of the patients with MCI and that of the NC individuals (see Fig. 2) under all attention conditions. In contrast, we found significant group differences in mean accuracy under the auditory selective attention and audiovisual divided attention conditions but not the visual selective attention conditions (see Fig. 2). Moreover, an analysis using CDFs revealed that the probability of response to the multisensory stimuli was significantly greater than that predicted by the total probability of the unimodal stimuli for all subject groups, which is consistent with our previous studies in healthy younger [19, 20] and older subjects [21]. However, we found that both the time window and the peak of the performance enhancements for the multi-sensory stimuli were different between the NC, MCI, and AD groups (see Table 4).

Accuracy and response time

Although the audiovisual integration task was designed to focus on response times to measure behavioral enhancement under audiovisual divided attention conditions, accuracies were also evaluated. Consistent with previous findings [21, 31], we did not find a significant difference in accuracy between different experimental conditions for each group (i.e., NC, MCI, and AD). The mean accuracies in the present study for the NC, MCI, and AD groups were greater than 90% for the audiovisual integration tasks and we also found no clearly differences of accuracy among the three groups. These results are understandable in the context of the human characteristics of decision making. A previous study [44] indicated that the decision making process involves a trade-off between the quality of an outcome (accuracy) and the response time at which that outcome is received. Because the subjects were instructed to press response key as accurately and quickly as possible when the stimulus was presented in this study, they were considered to put more emphasis on the accuracy. Therefore, no significant differences of accuracy were found among different experimental conditions. Even the previous study (see [30] for review) indicated that patients with both MCI and AD experienced attention impairment compared with normal aging, the impairments in attention associated with AD did not affect the accuracy of the present study.

In contrast, as shown in Fig. 2, the present results indicate that, under both divided and visual selective attention conditions, the RTs for the bimodal AV_stimuli were significantly faster than those for the unimodal V_stimuli and A_stimuli in the NC, MCI and AD groups. It is possible that the increase in response time observed in the multisensory trials was due to the association of both the auditory and visual stimuli with the same response choice during the multisensory trials [19, 20]. However, we did not find any significant decreases in response time for the bimodal AV_stimuli trials compared with those for the unimodal A_stimuli trials under the auditory selective attention conditions, as demonstrated in the previous study [31]. Although the patients with AD demonstrated the same speeding effect as the NC individuals, the RTs of the AD group were significantly longer than those of the MCI and NC groups under all conditions (see Fig. 2). According to previous studies [2, 32, 33], the RT performance of patients with AD is slowed by deficits in cognitive function including decision making compared with NC and MCI individuals. Therefore, our results suggest that the abnormal cognitive function associated with AD contributed to increased RTs under all conditions compared with NC and MCI.

Behavioral enhancements

The use of CDFs to evaluate the race model ensures that multisensory enhancement is identified using the entire RT distribution rather than a single central tendency score (i.e., the mean RT), which is more susceptible to bias associated with small sample sizes and skewed distributions [23, 24]. In the present study, comparisons with the race model revealed that all the subjects exhibited greater peaks and broader temporal windows of multisensory enhancement under the visual selective attention and divided attention conditions (see Table 4 and Figs. 3–5). These results suggest that, despite the decline in sensory processing that accompanies patients with MCI and AD, a multisensory benefit during the selected time window was observed. The following two points were thought to be responsible for the multisensory enhancement of the subjects with MCI and AD.

First, the visual and auditory stimuli presented occurred simultaneously during the bimodal AV_stimuli trials (for details, see above). Previous studies [34–36] have demonstrated that audiovisual enhancement only occurred when both visual and auditory events co-occurred in time and space. The strong audiovisual integration effect was obtained when the time window between the onset of auditory and visual events was less than 100 ms [34], and this integration effect reached a maximum when the auditory and visual events co-occurred temporally. Moreover, a previous EEG study [45] also recorded the brain activity when audiovisual events were presented with event onset asynchronies ranging from −125 to +125 ms. They found that the effects elicited by simultaneously presented audiovisual events were observed over both medial-frontal and occipital brain regions, and demonstrated that early audiovisual multisensory processing is highly sensitive to the relative onset timing of the auditory and visual component inputs. Therefore, the experimental design used in the present study was used to induce strong audiovisual integration for all the subjects.

Second, many cortical and subcortical brain areas are involved in multisensory integration. For instance, previous studies have demonstrated that audiovisual integration activated a number of brain areas, including early cortical areas, such as the primary visual and auditory cortices [37–39], higher cortical areas, such as the superior temporal sulcus and intra-parietal areas [38, 39] and also subcortical areas, such as the superior colliculus [37]. Moreover, both MCI and AD patients presented with varying degrees of cognitive impairment and cerebral atrophy [40], and there was a strong correlation between the severity of the cognitive impairment and cerebral atrophy. The mean MMSE scores of the MCI and AD groups in the present study were higher than 28 and 24, respectively. Therefore, despite the mild cerebral atrophy occurring in the MCI and AD patients from the present study, most parts of the neural pathway correlates of multisensory integration were activated when audiovisual stimuli were presented.

However, the group difference in onset time and peak of the performance enhancements must be considered. As shown in Table 4, the onset times and peak benefit of the NC group were faster than those of the MCI and AD groups. In addition, we observed that both the onset time and peak benefit exhibited by the MCI group were faster than those of the AD group. Although the mechanisms underlying the enhanced audiovisual performance in patients with MCI and AD remain unknown, the cerebral atrophy observed in MCI and AD individuals is a likely factor affecting onset time and peak benefit. For example, some neuroimaging studies [41–43] have revealed that abnormalities in the frontal, temporal, and parietal cortices contribute to the functional deficits in AD patients. The functional deficits in patients with MCI and AD appear to induce their late response for both the unimodal (i.e., auditory or visual) and bimodal (i.e., audiovisual) stimuli compared with NC individuals. Moreover, this late response in AD compared with MCI may correspond with the varying degree of cerebral atrophy in AD patients. Consequently, our results suggest that the functional deficits related to the cerebral atrophy observed in patients with AD contribute to the late onset time and peak performance enhancement in AD patients compared with MCI patients and NC individuals.

Acknowledgments

This study was partly supported by a Grant-in-Aid for Scientific Research (B) 21404002, Japan and by the AA Science Platform Program of the Japan Society for the Promotion of Science.

Footnotes

Authors’ disclosures available online (http://www.j-alz.com/disclosures/view.php?id=1396).

References

  • 1.Förstl H, Kurz A. Clinical features of Alzheimer’s disease. Eur Arch Psychiatry Clin Neurosci. 1999;249:288–290. doi: 10.1007/s004060050101. [DOI] [PubMed] [Google Scholar]
  • 2.Carlesimo GA, Oscar-Berman M. Memory deficits in Alzheimer’s patients: A comprehensive review. Neuropsychol Rev. 1992;3:119–169. doi: 10.1007/BF01108841. [DOI] [PubMed] [Google Scholar]
  • 3.McKhann G, Drachman D, Folstein M, Katzman R, Price D, Stadlan EM. Clinical diagnosis of Alzheimer’s disease: Report of the NINCDS-ADRDA work group under the auspices of Department of Health and Human Services task force on Alzheimer’s disease. Neurology. 1984;34:939–344. doi: 10.1212/wnl.34.7.939. [DOI] [PubMed] [Google Scholar]
  • 4.Petersen RC, Smith GE, Waring SC, Ivnik RJ, Tangalos EG, Kokmen E. Mild cognitive impairment: Clinical characterization and outcome. Arch Neurol. 1999;56:303–308. doi: 10.1001/archneur.56.3.303. [DOI] [PubMed] [Google Scholar]
  • 5.Reite M, Teale P, Zimmerman J, Davis K, Whalen J. Source location of a 50 msec latency auditory evoked field component. Electroencephalogr Clin Neurophysiol. 1988;70:490–498. doi: 10.1016/0013-4694(88)90147-2. [DOI] [PubMed] [Google Scholar]
  • 6.Siedenberg R, Goodin DS, Aminoff MJ, Rowley HA, Roberts TP. Comparison of late components in simultaneously recorded event-related electrical potentials and event-related magnetic fields. Electroencephalogr Clin Neurophysiol. 1996;99:191–197. doi: 10.1016/0013-4694(96)95215-3. [DOI] [PubMed] [Google Scholar]
  • 7.Petersen RC. Mild cognitive impairment as a diagnostic entity. J Intern Med. 2004;256:183–194. doi: 10.1111/j.1365-2796.2004.01388.x. [DOI] [PubMed] [Google Scholar]
  • 8.Elias MF, Beiser A, Wolf PA, Au R, White RF, D’Agostino RB. The preclinical phase of Alzheimer disease. A 22-year prospective study of the Framingham Cohort. Arch Neurol. 2000;57:808–813. doi: 10.1001/archneur.57.6.808. [DOI] [PubMed] [Google Scholar]
  • 9.Small BJ, Fratiglioni L, Viitanen M, Winblad B, Backman L. The course of cognitive impairment in preclinical Alzheimer disease. Three- and 6-year follow-up of a population-based sample. Arch Neurol. 2000;57:839–844. doi: 10.1001/archneur.57.6.839. [DOI] [PubMed] [Google Scholar]
  • 10.Welsh K, Butters N, Hughes J, Mohs R, Heyman A. Detection of abnormal memory decline in mild cases of Alzheimer’s disease using CERAD neuropsychological measures. Arch Neurol. 1991;48:278–281. doi: 10.1001/archneur.1991.00530150046016. [DOI] [PubMed] [Google Scholar]
  • 11.Talsma D, Senkowski D, Soto-Faraco S, Woldorff MG. The multifaceted interplay between attention and multisensory integration. Trends Cognit Sci. 2010;14:400–410. doi: 10.1016/j.tics.2010.06.008. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Giard MH, Peronnet F. Auditory-visual integration during multimodal object recognition in humans: A behavioral and electrophysiological study. J Cogn Neurosci. 1999;11:473–490. doi: 10.1162/089892999563544. [DOI] [PubMed] [Google Scholar]
  • 13.Fort A, Delpuech C, Pernier J, Giard MH. Early auditory-visual interactions in human cortex during nonredundant target identification. Brain Res Cogn Brain Res. 2002;14:20–30. doi: 10.1016/s0926-6410(02)00058-7. [DOI] [PubMed] [Google Scholar]
  • 14.Molholm S, Ritter W, Murray MM, Javitt DC, Schroeder CE, Foxe JJ. Multisensory auditory-visual interactions during early sensory processing in humans: A high-density electrical mapping study. Brain Res Cogn Brain Res. 2002;14:115–128. doi: 10.1016/s0926-6410(02)00066-6. [DOI] [PubMed] [Google Scholar]
  • 15.Teder-Salejarvi WA, McDonald JJ, Di Russo F, Hillyard SA. An analysis of audio-visual crossmodal integration by means of event-related potential (ERP) recordings. Brain Res Cogn Brain Res. 2002;14:106–114. doi: 10.1016/s0926-6410(02)00065-4. [DOI] [PubMed] [Google Scholar]
  • 16.Teder-Salejarvi WA, Di Russo F, McDonald JJ, Hillyard SA. Effects of spatial congruity on audio-visual multimodal integration. J Cogn Neurosci. 2005;17:1396–1409. doi: 10.1162/0898929054985383. [DOI] [PubMed] [Google Scholar]
  • 17.Vidal J, Giard MH, Roux S, Barthelemy C, Bruneau N. Cross-modal processing of auditory-visual stimuli in a no-task paradigm: A topographic event-related potential study. Clin Neurophysiol. 2008;119:763–771. doi: 10.1016/j.clinph.2007.11.178. [DOI] [PubMed] [Google Scholar]
  • 18.Szycik GR, Jansma H, Münte TF. Audiovisual integration during speech comprehension: An fMRI study comparing ROI-based and whole brain analyses. Hum Brain Mapp. 2009;30:1990–1999. doi: 10.1002/hbm.20640. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Wu J, Li Q, Bai O, Touge T. Multisensory interactions elicited by audiovisual stimuli presented peripherally in a visual attention task: A behavioral and event-related potential study in humans. J Clin Neurophysiol. 2009;26:407–413. doi: 10.1097/WNP.0b013e3181c298b1. [DOI] [PubMed] [Google Scholar]
  • 20.Li Q, Wu J, Touge T. Audiovisual interaction enhances auditory detection in late stage: An ERP study. Neuroreport. 2010;21 :173–178. doi: 10.1097/WNR.0b013e3283345f08. [DOI] [PubMed] [Google Scholar]
  • 21.Laurienti PJ, Burdette JH, Maldjian JA, Wallace MT. Enhanced multisensory integration in older adults. Neurobiol Aging. 2006;27:1155–1163. doi: 10.1016/j.neurobiolaging.2005.05.024. [DOI] [PubMed] [Google Scholar]
  • 22.Hugenschmidt CE, Mozolic JL, Laurienti PJ. Suppression of multisensory integration by modality-specific attention in aging. Neuroreport. 2009;20:349–353. doi: 10.1097/WNR.0b013e328323ab07. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Miller J. Divided attention: Evidence for coactivation with redundant signals. Cognit Psychol. 1982;14:247–279. doi: 10.1016/0010-0285(82)90010-x. [DOI] [PubMed] [Google Scholar]
  • 24.Miller J. Time course of coactivation in bimodal divided attention. Percept Psychophys. 1986;40:331–343. doi: 10.3758/bf03203025. [DOI] [PubMed] [Google Scholar]
  • 25.Folstein MF, Folstein SE, McHugh PR. ‘Mini-mental state’ A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12:189–198. doi: 10.1016/0022-3956(75)90026-6. [DOI] [PubMed] [Google Scholar]
  • 26.Hughes CP, Berg L, Danziger WL, Coben LA, Martin RL. A new clinical scale for the staging of dementia. Br J Psychiatry. 1982;140:566–572. doi: 10.1192/bjp.140.6.566. [DOI] [PubMed] [Google Scholar]
  • 27.Morris JC, Heyman A, Mohs RC, Hughes JP, van Belle G, Fillenbaum G, Mellits ED, Clark C The Consortium to Establish a Registry for Alzheimer’s Disease (CERAD) Part I. Clinical and neuropsychological assessment of Alzheimer’s disease. Neurology. 1989;39:1159–1165. doi: 10.1212/wnl.39.9.1159. [DOI] [PubMed] [Google Scholar]
  • 28.Rosen WG, Terry RD, Fuld PA, Katzman R, Peck A. Pathological verification of ischemic score in differentiation of dementias. Ann Neurol. 1980;7:486–488. doi: 10.1002/ana.410070516. [DOI] [PubMed] [Google Scholar]
  • 29.Robins LN, Wing J, Wittchen HU, Helzer JE, Babor TF, Burke J, Farmer A, Jablenski A, Pickens R, Regier DA, Sartorius N, Towle LH. The Composite International Diagnostic Interview. An epidemiologic Instrument suitable for use in conjunction with different diagnostic systems and in different cultures. Arch Gen Psychiatry. 1988;45:1069–1177. doi: 10.1001/archpsyc.1988.01800360017003. [DOI] [PubMed] [Google Scholar]
  • 30.Perry RJ, Hodges JR. Attention and executive deficits in Alzheimer’s disease. A critical review. Brain. 1999;122:383–404. doi: 10.1093/brain/122.3.383. [DOI] [PubMed] [Google Scholar]
  • 31.Mozolic JL, Hugenschmidt CE, Peiffer AM, Laurienti PJ. Modality-specific selective attention attenuates multisensory integration. Exp Brain Res. 2008;184:39–52. doi: 10.1007/s00221-007-1080-3. [DOI] [PubMed] [Google Scholar]
  • 32.Baddeley AD, Bressi S, Della Sala S, Logie R, Spinnler H. The decline of working memory in Alzheimer’s disease: A longitudinal study. Brain. 1991;114:2521–2542. doi: 10.1093/brain/114.6.2521. [DOI] [PubMed] [Google Scholar]
  • 33.Bäckman L, Small BJ. Cognitive deficits in preclinical Alzheimer’s disease and vascular dementia: Patterns of findings from the Kungsholmen Project. Physiol Behav. 2007;92:80–86. doi: 10.1016/j.physbeh.2007.05.014. [DOI] [PubMed] [Google Scholar]
  • 34.Frassinetti F, Bolognini N, Ladavas E. Enhancement of visual perception by crossmodal visuo-auditory interaction. Exp Brain Res. 2002;147:332–343. doi: 10.1007/s00221-002-1262-y. [DOI] [PubMed] [Google Scholar]
  • 35.Ernst MO, Bülthoff HH. Merging the senses into a robust percept. Trends Cogn Sci. 2004;8:162–169. doi: 10.1016/j.tics.2004.02.002. [DOI] [PubMed] [Google Scholar]
  • 36.Meredith MA, Nemitz JW, Stein BE. Determinants of multisensory integration in superior colliculus neurons 1. Temporal factors. J Neurosci. 1987;7:3215–3229. doi: 10.1523/JNEUROSCI.07-10-03215.1987. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Calvert GA, Thesen T. Multisensory integration: Methodological approaches and emerging principles in the human brain. J Physiol Paris. 2004;98:191–205. doi: 10.1016/j.jphysparis.2004.03.018. [DOI] [PubMed] [Google Scholar]
  • 38.Lewis JW, Van Essen DC. Corticocortical connections of visual, sensorimotor, and multimodal processing areas in the parietal lobe of the macaque monkey. J Comp Neurol. 2000;428:112–137. doi: 10.1002/1096-9861(20001204)428:1<112::aid-cne8>3.0.co;2-9. [DOI] [PubMed] [Google Scholar]
  • 39.Linden JF, Grunewald A, Andersen RA. Responses to auditory stimuli in macaque lateral intraparietal area II. Behavioral modulation. J Neurophysiol. 1999;82:343–358. doi: 10.1152/jn.1999.82.1.343. [DOI] [PubMed] [Google Scholar]
  • 40.Mouton PR, Martin LJ, Calhoun ME, Dal Forno G, Price DL. Cognitive decline strongly correlates with cortical atrophy in Alzheimer’s dementia. Neurobiol Aging. 1998;19:371–377. doi: 10.1016/s0197-4580(98)00080-3. [DOI] [PubMed] [Google Scholar]
  • 41.Huang S, Li J, Sun L, Ye J, Fleisher A, Wu T, Chen K, Reiman E Alzheimer’s Disease NeuroImaging Initiative. Learning brain connectivity of Alzheimer’s disease by sparse inverse covariance estimation. Neuroimage. 2010;50:935–949. doi: 10.1016/j.neuroimage.2009.12.120. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Delbeuck X, Van der Linden M, Collette F. Alzheimer’s disease as a disconnection syndrome? Neuropsychol Rev. 2003;13:79–92. doi: 10.1023/a:1023832305702. [DOI] [PubMed] [Google Scholar]
  • 43.Dickerson BC, Sperling RA. Large-scale functional brain network abnormalities in Alzheimer’s disease: Insights from functional neuroimaging. Behav Neurol. 2009;21:63–75. doi: 10.3233/BEN-2009-0227. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 44.Kable JW, Glimcher PW. An “as soon as possible” effect in human intertemporal decision making: Behavioral evidence and neural mechanisms. J Neurophysiol. 2010;103:2513–2531. doi: 10.1152/jn.00177.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Senkowski D, Talsma D, Grigutsch M, Herrmann CS, Woldorff MG. Good times for multisensory integration: Effects of the precision of temporal synchrony as revealed by gamma-band oscillations. Neuropsychologia. 2007;45:561–571. doi: 10.1016/j.neuropsychologia.2006.01.013. [DOI] [PubMed] [Google Scholar]

RESOURCES