Neural activities of tactile crossmodal working memory in humans: an event-related potential study

Abstract

In the present study, we examined the neural mechanisms underlying crossmodal working memory by analyzing scalp-recorded event-related potentials (ERPs) from normal human subjects performing tactile-tactile unimodal or tactile-auditory crossmodal delay tasks that consisted of stimulus-1 (S-1, tactile), interval (delay), and stimulus-2 (S-2, tactile or auditory). We hypothesized that there are sequentially discrete task-correlated changes in ERPs representing neural processes of tactile working memory, and in addition, significant differences would be observed in ERPs between the unimodal task and the crossmodal task.

In comparison to the ERP components in the unimodal task, two late positive ERP components (LPC-1 and LPC-2) evoked by the tactile S-1 in the delay of the crossmodal task were enhanced by expectation of the associated auditory S-2 presented at the end of the delay. Such enhancement might represent neural activities involved in crossmodal association between the tactile stimulus and the auditory stimulus. Later in the delay, a late negative component (LNC) was observed. The amplitude of LNC depended on information retained during the delay, and when the same information was retained, this amplitude was not influenced by modality or location of S-2 (auditory S-2 through headphones, or tactile S-2 on the left index finger). LNC might represent the neural activity involved in working memory. The above results suggest that the sequential ERP changes in the present study represent temporally distinguishable neural processes, such as the crossmodal association and crossmodal working memory.

Introduction

Memory is thought to be represented by changes in the same neural systems that perceive, analyze, and process sensory information (Squire 1986; Thompson 1986; Squire and Zola-Morgan, 1991; Zola-Morgan and Squire, 1993; Kosslyn et al., 1995; Gilbert 1998; Kosslyn et al., 2001; Super et al., 2001). In working memory, sensory information is temporarily stored, manipulated, and maintained in the neural system for later action (Baddeley 1992; Smith and Jonides, 1999; Fuster 1995).

Studies on monkey tactile working memory showed that in the somatosensory system, activities of neurons in somatosensory cortex (primary and secondary, SI and SII) and the posterior parietal cortex (PPC) were correlated with performance of tactile-tactile unimodal working memory tasks (e.g., Koch and Fuster, 1989; Zhou and Fuster, 1996; Bodner et al., 1997; Salinas et al., 2000; Bodner et al., 2005). In those studies, the neural activity of tactile working memory was investigated through analysis of single-unit data recorded in awake monkeys performing delayed matching-to-sample tasks in which a tactile sample stimulus was first presented, and the animal had to memorize the sample throughout the delay period until the tactile matching test stimulus was presented for choice. Many neurons recorded in the tasks showed sustained activation (increase in firing rate above baseline) during the delay period. These neurons, “memory cells”, were thought to be neural substrates of working memory (Fuster 1995).

A recent human working memory study (Harris et al., 2002) showed that in performance of a tactile unimodal working memory task, SI cortex retained a memory trace of the tactile stimulus for a short period. In a trial of the task, by comparing frequencies of two tactile vibrations separated by a retention interval of 1,500 ms, subjects indicated at the end of the trial if the frequency of the second vibration was higher or lower. A pulse of transcranial magnetic stimulation (TMS) delivered in the retention interval at 300 or 600 ms to the SI cortex contralateral to the stimulated hand significantly disrupted task performance. TMS did not affect performance if delivered to the ipsilateral SI at any time point. The researchers proposed that the tactile memory trace was held initially for a certain period of time in both SI and SII, but later it would be held in higher level of cortex, such as the premotor and prefrontal cortex (Romo et al., 1999; Brody et al., 2003; Passingham and Sakai, 2004; Romo et al., 2004; Hernandez et al., 2002). A similar conclusion was drawn by a recent human fMRI study on haptic working memory (Kaas et al., 2007). Mounting evidence in human studies also shows that PPC plays an important role in tactile working memory (e.g., Burton and Sinclair, 2000; Stoeckel et al., 2003; Ricciardi et al., 2006)).

Further, the neural activity in somatosensory cortex was shown to respond to task-related stimuli of more than one sensory modality in working memory tasks (Zhou and Fuster, 2000; Zhou and Fuster, 2004), or was modulated by non-tactile stimuli that were associated with the tactile stimulus in the task (Schaefer et al., 2006; Taylor-Clarke et al., 2002; Ku et al., 2007). In our previous study (Ohara et al., 2006), we recorded scalp event-related potentials (ERPs) in normal human subjects performing tactile-tactile unimodal delay tasks, or tactile-visual crossmodal delay tasks. We observed sequential changes in ERPs during the delay of tactile-visual crossmodal working memory tasks. Two ERP components (late positive components) in the earlier period of the delay were related to association between two stimuli (stimulus-1, S-1, and stimulus-2, S-2) that were separated by the delay. An ERP component (late negative component, LNC) shown later in the delay was related to working memory. When the same information defined by the tactile S-1 was retained in working memory, the LNC showed similar changes irrespective of which sensory modalities (tactile or visual) of S-2 (go-signals) were expected at the end of the delay period.

In the present study, we further examined sequentially discrete task-correlated changes in ERPs by recording them in tactile-auditory working memory tasks. We tested two hypotheses: first, that in the earlier period of the delay, neural activities were influenced by sensory modality changes of the expected S-2 (tactile, auditory), and thus were involved in crossmodal associations between S-1 and S-2; and second, that in the later period of the delay, neural activities represented activation of neural networks in working memory, which were independent of sensory modality changes of go-signals (S-2) in the tasks.

Materials and methods

Subjects and Stimuli

Fourteen normal adult human subjects (all males, aged 18–33 years) participated in the study. All of them signed informed consent. The protocols of the experiments were approved by the IRB of the Johns Hopkins School of Medicine.

Participants sitting in a comfortable chair in a quiet, dimly lit room faced at eye level a white fixation point, 10 mm in diameter and 1.5 m away, on a black background. Tactile stimuli generated by a mechanical vibrator (Chubbuck 1966) were delivered on the subject’s left index fingertip. The frequency of the tactile stimulus was either 80 or 150 Hz. During performance of tasks, participants placed their left hand on a support in a comfortable position, and their right hand was also placed on a support and positioned to press two buttons with their index finger (left button) and middle finger (right button). Auditory stimuli were generated by a frequency generator (Grass Click-Tone Generator, Astro-Med Inc., West Warwick, RI), and delivered bilaterally through headphones. The frequency of the auditory stimulus was either 1,000 Hz or 2,000 Hz and its intensity was adjusted to approximately 50dB above the subject’s auditory threshold.

ERP recording

An electroencephalogram (EEG) recording system (Neuroscan SynAmp) was used in the study. Ag-AgCl scalp electrodes (Quick-Caps, Neuroscan) were arranged in standard locations (Guideline thirteen, American Clinical Neurophysiology Society, 2003), and referenced to linked earlobes. The impedance of each electrode was kept below 5 k_. Electrooculograms (EOG) were recorded for monitoring both horizontal eye movements (HEOG) and vertical eye movements (VEOG). EEG and EOG were amplified by an amplifier with a 0.1–100 Hz band-pass filter (−6 dB), digitized at 500 Hz sample rate.

The scalp-ERPs were recorded from the subjects when they performed working memory tasks. The subjects were instructed to focus on the fixation point throughout experiments to avoid voluntary eye movements, and also to minimize blinking. The original EEG data in trials with eye-blinks, excessive eye movements, or excessive muscle artifacts were excluded from the analysis.

Behavioral tasks

To test effects of associated auditory stimulation on ERP components elicited by tactile stimulation, several unimodal and crossmodal delay tasks were used in the present study.

Unimodal and crossmodal delayed response tasks

In the tactile-tactile unimodal delayed response task (Figure 1), a trial started with stimulus-1 (S-1) that was a 100-ms tactile vibration (either 150 Hz or 80 Hz) applied to the subject’s left index finger. A delay of 1,500 ms followed S-1. At the end of the delay, stimulus-2 (S-2), a 100-ms tactile vibration again (either high or low frequency) was presented. The subject was instructed to use S-2 as a go-signal, irrespective of the features of the S-2 (frequency of the vibration). Immediately after the onset of S-2, the subject pressed one of two buttons to indicate whether S-1 was of high or low frequency (e.g., left button for high, right button for low).

Fig. 1.

Schematic description of delayed response tasks, and delayed matching-to-sample tasks. In delayed response tasks, stimulus-1 (S-1) is a tactile vibration (high frequency or low frequency) delivered on the subject’s left index fingertip. In the unimodal task, stimulus-2 (S-2) is also a tactile vibration but in the crossmodal task, S-2 is a single auditory tone delivered through headphones. The S-2 in both tasks is a go-signal followed by the subject’s action to indicate whether S-1 is of high or low frequency. In delayed matching-to-sample tasks, at the end of the delay the subject indicates whether S-2 matches S-1. Fifteen electrodes shown in the gray area are those where ERP data are statistically analyzed.

The tactile-auditory crossmodal delayed response task was identical to the unimodal task except that in this task S-2 was an auditory tone (1,000 Hz or 2,000 Hz, 100 ms duration) presented bilaterally through headphones. Again, the subject was instructed to take the tones as go-signals, irrespective of their frequencies. The assignment of the buttons indicating high or low frequency of the vibration was random among subjects, and counterbalanced across subjects. Presentation of S-1, high or low frequency, was randomized from trial-to-trial to prevent the subject from predicting the stimulus. The frequency of S-2 was also presented in random order from trial-to-trial. The reaction time to S-2 was recorded.

Unimodal and crossmodal delayed matching-to-sample tasks

To test further whether the associated auditory stimulus indeed had the crossmodal effect on tactile ERP components, matching tasks were used, in which the auditory stimulus had the same physical features as the stimulus in the delayed response tasks, but different cognitive contents that would initiate different behaviors from those in the delayed response tasks.

In the delayed matching-to-sample tasks, the subject was instructed to memorize the frequency of tactile S-1 during the delay, to expect S-2 at the end of the delay, and to indicate at the end of each trial whether S-2 was associated with (matching) S-1 by pressing one of two buttons (e.g., left for match, right for nonmatch). In the unimodal task, S-2 was a tactile vibration, and in the crossmodal task, S-2 was an auditory tone. Associations between the tactile stimuli and the auditory stimuli were assigned before the subject started performing the task, and counterbalanced across subjects. Again, the frequency of S-1 or S-2 was presented in random order from trial-to-trial.

Strategies for task performance

The subjects were trained with all the tasks a day before ERP recording. They were instructed to practice controlling their eye movements and eye blinking, and how to perform the tasks with strategies required by the study.

In the delayed response tasks, the subjects were required to decide which button (left or right) was associated with the tactile S-1 (association, high or low frequency, assigned before performance of the task) immediately after its presentation, and to memorize this button position during the delay.

In the crossmodal delayed matching-to-sample tasks, the subjects were instructed to retain information about the associated auditory stimulus, S-2 (e.g., high tone) in the rest of the delay immediately after the presentation of S-1 (e.g., high frequency), and expect the presence of this S-2 at the end of the delay for a matching. The association between S-1 and S-2 was assigned before performance of the task.

The training was considered completed after the correct rate of task performance had reached about 85%; the reaction time to S-2 in each task was stable; and the subject reported that the required strategies for task performance could be applied reliably.

Data acquisition and data analysis

Thirteen subjects were included in the data analysis. One was excluded because of poor task performance and excessive eye blinks.

Original EEG data from which trials with eye-blinks, excessive eye movements, or muscle artifacts had been excluded were filtered with a digital zero-phase filter (Finite Impulse Response filter, pass band 0.3 to 40 Hz). A two-way repeated measures analysis of variance (ANOVA) by TASK (delayed response and matching-to-sample) and MODALITY (crossmodal and unimodal) as the within-subject factors was performed for comparison of reaction time (RT) and correct rate (CR) of performance. The significance for F values was obtained after Greenhouse-Geisser correction. Significance level was set at p < 0.05.

ERP recording was taken when the subject performed the tasks, each of which was designed to contain 3 blocks (40 trials for each block). Four out of 13 subjects were only able to finish 2 blocks for each task, however. The sequence of blocks of the tasks was pseudorandomized and counterbalanced. To prevent fatigue from task performance, the subject was allowed a break whenever 4 blocks of trials had been performed. EEG waveforms were monitored in real-time. All task event markers of each trial were saved with EEG data for off-line analysis that was carried out by using MATLAB (Mathworks, Natick, MA). They were: 1) the onset and the off-set of S-1; 2) the onset and the off-set of S-2; 3) the button-press (end of the trial). These event markers defined discrete epochs in a complete trial: the baseline, the S-1 period, the delay period, the S-2 period, and the reaction period.

Raw EEG signals were recorded continuously throughout the whole recording session. The grand average of ERPs was computed for those trials that had correct behavioral responses (over 90% of total trials for each task per subject). The time-locking event for the analysis was the onset of stimulus-1 (S-1). The baseline value in the analysis was the mean value calculated from the period of 500 ms preceding the onset of S-1. The amplitude of an ERP component was the difference between its peak value and the baseline value. The latency of the component was the time difference between the onset of S-1 and the peak of the component.

A four-way repeated-measures multivariate analysis of variance (MANOVA; (Dien and Santuzzi, 2004)) was performed for comparison of peak amplitudes and latencies among ERP components recorded from 15 electrodes that covered frontal and parietal areas (Figure 1). The within-subject factors of the analysis were MODALITY (crossmodal or unimodal), TASK (matching or delayed response), LR (Left-right location: left, center, right – corresponding to scalp electrode locations of 3, z, and 4), and AP (anterior-posterior location: frontal, frontocentral, central, centroparietal, parietal levels – corresponding to scalp electrode locations of F, FC, C, CP, and P).

At a certain point after the peak of an ERP component, the difference in ERPs between unimodal and crossmodal delayed response tasks returned to the level shown at baseline. Latency of this point (“off-set point”) was estimated with the procedure as follows: 1) calculating the absolute mean value of the difference between two tasks during the baseline, 2) dividing the grand ERP average recorded in the unimodal and the crossmodal tasks into 20-ms bins, 3) determining bins that contain ERP difference between two tasks larger than the absolute mean value of the baseline difference plus 2 SEMs, and 4) counting bins starting from the onset of S-1 until the end of the last three consecutive “larger-difference” bins obtained in step 3. The latency of the “off-set point” was determined by the number of total bins times 20 ms.

Results

Task performance

Reaction time (RT) to S-2 and correct rate (CR) of task performance were analyzed (Table 1). Two-way (TASK, MODALITY) repeated measures ANOVAs showed the significant effect of TASK (df = 1, F = 57.4, p < 0.001), and MODALITY (df = 1, F = 13.2, p < 0.01) on RT. RTs in the matching tasks were found to be significantly longer than those in the delayed response tasks, and RTs in the unimodal tasks were found to be significantly longer than those in the corresponding crossmodal tasks. The MODALITY-TASK interaction was not significant (df = 1, F = 0.1, p = 0.788).

Table 1.

Behavioral performance

	matching		delayed response
	crossmodal	unimodal	crossmodal	unimodal
RT	609±55	692±39	303±23	379±26
CR	91.8±1.6	93.1±1.6	95.7±0.7	96.0±0.9

RT, reaction time.

CR, correct performance rate (mean±SEM).

A significant effect of TASK (df = 1, F = 7.4, p < 0.02) on task performance (CR) was also observed, but no such significant effect of MODALITY was found (df = 1, F = 0.6, p = 0.45). CRs were significantly higher in the delayed response tasks. The interaction between TASK and MODALITY was not significant (df = 1, F = 0.2, p = 0.703).

ERP recording

ERPs collected from 15 electrodes were compared among tasks (Figure 1). Three components were observed in the delay period of the tasks (Figure 2, 3). They were: late positive component-1 (LPC-1), late positive component-2 (LPC-2), and late negative component (LNC).

Fig. 2.

Grand average ERPs recorded at 15 electrode sites in performance of delayed response tasks. Negativity is depicted as an up-deflection. All ERPs are time-locked to the onset of S-1. LPC-1: late positive component-1. LPC-2: late positive component-2. LNC: late negative component. Both LPC components were enhanced in the crossmodal task but there was no notable difference in LNC between two tasks. Topographies show voltage distributions of the components within the time window of 50 ms before, to 50 ms after the peak. They are normalized by root mean square, and scaled in the range of −0.5 to 0.5.

Fig. 3.

Grand average ERPs recorded at 15 electrode sites in performance of delayed matching-to-sample tasks. Both LPC components were enhanced in the crossmodal task, and the significant difference was observed in LNC between two tasks. Topographies show voltage distributions of the components. LNC is more negative over right central and parietal areas than any other area in both tasks, and LNC recorded in frontal areas in the crossmdal task is more negative than that in the unimodal task.

LPC-1 and LPC-2 were measured as an average of 100 ms duration around the peak (+/−50 ms). LNC was measured as an average of activities in the 500-ms period before the onset of S-2 since the activity in this time period was sustained and usually incremental without a clear peak.

Late positive components 1 and 2 (LPC-1 and LPC-2)

LPC-1 amplitude showed very significant differences between modalities, and among electrode locations (AP and LR), but not among tasks (Table 2). LPC-1 amplitudes showed a trend towards being higher over parietal areas than over frontal areas, and higher in the left hemisphere than in the right hemisphere (Figure 4A). Interaction among the factors was not significant, except for the interaction between MODALITY and LR (df = 2, F = 4.5, p < 0.05).

Table 2.

	ERP Amplitude
	LPC-1	LPC-2	LNC
MODALITY	p < 0.005 (df = 1, F = 14.6)	p < 0.001 (df = 1, F = 69.2)	p = 0.526 (df = 1, F = 0.4)
AP	p < 0.001 (df = 4, F = 18.5)	p < 0.001 (df = 4, F = 16.2)	p < 0.001 (df = 4, F = 13.7)
LR	p < 0.001 (df = 2, F = 21.2)	p < 0.001 (df = 2, F = 30.8)	p < 0.050 (df = 2, F = 5.2)
TASK	p = 0.325 (df = 1, F = 1.1)	p = 0.426 (df = 1, F = 0.7)	p < 0.050 (df = 1, F = 6.5)

	ERP Latency
	LPC-1	LPC-2
MODALITY	p = 0.065 (df = 1, F = 4.1)	p = 0.426 (df = 1, F = 0.7)
AP	p = 0.261 (df = 4, F = 1.6)	p < 0.050 (df = 4, F = 4.9)
LR	p = 0.070 (df = 2, F = 3.4)	p = 0.312 (df = 2, F = 1.3)
TASK	p = 0.509 (df = 1, F = 0.5)	p = 0.493 (df = 1, F = 0.5)

ERP components: LPC-1, LPC-2, LNC.

The within-subject factors (MANOVA): MODALITY, AP, LR, TASK.

Fig. 4.

A. LPC-1 shows significant difference between modalities (p < 0.005), and among electrode locations (AP: p < 0.001; LR: p < 0.001). B. LPC-2 shows similar results (MODALITY: p < 0.001; AP: p < 0.001; LR: p < 0.001). Error bars represent SEMs in this and other figures.

The latency of LPC-1 was not significantly related to any of the factors (Table 2), nor was interaction among the factors. The mean latency of the LPC-1 peak was 324 ms at F, 323 ms at FC, 322 ms at C, 323 ms at CP, and 327 ms at P.

LPC-2 amplitude was also significantly affected by modalities, and electrode locations, but not by tasks (Table 2). LPC-2 amplitudes showed a trend similar to that of LPC-1 (Figure 4B). No interaction between any factors was significant, except for the interaction between MODALITY and LR (df = 2, F = 6.6, p < 0.05).

The latency of LPC-2 was related to AP but not to LR, MODALITY, or TASK (Table2). Interaction among the factors was not significant. Overall, the mean latency of the LPC-2 peak was 517 ms at F, 516 ms at FC, 519 ms at C, 522 ms at CP, and 526 ms at P.

Late negative component (LNC)

The amplitude of LNC showed significant dependence on electrode locations, and tasks, but not on modalities (Table 2). Interaction was significant between TASK, and AP (df = 4, F = 7.0, p < 0.01), and between TASK and LR (df = 2, F = 11.7, p < 0.005). LNC amplitudes in the matching tasks showed a clear trend towards being higher in the right hemisphere than in the left hemisphere (Figure 3, LNC-topographies). In general, the amplitude of the LNC increased towards the end of the delay. The difference in LNC amplitude between any two tasks was analyzed by the post hoc test (Tukey HSD test). Results (Figure 5A) showed that there were no significant differences at all electrode sites between delayed response tasks, but in other three comparisons, difference between tasks was significant at most of the electrodes. Similar results could also be observed from comparisons of LNC-topographies between tasks (Figure 2 & 3). The LNC results from the delayed response tasks in the present study were compared with the LNC results from similar tasks (control tasks) in our previous study (Ohara et al., 2006, plus results from 3 more subjects); and showed that changes in modality of go-signals (S-2, tactile, visual, and auditory) had no significant effect on LNC (Figure 5B).

Fig. 5.

A. The difference between tasks in LNC. The significance level of the difference is marked with asterisks. Upper, comparison between two tasks with the same modality; Lower, comparison between two tasks with different modalities. B. LNCs recorded in delayed response tasks in the present study (TA: Tactile-Auditory; T-Ta: Tactile-Tactile), and in the previous study (T-V: Tactile-Visual; T-Tv: Tactile-Tactile).

ERP changes in the range of 700 ms to 900 ms

The ERP activity in the period of 700–900 ms was averaged and statistically analyzed among the tasks. This activity was not related to either MODALITY or TASK but to electrode positions (AP: p < 0.001; LR: p < 0.01), reaching the maximum in frontal areas.

Latency of the off-set point

Latency of the off-set point of LPC-2 was estimated for electrodes CPz and Pz at the midline over parietal areas, where the significantly higher amplitude of LPC-2 was observed. A similar estimation was also made for those results in equivalent tasks (control tasks) from our previous study (Ohara et al., 2006). The latency of the off-set points in all the tasks was in a range between about 600 and 700 ms (Figure 6).

Fig. 6.

Latency of the points (indicated by the dotted line) shown at 2 midline electrodes, starting from which the difference in ERPs is no longer significant between unimodal and crossmodal delayed response tasks in the present study (Tactile-Auditory), and in the previous study (Tactile-Visual).

ERPs elicited by tactile stimulus-2 in the unimodal tasks

Behavioral results showed that RT to S-2 in the unimodal matching task was significantly longer than that in the delayed response task (Table 1). Apparently, in the matching task, extra time was needed for the subject to keep his attention on S-2 to perceive and retain its frequency, and to decide whether it matched S-1. ERPs elicited by S-2 in the unimodal tasks are shown in Figure 7A (data from 5 middle electrodes for presentation). Three temporal windows of 200 ms duration were defined from 0 to 600 ms. Four-way (TASK, AP, LR, DURATION (0–200 ms, 200–400 ms, or 400–600 ms) ) repeated measures MANOVAs showed that the ERPs were significantly affected by TASK (df = 1, F = 47.6, p < 0.0001 ), AP (df = 4, F = 5.6, p < 0.02 ), LR (df = 2, F = 12.8, p < 0.002 ), and DURATION (df = 2, F = 65.6, p < 0.00001). Interactions were significant between TASK and AP (df = 4, F = 12.3, p < 0.002), and TASK and DURATION (df = 2, F = 21.7, p < 0.0002). Post hoc test showed that ERP amplitude in the matching task was significantly higher than that in the delayed response task in durations of 200–400 ms, and 400–600 ms (Figure 7B). Post hoc test also showed that ERP amplitudes in the matching task recorded at FC, C, CP, and P were significantly higher than those recorded over the same areas in the delayed response task (Figure 7C).

Fig. 7.

A. Grand average ERPs at 5 midline electrode sites are time-locked to the onset of S-2, showing time course of ERP changes elicited by S-2 in unimodal matching and delay response tasks. ERP changes in both tasks are relative to the potential at time zero, which is assigned as zero volts. Average reaction times ± 2 standard errors (RT ± 2SE) in each task are indicated by a vertical line with its corresponding band. Three temporal windows of 200 ms duration are defined from 0 to 600 ms. The dashed line indicates the time of 200 ms. B. EPR amplitudes averaged at 15 electrodes in the matching task are significantly higher than those in the delayed response task in the second and third durations (200–600 ms). C. ERP amplitudes in the matching task at F, C, CP, and P are significantly higher than those recorded over the same areas in the delayed response task.

Discussion

Task performance

The reaction time (RT) to S-2 (auditory stimuli in the crossmodal task, tactile stimuli in the unimodal task) was recorded among tasks in the present study. The longest RT was shown in the unimodal matching task, and the shortest RT in the delayed response crossmodal task. The difference in RT among tasks suggested that levels of the difficulty of discrimination of S-2, and accordingly, levels of the subject’s attention on S-2 were different among the tasks. The difficulty of discrimination of S-1, and the subject’s attention on S-1, however, were essentially at the same level among all the tasks since S-1 used in each of those tasks was identical.

ERP components

The present results showed that during the delay of the tasks, ERP components were in temporally sequential order, and correlated with task performance. These findings together with findings in our previous study on tactile-visual crossmodal working memory (Ohara et al., 2006) provide clear evidence that during the memorization period of the tasks, neural activities contained temporally discrete elements that represented different cognitive functions in tactile working memory, unimodal or crossmodal.

LPC-components with crossmodal association

LPC-1 and LPC-2 showing differences between modalities appeared to include neural activities involved in crossmodal associations between the tactile stimulus and the expected auditory stimulus in the crossmodal tasks. They also showed greater activity recorded over parietal areas than that over frontal areas, and greater activity in the left hemisphere (ipsilateral to S-1) than that in the right hemisphere. Although further studies are needed to obtain more direct evidence of sources in the brain for those components, imaging studies indicate that posterior parietal cortex, insular-claustrum, and prefrontal cortex may be involved in such crossmodal neural processes (Downar et al., 2000; Calvert 2001; Gobbele et al., 2003), and other studies suggest that the degree of hemispheric lateralization depends on variations in task demands and that left lateralization in working memory likely involved in a verbal or symbolic encoding strategy (Smith and Jonides, 1997; Ungerleider et al., 1998; Talsma et al., 2001). The results of LPC-components in our present tactile-auditory study further confirm what we found in our previous tactile-visual study (Ohara et al., 2006).

Behavioral results in the present study showed that in the unimodal delayed response task, the subject’s response time from the onset of the tactile S-2 to button-press was a little over 300 ms, and in the unimodal matching task, the subject’s response time was a little over 600 ms. Our analysis showed that in the duration of 0–200 ms after the onset of the S-2, ERP changes were similar between the two tasks, but in the duration of 200–600 ms, significant differences were observed between the tasks. Those results suggested that detection of the occurrence of tactile S-2 required by both tasks likely occurred within 200 ms, but perception of the frequency of S-2 and process of matching required only by the matching task likely occurred later, within 200–600 ms. Compared to the time course of ERP changes elicited by the tactile S-2, LPCs elicited by the tactile S-1 (the same stimulus as the S-2) with latency in the range of 200–600 ms may represent perception of the frequency of S-1, and association between S-1 and S-2. Apparently, those task-related cognitive neural processes are influenced by non-tactile stimuli in crossmodal tasks.

Considering the time course of LPC-components, we believe that the results of our studies agree with the model proposed by Harris et al. (2002). They found that performance of a tactile working memory task was significantly disrupted when a transcranial magnetic stimulation (TMS) pulse was delivered to the SI contralateral to the stimulated hand early in the delay at 300 or 600 ms. This effect was not observed when a TMS pulse was delivered at 900 ms or later. The researchers proposed a model in which the tactile memory trace was held initially in the SI, but not for longer than 900 ms. They suggested that the maintained neuronal activity in SI might constitute the neural substrate of the working memory trace itself, and was essential to optimal tactile working memory performance. Latencies of LPC-1 and LPC-2 in the present study and in Ohara et al. (2006) were around 300 ms and 600 ms, respectively. In addition, our results suggested the likelihood that memorization of the same information for the final decision in the tasks began right after “off-set” point (around 700 ms or shorter). Thus, the neural processes represented by LPC-components were in a temporal range similar to the effective range of TMS in Harris’ study. It would be a reasonable assumption that since those neural processes (e.g., association between S-1 and S-2) required accurate information about tactile S-1, the original memory trace of S-1 in the somatosensory cortex held for a similar range of time would be critical to those processes, and consequently to performance of the tasks.

Immediately after the off-set of LPC-2, the ERP activity (700–900 ms) showed higher level in frontal areas, but showed no significant difference between modalities or among tasks. This activity may represent the neural activity in transition from association between S-1 and S-2 to working memory.

LNC with working memory

By statistical analysis of LNC distribution among the electrodes, and the comparison between LNC-topographies, significant differences in LNC were observed between any two tasks, except for unimodal and crossmodal delayed response tasks in which the subject retained the same information during the delay for the final action. Studies have shown that motor preparation and attention affect ERP components (Eimer and Driver, 2001; Eimer and Van Velzen, 2002), especially the late ERP component, CNV (contingent negative variation) (Brunia and Damen, 1988; Tecce and Cattanach, 1993; van Boxtel and Brunia, 1994; Brunia and van Boxtel, 2001; Smith et al., 2007). Nevertheless, motor preparation and attention might not be major factors responsible for the significant difference in LNC between tasks in the present study. The argument is based on our results showing that although the motor action at the end of a trial was similar in all tasks, the significant difference in LNC was observed between any two tasks except for unimodal and crossmodal delayed response tasks, and that although attention to the expected S-2 was spatially different between the delayed response tasks, LNC was almost identical in those two tasks. These results indicated that changes in LNC likely depended on the information retained by the subject during the delay, and therefore suggested that the LNC was correlated with short-term retention of information in the delay. The above conclusion is consistent with findings in a recent ERP study on visual working memory (McCollough et al., 2007).

The localization of the LNC topography over right central and parietal areas in the matching tasks of the present study suggested involvement of right parietal cortical areas, contralateral to the tactile stimulus, in tactile working memory. This is consistent with the idea that neural networks of the posterior association cortices specialize in particular modalities or aspects of sensory information (Fuster 2001; Fuster 2004). The activity of cells in the posterior parietal cortex (PPC) was shown to be correlated with working memory (Koch and Fuster, 1989; Quintana and Fuster, 1993; Iriki et al., 1996; Grunewald et al., 1999; Linden et al., 1999; Burton and Sinclair, 2000; Snyder et al., 2000; Andersen and Buneo, 2002; Pesaran et al., 2002; Constantinidis and Procyk, 2004). PPC was also shown in human imaging studies to be involved in crossmodal transfer of information between the tactile and visual representations (Hadjikhani and Roland, 1998; Saito et al., 2003).

Anatomical evidence indicates that posterior association cortical areas project to prefrontal cortex (PFC) and receive reciprocal projections from it (Pandya and Yeterian, 1985; Goldman-Rakic 1987; Fuster 1997; Barbas et al., 2005; Medalla and Barbas, 2006). In order to carry out its temporally integrative functions, the PFC probably cooperates with the PPC (Fuster et al., 1985; Friedman and Goldman-Rakic, 1994; Cohen et al., 1997; Tomita et al., 1999).

The PFC is essential for working memory and integrating sensory information in different modalities with subsequent action in goal-directed behavior (Fuster 1997; Braver et al., 2001; Levy and Goldman-Rakic, 2000; Miller 2000; Fuster 2001; Miller and Cohen, 2001; Tanji and Hoshi, 2001). In recent human studies, the PFC has been found to be involved in tactile object recognition (Deibert et al., 1999; Stoeckel et al., 2003), and tactile working memory (Ricciardi et al., 2006; Soros et al., 2007).

The amplitude of LNC in all tasks increased progressively towards the end of the delay (between 1,000 ms and 1,500 ms), suggesting that the LNC represented activities involved in retention of sensory information for execution of the tasks. This is consistent with single unit studies in monkeys, which showed that the firing frequency of cells in PFC ramped up towards the end of delays in working memory tasks, and therefore those cells were involved in execution of the tasks (Fuster 1997).

LNCs in both delayed response tasks appeared to be determined by information retained by the subject in working memory, but not to be influenced by the change in modality or location of the expected S-2 between the two tasks. This suggests that LNC likely represents activities in PFC and is consistent with studies both in humans (Sakai et al., 2002) and in monkeys (Miller et al., 1996), which showed that sustained activities in PFC during the delay in working memory tasks were able to resist various distractions.

Sequential neural processes in working memory

Although in our present study we were not able to determine the precise sources of the ERP components, our data indicate that in the relatively earlier part of the delay in tactile crossmodal working memory tasks, the activity in the brain might be involved in perception of S-1 and tactile crossmodal association between S-1 and S-2, which likely occurred in the parietal cortical areas. Our data also suggest that starting from the end of the association towards the end of the delay, the activity participated in working memory of sensory-related information for execution of a behavioral action, in which both PPC and PFC were likely involved. The delay activities thus represent different cognitive functions from lower level to higher level in tactile crossmodal working memory, and therefore fit the concept of hierarchy in working memory (Marshuetz and Smith, 2006) and the perception-action cycle (Fuster 2001; Fuster 2004) that describes the cortical neural dynamics of sensory-motor behaviors represented by activities of distributed, interactive, and overlapping cortical neuronetworks named perceptual cognits and executive cognits (Fuster 2006).

Abbreviations

ANOVA

analysis of variance

CR

correct rate

EEG

electroencephalogram

EOG

electrooculogram

ERP

event-related potential

LED

light-emitting diode

LNC

late negative component

LPC-1

late positive component-1

LPC-2

late positive component-2

MANOVA

multivariate analysis of variance

PFC

prefrontal cortex

PPC

posterior parietal cortex

RT

reaction time

SI

primary somatosensory cortex

SII

secondary somatosensory cortex

TMS

transcranial magnetic stimulation