What and When: Parallel and Convergent Processing in Motor Control

Abstract

Successful motor behavior requires making appropriate response (response selection) at the right time (timing adjustment). Earlier psychological studies have suggested that the response selection and timing adjustment processes are performed serially in separate stages. We tested this hypothesis using functional magnetic resonance imaging. The subjects performed a choice reaction time task in four conditions: two (on-line response selection required or not) by two (on-line timing adjustment required or not). We found that the neural correlates for the two processes were indeed separate: the anterior medial premotor cortex (presupplementary motor area) was selectively active in response selection, whereas the cerebellar posterior lobe was selectively active in timing adjustment. However, the functional separation was only partial in that the lateral premotor cortex and the intraparietal sulcus were active equally for response selection and timing adjustment. The lateral premotor cortex was most active when both processes were required, suggesting that it integrates the information on response selection and the information on timing adjustment; alternatively, it might contribute to the allocation of attentional resources during dual information processing. The intraparietal sulcus was equally active when either response selection or timing adjustment was required, suggesting that it modifies, rather than integrates, these processes. Furthermore, our results suggest that these activations related to response selection and timing adjustment were distinct from sensory or motor processes.

Reaction process in response to an external stimulus has been thought to take several steps, starting from the stimulus encoding process to the final motor execution process (Fig. 1a; Frowein et al., 1981). This serial processing stage model is based on the additive factor theory (Sternberg, 1969), which assumes that the reaction time (RT) reflects a series of independent processing stages. Posner et al. (1973), Sanders (1977), and Frowein et al. (1981), using a choice reaction time task (CRT), have shown that uncertainty about which response to make (response uncertainty) and uncertainty about when to make the response (time uncertainty) both brought about an increase in the RT and that the effects were additive. The finding suggested that the two kinds of uncertainty affected two distinct processing stages, which have been referred to as the response selection and timing adjustment stages, respectively (Fig. 1a).

Fig. 1.

a, Scheme for reaction processes proposed in the earlier psychological studies (Frowein et al., 1981). The overall processes were thought to take several discrete stages arranged in a serial manner, where response uncertainty and time uncertainty independently affect the response selection and timing adjustment stages, respectively. In the original scheme, the timing adjustment was termed as motor adjustment. We used the present term to stress on the timing aspect. b, Task procedures. Subjects were asked to press buttons in response to the two types of stimuli using the index and middle finger of the right hand. The order of the presentation of the two types of stimuli and ISI were set either regular or random, creating four conditions named control, response uncertainty, time uncertainty, and dual uncertainty. c,Procedures for fMRI experiments. Six runs of experiments were conducted, three using the auditory paradigm, and the other three using the visual paradigm. For each run, one of the uncertainty, control, and rest conditions were repeated for six times in a counterbalanced order.

However, it is unknown whether the response selection and timing adjustment processes are performed in separate brain regions. Although a number of studies have suggested that some brain regions are related to response selection (Tanji and Kurata, 1985; Kurata, 1993; Chen et al., 1995; Deiber et al., 1996; Humberstone et al., 1997) or timing adjustment processes (Ivry et al., 1988; Ivry and Keele, 1989; Perrett et al., 1993; Jueptner et al., 1995; Rao et al., 1997; Penhune et al., 1998), they were not intended to test the anatomical separation for the two processes. Crucial to prove the anatomical separation is to incorporate the two processes in a single experimental paradigm. In the present study, we have devised a CRT in which response uncertainty and time uncertainty were manipulated independently to change the amount of processing for response selection and timing adjustment. Functional magnetic resonance imaging (fMRI) was used to measure the brain activations while the normal human subjects performed the CRT. This factorially designed task procedure allowed us to test the anatomical separation and/or interaction of the response selection and timing adjustment processes. In addition, to identify the neural structures that are independent of the sensory modality of the stimuli, we tested auditory and visual paradigms and searched areas that were commonly active for the two paradigms.

MATERIALS AND METHODS

Subjects

Six normal human subjects (five men, one woman, ages 28–45 years, right-handed) participated in the study. Informed consents were obtained from all the subjects prior to the study. The experimental protocol was approved by the ethics committee of the Communications Research Laboratory.

Behavioral paradigm

The subjects lay supine inside the bore of the magnet and held a plate equipped with two buttons. The subjects were asked to press one of the two buttons with the right index or middle finger in response to an auditory or visual stimulus while they were looking at a fixation spot at the center of the screen through a mirror. We conducted two sets of experiments, one using auditory stimuli, and the other using visual stimuli. For auditory stimuli, we used two types of tones with different pitches (1 and 2 kHz; rise–fall time, 5 msec; plateau, 20 msec; intensity, 95 dB), which were presented through a headphone. For visual stimuli, we used two disk-shaped patterns with different colors (yellow or blue; diameter, 4° in visual angle; duration of presentation, 60 msec), which were presented 2° above the eye fixation spot on the screen.

The task was factorially designed such that the two factors, the order of the presentation of the two types of stimuli (order) and the interstimulus interval (ISI), were varied independently at two levels (regular/random), creating four conditions named “control”, “response uncertainty”, “time uncertainty”, and “dual uncertainty” (Fig. 1b). In control, the presentation of the two types of stimuli were alternated regularly and separated by a constant ISI of 1 sec. In response uncertainty, the two types of stimuli were presented in random orders but with a constant ISI of 1 sec. In time uncertainty, the two types of stimuli were presented in regular alternation with random ISIs ranging from 0.7 to 1.3 sec. In dual uncertainty, the two types of stimuli were presented in random orders with random ISIs (0.7–1.3 sec). Two types of stimuli presented in regular alternation provide advance information about the next response, whereas stimuli presented at a constant interval provide advance information about the timing of button presses. In contrast to these predictable situations, the randomness in the order of stimulus presentation and/or ISIs creates unpredictable situations that require the increased amount of processing for response selection and/or timing adjustment. Thus, the comparison between the response uncertainty and control would reflect the response selection process, whereas the comparison between the time uncertainty and control would reflect the timing adjustment process. The comparison between dual uncertainty and control would reflect the combination of the two processes. This task design allowed us to determine how each brain area contributes to the response selection and the timing adjustment: nonselectively for both functions, selectively for one function, or selectively for the conjoint function.

During the task, the state of the ongoing task condition was indicated on the screen below the fixation spot, so that the subjects could make use of the advance information about the type of response and/or the timing of response. Subjects were instructed to suppress anticipatory responding and also to minimize choice errors and were trained for 20 min before the fMRI experiments. Trials with RTs <100 msec (anticipatory responding) and those with choice errors accounted for 2.2 and 3.4% of all the trials, which were excluded from the RT data analysis.

Procedure of fMRI experiment

First, structure images were obtained for each subject [fast low angle snap shot; repetition time (TR), 2.8 sec; echo time (TE), 4 msec; inversion time (TI), 300 msec; flip angle (FA), 15°; thickness, 1 mm; resolution, 1 × 1 mm; matrix, 256 × 256). Subsequently, the subjects underwent six runs of fMRI experiments, three using auditory paradigm and the other three using visual paradigm (Fig. 1c). For each run, the subjects performed six blocks of the following three conditions: one of the three uncertainty conditions, control condition, and rest condition (Fig. 1c). In the rest condition, the subjects were asked only to focus their gaze on the fixation spot, and no stimuli were presented. Each condition block lasted for 35 sec. Within each run, the orders of the three conditions were counterbalanced across the six repetitions, and, also, the orders of the three runs for each of the auditory and visual paradigms were counterbalanced across the six subjects. For each run, the task condition was started after two dummy scans, whereas a time series of 126 scans (seven scans × three conditions × six repetitions) was performed at an interscan interval of 5 sec using a 1.5 T scanner (Siemens Vision, Erlangen, Germany) equipped with a circular-polarized head coil. In each single scan, 14 slices of T2*-weighted gradient-echo echoplanar images (TR, 5 sec; TE, 66 msec; FA, 90°; thickness, 7 mm; resolution, 1.8 × 1.8 mm; matrix, 128 × 128) were collected parallel to the anterior and posterior commissure (AC–PC) line, which was determined based on the structure images obtained beforehand.

Data analysis

For data analysis, we used two methods. First, we performed statistical parametric analysis on the population data from the six subjects to identify the candidate brain regions related to response selection and/or timing adjustment processes. This analysis provides global estimates for the candidate brain regions and also shows the difference in the activation patterns between the two processes. However, the analysis depends on the population data and cannot take into account the anatomical variations across subjects. In addition, we needed another approach for statistical testing of the difference in the activations for each brain region between the two processes. Therefore, as the next step, we investigated whether the candidate regions are selectively related to response selection or timing adjustment process based on the individual data for each subject. In this second analysis, we delineated each candidate region based solely on the anatomical landmarks for each subject. This procedure could take the anatomical variations across subjects into account.

Identification of neural correlates. Using SPM96 (http//:www.fil.ion.ucl.ac.uk/spm; Welcome Department of Cognitive Neurology, London, UK), the series of functional images for the six runs was realigned, normalized, and smoothed with a Gaussian filter of 4.5 mm full-width half-maximum. The time series of magnetic resonance signals for the two conditions of interest was cross-correlated with a boxcar reference function shifted for one data point (5 sec), whereas the confounding effect of variation in global signal intensity across subjects was removed by analysis of covariance. Statistical parametric map of the t statistics was constituted from the resulting voxel values and was transformed to the unit normal distribution (SPM{Z}; thresholded at Z = 3.09). Taking spatial extent of activation into consideration, a correctedp value of 0.05 was used as the final threshold for significance. To find the areas commonly active for the auditory and visual paradigms, we performed a conjunction analysis for the two paradigms (Price and Friston, 1997). The analysis first created a statistical parametric map that reflected the sum of the contrasts for the two paradigms and then eliminated regions where there were significant differences (p < 0.05) between the two. Thus, the procedure identifies areas that show significant difference in the activation between the two conditions of interest and that are independent of the sensory modality of the stimuli.

First, we identified the candidate brain regions related to response selection and/or timing adjustment processes. The conjunction analysis between the auditory and visual paradigms was performed respectively for the following three comparisons: response uncertainty versus control, time uncertainty versus control, and dual uncertainty versus control. Subsequently, the three statistical parametric maps were rendered onto the same standard brain template (Talairach and Tournoux, 1988) to show the areas active in any of the three comparisons. Thus, the map shows areas that were independent of the sensory modality of the stimuli and reflect (response uncertainty–control) ∪ (time uncertainty–control) ∪ (dual uncertainty–control). The identified areas would be related to response selection and/or timing adjustment processes.

Second, we identified the neural correlates for motor execution process. This process should be independent of the sensory modality of the stimuli, and, in addition, should be similarly involved in all of the three uncertainty conditions as well as control condition. Based on the idea, the conjunction analysis was initially performed between the auditory and visual paradigms, respectively for control versus rest, response uncertainty versus rest, time uncertainty versus rest, and dual uncertainty versus rest. Subsequently, we searched areas that were common to all of these four pairs of contrasts. Thus, the results should show areas that were independent of the sensory modality and reflect (control–rest) ∩ (response uncertainty–rest) ∩ (time uncertainty–rest) ∩ dual uncertainty–rest). The identified areas would be related to the motor execution process.

Testing the effect of uncertainty. For the next step, we tested the effect of the uncertainty about response and uncertainty about timing on the activations of the brain areas identified by the preceding analysis. As will be described in Results, we found the four candidate areas related to the response selection and/or timing adjustment processes [the presupplementary motor area/rostral cingulate motor area (PreSMA/rCMA), cerebellar posterior lobe (Cbll-post), lateral premotor cortex (PM), and an area around the intraparietal sulcus (IPS)] and the three candidate areas related to the motor execution process [the supplementary motor area proper (SMA-proper), primary motor cortex (M1), and cerebellar anterior lobe (Cbll-ant)].

The volumes of interest (VOIs) for these seven areas were determined for each subject based on the anatomical landmarks before the creation of activation maps according to the following criteria (see also Fig. 4a), and we calculated the sizes of active volumes within the VOIs as measures for the degree of activations.

Fig. 4.

Region-based analysis. a, Seven volumes of interest placed on both hemispheres are shown on four slice images of one subject, which were determined based on the anatomical landmarks. Light pink, PreSMA/rCMA; dark pink, SMA-proper; light blue, PM; dark blue, M1; light green, Cbll-ant; dark green, Cbll-post; yellow, IPS. b,The active volume sizes under the four conditions, control (cont), response uncertainty (resp), time uncertainty (time), and dual uncertainty (dual), relative to rest are shown in cubic millimeters for the seven volumes of interest. The values were combined for both hemispheres and the means and SEs for the six subjects are indicated.

PreSMA/rCMA: the rectangular portion over the medial premotor region, extending anteriorly for 25 mm from the plane passing the anterior commissure (VCA), and laterally for 10 mm from the midline to both sides (Picard and Strick, 1996). Additionally, the VOIs were separately determined for PreSMA and rCMA by assuming their border to be at the superior cingulate sulcus.

SMA-proper: the rectangular portion over the medial premotor region between the VCA and the coronal plane passing the posterior commissure (VCP) and above the superior cingulate sulcus (Picard and Strick, 1996).

Cbll-post: the portion of the cerebellum between the primary fissure and the horizontal fissure that corresponded to the H VI-VIIa (Larsell and Jansen, 1972).

Cbll-ant: the portion of the cerebellum anterior to the primary fissure.

PM: the area around the precentral sulcus above the junction of the precentral and inferior frontal sulci, extending anteriorly for 10 mm from the precentral sulcus and posteriorly to the midline between the precentral and central sulci (Fink et al., 1997).

M1: the area posterior to the midline between the precentral and central sulci, anterior to the central sulcus and above the junction of the precentral and inferior frontal sulci.

IPS: areas around the horizontal segment of the intraparietal sulcus, 10 mm medial or lateral to the sulcus.

Activation maps were created for each subject by comparing each of the control, response uncertainty, time uncertainty, and dual uncertainty conditions with the rest condition using the same statistical threshold as used in the population data analysis. Then, for each VOI, the active volume sizes were computed and expressed in cubic millimeters for the four conditions, on which two-factor ANOVA was performed to test the effect of uncertainty about response and uncertainty about timing.

Additionally, these measures of active volume sizes were used for statistical testing of the laterality of activations. For each area, the active volume sizes between the two hemispheres were compared using paired t test, respectively for the four conditions.

Dependence on sensory modality. In addition to the areas commonly active for both auditory and visual paradigms, we identified areas that were dependent on the sensory modality of the stimuli. For this purpose, the statistical parametric maps were created respectively for the auditory and visual paradigms. The results were rendered onto the same standard brain template (Talairach and Tournoux, 1988) to see the difference and overlap of activations between the two paradigms. The comparisons were respectively made for response uncertainty versus control, time uncertainty versus control, and dual uncertainty versus control to identify the response selection and/or timing adjustment processes that were dependent on the sensory modality. In addition, the comparisons were respectively made for control versus rest, response uncertainty versus rest, time uncertainty versus rest, and dual uncertainty versus rest to identify the early perceptual processes that were unmodulated by any of the uncertainty conditions.

RESULTS

Behavioral data

Figure 2a shows the RTs for button pressing under the four conditions: control, response uncertainty, time uncertainty, and dual uncertainty. For both auditory and visual paradigms, the RTs were longer in the three uncertainty conditions compared with the control. Two-factor ANOVA showed that both the response uncertainty and the time uncertainty increased the RT significantly (for auditory paradigm, the main effect of response uncertainty was F_(1,5) = 15.1,p < 0.05; the main effect of time uncertainty wasF_(1,5) = 10.1, p < 0.05; for visual paradigm, they wereF_(1,5) = 19.7, p < 0.05 and F_(1,5) = 6.68,p < 0.05, respectively). On the other hand, the interaction of the two kinds of uncertainty was not significant (auditory paradigm, F_(1,5) = 0.095,p > 0.1; visual paradigm,F_(1,5) = 0.012, p > 0.1). Relative to control, the increase of RT for dual uncertainty (mean increase of RT: 85.4 and 140.4 msec, respectively for auditory and visual paradigms) was close to the sum of the increase of RT for response uncertainty and RT for time uncertainty (50.9 + 33.3 and 70.8 + 50.3 msec, respectively, for auditory and visual paradigms).

Fig. 2.

Reaction times (a) and variance of reaction times (b) for the four conditions: control (cont), response uncertainty (resp), time uncertainty (time), and dual uncertainty (dual). Means and the SEs for the six subjects were separately shown for the auditory (left) and visual (right) paradigms.

To the contrary, for the variance of RT, the interaction of the two kinds of uncertainty was significant (auditory paradigm,F_(1,5) = 24.1, p < 0.05; visual paradigm, F_(1,5) = 7.73,p < 0.05; Fig. 2b). Relative to control, the increase of the RT variance for dual uncertainty condition (mean increase of RT variance, 2813 and 2743 msec²) was significantly shorter than the sum of the increase of RT variances for response uncertainty and time uncertainty (2824 + 2380 and 2717 + 2142 msec²). Thus, there was no additive effect on the variance of RT.

fMRI data

Using fMRI, we investigated whether different brain regions subserve the response selection and timing adjustment processes. We present the fMRI data in two parts: (1) show the candidate regions involved in response selection and/or timing adjustment; (2) examine whether the candidate regions are selectively related to response selection and/or timing adjustment. The results are shown separately for the areas that were independent of the sensory modality of the stimuli and for the areas that were dependent on the sensory modality of the stimuli.

Identification of neural correlates

Response selection and/or timing adjustment. We first determined the candidate brain regions related to response selection and/or timing adjustment. Initially, six statistical parametric maps for the group of six subjects were created by comparing each of the three uncertainty conditions with control, respectively for the auditory and visual paradigms. The coordinates of the active areas are indicated in Tables 1 and2. To determine the areas that were independent of the sensory modality of the stimuli, the conjunction analysis was performed between the auditory and visual paradigms, respectively, for the three comparisons (coordinates shown in Table 3). The areas commonly active for the two paradigms were coded yellow on the surface of the standard brain (Fig. 3a, left) and on the axial sections (Fig. 3a, right). The figures indicate areas that were active in any of the three uncertainty conditions relative to control. The areas included the anterior part of the medial premotor cortex above and below the superior cingulate sulcus. They were located anterior to the VCA and thus were considered to be the PreSMA (Picard and Strick, 1996; Tanji, 1996) and the rCMA (Dum and Strick, 1993; Picard and Strick, 1996). Because the activation was continuous over the superior cingulate sulcus, the two areas were, hereafter, jointly referred to as PreSMA/rCMA. The yellow areas were also found in the lateral part of the cerebellar posterior lobe on both sides, which were mainly localized in H VI-VIIa (Larsell and Jansen, 1972) (Cbll-post) (better seen in Fig. 3a, right). Other areas coded in yellow were the PM (Passingham, 1993; Fink et al., 1997;Wise et al., 1997) and the area around the IPS. PreSMA/rCMA, Cbll-post, and PM were active on both sides, whereas IPS was active only on the left side. These areas were considered to be related to response selection and/or timing adjustment processes, irrespective of the sensory modality of the stimuli.

Table 1.

Coordinates and Z scores of active areas in auditory paradigm

Comparison	Region	Coordinate	Zscores
Response uncertainty–Control	PM	−54, 6, 46	4.88
	PreSMA/rCMA	−6, 6, 52	5.02
		10, 12, 50	4.09
	IPS	−40, −46, 56	4.25
	STG-post	−70, −22, 18	4.44
		70, −26, 24	4.54
Time uncertainty– Control	PM	−50, 4, 42	4.22
	IPS	−40, −42, 54	4.25
	STG-post	−68, −26, 22	4.92
		70, −30, 20	4.04
	Cbll-post	−44, −76, −26	5.16
		38, −62, −24	4.11
Dual uncertainty– Control	PM	−52, 6, 48	5.02
		−34, 2, 64	5.11
		58, 6, 44	4.68
	PreSMA/rCMA	−4, 4, 54	4.24
		4, 6, 50	4.54
	IPS	−34, −42, 62	4.44
	STG-post	−56, −22, 16	4.23
		48, −24, 18	3.68
	Cbll-post	−32, −56, −24	3.83
		38, −62, −24	4.11
(Control–Rest)∩	M1	−42, −18, 48	6.60
(Response uncertainty–Rest)∩	SMA-proper	−4, −6, 60	3.82
(Time uncertainty–Rest)∩	IPS	−42, −46, 58	4.25
(Dual uncertainty–Rest)	STG-post	−42, −22, 18	4.68
	A1	−50, −6, 6	5.12
		54, −10, 8	4.88
	Cbll-ant	14, −46, −16	6.72

Cbll-ant, Cerebellar anterior lobe; Cbll-post, cerebellar posterior lobe; IPS, intraparietal sulcus; M1, primary motor cortex; PM, lateral premotor cortex; PreSMA, presupplementary motor area; rCMA, rostral cingulate motor area; SMA-proper, supplementary motor area proper; STG-post, posterior superior temporal gyrus.

Table 2.

Coordinates and Z scores of active areas in visual paradigm

Comparison	Region	Coordinate	Zscores
Response uncertainty–Control	PM	−50, 8, 50	4.60
	PreSMA/rCMA	−4, 4, 52	5.52
	IPS	−36, −42, 54	4.02
	V2/4	14, −72, −14	3.88
Time uncertainty– Control	PM	−44, 2, 50	4.12
	IPS	−34, −36, 60	4.56
	V2/4	−6, −66, −12	4.02
		14, −64, −10	3.64
	Cbll-post	−36, −68, −26	4.81
		36, −62, −24	4.76
Dual uncertainty– Control	PM	−52, 10, 46	4.98
		−30, −4, 62	5.66
		50, 2, 50	4.68
	PreSMA/rCMA	−6, 8, 52	5.02
		2, 2, 50	4.24
	IPS	−44, −34, 60	5.11
	V2/4	−6, −56, −6	5.42
		18, −60, −14	4.02
	Cbll-post	−42, −52, −26	4.98
		44, −62, −26	4.06
(Control–Rest)∩	M1	−36, −16, 50	5.42
(Response uncertainty–Rest)∩	SMA-proper	−4, −8, 58	4.02
(Time uncertainty–Rest)∩	IPS	−46, −40, 54	4.34
(Dual uncertainty–Rest)	V1/2	−8, −76, −4	5.22
		−4, −66, 2	4.88
		2, −70, 14	4.86
	Cbll-ant	16, −52, −20	5.02

For abbreviations, see Table 1 legend.

Table 3.

Coordinates and Z scores of commonly active areas for auditory and visual paradigms (Auditory ∩ Visual)

Comparison	Region	Coordinate	Z scores
Response uncertainty–Control	PM	−52, 2, 50	4.37
	PreSMA	−2, 8, 50	4.51
		6, 12, 54	4.23
	IPS	−36, −42, 54	4.03
Time uncertainty– Control	PM	−50, −4, 48	4.06
	IPS	−32, −44, 56	4.28
	Cbll-post	−32, −58, −22	4.34
		40, −62, −24	4.51
Dual uncertainty– Control	PM	−50, 2, 52	4.67
		34, 0, 60	4.56
	PreSMA	−2, 10, 52	4.84
		4, 2, 50	4.09
	IPS	−36, −42, 62	4.52
	Cbll-post	−40, −56, −26	4.82
		40, −62, −24	4.22
(Control–Rest)∩	M1	−36, −16, 46	5.26
(Response uncertainty–Rest)∩	SMA-proper	−4, −8, 62	4.32
(Time uncertainty–Rest)∩	IPS	−46, −44, 58	3.88
(Dual uncertainty–Rest)	Cbll-ant	16, −48, −18	4.86

For abbreviations, see Table 1 legend.

Fig. 3.

Statistical parametric analysis: sensory-modality-independent areas. Areas related to response selection and/or timing adjustment processes (a) and areas related to motor execution process (b) are shown in yellow on the surface of the standard brain (left three figures) and on the three axial slices of a single subject fitted into the Talairach space (Talairach and Tournoux, 1988) (right three figures). Theleft side of the figure indicates the left hemisphere.a, (response uncertainty–control) ∪ (time uncertainty–control) ∪ (dual uncertainty–control);b, (control–rest) ∩ (response uncertainty–rest) ∩ (time uncertainty–rest) ∩ (dual uncertainty–rest).

Motor execution. Note that the above activations were revealed with respect to the control condition. The comparison with the rest condition revealed a different set of brain areas, which were commonly active in the control and the three uncertainty conditions (Fig. 3b). The areas commonly active in the auditory and visual paradigms (shown in yellow) were the left medial premotor cortex posterior to VCA, which was considered to be the SMA-proper (Picard and Strick, 1996; Tanji, 1996), the left M1, the right Cbll-ant, and the left IPS. These areas were considered to be related to the motor execution process because they were active independently of the sensory modality of the stimuli and were active irrespective of the presence or absence of the response/time uncertainty.

Testing the effect of uncertainty

The determination of active brain areas so far was still tentative because the analysis was based on the population data. To further characterize the candidate brain areas, we performed the second part of analysis: we delineated the candidate areas for each subject and statistically examined the contribution of these areas to the response selection and the timing adjustment.

After determining the VOIs on both hemispheres for each subject (Fig.4a), the active volume sizes within the VOIs relative to the rest condition were computed respectively for the four task conditions (control and the three uncertainty conditions) (Fig. 4b). Two-factor ANOVA performed on the active volume sizes showed that, for both the auditory and visual paradigms, the uncertainty about response produced significantly increased activation in PreSMA/rCMA, PM, and IPS (p < 0.05), whereas the uncertainty about timing produced significantly increased activation in Cbll-post, PM, and IPS (p < 0.05) (Fig. 4b, Table4). The interaction of the two factors was found to be significant only for PM, which showed highest activity under the dual uncertainty condition (p < 0.05; corrected for multiple comparisons) (Fig. 4b, Table 4). In contrast, the activations of SMA-proper, M1, and Cbll-ant were not affected either by the response or time uncertainty (p > 0.1).

Table 4.

Effect of response uncertainty and time uncertainty on active volume size

	Response	Time	Response × Time
Auditory paradigm
PreSMA/rCMA	0.007 (19.3)	0.970 (0.002)	0.098 (4.14)
Cbll-post	0.832 (0.050)	0.012 (15.1)	0.549 (0.413)
PM	0.011 (15.8)	0.002 (34.5)	0.038 (7.87)
IPS	0.004 (25.4)	0.038 (7.89)	0.203 (2.15)
PreSMA	0.009 (16.8)	0.499 (0.531)	0.087 (4.51)
rCMA	0.022 (10.7)	0.584 (0.342)	0.356 (1.04)
SMA-proper	0.818 (0.059)	0.687 (0.182)	0.467 (0.620)
M1	0.800 (0.072)	0.878 (0.026)	0.607 (0.301)
Cbll-ant	0.609 (0.297)	0.560 (0.389)	0.564 (0.382)
Visual paradigm
PreSMA/rCMA	0.0001 (516)	0.670 (0.204)	0.861 (0.034)
Cbll-post	0.892 (0.020)	0.002 (34.0)	0.477 (0.591)
PM	0.002 (32.2)	0.0001 (216)	0.045 (7.07)
IPS	0.011 (15.4)	0.042 (7.41)	0.116 (3.62)
PreSMA	0.0001 (106)	0.252 (1.68)	0.319 (1.23)
rCMA	0.001 (51.2)	0.377 (0.939)	0.600 (0.314)
SMA-proper	0.496 (0.539)	0.967 (0.002)	0.954 (0.004)
M1	0.435 (0.721)	0.762 (0.102)	0.728 (0.136)
Cbll-ant	0.693 (0.175)	0.679 (0.192)	0.375 (0.950)

P and F values (within parentheses) for the effects of uncertainty about the response (Response), uncertainty about the timing (Time), and interaction of them (Response × Time) on the active volume sizes for the seven areas. For PreSMA and rCMA, the calculations were made on the combined values as well as the respective values. Underlines indicate the significant effect (p < 0.05).

For abbreviations, see Table 1 legend.

The comparisons of the active volume sizes between the two hemispheres showed that, for all the conditions, PreSMA/rCMA and Cbll-post were bilaterally active (paired t test; p > 0.1), whereas PM and IPS was significantly more active on the left side (p < 0.01). Activations of the SMA-proper, M1, and Cbll-ant were unilateral: the former two confined on the left hemisphere, and the latter confined on the right side.

Dependence on sensory modality

We also found brain activations that were selective for the sensory modality. Figure 5ashows the areas that were active in each of the three uncertainty conditions, separately for the sensory modalities: selectively active for the auditory paradigm (shown in green), selectively active for the visual paradigm (shown in red), and commonly active in both paradigms (shown in yellow). The posterior portion of the superior temporal gyrus (STG-post; Rivier and Clarke, 1997) was selectively active for the auditory paradigm, whereas the ventral part of the occipital areas probably corresponding to V2 and V4 (Zeki et al., 1991; Sereno et al., 1995) was selectively active for the visual paradigm. As shown in Figure 5a, these sensory–modality-dependent areas were similarly active for the three uncertainty conditions.

Fig. 5.

Statistical parametric analysis: sensory–modality-dependent areas. Areas of significant activation for the group of six subjects are indicated on the surface of the standard brain. a,Top row, (response uncertainty–control); second row, (time uncertainty–control); third row, (dual uncertainty–control); b, (control–rest) ∩ (response uncertainty–rest) ∩ (time uncertainty–rest) ∩ (dual uncertainty–rest).

Figure 5a also confirmed the differential involvement of PreSMA/rCMA, Cbll-post, and PM in response selection, timing adjustment, and dual processes. PreSMA/rCMA was active for the response uncertainty, but not for the time uncertainty, whereas Cbll-post was active for the time uncertainty, but not for the response uncertainty. The two areas were active for the dual uncertainty. In contrast, PM was bilaterally active only for the dual uncertainty. Whereas PM was active on the left side for the response uncertainty and time uncertainty, its activation was significantly higher for the dual uncertainty compared to the single uncertainty, as seen from Figure 4b. The left IPS was active to a similar degree for the three uncertainty conditions.

Figure 5b shows the areas that were active even without any uncertainty: the conjunction of the four comparison; (response uncertainty–rest), (time uncertainty–rest), (dual uncertainty–rest), (control–rest). Areas active for the auditory and visual paradigms were respectively coded in green and red. As shown, the anterior portion of the superior temporal gyrus (A1) and STG-post were active only in the auditory paradigm (coded green), whereas the medial and ventral occipital areas including V1 and V2/V4 were active only in the visual paradigm (coded red).

DISCUSSION

By changing response uncertainty and time uncertainty independently, we found that a set of brain areas were active selectively or conjointly for the response selection and timing adjustment processes. The activations were largely distinct from those related to the motor execution processes that were present without any uncertainty and were also distinct from those related to the sensory processes that were selective for the sensory modality used for the response cue. Based on these results, we propose a parallel and convergent processing model as shown in Figure6. In the following will be discussed the evidence for it and other possible interpretations of our findings.

Fig. 6.

Hypothetical neural mechanisms for the processing of response selection and timing adjustment; the processing (top) and the neural correlates (bottom). The scheme depicts the parallel processing of response selection and timing adjustment, which are subserved by PreSMA/rCMA and Cbll-post. PM may either integrate these kinds of information or allocate attentional resources during dual information processing.

Separate processing

The clear double dissociation between the PreSMA/rCMA and Cbll-post activations suggests independent neural computations for response selection and timing adjustment. This structural segregation makes it unlikely that the observed activations were merely the reflection of nonspecific attention. Instead, we consider that the activations reflect the on-line processing for selecting the appropriate response and that for determining the right timing of the response. Behaviorally, either process would lead to an increase in RT, as we found. However, it is unclear whether PreSMA/rCMA and Cbll-post actually compute the response and timing or they are related to the selective attention to stimulus features (Pardo et al., 1990) and timing.

PreSMA is anatomically and functionally distinct from the posteriorly situated SMA-proper (for review, see Picard and Strick, 1996; Tanji, 1996), and is particularly active when the subjects had to select the appropriate response on-line (Deiber et al., 1996; Humberstone et al., 1997; Petit et al., 1998; Ikeda et al., 1999; Sakai et al., 1999a), change or update motor plans (Matsuzaka and Tanji, 1996; Shima et al., 1996), or respond to an unpredictable visual stimulus (Dassonville et al., 1998). In our behavioral paradigms, the response was selected based on the auditory-motor or visuomotor mapping rule. This could be done by the direct connections from the auditory and visual association cortices (Luppino et al., 1993; Rizzolatti et al., 1998), as schematized in Figure 6. Alternatively, the information on the sensory cue may be relayed by way of other areas, such as the lateral prefrontal cortex (Bates and Goldman-Rakic, 1993; Luppino et al., 1993). rCMA behaved similarly to PreSMA (Table 4). This area has been shown to be connected with PreSMA (Luppino et al., 1993) and to play roles in higher-order motor control, especially in movement selection based on reward (Shima and Tanji, 1998).

Our data suggested that the posterior part of the cerebellum is related to the timing adjustment. Indeed, cerebellar patients show deficits in monitoring and reproducing timing (Ivry et al., 1988; Ivry and Keele, 1989; Nichelli et al., 1996). We now show, more specifically, that the timing-related area was localized bilaterally in the lateral part of the cerebellar posterior lobe, consistent with our preceding study (Sakai et al., 1998, 1999b). The finding is also consistent with the study of eyeblink conditioning showing that a part of Cbll-post (Larsell's H VI) plays a critical role in precise timing adjustment (Yeo and Hardiman, 1992; Gruart and Yeo, 1995). In our task the subjects might predict the time of the next cue stimulus and, if the actual time of the stimulus was different, adjust the timing of the response. Indeed, the RTs were shortest when the preceding ISI was around the mean value (1 sec) and increased when the preceding ISI was deviated from 1 sec [mean RTs were 320, 271, and 310 msec, when the preceding ISI fell within the range of 700–900, 900–1100, and 1100–1300 msec, respectively (ANOVA,F_(5,10) = 14.36, p < 0.01)]. This suggests that the activation of Cbll-post is related to the correction of timing error, consistent with the general view of the cerebellar function (Ito 1984; Thach et al., 1992; Kitazawa et al., 1998). Our data also suggest that both the auditory and visual information converge on the cerebellum (for review, see Schmahmann, 1996) (Fig. 6).

To summarize, PreSMA/rCMA and Cbll-post may contribute independently to the response selection and timing adjustment processes, or, in other words, to determine “what action to take” and “when to take it”. However, these data alone cannot indicate whether the two processes were performed in a serial or parallel manner. The interaction of these processes must be examined to answer this question.

Interaction

PM was significantly more active in the dual uncertainty condition than in the single uncertainty conditions. The result suggests two possibilities. First, PM may play a role in integrating the information for response and timing. “What” information and “when” information may be processed in parallel by PreSMA/rCMA and Cbll-post, which, then, converge on PM to generate a final motor program (Fig. 6). Consistent with this view, PM receives connections from both the pre-SMA (Barbas and Pandya, 1987; Kurata, 1991) and the cerebellum (Middleton and Strick, 1997), although it is unknown whether Cbll-post projects to PM. Also consistent is a physiological finding that neuronal activity in the dorsal PM was strongly influenced by the predictability of response and timing (Mauritz and Wise, 1986). As the second possibility, PM might contribute to the allocation of attentional resources during dual information processing (Iacoboni et al., 1998). In this case, PM would modify, rather than integrate, the response and timing processes (Fig. 6).

IPS was also related to both the response selection and timing adjustment processes. However, its activity was not enhanced in the dual uncertainty condition, suggesting that IPS does not play an integrative role. It has been shown that IPS is crucial for the selection of action (Snyder et al., 1997) and might encode the timing of events (Mackay and Crammond, 1987). Such selection signals and timing signals in IPS may be sent to the nonprimary sensory areas as top-down signals to improve their performance (Fig. 6). This can be regarded as an attentional mechanism in which IPS is considered to play an important role (Kalaska et al., 1983; Rushworth et al., 1997). Anatomically, IPS is densely connected with PM (Passingham, 1993; Wise et al., 1997; Rizzolatti et al., 1998), and may also be connected with PreSMA (Luppino et al., 1993) and the cerebellum (May and Andersen, 1986). IPS is also mutually connected with the nonprimary sensory areas (Blatt et al., 1990), consistent with our scheme (Fig. 6).

Motor execution

The present study has shown that the motor execution process is anatomically segregated from the response selection and timing adjustment processes, consistent with the suggestion in the earlier psychological study (Frowein and Sanders, 1978). The finding indicates the functional separation between PreSMA and SMA-proper and between Cbll-post and Cbll-ant, which was consistent with the preceding studies (Ivry et al., 1988; Picard and Strick, 1996; Tanji, 1996; Allen et al., 1997; Sakai et al., 1998, 1999b). The only exception is IPS, which was active both in motor execution and in response selection/timing adjustment processes. It remains open to future studies whether different subareas within IPS are selectively involved in these processes.

Perceptual processes

Some of the nonprimary sensory areas were related to the response selection and the timing adjustment, but selectively for the sensory modality: STG-post for the auditory paradigm and V4 for the visual paradigm. The activation of these areas may reflect the increased level of attention to the auditory (especially, pitch) (Woodruff et al., 1996; Tzourio et al., 1997; Benedict et al., 1998) or visual (especially, color) stimuli (Moran and Desimone, 1985; McAdams and Maunsell, 1999). We speculate that the attentional effects were exerted by the IPS (Fig. 6). On the other hand, the primary sensory areas (A1 and V1) seemed to be unaffected by the uncertainty conditions (Woodruff et al., 1996; Zeki and Marini, 1998; McAdams and Maunsell, 1999), suggesting that these areas are related to the initial processing of stimulus encoding.

Interpretation of the behavioral data

The increase of RT for dual uncertainty was close to the sum of the increase of RT for response uncertainty and RT for time uncertainty. This additive effect seems to support the serial model (Fig. 1a). However, the result on the variance of RT is inconsistent with the serial model. The serial model would require a similar additive effect for the variance of RT: the variance of RT for dual uncertainty should be the sum of the RT variances for the two single uncertainty conditions (Sternberg, 1969). Our result indicated, on the contrary, that the variance of RT for dual uncertainty was close to the variance of RT for response uncertainty (Fig. 2b). According to the model shown in Figure 6, the response selection and timing adjustment processes are performed in parallel, but their integration starts only after both of the two processes are finished. The variance of RT for dual processing would then be determined by the variance of RT for the slower process between the two (response selection in the present study). The additive effect of RT would be attributable to the time required for the integration of the two kinds of information.