Mental chronometry using latency-resolved functional MRI

Abstract

Vascular responses to neural activity are exploited as the basis of a number of brain imaging techniques. The vascular response is thought to be too slow to resolve the temporal sequence of events involved in cognitive tasks, and hence, imaging studies of mental chronometry have relied on techniques such as the evoked potential. Using rapid functional MRI (fMRI) of single trials of two simple behavioral tasks, we demonstrate that while the microvascular response to the onset of neural activity is delayed consistently by several seconds, the relative timing between the onset of the fMRI responses in different brain areas appears preserved. We examined a number of parameters that characterize the fMRI response and determined that its onset time is best defined by the inflection point from the resting baseline. We have found that fMRI onset latencies determined in this manner correlate well with independently measurable parameters of the tasks such as reaction time or stimulus presentation time and can be used to determine the origin of processing delays during cognitive or perceptual tasks with a temporal accuracy of tens of milliseconds and spatial resolution of millimeters.

A large body of research in human perception/cognition has been concerned with the analysis of mental events into their hierarchical processing stages, the temporal aspect of such processing being termed mental chronometry (1). Mental chronometric tasks have been used extensively in cognitive science to elucidate mechanisms underlying cognitive processing, often using reaction time (RT) as a variable for correlation. The traditional RT approach in studies of cognitive processing could be complemented by a measure of stimulus processing that is independent of explicit motor responses (2) and (ideally) spatially resolved within the brain. One such method is the evoked potential (EP) (3), but despite modern filtering and inversion algorithms, scalp EPs can give a somewhat distorted perspective of timing and origin of the evoked activity (4, 5), with surprisingly long latencies (>50 ms) relative to direct extracellular recordings in the human cortex (6, 7). Nonetheless, EPs remain the best available, noninvasive method for determining the sequence of activity in cognitive brain function.

On the other hand, functional brain mapping techniques that give precise spatial information about the activity of neural substrates rely on changes in the relatively slowly responding cerebral hemodynamic properties such as blood flow (8–11). Maps made with positron-emission tomography (PET) (9) and, more recently, with functional MRI (fMRI) (10, 11) indicate (directly or indirectly) local blood flow changes in response to neural activity modulation. However, they do not contain information about the relative onset of activity in different brain regions. Recent advances in averaged single-trial fMRI (12, 13) have made it possible to map brain activity by using paradigms similar to those used in the EP literature, which, in principle, allows the possibility of measuring the timing difference between activation of neural substrates in the brain.

The EP waveform has a number of characteristic features that could be used to extract timing between brain areas, and it took several decades to understand which of these features were relevant to mental processing load and timing (3). Although simpler, there are many features of the fMRI response that conceivably could be used to determine latency between brain areas, and, to date, none have been found to give robust timing information among activated brain regions. To compare our technique with previous EP literature, we have performed a well characterized experiment involving a hemifield checkerboard presentation (14) in which the temporal onset of the left and right hemifield was controlled accurately, using our latency-resolved fMRI technique.

We have also examined a simple, visually cued motor task similar to one used in monkey single-unit studies and were able to demonstrate robust single-trial maps, responses, and timing between neural substrates that correlate with known electrophysiological measurements as well as psychophysical measures of task performance. The ability to correlate psychophysical parameters such as reaction time with latency-resolved fMRI allows the determination of which neural substrates are involved in task-related processing and which ones are constants of the task. Our results suggest a new and important role for fMRI in studying the origin of processing delays in mental operations.

MATERIALS AND METHODS

Subjects.

All subjects were recruited from the academic environment of the University of Western Ontario and handedness was determined by the Edinburgh Handedness Inventory (15). The study was approved by the Review Board for Health Sciences Research Involving Human Subjects of the University of Western Ontario, and written informed consent was obtained from each subject. For the hemifield experiments, three male and one female right-handed subjects aged 28 ± 4 years (mean ± SD) were used. For the RT experiments, six subjects were tested, four male and two female, aged 24–30 years [mean = 27 ± 2 (SD)]. All subjects for RT experiments used their dominant arm as determined above. Five were determined to be right-handed and one (male) was left-handed.

Design of Experiment One.

In the first experiment, we investigated the relative and absolute latency of the fMRI onset time in the same vascular bed. We also attempted to discern which features of the fMRI response correlated with the temporal properties of the stimulus. To do this, the fMRI response in the left and right hemispheres of human primary visual cortex (V1) was measured by using a 45° wide, color-reversing (at 8 Hz) blue-yellow radial checkerboard on a gray background (see Fig. 1 A Insert), which was presented by using an LCD video projector (NEC, Ithasca, IL). The central 5° contained a fixation cross with no checks. Between visual stimulus presentations, subjects stayed fixated on a cross centered on a gray screen. During acquisition of a functional data set, the visual stimulus with the chosen delay was presented 10 times at the beginning of each regular interval of 25 s (10 single trials). The long delay between stimulus presentations was chosen to establish a flat baseline-image intensity. Each half of the checkerboard was presented for a 2-s duration. The left hemifield was presented first, then the right one at 0, 125, 250, 500, or 1,000 ms later. All subjects were tested with each of the five delays (by J.S.G.), but the order in which they were presented was randomized and the analysis was performed blinded (by D.C.L.). Latency maps were generated as described below, and time series of average image intensity in appropriate regions of interest (ROIs) as a function of image number were created separately for activated pixels in V1 of each hemisphere.

Figure 1.

Hemifield experiment. (A) Typical fMRI response from left and right hemispheres of V1 because of a 500-ms offset between left and right side checkerboard presentation of a single subject (Inset and see Materials and Methods). (B) Lissajous plot of left-hemisphere fMRI signal vs. right-hemisphere fMRI signal in V1 for a single subject and all presentation delays. Time evolution is indicated by the dark arrowheads. Phase delays during onset appear on or below the diagonal (right hemisphere leads left since left hemifield appears first), while phase delays on the falling side of the response appear above the diagonal. (C) Plot of fMRI onset delay for left hemisphere (see Materials and Methods) vs. actual presentation delay. (D) Plot of fMRI latency between hemispheres referenced to the onset of the right-hemisphere activity (see Materials and Methods) vs. actual presentation delay.

Design of Experiment Two.

In a second experiment, we examined the fMRI response onset delay between different vascular beds by using a cued visuomotor reaction time task to examine where in the processing chain between the stimulus detection and the muscle response the RT variations between subjects originated. A gray background with fixation cross was presented for 6 s, a bright-yellow 30° screen (target screen) with a green start box and a red target box was presented for the 2 s, followed again by the gray background and cross for 22 s (Fig. 2 A Inset). Subjects moved a small blue cursor from the start box to the target as rapidly and accurately as possible during the presentation of the yellow screen. Subjects had to hold the cursor in the target box for 1 s, after which the screen again turned gray and the subjects returned the nonmagnetic joystick (Model 521, Measurement Systems, Fairfield, CT) to the start position and held still. During acquisition of a functional data set, the target screen was presented 10 times at the beginning of each regular interval of 30 s (10 single trials). The kinematics of the movements, including position, velocity, and visuomotor RTs, were recorded simultaneously with the fMRI data by using a NI-DAQ card (National Instruments, Austin, TX) in a Macintosh 840 AV Quadra (Apple).

Figure 2.

Visuomotor reaction-time experiment. (A) Activation maps generated by using the cross-correlation approach of the areas invoked by the visuomotor task in a single subject. The activation maps, derived from EPI images with 1-cm slice thicknesses, have been interpolated onto the much thinner anatomic slices. preM, premotor area; M1, primary motor area; SMA, supplementary motor area; V5, motion-sensitive area of visual cortex; V1, primary visual cortex. These areas have been determined anatomically. (B) fMRI responses for 10 trials after bandpass filtering and Fourier interpolation in the four areas chosen for analysis (see Materials and Methods). The visual stimulus that cued subject motor response is shown as an Inset along with the direction of cursor movement. (C) Plot of fMRI onset delay between V1 and SMA (see Materials and Methods) vs. measured reaction time from kinematic trace as well as the onset differences between SMA and M1 vs. RT. Note there are eight points, as two of the six subjects were repeated to assess consistency. In order of increasing kinematic RT, the subjects are LS(2), BG, JD, ML, DB(2), LS(1), DB(1), and ET. The number in parentheses denotes the session number for those scanned twice. Note that RT improved on the second session in both these subjects, demonstrating some practice effect.

Latency maps were generated as described below, and time series of average image intensity as a function of image number were created separately for ROIs containing activated pixels in supplementary motor area (SMA), premotor area (preM), primary motor cortex (M1), and V1 areas of the brain.

Magnetic Resonance (MR) Procedures.

All MRI was performed on a Varian Unity Inova 4 Tesla whole-body research scanner equipped with AS-25 gradients (Varian NMR Instruments, Palo Alto, CA; Siemens, Iselin, NJ). T₁-weighted anatomic images were acquired for all relevant slice planes to overlay the functional data that were acquired with single-shot echo planar imaging (EPI). Data for the hemifield experiments were collected using a 13-cm-diameter quadrature radio frequency (RF) surface coil. Head motion was limited by use of a foam-padded vice. Single-slice, blood oxygenation level-dependent (BOLD) EPI images were acquired along the plane of the calcarine fissure. Typical fMRI EPI parameters were: echo time (TE) = 32 ms, repetition time (TR) = 100 ms, flip angle (FA) = 15°, pixel size = 3.1 × 4.2 × 10 mm. For each presentation delay, 2,750 images were acquired over 10 sequential trials, with the visual stimulus being triggered on the 251st, 501st … etc., image. The first 250 images were used to acclimate the subject in the scanner environment, and were discarded.

For experiment two, fMRI data were collected by using an all-stop birdcage quadrature RF head coil. EPI fMRI images were acquired from three contiguous axial slices selected from near the superior edge of the brain. Two additional slices, parallel to the first three, were located in the visual cortex. The 10 trials were triggered on the 61st, 121st, … etc., volume. In total, 660, five-slice volumes were acquired; 600 during each subject’s performance of the 10 trials of the motor task and 60 as an initial baseline. These first 60 volumes were discarded. Typical fMRI EPI parameters were TE = 32 ms, TR = 80 ms (per slice), FA = 40°, pixel size = 2.0 × 3.9 × 10 mm. The fifth slice acquisition was followed by a delay of 80 ms, giving an effective volume acquisition time of 5 × TR + 80 ms = 480 ms plus a 20-ms data transfer time per volume.

Activation Maps.

For each experiment, the time series of every pixel was bandpass-filtered at the paradigm frequency (i.e., the frequency of single-trial repetition) to eliminate high-frequency physiological noise and low-frequency drift. The bandpass was wide enough to include the first four harmonics of the paradigm frequency to avoid distortion of the fMRI-response waveform. Activated pixels then were identified by using period cross-correlation of each pixel’s time series against a “box-car” reference waveform at the paradigm-repetition frequency, allowing shifts of ±5 s (16). Pixels were deemed activated if they had a correlation coefficient greater than a threshold of 0.5 (for 2,500 images corresponding to a P value <10⁻⁴ corrected for the autocorrelation of the rapidly acquired data). Activated pixels whose mean percentage change due to activation was greater than 6% were excluded to reduce the contribution from large vessels (17, 18). A latency map, in which color was used to code delay relative to the reference vector, was generated. ROIs in anatomically well defined areas (left and right primary V1, M1, preM, and SMA) were drawn and average time courses were generated for these regions (see Fig. 1A for hemifield experiment). Care was taken not to include any visible veins in the ROIs since these exhibit unpredictable delays in response (19).

Timing Analysis.

Fig. 1A shows that between the initial upturn and peak, each fMRI response curve from an ROI is nearly linear. Linear regression was used to fit a line to this section of each curve (typically the region lying between 20 and 70% of the peak height), and the intercept of this line with zero intensity (baseline) was taken to be the onset point. Onset difference is defined to be the time difference between two onset points. Uncertainties in fMRI-detected onset differences were derived from the errors obtained from the linear regression fits, while uncertainties in the RT measurements were derived from the SD of the 10 single trials.

RESULTS

Experiment One.

The averaged response curves from each hemisphere of a single subject for a 500-ms hemifield onset delay is shown in Fig. 1A. To examine which features of the microvascular response could be used to extract timing, we invoked a common engineering methodology known as a Lissajous or X–Y plot. This involves the plotting of one temporally varying signal against another, usually on an oscilloscope. In a simple case, in which two sinusoidal signals are shifted in time, the Lissajous pattern formed is an ellipse, whose major and minor axes can be used to extract quantitatively the phase delay between the two signals. If one of the sinusoidal signals has a larger amplitude than the other, the major axis of the ellipse is tilted off the diagonal. Although the fMRI response is not sinusoidal, it is smooth and continuous, and the rough shape of an ellipse is maintained. Thus, to discern the timing relationship between the activated hemispheres we plotted the left hemisphere fMRI signal as a function of the right hemisphere signal (Fig. 1B). The solid arrows indicate the direction traced out by the Lissajous pattern as the signals from the two hemispheres evolve. If there were no temporal shift between hemispheres and the signal change was equal on both sides, the onset would follow the diagonal, i.e., left and right responses track identically. Deviation away from this diagonal (open arrow) is indicative of a temporal shift between the two time courses. A tilt of the major axis of the curve away from the diagonal indicates an asymmetry in the magnitude of activation between the hemispheres. Clearly, only the onset portion of the curves shows a systematic variation with presentation delay while other measures, such as time-to-peak, peak amplitude, or the falling edge of the response, are inconsistent.

In Fig. 1C, we have plotted the fMRI-determined onset latency in left hemisphere V1 relative to the right hemifield stimulus presentation delay time. Clearly, the fMRI-determined onset latency correlates well with presentation delay time of the checkerboard (r² = 0.952) but is time-shifted by a constant hemodynamic delay that varies somewhat between subjects. The mean hemodynamic delay for the positive-going fMRI response is 2.970 ± 0.083 (SD). To reduce the effects of this intersubject latency variance, we measured the left hemisphere V1 onset response relative to the onset response measured in the right hemisphere. Defining the onset time of the right hemisphere response as “time zero” is a procedure similar to aligning peristimulus histograms to a given marker event (20). This results in an improvement in the correlation between right hemisphere response and left hemifield presentation time (r² = 0.999), as shown in Fig. 1D. This is pursued further in the Discussion.

Experiment Two.

Functional magnetic resonance imaging during the visuomotor task detected large clusters of activated pixels in contralateral primary M1 of all six subjects even though the motion duration was <300 ms (Fig. 2A). There was also activation detected in M1 ipsilateral to the moving hand of the lone left-handed subject in the study, consistent with previous published results (21). Bilateral activation of the primary visual cortex and of the middle temporal area also was found. Other motor areas were activated, including preM, SMA, as well as areas in posterior parietal cortex (Fig. 2A). These, too, are consistent with previous neurophysiological and fMRI studies (20–23). The time series shown in Fig. 2B from M1 contralateral to the moving limb, contralateral preM, bilateral SMA, and bilateral V1 was analyzed for hemodynamic onset differences. To allow comparison of the timing of activity from a multislice fMRI experiment, each time series was Fourier-interpolated to obtain a data point every 80 ms, corresponding to the imaging time per slice. Then the time series from each slice was time-shifted an amount corresponding to when it was acquired, relative to the first slice. Average fMRI response curves were then created by summing the 10 trials to create one mean response curve per area, and the absolute onset time from stimulus presentation was determined for each area and for each subject as described in Materials and Methods. Onset times then were recalculated relative to that of V1 in a procedure similar to the hemifield experiments, to control for attentional effects, as indicated in the Discussion.

With the relatively coarse temporal resolution used in this multislice experiment, preM and SMA were not temporally distinguishable and, hence, are used interchangeably in the discussion that follows. However, we were able to decompose the observed V1-to-M1 latency as determined by the fMRI technique into a V1 → SMA delay and a SMA → M1 delay. These are plotted in Fig. 2C versus the RT determined from the kinematic recordings of joystick position. It is evident that the SMA → M1 delay is a constant of the task, while the V1 → SMA delay linearly scales with RT. Given that the intersubject variation in SMA → M1 latency was constant (28 ± 11 ms), we can conclude that the variation in RT between subjects occurs in the pathway between V1 and the motor planning areas and not in the motor program execution subsequent to stimulus detection.

DISCUSSION

In experiment 1, we determined the absolute hemodynamic delay for each subject (Fig. 1C) as well as the relative delay between activation of the left hemisphere and the right (Fig. 1D). The largest contributor to the r² of the fit shown in Fig. 2C was absolute latency variations between successive presentation delays. By normalizing the effective time zero to the onset in the right hemisphere, we were able to improve the r² dramatically. This was true in experiment 2 as well, in which we therefore defined time zero as the onset of fMRI activity in V1. We conclude that for optimum temporal resolution (as defined by the minimum variance in onset time between trials and subjects), attentional, eye position, or vascular variability effects are best controlled for by using an internal marker of activity in the brain as a reference time point and measuring other delays in hemodynamic response relative to the onset time of this internal marker. By doing this alignment procedure, similar to that used in electrophysiology, the variability becomes comparable to single-unit work (20).

The visuomotor experiment illustrates the power of our latency-resolved fMRI technique. In addition to obtaining static maps of the areas activated by the task, we are able to decompose their fMRI activation latencies into those that co-vary with RT and those that are constants of the task performance. However, this is a relative measure of fMRI latency between areas. To compare the fMRI-determined onset differences with the true absolute neural firing delay between V1 and M1, two factors need be considered. First, the onset of electrical activity in V1 is delayed by about 30 ms relative to visual stimulus onset (6, 7). Similarly, a latency of about 120 ms between M1 firing and movement onset has been reported (20, 22). Data from Wise et al. (20) suggest that the average time between discharge of cells in M1 and the onset of movement is 113 ± 60 ms, in a task where the measured RT was 266 ± 29 ms. Second, latency and RTs were measured by Georgopoulos et al. (22) while investigating the evolution of population vectors in the motor cortex of a rhesus monkey. They report a change in the length of the population vector of 125 ± 28 ms after the stimulus onset and that the movement began 260 ± 30 ms after stimulus onset. This suggests an upper limit of 135 ± 58 ms to the average time between discharge of cells in M1 and the onset of movement. Thus, if the delay in hemodynamic response to neural activity was, on average, the same in V1 and M1, the fMRI-determined onset delay should be about 150 ms less than the measured RT. However, we find that the fMRI-determined onset difference between V1 and M1 was almost identical to the measured RT (Fig. 2C, where the V1 → M1 delay is the sum of the V1 → SMA delay and the SMA → M1 delay).

One possible explanation for our skewed V1 → M1 result may be that the vascular response is faster in V1 than in M1. The exact mechanism coupling cerebral hemodynamic responses to neural activity remains elusive, but it is thought to involve a signaling mechanism between the neurons and the capillary endothelium, possibly involving astrocytes (24–26), which therefore would be sensitive to the capillary–neuron distance. It is known that a close relationship exists between capillary density in different brain structures and their local blood flow and metabolism (27). The primary motor cortex has a lower capillary density (28) than the striate cortex (29). Additionally, the neuronal density in V1 of monkeys is 50–100% greater than in M1 (30, 31). Since both neurons and capillaries are denser in V1, the average distance between neurons and capillaries is significantly shorter, resulting in a shorter diffusion time for the messenger molecule to travel from the neuron to the capillary. This would also be true if the messenger involved astrocyte–capillary endothelium interactions. Thus, the hemodynamic response, as detected by fMRI, may be hypothesized to be faster in V1 compared with M1, lengthening the hemodynamically determined latency relative to the electrical one and, consequently, limiting the use of this method in determining absolute timing. More work on the functional and anatomical relationships between neurons and their supporting cells obviously is needed.

We have demonstrated that the tight coupling of the microvascular response to the electrical activity of the brain can be exploited for studies of mental chronometry by using fMRI. Many unresolved issues remain with respect to the vascular bed variability and MRI scanner field strength. Within the same vascular bed, the fMRI results correspond exactly to the stimulus presentation timing, even across hemispheres and even when using repetition time values that are relatively long compared with the actual latency values. Between different vascular beds, the fMRI results correlate well with RT, but are offset by a constant possibly because of differences in the functional and anatomic relationships between cells and the microvasculature in the two areas. This may limit the utility of the technique in terms of absolute differences, but for experiments in which incremental scaling of a processing parameter is used [e.g., rotation angle (2)], useful information on timing among regions and its relationship to performance may be obtained. High-field fMRI measurements have considerable contrast-to-noise advantages relative to clinically available fields (32). This allows flexibility in experimental design by allowing various permutations of higher signal-to-noise ratio, shorter repetition times, whole-head RF coils, shorter experiments, and true single-trial studies without averaging. Nonetheless, because most higher-order cognitive tasks are fairly slow in their execution, it may be possible to get whole-brain coverage with sufficient signal by using standard clinical scanners. The early single-trial papers are promising in this regard (12, 13). With the repetition rates and the averaging used to date, the ability to extract timing information between brain areas should be feasible at the more commonly available 1.5 Tesla field strength.

The results presented here suggest that by focusing on the onset of the vascular response, the sequence of neural events during complex functional and cognitive tasks may be revealed using high spatial resolution techniques such as fMRI. The latency-resolved fMRI method does not attempt to extract absolute timing differences, which undoubtedly vary because of differences between the neurovascular coupling in different brain areas. Instead, we examine the fMRI signal to determine onset latencies that co-vary with stimulus timing or psychophysical measurements to determine which brain areas are involved in task-related processing and which brain areas are constants of the task.

ABBREVIATIONS

fMRI

functional MRI

EPI

echo planar imaging

ROI

region of interest

preM

premotor area

M1

primary motor area

SMA

supplementary motor area

V5

visual motion area

V1

primary visual cortex

RT

reaction time

EP

evoked potential