A temporal decomposition method for identifying venous effects in task-based fMRI

Abstract

The spatial resolution of functional magnetic resonance imaging (fMRI) is fundamentally limited by effects from large draining veins. Here, we describe a new analysis method that provides data-driven estimates of these effects in task-based fMRI. The method involves fitting a simple one-dimensional manifold that characterizes variation in response timecourses observed in a given dataset, and then using identified early and late timecourses as basis functions for decomposing responses into components related to the microvasculature (capillaries and small venules) and the macrovasculature (large veins), respectively. We show the removal of late components substantially reduces the superficial cortical depth bias of fMRI responses and helps eliminate artifacts in cortical activity maps. This powerful method provides insight into the origins of the fMRI signal and can be used to improve the spatial accuracy of fMRI.

Introduction

Among the handful of noninvasive techniques that permit the study of human brain activity, functional magnetic resonance imaging (fMRI) based on blood oxygenation level dependent (BOLD) contrast has emerged as the most widely used approach in cognitive neuroscience. A primary advantage of fMRI over other measurement techniques is its spatial resolution. However, efforts to increase the spatial resolution of fMRI—especially to reach the sub-millimeter scale of mesoscopic brain organization—face a major challenge imposed by the “draining vein” confound, first noted early in the history of fMRI¹. Venous effects may appear as activation displaced from the original site of neural activity by as much as 4 mm² and may reflect neural activity pooled over large spatial scales, thus degrading spatial specificity^3–6. The field has long sought to measure BOLD responses from the microvasculature (capillaries and small venules) while avoiding BOLD responses from the macrovasculature (large veins)^7–9. The problem of the macrovasculature is especially critical to resolve given the growing popularity in the neuroscience community of using fMRI to probe laminar-specific responses^10–12.

To avoid the specificity loss caused by veins, the field has traditionally turned to the use of spin-echo acquisition at ultra-high magnetic fields^8,13 instead of conventional gradient-echo acquisition. However, spin-echo involves increased energy deposition, longer volume acquisition times, and lower BOLD contrast-to-noise ratio. Thus, in order to maintain measurement sensitivity, the experimenter is generally forced to reduce spatial coverage and/or substantially increase the amount of data collected per experimental condition. These unappealing complications are often dealbreakers for neuroscientists, given that measuring multiple brain regions is often critical, the sensitivity of fMRI is already relatively low to start with, and increasing the duration of an experiment beyond more than a factor of two or so is often impractical.

Here, we introduce an analysis method, called Temporal Decomposition through Manifold Fitting (TDM), that identifies and removes venous-related signals from task-based fMRI data. The TDM method is simple, principled, and is compatible with a variety of experimental protocols including those based on gradient-echo acquisitions. We demonstrate TDM on visual experiments conducted in human subjects (BOLD fMRI at 7T and 3T), and show that TDM consistently removes unwanted venous effects while maintaining a reasonable level of sensitivity. The data used in this paper (both raw and pre-processed), code implementing TDM, and a video tutorial are freely available at https://osf.io/j2wsc/.

Results

TDM provides a method for visualizing timecourse variation

In our experimental datasets (summary provided in Supplementary Figure 1), we replicate prior observations^14–17 that BOLD timecourse amplitude, delay, and width increase as one moves from inner to outer cortical depths and as one moves from lighter to darker voxel intensities (Supplementary Note 1, Supplementary Figures 2–3). These effects are consistent with the interpretation that large draining veins (which tend to reside near the pial surface and cause signal dephasing) lead to changes in BOLD timecourses, and set the stage for the TDM method.

The first step in TDM is to visualize distributions of response timecourses in a low-dimensional space (Figure 1). Specifically, we use principal components analysis (PCA) to determine the three orthogonal timecourses that account for the most variance in the given timecourses. We then use these three timecourses as axes of a 3D space (PC1, PC2, PC3) in which each of the original timecourses corresponds to one point, or vector, in this space. To visualize the results, we map the timecourse vectors to the unit sphere (by normalizing them to unit length) and use an orthographic projection to visualize the density of the timecourses. Such a visualization reveals commonly occurring timecourse shapes, independent of timecourse amplitude (see ‘Density’ image in Figure 1). We separately visualize the amplitudes of the timecourses by computing the lengths of the original timecourse vectors and repeating the orthographic visualization (see ‘Vector length’ image in Figure 1).

Figure 1. Schematic of the TDM method.

Time-series data are fit with a finite impulse response model to estimate response timecourses. PCA is performed on the timecourses to reduce their dimensionality to three. Using orthographic projection in the direction of the first PC, a 2D histogram image is calculated (density). The same projection and binning scheme is used to calculate an image representing timecourse amplitude (vector length). The two images are combined and fit with a 2D Gaussian in order to determine an early timecourse and a late timecourse that together summarize the principal axis of variation. Finally, the time-series data are re-fit with a model incorporating the two timecourses.

Applying these visualization procedures to a representative dataset (similar results are obtained in every dataset; see Extended Data Figures 1–2), we find that timecourse shapes typically reside near the pole of the unit circle where PC1 is maximal, with some variability around this pole (Figure 2A, left). We find that timecourse amplitudes are large in a similar portion of the space except for a small extension towards the lower left (Figure 2A, middle). A separate plot shows the actual timecourses associated with the three PCs that define the axes of the space (Figure 2B). This plot reveals that timecourse shapes generally resemble a canonical hemodynamic response timecourse (Figure 2B, black line), with a major axis of variation corresponding to different loadings on a timecourse that shifts the peak either earlier or later in time (Figure 2B, dark gray line).

Figure 2. TDM captures timecourse variation along a 1D manifold.

Panels A–C show results for Dataset D9 (all datasets shown in Extended Data Figures 1–2). A, Quantities of interest. TDM calculates a density image indicating frequently occurring timecourse shapes (left), a vector-length image indicating timecourse amplitudes (middle), and an EPI-intensity image indicating bias-corrected EPI intensities (right). The black line indicates the identified 1D arc that connects the Early and Late timecourses. B, Timecourses. All timecourses are unit-length vectors. C, Gaussian fitting procedure. Density and vector-length images are DC-subtracted, scaled, and truncated, producing regularized images (upper left, upper right). These images are then averaged (lower left) and fit with an oriented 2D Gaussian (lower right). D, Dimensionality of response timecourses (Datasets D1–D12). We perform PCA on split-halves of each dataset and assess how well a limited number of PC timecourses from one half reconstruct timecourses measured in the other half.

TDM identifies an axis of timecourse variation

The second step in TDM is to identify an axis that captures the major variation in the observed response timecourses. As seen earlier (Figure 2A), response timecourses empirically occupy a small portion of the 3D space. Furthermore, the timecourses can be approximated in the 3D space by a simple line segment defined on the sphere (arc) that characterizes variation with respect to timecourse delay, i.e. early vs. late. The interpretation we adopt here is that (i) the endpoints of the line segment correspond to latent hemodynamic timecourses associated with the microvasculature and the macrovasculature and (ii) any single observed timecourse is simply a mixture of these two latent timecourses plus measurement noise (which causes deviation away from the line).

To calculate the axis of variation, TDM combines density and vector length (Figure 1, lower left), fits a 2D Gaussian to the result (Figure 1, bottom), and extracts points positioned at plus and minus one standard deviation along the major axis of the fitted Gaussian (Figure 1, bottom, red and blue points). Examining results obtained on the representative dataset, we see that the procedure works well: the combined image resembles both density and vector length (Figure 2C, bottom left), the fitted Gaussian approximates the data (Figure 2C, bottom right), and the extracted points reside in sensible locations (Figure 2A, first two plots). We reconstruct timecourses corresponding to the two extracted points (Figure 2B, red and blue lines), and then label the timecourses ‘Early’ and ‘Late’ based on the time-to-peak of the reconstructed timecourses. Note that different mixtures of the Early and Late timecourses trace out an arc on the unit sphere (Figure 2A, black line) and result in a continuum of timecourse shapes (Figure 1, bottom).

We interpret the Early and Late timecourses as reflecting the microvasculature and macrovasculature, respectively. We offer several lines of reasoning that suggest the validity of this interpretation. First, we find that the Late timecourse is consistently associated with large vector length (see Figure 2A, middle plot and Extended Data Figure 1), indicating that response timecourses resembling the Late timecourse tend to have large BOLD amplitudes. This makes sense given that veins exhibit large percent BOLD signal changes^1,16. Second, if we use the same visualization methods (orthographic projection of the unit sphere) to examine the relationship between timecourse shape and bias-corrected EPI intensity, we find that the Late timecourse is consistently positioned in a zone of low EPI intensity (see Figure 2A, right plot and Extended Data Figure 1). Since the TDM procedure does not make use of EPI intensities, this is an empirical finding that provides further evidence of validity, as it is known that veins cause static susceptibility effects in EPI images^1,4,18. Third, the idea that veins exhibit delayed BOLD responses is consistent with several previous experimental studies^14–17. Finally, a biophysical model of vascular dynamics has been proposed, and this model provides a potential explanation for why temporal delays occur in veins¹⁹. Additional control analyses further support the validity of TDM (see Supplementary Note 2).

Summary of TDM timecourses across subjects

For a comprehensive assessment of results, we plot the TDM-derived Early and Late timecourses obtained for each dataset (Figure 3A, thin lines). We see that on the whole, the timecourses are stereotyped and largely consistent across datasets. However, there is substantial quantitative variability, consistent with the well-established observation that BOLD timecourses vary across subjects^20,21. Computing the group-average Early and Late timecourses (Figure 3A, thick lines), we see that these timecourses are similar in overall shape and differ primarily in their delay and width. However, notice that the timecourses also exhibit differences in the timing and magnitude of the post-stimulus undershoot; this feature is presumably also important for accurate timecourse characterization. We next fit each group-average timecourse using a double-gamma function, and find that smooth parametric functions characterize the empirical results quite well (Figure 3B). Finally, as a point of comparison, we plot the predicted timecourse for a 4-s event using the default double-gamma hemodynamic response function implemented in SPM. Interestingly, this timecourse (Figure 3B, magenta line) coincides extremely well with the group-average Late timecourse (Figure 3B, blue line). This makes sense, given that the default timecourse parameters in SPM were derived from fMRI measurements conducted at low (2T) magnetic field strength²² where the BOLD response is dominated by contributions from large vessels^23,24.

Figure 3. Early and late timecourses found by TDM.

A, Summary of results. We plot unit-length-normalized Early and Late timecourses found in Datasets D1–D12 (thin lines) and their average (thick lines). Dots mark timecourse peaks and timecourse rise and fall times. Horizontal black and gray bars below the x-axis indicate the stimulus duration in the eccentricity and category experiments (3.5 s and 4 s, respectively). B, Group-average results and parametric model fit. Group-average Early and Late timecourses are normalized to peak at 1 (red and blue circles), and are fit with a double-gamma function as implemented in SPM’s spm_hrf.m (red and blue lines). The fitting was achieved by convolving a double-gamma function with a 4-s square wave and minimizing squared error with respect to the data. Estimated double-gamma parameters for the group-average Early and Late timecourses were [7.21 17.6 0.5 4.34 1.82 –3.09 50] and [5.76 21.6 1.11 1.72 3.34 0.193 50], respectively. As a comparison, we show the timecourse obtained using default SPM parameters [6 16 1 1 6 0 32] (magenta line).

Decomposition using TDM timecourses removes artifacts from cortical maps

The final step of TDM involves analyzing the fMRI data using a GLM that incorporates the Early and Late timecourses time-locked to the onsets of each experimental condition. Fitting this GLM produces, for each vertex (or voxel) and condition, an estimate of the BOLD response amplitude from the microvasculature and an estimate of the BOLD response amplitude from the macrovasculature, respectively. These response amplitudes, or betas, can then be used in subsequent analyses according to the goals of the researcher. Note that the Early and Late timecourses are often highly overlapping and quite correlated (in Figure 2A, notice the close proximity of the Early and Late timecourses on the unit sphere). In the GLM, the two timecourses are fit simultaneously to the data and are competing to account for response timecourses observed in the data (see Figure 1, bottom right).

To assess the quality of the Early and Late betas, we generate cortical surface visualizations and compare these against visualizations of betas obtained using a standard GLM that incorporates a single canonical hemodynamic response function time-locked to each condition. We focus specifically on visualizations for datasets D1–D5 which involved presentation of stimuli that vary in eccentricity (see Supplementary Figure 1). This is because studies of the visual system provide well-established ‘ground truth’ expectations for neural activity patterns elicited by stimuli varying in eccentricity: in brief, neurons in early visual cortex respond selectively to stimuli at specific eccentricities, and the preferred eccentricity varies smoothly from the fovea to the periphery along the posterior-to-anterior direction²⁵.

Inspecting results for a representative dataset (Figure 4; other datasets shown in Extended Data Figure 3; quantitative results described in Supplementary Note 3), we find that the Early and Late betas show strikingly different patterns. The Early betas are relatively homogeneous across the cortical surface, relatively flat across cortical depth, and are moderate in size at around 1–4% signal change. In contrast, the Late betas are quite heterogeneous across the cortical surface (sparsely distributed), heavily biased towards outer cortical depths, and are sometimes quite large in size, reaching 10% or more signal change. The observation of additional activations arising in the Late betas is consistent with the fact that the BOLD point-spread function appears larger when sampling late in the BOLD response⁶. Furthermore, comparing the spatial pattern of the Late betas against the spatial pattern of bias-corrected EPI intensities (Figure 4, fourth column), we see a general correspondence between the sparsely distributed locations where very large BOLD responses are observed and regions with dark EPI intensities.

Figure 4. TDM decomposes brain activity patterns into early and late components.

Here we show detailed results for Dataset D1 (summary results for all datasets provided in Extended Data Figure 3). Rows correspond to different cortical depths for a small patch covering primary visual cortex (flattened surface, left hemisphere). Three versions of results are shown: the first (Standard) reflects betas from a GLM incorporating a single canonical HRF, while the next two (Early, Late) reflect betas from a GLM incorporating the timecourses found by TDM. Four types of images are shown: (1) absolute value of betas averaged across conditions, (2) bias-corrected EPI intensities, (3) peak eccentricity quantified as the center-of-mass of the six betas corresponding to different eccentricities, and (4) single-condition activity patterns for the fifth stimulus. Blue, purple, and green arrows mark vertices illustrated in greater detail in Extended Data Figure 4. Thin gray lines are an example of the lines plotted in Figure 5.

We next consider maps of peak eccentricity tuning (Figure 4, fifth through seventh columns). All three versions of the betas (Standard, Early, Late) exhibit the expected smooth large-scale progression from foveal (blue) to peripheral (red) eccentricities as one moves posterior (left) to anterior (right) in early visual cortex. However, the quality or robustness of the eccentricity map is highest for the Standard betas, moderately high for the Early betas, and relatively low for the Late betas. Moreover, for the Late betas, there is a substantial decrease in quality moving from outer to inner cortical depths; this is consistent with the sharp fall-off in the magnitude of betas moving from outer to inner depths, as seen previously (see Figure 4, third column). We also observe that although large-scale eccentricity patterns are similar across the three versions of the betas, the maps show divergence at a fine scale. In particular, there are artifacts in eccentricity tuning that are present in the Standard and Late betas but absent in the Early betas.

To further clarify these results, we examine activity patterns elicited by a single experimental condition (Figure 4, eighth through tenth columns). Based on known tuning properties of early visual cortex^25,26, we expect a relatively compact ‘stripe’ of positive activity extending along the superior-inferior direction. All versions of the activity pattern (Standard, Early, Late) indeed show evidence of a stripe. However, only the Early version exhibits a well-behaved stripe that is relatively homogeneous within its spatial extent and relatively flat across cortical depth. For detailed illustrations of how TDM is able to de-mix Early and Late timecourses, please see Extended Data Figures 4–5.

Line profiles and comparison to simpler methods

For quantitative and more comprehensive assessment of single-condition activity patterns, we extract line profiles along iso-angle contours in primary visual cortex (V1) and show results from all subjects (Figure 5A). This analysis also provides an opportunity to directly compare against results of a simple alternative approach for avoiding responses from the macrovasculature, namely, sampling early timepoints in evoked BOLD responses^6,27. The results show that activity patterns from Standard and Late exhibit large and idiosyncratic responses that are highly variable across subjects and biased towards outer cortical depths. In contrast, activity patterns from Early exhibit more focal activations that are consistent across subjects and relatively homogeneous across depth. For the timepoint-based approach, we use timecourse estimates provided by a finite impulse response (FIR) model and examine responses at 2, 3, 4, and 5 s after trial onset. We find that at 2 s, responses are overly weak. At 5 s, responses are strong but closely resemble the large and idiosyncratic responses observed in the Standard and Late analyses. Intermediate timepoints 3 s and 4 s perform somewhat better at avoiding the idiosyncratic responses, but are weaker in magnitude.

Figure 5. TDM outperforms a simple timepoint-based analysis.

A, Line profiles. Each subplot shows BOLD activity evoked by the fourth eccentricity stimulus for 11 iso-angle lines in right hemisphere V1 (see thin gray lines in Figure 4). Each line indicates the mean across datasets (D1–D5); gray shaded regions indicate standard error across datasets. Responses are substantially larger at outer cortical depths for the Standard, Late, and FIR (5 s) analyses (red arrows), and often have large artifacts (blue arrows). In contrast, activity profiles from Early, and to some extent FIR (4 s), are more homogeneous across depth and more compact in shape (green arrows). B, Similarity analysis. We computed split-half correlations between activity profiles produced by different analysis approaches. The main plot shows the group average result; inset plots show results for individual datasets.

To summarize performance of the different methods, we quantify in a split-half analysis the similarity of activity profiles produced by all of the analysis approaches as well as additional analyses that either use a single hemodynamic response function (HRF) matched to the TDM-derived Early timecourse or a single HRF matched to the TDM-derived Late timecourse (Figure 5B). This analysis reveals that Early responses (location i; r = 0.76) are more reliable than FIR (3 s) responses (location ii; r = 0.63). Although Early responses (location i; r = 0.76) are less reliable than FIR (5 s) responses (location iii; r = 0.88), the FIR (5 s) responses resemble both Early and Late responses (location v; r = 0.61, r = 0.57), and thus cannot be interpreted as avoiding effects from the macrovasculature. Even FIR (3 s) responses have fairly equitable balance between Early and Late responses (location iv; r = 0.55, r = 0.31). This is not surprising given that macrovascular timecourses already exhibit substantial rise 3 s after trial onset (see Figure 3). We also observe that the analyses involving a single HRF produce responses that are quite similar to the Standard responses (location vi; r = 0.95, r = 0.97), and that using the TDM-derived Early timecourse as the single HRF still produces responses that reflect substantial influence of both Early and Late responses (location vii; r = 0.58, r = 0.66).

We acknowledge that the split-half analysis does not directly quantify the accuracy of the activity profiles generated by the different methods (e.g., against a quantitative ground-truth measure). Nevertheless, the analysis still reveals valuable insights. Specifically, the results indicate that although sampling early timepoints can help avoid macrovascular responses, TDM outperforms this simple timepoint-based analysis in terms of specificity (avoidance of the macrovasculature) and sensitivity (reliability of response estimates). In addition, the results indicate that the benefits of TDM come from simultaneously including both Early and Late timecourses in the model, and are not obtained by simply modifying the HRF used in a conventional GLM.

Discussion

Novel contributions

Our analysis approach can be viewed as comprising two distinct components. The first is our data-driven method for deriving meaningful Early and Late timecourses. This method is fully automated and principled (as opposed to a heuristic method that might attempt to quickly find a few “fast” voxels), and might be valuable in and of itself for extracting hemodynamic timecourses for comparison across brain regions, individuals, and/or groups. The second is the application we demonstrate for using these timecourses to estimate and reduce venous effects in task-based fMRI.

The value of the work presented here does not lie in the discovery of a new phenomenon: the idea that veins carry delayed responses is not novel^14,16,17, nor is the idea that early responses have the potential to be more spatially specific^6,16,28–30. However, making an observation regarding a phenomenon and having a robust method that can exploit that observation in practice to generate biologically informative results are two very different contributions. Our work introducing the design and validation of new analysis methodology primarily falls in the latter category. We provide code implementation of our algorithms, and we supply substantial empirical evidence that the algorithms produce correct results.

It is important to note that TDM incurs some loss in sensitivity due to correlation between Early and Late timecourses (see Supplementary Note 3). Nonetheless, we believe TDM has distinct advantages over other methods that have been proposed for dealing with venous effects in fMRI^{5,6,13,27,31–40} (see Supplementary Discussion).

Nature of the TDM method

Algorithmically, TDM uses a manifold-fitting method to characterize latent structure in timecourse variations. There are other methods that can characterize latent structure; two widely used methods are principal components analysis (PCA) and independent components analysis (ICA). Could these methods have been used instead? Due to the orthogonality constraint in PCA, it is necessarily the case that the PC timecourses returned by PCA are orthogonal. Although TDM does make use of PCA to determine the 3-dimensional space within which to perform further analyses, the PCA timecourses themselves do not constitute good candidates for latent timecourses. This is simply because there is no reason to expect hemodynamic timecourses in the brain to be orthogonal. Indeed, the bulk of empirically measured timecourses tend to reside in a small portion of the 3-dimensional space, and the early and late timecourses returned by TDM are nearby in this space and highly correlated (see Figure 2 and Extended Data Figure 1).

In contrast to PCA, ICA does not impose the constraint of orthogonality. Instead, ICA optimizes timecourses with respect to statistical independence, often through some measure of non-Gaussianity (e.g. kurtosis). We demonstrate that it is possible to construct an ICA-based procedure that can potentially derive early and late timecourses (Extended Data Figure 1). Though the derived timecourses from the ICA-based procedure are sometimes similar to those produced by TDM, there are clear advantages of the TDM method. First, the data visualization and explicit modeling performed by TDM allow the user to evaluate and confirm the data features that give rise to the derived timecourses. ICA, without further analysis, remains a ‘black box’ and it is difficult to understand the specific features of the data that give rise to its results. Second, there is no a priori reason to think that loadings on early and late hemodynamic timecourses must necessarily conform to statistical independence. Thus, relying on a procedure that is predicated on independence seems risky. Third, ICA alone does not identify the early and late timecourses; rather, we found it necessary to couple the results of ICA with several post-hoc procedures that are heuristic in nature and thus unsatisfying. On the whole, we suggest that TDM is more explicit, more direct, and more interpretable than ICA. Indeed, historically, the first method that we developed was the ICA-based procedure, and the shortcomings described above are what prompted us to develop the TDM method.

GLM-based analyses of fMRI time-series data sometimes allow flexible modeling of timecourse shape through the inclusion of a canonical hemodynamic response timecourse and its temporal derivative²² or some other basis function decomposition such as PCA⁴¹. While TDM shares the common feature of providing a means to capture timecourse variation, the key difference with respect to these alternative approaches lies in the specific timecourses that are chosen by TDM. The Early and Late timecourses found by TDM are often quite correlated (unlike a timecourse and its derivative or those returned by PCA). Moreover, the Early and Late timecourses have specific biological meanings, and so the beta loadings found for these timecourses have specific value. It is possible that alternative timecourse models can yield fits to a set of data that are as good as the fit achieved by TDM, but the beta loadings associated with these models cannot be interpreted in terms of the microvasculature and macrovasculature.

Validation of TDM

In this study, we demonstrated that TDM delivers robust and meaningful results in each of the 16 fMRI datasets collected (11 unique subjects). These datasets included not only high-resolution gradient-echo acquisitions but also spin-echo and low-resolution acquisitions (see Supplementary Note 4). The main lines of validation include sensible timecourses (the shapes of the Early and Late timecourses are plausible and consistent with simple inspections of response timecourses; see Supplementary Figures 2–3), co-variation of loadings on the Late timecourse with dark EPI intensities (see Figure 4 and Extended Data Figure 1) and with kurtotic BOLD amplitudes (see Extended Data Figure 6B), flattening of depth-dependent response profiles (see Extended Data Figure 6D), and reduction of artifacts in cortical maps for which we have ground-truth expectations (see Figures 4–5). Furthermore, we make freely available data and analysis code to ensure that the TDM method is transparent and reproducible⁴².

Efforts to further assess and validate the TDM method would nonetheless be useful. Further work could be directed at assessing and optimizing the technique with respect to experimental design characteristics such as the duration of experimental conditions, the spatial and temporal resolution of the acquisition, and the amount of data acquired. In addition, it would be worthwhile to test the technique on other types of experiments (other sensory, cognitive, and/or motor experiments) and other brain areas. It would be interesting to assess how well TDM can resolve fine-scale variation in neural representations, such as ocular dominance columns⁴³. Since TDM makes no restrictions on the spatial loadings of the Early and Late timecourses, the technique should in principle be applicable not only to large-scale neural representations like eccentricity but also fine-scale representations like ocular dominance.

Online Methods

Subjects

Eleven subjects (five males, six females; age range 19–37; one subject, S1, was an author (K.K.)) participated in the experiments described in this study. All subjects had normal or corrected-to-normal visual acuity. Informed written consent was obtained from all subjects, and the experimental protocol was approved by the University of Minnesota Institutional Review Board.

We conducted four experiments. Experiment E1 measured responses to eccentricity stimuli using a high-resolution (7T, 0.8 mm) gradient-echo protocol. Experiment E2 measured responses to category stimuli also using the high-resolution gradient-echo protocol; data from this experiment are the same as described in a previous publication⁴. Experiment E3 measured responses to the same eccentricity stimuli in E1 but used a spin-echo protocol (7T, 1.05 mm). Experiment E4 measured responses to the same eccentricity stimuli in E1 but used a low-resolution (3T, 2.4 mm) gradient-echo protocol.

A total of sixteen datasets (scan sessions) were collected: five corresponding to Experiment E1; seven corresponding to Experiment E2; two corresponding to Experiment E3; and two corresponding to Experiment E4. To facilitate direct comparison, Experiments E3 and E4 were conducted in subjects who also participated in Experiment E1. A full breakdown of subjects, experiments, and datasets is provided in Supplementary Figure 1.

Stimulus presentation

For the 7T datasets, stimuli were presented using a Cambridge Research Systems BOLDscreen 32 LCD monitor positioned at the head of the scanner bed (resolution 1920 × 1080 at 120 Hz; viewing distance 189.5 cm). For the 3T datasets, stimuli were presented using a NEC NP4100 DLP projector that was focused onto a backprojection screen positioned at the head of the scanner bore (resolution 1024 × 768 at 60 Hz; viewing distance 102 cm). Subjects viewed the monitor or backprojection screen via a mirror mounted on the RF coil. A Mac Pro (7T) or iMac (3T) computer controlled stimulus presentation using code based on Psychophysics Toolbox^44,45. Behavioral responses were recorded using a button box.

Experimental design

In the eccentricity experiment (Experiments E1, E3, E4), stimuli consisted of rings positioned at six different eccentricities (i.e. distances from the center of gaze), and were confined to a circular region with diameter 11°. Each ring was filled with a black-and-white contrast pattern that updated at 10 Hz. Ring size scaled with eccentricity, and rings were presented on a neutral gray background (Supplementary Figure 1). Stimuli were presented in 4-s trials. In a trial, one of the six rings was presented for 3.5 s (35 images presented sequentially, each with duration 0.1 s) and was followed by a brief gap of 0.5 s. Each run lasted 368.116 s and included 12 presentations of each of the 6 rings as well as blank trials (also of 4-s duration). Throughout stimulus presentation, a small semi-transparent dot (50% opacity) was present at the center of the stimulus. The color of the dot switched between red, white, and black every 1–5 s, and subjects were instructed to maintain fixation on the dot and to press a button whenever the color changed. A total of 9 runs were collected in each 7T scan session, and a total of 6 runs were collected in each 3T scan session. (The eccentricity experiment was time-locked to the refresh rate of the LCD monitor, which caused the additional 116 ms in the total run duration. To compensate for this slight offset, we pre-processed the fMRI data for the eccentricity experiment at a sampling rate of 1.000316 s, and then, for simplicity, treated the data in subsequent analyses as if the sampling rate was exactly 1.0 s.)

The category experiment (Experiment E2) was the same as the ‘functional localizer’ experiment conducted in a previous paper⁴. This experiment (http://vpnl.stanford.edu/fLoc/) was developed by the Grill-Spector lab⁴⁶. Stimuli consisted of grayscale images of different semantically meaningful categories. There were 10 categories, grouped into 5 stimulus domains: characters (word, number), body parts (body, limb), faces (adult, child), places (corridor, house), and objects (car, instrument). Each stimulus was presented on a scrambled background and occupied a square region with dimensions 10° × 10°. Stimuli were presented in 4-s trials. In a trial, 8 images from a given category were presented sequentially, each with duration 0.5 s. Each run lasted 312.0 s and included 6 presentations of each of the 10 categories as well as blank trials (also of 4-s duration). Throughout stimulus presentation, a small red fixation dot was present at the center of the stimulus. Subjects were instructed to maintain fixation on the dot and to press a button whenever they noticed an image in which only the background was present (“oddball” task). A total of 10–12 runs were collected in each scan session.

MRI data acquisition and pre-processing

Acquisition and pre-processing procedures are the same as described in a previous paper⁴, except for the addition of a spin-echo acquisition protocol. A summary of all procedures is provided below, and we refer the reader to the previous paper for details.

Acquisition

MRI data were collected at the Center for Magnetic Resonance Research at the University of Minnesota. Some data were collected using a 7T Siemens Magnetom scanner equipped with SC72 body gradients and a custom 4-channel-transmit, 32-channel-receive RF head coil. Other data were collected using a 3T Siemens Prisma scanner and a standard Siemens 32-channel RF head coil. Head motion was mitigated using standard foam padding.

Anatomical data were collected at 3T at 0.8-mm isotropic resolution. We used a whole-brain T1-weighted MPRAGE sequence (TR 2400 ms, TE 2.22 ms, TI 1000 ms, flip angle 8°, bandwidth 220 Hz/pixel, no partial Fourier, in-plane acceleration factor (iPAT) 2, TA 6.6 min/scan) and a whole-brain T2-weighted SPACE sequence (TR 3200 ms, TE 563 ms, bandwidth 744 Hz/pixel, no partial Fourier, in-plane acceleration factor (iPAT) 2, TA 6.0 min/scan). Several T1 and T2 scans were acquired for each subject in order to increase signal-to-noise ratio.

Functional data for Experiments E1 and E2 were collected at 7T using gradient-echo EPI at 0.8-mm isotropic resolution with partial-brain coverage (84 oblique slices covering occipitotemporal cortex, slice thickness 0.8 mm, slice gap 0 mm, field-of-view 160 mm (FE) × 129.6 mm (PE), phase-encode direction inferior-superior (F ≫ H in Siemens’ notation), matrix size 200 × 162, TR 2.2 s, TE 22.4 ms, flip angle 80°, echo spacing 1 ms, bandwidth 1136 Hz/pixel, partial Fourier 6/8, in-plane acceleration factor (iPAT) 3, multiband slice acceleration factor 2). Gradient-echo fieldmaps were also acquired for post-hoc correction of EPI spatial distortion (same slice slab as the EPI data, resolution 2 mm × 2 mm × 2.4 mm, TR 391 ms, TE1 4.59 ms, TE2 5.61 ms, flip angle 40°, bandwidth 260 Hz/pixel, no partial Fourier, TA 1.3 min). Fieldmaps were periodically acquired over the course of each scan session to track changes in the magnetic field.

Functional data for Experiment E3 were collected at 7T using spin-echo EPI at 1.05-mm isotropic resolution with partial-brain coverage (64 (or 48 for Dataset D14) slices, slice thickness 1.05 mm, slice gap 0 mm, field-of-view 128 mm (FE) × 111.2 mm (PE), phase-encode direction inferior-superior (F ≫ H in Siemens’ notation; Dataset D13 was reversed H ≪ F), matrix size 122 × 106, TR 2.2 s, TE 39 ms, flip angle 90°, echo spacing 1 ms, bandwidth 1138 Hz/pixel, partial Fourier 6/8, in-plane acceleration factor (iPAT) 2, multiband slice acceleration factor 2). Corresponding gradient-echo fieldmaps were also acquired.

Functional data for Experiment E4 were collected at 3T using gradient-echo EPI at 2.4-mm isotropic resolution with partial-brain coverage (30 slices, slice thickness 2.4 mm, slice gap 0 mm, field-of-view 192 mm (FE) × 192 mm (PE), phase-encode direction anterior-posterior (A ≫ P in Siemens’ notation), matrix size 80 × 80, TR 1.1 s, TE 30 ms, flip angle 62°, echo spacing 0.55 ms, bandwidth 2232 Hz/pixel, no partial Fourier, no in-plane acceleration, multiband slice acceleration factor 2). Corresponding gradient-echo fieldmaps were also acquired.

Pre-processing

T1- and T2-weighted anatomical volumes were corrected for gradient nonlinearities, co-registered, and averaged (within modality). The averaged T1 volume (0.8-mm resolution) was processed using FreeSurfer⁴⁷ version 6 beta (build-stamp 20161007) with the -hires option. We generated 6 cortical surfaces spaced equally between 10% and 90% of the distance between the pial surface and the boundary between gray and white matter, increased the density of surface vertices by bisecting each edge, and truncated the surfaces retaining only posterior cortex in order to reduce memory requirements. The resulting surfaces are termed ‘Depth 1’ through ‘Depth 6’ where 1 corresponds to the outermost surface and 6 corresponds to the innermost surface. Cortical surface visualizations were generated using nearest-neighbor interpolation of surface vertices onto image pixels.

Functional data were pre-processed by performing one temporal resampling and one spatial resampling. The temporal resampling consisted of one cubic interpolation of each voxel’s time-series data; this interpolation corrected differences in slice acquisition times and also upsampled the data to 1.0 s (Supplementary Figure 1). Data were prepared such that the first time-series data point coincides with the acquisition time of the first slice acquired in the first EPI volume. The motivation for upsampling is to exploit the intrinsic jitter between the data acquisition and the experimental paradigm. The spatial resampling consisted of one cubic interpolation of each volume; this interpolation corrected head motion (rigid-body transformation) and EPI distortion (determined by regularizing the fieldmaps and interpolating them over time) and also mapped the functional volumes onto the cortical surface representations (affine transformation between the EPI data and the averaged T2 volume). Note that the temporal correction is applied first, and then, based on the resulting temporally corrected volumes, the spatial correction is applied. Although the topic of optimal ordering is out of scope of the current paper, we believe that performing temporal correction first is most appropriate in datasets where head motion is relatively low.

After pre-processing, the data consisted of EPI time series sampled every 1.0 s at the vertices of the depth-dependent cortical surfaces (Depth 1–6). As a final pre-processing step, for the purposes of identifying vertices affected by venous susceptibility effects, we computed the mean of the EPI time-series data obtained for each vertex and divided the EPI intensities by a fitted 3D polynomial (up to degree 4); this produced bias-corrected EPI intensities that can be interpreted as percentages (e.g. 0.8 means 80% of the brightness of typical EPI intensities).

GLM analysis

We analyzed the pre-processed time-series data using three different GLM models (FIR, Standard, TDM).

The first GLM model, termed FIR (finite impulse response), is a GLM in which separate regressors are used to model each time point in the response to each experimental condition⁴⁸. Results from this model are used as inputs to the TDM method. The FIR model characterized the response from 0 s to 30 s after condition onset, yielding a total of 31 regressors for each condition. (Modeling the response to 30 s was sufficient to capture the majority of the hemodynamic responses; see Figure 3 and Supplementary Figure 3.) We divided the trials for each experimental condition into 2 groups using a “condition-split” strategy⁴, thereby producing two estimates for each response timecourse. Fitting the FIR model produced BOLD response timecourses (timecourses of betas) with dimensionality N vertices × 6 depths × M conditions × 31 time points × 2 condition-splits where N is the number of surface vertices for a given subject and M is the number of conditions in the experiment.

The second GLM model, termed Standard, is a GLM in which a canonical hemodynamic response function (getcanonicalhrf.m) is convolved with condition onsets to create a regressor for each experimental condition. We used six condition-splits, thereby producing six response estimates for each condition. Fitting the Standard model produced BOLD response amplitudes (betas) with dimensionality N vertices × 6 depths × M conditions × 6 condition-splits.

The third GLM model, termed TDM, is a GLM in which two hemodynamic timecourses (Early, Late) are separately convolved with condition onsets to create two regressors for each experimental condition. The exact nature of these timecourses is determined by the TDM method as described below. We used six condition-splits, thereby producing six response estimates for each combination of condition and timecourse. Fitting the TDM model produced BOLD response amplitudes (betas) with dimensionality N vertices × 6 depths × M conditions × 2 timecourses × 6 condition-splits.

GLMs were prepared and fit to the data using GLMdenoise^49,50. In GLMdenoise, the GLM consists of experimental regressors (which may take on different forms, as described above), polynomial regressors that characterize the baseline signal level in each run, and data-derived nuisance regressors. In the case of the FIR model, experimental regressors consisted of binary values (0s and 1s). In the case of the Standard and TDM models, experimental regressors consisted of the convolution of condition onsets (1s) with hemodynamic timecourses that are normalized to peak at 1 (for example, see Supplementary Figure 2). After fitting the GLMs, estimated betas were converted from raw scanner units to units of percent BOLD signal change by dividing by the mean signal intensity observed at each vertex and multiplying by 100.

Betas were further analyzed using simple summary metrics. To quantify overall BOLD activity at a given vertex, we calculated mean absolute beta (e.g. Figure 4, left) by averaging betas across condition-splits, taking the absolute value of the results, and then averaging across conditions. To summarize responses to the eccentricity stimuli, we calculated peak eccentricity (e.g. Figure 4, middle) by averaging betas across condition-splits, performing positive half-wave rectification (i.e. setting negative values to zero), and then calculating center-of-mass⁵¹. Specifically, center-of-mass was calculated as the weighted average of the integers 1–6 (corresponding to the 6 ring eccentricities from fovea to periphery) using the rectified betas as weights.

Timecourse quantification and metrics

In some analyses (see Supplementary Figures 2–3), we summarize the typical timecourse shape observed in a set of timecourses. This was accomplished using a PCA-based procedure (derivehrf.m). In the procedure, we first subtract off the mean of each timecourse. We then perform principal components analysis (PCA) on the entire set of timecourses and extract the first PC (this is the timecourse vector along which variance in the total set of timecourses is maximized). Next, we add a constant offset to the first PC such that the first time point equals 0, and, if necessary, flip the sign of the PC such that the mean over the range 0–10 s is positive. Finally, we calculate the weight that minimizes squared reconstruction error for each timecourse, compute the absolute value of these weights, and then scale the PC by the average weight. The motivation for de-meaning the timecourses prior to PCA is to suppress low-frequency noise present in weak BOLD responses. In general, the use of PCA for summarizing timecourse shape⁵² has advantages over simply computing the mean timecourse: PCA elegantly handles negative BOLD timecourses, and PCA allows timecourses with larger BOLD responses to have greater influence on the resulting timecourse shape (thereby producing more robust results).

Several timecourse metrics were computed (see Supplementary Figures 2–3, Figure 3). Given a timecourse, we upsampled the timecourse to a sampling rate of 0.01 s using sinc interpolation. We then identified the maximum of the resulting timecourse (peak amplitude) and its associated time (time-to-peak). We used linear interpolation to calculate the time at which the timecourse rises to half of the maximum value (rise time) and the time at which the timecourse falls to half of the maximum value (fall time). Finally, we computed the time elapsed between the rise time and the fall time (full-width-at-half-max or FWHM).

TDM method

Theory

TDM is a data-driven technique that identifies a principal axis of timecourse variation present in a set of experimentally measured timecourses. It does this by examining timecourses projected into a low-dimensional space defined by the first three principal components of the timecourses and extracting a one-dimensional manifold—specifically, an arc on the unit sphere—that captures the variation of interest. The procedure can be viewed as a powerful method for summarizing and extracting the signal present in timecourses which considered individually (i.e. one response timecourse at a time) would likely be insufficiently reliable. In our fMRI measurements of responses to 4-s visual stimuli, we consistently find that one endpoint of the line corresponds to an early timecourse peaking at around 5–7 s and the other endpoint of the line corresponds to a late timecourse peaking at around 6–9 s (see Figure 3A, Supplementary Figures 2–3). These timecourses are interpreted as reflecting hemodynamic responses from the microvasculature (capillaries and venules) and hemodynamic responses from the macrovasculature (veins), respectively. TDM then uses the identified timecourses in a regression model in order to decompose observed hemodynamic responses into early and late components. The researcher can choose to analyze further the early component, the late component, or both.

There are three main quantities involved in the TDM technique: density, referring to the timecourse shapes that tend to be present in the data; vector length, referring to the amplitudes of the timecourses in the data; and EPI intensity, referring to the bias-corrected EPI intensity of the vertex (or voxel) to which each timecourse belongs. (Note that these three quantities are distinct from the three principal component dimensions.) TDM combines density and vector length and fits an oriented 2D Gaussian to the result in order to identify the 1-dimensional arc. The motivation for incorporating vector length beyond density alone is to ensure that veins—which generate BOLD responses with large amplitudes but constitute only a fraction of the total set of responses—have sufficient influence on the determination of the arc. Note that EPI intensity does not directly participate in the determination of the arc, and can therefore provide useful validation of the results (see Figure 2A, Extended Data Figures 1–2).

There are a few important conceptual points regarding the nature of the TDM method. For any given voxel (or vertex), the BOLD response to an experimental event is expected to reflect a mixture of early (microvasculature) and late (macrovasculature) timecourses. The specific proportion of these timecourses is expected to vary from voxel to voxel simply due to heterogeneity in the spatial structure of the vasculature (e.g., one voxel might be centered on a large vein, whereas another voxel may only partially overlap the vein). Different proportions of the timecourses manifest in TDM as different points, and these points collectively trace out an arc on the unit sphere (see Figure 1). Empirically, we confirm that a diversity of proportions are observed in response timecourses (Extended Data Figure 5).

Another important point is that TDM is not equivalent to estimating a different hemodynamic response function for each voxel^52–54. Using a single hemodynamic timecourse for different experimental conditions (time-condition separability) yields at most one amplitude estimate (beta) for each condition. In contrast, TDM allows experimental conditions to have different loadings on the early and late timecourses, and yields two amplitude estimates (betas) for each condition (this critical feature is elaborated in Extended Data Figure 4).

Finally, note that the GLM analyses performed in TDM rely on the assumption of linear summation of BOLD responses over time. Our experiments, like many used in cognitive neuroscience, involve a large number of trials that are presented fairly rapidly in order to maximize statistical power (e.g. less than 10 s of rest in between trials). In such experiments, it is a practical necessity to assume temporal linearity. Moreover, nonlinear effects are likely to average out when using randomized experimental designs. Nevertheless, the accuracy with which TDM identifies timecourses may be limited if there exist nonlinear effects⁵⁵ and especially if these nonlinearities vary for different types of vasculature^27,56,57.

Prerequisites

The TDM method requires a task-based experiment in which neural events occur at prescribed times, and is therefore inapplicable to resting-state paradigms, at least in its current form. Despite this, our approach is still valuable to the wide array of task-based fMRI studies being conducted in the field of cognitive neuroscience. We suspect that TDM will be most effective for event-related paradigms where experimental events are somewhat short (e.g. 4 s or less). Block designs involving prolonged events (e.g. 16–30 s) or designs involving continuously changing experimental parameters (e.g. sinusoidal variation of a stimulus property) are likely to generate microvasculature-related and macrovascular-related timecourses that are more similar and therefore harder to disambiguate.

In terms of data acquisition, the TDM method is likely compatible with a broad range of acquisition styles. As we show, TDM can be applied to data from standard spatial resolutions (2–3 mm; see Extended Data Figures 2–3) or data from high spatial resolutions (<1 mm; see Figure 4). Presumably, the spatial resolution needs only to be high enough to allow diverse sampling of vasculature in the brain. With respect to temporal resolution, the temporal requirements of TDM are not stringent: we show that TDM can be successfully applied to data acquired at even fairly slow rates, such as the 2.2-s sampling rate used in Datasets D1–D14. This is likely aided by the fact that we jittered the acquired time points with respect to the experimental conditions (see Supplementary Figure 1).

Because TDM is a data-driven technique, it is necessary to acquire sufficient data to support the method. For example, there must be sufficient data to estimate response timecourses from the voxels in a dataset. If a given dataset is overly noisy or if not enough data are collected, timecourse estimates may be noisy and nearly isotropic in their distributions in the 3-dimensional PCA space (e.g. see Dataset D6 in Extended Data Figure 1), making it difficult to extract the underlying structure of the data.

Algorithm

The TDM algorithm starts with a set of timecourses and determines a pair of timecourses that characterize the overall variation in the timecourses. The following are the steps in the TDM algorithm (extracthrfmanifold.m):

1. Perform PCA on the timecourses. We collect timecourses into a 2D matrix of dimensionality L timecourses × T time points, and then perform singular value decomposition. This produces a matrix with dimensionality T time points × T eigenvectors where columns correspond to principal component (PC) timecourses in decreasing order of variance explained. Since the sign of the returned eigenvectors is arbitrary, we flip the sign of the first PC if necessary to ensure that the mean of the timecourse is positive over the range 0–10 s. Note that previous studies have applied PCA to fMRI response timecourses but in different contexts^41,58.

2. Use PC1–PC3 to define a 3-dimensional space for further analysis. Our convention for visualization is that PC1 points out of the page (positive z-axis), PC2 points to the right (positive x-axis), and PC3 points to the top (positive y-axis).

3. Map timecourses onto the unit sphere. We project each timecourse onto PC1, PC2, and PC3. One complication here is the possibility of negative BOLD timecourses. To first approximation, such timecourses can be treated as a sign-flipped version of positive hemodynamic responses⁵⁹. Thus, we take the coordinates of each timecourse and mirror these coordinates across the origin if necessary to ensure that the loading on PC1 is positive. In this way, negative BOLD timecourses are flipped and treated in the same way as positive BOLD timecourses. (If the user wishes to simply discard negative BOLD timecourses, this can be achieved by setting opt.ignorenegative to 1.) After projection, each timecourse is represented by a set of coordinates (loadings), and can be interpreted as a 3-dimensional vector. We normalize each vector to unit length (thereby placing the vector on the unit sphere), and also save the original vector length for later use.

4. Calculate a 2D image that represents density. We orthographically project timecourses onto the xy plane, and then calculate a 2D histogram. This produces a 2D image where pixel values represent frequency counts. This image indicates typical timecourse shapes found in the data.

5. Calculate 2D images that represent vector length and EPI intensity. Using the same orthographic projection and binning scheme of Step 4, we calculate the median vector length of the timecourses found in each bin. We also calculate the median bias-corrected EPI intensity of the voxels (vertices) associated with the timecourses in each bin. This produces 2D images where pixel values represent vector lengths (indicating timecourse amplitudes) and EPI intensities (indicating static susceptibility effects caused by veins), respectively.

6. Regularize the density image by subtracting a DC bias. We distribute a collection of particles on the unit sphere (S2 Sampling Toolbox, https://www.github.com/AntonSemechko/S2-Sampling-Toolbox/, assign timecourses to their nearest particles, and count the number of timecourses associated with each particle. We then calculate a histogram of these counts and determine the bin B with the highest frequency. Finally, we stochastically subsample the timecourses such that the number of timecourses associated with each particle is reduced by the middle value of bin B. The resulting subsampled timecourses are used to generate a new density image.

7. Scale the density image. The regularized density image from Step 6 is scaled such that 0 maps to 0 and the maximum value maps to 1. Values are then truncated to the range [0,1].

8. Regularize the vector-length image by subtracting a DC bias. This is accomplished in a similar manner as Step 6: we calculate a histogram of the values in the vector-length image, determine the bin B with the highest frequency, and subtract the middle value of bin B from all image pixels.

9. Scale the vector-length image. The regularized vector-length image from Step 8 is scaled such that 0 maps to 0 and the maximum value maps to 1. Values are then truncated to the range [0,1].

10. Average the density and vector-length images. Although the default behavior is to simply average the density and vector-length images, if the user desires a different weighting (e.g. giving more weight to the vector-length image), a flag can be used (opt.vlengthweight).

11. Fit 2D Gaussian. The image resulting from Step 10 is fit with a 2D Gaussian. This choice of model is simplistic but sufficient; for more complex distributions, one might consider the use of principal curves⁶⁰. The Gaussian is controlled by two parameters specifying the center, two parameters specifying the spreads along the major and minor axes, a rotation parameter, a gain parameter, and an offset parameter. In model fitting, the error metric is set up such that the image is interpreted as a probability distribution (pixels with larger values reflect higher density and thus contribute more heavily to the error metric).

12. Extract two points along the major axis of the Gaussian. We determine points corresponding to the mean plus or minus one standard deviation along the major axis of the Gaussian. The choice of one standard deviation is somewhat arbitrary but appears to produce satisfactory results.

13. Reconstruct timecourses corresponding to the identified points. We place the points determined in step 12 on the unit sphere, and use their associated coordinates to weight and sum the PC1, PC2, and PC3 timecourses. This yields two reconstructed timecourses. Based on time-to-peak, we label one timecourse as ‘Early’ and the other timecourse as ‘Late’.

While the algorithm described above has several parameters, default parameter values were used for all datasets in this paper. Some of the parameters are relatively minor and likely do not need adjustment. These include parameters related to constructing histograms and regularizing the images (opt.numspherehistbins, opt.bins, opt.sphereparticles) and a parameter for the temporal range over which to quantify timecourse sign (opt.rng). Other parameters are more significant, and may warrant adjustment. These include the relative weighting of the density and vector-length images (opt.vlengthweight) and the choice of one standard deviation for early and late timecourses (results.fullarc can be used to choose other points along the arc for the timecourses).

Application of algorithm

In our datasets, we obtained timecourses by fitting an FIR model to the fMRI time-series data. We then calculated the amount of variance explained (R²) by the FIR model. In order to focus the TDM algorithm on cortical locations with BOLD responses, we selected all vertices within a given region of interest that exceeded an automatically determined threshold (specifically, the value at which the posterior probability switches between two Gaussian distributions fitted as a mixture model to the data; see findtailthreshold.m). This produced a set of timecourses with dimensionality P vertices × M conditions × 31 time points × 2 condition-splits. We applied the TDM algorithm to the timecourses averaged across the condition-splits (thus, reflecting the entire dataset) and also to the timecourses from each condition-split separately in order to assess reliability. In both cases, the number of timecourses given to the TDM algorithm is L = P*M and the number of time points is T = 31. After completion of the TDM algorithm, the identified timecourses (Early, Late) were incorporated into a GLM model to decompose the fMRI time-series data into early and late components (see GLM analysis).

Region-of-interest (ROI) definition

We used two regions-of-interest (ROIs). For the eccentricity experiment, we used the union of visual areas V1, V2, and V3 from a publicly available atlas of visual topography⁶¹. For the category experiment, we used a manually defined region in occipital, parietal, and temporal cortex that covers visually responsive vertices (same region used in ref. 4). Both ROIs were defined in FreeSurfer’s fsaverage space and backprojected to individual subjects for the purposes of vertex selection.

Line profile analysis

A ‘line-profile’ analysis⁴ was used to quantify and summarize activity patterns observed in the eccentricity experiment. To define a set of lines, we downloaded the publicly available HCP 7T Retinotopy Dataset⁶², took the group-average results for population receptive field angle and eccentricity, and visualized these results on FreeSurfer’s fsaverage sphere surface. We then manually selected on each hemisphere 5 eccentricity × 3 angle = 15 vertices in primary visual cortex (V1) corresponding to the intersection of the vertical and horizontal meridians and eccentricity values 0.3°, 0.8°, 1.8°, 3.3°, and 6.4° (these values are approximately equally spaced along the cortical surface). These fsaverage vertices were mapped to subject-native surfaces via nearest-neighbor interpolation, and used to draw lines corresponding to iso-angle contours (lines are drawn on an orthographic projection of the sphere surface). Each line consists of 4 line segments (one for each successive pair of eccentricity values) and is represented as a sequence of surface vertices. To sample the full extent of V1, we used linear interpolation to create four lines evenly spaced between the vertical and horizontal meridians, yielding a total of 1 (lower vertical meridian) + 4 + 1 (horizontal meridian) + 4 + 1 (upper vertical meridian) = 11 lines representing iso-angle contours in each hemisphere of each subject (see Figure 4 for an example). Cortical distance was quantified using Euclidean distance between pairs of vertices on subject-native white surfaces.

We used the defined lines to generate group-level profiles of BOLD activity (see Figure 5). The core idea is to treat the vertices corresponding to the 5 selected eccentricity values as waypoints for the purposes of intersubject alignment. First, to determine approximate physical units, we computed the distance between successive pairs of waypoints and averaged the resulting values across lines and subjects. This yielded a sequence of averaged distances in millimeter units. Then, for each subject, the sequence of vertices between successive pairs of waypoints was linearly rescaled to match the averaged distances. Finally, beta weights from a given analysis of interest (e.g. Standard) were extracted for each vertex and regridded onto an evenly spaced 0.3-mm grid using linear interpolation. This process ultimately produced sets of activity profiles in V1 that extend from 0.3° to 6.4° eccentricity and that are directly comparable across lines, subjects, and analyses.

To quantify the similarity of different analysis approaches (see Figure 5B), we computed, for split-halves of each dataset, a full set of activity profiles for all lines, stimulus conditions, depths, and hemispheres. These activity profiles were then correlated across each pair of analysis approaches, averaging across resampling cases. (For example, activity profiles from Standard on split 1 were correlated with activity profiles from Late on split 2, activity profiles from Late on split 1 were correlated with activity profiles from Standard on split 2, and the two resulting correlation values were averaged.) The use of split-halves for the similarity analysis is valuable as it provides a measure of reliability for each analysis approach.

Statistics

Reliability of results was assessed using both within-session and across-session analyses, and is depicted by error bars in the various figures. In the within-session case, reliability was assessed by splitting experimental trials into non-overlapping groups (condition-splits), analyzing the groups separately, and then quantifying variability of results across the groups. For the FIR GLM model, two condition-splits were used; for the Standard and TDM GLM models, six condition-splits were used. In the across-session case, reliability was assessed by simply quantifying variability of results across sessions. TDM results were replicated in 16 scan sessions conducted in 11 distinct subjects. For some analyses, robustness of BOLD response amplitudes was quantified using t-values (mean divided by standard error across condition-splits). Effect sizes are expressed using percent BOLD signal change and Pearson’s correlation. Cross-validation was used to assess accuracy of modeling procedures (see Figure 2D and Supplementary Figure 6C). Monte Carlo simulations were used to formally assess the robustness of analysis procedures (see Supplementary Figures 5 and 7).

Reporting Summary

Data were primarily analyzed using custom code written in MATLAB R2018a. Further information on research design is available in the Life Sciences Reporting Summary attached to this article.

Data Availability Statement

Materials related to this paper, including all datasets used, are available at https://osf.io/j2wsc/. Raw data in BIDS format⁶³ are hosted at OpenNeuro at https://doi.org/10.18112/openneuro.ds002702.v1.0.1, whereas pre-processed data (i.e., temporally and spatially corrected fMRI time-series data in surface format) are provided on the OSF site.

Code Availability Statement

The OSF site (https://osf.io/j2wsc/) includes an archive of the code used in this paper, sample data and scripts demonstrating the TDM method, and a link to a detailed video tutorial demonstrating the scripts and discussing the methodology and rationale therein. TDM source code is available at https://github.com/kendrickkay/TDM/ and is licensed under the BSD 3-Clause License.

Extended Data

Extended Data Fig. 1. TDM results for the high-resolution gradient-echo datasets (D1–D12).

Same format as Figure 2, except for the following addition: magenta and cyan crosses indicate the early and late timecourses derived from the ICA-based procedure. TDM consistently identifies reasonable early and late timecourses in each dataset. The ICA-based procedure yields similar timecourses in some datasets (e.g. D4), but diverges substantially in others (e.g. D8). It appears that timecourses with very large BOLD responses (see D8) is a major factor that influences the timecourses returned by ICA.

Extended Data Fig. 2. TDM results for the alternative acquisition protocols (D13–D16).

Same format as Figure 2. To facilitate comparison, we place results obtained using the spin-echo and low-resolution protocols next to results obtained using the high-resolution gradient-echo protocol.

Extended Data Fig. 3. Decomposition of brain activity patterns across datasets and acquisition protocols.

Same format as Figure 4, except only two cortical depths (Depths 1 and 4) are displayed. On the left are results obtained using high-resolution (0.8-mm) 7T gradient-echo (Datasets D1–D5). On the right are results obtained using high-resolution (1.05-mm) 7T spin-echo (Datasets D13–D14) and low-resolution (2.4-mm) 3T gradient-echo (Dataset D15–D16). These alternative acquisition protocols were conducted in the same subjects as the high-resolution gradient-echo protocol (correspondence indicated by arrows).

Extended Data Fig. 4. Detailed inspection of example vertices.

A–C, Results for three surface vertices marked by arrows in Figure 4. At the upper left are FIR timecourses with ribbon center and width indicating mean and standard error across two condition-splits. Dotted lines indicate the overall fit of the TDM model for each condition (reflecting a weighted sum of the Early and Late timecourses). At the lower left are canonical and TDM-derived timecourses. On the right are the three versions of the betas with bars and error bars indicating mean and standard error across six condition-splits and black arrows indicating peak eccentricity. Rainbow colors indicate stimulus eccentricity (1 = most foveal, 6 = most peripheral). Notice that TDM decomposes BOLD responses into two sets of betas (Early, Late) that exhibit different stimulus selectivity, and that FIR timecourses for different experimental conditions have differing shapes, even for the same vertex.

Extended Data Fig. 5. Response timecourses exhibit diverse proportions of early and late timecourses.

Each subplot depicts results for a single condition at a single vertex (Dataset D1). The left shows FIR timecourses (black, with lines and error bars indicating mean and standard error across two condition-splits) and the overall fit of the TDM model (purple). The right shows beta estimates (bars and error bars indicate mean and standard error across six condition-splits). To select which cases to show, we first identified vertices whose R² under the TDM GLM is greater than 10%. We then examined the estimated betas and calculated their t-values (beta divided by standard error across condition-splits). We determined (i) all cases with a robust Early beta (t > 5) and a weak Late beta (absolute value less than 1/10 of the Early beta), (ii) all cases with robust Early and Late betas (t > 5) and where each beta is at least 9/10 of the other beta, and (iii) all cases with a robust Late beta (t > 5) and a weak Early beta (absolute value less than 1/10 of the Late beta). Finally, we randomly selected 20 cases from each of the three groups.

Extended Data Fig. 6. Quantitative assessment of BOLD amplitude estimates provided by TDM.

A, Histogram. The top plot shows distributions of BOLD amplitudes aggregated across Datasets D1–D12; the bottom plot shows results on a log scale and with a wider x-axis range. B, Kurtosis. Results are shown for individual datasets (thin lines, D1–D12) and the group average (thick black line). C, Standard deviation. Same format as panel B. D, Cortical depth profiles. The main plot shows the average depth profile observed in Datasets D1–D12, with ribbons indicating standard error across datasets; the inset plots show results for individual datasets (D1–D16), with ribbons indicating standard error across conditions. E, Reliability. Average correlation of betas across 6 splits of each dataset. Same format as panel D. F, Gradient-echo versus spin-echo. We re-plot results from panels D and E, directly comparing the gradient-echo and spin-echo datasets.