Attentional modulation of effective connectivity from V2 to V5/MT in humans

Abstract

The nonlinear nature of integration among cortical brain areas renders the effective connectivity between them inherently dynamic and context-sensitive. One emerging architectural principle of functional brain organization, which rests explicitly on these nonlinear interactions, is that neuronal responses expressed at any level in a sensory hierarchy reflect an interaction between (i) bottom up “driving” afferents from lower cortical areas and (ii) backwards “modulatory” inputs from higher areas that mediate top-down contextual effects. A compelling example is attentional modulation of responses in functionally specialized sensory areas. The aim of this work was to demonstrate that parietal regions may mediate selective attention to motion by modulating the effective connectivity from early visual cortex to the motion-sensitive area V5/MT. Using functional magnetic resonance imaging, and an analysis of effective connectivity based on nonlinear system identification, we found that backwards modulatory influences from the posterior parietal cortex are sufficient to account for a significant component of attentional modulation of V5/MT responses to “driving” inputs from V2. By explicitly modeling interactions among inputs to V5/MT, we were able to make inferences about the influences of V2 inputs and their concomitant activity-dependent modulation by parietal afferents. The latter effects embody dynamic changes in effective connectivity that may underlie attentional mechanisms. These results speak to the context-sensitive nature of functional integration in the brain and provide empirical evidence that attentional effects may be mediated by backwards connections, of a modulatory sort, in humans.

Imaging neuroscience has firmly established functional specialization as a principle of brain organization in humans (1). The functional integration of specialized areas has proven more difficult to assess (2–8). Functional integration is usually inferred on the basis of correlations among measurements of neuronal activity. However, correlations can arise in a variety of ways: For example, in multiunit electrode recordings, they can result from stimulus-locked transients evoked by a common input or can reflect stimulus-induced oscillations mediated by synaptic connections (4–6). Integration within a distributed system is usually better understood in terms of effective connectivity: Effective connectivity refers explicitly to the influence that one neural system exerts over another (7), either at a synaptic (i.e., synaptic efficacy) or population level. It has been proposed (6) that “the [electrophysiological] notion of effective connectivity should be understood as the experiment- and time-dependent, simplest possible circuit diagram that would replicate the observed timing relationships between the recorded neurons.” This speaks to two important points: (i) Effective connectivity is dynamic: i.e., activity- and time-dependent; and (ii) it depends on a model of the interactions. To date, the models used in functional neuroimaging have been linear (8, 9). There is a fundamental problem with these models because they assume that the multiple inputs to a region are linearly separable. This assumption precludes activity-dependent connections that are expressed in one sensorimotor or cognitive context and not in another. The resolution of this problem lies in adopting nonlinear models that include interactions among inputs. These interactions can be construed as a context- or activity-dependent modulation of the influence that one region exerts over another, where that context is instantiated by activity in further brain regions that exert modulatory effects. It follows that nonlinear models are necessary for a proper characterization of contextual changes in effective connectivity.

This work addresses the modulation of visual cortical responses by attentional mechanisms (10) and the mediating role of activity- or context-sensitive changes in effective connectivity. “The expression of attention in the brain appears to be described effectively as an enhancement of activity in the attended set of pathways relative to the unattended set” (11). We used attention to visual motion to examine the modulatory effect of higher areas on the effective connectivity among lower visual areas in humans. Area V5/MT (humans) or MT (non-human primates) receives parallel inputs from the lateral geniculate via V1 and V2, and extrageniculate pathways involving the pulvinar (12). V5/MT is specialized for motion (2, 13, 14). Neuroimaging (15–18) and unit-electrode recordings (10) show that V5/MT responses can be modulated by attention. A likely source of this modulation is the posterior parietal cortex (PPC) (19–21). PPC is part of a distributed system, subserving visual attention, that includes the frontal eye fields, cingulate cortex, prefrontal cortex, thalamic (pulvinar), and other regions. These areas have been implicated on the basis of electrophysiological studies (21), retrograde labeling (22), and lesion studies (23). For example, area 7a (PPC) unit firing is enhanced by attentive fixation (21). The frontal eye fields have been implicated in attention by ablation of areas 6 and 8, engendering the “premotor theory” of attention (24). Anatomically V5/MT and other motion-sensitive areas such as V3a (25) are reciprocally connected to V2. PPC is densely interconnected with V5/MT (26). Reciprocal projections from the PPC include areas 6 and 8 (frontal eye fields), frontal operculum, and area 46 (prefrontal cortex) (27–30). The specific hypothesis we wanted to test was that PPC exerts a backwards modulatory influence over the forwards driving connections from V2 to V5/MT. To do this, we had to adopt a nonlinear model of effective connectivity that incorporated this modulatory effect.

The model we used is based on nonlinear system identification. We assume that the activity in one region can be “explained” by a nonlinear convolution or filtering of the dynamics in regions that contribute inputs. The implicit nonlinear mixing of inputs over time allows source regions to “cause” responses in a target that may be more enduring than the input or, indeed, delayed. Furthermore, the nonlinearities allow for inputs to interact and sensitize the target to other inputs, again with any time course. This convolution model can, in principle, model any driving or modulatory effects that distant regions exert over a target and characterizes them in terms of Volterra kernels (31). Because the kernels are high-order they embody interactions among inputs. The influence of one region j on another i can therefore be divided into two components: (i) the “driving” influence of j on i, irrespective of the activities elsewhere; and (ii) an activity-dependent component that represents an interaction with inputs from the remaining regions. These two aspects of effective connectivity correspond to terms that: (i) involve only activity in area j; and (ii) the remaining terms that model an interaction between activity in j and other regions. By using low-order approximations, the kernels can be estimated by using ordinary least squares which, in turn, facilitates the use of standard inferential statistics. Connections (driving or modulatory) are assessed by testing the null hypothesis that the associated kernel is zero. This can be repeated at every voxel in some prespecified target region (e.g.V5/MT) to produce a statistical parametric map (32–34) that reflects the significance of the connection's influence. When making inferences about modulatory effects one is testing the null hypothesis that the interactions among inputs, causing a response in an area, are negligible. It is these interactions that distinguish the nonlinear model from an equivalent linear model. In this sense a significant modulatory effect implies that a linear model is not sufficient to explain the observed response. Having established that an effect is significant, the influence of remote regions (either driving or modulatory) can be characterized by using simulated inputs and the estimated kernels.

This approach differs from existing approaches [e.g., structural equation modeling (8, 18) and regression analyses (9)] in that the model parameters do not explicitly identify the effective connections but are used to characterize them in terms of responses to simulated inputs. Another important aspect of the current framework, which distinguishes it from conventional approaches to effective connectivity, is that it does not characterize a “network” but deals with the response of a single region given the inputs or causes of that response. This allows one to address very directed questions or hypotheses about regionally specific responses given the neurophysiological context in which they are expressed. In contradistinction to other approaches, the emphasis is not on estimating the parameters of an assumed connectivity architecture but on making statistical inferences about the integration of multiple inputs to a single area that elaborates a response.

Methods

The Nonlinear Model.

Neuronal systems are inherently nonlinear and lend themselves to modeling by dynamical systems. However, it is generally difficult to identify the appropriate analytic equations. An alternative is to take a generic model and obtain the specific parameters that enable it to describe the system in question (35). A common example of this approach is the use of Volterra series to model the nonlinear transformation of some inputs u(t) to an output y(t):

1

where Ω{u} is a functional Taylor expansion implementing a high-order convolution. y(t) is the hemodynamic response of region i over time, and u = u(t) is an m-vector function [u₁(t), … . u_m(t)]^T representing equivalent measures of activity in m other regions. h_j1… _jn(τ₁,… . τ_n) is the nth order Volterra kernel. Eq. 1 can be expressed in matrix form as the second order approximation:

2

where h is a constant, h = h(τ) is a column m-vector of first order Volterra kernels, H = H(τ₁, τ₂) is an m × m matrix of second order kernels, and * is the convolution operator. This equation is a general linear model with response variable y(t) and explanatory variables u. The unknown parameters are the kernels and can be estimated by using ordinary least squares, after expansion in terms of temporal basis functions as described by Friston et al. (31). The basis functions used are dictated by the nature of the time-series analyzed. For high temporal acuity data, like electroencephalographic or neuromagnetic time-series, a large number of basis functions might be appropriate, allowing the fine temporal structure of neuronal interactions or effective connectivity to be characterized. However, in fMRI, the underlying neuronal time-series is temporally “blurred” by the hemodynamic response function (31), and we can use a very parsimonious model (equivalent to two basis functions, a delta function and its temporal derivative): Assuming that the length of the kernels are small in relation to temporal smoothness of u(t) imposed by the hemodynamic response function, we can substitute the first order approximation u(t − τ) ≈ u(t) − τ.u̇(t) into Eq. 1, giving

3

where

4

Here the explanatory variables are [1, ũ(t)^T, ũ(t)^T ⊗ ũ(t)^T], where ⊗ is the Kronecker tensor product.§ ũ(t) represents activity measurements (and their derivatives) in the source regions. The estimation of g, γ, and Γ, which embody the effective connectivity, reduces to a simple regression problem in the context of serially correlated fMRI time-series (32, 33).

To understand the role played by the parameters of effective connectivity, consider connectivity C_ij{u} as the differential response in region i to a hemodynamic transient s(t) = [0,… . . s(t),… 0]^T in region j, for a fixed profile u of activities over regions:

5

Eq. 5 means that the effect of inputs can be decomposed into those that are activity-independent γ^Ts̃ + s̃^TΓs̃ = C_ij{0} and those that are activity-dependent ũ^TΓs̃. In fact, Eq. 5 is equivalent to the first order Taylor approximation (noting u̇ = 0):

6

The activity-independent component is simply C_ij{0}, the driving effect of the input that would have been observed in the absence of other inputs and depends on the first-order coefficients and the second-order coefficients that involve j and only j. The remaining second-order coefficients model interactions between the region j and inputs from other regions. ∂C_ij/∂u_k can be interpreted as a modulatory influence of region k on the connectivity from j to i (examples of C_ij{0} and ∂C_ij/∂u_k are given below). The explanatory variables pertinent to this effect are those that involve regions j and k. By partitioning the terms in this way, one can test separately for driving and modulatory effects. Significance is simply tested with the F statistic by treating the driving (or specified interaction) terms as interesting and the remainder as confounds or nuisance variables in a multiple regression analysis for serially correlated data (32, 33).

In summary one can take a highly nonlinear model of how different inputs interact to produce a response in a target area and, through sensible approximations, linearly separate the response into driving and modulatory components. The contribution of these components to the observed response, at any voxel, can be estimated by using standard regression techniques. One nice thing about the use of regression analysis is that the estimation of, and inferences about, the influence of any input are based on dynamics that are unique to the target and source. This follows because the effects of other sources and interactions are implicitly removed during the least squares estimation, eschewing the problem of common inputs. For example, the influence of one input will be discounted if it can be explained by another correlated input. This of course assumes that all of the relevant variables have been included in the model. It should be noted that the techniques used here are subject to the same qualifications as linear models of effective connectivity; namely, that the validity of the model is inherently determined by the validity of the architecture assumed (i.e., which regions and connections are included).

Experimental Design and Data Acquisition.

Subjects were studied with fMRI under identical stimulus conditions (visual motion subtended by radially moving dots) while manipulating the attentional component of the task (detection of velocity changes). The data were acquired from five young normal subjects at 2 Tesla by using a Magnetom VISION (Siemens, Erlangen, Germany) whole body MRI system, equipped with a head volume coil. Contiguous multislice T₂*-weighted fMRI images were obtained with a gradient echo-planar sequence (TE = 40 ms; TR = 3.22 s; matrix size = 64 × 64 × 32; voxel size = 3 × 3 × 3 mm). Each subject had four consecutive 100-scan sessions comprising a series of 10-scan blocks under five different conditions D F A F N F A F N S. The first condition (D) was a dummy condition to allow for magnetic saturation effects. F (Fixation) corresponds to a low-level baseline in which the subjects viewed a fixation point at the center of a screen area 17° in diameter. In condition A (Attention), the subject viewed 250 dots moving radially from the center at 4.7° per second and were asked to detect changes in radial velocity. In condition N (No attention), the subjects were asked simply to view the moving dots. In condition S (Stationary), the subjects viewed stationary dots. The order of A and N was swapped for the last two sessions. In all conditions, the subject fixated the center of the screen. In a prescanning session, the subjects were given five trials with five speed changes (reducing to 1%). During scanning, there were no speed changes. No overt response was required in any condition. We validated our manipulation of attentional set by showing that the motion after-effect was significantly more enduring in the attention conditions relative to the no attention conditions after scanning (18).

Data Analysis.

The data from each subject were analyzed separately as a series of case studies. First, the regions showing motion-sensitive responses and attentional modulation of these responses were identified in a conventional analysis. The results of this first analysis were used to identify the regional activities that entered into the second effective connectivity analysis.

The data were analyzed by using statistical parametric mapping (SPM96 Wellcome Department of Cognitive Neurology). The time-series were realigned, were corrected for movement-related effects, and were spatially normalized by using co-registered structural T₁ scans (37, 38). The data were smoothed in space (6-mm isotropic Gaussian kernel) and time (√8 second Gaussian kernel). Condition-specific effects were assessed by using multiple regression for serially correlated data (32, 33). Each condition was modeled as a box-car function convolved with a canonical hemodynamic response function. The statistical model included global and low frequency confounds. Comparisons among conditions were effected with the appropriate contrast of the condition-specific parameter estimates to give statistical parametric maps (SPMs) of regionally specific effects (32–34).

To examine the influences of PPC on the forward cortical and subcortical inputs to V5/MT, we specified the anatomical model depicted in Fig. 1. In this model, there are direct effects from V2 and the pulvinar and an interaction between V2 and PPC representing the modulatory effect of interest. The dynamics of regions contributing inputs to V5/MT (i.e., V2, pulvinar, and the PPC) were characterized in terms of the first principal component of the adjusted (for the effects of confounds) data from voxels within an 8-mm spherical volume of interest centered on the maximum of the appropriate SPM for each subject (see Table 1). Only voxels surviving an F threshold of P = 0.001 in the first SPM analysis were considered. This is a form of averaging that properly reflects the correlations among voxels to give a suitable spatial weighting over the volume of interest. The normalized (to zero mean and unit variance) dynamics of each region, of the first subject, are shown in Fig. 2 and demonstrate a progressive attentional modulation from lower to higher areas: The periodic response of V2 reflects successive periods of photic stimulation followed by fixation of a single point. During some of these periods the subject was asked to detect changes in the speed of the stimuli (that never actually occurred). The enhanced responses in V5/MT and the eight most pronounced peaks in the PPC time-series coincide with these attentive conditions (marked with an asterisk).

Figure 1.

Schematic depicting the influences included in the nonlinear model of effective connectivity. The model includes driving (with linear and nonlinear terms) cortical (V2) and subcortical (Pul) inputs (arrows). Given the relatively slow speed of the stimuli used (4.7° per second), we anticipated that the functionally expressed input to V5/MT would derive primarily from V2 (12). Consequently, we modeled a modulation of this input by backwards afferents from posterior parietal cortex (PPC) (thick line).

Table 1.

Location of the specified regions (x, y, and z in millimeters) in the standard space defined by Talairach and Tournoux (38) and the significance of the modulatory effect of PPC on V5/MT responses to V2 inputs (see Fig. 3)

Subject	1	2	3	4	5
Region definition
Pul	21 −27  9	21 −30   0	24 −27 − 3	27 −27   3	24 −30   3
V2	9 −93 12	12 −99   9	12 −90   9	12 −96   0	9 −96   3
V5	51 −60  3	51 −72 − 6	51 −54 − 9	49 −57 − 9	54 −72 − 6
PPC	27 −72 60	38 −51  51	30 −48  63	24 −66  57	33 −68  60
Statistical inference
Fmax	11.24	11.84	14.64	33.99	10.89
Location	51 −63  0	54 −74 −12	48 −51 − 9	48 −54 −12	54 −72 − 6
Voxels	75	64	43	72	33
P uncorrected	0.001	0.001	<0.001	<0.001	0.001
P corrected (34)	0.054	0.036	0.008	<0.001	0.029

By virtue of the region definition and spatial smoothing, the cortical region (V2) subsumes portions of V1. Similarly, the subcortical region (Pul) probably includes a contribution from both the pulvinar and the lateral geniculate nucleus. F_max corresponds to the largest F value in V5/MT. The effective degrees of freedom (corrected for serial correlations) were approximately 2 and 179 (33). The location of the voxel with the largest F value, the number of voxels comprising the V5/MT region, and the associated P values are provided for each subject. 

Figure 2.

Dynamics of the regions used in the analysis: Data from the first subject are shown in terms of the voxels used (white areas on a standard structural MRI scan) and the associated time-series. The time-series from the pulvinar (Pul), V2, and PPC were used in the subsequent analysis of modulatory effects and correspond to u(t) in the text.

The ability of these inputs and their interactions to predict the responses of every voxel within the V5/MT volume of interest were assessed with the appropriate SPM of the F statistic. In this analysis, the response variable y(t) in Eq. 1 was the voxel-specific activity in V5/MT voxels, and the explanatory variables u(t) were constructed by using the regional dynamics of V2, pulvinar, and the PPC illustrated in Fig. 2. Inferences about the significance of modulatory effects were made by considering the all second order terms involving PPC and V2 as interesting and the remainder as confounds. Corrected P values were assigned by using the maximal F value in the voxels comprising the V5/MT region (34). In principle, we could have included all brain voxels in this analysis, but the correction for multiple comparisons makes our restricted search more sensitive.

Results

The conventional analysis, in terms of regionally specific effects and structural equation modeling, has been reported fully elsewhere (18). In summary, attentional modulation of hemodynamic responses to optic flow stimuli were seen in an occipito-parieto-frontal network, including V3a, PPC (BA 7 extending into the intraparietal sulcus), frontal eye fields, anterior prefrontal cortex at the junction of the precentral sulcus and inferior frontal sulcus, and the lateral prefrontal cortex. Using the nonlinear model of effective connectivity described above, we were able to show that the modulatory effect of PPC on V2 inputs to V5/MT was significant and replicated this finding four times in independent studies of further subjects (Table 1) (strictly speaking, the first subject only achieved trend significance after correction for the volume of V5/MT analyzed).

Fig. 3 shows a characterization of this modulatory effect in terms of the increase in V5/MT responses, to a simulated V2 input, when PPC activity is zero (broken lines) and when it is high (solid lines), for the first three subjects. The broken lines in Fig. 3 represent estimates of C_V5,V2{0} according to Eq. 6 and corresponds to a driving effect. In this example, s(t), the simulated input from V2, was a 500-ms square wave convolved with a hemodynamic response function and scaled to unit height. The solid curves represent the same response when PPC activity is unity (i.e., C_V5,V2{0} + ∂C_V5,V2/∂u_PPC). It is evident that V2 has a driving effect on V5/MT and that PPC increases the responsiveness of V5/MT to these inputs. Quantitatively, there is an increase of about 30% in the response to V2 inputs for a unit increase in PPC activity. By virtue of the normalization applied to the time-series, the (dimensionless) activities are expressed in terms of the standard deviation of each region. The inserts show all of the voxels in V5/MT that evidenced a modulatory effect (P < 0.05 uncorrected). Results for the first three subjects are shown. Similar results were obtained for the remaining two subjects.

Figure 3.

Characterization of effects of V2 inputs on V5/MT and the modulation of these responses by PPC using simulated inputs at different levels of PPC activity. The broken lines represent estimates of C_V5,V2{0} according to Eq. 6, in which the V2 input s(t) was a 500-ms square wave convolved with the hemodynamic response function. The solid curves represent the same response when PPC activity is unity (i.e., C_V5,V2{0} + ∂C_V5,V2/∂u_PPC). The insets show all of the voxels in V5/MT that evidenced a modulatory effect (P < 0.05 uncorrected). These voxels were identified by thresholding SPMs of the F statistic testing for the contribution of second order explanatory variables involving V2 and PPC, while treating all others as confounds. Results for the first three subjects are shown.

Conclusions

Our demonstration of attentional modulation of V5/MT responses is compelling insofar as it uses a novel nonlinear characterization of effective connectivity to test a specific hypothesis that was motivated by basic neuroscience findings. It should be noted that the conclusions reached above are constrained inherently by the hypothesis tested. Alternative architectures could have been considered (e.g., those allowing for a modulatory influence of the pulvinar) that may have yielded similar results. In this sense, the analysis described in this report should not be construed as an exploratory characterization of network interactions but represents a test of a specific hypothesis. The most that can be concluded from our analysis is that the activity in PPC is sufficient to explain a significant component of attentional modulation of V5/MT responses. It cannot be concluded that other afferents do not play a role or that PPC modulation is necessary for these effects. A second limitation is imposed by the spatial and temporal resolution of neuroimaging data. This precludes inferences about laminar-specific interactions or anything more refined than large scale population dynamics. However, the wealth of data implicating PPC in an attentional role (19–22, 28) renders the current analysis congruent with a large body of convergent evidence.

Previous approaches to characterizing interactions among cortical areas, as measured by fMRI, use either structural equation modeling (8) or regression techniques (9). We have recently described how structural equation modeling can be extended to deal with nonlinear or modulatory effects by the use of “moderator” variables (18). However, structural equation modeling places restrictions on the number of parameters that can be estimated, making it difficult to include many areas or high-order terms. This is in contradistinction to regression approaches. The first regression analysis of modulatory effects (3), using fMRI, used piece-wise linear regression to examine interactions between V1 and V2. Although appropriate for the questions addressed, this framework only modeled modulation by intrinsic activity and did not allow for modulation by extrinsic inputs. Subsequently, high-order terms were introduced into simple regression models of effective connectivity (36). This enabled hypothesis testing about second-order interactions among inputs to an area, or indeed interactions between experimental factors and activity in a modulating source [referred to as psychophysiological interactions (36)]. However, these models assume that the activity in one region causes a response in another instantaneously. To allow for (i) temporal precedence of influences among cortical regions and (ii) the endogenous variability in the delay of the hemodynamic response, the current Volterra formulation was developed. The Volterra approach explicitly models interactions over time and has been established in the analysis of fMRI time-series from single voxel (31). This paper describes an application of this approach to questions about functional integration.

This report has only addressed the modulation of early connections in the dorsal visual pathway. Clearly, the same principles may apply at subsequent stages. For example backwards connections from prefrontal areas may modulate V5/MT inputs to the PPC (18). This leads to the notion of a serially coupled hierarchy, where each stage is driven by lower stages but, at the same time, modulates these driving areas. This has important implications for conceptual and mathematical models of information processing because the dynamics at any level are some nonlinear function of activity in both lower and higher levels. This precludes serial transformations of the sensory input as assumed in many information theoretic accounts of early visual processing but is much more consistent with a “generative” model approach, as nicely exemplified by Rao and Ballard (39).

In conclusion, we have confirmed the hypothesis that changes in attentional set are associated with augmented responses of V5/MT to driving inputs from V2 and that the activity of posterior parietal cortex is sufficient to account for this modulation. This does not preclude the role of other modulatory effects (e.g., those mediated by cortico-thalamic loops), nor does it imply that parietal modulation is necessary. However these results clearly show that parietal influences are sufficient to explain a significant component of attentional modulation in V5/MT.

Abbreviations

PPC

posterior parietal cortex

SPM

statistical parametric map