Dynamic adjustments in frontal, hippocampal, and inferior temporal interactions with increasing visual working memory load

Abstract

The active maintenance of visual stimuli across a delay interval in working memory tasks is thought to involve reverberant neural communication between the prefrontal cortex and posterior visual association areas. The hippocampus has also recently been attributed a role in this retention process, presumably via its reciprocal connectivity with visual regions. To characterize the nature of these inter-regional interactions, we applied a recently developed functional connectivity analysis method to an event-related fMRI experiment in which participants performed a delayed face recognition task. As the number of faces that participants were required to remember was parametrically increased, the right inferior frontal gyrus (IFG) showed a linearly decreasing degree of functional connectivity with the fusiform face area (FFA) during the delay period. In contrast, the hippocampus linearly increased its delay period functional connectivity with both the FFA and the IFG as the mnemonic load increased. Moreover, the degree to which participants’ FFA showed a load-dependent increase in its connectivity with the hippocampus predicted the degree to which its connectivity with the IFG decreased with load. Thus, these two neural circuits may dynamically trade off to accommodate the particular mnemonic demands of the task, with IFG-FFA interactions mediating maintenance at lower loads and hippocampal interactions supporting retention at higher loads.

Introduction

Theories of visual working memory¹ (WM) postulate that the prefrontal cortex (PFC) provides top-down signals to high-level visual processing regions in the inferior temporal cortex (ITC) in order to keep neural representations of behaviorally relevant sensory information activated when it is no longer present in the external environment (Fuster 2000; Knight and others 1999; Miller and D’Esposito 2005; Petrides 1994; Postle 2005a). Experimental evidence for PFC involvement in WM is abundant. PFC lesions in monkeys and humans often result in impaired performance on tasks requiring short-term maintenance, such as delayed response and delayed recognition tasks (for review, see Curtis and D’Esposito 2004). Single-unit recordings in primate PFC have identified neurons with sustained firing during the delay period of visual WM tasks (Fuster and Alexander 1971; Miller and others 1996; Rainer and others 1998). It has also been shown that the maintenance of visual information evokes sustained delay period firing of neurons in the ITC (Fuster and Jervey 1982; Mikami and Kubota 1980; Miller and others 1993). Functional magnetic resonance imaging (fMRI) has given researchers the opportunity to measure the blood oxygenation level dependent (BOLD) signal (Ogawa and others 1990), a correlate of neural activity (Logothetis and others 2001), when human participants engage in a WM task. A number of event-related fMRI studies, which separately assess activity levels during the cue, delay, and probe stages of a visual delayed recognition task (Zarahn and others 1997), have identified delay period activity in the PFC (for a review, see Courtney 2004; Curtis and D’Esposito 2003) and ITC (Postle and others 2003; Ranganath and others 2004; Yoon and others 2006). Interactivity between PFC and ITC is suggested by anatomical tract tracing studies, which have demonstrated their reciprocal connectivity (Petrides and Pandya 2002; Ungerleider and others 1989; Webster and others 1994), and by several elegant lesion/electrophysiological studies, which document alterations in ITC activity resulting from a loss of PFC-mediated top-down control (Barcelo and others 2000; Fuster and others 1985; Tomita and others 1999).

A growing body of evidence also indicates that medial temporal lobe (MTL) regions, which show extensive reciprocal connections with ITC (Yoshida and others 2003), contribute to visual memory even over very brief delays. MTL regions, such as the hippocampus, have not traditionally been considered a component of the WM system, since patients with MTL damage typically perform normally on tests requiring only the short-term maintenance of information, while exhibiting pronounced long-term memory impairments (Squire and others 2004). However, in a recent review of the neuropsychological literature on short-term memory, Ranganath & Blumenfeld (2005) concluded that while MTL-damaged patients do well on short-term memory tasks involving simple overlearned materials such as letters, words, or digits, they are often impaired on tasks requiring the maintenance of complex novel visual objects. These impairments exist even with retention delays as short as 2–10 s (Buffalo and others 1998; Holdstock and others 2000; Holdstock and others 1995; Owen and others 1995). In fact, two recent studies found MTL-damaged patients to be significantly impaired on a visual delayed recognition task requiring the maintenance of a single face stimulus across delays as short as 4 s (Olson and others 2006a) and 7 s (Nichols and others 2006). A study by Aggleton and others (1992) found MTL-damaged patients were most impaired at a visual delayed recognition task when the task required the retention of multiple items.

These data on the effects of naturally occurring lesions in humans are corroborated by the results of well-controlled lesion studies in rats and monkeys that demonstrate deficits in spatial and/or object short-term memory following MTL ablations (Lee and Kesner 2003; Murray and others 1989; Olton and Feustle 1981; Olton and others 1982; Raffaele and Olton 1988; Wan and others 1994; Zola-Morgan and Squire 1986). Further evidence for MTL involvement in WM tasks comes from neurophysiological findings of sustained delay period firing of hippocampal neurons in rats (Hampson and others 1993; Wible and others 1986) and monkeys (Cahusac and others 1989; Watanabe and Niki 1985). fMRI studies with humans have also revealed hippocampal activity during the delay period of WM tasks requiring the maintenance of novel visual stimuli (Nichols and others 2006; Park and others 2003; Ranganath and others 2005a; Ranganath and others 2004; Ranganath and D’Esposito 2001; Schon and others 2004), but not during the maintenance of highly familiar faces (Ranganath and D’Esposito 2001) or verbal stimuli, such as letters (Zarahn and others 2005).

Despite the suggestive evidence that the PFC and hippocampus interact with ITC in the service of visual WM, the differential contributions these structures to the maintenance of novel visual representations is unclear. Studying task-dependent changes in the functional interactions between these regions could yield valuable insights into the role they play in WM. In recent years, researchers have begun to take advantage of fact that fMRI records correlates of neural activity virtually simultaneously throughout the entire brain to explore how anatomically disparate brain areas interact during cognitive tasks. Such interactions have typically been characterized by identifying regions that show correlated fluctuations in their fMRI time series data, with the assumption that temporal correlations in BOLD signal reflect synchronous neural firing in communicating, or “functionally connected”, regions. Early fMRI studies of functional connectivity used blocked designs involving the continuous performance of a single task across an extended block of time (e.g., Lowe and others 2000). In order to study functionally connectivity during a particular stage of a multi-component task, such as the delay period of a delayed recognition task, it is necessary to have a method capable of obtaining stage-specific measures of inter-regional correlations in event-related fMRI designs.

A major limitation in using multivariate analysis techniques to probe delay period connectivity in event-related fMRI designs has been the challenge of generating delay period connectivity data that are uncontaminated by slowly evolving hemodynamic signal evoked during the preceding cue period or ensuing probe period. To address this limitation, we recently developed and validated a new multivariate analysis method designed specifically to characterize functional connectivity in an event-related fMRI dataset and measure inter-regional correlations during the individual stages of a multi-stage cognitive task (Rissman and others 2004). The method, beta series correlation analysis, employs a general linear model (GLM) approach (Friston and others 1995b), as do most univariate analyses for estimating stage-specific activity, but adapts the model such that distinct parameter estimates (beta values) are computed for each trial and then used as the dependent data in a correlation analysis. While standard univariate analyses inherently treat trial-to-trial variability as noise, beta series correlation analysis explicitly measures and capitalizes on this variability. If two areas of the brain are functionally interacting with each other during a particular stage of WM, then fluctuations in the amount of activity that the two areas exhibit during that stage should be correlated across trials. The method can be implemented either by selecting a region of interest, or “seed”, and determining the network of regions that correlate with it, or by defining a set of regions and assessing their correlations with each other. Several studies have now successfully employed the beta series correlation analysis method to yield novel insights into the inter-regional interactions occurring during the delay period of WM tasks (Buchsbaum and others 2005; Fiebach and others 2006; Gazzaley and others 2004; Ranganath and others 2005b).

In a recent study, we sought to characterize the WM maintenance network by using beta series correlation method to analyze data from a face WM task, in which participants had to maintain a single face stimulus across a 7–8 s delay period (Gazzaley and others 2004). Participants also performed an independent “localizer” task, in which the fusiform face area (FFA), a focal ITC region that exhibits a high degree of selectivity for faces (Kanwisher and others 1997; Puce and others 1995), was functionally defined in each participant’s right fusiform gyrus. Given the FFA’s important role in face processing, it was of interest to determine which cortical regions interact with the FFA to facilitate face WM. Using the beta series correlation analysis method, the functional connectivity between each participant’s FFA seed and every voxel in the brain was computed separately for the cue, delay, and probe stages of the task. The group-level delay period correlation map revealed a robust network of regions that correlated significantly with the FFA seed, including the dorsolateral PFC (DLPFC), ventrolateral PFC (VLPFC), and hippocampus.

In the present experiment, we seek to refine our understanding of the inter-regional interactions facilitating WM maintenance by examining how this system adapts to accommodate increasing mnemonic demands. By assessing how PFC and hippocampal regions differentially strengthen, sustain, or weaken the strength of their functional coupling with ITC as a function of increasing WM load, we hope to gain valuable insights into the nature of each region’s contribution to the short-term retention of task-relevant visual representations. To this end, we performed a functional connectivity analysis on an existing fMRI data set in which participants performed a visual WM task that involved a parametric manipulation of memory load. The data set has been previously analyzed with univariate methods and these results have been published elsewhere (Druzgal and D’Esposito 2001; Druzgal and D’Esposito 2003; Landau and others 2004). In the experiment, participants were presented with one to four face stimuli on each trial, which they were then required to remember across an 8 s delay interval. Applying beta series correlation analysis to this data set allows us to examine how the strength of connectivity between brain regions that functionally interact with the FFA during the delay period of a face WM task change as a function of the number of faces being maintained.

Materials and Methods

A complete description of the experimental design and scanning protocol can be found in Druzgal & D’Esposito (2003); the critical details are summarized below.

Participants

Ten right-handed participants (age range 22–27 years) were recruited from the University of Pennsylvania Medical Center, and gave written informed consent prior to their participation. All participants were screened against medical, neurological, and psychiatric illnesses, and were not currently taking any prescription medications. Due to reasons expounded upon below (see Region-of-Interest Analysis), the data from one participant were excluded from all analyses.

Experimental Task

Participants performed eight runs of 12 delayed recognition trials for a total of 96 trials (Figure 1). At the start of each trial, participants viewed four serially presented grayscale images, which were a combination of intact and scrambled faces. Each image was displayed for 1 s, and participants were instructed to remember all of the intact faces. On any given trial, one to four of the stimuli were intact faces and the remainder (if any) were scrambled faces. The inclusion of scrambled faces was intended to equate the overall amount of bottom-up visual stimulation across all four load conditions. Both the mnemonic load of each trial and the order in which intact face and scrambled face stimuli were presented were randomized so that the participants did not know how many faces they would have to remember until the end of the encoding period. After the four stimuli were presented, a fixation cross appeared for an 8 s delay period. Finally, a probe face appeared for 2 s, and participants made a key press indicating whether the probe face matched any of the previously presented face stimuli. A 16 s inter-trial interval (ITI) elapsed between trials.

Fig 1.

Structure of the behavioral task. On each trial, four cue stimuli were serially presented for 1 s each. The stimulus set contained one, two, three, or four intact faces, with the remainder of the images composed of scrambled faces. A trial with a memory load of three faces is illustrated here. Participants were instructed to remember all of the intact faces across an 8 s delay period. Following the delay, a probe face appeared for 1 s, and participants made a button press response indicating whether or not the probe face matched any of the cue faces. After responding, participants were instructed to fixate on a crosshair during a 16 s inter-trial interval (ITI).

All face stimuli were unfamiliar to the participants at the onset of the experiment and were cropped to an ovoid shape so that their external features, such as hair and head shape, were not visible. Over the course of the experiment, each face was presented as a cue stimulus an average of 2.5 times.

Prior to performing the working memory task in the scanner, participants performed a “localizer” task in which they passively viewed blocks of face and object stimuli to identify face-sensitive regions of the fusiform cortex (FFA) (Kanwisher and others 1997). None of the face stimuli presented in this task were also used in the main experiment.

fMRI Acquisition and Processing

All functional images were acquired on a 1.5 T General Electric scanner with a gradient-echo EPI sequence (TR = 2000 ms, TE = 50 ms, matrix size = 64 × 64, FOV = 24 cm²). Each functional volume consisted of 21 contiguous 5 mm thick axial slices. fMRI data processing included sinc interpolation in time to correct for between-slice timing differences in image acquisition, motion correction using a six-parameter, rigid-body transformation algorithm (Friston and others 1995a), normalization of the time series of each voxel by its mean signal value, and spatial smoothing with an 8 mm FWHM Gaussian kernel.

fMRI Data Analysis

BOLD responses during the cue, delay, and probe stages of the task were modeled as brief impulses of neural activity convolved with an in-house canonical HRF, obtained by averaging the empirically-derived HRFs (Aguirre and others 1998; Handwerker and others 2004) across a group of participants who performed fMRI experiments on this scanner. To minimize collinearity between temporally adjacent covariates, the delay period was not modeled as boxcar function beginning immediately after the offset of the cue stimuli and extending until the onset of the probe stimulus. Rather, care was taken to ensure that the onsets of temporally adjacent covariates were spaced at least 4 s apart (Zarahn and others 1997). This approach minimizes the contamination of the delay period covariate by residual cue period activity and has been used to successfully model delay period activity in numerous published studies (Barde and Thompson-Schill 2002; Curtis and others 2004; Druzgal and D’Esposito 2003; Pessoa and others 2002; Postle and others 2000; Ranganath and others 2004). Since the cue period entailed 4 s of visual stimulation, the cue period was modeled as the sum of two HRF functions, the first placed at the start of the trial and the second placed 2 s into the trial. The delay period was also modeled as the sum of two HRFs, the first positioned 6 s into the trial and the second positioned 8 s into the trial. The probe period was modeled with an HRF positioned 12 s into the trial.

Functional connectivity analyses were conducted using the beta series correlation analysis method (Rissman and others 2004). Because this approach requires that separate parameter estimates (beta values) be computed for each component of each individual trial, the cue, delay, and probe stages of each individual trial were coded with a unique covariate. This resulted in a total of 288 covariates of interest being entered into the GLM (3 task stages × 4 memory load conditions × 24 trials per condition). The GLM also included covariates of no interest to model the effects of shifting signal levels across runs. Band-pass filters were used to attenuate frequencies above 0.25 Hz and below 0.02 Hz. The GLM was run for each participant using the VoxBo analysis package (http://www.voxbo.org). The least squares solution of the GLM yielded a unique set of 288 beta values for each voxel in the brain. For each voxel, these beta values were sorted by the memory load condition as well as by the task stage from which they were derived to form 12 distinct beta series for that voxel. Each beta series thus reflects the voxel’s estimated activity during a particular task stage of each experimental trial of a given memory load condition. Only beta values from trials for which the participant produced the correct response were included in the beta series. The extent to which two brain regions interact during a particular task stage and memory load condition is quantified by the extent to which their respective beta series from that stage/condition are correlated.

The seven contiguous voxels in each participant’s right fusiform gyrus that exhibited the strongest response preference to faces versus objects in the localizer task, as assessed by a t-test, were defined as that participant’s FFA (Kanwisher and others 1997) and used as seeds in the subsequent correlation analyses. Our choice of seed size was somewhat arbitrary, but based on observations that seeds of this size are more robust to noise than those based on a single peak voxel and more selective to the activity profile of the FFA than larger clusters. We chose to use a right-lateralized FFA seed since lesion, electrophysiological, neuroimaging, and behavioral studies have shown the right hemisphere to play a dominant role in the perceptual analysis and recognition of faces (Bentin and others 1996; Hillger and Koenig 1991; Kanwisher and others 1997; Rossion and others 2003). Stage-specific seed correlation maps were obtained by calculating the correlation of the FFA seed’s beta series (averaged across the seven seed voxels) with that of all brain voxels. Separate beta series, and hence separate correlation maps, were derived for each of the four WM loads in addition to being subdivided by task stage.

To allow statistical conclusions to be made based on the correlation magnitudes, we applied an arc-hyperbolic tangent transform (Fisher 1921) to the correlation coefficients of all brain voxels. Since the correlation coefficient is inherently restricted to range from −1 to +1, this transformation serves to make its null hypothesis sampling distribution approach that of the normal distribution.

In addition to the functional connectivity analysis, a standard univariate analysis was also conducted using the same placement scheme for the cue, delay, and probe covariates described above. However, for this analysis, rather than using a unique set of covariates to model the activity from each individual trial, a single set of covariates was used to model the activity across all of the trials of each load condition.

Region-of-interest analysis

The general strategy of our region-of-interest (ROI) analysis was to identify the regions of the PFC and hippocampus that showed the strongest delay period functional connectivity with the FFA seed, averaged across all mnemonic loads, and then to probe how the functional connectivity of these regions changed as a function of load. In this way, we ensured that our procedure for defining the ROIs was not biased towards finding load effects. We based our ROI definition on the delay period in an effort to keep our investigation focused on the neural mechanisms subserving visual information processing in the absence of bottom-up sensory input. While maintenance processes are initiated during the cue period of the task, these processes are confounded with load-dependent changes in the overall amount of visual attention/processing (since the scrambled face stimuli do not need to be richly encoded). Likewise, connectivity during the probe period is difficult to interpret because of the multiple cognitive operations that must be implemented to process the probe stimulus, make a memory-guided decision, and implement the appropriate motor response. Moreover, we do not have a sufficient number of trials to separately assess inter-regional interactions during the probe period of match and non-match trials, which have been shown to evoke different profiles of brain activity (Druzgal and D’Esposito 2001).

We began by averaging the arc-hyperbolic tangent transformed delay period FFA correlation maps from all four load conditions to create a mean delay correlation map for each participant. ROIs in each participant’s right and left inferior frontal gyrus (IFG), middle frontal gyrus (MFG), and hippocampus were defined as the cluster of 7 contiguous voxels within each region exhibiting the highest average delay period correlation with the FFA seed. Seven voxel ROIs were defined so that the correlations would be computed between two identically-sized regions (since the FFA seed was also defined as a 7 voxel cluster). In order to ensure that a complete set of six ROIs were obtained for each participant, a lower-bound threshold was not applied to the mean correlation maps. Occasionally, it was not possible to isolate a 7 voxel cluster simply by titrating the threshold until one emerged, since sometimes narrow “bridges” formed between adjacent clusters. When this occurred, a mask was drawn around the cluster with the strongest mean correlation level. When multiple clusters were observed within a large anatomical region, such as the IFG, an effort was made to select the cluster that represented the most consistent anatomical location of that cluster across participants (in this case, the more ventral aspect of the IFG). Explicit anatomical masks were used, when necessary, to restrict the functionally-defined ROIs to the specific anatomical regions being interrogated. For the hippocampal ROIs, these anatomical masks were limited to the anterior two-thirds of the hippocampus, since the anterior portion of the hippocampus has previously been implicated as playing a role in visual working memory (Ranganath and D’Esposito 2001; Ranganath and others 2005b).

By defining our ROIs separately for each participant based on their native space correlation map, we sought to maximize the power of our group analysis, given our small sample size. However, since our functionally defined, anatomically constrained ROIs varied slightly across participants in their localization, the regional effects we report cannot be mapped onto precise atlas coordinates. In an effort to provide an approximation of the atlas-based localization of our ROIs, we warped each participant’s ROI mask images into standard Montreal Neurological Institute (MNI) atlas space using the normalization routine from SPM2 (http://www.fil.ion.ucl.ac.uk/spm). Since these mask images consist of 1’s at all locations included in the ROI and 0’s at all other locations, summing the spatially normalized mask images across participants yields a mask overlap map. In such a map, the value of each voxel indicates the number of participants whose normalized ROI mask included that voxel. From these maps, we determined a coordinate of maximal overlap for each of our six ROIs (in brackets), as well as the number of participants whose ROI included this coordinate (in braces): right IFG [52, 30, 8] {6}; left IFG [−45, 48, 8] {5}, right MFG [38, 50, 28] {5}, left MFG [−32, 56, 18] {5}, right hippocampus [26, −12, −18] {6}, left hippocampus [−22, −20, −14] {7}.

For each of the six ROIs, the mean beta series (averaging across the 7 voxels) was extracted for each task stage and load. These beta series were then correlated across trials with the corresponding beta series of the FFA to produce a set of stage-specific correlation values for each memory load condition. Load-dependent changes in each ROI’s delay period correlation with the FFA were then assessed statistically using linear contrast analyses (F tests) across all load conditions and pairwise comparisons (two-tailed paired sample t tests) between the two extreme conditions (“one face” vs. “four faces”), with participants as random factor.

During the process of defining individual participant ROIs based on the mean delay period FFA connectivity map, it became apparent that the correlation data from one participant was substantially different from that of the other 9 participants. Specifically, the participant’s FFA seed correlated strongly with virtually the entire brain volume; 68% of all his brain voxels had a correlation of r > .5 with the FFA seed during the delay period (and correlations were similarly high and non-selective during the cue and probe stages). Given that this value is 6 standard deviations above that of the other 9 participants (who showed an average of 13% of voxels exceeding an r > .5 correlation level) and likely indicates the presence of a global colored noise component, we elected to exclude this participant’s data from all analyses.

Results

Behavioral Data

Group-averaged behavioral data are presented in Figure 2. One participant was excluded from the behavioral analysis due to a hardware problem that prevented the collection of behavioral data. Participants’ accuracy declined significantly as a function of increasing memory load (one face (1F) vs. four faces (4F): t(7) = 11.91, p < .0001; linear contrast: F(1,7) = 71.35, p < .0001). On trials in which participants responded correctly, reaction time increased with mnemonic load (1F vs 4F: t(7) = −5.09, p < .001; linear contrast: F(1,7) = 23.94, p < .005).

Fig 2.

Mean accuracy and reaction time for each of the four load conditions. Error bars represent standard error of the mean (SEM).

Functional Connectivity Analysis

Of the four PFC ROIs tested (right and left IFG and MFG), only the right IFG showed a significant effect of memory load on its delay period correlation with the FFA (Figure 3). Specifically, this region’s correlation with the FFA decreased significantly from the 1F condition to the 4F condition, and there was a significant linear component to this decline (1F vs. 4F: t(8) = −3.15, p < .05; linear contrast: F(1,8) = 7.10, p < .05). In contrast, both the left and the right hippocampus ROIs revealed a pattern of increasing correlations with the FFA with increasing load. However, this effect achieved statistical significance only in left hippocampus ROI (1F vs. 4F: t(8) = −2.77, p < .05; linear contrast: F(1,8) = 8.55, p < .05) (Fig. 3). The right hippocampus ROI showed a marginally significant load-related increase in its correlation with the FFA (1F vs. 4F: t(8) = −1.88, p = .097), but the linear trend analysis was not significant (p = .24) (data not shown). The differential effect of load on the functional connectivity patterns of the right IFG and left hippocampus was further supported by a significant ROI × load interaction (F(3,24) = 3.19, p < .05).

Fig 3.

FFA delay period correlations as a function of mnemonic load. The mean correlation (Pearson’s r) of the right FFA’s delay period beta series with that of the right inferior frontal gyrus (IFG) and left hippocampus ROIs is depicted for each face load condition (load is indicated by the numbers below each bar). The FFA shows a linear decrease in its correlation with the right IFG and a linear increase in its correlation with the left hippocampus; the pairwise differences between the “1 Face” and “4 Faces” conditions are also significant for both ROIs (all ps<.05). Error bars indicate SEM.

To illustrate how this changing functional connectivity profile manifests itself in the context of the beta series correlation analysis method, a set of correlation scatter plots taken from a representative participant are shown in Figure 4. In these scatter plots, each data point represents the estimated activity level (beta value) from a single trial for the FFA (x axis; both charts) and either the right IFG (y axis; left-side chart) or the left hippocampus (y axis, right-side chart). To emphasize the load-dependent changes in correlations, the lowest load (1F; open diamonds, grey regression line) and the highest load (4F; filled circles, black regression line) are plotted. Mirroring the pattern of results observed at the group level, this participant’s FFA is more strongly correlated with the right IFG during the delay period of the 1F condition and more strongly correlated with the left hippocampus during the delay period of the 4F condition.

Fig 4.

Example of load-dependent functional connectivity changes in an individual participant. In these scatter plots, each data point represents the estimated delay period activity (beta value) from a single trial, taken either from the “1 Face” condition (open diamonds) or “4 Faces” condition (filled circles). The beta values obtained from the right FFA (x-axis; both charts) are plotted against those obtained in the right IFG (y-axis; left-side chart) and left hippocampus (y-axis, right-side chart) across all trials for which this participant made a correct response. The degree to which two regions show correlated fluctuations in their delay period activity across trials is taken as a measure of their functional connectivity.

To assess whether the load-dependent connectivity increases with the left hippocampus were systematically related to the load-dependent connectivity decreases with the right IFG, we evaluated load-dependent changes in these correlations across participants. Indeed, these two effects were found to be inversely correlated (r = .68, p < .05), such that the more a participant’s right IFG showed a load-dependent decrease in its correlation with the FFA, the more their left hippocampus showed a load-dependent increase in its correlation with the FFA (Figure 5). This inverse relationship between the load-dependent connectivity of frontal and hippocampal regions with the FFA suggests that increased hippocampal correlations may attempt to compensate for decreased frontal correlations at high loads. The fact that some participants showed this tradeoff more than others suggests that there may be individual differences in the extent to which participants adapt their maintenance circuitry to cope with increasing mnemonic load.

Fig 5.

Load-dependent trade-off between prefrontal and hippocampal connectivity with the FFA. The amount that each participant’s delay period correlation between the right FFA and right IFG changed as a function of load (y-axis) is plotted against the corresponding change in their correlation between the right FFA and left hippocampus (x-axis). The values of each axis correspond to the difference between the arc hyperbolic tangent-transformed correlation coefficients of the “4 faces” condition and “1 face” condition. A significant negative correlation was observed between these two connectivity change measures.

Next, we wished to assess whether the connectivity effects observed between these ROIs were accompanied by load-dependent changes in the mean BOLD signal level (univariate activity) with the ROIs during the delay period, as has been reported in other studies. The right IFG was the only ROI that showed a significant influence of load on its delay period univariate activity. In this region, we observed a significant linear increase in delay period activity with increasing load (F(1,8) = 5.87, p < .05). However, the direct contrast between the 1F and 4F conditions was only marginally significant (t(8) = −2.20, p = .059). Univariate load effects were not found in the other PFC or hippocampal ROIs, nor were they found in the FFA seed.

The fact that the right IFG ROI showed increasing univariate activity as a function of load implies that this region must serve an important role during the delay period of high load trials, despite its decreased coupling with the FFA. Although a detailed examination of this region’s functional interactions with the rest of the brain is beyond the scope of the present investigation, we decided to test the hypothesis that the right IFG increases its connectivity with the hippocampus at high loads. Interactions between ventrolateral PFC and medial temporal lobe regions are thought to play a critical role in the control of long-term memory processes (Simons and Spiers 2003). Given our supposition that long-term memory-like mechanisms in the MTL can contribute to the retention of visual information even over short delays (Ranganath and Blumenfeld 2005), we reasoned that increased frontal-hippocampal coupling might guide or supplement increased hippocampal-FFA coupling at high loads. To test this hypothesis, we used the same hippocampal and right IFG ROIs we had already defined for each participant based on their strong delay period connectivity with the FFA (averaged across loads) and assessed how connectivity between them changed as a function of load. Indeed, the right IFG ROI’s correlation with both the left and right hippocampus ROIs increased with load. For the left hippocampus ROI, this effect was significant for the 1F vs. 4F contrast (t(8) = −2.35, p < .05), but did not achieve significance for the linear contrast (p = .112). For the right hippocampus ROI, this effect was significant for both the 1F vs. 4F contrast (t(8) = −2.77, p < .05) and the linear contrast (F(1,8) = 7.41, p < .05). A schematic summary of these load-dependent changes in frontal, hippocampal, and inferotemporal connectivity is presented in Figure 6.

Fig 6.

Schematic summary of inter-regional connectivity at low and high loads. These diagrams depict the mean correlation level between the right FFA, right IFG, and left hippocampus ROIs during the delay period of the “1 Face” (left) and “4 Faces” (right) conditions. The thickness of the each arrow is scaled to reflect the mean correlation between that pair of nodes, the value of which is displayed for each path. All three paths show significant changes in their correlation between these load levels (all ps <.05).

Given that our six ROIs were defined based on their delay period connectivity with the FFA, our analysis procedure was not optimally structured for assessing load-effects on connectivity during the cue and probe stages of the task. However, it is worth noting that while all six ROIs exhibited statistically significant connectivity with the FFA seed during all task stages and loads, none of these ROIs showed significant load-dependent increases or decreases in their connectivity with the FFA during the cue and probe stages of the task.

Discussion

In the present study, we applied a recently developed functional connectivity analysis procedure (Rissman and others 2004) to an event-related fMRI data set to investigate how the inter-regional interactions subserving visual WM change as function of mnemonic load. Our ROI analyses yielded several novel and important results. First, we observed that while the right IFG interacted strongly with the FFA when participants were required to maintain a single face stimulus, functional connectivity between these two regions linearly decreased as the number of to-be-maintained faces increased. Second, we found that the hippocampus, an MTL region traditionally thought to be involved only in long-term memory processes, exhibited the opposite load-dependent connectivity profile – a linearly increasing degree of functional connectivity with the FFA. Third, these two complementary neural circuits supporting visual WM appeared to dynamically trade off as load increased, such that the amount that a participant’s IFG-FFA connectivity decreased with load was significantly correlated with the amount that their hippocampus-FFA connectivity increased with load. Fourth, this altered pattern of regional interactions was further reflected in the finding that the connectivity between the right IFG and the hippocampus increased significantly from low to high loads. Thus, when faced with increased mnemonic load, the IFG and FFA interacted less with each other, while these regions mutually strengthened their degree of functional coupling with the hippocampus.

Prefrontal interactions with the FFA

We interpret the strong functional connectivity observed between the IFG and FFA during the maintenance of a single face stimulus as reflecting the presence of a modulatory signal originating in the IFG that serves to sustain the activity of a sparse ensemble of neurons in the FFA in an effort to keep the stimulus representation active across the delay period (Gazzaley and others 2004). While this functional relationship must be predominantly top-down in nature since there is no bottom-up visual stimulation during delay period, the active maintenance of the relevant sensory information likely involves reverberant interactions between reciprocally connected IFG and FFA neurons (Fuster 2000). Since our measure of inter-regional coupling is inherently correlational, we cannot make claims about the causal relationship or directionality of these modulatory interactions. Although the specific neural mechanisms of these interactions remain to be elucidated, it is our view that the functional connectivity between IFG and FFA is a neural instantiation of the persistent allocation of attention to the to-be-maintained face representations in the service of WM. In fact, we consider the essential element of WM maintenance to be the sustained attention to relevant pre-existing cortical representations, rather than assuming a dedicated WM storage buffer. This perspective of short-term memory as the PFC-guided attention-based activation of long-term memory representations has recently been articulated by others (Cowan 1993; Postle 2005b; Ruchkin and others 2003).

To the extent that IFG-FFA connectivity during the delay period represents a neural correlate of active maintenance, one might have expected that the strength of this coupling would increase, or at least remain constant, as a function of WM load, since higher loads inherently require the retention of more visual information. However, our data revealed a pattern of decreasing functional connectivity between the right IFG and right FFA as the load increased. This result raises the possibility that IFG-FFA connectivity is not the principle mechanism by which face representations are maintained when the task requires the retention of multiple unique face stimuli. In fact, this finding suggests that this circuit is progressively less utilized for maintenance as the load increases. One plausible explanation for this load-related connectivity decrease is that the maintenance demands of the task exceed the capacity of visual WM at loads greater than a single face. The visual WM system is severely limited in the amount of information that can be simultaneously maintained in an active state (Alvarez and Cavanagh 2004; Cowan 2001; Luck and Vogel 1997; Marois and Ivanoff 2005; Phillips 1974). Indeed, the human capacity for maintaining complex visual objects, such as faces, has been estimated to be only 1 to 2 items, even over very brief delay periods (~1 s) in the absence of any distraction (Eng and others 2005; Jackson and Raymond 2004; Xu and Chun 2006). Such capacity estimates are thought to reflect the number of integrated chunks of information that can be maintained within the focus of attention at a time. Given the longer delay period (8 s) of the present study, along with the potentially distracting effect of the scrambled face stimuli that were intermixed with the memoranda, it is possible that participants were only able to effectively engage in active attention-based maintenance on trials in which a single face was presented. As the load increases, the IFG-FFA circuit is engaged to a lesser extent, potentially because its limited capabilities accomplish the maintenance of a progressively smaller percentage of the total amount of to-be-remembered information, thus decreasing its utility. This account of the IFG-FFA connectivity findings presumes that an alternative neural circuit becomes involved at higher loads to compensate. The putative role of the hippocampus in this process will be discussed in more detail below.

Of the PFC ROIs we queried, only the right IFG (roughly corresponding to Brodmann’s Area 45) showed a significant load effect on its delay period functional connectivity with the FFA. The emergence of this effect in this VLPFC region is logical given the functional and anatomical properties of this region. VLPFC regions are known to have direct long-range projections to and from posterior visual association cortices (Petrides and Pandya 2002; Ungerleider and others 1989; Webster and others 1994) and are thought to play a predominant role in visual object processing, while more dorsal PFC regions are thought to be more critical for spatial processing (Sala and others 2003; Wilson and others 1993). Indeed, face-selective neurons have been found in the primate VLPFC but not the DLPFC, and these neurons receive direct projections from the ITC (O’Scalaidhe and others 1997). In addition, several functional imaging studies have associated activity in the right IFG with the processing and maintenance of faces (Courtney and others 1997; Haxby and others 1995; Sala and others 2003; Ungerleider and others 1998). Right IFG regions are thought to play a greater relatively greater role in visuospatial WM than their left IFG counterparts, which are preferentially involved in the rehearsal of verbal materials (Braver and others 2001; Postle and D’Esposito 2000; Reuter-Lorenz and others 2000). The right hemisphere lateralization of the IFG load effect may also be partly attributable to our use of a right-lateralized FFA seed, given that PFC projections to posterior visual regions are predominantly intrahemispheric (Barcelo and others 2000; Chao and Knight 1998; Eacott and Gaffan 1992). Lastly, our association of IFG-FFA connectivity with maintenance-related processes is consistent with theories of functional specialization in the PFC that have proposed that the VLPFC is preferentially involved in the maintenance of information whereas DLPFC regions are recruited when cognitive tasks require higher-level executive control processes such as manipulation or monitoring (D’Esposito and others 1999; Owen and others 1999).

Load-dependent connectivity changes were not observed in our left IFG ROI or our two MFG ROIs. Given that these ROIs were specifically defined based on their having a high delay period correlation with the FFA (averaged across loads), a null effect of load should not be taken as strong evidence that these regions showed equivalently high correlations with the FFA at all loads. That said, the MFG-FFA correlations at all loads were of similar magnitude to those seen in the right IFG in the “one face” condition. This suggests that communication between the MFG and FFA, while most likely polysynaptic (Petrides and Pandya 1999), may make a functional contribution to the task in a load-independent manner. One speculative explanation is that top-down signals emanating from MFG regions tag face representations in the FFA as task relevant, regardless of the mnemonic load, which serves to facilitate task-set maintenance (cf. Fassbender and others 2006).

Hippocampal interactions with the FFA

In contrast to the load-related decreases in the delay period functional connectivity between the FFA and IFG, the strength of the interaction between the FFA and hippocampus increased linearly as a function of increasing mnemonic load. Indeed, the effect of load on hippocampus-FFA connectivity was inversely correlated with the effect of load on IFG-FFA connectivity. That is, the participants who showed the largest load-related increase in their hippocampus-FFA connectivity also showed the largest load-related decrease in their IFG-FFA connectivity. Thus, increased hippocampal connectivity at high loads may reflect a shift in neural processing away from PFC-mediated maintenance as it becomes increasingly difficult for participants to actively allocate attentional resources to the cortical representations of multiple face stimuli. The maintenance of the face stimuli may have been particularly challenging in the high load trials because each successive face that is presented to the participant at the start of the trial requires mnemonic encoding and essentially distracts the participant from maintaining a mental image of the previous face. Given this challenge, along with the extremely limited capacity of WM for faces (Eng and others 2005), most participants reported that on the high load trials they resorted to recognizing the probe faces based on their subjective familiarity. Functional connectivity between the hippocampus and FFA during the delay period may have served to strengthen the mnemonic traces of the face stimuli in a way that would make them more recognizable when the probe stimulus was presented and a match/nonmatch decision was required. The neural circuitry of the hippocampus, particularly the sparse representations of the dentate gyrus and CA3, afford it the unique ability to assign distinct pattern-separated representations to stimuli, facilitating rapid learning with minimal interference from similar stimuli (O’Reilly and Norman 2002). Increased MTL-FFA connectivity during the delay period of the high load trials may have also served to reactivate the representations of first few faces in the series after their attempted maintenance had been disrupted by the need to encode the subsequently presented faces (Sakai and Passingham 2004; Sakai and others 2002).

Previous studies suggesting that the hippocampus is not critically involved in WM have utilized tasks in which highly overlearned stimuli can be readily rehearsed during the delay period (Cave and Squire 1992; Zarahn and others 2005). Our findings suggest that hippocampal processing resources may only be utilized when the number (or complexity) of to-be-remembered stimuli exceeds the capacity limits of short-term memory. In other words, when a WM task cannot be accomplished using attention-based maintenance or verbal rehearsal, individuals may form and utilize hippocampus-dependent mnemonic representations. These representations are likely similar to those that support long-term memory, implying that performance of difficult short-term memory tasks may often be supported by rapidly formed “long-term” memories. Despite the seemingly compensatory reliance on hippocampal processing when faced with a high mnemonic load, task performance still decreases as a function of load. This suggests that the IFG-FFA connectivity that putatively supports active maintenance at low loads is capable of sustaining higher fidelity perceptual representations than those created via hippocampus-FFA connectivity. However, given that the IFG-FFA circuit may be only effective for maintaining a very limited amount of visual information, the MTL system may be recruited at higher loads to facilitate the retention of less veridical, yet longer-lasting and more distractor-resistant mnemonic representations.

In the present experiment, participants were only tested on their memory for face stimuli after an 8 s delay interval. Thus, we do not have the behavioral data to determine whether the increased hippocampus-FFA connectivity at high loads reflects the formation of new long-term memory traces that would lead to successful face recognition if participants were tested after a longer interval. Data from several recent studies suggest that hippocampal involvement during the delay period of visual WM tasks can lead to improved subsequent long-term memory for the stimuli being maintained. Ranganath et al. (2005b) employed the beta series correlation analysis method to examine functional connectivity in a WM task requiring the maintenance of complex novel objects. Using the left hippocampus as a seed, they determined that its delay period functional connectivity with a network of cortical regions, including ITC, was significantly greater on trials in which the participants subsequently remembered the stimuli on a surprise post-scan recognition memory test versus trials in which they subsequently failed to recognize the stimuli. Other studies have demonstrated that univariate activity levels in the hippocampus during the delay period of visual WM tasks also predict subsequent long-term recognition memory performance (Nichols and others 2006; Ranganath and others 2005a; Schon and others 2004).

Some investigators have noted that hippocampal damage produces a particularly severe impairment in the memory for the relations between multiple items (Ryan and others 2000). One recent experiment testing patients with hippocampal amnesia on task requiring the memory for the relations between faces and scenes or objects within a scene demonstrated significant impairments at delay lags as short as 3 s (Hannula and others 2006). Another recent experiment found that patients with MTL lesions showed severely impaired recognition memory for object-location conjunctions at 8 s delays (Olson and others 2006b). While the present experiment did not probe participants’ memory for the order of, or relations between, the sequentially presented face stimuli, it is possible that some of the hippocampal involvement at high loads may reflect implicit relational processing. Additional studies will be needed to examine the degree to which hippocampal connectivity with posterior sensory cortices scales with the demand for relational processing.

One neural mechanism that has been hypothesized to mediate the long-range communication between the hippocampus and FFA is synchronous oscillatory activity in the theta (4–12 Hz) range (Bastiaansen and Hagoort 2003; Guderian and Duzel 2005; Kirk and Mackay 2003). Recent evidence in monkeys suggests that theta phase locking may be particularly robust during the WM delay period (Lee and others 2005). It is important to note that our measure of functional connectivity only demonstrates that activity is correlated across a given pair of regions; it does not imply that these two regions are monosynaptically connected. Since processing in the hippocampus proper is channeled through a polysynaptic circuit, and hippocampal input and output is relayed through the entorhinal cortex, the functional coupling we observe between the hippocampus and FFA most likely reflects indirect neural communication.

It is unclear why the most robust load-dependent hippocampus-FFA connectivity increase emerged in the left hippocampus ROI. The left hippocampus is thought to be particularly important for verbal encoding, whereas visual encoding tends to engage the hippocampus bilaterally, albeit with a right-hemispheric bias for face stimuli (Powell and others 2005). However, a recent review exploring the nature of memory-related MTL activations found only a slight trend toward these material-specific lateralization effects (Henson 2005). It is interesting to note that fMRI studies utilizing visual WM tasks have documented a strong relationship between the delay period activity/connectivity of the left hippocampus and subsequent long-term recognition memory performance (Ranganath and others 2005a; Ranganath and others 2005b). However, given that our right hippocampus ROI also showed a marginally significant load-dependent increase in its connectivity with the FFA, we are hesitant to place too much weight on the fact that our strongest load effects were found in the left hippocampus. It is possible that with more statistical power we might have observed equivalent load-dependent connectivity increases between our right-lateralized FFA seed and the left and right hippocampi. Moreover, it is worth noting that our right IFG ROI showed significant load effects on its connectivity with both hippocampal ROIs. Further research will be needed to better differentiate the contributions of right and left hemisphere MTL regions to visual memory over short delays.

Interpreting PFC activity and connectivity increases

We have speculated that the load-related decreases in PFC-FFA connectivity reflect a diminished reliance on active attention-based maintenance at higher loads. This finding, which at first glance implies reduced prefrontal involvement with increasing load, may seem surprising in the context of the extant fMRI literature on WM load effects. Most event-related fMRI studies of WM that have included load manipulations have documented load-related increases in PFC activation during the delay period (Cairo and others 2004; Druzgal and D’Esposito 2003; Habeck and others 2005; Jha and McCarthy 2000; Leung and others 2004; Linden and others 2003; Rypma and others 2002; Zarahn and others 2005). Indeed, Druzgal & D’Esposito (2003), working with the same data set used in the present study, reported greater PFC activity during the delay period of high load trials. Our re-analysis of this dataset also documented a load-related increase in univariate delay period activity in our right IFG ROI, even though this ROI was defined based on its connectivity with the FFA during the delay period and not on univariate activity levels. The fact that the right IFG increased its delay period activity with load suggests that this region still makes important contributions to task-related processes at high loads, despite its diminished connectivity with the FFA.

Our finding that this right IFG region shows a load-dependent increase in its connectivity with the right and left hippocampi provides suggestive evidence for some potential roles that this prefrontal region may be playing at high loads. Since active PFC-guided maintenance of the detailed visual representations held in the FFA may be inadequate at supra-capacity loads, the IFG’s shift toward increased functional coupling with the hippocampus may indicate increased reliance on MTL representations. This increased frontal-hippocampal communication could signal the need for a shift away from attention-based maintenance toward a retention process maximally exploits the mnemonic codes of the MTL memory system. According to this view, top-down input from the IFG might orient the hippocampus toward the need to strengthen its processing of the decaying sensory representations in the FFA, thus triggering increased connectivity between the hippocampus and FFA. Alternatively, the IFG may establish reverberant communication with the MTL in a similar manner to its relationship with the FFA at low loads, in both cases with the goal of modulating and sustaining activity in a specific neural population containing task-relevant codes. One could speculate that, when faced with a high load, the IFG allocates its limited modulatory resources to sharpen relevant hippocampal representations. In doing so, the high-fidelity visual maintenance capabilities of the IFG-FFA circuit may be largely abandoned in order to capitalize on the lower-fidelity, yet more durable, mnemonic processing mechanisms of the hippocampus. Given the substantial inter-correlation of all nodes in our simplified network, we are unable to discern whether the IFG-hippocampus connectivity at high loads causes or supplements the hippocampus-FFA connectivity. Further research will be needed to determine how the frontal-hippocampal interactions observed in the present study relate to those thought to play a critical role in the control processes regulating the encoding and retrieval of long-term memories (Simons and Spiers 2003).