Selective Involvement of Superior Frontal Cortex During Working Memory for Shapes

Abstract

A spatial/nonspatial functional dissociation between the dorsal and ventral visual pathways is well established and has formed the basis of domain-specific theories of prefrontal cortex (PFC). Inconsistencies in the literature regarding prefrontal organization, however, have led to questions regarding whether the nature of the dissociations observed in PFC during working memory are equivalent to those observed in the visual pathways for perception. In particular, the dissociation between dorsal and ventral PFC during working memory for locations versus object identities has been clearly present in some studies but not in others, seemingly in part due to the type of objects used. The current study compared functional MRI activation during delayed-recognition tasks for shape or color, two object features considered to be processed by the ventral pathway for perceptual recognition. Activation for the shape-delayed recognition task was greater than that for the color task in the lateral occipital cortex, in agreement with studies of visual perception. Greater memory-delay activity was also observed, however, in the parietal and superior frontal cortices for the shape than for the color task. Activity in superior frontal cortex was associated with better performance on the shape task. Conversely, greater delay activity for color than for shape was observed in the left anterior insula and this activity was associated with better performance on the color task. These results suggest that superior frontal cortex contributes to performance on tasks requiring working memory for object identities, but it represents different information about those objects than does the ventral frontal cortex.

INTRODUCTION

Cells in the ventral occipitotemporal cortex are highly sensitive to multiple visual features that determine the identity of objects, including shape (Brincat and Connor 2004; Gross et al. 1972; Logothetis et al. 1995; Malach et al. 1995) and color (Desimone et al. 1985; Komatsu et al. 1992; McKeefry and Zeki 1997), and the integrity of ventral occipitotemporal cortex is necessary for object recognition (Ungerleider and Mishkin 1982). Cells in dorsal occipital and parietal areas, on the other hand, are sensitive to the locations of objects in multiple reference frames and are necessary for directing reaching and grasping movements toward visually presented objects (Creem and Proffitt 2001; Goodale and Milner 1992; Ungerleider and Mishkin 1982).

This characterization of the organization of visual extrastriate cortex has been extended to the realm of visual working memory and the organization of the prefrontal cortex (PFC) (Courtney 2004; Levy and Goldman-Rakic 2000). The dorsal visual pathway has a predominance of anatomical connections with dorsal frontal cortex and the ventral visual pathway has a predominance of anatomical connections with ventral frontal cortex (e.g., Barbas and Pandya 1989). However, seemingly contradictory findings have been reported from both single-cell recording and neuroimaging experiments regarding spatial versus object working-memory delay-period neural activity. Although some studies report a similar dissociation of dorsal and ventral PFC regions (e.g., Courtney et al. 1996, 1998; Mohr et al. 2006; Rämä et al. 2004; Sala and Courtney 2007; Sala et al. 2003; Volle et al. 2008; Wilson et al. 1993) others do not (e.g., Nystrom et al. 2000; Owen et al. 1996; Postle et al. 2000; Rao et al. 1997; Smith et al. 1995). These studies have raised questions about whether an analogous spatial versus nonspatial characterization of the dorsal versus ventral prefrontal cortex is appropriate (see reviews in Courtney 2007; Courtney et al. 2004). However, such inconsistencies can be explained if the processing or working-memory maintenance of some object features also relies on the same processing architecture as that which computes or maintains spatial locations and directs movements. Object shape may be one such feature.

Information about object shape is processed by both the ventral and dorsal visual streams. Explicit perceptual recognition of shapes is thought to primarily be the role of ventral visual areas such as lateral occipital cortex (review in Connor et al. 2007). However, shape-selective responses have been observed in individual cells in parietal cortex (Sereno and Maunsell 1998; Sereno et al. 2002) and functional magnetic resonance imaging (fMRI) activation in the parietal cortex has been observed for viewing intact versus scrambled shapes and objects (Kraut et al. 1997). In addition, a recent fMRI study of shape perception with both monkeys and humans indicates that there are regions in parietal cortex in both species that are shape sensitive (Denys et al. 2004; Sawamura et al. 2005). These two studies also suggest that shape selectivity in parietal cortex may be even stronger and more extensive in humans than in monkeys. In at least some parts of the parietal cortex, however, the representations of shape appear to be qualitatively different from those in ventral stream perceptual areas (Lehky and Sereno 2007; Srivastava et al. 2009). These dorsal stream representations seem well-suited for the known roles of the parietal cortex in directing grasping actions (e.g., Goodale and Milner 1992) and control of spatial attention (Kourtzi and Kanwisher 2000; Sereno and Amador 2006). Studying visual working memory for shape using a delayed-recognition task enables evaluation of both the functional organization of prefrontal cortex and its relationship with the dorsal and ventral visual streams, independent of motor behavior.

In the current study, fMRI activations during working memory for one of two different object features—shape and color—were directly contrasted. If the ventral and dorsal visual streams follow either the “what versus where” or “perception versus action” proposed functional organizations and if the organization of PFC mirrors this organization, then working memory for object location should depend on dorsal PFC as shown in previous studies, but working memory for object features, including both object shape and color in the current study, should depend only on the ventral PFC. Previous neuroimaging studies showing activation in dorsal PFC during object working memory tasks suggest that this hypothesis may not be correct. We hypothesized that shape processing would activate the dorsal PFC more than color processing would and that these activation modulations would be associated with performance on the tasks. The results agree with this hypothesis and suggest that superior frontal cortex contributes to performance on tasks requiring working memory for object identities, although it represents different information about those objects than does ventral frontal cortex.

METHODS

Subjects

Participants (n = 18) were recruited from the Johns Hopkins University community and were paid $50.00 for their participation. They were nonsmokers in good physical and mental health. They were free of any reported history of head injury, drug or alcohol abuse, neurological or mental disorders, and cardiovascular diseases. In addition, all participants were screened for color blindness. The experimental protocol was approved by the Institutional Review Boards of both the Johns Hopkins University and the Johns Hopkins Medical Institutions. All participants gave written informed consent and were debriefed afterward.

Stimuli

Stimuli consisted of abstract figures whose visual angle was 2.2 × 2.2° in size (Fig. 1). Each figure contained four vertices. The four sides connecting these vertices were unique combinations of an S-shaped curve, a concave curve, a convex curve, and a straight line. In all, 24 different shapes were generated. The shapes were filled in with different shades of blue. In total, 24 different shades were used to generate 24 colored versions for each shape, resulting in a total of 24 × 24 = 576 unique combinations of abstract shapes and colors. The colors were of different hues that varied in equal discrete increments from a greenish blue to a purplish blue. Brightness of all the different shades was kept constant. Two different levels of saturation (60 and 70%) were used, but the saturation level was kept constant within a trial. All the shapes were embedded in a black square canvas 3.3 × 3.3° visual angle in size, which was then presented on a white background.

FIG. 1.

Example trial sequence for the color-delayed recognition, shape-delayed recognition, and sensorimotor control tasks. The working-memory trials shown are both “match” trials. One representative shade from each color group of stimuli is shown.

Control stimuli were the same abstract colored shapes on their black square canvas, Fourier transformed, phase scrambled, and then inverse Fourier transformed. The resulting figures retained the same luminance and overall contrast and frequency information, but were unrecognizable as shapes (Fig. 1). These phase-scrambled stimuli were also presented on a white background.

A Power Macintosh G3 desktop computer using Superlab software was used to present stimuli and collect behavioral data. A liquid crystal display projector located outside the scanning room was used to project stimuli from the computer onto a screen located in the bore of the scanner, behind the subject's head. Subjects then viewed the stimuli via a mirror mounted to the top of the head coil and made responses with left or right thumb presses on button boxes held in each hand. Button boxes were connected via a fiber-optic cable to a Cedrus RB-6×0 Response Box to record the responses.

Tasks

Two delayed-recognition tasks and a sensorimotor control task were used in the experiment. Each trial in all three conditions consisted of the following: 1) an instruction cue of 1,500 ms; 2) an instruction delay, consisting of a fixation cross in the center of a white background, lasting for 1,500 ms; 3) presentation of two sample stimuli at fixation sequentially, each lasting 1,000 ms; 4) a delay period of varying length, lasting for 6, 7.5, or 9 s, during which a fixation cross in the center of a white background was shown; 5) a test stimulus presented at fixation for 3,000 ms; and 6) an intertrial interval of varying lengths, consisting of a fixation cross in the center of a white background, lasting for 1.5, 3, or 4.5 s (Fig. 1). Two instruction cues informed subjects whether they were to remember the shape or color, respectively, of the sample stimuli during the delay period after those stimuli were removed from view. They were told explicitly that shape was irrelevant during a color trial and vice versa. The two sample shapes or colors within a trial were always different and, in the sensorimotor control task, none of the three (two sample and one test) images was repeated.

Pilot testing revealed that if the color hues on a given trial were neighbors to each other they were very difficult to discern, resulting in worse overall performance on the color task than that on the shape task. Thus the 24 shades were divided into three groups of eight hues each, with the first group ranging from 195 to 210°, the second group ranging from 215 to 230°, and the third group ranging from 235 to 250° on the Hue-Saturation-Lightness scale. Within each trial, the two sample shades were chosen from any two of the three groups. The mismatch test shade was then chosen from the remaining group, so that there was a minimum “distance” in hue between the two sample stimuli and between the sample stimuli and a distractor test stimulus. This change reduced the difficulty in the color task and made the performance level similar to that of the shape task.

Subjects were instructed to look directly at the stimuli while they were present on the screen and to maintain fixation on the fixation cross otherwise (during the instruction delay, memory delay, and the intertrial interval). Subjects were instructed to make a response with a left or right thumb press on the button boxes held in both hands to indicate whether the test image was a match or a mismatch while it was present on the screen. A match in a shape trial was a test image having the same shape as either of the two samples, regardless of its color. A match in a color trial was a test image having the same color shade as that of either of the two samples, regardless of its shape. In control trials, subjects were instructed to look at the samples when they were presented but were not required to remember anything. They were instructed that during the test period of the control task they were to press both buttons at the same time when the test image was shown. This low-level sensorimotor task controlled for the amount of visual stimulation, anticipation of a motor response, and the motor output. It served as a baseline to evaluate overall activation patterns across both tasks. The more interesting contrast, however, was the evaluation of differences between the two well-matched working-memory tasks.

All three tasks (shape, color, and control) were used in every run. The two experimental conditions were blocked into groups of two, three, or four trials. Each experimental block was preceded and followed by one or two control trials. Each run consisted of 24 trials, with eight trials from each condition. There were six runs in the experiment; thus 48 trials were included for each condition. The order of the trials was counterbalanced within each run and across the entire experiment. Run order was counterbalanced across subjects. Before scanning, subjects were given two practice runs with feedback to familiarize them with the tasks.

Imaging Protocol

T2*-weighted, interleaved gradient echo, echo planar imaging (EPI) was used throughout performance of the tasks (27 axial slices; 3.00-mm thickness; 1.00-mm gap; repetition time [TR] = 1,500 ms; time to echo [TE] = 30 ms; flip angle = 65°; matrix: 80 × 80; field of view [FOV], 240 mm). An magnetization-prepared rapid gradient echo structural image (200 coronal slices; 1.0-mm thickness; flip angle = 8.00°; matrix: 256 × 256; FOV, 256 mm) was obtained from each subject in the middle of the scanning session (between the third and fourth functional runs) to allow for localization of anatomical structures in later data processing. All scans were performed at the F. M. Kirby Research Center for Functional Brain Imaging at the Kennedy Krieger Institute on a 3-Tesla Philips Gyroscan ACS-NT MR scanner (Philips Medical Systems).

Analysis of Time Series Data

Imaging data were preprocessed and statistically analyzed using AFNI shareware (Cox 1996). All functional images were phase-shifted to correct for slice acquisition time and aligned to the middle image of the session to correct for motion during the session. Anatomical images were coregistered to functional images and all images were spatially normalized into Talairach space (Talairach and Tournoux 1988). Global activity for each scan was corrected for by grand mean scaling. Separate square wave functions matching the time course of the four event types within a trial (instructional cue, sample presentation, delay period, and test period) for each trial type (color, shape, control) were convolved with a Gaussian model of the hemodynamic response function, creating twelve regressors of interest. Regressors of no interest included six regressors that modeled the motion parameters, and one regressor representing linear drift during each run of the session. These regressors were entered into a General Linear Model (Friston et al. 1995), where each regressor was scaled to fit individual subjects' response functions for each voxel. These individual subjects' maps were registered into Talairach space and spatially filtered with a 3 mm Gaussian kernel (FWHM), and the scalar beta weights for shape delay and color delay were then compared in a mixed-effects analysis with subjects as a random factor, on a voxelwise basis, creating statistical parametric t-maps. Significance of results was determined by an individual voxel t statistic threshold (P < 0.01) and a spatial extent threshold for contiguously activated voxels ( P < 0.05, corrected for multiple comparisons, Ward 2000; http://afni.nimh.nih.gov/pub/dist/doc/manual/AlphaSim.pdf). To correct for multiple comparisons a Monte Carlo simulation was run to determine the number of contiguous voxels that were required for these clusters to reach an experiment-wise significance level of P < 0.05. It was found that 14 contiguous activated voxels (504 μl) were required to reach this experiment-wise significance level.

Time Course Analysis

Time courses of the activated regions were estimated across trials using voxelwise deconvolution. The time courses for each of the three conditions (shape, color, and control) were then averaged across voxels within an activated cluster and across participants. In all, 25 time points (37.5 s) were included, beginning at the start of the sample presentation.

To determine the extent to which fMRI activity varied with performance, subjects were separated into groups based on their performance on the two types of working-memory tasks. Specifically, subjects were each given a composite z-score, based on the accuracy and response times relative to the group, for their performance on both the shape trials and the color trials. Participants were thus divided into two equal groups twice, once based on their overall performance on the color task and once based on their overall performance on the shape task. Activity from the onset of the sample through the next 10 time points (15 s) of shape trials within the shape > color regions was compared for good versus poor shape performers. Activity in color > shape regions was similarly compared for good versus poor color performers. To increase statistical power, the analyses were limited to the first 10 time points, which covered most of the trial-related activity changes. Later modeled time points, which reflected return to baseline activity, were not included. The relationship between trial-to-trial activation variation and task performance within subjects was not analyzed due to lack of statistical power from the low number of trials with incorrect responses.

RESULTS

Behavioral Data

Participants performed well in the experiment, with a mean accuracy of 82.3% across all trials. Accuracy on color trials (81.3 ± 1.9%) was similar to that on shape trials (83.4 ± 2.2%) [F(1,17) = 1.12, P = 0.304]. Reaction times (RTs) were also not significantly different between color trials (mean RT = 1,298 ± 59 ms) and shape trials (mean RT = 1,369 ± 53 ms) [F(1,17) = 3.33, P = 0.09]. Inaccurate responses were excluded from RT analyses. RT and accuracy for control trials were not subjected to statistical analyses. Average accuracies for the good and poor performing groups for each task are shown in Table 1.

Table 1.

Average accuracies for the good and poor performing groups for each task

Performance	Good Color Group	Poor Color Group	Good Shape Group	Poor Shape Group
Color task, mean % correct (SE)	87 (1)	76 (2)	85 (2)	78 (3)
Shape task, mean % correct (SE)	85 (3)	82 (3)	91 (1)	76 (7)

fMRI Data

Regions of Activation

Large regions of parietal cortex and superior, middle, and inferior frontal cortex were more active for both the color and shape working-memory tasks than for the sensorimotor control task. In addition to regions that may process both color and shape information, this contrast reflects the use of colored shapes as opposed to scrambled stimuli and the greater attention and other executive demands of both working-memory tasks. The more meaningful comparison for the purpose of the current study was the direct comparison between the color and shape tasks.

During the delay periods of the working-memory task there was a double dissociation regarding regions differentially activated for color versus shape trials in both extrastriate and prefrontal cortices. Color trials showed greater activation than did shape trials in the cuneus, anterior insula (INS), and inferior frontal gyrus (IFG). Shape trials, on the other hand, showed greater activation than did color trials in the parietal cortex, middle occipital gyrus (MOG), the middle frontal gyrus (MFG), and the superior frontal sulcus (SFS). All regions showing differential color versus shape activation during memory delays can be seen in Fig. 2.

FIG. 2.

Brain regions showing greater activity during shape than during color working-memory delays (green) and greater activity during color than during shape delays (red).

Relationship between Activation and Performance

Within the regions demonstrating greater activity for shape than for color trials, activity from the onset of the sample through the next 10 time points (15 s) of shape trials was compared for good shape performers and poor shape performers. Two of the regions that showed greater activity for shape than for color also showed reliable differences in activity across shape task performance groups. They included the more superior region along the SFS [t(18) = 5.7, P < 0.01] and the region near the right MOG [t(18) = 5.36, P < 0.01]. Figure 3 displays the time course of activity within these regions for both groups of performers, showing much greater activity in the region along the SFS throughout the delay period for participants who performed better on the shape task. Greater activity was also seen within the right MOG for those subjects who performed better on the shape task, although the time course of activity for both groups shows little sustained activity during the memory delay within this region, suggesting that it might be more critical for perceptual processing during the sample and test periods of the trials.

FIG. 3.

Differential activities in shape-sensitive (green) and color-sensitive (red) regions for good (solid line) and poor (dotted line) performers. A: left superior frontal sulcus. B: right middle occipital gyrus. C: left insula.

Similar analyses, comparing the estimated activation during color trials of good color performers and poor color performers found that one of the color-sensitive regions, the left insula, also showed performance-based modulation [t(18) = 4.3, P < 0.001]. As can be seen in Fig. 3, subjects who performed better during color trials showed greater activity within the left insula from the onset of the sample throughout the duration of the delay period. The activation in this region was not dependent on good versus poor performance on the shape task. Similarly, neither the SFS nor the MOG activation was dependent on good versus poor performance on the color task.

DISCUSSION

In summary, the current results demonstrate a double dissociation between cortical regions preferentially activated during working memory for shape, such as in the middle occipital gyrus, parietal cortex, and dorsolateral prefrontal cortex, and regions preferentially activated during working memory for color, such as in the cuneus, insula, and ventrolateral prefrontal cortex. Some regions differentially active for the two tasks appear to be important for performance on the task. Better working-memory performance for shapes is associated with increased activation in middle occipital and superior frontal regions, whereas better working memory performance for color is associated with increased activation in the left insular cortex. The different degrees to which the SFS and the insula were important for shape versus color working-memory performance, respectively, constitute a functional double dissociation, in addition to the activation magnitude dissociation. The results suggest a role for this region of superior frontal cortex in the explicit representation of shape information during visual working-memory delays.

Our results are consistent with previous observations demonstrating that the ventral occipitotemporal cortex plays a primary role in visual recognition of shapes. Cells in V4 and inferotemporal cortex are highly selective for object shape (Desimone et al. 1985; Gross et al. 1972). In addition, object recognition, which depends on shape representation, becomes disrupted when the ventral visual areas are lesioned (Ungerleider and Mishkin 1982). Neuroimaging studies have also observed modulation of activation in the ventral visual stream when shape information was to be attended (Cant and Goodale 2006; Corbetta et al. 1991). As in these previous neuroimaging studies, we found preferential shape activations in the MOG, a region in the ventral visual processing stream. Although MOG's activity was also found to correlate with good performance on the shape task, inspection of the time courses of activation in the two groups suggests that sample and test period activity played a greater role in this correlation than did delay activity. Thus MOG may be primarily important for perceptual encoding aspects of the task rather than maintaining shape information over the memory delay. The performance-dependent activation in the SFS, on the other hand, appeared to be sustained throughout the working-memory delay period.

The greater activation of parietal cortex for the shape than that for the color task in the current study is also consistent with a large number of previous studies indicating a role for parietal cortex in shape representation. Shape-selective responses in parietal cortex have been observed in both monkeys and humans (Denys et al. 2004; Kraut et al. 1997; Sawamura et al. 2005; Sereno and Maunsell 1998; Sereno et al. 2002). A study by Oliver and Thompson-Schill (2003) examined the brain areas involved in memory retrieval of the size, shape, or color of everyday objects. Greater parietal activation was observed during size and shape retrieval, compared with color retrieval. A study by Sala et al. (2003) suggested that greater activation in parietal and superior frontal cortices during working memory for some types of objects relative to others may be due to the need to represent spatial information about those objects, such as the relative spatial relationships among parts of an object. Individuals with Balint syndrome who have sustained bilateral parietal damage cannot perceive more than one object in their visual field at the same time, nor can they perceive the spatial configuration of elements within objects (Robertson et al. 1997). Particularly during working-memory tasks, the shapes might be represented in part according to their spatial configurations.

In the current study, some subjects might have strategically maintained representations of where the component lines were located instead of the holistic shapes per se. Such a strategy, however, would have resulted in the need to maintain the spatial relationships of eight lines (four for each of the two samples), a much higher memory load than two holistically represented shapes and, presumably, less effective. If the parietal activation were the result of a parts-based strategy, one would expect activation to be negatively correlated with performance on the shape task. Shape-selective activation in the parietal cortex was not significantly different between the good and poor performance groups, whereas the SFS activation was. It is unlikely that we lacked statistical power to detect a difference between groups in parietal cortex where the overall activation was greater than that in SFS. Instead, the activation in parietal cortex may have been modulated by attention to the relevant dimension of shape (or spatial configural information) during the delay period of the shape task. This selective-attention effect may have been present in both good and poor performers as they attempted to perform the tasks, but apparently this activation was not sufficient for optimal working-memory performance. Thus although both parietal and SFS regions were more active for the shape than for the color task, differences in their susceptibility to performance-based modulation suggests these regions play different roles in this neural system involved in the shape task. The results are consistent with recent findings that individuals with Williams Syndrome—which is thought to involve dysfunction of the dorsal stream–have perceptual deficits that are specific to spatial processing, but working-memory deficits for both spatial locations and object identities (O'Hearn et al. 2009).

Color-selective activation in the insula was elevated in good color task performers across all task components, but its activation peak was later than was the shape-selective activation peak in MOG or SFS, suggesting that its role may lie primarily at the recognition test stage of the trial (Owen et al. 1996). Together with ventrolateral PFC, the insula has been previously shown to have greater activity for nonspatial working memory than for spatial working memory (e.g., Munk et al. 2002; Rämä et al. 2004; Sala et al. 2003). Activation in this region has also been associated with selective attention (Corbetta et al. 1991), semantic knowledge about the colors of objects (Banich et al. 2001), and self-reflection (Modinos et al. 2009). This combination of previous reports might lead one to speculate that in the current study the greater activation during the color task for good color performers reflected successful use of a task strategy involving working-memory maintenance of autobiographical associations of the sample color with specific remembered objects, such as sky blue or cornflower blue.

In general, the relationship found here regarding the differences in overall performance between subject groups and the overall magnitude of activation in those subjects could reflect individual differences in strategy or ability, rather than the necessity of a particular brain region for task performance within all subjects. The small number of incorrect trials resulted in insufficient statistical power to do a trial-by-trial performance analysis within subjects. However, because the regions that are correlated with performance are different for the shape task than those for the color task, it is unlikely that the results can be explained by a general motivational or arousal effect. All subjects performed both tasks and the division of the group into “good” and “poor” performers was independent for each task.

The greater involvement of the SFS in working memory for shapes than working memory for color helps to explain the inconsistent results of previous studies that compared activations during working memory for object identities versus those during working memory for object locations. Multiple object properties can be used for recognition and working-memory maintenance. The particular type of object used and the dimensions along which distractor test items might differ from remembered sample items in an experiment could affect the degree to which an object working-memory task may depend on the SFS. Thus the functional distinction between the SFS and more ventral posterior PFC regions may be in the representation of spatial versus nonspatial information more generally, rather than working-memory tasks for the locations versus the identities of objects. Such spatial information could include information important for identifying objects, such as the configurations of object parts or some aspects of object shapes. The current results indicate that the SFS is involved in both spatial and object tasks, but that it maintains a different type of information about remembered objects than does the ventral frontal cortex.

These results fit into a growing body of evidence that suggests a yet-to-be-identified common underlying computation or representation of information that is necessary for many tasks, not just those that directly involve spatial location information. Parietal and superior frontal cortical regions have been shown to be involved in attention, updating, and other “control” processes in tasks that seem to be nonspatial (e.g., Petrides 2005; Roth et al. 2006; Serences et al. 2004). These regions have also been implicated as being commonly involved in the representation of space, time, and numerosity (see review by Bueti and Walsh 2009). Both these studies and the current study suggest that we must fine-tune our hypotheses about how best to characterize the types of information represented in both dorsal and ventral PFC, particularly concerning the role of dorsal frontal cortex in maintaining information about objects.

GRANTS

This work was funded in part by National Institute of Mental Health Grant R01 MH-061625 to S. M. Courtney.