Frontoparietal networks involved in categorization and item working memory

Abstract

Categorization and memory for specific items are fundamental processes that allow us to apply knowledge to novel stimuli. This study directly compares categorization and memory using delay match to category (DMC) and delay match to sample (DMS) tasks. In DMC participants view and categorize a stimulus, maintain the category across a delay, and at the probe phase view another stimulus and indicate whether it is in the same category or not. In DMS, a standard item working memory task, participants encode and maintain a specific individual item, and at probe decide if the stimulus is an exact match or not. Constrained Principal Components Analysis was used to identify and compare activity within neural networks associated with these tasks, and we relate these networks to those that have been identified with resting state-fMRI. We found that two frontoparietal networks of particular interest. The first network included regions associated with the dorsal attention network and frontoparietal salience network; this network showed patterns of activity consistent with a role in rapid orienting to and processing of complex stimuli. The second uniquely involved regions of the frontoparietal central-executive network; this network responded more slowly following each stimulus and showed a pattern of activity consistent with a general role in role in decision-making across tasks. Additional components were identified that were associated with visual, somatomotor and default mode networks.

1 Introduction

Categorization and specific-item memory are fundamental processes which allow us to apply knowledge to novel situations. Categorization requires abstraction from inherent stimulus features to generalizable latent features, and plays an important role in the flexible transfer of knowledge and skills across stimuli and tasks (for review, see Seger & Miller, 2010). In contrast, memory for specific items maintains these inherent stimulus features in order to enable us to make fine distinctions between items. Despite these fundamental differences, both categorization and specific item memory tasks recruit common cognitive control systems to support task performance (Cole et al., 2013; Gazzaley & Nobre, 2012; Seger & Peterson, 2013), raising the question of how the same neural systems can serve different ends. This study directly compares categorization and specific item memory using delayed-match-to-category (DMC) and delayed-match-to-sample (DMS) tasks in which participants encode a stimulus, maintain information across a delay, see a second stimulus and then decide if it matches the first. The tasks share similar structure, and therefore place similar demands on perceptual (stimulus encoding), motor (response execution) and some executive functions (working memory and decision-making). The tasks differ in what is encoded at the first stimulus and in the basis of the match-mismatch decision occurring at probe: specific item identity in the DMS task and category in the DMC task. Our design, therefore, allows us to isolate differences between processes associated with categorization and those associated with item-specific memory, and also to identify shared processes. Below we first discuss proposed shared cognitive control functions across categorization and specific-item tasks and how they may rely on intrinsically connected frontoparietal neural networks. We then discuss aspects of cognitive processing specific to categorization and to item working memory. Finally we describe our task and our predictions.

1.1 Shared cognitive control processes and intrinsic neural systems

Much recent research has focused on how frontoparietal networks can be flexibly recruited to support cognitive control in diverse task environments (Cole et al., 2013; Dumontheil, Thompson, & Duncan, 2011; Duncan, 2010). Multiple networks supporting cognitive control have been identified, and although there is currently little consensus concerning network nomenclature, we will focus on two networks that show coactivation across a variety of cognitive tasks and correlated patterns of intrinsic activity during resting-state fMRI (Dosenbach et al., 2007; Seeley et al., 2007). The first, the salience network (abbreviated here as SA), which has important nodes in the anterior insula / frontoinsular cortex and dorsal anterior cingulate (ACC) / medial frontal gyrus, is thought to play an important role in bottom-up detection of salient external events, the coordination of functional networks to meet task demands, and in moderating autonomic arousal (Medford & Critchley, 2010; Menon & Uddin, 2010; Menon, 2011; Sridharan, Levitin, & Menon, 2008). The second, the central executive network (FP-CEN), which has important nodes in the dorsolateral prefrontal cortex and posterior parietal cortex / intraparietal sulcus, is thought to operate on the salient stimuli marked by the SA network (Seeley et al., 2007), and to play an important role in the manipulation and maintenance of these representations in working memory and rule-based processes (Miller & Cohen, 2001).

Intrinsic connectivity has also identified other networks, such as the somatomotor network (SM; primary motor and somatosensory cortex), dorsal attentional network (DA; premotor and superior parietal cortex), default mode network (DMN; medial frontal and posterior cingulate regions, inferior parietal and medial temporal lobe), and visual network (VS; occipital and inferior temporal cortex) (Buckner, Krienen, Castellanos, Diaz, & Yeo, 2011; Choi, Yeo, & Buckner, 2012; Yeo et al., 2011). A focus of recent research has been to identify how these networks interact, and one particularly important finding is that the FP-CEN and SA, which show greater activity during cognitively-demanding tasks (Chen et al., 2013; Dang, Donde, & Madison, 2012; Vanhaudenhuyse & Demertzi, 2011) are anticorrelated with the DMN. The SA is thought to play an important role in mediating this anticorrelated relationship, and in switching from the DMN to the FP-CEN in response to salient external events (Bonnelle et al., 2012; Goulden et al., 2014; Menon, 2011; Palaniyappan, Simmonite, White, Liddle, & Liddle, 2013; Sridharan et al., 2008).

How these primarily cortical intrinsic connectivity networks interact with subcortical and cerebellar regions is an active area of research. Buckner and colleagues, for instance, examined functional connectivity between the cortical intrinsic connectivity networks and the basal ganglia and the cerebellum (Buckner, Krienen, Castellanos, Diaz, & Yeo, 2011; Choi, Yeo, & Buckner, 2012). Both basal ganglia and cerebellum had separate regions that correlated with each cortical network, consistent with known projections from cortex to these structures. Particularly relevant for our study are the interconnections with the FP-CEN, which are primarily correlated with the dorsal head and body of the caudate nucleus (Choi et al., 2012), and the lateral cerebellar hemisphere (Buckner et al., 2011).

1.2 Categorization

Cognitive neuroscience studies have associated categorization with a large distributed neural network including the basal ganglia (Seger, 2008), lateral frontal (Muhammad, Wallis, & Miller, 2006), lateral parietal cortex (Daniel et al., 2011; Freedman & Assad, 2009; Rishel, Huang, & Freedman, 2013), precuneus (Wenzlaff, Bauer, Maess, & Heekeren, 2011), premotor and supplementary motor areas (Ashby, Ennis, & Spiering, 2007; Little, Shin, Sisco, & Thulborn, 2006; Waldschmidt & Ashby, 2011). Although still an active area of research, clues are emerging as to the individual contributions made by each region. The basal ganglia has been associated with multiple processes: posterior regions are involved in mapping visual stimuli to category, and category to motor response, whereas anterior regions and the ventral striatum are associated with feedback and reward processing (Seger, 2008). Frontal regions have been associated with maintenance and implementation of categorization rules (Antzoulatos & Miller, 2011; Buschman, Denovellis, & Diogo, 2012; Freedman, Riesenhuber, Poggio, & Miller, 2001; Meyers, Freedman, Kreiman, Miller, & Poggio, 2008; Muhammad et al., 2006; Wallis & Miller, 2003). The parietal cortex combines category with motor response, and may be responsible for integration of relevant information for category membership (Freedman & Assad, 2009; Shadlen & Newsome, 2001; Swaminathan & Freedman, 2012). The precuneus and SMA, along with regions of the basal ganglia they interact with, may be associated with setting a response criterion (Forstmann et al., 2008; Wenzlaff et al., 2011). In addition, inferotemporal cortex performs relevant visual processing necessary for categorization, though it is still unclear the degree to which this region changes with learning and contributes to the representation of novel categories.

Previous categorization studies in humans have typically required participants to view, categorize and respond to single stimuli in rapid succession. However, like the DMC task, real life situations often have a delay between the categorization of a stimulus and a subsequent behavioral response, or require that multiple categorical representations be integrated in order to determine the correct response. The DMC task is advantageous in that it allows us to examine category maintenance and integration across stimuli, and also allows the dissociation of categorization processes from those related to motor preparation. The DMC task was originally developed for research with non-human primates, which find category sensitivity independent of motor response for neurons in inferotemporal, parietal, frontal, and basal ganglia regions (Freedman & Miller, 2008). Electrophysiological recordings suggest coordination between the frontoparietal network and regions within the inferior temporal lobe during this task, such that inferior temporal regions tend to be more sensitive to the visual features of individual exemplars, while prefrontal regions are more sensitive to features relevant for successful task performance (e.g., categorical-status of the first stimulus, and match-mismatch status of the second; Freedman et al., 2003; Meyers et al., 2008a).

1.3 Working Memory

The DMS task has been used extensively to investigate specific-item working memory in human and non-human primates. A large body of research finds that frontoparietal regions are recruited during the performance of DMS tasks (Sala, Rämä, & Courtney, 2003). However, there are some differences based on task demands; for example, working memory for objects rather than spatial location is particularly reliant on ventrolateral PFC regions in the middle and inferior frontal gyri (Sala et al., 2003). There is also evidence that working memory for objects involves interactions between these frontoparietal networks and higher order visual cortical regions involved in representing the objects (Gazzaley, Rissman, & D’Esposito, 2004), and evidence that memory for specific items can lead to interactions with the hippocampus (Rissman, Gazzaley, & D’Esposito, 2008). Neural networks recruited during DMS performance vary depending on task demands. Increased working memory demands have been associated with increased connectivity between regions in the inferotemporal cortex and the hippocampus, and decreased connectivity between inferotemporal regions and the inferior frontal gyrus (Rissman et al., 2008). Similarly, during the delay epoch, inferotemporal regions associated with task-relevant processes show increased connectivity with the frontoparietal network while regions associated with task-irrelevant processes show increased connectivity with the default mode network (Chadick and Gazzaley, 2011).

1.4 Present Study

In the present study, we directly compared patterns of activity during DMC and DMS tasks utilizing the same perceptually-similar stimuli (young Caucasian female faces) and the same timing and responses such that the tasks differed only in the requirement to either categorize the face or remember the specific face. As in several previous DMS studies, we chose to use facial stimuli, as the processing of these stimuli is known to occur with localized regions of the fusiform cortex (cf., Gazzaley et al., 2004; Rissman et al., 2008). Two versions of the DMC task were used, which we termed “Category” and “Label.” In both, participants viewed a face at encoding, categorized it, and maintained the category membership across a delay. At probe, the conditions differed: in the Category version participants viewed a second face, whereas in the Label version they viewed the category label (“A” or “B”). In both of these conditions, participants decided whether the categories matched. The Label condition allowed us to discriminate between activity due to comparing category labels and match-mismatch decision making, versus activity due to viewing and categorizing the face. Thus, we predicted that regions involved in stimulus categorization should have high activity at encoding for both Label and Category, but only for Category at probe. However, regions associated with category match-mismatch decision making should show activity more broadly in both tasks.

We predicted some common and some differing patterns of activity based on the shared and individual characteristics of categorization and item working memory. First, because of the paired task design that equated basic visual and motor demands, we predicted similar recruitment of the visual and motor systems. Second, as both tasks require weighing evidence towards the binary response options, we predicted that the tasks would commonly elicit activity in regions associated with general decision-making processes (Freedman & Assad, 2011; Seger & Peterson, 2013). We predicted that the primary differences between tasks would be found in regions associated with the FP-CEN. Categorization involves several different strategies that require cognitive control, including evaluating information with respect to categorization criteria and mapping the stimulus to category membership (Seger & Peterson, 2013). In contrast, working memory requires different control processes for encoding and maintenance, potentially via interactions with inferotemporal cortex and the hippocampus (Rissman et al., 2008).

2 Methods

2.1 Participants

Seventeen participants were recruited from the Colorado State University Community. All participants were healthy, right-handed adults (11 females, 6 males) with an average age of 27 (range: 20–37). Participants were screened for history of psychiatric or neurological disorders, for current use of psychoactive medications, and exclusionary criteria for fMRI (e.g., claustrophobia, metallic implants).

2.2 Stimuli

Twenty-five similar young adult female Caucasian faces were selected for the stimulus set. To discourage use of verbalizable memory strategies, all images were cropped so that the whole face, but no other defining characteristics, was shown. The faces were then warped and resized to subtend a visual angle of roughly 3.9 degrees horizontally and 6.9 degrees vertically. For each participant, eight stimuli were randomly assigned to category “A” and eight were randomly assigned to category “B.” This type of categorization task is sometimes referred to as arbitrary, or unstructured because the stimuli are randomly assigned to category and do not include any intentional within-category similarities. Unstructured tasks rely on procedural knowledge to a similar degree as structured implicit categorization tasks (Crossley, Madsen, & Ashby, 2012), and recruit similar cortical and striatal systems (Seger, Dennison, Lopez-Paniagua, Peterson, & Roark, 2011; Seger, Peterson, Cincotta, Lopez-Paniagua, & Anderson, 2010). The remaining stimuli were used in the Item condition.

2.3 Procedure

Prior to scanning, participants performed two tasks on a laptop computer; they first learned to categorize faces and then trained on a task that was similar to what they would later perform in the scanner. In the category-learning task, participants learned to categorize each of the 16 faces into category “A” or category “B” via trial and error. On each trial, a face was presented in the center of the computer screen, and the category labels were presented at the bottom left and right of the screen. To encourage participants to learn category labels rather than specific motor responses, the locations of the labels were randomly determined on each trial (i.e., “A” appeared at the bottom left of the screen on some trials, but at the bottom right on others). Participants responded by pressing the “d” key on the laptop keyboard if the chosen category label was on the bottom left side of the screen, and the “k” key if it was on the bottom right. Each image remained on the screen until the participant made a response. Following each response, auditory and visual feedback was presented for 0.75 seconds. Following correct responses, the word “Correct” was presented in the center of the computer screen with a pleasant tone. Following incorrect responses, the word “Wrong” was presented with an unpleasant tone. Every 100 trials, participants were given a self-paced break. Participants trained until they reached an 85% correct performance criterion on the final block of 100 trials.

To gain familiarity with the task that would later be used in the scanner, after reaching the 85% performance criterion, participants performed a second training task similar to that they would perform in the scanner (Figure 1 illustrates the task performed in the scanner). At the beginning of each trial, a cue was presented for 1.5 seconds, which instructed participants to either “Match the Specific Face,” or “Match the Category.” After this cue was presented, a face stimulus was presented for 1.5 seconds in the center of the computer screen. After a nine-second delay (during which time participants saw only a blank screen), a second face stimulus (or, in the Label condition, the category label “A” or “B”) was presented for three seconds. Two response cues, “Match” and “Mismatch” were then added to the display (at the bottom left and bottom right of the screen, respectively). On trials in which participants were cued to remember the specific face, participants had to indicate whether the second face was the same as the first. On trials in which participants were instructed to remember the category, they had to indicate whether the second stimulus (face or category label) belonged to the same category as the first. No feedback was delivered. Trials were separated by a 1.5 second inter-trial interval (ITI) during training. All participants performed 30 trials of the second training task. The assignment of condition to trial was random (selected with replacement), such that there were 10 trials per condition. As in the first training task, participants made their responses via the index fingers of their right and left hands using the “d” and “k” keys.

Figure 1.

During each trial, participants first saw a cue (Item condition: “Match the Specific Face”; Category and Label conditions: “Match the Category”) for 1.5 sec., they then saw the first stimulus (1.5 sec). After a brief delay (9 sec) they saw a second stimulus (3 sec). In the Category and Item conditions, the second stimulus was a face. In the Label condition, the second stimulus was the category label (“A” or “B”). After three seconds, a match mismatch cue was presented, and participants had to indicate whether the second stimulus matched the first. Trials were separated by a jittered ITI (1.5–9 sec).

In the scanner, the task was similar to the second pretraining task described above. Stimuli were, however, presented via a back-projection mirror positioned above the participant, and responses were made with fingers of the right and left hands via separate button boxes. The ITI was jittered according to a positively-skewed geometric distribution ranging from 1.5 to 9 seconds. Participants performed two 15-minute runs. In order to increase power for analyses of the Item trials in contrast with Categorical Encoding trials, we presented fewer Category (14) and Label (14) trials than Item trials (17) during each run. Both correct and incorrect trials were included in the analyses to maximize statistical power.

2.4 Image Acquisition

Images were obtained with a 3.0 Tesla MRI scanner (Siemens) at the Intermountain Neuroimaging Consortium (Boulder, CO). The scanner was equipped with a 12-channel head coil. Structural images were collected using a 3D T1-weighted rapid gradient-echo (MPRAGE) sequence (256 × 256 matrix; FOV, 256; 192 1-mm sagittal slices). Functional images were reconstructed from 28 axial oblique slices obtained using a T2* -weighted 2D-EPI sequence (TR, 1500ms; TE, 25ms; FA, 75; FOV, 220-mm, 96 × 96 matrix; 4.5-mm thick slices; no inter-slice gap). Each run consisted of 597 volumes. The first three volumes, which were collected before the magnetic field reached a steady state, were discarded.

2.4.1 Preprocessing

Image preprocessing was performed using SPM8 (http://www.fil.ion.ucl.ac.uk/spm/software/spm8). Preprocessing involved correction of slice time acquisition differences (images were adjusted to the 14th slice), motion correction of each volume to the first volume of the first run using 3rd degree B spline interpolation, coregistration of the functional to the structural data, normalization to the MNI template, smoothing (with a 6 mm Gaussian kernel), and temporal filtering (with a 128 sec. high-pass filter). One participant had excessive head-movement (defined as greater than 3mm translational or 2.5° rotational movement). This participant’s data were excluded from subsequent analyses.

2.5 FMRI Analyses

2.5.1 Univariate General Linear Model

Trial epochs (i.e., encoding, delay, and probe) were modeled as independent regressors in a univariate whole-brain analysis. All trials were included in this analysis. The encoding regressor was coded as a 1.5 s. boxcar coinciding with the presentation of the first stimulus (1.5–3 sec. after cue onset). The delay period was modeled as a 2 second boxcar placed halfway through the delay period (6–8 sec. after cue onset). The probe period was modeled as a 3 second boxcar coinciding with the presentation of the second stimulus (12–15 sec. after cue onset). As in previous research, the onsets of these regressors were placed at least 4 seconds apart to minimize the influence of preceding trial epochs (Barde & Thompson-Schill, 2002; Druzgal & D’Esposito, 2003; Gazzaley et al., 2007, 2004; Pessoa, Gutierrez, Bandettini, & Ungerleider, 2002; Postle, Zarahn, & D’Esposito, 2000; Rissman, Gazzaley, & D’Esposito, 2004; Rissman et al., 2008; Zarahn, Aguirre, & D’Esposito, 1997). We convolved each boxcar with the canonical SPM HRF. For each contrast, we generated maps at an uncorrected threshold of p < 0.001 and corrected for multiple comparisons using the topological false-discovery rate (Chumbley & Friston, 2009).

2.5.2 Constrained Principal Component Analyses (CPCA)

To investigate task-related differences across functional networks, we used Constrained Principal Component Analyses (CPCA) using a finite-impulse response (FIR) model, as implemented in the fMRI-CPCA toolbox (www.nitrc.org/projects/fmricpca). CPCA combines multivariate regression and principal component analysis to identify multiple functional networks involved in a given task, and has been used successfully with similar experimental paradigms (Metzak et al., 2012; Metzak et al., 2011; Woodward, Feredoes, Metzak, Takane, & Manoach, 2013). This approach is mathematically similar to Partial Least Squares analysis (McIntosh, Bookstein, Haxby, & Grady, 1996), and is attractive, as it allows estimation of changes in the BOLD response across peristimulus time within each functional network, and also allows statistical inference concerning the importance of each column of the design matrix for each component.

CPCA involves preparation of two matrices: Z, and G. Z contains the BOLD time course of each voxel, with one column per voxel and one row per scan. The design matrix, G contains a FIR model of the BOLD response related to the event onsets. The BOLD time-series in Z is regressed onto the design matrix, G, yielding a matrix, C, of regression weights. GC thus contains the variance in Z, that is accounted for by the design matrix, G. Components are then extracted from the variance in GC via singular value decomposition, yielding U, a matrix of left singular vectors, D a diagonal matrix of singular values, and V, a matrix of right singular vectors. The columns of VD, which reflect component loadings, can be overlaid on a structural image to visualize the functional networks. To maximize the variance of the squared loadings, we orthogonally rotated VD prior to display. The top 5% of these rotated loadings for each component are illustrated in Figures 3B, 4B, 5B, 6B and 7A. Several previous studies (e.g., Metzak et al., 2012; Metzak et al., 2011) have used a similar threshold. For each combination of peristimulus time-point, condition and participant, CPCA estimates a set of predictor weights (P), which are the values that relate the design matrix, G, to the networks associated with each component, such that U = G × P. All trials were included in this analysis.

Figure 3.

Component 1. Note the recruitment of regions involved in the salience network (inferior frontal/anterior insula and anterior cingulate) along with visual processing regions (fusiform gyrus and occipital lobe). A) The top 5% of component loadings overlaid on the MNI template provided by MRIcron (3d renderings, top) and the average structural image (slices, bottom). B) Predictor weight timecourse across peristimulus time. Error bars represent the standard error of the mean. Vertical lines indicate onsets of visual stimuli.

Figure 4.

Component 2. Note the recruitment of sensorimotor and premotor regions. A) The top 5% of component loadings overlaid on the MNI template provided by MRIcron (3d renderings, top) and an averaged structural image (slices, bottom). B) Predictor weight timecourse. Error bars represent the standard error of the mean. Vertical lines indicate onsets of visual stimuli.

Figure 5.

Component 3. Note the recruitment of FP-CEN regions including the lateral prefrontal cortex and intraparietal sulcus, along with the cerebellum and caudate. A) The top 5% of component loadings overlaid on the MNI template provided by MRIcron (3d renderings, top) and an averaged structural image (slices, bottom). B) Predictor weight timecourse. Error bars represent the standard error of the mean. Vertical lines indicate onsets of visual stimuli.

Figure 6.

Component 4. Note the recruitment of visual processing regions. A) The top 5% of component loadings overlaid on the MNI template provided by MRIcron (3d renderings, top) and an averaged structural image (slices, bottom).. B) Predictor weight timecourse. Error bars represent the standard error of the mean. Vertical lines indicate onsets of visual stimuli.

Figure 7.

Component 5. Note the recruitment of default mode network regions. A) The predictor weight timecourse. B) The top 5% of component loadings overlaid on the MNI template provided by MRIcron (3d rendering, left) and an averaged structural image (slices on the right). We changed the color map to cool colors to emphasize that Component 5 was anticorrelated with cognitive demands.

We conducted a repeated measures ANOVA using SPSS, which allowed us to investigate the consecutive scans where the slope of the predictor weight time course differed between conditions. This allows investigation of differences between conditions or contiguous time points, without considering the complex hemodynamic shape (cf., Metzak et al., 2012). For these analyses, and in Figures 3B, 4B, 5B, 6B and 7A, we adjusted the time-series so that the first observation was zero for all conditions (cf., Metzak et al., 2011, 2012; Woodward et al., 2013). We tested assumptions of sphericity, and controlled for violations using Greenhouse-Geisser adjusted degrees of freedom.

Readers more familiar with the interpretation of statistical maps derived from univariate analyses should take care when interpreting the multivariate results shown in Figures 3, 4, 5, 6 and 7. Whereas statistical maps derived from univariate analyses provide information about activity occurring within specific regions, each CPCA component reflects a pattern of task-related variance derived from all voxels in the brain. The maps derived from CPCA analyses, therefore, provide information about regions that cooperate to subserve a particular function. Whereas most univariate analyses assume a HDR shape by using a standard GLM with a canonical HRF, CPCA uses a FIR model which uncovers network-specific HDR shapes of task-related variance in a data-driven manner. This is valuable, as it can help segregate and characterize task-related processes that might not have been predicted by the experimenter.

In the present paper, we use univariate analyses to characterize activity occurring within specific regions of the brain, and we use CPCA to investigate how distributed regions coordinate to subserve different processes. In the tables provided, we label cluster peaks according to a 7-network parcellation identified in previous research (Buckner et al., 2011; Choi et al., 2012; Yeo et al., 2011), but refer to some regions that these papers term the ventral attentional network as the salience network (SA), in line with current usage (Buckner, Krienen, & Yeo, 2013).

3 Results

3.1 Behavioral

All trials for which participants made a behavioral response were included in all analyses. Accuracy was highest for the Item conditions (M = 94.87% correct, SD = 5.83), lower for the Label condition (M= 81.11%, SD=12.36) and lowest for the Category condition (M = 75.46%, SD = 16.71), F(2,45) = 10.25, p < .001, η² = 0.31. The accuracy difference between the Label and Category conditions is likely related to the different number of categorization decisions required for each condition: the Category trials required participants to categorize stimuli at encoding and probe, and an error on either decision could lead to an incorrect response, whereas the Label trials required participants to categorize stimuli only at encoding. Performance in the Label condition was close to that of the 85% accuracy criterion from the learning phase. We did not collect reaction time data because it was unlikely to be of interest due to the requirement that participants delay their response until the response cue was presented.

3.2 Neuroimaging

3.2.1 Univariate GLM Analyses

Whole brain GLM analyses were used to examine regions of activity during each trial epoch: encoding, delay and probe (cf., Gazzaley et al., 2007, 2004). Because the Category and Label trials were identical until the onset of the second stimulus, these conditions were combined as the “categorical-encoding” condition for examination of activity during the encoding and delay epochs. In this section we present univariate regions of activity across the whole brain; we also discuss these results masked by each component later following the CPCA results.

As can be seen in Table A.1 and Figure 2, during encoding, the Categorical-Encoding trials elicited greater activity than Item trials primarily within frontal lobe regions (middle cingulate, superior medial gyrus, inferior frontal / anterior insula) and subcortical regions that are known to interact with the frontal lobe (right caudate, left cerebellar lobule VI). These regions participate in the frontoparietal intrinsic connectivity network (Yeo et al., 2011). In addition, categorical encoding recruited visual regions including the bilateral calcarine gyri. The Item condition elicited greater activity than Categorical-Encoding trials in the left inferior frontal gyrus, regions of the temporal and occipital lobe associated with high level visual processing, and the bilateral hippocampus. All of these regions have been identified in previous studies as being recruited during visual working memory encoding (Gazzaley et al., 2007; Sala et al., 2003). The only region showing significant activation in response to both conditions (conjunction analysis, contrast with implicit baseline) at encoding was the crus I region of the right cerebellum.

Table A.1.

Univariate results for the encoding, delay and probe epochs. As Category and Label trials were methodologically identical during encoding and delay, they were combined into a single Categorical-Encoding condition.

Contrast			Peak MNI Coordinates
	Voxels	mm^3	x	y	z	Region	BA	Network
Encoding
Categorical Encoding >item
	1185	9480
			−2	−90	−4	Left Calcarine Gyrus	17	VS
			2	−76	8	Right Calcarine Gyrus	18	VS
	731	5848
			12	−64	40	Right Precuneus	7	FP-CEN
			−4	−64	44	Left Precuneus	7	DMN
	357	2856	−4	−28	28	Middle Cingulate	23	SA
	152	1216	12	6	14	Right Caudate Nucleus		FP-CEN
	106	848	−6	−18	10	Left Thalamus
	99	792	−12	−80	−22	Left Cerebelar Lobule VI (hem)	18	FP-CEN
	79	632	4	28	42	Right Superior Medial Gyrus	6	SA
	75	600	30	22	−4	Right Inferior Frontal / Anterior Insula	47	SA
Item> Categorical Encoding
	451	3608	−42	−66	−12	Left Inferior Occipital Gyrus	19	VS
	284	2272
			−46	−8	−30	Left Inferior Temporal Gyrus	20	DMN
			−24	−2	−22	Left Amygdala	28
			−36	−2	−20	Superior Temporal Pole	36
			−22	−12	−18	Left Hippocampus	35
	177	1416	−4	52	−16	Left Rectal Gyrus	11	DMN
	124	992	22	−10	−14	Right Hippocampus
	111	888	44	−62	−6	Right Inferior Temporal Gyrus	36	VS/DA
	91	728	−42	28	−16	Left Inferior Frontal Gyrus	38	DMN
	87	696	−44	−46	−14	Left Inferior Temporal Gyrus	37	DA
Conjunction (Item> Implicit Baseline & Categorical Encoding Trials > Implicit Baseline)
	100	800	40	−73	−29	Right Cerebellar Crus I		FP-CEN/DMN
Delay
Conjunction (Item>Categorical Encoding & Item > Implicit Baseline)
	131	1048	56	−18	10	Right Superior Temporal		SM
	52	416	−6	−18	44	Middle Cingulate		SM/VA
Probe
Item > Label
	1325	10600
			42	−82	−2	Right Inferior Occipital Gyrus	19	VS
			42	−74	−10	Right Inferior Occipital Gyrus	19	VS
			42	−46	−16	Right Fusiform Gyrus	37	DA
	937	7496
			−26	−94	−4	Left Inferior Occipital Gyrus	18	VS
			−38	−84	−14	Left Fusiform Gyrus	19	VS
Label > Item
	191	1528	16	−76	24	Right Cuneus	18	VS
	135	1080	−50	−50	6	Left Middle Temporal Gyrus/ Lateral Occipital	21	VA
	103	824	−14	−88	24	Left Superior Occipital Gyrus	18	VS
	84	672	−4	−68	46	Left Precuneus	7	FP-CEN
Label > Category
	513	4104
			18	−88	18	Right Superior Occipital Gyrus	18	VS
			18	−78	24	Right Cuneus	19	VS
	325	2600	−56	−56	10	Left Middle Temporal Gyrus	37	VA
	298	2384	−12	−90	24	Left Superior Occipital Gyrus	18	VS
	156	1248	52	−32	16	Right Superior Temporal Gyrus	41	SM
	120	960	−38	−18	−8	Left Insula (Id1)	20	VA
	112	896	−54	−2	−10	Left Superior Temporal Gyrus	22	DMN
	102	816	10	−70	−6	Right Lingual Gyrus	18	VS
	81	648	0	−24	60	Left Paracentral Lobule	6	SM
	80	640	68	−30	30	Right SupraMarginal Gyrus	2	VA
	77	616	−6	−70	−2	Left Lingual Gyrus	18	VS
	71	568	−32	0	12	Left Insula	48	VA
	67	536	−62	−24	16	Left SupraMarginal Gyrus	42	VA
	63	504	60	−58	20	Right Superior Temporal Gyrus	21	DMN
	62	496	−32	−38	−14	Left Fusiform Gyrus	37	VS
Category > Label
	1629	13032
			42	−88	−2	Right Inferior Occipital Gyrus	19	VS
			42	−74	−10	Right Inferior Occipital Gyrus	19	VS
			42	−52	−16	Right Fusiform Gyrus	37	VS
			32	−92	−2	Right Inferior Occipital Gyrus	18	VS
	1114	8912
			−26	−94	−4	Left Inferior Occipital Gyrus	18	VS
			−38	−82	−12	Left Inferior Occipital Gyrus	19	VS
	211	1688
			2	28	46	Right Superior Medial Gyrus	8	SA
			8	30	30	Right Middle Cingulate Cortex	32	SA
	148	1184	0	−44	20	Posterior Cingulate		DMN
	136	1088	−8	−74	−38	Left Cerebellar Lobule VIIb (Hem)		DA
	97	776	−6	−32	28	Posterior Cingulate	23	FP-CEN
	97	776	−2	−34	−6	Superior Colliculus
Item > Category
	122	976	−58	−2	−18	Left Middle Temporal Gyrus	21	DMN
Category > Item
	863	6904
			−6	−72	44	Left Precuneus	7	FP-CEN
			8	−70	44	Right Precuneus	7	FP-CEN
	758	6064	−34	−56	46	Left Inferior Parietal Lobule	7	FP-CEN
	555	4440
			0	28	42	Left Superior Medial Gyrus	6	SA
			−2	14	46	Left preSMA	6	SA
	312	2496
			−30	6	60	Left Middle Frontal Gyrus	6	SA
			−48	6	44	Left Precentral Gyrus	44	FP-CEN
	252	2016	34	−60	48	Right Angular Gyrus	7	FP-CEN
	91	728	−4	−38	24	Left Posterior Cingulate Cortex	26	FP-CEN
	76	608	−44	30	32	Left Middle Frontal Gyrus	45	FP
	71	568	−36	18	−2	Left Inferior Frontal / Anterior Insula	47	SA
	59	472	−8	−74	−26	Left Cerebellar Lobule VI (Hem)		FP-CEN
Conjunction(Category> Implicit, Label > Implicit, & Item> Implicit)
	121	968	−1	2	50	Bilateral preSMA	6	SA
	39	312	−16	−14	15	Left Thalamus
	26	208	19	−13	16	Right Thalamus

VS=Visual; SM=Somatomotor; DA=Dorsal Attention; FP-CEN=Central Executive; SA=Salience; DMN=Default Mode Network. Cluster volumes smaller than 10 voxels have been omitted.

Figure 2.

Whole brain univariate analyses: activity differing between conditions within individual trial epochs (Encoding and Probe). TOP FIGURE: Encoding epoch. Red: Categorical Encoding (Category and Label trials) greater than Item; Blue: Item greater than Categorization Encoding. BOTTOM FIGURES: Probe epoch. TOP: Green: Label greater than Item; Blue: Item greater than Label. MIDDLE: Green: Label greater than Category; Red: Category greater than Label. BOTTOM: Red: Category greater than Item. Blue: Item greater than Category. Regions of activity are overlaid on the average normalized anatomical image across subjects. For each contrast, we generated maps at an uncorrected threshold of p < 0.001 and corrected for multiple comparisons using the topological false-discovery rate (q < .05; Chumbley & Friston, 2009).

During the delay period, the majority of regions sensitive to differences between conditions showed patterns of activity suppressed below implicit baseline (cf. Gazzaley, Rissman, & D’Esposito, 2004). To avoid difficulties in interpreting deactivation, we conducted a conjunction analysis, and have reported only regions that showed activity greater than implicit baseline and were also sensitive to direct contrasts between conditions. We found that regions in the right superior temporal lobe and middle cingulate gyrus showed significantly greater activity during Item trials than during Categorical-Encoding trials. No regions showed greater activity during Categorical-Encoding trials than during Item trials.

During the probe epoch, Category and Label trials were analyzed separately, and compared with each other and with Item trials. Not surprisingly, during the probe epoch, conditions in which participants viewed faces (Category and Item) had greater activity in higher order visual processing regions than the Label condition (in which participants viewed the Category Label, “A” or “B”). As shown in Figure 2, these included bilateral inferior occipital and bilateral fusiform gyri. Conversely, Label trials led to greater activity than Category and Item trials in other visual processing regions including the right cuneus / superior occipital gyrus, and a region of the left fusiform (Label > Category only). These differences are likely due to visual processing differences between faces and letters. In addition to visual regions, the Category > Label contrast during the probe epoch revealed recruitment of frontoparietal regions including the right superior medial gyrus, and the middle and posterior cingulate, along with a region of the cerebellum (left cerebellar crus II).

The Category versus Item contrast (see Figure 2, bottom row) was the most direct comparison between categorization and item recognition; both conditions had similar requirements for viewing and processing face stimuli and making same-different judgments. The only region showing greater activity during Item trials than during Category trials was a region in the left middle temporal gyrus. Category trials elicited greater activity than Item trials in executive regions of the cerebellum, frontal (middle frontal, anterior insula/inferior frontal, and superior medial gyrus) and parietal regions (inferior parietal, angular gyrus, and precuneus), including the salience network. Finally a conjunction analysis revealed that motor planning regions of the SMA were recruited in all three conditions, consistent with the similar motor response demands across the conditions.

3.2.2 CPCA

The GLM model, GC, accounted for 36.31% of the variance in the BOLD signal. Based on inspection of the scree-plot, we extracted five components. After varimax rotation, the first through fifth components accounted for 14.15%, 8.13%, 4.95%, 4.90%, and 4.16% of the task-related variance, respectively.

3.2.2.1 Component 1

Component 1 is illustrated in Figure 3; cluster coordinates are given in Table A.2. This component was characterized by activity in the bilateral fusiform gyrus and occipital regions, bilateral thalamus, bilateral parietal regions, bilateral inferior frontal gyrus and anterior insula, and bilateral medial frontal gyrus. Overall this component overlapped with the visual, dorsal attentional, FP-CEN, and SA networks (Buckner et al., 2011; Dosenbach, Visscher, & Palmer, 2006; Seeley et al., 2007). Inspection of the predictor weights (Figure 3b) revealed a bimodal shape, and an ANOVA revealed a significant main effect of condition, F(2,30) = 6.1, p < .05, η²= .01, a significant main effect of time-point, F(17,255) = 27.29, p < .001, η² = .56, and an interaction between condition and time-point, F(34,510) = 9.88, p < .001, η² = .04. This interaction was driven by differences following the second stimulus. The slope of the predictor weight timecourse differed between the Category and Label trials for the two consecutive time points between 2.25 and 5.25 seconds (p<.001 for both time points), and the time points between 6.75 and 11.25 seconds (p<.05; p<.001; p<.001) following the second stimulus. The slope also differed between the Category and Item conditions (2.25 to 3.75, 6.75 to 8.25, 9.75 to 11.25, and 14.25 to 15.75 seconds; p<.05 for all time-points), and between the Item and Label trials (2.25 to 5.25 seconds and 6.75 to 12.75 seconds; p<.05 for all time points). Following the second stimulus, the predictor weights associated with the Category and Item conditions showed greater amplitude than those of the Label condition, suggesting that the Category and Item conditions placed greater demands on the Component 1 network than did the Label condition. This might have been due to the visual differences between faces and labels, or due to differences in higher-order cognitive demands.

Table A.2.

Cluster volumes and peak coordinates for the rotated top 5% of Component 1 loadings.

		Peak MNI Coordinates
Voxels	mm^3	x	y	z	Region	BA	Network
3755	30040
		32	−90	0	Right Middle Occipital Gyrus	18	VS
		42	−56	−20	Right Fusiform Gyrus	37	VS
		40	−78	−12	Right Inferior Occipital Gyrus	19	VS
		32	−60	46	Right Angular Gyrus	7	DA/FP-CEN
		30	−52	46	Right Inferior Parietal Lobule	7	DA/FP-CEN
2916	23328
		−32	−90	−6	Left Middle Occipital Gyrus	19	VS
		−38	−82	−12	Left Inferior Occipital Gyrus	19	VS
		−40	−72	−14	Left Fusiform Gyrus	19	VS
		−38	−56	−20	Left Fusiform Gyrus	37	VS
1021	8168
		−28	−70	26	Left Middle Occipital Gyrus	19	VS/DA
		−28	−62	48	Left Superior Parietal Lobule	7	DA
		−30	−48	42	Left Inferior Parietal Lobule	40	DA
627	5016
		−4	10	50	Left preSMA / Medial Frontal Gyrus	6	SA
		4	16	46	Right preSMA / Medial Frontal Gyrus	6	SA
525	4200
		48	12	30	Right Inferior Frontal Gyrus (p. Opercularis))	44	DA
		44	6	34	RightPrecentral Gyrus	44	DA
		48	32	18	Right Inferior Frontal Gyrus (p. Triangularis)	45	SA
113	904
		12	−64	42	Right Precuneus	7	FP-CEN
110	880
		−46	4	34	Left Precentral Gyrus	44	DA
		−46	12	26	Left Inferior Frontal Gyrus (p. Triangularis)	44	SA
95	760	32	24	−4	Right Anterior Insula	47	SA
89	712	−10	−72	44	Left Precuneus	7	FP-CEN
71	568	−6	−76	−24	Left Cerebellar Lobule VI		FP-CEN
68	544	10	−14	8	Right Thalamus		SA
63	504	−2	−30	26	Middle/Posterior Cingulate	23	FP-CEN
60	480	−10	−16	8	Left Thalamus		SA
49	392	−30	24	−2	Left Anterior Insula	47	SA
35	280	−48	0	50	Left Precentral Gyrus	6	DA/SM

VS=Visual; SM=Somatomotor; DA=Dorsal Attention; FP-CEN=Central Executive; SA=Salience. Cluster volumes smaller than 10 voxels have been omitted.

3.2.2.2 Component 2

Component 2 loadings (Figure 4 and Table A.3) were associated with regions largely involved in motor and visual processing, including motor and premotor cortex (e.g., precentral and postcentral gyri, and the SMA) and visual regions (superior occipital, lateral occipital and lingual gyri). This component largely overlaps with the visual and somatomotor networks, along with some adjoining areas of the salience network and dorsal and ventral attentional networks (Yeo et al., 2011). Inspection of the predictor weights revealed a unimodal peak occurring roughly 8 seconds after the onset of the second stimulus. An ANOVA on the predictor weights revealed a significant main effect of condition, F(2,30) = 19.23, p< .001, η²=.03, a main effect of time-point F(17,255) = 5.00, p < .001, η²=.18, and an interaction between condition and time-point F(34,510) = 2.9, p < .001, η²=.04. There was a significant difference in the slope of the predictor weight time course between the Category and Label condition following the first peak (6.75–8.25 seconds after the first stimulus; p< .05), but the effects predominantly followed the second stimulus. The slope between the Category and Label Trials differed between 2.25 and 3.75 seconds and 12.75 to 14.25 seconds following the second stimulus (p<.05 for both time points). The slopes associated with the Label and Item conditions differed for the consecutive time points between .75 and 3.75 seconds (p<.05); and the slopes of the Category and Item conditions differed between 3.75 and 5.25 seconds (p.<.05). Overall, following the second stimulus, the predictor weight peak was greatest for the Label condition followed by the Item condition, and then the Category Condition.

Table A.3.

Table A.3. Cluster volumes and peak coordinates for the rotated top 5% of Component 2 loadings.

		Peak MNI Coordinates
Voxels	mm^3	x	y	z	Region	BA	Network
3170	25360
		−56	−22	40	Left Inferior Parietal Lobule	3	DA
		−58	−20	18	Left Postcentral Gyrus	48	SM
		−38	−40	58	Left Postcentral Gyrus	2	SM
		−62	−36	24	Left SupraMarginal Gyrus	48	SA
		−46	−36	22	Left Superior Temporal Gyrus	48	SM
		−42	−12	52	Left Precentral Gyrus	6	SM
2679	21432
		56	−18	46	Right Postcentral Gyrus	1	SM
		48	−20	18	Right Rolandic Operculum	48	SM
		46	−26	58	Right Postcentral Gyrus	1	SM
		64	−40	22	Right Superior Temporal Gyrus	22	SA
		64	−24	24	Right SupraMarginal Gyrus	48	SA
		38	−26	62	Right Precentral Gyrus	4	SM
1243	9944
		2	−4	52	Right preSMA	6	SM
		−2	6	40	Left Middle Cingulate Cortex	24	SA
		12	−28	44	Right Middle Cingulate Cortex		SA
1085	8680
		−6	−72	−2	Left Lingual Gyrus	18	VS
		8	−68	−2	Right Lingual Gyrus	18	VS
		−20	−60	−14	Left Cerebellum Lobule VI	19
		20	−60	−12	Right Fusiform Gyrus	19	VS
406	3248
		−50	−2	0	Left Insula	48	VA/SM
		−44	−4	10	Left Insula	48	VA/SM
		−56	8	20	Left Inferior Frontal Gyrus	44	SA
306	2448	−8	−94	14	Left Cuneus	18	VS
277	2216
		44	2	6	Right Insula	48	SA
		58	10	8	Right Inferior Frontal Gyrus	44	SA
243	1944	14	−90	20	Right Cuneus	18	VS
54	432	20	−4	68	Right Superior Frontal Gyrus	6	VA/SM
45	360	50	−66	8	Right Lateral Occipital Gyrus	37	VS
36	288	−12	−26	40	Left Middle Cingulate Cortex		SA
32	256	−52	−74	8	Left Lateral Occipital Gyrus	19	VS
18	144	12	−18	6	Right Thalamus

VS=Visual; SM=Somatomotor; DA=Dorsal Attention; SA=Salience. Cluster volumes smaller than 10 voxels have been omitted.

3.2.2.3 Component 3

The top 5% of loadings on Component 3 (see Figure 5 and Table A.4) were associated primarily with regions within the FP-CEN network (Buckner et al., 2011; Choi et al., 2012; Yeo et al., 2011), including extensive regions of the inferior and medial parietal lobe, regions of the inferior, middle, and superior frontal gyri, and subcortical regions including the caudate nucleus and cerebellar regions. In addition, this component included primary visual cortex and regions of the medial frontal gyrus, inferior parietal lobe, and cerebellum associated with the default network. Inspection of the predictor weights associated with Component 3 revealed a bimodal shape similar to that of Component 1, but with peaks delayed several seconds in time. An ANOVA on the predictor weights (shown in Figure 5B) revealed a significant main effect of condition, F(2,30) = 13.24, p < .001, η²=.04, a main effect of time-point F(17,255) = 4.67, p < .001, η²=.16, and an interaction between condition and time-point F(34,510) = 2.34, p < .001, η²=.03. This interaction was driven by effects following the second stimulus. The predictor weight slopes were significantly steeper in the Category condition than in the Label conditions from 6.75 to 8.25 seconds following the second stimulus (p<.01). The slope was steeper in the item condition than the Label condition from 2.25 to 3.75 seconds (p<.05) and during the consecutive time points from 6.75 to 9.75 seconds (p<.01; p<.05). From 8.25 to 9.75 seconds, the slope associated with the Item condition was steeper than that of the Category condition (p < .05), an effect driven by a slightly shorter time-to-peak in the Category condition.

Table A.4.

Cluster volumes and peak coordinates for the rotated top 5% of Component 3 loadings.

		Peak MNI Coordinates
Voxels	mm^3	x	y	z	Region	BA	Network
5040	40320
		4	−72	46	Right Precuneus	7	FP-CEN
		−4	−66	58	Left Precuneus	7	DA
		−38	−58	38	Left Angular Gyrus	40	FP-CEN
		−32	−72	42	Left Inferior Parietal Lobule	7	FP-CEN
		38	−64	48	Right Angular Gyrus	7	FP-CEN
		−44	−58	48	Left Inferior Parietal Lobule	39	FP-CEN
		40	−56	42	Right Inferior Parietal Lobule	40	FP-CEN
		0	−34	24	Bilateral Posterior Cingulate	23	FP-CEN
2774	22192
		12	−88	−26	Right Cerebellum Crus I		DMN
		−4	−82	−22	Left Cerebellum Lobule VI		FP-CEN
		−8	−88	−26	Left Cerebellum Crus I		FP-CEN
		44	−72	−30	Right Cerebellum Crus I		DMN
		6	−94	−10	Right Calcarine Gyrus		VS
		−40	−78	−30	Left Cerebellum Crus I		DMN
1061	8488
		0	−8	10	Thalamus
		18	4	22	Right Caudate Nucleus		FP-CEN
		−10	−6	12	Thalamus
		−18	−14	24	Left Caudate Nucleus		FP-CEN
174	1392	46	38	28	Right Middle Frontal Gyrus	45	FP-CEN
165	1320	−38	58	2	Left Superior Frontal Gyrus	10	FP-CEN
158	1264
		0	34	38	Bilateral Superior Medial Gyrus	32	FP-CEN/DMN
		0	24	48	Bilateral preSMA	6	FP-SA
77	616	−12	−92	−2	Left Calcarine Gyrus	17	VS
71	568	−50	18	34	Left Middle Frontal Gyrus	45	FP-CEN
32	256	34	62	10	Right Superior Frontal Gyrus	10	FP-CEN

VS= Visual; SM=Somatomotor; DA=Dorsal Attention; FP-CEN=Central Executive; SA=Salience; = DMN=Default Mode Network. Cluster volumes smaller than 10 voxels have been omitted.

3.2.2.4 Component 4

Component 4 (see Figure 6 and Table A.5) was associated exclusively with regions of the visual network (Yeo et al., 2011), and included regions extending from the primary visual cortex superiorly to the precuneus and anteriorly to the inferior temporal lobe. An ANOVA on the predictor weight timecourse indicated a significant main effect of condition, F(1.44,21.63) = 4.55, p < .05, η²=.01, and a main effect of time-point F(4.25,63.78) = 7.76, p < .001, η²=.26. The interaction between condition and time-point was not significant, F(7.78,116.69) = 1.63, p >.05, η²=.02.

Table A 5.

Cluster volumes and peak coordinates for the rotated top 5% of Component 4 loadings.

		Peak MNI Coordinates
Voxels	mm^3	x	y	z	Region	BA	Network
9600	76800
		0	−80	8	Left Calcarine Gyrus	17	VS
		−40	−88	−2	Left Middle Occipital Gyrus	19	VS
		−38	−80	−14	Left Fusiform Gyrus	19	VS
		16	−94	−2	Right Calcarine Gyrus	17	VS
		0	−84	26	Left Cuneus	18	VS
		10	−64	8	Right Lingual Gyrus	17	VS
		−12	−62	4	Left Calcarine Gyrus	17	VS
		−36	−88	−12	Left Inferior Occipital Gyrus	19	VS
		−28	−68	−18	Left Cerebellum	19	VS
		6	−80	42	Right Cuneus	7	VS
		−4	−82	42	Left Superior Occipital Gyrus	19	VS
		36	−68	−20	Right Fusiform Gyrus	19	VS

VS=Visual. Cluster volumes smaller than 10 voxels have been omitted.

3.2.2.5 Component 5

The top 5% of loadings (Table A.6 & Figure 7) were associated primarily with regions within the default mode network, including medial parietal, medial prefrontal, middle temporal gyrus, and the hippocampus (Buckner et al., 2011; Choi et al., 2012; Yeo et al., 2011). An ANOVA on the predictor weight time course revealed a significant main effect of condition, F(2,30) = 12.56, p < .01, η²= .46, a significant main effect of time-point, F(17,255) = 19.24, p < .001, η² = .41, and an interaction between condition and time-point, F(34,510) = 2.5, p < .001, η² = .14.

Table A.6.

Cluster volumes and peak coordinates for the rotated top 5% of Component 5 loadings.

		Peak MNI Coordinates
Voxels	mm^3	x	y	z	Region	BA	Network
4231	33848
		2	54	2	Right Superior Medial Gyrus	10	DMN
		−2	52	−12	Left Mid Orbital Gyrus	11	DMN
		0	62	6	Left Superior Medial Gyrus		DMN
		6	52	16	Right Superior Medial Gyrus	32	DMN
		−8	42	−8	Left Mid Orbital Gyrus	11	DMN
		−12	66	22	Left Superior Frontal Gyrus	10	DMN
1770	14160
		−4	−56	18	Left Precuneus	7	DMN
		8	−52	8	Right Precuneus	17	DMN
1301	10408
		−50	−68	24	Left Angular Gyrus	39	DMN
		−62	−50	4	Left Middle Temporal Gyrus	21	DMN
523	4184	50	−10	−16	Right Middle Temporal Gyrus	20	DMN
514	4112	52	−56	18	Right Middle Temporal Gyrus	21	DA/DMN
424	3392
		−58	−2	−18	Left Middle Temporal Gyrus	21	DMN
		−54	−14	−14	Left Middle Temporal Gyrus	20	DMN
		−64	−18	−8	Left Middle Temporal Gyrus	21	DMN
258	2064	26	−20	−18	Hipp (SUB)	20
186	1488	−42	22	−20	Left Temporal Pole)	38	DMN
134	1072	−24	−22	−18	Hipp (SUB)	30
36	288	−20	−86	−10	Left Fusiform Gyrus	18	VS

VS=Visual; DA=Dorsal Attention; DMN=Default Mode Network. Cluster volumes smaller than 10 voxels have been omitted.

The interaction was driven by differences following the second stimulus: the slopes differed between the Category and Label conditions from 2.25 to 3.75 seconds (p<.05), from 5.25 8.25 (p<.01; p<.01), and between the Category and Item conditions from .75 to 2.25 (p<.05), and from 11.25 to 14.25 (p<.05; p<.05) seconds following the second stimulus. As predicted based on task difficulty, we found that Category (M = −0.11, SD= .08, t(15)=3.04, p < .01, g= .61) and Label (M= −.13, SD= .09, t(15)= 4.97, p < .001, g= 0.7) trials elicited greater suppression across the predictor weight time series than Item trials (M= −.05, SD= .11).

3.2.3 CPCA-masked Univariate GLM Analyses

In order to further explore how regions within CPCA components were affected by task differences, we examined our univariate results within masks formed by each individual component. However, it is important to understand the limitations of this approach. Whereas the CPCA results highlight patterns of task-related variance shared between brain regions, univariate analyses ignore these patterns of shared variance and consider only variance within specific clusters. As a result, univariate results can represent several sources of overlapping variance, and may, therefore, not closely resemble the patterns revealed by CPCA (e.g., Components 1 and 3 overlapped in regions of the bilateral precuneus, and Components 3 and 4 overlapped in bilateral regions of the lingual gyrus). While univariate analyses can be conceptualized as providing a view of task-related variance that slices across network variance, CPCA can be conceptualized as providing a view that slices across the variance within specific brain regions. The evaluation of the statistical reliability of the results from each analysis reflects this distinction; univariate results are evaluated for each cluster separately, while CPCA results are evaluated at the level of the hemodynamic response of the entire network. Additionally, whereas the univariate analyses assume a canonical hemodynamic response associated with each task epoch, CPCA uses a flexible FIR model, and is capable of uncovering patterns of task-related variance in a data-driven manner. Because the univariate analyses are sensitive to matches between the network-unspecific hemodynamic response and the canonical hemodynamic response function modeling each task epoch separately, whereas the multivariate analysis reflect network-specific, data-driven, hemodynamic response shapes during overall task performance, we consider the pattern of univariate results to convey a broader, network-unspecific view of task related activity, although this activity is restricted to matches to the canonical HRFs.

3.2.3.1 Component 1

As shown in Table A.7, consistent with the predictor weight time course, we found that visual regions within Component 1 tended to respond preferentially to visual stimulus features at probe; regions in the bilateral inferior occipital gyrus, bilateral fusiform, and the left calcarine showed greater activity when a face was presented (i.e., during both Category and Item trials) than when a Label was presented. Interestingly, neighboring regions within the fusiform showed greater activity during the encoding epoch when it was necessary to encode a specific stimulus (Item trials) than when it was necessary to categorize it (Category and Label trials), indicating that this region was sensitive to specific-item encoding demands. Taken together, these results indicate that these visual regions have both feature specific processing roles, and functional roles within the task-related salience network in responding to stimuli.

Table A.7.

Cluster volumes and MNI peak coordinates regions within Component 1 that showed significant differences between conditions, based on the full-brain univariate statistics.

			Peak MNI Coordinates
	Voxels	mm^3	x	y	z	Region	BA	Network
Encoding
Categorical Encoding > Item
	103	824	12	−64	40	Right Precuneus	7	FP-CEN
	55	440	−4	−28	28	Bilateral Posterior Cingulate	23	FP-CEN
	54	432	32	23	−4	Right Anterior Insula	47	SA
	51	408	4	28	42	Right Superior Medial Gyrus	6	SA
	35	280	−10	−78	−24	Left Cerebellar Lobule VI		FP-CEN
	35	280	−4	−70	46	Left Precuneus	7	FP-CEN
	32	256	−6	−18	10	Left Thalamus
Item > Categorical Encoding
	275	2200	−42	−66	−12	Left Inferior Occipital Gyrus	19	VS
	50	400	44	−62	−6	Right Inferior Temporal Gyrus	37	VS
	24	192	−42	−46	−16	Left Inferior Temporal Gyrus	37	DA
Probe
Item > Label
	1271	10168
			42	−82	−2	Right Inferior Occipital Gyrus	19	VS
			42	−74	−10	Right Inferior Occipital Gyrus	19	VS
			42	−46	−16	Right Fusiform Gyrus	37	DA/VS
	851	6808
			−26	−94	−4	Left Inferior Occipital Gyrus	18	VS
			−38	−84	−14	Left Fusiform Gyrus	19	VS
Label > Item
	11	88	−6	−68	46	Left Precuneus	7	FP-CEN
Category > Label
	1471	11768
			42	−88	−2	Right Inferior Occipital Gyrus	19	VS
			42	−52	−16	Right Fusiform Gyrus	37	VS/DA
			32	−92	−2	Right Inferior Occipital Gyrus	18	VS
			42	−50	−28	Right Cerebellar Lobule VI (Hem)	37	SA
	999	7992
			−26	−94	−4	Left Inferior Occipital Gyrus	18	VS
			−38	−82	−12	Left Inferior Occipital Gyrus	19	VS
			−40	−58	−24	Left Cerebellar Lobule VI	37	SA
	96	768	2	28	46	Bilateral Superior Medial Gyrus	8	SA
	24	192	−4	−32	28	Bilateral Posterior Cingulate	23	FP-CEN
Category > Item
	412	3296	−34	−56	46	Left Inferior Parietal Lobule	7	FP-CEN
	309	2472
			0	28	42	BIlateral Superior Medial Gyrus	32	SA
			−2	14	46	Left preSMA	6	SA
			−2	6	58	Left preSMA	6	SA
	190	1520	34	−60	48	Right Angular Gyrus	7	FP-CEN
	103	824	8	−70	44	Right Precuneus	7	FP-CEN
	75	600	−6	−72	44	Left Precuneus	7	FP-CEN
	30	240	−2	−36	24	Posterior Cingulate	23	FP-CEN

VS=Visual; SM=Somatomotor; DA=Dorsal Attention; FP-CEN=Central Executive; SA=Salience; DMN=Default Mode Network. Cluster volumes smaller than 10 voxels have been omitted.

A second pattern was that frontal and parietal regions associated with FP-CEN and SA tended to show sensitivity to categorization demands. Regions in the superior medial frontal cortex, the left cerebellar crus 1, and the bilateral precuneus showed greater activity for categorical encoding trials than for Item trials during encoding, and also showed greater activity for Category trials than for Item trials during probe. Interestingly, only the left precuneus, a region thought to be a hub in the FP-CEN (Niendam, Laird, & Ray, 2012; Spreng, Sepulcre, Turner, Stevens, & Schacter, 2013) also showed greater activity for the Label condition than the Item condition at probe. The right anterior insula, a key node in the salience network (Seeley et al., 2007), and a region of the thalamus known to be connected with the prefrontal cortex (Behrens et al., 2003), showed greater activity for the categorical-encoding trials than the Item trials during encoding, but showed no differences between conditions at probe.

3.2.3.2 Component 2

Mirroring the predictor weight timecourse, we found that visual cortical regions of Component 2 were sensitive to visual stimulus features, as detailed in Table A.8. Regions of the left lingual and superior occipital gyrus showed greater activity during categorical encoding than during item trials at encoding, while the right superior temporal lobe showed the opposite pattern (item > categorical encoding). The primary effect, however, was that regions of the bilateral lingual gyrus, bilateral occipital gyri, and left supramarginal gyrus were more active for Label than for Category and/or Item trials. The univariate results, however, did not reveal similar patterns of activity within the motor regions associated with Component 2.

Table A 8.

Cluster volumes and peak coordinates for the rotated top 5% of Component 4 loadings.

		Peak MNI Coordinates
Voxels	mm^3	x	y	z	Region	BA	Network
9600	76800
		0	−80	8	Left Calcarine Gyrus	17	VS
		−40	−88	−2	Left Middle Occipital Gyrus	19	VS
		−38	−80	−14	Left Fusiform Gyrus	19	VS
		16	−94	−2	Right Calcarine Gyrus	17	VS
		0	−84	26	Left Cuneus	18	VS
		10	−64	8	Right Lingual Gyrus	17	VS
		−12	−62	4	Left Calcarine Gyrus	17	VS
		−36	−88	−12	Left Inferior Occipital Gyrus	19	VS
		−28	−68	−18	Left Cerebellum	19	VS
		6	−80	42	Right Cuneus	7	VS
		−4	−82	42	Left Superior Occipital Gyrus	19	VS
		36	−68	−20	Right Fusiform Gyrus	19	VS

VS=Visual. Cluster volumes smaller than 10 voxels have been omitted.

3.2.3.3 Component 3

Overall, many regions within the component 3 mask in the univariate analysis (Table A.9) showed a pattern of higher activity for both Label and Category than Item at encoding, along with greater activity for Category than both Label and Item at probe. Specifically, at encoding, frontoparietal (right precuneus, left cerebellar lobule VI, and middle cingulate) and visual (left calcarine gyrus) regions showed greater activity during categorical-encoding trials than during Item-trials. At probe there was greater activity in FP-CEN regions during Category trials than during Label trials (right superior medial gyrus, posterior cingulate, bilateral cerebellar crus I), or Item trials (bilateral precuneus, left inferior parietal lobe, right angular gyrus, bilateral superior medial gyrus, and posterior cingulate). Overall, the univariate results were similar to the multivariate results for these regions, despite the fact that each analysis approach captures different sources of variance. However, the analysis approaches differed with respect to subcortical regions. Although the basal ganglia was involved in this component overall, its activity did not differ significantly across conditions in the univariate analysis, indicating that these regions likely played an important role across tasks. Similarly, there was widespread cerebellar activity in the component in the multivariate analysis, indicating cerebellar contributions to the functional network, but only small regions of cerebellum were present in the univariate analysis.

Table A.9.

Cluster volumes and peak MNI coordinates regions within Component 3 that showed significant differences between conditions, based on the full-brain univariate statistics.

			MNI Peak Coordinates
	Voxels	mm^3	x	y	z	Region	BA	Network
Encoding
Categorical Encoding > Item
	599	4792
			12	−64	40	Right Precuneus	7	FP-CEN
			−4	−64	44	Left Precuneus	7	DMN
	161	1288	−4	−28	28	Left Middle Cingulate/Medial Frontal Gyrus	23	SA
	65	520	−12	−80	−22	Left Cerebellar Lobule VI (Hem)	18	FP-CEN
	47	376	4	−88	−4	Left Calcarine Gyrus	17	VS
	16	128	−8	−88	−6	Left Calcarine Gyrus	18	VS
	14	112	−6	−8	8	Left Thalamus
Probe
Label > Item
	67	536	−4	−68	46	Left Precuneus	7	FP-CEN
	Category> Label
	40	320	2	26	46	Right Superior Medial Gyrus		SA
	33	264	−4	−34	28	Bilateral Posterior Cingulate	23	FP-CEN
	33	264	36	−62	−28	Right Cerebellar Lobule VIIa Crus I		FP-CEN
	30	240	2	−42	22	Bilateral Posterior Cingulate Cortex	26	DMN
	24	192	−8	−80	−30	Left Cerebellar Lobule VIIa Crus I		FP-CEN
Category > Item
	743	5944
			−6	−72	44	Left Precuneus	7	FP-CEN
			8	−70	44	Right Precuneus	7	FP-CEN
	556	4448	−34	−56	46	Left Inferior Parietal Lobule	7	FP-CEN
	168	1344	34	−60	46	Right Angular Gyrus	7	FP-CEN
	85	680	0	28	42	Bilateral Superior Medial Gyrus	32	SA
	68	544	−4	−38	24	Posterior Cingulate Cortex	26	FP-CEN
	14	112	−8	−84	−34	Left Cerebellar Lobule VIIa Crus II (Hem)		FP-CEN
	11	88	−50	22	34	Left Middle Frontal Gyrus	45	FP-CEN

VS=Visual; SM=Somatomotor; DA=Dorsal Attention; FP-CEN=Central Executive; SA=Salience; DMN=Default Mode Network. Cluster volumes smaller than 10 voxels have been omitted.

3.2.3.4 Components 4 & 5

As was the case for visual regions within Components 1 and 2, visual regions associated with Component 4 were sensitive to visual stimulus features, (Table A.10). Interestingly, there were differences in recruitment during encoding between the working memory and categorization conditions despite both conditions sharing the same stimulus types (faces). Activity within regions associated with Component 5 tended to be anticorrelated with task difficulty, mirroring results based on the predictor weight timecourse. Given the tangential nature of this component to our primary hypotheses, we do not provide a table of these results.

Table A.10.

Cluster volumes and peak MNI coordinates regions within Component 4 that showed significant differences between conditions, based on the full-brain univariate statistics.

			MNI Peak Coordinates
	Voxels	mm^3	x	y	z	Region	BA	Network
Encoding
Categorical Encoding > Item
	1040	8320
			−2	−90	−4	Left Calcarine Gyrus	17	VS
			2	−76	8	Right Calcarine Gyrus	18	VS
			16	−76	10	Right Calcarine Gyrus	17	VS
			−14	−62	2	Left Lingual Gyrus	17	VS
	20	160	−12	−74	−10	Left Lingual Gyrus	18	VS
Item > Categorical Encoding
	138	1104
			−42	−66	−12	Left Inferior Occipital Gyrus	19	VS
			−46	−74	−8	Left Inferior Occipital Gyrus	19	VS
Probe
Item>Label
	764	6112
			−26	−94	−4	Left Inferior Occipital Gyrus	18	VS
			−38	−84	−14	Left Fusiform Gyrus	19	VS
			−14	−96	−12	Left Lingual Gyrus	18	VS
	436	3488	42	−82	−2	Right Inferior Occipital Gyrus	19	VS
Label > Item
	56	448	−14	−88	24	Left Superior Occipital Gyrus	18	VS
	22	176	12	−86	16	Right Cuneus	18	VS
Category > Label
	861	6888
			−26	−94	−4	Left Inferior Occipital Gyrus	18	VS
			−24	−82	−16	Left Lingual Gyrus	18	VS
	591	4728
			42	−88	−2	Right Inferior Occipital Gyrus	19	VS
			22	−96	−4	Right Calcarine Gyrus	17	VS
Label > Category
	125	1000	−12	−90	26	Left Cuneus	18	VS
	165	1320	6	−86	18	Right Cuneus	18	VS
	82	656	10	−70	−6	Right Lingual Gyrus	18	VS
	59	472	−6	−70	−2	Left Lingual Gyrus	18	VS

VS=Visual; SM=Somatomotor; DA=Dorsal Attention; FP-CEN=Central Executive; SA=Salience; DMN=Default Mode Network. Cluster volumes smaller than 10 voxels have been omitted.

4 Discussion

We compared delayed match-to-sample and delayed match-to-category tasks to investigate how neural systems were recruited for categorization and item-specific processes across encoding, maintenance across a short delay, and match-mismatch decisions. In the match-to-sample task, optimal behavior could be subserved by a strategy wherein participants considered only the intrinsic visual features of the stimuli. The categorization task, however, required that participants make judgments based on latent categorical features (the category labels). We found that categorization and item specific memory recruited five neural networks (identified as CPCA components). Two of the components are of particular note: Component 1, which recruited key nodes of the salience network involved in immediate stimulus processing, and Component 3, which recruited fronto-parietal-striatal regions linked to executive function.

The first CPCA component had three important characteristics. First, it included regions associated with frontoparietal networks, especially the salience network (i.e., bilateral inferior frontal and anterior insula, and dorsal anterior cingulate/ SMA; Chiong et al., 2013; Ham, Leff, de Boissezon, Joffe, & Sharp, 2013; Menon & Uddin, 2010; Sridharan et al., 2008) along with visual processing regions. Second, it displayed hemodynamic response peaks occurring rapidly after stimulus onset. Third, activity in this component was significantly higher for both conditions that required face processing (Item and Category) than the Label conditions. These characteristics support the interpretation that this component was associated with the detection of behaviourally-salient events and the rapid allocation of cognitive resources to support task demands. The salience network has been previously associated with the coordination of large-scale brain networks to support advantageous behavioral responding (Eckert, Menon, & Walczak, 2009; Ham et al., 2013; Menon & Uddin, 2010; Sridharan et al., 2008). For instance, damage to the salience network has been linked to default mode network dysfunction (Bonnelle et al., 2012), and functional connectivity analyses have provided evidence that the salience network mediates the anti-correlated relationship between the frontoparietal network and the default mode network (Goulden et al., 2014; Menon, 2011; Palaniyappan et al., 2013; Sridharan et al., 2008).

Like Component 1, Component 3 was primarily associated with regions of the frontoparietal network (Buckner et al., 2011; Choi et al., 2012; Yeo et al., 2011), but unlike Component 1, these regions were primarily associated with the central executive network rather than the salience network (Goulden et al., 2014; Sridharan et al., 2008). The bimodal predictor-weight timecourse associated with Component 3 was similar to that of Component 1, but the peaks occurred later in time following each stimulus, consistent with a greater role in more time-demanding processes such as those involved in decision making, rather than rapid attentional orienting as in Component 1. Interestingly, univariate analyses indicated that subregions within this network tended to show greater activity when categorization was required than when it was not.

Many of the regions involved in Component 3 have been associated with categorization in previous studies. Notably Component 3 recruited frontal lobe regions known to play an important role in rule and category learning tasks in both monkeys (Antzoulatos & Miller, 2011; Freedman, Riesenhuber, Poggio, & Miller, 2001; Meyers et al., 2008; Wallis & Miller, 2003) and humans (Seger & Cincotta, 2006). Additionally, there was broad activity extending medially to laterally across the intraparietal sulcus region, which is thought to include the human homolog of LIP, shown to be category sensitive in monkey (Fitzgerald, Freedman, & Assad, 2011; Freedman & Assad, 2006, 2009). This region is also implicated in perceptual decision making more broadly, and is thought to subserve processes of accumulation of information from perceptual regions that can serve as input to regions involved in response selection (Ploran et al., 2007; Roitman & Shadlen, 2002; Shadlen & Newsome, 2001). Although univariate analyses indicated that subregions within this network were preferentially activated during conditions requiring categorization, the multivariate analyses indicated that this network was similarly recruited across tasks.

Component 3 was also the only component associated with widespread activity in the basal ganglia, specifically in the body of the caudate, a region associated with visual categorization in several previous studies (Lopez-Paniagua & Seger, 2011; Nomura et al., 2007; Seger & Cincotta, 2005, 2006; Seger, Peterson, Cincotta, Lopez-Paniagua, & Anderson, 2010). Univariate analyses, however, did not indicate that there were significant differences in basal ganglia activity between tasks; suggesting that this region may play a similar role across these tasks. An unexpected finding was that widespread regions of the cerebellum were also included in Component 3. Although these regions of the cerebellum are associated with frontal cognitive control system (Buckner et al., 2011), and are known to contribute to higher-order cognitive processes (Balsters, Whelan, Robertson, & Ramnani, 2012), the cerebellum is not commonly a focus of categorization and decision making research.

Component 5 closely resembled the DMN, and showed suppressed activity during the more cognitively demanding Category trials relative to the Item trials. This finding is in accordance with the known anti-correlated relationship between the default mode and frontoparietal networks (e.g., Menon & Uddin, 2010; Sridharan et al., 2008). Component 2 resembled the sensorimotor intrinsic connectivity network and showed a single peak corresponding to the behavioral response. However, this network additionally recruited regions within the salience and dorsal attention intrinsic connectivity networks. Many of these regions have been previously associated with abstract motor representation and motor preparation in functional tasks (Noppeney, Josephs, Kiebel, Friston, & Price, 2005; Rowe, Hughes, & Nimmo-Smith, 2010), and may therefore have activity patterns that correlated with the sensorimotor network during our task.

Component 4 was limited to regions within the visual intrinsic connectivity network, and as in resting state fcMRI, displayed a pattern of strong local connectivity (Yeo et al., 2011). Our multivariate analyses, however, suggested that these regions also interacted with different CPCA components. The bilateral lingual gyrus and cuneus, for instance, interacted with somatomotor regions in Component 2, while posterior occipital regions were associated with Component 4, and regions extending from the lateral occipital lobe down through the fusiform gyri interacted with Component 1. Univariate analyses provided evidence that visual regions were driven by stimulus type, but were insensitive to categorization demands; fusiform regions showed greater activity when a face was presented, while medial occipital regions showed greater activity when the category label (a single letter) was presented.

In this paper we report, for the first time, functional networks involved in the performance of a delayed matching task that were recruited during categorization and during the processing of specific items. Most importantly, we found two different frontoparietal networks, one of which (Component #1) acted on a faster time course, was sensitive to differences between conditions, and included regions of the salience network in conjunction with regions involved in higher level visual processing. The second, (Component #3) operated on a slower time course, and involved lateral parietal and lateral frontal regions, as well as the basal ganglia, all regions previously individually associated with categorization and decision-making.

Highlights.

• -Using fMRI, we compared delayed match-to-sample and delayed match-to-category tasks
• -We identified five intrinsic networks active during the tasks
• -The frontoparietal central-executive network played a key role in categorization
• -The frontoparietal salience network was sensitive to rapid visual feature processing in both tasks
• -Visual, motor, and default mode networks were activated similarly for both tasks