The Neural Substrates of Recognition Memory for Verbal Information: Spanning the Divide between Short- and Long-term Memory

Abstract

One of the classic categorical divisions in the history of memory research is that between short-term and long-term memory. Indeed, because memory for the immediate past (a few seconds) and memory for the relatively more remote past (several seconds and beyond) are assumed to rely on distinct neural systems, more often than not, memory research has focused either on short- (or “working memory”) or on long-term memory. Using an auditory–verbal continuous recognition paradigm designed for fMRI, we examined how the neural signatures of recognition memory change across an interval of time (from 2.5 to 30 sec) that spans this hypothetical division between short- and long-term memory. The results revealed that activity during successful auditory–verbal item recognition in inferior parietal cortex and the posterior superior temporal lobe was maximal for early lags, whereas, conversely, activity in the left inferior frontal gyrus increased as a function of lag. Taken together, the results reveal that as the interval between item repetitions increases, there is a shift in the distribution of memory-related activity that moves from posterior temporo-parietal cortex (lags 1–4) to inferior frontal regions (lags 5–10), indicating that as time advances, the burden of recognition memory is increasingly placed on top–down retrieval mechanisms that are mediated by structures in inferior frontal cortex.

INTRODUCTION

It is much easier to remember something that occurred a few seconds ago than it is to remember something that happened a minute ago. For instance, in tests of free recall for word lists, usually consisting of 15 or so common nouns, subjects are more likely to recall words presented at the end of the list than in the beginning or middle of the list (e.g., Glanzer & Cunitz, 1966; Waugh & Norman, 1965). It has also been shown that even with lists as short as three items (e.g., a “consonant trigram”), forgetting can occur very rapidly provided that an activity-filled delay, such as counting backwards by threes, is interposed between stimulus presentation and recall (Peterson & Peterson, 1959). One explanation for the superior recall of recent items is that a finite amount of information can be retrieved, with relative ease, from a STM store. As time elapses or new information enters the system, older information is no longer accessible in STM and must instead be retrieved from long-term memory (LTM) (Waugh & Norman, 1965).

The neuropsychological study of patients with memory disorders has generally supported the idea that there are dedicated and largely independent STM and LTM systems (Squire, 2009). For example, there are descriptions of patients with lesions to the medial temporal lobe (MTL) who have intact STM, as measured, for instance, by digit-span recall, but severely impaired long-term declarative memory (Corkin, 2002). There are other patients with left temporo-parietal lesions who perform normally on tests of LTM but have a digit span of only one or two items (e.g., Vallar & Baddeley, 1984; Warrington & Shallice, 1969). These neurological cases, which together constitute a double dissociation, offer extremely compelling evidence in favor of the existence of separate neural systems for STM and LTM. Indeed, the concept of separate memory components is so well established that, particularly in the fields of neuropsychology and cognitive neuroscience, STM and LTM are almost always studied separately (but see Cabeza, Dolcos, Graham, & Nyberg, 2002), and experimental tasks are deliberately constructed so as to tap into LTM or STM—but rarely both at the same time. The reasoning behind this is straightforward: Because STM and LTM depend on different brain systems, there is little to be learned about one system (e.g., LTM) by using a task that was designed to exercise the other (e.g., STM). In addition, cognitive neuroscience researchers generally take pains to ensure that their experimental task is a “pure” test of STM or a “pure” test LTM so that the elicited brain response can be confidently attributed to either one system or the other rather than some complex mixture of the two.

One reason that we might actually want to examine cognitive tasks that span or otherwise mix elements of STM and LTM is to define or delimit the boundaries and scope of these systems in a neurobiological context. If tasks designed only to probe STM or tasks designed only to examine LTM are always used, an unintended effect is to reify the very conceptual framework that guided the selection of the task in the first place. As recent work showing an unexpected role for the MTL in tests of STM illustrates (Olsen et al., 2009; Nichols, Kao, Verfaellie, & Gabrieli, 2006; Ranganath & D'Esposito, 2001; Stern, Sherman, Kirchhoff, & Hasselmo, 2001), however, the relationship between “memory systems,” brain areas, and psychological constructs such as STM and LTM is still not fully understood. Thus, despite the large body of evidence supporting both a functional and neural distinction between STM and LTM, it remains a challenge for cognitive neuroscience to objectively and precisely define them. A major aim of the present work is to take up this general challenge in the context of verbal memory, a domain in which most of the existing cognitive neuroscience research has investigated STM and LTM in isolation. Indeed, our own previous research on verbal working memory, which we now turn to, falls into this category.

In the past several years, we have investigated the neural basis of verbal STM (Buchsbaum & D'Esposito, 2009; Buchsbaum, Olsen, Koch, & Berman, 2005; Hickok, Buchsbaum, Humphries, & Muftuler, 2003; Buchsbaum, Hickok, & Humphries, 2001), focusing especially on how it is related to Baddeley and colleagues' classic model of verbal working memory, the phonological loop (Baddeley, 1992, 2003). In this model, verbal information is retained in memory through the coordinated activity of two cognitive components, an articulatory rehearsal process and a phonological store. Speech-based information can be held in the phonological store only for short periods because such information is subject to rapid decay (an item has a lifespan of approximately 2 sec). The mechanism of subvocal rehearsal is mediated by an articulatory control process that can reverse this decay process by “refreshing” memory traces in the phonological store. Thus, by “looping” through the contents of the store and subvocalizing each item in turn, a small amount (approximately as many items as can be spoken in 2 sec) (Schweickert & Boruff, 1986; Baddeley & Lewis, 1984) of verbal information can be retained in working memory for extended periods.

In a series of fMRI investigations of verbal working memory (for a review, see Buchsbaum & D'Esposito, 2008), we have shown that subvocal rehearsal is associated with a sustained delay period activity in posterior auditory cortex [area Spt in the left planum temporale, and the STS, bilaterally] and areas in frontal cortex associated with speech production and articulatory control, including the left posterior inferior frontal gyrus [IFG] (pars opercularis) and dorsal premotor cortex (DPMC). We have argued that the neural substrate of the phonological store, the component of Baddeley and colleagues' model of verbal working memory that is specialized for the brief retention of phonological information, is best thought of as emerging from the combined and coordinated activity of bilateral STS and Spt. For example, we showed (Buchsbaum et al., 2005) that STS activity associated with verbal memory maintenance was reliable only for the first few seconds of a 12-sec delay period. In contrast, area Spt showed strong activity that persisted across the retention period. Moreover, although delay period activity in the STS was larger for auditory–verbal than for visual–verbal stimuli, activity in Spt was not affected by stimulus input modality. We concluded that Spt supports a kind of memory code that is amenable to the kind of articulatory “refreshing” indicated in the phonological loop, whereas the mnemonic trace in the STS reflects a more transient, acoustic–phonetic (or “echoic”) code that is less easily reactivated by rehearsal mechanisms. Both structures are important for auditory–verbal storage in STM, however, the STS is critical for early maintenance of a fragile acoustic–phonetic trace that is converted to an articulatory code that is subsequently maintained by a network that includes Spt and the frontal speech system.

With regard to the apparently short-lived trace in STS, one might speculate that the decline in activation across the delay period in the STS reflected mnemonic trace decay, whereas the sustained activity in Spt reflected the top–down refreshing operation described in Baddeley and colleagues’ phonological loop model. Because performance in this study was at ceiling (due to low memory load—2 or 3 items), however, there is no objective way to link this suggestive physiological pattern of activation decline to the hypothetical process of “trace decay.” In point of fact, little is known about the neural basis of phonological trace decay, despite its theoretical importance in many (but not all; see Lewandowsky & Oberauer, 2009; Brown, Neath, & Chater, 2007) models of verbal STM. To examine the physiological basis of “decay,” however, one must employ a task that prohibits (or otherwise makes impracticable) the use of subvocal rehearsal— the purpose of which is precisely to counteract decay.

One task that is well suited for examining how memory performance changes as a function of time (and serial order) is the continuous recognition task. In this task (e.g., Hockley, 1982; Shepard & Teghtsoonian, 1961), subjects are confronted with a continuous stream of stimuli for which they must decide whether each item in the sequence is old (a repeated item) or new (a novel item). If variability in the lag between the first and second presentations of an item is sufficiently large (e.g., greater than a subject's memory span), then subvocal rehearsal is not an effective mnemonic aid. At any one time, because the likelihood that the currently presented item is within the subject's span is quite low, subvocal rehearsal has little obvious benefit. In addition, because the items arrive continuously, active rehearsal requires a difficult updating operation for each item. Moreover, if the interstimulus interval between successive items is short, then subjects are continually engaged in making recognition decisions and button responses; accordingly, little time or processing resources are available for a demanding form of subvocal rehearsal. If subjects do not engage in subvocal rehearsal, we can assume that items that enter the phonological store as the result of auditory encoding are left in a “pristine state”—that is, undisturbed by the trace-restoring effects of subvocal rehearsal.

The goal of the present study is to exploit this feature of the continuous recognition task to examine whether rapid time-based decay is evident in regions of the brain previously associated with phonological storage—as well as the more general question of whether the neural basis of STM and LTM can be delimited by task parameters such as time and serial order. If we begin with the premise that the dividing line between STM and LTM is ill-defined and possibly even nonexistent—and then assert that we have discovered a task that crosses that very line—our argument is admittedly a very circular one. To be clear, our strategy is to evaluate the plausibility of a particular estimate of STM as stipulated by the 2-sec temporal decay hypothesis of the phonological loop, as well as to sample across a range of lags wide enough to cover the most common and generally accepted estimates of STM capacity (e.g., Cowan, 2001; Oberauer, Demmrich, Mayr, & Kliegl, 2001) or duration (Mueller & Krawitz, 2009; Schweickert & Boruff, 1986), that is, covering from 2 to 7 sec or one to four items after initial encoding.

To that end, we scanned 16 subjects with fMRI while they performed an auditory–verbal continuous recognition task. We tested recognition memory for auditory–verbal items separated by “lags” ranging from 1 to 15 items (1, 2, 3, 4, 5, 6, 8, 10, 15). Although several previous functional neuroimaging studies have examined lag effects using the continuous recognition paradigm (e.g., Johnson, Muftuler, & Rugg, 2008; Brozinsky, Yonelinas, Kroll, & Ranganath, 2005), only one fMRI study that we are aware of has tested lags that span STM and LTM. Thus, Brozinsky et al. (2005) examined activity in the MTL for lags of 2, 8, 16, and 32 intervening items. Repetition suppression effects were observed in the parahippocampal gyrus and posterior hippocampus primarily for lag 2. That study, however, focused exclusively on the MTL and used a coarse sampling of lags in the critical transition zone between STM and LTM. Buchsbaum and D'Esposito (2009) also used a verbal continuous recognition paradigm with auditory- and visual– verbal stimuli that only employed short lags (1–5) and, therefore, may only have sampled lags that are arguably within the purview of STM.

If we assume that when the phonological trace of an item in the store fully decays, it can no longer be accessed—and thereby refreshed—either by way of self-initiated retrieval processes or by an external input stimulus, then the absence of a detectable physiological response to a repeated item might be taken as evidence of total trace decay. More generally, the existence of a parametric relationship between lag and activation magnitude might offer a description of the time course of mnemonic trace decay. As we and others have previously shown, subvocal rehearsal is associated with elevated activity in the STS and Spt, and perceptually driven trace reactivation is moreover similar to internal mnemonic refreshing (Postle, 2006; Buchsbaum et al., 2005; Pasternak & Greenlee, 2005; Slotnick, 2004; Wheeler, Petersen, & Buckner, 2000). Thus, we hypothesize that stimulus repetition occurring at short intervals (e.g., before total trace decay) should be associated with heightened activation in the posterior auditory cortical storage areas relative to baseline. An alternative hypothesis is that the physiological correlate of trace reactivation is reduced neural activity or repetition suppression (e.g., Grill-Spector, Henson, & Martin, 2006). Previous studies of auditory repetition, however, have typically observed such suppression effects only in the (relatively) anterior part of auditory cortex (Buchsbaum & D'Esposito, 2009; Dehaene-Lambertz et al., 2006; Cohen, Jobert, Le Bihan, & Dehaene, 2004), a region that has not typically been associated with phonological working memory.

In addition to the particular questions relating to phonological trace decay in verbal STM, we will address in a more exploratory fashion the general issue of delimiting the physiological signatures of STM and LTM that motivated the present study. For instance, we will examine the extent to which there exists any evidence for a sharp categorical shift in brain activation patterns as lag advances through the “gray area” that separates STM and LTM; or alternatively, whether brain activation changes more gradually as a function of lag. In addition, we will examine how individual differences in task performance relate to brain activation to assess the extent to which lag-sensitive areas also predict memory ability.

METHODS

Subjects

Sixteen healthy subjects (7 women, 9 men; 21–33 years old, mean age = 25.03 years), all native English speakers, participated in the study after giving informed written consent. The National Institute of Mental Health Institutional Review Board approved the experimental procedures. No subjects had any past history of psychiatric or neurological diseases. All subjects, as assessed by the Edinburgh Handedness Inventory, were determined to be strongly right-handed.

Task

The task was an auditory–verbal continuous recognition paradigm (Shepard & Teghtsoonian, 1961) with two- and three-syllable nouns. Words were presented through headphones in the auditory modality at a rate of one word every 2.5 sec. Subjects were instructed to judge whether each word in the continuous sequence was old (previously encountered word) or new (novel word), and to press the left button for the former and the right button for the latter. The experimental manipulation was the lag between an old item and its previous presentation (measured as the difference between the serial positions of the two items). No item was repeated more than once. “Filler” items were included to fill in the gaps in the randomly generated sequences. Such items are a subset of new items and are indistinguishable from other new items, except that they are presented once and never repeated. Old items were distributed equally across the following repetition lags: 1 (immediate repetition), 2, 3, 4, 5, 6, 8, 10, or 15. Each of eight scanning runs consisted of five old items per lag (45 total old events) and 75 new events (50 first-presentation words and 25 “filler” words), for a total of 125 word stimuli per run. Thus, the proportion of old items is 0.36 and the proportion of items corresponding to a repeat of a particular lag was 0.04. The ordering of the lag conditions was arranged in such a way that the differences are minimized in the (within-run) serial position across lags. This is necessary because whereas a lag 1 repeat can occur as early as the second absolute (within-run) serial position, a lag 15 cannot occur until the 16th serial position. Serial position across lag was balanced by randomly generating a set of 100 orderings, computing the average serial position across lag, and then taking the standard deviation of this average; the randomization with the smallest standard deviation was selected for each run.

Auditory Stimuli

A total of 600 two- and three-syllable nouns were generated with a text-to-speech synthesizer using the Nuance Speechify (http://www.nuance.com/) software with a female voice. The words were selected from the MRC psycholinguistic database (Coltheart, 1981) so as to exclude words with very high (>600) imageability ratings. Other relevant indices for the word set are as follows: average Kucera–Francis written frequency, mean = 43.8, SD = 64.6; number of syllables, mean = 2.46, SD = .5; number of letters, mean = 7.1, SD = 1.59; imageability index, mean = 474.9, SD = 97.9. There were no statistically significant differences in these word indices across experimental conditions.

MRI Data Acquisition

Functional and structural images were acquired with a 3.0-Tesla GE Signa scanner (Milwaukee, WI) using a GE birdcage head coil. Each subject performed eight scanning runs, each of which lasted 340 sec. Functional images were collected with a gradient-echo echo-planar imaging (EPI) sequence (TR = 2 sec; TE = 25 msec; FOV = 24 cm; flip angle = 90°; 128 × 128 matrix). Image volumes were acquired in 24 axial slices (thickness = 5 mm; in-plane resolution = 1.88 × 1.88 mm). In addition, high-resolution MP-RAGE structural images were acquired in 124 axial slices (thickness = 1.2 mm, in-plane resolution = 0.975 × 0.975 mm). The experimental paradigm was programmed using Presentation software version 5.5 (Neurobehavioral Systems, Albany, CA) and ran on a Dell laptop. Auditory stimuli were delivered via air conductance tubes connected to magnet-safe headphones (Avotec model SS-3100) placed around the subject's ears.

Image Preprocessing

The images of every scanning run were concatenated to form, for each subject, a set of eight 4-dimensional data “bricks.” Slice-timing adjustment and image realignment were carried out with the AFNI (http://afni.nimh.nih.gov/) program 3dVolreg. The mean volume for the session served as the registration reference image. The time series were normalized by the image mean and multiplied by 100. All image volumes were then smoothed with a 6-mm (FWHM) Gaussian kernel. Each subject's mean EPI image was aligned to that subject's high-resolution structural MRI with a rigid-body alignment procedure using FLIRT (Jenkinson & Smith, 2001). Highresolution structural MRIs were transformed to MNI space using a 12-parameter affine warp (Jenkinson & Smith, 2001). These two transforms were concatenated and used to transform each subject's native image space to the standard template for use in multisubject statistical analyses.

Statistical Analysis

A single-subject multiple regression analysis was carried out with the AFNI program 3dDeconvolve. Each event type was modeled as a convolution of a temporal onset vector with a gamma probability density function. Thus, separate regressors were created for new items (including both first presentation and “filler” words), and one for each of the nine lag conditions. All incorrect trials were modeled with a separate “error” regressor. A set of four orthogonal polynomials plus a constant term were also included (separately for each run) to model the shifts in the global mean as well as within-run low-frequency trends. Contrasts were estimated for each of the lag conditions versus novel items (e.g., lag 1 > novel, lag 2 > novel, … lag n > novel), and the main effect of old items was assessed by comparing the average repetition effect (e.g., the main effect of old items, collapsed across lag) against the novel “baseline.” Statistical analyses at the group level were carried out on the spatially normalized single-subject t-statistic maps using, where appropriate, either a one-sample t test (using AFNI program 3dttest) or, for the assessment of lag effects, a one-way repeated measures ANOVA (using AFNI program 3dANOVA). Additional linear trend analyses (using weighted contrast vectors) were conducted to assess the direction of the lag effects (e.g., increasing or decreasing with lag). Lastly, voxelwise correlations were performed to assess the degree of association between subject performance and the degree of activity during old trials.

RESULTS

Behavioral Data

A one-way repeated measures ANOVA, with factor lag as the independent variable and RT as the dependent measure, was statistically significant [F(8, 112) = 36.352, p < .0001]. The mean RT for new items was 539.6 msec greater than the mean RT for old items averaged across lag [F(1, 14) = 5.659, p < .033]; however, contrasts showed that the mean RT for lags 6, 8, 10, and 15 was not significantly different from the mean RT for new items (see Figure 1). The mean accuracy (proportion correct) for novel items was 0.914 and that for repeated items was 0.7822. A one-way repeated measures ANOVA for accuracy with factor lag as an independent variable was also significant [F(8, 112) = 20.617, p < .0001]. For several subjects, performance on lag 15 was below chance, probably owing to the relatively small percentage of old items (36%) and the five-item jump in lag from 10 to 15 (an increase of 50%). This condition was therefore eliminated from fMRI analyses due its status as an outlier condition in several subjects.

Figure 1.

Top: Accuracy data for item recognition scored as proportion correct (y-axis) and plotted as a function of lag (x-axis). Error bars represent ±1 standard error. Bottom: Mean RT (msec) plotted as a function of lag. Error bars represent ±1 standard error.

fMRI Analyses

Two main analyses were carried out on the set of spatially normalized single-subject contrasts. First, the main effect of repetition (old items − new items, excluding lag 15) was assessed at the group level with a one-sample t test. Figure 2 shows the effect of repetition (old > new; new > old) displayed on the cortical surface and in three axial slices. Repetition-related increases in activity were observed in posterior parietal cortex along the intraparietal sulcus, left anterior prefrontal cortex, the anterior insular bilaterally, and the dorsal precentral sulcus. Reductions in activity associated with stimulus repetition were observed in the parahippocampal gyrus and ventral occipital cortex bilaterally, medial fronto-polar cortex, and the anterior superior temporal sulcus (see Table 1 for the full set of activated regions). An additional repeated measures ANOVA with subject as a random factor and lag as the independent variable was carried out. The inputs to this analysis were the single-subject t statistics for each of the (lag_{(1,2,3,4,5,6,8,10)} − novel) contrasts.

Figure 2.

Top row: Surface rendering of group contrast old versus new, threshold at p < .001, two-tailed. Areas where old (hits) > new are shown in green colors. Areas where new (correct rejections) > old are shown in yellow colors. Bottom row: Three axial slices show old versus new group contrast with same color scheme as above.

Table 1.

Peak Voxel Coordinates, Anatomical Locations, and Brodmann’s Areas for Old/New Contrast

x	y	z	Size	t Statistic	p	Brodmann’s Area/Brain Region
Old > New
−38	−56	38	1399	13.7	<.0001	BA 40/L. inferior parietal lobule
47	−44	53	453	6.31	<.0001	BA 40/R. inferior parietal lobule
−8	11	1	347	11.4	<.0001	L. caudate
8	11	1	233	12.1	<.0001	R. caudate
−38	−4	53	188	6.64	<.0001	BA 6/L. precentral sulcus
−11	−86	−35	180	7.28	<.0001	L. cerebellum
−38	53	4	165	6.23	<.0001	L. middle frontal gyrus
35	23	−11	51	5.51	<.0001	R. inferior frontal gyrus/OFC
−5	32	38	28	4.6	.0004	BA 9/L. medial superior frontal gyrus
62	−41	−17	23	4.26	.0008	BA 21/R. inferior temporal gyrus
−41	29	23	19	4.35	.0006	L. inferior frontal gyrus
44	38	20	19	5.06	.0002	BA 46/R. middle frontal gyrus
8	−23	−26	10	4.65	.0004	Pons
−44	−65	−47	7	5.01	.0002	L. cerebellum
−32	−53	−53	7	4.78	.0003	L. cerebellum
−23	−65	−59	7	4.01	.001	R. cerebellum
New > Old
38	−38	−20	3535	11.4	<.0001	BA 20/R. fusiform gyrus
−17	32	50	900	6.55	<.0001	L. superior frontal gyrus
−53	−71	16	64	5.57	<.0001	BA 39/L. middle temporal gyrus
−56	5	−32	50	4.83	.00027	BA 21/L. middle temporal gyrus
−59	−10	−17	35	4.96	.00021	L. middle temporal gyrus
29	38	−17	27	4.76	.0003	R. middle frontal gyrus/OFC
26	41	35	25	4.92	.00023	BA 9/R. superior frontal gyrus
−35	35	−17	25	4.15	.0009	L. inferior frontal gyrus/OFC
38	5	−44	11	5.17	.00014	R. inferior temporal gyrus
−35	−4	−50	11	5.53	<.0001	L. inferior temporal gyrus
−29	−20	−26	8	4.1	.001	L. parahippocampal

The omnibus voxelwise F-test for the main effect of lag, shown in Figure 3, reveals a left inferior frontal, inferior parietal (bilaterally), as well as middle and superior temporal distribution (bilaterally) of lag-sensitive activity. The mean t statistics for each of these four clusters in the left hemisphere, plotted as a function of lag (relative to a baseline representing the average effect for new items), is shown in Figure 4. Strong lag effects were seen in the lateral and inferior portion of parietal cortex, the distribution of which overlapped slightly with the parietal region showing a generic old > new effect (see Figure 5). The full set of regions showing a main effect of lag is presented in Table 2.

Figure 3.

Top row: Surface rendering of omnibus F test for main effect of lag thresholded at p < .001. Bottom row: Set of four axial slices showing lag-sensitive activation.

Figure 4.

Left hemisphere surface rendering showing cluster maxima for the main effect of lag. Adjacent to each cluster is a plot of mean activity as a function of lag. 1 = superior temporal gyrus; 2 = superior temporal gyrus/superior temporal sulcus; 3 = left lateral parietal lobe; 4 = inferior frontal gyrus; 5 = ventral anterior prefrontal cortex.

Figure 5.

Posterior view of the parietal lobe showing overlap of old > new and lag effects in lateral parietal cortex and intraparietal sulcus, respectively. Lag effects (red color) are lateral and inferior, old/new effects (green colors) are more superior, and regions with joint effects are located in between (orange colors).

Table 2.

Peak Voxel Coordinates, Anatomical Locations, and Brodmann’s Areas for Main Effect of Lag

x	y	z	Size	F(8, 112)	p	Brodmann’s Area/Brain Region
62	−47	−2	1139	14.1	<.0001	BA 21/R. middle temporal gyrus
−53	−50	41	355	9.11	<.0001	L. inferior parietal lobule
−53	−32	−5	147	6.34	<.0001	L. middle temporal gyrus
−44	26	13	146	8	<.0001	L. inferior frontal gyrus
−47	−56	−17	52	8.27	<.0001	BA 37/L. fusiform gyrus
−8	20	50	48	6.61	<.0001	L. supplementary motor
−41	−56	−35	24	5.35	<.0001	L. cerebellum
−32	38	−17	22	5.46	<.0001	BA 11/L. inferior frontal gyrus/OFC
−35	41	35	18	5.12	<.0001	L. middle frontal gyrus
35	14	56	13	4.52	<.0001	BA 8/R. middle frontal gyrus
23	53	32	9	4.5	<.0001	R. superior frontal gyrus
−5	−26	35	7	4.23	.0002	L. cingulate gyrus
71	−16	1	6	4.01	.0003	R. superior temporal gyrus
−56	−7	50	5	4.28	.0002	BA 6/precentral sulcus
−32	26	−2	5	4.28	.0002	L. insula
11	−80	−29	5	4.38	.0001	R. cerebellum

In the IFG, activation increased with lag (see Figure 4) and the largest increases occurred for lags 1–3 before reaching a rough asymptote at lag 4. Conversely, the lateral/inferior parietal cluster showed strong activity at lags 1 and 2 before leveling off at lag 3 (though remaining above baseline for all lags). In the lateral superior and middle temporal region, we saw two distinct patterns of activity, as indicated by an examination of linear trend contrasts. In the left STS/middle temporal gyrus (MTG), activation was equally high at lags of 1 and 2, and then dropped off dramatically at lag 3, where it remained relatively constant across lags 3–10. On the other hand, the cluster in the superior temporal gyrus (STG, just anterior and superior to the STS/MTG cluster) initially showed a repetition suppression effect that dissipated (ultimately rising above baseline at lag 10) with increasing lag, paralleling the effect in the IFG. Note that the main effect of lag in these two regions formed a single cluster but have been split according to the direction of the linear trend (positive-going in the STG, negative-going in the STS/MTG) and plotted in two colors in Figure 4.

We next examined the extent to which individual differences in accuracy (percent correct) correlated with the measured activation in the old > new contrast. As might be expected, although mean accuracy systematically decreased as a function of lag, correlations between accuracy scores for different lag conditions were highly correlated across subjects. For instance, the correlation between accuracy scores for lag 3 and lag 10 was r = .906, p < .0001. Thus, a subject's performance at lag 3 was highly predictive of his or her performance at lag 10, and vice versa. On the other hand, performance at lag 1 was only weakly correlated with performance at lag 10 (r = 0.29, p = .29). This is, in part, attributable to the five-fold difference in variance between the percent correct scores at lag 1 and lag 3, respectively (variance lag 1 = 0.0029, variance lag 2 = 0.008, variance lag 3 = 0.016). Because the variance in accuracy at lags 1 and 2 was relatively low (owing to ceiling effects), and the correlations in accuracy across lags greater than 3 were quite high, a single behavioral index of performance, computed as the mean accuracy across lags 3–10, was computed. This index was then correlated with the old > new contrast, which was recalculated so as to exclude lags 1 and 2. This index of performance was only weakly correlated with mean RT for the same lags, r = −.44, p = .1. Areas significantly correlated with accuracy are reported in Table 3. Strong negative correlations were seen in left hemisphere regions including the posterior STS, IFG, ventral temporal cortex, and anterior hippocampus.

Table 3.

Peak Voxel Coordinates, Anatomical Locations, and Brodmann’s Areas for Behavioral Correlation Analysis

x	y	z	Size	Correlation	p	Brodmann’s Area/Brain Region
Negative Correlations
−47	20	7	25	−.859	<.0001	L. inferior frontal gyrus
−56	−41	1	24	−.802	<.0001	BA 22/L. middle temporal gyrus
−44	−50	−20	21	−.805	<.0001	L. fusiform gyrus
−23	−7	−26	17	−.756	<.001	L. hippocampus/amygdala
−20	5	1	8	−.761	<.001	L. putamen
29	−53	−35	6	−.789	<.001	R. cerebellum
Positive Correlations
26	59	16	43	.861	<.0001	BA 10/R. superior frontal gyrus
62	−53	32	22	.801	<.001	BA 40/R. supramarginal gyrus
−5	−23	35	8	.773	<.001	L. cingulate gyrus

Two of these regions, the posterior STS/MTG and the IFG, overlapped with areas also showing lag effects (see conjunction analysis in Figure 6), although the performance-related area in the IFG was shifted somewhat more inferior and anterior to the lag-sensitive cluster. The pattern of this effect across lag can be seen for this IFG region in Figure 6, where groups (good and poor performers) have been formed by a median split on the accuracy variable, and then plotted for each level of lag. It is clear from this figure that better accuracy on the recognition memory task is associated with less activity in the IFG across each level of lag. Although this disparity appears to be most pronounced at a lag of one, the Lag × Group (good or poor accuracy) interaction was not significant. Lastly, positive correlations between the old > new contrast and performance were also observed but were most prominent in right anterior prefrontal cortex (see Table 3 and Figure 7).

Figure 6.

Left: Plot of lag effect in the LIFG after dividing groups into good and poor performers. The negative correlation between performance and size of old > new effect is evident in the large separation between the two groups. Right: Left hemisphere surface rendering of conjunction analysis showing overlap (purple colors) of regions with a main effect of lag (red colors) and an inverse correlation (blue colors) between old > new contrast and subject accuracy.

Figure 7.

Left: Top shows clusters in the anterior hippocampus/amygdala and ventral temporal lobe showing negative correlation between performance and old > new effect. Bottom shows region in anterior PFC showing positive correlation with old > new effect. Right: Scatterplots of mean accuracy (across lags 3–10) versus old > new contrast in the anterior hippocampus/amygdala (bottom) and anterior PFC (top). Note that plots are shown for the purposes of quality assurance and may be biased due to post hoc ROI selection.

DISCUSSION

We have examined the relation between regional patterns in brain activity as a function of repetition lag in a continuous auditory–verbal recognition memory paradigm. The manipulation of repetition lag in the current study was chosen so as to cross the threshold that is commonly held to exist between STM and LTM in humans. Although the estimates with respect to the span or capacity of STM are variable and often paradigm-dependent, for the present study, we drew on the principle that the probability of an item being retrieved from STM is a monotonically decreasing function of lag. As is evident from the behavioral measures in this and previous studies, as repetition lag increases, accuracy declines and RT increases. For both measures, the function was steepest across the early and middle lags, and then leveled off for longer lags (e.g., lags 8, 10, and 15). This pattern was mirrored quite closely by the pattern of activation observed in left inferior prefrontal cortex, where the mean BOLD signal rapidly increases for the earliest lags before reaching an asymptote at about lag 6. In lateral parietal cortex, however, a different pattern was observed in which activation was greatest for the first two or three lags before rapidly reaching baseline levels after lag 4. In the middle-to-posterior STS, bilaterally, we observed strong activity for lags 1 and 2 followed by a precipitous decline thereafter 2 (Figure 7, area #2). Just anterior to this region on the bank of the STG, lateral and anterior to primary auditory cortex, activation was lowest for early lags and then gradually increased above baseline levels after lag 6. Finally, further anteriorly in the left STS, we also observed a repetition suppression effect (new > old) that, however, was not reliably modulated by lag.

Phonological Decay in Short-term Memory

We set out in search of physiological evidence in support of the claim implicit in the phonological loop model that verbal items in the phonological store undergo rapid time-based decay. Previous functional neuroimaging work (Buchsbaum et al., 2005; Hickok et al., 2003; Postle, Berger, & D'Esposito, 1999; Salmon et al., 1996; Paulesu, Frith, & Frackowiak, 1993) has identified regions in the posterior temporo-parietal zone that appear to support phonological storage in working memory. Thus, we reasoned that if trace decay is occurring, then physiological evidence for it should be sought in these regions. In addition, we assumed that time-based trace decay might be detected as a change in brain activity as a function of the lag between the first and second presentation of an auditory–verbal item. Finally, if phonological trace decay occurs as rapidly as is assumed by the phonological loop model (approximately 2 sec per item), then brain activity plotted as a function of lag should decline steeply after lag 1. In fact, this is precisely the pattern we observed in the left mid-to-posterior STS, where above baseline activity for lags 1 and 2 was followed by a cliff-like drop at lag 3, and leveling off thereafter. It was also in this area of the STS wherein Buchsbaum (2005) noted a decline in activity across the delay period in a task that actively encouraged verbal rehearsal. Thus, it may be that activity in this area, as we have previously suggested, is not much affected by rehearsal, and the present findings are consistent with that viewpoint.

In contrast to the lag effects we observed in the STS, the other area often associated with phonological working memory, area Spt in the posterior planum temporale, was not reliably modulated by lag. Buchsbaum and D'Esposito (2008), Hickok, Okada, and Serences (2009), and Jacquemot and Scott (2006) have previously suggested that the role of Spt in phonological STM is to interface between auditory and motor representations of speech in the context of speech production. This is supported by evidence from patients with conduction aphasia, a disorder that is often associated with lesions to the left temporo-parietal area that overlaps Spt (Buchsbaum & D'Esposito, 2009) and is characterized by an impairment in speech production and verbal repetition coupled with preserved auditory perception. The present work offers further evidence that the role of Spt in phonological memory is related to subvocal rehearsal rather than the sort of passive auditory–verbal storage required by the continuous recognition test. Taken together, the pattern of effects observed in auditory cortex reinforces the view (Buchsbaum & D'Esposito, 2008; Chein & Fiez, 2001; Becker, MacAndrew, & Fiez, 1999) that a one-to-one mapping between the concept of the phonological store and a particular brain region that embodies all its functional properties is unlikely to be found.

Memory Retrieval, Recency, and the Parietal Lobe

Although our primary goal was to examine lag effects in auditory cortical regions associated with phonological storage, a subsidiary aim was to track lag-related changes in other regions of the brain that have been associated with memory processing. The largest magnitude lag effects in the entire brain were observed bilaterally in the inferior parietal lobe and angular gyrus. This region showed a steep decline from lag 1 to lag 3 before leveling off at baseline levels thereafter. In functional neuroimaging studies of LTM, activity in this region has been consistently associated with recollective memory processes as indexed by source memory, remember/know, and related paradigms (Montaldi, Spencer, Roberts, & Mayes, 2006; Kahn, Davachi, & Wagner, 2004; Wheeler & Buckner, 2004; Dobbins, Rice, Wagner, & Schacter, 2003; Henson, Rugg, Shallice, Josephs, & Dolan, 1999). It has also been shown that activity in the inferior parietal area is positively associated with high confidence recognition (Kim & Cabeza, 2009) as well as the “amount” of information retrieved from LTM (Vilberg & Rugg, 2009a, 2009b). In light of these findings from neuroimaging studies of LTM, it seems plausible to attribute the lag effects in the inferior parietal lobe to a similar cause, such as high confidence or the quantity of retrieved information. Thus, although we did not require subjects to make confidence or remember/know judgments, other studies have shown that such measures decrease as a function of lag (Rubin, Hinton, & Wenzel, 1999). It seems likely, then, that the lag effects observed in parietal cortex in the present study are due to the same underlying neural mechanisms that give rise to confidence and recollection effects in studies of LTM. This is an example of a correspondence between STM and LTM that has probably gone unnoticed simply because researchers studying LTM avoid tasks that encroach upon STM.

Retrieval Demand, Semantics, and the Inferior Frontal Gyrus

As time and a succession of intervening items fill the interval between the first and the second occurrence of an item–probe pair, the task for the subject becomes increasingly difficult. Recent work has shown, in the context of LTM (Badre, Poldrack, Pare-Blagoev, Insler, & Wagner, 2005; Wagner, Pare-Blagoev, Clark, & Poldrack, 2001; Poldrack et al., 1999; Thompson-Schill, D'Esposito, Aguirre, & Farah, 1997), that the IFG is consistently associated with processes underlying controlled memory retrieval. In the present study, we found that the pars triangularis region of the IFG shows increasing activation as a function of repetition lag. Moreover, the degree of activation in this area is negatively correlated with individual differences in performance as indexed by mean accuracy across lags 3–10. One implication of this finding is that better-performing subjects have less need for recourse to controlled memory retrieval processes instantiated in lateral prefrontal cortex. Several previous studies of LTM have shown that the more anterior portion of the left IFG (LIFG) is activated more for semantic than for phonological retrieval (Binder, Desai, Graves, & Conant, 2009; Chee, Hon, Caplan, Lee, & Goh, 2002; Gold & Buckner, 2002; Poldrack et al., 1999). A classic finding from cognitive psychology is that, whereas acoustic or phonological factors tend to influence performance in tests of STM, semantic codes seem to play a more important role in LTM (Baddeley, 1966). It may be that as lag increases and sensory-based item codes decay, subjects shift to a strategy that relies more on the semantic retrieval function of the LIFG. Thus, increased activity in the LIFG does not necessarily reflect retrieval demand per se, but rather reflects a strategic shift to a more semantically driven search of memory.

Correlations between Memory Performance and Brain Activity

To assess the extent to which regions showing changes in brain activity as a function of lag might also be associated with interindividual variation in memory performance, we performed an exploratory voxel-by-voxel correlation analysis. Thus, we correlated each subject's mean accuracy for lags 3–10 with average activity across the corresponding single-subject contrast maps. Because all subjects performed well on lags 1 and 2, variance in performance as indexed by accuracy primarily reflects the memory for items presented at relatively longer lags. Negative correlations with recognition accuracy and mean BOLD activity (old > new contrast) were observed in a number of areas including the posterior STS/MTG and the anterior MTL (anterior hippocampus extending into amygdala and perirhinal cortex). That decreased activity in the posterior STS/MTG was associated with better performance on the recognition memory task is consistent with the idea that repetition-related reductions in neural activity in perceptual regions may reflect an increased efficiency of processing that occurs as the result of a sharpening or tuning of neural populations that code for a particular stimulus (Grill-Spector et al., 2006). A related interpretation, which is especially relevant to the negative correlations between brain activity and accuracy observed in the anterior MTL, is that repetition-related reductions in neural activity do not merely indicate a gain in processing efficiency but also constitute a “familiarity signal” that indicates the degree to which a repeated item matches previously encountered perceptual stimuli (Gonsalves, Kahn, Curran, Norman, & Wagner, 2005; Xiang & Brown, 1998). Notably, in our study, the cluster of activity in the anterior MTL includes part of the perirhinal cortex, which is thought to be the most important region for neural familiarity signaling, and has previously been identified as such in functional neuroimaging studies of recognition memory (Gonsalves et al., 2005; Henson, Cansino, Herron, Robb, & Rugg, 2003).

Negative correlations with performance were also observed in the STS/MTG and LIFG, a pair of brain regions that commonly coactivate in studies of verbal retrieval and lexical access (Badre & Wagner, 2007; Gold et al., 2006; Indefrey & Levelt, 2004; Gold & Buckner, 2002), and are thought to form the basis of a fronto-temporal system critical for word production. It should be noted, however, that although the LIFG showed both a main effect of lag across the group and a negative correlation with performance, the STS/MTG only showed the latter effect. In addition, the direction of the effect indicates that more poorly performing subjects activated these areas more than better-performing subjects, a finding that implies a behavioral cost to a more semantic mode of retrieval. This would seem paradoxical in light of evidence showing that semantic processing is generally beneficial to verbal memory (Craik, 2002) even at short delays (Hulme et al., 1997). One possibility is that better-performing subjects placed more emphasis on semantic processing at encoding, a strategy that lessened the burden on semantic search processes at the time of retrieval, thereby leading to more efficient neural processing in the LIFG and the STS/MTG for old items. If this is the case, it raises the possibility that the lag effects observed in the LIFG and other areas may be driven, in part, by variation in encoding processes even though lag is, at least conceptually, a retrieval manipulation. Thus, it may be that encoding processes are deployed only to the extent that retrieval processes fail. On short lag trials, where retrieval is generally successful, encoding processes are truncated. For longer lags, however, encoding processes carry on in parallel with retrieval processes, and the more difficult an item is to retrieve, the more processing resources are devoted to encoding it. To properly tease apart contributions from encoding and retrieval processes, however, requires a paradigm that orthogonally manipulates factors relating to encoding and retrieval, respectively.

Alternatively, the elevated activation in these areas in more poorly performing subjects reflects a general increase in retrieval effort—semantic or otherwise—that is a consequence of a less efficient overall memory system. This latter view is consistent with the finding that older adults with poorer memory often “overactivate” in prefrontal cortex during verbal retrieval (Velanova, Lustig, Jacoby, & Buckner, 2007).

Finally, a strong positive correlation was observed in right anterior prefrontal cortex in a region that has often been associated with episodic retrieval success (e.g., Donaldson, Petersen, & Buckner, 2001; Duzel et al., 1999; Buckner et al., 1998). Burgess, Dumontheil, and Gilbert (2007) have argued that one of the functions of the lateral part of anterior prefrontal cortex is to discriminate between perceived and imagined events. Thus, in a report by Simons, Davis, Gilbert, Frith, and Burgess (2006), subjects with greater activity in anterior prefrontal cortex were better able to distinguish between events that had been perceived compared with those that had only been imagined. Although our task did not explicitly involve imagined episodes, it is, nevertheless, an implicit requirement of successful recognition memory that one be able to consistently distinguish between memories for events that either did (a repeated item) or did not (a novel item) previously occur. Irrespective of the precise role of anterior prefrontal cortex in memory retrieval, the present results are consistent with previous studies that employed LTM paradigms. Once again, it seems to be the case that the type of processing involved, rather than the type of task employed by the experimenters, determines the elicited neural pattern.

Conclusion

In summary, we found that by smoothly varying the lag between encoding and retrieval in a continuous recognition memory paradigm, we were able to track the waxing and waning of neural activity that occurs as one crosses the hypothetical divide between STM and LTM. There are several regions, including the left STS and the inferior parietal lobe, which show a severe decline in activity as a function of lag—a profile that is consistent with the idea of “trace decay” in STM. Other areas, notably in lateral prefrontal cortex, showed an opposite effect: rapidly increasing activity across the first few lags followed thereafter by a gradual leveling. Insofar as the brain activity in both inferior prefrontal and temporo-parietal areas changes most rapidly across the first three or four most recent items, the evidence supports a qualitative—but not starkly categorical—difference in the neural computations that support recognition judgments for very recent and relatively more distant items. Further work, with more fine-grained cognitive manipulations, will be needed to identify the critical experimental factors and the underlying neural principles that produce these lag-related changes in brain activity.