Processing Stages Underlying Word Recognition in the Anteroventral Temporal Lobe

Abstract

The anteroventral temporal lobe integrates visual, lexical, semantic and mnestic aspects of word-processing, through its reciprocal connections with the ventral visual stream, language areas, and the hippocampal formation. We used linear microelectrode arrays to probe population synaptic currents and neuronal firing in different cortical layers of the anteroventral temporal lobe, during semantic judgments with implicit priming, and overt word recognition. Since different extrinsic and associative inputs preferentially target different cortical layers, this method can help reveal the sequence and nature of local processing stages at a higher resolution than was previously possible.

The initial response in inferotemporal and perirhinal cortices is a brief current sink beginning at ~120ms, and peaking at ~170ms. Localization of this initial sink to middle layers suggests that it represents feedforward input from lower visual areas, and simultaneously increased firing implies that it represents excitatory synaptic currents. Until ~800ms, the main focus of transmembrane current sinks alternates between middle and superficial layers, with the superficial focus becoming increasingly dominant after ~550ms. Since superficial layers are the target of local and feedback associative inputs, this suggests an alternation in predominant synaptic input between feedforward and feedback modes. Word repetition does not affect the initial perirhinal and inferotemporal middle layer sink, but does decrease later activity. Entorhinal activity begins later (~200ms), with greater apparent excitatory postsynaptic currents and multiunit activity in neocortically-projecting than hippocampal-projecting layers. In contrast to perirhinal and entorhinal responses, entorhinal responses are larger to repeated words during memory retrieval.

These results identify a sequence of physiological activation, beginning with a sharp activation from lower level visual areas carrying specific information to middle layers. This is followed by feedback and associative interactions involving upper cortical layers, which are abbreviated to repeated words. Following bottom-up and associative stages, top-down recollective processes may be driven by entorhinal cortex. Word processing involves a systematic sequence of fast feedforward information transfer from visual areas to anteroventral temporal cortex, followed by prolonged interactions of this feedforward information with local associations, and feedback mnestic information from the medial temporal lobe.

Introduction

The human anteroventral temporal lobe (avTL), comprised of inferotemporal (IT), perirhinal (PR), and entorhinal (ER) cortices, works with the hippocampal formation (HC) to perform an essential role in declarative memory (Squire et al., 2004). These structures are interconnected: the superficial layers of ER are the main source of afferents to HC (via the perforant path), and the deep layers of ER are the main recipient of HC output (via the tri-synaptic pathway and subicular complex) (Insausti and Amaral, 2004). Superficial ER receives input from, and deep ER projects to, widespread association cortex (AC). These projections include both direct connections, as well as relays via PR and IT. Basic physiological studies in rodents find that each step in this long multistage feedback loop between AC and HC, via IT, PR and ER, is excitatory (Biella et al., 2002a).

In addition to its crucial role in memory, anatomical (Felleman and VanEssen, 1991), physiological (Naya et al., 2001), and lesion (Murray and Bussey, 1999) evidence in primates strongly indicates that the avTL can also be viewed as the highest level of the ventral visual object-processing stream. A role in language processing is implied by the avTL atrophy that characterizes semantic dementia (Hodges et al., 1992), and the avTL hemodynamic activation evoked by semantic processing (Devlin et al., 2002). In primates, the avTL is reciprocally connected with cortical areas that may be homologous to Wernicke’s and Broca’s areas (Insausti and Amaral, 2004).

As the ventral object processing stream proceeds in the anterior and medial directions, stimulus characteristics evoking cellular responses in macaques become more complex and abstract, and ultimately appear to be associative. This increasing complexity lies on an unbroken continuum of visual processing with the more posterior visual areas. In contrast, the vivid memories evoked by avTL hyperactivation (Halgren et al., 1978b), as well as its crucial anatomical position relaying hippocampal formation output to association cortex, suggests a countercurrent return of information during memory retrieval from more medial structures (Buzsaki, 1996; Halgren, 1984; Merker, 2004; Qin et al., 1997).

These characteristics suggest a sequential evolution of neural information flow in the feedforward and then feedback directions. An overview of the spatiotemporal processing pattern that these pathways engage during verbal tasks can be found in the event-related potentials (ERPs) recorded locally by electrodes implanted for clinical purposes in epileptics. These studies typically find a more posterior peak at ~200ms possibly associated with word-form processing, and a later more anterior peak at ~400ms that is related to semantic manipulations, termed the N400 (Halgren et al., 1994; McCarthy et al., 1995; Smith et al., 1986). Event-related magnetoencephalographic (MEG) responses in normal subjects, with similar latency and repetition effects, appear to arise in the same location (Dale et al., 2000; Dhond et al., 2003; Marinkovic et al., 2003), and the cognitive correlates of the N400 have been confirmed and extended in normal subjects using scalp-recorded ERPs (Kutas and Federmeier, 2000).

One interpretation of these studies is that an initial wave of activity passes quickly through the avTL, and then is followed by sustained activity in all areas, continuing until well past the behavioral response (Dale et al., 2000; Halgren et al., 1994). However, the spatial resolution of ERP/MEG, or even intracranial macro-electrode recordings, is insufficient to determine if widespread extended areas are truly active during this entire period, or if different areas are active at different latencies but they are too close to be resolved. Similarly, these techniques lack the physiological resolution to distinguish synaptic inhibition from excitation, so simultaneous activity could actually represent inhibition in some areas and excitation in others. Finally, although ERPs and MEG are the direct instantaneous result of transmembrane currents caused directly or indirectly by synaptic activity, the presynaptic cell at the origin of that synaptic activity is difficult to infer, although this is crucial for functional interpretation.

These issues have been partially addressed in macaques, where the latencies and durations of unit responses in ventral visual stream areas imply their sequential then simultaneous activation, and the delayed onset of distinctive unit responses to certain visual stimulus distinctions suggests that some processing may require feedback interactions (Lamme and Roelfsema, 2000). However, others argue that macaque data support extraction of high-level information already in the first pass (VanRullen and Thorpe, 2002). In any case, the relation of unit responses in macaques to field potentials in humans is unknown, due not only to differences in the physiological measures, but also to the lack of language in macaques, and to the substantial expansion of avTL areas in evolution.

The current study used a novel technique in humans that is capable of localizing transmembrane currents and multiunit activity not only to particular cortical areas, but also to different layers in those areas. Since feedforward and feedback information flow tends to involve different cortical layers (Barbas and Rempel-Clower, 1997; Felleman and VanEssen, 1991), these data lead to hypotheses regarding the sequence of network interactions between and within these areas, and their relationship to more macroscopic measures. Recordings were made during overt word recognition as well as during implicit repetition during semantic tasks. The initial wave of activity through IT and PR appeared to reflect feedforward EPSCs in middle cortical layers, and was followed by apparent EPSCs that may represent feedback and/or associative processes. Consistent with non-invasive recordings, only the later stages in IT and PR showed repetition suppression. In contrast, ER showed repetition enhancement. These results suggest a spatiotemporal sequence of information processing supporting word processing, with repetition inducing facilitated processing in lateral structures, and explicit recollection in medial.

Materials and Methods

Subjects and Probes

Three patients with long-standing pharmaco-resistant complex partial seizures participated after fully informed consent according to NIH guidelines as monitored by the local Institutional Review Board. Participants were implanted with depth electrodes (fig. 1) in order to localize their seizure focus and thus direct surgical treatment (patient 1: male, 30 years old; pt. 2 female, 35 y.o.; pt. 3 male, 35 y.o.; all right handed, with normal intelligence and personality). Clinical electrodes were modified to be smaller diameter (350μm) in a 5 mm segment at their tips, containing 24 90%Pt-10%Ir contacts, each 40μ in diameter and separated by 110μm (Ulbert et al., 2001a). Simultaneous recordings from macrocontacts on the clinical electrodes were obtained in Pt3. The macrocontacts consisted of 1.3mm diameter cylinders, each 1.5mm long, and separated from the next contact by 3.5mm. MRIs taken with the probes in place (figure 1) show the tip to lie: in patient 1 in anteroventral IT; in patient 2 in PR; and in patient 3 in ER (Insausti and Amaral, 2004). Laminar contacts in gray matter, white matter, and CSF have characteristic activity patterns, permitting resolution of their entry and exit points. The recorded macro-contacts in patient 3 were located in the crown of the inferior temporal gyrus (probable IT) on the left, and fundus of the collateral sulcus (in or near PR) on the right. Structural MRI and/or histological examination of the surgical specimen was normal except for right hippocampal sclerosis in pt. 3. Seizure onset was found to lie outside of the locations reported here: right frontal in patients 1 and 2; right amygdala in patient 3. The decision to implant, the electrode targets and the duration of implantation were made entirely on clinical grounds without reference to this experiment.

Figure 1. Locations of recording sites in MRIs taken with the electrodes in situ.

Laminar probes are indicated by oblique arrows, and macroelectrode contacts by vertical arrows. The white MRI artifacts artifacts lateral to the probes are due to the clinical contacts (larger than the actual electrodes). Patient 1, inferotemporal cortex (IT): laminar tip in the lateral aspect of the right fusiform g., medial bank of lateral occipito-temporal s. (coordinates 38 lateral, 22 posterior, 11 down) (Talairach and Tournoux, 1988). Patient 2, perirhinal cortex (PR): laminar tip in the lateral aspect of the right parahippocampal g., medial bank of collateral s. (coordinates 31, -22, -16). Patient 3, entorhinal cortex (ER): laminar tip in the medial aspect of the left parahippocampal g. (coordinates -23, -17, -25). The left macro-electrode (↓) is in the crown of the inferior temporal gyrus (probable IT) and the right macrorecording (↑) is in the fundus of the collateral sulcus, in or near PR. Some MRIs are displayed with inverted contrast to maximize electrode visibility.

Recordings and Analysis

Differential recordings were made from 23 pairs of successive contacts, at 2kHz (16 bit) sampling rate for CSD and 20kHz (12 bit) for MUA, and stored continuously with stimulus markers. Population trans-membrane current flows were estimated using CSD analysis (Nicholson and Freeman, 1975), calculated as the second spatial derivative of field potentials (0.5–30Hz) after applying a 5-point Hamming filter (Ulbert et al., 2001a). Although the transmembrane currents localized with CSD are generally interpreted as due to transynaptic currents (Mitzdorf, 1985), voltage-gated currents may also contribute (Murakami et al., 2002). One-dimensional CSD analysis assumes that the cortical transmembrane currents are radially symmetrical around the electrode track. While cortical currents are thought to be primarily perpendicular to the local surface (Mitzdorf, 1985), neurons in layer II of ER are arranged in islands, ~200μM in diameter, separated by cell-free zones (Insausti and Amaral, 2004). These islands could result in the unpaired sources and sinks seen in CSD from superficial ER (see figure 4). CSD analysis also assumes that conductivity is uniform and isotropic in the tissue immediately surrounding the probe. This assumption has been tested in the HC where deviations from the homogeneous approximation were found to be too small to influence the spatial distribution of sources and sinks (Holsheimer, 1987). Variable electrode spacing or potential amplification could produce spurious CSD signals but these effects were evaluated experimentally in our system and found to be less than 5% (Ulbert et al., 2001a). CSD analysis will miss transmembrane currents if they do result in a net radial extracellular current, as might happen if they are produced by synapses on spherically symmetrical dendritic domains. CSD will also fail to detect currents that flow over distances that are small relative to the spatial sampling density. Modeling and experimental measures indicate that the center-to-center contact spacing of 150μM used in the current study is adequate to sample laminar CSD in macaque primary visual cortex (Schroeder et al., 1998; Tenke et al., 1993). The limiting factor was the dendritic domains of stellate cells in thalamorecipient layer IVc. Cortex is thicker in humans, and the sampled areas are not known to have a thin but important sublayer comparable to IVc. Nonetheless, it is likely that the CSD analysis reported here is relatively insensitive to synaptic activity on layer IV stellate cells. Finally, current sources or sinks can be missed if the laminar probe does not sample the entire cortical depth. This provides another possible explanation for the unpaired sinks and sources noted in ER.

Figure 4. Entorhinal population synaptic activity and neuronal firing.

CSD color maps (upper rows) show the spatiotemporal patterns of population synaptic activity across four tasks (arrayed in columns)., Waveforms below CSD maps show simultaneous MUA, CSD and macro-electrode recordings (third through fifth rows). The most prominent early activity during explicit Word Recognition is a superficial sink starting at ~200ms (↓) and peaking at ~370ms (a) with strongly increased firing in superficial and deep layers (a’). Sinks in deeper layers (b) with accompanying MUA (b’) also occur during this time. Both deep and superficial sinks are followed by sources in the same layers (c), beginning at ~700ms after the stimulus and continuing for ~900ms. The deep source is surrounded by sinks (e.g., d). Population firing in deep layers continues at a high level during this period (c’), whereas that in superficial layers returns to baseline. All components of the response are larger to old as compared to new words, with the CSD and MUA responses diverging in superficial layers at ~300–400ms (↑), and in deep layers at ~500–600ms (⇑). A broadly similar but substantially weaker spatiotemporal pattern is seen to implicit word repetition during Size Judgments (second column). In Learn/Retrieve (third column), subjects were explicitly asked to memorize the words during Learn, and then were presented with forced-choice explicit recognition. In addition to showing a larger synaptic response to old (e) as compared to new (e’) words during Retrieval, the response is larger to new words during Retrieval (e’) as opposed to new words during Learn (e”). In the fourth column, choice VisuoMotor reactions evoke a similar superficial sink (f) as in Word Recognition, but with less than half the amplitude or duration (a). The difference in neuronal firing is more striking, with VisuoMotor reactions evoking a decrease in firing (f’) whereas Word Recognition evokes a sustained increase (a’, b’, c’). Bipolar potential recordings from macro-electrodes in left IT and right PRshow distinct spatiotemporal patterns that resemble the IT and PR laminar recordings, beginning with a sharp initial peak before 200ms (g), followed by two components from ~240 to 620ms (h, k) that distinguish new and old words. In contrast to the ER laminar recordings, IT and PR potential gradients are largest during Learning (p). The x-axis is thickened when two conditions evoke significantly different activity (black lines for new versus old words, or choice versus simple reactions; orange lines for new-learn vs. new-retrieve in Learn/Retrieve); CSD and MUA waveforms are thickened when each is significantly different from zero (2-tailed p<0.01). MUA responses are only shown for Word Recognition and VisuoMotor tasks, because they were given on the first day of testing when the MUA recordings were higher quality. Macro-electrode recordings were not obtained for the VisuoMotor task.

Population neuronal firing (MUA) was estimated by rectifying high frequency activity (300–3000Hz) and smoothing with a 50hz low pass filter (Ulbert et al., 2001a). MUA was not recorded in Pt. 1 due to interference from the clinical telemetry system. Statistical significance of the difference between conditions for a particular recording channel, latency, and measure (CSD, MUA, or spectral power), was assessed using a t-test of values from individual trials. Significant deviations of responses from baseline were assessed using 1-sample t-tests of values from each trial. Threshold was set at p<0.01 (2-tailed).

Tasks

Subjects viewed single words presented on a computer monitor in Geneva font as white letters on a black background in the central ~5% of visual angle. Stimulus exposure was 240ms and stimulus onset asynchrony was 2400ms unless otherwise noted. The monitor was controlled, and keyboard response accuracy and latency were monitored, by MacProbe software (Hunt, 1994). Subjects remained in their hospital room under videotelemetry during the recordings.

All subjects performed the Word Recognition task to probe explicit recognition, and the Size Judgment task to probe implicit word priming:

Word Recognition (all patients)

Initially, the subjects were instructed to memorize 10 words, each presented 3 times. These words were then presented 12 times each, randomly intermixed with 120 novel words. Any given word repeated after an average delay of ~50 s and ~20 intervening stimuli. Subjects were instructed to press a key with their dominant hand within 1200 ms after presentation of a repeating word. At 1360 ms post-stimulus, a 55ms feedback tone indicated whether the response (or lack thereof) had been correct (1000 Hz) or wrong (200 Hz). In an identical task, large potentials were recorded in the ventral temporal lobe using depth electrodes in epileptic patients (Halgren et al., 1994; Smith et al., 1986).

Size Judgment (all patients)

Subjects pressed a key if the object or animal that the word represents is usually more than one foot in its longest dimension. The 160 words that were presented only once (‘new’) were randomly intermixed with 10 ‘old’ words that each repeated 16 times. Prior to the beginning of the recordings, these ten words were each presented 6 times for familiarization with the task. MEG and fMRI show avTL activation and strong repetition effects in the identical task (Dale et al., 2000; Marinkovic et al., 2003).

Individual patients also performed other supplemental tasks, and these results are shown when they help explicate the responses to the Word Recognition and/or Size Judgment tasks.

Abstractness Judgment /Delayed Retrieval (patient 2) tests explicit retrieval, like Word Recognition, but with a longer delay and less repetition. The subject initially made Abstractness Judgments on 480 visually presented words, without being aware that she would later be tested for recognition. Word presentation was 700ms. Following a 20-30min break, the subject underwent a Delayed Retrieval test phase, where she was shown 960 words, including the 480 previously shown (“Old”). He responded with her left hand, first to indicate whether the presented word was “New” or “Old” and then to rate her confidence in her response as “High” or “Low”. An unpublished MEG study by Dhond et al. inferred strong avTL activation in this task.

Learn/Retrieve (patient 3)

During Learning, 80 words were presented for study, for 300ms each, at 2000 SOA. During Retrieval, the initial 80 words were presented again, randomly intermixed with 80 novel words, and the subject responded to each indicating if it was novel or repeated. This task is modeled after one that has been reported to elicit medial temporal activation with fMRI (Weiss et al., 2004).

Verb Conjugation (patient 1) tests incidental word repetition, as does Size Judgment. The subject was shown 80 new (presented only once) regular verbs, 80 new irregular verbs, 5 old regular verbs (repeated 16 times each) and 5 old irregular verbs, for a total of 320 trials. Regular/irregular and new/old trials were fully crossed and randomly intermixed. Verbs were presented in the infinitive form; the subject silently generated the past tense form, and lifted his left index finger if it ended in “-ED”. MEG sources were inferred in the avTL in a study using the same task in normal subjects (Dhond et al., 2003).

Visuomotor (patient 3) probed simple sensorimotor processes. Targets were flashed for 60 ms in the left or right visual field in random order at ~8º of visual angle eccentricity, and the subject responded with the left or right hand under two ‘Simple’ instructions (press always left or right regardless of stimulus laterality), and two ‘Choice’ instructions (press contralateral or ipsilateral to the stimulus). Stimulus and response laterality were thus balanced between Simple Reactions and Choice Reactions. There were 197 trials for each of these four sections. Time out for producing a key press was 1230ms. Stimulus onset asynchrony was randomized from 1550-1950ms.

Behavioral Performance

The purpose of the behavioral tasks in the current study was not to probe the limits of behavioral performance, but to elicit synchronized neuronal responses that contribute to the semantic processing or recognition of words, and to observe how those responses are modulated when the word is repeated. For these reasons, each ‘old’ word in the main behavioral tasks was repeated 12 or 16 times during the recordings, and each had been repeated 3 or 6 times prior to the recordings. This degree of repetition assures excellent performance in all subjects with at least average intelligence and memory, as was the case for those participating in this study. Specifically, during Word Recognition, Pt. 2 pressed correctly to 111 of 120 repeating words with a reaction time of 762 ±204ms (mean±std dev), and correctly withheld pressing to all 120 new words; Pt. 3 pressed correctly to 117 of 120 repeating words with a reaction time of 850 ±121ms, and correctly withheld pressing to 119 of 120 new words. Behavioral results from pt.1 were not available. Similar results were obtained in larger groups of epileptic patients drawn from the same population, as well as a demographically similar control group (Smith et al., 1986). Due to technical problems, behavioral results are not available for the Size Judgment task. However, in the identical task, a group of young normal subjects responded correctly to 92% of the trials at a latency of 960±123ms to new words and 760±81ms to old words (d’=2.64 and 3.96 for new and old words, respectively), demonstrating strong behavioral priming (Marinkovic et al., 2003).

As noted above, supplemental tasks were also administered to individual patients (clinical exigencies prevented recording during these tasks in all subjects). During Abstractness Judgment/Delayed Retrieval, patient 2 responded to 82% of the input trials correctly at a latency of 1131 ±329ms, and to 65% of the recognition trials correctly at a latency of 2669±1611ms to hits and 1993±1800ms to correct rejects. This is similar in accuracy but slower than a group of normal subjects who on the same task who responded correctly to 86% of the input trials at a latency of 1104 ±128ms, and correctly to 65% of the recognition trials, at a latency of 1104±104ms to hits and 1274±117ms to correct rejects (Dhond et al, unpublished). Behavioral results are not available for the single subjects who performed Learn/Retrieve and Verb Conjugation. However, in the identical Verb Conjugation task, a group of young normal subjects responded correctly to 96% of the trials at a latency of 954±114ms to new words and 841±112ms to old words, demonstrating strong behavioral priming (Dhond et al., 2003). In the Visuomotor task, patient 3 responded to 97% of the Simple trials correctly at a latency of 325 ±15ms, and to 98%% of the Choice trials correctly at a latency of 739±98ms. In summary, the patients performed in the normal range consistent with their cognitive status. The main tasks, Word Recognition and Size Judgment, utilize multiple repetitions of a group of 10 old words to strongly distinguish their behavioral and neural responses from those evoked by randomly intermingled new words that are each presented only once.

Results

Inferotemporal cortex (IT)

Current source density (CSD) recordings of synaptic responses to words

CSD was calculated from the linear array of closely spaced microelectrode contacts in order to estimate the time course of synaptic activity in different cortical layers. In figure 2, upper panel, the CSD recorded in three tasks is shown in color maps of cortical depth versus time after stimulus onset. Red indicates current sinks, locations where the local transmembrane current flows into the cells, as happens at excitatory synapses. Blue indicates current sources, where the current flows out of the cell. In the lower panel of figure 2, waveforms from the middle and superficial cortical layers are shown.

Figure 2. Inferotemporal population synaptic activity evoked by words in memory tasks.

Responses to Word Recognition, Size Judgment and Verb Conjugation are arranged in columns. Averaged CSD color maps (upper two rows) show an initial transmembrane current sink in putative layer IV (red area, a), with a return source in more superficial layers (blue area, a’). These sinks and sources are longer duration to new words (b, b’). The sink may invert to a source, and vice versa, several times from ~300–800ms (c, c’), culminating in a sustained superficial sink (d) and middle layer source (d’). The same phenomena can be seen in the averaged CSD waveforms from selected superficial and middle layers in the lower 3 rows. Purple bars below x-axes indicate word presentation periods. The x-axis is thickened when new and old words evoke significantly different activity; CSD waveforms are thickened when significantly different from zero (2-tailed p<0.01). The synaptic response begins at ~120ms (↓), but differential activity to word repetition does not begin until ~220ms after stimulus onset (↑).

The columns of figure 2 show the responses during three tasks. In all tasks, subjects viewed words, and made a key-press to the target category, which occurred on 50% of the trials. In all tasks, half of the words were new, occurring only once in the task, and the other half were old, from a small set of repeating words. Responses are compared between an overt Word Recognition task, and Size Judgment or Verb Conjugation tasks where repetition is incidental. Separate color maps and waveforms are shown for new and old stimuli, for each of the three tasks.

Initial middle layer sink

In all tasks and to new and old words, activation begins with a sharp sink in putative layer IV, arising abruptly from the baseline (marked with a in the contour plot and waveforms in figure 2). This sink peaks at ~180ms, and is accompanied by a source in layer II/III (a’, figure 2). A t-test found that the responses significantly (p<0.01) deviated from baseline by ~120ms after stimulus onset. In upper layer IV, to old words, this sink inverts in ~100ms to a source. However, to new words, the sink is prolonged (b, figure 2), but it continues to be associated with a source in layers II/III (b’, figure 2). The prolonged sink in layer IV to new words results in a divergence in the response to new versus old words at ~220ms after stimulus onset (p<0.01). Thus, evoked activity was identical to new and old words for the initial ~100ms.

Later responses

In putative layer IV, the initial sink is followed by alternating sources and sinks until ~800ms (e.g., c, figure 2). These middle layer sources and sinks are matched by upper layer sinks and sources (e.g., c’, figure 2). Examination of the waveforms and color maps to old words reveals three peaks in the putative layer IV current sink, at ~200, 400 and 600ms, with variable merging of these peaks into a single sustained sink to new words. These variable alternating sources and sinks culminate in a prominent sustained sink in superficial cortical layers in all tasks to new and old words, generally from ~700–1200ms after stimulus onset (d, figure 2). It is accompanied by a source in middle cortical layers (d’, figure 2).

Perirhinal cortex (PR)

Simultaneous synaptic (CSD) and cellular (MUA, multi-unit activity) responses to words

CSD and MUA responses in PR during the overt recognition and incidental repetition tasks are illustrated in figure 3. Similar patterns are evoked by all tasks. As in IT, synaptic activity in PR begins with a sink in putative layer IV at ~120ms. The initial sharp component of this sink peaks at ~150ms, lasts ~100 ms and is accompanied by a sharp source in more superficial layers. This initial sharp sink is usually continued with a smaller sustained sink in putative layer IV until ~500–700ms. Sinks are also present in superficial layers at longer latencies. This especially clear during Size Judgment, where the superficial sink peaks at ~600ms.

Figure 3. Perirhinal activity during memory tasks.

Words evoke an initial sharp middle layer transmembrane current sink (a) and superficial source (a’), seen in the CSD color maps (upper two rows) and waveforms (middle two rows). The sink likely represents a population EPSC, since it is associated with increased population firing (lower two rows), especially in the middle layer (a”). This is followed by a sink in superficial layers (b), and then variable activity including a very late superficial sink during Size Judgments (c). Population firing and synaptic activity begins at ~120ms (↓), but differential activity to word repetition does not begin until ~220ms after stimulus onset (↑) when new words evoke stronger sustained firing in deep layers until 800–1100ms (d). Similar patterns are observed regardless of whether word repetition is explicit (Word Recognition, first column), or implicit (Size Judgment, second column). The third column also shows an explicit word recognition task, but with a single word presentation during Abstractness Judgments, and an ~1hr delay before Delayed Retrieval. In Word Recognition and Size Judgment, old words are seen several times with delays of ~1min. Although the repetition effect is smaller and later in Delayed Retrieval, it is still present (e). Note that compared to new words during the output period, new words during the input task evoke a larger middle layer sink (f) and more population firing (f’). The x-axis is thickened when responses to new and old words are significantly different from each other; CSD and MUA waveforms are thickened when they are significantly different from zero (2-tailed p<0.01). The thick orange baseline indicates significant differences between responses evoked by new words at input versus retrieval.

MUA, recorded simultaneously with CSD in PR, was most prominent in deeper layers, where it showed a monotonic increase from ~130ms to ~700–1200ms. MUA in putative layer IV showed a sharp increase simultaneous with the local initial sink, then a smaller prolonged response peaking at ~400ms.

When CSD current sinks are generated by excitatory post-synaptic currents (EPSCs), they generally are accompanied by passive return current sources (Nicholson and Freeman, 1975). Conversely, at the normal depolarized state of cortical neurons (Destexhe et al., 2003), inhibitory post-synaptic currents (IPSCs) could produce active current sources, and they would be accompanied by a passive current sink. Thus, the initial pattern of sources and sinks shown in figure 2 could represent either an EPSC in putative layer IV, or an IPSC in putative layers II/III. This ambiguity is not present in the recordings shown in figure 3, where the simultaneous MUA strongly increases from baseline, implying that the current sinks represent EPSCs rather than passive current returns.

Decreased responses to repeated words, across different delays, number of repetitions, and task instructions

PR was also similar to IT in demonstrating decreased activity to repeated words (figure 3). These effects were not apparent in the traces and did not become significant until ~220ms, i.e., ~100 ms after response onset. In PR, the most striking effect of repetition was a decrease in MUA in deep layers. This effect, as well as a decrease in putative layer IV synaptic currents, were significant even after a single repetition of a word, ~1 hour after the initial presentation (DR, figure 3). No differences between subsequently remembered versus not remembered words was found.

Again, as for IT, the effects of repetition in PR were similar regardless of whether they were targets in a recognition task, or repetition was incidental in a size judgment task (figure 3). In both MUA and CSD measures, the effects of repetition in the incidental memory condition were larger and more sustained than in the explicit recognition task, even though they used similar stimuli (words), exposures, delays, and repetitions, and were performed on the same day. It is possible that this observation is due to greater depth-of-processing during the semantic task. In any case, this effect shows that an overt memory task is not necessary in order for repetition-suppression to be observed in PR.

Furthermore, the task of explicit retrieval does not specifically drive PR activity, as shown by a comparison of responses during Abstractness Judgment, to those during Delayed Recognition of the same words an hour later (figure 3). Larger synaptic and unit responses are observed during the input semantic task, as compared to explicit retrieval, when comparing only novel words in both conditions.

Origin of the long latency repetition effect

The clearest repetition effects were noted during Size Judgment, which thus offers the clearest opportunity for examining the relationship of synaptic and neuronal activities in producing repetition effects. The sustained long duration MUA increase to new words in deep layers is associated with a simultaneous sink in middle and deep layers (figure 3). This sink greatly decreases (and may even invert to a source) for old words at ~500ms, at the same time that the MUA also becomes markedly smaller to old words, suggesting that the later part of the repetition effect is associated with a decreased excitatory drive to the lower cortical layers.

To summarize, the response in PR appears to have three phases: (1) an initial biphasic response from ~120–220ms that is not modulated by repetition, consisting of a layer IV sink-source (mirrored by superficial and deep source-sink sequences), and associated with a brief layer IV MUA increase; (2) a sustained layer IV sink from ~250–500ms accompanied by a sustained plateau of deep layer MUA, both of which are moderately changed by repetition; and (3) a deep sink and MUA to new words from ~500–1100ms, and relatively little or even opposite activity to old words (the last phase is most clear during Size Judgment).

Entorhinal cortex (ER)

Overall pattern of synaptic and unit responses in deep and superficial layers

In all tasks, the initial synaptic activities in ER are CSD sinks in superficial (II/III) as well as deep (V/VI) layers (figure 4). Both sinks usually start before 200ms, and the deeper sink is typically smaller and slightly earlier. Both sinks are sustained until ~500ms (range ~400–800). The deep sink is surrounded by weaker sources, whereas the return source for the superficial sink is presumably in more superficial layers that were not sampled in these recordings. Both deep and superficial sinks are followed by sources in the same layers, beginning at 400–800ms after the stimulus (depending upon the task and condition) and continuing in the overt word recognition tasks until more than 1500ms after stimulus onset. The deep source was in all cases surrounded by sinks; the sink corresponding to the superficial source again appeared to be absent due to inadequate sampling.

Distinct patterns of MUA activity were noted in deep versus superficial layers, most clearly during Word Recognition. In putative layers II/III, the time-course (peak, duration and shape) of the MUA closely resembles that of the local current sink, implying that it represents a population EPSC. However, in putative layers V/VI, while the initial peaks of the MUA and the local sink correspond, overall the MUA response is much more prolonged (>1500ms) than the local sink (~700ms). The deep MUA may be related to the prolonged sink in intermediate depths that occurs in most tasks. This intermediate level sink begins at ~400–800ms and continues for ~500–1000ms, depending on the task and condition.

Increased synaptic and unit responses to repeated words

As in IT and PR, CSD and MUA in ER are indistinguishable between new and old words for the initial ~100ms of the response. However, unlike the pattern described above in IT and PR, in ER old words consistently evoked more synaptic and unit activity than did new words following this initial period. The increased activity in Word Recognition consisted of a later and larger peak of neuronal firing in both superficial layers (peak at ~300ms to new and at ~350ms to old words) and in deep layers (peak ~300ms to new, ~700ms to old words). The MUA response duration was also longer to old as compared to new words, ending ~100ms later in superficial layers (at ~700ms), and >1000ms later in deep layers (after 1600ms).

The sinks in both superficial and deep layers were also larger, peaked later, and lasted longer to old compared to new words, as did the subsequent sources in both layers. These differences appeared to be true in all three tasks examining word repetition effects (with the caveat that good MUA recordings were only obtained in the Word Recognition task). The fact that there is a larger response to the old words both in the Word Recognition task (where the old stimuli are targets) and in Learn/Retrieve (where both new and old stimuli are targets), means that targetness is not the critical factor in evoking the greater response to olds. The response during Size Judgment to implicit repetition was markedly less than that to explicit repetition, unlike what has been described above for the same comparison in IT and PR.

Larger responses during retrieval than during learning

In addition to the larger response to old stimuli, activity in ER was distinguished by the larger response overall in tasks that required the overt retrieval of recently stored representations. This is illustrated in the Learn/Retrieve task, where the current sink evoked during learning is much smaller than that evoked during retrieval. This smaller response remains after controlling for repetition, i.e., when comparing new words during learning with new words presented during recognition testing (figure 4).

Weak but differential responses during choice reactions

The memory tests given in the current study all presented words visually and required a key press response. The possible effects of these stimulus-response aspects of the testing situation on ER activity were probed with a test that lacked words or word-repetition, but possessed the other formal properties of the memory tasks. In VisuoMotor, the subject makes a key press response depending on the properties of a simple visual stimulus. In some blocks the responses are always the same (Simple) and in others the laterality of the response depends on the laterality of the stimulus (Choice). Choice-related unit-responses were previously noted in this region in humans (Halgren, 1991; Halgren et al., 1978a; Heit et al., 1990), and animal studies suggest that the HC may be crucial for rapidly learning arbitrary sensorimotor mappings (Dypvik and Bland, 2004; Wise and Murray, 1999).

The overall pattern of synaptic activity evoked by VisuoMotor was similar to that evoked by the memory tasks (figure 4, compare CSD contour maps in the first and fourth columns), and the synaptic response in superficial layers was about twice as large when a choice reaction was being made, as opposed to a simple reaction. However, the CSD response was about three fold smaller during visuomotor Choice reactions, and the duration of the response was shorter, than during recognition judgments. Furthermore, the MUA response to VisuoMotor was a brief decrease relative to baseline, whereas the MUA response during Word Recognition was a prolonged increase. These data suggest that some aspect of making a choice may weakly engage ER.

IT and PR recordings show repetition-suppression simultaneous with ER repetition-enhancement

The laminar responses recorded in IT and PR were consistent with recordings from intracranial macroelectrodes in similar locations during the same tasks, both in the timing of the different field potential components, and in the suppression of the major component peaking at ~400ms by word repetition (Halgren et al., 1994; Smith et al., 1986). Generators with similar latencies, locations, and repetition-correlates have also been inferred from MEG in the same tasks (Dale et al., 2000; Marinkovic et al., 2003). Repetition suppression is also commonly found in macaque unit recordings from IT and PR (Miller and Desimone, 1994; Xiang and Brown, 1998). In contrast to the repetition-suppression of IT and PR CSD/MUA, the repetition-enhancement of ER CSD/MUA was not expected from previous recordings. Since only a single ventral temporal area was recorded with laminar probes in each subject, this raises the possibility that ER repetition-enhancement could reflect an idiosyncratic response of the subject rather than a characteristic property of ER.

In order to evaluate this possibility, bipolar macroelectrode recordings were obtained in or near left IT and right PR simultaneously with laminar recordings from ER in patient 3 (electrode sites shown in figure 1, recordings in figure 4). The potential gradients inverted both medially and laterally at the adjacent contact-pairs, strongly implying local generation. These areas showed typical ventral temporal IT or PR responses, with an initial sharp component peaking prior to 200ms that is identical to new and old words. This is followed by a large peak that resolves more quickly to old words, with the new/old difference visible as an inflection beginning at ~240ms. Furthermore, a larger macro IT and PR response is seen during the Learn than Retrieve segments of a memory task. Thus, in their latency, repetition suppression, and response during Learning, the IT and PR macro recordings in pt. 3 resemble previously reported IT macro recordings in large numbers of subjects, as well as the IT and PR laminar recordings in pts. 1 and 2, but they contrast with the simultaneous ER laminar recordings in pt. 3. Thus, the repetition-enhancements and long latencies seen in the laminar ER recordings do not appear to be due to a patient-specific abnormal response. Nonetheless, further replication of this finding is needed.

Discussion

The current study provides a high resolution window into the functional activity in avTL during memory and related tasks. Multiple stages of cortical response were present in all three structures examined, the inferotemporal (IT) and perirhinal (PR) cortices laterally, and the entorhinal (ER) cortex medially. These stages followed the pathways of inter-areal and intra-columnar cortico-cortical projections previously established in animal studies. Repetition-suppression in IT and PR, with repetition-enhancement in ER, suggests that the former may be more involved in incidental priming embedded in semantic processing, whereas the later is more involved in active retrieval.

Intracolumnar and intercolumnar circuitry of word processing

Synaptic activity in IT and PR begins at ~120ms with a sink in putative layer IV, lasting ~100 ms and accompanied by a sharp source in more superficial layers. This initial sink, peaking before 200ms, is usually followed by a sustained sink in putative layer IV until ~500ms or more, and then by a sink in more superficial layers peaking at ~600–900ms. In PR, the sinks are associated with an initial increase in neuronal firing in layer IV and a sustained increase in neuronal firing in layer V. This spatiotemporal pattern of current sinks and increased firing is consistent with an initial phasic layer IV EPSC, followed by long-lasting EPSCs first predominantly in layer IV and then layers II/III. The initial phasic EPSCs may be mainly in the dendrites of layer IV granule cells, with later sustained EPSCs involving the superficial dendritic domains of layer V/VI pyramidal cells.

This basic pattern of an initial layer IV sink followed by a sink in more superficial cortical layers has been repeatedly observed in response to visual, somatosensory and auditory input in animals (Barth and Di, 1990; Schroeder et al., 1998), suggesting engagement of a canonical cortical circuit. Anatomically, layer IV receives input from thalamic relay nuclei, or (‘lower’) areas closer to primary cortex (Barbas and Rempel-Clower, 1997; Felleman and VanEssen, 1991). Layer IV cells project locally to other layer IV cells as well as to layer II/III cells, which in turn project to layer V/VI. The layer II/III pyramids project onward to higher cortical areas, and layer V/VI pyramids project back to the superficial layers of lower cortical areas. Superficial layers also receive local recurrent collaterals. Intracellular (Shao and Burkhalter, 1999; Thomson and Bannister, 2003) and voltage-sensitive dye (Petersen and Sakmann, 2001) recordings in cortical slices confirm that these intra- and inter-columnar projections are excitatory and effective, as do limited recordings in vivo (Mignard and Malpeli, 1991). In vitro studies of the rat visual cortex show that feedforward projections are characterized by sharp excitation followed by immediate inhibitory feedback, whereas feedback is characterized by polysynaptic weaker but more sustained excitation that may be capable of overcoming simultaneous sustained inhibition, especially when feedforward and feedback paths are simultaneously active (Shao and Burkhalter, 1999). This ‘countercurrent’ pattern of convergent feedforward excitation carrying stimulus-specific information, and feedback excitation carrying contextual information, has been proposed as a general organizing principle of cortical function (Merker, 2004).

Specific studies of the avTL in animals tend to confirm the general plan of cortical connectivity noted above (Lavenex et al., 2004). Laminar recordings in PR (BA36) to neocortical stimulation in the isolated guinea pig brain show an initial middle layer current sink, followed by a superficial layer sink (together with middle layer source). Intracellular recordings show that the initial sink is due to monosynaptic EPSPs, and the later is due to polysynaptic EPSPs (Biella et al., 2001). Conversely, PR (BA36) projects widely to layer I in IT (Lavenex et al., 2002), and also within itself (Suzuki and Amaral, 1994). These studies suggest that the initial middle layer sink noted in the current study represents excitatory input from lower cortical areas; the following middle layer sink represents a period when the predominant excitatory input is feedforward; and the final superficial layer sink a period when top-down and local association afferents predominate.

The delay for the sink to rise from middle to upper cortical layers was as long as ~500 ms in the current study, whereas in human interictal spikes it is ~20ms (Ulbert et al., 2004). Comparable delays have been found in animal sensory areas (Barth and Di, 1990; Schroeder et al., 1998) and human V2 (personal observation). Closer examination of the period between the initial layer IV sink and late layer II/III sink, reveals that although a sustained layer IV sink may predominate (especially to new words that need to be processed deeply), in other circumstances, an alternating series of sinks in middle and superficial layers may be revealed. Thus, the dominance of feedforward, feedback and associative interactions may alternate in a manner that is sensitive to the contextual familiarity of the stimulus.

Compared to IT and PR, where a single sink appeared to dominate cortical activity at any given post-stimulus time, two or even three sinks were often simultaneously present in ER. This may reflect the fact that the superficial and deep layers of ER have quite distinct anatomical connectivity, are separated by a cell-free ‘lamina dissecans,’ and may be somewhat isolated physiologically from each other. The superficial ER layers receive input from PR and IT, and the duration of activity in superficial ER observed here is similar to that noted in these lateral areas. Furthermore, animal studies suggest that convergent and/or sustained activation may be necessary to evoke ER activity from PR and IT (Biella et al., 2002a), and that the presence of activity in deep layers of PR indicates that it has become sufficiently active to evoke an ER response (Kajiwara et al., 2003). In the current study, activity was simultaneously present in PR and IT, and the deep layers of PR showed strong sustained MUA, both suggesting that the lateral activation was sufficient to activate ER. Superficial ER also receives extensive projections from other parts of ER in rodents (Biella et al., 2002b). Activity in superficial ER is projected to the hippocampal formation, and after relaying through ~2–5 substructures, returns to deep ER. The much prolonged unit responses in deep ER may reflect the complexity, variety and synaptic depth of this long positive feedback route.

Long neocortical-hippocampal feedback loops are crucial to memory models that posit formation of specific connections in the hippocampal formation associating the novel constellation of elements in an episodic event, with these connections helping to guide the retrieval of that constellation at a later time (Buzsaki, 1996; Halgren, 1984; Qin et al., 1997). The high-fidelity correspondence of the connections to the event, and their ability to accurately guide event retrieval, are critically dependent on the passage of information between association cortices and HC, which must occur primarily through EC, PR and IT. The current results show that the successive links in this multistage system are co-activated despite the difficulties found in model systems to produce such co-activation alluded to above. A comparison of the known laminar termination of connections between these areas with the laminar dynamics of apparent EPSPs suggests that these areas are participating in alternating feedforward/feedback interactions during encoding/retrieval, with the initial feedforward projection gradually evolving to a dominance of feedback projections. Memory effects only occurred during this interactive period. It is possible that the human in vivo ventral temporal lobe physiological interactions are highly selective and information specific in a manner that cannot be replicated with electrical stimulation in anesthetized animals or in vitro.

Linking synaptic processing at the quasi-columnar level to macroscopic activity measures

The local and distant ERPs and MEG produced by a cortical column can be directly estimated from its laminar CSD pattern. Specifically, in IT and PR, the initial sharp sink in layer IV combined with a source in layers II/III would be expected to produce a surface positive local field potential peaking shortly before 200ms, whereas the late source in layer IV combined with a sink in layers II/III would be expected to produce a surface negative potential peaking at ~600ms. The intermediate period, dominated by a middle layer sink but with superimposed alternating sink/source patterns in middle/superficial layers, would be expected to produce a predominantly positive surface potential but with multiple peaks, including one at ~400ms. Thus, the typical CSD pattern observed in IT and PR would be expected to produce an initial surface-positive peak at ~200ms, a more variable positive peak at ~400ms and a negative peak at ~600ms.

Consistent with the CSD pattern described above, intracranial depth electrode macro-recordings from the human avTL during the Word Recognition task typically record potentials peaking at ~200, ~400 and ~600ms (Halgren et al., 1994; Smith et al., 1986). The peak at ~400ms has similar task correlates and latency to the scalp-recorded N400, associated with cognitive contextual integration of words and similar stimuli. While the avTL-N400 occasionally polarity-inverts across the collateral sulcus, it is largest, most reliable, and invariably negative in anterior locations immediately above the ventral surface of the anterior temporal lobe (Halgren et al., 1994; Smith et al., 1986). In contrast, when recorded in a similar task by intracranial subdural strips immediately below this surface, the corresponding potential is positive (McCarthy et al., 1995). This gross surface-positive/depth-negative field distribution inferred from intracranial recordings is consistent with the layer IV sink and layer II/III source in the ventral surface of the temporal lobe.

Major sources of the magnetic counterpart of the N400 (‘N400m’) have been inferred to lie in the avTL from non-invasive magnetic recordings in response to words in the same tasks as were used in the current study (Dale et al., 2000; Dhond et al., 2003; Marinkovic et al., 2003). The orientation of the inferred N400m generator is consistent with an equivalent current dipole in the temporal lobe with intracellular positive current flowing toward the neck. This would result from a layer IV sink and layer II/III source in the ventral surface of the temporal lobe, as recorded here with CSD. Thus, recordings during the same task using non-invasive MEG, intracranial macro-electrodes, and laminar microelectrode arrays provide a consistent multi-resolution view of the N400 during semantic processing.

Intracranial as well as non-invasive recordings have also identified activity in the avTL at ~200ms. In some limited locations this may be specific for words or other visual categories, and the polarity of the activity is less regular (Allison et al., 1999; Halgren et al., 1994). Intracranial recordings also identify a peak following the avTL-N400, peaking at ~600ms, that has been associated with the scalp late positive component or P3b (Halgren et al., 1994; McCarthy et al., 1995; Smith et al., 1986). The avTL-200 tends to be located more posteriorly in the ventral temporal lobe, and the avTL-600 more anteriorly; however, activity over this entire range can be recorded over the entire extent of the ventral temporal lobe (Halgren et al., 1994). Similarly, peaks at ~200 and 400ms followed by a peak of opposite polarity at ~600ms, has been inferred over the entire avTL using fMRI-biased MEG recordings during Size Judgment (see figure 8 of Dale et al., 2000). However, since cortical electromagnetic fields from different sources can superimpose in MEG and intracranial macroEEG recordings, these previous results are consistent with the different temporal components of the cortical response to words being located in different cortical regions.

Both CSD and MUA decline steeply with distance. Our simulations indicate that CSD amplitude declines ten fold in ~250μm (Wang et al., 2005). Theoretical and empirical studies in animals indicate that MUA should decline with distance at least as rapidly (Grover and Buchwald, 1970). Thus, the current study demonstrates for the first time that multiple processing stages over hundreds of ms are generated within the same IT, PR and EC cortical microdomains roughly corresponding to that of a cortical column.

Both anatomically-constrained MEG, as well as intracranial EEG recordings find that other structures, especially the ventrolateral prefrontal cortex, are also active during this entire time period. Taken together with the global view provided by non-invasive studies, these results suggest that words are processed through repeated activation cycles in an extended neural network that involves each of many cortical columns over sustained and cognitively varied processing stages.

Repetition and retrieval effects

Word repetition decreased the size and duration of the layer IV sink in IT, and of the layer V multiunit activity in PR. Repetition-suppression of the layer IV sink is consistent with it corresponding to the N400 which shows strong repetition-suppression as recorded with MEG (Dale et al., 2000; Marinkovic et al., 2003) or intracranial-EEG (Halgren et al., 1994; Smith et al., 1986). Decreased firing confirms similar results in macaques (Miller and Desimone, 1994; Xiang and Brown, 1998), and is consistent with the decreased hemodynamic activity evoked by repeated words in this area in humans (Buckner et al., 2000). The repetition effect was present regardless of whether repetition was task-relevant, and in fact was largest when repetition was implicit, presumably because in such tasks encoding was relatively deep. Overall, the synaptic and cellular responses in these lateral avTL areas were largest to new words when they were deeply processed.

The repetition-induced decrease in IT and PR reported here was not present in the initial ~100ms of the response. This is similar to the findings of MEG in the same tasks where activity spreads to widespread frontal, temporal and parietal areas before the repetition effects begin, more-or-less simultaneously, across the entire network (Dale et al., 2000; Marinkovic et al., 2003). The repetition effect seems to arise out of a sustained poly-synaptic interaction of an extended trans-cortical network, rather than focal plasticity in a small specific antecedent cell-group. In this study, the sustained but not the phasic CSD/MUA responses to words show the repetition and semantic effects that have been seen with fMRI in the same area. In contrast, it is the phasic rather than the sustained MUA/CSD response that correlates best with the fMRI response in the human visual motion area MT+ (Ulbert et al., 2001b). These data would suggest that the relationship between synaptic currents, action potentials, and hemodynamics may vary between tasks and location.

The initial layer IV sink is ~50–100ms in duration before it is cut off with an apparent IPSC. This initial sink probably mainly reflects glutaminergic activation of kainate receptors; the sink is too brief to dislodge the Mg++ ions blocking N-methyl-D-aspartate receptors. Thus, the initial rapid feedforward projection of activity through the visual and semantic systems may be excluded from plasticity. As a consequence, this first pass activity may not change with repetition, unlike the later components reflecting neural networks involved in sustained activity.

In striking contrast to IT and PR, putative EPSCs and neuronal firing in both superficial and deep layers of ER strongly increased to repeated stimuli. While this increase was present in tasks utilizing both implicit and explicit repetition, it was strongest in overt recognition memory tasks. Although this observation was only made in one subject and so needs further replication, within that subject, simultaneously recorded IT and PR macroelectrodes showed typical repetition-suppression. When tested within a given task, and after controlling for repetition, the ER synaptic and cellular response was larger during the retrieval phase, suggesting a role for ER in intentional retrieval. Increased ER firing to repeats has been observed in monkeys when the repeated stimuli are also the behavioral targets (Suzuki et al., 1997). Some hemodynamic studies have found specific medial temporal activation during intentional recall and experiential recognition over implicit repetition, and lesion studies have found a specific deficit after medial temporal lesions in the same circumstances (Yonelinas et al., 2001; Yonelinas et al., 2002), but both findings are inconsistent (Squire et al., 2004). A variety of evidence in animals has led to the suggestion that familiarity/recognition depends on the repetition-induced decreases in cell firing in more lateral neocortical segments of the ventral temporal lobe, whereas conscious retrieval and recollection depends on medial structures, and especially the HC (Brown and Aggleton, 2001).

The finding of selective ER activation during intentional retrieval is consistent with the observation that powerful feelings of familiarity (déjà vu) as well as intense reminiscences of previous events can be evoked by electrical stimulation of the human avTL (Bancaud et al., 1994; Halgren et al., 1978b), and perhaps especially ER (Bartolomei et al., 2004). These phenomena imply an active role of ER in shifting the mode of cerebral processing from one of identifying external stimuli to one of retrieving internal representations. In organizational analyses of the cerebral cortex, the hippocampal formation and ER stand at the top of a hierarchy extending down into all modalities (Felleman and VanEssen, 1991). Organization of the retrieval effort may include suppression of external input, and attention to imagination (Lepage et al., 2000). Conceivably, this shift in the leading source of information for defining mental contents would be implemented neurally as a shift in the relative excitation of medial as opposed to lateral avTL, between memory input and output, as was observed here.

Neocortical-hippocampal interaction during retrieval through the ventral temporal lobe

Anatomically and physiologically, IT is the most anterior part of visual association cortex. PR has many characteristics of IT, but also more strongly interacts with medial limbic cortex, suggesting that it contributes to both visual associations and recognition memory. Repetition probes both aspects of this transition/intersection between visual association of the object stream and memory cortex; the facilitated processing of repeated stimuli provides an implicit marker allowing recognition of familiarity. As visual processing involves increasingly complex and novel constellations of items, processing plasticity blends into associative memory, first as association of visual objects, and eventually as supramodal arbitrary associations. In contrast to this continuum on a stimulus level from priming to association, there is a discontinuity on a process level from facilitated processing to active retrieval.

Recognition involves a variety of processes, ranging from implicit (more rapid processing due to previous access of the same neural representation) to explicit (re-instatement of an entire experience within the context of the previous stream of consciousness). The current study demonstrates that all three structures are active and interactive during both input and output from memory, and in both semantic and mnestic tasks. Although all areas are highly differentiated, they are also highly integrated in a series of chained interlocking feedforward/feedback loops. There appears to be a division of labor across time, with activity at different latencies being associated with different aspects of memory. While all components are present in all structures, they are differentially distributed, suggesting that the same network is used across the memory situation, with different organizations emerging to support different aspects.

The first of these spatiotemporal multi-structure components is associated with the scalp-recorded N200. This stage rapidly forwards information through the cortical hierarchy, into layer IV, up to supragranular pyramids and on to the next stage, encoding the stimulus but not evaluating it. The following N400 through the neocortex may support priming. Intentional retrieval requires specialized late top-down physiological processes originating in the medial temporal lobe.