Internally Generated Reactivation of Single Neurons in Human Hippocampus During Free Recall

Abstract

The emergence of memory, a trace of things past, into human consciousness is one of the greatest mysteries of the human mind. Whereas the neuronal basis of recognition memory can be probed experimentally in human and nonhuman primates, the study of free recall requires that the mind declare the occurrence of a recalled memory (an event intrinsic to the organism and invisible to an observer). Here, we report the activity of single neurons in the human hippocampus and surrounding areas when subjects first view cinematic episodes consisting of audiovisual sequences and again later when they freely recall these episodes. A subset of these neurons exhibited selective firing, which often persisted throughout and following specific episodes for as long as 12 seconds. Verbal reports of memories of these specific episodes at the time of free recall were preceded by selective reactivation of the same hippocampal and entorhinal cortex neurons. We suggest that this reactivation is an internally generated neuronal correlate for the subjective experience of spontaneous emergence of human recollection.

The human hippocampus and its associated structures in the medial temporal lobe (MTL) transform present experience into future conscious recollections (1–4). Human MTL neurons respond in a highly specific manner to complex stimulus features (5), to complex stimulus categories (5, 6), to individual persons or landmarks (7–9), and to previously seen and novel stimuli (5, 10, 11). These responses have been demonstrated for stationary stimuli and are usually brief, often lasting between 300 and 600 ms following stimulus onset, and rarely persist beyond 1 to 2 s (8). However, the human experience is seldom that of stationary stimuli; rather, we live and operate in a constantly changing environment. In this environment, we encounter complex stimuli constituting episodes, series of variant multimodal representations linked in temporal succession. It is such temporally sequenced information that is processed by the human MTL (12, 13) and later becomes available for conscious recollection. For this reason, we set out to examine how neurons in the MTL respond to cinematic sequences depicting specific episodes, and, later, when these episodes spontaneously come to mind in the absence of external stimuli, in a free-recall situation that can be reported by individual subjects.

Subjects were patients with pharmacologically intractable epilepsy implanted with depth electrodes to localize the focus of seizure onset. For each patient, the placement of the depth electrodes was determined exclusively by clinical criteria (14, 15). Patients first participated in a viewing session in which they viewed a series of audiovisual clips lasting 5 to 10 s each. Each clip depicted an “episode” featuring famous people, characters, or animals engaged in activity, or landmarks described from various views, and was presented 5 to 10 times in a pseudorandomized order (15). After the viewing session, patients performed an intervening task (1 to 5 min) (15), after which they were asked to freely recall the clips they had seen and to verbally report immediately when a specific clip “came to mind” (free-recall session). Patients spontaneously recalled a mean of 83.2% (±5% SEM) of the clips presented.

Thirteen patients participated in a total of 43 viewing and recall sessions. We recorded from a total of 857 units (441 single units and 416 multi units) (15) in the MTL and the medial frontal cortex (table S1). A unit was considered responsive to a specific clip if it showed a consistent elevated pattern of firing in all trials of that clip. Overall, the majority of recorded neurons, 475 units (54.9%), showed a significant response to one or more of the clips, i.e., consistently increased firing rate in at least one 500-ms segment of clip presentation (15). There were no differences in proportion of responsive units among the various regions sampled in this study [P > 0.05, χ²(5) = 7.6] (table S1). Of the responsive units, 46 (9.7%) showed a sustained response to at least one clip, i.e., a significant elevation of firing rate through most of the clip duration (although not necessarily at a fixed level) (15). Of these 46 cells, 44 were in MTL and only 2 in medial frontal lobe [P < 0.03, χ²(1) = 5.2] (table S1, fig. S1). Twenty of these cells maintained their elevated firing rate at least 1 s beyond clip offset, and in some cases, up to 5 s beyond clip offset. Responses observed were as long as 12 s and were usually attenuated only by the onset of the next clip.

For example, a single unit in the right entorhinal cortex of a patient, presented with a selection of 48 different clips, responded in a sustained manner to an episode from the animated television (TV) series The Simpsons (Fig. 1A). Each time this clip was shown, the firing rate was elevated to an average of 15.57 Hz, compared with 2.11 and 2.23 Hz during other clips and blank periods, respectively (P < 10⁻⁹, two-sample t test). The response persisted for the entire 5-s duration of the clip, continued in some of the trials up to 5 s after clip offset, and appeared to be silenced only by the onset of a different clip (Fig. 1B and movie S1a).

Fig. 1.

A single-unit in the right entorhinal cortex was activated during viewing and recall of an episode from the TV series The Simpsons. (A) Cell responses to a selection of 48 different episodes (movie clips) presented to the patient in three different viewing sessions (parts 1 to 3). For each clip, the corresponding raster plots (six trials, order of trials is from top to bottom) and post–stimulus time histogram (500-ms bins) are given. Vertical dashed lines indicate clip onset and offset (5 s apart); 5-s blank periods were presented occasionally within groups of successive clips and were used to calculate the baseline firing rate, denoted by a gray horizontal line. Red boxes indicate sustained responses. (B) Trial-by-trial response of the neuron. Order of clips is for the purpose of illustration; more intervening clips separated successive Simpsons clips in the actual experiment. Spike raster plot and instantaneous firing rate (spike train convolved with a Gaussian of the full width at half maximum of 1200 ms) are displayed together. (C) Free-recall session that followed the third viewing session (part 3). (Bottom) Sound amplitude of patient voice; (top) a spike raster plot and instantaneous firing rate; gray dashed line denotes the average firing rate during the recall session + 3 SD; numbered dots denote onset time of verbal report of recall events, corresponding to clip numbers in (A). Note the distinct elevation of firing rate just before the patient reported the recall of the Simpsons clip (red arrow). (D) A 50-s window around the Simpsons recall event [blue area in (C)]. Patient’s words are below the bottom panel. Note that the cell’s firing rate rose significantly above baseline 1500 ms before onset of verbal report of the Simpsons clip and returned to baseline after more than 10 s.

The neuron did not respond exclusively to this Simpsons episode. Even within the limited selection of 48 clips, there was a considerably weaker, yet significant, response to another clip, an episode from the TV situational comedy (sitcom) Seinfeld. Of the 46 units with sustained responses, a unit responded in a sustained manner to an average of 1.4 ± 0.1 SEM clips (or an average of 5.7% ± 0.5 SEM of clips presented). For another example, see fig. S2.

In a second example, a neuron in left anterior hippocampus responded with elevated firing rate throughout a single clip from a choice of 48 clips—that of the actor Tom Cruise during an interview (Fig. 2, A and B, and movie S2a). Note that this cell also exhibited shorter, transient (15) neuronal responses to other clips, i.e., consistent elevation of firing rate above baseline only during particular segments of the clip (green boxes in Fig. 2A), possibly reflecting the preference of the cell to a specific feature of the clip or to an episode within the clip. Additional examples are shown in figs. S3 and S4. Overall, responsive units significantly responded in a sustained or transient manner to an average of 17.7% ± 0.7 SEM of the clips presented (or 4.0 ± 0.2 SEM clips) (fig. S5).

Fig. 2.

A single-unit in the left anterior hippocampus is activated during viewing and recall of an episode (conventions as in Fig. 1). (A and B) Note the sustained elevation of firing rate during the episode depicting actor Tom Cruise during an interview on the Oprah Winfrey Show (red box). Note also the transient responses to various clips (green boxes). (C and D) Free-recall session that followed the first viewing session (part 1). Note that the burst of spikes that accompanied the recall of the Tom Cruise clip began 1500 ms before onset of verbal report (“Tom Cruise … on Oprah”). Blue words indicate experimenter’s speech.

We next examined the neuronal firing at the time of free recall when no external representation of the stimulus was present, no external cue was provided, and no external constraint was placed on the recall process. We found that the neurons that responded during viewing of a particular clip also responded during recall of that clip, with a robust elevation of firing rate for several seconds that could be detected during a single recall trial. This is illustrated for the entorhinal cortex neuron that responded selectively during viewing of a 5-s video clip from the cartoon The Simpsons (Fig. 1, A and B); when all 16 clips were freely recalled, the maximal firing rate was obtained in conjunction with recall of The Simpsons episode (Fig. 1C). The unit’s firing rate rose to more than 3 SD above baseline (15) about 1500 ms before onset of the verbal report of recall, peaked about 100 ms before verbal report onset, but returned to baseline only after 10 s or more (see also movie S1b). Similar examples are illustrated in Fig. 2 and movie S2b, and in figures S6 to S10.

This recurrence of selective activity during recall was not an isolated observation found in a few neurons, but was also evident when the population of responsive units was examined as a whole. We calculated the average ratio of firing rate during viewing of clips to baseline firing rate (FR ratio) (15), for all responsive entorhinal cortex and hippocampal units (Fig. 3A, top). The FR ratio during viewing of the “preferred clip” (15) (red bars) is compared with the average FR ratio across all other clips with nonsignificant responses in the same viewing session (gray bars). The composite graph shows a marked elevation of the ratio following onset of the preferred clip, during viewing and, notably, also 3 s after clip offset. The FR ratio histogram for the recall is then shown in the bottom of Fig. 3A, averaged across the same units with respect to the onset time of verbal report of recall (15). During recall of the preferred clip (red), the averaged firing rate of the neurons increased significantly above baseline in the 3 s before onset of verbal report of recall and remained significantly above baseline in the ensuing 2 s (P < 0.05, Student’s t test) (15). These neurons remained at baseline firing during recall of clips that did not elicit significant responses during viewing (gray). This recurrence during free recall of the same selective neuronal responses present during viewing was found in the population of hippocampal and entorhinal units but not in medial frontal units (compare bottom panels of Fig. 3, A and B). These frontal units exhibited a significant selective increase in firing rate during viewing, but not during recall (Fig. 3B, top and bottom, respectively). This episode-specific reactivation phenomenon was weak in amygdala and absent from parahippocampal gyrus (fig. S11), but was particularly striking for hippocampal and entorhinal neurons with sustained responses (Fig. 3C). We also noted reactivation of inhibitory responses, but because of the low baseline firing rate, these inhibitory responses were evident only at the population level (fig. S12). This phenomenon was most prominent in entorhinal cortex cells. For further details, see supporting online text.

Fig. 3.

Average FR ratio histograms during viewing and during free recall. (A) (Top) Ratio of firing rate during viewing of clips to baseline firing rate is averaged across all responsive hippocampal and entorhinal cortex cells (n = 154). Vertical dashed lines denote clip onset and offset. Cells increased their firing rate significantly above baseline during and following viewing of their preferred clip (15) (red bars). Note small elevation before clip onset, probably attributable to anticipatory effect. These cells remained at baseline firing rate (FR ratio = 1) during viewing of other clips (15) (gray bars). (Bottom) FR ratio during recall events is averaged across the same cells from the top panel. Traces were aligned on the onset time of the verbal report of recall (zero time, vertical dashed line). Note that the same cells increased their firing rate significantly above baseline in the 3 s before onset of verbal report of their preferred clips (red bars) and maintained this elevated firing rate in the ensuing 2 s. However, these cells remained at baseline during recall of clips that did not elicit significant responses during viewing (gray bars). Stars denote statistical significance of P < 0.05 (t test) (15). (B) Same as (A) but for cells from anterior cingulate (n = 56). Note that, in contrast to hippocampal and entorhinal cortex cells, although these cells exhibited selectivity during viewing (top), this selectivity was not maintained during free recall (bottom). (C and D) are the same as (A) but for sustained and transient responses separately.

In conclusion, we report here that a subset of neurons in the human hippocampus and entorhinal cortex exhibited highly reliable and specific responses during viewing of video episodes. These responses persisted throughout an episode or appeared during specific segments. The same neurons showed an increased firing rate again with free conscious recall, before the verbal report, when the sequence of physical sensory stimuli was absent and no external cues were provided. This recurrence during recall of specific past neuronal activity was not observed in medial frontal cortex sites. However, it is possible that top-down early recall signals do originate in other frontal or temporal lobe regions not sampled in this study (16–20).

Could the findings reported here be attributed to the neurological pathology of the patients? Although these results should be viewed with caution, such interpretation is unlikely for various reasons. Epileptic activity is characterized by highly correlated activity in large groups of neighboring neurons. The neuronal responses reported here were extremely sparse and seen selectively in individual neurons out of dozens of nonresponsive neurons that were recorded in their immediate vicinity. Furthermore, only 27% of the units were recorded from within the epileptogenic seizure foci. No significant difference was detected when these units were excluded from the analysis (see supporting online text for more details).

The responses to episodes observed here were often remarkably selective and relatively sparse; yet it is clear, even by a simple statistical reasoning, that these neurons must display selective responses to multiple other clips that have not been presented, as only a minute fraction of the vast set of possible episodes was tested in our study (9). Whether multiple clips to which a neuron responds may be related by some abstract association rule is not clear at present. However, some intriguing examples in our data suggest that such rules may exist (see supporting online text and figs. S2, S9, and S13 to S15). It is also important to exercise caution in claims as to what exact aspect of the clips the cell responded to. The critical point, however, is that the same selective responses recur during free recall.

Several neuroimaging studies show that brain activity present during the learning of information, as indirectly measured by the BOLD signal, is reinstated during cued or free recall (21–24), although spatiotemporal limits of fMRI restrict the possible conclusions. Selective increases in single-unit activation during mental imagery of stationary stimuli have been reported (25). However, unlike the situation in our study, subjects were cued by an external, content-specific, sensory stimulus.

The sparse neuronal responses arising from a very low baseline to robust firing during a specific episode are reminiscent of the responses of hippocampal place cells in rodents (26), in which a cell responds whenever the animal is in a particular place in the environment. Internally generated replay of previous firing sequences by hippocampal neurons has been reported in rodents, mostly during sleep and rest states after locomotion (27–29), but also during the awake state (30, 31) and at decision points of a spatial memory task (32), where it might be predictive of the animal’s future choice (33). However, the relation of such replay in rodents to recall of past navigation events has been merely conjectural. Our results from conscious human patients, who can spontaneously declare their memories, now directly link free recall and neuronal replay in hippocampus and entorhinal cortex. The hippocampal and entorhinal machinery used in spatial navigation in rodents may have been preserved in humans but put to a more elaborate and abstract use (5, 34–36).