Human medial temporal lobe neurons respond preferentially to personally relevant images

Abstract

People with whom one is personally acquainted tend to elicit richer and more vivid memories than people with whom one does not have a personal connection. Recent findings from neurons in the human medial temporal lobe (MTL) have shown that individual cells respond selectively and invariantly to representations of famous people [Quian Quiroga R, Reddy L, Kreiman G, Koch C, Fried I (2005) Nature 435(7045):1102–1107]. Observing these cells, we wondered whether photographs of personally relevant individuals, such as family members, might be more likely to generate such responses. To address this issue, we recorded the activity of 2,330 neurons in the human MTL while patients viewed photographs of varying personal relevance: previously unknown faces and landscapes, familiar but not necessarily personally relevant faces and landscapes, and finally, photographs of the patients themselves, their families, and the experimenters. Our findings indicate that personally relevant photographs are indeed more likely to elicit selective responses in MTL neurons than photographs of individuals with whom the patients have had no personal contact. These findings further suggest that relevant stimuli are encoded by a larger proportion of neurons than less relevant stimuli, given that familiar or personally relevant items are linked to a larger variety of experiences and memories of these experiences.

Humans are self-absorbed by nature. One virtually infallible method of enhancing memory is simply to relate the to-be-remembered information to one's self. The self-reference effect (1) is a well-documented encoding enhancement: people are more likely to remember items that are personally relevant, than items that have undergone some other deep or semantically elaborative encoding processes (2). One mechanism by which the self-referent effect might operate is via the incidental recollection of a rich network of information related to past experiences with that particular item. In addition, incidental recollection of related autobiographical associations can lead to performance advantages such as enhanced memory and speeded responding (3). This incidental recollection has been shown to occur in the context of identifying famous people (3), and has also been shown to involve the hippocampus (4, 5). Furthermore, famous faces and names have long been used to query the integrity of recent and remote semantic memory in patients with temporal lobe epilepsy (TLE) (3, 6–11). Finally, autobiographical memory retrieval consistently engages the medial temporal lobe (12), suggesting that cells in this region may respond differentially to faces that elicit autobiographical memory retrieval.

Recently, recordings from the human medial temporal lobe (MTL) have shown that individual neurons can be highly selective in terms of the stimuli to which they respond (13). Out of a set of about 100 visual images across several categories, such as faces, animals, and landmarks, some cells showed robust responses to only a handful of pictures pertaining to a single conceptual category, such a particular famous person. This selectivity provides an important clue as to the mechanism by which the brain represents information currently in awareness (14). In animals, selective sparse coding has also been observed in recordings from the MTL (15, 16). In fact, the idea that the MTL represents information using a sparse code is one of the basic tenets of most contemporary memory models (17–19). Given that the MTL assigns a relatively small number of neurons to a specific stimulus (20), and that the number of stimuli in the environment is very large, does a feature such as personal relevance, which may be related to the incidental recollection of autobiographically significant information (3), make a stimulus more likely to elicit a selective excitatory response from a cell? Here, we address the question of whether individual neurons in the MTL show a preference for personally relevant pictures.

In this study, patients with intracranial electrodes implanted for clinical reasons were shown photographs of varying personal relevance: previously unknown faces and landmarks, familiar but not necessarily personally relevant faces and landmarks (e.g., pictures of celebrities and famous landmarks), photographs of the patients themselves and their relatives, and pictures of the researchers performing the experiments at the University of California at Los Angeles (UCLA), who had daily contact with the patients. Certainly, images of individuals with whom the patients are familiar, such as their family members or the experimenters, are more likely to generate richer autobiographical recollection than images of celebrities with whom they have had no personal interactions, or pictures of faces unknown to the patients. By comparing the propensity of single neurons in various regions to respond to images in each of these categories of stimuli, we show that personally relevant items are more likely to elicit selective, excitatory responses in MTL neurons than images of people with whom the patients are not familiar.

Results

Sixteen patients participated in our study over 34 recording sessions, yielding a total of 2,330 units in 4 regions of the MTL: the amygdala, the hippocampus, the parahippocampal cortex, and the entorhinal cortex. Of these, 874 were classified as single units and 1,456 as multiunits. According to our criterion of selective responsiveness, which excluded neurons that responded significantly to more than 5 different sets of images (i.e., more than 5 celebrities, landmarks, or a combination), selective responses were seen in 153 units This procedure resulted in the exclusion of only 7% of all of the responsive units (165 in total). A histogram depicting the units responding to different numbers of stimuli is shown in the online supporting information (SI). The selective excitatory responses were not equally distributed among MTL regions: there was a regional difference in terms of responsiveness [χ² (3) = 8.4, P < 0.05], and post-hoc tests demonstrated that the entorhinal cortex showed fewer responses than the hippocampus [χ² (1) = 6.0, P < 0.01] and parahippocampal cortex [χ² (1) = 5.4, P < 0.05]. The distribution of recorded and responsive units for each MTL area is shown in Table 1.

Table 1.

Distribution of units by MTL region

Region	Single units	Multiunits	Responsive cells		All units
			Single	Multi
Amygdala	341	453	26	26	794
Entorhinal cortex	237	365	15	11	602
Hippocampus	219	458	28	25	677
Parahippocampal cortex	77	180	7	15	257
Total	874	1,456	153		2,330

Responses by Personal Relevance.

Fig. 1 shows a typical response of a selective single unit in the amygdala to a subset of the photographs. This neuron responded only to a picture of one of the researchers performing experiments at UCLA and not to 100+ other stimuli that were perceptually similar.

Fig. 1.

A neuron in the amygdala with a selective excitatory response to an image of one of the researchers running studies with the patient at UCLA. (A) A sampling of 10 pictures eliciting the large responses and their corresponding raster plots and peri-stimulus time histograms (PSTH). Please note that due to copyright issues, the exact images of famous people used in the study were replaced by similar photographs in this figure. Dotted lines mark stimulus onset and offset, 1 sec apart. (B) Median number of spikes (across trials) for each of the pictures presented to the patient. The horizontal line marks the threshold for defining responsiveness (see Methods). Image numbers correspond to the ones shown at the top of each picture in (A).

We found that MTL neurons respond preferentially to personally relevant pictures, exemplified by the responses of the cell depicted in Fig. 1 to a picture of an experimenter who had daily contact with the patient. Overall, we found that the different categories of faces (unknown, famous, family, and experimenters) generated different proportions of responses across the MTL [χ² (3) = 28.2, P < 0.001]. Photographs of the experimenters were most likely to generate a response (22%), followed by photographs of the patients and their families (17%). Pictures of famous faces, in turn, yielded fewer responses (9%), and the unknown faces generated the fewest responses (5%). To further evaluate the differences in responses in terms of personal relevance, we conducted the following 3 planned comparisons: images of family members vs. images of experimenters, images of family members and experimenters vs. celebrities, and images of celebrities vs. unknown faces. Family members and experimenters did not differ in their ability to elicit responses (P = 0.47), but together, family members and experimenters were more likely to elicit responses than celebrities [χ² (1) = 13.1, P < 0.001], and celebrities were more likely to generate responses than unknown faces [χ² (1) = 4.5, P < 0.05]. These findings are shown in Fig. 2. Of note, we found only 3 units that responded to images of the patients themselves. Therefore, we did not differentiate between pictures of the self and pictures of family members. In contrast to cellular responses to faces, which showed a significant preference for famous faces, as compared with unknown faces [χ² (1) = 4.5, P < 0.05], there was no difference in the proportion of cells responding to famous versus unknown landmarks (P = 0.27), as 14% of the famous landmarks generated responses, whereas 11% of the unknown landmarks did the same.

Fig. 2.

Proportion of images in each category generating selective responses in the MTL. Asterisks indicate significant differences between categories at P < 0.05.

Responses by Region for Faces.

To investigate whether the different MTL regions showed different patterns of responses, we ran a 4(region) × 4(category) χ² analysis and found that the regions did in fact show significantly different patterns: [χ² (9) = 32.3, P < 0.0001].

Next, we examined the extent to which each of the different regions of the MTL coded for personal relevance. Therefore, we carried out the same χ² analysis investigating differences in responsiveness for family, experimenter, famous, and unknown faces for each of the 4 MTL regions, and again we included 3 planned comparisons to investigate the personal relevance effect more thoroughly in those regions that showed a significant difference across the 4 categories. These findings are shown in Fig. 3. In the hippocampus there was a significant difference among these categories [χ² (3) = 9.2, P < 0.05]. Furthermore, whereas categories of family members, experimenters, and celebrities showed no significant differences (P > 0.5), celebrities were more likely to generate responses than unknown faces: [χ² (1) = 10.9, P < 0.01]. In entorhinal cortex, we did not find any responses to unknown faces. For the other 3 categories, a χ² analysis showed that category differences were significant [χ² (2) = 9.1, P < 0.01]. Whereas pictures of family members and experimenters did not differ with respect to their propensity for eliciting responses (P > 0.5), together, these pictures were more likely to generate responses than photographs of celebrities: [χ² (1) = 9.1, P < 0.01]. In parahippocampal cortex, the overall χ² analysis showed a significant category effect: [χ² (3) = 47.1, P < 0.01]. And, as in the entorhinal cortex, categories of family members and experimenters did not differ (P > 0.5), though together they were more likely to elicit responses than famous faces: [χ² (1) = 42.3, P < 0.01]. Famous faces did not differ from unknown faces with respect to their ability to generate responses (P > 0.1). Finally in the amygdala, although the numbers of responses was small, we found a significant overall difference [χ² (3) = 7.91, P < 0.05], with experimenters showing a greater propensity for response elicitation than family members [χ² (1) = 3.94, P < 0.05], but no difference between family and experimenters together vs. celebrities (P > 0.4) and no difference between celebrities and unknown faces (P > 0.2).

Fig. 3.

Regional differences in the percentage of responses for each category of stimuli

Discussion

Our survey of responses to familiar and unfamiliar faces in the MTL demonstrated that the more personally relevant a photograph, the more likely that cells in the MTL will respond to it; specifically, cells were more likely to show selective excitatory responses to photographs of family members and personal acquaintances, such as the experimenters in this study, than to photographs of persons utterly unknown to the participants. The intermediate category of famous people with whom patients were likely to be familiar, albeit with limited personal relevance or experience, also yielded a higher proportion of responses in the hippocampus proper, but not in other regions. Taken together, these findings suggest that personally relevant photographs are encoded by a larger number of neurons than photographs of unfamiliar people, because these photographs are associated with a larger number and perhaps wider variety of experiences. Furthermore, the pattern of responses was not the same across the different MTL regions.

These data are compatible with the view that the selectively responsive MTL neurons are involved in the creation of new long-term memories by associating perception with experience (21). Information may be stored in long-term memory via the activity of these selective cells, because personally relevant images elicit the largest number of selective responses, and these images are also the most likely to be stored in long-term memory (1). The images of experimenters and family members and, to a lesser degree, famous people, are all likely to elicit a richer network of associations than unknown faces, and thus recruit more MTL neurons. Our findings provide support for the notion that the hippocampus and neighboring structures play a central role in the elaboration, encoding, and retrieval of associations, and that these associations affect neural activity even in individual cells (22–25).

The network of associations activated by viewing personally salient images is likely enhanced and may even be driven by the recollection of specific personal memories, which may include episodic, contextual details (3). For example, a picture of a patient's mother shown during the experiment may automatically generate retrieval of a personal memory, such as her recent visit to the hospital, or an incident on a family vacation that occurred many years ago. Personal memories may even be elicited, albeit to a lesser extent, by images of celebrities: seeing a picture of Barack Obama may lead to the recollection of an election-night party, or a particularly moving speech. The activation of the network of associations described could be due to the incidental recollection of autobiographically significant information, recently shown to involve the hippocampus bilaterally (5). Previous work has shown that the coincident recall of episodic memories associated with a particular semantic item confers a performance advantage in the form of enhanced recall, recognition, fame judgment, and speeded reading (3). Furthermore, this recall need not be intentional to involve the hippocampus; in a recent neuroimaging study, activation of the hippocampus was seen both during an intentional famous face recognition task, and a gender-discrimination task using famous faces (26). In fact, in people with damage to the MTL, such as those with Alzheimer's disease and MTL amnesia, autobiographical significance does not lead to a performance advantage on tests of memory, fame judgment, or speeded reading (4). This incidental recall of autobiographical information might explain why images of family members and personal acquaintances elicit responses in the MTL, but also why images of celebrities activate cells in the hippocampus proper. The hippocampus proper has been shown to play a special role in episodic memories, in both neuroimaging and lesion studies (27–31), and further support for this idea may be found in the finding that famous people were more likely than unknown people to generate responses only in the hippocampus.

The roles of the different regions of the MTL in declarative memory are still debated (32), despite the large number of studies designed to address this issue. Several current theoretical models suggest that MTL cortex alone can support familiarity-based memory (17, 33), whereas the hippocampus is necessary for the retrieval of episodic information, or recollection. A growing body of neuroimaging and neuropsychological studies now supports this idea (34, 35). Though some researchers maintain that the hippocampus is initially necessary for all declarative memory processes, but over time these memories become independent of the hippocampus, even if they involve the retrieval of episodic information (32), others suggest that the hippocampus is necessary for episodic retrieval regardless of how long ago the episode occurred (31, 36). Because we did not ask our patients if the images we showed them generated episodic memories, we cannot address this issue directly with our current study. The fact that personally relevant images of the patients, their families, and the experimenters were most likely to generate selective responses in the MTL cortex, however, suggests that cells in this region can encode the relative familiarity of stimuli, and may prefer images with which the patients are most personally familiar. These familiar images also likely elicit the retrieval of a network of associations, based on episodic memories involving these individuals, that is richer than the network elicited by images of unfamiliar people. This hypothesis is in line with previous work showing that cells in the hippocampus code the relative familiarity of stimuli via alterations in firing patterns, even after only a single presentation of any given stimulus (37, 38).

In the hippocampus, and surrounding neocortex, images of family members and experimenters were equally likely to elicit responses in these regions. Given that experimenters are more recent acquaintances of the patients than their family members, this finding may be interpreted in light of the recent controversy regarding the extent to which recent and remote personal memories are represented in the hippocampus and surrounding cortices. Whereas the standard model of consolidation (MCT) predicts that over long periods of time, memories eventually become independent of the hippocampus (39), the idea that this structure is involved in episodic memory retrieval without regard to the age of the memory is one of the main tenets of an alternative theory of MTL memory function—namely, the multiple trace theory (MTT) (40). In line with MTT, the fact that both remote and recent acquaintances elicit cellular responses in the hippocampus and surrounding cortex seems to suggest that remote memories may be represented in the hippocampus proper, as well as in the surrounding regions. Alternatively, proponents of MCT might argue that the specific memories contributing to the evocative power of images of family members and famous people in the hippocampus might be more recent in nature, or that the cells responding to images of remote acquaintances may be involved in reconsolidation or the formation of new memories involving viewing these images during the experimental protocol, or in the hospital setting. Future studies should address the extent to which the specific photographs of family members and famous people that generated responses in the hippocampus were accompanied by incidental recollection of personal memories related to those individuals.

Notably, in the amygdala, photographs of the experimenters generated the largest proportion of responses, suggesting that these cells may represent other related variables, such as a combination of novelty and emotional significance. Such an interpretation is supported by findings regarding the role of the amygdala in the processing of novel (41, 42) and emotionally salient information (43), and the fact that unknown faces were just as likely as familiar faces to elicit responses in this region.

Although the MTL regions showed different patterns of responsiveness to the different categories of photographs (see Fig. 3), the number of significant responses in each area was relatively low, and therefore these regional differences must be interpreted with caution, calling for further studies to elucidate the unique ways in which cells in these regions might code familiarity. As mentioned previously, such studies should include investigations of whether famous faces generating excitatory responses in the hippocampus are in fact associated with personal episodic recollection.

The responsivity of cells in the human MTL to faces in general underscores the importance of facial recognition processes for humans. Interestingly, recordings of firing patterns of MTL neurons in rodents have provided a wealth of evidence that, at least in these animals, the firing patterns are often tied to specific spatial locations (44). Whereas spatial information is particularly important to rodents, recognizing and interpreting facial expressions skillfully represents a critical human function, given the central role that social interactions play in the human experience (45). Though we did not find any significant effects of familiarity with respect to landmarks, we did not include personally familiar landmarks such as the patient's home, school, or work. Future studies including such landmarks are needed to investigate this issue further.

In summary, our findings provide compelling evidence that the relative familiarity and personal relevance of stimuli in the world is coded at the level of individual neurons in the MTL. Given that our patients likely had a greater number of associations created by memorable experiences with family members and experimenters than with celebrities and certainly with unknown faces, these cells may be part of the representation of long-term memories involving these individuals. Finally, there may be something unique to familiar human faces that generate responses in MTL cells, because relative familiarity did not have the same effect when patients viewed photographs of landmarks rather than faces.

Methods

Patients.

Patients with pharmacologically resistant epilepsy for whom extensive noninvasive evaluation failed to yield a single epileptogenic zone participated in our study. To obtain localizing information for potential curative resection, patients were stereotactically implanted with 6–14 depth electrodes from a lateral orthogonal approach aiming at targets selected using clinical criteria. Following implantation, patients remained between 1 and 2 weeks in the ward and were monitored for spontaneous seizures. All patients provided informed consent, and every session conformed to the guidelines of the Medical Institutional Review Board at UCLA.

Recordings.

At the tips of each depth electrode was a set of nine 40-μm platinum-iridium microwires; the ninth microwire had a lower impedance and served as a reference, and the other 8 microwires provided possible cellular signals. Anatomical locations of electrodes were verified via postplacement MRI scans and images created by fusing CT scans taken while electrodes were implanted with high-resolution MRI scans taken immediately before implantation (46). Signals from each microwire were amplified, digitally sampled at 28 kHz, and bandpass filtered between 1 and 9 kHz (Neuralynx). Spike sorting was performed using Wave_Clus (47), a recently proposed algorithm. After sorting, the clusters were classified into single or multiunits. This was done based on (i) the spike shape and its variance, (ii) the ratio between the spike peak value and the noise level, (iii) the interspike interval (ISI) distribution of each cluster, and (iv) the presence of a refractory period for the single units (i.e., <1% spikes within less than 3 ms ISI).

Task.

Patients were tested in their rooms, as they sat upright on the bed, facing a laptop computer. Each image covered about 1.5° of visual angle and was presented at the center of the screen. Each picture was shown 6 times for 1 s per trial. The order of the pictures and their repetition was randomized. Subjects were asked to indicate, after the image had been removed from the screen, whether the picture contained a human face or something else by pressing the Y and N keys, respectively. This simple task, on which performance was virtually flawless, ensured that patients attended to the pictures. We showed the patients 6 categories of images: photographs of the patient and his/her family and friends, photographs of the experimenters, photographs of famous celebrities, photographs of unknown faces, photographs of famous landmarks such as the Taj Mahal, and finally photographs of unknown landmarks, such as a Victorian house or an unknown building. All photographs of faces were cropped such that the field of view was generally limited to the head and shoulders of a single individual. In addition, in a few cases, the person or landmark represented in the images was shown in more than one view: in these cases, only the first representation of that item was considered for further analysis. Duplicate images of the same individual or landmark shown in a different view were excluded. This procedure ensured that no category contained a disproportionate number of individuals in multiple views. Therefore, for each individual landmark or person, only the 6 repetitions of the exact same image were included in this analysis. The number of images in each category varied by patient and across categories, thereby leading us to use proportions of images eliciting responses in each category as the metric of interest.

Data Analysis.

The response to a picture was defined as the median number of spikes across trials in the first second after stimulus onset. Similarly, the baseline for each picture was the median number of spikes in the second before stimulus onset. A unit was considered responsive if the activity to at least 1 stimulus fulfilled 3 criteria: (i) the median number of spikes was larger than the average baseline (for all repetitions of an image) plus 5 standard deviations; (ii) the median number of spikes was at least 2; and (iii) a t test comparing the baseline and response period for the particular stimulus showed a significant difference with P < 0.05.

According to the above criteria, a total of 165 units responded to at least one picture. To avoid a large weight of neurons with many responses in the statistical comparisons (e.g., 1 neuron responding to 20 landmarks counting the same as 20 neurons responding to 1 landmark each), 12 units that responded to more than 5 pictures were excluded from the analysis, thus giving a total of 153 units with which we compared responses across the different category of stimuli. To assess the differences in responsiveness between the categories of stimuli, we ran a χ² test including responses collapsed across all of the regions, a second χ² test (number of response-generating images per region) to check if there was a difference between regions in terms of the pattern of responses, and finally, a separate χ² test to investigate differences in category response patterns for each MTL subregion separately. To investigate simple effects, when a χ² test showed a significant effect of personal relevance, we then conducted the following 3 planned comparisons: images of family members vs. experimenters, images of family members and experimenters vs. celebrities, and images of celebrities vs. unknown faces.