Visual perception and memory systems: from cortex to medial temporal lobe

Abstract

Visual perception and memory are the most important components of vision processing in the brain. It was thought that the perceptual aspect of a visual stimulus occurs in visual cortical areas and that this serves as the substrate for the formation of visual memory in a distinct part of the brain called the medial temporal lobe. However, current evidence indicates that there is no functional separation of areas. Entire visual cortical pathways and connecting medial temporal lobe are important for both perception and visual memory. Though some aspects of this view are debated, evidence from both sides will be explored here. In this review, we will discuss the anatomical and functional architecture of the entire system and the implications of these structures in visual perception and memory.

Introduction

The first steps in seeing begin in the retina of the eye. When a visual signal reaches the receptor layer of retinal rods and cones, it is converted into an electrochemical signal [1]. These signals are then sent to the brain along the optic nerve formed by a population of ganglion cells. In the primate retina, at least 17 distinct ganglion cell types exist and at least 13 of them project to the lateral geniculate nucleus (LGN) of the thalamus and on to visual cortex [2]. Due to the anatomical bottleneck of the optic nerve, retinal output must be efficiently condensed. Thus, the strategy used by the mammalian visual system is to reduce the representation of the visual scene to a limited number of specialized parallel output channels [3].

There are three particularly well-characterized retinal ganglion cell types linked to parallel pathways that remain anatomically segregated through the LGN and into the input layers of primary visual cortex (V1) [4, 5]. In LGN of human and monkey brain, there are six distinctive layers. The inner two layers (1 and 2) are called magnocellular (M) layers, whereas the outer four layers (3, 4, 5, and 6) are called parvocellular (P) layers. The koniocellular (K) sublayers are found in between of all the M and P layers. Retinal parasol ganglion cells project to M layers of the LGN and on to layer 4Cα of V1 and midget ganglion cells project to P layers of the LGN and on to layer 4Cβ of V1. Small and large bistratified ganglion cells project to K layers of the LGN and on to the cytochrome oxidase (CO) expressing patches (or blobs) of layer 2/3 (Fig. 1) [3]. Thus, visual signals emerging from the LGN use separate channels arising from its internal layers to converge into V1. Although these ganglion cell types are numerically dominant within the retina, others are likely to subserve important parallel pathways that are yet to be identified.

Fig. 1.

Innervations from retinal ganglion cells to V2 visual cortex through LGN and area V1. This is to show how the forward connections in LGN, V1, and V2 are organized. LGN represents the real appearance of this structure in brain. The cylinder in layer 2/3 of area V1 indicates the blob area. Though there are six layers in the area V2, only the layers involved in the connections are shown here. Pa parasol ganglion cells, Mi midget ganglion cells, Bi bistratified ganglion cells, M magnocellular layers, P parvocellular layers, K koniocellular layers, LGN lateral geniculate nucleus, V1 primary visual cortex, V2 second secondary visual cortex

Once the condensed and parallel signals from the retina and LGN arrive in visual cortex, the original components of the visual scene must be extracted, elaborated upon and integrated into an unified percept. The visual cortex uses both hierarchical and modular processing in order to accomplish these goals [6]. Although the basic tuning properties of cells do not differ substantially between the retina and LGN, it has been shown that within the cortical neurons of V1 more complex information on orientation, direction, and color selectivity is extracted [7]. As visual information passes through V1 and on to extrastriate cortex, the response properties found within each subsequent area tend to increase in complexity and selectivity. There are approximately 32 visual areas in macaque brain and these are interconnected by more than 300 distinct cortico-cortical pathways. The great majority of these pathways are reciprocal; for example, V1 projects to area V2 of secondary visual cortex, and V2 projects back to V1. In most cases, there are pronounced asymmetries in the specific cortical layers in which reciprocal pathways originate and terminate. On this basis, the entire collection of visual areas can be arranged in a hierarchy that contains ten distinct levels of cortical processing [8].

The analysis of visual signals in cortex is processed through two major pathways [6, 7, 9, 10]. The dorsal pathway, sometimes called the “where pathway”, starts from V1 and goes through areas V2, V3, MT (V5) to the parietal lobe. This stream is associated with motion, representation of object locations, and control of the eyes and arms, especially when visual information is used to guide saccades or reaching. It is believed that the main function of this pathway is to guide in real time the actions that we direct at objects in the visual world by integrating the spatial relationships between our bodies and our environment. In contrast, the ventral or “what pathway” begins in V1, goes through area V2, and then through area V4 to the temporal lobe. This pathway is thought to be involved in the conscious perception, recognition, and identification of objects by processing their intrinsic visual properties, such as shape and color. The ventral pathway is also associated with the processing and storage of long-term object memory.

Organization of visual pathways

Based on monkey-brain studies, the dorsal and ventral streams have been shown to constitute separate but interconnected parallel pathways in the visual areas. It is believed that each of these pathways maintains a hierarchical relationship between all the areas implicated. The three LGN projections on to layers of V1 are further reorganized (Figs. 1, 2). More-recent results suggest that there are four distinct projections from V1 to V2: blob columns to thin stripes, blob/interblob border columns to thick stripes, interblob columns to pale lateral stripes, and layer 2/3-4A interblobs to pale medial stripes [11]. Classically, the blob-dominated (BD) and interblob-dominated (ID) projections are named after the compartments in V1 from which they originate, whereas the magno-dominated (MD) projection is named after its dominant source of subcortical inputs [12]. The dorsal stream is largely an extension of the MD projection, and it connects to the posterior parietal cortex, running from the occipital lobe where visual cortical areas are localized up to the parietal lobe. The dorsal pathway consists of a large number of interconnected extrastriate cortical areas and parietal areas downstream of V5, including medial superior temporal area (MST), fundus of the superior temporal sulcus (FST), superior temporal polysensory area (STP), ventral intraparietal (VIP), lateral intraparietal (LIP), and 7a to name just a few [13–17]. In contrast, the ventral stream includes the BD and ID projections, and it connects to the inferotemporal cortex (IT) complex, running from the occipital lobe down to the inferior temporal lobe. The ventral pathway also consists of a large number of interconnected extrastriate cortical areas and temporal areas downstream of V4, including temporal areas F (TF) and H (TH) and the various subdivisions of IT cortex [18–20]. IT cortex is believed to be the last visual processing region and it is comprised of at least two inferior temporal areas, TEO and TE, and the perirhinal cortex anteromedially [21].

Fig. 2.

Interconnections of areas implicated in the visual information processing. An overview of the two dominant visual pathways that continues to the MTL areas is shown in this illustration. Areas that are known to influence the visual information processing in MTL, in particular EC, are also depicted here. Each type of connection is color coded. V1 primary visual cortex, V2 second secondary visual area, V3 third secondary visual area, V4 visual area 4, MT middle temporal area, also known as visual area V5, TEO area TEO, TE area TE, PIT posterior inferotemporal cortex, CIT central inferotemporal cortex, AIT anterior inferotemporal cortex, FST fundus of superior temporal, STP superior temporal polysensory area, VIP ventral intraparietal cortex, LIP lateral intraparietal cortex, MST medial superior temporal cortex, SPL superior parietal lobule, IPL inferior parietal lobule, 7ª visual area 7a, PMd dorsal premotor area, PMv ventral premotor area, M1 primary motor area, PFC prefrontal cortex, VLPFC ventrolateral prefrontal cortex, DLPFC dorsolateral prefrontal cortex, MTL medial temporal lobe, TF temporal area F, TH temporal area H, PRh perirhinal cortex, EC entorhinal cortex, HC hippocampus, OF orbitofrontal cortex, I insular cortex, STS dorsal bank of the superior temporal sulcus, AC anterior cingulate cortex, RSP retrosplenial cortex, MF medial prefrontal (infralimbic) cortex

Although dorsal and ventral streams clearly make up two relatively separate circuits, the anatomical segregation between the two streams is by no means absolute. There is clear evidence of crosstalk between streams, such as the reported connections between V4 and areas V5 and LIP [13, 14, 17], as well as between anterior inferotemporal (AIT) cortex and areas FST, VIP and STP [15, 16, 22]. New subdivisions of parietal and IT cortex continue to be established [23–25], and the specific connection patterns of these areas have, for the most part, reinforced the notion of segregated but interacting dorsal and ventral processing streams [26, 27]. Different behavioral goals are subserved by the dorsal and ventral streams and the same sensory cues, such as motion (associated to dorsal), disparity and shape (specialized to ventral) are processed along both streams. However, distinct computations are performed on the same cues within each stream to support different behavioral goals [3]. Lesion studies have further corroborated the observed physiological differences along the two processing streams, with dorsal stream lesions affecting smooth pursuit eye movements, speed discriminations, complex motion perception and the accurate encoding of visual space [28–32] and ventral stream lesions affecting orientation, complex shape discriminations, perceptual invariance and attention [33–35]. Indeed, single cell-recording shows increasing evidence that a number of attributes, such as two-dimensional and three-dimensional shapes, are processed in both pathways [36, 37], even when there is no arbitrary mapping between shape and response [38].

Further, differences in information processing in direct (V1 → V5) [39] and indirect projection pathways (V1 → V2 or V3 → V5) within visual streams have been investigated in awake monkeys. When the indirect projection pathway was inactivated by cooling the lunate sulcus, disparity selectivity in V5 cells was severely disrupted, but direction and speed selectivity remained largely intact [40]. These results suggest that the direct pathway from V1 to V5 provides speed and direction of motion information, whereas the indirect pathway provides disparity information. Although much less is known about other projection pathways to V5 and pathways that provide connections to other extrastriate cortical areas, functional segregation along parallel input pathways in extrastriate cortices is very likely [3].

Beyond visual cortical streams

The neuronal connections originating from the visual cortex do not stop in the IT or parietal cortex. The outputs of the visual cortex are distributed to various other cortical areas in the temporal and frontal lobes and to numerous subcortical structures [12]. The medial temporal lobe (MTL) is a continuation of the ventral pathway and is made up of perirhinal cortex (PRh), parahippocampal cortex, entorhinal cortex (EC), and hippocampus (Fig. 2). MTL is crucial for both spatial and object recognition memory. All areas in MTL are interconnected and receive both unimodal and multimodal information from across the senses, and communicates with the prefrontal cortex (PFC) and other areas [41]. Within the areas of MTL, EC is uniquely positioned to serve as an interface and as gateway to bidirectional information exchange between the neocortex and hippocampus. Areas that provide the major cortical input to MTL are: orbitofrontal, medial frontal, anterior and posterior cingulate, retrosplenial, temporopolar, agranular insular (Ia) cortices, as well as anterior and posterior portions of the cortex along the upper bank of the superior temporal sulcus (STS) and the superior temporal gyrus (STG) [42].

In addition to MTL, anatomical data of the ventral visual pathway suggests strong neuronal projections of IT cortex to the PFC, especially to the ventrolateral prefrontal cortex (VLPFC) [43–46]. In contrast, the parietal cortex of dorsal visual pathway has strong neuronal projections from superior parietal lobule (SPL) to dorsal premotor area (PMd) and the inferior parietal lobule (IPL) to ventral premotor area (PMv) [3], primary motor (M1) areas and dorsolateral prefrontal cortex (DLPFC) [47, 48]. PFC has reciprocal connections with most or all of extrastriate visual cortex, and thus this area is thought to be involved in the modulation of extrastriate processing [45, 49, 50].

Visual signal processing

Virtually all visual processing tasks activate V1 and V2. However, as signal proceeds from one area to the next hierarchical area, the neuronal response properties become increasingly complex. For example, in monkeys, along the ventral pathway, many V1 cells function as local spatio-temporal filters and, respond to oriented bars [51]; V2 cells respond to illusory contours of figures [52]; some V4 cells respond only if a stimulus has a specific color or pattern [53, 54]; and in IT regions cells respond selectively to particular shapes [55–60]. The outputs from V1 and V2 to V5 of the dorsal pathway and to V4 of the ventral pathway represent the beginning of a more pronounced functional segregation of signals. As shown through different experiments using single-cell recording and targeted single-unit recording in alert macaque monkeys, V5 is specialized for processing motion and depth, whereas V4 is specialized for processing form and possibly color [54, 61–63]. Functional evidence provides support that layer 4B of V1 and the thick stripes of V2 have a high proportion of direction- and disparity-selective neurons, whereas layer 2/3 of V1 and the pale and thin stripes of V2 consist mainly of neurons that are selective for orientation and color (Fig. 3). It seems that one of the goals of the systematic integration of parallel inputs that occurs within V1 and V2 is to construct new output channels that can support at least two new streams of information flow [3]. Results from single-cell recordings from areas within the ventral and dorsal streams are consistent with this model of functional specialization. Thus, neuronal processing along the dorsal stream is characterized by direction of motion and binocular disparity selectivity in MT [64, 65], analysis of complex motion related to locomotion and pursuit/tracking in areas downstream from MT in the STS (MST, FST, and STP) [66, 67], and computation of information in target selection for arm and eye movements, object manipulation, visuospatial attention in areas of the IPL (LIP, VIP, and V6). By contrast, neuronal processing along the ventral stream is best characterized by color and contour selectivity in V4 [62, 63, 68], more complex combinations of colors, patterns, and/or shapes in posterior inferotemporal (PIT) cortex [56, 69], and invariant representations of two-dimensional and three-dimensional shapes and objects in AIT.

Fig. 3.

Functional depiction of the visual signal processing throughout the visual and MTL areas. This illustration shows how the various constituents of visual stimuli are processed in both the visual pathways that lead to either action or recognition. LGN lateral geniculate nucleus, V1 primary visual cortex, V2 second secondary visual cortex, V3 third secondary visual area, V4 visual area 4, MT middle temporal area, MTL medial temporal lobe

Contrary to the view of segregated processing pathways, evidence from functional brain imaging in humans demonstrated that object representations were not confined to the ventral pathway, but can also be found in several areas along the dorsal pathway. In both streams, areas at intermediate processing stages in extrastriate cortex (V4, V3A, MT and V7) showed object-selective but viewpoint- and size-specific responses. In contrast, higher-order areas in lateral occipital and posterior parietal cortex (LOC, IPS1 and IPS2) responded selectively to objects independent of image transformations. These findings indicate that basic object information related to shape, size, and viewpoint may be represented in both parallel visual pathways [70].

Perception

Perception involves the immediate apprehension and initial representation of stimuli currently impinging on a sensory area. It has been shown that perception of visual stimuli improves with experience or training and that this learning produces plasticity in adult visual cortex [71]. It has been suggested that perceptual learning in the visual system gives rise to nondeclarative perceptual memory. Along the ventral pathway, memory traces have been defined as nondeclarative (unconscious) or perceptual in the early visual cortex and as declarative in MTL areas [72, 73].

Perceptual learning

Visual perceptual learning is defined as a long-term improvement in performance on a visual task. It has often been suggested that the key plasticity locus in perceptual learning is likely to be in early visual cortex, as this learning is confined to the trained retinal position [74–76]. Based on this hypothesis, changes in the receptive field tuning properties of neurons in V1 might account for the specificity of learning effects to the trained visual field position and trained stimulus attribute. Indeed, recent imaging studies provide evidence for the involvement of V1 in object feature learning [77–79]. However, neurophysiological evidence for the contribution of V1 in behavioral improvement after training on visual discrimination remains controversial [80, 81]. There is some evidence for sharpening of orientation tuning after training [81], but no evidence for changes in the size of the cortical representation or the receptive field properties of neurons in V1 [75, 80].

However, recent studies combining behavior and functional magnetic resonance imaging (fMRI) examined the relationship between shape learning and experience-dependent reorganization of shape representation across stages of visual processing [82, 83]. The results suggest that shape representation might shift from higher to early visual areas, which support rapid and automatic search and detection in visual cluttered scenes independently of attentional control. These findings are consistent with the suggestion that learning moulds object representations not only by enhancing the processing of feature detectors with increasing complexity in a bottom-up manner but also in a top-down manner taking into account the relevant task dimensions and demands. In particular, it has been suggested that learning begins at higher visual areas for easy tasks and proceeds to early retinotopic areas that have higher resolution for finer and more difficult discriminations [84]. At the neuronal level, learning could be implemented by changes in core feedforward neuronal processing, especially at higher visual stages [85]. However, in another scenario, learning can also be processed through changes in the interactions between object analysis centers in temporal and frontal cortical areas and local connections in V1 visual cortex based on top-down feedback mechanisms. Indeed, learning has been suggested to modulate neuronal sensitivity in the early visual areas by modulating networks of lateral interactions and through feedback connections from higher visual cortical areas [75, 79, 86–89]. Such changes in the connectivity of visual analysis circuits might be more adaptive and efficient compared with changes in core feedforward visual processing (e.g., receptive fields) that might have deleterious consequences for the visual processing of the trained stimuli in another context or task.

Perception in MTL: the controversy

Classically, it was thought that perception of visual signals is processed in visual cortical areas. However, new evidence in literature suggests that not only the visual cortex but also MTL areas are important for visual perception [90–93]. The idea that the MTL, viewed as an exclusive declarative memory system [94], may also subserve perception has been debated fiercely [95–97]. At the center of this controversy, there are studies that have demonstrated visual discrimination deficits in monkeys and humans with MTL damage, impairment in implicit contextual learning, perceptual learning and complex spatial discrimination after hippocampal lesions and production of complex object discrimination deficits with PRh damage [92, 98–109]. The interpretation and implications of these findings have, however, been debated for several reasons. First, these discrimination impairments have not been replicated by other research groups [110–115]. Second, studies that have reported visual discrimination deficits after MTL damage have mostly relied on neuropsychological tasks that require participants to compare multiple simultaneously presented images, such as oddity judgment. Thus, it is plausible that memory and perception are confounded in these tests, and poor performance may reflect a deficit in MTL-mediated working memory processes [116–118] or the failure to use long-term memory representations [114] rather than a perceptual problem.

In a study adapting a classic object decision paradigm [119], amnesic patients without lateral temporal cortex damage were asked to assess the structural coherency of single drawings of novel objects. Performance on this task was compared with that on a test in which participants made same/different judgments of two novel objects presented simultaneously (a task that demanded the comparison of multiple stimuli but not higher-order object perception). Only the patient with perirhinal atrophy was found to be impaired on the perception of single objects [93]. The contribution of human PRh in perception was further investigated in healthy volunteers performing visual discrimination task [120]. The task was designed to engage perceptual, but not mnemonic, processes by presenting participants with an array of four images and forcing them to choose the odd one out. Because the duration of the stimuli on the screen was longer than the response time, the task did not require stimuli to be encoded into memory. In addition, unlike monkeys, who require extensive training to learn the task, humans are able to perform the task immediately with trial-unique stimuli, effectively eliminating any significant contributions of learning. PRh activation in functional imaging was revealed when the task required the integration of visual features into a view-invariant representation but not when it could be accomplished on the basis of simple features (e.g., color and shape). This activation pattern matched lateral inferotemporal regions classically associated with visual processing but differed from EC activation associated with memory encoding. Similarly, in another study, an increase in PRh activity was observed while healthy individuals performed oddity judgment for three types of trial-unique novel stimuli (novel objects, faces, and scenes) in a trial-unique version of the oddity discrimination task [121]. These findings support a role of MTL in visual perception.

Visual object memory and areas implicated in its processing

Object memory is a form of declarative memory that involves the ability to judge whether a stimulus has been encountered previously [122, 123]. It is believed that different subareas of MTL are dedicated for the processing and formation of this kind of memory. Within the MTL, the hippocampus receives visual information from IT cortex via projections through PRh and EC (Fig. 2) [124, 125]. PRh receives most of the neocortical input from association areas that are known to process unimodal sensory information about objects through ventral stream, whereas most of the neocortical input to the parahippocampal cortex (called postrhinal cortex in rodents) comes from dorsal stream areas that process polymodal spatial information. It is thought that the ventral and dorsal streams of processing remain largely segregated as the PRh projects primarily to the lateral entorhinal area, whereas the parahippocampal cortex projects to the medial entorhinal area. Therefore, it has been proposed that the information from both streams converge mainly within the hippocampus. Identity of a complex object is represented by area TE, a subarea of IT cortex, but the association with its name is represented by the PRh [126, 127]. In addition, PRh is also involved in object recognition and damage to this area impairs performance on recognition memory tasks [128, 129].

V2 visual cortex

Area V2 has been thought to be exclusively associated with the perceptual processing of visual signals. However, recent study provides evidence for the implication of this area in the formation of long-term object recognition memory (ORM) [130]. It was shown that stimulation of layer 6 neurons of V2 by overexpression of a protein called RGS-14₄₁₄ led to the conversion of ORM that usually lasts for 45 min in normal rats into long-term memory that could be observed even after many months. Furthermore, selective elimination of the same neurons of V2 by infusion of OX7-SAP, an immunotoxin that selectively kills neurons without affecting other structures, not only abolished the protein-mediated boost of ORM but also the normal ORM. Additionally, the capacity to retain information on multiple objects was also higher in these RGS-animals. Normal rats could retain the information of two different objects at once but were unable to do so for four objects. However, in contrast to normal animals, RGS-treated rats could retain information of six objects. These findings provide a new dimension to current understanding on the formation of ORM and further emphasize the importance of an early visual cortical area into object memory.

Inferior temporal cortex

Area TE of IT cortex and PRh are cytoarchitectonically distinct but mutually interconnected [131, 132]. Both areas have been implicated in different aspects of visual perception and memory. TE, a unimodal neocortex located at the final stage of the ventral visual stream, is involved in the processing of object vision [23, 133]. On the other hand, PRh is a polymodal association area and a component of the MTL that participates in the formation of declarative memory [134, 135]. Interactions between TE and PRh are thought to be a critical substrate for long-term visual memory [136–142]. IT has an important role not only for synthesizing the visual attribute into a unique configuration but also for the storage of visual memory in both monkeys and humans [143–145]. Single-unit recordings in the monkey IT found neuronal correlates of visual long-term memory [146], where IT neurons could reflect learned associative relations among stimuli. Additionally, it has been reported that processing of visual stimuli in TE is as complex as for PRh [143–145, 147].

In monkeys, cortical visual processing of object recognition is thought to be organized in a set of hierarchically connected cortical regions, including the peristriate cortex, posterior TEO, inferior temporal cortex (TE3, TEa, TEm), anterior temporal cortex (TE1, TE2), PRh, and EC. All these areas have been shown to play some role in memory for visual object information [10, 129, 148–151]. On the basis of anatomical evidence, a similar hierarchical organization has also been uncovered in the rat [152–156]. The medial extrastriate visual cortex has been found to be important in mediating visual object memory in rats [157]. Lesions of these posterior temporal cortical areas also produce deficits in an ORM task [158–160].

Perirhinal cortex

Perirhinal cortex is one of the most important structures for object recognition [110, 128, 129, 135, 159, 161–163]. This area appears to mediate memory acquisition, consolidation, and retrieval in the ORM task [164]. These functions have been shown to depend on specific neurotransmitter systems within PRh, including N-methyl-d-aspartic acid (NMDA) glutamatergic, muscarinic cholinergic, and GABA_A receptors [165–170]. Damage to this area disrupts object recognition in humans [110, 163], monkeys [115, 128, 129], and rats [159, 161, 171–173]. In monkeys, lesions of PRh cause far greater disruption of ORM than damage to any other single structure in the MTL [129]. EC lesions only produce mild [129] and transient impairments in ORM [174]. Additionally, it has been shown that transient inactivation of PRh with lidocaine disrupts encoding, retrieval and consolidation of an object information [164]. Rats with PRh lesions were not impaired on a spatial working memory test (reinforced T maze alternation), however, they were markedly impaired on a spontaneous ORM test [161]. In another study, lesions of the rat PRh spared the acquisition of a complex configural visual discrimination that required the animals to use strategy to solve the problems on a series of three visual discriminations but impaired their object recognition [175]. Furthermore, studies on amnesic patients with broad damage to MTL structures that included the hippocampus and PRh found an impairment on both face and scene memory. By contrast, participants with damage limited to the hippocampus showed deficits only in memory for scenes [163]. These results underline a critical role for PRh in ORM.

Recently, rats with PRh damage were shown to treat novel objects as they were familiar in a variation of the standard ORM task [176]. This observation led to the proposal that these animals are behaving in such manner because of confusion within memory rather than a loss of memory as it was thought previously. This concept was further supported by the demonstration of complete ORM rescue in rats with PRh lesions. These animals were maintained in darkness during the delay interval in the ORM test, thereby eliminating interfering visual information flow from impinging on the brain during that time [176]. Furthermore, this interpretation can also account for ORM deficits in aged rats, who also behave as if novel stimuli are familiar [177].

Hippocampus

Mounting evidence suggests the presence of functional double dissociation between PRh and hippocampus [178, 179]. This functional dissociation has been observed by imaging experiments monitoring the levels of expression of immediate early gene c-fos as an index of neuronal activation in response to stimulus exposure. PRh was activated significantly more by novel than familiar pictures of objects, whereas the hippocampus was insensitive [180]. Conversely, pictures of novel spatial arrangements of familiar objects significantly activated hippocampal CA1 area and PRh was unaffected. Together, these results suggest the importance of PRh in object recognition and the hippocampus in spatial [181] and other relational [182] functions. Similar dissociations have also been seen in lesion studies. Selective lesions of the hippocampus did not affect recognition of novel objects but severely impaired detection of new arrangements of the same objects [171, 183, 184]. In contrast, lesions of rhinal cortex impaired recognition of object novelty but did not affect memory for the arrangements of objects [171, 183]. The importance of the hippocampus for spatial memory [185] and PRh for object recognition [186] is also well documented in monkeys. Additionally, functional neuroimaging studies also showed a dissociation between PRh in encoding of object novelty and posterior hippocampus in encoding of novel spatial arrangements [187–191].

Lesion studies indicate that the dorsal hippocampus in rats (posterior hippocampus in primates) is critical for spatial learning and memory [192]. These observations are further corroborated by a single-unit recording study in monkeys demonstrating that activity observed after exposure to stimulus in spatial memory task was more common in the posterior hippocampus [193] and in rats showing greater prevalence and selectivity of spatial firing correlates in dorsal relative to ventral hippocampus [194]. A PET study also reported activation in the posterior part of the hippocampus during route recall [187, 195]. However, a recent observation suggests that rodent ventral hippocampus is also important for spatial memory at larger scales of representations [196]. Hippocampal involvement has been observed in learning of rewarded locations within complex visual scenes [197]. Furthermore, rats with fornix lesions performed as well as controls in an object–place recognition task, a test that requires animals to keep in memory the location in which an object was encountered, but were impaired on an object–place–context (“what–where–which”) memory task where rats have to remember not only the object and location but also the context in which they were encountered [198]. Consistent with the effects of fornix lesions, animals with bilateral hippocampal lesions showed impairment only on the object–place–context task and not on the object or object–place tasks [199]. Similarly, monkeys with neurotoxic hippocampal lesions performed normally in a one-trial object–place association memory task [200] and were able to learn large sets of object–place association problems [201]. These findings suggest that not all forms of spatial associative recognition depend on the integrity of the hippocampus.

Though the participation of the hippocampus in spatial memory is well accepted, its role in object recognition remains controversial [202, 203]. Although many lesion studies have shown that object recognition can occur normally in the absence of the hippocampus, a functional specialization within the hippocampus for the encoding of novel and previously experienced materials has been observed using neuroimaging paradigms. It was found that the anterior hippocampus is important in processing of novel objects [204] and novel scenes [205]. Furthermore, the hippocampus may be particularly necessary when recognition specifically engages recollection processes [206]. Altogether, these observations suggest that both the hippocampus and PRh contribute to recognition memory, but their respective roles are dependent on the type of stimulus to be remembered.

Prefrontal cortex

Although visual stimulus-related memory is processed in visual cortical and MTL areas, it is acknowledged that these are not the only structures in the brain that are involved in this activity. It has been shown that PFC plays an important role in visual and spatial working memory [207, 208] as well as in visual recognition memory [209] and scene-specific memory for objects [210]. PFC damage was implicated in patients with abnormally high false-alarm rates on recognition memory tests [211]. This observation is consistent with effects of medial frontal lesions on recognition memory in rats [212]. PFC has also been shown to take part in the retrieval of source information related to “where” component of the past experience in mice [213], congruent with impairments in source memory that accompany PFC damage in humans [214]. In a mice study, normal animals tested on the “what”, “where”, and “when” components of recognition memory were able to remember which objects they had explored, as well as when and where they were experienced. However, mice with damage to the hippocampus were impaired on all three components of the task. In contrast, animals with medial frontal cortical lesions were selectively impaired on the “where” component of the task, but had intact memory for “what” and “when”. These results are consistent with the hypothesis that the hippocampus integrates the “what”, “where”, and “when” features of unique experiences, whereas PFC makes a more selective contribution to retrieving source information about the context in which events occurred. Furthermore, it has been shown in rats that PRh is able to support recognition memory for individual items even when the hippocampus or PFC is damaged [215]. In contrast, an intact hippocampus and medial PFC is necessary when judgment of prior occurrence requires items be associated with a particular place (object-in-place recognition memory) or when discrimination needs to be made of the relative time when items were encountered (temporal order memory).

Familiarity and recollection

There are two ways to recognize a visual stimulus that has been encountered on a previous occasion. In some cases, we experience a general feeling of memory, or familiarity, which indicates that the presented event has been experienced before. On other occasions, we recollect contextual details from a previous encounter. This dual-process account of recognition memory was shown to be due to two separate component processes [125, 162, 216–221]. Although such views remain contentious, electrophysiological studies in monkeys and rats have provided support for a specific role of PRh and not of hippocampus in the familiarity judgment process [217, 222, 223]. Specifically, electrophysiological recordings from neurons in the MTL of monkeys or analogous regions in rats indicate that a good percentage of neurons (up to 25%) in PRh and adjacent cortical areas respond less vigorously to familiar visual stimuli than to novel visual stimuli [217, 224–228]. The responses of such cells are markedly reduced from the first to the second presentation of a visual stimulus. Such reduced response is rarely observed in the hippocampus, but is commonly reported in IT cortical regions, particularly PRh [222].

To distinguish recollection and familiarity, a strategy of signal detection has been applied based on the hypothesis that these two processes have different profiles. Receiver operating characteristic (ROC) is the function that relates the proportion of correct recognitions (i.e., the hit rate) to the proportion of incorrect recognitions (i.e., the false-alarm rate). For example, after studying a list of words, subjects are presented with a mixture of old and new words and are required to make recognition-based judgments on a scale ranging from “sure it was studied” to “sure it was not studied”. The performance is plotted as a function of confidence. The ROC curves have been used to characterize both forms of recognition memory in human subjects. This curve has a curvilinear shape, and it is also asymmetrical. The asymmetry has been interpreted as evidence for a threshold for recollection, and the curvilinear component as reflecting the strength of familiarity [216, 221]. The loss of asymmetry (an index of recollection) combined with the retained curvilinearity (an index of familiarity) following selective damage to the hippocampus in rats provides evidence that recollection and familiarity may have distinct neural substrates [206]. In addition, a disproportionate impairment of familiarity was observed in patient with resection in EC and PRh where the hippocampus was spared [219]. In an expanded study, individuals with varying levels of recollection and familiarity, which included older adults and patients with amnesic-mild cognitive impairment (MCI) and Alzheimer’s disease, showed a stronger relationship of recollection with the hippocampus and familiarity with the extrahippocampal MTL [220]. These results are consistent with the proposal of the dual-process model. Additional support for this hypothesis comes from another study where damage to the hippocampus causes a shift toward reliance on familiarity while eliminating reliance on recollection [229].

Both routes, familiarity and recollection, lead to recognition, but the way they operate is a matter of great dispute in the field [123, 230]. There is minimal consensus regarding the participation of MTL structures in supporting these two components. According to one view, different MTL structures are responsible for each form of recognition. Familiarity has been shown to rely on PRh, whereas recollection is thought to be mediated by the hippocampus [231, 232]. fMRI studies using a scanned encoding phase and a subsequent memory test have shown differential activations within the MTL. The hippocampus has shown a greater response to recollection, whereas the PRh appears more important for encoding familiar stimuli [232]. Other views, however, regard familiarity and recollection as products of varying degrees of strength where familiarity and recollection reflect the lower and the higher strengths, respectively [123], consistent with a single-process model. Thus, it was argued that all MTL structures contribute equally to familiarity and recollection. Accordingly, it has been shown that the activity in both the hippocampus and PRh during encoding of words predicts the strength of recognition memory [233]. More recent results from the same group suggest that although the structures of the MTL do not play functionally identical roles in recognition tasks, the apparent differences between them are not related to the distinction between recollection and familiarity [234].

The importance of the hippocampus in object recognition memory: the controversy

Although there are several reports showing the relationship of dysfunctional hippocampus with impaired recognition memory in humans [72, 163, 235, 236], monkeys [237–242], and rodents [243–255], many studies have also failed to find notable deficits in subjects with hippocampal damage [115, 157, 161, 173, 184, 186, 256–268]. Furthermore, it is argued that even in cases where hippocampal damage has been shown to disrupt object recognition, the impairment is often much less severe than the deficit caused by PRh lesions [202, 251, 269]. In contrast, in a non-lesion approach, authors specifically blocked the generation of new granule cells in dentate gyrus, a process that has previously been linked to learning and memory [254]. Rats with substantially reduced levels of newborn neurons showed less preference for the target zone in water maze trials. The same knockout animals were also impaired in the performance on ORM task. These findings reveal that the hippocampus is important for both spatial and recognition memory. A similar conclusion was drawn from another study where anterograde and retrograde effects of hippocampal lesions on object recognition was evaluated [255]. After exposure to two identical objects, rats received either bilateral lesions of the hippocampus 1 day, or 4, or 8 weeks after the exposure. On the object recognition test, 1-day and 4-week hippocampal lesion groups exhibited impaired ORM. In contrast, the 8-week hippocampal lesion group performed similar to controls where both groups exhibited a preference for the novel object. These same rats were then given four postoperative tests using unique object pairs and a 3-h delay between the exposure and the test. In this test, animals with hippocampal lesions produced moderate and reliable memory impairment.

A neural system of recognition memory

Recognition of prior occurrence for an individual item relies on the PRh, however, recognition memory that involves multiple items and their contextual associations or the temporal order in which items are encountered depends upon interactions between the PRh, hippocampus, and medial PFC [215]. The disconnection of any two of the structures of hippocampal, PRh, and medial prefrontal system has been shown to produce associative or temporal order recognition memory impairment, indicating that interactions among these structures are necessary. Furthermore, interactions involving the hippocampus and PRh are not sufficient to support performance on the object-in-place task in rats if interactions with medial PFC blocked. Thus, it has been suggested that successful performance of this task is a property of the PRh-hippocampal-medial PFC network, and that this network may engage to produce integrated recognition memories containing visual, spatial, and temporal components in day-to-day memory formation.

Spatial information processing and memory

The integration of spatial information of a visual stimulus is thought to be associated with cortical plasticity along the dorsal pathway. It has been shown that experience-dependent synaptic plasticity occurs in adult V1 visual cortex leading to the formation of long-term spatial memories [270–272]. In addition, synaptic plasticity occurring in V1 influences the ability of the hippocampus to integrate spatial episodes in time [273]. It has been suggested that this interaction may comprise a key element in neocortico-hippocampal transfer of new spatial information observed during exploratory behavior. Furthermore, viewing of novel spatial arrangements of familiar objects activates posterior hippocampal formation and posterior parahippocampal cortex [204]. Areas TF and TH of the parahippocampal cortex from monkey brain have been implicated in visuospatial processing [131, 274]. These areas receive projections from the retrosplenial cortex and in addition, area TF also receives direct input from the posterior parietal cortex [131]. Besides its retrosplenial and parietal connections, area TF also receives massive input from unimodal visual areas V4, caudal TE, and TEO [131]. These known anatomical connections further support a role of parahippocampal cortex in the processing of spatial memory.

Analysis of performance in different spatial learning tasks has led to the suggestion that the integrity of connections between the hippocampus, subiculum, and cortical areas is necessary for synthesis of all components of spatial learning [275, 276]. While exploring a space, modification of synaptic weights occurs very fast, which was shown to be responsible for the activation of the hippocampus [277]. It is thought that entorhinal-hippocampal network oscillations at theta frequency play a crucial role in this process [278]. It has become increasingly apparent that hippocampal long-term potentiation (LTP) and long-term depression (LTD) may engage in the encoding of different aspects of visual spatial information [279, 280]. Exposure to a new environment triggers LTP as a global response across the hippocampus. This is quite distinct from the involvement of LTD in information encoding [279]. Although LTD appears to occur intrinsically as a response to changes in environmental detail [281, 282], the hippocampal subregions appear to process different aspects of it. LTD in the CA1 region is facilitated by exposure to contextual changes in small characteristics of an environment [283], whereas LTD in the dentate gyrus is facilitated by contextual changes in landmark or orientational features of it [284]. This suggests LTP and LTD encode different aspects of spatial memory.

Superior colliculus: an alternate pathway in visual information processing

In contrast to both visual streams, a path through the superior colliculus (SC) and pulvinar that does not involve the LGN or V1, may subserve some of the perceptual aspects of visual processing to fasten the transfer of information for early detection of motion and the early planning of orienting eye gaze, features shared by the SC and dorsal pathway [285–289]. In addition, bypassing V1 suggests that this pathway is likely to be responsible for carrying visual information that underlies at least some aspects of blindsight associated with dorsal stream visual areas [290, 291]. It is argued that if these pathways are the substrate in mediating blindsight, then the blindsight patient should not be able to independently mediate conscious visual awareness. Consistent with this idea, it is shown that visual perception, the conscious experience of seeing [292], is mediated by the ventral stream and not the dorsal stream [293].

However, in contrast, recent study demonstrates that the thalamic LGN plays an important role in V1-independent processing of visual information [294]. Before LGN inactivation, high-contrast stimuli presented to the lesion-affected visual field (scotoma) produced significant V1-independent fMRI activation in the extrastriate cortical areas V2, V3, V4, V5 (MT), FST and LIP, and the animals correctly located the stimuli in a detection task. However, following reversible inactivation of the LGN in the V1-lesioned hemisphere, fMRI responses and behavioral detection were abolished.

Concluding remarks

Current evidence in the literature suggests that both segregated and hierarchical processing of visual information along the visual pathways may not be as solid as it was previously thought to be. In the past decade, our view of the role of visual cortical areas in processing of visual signals has drastically changed. These areas are not merely involved in the perceptual aspect of the visual information but are also actively engaged in the processing of visual working and long-term memory. Conversely, areas such as the MTL that were thought to be exclusive for memory processing have also been shown to be involved in visual perception. It is suggested that visual cortical and MTL areas may work together to analyze the complexity of visual signals and to facilitate the synthesis of visual long-term memory.

In the literature, a controversy was found over the implication of the hippocampus in the processing of object recognition. It is now clear that the hippocampus is crucial for the processing of spatial memory but its participation in ORM that does not contain a spatial component remains controversial. Emerging evidence has begun to unfold the complex nature of hippocampal functions in recognition memory. However, a full understanding of the function of this structure is essential to resolve this conflict.

Personal view

Most of the controversies related to perception and memory are based on the assumption that there is only one route/pathway for processing these visual functions in the brain. However, known anatomical circuits within visual cortical and MTL areas can, in fact, give rise to more than one pathway. This theory could account for many of the controversial results reported in the literature.