Visual Working Memory and Delay Activity in Highly Selective Neurons in the Inferior Temporal Cortex

Visual working memory (VWM) is the kind of active memory needed to represent and monitor visual information in the short term and is often studied with delayed match-to-sample (DMS) tasks. A number of studies have shown that both the inferior temporal cortex (ITC) and the prefrontal cortex (PFC) are important for performing tasks that involve such cross-temporal contingencies. A neuronal mechanism that has been suggested to support this ability is delay activity, the sustained cell activation during the delay period after the presentation of a stimulus, until a subsequent stimulus or the execution of a response. Particular emphasis has been put on the PFC role in working memory, based on deficits in performance of delay tasks after PFC lesions, persistence of delay activity in some PFC cells even when distractors appear during the delay period, and elevated PET and fMRI BOLD signals from the PFC in humans performing delay tasks (Fuster, 2008). Delay activity in the PFC was discovered already in 1971 (Fuster and Alexander, 1971; Kubota and Niki, 1971) and has been extremely influential in theories of PFC function (e.g. Levy and Goldman-Rakic, 2000), but it has since been reported in most cortical areas, e.g. V1 and ITC (Fuster, 1990), parietal and somatosensory cortex (Koch and Fuster, 1989). According to the current framework neural activity in IT (or other unimodal sensory cortex) represents the stimulus that is no longer visible/available, and this activity is under top-down control exerted by PFC (Ranganath and D'Esposito, 2005). Because IT delay activity is thought to be vulnerable to distractors (Baylis and Rolls, 1987; Miller et al., 1993), it has been proposed that for the reinstatement of the specific memory in ITC, feedback signals from the PFC are necessary (Miller et al., 1996; Pasternak and Greenlee, 2005).

In a recent paper Woloszyn and Sheinberg (2009) report highly relevant findings for the robustness of delay activity in the ITC of monkeys performing a delay task. The task has four different conditions, designed to test the effects of distractors on the delay activity of the neurons during the memory delay. In this DMS task a sample image is presented that needs to be encoded and remembered during a 1000-ms delay, and is then compared with a test image, which can be a match or a non-match to the sample, requiring a different kind of response (right or left button press). The first condition contains no distractor, so the screen is blank until the test stimulus appears. In the second condition the sample is gradually occluded by a square of the same colour as the background. The third condition is a slight variation of the second with the square occluder being black and hence visible during occlusion and throughout the delay period. The fourth and most interesting condition uses a complex visual image, so a true distractor during the delay period, which is replaced by the test stimulus at the end of the delay.

As expected, task performance reflected the higher difficulty of the complex distractor condition with slower reaction times and higher error rates (Woloszyn and Sheinberg, 2009, Figure 1F,G). During recordings 79 neurons with very high visual selectivity were isolated, which enabled the authors to test the hypothesis that delay activity is robust in highly selective IT cells (Mikami, 1995). In the first three conditions there was selective delay activity (evaluated with ROC analysis) at the level of the cell population, while in the fourth and most relevant condition, although the selectivity was not significant at the level of the population, it was evident in 10 cells (out of 79 or 12.7%), which is comparable to the proportion reported earlier (19/171 or 11.11% cells, Mikami, 1995). Furthermore, a linear Support Vector Machine classification analysis showed that combining the selective activity of a small number of cells (64) yields significant classification for effective vs. ineffective stimuli even for the delay activity of the complex distractor trials, with the reliability of the classification improving towards the end of the delay period (see Woloszyn and Sheinberg, 2009, Figure 5). It is important to note that the classification analysis provides supportive evidence for the information present at the population level, but the delay activity surviving the distractor presentation is already visible in the spike density functions of the cells (see Woloszyn and Sheinberg, 2009, Figure 3A). The finding that delay activity in IT can be robust and survive the presentation of distractors confirms a related finding reported by Takeda et al. (2005). In that study the monkeys performed a paired-associate task and recordings from both IT and PRh cortex showed that cells with target-specific delay activity can maintain their selectivity during the delay period after the presentation of two distractors.

The prevailing view is that robust delay activity in PFC is the main mechanism for information maintenance and processing that is fed back to IT (or other posterior cortical areas). Computational models of working memory primarily focus on PFC and have used extensively the delay activity as part of attractor or feedback networks (e.g. Wang, 2001), more recently enriching the models with distractors (Macoveanu et al., 2007), and dopamine modulation of the interaction between PFC and the basal ganglia (Gruber et al., 2006; Chumbley et al., 2008).

The accumulating evidence that IT delay activity can also be robust to the presentation of distractors, as well as that PFC delay activity can become weak and lose its selectivity following distractors (Warden and Miller, 2007; Kusunoki et al., 2009), call for a re-evaluation of the emphasis placed on PFC delay activity as the main mechanism supporting VWM. On the one hand, recent evidence suggests that sensory areas also have selective delay activity, so they are involved both in perception and memory (Pasternak and Greenlee, 2005). On the other hand, it is not clear if PFC delay activity is a necessary and adequate mechanism for VWM. Several studies have successfully explored alternative interpretations for the role of delay activity in the PFC, pointing to attentional control, stimulus and action selection (see Lebedev et al., 2004 and Woloszyn and Sheinberg, 2009 for relevant studies and discussion). Recent work proposes that VWM can be achieved with alternative mechanisms that do not rely on delay activity (Sugase-Miyamoto et al., 2008). According to this model IT neurons behave like match filters that compare the total activity elicited by the sample and test stimuli, and decision on the match is made when a certain activity threshold is exceeded. The memory trace is stored in the synaptic weights of fast adapting synapses and is independent of the presence of delay activity. The match filter relies on standard stimulus selectivity, and the large difference of activity for effective and ineffective stimuli (see Woloszyn and Sheinberg, 2009, Figure 2) creates very different, spatially distributed signals in the synaptic weights. The idea that memory can be stored in synaptic weights has also been modelled in papers addressing the mechanisms of repetition suppression (Sohal and Hasselmo, 2000; Bogacz and Brown, 2003).The important point about the Sugase-Miyamoto et al. model is that it does not rely on the presence of delay activity, and achieves high matching performance by comparing activity levels during the presentation of the sample and test stimuli only.

These recent findings taken together pose questions to our interpretation of the role of delay activity, which we should aim to answer with well-designed experiments and an open mind.