Barlow versus Hebb: When is it time to abandon the notion of feature detectors and adopt the cell assembly as the unit of cognition?

Abstract

Research on how information is encoded by the brain is largely based on studies of feature detector properties of single neurons, but considerable new data shows that single neurons in many brain areas have mixed selectivity for multiple features and change their tuning properties across realistic information processing situations. Here I consider new approaches that explore cell assemblies as the units of information processing and how these approaches are revealing the structure and organization of neural representations in perception and cognition.

Introduction

Two major themes have guided research on how neuronal activity encodes experience. One theme, originating with Horace Barlow (1,2), proposes that each neuron detects a specific feature of an event, typically characterized by selective tuning within a single dimension of a particular perceptual, behavioral, or cognitive domain. The other theme, begun by Donald Hebb (3), proposes that the unit of perceptual, behavioral, and cognitive representation is the cell assembly, an interconnected network of neurons whose collective activity codes for all elements of a particular meaningful event.

These views differ on fundamental properties of neural representation. The feature detector view focuses on single neurons, whereas the cell assembly view focuses on population activity patterns. The feature detector view focuses on tuning within a single stimulus dimension whereas the cell assembly view characterizes the combination of all dimensions of an experience. And the feature detector view assumes the response properties of a neuron are stable across all situations whereas the cell assembly view allows that population activity can vary depending on context and be dynamic across an experience.

Here I will discuss how these themes involve different experimental approaches and lead to distinct visions about how brain circuits and systems represent events. These considerations will support the notion that, at least for higher order brain areas, neuronal activity patterns are inconsistent with the feature detector view, and conversely, analyses of neural populations reveal information about the organization of information by cell assemblies that go beyond what can be demonstrated by single cell analyses.

Feature detector neurons as the functional unit of information processing

A guiding principle for understanding the brain is the neuron doctrine, which states that the unit of anatomy and function of the nervous system is the single neuron (4, 5). Our conception of the neuron as a functional unit grew with early descriptions of receptive fields of sensory neurons and was solidified by Barlow’s characterization of frog retinal ganglion cells as “fly detectors” (for reviews, see 2,6). The idea that single neurons are tuned to a particular point within a single perceptual dimension gained ground as various feature detector properties were identified for neurons in the frog tectum (7) and in other sensory systems, and this progress reached maturity in Hubel & Wiesel’s (8) studies on neurons in the mammalian lateral geniculate nucleus and visual cortex that are tuned to specific location and form of a visual stimulus within the visual fields. While there are different views on the response properties of visual neurons (e.g., that they are filters for different spatial and temporal frequencies; 9), the idea of neurons as detectors of specific features of sensory and behavioral events remains prominent.

Furthermore, the additional findings of Hubel & Wiesel on more complex response properties of neurons in the visual cortical areas reinforced the notion that the output of feature detectors in earlier stages converge at successive stages of information processing to build a hierarchy of feature coding, ultimately leading to what Konorski called “gnostic units” and Lettvin called “grandmother cells” (see 10).

Despite these successes, the feature detector approach has been met with two general challenges. First, feature coding encourages the experimenter to narrow the presented events to the simplest one that evokes a robust neural response, and to ignore how neurons might respond to the more complex stimuli that characterize the real world. However, the responses of these neurons to natural scenes is far more complex and an area of considerable uncertainty and current research (e.g., 12–14).

Second, identification of feature detector properties in higher cortical areas has had mixed success. Thus, neurons in the inferotemporal cortex (10), the prefrontal cortex (see 14), and the hippocampus (15, 16) have a breadth and flexibility of tuning that challenge the idea of feature detection by neurons in the very areas we most associate with the ultimate stages of perceptual, cognitive, and memory processing.

Cell assemblies as units of representation

The limitations of the feature detector hypothesis suggest that, under naturalistic conditions, neurons may not have a single target feature but may instead exhibit graded or nonlinear selectivity to multiple variations of a feature, multiple features within a domain, and even multiple domains of features. The theme that incorporates these qualities began with Hebb’s (3) notion of the cell assembly as a network of connected neurons that represent the composite features of a concept or event. Early on the cell assembly idea served primarily as a conceptual tool, but with the advent of techniques for simultaneous recording of a population of neighboring neurons that might compose a subset of a cell assembly, has received greater attention in experimental studies that reveal multidimensional representations of events from a combination of distributed responses, co-dependencies, and dynamic interactions (17).

It is important to note that a cell assembly is theoretical entity of a functionally interconnected network whereas population coding is derived from an analysis of conjoint activity of many neighboring neurons recorded in simultaneously or under identical conditions across recordings. A population code may well include neurons from different functional cell assemblies, potentially leading to confusing or misleading interpretations. Nevertheless, ensembles of neighboring neurons are largely interconnected and population analyses have yielded emergent characteristics that may be generated at the level of cell assemblies (6). Here I will focus on selected studies that have explored population coding in the three high order areas introduced above, the inferotemporal cortex, the prefrontal cortex, and the hippocampus in studies that monitor neural responses to a variety of events that include multiple dimensions and different cognitive demands.

Inferotemporal cortex

The inferior temporal (IT) cortex is well recognized as the final stage of purely visual processing where neurons are maximally driven by highly specific complex visual stimuli, including faces, and have become tempting exemplars of “grandmother cells” (e.g., 18,19). However, the same neurons typically also respond to diverse stimuli, suggesting that the high specificity with which we identify particular objects is derived only at the population level. Using a multidimensional scaling and principal components approaches, studies have shown that IT neuronal ensembles can identify individuals and relate them to one another (20), distinguish human identity and monkey facial expressions (21), and categorize objects (22).

A further study on categorization of natural object stimuli by IT populations revealed that IT cell assemblies distinguish intuitive object categories, including faces, hands, bodies, from inanimate objects, and an agglomerative hierarchical clustering analysis of response similarities revealed the higher order organization of these representations (23; Figure 1A). Furthermore, a direct comparison of these neural population responses with those from pseudo-population responses composed of high-resolution fMRI mutli-pixel patterns generated when humans viewed largely the same set of object stimuli revealed a highly similar pattern of organization of the representation of object categories (25), and further analysis of multivoxel patterns revealed a categorization of animals that paralleled the prototypical phylogenetic hierarchy (24). These observations show that cell assemblies can both identify specific events and organize knowledge about related events within a meaningful cognitive domain (see also 26,27).

Figure 1.

Hierarchical organization of knowledge in different brain areas: A. for visual object catagories in the inferotemporal cortex of monkeys (23), B. for reward outcomes, objects, their positions, and even versus odd numbered trials in orbitofrontal cortex in rats (43), and C. for contexts, positions, reward outcomes, and object identities in the dorsal hippocampus (56).

Prefrontal cortex

The prefrontal cortex (PFC) is generally thought to play an essential role in “executive function” via the control over many aspects of cognition, including strategy selection, working memory, decision making, and perception (28), and PFC performs its role via rapid formation of rule-based representations (14,29). This general function may be supported by adaptively mapping any task space onto a cell assembly that links current stimuli with specific goals to generate decisions and actions directed at specific outcomes and dynamics by which firing patterns change when rules and contingencies are altered (14, 30–40).

These properties of PFC neurons seem incompatible with the idea that they could be dedicated to coding single features of specific stimuli or behavioral events. Instead, new analyses reveal that these coding properties emerge primarily or exclusively in population coding (41). For example, one study focused on the activity patterns of PFC neurons in monkeys performing different tasks that demanded they remember their identity and order of presentation (42). Single neurons showed complex patterns of responses that typically reflected multiple dimensions of the tasks including stimulus identity, study or test phase of a trial, and recognition or recall memory demands. The firing patterns of single neurons did not reflect a simple linear summation of responses to specific dimensions but rather coded by mixed selectivity, and population analyses predicted accurate performance on the tasks much better than coding by single neurons.

Another study analyzed population activity of neurons in the orbitofrontal cortex (OFC) of rats performing a task where they learned opposite reward associations of the same objects depending on which of two spatial contexts in which the objects were presented (43). As observed in the dorsolateral PFC of monkeys, single neurons in the rodent OFC also showed nonlinear mixed selectivity that involved multiple task dimensions. Many of these OFC neurons distinguished combinations of objects and the context in which they were sampled that composed a common reward value of these events. Population analyses using the same similarity measures employed in the above described experiments on IT in monkeys and humans revealed distinct organizations that separated events differing in reward association and a systematic organization of the representations of objects in the contexts and positions where they occurred (Figure 1B). This hierarchical coding scheme is consistent with a mapping of contextual cues and specific stimuli to behavioral responses and associated reward values (30). Notably, the population code evolved during the course of a trial to sequentially represent the spatial context of the trial, then the position of an object to be sampled, then object identity and reward value. Combining the findings across these studies, despite major differences in the connections, architectures, and specific functions assigned to different PFC areas and species, there is remarkable convergence both in the generality of high dimensional coding and the adaptability of PFC networks revealed by population analyses.

Hippocampus

Many studies have reported that the firing patterns of neurons in the hippocampus and adjacent areas encode the location of an animal in allocentric space and other spatial features, including head direction, borders, and speed (15,44,45). Many view these cell types as the building blocks of spatial cognition, much as the visual feature detectors are viewed as building blocks for visual perception (15,45).

On the other hand, in more complex behavioral tasks, hippocampal neurons adapt to diverse dimensions of experience. For example, when environmental features are changed, hippocampal place cell populations partially “remap”, thus are mixed in selectivity for some features of the original environment combined with coding of new features (reviewed in 16). In a classical eye-blink conditioning paradigm where animals are restrained, hippocampal neurons are tuned to the profile of the conditioned response (46). In other studies where space is held constant and specific task demands are applied, hippocampal neurons can map a perceptual “space” (47) or a sequence of moments in a fixed temporal period (48,49,50). These findings indicate that hippocampal neurons adapt to represent relevant dimensions of a broad range of experiences.

Furthermore, in many situations hippocampal neurons show mixed selectivity for multiple dimensions of experience in addition to spatial position. For example, when animals run in opposite directions on a track, hippocampal neuronal activity is strongly modulated by direction of movement (51,52). In several complex behavioral tasks, hippocampal neuronal responses can be strongly dependent on categorical dimensions including specific objects and various associative and reward contingencies (53,54). Thus, as observed in PFC neurons, hippocampal neurons exhibit strong nonlinear high-dimensional coding.

An early approach to describing the population activity of hippocampal neurons focused on recordings hippocampal neuronal ensembles as rats performed a spatial delayed non-match to sample task (55). Individual neurons fired associated with combinations of spatial location, sample or test epoch, and match and non-match responses, and different nonlinear responses were observed to various combinations of these dimensions. A discriminant analysis revealed significant canonical roots for each of the critical task events (see also 56,57).

More recently, Mckenzie et al. (58) recorded CA1 and CA3 neuronal ensembles in rats performing the context-guided memory task introduced above with regard to OFC neuronal population coding, and the firing patterns of hippocampal neuronal populations were analyzed using the same similarity analysis as described above for OFC. By contrast to the findings on OFC, in the hippocampus, events that occurred in different contexts are widely separated in representational space by anti-correlated population activities (Figure 1C). Within each context-based network representation, hippocampal populations separated events by the positions where they occurred in each context, then events were separated according to the associated reward outcome, and finally events were separated by object identity. Also, the hippocampal population representation evolved during the course of each trial, such that, initially the code represented the spatial context of the trial, then the position of the object to be sampled, then the object identity, and finally the reward association (see also 59,60).

Additional studies on the closely associated areas of the lateral entorhinal cortex (LEC) and medial entorhinal cortex (MEC) offer further insights into the nature of population coding within the hippocampal system (61). It is generally accepted that the LEC is specialized for processing information about objects and events and the MEC is specialized for processing spatial information. However, contrary to this view, in animals performing a task that associates objects with the places they are rewarded, single neurons in LEC and MEC (including grid and border cells of MEC), demonstrate strong nonlinear high-dimensional selectivity for object and spatial dimensions of the task. Nevertheless, population analyses revealed a key distinction in the organization of information in these areas. As in the hippocampus, at the top of the hierarchy population representations of events that occur in different contexts are anti-correlated in both LEC and MEC. However, within each context-based network, LEC populations next distinguish events by the object involved, and within each object category, the positions where the objects occur are separated. In striking contrast, MEC populations next distinguish positions within each context, and separate object representations within each position category. Thus, although LEC and MEC exhibit similarity in mixed selectivity at the single neuron level, the population analysis reveals that information is organized in distinct but complementary ways in each area. Given that these hippocampal and entorhinal areas are highly interconnected, it is reasonable to expect that neural activity in all of these areas incorporate the same set of dimensions. These findings suggest that the key distinction between the areas is how cell assemblies organize the information more so than which dimensions are included.

Finally, it is important to acknowledge that the analyses described here assume that the task dimensions are hierarchically organized, and other approaches based on different assumptions could yield alternative organizational patterns. Nevertheless, the observations described here illustrate the potential for population analyses in discovery of al properties of population coding that go beyond the properties identified in single neuron analyses.

When are feature detectors sufficient, and when should we turn to cell assemblies?

This brief and selective survey of research on population coding suggests three basic properties of cell assembly representations that emerge only at the population level. First, individual neurons in cell assemblies exhibit graded or nonlinear responses to multiple closely related events. Second, individual elements of cell assemblies exhibit mixed selectivity for distinct dimensions within and across domains of information and are adaptable in their representations to task structure and cognitive demands. Third, population analyses reveal the organization of information in cell assemblies, both with respect to the structure of knowledge within the cell assembly and with respect to the temporal sequence of the dimensions represented.

Population analyses on IT networks have revealed that object identification is accomplished by large cell assemblies composed of graded tuning curves of individual neurons. The cell assembly perspective becomes essential to revealing how large neural populations can sort object stimuli into meaningful categories and knowledge organizations, the very stuff of visual cognition we seek to understand.

In many studies on primates and rodents, prefrontal neurons are activated during cognitive performance, but no specific consistent features of cognition have been assigned to the response profiles of single prefrontal neurons. Rather, the response patterns of single PFC neurons are characterized as bound to multiple task features by nonlinear mixed selectivity that is adapted to a very broad range of cognitive demands in virtually any ongoing task, and analyses of prefrontal neural populations offer insights into the nature and organization of task features by cell assemblies in this area.

Finally, when animals are performing complex learning tasks that are viewed as supported by the hippocampal system, hippocampal neurons take on characteristics of elements within cell assemblies. They exhibit nonlinear tuning to multiple task dimensions and the most comprehensive understanding of the contribution of hippocampal networks comes from population analyses that highlight both the adaptability of hippocampal networks and their capacity to link distinct events into an organization by task dimensions.

What is the take home message from these observations? In the 1950s when the primary tool of neurophysiologists was the single sharp electrode and experiments focused on early sensory processing, the field was dominated by search for unidimensional features that could be coded in single neurons. This work still proceeds and is, of course, a model for studying circuit properties that underlie sensory driven responses. However, as the stimuli become complex (e.g., visual scenes), the explanatory power of feature detection diminishes, even at early stages of sensory processing.

Early approaches to cell assemblies involved theoretical analyses using artificial neural networks to examine distributed codes and complex, non-linear “hidden” representations (e.g., 61,62). With the advent of methods for recording large numbers of neurons simultaneously, the ability to explore neural network activity became a reality and, as the experimental focus turned to behavior and cognition, the notion of the cell assembly is reaching maturity. This perspective forces us to re-think not only how neural networks within brain areas represent information and to think about how anatomically separated but interconnected networks interact within functional systems. Given the high interconnectivity of high order brain areas, it may have been simplistic to expect individual neurons to code just one feature of experience, and to be fixed to one domain of information. The new directions in cell assembly analyses are pointing to how multiple dimensions and domains of information can be incorporated and organized in areas that support behavior and cognition.

Highlights.

In higher order areas and complex tasks, neurons exhibit mixed selectivity and dependence on cognitive demands

• These properties are observed in inferotemporal cortex, prefrontal cortex, and the hippocampus

• New population analyses reveal how cell assemblies in these brain areas organize task relevant knowledge