The challenge of understanding the brain: where we stand in 2015

Abstract

Starting with the work of Cajal more than 100 years ago, neuroscience has sought to understand how the cells of the brain give rise to cognitive functions. How far has neuroscience progressed in this endeavor? This Perspective assesses progress in elucidating five basic brain processes: visual recognition, long-term memory, short-term memory, action selection, and motor control. Each of these processes entails several levels of analysis: the behavioral properties, the underlying computational algorithm, and the cellular/network mechanisms that implement that algorithm. At this juncture, while many questions remain unanswered, achievements in several areas of research have made it possible to relate specific properties of brain networks to cognitive functions. What has been learned reveals, at least in rough outline, how cognitive processes can be an emergent property of neurons and their connections.

The human brain, with its ~80 billion neurons, is one of the most complex systems on Earth. The field of neuroscience has striven to elucidate brain function for more than 100 years. My objective in this Perspective is to offer an overview of how far neuroscience has progressed in the endeavor to understand the brain.

At the outset, it is important to define what “understanding” the brain entails. Information about the brain has been obtained by many approaches, ranging from the cellular to the cognitive, and this information must be integrated. A useful framework for such integration was developed by David Marr, whose work in the 1970s pioneered efforts to understand specific brain networks (Marr, 2010). Marr argued that understanding a brain process requires efforts at three levels. First, the functional properties of the process must be defined and behaviorally characterized. Next, the computational algorithm that performs that process must be identified. And finally, how neurons and their network connections lead to the execution of that algorithm must be determined.

Although the subject of this Perspective is the understanding of the vertebrate brain, consideration of a simple network found in the eye of an invertebrate, the horseshoe crab, provides a helpful vehicle to illustrate how a brain process can be understood using Marr’s framework. The horseshoe crab eye has an array of cellular units, each of which is excited by the light that impinges on it (reviewed in (Ratliff, 1972)). It was found that this array processes the image in a way that enhances regions of contrast (Marr’s first level). This was shown by the fact that excitation of an illuminated cell was enhanced if nearby cells were kept in the dark, thereby generating contrast. Further analysis indicated that this interaction could be accounted for by a simple algorithm in which excited cells reduced the response of nearby cells (Marr’s second level). To achieve Marr’s third level, physiological and anatomical experiments on the network connections in the eye showed that cells reduce the response in nearby cells as a result of monosynaptic inhibitory connections, a network architecture called lateral inhibition (Fig.1A; for another example of an “understood” process in invertebrates, see Fig.1B).

Fig.1. Examples of simple neural networks that perform useful computations.

A. Lateral inhibition in the horseshoe crab eye. (Top) If only the middle cell is illuminated (yellow arrow), it fires strongly. (Bottom) If all cells are illuminated, the middle cell fires less because it receives inhibition from neighbors. The network thus produces contrast detection (Ratliff, 1972). B. A half-center oscillator that mediates rhythmic and alternating (e.g., walking) behaviors in invertebrates. A simplified diagram with only two cells. Each cell has intrinsic slow conductances that produce spontaneous bursts of action potentials. Because these cells mutually inhibit each other, the cells burst in alternation (Sharp et al., 1996). C. Autoassociative memory. Cells in such networks have axons that make excitatory (recurrent) connections with other cells in the network. In the example discussed in the text, an associative memory occurs by the strengthening of the synapses that link cells 1 and 4 (dark blue). Inhibitory cells are not shown. (McNaughton and Morris, 1987). D. Bump integrator. Each pyramidal cell excites nearby cells and inhibits more distant ones (only one set of connections shown). The excitation of nearby cells leads to a region of high activity (a “bump”). Without external inputs, the bump has a relatively stable position. External input activates asymmetrical connections (not shown) that move the bump along the length of the network. Because the distance moved is proportional to external input, this network is capable of integration (Zhang, 1996). For use of such bumps during decisions, see (Wang, 2012). E. Cumulative integrator. All cells receive the same external input pulse (not shown), but only cell 1 is initially bistable (e.g., able to produce persistent activity in response to a brief excitatory input). After the first such input, the persistent activity in cell 1is communicated to cell 2 by lateral connections, making cell 2 bistable (by activating its NMDA conductance); cell 2 can therefore become persistently active when excited by a second input pulse. Integration occurs because the more input pulses, the larger the number of persistently active cells (Koulakov et al., 2002). (See (Fisher et al., 2013; Joshua and Lisberger, 2014) for how integration may occur in the oculomotor system). F. Winner-take-all by gamma oscillations. Pyramidal cells excite an interneuron network (only one interneuron is shown), which then provides global feedback inhibition to pyramidal cells. As inhibition declines, the most excited cells will fire (the winners) and again produce feedback inhibition (de Almeida et al., 2009). This repeating inhibition generates gamma frequency oscillations of the network. Other functions of gamma are binding the different cells that represent an item into an ensemble (Engel and Singer, 2001) and formatting multi-item messages by creating a firing pause between different items (Fig.3C) (Lisman and Jensen, 2013).

Before proceeding with a discussion of the vertebrate brain, three prefatory comments about the scope and organization of this Perspective are in order. First, due to space limitations, I have selected for review only a subset of the important brain functions, namely visual recognition, memory (short-term and long-term), action selection, and motor control. These processes together are, in principle, sufficient to account for a simple visually evoked behavior (e.g., a rat conditioned to make a right turn in response to a particular visual stimulus). In this Perspective, I will begin with vision and end with motor control because this order conveys a sense of the flow of events in the brain that underlies a simple behavior. Second, Marr’s third level involves a description of how neuronal signals (i.e., action potentials and excitatory/inhibitory synaptic potentials) are generated as well as how they propagate through a network. Although much progress has been made in understanding signal generation, I will focus almost exclusively on network processes, again solely due to space limitations. And finally, in assessing our understanding of a brain function, it is important to discuss not only what is known, but also what is not known. For this reason I begin each section with a functional description of a process (Marr’s first level). This sets the stage for assessment of the extent to which neuroscience has achieved a satisfactory explanation of this process at Marr’s second and third levels.

Marr’s predicate, and indeed that of all modern neuroscience, is that cognitively important functions can be explained as an emergent property of neurons and their network connections. When Marr started his exploration, there were virtually no experimental data directly linking the network and cognitive levels. Over the ensuing decades, the situation has dramatically changed. The firing properties of neurons have been measured during many types of behavioral tasks. Advances in computational neuroscience have provided insight into how the particular architecture of a network (the pattern of excitatory and inhibitory connections) can lead to particular computations and observed firing patterns. More recently, the development of optogenetics (reviewed in (Fenno et al., 2011)) has made it possible to mimic or block neural activity, thereby enabling strong tests of hypotheses. Together, these methods have yielded a great deal of data and have led to data-driven models of how specific networks perform cognitively relevant functions. My hope is that this Perspective will convey an appreciation of this progress while at the same time indicating where major gaps in our knowledge remain.

Visual Recognition

Let us begin with a discussion of visual recognition, the most extensively studied sensory process. The visual system is fast, producing recognition in less than 200 milliseconds (VanRullen and Thorpe, 2001). This is quite remarkable given the complexity of the task. It is sometimes said that we never see the same thing twice; an object may be in a different position, may have a different size (because its distance from the viewer is different), or may have a different rotation (the TV image of your retina is rotated 90 degrees when you watch while lying on your side). To achieve recognition, the brain must somehow convert these varying images into a single invariant form that can then be compared to a stored pattern. A further remarkable and counterintuitive aspect of recognition is that visual scenes are not processed as a whole. Instead, a movable window of attention serially samples different subregions. Indeed, if no attention is turned to an object, the object is not perceived, even when in full view, a phenomenon called inattentional blindness (Simons and Chabris, 1999). Some movements of attention can be due to changes in the position of the eye but can also be produced covertly while the eye is stable. Such covert movements of attention can occur ~30 times per second, as first established psychophysically (Treisman and Gelade, 1980) and then physiologically (Buschman and Miller, 2009). Explaining recognition thus requires elucidation of the mechanisms of attention and, in particular, how serially sampled information is retained and combined. Perhaps the most sophisticated aspect of recognition is its use of context. As illustrated in Fig.2, recognition of a letter depends not just on its features, but also on other letters in the word and even other words in the sentence.

Fig.2. Role of context in recognition.

This figure demonstrates that the recognition of letters (H/B/A) is not only dependent on the features of that letter, but also on other letters in the word and even other words in the sentence.

A starting point for any model of visual recognition is the extensive literature on the response properties of cells in the vertebrate visual system. The flow of visual information starts in the retina, proceeds to the thalamus, and from there to a hierarchy of cortical areas (Fig.3).The output cells of the retina (ganglion cells) respond best to a small spot of light at a particular position and less to global illumination (much like the cells of the invertebrate eye discussed earlier). In contrast, cells in V1, the first region of the cortical hierarchy, have a different representation: so called simple cells respond best to a bar having a particular orientation and position (Hubel and Wiesel, 1962). A second set of cells, called complex cells, generalize the simple cell response: they respond best to a bar of particular orientation but, importantly (see below), tolerate substantial variation in position. The output of V1 is processed (bottom-up) by a hierarchy of intermediate and higher cortical areas (solid green areas in Fig.3), first in the occipital lobe (e.g. V2, V3) and then in the parietal and temporal lobes (Felleman and Van Essen, 1991).

Fig.3. Cortical and subcortical brain regions.

A. Cytoarchitectonic areas (outlined in black) of the macaque cortex have been identified by the number of cell layers, their thickness, and cell density. The cortical surface is viewed as a flatmap. Such a map is necessary to see all of the regions of the cortical surface, some of which are normally hidden within folds. Different lobes are outlined in different colors. The position of lobes is indicated on the lateral and medial views of the folded brain (bottom). Areas are grouped according to the theory of dual origin of cerebral cortex (Pandya and Yeterian, 1985): functional hierarchies occur in pairs (trends), one derived evolutionarily from the hippocampus (archi; saturated colors) and the other from the piriform cortex (paleo; pastel colors). The dividing line between the trends is marked by a white dashed line. The paired trends are color-coded according to the key at the bottom right. The most recent additions to each trend (neocortex) have six layers, whereas the most primitive areas that form the origin of trends (hippocampus and piriform cortex) have three. In the visual system, the paleo trend (ventral) is specialized for determining WHAT is present; the archi trend (dorsal) is specialized for determining WHERE information. In the auditory system, the paleo trend deals with sound identification (what), the archi trend with sound localization (where). In the motor and somatosensory systems, the paleo trend preferentially deals with the face and neck, the archi trend with the trunk and limbs (Kuypers, 1982). In the prefrontal system, the paleo trend deals with emotions, the archi trend with executive control. The dashed line is a thin layer of cells that connect the ventral hippocampus to the dorsal parts of the limbic system through a region not included in the flatmap. The flatmap was adapted from Fig.1 of (Markov et al., 2014) by Dr. Deepak Pandya. B. Flow of information through major brain regions. Uncolored shapes are subcortical regions.

Recordings from high regions in this hierarchy (perirhinal and entorhinal cortex and hippocampus; Fig.3) show that the invariance problem is indeed solved by the time that signals reach this level. There are cells in these regions that respond to a particular face irrespective of its size, position, and rotation (Perrett et al., 1982; Quiroga et al., 2008). Thus, a key question is the algorithm by which the output of V1 is processed by intermediate regions of the hierarchy to achieve this invariance.

One proposed algorithm (Riesenhuber and Poggio, 2002) posits that the operations of simple and complex cells of V1 are paradigmatic of operations that are repeated at various stages of the visual hierarchy to produce a family of templates that can be matched to incoming information. For instance, in V4, an intermediate region in the hierarchy, a new representation is developed in which cells represent shapes composed of several linear segments. In this way, the feature conjunctions (like a partially unfolded paper clip) form a template to which incoming information can subsequently be matched (Pasupathy and Connor, 2002). This is generalized by a second set of V4 cells making the response less dependent on position (Cadieu et al., 2007). Analogous operations at still higher levels make it possible for cells to become selective to yet more complex stimuli and categories and to do so with substantial invariance. The (Riesenhuber and Poggio, 2002) algorithm accounts for many physiological findings (but not rotation invariance) (for a review, see (DiCarlo et al., 2012)).

An alternative to templates is an algorithm based on spatial frequency, as in Fourier analysis (Cavanagh, 1985; Sountsov et al., 2011). Three findings support this type of algorithm: 1) Vision, as measured psychophysically, acts as if there are independent pathways for different spatial frequencies (Blakemore and Campbell, 1969); 2) V1 cells have spatial frequency tuning (i.e., sensitivity to the specific distance between edges); 3) Visual space is mapped over the entire V1 surface; however, within subregions, there is a detailed map in which log of spatial frequency is mapped orthogonally to orientation (Nauhaus et al., 2012). Simulations show that if such a mapping procedure was reapplied to the V1 output, the resulting activity map would be object-specific and invariant to the object’s size, position, and rotation (Sountsov et al., 2011), thus suggesting how the brain might produce an analytical solution to the invariance problem. Further experiments will be necessary to determine how invariance is actually achieved.

Other experiments have sought to determine whether there is a particular brain region where recognition actually occurs. Familiarity (this is feeling that you’ve seen somebody before but can’t remember when) can be measured in rodents because when placed near both a familiar object and a novel object, rodents preferentially explore the novel one. Experiments show that this preference depends on the perirhinal cortex, a region in the medial temporal lobe (Fig.3A) (reviewed in (Eichenbaum et al., 2007)). Cells in this region respond differentially to familiar vs. novel items, and recognition fails if this region is disabled (Brown et al., 2010; Tang et al., 1997). A yet higher structure, the hippocampus, is not required, as demonstrated by the fact that the patient HM retained this ability after removal of his hippocampi (Corkin, 2013).

As noted earlier, recognition involves serial movements of attention to different parts of an object. This implies that cortex must retain information about a sampled region while other regions are processed. Insight to this and other aspects of visual recognition has been achieved by recording from neurons during recognition tasks (Desimone and Duncan, 1995; Hanes and Schall, 1996; Shadlen and Newsome, 1996). Notably, studies of cells in LIP, a parietal lobe region, showed that the cortical network has integrator properties (Huk and Shadlen, 2005). Integrators provide a solution to the sampling problem because they sustain their activity after input ceases (for a model of how networks can integrate, see Fig.1E). The data indicates that there are different integrators that represent different possible items (Churchland et al., 2008) and that their firing rate is proportional to the probability of that item (a probabilistic population code) (Beck et al., 2008; Yang and Shadlen, 2007). Consider now an integrator that represents the item being presented; as information accumulates, the firing rate increases gradually---the moment of recognition occurs when a criterion rate is achieved (for a review, see (Shadlen and Kiani, 2013); see also section on consciousness). There has been progress in understanding the brain regions that control attention and the role of brain oscillations in the process (Baldauf and Desimone, 2014; Brooks et al., 2014; Buschman and Miller, 2009), but no clear mechanistic view of how the attentional window is selected has emerged.

Let us now turn to the most sophisticated aspect of cortical recognition, its dependence on context (Fig.2). The process of reading the sentence in Fig. 2 appears to involve two steps. The first is the determination that the context could be baseball; the second is the determination that, given this assumption, all of the ambiguous letters can be interpreted in such a way that the words are relevant to baseball. Such context-dependent interpretation of letters is likely to depend on a fundamental architectural design principle of the cortical hierarchy: that sequential levels are not only connected bottom-up, but also top-down (reviewed in (Gilbert and Li, 2013)). It is the top-down connections that may allow context to bias the interpretation of low-level features (Fenske et al., 2006; McClelland, 1986). In the example of Fig.2, high-level cortex may generate multiple hypotheses about the context, the most probable of which is a baseball. Top-down connections could then positively bias letter features contained within baseball words (predictive coding), thereby leading to context-dependent interpretation of ambiguous letters.

Evidence that the brain actually generates predictions about the features of incoming sensory items comes from the study of mismatch negativity. This EEG signal is generated by auditory cortex and occurs when an incoming signal violates expectations based on the regularity (e.g., in tone or interval) of previous auditory stimuli (Lieder et al., 2013). Another example of prediction arises when brain motor regions initiate an action; they send a corollary discharge to sensory regions predicting the sensory consequences of one’s own upcoming action. This allows these sensations to be minimized (the EEG evoked potential is smaller when you initiate the sound than when you listen to the same sound in recorded form). Recent experiments indicate that there are deficits of corollary discharge in schizophrenia and that these deficits may produce the loss of agency (sense of self) that occurs in this disease (Ford et al., 2014; Shergill et al., 2014). For a review or predictive coding, see (Summerfield and de Lange, 2014).

Give the importance of bottom-up flow of sensory information and the top-down flow of predictions, it is clear that the still mysterious algorithm of cortical computation will involve bidirectional information flow. Theoretical work is beginning to shed light on what that algorithm might be and what it can accomplish. One study showed how such an algorithm can identify the most informative regions of an image (Feldman and Friston, 2010). Another study simulated bidirectional information flow during recognition and successfully accounted for the experimental finding that recognition time depends logarithmically on the number of contextually possible items (Graboi and Lisman, 2003).

Considering these findings in the context of Marr’s framework, it is clear that our understanding of the visual recognition process is at a fairly early stage. There are now proposed algorithms (Marr’s second level) for several aspects of vision, but much further work is needed. Although a great deal is known about the response properties of cells in the visual system, there is little information about the particular cortical cell types that generate these responses (Marr’s third level), and models that assign function to particular cortical layers are still in early stages (Bastos et al., 2012; Grossberg and Pearson, 2008). I will return later to describe new approaches that may be powerful enough to finally solve the problem of cortex. That said, one success of the field deserves special emphasis: the demonstration (Perrett et al., 1982; Quiroga et al., 2008) that there are neurons in high-level cortex that represent particular items (e.g., a particular person) in an invariant way. These cells fire when a particular item is seen, imagined, or remembered (Gelbard-Sagiv et al., 2008; Kreiman et al., 2000). Such neurons can operate within memory circuits to form associations (e.g., connecting a face to a name) and within action selection circuits to connect a visual stimulus to an action, as described in subsequent parts of this Perspective.

Long-Term Episodic Memory

You may meet someone and know they are familiar; then a new fact may suddenly allow you to remember their name and the sequence of events when you met. Such recollection is termed episodic memory. Extensive studies of hippocampal lesions in humans and animals (reviewed in (Eichenbaum et al., 2007; Squire and Wixted, 2011)) have made it clear that the hippocampus is necessary for episodic memory (Marr’s first level). In the sections below, I will first describe the way memory recall is organized and the neural code that is utilized (Marr’s second level). I will then turn to what has been learned about the underlying network mechanisms (Marr’s third level).

A major advance in the study of memory has been the ability to observe memory sequences being replayed in the rodent hippocampus. The breakthrough that made this possible was the discovery of place cells (O’Keefe, 1976). These fire whenever a rat enters a small subregion of the environment (the place field). Different cells have place fields in different locations, thus collectively mapping the environment. Once the location of a cell’s place field is determined, the activity of that cell can be used to determine when the memory of that location is being replayed, even when a rat is elsewhere.

Experiments show the hippocampus produces two different forms of replay. One type is offline during rest or sleep and can be identified by sharp-wave ripples (SWR), an extracellular (field potential) signal that lasts for ~100 msec and reflects the average signal in nearby cells (Fig.4A). A key observation is that during a SWR, place cells often fire in the same sequence as they did during the actual experience of a path (Fig.4A), albeit sped up (~10x time compressed) (Nadasdy et al., 1999; Wikenheiser and Redish, 2013; Wilson and McNaughton, 1994). Such sweeps are interpreted to mean that what is being replayed is the memory of having traversed the path. Consistent with this interpretation, such sweeps do not occur if synaptic plasticity is blocked during the actual experience of the path (Dupret et al., 2010; Silva et al., 2012). A further major advance was the demonstration that such replay is functionally important. This was shown by evoking inhibition whenever the beginning of a SWR was detected, thereby blocking the remaining replay. It was found that this procedure compromised subsequent memory-guided behavior (Girardeau et al., 2009; Jadhav et al., 2012). The replay that occurs during SWRs is thought to have two functions. One is synaptic consolidation, a process that stabilizes the synaptic changes in the hippocampus that occurred during learning (Dudai, 2012). The other is systems consolidation. According to one view of this process, what the hippocampus stores is an index that points to more detailed memories in cortex (Teyler and Rudy, 2007). Systems consolidation strengthens and reorganizes these cortical representations (Diekelmann and Born, 2010; McClelland et al., 1995) (over many sleep periods), making it eventually possible to recall substantial information without the hippocampus (Tse et al., 2007).

Fig.4. Network and coding mechanisms of the hippocampus.

A. Memory replay during a single sharp-wave ripple (SWR) in the field potential (bottom trace). Top plot shows the replay: action potentials (dots) in nine cells fire in the same order as when the animal traversed the path. Cell number is ordered according the position of the cells’ place fields (Fig.5 of (Wikenheiser and Redish, 2013)). B. Theta and gamma oscillations in the extracellular field potential (top) and intracellular voltage (bottom) (Figure 4C of (Penttonen et al., 1998)). C. The theta/gamma code: an item is represented by a subset of cells (black) in the network (oval) that fire during a gamma cycle; different items are represented in different gamma cycles of a theta cycle. This raises the question of how a continuous world is discretized into items. According to one proposal, items become defined at lower levels of the hierarchy by a combination of processes of perceptual learning, driven by input statistics, and behavioral learning dominated by the individual ability of stimuli to predict reward/punishment (Verschure et al., 2003). D. Wiring diagram of excitatory connections in the hippocampus. Sensory information arrives from the lateral entorhinal and spatial information from the medial entorhinal cortex. Convergence of these inputs onto dentate granules cells produces a conjunctive representation (rate remapping) (Lu et al., 2013; Renno-Costa et al., 2010). Information is passed from the dentate to CA3, which has the abundant recurrent connections characteristic of attractor networks. The attractor produces pattern completion and error correction (Hasselmo et al., 1995; Treves and Rolls, 1992). Accurate recall of sequences (as in A) requires a chaining process in which one memory in the sequence triggers the next and a process of error correction after each chaining step (Sompolinsky and Kanter, 1986). According to one model, chaining is the result of backprojections from CA3 (and mossy cells) to granule cells (Lisman et al., 2005). After each chaining step, the item is sent from dentate to CA3 for error correction, followed by the next chaining step. The function of CA1is less clear, but recent work indicates that inputs from the thalamus, cortex, and CA2 serve to differentiate CA1 response patterns that occur in the same location but at a different time or working memory state (Ito et al., Personal Communication; Mankin et al., 2012; Wood et al., 2000).

A second type of memory replay occurs online—for instance, when a rat pauses at the choice point of a maze and has to decide which direction leads to reward (Johnson and Redish, 2007). During this pause, the replay is not accompanied by a SWR but, rather, by theta and gamma oscillations (the theta-gamma neural code will be described below). What is observed at the choice point is that the sequential positions along one arm of the maze replay during one theta cycle (~150 msec); moments later, the experience down the other arm is replayed during another theta cycle. These sequences are transferred to downstream structures (PFC and striatum) (Jones and Wilson, 2005; Pennartz et al., 2011; Tort et al., 2008; van der Meer and Redish, 2011) involved in making the decision about which direction to turn (see section on action selection). Indeed, the strength of the theta frequency coupling between the hippocampus and striatum correlates with the accuracy of the ultimate decision (DeCoteau et al., 2007).

The communication of multi-part messages (e.g., the route to a goal) from one brain region to another requires a neural code. Such a code specifies two conventions, one about how the multiple items that make up a longer message are formatted and the other about how items are represented. Consider the Morse code: letters are represented by specific sequences of dots and dashes. Formatting involves conventions about the duration of pauses (brief pauses separate the dots/dashes that represent letters; longer pauses separate letters).

In the hippocampus, there is now substantial evidence that formatting is done by the co-occurring oscillations in the gamma (30-100 Hz) and theta (4-10 Hz) ranges (Fig.4B) (Bragin et al., 1995; Lisman and Buzsaki, 2008; O’Keefe and Recce, 1993), which together organize a theta-gamma code. Using this code (Fig.4C), multi-item messages are formatted as follows: an individual item is represented by the subset of cells (an ensemble) that fires during a gamma cycle; an ordered multi-item message is organized by representing different items in the sequential gamma cycles within a theta cycle (for definition of an item, see Fig.4C caption). It was long suspected that ensembles could be represented by the cells synchronized to fire within a gamma cycle (Engel and Singer, 2001). With the advent of methods that allowed recordings from large numbers of hippocampal place cells, it has occasionally been possible to find two cells that code for the same place. In support of the code outlined above, these cells are synchronized to fire within a time window roughly equal to a gamma cycle (Dragoi and Buzsaki, 2006). Other experiments show that interfering with theta or gamma oscillations produces memory deficits (Carlen et al., 2012; Korotkova et al., 2010; Robbe and Buzsaki, 2009; Shirvalkar et al., 2010).

The second aspect of coding is representation, and here too there has been progress. So much work on the hippocampus dealt with place information that it led to the impression that the hippocampus only dealt with spatial information. But why then do hippocampal lesions interfere with the memory of sensory experience? Recent work resolved this problem by showing that sensory experience is indeed represented, albeit in an unusual way termed rate remapping. Consider the ensemble of several hundred CA3 place cells that fire when a rat is in a given place (this is less than 1% of all CA3 cells). Experiments show that alteration of the sensory environment causes some of these cells to increase their rate and others to decrease their rate (Komorowski et al., 2009; Leutgeb et al., 2005). In this way, the sensory stimulus at a given place is encoded by the pattern of rates among the place cells that code for that place. This special role of place in sensory coding may be the basis of the Method of Loci developed by the Greeks to remember lists. According to this method, one should form a visual image of each item and imagine each image in sequential positions along a well-known path. To recall the list, one imagines taking the path and visualizes the items placed there (reviewed in (Foer, 2011)). This method may work because rate remapping encodes item information in place cells and because there are pre-existing connections that link place cells representing sequential position.

Network mechanisms of long-term memory

The intricate set of connections that define the wiring diagram of the hippocampus (Fig.4D) fascinated Marr (Marr, 1971), and he attempted to relate specific aspects of the CA3 network to memory function (Marr’s third level). CA3 neurons have abundant recurrent synaptic connections by which they excite other CA3 cells. Importantly, these are the defining feature of the attractor networks that theorists have postulated to underlie memory storage (Hopfield, 1982).The following simplified example illustrates how these synapses might store the association between a face and a name. Suppose that cell 1 represents Sue’s face and cell 4 represents her name (Fig.1C). When you first met Sue and heard her name, both cells fired. As proposed by Hebb (Hebb, 1949) and then verified experimentally (Kelso et al., 1986), hippocampal synapses undergo long-term potentiation (LTP) when there is simultaneous presynaptic and postsynaptic firing. Because this is true for the synapses connecting cells 1 and 4, these synapses will be strengthened (the postsynaptic excitation produced by a presynaptic action potential becomes larger). Now suppose that sometime later you see Sue, leading to activation of cell 1. Cell 4 will be caused to fire because of the strengthened input from cell 1, thereby bringing Sue’s name to mind. This is termed pattern completion because activating part of the association (face) leads to activation of (attraction to) the complete memory (face + name).

The concept of CA3 as an attractor has been tested in several ways. In one study, the NMDA receptors necessary for LTP (Bliss and Collingridge, 1993) were genetically deleted in CA3 (Nakazawa et al., 2002). These mice were presented with a fragment of the cue by which the animal had been conditioned to make a particular behavioral response. In wild-type animals, this partial cue was sufficient to evoke the conditioned response; in the mice lacking CA3 NMDA receptors, it was not. This indicates that the crucial process of pattern completion is dependent on the proper function of CA3. In another type of experiment, recordings were used to directly observe the “attraction” process. In these experiments, rats first formed memories of a chamber, the walls of which were frequently morphed between a square or round configuration (Leutgeb et al., 2007), producing memories of these shapes. In the crucial experiments, the walls were morphed to a shape that was basically square but slightly round. This altered the firing pattern in the input region of the hippocampus (the dentate gyrus) away from that characteristic of the square environment. In contrast, CA3 cells, which are downstream from the dentate gyrus, fired in a manner similar to that in the square environment, as expected if the recurrent connections of CA3 attracted the activity back to a stored memory (Renno-Costa et al., 2014) (see also (Neunuebel and Knierim, 2014)).

The above results make the strong case that the memory of an environment (an item) is stored in the synapses of CA3. However, to produce the replay of sequences of items (Fig.4A), there must also be synaptic processes that link one item to the next. The synapses that store such linkages are not known, but one possibility is the feedback connections from the CA3/mossy cells to the dentate gyrus (see explanation in Fig.4D caption).

Given the occurrence of place fields in the hippocampus, it has been of great interest to determine the network operations that enable the spatial sense to be computed. One underlying algorithm for how an animal can determine its location in an environment is called path integration, a process that depends on integrating a velocity signal (this process is what helps you find the bathroom in the dark). Velocity can be derived from the vestibular system, from optic flow, and from the motor system (how many steps have been taken) (Calton et al., 2003; Dombeck et al., 2010; Jamali et al., 2009; Ravassard et al., 2013; Stackman et al., 2002; Terrazas et al., 2005). Current ideas about the network mechanism of integration are best understood by considering a simplified one-dimensional integrator network in which cells are arranged along a line (Fig.1D). Each cell excites local cells and inhibits more distant ones (Skaggs et al., 1995; Zhang, 1996). The local excitation produces local self-sustaining activity (termed a “bump”). Special synaptic inputs move the bump’s position in the network and do so in proportion to velocity; this integration allows the position of the bump to represent the computed position of the animal in the environment. A type of cell in the entorhinal cortex (grid cells) is thought to work analogously to represent position in two dimensions (Burak and Fiete, 2009), thereby forming a cognitive map.

A recent paper (Erdem and Hasselmo, 2012) proposes an algorithm for how rats can use such a map to find a short-cut to a reward site (Tolman et al., 1992). An artificial velocity signal having a particular direction is put into the integrator. This moves the bump, generating imagined movement along the direction of the vector. Vectors of different directions away from the current position are chosen sequentially; if a particular vector produces movement of the bump through the reward site, the direction of that vector is a good shortcut. This algorithm is an interesting example of how a cognitive function might be reduced to a defined set of neurally plausible operations (for an alternative algorithm, see (Sanchez Fibla et al., October 18-22, 2010)).

Less is known about the events that encode a memory and make it endure. Recent work suggests that the hippocampus, like a computer memory, has separate read/write modes (De Almeida et al., 2012) and that the flow of information is different in these modes (Colgin et al., 2009). Presumably, processes during the write mode account for the observed changes in hippocampal synapses during learning (Whitlock et al., 2006). However, there are not yet real-time methods for observing encoding, and so we know little about what type of activity patterns actually produce it. We also know relatively little about what makes some memories fade while others endure. Recent work is beginning to shed light on relevant factors. This work shows that motivation, novelty, and reward influence dopamine release, which acts to convert early LTP into a more permanent synaptic modification (Bethus et al., 2010; Hasselmo and Schnell, 1994; Lisman et al., 2011; Yagishita et al., 2014). Understanding these factors may have important implications for educational methods (Ballarini et al., 2013).

In summary, many of the core problems in episodic memory have been directly addressed (for reviews see (Buzsaki and Moser, 2013; Kandel et al., 2014)). Importantly, the concept of a memory ensemble has been verified (Liu et al., 2012), the neural code used by the hippocampus has been determined, and the replay of memory sequences has been directly observed. The connection of this replay with behavioral memory has been established. Furthermore, there are strong hypotheses about how particular cell types and network connections produce function, and some of these have been experimentally tested. Taken together, this is substantial progress by Marr’s criteria.

Short-Term Memory

The brain has special mechanisms for storing short-term memories (STM) on the time scale of seconds (Baddeley, 2012). This is evident from the fact that, whereas the capacity of long-term memory (LTM) is enormous, the capacity (span) of STM is remarkably limited (for a review, see (Luck and Vogel, 2013)). For instance, the span for digits is only approximately seven items, making it relatively easy to remember a seven-digit phone number but difficult to remember a longer list. The determinants of span are important to understand because span is a major determinant of intelligence, as estimated using standard intelligence tests (Fukuda et al., 2010). Furthermore, keeping information persistently present (e.g., information about a selected goal) may be important in influencing action selection (this is working memory, a term often used synonymously with STM).

Addressing Marr’s second level are experiments that have sought to determine the general strategy for encoding working memory. A major discovery was the finding of neurons that fire persistently during working memory tasks (Fuster and Alexander, 1971). In a particularly informative set of experiments, animals were trained to remember the location of a briefly presented visual object for several seconds (the delay period) and then make a saccade to that location (Funahashi et al., 1989). A population of cells in the prefrontal cortex fired during the entire delay period. Significantly, the particular cells that fired depended on the location of the object (e.g., left or right part of the screen) and could thus serve as a storage mechanism for the impending action. These results suggest that, whereas LTM is stored by synaptic modification, STM is stored by persistent firing.

Addressing Marr’s third level, there has been efforts to determine the mechanism of persistent firing. One possibility is that it arises from a network configuration in which cells, once activated by an external stimulus, persist in firing because they synaptically excite each other, a process called reverberation. Theoretical and experimental work suggested that the depolarizing current through NMDA channels has a particularly important role in this mutual excitation (Lisman et al., 1998; Major et al., 2008; Palmer et al., 2014), and recent experiments directly support this: NMDAR antagonist prevents persistent firing in the PFC in vivo and compromises STM (Wang et al., 2013). Still, it remains unclear whether all forms of STM are encoded by persistent activity, and recent experiments have raised doubts (Liu et al., 2014). It remains possible that some forms of STM are stored by short-term potentiation (STP), a rapidly developing but short-lived synaptic modification that is associative but more easily triggered than LTP (Erickson et al., 2010; Sanderson et al., 2009; Sugase Miyamoto et al., 2008).

What determines the capacity limit of STM? One possibility is a multiplexing process by which the same network keeps multiple items active but at different phases of an oscillation. A specific form of multiplexing based on a theta-gamma code (Lisman and Idiart, 1995) (Lisman and Buzsaki, 2008) suggests that capacity is determined by the number of gamma cycles within a theta cycle (Fig.4C). Some experiments point in this direction (De Almeida et al., 2012; Siegel et al., 2009), but strong support has not yet been obtained.

Testing this and other models of STM in humans has been complicated by uncertainty about the location of STM buffers. The particular cortical region involved depends on the type of information stored (Emrich et al., 2013; Harrison and Tong, 2009). STM for digits and visual patterns involves regions in the parietal lobe (Cave and Squire, 1992; Koenigs et al., 2011). However, it does not follow that STM is stored in a single region. Recent advances in high-resolution fMRI have shown that early sensory areas show an activity pattern during the delay period that reflects the particular stimulus stored in STM (Harrison and Tong, 2009). Complicating the picture further is that information may be independently stored in different networks using different representations. Notably, the parahippocampal region contains an episodic memory buffer, one function of which is to load STM information into hippocampal LTM stores (Baddeley, 2000; Newmark et al., 2012; Schon et al., 2005). In contrast to other STM buffers that use a phonetic representation (based on sounds), the episodic buffer uses a semantic representation (based on meaning).

In summary, by Marr’s standard, the understanding of STM remains at a fairly early stage. It seems likely that some forms of STM require persistent firing caused by reverbertion, but the specific network connections responsible for reverberation have not been identified. The basis of span, a key property of STM, remains to be determined.

Action Selection

We now turn to the problem of how actions are selected. As we saw in the section on vision, there are neurons high in the cortical hierarchy that can represent a sensory item in an invariant way. In the action selection process, such cells become more strongly connected to cells that trigger action. Remarkable progress has been made in understanding how and where this occurs using the two major laboratory models of action selection, classical (Pavlovian) and instrumental conditioning (reviewed in (van der Meer et al., 2012)).

Mechanisms of Pavlovian conditioning

An extensively studied example of Pavlovian conditioning involves the emotion of fear. Rats show fear by freezing, for instance, after being given a shock (the unconditioned stimulus). Normally, a moderate intensity tone does not produce fear; however, if a shock is given whenever the tone is given, the tone becomes a cue (the conditioned stimulus). Once this happens, the cue given alone will produce freezing.

The core problem in fear conditioning is determining where and how the tone signal becomes able to evoke fear. Experiments show that this occurs in the pyramidal cells of the lateral nucleus of the amygdala. These cells are hard-wired so that when they are excited by painful stimuli, they activate cells in the hypothalamus and central grey (Fig.3B) that produce freezing and other components of the emotional response (for reviews, see (Johansen et al., 2011; Lang and Davis, 2006)). The cells of the lateral nucleus also receive inputs that carry information about the tone that before conditioning are too weak to excite the cell. During conditioning, the shock results in strong depolarization of these cells. This, together with the glutamate released as result of the tone, activates NMDARs, which, in turn, produces LTP of the synapses that carry tone information. Once this potentiation occurs, the tone alone can activate the pyramidal cells and produce a fear response (Quirk et al., 1995). A method that specifically blocks LTP in these cells prevents fear conditioning (Rumpel et al., 2005). Furthermore, a synaptic stimulation procedure that weakens the potentiated synapses erases the fear response (Nabavi et al., 2014). What appears to be a key event in associating tone with shock is the depolarization of the pyramidal cells by the synaptic inputs that carry information about the shock. It is this depolarization that is necessary for the LTP of the synapse that carry tone information. It follows that, if tones and optogenetically produced depolarization were paired, it should be possible to make an animal fearful of a tone without ever shocking it. Experiments show that this is possible (Johansen et al., 2010). Thus, an important form of Pavlovian conditioning can be explained in terms of specific synaptic changes in an identified cell type.

Mechanisms of instrumental conditioning

Whereas fear conditioning involves a hard-wired component of behavior (e.g., the fear evoked by shock), instrumental conditioning does not. With instrumental conditioning by reward or aversive stimuli, almost any form of behavior can be produced. As an example, consider a rat being conditioned to make a right turn at the choice point of a maze. Under these conditions, the cue is the sensory properties of the choice point. If reward is consistently given immediately after the animal happens to make a right turn, the animal will eventually make right turns consistently. Alternatively, rats can be made to avoid taking a right turn if they are shocked whenever they do so. Over the animal’s life, various forms of behavior have been chosen, leading to reward and punishment. The action selection system takes this cumulative experience into consideration when actions are selected (Marr’s first level).As described in the following paragraphs, we now know a great deal about the underlying processes and how they are implemented at the network and cellular levels. The algorithm involved depends on a parallel comparison of different possible actions and has partially separate subsystems for storing the cumulative associations of rewards and punishments.

Experiments suggest that the synaptic changes that produce instrumental conditioning are at the synapses that carry the cue from sensory cortex to the medium spiny neurons (MSN) of the striatum, the input structure of the basal ganglia. These synaptic changes are controlled by dopamine, which acts as a teacher to signal whether reward or punishment has occurred. The understanding of these mechanisms has come from progress in two fields. In vivo recordings showed that unexpected rewards (positive reinforcement) produce a burst of firing in the dopamine cells that innervate the striatum (Cohen et al., 2012; Schultz et al., 1997). Meanwhile, experiments on striatal slices (Shen et al., 2008) showed that dopamine elevation increases the strength of active synapses onto a subset of MSNs that have D1 dopamine receptors and are part of what is called the “direct” pathway that promotes action (Go pathway in Fig.5). These D1-MSN neurons (and their downstream targets in the globus pallidus, thalamus, and premotor cortex) are organized into parallel streams that promote different actions (e.g., RIGHT or LEFT) (Cisek and Kalaska, 2010; Frank et al., 2004; Houk and Wise, 1995; Kelly and Strick, 2004). Thus, if, during the initial stages of conditioning, cortical synapses activate the RIGHT D1-MSNs (perhaps by chance) and this leads to reward, these synapses will be strengthened. As a result, on subsequent trials, the same cue will more strongly excite the RIGHT D1-MSNs and will lead, by the direct pathway, to increased probability of the animal taking a RIGHT turn.

Fig.5. Connections of the basal ganglia and premotor cortex that mediate instrumental conditioning (a cue leads to the conditioned response).

Sensory cortex carries cue information to the striatum through synapses onto medium spiny neurons (MSN). There are different MSNs for different actions (e.g., LEFT or RIGHT; L/R), and these form separate streams related to different actions in the downstream GPe, GPi, thalamus, and premotor cortex. Some MSNs have D1 receptors and form the direct pathway to the output structures of the basal ganglia, the Globus Pallidus interna (GPi) and Substantia Nigra pars reticulata (SNr), both of which are spontaneously active (Hikosaka, 2007) and reduce activity in premotor cortex via a thalamic intermediary (Goldberg et al., 2012). Reward, via dopamine elevation from the Substantia Nigra pars compacta (SNc), leads to LTP of cortical inputs to the MSN-D1 that led to the rewarded action. This change promotes the rewarded action on subsequent trials by the following mechanism: the strengthened input to the direct pathway inhibits GPi/SNr, thereby removing a negative bias from premotor cortex and allowing action selection by a winner-take-all process in that structure (Cisek and Kalaska, 2010). Aversive stimulation, via a decrease in dopamine, leads to LTP on the MSN-D2 cells that represent the chosen action. These cells, via the indirect pathway (with globus pallidus externa (GPe) as an intermediary), increase the inhibitory output of the basal ganglia, making action less likely. Connections from premotor cortex to striatum (dashed green lines) may selectively enable the possible actions in the current context (i.e., L and R). Furthermore, after the action is finally chosen in premotor cortex and then executed, “chose action” activity in this pathway (Lau and Glimcher, 2008) can insure that credit is correctly assigned to the chosen action (Fee, 2012).

This framework has now been tested in several different ways. Inactivation of the striatum (Miyachi et al., 1997) or, more specifically, MSN-D1 cells (Hikida et al., 2010) prevents reward-dependent conditioning. Conversely, optogenetic activation of these cells promotes action initiation (Kravitz et al., 2010). Most remarkable are experiments confirming the role of dopamine in conditioning. These experiments show that preventing burst-mediated release of dopamine prevents conditioning (Zweifel et al., 2009), whereas optogenetic stimulation of dopamine cells can produce conditioning (Fig.6), obviating the need for actual reward (Adamantidis et al., 2011; Kim et al., 2012).

Fig.6. Conditioning by optogenetic activation of dopamine cells instead of actual reward.

Channel-rhodopsin was expressed in dopamine cells. When the mouse made a “correct” response (in this case, a LEFT nose poke), a flash of light was given to activate channel-rhodopsin. Over days, the mouse showed conditioning by making more pokes to the left (thick line), with little change of pokes to the right (thin line). This conditioning underwent extinction when flashes were no longer given after day 9. From the study of (Kim et al., 2012) as supplied by E. Boyden.

What is described above is reinforcement by reward, but conditioning can also be produced by aversive stimuli (punishment or omission of expected reward). Reinforcement using aversive stimuli leads to synaptic strengthening in a second type of MSNs that contain D2 receptors and are part of the indirect pathway, a pathway that inhibits actions (No Go pathway in Fig.5). Again, the study of dopamine cells and plasticity mechanisms clarified the teaching mechanism. Whereas unexpected rewards increase the firing of dopaminergic cells, aversive stimuli decrease dopamine by inhibiting the spontaneous firing of dopaminergic neurons (reward prediction error) (Schultz et al., 1997). Experiments in the slice preparation indicate that a decrease of dopamine enhances LTP onto D2-MSNs (Shen et al., 2008). The stronger excitation of these cells will, by the indirect pathway (see Fig.5 caption), make it less likely that the punished action will be repeated.

This model of conditioning by aversive stimuli has been tested in several ways. Consistent with this model, inactivation of D2-MSNs of the indirect pathway interferes with the ability of aversive stimuli to produce conditioning but has no effect on the ability of reward to produce conditioning (Hikida et al., 2010). Furthermore, optogenetic stimulation of these cells produces a Parkinson’s-like state in which action initiation is reduced (Kravitz et al., 2010). Finally, optogenetic methods that transiently reduce dopamine release can be used to make animals avoid the region where dopamine reduction occurs (Carlen et al., 2012).

Overall, this basal ganglia/premotor system provides a voting process for action selection that depends on the history of conditioning (both L and R actions may have been rewarded or punished to varying extents in the past). When a cue is encountered, it is thus likely to lead to activation of MSNs in both the direct (votes for) and indirect (votes against) pathways and to do so for both L and R actions (Cui et al., 2013). The strength of these synapses, which depends on the history of conditioning, determines the number of votes (for or against) for L/R actions. These votes are funneled from MSNs to the basal ganglia output structure, the GPi/SNr (Hikosaka, 2007), from there to the thalamus (Goldberg et al., 2012), and finally to premotor cortex (and in some cases to the superior colliculus). It is there that votes for and against are tallied and the final winner-take-all decision for L or R action is made (for a review, see (Cisek and Kalaska, 2010)). An important feature of this mode of basal ganglia function is that action selection is achieved by a fast parallel voting process that reflects the entire history of previous conditioning. Much less is known about a more deliberative mode in which the basal ganglia chooses actions based on the serial recall of episodic memories from the hippocampus (Johnson and Redish, 2007).

The above framework for action selection helps to explain several consequences of basal ganglia function (Belin et al., 2009; Maia and Frank, 2011). One tragic consequence is addiction; addictive drugs lead to dopamine release (Hyman et al., 2006), thereby strengthening the “chosen action” that produced dopamine release (the taking of the drug). This makes it more and more likely that drug taking will be repeated in the future (Redish, 2004). Another consequence is that D2 agonists often turn patients into pathological gamblers (Lader, 2008). The above model suggests why. Gambling losses decrease dopamine, but this is not detected when the D2 receptors are occupied by agonist. Thus, the normal strengthening of the indirect pathway by losses, which depends on detection of a dopamine decrease, does not occur. In contrast, wins are registered when the resulting increase in dopamine strengthens the direct pathway by D1 action. The final result is that gambling becomes extraordinarily desirable because wins can affect the voting process, whereas losses cannot.

In summary, substantial progress has been made in understanding instrumental conditioning at all three of Marr’s levels. Notably, in current models, the synaptic sites that store the results of conditioning and the network mechanism by which this information is used to select an action are specified. The properties of basal ganglia networks have been incorporated into large-scale computer simulations, and these successfully account for many aspects of conditioning (Eliasmith, 2012; Frank et al., 2004; Grossberg, 2014; Gurney et al., 2004). Still, because much remains to be learned about these networks, the models are best considered strong working hypotheses. For instance, it will be important to verify in vivo that the key site of learning is at the MSNs. There also remain important aspects of action selection that are not well understood. For instance, dopamine, in addition to affecting synaptic plasticity, also affects the excitability of striatal cells, a modulation that may control the level of effort that an organism is willing to make (Dagher and Robbins, 2009; Niv et al., 2007). Furthermore, the mechanisms that enable dopamine cells to be a good teacher are just beginning to be elucidated (Lammel et al., 2012; Volman et al., 2013). As might be expected for such a crucial process, the complexity of control is high; the habenula, a structure that controls the dopamine system and that has been strongly implicated in depression, has no less than 15 subnuclei (Proulx et al., 2014). Finally, the picture that emerges from the above work is that learning occurs by trial and error; a behavioral variant is strengthened or weakened depending on the results. But this raises the important question of what produces variation. In evolution, this is due to mutation, but what produces variation in the brain? In the study of birdsong, a clear answer has been obtained: there is a specific cortical region, the function of which is to produce variation (Aronov et al., 2008; Stepanek and Doupe, 2010). The generality of this finding remains to be determined.

The Motor System

Once an action is selected, the motor system must execute the action, and this is no trivial matter. Our motor abilities are exquisite. You can easily make a reach to a visible target irrespective of whether your gaze is directed toward the target or whether the target is only in your peripheral vision. More remarkably, you can touch your ear lobe quite accurately even though you can’t see it. This is possible irrespective of the particular trajectory chosen and irrespective of the load on your muscles (try touching your ear lobe while holding a mug in the same hand). What algorithm could underlie this robust motor control? At present, we have only a very limited answer to this question.

As noted above, we can reach a target that we gaze at directly or one that is only in our peripheral vision. Given that information about gaze angle is available to the brain, visual information could be corrected for gaze angle. Indeed, recordings in the parietal lobe indicate that such correction is done; objects are represented in a coordinate system in which the position of an object is defined relative to the body and does not depend on gaze angle (Andersen and Mountcastle, 1983). Biologically plausible network operations that can produce such a coordinate transformation have been suggested (Andersen and Mountcastle, 1983; Salinas and Sejnowski, 2001). More difficult to imagine is how one can reach one’s ear lobe without visual guidance. This requires knowledge of the position of the target and the position of the limbs. It has only recently become clear that there is indeed a population code that represents hand position (Hauschild et al., 2012), but surprisingly little is known about the muscle and joint signals that allow this computation to be made (Weber et al., 2011).

Once the coordinate system for specifying a reach is established, the muscles needed to produce the reach must be triggered by activity in the primary motor cortex. Despite considerable effort, there remains no consensus about what motor cortex is specifying about the reach. It remains unclear whether cells represent a signal for muscle force, the direction of movement, or a more abstract end-goal of muscle action (Adelsberger et al., 2014; Georgopoulos et al., 1982; Graziano, 2006; Shenoy et al., 2013). One promising line of work suggests that subregions of the motor cortex regulate groups of muscles that act synergistically to produce motor primitives (multi-joint movements that can be combined in various proportions to produce a variety of actions) (Dominici et al., 2011; Overduin et al., 2012).

Any explanation of movement will have to specify not only which muscles are activated, but how their activation is precisely timed. The cerebellum appears critical for a range of timing processes that occur in the range of seconds (Manto et al., 2012). One of these is the generation of temporal waveforms that predict the sensory and somatosensory consequences of one’s own motor commands (forward models), predictions that are vital for motor and sensory processes (Bell et al., 2008; Franklin and Wolpert, 2011). Eye blink conditioning has provided important insights into another timing function of the cerebellum. In such conditioning, a tone is turned on and a puff of air is given to the eye hundreds of milliseconds later. After such conditioning, the tone produces a protective eye blink just before the air puff (reviewed in (Thompson, 2005)). Experiments show that after conditioning, the spontaneous firing of the output cells of the cerebellar cortex, the Purkinje cells, pauses just before the air puff (Jirenhed et al., 2007). Recent optogenetic experiments demonstrate that such pauses are sufficient to trigger motor action (Heiney et al., 2014). It remain controversial whether the pause is due to reduced excitation (Ito, 2005) or increased inhibition (Gao et al., 2012; Hirano and Kawaguchi, 2014; Schonewille et al., 2011; Welsh et al., 2005).

Although progress has been made in understanding some aspects of motor control, an overall view of how the system works is lacking, and the field is very far from satisfying Marr’s criteria. Advances are needed in understanding how cortex, basal ganglia, and cerebellum function together (for one possibility, see (Shadmehr and Krakauer, 2008)). One would hope that overall design principles could be elucidated. One suggestion is that motor control is organized by a ~10 Hz clock (Llinas, 2002). Indeed, the limit on independent finger movements (as in typing) is about 10 Hz (http://10fastfingers.com/typing-test/english/top50). If indeed the motor system is controlled by oscillations, actions longer than 100 msec must be fundamentally discontinuous. Support for such discontinuity comes from measurements of finger position during simple linear movements (Vallbo and Wessberg, 1993). Remarkably, there are pulses of acceleration every ~100 msec, suggesting that even linear movement has a 10 Hz discretization. This rhythmicity is coherent with signals in the cerebellum and motor cortex (Gross et al., 2002), consistent with the idea that the entire motor system is organized by a 10 Hz clock. Most persuasively, movement onset occurs at a preferred phase of ongoing cortical oscillations (Drewes and VanRullen, 2011; Igarashi et al., 2013). It thus seems likely that elucidating the role of oscillations in the motor system will be an important step toward understanding the still mysterious algorithms of motor control.

Higher Mental Functions

From the discussion thus far, one can see in rough outline how a “simple” behavior might occur. But what about more complex mental functions, such as executive control, thought, and consciousness? Below I very briefly outline current efforts to understand these higher functions.

Executive control

Action selection in the laboratory setting involves linking a sensory cue to an action but does not generally involve context. Action selection in real life situations takes an enormous amount of contextual information into consideration, including the existence of prepotent responses (Schall and Godlove, 2012), the assessment of what is possible (affordances) given the current goals (Cisek, 2007; Gibson, 1977), the application of abstract rules (Wallis et al., 2001), and the evaluation of potential rewards based on their magnitude (Madlon-Kay et al., 2013), delay (Ballard and Knutson, 2009), and uncertainty (Kepecs et al., 2008). This strong dependence on multiple constraints is not unlike the dependence of recognition on context (Fig.2) and thus is likely to depend on the hierarchical properties of the cortical regions that control action (Cooper and Shallice, 2006). The basal ganglia is also hierarchically organized (Haber and Calzavara, 2009; Yin and Knowlton, 2006); conceptual breakthroughs are needed to understand how these interacting hierarchies produce executive control.

Language-based control

The existence of language raises questions about the generality of the mechanisms of action selection revealed by the study of instrumental condition. Whereas animals have to be conditioned by the experimenter to make a particular action, a human can simply be asked (e.g., “please raise your hand”). One wonders whether some entirely new mechanism of action selection has evolved in humans or whether, as suggested in a recent paper (Kriete et al., 2013), modification of existing basal ganglia mechanisms has made action selection dependent on thought rather than by conditioning.

Consciousness

According to dualists, consciousness cannot be explained in terms of the physical properties of the brain. By contrast, neuroscience now assumes that consciousness is a result of specific properties of brain networks (Dehaene and Changeux, 2011; Edelman et al., 2011; Koch, 2009). Studying consciousness is not easy, but there are methods. Reportability is the standard experimental method for verifying human consciousness. Based on experiments that manipulate the ability of subjects to report visual stimuli, (Dehaene and Changeux, 2011) proposed that sensory cortex does local visual processing before information comes to consciousness. Then, at some critical juncture, information becomes widely communicated from the local source to a global neuronal workspace, thereby producing consciousness. What might the critical juncture be? There is some evidence that this occurs when a high-level cortical model is chosen that correctly predicts low-level sensory information (Graboi and Lisman, 2003; Pascual-Leone and Walsh, 2001; Pollen, 1999). A further question is how information is communicated to the global neuronal workspace. It is well known that there are direct connections between cortical regions, but (Sherman, 2005) emphasizes that there is also a thalamic route. Thus the thalamus, which has been implicated in attention (Crick, 1984; Halassa et al., 2014; Saalmann et al., 2012; Schmid et al., 2012; Zikopoulos and Barbas, 2006), must be considered a potential route to the global neuronal workspace.

Another approach to understanding consciousness comes from consideration of unconscious action (Lisman and Sternberg, 2013). A person can drive to work along a habitual route and also think about a problem. Upon arrival at work, the person may remember how they solved the problem but may be unable to report anything about the commute, thus indicating that driving was done unconsciously by habit. Relating such actions to habit opens new ways to explore the unconscious because habit (automatic) and non-habit (goal-directed) behaviors can be distinguished in rodents (Dickinson, 1985) and experiments show that different brain regions are involved (Daw et al., 2005; Smith et al., 2012; Yin and Knowlton, 2006). It may thus be possible to identify mechanisms unique to the regions that mediate non-habit behaviors, thereby providing clues about the mechanism of consciousness. One possible mechanism deserves special mention because of recent progress. The philosopher, John Locke, defined consciousness as “the perception of what passes in a man’s own mind” (Locke, 1689), implying that consciousness depends on a brain architecture in which thought can activate perceptual processes (e.g., imagery or auditory sensation). In a recent experiment, such activation was demonstrated directly ((Albers et al., 2013) (see also (Shergill et al., 2002)). Subjects were shown oriented bars and were asked to imagine the bars rotated by a certain amount; using fMRI it was found that the activation pattern in visual cortex was similar to that produced by viewing rotated bars. If Locke’s definition of consciousness is correct, understanding the top-down information flow that produces imagery will provide a mechanistic description of consciousness.

Conclusion

Let us return to the central question of this Perspective: where are we in the historical process of understanding the brain? There are major successes to point to that meet Marr’s three criteria for understanding. Through the study of the hippocampal system, it is now possible to directly observe memory sequences being recalled and to optogenetically manipulate memories. Furthermore, specific networks and synaptic connections have been demonstrated to form the associations that underlie some forms of memory. In the case of action selection, the synapses in the amygdala and basal ganglia where conditioning occurs have been identified. The understanding of Pavlovian and instrumental conditioning is not superficial; the fact that optogenetic stimulation of specific cells can altogether bypass the need for the unconditioned stimulus demonstrates that core processes are now understood. Current models of the hippocampus, amygdala, and basal ganglia are certainly incomplete, but they are unlikely to be fundamentally wrong.

Less progress has been made in understanding visual perception, working memory, and motor control. We cannot yet understand these processes because they are cortical and the role of different cortical layers and cell types has not yet been determined. It was thought that the canonical flow of information within a cortical column was from layer 4 to layers 2/3 to layers 5/6 (Douglas and Martin, 2007), but it now seems likely that this is not generally the case (Constantinople and Bruno, 2013). There are, however, reasons to be optimistic about prospects for understanding cortex. Experimental methods of enormous power are now being brought to bear on the problem. For example, in the somatosensory cortex, it is now possible to study the function of identified cell types in awake animals during sensory-guided decisions (Larkum, 2013; Xu et al., 2012). Furthermore, during such decisions, it is possible to inhibit or excite specific cell types using optogenetics (Guo et al., 2014). These new methods would seem to be sufficiently powerful to finally crack the cortex problem.

To be sure, the sheer number of cytoarchitectonic areas in cortex (Fig.3A) makes the prospect of understanding this part of the brain seem daunting. There are, however, regularities that may simplify the task. Different regions have similar cell types with common connection motifs (Klausberger and Somogyi, 2008; Pi et al., 2013). Furthermore, there are repeating rules for hierarchical organization: most of the cortex is accounted for by five paired cortical hierarchies arising from primitive regions ((Pandya and Yeterian, 1985); see Fig.3 caption). Similarly, the study of subcortical structures has identified repeating structural rules (Kinkhabwala et al., 2011; Swanson, 2005).

The skeptic might pose another ground for pessimism: that what we are learning about the rodent brain, the workhorse of modern neuroscience, may not apply to the human brain. This seems unlikely given the qualitative similarities of rodent and human neuroanatomy. But there is data in one research area that bears on this question. Recordings from the human hippocampal region made during surgery reveal place cells and grid cells with obvious similarity to those recorded in the rodent (Ekstrom et al., 2003; Jacobs et al., 2013). Thus, even though there are certainly differences between the human and rodent brain, there is little doubt that what is learned from the study of the rodent brain will take us a long way toward understanding the human brain.

Perhaps twenty years ago, one could have argued that the emergence of cognitive function from interconnected neurons was deeply mysterious. That does not seem true today. What has changed is that we now have a feel for how networks can produce cognitively relevant computations (Figs.1,4,5). In many areas of brain research, models of network function are being explored through the interplay of experimentation, theory, and computer modeling. This is leading to sound, tested concepts that address Marr’s three levels. In summary, there has been demonstrated success in providing an understanding of several brain processes, and there is every reason to expect further rapid progress. History is thus likely to look back on the first half of the twenty-first century as the period during which the brain came to be understood.