Neurons as will and representation

Abstract

Memory recollections and voluntary actions are often perceived as spontaneously generated irrespective of external stimuli. Although products of our neurons, they are only rarely accessible in humans at the neuronal level. Here I review insights gleaned from unique neurosurgical opportunities to record and stimulate single-neuron activity in people who can declare their thoughts, memories and wishes. I discuss evidence that the subjective experience of human recollection and that of voluntary action arise from the activity of two internal neuronal generators, the former from medial temporal lobe reactivation and the latter from frontoparietal preactivation. I characterize properties of these generators and their interaction, enabling flexible recruitment of memory-based choices for action as well as recruitment of action-based plans for the representation of conceptual knowledge in memories. Both internal generators operate on surprisingly explicit but different neuronal codes, which appear to arise with distinct single-neuron activity, often observed before participants’ reports of conscious awareness. I discuss prediction of behaviour based on these codes, and the potential for their modulation. The prospects of editing human memories and volitions by enhancement, inception or deletion of specific, selected content raise therapeutic possibilities and ethical concerns.

How do memories spontaneously come to mind and voluntary actions arise? Such internal generation has long been a source of neurophysiological query¹. However, it is extremely difficult to study, as it requires individuals who can declare their thoughts, memories and wishes. Although memory and action have been studied in non-human animals at the levels of single neurons and neuronal populations, subjective experiences cannot be readily studied in such animals. Functional neuroimaging has enabled spectacular investigations of cognition in humans, but the signals from this approach provide only surrogate markers for neuronal activity, and such imaging offers spatial and temporal resolutions that are far lower than those of recordings of single neurons and small neuronal assemblies.

Thus, recordings from, or stimulation of, single neurons and neuronal assemblies in humans who can declare their experiences have been crucial for studying human cognitive functions. Such opportunities arise in certain neurosurgical procedures^2,3, including during open brain surgery, implantation of deep brain stimulation and recording devices for various disorders, monitoring of epileptic activity via intracranial electrodes and implantation of brain–machine interfaces in patients with paralysis. Such studies are done in patients with neurological disease, and so the results should be viewed with caution and alongside findings from other neuroscience methods. Moreover, the brain regions sampled are limited by clinical considerations. Most single-neuron recordings from patients with epilepsy are obtained from frequently implanted sites in the temporal and frontal lobes⁴, and opportunities for recordings from brain–machine interfaces in patients with paralysis have been limited to motor and parietal cortices.

Here I present an overview of human single-neuron studies in two brain systems: the medial temporal lobe (MTL) and a frontoparietal system involving primarily the medial frontal lobe (MFL) and the posterior parietal cortex (PPC). These systems use neuronal codes, surprisingly explicit at the single-neuron level, to internally trigger representations of concepts and actions, respectively. I discuss the presence of an interaction between these systems, a conceptual–volition interface, where representations of actions and stored memories interact, sometimes without sensory input, or even overriding such input.

The MTL is the brain’s chief organ of declarative memory, translating external objects and events into consciously accessible mental representations that can later become retrievable, even without, or despite, external stimulation⁵. In rodents, the entorhinal–hippocampal machinery controlling navigation behaviour and the consolidation of spatial memory to durable neuronal representations has been well studied^6,7. These findings have been extended to non-human primates and humans^8,9, but their translation to non-spatial memory has been challenging¹⁰. Recently, however, a growing literature has attempted to apply neuronal coding principles used by rodents negotiating space to neuronal codes for negotiating multidimensional ‘cognitive’ or ‘conceptual’ space^11,12.

The MTL system for the encoding and maintenance of memory traces is complemented by a frontoparietal system translating action programmes into consciously accessible representations, namely self-recognized voluntary actions. This system has a large MFL component, including both medial frontal cortex and prefrontal cortex, and a PPC component, involving the superior and inferior parietal lobules and the intraparietal sulcus¹³.

By examining internally arising voluntary action and memory recall, in this Perspective, I address the content and temporal course of conscious experience and its neuronal correlates. Crick and Koch¹⁴ offered the formulation of the neural correlates of consciousness (NCC) as the minimal neural mechanisms jointly sufficient for any one conscious percept¹⁴. Here I focus on content-specific NCC, defined as the neural mechanisms that specify contents within consciousness, such as faces, places, intended actions and specific recollections. Experimentally, content-specific NCC are typically investigated by comparing neural responses in conditions in which specific conscious contents are present versus absent, as in flash suppression¹⁵ or binocular rivalry¹⁶. The process by which one becomes aware of specific NCC content could be studied by comparing contents that are externally triggered versus internally triggered. However, in practice, this proves extremely difficult, as the conscious recognition of such content involves processes or states regarded as unconscious or preconscious, which, as recently reviewed, are often assumed rather than demonstrated¹⁷.

Discussion of the major theories of consciousness is beyond the scope of this Perspective. However, the system involving the MTL and the MFL–PPC system serving voluntary action are compatible with both the global workspace theory of consciousness, which would argue for frontal activation, and the integrated information theory of consciousness, predicting parieto-occipital activation¹⁸.

Concept cells in the MTL

Forming mental objects

How do objects become ‘mental objects’, recognized by the self, and how do these mental objects become available for conscious representation long after the physical objects are gone? This transformation is the basis of declarative memory and underlies the ability to navigate in space and time among mental objects, both experienced and imagined. As we will see, a critical role in this transformation is played by unique cells in the MTL termed ‘concept cells’.

Decades of research based on lesion, electrophysiological and neuroimaging studies have singled out the MTL and its entorhinal–hippocampal system as having a crucial role in declarative memory and the maintenance of experience as durable representations⁵. To use these neuronal representations in the future and in different environments, this system must form invariants: patterns of neuronal activity that remain constant despite changing sensory input. Such invariants have already been described in the rodent hippocampal–entorhinal system and in the cases of hippocampal place cells⁶ and entorhinal grid cells⁷, as well as other cells, such as border cells and head direction cells¹⁹. Grid cells feed an organized representation of the two-dimensional environment to the hippocampus, resulting in the formation of invariants representing specific locations — the place fields. As well as imposing organization on the spatial environment of the rodent, these invariants can be combined in time to provide the rodent with path integration, enabling navigation in any new environment. Place cells and grid-like cells have also been discovered in humans^8,9; however, whereas in rodents these cells represent spatial architecture²⁰, in humans this system may provide non-spatial architecture for episodic memory. I previously hypothesized that similarly to the hippocampal system serving spatial navigation in rodents, the human MTL serves ‘cognitive navigation’ in a multidimensional space, where the dimensions are stimulus attributes or features²¹. A growing literature, based on the use of functional neuroimaging, electroencephalography (EEG) and magnetoencephalography (MEG) tools, attempts to link the specialized cellular machinery underlying rodent spatial navigation with so-called concept space^22,23.

Here I focus on the unique MTL neuronal code, as revealed by single-neuron recordings in patients able to declare their percepts and recollections. Importantly, this ‘conceptual code’ forms selective and invariant representations of external stimuli that become available for mental computations and memory. Similarly to path integration enabled by invariant responses of place cells, the ability of humans to navigate mental space depends on invariant responses of concept cells that provide stable codes for neuronal computations. The neuronal responses constituting this conceptual code have several unique features.

Coding properties of concept cells

Selectivity and invariance.

The most important property of the neuronal responses constituting the conceptual code is their combination of selectivity and invariance. Selectivity represents the uniqueness of individual experiences and invariance represents its abstraction. Some MTL neurons respond selectively to categories, such as people, animals or outdoor scenes^21,24,25, whereas others respond selectively to individual stimuli^26–28 (FIG. 1a). However, MTL neuronal responses to individual stimuli are not bound by the physical attributes of the stimulus, and therefore these neurons respond to distinctly different representations of the same individual stimulus²⁶. Thus, a neuron may respond selectively to a picture of President Clinton, to numerous physical iterations of Clinton and/or even to the written words ‘Bill Clinton’. Response invariance may cross modalities: there are similar responses to visual and auditory representations of the individual (FIG. 1a). Furthermore, this invariance is strikingly maintained across time: for example, a neuron responding invariantly to a person may fire throughout a 10-s audiovisual episode in which the same person appears²⁹ (FIG. 1b). The specificity and invariance of neuronal responses are more prominent in the hippocampus and entorhinal cortex than in the amygdala, parahippocampal gyrus or regions earlier in the ventral system, such as the fusiform gyrus³⁰.

Fig. 1 |. Responses of human medial temporal lobe neurons to images and episodes.

a | Selective responses of a concept neuron in the entorhinal cortex to various images of Saddam Hussein (stimuli 13, 28 and 29), to his written name (stimulus 52) and to the sound of his spoken name (stimulus 80). Raster plots and peristimulus time histograms for six trials are shown. No significant responses to 79 other stimuli were seen. Three of these other stimuli (26, 65 and 71) are shown for illustration purposes. The median number of spikes for stimuli with the largest responses is shown (in green and purple bars). Only the responses to stimuli representing Saddam Hussein (purple bars) were significant (more than five standard deviations above the baseline firing rate (FR)). b | A single unit in the left anterior hippocampus activated during viewing of many video episodes (10 s each, with onset and offset depicted by dashed vertical lines, and with a 2-s blank screen preceding each clip). Note the marked increase in FR throughout each 10-s presentation of an Emma Thompson episode (box with dashed line), which persisted for 1–2 s after clip offset. The FR persisted despite each clip containing multiple different frames including the actress with changing facial expressions during a monologue, as well as frames with her written name (without her image). Raster plots and peristimulus time histograms for six trials of each clip are shown. At the bottom are shown trial-by-trial responses of the neuron firing every time the Thompson clip is presented but not when intervening clips are presented. Only a few intervening clips are presented for illustration purposes. Part a is adapted with permission from REF.³⁰, Elsevier. Part b adapted, with permission, from REF.²⁹, AAAS.

Responses are late.

Neuronal responses in the MTL are relatively late, often 300–500 ms after stimulus onset. These responses progressively increase in latency along the ventral stream, with shorter latencies in the fusiform gyrus, longer latencies in the parahippocampal gyrus and latencies that are longer still in the hippocampus, entorhinal cortex and amygdala^31,32.

The code is sparse.

Highly selective cells respond to a small number of stimuli, suggesting that the code is sparse. The sparseness of the code is most evident in the hippocampus and entorhinal cortex³⁰. Consistent with this notion, a probabilistic analysis estimated the degree of sparseness of the code (that is, the percentage of cells that respond to a particular stimulus) ranges from 0.2% to 1%³³. By comparison, responses in the monkey superior temporal sulcus or inferotemporal cortex have a mean sparseness of 33%^34,35.

The code is associated with conscious perception.

Flash suppression, backward masking and attentional blink experiments show that the magnitude and timing of MTL responses are associated with conscious perception^15,36–38. Stimuli that are not consciously perceived following brief retinal presentation with backward masking also fail to elicit MTL neuron responses³⁶. In flash suppression, the perception of a monocular stimulus is suppressed upon a different stimulus being flashed in front of the contralateral eye. Therefore, although two different stimuli are presented to the two eyes, only one stimulus is consciously perceived. Here, again, neurons followed the perceived stimulus; that is, only the neuron responding to the flashed stimulus fired¹⁵.

A striking demonstration of this code in conscious perception was provided by a study in which two face stimuli (A and B) were morphed to form a ‘midtransition’ image with maximum ambiguity. When a participant was presented with successive images of A, followed by presentation of the ambiguous face, the latter was more likely to be recognized as B. If the images preceding presentation of the ambiguous face were of B, then the ambiguous face was perceived as A. Thus, the same physical stimulus elicited two different conscious percepts. Again, neurons followed the conscious percepts; that is, if a neuron encoded face A, it responded to the ambiguous face only when the conscious percept (reported by the participant) was that of face A³⁷.

The code is dynamic.

The presence of responses to particular individual stimuli can change with experience. Responses to family members and celebrities are more frequent than those to unknown faces²⁸. However, with increasing exposure to a new face, responses are more abundant, reflecting recruitment of more responding cells. Interestingly, patients develop cells representing the experimenters just a day or two after first meeting them³⁰.

The code is associative.

A remarkable property of the code is its dynamic associative nature. For example, place cells can become associated with objects encountered in those places and later reactivated when participants remember those objects³⁹. Moreover, if a cell responds to stimulus A, once stimulus B is presented together with stimulus A, the cell will start responding to stimulus B as well⁴⁰, even in the absence of explicit association between stimulus A and stimulus B, and with a similar response latency to stimulus A. Such observations may explain how single-neuronal responses expand to assimilate additional stimuli and mental objects, thus affording the dynamic generation of abstract invariants. Furthermore, such associations at the single-neuron level are long term⁴¹. Cells related to a concept or a word with rich semantic associations or similarity to other words may be more likely activated during retrieval, thus resulting in greater memorability of these words or concepts. Interestingly, words sharing greater semantic similarity with other words are more readily available during memory retrieval in a paired association task irrespective of the cue word, and are associated with faster neuronal reinstatement as measured by intracranial EEG power⁴².

The code is maintained active when the concept is held in mind.

Concept cells remain active when the concept is maintained in working memory, and this activity is predictive of the content of a memory^43,44. This finding suggests a role for concept cells in maintaining specific patterns of activity beyond the physical presentation of the stimulus and thus using these patterns for further neuronal computations.

The code is internally generated.

In the absence of external stimuli, the code can still generate responses that reinstate those generated in the presence of external stimuli. This is evident in mental imagery⁴⁵. Similarly, visual recall of previously viewed audiovisual episodes reinstates specific sustained responses selective for those episodes²⁹.

Pattern separation.

The transformation of external reality to durable representations in memory requires not only abstraction but also retaining discriminant information that can later be used to separate partially overlapping representations. The neuronal substrates of pattern completion versus pattern separation in the hippocampal–entorhinal system have been investigated at the cellular level in rodents and also in humans, through the use of neuroimaging^46,47. A single-neuron recording study in humans showed that firing of hippocampal neurons selective for individuals or faces decreases when very similar, yet non-identical stimuli (lures) are presented, providing evidence for pattern separation⁴⁸. The selectivity of hippocampal neurons for the stimuli correlated with memory of those items, and the proportion of neurons showing such selectivity was higher in patients with good learning performance than in individuals who performed poorly.

Temporal code.

In addition to firing rate, increasing evidence suggests that neuronal spike timing with respect to network oscillations is another critical component of the neural code^49–54. In rodents, position during navigation is represented by phase precession of hippocampal cell spikes relative to the theta oscillations^55,56. A similar phase code for navigational goals may exist in humans: neurons have been reported to respond to various goals with the same rate of firing, but at different phases of the theta oscillation⁵⁷, and phase precession in humans was recently described in a navigation task with respect to both place and goals⁵⁸.

‘Free’ recall?

During free recall, mental objects spontaneously emerge, and thus free recall is sometimes perceived as a process over which we have no control. Indeed, it seems to be a neuronal operation that occurs internally, in the absence of the physical object.

One analysis identified category-specific cortical blood oxygen level-dependent (BOLD) signal patterns present while participants studied three categories of pictures and also just before free recall of those categories⁵⁹. In another study, a subset of MTL neurons were reported to show remarkable selectivity for particular video episodes²⁹, with responses robustly sustained throughout an episode (FIGS 1b,2a). Remarkably, in the entorhinal cortex and hippocampus, but not in other temporal or frontal regions, the cells active while the patient was viewing an episode started firing just before the patient reported spontaneous recall of the episode²⁹ (FIG. 2b,c). Strikingly, the (normalized composite) firing rates of these neurons ramped up 1 s or more before the patient’s report of recollection. It is therefore conceivable that conscious recollection of the episode follows the reactivation of the same neuronal pool that is active when one is viewing the episode. These results unveil a neuronal signature that precedes free recall (FIG. 2c). The actual experience of free recall may be an ‘afterthought’ of neuronal events in the hippocampus, entorhinal cortex and, possibly, other brain regions not probed using such recordings. Such a neuronal signature may serve as a basis for potential preconscious modulation of human memory (BOX 1).

Fig. 2 |. Responses of medial temporal lobe neurons during encoding and free recall of episodes.

a,b | Trial-by-trial responses of neurons to successive film and television clips. A selection of 48 different episodes (film and television clips) were presented to a patient in three different viewing sessions (only a few are shown here, each depicted by one illustrative frame.) A single neuron in the right entorhinal cortex was activated during free viewing (part a) and recall (part b) of an episode consisting of a few segments from the television series The Simpsons. Note that the response was sustained throughout the blank period (shown after the clip of The Simpsons) even though the clip was no longer on the screen. In part b, the increased firing rate (FR) response indicating free recall of The Simpsons clip started more than 1.5 s before the verbal expression of that recollection. c | Ratio of FR during free recall of events (video clips) to the baseline FR averaged across 20 hippocampal and entorhinal cortex cells (Gaussian smooth bins, 200 ms; the shaded area denotes the standard error of the mean). Each of these cells responded with a selective sustained response to a particular clip (preferred clip; green line) in an earlier viewing session (that is, encoding). The response during the recall of another clip (neutral clip; purple line) that did not elicit a response during viewing is also plotted for comparison. Note the increased FR in the 2 s before free recall of the preferred clips (green line) persisting into the recall report (zero time denotes onset of verbal recall). Parts a and b adapted with permission from REF.¹¹⁸, MIT Press. Part c is based on data from REF.²⁹.

Box 1 |. Editing memory and will.

Decoding of memories or intentions from native neuronal activity presents an opportunity for modulation of specific brain functions. Such an intervention may be used to enhance motor, affective or cognitive faculties. Indeed, this is one of the central goals of brain–machine interfaces^97,119 as well as deep brain stimulation¹²⁰, which has proven quite effective in alleviating symptoms of Parkinson disease, but has over the past decade also been explored as treatment of various neuropsychiatric disorders¹²¹ and even in the enhancement of memory in the treatment of dementia or head injury¹²². The increasing abilities to decode memories and volitions sometimes even in the preconscious state set the stage for modulation of neuronal activity towards the desired behavioural outcome. The volitional signals from the posterior parietal cortex are already used in development of brain– machine interfaces for individuals with paralysis, in essence translating thought into action⁹⁷. These emerging capabilities will be fortified by high-channel recording devices^123,124 as well as the management of the resulting voluminous data afforded by artificial intelligence¹²⁵.

A cardinal challenge in memory modulation is the enhancement or the inception of specific memories. In rodents, in which selected engrams are associated with activation patterns of specific place cells, such a challenge has been met using optogenetic techniques¹²⁶. In another study, manipulation of place cells replayed during sleep in rodents incepted a memory for a particular place, which was evident by the preference of the animals for this place in subsequent waking behaviour¹²⁷. Given these findings, will we enter an era where human memories and volitions can be implicitly edited at a preconscious stage or even during sleep? already, clinical trials involving development and implementation of personalized deep brain stimulation involving interrogation of patient-specific brain circuits are under way^128,129. Personalized action invariants may be identified in the intentional posterior parietal cortex system and used to guide brain–machine interfaces⁹⁷. Such personalized brain modulation might include not only decoding and enhancing of human memories but also strengthening or deletions of specific wanted or unwanted memories or volition. one could envision the therapeutic potential of manipulating specific memories, such as in post-traumatic stress disorder¹³⁰, or specific volitions or compulsions, such as in Tourette syndrome or in obsessive–compulsive disorder¹³¹. However, we must also keep in mind ethical concerns for unwanted manipulation of the human mind at the volitional–conceptual interface.

Another neuronal signature preceding conscious recall — again by 1–2 s — is short bursts of high-frequency oscillations known as sharp wave–ripples (SWRs), generated in the hippocampus and associated with highly synchronized neuronal firing in this brain region. In rodents, SWRs have been linked to reinstatement and consolidation of stored representations of navigational experience⁶⁰. Disruption of SWRs during both the sleep state and the awake state has been shown to impair subsequent performance on navigational spatial memory tasks^61,62, whereas their prolongation was recently shown to enhance memory⁶³. In recordings from intracranial depth electrodes in patients with epilepsy, SWRs present during encoding of visual information were reactivated with some specificity before conscious recall⁶⁴. Furthermore, during recall, high-order visual areas showed content-specific reactivation coupled to SWRs. These data suggest that the neuronal signature preceding reported recall involves hippocampal–neocortical dialogue. Indeed, another study showed that coupled ripple oscillations between the MTL and the temporal neocortex preceded successful memory retrieval⁶⁵.

What is the relationship between these SWRs preceding recall and hippocampal concept cells? In rodents, sequences of hippocampal place cell activity, representing past spatial information, are briefly replayed during SWRs. Recent recordings in humans demonstrated that SWRs reflect bursts of neuronal firing organized in sequences that are replayed during successful memory retrieval⁶⁶. It is thus plausible that these are synchronized bursts of the concept cells shown to be reactivated before free recall of specific content encoded by these cells²⁹.

Free from the senses

The MTL code transforms sensory signals about physical objects to generate mental objects for mental computations, a process evident during working memory^43,44, imagining stimuli such as faces, animals, objects or places⁴⁵ and during free recollections²⁹. By transforming external stimuli to mental images and concepts, the brain enables human thoughts to transcend the physical limitations of space and time. That is, unlike in the physical world, single neurons can use mental representations to enable mental travel in space and time⁶⁷.

Concept cells may reflect an extension of coding principles by the entorhinal–hippocampal system beyond the spatial domain. The coding for space that has been studied in rodents reflects primarily a two-dimensional Euclidean space. However, this same neuronal architecture may have evolved to represent experience in a multidimensional non-Euclidean space, where the distance between objects may be a function of associative strength that changes on the basis of experience⁶⁸. The ability of concept cells to incorporate new associations⁴⁰ may be the basis for such changes, enabling an architecture of relational knowledge to become part of long-term memory⁴¹.

Coding along dimensions of non-spatial experience by the human entorhinal–hippocampal system may follow the same constraints and principles of the place cell system, grid cell system and other similarly specialized cell systems in rodents. For example, cells may fire in response to specific conjunctions or ‘places’ in a two-dimensional ‘space’ defined by the features of emotional facial expression and gender, or facial expression and facial identity²¹. In a recent study, hexadirectional BOLD signals were observed in the entorhinal cortex in a two-dimensional ‘bird space’ defined by the length of a bird’s legs and the length of its neck, where participants learned associations of specific combinations of these features with different objects⁶⁹. High-frequency MEG and intracranial EEG activity further suggests grid-like hexadirectional mapping in the anterior MTL of visual space during visual exploration²³. The analogy between rodent place cells and human concept cells may be limited, however, as the former may remap with only minor cue changes, whereas concept cells seem stabler across different situations and tasks, and similar considerations may apply to grid cells and other spatially tuned cells.

Recent reviews^12,70 further argue for representation of experience and knowledge by the cellular machinery of the entorhinal–hippocampal system and its connections to other brain regions, including the ventromedial prefrontal cortex and the posterior cingulate cortex. Such fundamental and general roles of this system were recently extended to the social domain, suggesting that navigation-like models may be applied to the social space⁷¹. In the social space, the position of an agent in the environment factors in not only the spatial coordinates of the agent in that environment but also the agent’s relationships — spatial, hierarchical and others — to other agents in the environment. In a recent study of freely ambulating patients with chronically implanted electrodes, their observing another agent navigating in the room instigated boundary-related modulations of MTL theta oscillations that were similar to those modulations during self-navigation⁷².

MFL–PPC code for volition

Electrical stimulation in the MFL and PPC

The definition of volition or voluntary action is challenging. Wittgenstein defined volition as “what is left over if I subtract the fact that my arm goes up from the fact that I raise my arm”⁷³, avoiding the complexity of human decision making affected by experience and goals. An important approach to volition has been the emphasis on its ‘stimulus independence’; that is, it is internally generated as opposed to externally triggered motor action¹. Such an approach must consider additional aspects of voluntary action, including the conscious subjective experience of such action, as well as its spontaneity, predictability and goal-directedness.

We have seen how electrical stimulation of the temporal lobe gives rise to experience akin to that of spontaneous memories. Analogously, electrical stimulation of the somatotopically organized supplementary motor area (SMA) and pre-SMA of patients undergoing neurosurgery gives rise to an ‘urge’ to perform specific movements⁷⁴. By contrast, stimulation of the primary motor area (M1) induced actual movements that patients viewed as being externally caused. These observations suggest that the intention to move may be elicited by the neuronal machinery of the SMA. Indeed, evidence suggests that the SMA is crucial for the urges that precede tics in individuals with Tourette syndrome⁷⁵.

Self-reports relating to volition for simple movements have limited value in assessing human will. More complex responses related to will were observed during electrical stimulation in the anterior part of the mid-cingulate region, part of the MFL, resulting in reports of ‘will to persevere’ by two patients. These responses involved a form of will or decision by the reporting patients, coupled with a perception of imminent challenge⁷⁶.

Some evidence suggests that parietal lobe stimulation can evoke the urge to move⁷⁷. Stimulation of the right inferior parietal lobule triggered a strong desire to move the contralateral hand, arm or foot, an intention to move the lips or less-specific intentions for movements that patients were unable to precisely describe⁷⁸. Thus, parietal stimulation was suggested to elicit the conscious intention to act, whereas SMA stimulation elicits the urge, reflecting the imminence of movement. Interestingly, patients with parietal lesions can report the execution of movements but not when they first became aware of their intention to move⁷⁹. Whereas increasing the current during MFL stimulation eventually yields a movement, increasing the current delivered to the parietal lobe does not induce overt movement, yet patients still report movement. It has been suggested that the volition-related responses in the parietal lobe may relate to the sensory predictions of the effects of action, whereas the MFL responses to stimulation relate to the motor commands themselves^13,77.

MFL and voluntary action

A large clinical literature connects the MFL to voluntary action. Unilateral resection of the SMA usually results in loss of voluntary contralateral movement depending on somatotopic organization⁸⁰. However, such deficits are usually transient, whereas bilateral mesial frontal lobe damage leads to permanent cessation of voluntary movement and akinetic mutism⁸¹. Damage to the dominant SMA may result in ‘alien hand syndrome’, in which the contralateral hand seems to move but without the patient perceiving it as voluntary⁸². The parietal variant of this syndrome differs in that the patient does not perceive the alien hand as belonging to the self⁸³.

In the 1960s, Benjamin Libet studied the so-called readiness potential, an EEG signal probably generated in the MFL that precedes the onset of voluntary movement by several hundred milliseconds⁸⁴. Libet asked participants to perform a movement when they “felt like it”, but requested that they notice the time when this happened so that they could later report it on a clock. By measuring a subjectively reported timing (denoted by W) of the volitional experience or will, Libet showed that the EEG signal preceded not only the volitional act but also the volitional experience. Such experiments are notoriously difficult, as they rely on the participants reporting a subjective conscious decision with high temporal acuity^85,86.

Many studies followed Libet’s original work, using extracranial EEG^84,87,88, intracranial EEG⁸⁹, MEG⁹⁰ and functional MRI^91,92 to demonstrate neural correlates of decision processes hundreds of milliseconds and even seconds before the moment that participants reported having consciously decided. Functional MRI studies identified an early preconscious BOLD signal in the frontopolar cortex and anterior cingulate cortex (ACC) encoding the content of a decision made several seconds later⁹¹. Such neuronal signatures in the MFL spanning from the SMA and pre-SMA region to the frontal pole anteriorly arguably reflect progression from simple ‘flicker of finger’ actions caudally to more cognitively complex decisions rostrally⁹³.

Libet’s paradigm rests heavily on the readiness potential, the gradual changes in neural activity approaching the W point. Therefore, it has faced considerable criticism on methodological and conceptual grounds. Schurger et al.⁹⁴ pointed out that the readiness potential may simply reflect random fluctuations of the time-locked autocorrelated EEG signal. Conceptually, Libet’s paradigm has stirred considerable debate because of potential implications for the philosophical question of free will. The arbitrary decision to move a finger, a decision with no significant consequence, obviously does not reflect human decision making in real life. To address this, Maoz et al.⁹⁵ compared arbitrary decision making with a more deliberate process in which the choice does make a difference and claimed that the readiness potential present in the former was absent in the latter.

The readiness potential is a gross signal with limited spatial resolution. Recording from single neurons in a paradigm similar to Libet’s, prominent gradual increases (FIG. 3a) or decreases (FIG. 3b) in the firing rate of MFL neurons (specifically, in the SMA, pre-SMA and ACC) were observed 1,000 ms or more before participants’ reported awareness of the urge or will to act⁹⁶ (that is, W; see FIG. 3a–c). Additionally, there was a progressive recruitment of these neurons as W was approached⁹⁶ (FIG. 3d). Thus, changes in MFL population activity preceded the recognized internal state of intention (FIG. 3c,d). Decreases in the firing rates of some neurons (FIG. 3b), combined with the excitatory patterns of others (FIG. 3a), may reflect the inhibition of alternative action plans during this preconscious period. The final single conscious intention may thus result from a gradually evolving balance between neuronal excitation and neuronal inhibition. Moreover, activity-encoded information in this preconscious interval before volition could predict the timing and content of the conscious decision. A similar study with a dual-choice task showed that the gamma-frequency band of intracranial cortical surface EEG could predict a driver’s decision to turn left or right, well before the conscious decision (or volition) of the participant⁸⁹.

Fig. 3 |. MFL neuronal activity preceding volition.

a,b | Histograms showing that neurons in the medial frontal lobe (MFL) significantly increased (part a) or decreased (part b) their firing rate (compared with the baseline) before W, the reported onset of volition (t = 0; solid vertical line). The number of trials is shown on the left of each histogram. Error bars denote the standard error of the mean. The neurons shown are from the left supplementary motor area (LSMA), right supplementary motor area (RSMA), right pre-supplementary motor area (RpSMA), left pre-supplementary motor area (LpSMA) and right rostral anterior cingulate cortex (RACCr). c | Corresponding average normalized responses (subtracted from the baseline) for MFL neurons with increasing (purple) or decreasing (green) firing rates before W. d | Percentage of MFL units showing a marked change in firing rate compared with the baseline as a function of time before W. The purple and green traces show the corresponding analyses restricted to multiunit activity (MUA; purple) and single-unit activity (SUA; green). Adapted with permission from REF.⁹⁶, Elsevier.

Although there is considerable evidence for preconscious changes in neuronal activity before a conscious decision, a hypothetical ‘threshold crossing’, which may explain why the W point is unique in time, is not clear. An extreme view may regard such a process as entirely random; however, the activity of populations of neurons that are gradually recruited and that alter their firing rate rides on a background of millions of neurons creating immense electrical variability that, to an extent, can be measured by local field potentials. One proposed model holds that spontaneous fluctuations in neural network activity near a threshold may bring the motor system closer to or beyond the threshold for motor intention and action in a stochastic manner⁹⁴. Accordingly, the readiness potential and changes in neural activity approaching the W point may reflect fluctuations of the time-locked autocorrelated EEG signal.

PPC and voluntary action

Recent opportunities to record single-neuron activity in the human PPC have added important insights into the role of this region in voluntary action and enabled synthesis of human and non-human primate research. These opportunities are offered in a few unique settings where neuroprosthetic devices have been chronically implanted in patients with tetraplegia. Accumulating evidence suggests that the PPC in monkeys and humans serves as a higher-order centre representing intentions for movements⁹⁷. Initial single-unit research in monkeys delineated intention signals in the PPC preceding voluntary movements of specific body parts^98,99. A map of intentions was found in the PPC, specifically in the intraparietal sulcus: different subregions were selective for intending reaches (the parietal reach region), saccades (the lateral intraparietal sulcus) and grasps (the anterior intraparietal sulcus)¹⁰⁰. Recent studies of the PPC in patients with tetraplegia who have brain machine–interface implants have begun to unveil various action-related properties encoded by single neurons⁹⁷. Intended intentions for movement of most body parts can be decoded from small populations of neurons¹⁰¹. Goals of movements and the trajectories to obtain these goals can also be decoded rapidly from PPC neurons. Data from macaques and rodents show that inhibiting or lesioning the lateral intraparietal sulcus impairs free choices but does not impair instructed or sensory-determined choices^102,103, supporting a role for the PPC in voluntary action.

Taken together, the monkey and human studies suggest that the PPC processes information in a high-dimensional feature space⁹⁷. The PPC then may generate action invariants characterized by fewer higher dimensions in this space, such as goals. This strategy of information processing is similar to the abstraction seen in the conceptual space generated in the MTL. There have been some suggestions, albeit still controversial, that motor actions are composed of modules, primitives or building blocks at different levels of motor hierarchy. These primitives are spatio-temporally invariant, and thus a relatively limited number of stored primitives can generate a large variety of actions¹⁰⁴. Microstimulation of the motor cortex and especially in the SMA evokes complex movement involving multiple muscles and joints, and complex movements are associated with single-neuron activity. Similar to the sequential activation of place cells in path integration, a ‘synfire’ chain of neuronal activation has been proposed for the sequential integration of action¹⁰⁵.

Conceptual–volitional interfaces

The nervous system interacts with the external world by assimilating sensory inputs and generating motor acts. Various mechanisms have evolved for this sensory–motor bridge, the shortest being reflex mechanisms. However, this bridge has become increasingly complex and temporally extended in evolution, enabling release from the immediacy of action¹⁰⁶ and culminating in high computational complexity. Although memory retrieval and voluntary choices are largely affected by external stimuli and their interactions with other internal representations, they can also be generated internally. However, this achievement relies on a complex architecture of relational conceptual knowledge available for retrieval and subsequent conscious elaboration and computation that can guide voluntary action.

Considerable recent research has focused on ‘the default mode’, a postulated internal mode of cognitive operations that are not sensory dependent but internally driven¹⁰⁷. Such a mode must operate on mental objects, memories, task sets and action invariants. Although discussion of the default mode is beyond the scope of this Perspective, I focus on the interactions between the MFL–PPC system and the MTL, termed ‘conceptual–volitional interfaces’ (CVIs). These interfaces extend the direct sensorimotor interfaces, enabling internally guided behaviour.

CVI during choice

Increasing evidence suggests that information retrieved from memory serves in decision making leading to voluntary action. The MFL presenting task sets with potential outcomes for making a voluntary choice interfaces with the MTL, which provides retrieved conceptual knowledge to guide choice. Supporting this model are recordings from 1,430 neurons in the MTL (hippocampus and amygdala) and the MFL (the ACC and pre-SMA) during a verbal memory-based choice. MFL cells showed phase-locking with MTL theta and gamma oscillations that predicted behavioural performance¹⁰⁸. In a task combining spatial memory and goal-directed behaviour of navigating to specific locations, both MTL neurons and MFL neurons fire in a similar manner of phase precession coordinated with theta oscillations⁵⁸.

In rodents, sequences of firing of place cells during immobility between periods of navigation reflect not only past experience but also future paths during goal-directed behaviour¹⁰⁹. However, the choice of future action based on past experiences is determined by coordinated hippocampal– prefrontal activity¹¹⁰. Similarly, human behaviour may use the interface of sequences of hippocampal–entorhinal cell firing reflecting replay of past episodes to provide choices among pre-play sequences for future goal-directed behaviour. Both MTL cells and MFL cells show phase precession — that is, coordinated firing with theta oscillations: MTL cells with respect to locations and MFL cells with respect to goal trajectories.

CVI in working memory

The ability of neural representations to remain available beyond a stimulus’s physical presence for short-term maintenance is explicit at the single-neuron level. Recent evidence points to cooperation between MTL neurons and MFL neurons in working memory. Concept neurons in the MTL remain active as the specific item they represent is held in memory, and indeed the content of memory can be predicted from the activity of these neurons. However, the quality of memory, measured by load and reaction time, is encoded by MFL neurons in the ACC and pre-SMA regions⁴³. The coupling between content-specific persistent firing in the human hippocampus and local field oscillation in the cortex is also reported to be important for working memory¹¹¹. Persistent neuronal activity when a target location is held in memory before an action (eye movement to the target) was also demonstrated in the PPC of macaques, albeit recordings in the MTL or the MFL were not performed in that study⁹⁸.

Volitional control of concept cells

MTL representations not only affect MFL-mediated volition but also may be modulated by volitions. Cerf et al.¹¹² reported on a brain–computer interface involving four MTL neurons, two of which were concept cells, with each coding for a different person. In this study, a hybrid image of these two people was presented on a screen and the participant was instructed to enhance (or fade in) the image of one person by concentrating on it. The recorded activities of the concept neurons were used to change the image on the screen in real time, creating a closed-loop brain–computer interface controlled by the participant’s neurons. Participants learned the task in just a few trials and were able to fade in the intended image. However, sometimes, a curious ‘competition’ occurred between the senses and volition, when the person nearly failed, and the wrong image was almost entirely faded in. Presumably, in these cases, the neuron responding to the increasingly visible distractor image became more active, while the neuron of the target image did not receive its external activating stimulus. However, the internal representation of the desired stimulus modulated by volition was sufficient to ‘rescue’ the shift towards the desired image, which was faded in at the last minute. Thus, in a sense, the participants’ volitionally mediated internal representations overrode sensory inputs.

CVI in binocular rivalry

In binocular rivalry paradigms, different images are shown separately to each eye, leading to transitions between two conscious experiences in the absence of any stimulus change. A non-human primate study identified neurons in the superior temporal sulcus that were active during subjective perception rather than presentation of the image to the retina¹¹³. In a microelectrode study of the human MTL and MFL during a binocular rivalry task (FIG. 4), cells responsive to one of the stimuli increased their firing rates before the emergence of conscious perception of that stimulus, suggesting a specific preconscious neuronal signature¹⁶ (FIG. 4a,b). Moreover, before the preconscious MTL signature, the activity of cells in the MFL also increased, although not in a stimulus-specific manner (FIG. 4c), suggesting an MFL-to-MTL sequence of neuronal activity in the mediation of conscious experience. Thus, in the same manner that MTL neurons may prime MFL-mediated volitional choice, MFL neurons may prime the MTL conceptual system during the generation of conscious experience. Although one may not rule out definitively that neuronal responses may be related to the report rather than to the switch, in a control experiment, the images were presented without rivalry, yet with the same transitions that were internally generated by the patients in the binocular rivalry condition, but now externally driven. In this condition (denoted ‘replay’), the onset of neuronal activity was significantly later than in the ‘rivalry’ condition, both in the MTL and in the MFL (FIG. 4). This unique experiment then suggests an MFL–MTL interface in the representation of conscious experience, and is thus compatible with the global workspace theory of consciousness.

Fig. 4 |. MTL and MFL single-neuron activity in binocular rivalry.

a | Activity of a single neuron in the amygdala during a binocular rivalry paradigm and during a control replay. The neuron responded to the image of a snake but not to the image of the actor Annette Bening. The panel on the left shows the neuronal firing around the report of transition onset to the snake image (Time = 0; end of Annette Bening exclusive image perception) during binocular rivalry (purple) and replay (cyan). The peristimulus time histograms were computed with a moving square window of 200 ms. Shaded areas represent the standard error. Purple and cyan horizontal bars denote periods where the instantaneous firing rate (FR) was markedly different from the baseline FR in rivalry and replay, respectively. Note the earlier activation during rivalry as compared with replay and the gradual increase in the FR of the neuron as the perceptual transition was approached¹⁶. This activation did not occur in the opposite transition, from the snake image to the image of Annette Bening (shown on the right). b | Medial temporal lobe (MTL) population-level activity started earlier during rivalry than during replay. Distribution of normalized FR of neurons that significantly increased or decreased their FR around the onset of the transition to the unit’s preferred image during rivalry transition (upper panel) versus neurons doing so during replay transition (lower panel). The responses for each neuron are colour coded (dark blue represents 0; dark red represents 1) with periods of significance at the unit level highlighted. c | Medial frontal lobe (MFL) activity preceding the perceptual change in rivalry. The average percentage change in FR (mean ± standard error) across all rivalry-active units in the MFL is shown. These increases were not stimulus selective. Note the very early anticipatory activity (almost 2 s before the report) of the rivalry-active units during rivalry. This activity occurred earlier than the activity in the MTL. Adapted from REF.¹⁶, CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/).

Sensory limits and spontaneity

According to Kant, the mind springs from two main sources: sensory receptivity for representations and the spontaneous production of concepts¹¹⁴. As described earlier herein, the transformation of physical stimuli into mental objects in the MTL represents one way that humans can become less dependent on the senses. Such operations provide ‘freedom from immediacy’ of decision making or action¹⁰⁶, and allow spontaneous memories and volitions. Yet how do these spontaneous memories and volitions arise? Here I consider the spontaneous fluctuation of neuronal activity and its inherent variability as a biological mechanism that may move a system beyond a threshold for conscious experience.

One proposed mechanism that might involve such variability is the generation of periodic variability by local field oscillations, which in turn may occasionally enhance activity beyond the threshold required. This view is consistent with the stochastic model of slow fluctuations towards a motor threshold, mentioned earlier herein⁹⁴. Accordingly, against a resting background of ultraslow fluctuations of low amplitude, integration of ongoing stochastic activity, either spontaneously or driven by a task, crosses a dynamic threshold and ignites, leading to content-specific awareness¹¹⁵. How this leads to the conscious experience of volition or memory recall is not clear, however, as the ignition dynamics also characterize bursts of activity in other states, such as slow-wave sleep, that do not evoke awareness. Intermediate (theta to gamma) frequencies of neural activity must also be considered to provide a temporal framework for neuronal firing. In this framework, fluctuations of electrical activity in the brain simultaneously provide a temporal matrix for neuronal firing, as well as variability that may drive a system beyond a threshold. However, a strict threshold model may be too limited, and perhaps a probabilistic temporal window better fits the behavioural uncertainty as to the timing and content of the experience of memory or will. Such a probabilistic process is supported by the representation of retrieval confidence by single neurons in the human MTL¹¹⁶. Recent evidence places participants’ reports of awareness of intention to move in a probabilistic model with a prospective component related to the presence of readiness potential and a retrospective component of actual motor execution¹¹⁷.

Conclusions

Stimulation and recording from neurons in the human brain, in people who can declare their conscious experience, reveal neuronal populations in two systems — the MTL system and the MFL–PPC system — that provide codes for stimulus-free, internal generation of memory and voluntary action, respectively. The formation of abstract invariants of experience — concepts and actions — governed by these neuronal codes enables recollections or imagery as well as volitional acts through CVIs. MTL neuronal trajectories in cognitive or conceptual space may determine future goal-directed volitional action invariants represented in the MFL– PPC system, and neuronal activity in the latter system may influence the formation of mental objects and concepts. Future research will need to address the stochastic nature of these codes, how neuronal trajectories in these two systems interact to produce continuous goal-directed behaviour and how other areas, such as subcortical nuclei, interact with these systems. It will also be of considerable interest to examine how these neuronal models of human thought may inform novel architectures of artificial intelligence and potential modulation of human cognition in health and in disease (BOX 1).