More than spikes: Common oscillatory mechanisms for content specific neural representations during perception and memory

Abstract

Although previous research into the mechanisms underlying sensory and episodic representations has primarily focused on changes in neural firing rate, more recent evidence suggests that neural oscillations also contribute to these representations. Here, we argue that multiplexed oscillatory power and phase contribute to neural representations at the mesoscopic scale, complementary to neuronal firing. Reviewing recent studies which used oscillatory activity to decipher content-specific neural representations, we identify oscillatory mechanisms common to both sensory and episodic memory representations and incorporate these into a model of episodic encoding and retrieval. This model advances the idea that oscillations provide a reference frame for phase-coded item representations during memory encoding and that shifts in oscillatory frequency and phase coordinate ensemble activity during memory retrieval.

Introduction

Reliable, flexible representations of the external world are crucial for encoding sensory perceptions into memories. Decades of research have documented changes in neural firing rate to varied and unique (“content specific”) environmental features. Beyond studies of single neuron firing rates, more recent work has investigated how the synchronized activity of many neurons, which results in oscillations of the local field potential (LFP) and the electroencephalogram (EEG), contributes to neural representation. Local and inter-areal synchronization supports dynamic coordination of incoming sensory information [1,2], inter-areal interactions [3,4], and synaptic plasticity [5,6], each of which are integral to both perception and memory. Thus, we argue that oscillations may be particularly relevant for encoding perceptual experiences into memory representations by coordinating spatially distributed and co-activated neuronal groups (i.e. cell assemblies). These coordinated representations thus constitute memory traces, or “engrams” [7,8].

Neural oscillations influence neural activity and human behavior

We begin by addressing general concerns regarding the role of oscillations in the “neural code”, which is distinguishable in two senses. Coding in a “strong” sense implies that neural computation is causally driven by some configuration of spikes or extracellular signal, which implies that the brain is using this code to represent information. Coding in a “weak” sense is descriptive and refers to the decodability of some mental state by observing a brain state characterized by different metrics such as power, phase, and frequency. Although there is limited evidence for oscillatory coding via ephaptic effects in the “strong” sense, considerable evidence using pattern classification and information theoretic approaches indicates a role for oscillatory coding in the “weak” sense, as discussed below.

The LFP is primarily a manifestation of aggregated synaptic activity and other transmembrane potentials [9,10]. Given that neurons process information through spiking, the LFP may only reflect a byproduct of neuronal firing without any further functional significance [11]. However, a small but growing body of evidence challenges this idea [12-14] , finding that the local oscillatory environment causally affects neural firing via ephaptic coupling (reviewed in [10]). While there is ongoing debate regarding the functional role of ephaptic coupling and a lack of evidence for similar effects in humans, these studies nonetheless suggest that changes in the LFP can impact neural firing [15].

Human stimulation studies also implicate specific oscillatory frequencies and phases in perception and memory. Helfrich et al [16] used transcranial alternating current stimulation (TACS) to entrain endogenous 10Hz posterior alpha activity during a target detection task. Targets presented during different oscillatory phases of the entrained rhythm led to differential behavioral performance, causally implicating oscillatory phase in sensory processing. In another study, Polonia et al [17] used TACS to induce different patterns of frontal-parietal theta phase relations during a letter recognition task. Whereas zero-lag (in-phase) patterns improved behavioral performance, 180° phase lag (anti-phasic) patterns diminished performance. Similarly, it has been observed that memory formation for words can be modulated by in-phase vs. anti-phasic electric stimulation of rhinal cortex and hippocampus [18] and that entorhinal stimulation modifies hippocampal theta phase to enhance spatial learning [19]. Although stimulation strength in these studies are orders of magnitude larger than that which occurs endogenously, these studies indicate that specific patterns of oscillatory phase support memory encoding and recognition, probably by promoting task-relevant neuronal communication and synaptic plasticity (reviewed in [5]). In sum, recent animal and human studies suggest that oscillations extend beyond a correlational epiphenomenon and are thus a plausible substrate contributing to the neural representation of an engram.

Independent contributions of LFP power and phase to neural representation

There is now abundant evidence for stimulus specific single neuron activity in humans (reviewed in [20]). Several studies have bridged the gap from single neurons to content-specific patterns in the LFP, showing both category [21,22] and stimulus [23,24] specific neuronal firing and LFP responses at low (below 10 Hz) and high (>30 Hz) frequencies in the human temporal lobe. Thus, activity in the theta and gamma bands appear particularly relevant for neural representation [25] (Figure 1A).

Figure 1.

Content-specific sensory and episodic information can be decoded from frequency and phase-specific oscillations. A) Importance map for decoding visual letters using human intracranial recordings as a function of time and frequency. Decoding is best when using power in the theta and high-gamma bands. Modified with permission from [25]. B) Decoding accuracy for naturalistic auditory stimuli using power or phase in monkeys (left) and humans (right). Note that decoding accuracy is largest using theta phase in both species and that theta phase is better than firing rate in monkeys. Modified with permission from [28]. C) Heat map depicting the relative contribution of power and phase (represented jointly in the complex plane) to representations of facial expressions. Maps are shown separately in different bins of the complex plane at two different frequencies. Note that representations of different aspects of the faces (eyes vs. mouths) are multiplexed at different frequencies and phases. Reproduced with permission from [29]. D) Phase-amplitude coupling (PAC) occurs at a variety of phases (left) and frequency combinations (right) in humans, fulfilling two of the requirements for a coding scheme based on frequency and phase-specific neural firing/PAC. The distribution of coupling phases may be either monophasic or biphasic (left). Whereas phase-providing frequencies varied in the delta-alpha band, amplitude-providing frequencies varied in the beta-gamma bands. Reproduced with permission from [47]. E) Difference in phase-locking for items encoded at a flickering frequency of 6Hz versus 10Hz (left). Successful recognition of items encoded at 6Hz was accompanied by frequency-specific phase-locking (right), implicating frequency-specificity in both encoding and retrieval. Modified with permission from [55]. F) Primate neurons fire to frequency-specific (lower) inter-regional oscillatory phase coupling patterns, which constitute an “internal receptive field” (IRF) for the neuron (upper left) and are a strong predictor of firing rate (upper right). Recreating an assembly’s IRF may be sufficient to activate it in support of memory retrieval for a content-specific event. Modified with permission from [36]. See original text for full details in each panel.

Power and phase (the squared-amplitude and momentary deflection angle of an oscillation, respectively) are theoretically independent aspects of the LFP [26] which could allow one to decode complementary information (see Box 1, B-C). Analyzing recordings from visual and auditory cortex during presentation of naturalistic stimuli, Belitski et al [27] demonstrated that power at different frequencies in the 1-10Hz band conveys unique information about the stimuli from both spike rate and from each other. Activity at higher frequencies (>50 Hz) carried more redundant information between bands and with spike rate. In another study, power and phase measurements from human electroencephalography (EEG) and monkey LFP recordings were used to decode naturalistic auditory stimuli [28]. Low-frequency oscillatory phase predicted representations for different sounds better than power in both species (Figure 1B), and in monkeys, oscillatory phase was a better predictor of perceived sound than firing rate. Finally, Schyns et al [29] directly compared the information in oscillatory frequency, power, and phase to the representation of different faces using human EEG, finding that low frequency (below ~15Hz) phase represented significantly more information than power. Moreover, they found that the combination of power and phase was better than either measure alone (Figure 1C), indicating a multiplexed coding scheme (discussed below). Together, these and other studies [30,31] confirm that content-specific representations can be decoded from aggregated neural activity.

Box 1.

Content specific decoding via oscillations. A) Hypothetical firing rate of a single neuron showing content specific firing for the orange stimulus (upper). Alternatively, firing rate may not predict content specific representation in perceptual and memory tasks (lower; as in [28,33,35]). Under these conditions, consideration of the power (B), phase (C), or both (D) of the LFP signal may provide additional information. B) Power and frequency-specific neuronal firing. Traces show idealized 2 and 4 Hz oscillations elicited by the red and blue stimuli along with associated spiking of a single neuron. Content-specific neurons fire at specific frequencies, but across an extended and therefore unspecific phase range. Content specific information (right panel) in this scenario can be recovered via frequency-specific power increases or by considering neuronal firing as a function of power [22,24,25,27,31]. C) Layout similar to B for a phase-specific firing scheme. In this case, the neuron fires at a specific narrow oscillatory phase across several frequencies. Content specific information is conveyed by the phase of oscillatory activity or by considering neuronal firing as a function of phase [26,28]. D) Layout similar to B for a power- and phase-specific firing scheme. In this case, neurons fire at specific frequencies and phases. Content specific information is conveyed by neural firing as a function of frequency and the phase of oscillations or by these measures alone [29,31,36,38,45-48]). For clarity, low-frequency oscillations are shown, although the principles depicted here generalize to other frequencies. Similarly, other possible mechanisms of content-specific representation, such as population coding or inter-regional measures [60,61], are not shown.

For human memory, phase-based intracranial EEG measures are better predictors for successful memory encoding of words than power-based measures [32]. Moreover, during a card-matching task, low-frequency (~2Hz) phase was a better predictor of correct performance than power in several brain regions [26]. Finally, hippocampal theta-band phase locking of single unit firing, instead of firing rate or power, predicts successful memory encoding [33]. These findings underscore phase as a coding mechanism and suggest that, in some cases, the LFP phase (which partially indexes the spatiotemporal coordination of spiking) may reflect behaviorally-relevant information which is not identifiable solely based on the spiking activity of the sampled neurons. [21,27,28,34,35]. We now explore how oscillations in various frequencies may combine to form content-specific representations.

Content-specific representations manifest in multiplexed oscillatory power and phase

Oscillations can occur simultaneously at multiple frequencies, raising the possibility that power and phase at different frequencies could represent distinct information (i.e. signal multiplexing, as in FM radio; Box 1D). Increasing empirical evidence suggests such a frequency-multiplexed coding scheme [29,36-39], which may allow for multiple processing streams [2,37,40] to code information with enhanced capacity [41,42]. During perception, frequency-specific power and phase represent different aspects of facial features [29] (Figure 1C) and human speech [38]. Regarding memory, inter-areal phase synchronization in distinct low frequency bands predicts different types of memory retrieval [37] and rodent low and high gamma activity convey unique spatial information [43]. Thus, multiplexing may be a general motif in neural representation during perception and memory [44]. We next consider how subcomponents of the LFP may combine to form content-specific representations during episodic encoding and retrieval.

Frequency and phase specific assembly coding for percepts and engrams

Several lines of evidence support a model whereby sensory and episodic experiences are encoded by cell assemblies firing at specific frequencies and phases of an underlying low-frequency rhythm (Box 1D, reviewed in [7]). First, low-frequency oscillations convey unique information about sensory stimuli between adjacent frequencies [27]. Both neuronal firing and gamma responses may be locked to specific phases of low-frequency oscillations [24,45], contributing to phase-amplitude coupling (PAC) between low-frequency phase and gamma amplitude [10]. In turn, spiking and PAC exhibits the diversity of frequencies and phases necessary for a frequency and phase-dependent coding scheme [33,36,46,47] (Figure 1D). Second, in accord with the notion of multiplexing, some human neurons phase-lock to multiple frequencies [46] and high-gamma amplitude may phase-lock to multiple low frequency phases simultaneously [39,47]. Third, prior work has found that content-specific representations manifest as neuronal firing and/or PAC at distinct phases [45,48-50], similar to the well-known phase precession of hippocampal place cells. Thus, single neuron firing and gamma are largest and convey the most content-specific information at particular frequencies and phases, suggesting that assemblies are organized by the frequency and phase of low-frequency oscillations [47,51]. Given that single-unit phase-locking [33] and PAC [52-54] are strongly associated with learning, they may also serve to transform sensory representations into frequency and phase-coded engrams.

Two mechanisms, phase entrainment and phase resetting, track rhythmic structure in the environment and could optimize encoding by aligning periods of local neuronal excitability with incoming rhythmic or arrhythmic sensory inputs, respectively. First, phase entrainment tracks rhythmic components of the external world such that the LFP mirrors the specific frequencies and phases of the sensory input [1,2,55]. Entrainment manifests as an increase in power and phase consistency at low frequencies [2,55] and may primarily determine the frequency at which cell assemblies are phase-coded. Second, phase resetting (i.e. stimulus-related phase shifting) occurs in response to both arrhythmic sensory inputs [38] and with “active” environmental sensing, such as rhythmic sampling of the environment [1,56]. Importantly, resetting is commonly observed in memory tasks [26,32,56,57] and is enhanced for items which are subsequently remembered [56], probably by promoting synaptic plasticity (reviewed in [58]). Thus, phase resetting may primarily specify the phase for an assembly by coordinating sensory inputs onto different phases of the underlying rhythm, forming cell assemblies which fire at unique phases for different sensory inputs. Together, entrainment and resetting can produce consistent oscillatory frequency and phase patterns across repetitions for a particular sensory input yet distinct patterns for different inputs, thereby organizing the frequency and phase of cell assembly formation, respectively.

Decoding oscillatory engrams during memory retrieval

How might phase and frequency coded cell assemblies be reactivated during retrieval? It has recently been reported that frequency-specific patterns of sensory inputs which occur during encoding reoccur during successful memory retrieval [55] (Figure 1E). Similarly, Canolty et al [36] found that spatially distributed groups of primate neurons are phase-locked to narrowband low-frequency oscillations, which have a particularly strong effect on neuronal spiking [12,36]. These neurons increased their firing to frequency-specific patterns of inter-regional oscillatory phase which acted as an “internal receptive field” (IRF) for the assembly (Figure 1F). Other studies have shown that IRFs vary with different behavioral conditions [59] and that inter-regional patterns of oscillatory phase allow for decoding of content-specific information in both monkeys [60] and humans [61]. Thus, recreating the IRF of an assembly should be sufficient to activate it.

We posit that retrieval is driven by dynamically shifting the frequency and phase of the LFP to the internal receptive field “IRF” [36] of a cell assembly [7], leading to its reactivation. While initial evidence for this idea has been reported using PCA analyses of intracranial EEG data [62], our model predicts that subtle changes in frequency and phase within the canonical bands may also be important (i.e. 2Hz vs. 4Hz, Box 1). Other findings also support this view. First, the LFP frequency helps determine several neuronal properties, including spike threshold, spike timing, and coincidence detection, and may also reflect coordination between neuronal groups [63]. Such frequency modulations are therefore likely to contribute to both encoding and retrieval likelihood. Second, human low-frequency phase coordinates single neuron firing [33,46] and exhibits rich spatiotemporal structure which varies with behavior [64]. Third, remembering spatial versus temporal information leads to frequency-specific changes in large scale patterns of oscillatory phase in humans [37]. Finally and perhaps most importantly, inter-regional oscillatory phase patterns are related to performance during retrieval [17,18] and successful retrieval is accompanied by frequency-specific input patterns which occurred during encoding [55]. These findings indicate that one aspect of the engram may be reflected in large scale patterns of oscillatory phase. It may therefore be possible to identify engrams by deciphering these frequency and phase-specific oscillatory patterns.

Concluding remarks and outstanding questions

Here, we have argued that oscillations and spike rates may each be useful for detecting neural representations of percepts and engrams. The firing of an assembly codes for content-specific memory traces and the large scale oscillatory environment reflects assembly communication and activation. Cell assembly firing at different frequencies and phases of low-frequency oscillations provide a mechanism for content-specific representation during perception and episodic encoding. Frequency and phase-tuned cell assembly organization arises out of oscillatory phase entrainment and phase-resetting, which together track environmental rhythmicity during initial perception. Retrieval is possible via the reinstatement of the large scale patterns of frequency-specific oscillatory phase which occurred during encoding, serving to select a cell assembly representing content specific information. This model therefore provides an account of how oscillations may reflect the organization of cell assemblies during initial sensory experience, encoding, and retrieval. Thus, it provides a more unified account of how oscillations coordinate cell assemblies in both perception and memory.

Given that the neuronal recording techniques necessary to identify distributed cell assemblies are not available in humans, testing these ideas may provide an alternate way to deduce content specific memory representations in humans. Yet, the above ideas and available literature also leave many questions unanswered. Do ephaptic effects exist in humans? Can frequency and phase-specific neural stimulation evoke content specific memory traces in humans? What are the relative contributions of synchronization and desynchronization at different frequencies to content specific representations [37,48,65]? How sparse or spatially distributed are these representations [25,36] and how do different neural codes spanning different areas combine [26,34]? What is the relation between intra- and inter-regional content specific representation [60,61]? Resolving these issues may ultimately lead to tracking distributed content-specific memory representations in humans – i.e., to finding the “engram”.

Highlights.

• Oscillations causally influence human perception and memory

• Decoding multiplexed power and phase reveals content specific representations

• Percepts and engrams are encoded by frequency and phase-coded cell assembly firing

• Reinstating an assembly’s frequency and phase reactivates it during retrieval