Flexible recruitment of memory-based choice representations by human medial-frontal cortex

Abstract

Decisions in complex environments rely on flexibly utilizing prior experience. This depends on the medial frontal cortex (MFC) and the medial temporal lobe, but it remains unknown how these structures implement selective memory retrieval. We recorded single neurons in MFC, amygdala, and hippocampus while human subjects switched between making recognition memory- and categorization-based decisions. MFC rapidly implemented changing task demands by utilizing different subspaces of neural activity and by representing the currently relevant task goal. Choices requiring memory retrieval selectively engaged phase-locking of MFC neurons to amygdala/hippocampus field potentials, thereby enabling the routing of memories. These findings reveal a mechanism for flexibly and selectively engaging memory retrieval and show that memory-based choices are preferentially represented in frontal cortex when required.

One sentence summary

Information stored in recognition memory is selectively used by human frontal cortex to make memory-based choices when required by a decision.

Behavior in complex environments requires decisions that flexibly combine stimulus representations with context, goals, and memory. Two key aspects of cognitive flexibility are to selectively utilize relevant information depending on task demands, and to retrieve information from memory when needed (1). We begin to understand the neural mechanisms that underlie flexible decisions in the case of perceptual decision-making (2–4), with evidence for both early gating, mediated by top-down attention (5), and late selection of relevant features in prefrontal cortex (3). In contrast, little is known about the mechanisms of decisions that also depend on associated category knowledge and memory. In particular, it remains unknown how memory retrieval is selectively engaged when decision-relevant information needs to be actively searched for in memory (6–8).

The MFC is critical for complex behavior, and registers cognitive conflict, errors, and choice outcomes (9–11). It supports flexible decision-making in two ways: i) by representing task sets (12–14) and context (15), and ii) by selectively engaging memory retrieval through functional interactions with other brain areas (16–18), in particular the hippocampus (19–21) and amygdala (22, 23). A mechanism that facilitates such inter-areal interactions is phase-locking of MFC activity to oscillations in the hippocampus or amygdala. This mechanism has been extensively investigated in rodents during spatial behavior (24–26) and fear conditioning (27, 28), but its broader function remains poorly understood (29), particularly in humans. Similarly, human neuroimaging studies indicate that the MFC are involved in memory search (8, 18, 30–34) and that patterns and level of activity and connectivity assessed by fMRI vary as a function of retrieval intentionality (35–38). It remains unknown what features of decisions and context are represented in human MFC, whether memory retrieval selectively engages synchrony between MFC and hippocampus/amygdala, and whether synchrony can be engaged dynamically when required. This lack of knowledge stands in stark contrast to the patent behavioral ability of humans to flexibly recruit memory processes in everyday life (39, 40), and to our detailed knowledge of memory representations in the human hippocampus and amygdala, where cells represent aspects of declarative memories such as the familiarity and the identity of a stimulus (41–43). To address these open questions, we utilized simultaneous recordings of single neurons and local field potentials in the human MFC, hippocampus, and amygdala.

Results

Task and behavior

We recorded from 1430 single neurons across four brain areas (Fig. 1C–D; see Table S1; 33 sessions in 13 subjects): n=203, 460, 329, and 438 neurons from anterior hippocampus (HF), amygdala (AMY), dorsal anterior cingulate cortex (dACC) and pre-supplementary motor area (pre-SMA), respectively. For brevity, we refer to HF and AMY together as HA (n=663 cells), and to dACC and pre-SMA together as MFC (n=767 cells).

Fig. 1. Task, electrode locations, and behavior.

(A) Task structure. A session consisted of eight blocks of 40 trials. The task switched with each block (blue=categorization, red=memory), and the response modality switched halfway within each block (saccade or button press; randomly assigned at the beginning of the block). The subject was instructed about the task at the beginning of each block (magenta arrow) and how to respond at the beginning and halfway point of each block (green arrow). (B) Example of screens shown to subject for two example trials. (C-D) Electrode locations. Each dot is the location of a microwire bundle in one subject. (E-F) Eye tracking data from one session from the button press (e) and eye movement (f) trials. (G) Reaction times as a function of task across all sessions (memory, μ ± sem, 1.27±0.02s; categorization, 0.90 ± 0.02s; p =7.6e-228, 2-sample KS test). (H) Memory performance improves over the course of the experiment (β = 0.56, p=8.42e-130, logistic mixed effects model). See also Fig. S1 for an extended summary of the behavior.

Subjects viewed a sequence of 320 images, grouped into 8 blocks of 40 images each, in each session (Fig. 1A–B). Subjects were instructed at the beginning of each block which decision to make and which response modality to use to communicate their decision. Subjects made a “yes” or “no” decision for each trial to either indicate whether an image belonged to a given visual category (“Categorization Task”) or whether an image had been seen before in the task or not (“Memory task”). No feedback was provided (see Methods for details on the task). Each image shown belonged to one of four visual categories: human faces, monkey faces, fruits, or cars. In each block, half of the images shown were repeated and half were novel (except in the first block, in which all images were novel).

Subjects indicated choices either using saccades (leftward or rightward) or button press with central fixation (Fig. 1E–F; mean ± s.d., 94 ± 15% of all gaze positions fell within the image shown). Reaction times (RT) were significantly longer in the memory compared to the categorization task (Fig. 1G, mean RT of 1.48s ± 1.1 vs. 1.19s ± 1.2 respectively, p<1e-20, 2-sample KS test, mean ± s.d. across all trials in a given task). Subjects performed with an average accuracy of 97 ± 6% vs. 71 ± 6% in the categorization and memory task, respectively (mean ± s.d. across n = 33 sessions). This difference in accuracy remained after we matched for RT between the two tasks (96 ± 6% vs. 72 ± 8% with matched RT of 1.23s±0.60 vs. 1.24s±0.60 for the categorization and memory task, respectively. Even without RT matching, the initial response in terms of arousal was not different between tasks as assessed by pupillometry, see Fig. S1J–L). In the memory task, accuracy increased as a function of how many times an image had been shown (Fig. 1H, β_appearances = 0.56, p <1e-20, mixed effects logistic regression; also see Fig. S1C–D for effect of target vs. non-target on memory performance). Subjects had shorter RTs on “yes” (seen before) decisions than on “no” (novel stimulus) decisions in the memory task (Fig. S1A, see legend for statistics), as expected from an MTL-dependent recognition memory task (41). In the categorization task, RT was not significantly different between the two responses (Fig. S1A), showing the absence of oddball effects.

Effects of task type and response modality in the medial frontal cortex

Instructions about the type of task and response modality were shown at the beginning of each block (Fig. 1A, B). Cells showed significant modulation of their firing rate during the baseline period as a function of task type (Fig. 2A–B shows an example in pre-SMA). At the single-neuron level, significantly more cells were modulated by task-type in the MFC compared to HA: 25% of MFC cells (165/767, 82 in dACC, 83 in pre-SMA; see Fig. S2B) vs. 12% of HA cells (79/663, 21 in HF, 58 in AMY), χ²-test of proportions, p = 1.5e-6. Similarly, at the population level, population decoding accuracy was significantly higher in MFC compared to HA (Fig. 2C; 90% vs. 70%, respectively, p<1e-3, Δ_true = 20% vs. empirical null distribution, see Methods), a conclusion that held regardless of number of neurons used (Fig. S2H). Cells also modulated their activity as a function of response modality during the baseline period (Fig. S2E shows an example). Similar to task type encoding, significantly more cells encoded response modality in MFC and this signal could be decoded with higher accuracy in MFC compared to HA (14% vs 10% of cells; 84/593 vs. 59/586 in MFC and HA respectively; 33 in ACC, 51 in pre-SMA, 27 in HF, 32 in AMY; χ²-test of proportions, p = 0.03; population decoding performance 72% vs. 64%, Fig. 2D; p<0.05, Δ_true = 8% vs. empirical null distribution; this conclusion held regardless of number of neurons used Fig. S2I.

Fig. 2. Representations of task type and response modality.

(A-B) Example pre-SMA neuron. (B) Average firing rate during the baseline period (−1 to 0s relative to stimulus onset) for each block for the cell shown in (a). Shown is the average baseline firing rate across all blocks of the same type. (C-D) Population decoding of task type (c) and response modality (d). (E) Cross-condition decoding approach. The background color denotes the type of trials that were used to train a given decoder. (F) Cross-response modality decoding of task type from the baseline firing rate of all recorded cells. (G) Cross-task decoding of response modality. (H) Decoding performance as a function of trial number relative to a task type switch (green arrows in Fig. 1a; transitions from categorization to memory and vice-versa were pooled). Error bars indicate standard deviation in all panels, with the exception of panel B where they indicate the standard error of the mean. (I) Baseline decoding of task-type for subsequent trials with short reaction times was more accurate than decoding on long reaction time trials. Shown separately for categorization and memory trials (p = 2e-11 and 7e-13 respectively, Wilcoxon rank sum test). Error bars denote standard error in decoding accuracy across trials (80 trials in each of the 4 groups). See Fig. S2 for additional analyses that break down context effects by specific anatomical regions.

After a task switch, contextual signals emerged rapidly within 1–2 trials in the new context in MFC (Fig. 2H). This was not a result of ongoing post-stimulus processing because task could still be decoded even if only considering the subset of task cells in the MFC which did not differentiate between the tasks around response time (see Fig. S2G). Task switching costs were also reflected in the subjects’ longer reaction times shortly after a change in task or effector type (Fig. S2A). Task type representations during the baseline period were stronger on trials where the subject subsequently produced a fast response versus those where the response time was slow (Fig. 2I), indicating behavioral relevance. Lastly, we tested whether the two types of contextual signals were sufficiently robust to avoid interference with one another, using a cross-condition generalization decoding analysis (44) (Fig. 2E). We first trained a decoder to discriminate task type on trials where the subject was instructed to reply with a button press, and then tested the performance of this decoder on trials where the subject was instructed to use saccades (and vice-versa). The two neural representations generalized in the MFC but not in the HA (Fig. 2F–G). For this reason, we focused on the MFC when conducting above analysis.

Cross-condition generalization of memory and image category

We next asked whether the neural representations of image category and familiarity are sensitive to task demands. We assessed two consequences of task demands: generalization across tasks and strength of representations within each task. At the single-unit level, we examined visually-selective (VS) cells (42), whose responses are thought to reflect input from high level visual cortex, and memory-selective (MS) cells (41), whose response signals stimulus familiarity (Fig. 3A–B shows examples). Of the HA cells, 40% were visually selective (264/663, 35 in HF and 229 in AMY) and 11% were memory selective (73/663, 10 in HF, 63 in AMY; 24/73 and 49/73 were old>new and new>old selective, respectively; see Methods for selection model). In the MFC, 13% (103/767) of the cells were visually selective and 11% (84/767) were memory selective. We first performed single-neuron analysis of the selected HA cells. Both visual and memory selectivity was present in both the memory and categorization blocks (Fig. S3D, E and S3H–I). MS cell responses reflected a memory process: they strengthened over blocks as memories became stronger (Fig. S3G) and differed between forgotten old (FN) and correctly identified new (TN) stimuli for both new>old (n=49) and old>new (n=24) preferring MS cells (Fig. S10F). Furthermore, these memory signals were behaviorally relevant: new/old decoding was significantly weaker in incorrect than correct trials (Fig. S10G).

Fig. 3. Representations of image category and familiarity (new/old).

(A-B) Example cells that (A) represent image category and (B) that differentiate between new and old stimuli. (C) Decoding accuracy of image category from all recorded cells was significantly higher in the HA relative to the MFC (Δ_true = 49%, p<0.001). (D) Decoding of new vs. old (ground truth) was similarly accurate in HA and MFC (Δ_true = 7%, p = 0.13). For new vs. old decoding, trials with images of monkeys were excluded since the recognition performance for these images was at chance (Fig. S1B). (E) Population activity of all recorded HA (left) and MFC (right) cells, plotted in 3D using MDS. Individual points show the mean activity of the population for that specific condition. The highlighted plane contains all locations of state space occupied by a given task for the case of fruits vs. faces as the binary category distinction (for illustration only; all analysis uses all categories). The geometry of the representation allows for a decoder that is trained on one task to generalize to the other task (see Fig. S4C for example decoder hyperplanes). (F) Approach used for the cross-condition generalization analysis. Color indicates task (blue=categorization, red=memory). (Top) We trained a decoder to discriminate between new vs. old trials on categorization trials and then tested its performance on new vs. old stimuli encountered during the memory condition (and vice-versa). (Bottom) Similarly, a decoder that is trained to discriminate between image categories (in this example face vs. fruits, all results include all 6 possible pairs) on categorization trials, was tested on memory trials. (G) Cross-condition generalization performance for image category. (H) Cross-condition generalization performance for new vs. old. (I) Difference in cross-task generalization decoding accuracy for familiarity and image category between HA and MFC. Difference is computed between the average cross-task performances in each area (i.e. average of memory → categorization and categorization → memory). The null distribution for the average was estimated from the empirical null estimated by shuffling the labels used to train the decoders. For both variables, decoding from HA had significantly greater cross-task generalization performance than decoding from MFC (difference in both cases is positive and outside of the 95^th percentile of the null distribution). (J) Generalization index (see Methods) for memory (left two data points) and image category (right two data points). For both image category and familiarity, generalization across task was higher in the HA population than the MFC population (see figure for statistics).

We next took a population-level approach (over all single units, without selection) to answer the question whether coding for visual and memory information generalizes across tasks. We used decoding performance on individual trials and single-neuron analysis (Fig. S4D–E) to assess whether the neural encoding of visual category and/or familiarity of a stimulus depended on task demands. In both HA and MFC, image category could be decoded (Fig. 3C, 98% and 49% in HA and MFC respectively; chance level = 25%). Category decoding performance was significantly higher in the HA than MFC (Δ_true = 49% vs. empirical null distribution, p<1e-3). In the HA, the ability to decode category was not significantly different between the two tasks (Fig. 3C, 96% vs. 99% in categorization and memory respectively; Δ_true = 3%, p =0.25) and could be decoded above chance in both the HF and AMY (Fig. S4A–B). Note that at the single-neuron level, HA neurons encoded significantly more information about category in the memory task (Fig. S4D), an effect that decoders were not sensitive to due to saturation. In the MFC, decoding accuracy for image category was significantly higher in the memory task (Fig. 3C, 60% vs. 36%, Δ_true = 24% vs. empirical null distribution, p<0.001). Memory was decodable in both the HA and MFC (Fig. 3D, 69% vs. 76% respectively), with no significant difference in decoding accuracy between the two tasks in HA (Fig. 3D, 67% vs. 71%, in categorization and memory trials respectively, Δ_true = 4%, p =0.3) and significantly better decoding ability in MFC in the memory task (Fig. 3D, 86% vs. 61%; Δ_true = 25% vs. empirical null, p = 0.001). Single-neuron analysis confirmed the impression from decoding that the strength of memory signals in the HA were not modulated by task demands whereas they were in MFC (Fig. S4E). In either task, at the population level, memory decoding was only possible in the amygdala (Fig. S4B). The population-level decoding of memory in the HA was principally a reflection of the signal carried by the MS cells (Fig. S10E), and was not due to repetition suppression of VS cells (Fig. S10B–C) because it was not possible to decode familiarity from VS cells alone (Fig. S10D).

To gain insight into the geometry of the population-level representations, we assessed whether the decoders trained to report familiarity and the category of the stimuli in one task would generalize to the other task (Fig. 3F schematizes our approach). Cross-task generalizability would indicate that familiarity and visual category are represented in an abstract format (44). First, cross-task generalization performance was greater in the HA than MFC for both image category (Fig. 3G,I; 98% vs. 41%, averaged across the two cross-condition decoding performances; Δ_true = 57% vs. empirical null, p<1e-3) and familiarity (Fig. 3H,I; 67% vs. 55%, Δ_true = 12% vs. empirical null, p<0.05). Second, to help understand the geometry of these neural representations, we projected the average HA and MFC population activity for all possible pairings of familiarity, image category, and task (8 different conditions) into a 3D state-space using multi-dimensional scaling. For illustration purposes, we show this for the two image categories (fruits, faces) for which memory performance was the best. In the HA (Fig. 3E, left), the relative positions of a “new face” with respect to an “old face” was preserved across tasks (shown as different colored planes). The relatively parallel location of the subspace of neural activity occupied by the two tasks permits across-task generalization for both image category and familiarity. In contrast, in the MFC (Fig. 3E, right), the relative positions of the four conditions were not preserved. This is consistent with weaker cross-task generalization performance we observed in MFC relative to HA (Fig. 3G–H), resulting in reduced generalization indices in MFC compared to HA (Fig. 3J; note this metric takes into account different levels within-task decoding accuracy, which is an upper bound for the achievable generalization performance; see supplementary text for details).

Representation of choice

We next investigated how the subject’s choice (yes or no) is represented. Decoding accuracy for choices was highest in the MFC (Fig. 4A shows examples), with average population decoding performance of 89%, compared to 68% in the HA (Fig. 4B; Δ_true = 19% vs. empirical null, p<1e-3; 61% in AMY and 57% in HF when trained separately; Fig. S7G shows this result as a function of number of neurons used). Further single-neuron (Fig. S11A–B) and population (Fig. 4E, S11C) analysis confirmed the impression that the choice signal was significantly stronger in the MFC regardless of selection threshold and quantification method. We therefore first analyzed choice information in the MFC (see further below for results in HA). Choice decoding in MFC was strongest shortly after stimulus onset well before the response was made (Fig. S7A). To disassociate representation of choice (yes vs. no) from the representation of ground truth (old vs. new) during the memory recognition task, we fit a choice decoder to a set of trials that were half-correct and half-incorrect. The activity of MFC cells predicted choice, but not the ground truth at levels significantly different from chance (Fig. 4C; choice decoding is above the 95^th percentile of the null distribution while new/old decoding is not; see Fig. S7D for this analysis shown over time). Choice could be decoded from MFC separately from both correct and incorrect trials (Fig. S5I). As a control for potential confounds due to RT differences between tasks (see Fig. S1A), we acquired data from a separate control task in which we eliminated RT differences behaviorally by adding a waiting period (6 sessions in 5 subjects; n=180 and 162 neurons in HA and MFC, respectively; see Methods and Fig. S6). Like in the original task, MFC cells represented the subject’s choice (Fig. S6C–G), thereby confirming that this separation is not due to RT differences.

Fig. 4. Task-specific representation of choice.

(A) Example MFC choice cells, split by choice (yes or no) and task. (B) Population choice decoding accuracy was significantly greater in MFC compared to HA (across all trials, Δ_true = 19% vs. empirical null, p<1e-3). (C) MFC cells represent choice and not the ground truth (i.e. new/old; memory trials only). (D) Population summary (neural state space) of choice-related activity in MFC, plotted in 3D space derived using MDS. (Top) Variability due to response modality. The highlighted planes connect the points of state space occupied by activity when utilizing button presses (purple) or saccades (green). (Bottom) Variability due to task type. The highlighted plans connect the points of state space occupied by activity in the same task. (E) Choice-decoders trained in one task do not generalize to the other task (binsize 500ms, 16ms step size). (F) Same as (E) but for a fixed 1s time window starting at 0.2s after stimulus onset. (G) Choice decoding generalizes across effectors (also see (D)). (H) Generalization index of choice decoding (see Methods for computation) to summarize (F-G). The representation of choices generalized across response modality but not task. (I) Generalization between different sub-tasks of the categorization task but not between task types. The colored bars indicate the 5^th-95^th percentile of the null distribution. (J) (Left) State-space trajectories for the four conditions arising from the combination of response (yes, no) and task (categorization, memory). (Right) Trajectory similarity, computed in a 8D latent space (recovered using GPFA, see Methods) across the eight conditions arising from the combinations of choice, effector type, and task. (K) Decoder weight of each cell for decoding choice during the categorization-and memory task. The cells in the top 25-th percentile are shown in black. The inset shows the angle created by the vector [${\omega }_i^{cat},{\omega }_i^{mem}$] with respect to the x-axis of the cells marked in black.

We used multi-dimensional scaling to visualize the population activity for the eight combinations of choices, task types, and response modality (Fig. 4D, see Methods). The resulting geometrical configuration indicates that choice decoding generalizes across response modality (Fig. 4D, top) but not across task types (Fig. 4D, bottom). We therefore computed the cross-task generalization performance of a decoder trained on choices during one task and tested on the other. We performed this analysis across time (Fig. 4E; see Fig. S7B for this analysis shown separately for preSMA and dACC), as well as in a single post-stimulus time bin (Fig. 4F). To avoid confounds due to response time differences, we performed the fixed window analysis (Fig. 4F) only for the control task, where the timing between tasks was identical (Fig. S6B). While the choice signal did not generalize across task types (Fig. 4F), it did generalize across response modality within the same task type (Fig. 4G). Quantifying this observation with the generalization index confirmed this impression (Fig. 4H). Lastly, we examined choice signals in the HA. While choice signals were comparatively weak in HA (Fig. S7E and 4B; see Fig. 4B for statistics), they nevertheless exhibited a pattern of generalization similar to that in the MFC (Fig. S7E–F).

To test the possibility that any task might yield a unique choice axis that does not generalize to any other task, we considered the four subtasks that make up the categorization trials in a given session (the target category can be any one of the four possible image categories). We tested whether the choice signal generalizes across these subtasks by training and testing across blocks requiring different categorizations. Choice decoding generalized across all sub-tasks in the categorization task, but not the memory task (Fig. 4I). Next we compared the dynamics of the population activity between the eight conditions arising from the combination of choice, effector type, and task in a state-space model recovered using Gaussian Process Factor Analysis (GPFA (45), see Methods). Comparing the pairwise similarity between the trajectories in state space (Fig. 4J, left) in the first 500ms after the stimulus onset revealed that the patterns of dynamics in state space cluster first by task type (Fig. 4J, right; also see Supplementary Video 1).

We next examined whether the population-level analysis relied on different sets of neurons to decode choice in each of the two tasks. We determined how individual cells are recruited by a linear decoder (44, 46). For each cell, we quantified its importance (46) for both the memory and categorization choice decoder (Fig. 4K). We then plotted the degree of specialization for each cell based on its importance in each task (see Methods). Cells that report choice independently of task should lie on the diagonal (i.e. an angle of π/4). Instead, the distribution of angles was significantly bimodal across all cells (Fig. 4K, inset plot, p<1e-5, Hartigan dip test), with modes centered away from the diagonal. Despite this bimodality, we could still use the cells that are the most useful in one task, to train a new decoder that can predict choice well above chance (although significantly weaker) in the other task (Fig. S7C). Note that this is not an example of cross-task generalization, since we are fitting a new decoder.

Task-dependent spike-field coherence between MFC cells and HA LFP

It is thought that the selective routing of decision-related information between the HA and MFC is coordinated by inter-areal spike-field coherence (24). We therefore asked whether MFC neurons phase-lock to ongoing oscillations in the LFP in the HA and, if so, whether the strength of such interactions is modulated by task demands. We performed this analysis for the 13 subjects and 33 sessions for which we simultaneously recorded from both areas (Fig. 5A). In the following, we only utilized neural activity from the 1s baseline period that precedes stimulus-onset to avoid confounds related to stimulus-onset evoked activity. Individual cells in the MFC showed strong task modulation of MFC to HA spike-field coherence (Fig. 5B shows a single-cell example in the dACC). At the population level, MFC cells showed significantly stronger theta-band coherence with HA oscillations during the memory compared to the categorization task (Fig. 5C, 8822 cell electrode pairs; p = 1.3e-7, paired t-test, measured at 5.5Hz; see Fig. S9B–C for additional controls). This was the case for both MFC-hippocampus, and MFC-amygdala interactions (Fig. 5D, n = 3939, p = 8.8e-4; n = 4884, p = 4.3e-5 respectively, paired t-test). This effect was due to changes in phase preference as there was no significant difference in HA LFP power between the tasks (Fig. 5E, p = 0.08, paired t-test of signal power at 5.5Hz, estimate across all 8822 cell-electrode pairs). Out of the 767 MFC cells, a significant subset of ~100 cells were phase-locked to the theta-band HA LFP (Fig. S9A), with the largest proportion preferring 3–8Hz.

Fig. 5. Modulation of interareal spike-field coherence by task demands.

(A) Analysis approach. Inset shows that only data from the baseline was used (except in panel H). (B) Spike-field coherence for a cell in dACC relative to all channels in the ipsilateral hippocampus. (C) Phase-locking of MFC cells to HA LFP. (top) Average inter-area pairwise phase consistency (PPC) of all cell-electrode pairs for each task. (bottom) Significance of difference between tasks; Peak difference was at f = 5.5Hz. Dashed line shows the threshold (P=0.05/56, Bonferroni corrected). (D) Difference in average inter-area PPC at f=5.5Hz between tasks conditions for all possible cell-electrode pairs (from left to right, n = 8822, p = 1.3e-7; n = 3938, p = 8.8e-4; n = 4884, p = 4.3e-5; paired t-test). (E) Average spike-triggered power was not significantly different between the two tasks (paired t-test, n = 8822 cell electrode pairs, p = 0.08). (F) Single-neuron analysis of choice cells. Importance index assigned by the decoder to each cell for decoding choices in either task. Selected choice cells are indicated in color. (G) MFC-HA spike-field coherence for choice cells. (top) Average PPC for all choice cells in MFC (209 cells, 2384 cell-electrode pairs) for the two tasks. (bottom) Significance of difference between tasks, shown separately for memory and categorization choice cells (n=1176 and n=906 cell-electrode pairs, respectively). Only memory choice cells show a significant difference in the gamma band (p = 2e-6, t-test). (H) Difference in spike-field coherence between true-positive (correct retrieval) vs. false negative (incorrect retrieval) trials measured in the [0.2 – 1.2s] window after the stimulus onset, shown separately for memory choice cells (left panel) and categorization choice cells (right panel) in the theta (4–10Hz) and gamma-frequency (30–80Hz) band. PPC was significantly stronger in correctly retrieved trials in the theta band for memory choice cells (Δ_mem = 0.003, p = 0.002; Δ_cat= - 0.001, p = 0.3; paired t-test) and in the gamma band for both types of choice cells (Δ_mem = 0.002, p = 0.005; Δ_cat=0.004, p = 7.2e-8; paired t-test). (I) Spike times of HA cells are coherent with local theta-band (3–8 Hz) LFP. (J) Average local PPC in the HA did not differ significantly as a function of the task (f = 5.5Hz; p = 0.51, paired t-test, n = 2321 cell-electrode pairs). Error bars in panels (D, H) denote the 95% confidence interval (bootstrap, n = 10000 iterations). All other error bars are s.e.m. otherwise.

To determine if there is a relationship between the tuning of cells in MFC and their inter-area coherence with HA, we selected for choice cells independently in the categorization and memory task (see Methods for selection model; note selection controls for RT differences). This revealed that 101/767 and 82/767 cells were significantly modulated by choice during the memory and categorization task, respectively (p<0.001 vs. chance for both; see Fig. 4A Cell 2 and 1 for an example, respectively). Single-neuron decoding showed that it was not possible to decode the subject’s choice in the categorization task from choice cells selected in the memory task and vice-versa (Fig. S5A–D). Single-neuron analysis revealed that cells preferring either “no” or “yes” choices were presently in approximately equal proportions in both tasks (Fig. S5B). The removal of the selected choice cells from a population decoding analysis with access to all recorded neurons significantly diminished decoding performance (Fig. S5F, G). Importantly, each of the selected cells had a high importance index, as determined from population decoding (Fig. 5F). Considering only the MFC choice cells revealed that this subset of cells was similarly increasing their phase-locking during the memory task (Fig. 5G, top), with the strongest effect again in the theta range (f_peak = 5 Hz, p = 1e-6, paired t-test). Both categorization and memory choice cells showed this pattern of modulation (Fig. 5G, bottom). The memory choice cells exhibited an increase in gamma band coherence (Fig. 5G, f_peak = 38.5Hz; p = 2e-6, paired t-test). The extent of phase-locking of choice cells following stimulus onset (0.2–1.2s) during the memory task was significantly stronger for correctly retrieved compared to forgotten old trials, indicating behavioral relevance for memory retrieval (Fig. 5H). Lastly, to exclude the possibility that this inter-area effect was due to task-dependent changes within the HA, we examined the phase-locking properties of HA cells to their own locally recorded LFP (LFP and spiking activity is recorded on separate electrodes, see Methods). The spiking activity of 331/663 HA cells was significantly related to the theta-frequency band LFP (Fig. 5I, shown for f = 5.5 Hz). The strength of this local spike-field coherence was, however, not significantly different between the two tasks (Fig. 5J; p = 0.61, paired t-test, n = 2321 cell-electrode pairs).

Discussion

We investigated the nature of flexible decision-making in the human brain by probing how the strength and/or geometry (44) of neural representations of stimulus memory, stimulus category, and choice is modified when subjects switch between a memory and a categorization task. We found evidence for both kinds of neural representation changes due to changing task demands for a subset of the studied variables. In the MFC, both the strength and geometry of representation of stimulus memory changed as a function of task demands. In contrast, in the HA, both the strength and geometry of the representation of stimulus memory were insensitive to task demands (Fig. 3D, H). Our finding of memory signals in the amygdala supports the hypothesis (47, 48) that the amygdala contributes to recognition memory by signaling stimulus familiarity. Representation strength of stimulus category in both the HA and MFC were stronger in the memory task, but the geometry of this representation was in addition also modulated by the task in the MFC (Fig. 3G, right). Overall, these results show that the geometry of the representations (as assessed by across-task generalization) of stimulus familiarity and memory were significantly less sensitive to task demands in the HA compared to the MFC (Fig. 3G–H).

At the population level in the MFC, choices in both the memory and categorization task were decodable with high reliability, but these decoders did not generalize across the two tasks. Choice decoding generalized across sub-tasks of the categorization task and changes of response modality within both tasks, indicating that changes in representations were due to switching between a task requiring memory retrieval and one that does not. Note that while the choice signal was significantly weaker in the HA, this same pattern of generalization also held for the HA, suggesting the possibility that the task-demand dependent choice representation we found in MFC is widely distributed in the brain. A group of task demand-dependent cells in the MFC were choice cells, which signal behavioral decisions preferentially for either memory- or categorization decisions irrespective of response modality and regardless of the ground truth of the decision. Thus, from the point of view of downstream areas, neurons formed two separate decision axes: one for memory-based decisions and one for categorization-based decisions. These two decision axes were instantiated selectively so that they were only present when the current task required them.

These findings contrast with prior work on task switching between different tasks that required purely perceptual decisions, which found a single decision axis in monkey pre-frontal cortex, with task-irrelevant attributes also represented (49). We found that memory-based choices add a second decision axis, which is present only when decisions engage memory retrieval processes. While task-sensitive representations of choice have been shown in recordings from nonhuman primates during perceptual decision-making (2, 49, 50) and human neuroimaging (51), our data reveal choice representations that specifically signal recognition memory-based choices at the single-cell level. It has long been appreciated that the frontal lobes are critical for initiating and controlling memory retrieval (30, 52–54). Neuroimaging reveals that patterns of activity within some frontal and parietal areas are modulated by memory retrieval demands (34, 35, 37, 38, 55), whereas memory-related activity patterns in the MTL are comparatively insensitive to retrieval demands (34). These finding have led to the proposal that the memory retrieval network consists of specialized processes separate from those utilized for other kinds of decisions (6, 56, 57). The memory-choice axis we describe is a potential cellular substrate for this critical aspect of human cognitive flexibility. Future work is needed to investigate whether similar principles also apply to hippocampus dependent associate or source memory (58–60) based decisions, which we did not assess here (we probed recognition memory).

A second group of cells we characterized in MFC signal the currently relevant goal (task type and response modality) throughout the task. These cells switched their activity pattern when instructions indicated a change in task demands. While these switches were rapid, they were not instantaneous, likely reflecting the cost of switching between memory retrieval and categorization modes (61–63). We hypothesize that these cells facilitate holding in working memory (64, 65) the active task set and configure brain networks in preparation for appropriate execution of the instructed task (12, 34, 66, 67). Task switching costs are a much investigated aspect of cognitive flexibility (39, 61–63), but it remains little understood how they arise, and why some task switches are more difficult than others. The MFC cells we describe here offer an avenue to directly investigate these questions.

Finally, we uncovered a possible mechanism by which memory-based information can be routed dynamically between MFC and hippocampus/amygdala when a task requires memory retrieval. Changing long-range synchronization of neural activity is thought to be a way by which functional connectivity between brain areas can be changed flexibly (68–71). Here, we reveal a specific instance of this phenomenon at the cellular level in humans in the form of changes in the strength of cortico-hippocampal and cortico-amygdala functional connectivity. Hippocampal-mPFC functional connectivity in rodents supports spatial working memory (24) and is prominent during both navigation and rest (72–74). Similarly, amygdala-mPFC function connectivity supports flexibly switching between aversive and neutral behaviors depending on learned cues (75). But it remains unknown whether these pathways serve a role in long-term memory retrieval in humans and, if so, whether it can be engaged selectively. Similarly, in humans, MTL-PFC connectivity changes as measured by fMRI have been related to control demands over memory retrieval (36, 76), but it remains unclear how these indirect metrics relate to long-range synchronization as measured in rodents. Here, we show that MFC-hippocampus/amygdala connectivity is selectively enhanced during the memory task, indicating that patterns of inter-areal connectivity change in preparation of initiating memory retrieval (77, 78). The extent of synchrony after stimulus onset is stronger when a memory is successfully retrieved compared to when it is forgotten. Memory choice cells in MFC exhibited enhanced gamma-frequency band coordination of their spiking activity with the hippocampus and amygdala LFP, and this modulation was behaviorally relevant after stimulus onset. This reveals a specific cellular-level instance of a role for gamma oscillation-mediated coordination of activity between distant brain regions (24, 79) in human memory retrieval.