Online evaluation of novel choices by simultaneous representation of multiple memories

Abstract

Prior experience plays a critical role in decision making. It enables explicit representation of potential outcomes and provides training to valuation mechanisms. However, we can also make choices in the absence of prior experience, by merely imagining the consequences of a new experience. Here, using fMRI repetition suppression in humans, we show how neuronal representations of novel rewards can be constructed and evaluated. A likely novel experience is constructed by invoking multiple independent memories within hippocampus and medial prefrontal cortex. This construction persists for only a short time period, during which new associations are observed between the memories for component items. Together these findings suggest that in the absence of direct experience, co-activation of multiple relevant memories can provide a training signal to the valuation system which allows the consequences of new experiences to be imagined and acted upon.

Humans display remarkable flexibility in their behavior. Like many other animals, we guide our behavior through direct experience, but we can also infer the likely consequences of actions never previously taken^1,2. Through generalizing principles and applying them to new situations^3,4, we can predict new relationships and statistical structures in our environment and use these to estimate the value of new events^1,5,6. Whilst some progress has been made in uncovering the brain regions that underlie these complex abilities^1,3–7, little or no progress has been made in understanding how neuronal networks support these complex computations, partly because it is unclear to what extent such computations exist in species where we can readily measure single cell activity.

One potential mechanism that allows for upcoming events to be evaluated involves using past experience to predict consequences of future possible scenarios. In rodents, hippocampal firing sequences at choice points predict or ‘preplay’ the forthcoming environment⁸, and the likely outcomes of their decision can later be decoded in the orbitofrontal cortex⁹. By contrast, when choosing between novel options, there is no direct experience from which to preplay and evaluate future options. However, it is possible that the representation of an upcoming novel outcome may be constructed by combining multiple distinct relevant experiences, preplayed simultaneously.

To test these predictions we required access to the information content of neural populations underlying the representation of a novel experience. Despite the poor spatial resolution of fMRI, there are well-validated strategies that can reveal underlying cellular representations. For example, fMRI adaptation takes advantage of the fact that activated cellular ensembles within a voxel show a relative suppression in their activity in response to repetition of a stimulus to which they recently responded. Despite ambiguity in the biophysical mechanism underlying repetition suppression¹⁰, when combined with careful experimental design the technique allows for inferences to be made about the underlying neuronal representations^12,13.

Here we used fMRI adaptation to probe the neural representation of a novel food reward. We hypothesized that if the representation of a novel food was constructed by explicit combination of multiple distinct experiences, we would observe fMRI adaptation when subjects evaluated a novel reward immediately after evaluating a component ingredient. Furthermore, if multiple experiences were replayed simultaneously, plasticity may result between the underlying neuronal assemblies. Hence, experiences used to construct the same novel good would later adapt to each other. Lastly, we hypothesized that this complex construction process would not be required after an independent neuronal representation of the novel good had been established. We should therefore observe a reduction in each adaptation effect after allowing the subjects either to experience the novel good directly, or to simulate the novel good repeatedly. This repetition suppression paradigm therefore allowed us to probe the neural mechanisms that underlie human capacity for flexible, online, value construction.

Results

Deciding between novel goods

We created thirteen ‘novel goods’ whose values were unknown to the subjects. However, each good was a novel combination of two different familiar foods (Fig. 1a). Participants were given the opportunity to observe these novel goods without being allowed to sample them either by taste or smell.

Figure 1. Experimental Design.

(a) Thirteen novel ‘goods’ were made, each from the combination of two familiar food types that had not previously been tasted together. Two examples are shown here: avocado and raspberry smoothie (‘AB’), and tea-jelly ( ‘CD’). (b) Participants made binary decisions between the novel goods whilst in the scanner. (c) Prior to entering the scanner, two of the novel goods were chosen for each participant. Participants learnt to associate each of these novel goods and their respective components with two abstract stimuli. (d) In the scanner, participants vividly imagined the sensory properties of the food items in response to each abstract stimulus presented.

To first establish that these goods activate known value-related brain regions, we measured fMRI activity in 19 subjects whilst they evaluated and chose between pairs of these novel goods (Fig. 1b). After the scan session, subjects performed a Becker-DeGroot-Marschak (BDM) auction¹⁴ that allowed us to measure subjects’ constructed value for each good. Consistent with reports in simpler valuation contexts, we observed a signal that correlated with the value of the chosen option in a network of brain regions that included ventral and dorsal medial prefrontal cortex ((v/d)mPFC), and posterior cingulate cortex, (mPFC: p = 0.001 FWE corrected on cluster level, peak t(17) = 6.30, Fig. 2a). The involvement of both vmPFC and dmPFC is of particular interest given that the task requires subjects to construct, and evaluate, a model of a future outcome. This involvement accords with recent evidence that vmPFC encodes value preference for executable choices while dmPFC does so for choices that are modeled abstractly⁷.

Figure 2. Neural correlates of constructing and evaluating a novel good.

(a) Whilst participants made binary choices between novel goods, the mPFC (extending into dmPFC) encoded chosen value. (b) The mPFC and hippocampus showed repetition suppression to a novel good when preceded by a related component (e.g. tea-jelly preceded by tea), compared to when preceded by an unrelated component (e.g. tea-jelly preceded by avocado). (c) The mPFC showed repetition suppression to a component food item when preceded by the related component (e.g. tea preceded by jelly), compared to when preceded by an unrelated component (e.g. tea preceded by avocado). (d)(e) In mPFC and hippocampus respectively, a significant positive correlation was revealed between the amount of suppression between related components (across all blocks) and the average value participants assigned to the novel goods (after removing effects attributable to the value of the components, for mPFC: r = 0.51, p = 0.015, and hippocampus: r = 0.60, p = 0.004). (f) Both adaptation effects showed comparable effect size across the ventral-to-dorsal gradient of mPFC (mean ± s.e. across participants). Location of the ROIs is shown, and the effect size for both adaptation measures was scaled such that the peak value was equal to one. There was no significant difference between the two adaptation effects at any point on this gradient.

To evaluate these novel goods, subjects could not rely on pre-learnt values. Hence, their only recourse was to construct, online, an expectation of the compound’s value from knowledge of the individual components. A key question therefore is whether subjects constructed a novel representation of the compound by explicitly combining the representations of each component and, if so, which brain regions support this construction process? We reasoned that this construction process could be measured using fMRI adaptation. Activity relating to the construction of the compound value would be suppressed when preceded by a related component if, and only if, the subject had engaged the neuronal ensembles of the components when constructing a representation of the compound.

Constructing representations of novel goods using memories

For each participant, we selected two of the thirteen novel compounds, here referred to as ‘AB’ and ‘CD’, made up of four ‘familiar’ individual components (‘A’, ‘B’, ‘C’ and ‘D’) that subjects had tasted immediately prior to the experiment (Fig. 1a). To avoid visual confounds in a later analysis, we trained subjects to associate each of the 6 component and compound foods (‘A’, ‘B’, ‘C’, ‘D’, ‘AB’, ‘CD’) with two different abstract shapes (Fig. 1c). Participants trained extensively on these associations between food items and abstract shapes. In the final block of trials the mean accuracy was 97.8%, with mean reaction time of 845.2ms.

On each trial in the scanner, we presented a distinct shape that served as an instruction cue for subjects to elicit an explicit mental representation of the associated food (Fig. 1d). The key comparison of interest here was brain activity elicited by novel goods when preceded by related components (e.g. ‘A’ or ‘B’ followed by ‘AB’) compared to novel goods when preceded by unrelated components (e.g. ‘C’ or ‘D’ followed by ‘AB’).

Early in the experiment (block 1 out of 3) we observed fMRI adaptation between the representation of novel goods and their constituent components in both mPFC (p<0.001 FWE-corrected on cluster level, peak t(18) = 4.45, Fig. 2b) and bilateral hippocampus (t(18) = 2.55, p = 0.010 using ROI analysis (see Methods), Fig. 2b). These two brain regions are components of a network commonly activated in studies of value^7,15–18, episodic memory^4,19,20 and spatial navigation¹². The present result implies that these brain regions construct a value representation of a novel item from component memories, and do so by simultaneously engaging neuronal representations of these components.

Plasticity between simultaneously active memories

If this is in fact the case then it follows that during the construction of the compound good ‘AB’, the neuronal ensembles representing components ‘A’ and ‘B’ should be simultaneously active. We reasoned that this simultaneous activity, which first occurred during the stimulus-item training phase prior to scanning, would induce experience dependent plasticity between cellular elements in these two ensembles- a plasticity evident in the scanning trials as a shadow of this value construction process. For example, after constructing a representation of ‘tea jelly’, we reasoned that cellular representations of ‘tea’ would induce activity in jelly-preferring ensembles, and vice versa. This can also be tested using fMRI adaptation which predicts a differential effect for components that were part of the same compound compared to components that were not.

Indeed, when we compared early trials of ‘A’ that were preceded by ‘B’, to those that were preceded by ‘C’, we again found relative suppression in medial prefrontal cortex activity (p = 0.014 FWE corrected on cluster level, peak t(18) = 4.24, Fig. 2c), but not hippocampus (t(18) = 0.34, p = 0.367 using ROI analysis, see Methods).

Notably, across all three blocks, the extent to which individual participants showed adaptation between related components in mPFC, but also in the hippocampus, was predicted by the average value of the novel items (mPFC: r = 0.47 and p = 0.040; hippocampus: r = 0.58 and p = 0.010) but not component items (mPFC: r = −0.05, p = 0.833; hippocampus: r = −0.09, p = 0.730). This suggests that the mechanism underlying this suppression occurred during the earlier construction of the novel good, and not during the participant’s elicitation of the component item at the time that this signal was measured. Indeed, in both structures, the correlation with the value of the novel good survived the removal of any signal attributable to the component values (mPFC: r = 0.51, p = 0.015, Fig. 2d; hippocampus: r = 0.60, p = 0.004, Fig. 2e; see Supplementary Fig. 1 and corresponding discussion). Together these findings support value dependent plasticity in related components as a consequence of co-activation during construction of the novel goods.

It is important to note here that these three de facto tests of mPFC function (valuation, construction and plasticity) do not rely on any of the same data. Despite slight differences in thresholded peak locations of the two adaptation effects, they each show similar patterns of activity within mPFC (Fig. 2f). Medial prefrontal cortex can therefore evaluate novel goods by constructing explicit representations of expected outcomes from familiar components, a process that engenders plasticity between simultaneously active component representations.

The influence of sensory experience upon construction

We then asked whether consummatory exposure to the novel goods would reduce a need to construct value online. To test this idea, we repeated the experiment in a second group of 20 subjects with one important difference. This second group (familiar) were given a single sample of each of the 13 novel compound goods to taste before the experiment. Notably, both groups underwent the same item-stimulus learning task prior to entering the scanner, and there was no significant difference between groups in reaction time or accuracy on the final block of trials during the learning task (see Supplementary Table 1). Any difference between the two groups in the representation or evaluation of novel goods could therefore be attributed to the impact of sensory exposure.

We first assessed value effects during decision trials. Both groups showed similar consistency in their choices (see Supplementary Fig. 2). As was the case for the unfamiliar group, the familiar group encoded chosen value activity in a network of value-related brain regions including mPFC (Fig. 3a). In both groups, the neural activity observed in mPFC was consistent with a role for this brain region in the evaluation of compound goods (Fig. 3b–c).

Figure 3. Sensory exposure to a novel good: comparison between the ‘unfamiliar’ and ‘familiar’ groups during the decision making task.

(a) In the familiar group, the mPFC correlated with chosen value during the decision making task (thresholded at p<0.01, uncorrected for visualisation). (b) ROI used to assess value signals in both groups of participants during the decision task. (c) During the decision making task the unfamiliar and familiar groups showed comparable chosen value signals in mPFC (left side: average of all task blocks for each group), and in the unfamiliar group there was no change in the chosen value signal across time (right side: block 1 versus blocks 2 and 3). Parameter estimates were extracted from ROI shown in b (mean ± s.e. across participants).

To test whether this single experience was enough to reduce a need for online value construction, we compared adaptation effects across the two groups. To avoid selection bias, we used ROIs derived from whole-brain adaptation effects averaged across both adaptation contrasts in the two groups (Fig. 4a–b, see Methods). A between group comparison within these ROIs revealed significant differences in the adaptation effects between the familiar and unfamiliar participants in both mPFC (3-way ANOVA (see methods): group*condition interaction, p = 0.018, F(1,144) = 5.76), and hippocampus (3-way ANOVA (see methods): group*adaptation_type*condition interaction, p = 0.035, F(1,144) = 4.52). Using post-hoc two-sample t-tests to decompose these interactions, we found relative to the unfamiliar group, the familiar group showed reduced adaptation between the novel goods and their related components in mPFC (group difference: trend, t(18) = 1.70, p = 0.053, Fig. 4c), and in the hippocampus (group difference: t(18) = 3.11, p = 0.003, Fig. 4c). Furthermore, the familiar group did not show plasticity in mPFC between the representation of the constituent components of a novel good (group difference: t(18) = 1.96, p = 0.033, Fig. 4c). Crucially, there was no significant difference between groups in their ability to accurately elicit the correct representations during the imagination task (group comparison of accuracy: p = 0.82, and reaction time: p = 0.89), nor in the average subjective value assigned to any of the novel goods used in the adaptation task (see Supplementary Fig. 3). This result therefore suggests that even a single previous experience of a good is sufficient to reduce a requirement for online value construction. This is particularly notable given that extensive experience is required to reduce goal-oriented behavior and establish habitual actions²¹.

Figure 4. Sensory exposure to a novel good: comparison between the ‘unfamiliar’ and ‘familiar’ groups during construction of a novel good.

(a) ROI used to compare mPFC adaptation effects. (b) ROI used to compare hippocampus adaptation effects. (c) In mPFC the familiar group showed less adaptation between the novel-goods and their related components (left: p = 0.053, trend) and significantly less adaptation between related components (middle: p = 0.033; both extracted from ROI shown in a). In the hippocampus, the familiar group showed significantly less adaptation between the novel-goods and their related components (right: p = 0.008, extracted from ROIs shown in b). The stars (*) indicate p<0.05; bars and vertical lines correspond to mean ± s.e. across participants.

Temporal dynamics of the construction mechanism

If experiential and constructed valuation use distinct neural mechanisms, it is possible that the value construction mechanism could itself substitute for a direct experience and train experiential valuation mechanisms. As the experiment progressed, subjects had substantial experience of constructing the representation of the novel good. We asked whether, after multiple previous simulations of an experience it was still necessary to construct and evaluate the representation of novel goods anew on each trial? Alternatively, were values learnt despite participants never having experienced the novel good? As our experiment extended over three separate blocks, we were able to study changes in value construction-related adaptation effects over time.

Previous studies have found goal directed choice mechanisms exhibit marked differences early and late in choice experiments¹⁷. In the present study, we used a three-way ANOVA (see Methods) to identify attenuation of adaptation effects in mPFC and hippocampus in the unfamiliar group across the scanning session (block*condition interaction for mPFC, p = 0.004, F_(1,144) = 8.44, and block*adaptation-type interaction for hippocampus, p = 0.011, F(1,144) = 6.56). Post-hoc t-tests comparing block 1 with all remaining blocks revealed a significant reduction in adaptation over time of a novel-good to its related component (mPFC: t(18) = 2.12, p = 0.024, and hippocampus: t(18) = 2.13, p = 0.024; Fig. 5a), and in the plasticity between related components (mPFC: t(18) = 1.85, p = 0.041, but not hippocampus: t(18) = 0.81, p = 0.785; Fig. 6a).

Figure 5. In the absence of sensory exposure, there was evidence for the construction mechanism only in early trials: block 1 compared to block 2 and 3 for unfamiliar subjects.

(a) There was significantly less adaptation in blocks 2 and 3 between the novel-goods and their related components in mPFC and hippocampus respectively (left and middle: p = 0.024 and p = 0.024 respectively, ROIs shown). There was no significant reduction across time in mPFC adaptation of a component item to itself, when predicted by two different stimuli (right: p = 0.326, ROI shown). (b)(c) On the imagination task, the unfamiliar group showed an increase in accuracy (b) and a decrease in reaction time (c) across blocks. The stars (*) indicate p<0.05; bars and vertical lines correspond to mean ± s.e. across participants.

Figure 6. In the absence of sensory exposure, repetition suppression between related components was maintained across the duration of the experiment only if participants assigned high value to the compound goods.

(a) Participants from the unfamiliar group showed significant reduction in adaptation between related components over time in mPFC, but not hippocampus (p = 0.041 and p = 0.785 respectively, ROIs shown). (b)(d) The correlations shown in Fig.2 were also significant in mPFC, b, and hippocampus, d, when considering suppression effects between related components in blocks 2 and 3 alone: the amount of suppression across participants correlated positively with the average value of the compound goods (mPFC: r = 0.64, p = 0.002; hippocampus: r = 0.63, p = 0.003). (c)(e) A median split of participants into those that assigned high and low values to the compound goods revealed significant suppression between related components in blocks 2 and 3 only in those participants who assigned high value (c ,mPFC: ‘High’ p = 0.022, and ‘High vs Low’ p = 0.028; e, hippocampus: ‘High’ p = 0.008, and ‘High vs Low’ p<0.001). The stars (*) indicate p<0.05; bars and vertical lines correspond to mean ± s.e. across participants.

To ensure sensitivity to the construction process was maintained across the duration of the experiment, we also considered temporal dynamics of other adaptation effects, and of value signals encoded on decision trials. In the unfamiliar group, both adaptation in mPFC to repetition of any item (but not stimulus) and adaptation in visual areas to repetition of a stimulus did not show reduction over time (one-tailed paired t-tests: t(18) = 0.46, p = 0.326, Fig. 5a; and t(18) = 0.50, p = 0.312, Supplementary Fig. 4a respectively). Furthermore, the chosen value signal encoded by mPFC also did not reduce over time, but instead remained consistent across sessions (Fig. 3c). In addition, performance on the imagination task improved across blocks (Fig. 5b–c). Rather than a loss of sensitivity, this suggests that the diminishing adaptation effects demonstrate that simulated experience is sufficient to establish an independent representation of the novel good that no longer needs to be reconstructed anew on each trial.

Despite the overall reduction of cross-component suppression over the course of the experiment, this was not true for components that had been used to construct high value novel goods. When averaging across the final two blocks, both the mPFC and hippocampus showed a significant positive correlation with the value of the compound items (mPFC: r = 0.64, p = 0.002, Fig. 6b; hippocampus: r = 0.63, p = 0.003, Fig. 6d; after accounting for variance explained by the value of the component items in both cases). After accounting for variance explained by component value, a median split of participants according to the value assigned to the novel goods verified that there was long lasting plasticity in mPFC and hippocampus in the final two blocks for those participants who attributed high but not low values to the novel goods (mPFC: ‘High’ t(8) = 2.84, p = 0.022, and ‘High vs Low’ t(8) = 2.68, p = 0.028, Fig. 6c; Hippocampus: ‘High’ t(8) = 3.52, p = 0.008, and ‘High vs Low’ t(8) = 5.36, p<0.001, Fig. 6e). Suggestive evidence that value-dependent adaptation between related component items emerged later in hippocampus relative to mPFC (‘High’, Fig. 6c vs 6e), could not be verified statistically (t(8) = 1.30, p = 0.229). Together these results suggest that the plasticity is long lasting if value is attributed to the original association.

Discussion

The role of memory in prospective evaluation and inference has previously been emphasized in both animals²² and humans^3,4,20. Simulation and preplay can be used to explore an ‘internal model’ of the environment and evaluate anticipated outcomes^8,23. However, the neural mechanisms by which these processes are achieved has remained unclear, particularly in circumstances where anticipated outcomes have not previously been experienced. Here we used repetition suppression in fMRI to reveal a neuronal mechanism that supports prospective representation and evaluation of novel experiences.

Repetition suppression has been used extensively in sensory brain regions to probe the information content of neural activations, and more recently in more frontal brain regions including orbitofrontal cortex²⁴. However, a number of different hypotheses have been proposed to explain the underlying physiological mechanisms behind the phenomenon, including fatigue, sparse coding and predictive coding^10,25–27. Although there is not yet a consensus on which mechanism provides the most appropriate explanation for the phenomenon, when used in a carefully controlled experimental design, the consequences of this ambiguity is mitigated since all models make the same prediction: if a neural population is sensitive to a particular feature or dimension, then suppression will occur in response to a repetition of this feature, but not others.

The repetition suppression paradigm used here was designed to allow interrogation of the underlying representation of a novel reward. By asking people to imagine and evaluate novel rewards in the scanner, we found that the neural representation of a novel reward was dependent upon representations of multiple related and previously experienced rewards. Our data suggest that neuronal networks can construct a novel experience by simultaneous activation of multiple previous memories, in order that this constructed experience may be evaluated. Whilst signals in the anterior hippocampus were found to be related to construction, those in medial prefrontal cortex were related to both construction and valuation.

Crucially, unlike other goal directed decision mechanisms that have been reported previously^21,23,28,29, we only found evidence for a construction mechanism when subjects had no direct experience of an outcome, and even then only fleetingly. It is possible, therefore, that constructed value can provide a substitute for direct experience, and train the experiential goal-directed systems that have been studied previously. This training signal may be considered analogous to off-line training of an habitual system which makes use of simulations from an internal goal-directed model^23,30–32. Whereas the teaching signal provided to a habitual system replicates, or fine-tunes, previous sensory experience, the teaching signal provided to a goal-directed system may establish an internal model of the future world by repeated imagination of a novel experience.

During the construction process, a second repetition suppression effect was observed between distinct and previously unassociated memories that contributed to the construction. This effect implies that the neural representation of related, compared to unrelated, component items became more similar as a consequence of the pre-scan training task, during which the participants were first exposed to the novel compounds. Notably, since the suppression was not observed in the familiar group it seems highly unlikely that this suppression effect reflects inherent similarity between related compared to unrelated components. Rather, the most plausible explanation for this change is that through repeated representation of a novel compound, previously unrelated memories were recruited simultaneously, inducing a form of plasticity between the underlying representations of necessary components.

Within both brain regions involved in construction, the mPFC and hippocampus, plasticity between related components was dependent upon the value of the novel compounds, but not the value of components. This value dependence effect suggests that the representations of the component memories were simultaneously present during valuation of the novel compounds. A number of different mechanistic explanations may underlie this dependency. For example, the occurrence of greater BOLD activity at the time of pairing may induce more plasticity, or alternatively when representing a higher value compound, the enhanced availability of neuromodulators, such as dopamine, may serve to facilitate plasticity.

Given that on average participants showed a reduction over time in the initial plasticity observed in mPFC, with comparable dynamics to the construction mechanism, it must be acknowledged that it remains ambiguous whether the adaptation observed between related components reflects classical Hebbian plasticity, or even occurs in the same regions to those where the repetition suppression is observed. However, those participants who assigned high value to the novel goods, showed plasticity in mPFC that outlasted the construction process. In the hippocampus, where plasticity was not observed early on in the experiment, the same participants showed plasticity late in the experiment. Therefore, the extent to which neural representations of related component became more similar to one another, but also the durability of the effect, was dependent upon value attributed to the novel compounds. Irrespective of the underlying nature of the plasticity, the influence of compound value upon component memories thus supports the claim that these representations are paired together at the time of construction of the novel compounds.

The medial prefrontal cortex is regularly activated in studies of valuation^{33,17,34–36,18}, and is particularly notable amongst such reward-related regions for the flexibility of the value signals that it contains. These computations may, for example, rely on an understanding of the complex structure of the environment⁵, the generalisation of concepts learnt in different situations³, or the integration of several disparate sources of information³⁷. If subjects are asked to ignore all of their own experiences and preferences, and instead to guess what a very different individual would choose, mPFC value signals can immediately reflect the preferences of this new individual^7,38. Such online evaluation is a hallmark property of ‘goal-directed’ choices, which are frequently contrasted with ‘habitual’ or overlearnt choices in studies of animal and human behaviour^{6,21,23,29,39,40}. Previous studies of goal-oriented behaviour have, however, focused on situations where values are known, but must be associated with a particular course of action by interfering the structure of the world^1,23,29. Our data suggest that medial prefrontal cortex can combine previous experiences to construct prospective outcomes de novo on each trial, and can then evaluate these constructed outcomes.

Hippocampal preplay mechanisms are known to be important substrates for goal-directed spatial decisions in rodents^8,41, and hippocampal value signals can be recorded in situations where outcomes must be inferred from knowledge of relationships between stimuli in the world^1,42. Notably, hippocampal activity is often recorded in concert with a network involving mPFC in studies of spatial memory and scene construction^12,19,43. Consistent with the proposed function of memory in prospective inference^44,45, the formation of associative links^46,47, and constructive episodic simulation^48,49, our data suggest that hippocampal activity can also play an active role in constructing de novo experiences in non-spatial contexts.

These findings show that a potential new experience can be prospectively represented and evaluated by invoking multiple memories simultaneously within hippocampus and medial prefrontal cortex. By highlighting this neuronal mechanism we provide unique insight into the neuronal computations underlying flexible behaviors that dominate human decision making and which are difficult to study in animal models.

Online Methods

Participants

Thirty-nine healthy volunteers participated in the fMRI experiment, and were assigned to one of two groups (unfamiliar and familiar) by drawing from Matlab’s pseudo-random number generator. One participant (from the familiar group) was excluded from all further analyses due to poor performance (less than 80% accuracy on task performance during any one session). All remaining participants (19 unfamiliar participants with mean age: 28.0, 13 females, and 19 familiar participants with mean age: 27.3, 10 females) were included in further analyses with the exception of one participant (from the unfamiliar group) who was excluded from analysis of the decision task due to parameter estimates being more than 3.5 standard deviations away from the group mean. The final sample size was comparable to that commonly used in fMRI studies. All participants refrained from eating for two hours prior to the start of the experiment. The study was approved by a local UCL ethics committee (ref. number 3486/001) and all participants gave informed written consent.

Experimental task

Behavioral training

Thirteen different novel food combinations, or ‘goods’, were presented to the participants along with their names: tea-jelly, tomato-jam, popcorn-jelly beans, beetroot-custard, onion-mints, pea-mousse, olive-strawberry, pesto-nutella, spinach-pineapple smoothie, raspberry-avocado smoothie, vanilla-salt, yoghurt-pretzels, and coffee-yoghurt. Each good was formed by combining two familiar component food types which had not previously been tasted together (Fig. 1a). The experimenter chose two novel goods, AB and CD, for each participant, under the constraint that the participant liked all four individual component foods (A and B, C and D) from which the two novel goods were formed. All participants were given a small sample of the components (A, B, C, D) to eat, but only participants in the familiar group were allowed to taste, smell and handle the novel goods.

The experimenter randomly assigned two abstract pink shapes to each novel good (AB and CD) and to their respective components (Fig. 1c). Participants were then actively trained on the twelve stimulus-item pairings using a reaction time task. On each trial one of the twelve abstract shapes was shown for 400ms before all six possible items were presented in randomized positions across the screen. Participants were instructed to press the button associated with the correct item as quickly and accurately as possible. Participants were required to continue with this stimulus-item learning task until their average reaction time per block approached 800ms with 100% accuracy.

Scanning

Whilst in the scanner, participants performed two different tasks, a decision making task and an imagination task. The tasks were evenly divided across the three twenty-minute scan sessions with 26 decision trials, and 240 imagination trials in each session.

During the decision task, participants performed 78 choice trials. On each trial, photographs of two novel goods were shown on the screen and participants were given four seconds to evaluate the two options, before being prompted to indicate their preference (Fig. 1b). To encourage veridical evaluation, participants agreed to eat the chosen food from one of their decision trials (chosen at random by the computer) after exiting the scanner.

During the imagination task, an abstract pink shape was presented for 800ms on each trial and this served as an instruction cue to vividly imagine the food item associated with the shape (Fig. 1d). The inter-trial interval was selected from a truncated gamma distribution with mean 2.5s. The trials were sorted into seven principal categories with thirty-two trials of each category per scan session, presented in a randomized order. The seven different categories were as follows: (1) novel good preceded by related component (AB preceded by A); (2) novel good preceded by unrelated component (AB preceded by C); (3) component preceded by the related component (A preceded by B); (4) any food item preceded by another unrelated food item in the same category, such as component preceded by unrelated component or novel good preceded by the other novel good (A preceded by C, or AB preceded by CD); (5) any food item (component or novel-good) preceded by the same food item but predicted by a different abstract stimulus (A preceded by A, or AB preceded by AB); (6) any food item (component or novel-good) preceded by the same food item and predicted by the same abstract stimulus (A preceded by A, or AB preceded by AB); (7) abstract stimulus which had no association with a food outcome preceded by itself. The remaining sixteen trials were necessary to ensure equal numbers of trials in each of the seven conditions.

Of particular interest for our analysis were conditions 1:4. We reasoned that neuronal assemblies which used the components to construct the novel goods would show adaptation (reduced response) in (1) relative to (2). Furthermore, if imagination of a component caused activation of the related component, we would expect reduced response in (3) relative to (4).

For each scan session, fourteen yes/no questions were randomly presented during the imagination task. Each question concerned properties of the food associated with the abstract shape on the last trial. For example asking ‘Was the outcome salty?’. Participants received fifty pence for each correct response. The adjectives used were chosen to encourage participants to elicit multisensory representations of each item, and concerned the appearance, texture, taste and smell of the food items.

Post-scan behavioral task

At the end of the scanning session, after participants had not eaten for a total of five hours, they were sold one of the novel-goods using a Becker-DeGroot-Marschak (BDM) auction procedure¹⁴, using a previously reported protocol⁵⁰. The BDM is known to elicit a measure of a participant’s willingness to pay for a good³⁵, therefore providing a measure of subjective value for each novel good.

fMRI data acquisition and pre-processing

T2*-weighted echo-planar images (EPI) with blood oxygen level-dependent (BOLD) contrast were acquired using a 32-channel head coil on a 3Tesla Trio MRI scanner (Siemens, Erlangen, Germany). A special sequence was used to minimize signal drop out in the OFC region and included an echo time (TE) of 70ms, a tilt of 30^o relative to the rostro-caudal axis and a local z-shim with a moment of −0.4 mT/m ms applied to the OFC region. To achieve whole-brain coverage, we used 43 transverse slices of 2mm thickness, with an inter-slice gap of 1mm and in-plane resolution of 3×3 mm, and collected slices in an ascending order. This lead to a repetition time (TR) of 3.01 seconds. In each session roughly 430 volumes were collected (~20 minutes) and the first five volumes were discarded to allow for T1 equilibration effects. A fieldmap with dual echo-time images (TE1= 10ms, TE2 = 14.76ms, whole brain coverage, voxel size 3×3×3 mm) and a single T1-weighted structural image with 1×1×1mm voxel resolution was acquired for each participant to correct for geometric distortions and co-register the epi images respectively.

Preprocessing and statistical analyses were carried out using SPM8 (Wellcome Trust Centre for Neuroimaging, London, UK, www.fil.ion.ucl.ac.uk/spm). After discarding the first five volumes, images were corrected for signal bias, realigned to the first volume, corrected for distortion using fieldmaps, normalized to a standard epi template, and smoothed using an 8mm full-width at half maximum Gaussian kernel.

Data analysis

Images were analyzed in an event related manner using a general linear model (GLM) involving thirty-two explanatory variables. Twenty-six explanatory variables corresponded to conditions 1 to 6 which were further divided into the different food types. Four additional explanatory variables described the ‘no-outcome’ trials, the time of question presentation, the response to question, and the time of evaluation during the decision trials. Two parametric regressors were included, corresponding to the participant’s subjective value of the two novel-goods and time locked to the onset the decision trials. An additional twenty-three nuisance regressors were included in the GLM to account for motion-related artifacts and physiological noise.

The primary aim of our analysis was to identify the neural mechanism underlying the construction of a novel-good. To detect brain regions involved in evaluating the novel goods we looked for activity modulated by chosen value during the decision task. To detect brain regions involved in constructing the novel-goods (‘component-to-compound’) we used the contrast: [(AB preceded by C) -(AB preceded by A)], averaging across all possible permutations (i.e. explanatory variables (2)-(1) from above). To detect plasticity effects between the related components (‘component-to-component’) we used the contrast [(A preceded by C)-(A preceded by B)], again averaging across all possible permutations (i.e. explanatory variables (4)-(3) from above). To detect brain regions showing adaptation to repeated item but not stimulus (‘item-to-self’) we used the contrast [(item preceded by different item)-(item preceded by itself but paired with a different stimulus)], i.e. explanatory variables (4)-(5) from above. To detect brain regions showing adaptation to repeated stimulus (‘stimulus adaptation’) we used the contrast [(stimulus preceded by different stimulus and different item)- (stimulus preceded by itself)], i.e. explanatory variables (4)-(6) from above. The contrast images of all participants were entered into a second level random effects analysis.

For our initial analyses, we assessed contrasts using whole-brain family wise error (FWE) corrected statistical significance. The cluster defining threshold was p<0.01 uncorrected and the corrected significance level defined as p<0.05. In the unfamiliar group, effects in mPFC were significant at the FWE corrected cluster level (shown in Figure 2a–c). To statistically assess hippocampal activity in our contrasts of interest (which did not survive cluster-based FWE thresholding), we tested the average signal from within a region of interest (ROI). This ROI approach was also used to test for a difference in mPFC signal across groups and across task blocks, and to test for repetition suppression effects in visual regions.

All ROIs were defined from contrasts that were orthogonal to the contrasts of interest, to allow statistical tests to be performed in an unbiased fashion. To define an ROI in the hippocampi we used the contrast identifying adaptation of ‘item-to-self’ averaged across all blocks (see above, thresholded at p<0.01 uncorrected). Firstly, a hippocampal ROI was defined from the unfamiliar group alone (see Fig. 2b), and secondly from the average of both groups (shown in Fig. 4b, Fig. 5a, and Fig. 6a, and used thereafter). To assess all construction related adaptation effects in mPFC, across groups and between blocks, we defined an ROI from the average of the two construction related contrasts (‘component-to-compound’ and ‘component-to-component’) from both groups and across all blocks (thresholded at p<0.001 uncorrected; ROI shown in Fig. 4a, Fig. 5a, Fig. 6a).

To compare the value signals in mPFC encoded by the two groups during the decision task, an ROI was defined from the average of the contrast for chosen value across the two groups (thresholded at p<0.01 uncorrected; shown in Fig. 3b). To investigate whether adaptation of component ‘item-to-self’ in mPFC reduced across the duration of the experiment, an ROI was defined using the ‘item-to-self’ contrast when including only component trials, and averaged across all blocks in the unfamiliar group (thresholded at p<0.01 uncorrected; shown in Fig. 5a). A final ROI was defined in visual regions to test the specificity and temporal dynamics of adaptation effects, and defined from a contrast identifying a main effect to any visual event, averaged across all blocks (thresholded at p<0.01 uncorrected; shown in Supplementary Fig. 4b). Note that differences between blocks or groups are, by definition, orthogonal to the group and block average effect.

The ROIs were then used to extract parameter estimates (as shown in the bar plots in Fig. 3:6) to test for significance between groups and across time. To assess differences between groups, we used a 3-way ANOVA to test for a main effect of and an interaction between: ‘group’, unfamiliar/familiar; ‘adaptation type’, component-to-compound/component-to-component; and ‘condition’, control/adaptation trial specific to the relevant adaptation type. To assess difference across time, we used a 3-way ANOVA to test for a main effect of and an interaction between: ‘block’, block 1/block 2 and 3; ‘adaptation type’, component-to-compound/component-to-component; and ‘condition’, control/adaptation trial specific to the relevant adaptation type. Post-hoc t-tests were then used to decompose the results of the ANOVA, using one-tailed t-tests to assess changes in signal within a group, and one-tailed two sample t-tests to assess differences across groups. The Kolmogorov-Smirnov goodness-of-fit hypothesis test was used to check that data were approximately normally distributed.

In the unfamiliar group, first a partial correlation was performed between ‘component-to-component’ suppression effects and the average value participants assigned to the two novel goods (during the BDM), after removing signal attributable to the component value (see Fig. 2d–e and Fig. 6b,d). The adaptation signal was extracted from mPFC ROI shown in Fig. 4a, 5a, and 6a, and hippocampus ROI shown in Fig. 4b, 5a and 6a, averaged across all blocks (Fig. 2d–e) and then repeated using the adaptation signal from the final two blocks of trials (Fig. 6b,d). The former correlations were compared with one between adaptation effect size and average component value, using a similar partial correlation with average component value after removing effects attributable to compound value (see Supplementary Fig. 1). One participant was excluded from these correlations due to missing data for the value of items. Finally, participants were divided using a median split according to high and low value attributed to the novel goods, and two-tailed t-tests and two-tailed paired t-tests used to assess adaptation in the latter two blocks between related components after variance attributable to the average component value had been removed.