Negative reinforcement impairs overnight memory consolidation

Abstract

Post-learning sleep is beneficial for human memory. However, it may be that not all memories benefit equally from sleep. Here, we manipulated a spatial learning task using monetary reward and performance feedback, asking whether enhancing the salience of the task would augment overnight memory consolidation and alter its incorporation into dreaming. Contrary to our hypothesis, we found that the addition of reward impaired overnight consolidation of spatial memory. Our findings seemingly contradict prior reports that enhancing the reward value of learned information augments sleep-dependent memory processing. Given that the reward followed a negative reinforcement paradigm, consolidation may have been impaired via a stress-related mechanism.

Why do we remember some things and not others? The human brain is constantly bombarded with a barrage of stimuli, and must decipher which newly encoded information is important enough to consolidate into long-term memory. It is now clear that memory consolidation is facilitated by sleep (Plihal and Born 1997; Stickgold et al. 2000; Mednick et al. 2002; Walker et al. 2002; Tucker et al. 2006; Drosopoulos et al. 2007; Ellenbogen et al. 2007; Alger et al. 2012). However, sleep may not benefit all memories equally—emerging evidence suggests that sleep-dependent memory consolidation is influenced by a variety of factors including emotion (Payne et al. 2008), expected future utility (Wilhelm et al. 2011a), and reward (Fischer and Born 2009; Oudiette et al. 2013). These seemingly disparate features of a learning experience may all enhance the “salience” of learned information, increasing the degree to which an experience is noticeable and/or important to the organism. Here, we examined the impact of task salience on overnight memory consolidation by manipulating the presence of (1) reward and (2) feedback, in the context of a virtual navigation task.

Several studies have now demonstrated that consolidation of human route-learning benefits from post-training sleep (Ferrara et al. 2006, 2008; Orban et al. 2006; Wamsley et al. 2010b; Nguyen et al. 2013). As for other forms of hippocampal-dependent learning (Plihal and Born 1997; Lau et al. 2011; Alger et al. 2012), human spatial learning may be facilitated by slow wave sleep (SWS) and/or slow wave activity (SWA) (Peigneux et al. 2004; Wamsley et al. 2010b). Because models of the brain basis of memory consolidation draw heavily on studies of spatial navigation in rodents, studying this memory system in humans provides an important analog to the rodent sleep and memory literature. Rodent studies have shown, for example, that recently encoded memories are reactivated during sharp wave-ripple events in the hippocampus during SWS (Wilson and McNaughton 1994; Kudrimoti et al. 1999; Ji and Wilson 2006), which may facilitate incremental consolidation and long-term storage in cortical circuits through a process of synaptic potentiation. An alternative mechanistic account suggests that sleep facilitates memory performance by the “competitive down-selection” of synapses that are activated infrequently and fit less well with the overall organization of memories (Tononi and Cirelli 2006; Nere et al. 2013).

While several studies have suggested that sleep preferentially “selects” some memories for consolidation over others, perhaps based on the importance of the learned information, it remains poorly understood why certain memories are preferentially consolidated during sleep. Payne et al. (2008, 2012) have shown that sleep preferentially consolidates memory for emotionally negative objects, while neutral objects and background information are forgotten. Two recent studies have reported that informing participants that they will be tested on material promotes its selective consolidation (Wilhelm et al. 2011a; van Dongen et al. 2012). Meanwhile, Fischer and Born (2009) observed that providing subjects with a reward incentive post-learning resulted in enhanced consolidation of a motor learning task during sleep. Similarly, using an object–location association task, Oudiette et al. (2013) reported that memory items with higher reward value show less deterioration across a night of sleep than those associated with a lower reward value. Together, these studies seem to suggest that something about the importance or salience of encoded information may mediate its consolidation during sleep.

However, the results of such studies have not been uniform. For example, Tucker et al. (2011) found no effect of reward on sleep-dependent consolidation of paired associates learning. Similarly, Baran et al. (2013) recently reported that while increasing the “value” of items to be learned boosted performance at encoding, item value had no effect on overnight consolidation of a declarative memory task.

The present study extends the concept of “selective” memory consolidation during sleep to the spatial domain, asking whether increasing the salience of a spatial navigation task leads to enhanced overnight memory consolidation. Extending the selective consolidation concept to new memory domains provides an opportunity to delineate the boundary conditions for this phenomenon, continuing to examine whether this purportedly generalizable concept indeed replicates under new paradigms. Participants trained on one of four closely matched versions of a virtual maze navigation task (VMT) (Fig. 1) prior to a night of sleep and were retested on the task the following morning. In the Reward group, a monetary reward value was displayed on the center of the screen and counted down as the participant played the maze—participants were informed that the remaining money was theirs to keep at the end of the trial, and that they would similarly receive a performance-based reward during the test trials the next morning. In the Feedback group, the sound of running water served as an auditory beacon, becoming louder with increasing linear distance from the exit door. The third version of the task contained both Reward and Feedback (Both) while the fourth version contained neither element (Neither). We selected these groups based on previous evidence that Reward may play a significant role in offline memory consolidation during sleep (Fischer and Born 2009; Oudiette et al. 2013), and based on the hypothesis that interactive feedback could similarly affect sleep-dependent consolidation by increasing participant engagement in the activity. We hypothesized that by enhancing the salience of the navigation task, these features would “flag” spatial information with higher priority for sleep-dependent memory consolidation and incorporation into dream content.

Figure 1.

The virtual maze task (VMT). (Right) In this spatial memory task, subjects explore the layout of a complex environment, attempting to navigate to the maze exit during a series of trials at evening training, and again at a delayed test the following morning. The screenshot shows participants’ view, including a landmark (sphere). In the Reward version of the task, the current reward value is displayed on the screen for the duration of each trial, counting down as the participant endeavors to reach the exit. (Left) An overhead view of the maze layout (not seen by participants). Star indicates the location of the exit door.

Sixty-five healthy volunteers between the ages of 18 and 30 (mean = 21 ± 2 SD) were included for analysis in the Feedback (n = 16), Reward (n = 17), Both (n = 17), and Neither (n = 15) groups (Table 1; see Supplemental Methods for exclusion criteria). All participants trained on the VMT prior to a night of laboratory-monitored sleep, and were tested on their memory for the virtual environment the following morning. Dependent measures for each trial included completion time (seconds to reach exit), distance traveled (path length to reach exit), backtracking (proportion of distance traveled attributed to “retracing steps”), and speed (distance traveled/min). For each measure, overnight performance improvement was quantified as percent improvement from the last evening training trial to mean performance on the morning test trials. At evening training and morning test, participants rated their sleepiness and alertness, as well as perceived difficulty, interest, emotional valence, and declarative knowledge of the exit location (see Supplemental Methods). After morning testing, participants completed questionnaires assessing task rehearsal and spatial strategy (egocentric vs. allocentric). Additional methodological details are described in the Supplemental Methods.

Table 1.

Baseline variables by condition

Contrary to our expectations, reward had a significant negative effect on overnight improvement in maze completion time (main effect of Reward: F_(1,55) = 5.32, P = 0.02, Fig. 2). On average, participants who were not rewarded (Feedback + Neither) showed an 18% ± 33% SD improvement across the night, while participants who received the monetary reward (Reward + Both) deteriorated in performance by 11% ± 60% SD, despite a lack of group differences at encoding. Reward also had a significant negative effect on overnight improvement in distance traveled (main effect of Reward: F_(1,55) = 5.18, P = 0.03), although not on backtracking (P = 0.40) or speed (P = 0.17). Importantly, during the evening training session, there were no group differences in any measure of VMT performance (all P > 0.11). Thus, Reward selectively affected overnight change in performance, with no confounding influence of encoding performance.

Figure 2.

Reward impairs overnight improvement in completion time. Reward led to deterioration in performance across a night of sleep. Error bars = SEM.

The reward manipulation also negatively affected participants’ perception of the task, and altered the spatial strategies that participants reported using. At the end of encoding, Reward participants perceived the task as more negatively emotional (trend level, main effect: F_(1,59) = 3.30, P = 0.09) and more difficult (main effect: F_(1,59) = 3.89, P = 0.05), relative to those who did not receive Reward. These effects were not seen for Feedback. Analysis of maze strategy revealed that Reward tended to decrease allocentric strategy use (main effect of Reward: F_(1,57) = 2.92, P = 0.09), especially in the absence of Feedback (Reward × Feedback interaction: F_(1,57) = 4.63, P = 0.04; see also Supplemental Material and Methods).

The effect of Feedback did not reach statistical significance for any objective performance measure. However, Feedback significantly enhanced self-reported declarative knowledge of the exit location, as assessed via questionnaire (Feedback main effect: F_(1,56) = 4.06, P < 0.05; see Supplemental Methods). In addition, there were trends toward a positive effect of Feedback on percent overnight improvement in completion time (main effect: F_(1,55) = 2.41, P = 0.13) and distance traveled (main effect: F_(1,55) = 2.37, P = 0.13). There was no effect of Feedback on backtracking (P = 0.51) or speed (P = 0.57). There was no effect of Reward on self-reported knowledge of the exit location, and there were no Reward × Feedback interactions for improvement on any dependent measure (all P-values >0.29).

Table 2 describes the effect of condition on the composition of sleep during the night. Reward increased total sleep time, while Feedback increased the percent of sleep time spent in Stage 2, and decreased both absolute and percent time spent in slow wave sleep (SWS). At the same time, selectively with the groups that received Feedback (Feedback + Both), there were strong associations between SWS and multiple measures of overnight improvement (Table 3; Fig. 3). This correlation of memory improvement with SWS echoes prior observations of the importance of this sleep stage to other forms of hippocampus-dependent memory consolidation (Plihal and Born 1997; Lau et al. 2011; Alger et al. 2012), which here appears to have been enhanced by feedback.

Table 2.

Comparison of sleep parameters between conditions

Table 3.

SWS correlation with overnight performance improvement

Figure 3.

Slow wave sleep correlated with overnight improvement selectively in feedback groups. Time in SWS was positively correlated with overnight improvement selectively in subjects who received performance Feedback (A; “Feedback” and “Both” groups), but not in subjects without Feedback (B; “Reward” and “Neither” groups).

Based on our prior work, we expected that effects of Reward and Feedback on overnight memory consolidation would be accompanied by changes in the incorporation of the VMT into dream content. Although participants did dream about the VMT, contrary to our hypotheses, there was no effect of experimental group on the appearance of the task in dream reports (see Supplemental Results).

Recently, the concept of “selective” sleep-dependent memory consolidation has received increasing research attention (Payne et al. 2008; Fischer and Born 2009; Wilhelm et al. 2011a; Oudiette et al. 2013; Stickgold and Walker 2013). Using a variety of manipulations, including monetary reward (Fischer and Born 2009; Oudiette et al. 2013), it has been argued that information which is perceived as having greater “importance” to an individual will be preferentially processed during sleep. Here, in direct contrast to our expectations, monetary reward actually impaired overnight memory consolidation.

This finding was initially perplexing as the reward was intended to provide a positive incentive for participants, and our observations seem to directly contradict those of at least two prior studies (Fischer and Born 2009; Oudiette et al. 2013). However, it is important to note that the design of the reward conditions followed a “negative reinforcement” paradigm—participants believed that they were continuously losing money throughout the duration of each trial, until they were able to successfully locate the maze exit. Successful navigation was thus reinforced via its association with escaping from the adverse experience of losing money. That reward participants rated the task as difficult and emotionally negative supports the view that these task versions created a stressful situation from which escape was desirable.

If the reward version of the task induced a stress-related cortisol increase, this may have impaired consolidation of this spatial learning task. Indeed, there is strong reason to think that stress prior to sleep could adversely affect consolidation, particularly of hippocampus-dependent spatial memory. Several studies have now demonstrated that elevated cortisol levels during early sleep impair overnight memory consolidation (Plihal and Born 1999; Plihal et al. 1999; Wilhelm et al. 2011b). This has led to the proposal that consolidation of hippocampus-dependent memory requires the natural inhibition of cortisol that occurs during early-night slow wave sleep, which would facilitate hippocampal-cortical communication during this time. Elevated cortisol levels during early-night sleep, which immediately followed training, could have interfered with this communication. In such a case, any positive impact of increased salience in the Reward groups would have been overwhelmed by stress effects, leading to a net diminished overnight memory consolidation. This resembles what is seen in PTSD (post-traumatic stress disorder), in which a failure to suppress excessive vigilance prevents normally sleep-dependent processing of the traumatic memory (Stickgold 2002, 2007). In fact, overnight consolidation of virtual maze learning is impaired in PTSD patients (Tempesta et al. 2011), in whom cortisol levels may be chronically elevated (Carrion et al. 2002).

Additionally, subjects in the Reward group reported greater dependence on egocentric, rather than allocentric, navigation strategies. Interestingly, stress provides a potential mechanism for this effect as well. Glucocorticoid receptors are strongly represented in the hippocampus, especially in the CA1 region, which is thought to be particularly important for allocentric spatial representation (Suthana et al. 2009). Meanwhile, exposure to stress has been shown to impair hippocampus-dependent allocentric spatial navigation, and increase reliance hippocampus-independent response strategies (Schwabe et al. 2007; Faraji et al. 2013). Thus, elevated cortisol could have altered hippocampal function in such a way that navigation strategy during both training and retest shifted away from the use of hippocampus-dependent allocentric representation and towards hippocampus-independent egocentric representation. In the presence of auditory feedback this effect may have been less pronounced, as the auditory beacon provided additional spatial information.

SWS was correlated with overnight improvement selectively in the Feedback groups, where consolidation was marginally enhanced. SWS is classically thought to be associated with consolidation of hippocampus-dependent learning (Plihal and Born 1997; Peigneux et al. 2004; Marshall and Born 2007; Mölle and Born 2009; Nguyen et al. 2013), and we thus speculate that Feedback and Reward may have led to qualitatively different forms of sleep-dependent processing, with the hippocampus being more strongly engaged following Feedback training. This interpretation is supported by the shift away from hippocampus-dependent allocentric strategy in the Reward group, and may also have resulted in the observed sleep architecture differences between groups.

Of course, neither stress nor cortisol levels were directly measured in the present study, and these mechanisms remain hypothetical. In order to test the hypothesis that elevated stress impairs sleep-dependent consolidation of spatial memory, subsequent studies might sample endogenous cortisol during encoding and consolidation, alongside subjective ratings of stress. Interestingly, a recent study sampled resting cortisol prior to encoding and emotional memory task and found that higher pre-encoding cortisol was associated with improved memory following a period of sleep (Bennion et al. 2013). The apparent contrast with our observations here might stem from the fact that participants in this study were not exposed to a stressful manipulation, and resting cortisol levels were low (Bennion et al. 2013). As postulated for waking memory encoding and consolidation (Salehi et al. 2010), stress and cortisol might enhance sleep-dependent memory consolidation at low levels, but have an impairing effect at higher levels.

The present experiment differs from prior studies of reward and sleep-dependent memory consolidation in that we employed a negative reinforcement paradigm, but it is not certain whether this feature of the design was the deciding factor in the observed outcome. By systematically varying the type of reinforcement scheme (e.g., negative versus positive reinforcement), future studies may be able to address whether the use of negative reinforcement universally impairs sleep-dependent memory consolidation.

We demonstrated an effect of sleep on this task in several prior studies (Wamsley et al. 2010a,b; Nguyen et al. 2013), and thus are confident that memory for the task is enhanced by post-training sleep, relative to an equivalent period of wakefulness. However, because the present study does not include waking control groups, we cannot say whether the detrimental effect of reward is truly specific to sleep, or whether a similar effect might have emerged across a waking consolidation period. In fact, several prior studies suggest that while elevated cortisol during encoding has a positive effect on emotional memory, memory for neutral information may fail to benefit or even be impaired (Buchanan and Lovallo 2001; Payne et al. 2006). From the present data, we can conclude that the reward manipulation impaired consolidation across a period of sleep, for a task in which consolidation is known to benefit from sleep. Future studies might further test the specificity of this effect.

Is “important” information preferentially consolidated during sleep? Our findings indicate that increasing the reward value of information does not uniformly enhance overnight memory consolidation. Prior studies have shown that tagging learned information with a variety of salience cues can boost sleep-dependent memory consolidation (Payne et al. 2008; Fischer and Born 2009; Wilhelm et al. 2011a; van Dongen et al. 2012). Yet under some conditions, seemingly similar manipulations can actually impair consolidation during sleep. Here negative reinforcement, rather than boosting memory consolidation, led to forgetting across the night. We propose this may have occurred via a stress-induced elevation of cortisol during consolidation, perhaps specifically during early-night sleep. This form of selective processing could reflect an evolutionarily adaptive mechanism whereby it is advantageous to recognize a situation worth avoiding rather than recalling the details of what occurred there.