Eye Tracking, Cortisol, and a Sleep vs. Wake Consolidation Delay: Combining Methods to Uncover an Interactive Effect of Sleep and Cortisol on Memory

Abstract

Although rises in cortisol can benefit memory consolidation, as can sleep soon after encoding, there is currently a paucity of literature as to how these two factors may interact to influence consolidation. Here we present a protocol to examine the interactive influence of cortisol and sleep on memory consolidation, by combining three methods: eye tracking, salivary cortisol analysis, and behavioral memory testing across sleep and wake delays. To assess resting cortisol levels, participants gave a saliva sample before viewing negative and neutral objects within scenes. To measure overt attention, participants’ eye gaze was tracked during encoding. To manipulate whether sleep occurred during the consolidation window, participants either encoded scenes in the evening, slept overnight, and took a recognition test the next morning, or encoded scenes in the morning and remained awake during a comparably long retention interval. Additional control groups were tested after a 20 min delay in the morning or evening, to control for time-of-day effects. Together, results showed that there is a direct relation between resting cortisol at encoding and subsequent memory, only following a period of sleep. Through eye tracking, it was further determined that for negative stimuli, this beneficial effect of cortisol on subsequent memory may be due to cortisol strengthening the relation between where participants look during encoding and what they are later able to remember. Overall, results obtained by a combination of these methods uncovered an interactive effect of sleep and cortisol on memory consolidation.

Download video file ^{(17.9MB, mp4)}

Introduction

The ability to consolidate information is dependent on countless factors, with sleep and the stress hormone cortisol being two of the most influential variables. Prior research has shown that elevated cortisol levels, either induced through exogenous cortisol administration or psychosocial stress, often are associated with a selective enhancement of memory for emotional relative to neutral stimuli^1-7. Sleep has been shown to have a similar selective effect on emotional memory^8,9: When participants are presented with scenes composed of a negative or neutral object placed on a neutral background, sleep unbinds the scenes, preserving memory for the negative objects, while allowing memory for the less salient scene components (neutral objects and neutral backgrounds) to decay^10-12.

While these two literatures focus on the independent effects of cortisol and sleep on emotional memory, it is possible that cortisol and sleep have an interactive effect. Interestingly, because studies investigating the effects of cortisol on memory include a retention delay of at least 24 hr, which necessarily includes a night of sleep, it is impossible to determine whether sleep is necessary for cortisol to have a facilitative effect on consolidation. Similarly, while research has shown that sleep preferentially benefits memory for emotional information, it is possible that these effects may be intensified in individuals with higher cortisol levels during encoding.

For these reasons, it is important to understand not only how sleep and cortisol independently support memory formation, but also to investigate whether sleep and cortisol interact to support memory formation. After all, there are important links between sleep and cortisol in normal aging¹³, and forms of psychopathology that are associated with memory deficits^14,15. Only by using a combination of methodologies is it possible to understand the complex interactions between sleep and cortisol, and their effects on memory consolidation. Specifically, by combining the method of salivary cortisol collection with a sleep versus wake design, it is possible to determine whether there is an interactive effect of these two variables on consolidation. Also valuable is that through the use of eye tracking to measure eye gaze during encoding, it is possible to elucidate a potential attentional mechanism underlying this effect.

Protocol

1. Participant Screening and Preparation for the Experiment

1. Recruit participants who are native English speakers with normal or corrected-to-normal vision. They should be free of neurological, psychiatric, and sleep disorders, and may not be taking any medications affecting the central nervous system or sleep architecture. Be sure to recruit a similar balance of male and female participants within the desired age range of interest (e.g., 18 to 35 years old), keeping in mind that changes in sleep architecture can occur as early as the mid-30s ¹⁶.

2. To most accurately assess the effects of sleep on cognition, have participants maintain a regular sleep schedule and restrict their alcohol consumption leading up to the study. Ask participants to sleep for at least 7 hr a night and be in bed by 2:00 am for the five nights preceding the study. Also ensure that they restrict their alcohol consumption to a maximum of 2 drinks during the 5 days preceding the study, with absolutely no alcohol consumption the day before or day of the study.

3. During scheduling, ensure that participants will comply with the guidelines related to the salivary cortisol sample (see also Section 4): They must refrain from physical activity, eating, drinking (anything besides water), smoking, and brushing their teeth for the 2 hr prior to encoding. They must also refrain from drinking water for at least 15 min prior to encoding.

4. Scheduling participants. If full random assignment is not possible, ensure that participants do not differ in age, scores on the Morningness-Eveningness Questionnaire¹⁷ (MEQ), Beck Depression Inventory¹⁸ (BDI), Beck Anxiety Inventory¹⁹ (BAI), and the amount of sleep obtained on the night before retrieval.

2. Conditions and Experimental Design

1. Schedule Sleep participants such that the encoding session occurs in the evening (7:00-10:00 pm) and the retrieval session occurs 12 hr later, following a full night of sleep in the laboratory. Schedule Wake participants such that the encoding session occurs in the morning (7:00-10:00 am) and the retrieval session occurs 12 hr later following a full day of wakefulness; ensure that they do not nap between sessions.

2. Include Morning and Evening Short Delay conditions (those that have only a 20-min delay between encoding and retrieval, as compared to a 12-hr delay for the Sleep and Wake groups) in the experimental design to minimize the concern that any differences found between the Sleep and Wake groups are due to the time of testing (morning vs. evening) rather than due to the sleep occurring during the consolidation delay. These Short Delay conditions can also be thought of as “circadian control” conditions.

   1. Arrange for participants in the Morning Short Delay condition to encode the stimuli between 7:00-10:00 am, and for participants in the Evening Short Delay condition to encode the stimuli between 7:00-10:00 pm. Test the participants 20 min after encoding.

3. Stimuli Construction

1. Stimuli construction for encoding. Select stimuli based on the specific experimental question. This protocol focuses on the effects of sleep and cortisol on emotional memory, and as such, the visual stimuli during encoding are scenes composed of either a negative object or a neutral object placed on a neutral background.

   1. Ensure that all emotional stimuli have either been previously rated for valence and arousal^20,21, or that they are rated by participants upon completing the study using a Likert scale from 1 to 7. Negative objects should be rated as highly arousing and low in valence (e.g., arousal: 5-7; valence < 3 on 7-point scales with high values indicating high arousal and high positivity, respectively), and neutral objects should be rated as non-arousing and neutral in valence (e.g., arousal < 4; valence: 3-5).
   2. Randomization and design. Randomly intermix the negative and neutral scenes between blocks (if applicable). The present study employs 2 blocks lasting approximately 10 min each, allowing participants to have a short break to rest their eyes from the eye tracker in between. Breaks may be helpful every 10-15 min for most young adult participants, but if testing a different population (e.g., children), more frequent breaks may be required.

2. Stimuli construction for retrieval. Select stimuli based on the specific experimental question. Here, memory for the objects is the focus, and as such, participants are presented with objects and backgrounds (half of which were presented during encoding, and half new) separately during retrieval.

4. Cortisol Procedure

1. Ensure that participants have followed all requirements in 1.3: No physical activity, eating, drinking (anything besides water), smoking, and brushing their teeth for the 2 hr prior to encoding, as well as no water for at least 15 min prior to encoding.

2. Immediately prior to encoding, instruct participants to rinse their mouths with approximately 1 ounce of water. Remind them not to swallow the water, as to avoid sample dilution.

3. Have participants salivate on an oral swab (see Materials) for 2 min.

4. Upon having participants place the oral swab in the swab storage tube, store the swabs at the approximate temperature of 0 ºF until analyzed.

5. Eye Tracking/Encoding Procedure

1. Eye tracking procedure. The eye tracker used here tracks participants’ left eye gaze patterns at 500 Hz (see Materials). Alternative trackers can be used; in order to most accurately assess attention during encoding, follow the instructions of the specific eye tracker used.

   1. First, ask participants to sit with their chin on the chinrest and forehead up against a bar. Make adjustments to the chair height and chinrest as needed, ensuring that the center of the screen aligns with participants’ eyes.
   2. Ensure that the eye tracker is accurately tracking participants’ gaze within 1° of accuracy by having each participant complete a calibration task. Ideally, a 9- or 17-point calibration would be used, depending on the system, but a 3- or 5-point calibration may also be sufficient.
   3. First, ask participants to follow a black dot with their eyes as it moves to different points on the screen and to fixate on it when it stops.
   4. Once the eye tracker is calibrated accurately, ask participants if they are ready to begin the task, and then press the “record” button.
2. Encoding procedure. Ask participants to perform a task that is likely to lead to deep encoding, such as having them indicate via mouse click whether they would approach or back away from the scene (e.g., left = approach; right = back away) if they were to encounter it in real life¹⁰. See Figure 2 for a visual depiction of the encoding procedure.

   1. Allow participants to have a short self-determined break (e.g., ~10-60 sec) between blocks so that they may sit back from the eye tracker and rest their eyes prior to continuing. Ask them to indicate when they are ready to continue.
3. Analysis of eye gaze data. To measure participants’ attention to certain parts of the scene, use software to draw Areas of Interest²² (AOIs) around those parts.

   1. After drawing the AOIs, calculate the proportion of time participants look at the AOI relative to the rest of the scene. Alternatively, count the number of saccades that participants make to that AOI within a certain time frame.

6. Study-test Delay

1. Ensure that the delay length between encoding and retrieval for the Sleep and Wake conditions is equal (e.g., 12 hr), as well as the delay length for the 2 control conditions (e.g., 20 min).

   1. For the Sleep participants, ensure that the 12 hr delay includes approximately 8 hr of sleep. Conversely, ensure that the Wake participants do not sleep or nap during this interval.
   2. Ask the Morning and Evening Short Delay participants to remain in the laboratory during their 20 min delay. Tell them that they may do whatever they please during this time, provided that they do not nap.

7. Recognition Procedure

1. Following the delay period, give participants a memory test.

   1. Ask participants to indicate whether the displayed stimulus is “old” (included in a previously studied scene), or “new” (not previously studied) by pressing corresponding keys on a keyboard (e.g., “1” = old; “2” = new).

Representative Results

Effects of Sleep and Cortisol on Memory for Emotional and Neutral Stimuli

The first hypothesis addressed is that elevated cortisol during encoding will facilitate memory for emotional more than neutral stimuli, and that this effect is dependent on sleep occurring between encoding and retrieval. Figure 4A plots the effect of cortisol on memory for negative objects. Standardized levels of cortisol (x-axis) and memory for negative objects (y-axis) were directly related in the Sleep group (in red) but not the Wake group (in gray). The Group (Sleep vs. Wake) by Cortisol interaction was significant [t(41) = 2.23, β = 2.92, p = 0.031]: Higher cortisol at encoding predicted memory for negative objects if participants slept between encoding and retrieval [t(24) = 2.31, β = 0.43, p = 0.031], but not if they stayed awake [t(16) = 0.40, β = 0.10, p = 0.70; see Figure 4A]. These significant effects were not due to gender, menstrual cycle, or critically, time of day, which was determined by running additional analyses with the Morning and Evening Short Delay groups. For neutral memory (see Figure 4B), a similar, but weaker pattern was observed. There was a marginally significant relation between cortisol levels prior to encoding (x-axis) and memory for neutral objects (y-axis) in the Sleep group [in blue; t(24) = 1.76, β = 0.34, p = .092], but not the Wake group [in gray; t(16) = 0.98, β = 0.25, p = 0.34]. The interaction between Cortisol and Group was marginally significant [t(41) = 1.95, β = 2.55, p = 0.059].

Effects of Sleep and Cortisol on the Interaction between Attention during Encoding and Consolidation

It was hypothesized that this beneficial effect of cortisol on emotional memory may be partly due to cortisol’s ability to ‘tag’ information as important to remember at the time of encoding, leading to the subsequent prioritization of that information during sleep. This “emotional tagging” concept suggests that encoding arousing stimuli activates neural mechanisms, leading to long-term plasticity in the synapses marked by the tag^23-25. It is possible that elevated cortisol during encoding helps to set these tags, leading to the selective preservation of this information during consolidation. To investigate this possibility, the eye-tracking data were analyzed to determine whether higher cortisol increases the likelihood that sleep-based consolidation processes preferentially strengthen memory for the information that receives the most attention during encoding. First, the proportion of time each participant spent looking at each object within each scene (i.e., the AOI) relative to the total scene viewing time was calculated. The scenes were then sorted on a post-hoc basis, using each participant’s recognition data to sort the scenes into those for which the participant later remembered the object and those for which the participant later forgot the object. Lastly, a score was computed to reflect the difference in looking time between subsequently remembered and subsequently forgotten objects (See Figure 5). For example, if participants looked at the objects they subsequently remembered for an average of 75% of the time that the scene was on the screen, and looked at the objects they subsequently forgot for an average of 65% of the time that the scene was on the screen, their difference in looking time score would be 10%.

Similar to the analyses conducted on the effects of sleep and cortisol on memory (Figure 4), a linear regression was used to test the effects of sleep and cortisol on this difference in looking time at encoding as a function of later memory. For negative objects (see Figure 6A), resting cortisol (x-axis) marginally predicted the difference in looking time at encoding as a function of later memory (y-axis) in the Sleep group [in red; t(23) = 1.869, β = 0.37, p = 0.075] but not the Wake group [in gray; t(16) = 0.168, β = 0.043, p = 0.87]. The interaction between Cortisol and Group was significant [t(40) = -2.04, β = -2.99, p = 0.049], and critically, this significant effect was not due to gender, menstrual cycle, or time of day. For neutral objects (see Figure 6B), there was no effect of cortisol in the Sleep group (in blue) nor the Wake group (in gray), and the interaction between Cortisol and Group was not significant.

Figure 1. Visual depiction of the procedure described in this video report, separated by Group. This figure displays the four groups of participants (Sleep, Wake, Morning Short Delay, and Evening Short Delay), as well as the timing of the pre-encoding cortisol sample, encoding, and retrieval for each group.

Figure 2. Visual depiction of stimuli used during encoding. This figure shows that each scene was composed of either a negative or a neutral object placed in front of a neutral background. It also shows that participants viewed these scenes for three seconds each, during which time they indicated whether they would approach or back away from the scene if they encountered it during real life.

Figure 3. Visual depiction of stimuli used during retrieval. This figure shows that participants were presented with objects and backgrounds (separately) during the recognition memory test. These objects and backgrounds were either previously presented during encoding (“old”) or had never before been seen in the context of the experiment (“new”).

Figure 4. Effect of cortisol on memory for negative and neutral objects. A plots the effect of standardized cortisol levels on memory for negative objects. B plots the effect of standardized cortisol levels on memory for neutral objects. Legend: Sleep [red diamonds (neg), blue diamonds (neu)], Wake [gray squares], Sleep Linear Fit [red line (neg), blue line (neu)], Wake Linear Fit [gray line].

Figure 5. Visual depiction of the encoding (left) and retrieval (right) procedure as related to the dependent variable (difference in looking time during encoding as a function of later memory) assessed in eye-tracking analyses. This figure shows how the dependent variable in eye-tracking analyses (the difference in looking time during encoding as a function of later memory) was calculated. Particularly, this score reflects the proportion of time during encoding that participants looked at objects that they later remembered (“hits”) minus the proportion of time during encoding that participants looked at objects that they later forgot (“misses”).

Figure 6. Effect of cortisol on the difference in looking time at encoding as a function of later memory for negative and neutral objects. Looking time was calculated as the proportion of total scene viewing time that participants spent looking at the object within the scene. A score was then computed to reflect the difference in looking time between subsequently remembered and subsequently forgotten objects, and a linear regression was used to test the effects of cortisol and sleep on this score. A plots the effect for negative objects, while B plots the effect for neutral objects. Legend: Sleep [red diamonds (neg), blue diamonds (neu)], Wake [gray squares], Sleep Linear Fit [red line (neg), blue line (neu)], Wake Linear Fit [gray line].

Discussion

This experimental design provided the first evidence that the beneficial effects of pre-encoding cortisol on memory are significant only when sleep occurs during the consolidation period. Only by both measuring cortisol levels and manipulating whether the consolidation interval included sleep was it possible to determine that sleep and cortisol have an interactive effect on memory. This design was critical in determining that, in the prior studies that have tied pre-learning cortisol to facilitated memory for negative stimuli^1,5, these effects of cortisol may manifest because of the sleep occurring during the consolidation interval.

This finding is made even more interesting considering that the sleep versus wake design necessitated testing participants in the two groups at different times of day. Cortisol follows a circadian rhythm²⁶, with the highest cortisol levels in the early morning, and the lowest in the evening. As such, the Sleep participants were all within a narrow range of relatively low cortisol levels, having encoded in the evening, while the Wake participants had higher and more variable cortisol levels, having encoded in the morning. By using this design, it appears that even small differences in relatively low cortisol levels are sufficient to influence memory consolidation over a period of sleep. At the same time, however, the diurnal variation in cortisol also complicates the design. Future work may consider how to control for the effects of the diurnal variation of cortisol in different ways. One such way would be to replicate the present findings using an afternoon nap paradigm. Because cortisol levels between a Nap and Wake condition would be statistically equivalent, circadian fluctuations in cortisol in such a paradigm would not be a concern.

While many questions remain about how cortisol interacts with sleep to enhance emotional memory consolidation, the current work provides evidence that the facilitative effect of pre-encoding cortisol on emotional memory following delays of at least 24 hr including sleep^1,5,27 may be due to interactions between cortisol and sleep-dependent consolidation processes. Without a design in which one condition’s delay includes sleep and the other does not, it would not be possible to isolate whether sleep is necessary for this effect to be observed. Furthermore, by assessing participants’ eye gaze during encoding, it was determined that this facilitative effect of cortisol on emotional memory may be due to cortisol modulating the relation between attention to emotional information at encoding and the subsequent consolidation of that information during sleep: With elevated cortisol, there is a stronger relation between what participants look at during encoding and what they later remember. It is possible that this is because elevated cortisol at encoding ‘tags’ this emotional information as important to remember, which leads sleep to selectively strengthen that salient information during the consolidation interval.

This unique combination of methods – a sleep versus wake design, a measurement of resting salivary cortisol values, an assessment of eye gaze during encoding, and an administration of a behavioral recognition memory test – led not only to uncovering an interactive effect of cortisol and sleep on consolidation, but also to determining a potential attentional mechanism behind this effect. Overall, this study can be used as an example of how to combine methodologies that are typically used independently in order to achieve a better understanding of complex interactions between variables that affect cognition.

Disclosures

The authors declare that they have no competing financial interests.